Bug 1471724: Ignore shutdown hangs in ASan code. r=jmaher draft
authorKris Maglione <maglione.k@gmail.com>
Thu, 28 Jun 2018 18:15:56 -0700
changeset 812294 43e12613deb63e5f195357a6545282596303c9a7
parent 812293 f6a3555dd8cbc74090946f7444e07934e2edc6f4
push id114528
push usermaglione.k@gmail.com
push dateFri, 29 Jun 2018 02:51:47 +0000
reviewersjmaher
bugs1471724
milestone63.0a1
Bug 1471724: Ignore shutdown hangs in ASan code. r=jmaher ASan currently unpredictably hangs at shutdown. That tends to be fairly rare, but after another patch I tried to land, it started happening once every couple thousand browser sessions on automation, which amounts to about a handful of times every full automation run. These hangs are non-critical, and ASan-only, so they shouldn't block us landing important code. They should ideally be fixed in a follow-up, though. MozReview-Commit-ID: 6k3YQ3HteN8
testing/mochitest/runtests.py
--- a/testing/mochitest/runtests.py
+++ b/testing/mochitest/runtests.py
@@ -2836,24 +2836,36 @@ toolbar#nav-bar {
         if self.manifest is not None:
             self.cleanup(options, False)
 
         return status
 
     def handleTimeout(self, timeout, proc, utilityPath, debuggerInfo,
                       browser_pid, processLog):
         """handle process output timeout"""
+        # ASan test runs currently randomly hang at shutdown every thousand or
+        # so browser sessions (bug 1472064). So, for now, we ignore those
+        # hangs.
+        expected = (mozinfo.info["asan"] and
+                    self.lastTestSeen == 'Last test finished')
+
         # TODO: bug 913975 : _processOutput should call self.processOutputLine
         # one more time one timeout (I think)
-        error_message = ("TEST-UNEXPECTED-TIMEOUT | %s | application timed out after "
-                         "%d seconds with no output") % (self.lastTestSeen, int(timeout))
+        error_message = ("TEST-%s-TIMEOUT | %s | application timed out after "
+                         "%d seconds with no output") % (
+                             'EXPECTED' if expected else 'UNEXPECTED',
+                             self.lastTestSeen,
+                             int(timeout))
         self.message_logger.dump_buffered()
         self.message_logger.buffering = False
         self.log.info(error_message)
-        self.log.error("Force-terminating active process(es).")
+        if expected:
+            self.log.warning("Force-terminating active process(es).")
+        else:
+            self.log.error("Force-terminating active process(es).")
 
         browser_pid = browser_pid or proc.pid
         child_pids = self.extract_child_pids(processLog, browser_pid)
         self.log.info('Found child pids: %s' % child_pids)
 
         if HAVE_PSUTIL:
             try:
                 browser_proc = [psutil.Process(browser_pid)]