Bug 1352463 - Increase wdspec timeouts; r?jgraham draft
authorAndreas Tolfsen <ato@mozilla.com>
Fri, 31 Mar 2017 17:42:26 +0100
changeset 564423 313b46b80ed1373aee2a59c09d0c1c561ef79363
parent 564422 c0020f7efea0c81648ad346eaade0c2fb0bd756c
child 624728 9c1185a4d2534776479ee857bcfd2014427d4e3c
push id54589
push userbmo:ato@mozilla.com
push dateTue, 18 Apr 2017 14:29:38 +0000
reviewersjgraham
bugs1352463
milestone55.0a1
Bug 1352463 - Increase wdspec timeouts; r?jgraham Increases WPT wdspec timeouts to more realistic values. Because wdspec tests interact with the browser from an OOP program, they require more time to run. Interactive browser tests are also known for generally being more expensive to run. 25 seconds for the default timeout and 120 seconds for the long timeout are values picked out of the air and likely needs to be further refined in the future. It is however the current belief that this moves us in the right direction. Further improvements to this approach may involve letting wdspec tests define timeouts on a per-file or a per-test function level through the use of pytest-timeouts, but this is purely speculative at this point. It is the current recommendation to adjust the number of tests and the runtime duration of the tests in a file according to these new defaults. MozReview-Commit-ID: 4I3Xz9G6lzv
testing/web-platform/harness/wptrunner/wpttest.py
--- a/testing/web-platform/harness/wptrunner/wpttest.py
+++ b/testing/web-platform/harness/wptrunner/wpttest.py
@@ -334,18 +334,18 @@ class ReftestTest(Test):
 
 
 class WdspecTest(Test):
 
     result_cls = WdspecResult
     subtest_result_cls = WdspecSubtestResult
     test_type = "wdspec"
 
-    default_timeout = 10
-    long_timeout = 60
+    default_timeout = 25
+    long_timeout = 120
 
 
 manifest_test_cls = {"reftest": ReftestTest,
                      "testharness": TestharnessTest,
                      "manual": ManualTest,
                      "wdspec": WdspecTest}