testing: look for Docker image environment variables (bug 1363509); r?glob draft
authorGregory Szorc <gps@mozilla.com>
Tue, 09 May 2017 16:11:50 -0700
changeset 11319 cd82c4b973e81f5685f654b3b14621fd294c756a
parent 11318 fa8cfd940e3a46920af40828a11abf57048205d8
child 11320 32903c78a1249f00c16fb4ab36d778bccbe49fab
push id1719
push userbmo:gps@mozilla.com
push dateSat, 24 Jun 2017 00:30:23 +0000
reviewersglob
bugs1363509
testing: look for Docker image environment variables (bug 1363509); r?glob hghave is used by the Mercurial test runner to determine what capabilities are available and whether to skip tests or parts of tests. Currently, our capabilities for Docker support (bmodocker, hgmodocker, and mozreviewdocker) simply return whether Docker is available. Our test runner - run-tests - differentiates between the various capability strings and ensures that required Docker images are built *before* invoking the Mercurial test harness. Our test harness also exports environment variables declaring the Docker image IDs to use. This prevents e.g. `d0cker` invocations in .t tests from having to rebuild the images (which takes a long time). This commit updates the hghave checks to look for the Docker image environment variables actually used by tests. In theory, behavior should be identical to before. However, checking for these environment variables in hghave has advantages. First, if we forget to populate a Docker image in run-tests, the test would generate the image and this would add a ton of overhead to each test. (This is why we generate images in run-tests). So checking for the environment variable in hghave will ensure the test is skipped and help flush out bugs in image population. Second, upcoming changes will allow our new, leaner development environments to support a subset of Docker. By looking for the exact Docker image requirements in hghave, we can skip tests requiring Docker images not "available" to the current environment. MozReview-Commit-ID: 7tOBPAEJmaA
testing/hghave.py
--- a/testing/hghave.py
+++ b/testing/hghave.py
@@ -25,38 +25,69 @@ if 'REPO_ROOT' not in globals():
 # We import Mercurial's own ``hghave.py`` so we can declare our own checks.
 HGHAVE_PY = os.path.join(REPO_ROOT, 'pylib', 'mercurial-support', 'hghave.py')
 execfile(HGHAVE_PY)
 
 # Need to supplement sys.path because PYTHONPATH isn't set properly
 # from the context of run-tests.py. This is very hacky.
 sys.path.insert(0, os.path.join(REPO_ROOT, 'testing'))
 
+def have_docker_images(images):
+    # These environment variables are exported by run-tests. If they aren't
+    # present, we assume the Docker image isn't built and available.
+    # If the environment variables aren't present, the test still works because
+    # d0cker will build images automatically if needed. This slows down tests
+    # drastically. So it is better to catch the issue sooner so the slowdown
+    # can be identified.
+    keys = [b'DOCKER_%s_IMAGE' % i.upper() for i in images]
+    return all(k in os.environ for k in keys)
+
+
 # Define custom checks for our environment.
 @check('docker', 'We can talk to Docker')
 def has_docker():
     if 'SKIP_DOCKER_TESTS' in os.environ:
         return False
 
     from vcttesting.docker import Docker, params_from_env
 
     url, tls = params_from_env(os.environ)
 
     tf = tempfile.NamedTemporaryFile()
     tf.close()
     d = Docker(tf.name, url, tls=tls)
     return d.is_alive()
 
+
 @check('hgmodocker', 'Require hgmo Docker pieces')
 def has_hgmodocker():
-    return has_docker()
+    images = (
+        'ldap',
+        'hgmaster',
+        'hgweb',
+        'pulse',
+    )
+    return has_docker() and have_docker_images(images)
+
 
 @check('mozreviewdocker', 'Require mozreview Docker pieces')
 def has_mozreviewdocker():
-    return has_docker()
+    images = (
+        'autolanddb',
+        'autoland',
+        'bmoweb',
+        'hgrb',
+        'hgweb',
+        'ldap',
+        'pulse',
+        'rbweb',
+        'treestatus',
+    )
+    return has_docker() and have_docker_images(images)
+
 
 @check('eslint', 'Require eslint')
 def has_eslint():
     from distutils.spawn import find_executable
     return find_executable('eslint') is not None
 
 @check('vcsreplicator', 'vcsreplicator Python modules')
 def has_vcsreplicator():
@@ -66,19 +97,21 @@ def has_vcsreplicator():
     except ImportError:
         return False
 
 @check('watchman', 'Require watchman')
 def has_watchman():
     from distutils.spawn import find_executable
     return find_executable('watchman') is not None
 
+
 @check('bmodocker', 'Require BMO Docker pieces')
 def has_bmodocker():
-    return has_docker()
+    return has_docker() and have_docker_images(['bmoweb'])
+
 
 def hgversion():
     v = os.environ['HGVERSION']
     v = v.split('+')[0]
     v = v.split('-')[0]
     return tuple(int(i) for i in v.split('.'))
 
 @check('hg31+', 'Running with Mercurial 3.1+')