testing: look for Docker image environment variables (
bug 1363509); r?glob
hghave is used by the Mercurial test runner to determine what
capabilities are available and whether to skip tests or parts of
tests.
Currently, our capabilities for Docker support (bmodocker, hgmodocker,
and mozreviewdocker) simply return whether Docker is available.
Our test runner - run-tests - differentiates between the various
capability strings and ensures that required Docker images are
built *before* invoking the Mercurial test harness. Our test harness
also exports environment variables declaring the Docker image IDs
to use. This prevents e.g. `d0cker` invocations in .t tests from
having to rebuild the images (which takes a long time).
This commit updates the hghave checks to look for the Docker
image environment variables actually used by tests. In theory,
behavior should be identical to before. However, checking for
these environment variables in hghave has advantages.
First, if we forget to populate a Docker image in run-tests, the
test would generate the image and this would add a ton of overhead
to each test. (This is why we generate images in run-tests). So
checking for the environment variable in hghave will ensure the test
is skipped and help flush out bugs in image population.
Second, upcoming changes will allow our new, leaner development
environments to support a subset of Docker. By looking for the
exact Docker image requirements in hghave, we can skip tests
requiring Docker images not "available" to the current environment.
MozReview-Commit-ID: 7tOBPAEJmaA
--- a/testing/hghave.py
+++ b/testing/hghave.py
@@ -25,38 +25,69 @@ if 'REPO_ROOT' not in globals():
# We import Mercurial's own ``hghave.py`` so we can declare our own checks.
HGHAVE_PY = os.path.join(REPO_ROOT, 'pylib', 'mercurial-support', 'hghave.py')
execfile(HGHAVE_PY)
# Need to supplement sys.path because PYTHONPATH isn't set properly
# from the context of run-tests.py. This is very hacky.
sys.path.insert(0, os.path.join(REPO_ROOT, 'testing'))
+def have_docker_images(images):
+ # These environment variables are exported by run-tests. If they aren't
+ # present, we assume the Docker image isn't built and available.
+ # If the environment variables aren't present, the test still works because
+ # d0cker will build images automatically if needed. This slows down tests
+ # drastically. So it is better to catch the issue sooner so the slowdown
+ # can be identified.
+ keys = [b'DOCKER_%s_IMAGE' % i.upper() for i in images]
+ return all(k in os.environ for k in keys)
+
+
# Define custom checks for our environment.
@check('docker', 'We can talk to Docker')
def has_docker():
if 'SKIP_DOCKER_TESTS' in os.environ:
return False
from vcttesting.docker import Docker, params_from_env
url, tls = params_from_env(os.environ)
tf = tempfile.NamedTemporaryFile()
tf.close()
d = Docker(tf.name, url, tls=tls)
return d.is_alive()
+
@check('hgmodocker', 'Require hgmo Docker pieces')
def has_hgmodocker():
- return has_docker()
+ images = (
+ 'ldap',
+ 'hgmaster',
+ 'hgweb',
+ 'pulse',
+ )
+ return has_docker() and have_docker_images(images)
+
@check('mozreviewdocker', 'Require mozreview Docker pieces')
def has_mozreviewdocker():
- return has_docker()
+ images = (
+ 'autolanddb',
+ 'autoland',
+ 'bmoweb',
+ 'hgrb',
+ 'hgweb',
+ 'ldap',
+ 'pulse',
+ 'rbweb',
+ 'treestatus',
+ )
+ return has_docker() and have_docker_images(images)
+
@check('eslint', 'Require eslint')
def has_eslint():
from distutils.spawn import find_executable
return find_executable('eslint') is not None
@check('vcsreplicator', 'vcsreplicator Python modules')
def has_vcsreplicator():
@@ -66,19 +97,21 @@ def has_vcsreplicator():
except ImportError:
return False
@check('watchman', 'Require watchman')
def has_watchman():
from distutils.spawn import find_executable
return find_executable('watchman') is not None
+
@check('bmodocker', 'Require BMO Docker pieces')
def has_bmodocker():
- return has_docker()
+ return has_docker() and have_docker_images(['bmoweb'])
+
def hgversion():
v = os.environ['HGVERSION']
v = v.split('+')[0]
v = v.split('-')[0]
return tuple(int(i) for i in v.split('.'))
@check('hg31+', 'Running with Mercurial 3.1+')