Bug 1448438 - Document how to upgrade Mercurial in CI; r?sheehan draft
authorGregory Szorc <gps@mozilla.com>
Fri, 23 Mar 2018 12:43:36 -0700
changeset 771783 248279a21016368d3234a950372b2cc374369046
parent 771782 6cabea2cc5ee06915540fffe8bf89df2ff043723
push id103788
push userbmo:gps@mozilla.com
push dateFri, 23 Mar 2018 21:28:59 +0000
reviewerssheehan
bugs1448438
milestone61.0a1
Bug 1448438 - Document how to upgrade Mercurial in CI; r?sheehan We don't have docs for how to upgrade Mercurial in CI. A lot of it is organic institutional knowledge that lives in various people's heads. That's not good for continuity of business. This commit starts to establish documentation for how to upgrade Mercurial in Firefox CI. The documentation for updating things in-tree is pretty robust (because I've done it a handful of times and know what is involved). But the docs for OpenCloudConfig and Puppet could certainly be improved. I'll poke the respective maintainers of these systems to help improve these docs as a follow-up. MozReview-Commit-ID: Aruh3N8Llls
taskcluster/docs/index.rst
taskcluster/docs/versioncontrol.rst
--- a/taskcluster/docs/index.rst
+++ b/taskcluster/docs/index.rst
@@ -26,10 +26,11 @@ check out the :doc:`how-to section <how-
     loading
     transforms
     optimization
     docker-images
     cron
     try
     actions
     release-promotion
+    versioncontrol
     how-tos
     reference
new file mode 100644
--- /dev/null
+++ b/taskcluster/docs/versioncontrol.rst
@@ -0,0 +1,100 @@
+=====================
+Version Control in CI
+=====================
+
+Upgrading Mercurial
+===================
+
+Upgrading Mercurial in CI requires touching a handful of different
+components.
+
+Vendored robustcheckout
+-----------------------
+
+The ``robustcheckout`` Mercurial extension is used throughout CI to
+perform clones and working directory updates. The canonical home of
+the extension is in the
+https://hg.mozilla.org/hgcustom/version-control-tools repository
+at the path ``hgext/robustcheckout/__init__.py``. A copy of the
+extension is vendored at
+``testing/mozharness/external_tools/robustcheckout.py``.
+
+When upgrading Mercurial, the ``robustcheckout`` extension should also
+be updated to ensure it is compatible with the version of Mercurial
+being upgraded to. Typically, one simplies copies the latest version
+from ``version-control-tools`` into the vendored location.
+
+Debian Packages for Debian Based Docker Images
+----------------------------------------------
+
+``taskcluster/ci/packages/kind.yml`` defines custom Debian packages for
+Mercurial. These are installed in various Docker images.
+
+To upgrade Mercurial, typically you just need to update the source URL
+and its hash in this file.
+
+Non-Debian Linux Docker Images
+------------------------------
+
+The ``taskcluster/docker/recipes/install-mercurial.sh`` script is sourced
+by a handful of Docker images to install Mercurial.
+
+The script references 3 tooltool artifacts:
+
+* A Mercurial source tarball (for ``pip`` installs).
+* A ``mercurial_*_amd64.deb`` Debian package.
+* A ``mercurial-common_*_all.deb`` Debian package.
+
+The source tarball is a vanilla Mercurial source distribution. The Debian
+archives will need to be produced manually.
+
+To produce the Debian archives,
+``hg clone https://www.mercurial-scm.org/repo/hg`` and ``hg update`` to
+the tag being built. Then run ``make docker-ubuntu-xenial``. This will
+build the Mercurial Debian packages in a Docker container. It will deposit
+the produced packages in ``packages/ubuntu-xenial/``.
+
+Once all 3 files are available, copy them to the same directory and
+upload them to tooltool.
+
+   $ tooltool.py add --public mercurial-x.y.z.tar.gz mercurial*.deb
+   $ tooltool.py upload --message 'Bug XXX - Mercurial x.y.z' --authentication-file ~/.tooltoolauth
+
+.. note::
+
+   See https://wiki.mozilla.org/ReleaseEngineering/Applications/Tooltool#How_To_Upload_To_Tooltool
+   for how to use tooltool and where to obtain ``tooltool.py``.
+
+Next, copy values from the ``manifest.tt`` file into
+``taskcluster/docker/recipes/install-mercurial.sh``. See revision
+``977768c296ca`` for an example upgrade.
+
+Windows AMIs
+------------
+
+https://github.com/mozilla-releng/OpenCloudConfig defines the Windows
+environment for various Windows AMIs used by Taskcluster. Several of
+the files reference a ``mercurial-x.y.z-*.msi`` installer. These references
+will need to be updated to the Mercurial version being upgraded to.
+
+The ``robustcheckout`` extension is also vendored into this repository
+at ``userdata/Configuration/FirefoxBuildResources/robustcheckout.py``. It
+should also be updated if needed.
+
+Puppet Maintained Hosts
+-----------------------
+
+Some hosts (namely macOS machines) are managed by Puppet and Puppet is used
+to install Mercurial.
+
+Puppet code lives in the https://hg.mozilla.org/build/puppet repository.
+Relevant files are in ``modules/mercurial/``,
+``modules/packages/manifests/mozilla/mozilla-python27-mercurial-debian/``,
+and ``modules/packages/manifests/mozilla/py27_mercurial*``. A copy of
+``robustcheckout`` is also vendored at
+``modules/mercurial/files/robustcheckout.py``.
+
+.. note::
+
+   The steps to upgrade Mercurial in Puppet aren't currently captured here.
+   Someone should capture those...