Bug 1290516 - Remove {{project}} from cache path for decision and action tasks; r?dustin
Currently, the gecko-decision and action tasks maintain a separate cache
for each "project" (Firefox repo) source checkout. In my opinion, this
is not necessary.
Separate caches/source checkouts mean a lower chance for a cache hit:
a lower chance that any given worker will have a source checkout readily
available for the "project" being built. They also mean workers store
more data and files in caches.
By using a shared cache for source checkouts, workers will ideally
only have a single source checkout cache and will be pretty much
guaranteed to have a checkout readily available (the main exception
being the first task on a freshly-provisioned worker).
A downside of a shared cache is that workers may spend more time doing
`hg update` operations. e.g. if the last task was from mozilla-release
and the current task is from mozilla-central, thousands of files will
be updated. However, I believe we will still be better off because
separate caches *increase* the probability for a full checkout, which
involves writing ~150,000 files! So I believe this change will lower
the overall amount of VCS work done by workers while possibly
sacrificing the lowest possible execution time (which is achieved by
minimizing the `hg update` work).
It's worth noting that `hg robustcheckout` (which is used by these
tasks) does the equivalent of an `hg purge --all`. This ensures that all
untracked and ignored files from previous tasks are deleted. So there
should be no contamination from previous tasks.
MozReview-Commit-ID: I5Ng3xFEfou
--- a/.taskcluster.yml
+++ b/.taskcluster.yml
@@ -82,17 +82,17 @@ tasks:
--base-repository='https://hg.mozilla.org/mozilla-central'
--head-repository='{{{url}}}'
--head-ref='{{revision}}'
--head-rev='{{revision}}'
--revision-hash='{{revision_hash}}'
cache:
level-{{level}}-hg-shared: /home/worker/hg-shared
- level-{{level}}-{{project}}-gecko-decision: /home/worker/workspace
+ level-{{level}}-gecko-decision: /home/worker/workspace
features:
taskclusterProxy: true
# Note: This task is built server side without the context or tooling that
# exist in tree so we must hard code the version
image: 'taskcluster/decision:0.1.2'
--- a/taskcluster/taskgraph/action.yml
+++ b/taskcluster/taskgraph/action.yml
@@ -32,17 +32,17 @@ payload:
GECKO_HEAD_REF: '{{head_ref}}'
GECKO_HEAD_REV: '{{head_rev}}'
ACTION_ARGS: >
--decision-id='{{decision_task_id}}'
--task-labels='{{task_labels}}'
cache:
level-{{level}}-hg-shared: /home/worker/hg-shared
- level-{{level}}-{{project}}-gecko-decision: /home/worker/workspace
+ level-{{level}}-gecko-decision: /home/worker/workspace
features:
taskclusterProxy: true
# Note: This task is built server side without the context or tooling that
# exist in tree so we must hard code the version
image: 'taskcluster/decision:0.1.2'