Bug 1290516 - Remove {{project}} from cache path for decision and action tasks; r?dustin draft
authorGregory Szorc <gps@mozilla.com>
Fri, 29 Jul 2016 10:09:06 -0700
changeset 394324 738db6c78ddebfcff0b129a760e12b59128ecae9
parent 394256 2ea3d51ba1bb9f5c3b6921c43ea63f70b4fdf5d2
child 394427 079417a03c9a476823ae2205dc34e351e861152a
push id24556
push userbmo:gps@mozilla.com
push dateFri, 29 Jul 2016 17:16:17 +0000
reviewersdustin
bugs1290516
milestone50.0a1
Bug 1290516 - Remove {{project}} from cache path for decision and action tasks; r?dustin Currently, the gecko-decision and action tasks maintain a separate cache for each "project" (Firefox repo) source checkout. In my opinion, this is not necessary. Separate caches/source checkouts mean a lower chance for a cache hit: a lower chance that any given worker will have a source checkout readily available for the "project" being built. They also mean workers store more data and files in caches. By using a shared cache for source checkouts, workers will ideally only have a single source checkout cache and will be pretty much guaranteed to have a checkout readily available (the main exception being the first task on a freshly-provisioned worker). A downside of a shared cache is that workers may spend more time doing `hg update` operations. e.g. if the last task was from mozilla-release and the current task is from mozilla-central, thousands of files will be updated. However, I believe we will still be better off because separate caches *increase* the probability for a full checkout, which involves writing ~150,000 files! So I believe this change will lower the overall amount of VCS work done by workers while possibly sacrificing the lowest possible execution time (which is achieved by minimizing the `hg update` work). It's worth noting that `hg robustcheckout` (which is used by these tasks) does the equivalent of an `hg purge --all`. This ensures that all untracked and ignored files from previous tasks are deleted. So there should be no contamination from previous tasks. MozReview-Commit-ID: I5Ng3xFEfou
.taskcluster.yml
taskcluster/taskgraph/action.yml
--- a/.taskcluster.yml
+++ b/.taskcluster.yml
@@ -82,17 +82,17 @@ tasks:
             --base-repository='https://hg.mozilla.org/mozilla-central'
             --head-repository='{{{url}}}'
             --head-ref='{{revision}}'
             --head-rev='{{revision}}'
             --revision-hash='{{revision_hash}}'
 
         cache:
           level-{{level}}-hg-shared: /home/worker/hg-shared
-          level-{{level}}-{{project}}-gecko-decision: /home/worker/workspace
+          level-{{level}}-gecko-decision: /home/worker/workspace
 
         features:
           taskclusterProxy: true
 
         # Note: This task is built server side without the context or tooling that
         # exist in tree so we must hard code the version
         image: 'taskcluster/decision:0.1.2'
 
--- a/taskcluster/taskgraph/action.yml
+++ b/taskcluster/taskgraph/action.yml
@@ -32,17 +32,17 @@ payload:
     GECKO_HEAD_REF: '{{head_ref}}'
     GECKO_HEAD_REV: '{{head_rev}}'
     ACTION_ARGS: >
       --decision-id='{{decision_task_id}}'
       --task-labels='{{task_labels}}'
 
   cache:
     level-{{level}}-hg-shared: /home/worker/hg-shared
-    level-{{level}}-{{project}}-gecko-decision: /home/worker/workspace
+    level-{{level}}-gecko-decision: /home/worker/workspace
 
   features:
     taskclusterProxy: true
 
   # Note: This task is built server side without the context or tooling that
   # exist in tree so we must hard code the version
   image: 'taskcluster/decision:0.1.2'