Bug 1341934 - Change how taskcluster urls are chosen between proxy and non-proxy. r?dustin draft
authorMike Hommey <mh+mozilla@glandium.org>
Thu, 23 Feb 2017 09:25:23 +0900
changeset 563438 e370363af20647b0e5554bae11b518e1d5a2bfb2
parent 563335 ce69b6e1773e9e0d0a190ce899f34b1658e66ca4
child 624465 f88b429f498e1b3fbdf62420a89b96717c558c26
push id54296
push userbmo:mh+mozilla@glandium.org
push dateMon, 17 Apr 2017 00:29:25 +0000
reviewersdustin
bugs1341934, 1356524, 1356683, 1356529, 1313111
milestone55.0a1
Bug 1341934 - Change how taskcluster urls are chosen between proxy and non-proxy. r?dustin Currently, opt_index_search chooses between taskcluster-proxy and normal urls based on TASK_ID. This only really works because the only job currently using this code runs on a taskcluster docker image that has access to the taskcluster proxy. As of bug 1356524, there is now a mach command using this code, and further down the road with bug 1356683, bug 1356529 and bug 1313111, that mach command will be used on all build jobs. But while TASK_ID is set on all jobs running on taskcluster, not all of them have access to the proxy. So we introduce a new environment variable TASKCLUSTER_PROXY, which is only set on jobs that have access to the taskcluster proxy, and rely on that in opt_index_search instead of TASK_ID.
.taskcluster.yml
taskcluster/taskgraph/optimize.py
taskcluster/taskgraph/transforms/task.py
--- a/.taskcluster.yml
+++ b/.taskcluster.yml
@@ -70,16 +70,17 @@ tasks:
         env:
           # checkout-gecko uses these to check out the source; the inputs
           # to `mach taskgraph decision` are all on the command line.
           GECKO_BASE_REPOSITORY: 'https://hg.mozilla.org/mozilla-unified'
           GECKO_HEAD_REPOSITORY: '{{{url}}}'
           GECKO_HEAD_REF: '{{revision}}'
           GECKO_HEAD_REV: '{{revision}}'
           HG_STORE_PATH: /home/worker/checkouts/hg-store
+          TASKCLUSTER_PROXY: 1
 
         cache:
           level-{{level}}-checkouts: /home/worker/checkouts
 
         features:
           taskclusterProxy: true
           chainOfTrust: true
 
--- a/taskcluster/taskgraph/optimize.py
+++ b/taskcluster/taskgraph/optimize.py
@@ -188,17 +188,17 @@ def optimization(name):
     return wrap
 
 
 @optimization('index-search')
 def opt_index_search(task, params, index_path):
     try:
         task_id = find_task_id(
             index_path,
-            use_proxy=bool(os.environ.get('TASK_ID')))
+            use_proxy=bool(os.environ.get('TASKCLUSTER_PROXY')))
 
         return True, task_id
     except requests.exceptions.HTTPError:
         pass
 
     return False, None
 
 
--- a/taskcluster/taskgraph/transforms/task.py
+++ b/taskcluster/taskgraph/transforms/task.py
@@ -538,16 +538,19 @@ def build_docker_worker_payload(config, 
         task_def['scopes'].append(
             'assume:project:taskcluster:level-{level}-sccache-buckets'.format(
                 level=config.params['level'])
         )
         worker['env']['USE_SCCACHE'] = '1'
     else:
         worker['env']['SCCACHE_DISABLE'] = '1'
 
+    if features.get('taskclusterProxy'):
+        worker['env']['TASKCLUSTER_PROXY'] = '1'
+
     capabilities = {}
 
     for lo in 'audio', 'video':
         if worker.get('loopback-' + lo):
             capitalized = 'loopback' + lo.capitalize()
             devices = capabilities.setdefault('devices', {})
             devices[capitalized] = True
             task_def['scopes'].append('docker-worker:capability:device:' + capitalized)