Bug 1446241: Set run-on-projects for upload-generated-sources and upload-symbols r=aki draft
authorChris AtLee <catlee@mozilla.com>
Mon, 19 Mar 2018 14:15:52 -0400
changeset 769525 153653733e01433ae894a0ba4fd99228e0936024
parent 769320 4f1014eb5039bdfdd7a39fb7785d102df1994a6f
push id103157
push usercatlee@mozilla.com
push dateMon, 19 Mar 2018 18:48:18 +0000
reviewersaki
bugs1446241
milestone61.0a1
Bug 1446241: Set run-on-projects for upload-generated-sources and upload-symbols r=aki If tasks still exist before the optimization phase, then any tasks these depend on will also be scheduled. In particular for devedition on m-r, that means that although the builds were being excluded from the target tasks due to the build's run-on-projects settings, the upload-symbols and upload-generated-sources tasks did exist in the target tasks, and so the builds got re-added to the target tasks during graph optimization. MozReview-Commit-ID: 1AWJuafULEE
taskcluster/ci/upload-generated-sources/kind.yml
taskcluster/ci/upload-symbols/kind.yml
taskcluster/taskgraph/transforms/task.py
--- a/taskcluster/ci/upload-generated-sources/kind.yml
+++ b/taskcluster/ci/upload-generated-sources/kind.yml
@@ -31,8 +31,12 @@ job-template:
         command: >
             cd /builds/worker/checkouts/gecko &&
             ./mach python build/upload_generated_sources.py ${ARTIFACT_URL}
         sparse-profile: upload-generated-sources
     optimization:
         only-if-dependencies-run: null
     scopes:
         - secrets:get:project/releng/gecko/build/level-{level}/gecko-generated-sources-upload
+    run-on-projects:
+        by-build-platform:
+            .*devedition.*: ['mozilla-beta', 'maple']
+            default: ['all']
--- a/taskcluster/ci/upload-symbols/kind.yml
+++ b/taskcluster/ci/upload-symbols/kind.yml
@@ -47,8 +47,12 @@ job-template:
         command: >
             cd /builds/worker/checkouts/gecko &&
             ./mach python toolkit/crashreporter/tools/upload_symbols.py https://queue.taskcluster.net/v1/task/${ARTIFACT_TASKID}/artifacts/public/build/target.crashreporter-symbols-full.zip
         sparse-profile: upload-symbols
     optimization:
         only-if-dependencies-run: null
     scopes:
         - secrets:get:project/releng/gecko/build/level-{level}/gecko-symbol-upload
+    run-on-projects:
+        by-build-platform:
+            .*devedition.*: ['mozilla-beta', 'maple']
+            default: ['all']
--- a/taskcluster/taskgraph/transforms/task.py
+++ b/taskcluster/taskgraph/transforms/task.py
@@ -180,17 +180,17 @@ task_description_schema = Schema({
         ),
 
         'channel': optionally_keyed_by('project', basestring),
     },
 
     # The `run_on_projects` attribute, defaulting to "all".  This dictates the
     # projects on which this task should be included in the target task set.
     # See the attributes documentation for details.
-    Optional('run-on-projects'): [basestring],
+    Optional('run-on-projects'): optionally_keyed_by('build-platform', [basestring]),
 
     # The `shipping_phase` attribute, defaulting to None. This specifies the
     # release promotion phase that this task belongs to.
     Required('shipping-phase'): Any(
         None,
         'build',
         'promote',
         'push',
@@ -1625,16 +1625,20 @@ def build_task(config, tasks):
                 config.params['project'], branch_rev)
             task_def['metadata']['description'] += ' ([Treeherder push]({}))'.format(
                 th_push_link)
 
         # add the payload and adjust anything else as required (e.g., scopes)
         payload_builders[task['worker']['implementation']](config, task, task_def)
 
         attributes = task.get('attributes', {})
+        # Resolve run-on-projects
+        build_platform = attributes.get('build_platform')
+        resolve_keyed_by(task, 'run-on-projects', item_name=task['label'],
+                         **{'build-platform': build_platform})
         attributes['run_on_projects'] = task.get('run-on-projects', ['all'])
         attributes['always_target'] = task['always-target']
         # This logic is here since downstream tasks don't always match their
         # upstream dependency's shipping_phase.
         # A basestring task['shipping-phase'] takes precedence, then
         # an existing attributes['shipping_phase'], then fall back to None.
         if task.get('shipping-phase') is not None:
             attributes['shipping_phase'] = task['shipping-phase']