Bug 1372892: make tasks with optimized dependencies depend on decision task; r?aki draft
authorDustin J. Mitchell <dustin@mozilla.com>
Sat, 01 Jul 2017 17:01:42 -0400
changeset 603298 3a9675758841e04f2edbcdc42d148e92b5ae99a7
parent 603297 d35a0173ec7c7b50d8f48042ec6a8225cc11cd9f
child 635874 2b90c7318c83beb4aac3fd9ab8262ab2f66149b7
push id66727
push userdmitchell@mozilla.com
push dateMon, 03 Jul 2017 15:24:37 +0000
reviewersaki
bugs1372892, 1372817
milestone56.0a1
Bug 1372892: make tasks with optimized dependencies depend on decision task; r?aki This addresses the issue where tasks with a dependency which is already complete started immediately, without waiting for the decision task (bug 1372817). It does not address the issue where a re-run of a failed decision task will allow tasks created by the first run to run, in addition to creating an entirely new set of tasks. MozReview-Commit-ID: EdNZSrNw3F6
taskcluster/taskgraph/create.py
--- a/taskcluster/taskgraph/create.py
+++ b/taskcluster/taskgraph/create.py
@@ -55,21 +55,25 @@ def create_tasks(taskgraph, label_to_tas
         # Using visit_postorder() here isn't the most efficient: we'll
         # block waiting for dependencies of task N to submit even though
         # dependencies for task N+1 may be finished. If we need to optimize
         # this further, we can build a graph of task dependencies and walk
         # that.
         for task_id in taskgraph.graph.visit_postorder():
             task_def = taskgraph.tasks[task_id].task
             attributes = taskgraph.tasks[task_id].attributes
-            # if this task has no dependencies, make it depend on this decision
-            # task so that it does not start immediately; and so that if this loop
-            # fails halfway through, none of the already-created tasks run.
-            if decision_task_id and not task_def.get('dependencies'):
-                task_def['dependencies'] = [decision_task_id]
+
+            # if this task has no dependencies *within* this taskgraph, make it
+            # depend on this decision task. If it has another dependency within
+            # the taskgraph, then it already implicitly depends on the decision
+            # task.  The result is that tasks do not start immediately. if this
+            # loop fails halfway through, none of the already-created tasks run.
+            if decision_task_id:
+                if not any(t in taskgraph.tasks for t in task_def.get('dependencies', [])):
+                    task_def.setdefault('dependencies', []).append(decision_task_id)
 
             task_def['taskGroupId'] = task_group_id
             task_def['schedulerId'] = scheduler_id
 
             # Wait for dependencies before submitting this.
             deps_fs = [fs[dep] for dep in task_def.get('dependencies', [])
                        if dep in fs]
             for f in futures.as_completed(deps_fs):