Bug 1903502 - Use in-tree robustcheckout.py in tasks running on generic-worker. r=releng-reviewers,sheehan,taskgraph-reviewers,jcristau

The robustcheckout.py that is in-tree ends up in docker images, used by
tasks running on docker-worker, but for tasks running on generic-worker,
we currently rely on whatever version of robustcheckout.py is there.
It currently seems out of date, but whenever we make changes to the
in-tree version, we don't end up with the fix on these jobs without some
additional manual work.

With this change, we do the same as with run-task and fetch-content: we
pull the in-tree version that we store as an artifact of the decision
task.

Differential Revision: https://phabricator.services.mozilla.com/D214234

taskcluster/gecko_taskgraph/decision.py

@@ -231,12 +231,17 @@ def taskgraph_decision(options, parameters=None):
     if len(push_schedules) > 0:
         write_artifact("bugbug-push-schedules.json", push_schedules.popitem()[1])
 
-    # cache run-task & misc/fetch-content
+    # cache run-task, misc/fetch-content & robustcheckout.py
     scripts_root_dir = os.path.join(GECKO, "taskcluster/scripts")
     run_task_file_path = os.path.join(scripts_root_dir, "run-task")
     fetch_content_file_path = os.path.join(scripts_root_dir, "misc/fetch-content")
+    robustcheckout_path = os.path.join(
+        GECKO,
+        "testing/mozharness/external_tools/robustcheckout.py",
+    )
     shutil.copy2(run_task_file_path, ARTIFACTS_DIR)
     shutil.copy2(fetch_content_file_path, ARTIFACTS_DIR)
+    shutil.copy2(robustcheckout_path, ARTIFACTS_DIR)
 
     # actually create the graph
     create_tasks(

taskcluster/gecko_taskgraph/transforms/job/run_task.py

@@ -203,6 +203,17 @@ def generic_worker_run_task(config, job, taskdesc):
                 "file": "./fetch-content",
             }
         )
+    # Mac workers are still using a python2-based Mercurial not compatible with the
+    # in-tree robustcheckout. bug #1626357
+    if run.get("checkout") and not is_mac:
+        worker["mounts"].append(
+            {
+                "content": {
+                    "url": script_url(config, "robustcheckout.py"),
+                },
+                "file": "./robustcheckout.py",
+            }
+        )
 
     run_command = run["command"]
     run_cwd = run.get("cwd")

taskcluster/scripts/run-task

@@ -431,6 +431,13 @@ def vcs_checkout(source_repo, dest, store_path,
         '--purge',
     ]
 
+    robustcheckout = os.path.join(os.path.dirname(__file__), 'robustcheckout.py')
+    if os.path.isfile(robustcheckout):
+        args.extend([
+            '--config',
+            f'extensions.robustcheckout={robustcheckout}',
+        ])
+
     # Obtain certificate fingerprints.  Without this, the checkout will use the fingerprint
     # on the system, which is managed some other way (such as puppet)
     if fetch_hgfingerprint: