Commit Graph
156 Commits
Author SHA1 Message Date
Jonathan Moss 710adb847d Bug 1944610 - Read Windows cache paths from worker environment. r=taskgraph-reviewers,bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D261470
2026-06-08 14:48:26 +00:00
abhishekmadan30 fff9fde7e9 Bug 2002002 - Stop using vendored taskcluster-taskgraph, r=firefox-build-system-reviewers,taskgraph-reviewers,releng-reviewers,mach-reviewers,ahal,ahochheiden
Differential Revision: https://phabricator.services.mozilla.com/D273841
2026-02-02 20:09:02 +00:00
Serban Stanca 9e40b6bb18 Revert "Bug 2002002 - Make toolchain_task_defintions uses the ./mach taskgraph command instead of calling a function to ensure it uses the taskgraph virtual env. r=firefox-build-system-reviewers,taskgraph-reviewers,ahal,ahochheiden" for breaking retrigger hook.
This reverts commit 540cd75043.

This reverts commit afef97fa49.

This reverts commit 3f4efe4bbd.
2026-01-29 00:20:18 +00:00
abhishekmadan30 3f4efe4bbd Bug 2002002 - Stop using vendored taskcluster-taskgraph, r=firefox-build-system-reviewers,taskgraph-reviewers,releng-reviewers,mach-reviewers,ahal,ahochheiden
Differential Revision: https://phabricator.services.mozilla.com/D273841
2026-01-28 23:24:25 +00:00
agoloman f0a2cb1fb5 Revert "Bug 2002002 - Make toolchain_task_defintions uses the ./mach taskgraph command instead of calling a function to ensure it uses the taskgraph virtual env. r=firefox-build-system-reviewers,taskgraph-reviewers,ahal,ahochheiden" for busting bootstrapping.
This reverts commit d240ad8fd9.

Revert "Bug 2002002 - Remove taskcluster-taskgraph vendor, r=taskgraph-reviewers,mach-reviewers,ahal,ahochheiden"

This reverts commit 864d327065.

Revert "Bug 2002002 - Stop using vendored taskcluster-taskgraph, r=firefox-build-system-reviewers,taskgraph-reviewers,releng-reviewers,mach-reviewers,ahal,ahochheiden"

This reverts commit 9083d55fb6.
2026-01-22 01:21:11 +00:00
abhishekmadan30 9083d55fb6 Bug 2002002 - Stop using vendored taskcluster-taskgraph, r=firefox-build-system-reviewers,taskgraph-reviewers,releng-reviewers,mach-reviewers,ahal,ahochheiden
Differential Revision: https://phabricator.services.mozilla.com/D273841
2026-01-21 15:42:57 +00:00
Alexandru Marc f85620682b Revert "Bug 2002002 - Create a venv for repackage tasks to include taskgraph, r=firefox-build-system-reviewers,taskgraph-reviewers,bhearsum,ahochheiden" for causing Bug 2010496
This reverts commit e049bb0047.

Revert "Bug 2002002 - Remove taskcluster-taskgraph vendor, r=taskgraph-reviewers,mach-reviewers,ahal,ahochheiden"

This reverts commit 3f8cf62296.

Revert "Bug 2002002 - Stop using vendored taskcluster-taskgraph, r=firefox-build-system-reviewers,taskgraph-reviewers,releng-reviewers,mach-reviewers,ahal,ahochheiden"

This reverts commit 84cbc516db.
2026-01-15 09:33:16 +00:00
abhishekmadan30 84cbc516db Bug 2002002 - Stop using vendored taskcluster-taskgraph, r=firefox-build-system-reviewers,taskgraph-reviewers,releng-reviewers,mach-reviewers,ahal,ahochheiden
Differential Revision: https://phabricator.services.mozilla.com/D273841
2026-01-14 20:07:00 +00:00
Alexandru Marc f1b2c5651e Revert "Bug 2002002: apply code formatting via Lando" for causing mac repackage bustages
This reverts commit c903ab15c4.

Revert "Bug 2002002 - Remove taskcluster-taskgraph vendor, r=taskgraph-reviewers,mach-reviewers,ahal,ahochheiden"

This reverts commit 251f05311e.

Revert "Bug 2002002 - Stop using vendored taskcluster-taskgraph, r=firefox-build-system-reviewers,taskgraph-reviewers,releng-reviewers,mach-reviewers,ahal,ahochheiden"

This reverts commit 3fa95d1a9b.
2026-01-09 08:50:36 +00:00
abhishekmadan30 3fa95d1a9b Bug 2002002 - Stop using vendored taskcluster-taskgraph, r=firefox-build-system-reviewers,taskgraph-reviewers,releng-reviewers,mach-reviewers,ahal,ahochheiden
Differential Revision: https://phabricator.services.mozilla.com/D273841
2026-01-09 03:25:55 +00:00
Julien Cristau b4d43348bd Bug 1967974 - run-task: wait on child processes when we're pid 1. r=releng-reviewers,bhearsum
Avoid leaving zombie processes around while the task is running.

Differential Revision: https://phabricator.services.mozilla.com/D253501
2025-07-22 10:11:10 +00:00
Alexandru Marc de466d429d Revert "Bug 1967974 - run-task: wait on child processes when we're pid 1. r=releng-reviewers,bhearsum" for causing dxc toolchains bustages
This reverts commit 6ee91c51eb.
2025-06-16 18:00:17 +00:00
Julien Cristau 6ee91c51eb Bug 1967974 - run-task: wait on child processes when we're pid 1. r=releng-reviewers,bhearsum
Avoid leaving zombie processes around while the task is running.

Differential Revision: https://phabricator.services.mozilla.com/D253501
2025-06-16 14:22:47 +00:00
Cristina Horotan 3797115949 Revert "Bug 1967974 - run-task: wait on child processes when we're pid 1. r=releng-reviewers,bhearsum" for Android bustage
This reverts commit 7eceeb7f04.
2025-06-13 13:48:10 +00:00
Julien Cristau 7eceeb7f04 Bug 1967974 - run-task: wait on child processes when we're pid 1. r=releng-reviewers,bhearsum
Avoid leaving zombie processes around while the task is running.

Differential Revision: https://phabricator.services.mozilla.com/D253501
2025-06-13 08:56:17 +00:00
Alex Hochheiden bca2ccd051 Bug 1968592 - Enable use-uv by default for test and source-test kinds r=ahal,taskgraph-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D251210
2025-06-06 21:37:02 +00:00
Julien Cristau 6a28550d7b Bug 1970105 - run-task: fix ruff/black warnings r=releng-reviewers,Eijebong
Differential Revision: https://phabricator.services.mozilla.com/D252320
2025-06-03 13:18:22 +00:00
Julien Cristau 472eeff605 Bug 1970105 - run-task: silence python 3.12 deprecation warning r=releng-reviewers,Eijebong
Differential Revision: https://phabricator.services.mozilla.com/D252319
2025-06-03 13:18:19 +00:00
Alexandru Marc 390509a00f Revert "Bug 1934096 - Minimize differences between site.py and python.py implementations of pip_command functions r=ahal" for causing build bustages
This reverts commit e5ef1c9835.

Revert "Bug 1934096 - Use `uv` in `mozharness/base/python.py` r=ahal"

This reverts commit 48b3d309b5.

Revert "Bug 1968593 - Upgrade `uv` toolchain version to `0.7.2` r=firefox-build-system-reviewers,sergesanspaille"

This reverts commit 8b828d5f58.

Revert "Bug 1968592 - Fix `test_site_activation.py` test in CI after `uv` enable r=firefox-build-system-reviewers,sergesanspaille"

This reverts commit 8cfd9ee70b.

Revert "Bug 1968592: Enable `use-uv` by default for `test` and `source-test` kinds r=ahal,taskgraph-reviewers"

This reverts commit 994220ef5d.

Revert "Bug 1962485 - Fix `jj` tests in `mozversioncontrol` r=sfink,ahal"

This reverts commit 52e5f009bd.

Revert "Bug 1962485 - Add `jj` toolchain to `mozversioncontrol` Python source test r=firefox-build-system-reviewers,nalexander"

This reverts commit f0c553f479.

Revert "Bug 1962485 - Add `jj` toolchain tasks r=firefox-build-system-reviewers,nalexander"

This reverts commit 166ddbd3fe.

Revert "Bug 1967634 - Bump vendored `pyyaml` version to `6.0.2` to match mozharness and grizzly r=ahal,mach-reviewers"

This reverts commit 495e3e0a95.

Revert "Bug 1967634 - Apply fixes from `python-sites` lint r=ahal,mach-reviewers"

This reverts commit 5953da8cb8.

Revert "Bug 1967634 - Add `python-sites` lint to facilitate adding rules specific to `python/sites/` r=ahal"

This reverts commit 83917ff26d.

Revert "Bug 1965907 - Re-enable `test_vendor.py` r=ahal"

This reverts commit 6777024795.
2025-05-30 20:56:55 +00:00
Alexandru Marc 3b440c8738 Reapply "Bug 1898654 - Make EXPANDABLE/EXPANDED/COLLAPSED states consistent. r=Jamie" for causing ba failures at browser_simplePatterns.js for causing build bustage
This reverts commit fa47313db2.
2025-05-30 20:56:48 +00:00
Cristina Horotan fa47313db2 Revert "Bug 1898654 - Make EXPANDABLE/EXPANDED/COLLAPSED states consistent. r=Jamie" for causing ba failures at browser_simplePatterns.js for causing build bustage
This reverts commit 779b2feecd.

Revert "Bug 1899960 - P2: Imply an unavailable state if focusable is in unavailable subtree. r=Jamie"

This reverts commit 6e4b35e663.

Revert "Bug 1899960 - P1: Don't cache implicit states. r=Jamie"

This reverts commit eacb22ed13.

Revert "Bug 1967813 - Check menu's frame IsOpen to determine if menu is active. r=ayeddi"

This reverts commit f1f8ac785d.

Revert "Bug 1934096 - Minimize differences between `site.py` and `python.py` implementations of `pip_command` functions r=ahal"

This reverts commit e5ef1c9835.

Revert "Bug 1934096 - Use `uv` in `mozharness/base/python.py` r=ahal"

This reverts commit 48b3d309b5.

Revert "Bug 1968593 - Upgrade `uv` toolchain version to `0.7.2` r=firefox-build-system-reviewers,sergesanspaille"

This reverts commit 8b828d5f58.

Revert "Bug 1968592 - Fix `test_site_activation.py` test in CI after `uv` enable r=firefox-build-system-reviewers,sergesanspaille"

This reverts commit 8cfd9ee70b.

Revert "Bug 1968592: Enable `use-uv` by default for `test` and `source-test` kinds r=ahal,taskgraph-reviewers"

This reverts commit 994220ef5d.

Revert "Bug 1962485 - Fix `jj` tests in `mozversioncontrol` r=sfink,ahal"

This reverts commit 52e5f009bd.

Revert "Bug 1962485 - Add `jj` toolchain to `mozversioncontrol` Python source test r=firefox-build-system-reviewers,nalexander"

This reverts commit f0c553f479.

Revert "Bug 1962485 - Add `jj` toolchain tasks r=firefox-build-system-reviewers,nalexander"

This reverts commit 166ddbd3fe.

Revert "Bug 1967634 - Bump vendored `pyyaml` version to `6.0.2` to match mozharness and grizzly r=ahal,mach-reviewers"

This reverts commit 495e3e0a95.

Revert "Bug 1969203 - Fix `.gitignore` pattern regarding `.egg-info` for vendored packages in `jj` r=ahal"

This reverts commit b993509711.

Revert "Bug 1967634 - Apply fixes from `python-sites` lint r=ahal,mach-reviewers"

This reverts commit 5953da8cb8.

Revert "Bug 1967634 - Add `python-sites` lint to facilitate adding rules specific to `python/sites/` r=ahal"

This reverts commit 83917ff26d.

Revert "Bug 1965907 - Re-enable `test_vendor.py` r=ahal"

This reverts commit 6777024795.
2025-05-30 20:33:45 +00:00
Alex Hochheiden 994220ef5d Bug 1968592: Enable use-uv by default for test and source-test kinds r=ahal,taskgraph-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D251210
2025-05-30 17:42:19 +00:00
Andrew Halberstadt d2d17b6243 Bug 1937801 - [ci] Revert {task_workdir} interpolation in environment variables, r=taskgraph-reviewers,jcristau,Eijebong
Initially I implemented this feature to solve the problem about how to
make a path absolute to protect against a task changing the working
directory. This is a problem because generic-worker creates a random
task user for each task, so the absolute path can't be known at runtime.

Before my patches, we "solved" this by hardcoding a set of envs in the
`run-task` script, and then calling `os.path.abspath` on them. This
worked for the most part, except for Windows tasks that don't run
through a shell. In that case, the paths would remain relative, which
tbh, mostly just works.

Still, I thought it would be best if we could guarantee that paths are
absolute, and also would be best to avoid hardcoding specific envs in
`run-task`. To that end, I introduced a `{task_workdir}` substitution
where any env containing this string would be interpolated with the
task user's home directory.

In hindsight, this was a bad idea. My patch got backed out 4-5 times due
to subtle errors it introduced to miscellaneous tasks. Then after
finally getting it landed, it started causing havoc in Thunderbird.
What's more is I still had to avoid using `{task_workdir}` in the
Windows without a shell case and rely on relative paths. So the feature
didn't even accomplish what I originally intended it to.

Instead, I recently landed a patch to generic-worker, so *it* will
populate the $TASK_WORKDIR env. this will be available across all
platforms, whether the task is using a shell or not.

So for all these reasons, I'm reverting the `{task_workdir}` change and
returning to using `os.path.abspath`. It's not perfect, but it's good
enough for now. Once we have a modern generic-worker in all our pools,
we can start relying on $TASK_WORKDIR and get the best of both worlds.

Differential Revision: https://phabricator.services.mozilla.com/D238217
2025-02-18 15:01:23 +00:00
Andrew Halberstadt be8868f272 Bug 1937801 - Normalize $TASK_WORKDIR to forward slashes, r=taskgraph-reviewers,bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D237808
2025-02-12 20:57:18 +00:00
Andrew Halberstadt b2b445d199 Bug 1937801 - [ci] Interpolate {task_workdir} in run-task script, r=taskgraph-reviewers,bhearsum
This is a port from upstream Taskgraph:
https://github.com/taskcluster/taskgraph/commit/eb4d396ca9a4d431e756ed3eb56acf63e1cb3a63

Differential Revision: https://phabricator.services.mozilla.com/D235877
2025-02-12 20:57:16 +00:00
Norisz Fay a8465b5c12 Backed out 4 changesets (bug 1937801) for causing Talos and Btime failures on Windows 11
Backed out changeset 6e19c19827e1 (bug 1937801)
Backed out changeset 4b1921a925d2 (bug 1937801)
Backed out changeset bc3752d1ad6b (bug 1937801)
Backed out changeset 1285a3e43a83 (bug 1937801)
2025-02-10 22:19:47 +02:00
Andrew Halberstadt 2e4fb8dcd5 Bug 1937801 - [ci] Interpolate {task_workdir} in run-task script, r=taskgraph-reviewers,bhearsum
This is a port from upstream Taskgraph:
https://github.com/taskcluster/taskgraph/commit/eb4d396ca9a4d431e756ed3eb56acf63e1cb3a63

Differential Revision: https://phabricator.services.mozilla.com/D235877
2025-02-10 15:14:40 +00:00
Alexandru Marc 363f53a3db Backed out 3 changesets (bug 1937801) for causing build bustages CLOSED TREE
Backed out changeset 03988b247053 (bug 1937801)
Backed out changeset 85ed7c0ceef2 (bug 1937801)
Backed out changeset dd5d06b71f4c (bug 1937801)
2025-02-01 00:59:11 +02:00
Andrew Halberstadt b74809f589 Bug 1937801 - [ci] Interpolate {task_workdir} in run-task script, r=taskgraph-reviewers,bhearsum
This is a port from upstream Taskgraph:
https://github.com/taskcluster/taskgraph/commit/eb4d396ca9a4d431e756ed3eb56acf63e1cb3a63

Differential Revision: https://phabricator.services.mozilla.com/D235877
2025-01-31 20:31:36 +00:00
Goloman Adrian 3e19c43d4e Backed out 3 changesets (bug 1937801) for causing toolchains failures @public/build. CLOSED TREE
Backed out changeset 254c5f7b0895 (bug 1937801)
Backed out changeset f442b0d4e5d3 (bug 1937801)
Backed out changeset 2ef51252d2c9 (bug 1937801)
2025-01-31 01:43:07 +02:00
Andrew Halberstadt 21b2d9377a Bug 1937801 - [ci] Interpolate {task_workdir} in run-task script, r=taskgraph-reviewers,bhearsum
This is a port from upstream Taskgraph:
https://github.com/taskcluster/taskgraph/commit/eb4d396ca9a4d431e756ed3eb56acf63e1cb3a63

Differential Revision: https://phabricator.services.mozilla.com/D235877
2025-01-30 21:04:43 +00:00
Julien Cristau 86c624b9ee Bug 1930944 - start dbus system bus in interactive docker-worker test tasks. r=releng-reviewers,bhearsum
Work around a docker-worker bug where it overrides the docker image's
entry point for interactive tasks, and doesn't seem to actually run the
original one as we'd expect.

Differential Revision: https://phabricator.services.mozilla.com/D229127
2024-12-09 09:43:33 +00:00
Julien Cristau bb9fe6a7c6 Bug 1796139 - use fetch-content script from taskgraph. r=taskgraph-reviewers,gabriel
Differential Revision: https://phabricator.services.mozilla.com/D217296
2024-07-23 07:08:11 +00:00
Julien Cristau 9a434d9d43 Bug 1906899 - remove unused resource-monitor code. r=taskgraph-reviewers,bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D216058
2024-07-17 08:57:42 +00:00
Mike Hommey a290a45515 Bug 1903502 - Use in-tree robustcheckout.py in tasks running on generic-worker. r=releng-reviewers,sheehan,taskgraph-reviewers,jcristau
The robustcheckout.py that is in-tree ends up in docker images, used by
tasks running on docker-worker, but for tasks running on generic-worker,
we currently rely on whatever version of robustcheckout.py is there.
It currently seems out of date, but whenever we make changes to the
in-tree version, we don't end up with the fix on these jobs without some
additional manual work.

With this change, we do the same as with run-task and fetch-content: we
pull the in-tree version that we store as an artifact of the decision
task.

Differential Revision: https://phabricator.services.mozilla.com/D214234
2024-06-25 00:27:58 +00:00
Sandor Molnar e016779c4f Backed out changeset 2147e0051e23 (bug 1903502) for causing build bustages. CLOSED TREE 2024-06-25 02:58:29 +03:00
Mike Hommey 3a42014dc3 Bug 1903502 - Use in-tree robustcheckout.py in tasks running on generic-worker. r=releng-reviewers,sheehan,taskgraph-reviewers,jcristau
The robustcheckout.py that is in-tree ends up in docker images, used by
tasks running on docker-worker, but for tasks running on generic-worker,
we currently rely on whatever version of robustcheckout.py is there.
It currently seems out of date, but whenever we make changes to the
in-tree version, we don't end up with the fix on these jobs without some
additional manual work.

With this change, we do the same as with run-task and fetch-content: we
pull the in-tree version that we store as an artifact of the decision
task.

Differential Revision: https://phabricator.services.mozilla.com/D214234
2024-06-24 23:40:38 +00:00
serge-sans-paille db0b7d8ffd Bug 1687732 - Bundle certifi certificates in taskcluster's Python on OSX r=jmaher
Instead of trying to find the system certificates, something that may
depend on the running system, use the certifi package and have ssl point
to it.

Differential Revision: https://phabricator.services.mozilla.com/D201596
2024-02-13 10:16:53 +00:00
Mike Hommey 04c3ebfe79 Bug 1539681 - Remove mercurial clone on failure. r=releng-reviewers,bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D182129
2023-06-27 21:18:20 +00:00
serge-sans-paille ac4eaf1035 Bug 1806664 - Use python toolchain tasks to build firefox r=glandium,taskgraph-reviewers,ahal
This patch introduces:

1. A new task property named use-system-python, defaults to true
2. Task transform to add the appropriate python dependency and set the
   PATH accordingly when use-system-python is set to False
3. A new environment variable MOZ_PYTHON_ROOT that can be used in task
   to refer to the local python environment (when available).

Differential Revision: https://phabricator.services.mozilla.com/D165171
2023-04-06 13:08:18 +00:00
Cristian Tuns 6f04275f71 Backed out changeset be6ab6cd894d (bug 1806664) for causing static-analysis, repackage-msi bustages CLOSED TREE 2023-04-05 17:19:33 -04:00
serge-sans-paille 6c5bb20ca0 Bug 1806664 - Use python toolchain tasks to build firefox r=glandium,taskgraph-reviewers,ahal
This patch introduces:

1. A new task property named use-system-python, defaults to true
2. Task transform to add the appropriate python dependency and set the
   PATH accordingly when use-system-python is set to False
3. A new environment variable MOZ_PYTHON_ROOT that can be used in task
   to refer to the local python environment (when available).

Differential Revision: https://phabricator.services.mozilla.com/D165171
2023-04-05 09:18:06 +00:00
Julien Cristau 912d6ca532 Bug 1792076 - add new hg.mozilla.org certificate fingerprint to run-task. r=dhouse DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D158746
2022-10-07 08:12:06 +00:00
Connor Sheehan 733353b59e Bug 1733180: add 2021/2022 hgmo SSL certificate fingerprint to FALLBACK_FINGERPRINT r=dhouse
Differential Revision: https://phabricator.services.mozilla.com/D129071
2021-10-21 14:02:11 +00:00
Steve Fink d5a14d7fbc Bug 1724969 - Fix run-task when not running_as_root r=releng-reviewers,jmaher
Differential Revision: https://phabricator.services.mozilla.com/D122265
2021-08-10 20:54:58 +00:00
Connor Sheehan 13406e70f0 Bug 1670031: update fallback fingerprint in run-task r=zeid
Differential Revision: https://phabricator.services.mozilla.com/D94018
2020-10-19 15:37:23 +00:00
Mike Hommey 9e8c7f4f2a Bug 1669879 - Always use hg in run-task on mac. r=releng-reviewers,mtabara
All the workers that had hg under /tools/python27-mercurial are gone.
This also fixes run-task for workers running > 10.14.

Differential Revision: https://phabricator.services.mozilla.com/D92867
2020-10-08 09:40:25 +00:00
Bogdan Tara d0f521a520 Backed out 2 changesets (bug 1669879) for docker failures CLOSED TREE
Backed out changeset f484bfd1ad97 (bug 1669879)
Backed out changeset 86061c7de217 (bug 1669879)
2020-10-09 07:50:02 +03:00
Mike Hommey 65f25600f0 Bug 1669879 - Always use hg in run-task on mac. r=releng-reviewers,mtabara
All the workers that had hg under /tools/python27-mercurial are gone.
This also fixes run-task for workers running > 10.14.

Differential Revision: https://phabricator.services.mozilla.com/D92867
2020-10-08 09:40:25 +00:00
Tom Prince 5f9c760bfb Bug 1492362: [taskgraph] Don't delete fetches after running a task; r=glandium
In Bug 1466660, we started deleting the fetches after a task had run, to avoid
interference between tasks. It turns out the only tasks this was for were the
`source-test-jsshell` tasks, which were changed to use an absolute directory in
Bug 1465181. However, since Bug 1568460 we've always used a per-task directory
for fetches, so can remove the work-around of removing fethes.

Differential Revision: https://phabricator.services.mozilla.com/D86670
2020-08-19 11:02:22 +00:00