Commit Graph
578 Commits
Author SHA1 Message Date
Mike Hommey 8514acf075 Bug 1906191 - Change how ProcessHandler handles output_timeout. r=releng-reviewers,gbrown
First and foremost, this undoes the patch for bug 1845125, which causes
other problems (e.g. output being dropped past a certain point).

The rest of the patch addresses bug 1845125 at the root: Ultimately, this
is a similar problem to bug 1863675.

wait is blocked on joining the stream reader thread, but the stream
reader thread is blocked on readline because the process is finished but
hasn't been waited on yet.

So joining the stream reader thread after the process is known to have
been finished is expected to work more reliably... except when a child
has been spun up that keeps it alive. In this case, we just keep the
reader thread working in the background, which is a similar strategy to
what was done in bug 1863675.

Differential Revision: https://phabricator.services.mozilla.com/D218389
2024-08-26 23:49:06 +00:00
ahochheiden 3ca4c07b58 Bug 1887885 - Collect the target architecture of a build r=firefox-build-system-reviewers,glandium
Added one field:
 - The `--target` of a build.

Re-added `python/mach/metrics.yaml` to `./mach doc mach-telemetry` generation (Not sure why it was missing).

Re-generated the associated docs. (It looks like this was not done for previously when other fields were added.

Differential Revision: https://phabricator.services.mozilla.com/D208811
2024-07-09 19:57:45 +00:00
ahochheiden f625e0fbea Bug 1894154 - Enable telemetry in CI r=firefox-build-system-reviewers,glandium
Also collect boolean flag `moz_automation` to allow filtering between
CI/Automation telemetry and the rest.

Differential Revision: https://phabricator.services.mozilla.com/D209272
2024-07-09 19:57:45 +00:00
Emilio Cobos Álvarez 1d9fc24322 Bug 1898258 - Add and document mach gen-uuid. r=smaug
The website in our docs no longer works for $reasons, this is a direct
port to python.

Differential Revision: https://phabricator.services.mozilla.com/D211219
2024-05-22 14:52:20 +00:00
Chris H-C 45bb614778 Bug 1896732 - Introducing mach gifft to generate Glean metric definitions for named Legacy probes r=TravisLong
Currently only supports events.

Differential Revision: https://phabricator.services.mozilla.com/D210690
2024-05-17 13:24:50 +00:00
Tamas Szentpeteri 6e36477758 Backed out changeset 0a1f42ba0143 (bug 1896732) for causing fatal errors while processing mozzbuild file. CLOSED TREE 2024-05-16 23:32:08 +03:00
Chris H-C 2501b5911b Bug 1896732 - Introducing mach gifft to generate Glean metric definitions for named Legacy probes r=TravisLong
Currently only supports events.

Differential Revision: https://phabricator.services.mozilla.com/D210690
2024-05-16 20:08:15 +00:00
ahochheiden 36bd40c694 Bug 1894157 - Remove Python2 compatibility code in Mach r=firefox-build-system-reviewers,glandium
Mach can currently only run on Python version 3.8 or higher, so it
doesn't make sense to continue having dead code that provides support
for Python2.

Differential Revision: https://phabricator.services.mozilla.com/D209030
2024-05-01 02:32:23 +00:00
ahochheiden 0477c16a17 Bug 1894156 - Remove usage of six in mach/telemetry.py r=firefox-build-system-reviewers,glandium
Differential Revision: https://phabricator.services.mozilla.com/D209029
2024-05-01 02:32:23 +00:00
ahochheiden b3cf71f41e Bug 1894155 - Replace [email protected] with [email protected] from notification_emails section of pings.yaml and metrics.yaml r=sylvestre
Mitch has been inactive for ~2 years now. I don't necessarily needs to
add myself in his stead, but I think it makes sense to have me on there
as the current triage owner.

Differential Revision: https://phabricator.services.mozilla.com/D209031
2024-05-01 00:12:25 +00:00
Tooru Fujisawa 343a1fa4b5 Bug 1893252 - Skip MozillaBuild version check outside of MozillaBuild. r=firefox-build-system-reviewers,ahochheiden
Differential Revision: https://phabricator.services.mozilla.com/D208561
2024-04-25 01:19:51 +00:00
serge-sans-paille e4edb9b9d7 Bug 1891778 - Move mozilla_build_version out of mozbuild/util.py r=ahochheiden
Differential Revision: https://phabricator.services.mozilla.com/D207594
2024-04-19 09:08:57 +00:00
Tooru Fujisawa 4b7697e67b Bug 1892103 - Remove ./mach esmify command. r=Standard8
Differential Revision: https://phabricator.services.mozilla.com/D207927
2024-04-19 08:21:17 +00:00
Tim Giles e31de321d5 Bug 1888952 - Add 'buildtokens' mach command. r=reusable-components-reviewers,mach-reviewers,hjones
This new command will make it easier to work with the JSON source of
truth for our design tokens. Unfortunately the noisy output when
building the tokens is expected. We can fix this issue by upgrading
to Style Dictionary v4 when it is available and changing the logging
level.[1]

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1889645

Differential Revision: https://phabricator.services.mozilla.com/D206561
2024-04-05 17:43:11 +00:00
Tim Giles f2252d8bd7 Bug 1889469 - Add 'addstory' command to MACH_COMMANDS. r=reusable-components-reviewers,mstriemer,mach-reviewers
Looks like this command didn't get added to the MACH_COMMANDS list which
means 'addstory' will throw an error when you try and use it.

Differential Revision: https://phabricator.services.mozilla.com/D206548
2024-04-05 17:43:11 +00:00
Perry McManis 840742a28d Bug 1888703 - add a mach command to create a legacy event definition from a glean one r=chutten
Differential Revision: https://phabricator.services.mozilla.com/D206281
2024-04-04 13:01:07 +00:00
Tomislav Jovanovic f0143fd784 Bug 1880764 - Initial TypeScript tooling r=mossop,Standard8
* Initial tools/ts setup.
* Mach commands for buidling xpcom related typelibs.
* Mach command for updating the typelib references.
* Mach command for type-checking js projects.

Also included the dom typelib for reference.

Differential Revision: https://phabricator.services.mozilla.com/D197620
2024-03-15 19:20:45 +00:00
Haik Aftandilian 4dc7e58ebd Bug 1876902 - Provide a way to sign macOS builds locally r=mach-reviewers,hneiva r!#mach-reviewers
Introduce a `./mach macos-sign` command that can be used to sign local packaged builds for manual testing.

Differential Revision: https://phabricator.services.mozilla.com/D199804
2024-02-29 22:58:17 +00:00
Cristian Tuns 39f3493687 Backed out changeset e8d20bbb8f68 (bug 1876902) for causing rst lint failures in signing_macos_build.rst CLOSED TREE 2024-02-29 15:11:00 -05:00
Haik Aftandilian 9573d29dbc Bug 1876902 - Provide a way to sign macOS builds locally r=mach-reviewers,hneiva r!#mach-reviewers
Introduce a `./mach macos-sign` command that can be used to sign local packaged builds for manual testing.

Differential Revision: https://phabricator.services.mozilla.com/D199804
2024-02-29 19:53:53 +00:00
ahochheiden dff7b658ed Bug 1877347 - Replace distutils.util.strtobool with a custom strtobool within mach r=firefox-build-system-reviewers,glandium
The migration advice (https://peps.python.org/pep-0632/#migration-advice)
suggests reimplementing the functionality as per the docs (https://docs.python.org/3.9/distutils/apiref.html#distutils.util.strtobool)
which is exactly what I've done here.

Differential Revision: https://phabricator.services.mozilla.com/D199950
2024-02-26 18:01:04 +00:00
Cristian Tuns a0a92fc71b Backed out changeset 9b5ebc2d3b4b (bug 1877347) as requested by Aryx on element. CLOSED TREE 2024-02-09 18:51:43 -05:00
ahochheiden 0962c229c3 Bug 1877347 - Replace distutils.util.strtobool with a custom strtobool within mach r=firefox-build-system-reviewers,glandium
The migration advice (https://peps.python.org/pep-0632/#migration-advice)
suggests reimplementing the functionality as per the docs (https://docs.python.org/3.9/distutils/apiref.html#distutils.util.strtobool)
which is exactly what I've done here.

Differential Revision: https://phabricator.services.mozilla.com/D199950
2024-02-09 23:41:09 +00:00
Butkovits Atila 9b1d3d8da9 Backed out changeset 3ccc5333fce1 (bug 1877347) for causing partial generation build bustages. a=backout 2024-02-09 13:39:27 +02:00
ahochheiden 89eb6c797b Bug 1877348 - Remove the try/except wrappers that end up calling distutils.core.setup r=firefox-build-system-reviewers,glandium
This is one of the last references to `distutils` within `mach`, so
let's get rid of it to be thorough.

Differential Revision: https://phabricator.services.mozilla.com/D199951
2024-02-08 21:07:27 +00:00
ahochheiden 21b62ee8eb Bug 1877347 - Replace distutils.util.strtobool with a custom strtobool within mach r=firefox-build-system-reviewers,glandium
The migration advice (https://peps.python.org/pep-0632/#migration-advice)
suggests reimplementing the functionality as per the docs (https://docs.python.org/3.9/distutils/apiref.html#distutils.util.strtobool)
which is exactly what I've done here.

Differential Revision: https://phabricator.services.mozilla.com/D199950
2024-02-08 21:07:26 +00:00
ahochheiden 1d5fe74cc7 Bug 1878021 - Print the messages for InvalidRepoPath and MissingVCSTool when _get_repository_object encounters those exceptions r=firefox-build-system-reviewers,glandium
This should make it easier for a user to troubleshoot the problem,
instead of trying to decipher a subsequent exception from a function
call being attempted on the `None` returned by this function.

Differential Revision: https://phabricator.services.mozilla.com/D200496
2024-02-08 07:08:53 +00:00
ahochheiden 7ff71acb2d Bug 1874208 - Remove the always printed Site not up-to-date reason r=firefox-build-system-reviewers,glandium
While this message was sometimes useful to help troubleshoot for end
users, it's causing issues parsing `stdout` (since it can't use the mach
logging infrastructure due to being too early in the startup). We also
can't use `stderr` since some mozharness scripts treat any `stderr`
output as a full error.

Differential Revision: https://phabricator.services.mozilla.com/D198451
2024-01-16 17:49:06 +00:00
Narcis Beleuzu 6a35db1169 Backed out changeset 120bb0115412 (bug 1874208) for causing BP bustages. 2024-01-12 09:02:06 +02:00
ahochheiden 128457ebda Bug 1874208 - Only output the Site not up-to-date reason to stderr r=firefox-build-system-reviewers,glandium
It's not technically an error, but we can't make it debug level output
since the mach logging isn't available this early on startup. There's
precedent for using `stderr` output elsewhere in `site.py`, though it
usually precedes an exception being raised.

Differential Revision: https://phabricator.services.mozilla.com/D198307
2024-01-12 02:11:35 +00:00
ogiorgis d916e7edbe Bug 1840493 - doc/rst: fix some languages declaration r=firefox-source-docs-reviewers,webdriver-reviewers,necko-reviewers,geckoview-reviewers,devtools-reviewers,profiler-reviewers,championshuttler,whimboo,nchevobbe,julienw,amejiamarmol
Differential Revision: https://phabricator.services.mozilla.com/D196268
2023-12-13 18:32:20 +00:00
Joel Maher 93a74ef1b8 Bug 1869430 - Batch 19.2 convert python unittest .ini -> .toml. r=gbrown,taskgraph-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D196124
2023-12-12 17:40:03 +00:00
ahochheiden a7c29534e9 Bug 1843209 - Regenerate the third_party/python lockfile with Python 3.8 as the minimum version r=firefox-build-system-reviewers,glandium,mach-reviewers,ahal
If this is not done, the next person to vendor will also make all of the changes in this commit, which is unnecessary noise.

Differential Revision: https://phabricator.services.mozilla.com/D183450
2023-12-12 03:52:01 +00:00
ogiorgis b341264881 Bug 1749473 - fix autodoc import warning r=sylvestre DONTBUILD
NameError: name 'self' is not defined

Depends on D196011

Differential Revision: https://phabricator.services.mozilla.com/D196012
2023-12-11 10:09:09 +00:00
Andrew Halberstadt 255518ea5c Bug 1867990 - Fix black formatting errors that snuck in due to regression, r=linter-reviewers,taskgraph-reviewers,releng-reviewers,bhearsum,sylvestre
These were not caught due to bug 1865672.

Depends on D195431

Differential Revision: https://phabricator.services.mozilla.com/D195432
2023-12-04 14:51:39 +00:00
Tom Marble 8514021d85 Bug 1682371 - mach skipfails part 1: add dependencies r=jmaher,mach-reviewers,ahal
Differential Revision: https://phabricator.services.mozilla.com/D193150
2023-11-29 00:12:23 +00:00
Cosmin Sabou ef2484fe0d Backed out changeset 4c5b3ec73a9a (bug 1682371) for python related failures. CLOSED TREE 2023-11-28 01:20:46 +02:00
Tom Marble 023771f69a Bug 1682371 - mach skipfails part 1: add dependencies r=jmaher,mach-reviewers,ahal
Differential Revision: https://phabricator.services.mozilla.com/D193150
2023-11-27 23:00:41 +00:00
ahochheiden 4815b23045 Bug 1865915 - Make the suggest_command code run before activating a virtualenv so that suggest command's venv is activated r=firefox-build-system-reviewers,sergesanspaille
I wasn't able to modify `sys.argv[1]` with the `suggested_command` and have
it propagate to `dispatcher.py`, so the code still has to run in both
places, otherwise there's crashes for trying to access invalid command
handlers.

Differential Revision: https://phabricator.services.mozilla.com/D194599
2023-11-27 07:50:18 +00:00
ahochheiden 495d72d453 Bug 1866382 - Properly get the aliased command name for dictionary lookup r=firefox-build-system-reviewers,sergesanspaille
The previous implementation had an error where only aliases that were 1:1 with the dictionary would work.

Differential Revision: https://phabricator.services.mozilla.com/D194591
2023-11-27 07:50:18 +00:00
Cosmin Sabou 4d28b93ff6 Backed out 2 changesets (bug 1843209) for python3 related browser accessible failures. CLOSED TREE
Backed out changeset 052214d67a99 (bug 1843209)
Backed out changeset 36c5e6c2819f (bug 1843209)
2023-11-23 02:41:22 +02:00
ahochheiden 6d5d8d77dd Bug 1843209 - Regenerate the third_party/python lockfile with Python 3.8 as the minimum version r=firefox-build-system-reviewers,glandium,mach-reviewers,ahal
If this is not done, the next person to vendor will also make all of the changes in this commit, which is unnecessary noise.

Differential Revision: https://phabricator.services.mozilla.com/D183450
2023-11-22 20:07:42 +00:00
Chris H-C 1892a9e1eb Bug 1852098 - mach command gen-use-counter-metrics r=emilio,firefox-build-system-reviewers,ahochheiden
Takes use counter definitions and generates a use_counter_metrics.yaml with
`counter` metric definitions for each use counter and denominator.

Total use counters at time of writing: 2271 (excludes denominators)

Differential Revision: https://phabricator.services.mozilla.com/D193246
2023-11-21 17:09:52 +00:00
ahochheiden d2f27fced2 Bug 1857279 - Load @SettingsProvider settings early so that we can parse set aliases and load the correct virtualenv r=ahal
Aliased commands have not loaded the virtualenv associated with a
command/subcommand since bug 1695312. It has been defaulting to the
`common` virtualenv, which worked most of the time since most commands
use `common` and it also contains a large share of our dependencies.

Differential Revision: https://phabricator.services.mozilla.com/D192376
2023-11-20 18:05:57 +00:00
ahochheiden 13c0c8057d Bug 1857279 - Consolidate all mach @SettingsProvider hooks in one file and class and have them execute very early during mach_initialize() r=ahal
The only remaining @SettingsProvider hooks should be in test files.

Differential Revision: https://phabricator.services.mozilla.com/D192375
2023-11-20 18:05:57 +00:00
Narcis Beleuzu aeac4d77e1 Backed out 7 changesets (bug 1852098) for bustages on UseCounterMetrics.h . CLOSED TREE
Backed out changeset 3b87419a9eea (bug 1852098)
Backed out changeset d9c7c84c82df (bug 1852098)
Backed out changeset e594c7eebb91 (bug 1852098)
Backed out changeset 15e0b0bd3bfb (bug 1852098)
Backed out changeset 220e9ee63ec3 (bug 1852098)
Backed out changeset 9edef145d1c5 (bug 1852098)
Backed out changeset 920a798dbb8a (bug 1852098)
2023-11-16 01:00:42 +02:00
Chris H-C 6a8a5c1490 Bug 1852098 - mach command gen-use-counter-metrics r=emilio,firefox-build-system-reviewers,ahochheiden
Takes use counter definitions and generates a use_counter_metrics.yaml with
`counter` metric definitions for each use counter and denominator.

Total use counters at time of writing: 2271 (excludes denominators)

Differential Revision: https://phabricator.services.mozilla.com/D193246
2023-11-15 21:32:40 +00:00
Stanca Serban 7f8a3ccc5e Backed out 7 changesets (bug 1852098) for causing build bustages in UseCounterMetrics.cpp. CLOSED TREE
Backed out changeset 2caabd4bfe86 (bug 1852098)
Backed out changeset 5c174b0ee1ab (bug 1852098)
Backed out changeset 05ba23510b93 (bug 1852098)
Backed out changeset 256984effaa2 (bug 1852098)
Backed out changeset 8bed08cf2a68 (bug 1852098)
Backed out changeset 16b0698a239d (bug 1852098)
Backed out changeset 6e8d54a51fb7 (bug 1852098)
2023-11-15 23:02:32 +02:00
Chris H-C dd6bfdd958 Bug 1852098 - mach command gen-use-counter-metrics r=emilio,firefox-build-system-reviewers,ahochheiden
Takes use counter definitions and generates a use_counter_metrics.yaml with
`counter` metric definitions for each use counter and denominator.

Total use counters at time of writing: 2271 (excludes denominators)

Differential Revision: https://phabricator.services.mozilla.com/D193246
2023-11-15 17:01:28 +00:00
Cristian Tuns 0a80141e7f Backed out 3 changesets (bug 1857279) for causing try failures with exit status 1 CLOSED TREE
Backed out changeset 4c1965a65011 (bug 1857279)
Backed out changeset b9885d1b6980 (bug 1857279)
Backed out changeset 76706ea52991 (bug 1857279)
2023-11-06 13:12:25 -05:00