Ideally 'manifest' and 'manifest_relpath' should be normalized to forward
slashes upstream in the TestResolver class. But normalizing them there could
potentially break other uses in-tree, and I don't have bandwidth to do a proper
audit to be confident I'm not breaking something elsewhere.
Differential Revision: https://phabricator.services.mozilla.com/D136183
This helps when dealing with threads that are not registered, e.g.: the Java thread, and the upcoming whole-process thread. And it removes some object copies.
Differential Revision: https://phabricator.services.mozilla.com/D133172
`PrintUsageThenExit(code)` was supposed to exit when `code` was not zero, but:
- The name didn't reflect that, so it was confusing that `PrintUsageThenExit(0)` would *not* exit.
- The implementation in the Base Profiler exited anyway! This caused issues with some legacy code that still used the now-removed "threads" feature.
This patch renames the function to just `PrintUsage()` and never exits, leaving the caller to invoke `exit(code)` as needed -- with the added benefit that it's possible to exit with a zero code, useful in cases where an exit is not actually an error.
Differential Revision: https://phabricator.services.mozilla.com/D135666
This generates a list of all known targets in an existing build of the
docs. That makes it easier/possible to figure out what references
exist and can be used.
Differential Revision: https://phabricator.services.mozilla.com/D135596
The idea is to capture the warnings in a temporary file, and then
apply a set of regex to find any that should be treated as fatal.
This allows us to fix warnings one type at a time, and prevents us
regressing the warnings that are already fixed.
The "reference target not count" warning is added to the initial
forbidden list, so we can ensure we don't end up with internal links
pointing to nowhere.
Differential Revision: https://phabricator.services.mozilla.com/D135389
Black and PyLint linters fail to install with Python 3.10 because of typed-ast.
typed-ast 1.4.3 added support for Python 3.10.
Differential Revision: https://phabricator.services.mozilla.com/D135538
Rather than defining a distinct `Parameter` class for Gecko, this now uses the
`extend_parameters_schema` utility function (which mobile repos are already
using).
As a consequence, shared parameters are now defined in standalone taskgraph.
And only Gecko-specific parameters are listed in
`gecko_taskgraph/parameters.py`
The only exception is `project` which gets redefined so we can override the
standalone taskgraph default (since it derives `project` from the repo name,
which doesn't work for Gecko).
Differential Revision: https://phabricator.services.mozilla.com/D134515
Many revisions later... this is my best shoehorning attempt of webrtc's trace
events into the profiler.
It is not an optimal fit as the profiler wants a static schema, and the trace
event framework provides labels and values for the args dynamically.
But it allows displaying all args in the profiler which is a big step for
low-maintenance webrtc trace event integration into the profiler.
Differential Revision: https://phabricator.services.mozilla.com/D135031
This patch adds a part of TraceEvent.h that we had previously skipped. It
handles the conversion from the arg-ful TRACE_EVENTn forms to the internal
representation which carries around a dynamic number of args (num_args and
friends).
This allows us to hook into GeckoTraceEvent.h at a higher level, where all trace
events end up, with or without args.
Differential Revision: https://phabricator.services.mozilla.com/D135029
Many revisions later... this is my best shoehorning attempt of webrtc's trace
events into the profiler.
It is not an optimal fit as the profiler wants a static schema, and the trace
event framework provides labels and values for the args dynamically.
But it allows displaying all args in the profiler which is a big step for
low-maintenance webrtc trace event integration into the profiler.
Differential Revision: https://phabricator.services.mozilla.com/D135031
This patch adds a part of TraceEvent.h that we had previously skipped. It
handles the conversion from the arg-ful TRACE_EVENTn forms to the internal
representation which carries around a dynamic number of args (num_args and
friends).
This allows us to hook into GeckoTraceEvent.h at a higher level, where all trace
events end up, with or without args.
Differential Revision: https://phabricator.services.mozilla.com/D135029
The name of the main thread outside the profiler should not be set by the profiler (to "GeckoMain"), as this may affect the name of the application itself on some systems like Linux.
Added tests to ensure that the profiler doesn't set that main thread public name to "GeckoMain", and also that other threads are publicly named when first registered with the profiler.
Differential Revision: https://phabricator.services.mozilla.com/D134847
This patch resolves cases like the following:
1. The system Python has `zstandard`.
2. `MOZ_AUTOMATION=1 ./mach python --virtualenv psutil <script>`
is run, adding `psutil` to the import scope.
3. `<script>` runs Mach a _second_ time, and this time Mach needs to
be able to import `zstandard` (in this case, it should be able
to fetch it from the system Python's site-packages).
The previous behaviour would add the "site-packages" of the //invoking//
Python interpreter, but ancestor packages would get dropped.
To rectify this issue, this patch changes "import inheritance" to keep
more of the `sys.path`, rather than just
`<external-python>.all_site_packages_dirs()`.
Note: the original implementation of this patch passed forward *all*
of the `sys.path` when creating virtualenvs. However, this caused issues
when Mach ran `pip`, because `pip` was no longer able to discover the
"standard library" (it was failing because it assumed all paths defined
in a virtualenv's site were non-standard-library paths, and the original
implementation broke that assumption).
As part of this, a distinction was defined between the "current" Python
interpreter (external_python) and the top-level Python interpreter
(original_python). This was needed to enable discovering which paths
are "standard library paths".
Differential Revision: https://phabricator.services.mozilla.com/D134201
`pytest` requires `pluggy<0.7,>=0.5` and `py>=1.5.0`.
Since `python-test` jobs use a Docker image that installs
`tox_requirements`, and the system packages are used (due to `zstandard`
existing), we need to ensure compatibility.
Depends on D132082
Differential Revision: https://phabricator.services.mozilla.com/D134451
I've erred on the side of removing code here. I think that I got most of it,
but there was quite a bit of accrued code.
Thanks to Gijs for fluent fixups (and code cleanup). This retains some notice
of the failed version.
Bug 1586846, Bug 1579285, Bug 1586846, and Bug 1617275 all added code here.
Differential Revision: https://phabricator.services.mozilla.com/D133591
We clean up our Gecko clone between tasks via `hg robustcheckout --purge`, which runs `hg purge`. This is very effective, *but* it doesn't detect or clean up any nested clones.
Because we run cross-channel on Gecko workers, and because we clone `comm/` in cross-channel and haven't cleaned it up, and because `hg purge` doesn't detect or clean up nested clones, and because our current virtualenv setup code traverses the tree and can error out on `comm/` clones, let's clean up `comm/` after running cross-channel.
We'll be moving TB cross-channel to different tasks/workers in bug 1742711, and ideally we can update robustcheckout and/or `hg purge` to be able to detect and/or clean up nested clones.
Differential Revision: https://phabricator.services.mozilla.com/D134582
We clean up our Gecko clone between tasks via `hg robustcheckout --purge`, which runs `hg purge`. This is very effective, *but* it doesn't detect or clean up any nested clones.
Because we run cross-channel on Gecko workers, and because we clone `comm/` in cross-channel and haven't cleaned it up, and because `hg purge` doesn't detect or clean up nested clones, and because our current virtualenv setup code traverses the tree and can error out on `comm/` clones, let's clean up `comm/` after running cross-channel.
We'll be moving TB cross-channel to different tasks/workers in bug 1742711, and ideally we can update robustcheckout and/or `hg purge` to be able to detect and/or clean up nested clones.
Differential Revision: https://phabricator.services.mozilla.com/D134582
Avoid "percentile" for user output, since it's not clear if e.g. a 10th
percentile push duration is good or bad.
(Also the previous output incorrectly displayed "10% percentile" for a
90th percentile push duration estimate)
Differential Revision: https://phabricator.services.mozilla.com/D134112
This is necessary, because the next patch will remove the "threads" feature, and some tests only add that one feature so now they will have an empty feature list, equivalent to a feature bitset of 0 (zero).
Differential Revision: https://phabricator.services.mozilla.com/D134136
On macOS, we get intermittent failures when comparing a marker start time with the start of the test, with a difference of 0.00004166666666094443, which is very close to 24kHZ! I'm not too sure of the significance of this time difference (OS time slice resolution), but an easy fix for this intermittent is to loosen the expected time from 1e-6 to 5e-5.
Differential Revision: https://phabricator.services.mozilla.com/D134258
This patch was r+ed before by Ted, but it never landed because I initially intended
to address Ted's review comment (about making it work on 10.11 and below), and
because it needed to be rebased around bug 1374888.
The rebase turned out to be really simple, and Ted's review comment no longer applies
because Firefox no longer runs on 10.11 and below.
Profile with fix: https://share.firefox.dev/3oYzvO6
Differential Revision: https://phabricator.services.mozilla.com/D134008
