Commit Graph

5994 Commits

Author SHA1 Message Date
Tarek Ziadé
7c486e2ce0 Bug 1641834 - started a Sphinx doc r=sparky,acreskey
Initial Sphinx structure with minimal content.
This is mostly to get started in writing doc

Differential Revision: https://phabricator.services.mozilla.com/D77440
2020-06-03 08:17:06 +00:00
Bogdan Tara
979c77d94e Backed out changeset 83ee79caab92 (bug 1622963) for python related bustages CLOSED TREE 2020-06-03 10:23:57 +03:00
Mike Hommey
323ea73e5a Bug 1622963 - Require python 3.6 for the build system. r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D77754
2020-06-03 02:16:48 +00:00
Tom Prince
581eba974b Bug 1641971: Pass mach perftest options explicitly, rather than inpsect parameters; r=tarek
Differential Revision: https://phabricator.services.mozilla.com/D77548
2020-06-03 02:35:09 +00:00
Narcis Beleuzu
479c45bbea Backed out 7 changesets (bug 1641971) for breaking Gecko Decision Task opt . CLOSED TREE
Backed out changeset a039e60a26d2 (bug 1641971)
Backed out changeset e7d4c490c31a (bug 1641971)
Backed out changeset f2322981b294 (bug 1641971)
Backed out changeset d123b08d2c20 (bug 1641971)
Backed out changeset 57772c0b9b3e (bug 1641971)
Backed out changeset 11a887f73dbb (bug 1641971)
Backed out changeset e9c7a59d4bc5 (bug 1641971)
2020-06-03 05:30:51 +03:00
Tom Prince
11aedca9b2 Bug 1641971: Pass mach perftest options explicitly, rather than inpsect parameters; r=tarek
Differential Revision: https://phabricator.services.mozilla.com/D77548
2020-06-02 18:23:07 +00:00
Andy Grover
95b90fb23e Bug 1633006 - Update to Neqo 0.4.0 r=dragana,necko-reviewers
Document qlog uses 2 clause bsd license.

Adapt to API changes in glue and http3server

Differential Revision: https://phabricator.services.mozilla.com/D77903
2020-06-02 21:28:31 +00:00
Riley Byrd
ad82a96dad Bug 1518234 - Migrate migration wizard to Fluent r=flod,Gijs,fluent-reviewers
Based on a patch originally written by Ian Kirkpatrick <kirkpa47@msu.edu>

Differential Revision: https://phabricator.services.mozilla.com/D17393
2020-06-02 06:46:09 +00:00
Ricky Stewart
d2861db731 Bug 1642439 - Fix test_error_output.py r=froydnj
These variables were renamed in bug 1641992, and furthermore they were changed to "templates" (with inline `%s` for string formatting), so fix the tests with that in mind.

Differential Revision: https://phabricator.services.mozilla.com/D77697
2020-06-01 19:33:29 +00:00
Emily McDonough
c6e4f99f13 Bug 1642040 - Add VoidLinux support to mach bootstrap r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D77571
2020-06-01 19:37:06 +00:00
Ricky Stewart
7ed5e2165a Bug 1642418 - Fix mach completion unit test by updating bad conditional r=froydnj
I misinterpreted one of the conditionals when working on bug 1641991 (the "not X or Y" formulation confused me :( ). The correct fix here is to remove this conditional which isn't relevant any more after we removed all the old pre-`mochitest` mach commands.

Differential Revision: https://phabricator.services.mozilla.com/D77692
2020-06-01 18:59:46 +00:00
Ricky Stewart
7bee84802f Bug 1641992 - Add argument to mach busted file to specify which component the bug should be filed against r=froydnj
With the addition of this change, a lone `mach busted file` will throw an error (since this should only ever be used by actual humans, and not in automation or anything, the backwards-compatibility breakage isn't a huge deal). Now it's expected to pass in `mach busted file $COMMAND`, where $COMMAND is the mach command that you were running when the error occurred, and we'll figure out which bugzilla component to file the bug against for you instead of directing it to `Firefox Build System :: General` regardless of whether the issue has *anything* to do with the build system. We preserve `mach busted file general` as a backup command that doesn't do any of this heavy logic.

Differential Revision: https://phabricator.services.mozilla.com/D77538
2020-06-01 16:23:48 +00:00
Ricky Stewart
7f3fa0551b Bug 1641991 - Remove deprecated mochitest mach commands r=ahal
These commands were removed 5 years ago (with the exception of robocop, which was removed about a year ago). We're way past the point where anyone would glean useful info from keeping the stubs here with an error message.

Differential Revision: https://phabricator.services.mozilla.com/D77536
2020-05-29 20:43:05 +00:00
Ricky Stewart
0bbaac721b Bug 1641693 - Replace a bunch of uses of GENERATED_FILES with the GeneratedFile template r=necko-reviewers,geckoview-reviewers,aklotz,dragana,froydnj
Also update documentation to suggest using the `GeneratedFile` template rather than directly referencing `GENERATED_FILES` where possible.

Differential Revision: https://phabricator.services.mozilla.com/D77496
2020-06-01 16:00:28 +00:00
Mike Hommey
9088528947 Bug 1635764 - Move --with-system-icu and --with-intl-api to python configure. r=firefox-build-system-reviewers,rstewart
At the same time, because it's now simpler to do so, set the right data
file name for big-endians, even though we don't have or produce it
(bug #1264836). Also remove USE_ICU, which is redundant with
JS_HAS_INTL_API, and actively break the build at configure time when
using --without-intl-api with Firefox because this hasn't actually
worked for close to 3 years (since bug 1402048).

Differential Revision: https://phabricator.services.mozilla.com/D77416
2020-05-29 21:29:52 +00:00
Mitchell Hentges
811fbf178f Bug 1640064: Don't attempt to install python-pip (py2) on Debian Sid r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D77517
2020-05-29 20:17:40 +00:00
Mitchell Hentges
1b525e3c0c Bug 1640815: Raise helpful error if no non-public commits on bootstrap r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D77505
2020-05-29 18:35:35 +00:00
Ricky Stewart
7a15c311ac Bug 1639313 - Print usage if you call a bare mach vendor, mach cargo, or mach remote r=remote-protocol-reviewers,froydnj,whimboo
The architecture of `mach` does not make this easy to do from within one process, not least of all due to persistent global, mutable state. :( There may be another way to do this, I'm not saying it's completely impossible (and obviously with a significant enough refactor there's a way to make this happen), but subprocessing is a foolproof way to accomplish the same.

Differential Revision: https://phabricator.services.mozilla.com/D77374
2020-05-29 16:16:31 +00:00
Gregory Mierzwinski
55c95e297a Bug 1637994 - Add a transformer for extracting times/durations from logcat logs. r=acreskey,mozperftest-reviewers,tarek
This patch adds a transformer that is capable of extracting times/entries or durations for logcat logs. Here, we also implement the ability for users to pass options to the transformers so that they can be more generic (through transformer-options). The logcat transformer makes use of this, and also shows how a custom function can be used within transformers. A new test file is added for the transformer, as well as a test in test_perfherder.py so we can test the full pipeline.

Differential Revision: https://phabricator.services.mozilla.com/D77094
2020-05-29 12:40:40 +00:00
Gregory Mierzwinski
115d1b4294 Bug 1641566 - Update in-tree browsertime to latest. r=tarek,perftest-reviewers,Bebe
Differential Revision: https://phabricator.services.mozilla.com/D77257
2020-05-29 12:24:33 +00:00
Tarek Ziadé
3c6bbe84ba Bug 1641549 - Run the metrics layer separately r=sparky
Differential Revision: https://phabricator.services.mozilla.com/D77292
2020-05-29 12:21:52 +00:00
Tarek Ziadé
77da205d14 Bug 1640627 - added android logs options r=sparky
--android-capture-logcat  - captures the logcat
--android-clear-logcat - clears the logcat on start
--android-capture-adb - captures the adb commands

Differential Revision: https://phabricator.services.mozilla.com/D77038
2020-05-28 16:42:37 +00:00
Nils Ohlmeier [:drno]
18ef0cb66a Bug 1641289: Bye, bye Steeplechase. r=dminor,froydnj
Differential Revision: https://phabricator.services.mozilla.com/D77114
2020-05-29 03:19:27 +00:00
Ricky Stewart
d620b9e6a6 Bug 1641242 - Handle difference in API of ImportError between py2 and py3 in artifact_commands.py r=catlee
Differential Revision: https://phabricator.services.mozilla.com/D77337
2020-05-28 20:33:03 +00:00
Axel Hecht
1343d506c0 Bug 1641184, use only the merge dir for repacks, r=nalexander
l10n-merge creates a full merge dir for a while now, let's
simplify the build logic to only read from that directory
during repacks and langpacks.

Differential Revision: https://phabricator.services.mozilla.com/D77023
2020-05-27 17:04:06 +00:00
Tarek Ziadé
c0c06d6f1f Bug 1641351 - raise on node error in BT r=acreskey
We forgot to check that the node command succeeded.
This patch fixes it.

Differential Revision: https://phabricator.services.mozilla.com/D77148
2020-05-27 21:43:38 +00:00
Geoff Brown
21053c0624 Bug 1638965 - Run 'mach gtest' in python 3; r=bc
Differential Revision: https://phabricator.services.mozilla.com/D77092
2020-05-27 20:53:22 +00:00
Artem
30dba11bdd Bug 1609556 - Migrate toolbar-context-menu to Fluent r=Gijs,fluent-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D76016
2020-05-27 18:09:40 +00:00
Mitchell Hentges
0482fe58ed Bug 1641073: improve BaseFile.copy() performance r=rstewart
Arrays are mutable, so appending each substring at a time and joining at the endis much faster than re-allocating
a new string on each loop.

Differential Revision: https://phabricator.services.mozilla.com/D76945
2020-05-27 16:52:51 +00:00
Ricky Stewart
1d10fc6783 Bug 1639624 - Don't import from distutils unless absolutely necessary in virtualenv.py r=froydnj
Similar reasoning to bug 1638780.

Differential Revision: https://phabricator.services.mozilla.com/D76221
2020-05-27 15:07:47 +00:00
Mike Hommey
a97c7ebfe7 Bug 1640578 - Remove --disable-install-strip from mac mozconfigs. r=froydnj
The need for --disable-install-strip in the mac mozconfigs comes from a
discrepancy in how stripping is handled between platforms. On Windows,
there is no stripping. On non-Mac unix, `strip` removes local symbols as
well as debug info and debug symbols. On Mac, it actually removes too
much, and one has to pass flags to remove both local symbols (`-x`) and
debug symbols (`-S`). Debug info is already in a separate file
(`.dSYM`).

For profiling reasons, we do ship e.g. nightlies with local symbols but
not debug info or symbols (or at least that's the intent). On Windows,
again, nothing to do. On non-Mac unix, we pass `--strip-debug` to
`strip` so that it keeps local symbols. That's where the discrepancy
comes in for Mac: the build system doesn't handle this at all, so the
mozconfigs contain --disable-install-strip to avoid stripping.

The build system should be doing what it's expected to be doing from the
start, without mozconfigs opting into anything.

AFAIK, we only really need the local symbols, so we can `strip -S` on
Mac when profiling is enabled, rather than `strip -x -S`. This also
significantly reduces the size of the installer for nightlies.

And while we're here, move the logic out of old-configure and into
python configure.

Differential Revision: https://phabricator.services.mozilla.com/D76789
2020-05-27 01:42:07 +00:00
Andi-Bogdan Postelnicu
622cf2a71a Bug 1640878 - for clang-tidy static-analysis keep a deep copy of the checks. r=froydnj
`StaticAnalysisMonitor` uses `clang-tidy` config dictionary in order to
match different options for the checkers that are being used. For this `StaticAnalysisMonitor`
modifies this dictionary. The dictionary not being a deep copy of the original one, the
modifications are echoed in the initial dictionary thus having the potential to lead to
unexpected behavior from `clang-tidy` when generating the checkers list.

Differential Revision: https://phabricator.services.mozilla.com/D76834
2020-05-26 17:23:38 +00:00
egao
c37762fb9f Bug 1640716 - correct string substitution in mozbuild/frontend/reader.py r=jmaher
Changes:

Current in-tree code will simply print `{path}` as many times as it reads a path.

This change will ensure that paths that have been read are output to the screen, which is much more useful.

Differential Revision: https://phabricator.services.mozilla.com/D76729
2020-05-26 09:57:17 +00:00
Tarek Ziadé
bddb8551ed Bug 1640747 - Add test-level support for handling exceptions while running the tests r=acreskey
This patch adds the `on_exception` hook, that will let developers
drive the runner behavior when a layer fails.

Differential Revision: https://phabricator.services.mozilla.com/D76794
2020-05-26 13:35:00 +00:00
Tarek Ziadé
92a2c142f7 Bug 1640649 - add --android-timeout r=sparky
This patch surfaces the timeout value for ADBDevice.
It also adds the ability to run a single test with
mach perftest-test and a new --skip-lint option
to skip black/flake8

Differential Revision: https://phabricator.services.mozilla.com/D76791
2020-05-26 14:42:58 +00:00
Tarek Ziadé
af3ea56177 Bug 1639321 - increase the coverage to 85 r=sparky
Increase the coverage to 85%

Differential Revision: https://phabricator.services.mozilla.com/D76048
2020-05-26 14:44:14 +00:00
Gregory Mierzwinski
f94f60a9c2 Bug 1640171 - Make the test file a str before using it. r=mozperftest-reviewers,tarek
This patch fixes a bug where a PosixPath is attempted to be used rather than a string path. There's also a regression in the coverage test run in `./mach perftest-test` on some platforms that is fixed by this patch (the attrs module being used by pytest is too old, so we update during setup). Lastly, a regression test is added to the browsertime tests to ensure that the command only contains knowingly compatible types.

Depends on D76713

Differential Revision: https://phabricator.services.mozilla.com/D76599
2020-05-25 20:07:58 +00:00
Geoff Brown
9e26d9583b Bug 1636797 - Purge __pycache__ in 'mach clobber python'; r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D76560
2020-05-23 19:37:09 +00:00
Gijs Kruitbosch
cb82792e02 Bug 1626842 - migrate the browser window title to fluent, r=zbraniecki,fluent-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D75854
2020-05-25 10:01:42 +00:00
Tarek Ziadé
d634c1741b Bug 1639493 - improved README r=acreskey
Added some minimal doc

Differential Revision: https://phabricator.services.mozilla.com/D76107
2020-05-20 13:12:15 +00:00
Tilden Windsor
b7355730e6 Bug 1640142 - Added spaces to "mach clang-format" help message. r=sylvestre
Differential Revision: https://phabricator.services.mozilla.com/D76467
2020-05-22 11:53:48 +00:00
Ricky Stewart
6cda2c27f5 Bug 1640261 - Expand set of possible Python 3's you can use in bootstrap/debian.py r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D76529
2020-05-22 19:34:08 +00:00
Ricky Stewart
e2fe57521b Bug 1638060 - Standardize interface of mozfile classes as bytes-based streams r=glandium
At the beginning of the Python 3 migration (circa bug 1602540), we made an update to the interface of `mozpack/files.py` in the direction of aligning with Python 3's built-in `file` support; namely, that opening a file in text mode returns a stream of `str` (text), and that opening a file in binary mode returns a stream of `bytes`. This was deemed to be more trouble than it was worth. This patch undoes all of those changes to the interface in favor of moving back to the Python 2 style, where all files are bytestreams.

Differential Revision: https://phabricator.services.mozilla.com/D75424
2020-05-22 01:11:29 +00:00
Andrew Halberstadt
0591e31c1f Bug 1639349 - [ci] Set up a SCHEDULES rule to exclude non-webrender tests on webrender-only pushes, r=gbrown
Depends on D76433

Differential Revision: https://phabricator.services.mozilla.com/D76434
2020-05-22 14:03:46 +00:00
adiabtaic28394
43294f5124 Bug 1632345 - mach doctor does not complain about noatime/reltime on tmpfs anymore r=Callek,glandium
Differential Revision: https://phabricator.services.mozilla.com/D75917
2020-05-22 13:36:42 +00:00
Mike Hommey
6ba1065508 Bug 1639815 - Move --enable-strip and --enable-install-strip to python configure. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D76291
2020-05-21 23:51:58 +00:00
Ricky Stewart
0a8ac76fef Bug 1639986 - Add hack to bootstrap to install Python 2 on macOS r=firefox-build-system-reviewers,nalexander
This is ugly and complicates the code some but it's manageable and allows us to keep things afloat on macOS while the testing team plugs along with the `mach` migration.

Differential Revision: https://phabricator.services.mozilla.com/D76386
2020-05-21 20:25:34 +00:00
Chris AtLee
cd35328624 Bug 1637381: Add support for extracting zst files to mozbuild tooltool r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D75203
2020-05-21 13:28:33 +00:00
Chris AtLee
ef685f3549 Bug 1637379: Optimize android toolchains r=gbrown
Create an emulator toolchain
Stop including mozboot/sdk-*.zip in our tarball

Differential Revision: https://phabricator.services.mozilla.com/D74931
2020-05-21 13:28:10 +00:00
Cameron McCormack
252e4e5c55 Bug 1639788 - Update IRC reference in mach bootstrap output. r=glob
Differential Revision: https://phabricator.services.mozilla.com/D76271
2020-05-21 05:39:24 +00:00