Commit Graph

5970 Commits

Author SHA1 Message Date
Axel Hecht
1343d506c0 Bug 1641184, use only the merge dir for repacks, r=nalexander
l10n-merge creates a full merge dir for a while now, let's
simplify the build logic to only read from that directory
during repacks and langpacks.

Differential Revision: https://phabricator.services.mozilla.com/D77023
2020-05-27 17:04:06 +00:00
Tarek Ziadé
c0c06d6f1f Bug 1641351 - raise on node error in BT r=acreskey
We forgot to check that the node command succeeded.
This patch fixes it.

Differential Revision: https://phabricator.services.mozilla.com/D77148
2020-05-27 21:43:38 +00:00
Geoff Brown
21053c0624 Bug 1638965 - Run 'mach gtest' in python 3; r=bc
Differential Revision: https://phabricator.services.mozilla.com/D77092
2020-05-27 20:53:22 +00:00
Artem
30dba11bdd Bug 1609556 - Migrate toolbar-context-menu to Fluent r=Gijs,fluent-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D76016
2020-05-27 18:09:40 +00:00
Mitchell Hentges
0482fe58ed Bug 1641073: improve BaseFile.copy() performance r=rstewart
Arrays are mutable, so appending each substring at a time and joining at the endis much faster than re-allocating
a new string on each loop.

Differential Revision: https://phabricator.services.mozilla.com/D76945
2020-05-27 16:52:51 +00:00
Ricky Stewart
1d10fc6783 Bug 1639624 - Don't import from distutils unless absolutely necessary in virtualenv.py r=froydnj
Similar reasoning to bug 1638780.

Differential Revision: https://phabricator.services.mozilla.com/D76221
2020-05-27 15:07:47 +00:00
Mike Hommey
a97c7ebfe7 Bug 1640578 - Remove --disable-install-strip from mac mozconfigs. r=froydnj
The need for --disable-install-strip in the mac mozconfigs comes from a
discrepancy in how stripping is handled between platforms. On Windows,
there is no stripping. On non-Mac unix, `strip` removes local symbols as
well as debug info and debug symbols. On Mac, it actually removes too
much, and one has to pass flags to remove both local symbols (`-x`) and
debug symbols (`-S`). Debug info is already in a separate file
(`.dSYM`).

For profiling reasons, we do ship e.g. nightlies with local symbols but
not debug info or symbols (or at least that's the intent). On Windows,
again, nothing to do. On non-Mac unix, we pass `--strip-debug` to
`strip` so that it keeps local symbols. That's where the discrepancy
comes in for Mac: the build system doesn't handle this at all, so the
mozconfigs contain --disable-install-strip to avoid stripping.

The build system should be doing what it's expected to be doing from the
start, without mozconfigs opting into anything.

AFAIK, we only really need the local symbols, so we can `strip -S` on
Mac when profiling is enabled, rather than `strip -x -S`. This also
significantly reduces the size of the installer for nightlies.

And while we're here, move the logic out of old-configure and into
python configure.

Differential Revision: https://phabricator.services.mozilla.com/D76789
2020-05-27 01:42:07 +00:00
Andi-Bogdan Postelnicu
622cf2a71a Bug 1640878 - for clang-tidy static-analysis keep a deep copy of the checks. r=froydnj
`StaticAnalysisMonitor` uses `clang-tidy` config dictionary in order to
match different options for the checkers that are being used. For this `StaticAnalysisMonitor`
modifies this dictionary. The dictionary not being a deep copy of the original one, the
modifications are echoed in the initial dictionary thus having the potential to lead to
unexpected behavior from `clang-tidy` when generating the checkers list.

Differential Revision: https://phabricator.services.mozilla.com/D76834
2020-05-26 17:23:38 +00:00
egao
c37762fb9f Bug 1640716 - correct string substitution in mozbuild/frontend/reader.py r=jmaher
Changes:

Current in-tree code will simply print `{path}` as many times as it reads a path.

This change will ensure that paths that have been read are output to the screen, which is much more useful.

Differential Revision: https://phabricator.services.mozilla.com/D76729
2020-05-26 09:57:17 +00:00
Tarek Ziadé
bddb8551ed Bug 1640747 - Add test-level support for handling exceptions while running the tests r=acreskey
This patch adds the `on_exception` hook, that will let developers
drive the runner behavior when a layer fails.

Differential Revision: https://phabricator.services.mozilla.com/D76794
2020-05-26 13:35:00 +00:00
Tarek Ziadé
92a2c142f7 Bug 1640649 - add --android-timeout r=sparky
This patch surfaces the timeout value for ADBDevice.
It also adds the ability to run a single test with
mach perftest-test and a new --skip-lint option
to skip black/flake8

Differential Revision: https://phabricator.services.mozilla.com/D76791
2020-05-26 14:42:58 +00:00
Tarek Ziadé
af3ea56177 Bug 1639321 - increase the coverage to 85 r=sparky
Increase the coverage to 85%

Differential Revision: https://phabricator.services.mozilla.com/D76048
2020-05-26 14:44:14 +00:00
Gregory Mierzwinski
f94f60a9c2 Bug 1640171 - Make the test file a str before using it. r=mozperftest-reviewers,tarek
This patch fixes a bug where a PosixPath is attempted to be used rather than a string path. There's also a regression in the coverage test run in `./mach perftest-test` on some platforms that is fixed by this patch (the attrs module being used by pytest is too old, so we update during setup). Lastly, a regression test is added to the browsertime tests to ensure that the command only contains knowingly compatible types.

Depends on D76713

Differential Revision: https://phabricator.services.mozilla.com/D76599
2020-05-25 20:07:58 +00:00
Geoff Brown
9e26d9583b Bug 1636797 - Purge __pycache__ in 'mach clobber python'; r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D76560
2020-05-23 19:37:09 +00:00
Gijs Kruitbosch
cb82792e02 Bug 1626842 - migrate the browser window title to fluent, r=zbraniecki,fluent-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D75854
2020-05-25 10:01:42 +00:00
Tarek Ziadé
d634c1741b Bug 1639493 - improved README r=acreskey
Added some minimal doc

Differential Revision: https://phabricator.services.mozilla.com/D76107
2020-05-20 13:12:15 +00:00
Tilden Windsor
b7355730e6 Bug 1640142 - Added spaces to "mach clang-format" help message. r=sylvestre
Differential Revision: https://phabricator.services.mozilla.com/D76467
2020-05-22 11:53:48 +00:00
Ricky Stewart
6cda2c27f5 Bug 1640261 - Expand set of possible Python 3's you can use in bootstrap/debian.py r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D76529
2020-05-22 19:34:08 +00:00
Ricky Stewart
e2fe57521b Bug 1638060 - Standardize interface of mozfile classes as bytes-based streams r=glandium
At the beginning of the Python 3 migration (circa bug 1602540), we made an update to the interface of `mozpack/files.py` in the direction of aligning with Python 3's built-in `file` support; namely, that opening a file in text mode returns a stream of `str` (text), and that opening a file in binary mode returns a stream of `bytes`. This was deemed to be more trouble than it was worth. This patch undoes all of those changes to the interface in favor of moving back to the Python 2 style, where all files are bytestreams.

Differential Revision: https://phabricator.services.mozilla.com/D75424
2020-05-22 01:11:29 +00:00
Andrew Halberstadt
0591e31c1f Bug 1639349 - [ci] Set up a SCHEDULES rule to exclude non-webrender tests on webrender-only pushes, r=gbrown
Depends on D76433

Differential Revision: https://phabricator.services.mozilla.com/D76434
2020-05-22 14:03:46 +00:00
adiabtaic28394
43294f5124 Bug 1632345 - mach doctor does not complain about noatime/reltime on tmpfs anymore r=Callek,glandium
Differential Revision: https://phabricator.services.mozilla.com/D75917
2020-05-22 13:36:42 +00:00
Mike Hommey
6ba1065508 Bug 1639815 - Move --enable-strip and --enable-install-strip to python configure. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D76291
2020-05-21 23:51:58 +00:00
Ricky Stewart
0a8ac76fef Bug 1639986 - Add hack to bootstrap to install Python 2 on macOS r=firefox-build-system-reviewers,nalexander
This is ugly and complicates the code some but it's manageable and allows us to keep things afloat on macOS while the testing team plugs along with the `mach` migration.

Differential Revision: https://phabricator.services.mozilla.com/D76386
2020-05-21 20:25:34 +00:00
Chris AtLee
cd35328624 Bug 1637381: Add support for extracting zst files to mozbuild tooltool r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D75203
2020-05-21 13:28:33 +00:00
Chris AtLee
ef685f3549 Bug 1637379: Optimize android toolchains r=gbrown
Create an emulator toolchain
Stop including mozboot/sdk-*.zip in our tarball

Differential Revision: https://phabricator.services.mozilla.com/D74931
2020-05-21 13:28:10 +00:00
Cameron McCormack
252e4e5c55 Bug 1639788 - Update IRC reference in mach bootstrap output. r=glob
Differential Revision: https://phabricator.services.mozilla.com/D76271
2020-05-21 05:39:24 +00:00
Csoregi Natalia
160736caed Backed out changeset 4376d47f325b (bug 1639624) for bustage. CLOSED TREE 2020-05-21 03:58:54 +03:00
Ricky Stewart
5aefcd726b Bug 1639624 - Don't unconditionally import VirtualenvManager in python/mach_commands.py. r=froydnj
Similar reasoning to bug 1638780.

Differential Revision: https://phabricator.services.mozilla.com/D76221
2020-05-20 23:32:43 +00:00
Dan Minor
fdcf957cca Bug 1634675 - Remove webrtc-gtests r=drno,froydnj
Differential Revision: https://phabricator.services.mozilla.com/D73451
2020-05-20 19:59:45 +00:00
Ricky Stewart
e8d734c549 Bug 1639521 - Fix Py3 incompatibility in mach artifact r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D76215
2020-05-20 20:34:51 +00:00
Yue Zhang
d8d83da843 Bug 1636535 - Remove subclass requirement for custom data transformers. r=sparky
Differential Revision: https://phabricator.services.mozilla.com/D75394
2020-05-20 19:48:58 +00:00
Alex Vincent
449dbae8b1 Bug 1588166 - Remove python-dbus build dependency. (notify-send replaced it and it's no longer needed.) r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D74798
2020-05-20 15:19:13 +00:00
Ricky Stewart
1a44b2a1d9 Bug 1638983 - Run mach repackage with Python 3 r=firefox-build-system-reviewers,glandium
Differential Revision: https://phabricator.services.mozilla.com/D76058
2020-05-20 15:35:58 +00:00
Tarek Ziadé
f8d5627e71 Bug 1639212 - allow URLs for hooks and scripts r=sparky
This patch allows using URLS for --hooks and scripts

Differential Revision: https://phabricator.services.mozilla.com/D75967
2020-05-19 20:27:27 +00:00
Gregory Mierzwinski
7788d4b576 Bug 1639211 - Implement --perfherder-app in mozperftest Perfherder layer. r=mozperftest-reviewers,tarek
This patch adds a new --perfherder-app that can be used to specify the shorthand name of the application being tested. The choices here are limited to what the perfherder schema can accept. We also add these settings to the existing taskcluster tasks to split the perfherder data. A couple tests are also added for this new flag.

Differential Revision: https://phabricator.services.mozilla.com/D76010
2020-05-19 19:07:27 +00:00
Ricky Stewart
aefac46494 Bug 1638951 - Run mach cargo with Python 3 r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D75999
2020-05-19 19:35:52 +00:00
Gregory Mierzwinski
99953d32a0 Bug 1638001 - Add test name from metadata to perfherder data. r=acreskey,mozperftest-reviewers,tarek
This patch adds a `test_name` field to the test metadata that can be used to change the name of the test when output in the perfherder format. It also implements this for the app-link tests.

Differential Revision: https://phabricator.services.mozilla.com/D75544
2020-05-19 19:52:52 +00:00
Tarek Ziadé
7d6014b5a8 Bug 1635573 - vendor coverage r=ahal
Vendor coverage, and make sure we can run out of third_party as much as possible

Differential Revision: https://phabricator.services.mozilla.com/D74531
2020-05-19 15:04:28 +00:00
Ricky Stewart
57d020b2e6 Bug 1638780 - Don't import virtualenv unless necessary in mach commands r=froydnj
The module `distutils.sysconfig` isn't installed by default on Ubuntu -- the package `python3-distutils` exposes it. That's fine, except we unconditionally import it in `virtualenv.py`, which gets up indirectly being imported whenever you run any `mach` command, which will cause `bootstrap` to break before it even has the chance to install it.

`python3-pip` seems to rely on `python3-distutils` being installed so `bootstrap` will install it, so all we need to do is make sure it doesn't import `virtualenv` in any circumstance unless it's necessary (when surfacing an error in the module would be appropriate).

Differential Revision: https://phabricator.services.mozilla.com/D75833
2020-05-19 12:07:47 +00:00
Nathan Froyd
9f876bab7a Bug 1638012 - commonize taskcluster-related bootstrappers for Linux; r=nalexander
This change doesn't fix all of the boilerplate involved in declaring
that certain packages should be fetched from taskcluster, but it's a
start, at least.

Differential Revision: https://phabricator.services.mozilla.com/D75330
2020-05-18 16:27:09 +00:00
Andi-Bogdan Postelnicu
86f93ce272 Bug 1635112 - Mark clang-diagnostic-error as publishable in the clang-tidy artifact. r=marco
For `clang-tidy` static-analysis mark all diagnostic-messages that are `clang-diagnostic-error` as publishable.

Differential Revision: https://phabricator.services.mozilla.com/D74229
2020-05-08 07:19:51 +00:00
Andi-Bogdan Postelnicu
0b18f7ab13 Bug 1635112 - Disable the build error reporting at review-phase using Coverity. r=sylvestre,marco
Differential Revision: https://phabricator.services.mozilla.com/D74003
2020-05-08 09:30:02 +00:00
Andi-Bogdan Postelnicu
ac1e721802 Bug 1635112 - Use clang-tidy as warning build error detector for our static-analysis targets. r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D73692
2020-05-08 07:19:23 +00:00
Andi-Bogdan Postelnicu
7a23d681c5 Bug 1635112 - remove MSVC support from WarningsCollector. r=glandium
MSVC is not supported by the build system anymore.

Differential Revision: https://phabricator.services.mozilla.com/D73809
2020-05-08 07:19:02 +00:00
Rob Lemley
bd812fce54 Bug 1631197 - Make mach valgrind-test work for Thunderbird builds. r=froydnj
Create a new MachCommandCondition, "is_firefox_or_thunderbird" which is then used
to allow mach valgrind-test work for Thunderbird builds.

Differential Revision: https://phabricator.services.mozilla.com/D73153
2020-05-19 00:10:06 +00:00
Ricky Stewart
ab8348bc51 Bug 1638783 - Install Mercurial with pip3 rather than pip, which no longer exists on new Ubuntu/Debian versions r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D75816
2020-05-18 20:10:19 +00:00
Corentin Arnould
e7a750541a Bug 1638799 - Added python-pip to archlinux's bootstrap. r=froydnj
pip3 is an optional dependency of python on archlinux. It needs to be installed explicitly.

Differential Revision: https://phabricator.services.mozilla.com/D75785
2020-05-18 12:50:54 +00:00
Dão Gottwald
379b7cff73 Bug 1638362 - Rename urlbar-go-end-cap l10n id to urlbar-go-button. r=fluent-reviewers,flod
Differential Revision: https://phabricator.services.mozilla.com/D75558
2020-05-15 16:08:22 +00:00
Tarek Ziadé
ffc7bb2c00 Bug 1637959 - Added more test coverage r=sparky
Differential Revision: https://phabricator.services.mozilla.com/D75309
2020-05-15 07:19:39 +00:00
Mike Hommey
28de21d935 Bug 1638193 - Remove build system support for iOS. r=nalexander
iOS support for Gecko has not been tested in years and is most probably
out of date. The build system part of it, specifically the checks in
build/autoconf/ios.m4, are not trivial to port to python configure, and
they prevent other things from moving to python configure (because some
of them change value when MOZ_IOS is set).

The code is left alone, although it could probably be stripped off as
well, but I'll leave that as an exercise for someone else.

Differential Revision: https://phabricator.services.mozilla.com/D75463
2020-05-15 03:56:16 +00:00