Commit Graph

6125 Commits

Author SHA1 Message Date
Geoff Brown
6612d6856b Bug 1648506 - Disable some python tests on py2 on mac; r=mhentges
These tests fail on py2 on osx when 'mach python-test' runs with python 3, blocking
bug 1594914.

Differential Revision: https://phabricator.services.mozilla.com/D81247
2020-06-25 23:47:38 +00:00
Jon Bauman
e4bc88f3e9 Bug 1648278 - Document how DEFINES works with the assembler. r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D80977
2020-06-25 17:57:02 +00:00
Ben Hearsum
8be667a9ad Bug 1631409: run multi-l10n step for artifact builds to fix test failures on try r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D80701
2020-06-24 20:54:28 +00:00
Ricky Stewart
d44b1b67a3 Bug 1647873 - Remove "mach bootstrap is not fully implemented in MozillaBuild" message r=nalexander
This message dates from bug 1177128, the commit where this bootstrapper was added. At the time of writing that was certainly true, but today it doesn't appear to be true in any meaningful way. There's certainly an expectation that installing MozillaBuild and running `mach bootstrap` will get your machine ready to build Firefox; or at least, if it doesn't, it prints out some error messages telling you to do extra stuff, like we see in bug 1644794 for example. Therefore, I don't see how this message is relevant at all any more.

Differential Revision: https://phabricator.services.mozilla.com/D80730
2020-06-24 02:47:43 +00:00
Francesco Lodolo (:flod)
a4b5e4d9bd Bug 1647692 - Add language packs and dictionaries to about:support, r=Gijs,fluent-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D80808
2020-06-24 13:30:22 +00:00
Mike Hommey
a87a1c3b54 Bug 1645986 - Move most GeneratedFile generation to top-level backend.mk. r=firefox-build-system-reviewers,rstewart
While ideally we'd just move all of them at the top-level, there are
other things that depend on them that wouldn't then get the right
dependencies. One of them is install steps in dist/something, but there
are others, and that's a rather long pole of things requiring many
changes along with chances to break things.

So instead, we go with a simpler and more limited approach, where we
still recurse into directories to run their export tier (to run whatever
else than GeneratedFiles they have), but ensure the GeneratedFiles we
moved at the top-level are built before recursing by making
directory/export depend on them.

Differential Revision: https://phabricator.services.mozilla.com/D80608
2020-06-24 04:23:02 +00:00
Mike Hommey
0636209ca5 Bug 1645986 - Properly mark generated files required during compile as such. r=firefox-build-system-reviewers,rstewart
Ideally, this kind of manual work wouldn't be required, but the frontend
doesn't provide enough information and while backend could correlate the
information, they currently don't.

Differential Revision: https://phabricator.services.mozilla.com/D80606
2020-06-24 05:07:25 +00:00
Dorel Luca
fdb71d7688 Backed out 4 changesets (bug 1645986) for Valgrid failure. CLOSED TREE
Backed out changeset 602ea804234f (bug 1645986)
Backed out changeset c43f9360b0a5 (bug 1645986)
Backed out changeset fe1142397e72 (bug 1645986)
Backed out changeset faf6de9409f9 (bug 1645986)
2020-06-24 07:16:13 +03:00
Mike Hommey
8a86310ac3 Bug 1645986 - Move most GeneratedFile generation to top-level backend.mk. r=firefox-build-system-reviewers,rstewart
While ideally we'd just move all of them at the top-level, there are
other things that depend on them that wouldn't then get the right
dependencies. One of them is install steps in dist/something, but there
are others, and that's a rather long pole of things requiring many
changes along with chances to break things.

So instead, we go with a simpler and more limited approach, where we
still recurse into directories to run their export tier (to run whatever
else than GeneratedFiles they have), but ensure the GeneratedFiles we
moved at the top-level are built before recursing by making
directory/export depend on them.

Differential Revision: https://phabricator.services.mozilla.com/D80608
2020-06-24 00:45:25 +00:00
Mike Hommey
94ff33a728 Bug 1645986 - Properly mark generated files required during compile as such. r=firefox-build-system-reviewers,rstewart
Ideally, this kind of manual work wouldn't be required, but the frontend
doesn't provide enough information and while backend could correlate the
information, they currently don't.

Differential Revision: https://phabricator.services.mozilla.com/D80606
2020-06-24 01:03:40 +00:00
Dorel Luca
b159956383 Backed out 4 changesets (bug 1645986) for Build bustage
Backed out changeset ac871b2f34dc (bug 1645986)
Backed out changeset 07b14b6a0b91 (bug 1645986)
Backed out changeset fdd319bdbfa7 (bug 1645986)
Backed out changeset ce213e6bbc1f (bug 1645986)
2020-06-24 03:42:47 +03:00
Mike Hommey
f6fcdd8d5c Bug 1645986 - Move most GeneratedFile generation to top-level backend.mk. r=firefox-build-system-reviewers,rstewart
While ideally we'd just move all of them at the top-level, there are
other things that depend on them that wouldn't then get the right
dependencies. One of them is install steps in dist/something, but there
are others, and that's a rather long pole of things requiring many
changes along with chances to break things.

So instead, we go with a simpler and more limited approach, where we
still recurse into directories to run their export tier (to run whatever
else than GeneratedFiles they have), but ensure the GeneratedFiles we
moved at the top-level are built before recursing by making
directory/export depend on them.

Differential Revision: https://phabricator.services.mozilla.com/D80608
2020-06-23 16:07:26 +00:00
Mike Hommey
c54b26945c Bug 1645986 - Properly mark generated files required during compile as such. r=firefox-build-system-reviewers,rstewart
Ideally, this kind of manual work wouldn't be required, but the frontend
doesn't provide enough information and while backend could correlate the
information, they currently don't.

Differential Revision: https://phabricator.services.mozilla.com/D80606
2020-06-23 20:47:06 +00:00
Sylvestre Ledru
56fbafd822 Bug 1647265 - mozlint: when type is 'regex', add the capability to ignore the case r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D80477
2020-06-23 21:45:16 +00:00
Mitchell Hentges
7914702f50 Bug 1647865: Bootstrap should stop if run with Python 2 r=rstewart
This uses sys.exit() instead of throwing an exception since:
1. It isn't an error, and bug 1646406 isn't finished yet
2. The output is more straightforward for the user

This verification happens separately from the other python-verification logic, but that's
because I wanted the validation to happen as soon as possible - before the user invests too
much time, and before too much of the script ran with the wrong interpreter.

Differential Revision: https://phabricator.services.mozilla.com/D80726
2020-06-23 21:40:55 +00:00
Mike Hommey
f3c4e8c67a Bug 1644371 - Update the cargo version requirement for mach vendor rust. r=dmajor
1.38 changed the Cargo.lock format, and the current minimum (1.37) can't
read it, so the minimum version was already wrong.

1.41 has a problem vendoring nom for some reason, that appears to be
fixed in 1.42.

Differential Revision: https://phabricator.services.mozilla.com/D80593
2020-06-23 13:13:36 +00:00
Andi-Bogdan Postelnicu
d0f21af448 Bug 1647629 - Make infer static-analysis compatible with python3. r=sylvestre
Differential Revision: https://phabricator.services.mozilla.com/D80633
2020-06-23 11:42:51 +00:00
James Graham
2ade9fa1fd Bug 1644395 - Add wpt print reftests to TC config, r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D79082
2020-06-23 10:00:28 +00:00
Narcis Beleuzu
a0c8ce672c Backed out 4 changesets (bug 1644395) for bc failures on browser_all_files_referenced.js . CLOSED TREE
Backed out changeset 582c1dd945ca (bug 1644395)
Backed out changeset 9dbba532a700 (bug 1644395)
Backed out changeset 59e998eb5e59 (bug 1644395)
Backed out changeset 035ddba4a4d9 (bug 1644395)
2020-06-23 00:56:49 +03:00
Andrew Halberstadt
15aaba2531 Bug 1646427 - [vendor] Exclude test and documentation dirs when vendoring Python packages, r=rstewart
This adds a way to exclude certain file patterns when extracting Python
packages in |mach vendor python|. For now I've excluded likely test and doc
dirs, though we may want to expand on the list in the future.

Differential Revision: https://phabricator.services.mozilla.com/D80209
2020-06-22 13:20:57 +00:00
James Graham
5706e82bce Bug 1644395 - Add wpt print reftests to TC config, r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D79082
2020-06-22 13:44:13 +00:00
Ricky Stewart
eb00073e1a Bug 1646840 - Ensure mach vendor rust writes out .cargo/config.in with LF newlines on Windows r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D80474
2020-06-22 15:24:12 +00:00
John Paul Adrian Glaubitz
41f3e19379 Bug 1318905 - build: Add riscv64 as target architecture to mozbuild r=glandium
Adds the basic definitions for riscv64 to mozbuild, allowing to build Spidermonkey.

Differential Revision: https://phabricator.services.mozilla.com/D78623
2020-06-18 07:11:09 +00:00
Sylvestre Ledru
c5d947b77f Bug 1646421 - flake8 followup, remove the noqa as it breaks the test
Instead, ignore this specific test for this file

Differential Revision: https://phabricator.services.mozilla.com/D80374
2020-06-20 12:21:27 +00:00
Sylvestre Ledru
46af48052b Bug 1646421 - flake8 Fix a bunch of actual errors r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D80078
2020-06-20 09:46:45 +00:00
Geoff Brown
dbfd31bcb6 Bug 1644876 - Add --enable-fission option to 'mach run'; r=kashav
Differential Revision: https://phabricator.services.mozilla.com/D80298
2020-06-19 16:20:00 +00:00
Mike Hommey
7fc03d8e13 Bug 1600530 - Consider MOZ_OBJDIR from the environment when there is no mozconfig. r=nalexander
We currently only deal with MOZ_OBJDIR in the environment when there is
a mozconfig.

Differential Revision: https://phabricator.services.mozilla.com/D80308
2020-06-19 03:31:11 +00:00
Mike Hommey
eab19f0855 Bug 1643258 - Avoid the use of the __cxa_thread_atexit_impl symbol. r=froydnj
When linking a weak symbol in an object against a library where the
symbol is provided with a version, the final binary get a weak versioned
symbol reference.

It turns out weak versioned symbols still make the dynamic linker need
the symbol version, even if all symbols needed with that version are
weak.

Practically speaking, that means with bug 1634204, we now end up with
a weak versioned symbol reference to __cxa_thread_atexit_impl with
version GLIBC_2.18, and glibcs without the symbol can't fulfil that
version, even though the weak symbol is the only thing we need from that
version.

This means the check_binary changes in bug 1634204 are too
relaxed, so we revert them (although we keep the easier to read
conditions in check_dep_versions).

We also introduce a hack in stdc++compat.cpp (although it's not
technically entirely about libstdc++ compat) so that we avoid the weak
symbol reference while keeping the intended baseline for libstdc++ and
glibc.

Differential Revision: https://phabricator.services.mozilla.com/D79773
2020-06-18 23:45:41 +00:00
Gregory Mierzwinski
91291fe122 Bug 1646697 - Modify how the multi-commit tasks work. r=mozperftest-reviewers,tarek
This patch adds a `--test-date` option to mozperftest that lets us specify the date that we are testing on. This lets us trigger multi-commit tests on past dates, and also ensures that retriggers of the multi-commit tests use the same day that they were pushed on (giving us reproducibility).

Differential Revision: https://phabricator.services.mozilla.com/D80173
2020-06-18 22:37:35 +00:00
Gregory Mierzwinski
b1676248ea Bug 1646368 - Add performance tuning feature to android layer. r=mozperftest-reviewers,tarek
This patch ports the performance tuning functionality from Raptor to mozperftest and adds a small test for it.

Differential Revision: https://phabricator.services.mozilla.com/D80168
2020-06-18 22:37:27 +00:00
Mitchell Hentges
f3b96a236f Bug 1645913: show mach command in Sentry breadcrumbs r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D80124
2020-06-18 21:31:46 +00:00
Dorel Luca
f7cdd7d12e Backed out changeset b707f591bef5 (bug 1643258) for build bustage. CLOSED TREE 2020-06-19 00:26:24 +03:00
Mike Hommey
39efc5bc7b Bug 1646653 - Remove dependencies on xpcom/xpidl/export. r=froydnj
They were necessary some time ago, but:
- there is no js/xpconnect/src/export target anymore,
- accessibility/xpcom/export had a dependency on xpcom/xpidl/export for
  the ply cache files, but they're not used as of bug 1633156.
- the recursive make backend was adding automatic dependencies on
  xpcom/xpidl/export for the ply cache files.

Differential Revision: https://phabricator.services.mozilla.com/D80150
2020-06-18 11:02:46 +00:00
Mike Hommey
9d26424253 Bug 1643258 - Avoid the use of the __cxa_thread_atexit_impl symbol. r=froydnj
When linking a weak symbol in an object against a library where the
symbol is provided with a version, the final binary get a weak versioned
symbol reference.

It turns out weak versioned symbols still make the dynamic linker need
the symbol version, even if all symbols needed with that version are
weak.

Practically speaking, that means with bug 1634204, we now end up with
a weak versioned symbol reference to __cxa_thread_atexit_impl with
version GLIBC_2.18, and glibcs without the symbol can't fulfil that
version, even though the weak symbol is the only thing we need from that
version.

This means the check_binary changes in bug 1634204 are too
relaxed, so we revert them (although we keep the easier to read
conditions in check_dep_versions).

We also introduce a hack in stdc++compat.cpp (although it's not
technically entirely about libstdc++ compat) so that we avoid the weak
symbol reference while keeping the intended baseline for libstdc++ and
glibc.

Differential Revision: https://phabricator.services.mozilla.com/D79773
2020-06-18 11:02:43 +00:00
Mitchell Hentges
c9ee95076b Bug 1645438: Sentry no longer patches <topobjdir> path r=rstewart
Sentry needs to be able to send data in a minimal environment, and intelligently determining
the "topobjdir" (without leaning on Mach) is tough.

Instead, since the "topobjdir" usually falls under the "topsrcdir", just leaning on <topsrcdir>
patching should be sufficient.

Differential Revision: https://phabricator.services.mozilla.com/D80122
2020-06-18 13:47:21 +00:00
Sylvestre Ledru
4b7c742212 Bug 1646405 - update the code using pyyaml to use the new path r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D80025
2020-06-18 19:40:28 +00:00
Yue Zhang
f9a1c0c02d Bug 1640951 - Implement a jsonschema for results coming from transformer merge functions. r=sparky
Differential Revision: https://phabricator.services.mozilla.com/D78622
2020-06-18 16:48:05 +00:00
Ricky Stewart
c371fbe069 Bug 1646748 - mach clobber python no longer deletes arbitrary untracked files in python/ r=nalexander
This is a documented feature of `mach clobber python`, which I never noticed until bug 1645889 landed. I think it's fair to say that this is not how one would intuitively expect the command to work given that I'm the "owner" of this `mach` command and I never noticed this quirk until today. :)

`python/` shouldn't contain compiled C extensions or anything and the existing `*.py[cdo]` and `*/__pycache__` clauses should get rid of most, if not all, of the relevant garbage in the directory.

If the original behavior turns out to be desirable, we can add it back in (maybe we can use the `--full` command line option?)

Differential Revision: https://phabricator.services.mozilla.com/D80204
2020-06-18 16:42:51 +00:00
Kagami Sascha Rosylight
0c5130e3a4 Bug 1643192 - Use git log to get base ref r=glandium
Since there are still needs for .base_ref(), this uses git log way to get base commit.

Differential Revision: https://phabricator.services.mozilla.com/D78183
2020-06-18 06:39:55 +00:00
Butkovits Atila
40766bf579 Backed out 2 changesets (bug 1643258) for causing Bug 1646625.
Backed out changeset 4c359d743fb2 (bug 1643258)
Backed out changeset 439eee50f354 (bug 1643258)
2020-06-18 08:56:12 +03:00
Jim Porter
c9a16c40ed Bug 1631806 - [part 1] Update perftests to use new API and fix --proxy option; r=tarek
This patch fixes the tests in testing/performance/ that hadn't been updated to
the new API, as well as fixing a couple issues with the `--proxy` option:

 * `policy.json` wasn't being logged correctly, since mozlog was treating the
   data as a format string

 * Proxy prefs weren't being set correctly for the profile

Differential Revision: https://phabricator.services.mozilla.com/D79937
2020-06-18 03:59:54 +00:00
Mitchell Hentges
bfb8185ee4 Bug 1645445: Perform case-insensitive path-patching in Sentry r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D80113
2020-06-17 23:03:33 +00:00
Mitchell Hentges
afebc69970 Bug 1646099: Scrub hostname from sentry events r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D80112
2020-06-17 22:52:04 +00:00
Gijs Kruitbosch
67234feea3 Bug 1257155 - make mach run create console output by default on windows so dump/printf works in the parent process, r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D80095
2020-06-17 22:39:37 +00:00
Mike Hommey
f764089a91 Bug 1643258 - Avoid the use of the __cxa_thread_atexit_impl symbol. r=froydnj
When linking a weak symbol in an object against a library where the
symbol is provided with a version, the final binary gets a weak versioned
symbol reference.

It turns out weak versioned symbols still make the dynamic linker need
the symbol version, even if all symbols needed with that version are
weak.

Practically speaking, that means with bug 1634204, we now end up with
a weak versioned symbol reference to __cxa_thread_atexit_impl with
version GLIBC_2.18, and glibcs without the symbol can't fulfil that
version, even though the weak symbol is the only thing we need from that
version.

This means the check_binary changes in bug 1634204 are too
relaxed, so we revert them (although we keep the easier to read
conditions in check_dep_versions).

We also introduce a hack in stdc++compat.cpp (although it's not
technically entirely about libstdc++ compat) so that we avoid the weak
symbol reference while keeping the intended baseline for libstdc++ and
glibc.

Differential Revision: https://phabricator.services.mozilla.com/D79773
2020-06-17 03:16:08 +00:00
Mihai Alexandru Michis
6387c1ed23 Backed out changeset fa227b47dac7 (bug 1631806) for causing awsy failures regarding mitmproxy.
CLOSED TREE
2020-06-18 00:27:47 +03:00
Jim Porter
da72965e62 Bug 1631806 - [part 1] Update perftests to use new API and fix --proxy option; r=tarek
This patch fixes the tests in testing/performance/ that hadn't been updated to
the new API, as well as fixing a couple issues with the `--proxy` option:

 * `policy.json` wasn't being logged correctly, since mozlog was treating the
   data as a format string

 * Proxy prefs weren't being set correctly for the profile

Differential Revision: https://phabricator.services.mozilla.com/D79937
2020-06-17 09:41:24 +00:00
Mitchell Hentges
58848678dc Bug 1644794: Improve "missing JDK" error message and docs r=rstewart,geckoview-reviewers,agi
Additionally, recommends the installation of AdoptOpenJDK instead of the Oracle JDK.
This required a different "JAVA_HOME" detection mechanism, which is implemented in this revision.

Differential Revision: https://phabricator.services.mozilla.com/D79547
2020-06-17 12:17:48 +00:00
Ricky Stewart
40da5b0814 Bug 1646431 - Fix type error in mach file-info bugzilla-component r=nalexander
The `key` function to the `sort` call here was janky, so this produced an error if you tried to invoke this command on a path that doesn't have an owner (`TypeError: '<' not supported between instances of 'TypedTuple' and 'NoneType'`).

Differential Revision: https://phabricator.services.mozilla.com/D80043
2020-06-17 17:16:06 +00:00
Ricky Stewart
ce17cf368b Bug 1645179 - Squash remaining dead code after removal of IMPACTED_TESTS r=ahal,froydnj
There is some remaining code in central originating from bug 1184405, which sought to associate source files with their "affected" test files. That ended up not panning out, and bug 1644228 removed a lot of that code, but left some remnants in the `Files` object which are still referenced in a couple different places. I'm deleting all of that code in `context.py` plus everything that references it for the following reasons:

1. Right now, `Files.{test_files,test_tags,test_flavors}` do get populated, but only ever with "default" values -- namely `moz.build` files that are above the files in question in the directory hierarchy. This is a heuristic that doesn't actually have anything to do with mapping source files to their corresponding test files, which is misleading.

2. Those attributes are accessed in two places. The first is in the `mach file-info dep-tests` command. This command isn't referenced anywhere else in tree and I don't have any evidence anyone ever uses it. Even if they do, I would claim that doing so is a mistake (because the results of the command aren't meaningful and are just populated by the "defaults" described above), and that person's workflow should be migrated to something else that *is* meaningful.

3. The second place where this metadata is accessed is in `testing/mozbase/moztest/moztest/resolve.py`; that method is invoked in `tools/tryselect/selectors/syntax.py`, but only if you pass `--detect-paths` to `mach try syntax`. This is [entirely broken](https://bugzilla.mozilla.org/show_bug.cgi?id=1614614), and even if we made an effort to fix it, it wouldn't do anything resembling what the documentation of `--detect-paths` suggests it would do (again, because the data isn't populated meaningfully). So I'm deleting the command line option entirely.

Differential Revision: https://phabricator.services.mozilla.com/D79711
2020-06-17 13:30:49 +00:00