- moved placement of the raise_exception computation to be after the initial objdir path computation.
- original implementation was missing detection for cases where user may pass in a False flag to explicitly suppress errors.
- added unit tests to check for scenarios where raise_exception flag is passed in as part of positional argument.
- changed argument to include a **kwargs argument for caller to modify default exception behavior.
- default behavior is to raise exceptions if mozinfo.json cannot be found.
- disabled TreeMetadataEmitter from calling mozinfo.find_and_update_from_json and setting the self.info variable since it was not referenced anywhere else after the initial setup.
Depends on D6859
Differential Revision: https://phabricator.services.mozilla.com/D6860
--HG--
extra : moz-landing-system : lando
There is only a single linter (test-disable.yml) that uses a glob in any
include path, and that usage is easily replaced by using the 'extensions' key
instead.
Since globs in include directives aren't very useful, let's disallow them. This
will allow us to simplify the 'filterpaths' logic quite substantially and make
future refactorings in this area easier.
Differential Revision: https://phabricator.services.mozilla.com/D6798
--HG--
extra : moz-landing-system : lando
This extracts the current logic for finding nodejs into its own module in mozbuild. Configure and ESLint then use it.
For ESLint, this will change the first location it looks for nodejs to be the .mozbuild directory.
Differential Revision: https://phabricator.services.mozilla.com/D6430
--HG--
extra : moz-landing-system : lando
This makes things more explicit. Previously we were relying on those magic
global "linters" variables, and it turned out that one of the tests was
actually linting a completely different set of linters than I was expecting.
This changes things so each test needs to explicitly define which linters it
wants to use.
Depends on D6410
Differential Revision: https://phabricator.services.mozilla.com/D6412
--HG--
extra : moz-landing-system : lando
This makes this test match all the other tests (which are named after the module
they are testing).
Also rename the test function to 'test_filterpaths'.
Differential Revision: https://phabricator.services.mozilla.com/D6410
--HG--
rename : python/mozlint/test/test_filterpaths.py => python/mozlint/test/test_pathutils.py
extra : moz-landing-system : lando
This patch gets us building an updater binary that always embeds the dep certificates (instead of release or nightly), and builds a new tests package that includes it.
This was originally D5900, but that was backed out due to busting artifact builds. I've fixed that by removing the Makefile that Ted pointed out is unnecessary.
Differential Revision: https://phabricator.services.mozilla.com/D6418
--HG--
extra : moz-landing-system : lando
Migrates strings from aboutUrlClassifier.dtd and aboutUrlClassifier.properties files to aboutUrlClassifier.ftl
Modifies aboutUrlClassifier.xhtml and aboutUrlClassifier.js to use fluent.
Differential Revision: https://phabricator.services.mozilla.com/D5156
--HG--
extra : moz-landing-system : lando
Changes the wpt manifest path to the topobjdir instead so it can be moved out of tree.
Other changes so that the manifest download and update, and |mach wpt| and |mach test <wpt-test>| work with the new path.
The manifest is also downloaded and updated when creating the tests-archive to ensure that it exists when we run tests on TC.
MozReview-Commit-ID: Fp6UsKJjhTU
Differential Revision: https://phabricator.services.mozilla.com/D5312
--HG--
extra : moz-landing-system : lando
Add support for |with Files('a/**', 'b/**')| in mozbuild config files.
MozReview-Commit-ID: IoM4qfEhXXc
Differential Revision: https://phabricator.services.mozilla.com/D5315
--HG--
extra : moz-landing-system : lando
This is a quick fix to workaround linking in rkv and lmdb-sys in the tup
backend. Bug 1492291 is a followup to actually fix the problem more
generally.
MozReview-Commit-ID: 523it7WEUad
Differential Revision: https://phabricator.services.mozilla.com/D6285
--HG--
extra : moz-landing-system : lando
This patch gets us building an updater binary that always embeds the dep certificates (instead of release or nightly), and builds a new tests package that includes it.
I had a lot of trouble getting the test package generated correctly due to the fact that things in "_tests" won't be included for test packages that aren't "common". My fix for that isn't ideal - I'm open to something better.
Differential Revision: https://phabricator.services.mozilla.com/D5900
--HG--
extra : moz-landing-system : lando
Changes the wpt manifest path to the topobjdir instead so it can be moved out of tree.
Other changes so that the manifest download and update, and |mach wpt| and |mach test <wpt-test>| work with the new path.
The manifest is also downloaded and updated when creating the tests-archive to ensure that it exists when we run tests on TC.
MozReview-Commit-ID: Fp6UsKJjhTU
Differential Revision: https://phabricator.services.mozilla.com/D5312
--HG--
extra : moz-landing-system : lando
Many Windows tools don't deal well with absolute paths for filenames,
oddly enough, so we want to pass relative paths into the source
directory where possible. These relative paths also mean that they look
like "normal" unix paths to WSL and therefore don't require any special
handling.
All but one of the current uses of DEFFILE use `SRCDIR + '/file.def'` to
get a srcdir-relative path anyway, and the other one wants an
objdir-relative path, so using Path makes everything clearer.
This makes it more straightforward to translate the paths for the WSL
build.
Add support for |with Files('a/**', 'b/**')| in mozbuild config files.
MozReview-Commit-ID: IoM4qfEhXXc
Differential Revision: https://phabricator.services.mozilla.com/D5315
--HG--
extra : moz-landing-system : lando
This will make it easier to add new test cases in the future. It'll
also make it easier to print debug, as only the output for each
specific test case will be printed on failure.
Differential Revision: https://phabricator.services.mozilla.com/D5862
--HG--
extra : moz-landing-system : lando
This introduces two new crates:
- jsrust, for standalone builds. This crate is compiled into a static library
libjsrust.a, which gets linked into the shared Spidermonkey library when it's
built, or into the static Spidermonkey library otherwise. This is just a
static library wrapping jsrust_shared below.
- jsrust_shared, for Gecko embedding. It just references other Rust
crates actively used in Spidermonkey. It is used to be embedded as part of
a new Rust dependency in Gecko (in gkrust).
--HG--
rename : js/src/wasm/cranelift/Cargo.toml => js/src/rust/Cargo.toml
extra : rebase_source : 84e440e3f669b73776653182cb7b006cc7febb10
extra : histedit_source : 3a67575ff6871b7dc3558c10a0251b73cedb090c
Now that we ship builds using clang on all platforms, pick it during
configure. It is still possible to opt-in to building other compilers by
setting CC/CXX (or even only CC) to the desired compiler.
Depends on D5829
Differential Revision: https://phabricator.services.mozilla.com/D5637
--HG--
extra : moz-landing-system : lando
While those builds are not fully supported yet, it's better to ensure
the coming changes to toolchain.configure won't break them.
Differential Revision: https://phabricator.services.mozilla.com/D5828
--HG--
extra : moz-landing-system : lando
This will be used when we migrate away from XBL and to a Custom Element
in the following changesets.
Differential Revision: https://phabricator.services.mozilla.com/D5598
--HG--
extra : moz-landing-system : lando
Currently, when building for Android x86-64 on Linux x86-64, we drop the
'--target' flag, which causes the build to fail. This patch adds a check
for OS mismatch, so we keep the '--target' flag in this situation.
Differential Revision: https://phabricator.services.mozilla.com/D4483
Imports the changes to the UpdateVerifyConfig class, and sets --override-certs for staging releases.
Differential Revision: https://phabricator.services.mozilla.com/D5705
--HG--
extra : moz-landing-system : lando
This patch makes the bootstrap code "ask" Gentoo's package manager for
human-readable information on how to get the latest JDK, then parses it and
locates the tarball URL on its own. The whole procedure is now fully automatic
(until the package output or Oracle web pages change).
Differential Revision: https://phabricator.services.mozilla.com/D5544
--HG--
extra : moz-landing-system : lando
This change is necessary for constructing relative paths between the
objdir and srcdir to succeed, and has been (I believe) the implicit
requirement of builds since time immemorial.
clang can handle MSVC-like codepaths generally, so we want to use those
when building with clang for Windows. So we switch _MSC_VER over to _WIN32
to pick up those codepaths when compiling for Windows with clang.
Additionally, we relax the ordering of sections for the same scenario.
Note that we do need to tell clang to use -fms-extensions with the MSVC code,
we do that in the mingw clang build job patch.
Differential Revision: https://phabricator.services.mozilla.com/D3526
--HG--
extra : moz-landing-system : lando
We don't actually ship XPT files anymore, but it's still useful for the
packager code to handle old Firefox versions. But for that, we don't
really need the complexity of "linking" XPT files in a single unit per
directory. We can just as well keep the XPT files intact, as long as we
retain individual `interfaces` manifest entries for each.
And since those entries used to be all merged into one, we now instead
group them all together in manifests (which also happens to make it
easier on unit test changes).
Differential Revision: https://phabricator.services.mozilla.com/D5221
--HG--
extra : moz-landing-system : lando
:mreid sent a review of my pull request and informed me that the
JSON schemas used in the data pipeline are all 2-space indented.
If we want to use this script to generate any future changes to
the build system schema, we should make sure it outputs data
correctly.
Differential Revision: https://phabricator.services.mozilla.com/D4149
--HG--
extra : moz-landing-system : lando
This patch rewrites `gather_telemetry` to collect data matching the new schema.
This includes all required fields and most of the optional fields. Some fields
are not currently recorded and followup bugs have been filed to track their
implementation.
Differential Revision: https://phabricator.services.mozilla.com/D4598
--HG--
extra : rebase_source : 0819e8519cfcf75ba0f554c110d1815b6f71af63
We already trim the display of output lists for GENERATED_FILES scripts
that produce many outputs, so we should do the same for rust build
scripts. This makes the terminal output of the build and the nodes from
'tup graph' more readable.
MozReview-Commit-ID: AftmrA4qJlr
Differential Revision: https://phabricator.services.mozilla.com/D4797
--HG--
extra : moz-landing-system : lando
Use the unused 'p' bit in the version code to denote 64-bit builds, so
we have different version codes for 64-bit builds on aarch64 and x86-64.
Differential Revision: https://phabricator.services.mozilla.com/D4260
--HG--
extra : moz-landing-system : lando
This warning spam happens particularly with WebRTC, but I can see it
happening for any third-party software whose use of WINVER and friends
conflicts with our own. Let's just change mozilla-config.h to avoid
defining these macros if they're already defined via the command line.
Use shutil.which in mozbuild for Python 3 instead of vendored third-party package, and enable mozversion tests that are fixed under Python 3 by this change.
Differential Revision: https://phabricator.services.mozilla.com/D4184
--HG--
extra : moz-landing-system : lando
After fixing the absolute path issue in codespell, I noticed that the stylish
formatter doesn't indent lint issues that don't have a column properly. This
was never noticed before since most other linters have a column attribute.
Depends on D4012
Differential Revision: https://phabricator.services.mozilla.com/D4013
--HG--
extra : moz-landing-system : lando
Currently there are 3 things that can impact the result of a lint run:
1. The list of lint issues found
2. The set of failures that happened during the setup phase
3. The set of failures that happened during the execution phase
All three of these things are stored as instance variables on the LintRoller
object, and then passed into a formatter when it comes time to print the
results. I'd like to add even more things that can impact the result, and it
became clear that the current scenario does not scale well.
This patch moves all data that could impact the end result of a lint run off of
the LintRoller object and onto a new 'result.ResultSummary' class. To avoid
confusion, this patch also renames the 'result.ResultContainer' class to
'result.Issue'.
With this new nomenclature:
result -> overall state of an entire lint run (can comprise multiple linters)
issue -> one specific lint infraction (at either 'warning' or 'error' level)
failure -> a non-recoverable error in the linter implementation itself
A "result" is comprised of 0 or more "issues" and 0 or more "failures".
Differential Revision: https://phabricator.services.mozilla.com/D3819
--HG--
extra : moz-landing-system : lando
As of this patch, any lint issue at the "warning" level will *only* be displayed
if --warnings is passed in. This gives us some flexibility to track issues that
are "recommended to fix" but that aren't required (aka don't cause a backout).
I think it would be ideal if the reviewbot ran with warnings enabled, and CI
ran without warnings. This way these issues will be surfaced to developers
(and hopefully get fixed prior to landing), but developers will always be able
to ignore them if the situation warrants it.
Since the last change converted everything to use errors, this patch should
be a no-op for now. However as we move forward (and potentially decide that
certain types of lint issues should result in a backout), this feature will
start seeing more and more use.
Depends on D3820
Differential Revision: https://phabricator.services.mozilla.com/D3821
--HG--
extra : moz-landing-system : lando
moz.configure looks for rustc/cargo on PATH and in ~/.cargo/bin.
Bootstrap only looks on PATH and not in ~/.cargo/bin, though it is smart
enough to complain if rustc/cargo can't be found on PATH and you have
them in ~/.cargo/bin. Bootstrap should look in both places by default,
and be content if it finds them wherever they are, so long as
moz.configure can find them.
The state directory is in $HOME by default, so should be fine to just create it
if we get --no-interactive I think.
Differential Revision: https://phabricator.services.mozilla.com/D3838
By default, when the launcher process is enabled, it does not wait for the
browser process to complete before terminating. mach run expects its child
process to keep running until the browser is terminated.
If we pass -wait-for-browser to the launcher process, the launcher will not
terminate until the browser process has finished, thus preserving the existing
semantics.
--HG--
extra : amend_source : cc848f955e14d7a97c1c506247fa2788261912f2
The telemetry gathering code is currently split in two places, so move it
all to a common place. Followup patches will rewrite most of this code.
--HG--
extra : rebase_source : afd14940f7175f7ca730a3b90ad770f143bcb6c7
cargo build is no longer called, so bootstrap will be faster.
All dependencies (including cbindgen) are usually up-to-date on BSDs
because there's no fallback to Mozilla binaries.
armasm64 doesn't accept the same options as its x86-ish counterparts,
and passing options it doesn't understand causes assembly to fail. So
let's just not pass any flags to the assembler for the moment.
If the wpt manifest format changes we bump the version number, causing
loading the old manifest to throw an error. We weren't correctly
handling this error when trying to update the manifest (by creating a
new empty manifest) so updates after the version number changed broke.
MozReview-Commit-ID: 4H1nMtRI9PZ
Differential Revision: https://phabricator.services.mozilla.com/D3771
--HG--
extra : moz-landing-system : lando
External systems such as the generic ingestion service will want to work with
the more standard json-schema format. This commit adds a script to convert the
voluptuous schema to json-schema format using the `luscious` Python module.
Since that module has not been updated recently, we install and use a fork with
some changes.
Since this is a single-purpose command that's unlikely to be used by many
people it's not implemented as a mach command, but simply a standalone script
that can be invoked via `mach python`.
Differential Revision: https://phabricator.services.mozilla.com/D2841
--HG--
extra : rebase_source : d35ed221d05d1d56b96604b931c22b700c10e476
This change adds a voluptuous schema for build system telemetry, replacing
the existing json schema file. Using voluptuous will make it easier to work
with the schema from Python code in the build system. A future commit will
use a Python module to provide a mach command to convert the voluptuous
schema to json schema format for consumption by other systems.
Differential Revision: https://phabricator.services.mozilla.com/D2840
--HG--
extra : rebase_source : 067995385334d1dbc123f2db4245ef4e69d076c3
This excludes directories, and returns true only if it's an executable file.
Differential Revision: https://phabricator.services.mozilla.com/D3366
--HG--
extra : moz-landing-system : lando
If you specify 'mk_add_options MOZ_PARALLEL_BUILD=X' in your mozconfig,
this variable ends up being passed into client.mk, which tells the make
backend to use that as the number of parallel jobs. However, there is no
equivalent in alternate backends aside from specifying '-jX' at the
commandline on each build invocation.
Rather than making a new mechanism to do this, we can check the
mozconfig for the MOZ_PARALLEL_BUILD flag and use that number for the
jobs parameter if no override was specified on the commandline.
MozReview-Commit-ID: 4YHG30N6tmi
Differential Revision: https://phabricator.services.mozilla.com/D3362
--HG--
extra : moz-landing-system : lando
And require it for taskcluster build already, because it doesn't harm and lets
me put all the yml changes in the same commit.
I gave up cross-compiling for OSX after a few tries and after realizing it
wasn't enough with cctools and such, but that I also needed the Mac SDK, for
which I don't have permission...
Differential Revision: https://phabricator.services.mozilla.com/D2664
--HG--
extra : moz-landing-system : lando
We're currently using NDK r15c, which is rather old, and happens to come
with a buggy gold linker. Let's use a more recent NDK, with a fixed
linker.
Unfortunately, we're currently at NDK API level 9, which the newer NDK
doesn't provide for x86 anymore. But that corresponds to Gingerbread
(2.3), which we've long stopped supporting. On the SDK side, we already
dropped support of versions before Jelly Bean, so we can do the same on
the NDK side. That corresponds to API level 16. So let's just use that
as a baseline.
Another change in the newer NDK is that the target-name changed from
i386-linux-android to i686-linux-android, so adjust for that in the
android x86 mozconfigs.
We're currently using NDK r15c, which is rather old, and happens to come
with a buggy gold linker. Let's use a more recent NDK, with a fixed
linker.
Unfortunately, we're currently at NDK API level 9, which the newer NDK
doesn't provide for x86 anymore. But that corresponds to Gingerbread
(2.3), which we've long stopped supporting. On the SDK side, we already
dropped support of versions before Jelly Bean, so we can do the same on
the NDK side. That corresponds to API level 16. So let's just use that
as a baseline.
Another change in the newer NDK is that the target-name changed from
i386-linux-android to i686-linux-android, so adjust for that in the
android x86 mozconfigs.
If a backend output file is deleted outside of the build system the non-zero
"created" count can make the current check erroneously think the list of
outputs has changed.
Differential Revision: https://phabricator.services.mozilla.com/D3129
--HG--
extra : moz-landing-system : lando
Similar to what we do for libstdc++ and glibc, we need to ensure that
we don't somehow end up with a dependency on a too new libgcc, which
can happen if it comes from gcc 7.0.
This patch adds JaCoCo as a dependency for the geckoview androidTest configurations, as well as
the `mach android archive-geckoview-coverage-artifacts` command, and the `--enable-java-coverage`
mozconfig flag.
MozReview-Commit-ID: 36jNAzK44g3
--HG--
extra : rebase_source : 9edc37913a3929ad045270c601c77791d122e363
With this artifact, we will have the total wall time for all files in the build
graph.
MozReview-Commit-ID: I6hLxavX3QY
--HG--
extra : rebase_source : dc5f827ea1fe83e9f04e9d9c1d19f3824e9a9a3f
Having an output to !/js/src/js causes problems for tup, because many
compilation invocations will attempt to find headers in !/js/src/js,
meaning tup thinks these invocations should depend on generating
!/js/src/js (the file).
MozReview-Commit-ID: CHP20RqH1OM
--HG--
extra : rebase_source : d2fd8ac808a2c9c548eb3962abae5002366d1ea3
Having an output to !/js/src/js causes problems for tup, because many
compilation invocations will attempt to find headers in !/js/src/js,
meaning tup thinks these invocations should depend on generating
!/js/src/js (the file).
MozReview-Commit-ID: Ky6kq82vlO8
--HG--
extra : rebase_source : 112715194f0ce53c5645a1d7f4a088b1a6120552
Summary:
The about:crashes page is being updated (bug 1463515). To facilitate these changes,
this patch changes the about:crashes page to use Fluent for localization instead of the old systems.
This also includes a script to migrate strings from the old .DTD and .properties files
to the new .ftl one.
Test Plan:
1. build Firefox with the changes
2. run Firefox
3. go to the about:crashes page
4. expect nothing to be different
This extension: https://github.com/rhelmer/webext-experiment-crashme can be used to
add local crash reports for verifying the different states of the about:crashes page.
Reviewers: flod, Pike, jchen, snorp
Reviewed By: flod, Pike, jchen, snorp
Subscribers: nalexander, reviewbot
Bug #: 1476034
Differential Revision: https://phabricator.services.mozilla.com/D2225
--HG--
extra : rebase_source : 0ca9516b4df78e735fd03907f2ea324bc72ca893
The --no-sync flag turns off the SQLite synchronous pragma, so we don't
force the disk to sync on every 'mach build'. Since mach always touches
a few files (eg: warnings.json, .purgecaches, etc), there is always a
minor amount of updates to the database, and syncing them every time
just forces an unnecessary delay.
The --debug-logging flag saves logs under .tup/log to show what files
changes or were deleted, as well as the corresponding build DAG. This
can be helpful when trying to investigate why something was rebuilt.
MozReview-Commit-ID: L0m7IsYlFUI
--HG--
extra : rebase_source : 0a26bf49778dfa13644f200be3e7061491b135c0
Currently the recursive make backend skips emitting COMPUTED_CFLAGS etc for
Rust-only directories as an optimization. However, we need to pass CFLAGS down
to Rust build scripts that compile C/C++ code, so having the COMPUTED_
variables will make it easier to pass the right set of flags.
Additionally, ensure that Rust programs wind up in linkables/host_linkables so
that they get computed flags emitted.
MozReview-Commit-ID: Ld41080dOZN
--HG--
extra : rebase_source : 671f1e4601d918eeb9de5ffa4c0cf53d821640de
A frequent question when mentoring new contributors is what "your
mozconfig file" is. By suggesting to create the file if it does
not exist, we can hopefully alleviate some new contributor frustration.
This change does unfortunately not take into account that the
mozconfig file can be named .mozconfig or even be in a designated
location defined by the MOZCONFIG environment variable, but it
seems reasonable to assume that developers who already know about
those alternatives will know which file to edit, and that what we
should optimise for during the bootstrapping process is to get new
contributors up and running quickly.
