They are equivalent, except for the fact that MACOSX_DEPLOYMENT_TARGET
may apply to more processes in a normal build. In practice, all the
processes that matter are covered through compiler flags.
On the opposite end, MACOSX_DEPLOYMENT_TARGET isn't necessary passed in
all cases (like clangd, mach static-analysis, etc.), while compiler
flags are.
Differential Revision: https://phabricator.services.mozilla.com/D122145
Sanitizers try to intercept __tls_get_addr to find DTLS blocks. In some
cases, they fail miserably with a crash. And in the cases they don't
crash, they don't seem to actually work properly anyways.
This affects both build time when running host programs compiled with
sanitizer flags (this only actually affects rust build
scripts/procedural macros on non-cross builds, not C/C++ host programs),
and execution time, e.g. when running tests.
Differential Revision: https://phabricator.services.mozilla.com/D120574
The `-/*` filter also removes the alias (`%%ALIAS`) and parent locale (`%%Parent`)
links, so we have to manually add them back when we're using `-/*`.
The ICU docs warn about this issue, but unfortunately that warning wasn't
present when we originally created our filter file, so this bug went unnoticed
for a long time.
Differential Revision: https://phabricator.services.mozilla.com/D116981
It's not practically make a difference (the flags we end up setting it
to don't do anything when linking all the wasm files into one).
Differential Revision: https://phabricator.services.mozilla.com/D116437
This last part to support "calendar" requires a bit more code.
`GetCalendarDisplayName()` performs the following steps:
1. Validate the input matches the `type` nonterminal, per
CanonicalCodeForDisplayNames, step 4.a.
2. Then convert it to lower case, because ICU expects lower case names and it
needs to be in lower case for `LanguageTag::replaceUnicodeExtensionType()`.
3. Replace calendar name aliases with the preferred name, e.g. use "ethioaa"
instead of "ethiopic-amete-alem".
4. Check the input isn't too large, just in case ICU somehow doesn't properly
handle large string inputs.
5. Retrieve the old-style value of the calendar name, for example "gregorian"
for the calendar name "gregory".
6. And finally call `uldn_keyValueDisplayName` to search for the localised name.
Our ICU data file doesn't yet include calendar names. To include calendar names
we need to enable them in "data_filter.json", rebuild the ICU data file, and
then reapply the update for tzdata 2021a. (Reapplying the tzdata changed
"intl/tzdata/GIT-INFO", but otherwise didn't change any time zone files.)
Adding the calendar names to the ICU data file results in a slight increase of
the overall data file size (79,184 bytes).
The existing "calendar.js" test file was renamed to "month-calendar.js", so
the test file for the new "calendar" type can be named "calendar.js".
Differential Revision: https://phabricator.services.mozilla.com/D116008
This adds a fallible factory method to create new NumberFormat instances. This
allows us to report initialization errors at time of initialization, rather than
when format is called, and remove internal checks in the implementation for
successful initialization.
The existing fluent code assumes that creating a NumberFormat instance always
succeeds. This patch updates that code to handle failures.
Differential Revision: https://phabricator.services.mozilla.com/D114593
This temporarily adds intl::NumberFormatterSkeleton to the public API so that
we can use it in PluralRules. This allows us to remove the SpiderMonkey
NumberFormatterSkeleton implementation when we convert SpiderMonkey to use
intl::NumberFormat.
We can remove intl::NumberFormatterSkeleton from the public API when we
implement intl::PluralRules in Bug 1704509.
Differential Revision: https://phabricator.services.mozilla.com/D111887
This adds a fallible factory method to create new NumberFormat instances. This
allows us to report initialization errors at time of initialization, rather than
when format is called, and remove internal checks in the implementation for
successful initialization.
The existing fluent code assumes that creating a NumberFormat instance always
succeeds. This patch updates that code to handle failures.
Differential Revision: https://phabricator.services.mozilla.com/D114593
This temporarily adds intl::NumberFormatterSkeleton to the public API so that
we can use it in PluralRules. This allows us to remove the SpiderMonkey
NumberFormatterSkeleton implementation when we convert SpiderMonkey to use
intl::NumberFormat.
We can remove intl::NumberFormatterSkeleton from the public API when we
implement intl::PluralRules in Bug 1704509.
Differential Revision: https://phabricator.services.mozilla.com/D111887
- We can and actually should remove the -fsanitize=% flags consistently
on all cargo targets, not only programs, but because of limitations in
cargo we can't really do that everywhere. We do need it for TSAN,
though.
- Because of the above, we actually don't need to single-out NATIVE_TSAN
builds anymore.
- The comment regarding the initial filtering of MOZ_CARGO_WRAP_LDFLAGS
somehow got separated from where it's happening, so we move it back to
where it belongs.
Differential Revision: https://phabricator.services.mozilla.com/D114579
Credit goes to dmajor.
Rust gets LLVM 12 - https://github.com/rust-lang/rust/pull/81451.
The PGO format changed between 11 and 12, and builds will have poor optimization if we mix versions in xLTO.
The above PR has landed on Rust nightly 1.52.
Until we move to 1.52 nightly we should have this landed.
In the meantime, for unblocking try builds, we can disable value-PGO in Rust.
Differential Revision: https://phabricator.services.mozilla.com/D113327
When "build targets" are used, the sccache server isn't
being used/integrated properly with make.
By moving rusttests to a separate build "application",
`client.mk` is invoked normally, and sccache is happy.
* Move rusttests to a new build "application".
* Move and denormalize rusttests mozconfigs. The new rusttests
mozconfigs should expand to be mostly identical to their old
versions, except with `MOZ_AUTOMATION_*=0` and
`--enable-application=tools/rusttests`.
They will differ a bit from their old variants because some
of them originally didn't source from their
respective `$app_mozconfigs/common`.
* Modify `baseconfig.mk` to change tiers if running rusttests.
* Adjust CI config to use new "rusttests" build application.
Differential Revision: https://phabricator.services.mozilla.com/D106294
While we could change qcms, encoding_rs and packed_simd to not emit
RUSTC_BOOTSTRAP on newer versions of rust, like we do for gkrust-shared,
it's not worth the effort (especially for those that are vendored).
Differential Revision: https://phabricator.services.mozilla.com/D105423
Bug 1690930 added sysroots that can be bootstrapped. With this change,
we allow --enable-bootstrap=install to pull the right sysroot for the
configured target, and --enable-bootstrap to update it if it was already
there.
Differential Revision: https://phabricator.services.mozilla.com/D104797
Back when the flag was added, the libffi source code didn't build for
ARM with the clang integrated assembler. That has been fixed with the
upgrade to libffi v3.3 in bug 1659906, so we can use the integrated
assembler now.
Differential Revision: https://phabricator.services.mozilla.com/D102261
* We add a config option for setting the number of content services;
* We add a config option to indicate whether content services should be isolated.
This one is just a `project_flag` since it doesn't really need the ability to
be overridden; it's something whose default we would want to flip when the
time comes;
* We set a dependency so that mobile/android/base/pre-export is executed;
* We add the `gen_from_jinja.py` script which is mostly just a dumb shim that
takes the input template and the config arguments, instantiates jinja,
generates the final output, and dumps it to the output fd;
* We add the requisite `moz.build` statements to generate the manifest overlay
and the service definitions;
* We update `build.gradle` so that Gradle knows to look for the generated files
when building the apk.
Differential Revision: https://phabricator.services.mozilla.com/D82577
* We add a config option for setting the number of content services;
* We add a config option to indicate whether content services should be isolated.
This one is just a `project_flag` since it doesn't really need the ability to
be overridden; it's something whose default we would want to flip when the
time comes;
* We set a dependency so that mobile/android/base/pre-export is executed;
* We add the `gen_from_jinja.py` script which is mostly just a dumb shim that
takes the input template and the config arguments, instantiates jinja,
generates the final output, and dumps it to the output fd;
* We add the requisite `moz.build` statements to generate the manifest overlay
and the service definitions;
* We update `build.gradle` so that Gradle knows to look for the generated files
when building the apk.
Differential Revision: https://phabricator.services.mozilla.com/D82577
With clang 12, win32 debug tests run out of stack in the 1000-deep layout crashtests.
Usually when this type of thing happens, it's because the compiler made a drastically heavier inlining decision along the recursive part of the stack, with a difference in the hundreds of bytes, and we pay that price of that space 1000 times. In those cases we normally try to snip out the heavy piece with a MOZ_NEVER_INLINE.
However, in this case, the difference between old and new compilers was only something like 16 bytes of the cyclical piece. This suggests that we're already up pretty close against the limit, and we're only a bad day away from failing tests even with existing clang. This only happens in debug builds since they have lots of logging helpers on the stack. Release builds are still well under the limit.
So I propose to increase the stack limit in debug. While we normally have to be pretty careful about address space on win32, it should be less of a concern with these non-shipping builds.
Differential Revision: https://phabricator.services.mozilla.com/D101196
Also rename `branchIfNegativeBigInt` to `branchIfBigIntIsNegative` to match
the other branch methods from part 2 and part 8.
Differential Revision: https://phabricator.services.mozilla.com/D98158
Also rename `branchIfNegativeBigInt` to `branchIfBigIntIsNegative` to match
the other branch methods from part 2 and part 8.
Differential Revision: https://phabricator.services.mozilla.com/D98158
On a very parallel debug build, I see a long time just waiting for
bindgen / style compilation / geckoservo.
Turns out that a bunch of this is just proc macros / build scripts.
Optimizing it saves between 10 and 17 seconds of my debug build. We
might want to consider running bindgen much like cbindgen rather than
rebuilding it all the time, which should help a lot more, but my guess
is that this should still help with the pretty hot custom derives that
the style crate runs.
This needs rust 1.41, so the requirement for tools/crashreporter needs
to be bumped as a consequence. To make things simpler, it was bumped
to 1.47 while we're at it.
Differential Revision: https://phabricator.services.mozilla.com/D98366
* Bumps the tsan toolchain to rust-nightly-2020-11-14 that has my patches to make -Zbuild-std work in vendored environments:
* https://github.com/rust-lang/cargo/pull/8834
* https://github.com/rust-lang/rust/pull/78790
* Passes -Zbuild-std to cargo when MOZ_TSAN is defined (mk_add_options --enable-thread-sanitizer)
* Removes generic Rust supressions and adds much more specific ones
* One presumed upstream false positive from tsan not understanding the code
* One actual upstream bug tsan found (yay!)
* One new real issue uncovered
* One issue that probably already existed intermittently but I happened to hit
Differential Revision: https://phabricator.services.mozilla.com/D97165
This makes --enable-thread-sanitizer turn on Rust tsan (-Zsanitizer=thread).
This requires changing SpiderMonkey tsan to use the tsan rust nightly.
In future changes, more Rust tsan integration will key off of MOZ_TSAN.
Differential Revision: https://phabricator.services.mozilla.com/D96453
This makes --enable-thread-sanitizer turn on Rust tsan (-Zsanitizer=thread).
This requires changing SpiderMonkey tsan to use the tsan rust nightly.
In future changes, more Rust tsan integration will key off of MOZ_TSAN.
Differential Revision: https://phabricator.services.mozilla.com/D96453
These files were omitted from the original patch because reformatting them required some manual intervention in order to avoid breaking unit tests. Generally the `noqa` lines were already there and just needed to be moved from one line to another (due to the reformatting by `black`), but sometimes `black` saw fit to move a bunch of stuff all onto one line, requiring me to introduce new `noqa` lines.
Besides the autoformat by `black` and some manual fixups, this patch contains no other changes.
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94052
Depends on D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
These files were omitted from the original patch because reformatting them required some manual intervention in order to avoid breaking unit tests. Generally the `noqa` lines were already there and just needed to be moved from one line to another (due to the reformatting by `black`), but sometimes `black` saw fit to move a bunch of stuff all onto one line, requiring me to introduce new `noqa` lines.
Besides the autoformat by `black` and some manual fixups, this patch contains no other changes.
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94052
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
This commit also allows `memfd_create` in the seccomp-bpf policy for all
process types.
`memfd_create` is an API added in Linux 3.17 (and adopted by FreeBSD
for the upcoming version 13) for creating anonymous shared memory
not connected to any filesystem. Supporting it means that sandboxed
child processes on Linux can create shared memory directly instead of
messaging a broker, which is unavoidably slower, and it should avoid
the problems we'd been seeing with overly small `/dev/shm` in container
environments (which were causing serious problems for using Firefox for
automated testing of frontend projects).
`memfd_create` also introduces the related operation of file seals:
irrevocably preventing types of modifications to a file. Unfortunately,
the most useful one, `F_SEAL_WRITE`, can't be relied on; see the large
comment in `SharedMemory:ReadOnlyCopy` for details. So we still use
the applicable seals as defense in depth, but read-only copies are
implemented on Linux by using procfs (and see the comments on the
`ReadOnlyCopy` function in `shared_memory_posix.cc` for the subtleties
there).
There's also a FreeBSD implementation, using `cap_rights_limit` for
read-only copies, if the build host is new enough to have the
`memfd_create` function.
The support code for Android, which doesn't support shm_open and can't
use the memfd backend because of issues with its SELinux policy (see bug
1670277), has been reorganized to reflect that we'll always use its own
API, ashmem, in that case.
Differential Revision: https://phabricator.services.mozilla.com/D90605
This commit also allows `memfd_create` in the seccomp-bpf policy for all
process types.
`memfd_create` is an API added in Linux 3.17 (and adopted by FreeBSD
for the upcoming version 13) for creating anonymous shared memory
not connected to any filesystem. Supporting it means that sandboxed
child processes on Linux can create shared memory directly instead of
messaging a broker, which is unavoidably slower, and it should avoid
the problems we'd been seeing with overly small `/dev/shm` in container
environments (which were causing serious problems for using Firefox for
automated testing of frontend projects).
`memfd_create` also introduces the related operation of file seals:
irrevocably preventing types of modifications to a file. Unfortunately,
the most useful one, `F_SEAL_WRITE`, can't be relied on; see the large
comment in `SharedMemory:ReadOnlyCopy` for details. So we still use
the applicable seals as defense in depth, but read-only copies are
implemented on Linux by using procfs (and see the comments on the
`ReadOnlyCopy` function in `shared_memory_posix.cc` for the subtleties
there).
There's also a FreeBSD implementation, using `cap_rights_limit` for
read-only copies, if the build host is new enough to have the
`memfd_create` function.
The support code for Android, which doesn't support shm_open and can't
use the memfd backend because of issues with its SELinux policy (see bug
1670277), has been reorganized to reflect that we'll always use its own
API, ashmem, in that case.
Differential Revision: https://phabricator.services.mozilla.com/D90605
These files were omitted from the original patch because reformatting them required some manual intervention in order to avoid breaking unit tests. Generally the `noqa` lines were already there and just needed to be moved from one line to another (due to the reformatting by `black`), but sometimes `black` saw fit to move a bunch of stuff all onto one line, requiring me to introduce new `noqa` lines.
Besides the autoformat by `black` and some manual fixups, this patch contains no other changes.
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94052
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
This avoids a sort of duplication of work between both, because the
linker will eventually LTO-compile everything, so we technically don't
really need the extra step of the rust compiler doing an intermediate
LTO on the static libraries it produces.
Differential Revision: https://phabricator.services.mozilla.com/D94224
Rustc >= 1.44 changed the file names of the static libraries it
produces with -windows-gnu targets, to match that of mingw clang/gcc.
Considering we still build on 1.43, the best fix would be to derive the
prefix/suffix based on the version of rust, but that actually turns into
a hard-to-solve problem because of configure tests for bindgen also
depending on the prefix/suffix value to be known.
On the other hand, we're soon due to an update to 1.47, so the simpler
solution is to just push mingw builds to require 1.44 (settling for the
smallest upgrade possible for now) and to remove the split between C and
rust library prefix/suffixes.
Differential Revision: https://phabricator.services.mozilla.com/D93726
The `clobber` targets are superseded by `mach clobber`, so we don't need them for any reason. The `clean` target is meant to get you to a post-`configure` state, but it doesn't really work, and if it's necessary for you to be in that state for some reason you can just clobber and re-`configure`, so it doesn't seem worth it to get it working again. Instead, delete all of them. Also delete `everything` which is not useful when `clobber` doesn't exist.
Differential Revision: https://phabricator.services.mozilla.com/D93514
This commit also allows `memfd_create` in the seccomp-bpf policy for all
process types.
`memfd_create` is an API added in Linux 3.17 (and adopted by FreeBSD
for the upcoming version 13) for creating anonymous shared memory
not connected to any filesystem. Supporting it means that sandboxed
child processes on Linux can create shared memory directly instead of
messaging a broker, which is unavoidably slower, and it should avoid
the problems we'd been seeing with overly small `/dev/shm` in container
environments (which were causing serious problems for using Firefox for
automated testing of frontend projects).
`memfd_create` also introduces the related operation of file seals:
irrevocably preventing types of modifications to a file. Unfortunately,
the most useful one, `F_SEAL_WRITE`, can't be relied on; see the large
comment in `SharedMemory:ReadOnlyCopy` for details. So we still use
the applicable seals as defense in depth, but read-only copies are
implemented on Linux by using procfs (and see the comments on the
`ReadOnlyCopy` function in `shared_memory_posix.cc` for the subtleties
there).
There's also a FreeBSD implementation, using `cap_rights_limit` for
read-only copies, if the build host is new enough to have the
`memfd_create` function.
Differential Revision: https://phabricator.services.mozilla.com/D90605
It is only really used in js/src/devtools/rootAnalysis/Makefile.in,
and even there, the way it is used seems wrong, so fix that at the
same time (binaries have been linked into $DIST/bin directly for a
while).
Differential Revision: https://phabricator.services.mozilla.com/D92721
Bug 1573566 moved libxul from toolkit/library to toolkit/library/build,
and that should be reflected in config/recurse.mk.
It's amazing the race condition hasn't caused problems earlier than now.
Differential Revision: https://phabricator.services.mozilla.com/D92708
This commit also allows `memfd_create` in the seccomp-bpf policy for all
process types.
`memfd_create` is an API added in Linux 3.17 (and adopted by FreeBSD
for the upcoming version 13) for creating anonymous shared memory
not connected to any filesystem. Supporting it means that sandboxed
child processes on Linux can create shared memory directly instead of
messaging a broker, which is unavoidably slower, and it should avoid
the problems we'd been seeing with overly small `/dev/shm` in container
environments (which were causing serious problems for using Firefox for
automated testing of frontend projects).
`memfd_create` also introduces the related operation of file seals:
irrevocably preventing types of modifications to a file. Unfortunately,
the most useful one, `F_SEAL_WRITE`, can't be relied on; see the large
comment in `SharedMemory:ReadOnlyCopy` for details. So we still use
the applicable seals as defense in depth, but read-only copies are
implemented on Linux by using procfs (and see the comments on the
`ReadOnlyCopy` function in `shared_memory_posix.cc` for the subtleties
there).
There's also a FreeBSD implementation, using `cap_rights_limit` for
read-only copies, if the build host is new enough to have the
`memfd_create` function.
Differential Revision: https://phabricator.services.mozilla.com/D90605
Bug 1645986 solved the problem for most generated files by moving their
rules to the top-level, but we're going to add rules that will end up in
subdirectories, so we have to solve the same problem again, in the
subdirectories.
Differential Revision: https://phabricator.services.mozilla.com/D88389
Somehow we have had it be a general define, but it has not mattered...at
least it won't matter until we upgrade to clang 11, at which point the
compiler will start to complain at us about the codepaths this define
activates.
Differential Revision: https://phabricator.services.mozilla.com/D88056
