This was the last flag that the PrintOptions bitfield was tracking.
So, this patch is effectively converting that bitfield (and its alias
"PrintOptionsBits") into a new, simpler boolean field named
"isPrintSelectionRBEnabled".
Differential Revision: https://phabricator.services.mozilla.com/D92542
This commit also allows `memfd_create` in the seccomp-bpf policy for all
process types.
`memfd_create` is an API added in Linux 3.17 (and adopted by FreeBSD
for the upcoming version 13) for creating anonymous shared memory
not connected to any filesystem. Supporting it means that sandboxed
child processes on Linux can create shared memory directly instead of
messaging a broker, which is unavoidably slower, and it should avoid
the problems we'd been seeing with overly small `/dev/shm` in container
environments (which were causing serious problems for using Firefox for
automated testing of frontend projects).
`memfd_create` also introduces the related operation of file seals:
irrevocably preventing types of modifications to a file. Unfortunately,
the most useful one, `F_SEAL_WRITE`, can't be relied on; see the large
comment in `SharedMemory:ReadOnlyCopy` for details. So we still use
the applicable seals as defense in depth, but read-only copies are
implemented on Linux by using procfs (and see the comments on the
`ReadOnlyCopy` function in `shared_memory_posix.cc` for the subtleties
there).
There's also a FreeBSD implementation, using `cap_rights_limit` for
read-only copies, if the build host is new enough to have the
`memfd_create` function.
Differential Revision: https://phabricator.services.mozilla.com/D90605
Before, this would be written to `sitecustomize.py` irrespective of the value of `populate_local_paths`. This doesn't make sense -- since the local paths aren't included in the `virtualenv`'s `PYTHONPATH` when Python starts up, it doesn't know how to `import mach_bootstrap`. Since on `mach` startup the import hook will be loaded anyway, and the `virtualenv`s in `~/.mozbuild` (i.e. the only `virtualenv`s for which we don't `populate_local_paths`) are just used to run `mach`, this is fine and won't regress anything.
Also, since the `import` hook is only necessary for Python 2, add a couple conditional checks to get rid of the added overhead when we're running with Python 3.
This was never noticed because importing `sitecustomize` is allowed to throw an `ImportError`, which failure is ignored silently. This may be fixed in the latest version of `virtualenv`.
Differential Revision: https://phabricator.services.mozilla.com/D92290
The patch no longer applies cleanly because of nearby changes in 4b8cb665a1.
Instead of forking a clang-12 version of this patch, we can get away with carrying a little less context in our patch file.
Differential Revision: https://phabricator.services.mozilla.com/D91625
1. Provide a new backend dedicated for C++ static-code_analysis
2. Build a list with directories, from non-unified-compat that have been fixed and
permit compiling of C++ files outside of the unified environment. With this list
we eliminate the unified sources and instead use the original source for command
attribute from compile_commands.json.
In this way if a regression appears clang-tidy will report it since it no longer uses
the unified environment for files that are compatible to be compiled standalone.
3. Remove the coverity functionality that was reading and using non-unified build files
since in practive it proved to be sub-optimal.
Differential Revision: https://phabricator.services.mozilla.com/D91011
Minor fix to remoteautomation.py: Increment stdoutlen before any type conversions,
to ensure that it accurately reflects the byte offset in the file.
With this last change, 'mach mochitest' appears to run correctly on Android with Python 3:
switch it over to Python 3.
Differential Revision: https://phabricator.services.mozilla.com/D91586
2020-09-18 Kevin Jacobs <kjacobs@mozilla.com>
* automation/abi-check/previous-nss-release, lib/nss/nss.h,
lib/softoken/softkver.h, lib/util/nssutil.h:
Set version numbers to 3.58 Beta
[c28e20f61e5d] [tip]
* .hgtags:
Added tag NSS_3_57_RTM for changeset cf7e3e8abd77
[a963849538ca] <NSS_3_57_BRANCH>
* lib/nss/nss.h, lib/softoken/softkver.h, lib/util/nssutil.h:
Set version numbers to 3.57 final
[cf7e3e8abd77] [NSS_3_57_RTM] <NSS_3_57_BRANCH>
Differential Revision: https://phabricator.services.mozilla.com/D91070
Since `install-moz-phab` is meant to simplify the moz-phab setup flow,
automatically prompting for Phabricator credentials removes an otherwise
manual step.
Detecting the "console_script" location of a package in a
cross-platform, virtualenv-supporting and "--user"-supporting way is
tough, and the most consistent solution seems to be to list the package
contents of moz-phab and look for the one that seems to be the entry
point.
Differential Revision: https://phabricator.services.mozilla.com/D90642
It was added in bug 1320656 because back then we were building as C++14
with warnings about future incompatibilities with C++17. Since then,
we've switched to C++17, which means we had to fix those
incompatibilities, and thus they don't exist anymore. A local build with
-Werror=noexcept-type finishes just fine.
This removes the only difference between top-level and js warning flags.
Differential Revision: https://phabricator.services.mozilla.com/D90521
I think they're remnants from the past that we don't really need anymore.
And they're making things more complicated for some pending work of mine.
Differential Revision: https://phabricator.services.mozilla.com/D89687
This extracts an install-meson.sh helper script to install meson in both
the wrench-deps task for Firefox CI and the taskcluster.yml in WebRender CI.
Differential Revision: https://phabricator.services.mozilla.com/D90441
Allows mach commands to define their own glean metrics with the `metrics_path` @CommandProvider parameter.
When `metrics_path` is defined:
* A `metrics` kwarg is provided to the decorated class. This `metrics` handle is a Glean instance, so Glean documentation should be consulted for usage information.
* When `mach doc telemetry` is run, metrics docs will be generated from all the registered metrics files.
Note: there was some consideration between making `metrics_path` a @CommandProvider or @Command parameter.
In the end, @CommandProvider seemed like a better fit because:
* Metrics seem to be more associated with the entire class than a specific command/method. This is because a class represents a "domain", and that domain may have different commands that have overlapping metrics. Accordingly, all the metrics should be defined once as available to the entire class.
* Currently, @Command methods only take parameters that map one-to-one with CLI arguments. It could seem inconsistent to have one exception: the metrics handle
Differential Revision: https://phabricator.services.mozilla.com/D85953
resolver_64.cc should have been on the list all along, because the InternalThunk
constructor runs before ASan init. It was probably just accident (maybe inlining?)
that we got away with not including it in the past.
Differential Revision: https://phabricator.services.mozilla.com/D89669
This was originally meant to allow `virtualenv`s to use packages from a parent Python environment without having to re-install them. This turned out to not pan out as we would have liked, so we're going another way to solve the same problem. Bug 1660351 walked back a bunch of this logic; this patch deletes the rest of it.
Differential Revision: https://phabricator.services.mozilla.com/D89492
I think they're remnants from the past that we don't really need anymore.
And they're making things more complicated for some pending work of mine.
Differential Revision: https://phabricator.services.mozilla.com/D89687
resolver_64.cc should have been on the list all along, because the InternalThunk
constructor runs before ASan init. It was probably just accident (maybe inlining?)
that we got away with not including it in the past.
Differential Revision: https://phabricator.services.mozilla.com/D89669
In addition to the existing build telemetry, also gather the stats and
report with Glean. This new telemetry is reported in tandem with the existing
telemetry to allow testing and confidence before a full roll-out.
Additionally, Glean isn't compatible with Python 2, so the new telemetry only runs
on Python 3 mach commands.
Differential Revision: https://phabricator.services.mozilla.com/D83572
The files are copied verbatim from upstream autoconf 2.13 (but only the
files we need) and old.configure is adapted to use the vendored version.
Differential Revision: https://phabricator.services.mozilla.com/D89554
This was originally meant to allow `virtualenv`s to use packages from a parent Python environment without having to re-install them. This turned out to not pan out as we would have liked, so we're going another way to solve the same problem. Bug 1660351 walked back a bunch of this logic; this patch deletes the rest of it.
Differential Revision: https://phabricator.services.mozilla.com/D89492
This commit does the following.
- Renames `slashslash` as `dumbComments`. As a result, it now comes before
`emptyLines` in alphabetical ordering, which means that if you apply both
`dumbComments` and `emptyLines`, lines that contain only comments will be
fully removed.
(I contemplated changing the filter ordering to match the order specified,
rather than using alphabetical ordering, but that was more invasive and not
obviously better.)
- Changes `dumbComments` so it only applies if the comment is at the start of
the line (with optional leading whitespace). This is so it can be used with
prefs files, which contain lines like `pref("foo", "https://mozilla.org");`
where the `//` must not be treated as a comment.
Note that `slashslash` wasn't being used anywhere.
Depends on D88240
Differential Revision: https://phabricator.services.mozilla.com/D88242
It's not used, probably because it's pretty strange and hard to imagine using
safely. (Stripping leading and trailing space could be useful, but collapsing
sequences of spaces? Hmm.)
Differential Revision: https://phabricator.services.mozilla.com/D88240
When configuring without system NSPR the configuration variable
PKGCONF_REQUIRES_PRIVATE isn't ever set, leading to a .pc file that still
contains the @PKGCONF_REQUIRES_PRIVATE@ stub.
Ensure that we always define PKGCONF_REQUIRES_PRIVATE, by setting it to an
empty string in case no system-nsrp is enabled.
In this way, the pkg-config file stub will be always replaced.
Differential Revision: https://phabricator.services.mozilla.com/D88179
widl output for dlldata has #defines immediately followed by #includes,
so looking for empty lines when we observer a #define doesn't work. We
instead look for #defines.
Differential Revision: https://phabricator.services.mozilla.com/D88938
The work in bug 1620133 ended up moving the execution of some of the
midl commands to the top-level, which was alleviated by adding the -out
midl command line option to make midl place its output in the expected
directory. Unfortunately, that option is not handled by widl, which is
the alternative command used in mingw builds.
So instead of using -out, we set the cwd for the midl command, and
readjust the command line arguments to be relative to that.
Differential Revision: https://phabricator.services.mozilla.com/D88937
For not-well-understood reasons, ld's `--gc-sections` discards a large number of the PGO bookkeeping structures that enable us to keep track of function counters, and the effect gets worse in object files generated by clang-10.
As much as I'd like to understand this better, the investigations take way too much time. As a path of least resistance, we can disable `--gc-sections` for the instrumentation phase of PGO builds. It won't harm anything since users never see those builds, and it will improve the performance of the optimized phase greatly.
Differential Revision: https://phabricator.services.mozilla.com/D78112
We don't anticipate end users will actually care to do this, but it's useful especially for unit tests. For example, after bug 1659539, Python `configure` tests will run in a new, non-`init_py3` `virtualenv`, and we'll want to target that `virtualenv` for `configure` rather than having it create a new `virtualenv` for no reason.
Differential Revision: https://phabricator.services.mozilla.com/D88661
2020-08-21 Kevin Jacobs <kjacobs@mozilla.com>
* automation/abi-check/previous-nss-release, lib/nss/nss.h,
lib/softoken/softkver.h, lib/util/nssutil.h:
Set version numbers to 3.57 Beta
[783f49ae6126]
2020-08-24 Kevin Jacobs <kjacobs@mozilla.com>
* gtests/ssl_gtest/ssl_auth_unittest.cc, lib/ssl/dtls13con.c,
lib/ssl/dtlscon.c, lib/ssl/ssl3con.c, lib/ssl/sslimpl.h,
lib/ssl/sslnonce.c:
Bug 1653641 - Cleanup inaccurate DTLS comments, code review fixes.
r=mt
[0e1b5c711cb9]
2020-08-24 Robert Relyea <rrelyea@redhat.com>
* lib/freebl/fipsfreebl.c, lib/softoken/fipstest.c,
lib/softoken/kbkdf.c, lib/softoken/lowpbe.c, lib/softoken/lowpbe.h,
lib/softoken/pkcs11c.c, lib/softoken/pkcs11i.h,
lib/softoken/sftkhmac.c, lib/softoken/sftkike.c:
Bug 1660304 New FIPS IG requires self-tests for approved kdfs.
r=ueno comments=kjacobs
FIPS guidance now requires self-tests for our kdfs. It also requires
self-tests for cmac which we didn't have in the cmac patch.
Currently only one test per kdf is necessary. Specifially for
SP-800-108, only one of the three flavors are needed (counter,
feedback, or pipeline). This patch includes more complete testing
but it has been turned off the currently extraneous tests under the
assumption that NIST guidance may require them in the future. HKDF
is currently not included in FIPS, but is on track to be included,
so hkdf have been included in this patch.
Because the test vectors are const strings, the patch pushes some
const definitions that were missing in existing private interfaces.
There are three flavors of self-tests: Function implemented in
freebl are added to the freebl/fipsfreebl.c Functions implemented in
pkcs11c.c have selftests completely implemented in
softoken/fipstest.c Functions implemented in their own .c file have
their selftest function implemented in that .c file and called by
fipstests.c These are consistant with the previous choices for
selftests.
Some private interfaces that took in keys from pkcs #11 structures
or outputted keys to pkcs #11 structures were modified to optionally
take keys in by bytes and output keys as bytes so the self-tests can
work in just bytes.
[5dca54fe61c2]
2020-08-25 Daiki Ueno <dueno@redhat.com>
* lib/softoken/manifest.mn:
Bug 1659252, disable building libnssdbm3.so if NSS_DISABLE_DBM=1,
r=rrelyea
Reviewers: rrelyea
Reviewed By: rrelyea
Bug #: 1659252
[4d55d36ca6ef]
2020-08-24 Kevin Jacobs <kjacobs@mozilla.com>
* lib/pk11wrap/pk11cxt.c, lib/softoken/pkcs11c.c, lib/softoken/sdb.c,
lib/softoken/sftkpwd.c:
Bug 1651834 - Fix various static analyzer warnings. r=rrelyea
[ab04fd73fd6d]
2020-08-28 Mike Hommey <mh@glandium.org>
* lib/freebl/blapii.h:
Bug 1661810 - Define pre_align/post_align based on the compiler.
r=jcj
Things worked fine before we upgraded to clang 11 presumably because
the stack was always 16-bytes aligned in the first place, or
something akin to that, and the lack of pre_align/post_align doing
anything didn't matter. The runtime misalignment of the stack may
well be a clang > 9 bug, but keeping pre_align/post_align tied to
the x86/x64 is a footgun anyways.
[c100e11991f6] [tip]
Differential Revision: https://phabricator.services.mozilla.com/D88876
