This only solves the easy half of the problem outlined in the bug,
leaving the other half for later.
iostream::tellg() actually returns streampos, which is able to support
files larger than 4GiB with libstdc++, but converting to an int
obviously truncated that, as well as transformed values between 2GiB and
4GiB into invalid negative numbers.
iostream::seekg() also takes a streampos, so storing the streampos as-is
is enough to address the problem with tellg()/seekg() sequences.
The other half of the problem involves elfhack converting 64-bits ELF
headers to 32-bits headers internally, which requires deeper changes.
This change however, is enough to support files up to 4GiB, which is
already a good first step.
Differential Revision: https://phabricator.services.mozilla.com/D94252
This avoids a sort of duplication of work between both, because the
linker will eventually LTO-compile everything, so we technically don't
really need the extra step of the rust compiler doing an intermediate
LTO on the static libraries it produces.
Differential Revision: https://phabricator.services.mozilla.com/D94224
2020-10-13 Mike Hommey <mh@glandium.org>
* lib/freebl/freebl.gyp:
Bug 1670839 - Use ARM crypto extension for AES, SHA1 and SHA2 on
mac. r=kjacobs
AFAICT, the Makefile equivalent already does.
[58dc3216d518] [tip]
* lib/freebl/sha1-armv8.c:
Bug 1670839 - Only build sha1-armv8.c code when USE_HW_SHA1 is
defined. r=kjacobs
This matches what is done in sha256-armv8.c, and avoids
inconsistency with sha1-fast.c, which will define the same functions
in the case USE_HW_SHA1 is not defined.
[54be084e3ba8]
2020-10-16 J.C. Jones <jjones@mozilla.com>
* automation/abi-check/expected-report-libnss3.so.txt, automation/abi-
check/previous-nss-release, lib/nss/nss.h, lib/softoken/softkver.h,
lib/util/nssutil.h:
Set version numbers to 3.59 Beta
[d4b21706e432]
Differential Revision: https://phabricator.services.mozilla.com/D94070
Adds secureonly.example.com:443 to server-locations.txt - this host is only available on HTTPS.
Regenerates certs using `./mach python build/pgo/genpgocert.py` command.
Sets network.dns.native-is-localhost pref in test so we don't trigger assertion.
Differential Revision: https://phabricator.services.mozilla.com/D94005
Rustc >= 1.44 changed the file names of the static libraries it
produces with -windows-gnu targets, to match that of mingw clang/gcc.
Considering we still build on 1.43, the best fix would be to derive the
prefix/suffix based on the version of rust, but that actually turns into
a hard-to-solve problem because of configure tests for bindgen also
depending on the prefix/suffix value to be known.
On the other hand, we're soon due to an update to 1.47, so the simpler
solution is to just push mingw builds to require 1.44 (settling for the
smallest upgrade possible for now) and to remove the split between C and
rust library prefix/suffixes.
Differential Revision: https://phabricator.services.mozilla.com/D93726
The `clobber` targets are superseded by `mach clobber`, so we don't need them for any reason. The `clean` target is meant to get you to a post-`configure` state, but it doesn't really work, and if it's necessary for you to be in that state for some reason you can just clobber and re-`configure`, so it doesn't seem worth it to get it working again. Instead, delete all of them. Also delete `everything` which is not useful when `clobber` doesn't exist.
Differential Revision: https://phabricator.services.mozilla.com/D93514
This commit also allows `memfd_create` in the seccomp-bpf policy for all
process types.
`memfd_create` is an API added in Linux 3.17 (and adopted by FreeBSD
for the upcoming version 13) for creating anonymous shared memory
not connected to any filesystem. Supporting it means that sandboxed
child processes on Linux can create shared memory directly instead of
messaging a broker, which is unavoidably slower, and it should avoid
the problems we'd been seeing with overly small `/dev/shm` in container
environments (which were causing serious problems for using Firefox for
automated testing of frontend projects).
`memfd_create` also introduces the related operation of file seals:
irrevocably preventing types of modifications to a file. Unfortunately,
the most useful one, `F_SEAL_WRITE`, can't be relied on; see the large
comment in `SharedMemory:ReadOnlyCopy` for details. So we still use
the applicable seals as defense in depth, but read-only copies are
implemented on Linux by using procfs (and see the comments on the
`ReadOnlyCopy` function in `shared_memory_posix.cc` for the subtleties
there).
There's also a FreeBSD implementation, using `cap_rights_limit` for
read-only copies, if the build host is new enough to have the
`memfd_create` function.
Differential Revision: https://phabricator.services.mozilla.com/D90605
Now that we don't recurse into the js python configure, we don't need to
have a special treatment for the options that need to be passed down to
that subconfigure, which is what js_option was for.
Differential Revision: https://phabricator.services.mozilla.com/D92727
It was only meant to be used internally, when the top-level python
configure invoked the js python subconfigure. Now that this doesn't
happen, we can remove the option, and consolidate js_standalone and
building_js, which are now roughly synonyms.
Differential Revision: https://phabricator.services.mozilla.com/D92726
Instead, we now run js/src/old-configure from the top-level configure
after having run old-configure and extracted a few variables to inherit
from it.
Because we're now running from the top-level, $_objdir is always the
top-level objdir, which simplifies some things. The topobjdir in
js/src/config.status, however, needs to stay in js/src because of the
build frontend expecting it there.
When running js/src/old-configure, we used to need some special
treatment for a large number of variables for historic reasons, where
we'd take values from the assigned values before running old-configure
for some, or from AC_SUBSTs after running old-configure.
Now that both old-configure and js/src/old-configure get the same
assignments from old-configure.vars, we don't need anything special for
the former. And only a few remaining variables still need manual work
for the latter.
One notable difference, though, is that the new code doesn't try to
avoid running js subconfigure, which added complexity, and was actually
error-prone.
Differential Revision: https://phabricator.services.mozilla.com/D92725
Preparing to run both old-configure and js/src/old-configure from the
same python configure run, we refactor things such that shared parts are
separate.
Differential Revision: https://phabricator.services.mozilla.com/D92722
It is only really used in js/src/devtools/rootAnalysis/Makefile.in,
and even there, the way it is used seems wrong, so fix that at the
same time (binaries have been linked into $DIST/bin directly for a
while).
Differential Revision: https://phabricator.services.mozilla.com/D92721
In order to be able to run both old-configure and js/src/old-configure
from the same python configure run, we need to stop setting the items
set by old-configure into the global sandbox config, and instead store
them to be later handled by configure.py.
Differential Revision: https://phabricator.services.mozilla.com/D92718
This was the last flag that the PrintOptions bitfield was tracking.
So, this patch is effectively converting that bitfield (and its alias
"PrintOptionsBits") into a new, simpler boolean field named
"isPrintSelectionRBEnabled".
Differential Revision: https://phabricator.services.mozilla.com/D92542
This commit also allows `memfd_create` in the seccomp-bpf policy for all
process types.
`memfd_create` is an API added in Linux 3.17 (and adopted by FreeBSD
for the upcoming version 13) for creating anonymous shared memory
not connected to any filesystem. Supporting it means that sandboxed
child processes on Linux can create shared memory directly instead of
messaging a broker, which is unavoidably slower, and it should avoid
the problems we'd been seeing with overly small `/dev/shm` in container
environments (which were causing serious problems for using Firefox for
automated testing of frontend projects).
`memfd_create` also introduces the related operation of file seals:
irrevocably preventing types of modifications to a file. Unfortunately,
the most useful one, `F_SEAL_WRITE`, can't be relied on; see the large
comment in `SharedMemory:ReadOnlyCopy` for details. So we still use
the applicable seals as defense in depth, but read-only copies are
implemented on Linux by using procfs (and see the comments on the
`ReadOnlyCopy` function in `shared_memory_posix.cc` for the subtleties
there).
There's also a FreeBSD implementation, using `cap_rights_limit` for
read-only copies, if the build host is new enough to have the
`memfd_create` function.
Differential Revision: https://phabricator.services.mozilla.com/D90605
Before, this would be written to `sitecustomize.py` irrespective of the value of `populate_local_paths`. This doesn't make sense -- since the local paths aren't included in the `virtualenv`'s `PYTHONPATH` when Python starts up, it doesn't know how to `import mach_bootstrap`. Since on `mach` startup the import hook will be loaded anyway, and the `virtualenv`s in `~/.mozbuild` (i.e. the only `virtualenv`s for which we don't `populate_local_paths`) are just used to run `mach`, this is fine and won't regress anything.
Also, since the `import` hook is only necessary for Python 2, add a couple conditional checks to get rid of the added overhead when we're running with Python 3.
This was never noticed because importing `sitecustomize` is allowed to throw an `ImportError`, which failure is ignored silently. This may be fixed in the latest version of `virtualenv`.
Differential Revision: https://phabricator.services.mozilla.com/D92290
The patch no longer applies cleanly because of nearby changes in 4b8cb665a1.
Instead of forking a clang-12 version of this patch, we can get away with carrying a little less context in our patch file.
Differential Revision: https://phabricator.services.mozilla.com/D91625
1. Provide a new backend dedicated for C++ static-code_analysis
2. Build a list with directories, from non-unified-compat that have been fixed and
permit compiling of C++ files outside of the unified environment. With this list
we eliminate the unified sources and instead use the original source for command
attribute from compile_commands.json.
In this way if a regression appears clang-tidy will report it since it no longer uses
the unified environment for files that are compatible to be compiled standalone.
3. Remove the coverity functionality that was reading and using non-unified build files
since in practive it proved to be sub-optimal.
Differential Revision: https://phabricator.services.mozilla.com/D91011
Minor fix to remoteautomation.py: Increment stdoutlen before any type conversions,
to ensure that it accurately reflects the byte offset in the file.
With this last change, 'mach mochitest' appears to run correctly on Android with Python 3:
switch it over to Python 3.
Differential Revision: https://phabricator.services.mozilla.com/D91586
2020-09-18 Kevin Jacobs <kjacobs@mozilla.com>
* automation/abi-check/previous-nss-release, lib/nss/nss.h,
lib/softoken/softkver.h, lib/util/nssutil.h:
Set version numbers to 3.58 Beta
[c28e20f61e5d] [tip]
* .hgtags:
Added tag NSS_3_57_RTM for changeset cf7e3e8abd77
[a963849538ca] <NSS_3_57_BRANCH>
* lib/nss/nss.h, lib/softoken/softkver.h, lib/util/nssutil.h:
Set version numbers to 3.57 final
[cf7e3e8abd77] [NSS_3_57_RTM] <NSS_3_57_BRANCH>
Differential Revision: https://phabricator.services.mozilla.com/D91070
Since `install-moz-phab` is meant to simplify the moz-phab setup flow,
automatically prompting for Phabricator credentials removes an otherwise
manual step.
Detecting the "console_script" location of a package in a
cross-platform, virtualenv-supporting and "--user"-supporting way is
tough, and the most consistent solution seems to be to list the package
contents of moz-phab and look for the one that seems to be the entry
point.
Differential Revision: https://phabricator.services.mozilla.com/D90642
It was added in bug 1320656 because back then we were building as C++14
with warnings about future incompatibilities with C++17. Since then,
we've switched to C++17, which means we had to fix those
incompatibilities, and thus they don't exist anymore. A local build with
-Werror=noexcept-type finishes just fine.
This removes the only difference between top-level and js warning flags.
Differential Revision: https://phabricator.services.mozilla.com/D90521
I think they're remnants from the past that we don't really need anymore.
And they're making things more complicated for some pending work of mine.
Differential Revision: https://phabricator.services.mozilla.com/D89687
This extracts an install-meson.sh helper script to install meson in both
the wrench-deps task for Firefox CI and the taskcluster.yml in WebRender CI.
Differential Revision: https://phabricator.services.mozilla.com/D90441
Allows mach commands to define their own glean metrics with the `metrics_path` @CommandProvider parameter.
When `metrics_path` is defined:
* A `metrics` kwarg is provided to the decorated class. This `metrics` handle is a Glean instance, so Glean documentation should be consulted for usage information.
* When `mach doc telemetry` is run, metrics docs will be generated from all the registered metrics files.
Note: there was some consideration between making `metrics_path` a @CommandProvider or @Command parameter.
In the end, @CommandProvider seemed like a better fit because:
* Metrics seem to be more associated with the entire class than a specific command/method. This is because a class represents a "domain", and that domain may have different commands that have overlapping metrics. Accordingly, all the metrics should be defined once as available to the entire class.
* Currently, @Command methods only take parameters that map one-to-one with CLI arguments. It could seem inconsistent to have one exception: the metrics handle
Differential Revision: https://phabricator.services.mozilla.com/D85953
resolver_64.cc should have been on the list all along, because the InternalThunk
constructor runs before ASan init. It was probably just accident (maybe inlining?)
that we got away with not including it in the past.
Differential Revision: https://phabricator.services.mozilla.com/D89669
This was originally meant to allow `virtualenv`s to use packages from a parent Python environment without having to re-install them. This turned out to not pan out as we would have liked, so we're going another way to solve the same problem. Bug 1660351 walked back a bunch of this logic; this patch deletes the rest of it.
Differential Revision: https://phabricator.services.mozilla.com/D89492
