We are currently relying on two different ways to insert side-by-side
manifests in binaries on Windows: through resource files, or through
the use of MT. The latter is not supported on mingw builds, which is
not great.
Link.exe has options to add a manifest at link time without relying on
either method above, but that's not supported on mingw either.
So the best we can do is to move everything to using resource files.
This also avoids using MT, which, on cross builds, requires using wine.
Ideally, the manifests would be declared in moz.build, but that
complicates things for cases like TestDllInterceptor, where there are
multiple binaries in the same directory, but only one of them needs the
manifest. This keeps the status quo of getting the manifest
automatically from the source directory.
Differential Revision: https://phabricator.services.mozilla.com/D85382
We are currently relying on two different ways to insert side-by-side
manifests in binaries on Windows: through resource files, or through
the use of MT. The latter is not supported on mingw builds, which is
not great.
Link.exe has options to add a manifest at link time without relying on
either method above, but that's not supported on mingw either.
So the best we can do is to move everything to using resource files.
This also avoids using MT, which, on cross builds, requires using wine.
Ideally, the manifests would be declared in moz.build, but that
complicates things for cases like TestDllInterceptor, where there are
multiple binaries in the same directory, but only one of them needs the
manifest. This keeps the status quo of getting the manifest
automatically from the source directory.
Differential Revision: https://phabricator.services.mozilla.com/D85382
clang-11 has a new warning that fires when you pass a 256-bit vector type as a parameter or return value, and your compilation doesn't enable AVX. The aim is to warn you that the ABI is different depending on whether you enable AVX, which could become a problem if AVX and non-AVX files communicate through such parameters.
While the intent is good, it's not really worth 1800 lines of log spam for us. We have only a tiny number of AVX compilations in media codecs, and the only things they interchange with other code are buffer pointers.
Differential Revision: https://phabricator.services.mozilla.com/D86107
Add a modular approach for the integration of `static-analysis` module in order
to be able to share components of it with other modules, like the integration of
`clangd` in `vscode` where we need to have access to the configuration of `clang-tidy`
in order to have `in-ide` `static-analysis` messages.
In this initial step we make a separate module for the clang-tidy configuration.
Differential Revision: https://phabricator.services.mozilla.com/D85979
2020-07-27 Jan-Marek Glogowski <glogow@fbihome.de>
* lib/freebl/Makefile:
Bug 1652032 Disable all freebl assembler code for MSVC arm64
r=rrelyea,bbeurdouche
There are two places, where NSS tries to compile either x86_64 MSVC
assembler or GCC aarch64 code, which will fail the build. And also
drop the non-MSVC arch build flags for them.
AFAI could identify, there isn't any armasm64 compatible asm code in
the whole NSS library, so I don't even adapt AS for the build. The
cross-build finishes this way.
[d98bbb6168f4]
2020-07-24 Benjamin Beurdouche <bbeurdouche@mozilla.com>
* cmd/bltest/blapitest.c, coreconf/config.gypi, coreconf/config.mk,
lib/freebl/alg2268.c, lib/freebl/deprecated/alg2268.c,
lib/freebl/freebl_base.gypi, lib/freebl/ldvector.c,
lib/freebl/loader.c, lib/freebl/loader.h, lib/freebl/manifest.mn,
lib/softoken/lowpbe.c, lib/softoken/pkcs11c.c:
Bug 1652729 - Add build flag to disable RC2 and relocate to
lib/freebl/deprecated. r=kjacobs
[e6c6f1d2d544]
2020-07-27 Robert Relyea <rrelyea@redhat.com>
* gtests/softoken_gtest/manifest.mn,
gtests/softoken_gtest/softoken_dh_vectors.h,
gtests/softoken_gtest/softoken_gtest.cc,
gtests/softoken_gtest/softoken_gtest.gyp, lib/freebl/blapi.h,
lib/freebl/dh.c, lib/freebl/ldvector.c, lib/freebl/loader.c,
lib/freebl/loader.h, lib/softoken/manifest.mn,
lib/softoken/pkcs11.c, lib/softoken/pkcs11c.c,
lib/softoken/pkcs11i.h, lib/softoken/pkcs11u.c,
lib/softoken/sftkdhverify.c, lib/softoken/softoken.gyp:
Bug 1648822 Add stricter validation of DH keys when in FIPS mode.
Update: FIPS now also requires us to do y^q mod p testing on key
generation (always). We now do that in FIPS mode only, but in all
modes we do full DH verification for DH and ECDH. Because of this,
the path has now separated out the prime checks, which are now only
done for the DH operation if we aren't using a known prime and the
subprime value has been provided. I've also learned we can accept
keys that we do full validation on in FIPS mode, so I've added that
to this patch, though we still can't generate those kinds of keys
without adding the subprime at keygen time.
The new FIPS standard is dh operations must use approved primes.
Approved primes are those selected in the tls and ike RFCs.
Currently tls and ike have modes with checks whether the primes are
approved, but the check may not always happen. The safest thing to
do in FIPS mode is only allow those primes. In addition, FIPS
requires 1< y < p-1 (or technically 2<=y<=p-2, since y is an integer
those two tests are identical).
While making changes I realized we would want a mode where we can do
more strict checks on the prime while not requiring that the prime
be an approved prime. We already allow for strict checking if q is
supplied with the private key, but there were a couple of issues
with that check:
1. there was no way of actually setting q in the current NSS
pk11wrap interfaces. 2. If the prime was a safe prime, but g was an
actual generator, then we would fail the y^q mod p = 1 tests for 50%
of the keys, even though those keys are safe. 3. We weren't checking
primality of p and q.
So the old code:
if (q) { check y^q mod p = 1 if not fail }
check 1 <y < p-1 (done in DH_Derive).
New code:
if (! p is approved prime) { if (FIPS) fail; if (q) { y_test = y if
(p,q-> p is a safe prime) { y_test = 1 } check prime is prime Fail
if not check subprime is subprime fail if not y_test^q mod p = 1 } }
check 1 < y < p-1 (done in DH_Derive)
This means:
Existing code non-fips without setting the subprime continues to run
as before. Non-fips code which sets the subprime now runs slower,
but p and q are checked if p or q where not prime, the derive fails
(which it should). In FIPS mode only approved primes will succeed
now. Non-fips code can now set the subprime to q=(p-1)/2 if it
doesn't have an explicit q value (like in tls). If the derive
succeeds, we know that p is a safe prime. If p is approved, the
checks are skipped because we already know that p is a safe prime.
Code can optionally do a test derive on a new p and remember it's
safe so that we know longer need to check ever call (though if q is
not (p-1)/2, you will need to continue to do the checks each call
because y could still be a small subgroup).
This patch:
gtests/softoken_gtest
1. Added New dh tests to softoken_gtests. The tests were added to
softoken_gtests because we need to test both non-FIPS and FIPS mode.
Test vectors include a category, so the same test vectors can be
used in FIPS and non-FIPS even though each class may have different
results. Most of the test vectors where created either by dhparams
command in openssl, dsaparams in openssl, and the nss makepqg
command. Each vector includes a label, prime, base, optional
subprime, optional public key, test type, and key class (basically
size). 2. If public key is not supplied, we use a generated public
key. 3. If subPrime is supplied to wet it on the private key after
generation.
lib/freebl/dh.c
add primality tests to KEA_VerifyKey().
lib/softokn/
1. Allow CKA_SUBPRIME to be set after key generation or import.
This affects how we test for it's existance, since it is now always
there on the key, we check it's length to make sure it's non-zero.
2. We implement the psuedocode above as real code. 3. We create two
new functions: sftl_VerifyDH_Prime which return SECSuccess if Prime
is an approved prime. sftk_IsSafePrime which returns SECSuess of
both prime and subprime look reasonable, and sets a Bool to PR_TRUE
is subprime -> prime is safe (subprime = (prime-1)/2. These
functions are implemented in sftkdhverify.c 4.Cleanup incorrect
nominclature on primes (safe primes are not strong primes).
[0be91fa2217a]
* gtests/softoken_gtest/softoken_dh_vectors.h,
gtests/softoken_gtest/softoken_gtest.cc:
Fix more of the timeout issues on tests. (Drop expensive 4098 dh
tests ).
[4014c075a31b]
2020-07-29 Makoto Kato <m_kato@ga2.so-net.ne.jp>
* coreconf/config.gypi, lib/freebl/Makefile, lib/freebl/blinit.c,
lib/freebl/freebl.gyp, lib/freebl/sha1-armv8.c,
lib/freebl/sha_fast.c, lib/freebl/sha_fast.h:
Bug 1650702 - Use ARM's crypt extension for SHA1. r=kjacobs
ARM Crypto extension has SHA1 acceleration. Using this, SHA1 is 3
times faster on ARMv8 CPU. The following data is AWS's a1 instance
(Cortex-A72).
Before ====== ``` # mode in opreps cxreps context op time(sec)
thrgput sha1_e 954Mb 31M 0 0.000 10000.000 10.000 95Mb ```
After ===== ``` # mode in opreps cxreps context op time(sec) thrgput
sha1_e 2Gb 94M 0 0.000 10000.000 10.000 288Mb ```
[68b6eb737689]
2020-07-29 Jan-Marek Glogowski <glogow@fbihome.de>
* manifest.mn:
Bug 1653975 - Set "all" as the default Makefile target r=jcj,rrelyea
Just reorder the rules in manifest.mn, so all is again the first
rule. This restores pre-3.53 Makefile defaults.
[eb52747b7000]
2020-07-31 Makoto Kato <m_kato@ga2.so-net.ne.jp>
* lib/freebl/blapii.h, lib/freebl/blinit.c, nss-tool/hw-support.c:
Bug 1654142 - Add CPU feature detection for Intel SHA extension.
r=kjacobs
[e6b77a9c417a]
2020-08-03 Nathan Froyd <froydnj@mozilla.com>
* coreconf/detect_host_arch.py:
Bug 1656986 - special-case arm64 in detect_host_arch.py; r=jcj
This case comes up when attempting to build NSS on ARM64 Mac. If we
don't do this, we wind up detecting arm64 as "arm", with predictably
bad consequences.
[afa38fb2f0b5] [tip]
Differential Revision: https://phabricator.services.mozilla.com/D85888
The order in which toolchain binaries are resolved change based on environmental factors,
such as whether Firefox is being built in release mode or not.
An informative log was added in either case.
Differential Revision: https://phabricator.services.mozilla.com/D84439
This has the side effect of not initializing fontconfig before the
valgrind test itself runs, which changes the code path leading to
`FcConfigAddDirList`, which eventually leads to suppressed leaks.
Those leaks are then not discarded because the caller doesn't match what
is in the suppression file anymore, so we remove the caller.
Differential Revision: https://phabricator.services.mozilla.com/D85353
This solves the same problem we attempted to solve in bug 1654663. That was a low-cost, sensible solution when there was only one in-build reference to `glean_parser`, but with project FOG we're about to drastically increase the in-build reliance on the library, so the ad-hoc `sys.path` manipulation is an increasingly insensible solution. Here we address this in a first-class way by specifying that `glean_parser` should be imported in `virtualenv`s, but NOT by top-level `mach` commands that run outside of an in-`objdir` `virtualenv`.
Differential Revision: https://phabricator.services.mozilla.com/D85182
It turns out setting CARGO_PROFILE_RELEASE_LTO has unwanted side
effects.
First it's not actually strictly equivalent to using `cargo rustc --
-Clto`. For instance, it apparently also enables cross-language LTO in
newer versions of cargo.
Second, it changes the rust computed hash for all the dependencies of
the crate being built with the variable set, which makes them diverge
from when the same dependencies are built through another crate in the
tree that is not LTOed. This effectively makes us build a _lot_ of
crates twice, many of which are not cacheable.
Since the original problem is that cargo >= 1.45 passes extra flags (`-C
embed-bitcode=no`) to rustc that are incompatible with `-Clto`, and while
it knows to adjust based on the `lto` setting in the build profile
(which CARGO_PROFILE_RELEASE_LTO overrides the default of), cargo
ignores flags passed via `cargo rustc -- ...` when making those
adjustments.
So, we need to override with `-C embed-bitcode=yes` on our own at the
same time we pass `-Clto`. But doing that through `cargo rustc -- ...`
is not enough because all the dependencies of the crate built with
`-Clto` need to be built with `-C embed-bitcode=yes`. So we need to
override with `RUSTFLAGS`, which will affect all the dependencies.
But we also need to do this consistently across all crates, not only the
dependencies of crates built with `-Clto`, otherwise we'd still end up
building crates twice (once with and once without the override).
Unfortunately, the `-C embed-bitcode=*` flag is also not supported in
versions older than 1.45, so we have to avoid adding it on older
versions.
We unfortunately support a large range of versions of rustc (albeit only
for tools/crashreporter), but we actually need to upgrade the smaller
supported version because rustc < 1.38 doesn't support our top-level
Cargo.lock. This makes the version check slightly less awful.
Differential Revision: https://phabricator.services.mozilla.com/D84652
In preparation for Glean telemetry, we scope the availability of the out-of-date vendored
"glean_parser" library to its one usage: "run_glean_parser.py".
This allows Glean telemetry to load its modern "glean_parser" dependency from the
"--user" package environment.
Differential Revision: https://phabricator.services.mozilla.com/D84610
afa1afd410 changed a line at the edge of the context of this patch. I'm really not keen to fork this patch into a separate clang-12 version, so I'd prefer to just shrink the context a little.
Differential Revision: https://phabricator.services.mozilla.com/D84609
Two changes happened during the LLVM 11 timeframe that break our Searchfox plugin.
First, the conversion from `llvm::StringRef` to `std::string` became explicit: adcd026838 This is easy enough to fix in a version-agnostic way.
Second, `mangleCXXCtor` no longer exists: 29e1a16be8 Since there isn't a one-size-fits-all fix, I had to use an ifdef. I mostly cargo-culted the change from 29e1a16be8 (diff-dac09655ff6a54658c320a28a6ea297c).
Differential Revision: https://phabricator.services.mozilla.com/D83838
