This patch also changes how pdbs for the ASAN job are copied:
we relax restrictions so that pdbs if present) are always copied out
and add an environment variable MOZ_COPY_PDBS to indicate when we
want to produce pdbs for copying.
This also requires the 64-bits rust compiler and some build system
tweaks.
And since we make the 32-bits builds cross-compiles on CI, we also need
to adjust the MSVC build mozconfigs such that the host compiler points
to the right MSVC cl. Likewise, the DIA SDK is used for host things, so
use the 64-bits version or it.
Differential Revision: https://phabricator.services.mozilla.com/D7845
--HG--
extra : moz-landing-system : lando
This commit also removes dwarf-exceptions from the x64 build.
sjlj exceptions are needed on x86 because there is a bug currently involving
SEH exceptions on x86. However on x64 there is not, so we can use the
default SEH and get rid of dwarf exceptions. Additionally, to use SEH
exceptions, we need to -fuse-cxa-atexit
Differential Revision: https://phabricator.services.mozilla.com/D7759
--HG--
extra : moz-landing-system : lando
This commit also removes dwarf-exceptions from the x64 build.
sjlj exceptions are needed on x86 because there is a bug currently involving
SEH exceptions on x86. However on x64 there is not, so we can use the
default SEH and get rid of dwarf exceptions. Additionally, to use SEH
exceptions, we need to -fuse-cxa-atexit
Differential Revision: https://phabricator.services.mozilla.com/D7759
--HG--
extra : moz-landing-system : lando
This also requires the 64-bits rust compiler and some build system
tweaks.
Differential Revision: https://phabricator.services.mozilla.com/D7845
--HG--
extra : moz-landing-system : lando
DONTBUILD as the coverage builds are only run on mozilla-central
--HG--
extra : rebase_source : 6ffbd33524b8d4100d33871aeb14c1011a61bee5
extra : histedit_source : d2b54dde2836f5e13017b6bf587f82629de77b80
We need to sign parts of the contents of the archives, so the mar's that we
ship get built as part of the repackage task. Thus, there is no reason to also
create and upload as part of the build, just to throw them away.
Differential Revision: https://phabricator.services.mozilla.com/D6213
--HG--
extra : moz-landing-system : lando
Function names in gcda are just here to check that they match the ones we've in gcno: this is an extra check since it already exists a function checksum for this purpose.
So the size of gcda and the time to generate them will decrease.
Differential Revision: https://phabricator.services.mozilla.com/D6413
--HG--
extra : moz-landing-system : lando
Some mozconfigs actually rely on testing whether the variable is set or
not, which may or may not depending on the mozharness configuration,
and doesn't necessarily match what the mozconfigs do.
So in all mozconfigs that enable PGO, make them use an environment
rathen than ac_add_options.
Differential Revision: https://phabricator.services.mozilla.com/D5843
--HG--
extra : moz-landing-system : lando
Now that Linux PGO builds also do LTO and all the Linux builds use
clang, there's not much use for separate LTO builds.
Differential Revision: https://phabricator.services.mozilla.com/D5632
Bug 1473786 enabled LTO on mac builds for nightlies, but this had the
side effect of enabling it on unrelated builds that just happen to
include the nightly mozconfig just because out mozconfigs kind of suck.
Differential Revision: https://phabricator.services.mozilla.com/D5482
--HG--
extra : moz-landing-system : lando
This is the equivalent of bug 1432475 but for macOS.
MozReview-Commit-ID: DT8V9Vd9eLS
--HG--
extra : rebase_source : 982e2fcb8aed0640acd80025319fdcac24634c47
The js shell is symlinked back to the js objdir by a one off rule.
This fails in the current tup build because the symlink rule is
written in the Tupfile before the rule to build the shell.
MozReview-Commit-ID: 2FOv9FovXLm
--HG--
extra : rebase_source : de8dce6a1b50509b1d0da33eb7b6c35749269733
The js shell is symlinked back to the js objdir by a one off rule.
This fails in the current tup build because the symlink rule is
written in the Tupfile before the rule to build the shell.
MozReview-Commit-ID: HR04x8lyEkg
--HG--
extra : rebase_source : 06a3059c8aa5f454e17c7ca8e54d39cb9b50c3cb
This build target doesn't have LTO enabled on it (yet)
MozReview-Commit-ID: 56tAHMyvH7o
--HG--
extra : rebase_source : 90039cd8e97332e2ef8aad7908b8a04b2869f4a5
Because we don't care about them for this build configuration.
MozReview-Commit-ID: JKEN2pN2x4K
--HG--
extra : rebase_source : b7ce0228f7086a5f933a7cdd6c4695eabb1530f1
This commit adds CI tasks to perform "plain" builds. These builds use
the same toolchains used by official builds. But that's about it. The
mozconfig changes are minimal and only set up paths to toolchain
artifacts. sccache is not used.
The main goal of these builds is to have a "reference" build that
matches an out-of-the-box build environment as much as possible. We want
this mainly so we have timing and behavior information that matches what
developers are doing.
The Windows/generic Taskcluster worker doesn't like it when you specify
an artifact directory that doesn't exist. So we needed to add a
mozharness step to ensure UPLOAD_PATH exists to prevent those tasks from
erroring.
I'm not super thrilled about using mozharness here. We definitely don't
really need mozharness. But the main thing it is providing that we care
about is the Perfherder metrics data. I don't feel like scope bloating
to move that out of mozharness at this time.
I only implemented Linux64 and Windows64 because I'm not convinced
coverage on 32-bit build variations is useful. Tasks only run on trunk
because they are informational only and we don't need to waste resources
running these on release repos and on Try. They are tier 2 because they
aren't critical to shipping Firefox.
MozReview-Commit-ID: Gl6hGYbFX9b
--HG--
extra : rebase_source : 05360d2f6e5dbfed5543726a1be4b0e5d00e0b3d
The previous commit moved creating installers to be side effect of creating
packages. This makes the installer step not actually do anything. So remove the
step from automation.
Differential Revision: https://phabricator.services.mozilla.com/D864
--HG--
extra : rebase_source : 75b67a6e57031ae189dafe1d9854e4105aa22621
extra : source : fcb756834abbdbe0ae6e59a8cfdf39024f50a226
The previous commit moved creating installers to be side effect of creating
packages. This makes the installer step not actually do anything. So remove the
step from automation.
Differential Revision: https://phabricator.services.mozilla.com/D864
--HG--
extra : rebase_source : 174a366890da295634ef8971d0608ea60979f110
extra : histedit_source : 06fdd0e2ccb93f061ba5a40c2a4137eed9898916
If you happen to be unfortunate enough to be testing changes that crash
xpcshell during the xpcshell self-tests, you'll be presented with error
messages like:
PROCESS-CRASH | test_child_assertions.js | application crashed [unknown top frame]
Crash dump filename: c:\users\task_1522676338\appdata\local\temp\xpc-other-mtot6h\cfaea928-e995-4430-baf9-0066c6b91be9.dmp
MINIDUMP_STACKWALK binary not found: z:\build\build\tools\breakpad\win64\minidump_stackwalk.exe
and then be told that, essentially, "your test failed because it
failed", which is unhelpful for figuring out what went wrong.
Apparently we had MINIDUMP_STACKWALK set at one point, and then it got
moved. Let's fix this situation by a) downloading minidump_stackwalk
out of tooltool (our test harnesses do this already); and b) pointing
MINIDUMP_STACKWALK at the correct location. With these changes, we can
once again get crash stacks if xpcshell (and probably a few other
things) fail their self tests.
This keeps --disable-stylo working and --enable-stylo=build with the same
semantics, but it makes also --enable-stylo / and the default to not build the
old style system at all.
This also removes the stylo-only platforms, since they're now the default.
MozReview-Commit-ID: DL2eZZn9suE
This requires unlocking the unstable features in the rust compiler
by using RUSTC_BOOTSTRAP=1
MozReview-Commit-ID: 1uUG1Ekp1YH
--HG--
extra : rebase_source : d8a5aa5d13f3ee055de8a544b0d5ca8af0aab751
As of bug 1430036 it was only set when building on CentOS, and as of bug
1432398, we don't have CentOS-based docker images anymore.
--HG--
extra : rebase_source : 5ade9bee773bca3283cfdb9d69209033fe82253f
The binutils we currently use as part of our GCC toolchain artifact
doesn't understand some relocations in the CRT objects on Debian
stretch, making the embedded CRT objects from bug 1427344, which we want
to remove in bug 1431251, necessary.
OTOH, there is no benefit from using our GCC toolchain artifact for host
compilations on those builds. In fact, Android builds, which are in a
similar position, being built on Debian stretch and being cross-builds,
don't care to use our GCC toolchain artifact.
It's arguably a good thing that those builds are not tied to the version
of GCC we use to build Firefox for linux, so let's remove this
dependency.
--HG--
extra : rebase_source : a80d4e4fb01a4862b844ebde0c521a635f462c0a
moz.configure automatically enables profiling if the milestone is
Nightly (see js/moz.configure:226). So, --enable-profiling in the
nightly mozconfigs is redundant and can be removed.
The whitelist has also been updated to reflect the removal of this
line.
MozReview-Commit-ID: 7nUJVcFOp6c
--HG--
extra : rebase_source : 86db8c2bf646c83701ade8c4a10d667d1c2da6f1
mk_add_options has this kind of awkward feature where
mk_add_options VAR=value
would set VAR for the build through client.mk, but not when running
make -C objdir target. But
mk_add_options "export VAR=value"
does.
We might want to change that on the long run, but the side effects would
have to be calculated first.
OTOH, we have automation jobs that run compilations during `make check`
(e.g. rusttests), which is not invoked through client.mk. So they
currently don't get the same PATH as the build part, meaning that
they're using system binutils instead of the one from the GCC toolchain
package.
--HG--
extra : rebase_source : aab7f221243c486cf70c7b0c91b9313231050ed8
OSX (cross) repackages are currently using a tooltool manifest to get
libdmg and hfsplus. Change those jobs to use the toolchain artifacts
instead.
At the same time, modify the repackage mozharness script's _run_tooltool
so that it doesn't fail with MOZ_TOOLCHAINS being set but without a
tooltool_manifest_src, matching the similar function in buildbase.py.
--HG--
extra : rebase_source : d128d4709c5d1d28d1a6b9c585fde82e99f725c7
The reason it was set from mozconfigs is that profiling require it. But
since it was added, bug 751355 made it implied by --enable-profiling,
and bug 1144842 further made sure that profiling and STRIP_FLAGS were
tied together.
--HG--
extra : rebase_source : c2a6b03bf007e661db48e40cca98e81aaa04c878
It now only does something trivial, which also happens to be a no-op
because it's the default. It does have a commented entry for possible
gtk+2 builds, but we're soon going to remove that possibility anyways in
bug 1278282.
--HG--
extra : rebase_source : 9ac927bb7bd8c057264c8f6f9ca5cbf79a839c4e
It turns out that in all cases it was the last tooltool manifest entry,
so we can remove the tooltool manifests entirely, and remove all
references to them.
--HG--
extra : rebase_source : d8447b5422e63e88444008fddb76d658829694de
Now that build environment docker images have gtk+3 installed in
/usr/local, adjust mozconfigs to point pkg-config there, and remove
all the glue that was required to build using the tooltool package.
Also remove the --x-libraries=/usr/lib on 32-bits builds, which only
confuses the linker.
--HG--
extra : rebase_source : c7de7b3959a3c6b77ea202d9609c891b5b7ec442
It now only does something trivial, which also happens to be a no-op
because it's the default. It does have a commented entry for possible
gtk+2 builds, but we're soon going to remove that possibility anyways in
bug 1278282.
--HG--
extra : rebase_source : 0de751e523ee002bbe6638d223eb384364edd22b
It turns out that in all cases it was the last tooltool manifest entry,
so we can remove the tooltool manifests entirely, and remove all
references to them.
--HG--
extra : rebase_source : 0aa9ef8151c2fccf62507dfecc0bc57b157772e1
Now that build environment docker images have gtk+3 installed in
/usr/local, adjust mozconfigs to point pkg-config there, and remove
all the glue that was required to build using the tooltool package.
Also remove the --x-libraries=/usr/lib on 32-bits builds, which only
confuses the linker.
--HG--
extra : rebase_source : 22b1273ae4b78807b355d33ed5895bdfe83a141d
At the same time, we make it actually do something on spidermonkey
builds. We also add an environment variable alternative, that we use
in mozconfig.stdcxx, allowing for mozconfig.no-compile to override it
and avoid configure failures on e.g. artifact builds.
--HG--
extra : rebase_source : b68d362025e0c99f9184a03391c652ec2c9357ad
The "contract" for toolchains is that extracting foo.tar.xz creates a
directory named foo/. That is however not true for mingw32.tar.xz, which
extracts into gcc/, possibly overwriting files from the gcc.tar.xz
archive (which is also used for mingw builds, for the host part).
This is also not true for nsis.tar.xz, but it reportedly has problems
when it's not in the same directory as mingw32.
But mingw32 doesn't actually need to be mixed with gcc, so it's better
to separate them as they are supposed to be.
--HG--
extra : rebase_source : 30d90af64459bbb31bc076e48f3c661fa9cd4a79
With all of our builds in Taskcluster now, we should never be uploading
symbols from build tasks. Unfortunately Windows builds were still doing so.
This patch removes MOZ_AUTOMATION_UPLOAD_SYMBOLS from all the in-tree
mozconfigs and a few other places so that it should always default off
(per moz-automation.mk). The rest of the uploadsymbols bits will be
removed once Thunderbird fixes their automation.
This patch was mostly autogenerated by running:
rg --files-with-matches UPLOAD_SYMBOLS browser/config/mozconfigs/ mobile/android/config/mozconfigs/ | xargs sed -ri '/.*UPLOAD_SYMBOLS.*/d'
sed -ri '/.*UPLOAD_SYMBOLS.*/d' build/unix/mozconfig.linux build/mozconfig.win-common build/macosx/local-mozconfig.common build/mozconfig.automation
Then mobile/android/config/mozconfigs/common and
taskcluster/scripts/builder/build-linux.sh were hand-edited.
MozReview-Commit-ID: Cy8kSEodSg4
--HG--
extra : rebase_source : 01caf1651b4eb428313e1f371aa585f8f34c4151
exe_7z_archive.py runs during the MinGW build L10N check step, and
hardcodes 7z as the 7zip executable. This works on Windows, but not
Linux. We need to pass it the correct executable for 7zip, which is
located during configure.
However, repacks (repackage-winXX-nightly) don't do configure, and
don't have the tools to do configure. So we leave the default
command in the python script if one is not supplied.
MozReview-Commit-ID: B7GEKRYEJTD
--HG--
extra : rebase_source : 10ec7e688d53341625217306e88f2e647195ce8d
None of these are defined in any "nightly" mozconfigs.
Some of these lines may occur and be set by mozconfigs. But the
entire premise of compare-mozconfigs is flawed because it only compares
file content: not the end result of mozconfig evaluation. (mozconfigs
are shell scripts and a lot of logic occurs in other sourced shell
scripts.)
MozReview-Commit-ID: 2Nwo3ePJreb
--HG--
extra : rebase_source : 4fef846c6e501cef29ca9bca573f376e9ac2f97c
This variable isn't consumed anywhere. I have no clue where it
was consumed. It appears to be the product of an old era.
MozReview-Commit-ID: 6iRdLW89ZjW
--HG--
extra : rebase_source : d338287d78462a78cf70671b00d7cf1fd9520785
extra : source : aa0428bd81463d7e12454238681f27085678c92a
There are no other references to PROFILE_GEN_SCRIPT outside
of the mozconfig whitelist file. The deleted entries from the
whitelist file were meaningless.
MozReview-Commit-ID: IhOV8Hlfmhl
--HG--
extra : rebase_source : e118008a8a87c83e6e025eca9e7ce51ce8fce0a3
extra : source : 5280a8429e4174e1cb1e0dd4c5e34e9fe539b171
We no longer define MOZ_MAKE_FLAGS in the in-repo mozconfigs. Nor do we
pass MOZ_MAKE_FLAGS from any automation configs as far as I can tell.
MozReview-Commit-ID: LYU1dI44uhX
--HG--
extra : rebase_source : 08a9e7801ae619ba1cd36f6c262c29a4735035d0
extra : intermediate-source : 8430d8501abcd30d15e0ef73d9bfa64d6ede8697
extra : source : 3f151657ee8d7a5a87629826f654ee5c807b5346
--enable-elf-hack is the default on all platforms where it's supported,
and is completely ignored on platforms where it's not supported.
While moving the flag to moz.configure, we're going to make it only
work on platforms where elfhack is supported, so we at least need to
remove it from mozconfigs for those platforms where it's not supported.
But generally speaking, we want less things in mozconfigs, so just
remove it from there, since it's the default anyways.
With that fixed, Re-enable MOZ_AUTOMATION_L10N_CHECK for MinGW
MozReview-Commit-ID: BDTtY1J7sBj
--HG--
extra : rebase_source : 162267bf08a6e477af3b8b119a9ef9df65ba8267
This line was added to the nightly macosx64 mozconfig in 9312a1903bf4
(bug 1391643). But we didn't catch it because compare-mozconfigs
only analyzes the current platform and we cross compile for MacOS
from Linux.
MozReview-Commit-ID: 35Q2c0mybPi
--HG--
extra : rebase_source : 25707037ca07ec22de283abcfe96c4faea71d55e
The mechanism by which PGO builds are kicked off is kinda wonky.
The MOZ_PGO environment variable is recognized by configure and setting
it will result in MOZ_PGO being defined in substs. In addition, the
build backend (previously client.mk, now Makefile.in) also recognizes
MOZ_PGO (from mozconfig or environment) and takes appropriate action.
In-tree mozconfigs set MOZ_PGO via mk_add_options. mk_add_options
is intended as a mechanism to inject state into client.mk and the
make-based build system.
In addition, there is code in mozharness (unchanged by this commit)
that sets MOZ_PGO if appropriate.
A PGO build configuration is different from a non-PGO build
configuration. Therefore a make-centric environment variable to
control PGO is not ideal. Instead, this should be defined as a
configure-time flag and the build invocation should key off that.
This commit normalizes in-tree mozconfigs to set MOZ_PGO via
ac_add_options and updates the PGO documentation to recommend
this method.
MozReview-Commit-ID: k6AZyJuXjs
--HG--
extra : rebase_source : b1a6348611eba08dd67ec938cca5586fbe8e6910
extra : source : 25c7ebc7c44dd253f421b6de3d0337635d0c99d0
The new VS package is now based on update 15.4.2, although that release didn't affect any files in our package. I'm picking up the update mostly just to make filename unique.
This also enables the crash reporter on the MinGW build, as this is the
only thing blocking that from working.
MozReview-Commit-ID: Hygd7UUQvwl
--HG--
extra : rebase_source : a4a12b8edaa5b1fba869d6f7c21fc8444be2d9d7
Repack of the new Visual Studio release using the packaging
scripts from bug 1407678. This version also includes the
pgo runtime, resolving a performance regression from the
previous package.
MozReview-Commit-ID: LhoVyG4IwmP
--HG--
extra : rebase_source : 0d3d2f28f05335976d236e5f76893307c252dc96
Repack of Visual Studio 2017 15.4.0 with SDK 10.0.16299.0
created by David Major by running the installer on a fresh
VM and then running the updated packaging script from
bug 1407678.
MozReview-Commit-ID: 7ZKoA6ncOPr
--HG--
extra : rebase_source : dcb72a71f34ada6ead631fe8fac0b31f0ddb8e29
We need mozharness configurations and mozconfigs for rusttests. We are
explicitly not doing Windows debug configurations currently because of
peculiar link errors in such configurations.
Add a toolchain job description which calls the
repack_rust.py script to package the requested
upstream build of Rust and its standard libraries
for use in gecko builds.
Links are added to these new toolchains for various build
and analysis tasks as appropriate. The base-toolchain
tasks use an explicitly-versioned toolchain since those
can be different from the current release used for most builds.
The corresponding tooltool manifest entries are removed
now that taskcluster artifact versions are available.
This simplifies the update process since new toolchains
can be packaged and used automatically by just updating
the versions in the task descriptions.
A 'linux64-rust' toolchain can be added to other tasks
as a dependency and artifact. It supports linux64-
hosted builds of Rust code targeting linux64 or linux32.
A 'linux64-rust-macos' toolchain targets linux64-hosted
builds of Rust code targeting macOS on x86_64.
A 'linux64-rust-android' toolchain targets linux64-hosted
builds of Rust code targeting various Android architectures.
Two 'win64-rust' and 'win32-rust' toolchain tasks create
similar entries for Windows-hosted builds. All our automation
builds are hosted on win64, so we could use one artifact
with support for both targets, but currently this doesn't
work because of cross-compilation issues in some crates.
This patch maintains the previous separation between
win32 and win64 rust toolchains until that can be addressed.
MozReview-Commit-ID: GRiJml8CtzO
--HG--
extra : rebase_source : 09a3698ce7f9a8b5f2b5d9b5a1fde9c05dc6b540
