Firefox uses multiple processes. It has intentional leaks, and when
running with ASAN, we have suppressions to eliminate those. When running
ASAN builds through CI tests, when Firefox exits, each of the processes
(parent and child) exits and goes through its leaks and when there are
(which is a given), the ASAN runtime runs llvm-symbolizer to symbolicate
and match against suppressions. So each process runs llvm-symbolizer. At
the same time.
Some of the addresses to symbolicate are in libxul. Which contains all
DWARF info, making it a ~1GB monster. Oh, and because you're lucky,
things align perfectly such that libxul size is a multiple of the page
size. That makes llvm-symbolizer pread() the file instead of mmap()ing
it. Did I say there are multiple processes? So suddenly you have n
processes simultaneously allocating and filling 1GB of memory each, on
CI machines that have enough memory for the job they usually run, but
not enough for a sudden rush of n GB.
And things go awry. When you're lucky and the OOM killer didn't take
care of killing the CI entirely, symbolication couldn't happen and the
suppressions are not matched, and leaks are reported.
This all turns out it originates in how llvm-symbolicate chooses between
pread() and mmap(), which turns out is just defaults not being made for
binary files.
Differential Revision: https://phabricator.services.mozilla.com/D16010
--HG--
extra : moz-landing-system : lando
In bug 1514288 we started to save the output from running Firefox during
PGO builds into log files, but they aren't correctly uploaded if the run
fails. This presents a problem for categorizing failed PGO builds if the
profileserver returns an error code (eg: bug 1517939), since the error
messages could be hidden away in log files that aren't uploaded.
Instead, we can generate them directly into the artifacts directory so
that they are always uploaded. Additionally, the log files are displayed
if the run fails so that the error messages are displayed in the log for
easier bug categorization.
Differential Revision: https://phabricator.services.mozilla.com/D15881
--HG--
extra : moz-landing-system : lando
Instead of per-document. This also allows to reuse this thread-pool if needed
for other stuff, like parallel CSS parsing (#22478), and to share more code with
Gecko, which is always nice.
This cherry-picks https://github.com/servo/servo/pull/22487, with a few minor
fixes to the build that are landing as part of the sync associated to this bug,
and an lsan exception tweak to point to the right module since it's moving.
The output from running the browser during PGO builds can have innocuous
error messages in them, but show up in treeherder as potential messages
to include when filing bugs. We can just save the output from these runs
as files and upload them as artifacts instead, so they don't show up in
the build log but are available for inspection if necessary.
MozReview-Commit-ID: 3VdVCKVkZNI
Differential Revision: https://phabricator.services.mozilla.com/D15154
--HG--
extra : moz-landing-system : lando
In bug 1259382, some workarounds were added to make the build system
alter PATH and not use absolute paths for toolchain programs, because
autoconf and the build system doesn't deal with spaces in those very
well. But later in bug 1290040, we made find_program return Windows
short paths (without spaces), which alleviates the need for those
workarounds.
We still, however, and unfortunately, need to alter PATH to account for
the fact that MSVC DLLs are not necessarily alongside the compiler
executables...
Depends on D15181
Differential Revision: https://phabricator.services.mozilla.com/D15182
--HG--
extra : moz-landing-system : lando
Like for other windows platforms. This currently doesn't make a
difference, but will with next change.
Differential Revision: https://phabricator.services.mozilla.com/D15181
--HG--
extra : moz-landing-system : lando
We remove --disable-libjpeg-turbo because that's only useful when Yasm
is too old, and the required version is now almost 8 years old, so we
can reasonably require people to upgrade rather than workaround with a
--disable option.
The valid_yasm_version function can seem overkill, but that's because
future moves of other things to python configure will pile up.
Differential Revision: https://phabricator.services.mozilla.com/D15184
--HG--
extra : moz-landing-system : lando
Now that we're not even building host static libraries, we don't need
variables for the tools used to build them.
Ironically, we weren't even running HOST_RANLIB.
Depends on D15172
Differential Revision: https://phabricator.services.mozilla.com/D15173
--HG--
extra : moz-landing-system : lando
So far, the main subject of cross-compiles was to cross compile from one
OS to another (e.g. {linux,osx} -> android), but there are a few useful
cases where the OS doesn't change, and, with --host being guessed, we
can just have developers pass --target=$cpu instead of a complete
target triplet. This can be useful to do x86 Linux builds on x86-64
Linux hosts, or aarch64 Windows builds on x86-64 Windows hosts.
Differential Revision: https://phabricator.services.mozilla.com/D15063
--HG--
extra : moz-landing-system : lando
Remove the version check for WINDRES, because, as per bug 454112, it
didn't actually work, and, making it work actually causes problems
because llvm's windres, used with mingw clang, has version 0.1.
Differential Revision: https://phabricator.services.mozilla.com/D15070
--HG--
extra : moz-landing-system : lando
vswhere only searches in Community, Professional and Enterprise, but one can
also install BuildTools only, which has a different product name.
Differential Revision: https://phabricator.services.mozilla.com/D15056
--HG--
extra : moz-landing-system : lando
It will be useful to run tests like try_compile, with different flags and different
kinds of sources.
Depends on D14949
Differential Revision: https://phabricator.services.mozilla.com/D14950
--HG--
extra : moz-landing-system : lando
Although this task technically doesn't build a toolchain, the set of
steps it needs to do is very similar to what a toolchain build does, so
we're shoehorning this task into the toolchain kind. The task basically
runs `cargo vendor` on the gfx/wr/Cargo.lock file (if/when it changes)
and exports a tarball of the resulting vendored crates. This allows
downstream tasks that build stuff in gfx/wr to not have to re-fetch
these crates from crates.io on every test run.
Differential Revision: https://phabricator.services.mozilla.com/D14406
--HG--
extra : moz-landing-system : lando
