The llvm-symbolizer tasks currently extract a llvm-symbolizer from clang
tasks. Changes in clang 14 make the hack that we have in place to keep
llvm-symbolizer statically linked to libllvm while clang uses a dynamic
libllvm not work anymore, so it's time to bite the bullet and build
llvm-symbolizer separately.
We share most of the build setup with the compiler-rt build.
Differential Revision: https://phabricator.services.mozilla.com/D140711
Use the SHT_RELR format which significantly improves the size reduction
from elfhack:
total size of .rel.* + .elfhack.* sections
x86 x86_64 android-arm
plain 3532904 10739544 3488888
current-elfhack 1085815 1155994 1042048
relr-elfhack 130219 193552 113840
Differential Revision: https://phabricator.services.mozilla.com/D134756
Elfhack currently can't deal with files larger than 4GiB because it
translates all ELF data structures to the 32-bits variant, even for
64-bits ELF files. So if the original file has e.g. sections that start
after the 4GiB boundary, they can't be represented in memory.
Practically speaking, this is not causing problems, but has prevented a
working elfhack for aarch64 because e.g. some relocation types don't fit
in the 32-bits ELF representation.
Differential Revision: https://phabricator.services.mozilla.com/D134745
We leave the following ones unchanged:
- geckodriver because the results are used to releases on github.
- sixgill because the script that creates it is not in-tree.
- *-dist-toolchain because sccache is not expecting a .tar.zst.
We use native tar support in most cases, except for toolchain scripts also
used on Windows, for which we use our zstdpy script.
Differential Revision: https://phabricator.services.mozilla.com/D124733
We leave the following ones unchanged:
- geckodriver because the results are used to releases on github.
- sixgill because the script that creates it is not in-tree.
- *-dist-toolchain because sccache is not expecting a .tar.zst.
We use native tar support in most cases, except for toolchain scripts also
used on Windows, for which we use our zstdpy script.
Differential Revision: https://phabricator.services.mozilla.com/D124733
We leave the following ones unchanged:
- geckodriver because the results are used to releases on github.
- sixgill because the script that creates it is not in-tree.
- *-dist-toolchain because sccache is not expecting a .tar.zst.
We use native tar support in most cases, except for toolchain scripts also
used on Windows, for which we use our zstdpy script.
Differential Revision: https://phabricator.services.mozilla.com/D124733
We leave the following ones unchanged:
- geckodriver because the results are used to releases on github.
- sixgill because the script that creates it is not in-tree.
- *-dist-toolchain because sccache is not expecting a .tar.zst.
We use native tar support in most cases, except for toolchain scripts also
used on Windows, for which we use our zstdpy script.
Differential Revision: https://phabricator.services.mozilla.com/D124733
librnp is used by Thunderbird for OpenPGP support. Until now, official builds
have been built statically linked to Clang/LLVM's libc++ to avoid problems with
libstdc++ symbols.
Recent build changes have narrowed the gap significantly, leaving out_of_range
and invalid_argument called with char const* rather than std::string.
Differential Revision: https://phabricator.services.mozilla.com/D123381
As this means stdc++compat also would need to be built when
cross-building for Windows on Linux, we also recurse in the stdc++compat
directory in that case. It was already possible to use
--enable-stdcxx-compat in that configuration, and that wasn't working
(there are other problems that prevent it from working, but we're going
to fix them shortly).
Differential Revision: https://phabricator.services.mozilla.com/D122195
Since the minimum required version of gcc and libstdc++ is 7.1 as of bug
1536848, the _GLIBCXX_RELEASE macro is available on all supported
versions. Which means instead of grabbing the largest version of the
libstdc++ symbols and tweaking for that, we can just tweak for
_GLIBCXX_RELEASE. We also remove the conditions that are always true due
to the versions involved.
Differential Revision: https://phabricator.services.mozilla.com/D122194
A lot of work happens on the wayland backend, and it regularly breaks tier-3
systems where wayland is not supported. Setting up X11-only builds will help
catch those breakages earlier.
Differential Revision: https://phabricator.services.mozilla.com/D121691
This allows to use the same toolchain docker images as other toolchains,
based on Debian buster.
While here, use the default max-run-time, which is more than enough for
this toolchain.
Differential Revision: https://phabricator.services.mozilla.com/D119137
Because GCC is built in stages, the final stage is built with
intermediate stages's GCC, which handles the sysroot correctly, so we
end up with headers and libraries with the expected compatibility.
This allows to use the same toolchain docker images as other toolchains,
based on Debian buster.
Differential Revision: https://phabricator.services.mozilla.com/D119136
Bug 1634204 bumped the maximum version of symbols allowed in our
dependency upon libstdc++, which effectively makes some of the
stdc++compat code dead. We can now remove it.
Differential Revision: https://phabricator.services.mozilla.com/D104617
When using the --sysroot argument to clang, clang changes where it
searches for libraries in its own directory, and excludes the lib and
lib32 subdirectories. So we need to move the gcc files to a place where
it does look (and that it also looks without --sysroot).
We however still keep a copy of libstdc++ in the lib directory for
runtime purposes.
Differential Revision: https://phabricator.services.mozilla.com/D104123
There is only one place where it's used:
config/check_vanilla_allocations.py, which is only executed from
js/src/build/Makefile.in on the condition that the build is targeting
Linux and not LTO. But the LTO test is actually outdated, because we
don't build with `-flto`, but `-flto=thin`, so the exclusion doesn't
work anymore.
There is however no AC_CHECK_PROG, and we currently rely on NM to be
given, or fall back to "nm", which works in most cases, except LTO with
clang. It works on CI because in LTO builds we explicitly set NM to
llvm-nm (which can output symbols from LLVM bitcode objects), but we
could also do that automatically.
So we add a full detection of nm/llvm-nm to python configure, and limit
it to Linux, since we only ever use it there.
Differential Revision: https://phabricator.services.mozilla.com/D101681
Only sanitizer builds require a native llvm-symbolizer executable.
Ideally, we'd build llvm-symbolizer from scratch, which would be faster,
but for now, let's go the easy route and just extract it from the
corresponding native clang builds.
We don't actually do anything with the llvm-symbolizer executable on
android builds, so we don't install it in $FINAL_TARGET, avoilding
the dependency on android builds (plus, we actually don't have an
android-native llvm-symbolizer, so even if it were already shipped, it
would be the wrong file).
Differential Revision: https://phabricator.services.mozilla.com/D101076
Only sanitizer builds require a native llvm-symbolizer executable.
Ideally, we'd build llvm-symbolizer from scratch, which would be faster,
but for now, let's go the easy route and just extract it from the
corresponding native clang builds.
We don't actually do anything with the llvm-symbolizer executable on
android builds, so we don't install it in $FINAL_TARGET, avoilding
the dependency on android builds (plus, we actually don't have an
android-native llvm-symbolizer, so even if it were already shipped, it
would be the wrong file).
Differential Revision: https://phabricator.services.mozilla.com/D101076
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
This only solves the easy half of the problem outlined in the bug,
leaving the other half for later.
iostream::tellg() actually returns streampos, which is able to support
files larger than 4GiB with libstdc++, but converting to an int
obviously truncated that, as well as transformed values between 2GiB and
4GiB into invalid negative numbers.
iostream::seekg() also takes a streampos, so storing the streampos as-is
is enough to address the problem with tellg()/seekg() sequences.
The other half of the problem involves elfhack converting 64-bits ELF
headers to 32-bits headers internally, which requires deeper changes.
This change however, is enough to support files up to 4GiB, which is
already a good first step.
Differential Revision: https://phabricator.services.mozilla.com/D94252
The `clobber` targets are superseded by `mach clobber`, so we don't need them for any reason. The `clean` target is meant to get you to a post-`configure` state, but it doesn't really work, and if it's necessary for you to be in that state for some reason you can just clobber and re-`configure`, so it doesn't seem worth it to get it working again. Instead, delete all of them. Also delete `everything` which is not useful when `clobber` doesn't exist.
Differential Revision: https://phabricator.services.mozilla.com/D93514
When linking a weak symbol in an object against a library where the
symbol is provided with a version, the final binary get a weak versioned
symbol reference.
It turns out weak versioned symbols still make the dynamic linker need
the symbol version, even if all symbols needed with that version are
weak.
Practically speaking, that means with bug 1634204, we now end up with
a weak versioned symbol reference to __cxa_thread_atexit_impl with
version GLIBC_2.18, and glibcs without the symbol can't fulfil that
version, even though the weak symbol is the only thing we need from that
version.
This means the check_binary changes in bug 1634204 are too
relaxed, so we revert them (although we keep the easier to read
conditions in check_dep_versions).
We also introduce a hack in stdc++compat.cpp (although it's not
technically entirely about libstdc++ compat) so that we avoid the weak
symbol reference while keeping the intended baseline for libstdc++ and
glibc.
Differential Revision: https://phabricator.services.mozilla.com/D79773
When linking a weak symbol in an object against a library where the
symbol is provided with a version, the final binary get a weak versioned
symbol reference.
It turns out weak versioned symbols still make the dynamic linker need
the symbol version, even if all symbols needed with that version are
weak.
Practically speaking, that means with bug 1634204, we now end up with
a weak versioned symbol reference to __cxa_thread_atexit_impl with
version GLIBC_2.18, and glibcs without the symbol can't fulfil that
version, even though the weak symbol is the only thing we need from that
version.
This means the check_binary changes in bug 1634204 are too
relaxed, so we revert them (although we keep the easier to read
conditions in check_dep_versions).
We also introduce a hack in stdc++compat.cpp (although it's not
technically entirely about libstdc++ compat) so that we avoid the weak
symbol reference while keeping the intended baseline for libstdc++ and
glibc.
Differential Revision: https://phabricator.services.mozilla.com/D79773
When linking a weak symbol in an object against a library where the
symbol is provided with a version, the final binary gets a weak versioned
symbol reference.
It turns out weak versioned symbols still make the dynamic linker need
the symbol version, even if all symbols needed with that version are
weak.
Practically speaking, that means with bug 1634204, we now end up with
a weak versioned symbol reference to __cxa_thread_atexit_impl with
version GLIBC_2.18, and glibcs without the symbol can't fulfil that
version, even though the weak symbol is the only thing we need from that
version.
This means the check_binary changes in bug 1634204 are too
relaxed, so we revert them (although we keep the easier to read
conditions in check_dep_versions).
We also introduce a hack in stdc++compat.cpp (although it's not
technically entirely about libstdc++ compat) so that we avoid the weak
symbol reference while keeping the intended baseline for libstdc++ and
glibc.
Differential Revision: https://phabricator.services.mozilla.com/D79773
When linking a weak symbol in an object against a library where the
symbol is provided with a version, the final binary gets a weak versioned
symbol reference.
It turns out weak versioned symbols still make the dynamic linker need
the symbol version, even if all symbols needed with that version are
weak.
Practically speaking, that means with bug 1634204, we now end up with
a weak versioned symbol reference to __cxa_thread_atexit_impl with
version GLIBC_2.18, and glibcs without the symbol can't fulfil that
version, even though the weak symbol is the only thing we need from that
version.
This means the check_binary changes in bug 1634204 are too
relaxed, so we revert them (although we keep the easier to read
conditions in check_dep_versions).
We also introduce a hack in stdc++compat.cpp (although it's not
technically entirely about libstdc++ compat) so that we avoid the weak
symbol reference while keeping the intended baseline for libstdc++ and
glibc.
Differential Revision: https://phabricator.services.mozilla.com/D79773
This is both for future proofing (fetches could move any time although
they likely won't), and to fix the path on the future Windows PGO
cross builds, where the fetches path is not under $WORKSPACE.
Differential Revision: https://phabricator.services.mozilla.com/D66358
--HG--
extra : moz-landing-system : lando
GENERATED_FILES now defaults to python3 unless py2=True is specified as
an argument. All existing GENERATED_FILES scripts and GeneratedFile
templates have the py2=True attribute added, so this patch should
effectively be a no-op.
Going forward, individual scripts can be converted to python3 and their
corresponding py2=True attribute can be deleted. In effect, this patch
will be backed out in pieces until all scripts run in python3, at which
point the py2 attribute itself can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D60919
--HG--
extra : moz-landing-system : lando
GENERATED_FILES now defaults to python3 unless py2=True is specified as
an argument. All existing GENERATED_FILES scripts and GeneratedFile
templates have the py2=True attribute added, so this patch should
effectively be a no-op.
Going forward, individual scripts can be converted to python3 and their
corresponding py2=True attribute can be deleted. In effect, this patch
will be backed out in pieces until all scripts run in python3, at which
point the py2 attribute itself can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D60919
--HG--
extra : moz-landing-system : lando
GENERATED_FILES now defaults to python3 unless py2=True is specified as
an argument. All existing GENERATED_FILES scripts and GeneratedFile
templates have the py2=True attribute added, so this patch should
effectively be a no-op.
Going forward, individual scripts can be converted to python3 and their
corresponding py2=True attribute can be deleted. In effect, this patch
will be backed out in pieces until all scripts run in python3, at which
point the py2 attribute itself can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D60919
--HG--
extra : moz-landing-system : lando
MOZ_PGO_PROFILE_USE is used to enable certain features in the automation
mozconfigs for profile-use builds, including the MOZ_PROFILE_USE
configure option. They aren't easily combined into one flag due to the
extra settings that are enabled only in automation mozconfigs.
Renaming the former variable to TASKCLUSTER_PGO_PROFILE_USE makes it
slightly more obvious where it comes from and avoids confusion.
Differential Revision: https://phabricator.services.mozilla.com/D56117
--HG--
extra : moz-landing-system : lando
With Bug 1579189 we are going to raise the minimum clang version to 5. But in clang 5
and clang 6 an issue has been introduced where the `Decl` nodes from the `AST` don't
contain all of the annotation attributes. The missing attributes can cause static
analysis failures. We are therefore going to disable the static analysis for the
base-toolchain clang builds as a workaround.
Differential Revision: https://phabricator.services.mozilla.com/D52025
--HG--
extra : moz-landing-system : lando
Setting this flag still attempts to build with 1-tier PGO, unless we
detect that 3-tier PGO is enabled. Since we should be using 3-tier PGO
everywhere, the flag should be redundant in our automation mozconfigs.
It can still be used locally in a mozconfig to do a 1-tier PGO build.
Depends on D46069
Differential Revision: https://phabricator.services.mozilla.com/D46070
--HG--
extra : moz-landing-system : lando
