At the moment, everything we're putting in a wasm sandbox is not
performance-critical, so we don't need PGO. It's also not clear that
PGO would actually work properly with code that's been run through
wasm. Let's leave figuring that out until, at the very least, the wasm
toolchain is a little more mature.
Differential Revision: https://phabricator.services.mozilla.com/D58513
--HG--
extra : moz-landing-system : lando
In 1-tier PGO builds that shared the objdir between the instrumented and
profile-use builds, the instrumented build objects used a different
suffix (.i_o) to separate them from the profile-use build (which uses
the default .o suffix). These builds are now always in separate objdirs,
and don't need special suffix rules anymore.
As a bonus, this helps fix an issue with buildid.cpp continually
rebuilding because libxul_so.list always lists the inputs as *.o, which
don't exist if we're using a .i_o suffix. Make would always re-create
buildid.cpp and therefore libxul.so in the instrumented build even when
nothing has changed.
Differential Revision: https://phabricator.services.mozilla.com/D56115
--HG--
extra : moz-landing-system : lando
Add backend stuff to build sandboxed wasm libraries. (Don't actually update any moz.build files to consume this yet.)
Differential Revision: https://phabricator.services.mozilla.com/D54152
--HG--
extra : moz-landing-system : lando
Actually we set _DEPEND_CFLAGS to both host and target compiler. But if host and target are different compiler type, we may pass invalid option.
Differential Revision: https://phabricator.services.mozilla.com/D38457
--HG--
extra : moz-landing-system : lando
Bug 1560527 was not supposed to change verbosity for mach build, but it
turns out it did, because the ifeq it copied from one place to another
was wrong in the first place.
While here, replace a ifeq that did work with the now equivalent
BUILD_VERBOSE_LOG.
Differential Revision: https://phabricator.services.mozilla.com/D35966
--HG--
extra : moz-landing-system : lando
This makes running without mach more consistent. e.g. running
`make -C $objdir/toolkit/library/rust target` makes the cargo log
verbose, and adding `-s` makes it less verbose.
Differential Revision: https://phabricator.services.mozilla.com/D35521
--HG--
extra : moz-landing-system : lando
We're moving to IR-level PGO instrumentation for clang-cl. We've also
moved to using static linker ordering files, which was the primary
application of the previous style of PGO instrumentation. We therefore
we no longer need this code.
Differential Revision: https://phabricator.services.mozilla.com/D31134
--HG--
extra : moz-landing-system : lando
On one-go MOZ_PGO builds, it's generally not wanted to do LTO during the
profile-generate phase. And the build system doesn't really support
different build options between both phases in this case, so we relied
on MOZ_PROFILE_GENERATE to disable the LTO flags.
However, in standalone profile-generate builds, if --enable-lto is
passed explicitly, the build should respect that choice.
So instead of checking MOZ_PROFILE_GENERATE to disable the LTO flags,
we disable them when MOZ_LTO is not set, and we force it to be disabled
during the profile-generate phase of one-go MOZ_PGO builds.
Differential Revision: https://phabricator.services.mozilla.com/D21659
--HG--
extra : moz-landing-system : lando
This patch adds detection for when icecream is in use to build telemetry.
icecream is commonly enabled in two ways: by either setting CC/CXX to point
to icecream's cc/c++ symlinks, or by setting adding
mk_add_options 'export CCACHE_PREFIX=icecc' to a mozconfig when also using
ccache. For the former, this patch adds a simple configure check to see
if CXX is a symlink to a file named 'icecc'. For the latter this patch adds
CCACHE_PREFIX as a configure subst to capture the value.
We don't currently have a facility for writing telemetry tests that depend on
configure values. Local manual testing shows that it does work as expected.
Differential Revision: https://phabricator.services.mozilla.com/D18138
--HG--
extra : moz-landing-system : lando
Consequently, this removes:
- MOZ_LIBPRIO, which is now always enabled.
- non_msvc_compiler, which is now always true.
- The cl.py wrapper, since it's not used anymore.
- CL_INCLUDES_PREFIX, which was only used for the cl.py wrapper.
- NONASCII, which was only there to ensure CL_INCLUDES_PREFIX still
worked in non-ASCII cases.
This however keeps a large part of detecting and configuring for MSVC,
because we still do need it for at least headers, libraries, and midl.
Depends on D19614
Differential Revision: https://phabricator.services.mozilla.com/D19615
--HG--
extra : moz-landing-system : lando
This was added back in bug 1414287, as a sort of quick hack, but since
then, bug 1515528 fixed things such that the hack is not necessary
anymore, and bug 1523201 allows for things to work on automation
(HOST_LINKER ended up being wrong because of the lack of "proper" VC
configuration on automation)
Differential Revision: https://phabricator.services.mozilla.com/D17790
--HG--
extra : moz-landing-system : lando
The only use in configure itself is for a MSVC version check that is now
always true (we don't accept versions < 19.15 anymore).
The only uses in the build system are in code that could just use
CC_TYPE instead.
Differential Revision: https://phabricator.services.mozilla.com/D17207
--HG--
extra : moz-landing-system : lando
At some point we made L10NBASEDIR required. That means that
env L10NBASEDIR=... make chrome-AB_CD
takes the value set by configure. That is different than
make chrome-AB_CD L10NBASEDIR=...
which uses the value passed on the command line. Rather than making
the latter style work with `mach build`, we instead set the "correct"
value for L10NBASEDIR in automation.
We could remove the --with-l10n-base stanzas from many automation
mozconfigs, but there's some small advantage to keeping them explicit.
Perhaps eventually we will remove them -- hopefully after
standardizing l10n vs l10n-central!
Differential Revision: https://phabricator.services.mozilla.com/D15776
--HG--
extra : moz-landing-system : lando
This also requires the 64-bits rust compiler and some build system
tweaks.
And since we make the 32-bits builds cross-compiles on CI, we also need
to adjust the MSVC build mozconfigs such that the host compiler points
to the right MSVC cl. Likewise, the DIA SDK is used for host things, so
use the 64-bits version or it.
Differential Revision: https://phabricator.services.mozilla.com/D7845
--HG--
extra : moz-landing-system : lando
This also requires the 64-bits rust compiler and some build system
tweaks.
Differential Revision: https://phabricator.services.mozilla.com/D7845
--HG--
extra : moz-landing-system : lando
This merely centralizes logic that was formerly used at multiple sites
into the single remaining use site.
None of the JAVA* flags have been used for a long time.
Differential Revision: https://phabricator.services.mozilla.com/D7313
--HG--
extra : moz-landing-system : lando
BUILD_TOOLS was only ever used for things that another variable provides
equally well. Removing BUILD_TOOLS means that we can remove win_srcdir
and WIN_TOP_SRC as well.
Currently mozconfig.cache overrides a few build options for sccache.
This patch moves them into toolchain.configure so that the build system
will set them properly when sccache is in use. Additionally,
{CC,CXX}_WRAPPER are set in config.mk, so just avoid setting them when
sccache is in use.
We perform, on the binaries we build, a series of check, that are
implemented as half-baked make commands, invoked after linking them.
- check libstdc++ symbol versions to ensure binary compatibility with
a baseline.
- check glibc symbol versions to ensure binary compatibility with a
baseline.
- check that target binaries don't contain text relocations.
- check that libmozglue is linked before libc on android.
- on libxul, check that NSModules are laid out correctly.
- on libxul, check that there is more than one PT_LOAD segment.
Those checks happen to work where they matter, but their setup is
unreliable. For example, the checks for symbol versions are supposed to
work for libclang-plugin on cross osx builds, but in fact, don't,
because the readelf path doesn't exist, and the command doesn't fail in
that case.
So move them all to a standalone script, performing the checks more
thoroughly (especially the NSModules one, where we now also check that
they are all adjacent), and more verbosely.
--HG--
extra : rebase_source : 7072e622e95f363d4a6c3a8e272d3445d998b592
The make backend was treating the first output of a GENERATED_FILES rule
specially, since it was the target of the rule containing the script
invocation. We want the outputs of GENERATED_FILES rules to be
FileAvoidWrite so that we avoid triggering downstream rules if the
outputs are unchanged, but if the target of the script invocation is
FileAvoidWrite, then make may continually re-run the script during a
no-op build.
The solution here is to use a stub file as the target of the script
invocation which will always be touched when the script runs. Since
nothing else in the build depends on the stub, we don't need to
FileAvoidWrite it. All actual outputs of the script can be
FileAvoidWrite, and make can properly avoid work for files that haven't
changed.
MozReview-Commit-ID: 3GejZw2tpqu
--HG--
extra : rebase_source : 2b9be82f893e89a4c2f254f05b1e8b9a0f9c631b
The uninstaller was being built as a side-effect of building `setup.exe`. In
Bug 1385227, that was moved from "somewhere" to part of the windows installer
packaging, which happens after the zip and mar are generated. Since the
installer we ship is actually repackaged from the zip[1], we stopped shipping
translated uninstallers.
This changes things around so that the uninstaller gets translated:
- Explicitly build the uninstaller as part of the L10n repack step.
- Use the same logic to build the installer locally as we do to create the ones
we ship.
[1] Except on Thunderbird
Differential Revision: https://phabricator.services.mozilla.com/D672
--HG--
extra : rebase_source : 05fe935c1d2a9fbfeef786819bfe5913ed8ef862
extra : source : d6bf22099e2195dcb64c3c3d7700d3edd0850a3a
The uninstaller was being built as a side-effect of building `setup.exe`. In
Bug 1385227, that was moved from "somewhere" to part of the windows installer
packaging, which happens after the zip and mar are generated. Since the
installer we ship is actually repackaged from the zip[1], we stopped shipping
translated uninstallers.
This changes things around so that the uninstaller gets translated:
- Explicitly build the uninstaller as part of the L10n repack step.
- Use the same logic to build the installer locally as we do to create the ones
we ship.
[1] Except on Thunderbird
Differential Revision: https://phabricator.services.mozilla.com/D672
--HG--
extra : rebase_source : 2b28b9ff7196d12f4a188c8dddf750b9a5efac5b
extra : histedit_source : 9bc28891950ae8c226cfdefef6f8121ce0b51f58
We use a wrapper script when compiling with MSVC to parse the
/showIncludes output and thereby generate a Makefile dependency
fragment. This fragment enables us to do correct and faster incremental
builds. But the cost of invoking the wrapper script can be significant;
it's an extra process or two to launch for every single compilation.
Instead, let's have clang-cl generate the dependencies directly, which
should be somewhat faster.
There are a lot of choices and moving pieces in this commit. I elected
to include the mechanics and the target use case in the same commit so
that readers can compare and contrast the implementation and final
expression in one review window.
- Initially, I wanted to make the {AB_CD} substitutions in
LOCALIZED_FILES and not in LOCALIZED_GENERATED_FILES. However, I ran
into conceptual blockers doing this. Fundamentally, LOCALIZED_FILES
is FINAL_TARGET_FILES, and my use case should _not_ be putting files
anywhere near dist/bin. In addition, LOCALIZED_FILES
(FINAL_TARGET_FILES) is handled using manifests, which would need to
grow locale-aware functionality to handle this. That's not desirable.
In addition, if we use manifests, then we lose the powerful locality
of |mach build mobile/android{/base}| re-generating changed
locale-dependent resources. This is similar to how the build system
plumbs dist/idl manifest processing throughout the build: we're
repairing local workflows after moving work into a global process.
For these reasons, this doesn't support {AB_CD} in LOCALIZED_FILES.
- There is even another layer of complexity! There are two axes
involved with these files: AB_CD controls localization and the Make
target controls destination. For the record, it is:
regular builds - AB_CD unset
multi-locale builds - AB_CD set
single-locale repacks - AB_CD set
For the record, the existing logic (before any changes) is:
regular builds - Make target is `libs` in mobile/android/base/locales
multi-locale builds - Make target is `chrome-%` in mobile/android/base/locales
single-locale repacks - Make target is `libs` in mobile/android/base/locales
This commit adds targets for both destinations, and uses Make
chrome-%:: and libs:: magic to control what is invoked in the various
situations. Tricky!
- I added MERGE_RELATIVE_FILES in order to be able to follow-up this
patch with more patches that will get rid of
m/a/base/locales/{moz.build,Makefile.in} altogether, and fold this work
into m/a/base. As it stands, we're already reaching from
m/a/base/locales all the way out to
mobile/locales/.../region.properties, so the existing code doesn't
follow the layout expected between mozilla-central and
l10n-central/$(AB_CD). But that'll impedance will get worse as we
improve the build system dependencies, not better, so we should grow
support for localized resources that aren't exactly as expected.
- I chose to follow Python's syntax for string substitutions. I
would have preferred to mark files that should be localized with a
leading '%'... but I took that for filesystem absolute paths in
moz.build files already. I also considered @AB_CD@ to echo the
preprocessor, but didn't want to open the door to an expecation that
_all_ preprocessor DEFINEs will work in the way {AB_CD} does.
- The generate_*py script changes required a bit of a hack to "turn
off" locale dependent resources. This would have been nicer if we had
marked localized resources with '%'... but we didn't. See the
--fallback flag. The real reason this is needed is that we're doing
work which is more like the work of compare-locales (merging
locale-dependent resources) at build-time rather than repack time. I
don't know why that's the case -- probably when we (I) implemented it,
compare-locales and the whole l10n process was entirely opaque. It's
not worth changing it now, so we use this --fallback flag approach.
- I didn't get to tup support. This should gently fail without
breaking tup builds: any {AB_CD} substitutions just won't be
expanded. I haven't a clue how this should work in tup in the future
(or, more generally, how to make any sense of repacks without
declaring the full set of expected locales at configure time.)
- strings.xml can't be a LOCALIZED_PP_FILES, since we need to
customize the output location based on AB_rCD, and since we need a
little more flexibility than PP_FILES gives for our inputs.
MozReview-Commit-ID: MyfIkNSEzt
--HG--
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/en-US/localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/en-US/localized-input
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/foo-data => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/foo-data
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/generate-foo.py => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/generate-foo.py
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/en-US/localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/inner/locales/en-US/localized-input
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/moz.build => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/moz.build
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/non-localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/non-localized-input
extra : rebase_source : 816b6f220758f2bb3bdd3ec81a2cb02269c6de5b
I wanted to lift this next to the definition of AB_CD, but that
doesn't allow to use it in a backend.mk file, due to the order in
which Makefile, config.mk, rules.mk, and backend.mk are processed.
Therefore, I've put it in a tiny include file, so that it can be used
by a Makefile and a backend.mk file.
This allows the `RecursiveMake` backend to owning defining AB_rCD in
backend.mk files, while not requiring consumers to arrange for AB_rCD
in a sibling Makefile.in file.
Other build backends will need to arrange for AB_rCD themselves: see
following commits.
MozReview-Commit-ID: I7GIzRbCCtf
--HG--
extra : rebase_source : 3277fedb43bc3d8007287c223554a085dae2f198
extra : source : 854c0f43a1f74b4e22aa7638b407580240c90dd5
I very much doubt these are used, but even if we are -- we shouldn't
support this type of local customization, since it doesn't extend to
non-Make-based backends.
With the customization point removed, there's no way to set ETAGS, so
we remove what little support there was for generating Emacs tags.
MozReview-Commit-ID: IEF2Q4tISEn
--HG--
extra : rebase_source : 3bc8e651c03517edb797032db6ce60ed8852d9fa
