XPIDL files are logically grouped together into a module, but the
current model for the moz.build frontend is that we emit individual
XPIDLFile objects, and leave it to the backend to reconstruct
module-ness from those. This setup causes a small amout of useless work
to be done (e.g. see XPIDLFile handling in
RecursiveMakeBackend.consume_object; such handling should only be done
once), and it would be cleaner to have the objects emitted reflect the
build system concepts as closely as possible. To that end, let's emit
XPIDLModule objects instead, which fortunately requires relatively few
changes.
The IDLManager in the moz.build backend is a bit weird. It maintains a
bunch of per-IDL file information, some of which isn't used, and
attempts to map module names to information about what entities the
module needs to be built. The former is done with Python dicts, and the
latter with Python tuples, both resulting in some contortions by the
clients of IDLManager to specify exactly what they need.
Let's clean this up, by making IDLManager to more clearly do two jobs:
1. Keep track of whether IDL files are globally unique; and
2. Map module names to the information needed to build them.
In the case of #2, we store everything as a straight Python object, so
we can use actual property accesses everywhere. We also provide a
stems() function on IDLManager to make some client code more
straightforward.
Doing this makes IDLManager much more XPIDL module-centric, and paves
the way for the same change to be made in the frontend as well.
XPIDL_SOURCES would benefit from being more explicit about what sort of
values it contains; let's define it as a list of SourcePath objects, and
propagate those objects into XPIDLFile frontend objects as well.
Co-authored by Nick Alexander <nalexander@mozilla.com>
This patch adds support to the fastermake backend for taking localized files
from a locale directory as specified by the combination of --with-l10n-base
and --enable-ui-locale. This allows artifact builds to be built localized
with a different locale.
MozReview-Commit-ID: 1bD9Gy0ewJ2
--HG--
extra : rebase_source : 7e7d52142e48c0e57ead11df4d3d721a3b10b3e0
XPIDL files are logically grouped together into a module, but the
current model for the moz.build frontend is that we emit individual
XPIDLFile objects, and leave it to the backend to reconstruct
module-ness from those. This setup causes a small amout of useless work
to be done (e.g. see XPIDLFile handling in
RecursiveMakeBackend.consume_object; such handling should only be done
once), and it would be cleaner to have the objects emitted reflect the
build system concepts as closely as possible. To that end, let's emit
XPIDLModule objects instead, which fortunately requires relatively few
changes.
The IDLManager in the moz.build backend is a bit weird. It maintains a
bunch of per-IDL file information, some of which isn't used, and
attempts to map module names to information about what entities the
module needs to be built. The former is done with Python dicts, and the
latter with Python tuples, both resulting in some contortions by the
clients of IDLManager to specify exactly what they need.
Let's clean this up, by making IDLManager to more clearly do two jobs:
1. Keep track of whether IDL files are globally unique; and
2. Map module names to the information needed to build them.
In the case of #2, we store everything as a straight Python object, so
we can use actual property accesses everywhere. We also provide a
stems() function on IDLManager to make some client code more
straightforward.
Doing this makes IDLManager much more XPIDL module-centric, and paves
the way for the same change to be made in the frontend as well.
XPIDL_SOURCES would benefit from being more explicit about what sort of
values it contains; let's define it as a list of SourcePath objects, and
propagate those objects into XPIDLFile frontend objects as well.
Co-authored by Nick Alexander <nalexander@mozilla.com>
This patch adds support to the fastermake backend for taking localized files
from a locale directory as specified by the combination of --with-l10n-base
and --enable-ui-locale. This allows artifact builds to be built localized
with a different locale.
MozReview-Commit-ID: 1bD9Gy0ewJ2
--HG--
extra : rebase_source : 9f457c8565ff0297589533ab91b44c16edc86be3
We shouldn't call _get_backend_file during TupBackend:_init() since
self.dry_run is not set yet. Instead we can just call it when we need to
add rust rules.
MozReview-Commit-ID: 5elqXGFMSuG
--HG--
extra : rebase_source : 880a754f631f691aaf85bfda610e29b7bfcb89b4
a35b188d0e44 inadvertently regressed behavior in the case where
the Git status.showUntrackedFiles config option was set and
we want to purge untracked files.
Differential Revision: https://phabricator.services.mozilla.com/D2141
--HG--
extra : moz-landing-system : lando
The doc string for the "working_directory_clean" function states:
> By default, untracked and ignored files are not considered.
But the git implementation for this function used to run:
git status --porcelain
Which *does* consider untracked files by default. Instead, we need to run:
git status --porcelain --untracked-files=no
Differential Revision: https://phabricator.services.mozilla.com/D2134
--HG--
extra : moz-landing-system : lando
This patch uses the PIPENV_PYTHON environment variable to append a suffix to the created virtual environment path according to the version specified. It also uses the PIPENV_DEFAULT_PYTHON_VERSION environment variable to avoid recreating the virtual environment every time. With these changes we are able to switch back and forth between Python versions without the expense of recreating environments, however there is a risk of these environments becoming stale. In this scenario it may be necessary to clobber the virtual environment root within the obj dir.
MozReview-Commit-ID: C4vuwNh04CP
--HG--
extra : rebase_source : 301c8418fe5b873ffadec37af11e98cfda8667b8
Most HG commands use subprocesses, even if a context manager (and therefore an
hglib client) has been created. There are only two commands that make use of
the client, but they *only* work inside a context manager. I don't think
there are any technical reason these two commands *need* to use the context
manager.
This patch merges the HgRepository._run_in_client function with
HgRepository._run(). If a client exists, that will be used, otherwise a
subprocess will be used.
Differential Revision: https://phabricator.services.mozilla.com/D1809
--HG--
extra : moz-landing-system : lando
This patch uses the PIPENV_PYTHON environment variable to append a suffix to the created virtual environment path according to the version specified. It also uses the PIPENV_DEFAULT_PYTHON_VERSION environment variable to avoid recreating the virtual environment every time. With these changes we are able to switch back and forth between Python versions without the expense of recreating environments, however there is a risk of these environments becoming stale. In this scenario it may be necessary to clobber the virtual environment root within the obj dir.
MozReview-Commit-ID: C4vuwNh04CP
--HG--
extra : rebase_source : ba34e415fa21683bc2a39231651587fa30ad0242
For clang-cl, we want to add code to libxul that only exists during the
PGO generation phase, so we can collect data. The most expedient way to
do that is to enable certain files in SOURCES to be marked as to only be
compiled during the PGO generation step.
This adds everything to a default output group so we can run `tup upd $objdir/<default>`
during `mach build` and `tup upd $objdir/<gtest>` for instance when running `mach gtest`
as a way to conditionally build parts of the tree.
MozReview-Commit-ID: 8jpkru1EgAC
--HG--
extra : rebase_source : 5a2c4fd0e0b1b62f8bd65dd1a23ba2086ba63eeb
Most HG commands use subprocesses, even if a context manager (and therefore an
hglib client) has been created. There are only two commands that make use of
the client, but they *only* work inside a context manager. I don't think
there are any technical reason these two commands *need* to use the context
manager.
This patch merges the HgRepository._run_in_client function with
HgRepository._run(). If a client exists, that will be used, otherwise a
subprocess will be used.
Differential Revision: https://phabricator.services.mozilla.com/D1809
--HG--
extra : moz-landing-system : lando
This adds just enough host shared library support for this one use case,
but also takes shortcuts, because fully supporting host shared library
is a deep rabbit hole I'm not ready to take just to fix --enable-lto
--enable-clang-plugin on mac builds.
One downside is that one my machine the plugin now takes > 80s to build,
instead of 15s before, thanks to the lack of unified sources.
--HG--
extra : rebase_source : bf52a72a01d4e3eb77cf52b646b19734b9273075
The test checks the modules imported, which means it will fail if we import mozbuild.base to determine the topsrcdir. Rather than include the additional modules, which may change in the future, I have narrowed the scope of the test to the mozbuild/tests module.
MozReview-Commit-ID: GiaPzp19adE
--HG--
extra : rebase_source : 9d79ad79bef2b45e3f62190200bc46f21d57f90d
Because we have no linking configure test in python configure (yet), we
just make old-configure tests use LINKER_LDFLAGS, and make those flags
added to LDFLAGS by old-configure at the same time.
--HG--
extra : rebase_source : 80ab7c5021c7ddd1b53d58ef76cd4372a524d3cb
vendor_rust.py checks all dirs in third_party/rust/ for license information, but it wasn't distinguishing between subdirs and files when listing the contents of that directory, and the script earlier causes macOS to create a .DS_Store file in the directory, which the script then treats as a subdir, causing an IOError "not a directory". This change excludes the .DS_Store file (and other non-dirs) from the list of subdirs of third_party/rust/ to check for license info.
MozReview-Commit-ID: 4CrJH9tZLnG
--HG--
extra : rebase_source : 464ad17f4c9b76303f9024d8eaad475b1fc248f7
Added `./mach python-safety`, distinct from python-test so it doesn't have
to be run on every CI job - its errors may not depend on the area the push has changed.
Added the python/safety directory to ensure a different Pipfile is used, avoiding
conflicts with python-test.
Differential Revision: https://phabricator.services.mozilla.com/D1825
--HG--
extra : moz-landing-system : lando
We're well overdue for an upgrade of the rust compiler requirements.
Now that we're building with 1.28 (albeit a beta, due to be bumped when
it's released), we can bump the requirement away from 1.24 which is now
old. 1.27 is too new, though, so settle for the older 1.26.
--HG--
extra : rebase_source : a17aa496bf3d4af4d1349d69a637c686c6817d0f
This will make sure that when running |mach python-test --python 3| locally,
we only run the tests that also run in CI with python 3 (and therefore pass
presumably).
MozReview-Commit-ID: 3OBr9yLSlSq
--HG--
extra : rebase_source : 456340d0ecdddf1078f2b5b4ebb1eddf3813b26a
By making the archive URL dynamic, we can fetch an old version of the
bootstrap files. This will make it easier to test the bootstrapper in
CI.
Differential Revision: https://phabricator.services.mozilla.com/D1698
--HG--
extra : rebase_source : 9ba582cf3c138dba433e2bb354650f14b3f16aa7
extra : amend_source : 8a515bb755187e7f0d87b90a25a99f3803ea9e0f
extra : source : 1dcd43dd2a7b04e2bb714349033a456ea5158f3e
The previously listed server wasn't working. This has likely been
broken for years (I initially authored this commit in November 2016).
Differential Revision: https://phabricator.services.mozilla.com/D1697
We're well overdue for an upgrade of the rust compiler requirements.
Now that we're building with 1.28 (albeit a beta, due to be bumped when
it's released), we can bump the requirement away from 1.24 which is now
old. 1.27 is too new, though, so settle for the older 1.26.
--HG--
extra : rebase_source : c788ef4f7da9949b81df2f0577af6f6039ea63d8
Overall, this makes the whole setup less fragile, and make it work with
LTO in more situations.
--HG--
extra : rebase_source : de968c61dc4ef337fdc28745c202334ac41763cd
We perform, on the binaries we build, a series of check, that are
implemented as half-baked make commands, invoked after linking them.
- check libstdc++ symbol versions to ensure binary compatibility with
a baseline.
- check glibc symbol versions to ensure binary compatibility with a
baseline.
- check that target binaries don't contain text relocations.
- check that libmozglue is linked before libc on android.
- on libxul, check that NSModules are laid out correctly.
- on libxul, check that there is more than one PT_LOAD segment.
Those checks happen to work where they matter, but their setup is
unreliable. For example, the checks for symbol versions are supposed to
work for libclang-plugin on cross osx builds, but in fact, don't,
because the readelf path doesn't exist, and the command doesn't fail in
that case.
So move them all to a standalone script, performing the checks more
thoroughly (especially the NSModules one, where we now also check that
they are all adjacent), and more verbosely.
--HG--
extra : rebase_source : 7072e622e95f363d4a6c3a8e272d3445d998b592
yaml.load() is unsafe and can lead to arbitrary code execution via
syntax like `!!python/object/apply:os.system`. yaml.safe_load() is
more reasonable.
Differential Revision: https://phabricator.services.mozilla.com/D1738
--HG--
extra : rebase_source : 597c07b3c1538dc27ad6f46e01cdb7f48755d0bc
extra : histedit_source : 131d570f8ac1ee047487cba54822dbf20abf6681
The bitstream is frozen and we're updating to v1.0.0. There is no longer any need
to indicate which revision we're using in the mimetype.
--HG--
extra : rebase_source : 5f5bf8649bd21610ebf04661e8f80bacbb69ca09