Some GN actions pass large lists of input files through a response file
rather than directly on the command line. We need to support this in
order to build ANGLE with moz.build files generated by gn_processor.py.
The only case we currently need to support is ANGLE's
angle_program_version_id action. The response_file_contents field in
this case is a list of relative paths from the GN gen dir to files in
the source tree, but the action is executed at build time with a cwd in
the objdir. At vendor time, gn_processor.py cannot know the eventual
source directory, object directory, or relative path between the two.
To handle this, gn_processor.py encodes those paths into a vendored
.rsp.in sidecar file using mozbuild-style path prefixes. At build time,
a new write_path_list.py action converts the .rsp.in file into a real
response file in the objdir, resolving those prefixes to absolute paths
and shell-quoting them as required by GN's response file format. A
second GeneratedFile invokes the original action via the existing
file_generate_wrapper.py entry point, substituting the
{{response_file_name}} placeholder in the action's args with the path of
the generated response file.
Since response file contents can vary between build configurations,
the .rsp.in filename includes a hash of its contents so that each
distinct variant has a unique name and all can coexist in-tree. Where
the .rsp.in differs between configurations, the moz.build writer emits
the GeneratedFile statements that generate the real response file
conditionally.
The `GeneratedFiles` context attr is now stored as a list so that a GN
action can produce both the response file and the action output. The
stale generated-file cleanup is also updated to retain these extra
sidecar files while still removing stale generated outputs.
Additionally, we must make the raw project.json path rewriting more
specific. The previous broad target_dir replacements could modify paths
in response_file_contents and make them point at files that do not
exist.
Note that a response file can technically contain arbitrary data, but
this currently only supports the case where it is a list of paths. If we
need to support arbitrary response file contents in the future then this
will need to be adapted.
Differential Revision: https://phabricator.services.mozilla.com/D300873
Linker inputs referenced only through `LDFLAGS` are
invisible to the build backend, so make can run the
link before those files exist. Add `EXTRA_LINK_DEPS`
to declare those inputs explicitly, and use it for
the macOS updater plist files.
Differential Revision: https://phabricator.services.mozilla.com/D302162
Some `GeneratedFile` scripts open additional files at runtime
(e.g. #include @TOPOBJDIR@/...), so backends need a way to
declare those as prerequisites without passing them as
positional args.
Also normalizes the `_target_per_file` key to an absolute
path so `GENERATED_FILES` dependencies that cross directory
boundaries actually resolve.
Differential Revision: https://phabricator.services.mozilla.com/D302161
This is part of fixing clangd indexing for out-of-tree object
directories. For example:
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../obj/$(basename $PWD)-no-opt
The problem is that clangd (at least in VSCode) doesn't index files
outside the project root(s). You can add directories as extra project
roots, but it will still fail to index them, since when it walks up the
parent directory chain it won't encounter the .clangd that `mach ide
vscode` creates at the root of the source directory.
This patch does two things:
- When the .clangd file is created in the root of the source
directory, it also copies it to the root of the object directory.
- It protects the .clangd file in the object directory from being
removed during a clobber, since users aren't going to know they'd
need to make a manual intervention to restor it, or to re-run `mach
ide vscode`, which wouldn't be great anyway).
Having a copy of .clangd in the object directory is harmless for in-tree
object directories, so no effort is make to avoid that.
Differential Revision: https://phabricator.services.mozilla.com/D303661
Track include_skip_if (the skip condition inherited from the enclosing
include directive) and tests_skip_if (AND-accumulated from all protocol-URL
test lines in the manifest) for each visited manifest path.
Differential Revision: https://phabricator.services.mozilla.com/D302340
This follows the same pattern as `SharedLibrary` and `RustLibrary`,
letting us exclude a Rust program from the default build and build
it on demand via `./mach build <category>`.
Differential Revision: https://phabricator.services.mozilla.com/D292597
`--maven-zip` was hardcoded to download `target.maven.zip` and nothing
else. Generalize it to `--artifact-filter`, which accepts arbitrary
artifact paths (e.g. `public/build/target.maven.zip`) and can be
specified multiple times. The fat-aar caller switches to the new flag
with no behavioral change.
Differential Revision: https://phabricator.services.mozilla.com/D292608
When vendoring, fetch_and_unpack keeps in-tree moz.build files but then
copy_contents overwrites them with the upstream versions (which in this
case already list the source files).
When these files were first vendored, stub.cpp was "new" from VCS perspective.
update_moz_build then calls add_file_to_moz_build_file for it,
then appends the file even though it already exists.
This also means that when the same vendor re-runs, the files are no longer "new",
so that code doesn't run. As a result, the duplicates are removed on subsequent vendors -
ie, executing the exact same "vendor" command twice ends up with different trees,
which is clearly not the intent.
Differential Revision: https://phabricator.services.mozilla.com/D299026
velzie