Previously, some tools such as `clang` or `lld` which require strict
order for certain command-line options, such as `clang -cc1` or `lld
-flavor`, would not longer work on Windows, when these tools were linked
as part of `llvm-driver`. This was caused by `InitLLVM` which was part
of the `*_main()` function of these tools, which in turn calls
`windows::GetCommandLineArguments`. That function completly replaces
argc/argv by new UTF-8 contents, so any ajustements to argc/argv made by
`llvm-driver` prior to calling these tools was reset.
`InitLLVM` is now called by the `llvm-driver`. Any tool that
participates in (or is part of) the `llvm-driver` doesn't call
`InitLLVM` anymore.
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.
I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
Make it easier to control which optimizations are enabled by making
OptimizeArgs a bit masked enum. There's currently only one such
optimization, but more will be added in followup commits.
This patch makes the generation of command lines for modular
dependencies lazy/on-demand. That operation is somewhat expensive and
prior to this patch used to be performed multiple times for the
identical `ModuleDeps` (i.e. when they were imported from multiple
different TUs).
D152495 makes clang warn on unused variables that are declared in conditions like `if (int var = init) {}`
This patch is an NFC fix to suppress the new warning in llvm,clang,lld builds to pass CI in the above patch.
Differential Revision: https://reviews.llvm.org/D158016
All command-line tools using `llvm::opt` create an enum of option IDs and a table of `OptTable::Info` object. Most of the tools use the same ID (`OPT_##ID`), kind (`Option::KIND##Class`), group ID (`OPT_##GROUP`) and alias ID (`OPT_##ALIAS`). This patch extracts that common code into canonical macros. This results in fewer changes when tweaking the `OPTION` macros emitted by the TableGen backend.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D157028
Scanner's C++ API accepts a set of modular dependencies the client has already seen and for which it doesn't need the full details. This is currently a set of strings, which somewhat implies that it should contain the set of module names. However, scanner internally expects the values to be in the format "{hash}{name}". Besides not being documented, this is very unintuitive. This patch makes this expectation explicit by changing the type to set of `ModuleID`.
Reviewed By: benlangmuir
Differential Revision: https://reviews.llvm.org/D156492
I once again stumbled across `IndexedModuleID` and was confused by the mutable `InputIndex` field.
This patch adds comments around that piece of code and unifies/simplifies related functions.
Reviewed By: Bigcheese
Differential Revision: https://reviews.llvm.org/D145197
This patch adds new `-print-timing` option to `clang-scan-deps`. It measures the wall and process time taken to scan dependencies for the compilation database. This provides more representative data compared to measuring the timing for the whole tool invocation, since that includes parsing and generating JSON files, which can be significant for larger inputs.
Reviewed By: akyrtzi
Differential Revision: https://reviews.llvm.org/D147815
This patch simplifies the loop inside each worker by extracting index retrieval into a lambda function.
Reviewed By: benlangmuir
Differential Revision: https://reviews.llvm.org/D145101
This patch makes sure the ordering of TUs in the input CDB is preserved in the full output. Previously, the results were pushed into a global vector, potentially out-of-order and then sorted by the input file name. This is non-deterministic when the CDB contains multiple entries with the same input file. This patch fixes that by pre-allocating the output vector and writing directly to the position corresponding to the current input. This also eliminates one critical section.
Reviewed By: benlangmuir
Differential Revision: https://reviews.llvm.org/D145098
Required in https://reviews.llvm.org/D137534.
The build systems needs the information to know that "header X changed,
scanning may have changed, so please rerun scanning". Although it is
possible to get the information by running clang-scan-deps for the
second time with make format, it is not user friendly clearly.
Reviewed By: jansvoboda11
Differential Revision: https://reviews.llvm.org/D139168
In a private chat with @ben.boeckel , we get in consensus it would be
great for cmake if the invocation of clang-scan-deps can get rid of
compilation database. Due to the compilation database can't do very well
for the files which are not existed yet. @ben.boeckel may have more
context to add here.
This patch should be innocent for others usages.
Reviewed By: jansvoboda11
Differential Revision: https://reviews.llvm.org/D137534
The forwarding header is left in place because of its use in
`polly/lib/External/isl/interface/extract_interface.cc`, but I have
added a GCC warning about the fact it is deprecated, because it is used
in `isl` from where it is included by Polly.
This reverts commit e1354763b6e641e45cc2371270883fcd26edf705.
There is a build failure in m68k-linux testing bot
(https://lab.llvm.org/buildbot/#/builders/192/builds/267), which is
weird. Revert this for now and look at the reasons.
Required in https://reviews.llvm.org/D137534.
The build systems needs the information to know that "header X changed,
scanning may have changed, so please rerun scanning". Although it is
possible to get the information by running clang-scan-deps for the
second time with make format, it is not user friendly clearly.
Reviewed By: jansvoboda11
Differential Revision: https://reviews.llvm.org/D139168
In a private chat with @ben.boeckel , we get in consensus it would be
great for cmake if the invocation of clang-scan-deps can get rid of
compilation database. Due to the compilation database can't do very well
for the files which are not existed yet. @ben.boeckel may have more
context to add here.
This patch should be innocent for others usages.
Reviewed By: jansvoboda11
Differential Revision: https://reviews.llvm.org/D137534
When reseting modular options, propagate the values from certain options
that have ImpliedBy relations instead of setting to the default. Also,
verify in clang-scan-deps that the command line produced round trips
exactly.
Ideally we would automatically derive the set of options that need this
kind of propagation, but for now there aren't very many impacted.
rdar://105148590
Differential Revision: https://reviews.llvm.org/D143446
D140176 introduced new `FullDeps` API that's not thread-safe, breaking the class invariant. This was causing race condition when `clang-scan-deps` was run with multiple threads.
Reviewed By: steven_wu, akyrtzi
Differential Revision: https://reviews.llvm.org/D143428
For users of the C++ API, the return type of `getFullDependencies` doesn't make sense when asking for dependencies of a module. In the returned `FullDependenciesResult` instance, only `DiscoveredModules` is useful (the graph of modular dependecies). The `FullDeps` member is trying to describe a translation unit it was never given. Its command line also refers to a file in the in-memory VFS we create in the scanner, leaking the implementation detail.
This patch splits the API and improves layering and naming of the return types.
Depends on D140175.
Reviewed By: artemcm
Differential Revision: https://reviews.llvm.org/D140176
This API is no longer necessary, so let's remove it to simplify the internal APIs.
Reviewed By: benlangmuir, artemcm
Differential Revision: https://reviews.llvm.org/D140175
This patch removes the ability of a dependency scanning worker to share a `FileManager` instance between individual scans. It's not sound and doesn't provide performance benefits (due to the underlying caching VFS).
Reviewed By: benlangmuir
Differential Revision: https://reviews.llvm.org/D134976
Attempt to fix the test failures observed in CI:
* Add Option dependency, which caused BUILD_SHARED_LIBS builds to fail
* Adapt tests that accidentally depended on the host platform: platforms
that don't use an integrated assembler (e.g. AIX) get a different set
of commands from the driver. Most dependency scanner tests can use
-fsyntax-only or -E instead of -c to avoid this, and in the rare case
we want to check -c specifically, set an explicit target so the
behaviour is independent of the host.
Original commit message follows.
---
Instead of trying to "fix" the original driver invocation by appending
arguments to it, split it into multiple commands, and for each -cc1
command use a CompilerInvocation to give precise control over the
invocation.
This change should make it easier to (in the future) canonicalize the
command-line (e.g. to improve hits in something like ccache), apply
optimizations, or start supporting multi-arch builds, which would
require different modules for each arch.
In the long run it may make sense to treat the TU commands as a
dependency graph, each with their own dependencies on modules or earlier
TU commands, but for now they are simply a list that is executed in
order, and the dependencies are simply duplicated. Since we currently
only support single-arch builds, there is no parallelism available in
the execution.
Differential Revision: https://reviews.llvm.org/D132405
Instead of trying to "fix" the original driver invocation by appending
arguments to it, split it into multiple commands, and for each -cc1
command use a CompilerInvocation to give precise control over the
invocation.
This change should make it easier to (in the future) canonicalize the
command-line (e.g. to improve hits in something like ccache), apply
optimizations, or start supporting multi-arch builds, which would
require different modules for each arch.
In the long run it may make sense to treat the TU commands as a
dependency graph, each with their own dependencies on modules or earlier
TU commands, but for now they are simply a list that is executed in
order, and the dependencies are simply duplicated. Since we currently
only support single-arch builds, there is no parallelism available in
the execution.
Differential Revision: https://reviews.llvm.org/D132405
The invocation is only ever used to serialize cc1 arguments from, so
instead serialize the arguments inside the dep scanner to simplify the
interface.
Differential Revision: https://reviews.llvm.org/D132616
This patch introduces new option `-eager-load-pcm` to `clang-scan-deps`, which controls whether the resulting command-lines will load PCM files eagerly (at the start of compilation) or lazily (when handling import directive). This patch also switches the default from eager to lazy.
To reduce the potential for churn in LIT tests in the future, this patch also removes redundant checks of command-line arguments and introduces new test `modules-dep-args.c` as a substitute.
Reviewed By: benlangmuir
Differential Revision: https://reviews.llvm.org/D132066
Instead of delaying the generation of command-lines to after all
dependencies are reported, compute them immediately. This is partly in
preparation for splitting the TU driver command into its constituent cc1
and other jobs, but it also just simplifies working with the compiler
invocation for modules if they are not "without paths".
Also change the computation of the default output path in
clang-scan-deps to scrape the implicit module cache from the
command-line rather than get it from the dependency, since that is now
unavailable at the time we make the callback.
Differential Revision: https://reviews.llvm.org/D131934
Since D129389 (and downstream PR https://github.com/apple/llvm-project/pull/4965), the dependency scanner is responsible for generating full command-lines, including the modules paths. This patch removes the flag that was making this an opt-in behavior in clang-scan-deps.
Reviewed By: benlangmuir
Differential Revision: https://reviews.llvm.org/D131420
Follow-up to 6626f6fec3d3, this fixes the handling of -MT
* If no targets are provided, we need to invent one since cc1 expects
the driver to have handled it. The default is to use -o, quoting as
necessary for a make target.
* Fix the splitting for empty string, which was incorrectly treated as
{""} instead of {}.
* Add a way to test this behaviour in clang-scan-deps.
Differential Revision: https://reviews.llvm.org/D129607
When building modules, override secondary outputs (dependency file,
dependency targets, serialized diagnostic file) in addition to the pcm
file path. This avoids inheriting per-TU command-line options that
cause non-determinism in the results (non-deterministic command-line for
the module build, non-determinism in which TU's .diag and .d files will
contain the module outputs). In clang-scan-deps we infer whether to
generate dependency or serialized diagnostic files based on an original
command-line. In a real build system this should be modeled explicitly.
Differential Revision: https://reviews.llvm.org/D129389
This is first of a series of patches for making the special lexing for dependency scanning a first-class feature of the `Preprocessor` and `Lexer`.
This patch only includes NFC renaming changes to make reviewing of the functionality changing parts easier.
Differential Revision: https://reviews.llvm.org/D125484
This is to improve maintenance a bit and remove need to maintain the additional option and related code-paths.
Differential Revision: https://reviews.llvm.org/D124558
With explicit modules build, the '-fmodules-cache-path=' argument is unused.
This patch removes the argument to avoid warnings or errors (with '-Werror') stemming from that.
Depends on D118915.
Reviewed By: dexonsmith
Differential Revision: https://reviews.llvm.org/D120474
The `clang-scan-deps` tool currently generates `-fmodule-file=` command-line arguments for the whole transitive closure of modular dependencies. This is not necessary, we only need to provide the direct dependencies on the command line. Information about transitive dependencies is stored within the `.pcm` files of direct dependencies. This makes the command lines shorter, but should be a NFC otherwise (unless there are bugs in the loading mechanism for explicit modules).
Depends on D120465.
Reviewed By: Bigcheese
Differential Revision: https://reviews.llvm.org/D118915
