Fixing Windows buildbot by using the same separators for `-F` and `-I`
paths both in VFS overlay and on command line.
Fix errors like
> module 'MultiPath' is defined in both 'path/to/modules.cache/3JR48BPRU7BCG/MultiPath-1352QHUF8RNMU.pcm' and 'path/to/modules.cache/3JR48BPRU7BCG/MultiPath-20HNSLLIUDDV1.pcm'
To avoid building extra identical modules `-ivfsoverlay` option is not a
part of the hash like "/3JR48BPRU7BCG/". And it is build system's
responsibility to provide `-ivfsoverlay` options that don't cause
observable differences. We also need to make sure the hash like
"-1352QHUF8RNMU" is not affected by `-ivfsoverlay`. As this hash is
defined by the module map path, use the path prior to any VFS
remappings.
rdar://111921464
Differential Revision: https://reviews.llvm.org/D156749
Fix errors like
> module 'MultiPath' is defined in both 'path/to/modules.cache/3JR48BPRU7BCG/MultiPath-1352QHUF8RNMU.pcm' and 'path/to/modules.cache/3JR48BPRU7BCG/MultiPath-20HNSLLIUDDV1.pcm'
To avoid building extra identical modules `-ivfsoverlay` option is not a
part of the hash like "/3JR48BPRU7BCG/". And it is build system's
responsibility to provide `-ivfsoverlay` options that don't cause
observable differences. We also need to make sure the hash like
"-1352QHUF8RNMU" is not affected by `-ivfsoverlay`. As this hash is
defined by the module map path, use the path prior to any VFS
remappings.
rdar://111921464
Differential Revision: https://reviews.llvm.org/D156749
The current `ASTReader::visitInputFiles()` function calls into `FileManager` to create `FileEntryRef` objects. This ends up being fairly costly in `clang-scan-deps`, where we mostly only care about file paths.
This patch introduces new `ASTReader` API that gives clients access to just the serialized paths. Since the scanner needs both the as-requested path and the on-disk one (and doesn't want to transform the former into the latter via `FileManager`), this patch starts serializing both of them into the PCM file if they differ.
This increases the size of scanning PCMs by 0.1% and speeds up scanning by 5%.
Reviewed By: benlangmuir, vsapsai
Differential Revision: https://reviews.llvm.org/D157066
This reverts commit 0d12683046ca75fb08e285f4622f2af5c82609dc and
reapplies ef9ec4bbcca2fa4f64df47bc426f1d1c59ea47e2 with an extension to
fix the Flang build.
Differential Revision: https://reviews.llvm.org/D156184
CUDA and HIP have kernel attributes to tune the code generation (in the
backend). To reuse this functionality for OpenMP target regions we
introduce the `ompx_attribute` clause that takes these kernel
attributes and emits code as if they had been attached to the kernel
fuction (which is implicitly generated).
To limit the impact, we only support three kernel attributes:
`amdgpu_waves_per_eu`, for AMDGPU
`amdgpu_flat_work_group_size`, for AMDGPU
`launch_bounds`, for NVPTX
The existing implementations of those attributes are used for error
checking and code generation. `ompx_attribute` can be attached to any
executable target region and it can hold more than one kernel attribute.
Differential Revision: https://reviews.llvm.org/D156184
This is a prep patch for avoiding the quadratic number of calls to `HeaderSearch::lookupModule()` in `ASTReader` for each (transitively) loaded PCM file. (Specifically in the context of `clang-scan-deps`).
This patch explicitly serializes `Module::DefinitionLoc` so that we can stop relying on it being filled by the module map parser. This change also required change to the module map parser, where we used the absence of `DefinitionLoc` to determine whether a file came from a PCM file. We also need to make sure we consider the "containing" module map affecting when writing a PCM, so that it's not stripped during serialization, which ensures `DefinitionLoc` still ends up pointing to the correct offset. This is intended to be a NFC change.
Reviewed By: benlangmuir
Differential Revision: https://reviews.llvm.org/D150292
In D114095, `HeaderFileInfo::NumIncludes` was moved into `Preprocessor`. This still makes sense, because we want to track this on the granularity of submodules (D112915, D114173), but the way this information is serialized is not ideal. In `ASTWriter`, the set of included files gets deserialized eagerly, issuing lots of calls to `FileManager::getFile()` for input files the PCM consumer might not be interested in.
This patch makes the information part of the header file info table, taking advantage of its lazy deserialization which typically happens when a file is about to be included.
Reviewed By: benlangmuir
Differential Revision: https://reviews.llvm.org/D155131
This is recommit of 98390ccb80569e8fbb20e6c996b4b8cff87fbec6, reverted
in 82a3969d710f5fb7a2ee4c9afadb648653923fef, because it caused
https://github.com/llvm/llvm-project/issues/63542. Although the problem
described in the issue is independent of the reverted patch, fail of
PCH/late-parsed-instantiations.cpp indeed obseved on PowerPC and is
likely to be caused by wrong serialization of `LateParsedTemplate`
objects. In this patch the serialization is fixed.
Original commit message is below.
Previously function template instantiations occurred with FP options
that were in effect at the end of translation unit. It was a problem
for late template parsing as these FP options were used as attributes of
AST nodes and may result in crash. To fix it FP options are set to the
state of the point of template definition.
Differential Revision: https://reviews.llvm.org/D143241
In `clang-scan-deps` contexts, the number of interesting identifiers in PCM files is fairly low (only macros), while the number of identifiers in the importing instance is high (builtins). Marking the whole identifier table out-of-date triggers lots of benign and expensive calls to `ASTReader::updateOutOfDateIdentifiers()`. (That unfortunately happens even for unused identifiers due to `SemaRef.IdResolver.begin(II)` line in `ASTWriter::WriteASTCore()`.)
This patch makes the main code path more similar to C++ modules, where the PCM files have `INTERESTING_IDENTIFIERS` section which lists identifiers that get created in the identifier table of the importing instance and marked as out-of-date. The only difference is that the main code path doesn't *create* identifiers in the table and relies on the importing instance calling `ASTReader::get()` when creating new identifier on-demand. It only marks existing identifiers as out-of-date.
This speeds up `clang-scan-deps` by 5-10%.
Reviewed By: Bigcheese, benlangmuir
Differential Revision: https://reviews.llvm.org/D151277
When writing a pcm, we serialize diagnostic mappings in order to
accurately reproduce the diagnostic environment inside any headers from
that module. However, the diagnostic state mapping table contains
entries for every diagnostic ID ever accessed, while we only want to
serialize the ones that are actually modified from their default value.
Futher, we need to serialize them in a deterministic order.
rdar://111477511
Differential Revision: https://reviews.llvm.org/D154016
This patch swaps out the `void *` behind `CXFile` from `FileEntry *` to `FileEntryRef::MapEntry *`. This allows us to remove some deprecated uses of `FileEntry::getName()`.
Depends on D151854.
Reviewed By: benlangmuir
Differential Revision: https://reviews.llvm.org/D151938
This patch forbids to write comment to BMIs for C++20 Named Modules.
Originally I thought this was helpful for language services like clangd.
But I found clangd don't want the BMI to contain comments actually. So
it is meaningless for C++20 Named Modules to keep such comments in
their BMI.
It is simple to enable this when someday we found we want this actually.
This mimics the `ModuleMap` API and enables D151854, where the `AllowCreation = true` function needs `FileEntryRef` but `AllowCreation = false` functions is happy with plain `FileEntry`. No functional change intended.
Reviewed By: benlangmuir
Differential Revision: https://reviews.llvm.org/D151853
Platform-specific language extensions often want to provide a way of
indicating that certain functions should be called in a different way,
compiled in a different way, or otherwise treated differently from a
“normal” function. Honoring these indications is often required for
correctness, rather being than an optimization/QoI thing.
If a function declaration has a property P that matters for correctness,
it will be ODR-incompatible with a function that does not have property P.
If a function type has a property P that affects the calling convention,
it will not be two-way compatible with a function type that does not
have property P. These properties therefore affect language semantics.
That in turn means that they cannot be treated as standard [[]]
attributes.
Until now, many of these properties have been specified using GNU-style
attributes instead. GNU attributes have traditionally been more lax
than standard attributes, with many of them having semantic meaning.
Examples include calling conventions and the vector_size attribute.
However, there is a big drawback to using GNU attributes for semantic
information: compilers that don't understand the attributes will
(by default) emit a warning rather than an error. They will go on to
compile the code as though the attributes weren't present, which will
inevitably lead to wrong code in most cases. For users who live
dangerously and disable the warning, this wrong code could even be
generated silently.
A more robust approach would be to specify the properties using
keywords, which older compilers would then reject. Some vendor-specific
extensions have already taken this approach. But traditionally, each
such keyword has been treated as a language extension in its own right.
This has three major drawbacks:
(1) The parsing rules need to be kept up-to-date as the language evolves.
(2) There are often corner cases that similar extensions handle differently.
(3) Each extension requires more custom code than a standard attribute.
The underlying problem for all three is that, unlike for true attributes,
there is no established template that extensions can reuse. The purpose
of this patch series is to try to provide such a template.
One option would have been to pick an existing keyword and do whatever
that keyword does. The problem with that is that most keywords only
apply to specific kinds of types, kinds of decls, etc., and so the
parsing rules are (for good reason) not generally applicable to all
types and decls.
Really, the “only” thing wrong with using standard attributes is that
standard attributes cannot affect semantics. In all other respects
they provide exactly what we need: a well-defined grammar that evolves
with the language, clear rules about what an attribute appertains to,
and so on.
This series therefore adds keyword “attributes” that can appear
exactly where a standard attribute can appear and that appertain
to exactly what a standard attribute would appertain to. The link is
mechanical and no opt-outs or variations are allowed. This should
make the keywords predictable for programmers who are already
familiar with standard attributes.
This does mean that these keywords will be accepted for parsing purposes
in many more places than necessary. Inappropriate uses will then be
diagnosed during semantic analysis. However, the compiler would need
to reject the keywords in those positions whatever happens, and treating
them as ostensible attributes shouldn't be any worse than the alternative.
In some cases it might even be better. For example, SME's
__arm_streaming attribute would make conceptual sense as a statement
attribute, so someone who takes a “try-it-and-see” approach might write:
__arm_streaming { …block-of-code…; }
In fact, we did consider supporting this originally. The reason for
rejecting it was that it was too difficult to implement, rather than
because it didn't make conceptual sense.
One slight disadvantage of the keyword-based approach is that it isn't
possible to use #pragma clang attribute with the keywords. Perhaps we
could add support for that in future, if it turns out to be useful.
For want of a better term, I've called the new attributes "regular"
keyword attributes (in the sense that their parsing is regular wrt
standard attributes), as opposed to "custom" keyword attributes that
have their own parsing rules.
This patch adds the Attr.td support for regular keyword attributes.
Adding an attribute with a RegularKeyword spelling causes tablegen
to define the associated tokens and to record that attributes created
with that syntax are regular keyword attributes rather than custom
keyword attributes.
A follow-on patch contains the main Parse and Sema support,
which is enabled automatically by the Attr.td definition.
Other notes:
* The series does not allow regular keyword attributes to take
arguments, but this could be added in future.
* I wondered about trying to use tablegen for
TypePrinter::printAttributedAfter too, but decided against it.
RegularKeyword is really a spelling-level classification rather
than an attribute-level classification, and in general, an attribute
could have both GNU and RegularKeyword spellings. In contrast,
printAttributedAfter is only given the attribute kind and the type
that results from applying the attribute. AFAIK, it doesn't have
access to the original attribute spelling. This means that some
attribute-specific or type-specific knowledge might be needed
to print the attribute in the best way.
* Generating the tokens automatically from Attr.td means that
pseudo's libgrammar does now depend on tablegen.
* The patch uses the SME __arm_streaming attribute as an example
for testing purposes. The attribute does not do anything at this
stage. Later SME-specific patches will add proper semantics for it,
and add other SME-related keyword attributes.
Differential Revision: https://reviews.llvm.org/D148700
Most users of `Module::Header` already assume its `Entry` is populated. Enforce this assumption in the type system and handle the only case where this is not the case by wrapping the whole struct in `std::optional`. Do the same for `Module::DirectoryName`.
Depends on D151584.
Reviewed By: benlangmuir
Differential Revision: https://reviews.llvm.org/D151586
For modules with umbrellas, we track how they were written in the module map. Unfortunately, the getter for the umbrella directory conflates the "as written" directory and the "effective" directory (either the written one or the parent of the written umbrella header).
This patch makes the distinction between "as written" and "effective" umbrella directories clearer. No functional change intended.
Reviewed By: benlangmuir
Differential Revision: https://reviews.llvm.org/D151581
Close https://github.com/llvm/llvm-project/issues/62796.
Previously, we didn't serialize the evaluated result for VarDecl. This
caused the compilation of template metaprogramming become slower than
expect. This patch fixes the issue.
This is a recommit tested with asan built clang.
Close https://github.com/llvm/llvm-project/issues/62796.
Previously, we didn't serialize the evaluated result for VarDecl. This
caused the compilation of template metaprogramming become slower than
expect. This patch fixes the issue.
A step to address https://github.com/llvm/llvm-project/issues/62707.
It is not user friendly enough to drop the implicitly generated path
directly. Let's emit the warning first and drop it in the next version.
In 9c254184 `ASTWriter` stopped writing identifiers that are not interesting. Taking it a bit further, we don't need to sort the whole identifier table, just the interesting identifiers. This reduces the size of sorted vector from ~10k (including lots of builtins) to 2 (`__VA_ARGS__` and `__VA_OPT__`) in a typical Xcode project, improving `clang-scan-deps` performance.
Reviewed By: benlangmuir
Differential Revision: https://reviews.llvm.org/D150494
Avoid inferring new submodules for headers in ASTWriter's collection of
affecting modulemap files, since we don't want to pick up dependencies
that didn't actually exist during parsing.
rdar://109112624
Differential Revision: https://reviews.llvm.org/D150151
Close https://github.com/llvm/llvm-project/issues/62269
Currently, the compiler will emit errors when we compile C++20 modules
if the referenced files changed or got removed. This is because we reuse
the existing logic from Clang implicit modules. It is helpful for clang
implicit modules since it is implicit and we want to be sure things
don't go wrong. But it is not necessary for C++20 modules. The C++20
modules is explicit and it is build systems' responsibility to maintain
the dependencies. So the check in the compiler side may be an overkill.
Reported by Coverity:
1. Inside "ASTReader.cpp" file, in clang::ASTReader::FindExternalLexicalDecls(clang::DeclContext const *, llvm::function_ref<bool (clang::Decl::Kind)>, llvm::SmallVectorImpl<clang::Decl *> &): Using the auto keyword without an & causes a copy.
auto_causes_copy: Using the auto keyword without an & causes the copy of an object of type pair.
2. Inside "ASTReader.cpp" file, in clang::ASTReader::ReadAST(llvm::StringRef, clang::serialization::ModuleKind, clang::SourceLocation, unsigned int, llvm::SmallVectorImpl<clang::ASTReader::ImportedSubmodule> *): Using the auto keyword without an & causes a copy.
auto_causes_copy: Using the auto keyword without an & causes the copy of an object of type DenseMapPair.
3. Inside "CGOpenMPRuntimeGPU.cpp" file, in clang::CodeGen::CGOpenMPRuntimeGPU::emitGenericVarsEpilog(clang::CodeGen::CodeGenFunction &, bool): Using the auto keyword without an & causes a copy.
auto_causes_copy: Using the auto keyword without an & causes the copy of an object of type pair.
4. Inside "ASTWriter.cpp" file, in clang::ASTWriter::WriteHeaderSearch(clang::HeaderSearch const &): Using the auto keyword without an & causes a copy.
auto_causes_copy: Using the auto keyword without an & causes the copy of an object of type UnresolvedHeaderDirective.
Reviewed By: tahonermann
Differential Revision: https://reviews.llvm.org/D149461
Fix several problems related to serialization causing command line
defines to be reported as being built-in defines:
* When serializing the <built-in> and <command line> files don't
convert them into absolute paths.
* When deserializing SM_SLOC_BUFFER_ENTRY we need to call
setHasLineDirectives in the same way as we do for
SM_SLOC_FILE_ENTRY.
* When created suggested predefines based on the current command line
options we need to add line markers in the same way that
InitializePreprocessor does.
* Adjust a place in clangd where it was implicitly relying on command
line defines being treated as builtin.
Differential Revision: https://reviews.llvm.org/D144651
In file `clang/lib/Basic/Module.cpp` the `Module` class had `submodule_begin()` and `submodule_end()` functions to retrieve corresponding iterators for private vector of Modules. This commit removes mentioned functions, and replaces all of theirs usages with `submodules()` function and range-based for-loops.
Differential Revision: https://reviews.llvm.org/D148954
This reverts commit 67b298f6d82e0b4bb648ac0dabe895e816a77ef1.
We got linker errors with undefined symbols during a compiler release
and tracked it down to this change. I am in the process of understanding
what is happening and getting a reproducer.
Sorry for reverting this again.
I will reopen#61065 until we fix this.
Fixes#61065.
This reverts commit 363c98b2d67986b9766bb1426739970ce6d9a6f3 and relands
db987b9589be1eb604fcb74c85b410469e31485f with fixes from
bc95f27337c7ed77c28e713c855272848f01802a.
The module-related issues surfaced there are fixed in the
previous commit.
This reverts commit bc95f27337c7ed77c28e713c855272848f01802a, originally db987b9589be1eb604fcb74c85b410469e31485f.
clang/test/Modules/pr61065.cppm is retained to make relands show less diff.
There are other module-related issues that were not caught, related to
false positive errors like
"error: no matching constructor for initialization of 'union (anonymous union at ..."
Reopen#61065
This re-lands commit db987b9589be1eb604fcb74c85b410469e31485f,
previously reverted in commit 1e0709167f5edd330889f51bb203c458bdb5e359,
with a fix for a pre-existing bug in how we write constructor names into
lookup tables that was exposed by that change.
As yet, we've been unable to produce a reduced testcase for this bug,
but we understand the problem: if we write out a lookup table for a
class which has two constructor names, and we only have local lookup
results for one of those constructor names, then we will write out a
lookup table where the "all the constructors" lookup result contains
results for only one of the constructor names.
Fix a bug in the MODULE_DIRECTORY serialization logic
that would cause MODULE_DIRECTORY to be serialized when
`-fmodule-file-home-is-cwd` is specified.
This matches the original logic added in:
f7b41371d9
Reviewed By: keith
Differential Revision: https://reviews.llvm.org/D147561
Previously, distinct lambdas would get merged, and multiple definitions
of the same lambda would not get merged, because we attempted to
identify lambdas by their ordinal position within their lexical
DeclContext. This failed for lambdas within namespace-scope variables
and within variable templates, where the lexical position in the context
containing the variable didn't uniquely identify the lambda.
In this patch, we instead identify lambda closure types by index within
their context declaration, which does uniquely identify them in a way
that's consistent across modules.
This change causes a deserialization cycle between the type of a
variable with deduced type and a lambda appearing as the initializer of
the variable -- reading the variable's type requires reading and merging
the lambda, and reading the lambda requires reading and merging the
variable. This is addressed by deferring loading the deduced type of a
variable until after we finish recursive deserialization.
This also exposes a pre-existing subtle issue where loading a
variable declaration would trigger immediate loading of its initializer,
which could recursively refer back to properties of the variable. This
particularly causes problems if the initializer contains a
lambda-expression, but can be problematic in general. That is addressed
by switching to lazily loading the initializers of variables rather than
always loading them with the variable declaration. As well as fixing a
deserialization cycle, that should improve laziness of deserialization
in general.
LambdaDefinitionData had 63 spare bits in it, presumably caused by an
off-by-one-error in some previous change. This change claims 32 of those bits
as a counter for the lambda within its context. We could probably move the
numbering to separate storage, like we do for the device-side mangling number,
to optimize the likely-common case where all three numbers (host-side mangling
number, device-side mangling number, and index within the context declaration)
are zero, but that's not done in this change.
Fixes#60985.
Reviewed By: #clang-language-wg, aaron.ballman
Differential Revision: https://reviews.llvm.org/D145737
We need to be able to distinguish individual TUs from the same module in cases
where TU-local entities either need to be hidden (or, for some cases of ADL in
template instantiation, need to be detected as exposures).
This creates a module type for the implementation which implicitly imports its
primary module interface per C++20:
[module.unit/8] 'A module-declaration that contains neither an export-keyword
nor a module-partition implicitly imports the primary module interface unit of
the module as if by a module-import-declaration.
Implementation modules are never serialized (-emit-module-interface for an
implementation unit is diagnosed and rejected).
Differential Revision: https://reviews.llvm.org/D126959
This reverts commit c6e9823724ef6bdfee262289ee34d162db436af0.
Reason: Broke the ASan buildbots, see https://reviews.llvm.org/D126959
(the original phabricator review) for more info.
We need to be able to distinguish individual TUs from the same module in cases
where TU-local entities either need to be hidden (or, for some cases of ADL in
template instantiation, need to be detected as exposures).
This creates a module type for the implementation which implicitly imports its
primary module interface per C++20:
[module.unit/8] 'A module-declaration that contains neither an export-keyword
nor a module-partition implicitly imports the primary module interface unit of
the module as if by a module-import-declaration.
Implementation modules are never serialized (-emit-module-interface for an
implementation unit is diagnosed and rejected).
Differential Revision: https://reviews.llvm.org/D126959
Close https://github.com/llvm/llvm-project/issues/61065.
We will avoid writing the names from external AST naturally. But
currently its check is often false positive since we may have already
marked the declarations as external but
DeclContext::hasNeedToReconcileExternalVisibleStorage would be false
after reconciling.
Tested with libcxx's modular build.
This patch can improve 8% compilation time in an internal workloads.
See the discussion in
https://reviews.llvm.org/rG1e0709167f5edd330889f51bb203c458bdb5e359
to see the information for recommitting.
Fix several problems related to serialization causing command line
defines to be reported as being built-in defines:
* When serializing the <built-in> and <command line> files don't
convert them into absolute paths.
* When deserializing SM_SLOC_BUFFER_ENTRY we need to call
setHasLineDirectives in the same way as we do for
SM_SLOC_FILE_ENTRY.
* When created suggested predefines based on the current command line
options we need to add line markers in the same way that
InitializePreprocessor does.
* Adjust a place in clangd where it was implicitly relying on command
line defines being treated as builtin.
Differential Revision: https://reviews.llvm.org/D144651
This reverts commit db987b9589be1eb604fcb74c85b410469e31485f.
We're seeing failures in modules-enabled builds from within stdlib after
this commit. Errors look like:
In module '...stl_cc_library':
...optional:560:38: error: 'std::__optional_copy_assign_base<unsigned
long>::__optional_copy_assign_base' from module '...optional' is not
present in definition of 'std::__optional_copy_assign_base<unsigned
long>' in module '...optional'
using __optional_move_base<_Tp>::__optional_move_base;
In module '...stl_cc_library':
...optional:771:11: error: no matching constructor for initialization of '__optional_move_assign_base<unsigned long>'
: __base(in_place, _VSTD::forward<_Up>(__v)) {}
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
llvm-project/clang/include/clang/Basic/CodeGenOptions.h:448:57: note: in instantiation of function template specialization 'std::optional<unsigned long>::optional<int, 0>' requested here
std::optional<uint64_t> DiagnosticsHotnessThreshold = 0;
I don't have a self-contained reproducer at this point. I'm hoping that
we may be able to share more information about these issues later, if
necessary.
Close https://github.com/llvm/llvm-project/issues/61065.
We will avoid writing the names from external AST naturally. But
currently its check is often false positive since we may have already
marked the declarations as external but
DeclContext::hasNeedToReconcileExternalVisibleStorage would be false
after reconciling.
Tested with libcxx's modular build.
This patch can improve 8% compilation time in an internal workloads.
Dynamic memory allows users to allocate fast shared memory when a kernel
is launched. We support a single size for all kernels via the
`LIBOMPTARGET_SHARED_MEMORY_SIZE` environment variable but now we can
control it per kernel invocation, hence allow computed values.
Note: Only the nextgen plugins will allocate memory based on the clause,
the old plugins will silently miscompile.
Differential Revision: https://reviews.llvm.org/D141233
