45 Commits

Author SHA1 Message Date
Kazu Hirata
aa8feeefd3 Don't use Optional::hasValue (NFC) 2022-06-25 11:55:57 -07:00
Kazu Hirata
06decd0b41 [clang] Use value_or instead of getValueOr (NFC) 2022-06-18 23:21:34 -07:00
Kazu Hirata
f13019f836 [clang] Use any_of and none_of (NFC) 2022-06-12 10:17:12 -07:00
Daniel Grumberg
504736cedf [clang][extract-api] Don't emit symbols prefixed with an underscore
These symbols are understood to not be used for client API consumption
by convention so they should not appear in the generated symbol graph.

Differential Revision: https://reviews.llvm.org/D125678
2022-05-25 19:02:18 +01:00
Erich Keane
017abbb258 Revert ""Re-apply 4b6c2cd642 "Deferred Concept Instantiation Implementation"""""
This reverts commit a425cac31e2e4cee8e14b7b9a99c8ba17c1ebb52.

There is another libc++ test, that this time causes us to hit an
assertion. Reverting, likely for a while this time.
2022-05-09 09:12:05 -07:00
Erich Keane
a425cac31e "Re-apply 4b6c2cd642 "Deferred Concept Instantiation Implementation""""
This includes a fix for the libc++ issue I ran across with friend
declarations not properly being identified as overloads.

This reverts commit 45c07db31cc76802a1a2e41bed1ce9c1b8198181.
2022-05-09 06:29:47 -07:00
Adrian Kuegel
cc344d262a [clang] Add static_cast to fix Bazel build.
Differential Revision: https://reviews.llvm.org/D124995
2022-05-05 10:29:47 +02:00
Zixu Wang
cb5bb28511 Revert "Revert "[clang][extract-api] Use relative includes""
Reapply the change after fixing sanitizer errors.
The original problem was that `StringRef`s in `Matches` are pointing to
temporary local `std::string`s created by `path::convert_to_slash` in
the regex match call. This patch does the conversion up front in
container `FilePath`.

This reverts commit 2966f0fa505266735dbc8324b8821b7f0aa901ff.

Differential Revision: https://reviews.llvm.org/D124964
2022-05-04 14:52:45 -07:00
Zixu Wang
2966f0fa50 Revert "[clang][extract-api] Use relative includes"
This reverts commit 4c262fee08b5383c96857d77eefe80d61c41d2b0.
Revert to fix Msan and Asan errors.
2022-05-04 12:27:20 -07:00
Zixu Wang
4c262fee08 [clang][extract-api] Use relative includes
This patch transforms the given input headers to relative include names
using header search entries and some heuritics.
For example: `/Path/To/Header.h` will be included as `<Header.h>` with a
search path of `-I /Path/To/`; and
`/Path/To/Framework.framework/Headers/Header.h` will be included as
`<Framework/Header.h>`, given a search path of `-F /Path/To`.
Headermaps will also be queried in reverse to find a spelled name to
include headers.

Differential Revision: https://reviews.llvm.org/D123831
2022-05-04 10:28:01 -07:00
Erich Keane
45c07db31c Revert "Re-apply 4b6c2cd642 "Deferred Concept Instantiation Implementation"""
This reverts commit a97899108e495147985e5e9492e742d51d5cc97a.

The patch caused some problems with the libc++ `__range_adaptor_closure`
that I haven't been able to figure out the cause of, so I am reverting
while I figure out whether this is a solvable problem/issue with the
  CFE, or libc++ depending on an older 'incorrect' behavior.
2022-05-02 11:12:14 -07:00
Erich Keane
a97899108e Re-apply 4b6c2cd642 "Deferred Concept Instantiation Implementation""
This reverts commit 0c31da48389754822dc3eecc4723160c295b9ab2.

I've solved the issue with the PointerUnion by making the
`FunctionTemplateDecl` pointer be a NamedDecl, that could be a
`FunctionDecl` or `FunctionTemplateDecl` depending.  This is enforced
with an assert.
2022-05-02 07:49:26 -07:00
Erich Keane
0c31da4838 Revert "Deferred Concept Instantiation Implementation"
This reverts commit 4b6c2cd647e9e5a147954886338f97ffb6a1bcfb.

The patch caused numerous ARM 32 bit build failures, since we added a
5th item to the PointerUnion, and went over the 2-bits available in the
32 bit pointers.
2022-05-02 06:25:38 -07:00
Erich Keane
4b6c2cd647 Deferred Concept Instantiation Implementation
As reported here: https://github.com/llvm/llvm-project/issues/44178

Concepts are not supposed to be instantiated until they are checked, so
this patch implements that and goes through significant amounts of work
to make sure we properly re-instantiate the concepts correctly.

Differential Revision: https://reviews.llvm.org/D119544
2022-05-02 05:49:15 -07:00
Duncan P. N. Exon Smith
2d13386783 Frontend: Delete output streams before closing CompilerInstance outputs
Delete the output streams coming from
CompilerInstance::createOutputFile() and friends once writes are
finished. Concretely, replacing `OS->flush()` with `OS.reset()` in:

- `ExtractAPIAction::EndSourceFileAction()`
- `PrecompiledPreambleAction::setEmittedPreamblePCH()`
- `cc1_main()'s support for `-ftime-trace`

This fixes theoretical bugs related to proxy streams, which may have
cleanups to run in their destructor. For example, a proxy that
CompilerInstance sometimes uses is `buffer_ostream`, which wraps a
`raw_ostream` lacking pwrite support and adds it. `flush()` does not
promise that output is complete; `buffer_ostream` needs to wait until
the destructor to forward anything so that it can service later calls to
`pwrite()`. If the destructor isn't called then the proxied stream
hasn't received any content.

This also protects against some logic bugs, triggering a null
dereference on a later attempt to write to the stream.

No tests, since in practice these particular code paths never use
use `buffer_ostream`; you need to be writing a binary file to a
pipe (such as stdout) to hit it, but `-extract-api` writes a text file
and the other two use computed filenames that will never (in practice)
be a pipe. This is effectively NFC, for now.

But I have some other patches in the works that add guard rails,
crashing if the stream hasn't been destructed by the time the
CompilerInstance is told to keep the output file, since in most cases
this is a problem.

Differential Revision: https://reviews.llvm.org/D124635
2022-04-28 19:07:40 -07:00
Daniel Grumberg
7443a504bf [clang][extract-api] Add support for true anonymous enums
Anonymous enums without a typedef should have a "(anonymous)" identifier.

Differential Revision: https://reviews.llvm.org/D123533
2022-04-12 20:42:17 +01:00
Daniel Grumberg
236b6a0eb4 [clang][extract-api] Emit "functionSignature" in SGF for ObjC methods.
- Split GlobalRecord into two distinct types to be able to introduce
has_function_signature type trait.
- Add has_function_signature type trait.
- Serialize function signatures as part of serializeAPIRecord for
records that are known to have a function signature.

Differential Revision: https://reviews.llvm.org/D123304
2022-04-11 19:05:24 +01:00
Daniel Grumberg
80ae366592 [clang][extract-api] Emit "navigator" property of "name" in SymbolGraph
Differential Revision: https://reviews.llvm.org/D123391
2022-04-08 17:29:00 +01:00
Daniel Grumberg
1015592251 [clang][extract-api][NFC] Use dedicated API to check for macro equality
Differential Revision: https://reviews.llvm.org/D123295
2022-04-07 19:08:17 +01:00
Zixu Wang
4048aad85a [clang][ExtractAPI] Fix declaration fragments for ObjC methods
Objective-C methods selector parts should be considered as identifiers.

Depends on D123259

Differential Revision: https://reviews.llvm.org/D123261
2022-04-07 10:22:41 -07:00
Daniel Grumberg
aebe5fc6e7 [clang][extract-api] Process only APIs declared in inputs
We should only process APIs declared in the command line inputs to avoid
drowning the ExtractAPI output with symbols the user doesn't care about.
This is achieved by keeping track of the provided input files and
checking that the associated Decl or Macro is declared in one of those files.

Differential Revision: https://reviews.llvm.org/D123148
2022-04-07 17:49:05 +01:00
Zixu Wang
fe2c77a006 [clang][ExtractAPI] Fix appendSpace in DeclarationFragments
There is a bug in `DeclarationFragments::appendSpace` where the space
character is added to a local copy of the last fragment.

Differential Revision: https://reviews.llvm.org/D123259
2022-04-07 09:17:30 -07:00
Zixu Wang
178aad9b94 [clang][extract-api] Add Objective-C Category support
Add (partial) support for Objective-C category records in ExtractAPI.
The current ExtractAPI collects everything for an Objective-C category,
but not fully serialized in the SymbolGraphSerializer. Categories
extending external interfaces are disgarded during serialization, and
categories extending known interfaces are merged (all members surfaced)
into the interfaces.

Differential Revision: https://reviews.llvm.org/D122774
2022-04-06 12:00:12 -07:00
Daniel Grumberg
9fc45ca00a [clang][extract-api] Add support for typedefs
Typedef records consist of the symbol associated with the underlying
TypedefDecl and a SymbolReference to the underlying type. Additionally
typedefs for anonymous TagTypes use the typedef'd name as the symbol
name in their respective records and USRs. As a result the declaration
fragments for the anonymous TagType are those for the associated
typedef. This means that when the user is defining a typedef to a
typedef to a anonymous type, we use a reference the anonymous TagType
itself and do not emit the typedef to the anonymous type in the
generated symbol graph, including in the type destination of further
typedef symbol records.

Differential Revision: https://reviews.llvm.org/D123019
2022-04-06 19:14:05 +01:00
Daniel Grumberg
28d793144f [clang][extract-api] Fix small issues with SymbolGraphSerializer
This includes:
- replacing "relationhips" with "relationships"
- emitting the "pathComponents" property on symbols
- emitting the "accessLevel" property on symbols

Differential Revision: https://reviews.llvm.org/D123045
2022-04-06 18:42:55 +01:00
Daniel Grumberg
8b63622b9f [clang][extract-api] Undefining macros should not result in a crash
This fixes the situation where a undefining a not previously defined
macro resulted in a crash. Before trying to remove a definition from
PendingMacros we first check to see if the macro did indeed have a
previous definition.

Differential Revision: https://reviews.llvm.org/D123056
2022-04-05 11:42:45 +01:00
Benjamin Kramer
302fe7b3c4 Remove libc++ test from clang lib.
This was added in 3ba8548c8e04bb301c4243887362c54bfbd4af8b
2022-04-05 11:33:55 +02:00
Nikolas Klauser
3ba8548c8e [libc++][ranges] Implement ranges::transform
Reviewed By: ldionne, var-const, #libc

Spies: libcxx-commits, mgorny

Differential Revision: https://reviews.llvm.org/D122173
2022-04-05 11:06:28 +02:00
Daniel Grumberg
985eaa1a3d [clang][extract-api][NFC] Don't remap the generated input buffer in PPOpts
This was triggering some build failures so removing this change for now.
2022-03-30 20:38:09 +01:00
Michael Liao
73ab5fd17b [clang] Fix shared build. NFC. 2022-03-30 14:05:14 -04:00
Daniel Grumberg
529a0570f7 [clang][extract-api] Add support for macros
To achieve this we hook into the preprocessor during the
ExtractAPIAction and record definitions for macros that don't get
undefined during preprocessing.
2022-03-30 18:33:10 +01:00
Daniel Grumberg
a9909d23e9 [clang][extractapi] Tie API and serialization to the FrontendAction
Make the API records a property of the action instead of the ASTVisitor
so that it can be accessed outside the AST visitation and push back
serialization to the end of the frontend action.

This will allow accessing and modifying the API records outside of the
ASTVisitor, which is a prerequisite for supporting macros.
2022-03-30 18:32:58 +01:00
Zixu Wang
d1d34bafef [clang][extract-api] Add Objective-C protocol support
Add support for Objective-C protocol declarations in ExtractAPI.

Depends on D122446

Differential Revision: https://reviews.llvm.org/D122511
2022-03-29 14:44:49 -07:00
Zixu Wang
9b36e126fd [clang][extract-api] Add Objective-C interface support
Add support for Objective-C interface declarations in ExtractAPI.

Depends on D122495

Differential Revision: https://reviews.llvm.org/D122446
2022-03-29 14:29:39 -07:00
Zixu Wang
15bf0e5673 [clang][extract-api] Use correct language info from inputs
The current way of getting the `clang::Language` from `LangOptions` does
not handle Objective-C correctly because `clang::Language::ObjC` does
not correspond to any `LangStandard`. This patch passes the correct
`Language` from the frontend input information.

Differential Revision: https://reviews.llvm.org/D122495
2022-03-29 10:06:08 -07:00
Chris Bieneman
d394f9f897 Add HLSL Language Option and Preprocessor
Bringing in HLSL as a language as well as language options for each of
the HLSL language standards.

While the HLSL language is unimplemented, this patch adds the
HLSL-specific preprocessor defines which enables testing of the command
line options through the driver.

Reviewed By: pete, rnk

Differential Revision: https://reviews.llvm.org/D122087
2022-03-28 16:16:17 -05:00
Zixu Wang
826e661a96 [NFC][clang][extract-api] Rename variable
Rename a local variable name to avoid potential ambiguity/conflict for
some compilers.
2022-03-24 15:12:40 -07:00
Zixu Wang
e5a7d272ab [NFC][clang][extract-api] Add missing virtual anchors
Add missing virtual method anchors for structs in ExtractAPI/API.h
2022-03-24 14:30:14 -07:00
Daniel Grumberg
0ee06c31aa [clang][extract-api] Stop allocating APIRecords via BumpPtrAllocator
Using a BumpPtrAllocator introduced memory leaks for APIRecords as they
contain a std::vector. This meant that we needed to always keep a
reference to the records in APISet and arrange for their destructor to
get called appropriately. This was further complicated by the need for
records to own sub-records as these subrecords would still need to be
allocated via the BumpPtrAllocator and the owning record would now need
to arrange for the destructor of its subrecords to be called
appropriately.

Since APIRecords contain a std::vector so whenever elements get added to
that there is an associated heap allocation regardless. Since
performance isn't currently our main priority it makes sense to use
regular unique_ptr to keep track of APIRecords, this way we don't need
to arrange for destructors to get called.

The BumpPtrAllocator is still used for strings such as USRs so that we
can easily de-duplicate them as necessary.

Differential Revision: https://reviews.llvm.org/D122331
2022-03-24 17:44:00 +00:00
Daniel Grumberg
f833aab0d0 [clang][extract-api] Enable processing of multiple headers
Before actually executing the ExtractAPIAction, clear the
CompilationInstance's input list and replace it with a single
synthesized file that just includes (or imports in ObjC) all the inputs.

Depends on D122141

Differential Revision: https://reviews.llvm.org/D122175
2022-03-23 19:05:19 +00:00
Zixu Wang
5bb5704c1b [clang][extract-api] Add struct support
- Add `StructFieldRecord` and `StructRecord` to store API information
  for structs
- Implement `VisitRecordDecl` in `ExtractAPIVisitor`
- Implement Symbol Graph serialization for struct records.
- Add test case for struct records.

Depends on D121873

Differential Revision: https://reviews.llvm.org/D122202
2022-03-23 09:45:06 -07:00
Zixu Wang
71b4c22612 [clang][extract-api] Add enum support
Add support for enum records
- Add `EnumConstantRecord` and `EnumRecord` to store API information for
  enums
- Implement `VisitEnumDecl` in `ExtractAPIVisitor`
- Implement serializatin for enum records and `MemberOf` relationship
- Add test case for enum records
- Few other improvements

Depends on D122160

Differential Revision: https://reviews.llvm.org/D121873
2022-03-23 09:41:21 -07:00
Daniel Grumberg
5ef2ec7e4e [clang][extract-api] Suppprt for the module name property in SymbolGraph
Adds `--product-name=` flag to the clang driver. This gets forwarded to
cc1 only when we are performing a ExtractAPI Action. This is used to
populate the `name` field of the module object in the generated SymbolGraph.

Differential Revision: https://reviews.llvm.org/D122141
2022-03-23 16:34:08 +00:00
Zixu Wang
b62d40216f [NFC][clang][extract-api] Use proper name string for Objective-C language
Change the Symbol Graph serializer for ExtractAPI to use `objective-c`
for the language name string for Objective-C, to align with clang
frontend standards.
2022-03-22 14:18:26 -07:00
Zixu Wang
89f6b26f1b [clang][extract-api] Refactor ExtractAPI and improve docs
- The name SymbolGraph is inappropriate and confusing for the new library
  for clang-extract-api. Refactor and rename things to make it clear that
  ExtractAPI is the core functionality and SymbolGraph is one serializer
  for the API information.
- Add documentation comments to ExtractAPI classes and methods to improve
  readability and clearness of the ExtractAPI work.

Differential Revision: https://reviews.llvm.org/D122160
2022-03-22 13:21:57 -07:00