Rather than relying on the structural equivalence of DICompositeType to
merge type definitions, use an explicit map on the LLVMContext that
LLParser and BitcodeReader consult when constructing new nodes.
Each non-forward-declaration DICompositeType with a non-empty
'identifier:' field is stored/loaded from the type map, and the first
definiton will "win".
This map is opt-in: clients that expect ODR types from different modules
to be merged must call LLVMContext::ensureDITypeMap.
- Clients that just happen to load more than one Module in the same
LLVMContext won't magically merge types.
- Clients (like LTO) that want to continue to merge types based on ODR
identifiers should opt-in immediately.
I have updated LTOCodeGenerator.cpp, the two "linking" spots in
gold-plugin.cpp, and llvm-link (unless -disable-debug-info-type-map) to
set this.
With this in place, it will be straightforward to remove the DITypeRef
concept (i.e., referencing types by their 'identifier:' string rather
than pointing at them directly).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266549 91177308-0d34-0410-b5e6-96231b3b80d8
(Resubmitting after fixing missing file issue)
With the changes in r263275, there are now more than just functions in
the summary. Completed the renaming of data structures (started in
r263275) to reflect the wider scope. In particular, changed the
FunctionIndex* data structures to ModuleIndex*, and renamed related
variables and comments. Also renamed the files to reflect the changes.
A companion clang patch will immediately succeed this patch to reflect
this renaming.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263513 91177308-0d34-0410-b5e6-96231b3b80d8
With the changes in r263275, there are now more than just functions in
the summary. Completed the renaming of data structures (started in
r263275) to reflect the wider scope. In particular, changed the
FunctionIndex* data structures to ModuleIndex*, and renamed related
variables and comments. Also renamed the files to reflect the changes.
A companion clang patch will immediately succeed this patch to reflect
this renaming.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263490 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch adds support for including a full reference graph including
call graph edges and other GV references in the summary.
The reference graph edges can be used to make importing decisions
without materializing any source modules, can be used in the plugin
to make file staging decisions for distributed build systems, and is
expected to have other uses.
The call graph edges are recorded in each function summary in the
bitcode via a list of <CalleeValueIds, StaticCount> tuples when no PGO
data exists, or <CalleeValueId, StaticCount, ProfileCount> pairs when
there is PGO, where the ValueId can be mapped to the function GUID via
the ValueSymbolTable. In the function index in memory, the call graph
edges reference the target via the CalleeGUID instead of the
CalleeValueId.
The reference graph edges are recorded in each summary record with a
list of referenced value IDs, which can be mapped to value GUID via the
ValueSymbolTable.
Addtionally, a new summary record type is added to record references
from global variable initializers. A number of bitcode records and data
structures have been renamed to reflect the newly expanded scope of the
summary beyond functions. More cleanup will follow.
Reviewers: joker.eph, davidxl
Subscribers: joker.eph, llvm-commits
Differential Revision: http://reviews.llvm.org/D17212
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263275 91177308-0d34-0410-b5e6-96231b3b80d8
LLVM Gold plugin decides which instance of a common symbol it wants
based on the symbol size in claim_file_hook. If the file that
contains the chosen instance is later dropped from the link, we end
up with an undefined reference.
This change delays this decision until the set of the included files
is known.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263180 91177308-0d34-0410-b5e6-96231b3b80d8
The split code gen task ID should not be appended to save-temps output
file when the parallelism factor is 1 (not actually splitting).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262731 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Since IR files are all compiled into separate independent object files
in ThinLTO mode, the prevailing linkonce symbols must be emitted in its
object file even if it is no longer referenced there, e.g. if no
references remain in the module after inlining, since it may be
referenced by another ThinLTO compiled object file. This is done by
changing LDPR_PREVAILING_DEF_IRONLY* symbols to LDPR_PREVAILING_DEF,
which converts the prevailing linkonce to weak. We also don't need the
other prevailing IRONLY handling for internalization, which is not
currently performed for ThinLTO.
Test case included.
Reviewers: davidxl, rafael
Subscribers: rafael, joker.eph, llvm-commits
Differential Revision: http://reviews.llvm.org/D16173
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262727 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Launch ThinLTO backends (LTO and codegen pipelines with importing) in
parallel using a ThreadPool, after creating the combined index.
The number of threads is controlled by the existing -jobs gold plugin
option, or the hardware concurrency if not specified.
The old behavior of exiting after creating the combined index can be
invoked via a new thinlto-index-only plugin option.
This commit involves just the ThinLTO-specific pieces of D15390, the NFC
and other restructuring pieces were committed independently:
r262677: Add hardware_concurrency interface to llvm::thread (NFC)
r262719: Change split code gen to use ThreadPool
r262721: Refactor gold-plugin codegen to prepare for ThinLTO threads (NFC)
Reviewers: pcc, joker.eph, rafael
Subscribers: rafael, davidxl, llvm-commits, joker.eph
Differential Revision: http://reviews.llvm.org/D15390
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262724 91177308-0d34-0410-b5e6-96231b3b80d8
This is the NFC part remaining from D15390, which refactors the
current codegen() into a CodeGen class with various modular methods and
other helper functions that will be used by the follow-on ThinLTO piece.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262721 91177308-0d34-0410-b5e6-96231b3b80d8
Gold has a newly added LDPT_GET_SYMBOLS_V3 callback that can
distinguish between a module that is not included in the link, and
one that is included but has its entire interface preempted by others.
Fixes PR26674.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262676 91177308-0d34-0410-b5e6-96231b3b80d8
It turns out that his is *really* slow. With this change the link of
clang with plugin-opt=emit-llvm goes from 41 to 26 seconds.
We can add an option to enable it again if needed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@256302 91177308-0d34-0410-b5e6-96231b3b80d8
This patch converts code that has access to a LLVMContext to not take a
diagnostic handler.
This has a few advantages
* It is easier to use a consistent diagnostic handler in a single program.
* Less clutter since we are not passing a handler around.
It does make it a bit awkward to implement some C APIs that return a
diagnostic string. I will propose new versions of these APIs and
deprecate the current ones.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@255571 91177308-0d34-0410-b5e6-96231b3b80d8
Ensure we release the files even when they don't hold a function index
summary section, by restructuring the control flow a little bit.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@255256 91177308-0d34-0410-b5e6-96231b3b80d8
A linker normally has two stages: symbol resolution and "moving stuff".
In lib/Linker there is the complication of lazy linking some globals,
but it was still far more mixed than it needed to.
This splits the linker into a lower level IRMover and the linker proper.
The IRMover just takes a list of globals to move and a callback that
lets the user control what is lazy linked.
The main motivation is that now tools/gold (and soon lld) can use their
own symbol resolution to instruct IRMover what to do.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@255254 91177308-0d34-0410-b5e6-96231b3b80d8
This wasn't causing an issue since at HEAD we exit the linker completely
after creating the combined index.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@255156 91177308-0d34-0410-b5e6-96231b3b80d8
Before this patch the diagnostic handler was optional. If it was not
passed, the one in the LLVMContext was used.
That is probably not a pattern we want to follow. If each area has an
optional callback, there is a sea of callbacks and it is hard to follow
which one is called.
Doing this also found cases where the callback is a nice addition, like
testing that no errors or warnings are reported.
The other option is to always use the diagnostic handler in the
LLVMContext. That has a few problems
* To implement the C API we would have to set the diag handler and then
set it back to the original value.
* Code that creates the context might be far away from code that wants
the diagnostics.
I do have a patch that implements the second option and will send that as
an RFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254777 91177308-0d34-0410-b5e6-96231b3b80d8
The linker never takes ownership of a module or changes which module it
is refering to, making it natural to use references.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254449 91177308-0d34-0410-b5e6-96231b3b80d8
Now the ValueMapper has two callbacks. The first one maps the
declaration. The ValueMapper records the mapping and then materializes
the body/initializer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254209 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Several fixes to the handling of bitcode files without function summary
sections so that they are skipped during ThinLTO processing in llvm-lto
and the gold plugin when appropriate instead of aborting.
1 Don't assert when trying to add a FunctionInfo that doesn't have
a summary attached.
2 Skip FunctionInfo structures that don't have attached function summary
sections when trying to create the combined function summary.
3 In both llvm-lto and gold-plugin, check whether a bitcode file has
a function summary section before trying to parse the index, and skip
the bitcode file if it does not.
4 Fix hasFunctionSummaryInMemBuffer in BitcodeReader, which had a bug
where we returned to early while looking for the summary section.
Also added llvm-lto and gold-plugin based tests for cases where we
don't have function summaries in the bitcode file. I verified that
either the first couple fixes described above are enough to avoid the
crashes, or fixes 1,3,4. But have combined them all here for added
robustness.
Reviewers: joker.eph
Subscribers: llvm-commits, joker.eph
Differential Revision: http://reviews.llvm.org/D14903
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@253796 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
The bitcode format is described in this document:
https://drive.google.com/file/d/0B036uwnWM6RWdnBLakxmeDdOeXc/view
For more info on ThinLTO see:
https://sites.google.com/site/llvmthinlto
The first customer is ThinLTO, however the data structures are designed
and named more generally based on prior feedback. There are a few
comments regarding how certain interfaces are used by ThinLTO, and the
options added here to gold currently have ThinLTO-specific names as the
behavior they provoke is currently ThinLTO-specific.
This patch includes support for generating per-module function indexes,
the combined index file via the gold plugin, and several tests
(more are included with the associated clang patch D11908).
Reviewers: dexonsmith, davidxl, joker.eph
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D13107
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249270 91177308-0d34-0410-b5e6-96231b3b80d8
The plugin now save the bitcode before and after optimizations and the
.o that is passed to the linker.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@239726 91177308-0d34-0410-b5e6-96231b3b80d8
Change the callers of `WriteToBitcodeFile()` to pass `true` or
`shouldPreserveBitcodeUseListOrder()` explicitly. I left the callers
that want to send `false` alone.
I'll keep pushing the bit higher until hopefully I can delete the global
`cl::opt` entirely.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@234957 91177308-0d34-0410-b5e6-96231b3b80d8
formatted_raw_ostream is a wrapper over another stream to add column and line
number tracking.
It is used only for asm printing.
This patch moves the its creation down to where we know we are printing
assembly. This has the following advantages:
* Simpler lifetime management: std::unique_ptr
* We don't compute column and line number of object files :-)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@234535 91177308-0d34-0410-b5e6-96231b3b80d8
Revert "Add classof implementations to the raw_ostream classes."
Revert "Use the cast machinery to remove dummy uses of formatted_raw_ostream."
The underlying issue can be fixed without classof.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@234495 91177308-0d34-0410-b5e6-96231b3b80d8
If we know we are producing an object, we don't need to wrap the stream
in a formatted_raw_ostream anymore.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@234461 91177308-0d34-0410-b5e6-96231b3b80d8
Keep a note in the materializer that we are stripping debug info so that
user doing a lazy read of the module don't hit outdated formats.
Thanks to Duncan for suggesting the fix.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@233603 91177308-0d34-0410-b5e6-96231b3b80d8
Dropping old debug format requires the entire module to be read upfront.
This was failing only with the gold plugin, but that is just because
llvm-link was not upgrading metadata.
The new testcase using llvm-link shows the problem.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@233381 91177308-0d34-0410-b5e6-96231b3b80d8
This speeds up llvm-ar building lib64/libclangSema.a with debug IR files
from 8.658015807 seconds to just 0.351036519 seconds :-)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@232221 91177308-0d34-0410-b5e6-96231b3b80d8
The issue was that we were always printing the remarks. Fix that and add a test
showing that it prints nothing if -pass-remarks is not given.
Original message:
Correctly handle -pass-remarks in the gold plugin.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@231273 91177308-0d34-0410-b5e6-96231b3b80d8
The issue is that now we have a diag handler during optimizations
and get forward every optimization remark, flooding stdout.
The same filtering should probably be done with or without a
custom handler, but for now just ignore remarks.
Original message:
gold-plugin: "Upgrade" debug info and handle its warnings.
The gold plugin never calls MaterializeModule, so any old debug info
was not deleted and could cause crashes.
Now that it is being "upgraded", the plugin also has to handle warnings
and create Modules with a nice id (it shows in the warning).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@230991 91177308-0d34-0410-b5e6-96231b3b80d8
It emits *millions of warnings* during selfhosting LTO build, to choke the buildbot with gigbytes of log.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@230885 91177308-0d34-0410-b5e6-96231b3b80d8
The gold plugin never calls MaterializeModule, so any old debug info
was not deleted and could cause crashes.
Now that it is being "upgraded", the plugin also has to handle warnings
and create Modules with a nice id (it shows in the warning).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@230655 91177308-0d34-0410-b5e6-96231b3b80d8
because I didn't have binutils set up properly to build the gold plugin.
Fixes PR22581 which was filed because this broke the build for folks
relying on the plugin. Very sorry! =]
I've gotten the plugin stuff building now as well so it shouldn't keep
happening.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229156 91177308-0d34-0410-b5e6-96231b3b80d8
LLVM's include tree and the use of using declarations to hide the
'legacy' namespace for the old pass manager.
This undoes the primary modules-hostile change I made to keep
out-of-tree targets building. I sent an email inquiring about whether
this would be reasonable to do at this phase and people seemed fine with
it, so making it a reality. This should allow us to start bootstrapping
with modules to a certain extent along with making it easier to mix and
match headers in general.
The updates to any code for users of LLVM are very mechanical. Switch
from including "llvm/PassManager.h" to "llvm/IR/LegacyPassManager.h".
Qualify the types which now produce compile errors with "legacy::". The
most common ones are "PassManager", "PassManagerBase", and
"FunctionPassManager".
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229094 91177308-0d34-0410-b5e6-96231b3b80d8
bfd creates the output file early, so calling exit(0) is not enough, the file needs to be explicitly deleted.
Patch by: H.J. Lu <hjl.tools@gmail.com>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228946 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Move calls to get_input_file and release_input_file out of
getModuleForFile(). Otherwise release_input_file may end up
unmapping a view of the file while the view is still being
used by the Module (on 32-bit hosts).
Fix for PR22482.
Test Plan: Add test using --no-map-whole-files.
Reviewers: rafael, nlewycky
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D7539
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228842 91177308-0d34-0410-b5e6-96231b3b80d8
While the term "Target" is in the name, it doesn't really have to do
with the LLVM Target library -- this isn't an abstraction which LLVM
targets generally need to implement or extend. It has much more to do
with modeling the various runtime libraries on different OSes and with
different runtime environments. The "target" in this sense is the more
general sense of a target of cross compilation.
This is in preparation for porting this analysis to the new pass
manager.
No functionality changed, and updates inbound for Clang and Polly.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226078 91177308-0d34-0410-b5e6-96231b3b80d8
This can happen if:
* It is present in a comdat in one file.
* It is not present in the comdat of the file that is kept.
* Is is not used.
This should fix the LTO boostrap.
Thanks to Takumi NAKAMURA for setting up the bot!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225983 91177308-0d34-0410-b5e6-96231b3b80d8
The bitcode reading interface used std::error_code to report an error to the
callers and it is the callers job to print diagnostics.
This is not ideal for error handling or diagnostic reporting:
* For error handling, all that the callers care about is 3 possibilities:
* It worked
* The bitcode file is corrupted/invalid.
* The file is not bitcode at all.
* For diagnostic, it is user friendly to include far more information
about the invalid case so the user can find out what is wrong with the
bitcode file. This comes up, for example, when a developer introduces a
bug while extending the format.
The compromise we had was to have a lot of error codes.
With this patch we use the DiagnosticHandler to communicate with the
human and std::error_code to communicate with the caller.
This allows us to have far fewer error codes and adds the infrastructure to
print better diagnostics. This is so because the diagnostics are printed when
he issue is found. The code that detected the problem in alive in the stack and
can pass down as much context as needed. As an example the patch updates
test/Bitcode/invalid.ll.
Using a DiagnosticHandler also moves the fatal/non-fatal error decision to the
caller. A simple one like llvm-dis can just use fatal errors. The gold plugin
needs a bit more complex treatment because of being passed non-bitcode files. An
hypothetical interactive tool would make all bitcode errors non-fatal.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225562 91177308-0d34-0410-b5e6-96231b3b80d8
The complicated situation is when we have to keep an alias but drop a GV
that is part of the aliasee.
We used to clone the dropped GV and make the clone internal. This is wasteful
as we know the original will be dropped.
With this patch what is done instead is set the linkage of the original to
internal and replace all uses (but the one in the alias) with a new
declaration that takes the name of the old GV. This saves us from having
to copy the body.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@223863 91177308-0d34-0410-b5e6-96231b3b80d8
It is a static method of IRObjectFile, so having to use
IRObjectFile::createIRObjectFile was redundant.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@223822 91177308-0d34-0410-b5e6-96231b3b80d8
This allows it to work with non trivial manglings like the one in COFF.
Amusingly, this can be tested with gold, as emit-llvm causes the plugin to
exit before any COFF is generated.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@223790 91177308-0d34-0410-b5e6-96231b3b80d8
Instead, walk the obj symbol list in parallel to find the GV. This shouldn't
change anything on ELF where global symbols are not mangled, but it is a step
toward supporting other object formats.
Gold itself is ELF only, but bfd ld supports COFF and the logic in the gold
plugin could be reused on lld.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@223780 91177308-0d34-0410-b5e6-96231b3b80d8
This removes calls to isMaterializable in the following cases:
* It was redundant with a call to isDeclaration now that isDeclaration returns
the correct answer for materializable functions.
* It was followed by a call to Materialize. Just call Materialize and check EC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@221050 91177308-0d34-0410-b5e6-96231b3b80d8
The also-emit-llvm option only supported getting the IR before optimizations.
This patch replaces it with a more generic save-temps option that saves the IR
both before and after optimizations.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@220885 91177308-0d34-0410-b5e6-96231b3b80d8
To do this, change the representation of lazy loaded functions.
The previous representation cannot differentiate between a function whose body
has been removed and one whose body hasn't been read from the .bc file. That
means that in order to drop a function, the entire body had to be read.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@220580 91177308-0d34-0410-b5e6-96231b3b80d8
The plugin API doesn't have the notion of linkonce, only weak. It is up to the
plugin to figure out if a symbol used only for the symbol table can be dropped.
In particular, it has to avoid dropping a linkonce_odr selected by gold if there
is also a weak_odr.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@219188 91177308-0d34-0410-b5e6-96231b3b80d8
The call to copyAttributesFrom will copy the visibility, which might assert
if it were to produce something invalid like "internal hidden". We avoid it
by first creating the replacement with the original linkage and then setting
it to internal affter the call to copyAttributesFrom.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@219184 91177308-0d34-0410-b5e6-96231b3b80d8
When creating an internal function replacement for use in an alias we were
not remapping the argument uses in the instructions to point to the new
arguments.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@219177 91177308-0d34-0410-b5e6-96231b3b80d8
This format is simply a regular object file with the bitcode stored in a
section named ".llvmbc", plus any number of other (non-allocated) sections.
One immediate use case for this is to accommodate compilation processes
which expect the object file to contain metadata in non-allocated sections,
such as the ".go_export" section used by some Go compilers [1], although I
imagine that in the future we could consider compiling parts of the module
(such as large non-inlinable functions) directly into the object file to
improve LTO efficiency.
[1] http://golang.org/doc/install/gccgo#Imports
Differential Revision: http://reviews.llvm.org/D4371
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218078 91177308-0d34-0410-b5e6-96231b3b80d8
With this a DataLayoutPass can be reused for multiple modules.
Once we have doInitialization/doFinalization, it doesn't seem necessary to pass
a Module to the constructor.
Overall this change seems in line with the idea of making DataLayout a required
part of Module. With it the only way of having a DataLayout used is to add it
to the Module.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217548 91177308-0d34-0410-b5e6-96231b3b80d8
