Cache the result of mapping metadata nodes between instances of IRLinker
(i.e., for the lifetime of IRMover). There shouldn't be any real
functional change here, but this should give a major speedup. I had
loaned this to Mehdi when he tested performance of r266446, and the two
patches together gave a 10x speedup in metadata mapping.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266579 91177308-0d34-0410-b5e6-96231b3b80d8
Add an assertion to ValueMapper that prevents double-scheduling of
GlobalValues to remap, and fix the one place it happened. There are
tons of tests that fail with this assertion in place and without the
code change, so I'm not adding another.
Although it looks related, r266563 was, indeed, removing dead code.
AFAICT, this cross-file double-scheduling started in r266510 when the
cross-file recursion was removed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266569 91177308-0d34-0410-b5e6-96231b3b80d8
This reverts commit r266507, reapplying r266503 (and r266505
"ValueMapper: Use API from r266503 in unit tests, NFC") completely
unchanged.
I reverted because of a bot failure here:
http://lab.llvm.org:8011/builders/lld-x86_64-freebsd/builds/16810/
However, looking more closely, the failure was from a host-compiler
crash (clang 3.7.1) when building:
lib/CodeGen/AsmPrinter/CMakeFiles/LLVMAsmPrinter.dir/DwarfAccelTable.cpp.o
I didn't modify that file, or anything it includes, with that commit.
The next build (which hadn't picked up my revert) got past it:
http://lab.llvm.org:8011/builders/lld-x86_64-freebsd/builds/16811/
I think this was just unfortunate timing. I suppose the bot must be
flakey.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266510 91177308-0d34-0410-b5e6-96231b3b80d8
Eliminate co-recursion of Mapper::mapValue through
ValueMaterializer::materializeInitFor, through a major redesign of the
ValueMapper.cpp interface.
- Expose a ValueMapper class that controls the entry points to the
mapping algorithms.
- Change IRLinker to use ValueMapper directly, rather than
llvm::RemapInstruction, llvm::MapValue, etc.
- Use (e.g.) ValueMapper::scheduleMapGlobalInit to add mapping work to
a worklist in ValueMapper instead of recursing.
There were two fairly major complications.
Firstly, IRLinker::linkAppendingVarProto incorporates an on-the-fly IR
ugprade that I had to split apart. Long-term, this upgrade should be
done in the bitcode reader (and we should only accept the "new" form),
but for now I've just made it work and added a FIXME. The hold-op is
that we need to deprecate C API that relies on this.
Secondly, IRLinker has special logic to correctly implement aliases with
comdats, and uses two ValueToValueMapTy instances and two
ValueMaterializers. I supported this by allowing clients to register an
alternate mapping context, whose MCID can be passed in when scheduling
new work.
While out of scope for this commit, it should now be straightforward to
remove recursion from Mapper::mapValue.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266503 91177308-0d34-0410-b5e6-96231b3b80d8
Currently each Function points to a DISubprogram and DISubprogram has a
scope field. For member functions the scope is a DICompositeType. DIScopes
point to the DICompileUnit to facilitate type uniquing.
Distinct DISubprograms (with isDefinition: true) are not part of the type
hierarchy and cannot be uniqued. This change removes the subprograms
list from DICompileUnit and instead adds a pointer to the owning compile
unit to distinct DISubprograms. This would make it easy for ThinLTO to
strip unneeded DISubprograms and their transitively referenced debug info.
Motivation
----------
Materializing DISubprograms is currently the most expensive operation when
doing a ThinLTO build of clang.
We want the DISubprogram to be stored in a separate Bitcode block (or the
same block as the function body) so we can avoid having to expensively
deserialize all DISubprograms together with the global metadata. If a
function has been inlined into another subprogram we need to store a
reference the block containing the inlined subprogram.
Attached to https://llvm.org/bugs/show_bug.cgi?id=27284 is a python script
that updates LLVM IR testcases to the new format.
http://reviews.llvm.org/D19034
<rdar://problem/25256815>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266446 91177308-0d34-0410-b5e6-96231b3b80d8
Strip out the remapping parts of IRLinker::linkFunctionBody and put them
in ValueMapper.cpp under the name Mapper::remapFunction (with a
top-level entry-point llvm::RemapFunction).
This is a nice cleanup on its own since it puts the remapping code
together and shares a single Mapper context for the entire
IRLinker::linkFunctionBody Call. Besides that, this will make it easier
to break the co-recursion between IRMover.cpp and ValueMapper.cpp in
follow ups.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@265835 91177308-0d34-0410-b5e6-96231b3b80d8
This is a cleanup after clarifying the meaning of RF_IgnoreMissingLocals
in r265628 and truly limiting it to locals in r265768.
This should have no functionality change, since the only context that
the flag has an effect is when we could hit function-local Value and
Metadata, and we were already passing it in those contexts.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@265831 91177308-0d34-0410-b5e6-96231b3b80d8
Clarify what this RemapFlag actually means.
- Change the flag name to match its intended behaviour.
- Clearly document that it's not supposed to affect globals.
- Add a host of FIXMEs to indicate how to fix the behaviour to match
the intent of the flag.
RF_IgnoreMissingLocals should only affect the behaviour of
RemapInstruction for function-local operands; namely, for operands of
type Argument, Instruction, and BasicBlock. Currently, it is *only*
passed into RemapInstruction calls (and the transitive MapValue calls
that it makes).
When I split Metadata from Value I didn't understand the flag, and I
used it in a bunch of places for "global" metadata.
This commit doesn't have any functionality change, but prepares to
cleanup MapMetadata and MapValue.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@265628 91177308-0d34-0410-b5e6-96231b3b80d8
Instead of copying arguments from the source function to the
destination, steal them. This has a few advantages.
- The ValueMap doesn't need to be seeded with (or cleared of)
Arguments.
- Often the destination function won't have created any arguments yet,
so this avoids malloc traffic.
- Argument names don't need to be copied.
Because argument lists are lazy, this required a new
Function::stealArgumentListFrom helper.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@265519 91177308-0d34-0410-b5e6-96231b3b80d8
Split the loop through compile units in mapUnneededSubprograms in two.
First, visit imported entities to ensure that we've visited all need
subprograms. Second, visit subprograms, and drop the ones we don't
need.
Hypothetically this protects against a subprogram from one compile unit
being referenced from an imported entity in a different compile unit. I
don't think that's valid IR (a debug info expert could confirm), but I
think the refactor makes the code more clear.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@265233 91177308-0d34-0410-b5e6-96231b3b80d8
IRLinker::mapUnneededSubprograms has to be sure that any "needed"
subprograms get linked in. Rather than traversing through imported
entities using llvm::getSubprogram, call MapMetadata. The latter
memoizes the result in the ValueMap (sharing work with
IRLinker::linkNamedMDNodes proper), and makes the local SmallPtrSet
redundant.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@265231 91177308-0d34-0410-b5e6-96231b3b80d8
Instead of checking live during MapMetadata whether a subprogram is
needed, seed the ValueMap with `nullptr` up-front.
There is a small hypothetical functionality change. Previously, calling
MapMetadataOp on a node whose "scope:" chain led to an unneeded
subprogram would return nullptr. However, if that were ever called,
then the subprogram would be needed; a situation that the IRMover is
supposed to avoid a priori!
Besides cleaning up the code a little, this restores a nice property:
MapMetadataOp returns the same as MapMetadata.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@265229 91177308-0d34-0410-b5e6-96231b3b80d8
Support seeding a ValueMap with nullptr for Metadata entries, a
situation I didn't consider in the Metadata/Value split.
I added a ValueMapper::getMappedMD accessor that returns an
Optional<Metadata*> with the mapped (possibly null) metadata. IRMover
needs to use this to avoid modifying the map when it's checking for
unneeded subprograms. I updated a call from bugpoint since I find the
new code clearer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@265228 91177308-0d34-0410-b5e6-96231b3b80d8
Since we have moved to a model where functions are imported in bulk from
each source module after making summary-based importing decisions, there
is no longer a need to link metadata as a postpass, and all users have
been removed.
This essentially reverts r255909 and follow-on fixes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@264763 91177308-0d34-0410-b5e6-96231b3b80d8
After comdat processing, the symbols still go through regular symbol
resolution.
We were not doing it for linkonce symbols since they are lazy linked.
This fixes pr27044.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@264288 91177308-0d34-0410-b5e6-96231b3b80d8
If not for lazy linking of linkonce GVs, comdats are just a
preprocessing before symbol resolution.
Lazy linking complicates it since when we pick a visible member of
comdat, we have to make sure the rest of it passes symbol resolution
too.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@264223 91177308-0d34-0410-b5e6-96231b3b80d8
If a comdat is dropped, all symbols in it are dropped.
If a comdat is kept, the symbols survive to pass regular symbol
resolution.
With this patch we do that for all global symbols.
The added test is a copy of test/tools/gold/X86/comdat.ll that we now
pass.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@264192 91177308-0d34-0410-b5e6-96231b3b80d8
A really unfortunate design of llvm-link and related libraries is that
they operate one module at a time.
This means they can copy a GV to the destination module that should not
be there in the final result because a later bitcode file takes
precedence.
We already handled cases like a strong GV replacing a weak for example.
One case that is not currently handled is a comdat replacing another.
This doesn't happen in ELF, but with COFF largest selection kind it is
possible.
In "llvm-link a.ll b.ll" if the selected comdat was from a.ll,
everything will work and we will not copy the comdat from b.ll.
But if we run "llvm-link b.ll a.ll", we fail to delete the already
copied comdat from b.ll. This patch fixes that.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@264103 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
ThinLTO is relying on linkInModule to import selected function.
However a lot of "magic" was hidden in linkInModule and the IRMover,
who would rename and promote global variables on the fly.
This is moving to an approach where the steps are decoupled and the
client is reponsible to specify the list of globals to import.
As a consequence some test are changed because they were relying on
the previous behavior which was importing the definition of *every*
single global without control on the client side.
Now the burden is on the client to decide if a global has to be imported
or not.
Reviewers: tejohnson
Subscribers: joker.eph, llvm-commits
Differential Revision: http://reviews.llvm.org/D18122
From: Mehdi Amini <mehdi.amini@apple.com>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263863 91177308-0d34-0410-b5e6-96231b3b80d8
(Resubmitting after fixing missing file issue)
With the changes in r263275, there are now more than just functions in
the summary. Completed the renaming of data structures (started in
r263275) to reflect the wider scope. In particular, changed the
FunctionIndex* data structures to ModuleIndex*, and renamed related
variables and comments. Also renamed the files to reflect the changes.
A companion clang patch will immediately succeed this patch to reflect
this renaming.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263513 91177308-0d34-0410-b5e6-96231b3b80d8
With the changes in r263275, there are now more than just functions in
the summary. Completed the renaming of data structures (started in
r263275) to reflect the wider scope. In particular, changed the
FunctionIndex* data structures to ModuleIndex*, and renamed related
variables and comments. Also renamed the files to reflect the changes.
A companion clang patch will immediately succeed this patch to reflect
this renaming.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263490 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Unless we plan to do later postpass metadata linking (ThinLTO special mode),
always invoke metadata materialization at the start of IRLinker::run().
This avoids the need for clients who use lazy metadata loading to
explicitly invoke materializeMetadata before the IRMover, which in
turn invokes IRLinker::run and needs materialized metadata for mapping.
Came up in the context of an LLD issue (D17982).
Reviewers: rafael
Subscribers: silvas, llvm-commits
Differential Revision: http://reviews.llvm.org/D17992
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263143 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
As discussed on IRC, move the ThinLTOGlobalProcessing code out of
the linker, and into TransformUtils. The name of the class is changed
to FunctionImportGlobalProcessing.
Reviewers: joker.eph, rafael
Subscribers: joker.eph, llvm-commits
Differential Revision: http://reviews.llvm.org/D17081
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260395 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Available externally definitions are considered declarations for the
linker and eventually dropped. As such they are not allowed to be
in comdats. Remove any such imported functions from comdats.
Reviewers: rafael
Subscribers: davidxl, llvm-commits, joker.eph
Differential Revision: http://reviews.llvm.org/D16120
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260122 91177308-0d34-0410-b5e6-96231b3b80d8
The stripNullSubprograms function is very inefficient because
it walks all subprograms in all compile units in the dest module
any time a new module is linked in. For LTO in particular this will
get increasingly expensive as more modules are linked.
This patch improves the efficiency in several ways. The first is that
no scanning is necessary when there were no unneeded subprograms
identified in the first place. The second is that only the newly-linked
module's compile unit metadata should be examined.
Fixes PR26346.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@259049 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch is provided in preparation for removing autoconf on 1/26. The proposal to remove autoconf on 1/26 was discussed on the llvm-dev thread here: http://lists.llvm.org/pipermail/llvm-dev/2016-January/093875.html
"I felt a great disturbance in the [build system], as if millions of [makefiles] suddenly cried out in terror and were suddenly silenced. I fear something [amazing] has happened."
- Obi Wan Kenobi
Reviewers: chandlerc, grosbach, bob.wilson, tstellarAMD, echristo, whitequark
Subscribers: chfast, simoncook, emaste, jholewinski, tberghammer, jfb, danalbert, srhines, arsenm, dschuff, jyknight, dsanders, joker.eph, llvm-commits
Differential Revision: http://reviews.llvm.org/D16471
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@258861 91177308-0d34-0410-b5e6-96231b3b80d8
For metadata postpass linking, after importing all functions, we need
to recursively walk through any nodes reached via imported functions to
locate needed subprogram metadata. Some might only be reached indirectly
via the variable list for an inlined function.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@258728 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Since we are currently not doing incremental importing there is
no need to link metadata as a postpass. The module linker will
only link in the imported subroutines due to the functionality
added by r256003.
(Note that the metadata postpass linking functionalitiy is still
used by llvm-link, and may be needed here in the future if a more
incremental strategy is adopted.)
Reviewers: joker.eph
Subscribers: joker.eph, llvm-commits
Differential Revision: http://reviews.llvm.org/D16424
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@258458 91177308-0d34-0410-b5e6-96231b3b80d8
Replace sequences of count() followed by operator[] with either
find() or insert(), depending on the context.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@258405 91177308-0d34-0410-b5e6-96231b3b80d8
Fix the condition for when the new global takes over the name of
the existing one to be the negation of the condition for the new
global to get internal linkage.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@258355 91177308-0d34-0410-b5e6-96231b3b80d8
Function::copyAttributesFrom will copy the personality function, prefix
data and prolog data from the source function to the new function, and
is invoked when the IRMover copies the function prototype. This puts a
reference to a constant in the source module on a function in the dest
module, which causes an error when deleting the source module after
importing, since the personality function in the source module still has
uses (this would presumably also be an issue for the prologue and prefix
data). Remove the copies added to the dest copy when creating the new
prototype, as they are mapped properly when/if we link the function body.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257420 91177308-0d34-0410-b5e6-96231b3b80d8
Due to the new in-place ThinLTO symbol handling support added in
r257174, we now invoke renameModuleForThinLTO on the current
module from within the FunctionImport pass.
Additionally, renameModuleForThinLTO no longer needs to return the
Module as it is performing the renaming in place on the one provided.
This commit will be immediately preceeded by a companion clang patch to
remove its invocation of renameModuleForThinLTO.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257181 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Move ThinLTO global value processing functions out of ModuleLinker and
into a new ThinLTOGlobalProcessor class, which performs any necessary
linkage and naming changes on the given module in place.
As a result, renameModuleForThinLTO no longer needs to create a new
Module when performing any necessary local to global promotion on a
module that we are possibly exporting from during a ThinLTO backend
compilation.
During function importing the ThinLTO processing is still invoked from
the ModuleLinker (via the new class), as it needs to perform renaming and
linkage changes on the source module, e.g. in order to get the correct
renaming during local to global promotion.
Reviewers: joker.eph
Subscribers: davidxl, llvm-commits, joker.eph
Differential Revision: http://reviews.llvm.org/D15696
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257174 91177308-0d34-0410-b5e6-96231b3b80d8
Follow-up to r257000: DIImportedEntity can reach a DISubprogram via
its entity, but also via its scope. Handle the latter case as well.
PR26037.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257019 91177308-0d34-0410-b5e6-96231b3b80d8
It is illegal to have a null entity in a DIImportedEntity, so
we must link in a DISubprogram metadata node referenced by one,
even if the associated function is not linked in or inlined anywhere.
Fixes PR26037.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257000 91177308-0d34-0410-b5e6-96231b3b80d8
