Dependent libraries support for the legacy api was committed in a
broken state (see: https://reviews.llvm.org/D60274). This was missed
due to the painful nature of having to integrate the changes into a
linker in order to test. This change implements support for dependent
libraries in the legacy LTO api:
- I have removed the current api function, which returns a single
string, and added functions to access each dependent library
specifier individually.
- To reduce the testing pain, I have made the api functions as thin as
possible to maximize coverage from llvm-lto.
- When doing ThinLTO the system linker will load the modules lazily
when scanning the input files. Unfortunately, when modules are
lazily loaded there is no access to module level named metadata. To
fix this I have added api functions that allow querying the IRSymtab
for the dependent libraries. I hope to expand the api in the future
so that, eventually, all the information needed by a client linker
during scan can be retrieved from the IRSymtab.
Differential Revision: https://reviews.llvm.org/D62935
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363140 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
For the most part this consists of replacing ${LLVM_TARGETS_TO_BUILD} with
some combination of AllTargets* so that they depend on specific components
of a target backend rather than all of it. The overall effect of this is
that, for example, tools like opt no longer falsely depend on the
disassembler, while tools like llvm-ar no longer depend on the code
generator.
There's a couple quirks to point out here:
* AllTargetsCodeGens is a bit more prevalent than expected. Tools like dsymutil
seem to need it which I was surprised by.
* llvm-xray linked to all the backends but doesn't seem to need any of them.
It builds and passes the tests so that seems to be correct.
* I left gold out as it's not built when binutils is not available so I'm
unable to test it
Reviewers: bogner, JDevlieghere
Reviewed By: bogner
Subscribers: mehdi_amini, mgorny, steven_wu, dexonsmith, rupprecht, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D62331
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@361567 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Reapply r357931 with fixes to ThinLTO testcases and llvm-lto tool.
ThinLTOCodeGenerator currently does not preserve llvm.used symbols and
it can internalize them. In order to pass the necessary information to the
legacy ThinLTOCodeGenerator, the input to the code generator is
rewritten to be based on lto::InputFile.
Now ThinLTO using the legacy LTO API will requires data layout in
Module.
"internalize" thinlto action in llvm-lto is updated to run both
"promote" and "internalize" with the same configuration as
ThinLTOCodeGenerator. The old "promote" + "internalize" option does not
produce the same output as ThinLTOCodeGenerator.
This fixes: PR41236
rdar://problem/49293439
Reviewers: tejohnson, pcc, kromanova, dexonsmith
Reviewed By: tejohnson
Subscribers: ormris, bd1976llvm, mehdi_amini, inglorion, eraman, hiraditya, jkorous, dexonsmith, arphaman, dang, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D60421
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358601 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
ThinLTOCodeGenerator currently does not preserve llvm.used symbols and
it can internalize them. In order to pass the necessary information to the
legacy ThinLTOCodeGenerator, the input to the code generator is
rewritten to be based on lto::InputFile.
This fixes: PR41236
rdar://problem/49293439
Reviewers: tejohnson, pcc, dexonsmith
Reviewed By: tejohnson
Subscribers: mehdi_amini, inglorion, eraman, hiraditya, jkorous, dang, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D60226
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@357931 91177308-0d34-0410-b5e6-96231b3b80d8
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@351636 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
We can sometimes end up with multiple copies of a local variable that
have the same GUID in the index. This happens when there are local
variables with the same name that are in different source files having the
same name/path at compile time (but compiled into different bitcode objects).
In this case make sure we import the copy in the caller's module.
This enables importing both of the variables having the same GUID
(but which will have different promoted names since the module paths,
and therefore the module hashes, will be distinct).
Importing the wrong copy is particularly problematic for read only
variables, since we must import them as a local copy whenever
referenced. Otherwise we get undefs at link time.
Note that the llvm-lto.cpp and ThinLTOCodeGenerator changes are needed
for testing the distributed index case via clang, which will be sent as
a separate clang-side patch shortly. We were previously not doing the
dead code/read only computation before computing imports when testing
distributed index generation (like it was for testing importing and
other ThinLTO mechanisms alone).
Reviewers: evgeny777
Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, dang, llvm-commits
Differential Revision: https://reviews.llvm.org/D55047
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@347886 91177308-0d34-0410-b5e6-96231b3b80d8
Two cases in a ThinLTO test were passing for the wrong reasons, since
rL340374. The tests were supposed to be testing that files were being
pruned due to the cache size, but they were in fact being pruned because
they were older than the default expiration period of 1 week.
This change fixes the tests by explicitly setting the expiration time to
the maximum value. This required the option to be exposed in llvm-lto.
By assigning all files in the cache a similar time, it is possible to see
that the newest files are still being kept, and that we aren't passing
for the wrong reason again. In the event that the entry expiration were
to expire for them, then the test would start failing, because these
files would be removed too.
Reviewed by: rnk, inglorion
Differential Revision: https://reviews.llvm.org/D51992
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343687 91177308-0d34-0410-b5e6-96231b3b80d8
The original was reverted due to an apparent build-bot test failure,
but it looks like this is just a flaky test.
Also added a C-interface function for large values, and updated
llvm-lto's --thinlto-cache-max-size-bytes switch to take a type larger
than int.
The maximum cache size in terms of bytes is a 64-bit number. However,
the methods to set it only took unsigned previously, which meant that
the maximum cache size could not be specified above 4GB. That's quite
small compared to the output of some projects, so it makes sense to
provide the ability to set larger values in that field.
We also needed a C-interface function that provides a greater range
than the existing thinlto_codegen_set_cache_size_bytes, which also only
takes an unsigned, so this change also adds
hinlto_codegen_set_cache_size_megabytes.
Reviewed by: mehdi_amini, tejohnson, steven_wu
Differential Revision: https://reviews.llvm.org/D52023
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@342366 91177308-0d34-0410-b5e6-96231b3b80d8
This caused LLD test failures, which I've been unable to reproduce.
Reverting to allow for further investigation next week.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@342244 91177308-0d34-0410-b5e6-96231b3b80d8
Also added a C-interface function for large values, and updated
llvm-lto's --thinlto-cache-max-size-bytes switch to take a type larger
than int.
The maximum cache size in terms of bytes is a 64-bit number. However,
the methods to set it only took unsigned previously, which meant that
the maximum cache size could not be specified above 4GB. That's quite
small compared to the output of some projects, so it makes sense to
provide the ability to set larger values in that field.
We also needed a C-interface function that provides a greater range
than the existing thinlto_codegen_set_cache_size_bytes, which also only
takes an unsigned, so this change also adds
hinlto_codegen_set_cache_size_megabytes.
Reviewed by: mehdi_amini, tejohnson, steven_wu
Differential Revision: https://reviews.llvm.org/D52023
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@342233 91177308-0d34-0410-b5e6-96231b3b80d8
With the upcoming patch to add summary parsing support, IsAnalysis would
be true in contexts where we are not performing module summary analysis.
Rename to the more specific and approprate HaveGVs, which is essentially
what this flag is indicating.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334140 91177308-0d34-0410-b5e6-96231b3b80d8
The module ID numbering typically starts at 0 (in both the new and old
LTO APIs, used by linkers). Make llvm-lto consistent with that.
Split out of D46699.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332476 91177308-0d34-0410-b5e6-96231b3b80d8
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.
Patch produced by
for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done
Differential Revision: https://reviews.llvm.org/D46290
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@331272 91177308-0d34-0410-b5e6-96231b3b80d8
We have a few functions that virtually all command wants to run on
process startup/shutdown. This patch adds InitLLVM class to do that
all at once, so that we don't need to copy-n-paste boilerplate code
to each llvm command's main() function.
Differential Revision: https://reviews.llvm.org/D45602
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330046 91177308-0d34-0410-b5e6-96231b3b80d8
These aren't the .def style files used in LLVM that require a macro
defined before their inclusion - they're just basic non-modular includes
to stamp out command line flag variables.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@329840 91177308-0d34-0410-b5e6-96231b3b80d8
- thinlto_codegen_set_cache_size_bytes to control the absolute size of cache directory.
- thinlto_codegen_set_cache_size_files the size and amount of files in cache directory.
These functions have been supported in C++ LTO API for a long time, but were absent in C LTO API.
Differential Revision: https://reviews.llvm.org/D42446
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@326537 91177308-0d34-0410-b5e6-96231b3b80d8
borked by: rL284966 (see: https://reviews.llvm.org/D25730).
Previously, Interval was unsigned (see: CachePruning.h), replacing the type with std::chrono::seconds (which is signed) causes a regression in behaviour because the c-api intends negative values to translate to large positive intervals to *effectively* disable the pruning (see comments on: setCachePruningInterval()).
Differential Revision: https://reviews.llvm.org/D41231
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@321077 91177308-0d34-0410-b5e6-96231b3b80d8
Since this isn't a real header - it includes static functions and had
external linkage variables (though this change makes them static, since
that's what they should be) so can't be included more than once in a
program.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@319082 91177308-0d34-0410-b5e6-96231b3b80d8
It enables OptimizationRemarkEmitter::allowExtraAnalysis and MachineOptimizationRemarkEmitter::allowExtraAnalysis to return true not only for -fsave-optimization-record but when specific remarks are requested with
command line options.
The diagnostic handler used to be callback now this patch adds a class
DiagnosticHandler. It has virtual method to provide custom diagnostic handler
and methods to control which particular remarks are enabled.
However LLVM-C API users can still provide callback function for diagnostic handler.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313390 91177308-0d34-0410-b5e6-96231b3b80d8
It enables OptimizationRemarkEmitter::allowExtraAnalysis and MachineOptimizationRemarkEmitter::allowExtraAnalysis to return true not only for -fsave-optimization-record but when specific remarks are requested with
command line options.
The diagnostic handler used to be callback now this patch adds a class
DiagnosticHandler. It has virtual method to provide custom diagnostic handler
and methods to control which particular remarks are enabled.
However LLVM-C API users can still provide callback function for diagnostic handler.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313382 91177308-0d34-0410-b5e6-96231b3b80d8
This reverts commit r302108. This causes crash in clang bootstrap with LTO.
Contacted the auther in the original commit.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@302140 91177308-0d34-0410-b5e6-96231b3b80d8
When profiling a no-op incremental link of Chromium I found that the functions
computeImportForFunction and computeDeadSymbols were consuming roughly 10% of
the profile. The goal of this change is to improve the performance of those
functions by changing the map lookups that they were previously doing into
pointer dereferences.
This is achieved by changing the ValueInfo data structure to be a pointer to
an element of the global value map owned by ModuleSummaryIndex, and changing
reference lists in the GlobalValueSummary to hold ValueInfos instead of GUIDs.
This means that a ValueInfo will take a client directly to the summary list
for a given GUID.
Differential Revision: https://reviews.llvm.org/D32471
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@302108 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
The motivation is to support better the -object_path_lto option on
Darwin. The linker needs to write down the generate object files on
disk for later use by lldb or dsymutil (debug info are not present
in the final binary). We're moving this into libLTO so that we can
be smarter when a cache is enabled and hard-link when possible
instead of duplicating the files.
Reviewers: tejohnson, deadalnix, pcc
Subscribers: dexonsmith, llvm-commits
Differential Revision: https://reviews.llvm.org/D27507
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289631 91177308-0d34-0410-b5e6-96231b3b80d8
llvm-lto.cpp has the following include chain:
llvm/Bitcode/BitcodeReader.h
llvm/IR/ModuleSummaryIndex.h
llvm/IR/Module.h
llvm/IR/Function.h
llvm/IR/Argument.h
llvm/IR/Attributes.h
llvm/IR/Attributes.gen
This means llvm-lto needs to depend on intrinsics_gen.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287398 91177308-0d34-0410-b5e6-96231b3b80d8
The functions getBitcodeTargetTriple(), isBitcodeContainingObjCCategory(),
getBitcodeProducerString() and hasGlobalValueSummary() now return errors
via their return value rather than via the diagnostic handler.
To make this work, re-implement these functions using non-member functions
so that they can be used without the LLVMContext required by BitcodeReader.
Differential Revision: https://reviews.llvm.org/D26532
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286623 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Split ReaderWriter.h which contains the APIs into both the BitReader and
BitWriter libraries into BitcodeReader.h and BitcodeWriter.h.
This is to address Chandler's concern about sharing the same API header
between multiple libraries (BitReader and BitWriter). That concern is
why we create a single bitcode library in our downstream build of clang,
which led to r286297 being reverted as it added a dependency that
created a cycle only when there is a single bitcode library (not two as
in upstream).
Reviewers: mehdi_amini
Subscribers: dlj, mehdi_amini, llvm-commits
Differential Revision: https://reviews.llvm.org/D26502
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286566 91177308-0d34-0410-b5e6-96231b3b80d8
The core of the change is supposed to be NFC, however it also fixes
what I believe was an undefined behavior when calling:
va_start(ValueArgs, Desc);
with Desc being a StringRef.
Differential Revision: https://reviews.llvm.org/D25342
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283671 91177308-0d34-0410-b5e6-96231b3b80d8
We need to add an entry in the combined-index for modules that have
a hash but otherwise empty summary, this is needed so that we can
get the hash for the module.
Also, if no entry is present in the combined index for a module, we
need to skip it when trying to compute a cache entry.
Differential Revision: https://reviews.llvm.org/D25300
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283654 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Fixed a bug in ThinLTOCodeGenerator's temp file dumping. The Twine
needs to be passed directly as an argument, or a copy saved into a
std::string.
It doesn't seem there are any consumers of this, so I added a new option
to llvm-lto to enable saving of temp files during ThinLTO, and augmented
a test to use it to check post-import but pre-opt bitcode.
Reviewers: mehdi_amini
Subscribers: llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D23525
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278761 91177308-0d34-0410-b5e6-96231b3b80d8
