This reverts commit r293970.
After more discussion, this belongs to the linker side and
there is no added value to do it at this level.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293993 91177308-0d34-0410-b5e6-96231b3b80d8
When a symbol is not exported outside of the
DSO, it is can be hidden. Usually we try to internalize
as much as possible, but it is not always possible, for
instance a symbol can be referenced outside of the LTO
unit, or there can be cross-module reference in ThinLTO.
This is a recommit of r293912 after fixing build failures,
and a recommit of r293918 after fixing LLD tests.
Differential Revision: https://reviews.llvm.org/D28978
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293970 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: Some compilers, including MSVC and Clang, allow linker options to be specified in source files. In the legacy LTO API, there is a getLinkerOpts() method that returns linker options for the bitcode module being processed. This change adds that method to the new API, so that the COFF linker can get the right linker options when using the new LTO API.
Reviewers: pcc, ruiu, mehdi_amini, tejohnson
Reviewed By: pcc
Differential Revision: https://reviews.llvm.org/D29207
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293950 91177308-0d34-0410-b5e6-96231b3b80d8
When a symbol is not exported outside of the
DSO, it is can be hidden. Usually we try to internalize
as much as possible, but it is not always possible, for
instance a symbol can be referenced outside of the LTO
unit, or there can be cross-module reference in ThinLTO.
This is a recommit of r293912 after fixing build failures.
Differential Revision: https://reviews.llvm.org/D28978
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293918 91177308-0d34-0410-b5e6-96231b3b80d8
When a symbol is not exported outside of the
DSO, it is can be hidden. Usually we try to internalize
as much as possible, but it is not always possible, for
instance a symbol can be referenced outside of the LTO
unit, or there can be cross-module reference in ThinLTO.
Differential Revision: https://reviews.llvm.org/D28978
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293912 91177308-0d34-0410-b5e6-96231b3b80d8
These linkages mean that the ultimately prevailing symbol will have the same
semantics as any non-prevailing copy of the symbol, so we are free to ignore
the linker's resolution.
Differential Revision: https://reviews.llvm.org/D29367
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293865 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Allow non-ODR weak/linkonce non-prevailing copies to be marked
as available_externally in the index. Add support for dropping these to
declarations in the backend.
Reviewers: mehdi_amini, pcc
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D28806
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292656 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Using the linker-supplied list of "preserved" symbols, we can compute
the list of "dead" symbols, i.e. the one that are not reachable from
a "preserved" symbol transitively on the reference graph.
Right now we are using this information to mark these functions as
non-eligible for import.
The impact is two folds:
- Reduction of compile time: we don't import these functions anywhere
or import the function these symbols are calling.
- The limited number of import/export leads to better internalization.
Patch originally by Mehdi Amini.
Reviewers: mehdi_amini, pcc
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D23488
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291177 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
When reading the metadata bitcode, create a type declaration when
possible for composite types when we are importing. Doing this in
the bitcode reader saves memory. Also it works naturally in the case
when the type ODR map contains a definition for the same composite type
because it was used in the importing module (buildODRType will
automatically use the existing definition and not create a type
declaration).
For Chromium built with -g2, this reduces the aggregate size of the
generated native object files by 66% (from 31G to 10G). It reduced
the time through the ThinLTO link and backend phases by about 20% on
my machine.
Reviewers: mehdi_amini, dblaikie, aprantl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D27775
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289993 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: ThinLTO needs to invoke SampleProfileLoader pass during link time in order to annotate profile correctly after module importing.
Reviewers: davidxl, mehdi_amini, tejohnson
Subscribers: pcc, davide, llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D27790
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289957 91177308-0d34-0410-b5e6-96231b3b80d8
Also, udpate the ~60 failing tests in the tree which did
not contain a valid datalayout.
This fixes PR31123. lld will be updated in a following patch,
immediately after this is committed.
Differential Revision: https://reviews.llvm.org/D27082
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289719 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
As discussed on mailing list, for ThinLTO importing we don't need
to import all the fields of the DICompileUnit. Don't import enums,
macros, retained types lists. Also only import local scoped imported
entities. Since we don't currently import any global variables,
we also don't need to import the list of global variables (added an
assert to verify none are being imported).
This is being done by pre-populating the value map entries to map
the unneeded metadata to nullptr. For the imported entities, we can
simply replace the source module's list with a new list containing
only those needed imported entities. This is done in the IRLinker
constructor so that value mapping automatically does the desired
mapping.
Reviewers: mehdi_amini, dexonsmith, dblaikie, aprantl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D27635
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289441 91177308-0d34-0410-b5e6-96231b3b80d8
The functions getBitcodeTargetTriple(), isBitcodeContainingObjCCategory(),
getBitcodeProducerString() and hasGlobalValueSummary() now return errors
via their return value rather than via the diagnostic handler.
To make this work, re-implement these functions using non-member functions
so that they can be used without the LLVMContext required by BitcodeReader.
Differential Revision: https://reviews.llvm.org/D26532
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286623 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Split ReaderWriter.h which contains the APIs into both the BitReader and
BitWriter libraries into BitcodeReader.h and BitcodeWriter.h.
This is to address Chandler's concern about sharing the same API header
between multiple libraries (BitReader and BitWriter). That concern is
why we create a single bitcode library in our downstream build of clang,
which led to r286297 being reverted as it added a dependency that
created a cycle only when there is a single bitcode library (not two as
in upstream).
Reviewers: mehdi_amini
Subscribers: dlj, mehdi_amini, llvm-commits
Differential Revision: https://reviews.llvm.org/D26502
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286566 91177308-0d34-0410-b5e6-96231b3b80d8
Unique ownership is just one possible ownership pattern for the memory buffer
underlying the bitcode reader. In practice, as this patch shows, ownership can
often reside at a higher level. With the upcoming change to allow multiple
modules in a single bitcode file, it will no longer be appropriate for
modules to generally have unique ownership of their memory buffer.
The C API exposes the ownership relation via the LLVMGetBitcodeModuleInContext
and LLVMGetBitcodeModuleInContext2 functions, so we still need some way for
the module to own the memory buffer. This patch does so by adding an owned
memory buffer field to Module, and using it in a few other places where it
is convenient.
Differential Revision: https://reviews.llvm.org/D26384
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286214 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
There is no point to importing at -O0, since we won't inline. We should
also disable other cross-module optimizations.
(Plan to backport this fix to the 3.9 branch to fix PR30774)
Reviewers: pcc
Subscribers: johanengelen, mehdi_amini
Differential Revision: https://reviews.llvm.org/D25918
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@285648 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
When we have an aliasee that is linkonce, while we can't convert
the non-prevailing copies to available_externally, we still need to
convert the prevailing copy to weak. If a reference to the aliasee
is exported, not converting a copy to weak will result in undefined
references when the linkonce is removed in its original module.
Add a new test and update existing tests.
Reviewers: mehdi_amini
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D26076
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@285512 91177308-0d34-0410-b5e6-96231b3b80d8
In an IR symbol table I would expect the comdats to be represented as:
- A table of strings, one for each comdat name.
- Each symbol has an optional index into that table.
The natural api for accessing that would be
InputFile:
ArrayRef<StringRef> getComdatTable() const;
Symbol:
int getComdatIndex() const;
This patch implements an API as close to that as possible. The
implementation on top of the current IRObjectFile is a bit hackish,
but should map just fine over a symbol table and is very convenient to
use.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@285061 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Changes default backend parallelism from thread::hardware_concurrency to
the new llvm::heavyweight_hardware_concurrency, which for X86 Linux
defaults to the number of physical cores (and will fall back to
thread::hardware_concurrency otherwise). This avoid oversubscribing
the physical cores using hyperthreading.
Reviewers: mehdi_amini, pcc
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D25775
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284618 91177308-0d34-0410-b5e6-96231b3b80d8
Module inline asm was always being linked/concatenated
when running the IRLinker. This is correct for full LTO but not when
we are importing for ThinLTO, as it can result in multiply defined
symbols when the module asm defines a global symbol.
In order to test with llvm-lto2, I had to work around PR30396,
where a symbol that is defined in module assembly but defined in the
LLVM IR appears twice. Added workaround to llvm-lto2 with a FIXME.
Fixes PR30610.
Reviewers: mehdi_amini
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D25359
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284030 91177308-0d34-0410-b5e6-96231b3b80d8
We need to add an entry in the combined-index for modules that have
a hash but otherwise empty summary, this is needed so that we can
get the hash for the module.
Also, if no entry is present in the combined index for a module, we
need to skip it when trying to compute a cache entry.
Differential Revision: https://reviews.llvm.org/D25300
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283654 91177308-0d34-0410-b5e6-96231b3b80d8
The NativeObjectOutput class has a design problem: it mixes up the caching
policy with the interface for output streams, which makes the client-side
code hard to follow and would for example make it harder to replace the
cache implementation in an arbitrary client.
This change separates the two aspects by moving the caching policy
to a separate field in Config, replacing NativeObjectOutput with a
NativeObjectStream class which only deals with streams and does not need to
be overridden by most clients and introducing an AddFile callback for adding
files (e.g. from the cache) to the link.
Differential Revision: https://reviews.llvm.org/D24622
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282299 91177308-0d34-0410-b5e6-96231b3b80d8
With the new LTO API in r278338, we stopped emitting the individual
index files and imports files for some modules in the distributed backend
case (thinlto-index-only plugin option).
Specifically, this is when the linker decides not to include a module in the
link, because it was in an archive library and did not have a strong
reference to it. Not creating the expected output files makes the
distributed build system implementation more difficult, in terms of
checking for the expected outputs of the thin link, and scheduling the
backend jobs. To address this, the gold-plugin will write dummy empty
.thinlto.bc and .imports files for modules not included in the link
(which LTO never sees).
Augmented a gold v1.12+ test, since that version of gold has the handling
for notifying on modules not being included in the link.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282100 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Emit an empty summary section, instead of no summary section, when
there are no global variables in the index. This ensures that LTO
will treat these files as ThinLTO inputs, instead of as regular
LTO inputs.
In addition to not being what the user likely intended when
compiling with -flto=thin, the current behavior is problematic for
distributed build systems that expect to get ThinLTO index and imports
files back for each input compiled with -flto=thin. Combining into
a single regular LTO module also reduces the backend parallelism.
And in the case where the index was suppressed due to uses in
inline assembly, combining into a single LTO module could provoke
renaming of duplicates that we were trying to prevent by suppressing
the index.
This change required a couple of fixes to handle the empty summary
section.
Reviewers: mehdi_amini
Subscribers: mehdi_amini, llvm-commits, pcc
Differential Revision: https://reviews.llvm.org/D24779
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282037 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
In runThinLTO we start the task numbering for ThinLTO backend
tasks depending on whether there was also a regular LTO object
(CombinedModule). However, the CombinedModule is moved at
the end of runRegularLTO, so we need to save this information and
pass it into runThinLTO. Otherwise the AddOutput callback to the client
will use the same task number for both the regular LTO object
and the first ThinLTO object, which in gold-plugin caused only
one to be end up in the output filename array and therefore passed
back to gold for the final native link.
Reviewers: pcc, mehdi_amini
Subscribers: mehdi_amini, kromanova
Differential Revision: https://reviews.llvm.org/D24643
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281725 91177308-0d34-0410-b5e6-96231b3b80d8
Previously the prevailing information was not honored, and commons
symbols could override a strong definition. This patch fixes it and
propose the following semantic for commons: the client should mark
as prevailing the commons that it expects the LTO implementation to
merge (i.e. take the maximum size and alignment).
It implies that commons are allowed to have multiple prevailing
definitions.
Differential Revision: https://reviews.llvm.org/D24545
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281538 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This addresses a regression in common handling from the new LTO
API in r278338. Only create a new common if the size is different.
The type comparison against an array type fails when the size is
different but not an array. GlobalMerge does not handle the
array types as well and we lose some global merging opportunities.
Reviewers: mehdi_amini
Subscribers: junbuml, llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D23955
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279911 91177308-0d34-0410-b5e6-96231b3b80d8
