18 Commits

Author SHA1 Message Date
Peter Collingbourne
a4799adf39 Object: Teach irsymtab::read() to try to use the irsymtab that we wrote to disk.
Fixes PR27551.

Differential Revision: https://reviews.llvm.org/D33974

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306488 91177308-0d34-0410-b5e6-96231b3b80d8
2017-06-27 23:50:24 +00:00
Peter Collingbourne
95d9f1b4f9 Apply summary-based dead stripping to regular LTO modules with summaries.
If a regular LTO module has a summary index, then instead of linking
it into the combined regular LTO module right away, add it to the
combined summary index and associate it with a special module that
represents the combined regular LTO module.

Any such modules are linked during LTO::run(), at which time we use
the results of summary-based dead stripping to control whether to
link prevailing symbols.

Differential Revision: https://reviews.llvm.org/D33922

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@305482 91177308-0d34-0410-b5e6-96231b3b80d8
2017-06-15 17:26:13 +00:00
Peter Collingbourne
b9fc96d9dd Bitcode: Introduce a BitcodeFileContents data type. NFCI.
This data type includes the contents of a bitcode file.
Right now a bitcode file can only contain modules, but
a later change will add a symbol table.

Differential Revision: https://reviews.llvm.org/D33969

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@305019 91177308-0d34-0410-b5e6-96231b3b80d8
2017-06-08 22:00:24 +00:00
Teresa Johnson
91c3ccb649 Remove ignore-empty-index-file option
Summary:
As discussed in the D32195 review thread and on IRC, remove this option
and replace with parameter, which will be set to true when invoked
from clang in the context of a ThinLTO distributed backend.

Reviewers: pcc

Subscribers: mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D33133

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@302939 91177308-0d34-0410-b5e6-96231b3b80d8
2017-05-12 19:32:11 +00:00
Peter Collingbourne
10dbf12dd6 Bitcode: Make the summary reader responsible for merging. NFCI.
This is to prepare for an upcoming change which uses pointers instead of
GUIDs to represent references.

Differential Revision: https://reviews.llvm.org/D32469

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@301843 91177308-0d34-0410-b5e6-96231b3b80d8
2017-05-01 22:04:36 +00:00
Peter Collingbourne
6a2cc4c156 Object: Remove ModuleSummaryIndexObjectFile class.
Differential Revision: https://reviews.llvm.org/D32195

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@301832 91177308-0d34-0410-b5e6-96231b3b80d8
2017-05-01 20:42:32 +00:00
Peter Collingbourne
6163b4af73 Bitcode: Add a string table to the bitcode format.
Add a top-level STRTAB block containing a string table blob, and start storing
strings for module codes FUNCTION, GLOBALVAR, ALIAS, IFUNC and COMDAT in
the string table.

This change allows us to share names between globals and comdats as well
as between modules, and improves the efficiency of loading bitcode files by
no longer using a bit encoding for symbol names. Once we start writing the
irsymtab to the bitcode file we will also be able to share strings between
it and the module.

On my machine, link time for Chromium for Linux with ThinLTO decreases by
about 7% for no-op incremental builds or about 1% for full builds. Total
bitcode file size decreases by about 3%.

As discussed on llvm-dev:
http://lists.llvm.org/pipermail/llvm-dev/2017-April/111732.html

Differential Revision: https://reviews.llvm.org/D31838

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@300464 91177308-0d34-0410-b5e6-96231b3b80d8
2017-04-17 17:51:36 +00:00
Teresa Johnson
f638013746 [ThinLTO] Import composite types as declarations
Summary:
When reading the metadata bitcode, create a type declaration when
possible for composite types when we are importing. Doing this in
the bitcode reader saves memory. Also it works naturally in the case
when the type ODR map contains a definition for the same composite type
because it was used in the importing module (buildODRType will
automatically use the existing definition and not create a type
declaration).

For Chromium built with -g2, this reduces the aggregate size of the
generated native object files by 66% (from 31G to 10G). It reduced
the time through the ThinLTO link and backend phases by about 20% on
my machine.

Reviewers: mehdi_amini, dblaikie, aprantl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D27775

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289993 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-16 21:25:01 +00:00
Peter Collingbourne
08850161ec LTO: Add support for multi-module bitcode files.
Differential Revision: https://reviews.llvm.org/D27313

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289621 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-14 01:17:59 +00:00
Peter Collingbourne
51b7aa5d98 Bitcode: Introduce BitcodeModule::{has,get}Summary().
These are equivalent to hasGlobalValueSummary() and getModuleSummaryIndex().

Differential Revision: https://reviews.llvm.org/D27242

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288317 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-01 06:00:53 +00:00
Peter Collingbourne
c67ca17e5e Add llvm-modextract tool.
This program is for testing features that rely on multi-module bitcode files.
It takes a multi-module bitcode file, extracts one of the modules and writes
it to the output file.

Differential Revision: https://reviews.llvm.org/D26778

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288201 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-29 21:54:33 +00:00
Peter Collingbourne
e6480e2add Bitcode: Introduce BitcodeWriter interface.
This interface allows clients to write multiple modules to a single
bitcode file. Also introduce the llvm-cat utility which can be used
to create a bitcode file containing multiple modules.

Differential Revision: https://reviews.llvm.org/D26179

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288195 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-29 20:43:47 +00:00
Peter Collingbourne
a0894a8257 Bitcode: Change expected layout of module blocks.
We now expect each module's identification block to appear immediately before
the module block. Any module block that appears without an identification block
immediately before it is interpreted as if it does not have a module block.

Also change the interpretation of VST and function offsets in bitcode.
The offset is always taken as relative to the start of the identification
(or module if not present) block, minus one word. This corresponds to the
historical interpretation of offsets, i.e. relative to the start of the file.

These changes allow for bitcode modules to be concatenated by copying bytes.

Differential Revision: https://reviews.llvm.org/D27184

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288098 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-29 02:27:04 +00:00
Peter Collingbourne
34c84f49d8 Bitcode: Introduce initial multi-module reader API.
Implement getLazyBitcodeModule() and parseBitcodeFile() in terms of it.

Differential Revision: https://reviews.llvm.org/D26719

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287156 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-16 21:44:45 +00:00
Peter Collingbourne
dead081fb2 Bitcode: Change module reader functions to return an llvm::Expected.
Differential Revision: https://reviews.llvm.org/D26562

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286752 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-13 07:00:17 +00:00
Peter Collingbourne
aeb2eff8d2 Bitcode: Change getModuleSummaryIndex() to return an llvm::Expected.
Differential Revision: https://reviews.llvm.org/D26539

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286624 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-11 19:50:39 +00:00
Peter Collingbourne
9b252f03d9 Bitcode: Clean up error handling for certain bitcode query functions.
The functions getBitcodeTargetTriple(), isBitcodeContainingObjCCategory(),
getBitcodeProducerString() and hasGlobalValueSummary() now return errors
via their return value rather than via the diagnostic handler.

To make this work, re-implement these functions using non-member functions
so that they can be used without the LLVMContext required by BitcodeReader.

Differential Revision: https://reviews.llvm.org/D26532

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286623 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-11 19:50:24 +00:00
Teresa Johnson
a547919737 Split Bitcode/ReaderWriter.h into separate reader and writer headers
Summary:
Split ReaderWriter.h which contains the APIs into both the BitReader and
BitWriter libraries into BitcodeReader.h and BitcodeWriter.h.

This is to address Chandler's concern about sharing the same API header
between multiple libraries (BitReader and BitWriter). That concern is
why we create a single bitcode library in our downstream build of clang,
which led to r286297 being reverted as it added a dependency that
created a cycle only when there is a single bitcode library (not two as
in upstream).

Reviewers: mehdi_amini

Subscribers: dlj, mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D26502

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286566 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-11 05:34:58 +00:00