Introduce symbol table data structures that can be potentially written to
disk, have the LTO library build those data structures using temporarily
constructed modules and redirect the LTO library implementation to go through
those data structures. This allows us to remove the LLVMContext and Modules
owned by InputFile.
With this change I measured a peak memory consumption decrease from 5.4GB to
2.8GB in a no-op incremental ThinLTO link of Chromium on Linux. The impact on
memory consumption is larger in COFF linkers where we are currently forced
to materialize all metadata in order to read linker options. Peak memory
consumption linking a large piece of Chromium for Windows with full LTO and
debug info decreases from >64GB (OOM) to 15GB.
Part of PR27551.
Differential Revision: https://reviews.llvm.org/D31364
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@299168 91177308-0d34-0410-b5e6-96231b3b80d8
BIND_OPCODE_DONE/REBASE_OPCODE_DONE may appear at the end of the opcode array,
but they are not required to. The linker only adds them as padding to align the
opcodes to pointer size.
This fixes rdar://problem/31285560.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@299104 91177308-0d34-0410-b5e6-96231b3b80d8
Mostly this change adds support converting to and from
YAML which will allow us to write more test cases for
the WebAssembly MC and lld ports.
Better support for objdump, readelf, and nm will be in
followup CLs.
I had to update the two wasm test binaries because they
used the old style 'name' section which is no longer
supported.
Differential Revision: https://reviews.llvm.org/D31099
Patch by Sam Clegg
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@299101 91177308-0d34-0410-b5e6-96231b3b80d8
Do that by creating a global_values, which is similar to
global_objects, but also iterates over aliases and ifuncs.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@299018 91177308-0d34-0410-b5e6-96231b3b80d8
rebase entry errors and test cases for each of the error checks.
Also verified with Nick Kledzik that a BIND_OPCODE_SET_ADDEND_SLEB
opcode is legal in a lazy bind table, so code that had that as an error
check was removed.
With MachORebaseEntry and MachOBindEntry classes now returning
an llvm::Error in all cases for malformed input the variables Malformed
and logic to set use them is no longer needed and has been removed
from those classes.
Also in a few places, removed the redundant Done assignment to true
when also calling moveToEnd() as it does that assignment.
This only leaves the dyld compact export entries left to have
error handling yet to be added for the dyld compact info.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@298883 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
The cumulative size of the bitcode files for a very large application
can be huge, particularly with -g. In a distributed build environment,
all of these files must be sent to the remote build node that performs
the thin link step, and this can exceed size limits.
The thin link actually only needs the summary along with a bitcode
symbol table. Until we have a proper bitcode symbol table, simply
stripping the debug metadata results in significant size reduction.
Add support for an option to additionally emit minimized bitcode
modules, just for use in the thin link step, which for now just strips
all debug metadata. I plan to add a cc1 option so this can be invoked
easily during the compile step.
However, care must be taken to ensure that these minimized thin link
bitcode files produce the same index as with the original bitcode files,
as these original bitcode files will be used in the backends.
Specifically:
1) The module hash used for caching is typically produced by hashing the
written bitcode, and we want to include the hash that would correspond
to the original bitcode file. This is because we want to ensure that
changes in the stripped portions affect caching. Added plumbing to emit
the same module hash in the minimized thin link bitcode file.
2) The module paths in the index are constructed from the module ID of
each thin linked bitcode, and typically is automatically generated from
the input file path. This is the path used for finding the modules to
import from, and obviously we need this to point to the original bitcode
files. Added gold-plugin support to take a suffix replacement during the
thin link that is used to override the identifier on the MemoryBufferRef
constructed from the loaded thin link bitcode file. The assumption is
that the build system can specify that the minimized bitcode file has a
name that is similar but uses a different suffix (e.g. out.thinlink.bc
instead of out.o).
Added various tests to ensure that we get identical index files out of
the thin link step.
Reviewers: mehdi_amini, pcc
Subscribers: Prazek, llvm-commits
Differential Revision: https://reviews.llvm.org/D31027
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@298638 91177308-0d34-0410-b5e6-96231b3b80d8
and test cases for each of the error checks.
To do this more plumbing was needed so that the segment indexes and
segment offsets can be checked. Basically what was done was the SegInfo
from llvm-objdump’s MachODump.cpp was moved into libObject for Mach-O
objects as BindRebaseSegInfo and it is only created when an iterator for
bind or rebase entries are created.
This commit really only adds the error checking and test cases for the
bind table entires and the checking for the lazy bind and weak bind entries
are still to be fully done as well as the rebase entires. Though some of
the plumbing for those are added with this commit. Those other error
checks and test cases will be added in follow on commits.
Note, the two llvm_unreachable() calls should now actually be unreachable
with the error checks in place and would take a logic bug in the error
checking code to be reached if the segment indexes and segment
offsets are used from a checked bind entry. Comments have been added
to the methods that require the arguments to have been checked
prior to calling.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@298292 91177308-0d34-0410-b5e6-96231b3b80d8
On Solaris ld (and some other tools that use the underlying utility
libraries, such as elfdump) chokes on an archive library that has no
symbol table. The Solaris tools always create one, even if it's empty.
That bug has been fixed in the latest development line, and can
probably be backported to a supported release, but it would be nice if
LLVM's archiver could emit the empty symbol table, too.
Patch by Danek Duvall!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@297773 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
In a .symver assembler directive like:
.symver name, name2@@nodename
"name2@@nodename" should get the same symbol binding as "name".
While the ELF object writer is updating the symbol binding for .symver
aliases before emitting the object file, not doing so when the module
inline assembly is handled by the RecordStreamer is causing the wrong
behavior in *LTO mode.
E.g. when "name" is global, "name2@@nodename" must also be marked as
global. Otherwise, the symbol is skipped when iterating over the LTO
InputFile symbols (InputFile::Symbol::shouldSkip). So, for example,
when performing any *LTO via the gold-plugin, the versioned symbol
definition is not recorded by the plugin and passed back to the
linker. If the object was in an archive, and there were no other symbols
needed from that object, the object would not be included in the final
link and references to the versioned symbol are undefined.
The llvm-lto2 tests added will give an error about an unused symbol
resolution without the fix.
Reviewers: rafael, pcc
Reviewed By: pcc
Subscribers: mehdi_amini, llvm-commits
Differential Revision: https://reviews.llvm.org/D30485
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@297332 91177308-0d34-0410-b5e6-96231b3b80d8
other tables. Providing a helpful error message to what the error is and
where the error occurred based on which opcode it was associated with.
There have been handful of bug fixes dealing with bad bind info in
object files, r294021 and r249845, which only put a band aid on the
problem after a bad bind table was created after unpacking from
its compact info. In these cases a bind table should have never been
created and an error should have simply been generated.
This change puts in place the plumbing to allow checking and returning
of an error when the compact info is unpacked. This follows the model
of iterators that can fail that Lang Hanes designed when fixing the problem
for bad archives r275316 (or r275361).
This change uses one of the existing test cases that now causes an
error instead of printing <<bad library ordinal>> after a bad bind table
is created. The error uses the offset into the opcode table as shown with
the macOS dyldinfo(1) tool to indicate where the error is and which
opcode and which parameter is in error.
For example the exiting test case has this lazy binding opcode table:
% dyldinfo -opcodes test/tools/llvm-objdump/Inputs/bad-ordinal.macho-x86_64
…
lazy binding opcodes:
0x0000 BIND_OPCODE_SET_SEGMENT_AND_OFFSET_ULEB(0x02, 0x00000010)
0x0002 BIND_OPCODE_SET_DYLIB_ORDINAL_IMM(2)
In the test case the binary only has one library so setting the library
ordinal to the value of 2 in the BIND_OPCODE_SET_DYLIB_ORDINAL_IMM
opcode at 0x0002 above is an error. This now produces this error message:
% llvm-objdump -lazy-bind bad-ordinal.macho-x86_64
…
llvm-objdump: 'bad-ordinal.macho-x86_64': truncated or malformed object (for BIND_OPCODE_SET_DYLIB_ORDINAL_ULEB bad library ordinal: 2 (max 1) for opcode at: 0x2)
This change provides the plumbing for the error handling and one example
of an error message. Other error checks and test cases will be added in follow
on commits.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296527 91177308-0d34-0410-b5e6-96231b3b80d8
For whatever reason ld64 requires that member headers (not the member
themselves) should be aligned. The only way to do that is to edit the
previous member so that it ends at an aligned boundary.
Since modifying data put in an archive is an undesirable property,
llvm-ar should only do it when it is absolutely necessary.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295765 91177308-0d34-0410-b5e6-96231b3b80d8
in this case for CPU_SUBTYPE_ARM64_ALL.
For this cpusubtype it should default to a cyclone CPU
to give proper disassembly without a -mcpu= flag.
rdar://27767188
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@294771 91177308-0d34-0410-b5e6-96231b3b80d8
This makes sure we get the same redefinition rules regardless of who
is printing (asm parser, codegen) and to what (asm, obj).
This fixes an unintentional regression in r293936.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@294752 91177308-0d34-0410-b5e6-96231b3b80d8
ld64 requires its archive members to be 8-byte aligned for 64-bit
content and 4-byte aligned for 32-bit content. Opt for the larger
alignment requirement. This ensures that ld64 can consume archives
generated by llvm-ar.
Thanks to Kevin Enderby for the hint about the ld64/cctools behaviours!
Resolves PR28361!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@294615 91177308-0d34-0410-b5e6-96231b3b80d8
Add a note about the reason for the divergence from the specification
for ld64. Addresses post-commit review comments from Davide. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@294594 91177308-0d34-0410-b5e6-96231b3b80d8
cctools would pad the string table to a sizeof(int32_t) (explicitly
printed out by cctools rather than 4). This adjusts the string table to
make it more compatible with cctools, but is insufficient to make ld64
happy.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@294557 91177308-0d34-0410-b5e6-96231b3b80d8
it was printing the field name fileoff instead of filesize. The original check
was added in r278557.
This was found in tracking down the problem that lead to the fix in
r293842 - [dsymutil] Fix __LINKEDIT vmsize in dsymutil upgrade path
rdar://30386075
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@294354 91177308-0d34-0410-b5e6-96231b3b80d8
for CPU_SUBTYPE_ARM_V7S and CPU_SUBTYPE_ARM_V7K.
For these two cpusubtypes they should default to a cortex-a7 CPU
to give proper disassembly without a -mcpu= flag.
rdar://27431703
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292993 91177308-0d34-0410-b5e6-96231b3b80d8
in llvm-objdump for Mach-O files add the printing of the
x86_thread_state32_t in the same format as
otool-classic(1) on darwin.
To do this the 32-bit x86 general tread state
needed to be defined in include/llvm/Support/MachO.h .
rdar://30110111
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292829 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Add a new load command LC_BUILD_VERSION. It is a generic version of
LC_*_VERSION_MIN load_command used on Apple platforms. Instead of having
a seperate load command for each platform, LC_BUILD_VERSION is recording
platform info as an enum. It also records SDK version, min_os, and tools
that used to build the binary.
rdar://problem/29781291
Reviewers: enderby
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D29044
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292824 91177308-0d34-0410-b5e6-96231b3b80d8
It describes a region of arbitrary data included in a Mach-O file.
Its initial use is to record extra data in MH_CORE files.
rdar://30001545
rdar://30001731
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292500 91177308-0d34-0410-b5e6-96231b3b80d8
An ELFObjectFile can now create SubtargetFeatures from the available
ARM build attributes, in a similar manner to MIPS. I've moved the
MIPS code into its own function and the ARM handler also has a
separate function.
Differential Revision: https://reviews.llvm.org/D28291
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292403 91177308-0d34-0410-b5e6-96231b3b80d8
Enable an ELFObjectFile to read the its arm build attributes to
produce a target triple with a specific ARM architecture.
llvm-objdump now uses this functionality to automatically produce
a more accurate target.
Differential Revision: https://reviews.llvm.org/D28769
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292366 91177308-0d34-0410-b5e6-96231b3b80d8
No any changes, will follow up with D28807 commit containing APLi change for clang
to fix build issues happened.
Original commit message:
[Support/Compression] - Change zlib API to return Error instead of custom status.
Previously API returned custom enum values.
Patch changes it to return Error with string description.
That should help users to report errors in universal way.
Differential revision: https://reviews.llvm.org/D28684
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292226 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Revert [ARM] Fix ubig32_t read in ARMAttributeParser
Now using support functions to read data instead of trying to
perform casts.
===========================================================
Revert [ARM] Enable objdump to construct triple for ARM
Now that The ARMAttributeParser has been moved into the library,
it has been modified so that it can parse the attributes without
printing them and stores them in a map. ELFObjectFile now queries
the attributes to fill out the architecture details of a provided
triple for 'arm' and 'thumb' targets. llvm-objdump uses this new
functionality.
Subscribers: llvm-commits, samparker, aemerson, mgorny
Differential Revision: https://reviews.llvm.org/D28683
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291911 91177308-0d34-0410-b5e6-96231b3b80d8
Now that The ARMAttributeParser has been moved into the library,
it has been modified so that it can parse the attributes without
printing them and stores them in a map. ELFObjectFile now queries
the attributes to fill out the architecture details of a provided
triple for 'arm' and 'thumb' targets. llvm-objdump uses this new
functionality.
Differential Revision: https://reviews.llvm.org/D28281
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291898 91177308-0d34-0410-b5e6-96231b3b80d8
Decompressor intention is to reduce duplication of code.
Currently LLD has own implementation of decompressor
for compressed debug sections.
This class helps to avoid it and share the code.
LLD patch for reusing it is D28106
Differential revision: https://reviews.llvm.org/D28105
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291675 91177308-0d34-0410-b5e6-96231b3b80d8
Running a Debug build of objdump -objc-meta-data with a large Mach-O file is
currently unnecessarily slow.
With some local test input, this change reduces the run time from 75-85s down
to 15-20s.
The two changes are:
Assert on pointer equality not array equality
Replace vector<pair<address, symbol>> with DenseMap<address, symbol>
Additionally, use a std::unique_ptr rather than handling the memory manually.
Patch by Dave Lee!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291398 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
In order to simplify distributed build system integration, where actions
may be scheduled before the Thin Link which determines the list of
objects selected by the linker. The gold plugin currently will emit
0-sized index files for objects not selected by the link, to enable
checking for expected output files by the build system. If the build
system then schedules a backend action for these bitcode files, we want
to be able to fall back to normal compilation instead of failing.
This is the LLVM side support for optionally enabling fallback
instead of issuing an error. Return a null CombinedIndex from
llvm::getModuleSummaryIndexForFile under the option when the file
is empty. Clang can then ignore the index when it is null.
Clang patch is D28362.
Reviewers: mehdi_amini
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D28410
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291302 91177308-0d34-0410-b5e6-96231b3b80d8
The Mach-O command line flag like "-arch armv7m" does not match the
arch name part of its llvm Triple which is "thumbv7m-apple-darwin”.
I think the best way to fix this is to have
llvm::object::MachOObjectFile::getArchTriple() optionally return the
name of the Mach-O arch flag that would be used with -arch that
matches the CPUType and CPUSubType. Then change
llvm::object::MachOUniversalBinary::ObjectForArch::getArchTypeName()
to use that and change it to getArchFlagName() as the type name is
really part of the Triple and the -arch flag name is a Mach-O thing
for a specific Triple with a specific Mcpu value.
rdar://29663637
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290001 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
When reading the metadata bitcode, create a type declaration when
possible for composite types when we are importing. Doing this in
the bitcode reader saves memory. Also it works naturally in the case
when the type ODR map contains a definition for the same composite type
because it was used in the importing module (buildODRType will
automatically use the existing definition and not create a type
declaration).
For Chromium built with -g2, this reduces the aggregate size of the
generated native object files by 66% (from 31G to 10G). It reduced
the time through the ThinLTO link and backend phases by about 20% on
my machine.
Reviewers: mehdi_amini, dblaikie, aprantl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D27775
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289993 91177308-0d34-0410-b5e6-96231b3b80d8
Changes all static helper functions in MachOObjectFile.cpp that expect a
non-null MachOObjectFile pointer to take a reference instead.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288608 91177308-0d34-0410-b5e6-96231b3b80d8
The file does not seems to use c++ iostreams (and is is llvm policy to avoid
that). Committing as obvious.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288364 91177308-0d34-0410-b5e6-96231b3b80d8
