InProcessMemoryManager used to make separate memory allocation calls for each
permission level (RW, RX, RO), which could lead to target-out-of-range errors
if data and code were placed too far apart (this was the source of failures in
the JITLink/AArch64 testcase when it was first landed).
This patch updates InProcessMemoryManager to allocate a single slab which is
subdivided between text and data. This should guarantee that accesses remain
in-range provided that individual object files do not exceed 1Mb in size.
This patch also re-enables the JITLink/AArch64 testcase.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374948 91177308-0d34-0410-b5e6-96231b3b80d8
The windows bots are failing due to a memory layout error. Temporarily disabling
while I investigate whether this can be worked around, or whether the test
should be disabled on Windows.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374500 91177308-0d34-0410-b5e6-96231b3b80d8
This implementation has support for all relocation types except TLV.
Compact unwind sections are not yet supported, so exceptions/unwinding will not
work.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374476 91177308-0d34-0410-b5e6-96231b3b80d8
In the Atom model the symbols, content and relocations of a relocatable object
file are represented as a graph of atoms, where each Atom represents a
contiguous block of content with a single name (or no name at all if the
content is anonymous), and where edges between Atoms represent relocations.
If more than one symbol is associated with a contiguous block of content then
the content is broken into multiple atoms and layout constraints (represented by
edges) are introduced to ensure that the content remains effectively contiguous.
These layout constraints must be kept in mind when examining the content
associated with a symbol (it may be spread over multiple atoms) or when applying
certain relocation types (e.g. MachO subtractors).
This patch replaces the Atom model in JITLink with a blocks-and-symbols model.
The blocks-and-symbols model represents relocatable object files as bipartite
graphs, with one set of nodes representing contiguous content (Blocks) and
another representing named or anonymous locations (Symbols) within a Block.
Relocations are represented as edges from Blocks to Symbols. This scheme
removes layout constraints (simplifying handling of MachO alt-entry symbols,
and hopefully ELF sections at some point in the future) and simplifies some
relocation logic.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@373689 91177308-0d34-0410-b5e6-96231b3b80d8
If content sections have lower alignment than zero-fill sections then bump the
overall segment alignment to avoid under-aligning the zero-fill sections.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@370072 91177308-0d34-0410-b5e6-96231b3b80d8
This patch replaces the JITDylib::DefinitionGenerator typedef with a class of
the same name, and adds support for attaching a sequence of DefinitionGeneration
objects to a JITDylib.
This patch also adds a new definition generator,
StaticLibraryDefinitionGenerator, that can be used to add symbols fom a static
library to a JITDylib. An object from the static library will be added (via
a supplied ObjectLayer reference) whenever a symbol from that object is
referenced.
To enable testing, lli is updated to add support for the --extra-archive option
when running in -jit-kind=orc-lazy mode.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@368707 91177308-0d34-0410-b5e6-96231b3b80d8
MachO/x86-64 UNSIGNED relocs are almost always 64-bit (length=3), but UNSIGNED
relocs of length=2 are allowed if the target resides in the low 32-bits. This
patch adds support for such relocations in JITLink (previously they would have
triggered an unsupported relocation error).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367764 91177308-0d34-0410-b5e6-96231b3b80d8
increase the MachO/x86-64 stub alignment to 8.
Stub alignment should be guaranteed for any section containing RuntimeDyld
stubs/GOT-entries. To do this we should pad and align all sections containing
stubs, not just code sections.
This commit also bumps the MachO/x86-64 stub alignment to 8, so that GOT entries
will be aligned.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@362139 91177308-0d34-0410-b5e6-96231b3b80d8
Previously we had only honored alignments on individual atoms, but
tools/runtimes may assume that the section alignment is respected too.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@360555 91177308-0d34-0410-b5e6-96231b3b80d8
Also updates RuntimeDyldChecker and llvm-rtdyld to support zero-fill tests by
returning a content address of zero (but no error) for zero-fill atoms, and
treating loads from zero as returning zero.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@360547 91177308-0d34-0410-b5e6-96231b3b80d8
If a MachO section has the no-dead-strip attribute set then its atoms should
be preserved, regardless of whether they're public or referenced elsewhere in
the object.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@360477 91177308-0d34-0410-b5e6-96231b3b80d8
Subtractor relocation addends are signed, so we need to read them via signed
int pointers. Accidentally treating 32-bit addends as unsigned leads to
out-of-range errors when we try to add very large (>INT32_MAX) bogus addends.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@360392 91177308-0d34-0410-b5e6-96231b3b80d8
This patch modifies MachOAtomGraphBuilder to use setLayoutNext rather than
addEdge, and fixes a bug in the section layout algorithm that could result in
atoms appearing more than once in the section ordering (which resulted in those
atoms being assigned invalid addresses during layout).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@360205 91177308-0d34-0410-b5e6-96231b3b80d8
The MachO .alt_entry directive is applied to a symbol to indicate that it is
locked (in terms of address layout and liveness) to its predecessor atom. I.e.
it is an alternate entry point, at a fixed offset, for the previous atom.
This patch updates MachOAtomGraphBuilder to check for the .alt_entry flag on
symbols and add a corresponding LayoutNext edge to the atom-graph. It also
updates MachOAtomGraphBuilder_x86_64 to generalize handling of the
X86_64_RELOC_SUBTRACTOR relocation: previously either the minuend or
subtrahend of the subtraction had to be the same as the atom being fixed up,
now it is only necessary for the minuend or subtrahend to be locked (via any
chain of alt_entry directives) to the atom being fixed up.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@360194 91177308-0d34-0410-b5e6-96231b3b80d8
Several X86_64_RELOC_SUBTRACTOR tests for subtrahend handling were incorrectly
labeled as tests for kinds of minuend handling.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@360160 91177308-0d34-0410-b5e6-96231b3b80d8
We use both -long-option and --long-option in tests. Switch to --long-option for consistency.
In the "llvm-readelf" mode, -long-option is discouraged as it conflicts with grouped short options and it is not accepted by GNU readelf.
While updating the tests, change llvm-readobj -s to llvm-readobj -S to reduce confusion ("s" is --section-headers in llvm-readobj but --symbols in llvm-readelf).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@359649 91177308-0d34-0410-b5e6-96231b3b80d8
ObjectLinkingLayer::Plugin provides event notifications when objects are loaded,
emitted, and removed. It also provides a modifyPassConfig callback that allows
plugins to modify the JITLink pass configuration.
This patch moves eh-frame registration into its own plugin, and teaches
llvm-jitlink to only add that plugin when performing execution runs on
non-Windows platforms. This should allow us to re-enable the test case that was
removed in r359198.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@359357 91177308-0d34-0410-b5e6-96231b3b80d8
This should fix the MachO/x86-64 eh-frame regression test by ensuring that
the symbols __ZTIi and ___gxx_personality_v0 are defined on all platforms.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@359169 91177308-0d34-0410-b5e6-96231b3b80d8
Frame Descriptor Entries (FDEs) have a pointer back to a Common Information
Entry (CIE) that describes how the rest FDE should be parsed. JITLink had been
assuming that FDEs always referred to the most recent CIE encountered, but the
spec allows them to point back to any previously encountered CIE. This patch
fixes JITLink to look up the correct CIE for the FDE.
The testcase is a MachO binary with an FDE that refers to a CIE that is not the
one immediately proceeding it (the layout can be viewed wit
'dwarfdump --eh-frame <testcase>'. This test case had to be a binary as llvm-mc
now sorts FDEs (as of r356216) to ensure FDEs *do* point to the most recent CIE.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@359105 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
JITLink is a jit-linker that performs the same high-level task as RuntimeDyld:
it parses relocatable object files and makes their contents runnable in a target
process.
JITLink aims to improve on RuntimeDyld in several ways:
(1) A clear design intended to maximize code-sharing while minimizing coupling.
RuntimeDyld has been developed in an ad-hoc fashion for a number of years and
this had led to intermingling of code for multiple architectures (e.g. in
RuntimeDyldELF::processRelocationRef) in a way that makes the code more
difficult to read, reason about, extend. JITLink is designed to isolate
format and architecture specific code, while still sharing generic code.
(2) Support for native code models.
RuntimeDyld required the use of large code models (where calls to external
functions are made indirectly via registers) for many of platforms due to its
restrictive model for stub generation (one "stub" per symbol). JITLink allows
arbitrary mutation of the atom graph, allowing both GOT and PLT atoms to be
added naturally.
(3) Native support for asynchronous linking.
JITLink uses asynchronous calls for symbol resolution and finalization: these
callbacks are passed a continuation function that they must call to complete the
linker's work. This allows for cleaner interoperation with the new concurrent
ORC JIT APIs, while still being easily implementable in synchronous style if
asynchrony is not needed.
To maximise sharing, the design has a hierarchy of common code:
(1) Generic atom-graph data structure and algorithms (e.g. dead stripping and
| memory allocation) that are intended to be shared by all architectures.
|
+ -- (2) Shared per-format code that utilizes (1), e.g. Generic MachO to
| atom-graph parsing.
|
+ -- (3) Architecture specific code that uses (1) and (2). E.g.
JITLinkerMachO_x86_64, which adds x86-64 specific relocation
support to (2) to build and patch up the atom graph.
To support asynchronous symbol resolution and finalization, the callbacks for
these operations take continuations as arguments:
using JITLinkAsyncLookupContinuation =
std::function<void(Expected<AsyncLookupResult> LR)>;
using JITLinkAsyncLookupFunction =
std::function<void(const DenseSet<StringRef> &Symbols,
JITLinkAsyncLookupContinuation LookupContinuation)>;
using FinalizeContinuation = std::function<void(Error)>;
virtual void finalizeAsync(FinalizeContinuation OnFinalize);
In addition to its headline features, JITLink also makes other improvements:
- Dead stripping support: symbols that are not used (e.g. redundant ODR
definitions) are discarded, and take up no memory in the target process
(In contrast, RuntimeDyld supported pointer equality for weak definitions,
but the redundant definitions stayed resident in memory).
- Improved exception handling support. JITLink provides a much more extensive
eh-frame parser than RuntimeDyld, and is able to correctly fix up many
eh-frame sections that RuntimeDyld currently (silently) fails on.
- More extensive validation and error handling throughout.
This initial patch supports linking MachO/x86-64 only. Work on support for
other architectures and formats will happen in-tree.
Differential Revision: https://reviews.llvm.org/D58704
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358818 91177308-0d34-0410-b5e6-96231b3b80d8
This patch reduces the number of functions in the interface between RuntimeDyld
and RuntimeDyldChecker by combining "GetXAddress" and "GetXContent" functions
into "GetXInfo" functions that return a struct describing both the address and
content. The GetStubOffset function is also replaced with a pair of utilities,
GetStubInfo and GetGOTInfo, that fit the new scheme. For RuntimeDyld both of
these functions will return the same result, but for the new JITLink linker
(https://reviews.llvm.org/D58704) these will provide the addresses of PLT stubs
and GOT entries respectively.
For JITLink's use, a 'got_addr' utility has been added to the rtdyld-check
language, and the syntax of 'got_addr' and 'stub_addr' has been changed: both
functions now take two arguments, a 'stub container name' and a target symbol
name. For llvm-rtdyld/RuntimeDyld the stub container name is the object file
name and section name, separated by a slash. E.g.:
rtdyld-check: *{8}(stub_addr(foo.o/__text, y)) = y
For the upcoming llvm-jitlink utility, which creates stubs on a per-file basis
rather than a per-section basis, the container name is just the file name. E.g.:
jitlink-check: *{8}(got_addr(foo.o, y)) = y
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358295 91177308-0d34-0410-b5e6-96231b3b80d8
Prior to this change, the "Symbol" field of a relocation would always be
assumed to be a symbol name, and if no such symbol existed, the
relocation would reference index 0. This confused me when I tried to use
a literal symbol index in the field: since "0x1" was not a known symbol
name, the symbol index was set as 0. This change falls back to treating
unknown symbol names as integers, and emits an error if the name is not
found and the string is not an integer.
Note that the Symbol field is optional, so if a relocation doesn't
reference a symbol, it shouldn't be specified. The new error required a
number of test updates.
Reviewed by: grimar, ruiu
Differential Revision: https://reviews.llvm.org/D58510
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355938 91177308-0d34-0410-b5e6-96231b3b80d8
lldb on Windows uses the ExecutionEngine for expression evaluation
and hits the llvm_unreachable due to this relocation. Thus, implement
the relocation and add a test to verify it's function.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@348904 91177308-0d34-0410-b5e6-96231b3b80d8
Doesn't build on Windows. The call to 'lookup' is ambiguous. Clang and
MSVC agree, anyway.
http://lab.llvm.org:8011/builders/clang-x64-windows-msvc/builds/787
C:\b\slave\clang-x64-windows-msvc\build\llvm.src\unittests\ExecutionEngine\Orc\CoreAPIsTest.cpp(315): error C2668: 'llvm::orc::ExecutionSession::lookup': ambiguous call to overloaded function
C:\b\slave\clang-x64-windows-msvc\build\llvm.src\include\llvm/ExecutionEngine/Orc/Core.h(823): note: could be 'llvm::Expected<llvm::JITEvaluatedSymbol> llvm::orc::ExecutionSession::lookup(llvm::ArrayRef<llvm::orc::JITDylib *>,llvm::orc::SymbolStringPtr)'
C:\b\slave\clang-x64-windows-msvc\build\llvm.src\include\llvm/ExecutionEngine/Orc/Core.h(817): note: or 'llvm::Expected<llvm::JITEvaluatedSymbol> llvm::orc::ExecutionSession::lookup(const llvm::orc::JITDylibSearchList &,llvm::orc::SymbolStringPtr)'
C:\b\slave\clang-x64-windows-msvc\build\llvm.src\unittests\ExecutionEngine\Orc\CoreAPIsTest.cpp(315): note: while trying to match the argument list '(initializer list, llvm::orc::SymbolStringPtr)'
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@345078 91177308-0d34-0410-b5e6-96231b3b80d8
In the new scheme the client passes a list of (JITDylib&, bool) pairs, rather
than a list of JITDylibs. For each JITDylib the boolean indicates whether or not
to match against non-exported symbols (true means that they should be found,
false means that they should not). The MatchNonExportedInJD and MatchNonExported
parameters on lookup are removed.
The new scheme is more flexible, and easier to understand.
This patch also updates JITDylib search orders to be lists of (JITDylib&, bool)
pairs to match the new lookup scheme. Error handling is also plumbed through
the LLJIT class to allow regression tests to fail predictably when a lookup from
a lazy call-through fails.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@345077 91177308-0d34-0410-b5e6-96231b3b80d8
Non-loaded sections (whose unused load-address defaults to zero) should not
be taken into account when calculating ImageBase, or ImageBase will be
incorrectly set to 0.
Patch by Andrew Scheidecker. Thanks Andrew!
https://reviews.llvm.org/D51343
+ // The Sections list may contain sections that weren't loaded for
+ // whatever reason: they may be debug sections, and ProcessAllSections
+ // is false, or they may be sections that contain 0 bytes. If the
+ // section isn't loaded, the load address will be 0, and it should not
+ // be included in the ImageBase calculation.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@344995 91177308-0d34-0410-b5e6-96231b3b80d8
minimal.ll contains a main function that returns zero, and
single-function-call.ll contains a main function that calls a foo function that
returns zero. These minimal tests can help to rule out some trivial JIT bugs
when other tests fail.
This commit also renames hello.ll to global-ctors-and-dtors.ll, which better
reflects what it is actually testing.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@344863 91177308-0d34-0410-b5e6-96231b3b80d8
CompileOnDemandLayer2 now supports user-supplied partition functions (the
original CompileOnDemandLayer already supported these).
Partition functions are called with the list of requested global values
(i.e. global values that currently have queries waiting on them) and have an
opportunity to select extra global values to materialize at the same time.
Also adds testing infrastructure for the new feature to lli.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343396 91177308-0d34-0410-b5e6-96231b3b80d8
Modifies lit to add a 'thread_support' feature that can be used in lit test
REQUIRES clauses. The thread_support flag is set if -DLLVM_ENABLE_THREADS=ON
and unset if -DLLVM_ENABLE_THREADS=OFF. The lit flag is used to disable the
multiple-compile-threads-basic.ll testcase when threading is disabled.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343122 91177308-0d34-0410-b5e6-96231b3b80d8
This doesn't work well in builds configured with LLVM_ENABLE_THREADS=OFF,
causing the following assert when running
ExecutionEngine/OrcLazy/multiple-compile-threads-basic.ll:
lib/ExecutionEngine/Orc/Core.cpp:1748: Expected<llvm::JITEvaluatedSymbol>
llvm::orc::lookup(const llvm::orc::JITDylibList &, llvm::orc::SymbolStringPtr):
Assertion `ResultMap->size() == 1 && "Unexpected number of results"' failed.
> LLJIT and LLLazyJIT can now be constructed with an optional NumCompileThreads
> arguments. If this is non-zero then a thread-pool will be created with the
> given number of threads, and compile tasks will be dispatched to the thread
> pool.
>
> To enable testing of this feature, two new flags are added to lli:
>
> (1) -compile-threads=N (N = 0 by default) controls the number of compile threads
> to use.
>
> (2) -thread-entry can be used to execute code on additional threads. For each
> -thread-entry argument supplied (multiple are allowed) a new thread will be
> created and the given symbol called. These additional thread entry points are
> called after static constructors are run, but before main.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343099 91177308-0d34-0410-b5e6-96231b3b80d8
for lazy compilation, rather than a callback manager.
The new mechanism does not block compile threads, and does not require
function bodies to be renamed.
Future modifications should allow laziness on a per-module basis to work
without any modification of the input module.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343065 91177308-0d34-0410-b5e6-96231b3b80d8
LLJIT and LLLazyJIT can now be constructed with an optional NumCompileThreads
arguments. If this is non-zero then a thread-pool will be created with the
given number of threads, and compile tasks will be dispatched to the thread
pool.
To enable testing of this feature, two new flags are added to lli:
(1) -compile-threads=N (N = 0 by default) controls the number of compile threads
to use.
(2) -thread-entry can be used to execute code on additional threads. For each
-thread-entry argument supplied (multiple are allowed) a new thread will be
created and the given symbol called. These additional thread entry points are
called after static constructors are run, but before main.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343058 91177308-0d34-0410-b5e6-96231b3b80d8
This patch adds support for ORC JIT for mips/mips64 architecture.
In common code $static is changed to __ORCstatic because on MIPS
architecture "$" is a reserved character.
Patch by Luka Ercegovcevic
Differential Revision: https://reviews.llvm.org/D49665
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@341934 91177308-0d34-0410-b5e6-96231b3b80d8
