(1) Adds comments for the API.
(2) Removes the setArch method: This is redundant: the setArchStr method on the
triple should be used instead.
(3) Turns EmulatedTLS on by default. This matches EngineBuilder's behavior.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343423 91177308-0d34-0410-b5e6-96231b3b80d8
Explicitly defines ThreadSafeModule's move-assignment operator to move fields in
reverse order. This is required to ensure that the context field outlives the
module field.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343149 91177308-0d34-0410-b5e6-96231b3b80d8
destroyed before its ThreadSharedContext.
Destroying the context first is an error if this ThreadSafeModule is the only
owner of its underlying context.
Add a unit test for ThreadSafeModule/ThreadSafeContext to catch this and other
basic usage issues.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343129 91177308-0d34-0410-b5e6-96231b3b80d8
implementation as lazy compile callbacks, and a "lazy re-exports" utility that
builds lazy call-throughs.
Lazy call-throughs are similar to lazy compile callbacks (and are based on the
same underlying state saving/restoring trampolines) but resolve their targets
by performing a standard ORC lookup rather than invoking a user supplied
compiler callback. This allows them to inherit the thread-safety of ORC lookups
while blocking only the calling thread (whereas compile callbacks also block one
compile thread).
Lazy re-exports provide a simple way of building lazy call-throughs. Unlike a
regular re-export, a lazy re-export generates a new address (a stub entry point)
that will act like the re-exported symbol when called. The first call via a
lazy re-export will trigger compilation of the re-exported symbol before calling
through to it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343061 91177308-0d34-0410-b5e6-96231b3b80d8
This will allow trampoline pools to be re-used for a new lazy-reexport utility
that generates looks up function bodies using the standard symbol lookup process
(rather than using a user provided compile function). This new utility provides
the same capabilities (since MaterializationUnits already allow user supplied
compile functions to be run) as JITCompileCallbackManager, but can use the new
asynchronous lookup functions to avoid blocking a compile thread.
This patch also updates createLocalCompileCallbackManager to return an error if
a callback manager can not be created, and updates clients of that API to
account for the change. Finally, the OrcCBindingsStack is updates so that if
a callback manager is not available for the target platform a valid stack
(without support for lazy compilation) can still be constructed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343059 91177308-0d34-0410-b5e6-96231b3b80d8
compilation of IR in the JIT.
ThreadSafeContext is a pair of an LLVMContext and a mutex that can be used to
lock that context when it needs to be accessed from multiple threads.
ThreadSafeModule is a pair of a unique_ptr<Module> and a
shared_ptr<ThreadSafeContext>. This allows the lifetime of a ThreadSafeContext
to be managed automatically in terms of the ThreadSafeModules that refer to it:
Once all modules using a ThreadSafeContext are destructed, and providing the
client has not held on to a copy of shared context pointer, the context will be
automatically destructed.
This scheme is necessary due to the following constraits: (1) We need multiple
contexts for multithreaded compilation (at least one per compile thread plus
one to store any IR not currently being compiled, though one context per module
is simpler). (2) We need to free contexts that are no longer being used so that
the JIT does not leak memory over time. (3) Module lifetimes are not
predictable (modules are compiled as needed depending on the flow of JIT'd
code) so there is no single point where contexts could be reclaimed.
JIT clients not using concurrency can safely use one ThreadSafeContext for all
ThreadSafeModules.
JIT clients who want to be able to compile concurrently should use a different
ThreadSafeContext for each module, or call setCloneToNewContextOnEmit on their
top-level IRLayer. The former reduces compile latency (since no clone step is
needed) at the cost of additional memory overhead for uncompiled modules (as
every uncompiled module will duplicate the LLVM types, constants and metadata
that have been shared).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343055 91177308-0d34-0410-b5e6-96231b3b80d8
construction, a new convenience lookup method, and add-to layer methods.
ExecutionSession now creates a special 'main' JITDylib upon construction. All
subsequently created JITDylibs are added to the main JITDylib's search order by
default (controlled by the AddToMainDylibSearchOrder parameter to
ExecutionSession::createDylib). The main JITDylib's search order will be used in
the future to properly handle cross-JITDylib weak symbols, with the first
definition in this search order selected.
This commit also adds a new ExecutionSession::lookup convenience method that
performs a blocking lookup using the main JITDylib's search order, as this will
be a very common operation for clients.
Finally, new convenience overloads of IRLayer and ObjectLayer's add methods are
introduced that add the given program representations to the main dylib, which
is likely to be the common case.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@342086 91177308-0d34-0410-b5e6-96231b3b80d8
The existing memory manager API can not be shared between objects when linking
concurrently (since there is no way to know which concurrent allocations were
performed on behalf of which object, and hence which allocations would be safe
to finalize when finalizeMemory is called). For now, we can work around this by
requiring a new memory manager for each object.
This change only affects the concurrent version of the ORC APIs.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@341579 91177308-0d34-0410-b5e6-96231b3b80d8
ExecutionSession defaults to creating a new shared pool if none is provided,
so explicitly passing one in is unnecessary.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@341494 91177308-0d34-0410-b5e6-96231b3b80d8
Removes the implicit conversion to the underlying type for
JITSymbolFlags::FlagNames and replaces it with some bitwise and comparison
operators.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@341282 91177308-0d34-0410-b5e6-96231b3b80d8
management and materialization responsibility registration.
The setOverrideObjectFlagsWithResponsibilityFlags method instructs
RTDyldObjectlinkingLayer2 to override the symbol flags produced by RuntimeDyld with
the flags provided by the MaterializationResponsibility instance. This can be used
to enable symbol visibility (hidden/exported) for COFF object files, which do not
currently support the SF_Exported flag.
The setAutoClaimResponsibilityForObjectSymbols method instructs
RTDyldObjectLinkingLayer2 to claim responsibility for any symbols provided by a
given object file that were not already in the MaterializationResponsibility
instance. Setting this flag allows higher-level program representations (e.g.
LLVM IR) to be added based on only a subset of the symbols they provide, without
having to write intervening layers to scan and add the additional symbols. This
trades diagnostic quality for convenience however: If all symbols are enumerated
up-front then clashes can be detected and reported early. If this option is set,
clashes for the additional symbols may not be detected until late, and detection
may depend on the flow of control through JIT'd code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@341154 91177308-0d34-0410-b5e6-96231b3b80d8
The new method name/behavior more closely models the way it was being used.
It also fixes an assertion that can occur when using the new ORC Core APIs,
where flags alone don't necessarily provide enough context to decide whether
the caller is responsible for materializing a given symbol (which was always
the reason this API existed).
The default implementation of getResponsibilitySet uses lookupFlags to determine
responsibility as before, so existing JITSymbolResolvers should continue to
work.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@340874 91177308-0d34-0410-b5e6-96231b3b80d8
The new unit tests match the old ones, which will remain in tree until the
old RTDyldObjectLinkingLayer is removed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@340786 91177308-0d34-0410-b5e6-96231b3b80d8
SymbolStringPool ref counts are now size_t, rather than uint64_t, so I do not
think this is necessary any more.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@340704 91177308-0d34-0410-b5e6-96231b3b80d8
An emitted symbol has had its contents written and its memory protections
applied, but it is not automatically ready to execute.
Prior to ORC supporting concurrent compilation, the term "finalized" could be
interpreted two different (but effectively equivalent) ways: (1) The finalized
symbol's contents have been written and its memory protections applied, and (2)
the symbol is ready to run. Now that ORC supports concurrent compilation, sense
(1) no longer implies sense (2). We have already introduced a new term, 'ready',
to capture sense (2), so rename sense (1) to 'emitted' to avoid any lingering
confusion.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@340115 91177308-0d34-0410-b5e6-96231b3b80d8
VSO was a little close to VDSO (an acronym on Linux for Virtual Dynamic Shared
Object) for comfort. It also risks giving the impression that instances of this
class could be shared between ExecutionSessions, which they can not.
JITDylib seems moderately less confusing, while still hinting at how this
class is intended to be used, i.e. as a JIT-compiled stand-in for a dynamic
library (code that would have been a dynamic library if you had wanted to
compile it ahead of time).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@340084 91177308-0d34-0410-b5e6-96231b3b80d8
An instance of ReexportsFallbackDefinitionGenerator can be attached to a VSO
(via setFallbackDefinitionGenerator) to re-export symbols on demandy from a
backing VSO.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@338764 91177308-0d34-0410-b5e6-96231b3b80d8
Also, make SerializationTraits for pairs forward the actual pair
template type arguments to the underlying serializer. This allows, for example,
std::pair<StringRef, bool> to be passed as an argument to an RPC call expecting
a std::pair<std::string, bool>, since there is an underlying serializer from
StringRef to std::string that can be used.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@338305 91177308-0d34-0410-b5e6-96231b3b80d8
deprecating SymbolResolver and AsynchronousSymbolQuery.
Both lookup overloads take a VSO search order to perform the lookup. The first
overload is non-blocking and takes OnResolved and OnReady callbacks. The second
is blocking, takes a boolean flag to indicate whether to wait until all symbols
are ready, and returns a SymbolMap. Both overloads take a RegisterDependencies
function to register symbol dependencies (if any) on the query.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@337595 91177308-0d34-0410-b5e6-96231b3b80d8
This discards the unresolved symbols set and returns the flags map directly
(rather than mutating it via the first argument).
The unresolved symbols result made it easy to chain lookupFlags calls, but such
chaining should be rare to non-existant (especially now that symbol resolvers
are being deprecated) so the simpler method signature is preferable.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@337594 91177308-0d34-0410-b5e6-96231b3b80d8
A search order is a list of VSOs to be searched linearly to find symbols. Each
VSO now has a search order that will be used when fixing up definitions in that
VSO. Each VSO's search order defaults to just that VSO itself.
This is a first step towards removing symbol resolvers from ORC altogether. In
practice symbol resolvers tended to be used to implement a search order anyway,
sometimes with additional programatic generation of symbols. Now that VSOs
support programmatic generation of definitions via fallback generators, search
orders provide a cleaner way to achieve the desired effect (while removing a lot
of boilerplate).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@337593 91177308-0d34-0410-b5e6-96231b3b80d8
symbols in another VSO).
Also fixes a bug where chained aliases within a single VSO would deadlock on
materialization.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@336741 91177308-0d34-0410-b5e6-96231b3b80d8
delegate method (and unit test).
The name 'replace' better captures what the old delegate method did: it
returned materialization responsibility for a set of symbols to the VSO.
The new delegate method delegates responsibility for a set of symbols to a new
MaterializationResponsibility instance. This can be used to split responsibility
between multiple threads, or multiple materialization methods.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@336603 91177308-0d34-0410-b5e6-96231b3b80d8
Once a symbol has been selected for materialization it can no longer be
overridden. Stripping the weak flag guarantees this (override attempts will
then be treated as duplicate definitions and result in a DuplicateDefinition
error).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334771 91177308-0d34-0410-b5e6-96231b3b80d8
If a VSO has a fallback definition generator attached it will be called during
lookup (and lookupFlags) for any unresolved symbols. The definition generator
can add new definitions to the VSO for any unresolved symbol. This allows VSOs
to generate new definitions on demand.
The immediate use case for this code is supporting VSOs that can import
definitions found via dlsym on demand.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334538 91177308-0d34-0410-b5e6-96231b3b80d8
Existing implementations of these methods do not require lazy materialization,
and switching to JITEvaluatedSymbol allows us to remove error checking on the
client side.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333835 91177308-0d34-0410-b5e6-96231b3b80d8
This method returns the set of symbols in the target VSO that have queries
waiting on them. This can be used to make decisions about which symbols to
delegate to another MaterializationUnit (typically this will involve
delegating all symbols that have *not* been requested to another
MaterializationUnit so that materialization of those symbols can be
deferred until they are requested).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333684 91177308-0d34-0410-b5e6-96231b3b80d8
Previously JITCompileCallbackManager only supported single threaded code. This
patch embeds a VSO (see include/llvm/ExecutionEngine/Orc/Core.h) in the callback
manager. The VSO ensures that the compile callback is only executed once and that
the resulting address cached for use by subsequent re-entries.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333490 91177308-0d34-0410-b5e6-96231b3b80d8
Re-appply r333147, reverted in r333152 due to a pre-existing bug. As
D47308 has been merged in r333206, the OSX issue should now be
resolved.
In many cases JIT users will know in which module a symbol
resides. Avoiding to search other modules can be more efficient. It
also allows to handle duplicate symbol names between modules.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D44889
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333215 91177308-0d34-0410-b5e6-96231b3b80d8
This reverts r333147 until https://reviews.llvm.org/D47308 is ready to
be reviewed. r333147 exposed a behavioural difference between
OrcCBindingsStack::findSymbolIn() and OrcCBindingsStack::findSymbol(),
where only the latter does name mangling. After r333147 that causes a
test failure on OSX, because the new test looks for main using
findSymbolIn() but the mangled name is _main.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333152 91177308-0d34-0410-b5e6-96231b3b80d8
