deprecating SymbolResolver and AsynchronousSymbolQuery.
Both lookup overloads take a VSO search order to perform the lookup. The first
overload is non-blocking and takes OnResolved and OnReady callbacks. The second
is blocking, takes a boolean flag to indicate whether to wait until all symbols
are ready, and returns a SymbolMap. Both overloads take a RegisterDependencies
function to register symbol dependencies (if any) on the query.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@337595 91177308-0d34-0410-b5e6-96231b3b80d8
This discards the unresolved symbols set and returns the flags map directly
(rather than mutating it via the first argument).
The unresolved symbols result made it easy to chain lookupFlags calls, but such
chaining should be rare to non-existant (especially now that symbol resolvers
are being deprecated) so the simpler method signature is preferable.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@337594 91177308-0d34-0410-b5e6-96231b3b80d8
A search order is a list of VSOs to be searched linearly to find symbols. Each
VSO now has a search order that will be used when fixing up definitions in that
VSO. Each VSO's search order defaults to just that VSO itself.
This is a first step towards removing symbol resolvers from ORC altogether. In
practice symbol resolvers tended to be used to implement a search order anyway,
sometimes with additional programatic generation of symbols. Now that VSOs
support programmatic generation of definitions via fallback generators, search
orders provide a cleaner way to achieve the desired effect (while removing a lot
of boilerplate).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@337593 91177308-0d34-0410-b5e6-96231b3b80d8
symbols in another VSO).
Also fixes a bug where chained aliases within a single VSO would deadlock on
materialization.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@336741 91177308-0d34-0410-b5e6-96231b3b80d8
delegate method (and unit test).
The name 'replace' better captures what the old delegate method did: it
returned materialization responsibility for a set of symbols to the VSO.
The new delegate method delegates responsibility for a set of symbols to a new
MaterializationResponsibility instance. This can be used to split responsibility
between multiple threads, or multiple materialization methods.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@336603 91177308-0d34-0410-b5e6-96231b3b80d8
Once a symbol has been selected for materialization it can no longer be
overridden. Stripping the weak flag guarantees this (override attempts will
then be treated as duplicate definitions and result in a DuplicateDefinition
error).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334771 91177308-0d34-0410-b5e6-96231b3b80d8
If a VSO has a fallback definition generator attached it will be called during
lookup (and lookupFlags) for any unresolved symbols. The definition generator
can add new definitions to the VSO for any unresolved symbol. This allows VSOs
to generate new definitions on demand.
The immediate use case for this code is supporting VSOs that can import
definitions found via dlsym on demand.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334538 91177308-0d34-0410-b5e6-96231b3b80d8
Existing implementations of these methods do not require lazy materialization,
and switching to JITEvaluatedSymbol allows us to remove error checking on the
client side.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333835 91177308-0d34-0410-b5e6-96231b3b80d8
This method returns the set of symbols in the target VSO that have queries
waiting on them. This can be used to make decisions about which symbols to
delegate to another MaterializationUnit (typically this will involve
delegating all symbols that have *not* been requested to another
MaterializationUnit so that materialization of those symbols can be
deferred until they are requested).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333684 91177308-0d34-0410-b5e6-96231b3b80d8
Previously JITCompileCallbackManager only supported single threaded code. This
patch embeds a VSO (see include/llvm/ExecutionEngine/Orc/Core.h) in the callback
manager. The VSO ensures that the compile callback is only executed once and that
the resulting address cached for use by subsequent re-entries.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333490 91177308-0d34-0410-b5e6-96231b3b80d8
Re-appply r333147, reverted in r333152 due to a pre-existing bug. As
D47308 has been merged in r333206, the OSX issue should now be
resolved.
In many cases JIT users will know in which module a symbol
resides. Avoiding to search other modules can be more efficient. It
also allows to handle duplicate symbol names between modules.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D44889
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333215 91177308-0d34-0410-b5e6-96231b3b80d8
This reverts r333147 until https://reviews.llvm.org/D47308 is ready to
be reviewed. r333147 exposed a behavioural difference between
OrcCBindingsStack::findSymbolIn() and OrcCBindingsStack::findSymbol(),
where only the latter does name mangling. After r333147 that causes a
test failure on OSX, because the new test looks for main using
findSymbolIn() but the mangled name is _main.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333152 91177308-0d34-0410-b5e6-96231b3b80d8
In many cases JIT users will know in which module a symbol
resides. Avoiding to search other modules can be more efficient. It
also allows to handle duplicate symbol names between modules.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D44889
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333147 91177308-0d34-0410-b5e6-96231b3b80d8
The lookup function provides blocking symbol resolution for JIT clients (not
layers themselves) so it does not need to track symbol dependencies via a
MaterializationResponsibility.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332897 91177308-0d34-0410-b5e6-96231b3b80d8
notifyFailed method rather than passing in an error generator.
VSO::notifyFailed is responsible for notifying queries that they will not
succeed due to error. In practice the queries don't care about the details
of the failure, just the fact that a failure occurred for some symbols.
Having VSO::notifyFailed take care of this simplifies the interface.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332666 91177308-0d34-0410-b5e6-96231b3b80d8
VSOs now track dependencies for materializing symbols. Each symbol must have its
dependencies registered with the VSO prior to finalization. Usually this will
involve registering the dependencies returned in
AsynchronousSymbolQuery::ResolutionResults for queries made while linking the
symbols being materialized.
Queries against symbols are notified that a symbol is ready once it and all of
its transitive dependencies are finalized, allowing compilation work to be
broken up and moved between threads without queries returning until their
symbols fully safe to access / execute.
Related utilities (VSO, MaterializationUnit, MaterializationResponsibility) are
updated to support dependence tracking and more explicitly track responsibility
for symbols from the point of definition until they are finalized.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332541 91177308-0d34-0410-b5e6-96231b3b80d8
See r331124 for how I made a list of files missing the include.
I then ran this Python script:
for f in open('filelist.txt'):
f = f.strip()
fl = open(f).readlines()
found = False
for i in xrange(len(fl)):
p = '#include "llvm/'
if not fl[i].startswith(p):
continue
if fl[i][len(p):] > 'Config':
fl.insert(i, '#include "llvm/Config/llvm-config.h"\n')
found = True
break
if not found:
print 'not found', f
else:
open(f, 'w').write(''.join(fl))
and then looked through everything with `svn diff | diffstat -l | xargs -n 1000 gvim -p`
and tried to fix include ordering and whatnot.
No intended behavior change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@331184 91177308-0d34-0410-b5e6-96231b3b80d8
LLVM_ON_WIN32 is set exactly with MSVC and MinGW (but not Cygwin) in
HandleLLVMOptions.cmake, which is where _WIN32 defined too. Just use the
default macro instead of a reinvented one.
See thread "Replacing LLVM_ON_WIN32 with just _WIN32" on llvm-dev and cfe-dev.
No intended behavior change.
This moves over all uses of the macro, but doesn't remove the definition
of it in (llvm-)config.h yet.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@331127 91177308-0d34-0410-b5e6-96231b3b80d8
/usr/local/bin/ld.lld: error: undefined symbol: llvm::createAggressiveInstCombinerPass()
>>> referenced by cc1_main.cpp
>>> tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o:(_GLOBAL__sub_I_cc1_main.cpp)
And so on
The bot coverage is clearly missing.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330693 91177308-0d34-0410-b5e6-96231b3b80d8
materializing function definitions.
MaterializationUnit instances are responsible for resolving and finalizing
symbol definitions when their materialize method is called. By contract, the
MaterializationUnit must materialize all definitions it is responsible for and
no others. If it can not materialize all definitions (because of some error)
then it must notify the associated VSO about each definition that could not be
materialized. The MaterializationResponsibility class tracks this
responsibility, asserting that all required symbols are resolved and finalized,
and that no extraneous symbols are resolved or finalized. In the event of an
error it provides a convenience method for notifying the VSO about each
definition that could not be materialized.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330142 91177308-0d34-0410-b5e6-96231b3b80d8
notifyMaterializationFailed.
The notifyMaterializationFailed method can determine which error to raise by
looking at which queue the pending queries are in (resolution or finalization).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330141 91177308-0d34-0410-b5e6-96231b3b80d8
This check prevents the ORC execution tests from running on Windows (which is
not supported yet).
This should fix the windows bots.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@328706 91177308-0d34-0410-b5e6-96231b3b80d8
Previously this crashed because a nullptr (returned by
createLocalIndirectStubsManagerBuilder() on platforms without
indirection support) functor was unconditionally invoked.
Patch by Andres Freund. Thanks Andres!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@328687 91177308-0d34-0410-b5e6-96231b3b80d8
There's are race between this thread and the destructor of the test ORC
components on the main threads. I saw flaky failures there in about 4%
of the runs of this unit test.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@328300 91177308-0d34-0410-b5e6-96231b3b80d8
operation all-or-nothing, rather than allowing materialization on a per-symbol
basis.
This addresses a shortcoming of per-symbol materialization: If a
MaterializationUnit (/SymbolSource) wants to materialize more symbols than
requested (which is likely: most materializers will want to materialize whole
modules) then it needs a way to notify the symbol table about the extra symbols
being materialized. This process (checking what has been requested against what
is being provided and notifying the symbol table about the difference) has to
be repeated at every level of the JIT stack. Making materialization
all-or-nothing eliminates this issue, simplifying both materializer
implementations and the symbol table (VSO class) API. The cost is that
per-symbol materialization (e.g. for individual symbols in a module) now
requires multiple MaterializationUnits.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@327946 91177308-0d34-0410-b5e6-96231b3b80d8
This reverts commit r327566, it breaks
test/ExecutionEngine/OrcMCJIT/test-global-ctors.ll.
The test doesn't crash with a stack trace, unfortunately. It merely
returns 1 as the exit code.
ASan didn't produce a report, and I reproduced this on my Linux machine
and Windows box.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@327576 91177308-0d34-0410-b5e6-96231b3b80d8
Layer implementations typically mutate module state, and this is better
reflected by having layers own the Module they are operating on.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@327566 91177308-0d34-0410-b5e6-96231b3b80d8
The lookup function takes a list of VSOs, a set of symbol names (or just one
symbol name) and a materialization function object. It returns an
Expected<SymbolMap> (if given a set of names) or an Expected<JITEvaluatedSymbol>
(if given just one name). The lookup method constructs an
AsynchronousSymbolQuery for the given names, applies that query to each VSO in
the list in turn, and then blocks waiting for the query to complete. If
threading is enabled then the materialization function object can be used to
execute the materialization on different threads. If threading is disabled the
MaterializeOnCurrentThread utility must be used.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@327474 91177308-0d34-0410-b5e6-96231b3b80d8
This makes it easy to free a SymbolSource (and any related
resources) when the last reference in a VSO is dropped.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@325727 91177308-0d34-0410-b5e6-96231b3b80d8
than a shared ObjectFile/MemoryBuffer pair.
There's no need to pre-parse the buffer into an ObjectFile before passing it
down to the linking layer, and moving the parsing into the linking layer allows
us remove the parsing code at each call site.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@325725 91177308-0d34-0410-b5e6-96231b3b80d8
unified GetResources callback.
Having a single 'GetResources' callback will simplify adding new resources in
the future.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@325180 91177308-0d34-0410-b5e6-96231b3b80d8
Handles were returned by addModule and used as keys for removeModule,
findSymbolIn, and emitAndFinalize. Their job is now subsumed by VModuleKeys,
which simplify resource management by providing a consistent handle across all
layers.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@324700 91177308-0d34-0410-b5e6-96231b3b80d8
In particular this patch switches RTDyldObjectLinkingLayer to use
orc::SymbolResolver and threads the requried changse (ExecutionSession
references and VModuleKeys) through the existing layer APIs.
The purpose of the new resolver interface is to improve query performance and
better support parallelism, both in JIT'd code and within the compiler itself.
The most visibile change is switch of the <Layer>::addModule signatures from:
Expected<Handle> addModule(std::shared_ptr<ModuleType> Mod,
std::shared_ptr<JITSymbolResolver> Resolver)
to:
Expected<Handle> addModule(VModuleKey K, std::shared_ptr<ModuleType> Mod);
Typical usage of addModule will now look like:
auto K = ES.allocateVModuleKey();
Resolvers[K] = createSymbolResolver(...);
Layer.addModule(K, std::move(Mod));
See the BuildingAJIT tutorial code for example usage.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@324405 91177308-0d34-0410-b5e6-96231b3b80d8
