CompileOnDemandLayer2 is a replacement for CompileOnDemandLayer built on the ORC
Core APIs. Functions in added modules are extracted and compiled lazily.
CompileOnDemandLayer2 supports multithreaded JIT'd code, and compilation on
multiple threads.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334967 91177308-0d34-0410-b5e6-96231b3b80d8
materializing weak symbols as strong.
This removes some elaborate flag tweaking and plays nicer with RuntimeDyld,
which relies of weak/common flags to determine whether it should emit a given
weak definition. (Switching to strong up-front makes it appear as if there is
already an overriding definition, which would require an extra back-channel to
override).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334966 91177308-0d34-0410-b5e6-96231b3b80d8
symbols in debug mode.
The MaterializationResponsibility class hijacks the Materializing flag to track
symbols that have not yet been resolved in order to guard against redundant
resolution. Since this is an API contract check and only enforced in debug mode
there is no reason to maintain the flag state in release mode.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334909 91177308-0d34-0410-b5e6-96231b3b80d8
Once a symbol has been selected for materialization it can no longer be
overridden. Stripping the weak flag guarantees this (override attempts will
then be treated as duplicate definitions and result in a DuplicateDefinition
error).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334771 91177308-0d34-0410-b5e6-96231b3b80d8
If WaitUntilReady is set to true then blockingLookup will return once all
requested symbols are ready. If WaitUntilReady is set to false then
blockingLookup will return as soon as all requested symbols have been
resolved. In the latter case, if any error occurs in finalizing the symbols it
will be reported to the ExecutionSession, rather than returned by
blockingLookup.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334722 91177308-0d34-0410-b5e6-96231b3b80d8
If a VSO has a fallback definition generator attached it will be called during
lookup (and lookupFlags) for any unresolved symbols. The definition generator
can add new definitions to the VSO for any unresolved symbol. This allows VSOs
to generate new definitions on demand.
The immediate use case for this code is supporting VSOs that can import
definitions found via dlsym on demand.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334538 91177308-0d34-0410-b5e6-96231b3b80d8
pre-existing SymbolFlags and SymbolToDefinition maps.
This constructor is useful when delegating work from an existing
IRMaterialiaztionUnit to a new one, as it avoids the cost of re-computing these
maps.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333852 91177308-0d34-0410-b5e6-96231b3b80d8
This method returns the set of symbols in the target VSO that have queries
waiting on them. This can be used to make decisions about which symbols to
delegate to another MaterializationUnit (typically this will involve
delegating all symbols that have *not* been requested to another
MaterializationUnit so that materialization of those symbols can be
deferred until they are requested).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333684 91177308-0d34-0410-b5e6-96231b3b80d8
and make it protected rather than private.
The new name reflects the actual information in the map, and this information
can be useful to derived classes (for example, to quickly look up the IR
definition of a requested symbol).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333683 91177308-0d34-0410-b5e6-96231b3b80d8
Previously JITCompileCallbackManager only supported single threaded code. This
patch embeds a VSO (see include/llvm/ExecutionEngine/Orc/Core.h) in the callback
manager. The VSO ensures that the compile callback is only executed once and that
the resulting address cached for use by subsequent re-entries.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333490 91177308-0d34-0410-b5e6-96231b3b80d8
Currently RTDyldObjectLinkingLayer makes it hard to support
JITEventListeners. Which in turn means debugging and profiling JIT
generated code hard.
Supporting JITEventListeners at minimum requries a freed
callback (added).
As listeners expect the ObjectFile to be passed as well, an adaptor
between RTDyldObjectLinkingLayer and JITEventListeners would currently
need to also maintain ObjectFiles for all loaded modules. To make that
less awkward, extend the callbacks to pass the ObjectFile to both
Finalized and Freed callbacks. That requires extending the lifetime
of the object file when callbacks are present.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D44890
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333227 91177308-0d34-0410-b5e6-96231b3b80d8
Re-appply r333147, reverted in r333152 due to a pre-existing bug. As
D47308 has been merged in r333206, the OSX issue should now be
resolved.
In many cases JIT users will know in which module a symbol
resides. Avoiding to search other modules can be more efficient. It
also allows to handle duplicate symbol names between modules.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D44889
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333215 91177308-0d34-0410-b5e6-96231b3b80d8
The lack of name mangling caused a unittest failure after r333147 (in
TestEagerIRCompilation), as OSX prefixes symbol names with '_'. The
lack of name mangling therefore leads to a NULL pointer being returned
and then called, hence the failure.
While it may look like it, this isn't an actual behavioral change, as
findSymbolIn() previously was not exposed externally, and essentially
dead code. Which explains why nobody noticed the issue previously.
Reviewers: lhames
Reviewed By: lhames
Subscribers: chandlerc, llvm-commits
Differential Revision: https://reviews.llvm.org/D47308
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333206 91177308-0d34-0410-b5e6-96231b3b80d8
This reverts r333147 until https://reviews.llvm.org/D47308 is ready to
be reviewed. r333147 exposed a behavioural difference between
OrcCBindingsStack::findSymbolIn() and OrcCBindingsStack::findSymbol(),
where only the latter does name mangling. After r333147 that causes a
test failure on OSX, because the new test looks for main using
findSymbolIn() but the mangled name is _main.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333152 91177308-0d34-0410-b5e6-96231b3b80d8
In many cases JIT users will know in which module a symbol
resides. Avoiding to search other modules can be more efficient. It
also allows to handle duplicate symbol names between modules.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D44889
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333147 91177308-0d34-0410-b5e6-96231b3b80d8
to a base class (IRMaterializationUnit).
The new class, IRMaterializationUnit, provides a convenient base for any client
that wants to write a materializer for LLVM IR.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332993 91177308-0d34-0410-b5e6-96231b3b80d8
Also tightens the behavior of ExecutionSession::failQuery. Queries can usually
only be failed by marking a symbol as failed-to-materialize, but
ExecutionSession::failQuery provides a second route, and both routes may be
executed from different threads. In the case that a query has already been
failed due to a materialization error, ExecutionSession::failQuery will
direct the error to ExecutionSession::reportError instead.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332898 91177308-0d34-0410-b5e6-96231b3b80d8
The lookup function provides blocking symbol resolution for JIT clients (not
layers themselves) so it does not need to track symbol dependencies via a
MaterializationResponsibility.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332897 91177308-0d34-0410-b5e6-96231b3b80d8
notifyFailed method rather than passing in an error generator.
VSO::notifyFailed is responsible for notifying queries that they will not
succeed due to error. In practice the queries don't care about the details
of the failure, just the fact that a failure occurred for some symbols.
Having VSO::notifyFailed take care of this simplifies the interface.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332666 91177308-0d34-0410-b5e6-96231b3b80d8
VSOs now track dependencies for materializing symbols. Each symbol must have its
dependencies registered with the VSO prior to finalization. Usually this will
involve registering the dependencies returned in
AsynchronousSymbolQuery::ResolutionResults for queries made while linking the
symbols being materialized.
Queries against symbols are notified that a symbol is ready once it and all of
its transitive dependencies are finalized, allowing compilation work to be
broken up and moved between threads without queries returning until their
symbols fully safe to access / execute.
Related utilities (VSO, MaterializationUnit, MaterializationResponsibility) are
updated to support dependence tracking and more explicitly track responsibility
for symbols from the point of definition until they are finalized.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332541 91177308-0d34-0410-b5e6-96231b3b80d8
This is a follow-up to r331272.
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.
Patch produced by
for i in $(git grep -l '\@brief'); do perl -pi -e 's/\@brief //g' $i & done
https://reviews.llvm.org/D46290
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@331275 91177308-0d34-0410-b5e6-96231b3b80d8
See r331124 for how I made a list of files missing the include.
I then ran this Python script:
for f in open('filelist.txt'):
f = f.strip()
fl = open(f).readlines()
found = False
for i in xrange(len(fl)):
p = '#include "llvm/'
if not fl[i].startswith(p):
continue
if fl[i][len(p):] > 'Config':
fl.insert(i, '#include "llvm/Config/llvm-config.h"\n')
found = True
break
if not found:
print 'not found', f
else:
open(f, 'w').write(''.join(fl))
and then looked through everything with `svn diff | diffstat -l | xargs -n 1000 gvim -p`
and tried to fix include ordering and whatnot.
No intended behavior change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@331184 91177308-0d34-0410-b5e6-96231b3b80d8
This forces these operations to be carried out via a
MaterializationResponsibility instance, ensuring responsibility is explicitly
tracked.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330356 91177308-0d34-0410-b5e6-96231b3b80d8
materializing function definitions.
MaterializationUnit instances are responsible for resolving and finalizing
symbol definitions when their materialize method is called. By contract, the
MaterializationUnit must materialize all definitions it is responsible for and
no others. If it can not materialize all definitions (because of some error)
then it must notify the associated VSO about each definition that could not be
materialized. The MaterializationResponsibility class tracks this
responsibility, asserting that all required symbols are resolved and finalized,
and that no extraneous symbols are resolved or finalized. In the event of an
error it provides a convenience method for notifying the VSO about each
definition that could not be materialized.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330142 91177308-0d34-0410-b5e6-96231b3b80d8
notifyMaterializationFailed.
The notifyMaterializationFailed method can determine which error to raise by
looking at which queue the pending queries are in (resolution or finalization).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330141 91177308-0d34-0410-b5e6-96231b3b80d8
Previously this crashed because a nullptr (returned by
createLocalIndirectStubsManagerBuilder() on platforms without
indirection support) functor was unconditionally invoked.
Patch by Andres Freund. Thanks Andres!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@328687 91177308-0d34-0410-b5e6-96231b3b80d8
operation all-or-nothing, rather than allowing materialization on a per-symbol
basis.
This addresses a shortcoming of per-symbol materialization: If a
MaterializationUnit (/SymbolSource) wants to materialize more symbols than
requested (which is likely: most materializers will want to materialize whole
modules) then it needs a way to notify the symbol table about the extra symbols
being materialized. This process (checking what has been requested against what
is being provided and notifying the symbol table about the difference) has to
be repeated at every level of the JIT stack. Making materialization
all-or-nothing eliminates this issue, simplifying both materializer
implementations and the symbol table (VSO class) API. The cost is that
per-symbol materialization (e.g. for individual symbols in a module) now
requires multiple MaterializationUnits.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@327946 91177308-0d34-0410-b5e6-96231b3b80d8
