CompileOnDemandLayer2 is a replacement for CompileOnDemandLayer built on the ORC
Core APIs. Functions in added modules are extracted and compiled lazily.
CompileOnDemandLayer2 supports multithreaded JIT'd code, and compilation on
multiple threads.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334967 91177308-0d34-0410-b5e6-96231b3b80d8
If WaitUntilReady is set to true then blockingLookup will return once all
requested symbols are ready. If WaitUntilReady is set to false then
blockingLookup will return as soon as all requested symbols have been
resolved. In the latter case, if any error occurs in finalizing the symbols it
will be reported to the ExecutionSession, rather than returned by
blockingLookup.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334722 91177308-0d34-0410-b5e6-96231b3b80d8
If a VSO has a fallback definition generator attached it will be called during
lookup (and lookupFlags) for any unresolved symbols. The definition generator
can add new definitions to the VSO for any unresolved symbol. This allows VSOs
to generate new definitions on demand.
The immediate use case for this code is supporting VSOs that can import
definitions found via dlsym on demand.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334538 91177308-0d34-0410-b5e6-96231b3b80d8
pre-existing SymbolFlags and SymbolToDefinition maps.
This constructor is useful when delegating work from an existing
IRMaterialiaztionUnit to a new one, as it avoids the cost of re-computing these
maps.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333852 91177308-0d34-0410-b5e6-96231b3b80d8
Existing implementations of these methods do not require lazy materialization,
and switching to JITEvaluatedSymbol allows us to remove error checking on the
client side.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333835 91177308-0d34-0410-b5e6-96231b3b80d8
This method returns the set of symbols in the target VSO that have queries
waiting on them. This can be used to make decisions about which symbols to
delegate to another MaterializationUnit (typically this will involve
delegating all symbols that have *not* been requested to another
MaterializationUnit so that materialization of those symbols can be
deferred until they are requested).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333684 91177308-0d34-0410-b5e6-96231b3b80d8
and make it protected rather than private.
The new name reflects the actual information in the map, and this information
can be useful to derived classes (for example, to quickly look up the IR
definition of a requested symbol).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333683 91177308-0d34-0410-b5e6-96231b3b80d8
Previously JITCompileCallbackManager only supported single threaded code. This
patch embeds a VSO (see include/llvm/ExecutionEngine/Orc/Core.h) in the callback
manager. The VSO ensures that the compile callback is only executed once and that
the resulting address cached for use by subsequent re-entries.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333490 91177308-0d34-0410-b5e6-96231b3b80d8
This is an ODR violation, for example in ExecutionEngine/MCJIT/MCJIT.o: multiple definition of 'LLVMCreateIntelJITEventListener'.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333265 91177308-0d34-0410-b5e6-96231b3b80d8
Currently RTDyldObjectLinkingLayer makes it hard to support
JITEventListeners. Which in turn means debugging and profiling JIT
generated code hard.
Supporting JITEventListeners at minimum requries a freed
callback (added).
As listeners expect the ObjectFile to be passed as well, an adaptor
between RTDyldObjectLinkingLayer and JITEventListeners would currently
need to also maintain ObjectFiles for all loaded modules. To make that
less awkward, extend the callbacks to pass the ObjectFile to both
Finalized and Freed callbacks. That requires extending the lifetime
of the object file when callbacks are present.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D44890
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333227 91177308-0d34-0410-b5e6-96231b3b80d8
to a base class (IRMaterializationUnit).
The new class, IRMaterializationUnit, provides a convenient base for any client
that wants to write a materializer for LLVM IR.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332993 91177308-0d34-0410-b5e6-96231b3b80d8
Also tightens the behavior of ExecutionSession::failQuery. Queries can usually
only be failed by marking a symbol as failed-to-materialize, but
ExecutionSession::failQuery provides a second route, and both routes may be
executed from different threads. In the case that a query has already been
failed due to a materialization error, ExecutionSession::failQuery will
direct the error to ExecutionSession::reportError instead.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332898 91177308-0d34-0410-b5e6-96231b3b80d8
The lookup function provides blocking symbol resolution for JIT clients (not
layers themselves) so it does not need to track symbol dependencies via a
MaterializationResponsibility.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332897 91177308-0d34-0410-b5e6-96231b3b80d8
notifyFailed method rather than passing in an error generator.
VSO::notifyFailed is responsible for notifying queries that they will not
succeed due to error. In practice the queries don't care about the details
of the failure, just the fact that a failure occurred for some symbols.
Having VSO::notifyFailed take care of this simplifies the interface.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332666 91177308-0d34-0410-b5e6-96231b3b80d8
MaterializationResponsibility instances should be created by
MaterializationUnits only. Making the constructor private enforces this.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332665 91177308-0d34-0410-b5e6-96231b3b80d8
VSOs now track dependencies for materializing symbols. Each symbol must have its
dependencies registered with the VSO prior to finalization. Usually this will
involve registering the dependencies returned in
AsynchronousSymbolQuery::ResolutionResults for queries made while linking the
symbols being materialized.
Queries against symbols are notified that a symbol is ready once it and all of
its transitive dependencies are finalized, allowing compilation work to be
broken up and moved between threads without queries returning until their
symbols fully safe to access / execute.
Related utilities (VSO, MaterializationUnit, MaterializationResponsibility) are
updated to support dependence tracking and more explicitly track responsibility
for symbols from the point of definition until they are finalized.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332541 91177308-0d34-0410-b5e6-96231b3b80d8
The DEBUG() macro is very generic so it might clash with other projects.
The renaming was done as follows:
- git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g'
- git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM
- Manual change to APInt
- Manually chage DOCS as regex doesn't match it.
In the transition period the DEBUG() macro is still present and aliased
to the LLVM_DEBUG() one.
Differential Revision: https://reviews.llvm.org/D43624
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332240 91177308-0d34-0410-b5e6-96231b3b80d8
This is a follow-up to r331272.
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.
Patch produced by
for i in $(git grep -l '\@brief'); do perl -pi -e 's/\@brief //g' $i & done
https://reviews.llvm.org/D46290
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@331275 91177308-0d34-0410-b5e6-96231b3b80d8
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.
Patch produced by
for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done
Differential Revision: https://reviews.llvm.org/D46290
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@331272 91177308-0d34-0410-b5e6-96231b3b80d8
This forces these operations to be carried out via a
MaterializationResponsibility instance, ensuring responsibility is explicitly
tracked.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330356 91177308-0d34-0410-b5e6-96231b3b80d8
materializing function definitions.
MaterializationUnit instances are responsible for resolving and finalizing
symbol definitions when their materialize method is called. By contract, the
MaterializationUnit must materialize all definitions it is responsible for and
no others. If it can not materialize all definitions (because of some error)
then it must notify the associated VSO about each definition that could not be
materialized. The MaterializationResponsibility class tracks this
responsibility, asserting that all required symbols are resolved and finalized,
and that no extraneous symbols are resolved or finalized. In the event of an
error it provides a convenience method for notifying the VSO about each
definition that could not be materialized.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330142 91177308-0d34-0410-b5e6-96231b3b80d8
notifyMaterializationFailed.
The notifyMaterializationFailed method can determine which error to raise by
looking at which queue the pending queries are in (resolution or finalization).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330141 91177308-0d34-0410-b5e6-96231b3b80d8
This is a layering violation. LTO shouldn't depend on MCJIT. The right
fix for this is moving the class somewhere else.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@329929 91177308-0d34-0410-b5e6-96231b3b80d8
operation all-or-nothing, rather than allowing materialization on a per-symbol
basis.
This addresses a shortcoming of per-symbol materialization: If a
MaterializationUnit (/SymbolSource) wants to materialize more symbols than
requested (which is likely: most materializers will want to materialize whole
modules) then it needs a way to notify the symbol table about the extra symbols
being materialized. This process (checking what has been requested against what
is being provided and notifying the symbol table about the difference) has to
be repeated at every level of the JIT stack. Making materialization
all-or-nothing eliminates this issue, simplifying both materializer
implementations and the symbol table (VSO class) API. The cost is that
per-symbol materialization (e.g. for individual symbols in a module) now
requires multiple MaterializationUnits.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@327946 91177308-0d34-0410-b5e6-96231b3b80d8
This reverts commit r327566, it breaks
test/ExecutionEngine/OrcMCJIT/test-global-ctors.ll.
The test doesn't crash with a stack trace, unfortunately. It merely
returns 1 as the exit code.
ASan didn't produce a report, and I reproduced this on my Linux machine
and Windows box.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@327576 91177308-0d34-0410-b5e6-96231b3b80d8
Layer implementations typically mutate module state, and this is better
reflected by having layers own the Module they are operating on.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@327566 91177308-0d34-0410-b5e6-96231b3b80d8
This can be used to extract the symbol table from a RuntimeDyld instance prior
to disposing of it.
This patch also updates RTDyldObjectLinkingLayer to use the new method, rather
than requesting symbols one at a time via getSymbol.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@327476 91177308-0d34-0410-b5e6-96231b3b80d8
The lookup function takes a list of VSOs, a set of symbol names (or just one
symbol name) and a materialization function object. It returns an
Expected<SymbolMap> (if given a set of names) or an Expected<JITEvaluatedSymbol>
(if given just one name). The lookup method constructs an
AsynchronousSymbolQuery for the given names, applies that query to each VSO in
the list in turn, and then blocks waiting for the query to complete. If
threading is enabled then the materialization function object can be used to
execute the materialization on different threads. If threading is disabled the
MaterializeOnCurrentThread utility must be used.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@327474 91177308-0d34-0410-b5e6-96231b3b80d8
This makes it easy to free a SymbolSource (and any related
resources) when the last reference in a VSO is dropped.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@325727 91177308-0d34-0410-b5e6-96231b3b80d8
