We were able to figure out that the result of a call is some constant.
While propagating that fact, we added the constant to the value map.
This is problematic because it results in us losing the call site when
processing the value map.
This fixes PR28802.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277611 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: Thumb2 supports encoding immediates with specific patterns into mov.w by splatting the low 8 bits into other bytes.
Reviewers: john.brawn, jmolloy
Subscribers: jmolloy, aemerson, rengolin, samparker, llvm-commits
Differential Revision: https://reviews.llvm.org/D23090
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277610 91177308-0d34-0410-b5e6-96231b3b80d8
MappedBlockSTream can work with any sequence of block data where
the ordering is specified by a list of block numbers. So rather
than manually stitch them together in the case of the FPM, reuse
this functionality so that we can treat the FPM as if it were
contiguous.
Reviewed By: ruiu
Differential Revision: https://reviews.llvm.org/D23066
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277609 91177308-0d34-0410-b5e6-96231b3b80d8
When expanding FP constants, we attempt to shrink doubles to floats and perform an extending load.
However, on SystemZ, and possibly on other targets (I've only confirmed the problem on SystemZ), the FP extending load instruction may convert SNaN into QNaN, or may cause an exception. So in the general case, we would still like to shrink FP constants, but SNaNs should be left as doubles.
Differential Revision: https://reviews.llvm.org/D22685
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277602 91177308-0d34-0410-b5e6-96231b3b80d8
When the same base address is used to load two different data types, LSR
would assume a memory type of "void". This type is not sized and has no
alignment information. Checking for it causes a crash.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277601 91177308-0d34-0410-b5e6-96231b3b80d8
called "FPM" instead of "LPM" in a hold-over from when the code was
modeled on that used to parse function passes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277584 91177308-0d34-0410-b5e6-96231b3b80d8
manager.
While this has some utility for debugging and testing on its own, it is
primarily intended to demonstrate the technique for adding custom
wrappers that can provide more interesting interation behavior in
a nice, orthogonal, and composable layer.
Being able to write these kinds of very dynamic and customized controls
for running passes was one of the motivating use cases of the new pass
manager design, and this gives a hint at how they might look. The actual
logic is tiny here, and most of this is just wiring in the pipeline
parsing so that this can be widely used.
I'm adding this now to show the wiring without a lot of business logic.
This is a precursor patch for showing how a "iterate up to N times as
long as we devirtualize a call" utility can be added as a separable and
composable component along side the CGSCC pass management.
Differential Revision: https://reviews.llvm.org/D22405
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277581 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
We also add a test to show what currently happens when we create a
section per function and emit an xray_instr_map. This illustrates the
relationship (or lack thereof) between the per-function section and the
xray_instr_map section.
We also change the code generation slightly so that we don't always
create group sections, but rather only do so if a function where the
table is associated with is in a group.
Also in this change:
- Remove the "merge" flag on the xray_instr_map section.
- Test that we're generating the right table for comdat and non-comdat functions.
Reviewers: echristo, majnemer
Subscribers: llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D23104
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277580 91177308-0d34-0410-b5e6-96231b3b80d8
IfConversion used to always add the undef flag when adding a use operand
on a newly predicated instruction. This would be an operand for the register
being conditionally redefined. Due to the undef flag, the liveness of this
register prior to the predicated instruction would get lost.
This patch changes this so that such use operands are added only when the
register is live, without the undef flag.
This was reverted but pushed again now, for details follow link below.
Reviewed by Quentin Colombet.
http://reviews.llvm.org/D209077
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277571 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This is the first refactoring before adding new functionality.
Add a class wrapper for the functions and container for
state associated with the transformation.
No functional change
Reviewers: majnemer, nadav, mehdi_amini
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D23065
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277565 91177308-0d34-0410-b5e6-96231b3b80d8
I forgot to do this initially, and added when I saw this fail in
a no-asserts build, but managed to loose the diff from the actual patch
that got submitted. Very sorry.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277562 91177308-0d34-0410-b5e6-96231b3b80d8
reason about and less error prone.
The core idea is to fully parse the text without trying to identify
passes or structure. This is done with a single state machine. There
were various bugs in the logic around this previously that were repeated
and scattered across the code. Having a single routine makes it much
easier to fix and get correct. For example, this routine doesn't suffer
from PR28577.
Then the actual pass construction is handled using *much* easier to read
code and simple loops, with particular pass manager construction sunk to
live with other pass construction. This is especially nice as the pass
managers *are* in fact passes.
Finally, the "implicit" pass manager synthesis is done much more simply
by forming "pre-parsed" structures rather than having to duplicate tons
of logic.
One of the bugs fixed by this was evident in the tests where we accepted
a pipeline that wasn't really well formed. Another bug is PR28577 for
which I have added a test case.
The code is less efficient than the previous code but I'm really hoping
that's not a priority. ;]
Thanks to Sean for the review!
Differential Revision: https://reviews.llvm.org/D22724
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277561 91177308-0d34-0410-b5e6-96231b3b80d8
This fixes a bug where we'd sometimes cache overly-conservative results
with our walker. This bug was made more obvious by r277480, which makes
our cache far more spotty than it was. Test case is llvm-unit, because
we're likely going to use CachingWalker only for def optimization in the
future.
The bug stems from that there was a place where the walker assumed that
`DefNode.Last` was a valid target to cache to when failing to optimize
phis. This is sometimes incorrect if we have a cache hit. The fix is to
use the thing we *can* assume is a valid target to cache to. :)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277559 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Sometimes, bitsets could get really large (>300k entries) and
we might want to drop a check, as it would have a too much cost.
Adding a flag to control how much penalty are we willing to pay
for bitsets.
Reviewers: kcc
Differential Revision: https://reviews.llvm.org/D23088
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277556 91177308-0d34-0410-b5e6-96231b3b80d8
Leftover from D22686; the passes can handle all the instructions
unconditionally; only isel needs to care whether to generate them.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277549 91177308-0d34-0410-b5e6-96231b3b80d8
In this particular example we wouldn't want the smmls anyway (the value is
actually unused), but in general smmls does not provide the required flags
register so if that SUBE result is used we can't replace it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277541 91177308-0d34-0410-b5e6-96231b3b80d8
Fixed the last incorrect uses of llvm_unreachable() in the code
which were actually just cases of errors in the input Archives.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277540 91177308-0d34-0410-b5e6-96231b3b80d8
There were issues with simply reporting AttrUnknown on
previously-unknown values in CFLAnders. So, we now act *entirely*
conservatively for values we haven't seen before. As in the prior patch
(r277362), writing a lit test for this isn't exactly trivial. If someone
wants a test badly, I'm willing to try to write one.
Patch by Jia Chen.
Differential Revision: https://reviews.llvm.org/D23077
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277533 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: We really want to move towards MemoryLocOrCall (or fix AA) everywhere, but for now, this lets us have a single instructionClobbersQuery.
Reviewers: george.burgess.iv
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D23072
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277530 91177308-0d34-0410-b5e6-96231b3b80d8
As agreed in post-commit review of r265388, I'm switching the flag to
its original value until the 90% runtime performance regression on
SingleSource/Benchmarks/Stanford/Bubblesort is addressed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277524 91177308-0d34-0410-b5e6-96231b3b80d8
We were relying on the misleadingly-names $status result to actually be the
status. Actually it's just a scratch register that may or may not be valid (and
is the inverse of the real ststus anyway). Success can be determined by
comparing the value loaded against the one we wanted to see for "cmpxchg
strong" loops like this.
Should fix PR28819.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277513 91177308-0d34-0410-b5e6-96231b3b80d8
This changes IRCE to "preserve" LCSSA and DomTree by recomputing them.
It still does not preserve LoopSimplify.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277505 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Two types of stores are possible in pixel shaders: stores to memory that are
explicitly requested at the API level, and stores that are an implementation
detail of register spilling or lowering of arrays.
For the first kind of store, we must ensure that helper pixels have no effect
and hence WQM must be disabled. The second kind of store must always be
executed, because the written value may be loaded again in a way that is
relevant for helper pixels as well -- and there are no externally visible
effects anyway.
This is a candidate for the 3.9 release branch.
Reviewers: arsenm, tstellarAMD, mareko
Subscribers: arsenm, kzhuravl, llvm-commits
Differential Revision: https://reviews.llvm.org/D22675
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277504 91177308-0d34-0410-b5e6-96231b3b80d8
Attempt 2: Retryign after Tsan.mman test fix.
Attempt 1: Recommitting after fixing test.
When parsing assembly where the line comment syntax is not hash, the
lexer cannot distinguish between hash's that start a hash line comment
and one that is part of an assembly statement and must be distinguished
during parsing. Previously, this was incompletely handled by not checking
for EndOfStatement at the end of statements and interpreting hash
prefixed statements as comments.
Change EndOfStatement Parsing to check for Hash comments and reintroduce
Hash statement parsing to catch previously handled cases.
Reviewers: rnk, majnemer
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D23017
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277501 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
There are cases where uniform branch conditions are computed in VGPRs, and
we didn't correctly mark those as WQM.
The stray change in basic-branch.ll is because invoking the LiveIntervals
analysis leads to the detection of a dead register that would otherwise not
be seen at -O0.
This is a candidate for the 3.9 branch, as it fixes a possible hang.
Reviewers: arsenm, tstellarAMD, mareko
Subscribers: arsenm, llvm-commits, kzhuravl
Differential Revision: https://reviews.llvm.org/D22673
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277500 91177308-0d34-0410-b5e6-96231b3b80d8
Identify patterns where the address is aligned to an 8-byte boundary,
but both the base address and the constant offset are both proper
multiples of 4. In such cases, extract Base+4 into a separate instruc-
tion, and use S2_storerd_io, instead of using S4_storerd_rr.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277497 91177308-0d34-0410-b5e6-96231b3b80d8
Recommitting after fixing overaggressive fastpath return in parsing.
Fix intel syntax special case identifier operands that refer to a constant
(e.g. .set <ID> n) to be interpreted as immediate not memory in parsing.
Associated commit to fix clang test commited shortly.
Reviewers: rnk
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D22585
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277489 91177308-0d34-0410-b5e6-96231b3b80d8
None of GlobalISel requires the property, but this lets us use the
verifier instead of rolling our own "all instructions selected" check.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277484 91177308-0d34-0410-b5e6-96231b3b80d8
After instruction selection, there should be no pre-isel generic
instructions remaining, nor should generic virtual registers be
used. Verify that.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277483 91177308-0d34-0410-b5e6-96231b3b80d8
Selected: the InstructionSelect pass ran and all pre-isel generic
instructions have been eliminated; i.e., all instructions are now
target-specific or non-pre-isel generic instructions (e.g., COPY).
Since only pre-isel generic instructions can have generic virtual register
operands, this also means that all generic virtual registers have been
constrained to virtual registers (assigned to register classes) and that
all sizes attached to them have been eliminated.
This lets us enforce certain invariants across passes.
This property is GlobalISel-specific, but is always available.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277482 91177308-0d34-0410-b5e6-96231b3b80d8
Fixes PR28670
Summary:
Rewrite the use optimizer to be less memory intensive and 50% faster.
Fixes PR28670
The new use optimizer works like a standard SSA renaming pass, storing
all possible versions a MemorySSA use could get in a stack, and just
tracking indexes into the stack.
This uses much less memory than caching N^2 alias query results.
It's also a lot faster.
The current version defers phi node walking to the normal walker.
Reviewers: george.burgess.iv
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D23032
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277480 91177308-0d34-0410-b5e6-96231b3b80d8
RegBankSelected functions shouldn't have any generic virtual
register not assigned to a bank. Verify that.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277476 91177308-0d34-0410-b5e6-96231b3b80d8
RegBankSelected: the RegBankSelect pass ran and all generic virtual
registers have been assigned to a register bank.
This lets us enforce certain invariants across passes.
This property is GlobalISel-specific, but is always available.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277475 91177308-0d34-0410-b5e6-96231b3b80d8
This patch enables the vectorizer to generate both scalar and vector versions
of an integer induction variable for a given loop. Previously, we only
generated a scalar induction variable if we knew all its users were going to be
scalar. Otherwise, we generated a vector induction variable. In the case of a
loop with both scalar and vector users of the induction variable, we would
generate the vector induction variable and extract scalar values from it for
the scalar users. With this patch, we now generate both versions of the
induction variable when there are both scalar and vector users and select which
version to use based on whether the user is scalar or vector.
Differential Revision: https://reviews.llvm.org/D22869
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277474 91177308-0d34-0410-b5e6-96231b3b80d8
RegBankSelect and InstructionSelect run after the legalizer and
require a Legalized function: check that all instructions are legal.
Note that this should be in the MachineVerifier, but it can't use the
MachineLegalizer as it's currently in the separate GlobalISel library.
Note that the RegBankSelect verifier checks have the same layering
problem, but we only use inline methods so end up not needing to link
against the GlobalISel library.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277472 91177308-0d34-0410-b5e6-96231b3b80d8
We currently use and test these, and select most of them. Mark them
as legal even though we don't go through the full ir->asm flow yet.
This doesn't currently have standalone tests, but the verifier will
soon learn to check that the regbankselect/select tests are legal.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277471 91177308-0d34-0410-b5e6-96231b3b80d8
Legalized: The MachineLegalizer ran; all pre-isel generic instructions
have been legalized, i.e., all instructions are now one of:
- generic and always legal (e.g., COPY)
- target-specific
- legal pre-isel generic instructions.
This lets us enforce certain invariants across passes.
This property is GlobalISel-specific, but is always available.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277470 91177308-0d34-0410-b5e6-96231b3b80d8
This is only used for debug prints, but the previous hardcoded ", "
caused it to be printed unnecessarily when OnlySet, and is annoying
when adding new properties.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277465 91177308-0d34-0410-b5e6-96231b3b80d8
This patch refactors the logic in collectLoopUniforms and
collectValuesToIgnore, untangling the concepts of "uniform" and "scalar". It
adds isScalarAfterVectorization along side isUniformAfterVectorization to
distinguish the two. Known scalar values include those that are uniform,
getelementptr instructions that won't be vectorized, and induction variables
and induction variable update instructions whose users are all known to be
scalar.
This patch includes the following functional changes:
- In collectLoopUniforms, we mark uniform the pointer operands of interleaved
accesses. Although non-consecutive, these pointers are treated like
consecutive pointers during vectorization.
- In collectValuesToIgnore, we insert a value into VecValuesToIgnore if it
isScalarAfterVectorization rather than isUniformAfterVectorization. This
differs from the previous functionaly in that we now add getelementptr
instructions that will not be vectorized into VecValuesToIgnore.
This patch also removes the ValuesNotWidened set used for induction variable
scalarization since, after the above changes, it is now equivalent to
isScalarAfterVectorization.
Differential Revision: https://reviews.llvm.org/D22867
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277460 91177308-0d34-0410-b5e6-96231b3b80d8
Recommitting after fixing test.
When parsing assembly where the line comment syntax is not hash, the
lexer cannot distinguish between hash's that start a hash line comment
and one that is part of an assembly statement and must be distinguished
during parsing. Previously, this was incompletely handled by not checking
for EndOfStatement at the end of statements and interpreting hash
prefixed statements as comments.
Change EndOfStatement Parsing to check for Hash comments and reintroduce
Hash statement parsing to catch previously handled cases.
Reviewers: rnk, majnemer
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D23017
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277459 91177308-0d34-0410-b5e6-96231b3b80d8
Added (sra (shl x, 16), 16) to the sext_16_node PatLeaf for ARM to
simplify some pattern matching. This has allowed several patterns
for smul* and smla* to be removed as well as making it easier to add
the matching for the corresponding instructions for Thumb2 targets.
Also added two Pat classes that are predicated on Thumb2 with the
hasDSP flag and UseMulOps flags. Updated the smul codegen test with
the wider range of patterns plus the ThumbV6 and ThumbV6T2 targets.
Differential Revision: https://reviews.llvm.org/D22908
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277450 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Commit 276701 requires that targets have the DSP extensions to use
certain saturating instructions. This requires some corrections.
For ARM ISA the instructions in question are available in all v6*
architectures.
For Thumb2, the instructions in question are available from v6T2.
SSAT and USAT are part of the base architecture while SSAT16 and
USAT16 require the DSP extensions.
Reviewers: rengolin
Subscribers: aemerson, rengolin, samparker, llvm-commits
Differential Revision: https://reviews.llvm.org/D23010
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277439 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This commit changes the Verifier class to accept a Module via the
constructor to make it obvious that a specific instance of the class is
only intended to work with a specific module. The `updateModule` setter
(despite being private) was making this fact less transparent.
There are fields in the `Verifier` class like `DeoptimizeDeclarations`
and `GlobalValueVisited` which are module specific, so a given
Verifier instance will not in fact work across multiple modules today.
This change just makes that more obvious.
The motivation is to make it easy to get to the datalayout of the
module unambiguously. That is required to verify that `inttoptr` and
`ptrtoint` constant expressions are well typed in the face of
non-integral pointer types.
Reviewers: dexonsmith, bkramer, majnemer, chandlerc
Subscribers: mehdi_amini, mcrosier, llvm-commits
Differential Revision: https://reviews.llvm.org/D23040
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277409 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
When parsing assembly where the line comment syntax is not hash, the
lexer cannot distinguish between hash's that start a hash line comment
and one that is part of an assembly statement and must be distinguished
during parsing. Previously, this was incompletely handled by not checking
for EndOfStatement at the end of statements and interpreting hash
prefixed statements as comments.
Change EndOfStatement Parsing to check for Hash comments and reintroduce
Hash statement parsing to catch previously handled cases.
Reviewers: rnk, majnemer
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D23017
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277407 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: By generalize the interface, users are able to inject more flexible Node token into the algorithm, for example, a pair of vector<Node>* and index integer. Currently I only migrated SCCIterator to use NodeRef, but more is coming. It's a NFC.
Reviewers: dblaikie, chandlerc
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D22937
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277399 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: This patch implements CFI for WebAssembly. It modifies the
LowerTypeTest pass to pre-assign table indexes to functions that are
called indirectly, and lowers type checks to test against the
appropriate table indexes. It also modifies the WebAssembly backend to
support a special ".indidx" assembly directive that propagates the table
index assignments out to the linker.
Patch by Dominic Chen
Differential Revision: https://reviews.llvm.org/D21768
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277398 91177308-0d34-0410-b5e6-96231b3b80d8
Common symbol support in ORC was broken in r270716 when the symbol resolution
rules in RuntimeDyld were changed. With the switch to lazily materialized
symbols in r277386, common symbols can be supported by having
RuntimeDyld::emitCommonSymbols search for (but not materialize!) definitions
elsewhere in the logical dylib.
This patch adds the 'Common' flag to JITSymbolFlags, and the necessary check
to RuntimeDyld::emitCommonSymbols.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277397 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: This patch includes asm.js-style exception handling support for
WebAssembly. The WebAssembly MVP does not have any support for
unwinding or non-local control flow. In order to support C++ exceptions,
emscripten currently uses JavaScript exceptions along with some support
code (written in JavaScript) that is bundled by emscripten with the
generated code.
This scheme lowers exception-related instructions for wasm such that
wasm modules can be compatible with emscripten's existing scheme and
share the support code.
Patch by Heejin Ahn
Differential Revision: https://reviews.llvm.org/D22958
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277391 91177308-0d34-0410-b5e6-96231b3b80d8
The FPM is split at regular intervals across the MSF file, as the MS code
suggests. It turns out that the value of the interval is precisely the
block size. If the block size is 4096, then there are two Fpm pages every
4096 blocks.
So here we teach the PDBFile class to parse a split FPM, and also add more
options when dumping the FPM to display some additional information such
as orphaned pages (pages which the FPM says are allocated, but which
nothing appears to use), use after free pages (pages which the FPM says
are not allocated, but which are referenced by a stream), and multiple use
pages (pages which the FPM says are allocated but are used more than
once).
Reviewed By: ruiu
Differential Revision: https://reviews.llvm.org/D23022
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277388 91177308-0d34-0410-b5e6-96231b3b80d8
This patch replaces RuntimeDyld::SymbolInfo with JITSymbol: A symbol class
that is capable of lazy materialization (i.e. the symbol definition needn't be
emitted until the address is requested). This can be used to support common
and weak symbols in the JIT (though this is not implemented in this patch).
For consistency, RuntimeDyld::SymbolResolver is renamed to JITSymbolResolver.
For space efficiency a new class, JITEvaluatedSymbol, is introduced that
behaves like the old RuntimeDyld::SymbolInfo - i.e. it is just a pair of an
address and symbol flags. Instances of JITEvaluatedSymbol can be used in
symbol-tables to avoid paying the space cost of the materializer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277386 91177308-0d34-0410-b5e6-96231b3b80d8
We used to combine "sext(setcc x, y, cc) -> (select (setcc x, y, cc), -1, 0)"
Instead, we should combine to (select (setcc x, y, cc), T, 0) where the value
of T is 1 or -1, depending on the type of the setcc, and getBooleanContents()
for the type if it is not i1.
This fixes PR28504.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277371 91177308-0d34-0410-b5e6-96231b3b80d8
As it turns out, modref queries are broken with CFLAA. Specifically,
the data source we were using for determining modref behaviors
explicitly ignores operations on non-pointer values. So, it wouldn't
note e.g. storing an i32 to an i32* (or loading an i64 from an i64*).
It also ignores external function calls, rather than acting
conservatively for them.
(N.B. These operations, where necessary, *are* tracked by CFLAA; we just
use a different mechanism to do so. Said mechanism is relatively
imprecise, so it's unlikely that we can provide reasonably good modref
answers with it as implemented.)
Patch by Jia Chen.
Differential Revision: https://reviews.llvm.org/D22978
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277366 91177308-0d34-0410-b5e6-96231b3b80d8
Currently, CFLAnders assumes that values it hasn't seen don't alias
anything. This patch fixes that. Given that the only way for this to
happen is to query AA, rely on specific transformations happening, then
query AA again (looking for a specific set of queries), lit testing is a
bit difficult. If someone really wants a test, I'm happy to add one.
Patch by Jia Chen.
Differential Revision: https://reviews.llvm.org/D22981
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277362 91177308-0d34-0410-b5e6-96231b3b80d8
Scavenging slots were only reserved when pseudo-instruction expansion in
frame lowering created new virtual registers. It is possible to still
need a scavenging slot even if no virtual registers were created, in cases
where the stack is large enough to overflow instruction offsets.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277355 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Allocating an AFGR64 shadows two GPR32's instead of just one.
This fixes an LNT regression detected by our internal buildbots.
Reviewers: sdardis
Subscribers: dsanders, sdardis, llvm-commits
Differential Revision: https://reviews.llvm.org/D23012
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277348 91177308-0d34-0410-b5e6-96231b3b80d8
Using RAUW was wrong here; if we have a switch transform such as:
18 -> 6 then
6 -> 0
If we use RAUW, while performing the second transform the *transformed* 6
from the first will be also replaced, so we end up with:
18 -> 0
6 -> 0
Found by clang stage2 bootstrap; testcase added.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277332 91177308-0d34-0410-b5e6-96231b3b80d8
The branch relaxation pass is computing the wrong offsets because it assumes
TLSDESC_CALLSEQ eats up 4 bytes, when in fact it is lowered to an instruction
sequence taking up 16 bytes. This can become a problem in huge files with lots
of TLS accesses, as it may slowly move branch targets out of the range computed
by the branch relaxation pass.
Fixes PR24234 https://llvm.org/bugs/show_bug.cgi?id=24234
Differential Revision: https://reviews.llvm.org/D22870
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277331 91177308-0d34-0410-b5e6-96231b3b80d8
If a switch is sparse and all the cases (once sorted) are in arithmetic progression, we can extract the common factor out of the switch and create a dense switch. For example:
switch (i) {
case 5: ...
case 9: ...
case 13: ...
case 17: ...
}
can become:
if ( (i - 5) % 4 ) goto default;
switch ((i - 5) / 4) {
case 0: ...
case 1: ...
case 2: ...
case 3: ...
}
or even better:
switch ( ROTR(i - 5, 2) {
case 0: ...
case 1: ...
case 2: ...
case 3: ...
}
The division and remainder operations could be costly so we only do this if the factor is a power of two, and emit a right-rotate instead of a divide/remainder sequence. Dense switches can be lowered significantly better than sparse switches and can even be transformed into lookup tables.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277325 91177308-0d34-0410-b5e6-96231b3b80d8
Initialize all AArch64-specific passes in the TargetMachine so they can be run
by llc. This can lead to conflicts in opt with some command line options that
share the same name as the pass, so I took this opportunity to do some cleanups:
* rename all relevant command line options from "aarch64-blah" to
"aarch64-enable-blah" and update the tests accordingly
* run clang-format on their declarations
* move all these declarations to a common place (the TargetMachine) as opposed
to having them scattered around (AArch64BranchRelaxation and
AArch64AddressTypePromotion were the only offenders)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277322 91177308-0d34-0410-b5e6-96231b3b80d8
Had to update a stack folding test to clobber the other 16 registers since this now made them get used instead of spilling.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277321 91177308-0d34-0410-b5e6-96231b3b80d8
When extracting a set of blocks make sure to inherit all of the target
dependent attributes to make sure that the function will be valid for
lowering. One example is the "target-features" attribute for x86, if the
extracted region has functionality that relies on a specific feature it
will fail to be lowered.
This also allows for extracted functions to be valid for inlining, at
least back into the parent function, as the target attributes are tested
when inlining for compatibility.
Patch by River Riddle!
Differential Revision: https://reviews.llvm.org/D22713
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277315 91177308-0d34-0410-b5e6-96231b3b80d8
We had import_directory_table_entry and
coff_import_directory_table_entry, remove one. Also, factor out the
logic which determins if a descriptor is a terminator.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277296 91177308-0d34-0410-b5e6-96231b3b80d8
The 2i32-2i64 legalization means that we can use the slightly quicker double bits + fptrunc approach for the same results
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277271 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
When performing cmp for EQ/NE and the operand is sign extended, we can
avoid the truncaton if the bits to be tested are no less than origianl
bits.
Reviewers: eli.friedman
Subscribers: eli.friedman, aemerson, nemanjai, t.p.northover, llvm-commits
Differential Revision: https://reviews.llvm.org/D22933
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277252 91177308-0d34-0410-b5e6-96231b3b80d8
These come in two variants for now: G_INTRINSIC and G_INTRINSIC_W_SIDE_EFFECTS.
We may decide to split the latter up with finer-grained restrictions later, if
necessary.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277224 91177308-0d34-0410-b5e6-96231b3b80d8
in r277177 and added back this test which was deleted in r277196 while
I tracked down these problems.
Changed from constructing Twine's to std::string's as Twine's don't work
across statements. Also removed a few unneeded Twine() constructions.
Fix the write_escaped() calls to not pass the unintended second argument
fixing the warning on the ld-x86_64-win7 bot.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277223 91177308-0d34-0410-b5e6-96231b3b80d8
Up until now, we only had code to match PSADBW patterns that look like what
comes out of the loop vectorizer - a partial reduction inside the loop body
that gets fed into a horizontal operation in a different basic block.
This adds support for straight-line patterns, like those generated by the
SLP vectorizer.
Differential Revision: https://reviews.llvm.org/D22889
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277219 91177308-0d34-0410-b5e6-96231b3b80d8
Support for lowering to VBROADCASTF128 etc. in D22460 was not correctly ensuring that the only users of the 128-bit vector load were the insertions of the vector into the lower/upper subvectors.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277214 91177308-0d34-0410-b5e6-96231b3b80d8
Previously this change was submitted from a Windows machine, so
changes made to the case of filenames and directory names did
not survive the commit, and as a result the CMake source file
names and the on-disk file names did not match on case-sensitive
file systems.
I'm resubmitting this patch from a Linux system, which hopefully
allows the case changes to make it through unfettered.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277213 91177308-0d34-0410-b5e6-96231b3b80d8
This will be used during GlobalISel, where we need a more robust and readable
way to write tests than a simple immediate ID.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277209 91177308-0d34-0410-b5e6-96231b3b80d8
LoopUnroll is a loop pass, so the analysis of OptimizationRemarkEmitter
is added to the common function analysis passes that loop passes
depend on.
The BFI and indirectly BPI used in this pass is computed lazily so no
overhead should be observed unless -pass-remarks-with-hotness is used.
This is how the patch affects the O3 pipeline:
Dominator Tree Construction
Natural Loop Information
Canonicalize natural loops
Loop-Closed SSA Form Pass
Basic Alias Analysis (stateless AA impl)
Function Alias Analysis Results
Scalar Evolution Analysis
+ Lazy Branch Probability Analysis
+ Lazy Block Frequency Analysis
+ Optimization Remark Emitter
Loop Pass Manager
Rotate Loops
Loop Invariant Code Motion
Unswitch loops
Simplify the CFG
Dominator Tree Construction
Basic Alias Analysis (stateless AA impl)
Function Alias Analysis Results
Combine redundant instructions
Natural Loop Information
Canonicalize natural loops
Loop-Closed SSA Form Pass
Scalar Evolution Analysis
+ Lazy Branch Probability Analysis
+ Lazy Block Frequency Analysis
+ Optimization Remark Emitter
Loop Pass Manager
Induction Variable Simplification
Recognize loop idioms
Delete dead loops
Unroll loops
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277203 91177308-0d34-0410-b5e6-96231b3b80d8
We used the wrong type for constructing a zero vector element which led
to type mismatches.
This fixes PR28771.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277197 91177308-0d34-0410-b5e6-96231b3b80d8
In a previous patch, it was suggested to use all caps instead of
rolling caps for initialisms, so this patch changes everything
to do this.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277190 91177308-0d34-0410-b5e6-96231b3b80d8
Patch by Sunita Marathe
Third try, now following fixes to MSan to handle mempcy in such a way that this commit won't break the MSan buildbots. (Thanks, Evegenii!)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277189 91177308-0d34-0410-b5e6-96231b3b80d8
The following pattern was being layed out poorly:
A
/ \
B C
/ \ / \
D E ? (Doesn't matter)
Where A->B is far more likely than A->C, and prob(B->D) = prob(B->E)
The current algorithm gives:
A,B,C,E (D goes on worklist)
It does this even if C has a frequency count of 0. This patch
adjusts the layout calculation so that if freq(B->E) >> freq(C->E)
then we go ahead and layout E rather than C. Fallthrough half the time
is better than fallthrough never, or fallthrough very rarely. The
resulting layout is:
A,B,E, (C and D are in a worklist)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277187 91177308-0d34-0410-b5e6-96231b3b80d8
Just the basic equivalent to DAG's condbr for now, we'll get to things like
br_cc when we start doing more legalization.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277184 91177308-0d34-0410-b5e6-96231b3b80d8
The DAG combiner tries to merge stores to adjacent vector wide memory
locations by creating stores which are integral multiples of the vector
width. Discourage this by informing it that this is slow. This should
not affect legalization passes, because all of them ignore the "Fast"
argument.
Patch by Pranav Bhandarkar.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277178 91177308-0d34-0410-b5e6-96231b3b80d8
As mentioned in commit log for r276686 this next step is adding a new
method in the ArchiveMemberHeader class to get the full name that
does proper error checking, and can be use for error messages.
To do this the name of ArchiveMemberHeader::getName() is changed to
ArchiveMemberHeader::getRawName() to be consistent with
Archive::Child::getRawName(). Then the “new” method is the addition
of a new implementation of ArchiveMemberHeader::getName() which gets
the full name and provides proper error checking. Which is mostly a rewrite
of what was Archive::Child::getName() and cleaning up incorrect uses of
llvm_unreachable() in the code which were actually just cases of errors
in the input Archives.
Then Archive::Child::getName() is changed to return Expected<> and use
the new implementation of ArchiveMemberHeader::getName() .
Also needed to change Archive::getMemoryBufferRef() with these
changes to return Expected<> as well to propagate Errors up.
As well as changing Archive::isThinMember() to return Expected<> .
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277177 91177308-0d34-0410-b5e6-96231b3b80d8
For MachineInstrBuilder, having to manually use RegState::Define is ugly and
makes register definitions clunkier than they need to be, so this adds two
convenience functions: addDef and addUse.
For MachineIRBuilder, we want to avoid BuildMI's first-reg-is-def rule because
it's hidden away and causes bugs. So this patch switches buildInstr to
returning a MachineInstrBuilder and adding *all* operands via addDef/addUse.
NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277176 91177308-0d34-0410-b5e6-96231b3b80d8
Mostly straightforward as we ignore addressing modes and just
use the base + unsigned immediate offset (always 0) variants.
This currently fails to select extloads because we have yet to
agree on a representation.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277171 91177308-0d34-0410-b5e6-96231b3b80d8
Software pipelining is an optimization for improving ILP by
overlapping loop iterations. Swing Modulo Scheduling (SMS) is
an implementation of software pipelining that attempts to
reduce register pressure and generate efficient pipelines with
a low compile-time cost.
This implementaion of SMS is a target-independent back-end pass.
When enabled, the pass should run just prior to the register
allocation pass, while the machine IR is in SSA form. If the pass
is successful, then the original loop is replaced by the optimized
loop. The optimized loop contains one or more prolog blocks, the
pipelined kernel, and one or more epilog blocks.
This pass is enabled for Hexagon only. To enable for other targets,
a couple of target specific hooks must be implemented, and the
pass needs to be called from the target's TargetMachine
implementation.
Differential Review: http://reviews.llvm.org/D16829
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277169 91177308-0d34-0410-b5e6-96231b3b80d8
If the mask of a vector shuffle has alternating odd or even numbers
starting with 1 or 0 respectively up to the largest possible index
for the given type in the given HVX mode (single of double) we can
generate vpacko or vpacke instruction respectively.
E.g.
%42 = shufflevector <32 x i16> %37, <32 x i16> %41,
<32 x i32> <i32 1, i32 3, ..., i32 63>
is %42.h = vpacko(%41.w, %37.w)
Patch by Pranav Bhandarkar.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277168 91177308-0d34-0410-b5e6-96231b3b80d8
Rebalances address calculation trees and applies Hexagon-specific
optimizations to the trees to improve instruction selection.
Patch by Tobias Edler von Koch.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277151 91177308-0d34-0410-b5e6-96231b3b80d8
The post register allocator scheduler can generate poor schedules
because the scoreboard hazard recognizer is unable to identify
hazards for Hexagon precisely. Instead, Hexagon should use a DFA
based hazard recognizer.
Patch by Brendon Cahoon.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277143 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Implements fastLowerArguments() to avoid the need to fall back on
SelectionDAG for 0-4 argument functions that don't do tricky things like
passing double in a pair of i32's.
This allows us to move all except one test to -fast-isel-abort=3. The
remaining one has function prototypes of the form 'i32 (i32, double, double)'
which requires floats to be passed in GPR's.
The previous commit had an uninitialized variable that caused the incoming
argument region to have undefined size. This has been fixed.
Reviewers: sdardis
Subscribers: dsanders, llvm-commits, sdardis
Differential Revision: https://reviews.llvm.org/D22680
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277136 91177308-0d34-0410-b5e6-96231b3b80d8
[DAG] Check debug values for invalidation before transferring and mark
old debug values invalid when transferring to another SDValue.
This fixes PR28613.
Reviewers: jyknight, hans, dblaikie, echristo
Subscribers: yaron.keren, ismail, llvm-commits
Differential Revision: https://reviews.llvm.org/D22858
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277135 91177308-0d34-0410-b5e6-96231b3b80d8
We currently default to using either generic shuffles or MASK+PACKUS/PACKSS to truncate all integer vectors. For vector comparisons, we know that the result will be either all or zero bits in every element, which can be efficiently truncated by directly using PACKSS to repeatedly halve the size of each element.
Due to the limited input values (-1 or 0) we don't need to account for vector element size, so for simplicity we just use the PACKSS(vXi16,vXi16) implementation in all cases. Additionally for AVX2 PACKSS of 256bit data we must perform a PERMQ shuffle to reorder the data into the correct order. I did investigate performing a single shuffle after all the PACKSS calls but the need to cross 128bit lanes makes this difficult to achieve efficiently.
We avoid performing this on AVX512 as it should have better alternative truncation instructions.
Differential Revision: https://reviews.llvm.org/D22814
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277132 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
The MOV/MOVT instructions being chosen for struct_byval predicates was
conditional only on Thumb2, resulting in an ARM MOV/MOVT instruction
being incorrectly emitted in Thumb1 mode. This is especially apparent
with v8-m.base targets. This patch ensures that Thumb instructions are
emitted in both Thumb modes.
Reviewers: rengolin, t.p.northover
Subscribers: llvm-commits, aemerson, rengolin
Differential Revision: https://reviews.llvm.org/D22865
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277128 91177308-0d34-0410-b5e6-96231b3b80d8
Some instructions may have their uses replaced with a symbolic constant.
However, the instruction may still have side effects which percludes it
from being removed from the function. EarlyCSE treated such an
instruction as if it were removed, resulting in PR28763.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277114 91177308-0d34-0410-b5e6-96231b3b80d8
