// shuffle (concat X, undef), (concat Y, undef), Mask -->
// concat (shuffle X, Y, Mask0), (shuffle X, Y, Mask1)
The ARM changes with 'vtrn' and narrowed 'vuzp' are improvements.
The x86 changes look neutral or better. There's one test with an
extra instruction, but that could be reversed for a subtarget with
the right attributes. But by default, we want to avoid the 256-bit
op when possible (in my motivating benchmark, a handful of ymm ops
sprinkled into a sequence of xmm ops are triggering frequency
throttling on Haswell resulting in significantly worse perf).
Differential Revision: https://reviews.llvm.org/D60545
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358291 91177308-0d34-0410-b5e6-96231b3b80d8
Currently combineHorizontalPredicateResult only handles anyof/allof reduction patterns of legal types, which can be tricky to match as type legalization of bools can introduce bitcasts/truncs/extensions.
This patch extends combineHorizontalPredicateResult to recognise vXi1 bool reductions as well and uses the existing combineBitcastvxi1 helper to create the MOVMSK necessary to then compare the signmask result.
This ensures the accuracy of the reduction costs added in D60403 which assume the MOVMSK generation.
Differential Revision: https://reviews.llvm.org/D60610
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358286 91177308-0d34-0410-b5e6-96231b3b80d8
It causes clang to crash while building Chromium. See https://crbug.com/952230
for reproducer.
> The PrologEpilogInserter need to insert a DW_OP_deref_size before
> prepending a memory location expression to an already implicit
> expression to avoid having the existing expression act on the memory
> address instead of the value behind it.
>
> The reason for using DW_OP_deref_size and not plain DW_OP_deref is that
> big-endian targets need to read the right size as simply truncating a
> larger read would yield the wrong result (LSB bytes are not at the lower
> address).
>
> Differential Revision: https://reviews.llvm.org/D59687
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358281 91177308-0d34-0410-b5e6-96231b3b80d8
The PrologEpilogInserter need to insert a DW_OP_deref_size before
prepending a memory location expression to an already implicit
expression to avoid having the existing expression act on the memory
address instead of the value behind it.
The reason for using DW_OP_deref_size and not plain DW_OP_deref is that
big-endian targets need to read the right size as simply truncating a
larger read would yield the wrong result (LSB bytes are not at the lower
address).
Differential Revision: https://reviews.llvm.org/D59687
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358268 91177308-0d34-0410-b5e6-96231b3b80d8
If the upper bits of the SHL result aren't used, we might be able to use a narrower shift. For example, on X86 this can turn a 64-bit into 32-bit enabling a smaller encoding.
Differential Revision: https://reviews.llvm.org/D60358
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358257 91177308-0d34-0410-b5e6-96231b3b80d8
If the vector setcc has been legalized then we will need to convert a vector boolean of 0 or -1 to a scalar boolean of 0 or 1.
The added test case previously crashed in 32-bit mode by creating a setcc with an i64 condition that type legalization couldn't expand.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358218 91177308-0d34-0410-b5e6-96231b3b80d8
This patch adds patterns for turning bitcasted atomic load/store into movss/sd.
It also removes the pseudo instructions for atomic RMW fadd. Instead just adding isel patterns for folding an atomic load into addss/sd. And relying on the new movss/sd store pattern to handle the write part.
This also makes the fadd patterns use VEX and EVEX instructions when AVX or AVX512F are enabled.
Differential Revision: https://reviews.llvm.org/D60394
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358215 91177308-0d34-0410-b5e6-96231b3b80d8
With correct test checks this time.
If we have X87, but not SSE2 we can atomicaly load an i64 value into the significand of an 80-bit extended precision x87 register using fild. We can then use a fist instruction to convert it back to an i64 integ
This matches what gcc and icc do for this case and removes an existing FIXME.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358214 91177308-0d34-0410-b5e6-96231b3b80d8
If we have X87, but not SSE2 we can atomicaly load an i64 value into the significand of an 80-bit extended precision x87 register using fild. We can then use a fist instruction to convert it back to an i64 integer and store it to a stack temporary. From there we can do two 32-bit loads to get the value into integer registers without worrying about atomicness.
This matches what gcc and icc do for this case and removes an existing FIXME.
Differential Revision: https://reviews.llvm.org/D60156
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358211 91177308-0d34-0410-b5e6-96231b3b80d8
Completes SimplifyDemandedVectorElts's basic variable shuffle mask support which should help D60512 + D60562
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358186 91177308-0d34-0410-b5e6-96231b3b80d8
Original test was too dependent on the order of the combines that could cause the inserted element being demanded after all
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358182 91177308-0d34-0410-b5e6-96231b3b80d8
// bo (build_vec ...undef, x, undef...), (build_vec ...undef, y, undef...) -->
// build_vec ...undef, (bo x, y), undef...
The lifetime of the nodes in these examples is different for variables versus constants,
but they are all build vectors briefly, so I'm proposing to catch them in this form to
handle all of the leading examples in the motivating test file.
Before we have build vectors, we might have insert_vector_element. After that, we might
have scalar_to_vector and constant pool loads.
It's going to take more work to ensure that FP vector operands are getting simplified
with undef elements, so this transform can apply more widely. In a non-loose FP environment,
we are likely simplifying FP elements to NaN values rather than undefs.
We also need to allow more opcodes down this path. Eg, we don't handle FP min/max flavors
yet.
Differential Revision: https://reviews.llvm.org/D60514
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358172 91177308-0d34-0410-b5e6-96231b3b80d8
We need to add support for all variable shuffle mask ops, but VPPERM is the only one that already has test coverage.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358165 91177308-0d34-0410-b5e6-96231b3b80d8
foldMaskedShiftToScaledMask tries to reorder and & shl to enable the shl to fold into an LEA. But if there is an any_extend between them it doesn't work.
This patch modifies the code to look through any_extend from i32 to i64 when the and mask only uses bits that weren't from the extended part.
This will prevent a regression from D60358 caused by 64-bit SHL being narrowed to 32-bits when their upper bits aren't demanded.
Differential Revision: https://reviews.llvm.org/D60532
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358139 91177308-0d34-0410-b5e6-96231b3b80d8
If we have an (add X, (and (aext (shl Y, C1)), C2)), we can pull the shift through and+aext to fold into an LEA with the.
Assuming C1 is small enough and C2 masks off all of the extend bits.
This pattern showed up in D60358. And we need to handle it to prevent a regression.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358124 91177308-0d34-0410-b5e6-96231b3b80d8
Certain optimisations from ConstantHoisting and CGP rely on Selection DAG not
seeing through to the constant in other blocks. Revert this patch while we come
up with a better way to handle that.
I will try to follow this up with some better tests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358113 91177308-0d34-0410-b5e6-96231b3b80d8
This lines up with what we do for regular subtract and it matches up better with X86 assumptions in isel patterns that add with immediate is more canonical than sub with immediate.
Differential Revision: https://reviews.llvm.org/D60020
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358027 91177308-0d34-0410-b5e6-96231b3b80d8
When bitcasting from a source op to a larger bitwidth op, split the demanded bits and OR them on top of one another and demand those merged bits in the SimplifyDemandedBits call on the source op.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@357992 91177308-0d34-0410-b5e6-96231b3b80d8
Be more selective in the SimplifyDemandedBits -> SimplifyDemandedVectorElts bitcast call based on the demanded elts.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@357942 91177308-0d34-0410-b5e6-96231b3b80d8
I was looking at a potential DAGCombiner fix for 1 of the regressions in D60278, and it caused severe regression test pain because x86 TLI lies about the desirability of 8-bit shift ops.
We've hinted at making all 8-bit ops undesirable for the reason in the code comment:
// TODO: Almost no 8-bit ops are desirable because they have no actual
// size/speed advantages vs. 32-bit ops, but they do have a major
// potential disadvantage by causing partial register stalls.
...but that leads to massive diffs and exposes all kinds of optimization holes itself.
Differential Revision: https://reviews.llvm.org/D60286
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@357912 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Previously we would use MOVZXrm8/MOVZXrm16, but those are longer encodings.
This is similar to what we do in the loadi32 predicate.
Reviewers: RKSimon, spatel
Reviewed By: RKSimon
Subscribers: hiraditya, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D60341
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@357875 91177308-0d34-0410-b5e6-96231b3b80d8
In the case where we only want the sign bit (e.g. when using PACKSS truncation of comparison results for MOVMSK) then we can just demand the sign bit of the source operands.
This makes use of the fact that PACKSS saturates out of range values to the min/max int values - so the sign bit is always preserved.
Differential Revision: https://reviews.llvm.org/D60333
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@357859 91177308-0d34-0410-b5e6-96231b3b80d8
This function reorders AND and SHL to enable the SHL to fold into an LEA. The
upper bits of the AND will be shifted out by the SHL so it doesn't matter what
mask value we use for these bits. By using sign bits from the original mask in
these upper bits we might enable a shorter immediate encoding to be used.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@357846 91177308-0d34-0410-b5e6-96231b3b80d8
When we shift the AND mask over we should shift in sign bits instead of zero bits. The scale in the LEA will shift these bits out so it doesn't matter whether we mask the bits off or not. Using sign bits will potentially allow a sign extended immediate to be used.
Also add some other test cases for cases that are currently optimal.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@357845 91177308-0d34-0410-b5e6-96231b3b80d8
The expansion of TCRETURNri(64) would not keep operand flags like
undef/renamable/etc. which can result in machine verifier issues.
Also add plumbing to be able to use `-run-pass=x86-pseudo`.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@357808 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This avoids needing an isel pattern for each condition code. And it removes translation switches for converting between Jcc instructions and condition codes.
Now the printer, encoder and disassembler take care of converting the immediate. We use InstAliases to handle the assembly matching. But we print using the asm string in the instruction definition. The instruction itself is marked IsCodeGenOnly=1 to hide it from the assembly parser.
Reviewers: spatel, lebedev.ri, courbet, gchatelet, RKSimon
Reviewed By: RKSimon
Subscribers: MatzeB, qcolombet, eraman, hiraditya, arphaman, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D60228
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@357802 91177308-0d34-0410-b5e6-96231b3b80d8
