The default promotion for the add_sat/sub_sat nodes currently does:
ANY_EXTEND iN to iM
SHL by M-N
[US][ADD|SUB]SAT
L/ASHR by M-N
If the promoted add_sat or sub_sat node is not legal, this can produce code
that effectively does a lot of shifting (and requiring large constants to be
materialised) just to use the overflow flag. It is simpler to just do the
saturation manually, using the higher bitwidth addition and a min/max against
the saturating bounds. That is what this patch attempts to do.
Differential Revision: https://reviews.llvm.org/D68926
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@375211 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
In the long run we should come up with another mechanism for marking
call instructions as heap allocation sites, and remove this workaround.
For now, we've had two bug reports about this, so let's apply this
workaround. SLH (the other client of instruction labels) probably has
the same bug, but the solution there is more likely to be to mark the
call instruction as not duplicatable, which doesn't work for debug info.
Reviewers: akhuang
Subscribers: aprantl, hiraditya, aganea, chandlerc, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D69068
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@375137 91177308-0d34-0410-b5e6-96231b3b80d8
Add some extra sat tests and adjust some of the existing tests to use signext where it would naturally be.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@375009 91177308-0d34-0410-b5e6-96231b3b80d8
PromoteTargetBoolean calls getSetccResultType to get the return
type. But we were passing it the setcc result type rather than the
setcc input type. This causes an issue on X86 with avx512vl where
the setcc result type for vXf16 vectors is vXi16 while the
result type for vXi16 vectors is vXi1.
There's really no guarantee that getSetccResultType is the type
we need here. So now we just grab the extend type from
getExtendForContent and extend to the original result VT of the
node we're splitting.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374970 91177308-0d34-0410-b5e6-96231b3b80d8
Examples:
i32 X > -1 ? C1 : -1 --> (X >>s 31) | C1
i8 X < 0 ? C1 : 0 --> (X >>s 7) & C1
This is a small generalization of a fold requested in PR43650:
https://bugs.llvm.org/show_bug.cgi?id=43650
The sign-bit of the condition operand can be used as a mask for the true operand:
https://rise4fun.com/Alive/paT
Note that we already handle some of the patterns (isNegative + scalar) because
there's an over-specialized, yet over-reaching fold for that in foldSelectCCToShiftAnd().
It doesn't use any TLI hooks, so I can't easily rip out that code even though we're
duplicating part of it here. This fold is guarded by TLI.convertSelectOfConstantsToMath(),
so it should not cause problems for targets that prefer select over shift.
Also worth noting: I thought we could generalize this further to include the case where
the true operand of the select is not constant, but Alive says that may allow poison to
pass through where it does not in the original select form of the code.
Differential Revision: https://reviews.llvm.org/D68949
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374902 91177308-0d34-0410-b5e6-96231b3b80d8
Add a pass to lower is.constant and objectsize intrinsics
This pass lowers is.constant and objectsize intrinsics not simplified by
earlier constant folding, i.e. if the object given is not constant or if
not using the optimized pass chain. The result is recursively simplified
and constant conditionals are pruned, so that dead blocks are removed
even for -O0. This allows inline asm blocks with operand constraints to
work all the time.
The new pass replaces the existing lowering in the codegen-prepare pass
and fallbacks in SDAG/GlobalISEL and FastISel. The latter now assert
on the intrinsics.
Differential Revision: https://reviews.llvm.org/D65280
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374784 91177308-0d34-0410-b5e6-96231b3b80d8
This prevents isel from emitting a TEST instruction that
optimizeCompareInstr will need to remove later.
In some of the modified tests, the SUB gets duplicated due to
the flags being needed in two places and being clobbered in
between. optimizeCompareInstr was able to optimize away the TEST
that was using the result of one of them, but optimizeCompareInstr
doesn't know to turn SUB into CMP after removing the TEST. It
only knows how to turn SUB into CMP if the result was already
dead.
With this change the TEST never exists, so optimizeCompareInstr
doesn't have to remove it. Then it can just turn the SUB into
CMP immediately.
Fixes PR43649.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374755 91177308-0d34-0410-b5e6-96231b3b80d8
This pass lowers is.constant and objectsize intrinsics not simplified by
earlier constant folding, i.e. if the object given is not constant or if
not using the optimized pass chain. The result is recursively simplified
and constant conditionals are pruned, so that dead blocks are removed
even for -O0. This allows inline asm blocks with operand constraints to
work all the time.
The new pass replaces the existing lowering in the codegen-prepare pass
and fallbacks in SDAG/GlobalISEL and FastISel. The latter now assert
on the intrinsics.
Differential Revision: https://reviews.llvm.org/D65280
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374743 91177308-0d34-0410-b5e6-96231b3b80d8
This enables use of the saturating truncate instructions when the
result type is less than 128 bits. It also enables the use of
saturating truncate instructions on KNL when the input is less
than 512 bits. We can do this by widening the input and then
extracting the result.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374731 91177308-0d34-0410-b5e6-96231b3b80d8
This seems to improve std::midpoint code where we have a min and
a max with the same condition. If we split the setcc we can end
up with two compares if the one of the operands is a constant.
Since we aggressively canonicalize compares with constants.
For non-constants it can interfere with our ability to share
control flow if we need to expand cmovs into control flow.
I'm also not sure I understand this min/max canonicalization code.
The motivating case talks about comparing with 0. But we don't
check for 0 explicitly.
Removes one instruction from the codegen for PR43658.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374706 91177308-0d34-0410-b5e6-96231b3b80d8
Since the input type is larger than 256-bits we'll need to some
concatenating to reassemble the results. The pack instructions
ability to concatenate while packing make this a shorter/faster
sequence.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374643 91177308-0d34-0410-b5e6-96231b3b80d8
This adds "min-legal-vector-width"="256" function attributes to
all the tests for a larger than 256-bit input. Also switch any
larger than 512-bit inputs to use a load. This makes the
arguments consistent with min-legal-vector-width attribute which
should usually be at least as large as the arguments.
The SKX configuration will avoid using zmm registers on the
modified test cases. For many of them we should use something
closer to the AVX2 codegen with pack instructions instead of
the avx512 saturating truncates.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374642 91177308-0d34-0410-b5e6-96231b3b80d8
We already did this for VTRUNCUS with a specific combination of
types. This extends this to VTRUNCS and handles any types where
a truncating store is legal.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374615 91177308-0d34-0410-b5e6-96231b3b80d8
This commit is not extending the promoted integers as it should. Reverting
whilst I look into the details.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374592 91177308-0d34-0410-b5e6-96231b3b80d8
The diffs suggest that we are missing some more basic
analysis/transforms, but this keeps the vector path in
sync with the scalar (rL374397). This is again a
preliminary step for introducing the reverse transform
in IR as proposed in D63382.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374555 91177308-0d34-0410-b5e6-96231b3b80d8
I wonder if we should split the v8i8 stores in order to form
two v4i8 saturating truncating stores. This would remove the
unpckl needed to concatenated the v4i8 results to make a
single store.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374519 91177308-0d34-0410-b5e6-96231b3b80d8
If we don't have VLX we won't end up selecting a saturating
truncate for 256-bit or smaller vectors so we should just use
the pack lowering.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374487 91177308-0d34-0410-b5e6-96231b3b80d8
The attribute is needed to avoid zmm registers. Using memory
avoids argument splitting for large vectors.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374486 91177308-0d34-0410-b5e6-96231b3b80d8
If we've disable zmm registers, the v16i32 will need to be split. This split will propagate through min/max the truncate. This creates two sequences that need to be concatenated back to v16i8. We can instead use packusdw to do part of the clamping, truncating, and concatenating all at once. Then we can use a vpmovuswb to finish off the clamp.
Differential Revision: https://reviews.llvm.org/D68763
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374431 91177308-0d34-0410-b5e6-96231b3b80d8
This reverses the scalar canonicalization proposed in D63382.
Pre: isPowerOf2(C1)
%r = select i1 %cond, i32 C1, i32 0
=>
%z = zext i1 %cond to i32
%r = shl i32 %z, log2(C1)
https://rise4fun.com/Alive/Z50
x86 already tries to fold this pattern, but it isn't done
uniformly, so we still see a diff. AArch64 probably should
enable the TLI hook to benefit too, but that's a follow-on.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374397 91177308-0d34-0410-b5e6-96231b3b80d8
The default promotion for the add_sat/sub_sat nodes currently does:
1. ANY_EXTEND iN to iM
2. SHL by M-N
3. [US][ADD|SUB]SAT
4. L/ASHR by M-N
If the promoted add_sat or sub_sat node is not legal, this can produce code
that effectively does a lot of shifting (and requiring large constants to be
materialised) just to use the overflow flag. It is simpler to just do the
saturation manually, using the higher bitwidth addition and a min/max against
the saturating bounds. That is what this patch attempts to do.
Differential Revision: https://reviews.llvm.org/D68643
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374373 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: It ensures that the bswap is generated even when a part of the subtree already matches a bswap transform.
Reviewers: craig.topper, efriedma, RKSimon, lebedev.ri
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D68250
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374340 91177308-0d34-0410-b5e6-96231b3b80d8
This adds some extra testing to the existing [su][add/sub]_sat X86 and AArch64
tests and adds equivalent tests for ARM.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374169 91177308-0d34-0410-b5e6-96231b3b80d8
Gather instructions can use i32 or i64 elements for indices. If
the index is zero extended from a type smaller than i32 to i64, we
can shrink the extend to just extend to i32.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@373982 91177308-0d34-0410-b5e6-96231b3b80d8
Simon Pilgrim