Earlier in the year intrinsics for lrint, llrint, lround and llround were
added to llvm. The constrained versions are now implemented here.
Reviewed by: andrew.w.kaylor, craig.topper, cameron.mcinally
Approved by: craig.topper
Differential Revision: https://reviews.llvm.org/D64746
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@373900 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Previously IntrinsicInfo::size was an unsigned what can't represent the
64 bit value used by MemoryLocation::UnknownSize.
Reviewers: jmolloy
Subscribers: hiraditya, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D68219
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@373214 91177308-0d34-0410-b5e6-96231b3b80d8
Merge the calls, just adjust the demandedelts if we have a valid extract_subvector constant index, else demand all elts.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@372521 91177308-0d34-0410-b5e6-96231b3b80d8
As commented on D67557 we have a lot of uses of depth checks all using magic numbers.
This patch adds the SelectionDAG::MaxRecursionDepth constant and moves over some general cases to use this explicitly.
Differential Revision: https://reviews.llvm.org/D67711
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@372315 91177308-0d34-0410-b5e6-96231b3b80d8
This is the first sweep of generic code to add isAtomic bailouts where appropriate. The intention here is to have the switch from AtomicSDNode to LoadSDNode/StoreSDNode be close to NFC; that is, I'm not looking to allow additional optimizations at this time. That will come later. See D66309 for context.
Differential Revision: https://reviews.llvm.org/D66318
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371786 91177308-0d34-0410-b5e6-96231b3b80d8
This implements constrained floating point intrinsics for FP to signed and
unsigned integers.
Quoting from D32319:
The purpose of the constrained intrinsics is to force the optimizer to
respect the restrictions that will be necessary to support things like the
STDC FENV_ACCESS ON pragma without interfering with optimizations when
these restrictions are not needed.
Reviewed by: Andrew Kaylor, Craig Topper, Hal Finkel, Cameron McInally, Roman Lebedev, Kit Barton
Approved by: Craig Topper
Differential Revision: http://reviews.llvm.org/D63782
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@370228 91177308-0d34-0410-b5e6-96231b3b80d8
Copied directly from the IR version.
Most of the testcases I've added for this are somewhat problematic
because they really end up testing the yet to be implemented version
for MUL_I24/MUL_U24.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@370099 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Before this patch MGATHER/MSCATTER is capable of representing all
common addressing modes, but only when illegal types are used.
This patch adds an IndexType property so more representations
are available when using legal types only.
Original modes:
vector of bases
base + vector of signed scaled offsets
New modes:
base + vector of signed unscaled offsets
base + vector of unsigned scaled offsets
base + vector of unsigned unscaled offsets
The current behaviour of addressing modes for gather/scatter remains
unchanged.
Patch by Paul Walker.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D65636
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@368008 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: Honoring no signed zeroes is also available as a user control through clang separately regardless of fastmath or UnsafeFPMath context, DAG guards should reflect this context.
Reviewers: spatel, arsenm, hfinkel, wristow, craig.topper
Reviewed By: spatel
Subscribers: rampitec, foad, nhaehnle, wuzish, nemanjai, jvesely, wdng, javed.absar, MaskRay, jsji
Differential Revision: https://reviews.llvm.org/D65170
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367486 91177308-0d34-0410-b5e6-96231b3b80d8
This makes the field wider than MachineOperand::SubReg_TargetFlags so that
we don't end up silently truncating any higher bits. We should still catch
any bits truncated from the MachineOperand field as a consequence of the
assertion in MachineOperand::setTargetFlags().
Differential Revision: https://reviews.llvm.org/D65465
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367474 91177308-0d34-0410-b5e6-96231b3b80d8
If anything called the recursive isKnownNeverNaN/computeKnownBits/ComputeNumSignBits/SimplifyDemandedBits/SimplifyMultipleUseDemandedBits with an incorrect depth then we could continue to recurse if we'd already exceeded the depth limit.
This replaces the limit check (Depth == 6) with a (Depth >= 6) to make sure that we don't circumvent it.
This causes a couple of regressions as a mixture of calls (SimplifyMultipleUseDemandedBits + combineX86ShufflesRecursively) were calling with depths that were already over the limit. I've fixed SimplifyMultipleUseDemandedBits to not do this. combineX86ShufflesRecursively is trickier as we get a lot of regressions if we reduce its own limit from 8 to 6 (it also starts at Depth == 1 instead of Depth == 0 like the others....) - I'll see what I can do in future patches.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367171 91177308-0d34-0410-b5e6-96231b3b80d8
Eventually all of these will be moved over, but we create nodes in GetDemandedBits recursion at the moment which causes regressions when we try to remove them all.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367092 91177308-0d34-0410-b5e6-96231b3b80d8
This patch adds support for recognizing cases where a larger vector type is being used to reduce just the elements in the lower subvector:
e.g. <8 x i32> reduction pattern in a <16 x i32> vector:
<4,5,6,7,u,u,u,u,u,u,u,u,u,u,u,u>
<2,3,u,u,u,u,u,u,u,u,u,u,u,u,u,u>
<1,u,u,u,u,u,u,u,u,u,u,u,u,u,u,u>
matchBinOpReduction returns the lower extracted subvector in such cases, assuming isExtractSubvectorCheap accepts the extraction.
I've only enabled it for X86 reduction sums so far. I intend to enable it for the bitop/minmax cases in future patches, and eventually I think its worth turning it on all the time. This is mainly just a case of ensuring calls to matchBinOpReduction don't make assumptions on the vector width based on the original vector extraction.
Fixes the x86 partial reduction sum cases in PR33758 and PR42023.
Differential Revision: https://reviews.llvm.org/D65047
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366933 91177308-0d34-0410-b5e6-96231b3b80d8
If we are already using the same chain for the old/new memory ops then just return.
Fixes PR42727 which had getLoad() reusing an existing node.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366922 91177308-0d34-0410-b5e6-96231b3b80d8
When a target intrinsic has been determined to touch memory, we construct a MachineMemOperand during SDAG construction. In this case, we should propagate AAMDNodes metadata to the MachineMemOperand where available.
Differential revision: https://reviews.llvm.org/D64131
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@365043 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
(Not so) boringly identical to pattern a (D62786)
Not yet sure how do deal with the last pattern c.
Reviewers: RKSimon, craig.topper, spatel
Reviewed By: RKSimon
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D62793
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@364418 91177308-0d34-0410-b5e6-96231b3b80d8
Use matchBinaryPredicate instead of isConstOrConstSplat to let us handle non-uniform shift cases.
This requires us to tweak matchBinaryPredicate to allow it to (optionally) handle constants with different type widths.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363792 91177308-0d34-0410-b5e6-96231b3b80d8
This is already done in DAGCombiner::visitINSERT_SUBVECTOR, but this helps a number of shuffles across different vector widths recognise when they come from the same source.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363542 91177308-0d34-0410-b5e6-96231b3b80d8
As I mentioned on D61887 we don't get many hits on ComputeNumSignBits as we did on computeKnownBits.
The case we do get is interesting though - it allows us to use the 'ConditionalNegate' combine in combineLogicBlendIntoPBLENDV to remove a select.
It comes too late for SSE41 (BLENDV) cases, but SSE2 tests can hit it now. We should probably try to make use of this for SSE41+ targets as well - avoiding variable blends is usually a good idea. I'll investigate as a followup.
Differential Revision: https://reviews.llvm.org/D62777
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@362486 91177308-0d34-0410-b5e6-96231b3b80d8
Add (opt-in) support for implicit truncation to isConstOrConstSplat, which allows us to match truncated 'all ones' cases in isBitwiseNot.
PR41020 compares against using ISD::isBuildVectorAllOnes() instead, but that predicate silently accepts any UNDEF elements in the build vector which might not be what we want in isBitwiseNot - so I've added an opt-in 'AllowUndefs' flag that is set to false by default but will allow us to enable it on individual cases where its safe.
Differential Revision: https://reviews.llvm.org/D62783
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@362323 91177308-0d34-0410-b5e6-96231b3b80d8
Just copy all of the operands except the chain and call MorphNode on that.
This removes the IsUnary and IsTernary flags.
Also always get the result type from the result type of the original
nodes. Previously we got it from the operand except for two nodes
where that didn't work.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@362269 91177308-0d34-0410-b5e6-96231b3b80d8
This is derived from the related fold for build vectors.
We also have a version of this in DAGCombiner. The benefit of
having this fold at node creation time is (1) efficiency and
(2) preventing infinite looping from creating patterns that
should not exist in the first place.
Currently, the inf-loop could happen with MergeConsecutiveStores()
because it naively creates concat of extracts when forming a wider
vector store. That could fight with target-specific store narrowing.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@361780 91177308-0d34-0410-b5e6-96231b3b80d8
There's a possible missing fold here for extracting from the
same source vector. It's similar to a check that we use to
squash a build vector with all extracted elements from the
same source vector.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@361778 91177308-0d34-0410-b5e6-96231b3b80d8
The DemandedElts variable is pretty much inert at the moment - the original GetDemandedBits implementation calls it with an 'all ones' DemandedElts value so the function is active and behaves exactly as it used to.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@361773 91177308-0d34-0410-b5e6-96231b3b80d8
Will be used in an upcoming patch but I've updated the original implementation to call this to ensure test coverage.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@361738 91177308-0d34-0410-b5e6-96231b3b80d8
This patch adds the overridable TargetLowering::getTargetConstantFromLoad function which allows targets to return any constant value loaded by a LoadSDNode node - only X86 makes use of this so far but everything should be in place for other targets.
computeKnownBits then uses this function to improve codegen, notably vector code after legalization.
A future commit will do the same for ComputeNumSignBits but computeKnownBits sees the bigger benefit.
This required a couple of fixes:
* SimplifyDemandedBits must early-out for getTargetConstantFromLoad cases to prevent infinite loops of constant regeneration (similar to what we already do for BUILD_VECTOR).
* Fix a DAGCombiner::visitTRUNCATE issue as we had trunc(shl(v8i32),v8i16) <-> shl(trunc(v8i16),v8i32) infinite loops after legalization on AVX512 targets.
Differential Revision: https://reviews.llvm.org/D61887
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@361620 91177308-0d34-0410-b5e6-96231b3b80d8
DAGCombiner simplifies this more liberally as:
// If inserting an UNDEF, just return the original vector.
if (N1.isUndef())
return N0;
So there's no way to make this visible in output AFAIK, but
doing this at node creation time should be slightly more efficient.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@361287 91177308-0d34-0410-b5e6-96231b3b80d8
Before this change, they were erroneously constructed with the EH_LABEL
SDNode opcode, which caused other passes to interact with them in
incorrect ways. See the FIXME about fastisel that this addresses in the
existing test case.
Fixes PR41890
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@360818 91177308-0d34-0410-b5e6-96231b3b80d8
The new fptrunc and fpext intrinsics are constrained versions of the
regular fptrunc and fpext instructions.
Reviewed by: Andrew Kaylor, Craig Topper, Cameron McInally, Conner Abbot
Approved by: Craig Topper
Differential Revision: https://reviews.llvm.org/D55897
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@360581 91177308-0d34-0410-b5e6-96231b3b80d8
This is extracted from the original draft of D61419 with some additional tests.
We don't currently get this in IR (it's conservatively turned into a NaN),
but presumably that'll get updated as we add real IR support for 'fneg'
rather than 'fsub -0.0, x'.
The x86-32 run shows the following, and I haven't looked further to see why,
but that seems to be independent:
Legalizing: t1: f32 = undef
Trying to expand node
Creating fp constant: t4: f32 = ConstantFP<0.000000e+00>
Differential Revision: https://reviews.llvm.org/D61516
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@360296 91177308-0d34-0410-b5e6-96231b3b80d8
