proven larger than the loop-count
This fixes PR31098: Try to resolve statically data-dependences whose
compile-time-unknown distance can be proven larger than the loop-count,
instead of resorting to runtime dependence checking (which are not always
possible).
For vectorization it is sufficient to prove that the dependence distance
is >= VF; But in some cases we can prune unknown dependence distances early,
and even before selecting the VF, and without a runtime test, by comparing
the distance against the loop iteration count. Since the vectorized code
will be executed only if LoopCount >= VF, proving distance >= LoopCount
also guarantees that distance >= VF. This check is also equivalent to the
Strong SIV Test.
Reviewers: mkuper, anemet, sanjoy
Differential Revision: https://reviews.llvm.org/D28044
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@294892 91177308-0d34-0410-b5e6-96231b3b80d8
Make SolveLinEquationWithOverflow take the start as a SCEV, so we can
solve more cases. With that implemented, get rid of the special case
for powers of two.
The additional functionality probably isn't particularly useful,
but it might help a little for certain cases involving pointer
arithmetic.
Differential Revision: https://reviews.llvm.org/D28884
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293576 91177308-0d34-0410-b5e6-96231b3b80d8
This is fixing pr31761: BasicAA is deducing NoAlias
on the result of the GEP if the base pointer is itself NoAlias.
This is possible only if the NoAlias on the base pointer is
deduced with a non-sized query: this should guarantee that
the pointers are belonging to different memory allocation
and that the GEP can't legally jump from one to another.
Differential Revision: https://reviews.llvm.org/D29216
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293293 91177308-0d34-0410-b5e6-96231b3b80d8
Inlining in getAddExpr() can cause abnormal computational time in some cases.
New parameter -scev-addops-inline-threshold is intruduced with default value 500.
Reviewers: sanjoy
Subscribers: mzolotukhin, llvm-commits
Differential Revision: https://reviews.llvm.org/D28812
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293176 91177308-0d34-0410-b5e6-96231b3b80d8
This reverts commit r292680. It is causing significantly worse
performance and test timeouts in our internal builds. I have already
routed reproduction instructions your way.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293092 91177308-0d34-0410-b5e6-96231b3b80d8
bots ever since d0k fixed the CHECK lines so that it did something at
all.
It isn't actually testing SCEV directly but LSR, so move it into LSR and
the x86-specific tree of tests that already exists there. Target
dependence is common and unavoidable with the current design of LSR.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292774 91177308-0d34-0410-b5e6-96231b3b80d8
become unavailable.
The AssumptionCache is now immutable but it still needs to respond to
DomTree invalidation if it ended up caching one.
This lets us remove one of the explicit invalidates of LVI but the
other one continues to avoid hitting a latent bug.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292769 91177308-0d34-0410-b5e6-96231b3b80d8
Newer ppc supports unaligned memory access, it reduces the cost of unaligned memory access significantly. This patch handles this case in PPCTTIImpl::getMemoryOpCost.
This patch fixes pr31492.
Differential Revision: https://reviews.llvm.org/D28630
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292680 91177308-0d34-0410-b5e6-96231b3b80d8
To avoid regressions, make ScalarEvolution::createSCEV a bit more
clever.
Also get rid of some useless code in ScalarEvolution::howFarToZero
which was hiding this bug.
No new testcase because it's impossible to actually expose this bug:
we don't have any in-tree users of getUDivExactExpr besides the two
functions I just mentioned, and they both dodged the problem. I'll
try to add some interesting users in a followup.
Differential Revision: https://reviews.llvm.org/D28587
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292449 91177308-0d34-0410-b5e6-96231b3b80d8
runnig LCSSA over them prior to running the loop pipeline.
This also teaches the loop PM to verify that LCSSA form is preserved
throughout the pipeline's run across the loop nest.
Most of the test updates just leverage this new functionality. One has to be
relaxed with the new PM as IVUsers is less powerful when it sees LCSSA input.
Differential Revision: https://reviews.llvm.org/D28743
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292241 91177308-0d34-0410-b5e6-96231b3b80d8
First, I've moved a test of IVUsers from the LSR tree to a dedicated
IVUsers test directory. I've also simplified its RUN line now that the
new pass manager's loop PM is providing analyses on their own.
No functionality changed, but it makes subsequent changes cleaner.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292060 91177308-0d34-0410-b5e6-96231b3b80d8
mark it as never invalidated in the new PM.
The old PM already required this to work, and after a discussion with
Hal this seems to really be the only sensible answer. The cache
gracefully degrades as the IR is mutated, and most things which do this
should already be incrementally updating the cache.
This gets rid of a bunch of logic preserving and testing the
invalidation of this analysis.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292039 91177308-0d34-0410-b5e6-96231b3b80d8
updated instructions:
pmulld, pmullw, pmulhw, mulsd, mulps, mulpd, divss, divps, divsd, divpd, addpd and subpd.
special optimization case which replaces pmulld with pmullw\pmulhw\pshuf seq.
In case if the real operands bitwidth <= 16.
Differential Revision: https://reviews.llvm.org/D28104
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291657 91177308-0d34-0410-b5e6-96231b3b80d8
The original code considered only v2i64 as slow for this feature. This patch
consider all 128-bit long vector types as slow candidates.
In internal tests, extending this feature to all 128-bit vector types
resulted in an overall improvement of 1% on Exynos M1.
Differential revision: https://reviews.llvm.org/D27998
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291616 91177308-0d34-0410-b5e6-96231b3b80d8
invalid.
This fixes use-after-free bugs that will arise with any interesting use
of SCEV.
I've added a dedicated test that works diligently to trigger these kinds
of bugs in the new pass manager and also checks for them explicitly as
well as triggering ASan failures when things go squirly.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291426 91177308-0d34-0410-b5e6-96231b3b80d8
The 'fast' costs should only work for shifts by uniform constants (uniform non-constant are lowered using the slow default implementation).
Logical shifts were not taking into account that we must mask the psrlw result, so the costs needed to be doubled.
Added missing AVX2/AVX512BW costs as well.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291391 91177308-0d34-0410-b5e6-96231b3b80d8
SSE41 provides pmulld which allows the simpler pslld/paddd/cvttps2dq/pmulld pattern than SSE2's use of pmuludq.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291372 91177308-0d34-0410-b5e6-96231b3b80d8
Without this CHECK line, we may not detect incorrectly detected additional
regions at the end of the region tree.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290994 91177308-0d34-0410-b5e6-96231b3b80d8
This test case has been reduced from test/Analysis/RegionInfo/mix_1.ll and
provides us with a minimal example of a test case which caused problems while
working on an improved version of the RegionInfo analysis. We upstream this
test case, as it certainly can be helpful in future debugging and optimization
tests.
Test case reduced by Pratik Bhatu <cs12b1010@iith.ac.in>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290974 91177308-0d34-0410-b5e6-96231b3b80d8
Sanjay Patel