Commit Graph

1207 Commits

Author SHA1 Message Date
Chad Rosier
db638de2de [AliasSetTracker] Add support for memcpy and memmove.
Differential Revision: https://reviews.llvm.org/D25776

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284630 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-19 19:09:03 +00:00
John Brawn
a6faf3af51 [SCEV] More accurate calculation of max backedge count of some less-than loops
In loops that look something like
 i = n;
 do {
  ...
 } while(i++ < n+k);
where k is a constant, the maximum backedge count is k (in fact the backedge
count will be either 0 or k, depending on whether n+k wraps). More generally
for LHS < RHS if RHS-(LHS of first comparison) is a constant then the loop will
iterate either 0 or that constant number of times.

This allows for more loop unrolling with the recent upper bound loop unrolling
changes, and I'm working on a patch that will let loop unrolling additionally
make use of the loop being executed either 0 or k times (we need to retain the
loop comparison only on the first unrolled iteration).

Differential Revision: https://reviews.llvm.org/D25607


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284465 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-18 10:10:53 +00:00
Simon Pilgrim
e615ec6e15 [X86][SSE] Add lowering to cvttpd2dq/cvttps2dq for sitofp v2f64/2f32 to 2i32
As discussed on PR28461 we currently miss the chance to lower "fptosi <2 x double> %arg to <2 x i32>" to cvttpd2dq due to its use of illegal types.

This patch adds support for fptosi to 2i32 from both 2f64 and 2f32.

It also recognises that cvttpd2dq zeroes the upper 64-bits of the xmm result (similar to D23797) - we still don't do this for the cvttpd2dq/cvttps2dq intrinsics - this can be done in a future patch.

Differential Revision: https://reviews.llvm.org/D23808

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284459 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-18 07:42:15 +00:00
Tobias Grosser
a00e39fd38 [SCEV] Consider delinearization pattern with extension with identity factor
Summary: The delinearization algorithm did not consider terms which had an extension without a multiply factor, i.e. a identify factor. We lose cases where size is char type where there will no multiply factor.

Reviewers: sanjoy, grosser

Subscribers: mzolotukhin, Eugene.Zelenko, llvm-commits, mssimpso, sanjoy, grosser

Differential Revision: https://reviews.llvm.org/D16492

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284378 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-17 11:56:26 +00:00
Tom Stellard
8b2bb5de9a [ValueTracking] Fix crash in GetPointerBaseWithConstantOffset()
Summary:
While walking defs of pointer operands we were assuming that the pointer
size would remain constant.  This is not true, because addresspacecast
instructions may cast the pointer to an address space with a different
pointer width.

This partial reverts r282612, which was a more conservative solution
to this problem.

Reviewers: reames, sanjoy, apilipenko

Subscribers: wdng, llvm-commits

Differential Revision: https://reviews.llvm.org/D24772

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283557 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-07 14:23:29 +00:00
Bjorn Pettersson
c4e191a67a [ValueTracking] Teach computeKnownBits and ComputeNumSignBits to look through ExtractElement.
Summary:
The computeKnownBits and ComputeNumSignBits functions in ValueTracking can now do a simple look-through of ExtractElement.

Reviewers: majnemer, spatel

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D24955

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283434 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-06 09:56:21 +00:00
Eli Friedman
aba0070ed8 Make GlobalsAA ignore dead constant expressions.
Slightly improves the precision of GlobalsAA in certain situations, and
makes the behavior of optimization passes more predictable.

Differential Revision: https://reviews.llvm.org/D24104



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283165 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-04 00:03:55 +00:00
Sanjay Patel
7b05e5c94e [x86, SSE/AVX] allow 128/256-bit lowering for copysign vector intrinsics (PR30433)
This should fix:
https://llvm.org/bugs/show_bug.cgi?id=30433

There are a couple of open questions about the codegen:
1. Should we let scalar ops be scalars and avoid vector constant loads/splats?
2. Should we have a pass to combine constants such as the inverted pair that we have here?

Differential Revision: https://reviews.llvm.org/D25165
 


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283119 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-03 16:38:27 +00:00
Simon Pilgrim
065e5924cd [CostModel][X86] Added tests for current fptosi/fptoui costs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283047 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-01 19:09:59 +00:00
Simon Pilgrim
1df457ad5e [CostModel][X86] Added fcopysign costs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283044 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-01 16:41:52 +00:00
Simon Pilgrim
089e03e79a [CostModel][X86] Added fabs costs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283042 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-01 16:30:13 +00:00
Jun Bum Lim
32ed609373 Enhance calcColdCallHeuristics for InvokeInst
Summary: When identifying cold blocks, consider only the edge to the normal destination if the terminator is InvokeInst and let calcInvokeHeuristics() decide edge weights for the InvokeInst.

Reviewers: mcrosier, hfinkel, davidxl

Subscribers: mcrosier, llvm-commits

Differential Revision: https://reviews.llvm.org/D24868

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282262 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-23 17:26:14 +00:00
Simon Pilgrim
08a3649b8b [CostModel][X86] Added scalar float op costs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281864 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-18 21:01:20 +00:00
Sanjay Patel
58071c976f auto-generate checks
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281756 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-16 17:54:52 +00:00
David L Kreitzer
cd01b58518 Reapplying r278731 after fixing the problem that caused it to be reverted.
Enhance SCEV to compute the trip count for some loops with unknown stride.

Patch by Pankaj Chawla

Differential Revision: https://reviews.llvm.org/D22377


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281732 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-16 14:38:13 +00:00
Wei Mi
81a2b8958e Create a getelementptr instead of sub expr for ValueOffsetPair if the
value is a pointer.

This patch is to fix PR30213. When expanding an expr based on ValueOffsetPair,
if the value is of pointer type, we can only create a getelementptr instead
of sub expr.

Differential Revision: https://reviews.llvm.org/D24088


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281439 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-14 04:39:50 +00:00
Elena Demikhovsky
1b2a8500af [Loop Vectorizer] Fixed memory confilict checks.
Fixed a bug in run-time checks for possible memory conflicts inside loop.
The bug is in Low <-> High boundaries calculation. The High boundary should be calculated as "last memory access pointer + element size".

Differential revision: https://reviews.llvm.org/D23176



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279930 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-28 08:53:53 +00:00
Wei Mi
8ce317bf15 [UNROLL] Postpone ScalarEvolution::forgetLoop after TripCountSC is expanded
when unroll runtime iteration loop.

In llvm::UnrollRuntimeLoopRemainder, if the loop to be unrolled is the inner
loop inside a loop nest, the scalar evolution needs to be dropped for its
parent loop which is done by ScalarEvolution::forgetLoop. However, we can
postpone forgetLoop to the end of UnrollRuntimeLoopRemainder so TripCountSC
expansion can still reuse existing value.

Differential Revision: https://reviews.llvm.org/D23572


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279748 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-25 16:17:18 +00:00
Evgeny Stupachenko
2af858265d The patch improves ValueTracking on left shift with nsw flag.
Summary:
The patch fixes PR28946.

Reviewers: majnemer, sanjoy

Differential Revision: http://reviews.llvm.org/D23296

From: Li Huang


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279684 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-24 23:01:33 +00:00
Artur Pilipenko
cd61ceee32 Remove missing file from r279433 reversal
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279434 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-22 13:18:19 +00:00
Simon Pilgrim
fc26436d17 [CostModel][X86] Removed shift tests
There are more thorough tests found in vshift-*-cost.ll 

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279406 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-21 19:56:02 +00:00
Simon Pilgrim
26eaf4f816 [CostModel][X86] Added costs for vXi16 and vXi8 vectors for add/sub/mul/and/or/xor tests
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279405 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-21 19:44:44 +00:00
Simon Pilgrim
799c5ebb1b [CostModel][X86] Replaced SSSE3 with SSE2 costs to create a better baseline
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279404 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-21 19:14:48 +00:00
Simon Pilgrim
4630834e7c [CostModel][X86] Added fsqrt and fma costs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279403 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-21 19:06:25 +00:00
Simon Pilgrim
4597dd96a3 [CostModel][X86] Split off float arithmetic cost tests
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279402 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-21 18:34:47 +00:00
Simon Pilgrim
18333ab15e [CostModel][X86] Added sub, or, and, fadd and fsub costs and missing 512-bit mul costs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279301 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-19 19:07:10 +00:00
Simon Pilgrim
07010a7197 [CostModel][X86] Added some AVX512 and 512-bit vector cost tests
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279291 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-19 18:24:10 +00:00
Simon Pilgrim
50fbf501e3 [CostModel][X86] Add fdiv + frem cost tests
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279283 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-19 17:39:00 +00:00
Michael Kuperstein
15095e4af8 [AliasSetTracker] Degrade AliasSetTracker when may-alias sets get too large.
Repeated inserts into AliasSetTracker have quadratic behavior - inserting a
pointer into AST is linear, since it requires walking over all "may" alias
sets and running an alias check vs. every pointer in the set.

We can avoid this by tracking the total number of pointers in "may" sets,
and when that number exceeds a threshold, declare the tracker "saturated".
This lumps all pointers into a single "may" set that aliases every other
pointer.

(This is a stop-gap solution until we migrate to MemorySSA)

This fixes PR28832.
Differential Revision: https://reviews.llvm.org/D23432


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279274 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-19 17:05:22 +00:00
Hans Wennborg
3a15af3d32 SCEV: Don't assert about non-SCEV-able value in isSCEVExprNeverPoison() (PR28932)
Differential Revision: https://reviews.llvm.org/D23594

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278999 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-17 22:50:18 +00:00
Reid Kleckner
8f5027f8f0 Revert "Enhance SCEV to compute the trip count for some loops with unknown stride."
This reverts commit r278731. It caused http://crbug.com/638314

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278853 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-16 21:02:04 +00:00
Sanjoy Das
b4ca813e88 Revert "[ValueTracking] Improve ValueTracking on left shift with nsw flag"
This reverts commit r278172.  It causes PR28946.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278740 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-15 21:01:31 +00:00
David L Kreitzer
1e230bc8b4 Enhance SCEV to compute the trip count for some loops with unknown stride.
Patch by Pankaj Chawla

Differential Revision: https://reviews.llvm.org/D22377


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278731 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-15 20:21:41 +00:00
Eli Friedman
35ff6f0857 [DSE] Don't remove stores made live by a call which unwinds.
Issue exposed by noalias or more aggressive alias analysis.

Fixes http://llvm.org/PR25422.

Differential revision: https://reviews.llvm.org/D21007



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278451 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-12 01:09:53 +00:00
Andrew Kaylor
72626e148e [ValueTracking] An improvement to IR ValueTracking on Non-negative Integers
Patch by Li Huang

Differential Revision: https://reviews.llvm.org/D18777



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278267 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-10 18:47:19 +00:00
Andrew Kaylor
6c6978f931 [ValueTracking] Improve ValueTracking on left shift with nsw flag
Patch by Li Huang

Differential Revison: https://reviews.llvm.org/D23296



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278172 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-09 22:41:35 +00:00
Wei Mi
fc25cfb37a Fix the runtime error caused by "Use ValueOffsetPair to enhance value reuse during SCEV expansion".
The patch is to fix the bug in PR28705. It was caused by setting wrong return
value for SCEVExpander::findExistingExpansion. The return values of findExistingExpansion
have different meanings when the function is used in different ways so it is easy to make
mistake. The fix creates two new interfaces to replace SCEVExpander::findExistingExpansion,
and specifies where each interface is expected to be used.

Differential Revision: https://reviews.llvm.org/D22942


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278161 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-09 20:40:03 +00:00
Wei Mi
8b93225c07 Recommit "Use ValueOffsetPair to enhance value reuse during SCEV expansion".
The fix for PR28705 will be committed consecutively.

In D12090, the ExprValueMap was added to reuse existing value during SCEV expansion.
However, const folding and sext/zext distribution can make the reuse still difficult.

A simplified case is: suppose we know S1 expands to V1 in ExprValueMap, and
  S1 = S2 + C_a
  S3 = S2 + C_b
where C_a and C_b are different SCEVConstants. Then we'd like to expand S3 as
V1 - C_a + C_b instead of expanding S2 literally. It is helpful when S2 is a
complex SCEV expr and S2 has no entry in ExprValueMap, which is usually caused
by the fact that S3 is generated from S1 after const folding.

In order to do that, we represent ExprValueMap as a mapping from SCEV to
ValueOffsetPair. We will save both S1->{V1, 0} and S2->{V1, C_a} into the
ExprValueMap when we create SCEV for V1. When S3 is expanded, it will first
expand S2 to V1 - C_a because of S2->{V1, C_a} in the map, then expand S3 to
V1 - C_a + C_b.

Differential Revision: https://reviews.llvm.org/D21313


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278160 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-09 20:37:50 +00:00
Sanjoy Das
549a67c20d [SCEV] Un-grep'ify tests; NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277861 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-05 20:33:49 +00:00
Sanjoy Das
a2ef5492ed [SCEV] Don't infinitely recurse on unreachable code
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277848 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-05 18:34:14 +00:00
Michael Kuperstein
e788186982 [LV, X86] Be more optimistic about vectorizing shifts.
Shifts with a uniform but non-constant count were considered very expensive to
vectorize, because the splat of the uniform count and the shift would tend to
appear in different blocks. That made the splat invisible to ISel, and we'd
scalarize the shift at codegen time.

Since r201655, CodeGenPrepare sinks those splats to be next to their use, and we
are able to select the appropriate vector shifts. This updates the cost model to
to take this into account by making shifts by a uniform cheap again.

Differential Revision: https://reviews.llvm.org/D23049


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277782 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-04 22:48:03 +00:00
Simon Pilgrim
2bdfdf95e6 [X86] Dropped XOP ctbits checks - they match the AVX checks
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277718 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-04 11:04:13 +00:00
Simon Pilgrim
d81c2d5aa5 [X86][SSE] Add initial costs for vector CTTZ/CTLZ
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277716 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-04 10:51:41 +00:00
George Burgess IV
4df351da7d [CFLAA] Remove modref queries from CFLAA.
As it turns out, modref queries are broken with CFLAA. Specifically,
the data source we were using for determining modref behaviors
explicitly ignores operations on non-pointer values. So, it wouldn't
note e.g. storing an i32 to an i32* (or loading an i64 from an i64*).
It also ignores external function calls, rather than acting
conservatively for them.

(N.B. These operations, where necessary, *are* tracked by CFLAA; we just
use a different mechanism to do so. Said mechanism is relatively
imprecise, so it's unlikely that we can provide reasonably good modref
answers with it as implemented.)

Patch by Jia Chen.

Differential Revision: https://reviews.llvm.org/D22978


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277366 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-01 18:47:28 +00:00
Adam Nemet
b17a45cae0 [BPI] Add new LazyBPI analysis
Summary:
The motivation is the same as in D22141: In order to add the hotness
attribute to optimization remarks we need BFI to be available in all
passes that emit optimization remarks.  BFI depends on BPI so unless we
make this lazy as well we would still compute BPI unconditionally.

The solution is to use the new LazyBPI pass in LazyBFI and only compute
BPI when computation of BFI is requested by the client.

I extended the laziness test using a LoopDistribute test to also cover
BPI.

Reviewers: hfinkel, davidxl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D22835

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277083 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-28 23:31:12 +00:00
George Burgess IV
aa6ca74bfc [CFLAA] Add getModRefBehavior to CFLAnders.
This patch lets CFLAnders respond to mod-ref queries. It also includes
a small bugfix to CFLSteens.

Patch by Jia Chen.

Differential Revision: https://reviews.llvm.org/D22823


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276939 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-27 23:07:07 +00:00
Hans Wennborg
df483ab5a3 Revert r276136 "Use ValueOffsetPair to enhance value reuse during SCEV expansion."
It causes Clang tests to fail after Windows self-host (PR28705).

(Also reverts follow-up r276139.)

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276822 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-26 23:25:13 +00:00
Wei Mi
feca1dba45 Remove useless pass from the pipeline in test/Analysis/Dominators/2007-01-14-BreakCritEdges.ll.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276644 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-25 16:27:34 +00:00
Sanjoy Das
fa29211988 [SCEV] Make isImpliedCondOperandsViaRanges smarter
This change lets us prove things like

  "{X,+,10} s< 5000" implies "{X+7,+,10} does not sign overflow"

It does this by replacing replacing getConstantDifference by
computeConstantDifference (which is smarter) in
isImpliedCondOperandsViaRanges.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276505 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-23 00:54:36 +00:00
Wei Mi
a6c7a032fe [PM] Port BreakCriticalEdges to the new PM.
Differential Revision: https://reviews.llvm.org/D22688


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276449 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-22 18:04:25 +00:00