Commit Graph

191 Commits

Author SHA1 Message Date
Junmo Park
4ee5ab9daf [SCEVExpander] Make findExistingExpansion smarter
Summary:
Extending findExistingExpansion can use existing value in ExprValueMap.
This patch gives 0.3~0.5% performance improvements on 
benchmarks(test-suite, spec2000, spec2006, commercial benchmark)
   
Reviewers: mzolotukhin, sanjoy, zzheng

Differential Revision: http://reviews.llvm.org/D15559


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260938 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-16 06:46:58 +00:00
Matt Arsenault
fae18e933b AMDGPU: Remove some old intrinsic uses from tests
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260493 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-11 06:02:01 +00:00
Justin Bogner
2ab07ac8fe LoopUnroll: Use the optsize threshold for minsize as well
Currently we're unrolling loops more in minsize than in optsize, which
means -Oz will have a larger code size than -Os. That doesn't make any
sense.

This resolves the FIXME about this in LoopUnrollPass and extends the
optsize test to make sure we use the smaller threshold for minsize as
well.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257402 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-11 22:39:43 +00:00
Justin Bogner
fbbc16fa87 LoopInfo: Simplify ownership of Loop objects
It's strange that LoopInfo mostly owns the Loop objects, but that it
defers deleting them to the loop pass manager. Instead, change the
oddly named "updateUnloop" to "markAsRemoved" and have it queue the
Loop object for deletion. We can't delete the Loop immediately when we
remove it, since we need its pointer identity still, so we'll mark the
object as "invalid" so that clients can see what's going on.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257191 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-08 19:08:53 +00:00
Matt Arsenault
7aed0ccd46 AMDGPU: Switch barrier intrinsics to using convergent
noduplicate prevents unrolling of small loops that happen to have
barriers in them. If a loop has a barrier in it, it is OK to duplicate
it for the unroll.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@256075 91177308-0d34-0410-b5e6-96231b3b80d8
2015-12-19 01:46:41 +00:00
Michael Zolotukhin
09dcb16d13 Don't recompute LCSSA after loop-unrolling when possible.
Summary:
Currently we always recompute LCSSA for outer loops after unrolling an
inner loop. That leads to compile time problem when we have big loop
nests, and we can solve it by avoiding unnecessary work. For instance,
if w eonly do partial unrolling, we don't break LCSSA, so we don't need
to rebuild it. Also, if all exits from the inner loop are inside the
enclosing loop, then complete unrolling won't break LCSSA either.

I replaced unconditional LCSSA recomputation with conditional recomputation +
unconditional assert and added several tests, which were failing when I
experimented with it.

Soon I plan to follow up with a similar patch for recalculation of dominators
tree.

Reviewers: hfinkel, dexonsmith, bogner, joker.eph, chandlerc

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D14526

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@253126 91177308-0d34-0410-b5e6-96231b3b80d8
2015-11-14 05:51:41 +00:00
Peter Collingbourne
5f220beefc DI: Reverse direction of subprogram -> function edge.
Previously, subprograms contained a metadata reference to the function they
described. Because most clients need to get or set a subprogram for a given
function rather than the other way around, this created unneeded inefficiency.

For example, many passes needed to call the function llvm::makeSubprogramMap()
to build a mapping from functions to subprograms, and the IR linker needed to
fix up function references in a way that caused quadratic complexity in the IR
linking phase of LTO.

This change reverses the direction of the edge by storing the subprogram as
function-level metadata and removing DISubprogram's function field.

Since this is an IR change, a bitcode upgrade has been provided.

Fixes PR23367. An upgrade script for textual IR for out-of-tree clients is
attached to the PR.

Differential Revision: http://reviews.llvm.org/D14265

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@252219 91177308-0d34-0410-b5e6-96231b3b80d8
2015-11-05 22:03:56 +00:00
Tobias Grosser
0efe7b620e Revert "[IndVarSimplify] Rewrite loop exit values with their initial values from loop preheader"
Commit 251839 triggers miscompiles on some bots:

http://lab.llvm.org:8011/builders/perf-x86_64-penryn-O3-polly-fast/builds/13723

(The commit is listed in 13722, but due to an existing failure introduced in
13721 and reverted in 13723 the failure is only visible in 13723)

To verify r251839 is indeed the only change that triggered the buildbot failures
and to ensure the buildbots remain green while investigating I temporarily
revert this commit. At the current state it is unclear if this commit introduced
some miscompile or if it only exposed code to Polly that is subsequently
miscompiled by Polly.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@251901 91177308-0d34-0410-b5e6-96231b3b80d8
2015-11-03 07:14:39 +00:00
Chen Li
fb81c7b828 [IndVarSimplify] Rewrite loop exit values with their initial values from loop preheader
Summary:
This patch adds support to check if a loop has loop invariant conditions which lead to loop exits. If so, we know that if the exit path is taken, it is at the first loop iteration. If there is an induction variable used in that exit path whose value has not been updated, it will keep its initial value passing from loop preheader. We can therefore rewrite the exit value with
its initial value. This will help remove phis created by LCSSA and enable other optimizations like loop unswitch.


Reviewers: sanjoy

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D13974

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@251839 91177308-0d34-0410-b5e6-96231b3b80d8
2015-11-02 22:00:15 +00:00
Chen Li
b63862ff47 Revert r251492 "[IndVarSimplify] Rewrite loop exit values with their
initial values from loop preheader", because it broke some bots.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@251498 91177308-0d34-0410-b5e6-96231b3b80d8
2015-10-28 05:15:51 +00:00
Chen Li
0c46270ed3 [IndVarSimplify] Rewrite loop exit values with their initial values from loop preheader
Summary:
This patch adds support to check if a loop has loop invariant conditions which lead to loop exits. If so, we know that if the exit path is taken, it is at the first loop iteration. If there is an induction variable used in that exit path whose value has not been updated, it will keep its initial value passing from loop preheader. We can therefore rewrite the exit value with
its initial value. This will help remove phis created by LCSSA and enable other optimizations like loop unswitch.


Reviewers: sanjoy

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D13974

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@251492 91177308-0d34-0410-b5e6-96231b3b80d8
2015-10-28 04:45:47 +00:00
Michael Zolotukhin
c04323ec72 [Tests] Add one more case to LoopUnroll/pr18861.ll for better coverage.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249174 91177308-0d34-0410-b5e6-96231b3b80d8
2015-10-02 19:21:52 +00:00
Michael Zolotukhin
e18914ee57 [Tests] Give meaningful names to blocks in LoopUnroll/pr18861.ll, add a description of what's going on.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249173 91177308-0d34-0410-b5e6-96231b3b80d8
2015-10-02 19:21:49 +00:00
Michael Zolotukhin
c1aaab5c08 [Tests] Slightly reduce test LoopUnroll/pr18861.ll.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249172 91177308-0d34-0410-b5e6-96231b3b80d8
2015-10-02 19:21:43 +00:00
Michael Zolotukhin
3286c1f693 [Unroll] Do not crash trying to propagate a value to vector load.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248333 91177308-0d34-0410-b5e6-96231b3b80d8
2015-09-22 22:27:12 +00:00
Michael Zolotukhin
831687d157 [Unroll] Follow-up for r247769: fix a bug in UnrolledInstAnalyzer::visitLoad.
Apart from checking that GlobalVariable is a constant, we should check
that it's not a weak constant, in which case we can't propagate its
value.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248327 91177308-0d34-0410-b5e6-96231b3b80d8
2015-09-22 21:41:29 +00:00
Michael Zolotukhin
41591e7e30 [Unroll] Fix a bug in UnrolledInstAnalyzer::visitLoad.
We only checked that a global is initialized with constants, which is
incorrect. We should be checking that GlobalVariable *is* a constant,
not just initialized with it.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@247769 91177308-0d34-0410-b5e6-96231b3b80d8
2015-09-16 03:25:09 +00:00
Duncan P. N. Exon Smith
a5ae7c1c9f DI: Require subprogram definitions to be distinct
As a follow-up to r246098, require `DISubprogram` definitions
(`isDefinition: true`) to be 'distinct'.  Specifically, add an assembler
check, a verifier check, and bitcode upgrading logic to combat testcase
bitrot after the `DIBuilder` change.

While working on the testcases, I realized that
test/Linker/subprogram-linkonce-weak-odr.ll isn't relevant anymore.  Its
purpose was to check for a corner case in PR22792 where two subprogram
definitions match exactly and share the same metadata node.  The new
verifier check, requiring that subprogram definitions are 'distinct',
precludes that possibility.

I updated almost all the IR with the following script:

    git grep -l -E -e '= !DISubprogram\(.* isDefinition: true' |
    grep -v test/Bitcode |
    xargs sed -i '' -e 's/= \(!DISubprogram(.*, isDefinition: true\)/= distinct \1/'

Likely some variant of would work for out-of-tree testcases.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246327 91177308-0d34-0410-b5e6-96231b3b80d8
2015-08-28 20:26:49 +00:00
Mark Heffernan
214aad9f31 Add new llvm.loop.unroll.enable metadata.
This change adds the unroll metadata "llvm.loop.unroll.enable" which directs
the optimizer to unroll a loop fully if the trip count is known at compile time, and
unroll partially if the trip count is not known at compile time. This differs from
"llvm.loop.unroll.full" which explicitly does not unroll a loop if the trip count is not
known at compile time.

The "llvm.loop.unroll.enable" is intended to be added for loops annotated with
"#pragma unroll".



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@244466 91177308-0d34-0410-b5e6-96231b3b80d8
2015-08-10 17:28:08 +00:00
Chandler Carruth
6e3744f374 [Unroll] Improve the brute force loop unroll estimate by propagating
through PHI nodes across iterations.

This patch teaches the new advanced loop unrolling heuristics to propagate
constants into the loop from the preheader and around the backedge after
simulating each iteration. This lets us brute force solve simple recurrances
that aren't modeled effectively by SCEV. It also makes it more clear why we
need to process the loop in-order rather than bottom-up which might otherwise
make much more sense (for example, for DCE).

This came out of an attempt I'm making to develop a principled way to account
for dead code in the unroll estimation. When I implemented
a forward-propagating version of that it produced incorrect results due to
failing to propagate *cost* between loop iterations through the PHI nodes, and
it occured to me we really should at least propagate simplifications across
those edges, and it is quite easy thanks to the loop being in canonical and
LCSSA form.

Differential Revision: http://reviews.llvm.org/D11706

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@243900 91177308-0d34-0410-b5e6-96231b3b80d8
2015-08-03 20:32:27 +00:00
Michael Zolotukhin
607fe5bb28 [Unroll] Handle SwitchInst properly.
Previously successor selection was simply wrong.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@243545 91177308-0d34-0410-b5e6-96231b3b80d8
2015-07-29 18:10:33 +00:00
Michael Zolotukhin
815580fe7a [Unroll] Don't crash when simplified branch condition is undef.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@243544 91177308-0d34-0410-b5e6-96231b3b80d8
2015-07-29 18:10:29 +00:00
Michael Zolotukhin
952e40ff00 Rename test full-unroll-bad-geps.ll to full-unroll-crashers.ll.
No reason to limit it only to GEP-related crashes. More tests are to
come here.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@243543 91177308-0d34-0410-b5e6-96231b3b80d8
2015-07-29 18:10:23 +00:00
Jingyue Wu
580991b5c9 Roll forward r243250
r243250 appeared to break clang/test/Analysis/dead-store.c on one of the build
slaves, but I couldn't reproduce this failure locally. Probably a false
positive as I saw this test was broken by r243246 or r243247 too but passed
later without people fixing anything.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@243253 91177308-0d34-0410-b5e6-96231b3b80d8
2015-07-26 19:10:03 +00:00
Jingyue Wu
e48b1257f1 Revert r243250
breaks tests


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@243251 91177308-0d34-0410-b5e6-96231b3b80d8
2015-07-26 18:30:13 +00:00
Jingyue Wu
9f141640b5 [TTI/CostModel] improve TTI::getGEPCost and use it in CostModel::getInstructionCost
Summary:
This patch updates TargetTransformInfoImplCRTPBase::getGEPCost to consider
addressing modes. It now returns TCC_Free when the GEP can be completely folded
to an addresing mode.

I started this patch as I refactored SLSR. Function isGEPFoldable looks common
and is indeed used by some WIP of mine. So I extracted that logic to getGEPCost.

Furthermore, I noticed getGEPCost wasn't directly tested anywhere. The best
testing bed seems CostModel, but its getInstructionCost method invokes
getAddressComputationCost for GEPs which provides very coarse estimation. So
this patch also makes getInstructionCost call the updated getGEPCost for GEPs.
This change inevitably breaks some tests because the cost model changes, but
nothing looks seriously wrong -- if we believe the new cost model is the right
way to go, these tests should be updated.

This patch is not perfect yet -- the comments in some tests need to be updated.
I want to know whether this is a right approach before fixing those details.

Reviewers: chandlerc, hfinkel

Subscribers: aschwaighofer, llvm-commits, aemerson

Differential Revision: http://reviews.llvm.org/D9819

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@243250 91177308-0d34-0410-b5e6-96231b3b80d8
2015-07-26 17:28:13 +00:00
Michael Zolotukhin
fc561accb7 Handle resolvable branches in complete loop unroll heuristic.
Summary:
Resolving a branch allows us to ignore blocks that won't be executed, and thus make our estimate more accurate.
This patch is intended to be applied after D10205 (though it could be applied independently).

Reviewers: chandlerc

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D10206

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@243084 91177308-0d34-0410-b5e6-96231b3b80d8
2015-07-24 01:53:04 +00:00
Michael Zolotukhin
72d14a0792 Tidy-up test case from r242257.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@242268 91177308-0d34-0410-b5e6-96231b3b80d8
2015-07-15 01:51:51 +00:00
Michael Zolotukhin
67ee52cf04 [LoopUnrolling] Handle cast instructions.
During estimation of unrolling effect we should be able to propagate
constants through casts.

Differential Revision: http://reviews.llvm.org/D10207

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@242257 91177308-0d34-0410-b5e6-96231b3b80d8
2015-07-15 00:19:51 +00:00
Mark Heffernan
8a9e01d606 Enable runtime unrolling with unroll pragma metadata
Enable runtime unrolling for loops with unroll count metadata ("#pragma unroll N")
and a runtime trip count. Also, do not unroll loops with unroll full metadata if the
loop has a runtime loop count. Previously, such loops would be unrolled with a
very large threshold (pragma-unroll-threshold) if runtime unrolled happened to be
enabled resulting in a very large (and likely unwise) unroll factor.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@242047 91177308-0d34-0410-b5e6-96231b3b80d8
2015-07-13 18:26:27 +00:00
David Majnemer
787b9b4e95 [LoopUnroll] Use undef for phis with no value live
We would create a phi node with a zero initialized operand instead of
undef in the case where no value was originally available.  This was
problematic for x86_mmx which has no null value.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@241143 91177308-0d34-0410-b5e6-96231b3b80d8
2015-07-01 05:38:07 +00:00
Alexey Samsonov
bc482286fc Set proper debug location for branch added in BasicBlock::splitBasicBlock().
This improves debug locations in passes that do a lot of basic block
transformations. Important case is LoopUnroll pass, the test for correct
debug locations accompanies this change.

Test Plan: regression test suite

Reviewers: dblaikie, sanjoy

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D10367

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@239551 91177308-0d34-0410-b5e6-96231b3b80d8
2015-06-11 18:25:54 +00:00
Sanjoy Das
68081f41fa [LoopUnroll] Fix truncation bug in canUnrollCompletely.
Summary:
canUnrollCompletely takes `unsigned` values for `UnrolledCost` and
`RolledDynamicCost` but is passed in `uint64_t`s that are silently
truncated.  Because of this, when `UnrolledSize` is a large integer
that has a small remainder with UINT32_MAX, LLVM tries to completely
unroll loops with high trip counts.

Reviewers: mzolotukhin, chandlerc

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D10293

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@239218 91177308-0d34-0410-b5e6-96231b3b80d8
2015-06-06 05:24:10 +00:00
Chandler Carruth
862b2ad204 [Unroll] Rework the naming and structure of the new unroll heuristics.
The new naming is (to me) much easier to understand. Here is a summary
of the new state of the world:

- '*Threshold' is the threshold for full unrolling. It is measured
  against the estimated unrolled cost as computed by getUserCost in TTI
  (or CodeMetrics, etc). We will exceed this threshold when unrolling
  loops where unrolling exposes a significant degree of simplification
  of the logic within the loop.
- '*PercentDynamicCostSavedThreshold' is the percentage of the loop's
  estimated dynamic execution cost which needs to be saved by unrolling
  to apply a discount to the estimated unrolled cost.
- '*DynamicCostSavingsDiscount' is the discount applied to the estimated
  unrolling cost when the dynamic savings are expected to be high.

When actually analyzing the loop, we now produce both an estimated
unrolled cost, and an estimated rolled cost. The rolled cost is notably
a dynamic estimate based on our analysis of the expected execution of
each iteration.

While we're still working to build up the infrastructure for making
these estimates, to me it is much more clear *how* to make them better
when they have reasonably descriptive names. For example, we may want to
apply estimated (from heuristics or profiles) dynamic execution weights
to the *dynamic* cost estimates. If we start doing that, we would also
need to track the static unrolled cost and the dynamic unrolled cost, as
only the latter could reasonably be weighted by profile information.

This patch is sadly not without functionality change for the new unroll
analysis logic. Buried in the heuristic management were several things
that surprised me. For example, we never subtracted the optimized
instruction count off when comparing against the unroll heursistics!
I don't know if this just got lost somewhere along the way or what, but
with the new accounting of things, this is much easier to keep track of
and we use the post-simplification cost estimate to compare to the
thresholds, and use the dynamic cost reduction ratio to select whether
we can exceed the baseline threshold.

The old values of these flags also don't necessarily make sense. My
impression is that none of these thresholds or discounts have been tuned
yet, and so they're just arbitrary placehold numbers. As such, I've not
bothered to adjust for the fact that this is now a discount and not
a tow-tier threshold model. We need to tune all these values once the
logic is ready to be enabled.

Differential Revision: http://reviews.llvm.org/D9966

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@239164 91177308-0d34-0410-b5e6-96231b3b80d8
2015-06-05 17:01:43 +00:00
Hal Finkel
5368a26e16 [PPC/LoopUnrollRuntime] Don't avoid high-cost trip count computation on the PPC/A2
On X86 (and similar OOO cores) unrolling is very limited, and even if the
runtime unrolling is otherwise profitable, the expense of a division to compute
the trip count could greatly outweigh the benefits. On the A2, we unroll a lot,
and the benefits of unrolling are more significant (seeing a 5x or 6x speedup
is not uncommon), so we're more able to tolerate the expense, on average, of a
division to compute the trip count.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@237947 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-21 20:30:23 +00:00
Wei Mi
95f7517f1b Add another InstCombine pass after LoopUnroll.
This is to cleanup some redundency generated by LoopUnroll pass. Such redundency may not be cleaned up by existing passes after LoopUnroll.

Differential Revision: http://reviews.llvm.org/D9777


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@237395 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-14 22:02:54 +00:00
Michael Zolotukhin
e2d9012a38 Reimplement heuristic for estimating complete-unroll optimization effects.
Summary:
This patch reimplements heuristic that tries to estimate optimization beneftis
from complete loop unrolling.

In this patch I kept the minimal changes - e.g. I removed code handling
branches and folding compares. That's a promising area, but now there
are too many questions to discuss before we can enable it.

Test Plan: Tests are included in the patch.

Reviewers: hfinkel, chandlerc

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D8816

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@237156 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-12 17:20:03 +00:00
Sanjoy Das
888e8e3a66 [LoopUnrollRuntime] Avoid high-cost trip count computation.
Summary:
Runtime unrolling of loops needs to emit an expression to compute the
loop's runtime trip-count.  Avoid runtime unrolling if this computation
will be expensive.

Depends on D8993.

Reviewers: atrick

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D8994

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@234846 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-14 03:20:38 +00:00
Sanjoy Das
e0f4a11a89 [LoopUnrollRuntime] Clean up a predicate.
Clean up a predicate I added in r229731, fix the relevant comment and
add a test case.  The earlier version is confusing to read and was also
buggy (probably not a coincidence) till Alexey fixed it in r233881.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@234701 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-12 01:24:01 +00:00
Kevin Qin
a56999c5de Reapply 'Run LICM pass after loop unrolling pass.'
It's firstly committed at r231630, and reverted at r231635.

Function pass InstructionSimplifier is inserted as barrier to
make sure loop unroll pass won't affect on LICM pass.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@232011 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-12 05:36:01 +00:00
Kevin Qin
1428f080a8 Revert r231630 - Run LICM pass after loop unrolling pass.
As it broke llvm bootstrap.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@231635 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-09 07:26:37 +00:00
Kevin Qin
40e66277f7 [AArch64] Enable partial & runtime unrolling on cortex-a57
For inner one of nested loops, it is more likely to be a hot loop,
and the runtime check can be promoted out from patch 0001, so the
overhead is less, we can try a doubled threshold to unroll more loops.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@231632 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-09 06:14:28 +00:00
Kevin Qin
994b4c784f Introduce runtime unrolling disable matadata and use it to mark the scalar loop from vectorization.
Runtime unrolling is an expensive optimization which can bring benefit
only if the loop is hot and iteration number is relatively large enough.
For some loops, we know they are not worth to be runtime unrolled.
The scalar loop from vectorization is one of the cases.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@231631 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-09 06:14:18 +00:00
Kevin Qin
3836dd0315 Run LICM pass after loop unrolling pass.
Runtime unrollng will introduce a runtime check in loop prologue.
If the unrolled loop is a inner loop, then the proglogue will be inside
the outer loop. LICM pass can help to promote the runtime check out if
the checked value is loop invariant.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@231630 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-09 06:14:07 +00:00
David Blaikie
7c9c6ed761 [opaque pointer type] Add textual IR support for explicit type parameter to load instruction
Essentially the same as the GEP change in r230786.

A similar migration script can be used to update test cases, though a few more
test case improvements/changes were required this time around: (r229269-r229278)

import fileinput
import sys
import re

pat = re.compile(r"((?:=|:|^)\s*load (?:atomic )?(?:volatile )?(.*?))(| addrspace\(\d+\) *)\*($| *(?:%|@|null|undef|blockaddress|getelementptr|addrspacecast|bitcast|inttoptr|\[\[[a-zA-Z]|\{\{).*$)")

for line in sys.stdin:
  sys.stdout.write(re.sub(pat, r"\1, \2\3*\4", line))

Reviewers: rafael, dexonsmith, grosser

Differential Revision: http://reviews.llvm.org/D7649

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@230794 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-27 21:17:42 +00:00
David Blaikie
198d8baafb [opaque pointer type] Add textual IR support for explicit type parameter to getelementptr instruction
One of several parallel first steps to remove the target type of pointers,
replacing them with a single opaque pointer type.

This adds an explicit type parameter to the gep instruction so that when the
first parameter becomes an opaque pointer type, the type to gep through is
still available to the instructions.

* This doesn't modify gep operators, only instructions (operators will be
  handled separately)

* Textual IR changes only. Bitcode (including upgrade) and changing the
  in-memory representation will be in separate changes.

* geps of vectors are transformed as:
    getelementptr <4 x float*> %x, ...
  ->getelementptr float, <4 x float*> %x, ...
  Then, once the opaque pointer type is introduced, this will ultimately look
  like:
    getelementptr float, <4 x ptr> %x
  with the unambiguous interpretation that it is a vector of pointers to float.

* address spaces remain on the pointer, not the type:
    getelementptr float addrspace(1)* %x
  ->getelementptr float, float addrspace(1)* %x
  Then, eventually:
    getelementptr float, ptr addrspace(1) %x

Importantly, the massive amount of test case churn has been automated by
same crappy python code. I had to manually update a few test cases that
wouldn't fit the script's model (r228970,r229196,r229197,r229198). The
python script just massages stdin and writes the result to stdout, I
then wrapped that in a shell script to handle replacing files, then
using the usual find+xargs to migrate all the files.

update.py:
import fileinput
import sys
import re

ibrep = re.compile(r"(^.*?[^%\w]getelementptr inbounds )(((?:<\d* x )?)(.*?)(| addrspace\(\d\)) *\*(|>)(?:$| *(?:%|@|null|undef|blockaddress|getelementptr|addrspacecast|bitcast|inttoptr|\[\[[a-zA-Z]|\{\{).*$))")
normrep = re.compile(       r"(^.*?[^%\w]getelementptr )(((?:<\d* x )?)(.*?)(| addrspace\(\d\)) *\*(|>)(?:$| *(?:%|@|null|undef|blockaddress|getelementptr|addrspacecast|bitcast|inttoptr|\[\[[a-zA-Z]|\{\{).*$))")

def conv(match, line):
  if not match:
    return line
  line = match.groups()[0]
  if len(match.groups()[5]) == 0:
    line += match.groups()[2]
  line += match.groups()[3]
  line += ", "
  line += match.groups()[1]
  line += "\n"
  return line

for line in sys.stdin:
  if line.find("getelementptr ") == line.find("getelementptr inbounds"):
    if line.find("getelementptr inbounds") != line.find("getelementptr inbounds ("):
      line = conv(re.match(ibrep, line), line)
  elif line.find("getelementptr ") != line.find("getelementptr ("):
    line = conv(re.match(normrep, line), line)
  sys.stdout.write(line)

apply.sh:
for name in "$@"
do
  python3 `dirname "$0"`/update.py < "$name" > "$name.tmp" && mv "$name.tmp" "$name"
  rm -f "$name.tmp"
done

The actual commands:
From llvm/src:
find test/ -name *.ll | xargs ./apply.sh
From llvm/src/tools/clang:
find test/ -name *.mm -o -name *.m -o -name *.cpp -o -name *.c | xargs -I '{}' ../../apply.sh "{}"
From llvm/src/tools/polly:
find test/ -name *.ll | xargs ./apply.sh

After that, check-all (with llvm, clang, clang-tools-extra, lld,
compiler-rt, and polly all checked out).

The extra 'rm' in the apply.sh script is due to a few files in clang's test
suite using interesting unicode stuff that my python script was throwing
exceptions on. None of those files needed to be migrated, so it seemed
sufficient to ignore those cases.

Reviewers: rafael, dexonsmith, grosser

Differential Revision: http://reviews.llvm.org/D7636

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@230786 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-27 19:29:02 +00:00
Sanjoy Das
6da5a456f4 Partial fix for bug 22589
Don't spend the entire iteration space in the scalar loop prologue if
computing the trip count overflows.  This change also gets rid of the
backedge check in the prologue loop and the extra check for
overflowing trip-count.

Differential Revision: http://reviews.llvm.org/D7715



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229731 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-18 19:32:25 +00:00
Chandler Carruth
25fa343bd8 [unroll] Concede defeat and disable the unroll analyzer for now.
The issues with the new unroll analyzer are more fundamental than code
cleanup, algorithm, or data structure changes. I've sent an email to the
original commit thread with details and a proposal for how to redesign
things. I'm disabling this for now so that we don't spend time
debugging issues with it in its current state.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229064 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-13 05:31:46 +00:00
Michael Zolotukhin
cd35fcecc2 Testcase for r228988.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228995 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-13 00:35:45 +00:00
Michael Zolotukhin
261a3a361b Add a test case for new unrolling heuristics.
THe heuristics were added in r228265 and r228434.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228713 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-10 17:54:54 +00:00
Alexander Potapenko
506c6ec22a Use a smaller pragma unroll threshold to reduce test execution time.
When opt is compiled with AddressSanitizer it takes more than 30 seconds
to unroll the loop in unroll_1M().


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226660 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-21 13:52:02 +00:00
Hal Finkel
6829815d96 [PowerPC] Readjust the loop unrolling threshold
Now that the way that the partial unrolling threshold for small loops is used
to compute the unrolling factor as been corrected, a slightly smaller threshold
is preferable. This is expected; other targets may need to re-tune as well.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225566 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-10 00:31:10 +00:00
Hal Finkel
a14d6f1ea5 [LoopUnroll] Fix the partial unrolling threshold for small loop sizes
When we compute the size of a loop, we include the branch on the backedge and
the comparison feeding the conditional branch. Under normal circumstances,
these don't get replicated with the rest of the loop body when we unroll. This
led to the somewhat surprising behavior that really small loops would not get
unrolled enough -- they could be unrolled more and the resulting loop would be
below the threshold, because we were assuming they'd take
(LoopSize * UnrollingFactor) instructions after unrolling, instead of
(((LoopSize-2) * UnrollingFactor)+2) instructions. This fixes that computation.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225565 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-10 00:30:55 +00:00
Hal Finkel
139bfee84c [PowerPC] Enable late partial unrolling on the POWER7
The P7 benefits from not have really-small loops so that we either have
multiple dispatch groups in the loop and/or the ability to form more-full
dispatch groups during scheduling. Setting the partial unrolling threshold to
44 seems good, empirically, for the P7. Compared to using no late partial
unrolling, this yields the following test-suite speedups:

SingleSource/Benchmarks/Adobe-C++/simple_types_constant_folding
	-66.3253% +/- 24.1975%
SingleSource/Benchmarks/Misc-C++/oopack_v1p8
	-44.0169% +/- 29.4881%
SingleSource/Benchmarks/Misc/pi
	-27.8351% +/- 12.2712%
SingleSource/Benchmarks/Stanford/Bubblesort
	-30.9898% +/- 22.4647%

I've speculatively added a similar setting for the P8. Also, I've noticed that
the unroller does not quite calculate the unrolling factor correctly for really
tiny loops because it neglects to account for the fact that not every loop body
replicant contains an ending branch and counter increment. I'll fix that later.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225522 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-09 15:51:16 +00:00
Duncan P. N. Exon Smith
f416d72973 IR: Add 'distinct' MDNodes to bitcode and assembly
Propagate whether `MDNode`s are 'distinct' through the other types of IR
(assembly and bitcode).  This adds the `distinct` keyword to assembly.

Currently, no one actually calls `MDNode::getDistinct()`, so these nodes
only get created for:

  - self-references, which are never uniqued, and
  - nodes whose operands are replaced that hit a uniquing collision.

The concept of distinct nodes is still not quite first-class, since
distinct-ness doesn't yet survive across `MapMetadata()`.

Part of PR22111.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225474 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-08 22:38:29 +00:00
Duncan P. N. Exon Smith
1ef70ff39b IR: Make metadata typeless in assembly
Now that `Metadata` is typeless, reflect that in the assembly.  These
are the matching assembly changes for the metadata/value split in
r223802.

  - Only use the `metadata` type when referencing metadata from a call
    intrinsic -- i.e., only when it's used as a `Value`.

  - Stop pretending that `ValueAsMetadata` is wrapped in an `MDNode`
    when referencing it from call intrinsics.

So, assembly like this:

    define @foo(i32 %v) {
      call void @llvm.foo(metadata !{i32 %v}, metadata !0)
      call void @llvm.foo(metadata !{i32 7}, metadata !0)
      call void @llvm.foo(metadata !1, metadata !0)
      call void @llvm.foo(metadata !3, metadata !0)
      call void @llvm.foo(metadata !{metadata !3}, metadata !0)
      ret void, !bar !2
    }
    !0 = metadata !{metadata !2}
    !1 = metadata !{i32* @global}
    !2 = metadata !{metadata !3}
    !3 = metadata !{}

turns into this:

    define @foo(i32 %v) {
      call void @llvm.foo(metadata i32 %v, metadata !0)
      call void @llvm.foo(metadata i32 7, metadata !0)
      call void @llvm.foo(metadata i32* @global, metadata !0)
      call void @llvm.foo(metadata !3, metadata !0)
      call void @llvm.foo(metadata !{!3}, metadata !0)
      ret void, !bar !2
    }
    !0 = !{!2}
    !1 = !{i32* @global}
    !2 = !{!3}
    !3 = !{}

I wrote an upgrade script that handled almost all of the tests in llvm
and many of the tests in cfe (even handling many `CHECK` lines).  I've
attached it (or will attach it in a moment if you're speedy) to PR21532
to help everyone update their out-of-tree testcases.

This is part of PR21532.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@224257 91177308-0d34-0410-b5e6-96231b3b80d8
2014-12-15 19:07:53 +00:00
Michael Zolotukhin
4e7b10b07f Fix a trip-count overflow issue in LoopUnroll.
Currently LoopUnroll generates a prologue loop before the main loop
body to execute first N%UnrollFactor iterations. Also, this loop is
used if trip-count can overflow - it's determined by a runtime check.

However, we've been mistakenly optimizing this loop to a linear code for
UnrollFactor = 2, not taking into account that it also serves as a safe
version of the loop if its trip-count overflows.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@222451 91177308-0d34-0410-b5e6-96231b3b80d8
2014-11-20 20:19:55 +00:00
Bradley Smith
8cff277de2 [SCEV] Improve Scalar Evolution's use of no {un,}signed wrap flags
In a case where we have a no {un,}signed wrap flag on the increment, if
RHS - Start is constant then we can avoid inserting a max operation bewteen
the two, since we can statically determine which is greater.

This allows us to unroll loops such as:

 void testcase3(int v) {
   for (int i=v; i<=v+1; ++i)
     f(i);
 }


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@220960 91177308-0d34-0410-b5e6-96231b3b80d8
2014-10-31 11:40:32 +00:00
Mark Heffernan
ed05e3703e This patch de-pessimizes the calculation of loop trip counts in
ScalarEvolution in the presence of multiple exits. Previously all
loops exits had to have identical counts for a loop trip count to be
considered computable. This pessimization was implemented by calling
getBackedgeTakenCount(L) rather than getExitCount(L, ExitingBlock)
inside of ScalarEvolution::getSmallConstantTripCount() (see the FIXME
in the comments of that function). The pessimization was added to fix
a corner case involving undefined behavior (pr/16130). This patch more
precisely handles the undefined behavior case allowing the pessimization
to be removed.

ControlsExit replaces IsSubExpr to more precisely track the case where
undefined behavior is expected to occur. Because undefined behavior is
tracked more precisely we can remove MustExit from ExitLimit. MustExit
was used to track the case where the limit was computed potentially
assuming undefined behavior even if undefined behavior didn't necessarily
occur.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@219517 91177308-0d34-0410-b5e6-96231b3b80d8
2014-10-10 17:39:11 +00:00
Duncan P. N. Exon Smith
de7465af34 LoopUnroll: Create sub-loops in LoopInfo
`LoopUnrollPass` says that it preserves `LoopInfo` -- make it so.  In
particular, tell `LoopInfo` about copies of inner loops when unrolling
the outer loop.

Conservatively, also tell `ScalarEvolution` to forget about the original
versions of these loops, since their inputs may have changed.

Fixes PR20987.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@219241 91177308-0d34-0410-b5e6-96231b3b80d8
2014-10-07 21:19:00 +00:00
Kevin Qin
dbaeb6e7cb Use a loop to simplify the runtime unrolling prologue.
Runtime unrolling will create a prologue to execute the extra
iterations which is can't divided by the unroll factor. It
generates an if-then-else sequence to jump into a factor -1
times unrolled loop body, like

    extraiters = tripcount % loopfactor
    if (extraiters == 0) jump Loop:
    if (extraiters == loopfactor) jump L1
    if (extraiters == loopfactor-1) jump L2
    ...
    L1:  LoopBody;
    L2:  LoopBody;
    ...
    if tripcount < loopfactor jump End
    Loop:
    ...
    End:

It means if the unroll factor is 4, the loop body will be 7
times unrolled, 3 are in loop prologue, and 4 are in the loop.
This commit is to use a loop to execute the extra iterations
in prologue, like

        extraiters = tripcount % loopfactor
        if (extraiters == 0) jump Loop:
        else jump Prol
 Prol:  LoopBody;
        extraiters -= 1                 // Omitted if unroll factor is 2.
        if (extraiters != 0) jump Prol: // Omitted if unroll factor is 2.
        if (tripcount < loopfactor) jump End
 Loop:
 ...
 End:

Then when unroll factor is 4, the loop body will be copied by
only 5 times, 1 in the prologue loop, 4 in the original loop.
And if the unroll factor is 2, new loop won't be created, just
as the original solution.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218604 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-29 11:15:00 +00:00
David Peixotto
ea468dddfe Ignore annotation function calls in cost computation
The annotation instructions are dropped during codegen and have no
impact on size.  In some cases, the annotations were preventing the
unroller from unrolling a loop because the annotation calls were
pushing the cost over the unrolling threshold.

Differential Revision: http://reviews.llvm.org/D5335



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218525 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-26 17:48:40 +00:00
Hal Finkel
3d03d60ca8 Add functions for finding ephemeral values
This adds a set of utility functions for collecting 'ephemeral' values. These
are LLVM IR values that are used only by @llvm.assume intrinsics (directly or
indirectly), and thus will be removed prior to code generation, implying that
they should be considered free for certain purposes (like inlining). The
inliner's cost analysis, and a few other passes, have been updated to account
for ephemeral values using the provided functionality.

This functionality is important for the usability of @llvm.assume, because it
limits the "non-local" side-effects of adding llvm.assume on inlining, loop
unrolling, etc. (these are hints, and do not generate code, so they should not
directly contribute to estimates of execution cost).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217335 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-07 13:49:57 +00:00
Mark Heffernan
d10aa6f8b2 After unrolling a loop with llvm.loop.unroll.count metadata (unroll factor
hint) the loop unroller replaces the llvm.loop.unroll.count metadata with
llvm.loop.unroll.disable metadata to prevent any subsequent unrolling
passes from unrolling more than the hint indicates.  This patch fixes
an issue where loop unrolling could be disabled for other loops as well which
share the same llvm.loop metadata.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213900 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-24 22:36:40 +00:00
Mark Heffernan
d55c7c7f42 Do not add unroll disable metadata after unrolling pass for loops with #pragma clang loop unroll(full).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213789 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-23 20:05:44 +00:00
Mark Heffernan
e8d7ebcd5a In unroll pragma syntax and loop hint metadata, change "enable" forms to a new form using the string "full".
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213772 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-23 17:31:37 +00:00
Mark Heffernan
354f2afffd Remove unroll pragma metadata after it is used.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213412 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-18 21:04:33 +00:00
Eli Bendersky
bb167336b3 Rename loop unrolling and loop vectorizer metadata to have a common prefix.
[LLVM part]

These patches rename the loop unrolling and loop vectorizer metadata
such that they have a common 'llvm.loop.' prefix.  Metadata name
changes:

llvm.vectorizer.* => llvm.loop.vectorizer.*
llvm.loopunroll.* => llvm.loop.unroll.*

This was a suggestion from an earlier review
(http://reviews.llvm.org/D4090) which added the loop unrolling
metadata. 

Patch by Mark Heffernan.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211710 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-25 15:41:00 +00:00
Benjamin Kramer
6b7ff6be9c LoopUnrollRuntime: Check for overflow in the trip count calculation.
Fixes PR19823.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211436 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-21 13:46:25 +00:00
Eli Bendersky
4046db0cdb Teach LoopUnrollPass to respect loop unrolling hints in metadata.
[This is resubmitting r210721, which was reverted due to suspected breakage
which turned out to be unrelated].

Some extra review comments were addressed. See D4090 and D4147 for more details.

The Clang change that produces this metadata was committed in r210667

Patch by Mark Heffernan.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211076 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-16 23:53:02 +00:00
Eli Bendersky
26278061ee Revert r210721 as it causes breakage in internal builds (and possibly GDB).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210807 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-12 18:05:39 +00:00
Eli Bendersky
87fcb536ab Teach LoopUnrollPass to respect loop unrolling hints in metadata.
See http://reviews.llvm.org/D4090 for more details.

The Clang change that produces this metadata was committed in r210667

Patch by Mark Heffernan.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210721 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 23:15:35 +00:00
Alp Toker
8aeca44558 Reduce verbiage of lit.local.cfg files
We can just split targets_to_build in one place and make it immutable.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210496 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-09 22:42:55 +00:00
Dinesh Dwivedi
2a747bf1c5 LCSSA should be performed on the outermost affected loop while unrolling loop.
During loop-unroll, loop exits from the current loop may end up in in different
outer loop. This requires to re-form LCSSA recursively for one level down from
the outer most loop where loop exits are landed during unroll. This fixes PR18861.

Differential Revision: http://reviews.llvm.org/D2976



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209796 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-29 06:47:23 +00:00
Hal Finkel
f35ce2376c Move late partial-unrolling thresholds into the processor definitions
The old method used by X86TTI to determine partial-unrolling thresholds was
messy (because it worked by testing target features), and also would not
correctly identify the target CPU if certain target features were disabled.
After some discussions on IRC with Chandler et al., it was decided that the
processor scheduling models were the right containers for this information
(because it is often tied to special uop dispatch-buffer sizes).

This does represent a small functionality change:
 - For generic x86-64 (which uses the SB model and, thus, will get some
   unrolling).
 - For AMD cores (because they still currently use the SB scheduling model)
 - For Haswell (based on benchmarking by Louis Gerbarg, it was decided to bump
   the default threshold to 50; we're working on a test case for this).
Otherwise, nothing has changed for any other targets. The logic, however, has
been moved into BasicTTI, so other targets may now also opt-in to this
functionality simply by setting LoopMicroOpBufferSize in their processor
model definitions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@208289 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-08 09:14:44 +00:00
Benjamin Kramer
99b03e3401 LoopUnroll: If we're doing partial unrolling, use the PartialThreshold to limit unrolling.
Otherwise we use the same threshold as for complete unrolling, which is
way too high. This made us unroll any loop smaller than 150 instructions
by 8 times, but only if someone specified -march=core2 or better,
which happens to be the default on darwin.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207940 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-04 19:12:38 +00:00
Diego Novillo
55deff895d Fix vectorization remarks.
This patch changes the vectorization remarks to also inform when
vectorization is possible but not beneficial.

Added tests to exercise some loop remarks.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207574 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-29 20:06:10 +00:00
Hal Finkel
e30aa957e3 Implement X86TTI::getUnrollingPreferences
This provides an initial implementation of getUnrollingPreferences for x86.
getUnrollingPreferences is used by the generic (concatenation) unroller, which
is distinct from the unrolling done by the loop vectorizer. Many modern x86
cores have some kind of uop cache and loop-stream detector (LSD) used to
efficiently dispatch small loops, and taking full advantage of this requires
unrolling small loops (small here means 10s of uops).

These caches also have limits on the number of taken branches in the loop, and
so we also cap the loop unrolling factor based on the maximum "depth" of the
loop. This is currently calculated with a partial DFS traversal (partial
because it will stop early if the path length grows too much). This is still an
approximation, and one that is both conservative (because it does not account
for branches eliminated via block placement) and optimistic (because it is only
recording the maximum depth over minimum paths). Nevertheless, because the
loops that fit in these uop caches are so small, it is not clear how much the
details matter.

The original set of patches posted for review produced the following test-suite
performance results (from the TSVC benchmark) at that time:
  ControlLoops-dbl - 13% speedup
  ControlLoops-flt - 15% speedup
  Reductions-dbl - 7.5% speedup

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205348 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-01 18:50:34 +00:00
Hal Finkel
a5d756ca39 Implement TTI getUnrollingPreferences for PowerPC
The PowerPC A2 core greatly benefits from aggressive concatenation unrolling;
use the new getUnrollingPreferences to enable this by default when targeting
the PPC A2 core.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190549 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-11 21:20:40 +00:00
Daniel Dunbar
24ec2e5a72 [tests] Cleanup initialization of test suffixes.
- Instead of setting the suffixes in a bunch of places, just set one master
   list in the top-level config. We now only modify the suffix list in a few
   suites that have one particular unique suffix (.ml, .mc, .yaml, .td, .py).

 - Aside from removing the need for a bunch of lit.local.cfg files, this enables
   4 tests that were inadvertently being skipped (one in
   Transforms/BranchFolding, a .s file each in DebugInfo/AArch64 and
   CodeGen/PowerPC, and one in CodeGen/SI which is now failing and has been
   XFAILED).

 - This commit also fixes a bunch of config files to use config.root instead of
   older copy-pasted code.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188513 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-16 00:37:11 +00:00
Stephen Lin
181a289256 Fixup to r186268 and r186269: don't append -LABEL to CHECK-NOT. No functionality change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186271 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-14 02:10:57 +00:00
Stephen Lin
39f4e8d9cc Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.
This update was done with the following bash script:

  find test/Transforms -name "*.ll" | \
  while read NAME; do
    echo "$NAME"
    if ! grep -q "^; *RUN: *llc" $NAME; then
      TEMP=`mktemp -t temp`
      cp $NAME $TEMP
      sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \
      while read FUNC; do
        sed -i '' "s/;\(.*\)\([A-Za-z0-9_]*\):\( *\)@$FUNC\([( ]*\)\$/;\1\2-LABEL:\3@$FUNC(/g" $TEMP
      done
      mv $TEMP $NAME
    fi
  done


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186268 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-14 01:42:54 +00:00
Stephen Lin
82e539d037 Modify two Transforms tests to explicitly check for full function names in some cases, rather than just a common prefix. No functionality change.
(This is to avoid confusing a scripted mass update of these tests to use CHECK-LABEL)


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186267 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-14 01:38:19 +00:00
Andrew Trick
cd8e3c4dcf Prevent loop-unroll from making assumptions about undefined behavior.
Fixes rdar:14036816, PR16130.

There is an opportunity to compute precise trip counts for 'or'
expressions and multi-exit loops.
rdar:14038809: Optimize trip count computation for multi-exit loops.

To do this we need to record the fact that ExitLimit assumes NSW. When
it does not we can safely assume that the loop trip count is the
minimum ExitLimt across all subexpressions and loop exits.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183060 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-31 23:34:46 +00:00
Jan Wen Voung
4323665bd8 Revert the test moves from 176733. Use "REQUIRES: asserts" instead.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176873 91177308-0d34-0410-b5e6-96231b3b80d8
2013-03-12 16:27:52 +00:00
Jan Wen Voung
fa785cb22d Disable statistics on Release builds and move tests that depend on -stats.
Summary:
Statistics are still available in Release+Asserts (any +Asserts builds),
and stats can also be turned on with LLVM_ENABLE_STATS.

Move some of the FastISel stats that were moved under DEBUG()
back out of DEBUG(), since stats are disabled across the board now.

Many tests depend on grepping "-stats" output.  Move those into
a orig_dir/Stats/. so that they can be marked as unsupported
when building without statistics.

Differential Revision: http://llvm-reviews.chandlerc.com/D486

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176733 91177308-0d34-0410-b5e6-96231b3b80d8
2013-03-08 22:56:31 +00:00
James Molloy
67ae135759 Add a new attribute, 'noduplicate'. If a function contains a noduplicate call, the call cannot be duplicated - Jump threading, loop unrolling, loop unswitching, and loop rotation are inhibited if they would duplicate the call.
Similarly inlining of the function is inhibited, if that would duplicate the call (in particular inlining is still allowed when there is only one callsite and the function has internal linkage).



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170704 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20 16:04:27 +00:00
Hal Finkel
8c65549318 getSmallConstantTripMultiple should never return zero.
When the trip count is -1, getSmallConstantTripMultiple could return zero,
and this would cause runtime loop unrolling to assert. Instead of returning
zero, one is now returned (consistent with the existing overflow cases).
Fixes PR14167.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166612 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 19:46:44 +00:00
Benjamin Kramer
e7381cc99c Fix tests that didn't test anything.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164686 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-26 09:51:39 +00:00
Andrew Trick
d9fc1ce809 Fix 12513: Loop unrolling breaks with indirect branches.
Take this opportunity to generalize the indirectbr bailout logic for
loop transformations. CFG transformations will never get indirectbr
right, and there's no point trying.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154386 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-10 05:14:42 +00:00
Hongbin Zheng
99c8a5a64a Add testcase for r154007, when a function has the optsize attribute,
the loop should be unrolled according the value of OptSizeUnrollThreshold.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154014 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-04 13:24:40 +00:00
Andrew Trick
dba95737e0 Remove redundant -enable-iv-rewrite=false flags from test cases.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153255 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-22 17:09:04 +00:00
Eli Bendersky
0f0c411079 Replace all instances of dg.exp file with lit.local.cfg, since all tests are run with LIT now and now Dejagnu. dg.exp is no longer needed.
Patch reviewed by Daniel Dunbar. It will be followed by additional cleanup patches.




git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@150664 91177308-0d34-0410-b5e6-96231b3b80d8
2012-02-16 06:28:33 +00:00
Andrew Trick
5d73448bb7 Add -unroll-runtime for unrolling loops with run-time trip counts.
Patch by Brendon Cahoon!

This extends the existing LoopUnroll and LoopUnrollPass. Brendon
measured no regressions in the llvm test suite with -unroll-runtime
enabled. This implementation works by using the existing loop
unrolling code to unroll the loop by a power-of-two (default 8). It
generates an if-then-else sequence of code prior to the loop to
execute the extra iterations before entering the unrolled loop.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146245 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-09 06:19:40 +00:00
Andrew Trick
5865a8dfde Fix a corner case in updating LoopInfo after fully unrolling an outer loop.
The loop tree's inclusive block lists are painful and expensive to
update. (I have no idea why they're inclusive). The design was
supposed to handle this case but the implementation missed it and my
unit tests weren't thorough enough.

Fixes PR11335: loop unroll update.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@144970 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-18 03:42:41 +00:00
Nick Lewycky
d7ecff49d9 Don't try to loop on iterators that are potentially invalidated inside the loop. Fixes PR11361!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@144454 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-12 03:09:12 +00:00
Andrew Trick
d974ea22a5 Unit test for r140919, loop unroll heuristics.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@141049 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-04 00:07:02 +00:00
Andrew Trick
b0598b14aa Reapply r139759. Disable IV rewriting by default. See PR10916.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139842 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-15 20:58:37 +00:00
Andrew Trick
effb647154 [indvars] Revert r139579 until 401.bzip -arch i386 miscompilation is fixed. PR10920.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139583 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-13 05:23:49 +00:00
Andrew Trick
dd9d758a31 Disable IV rewriting by default. See PR10916.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139579 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-13 03:23:21 +00:00