Nadav Rotem 4bf6326d08 LoopVectorize: Implement partial loop unrolling when vectorization is not profitable. ...

This patch enables unrolling of loops when vectorization is legal but not profitable.
We add a new class InnerLoopUnroller, that extends InnerLoopVectorizer and replaces some of the vector-specific logic with scalars.

This patch does not introduce any runtime regressions and improves the following workloads:

SingleSource/Benchmarks/Shootout/matrix -22.64%
SingleSource/Benchmarks/Shootout-C++/matrix -13.06%
External/SPEC/CINT2006/464_h264ref/464_h264ref  -3.99%
SingleSource/Benchmarks/Adobe-C++/simple_types_constant_folding -1.95%



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189281 91177308-0d34-0410-b5e6-96231b3b80d8

2013-08-26 22:33:26 +00:00

..

ARM

[tests] Cleanup initialization of test suffixes.

2013-08-16 00:37:11 +00:00

X86

[tests] Cleanup initialization of test suffixes.

2013-08-16 00:37:11 +00:00

12-12-11-if-conv.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

2012-10-20-infloop.ll

Force a fixed unroll count on the target independent tests.

2013-01-05 00:58:48 +00:00

2012-10-22-isconsec.ll

TBAA: remove !tbaa from testing cases if not used.

2013-05-02 18:11:35 +00:00

bsd_regex.ll

LoopVectorizer: Bail out if we don't have datalayout we need it

2013-04-24 16:15:58 +00:00

bzip_reverse_loops.ll

TBAA: remove !tbaa from testing cases if not used.

2013-05-02 18:11:35 +00:00

calloc.ll

TBAA: remove !tbaa from testing cases if not used.

2013-04-30 17:52:57 +00:00

cast-induction.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

cpp-new-array.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

dbg.value.ll

Use function attributes to indicate that we don't want to realign the stack.

2013-08-01 21:42:05 +00:00

debugloc.ll

[Debug Info Tests] Update testing cases.

2013-08-22 17:11:18 +00:00

flags.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

float-reduction.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

funcall.ll

LoopVectorize: Math functions only read rounding mode

2013-07-01 00:54:44 +00:00

gcc-examples.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

global_alias.ll

Catch more CHECK that can be converted to CHECK-LABEL in Transforms for easier debugging. No functionality change.

2013-07-14 01:50:49 +00:00

hoist-loads.ll

LoopVectorize: Hoist conditional loads if possible

2013-05-15 01:44:30 +00:00

i8-induction.ll

TBAA: remove !tbaa from testing cases if not used.

2013-04-30 17:52:57 +00:00

if-conv-crash.ll

LoopVectorize: Handle single edge PHIs

2013-05-18 18:38:34 +00:00

if-conversion-edgemasks.ll

LoopVectorize: Cache edge masks created during if-conversion

2013-06-27 20:31:06 +00:00

if-conversion-nest.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

if-conversion-reduction.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

if-conversion.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

increment.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

induction_plus.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

induction.ll

LoopVectorize: Handle loops with multiple forward inductions

2013-05-14 00:21:18 +00:00

infiniteloop.ll

LoopVectorizer: Don't assert on the absence of induction variables

2013-05-09 00:32:18 +00:00

intrinsic.ll

Add a llvm.copysign intrinsic

2013-08-19 23:35:46 +00:00

lcssa-crash.ll

LoopVectorize: LoopSimplify can't canonicalize loops with an indirectbr in it, don't assert on those cases.

2013-05-24 18:05:35 +00:00

lifetime.ll

LoopVectorize: Allow vectorization of loops with lifetime markers

2013-08-06 22:37:52 +00:00

memdep.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

metadata-unroll.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

metadata-width.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

minmax_reduction.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

multiple-address-spaces.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

no_idiv_reduction.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

no_int_induction.ll

Teach LoopVectorize about address space sizes

2013-08-22 02:42:55 +00:00

no_outside_user.ll

LoopVectorize: PHIs with only outside users should prevent vectorization

2013-05-31 19:53:50 +00:00

nofloat.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

non-const-n.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

nsw-crash.ll

LoopVectorizer: Fix a bug in the vectorization of BinaryOperators. The BinaryOperator can be folded to an Undef, and we don't want to set NSW flags to undef vals.

2013-01-10 17:34:39 +00:00

phi-hang.ll

LoopVectorize: Use a set to avoid longer cycles in the reduction chain too.

2013-04-18 14:29:13 +00:00

ptr_loops.ll

Add support for reverse pointer induction variables. These are loops that contain pointers that count backwards.

2013-01-23 01:35:00 +00:00

read-only.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

reduction.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

reverse_induction.ll

LoopVectorize: Use the widest induction variable type

2013-05-11 23:04:28 +00:00

reverse_iter.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

runtime-check-readonly.ll

LoopVectorizer: No need to generate pointer disambiguation checks between readonly pointers.

2013-04-25 19:55:03 +00:00

runtime-check.ll

Reapply 184685 after the SetVector iteration order fix.

2013-06-24 12:09:15 +00:00

runtime-limit.ll

LoopVectorizer: Calculate the number of pointers to disambiguate at runtime based on the numbers of reads and writes.

2013-04-26 05:08:59 +00:00

safegep.ll

LoopVectorize: Vectorize all accesses in address space zero with unit stride

2013-07-11 15:21:55 +00:00

same-base-access.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

scalar-select.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

scev-exitlim-crash.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

simple-unroll.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

small-loop.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

start-non-zero.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

store-shuffle-bug.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

struct_access.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

undef-inst-bug.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

unroll_novec.ll

LoopVectorize: Implement partial loop unrolling when vectorization is not profitable.

2013-08-26 22:33:26 +00:00

value-ptr-bug.ll

LoopVectorize: Make Value pointers that could be RAUW'ed a VH

2013-05-22 16:54:56 +00:00

vectorize-once.ll

Add support for llvm.vectorizer metadata

2013-05-28 20:00:34 +00:00

write-only.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00