Zvi Rackover 3438d07f09 LoopVectorize: MaxVF should not be larger than the loop trip count ...

Summary:
Improve how MaxVF is computed while taking into account that MaxVF should not be larger than the loop's trip count.

Other than saving on compile-time by pruning the possible MaxVF candidates, this patch fixes pr34438 which exposed the following flow:
1. Short trip count identified -> Don't bail out, set OptForSize:=True to avoid tail-loop and runtime checks.
2. Compute MaxVF returned 16 on a target supporting AVX512.
3. OptForSize -> choose VF:=MaxVF.
4. Bail out because TripCount = 8, VF = 16, TripCount % VF !=0 means we need a tail loop.

With this patch step 2. will choose MaxVF=8 based on TripCount.

Reviewers: Ayal, dorit, mkuper, hfinkel

Reviewed By: hfinkel

Subscribers: hfinkel, llvm-commits

Differential Revision: https://reviews.llvm.org/D37425

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312472 91177308-0d34-0410-b5e6-96231b3b80d8

2017-09-04 08:35:13 +00:00

..

AArch64

[LV] Fix PR34248 - recommit D32871 after revert r311304

2017-08-27 12:55:46 +00:00

AMDGPU

LV: Don't insert runtime ptr checks on divergent targets

2017-08-02 21:43:08 +00:00

ARM

Revert "r306473 - re-commit r306336: Enable vectorizer-maximize-bandwidth by default."

2017-07-01 03:24:09 +00:00

PowerPC

[ppc] Correctly compute the cost of loading 32/64 bit memory into VSR

2016-12-03 00:41:43 +00:00

SystemZ

[LV] Fix PR34248 - recommit D32871 after revert r311304

2017-08-27 12:55:46 +00:00

X86

LoopVectorize: MaxVF should not be larger than the loop trip count

2017-09-04 08:35:13 +00:00

XCore

[opaque pointer type] Add textual IR support for explicit type parameter to getelementptr instruction

2015-02-27 19:29:02 +00:00

12-12-11-if-conv.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

2012-10-20-infloop.ll

Rename getMaximumUnrollFactor -> getMaxInterleaveFactor; also rename option names controlling this variable.

2014-09-10 17:58:16 +00:00

2012-10-22-isconsec.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

2016-07-27-loop-vec.ll

Fix the assertion error in collectLoopUniforms caused by empty Worklist before expanding.

2016-07-27 23:53:58 +00:00

align.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

bsd_regex.ll

[opaque pointer type] Add textual IR support for explicit type parameter to getelementptr instruction

2015-02-27 19:29:02 +00:00

bzip_reverse_loops.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

calloc.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

cast-induction.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

conditional-assignment.ll

Reapply "[LV] Enable vectorization of loops with conditional stores by default"

2016-12-16 19:12:02 +00:00

consec_no_gep.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

consecutive-ptr-uniforms.ll

[LV] Consider users that are memory accesses in uniforms expansion step

2017-03-07 18:47:30 +00:00

control-flow.ll

[LV] Stop saying "use -Rpass-analysis=loop-vectorize"

2016-11-11 22:51:46 +00:00

cpp-new-array.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

dbg.value.ll

Canonicalize the representation of empty an expression in DIGlobalVariableExpression

2017-08-30 18:06:51 +00:00

dead_instructions.ll

[LV] Avoid emitting trivially dead instructions

2016-10-19 19:22:02 +00:00

debugloc.ll

Remove the obsolete offset parameter from @llvm.dbg.value

2017-07-28 20:21:02 +00:00

diag-missing-instr-debug-loc.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

diag-with-hotness-info-2.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

diag-with-hotness-info.ll

[LoopVersioning] Require loop-simplify form for loop versioning.

2016-12-19 17:13:37 +00:00

discriminator.ll

Encode duplication factor from loop vectorization and loop unrolling to discriminator.

2017-02-10 21:09:07 +00:00

duplicated-metadata.ll

[Loop Vectorize] Added a separate metadata

2017-08-20 10:32:41 +00:00

ee-crash.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

exact.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

fcmp-vectorize.ll

[LoopVectorize] Fix assertion failure in Fcmp vectorization

2017-08-08 18:07:44 +00:00

first-order-recurrence.ll

[LV] Fix PR34248 - recommit D32871 after revert r311304

2017-08-27 12:55:46 +00:00

flags.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

float-induction.ll

[SimplifyCFG] Defer folding unconditional branches to LateSimplifyCFG if it can destroy canonical loop structure.

2017-07-19 08:53:34 +00:00

float-reduction.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

funcall.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

gcc-examples.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

gep_with_bitcast.ll

[LV] Don't widen trivial induction variables

2016-07-06 14:26:59 +00:00

global_alias.ll

[LV] Scalarize instructions marked scalar after vectorization

2016-09-26 17:08:37 +00:00

hints-trans.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

hoist-loads.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

i8-induction.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

icmp-uniforms.ll

[LV] Don't mark multi-use branch conditions uniform

2016-10-07 15:20:13 +00:00

if-conv-crash.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

if-conversion-edgemasks.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

if-conversion-nest.ll

[Loop Vectorize] Added a separate metadata

2017-08-20 10:32:41 +00:00

if-conversion-reduction.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

if-conversion.ll

[LoopVectorize] partly revert r307475

2017-07-08 16:34:46 +00:00

if-pred-non-void.ll

[LV] Fix PR34248 - recommit D32871 after revert r311304

2017-08-27 12:55:46 +00:00

if-pred-not-when-safe.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

if-pred-stores.ll

[LV] Avoid redundant operations manipulating masks

2017-07-31 13:21:42 +00:00

incorrect-dom-info.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

increment.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

induction_plus.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

induction-step.ll

[LV] Test once if vector trip count is zero, instead of twice

2017-07-19 05:16:39 +00:00

induction.ll

[LV] Test once if vector trip count is zero, instead of twice

2017-07-19 05:16:39 +00:00

infiniteloop.ll

Fix missing CHECK-LABELs

2013-10-02 20:29:00 +00:00

interleaved-accesses-1.ll

Second attempt at r285517.

2016-10-31 13:17:31 +00:00

interleaved-accesses-2.ll

Second attempt at r285517.

2016-10-31 13:17:31 +00:00

interleaved-accesses-3.ll

Second attempt at r285517.

2016-10-31 13:17:31 +00:00

interleaved-accesses-pred-stores.ll

[LV] Test once if vector trip count is zero, instead of twice

2017-07-19 05:16:39 +00:00

interleaved-accesses.ll

[LV] Test once if vector trip count is zero, instead of twice

2017-07-19 05:16:39 +00:00

intrinsic.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

iv_outside_user.ll

[LV] Test once if vector trip count is zero, instead of twice

2017-07-19 05:16:39 +00:00

lcssa-crash.ll

[LV] Fix an issue where forming LCSSA in the place that we did would

2017-01-26 10:41:09 +00:00

lifetime.ll

Add address space mangling to lifetime intrinsics

2017-04-10 20:18:21 +00:00

loop-form.ll

[opaque pointer type] Add textual IR support for explicit type parameter to getelementptr instruction

2015-02-27 19:29:02 +00:00

loop-scalars.ll

Reapply r298620: [LV] Vectorize GEPs

2017-04-07 14:15:34 +00:00

loop-vect-memdep.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

memdep.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

metadata-unroll.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

metadata-width.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

metadata.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

miniters.ll

[LV] Test once if vector trip count is zero, instead of twice

2017-07-19 05:16:39 +00:00

minmax_reduction.ll

[InstCombine] canonicalize icmp predicate feeding select

2017-06-27 17:53:22 +00:00

multi-use-reduction-bug.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

multiple-address-spaces.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

multiple-strides-vectorization.ll

[Loop Vectorize] Added a separate metadata

2017-08-20 10:32:41 +00:00

no_array_bounds.ll

[PR27284] Reverse the ownership between DICompileUnit and DISubprogram.

2016-04-15 15:57:41 +00:00

no_idiv_reduction.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

no_int_induction.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

no_outside_user.ll

Recommit [LV] Enable vectorization of loops where the IV has an external use

2016-06-15 00:35:26 +00:00

no_switch.ll

[LV] Stop saying "use -Rpass-analysis=loop-vectorize"

2016-11-11 22:51:46 +00:00

noalias-md-licm.ll

[LoopVectorize] Annotate versioned loop with noalias metadata

2016-03-17 20:32:37 +00:00

noalias-md.ll

[LoopVectorize] Annotate versioned loop with noalias metadata

2016-03-17 20:32:37 +00:00

nofloat.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

non-const-n.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

nontemporal.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

nsw-crash.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

opt.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

optsize.ll

fix minsize detection: minsize attribute implies optimizing for size

2015-08-11 15:56:31 +00:00

partial-lcssa.ll

[LV] Run loop-simplify and LCSSA explicitly instead of "requiring" them

2017-01-19 00:42:28 +00:00

phi-cost.ll

[InstCombine] Teach select01 helper of foldSelectIntoOp to handle vector splats

2017-08-28 22:00:27 +00:00

phi-hang.ll

[Verifier] Reject PHIs using defs from own block.

2016-03-26 23:32:57 +00:00

pr25281.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

pr28541.ll

[LoopVectorize] Detect loops in the innermost loop before creating InnerLoopVectorizer

2016-08-12 22:47:13 +00:00

pr30654-phiscev-sext-trunc.ll

PSCEV] Create AddRec for Phis in cases of possible integer overflow,

2017-07-18 11:57:08 +00:00

pr31098.ll

[LV/LoopAccess] Check statically if an unknown dependence distance can be

2017-02-12 09:32:53 +00:00

pr31190.ll

[LV] Run loop-simplify and LCSSA explicitly instead of "requiring" them

2017-01-19 00:42:28 +00:00

pr32859.ll

[LoopOptimizer][Fix]PR32859, PR24738

2017-05-13 13:25:57 +00:00

pr33706.ll

[LV] Don't allow outside uses of IVs if the SCEV is predicated on loop conditions.

2017-07-12 19:53:55 +00:00

ptr_loops.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

ptr-induction.ll

Reverted patch 273864

2016-06-29 10:01:06 +00:00

read-only.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

reduction.ll

[LV] Allow reductions that have several uses outside the loop

2017-01-18 19:02:52 +00:00

reverse_induction.ll

[LV] Don't emit unused scalars for uniform instructions

2016-09-21 16:50:24 +00:00

reverse_iter.ll

Reapply "[LV] Extend trunc optimization to all IVs with constant integer steps"

2017-02-14 16:28:32 +00:00

runtime-check-address-space.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

runtime-check-readonly-address-space.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

runtime-check-readonly.ll

[LV] Test once if vector trip count is zero, instead of twice

2017-07-19 05:16:39 +00:00

runtime-check.ll

[LV] Test once if vector trip count is zero, instead of twice

2017-07-19 05:16:39 +00:00

runtime-limit.ll

Fix test introduced in r246187 that failed on some systems.

2015-08-27 20:43:29 +00:00

safegep.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

same-base-access.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

scalar_after_vectorization.ll

[LV] Scalarize instructions marked scalar after vectorization

2016-09-26 17:08:37 +00:00

scalar-select.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

scev-exitlim-crash.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

simple-unroll.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

small-loop.ll

[LV] Optimize for size when vectorizing loops with tiny trip count

2017-06-30 08:02:35 +00:00

start-non-zero.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

store-shuffle-bug.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

struct_access.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

tbaa-nodep.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

tripcount.ll

Improve profile-guided heuristics to use estimated trip count.

2017-06-19 18:48:58 +00:00

undef-inst-bug.ll

[opaque pointer type] Add textual IR support for explicit type parameter to getelementptr instruction

2015-02-27 19:29:02 +00:00

unroll_novec.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

unroll-novec-memcheck-metadata.ll

[LV] Add regression test for r297610

2017-03-23 20:02:23 +00:00

unroll.ll

[X86] Disable loop unrolling in loop vectorization pass when VF is 1.

2015-05-06 17:12:25 +00:00

unsafe-dep-remark.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

unsized-pointee-crash.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

value-ptr-bug.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

vect.omp.persistence.ll

[tests] Cleanup vect.omp.persistence.ll test.

2017-07-25 10:35:16 +00:00

vect.stats.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

vector-geps.ll

Reapply r298620: [LV] Vectorize GEPs

2017-04-07 14:15:34 +00:00

vectorize-once.ll

[Loop Vectorize] Added a separate metadata

2017-08-20 10:32:41 +00:00

version-mem-access.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

write-only.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00

zero-sized-pointee-crash.ll

[LV] Remove triples from target-independent vectorizer tests. NFC.

2016-10-06 23:57:25 +00:00