Michael Kuperstein e5b8fbc43a [SLPVectorizer] Vectorize reverse-order loads in horizontal reductions ...

When vectorizing a tree rooted at a store bundle, we currently try to sort the
stores before building the tree, so that the stores can be vectorized. For other
trees, the order of the root bundle - which determines the order of all other
bundles - is arbitrary. That is bad, since if a leaf bundle of consecutive loads
happens to appear in the wrong order, we will not vectorize it.

This is partially mitigated when the root is a binary operator, by trying to
build a "reversed" tree when that's considered profitable. This patch extends the
workaround we have for binops to trees rooted in a horizontal reduction.

This fixes PR28474.

Differential Revision: https://reviews.llvm.org/D22554


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276477 91177308-0d34-0410-b5e6-96231b3b80d8

2016-07-22 21:28:48 +00:00

..

addsub.ll

[opaque pointer type] Add textual IR support for explicit type parameter to gep operator

2015-03-13 18:20:45 +00:00

align.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

atomics.ll

[opaque pointer type] Add textual IR support for explicit type parameter to gep operator

2015-03-13 18:20:45 +00:00

bad_types.ll

[SLP] Don't vectorize loads of non-packed types (like i1, i2).

2015-09-30 21:05:43 +00:00

barriercall.ll

[opaque pointer type] Add textual IR support for explicit type parameter to the call instruction

2015-04-16 23:24:18 +00:00

bitreverse.ll

[CostModel][X86][SSE] Updated costs for vector BITREVERSE ops on SSSE3+ targets

2016-06-11 19:23:02 +00:00

bswap.ll

[X86][SSE] Add cost model for BSWAP of vectors

2016-06-20 23:08:21 +00:00

call.ll

[SLPVectorizer] Vectorizing the libm sqrt to llvm's sqrt intrinsic requires nnan

2016-04-06 07:04:53 +00:00

cast.ll

[SLPVectorizer][X86] Regenerated SEXT/ZEXT cast vectorization tests

2016-05-06 22:22:18 +00:00

cmp_sel.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

commutativity.ll

Fix SLPVectorizer commutativity reordering

2015-11-06 20:17:51 +00:00

compare-reduce.ll

[opaque pointer type] Add textual IR support for explicit type parameter to the call instruction

2015-04-16 23:24:18 +00:00

consecutive-access.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

continue_vectorizing.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

crash_7zip.ll

[opaque pointer type] Add textual IR support for explicit type parameter to getelementptr instruction

2015-02-27 19:29:02 +00:00

crash_binaryop.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

crash_bullet3.ll

[opaque pointer type] Add textual IR support for explicit type parameter to getelementptr instruction

2015-02-27 19:29:02 +00:00

crash_bullet.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

crash_cmpop.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

crash_dequeue.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

crash_flop7.ll

SLPVectorizer: Implement DCE as part of vectorization.

2013-07-07 06:57:07 +00:00

crash_gep.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

crash_lencod.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

crash_mandeltext.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

crash_netbsd_decompress.ll

[opaque pointer type] Add textual IR support for explicit type parameter to gep operator

2015-03-13 18:20:45 +00:00

crash_scheduling.ll

[opaque pointer type] Add textual IR support for explicit type parameter to getelementptr instruction

2015-02-27 19:29:02 +00:00

crash_sim4b1.ll

[opaque pointer type] Add textual IR support for explicit type parameter to getelementptr instruction

2015-02-27 19:29:02 +00:00

crash_smallpt.ll

[opaque pointer type] Add textual IR support for explicit type parameter to getelementptr instruction

2015-02-27 19:29:02 +00:00

crash_vectorizeTree.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

cross_block_slp.ll

[opaque pointer type] Add textual IR support for explicit type parameter to the call instruction

2015-04-16 23:24:18 +00:00

cse.ll

[SLPVectorizer] Try different vectorization factors for store chains

2015-07-08 23:40:55 +00:00

ctlz.ll

[SLPVectorizer][X86] Added CTPOP/CTLZ/CTTZ vectorization tests

2016-05-06 21:33:01 +00:00

ctpop.ll

[X86][SSE] Add cost model values for CTPOP of vectors

2016-07-20 10:41:28 +00:00

cttz.ll

[SLPVectorizer][X86] Added CTPOP/CTLZ/CTTZ vectorization tests

2016-05-06 21:33:01 +00:00

cycle_dup.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

debug_info.ll

[PR27284] Reverse the ownership between DICompileUnit and DISubprogram.

2016-04-15 15:57:41 +00:00

diamond.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

external_user.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

extract_in_tree_user.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

extract.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

extractcost.ll

[opaque pointer type] Add textual IR support for explicit type parameter to getelementptr instruction

2015-02-27 19:29:02 +00:00

flag.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

fma.ll

[SLPVectorizer][X86] Added fma vectorization tests

2016-07-08 17:19:13 +00:00

fround.ll

[SLPVectorizer][X86] Added ceil/floor/nearbyint/rint/trunc vectorization tests

2016-06-22 14:07:46 +00:00

funclet.ll

[SLPVectorizer] Add operand bundles to vectorized functions

2016-04-29 07:09:51 +00:00

gep_mismatch.ll

[SLPVectorizer] Handle GEP with differing constant index types

2016-06-08 21:55:16 +00:00

gep.ll

[PM] Port SLPVectorizer to the new PM

2016-06-15 08:43:40 +00:00

hoist.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

horizontal.ll

[NFC] Update horizontal reduction test cases.

2015-12-16 17:22:24 +00:00

implicitfloat.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

in-tree-user.ll

[opaque pointer type] Add textual IR support for explicit type parameter to the call instruction

2015-04-16 23:24:18 +00:00

insert-element-build-vector.ll

[SLPVectorize] Basic ephemeral-value awareness

2014-10-15 17:35:01 +00:00

insertvalue.ll

[SLPVectorizer] Extend SLP Vectorizer to deal with aggregates.

2016-04-28 16:11:45 +00:00

intrinsic.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

lit.local.cfg

Reduce verbiage of lit.local.cfg files

2014-06-09 22:42:55 +00:00

long_chains.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

loopinvariant.ll

[SLPVectorizer] Try different vectorization factors for store chains

2015-07-08 23:40:55 +00:00

metadata.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

multi_block.ll

[opaque pointer type] Add textual IR support for explicit type parameter to the call instruction

2015-04-16 23:24:18 +00:00

multi_user.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

non-vectorizable-intrinsic.ll

SLPVectorizer: Only vectorize intrinsics whose operands are widened equally

2014-04-09 14:20:47 +00:00

odd_store.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

operandorder.ll

[opaque pointer type] Add textual IR support for explicit type parameter to gep operator

2015-03-13 18:20:45 +00:00

opt.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

ordering.ll

Fix tests that used CHECK-NEXT-NOT and CHECK-DAG-NOT.

2016-02-26 19:40:34 +00:00

phi3.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

phi_landingpad.ll

Move the personality function from LandingPadInst to Function

2015-06-17 20:52:32 +00:00

phi_overalignedtype.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

phi.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

powof2div.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

pr16571.ll

Fix PR16571, which is a bug in the code that checks that all of the types in the bundle are uniform.

2013-07-09 21:38:08 +00:00

pr16628.ll

[opaque pointer type] Add textual IR support for explicit type parameter to the call instruction

2015-04-16 23:24:18 +00:00

pr16899.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

pr18060.ll

PR1860 - We can't save a list of ExtractElement instructions to CSE because some of these instructions

2013-11-26 22:24:25 +00:00

pr19657.ll

[SLPVectorizer] Try different vectorization factors for store chains

2015-07-08 23:40:55 +00:00

pr23510.ll

[SLP vectorizer]: Choose the best consecutive candidate to pair with a store instruction.

2015-07-30 17:40:39 +00:00

pr27163.ll

[SLPVectorizer] Don't insert an extractelement before a catchswitch

2016-04-01 17:28:15 +00:00

propagate_ir_flags.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

reduction2.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

reduction_loads.ll

[SLPVectorizer] Vectorize reverse-order loads in horizontal reductions

2016-07-22 21:28:48 +00:00

reduction.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

return.ll

[opaque pointer type] Add textual IR support for explicit type parameter to gep operator

2015-03-13 18:20:45 +00:00

rgb_phi.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

saxpy.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

schedule_budget.ll

SLPVectorizer: add a test to check if the minimum region size works.

2015-09-30 17:28:19 +00:00

scheduling.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

simple-loop.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

simplebb.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

sqrt.ll

[SLPVectorizer][X86] Added sqrt vectorization tests

2016-07-18 13:20:54 +00:00

tiny-tree.ll

[SLP] Vectorize for all-constant entries.

2015-06-19 17:40:15 +00:00

unreachable.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

value-bug.ll

SLPVectorizer: Bring back the insertelement patch (r205965) with fixes

2014-05-04 17:10:15 +00:00

vector_gep.ll

Vector GEP test: renamed + some comments

2016-07-06 08:11:23 +00:00

vector.ll

Fix PR15800. Do not try to vectorize vectors and structs.

2013-04-20 22:29:43 +00:00