Now that we legalize by widening, the element types here won't change. Previously these were modeled as the elements being widened and then the instruction might become an AND or SHL/ASHR pair. But now they'll become something like a ZERO_EXTEND_VECTOR_INREG/SIGN_EXTEND_VECTOR_INREG.
For AVX2, when the destination type is legal its clear the cost should be 1 since we have extend instructions that can produce 256 bit vectors from less than 128 bit vectors. I'm a little less sure about AVX1 costs, but I think the ones I changed were definitely too high, but they might still be too high.
Differential Revision: https://reviews.llvm.org/D66169
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@368858 91177308-0d34-0410-b5e6-96231b3b80d8
The assert that caused this to be reverted should be fixed now.
Original commit message:
This patch changes our defualt legalization behavior for 16, 32, and
64 bit vectors with i8/i16/i32/i64 scalar types from promotion to
widening. For example, v8i8 will now be widened to v16i8 instead of
promoted to v8i16. This keeps the elements widths the same and pads
with undef elements. We believe this is a better legalization strategy.
But it carries some issues due to the fragmented vector ISA. For
example, i8 shifts and multiplies get widened and then later have
to be promoted/split into vXi16 vectors.
This has the potential to cause regressions so we wanted to get
it in early in the 10.0 cycle so we have plenty of time to
address them.
Next steps will be to merge tests that explicitly test the command
line option. And then we can remove the option and its associated
code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@368183 91177308-0d34-0410-b5e6-96231b3b80d8
This patch changes our defualt legalization behavior for 16, 32, and
64 bit vectors with i8/i16/i32/i64 scalar types from promotion to
widening. For example, v8i8 will now be widened to v16i8 instead of
promoted to v8i16. This keeps the elements widths the same and pads
with undef elements. We believe this is a better legalization strategy.
But it carries some issues due to the fragmented vector ISA. For
example, i8 shifts and multiplies get widened and then later have
to be promoted/split into vXi16 vectors.
This has the potential to cause regressions so we wanted to get
it in early in the 10.0 cycle so we have plenty of time to
address them.
Next steps will be to merge tests that explicitly test the command
line option. And then we can remove the option and its associated
code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367901 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
In D62801, new function attribute `willreturn` was introduced. In short, a function with `willreturn` is guaranteed to come back to the call site(more precise definition is in LangRef).
In this patch, willreturn is annotated for LLVM intrinsics.
Reviewers: jdoerfert
Reviewed By: jdoerfert
Subscribers: jvesely, nhaehnle, sstefan1, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D64904
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367184 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
- As the pointer stripping now tracks through `addrspacecast`, prepare
to handle the bit-width difference from the result pointer.
Reviewers: jdoerfert
Subscribers: jvesely, nhaehnle, hiraditya, arphaman, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D64928
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366470 91177308-0d34-0410-b5e6-96231b3b80d8
As there are some reported miscompiles with AVX512 and performance regressions
in Eigen. Verified with the original committer and testcases will be forthcoming.
This reverts commit r364964.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366154 91177308-0d34-0410-b5e6-96231b3b80d8
I'm not sure if this falls within the scope of SLP,
but we could create vector loads for some of these
patterns.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@365055 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: This patch introduces a new heuristic for guiding operand reordering. The new "look-ahead" heuristic can look beyond the immediate predecessors. This helps break ties when the immediate predecessors have identical opcodes (see lit test for an example).
Reviewers: RKSimon, ABataev, dtemirbulatov, Ayal, hfinkel, rnk
Reviewed By: RKSimon, dtemirbulatov
Subscribers: hiraditya, phosek, rnk, rcorcs, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D60897
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@364964 91177308-0d34-0410-b5e6-96231b3b80d8
This reverts r364478 (git commit 574cb0eb3a7ac95e62d223a60bef891171dfe321)
The patch is causing compilation timeouts.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@364846 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: This patch introduces a new heuristic for guiding operand reordering. The new "look-ahead" heuristic can look beyond the immediate predecessors. This helps break ties when the immediate predecessors have identical opcodes (see lit test for an example).
Reviewers: RKSimon, ABataev, dtemirbulatov, Ayal, hfinkel, rnk
Reviewed By: RKSimon, dtemirbulatov
Subscribers: rnk, rcorcs, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D60897
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@364478 91177308-0d34-0410-b5e6-96231b3b80d8
This reverts r364084 (git commit 5698921be2d567f6abf925479ac9f5a376d6d74f)
It caused crashes while compiling a file in Chrome. Reduction
forthcoming.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@364111 91177308-0d34-0410-b5e6-96231b3b80d8
This patch introduces a new heuristic for guiding operand reordering. The new "look-ahead" heuristic can look beyond the immediate predecessors. This helps break ties when the immediate predecessors have identical opcodes (see lit test for an example).
Committed on behalf of @vporpo (Vasileios Porpodas)
Differential Revision: https://reviews.llvm.org/D60897
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@364084 91177308-0d34-0410-b5e6-96231b3b80d8
This patch uses the mechanism from D62995 to strengthen the
definitions of the reduction intrinsics by letting the scalar
result/accumulator type be overloaded from the vector element type.
For example:
; The LLVM LangRef specifies that the scalar result must equal the
; vector element type, but this is not checked/enforced by LLVM.
declare i32 @llvm.experimental.vector.reduce.or.i32.v4i32(<4 x i32> %a)
This patch changes that into:
declare i32 @llvm.experimental.vector.reduce.or.v4i32(<4 x i32> %a)
Which has the type-constraint more explicit and causes LLVM to check
the result type with the vector element type.
Reviewers: RKSimon, arsenm, rnk, greened, aemerson
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D62996
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363240 91177308-0d34-0410-b5e6-96231b3b80d8
This patch fixes a regression caused by the operand reordering refactoring patch https://reviews.llvm.org/D59973 .
The fix changes the strategy to Splat instead of Opcode, if broadcast opportunities are found.
Please see the lit test for some examples.
Committed on behalf of @vporpo (Vasileios Porpodas)
Differential Revision: https://reviews.llvm.org/D62427
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@362613 91177308-0d34-0410-b5e6-96231b3b80d8
The original costs stopped at SSE42, I've added conservative estimates for everything down to SSE1/SSE2 and moved some of the SSE42 costs to SSE41 (really only the addition of PCMPGT makes any difference).
I've also added missing vXi8 costs (we use PHMINPOSUW for i8/i16 for scarily quick results) and 256-bit vector costs for AVX1.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@360528 91177308-0d34-0410-b5e6-96231b3b80d8
The code in this test is not vectorized by SLP because its operand reordering cannot look beyond the immediate predecessors.
This will get fixed in a follow-up patch that introduces the look-ahead operand reordering heuristic.
Committed on behalf of @vporpo (Vasileios Porpodas)
Differential Revision: https://reviews.llvm.org/D61283
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@359553 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: The code did not check if operand was undef before casting it to Instruction.
Reviewers: RKSimon, ABataev, dtemirbulatov
Reviewed By: ABataev
Subscribers: uabelho
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D61024
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@359136 91177308-0d34-0410-b5e6-96231b3b80d8
As it's causing some bot failures (and per request from kbarton).
This reverts commit r358543/ab70da07286e618016e78247e4a24fcb84077fda.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358546 91177308-0d34-0410-b5e6-96231b3b80d8
This is a refactoring patch which should have all the functionality of the current code. Its goal is twofold:
i. Cleanup and simplify the reordering code, and
ii. Generalize reordering so that it will work for an arbitrary number of operands, not just 2.
This is the second patch in a series of patches that will enable operand reordering across chains of operations. An example of this was presented in EuroLLVM'18 https://www.youtube.com/watch?v=gIEn34LvyNo .
Committed on behalf of @vporpo (Vasileios Porpodas)
Differential Revision: https://reviews.llvm.org/D59973
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358519 91177308-0d34-0410-b5e6-96231b3b80d8
In PR41304:
https://bugs.llvm.org/show_bug.cgi?id=41304
...we have a case where we want to fold a binop of select-shuffle (blended) values.
Rather than try to match commuted variants of the pattern, we can canonicalize the
shuffles and check for mask equality with commuted operands.
We don't produce arbitrary shuffle masks in instcombine, but select-shuffles are a
special case that the backend is required to handle because we already canonicalize
vector select to this shuffle form.
So there should be no codegen difference from this change. It's possible that this
improves CSE in IR though.
Differential Revision: https://reviews.llvm.org/D60016
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@357366 91177308-0d34-0410-b5e6-96231b3b80d8
For the cases where the icmp/fcmp predicate is commutative, use reorderInputsAccordingToOpcode to collect and commute the operands.
This requires a helper to recognise commutativity in both general Instruction and CmpInstr types - the CmpInst::isCommutative doesn't overload the Instruction::isCommutative method for reasons I'm not clear on (maybe because its based on predicate not opcode?!?).
Differential Revision: https://reviews.llvm.org/D59992
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@357266 91177308-0d34-0410-b5e6-96231b3b80d8
Some predicates are fully commutative - we should be able to easily commute their operands to improve vectorization
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@357202 91177308-0d34-0410-b5e6-96231b3b80d8
As discussed on D59738, this generalizes reorderInputsAccordingToOpcode to handle multiple + non-commutative instructions so we can get rid of reorderAltShuffleOperands and make use of the extra canonicalizations that reorderInputsAccordingToOpcode brings.
Differential Revision: https://reviews.llvm.org/D59784
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@356939 91177308-0d34-0410-b5e6-96231b3b80d8
Remove attempts to commute non-Instructions to the LHS - the codegen changes appear to rely on chance more than anything else and also have a tendency to fight existing instcombine canonicalization which moves constants to the RHS of commutable binary ops.
This is prep work towards:
(a) reusing reorderInputsAccordingToOpcode for alt-shuffles and removing the similar reorderAltShuffleOperands
(b) improving reordering to optimized cases with commutable and non-commutable instructions to still find splat/consecutive ops.
Differential Revision: https://reviews.llvm.org/D59738
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@356913 91177308-0d34-0410-b5e6-96231b3b80d8
If either the main/alt opcodes isn't commutable we may end up with the splats not correctly commuted to the same side.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@356837 91177308-0d34-0410-b5e6-96231b3b80d8
'tmpXXX' conflicts with the auto-generated script regex names.
That could cause mask a bug or fail if the output changes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@356790 91177308-0d34-0410-b5e6-96231b3b80d8
x86-64 is an invalid architecture in triples. Changing it to the correct
triple (x86_64) changes some tests, because SLP is not deemed profitable
any more.
Reviewers: ABataev, RKSimon, spatel
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D58931
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355420 91177308-0d34-0410-b5e6-96231b3b80d8
This requires a couple of tweaks to existing vectorization functions as they were assuming that only the second call argument (ctlz/cttz/powi) could ever be the 'always scalar' argument, but for smul.fix + umul.fix its the third argument.
Differential Revision: https://reviews.llvm.org/D58616
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354790 91177308-0d34-0410-b5e6-96231b3b80d8
As this has broken the lto bootstrap build for 3 days and is
showing a significant regression on the Dither_benchmark results (from
the LLVM benchmark suite) -- specifically, on the
BENCHMARK_FLOYD_DITHER_128, BENCHMARK_FLOYD_DITHER_256, and
BENCHMARK_FLOYD_DITHER_512; the others are unchanged. These have
regressed by about 28% on Skylake, 34% on Haswell, and over 40% on
Sandybridge.
This reverts commit r353923.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354434 91177308-0d34-0410-b5e6-96231b3b80d8
Sanjay Patel