Convert vector increment or decrement to sub/add with an all-ones constant:
add X, <1, 1...> --> sub X, <-1, -1...>
sub X, <1, 1...> --> add X, <-1, -1...>
The all-ones vector constant can be materialized using a pcmpeq instruction that is
commonly recognized as an idiom (has no register dependency), so that's better than
loading a splat 1 constant.
AVX512 uses 'vpternlogd' for 512-bit vectors because there is apparently no better
way to produce 512 one-bits.
The general advantages of this lowering are:
1. pcmpeq has lower latency than a memop on every uarch I looked at in Agner's tables,
so in theory, this could be better for perf, but...
2. That seems unlikely to affect any OOO implementation, and I can't measure any real
perf difference from this transform on Haswell or Jaguar, but...
3. It doesn't look like it from the diffs, but this is an overall size win because we
eliminate 16 - 64 constant bytes in the case of a vector load. If we're broadcasting
a scalar load (which might itself be a bug), then we're replacing a scalar constant
load + broadcast with a single cheap op, so that should always be smaller/better too.
4. This makes the DAG/isel output more consistent - we use pcmpeq already for padd x, -1
and psub x, -1, so we should use that form for +1 too because we can. If there's some
reason to favor a constant load on some CPU, let's make the reverse transform for all
of these cases (either here in the DAG or in a later machine pass).
This should fix:
https://bugs.llvm.org/show_bug.cgi?id=33483
Differential Revision: https://reviews.llvm.org/D34336
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306289 91177308-0d34-0410-b5e6-96231b3b80d8
Csmith discovered that this function can be called with a zero argument,
in which case an assert for this triggered.
This patch also adds a guard before the other call to this function since
it was missing, although the test only covers the case where it was
discovered.
Reduced test case attached as CodeGen/SystemZ/int-cmp-54.ll.
Review: Ulrich Weigand
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306287 91177308-0d34-0410-b5e6-96231b3b80d8
As noted on PR32585, the change in D29780/rL295325 resulted in calls to Rand32() (values 0 -> 0xFFFFFFFF) but the min()/max() operators indicated it would be (0 -> 0x7FFFF).
This patch changes the random operator to call Rand() instead which does respect the 0 -> 0x7FFFF range and asserts that the value is in range as well.
Differential Revision: https://reviews.llvm.org/D34089
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306281 91177308-0d34-0410-b5e6-96231b3b80d8
This method doesn't do any initializing. It just contains asserts. So renaming to AssertOK makes it consistent with similar instructions in other Instruction classes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306277 91177308-0d34-0410-b5e6-96231b3b80d8
Revert "[MBP] do not rotate loop if it creates extra branch"
It breaks the sanitizer build bots. Need to fix this.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306276 91177308-0d34-0410-b5e6-96231b3b80d8
We added the initilization of disassembler passes in r306208 with the goal to
bring bugpoint in line with 'opt'. However, 'opt' does itself not initialize
dissassembler passes. As our goal was consistency, we drop the initialization
of dissassembler passes again from bugpoint.
Thanks to Chandler for pointing this out!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306275 91177308-0d34-0410-b5e6-96231b3b80d8
This is a last fix for the corner case of PR32214. Actually this is not really corner case in general.
We should not do a loop rotation if we create an additional branch due to it.
Consider the case where we have a loop chain H, M, B, C , where
H is header with viable fallthrough from pre-header and exit from the loop
M - some middle block
B - backedge to Header but with exit from the loop also.
C - some cold block of the loop.
Let's H is determined as a best exit. If we do a loop rotation M, B, C, H we can introduce the extra branch.
Let's compute the change in number of branches:
+1 branch from pre-header to header
-1 branch from header to exit
+1 branch from header to middle block if there is such
-1 branch from cold bock to header if there is one
So if C is not a predecessor of H then we introduce extra branch.
This change actually prohibits rotation of the loop if both true
1) Best Exit has next element in chain as successor.
2) Last element in chain is not a predecessor of first element of chain.
Reviewers: iteratee, xur
Reviewed By: iteratee
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D34271
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306272 91177308-0d34-0410-b5e6-96231b3b80d8
metadata out of InstCombine and into helpers.
NFC, this just exposes the logic used by InstCombine when propagating
metadata from one load instruction to another. The plan is to use this
in SROA to address PR32902.
If anyone has better ideas about how to factor this or name variables,
I'm all ears, but this seemed like a pretty good start and lets us make
progress on the PR.
This is based on a patch by Ariel Ben-Yehuda (D34285).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306267 91177308-0d34-0410-b5e6-96231b3b80d8
This should not be treated as a different version of
private_segment_buffer. These are distinct things with
different uses and register classes, and requires the
function argument info to have more context about the
function's type and environment.
Also add missing test coverage for the intrinsic, and
emit an error for HSA. This also encovers that the intrinsic
is broken unless there happen to be stack objects.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306264 91177308-0d34-0410-b5e6-96231b3b80d8
This was reverted in r306252, but I already had the bug fixed and was
just trying to form a test case.
The original commit factored the logic for forming dedicated exits
inside of LoopSimplify into a helper that could be used elsewhere and
with an approach that required fewer intermediate data structures. See
that commit for full details including the change to the statistic, etc.
The code looked fine to me and my reviewers, but in fact didn't handle
indirectbr correctly -- it left the 'InLoopPredecessors' vector dirty.
If you have code that looks *just* right, you can end up leaking these
predecessors into a subsequent rewrite, and crash deep down when trying
to update PHI nodes for predecessors that don't exist.
I've added an assert that makes the bug much more obvious, and then
changed the code to reliably clear the vector so we don't get this bug
again in some other form as the code changes.
I've also added a test case that *does* manage to catch this while also
giving some nice positive coverage in the face of indirectbr.
The real code that found this came out of what I think is CPython's
interpreter loop, but any code with really "creative" interpreter loops
mixing indirectbr and other exit paths could manage to tickle the bug.
I was hard to reduce the original test case because in addition to
having a particular pattern of IR, the whole thing depends on the order
of the predecessors which is in turn depends on use list order. The test
case added here was designed so that in multiple different predecessor
orderings it should always end up going down the same path and tripping
the same bug. I hope. At least, it tripped it for me without
manipulating the use list order which is better than anything bugpoint
could do...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306257 91177308-0d34-0410-b5e6-96231b3b80d8
I did some basic testing while looking for a bug in my recent change to
loop simplify and even though it didn't find the bug it seems like
a useful improvement anyways.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306256 91177308-0d34-0410-b5e6-96231b3b80d8
Recommit NFC patch (rL306157) where I missed incrementing the basic block iterator,
which caused loop deletion tests to hang due to infinite loop.
Had reverted it in rL306162.
rL306157 commit message:
Currently, the implementation of delete dead loops has a special case
when the loop being deleted is never executed. This special case
(updating of exit block's incoming values for phis) can be
run as a prepass for non-executable loops before performing
the actual deletion.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306254 91177308-0d34-0410-b5e6-96231b3b80d8
This leads to a segfault. Chandler already has a test case and should be
able to recommit with a fix soon.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306252 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch does a few things that should remove some copies around PatternsToMatch. These were noticed while reviewing code for D34341.
Change constructor to take Dstregs by value and move it into the class. Change one of the callers to add std::move to the argument so that it gets moved.
Make AddPatternToMatch take PatternToMatch by rvalue reference so we can move it into the PatternsToMatch vector. I believe we should have a implicit default move constructor available on PatternToMatch. I chose rvalue reference because both callers call it with temporaries already.
Reviewers: RKSimon, aymanmus, spatel
Reviewed By: aymanmus
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D34411
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306251 91177308-0d34-0410-b5e6-96231b3b80d8
http://rise4fun.com/Alive/i8Q
A narrow bitwise logic op is obviously better than math for value tracking,
and zext is better than sext. Typically, the 'not' will be folded into an
icmp predicate.
The IR difference would even survive through codegen for x86, so we would see
worse code:
https://godbolt.org/g/C14HMF
one_or_zero(int, int): # @one_or_zero(int, int)
xorl %eax, %eax
cmpl %esi, %edi
setle %al
retq
one_or_zero_alt(int, int): # @one_or_zero_alt(int, int)
xorl %ecx, %ecx
cmpl %esi, %edi
setg %cl
movl $1, %eax
subl %ecx, %eax
retq
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306243 91177308-0d34-0410-b5e6-96231b3b80d8
The compiler fails with assertion during legalization of SETCC for <3 x i8> operands.
The result is extended to <4 x i8> and then truncated <4 x i1>. It does not happen on AVX2, because the final result of SETCC is <4 x i32>.
Differential Revision: https://reviews.llvm.org/D34503
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306242 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Make sure we are comparing the unknown instructions in the alias set and the instruction interested in.
I believe this is clearly a bug (missed opportunity). I can also add some test cases if desired.
Reviewers: hfinkel, davide, dberlin
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D34597
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306241 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Support vector type G_EXTRACT selection. For now G_EXTRACT marked as legal for any type, so nothing to do in legalizer.
Split from https://reviews.llvm.org/D33665
Reviewers: qcolombet, t.p.northover, zvi, guyblank
Reviewed By: guyblank
Subscribers: guyblank, rovka, llvm-commits, kristof.beyls
Differential Revision: https://reviews.llvm.org/D33957
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306240 91177308-0d34-0410-b5e6-96231b3b80d8
The cost of an interleaved access was only implemented for AVX512. For other
X86 targets an overly conservative Base cost was returned, resulting in
avoiding vectorization where it is actually profitable to vectorize.
This patch starts to add costs for AVX2 for most prominent cases of
interleaved accesses (stride 3,4 chars, for now).
Note1: Improvements of up to ~4x were observed in some of EEMBC's rgb
workloads; There is also a known issue of 15-30% degradations on some of these
workloads, associated with an interleaved access followed by type
promotion/widening; the resulting shuffle sequence is currently inefficient and
will be improved by a series of patches that extend the X86InterleavedAccess pass
(such as D34601 and more to follow).
Note 2: The costs in this patch do not reflect port pressure penalties which can
be very dominant in the case of interleaved accesses since most of the shuffle
operations are restricted to a single port. Further tuning, that may incorporate
these considerations, will be done on top of the upcoming improved shuffle
sequences (that is, along with the abovementioned work to extend
X86InterleavedAccess pass).
Differential Revision: https://reviews.llvm.org/D34023
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306238 91177308-0d34-0410-b5e6-96231b3b80d8
If you dump a pdb to yaml, and then round-trip it back to a pdb,
and run cvdump -l <file> on the new pdb, cvdump will generate
output such as this.
*** LINES
** Module: "d:\src\llvm\test\DebugInfo\PDB\Inputs\empty.obj"
Error: Line number corrupted: invalid file id 0
<Unknown> (MD5), 0001:00000010-0000001A, line/addr pairs = 3
5 00000010 6 00000013 7 00000018
Note the error message about the corrupted line number.
It turns out that the problem is that cvdump cannot find the
/names stream (e.g. the global string table), and the reason it
can't find the /names stream is because it doesn't understand
the NameMap that we serialize which tells pdb consumers which
stream has the string table.
Some experimentation shows that if we add items to the hash
table in a specific order before serializing it, cvdump can read
it. This suggests that either we're using the wrong hash function,
or we're serializing something incorrectly, but it will take some
deeper investigation to figure out how / why. For now, this at
least allows cvdump to read our line information (and incidentally,
produces an identical byte sequence to what Microsoft tools
produce when writing the named stream map).
Differential Revision: https://reviews.llvm.org/D34491
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306233 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch changes getRange to getRangeRef and returns a reference to the ConstantRange object stored inside the DenseMap caches. We then take advantage of that to add new helper methods that can return min/max value of a signed or unsigned ConstantRange using that reference without first copying the ConstantRange.
getRangeRef calls itself recursively and I believe the reference return is fine for those calls.
I've left getSignedRange and getUnsignedRange returning a ConstantRange object so they will make a copy now. This is to ensure safety since the reference will be invalidated if the DenseMap changes.
I'm sure there are still more places that can take advantage of the reference and I'll submit future patches as I find them.
Reviewers: sanjoy, davide
Reviewed By: sanjoy
Subscribers: zzheng, llvm-commits, mzolotukhin
Differential Revision: https://reviews.llvm.org/D32978
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306229 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Turns out creating matchers with combineOr isn't very efficient as we have to build matcher objects for both sides of the OR. Those objects aren't free, the trees usually contain several objects that contain a reference to a Value *, ConstantInt *, APInt * or some such thing. The compiler isn't always willing to inline all the matcher code to get rid of these member variables. Thus we end up loads and stores of these variables.
Using combineOR ends up creating two complete copies of the tree and the associated stores. I believe we're also paying for the opcode check twice.
This patch adds a commutable mode to several of the matcher objects as a bool template parameter that can be used to enable commutable support directly in the match functions of the corresponding objects. This avoids the duplicate object creation and the opcode checks.
This shows about an ~7-8k reduction in the opt binary size on my local build.
Reviewers: spatel, majnemer, davide
Reviewed By: majnemer
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D34592
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306226 91177308-0d34-0410-b5e6-96231b3b80d8
