Summary:
This lets us avoid creating and destroying a CallbackVH every time we
check the cache.
This is good for a 2% e2e speedup when compiling one of the large Eigen
tests at -O3.
FTR, I tried making the ValueCache hashtable one-level -- i.e., mapping
a pair (Value*, BasicBlock*) to a lattice value, and that didn't seem to
provide any additional improvement. Saving a word in LVILatticeVal by
merging the Tag and Val fields also didn't yield a speedup.
Reviewers: reames
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D21951
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276926 91177308-0d34-0410-b5e6-96231b3b80d8
This is a bit gnarly since LVI is maintaining its own cache.
I think this port could be somewhat cleaner, but I'd rather not spend
too much time on it while we still have the old pass hanging around and
limiting how much we can clean things up.
Once the old pass is gone it will be easier (less time spent) to clean
it up anyway.
This is the last dependency needed for porting JumpThreading which I'll
do in a follow-up commit (there's no printer pass for LVI or anything to
test it, so porting a pass that depends on it seems best).
I've been mostly following:
r269370 / D18834 which ported Dependence Analysis
r268601 / D19839 which ported BPI
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272593 91177308-0d34-0410-b5e6-96231b3b80d8
When encountering a non-local pointer, LVI would eagerly scan the block for dereferences of the given object to prove the pointer to be non null. That's all well and good, but *then* we'd go recurse through our input blocks. As a result, we could end up scanning each and every block we traverse, even if the final definition was obviously non null or we found a constant value somewhere up the chain. The previous code papered over this by using the isKnownNonNull routine from value tracking. This made the duplication less painful in the common case.
Instead, we know do the block scan only *after* we've gotten the recursive results back. This lets us stop scanning individual blocks as soon as we've determined it to be non-null in any predecessor block and use our usual merge rules to propagate that information cheaply through successor blocks. For a pointer which can be found non-null, this does strictly less work and sometimes substaintially so.
Note that the case where we *can't* prove something non-null is still the really expensive case. We end up scanning each and every block looking for a dereference and never end up finding one.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267642 91177308-0d34-0410-b5e6-96231b3b80d8
Previously we were recursing on our operands for unary and binary operators regardless of whether we knew how to reason about the operator in question. This has the effect of doing a potentially large amount of work, only to throw it away. By checking whether the operation is one LVI can handle, we can cut short the search and return the (overdefined) answer more quickly. The quality of the results produced should not change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267626 91177308-0d34-0410-b5e6-96231b3b80d8
As pointed out by John Regehr over in http://reviews.llvm.org/D19485, LVI was being incredibly stupid about applying its transfer rules. Rather than gathering local facts from the expression itself, it was simply giving up entirely if one of the inputs was overdefined. This greatly impacts the precision of the overall analysis and makes it far more fragile as well.
This patch builds on 267609 which did the same thing for unary casts.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267620 91177308-0d34-0410-b5e6-96231b3b80d8
Essentially, I was using the wrong size function. For types which were sized, but not primitive, I wasn't getting a useful size for the operand and failed an assert. I fixed this, and also added a guard that the input is a sized type. Test case is for the original mistake. I'm not sure how to actually exercise the sized type check.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267618 91177308-0d34-0410-b5e6-96231b3b80d8
I'll clean this up and add a test case shortly. I want to make sure this does actually fix the bots; if not, I'll revert.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267617 91177308-0d34-0410-b5e6-96231b3b80d8
As pointed out by John Regehr over in http://reviews.llvm.org/D19485, LVI was being incredibly stupid about applying its transfer rules. Rather than gathering local facts from the expression itself, it was simply giving up entirely if one of the inputs was overdefined. This greatly impacts the precision of the overall analysis and makes it far more fragile as well.
This patch implements only the unary operation case. Once this is in, I'll implement the same for the binary operations.
Differential Revision: http://reviews.llvm.org/D19492
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267609 91177308-0d34-0410-b5e6-96231b3b80d8
There has been much recent confusion about the partition in the lattice between constant and non-constant values. Hopefully, documenting this will prevent confusion going forward.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267440 91177308-0d34-0410-b5e6-96231b3b80d8
This function handled both unary and binary operators. Cloning and specializing leads to much easier to follow code with minimal duplicatation.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267438 91177308-0d34-0410-b5e6-96231b3b80d8
The diff is relatively large since I took a chance to rearrange the code I had to touch in a more obvious way, but the key bit is merely using the !range metadata when we can't analyze the instruction further. The previous !range metadata code was essentially just dead since no binary operator or cast will have !range metadata (per Verifier) and it was otherwise dropped on the floor.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262751 91177308-0d34-0410-b5e6-96231b3b80d8
Most of this is fairly straight forward. Add handling for min/max via existing matcher utility and ConstantRange routines. Add handling for clamp by exploiting condition constraints on inputs.
Note that I'm only handling two constant ranges at this point. It would be reasonable to consider treating overdefined as a full range if the instruction is typed as an integer, but that should be a separate change.
Differential Revision: http://reviews.llvm.org/D17184
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262085 91177308-0d34-0410-b5e6-96231b3b80d8
No functional change intended. Copying small (<= 64 bits) APInts isn't
expensive but bloats code by generating the slow path everywhere. Moving
doesn't care about the size of the value.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@261426 91177308-0d34-0410-b5e6-96231b3b80d8
The root issue appears to be a confusion around what makeNoWrapRegion actually does. It seems likely we need two versions of this function with slightly different semantics.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260981 91177308-0d34-0410-b5e6-96231b3b80d8
As the title says. Modelled after similar code in SCEV.
This is useful when analysing induction variables in loops which have been canonicalized by other passes. I wrote the tests as non-loops specifically to avoid the generality introduced in http://reviews.llvm.org/D17174. While that can handle many induction variables without *needing* to exploit nsw, there's no reason not to use it if we've already proven it.
Differential Revision: http://reviews.llvm.org/D17177
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260705 91177308-0d34-0410-b5e6-96231b3b80d8
This patches teaches LVI to recognize clamp idioms (e.g. select(a > 5, a, 5) will always produce something greater than 5.
The tests end up being somewhat simplistic because trying to exercise the case I actually care about (a loop with a range check on a clamped secondary induction variable) ends up tripping across a couple of other imprecisions in the analysis. Ah, the joys of LVI...
Differential Revision: http://reviews.llvm.org/D16827
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260627 91177308-0d34-0410-b5e6-96231b3b80d8
There's nothing preventing callers of LVI from asking for lattice values representing a Constant. In fact, given that several callers are walking back through PHI nodes and trying to simplify predicates, such queries are actually quite common. This is mostly harmless today, but we start volatiling assertions if we add new calls to getBlockValue in otherwise reasonable places.
Note that this change is not NFC. Specifically:
1) The result returned through getValueAt will now be more precise. In principle, this could trigger any latent infinite optimization loops in callers, but in practice, we're unlikely to see this.
2) The result returned through getBlockValueAt is potentially weakened for non-constants that were previously queried. With the old code, you had the possibility that a later query might bypass the cache and discover some information the original query did not. I can't find a scenario which actually causes this to happen, but it was in principle possible. On the other hand, this may end up reducing compile time when the same value is queried repeatedly.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260439 91177308-0d34-0410-b5e6-96231b3b80d8
Due to staleness in a patch I committed yesterday, the debug output was reporting overdefined cases as being undefined. Confusing to say the least. The mistake appears to have only effected the debug output thankfully.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@259594 91177308-0d34-0410-b5e6-96231b3b80d8
I introduced a declaration in 259583 to keep the diff readable. This change just moves the definition up to remove the declaration again.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@259585 91177308-0d34-0410-b5e6-96231b3b80d8
This patch uses the newly introduced 'intersect' utility (from 259461: [LVI] Introduce an intersect operation on lattice values) to simplify existing code in LVI.
While not introducing any new concepts, this change is probably not NFC. The common 'intersect' function is more powerful that the ad-hoc implementations we'd had in a couple of places. Given that, we may see optimizations triggering a bit more often.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@259583 91177308-0d34-0410-b5e6-96231b3b80d8
LVI has several separate sources of facts - edge local conditions, recursive queries, assumes, and control independent value facts - which all apply to the same value at the same location. The existing implementation was very conservative about exploiting all of these facts at once.
This change introduces an "intersect" function specifically to abstract the action of picking a good set of facts from all of the separate facts given. At the moment, this function is relatively simple (i.e. mostly just reuses the bits which were already there), but even the minor additions reveal the inherent power. For example, JumpThreading is now capable of doing an inductive proof that a particular value is always positive and removing a half range check.
I'm currently only using the new intersect function in one place. If folks are happy with the direction of the work, I plan on making a series of small changes without review to replace mergeIn with intersect at all the appropriate places.
Differential Revision: http://reviews.llvm.org/D14476
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@259461 91177308-0d34-0410-b5e6-96231b3b80d8
This routine was returning Undefined for most queries. This was utterly wrong. Amusingly, we do not appear to have any callers of this which are actually trying to exploit unreachable code or this would have broken the world.
A better approach would be to explicit describe the intersection of facts. That's blocked behind http://reviews.llvm.org/D14476 and I wanted to fix the current bug.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@259446 91177308-0d34-0410-b5e6-96231b3b80d8
I'll submit a test case shortly which covers this, but it's causing clang self host problems in the builders so I wanted to get it removed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@259432 91177308-0d34-0410-b5e6-96231b3b80d8
Teach LVI to handle select instructions in the exact same way it handles PHI nodes. This is useful since various parts of the optimizer convert PHI nodes into selects and we don't want these transformations to cause inferior optimization.
Note that this patch does nothing to exploit the implied constraint on the inputs represented by the select condition itself. That will be a later patch and is blocked on http://reviews.llvm.org/D14476
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@259429 91177308-0d34-0410-b5e6-96231b3b80d8
reduce memory usage.
Previously, LazyValueInfoCache inserted overdefined lattice values into
both ValueCache and OverDefinedCache. This wasn't necessary and was
causing LazyValueInfo to use an excessive amount of memory in some cases.
This patch changes LazyValueInfoCache to insert overdefined values only
into OverDefinedCache. The memory usage decreases by 70 to 75% when one
of the files in llvm is compiled.
rdar://problem/11388615
Differential revision: http://reviews.llvm.org/D15391
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@255320 91177308-0d34-0410-b5e6-96231b3b80d8
Somewhat shockingly for an analysis pass which is computing constant ranges, LVI did not understand the ranges provided by range metadata.
As part of this change, I included a change to CVP primarily because doing so made it much easier to write small self contained test cases. CVP was previously only handling the non-local operand case, but given that LVI can sometimes figure out information about instructions standalone, I don't see any reason to restrict this. There could possibly be a compile time impact from this, but I suspect it should be minimal. If anyone has an example which substaintially regresses, please let me know. I could restrict the block local handling to ICmps feeding Terminator instructions if needed.
Note that this patch continues a somewhat bad practice in LVI. In many cases, we know facts about values, and separate context sensitive facts about values. LVI makes no effort to distinguish and will frequently cache the same value fact repeatedly for different contexts. I would like to change this, but that's a large enough change that I want it to go in separately with clear documentation of what's changing. Other examples of this include the non-null handling, and arguments.
As a meta comment: the entire motivation of this change was being able to write smaller (aka reasonable sized) test cases for a future patch teaching LVI about select instructions.
Differential Revision: http://reviews.llvm.org/D13543
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@251606 91177308-0d34-0410-b5e6-96231b3b80d8
Currently LazyValueInfo will report only alloca's as having nonnull range.
For loads with !nonnull metadata it will bailout with no additional information.
Same is true for calls returning nonnull pointers.
This change extends LazyValueInfo to handle additional nonnull instructions.
Differential Revision: http://reviews.llvm.org/D12932
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@247985 91177308-0d34-0410-b5e6-96231b3b80d8
If asked to prove a predicate about a value produced by a PHI node, LazyValueInfo was unable to do so even if the predicate was known to be true for each input to the PHI. This prevented JumpThreading from eliminating a provably redundant branch.
The problematic test case looks something like this:
ListNode *p = ...;
while (p != null) {
if (!p) return;
x = g->x; // unrelated
p = p->next
}
The null check at the top of the loop is redundant since the value of 'p' is null checked on entry to the loop and before executing the backedge. This resulted in us a) executing an extra null check per iteration and b) not being able to LICM unrelated loads after the check since we couldn't prove they would execute or that their dereferenceability wasn't effected by the null check on the first iteration.
Differential Revision: http://reviews.llvm.org/D12383
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246465 91177308-0d34-0410-b5e6-96231b3b80d8
Historically there seems to be some resistance regarding the change to DenseMap
(r147980). However, I couldn't find cases of iterator invalidation for
ValueCacheEntryTy, but only for ValueCache, which I left untouched.
This reduces 20s on an internal testcase. Follow up from r245309.
Differential Revision: http://reviews.llvm.org/D11651
rdar://problem/21320066
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@245314 91177308-0d34-0410-b5e6-96231b3b80d8
Changes in LoopUnroll in the past six months exposed scalability
issues in LazyValueInfo when used from JumpThreading. One internal test
that used to take 20s under -O2 now takes 6min.
This commit change the OverDefinedCache from
DenseSet<std::pair<AssertingVH<BasicBlock>, Value*>> to
DenseMap<AssertingVH<BasicBlock>, SmallPtrSet<Value *, 4>>
and reduces compile time down to 1m40s.
Differential Revision: http://reviews.llvm.org/D11651
rdar://problem/21320066
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@245309 91177308-0d34-0410-b5e6-96231b3b80d8
