The branch folding pass tail merges blocks into a common-tail. However, the
tail retains the debug information from one of the original inputs to the
merge (chosen randomly). This is a problem for sampled-based PGO, as hits
on the common-tail will be attributed to whichever block was chosen,
irrespective of which path was actually taken to the common-tail.
This patch fixes the issue by nulling the debug location for the common-tail.
Differential Revision: https://reviews.llvm.org/D25742
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@285093 91177308-0d34-0410-b5e6-96231b3b80d8
Branch folder removes implicit defs if they are the only non-branching
instructions in a block, and the branches do not use the defined registers.
The problem is that in some cases these implicit defs are required for
the liveness information to be correct.
Differential Revision: https://reviews.llvm.org/D25478
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284036 91177308-0d34-0410-b5e6-96231b3b80d8
When branching to a block that immediately tail calls, it is possible to fold
the call directly into the branch if the call is direct and there is no stack
adjustment, saving one byte.
Example:
define void @f(i32 %x, i32 %y) {
entry:
%p = icmp eq i32 %x, %y
br i1 %p, label %bb1, label %bb2
bb1:
tail call void @foo()
ret void
bb2:
tail call void @bar()
ret void
}
before:
f:
movl 4(%esp), %eax
cmpl 8(%esp), %eax
jne .LBB0_2
jmp foo
.LBB0_2:
jmp bar
after:
f:
movl 4(%esp), %eax
cmpl 8(%esp), %eax
jne bar
.LBB0_1:
jmp foo
I don't expect any significant size savings from this (on a Clang bootstrap I
saw 288 bytes), but it does make the code a little tighter.
This patch only does 32-bit, but 64-bit would work similarly.
Differential Revision: https://reviews.llvm.org/D24108
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280832 91177308-0d34-0410-b5e6-96231b3b80d8
This is prep work for allowing the threshold to be different during layout,
and to enforce a single threshold between merging and duplicating during
layout. No observable change intended.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279117 91177308-0d34-0410-b5e6-96231b3b80d8
Fixed a bug in the test case.
To fix PR28104, this patch restricts tail merging to blocks that belong to the
same loop after MBP.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278575 91177308-0d34-0410-b5e6-96231b3b80d8
This change makes it possible for tail-duplication and tail-merging to
be disjoint. By being less aggressive when merging during layout, there are no
overlapping cases between tail-duplication and tail-merging, provided the
thresholds are disjoint.
There is a remaining TODO to benchmark the succ_size() test for non-layout tail
merging.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278265 91177308-0d34-0410-b5e6-96231b3b80d8
Use LivePhysRegs with a backwards walking algorithm to update live in
lists, this way the results do not depend on the presence of kill flags
anymore.
This patch also reduces the number of registers added as live-in.
Previously all pristine registers as well as all sub registers of a
super register were added resulting in unnecessarily large live in
lists. This fixed https://llvm.org/PR25263.
Differential Revision: http://reviews.llvm.org/D22027
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275201 91177308-0d34-0410-b5e6-96231b3b80d8
Blocks to be tail-merged may share more than one successor. Correct the
comment to state that they share a specific successor, SuccBB, rather
than a single successor, which is not true.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275104 91177308-0d34-0410-b5e6-96231b3b80d8
Tail merge was making the assumption that a layout successor or
predecessor was always a cfg successor/predecessor. Remove that
assumption. Changes to tests are necessary because the errant cfg edges
were preventing optimizations.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@273700 91177308-0d34-0410-b5e6-96231b3b80d8
This reapplies commit r271930, r271915, r271923. They hit a bug in
Thumb which is fixed in r272258 now.
The original message:
The code layout that TailMerging (inside BranchFolding) works on is not the
final layout optimized based on the branch probability. Generally, after
BlockPlacement, many new merging opportunities emerge.
This patch calls Tail Merging after MBP and calls MBP again if Tail Merging
merges anything.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272267 91177308-0d34-0410-b5e6-96231b3b80d8
The code layout that TailMerging (inside BranchFolding) works on is not the
final layout optimized based on the branch probability. Generally, after
BlockPlacement, many new merging opportunities emerge.
This patch calls Tail Merging after MBP and calls MBP again if Tail Merging
merges anything.
Differential Revision: http://reviews.llvm.org/D20276
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271925 91177308-0d34-0410-b5e6-96231b3b80d8
Move the register stackification and coloring passes to run very late, after
PEI, tail duplication, and most other passes. This means that all code emitted
and expanded by those passes is now exposed to these passes. This also
eliminates the need for prologue/epilogue code to be manually stackified,
which significantly simplifies the code.
This does require running LiveIntervals a second time. It's useful to think
of these late passes not as late optimization passes, but as a domain-specific
compression algorithm based on knowledge of liveness information. It's used to
compress the code after all conventional optimizations are complete, which is
why it uses LiveIntervals at a phase when actual optimization passes don't
typically need it.
Differential Revision: http://reviews.llvm.org/D20075
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269012 91177308-0d34-0410-b5e6-96231b3b80d8
Many files include Passes.h but only a fraction needs to know about the
TargetPassConfig class. Move it into an own header. Also rename
Passes.cpp to TargetPassConfig.cpp while we are at it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269011 91177308-0d34-0410-b5e6-96231b3b80d8
This patch implements a optimization bisect feature, which will allow optimizations to be selectively disabled at compile time in order to track down test failures that are caused by incorrect optimizations.
The bisection is enabled using a new command line option (-opt-bisect-limit). Individual passes that may be skipped call the OptBisect object (via an LLVMContext) to see if they should be skipped based on the bisect limit. A finer level of control (disabling individual transformations) can be managed through an addition OptBisect method, but this is not yet used.
The skip checking in this implementation is based on (and replaces) the skipOptnoneFunction check. Where that check was being called, a new call has been inserted in its place which checks the bisect limit and the optnone attribute. A new function call has been added for module and SCC passes that behaves in a similar way.
Differential Revision: http://reviews.llvm.org/D19172
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267022 91177308-0d34-0410-b5e6-96231b3b80d8
Make it obvious that the argument cannot be nullptr.
Remove an unnecessary nullptr check in initRegState.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@265511 91177308-0d34-0410-b5e6-96231b3b80d8
If the lhs is evaluated before the rhs, FuncletI's operator-> can trigger the
assert(isHandleInSync() && "invalid iterator access!");
at include/llvm/ADT/DenseMap.h:1061. (Happens e.g. when compiled with GCC 6.)
Differential Revision: http://reviews.llvm.org/D18440
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@265024 91177308-0d34-0410-b5e6-96231b3b80d8
Change MachineInstr API to prefer MachineInstr& over MachineInstr*
whenever the parameter is expected to be non-null. Slowly inching
toward being able to fix PR26753.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262149 91177308-0d34-0410-b5e6-96231b3b80d8
Change TargetInstrInfo API to take `MachineInstr&` instead of
`MachineInstr*` in the functions related to predicated instructions
(I'll try to come back later and get some of the rest). All of these
functions require non-null parameters already, so references are more
clear. As a bonus, this happens to factor away a host of implicit
iterator => pointer conversions.
No functionality change intended.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@261605 91177308-0d34-0410-b5e6-96231b3b80d8
This is a recommit of r257253 which was reverted in r257270.
Previous testcase can make failure on some targets due to using opt with O3 option.
Original Summary:
Merge MBBICommon and MBBI's MMOs.
Differential Revision: http://reviews.llvm.org/D15990
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257317 91177308-0d34-0410-b5e6-96231b3b80d8
Move the logic from BranchFolding to use the shared infrastructure for merging MMOs introduced in 256909. This has the effect of making BranchFolding more capable.
In the process, fix a latent bug. The existing handling for merging didn't handle the case where one of the instructions being merged had overflowed and dropped MemRefs. This was a latent bug in the places the code was commoned from, but potentially reachable in BranchFolding.
Once this is in, we're left with a single place to consider implementing MMO unique-ing as proposed in http://reviews.llvm.org/D15230.
Differential Revision: http://reviews.llvm.org/D15913
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@256966 91177308-0d34-0410-b5e6-96231b3b80d8
Clarify a comment about what it means to drop memory operands from an instruction. While I'm adding change the name of the method slightly to make it a bit more clear what's going on when reading calling code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@256346 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
If we remove the MMOs from Load/Store instructions,
they are treated as volatile. This makes other optimization passes unhappy.
eg. Load/Store Optimization
So, it looks better to merge, not remove.
Reviewers: gberry, mcrosier
Subscribers: gberry, llvm-commits
Differential Revision: http://reviews.llvm.org/D14797
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254694 91177308-0d34-0410-b5e6-96231b3b80d8
(This is the second attempt to submit this patch. The first caused two assertion
failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687)
The patch in http://reviews.llvm.org/D13745 is broken into four parts:
1. New interfaces without functional changes (http://reviews.llvm.org/D13908).
2. Use new interfaces in SelectionDAG, while in other passes treat probabilities
as weights (http://reviews.llvm.org/D14361).
3. Use new interfaces in all other passes.
4. Remove old interfaces.
This patch is 3+4 above. In this patch, MBB won't provide weight-based
interfaces any more, which are totally replaced by probability-based ones.
The interface addSuccessor() is redesigned so that the default probability is
unknown. We allow unknown probabilities but don't allow using it together
with known probabilities in successor list. That is to say, we either have a
list of successors with all known probabilities, or all unknown
probabilities. In the latter case, we assume each successor has 1/N
probability where N is the number of successors. An assertion checks if the
user is attempting to add a successor with the disallowed mixed use as stated
above. This can help us catch many misuses.
All uses of weight-based interfaces are now updated to use probability-based
ones.
Differential revision: http://reviews.llvm.org/D14973
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254377 91177308-0d34-0410-b5e6-96231b3b80d8
and the follow-up r254356: "Fix a bug in MachineBlockPlacement that may cause assertion failure during BranchProbability construction."
Asserts were firing in Chromium builds. See PR25687.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254366 91177308-0d34-0410-b5e6-96231b3b80d8
The patch in http://reviews.llvm.org/D13745 is broken into four parts:
1. New interfaces without functional changes (http://reviews.llvm.org/D13908).
2. Use new interfaces in SelectionDAG, while in other passes treat probabilities
as weights (http://reviews.llvm.org/D14361).
3. Use new interfaces in all other passes.
4. Remove old interfaces.
This patch is 3+4 above. In this patch, MBB won't provide weight-based
interfaces any more, which are totally replaced by probability-based ones.
The interface addSuccessor() is redesigned so that the default probability is
unknown. We allow unknown probabilities but don't allow using it together
with known probabilities in successor list. That is to say, we either have a
list of successors with all known probabilities, or all unknown
probabilities. In the latter case, we assume each successor has 1/N
probability where N is the number of successors. An assertion checks if the
user is attempting to add a successor with the disallowed mixed use as stated
above. This can help us catch many misuses.
All uses of weight-based interfaces are now updated to use probability-based
ones.
Differential revision: http://reviews.llvm.org/D14973
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254348 91177308-0d34-0410-b5e6-96231b3b80d8
Remove a few more implicit ilist iterator conversions, this time from
Analysis.cpp and BranchFolding.cpp.
I added a few overloads for `remove()` and `erase()`, which quite
naturally take pointers as well as iterators as parameters. This will
reduce the churn at least in the short term, but I don't really have a
problem with these existing for longer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249867 91177308-0d34-0410-b5e6-96231b3b80d8
Track which basic blocks belong to which funclets. Permit branch
folding to fire but only if it can prove that doing so will not cause
code in one funclet to be reused in another.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249257 91177308-0d34-0410-b5e6-96231b3b80d8
