Re-apply r276044/r279124/r305516. Fixed a problem where we would refuse
to place spills as the very first instruciton of a basic block and thus
artifically increase pressure (test in
test/CodeGen/PowerPC/scavenging.mir:spill_at_begin)
This is a variant of scavengeRegister() that works for
enterBasicBlockEnd()/backward(). The benefit of the backward mode is
that it is not affected by incomplete kill flags.
This patch also changes
PrologEpilogInserter::doScavengeFrameVirtualRegs() to use the register
scavenger in backwards mode.
Differential Revision: http://reviews.llvm.org/D21885
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@305625 91177308-0d34-0410-b5e6-96231b3b80d8
Revert because of reports of some PPC input starting to spill when it
was predicted that it wouldn't and no spillslot was reserved.
This reverts commit r305516.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@305566 91177308-0d34-0410-b5e6-96231b3b80d8
Re-apply r276044/r279124. Trying to reproduce or disprove the ppc64
problems reported in the stage2 build last time, which I cannot
reproduce right now.
This is a variant of scavengeRegister() that works for
enterBasicBlockEnd()/backward(). The benefit of the backward mode is
that it is not affected by incomplete kill flags.
This patch also changes
PrologEpilogInserter::doScavengeFrameVirtualRegs() to use the register
scavenger in backwards mode.
Differential Revision: http://reviews.llvm.org/D21885
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@305516 91177308-0d34-0410-b5e6-96231b3b80d8
This pass allows to run the register scavenging independently of
PrologEpilogInserter to allow targeted testing.
Also adds some basic register scavenging tests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@304606 91177308-0d34-0410-b5e6-96231b3b80d8
These parts do not depend on any PrologEpilogInserter logic and
therefore better fits RegisterScaveging.cpp.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@304596 91177308-0d34-0410-b5e6-96231b3b80d8
1. RegisterClass::getSize() is split into two functions:
- TargetRegisterInfo::getRegSizeInBits(const TargetRegisterClass &RC) const;
- TargetRegisterInfo::getSpillSize(const TargetRegisterClass &RC) const;
2. RegisterClass::getAlignment() is replaced by:
- TargetRegisterInfo::getSpillAlignment(const TargetRegisterClass &RC) const;
This will allow making those values depend on subtarget features in the
future.
Differential Revision: https://reviews.llvm.org/D31783
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@301221 91177308-0d34-0410-b5e6-96231b3b80d8
This is a set of register units intended to track register liveness, it
is similar in spirit to LivePhysRegs.
You can also think of this as the liveness tracking parts of the
RegisterScavenger factored out into an own class.
This was proposed in http://llvm.org/PR27609
Differential Revision: http://reviews.llvm.org/D21916
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292542 91177308-0d34-0410-b5e6-96231b3b80d8
Add an assert that checks whether liveins are up to date before they are
used.
- Do not print liveins into .mir files anymore in situations where they
are out of date anyway.
- The assert in the RegisterScavenger is superseded by the new one in
livein_begin().
- Skip parts of the liveness updating logic in IfConversion.cpp when
liveness isn't tracked anymore (just enough to avoid hitting the new
assert()).
Differential Revision: https://reviews.llvm.org/D27562
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291169 91177308-0d34-0410-b5e6-96231b3b80d8
Specifically avoid implicit conversions from/to integral types to
avoid potential errors when changing the underlying type. For example,
a typical initialization of a "full" mask was "LaneMask = ~0u", which
would result in a value of 0x00000000FFFFFFFF if the type was extended
to uint64_t.
Differential Revision: https://reviews.llvm.org/D27454
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289820 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
In addition to not including the register operand of the current
instruction also don't include any aliasing registers. We can't consider
these as candidates because using them will clobber the corresponding
register operand of the current instruction.
This change doesn't include a test case and it would probably be difficult
to produce a stable one since the bug depends on the results of register
allocation.
Reviewers: MatzeB, qcolombet, hfinkel
Subscribers: hfinkel, llvm-commits
Differential Revision: https://reviews.llvm.org/D24130
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280698 91177308-0d34-0410-b5e6-96231b3b80d8
The ppc64 multistage bot fails on this.
This reverts commit r279124.
Also Revert "CodeGen: Add/Factor out LiveRegUnits class; NFCI" because it depends on the previous change
This reverts commit r279171.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279199 91177308-0d34-0410-b5e6-96231b3b80d8
This is a set of register units intended to track register liveness, it
is similar in spirit to LivePhysRegs.
You can also think of this as the liveness tracking parts of the
RegisterScavenger factored out into an own class.
This was proposed in http://llvm.org/PR27609
Differential Revision: http://reviews.llvm.org/D21916
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279171 91177308-0d34-0410-b5e6-96231b3b80d8
Re-apply r276044 with off-by-1 instruction fix for the reload placement.
This is a variant of scavengeRegister() that works for
enterBasicBlockEnd()/backward(). The benefit of the backward mode is
that it is not affected by incomplete kill flags.
This patch also changes
PrologEpilogInserter::doScavengeFrameVirtualRegs() to use the register
scavenger in backwards mode.
Differential Revision: http://reviews.llvm.org/D21885
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279124 91177308-0d34-0410-b5e6-96231b3b80d8
Reverting this commit for now as it seems to be causing failures on
test-suite tests on the clang-ppc64le-linux-lnt bot.
This reverts commit r276044.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276068 91177308-0d34-0410-b5e6-96231b3b80d8
This is a variant of scavengeRegister() that works for
enterBasicBlockEnd()/backward(). The benefit of the backward mode is
that it is not affected by incomplete kill flags.
This patch also changes
PrologEpilogInserter::doScavengeFrameVirtualRegs() to use the register
scavenger in backwards mode.
Differential Revision: http://reviews.llvm.org/D21885
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276044 91177308-0d34-0410-b5e6-96231b3b80d8
This adds two pieces:
- RegisterScavenger:::enterBasicBlockEnd() which behaves similar to
enterBasicBlock() but starts tracking at the end of the basic block.
- A RegisterScavenger::backward() method. It is subtly different
from the existing unprocess() method which only considers uses with
the kill flag set: If a value is dead at the end of a basic block with
a last use inside the basic block, unprocess() will fail to mark it as
live. However we cannot change/fix this behaviour because unprocess()
needs to perform the exact reverse operation of forward().
Differential Revision: http://reviews.llvm.org/D21873
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276043 91177308-0d34-0410-b5e6-96231b3b80d8
Prefer MachineInstr& in order to avoid implicit conversions from
MachineInstrBundleIterator to MachineInstr*.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@274888 91177308-0d34-0410-b5e6-96231b3b80d8
- Use range based for loops
- No need for some !Reg checks: isPhysicalRegister() reports false for
NoRegister anyway
- Do not repeat function name in documentation comment.
- Do not repeat documentation comment in implementation when we already
have one at the declaration.
- Factor some common subexpressions out.
- Change file comments to use doxygen syntax.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@274194 91177308-0d34-0410-b5e6-96231b3b80d8
Depending on the compiler used to build LLVM, llvm_unreachable can either
expand to a call to abort(), or to a __builtin_unreachable. The latter
does not have a predictable behavior at runtime.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270260 91177308-0d34-0410-b5e6-96231b3b80d8
- Do not store Twine objects.
- Remove report_fatal_error, since llvm_unreachable does terminate the
program in release mode.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270233 91177308-0d34-0410-b5e6-96231b3b80d8
When looking for an available spill slot, the register scavenger would stop
after finding the first one with no register assigned to it. That slot may
have size and alignment that do not meet the requirements of the register
that is to be spilled. Instead, find an available slot that is the closest
in size and alignment to one that is needed to spill a register from RC.
Differential Revision: http://reviews.llvm.org/D20295
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269969 91177308-0d34-0410-b5e6-96231b3b80d8
Make it obvious that the argument cannot be nullptr.
Remove an unnecessary nullptr check in initRegState.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@265511 91177308-0d34-0410-b5e6-96231b3b80d8
With subregister liveness enabled we can detect the case where only
parts of a register are live in, this is expressed as a 32bit lanemask.
The current code only keeps registers in the live-in list and therefore
enumerated all subregisters affected by the lanemask. This turned out to
be too conservative as the subregister may also cover additional parts
of the lanemask which are not live. Expressing a given lanemask by
enumerating a minimum set of subregisters is computationally expensive
so the best solution is to simply change the live-in list to store the
lanemasks as well. This will reduce memory usage for targets using
subregister liveness and slightly increase it for other targets
Differential Revision: http://reviews.llvm.org/D12442
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@247171 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
The RegisterScavenger explicitly ignores <kill> flags on operands of
predicated instructions and therefore assumes that such registers remain
live. When it then scavenges such a register, it inserts a spill of this
(killed) register. This is invalid code and gets flagged up by the
verifier.
Nowadays kill flags are set correctly on predicated instructions. This
patch makes the Scavenger respect them.
The bug has so far only been triggered by an internal pass, so I don't
have a test case unfortunately.
Fixes PR23119.
Reviewers: hfinkel, tobiasvk_caf
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D9039
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@239439 91177308-0d34-0410-b5e6-96231b3b80d8
About pristine regsiters:
Pristine registers "hold a value that is useless to the current
function, but that must be preserved - they are callee saved registers
that have not been saved." This concept saves compile time as it frees
the prologue/epilogue inserter from adding every such register to every
basic blocks live-in list.
However the current code in getPristineRegs is formulated in a
complicated way: Inside the function prologue and epilogue all callee
saves are considered pristine, while in the rest of the code only the
non-saved ones are considered pristine. This requires logic to
differentiate between prologue/epilogue and the rest and in the presence
of shrink-wrapping this even becomes complicated/expensive. It's also
unnecessary because the prologue epilogue inserters already mark
callee-save registers that are saved/restores properly in the respective
blocks in the prologue/epilogue (see updateLiveness() in
PrologueEpilogueInserter.cpp). So only declaring non-saved/restored
callee saved registers as pristine just works.
Differential Revision: http://reviews.llvm.org/D10101
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238524 91177308-0d34-0410-b5e6-96231b3b80d8
define below all header includes in the lib/CodeGen/... tree. While the
current modules implementation doesn't check for this kind of ODR
violation yet, it is likely to grow support for it in the future. It
also removes one layer of macro pollution across all the included
headers.
Other sub-trees will follow.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206837 91177308-0d34-0410-b5e6-96231b3b80d8
When computing currently-live registers, the register scavenger excludes undef
uses. As a result, undef uses are ignored when computing the restore points of
registers spilled into the emergency slots. While the register scavenger
normally excludes from consideration, when scavenging, registers used by the
current instruction, we need to not exclude undef uses. Otherwise, we might end
up requiring more emergency spill slots than we have (in the case where the
undef use *is* the currently-spilled register).
Another bug found by llvm-stress.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186067 91177308-0d34-0410-b5e6-96231b3b80d8
This fixes PEI as previously described, but correctly handles the case where
the instruction defining the virtual register to be scavenged is the first in
the block. Arnold provided me with a bugpoint-reduced test case, but even that
seems too large to use as a regression test. If I'm successful in cleaning it
up then I'll commit that as well.
Original commit message:
This change fixes a bug that I introduced in r178058. After a register is
scavenged using one of the available spills slots the instruction defining the
virtual register needs to be moved to after the spill code. The scavenger has
already processed the defining instruction so that registers killed by that
instruction are available for definition in that same instruction. Unfortunately,
after this, the scavenger needs to iterate through the spill code and then
visit, again, the instruction that defines the now-scavenged register. In order
to avoid confusion, the register scavenger needs the ability to 'back up'
through the spill code so that it can again process the instructions in the
appropriate order. Prior to this fix, once the scavenger reached the
just-moved instruction, it would assert if it killed any registers because,
having already processed the instruction, it believed they were undefined.
Unfortunately, I don't yet have a small test case. Thanks to Pranav Bhandarkar
for diagnosing the problem and testing this fix.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@178919 91177308-0d34-0410-b5e6-96231b3b80d8
Reverting because this breaks one of the LTO builders. Original commit message:
This change fixes a bug that I introduced in r178058. After a register is
scavenged using one of the available spills slots the instruction defining the
virtual register needs to be moved to after the spill code. The scavenger has
already processed the defining instruction so that registers killed by that
instruction are available for definition in that same instruction. Unfortunately,
after this, the scavenger needs to iterate through the spill code and then
visit, again, the instruction that defines the now-scavenged register. In order
to avoid confusion, the register scavenger needs the ability to 'back up'
through the spill code so that it can again process the instructions in the
appropriate order. Prior to this fix, once the scavenger reached the
just-moved instruction, it would assert if it killed any registers because,
having already processed the instruction, it believed they were undefined.
Unfortunately, I don't yet have a small test case. Thanks to Pranav Bhandarkar
for diagnosing the problem and testing this fix.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@178916 91177308-0d34-0410-b5e6-96231b3b80d8
This change fixes a bug that I introduced in r178058. After a register is
scavenged using one of the available spills slots the instruction defining the
virtual register needs to be moved to after the spill code. The scavenger has
already processed the defining instruction so that registers killed by that
instruction are available for definition in that same instruction. Unfortunately,
after this, the scavenger needs to iterate through the spill code and then
visit, again, the instruction that defines the now-scavenged register. In order
to avoid confusion, the register scavenger needs the ability to 'back up'
through the spill code so that it can again process the instructions in the
appropriate order. Prior to this fix, once the scavenger reached the
just-moved instruction, it would assert if it killed any registers because,
having already processed the instruction, it believed they were undefined.
Unfortunately, I don't yet have a small test case. Thanks to Pranav Bhandarkar
for diagnosing the problem and testing this fix.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@178845 91177308-0d34-0410-b5e6-96231b3b80d8
This is a follow-up to r178073 (which should actually make target-customized
spilling work again).
I still don't have a regression test for this (but it would be good to have
one; Thumb 1 and Mips16 use this callback as well).
Patch by Richard Sandiford.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@178137 91177308-0d34-0410-b5e6-96231b3b80d8
As pointed out by Richard Sandiford, my recent updates to the register
scavenger broke targets that use custom spilling (because the new code assumed
that if there were no valid spill slots, than spilling would be impossible).
I don't have a test case, but it should be possible to create one for Thumb 1,
Mips 16, etc.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@178073 91177308-0d34-0410-b5e6-96231b3b80d8
The previous algorithm could not deal properly with scavenging multiple virtual
registers because it kept only one live virtual -> physical mapping (and
iterated through operands in order). Now we don't maintain a current mapping,
but rather use replaceRegWith to completely remove the virtual register as
soon as the mapping is established.
In order to allow the register scavenger to return a physical register killed
by an instruction for definition by that same instruction, we now call
RS->forward(I) prior to eliminating virtual registers defined in I. This
requires a minor update to forward to ignore virtual registers.
These new features will be tested in forthcoming commits.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@178058 91177308-0d34-0410-b5e6-96231b3b80d8
