This re-applies r262886 with a fix for 32 bit platforms that have 8 byte
pointer alignment, effectively reverting r262892.
Original Message:
Currently some SDNode operands are malloc'd, some are stored inline in
subclasses of SDNode, and some are thrown into a BumpPtrAllocator.
This scheme is complex, inconsistent, and makes refactoring SDNodes
fairly difficult.
Instead, we can allocate all of the operands using an ArrayRecycler
that wraps a BumpPtrAllocator. This keeps the cache locality when
iterating operands, improves locality when iterating SDNodes without
looking at operands, and vastly simplifies the ownership semantics.
It also means we stop overallocating SDNodes by 2-3x and will make it
simpler to fix the rampant undefined behaviour we have in how we
mutate SDNodes from one kind to another (See llvm.org/pr26808).
This is NFC other than the changes in memory behaviour, and I ran some
LNT tests to make sure this didn't hurt compile time. Not many tests
changed: there were a couple of 1-2% regressions reported, but there
were more improvements (of up to 4%) than regressions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262902 91177308-0d34-0410-b5e6-96231b3b80d8
Looks like the largest SDNode is different between 32 and 64 bit now,
so this is breaking 32 bit bots. Reverting while I figure out a fix.
This reverts r262886.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262892 91177308-0d34-0410-b5e6-96231b3b80d8
Currently some SDNode operands are malloc'd, some are stored inline in
subclasses of SDNode, and some are thrown into a BumpPtrAllocator.
This scheme is complex, inconsistent, and makes refactoring SDNodes
fairly difficult.
Instead, we can allocate all of the operands using an ArrayRecycler
that wraps a BumpPtrAllocator. This keeps the cache locality when
iterating operands, improves locality when iterating SDNodes without
looking at operands, and vastly simplifies the ownership semantics.
It also means we stop overallocating SDNodes by 2-3x and will make it
simpler to fix the rampant undefined behaviour we have in how we
mutate SDNodes from one kind to another (See llvm.org/pr26808).
This is NFC other than the changes in memory behaviour, and I ran some
LNT tests to make sure this didn't hurt compile time. Not many tests
changed: there were a couple of 1-2% regressions reported, but there
were more improvements (of up to 4%) than regressions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262886 91177308-0d34-0410-b5e6-96231b3b80d8
Now the type API is always available, but when global-isel is not
built the implementation does nothing.
Note: The implementation free of ifdefs is WIP and tracked here in PR26576.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262873 91177308-0d34-0410-b5e6-96231b3b80d8
We have known UB in some ilists where we static cast half nodes to
(larger) derived types and use the address. See llvm.org/PR26753.
This needs to be fixed, but in the meantime it'd be nice if running
ubsan didn't complain. This adds annotations in the two places where
ubsan complains while running check-all of a sanitized clang build.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262683 91177308-0d34-0410-b5e6-96231b3b80d8
The vast majority of LiveRanges (ie, 4/5) have exactly 1 segment and 1
value number, and a good chunk of the rest have 2 of each, so
allocating space for 4 is wasteful. This is especially noticeable when
dealing with a very large number of vregs, and I have an internal case
where dropping this to 2 shaves over 5% off of peak memory when
compiling a particularly large function.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262681 91177308-0d34-0410-b5e6-96231b3b80d8
The placement new calls here were all calling the allocation function
in RecyclingAllocator/Recycler for SDNode, instead of the function for
the specific subclass we were constructing.
Since this particular allocator always overallocates it more or less
worked, but would hide what we're actually doing from any memory
tools. Also, if you tried to change this allocator so something like a
BumpPtrAllocator or MallocAllocator, the compiler would crash horribly
all the time.
Part of llvm.org/PR26808.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262500 91177308-0d34-0410-b5e6-96231b3b80d8
In the code below on 32-bit targets, x would previously get forwarded to g()
without sign-extension to 32 bits as required by the parameter attribute.
void g(signed short);
void f(unsigned short x) {
g(x);
}
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262352 91177308-0d34-0410-b5e6-96231b3b80d8
Change MachineInstr API to prefer MachineInstr& over MachineInstr*
whenever the parameter is expected to be non-null. Slowly inching
toward being able to fix PR26753.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262149 91177308-0d34-0410-b5e6-96231b3b80d8
In all but one case, change the DFAPacketizer API to take MachineInstr&
instead of MachineInstr*. In DFAPacketizer::endPacket(), take
MachineBasicBlock::iterator. Besides cleaning up the API, this is in
search of PR26753.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262142 91177308-0d34-0410-b5e6-96231b3b80d8
Update APIs in MachineInstrBundle.h to take and return MachineInstr&
instead of MachineInstr* when the instruction cannot be null. Besides
being a nice cleanup, this is tacking toward a fix for PR26753.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262141 91177308-0d34-0410-b5e6-96231b3b80d8
Avoid another implicit conversion from MachineInstrBundleIterator to
MachineInstr*, this time in MachineInstrBuilder.h (this is in pursuit of
PR26753).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262118 91177308-0d34-0410-b5e6-96231b3b80d8
Take MachineInstr by reference instead of by pointer in SlotIndexes and
the SlotIndex wrappers in LiveIntervals. The MachineInstrs here are
never null, so this cleans up the API a bit. It also incidentally
removes a few implicit conversions from MachineInstrBundleIterator to
MachineInstr* (see PR26753).
At a couple of call sites it was convenient to convert to a range-based
for loop over MachineBasicBlock::instr_begin/instr_end, so I added
MachineBasicBlock::instrs.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262115 91177308-0d34-0410-b5e6-96231b3b80d8
(This is the second attemp to commit this patch, after fixing pr26652 & pr26653).
This patch detects vector reductions before instruction selection. Vector
reductions are vectorized reduction operations, and for such operations we have
freedom to reorganize the elements of the result as long as the reduction of them
stay unchanged. This will enable some reduction pattern recognition during
instruction combine such as SAD/dot-product on X86. A flag is added to
SDNodeFlags to mark those vector reduction nodes to be checked during instruction
combine.
To detect those vector reductions, we search def-use chains starting from the
given instruction, and check if all uses fall into two categories:
1. Reduction with another vector.
2. Reduction on all elements.
in which 2 is detected by recognizing the pattern that the loop vectorizer
generates to reduce all elements in the vector outside of the loop, which
includes several ShuffleVector and one ExtractElement instructions.
Differential revision: http://reviews.llvm.org/D15250
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@261804 91177308-0d34-0410-b5e6-96231b3b80d8
This fixes bugs in copy elimination code in llvm. It slightly changes the
semantics of clearRegisterKills(). This is appropriate because:
- Users in lib/CodeGen/MachineCopyPropagation.cpp and
lib/Target/AArch64RedundantCopyElimination.cpp and
lib/Target/SystemZ/SystemZElimCompare.cpp are incorrect without it
(see included testcase).
- All other users in llvm are unaffected (they pass TRI==nullptr)
- (Kill flags are optional anyway so removing too many shouldn't hurt.)
Differential Revision: http://reviews.llvm.org/D17554
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@261763 91177308-0d34-0410-b5e6-96231b3b80d8
This is a little embarrassing.
When I reverted r261504 (getIterator() => getInstrIterator()) in
r261567, I did a `git grep` to see if there were new calls to
`getInstrIterator()` that I needed to migrate. There were 10-20 hits,
and I blindly did a `sed ...` before calling `ninja check`.
However, these were `MachineInstrBundleIterator::getInstrIterator()`,
which predated r261567. Perhaps coincidentally, these had an identical
name and return type.
This commit undoes my careless sed and restores
`MachineBasicBlock::iterator::getInstrIterator()`.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@261577 91177308-0d34-0410-b5e6-96231b3b80d8
This reverts commit r261510, effectively reapplying r261509. The
original commit missed a caller in AArch64ConditionalCompares.
Original commit message:
Pass non-null arguments by reference in MachineTraceMetrics::Trace,
simplifying future work to remove implicit iterator => pointer
conversions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@261511 91177308-0d34-0410-b5e6-96231b3b80d8
Pass non-null arguments by reference in MachineTraceMetrics::Trace,
simplifying future work to remove implicit iterator => pointer
conversions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@261509 91177308-0d34-0410-b5e6-96231b3b80d8
Delete MachineInstr::getIterator(), since the term "iterator" is
overloaded when talking about MachineInstr.
- Downcast to ilist_node in iplist::getNextNode() and getPrevNode() so
that ilist_node::getIterator() is still available.
- Add it back as MachineInstr::getInstrIterator(). This matches the
naming in MachineBasicBlock.
- Add MachineInstr::getBundleIterator(). This is explicitly called
"bundle" (not matching MachineBasicBlock) to disintinguish it clearly
from ilist_node::getIterator().
- Update all calls. Some of these I switched to `auto` to remove
boiler-plate, since the new name is clear about the type.
There was one call I updated that looked fishy, but it wasn't clear what
the right answer was. This was in X86FrameLowering::inlineStackProbe(),
added in r252578 in lib/Target/X86/X86FrameLowering.cpp. I opted to
leave the behaviour unchanged, but I'll reply to the original commit on
the list in a moment.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@261504 91177308-0d34-0410-b5e6-96231b3b80d8
Split MachineBasicBlock::bundle_iterator into a separate file, and
rename the class to MachineBundleIterator.
This is a precursor to adding a `MachineInstr::getBundleIterator()`
accessor, which will eventually let us delete the final call to
getNodePtrUnchecked(), and then remove the UB from ilist_iterator.
As a drive-by, I removed an unnecessary second template parameter.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@261502 91177308-0d34-0410-b5e6-96231b3b80d8
COFF doesn't have sections with mergeable contents. Instead, each
constant pool entry ends up in a COMDAT section. The linker, when
choosing between COMDAT sections, doesn't choose the max alignment of
the two sections. You just get whatever alignment was on the section.
If one constant needed a higher alignment in one object file from
another one, then we will get into trouble if the linker chooses the
lower alignment one.
Instead, lets promote the alignment of the constant pool entry to make
sure we don't use an under aligned constant with an instruction which
assumed otherwise.
This fixes PR26680.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@261462 91177308-0d34-0410-b5e6-96231b3b80d8
This patch detects vector reductions before instruction selection. Vector
reductions are vectorized reduction operations, and for such operations we have
freedom to reorganize the elements of the result as long as the reduction of them
stay unchanged. This will enable some reduction pattern recognition during
instruction combine such as SAD/dot-product on X86. A flag is added to
SDNodeFlags to mark those vector reduction nodes to be checked during instruction
combine.
To detect those vector reductions, we search def-use chains starting from the
given instruction, and check if all uses fall into two categories:
1. Reduction with another vector.
2. Reduction on all elements.
in which 2 is detected by recognizing the pattern that the loop vectorizer
generates to reduce all elements in the vector outside of the loop, which
includes several ShuffleVector and one ExtractElement instructions.
Differential revision: http://reviews.llvm.org/D15250
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@261070 91177308-0d34-0410-b5e6-96231b3b80d8
Original message:
Get rid of the ifdefs in TargetLowering.
Introduce a new API used only by GlobalISel: CallLowering.
This API will contain target hooks dedicated to call lowering.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260998 91177308-0d34-0410-b5e6-96231b3b80d8
