This patch splits atomics out of the generic G_LOAD/G_STORE and into their own
G_ATOMIC_LOAD/G_ATOMIC_STORE. This is a pragmatic decision rather than a
necessary one. Atomic load/store has little in implementation in common with
non-atomic load/store. They tend to be handled very differently throughout the
backend. It also has the nice side-effect of slightly improving the common-case
performance at ISel since there's no longer a need for an atomicity check in the
matcher table.
All targets have been updated to remove the atomic load/store check from the
G_LOAD/G_STORE path. AArch64 has also been updated to mark
G_ATOMIC_LOAD/G_ATOMIC_STORE legal.
There is one issue with this patch though which also affects the extending loads
and truncating stores. The rules only match when an appropriate G_ANYEXT is
present in the MIR. For example,
(G_ATOMIC_STORE (G_TRUNC:s16 (G_ANYEXT:s32 (G_ATOMIC_LOAD:s16 X))))
will match but:
(G_ATOMIC_STORE (G_ATOMIC_LOAD:s16 X))
will not. This shouldn't be a problem at the moment, but as we get better at
eliminating extends/truncates we'll likely start failing to match in some
cases. The current plan is to fix this in a patch that changes the
representation of extending-load/truncating-store to allow the MMO to describe
a different type to the operation.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@319691 91177308-0d34-0410-b5e6-96231b3b80d8
GIM_CheckNonAtomic has been replaced by GIM_CheckAtomicOrdering to allow it to support a wider
range of orderings. This has then been used to import patterns using nodes such
as atomic_cmp_swap, atomic_swap, and atomic_load_*.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@319232 91177308-0d34-0410-b5e6-96231b3b80d8
ptypeN is functionally the same as typeN except that it informs the
SelectionDAG importer that an operand should be treated as a pointer even
if it was written as iN. This is important for patterns that use iN instead
of iPTR to represent pointers. E.g.:
(set GPR64:$dst, (load GPR64:$addr))
Previously, this was handled as a hardcoded special case for the appropriate
operands to G_LOAD and G_STORE.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@318574 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch adds a LLVM_ENABLE_GISEL_COV which, like LLVM_ENABLE_DAGISEL_COV,
causes TableGen to instrument the generated table to collect rule coverage
information. However, LLVM_ENABLE_GISEL_COV goes a bit further than
LLVM_ENABLE_DAGISEL_COV. The information is written to files
(${CMAKE_BINARY_DIR}/gisel-coverage-* by default). These files can then be
concatenated into ${LLVM_GISEL_COV_PREFIX}-all after which TableGen will
read this information and use it to emit warnings about untested rules.
This technique could also be used by SelectionDAG and can be further
extended to detect hot rules and give them priority over colder rules.
Usage:
* Enable LLVM_ENABLE_GISEL_COV in CMake
* Build the compiler and run some tests
* cat gisel-coverage-[0-9]* > gisel-coverage-all
* Delete lib/Target/*/*GenGlobalISel.inc*
* Build the compiler
Known issues:
* ${LLVM_GISEL_COV_PREFIX}-all must be generated as a manual
step due to a lack of a portable 'cat' command. It should be the
concatenation of all ${LLVM_GISEL_COV_PREFIX}-[0-9]* files.
* There's no mechanism to discard coverage information when the ruleset
changes
Depends on D39742
Reviewers: ab, qcolombet, t.p.northover, aditya_nandakumar, rovka
Reviewed By: rovka
Subscribers: vsk, arsenm, nhaehnle, mgorny, kristof.beyls, javed.absar, igorb, llvm-commits
Differential Revision: https://reviews.llvm.org/D39747
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@318356 91177308-0d34-0410-b5e6-96231b3b80d8
Allow a pattern rewriter to be installed in CodeGenDAGPatterns and use it to
correct situations where SelectionDAG and GlobalISel disagree on
representation. For example, it would rewrite:
(sextload:i32 $ptr)<<unindexedload>><<sextload>><<sextloadi16>
to:
(sext:i32 (load:i16 $ptr)<<unindexedload>>)
I'd have preferred to replace the fragments and have the expansion happen
naturally as part of PatFrag expansion but the type inferencing system can't
cope with loads of types narrower than those mentioned in register classes.
This is because the SDTCisInt's on the sext constrain both the result and
operand to the 'legal' integer types (where legal is defined as 'a register
class can contain the type') which immediately rules the narrower types out.
Several targets (those with only one legal integer type) would then go on to
crash on the SDTCisOpSmallerThanOp<> when it removes all the possible types
for the result of the extend.
Also, improve isObviouslySafeToFold() slightly to automatically return true for
neighbouring instructions. There can't be any re-ordering problems if
re-ordering isn't happenning. We'll need to improve it further to handle
sign/zero-extending loads when the extend and load aren't immediate neighbours
though.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317971 91177308-0d34-0410-b5e6-96231b3b80d8
The GlobalISel TableGen backend didn't check for predicates on the
source children. This caused it to generate code for ARM patterns such
as SMLABB or similar, but without properly checking for the sext_16_node
part of the operands. This in turn meant that we would select SMLABB
instead of MLA for simple sequences such as s32 + s32 * s32, which is
wrong (we want a MLA on the full operands, not just their bottom 16
bits).
This patch forces TableGen to skip patterns with predicates on the src
children, so it doesn't generate code for SMLABB and other similar ARM
instructions at all anymore. AArch64 and X86 are not affected.
Differential Revision: https://reviews.llvm.org/D39554
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317313 91177308-0d34-0410-b5e6-96231b3b80d8
The importer will now accept nested instructions in the result pattern such as
(ADDWrr $a, (SUBWrr $b, $c)). This is only valid when the nested instruction
def's a single vreg and the parent instruction consumes a single vreg where a
nested instruction is specified. The importer will automatically create a vreg
to connect the two using the type information from the pattern. This vreg will
be constrained to the register classes given in the instruction definitions*.
* REG_SEQUENCE is explicitly rejected because of this. The definition doesn't
constrain to a register class and it therefore needs special handling.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317117 91177308-0d34-0410-b5e6-96231b3b80d8
The next commit will add support for multi-instruction emission so we need to
start allocating instruction ID's instead of hard-coding them to 0.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317057 91177308-0d34-0410-b5e6-96231b3b80d8
Multi-instruction emission needs to ensure the the instructions are generated
a depth-first fashion. For example:
(ADDWrr (SUBWrr a, b), c)
needs to emit the SUBWrr before the ADDWrr. However, our walk over
TreePatternNode's is highly context sensitive which makes it difficult to append
BuildMIActions in the order we want. To fix this, we now keep track of the
insertion point as we add actions. This will allow multi-insn emission to insert
BuildMI's in the correct place.
The previous commit failed on the Ubuntu bots using GCC 4.8. These bots lack the
const_iterator forms of insert() and emplace() that were added in C++11. As a
result I've switched the const_iterators to iterators.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317049 91177308-0d34-0410-b5e6-96231b3b80d8
The same bots fail but I believe I know what the issue is now. These bots are
missing the const_iterator versions of insert/emplace/etc. that were introduced
in C++11.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317042 91177308-0d34-0410-b5e6-96231b3b80d8
Multi-instruction emission needs to ensure the the instructions are generated
a depth-first fashion. For example:
(ADDWrr (SUBWrr a, b), c)
needs to emit the SUBWrr before the ADDWrr. However, our walk over
TreePatternNode's is highly context sensitive which makes it difficult to append
BuildMIActions in the order we want. To fix this, we now keep track of the
insertion point as we add actions. This will allow multi-insn emission to insert
BuildMI's in the correct place.
The previous commit failed on the Ubuntu bots using GCC 4.8. These bots didn't
like a call to emplace(). I've replaced it with insert() to see if it's a quirk
of the C++11 support.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317040 91177308-0d34-0410-b5e6-96231b3b80d8
Multi-instruction emission needs to ensure the the instructions are generated
a depth-first fashion. For example:
(ADDWrr (SUBWrr a, b), c)
needs to emit the SUBWrr before the ADDWrr. However, our walk over
TreePatternNode's is highly context sensitive which makes it difficult to append
BuildMIActions in the order we want. To fix this, we now keep track of the
insertion point as we add actions. This will allow multi-insn emission to insert
BuildMI's in the correct place.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317029 91177308-0d34-0410-b5e6-96231b3b80d8
Multi-instruction emission will require that we have separate handling for
the defs between the implicitly created temporaries and the rule outputs.
The former require new temporary vregs while the latter should copy existing
operands. Factor out the implicit def/use renderers to minimize the code
duplication when we implement that.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317025 91177308-0d34-0410-b5e6-96231b3b80d8
Prepare for multiple instruction emission by allowing BuildMIAction to
search for a suitable matcher that will support mutation.
This patch deliberately neglects to add matchers aside from the root to
preserve NFC. That said, it should be noted that until we support mutations
other than just the opcode the chances of finding a non-root instruction
for which canMutate() is true, is essentially zero. Furthermore in the
presence of multi-instruction emission the chances of finding any
instruction for which canMutate() is true is also zero. Nevertheless, we
can't continue to require that all BuildMIAction's consider the root of the match
to be recyclable due to the risk of recycling it twice in the same rule.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317022 91177308-0d34-0410-b5e6-96231b3b80d8
I missed a dereference of `Matched` that preceeded the new check. Thanks to
Justin Bogner for spotting it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316480 91177308-0d34-0410-b5e6-96231b3b80d8
When multi-instruction emission is supported, it will no longer be guaranteed
that every BuildMIAction has a corresponding matched instruction. BuildMIAction
should support not having one to cover the case where a rule produces more
instructions than it matched.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316463 91177308-0d34-0410-b5e6-96231b3b80d8
This patch enables the import of stores. Unfortunately, doing so by itself,
loses an optimization where storing 0 to memory makes use of WZR/XZR.
To mitigate this, this patch also introduces a new feature that allows register
operands to nominate a zero register. When this is done, GlobalISel will
substitute (G_CONSTANT 0) with the nominated register automatically. This
is currently configured to only apply to the stores.
Applying it to GPR32/GPR64 register classes in general will be done after
review see (https://reviews.llvm.org/D39150).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316360 91177308-0d34-0410-b5e6-96231b3b80d8
ComplexRendererFn -> ComplexRendererFns
Corrected a couple lingering references to tied operands that were missed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316237 91177308-0d34-0410-b5e6-96231b3b80d8
MSVC doesn't seem to like implicitly instantiating addPredicate and then
explicitly specializing it later. It causes an internal compiler error.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315930 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
iPTR is a pointer of subtarget-specific size to any address space. Therefore
type checks on this size derive the SizeInBits from a subtarget hook.
At this point, we can import the simplests G_LOAD rules and select load
instructions using them. Further patches will support for the predicates to
enable additional loads as well as the stores.
The previous commit failed on MSVC due to a failure to convert an
initializer_list to a std::vector. Hopefully, MSVC will accept this version.
Depends on D37457
Reviewers: ab, qcolombet, t.p.northover, rovka, aditya_nandakumar
Reviewed By: qcolombet
Subscribers: kristof.beyls, javed.absar, llvm-commits, igorb
Differential Revision: https://reviews.llvm.org/D37458
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315887 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
iPTR is a pointer of subtarget-specific size to any address space. Therefore
type checks on this size derive the SizeInBits from a subtarget hook.
At this point, we can import the simplests G_LOAD rules and select load
instructions using them. Further patches will support for the predicates to
enable additional loads as well as the stores.
Depends on D37457
Reviewers: ab, qcolombet, t.p.northover, rovka, aditya_nandakumar
Reviewed By: qcolombet
Subscribers: kristof.beyls, javed.absar, llvm-commits, igorb
Differential Revision: https://reviews.llvm.org/D37458
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315885 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This includes some context-sensitivity in the MVT to LLT conversion so that
pointer types are tested correctly.
FIXME: I'm not happy with the way this is done since everything is a
special-case. I've yet to find a reasonable way to implement it.
select-load.mir fails because <1 x s64> loads in tablegen get priority over s64
loads. This is fixed in the next patch and as such they should be committed
together, I've posted them separately to help with the review.
Depends on D37456
Reviewers: ab, qcolombet, t.p.northover, rovka, aditya_nandakumar
Subscribers: kristof.beyls, javed.absar, llvm-commits, igorb
Differential Revision: https://reviews.llvm.org/D37457
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315884 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
It's possible for a ComplexPattern to be used as an operator in a match
pattern. This is used by the load/store patterns in AArch64 to name the
suboperands returned by ComplexPattern predicate so that they can be broken
apart and referenced independently in the result pattern.
This patch adds support for this in order to enable the import of load/store
patterns.
Depends on D37445
Hopefully fixed the ambiguous constructor that a large number of bots reported.
Reviewers: ab, qcolombet, t.p.northover, rovka, aditya_nandakumar
Reviewed By: qcolombet
Subscribers: aemerson, javed.absar, igorb, llvm-commits, kristof.beyls
Differential Revision: https://reviews.llvm.org/D37456
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315869 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
It's possible for a ComplexPattern to be used as an operator in a match
pattern. This is used by the load/store patterns in AArch64 to name the
suboperands returned by ComplexPattern predicate so that they can be broken
apart and referenced independently in the result pattern.
This patch adds support for this in order to enable the import of load/store
patterns.
Depends on D37445
Reviewers: ab, qcolombet, t.p.northover, rovka, aditya_nandakumar
Reviewed By: qcolombet
Subscribers: aemerson, javed.absar, igorb, llvm-commits, kristof.beyls
Differential Revision: https://reviews.llvm.org/D37456
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315863 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
There is an important mismatch between ISD::LOAD and G_LOAD (and likewise for
ISD::STORE and G_STORE). In SelectionDAG, ISD::LOAD is a non-atomic load
and atomic loads are handled by a separate node. However, this is not true of
GlobalISel's G_LOAD. For G_LOAD, the MachineMemOperand indicates the atomicity
of the operation. As a result, this mapping must also add a predicate that
checks for non-atomic MachineMemOperands.
This is NFC since these nodes always have predicates in practice and are
therefore always rejected at the moment.
Depends on D37443
Reviewers: ab, qcolombet, t.p.northover, rovka, aditya_nandakumar
Reviewed By: qcolombet
Subscribers: kristof.beyls, llvm-commits, igorb
Differential Revision: https://reviews.llvm.org/D37445
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315843 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
GlobalISel and SelectionDAG require different code for the common
load/store predicates due to differences in the representation.
For example:
SelectionDAG: (load<signext,i8>:i32 GPR32:$addr) // The <> denote properties of the SDNode that are not printed in the DAG
GlobalISel: (G_SEXT:s32 (G_LOAD:s8 GPR32:$addr))
Even without that, differences in the IR (SDNode vs MachineInstr) require
differences in the C++ predicate.
This patch moves the implementation of the common load/store predicates
into tablegen so that it can handle these differences.
It's NFC for SelectionDAG since it emits equivalent code and it's NFC for
GlobalISel since the rules involving the relevant predicates are still
rejected by the importer.
Depends on D36618
Reviewers: ab, qcolombet, t.p.northover, rovka, aditya_nandakumar
Subscribers: llvm-commits, igorb
Differential Revision: https://reviews.llvm.org/D37443
Includes a partial revert of r315826 since this patch makes it necessary for
getPredCode() to return a std::string and getImmCode() should have the same
interface as getPredCode().
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315841 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Operand variable lookups are now performed by the RuleMatcher rather than
searching the whole matcher hierarchy for a match. This revealed a wrong-code
bug that currently affects ARM and X86 where patterns that use a variable more
than once in the match pattern will be imported but won't check that the
operands are identical. This can cause the tablegen-erated matcher to
accept matches that should be rejected.
Depends on D36569
Reviewers: ab, t.p.northover, qcolombet, rovka, aditya_nandakumar
Subscribers: aemerson, igorb, llvm-commits, kristof.beyls
Differential Revision: https://reviews.llvm.org/D36618
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315780 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
There's only a tablegen testcase for IntImmLeaf and not a CodeGen one
because the relevant rules are rejected for other reasons at the moment.
On AArch64, it's because there's an SDNodeXForm attached to the operand.
On X86, it's because the rule either emits multiple instructions or has
another predicate using PatFrag which cannot easily be supported at the
same time.
Reviewers: ab, t.p.northover, qcolombet, rovka, aditya_nandakumar
Reviewed By: qcolombet
Subscribers: aemerson, javed.absar, igorb, llvm-commits, kristof.beyls
Differential Revision: https://reviews.llvm.org/D36569
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315761 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
The purpose of this patch is to expose more information about ImmLeaf-like
PatLeaf's so that GlobalISel can learn to import them. Previously, ImmLeaf
could only be used to test int64_t's produced by sign-extending an APInt.
Other tests on immediates had to use the generic PatLeaf and extract the
constant using C++.
With this patch, tablegen will know how to generate predicates for APInt,
and APFloat. This will allow it to 'do the right thing' for both SelectionDAG
and GlobalISel which require different methods of extracting the immediate
from the IR.
This is NFC for SelectionDAG since the new code is equivalent to the
previous code. It's also NFC for FastISel because FastIselShouldIgnore is 1
for the ImmLeaf subclasses. Enabling FastIselShouldIgnore == 0 for these new
subclasses will require a significant re-factor of FastISel.
For GlobalISel, it's currently NFC because the relevant code to import the
affected rules is not yet present. This will be added in a later patch.
Depends on D36086
Reviewers: ab, t.p.northover, qcolombet, rovka, aditya_nandakumar
Reviewed By: qcolombet
Subscribers: bjope, aemerson, rengolin, javed.absar, igorb, llvm-commits, kristof.beyls
Differential Revision: https://reviews.llvm.org/D36534
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315747 91177308-0d34-0410-b5e6-96231b3b80d8
It's rare but there are a small number of patterns like this:
(set i64:$dst, (add i64:$src1, i64:$src2))
These should be equivalent to register classes except they shouldn't check for
a specific register bank.
This doesn't occur in AArch64/ARM/X86 but does occasionally come up in other
in-tree targets such as BPF.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315226 91177308-0d34-0410-b5e6-96231b3b80d8
This replaces TableGen's type inference to operate on parameterized
types instead of MVTs, and as a consequence, some interfaces have
changed:
- Uses of MVTs are replaced by ValueTypeByHwMode.
- EEVT::TypeSet is replaced by TypeSetByHwMode.
This affects the way that types and type sets are printed, and the
tests relying on that have been updated.
There are certain users of the inferred types outside of TableGen
itself, namely FastISel and GlobalISel. For those users, the way
that the types are accessed have changed. For typical scenarios,
these replacements can be used:
- TreePatternNode::getType(ResNo) -> getSimpleType(ResNo)
- TreePatternNode::hasTypeSet(ResNo) -> hasConcreteType(ResNo)
- TypeSet::isConcrete -> TypeSetByHwMode::isValueTypeByHwMode(false)
For more information, please refer to the review page.
Differential Revision: https://reviews.llvm.org/D31951
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313271 91177308-0d34-0410-b5e6-96231b3b80d8
This fixes a warning when there are zero defined predicates and also fixes an
unnoticed bug where the first predicate in the table was unusable.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311684 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch adds support for predicates on imm nodes but only for ImmLeaf and not
for PatLeaf or PatFrag and only where the value does not need to be transformed
before being rendered into the instruction.
The limitation on PatLeaf/PatFrag/SDNodeXForm is due to differences in the
necessary target-supplied C++ for GlobalISel.
Depends on D36085
The previous commit was reverted for breaking the build but this appears to have
been the recurring problem on the Windows bots with tablegen not being re-run
when llvm-tblgen is changed but the .td's aren't. If it re-occurs then forcing a
build with clean=True should fix it but this string should do this in advance:
Requires a clean build.
Reviewers: ab, t.p.northover, qcolombet, rovka, aditya_nandakumar
Reviewed By: rovka
Subscribers: kristof.beyls, javed.absar, igorb, llvm-commits
Differential Revision: https://reviews.llvm.org/D36086
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311645 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch adds support for predicates on imm nodes but only for ImmLeaf and not for PatLeaf or PatFrag and only where the value does not need to be transformed before being rendered into the instruction.
The limitation on PatLeaf/PatFrag/SDNodeXForm is due to differences in the necessary target-supplied C++ for GlobalISel.
Depends on D36085
Reviewers: ab, t.p.northover, qcolombet, rovka, aditya_nandakumar
Reviewed By: rovka
Subscribers: kristof.beyls, javed.absar, igorb, llvm-commits
Differential Revision: https://reviews.llvm.org/D36086
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311546 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Generate the type table from the types used by a target rather than hard-coding
the union of types used by all targets.
Depends on D36084
Reviewers: ab, t.p.northover, qcolombet, rovka, aditya_nandakumar
Reviewed By: rovka
Subscribers: kristof.beyls, igorb, llvm-commits
Differential Revision: https://reviews.llvm.org/D36085
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311084 91177308-0d34-0410-b5e6-96231b3b80d8
