Commit Graph

2091 Commits

Author SHA1 Message Date
Stanislav Mekhanoshin
37bcd272bb [AMDGPU] fix commuted case of sub combine
Differential Revision: https://reviews.llvm.org/D58481

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354543 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-21 02:58:00 +00:00
Tom Stellard
21d7b5a1d6 AMDGPU/GlobalISel: Move SMRD selection logic to TableGen
Reviewers: arsenm

Reviewed By: arsenm

Subscribers: volkan, kzhuravl, jvesely, wdng, nhaehnle, yaxunl, rovka, kristof.beyls, dstuttard, tpr, t-tye, llvm-commits

Differential Revision: https://reviews.llvm.org/D52922

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354516 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-20 21:02:37 +00:00
Matt Arsenault
1b59f4c380 GlobalISel: Fix fewerElementsVector for ctlz with different result type
Also complete the set of related operations.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354480 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-20 16:42:52 +00:00
Matt Arsenault
7e1a65dad5 GlobalISel: Implement moreElementsVector for g_insert results
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354477 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-20 16:11:22 +00:00
Matt Arsenault
379689ce0c GlobalISel: Implement moreElementsVector for select
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354354 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-19 17:03:09 +00:00
Matt Arsenault
406dc2a0d5 GlobalISel: Implement moreElementsVector for G_EXTRACT source
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354348 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-19 16:44:22 +00:00
Matt Arsenault
47f8b7cd25 GlobalISel: Implement moreElementsVector for bit ops
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354345 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-19 16:30:19 +00:00
Changpeng Fang
1e3c4790ab AMDGPU: Use MachineInstr::mayAlias to replace areMemAccessesTriviallyDisjoint in LoadStoreOptimizer pass.
Summary:
  This is to fix a memory dependence bug in LoadStoreOptimizer.

Reviewers:
  arsenm, rampitec

Differential Revision:
  https://reviews.llvm.org/D58295

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354295 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-18 23:00:26 +00:00
Matt Arsenault
b1b624d08a GlobalISel: Implement widenScalar for g_extract scalar results
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354293 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-18 22:39:27 +00:00
Matt Arsenault
57078b6e3a Try to organize MachineVerifier tests
The Verifier is separate from the MachineVerifier, so move it to a
different directory. Some other verifier tests were scattered in
target codegen tests as well (although I'm sure I missed some). Work
towards using a more consistent naming scheme to make it clearer where
the gaps still are for generic instructions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354138 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-15 15:24:31 +00:00
Konstantin Zhuravlyov
1d02248c5f AMDGPU: Set ABI version to 1 for code object v3
Differential Revision: https://reviews.llvm.org/D57811


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354085 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-14 23:56:04 +00:00
Matt Arsenault
217f03f322 AMDGPU/GlobalISel: Fix RegBankSelect for GEP.
This is basically a pointer typed add, so shouldn't be any different.
This was assuming everything was an SGPR, which is not true.

Also cleanup legality for GEP. I don't seem to be seeing the problem
the hack marking s64 as a legal pointer type the comment mentions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354067 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-14 22:24:28 +00:00
Stanislav Mekhanoshin
2f5fd7e3bd [AMDGPU] Ressociate 'add (add x, y), z' to use SALU
Reassociate adds to collect scalar operands in a single
instruction when possible. That will result in a scalar
add followed by vector instead of two vector adds, thus
better utilizing SALU.

Differential Revision: https://reviews.llvm.org/D58220

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354066 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-14 22:11:25 +00:00
Matt Arsenault
9b6003fe19 AMDGPU/GlobalISel: Handle split for 64-bit VALU select
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354065 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-14 21:58:12 +00:00
David Stenberg
ac705e8410 [DebugInfo] Stop changing labels for register-described parameter DBG_VALUEs
Summary:
This is a follow-up to D57510. This patch stops DebugHandlerBase from
changing the starting label for the first non-overlapping,
register-described parameter DBG_VALUEs to the beginning of the
function. That code did not consider what defined the registers, which
could result in the ranges for the debug values starting before their
defining instructions. We currently do not emit debug values for
constant values directly at the start of the function, so this code is
still useful for such values, but my intention is to remove the code
from DebugHandlerBase completely when we get there. One reason for
removing it is that the code violates the history map's ranges, which I
think can make it quite confusing when troubleshooting.

In D57510, PrologEpilogInserter was amended so that parameter DBG_VALUEs
now are kept at the start of the entry block, even after emission of
prologue code. That was done to reduce the degradation of debug
completeness from this patch. PR40638 is another example, where the
lexical-scope trimming that LDV does, in combination with scheduling,
results in instructions after the prologue being left without locations.
There might be other cases where the DBG_VALUEs are pushed further down,
for which the DebugHandlerBase code may be helpful, but as it now quite
often result in incorrect locations, even after the prologue, it seems
better to remove that code, and try to work our way up with accurate
locations.

In the long run we should maybe not aim to provide accurate locations
inside the prologue. Some single location descriptions, at least those
referring to stack values, generate inaccurate values inside the
epilogue, so we maybe should not aim to achieve accuracy for location
lists. However, it seems that we now emit line number programs that can
result in GDB and LLDB stopping inside the prologue when doing line
number stepping into functions. See PR40188 for more information.

A summary of some of the changed test cases is available in PR40188#c2.

Reviewers: aprantl, dblaikie, rnk, jmorse

Reviewed By: aprantl

Subscribers: jdoerfert, jholewinski, jvesely, javed.absar, llvm-commits

Tags: #debug-info, #llvm

Differential Revision: https://reviews.llvm.org/D57511

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353928 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-13 09:34:07 +00:00
Matt Arsenault
d135f2406f AMDGPU/GlobalISel: Add more insert/extract testcases
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353848 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-12 15:04:03 +00:00
Matt Arsenault
bb8d9165a4 AMDGPU/GlobalISel: Only make f16 constants legal on f16 targets
We could deal with it, but there's no real point.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353845 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-12 14:54:55 +00:00
Matt Arsenault
45f5ca3e2d GlobalISel: Verify G_EXTRACT
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353759 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-11 22:12:43 +00:00
Matt Arsenault
f3f4691605 GlobalISel: Implement moreElementsVector for implicit_def
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353754 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-11 22:00:39 +00:00
Matt Arsenault
c0665d4bcd GlobalISel: Add G_FCANONICALIZE instruction
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353719 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-11 17:05:20 +00:00
Valery Pykhtin
14478b44a8 [AMDGPU] fix atomic_optimizations_buffer.ll test after DPP combiner was enabled by default.
Related commits: rL353691, rL353703.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353717 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-11 16:28:42 +00:00
Neil Henning
4fee3cff4d [AMDGPU] Fix DPP sequence in atomic optimizer.
This commit fixes the DPP sequence in the atomic optimizer (which was
previously missing the row_shr:3 step), and works around a read_register
exec bug by using a ballot instead.

Differential Revision: https://reviews.llvm.org/D57737

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353703 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-11 14:44:14 +00:00
Stanislav Mekhanoshin
469309ae52 [AMDGPU] Split idot4/8 signed and unsigned tests. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353593 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-09 01:02:28 +00:00
Matt Arsenault
4390f003b3 AMDGPU/GlobalISel: Fix broken tests
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353559 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-08 19:59:39 +00:00
Matt Arsenault
163f2c349f AMDGPU: Remove GCN features and predicates
These are no longer necessary since the R600 tablegen files are split
out now.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353548 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-08 19:18:01 +00:00
Carl Ritson
46ba685628 [AMDGPU] Fix CS scratch setup on pre-GCN3 ASICs
Summary:
Prior to GCN3 s_load_dword offsets are in dwords rather than bytes.
Thus the scratch buffer descriptor offset must be adjusted for pre-GCN3 ASICs.

Reviewers: nhaehnle, tpr

Reviewed By: nhaehnle

Subscribers: sheredom, arsenm, kzhuravl, jvesely, wdng, yaxunl, dstuttard, t-tye, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D56496

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353530 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-08 15:41:11 +00:00
Matt Arsenault
7772407272 AMDGPU/GlobalISel: Fix shift legalization for non-power-of-2
clampScalar doesn't do anything for non-power-of-2 in range.
There should probably be a combination rule to reduce the number
of matching rules.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353526 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-08 15:06:24 +00:00
Matt Arsenault
b997233527 AMDGPU/GlobalISel: Fix non-power-of-2 implicit_def
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353522 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-08 14:46:27 +00:00
Petar Avramovic
005af434d1 [MIPS GlobalISel] Select any extending load and truncating store
Make behavior of G_LOAD in widenScalar same as for G_ZEXTLOAD and
G_SEXTLOAD. That is perform widenScalarDst to size given by the target
and avoid additional checks in common code. Targets can reorder or add
additional rules in LegalizeRuleSet for the opcode to achieve desired
behavior.

Select extending load that does not have specified type of extension
into zero extending load.

Select truncating store that stores number of bytes indicated by size
in MachineMemoperand.

Differential Revision: https://reviews.llvm.org/D57454


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353520 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-08 14:27:23 +00:00
Matt Arsenault
4473e73049 AMDGPU/GlobalISel: Don't use a copy in addrspacecast lowering
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353516 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-08 14:16:11 +00:00
Valery Pykhtin
3a178e01c4 [AMDGPU] Fix DPP combiner
Differential revision: https://reviews.llvm.org/D55444

dpp move with uses and old reg initializer should be in the same BB.
bound_ctrl:0 is only considered when bank_mask and row_mask are fully enabled (0xF). Otherwise the old register value is checked for identity.
Added add, subrev, and, or instructions to the old folding function.
Kill flag is cleared for the src0 (DPP register) as it may be copied into more than one user.

The pass is still disabled by default.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353513 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-08 11:59:48 +00:00
Matt Arsenault
e6576d59f1 AMDGPU/GlobalISel: Legalize addrspacecast
Use a placeholder constant for now on targets
that need the load from the queue ptr.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353497 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-08 02:40:47 +00:00
Nikita Popov
37e240c2fe [CodeGen] Handle vector UADDO, SADDO, USUBO, SSUBO
This is part of https://bugs.llvm.org/show_bug.cgi?id=40442.

Vector legalization is implemented for the add/sub overflow opcodes.
UMULO/SMULO are also handled as far as legalization is concerned, but
they don't support vector expansion yet (so no tests for them).

The vector result widening implementation is suboptimal, because it
could result in a legalization loop.

Differential Revision: https://reviews.llvm.org/D57639

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353464 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-07 21:02:22 +00:00
Matt Arsenault
c0f816e09c GlobalISel: Implement narrowScalar for shift main type
This is pretty much directly ported from SelectionDAG. Doesn't include
the shift by non-constant but known bits version, since there isn't a
globalisel version of computeKnownBits yet.

This shows a disadvantage of targets not specifically which type
should be used for the shift amount. If type 0 is legalized before
type 1, the operations on the shift amount type use the wider type
(which are also less likely to legalize). This can be avoided by
targets specifying legalization actions on type 1 earlier than for
type 0.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353455 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-07 19:37:44 +00:00
Matt Arsenault
cceef2f736 AMDGPU/GlobalISel: Restrict g_implicit_def legality
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353452 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-07 19:10:15 +00:00
Matt Arsenault
c9e555791d GlobalISel: Fix artifact combiner constant legality checks for vectors
Since G_CONSTANT is illegal for vectors, this needs to check
what buildConstant will produce for a splat vector.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353449 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-07 18:58:28 +00:00
Matt Arsenault
7c32994026 AMDGPU/GlobalISel: Don't use g_implicit_def in a few tests
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353443 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-07 18:33:22 +00:00
Matt Arsenault
4bafedb149 AMDGPU/GlobalISel: Legalize fsqrt
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353438 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-07 18:14:39 +00:00
Matt Arsenault
1faffeed21 AMDGPU/GlobalISel: Legalize some f16 operations
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353436 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-07 18:03:11 +00:00
Matt Arsenault
f2e3c26215 GlobalISel: Implement fewerElementsVector for shifts
Introduce a new function which handles instructions with multiple type
indices, but have the same number of vector elements.

Also legalize v2s16 shifts when applicable.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353432 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-07 17:38:00 +00:00
Scott Linder
2221866e2e [AMDGPU] Consider XOR in waterfall loop as a terminator
Ensure the XOR in the waterfall loop for indirect addressing is considered a terminator.

Differential Revision: https://reviews.llvm.org/D57703

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353207 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-05 19:50:32 +00:00
Matt Arsenault
f9c9936f9f AMDGPU: Fix assert on trunc from bitcast of build_vector
The v2i64 argument is lowered to a bitcast of v4i32 build_vector.
This would then attempt to use the i32-element as the source of the
vector truncate. This really would need to collect 2 elements from the
build_vector to produce the intended truncate.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353202 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-05 19:23:57 +00:00
Matt Arsenault
4966488eee GlobalISel: Consolidate load/store legalization
The fewerElementsVectors implementation for load/stores
handles the scalar reduction case just as well, so drop
the redundant code in narrowScalar. This also introduces
support for narrowing irregular size breakdowns for
scalars.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353125 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-05 00:26:12 +00:00
Matt Arsenault
e3333a3a48 GlobalISel: Implement narrowScalar for select
Don't handle vector conditions.

I think this can be merged in the future with
fewerElementsVectorSelect, although this becomes slightly tricky with
a vector condition.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353122 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-05 00:13:44 +00:00
Matt Arsenault
757bf95c73 GlobalISel: Combine g_extract with g_merge_values
Try to use the underlying source registers.

This enables legalization in more cases where some irregular
operations are widened and others narrowed.

This seems to make the test_combines_2 AArch64 test worse, since the
MERGE_VALUES has multiple uses. Since this should be required for
legalization, a hasOneUse check is probably inappropriate (or maybe
should only be used if the merge is legal?).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353121 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-04 23:41:59 +00:00
Matt Arsenault
7c4ac52d75 GlobalISel: Enforce operand types for constants
A number of of tests were using imm operands, not cimm. Since CSE
relies on the exact ConstantInt* pointer used, and implicit
conversions are generally evil, also enforce the bitsize of the types.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353113 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-04 23:29:31 +00:00
Matt Arsenault
733c08b6b3 GlobalISel: Fix not calling observer when legalizing bitcount ops
This was hiding bugs from never legalizing the source type.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353102 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-04 22:26:33 +00:00
Matt Arsenault
5d2f3ca3d6 AMDGPU: Don't rematerialize mov with implicit operands
This was pulling the mov used for register indexing on gfx9 out of the
loop.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353101 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-04 22:26:21 +00:00
Scott Linder
6186e8e51b [AMDGPU] Support emitting GOT relocations for function calls
Differential Revision: https://reviews.llvm.org/D57416

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353083 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-04 20:00:07 +00:00
Matt Arsenault
cd93dd44c7 AMDGPU/GlobalISel: Legalize select for v4s16
Also add some more select tests to help show future legalization
changes.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353045 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-04 14:04:52 +00:00