Commit Graph

206 Commits

Author SHA1 Message Date
Matt Arsenault
54dea4c5ee AMDGPU/GlobalISel: Legalize s1 source G_[SU]ITOFP
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371952 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-16 00:37:10 +00:00
Matt Arsenault
83c97ac441 AMDGPU/GlobalISel: Select S16->S32 fptoint
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371950 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-16 00:32:56 +00:00
Matt Arsenault
f00688b213 AMDGPU/GlobalISel: Legalize s32->s16 G_SITOFP/G_UITOFP
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371811 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-13 04:04:55 +00:00
Matt Arsenault
8fc9eed989 AMDGPU/GlobalISel: Legalize G_FFLOOR
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371803 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-13 01:48:15 +00:00
Matt Arsenault
bf6eeaf07e AMDGPU/GlobalISel: Legalize G_FMAD
Unlike SelectionDAG, treat this as a normally legalizable operation.
In SelectionDAG this is supposed to only ever formed if it's legal,
but I've found that to be restricting. For AMDGPU this is contextually
legal depending on whether denormal flushing is allowed in the use
function.

Technically we currently treat the denormal mode as a subtarget
feature, so custom lowering could be avoided. However I consider this
to be a defect, and this should be contextually dependent on the
controllable rounding mode of the parent function.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371800 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-13 00:44:35 +00:00
Matt Arsenault
cae6fe6cdb AMDGPU/GlobalISel: Select G_FABS/G_FNEG
f64 doesn't work yet because tablegen currently doesn't handlde
REG_SEQUENCE.

This does regress some multi use VALU fneg cases since now the
immediate remains in an SGPR, and more moves are used for legalizing
the xor. This is a SIFixSGPRCopies deficiency.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371540 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-10 17:19:46 +00:00
Matt Arsenault
6c1da931e7 AMDGPU/GlobalISel: RegBankSelect for G_ZEXTLOAD/G_SEXTLOAD
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371536 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-10 16:42:37 +00:00
Matt Arsenault
f37f483128 AMDGPU/GlobalISel: Legalize constant 32-bit loads
Legalize by casting to a 64-bit constant address. This isn't how the
DAG implements it, but it should.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371535 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-10 16:42:31 +00:00
Matt Arsenault
7b1d1c328c AMDGPU/GlobalISel: First pass at attempting to legalize load/stores
There's still a lot more to do, but this handles decomposing due to
alignment. I've gotten it to the point where nothing crashes or
infinite loops the legalizer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371533 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-10 16:20:14 +00:00
Matt Arsenault
989241eebf AMDGPU/GlobalISel: Fix insert point when lowering fminnum/fmaxnum
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371471 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-09 23:30:11 +00:00
Austin Kerbow
0bbba0846f AMDGPU/GlobalISel: Rename MIRBuilder to B. NFC
Reviewers: arsenm

Reviewed By: arsenm

Subscribers: kzhuravl, jvesely, wdng, nhaehnle, yaxunl, rovka, dstuttard, tpr, t-tye, hiraditya, Petar.Avramovic, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D67374

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371467 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-09 23:06:13 +00:00
Matt Arsenault
2cfa868dbf AMDGPU/GlobalISel: Legalize G_BUILD_VECTOR v2s16
Handle it the same way as G_BUILD_VECTOR_TRUNC. Arguably only
G_BUILD_VECTOR_TRUNC should be legal for this, but G_BUILD_VECTOR will
probably be more convenient in most cases.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371440 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-09 18:57:51 +00:00
Matt Arsenault
a81f890ca1 AMDGPU/GlobalISel: Implement LDS G_GLOBAL_VALUE
Handle the simple case that lowers to a constant.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371424 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-09 17:13:44 +00:00
Matt Arsenault
93eb1d1364 AMDGPU/GlobalISel: Legalize G_BUILD_VECTOR_TRUNC
Treat this as legal on gfx9 since it can use S_PACK_* instructions for
this.

This isn't used by anything yet. The same will probably apply to
16-bit G_BUILD_VECTOR without the trunc.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371423 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-09 17:04:18 +00:00
Matt Arsenault
ec69d2c9e6 AMDGPU/GlobalISel: Select G_PTR_MASK
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371412 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-09 15:46:13 +00:00
Matt Arsenault
cedf41585f AMDGPU/GlobalISel: Legalize wavefrontsize intrinsic
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371407 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-09 15:20:49 +00:00
Matt Arsenault
9f717fa86d AMDGPU: Add intrinsics for address space identification
The library currently uses ptrtoint and directly checks the queue ptr
for this, which counts as a pointer capture.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371009 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-05 02:20:39 +00:00
Matt Arsenault
ef8535baee AMDGPU/GlobalISel: Restore insert point when getting aperture
Avoids SSA violations in a future patch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371008 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-05 02:20:32 +00:00
Matt Arsenault
6c2a25a8f6 AMDGPU/GlobalISel: Fix placeholder value used for addrspacecast
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371007 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-05 02:20:29 +00:00
Matt Arsenault
b171b8eace GlobalISel: Add basic legalization for G_BITREVERSE
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@370979 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-04 20:46:15 +00:00
Matt Arsenault
d6d1c46508 AMDGPU/GlobalISel: Make 16-bit constants legal
This is mostly for the benefit of patterns which use 16-bit constants.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@370921 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-04 16:19:45 +00:00
Reid Kleckner
9a699c8345 Fix the build for MSVC builds using M_PI
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@370405 91177308-0d34-0410-b5e6-96231b3b80d8
2019-08-29 20:32:53 +00:00
Matt Arsenault
b11bd36479 AMDGPU/GlobalISel: Legalize sin/cos
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@370402 91177308-0d34-0410-b5e6-96231b3b80d8
2019-08-29 20:06:48 +00:00
Matt Arsenault
6881419669 AMDGPU/GlobalISel: Implement addrspacecast for 32-bit constant addrspace
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@370140 91177308-0d34-0410-b5e6-96231b3b80d8
2019-08-28 00:58:24 +00:00
Matt Arsenault
7c120abd60 GlobalISel: Implement moreElementsVector for G_UNMERGE_VALUES sources
This is necessary for handling <3 x s16> on AMDGPU, assuming this
should be handled as 2 separate legalization actions. The alternative
would be for fewerElementsVector to handle 3->2.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@369547 91177308-0d34-0410-b5e6-96231b3b80d8
2019-08-21 16:59:10 +00:00
Matt Arsenault
5508afab5c GlobalISel: Partially implement fewerElementsVector G_UNMERGE_VALUES
Odd sized vectors aren't handled yet.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@368713 91177308-0d34-0410-b5e6-96231b3b80d8
2019-08-13 16:26:28 +00:00
Matt Arsenault
b5027beff2 GlobalISel: Implement lower for G_SHUFFLE_VECTOR
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@368709 91177308-0d34-0410-b5e6-96231b3b80d8
2019-08-13 16:09:07 +00:00
Daniel Sanders
2d2b63fa73 [globalisel] Add G_SEXT_INREG
Summary:
Targets often have instructions that can sign-extend certain cases faster
than the equivalent shift-left/arithmetic-shift-right. Such cases can be
identified by matching a shift-left/shift-right pair but there are some
issues with this in the context of combines. For example, suppose you can
sign-extend 8-bit up to 32-bit with a target extend instruction.
  %1:_(s32) = G_SHL %0:_(s32), i32 24 # (I've inlined the G_CONSTANT for brevity)
  %2:_(s32) = G_ASHR %1:_(s32), i32 24
  %3:_(s32) = G_ASHR %2:_(s32), i32 1
would reasonably combine to:
  %1:_(s32) = G_SHL %0:_(s32), i32 24
  %2:_(s32) = G_ASHR %1:_(s32), i32 25
which no longer matches the special case. If your shifts and extend are
equal cost, this would break even as a pair of shifts but if your shift is
more expensive than the extend then it's cheaper as:
  %2:_(s32) = G_SEXT_INREG %0:_(s32), i32 8
  %3:_(s32) = G_ASHR %2:_(s32), i32 1
It's possible to match the shift-pair in ISel and emit an extend and ashr.
However, this is far from the only way to break this shift pair and make
it hard to match the extends. Another example is that with the right
known-zeros, this:
  %1:_(s32) = G_SHL %0:_(s32), i32 24
  %2:_(s32) = G_ASHR %1:_(s32), i32 24
  %3:_(s32) = G_MUL %2:_(s32), i32 2
can become:
  %1:_(s32) = G_SHL %0:_(s32), i32 24
  %2:_(s32) = G_ASHR %1:_(s32), i32 23

All upstream targets have been configured to lower it to the current
G_SHL,G_ASHR pair but will likely want to make it legal in some cases to
handle their faster cases.

To follow-up: Provide a way to legalize based on the constant. At the
moment, I'm thinking that the best way to achieve this is to provide the
MI in LegalityQuery but that opens the door to breaking core principles
of the legalizer (legality is not context sensitive). That said, it's
worth noting that looking at other instructions and acting on that
information doesn't violate this principle in itself. It's only a
violation if, at the end of legalization, a pass that checks legality
without being able to see the context would say an instruction might not be
legal. That's a fairly subtle distinction so to give a concrete example,
saying %2 in:
  %1 = G_CONSTANT 16
  %2 = G_SEXT_INREG %0, %1
is legal is in violation of that principle if the legality of %2 depends
on %1 being constant and/or being 16. However, legalizing to either:
  %2 = G_SEXT_INREG %0, 16
or:
  %1 = G_CONSTANT 16
  %2:_(s32) = G_SHL %0, %1
  %3:_(s32) = G_ASHR %2, %1
depending on whether %1 is constant and 16 does not violate that principle
since both outputs are genuinely legal.

Reviewers: bogner, aditya_nandakumar, volkan, aemerson, paquette, arsenm

Subscribers: sdardis, jvesely, wdng, nhaehnle, rovka, kristof.beyls, javed.absar, hiraditya, jrtc27, atanasyan, Petar.Avramovic, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61289

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@368487 91177308-0d34-0410-b5e6-96231b3b80d8
2019-08-09 21:11:20 +00:00
Matt Arsenault
abab6b93e0 GlobalISel: Lower scalarizing unmerge of a vector to shifts
AMDGPU sometimes has legal s16 and <2 x s16> operations, but all
registers are really 32-bit. An unmerge destination really should ben
widened to a 32-bit register. If widening a scalarizing vector with a
target size that matches the vector size, bitcast to integer and
extract the relevant bits with shifts.

I'm not sure if this is the right place for this. This could arguably
be part of widenScalar for the result. I also have a growing feeling
that we're missing a bitcast legalize action.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367604 91177308-0d34-0410-b5e6-96231b3b80d8
2019-08-01 19:10:05 +00:00
Matt Arsenault
9455340d7b AMDGPU/GlobalISel: Handle G_ATOMICRMW_FADD
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367509 91177308-0d34-0410-b5e6-96231b3b80d8
2019-08-01 03:33:15 +00:00
Matt Arsenault
9676b30e9f GlobalISel: moreElementsVector for G_LOAD/G_STORE
AMDGPU change and test is a placeholder until a future patch with
complete handling.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367503 91177308-0d34-0410-b5e6-96231b3b80d8
2019-08-01 01:44:22 +00:00
Austin Kerbow
2664d7d186 [AMDGPU/GlobalISel] Add llvm.amdgcn.fdiv.fast legalization.
Reviewers: arsenm

Reviewed By: arsenm

Subscribers: volkan, kzhuravl, jvesely, wdng, nhaehnle, yaxunl, rovka, dstuttard, tpr, t-tye, hiraditya, Petar.Avramovic, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64966

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367344 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-30 18:49:16 +00:00
Matt Arsenault
65ece44b15 AMDGPU/GlobalISel: Legalize GEP for other 32-bit address spaces
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366621 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-19 22:28:44 +00:00
Matt Arsenault
b18edea81b AMDGPU/GlobalISel: Select flat loads
Now that the patterns use the new PatFrag address space support, the
only blocker to importing most load patterns is the addressing mode
complex patterns.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366237 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-16 18:05:29 +00:00
Matt Arsenault
aeefee98c1 AMDGPU/GlobalISel: Fix test failures in release build
Apparently the check for legal instructions during instruction
select does not happen without an asserts build, so these would
successfully select in release, and fail in debug.

Make s16 and/or/xor legal. These can just be selected directly
to the 32-bit operation, as is already done in SelectionDAG, so just
make them legal.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366210 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-16 14:28:30 +00:00
Matt Arsenault
e86254e20e AMDGPU/GlobalISel: Custom legalize G_INSERT_VECTOR_ELT
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366116 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-15 19:43:04 +00:00
Matt Arsenault
a00c6918e7 AMDGPU/GlobalISel: Custom legalize G_EXTRACT_VECTOR_ELT
Turn the constant cases into G_EXTRACTs.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366115 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-15 19:40:59 +00:00
Matt Arsenault
e278479622 AMDGPU/GlobalISel: Widen vector extracts
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366103 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-15 18:31:10 +00:00
Matt Arsenault
2e07d5cebb GlobalISel: Legalization for G_FMINNUM/G_FMAXNUM
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@365658 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-10 16:31:19 +00:00
Tom Stellard
b97830df46 AMDGPU/GlobalISel: Add support for wide loads >= 256-bits
Summary:
This adds support for the most commonly used wide load types:
<8xi32>, <16xi32>, <4xi64>, and <8xi64>

Reviewers: arsenm

Reviewed By: arsenm

Subscribers: hiraditya, kzhuravl, jvesely, wdng, nhaehnle, yaxunl, rovka, kristof.beyls, dstuttard, tpr, t-tye, volkan, Petar.Avramovic, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D57399

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@365586 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-10 00:22:41 +00:00
Matt Arsenault
13b473d248 GlobalISel: Implement lower for G_FCOPYSIGN
In SelectionDAG AMDGPU treated these as legal, but this was mostly
because the bitcasts required for FP types were painful. Theoretically
the bitpattern should eventually match to bfi, so don't bother trying
to get the patterns to import.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@365583 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-09 23:34:29 +00:00
Matt Arsenault
9b44e7d33a AMDGPU/GlobalISel: Fix legality for G_BUILD_VECTOR
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@365575 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-09 22:48:04 +00:00
Matt Arsenault
0d10177cd3 AMDGPU/GlobalISel: Legalize more concat_vectors
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@365488 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-09 14:17:31 +00:00
Matt Arsenault
a568e61708 AMDGPU/GlobalISel: Make s16 G_ICMP legal
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@365486 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-09 14:10:43 +00:00
Matt Arsenault
fd0d82251d AMDGPU/GlobalISel: Select G_MERGE_VALUES
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@365482 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-09 14:02:20 +00:00
Matt Arsenault
d51fdec4e0 AMDGPU/GlobalISel: Handle more input argument intrinsics
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@364836 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-01 18:50:50 +00:00
Matt Arsenault
51744f169f AMDGPU/GlobalISel: Lower kernarg segment ptr intrinsics
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@364835 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-01 18:49:01 +00:00
Matt Arsenault
c087c304f7 AMDGPU/GlobalISel: Legalize workgroup ID intrinsics
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@364834 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-01 18:47:22 +00:00
Matt Arsenault
fa7c34bbaf AMDGPU/GlobalISel: Legalize workitem ID intrinsics
Tests don't cover the masked input path since non-kernel arguments
aren't lowered yet.

Test is copied directly from the existing test, with 2 additions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@364833 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-01 18:45:36 +00:00
Matt Arsenault
643e4ad7f2 AMDGPU/GlobalISel: Custom lower control flow intrinsics
Replace the brcond for the 2 cases that act as branches. For now
follow how the current system works, although I think we can
eventually get rid of the pseudos.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@364832 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-01 18:40:23 +00:00