Commit Graph

241 Commits

Author SHA1 Message Date
Matt Arsenault
62b3258a7c AMDGPU: Add replacement export intrinsics
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292205 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-17 07:26:53 +00:00
Benjamin Kramer
1fb85c6675 Apply clang-tidy's performance-unnecessary-value-param to LLVM.
With some minor manual fixes for using function_ref instead of
std::function. No functional change intended.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291904 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-13 14:39:03 +00:00
Diana Picus
8a47810cd6 [CodeGen] Rename MachineInstrBuilder::addOperand. NFC
Rename from addOperand to just add, to match the other method that has been
added to MachineInstrBuilder for adding more than just 1 operand.

See https://reviews.llvm.org/D28057 for the whole discussion.

Differential Revision: https://reviews.llvm.org/D28556

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291891 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-13 09:58:52 +00:00
Matt Arsenault
f7c0d4013c AMDGPU: Add Assert[SZ]Ext during argument load creation
For i16 zeroext arguments when i16 was a legal type, the
known bits information from the truncate was lost. Insert
a zeroext so the known bits optimizations work with the 32-bit
loads.

Fixes code quality regressions vs. SI in min.ll test.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291461 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-09 18:52:39 +00:00
Jan Vesely
0835374acb AMDGPU/R600: Don't use REGISTER_{LOAD,STORE} ISD nodes
This will make transition to SCRATCH_MEMORY easier

Differential Revision: https://reviews.llvm.org/D24746

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291279 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-06 21:00:46 +00:00
Jan Vesely
bf64cb107c AMDGPU/SI: Implement sendmsghalt intrinsic
v2: expose using amdgcn prefix

Differential Revision: https://reviews.llvm.org/D23511

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290977 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-04 18:06:55 +00:00
Matt Arsenault
ad47821c65 AMDGPU: Use i16 for i16 shift amount
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290351 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-22 16:36:25 +00:00
Matt Arsenault
8d973070a0 AMDGPU: Use i16 comparison instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290348 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-22 16:27:11 +00:00
Matt Arsenault
46e5f1c88d AMDGPU: Swap order of operands in fadd/fsub combine
FMA is canonicalized to constant in the middle operand. Do
the same so fmad matches and avoid an extra combine step.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290313 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-22 04:03:40 +00:00
Matt Arsenault
121f8654d3 AMDGPU: Check fast math flags in fadd/fsub combines
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290312 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-22 04:03:35 +00:00
Matt Arsenault
ff4096b8f8 AMDGPU: Form more FMAs if fusion is allowed
Extend the existing fadd/fsub->fmad combines to produce
FMA if allowed.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290311 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-22 03:55:35 +00:00
Matt Arsenault
121cd504bf AMDGPU: Move combines into separate functions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290309 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-22 03:44:42 +00:00
Matt Arsenault
75c32f5150 AMDGPU: Enable some f32 fadd/fsub combines for f16
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290308 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-22 03:40:39 +00:00
Matt Arsenault
cee1c4614a AMDGPU: Implement isFMAFasterThanFMulAndFAdd for f16
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290307 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-22 03:21:48 +00:00
Matt Arsenault
a8dff18ebc AMDGPU: Allow rcp and rsq usage with f16
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290302 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-22 03:05:44 +00:00
Matt Arsenault
4bb99910b0 AMDGPU: Custom lower f16 fdiv
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290301 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-22 03:05:41 +00:00
Matt Arsenault
0bb2ef4a14 AMDGPU: Implement f16 fcanonicalize
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290300 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-22 03:05:37 +00:00
Matt Arsenault
256f8018fa AMDGPU: Allow 16-bit types in inline asm constraints
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290193 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-20 19:06:12 +00:00
Tom Stellard
38206ae07e AMDGPU/SI: Add a MachineMemOperand when lowering llvm.amdgcn.buffer.load.*
Reviewers: arsenm, nhaehnle, mareko

Subscribers: kzhuravl, wdng, yaxunl, llvm-commits, tony-tye

Differential Revision: https://reviews.llvm.org/D27834

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290184 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-20 17:19:44 +00:00
Tom Stellard
11d071bf72 AMDGPU/SI: Add a MachineMemOperand to MIMG instructions
Summary:
Without a MachineMemOperand, the scheduler was assuming MIMG instructions
were ordered memory references, so no loads or stores could be reordered
across them.

Reviewers: arsenm

Subscribers: arsenm, kzhuravl, wdng, nhaehnle, yaxunl, tony-tye

Differential Revision: https://reviews.llvm.org/D27536

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290179 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-20 15:52:17 +00:00
Matt Arsenault
0b698fea77 AMDGPU: Select branch on undef to uniform scc branch
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289877 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-15 21:57:11 +00:00
Alexander Timofeev
82c3914258 Fix for regression after Global Load Scalarization patch
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289822 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-15 15:17:19 +00:00
Matt Arsenault
f811020a19 AMDGPU: Fix isTypeDesirableForOp for i16
This should do nothing for targets without i16.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289235 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-09 17:57:43 +00:00
Matt Arsenault
beec226db5 AMDGPU: Make f16 ConstantFP legal
Not having this legal led to combine failures, resulting
in dumb things like bitcasts of constants not being folded
away.

The only reason I'm leaving the v_mov_b32 hack that f32
already uses is to avoid madak formation test regressions.
PeepholeOptimizer has an ordering issue where the immediate
fold attempt is into the sgpr->vgpr copy instead of the actual
use. Running it twice avoids that problem.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289096 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-08 20:14:46 +00:00
Alexander Timofeev
9ae2b48b84 [AMDGPU] Scalarization of global uniform loads.
Summary:
LC can currently select scalar load for uniform memory access
basing on readonly memory address space only. This restriction
originated from the fact that in HW prior to VI vector and scalar caches
are not coherent. With MemoryDependenceAnalysis we can check that the
memory location corresponding to the memory operand of the LOAD is not
clobbered along the all paths from the function entry.

Reviewers: rampitec, tstellarAMD, arsenm

Subscribers: wdng, arsenm, nhaehnle

Differential Revision: https://reviews.llvm.org/D26917

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289076 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-08 17:28:47 +00:00
Tom Stellard
c53f76cc0b AMDGPU : Add S_SETREG instructions to fix fdiv precision issues.
Patch By: Wei Ding

Summary: This patch fixes the fdiv precision issues.

Reviewers: b-sumner, cfang, wdng, arsenm

Subscribers: kzhuravl, nhaehnle, yaxunl, tony-tye

Differential Revision: https://reviews.llvm.org/D26424

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288879 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-07 02:42:15 +00:00
Tom Stellard
89ce12495b AMDGPU: Add llvm.amdgcn.interp.mov intrinsic
Reviewers: arsenm, nhaehnle

Subscribers: kzhuravl, wdng, yaxunl, tony-tye, llvm-commits

Differential Revision: https://reviews.llvm.org/D26725

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288865 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-06 23:52:13 +00:00
Matt Arsenault
350d0dab1e AMDGPU: Refactor exp instructions
Structure the definitions a bit more like the other classes.

The main change here is to split EXP with the done bit set
to a separate opcode, so we can set mayLoad = 1 so that it won't
be reordered before the other exp stores, since this has the special
constraint that if the done bit is set then this should be the last
exp in she shader.

Previously all exp instructions were inferred to have unmodeled
side effects.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288695 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-05 20:23:10 +00:00
Matt Arsenault
76a17e03e0 AMDGPU: Implement isCheapAddrSpaceCast
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288523 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-02 18:12:53 +00:00
Matt Arsenault
c562d96ea6 AMDGPU: Use SGPR_64 for argument lowerings
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288190 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-29 19:39:48 +00:00
Tom Stellard
0bada48294 AMDGPU/SI: Use float as the operand type for amdgcn.interp intrinsics
Reviewers: arsenm, nhaehnle

Subscribers: kzhuravl, wdng, yaxunl, llvm-commits, tony-tye

Differential Revision: https://reviews.llvm.org/D26724

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287962 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-26 02:26:04 +00:00
Marek Olsak
2a24827c23 AMDGPU/SI: Add back reverted SGPR spilling code, but disable it
suggested as a better solution by Matt

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287942 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-25 17:37:09 +00:00
Marek Olsak
2acdc08776 Revert "AMDGPU: Make m0 unallocatable"
This reverts commit 124ad83dae.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287932 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-25 16:03:15 +00:00
Matt Arsenault
124ad83dae AMDGPU: Make m0 unallocatable
m0 may need to be written for spill code, so
we don't want general code uses relying on the
value stored in it.

This introduces a few code quality regressions where copies
from m0 are not coalesced into copies of a copy of m0.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287841 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-24 00:26:40 +00:00
Matt Arsenault
13892fc867 AMDGPU: Fix unused variable warning
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287362 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-18 18:33:36 +00:00
Matt Arsenault
cbafc5829d AMDGPU: Fix crash on illegal type for inlineasm
There are still crashes on non-MVT types in other
places.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287310 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-18 04:42:57 +00:00
Konstantin Zhuravlyov
0c92298282 [AMDGPU] Custom lower f16 = fp_round f64
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287203 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-17 04:28:37 +00:00
Konstantin Zhuravlyov
8fd8772fcd [AMDGPU] Promote f16/i16 conversions to f32/i32
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287201 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-17 04:00:46 +00:00
Konstantin Zhuravlyov
54556b5c81 [AMDGPU] Expand br_cc for f16
Differential Revision: https://reviews.llvm.org/D26732


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287199 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-17 03:49:01 +00:00
Konstantin Zhuravlyov
7f4d4fdf3f [AMDGPU] Handle f16 select{_cc}
- Select `select` to `v_cndmask_b32`
- Expand `select_cc`
- Refactor patterns

Differential Revision: https://reviews.llvm.org/D26714


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287074 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-16 03:16:26 +00:00
Changpeng Fang
af76145600 AMDGPU/SI: Support data types other than V4f32 in image intrinsics
Summary:
  Extend image intrinsics to support data types of V1F32 and V2F32.

  TODO: we should define a mapping table to change the opcode for data type of V2F32 but just one channel is active,
  even though such case should be very rare.

Reviewers:
  tstellarAMD

Differential Revision:
  http://reviews.llvm.org/D26472

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286860 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-14 18:33:18 +00:00
Konstantin Zhuravlyov
9027123253 [AMDGPU] Add f16 support (VI+)
Differential Revision: https://reviews.llvm.org/D25975


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286753 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-13 07:01:11 +00:00
Tom Stellard
da5a5c79fa AMDGPU/SI: Promote i16 = fp_[us]int f32 for VI
Summary: This fixes a regression caused by r286464.

Reviewers: arsenm

Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, llvm-commits, tony-tye

Differential Revision: https://reviews.llvm.org/D26570

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286687 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-12 00:19:11 +00:00
Tom Stellard
0deee390af AMDGPU: Add VI i16 support
Patch By: Wei Ding

Differential Revision: https://reviews.llvm.org/D18049

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286464 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-10 16:02:37 +00:00
Tom Stellard
ae152bde49 Revert "AMDGPU: Add VI i16 support"
This reverts commit r285939 and r285948.  These broke some conformance tests.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@285995 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-04 13:06:34 +00:00
Tom Stellard
7c173dd5fa AMDGPU: Add VI i16 support
Patch By: Wei Ding

Differential Revision: https://reviews.llvm.org/D18049

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@285939 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-03 17:13:50 +00:00
Tom Stellard
1a633d108a AMDGPU/SI: Don't emit multi-dword flat memory ops when they might access scratch
Summary:
A single flat memory operations that might access the scratch buffer
can only access MaxPrivateElementSize bytes.

Reviewers: arsenm

Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, tony-tye, llvm-commits

Differential Revision: https://reviews.llvm.org/D25788

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@285198 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-26 14:38:47 +00:00
Nicolai Haehnle
e07033aa42 AMDGPU: Fix Two Address problems with v_movreld
Summary:
The v_movreld machine instruction is used with three operands that are
in a sense tied to each other (the explicit VGPR_32 def and the implicit
VGPR_NN def and use). There is no way to express that using the currently
available operand bits, and indeed there are cases where the Two Address
instructions pass does the wrong thing.

This patch introduces a new set of pseudo instructions that are identical
in intended semantics as v_movreld, but they only have two tied operands.

Having to add a new set of pseudo instructions is admittedly annoying, but
it's a fairly straightforward and solid approach. The only alternative I
see is to try to teach the Two Address instructions pass about Three Address
instructions, and I'm afraid that's trickier and is going to end up more
fragile.

Note that v_movrels does not suffer from this problem, and so this patch
does not touch it.

This fixes several GL45-CTS.shaders.indexing.* tests.

Reviewers: tstellarAMD, arsenm

Subscribers: kzhuravl, wdng, yaxunl, llvm-commits, tony-tye

Differential Revision: https://reviews.llvm.org/D25633

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284980 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-24 14:56:02 +00:00
Konstantin Zhuravlyov
25a30cdacb [AMDGPU] Perform uchar to float combine for ISD::SINT_TO_FP
This will prevent following regression when enabling i16 support (D18049):
  test/CodeGen/AMDGPU/cvt_f32_ubyte.ll

Differential Revision: https://reviews.llvm.org/D25805


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284891 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-21 22:10:03 +00:00
Konstantin Zhuravlyov
e0e811c4ce [AMDGPU] Emit constant address space data in .rodata section and use relocations instead of fixups (amdhsa only)
Differential Revision: https://reviews.llvm.org/D25693


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284759 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-20 18:12:38 +00:00