67 Commits

Author SHA1 Message Date
Matt Arsenault
2415a7067f AMDGPU: Cleanup fmin/fmax legacy function
Use a more specific subtarget check and combine hasOneUse checks

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293726 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-01 00:42:40 +00:00
Tom Stellard
d367f44048 AMDGPU add support for spilling to a user sgpr pointed buffers
Summary:
This lets you select which sort of spilling you want, either s[0:1] or 64-bit loads from s[0:1].

Patch By: Dave Airlie

Reviewers: nhaehnle, arsenm, tstellarAMD

Reviewed By: arsenm

Subscribers: mareko, llvm-commits, kzhuravl, wdng, yaxunl, tony-tye

Differential Revision: https://reviews.llvm.org/D25428

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293000 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-25 01:25:13 +00:00
Jan Vesely
5709854b8d AMDGPU/R600: Serialize vector trunc stores to private AS
Add DUMMY_CHAIN SDNode to denote stores of interest

Bugzilla: https://llvm.org/bugs/show_bug.cgi?id=28915
Bugzilla: https://llvm.org/bugs/show_bug.cgi?id=30411

Differential Revision: https://reviews.llvm.org/D27964

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292651 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-20 21:24:26 +00:00
Jan Vesely
6d821c2f7c ADMGPU/EG,CM: Implement _noret global atomics
_RTN versions will be a lot more complicated

Differential Revision: https://reviews.llvm.org/D28067

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292162 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-16 21:20:13 +00:00
Diana Picus
8a47810cd6 [CodeGen] Rename MachineInstrBuilder::addOperand. NFC
Rename from addOperand to just add, to match the other method that has been
added to MachineInstrBuilder for adding more than just 1 operand.

See https://reviews.llvm.org/D28057 for the whole discussion.

Differential Revision: https://reviews.llvm.org/D28556

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291891 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-13 09:58:52 +00:00
Jan Vesely
0835374acb AMDGPU/R600: Don't use REGISTER_{LOAD,STORE} ISD nodes
This will make transition to SCRATCH_MEMORY easier

Differential Revision: https://reviews.llvm.org/D24746

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291279 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-06 21:00:46 +00:00
Eugene Zelenko
43dec7d682 [AMDGPU, PowerPC, TableGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289282 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-09 22:06:55 +00:00
Matt Arsenault
350d0dab1e AMDGPU: Refactor exp instructions
Structure the definitions a bit more like the other classes.

The main change here is to split EXP with the done bit set
to a separate opcode, so we can set mayLoad = 1 so that it won't
be reordered before the other exp stores, since this has the special
constraint that if the done bit is set then this should be the last
exp in she shader.

Previously all exp instructions were inferred to have unmodeled
side effects.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288695 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-05 20:23:10 +00:00
Tom Stellard
77361ae206 AMDGPU: Refactor kernel argument lowering
Summary:
The main challenge in lowering kernel arguments for AMDGPU is determing the
memory type of the argument.  The generic calling convention code assumes
that only legal register types can be stored in memory, but this is not the
case for AMDGPU.

This consolidates all the logic AMDGPU uses for deducing memory types into a single
function.  This will make it much easier to support different ABIs in the future.

Reviewers: arsenm

Subscribers: arsenm, wdng, nhaehnle, llvm-commits, yaxunl

Differential Revision: https://reviews.llvm.org/D24614

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281781 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-16 21:53:00 +00:00
Justin Lebar
c71d5b41ef [CodeGen] Split out the notions of MI invariance and MI dereferenceability.
Summary:
An IR load can be invariant, dereferenceable, neither, or both.  But
currently, MI's notion of invariance is IR-invariant &&
IR-dereferenceable.

This patch splits up the notions of invariance and dereferenceability at
the MI level.  It's NFC, so adds some probably-unnecessary
"is-dereferenceable" checks, which we can remove later if desired.

Reviewers: chandlerc, tstellarAMD

Subscribers: jholewinski, arsenm, nemanjai, llvm-commits

Differential Revision: https://reviews.llvm.org/D23371

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281151 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-11 01:38:58 +00:00
Jan Vesely
bf9a94abff AMDGPU/R600: EXTRACT_VECT_ELT should only bypass BUILD_VECTOR if the vectors have the same number of elements.
Fixes R600 piglit regressions since r280298

Differential Revision: https://reviews.llvm.org/D24174

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280535 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-02 20:13:19 +00:00
Jan Vesely
1c5de877c4 AMDGPU/R600: Expand unaligned writes to local and global AS
LOCAL and GLOBAL AS only
PRIVATE needs special treatment

Differential Revision: https://reviews.llvm.org/D23971

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280526 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-02 19:07:06 +00:00
Jan Vesely
460ff94b82 AMDGPU/R600: Cleanup DAGCombine
Move SDLoc initialization to comon place.
fall back to AMDGPU version in one place

Differential Revision: https://reviews.llvm.org/D23900

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280030 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-29 23:21:46 +00:00
Jan Vesely
79944cc240 AMDGPU/R600: Remove MergeVectorStores from legalization
This is handled by DAGCombiner in a more generic way

Differential Revision: https://reviews.llvm.org/D23970

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280019 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-29 22:05:06 +00:00
Jan Vesely
3ed393863d AMDGPU/R600: Enable Load combine
Fix and improve tests

Differential Revision: https://reviews.llvm.org/D23899

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279925 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-27 19:09:43 +00:00
Justin Bogner
6673ea81f6 Replace "fallthrough" comments with LLVM_FALLTHROUGH
This is a mechanical change of comments in switches like fallthrough,
fall-through, or fall-thru to use the LLVM_FALLTHROUGH macro instead.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278902 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-17 05:10:15 +00:00
Matt Arsenault
129b27ab08 AMDGPU/R600: Remove macros
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278588 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-13 01:43:46 +00:00
Hans Wennborg
46b81ba7eb Fix more dereferenced end() iterators after r278532
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278587 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-13 01:12:49 +00:00
Matt Arsenault
252b5ebfdd AMDGPU/R600: Remove dead custom inserters
The intrinsics for these were removed, so this is dead.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276805 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-26 21:03:38 +00:00
Matt Arsenault
d506595769 AMDGPU: Make AMDGPUMachineFunction fields private
ABIArgOffset is a problem because properly fsetting the
KernArgSize requires that the reserved area before the
real kernel arguments be correctly aligned, which requires
fixing clover.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276766 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-26 16:45:58 +00:00
Jan Vesely
4a44da0c82 AMDGPU: Remove read_workdim intrinsic
Differential revision: https://reviews.llvm.org/D22732

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276682 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-25 20:17:02 +00:00
Matt Arsenault
c28b821881 AMDGPU: Delete more dead code
Remove dead code from r600 intrinsic removal.
Remove unset members, rename StackSize to be less ambiguous.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276436 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-22 17:01:25 +00:00
Matt Arsenault
7488ab3114 AMDGPU: Fix i1 fp_to_int
R600's i1 fp_to_uint selected but was incorrect according to
what instcombine constant folds to.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276435 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-22 17:01:21 +00:00
Matt Arsenault
5fecfa22e5 AMDGPU: Fix TargetPrefix for remaining r600 intrinsics
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275619 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-15 21:27:08 +00:00
Matt Arsenault
7150fbf236 AMDGPU: Remove legacy rsq.clamped intrinsic
Mesa still has a use of llvm.AMDGPU.rsq.f64 remaining.

Also fix mismatch with non-IEEE rsq selecting to IEEE rsq.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275617 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-15 21:26:52 +00:00
Justin Lebar
b2d6ad7cfd [SelectionDAG] Get rid of bool parameters in SelectionDAG::getLoad, getStore, and friends.
Summary:
Instead, we take a single flags arg (a bitset).

Also add a default 0 alignment, and change the order of arguments so the
alignment comes before the flags.

This greatly simplifies many callsites, and fixes a bug in
AMDGPUISelLowering, wherein the order of the args to getLoad was
inverted.  It also greatly simplifies the process of adding another flag
to getLoad.

Reviewers: chandlerc, tstellarAMD

Subscribers: jholewinski, arsenm, jyknight, dsanders, nemanjai, llvm-commits

Differential Revision: http://reviews.llvm.org/D22249

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275592 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-15 18:27:10 +00:00
Matt Arsenault
4d120e9b24 AMDGPU/R600: Delete/rename intrinsics no longer used by mesa
Use the replacement pass to update the tests, and delete old names.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275375 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-14 05:47:17 +00:00
Matt Arsenault
04e6d2604d AMDGPU/R600: Remove intrinsics with no tests and no users
Mesa removed this path, so nothing is using these anymore.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275372 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-14 05:23:23 +00:00
Jan Vesely
e2b4643334 AMDGPU/R600: Add implicitarg.ptr intrinsic
Differential Revision: http://reviews.llvm.org/D21622

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275024 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-10 21:20:29 +00:00
Duncan P. N. Exon Smith
a204da23db CodeGen: Use MachineInstr& in TargetLowering, NFC
This is a mechanical change to make TargetLowering API take MachineInstr&
(instead of MachineInstr*), since the argument is expected to be a valid
MachineInstr.  In one case, changed a parameter from MachineInstr* to
MachineBasicBlock::iterator, since it was used as an insertion point.

As a side effect, this removes a bunch of MachineInstr* to
MachineBasicBlock::iterator implicit conversions, a necessary step
toward fixing PR26753.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@274287 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-30 22:52:52 +00:00
Duncan P. N. Exon Smith
567409db69 CodeGen: Use MachineInstr& in TargetInstrInfo, NFC
This is mostly a mechanical change to make TargetInstrInfo API take
MachineInstr& (instead of MachineInstr* or MachineBasicBlock::iterator)
when the argument is expected to be a valid MachineInstr.  This is a
general API improvement.

Although it would be possible to do this one function at a time, that
would demand a quadratic amount of churn since many of these functions
call each other.  Instead I've done everything as a block and just
updated what was necessary.

This is mostly mechanical fixes: adding and removing `*` and `&`
operators.  The only non-mechanical change is to split
ARMBaseInstrInfo::getOperandLatencyImpl out from
ARMBaseInstrInfo::getOperandLatency.  Previously, the latter took a
`MachineInstr*` which it updated to the instruction bundle leader; now,
the latter calls the former either with the same `MachineInstr&` or the
bundle leader.

As a side effect, this removes a bunch of MachineInstr* to
MachineBasicBlock::iterator implicit conversions, a necessary step
toward fixing PR26753.

Note: I updated WebAssembly, Lanai, and AVR (despite being
off-by-default) since it turned out to be easy.  I couldn't run tests
for AVR since llc doesn't link with it turned on.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@274189 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-30 00:01:54 +00:00
Matt Arsenault
759ed7e410 AMDGPU: Cleanup subtarget handling.
Split AMDGPUSubtarget into amdgcn/r600 specific subclasses.
This removes most of the static_casting of the basic codegen
classes everywhere, and tries to restrict the features
visible on the wrong target.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@273652 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-24 06:30:11 +00:00
Benjamin Kramer
af18e017d2 Pass DebugLoc and SDLoc by const ref.
This used to be free, copying and moving DebugLocs became expensive
after the metadata rewrite. Passing by reference eliminates a ton of
track/untrack operations. No functionality change intended.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272512 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-12 15:39:02 +00:00
Matt Arsenault
29d0ea4bc8 AMDGPU: Cleanup load tests
There are a lot of different kinds of loads to test for,
and these were scattered around inconsistently with
some redundancy. Try to comprehensively test all loads
in a consistent way.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271571 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-02 19:54:26 +00:00
Matt Arsenault
be522c6214 AMDGPU: Cleanup lowering actions
These are kind of a mess and hard to follow, particularly
for loads and stores. Fix various redundant, unnecessary
and dead settings.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270307 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-21 02:27:49 +00:00
Jan Vesely
350e40ffb2 AMDGPU/R600: Use correct number of vector elements when lowering private loads
Reviewer: tstellardAMD, arsenm

Subscribers: arsenm, kzhuravl, llvm-commits

Differential Revision: http://reviews.llvm.org/D20032

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269725 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-16 23:56:32 +00:00
Jan Vesely
d75c9ce18b AMDGPU/R600: Fold global address operand
Reviewers: tstellard

Subscribers: arsenm

Differential Revision: http://reviews.llvm.org/D19793

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269480 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-13 20:39:31 +00:00
Jan Vesely
fbff874b03 AMDGPU/R600: Implement memory loads from constant AS
Reviewers: tstellard

Subscribers: arsenm

Differential Revision: http://reviews.llvm.org/D19792

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269479 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-13 20:39:29 +00:00
Tom Stellard
c6d991c3a5 AMDGPU: Move R600 specific code out of AMDGPUISelLowering.cpp
Reviewers: arsenm

Subscribers: jvesely, arsenm, llvm-commits

Differential Revision: http://reviews.llvm.org/D19736

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268267 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-02 18:05:17 +00:00
Ahmed Bougacha
fb52dfa04d [CodeGen] Add getBuildVector and getSplatBuildVector helpers. NFCI.
Differential Revision: http://reviews.llvm.org/D17176

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267606 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-26 21:15:30 +00:00
Matt Arsenault
176a9b27e6 AMDGPU: Remove custom load/store scalarization
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266385 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-14 23:31:26 +00:00
Nicolai Haehnle
ea7a0c0467 AMDGPU: Add a shader calling convention
This makes it possible to distinguish between mesa shaders
and other kernels even in the presence of compute shaders.

Patch By: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>

Differential Revision: http://reviews.llvm.org/D18559

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@265589 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-06 19:40:20 +00:00
Sanjay Patel
3e87fcf215 [DAG] use isUndef() ; NFCI
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263448 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-14 17:28:46 +00:00
Matt Arsenault
7137d0dce6 AMDGPU: R600 code splitting cleanup
Move a few functions only used by R600 to R600 specific code,
fix header macros to stop using R600, mark classes as final.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263204 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-11 08:00:27 +00:00
Matt Arsenault
11f560a199 AMDGPU: Move function only used by R600
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262853 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-07 21:10:13 +00:00
Matt Arsenault
2f65ff664c AMDGPU/R600: Implement allowsMisalignedMemoryAccess
This avoids some test regressions in a future commit
when unaligned operations are expanded when they
have custom lowering.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@261570 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-22 21:04:16 +00:00
Matt Arsenault
a4c1dc826a AMDGPU: Rename intrinsic to better match instruction name
Also fixes missing f32 test.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260780 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-13 01:03:00 +00:00
Matt Arsenault
b49a0edca2 AMDGPU: Split R600 and SI store lowering
These were only sharing some somewhat incorrect
logic for when to scalarize or split vectors.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260490 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-11 05:32:46 +00:00
Matt Arsenault
e6640ee461 AMDGPU: Split R600 and SI load lowering
These weren't actually sharing anything in the common
LowerLOAD.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260398 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-10 18:21:39 +00:00
Ahmed Bougacha
3248c624fa [CodeGen] Prefer "if (SDValue R = ...)" to "if (R.getNode())". NFCI.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260316 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-09 22:54:12 +00:00