36 Commits

Author SHA1 Message Date
Wei Ding 29de9d738e Implement custom lowering for ISD::CTTZ_ZERO_UNDEF and ISD::CTTZ.
Differential Revision: http://reviews.llvm.org/D37348

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315610 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-12 19:37:14 +00:00
Matt Arsenault 45424dbebb AMDGPU: Start adding tail call support
Handle the sibling call cases.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310753 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-11 20:42:08 +00:00
Matt Arsenault 43950949ad AMDGPU: Initial implementation of calls
Includes a hack to fix the type selected for
the GlobalAddress of the function, which will be
fixed by changing the default datalayout to use
generic pointers for 0.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309732 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-01 19:54:18 +00:00
Stanislav Mekhanoshin 1a1f544263 [AMDGPU] simplify add x, *ext (setcc) => addc|subb x, 0, setcc
This simplification allows to avoid generating v_cndmask_b32
to serialize condition code between compare and use.

Differential Revision: https://reviews.llvm.org/D34300

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@305962 91177308-0d34-0410-b5e6-96231b3b80d8
2017-06-21 22:05:06 +00:00
Matt Arsenault a0540d3468 AMDGPU: Start defining a calling convention
Partially implement callee-side for arguments and return values.
byval doesn't work properly, and most likely sret or other on-stack
return values most as well.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@303308 91177308-0d34-0410-b5e6-96231b3b80d8
2017-05-17 21:56:25 +00:00
Marek Olsak 007530e1a2 AMDGPU: Add new amdgcn.init.exec intrinsics
v2: More tests, bug fixes, cosmetic changes.

Subscribers: arsenm, kzhuravl, wdng, nhaehnle, yaxunl, dstuttard, tpr, llvm-commits, t-tye

Differential Revision: https://reviews.llvm.org/D31762

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@301677 91177308-0d34-0410-b5e6-96231b3b80d8
2017-04-28 20:21:58 +00:00
Matt Arsenault 666020a37d AMDGPU: Move trap lowering to DAG
Fixes traps in any block besides the entry block,
and fixes depending on a live-in physical register
by using a virtual register copy.

Also happens to stop emitting a nop in the case
debug trap is not supported.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@301206 91177308-0d34-0410-b5e6-96231b3b80d8
2017-04-24 17:49:13 +00:00
Matt Arsenault c4de629ce2 AMDGPU: Remove unnecessary ands when f16 is legal
Add a new node to act as a fancy bitcast from f16 operations to
i32 that implicitly zero the high 16-bits of the result.

Alternatively could try making v2f16 legal and canonicalizing
on build_vectors.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@299246 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-31 19:53:03 +00:00
Matt Arsenault dc55587b7f AMDGPU: Rename SI_RETURN
This is used for a specific type of return to a shader part's
epilog code. Rename to try avoiding confusion from a true
call's return.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@298452 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-21 22:18:10 +00:00
Matt Arsenault 33eb1b078c AMDGPU: Cleanup control flow intrinsics
Move backend internal intrinsics along with the rest of the
normal intrinsics, and use the Intrinsic::getDeclaration
API instead of manually constructing the type list.

It's surprising this was working before. fdiv.fast had
the wrong number of parameters. The control flow intrinsic
declaration attributes were not being applied, and
their types were inconsistent. The actual IR use types
did not match the declaration, and were closer to the
types used for the patterns. The brcond lowering
was changing the types, so introduce new nodes for those.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@298119 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-17 20:41:45 +00:00
Matt Arsenault 0c52bece01 AMDGPU: Fix unnecessary ands when packing f16 vectors
computeKnownBits didn't handle fp_to_fp16 to report
the high bits as 0. ARM maps the generic node to an instruction
that does not modify the high bits of the register, so introduce
a target node where the high bits are known 0.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@297873 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-15 19:04:26 +00:00
Wei Ding 5d1e915557 AMDGPU : Replace FMAD with FMA when denormals are enabled.
Differential Revision: http://reviews.llvm.org/D29958

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296186 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-24 23:00:29 +00:00
Matt Arsenault 7d65faa5cc AMDGPU: Add cvt.pkrtz intrinsic
Convert llvm.SI.packf16 test uses

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295797 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-22 00:27:34 +00:00
Matt Arsenault aac82e218f AMDGPU: Redefine clamp node as clamp 0.0-1.0
Change implementation to use max instead of add.
min/max/med3 do not flush denormals regardless of the mode,
so it is OK to use it whether or not they are enabled.

Also allow using clamp with f16, and use knowledge
of dx10_clamp.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295788 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-21 23:35:48 +00:00
Matt Arsenault 644ed4d33f AMDGPU: Remove dead node definitions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295247 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-15 22:23:04 +00:00
Jan Vesely 5709854b8d AMDGPU/R600: Serialize vector trunc stores to private AS
Add DUMMY_CHAIN SDNode to denote stores of interest

Bugzilla: https://llvm.org/bugs/show_bug.cgi?id=28915
Bugzilla: https://llvm.org/bugs/show_bug.cgi?id=30411

Differential Revision: https://reviews.llvm.org/D27964

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292651 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-20 21:24:26 +00:00
Matt Arsenault 62b3258a7c AMDGPU: Add replacement export intrinsics
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292205 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-17 07:26:53 +00:00
Jan Vesely bf64cb107c AMDGPU/SI: Implement sendmsghalt intrinsic
v2: expose using amdgcn prefix

Differential Revision: https://reviews.llvm.org/D23511

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290977 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-04 18:06:55 +00:00
Tom Stellard c53f76cc0b AMDGPU : Add S_SETREG instructions to fix fdiv precision issues.
Patch By: Wei Ding

Summary: This patch fixes the fdiv precision issues.

Reviewers: b-sumner, cfang, wdng, arsenm

Subscribers: kzhuravl, nhaehnle, yaxunl, tony-tye

Differential Revision: https://reviews.llvm.org/D26424

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288879 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-07 02:42:15 +00:00
Matt Arsenault 350d0dab1e AMDGPU: Refactor exp instructions
Structure the definitions a bit more like the other classes.

The main change here is to split EXP with the done bit set
to a separate opcode, so we can set mayLoad = 1 so that it won't
be reordered before the other exp stores, since this has the special
constraint that if the done bit is set then this should be the last
exp in she shader.

Previously all exp instructions were inferred to have unmodeled
side effects.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288695 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-05 20:23:10 +00:00
Matt Arsenault 95ec13b22a AMDGPU: Select mulhi 24-bit instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279902 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-27 01:32:27 +00:00
Wei Ding ee8c4ca1e1 AMDGPU : Add intrinsics for compare with the full wavefront result
Differential Revision: http://reviews.llvm.org/D22482

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276998 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-28 16:42:13 +00:00
Matt Arsenault ee4cdb7b75 AMDGPU: Add fp legacy instruction intrinsics
This could use some additional optimization work
to use mad/mac legacy.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276764 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-26 16:45:45 +00:00
Matt Arsenault 1ce58d721f AMDGPU: Only use legal inline immediates with kill pseudo
Only if the value is negative or positive is what matters,
so use a constant that doesn't require an instruction to
materialize.

These should really just emit the write exec directly,
but for stick with the kill pseudo-terminator.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275988 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-19 16:27:56 +00:00
Matt Arsenault bb09cfd86f AMDGPU: Add intrinsic for s_flbit_i32/v_ffbh_i32
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275871 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-18 18:35:05 +00:00
Matt Arsenault e22857013f AMDGPU: Fix verifier errors in SILowerControlFlow
The main sin this was committing was using terminator
instructions in the middle of the block, and then
not updating the block successors / predecessors.
Split the blocks up to avoid this and introduce new
pseudo instructions for branches taken with exec masking.

Also use a pseudo instead of emitting s_endpgm and erasing
it in the special case of a non-void return.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@273467 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-22 20:15:28 +00:00
Jan Vesely 03bb0f57a8 AMDGPU: Make CONST_DATA_PTR available to R600
Rename to AMDGPUconstdata_ptr

Reviewers: tstellard

Subscribers: arsenm

Differential Revision: http://reviews.llvm.org/D19786

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269474 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-13 20:39:18 +00:00
Tom Stellard 059753cf8e AMDGPU: Implement {BUFFER,FLAT}_ATOMIC_CMPSWAP{,_X2}
Summary:
Implement BUFFER_ATOMIC_CMPSWAP{,_X2} instructions on all GCN targets, and FLAT_ATOMIC_CMPSWAP{,_X2} on CI+.

32-bit instruction variants tested manually on Kabini and Bonaire. Tests and parts of code provided by Jan Veselý.

Patch by: Vedran Miletić

Reviewers: arsenm, tstellarAMD, nhaehnle

Subscribers: jvesely, scchan, kanarayan, arsenm

Differential Revision: http://reviews.llvm.org/D17280

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@265170 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-01 18:27:37 +00:00
Matt Arsenault a4c1dc826a AMDGPU: Rename intrinsic to better match instruction name
Also fixes missing f32 test.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260780 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-13 01:03:00 +00:00
Matt Arsenault 6a2bf372b8 AMDGPU: Match some med3 patterns
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@259089 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-28 20:53:42 +00:00
Marek Olsak 7207597936 AMDGPU/SI: Add support for non-void functions
Summary:
Return values can be stored in SGPRs (i32) and VGPRs (f32).

This will be used by functions which expect some bytecode or other binary to
be appended at the end. It allows defining in which registers the return
values will be stored.

v2: don't do this for compute shaders

Reviewers: tstellarAMD, arsenm

Subscribers: arsenm

Differential Revision: http://reviews.llvm.org/D16033

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257621 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-13 17:23:04 +00:00
Matt Arsenault f12a12cd25 AMDGPU: Pattern match ffbh pattern to instruction.
The hardware instruction's output on 0 is -1 rather than 32.
Eliminate a test and select to -1. This removes an extra instruction
from the compatability function with HSAIL's firstbit instruction.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257352 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-11 17:02:00 +00:00
Matt Arsenault 1451e94ee0 AMDGPU: Use generic bitreverse intrinsic
Also fix bug in vector legalization for bitreverse.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@255512 91177308-0d34-0410-b5e6-96231b3b80d8
2015-12-14 17:25:38 +00:00
Tom Stellard 953c681473 R600 -> AMDGPU rename
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@239657 91177308-0d34-0410-b5e6-96231b3b80d8
2015-06-13 03:28:10 +00:00
Tom Stellard b1162b8d4b Revert "AMDGPU: Add core backend files for R600/SI codegen v6"
This reverts commit 4ea70107c5e51230e9e60f0bf58a0f74aa4885ea.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@160303 91177308-0d34-0410-b5e6-96231b3b80d8
2012-07-16 18:19:53 +00:00
Tom Stellard 23dc769a9b AMDGPU: Add core backend files for R600/SI codegen v6
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@160270 91177308-0d34-0410-b5e6-96231b3b80d8
2012-07-16 14:17:08 +00:00