Commit Graph

2080 Commits

Author SHA1 Message Date
Davide Italiano
1546bf0dba [AMDGPU] Remove unused function. NFCI.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312836 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-08 23:54:11 +00:00
Matt Arsenault
75448f1d3b AMDGPU: Start using !con operator
We have a lot of operand definition work essentially producing
every valid permutation of operands to workaround builiding
operand lists based on the instruction features. Apparently tablegen
already has a mostly undocumented operator to concat dags which
simplies this.

Convert one simple place to use this. The BUF instruction definitions
have much more complicated logic that can be totally rewritten now.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312822 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-08 19:09:13 +00:00
Matt Arsenault
fadb61df65 AMDGPU: Recompute scc liveness
The various scalar bit operations set SCC,
so one is erased or moved it needs to be recomputed.
Not sure why the existing tests don't fail on this.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312819 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-08 18:51:26 +00:00
Matt Arsenault
0bb6355f63 AMDGPU: Start selecting v_mad_mix_f32
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312732 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-07 18:05:07 +00:00
Konstantin Zhuravlyov
3964b8bfc8 AMDGPU: Handle non-temporal loads and stores
Differential Revision: https://reviews.llvm.org/D36862


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312729 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-07 17:14:54 +00:00
Konstantin Zhuravlyov
b6f64be453 AMDGPU: Handle more than one memory operand in SIMemoryLegalizer
Differential Revision: https://reviews.llvm.org/D37397


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312725 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-07 16:14:21 +00:00
Matt Arsenault
ca22b05483 AMDGPU: Don't legalize i16 extloads to i32 with legal i16
Keeping non-i16 extloads makes it easier to match some new
gfx9 load instructions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312699 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-07 05:37:34 +00:00
Stanislav Mekhanoshin
6148c30603 [AMDGPU] Use v_pk_max_f16 for fcanonicalize
Differential Revision: https://reviews.llvm.org/D37325

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312676 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-06 22:27:29 +00:00
Stanislav Mekhanoshin
953b70393a [AMDGPU] Fixed encoding of v_pk_mul_f16 in fcanonicalize
Differential Revision: https://reviews.llvm.org/D37522

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312660 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-06 18:29:51 +00:00
Stanislav Mekhanoshin
651c4efd77 [AMDGPU] Fix shouldClusterMemOps to process flat loads
Flat loads do not have vdata operand but have vdst instead.

Differential Revision: https://reviews.llvm.org/D37502

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312640 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-06 15:31:30 +00:00
Nicolai Haehnle
adf1cb63f2 AMDGPU: Make worst-case assumption about the wait states in inline assembly
Summary:
Mesa still uses a hack where empty inline assembly is used as a kind of
optimization barrier. This exposed a problem where not enough wait states
were inserted, because the hazard recognizer implicitly assumed that each
inline assembly "instruction" has at least one wait state.

Reviewers: arsenm

Subscribers: kzhuravl, wdng, yaxunl, dstuttard, tpr, llvm-commits, t-tye

Differential Revision: https://reviews.llvm.org/D37205

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312635 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-06 13:50:13 +00:00
Yaxun Liu
1e1d0b01c1 [AMDGPU] Transform __read_pipe_* and __write_pipe_*
When packet size equals packet align and is power of 2, transform
__read_pipe* and __write_pipe* to specialized library function.

Differential Revision: https://reviews.llvm.org/D36831


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312598 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-06 00:30:27 +00:00
Konstantin Zhuravlyov
9e6f849b2e AMDGPU: Cleanup/refactor SIMemoryLegalizer [3]:
- Refactor SIMemOpInfo's constructors
  - Allow construction of NotAtomic SIMemOpInfo

Differential Revision: https://reviews.llvm.org/D37396


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312563 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-05 19:01:10 +00:00
Matt Arsenault
4e0c4fb9c1 AMDGPU: Fix not accounting for tail call resource usage
If the only call in a function is a tail call, the
function isn't considered to have a call since it's a
type of return.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312561 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-05 18:36:36 +00:00
Konstantin Zhuravlyov
f9ab88e18d AMDGPU/NFC: Cleanup/refactor SIMemoryLegalizer [2]:
- Make SIMemOpInfo a class
  - Add accessor methods to SIMemOpInfo
  - Move get*Info methods to SIMemOpInfo

Differential Revision: https://reviews.llvm.org/D37395


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312541 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-05 16:41:25 +00:00
Konstantin Zhuravlyov
c0c4768b6b AMDGPU/NFC: Cleanup/refactor SIMemoryLegalizer [1]:
- Rename MemOpInfo -> SIMemOpInfo
  - Move SIMemOpInfo class out of SIMemoryLegalizer class

Differential Revision: https://reviews.llvm.org/D37394


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312540 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-05 16:18:05 +00:00
Stanislav Mekhanoshin
f3b5f2ad4a [AMDGPU] Prevent infinite recursion in DAG.computeKnownBits()
Differential Revision: https://reviews.llvm.org/D37392

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312364 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-01 20:43:20 +00:00
Matt Arsenault
757642511d AMDGPU: Add ds_{read|write}_addtid_b32 definitions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312349 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-01 18:38:02 +00:00
Matt Arsenault
6a29a225d2 AMDGPU: Add most d16 load/store instruction definitions
Doesn't include the tied operand necessary for the loads,
but is enough for the assembler to work.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312347 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-01 18:36:06 +00:00
Nicolai Haehnle
96b6414540 AMDGPU: IMPLICIT_DEFs and DBG_VALUEs do not contribute to wait states
Summary:
This fixes a bug that was exposed on gfx9 in various
GL45-CTS.shaders.loops.*_iterations.select_iteration_count_fragment tests,
e.g. GL45-CTS.shaders.loops.do_while_uniform_iterations.select_iteration_count_fragment

Reviewers: arsenm

Subscribers: kzhuravl, wdng, yaxunl, dstuttard, tpr, t-tye, llvm-commits

Differential Revision: https://reviews.llvm.org/D36193

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312337 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-01 16:56:32 +00:00
Matt Arsenault
fcd77e8a04 AMDGPU: Fold clamp modifier for packed instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312297 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-31 23:53:50 +00:00
Eugene Zelenko
046ca04445 [Analysis] Fix some Clang-tidy modernize-use-using and Include What You Use warnings; other minor fixes. Also affected in files (NFC).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312289 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-31 21:56:16 +00:00
Matt Arsenault
d213820974 AMDGPU: Turn int pack pattern into build_vector
build_vector is a more useful canonical form when
pattern matching packed operations, so turn shift
into high element into a build_vector.

Should show no change for now.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312282 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-31 21:17:22 +00:00
Matt Arsenault
c3f95e0648 AMDGPU: Don't assert in TTI with fp32 denorms enabled
Also refine for f16 and rcp cases.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312213 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-31 05:47:00 +00:00
Matt Arsenault
e02cff1224 AMDGPU: Use set for tracked registers
The majority of the time spent in the pass checking
for the register reads. Rather than searching all of
the defined registers for uses in each instruction,
use a set of defined registers and check the operands
of the instruction.

This process still is algorithmically not great,
but with the additional trick of skipping the analysis
for addresses with one use, this brings one slow
testcase into a reasonable range.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312206 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-31 01:53:09 +00:00
Matt Arsenault
73dd3b8c43 AMDGPU: Correct operand types for v_mad_mix*
These aren't really packed instructions, so the default
op_sel_hi should be 0 since this indicates a conversion.
The operand types are scalar values that behave similar
to an f16 scalar that may be converted to f32.

Doesn't change the default printing for op_sel_hi, just
the parsing.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312179 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-30 22:18:40 +00:00
Matt Arsenault
b0edfb8160 AMDGPU: Don't look for DS merge candidates with one use address
The merge is only possible if the base address register is the
same for the two instructions. If there is only the one use,
there's no point in doing an expensive forward scan checking
for memory interference looking for a merge candidate.

This gives a signficant improvement in one extreme testcase.
The code to do the scan is still algorithmically terrible,
so this is still the slowest pass in that example.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312096 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-30 03:26:18 +00:00
Stanislav Mekhanoshin
0a4c4f2a1d [AMDGPU] Use v_max_f* for fcanonicalize
If denorms are not flushed we can use max instead of multiplication
by 1. For double that is simply faster, while for float and half
it is shorter, because mul uses constant bus and VOP3.

Differential Revision: https://reviews.llvm.org/D36856

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312095 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-30 03:03:38 +00:00
Matt Arsenault
90ea18a51f AMDGPU: Select clamp pattern with v2f16
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312087 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-30 01:20:17 +00:00
Matt Arsenault
e5eabbf0bd AMDGPU: Fix typo
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312040 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-29 21:25:51 +00:00
Stanislav Mekhanoshin
9324a77aa4 [AMDGPU] Fix regression in AMDGPULibCalls allowing native for doubles
Under -cl-fast-relaxed-math we could use native_sqrt, but f64 was
allowed to produce HSAIL's nsqrt instruction. HSAIL is not here
and we stick with non-existing native_sqrt(double) as a result.

Add check for f64 to not return native functions and also remove
handling of f64 case for fold_sqrt.

Differential Revision: https://reviews.llvm.org/D37223

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311900 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-28 18:00:08 +00:00
Stanislav Mekhanoshin
f4dd1bdd9a [AMDGPU] computeKnownBitsForTargetNode for 24 bit mul
Differential Revision: https://reviews.llvm.org/D37168

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311896 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-28 16:35:37 +00:00
Konstantin Zhuravlyov
a698ffcfb3 AMDGPU: Fix gfx801 features
gfx801 has 1/2 rate F64, Fast F32 FMA

Differential Revision: https://reviews.llvm.org/D36981


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311694 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-24 20:03:07 +00:00
Benjamin Kramer
c773276189 Move helper classes into anonymous namespaces.
No functionality change intended.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311288 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-20 13:03:48 +00:00
Konstantin Zhuravlyov
6930800b8c AMDGPU/NFC: Reorder functions in SIMemoryLegalizer:
- Move *load* functions before *atomic* functions
  - Move *store* functions before *atomic* functions


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311256 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-19 18:44:27 +00:00
Konstantin Zhuravlyov
230b5542e4 AMDGPU/NFC: Rename few things in SIMemoryLegalizer:
- AtomicInfo -> MemOpInfo
  - getAtomicLoadInfo -> getLoadInfo
  - getAtomicStoreInfo -> getStoreInfo
  - expandAtomicLoad -> expandLoad
  - expandAtomicStore -> expandStore

Differential Revision: https://reviews.llvm.org/D36861


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311179 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-18 17:30:02 +00:00
Tom Stellard
7dcd9e77f5 AMDGPU: Add R600InstPrinter class
Summary:
This is step towards separating the GCN and R600 tablegen'd code.

This is a little awkward for now, because the R600 functions won't have the
MCSubtargetInfo parameter, so we need to have AMDMGPUInstPrinter
delegate to R600InstPrinter, but once the tablegen'd code is split,
we will be able to drop the delegation and use R600InstPrinter directly.

Reviewers: arsenm

Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, llvm-commits

Differential Revision: https://reviews.llvm.org/D36444

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311128 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-17 22:20:04 +00:00
Evgeny Mankov
cb139f4145 [AMDGPU] NFC: test commit
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311019 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-16 16:47:29 +00:00
Konstantin Zhuravlyov
d8e6ba7a8d AMDGPU/NFC: Sort files in CMakeLists.txt alphabetically
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311017 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-16 16:23:32 +00:00
Dmitry Preobrazhensky
917eb1c735 [AMDGPU][MC][GFX9] Added op_sel support for v_mad_*16, v_fma_f16, v_div_fixup_f16
This change implements features postponed in https://reviews.llvm.org/D35424 because of a dependency on https://reviews.llvm.org/D36322

Reviewers: SamWot, artem.tamazov, arsenm

Differential Revision: https://reviews.llvm.org/D36694

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311011 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-16 15:16:32 +00:00
Dmitry Preobrazhensky
600899c871 [AMDGPU][MC][GFX9] Added integer clamping support for VOP3 opcodes
See Bug 34152: https://bugs.llvm.org//show_bug.cgi?id=34152

Reviewers: SamWot, artem.tamazov, arsenm

Differential Revision: https://reviews.llvm.org/D36674

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311006 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-16 13:51:56 +00:00
Stanislav Mekhanoshin
2df3fafbea [AMDGPU] Eliminate no effect instructions before s_endpgm
Differential Revision: https://reviews.llvm.org/D36585

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310987 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-16 04:43:49 +00:00
Quentin Colombet
8e98e02784 Reapply "[GlobalISel] Remove the GISelAccessor API."
This reverts commit r310425, thus reapplying r310335 with a fix for link
issue of the AArch64 unittests on Linux bots when BUILD_SHARED_LIBS is ON.

Original commit message:
[GlobalISel] Remove the GISelAccessor API.

Its sole purpose was to avoid spreading around ifdefs related to
building global-isel. Since r309990, GlobalISel is not optional anymore,
thus, we can get rid of this mechanism all together.

NFC.

----
The fix for the link issue consists in adding the GlobalISel library in
the list of dependencies for the AArch64 unittests. This dependency
comes from the use of AArch64Subtarget that needs to know how
to destruct the GISel related APIs when being detroyed.

Thanks to Bill Seurer and Ahmed Bougacha for helping me reproducing and
understand the problem.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310969 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-15 22:31:51 +00:00
Matt Arsenault
45424dbebb AMDGPU: Start adding tail call support
Handle the sibling call cases.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310753 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-11 20:42:08 +00:00
Stanislav Mekhanoshin
911c1e458f [AMDGPU] Fix santizer error after last commit
Removed useless assert.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310738 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-11 17:54:43 +00:00
Stanislav Mekhanoshin
9643e6bd78 [AMDGPU] Ported and adopted AMDLibCalls pass
The pass does simplifications of well known AMD library calls.
If given -amdgpu-prelink option it works in a pre-link mode which
allows to reference new library functions which will be linked in
later.

In addition it also used to process traditional AMD option
-fuse-native which allows to replace some of the functions with
their fast native implementations from the library.

The necessary glue to pass the prelink option and translate
-fuse-native is to be added to the driver.

Differential Revision: https://reviews.llvm.org/D36436

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310731 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-11 16:42:09 +00:00
Eugene Zelenko
75134ec482 [AMDGPU] Fix some Clang-tidy modernize-use-using and Include What You Use warnings; other minor fixes (NFC).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310541 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-10 00:46:15 +00:00
Matt Arsenault
e695a23276 AMDGPU: Fix assert on n inline asm constraint
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310515 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-09 20:09:35 +00:00
Dmitry Preobrazhensky
cb7edc0ee9 [AMDGPU][MC][GFX9] Added 16-bit renamed and "_legacy" VALU opcodes
See Bug 33629: https://bugs.llvm.org//show_bug.cgi?id=33629

Reviewers: vpykhtin, SamWot, arsenm

Differential Revision: https://reviews.llvm.org/D36322

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310497 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-09 17:10:47 +00:00
Gabor Horvath
4e95657190 Suppress a warning. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310459 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-09 10:38:53 +00:00