Commit Graph

25 Commits

Author SHA1 Message Date
Nicolai Haehnle
adf1cb63f2 AMDGPU: Make worst-case assumption about the wait states in inline assembly
Summary:
Mesa still uses a hack where empty inline assembly is used as a kind of
optimization barrier. This exposed a problem where not enough wait states
were inserted, because the hazard recognizer implicitly assumed that each
inline assembly "instruction" has at least one wait state.

Reviewers: arsenm

Subscribers: kzhuravl, wdng, yaxunl, dstuttard, tpr, llvm-commits, t-tye

Differential Revision: https://reviews.llvm.org/D37205

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312635 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-06 13:50:13 +00:00
Nicolai Haehnle
96b6414540 AMDGPU: IMPLICIT_DEFs and DBG_VALUEs do not contribute to wait states
Summary:
This fixes a bug that was exposed on gfx9 in various
GL45-CTS.shaders.loops.*_iterations.select_iteration_count_fragment tests,
e.g. GL45-CTS.shaders.loops.do_while_uniform_iterations.select_iteration_count_fragment

Reviewers: arsenm

Subscribers: kzhuravl, wdng, yaxunl, dstuttard, tpr, t-tye, llvm-commits

Differential Revision: https://reviews.llvm.org/D36193

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312337 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-01 16:56:32 +00:00
Connor Abbott
1a05d247fa [AMDGPU] Add missing hazard for DPP-after-EXEC-write
Summary:
Following the docs, we need at least 5 wait states between an EXEC write
and an instruction that uses DPP.

Reviewers: tstellar, arsenm

Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, llvm-commits

Differential Revision: https://reviews.llvm.org/D34849

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310013 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-04 01:09:43 +00:00
Chandler Carruth
e3e43d9d57 Sort the remaining #include lines in include/... and lib/....
I did this a long time ago with a janky python script, but now
clang-format has built-in support for this. I fed clang-format every
line with a #include and let it re-sort things according to the precise
LLVM rules for include ordering baked into clang-format these days.

I've reverted a number of files where the results of sorting includes
isn't healthy. Either places where we have legacy code relying on
particular include ordering (where possible, I'll fix these separately)
or where we have particular formatting around #include lines that
I didn't want to disturb in this patch.

This patch is *entirely* mechanical. If you get merge conflicts or
anything, just ignore the changes in this patch and run clang-format
over your #include lines in the files.

Sorry for any noise here, but it is important to keep these things
stable. I was seeing an increasing number of patches with irrelevant
re-ordering of #include lines because clang-format was used. This patch
at least isolates that churn, makes it easy to skip when resolving
conflicts, and gets us to a clean baseline (again).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@304787 91177308-0d34-0410-b5e6-96231b3b80d8
2017-06-06 11:49:48 +00:00
Matt Arsenault
6cf5553d32 AMDGPU: Fix broken condition in hazard recognizer
Fixes bug 32248.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@298125 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-17 21:36:28 +00:00
Simon Pilgrim
c12050bd3a Reverted unintended commit
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@297841 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-15 14:47:30 +00:00
Simon Pilgrim
a6a5484b63 Fix Wint-in-bool-context warning (PR32248)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@297840 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-15 14:38:19 +00:00
Matt Arsenault
83c857cd3a AMDGPU: Merge initial gfx9 support
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295554 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-18 18:29:53 +00:00
Eugene Zelenko
68c521d030 [AMDGPU] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292623 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-20 17:52:16 +00:00
Matt Arsenault
a89dd1d1c4 AMDGPU: Rename flat operands to match mubuf
Use vaddr/vdst for the same purposes.

This also fixes a beg in SIInsertWaits for the
operand check. The stored value operand is currently called
data0 in the single offset case, not data.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288188 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-29 19:30:44 +00:00
Jan Vesely
ea73d16588 AMDGPU/GCN: Exit early in hazard recognizer if there is no vreg argument
wbinvl.* are vector instruction that do not sue vector registers.

v2: check only M?BUF instructions

Differential Revision: https://reviews.llvm.org/D26633

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287056 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-15 23:55:15 +00:00
Tom Stellard
a911f5ff01 AMDGPU/SI: Handle hazard with s_rfe_b64
Reviewers: arsenm

Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, llvm-commits, tony-tye

Differential Revision: https://reviews.llvm.org/D25638

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@285368 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-27 23:50:21 +00:00
Tom Stellard
8434132101 AMDGPU/SI: Handle hazard with sgpr lane selects for v_{read,write}lane
Reviewers: arsenm

Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, tony-tye, llvm-commits

Differential Revision: https://reviews.llvm.org/D25637

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@285367 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-27 23:42:29 +00:00
Tom Stellard
0b23373c51 AMDGPU/SI: Fix unused variable warning on non-debug builds
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@285363 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-27 23:28:03 +00:00
Tom Stellard
5480a2423d AMDGPU/SI: Handle hazard with > 8 byte VMEM stores
Reviewers: arsenm

Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, tony-tye, llvm-commits

Differential Revision: https://reviews.llvm.org/D25577

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@285359 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-27 23:05:31 +00:00
Tom Stellard
79758d450e AMDGPU/SI: Handle s_setreg hazard in GCNHazardRecognizer
Reviewers: arsenm

Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, llvm-commits, tony-tye

Differential Revision: https://reviews.llvm.org/D25528

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@285338 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-27 20:39:09 +00:00
Tom Stellard
6b339ba8ac AMDGPU/SI: Handle s_getreg hazard in GCNHazardRecognizer
Reviewers: arsenm

Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, llvm-commits, tony-tye

Differential Revision: https://reviews.llvm.org/D25526

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284298 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-15 00:58:14 +00:00
Tom Stellard
4e0ea1b72b AMDGPU/SI: Handle div_fmas hazard in GCNHazardRecognizer
Reviewers: arsenm

Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, tony-tye, llvm-commits

Differential Revision: https://reviews.llvm.org/D25250

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283622 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-07 23:42:48 +00:00
Matt Arsenault
759ed7e410 AMDGPU: Cleanup subtarget handling.
Split AMDGPUSubtarget into amdgcn/r600 specific subclasses.
This removes most of the static_casting of the basic codegen
classes everywhere, and tries to restrict the features
visible on the wrong target.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@273652 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-24 06:30:11 +00:00
Benjamin Kramer
022a8990dd Move instances of std::function.
Or replace with llvm::function_ref if it's never stored. NFC intended.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272513 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-12 16:13:55 +00:00
Aaron Ballman
d80450e58d Silence unused variable warning; NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268392 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-03 15:17:25 +00:00
Tom Stellard
8478bd5765 AMDGPU/SI: Use the hazard recognizer to break SMEM soft clauses
Summary:
Add support for detecting hazards in SMEM soft clauses, so that we only
break the clauses when necessary, either by adding s_nop or re-ordering
other alu instructions.

Reviewers: arsenm

Subscribers: arsenm, llvm-commits

Differential Revision: http://reviews.llvm.org/D18870

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268260 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-02 17:39:06 +00:00
Tom Stellard
b8a2cc5119 AMDGPU/SI: Use hazard recognizer to detect DPP hazards
Reviewers: arsenm

Subscribers: arsenm, llvm-commits

Differential Revision: http://reviews.llvm.org/D18603

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268247 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-02 16:23:09 +00:00
Aaron Ballman
f1f452f373 Silence unused variable warnings; NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268234 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-02 14:48:03 +00:00
Tom Stellard
6ab99c7ca6 AMDGPU/SI: Enable the post-ra scheduler
Summary:
This includes a hazard recognizer implementation to replace some of
the hazard handling we had during frame index elimination.

Reviewers: arsenm

Subscribers: qcolombet, arsenm, llvm-commits

Differential Revision: http://reviews.llvm.org/D18602

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268143 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-30 00:23:06 +00:00