2193 Commits

Author SHA1 Message Date
David Blaikie
48319238e4 Target/TargetInstrInfo.h -> CodeGen/TargetInstrInfo.h to match layering
This header includes CodeGen headers, and is not, itself, included by
any Target headers, so move it into CodeGen to match the layering of its
implementation.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317647 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-08 01:01:31 +00:00
Matt Arsenault
a932c3f118 AMDGPU: Set correct sched model on v_mad_u64_u32
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317645 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-08 00:48:25 +00:00
Matt Arsenault
db6cc311a0 AMDGPU: Remove redundant combine
This combine was already done in two places. The
generic combiner already has done this since
r217610, for adds (with a single use).

This one was added in r303641, and added support for handling
or as well. r313251 later added support to the generic
combine for or. It also turns out the isOrEquivalentToAdd
check is not necessary for this combine.

Additionally, we already reproduce this combine in yet
another place in the backend, although in that version
multiple uses of the add are still folded if it will
allow a fold into the addressing mode. That version needs
to be improved to understand ors though, as well as the
correct legal offsets for private.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317526 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-07 00:06:32 +00:00
Matt Arsenault
bd04b64cd1 AMDGPU: Select v_mad_u64_u32 and v_mad_i64_i32
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317492 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-06 17:04:37 +00:00
Sanjay Patel
00e900afdb [IR] redefine 'UnsafeAlgebra' / 'reassoc' fast-math-flags and add 'trans' fast-math-flag
As discussed on llvm-dev:
http://lists.llvm.org/pipermail/llvm-dev/2016-November/107104.html
and again more recently:
http://lists.llvm.org/pipermail/llvm-dev/2017-October/118118.html

...this is a step in cleaning up our fast-math-flags implementation in IR to better match
the capabilities of both clang's user-visible flags and the backend's flags for SDNode.

As proposed in the above threads, we're replacing the 'UnsafeAlgebra' bit (which had the 
'umbrella' meaning that all flags are set) with a new bit that only applies to algebraic 
reassociation - 'AllowReassoc'.

We're also adding a bit to allow approximations for library functions called 'ApproxFunc' 
(this was initially proposed as 'libm' or similar).

...and we're out of bits. 7 bits ought to be enough for anyone, right? :) FWIW, I did 
look at getting this out of SubclassOptionalData via SubclassData (spacious 16-bits), 
but that's apparently already used for other purposes. Also, I don't think we can just 
add a field to FPMathOperator because Operator is not intended to be instantiated. 
We'll defer movement of FMF to another day.

We keep the 'fast' keyword. I thought about removing that, but seeing IR like this:
%f.fast = fadd reassoc nnan ninf nsz arcp contract afn float %op1, %op2
...made me think we want to keep the shortcut synonym.

Finally, this change is binary incompatible with existing IR as seen in the 
compatibility tests. This statement:
"Newer releases can ignore features from older releases, but they cannot miscompile 
them. For example, if nsw is ever replaced with something else, dropping it would be 
a valid way to upgrade the IR." 
( http://llvm.org/docs/DeveloperPolicy.html#ir-backwards-compatibility )
...provides the flexibility we want to make this change without requiring a new IR 
version. Ie, we're not loosening the FP strictness of existing IR. At worst, we will 
fail to optimize some previously 'fast' code because it's no longer recognized as 
'fast'. This should get fixed as we audit/squash all of the uses of 'isFast()'.

Note: an inter-dependent clang commit to use the new API name should closely follow 
commit.

Differential Revision: https://reviews.llvm.org/D39304



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317488 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-06 16:27:15 +00:00
Yaxun Liu
ec1f0cc416 [AMDGPU] Change alloca addr space of r600 to 5 for amdgiz environment
Differential Revision: https://reviews.llvm.org/D39657


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317479 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-06 14:32:33 +00:00
Yaxun Liu
13a223e6e6 [AMDGPU] Fix assertion due to assuming pointer in default addr space is 32 bit
The backend assumes pointer in default addr space is 32 bit, which is not
true for the new addr space mapping and causes assertion for unresolved
functions.

This patch fixes that.

Differential Revision: https://reviews.llvm.org/D39643


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317476 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-06 13:01:33 +00:00
Yaxun Liu
35934f8100 [AMDGPU] Remove hardcoded address space value from AMDGPULibFunc
AMDGPULibFunc hardcodes address space values of the old address space mapping,
which causes invalid addrspacecast instructions and undefined functions in
APPSDK sample MonteCarloAsianDP.

This patch fixes that.

Differential Revision: https://reviews.llvm.org/D39616


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317409 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-04 17:37:43 +00:00
David Blaikie
803f827385 Move TargetFrameLowering.h to CodeGen where it's implemented
This header already includes a CodeGen header and is implemented in
lib/CodeGen, so move the header there to match.

This fixes a link error with modular codegeneration builds - where a
header and its implementation are circularly dependent and so need to be
in the same library, not split between two like this.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317379 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-03 22:32:11 +00:00
Konstantin Zhuravlyov
f79fab6f98 AMDGPU: Fix warning discovered by r317266 [-Wunused-private-field]
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317280 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-02 22:35:22 +00:00
Konstantin Zhuravlyov
37bbee84d8 AMDGPU: Remove outdated fixme (it was already fixed)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317266 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-02 20:48:06 +00:00
Konstantin Zhuravlyov
63dcaeaca4 AMDGPU: Fix set but not used warnings related to AMDGPUAS
Differential Revision: https://reviews.llvm.org/D39499


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317114 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-01 19:12:38 +00:00
NAKAMURA Takumi
598658d792 Fix warnings discovered by rL317076. [-Wunused-private-field]
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317091 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-01 13:47:55 +00:00
Benjamin Kramer
63e6891819 [AMDGPU] Clean up symbols in the global namespace.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317051 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-31 23:21:30 +00:00
Marek Olsak
0ce6825d9e AMDGPU: Select s_buffer_load_dword with a non-constant SGPR offset
Summary:
Apps that benefit:
- alien isolation
- bioshock infinite
- civilization: beyond earth
- company of heroes 2
- dirt showdown
- dota 2
- F1 2015
- grid autosport
- hitman
- legend of grimrock
- serious sam 3: bfe
- shadow warrior
- talos principle
- total war: warhammer
- UE4 demos: effects cave, elemental, sun temple

Reviewers: arsenm, nhaehnle

Subscribers: kzhuravl, wdng, yaxunl, dstuttard, tpr, llvm-commits, t-tye

Differential Revision: https://reviews.llvm.org/D38914

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317038 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-31 21:06:42 +00:00
Yaxun Liu
a52756b2c9 [AMDGPU] Emit metadata for hidden arguments for kernel enqueue
Identifies kernels which performs device side kernel enqueues and emit
metadata for the associated hidden kernel arguments. Such kernels are
marked with calls-enqueue-kernel function attribute by
AMDGPUOpenCLEnqueueKernelLowering pass and later on
hidden kernel arguments metadata HiddenDefaultQueue and
HiddenCompletionAction are emitted for them.

Differential Revision: https://reviews.llvm.org/D39255


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316907 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-30 14:30:28 +00:00
Tom Stellard
f94e4f3ed7 AMDGPU/GlobalISel: Mark 32-bit G_FADD as legal
Reviewers: arsenm

Reviewed By: arsenm

Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, rovka, kristof.beyls, igorb, dstuttard, tpr, t-tye, llvm-commits

Differential Revision: https://reviews.llvm.org/D38439

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316815 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-27 23:57:41 +00:00
Marek Olsak
b25352f371 AMDGPU: Handle s_buffer_load_dword hazard on SI
Reviewers: arsenm, nhaehnle

Subscribers: kzhuravl, wdng, yaxunl, dstuttard, tpr, llvm-commits, t-tye

Differential Revision: https://reviews.llvm.org/D39171

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316666 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-26 14:43:02 +00:00
Matt Arsenault
b6397326b8 AMDGPU: Add max-mix-insts subtarget feature
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316553 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-25 07:00:51 +00:00
Marek Olsak
4fda278e9b AMDGPU: Add new intrinsic llvm.amdgcn.kill(i1)
Summary:
Kill the thread if operand 0 == false.
llvm.amdgcn.wqm.vote can be applied to the operand.

Also allow kill in all shader stages.

Reviewers: arsenm, nhaehnle

Subscribers: kzhuravl, wdng, yaxunl, dstuttard, tpr, t-tye, llvm-commits

Differential Revision: https://reviews.llvm.org/D38544

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316427 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-24 10:27:13 +00:00
Marek Olsak
7525c08782 AMDGPU: Add llvm.amdgcn.wqm.vote intrinsic
Reviewers: arsenm, nhaehnle

Subscribers: kzhuravl, wdng, yaxunl, dstuttard, tpr, t-tye

Differential Revision: https://reviews.llvm.org/D38543

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316426 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-24 10:26:59 +00:00
Konstantin Zhuravlyov
24cbf50e81 AMDGPU: Initialize WavefrontSize from TD files
Differential Revision: https://reviews.llvm.org/D39205


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316389 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-23 23:02:39 +00:00
Matt Arsenault
7f8c3f0208 AMDGPU: Cleanup local atomic node names
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316349 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-23 17:16:43 +00:00
Matt Arsenault
367cfd84c3 AMDGPU: Fix default range in non-kernel functions
The range should be assumed to be the hardware maximum
if a workitem intrinsic is used in a callable function
which does not know the restricted limit of the calling
kernel.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316346 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-23 17:09:35 +00:00
Konstantin Zhuravlyov
28cb7901b7 AMDGPU: Rename MaxFlatWorkgroupSize to MaxFlatWorkGroupSize for consistency
Differential Revision: https://reviews.llvm.org/D38957


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316097 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-18 17:31:09 +00:00
NAKAMURA Takumi
d8ff2f49ce Untabify.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316079 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-18 13:31:28 +00:00
Wei Ding
607acf30af AMDGPU : Fix an error for the llvm.cttz implementation.
Differential Revision: http://reviews.llvm.org/D39014

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316037 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-17 21:49:52 +00:00
Eugene Zelenko
1c51faba7a [Transforms] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316034 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-17 21:27:42 +00:00
Konstantin Zhuravlyov
cecf102e0c AMDGPU: Start generating metadata for MaxFlatWorkGroupSize
Differential Revision: https://reviews.llvm.org/D38958


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316024 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-17 20:03:21 +00:00
Mark Searles
df60d8e59e Use the return value of UpdateNodeOperands(); in some cases, UpdateNodeOperands() modifies the node in-place and using the return value isn’t strictly necessary. However, it does not necessarily modify the node, but may return a resultant node if it already exists in the DAG. See comments in UpdateNodeOperands(). In that case, the return value must be used to avoid such scenarios as an infinite loop (node is assumed to have been updated, so added back to the worklist, and re-processed; however, node hasn’t changed so it is once again passed to UpdateNodeOperands(), assumed modified, added back to worklist; cycle infinitely repeats).
Differential Revision: https://reviews.llvm.org/D38466

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315957 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-16 23:38:53 +00:00
Krzysztof Parzyszek
7b72f3902c Add iterator range MachineRegisterInfo::liveins(), adopt users, NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315927 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-16 19:08:41 +00:00
Aaron Ballman
1d03d382c1 Reverting r315590; it did not include changes for llvm-tblgen, which is causing link errors for several people.
Error LNK2019 unresolved external symbol "public: void __cdecl `anonymous namespace'::MatchableInfo::dump(void)const " (?dump@MatchableInfo@?A0xf4f1c304@@QEBAXXZ) referenced in function "public: void __cdecl `anonymous namespace'::AsmMatcherEmitter::run(class llvm::raw_ostream &)" (?run@AsmMatcherEmitter@?A0xf4f1c304@@QEAAXAEAVraw_ostream@llvm@@@Z) llvm-tblgen D:\llvm\2017\utils\TableGen\AsmMatcherEmitter.obj 1


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315854 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-15 14:32:27 +00:00
Vitaly Buka
eec5b16c88 Remove unused variables
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315847 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-15 05:35:02 +00:00
Konstantin Zhuravlyov
38ae50b093 AMDGPU: Don't use TargetStreamer if it has not been initialized
Fixes cfe/trunk/test/Misc/backend-resource-limit-diagnostics.cl
test after r315808

We may hit few other similar issues, but I want to discuss good
solution offline.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315830 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-14 22:16:26 +00:00
Konstantin Zhuravlyov
5556d8485b AMDGPU: Bring HSA metadata on par with the specification
Differential Revision: https://reviews.llvm.org/D38753


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315821 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-14 19:03:51 +00:00
Konstantin Zhuravlyov
e7b2dc3c3a AMDGPU: Improve note directive verification in assembler
- Do not allow amd_amdgpu_isa directives on non-amdgcn architectures
  - Do not allow amd_amdgpu_hsa_metadata on non-amdhsa OSes
  - Do not allow amd_amdgpu_pal_metadata on non-amdpal OSes

Differential Revision: https://reviews.llvm.org/D38750


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315812 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-14 16:15:28 +00:00
Konstantin Zhuravlyov
473d951406 AMDGPU: Do not emit deprecated notes for code object v3
Differential Revision: https://reviews.llvm.org/D38749


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315810 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-14 15:59:07 +00:00
Konstantin Zhuravlyov
eb211af057 AMDGPU: Add support for isa version note
- Emit NT_AMD_AMDGPU_ISA
  - Add assembler parsing for isa version directive
    - If isa version directive does not match command line arguments, then return error

Differential Revision: https://reviews.llvm.org/D38748


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315808 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-14 15:40:33 +00:00
Matt Arsenault
93f0e0c1d1 AMDGPU: Implement hasBitPreservingFPLogic
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315754 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-13 21:10:22 +00:00
Matt Arsenault
b1763ab78e AMDGPU: Look for src mods before fp_extend
When selecting modifiers for mad_mix instructions,
look at fneg/fabs that occur before the conversion.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315748 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-13 20:45:49 +00:00
Matt Arsenault
9a6875264b AMDGPU: Implement isFPExtFoldable
This helps match v_mad_mix* in some cases.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315744 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-13 20:18:59 +00:00
Matthias Braun
9385cf15d1 Revert "TargetMachine: Merge TargetMachine and LLVMTargetMachine"
Reverting to investigate layering effects of MCJIT not linking
libCodeGen but using TargetMachine::getNameWithPrefix() breaking the
lldb bots.

This reverts commit r315633.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315637 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-12 22:57:28 +00:00
Matthias Braun
a063107f8d TargetMachine: Merge TargetMachine and LLVMTargetMachine
Merge LLVMTargetMachine into TargetMachine.

- There is no in-tree target anymore that just implements TargetMachine
  but not LLVMTargetMachine.
- It should still be possible to stub out all the various functions in
  case a target does not want to use lib/CodeGen
- This simplifies the code and avoids methods ending up in the wrong
  interface.

Differential Revision: https://reviews.llvm.org/D38489

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315633 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-12 22:28:54 +00:00
Wei Ding
29de9d738e Implement custom lowering for ISD::CTTZ_ZERO_UNDEF and ISD::CTTZ.
Differential Revision: http://reviews.llvm.org/D37348

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315610 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-12 19:37:14 +00:00
Konstantin Zhuravlyov
fb0b5f597f AMDGPU/NFC: Move AMDGPU specific note types to ELF.h
Differential Revision: https://reviews.llvm.org/D38747


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315608 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-12 18:59:54 +00:00
Konstantin Zhuravlyov
499a404a37 AMDGPU: Fix warnings introduced in r315526
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315596 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-12 17:34:05 +00:00
Tim Renouf
fadd83b09c [AMDGPU] For amdpal, widen interpolation mode workaround
Summary:
The interpolation mode workaround ensures that at least one
interpolation mode is enabled in PSInputAddr. It does not also check
PSInputEna on the basis that the user might enable bits in that
depending on run-time state.

However, for amdpal os type, the user does not enable some bits after
compilation based on run-time states; the register values being
generated here are the final ones set in the hardware. Therefore, apply
the workaround to PSInputAddr and PSInputEnable together. (The case
where a bit is set in PSInputAddr but not in PSInputEnable is where the
frontend set up an input arg for a particular interpolation mode, but
nothing uses that input arg. Really we should have an earlier pass that
removes such an arg.)

Reviewers: arsenm, nhaehnle, dstuttard

Subscribers: kzhuravl, wdng, yaxunl, t-tye, llvm-commits

Differential Revision: https://reviews.llvm.org/D37758

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315591 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-12 16:16:41 +00:00
Don Hinton
5298935fe7 [dump] Remove NDEBUG from test to enable dump methods [NFC]
Summary:
Add LLVM_FORCE_ENABLE_DUMP cmake option, and use it along with
LLVM_ENABLE_ASSERTIONS to set LLVM_ENABLE_DUMP.

Remove NDEBUG and only use LLVM_ENABLE_DUMP to enable dump methods.

Move definition of LLVM_ENABLE_DUMP from config.h to llvm-config.h so
it'll be picked up by public headers.

Differential Revision: https://reviews.llvm.org/D38406

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315590 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-12 16:16:06 +00:00
Reid Kleckner
1c59d0201f Fix AMDGPU build issue
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315535 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-11 23:53:36 +00:00
Lang Hames
806f68bbab [MC] Have MCObjectStreamer take its MCAsmBackend argument via unique_ptr.
MCObjectStreamer owns its MCCodeEmitter -- this fixes the types to reflect that,
and allows us to remove the last instance of MCObjectStreamer's weird "holding
ownership via someone else's reference" trick.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315531 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-11 23:34:47 +00:00