Tim Renouf ee9a2bdfd5 [AMDGPU] Add support for multi-dword s.buffer.load intrinsic ...

Summary:
Patch by Marek Olsak and David Stuttard, both of AMD.

This adds a new amdgcn intrinsic supporting s.buffer.load, in particular
multiple dword variants. These are convenient to use from some front-end
implementations.

Also modified the existing llvm.SI.load.const intrinsic to common up the
underlying implementation.

This modification also requires that we can lower to non-uniform loads correctly
by splitting larger dword variants into sizes supported by the non-uniform
versions of the load.

V2: Addressed minor review comments.
V3: i1 glc is now i32 cachepolicy for consistency with buffer and
    tbuffer intrinsics, plus fixed formatting issue.
V4: Added glc test.

Subscribers: arsenm, kzhuravl, jvesely, wdng, nhaehnle, yaxunl, dstuttard, t-tye, llvm-commits

Differential Revision: https://reviews.llvm.org/D51098

Change-Id: I83a6e00681158bb243591a94a51c7baa445f169b

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@340684 91177308-0d34-0410-b5e6-96231b3b80d8

2018-08-25 14:53:17 +00:00

..

ADCE

[DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label.

2018-05-09 02:40:45 +00:00

AddDiscriminators

[DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label.

2018-05-09 02:40:45 +00:00

AggressiveInstCombine

[AggressiveInstCombine] avoid crashing on unsimplified code (PR37446)

2018-05-14 13:43:32 +00:00

AlignmentFromAssumptions

[AlignmentFromAssumptions] Set source and dest alignments of memory intrinsiscs separately

2018-02-22 18:55:59 +00:00

ArgumentPromotion

[ArgumentPromotion] don't break musttail invariant PR36543

2018-03-02 00:59:27 +00:00

AtomicExpand

[AtomicExpandPass] Widen partword atomicrmw or/xor/and before tryExpandAtomicRMW

2018-08-17 14:03:37 +00:00

BDCE

[DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label.

2018-05-09 02:40:45 +00:00

BlockExtractor

BlockExtractor: Don’t delete functions directly

2018-03-12 22:28:18 +00:00

BranchFolding

…

CalledValuePropagation

Add CalledValuePropagation pass

2017-10-25 13:40:08 +00:00

CallSiteSplitting

[CallSiteSplit] Make sure we remove nonnull if the parameter turns out to be a constant.

2018-04-23 20:09:08 +00:00

CodeExtractor

[CodeExtractor] Use 'normal destination' BB as insert point to store invoke results.

2018-08-21 20:07:46 +00:00

CodeGenPrepare

[CodeGenPrepare] Set debug locs when folding a comparison into a uadd.with.overflow

2018-08-22 18:15:03 +00:00

ConstantHoisting

[DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label.

2018-05-09 02:40:45 +00:00

ConstantMerge

Canonicalize the representation of empty an expression in DIGlobalVariableExpression

2017-08-30 18:06:51 +00:00

ConstProp

[X86] Add constant folding for AVX512 versions of scalar floating point to integer conversion intrinsics.

2018-08-12 22:09:54 +00:00

Coroutines

[Coroutines] PR34897: Fix incorrect elisions

2018-05-11 03:12:28 +00:00

CorrelatedValuePropagation

[CVP] Extend tests to illustrate an old patch isn't needed

2018-08-24 21:56:43 +00:00

CrossDSOCFI

[cfi] Build __cfi_check as Thumb when applicable.

2017-08-29 22:29:15 +00:00

DCE

[DCE] Salvage debug info from dead insts

2018-02-15 22:26:18 +00:00

DeadArgElim

[DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label.

2018-05-09 02:40:45 +00:00

DeadStoreElimination

[BasicAA] Don't assume tail calls with byval don't alias allocas

2018-08-14 01:24:35 +00:00

DivRemPairs

[DivRemPairs] split tests per target to account for bots that don't build for all targets

2017-09-09 14:10:59 +00:00

EarlyCSE

[AMDGPU] Add support for multi-dword s.buffer.load intrinsic

2018-08-25 14:53:17 +00:00

EliminateAvailableExternally

…

EntryExitInstrumenter

[DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label.

2018-05-09 02:40:45 +00:00

ExpandMemCmp/X86

[x86, MemCmpExpansion] allow 2 pairs of loads per block (PR33325)

2018-01-06 16:16:04 +00:00

Float2Int

[PM] Port float2int to the new pass manager

2016-06-24 23:32:02 +00:00

ForcedFunctionAttrs

…

FunctionAttrs

llvm: Add support for "-fno-delete-null-pointer-checks"

2018-07-09 22:27:23 +00:00

FunctionImport

Restore "[ThinLTO] Ensure we always select the same function copy to import"

2018-07-16 15:30:27 +00:00

GCOVProfiling

[DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label.

2018-05-09 02:40:45 +00:00

GlobalDCE

[PM] Teach the PGO instrumentation pasess to run GlobalDCE before

2017-05-25 07:15:09 +00:00

GlobalMerge

[GlobalMerge] Allow merging globals with explicit section markings.

2018-08-02 23:54:16 +00:00

GlobalOpt

[GlobalOpt] Don't apply fastcc if it would break inalloca invariants

2018-08-09 17:29:26 +00:00

GlobalSplit

Fix one-after-the-end type metadata handling in globalsplit.

2017-03-07 22:18:48 +00:00

GuardWidening

[GuardWidening] Ignore guards with trivial conditions

2018-08-22 02:40:49 +00:00

GVN

[GVN] Invalidate cached info for phis when setting dead predecessors to undef

2018-08-23 12:48:17 +00:00

GVNHoist

[Local] Make DoesKMove required for combineMetadata.

2018-08-24 11:40:04 +00:00

GVNSink

[Local] Make DoesKMove required for combineMetadata.

2018-08-24 11:40:04 +00:00

IndirectBrExpand

Introduce the "retpoline" x86 mitigation technique for variant #2 of the speculative execution vulnerabilities disclosed today, specifically identified by CVE-2017-5715, "Branch Target Injection", and is one of the two halves to Spectre..

2018-01-22 22:05:25 +00:00

IndVarSimplify

[SimplifyIndVar] Canonicalize comparisons to unsigned while eliminating truncs

2018-07-27 09:43:39 +00:00

InferAddressSpaces

[AMDGPU] Switch to the new addr space mapping by default

2018-02-02 16:07:16 +00:00

InferFunctionAttrs

[LibCalls] Added returned attribute to libcalls

2018-08-23 05:18:23 +00:00

Inline

Revert [Inliner] Attribute callsites with inline remarks

2018-08-24 16:39:41 +00:00

InstCombine

[InstCombine] add tests for shuffle+binop transform; NFC

2018-08-25 14:37:08 +00:00

InstMerge

[MergedLoadStoreMotion] Fix a debug invariant bug in mergeStores

2018-05-09 06:52:12 +00:00

InstNamer

Add basic test case for -instnamer

2017-05-08 23:18:46 +00:00

InstSimplify

[InstSimplify] use isKnownNeverNaN to fold more fcmp ord/uno

2018-08-21 14:45:13 +00:00

InterleavedAccess

[X86][LLVM]Expanding Supports lowerInterleaved{store|load}() in X86InterleavedAccess (VF64 stride 3-4)

2017-10-02 07:35:25 +00:00

Internalize

…

IPConstantProp

Recommit r333268: [IPSCCP] Use PredicateInfo to propagate facts from cmp instructions.

2018-08-23 11:04:00 +00:00

IRCE

[IRCE] Fix miscompile with range checks against negative values

2018-05-19 13:06:37 +00:00

JumpThreading

[Local] Make DoesKMove required for combineMetadata.

2018-08-24 11:40:04 +00:00

LCSSA

[DebugInfo][LCSSA] Preserve debug location in lcssa phis

2018-07-31 14:54:52 +00:00

LICM

[LICM] Hoist an invariant_start out of loops if there are no stores executed before it

2018-08-24 16:24:48 +00:00

LoadStoreVectorizer

[LSV] Look through selects for consecutive addresses

2018-07-25 21:33:00 +00:00

LoopDataPrefetch

[DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label.

2018-05-09 02:40:45 +00:00

LoopDeletion

[LoopDeletion] Handle users in unreachable block

2018-01-12 07:24:43 +00:00

LoopDistribute

[DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label.

2018-05-09 02:40:45 +00:00

LoopIdiom

SCEV should forget all loops containing a deleted block.

2018-08-09 17:53:26 +00:00

LoopInstSimplify

Update MemorySSA in LoopInstSimplify.

2018-08-22 20:05:21 +00:00

LoopInterchange

NFC - Various typo fixes in tests

2018-07-04 13:28:39 +00:00

LoopLoadElim

[PM] Port LoopLoadElimination to the new pass manager and wire it into

2017-01-27 01:32:26 +00:00

LoopPredication

[LoopPredication] Add profitability check based on BPI

2018-03-22 16:03:59 +00:00

LoopReroll

[LoopReroll] Rewrite induction variable rewriting.

2018-06-22 22:58:55 +00:00

LoopRotate

[ADCE] Remove the need of DomTree

2018-08-04 02:50:12 +00:00

LoopSimplify

[DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label.

2018-05-09 02:40:45 +00:00

LoopSimplifyCFG

Update MemorySSA in LoopSimplifyCFG.

2018-08-22 20:10:21 +00:00

LoopStrengthReduce

SCEVExpander::expandAddRecExprLiterally(): check before casting as Instruction

2018-06-29 07:44:20 +00:00

LoopUnroll

LoopUnroll: Allow analyzing intrinsic call costs

2018-06-26 18:51:17 +00:00

LoopUnrollAndJam

[UnJ] Improve explicit loop count checks

2018-08-11 07:37:31 +00:00

LoopUnswitch

[LoopUnswitch] Fix SCEV invalidation in unswitching

2018-05-23 10:09:53 +00:00

LoopVectorize

Avoid dbg.value use-before-def in a few tests (NFC)

2018-08-21 23:42:08 +00:00

LoopVersioning

Re-apply "[SCEV] Strengthen StrengthenNoWrapFlags (reapply r334428)."

2018-07-13 23:58:46 +00:00

LoopVersioningLICM

[LAA] Correctly return a half-open range in expandBounds

2017-04-05 09:24:26 +00:00

LowerAtomic

LowerAtomic: Don't skip optnone functions; atomic still need lowering (PR34020)

2017-08-23 15:43:28 +00:00

LowerExpectIntrinsic

Fix builin_expect lowering bug

2017-06-07 18:32:24 +00:00

LowerGuardIntrinsic

[PM] Port LowerGuardIntrinsic to the new PM.

2016-07-28 22:08:41 +00:00

LowerInvoke

[PM] Port LowerInvoke to the new pass manager

2016-08-12 17:28:27 +00:00

LowerSwitch

LowerSwitch: Fix producing invalid IR on unreachable code

2017-04-21 23:54:12 +00:00

LowerTypeTests

[LowerTypeTests] Limit when icall jumptable entries are emitted

2018-07-13 19:57:39 +00:00

Mem2Reg

Improve ConvertDebugDeclareToDebugValue

2018-06-26 06:17:00 +00:00

MemCpyOpt

[MemCpyOpt] Skip optimizing basic blocks not reachable from entry

2018-04-23 19:55:04 +00:00

MergeFunc

[MergeFunctions] Fix merging of small weak functions

2018-05-15 11:31:07 +00:00

MergeICmps

[MergeICmps] Don't crash when memcmp is not available

2018-05-19 12:51:59 +00:00

MetaRenamer

Remove alignment argument from memcpy/memmove/memset in favour of alignment attributes (Step 1)

2018-01-19 17:13:12 +00:00

NameAnonGlobals

[ThinLTO] Ensure anonymous globals renamed even at -O0

2016-09-17 20:40:16 +00:00

NaryReassociate

[NaryReassociate] Detect deleted instr with WeakVH

2018-05-24 06:09:02 +00:00

NewGVN

[NewGVN] Add tests for r340031.

2018-08-17 14:39:53 +00:00

ObjCARC

[ObjCARC] Prevent code motion into a catchswitch

2018-05-16 04:52:18 +00:00

PartiallyInlineLibCalls

[PartiallyInlineLibCalls][x86] add TTI hook to allow sqrt inlining to depend on arg rather than result

2017-11-27 21:15:43 +00:00

PGOProfile

[PGO] add target md5sum in warning message for icall

2018-08-24 21:38:24 +00:00

PhaseOrdering

[PhaseOrdering] remove stale comments; NFC

2018-05-09 23:10:46 +00:00

PlaceSafepoints

Remove alignment argument from memcpy/memmove/memset in favour of alignment attributes (Step 1)

2018-01-19 17:13:12 +00:00

PreISelIntrinsicLowering

[PM] Port PreISelIntrinsicLowering to the new PM

2016-06-24 20:13:42 +00:00

PruneEH

[PM][FunctionAttrs] add NoUnwind attribute inference to PostOrderFunctionAttrs pass

2018-03-23 21:46:16 +00:00

Reassociate

[Constants] add identity constants for fadd/fmul

2018-07-03 17:12:59 +00:00

Reg2Mem

…

RewriteStatepointsForGC

NFC - Various typo fixes in tests

2018-07-04 13:28:39 +00:00

SafeStack

[SafeStack] Set debug location for calls to __safestack_pointer_address.

2018-08-24 20:42:32 +00:00

SampleProfile

[SampleFDO] Fix a bug in getOrCompHotCountThreshold/getOrCompColdCountThreshold

2018-08-07 18:13:10 +00:00

Scalarizer

[DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label.

2018-05-09 02:40:45 +00:00

SCCP

[BasicAA] Use PhiValuesAnalysis if available when handling phi alias

2018-07-30 11:52:08 +00:00

SeparateConstOffsetFromGEP

[Split GEP] handle trunc() in separate-const-offset-from-gep pass.

2018-05-11 21:13:19 +00:00

SimpleLoopUnswitch

[PM/Unswitch] Fix a collection of closely related issues with trivial

2018-07-10 08:36:05 +00:00

SimplifyCFG

[NFC] Fixed build, updated tests

2018-08-12 18:32:53 +00:00

Sink

[Sink] Really really fix predicate in legality check

2018-01-11 21:28:57 +00:00

SLPVectorizer

[SLP] Fix insert point for reused extract instructions.

2018-08-07 19:21:05 +00:00

SpeculateAroundPHIs

Add a new pass to speculate around PHI nodes with constant (integer) operands when profitable.

2017-11-28 11:32:31 +00:00

SpeculativeExecution

Revert r301950: SpeculativeExecution: Stop using whitelist for costs

2017-05-10 12:30:07 +00:00

SROA

[SROA] Preserve DebugLoc when rewriting alloca partitions

2018-06-28 18:58:30 +00:00

StraightLineStrengthReduce

AMDGPU: Mark all unspecified CC functions in tests as amdgpu_kernel

2017-03-21 21:39:51 +00:00

StripDeadPrototypes

…

StripSymbols

Canonicalize the representation of empty an expression in DIGlobalVariableExpression

2017-08-30 18:06:51 +00:00

StructurizeCFG

StructurizeCFG: Adjust the loop depth for a subregion to order the nodes correctly

2018-05-23 18:34:48 +00:00

SyntheticCountsPropagation

Add a pass to generate synthetic function entry counts.

2018-01-09 19:39:35 +00:00

TailCallElim

[TailCallElim] Preserve DT and PDT

2018-08-04 08:13:47 +00:00

ThinLTOBitcodeWriter

[ThinLTOBitcodeWriter] Emit summaries for regular LTO modules

2018-06-01 15:20:47 +00:00

Util

[PredicateInfo] Use custom mangling to support ssa_copy with unnamed types.

2018-07-24 14:49:52 +00:00

WholeProgramDevirt

Runtime flag to control branch funnel threshold

2018-04-06 21:32:36 +00:00