Commit Graph

32 Commits

Author SHA1 Message Date
Matt Arsenault 9f717fa86d AMDGPU: Add intrinsics for address space identification
The library currently uses ptrtoint and directly checks the queue ptr
for this, which counts as a pointer capture.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371009 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-05 02:20:39 +00:00
Matt Arsenault 6b4d361457 InferAddressSpaces: Fix cloning original addrspacecast
If an addrspacecast needed to be inserted again, this was creating a
clone of the original cast for each user. Just use the original, which
also saves losing the value name.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363562 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-17 14:13:29 +00:00
Matt Arsenault 6f9792a238 AMDGPU: Ignore subtarget for InferAddressSpaces
Even if the target doesn't have flat instructions, addrspace(0) is
still flat. It just happens to not work.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363561 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-17 14:13:24 +00:00
Fangrui Song 1002960b9d [lit] Delete empty lines at the end of lit.local.cfg NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363538 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-17 09:51:07 +00:00
Michael Liao dbb747a8e2 [InferAddressSpaces] Enhance the handling of cosntexpr.
Summary:
- Constant expressions may not be added in strict postorder as the
  forward instruction scan order. Thus, for a constant express (CE0), if
  its operand (CE1) is used in an previous instruction, they are not in
  postorder. However, different from
  `cloneInstructionWithNewAddressSpace`,
  `cloneConstantExprWithNewAddressSpace` doesn't bookkeep uninferred
  instructions for later resolving. That results in failure of inferring
  constant address.
- This patch adds the support to infer constant expression operand
  recursively, since there won't be loop, if that operand is another
  constant expression.

Reviewers: arsenm

Subscribers: jholewinski, jvesely, wdng, nhaehnle, hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61760

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@360431 91177308-0d34-0410-b5e6-96231b3b80d8
2019-05-10 14:57:42 +00:00
Eric Christopher 598198edbc Revert "Temporarily Revert "Add basic loop fusion pass.""
The reversion apparently deleted the test/Transforms directory.

Will be re-reverting again.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358552 91177308-0d34-0410-b5e6-96231b3b80d8
2019-04-17 04:52:47 +00:00
Eric Christopher 02cc44c1b9 Temporarily Revert "Add basic loop fusion pass."
As it's causing some bot failures (and per request from kbarton).

This reverts commit r358543/ab70da07286e618016e78247e4a24fcb84077fda.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358546 91177308-0d34-0410-b5e6-96231b3b80d8
2019-04-17 02:12:23 +00:00
Matt Arsenault 6e8fb99b69 IR: Add immarg attribute
This indicates an intrinsic parameter is required to be a constant,
and should not be replaced with a non-constant value.

Add the attribute to all AMDGPU and generic intrinsics that comments
indicate it should apply to. I scanned other target intrinsics, but I
don't see any obvious comments indicating which arguments are intended
to be only immediates.

This breaks one questionable testcase for the autoupgrade. I'm unclear
on whether the autoupgrade is supposed to really handle declarations
which were never valid. The verifier fails because the attributes now
refer to a parameter past the end of the argument list.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355981 91177308-0d34-0410-b5e6-96231b3b80d8
2019-03-12 21:02:54 +00:00
Xing GUO ca992cddc2 [Transforms] fix typo in test case. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355265 91177308-0d34-0410-b5e6-96231b3b80d8
2019-03-02 08:32:32 +00:00
Joey Gouly 1730d3775c [InferAddressSpaces] Fix crash on select of non-ptr operands
Check the operands of a select are pointers, to determine if it is an address
expression or not.

https://reviews.llvm.org/D58226


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354576 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-21 12:31:36 +00:00
Erik Pilkington 99ef84fa6f Add a 'dynamic' parameter to the objectsize intrinsic
This is meant to be used with clang's __builtin_dynamic_object_size.
When 'true' is passed to this parameter, the intrinsic has the
potential to be folded into instructions that will be evaluated
at run time. When 'false', the objectsize intrinsic behaviour is
unchanged.

rdar://32212419

Differential revision: https://reviews.llvm.org/D56761

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@352664 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-30 20:34:35 +00:00
Matt Arsenault 7c299a76cf AMDGPU: Fix tests using old number for constant address space
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@341770 91177308-0d34-0410-b5e6-96231b3b80d8
2018-09-10 02:54:25 +00:00
Yaxun Liu a2386c3b26 [AMDGPU] Switch to the new addr space mapping by default
This requires corresponding clang change.

Differential Revision: https://reviews.llvm.org/D40955


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@324101 91177308-0d34-0410-b5e6-96231b3b80d8
2018-02-02 16:07:16 +00:00
Daniel Neilson afa2e7e6a6 Remove alignment argument from memcpy/memmove/memset in favour of alignment attributes (Step 1)
Summary:
 This is a resurrection of work first proposed and discussed in Aug 2015:
   http://lists.llvm.org/pipermail/llvm-dev/2015-August/089384.html
and initially landed (but then backed out) in Nov 2015:
   http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20151109/312083.html

 The @llvm.memcpy/memmove/memset intrinsics currently have an explicit argument
which is required to be a constant integer. It represents the alignment of the
dest (and source), and so must be the minimum of the actual alignment of the
two.

 This change is the first in a series that allows source and dest to each
have their own alignments by using the alignment attribute on their arguments.

 In this change we:
1) Remove the alignment argument.
2) Add alignment attributes to the source & dest arguments. We, temporarily,
   require that the alignments for source & dest be equal.

 For example, code which used to read:
  call void @llvm.memcpy.p0i8.p0i8.i32(i8* %dest, i8* %src, i32 100, i32 4, i1 false)
will now read
  call void @llvm.memcpy.p0i8.p0i8.i32(i8* align 4 %dest, i8* align 4 %src, i32 100, i1 false)

 Downstream users may have to update their lit tests that check for
@llvm.memcpy/memmove/memset call/declaration patterns. The following extended sed script
may help with updating the majority of your tests, but it does not catch all possible
patterns so some manual checking and updating will be required.

s~declare void @llvm\.mem(set|cpy|move)\.p([^(]*)\((.*), i32, i1\)~declare void @llvm.mem\1.p\2(\3, i1)~g
s~call void @llvm\.memset\.p([^(]*)i8\(i8([^*]*)\* (.*), i8 (.*), i8 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.memset.p\1i8(i8\2* \3, i8 \4, i8 \5, i1 \6)~g
s~call void @llvm\.memset\.p([^(]*)i16\(i8([^*]*)\* (.*), i8 (.*), i16 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.memset.p\1i16(i8\2* \3, i8 \4, i16 \5, i1 \6)~g
s~call void @llvm\.memset\.p([^(]*)i32\(i8([^*]*)\* (.*), i8 (.*), i32 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.memset.p\1i32(i8\2* \3, i8 \4, i32 \5, i1 \6)~g
s~call void @llvm\.memset\.p([^(]*)i64\(i8([^*]*)\* (.*), i8 (.*), i64 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.memset.p\1i64(i8\2* \3, i8 \4, i64 \5, i1 \6)~g
s~call void @llvm\.memset\.p([^(]*)i128\(i8([^*]*)\* (.*), i8 (.*), i128 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.memset.p\1i128(i8\2* \3, i8 \4, i128 \5, i1 \6)~g
s~call void @llvm\.memset\.p([^(]*)i8\(i8([^*]*)\* (.*), i8 (.*), i8 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.memset.p\1i8(i8\2* align \6 \3, i8 \4, i8 \5, i1 \7)~g
s~call void @llvm\.memset\.p([^(]*)i16\(i8([^*]*)\* (.*), i8 (.*), i16 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.memset.p\1i16(i8\2* align \6 \3, i8 \4, i16 \5, i1 \7)~g
s~call void @llvm\.memset\.p([^(]*)i32\(i8([^*]*)\* (.*), i8 (.*), i32 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.memset.p\1i32(i8\2* align \6 \3, i8 \4, i32 \5, i1 \7)~g
s~call void @llvm\.memset\.p([^(]*)i64\(i8([^*]*)\* (.*), i8 (.*), i64 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.memset.p\1i64(i8\2* align \6 \3, i8 \4, i64 \5, i1 \7)~g
s~call void @llvm\.memset\.p([^(]*)i128\(i8([^*]*)\* (.*), i8 (.*), i128 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.memset.p\1i128(i8\2* align \6 \3, i8 \4, i128 \5, i1 \7)~g
s~call void @llvm\.mem(cpy|move)\.p([^(]*)i8\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i8 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.mem\1.p\2i8(i8\3* \4, i8\5* \6, i8 \7, i1 \8)~g
s~call void @llvm\.mem(cpy|move)\.p([^(]*)i16\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i16 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.mem\1.p\2i16(i8\3* \4, i8\5* \6, i16 \7, i1 \8)~g
s~call void @llvm\.mem(cpy|move)\.p([^(]*)i32\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i32 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.mem\1.p\2i32(i8\3* \4, i8\5* \6, i32 \7, i1 \8)~g
s~call void @llvm\.mem(cpy|move)\.p([^(]*)i64\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i64 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.mem\1.p\2i64(i8\3* \4, i8\5* \6, i64 \7, i1 \8)~g
s~call void @llvm\.mem(cpy|move)\.p([^(]*)i128\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i128 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.mem\1.p\2i128(i8\3* \4, i8\5* \6, i128 \7, i1 \8)~g
s~call void @llvm\.mem(cpy|move)\.p([^(]*)i8\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i8 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.mem\1.p\2i8(i8\3* align \8 \4, i8\5* align \8 \6, i8 \7, i1 \9)~g
s~call void @llvm\.mem(cpy|move)\.p([^(]*)i16\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i16 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.mem\1.p\2i16(i8\3* align \8 \4, i8\5* align \8 \6, i16 \7, i1 \9)~g
s~call void @llvm\.mem(cpy|move)\.p([^(]*)i32\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i32 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.mem\1.p\2i32(i8\3* align \8 \4, i8\5* align \8 \6, i32 \7, i1 \9)~g
s~call void @llvm\.mem(cpy|move)\.p([^(]*)i64\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i64 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.mem\1.p\2i64(i8\3* align \8 \4, i8\5* align \8 \6, i64 \7, i1 \9)~g
s~call void @llvm\.mem(cpy|move)\.p([^(]*)i128\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i128 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.mem\1.p\2i128(i8\3* align \8 \4, i8\5* align \8 \6, i128 \7, i1 \9)~g

 The remaining changes in the series will:
Step 2) Expand the IRBuilder API to allow creation of memcpy/memmove with differing
   source and dest alignments.
Step 3) Update Clang to use the new IRBuilder API.
Step 4) Update Polly to use the new IRBuilder API.
Step 5) Update LLVM passes that create memcpy/memmove calls to use the new IRBuilder API,
        and those that use use MemIntrinsicInst::[get|set]Alignment() to use
        getDestAlignment() and getSourceAlignment() instead.
Step 6) Remove the single-alignment IRBuilder API for memcpy/memmove, and the
        MemIntrinsicInst::[get|set]Alignment() methods.

Reviewers: pete, hfinkel, lhames, reames, bollu

Reviewed By: reames

Subscribers: niosHD, reames, jholewinski, qcolombet, jfb, sanjoy, arsenm, dschuff, dylanmckay, mehdi_amini, sdardis, nemanjai, david2050, nhaehnle, javed.absar, sbc100, jgravelle-google, eraman, aheejin, kbarton, JDevlieghere, asb, rbar, johnrusso, simoncook, jordy.potman.lists, apazos, sabuasal, llvm-commits

Differential Revision: https://reviews.llvm.org/D41675

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@322965 91177308-0d34-0410-b5e6-96231b3b80d8
2018-01-19 17:13:12 +00:00
Yaxun Liu 2bd22dfce7 InferAddressSpaces: Fix bug about replacing addrspacecast
InferAddressSpaces assumes the pointee type of addrspacecast
is the same as the operand, which is not always true and causes
invalid IR.

This bug cause build failure in HCC.

This patch fixes that.

Differential Revision: https://reviews.llvm.org/D39432


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316957 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-30 21:19:41 +00:00
Yaxun Liu d0585d352f [InferAddressSpaces] Fix assertion about null pointer
InferAddressSpaces does not check address space in collectFlatAddressExpressions,
which causes values with non flat address space put into Postorder and causes
assertion in cloneValueWithNewAddressSpace.

This patch fixes assertion in OpenCL 2.0 conformance test generic_address_space
subtest for amdgcn target.

Differential Revision: https://reviews.llvm.org/D34991


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@307349 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-07 02:40:13 +00:00
Nirav Dave d2492e7267 InferAddressSpaces: Avoid assertion failure with replacing identical
cloned constexpr

Have cloneConstantExprWithNewAddressSpaces return nullptr when
returning initial ConstantExpr.

Reviewers: arsenm

Subscribers: jholewinski, wdng, llvm-commits

Differential Revision: https://reviews.llvm.org/D33995

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@304975 91177308-0d34-0410-b5e6-96231b3b80d8
2017-06-08 13:20:55 +00:00
Matt Arsenault 754511fb00 InferAddressSpaces: Search constant expressions for addrspacecasts
These are pretty common when using local memory, and the 64-bit generic
addressing is much more expensive to compute.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@301711 91177308-0d34-0410-b5e6-96231b3b80d8
2017-04-28 22:52:41 +00:00
Matt Arsenault afc9030f67 InferAddressSpaces: Infer from just addrspacecasts
Eliminates some more cases where some subset of the addressing
computation remains flat. Some cases with addrspacecasts
in nested constant expressions are still left behind however.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@301704 91177308-0d34-0410-b5e6-96231b3b80d8
2017-04-28 22:18:08 +00:00
Matt Arsenault cfadbaaf1a InferAddressSpaces: Infer for just GEPs
Fixes leaving intermediate flat addressing computations
where a GEP instruction's source is a constant expression.

Still leaves behind a trivial addrspacecast + gep pair that
instcombine is able to handle, which ideally could be folded
here directly.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@301044 91177308-0d34-0410-b5e6-96231b3b80d8
2017-04-21 21:35:04 +00:00
Matt Arsenault 3c1dcddf86 AMDGPU: Add all atomicrmw fields to atomic.inc/dec
Add scope, order, isVolatile

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@299122 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-30 22:21:40 +00:00
Matt Arsenault d706d030af AMDGPU: Mark all unspecified CC functions in tests as amdgpu_kernel
Currently the default C calling convention functions are treated
the same as compute kernels. Make this explicit so the default
calling convention can be changed to a non-kernel.

Converted with perl -pi -e 's/define void/define amdgpu_kernel void/'
on the relevant test directories (and undoing in one place that actually
wanted a non-kernel).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@298444 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-21 21:39:51 +00:00
George Burgess IV 3479ed63a6 Let llvm.objectsize be conservative with null pointers
This adds a parameter to @llvm.objectsize that makes it return
conservative values if it's given null.

This fixes PR23277.

Differential Revision: https://reviews.llvm.org/D28494


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@298430 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-21 20:08:59 +00:00
Matt Arsenault e67c4a1dbe Bug 31948: Fix assertion when bitcasting constantexpr pointers
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295387 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-17 00:32:19 +00:00
Matt Arsenault b324cd9254 InferAddressSpaces: Handle more cases with constant select operands
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293859 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-02 03:37:22 +00:00
Matt Arsenault 40fe3a93e9 InferAddressSpaces: Handle select
This fails to handle some cases where one of the inputs is
a constant to be fixed in a later commit.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293723 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-01 00:08:53 +00:00
Matt Arsenault 34a0d5fe90 InferAddressSpaces: Fix broken casting of constants
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293718 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-31 23:48:40 +00:00
Matt Arsenault 1394a16b28 InferAddressSpaces: Handle icmp
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293593 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-31 02:17:32 +00:00
Matt Arsenault 35b092a75f InferAddressSpaces: Support memory intrinsics
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293587 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-31 01:56:57 +00:00
Matt Arsenault 6be67912a8 InferAddressSpaces: Support atomics
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293584 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-31 01:40:38 +00:00
Matt Arsenault de6cb7e695 InferAddressSpaces: Don't replace volatile users
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293582 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-31 01:30:16 +00:00
Matt Arsenault 264e91f294 AMDGPU: Implement hook for InferAddressSpaces
For now just port some of the existing NVPTX tests
and from an old HSAIL optimization pass which
approximately did the same thing.

Don't enable the pass yet until more testing is done.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293580 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-31 01:20:54 +00:00