193 Commits

Author SHA1 Message Date
Craig Topper
d40aee4db0 [AVX-512] Remove and autoupgrade the masked integer compare intrinsics
Summary:
These intrinsics aren't used by clang and haven't been for a while.

There's some really terrible codegen in the 32-bit target for avx512bw due to i64 not being legal. But as I said these intrinsics aren't used by clang even before this patch so this codegen reflects our clang behavior today.

Reviewers: spatel, RKSimon, zvi, igorb

Reviewed By: RKSimon

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34389

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306047 91177308-0d34-0410-b5e6-96231b3b80d8
2017-06-22 20:11:01 +00:00
Galina Kistanova
fc18e2f236 Added missing break.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@303454 91177308-0d34-0410-b5e6-96231b3b80d8
2017-05-19 20:31:51 +00:00
Elad Cohen
ea59a24717 Support arbitrary address space pointers in masked gather/scatter intrinsics.
Fixes PR31789 - When loop-vectorize tries to use these intrinsics for a
non-default address space pointer we fail with a "Calling a function with a
bad singature!" assertion. This patch solves this by adding the 'vector of
pointers' argument as an overloaded type which will determine the address
space.

Differential revision: https://reviews.llvm.org/D31490



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@302018 91177308-0d34-0410-b5e6-96231b3b80d8
2017-05-03 12:28:54 +00:00
Simon Pilgrim
99a634fdf1 [X86][SSE] Update MOVNTDQA non-temporal loads to generic implementation (LLVM)
MOVNTDQA non-temporal aligned vector loads can be correctly represented using generic builtin loads, allowing us to remove the existing x86 intrinsics.

Clang companion patch: D31766.

Differential Revision: https://reviews.llvm.org/D31767

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@300325 91177308-0d34-0410-b5e6-96231b3b80d8
2017-04-14 15:05:35 +00:00
Matt Arsenault
bdbe8280f2 Add address space mangling to lifetime intrinsics
In preparation for allowing allocas to have non-0 addrspace.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@299876 91177308-0d34-0410-b5e6-96231b3b80d8
2017-04-10 20:18:21 +00:00
Michael Zuckerman
3341db988b [X86][LLVM] Converting __mm{|256|512}_movm_epi{8|16|32|64} LLVMIR call into generic intrinsics.
This patch is a part one of two reviews, one for the clang and the other for LLVM. 
The patch deletes the back-end intrinsics and adds support for them in the auto upgrade.

Differential Revision: https://reviews.llvm.org/D31393


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@299432 91177308-0d34-0410-b5e6-96231b3b80d8
2017-04-04 13:32:14 +00:00
George Burgess IV
3479ed63a6 Let llvm.objectsize be conservative with null pointers
This adds a parameter to @llvm.objectsize that makes it return
conservative values if it's given null.

This fixes PR23277.

Differential Revision: https://reviews.llvm.org/D28494


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@298430 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-21 20:08:59 +00:00
Daniel Berlin
ee517039d6 Keep attributes, calling convention, etc, when remangling intrinsic
Summary: Fix issue reported where intrinsic calling convention is dropped after r295253.

Reviewers: sanjoy

Subscribers: materi, llvm-commits

Differential Revision: https://reviews.llvm.org/D30422

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296563 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-01 01:49:13 +00:00
Craig Topper
c908a7f8b4 [IR][X86] Fix llvm version number in comments in AutoUpgrade. Forgot the next release is 5.0 not 4.1
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296092 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-24 05:35:07 +00:00
Craig Topper
474a418aa7 [AVX-512] Remove lzcnt intrinsics and autoupgrade them to generic ctlz intrinsics with select.
Clang has been emitting cltz intrinsics for a while now.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296091 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-24 05:35:04 +00:00
Craig Topper
59900325da [X86][IR] In AutoUpgrade, check explicitly for xop.vpcmov and xop.vpcmov.256 instead of anything starting with xop.vpcmov
There were some older intrinsics that only existed for less than a month in 2012 that still exist in some out of tree test files that start with this string, but aren't able to be handled by the current upgrade code and fire an assert. Now we'll go back to treating them as not intrinsics at all and just passing them through to output.

Fixes PR32041, sort of.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295930 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-23 03:22:14 +00:00
Craig Topper
58ee25f913 Recommit "[X86] Remove XOP VPCMOV intrinsics and autoupgrade them to native IR."
Clang has now been fixed to not use these intrinsics.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295571 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-18 21:50:58 +00:00
Craig Topper
b3d03a9308 Revert "[X86] Remove XOP VPCMOV intrinsics and autoupgrade them to native IR."
This reverts r295564. I missed that clang was still using the intrinsics despite our half implemented autoupgrade support.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295565 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-18 20:14:20 +00:00
Craig Topper
aa2f6f93a2 [X86] Remove XOP VPCMOV intrinsics and autoupgrade them to native IR.
It seems we were already upgrading 128-bit VPCMOV, but the intrinsic was still defined and being used in isel patterns. While I was here I also simplified the tablegen multiclasses.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295564 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-18 19:51:25 +00:00
Craig Topper
a6bf7e7433 [X86][IR] Simplify the XOP vpcmov autoupgrade code. NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295563 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-18 19:51:19 +00:00
Craig Topper
fc7a041c9e [X86][IR] Merge together some very similar AutoUpgrade handling. NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295562 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-18 19:51:14 +00:00
Craig Topper
47cf6aadec [AVX-512] Remove 128/256-bit masked fp max/min intrinsics. Upgrade them to legacy unmasked intrinsics and select instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295543 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-18 07:07:50 +00:00
Craig Topper
ff64e4651d [IR][X86] Simplify some AutoUpgrade code slightly. NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295426 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-17 07:07:24 +00:00
Craig Topper
33dd89087d [IR][X86] Rename an AutoUpgrade helper function to more accurately match what intrinsics it handles. NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295425 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-17 07:07:21 +00:00
Craig Topper
aace5e8709 [IR][X86] Move X86 specific portions of UpgradeIntrinsicFunction1 to a couple helper functions. NFC
This enables some early outs to avoid repeatedly using IsX86 check to qualify. I hope to continue to improve this to shorten the lengths of some of the string comparisons.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295424 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-17 07:07:19 +00:00
Craig Topper
fdf5214704 [AVX-512] Remove masked packss/packus intrinsics and autoupgrade to unmasked intrinsics with select instructions. For 512-bit add new unmasked intrinsics.
The new 512-bit unmasked intrinsics will make it easy to handle these with the SSE/AVX intrinsics in InstCombine where we currently have a TODO.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295290 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-16 06:31:54 +00:00
Daniel Berlin
d17bca97a5 Implement intrinsic mangling for literal struct types.
Fixes PR 31921

Summary:
Predicateinfo requires an ugly workaround to try to avoid literal
struct types due to the intrinsic mangling not being implemented.
This workaround actually does not work in all cases (you can hit the
assert by bootstrapping with -print-predicateinfo), and can't be made
to work without DFS'ing the type (IE copying getMangledStr and using a
version that detects if it would crash).

Rather than do that, i just implemented the mangling.  It seems
simple, since they are unified structurally.

Looking at the overloaded-mangling testcase we have, it actually turns
out the gc intrinsics will *also* crash if you try to use a literal
struct.  Thus, the testcase added fails before this patch, and works
after, without needing to resort to predicateinfo.

Reviewers: chandlerc, davide

Subscribers: llvm-commits, sanjoy

Differential Revision: https://reviews.llvm.org/D29925

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295253 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-15 23:16:20 +00:00
Justin Lebar
e5aefeb9b8 [NVPTX] Auto-upgrade some NVPTX intrinsics to LLVM target-generic code.
Summary:
Specifically, we upgrade llvm.nvvm.:

 * brev{32,64}
 * clz.{i,ll}
 * popc.{i,ll}
 * abs.{i,ll}
 * {min,max}.{i,ll,u,ull}
 * h2f

These either map directly to an existing LLVM target-generic
intrinsic or map to a simple LLVM target-generic idiom.

In all cases, we check that the code we generate is lowered to PTX as we
expect.

These builtins don't need to be backfilled in clang: They're not
accessible to user code from nvcc.

Reviewers: tra

Subscribers: majnemer, cfe-commits, llvm-commits, jholewinski

Differential Revision: https://reviews.llvm.org/D28793

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292694 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-21 01:00:32 +00:00
Chad Rosier
ef64b17169 [ARM] Remove rbit intrinsics and autoupgrade to generic bitreverse.
Testing already covered by CodeGen/ARM/rbit.ll

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291587 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-10 19:23:51 +00:00
Chad Rosier
33d492eb19 [AArch64] Add support for lowering bitreverse to the rbit instruction.
Differential Revision: https://reviews.llvm.org/D28379

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291575 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-10 17:20:33 +00:00
Craig Topper
a3ed852d30 [AVX-512] Remove vinsert intrinsics and autoupgrade to native shufflevectors. There are some codegen problems here that I'll try to fix in future commits.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290864 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-03 05:45:57 +00:00
Craig Topper
be3e531ac2 [AVX-512] Remove vextract intrinsics and autoupgrade to native shufflevectors. This unfortunately generates some really terrible code without VLX support due to v2i1 and v4i1 not being legal.
Hopefully we can improve that in future patches.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290863 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-03 05:45:46 +00:00
Craig Topper
48ee9a80ae [AVX-512] Remove masked pmuldq and pmuludq intrinsics and autoupgrade them to unmasked intrinsics plus a select.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290583 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-27 05:30:14 +00:00
Craig Topper
2d270b3115 [X86] Remove masking from 512-bit VPERMIL intrinsics in preparation for being able to constant fold them in InstCombineCalls like we do for 128/256-bit.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289350 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-11 01:26:44 +00:00
Craig Topper
df9e980077 [X86] Remove masking from 512-bit PSHUFB intrinsics in preparation for being able to constant fold it in InstCombineCalls like we do for 128/256-bit.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289344 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-10 23:09:43 +00:00
Craig Topper
4715a2cff9 [AVX-512] Remove 128/256 masked vpermil instrinsics and autoupgrade to a select around the unmasked avx1 intrinsics.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289340 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-10 21:15:52 +00:00
Craig Topper
a1123a921e [X86][IR] Move the autoupgrading of store intrinsics out of the main nested if/else chain. This should buy a little more time against the MSVC limit mentioned in PR31034.
The handlers for stores all return at the end of their block so they can be picked off early.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289339 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-10 21:15:48 +00:00
Craig Topper
fa494a82ff [AVX-512] Remove intrinsics for valignd/q and autoupgrade them to native shuffles.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287744 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-23 06:54:55 +00:00
Craig Topper
e5e77e4a92 [AVX-512] Replace masked 16-bit element variable shift intrinsics with new unmasked versions and selects.
The same thing was done to 32-bit and 64-bit element sizes previously.

This will allow us to support these shuffls in InstCombineCalls along with the other variable shift intrinsics.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287312 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-18 05:04:44 +00:00
Simon Pilgrim
f1f243fd34 [X86][AVX512] Autoupgrade lossless i32/u32 to f64 conversion intrinsics with generic IR
Both the (V)CVTDQ2PD (i32 to f64) and (V)CVTUDQ2PD (u32 to f64) conversion instructions are lossless and can be safely represented as generic SINT_TO_FP/UINT_TO_FP calls instead of x86 intrinsics without affecting final codegen.

LLVM counterpart to D26686

Differential Revision: https://reviews.llvm.org/D26736

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287108 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-16 14:48:32 +00:00
Ayman Musa
49b99259ed [X86][AVX512] Removing llvm x86 intrinsics for _mm_mask_move_{ss|sd} intrinsics.
Differential Revision: https://reviews.llvm.org/D26128



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287087 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-16 09:00:28 +00:00
Craig Topper
614df99de4 [X86] Remove the scalar intrinsics for fadd/fsub/fdiv/fmul
Summary: These intrinsics have been unused for clang for a while. This patch removes them. We auto upgrade them to extractelements, a scalar operation and then an insertelement. This matches the sequence used by clangs intrinsic file.

Reviewers: zvi, delena, RKSimon

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D26660

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287083 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-16 05:24:10 +00:00
Craig Topper
436c13969b [X86] Add LLVM version number for each intrinsic handled by auto upgrade for age tracking.
One day we'd like to remove some of this autoupgrade support and it will be easier if we know how long some of it has been around.

Differential Revision: https://reviews.llvm.org/D26321

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286933 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-15 05:04:51 +00:00
Craig Topper
d03c7ac104 [AVX-512] Remove and autoupgrade masked dword/qword variable shift intrinsics to the new unmasked versions and selects.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286786 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-14 01:53:22 +00:00
Craig Topper
5eb34e8e29 [X86][IR] Reduce the number of full string comparisons in the code that autoupgrades masked shift intrinsics.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286768 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-13 19:09:56 +00:00
Igor Breger
b70e96e258 revert commit r286761, some builds failed on Win platforms
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286765 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-13 15:48:11 +00:00
Ayman Musa
061d58f00f [X86][AVX512] Removing llvm x86 intrinsics for _mm_mask_move_{ss|sd} intrinsics.
Differential Revision: https://reviews.llvm.org/D26128



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286761 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-13 14:51:25 +00:00
Craig Topper
7aa2751105 [AVX-512] Remove the remaining masked shift by immediate or by single value. Autoupgrade them to recently introduced unmasked versions and a select.
After this I'll add the unmasked intrinsics to InstCombineCalls to finish making our handling of these types of shuffles consistent between AVX-512 and the legacy intrinsics.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286725 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-12 18:04:46 +00:00
George Burgess IV
ba372980dc Add a missing break statement. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286203 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-08 04:01:50 +00:00
Craig Topper
e69277c6bf [AVX-512] Remove masked pmovzx/pmovsx builtins and autoupgrade them to selects and native zext/sext.
This mostly reuses earlier autoupgrade support for the sse and avx equivalents. Just needed to add the code to add the select.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286092 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-07 02:12:57 +00:00
Craig Topper
e132a76033 [X86] Use StringRef::startswith to reduce a few compares in the intrinsic autoupgrade code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286090 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-07 00:13:42 +00:00
Craig Topper
9fd28c59b7 [AVX-512] Remove 128/256 masked pshufb intrinsics. Autoupgrade them to legacy intrinsics and a select.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286089 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-07 00:13:39 +00:00
Craig Topper
90419a0ef5 [AVX-512] Remove intrinsics for 128/256-bit masked variable shift. Instead upgrade them to a select and the older AVX2 intrinsic.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286073 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-06 16:29:19 +00:00
Craig Topper
4734b821e2 [AVX-512] Remove intrinsics for 128/256-bit masked shift by immediate. Instead upgrade them to a select and the older SSE/AVX2 intrinsic.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286072 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-06 16:29:14 +00:00
Craig Topper
cd57e732e4 [AVX-512] Remove intrinsics for 128/256-bit masked shift by single element in xmm. Instead upgrade them to a select and the older SSE/AVX2 intrinsic.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286070 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-06 16:29:08 +00:00