7832 Commits

Author SHA1 Message Date
Craig Topper
f543143a73 [X86] Add AVX 256-bit load and stores to fast isel.
I'm not sure why this was missing for so long.

This also exposed that we were picking floating point 256-bit VMOVNTPS for some integer types in normal isel for AVX1 even though VMOVNTDQ is available. In practice it doesn't matter due to the execution dependency fix pass, but it required extra isel patterns. Fixing that in a follow up commit.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271481 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-02 04:19:45 +00:00
Craig Topper
11a12c2c07 [AVX512] Remove masked load intrinsics. Clang now emits generic masked load intrinsics instead.
The intrinsics will be autoupgraded to the same generic masked loads.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271478 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-02 04:19:36 +00:00
Sanjay Patel
7cff23cad5 [x86, AVX2] regenerate checks
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271434 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-01 21:32:56 +00:00
Michael Kuperstein
d99a2e3ae2 [DAG] Improve legalization of INSERT_SUBVECTOR
When the index is known to be constant 0, insert directly into the the low half,
instead of spilling, performing the insert in-memory, and reloading.

Differential Revision: http://reviews.llvm.org/D20763


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271428 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-01 20:49:35 +00:00
Than McIntosh
90ac7e3463 Better fix for PR27903.
Summary:
Re-enable lifetime-start-on-first-use for stack coloring,
but explicitly disable it for slots with more than one start
or end lifetime marker.

Bug: 27903

Reviewers: wmi, tejohnson, qcolombet, gbiv

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D20739

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271412 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-01 17:55:10 +00:00
Simon Pilgrim
90828513f4 [X86][SSE] Added non-temporal store tests for all 512-bit vector types
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271393 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-01 13:58:00 +00:00
Simon Pilgrim
9bc536be82 [X86][SSE] Added non-temporal store tests for all 256-bit vector types
Also added KNL AVX-512 checks

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271391 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-01 13:20:25 +00:00
Simon Pilgrim
b79ae3d5b7 [X86][SSE] Added non-temporal store tests for all 128-bit integer vector types
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271389 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-01 13:05:00 +00:00
Michael Zuckerman
ac3b4c962d Adding back-end support to two bit scanning intrinsics
Adding LLVM back-end support to two intrinsics dealing with bit scan: _bit_scan_forward and _bit_scan_reverse.
Their functionality is as described in Intel intrinsics guide:
https://software.intel.com/sites/landingpage/IntrinsicsGuide/#text=_bit_scan_forward&expand=371,370
https://software.intel.com/sites/landingpage/IntrinsicsGuide/#text=_bit_scan_reverse&expand=371,370

Commit on behalf of Omer Paparo Bivas


Differential Revision: http://reviews.llvm.org/D19915



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271386 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-01 12:02:37 +00:00
Craig Topper
323e9f1870 Revert r271362 "[AVX512] Remove masked load intrinsics. Clang now emits generic masked load intrinsics instead."
Looks like something isn't quite right still. Also forgot to move the test cases to an autoupgrade test.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271363 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-01 05:57:55 +00:00
Craig Topper
c1bee8aab0 [AVX512] Remove masked load intrinsics. Clang now emits generic masked load intrinsics instead.
The intrinsics will be autoupgraded to the same generic masked loads.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271362 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-01 05:35:16 +00:00
Kevin B. Smith
5830c5c818 [X86]: Add a pattern that uses GR16_ABCD rather than GR32_ABCD to avoid falsely marking whole 32 bit register as live.
Differential Revision: http://reviews.llvm.org/D20649


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271341 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-31 22:00:12 +00:00
Simon Pilgrim
a8a5e74bf9 [X86][SSE] Add load-folding patterns for (V)CVTDQ2PD (PR27291)
Added patterns for (V)CVTDQ2PD -> 2f64 loading from a 64-bit source.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271269 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-31 12:04:35 +00:00
Igor Breger
ae9c817ebb [AVX512] Fix intrinsic vcvtps2ph lowering.
Differential Revision: http://reviews.llvm.org/D20788

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271255 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-31 08:04:21 +00:00
Igor Breger
7720e47020 Fix intrinsic vbroadcast{i32|f32}x2 lowering.
Differential Revision: http://reviews.llvm.org/D20780

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271254 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-31 07:43:39 +00:00
Craig Topper
acc7ed7b83 [AVX512] Remove masked store intrinsics. Clang now emits generic masked store intrinsics instead.
The intrinsics will be autoupgraded to the same generic masked stores.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271245 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-31 01:50:02 +00:00
Saleem Abdulrasool
a2bdb6de81 X86: permit using SjLj EH on x86 targets as an option
This adds support to the backed to actually support SjLj EH as an exception
model.  This is *NOT* the default model, and requires explicitly opting into it
from the frontend.  GCC supports this model and for MinGW can still be enabled
via the `--using-sjlj-exceptions` options.

Addresses PR27749!

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271244 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-31 01:48:07 +00:00
Craig Topper
2463f3bdaa [X86] Remove SSE/AVX unaligned store intrinsics as clang no longer uses them. Auto upgrade to native unaligned store instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271236 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-30 23:15:56 +00:00
Craig Topper
83ed492499 [X86] Use update_llc_test_checks.py to re-generate a test in preparation for an upcoming commit. NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271234 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-30 22:54:14 +00:00
Simon Pilgrim
47ca2d83df [X86][XOP] Split off auto-upgraded xop intrinsics
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271228 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-30 19:50:56 +00:00
Simon Pilgrim
4dedbb6410 [X86][SSE] Renamed pmovxrm tests
These aren't intrinsics anymore - as discussed on D20686

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271226 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-30 19:14:37 +00:00
Simon Pilgrim
16a2da8f0e [X86][AVX2] Regenerated AVX2 extension tests
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271224 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-30 18:49:57 +00:00
Simon Pilgrim
e03d7f3fb0 [X86][SSE] Updated storeu fast-isel tests to match clang builtin tests
Since rL271214 the headers have no longer used the storeu intrinsic

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271222 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-30 18:42:51 +00:00
Simon Pilgrim
5aef47d5b7 [X86][SSE2] Updated _mm_store_pd1/_mm_store1_pd fast-isel tests to match D20617
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271220 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-30 18:18:44 +00:00
Simon Pilgrim
687467768e [X86][SSE] (Reapplied) Replace (V)PMOVSX and (V)PMOVZX integer extension intrinsics with generic IR (llvm)
This patch removes the llvm intrinsics VPMOVSX and (V)PMOVZX sign/zero extension intrinsics and auto-upgrades to SEXT/ZEXT calls instead. We already did this for SSE41 PMOVSX sometime ago so much of that implementation can be reused.

Reapplied now that the the companion patch (D20684) removes/auto-upgrade the clang intrinsics has been committed.

Differential Revision: http://reviews.llvm.org/D20686

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271131 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-28 18:03:41 +00:00
Sanjay Patel
aea5485cfe [x86] avoid printing unnecessary sign bits of hex immediates in asm comments (PR20347)
It would be better to check the valid/expected size of the immediate operand, but this is
generally better than what we print right now.

Differential Revision: http://reviews.llvm.org/D20385



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271114 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-28 14:58:37 +00:00
Ahmed Bougacha
fe422064d9 [X86] Try to zero elts when lowering 256-bit shuffle with PSHUFB.
Otherwise we fallback to a blend of PSHUFBs later on.

Differential Revision: http://reviews.llvm.org/D19661

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271113 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-28 14:38:04 +00:00
Michael Kuperstein
780aab5bdb [X86] Detect SAD patterns and emit psadbw instructions.
This recommits r267649 with a fix for PR27539.

Differential Revision: http://reviews.llvm.org/D20598


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271033 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-27 18:53:22 +00:00
Simon Pilgrim
8c5e8cefea [X86][AVX] Removed some remains of old (pre-regeneration) filechecks
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271007 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-27 15:56:19 +00:00
Than McIntosh
019fdc9db3 Disable lifetime-start-on-first-use analysis.
Summary:
Turn off lifetime-start-on-first-use enhancement for the moment
pending a fix for bug 27903.

Bug: 27903

Reviewers: tejohnson, wmi, qcolombet, gbiv

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D20731

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271003 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-27 15:27:51 +00:00
Simon Pilgrim
3808026456 Revert: r270973 - [X86][SSE] Replace (V)PMOVSX and (V)PMOVZX integer extension intrinsics with generic IR (llvm)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270976 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-27 09:02:25 +00:00
Simon Pilgrim
337a02bbe0 [X86][SSE] Replace (V)PMOVSX and (V)PMOVZX integer extension intrinsics with generic IR (llvm)
This patch removes the llvm intrinsics VPMOVSX and (V)PMOVZX sign/zero extension intrinsics and auto-upgrades to SEXT/ZEXT calls instead. We already did this for SSE41 PMOVSX sometime ago so much of that implementation can be reused.

A companion patch (D20684) removes/auto-upgrade the clang intrinsics.

Differential Revision: http://reviews.llvm.org/D20686

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270973 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-27 08:49:15 +00:00
Mitch Bodart
f688cf78d6 [CodeGen] Fix problem with X86 byte registers in CriticalAntiDepBreaker
CriticalAntiDepBreaker was not correctly tracking defs of the high X86 byte
registers, leading to incorrect use of a busy register to break an
antidependence.

Fixes pr27681, and its duplicates pr27580, pr27804.

Differential Revision: http://reviews.llvm.org/D20456


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270935 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-26 23:08:52 +00:00
Simon Pilgrim
1f433c478c [X86][SSE] When lowering a 256-bit shuffle as PMOVZX, reduce the input vector to the lower 128-bit subvector.
Most often as not this is what it started out as, the extraction is zero-cost on AVX and the PMOVZX/PMOVSX folding logic is based around 128-bit loads.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270858 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-26 15:40:36 +00:00
Simon Pilgrim
7b28fe4dd5 [X86][SSE] Added load_zext_16i8_to_8i32 test
Odd issue with input vector not being folded into pmovzx on AVX2+ targets

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270852 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-26 14:45:30 +00:00
Igor Breger
10c072c9b4 [AVX512] Fix intrinsic cmp{sd|ss} lowering.
Differential Revision: http://reviews.llvm.org/D20615

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270843 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-26 12:42:25 +00:00
Simon Pilgrim
c8706b2d4f [X86][F16C] Added F16C fast-isel tests to match clang/test/CodeGen/f16c-builtins.c
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270837 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-26 10:26:56 +00:00
Simon Pilgrim
30719ebc26 [X86][AVX2] Added gather fast-isel tests to match clang/test/CodeGen/avx2-builtins.c
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270835 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-26 10:07:05 +00:00
Simon Pilgrim
b9dfbbc939 [X86][SSE41] Removed pblendw intrinsics tests - they are auto-upgraded
Equivalent tests included in sse41-intrinsics-x86-upgrade.ll - the i8/i32 immediate diff doesn't matter anymore

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270767 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 21:27:58 +00:00
Simon Pilgrim
e4a60f4f50 [X86][SSE41] Regenerated intrinsics tests
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270764 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 21:21:51 +00:00
Simon Pilgrim
05bd592af9 [X86][SSE41] Removed blendpd/blendps intrinsics tests - they are auto-upgraded
Equivalent tests included in sse41-intrinsics-x86-upgrade.ll

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270761 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 21:06:36 +00:00
Simon Pilgrim
07cea65abb [X86][AVX2] Regenerate avx2 vector shift tests
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270756 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 21:00:40 +00:00
Rafael Espindola
6edb5180dd Fix shouldAssumeDSOLocal for private linkage.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270746 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 19:55:16 +00:00
Hal Finkel
637f75559d [SDAG] Add a fallback multiplication expansion
LegalizeIntegerTypes does not have a way to expand multiplications for large
integer types (i.e. larger than twice the native bit width). There's no
standard runtime call to use in that case, and so we'd just assert.

Unfortunately, as it turns out, it is possible to hit this case from
standard-ish C code in rare cases. A particular case a user ran into yesterday
involved an __int128 induction variable and a loop with a quadratic (not
linear) recurrence which triggered some backend logic using SCEVExpander. In
this case, the BinomialCoefficient code in SCEV generates some i129 variables,
which get widened to i256. At a high level, this is not actually good (i.e. the
underlying optimization, PPCLoopPreIncPrep, should not be transforming the loop
in question for performance reasons), but regardless, the backend shouldn't
crash because of cost-modeling issues in the optimizer.

This is a straightforward implementation of the multiplication expansion, based
on the algorithm in Hacker's Delight. I validated it against the code for the
mul256b function from http://locklessinc.com/articles/256bit_arithmetic/ using
random inputs. There should be no functional change for previously-working code
(the new expansion code only replaces an assert).

Fixes PR19797.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270720 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 16:50:22 +00:00
Sanjay Patel
efe6410a4b [x86, AVX] allow explicit calls to VZERO* to modify state in VZeroUpperInserter pass (PR27823)
As noted in the review, there are still problems, so this doesn't the bug completely.

Differential Revision: http://reviews.llvm.org/D20529



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270718 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 16:39:47 +00:00
Simon Pilgrim
b05388fe7e [X86][AVX] Sync with clang/test/CodeGen/avx2-builtins.c
Only tests for the gather intrinsic are still to be added

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270710 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 15:30:08 +00:00
Simon Pilgrim
2475758d0c [X86][AVX2] Added more fast-isel tests to match clang/test/CodeGen/avx2-builtins.c
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270685 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 10:56:23 +00:00
Simon Pilgrim
df706b0baf [X86][AVX2] Begun adding fast-isel tests to match clang/test/CodeGen/avx2-builtins.c
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270683 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 10:15:06 +00:00
Simon Pilgrim
5c780f78b4 [X86][SSE2] Use storeu intrinsics for _mm_storeu_pd/_mm_storeu_pd tests
Also fixed name of _mm_store1_pd test

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270681 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 09:42:29 +00:00
Simon Pilgrim
88c2d3172d [X86][SSE] Use storeu intrinsics for _mm_storeu_ps test
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270680 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 09:28:06 +00:00