Commit Graph

351 Commits

Author SHA1 Message Date
Tim Northover
fbb631183a AArch64: fall back to generic code for out of range extract/insert.
rdar://problem/17624784

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213059 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-15 10:00:26 +00:00
Tim Northover
26012cec89 AArch64: remove unnecessary pseudo-instruction.
Sufficiently twisted use of TableGen lets us write patterns directly for f16
(as an i16 promoted to i32) -> f32 conversion.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212933 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-14 11:16:02 +00:00
Saleem Abdulrasool
01c06d7954 AArch64: add support for llvm.aarch64.hint intrinsic
This adds a llvm.aarch64.hint intrinsic to mirror the llvm.arm.hint in order to
support the various hint intrinsic functions in the ACLE.

Add an optional pattern field that permits the subclass to specify the pattern
that matches the selection.  The intrinsic pattern is set as mayLoad, mayStore,
so overload the value for the definition of the hint instruction.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212883 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-12 21:20:49 +00:00
Oliver Stannard
cb047f2a74 ARM: Allow __fp16 as a function arg or return type for AArch64
ACLE 2.0 allows __fp16 to be used as a function argument or return
type. This enables this for AArch64.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212812 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-11 13:33:46 +00:00
Tim Northover
6b0ac2aa02 AArch64: correctly fast-isel i8 & i16 multiplies
We were asking for a register for type i8 or i16 which caused an assert.

rdar://problem/17620015

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212718 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-10 14:18:46 +00:00
Hao Liu
a3c15c19b8 [AArch64]Fix an assertion failure in DAG Combiner about concating 2 build_vector.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212677 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-10 03:41:50 +00:00
Jim Grosbach
a3edd6a038 AArch64: Better codegen for storing to __fp16.
Storing will generally be immediately preceded by rounding from an f32
or f64, so make sure to match those patterns directly to convert into the
FPR16 register class directly rather than going through the integer GPRs.

This also eliminates an extra step in the convert-from-f64 path
which was first converting to f32 and then to f16 from there.

rdar://17594379

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212638 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-09 18:55:52 +00:00
Jim Grosbach
05bb7c5045 AArch64: Better codegen for loading from __fp16.
Loading will generally extend to an f32 or an 64, so make sure
to match those patterns directly to load into the FPR16 register
class directly rather than going through the integer GPRs.

This also eliminates an extra step in the convert-to-f64 path
which was first converting to f32 and then to f64 from there.

rdar://17594379

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212573 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-08 23:28:48 +00:00
Louis Gerbarg
e7f8191b18 Allow AArch64FastISel to degrade graceully in the presence of an MVT::i128
Currently AArch64FastISel crashes if it tries to extend an integer into an
MVT::i128. This can happen by creating 128 bit integers like so:

  typedef unsigned int uint128_t __attribute__((mode(TI)));
  typedef int sint128_t __attribute__((mode(TI)));

This patch makes EmitIntExt check for their presence and then falls back to
SelectionDAG.

Tests included.

rdar://17516686

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212492 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-07 21:37:51 +00:00
Tim Northover
3e16b022be CodeGen: it turns out that NAND is not the same thing as BIC. At all.
We've been performing the wrong operation on ARM for "atomicrmw nand" for
years, since "a NAND b" is "~(a & b)" rather than ARM's very tempting "a & ~b".
This bled over into the generic expansion pass.

So I assume no-one has ever actually tried to do an atomic nand in the real
world. Oh well.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212443 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-07 09:06:35 +00:00
Kevin Qin
307e97d066 [AArch64] Normalize all constants to build a vector.
The value of constant operands will be truncated to fit element width.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212428 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-07 02:45:40 +00:00
Chandler Carruth
c179202bb4 [aarch64] Add a test that should have been in r212242 but I forgot to
add it. Sorry about that.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212251 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-03 02:12:26 +00:00
Chandler Carruth
70968365db [codegen,aarch64] Add a target hook to the code generator to control
vector type legalization strategies in a more fine grained manner, and
change the legalization of several v1iN types and v1f32 to be widening
rather than scalarization on AArch64.

This fixes an assertion failure caused by scalarizing nodes like "v1i32
trunc v1i64". As v1i64 is legal it will fail to scalarize v1i32.

This also provides a foundation for other targets to have more granular
control over how vector types are legalized.

Patch by Hao Liu, reviewed by Tim Northover. I'm committing it to allow
some work to start taking place on top of this patch as it adds some
really important hooks to the backend that I'd like to immediately start
using. =]

http://reviews.llvm.org/D4322

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212242 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-03 00:23:43 +00:00
Duncan P. N. Exon Smith
9b4509a759 AArch64: Re-enable AArch64AddressTypePromotion
This reverts commits r212189 and r212190.

While this pass was accidentally disabled (until r212073), r205437
slipped in a use of `auto` that should have been `auto&`.

This fixes PR20188.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212201 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-02 18:17:40 +00:00
Duncan P. N. Exon Smith
0c4d7a1ba5 XFAIL the test to go with r202189
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212190 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-02 17:07:03 +00:00
Chad Rosier
5ee1c8f6f0 Revert "Revert "MachineScheduler: better book-keeping for asserts.""
This reverts commit r212109, which reverted r212088.

However, disable the assert as it's not necessary for correctness.  There are
several corner cases that the assert needed to handle better for in-order
scheduling, but none of them are incorrect scheduler behavior. The assert is
mainly there to collect good unit tests like this and ensure that the
target-independent scheduler is working as expected with the various machine
models.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212187 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-02 16:46:08 +00:00
Chad Rosier
582dd1e608 Revert "MachineScheduler: better book-keeping for asserts."
This reverts commit r212088, which is causing a number of spec
failures.  Will provide reduced test cases shortly.
PR20057

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212109 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-01 17:23:11 +00:00
Andrew Trick
5c7ec29ebc MachineScheduler: better book-keeping for asserts.
Fixes another test case under PR20057.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212088 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-01 03:23:13 +00:00
Duncan P. N. Exon Smith
91fa94884d AArch64: Actually do address type promotion
AArch64AddressTypePromotion was doing nothing because it was using the
old semantics of `Use` and `uses()`, when it really wanted to get at the
`users()`.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212073 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-30 23:42:14 +00:00
Chad Rosier
99f2d6fcc2 [AArch64] Unsized types don't specify an alignment.
PR20109

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212045 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-30 15:03:00 +00:00
Chad Rosier
e7dfa85e85 [AArch64] Convert mul x, -(pow2 +/- 1) to shift + add/sub.
The combine for mul x, pow2 +/- 1 is unchanged. Test cases for
both combines as well as mul x, pow2 have been added as well.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212044 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-30 14:51:14 +00:00
Chad Rosier
d7be29696d [AArch64] Fix memset ICE when memset value is f128.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211960 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 21:05:09 +00:00
Andrew Trick
e8f8db1c5a MachineScheduler: add some book-keeping to fix an assert.
Fixe for Bug 20057 - Assertion failied in llvm::SUnit* llvm::SchedBoundary::pickOnlyChoice(): Assertion `i <= (HazardRec->getMaxLookAhead() + MaxObservedStall) && "permanent hazard"'

Thanks to Chad for the test case.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211865 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 04:57:05 +00:00
Weiming Zhao
c33b4883b3 Resubmit commit r211533
"Fix PR20056: Implement pseudo LDR <reg>, =<literal/label> for AArch64"
Missed files are added in this commit.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211605 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-24 16:21:38 +00:00
Kevin Qin
8c0787e83a [AArch64] Fix a build_vector pattern match fail
caused by defect in isBuildVectorAllZeros().

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211567 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-24 05:37:27 +00:00
Arnold Schwaighofer
5d5ddf9663 Add a triple so that right syntax is choosen on mac osx systems
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211188 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 17:20:49 +00:00
Kevin Qin
74287ec34c [AArch64] Fix a pattern match failure caused by creating improper CONCAT_VECTOR.
ReconstructShuffle() may wrongly creat a CONCAT_VECTOR trying to
concat 2 of v2i32 into v4i16. This commit is to fix this issue and
try to generate UZP1 instead of lots of MOV and INS.
Patch is initalized by Kevin Qin, and refactored by Tim Northover.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211144 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 05:54:42 +00:00
Tim Northover
c22960dba6 AArch64: estimate inline asm length during branch relaxation
To make sure branches are in range, we need to do a better job of estimating
the length of an inline assembly block than "it's probably 1 instruction, who'd
write asm with more than that?".

Fortunately there's already a (highly suspect, see how many ways you can think
of to break it!) callback for this purpose, which is used by the other targets.

rdar://problem/17277590

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211095 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 11:31:42 +00:00
James Molloy
b3820b4289 [AArch64] Fix a fencepost error in lowering for llvm.aarch64.neon.uqshl.
Patch by Jiangning Liu!



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211014 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-16 10:39:21 +00:00
Tim Northover
8bfc50e4a9 AArch64: improve handling & modelling of FP_TO_XINT nodes.
There's probably no acatual change in behaviour here, just updating
the LowerFP_TO_INT function to be more similar to the reverse
implementation and updating costs to current CodeGen.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210985 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-15 09:27:15 +00:00
Tim Northover
94fe5c1fe2 AArch64: improve vector [su]itofp handling.
This somehow got missed in the AArch64 merge, so should fix a
performance regression since 3.4.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210984 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-15 09:27:06 +00:00
Jiangning Liu
c5bc067a0f Move GlobalMerge from Transform to CodeGen.
This patch is to move GlobalMerge pass from Transform/Scalar                                                           
to CodeGen, because GlobalMerge depends on TargetMachine.
In the mean time, the macro INITIALIZE_TM_PASS is also moved
to CodeGen/Passes.h. With this fix we can avoid making
libScalarOpts depend on libCodeGen.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210951 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-13 22:57:59 +00:00
Tim Northover
8f2a85e099 IR: add "cmpxchg weak" variant to support permitted failure.
This commit adds a weak variant of the cmpxchg operation, as described
in C++11. A cmpxchg instruction with this modifier is permitted to
fail to store, even if the comparison indicated it should.

As a result, cmpxchg instructions must return a flag indicating
success in addition to their original iN value loaded. Thus, for
uniformity *all* cmpxchg instructions now return "{ iN, i1 }". The
second flag is 1 when the store succeeded.

At the DAG level, a new ATOMIC_CMP_SWAP_WITH_SUCCESS node has been
added as the natural representation for the new cmpxchg instructions.
It is a strong cmpxchg.

By default this gets Expanded to the existing ATOMIC_CMP_SWAP during
Legalization, so existing backends should see no change in behaviour.
If they wish to deal with the enhanced node instead, they can call
setOperationAction on it. Beware: as a node with 2 results, it cannot
be selected from TableGen.

Currently, no use is made of the extra information provided in this
patch. Test updates are almost entirely adapting the input IR to the
new scheme.

Summary for out of tree users:
------------------------------

+ Legacy Bitcode files are upgraded during read.
+ Legacy assembly IR files will be invalid.
+ Front-ends must adapt to different type for "cmpxchg".
+ Backends should be unaffected by default.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210903 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-13 14:24:07 +00:00
Chad Rosier
2e4cd27799 [AArch64] Basic Sched Model for Cortex-A57.
Patch by Dave Estes<cestes@codeaurora.org>
Differential Revision: http://reviews.llvm.org/D4008

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210705 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 21:06:56 +00:00
Jiangning Liu
f847ccb87a Global merge for global symbols.
This commit is to improve global merge pass and support global symbol merge.
The global symbol merge is not enabled by default. For aarch64, we need some
more back-end fix to make it really benifit ADRP CSE.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210640 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 06:44:53 +00:00
Chad Rosier
0db9526c1a [AArch64] Emit .ident compiler version attribute.
Patch by Ana Pazos<apazos@codeaurora.org>!

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210535 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 14:32:08 +00:00
Tim Northover
e1db6ac10b AArch64: disallow x30 & x29 as the destination for indirect tail calls
As Ana Pazos pointed out, these have to be restored to their incoming values
before a function returns; i.e. before the tail call. So they can't be used
correctly as the destination register.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210525 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 10:50:24 +00:00
Tim Northover
46b3076cd0 AArch64: teach FastISel how to handle offset FrameIndices
Previously we were abandonning the attempt, leading to some combination of
extra work (when selection of a load/store fails completely) and inferior code
(when this leads to a real memcpy call instead of inlining).

rdar://problem/17187463

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210520 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 09:52:44 +00:00
Tim Northover
292c7c6a48 AArch64: make FastISel memcpy emission more robust.
We were hitting an assert if FastISel couldn't create the load or store we
requested. Currently this happens for large frame-local addresses, though
CodeGen could be improved there.

rdar://problem/17187463

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210519 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 09:52:40 +00:00
Alp Toker
8aeca44558 Reduce verbiage of lit.local.cfg files
We can just split targets_to_build in one place and make it immutable.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210496 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-09 22:42:55 +00:00
Chad Rosier
451cc566c1 [AArch64] Fix the ordering of the accumulate operand in SchedRW list.
Patch by Dave Estes <cestes@codeaurora.org>
http://reviews.llvm.org/D4037

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210446 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-09 01:54:00 +00:00
Chad Rosier
0607e82c0a [AArch64] When combining constant mul of power of 2 plus/minus 1, prefer shift
plus add.  The shift can be folded into the add.  This only effects codegen
when the constant is 3.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210445 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-09 01:25:51 +00:00
Tilmann Scheller
9f039304b3 [AArch64] Add regression tests for the load/store optimizer which cover post-index update folding with sub rather than add.
The tests check that the following transform happens:

  (ldr|str) X, [x20]
   ...
  sub x20, x20, 
   ->
  (ldr|str) X, [x20], #-16

with X being either w0, x0, s0, d0 or q0.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210113 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-03 16:03:00 +00:00
Tim Northover
1410a2c906 AArch64: mark small types (i1, i8, i16) as promoted
This means the output of LowerFormalArguments returns a lowered
SDValue with the correct type (expected in SelectionDAGBuilder).
Without this, an assertion under a DEBUG macro triggers when those
types are passed on the stack.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210102 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-03 13:54:53 +00:00
Jiangning Liu
9a2d239740 [AArch64] Correctly deal with VPR stack parameter passing.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210067 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-03 03:25:09 +00:00
Tilmann Scheller
cca9c26920 [AArch64] Add some more regression tests for store pre-index update folding in the load/store optimizer.
Add tests for the following transform:

 add x8, x8, 
  ...
 str X, [x8]
  ->
 str X, [x8, ]!

with X being either w0, x0, s0, d0 or q0.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210021 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-02 12:33:33 +00:00
Tilmann Scheller
ef8a99c807 [AArch64] Add some more regression tests for load pre-index update folding in the load/store optimizer.
Add tests for the following transform:

 add x8, x8, 
  ...
 ldr X, [x8]
  ->
 ldr X, [x8, ]!

with X being either w0, x0, s0, d0 or q0.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210018 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-02 11:57:09 +00:00
Tim Northover
d0dbe02fd2 ARM & AArch64: make use of common cmpxchg idioms after expansion
The C and C++ semantics for compare_exchange require it to return a bool
indicating success. This gets mapped to LLVM IR which follows each cmpxchg with
an icmp of the value loaded against the desired value.

When lowered to ldxr/stxr loops, this extra comparison is redundant: its
results are implicit in the control-flow of the function.

This commit makes two changes: it replaces that icmp with appropriate PHI
nodes, and then makes sure earlyCSE is called after expansion to actually make
use of the opportunities revealed.

I've also added -{arm,aarch64}-enable-atomic-tidy options, so that
existing fragile tests aren't perturbed too much by the change. Many
of them either rely on undef/unreachable too pervasively to be
restored to something well-defined (particularly while making sure
they test the same obscure assert from many years ago), or depend on a
particular CFG shape, which is disrupted by SimplifyCFG.

rdar://problem/16227836

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209883 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-30 10:09:59 +00:00
Tim Northover
7be505ae88 AArch64 & ARM: remove undefined behaviour from some tests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209880 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-30 08:59:55 +00:00
Hao Liu
fd481d05be Test cases named with dates is a legacy rule not used now. Rename several test cases.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209877 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-30 05:58:19 +00:00