197 Commits

Author SHA1 Message Date
Nadav Rotem
14925e6b88 ARM Cost model: Use the size of vector registers and widest vectorizable instruction to determine the max vectorization factor.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172010 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-09 22:29:00 +00:00
Nadav Rotem
83be7b0dd3 Cost Model: Move the 'max unroll factor' variable to the TTI and add initial Cost Model support on ARM.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171928 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-09 01:15:42 +00:00
Nadav Rotem
8327474e4b Code cleanup: refactor the switch statements in the generation of reduction variables into an IR builder call.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171871 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-08 17:37:45 +00:00
Nadav Rotem
aae3d6fb53 Rename the enum members to match the LLVM coding style.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171868 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-08 17:23:17 +00:00
Nadav Rotem
111e5fe7e0 LoopVectorizer: Add support for floating point reductions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171812 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-07 23:13:00 +00:00
Nadav Rotem
9a6c6a3736 LoopVectorizer: When we vectorizer and widen loops we process many elements at once. This is a good thing, except for
small loops. On small loops post-loop that handles scalars (and runs slower) can take more time to execute than the
rest of the loop. This patch disables widening of loops with a small static trip count.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171798 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-07 21:54:51 +00:00
Chandler Carruth
1cbeaeb194 Simplify LoopVectorize to require target transform info and rely on it
being present. Make a member of one of the helper classes a reference as
part of this.

Reformatting goodness brought to you by clang-format.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171726 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-07 11:12:29 +00:00
Chandler Carruth
f3252b12e0 Merge the unused header file for LoopVectorizer into the source file.
This makes the loop vectorizer match the pattern followed by roughly all
other passses. =]

Notably, this header file was braken in several regards: it contained
a using namespace directive, global #define's that aren't globaly
appropriate, and global constants defined directly in the header file.

As a side benefit, lots of the types in this file become internal, which
will cause the optimizer to chew on this pass more effectively.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171723 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-07 10:44:06 +00:00
Chandler Carruth
8bd6c52396 Switch BBVectorize to directly depend on having a TTI analysis.
This could be simplified further, but Hal has a specific feature for
ignoring TTI, and so I preserved that.

Also, I needed to use it because a number of tests fail when switching
from a null TTI to the NoTTI nonce implementation. That seems suspicious
to me and so may be something that you need to look into Hal. I worked
it by preserving the old behavior for these tests with the flag that
ignores all target info.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171722 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-07 10:22:36 +00:00
Chandler Carruth
cf8814f975 Fix a slew of indentation and parameter naming style issues. This 80% of
this patch brought to you by the tool clang-format.

I wanted to fix up the names of constructor parameters because they
followed a bit of an anti-pattern by naming initialisms with CamelCase:
'Tti', 'Se', etc. This appears to have been in an attempt to not overlap
with the names of member variables 'TTI', 'SE', etc. However,
constructor arguments can very safely alias members, and in fact that's
the conventional way to pass in members. I've fixed all of these I saw,
along with making some strang abbreviations such as 'Lp' be simpler 'L',
or 'Lgl' be the word 'Legal'.

However, the code I was touching had indentation and formatting somewhat
all over the map. So I ran clang-format and fixed them.

I also fixed a few other formatting or doxygen formatting issues such as
using ///< on trailing comments so they are associated with the correct
entry.

There is still a lot of room for improvement of the formating and
cleanliness of this code. ;] At least a few parts of the coding
standards or common practices in LLVM's code aren't followed, the enum
naming rules jumped out at me. I may mix some of these while I'm here,
but not all of them.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171719 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-07 09:57:00 +00:00
Chandler Carruth
bb00800ff4 Fix the enumerator names for ShuffleKind to match tho coding standards,
and make its comments doxygen comments.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171688 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-07 03:20:02 +00:00
Chandler Carruth
be04929f7f Move TargetTransformInfo to live under the Analysis library. This no
longer would violate any dependency layering and it is in fact an
analysis. =]

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171686 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-07 03:08:10 +00:00
Chandler Carruth
be73c7b903 Switch the loop vectorizer from VTTI to just use TTI directly.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171620 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-05 10:16:02 +00:00
Chandler Carruth
abc227d9b3 Switch the BB vectorizer from the VTTI interface to the simple TTI
interface.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171618 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-05 10:05:28 +00:00
Nadav Rotem
d5b92c3891 iLoopVectorize: Non commutative operators can be used as reduction variables as long as the reduction chain is used in the LHS.
PR14803.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171583 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-05 01:15:47 +00:00
Paul Redmond
5767d91956 Do not vectorize loops with subtraction reductions
Since subtraction does not commute the loop vectorizer incorrectly vectorizes
reductions such as x = A[i] - x.

Disabling for now.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171537 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-04 22:10:16 +00:00
Nadav Rotem
608467d1e1 Fix a warning
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171525 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-04 21:08:44 +00:00
Nadav Rotem
e503319874 LoopVectorizer:
1. Add code to estimate register pressure.
2. Add code to select the unroll factor based on register pressure.
3. Add bits to TargetTransformInfo to provide the number of registers.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171469 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-04 17:48:25 +00:00
Nadav Rotem
e4159491a7 LoopVectorizer: Add support for loop-unrolling during vectorization for increasing the ILP. At the moment this feature is disabled by default and this commit should not cause any functional changes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171436 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-03 00:52:27 +00:00
Nadav Rotem
00a6bcaeb4 Avoid vectorization when the function has the "noimplicitflot" attribute.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171429 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-02 23:54:43 +00:00
Chandler Carruth
0b8c9a80f2 Move all of the header files which are involved in modelling the LLVM IR
into their new header subdirectory: include/llvm/IR. This matches the
directory structure of lib, and begins to correct a long standing point
of file layout clutter in LLVM.

There are still more header files to move here, but I wanted to handle
them in separate commits to make tracking what files make sense at each
layer easier.

The only really questionable files here are the target intrinsic
tablegen files. But that's a battle I'd rather not fight today.

I've updated both CMake and Makefile build systems (I think, and my
tests think, but I may have missed something).

I've also re-sorted the includes throughout the project. I'll be
committing updates to Clang, DragonEgg, and Polly momentarily.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171366 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-02 11:36:10 +00:00
Benjamin Kramer
6c30749583 Add IRBuilder::CreateVectorSplat and use it to simplify code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171349 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-01 19:55:16 +00:00
Bill Wendling
831737d329 Remove the Function::getFnAttributes method in favor of using the AttributeSet
directly.

This is in preparation for removing the use of the 'Attribute' class as a
collection of attributes. That will shift to the AttributeSet class instead.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171253 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-30 10:32:01 +00:00
Nadav Rotem
db2367512e LoopVectorizer: Fix a bug in the code that updates the loop exiting block.
LCSSA PHIs may have undef values. The vectorizer updates values that are used by outside users such as PHIs.
The bug happened because undefs are not loop values. This patch handles these PHIs.

PR14725



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171251 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-30 07:47:00 +00:00
Nadav Rotem
5dd839430c If all of the write objects are identified then we can vectorize the loop even if the read objects are unidentified.
PR14719.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171124 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-26 23:30:53 +00:00
Nadav Rotem
13eb1e7817 LoopVectorizer: Optimize the vectorization of consecutive memory access when the iteration step is -1
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171114 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-26 19:08:17 +00:00
Hal Finkel
a777284158 BBVectorize: Use VTTI to compute costs for intrinsics vectorization
For the time being this includes only some dummy test cases. Once the
generic implementation of the intrinsics cost function does something other
than assuming scalarization in all cases, or some target specializes the
interface, some real test cases can be added.

Also, for consistency, I changed the type of IID from unsigned to Intrinsic::ID
in a few other places.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171079 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-26 01:36:57 +00:00
Hal Finkel
1d59f5fa53 LoopVectorize: Enable vectorization of the fmuladd intrinsic
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171076 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-25 23:21:29 +00:00
Hal Finkel
64a7a24edf BBVectorize: Enable vectorization of the fmuladd intrinsic
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171075 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-25 22:36:08 +00:00
Nadav Rotem
9e5329d77e LoopVectorizer: When checking for vectorizable types, also check
the StoreInst operands.

PR14705.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171023 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-24 09:14:18 +00:00
Nadav Rotem
470ea9b72f LoopVectorizer: Fix an endless loop in the code that looks for reductions.
The bug was in the code that detects PHIs in if-then-else block sequence.

PR14701.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171008 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-24 01:22:06 +00:00
Benjamin Kramer
a1acf55738 LoopVectorize: Fix accidentaly inverted condition.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171001 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-23 13:21:41 +00:00
Benjamin Kramer
417872ed08 LoopVectorize: For scalars and void types there is no need to compute vector insert/extract costs.
Fixes an assert during the build of oggenc in the test suite.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171000 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-23 13:19:18 +00:00
Nadav Rotem
d54fed2786 Loop Vectorizer: Update the cost model of scatter/gather operations and make
them more expensive.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170995 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-23 07:23:55 +00:00
Bill Wendling
629fb82419 Change 'AttrVal' to 'AttrKind' to better reflect that it's a kind of attribute instead of the value of the attribute.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170972 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-22 00:37:52 +00:00
Roman Divacky
38b06020db Remove duplicate includes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170902 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-21 17:06:44 +00:00
Nadav Rotem
ebf395d39e Enable if-conversion.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170841 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-21 04:47:54 +00:00
Nadav Rotem
c2a537bd08 BB-Vectorizer: Check the cost of the store pointer type
and not the return type, which is void. A number of test
cases fail after adding the assertion in TTImpl.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170828 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-21 01:24:36 +00:00
Nadav Rotem
55306bdea5 Fix a bug in the code that checks if we can vectorize loops while using dynamic
memory bound checks.  Before the fix we were able to vectorize this loop from
the Livermore Loops benchmark:

for ( k=1 ; k<n ; k++ )
  x[k] = x[k-1] + y[k];



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170811 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-21 00:07:35 +00:00
Nadav Rotem
8386acd734 LoopVectorize: Fix a bug in the scalarization of instructions.
Before if-conversion we could check if a value is loop invariant
if it was declared inside the basic block. Now that loops have
multiple blocks this check is incorrect.

This fixes External/SPEC/CINT95/099_go/099_go



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170756 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20 20:24:40 +00:00
Nadav Rotem
d5d46ace89 Loop Vectorizer: turn-off if-conversion.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170708 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20 17:42:53 +00:00
Nadav Rotem
d3071bb042 Loop Vectorizer: Enable if-conversion.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170632 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20 02:00:02 +00:00
Nadav Rotem
a6603b35a9 whitespace
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170626 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20 00:49:56 +00:00
Benjamin Kramer
8b7c89f2d4 LoopVectorize: Make iteration over induction variables not depend on pointer values.
MapVector is a bit heavyweight, but I don't see a simpler way. Also the
InductionList is unlikely to be large. This should help 3-stage selfhost
compares (PR14647).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170528 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19 11:09:15 +00:00
Bill Wendling
034b94b170 Rename the 'Attributes' class to 'Attribute'. It's going to represent a single attribute in the future.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170502 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19 07:18:57 +00:00
Benjamin Kramer
0ef0e2e6d0 LoopVectorize: Emit reductions as log2(vectorsize) shuffles + vector ops instead of scalar operations.
For example on x86 with SSE4.2 a <8 x i8> add reduction becomes
	movdqa	%xmm0, %xmm1
	movhlps	%xmm1, %xmm1            ## xmm1 = xmm1[1,1]
	paddw	%xmm0, %xmm1
	pshufd	$1, %xmm1, %xmm0        ## xmm0 = xmm1[1,0,0,0]
	paddw	%xmm1, %xmm0
	phaddw	%xmm0, %xmm0
	pextrb	$0, %xmm0, %edx

instead of
	pextrb	$2, %xmm0, %esi
	pextrb	$0, %xmm0, %edx
	addb	%sil, %dl
	pextrb	$4, %xmm0, %esi
	addb	%dl, %sil
	pextrb	$6, %xmm0, %edx
	addb	%sil, %dl
	pextrb	$8, %xmm0, %esi
	addb	%dl, %sil
	pextrb	$10, %xmm0, %edi
	pextrb	$14, %xmm0, %edx
	addb	%sil, %dil
	pextrb	$12, %xmm0, %esi
	addb	%dil, %sil
	addb	%sil, %dl

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170439 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-18 18:40:20 +00:00
Nadav Rotem
9ad73e93a5 Enable the Loop Vectorizer by default for O2 and O3. Disable if-conversion by default. I plan to revert this patch later today.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170157 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-13 23:11:54 +00:00
Nadav Rotem
807dad62a0 Teach the cost model about the optimization in r169904: Truncation of induction variables costs the same as scalar trunc.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170051 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-13 00:21:03 +00:00
Nadav Rotem
d0b144c04c Fix indentation.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170005 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12 19:39:36 +00:00
Nadav Rotem
ae3b652f5c LoopVectorizer: Use the "optsize" attribute to decide if we are allowed to increase the function size.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170004 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12 19:29:45 +00:00