Nadav Rotem
f1e0aebca1
PR16628: Fix a bug in the code that merges compares.
...
Compares return i1 but they compare different types.
llvm-svn: 186359
2013-07-15 22:52:48 +00:00
Nadav Rotem
37c18f5cda
SLPVectorizer: change the order in which we search for vectorization candidates. Do stores first and PHIs second.
...
llvm-svn: 186277
2013-07-14 06:15:46 +00:00
Nadav Rotem
ee62470368
SLPVectorizer: Sink and enable CSE for ExtractElements.
...
llvm-svn: 186145
2013-07-12 06:09:24 +00:00
Nadav Rotem
1e6246b38c
SLPVectorize: Replace the code that checks for vectorization candidates in successor blocks with code that scans PHINodes.
...
Before we could vectorize PHINodes scanning successors was a good way of finding candidates. Now we can vectorize the phinodes which is simpler.
llvm-svn: 186139
2013-07-12 00:04:18 +00:00
Nadav Rotem
cd9c4e430f
Remove an argument that we dont use anymore.
...
llvm-svn: 186116
2013-07-11 20:56:13 +00:00
Nadav Rotem
965af9cb85
Fix a warning.
...
llvm-svn: 186064
2013-07-11 05:39:02 +00:00
Nadav Rotem
8e1d89128f
SLPVectorizer: refactor the code that places extracts. Place the code that decides where to put extracts in the build-tree phase. This allows us to take the cost of the extracts into account.
...
llvm-svn: 186058
2013-07-11 04:54:05 +00:00
Nadav Rotem
417c1a3150
Fix PR16571, which is a bug in the code that checks that all of the types in the bundle are uniform.
...
llvm-svn: 185970
2013-07-09 21:38:08 +00:00
Nadav Rotem
07b212b07b
Set the default insert point to the first instruction, and not to end()
...
llvm-svn: 185953
2013-07-09 17:55:36 +00:00
Nadav Rotem
c43699ed15
This patch changes the saved IRBuilder insert point from BasicBlock::iterator to AssertingVH.
...
Commit 185883 fixes a bug in the IRBuilder that should fix the ASan bot. AssertingVH can help in exposing some RAUW problems.
Thanks Ben and Alexey!
llvm-svn: 185886
2013-07-08 23:31:13 +00:00
Nadav Rotem
709b733114
Clear the builder insert point between tree-vectorization phases.
...
llvm-svn: 185777
2013-07-07 14:57:18 +00:00
Nadav Rotem
883fb8ad80
SLPVectorizer: Implement DCE as part of vectorization.
...
This is a complete re-write if the bottom-up vectorization class.
Before this commit we scanned the instruction tree 3 times. First in search of merge points for the trees. Second, for estimating the cost. And finally for vectorization.
There was a lot of code duplication and adding the DCE exposed bugs. The new design is simpler and DCE was a part of the design.
In this implementation we build the tree once. After that we estimate the cost by scanning the different entries in the constructed tree (in any order). The vectorization phase also works on the built tree.
llvm-svn: 185774
2013-07-07 06:57:07 +00:00
Craig Topper
783617eba7
Use SmallVectorImpl::iterator/const_iterator instead of SmallVector to avoid specifying the vector size.
...
llvm-svn: 185606
2013-07-04 01:31:24 +00:00
Nadav Rotem
201beeb585
We preserve the CFG and some of the analysis passes.
...
llvm-svn: 185251
2013-06-29 05:38:15 +00:00
Nadav Rotem
aed4323517
Update docs.
...
llvm-svn: 185250
2013-06-29 05:37:19 +00:00
Nadav Rotem
12c3b510fa
SLP Vectorizer: Add support for trees with external users.
...
To support this we have to insert 'extractelement' instructions to pick the right lane.
We had this functionality before but I removed it when we moved to the multi-block design because it was too complicated.
llvm-svn: 185230
2013-06-28 22:07:09 +00:00
Nadav Rotem
195bbbe54b
No need to use a Set when a vector would do.
...
llvm-svn: 185047
2013-06-27 00:14:13 +00:00
Nadav Rotem
897ca82595
SLP: When searching for vectorization opportunities scan the blocks in post-order because we grow chains upwards.
...
llvm-svn: 185041
2013-06-26 23:44:45 +00:00
Nadav Rotem
962b32446e
SLP: Dont erase instructions during vectorization because it prevents the outerloops from iterating over the instructions.
...
llvm-svn: 185040
2013-06-26 23:43:23 +00:00
Nadav Rotem
860aebf69a
Erase all of the instructions that we RAUWed
...
llvm-svn: 184969
2013-06-26 17:16:09 +00:00
Nadav Rotem
e0a5a586b8
Do not add cse-ed instructions into the visited map because we dont want to consider them as a candidate for replacement of instructions to be visited.
...
llvm-svn: 184966
2013-06-26 16:54:53 +00:00
Nadav Rotem
a8fba65221
SLPVectorizer: support slp-vectorization of PHINodes between basic blocks
...
llvm-svn: 184888
2013-06-25 23:04:09 +00:00
Nadav Rotem
8fcb707c24
Fix a typo in the code that collected the costs recursively.
...
llvm-svn: 184827
2013-06-25 05:30:56 +00:00
Nadav Rotem
eff545235c
Rename the variable to fix a warning. Thanks Andy Gibbs.
...
llvm-svn: 184749
2013-06-24 15:59:47 +00:00
Nadav Rotem
6c2ae14dc5
SLP Vectorizer: Add support for vectorizing parts of the tree.
...
Untill now we detected the vectorizable tree and evaluated the cost of the
entire tree. With this patch we can decide to trim-out branches of the tree
that are not profitable to vectorizer.
Also, increase the max depth from 6 to 12. In the worse possible case where all
of the code is made of diamond-shaped graph this can bring the cost to 2**10,
but diamonds are not very common.
llvm-svn: 184681
2013-06-24 02:52:43 +00:00
Nadav Rotem
5f8e32a66f
SLP Vectorizer: Fix a bug in the code that does CSE on the generated gather sequences.
...
Make sure that we don't replace and RAUW two sequences if one does not dominate the other.
llvm-svn: 184674
2013-06-23 21:57:27 +00:00
Nadav Rotem
8aa1211383
SLP Vectorizer: Erase instructions outside the vectorizeTree method.
...
The RAII builder location guard is saving a reference to instructions, so we can't erase instructions during vectorization.
llvm-svn: 184671
2013-06-23 19:38:56 +00:00
Nadav Rotem
03f4c0b02d
SLP Vectorizer: Implement a simple CSE optimization for the gather sequences.
...
llvm-svn: 184660
2013-06-23 06:15:46 +00:00
Nadav Rotem
3dc5b0a65a
SLP Vectorizer: Implement multi-block slp-vectorization.
...
Rewrote the SLP-vectorization as a whole-function vectorization pass. It is now able to vectorize chains across multiple basic blocks.
It still does not vectorize PHIs, but this should be easy to do now that we scan the entire function.
I removed the support for extracting values from trees.
We are now able to vectorize more programs, but there are some serious regressions in many workloads (such as flops-6 and mandel-2).
llvm-svn: 184647
2013-06-22 21:34:10 +00:00
Nadav Rotem
9191086b4b
Clang-format the SLP vectorizer. No functionality change.
...
llvm-svn: 184446
2013-06-20 17:54:36 +00:00
Nadav Rotem
83cdea61cd
SLPVectorization: Add a basic support for cross-basic block slp vectorization.
...
We collect gather sequences when we vectorize basic blocks. Gather sequences are excellent
hints for vectorization of other basic blocks.
llvm-svn: 184444
2013-06-20 17:41:45 +00:00
Nadav Rotem
b87c78377e
Change the debug type to match the debug type that is used by vecutils.cpp.
...
This change makes it easier to filter debug messages.
llvm-svn: 184440
2013-06-20 16:38:05 +00:00
Nadav Rotem
6796716256
Scan the successor blocks and use the PHI nodes as a hint for possible chain roots.
...
llvm-svn: 184201
2013-06-18 15:58:05 +00:00
Nadav Rotem
f4096f7321
SLPVectorizer: Change the order in which new instructions are added to the function.
...
We are not working on a DAG and I ran into a number of problems when I enabled the vectorizations of 'diamond-trees' (trees that share leafs).
* Imroved the numbering API.
* Changed the placement of new instructions to the last root.
* Fixed a bug with external tree users with non-zero lane.
* Fixed a bug in the placement of in-tree users.
llvm-svn: 182508
2013-05-22 19:47:32 +00:00
Nadav Rotem
4eab353eed
Add a debug print
...
llvm-svn: 181647
2013-05-10 22:56:18 +00:00
Nadav Rotem
a18fab3891
Fix a typo
...
llvm-svn: 180806
2013-04-30 21:04:51 +00:00
Nadav Rotem
069e6d9a7f
Fix PR15800. Do not try to vectorize vectors and structs.
...
llvm-svn: 179960
2013-04-20 22:29:43 +00:00
Benjamin Kramer
e587f9b0bb
SLPVectorizer: Strength reduce SmallVectors to ArrayRefs.
...
Avoids a couple of copies and allows more flexibility in the clients.
llvm-svn: 179935
2013-04-20 09:49:10 +00:00
Nadav Rotem
a244928706
SLPVectorizer: Reduce the compile time by eliminating the search for some of the more expensive patterns. After this change will only check basic arithmetic trees that start at cmpinstr.
...
llvm-svn: 179933
2013-04-20 07:29:34 +00:00
Nadav Rotem
0d6e377312
refactor tryToVectorizePair to a new method that supports vectorization of lists.
...
llvm-svn: 179932
2013-04-20 07:22:58 +00:00
Nadav Rotem
594ff6b41e
Fix an unused variable warning.
...
llvm-svn: 179931
2013-04-20 06:40:28 +00:00
Nadav Rotem
ed63de18d3
SLPVectorizer: Improve the cost model for loop invariant broadcast values.
...
llvm-svn: 179930
2013-04-20 06:13:47 +00:00
Nadav Rotem
571d90215a
Report the number of stores that were found in the debug message.
...
llvm-svn: 179929
2013-04-20 05:23:11 +00:00
Nadav Rotem
40ad92b46d
SLPVectorizer: Make it a function pass and add code for hoisting the vector-gather sequence out of loops.
...
llvm-svn: 179562
2013-04-15 22:00:26 +00:00
Nadav Rotem
7ab2574900
SLPVectorizer: Add support for vectorizing trees that start at compare instructions.
...
llvm-svn: 179504
2013-04-15 04:25:27 +00:00
Nadav Rotem
e380208d4f
SLPVectorizer: Add support for trees that don't start at binary operators, and add the cost of extracting values from the roots of the tree.
...
llvm-svn: 179475
2013-04-14 05:15:53 +00:00
Nadav Rotem
433f05f5de
SLPVectorizer: add initial support for reduction variable vectorization.
...
llvm-svn: 179470
2013-04-14 03:22:20 +00:00
Nadav Rotem
0b2109a86c
Add debug prints.
...
llvm-svn: 179412
2013-04-12 21:11:14 +00:00
Nadav Rotem
6a6b998435
Make the SLP store-merger less paranoid about function calls. We check for function calls when we check if it is safe to sink instructions.
...
llvm-svn: 179207
2013-04-10 19:41:36 +00:00
Nadav Rotem
aa6eefd489
We require DataLayout for analyzing the size of stores.
...
llvm-svn: 179206
2013-04-10 18:57:27 +00:00