Commit Graph

3865 Commits

Author SHA1 Message Date
Justin Lebar
cfc2fa9b55 Revert "[SCEV] Use LLVM_MARK_AS_BITMASK_ENUM in SCEV." -- breaks MSVC builds.
This reverts D48237.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334878 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-16 00:14:10 +00:00
Justin Lebar
5802a4a49f [SCEV] Use LLVM_MARK_AS_BITMASK_ENUM in SCEV.
Summary:
Obviates the need for mask/clear/setFlags helpers.

There are some expressions here which can be simplified, but to keep
this easy to review, I have not simplified them in this patch.

No functional change.

Reviewers: sanjoy

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D48237

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334874 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-15 23:51:57 +00:00
George Burgess IV
1967f637b4 [MSSA] Print more optimization information
In particular, when asked to print a MemoryAccess, we'll now print where
defs are optimized to, and we'll print optimized access types.

This patch also introduces an operator<< to make printing AliasResults
easier.

Patch by Juneyoung Lee!

Differential Revision: https://reviews.llvm.org/D47860


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334760 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-14 19:55:53 +00:00
Simon Pilgrim
21582f2af6 [CostModel] Replace ShuffleKind::SK_Alternate with ShuffleKind::SK_Select (PR33744)
As discussed on PR33744, this patch relaxes ShuffleKind::SK_Alternate which requires shuffle masks to only match an alternating pattern from its 2 sources:

e.g. v4f32: <0,5,2,7> or <4,1,6,3>

This seems far too restrictive as most SIMD hardware which will implement it using a general blend/bit-select instruction, so replaces it with SK_Select, permitting elements from either source as long as they are inline:

e.g. v4f32: <0,5,2,7>, <4,1,6,3>, <0,1,6,7>, <4,1,2,3> etc.

This initial patch just updates the name and cost model shuffle mask analysis, later patch reviews will update SLP to better utilise this - it still limits itself to SK_Alternate style patterns.

Differential Revision: https://reviews.llvm.org/D47985

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334513 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-12 16:12:29 +00:00
Krzysztof Parzyszek
4b6d3f98cd [SCEV] Look through zero-extends in howFarToZero
An expression like
  (zext i2 {(trunc i32 (1 + %B) to i2),+,1}<%while.body> to i32)
will become zero exactly when the nested value becomes zero in its type.
Strip injective operations from the input value in howFarToZero to make
the value simpler.

Differential Revision: https://reviews.llvm.org/D47951


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334318 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-08 20:43:07 +00:00
David Blaikie
1fb574d534 Add missing header
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333957 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-04 21:33:56 +00:00
David Blaikie
8325fb20d4 Move Analysis/Utils/Local.h back to Transforms
Review feedback from r328165. Split out just the one function from the
file that's used by Analysis. (As chandlerc pointed out, the original
change only moved the header and not the implementation anyway - which
was fine for the one function that was used (since it's a
template/inlined in the header) but not in general)

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333954 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-04 21:23:21 +00:00
Benjamin Kramer
6450da6b0c [CalledValuePropagation] Just use a sorted vector instead of a set.
The set properties are never used, so a vector is enough. No
functionality change intended.

While there add some std::moves to SparseSolver.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333582 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-30 19:31:11 +00:00
Daniel Neilson
136d97ecc4 [AliasSet] Teach the alias set how to handle atomic memcpy/memmove/memset
Summary:
The atomic variants of the memcpy/memmove/memset intrinsics can be treated
the same was as the regular forms, with respect to aliasing. Update the
AliasSetTracker to treat the atomic forms the same was as the regular forms.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333551 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-30 14:43:39 +00:00
Chandler Carruth
95ab02a5c0 [LoopInstSimplify] Re-implement the core logic of loop-instsimplify to
be both simpler and substantially more efficient.

Rather than use a hand-rolled iteration technique that isn't quite the
same as RPO, use the pre-built RPO loop body traversal utility.

Once visiting the loop body in RPO, we can assert that we visit defs
before uses reliably. When this is the case, the only need to iterate is
when simplifying a def that is used by a PHI node along a back-edge.
With this patch, the first pass over the loop body is just a complete
simplification of every instruction across the loop body. When we
encounter a use of a simplified instruction that stems from a PHI node
in the loop body that has already been visited (due to some cyclic CFG,
potentially the loop itself, or a nested loop, or unstructured control
flow), we recall that specific PHI node for the second iteration.
Nothing else needs to be preserved from iteration to iteration.

On the second and later iterations, only instructions known to have
simplified inputs are considered, each time starting from a set of PHIs
that had simplified inputs along the backedges.

Dead instructions are collected along the way, but deleted in a batch at
the end of each iteration making the iterations themselves substantially
simpler. This uses a new batch API for recursively deleting dead
instructions.

This alsa changes the routine to visit subloops. Because simplification
is fundamentally transitive, we may need to visit the entire loop body,
including subloops, to handle knock-on simplification.

I've added a basic test file that helps demonstrate that all of these
changes work. It includes both straight-forward loops with
simplifications as well as interesting PHI-structures, CFG-structures,
and a nested loop case.

Differential Revision: https://reviews.llvm.org/D47407

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333461 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-29 20:15:38 +00:00
David Green
00d34a85c6 Revert 333358 as it's failing on some builders.
I'm guessing the tests reply on the ARM backend being built.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333359 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-27 12:54:33 +00:00
David Green
3dde46793c [UnrollAndJam] Add a new Unroll and Jam pass
This is a simple implementation of the unroll-and-jam classical loop
optimisation.

The basic idea is that we take an outer loop of the form:

for i..
  ForeBlocks(i)
  for j..
    SubLoopBlocks(i, j)
  AftBlocks(i)

Instead of doing normal inner or outer unrolling, we unroll as follows:

for i... i+=2
  ForeBlocks(i)
  ForeBlocks(i+1)
  for j..
    SubLoopBlocks(i, j)
    SubLoopBlocks(i+1, j)
  AftBlocks(i)
  AftBlocks(i+1)
Remainder

So we have unrolled the outer loop, then jammed the two inner loops into
one. This can lead to a simpler inner loop if memory accesses can be shared
between the now-jammed loops.

To do this we have to prove that this is all safe, both for the memory
accesses (using dependence analysis) and that ForeBlocks(i+1) can move before
AftBlocks(i) and SubLoopBlocks(i, j).

Differential Revision: https://reviews.llvm.org/D41953



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333358 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-27 12:11:21 +00:00
George Burgess IV
3bf8298d82 Replace AA's uses of uint64_t with LocationSize; NFC.
The uint64_ts that we pass around AA to represent MemoryLocation sizes
are logically an Optional<uint64_t>. In D44748, we want to add an extra
'imprecise' bit to this Optional<uint64_t> to represent whether a given
MemoryLocation size is an upper-bound or an exact size. For more context
on why, please see D44748.

That patch is quite large, but reviewers seem to be OK with the
approach. In D45581 (my first attempt to split 'noise' out of D44748),
reames asked that I land a precursor that is solely replacing uint64_t
with LocationSize, which starts out as `using LocationSize = uint64_t;`.
He also gave me the OK to submit this rename without further review.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333314 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-25 21:16:58 +00:00
Jakub Kuderski
ce531b3cc4 [Dominators] Add PDT constructor from Function
Summary: This patch adds a PDT constructor from Function and lets codes previously using a local class to do this use PostDominatorTree class directly.

Reviewers: davide, kuhar, grosser, dberlin

Reviewed By: kuhar

Author: NutshellySima

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D46709

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333102 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-23 17:29:21 +00:00
Piotr Padlewski
2e3226ade6 Fix aliasing of launder.invariant.group
Summary:
Patch for capture tracking broke
bootstrap of clang with -fstict-vtable-pointers
which resulted in debbugging nightmare. It was fixed
https://reviews.llvm.org/D46900 but as it turned
out, there were other parts like inliner (computing of
noalias metadata) that I found after bootstraping with enabled
assertions.

Reviewers: hfinkel, rsmith, chandlerc, amharc, kuhar

Subscribers: JDevlieghere, eraman, llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D47088

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333070 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-23 09:16:44 +00:00
David Bolvansky
15b63213e0 [InstCombine] Remove calloc transformations
Summary: Previous patch does not care if a value is changed between calloc and strlen. This needs to be removed from InstCombine and maybe moved to DSE later after some rework.

Reviewers: efriedma

Reviewed By: efriedma

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D47218

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333022 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-22 20:27:36 +00:00
David Bolvansky
851bb222e8 [InstCombine] Calloc-ed strings optimizations
Summary:
Example cases:
strlen(calloc(...)) -> 0

Reviewers: efriedma, bkramer

Reviewed By: bkramer

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D47059

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332990 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-22 15:41:23 +00:00
Craig Topper
77438c3c98 [EarlyCSE] Improve EarlyCSE of some absolute value cases.
Change matchSelectPattern to return X and -X for ABS/NABS in a well defined order. Adjust EarlyCSE to account for this. Ensure the SPF result is some kind of min/max and not abs/nabs in one place in InstCombine that made me nervous.

Prevously we returned the two operands of the compare part of the abs pattern. The RHS is always going to be a 0i, 1 or -1 constant. This isn't a very meaningful thing to return for any one. There's also some freedom in the abs pattern as to what happens when the value is equal to 0. This freedom led to early cse failing to match when different constants were used in otherwise equivalent operations. By returning the input and its negation in a defined order we can ensure an exact match. This also makes sure both patterns use the exact same subtract instruction for the negation. I believe CSE should evebntually make this happen and properly merge the nsw/nuw flags. But I'm not familiar with CSE and what order it does things in so it seemed like it might be good to really enforce that they were the same.

Differential Revision: https://reviews.llvm.org/D47037

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332865 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-21 18:42:42 +00:00
Piotr Padlewski
2fd5691529 [MemDep] Fixed handling of invariant.group
Summary:
Memdep had funny bug related to invariant.groups - because it did not
invalidated cache, in some very rare cases it was possible to show memory
dependence of the instruction that was deleted, but because other
instruction took it's place it resulted in call to vtable!
Thanks @amharc for repro!.

Reviewers: dberlin, kuhar, amharc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D45320

Co-authored-by: Krzysztof Pszeniczny <krzysztof.pszeniczny@gmail.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332781 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-18 22:40:34 +00:00
Heejin Ahn
5b752cf527 [WebAssembly] Add Wasm personality and isScopedEHPersonality()
Summary:
- Add wasm personality function
- Re-categorize the existing `isFuncletEHPersonality()` function into
two different functions: `isFuncletEHPersonality()` and
`isScopedEHPersonality(). This becomes necessary as wasm EH uses scoped
EH instructions (catchswitch, catchpad/ret, and cleanuppad/ret) but not
outlined funclets.
- Changed some callsites of `isFuncletEHPersonality()` to
`isScopedEHPersonality()` if they are related to scoped EH IR-level
stuff.

Reviewers: majnemer, dschuff, rnk

Subscribers: jfb, sbc100, jgravelle-google, eraman, JDevlieghere, sunfish, llvm-commits

Differential Revision: https://reviews.llvm.org/D45559

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332667 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-17 20:52:03 +00:00
David Bolvansky
cb037efeac [SimplifyLibcalls] Replace locked IO with unlocked IO
Summary: If file stream arg is not captured and source is fopen, we could replace IO calls by unlocked IO ("_unlocked" function variants) to gain better speed,

Reviewers: efriedma, RKSimon, spatel, sanjoy, hfinkel, majnemer, lebedev.ri, rja

Reviewed By: rja

Subscribers: rja, srhines, efriedma, lebedev.ri, llvm-commits

Differential Revision: https://reviews.llvm.org/D45736

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332452 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-16 11:39:52 +00:00
Michael Zolotukhin
26e0acdad0 [MemorySSA] Don't sort IDF blocks.
Summary:
After r332167 we started to sort the IDF blocks inside IDF calculation, so
there is no need to re-sort them on the user site. The test changes are due to
a slightly different order we're using now (originally we used DFSInNumber and
now the blocks are sorted by a pair (LevelFromRoot, DFSInNumber)).

Reviewers: dberlin, mgrang

Subscribers: Prazek, hiraditya, george.burgess.iv, llvm-commits

Differential Revision: https://reviews.llvm.org/D46899

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332385 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-15 18:40:29 +00:00
Nicola Zaghen
0818e789cb Rename DEBUG macro to LLVM_DEBUG.
The DEBUG() macro is very generic so it might clash with other projects.
The renaming was done as follows:
- git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g'
- git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM
- Manual change to APInt
- Manually chage DOCS as regex doesn't match it.

In the transition period the DEBUG() macro is still present and aliased
to the LLVM_DEBUG() one.

Differential Revision: https://reviews.llvm.org/D43624



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332240 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-14 12:53:11 +00:00
Alina Sbirlea
8e3c0384fd [MemorySSA] getIncomingValueForBlock should return a MemoryAccess.
Summary: getIncomingValueForBlock is just a wrapper API that should return a MemoryAccess, instead of a generic Value.

Reviewers: george.burgess.iv

Subscribers: sanjoy, jlebar, Prazek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46779

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332153 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-11 22:59:37 +00:00
Wei Mi
f2885f7297 [SampleFDO] Don't treat warm callsite with inline instance in the profile as cold
We found current sampleFDO had a performance issue when triaging a regression.
For a callsite with inline instance in the profile, even if hot callsite inliner
cannot inline it, it may still execute enough times and should not be treated as
cold in regular inliner later. However, currently if such callsite is not inlined
by hot callsite inliner, and the BB where the callsite locates doesn't get
samples from other instructions inside of it, the callsite will have no profile
metadata annotated. In regular inliner cost analysis, if the callsite has no
profile annotated and its caller has profile information, it will be treated as
cold.

The fix changes the isCallsiteHot check and chooses to compare
CallsiteTotalSamples with hot cutoff value computed by ProfileSummaryInfo.

Differential Revision: https://reviews.llvm.org/D45377


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332058 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-10 23:02:27 +00:00
Omer Paparo Bivas
55a66360d5 [InstCombine] Moving overflow computation logic from InstCombine to ValueTracking; NFC
Differential Revision: https://reviews.llvm.org/D46704

Change-Id: Ifabcbe431a2169743b3cc310f2a34fd706f13f02

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332026 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-10 19:46:19 +00:00
Shiva Chen
a8a13bc662 [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label.
In order to set breakpoints on labels and list source code around
labels, we need collect debug information for labels, i.e., label
name, the function label belong, line number in the file, and the
address label located. In order to keep these information in LLVM
IR and to allow backend to generate debug information correctly.
We create a new kind of metadata for labels, DILabel. The format
of DILabel is

!DILabel(scope: !1, name: "foo", file: !2, line: 3)

We hope to keep debug information as much as possible even the
code is optimized. So, we create a new kind of intrinsic for label
metadata to avoid the metadata is eliminated with basic block.
The intrinsic will keep existing if we keep it from optimized out.
The format of the intrinsic is

llvm.dbg.label(metadata !1)

It has only one argument, that is the DILabel metadata. The
intrinsic will follow the label immediately. Backend could get the
label metadata through the intrinsic's parameter.

We also create DIBuilder API for labels to be used by Frontend.
Frontend could use createLabel() to allocate DILabel objects, and use
insertLabel() to insert llvm.dbg.label intrinsic in LLVM IR.

Differential Revision: https://reviews.llvm.org/D45024

Patch by Hsiangkai Wang.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@331841 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-09 02:40:45 +00:00
Adrian Prantl
0b24b74655 Remove @brief commands from doxygen comments, too.
This is a follow-up to r331272.

We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.

Patch produced by
  for i in $(git grep -l '\@brief'); do perl -pi -e 's/\@brief //g' $i & done

https://reviews.llvm.org/D46290

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@331275 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-01 16:10:38 +00:00
Adrian Prantl
26b584c691 Remove \brief commands from doxygen comments.
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.

Patch produced by

  for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done

Differential Revision: https://reviews.llvm.org/D46290

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@331272 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-01 15:54:18 +00:00
Nico Weber
0f38c60baf IWYU for llvm-config.h in llvm, additions.
See r331124 for how I made a list of files missing the include.
I then ran this Python script:

    for f in open('filelist.txt'):
        f = f.strip()
        fl = open(f).readlines()

        found = False
        for i in xrange(len(fl)):
            p = '#include "llvm/'
            if not fl[i].startswith(p):
                continue
            if fl[i][len(p):] > 'Config':
                fl.insert(i, '#include "llvm/Config/llvm-config.h"\n')
                found = True
                break
        if not found:
            print 'not found', f
        else:
            open(f, 'w').write(''.join(fl))

and then looked through everything with `svn diff | diffstat -l | xargs -n 1000 gvim -p`
and tried to fix include ordering and whatnot.

No intended behavior change.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@331184 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-30 14:59:11 +00:00
Serguei Katkov
fe97d18817 [SCEV] Introduce bulk umin creation utilities
Add new umin creation method which accepts a list of operands.

SCEV does not represents umin which is required in getExact, so
it transforms umin to umax with not. As a result the transformation of
tree of max to max with several operands does not work.
We just use the new introduced method for creation umin from several operands.

Reviewers: sanjoy, mkazantsev
Reviewed By: sanjoy
Subscribers: javed.absar, llvm-commits
Differential Revision: https://reviews.llvm.org/D46047


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@331015 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-27 03:56:53 +00:00
Matt Morehouse
d0cb241845 Revert "[SimplifyLibcalls] Replace locked IO with unlocked IO"
This reverts r331002 due to sanitizer bot breakage.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@331011 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-27 01:48:09 +00:00
David Bolvansky
c64f4dbfe3 [SimplifyLibcalls] Replace locked IO with unlocked IO
Summary: If file stream arg is not captured and source is fopen, we could replace IO calls by unlocked IO ("_unlocked" function variants) to gain better speed,

Reviewers: efriedma, RKSimon, spatel, sanjoy, hfinkel, majnemer

Subscribers: lebedev.ri, llvm-commits

Differential Revision: https://reviews.llvm.org/D45736

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@331002 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-26 22:31:43 +00:00
Matt Davis
d602d5b93e [Docs] Escape the @ symbol, so that it appears in documentation output. [NFC]
Summary:
The '@' character is a special character in Doxygen.  In a handful of cases we were not escaping this character which resulted in llvm intrinsics not being rendered properly.  Specifically, the @llvm part was removed.  

For example, see https://llvm.org/doxygen/classllvm_1_1AssumptionCache.html.  There are a few references to '.assume' without the @llvm. prefix.  This patch corrects this.

Reviewers: sanjoy

Reviewed By: sanjoy

Subscribers: llvm-commits, sanjoy

Differential Revision: https://reviews.llvm.org/D45981

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330998 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-26 21:55:45 +00:00
Matthew Simpson
daa39fa144 [TTI, AArch64] Add transpose shuffle kind
This patch adds a new shuffle kind useful for transposing a 2xn matrix. These
transpose shuffle masks read corresponding even- or odd-numbered vector
elements from two n-dimensional source vectors and write each result into
consecutive elements of an n-dimensional destination vector. The transpose
shuffle kind is meant to model the TRN1 and TRN2 AArch64 instructions. As such,
this patch also considers transpose shuffles in the AArch64 implementation of
getShuffleCost.

Differential Revision: https://reviews.llvm.org/D45982

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330941 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-26 13:48:33 +00:00
Daniel Neilson
608fcd9862 [CaptureTracking] Fixup const correctness of DomTree arg (NFC)
Summary:
The PointerMayBeCapturedBefore function's DomTree arg should be
const instead of non-const. There are no non-const uses of it
in the function.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330769 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-24 21:12:45 +00:00
Florian Hahn
0c47f856a1 [LoopInfo] Verify BBMap tracks innermost loops for BBs.
By checking that none of the child loops contain a BB we make sure BBMap
contains the innermost loop defining BB. This invariant was violated in
LoopInterchange and got caught by this assertion.

Reviewers: chandlerc, mzolotukhin, sanjoy, mehdi_amini, efriedma

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D45971


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330698 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-24 09:10:05 +00:00
Daniel Neilson
6c92ba3e0b [DSE] Teach the pass that atomic memory intrinsics are stores.
Summary:
This change teaches DSE that the atomic memory intrinsics are stores
that can be eliminated, and can allow other stores to be eliminated.
This change specifically does not teach DSE that these intrinsics
can be partially eliminated (i.e. length reduced, and dest/src changed);
that will be handled in another change.

Reviewers: mkazantsev, skatkov, apilipenko, efriedma, rsmith

Reviewed By: efriedma

Subscribers: dmgreen, llvm-commits

Differential Revision: https://reviews.llvm.org/D45535

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330629 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-23 19:06:49 +00:00
Max Kazantsev
d356fd6732 [LoopSimplify] Fix incorrect SCEV invalidation
In the function `simplifyOneLoop` we optimistically assume that changes in the
inner loop only affect this very loop and have no impact on its parents. In fact,
after rL329047 has been merged, we can now calculate exit counts for outer
loops which may depend on inner loops. Thus, we need to invalidate all parents
when we do something to a loop.

There is an evidence of incorrect behavior of `simplifyOneLoop`: when we insert
`SE->verify()` check in the end of this funciton, it fails on a bunch of existing
test, in particular:

    LLVM :: Transforms/LoopUnroll/peel-loop-not-forced.ll
    LLVM :: Transforms/LoopUnroll/peel-loop-pgo.ll
    LLVM :: Transforms/LoopUnroll/peel-loop.ll
    LLVM :: Transforms/LoopUnroll/peel-loop2.ll

Note that previously we have fixed issues of this variety, see rL328483.
This patch makes this function invalidate the outermost loop properly.

Differential Revision: https://reviews.llvm.org/D45937
Reviewed By: chandlerc


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330576 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-23 10:32:37 +00:00
Krzysztof Parzyszek
c6388fd9c7 [LV] Introduce TTI::getMinimumVF
The function getMinimumVF(ElemWidth) will return the minimum VF for
a vector with elements of size ElemWidth bits. This value will only
apply to targets for which TTI::shouldMaximizeVectorBandwidth returns
true. The value of 0 indicates that there is no minimum VF.

Differential Revision: https://reviews.llvm.org/D45271


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330062 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-13 20:16:32 +00:00
Hiroshi Inoue
ef1bc2d3ca [NFC] fix trivial typos in documents and comments
"is is" -> "is", "if if" -> "if", "or or" -> "or"



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@329878 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-12 05:53:20 +00:00
Zhaoshi Zheng
b26999a62d [MemorySSAUpdater] Mark Phi users of a node being moved as non-optimize
Fix PR36484, as suggested:

<quote>
during moves, mark the direct users of the erased things that were phis as "not to be optimized"
<quote>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@329621 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-09 20:55:37 +00:00
Eric Fiselier
75627b1a25 [Analysis] Support aligned new/delete functions.
Summary:
Clang's __builtin_operator_new/delete was recently taught about the aligned allocation overloads (r328134). This patch makes LLVM aware of them as well.
This allows the compiler to perform certain optimizations including eliding new/delete calls.

Reviewers: rsmith, majnemer, dblaikie, vsk, bkramer

Reviewed By: bkramer

Subscribers: ckennelly, llvm-commits

Differential Revision: https://reviews.llvm.org/D44769

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@329218 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-04 19:01:51 +00:00
Eric Fiselier
b9f31323a6 Revert "[Analysis] Support aligned new/delete functions."
This reverts commit bee3bbd9bd.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@329217 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-04 18:23:00 +00:00
Eric Fiselier
bee3bbd9bd [Analysis] Support aligned new/delete functions.
Summary:
Clang's __builtin_operator_new/delete was recently taught about the aligned allocation overloads (r328134). This patch makes LLVM aware of them as well.
This allows the compiler to perform certain optimizations including eliding new/delete calls.

Reviewers: rsmith, majnemer, dblaikie, vsk, bkramer

Reviewed By: bkramer

Subscribers: ckennelly, llvm-commits

Differential Revision: https://reviews.llvm.org/D44769

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@329215 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-04 18:12:01 +00:00
Nicolai Haehnle
036d645a4c StructurizeCFG: Test for branch divergence correctly
Fixes cases like the new test @nonuniform. In that test, %cc itself
is a uniform value; however, when reading it after the end of the loop in
basic block %if, its value is effectively non-uniform, so the branch is
non-uniform.

This problem was encountered in
https://bugs.freedesktop.org/show_bug.cgi?id=103743; however, this change
in itself is not sufficient to fix that bug, as there is another issue
in the AMDGPU backend.

As discovered after committing an earlier version of this change, this
exposes a subtle interaction between this pass and DivergenceAnalysis:
since we remove and re-create branch instructions, we can no longer rely
on DivergenceAnalysis for branches in subregions that were already
processed by the pass.

Explicitly remove branch instructions from DivergenceAnalysis to
avoid dangling pointers as a matter of defensive programming, and
change how we detect non-uniform subregions.

Change-Id: I32bbffece4a32f686fab54964dae1a5dd72949d4

Differential Revision: https://reviews.llvm.org/D43743

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@329165 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-04 10:58:15 +00:00
Max Kazantsev
30f7325162 [SCEV] Prove implications for SCEVUnknown Phis
This patch teaches SCEV how to prove implications for SCEVUnknown nodes that are Phis.
If we need to prove `Pred` for `LHS, RHS`, and `LHS` is a Phi with possible incoming values
`L1, L2, ..., LN`, then if we prove `Pred` for `(L1, RHS), (L2, RHS), ..., (LN, RHS)` then we can also
prove it for `(LHS, RHS)`. If both `LHS` and `RHS` are Phis from the same block, it is sufficient
to prove the predicate for values that come from the same predecessor block.

The typical case that it handles is that we sometimes need to prove that `Phi(Len, Len - 1) >= 0`
given that `Len > 0`. The new logic was added to `isImpliedViaOperations` and only uses it and
non-recursive reasoning to prove the facts we need, so it should not hurt compile time a lot.

Differential Revision: https://reviews.llvm.org/D44001
Reviewed By: anna


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@329150 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-04 05:46:47 +00:00
Alexey Bataev
91811bc488 [SLP] Fix PR36481: vectorize reassociated instructions.
Summary:
If the load/extractelement/extractvalue instructions are not originally
consecutive, the SLP vectorizer is unable to vectorize them. Patch
allows reordering of such instructions.

Patch does not support reordering of the repeated instruction, this must
be handled in the separate patch.

Reviewers: RKSimon, spatel, hfinkel, mkuper, Ayal, ashahid

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D43776

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@329085 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-03 17:14:47 +00:00
Benjamin Kramer
4832f865cf Revert "[SLP] Fix PR36481: vectorize reassociated instructions."
This reverts commit r328980 and r329046. Makes the vectorizer crash.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@329071 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-03 14:40:33 +00:00
Alexey Bataev
6616787959 [SLP] Fix PR36481: vectorize reassociated instructions.
Summary:
If the load/extractelement/extractvalue instructions are not originally
consecutive, the SLP vectorizer is unable to vectorize them. Patch
allows reordering of such instructions.

Reviewers: RKSimon, spatel, hfinkel, mkuper, Ayal, ashahid

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D43776

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@328980 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-02 14:51:37 +00:00