This bug was introduced in r269728 and is the likely cause of many stage 2 ubsan bot failures.
I'll add a test in a follow-up commit assuming this fixes things properly.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269797 91177308-0d34-0410-b5e6-96231b3b80d8
Guards are expected to basically never fail. Reflect this in the branch
probabilities in their lowered form.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269791 91177308-0d34-0410-b5e6-96231b3b80d8
This is assertion is no longer necessary since we never record
constants in the live set anyway. (They are never recorded in
the initial live set, and constant bases are removed near line 2119)
Differential Revision: http://reviews.llvm.org/D20293
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269764 91177308-0d34-0410-b5e6-96231b3b80d8
Fix a bug introduced with rL269426 :
[InstCombine] canonicalize* LE/GE vector integer comparisons to LT/GT (PR26701, PR26819)
We were assuming that a ConstantDataVector / ConstantVector / ConstantAggregateZero operand of
an ICMP was composed of ConstantInt elements, but it might have ConstantExpr or UndefValue
elements. Handle those appropriately.
Also, refactor this function to join the scalar and vector paths and eliminate the switches.
Differential Revision: http://reviews.llvm.org/D20289
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269728 91177308-0d34-0410-b5e6-96231b3b80d8
Transition InstrProf and Coverage over to the stricter Error/Expected
interface.
Changes since the initial commit:
- Address undefined-var-template warning.
- Fix error message printing in llvm-profdata.
- Check errors in loadTestingFormat() + annotateAllFunctions().
- Defer error handling in InstrProfIterator to InstrProfReader.
Differential Revision: http://reviews.llvm.org/D19901
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269694 91177308-0d34-0410-b5e6-96231b3b80d8
The selection of the vectorization factor currently doesn't consider
interleaved accesses. The vectorization factor is based on the maximum safe
dependence distance computed by LAA. However, for loops with interleaved
groups, we should instead base the vectorization factor on the maximum safe
dependence distance divided by the maximum interleave factor of all the
interleaved groups. Interleaved accesses not in a group will be scalarized.
Differential Revision: http://reviews.llvm.org/D20241
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269659 91177308-0d34-0410-b5e6-96231b3b80d8
TargetLibraryInfoWrapperPass is a dependency of
SCCP but it's not listed as such. Chandler pointed
out this is an easy mistake to make which only
surfaces in weird crashes with some flag combinations.
This code will go away anyway at some point in the
future, but as long as it's (still) exercised, try
to make it correct.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269589 91177308-0d34-0410-b5e6-96231b3b80d8
This reverts commit r269491. It triggers warnings with Clang, breaking
builds for -Werror users including several build bots.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269547 91177308-0d34-0410-b5e6-96231b3b80d8
Transition InstrProf and Coverage over to the stricter Error/Expected
interface.
Changes since the initial commit:
- Fix error message printing in llvm-profdata.
- Check errors in loadTestingFormat() + annotateAllFunctions().
- Defer error handling in InstrProfIterator to InstrProfReader.
Differential Revision: http://reviews.llvm.org/D19901
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269491 91177308-0d34-0410-b5e6-96231b3b80d8
Currently there is no reasonable way to control the warnings in the 'use' phase
of the IRPGO pass. This is problematic because the output can be somewhat
spammy. This patch adds some flags which allow us to optionally disable these
warnings. The current upstream behavior will remain the default.
Patch by Jake VanAdrighem (jvanadrighem@gmail.com)
Differential Revision: http://reviews.llvm.org/D20195
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269437 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: This change fix the bug in isProfitableToUseMemset() where MaxIntSize shoule be in byte, not bit.
Reviewers: arsenm, joker.eph, mcrosier
Subscribers: mcrosier, llvm-commits
Differential Revision: http://reviews.llvm.org/D20176
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269433 91177308-0d34-0410-b5e6-96231b3b80d8
*We don't currently handle the edge case constants (min/max values), so it's not a complete
canonicalization.
To fully solve the motivating bugs, we need to enhance this to recognize a zero vector
too because that's a ConstantAggregateZero which is a ConstantData, not a ConstantVector
or a ConstantDataVector.
Differential Revision: http://reviews.llvm.org/D17859
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269426 91177308-0d34-0410-b5e6-96231b3b80d8
This reverts commit r269388.
It caused some bots to fail, I'm reverting it until I investigate the
issue.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269395 91177308-0d34-0410-b5e6-96231b3b80d8
This should fix some compile-time regressions after r267672. Thanks to
Chris Matthews for bisecting it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269392 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
...loop after the last iteration.
This is really hard to do correctly. The core problem is that we need to
model liveness through the induction PHIs from iteration to iteration in
order to get the correct results, and we need to correctly de-duplicate
the common subgraphs of instructions feeding some subset of the
induction PHIs. All of this can be driven either from a side effect at
some iteration or from the loop values used after the loop finishes.
This patch implements this by storing the forward-propagating analysis
of each instruction in a cache to recall whether it was free and whether
it has become live and thus counted toward the total unroll cost. Then,
at each sink for a value in the loop, we recursively walk back through
every value that feeds the sink, including looping back through the
iterations as needed, until we have marked the entire input graph as
live. Because we cache this, we never visit instructions more than twice
-- once when we analyze them and put them into the cache, and once when
we count their cost towards the unrolled loop. Also, because the cache
is only two bits and because we are dealing with relatively small
iteration counts, we can store all of this very densely in memory to
avoid this from becoming an excessively slow analysis.
The code here is still pretty gross. I would appreciate suggestions
about better ways to factor or split this up, I've stared too long at
the algorithmic side to really have a good sense of what the design
should probably look at.
Also, it might seem like we should do all of this bottom-up, but I think
that is a red herring. Specifically, the simplification power is *much*
greater working top-down. We can forward propagate very effectively,
even across strange and interesting recurrances around the backedge.
Because we use data to propagate, this doesn't cause a state space
explosion. Doing this level of constant folding, etc, would be very
expensive to do bottom-up because it wouldn't be until the last moment
that you could collapse everything. The current solution is essentially
a top-down simplification with a bottom-up cost accounting which seems
to get the best of both worlds. It makes the simplification incremental
and powerful while leaving everything dead until we *know* it is needed.
Finally, a core property of this approach is its *monotonicity*. At all
times, the current UnrolledCost is a conservatively low estimate. This
ensures that we will never early-exit from the analysis due to exceeding
a threshold when if we had continued, the cost would have gone back
below the threshold. These kinds of bugs can cause incredibly hard to
track down random changes to behavior.
We could use a techinque similar (but much simpler) within the inliner
as well to avoid considering speculated code in the inline cost.
Reviewers: chandlerc
Subscribers: sanjoy, mzolotukhin, llvm-commits
Differential Revision: http://reviews.llvm.org/D11758
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269388 91177308-0d34-0410-b5e6-96231b3b80d8
Ported DA to the new PM by splitting the former DependenceAnalysis Pass
into a DependenceInfo result type and DependenceAnalysisWrapperPass type
and adding a new PM-style DependenceAnalysis analysis pass returning the
DependenceInfo.
Patch by Philip Pfaffe, most of the review by Justin.
Differential Revision: http://reviews.llvm.org/D18834
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269370 91177308-0d34-0410-b5e6-96231b3b80d8
Split FCMP//ICMP/SEL from the basic arithmetic cost functions. They were not sharing any notable code path (just the return) and were repeatedly testing the opcode.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269348 91177308-0d34-0410-b5e6-96231b3b80d8
LoopVectorBody was changed from a single pointer to a SmallVector when
store predication was introduced in r200270. Since r247139, store predication
no longer splits the vector loop body in-place, so we can go back to having
a single LoopVectorBody block.
This reverts the no-longer-needed changes from r200270.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269321 91177308-0d34-0410-b5e6-96231b3b80d8
Shifts beyond the bitwidth are undef but SCCP resolved them to zero.
Instead, DTRT and resolve them to undef.
This reimplements the transform which caused PR27712.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269269 91177308-0d34-0410-b5e6-96231b3b80d8
This new verifier rule lets us unambigously pick a calling convention
when creating a new declaration for
`@llvm.experimental.deoptimize.<ty>`. It is also congruent with our
lowering strategy -- since all calls to `@llvm.experimental.deoptimize`
are lowered to calls to `__llvm_deoptimize`, it is reasonable to enforce
a unique calling convention.
Some of the tests that were breaking this verifier rule have had to be
split up into different .ll files.
The inliner was violating this rule as well, and has been fixed to avoid
producing invalid IR.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269261 91177308-0d34-0410-b5e6-96231b3b80d8
This should just be a compile-time change. Correct the check for whether
we have already analyzed the callee when making summary based decisions.
There is no need to reprocess one at the same threshold as when it was
last processed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269251 91177308-0d34-0410-b5e6-96231b3b80d8
Use WeakAny linkage instead of LinkOnceAny, as the symbol can be removed with
LinkOnceAny in O2 (not referenced).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269146 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: In sample profile, some branches may have profile missing due to profile inaccuracy. We want existing branch probability still valid after propagation.
Reviewers: hfinkel, davidxl, spatel
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D19948
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269137 91177308-0d34-0410-b5e6-96231b3b80d8
