There are cases when we have to merge TBAA access tags with the
same base access type, but different final access types. For
example, accesses to different members of the same structure may
be vectorized into a single load or store instruction. Since we
currently assume that the tags to merge always share the same
final access type, we incorrectly return a tag that describes an
access to one of the original final access types as the generic
tag. This patch fixes that by producing generic tags for the
common type and not the final access types of the original tags.
Resolves:
PR35225: Wrong tbaa metadata after load store vectorizer due to
recent change
https://bugs.llvm.org/show_bug.cgi?id=35225
Differential Revision: https://reviews.llvm.org/D39732
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317682 91177308-0d34-0410-b5e6-96231b3b80d8
My fix is conservative and will make us return may-alias instead.
The test case is:
check(gep(x, 0), n, gep(x, n), -1) with n == sizeof(x)
Here, the first value accesses the whole object, but the second access
doesn't access anything. The semantics of -1 is read until the end of the
object, which in this case means read nothing.
No test case, since isn't trivial to exploit this one, but I've proved it correct.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317680 91177308-0d34-0410-b5e6-96231b3b80d8
As discussed in D39204, this is effectively a revert of rL265521 which required nnan
to vectorize sqrt libcalls based on the old LangRef definition of llvm.sqrt. Now that
the definition has been updated so the libcall and intrinsic have the same semantics
apart from potentially setting errno, we can remove the nnan requirement.
We have the right check to know that errno is not set:
if (!ICS.onlyReadsMemory())
...ahead of the switch.
This will solve https://bugs.llvm.org/show_bug.cgi?id=27435 assuming that's being
built for a target with -fno-math-errno.
Differential Revision: https://reviews.llvm.org/D39642
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317519 91177308-0d34-0410-b5e6-96231b3b80d8
single-iteration loop
This fixes PR34681. Avoid adding the "Stride == 1" predicate when we know that
Stride >= Trip-Count. Such a predicate will effectively optimize a single
or zero iteration loop, as Trip-Count <= Stride == 1.
Differential Revision: https://reviews.llvm.org/D38785
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317438 91177308-0d34-0410-b5e6-96231b3b80d8
Now that we have a way to mark GlobalValues as local we can use the symbol
resolutions that the linker plugin provides as part of lto/thinlto link
step to refine the compilers view on what symbols will end up being local.
Originally commited as r317374, but reverted in r317395 to update some missed
tests.
Differential Revision: https://reviews.llvm.org/D35702
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317408 91177308-0d34-0410-b5e6-96231b3b80d8
Now that we have a way to mark GlobalValues as local we can use the symbol
resolutions that the linker plugin provides as part of lto/thinlto link
step to refine the compilers view on what symbols will end up being local.
Differential Revision: https://reviews.llvm.org/D35702
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317374 91177308-0d34-0410-b5e6-96231b3b80d8
This patch combines the code that matches and merges TBAA access
tags. The aim is to simplify future changes and making sure that
these operations produce consistent results.
Differential Revision: https://reviews.llvm.org/D39463
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317311 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Currently the block frequency analysis is an approximation for irreducible
loops.
The new irreducible loop metadata is used to annotate the irreducible loop
headers with their header weights based on the PGO profile (currently this is
approximated to be evenly weighted) and to help improve the accuracy of the
block frequency analysis for irreducible loops.
This patch is a basic support for this.
Reviewers: davidxl
Reviewed By: davidxl
Subscribers: mehdi_amini, llvm-commits, eraman
Differential Revision: https://reviews.llvm.org/D39028
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317278 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Compute the strongly connected components of the CFG and fall back to
use these for blocks that are in loops that are not detected by
LoopInfo when computing loop back-edge and exit branch probabilities.
Reviewers: dexonsmith, davidxl
Subscribers: mcrosier, llvm-commits
Differential Revision: https://reviews.llvm.org/D39385
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317094 91177308-0d34-0410-b5e6-96231b3b80d8
Issue found by llvm-isel-fuzzer on OSS fuzz, https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3725
If anyone actually cares about > 64 bit arithmetic, there's a lot more to do in this area. There's a bunch of obviously wrong code in the same function. I don't have the time to fix all of them and am just using this to understand what the workflow for fixing fuzzer cases might look like.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316967 91177308-0d34-0410-b5e6-96231b3b80d8
- Targets that want to support memcmp expansions now return the list of
supported load sizes.
- Expansion codegen does not assume that all power-of-two load sizes
smaller than the max load size are valid. For examples, this is not the
case for x86(32bit)+sse2.
Fixes PR34887.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316905 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
ValueTracking was recognizing not all variations of clamp. Swapping of
true value and false value of select was added to fix this problem. The
first patch was reverted because it caused miscompile in NVPTX target.
Added corresponding test cases.
Reviewers: spatel, majnemer, efriedma, reames
Subscribers: llvm-commits, jholewinski
Differential Revision: https://reviews.llvm.org/D39240
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316795 91177308-0d34-0410-b5e6-96231b3b80d8
Max backedge taken count is always expected to be a constant; and this is
usually true by construction -- it is a SCEV expression with constant inputs.
However, if the max backedge expression ends up being computed to be a udiv with
a constant zero denominator[0], SCEV does not fold the result to a constant
since there is no constant it can fold it to (SCEV has no representation for
"infinity" or "undef").
However, in computeMaxBECountForLT we already know the denominator is positive,
and thus at least 1; and we can use this fact to avoid dividing by zero.
[0]: We can end up with a constant zero denominator if the signed range of the
stride is more precise than the unsigned range.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316615 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Memory dependence analysis no longer counts DbgInfoIntrinsics towards the
limit where to abort the analysis. Before, a bunch of calls to dbg.value
could affect the generated code, meaning that with -g we could generate
different code than without.
Reviewers: chandlerc, Prazek, davide, efriedma
Reviewed By: efriedma
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D39181
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316551 91177308-0d34-0410-b5e6-96231b3b80d8
If particular target supports volatile memory access operations, we can
avoid AS casting to generic AS. Currently it's only enabled in NVPTX for
loads and stores that access global & shared AS.
Differential Revision: https://reviews.llvm.org/D39026
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316495 91177308-0d34-0410-b5e6-96231b3b80d8
Neither of these cases really require a temporary APInt outside the loop. For the ConstantDataSequential case the APInt will never be larger than 64-bits so its fine to just call getElementAsAPInt. For ConstantVector we can get the APInt by reference and only make a copy where the inversion is needed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316265 91177308-0d34-0410-b5e6-96231b3b80d8
(recommit #2 after checking for timeout issue).
The original patch was an improvement to IR ValueTracking on
non-negative integers. It has been checked in to trunk (D18777,
r284022). But was disabled by default due to performance regressions.
Perf impact has improved. The patch would be enabled by default.
Reviewers: reames, hfinkel
Differential Revision: https://reviews.llvm.org/D34101
Patch by: Olga Chupina <olga.chupina@intel.com>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316208 91177308-0d34-0410-b5e6-96231b3b80d8
Added check that type of CmpConst and source type of trunc are equal
for correct matching of the case when we can set widened C constant
equal to CmpConstant.
%cond = cmp iN %x, CmpConst
%tr = trunc iN %x to iK
%narrowsel = select i1 %cond, iK %t, iK C
Patch by: Gainullin, Artur <artur.gainullin@intel.com>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316082 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
When we have the following case:
%cond = cmp iN %x, CmpConst
%tr = trunc iN %x to iK
%narrowsel = select i1 %cond, iK %t, iK C
We could possibly match only min/max pattern after looking through cast.
So it is more profitable if widened C constant will be equal CmpConst.
That is why just set widened C constant equal to CmpConst, because there
is a further check in this function that trunc CmpConst == C.
Also description for lookTroughCast function was added.
Reviewers: spatel
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D38536
Patch by: Artur Gainullin <artur.gainullin@intel.com>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316070 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
If a compare instruction is same or inverse of the compare in the
branch of the loop latch, then return a constant evolution node.
Currently scope of evaluation is limited to SCEV computation for
PHI nodes.
This shall facilitate computations of loop exit counts in cases
where compare appears in the evolution chain of induction variables.
Will fix PR 34538
Reviewers: sanjoy, hfinkel, junryoungju
Reviewed By: junryoungju
Subscribers: javed.absar, llvm-commits
Differential Revision: https://reviews.llvm.org/D38494
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316054 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
ValueTracking was recognizing not all variations of clamp. Swapping of
true value and false value of select was added to fix this problem. This
change breaks the canonical form of cmp inside the matchMinMax function,
that is why additional checks for compare predicates is needed. Added
corresponding test cases.
Reviewers: spatel
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D38531
Patch by: Artur Gainullin <artur.gainullin@intel.com>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315992 91177308-0d34-0410-b5e6-96231b3b80d8
This reverts commit r315713. It causes PR34968.
I think I know what the problem is, but I don't think I'll have time to fix it
this week.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315962 91177308-0d34-0410-b5e6-96231b3b80d8
This patch moves some common utility functions out of IPSCCP and makes them
available globally. The functions determine if interprocedural data-flow
analyses can propagate information through function returns, arguments, and
global variables.
Differential Revision: https://reviews.llvm.org/D37638
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315719 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This change uses the loop use list added in the previous change to remember the
loops that appear in the trip count expressions of other loops; and uses it in
forgetLoop. This lets us not scan every loop in the function on a forgetLoop
call.
With this change we no longer invalidate clear out backedge taken counts on
forgetValue. I think this is fine -- the contract is that SCEV users must call
forgetLoop(L) if their change to the IR could have changed the trip count of L;
solely calling forgetValue on a value feeding into the backedge condition of L
is not enough. Moreover, I don't think we can strengthen forgetValue to be
sufficient for invalidating trip counts without significantly re-architecting
SCEV. For instance, if we have the loop:
I = *Ptr;
E = I + 10;
do {
// ...
} while (++I != E);
then the backedge taken count of the loop is 9, and it has no reference to
either I or E, i.e. there is no way in SCEV today to re-discover the dependency
of the loop's trip count on E or I. So a SCEV client cannot change E to (say)
"I + 20", call forgetValue(E) and expect the loop's trip count to be updated.
Reviewers: atrick, sunfish, mkazantsev
Subscribers: mcrosier, llvm-commits
Differential Revision: https://reviews.llvm.org/D38435
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315713 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch teaches SCEV to calculate the maxBECount when the end bound
of the loop can vary. Note that we cannot calculate the exactBECount.
This will only be done when both conditions are satisfied:
1. the loop termination condition is strictly LT.
2. the IV is proven to not overflow.
This provides more information to users of SCEV and can be used to
improve identification of finite loops.
Reviewers: sanjoy, mkazantsev, silviu.baranga, atrick
Reviewed by: mkazantsev
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D38825
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315683 91177308-0d34-0410-b5e6-96231b3b80d8
Significantly reduces performancei (~30%) of gipfeli
(https://github.com/google/gipfeli)
I have not yet managed to reproduce this regression with the open-source
version of the benchmark on github, but will work with others to get a
reproducer to you later today.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315680 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Currently we do not correctly invalidate memoized results for add recurrences
that were created directly (i.e. they were not created from a `Value`). This
change fixes this by keeping loop use lists and using the loop use lists to
determine which SCEV expressions to invalidate.
Here are some statistics on the number of uses of in the use lists of all loops
on a clang bootstrap (config: release, no asserts):
Count: 731310
Min: 1
Mean: 8.555150
50th %time: 4
95th %tile: 25
99th %tile: 53
Max: 433
Reviewers: atrick, sunfish, mkazantsev
Subscribers: mcrosier, llvm-commits
Differential Revision: https://reviews.llvm.org/D38434
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315672 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Add LLVM_FORCE_ENABLE_DUMP cmake option, and use it along with
LLVM_ENABLE_ASSERTIONS to set LLVM_ENABLE_DUMP.
Remove NDEBUG and only use LLVM_ENABLE_DUMP to enable dump methods.
Move definition of LLVM_ENABLE_DUMP from config.h to llvm-config.h so
it'll be picked up by public headers.
Differential Revision: https://reviews.llvm.org/D38406
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315590 91177308-0d34-0410-b5e6-96231b3b80d8
This patch fixes the bug introduced in https://reviews.llvm.org/D35907; the bug is reported by http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20171002/491452.html.
Before D35907, when GetUnderlyingObjects fails to find an identifiable object, allMMOsOkay lambda in getUnderlyingObjectsForInstr returns false and Objects vector is cleared. This behavior is unintentionally changed by D35907.
This patch makes the behavior for such case same as the previous behavior.
Since D35907 introduced a wrapper function getUnderlyingObjectsForCodeGen around GetUnderlyingObjects, getUnderlyingObjectsForCodeGen is modified to return a boolean value to ask the caller to clear the Objects vector.
Differential Revision: https://reviews.llvm.org/D38735
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315565 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch fixes an error in the patch to ScalarEvolution::createAddRecFromPHIWithCastsImpl
made in D37265. In that patch we handle the cases where the either the start or accum values can be
zero after truncation. But, we assume that the start value must be a constant if the accum is
zero. This is clearly an erroneous assumption. This change removes that assumption.
Reviewers: sanjoy, dorit, mkazantsev
Reviewed By: sanjoy
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D38814
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315491 91177308-0d34-0410-b5e6-96231b3b80d8
Recommitting r314517 with the fix for handling ConstantExpr.
Original commit message:
Currently, getGEPCost() returns TCC_FREE whenever a GEP is a legal addressing
mode in the target. However, since it doesn't check its actual users, it will
return FREE even in cases where the GEP cannot be folded away as a part of
actual addressing mode. For example, if an user of the GEP is a call
instruction taking the GEP as a parameter, then the GEP may not be folded in
isel.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@314923 91177308-0d34-0410-b5e6-96231b3b80d8
