We were bailing to two places if our runtime checks failed. If the initial overflow check failed, we'd go to ScalarPH. If any other check failed, we'd go to MiddleBlock. This caused us to have to have an extra PHI per induction and reduction as the vector loop's exit block was not dominated by its latch.
There's no need to have this behavior - if we just always go to ScalarPH we can get rid of a bunch of complexity.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246637 91177308-0d34-0410-b5e6-96231b3b80d8
This reduces the complexity of createEmptyBlock() and will open the door to further refactoring.
The test change is simply because we're now constant folding a trivial test.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246634 91177308-0d34-0410-b5e6-96231b3b80d8
... and do a tad of tidyup while we're at it. Because StartIdx must now be zero, there's no difference between Count and EndIdx.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246633 91177308-0d34-0410-b5e6-96231b3b80d8
It makes things easier to understand if this is in a helper method. This is part of my ongoing spaghetti-removal operation on createEmptyLoop.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246632 91177308-0d34-0410-b5e6-96231b3b80d8
There's no need to widen canonical induction variables. It's just as efficient to create a *new*, wide, induction variable.
Consider, if we widen an indvar, then we'll have to truncate it before its uses anyway (1 trunc). If we create a new indvar instead, we'll have to truncate that instead (1 trunc) [besides which IndVars should go and clean up our mess after us anyway on principle].
This lets us remove a ton of special-casing code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246631 91177308-0d34-0410-b5e6-96231b3b80d8
Vectorized loops only ever have one induction variable. All induction PHIs from the scalar loop are rewritten to be in terms of this single indvar.
We were trying very hard to pick an indvar that already existed, even if that indvar wasn't canonical (didn't start at zero). But trying so hard is really fruitless - creating a new, canonical, indvar only results in one extra add in the worst case and that add is trivially easy to push through the PHI out of the loop by instcombine.
If we try and be less clever here and instead let instcombine clean up our mess (as we do in many other places in LV), we can remove unneeded complexity.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246630 91177308-0d34-0410-b5e6-96231b3b80d8
Vector 'getelementptr' with scalar base is an opportunity for gather/scatter intrinsic to generate a better sequence.
While looking for uniform base, we want to use the scalar base pointer of GEP, if exists.
Differential Revision: http://reviews.llvm.org/D11121
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246622 91177308-0d34-0410-b5e6-96231b3b80d8
The MS incremental linker seems to inspect the timestamp written into
the object file to determine whether or not it's contents need to be
considered. Failing to set the timestamp to a date newer than the
executable will result in the object file not participating in
subsequent links. To ameliorate this, write the current time into the
object file's TimeDateStamp field.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246607 91177308-0d34-0410-b5e6-96231b3b80d8
We can just ask the ObjectWriter for it's stream instead of caching
around our own reference to it. No functionality change is intended.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246604 91177308-0d34-0410-b5e6-96231b3b80d8
The code introduced in r244314 assumed that EXTRACT_VECTOR_ELT only
takes constant indices, but it does accept variables.
Bail out for those: we can't use them, as the shuffles we want to
reconstruct do require constant masks.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246594 91177308-0d34-0410-b5e6-96231b3b80d8
COFF sections are accompanied with an auxiliary symbol which includes a
checksum. This checksum used to be filled with just zero but this seems
to upset LINK.exe when it is processing a /INCREMENTAL link job.
Instead, fill the CheckSum field with the JamCRC of the section
contents. This matches MSVC's behavior.
This fixes PR19666.
N.B. A rather simple implementation of JamCRC is given. It implements
a byte-wise calculation using the method given by Sarwate. There are
implementations with higher throughput like slice-by-eight and making
use of PCLMULQDQ. We can switch to one of those techniques if it turns
out to be a significant use of time.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246590 91177308-0d34-0410-b5e6-96231b3b80d8
This was last used by the pre-MC object emitter and has been dead for
quite a while. We have better ways to emit endian-dependent stuff now.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246571 91177308-0d34-0410-b5e6-96231b3b80d8
This matches the ARM behavior. In both cases, the register is part
of the optional Performance Monitors extension, so, add the feature,
and enable it for the A-class processors we support.
Differential Revision: http://reviews.llvm.org/D12425
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246555 91177308-0d34-0410-b5e6-96231b3b80d8
There are occasions where it is useful to consider the entirety of the
contents of a section. For example, compressed debug info needs the
entire section available before it can compress it and write it out.
The compressed debug info scenario was previously implemented by
mirroring the implementation of writeSectionData in the ELFObjectWriter.
Instead, allow the output stream to be swapped on demand. This lets
callers redirect the output stream to a more convenient location before
it hits the object file.
No functionality change is intended.
Differential Revision: http://reviews.llvm.org/D12509
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246554 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This change turns on by default interleaved access vectorization
for AArch64.
We also clean up some tests which were spedifically enabling this
behaviour.
Reviewers: rengolin
Subscribers: aemerson, llvm-commits, rengolin
Differential Revision: http://reviews.llvm.org/D12149
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246542 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This change turns on by default interleaved access vectorization on ARM,
as it has shown to be beneficial on ARM.
Reviewers: rengolin
Subscribers: aemerson, llvm-commits, rengolin
Differential Revision: http://reviews.llvm.org/D12146
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246541 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Interleaved access lowering removes a memory operation and a
sequence of vector shuffles and replaces it with a series of
memory operations. This should be always beneficial.
This pass in only enabled on ARM/AArch64.
Reviewers: rengolin
Subscribers: aemerson, llvm-commits, rengolin
Differential Revision: http://reviews.llvm.org/D12145
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246540 91177308-0d34-0410-b5e6-96231b3b80d8
If a symbol is marked as "data", the symbol should be exported
with __imp_ prefix. Previously, the symbol was exported as-is.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246532 91177308-0d34-0410-b5e6-96231b3b80d8
Currently, if you call cmake with a typo in an LLVM_USE_SANITIZER
value, there's a cmake warning and the build goes on with no
sanitizers at all. This isn't a good behaviour, since cmake warnings
are fairly easy to miss and the resulting behaviour is that it looks
like the build is sanitizer clean.
Upgrade these warnings to errors so misconfigurations are more
obvious.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246531 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Three closely related changes, to have a mode in which we link all
executables and shared libraries against libLLVM.
1. Add a new LLVM_LINK_LLVM_DYLIB cmake option, which, when ON, will link
executables and shared libraries against libLLVM. For this to work, it
is necessary to also set LLVM_BUILD_LLVM_DYLIB and LLVM_DYLIB_EXPORT_ALL.
It is not strictly necessary to set LLVM_DISABLE_LLVM_DYLIB_ATEXIT, but
we also default to OFF in this mode, or tools tend to misbehave (e.g.
stdout may not flush on exit when output is buffered.)
llvm-config and Tablegen do not use libLLVM, as they are dependencies of
libLLVM.
2. Modify llvm-go to take a new flag, "linkmode=component-libs|dylib".
Depending on which one is passed (default is component-libs), we link
with the individual libraries or libLLVM respectively. We pass in dylib
when LLVM_LINK_LLVM_DYLIB is ON.
3. Fix LLVM_DYLIB_EXPORT_ALL on Linux, and expand the symbols exported to
actually export all. Don't strip leading underscore from symbols on Linux,
and make sure we get all exported symbols and weak-with-default symbols
("W" in nm output). Without these changes, passes won't load because
the "Annotate..." symbols defined in lib/Support/Valigrind.cpp are not
found.
Testing:
- Ran default build ("ninja") with LLVM, clang, compiler-rt, llgo, lldb.
- Ran "check", "check-clang", "check-tsan", "check-libgo" targets. I've
never had much success with LLDB tests, and llgoi is currently broken
so check-llgo fails for an unrelated reason.
- Ran "lldb" to ensure it loads.
Reviewers: chandlerc, beanz, pcc, rnk
Subscribers: rnk, chapuni, sylvestre.ledru, llvm-commits
Differential Revision: http://reviews.llvm.org/D12488
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246527 91177308-0d34-0410-b5e6-96231b3b80d8
These are already added during the MachineInstr construction,
so this was adding the implicit registers twice.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246525 91177308-0d34-0410-b5e6-96231b3b80d8
Currently, when edge weights are assigned to edges that are created when lowering switch statement, the weight on the edge to default statement (let's call it "default weight" here) is not considered. We need to distribute this weight properly. However, without value profiling, we have no idea how to distribute it. In this patch, I applied the heuristic that this weight is evenly distributed to successors.
For example, given a switch statement with cases 1,2,3,5,10,11,20, and every edge from switch to each successor has weight 10. If there is a binary search tree built to test if n < 10, then its two out-edges will have weight 4x10+10/2 = 45 and 3x10 + 10/2 = 35 respectively (currently they are 40 and 30 without considering the default weight). Each distribution (which is 5 here) will be stored in each SwitchWorkListItem for further distribution.
There are some exceptions:
For a jump table header which doesn't have any edge to default statement, we don't distribute the default weight to it.
For a bit test header which covers a contiguous range and hence has no edges to default statement, we don't distribute the default weight to it.
When the branch checks a single value or a contiguous range with no edge to default statement, we don't distribute the default weight to it.
In other cases, the default weight is evenly distributed to successors.
Differential Revision: http://reviews.llvm.org/D12418
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246522 91177308-0d34-0410-b5e6-96231b3b80d8
Follow LLVM style for the parameter names (`CamelCase` not `camelCase`),
and surface the header docs in doxygen. No functionality change
intended.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246509 91177308-0d34-0410-b5e6-96231b3b80d8
SETCC is one of those special node types for which operation actions (legality,
etc.) is keyed off of an operand type, not the node's value type. This makes
sense because the value type of a legal SETCC node is determined by its
operands' value type (via the TLI function getSetCCResultType). When the
SDAGBuilder creates SETCC nodes, it either creates them with an MVT::i1 value
type, or directly with the value type provided by TLI.getSetCCResultType.
The first problem being fixed here is that DAGCombine had several places
querying TLI.isOperationLegal on SETCC, but providing the return of
getSetCCResultType, instead of the operand type directly. This does not mean
what the author thought, and "luckily", most in-tree targets have SETCC with
Custom lowering, instead of marking them Legal, so these checks return false
anyway.
The second problem being fixed here is that two of the DAGCombines could create
SETCC nodes with arbitrary (integer) value types; specifically, those that
would simplify:
(setcc a, b, op1) and|or (setcc a, b, op2) -> setcc a, b, op3
(which is possible for some combinations of (op1, op2))
If the operands of the and|or node are actual setcc nodes, then this is not an
issue (because the and|or must share the same type), but, the relevant code in
DAGCombiner::visitANDLike and DAGCombiner::visitORLike actually calls
DAGCombiner::isSetCCEquivalent on each operand, and that function will
recognise setcc-like select_cc nodes with other return types. And, thus, when
creating new SETCC nodes, we need to be careful to respect the value-type
constraint. This is even true before type legalization, because it is quite
possible for the SELECT_CC node to have a legal type that does not happen to
match the corresponding TLI.getSetCCResultType type.
To be explicit, there is nothing that later fixes the value types of SETCC
nodes (if the type is legal, but does not happen to match
TLI.getSetCCResultType). Creating SETCCs with an MVT::i1 value type seems to
work only because, either MVT::i1 is not legal, or it is what
TLI.getSetCCResultType returns if it is legal. Fixing that is a larger change,
however. For the time being, restrict the relevant transformations to produce
only SETCC nodes with a value type matching TLI.getSetCCResultType (or MVT::i1
prior to type legalization).
Fixes PR24636.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246507 91177308-0d34-0410-b5e6-96231b3b80d8
Hopefully this will end the GEPs saga!
This commit reverts r245394, i.e., it reapplies r221876 while incorporating the
fixes from D11847.
r221876 was not reapplied alone because it was not safe and D11847 was not
applied alone because it needs r221876 to produce correct results.
This should fix PR24596.
Original commit message for r221876:
Let's try this again...
This reverts r219432, plus a bug fix.
Description of the bug in r219432 (by Nick):
The bug was using AllPositive to break out of the loop; if the loop break
condition i != e is changed to i != e && AllPositive then the
test_modulo_analysis_with_global test I've added will fail as the Modulo will
be calculated incorrectly (as the last loop iteration is skipped, so Modulo
isn't updated with its Scale).
Nick also adds this comment:
ComputeSignBit is safe to use in loops as it takes into account phi nodes, and
the == EK_ZeroEx check is safe in loops as, no matter how the variable changes
between iterations, zero-extensions will always guarantee a zero sign bit. The
isValueEqualInPotentialCycles check is therefore definitely not needed as all
the variable analysis holds no matter how the variables change between loop
iterations.
And this patch also adds another enhancement to GetLinearExpression - basically
to convert ConstantInts to Offsets (see test_const_eval and
test_const_eval_scaled for the situations this improves).
Original commit message:
This reverts r218944, which reverted r218714, plus a bug fix.
Description of the bug in r218714 (by Nick):
The original patch forgot to check if the Scale in VariableGEPIndex flipped the
sign of the variable. The BasicAA pass iterates over the instructions in the
order they appear in the function, and so BasicAliasAnalysis::aliasGEP is
called with the variable it first comes across as parameter GEP1. Adding a
%reorder label puts the definition of %a after %b so aliasGEP is called with %b
as the first parameter and %a as the second. aliasGEP later calculates that %a
== %b + 1 - %idxprom where %idxprom >= 0 (if %a was passed as the first
parameter it would calculate %b == %a - 1 + %idxprom where %idxprom >= 0) -
ignoring that %idxprom is scaled by -1 here lead the patch to incorrectly
conclude that %a > %b.
Revised patch by Nick White, thanks! Thanks to Lang to isolating the bug.
Slightly modified by me to add an early exit from the loop and avoid
unnecessary, but expensive, function calls.
Original commit message:
Two related things:
1. Fixes a bug when calculating the offset in GetLinearExpression. The code
previously used zext to extend the offset, so negative offsets were converted
to large positive ones.
2. Enhance aliasGEP to deduce that, if the difference between two GEP
allocations is positive and all the variables that govern the offset are also
positive (i.e. the offset is strictly after the higher base pointer), then
locations that fit in the gap between the two base pointers are NoAlias.
Patch by Nick White!
Message from D11847:
Un-revert of r241981 and fix for PR23626. The 'Or' case of GetLinearExpression
delegates to 'Add' if possible, and if not it returns an Opaque value.
Unfortunately the Scale and Offsets weren't being set (and so defaulted to 0) -
and a scale of zero effectively removes the variable from the GEP instruction.
This meant that BasicAA would return MustAliases when it should have been
returning PartialAliases (and PR23626 was an example of the GVN pass using an
incorrect MustAlias to merge loads from what should have been different
pointers).
Differential Revision: http://reviews.llvm.org/D11847
Patch by Nick White <n.j.white@gmail.com>!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246502 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: This handles all load/store operations that WebAssembly defines, and handles those necessary for C++ such as i1. I left a FIXME for outstanding features which aren't required for now.
Reviewers: sunfish
Subscribers: jfb, llvm-commits, dschuff
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246500 91177308-0d34-0410-b5e6-96231b3b80d8
This was part of D7208 (r227242), but that commit was reverted because it exposed
a bug in AArch64 lowering. I should have that fixed and the rest of the commit
reinstated soon.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246493 91177308-0d34-0410-b5e6-96231b3b80d8
