123 Commits

Author SHA1 Message Date
Dmitri Gribenko
be9f44f943 Revert "Reland "r364412 [ExpandMemCmp][MergeICmps] Move passes out of CodeGen into opt pipeline.""
This reverts commit r371502, it broke tests
(clang/test/CodeGenCXX/auto-var-init.cpp).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371507 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-10 10:39:09 +00:00
Clement Courbet
9f23a59130 Reland "r364412 [ExpandMemCmp][MergeICmps] Move passes out of CodeGen into opt pipeline."
With a fix for sanitizer breakage (see explanation in D60318).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@371502 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-10 09:18:00 +00:00
Benjamin Kramer
8a67530447 Do a sweep of symbol internalization. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@369803 91177308-0d34-0410-b5e6-96231b3b80d8
2019-08-23 19:59:23 +00:00
Jonas Devlieghere
114087caa6 [llvm] Migrate llvm::make_unique to std::make_unique
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@369013 91177308-0d34-0410-b5e6-96231b3b80d8
2019-08-15 15:54:37 +00:00
Fangrui Song
727b16e096 Delete dead stores
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@365903 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-12 14:58:15 +00:00
Clement Courbet
6f6d98e186 Revert "r364412 [ExpandMemCmp][MergeICmps] Move passes out of CodeGen into opt pipeline."
Breaks sanitizers:
    libFuzzer :: cxxstring.test
    libFuzzer :: memcmp.test
    libFuzzer :: recommended-dictionary.test
    libFuzzer :: strcmp.test
    libFuzzer :: value-profile-mem.test
    libFuzzer :: value-profile-strcmp.test

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@364416 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-26 12:13:13 +00:00
Clement Courbet
e0fc543f4c [ExpandMemCmp][MergeICmps] Move passes out of CodeGen into opt pipeline.
This allows later passes (in particular InstCombine) to optimize more
cases.

One that's important to us is `memcmp(p, q, constant) < 0` and memcmp(p, q, constant) > 0.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@364412 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-26 11:50:18 +00:00
Matt Arsenault
5b56cc85b0 Rename ExpandISelPseudo->FinalizeISel, delay register reservation
This allows targets to make more decisions about reserved registers
after isel. For example, now it should be certain there are calls or
stack objects in the frame or not, which could have been introduced by
legalization.

Patch by Matthias Braun

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363757 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-19 00:25:39 +00:00
Jonas Paulsson
ff136c756d [SystemZ, RegAlloc] Favor 3-address instructions during instruction selection.
This patch aims to reduce spilling and register moves by using the 3-address
versions of instructions per default instead of the 2-address equivalent
ones. It seems that both spilling and register moves are improved noticeably
generally.

Regalloc hints are passed to increase conversions to 2-address instructions
which are done in SystemZShortenInst.cpp (after regalloc).

Since the SystemZ reg/mem instructions are 2-address (dst and lhs regs are
the same), foldMemoryOperandImpl() can no longer trivially fold a spilled
source register since the reg/reg instruction is now 3-address. In order to
remedy this, new 3-address pseudo memory instructions are used to perform the
folding only when the dst and lhs virtual registers are known to be allocated
to the same physreg. In order to not let MachineCopyPropagation run and
change registers on these transformed instructions (making it 3-address), a
new target pass called SystemZPostRewrite.cpp is run just after
VirtRegRewriter, that immediately lowers the pseudo to a target instruction.

If it would have been possibe to insert a COPY instruction and change a
register operand (convert to 2-address) in foldMemoryOperandImpl() while
trusting that the caller (e.g. InlineSpiller) would update/repair the
involved LiveIntervals, the solution involving pseudo instructions would not
have been needed. This is perhaps a potential improvement (see Phabricator
post).

Common code changes:

* A new hook TargetPassConfig::addPostRewrite() is utilized to be able to run a
target pass immediately before MachineCopyPropagation.

* VirtRegMap is passed as an argument to foldMemoryOperand().

Review: Ulrich Weigand, Quentin Colombet
https://reviews.llvm.org/D60888

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@362868 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-08 06:19:15 +00:00
Clement Courbet
a6ff3047e7 [MergeICmps] Make the pass compatible with the new pass manager.
Reviewers: gchatelet, spatel

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D62287

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@361490 91177308-0d34-0410-b5e6-96231b3b80d8
2019-05-23 12:35:26 +00:00
Amara Emerson
040f61e117 [GlobalISel] Enable CSE in the IRTranslator & legalizer for -O0 with constants only.
Other opcodes shouldn't be CSE'd until we can be sure debug info quality won't
be degraded.

This change also improves the IRTranslator so that in most places, but not all,
it creates constants using the MIRBuilder directly instead of first creating a
new destination vreg and then creating a constant. By doing this, the
buildConstant() method can just return the vreg of an existing G_CONSTANT
instead of having to create a COPY from it.

I measured a 0.2% improvement in compile time and a 0.9% improvement in code
size at -O0 ARM64.

Compile time:
Program                                        base   cse    diff
test-suite...ark/tramp3d-v4/tramp3d-v4.test     9.04   9.12  0.8%
test-suite...Mark/mafft/pairlocalalign.test     2.68   2.66 -0.7%
test-suite...-typeset/consumer-typeset.test     5.53   5.51 -0.4%
test-suite :: CTMark/lencod/lencod.test         5.30   5.28 -0.3%
test-suite :: CTMark/Bullet/bullet.test        25.82  25.76 -0.2%
test-suite...:: CTMark/ClamAV/clamscan.test     6.92   6.90 -0.2%
test-suite...TMark/7zip/7zip-benchmark.test    34.24  34.17 -0.2%
test-suite :: CTMark/SPASS/SPASS.test           6.25   6.24 -0.1%
test-suite...:: CTMark/sqlite3/sqlite3.test     1.66   1.66 -0.1%
test-suite :: CTMark/kimwitu++/kc.test         13.61  13.60 -0.0%
Geomean difference                                          -0.2%

Code size:
Program                                        base     cse      diff
test-suite...-typeset/consumer-typeset.test    1315632  1266480 -3.7%
test-suite...:: CTMark/ClamAV/clamscan.test    1313892  1297508 -1.2%
test-suite :: CTMark/lencod/lencod.test        1439504  1423112 -1.1%
test-suite...TMark/7zip/7zip-benchmark.test    2936980  2904172 -1.1%
test-suite :: CTMark/Bullet/bullet.test        3478276  3445460 -0.9%
test-suite...ark/tramp3d-v4/tramp3d-v4.test    8082868  8033492 -0.6%
test-suite :: CTMark/kimwitu++/kc.test         3870380  3853972 -0.4%
test-suite :: CTMark/SPASS/SPASS.test          1434904  1434896 -0.0%
test-suite...Mark/mafft/pairlocalalign.test    764528   764528   0.0%
test-suite...:: CTMark/sqlite3/sqlite3.test    782092   782092   0.0%
Geomean difference                                              -0.9%

Differential Revision: https://reviews.llvm.org/D60580

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358369 91177308-0d34-0410-b5e6-96231b3b80d8
2019-04-15 05:04:20 +00:00
Amara Emerson
3d4f91d8d0 [GlobalISel] Introduce a CSEConfigBase class to allow targets to define their own CSE configs.
Because CodeGen can't depend on GlobalISel, we need a way to encapsulate the CSE
configs that can be passed between TargetPassConfig and the targets' custom
pass configs. This CSEConfigBase allows targets to create custom CSE configs
which is then used by the GISel passes for the CSEMIRBuilder.

This support will be used in a follow up commit to allow constant-only CSE for
-O0 compiles in D60580.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358368 91177308-0d34-0410-b5e6-96231b3b80d8
2019-04-15 04:53:46 +00:00
Matt Arsenault
951c9d9b26 CodeGen: Refactor regallocator command line and target selection
This will allow targets more flexibility to replace the
register allocator core passes. In a future commit,
AMDGPU will run the core register assignment passes
twice, and will also want to disallow using the
standard -regalloc option.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@356506 91177308-0d34-0410-b5e6-96231b3b80d8
2019-03-19 19:33:12 +00:00
Daniel Sanders
d57d5e8d9c Restore ability for C++ API users to Enable IPRA.
Summary:
Prior to r310876 one of our out-of-tree targets was enabling IPRA by modifying
the TargetOptions::EnableIPRA. This no longer works on current trunk since the
useIPRA() hook overrides any values that are set in advance. This patch adjusts
the behaviour of the hook so that API users and useIPRA() can both enable it
but useIPRA() cannot disable it if the API user already enabled it.

Reviewers: arsenm

Reviewed By: arsenm

Subscribers: wdng, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D38043

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354692 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-22 20:59:07 +00:00
Matt Arsenault
664a45f97e CodeGen: Make RegAllocRegistry a template class
Will allow re-using the machinery for independent
sets of register allocators.

This will allow AMDGPU to use separate command line
options for the allocator to use for SGPRs separate
from VGPRs.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354687 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-22 19:16:52 +00:00
Aditya Nandakumar
47a2c804dd [GISel]: Change how CSE is enabled by default for each pass
https://reviews.llvm.org/D57178

Now add a hook in TargetPassConfig to query if CSE needs to be
enabled. By default this hook returns false only for O0 opt level but
this can be overridden by the target.
As a consequence of the default of enabled for non O0, a few tests
needed to be updated to not use CSE (by passing in -O0) to the run
line.

reviewed by: arsenm

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@352126 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-24 23:11:25 +00:00
Chandler Carruth
6b547686c5 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@351636 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-19 08:50:56 +00:00
Petr Pavlu
7885d2ca5a [GlobalISel] Fix choice of instruction selector for AArch64 at -O0 with -global-isel=0
Commit rL347861 introduced an unintentional change in the behaviour when
compiling for AArch64 at -O0 with -global-isel=0. Previously, explicitly
disabling GlobalISel resulted in using FastISel but an updated condition
in the commit changed it to using SelectionDAG. The patch fixes this
condition and slightly better organizes the code that chooses the
instruction selector.

Fixes PR40131.

Differential Revision: https://reviews.llvm.org/D56266


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@350626 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-08 14:19:06 +00:00
Matt Arsenault
4c2adea438 MIR: Add method to stop after specific runs of passes
Currently if you use -{start,stop}-{before,after}, it picks
the first instance with the matching pass name. If you run
the same pass multiple times, there's no way to distinguish them.

Allow specifying a run index wih ,N to specify which you mean.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@348285 91177308-0d34-0410-b5e6-96231b3b80d8
2018-12-04 17:45:12 +00:00
Petr Pavlu
3834f85200 [GlobalISel] Make EnableGlobalISel always set when GISel is enabled
Change meaning of TargetOptions::EnableGlobalISel. The flag was
previously set only when a target switched on GlobalISel but it is now
always set when the GlobalISel pipeline is enabled. This makes the flag
consistent with TargetOptions::EnableFastISel and allows its use in
other parts of the compiler to determine when GlobalISel is enabled.

The EnableGlobalISel flag had previouly only one use in
TargetPassConfig::isGlobalISelAbortEnabled(). The method used its value
to determine if GlobalISel was enabled by a target and returned false in
such a case. To preserve the current behaviour, a new flag
TargetOptions::GlobalISelAbort is introduced to separately record the
abort behaviour.

Differential Revision: https://reviews.llvm.org/D54518


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@347861 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-29 12:56:32 +00:00
Serge Guelton
bbe5fdb08d Type safe version of MachinePassRegistry
Previous version used type erasure through a `void* (*)()` pointer,
which triggered gcc warning and implied a lot of reinterpret_cast.

This version should make it harder to hit ourselves in the foot.

Differential revision: https://reviews.llvm.org/D54203

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@346522 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-09 17:19:45 +00:00
Matthias Braun
94f7fc2ccd LLVMTargetMachine/TargetPassConfig: Simplify handling of start/stop options; NFC
- Make some TargetPassConfig methods that just check whether options have
  been set static.
- Shuffle code in LLVMTargetMachine around so addPassesToGenerateCode
  only deals with TargetPassConfig now (but not with MCContext or the
  creation of MachineModuleInfo)

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@345918 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-02 01:31:50 +00:00
Francis Visoiu Mistrih
6165c5d19d [llc] Error out when -print-machineinstrs is used with an unknown pass
We used to assert instead of reporting an error.

PR39494

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@345589 91177308-0d34-0410-b5e6-96231b3b80d8
2018-10-30 12:07:18 +00:00
Daniel Sanders
9f149bc4ff Correct implementation of -verify-machineinstrs such that it's still overridable for EXPENSIVE_CHECKS
-verify-machineinstrs was implemented as a simple bool. As a result, the
'VerifyMachineCode == cl::BOU_UNSET' used by EXPENSIVE_CHECKS to make it on by
default but possible to disable didn't work as intended. Changed
-verify-machineinstrs to a boolOrDefault to correct this.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343696 91177308-0d34-0410-b5e6-96231b3b80d8
2018-10-03 16:29:24 +00:00
Daniel Sanders
8f3aebf19b [globalisel][verifier] Run the MachineVerifier from IRTranslator onwards
-verify-machineinstrs inserts the MachineVerifier after every MachineInstr-based
pass. However, GlobalISel creates MachineInstr-based passes earlier than DAGISel
and the corresponding verifiers are not being added. This patch fixes that.

If GlobalISel triggers the fallback path then the MIR can be left in a bad
state that is going to be cleared by ResetMachineFunctions. In this situation
verifying between GlobalISel passes will prevent the fallback path from
recovering from this. As a result, we bail out of verifying a function if the
FailedISel attribute is present.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343613 91177308-0d34-0410-b5e6-96231b3b80d8
2018-10-02 17:56:58 +00:00
Fangrui Song
af7b1832a0 Remove trailing space
sed -Ei 's/[[:space:]]+$//' include/**/*.{def,h,td} lib/**/*.{cpp,h}

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@338293 91177308-0d34-0410-b5e6-96231b3b80d8
2018-07-30 19:41:25 +00:00
Jessica Paquette
c549767db5 [MachineOutliner] Add support for target-default outlining.
This adds functionality to the outliner that allows targets to
specify certain functions that should be outlined from by default.

If a target supports default outlining, then it specifies that in
its TargetOptions. In the case that it does, and the user hasn't
specified that they *never* want to outline, the outliner will
be added to the pass pipeline and will run on those default functions.

This is a preliminary patch for turning the outliner on by default
under -Oz for AArch64.

https://reviews.llvm.org/D48776

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@336040 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-30 03:56:03 +00:00
Jessica Paquette
d1bf1e8336 [MachineOutliner] Add always and never options to -enable-machine-outliner
This is a recommit of r335887, which was erroneously committed earlier.

To enable the MachineOutliner by default on AArch64, we need to be able to
disable the MachineOutliner and also provide an option to "always" enable the
outliner.

This adds that capability. It allows the user to still use the old
-enable-machine-outliner option, which defaults to "always". This is building
up to allowing the user to specify "always" versus the target default
outlining behaviour.

https://reviews.llvm.org/D48682

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@335986 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-29 16:12:45 +00:00
Jessica Paquette
519acca334 [MachineOutliner] Never add the outliner in -O0
This is a recommit of r335879.

We shouldn't add the outliner when compiling at -O0 even if
-enable-machine-outliner is passed in. This makes sure that we
don't add it in this case.

This also removes -O0 from the outliner DWARF test.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@335930 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-28 21:49:24 +00:00
Jessica Paquette
d6b8d8c07b [MachineOutliner] Define MachineOutliner support in TargetOptions
Targets should be able to define whether or not they support the outliner
without the outliner being added to the pass pipeline. Before this, the
outliner pass would be added, and ask the target whether or not it supports the
outliner.

After this, it's possible to query the target in TargetPassConfig, before the
outliner pass is created. This ensures that passing -enable-machine-outliner
will not modify the pass pipeline of any target that does not support it.

https://reviews.llvm.org/D48683

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@335887 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-28 17:45:43 +00:00
Jessica Paquette
9bc4099b79 Revert "[MachineOutliner] Add always and never options to -enable-machine-outliner"
I accidentally committed this instead of D48683 because I haven't had coffee
yet.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@335883 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-28 17:26:19 +00:00
Jessica Paquette
be2cdf90a3 Revert "[MachineOutliner] Never add the outliner in -O0"
This reverts commit 9c7c10e4073a0bc6a759ce5cd33afbac74930091.

It relies on r335872 since that introduces the machine outliner
flags test. I meant to commit D48683 in that commit, but got mixed
up and committed D48682 instead. So, I'm reverting this and
r335872, since D48682 hasn't made it through review yet.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@335882 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-28 17:26:18 +00:00
Jessica Paquette
9c7c10e407 [MachineOutliner] Never add the outliner in -O0
We shouldn't add the outliner when compiling at -O0 even if
-enable-machine-outliner is passed in. This makes sure that we
don't add it in this case.

This also updates machine-outliner-flags to reflect the change
and improves the comment describing what that test does.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@335879 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-28 17:05:57 +00:00
Jessica Paquette
bd0a2a58e5 [MachineOutliner] Add always and never options to -enable-machine-outliner
To enable the MachineOutliner by default on AArch64, we need to be able to
disable the MachineOutliner and also provide an option to "always" enable the
outliner.

This adds that capability. It allows the user to still use the old
-enable-machine-outliner option, which defaults to "always". This is building
up to allowing the user to specify "always" versus the target-default
outlining behaviour.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@335872 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-28 16:39:42 +00:00
Heejin Ahn
a6e37da488 [WebAssembly] Add Wasm exception handling prepare pass
Summary:
This adds a pass that transforms a program to be prepared for Wasm
exception handling. This is using Windows EH instructions and based on
the previous Wasm EH proposal.
(https://github.com/WebAssembly/exception-handling/blob/master/proposals/Exceptions.md)

Reviewers: dschuff, majnemer

Subscribers: jfb, mgorny, sbc100, jgravelle-google, JDevlieghere, sunfish, llvm-commits

Differential Revision: https://reviews.llvm.org/D43746

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333696 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-31 22:02:34 +00:00
Jessica Paquette
7256bf169a [MachineOutliner] NFC: Move EnableLinkOnceODROutlining into MachineOutliner.cpp
This moves the EnableLinkOnceODROutlining flag from TargetPassConfig.cpp into
MachineOutliner.cpp. It also removes OutlineFromLinkOnceODRs from the
MachineOutliner constructor. This is now handled by the moved command-line
flag.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330373 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-19 22:17:07 +00:00
David Blaikie
49ca55e381 Transforms: Introduce Transforms/Utils.h rather than spreading the declarations amongst Scalar.h and IPO.h
Fixes layering - Transforms/Utils shouldn't depend on including a Scalar
or IPO header, because Scalar and IPO depend on Utils.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@328717 91177308-0d34-0410-b5e6-96231b3b80d8
2018-03-28 17:44:36 +00:00
Jun Bum Lim
af56683019 [CodeGen] Add a new pass for PostRA sink
Summary:
This pass sinks COPY instructions into a successor block, if the COPY is not
used in the current block and the COPY is live-in to a single successor
(i.e., doesn't require the COPY to be duplicated).  This avoids executing the
the copy on paths where their results aren't needed.  This also exposes
additional opportunites for dead copy elimination and shrink wrapping.

These copies were either not handled by or are inserted after the MachineSink
pass. As an example of the former case, the MachineSink pass cannot sink
COPY instructions with allocatable source registers; for AArch64 these type
of copy instructions are frequently used to move function parameters (PhyReg)
into virtual registers in the entry block..

For the machine IR below, this pass will sink %w19 in the entry into its
successor (%bb.1) because %w19 is only live-in in %bb.1.

```
   %bb.0:
      %wzr = SUBSWri %w1, 1
      %w19 = COPY %w0
      Bcc 11, %bb.2
    %bb.1:
      Live Ins: %w19
      BL @fun
      %w0 = ADDWrr %w0, %w19
      RET %w0
    %bb.2:
      %w0 = COPY %wzr
      RET %w0
```
As we sink %w19 (CSR in AArch64) into %bb.1, the shrink-wrapping pass will be
able to see %bb.0 as a candidate.

With this change I observed 12% more shrink-wrapping candidate and 13% more dead copies deleted  in spec2000/2006/2017 on AArch64.

Reviewers: qcolombet, MatzeB, thegameg, mcrosier, gberry, hfinkel, john.brawn, twoh, RKSimon, sebpop, kparzysz

Reviewed By: sebpop

Subscribers: evandro, sebpop, sfertile, aemerson, mgorny, javed.absar, kristof.beyls, llvm-commits

Differential Revision: https://reviews.llvm.org/D41463

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@328237 91177308-0d34-0410-b5e6-96231b3b80d8
2018-03-22 20:06:47 +00:00
Clement Courbet
cae812cc14 [MergeICmps] Re-land 324317 "Enable the MergeICmps Pass by default."
Now that PR36557 is fixed.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@327840 91177308-0d34-0410-b5e6-96231b3b80d8
2018-03-19 13:37:04 +00:00
Clement Courbet
af45cfce8b [MergeICmps] Revert 324317 "Enable the MergeICmps Pass by default."
While working on PR36557.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@326575 91177308-0d34-0410-b5e6-96231b3b80d8
2018-03-02 14:34:49 +00:00
Chih-Hung Hsieh
70716e54e0 [TLS] use emulated TLS if the target supports only this mode
Emulated TLS is enabled by llc flag -emulated-tls,
which is passed by clang driver.
When llc is called explicitly or from other drivers like LTO,
missing -emulated-tls flag would generate wrong TLS code for targets
that supports only this mode.
Now use useEmulatedTLS() instead of Options.EmulatedTLS to decide whether
emulated TLS code should be generated.
Unit tests are modified to run with and without the -emulated-tls flag.

Differential Revision: https://reviews.llvm.org/D42999



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@326341 91177308-0d34-0410-b5e6-96231b3b80d8
2018-02-28 17:48:55 +00:00
Geoff Berry
1bfec906a8 Re-enable "[MachineCopyPropagation] Extend pass to do COPY source forwarding"
Re-enable commit r323991 now that r325931 has been committed to make
MachineOperand::isRenamable() check more conservative w.r.t. code
changes and opt-in on a per-target basis.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@326208 91177308-0d34-0410-b5e6-96231b3b80d8
2018-02-27 16:59:10 +00:00
Heejin Ahn
62779f87d5 [WebAssembly] Add exception handling option and feature
Summary:
Add a llc command line option and WebAssembly architecture feature for
exception handling.

Reviewers: dschuff

Subscribers: jfb, sbc100, jgravelle-google, sunfish, llvm-commits

Differential Revision: https://reviews.llvm.org/D43683

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@326004 91177308-0d34-0410-b5e6-96231b3b80d8
2018-02-24 00:40:50 +00:00
Quentin Colombet
0d9a8774c9 Revert "[MachineCopyPropagation] Extend pass to do COPY source forwarding"
This reverts commit r323991.

This commit breaks target that don't model all the register constraints
in TableGen. So far the workaround was to set the
hasExtraXXXRegAllocReq, but it proves that it doesn't cover all the
cases.
For instance, when mutating an instruction (like in the lowering of
COPYs) the isRenamable flag is not properly updated. The same problem
will happen when attaching machine operand from one instruction to
another.

Geoff Berry is working on a fix in https://reviews.llvm.org/D43042.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@325421 91177308-0d34-0410-b5e6-96231b3b80d8
2018-02-17 03:05:33 +00:00
Clement Courbet
ee43c8204a [MergeICmps] Re-commit rL324317 "Enable the MergeICmps Pass by default."
With fixes from rL324341.

Original commit message:

[MergeICmps] Enable the MergeICmps Pass by default.

Summary: Now that PR33325 is fixed, this should always improve the generated code.

Reviewers: spatel

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D42793

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@324465 91177308-0d34-0410-b5e6-96231b3b80d8
2018-02-07 09:58:55 +00:00
Clement Courbet
74c3cb19aa Revert "[MergeICmps] Enable the MergeICmps Pass by default."
Breaks clang-ppc64be-linux-multistage buildbot.

This reverts commit 515bab711f308c2e8299c49dd8c84ea6a2e0b60e.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@324319 91177308-0d34-0410-b5e6-96231b3b80d8
2018-02-06 08:40:18 +00:00
Clement Courbet
b458d85f09 [MergeICmps] Enable the MergeICmps Pass by default.
Summary: Now that PR33325 is fixed, this should always improve the generated code.

Reviewers: spatel

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D42793

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@324317 91177308-0d34-0410-b5e6-96231b3b80d8
2018-02-06 07:20:33 +00:00
Geoff Berry
c1676d5461 [MachineCopyPropagation] Extend pass to do COPY source forwarding
Summary:
This change extends MachineCopyPropagation to do COPY source forwarding
and adds an additional run of the pass to the default pass pipeline just
after register allocation.

This version of this patch uses the newly added
MachineOperand::isRenamable bit to avoid forwarding registers is such a
way as to violate constraints that aren't captured in the
Machine IR (e.g. ABI or ISA constraints).

This change is a continuation of the work started in D30751.

Reviewers: qcolombet, javed.absar, MatzeB, jonpa, tstellar

Subscribers: tpr, mgorny, mcrosier, nhaehnle, nemanjai, jyknight, hfinkel, arsenm, inouehrs, eraman, sdardis, guyblank, fedor.sergeev, aheejin, dschuff, jfb, myatsina, llvm-commits

Differential Revision: https://reviews.llvm.org/D41835

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@323991 91177308-0d34-0410-b5e6-96231b3b80d8
2018-02-01 18:54:01 +00:00
Amara Emerson
542b076346 [GlobalISel] Don't fall back to FastISel.
Apparently checking the pass structure isn't enough to ensure that we don't fall
back to FastISel, as it's set up as part of the SelectionDAGISel.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@323369 91177308-0d34-0410-b5e6-96231b3b80d8
2018-01-24 19:59:29 +00:00
Chandler Carruth
fd5a8723ce Introduce the "retpoline" x86 mitigation technique for variant #2 of the speculative execution vulnerabilities disclosed today, specifically identified by CVE-2017-5715, "Branch Target Injection", and is one of the two halves to Spectre..
Summary:
First, we need to explain the core of the vulnerability. Note that this
is a very incomplete description, please see the Project Zero blog post
for details:
https://googleprojectzero.blogspot.com/2018/01/reading-privileged-memory-with-side.html

The basis for branch target injection is to direct speculative execution
of the processor to some "gadget" of executable code by poisoning the
prediction of indirect branches with the address of that gadget. The
gadget in turn contains an operation that provides a side channel for
reading data. Most commonly, this will look like a load of secret data
followed by a branch on the loaded value and then a load of some
predictable cache line. The attacker then uses timing of the processors
cache to determine which direction the branch took *in the speculative
execution*, and in turn what one bit of the loaded value was. Due to the
nature of these timing side channels and the branch predictor on Intel
processors, this allows an attacker to leak data only accessible to
a privileged domain (like the kernel) back into an unprivileged domain.

The goal is simple: avoid generating code which contains an indirect
branch that could have its prediction poisoned by an attacker. In many
cases, the compiler can simply use directed conditional branches and
a small search tree. LLVM already has support for lowering switches in
this way and the first step of this patch is to disable jump-table
lowering of switches and introduce a pass to rewrite explicit indirectbr
sequences into a switch over integers.

However, there is no fully general alternative to indirect calls. We
introduce a new construct we call a "retpoline" to implement indirect
calls in a non-speculatable way. It can be thought of loosely as
a trampoline for indirect calls which uses the RET instruction on x86.
Further, we arrange for a specific call->ret sequence which ensures the
processor predicts the return to go to a controlled, known location. The
retpoline then "smashes" the return address pushed onto the stack by the
call with the desired target of the original indirect call. The result
is a predicted return to the next instruction after a call (which can be
used to trap speculative execution within an infinite loop) and an
actual indirect branch to an arbitrary address.

On 64-bit x86 ABIs, this is especially easily done in the compiler by
using a guaranteed scratch register to pass the target into this device.
For 32-bit ABIs there isn't a guaranteed scratch register and so several
different retpoline variants are introduced to use a scratch register if
one is available in the calling convention and to otherwise use direct
stack push/pop sequences to pass the target address.

This "retpoline" mitigation is fully described in the following blog
post: https://support.google.com/faqs/answer/7625886

We also support a target feature that disables emission of the retpoline
thunk by the compiler to allow for custom thunks if users want them.
These are particularly useful in environments like kernels that
routinely do hot-patching on boot and want to hot-patch their thunk to
different code sequences. They can write this custom thunk and use
`-mretpoline-external-thunk` *in addition* to `-mretpoline`. In this
case, on x86-64 thu thunk names must be:
```
  __llvm_external_retpoline_r11
```
or on 32-bit:
```
  __llvm_external_retpoline_eax
  __llvm_external_retpoline_ecx
  __llvm_external_retpoline_edx
  __llvm_external_retpoline_push
```
And the target of the retpoline is passed in the named register, or in
the case of the `push` suffix on the top of the stack via a `pushl`
instruction.

There is one other important source of indirect branches in x86 ELF
binaries: the PLT. These patches also include support for LLD to
generate PLT entries that perform a retpoline-style indirection.

The only other indirect branches remaining that we are aware of are from
precompiled runtimes (such as crt0.o and similar). The ones we have
found are not really attackable, and so we have not focused on them
here, but eventually these runtimes should also be replicated for
retpoline-ed configurations for completeness.

For kernels or other freestanding or fully static executables, the
compiler switch `-mretpoline` is sufficient to fully mitigate this
particular attack. For dynamic executables, you must compile *all*
libraries with `-mretpoline` and additionally link the dynamic
executable and all shared libraries with LLD and pass `-z retpolineplt`
(or use similar functionality from some other linker). We strongly
recommend also using `-z now` as non-lazy binding allows the
retpoline-mitigated PLT to be substantially smaller.

When manually apply similar transformations to `-mretpoline` to the
Linux kernel we observed very small performance hits to applications
running typical workloads, and relatively minor hits (approximately 2%)
even for extremely syscall-heavy applications. This is largely due to
the small number of indirect branches that occur in performance
sensitive paths of the kernel.

When using these patches on statically linked applications, especially
C++ applications, you should expect to see a much more dramatic
performance hit. For microbenchmarks that are switch, indirect-, or
virtual-call heavy we have seen overheads ranging from 10% to 50%.

However, real-world workloads exhibit substantially lower performance
impact. Notably, techniques such as PGO and ThinLTO dramatically reduce
the impact of hot indirect calls (by speculatively promoting them to
direct calls) and allow optimized search trees to be used to lower
switches. If you need to deploy these techniques in C++ applications, we
*strongly* recommend that you ensure all hot call targets are statically
linked (avoiding PLT indirection) and use both PGO and ThinLTO. Well
tuned servers using all of these techniques saw 5% - 10% overhead from
the use of retpoline.

We will add detailed documentation covering these components in
subsequent patches, but wanted to make the core functionality available
as soon as possible. Happy for more code review, but we'd really like to
get these patches landed and backported ASAP for obvious reasons. We're
planning to backport this to both 6.0 and 5.0 release streams and get
a 5.0 release with just this cherry picked ASAP for distros and vendors.

This patch is the work of a number of people over the past month: Eric, Reid,
Rui, and myself. I'm mailing it out as a single commit due to the time
sensitive nature of landing this and the need to backport it. Huge thanks to
everyone who helped out here, and everyone at Intel who helped out in
discussions about how to craft this. Also, credit goes to Paul Turner (at
Google, but not an LLVM contributor) for much of the underlying retpoline
design.

Reviewers: echristo, rnk, ruiu, craig.topper, DavidKreitzer

Subscribers: sanjoy, emaste, mcrosier, mgorny, mehdi_amini, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D41723

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@323155 91177308-0d34-0410-b5e6-96231b3b80d8
2018-01-22 22:05:25 +00:00