69 Commits

Author SHA1 Message Date
Teresa Johnson
e07c2606ba [ThinLTO] Rename index IsAnalysis flag to HaveGVs (NFC)
With the upcoming patch to add summary parsing support, IsAnalysis would
be true in contexts where we are not performing module summary analysis.
Rename to the more specific and approprate HaveGVs, which is essentially
what this flag is indicating.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334140 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-06 22:22:01 +00:00
Benjamin Kramer
f4eac50c9d Extend the GlobalObject metadata interface
- Make eraseMetadata return whether it changed something
- Wire getMetadata for a single MDNode efficiently into the attachment
map
- Add hasMetadata, which is less weird than checking getMetadata ==
nullptr on a multimap.

Use it to simplify code.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333649 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-31 13:29:58 +00:00
Vitaly Buka
af98dad7c6 Fix warning by cl::opt<int> -> cl::opt<unsigned>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@329461 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-06 21:41:17 +00:00
Vitaly Buka
b67d096388 Runtime flag to control branch funnel threshold
Reviewers: pcc

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D45193

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@329459 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-06 21:32:36 +00:00
Peter Collingbourne
68d73644cd Use branch funnels for virtual calls when retpoline mitigation is enabled.
The retpoline mitigation for variant 2 of CVE-2017-5715 inhibits the
branch predictor, and as a result it can lead to a measurable loss of
performance. We can reduce the performance impact of retpolined virtual
calls by replacing them with a special construct known as a branch
funnel, which is an instruction sequence that implements virtual calls
to a set of known targets using a binary tree of direct branches. This
allows the processor to speculately execute valid implementations of the
virtual function without allowing for speculative execution of of calls
to arbitrary addresses.

This patch extends the whole-program devirtualization pass to replace
certain virtual calls with calls to branch funnels, which are
represented using a new llvm.icall.jumptable intrinsic. It also extends
the LowerTypeTests pass to recognize the new intrinsic, generate code
for the branch funnels (x86_64 only for now) and lay out virtual tables
as required for each branch funnel.

The implementation supports full LTO as well as ThinLTO, and extends the
ThinLTO summary format used for whole-program devirtualization to
support branch funnels.

For more details see RFC:
http://lists.llvm.org/pipermail/llvm-dev/2018-January/120672.html

Differential Revision: https://reviews.llvm.org/D42453

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@327163 91177308-0d34-0410-b5e6-96231b3b80d8
2018-03-09 19:11:44 +00:00
Eugene Leviant
731becc910 [ThinLTO] Re-commit of dot dumper after test fix
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@323116 91177308-0d34-0410-b5e6-96231b3b80d8
2018-01-22 13:35:40 +00:00
Eugene Leviant
93150c76d7 Temporarily revert r323062 to investigate buildbot failures
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@323065 91177308-0d34-0410-b5e6-96231b3b80d8
2018-01-21 10:22:19 +00:00
Eugene Leviant
7ec5710f32 [ThinLTO] Implement summary visualizer
Differential revision: https://reviews.llvm.org/D41297


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@323062 91177308-0d34-0410-b5e6-96231b3b80d8
2018-01-21 07:27:32 +00:00
Peter Collingbourne
0573715330 WholeProgramDevirt: Simplify ORE getter mechanism for old PM. NFCI.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@321841 91177308-0d34-0410-b5e6-96231b3b80d8
2018-01-05 00:27:51 +00:00
Adrian Prantl
c0ade050e0 Silence a bunch of implicit fallthrough warnings
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@321114 91177308-0d34-0410-b5e6-96231b3b80d8
2017-12-19 22:05:25 +00:00
Michael Zolotukhin
d5c7271167 Remove redundant includes from lib/Transforms.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@320628 91177308-0d34-0410-b5e6-96231b3b80d8
2017-12-13 21:31:01 +00:00
Adam Nemet
3b8950a6d6 Rename OptimizationDiagnosticInfo.* to OptimizationRemarkEmitter.*
Sync it up with the name of the class actually defined here.  This has been
bothering me for a while...

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315249 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-09 23:19:02 +00:00
Peter Collingbourne
37911b93a0 WholeProgramDevirt: Add import/export support for targets without absolute symbol constants.
Not all targets support the use of absolute symbols to export
constants. In particular, ARM has a wide variety of constant encodings
that cannot currently be relocated by linkers. So instead of exporting
the constants using symbols, export them directly in the summary.
The values of the constants are left as zeroes on targets that support
symbolic exports.

This may result in more cache misses when targeting those architectures
as a result of arbitrary changes in constant values, but this seems
somewhat unavoidable for now.

Differential Revision: https://reviews.llvm.org/D37407

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312967 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-11 22:34:42 +00:00
Peter Collingbourne
7480eb9933 WholeProgramDevirt: When promoting for single-impl devirt, also rename the comdat.
This is required when targeting COFF, as the comdat name must match
one of the names of the symbols in the comdat.

Differential Revision: https://reviews.llvm.org/D37550

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312767 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-08 00:10:53 +00:00
Peter Collingbourne
bb516bcc22 WholeProgramDevirt: Create bitcast to i8* at each virtual call site.
We can't reuse the llvm.assume instruction's bitcast because it may not
dominate every user of the vtable pointer.

Differential Revision: https://reviews.llvm.org/D36994

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311491 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-22 21:41:19 +00:00
Sam Elliott
6930fbfd6f Migrate WholeProgramDevirt to new Optimization Remark API
Summary:
This is an attempt to move WholeProgramDevirt to the new remark API.

https://bugs.llvm.org/show_bug.cgi?id=33793

Reviewers: anemet

Reviewed By: anemet

Subscribers: fhahn, llvm-commits

Differential Revision: https://reviews.llvm.org/D36943

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311352 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-21 16:57:21 +00:00
Chandler Carruth
e3e43d9d57 Sort the remaining #include lines in include/... and lib/....
I did this a long time ago with a janky python script, but now
clang-format has built-in support for this. I fed clang-format every
line with a #include and let it re-sort things according to the precise
LLVM rules for include ordering baked into clang-format these days.

I've reverted a number of files where the results of sorting includes
isn't healthy. Either places where we have legacy code relying on
particular include ordering (where possible, I'll fix these separately)
or where we have particular formatting around #include lines that
I didn't want to disturb in this patch.

This patch is *entirely* mechanical. If you get merge conflicts or
anything, just ignore the changes in this patch and run clang-format
over your #include lines in the files.

Sorry for any noise here, but it is important to keep these things
stable. I was seeing an increasing number of patches with irrelevant
re-ordering of #include lines because clang-format was used. This patch
at least isolates that churn, makes it easy to skip when resolving
conflicts, and gets us to a clean baseline (again).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@304787 91177308-0d34-0410-b5e6-96231b3b80d8
2017-06-06 11:49:48 +00:00
Peter Collingbourne
e611018a3f Re-apply r302108, "IR: Use pointers instead of GUIDs to represent edges in the module summary. NFCI."
with a fix for the clang backend.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@302176 91177308-0d34-0410-b5e6-96231b3b80d8
2017-05-04 18:03:25 +00:00
Eric Liu
1c442aa9b7 Revert "IR: Use pointers instead of GUIDs to represent edges in the module summary. NFCI."
This reverts commit r302108. This causes crash in clang bootstrap with LTO.

Contacted the auther in the original commit.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@302140 91177308-0d34-0410-b5e6-96231b3b80d8
2017-05-04 11:49:39 +00:00
Peter Collingbourne
df2206086c IR: Use pointers instead of GUIDs to represent edges in the module summary. NFCI.
When profiling a no-op incremental link of Chromium I found that the functions
computeImportForFunction and computeDeadSymbols were consuming roughly 10% of
the profile. The goal of this change is to improve the performance of those
functions by changing the map lookups that they were previously doing into
pointer dereferences.

This is achieved by changing the ValueInfo data structure to be a pointer to
an element of the global value map owned by ModuleSummaryIndex, and changing
reference lists in the GlobalValueSummary to hold ValueInfos instead of GUIDs.
This means that a ValueInfo will take a client directly to the summary list
for a given GUID.

Differential Revision: https://reviews.llvm.org/D32471

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@302108 91177308-0d34-0410-b5e6-96231b3b80d8
2017-05-04 03:36:16 +00:00
Serge Guelton
9d54400bba Module::getOrInsertFunction is using C-style vararg instead of variadic templates.
From a user prospective, it forces the use of an annoying nullptr to mark the end of the vararg, and there's not type checking on the arguments.
The variadic template is an obvious solution to both issues.

Differential Revision: https://reviews.llvm.org/D31070



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@299949 91177308-0d34-0410-b5e6-96231b3b80d8
2017-04-11 15:01:18 +00:00
Diana Picus
1d02724c71 Revert "Turn some C-style vararg into variadic templates"
This reverts commit r299925 because it broke the buildbots. See e.g.
http://lab.llvm.org:8011/builders/clang-cmake-armv7-a15/builds/6008

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@299928 91177308-0d34-0410-b5e6-96231b3b80d8
2017-04-11 10:07:12 +00:00
Serge Guelton
ec124b3a6f Turn some C-style vararg into variadic templates
Module::getOrInsertFunction is using C-style vararg instead of
variadic templates.

From a user prospective, it forces the use of an annoying nullptr
to mark the end of the vararg, and there's not type checking on the
arguments. The variadic template is an obvious solution to both
issues.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@299925 91177308-0d34-0410-b5e6-96231b3b80d8
2017-04-11 08:36:52 +00:00
Mehdi Amini
8701bbc75d Revert "Turn some C-style vararg into variadic templates"
This reverts commit r299699, the examples needs to be updated.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@299702 91177308-0d34-0410-b5e6-96231b3b80d8
2017-04-06 20:23:57 +00:00
Mehdi Amini
753bd2a772 Turn some C-style vararg into variadic templates
Module::getOrInsertFunction is using C-style vararg instead of
variadic templates.

From a user prospective, it forces the use of an annoying nullptr
to mark the end of the vararg, and there's not type checking on the
arguments. The variadic template is an obvious solution to both
issues.

Patch by: Serge Guelton <serge.guelton@telecom-bretagne.eu>

Differential Revision: https://reviews.llvm.org/D31070

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@299699 91177308-0d34-0410-b5e6-96231b3b80d8
2017-04-06 20:09:31 +00:00
Peter Collingbourne
e53e585ce9 IPO: Const correctness for summaries passed into passes.
Pass const qualified summaries into importers and unqualified summaries into
exporters. This lets us const-qualify the summary argument to thinBackend.

Differential Revision: https://reviews.llvm.org/D31230

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@298534 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-22 18:22:59 +00:00
Peter Collingbourne
49df148534 IR: Fix a race condition in type id clients of ModuleSummaryIndex.
Add a const version of the getTypeIdSummary accessor that avoids
mutating the TypeIdMap.

Differential Revision: https://reviews.llvm.org/D31226

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@298531 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-22 18:04:39 +00:00
Peter Collingbourne
bfecb4640b WholeProgramDevirt: Implement export/import support for VCP.
Differential Revision: https://reviews.llvm.org/D30017

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@297503 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-10 20:13:58 +00:00
Peter Collingbourne
eb3c7034f3 WholeProgramDevirt: Implement export/import support for unique ret val opt.
Differential Revision: https://reviews.llvm.org/D29917

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@297502 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-10 20:09:11 +00:00
George Rimar
a312eded5f WholeProgramDevirt: Fixed compilation error under MSVS2015.
It was introduced in:

r296945
WholeProgramDevirt: Implement exporting for single-impl devirtualization.
---------------------
r296939
WholeProgramDevirt: Add any unsuccessful llvm.type.checked.load devirtualizations to the list of llvm.type.test users.
---------------------

Microsoft Visual Studio Community 2015
Version 14.0.23107.0 D14REL
Does not compile that code without additional brackets, showing multiple error like below:

WholeProgramDevirt.cpp(1216): error C2958: the left bracket '[' found at 'c:\access_softek\llvm\lib\transforms\ipo\wholeprogramdevirt.cpp(1216)' was not matched correctly
WholeProgramDevirt.cpp(1216): error C2143: syntax error: missing ']' before '}'
WholeProgramDevirt.cpp(1216): error C2143: syntax error: missing ';' before '}'
WholeProgramDevirt.cpp(1216): error C2059: syntax error: ']'


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@297451 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-10 10:31:56 +00:00
Peter Collingbourne
1166572ca3 WholeProgramDevirt: Implement importing for uniform ret val opt.
Differential Revision: https://reviews.llvm.org/D29854

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@297350 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-09 01:11:15 +00:00
Peter Collingbourne
0e534009f0 WholeProgramDevirt: Implement importing for single-impl devirtualization.
Differential Revision: https://reviews.llvm.org/D29844

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@297333 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-09 00:21:25 +00:00
Peter Collingbourne
708709f2dc Fix build.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296949 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-04 01:38:05 +00:00
Peter Collingbourne
3f44bcbdcb WholeProgramDevirt: Implement exporting for uniform ret val opt.
Differential Revision: https://reviews.llvm.org/D29846

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296948 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-04 01:34:53 +00:00
Peter Collingbourne
d1e011ddda WholeProgramDevirt: Implement exporting for single-impl devirtualization.
Differential Revision: https://reviews.llvm.org/D29811

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296945 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-04 01:31:01 +00:00
Peter Collingbourne
d31e3edb6f WholeProgramDevirt: Add any unsuccessful llvm.type.checked.load devirtualizations to the list of llvm.type.test users.
Any unsuccessful llvm.type.checked.load devirtualizations will be translated
into uses of llvm.type.test, so we need to add the resulting llvm.type.test
intrinsics to the function summaries so that the LowerTypeTests pass will
export them.

Differential Revision: https://reviews.llvm.org/D29808

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296939 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-04 01:23:30 +00:00
Adam Nemet
0f085888b6 [OptDiag] Hide legacy remark ctors
These are only used when emitting remarks without ORE directly using the free
functions emitOptimizationRemark*.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296037 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-23 23:11:11 +00:00
Justin Bogner
4be00b7013 OptDiag: Allow constructing DiagnosticLocation from DISubprograms
This avoids creating a DILocation just to represent a line number,
since creating Metadata is expensive. Creating a DiagnosticLocation
directly is much cheaper.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295531 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-18 02:00:27 +00:00
Peter Collingbourne
063d4cbbef WholeProgramDevirt: For VCP use a 32-bit ConstantInt for the byte offset.
A future change will cause this byte offset to be inttoptr'd and then exported
via an absolute symbol. On the importing end we will expect the symbol to be
in range [0,2^32) so that it will fit into a 32-bit relocation. The problem
is that on 64-bit architectures if the offset is negative it will not be in
the correct range once we inttoptr it.

This change causes us to use a 32-bit integer so that it can be inttoptr'd
(which zero extends) into the correct range.

Differential Revision: https://reviews.llvm.org/D30016

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295487 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-17 19:43:45 +00:00
Peter Collingbourne
72258b42b0 WholeProgramDevirt: Examine the function body when deciding whether functions are readnone.
The goal is to get an analysis result even for de-refineable functions.

Differential Revision: https://reviews.llvm.org/D29803

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295472 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-17 18:17:04 +00:00
Peter Collingbourne
33a52c8849 Re-apply r295110 and r295144 with a fix for the ASan issue.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295241 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-15 21:56:51 +00:00
Daniel Jasper
96670d1a33 Revert r295110 and r295144.
This fails under ASAN:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/builds/798/steps/check-llvm%20asan/logs/stdio

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295162 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-15 09:56:08 +00:00
Peter Collingbourne
a7968ae5b5 WholeProgramDevirt: Separate the code that applies optzns from the code that decides whether to apply them. NFCI.
The idea is that the apply* functions will also be called when importing
devirt optimizations.

Differential Revision: https://reviews.llvm.org/D29745

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295144 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-15 02:13:08 +00:00
Peter Collingbourne
f2e64c5579 WholeProgramDevirt: Change internal vcall data structures to match summary.
Group calls into constant and non-constant arguments up front, and use uint64_t
instead of ConstantInt to represent constant arguments. The goal is to allow
the information from the summary to fit naturally into this data structure in
a future change (specifically, it will be added to CallSiteInfo).

This has two side effects:
- We disallow VCP for constant integer arguments of width >64 bits.
- We remove the restriction that the bitwidth of a vcall's argument and return
  types must match those of the vfunc definitions.
I don't expect either of these to matter in practice. The first case is
uncommon, and the second one will lead to UB (so we can do anything we like).

Differential Revision: https://reviews.llvm.org/D29744

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295110 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-14 22:12:23 +00:00
Peter Collingbourne
a5035323ac IR: Type ID summary extensions for WPD; thread summary into WPD pass.
Make the whole thing testable by adding YAML I/O support for the WPD
summary information and adding some negative tests that exercise the
YAML support.

Differential Revision: https://reviews.llvm.org/D29782

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@294981 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-13 19:26:18 +00:00
Peter Collingbourne
ae2612e12e WholeProgramDevirt: Check that VCP candidate functions are defined before evaluating them.
This was crashing before.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@294666 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-09 23:46:26 +00:00
Peter Collingbourne
c8e39ef801 WholeProgramDevirt: Teach the pass to handle structs of arrays.
This will become necessary in some cases once D22296 lands.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289165 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-09 01:10:11 +00:00
Peter Collingbourne
5ecad331e4 Make WholeProgramDevirt understand ConstStruct vtables.
Based on a patch by LemonBoy!

Differential Revision: https://reviews.llvm.org/D26581

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289162 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-09 00:33:27 +00:00
Adam Nemet
0552c8c45a Shorten DiagnosticInfoOptimizationRemark* to OptimizationRemark*. NFC
With the new streaming interface, these class names need to be typed a
lot and it's way too looong.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282544 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-27 22:19:23 +00:00
Ivan Krasin
d79f1f1720 WholeProgramDevirt: initialize WasDevirt in all constructors.
Summary: This is a follow up to r278389 and r278442.

Differential Revision: https://reviews.llvm.org/D23438

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278455 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-12 01:40:10 +00:00