Commit Graph

652 Commits

Author SHA1 Message Date
Francis Visoiu Mistrih
1cb383ef62 [Remarks] Extend -fsave-optimization-record to specify the format
Use -fsave-optimization-record=<format> to specify a different format
than the default, which is YAML.

For now, only YAML is supported.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363573 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-17 16:06:00 +00:00
Aaron Puchert
42e2da74cf [Clang] Harmonize Split DWARF options with llc
Summary:
With Split DWARF the resulting object file (then called skeleton CU)
contains the file name of another ("DWO") file with the debug info.
This can be a problem for remote compilation, as it will contain the
name of the file on the compilation server, not on the client.

To use Split DWARF with remote compilation, one needs to either

* make sure only relative paths are used, and mirror the build directory
  structure of the client on the server,
* inject the desired file name on the client directly.

Since llc already supports the latter solution, we're just copying that
over. We allow setting the actual output filename separately from the
value of the DW_AT_[GNU_]dwo_name attribute in the skeleton CU.

Fixes PR40276.

Reviewers: dblaikie, echristo, tejohnson

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D59673

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363496 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-15 15:38:51 +00:00
Francis Visoiu Mistrih
6058aa9ef5 Reland: [Remarks] Refactor optimization remarks setup
* Add a common function to setup opt-remarks
* Rename common options to the same names
* Add error types to distinguish between file errors and regex errors

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363415 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-14 16:20:51 +00:00
Francis Visoiu Mistrih
0f00485ae2 Revert "[Remarks] Refactor optimization remarks setup"
This reverts commit 6e6e3af55bb97e1a4c97375c15a2b0099120c5a7.

This breaks greendragon.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363343 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-14 00:05:56 +00:00
Francis Visoiu Mistrih
eb1d34aaa7 [Remarks] Refactor optimization remarks setup
* Add a common function to setup opt-remarks
* Rename common options to the same names
* Add error types to distinguish between file errors and regex errors

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363328 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-13 21:46:57 +00:00
Ben Dunbobbin
c543215de3 [ThinLTO]LTO]Legacy] Fix dependent libraries support by adding querying of the IRSymtab
Dependent libraries support for the legacy api was committed in a
broken state (see: https://reviews.llvm.org/D60274). This was missed
due to the painful nature of having to integrate the changes into a
linker in order to test. This change implements support for dependent
libraries in the legacy LTO api:

- I have removed the current api function, which returns a single
string, and   added functions to access each dependent library
specifier individually.

- To reduce the testing pain, I have made the api functions as thin as
possible to   maximize coverage from llvm-lto.

- When doing ThinLTO the system linker will load the modules lazily
when scanning   the input files. Unfortunately, when modules are
lazily loaded there is no access   to module level named metadata. To
fix this I have added api functions that allow   querying the IRSymtab
for the dependent libraries. I hope to expand the api in the   future
so that, eventually, all the information needed by a client linker
during   scan can be retrieved from the IRSymtab.

Differential Revision: https://reviews.llvm.org/D62935

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363140 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-12 11:07:56 +00:00
Johannes Doerfert
f64487e26f [Attributor] Pass infrastructure and fixpoint framework
NOTE: Note that no attributes are derived yet. This patch will not go in
      alone but only with others that derive attributes. The framework is
      split for review purposes.

This commit introduces the Attributor pass infrastructure and fixpoint
iteration framework. Further patches will introduce abstract attributes
into this framework.

In a nutshell, the Attributor will update instances of abstract
arguments until a fixpoint, or a "timeout", is reached. Communication
between the Attributor and the abstract attributes that are derived is
restricted to the AbstractState and AbstractAttribute interfaces.

Please see the file comment in Attributor.h for detailed information
including design decisions and typical use case. Also consider the class
documentation for Attributor, AbstractState, and AbstractAttribute.

Reviewers: chandlerc, homerdin, hfinkel, fedor.sergeev, sanjoy, spatel, nlopes, nicholas, reames

Subscribers: mehdi_amini, mgorny, hiraditya, bollu, steven_wu, dexonsmith, dang, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59918

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@362578 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-05 03:02:24 +00:00
Sam Clegg
15927a9c4b Fix -DBUILD_SHARED_LIBS=ON build after rL362160
Differential Revision: https://reviews.llvm.org/D62709

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@362180 91177308-0d34-0410-b5e6-96231b3b80d8
2019-05-31 01:04:00 +00:00
Francis Visoiu Mistrih
890aba409a [Remarks][NFC] Move the serialization to lib/Remarks
Separate the remark serialization to YAML from the LLVM Diagnostics.

This adds a new serialization abstraction: remarks::Serializer. It's
completely independent from lib/IR and it provides an easy way to
replace YAML by providing a new remarks::Serializer.

Differential Revision: https://reviews.llvm.org/D62632

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@362160 91177308-0d34-0410-b5e6-96231b3b80d8
2019-05-30 21:45:59 +00:00
Ben Dunbobbin
7398f4a5f8 [ELF] Implement Dependent Libraries Feature
This patch implements a limited form of autolinking primarily designed to allow
either the --dependent-library compiler option, or "comment lib" pragmas (
https://docs.microsoft.com/en-us/cpp/preprocessor/comment-c-cpp?view=vs-2017) in
C/C++ e.g. #pragma comment(lib, "foo"), to cause an ELF linker to automatically
add the specified library to the link when processing the input file generated
by the compiler.

Currently this extension is unique to LLVM and LLD. However, care has been taken
to design this feature so that it could be supported by other ELF linkers.

The design goals were to provide:

- A simple linking model for developers to reason about.
- The ability to to override autolinking from the linker command line.
- Source code compatibility, where possible, with "comment lib" pragmas in other
  environments (MSVC in particular).

Dependent library support is implemented differently for ELF platforms than on
the other platforms. Primarily this difference is that on ELF we pass the
dependent library specifiers directly to the linker without manipulating them.
This is in contrast to other platforms where they are mapped to a specific
linker option by the compiler. This difference is a result of the greater
variety of ELF linkers and the fact that ELF linkers tend to handle libraries in
a more complicated fashion than on other platforms. This forces us to defer
handling the specifiers to the linker.

In order to achieve a level of source code compatibility with other platforms
we have restricted this feature to work with libraries that meet the following
"reasonable" requirements:

1. There are no competing defined symbols in a given set of libraries, or
   if they exist, the program owner doesn't care which is linked to their
   program.
2. There may be circular dependencies between libraries.

The binary representation is a mergeable string section (SHF_MERGE,
SHF_STRINGS), called .deplibs, with custom type SHT_LLVM_DEPENDENT_LIBRARIES
(0x6fff4c04). The compiler forms this section by concatenating the arguments of
the "comment lib" pragmas and --dependent-library options in the order they are
encountered. Partial (-r, -Ur) links are handled by concatenating .deplibs
sections with the normal mergeable string section rules. As an example, #pragma
comment(lib, "foo") would result in:

.section ".deplibs","MS",@llvm_dependent_libraries,1
         .asciz "foo"

For LTO, equivalent information to the contents of a the .deplibs section can be
retrieved by the LLD for bitcode input files.

LLD processes the dependent library specifiers in the following way:

1. Dependent libraries which are found from the specifiers in .deplibs sections
   of relocatable object files are added when the linker decides to include that
   file (which could itself be in a library) in the link. Dependent libraries
   behave as if they were appended to the command line after all other options. As
   a consequence the set of dependent libraries are searched last to resolve
   symbols.
2. It is an error if a file cannot be found for a given specifier.
3. Any command line options in effect at the end of the command line parsing apply
   to the dependent libraries, e.g. --whole-archive.
4. The linker tries to add a library or relocatable object file from each of the
   strings in a .deplibs section by; first, handling the string as if it was
   specified on the command line; second, by looking for the string in each of the
   library search paths in turn; third, by looking for a lib<string>.a or
   lib<string>.so (depending on the current mode of the linker) in each of the
   library search paths.
5. A new command line option --no-dependent-libraries tells LLD to ignore the
   dependent libraries.

Rationale for the above points:

1. Adding the dependent libraries last makes the process simple to understand
   from a developers perspective. All linkers are able to implement this scheme.
2. Error-ing for libraries that are not found seems like better behavior than
   failing the link during symbol resolution.
3. It seems useful for the user to be able to apply command line options which
   will affect all of the dependent libraries. There is a potential problem of
   surprise for developers, who might not realize that these options would apply
   to these "invisible" input files; however, despite the potential for surprise,
   this is easy for developers to reason about and gives developers the control
   that they may require.
4. This algorithm takes into account all of the different ways that ELF linkers
   find input files. The different search methods are tried by the linker in most
   obvious to least obvious order.
5. I considered adding finer grained control over which dependent libraries were
   ignored (e.g. MSVC has /nodefaultlib:<library>); however, I concluded that this
   is not necessary: if finer control is required developers can fall back to using
   the command line directly.

RFC thread: http://lists.llvm.org/pipermail/llvm-dev/2019-March/131004.html.

Differential Revision: https://reviews.llvm.org/D60274

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@360984 91177308-0d34-0410-b5e6-96231b3b80d8
2019-05-17 03:44:15 +00:00
Eugene Leviant
fd1c3442d9 [ThinLTO] Don't internalize weak writeable variables
Variables with linkonce_odr and weak_odr linkage shouldn't be internalized
if they're not readonly. Otherwise we may end up with multiple copies of
such variable, so reads and writes will become inconsistent

Differential revision: https://reviews.llvm.org/D61255


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@360577 91177308-0d34-0410-b5e6-96231b3b80d8
2019-05-13 11:53:05 +00:00
Teresa Johnson
9488ca4c3f [ThinLTO] Auto-hide prevailing linkonce_odr only when all copies eligible
Summary:
We hit undefined references building with ThinLTO when one source file
contained explicit instantiations of a template method (weak_odr) but
there were also implicit instantiations in another file (linkonce_odr),
and the latter was the prevailing copy. In this case the symbol was
marked hidden when the prevailing linkonce_odr copy was promoted to
weak_odr. It led to unsats when the resulting shared library was linked
with other code that contained a reference (expecting to be resolved due
to the explicit instantiation).

Add a CanAutoHide flag to the GV summary to allow the thin link to
identify when all copies are eligible for auto-hiding (because they were
all originally linkonce_odr global unnamed addr), and only do the
auto-hide in that case.

Most of the changes here are due to plumbing the new flag through the
bitcode and llvm assembly, and resulting test changes. I augmented the
existing auto-hide test to check for this situation.

Reviewers: pcc

Subscribers: mehdi_amini, inglorion, eraman, dexonsmith, arphaman, dang, llvm-commits, steven_wu, wmi

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59709

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@360466 91177308-0d34-0410-b5e6-96231b3b80d8
2019-05-10 20:08:24 +00:00
Steven Wu
036707485e [ThinLTO] Adding architecture name into saved object filename
Summary:
For ThinLTOCodegenerator, it has an option to save the object file
outputs into a directory which is essential for debug info. Tools like lldb
and dsymutil will look for these object files for debug info.

On Darwin platform, you can link fat binaries with one single clang
driver invocation like:
 $ clang -arch x86_64 -arch i386 -Wl,-object_path_lto,$TMPDIR ...
Unfornately, the output object files for one architecture is going to
overwrite the previous ones and one architecture slice will end up with
no debug info. One example for this is to turn on ThinLTO for sanitizer
dylibs in compiler-rt project.

To fix the issue, add the name for the architecture into the name of the
output object file.

rdar://problem/35482935

Reviewers: tejohnson, bd1976llvm, dexonsmith, JDevlieghere

Reviewed By: dexonsmith

Subscribers: mehdi_amini, aprantl, inglorion, eraman, hiraditya, jkorous, dang, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60924

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@359508 91177308-0d34-0410-b5e6-96231b3b80d8
2019-04-29 21:39:54 +00:00
Alina Sbirlea
a837603c47 [NewPassManager] Adding pass tuning options: loop vectorize.
Summary:
Trying to add the plumbing necessary to add tuning options to the new pass manager.
Testing with the flags for loop vectorize.

Reviewers: chandlerc

Subscribers: sanjoy, mehdi_amini, jlebar, steven_wu, dexonsmith, dang, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59723

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358763 91177308-0d34-0410-b5e6-96231b3b80d8
2019-04-19 16:11:59 +00:00
Florian Hahn
79b3f9b308 [LTO] Add plumbing to save stats during LTO on Darwin.
Gold and ld on Linux already support saving stats, but the
infrastructure is missing on Darwin. Unfortunately it seems like the
configuration from lib/LTO/LTO.cpp is not used.

This patch adds a new LTOStatsFile option and adds plumbing in Clang to
use it on Darwin, similar to the way remarks are handled.

Currnetly the handling of LTO flags seems quite spread out, with a bunch
of duplication. But I am not sure if there is an easy way to improve
that?

Reviewers: anemet, tejohnson, thegameg, steven_wu

Reviewed By: steven_wu

Differential Revision: https://reviews.llvm.org/D60516

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358753 91177308-0d34-0410-b5e6-96231b3b80d8
2019-04-19 12:36:41 +00:00
Steven Wu
f8c4f81342 [ThinLTO] Fix ThinLTOCodegenerator to export llvm.used symbols
Summary:
Reapply r357931 with fixes to ThinLTO testcases and llvm-lto tool.

ThinLTOCodeGenerator currently does not preserve llvm.used symbols and
it can internalize them. In order to pass the necessary information to the
legacy ThinLTOCodeGenerator, the input to the code generator is
rewritten to be based on lto::InputFile.

Now ThinLTO using the legacy LTO API will requires data layout in
Module.

"internalize" thinlto action in llvm-lto is updated to run both
"promote" and "internalize" with the same configuration as
ThinLTOCodeGenerator. The old "promote" + "internalize" option does not
produce the same output as ThinLTOCodeGenerator.

This fixes: PR41236
rdar://problem/49293439

Reviewers: tejohnson, pcc, kromanova, dexonsmith

Reviewed By: tejohnson

Subscribers: ormris, bd1976llvm, mehdi_amini, inglorion, eraman, hiraditya, jkorous, dexonsmith, arphaman, dang, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60421

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@358601 91177308-0d34-0410-b5e6-96231b3b80d8
2019-04-17 17:38:09 +00:00
Steven Wu
65881148e9 Revert [ThinLTO] Fix ThinLTOCodegenerator to export llvm.used symbols
This reverts r357931 (git commit 8b70a5c11e08116955a875b9085433f14737bcaf)

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@357932 91177308-0d34-0410-b5e6-96231b3b80d8
2019-04-08 18:53:21 +00:00
Steven Wu
13456d67c9 [ThinLTO] Fix ThinLTOCodegenerator to export llvm.used symbols
Summary:
ThinLTOCodeGenerator currently does not preserve llvm.used symbols and
it can internalize them. In order to pass the necessary information to the
legacy ThinLTOCodeGenerator, the input to the code generator is
rewritten to be based on lto::InputFile.

This fixes: PR41236
rdar://problem/49293439

Reviewers: tejohnson, pcc, dexonsmith

Reviewed By: tejohnson

Subscribers: mehdi_amini, inglorion, eraman, hiraditya, jkorous, dang, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60226

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@357931 91177308-0d34-0410-b5e6-96231b3b80d8
2019-04-08 18:24:10 +00:00
Francis Visoiu Mistrih
1e252811d6 Reland "[Remarks] Add -foptimization-record-passes to filter remark emission"
Currently we have -Rpass for filtering the remarks that are displayed as
diagnostics, but when using -fsave-optimization-record, there is no way
to filter the remarks while generating them.

This adds support for filtering remarks by passes using a regex.
Ex: `clang -fsave-optimization-record -foptimization-record-passes=inline`

will only emit the remarks coming from the pass `inline`.

This adds:

* `-fsave-optimization-record` to the driver
* `-opt-record-passes` to cc1
* `-lto-pass-remarks-filter` to the LTOCodeGenerator
* `--opt-remarks-passes` to lld
* `-pass-remarks-filter` to llc, opt, llvm-lto, llvm-lto2
* `-opt-remarks-passes` to gold-plugin

Differential Revision: https://reviews.llvm.org/D59268

Original llvm-svn: 355964

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355984 91177308-0d34-0410-b5e6-96231b3b80d8
2019-03-12 21:22:27 +00:00
Francis Visoiu Mistrih
a826459b5e Revert "[Remarks] Add -foptimization-record-passes to filter remark emission"
This reverts commit 20fff32b7d1f1a1bd417b22aa9f26ededd97a3e5.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355976 91177308-0d34-0410-b5e6-96231b3b80d8
2019-03-12 20:54:18 +00:00
Francis Visoiu Mistrih
1bd774409d [Remarks] Add -foptimization-record-passes to filter remark emission
Currently we have -Rpass for filtering the remarks that are displayed as
diagnostics, but when using -fsave-optimization-record, there is no way
to filter the remarks while generating them.

This adds support for filtering remarks by passes using a regex.
Ex: `clang -fsave-optimization-record -foptimization-record-passes=inline`

will only emit the remarks coming from the pass `inline`.

This adds:

* `-fsave-optimization-record` to the driver
* `-opt-record-passes` to cc1
* `-lto-pass-remarks-filter` to the LTOCodeGenerator
* `--opt-remarks-passes` to lld
* `-pass-remarks-filter` to llc, opt, llvm-lto, llvm-lto2
* `-opt-remarks-passes` to gold-plugin

Differential Revision: https://reviews.llvm.org/D59268

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355964 91177308-0d34-0410-b5e6-96231b3b80d8
2019-03-12 20:28:50 +00:00
Francis Visoiu Mistrih
78ee34a412 Reland "[Remarks] Refactor remark diagnostic emission in a RemarkStreamer"
This allows us to store more info about where we're emitting the remarks
without cluttering LLVMContext. This is needed for future support for
the remark section.

Differential Revision: https://reviews.llvm.org/D58996

Original llvm-svn: 355507

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355514 91177308-0d34-0410-b5e6-96231b3b80d8
2019-03-06 15:20:13 +00:00
Francis Visoiu Mistrih
4107715f75 Revert "[Remarks] Refactor remark diagnostic emission in a RemarkStreamer"
This reverts commit 2e8c4997a2089f8228c843fd81b148d903472e02.

Breaks bots.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355511 91177308-0d34-0410-b5e6-96231b3b80d8
2019-03-06 14:52:37 +00:00
Francis Visoiu Mistrih
aa137aa9ea [Remarks] Refactor remark diagnostic emission in a RemarkStreamer
This allows us to store more info about where we're emitting the remarks
without cluttering LLVMContext. This is needed for future support for
the remark section.

Differential Revision: https://reviews.llvm.org/D58996

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355507 91177308-0d34-0410-b5e6-96231b3b80d8
2019-03-06 14:32:08 +00:00
Rong Xu
0d5bf69c9c [PGO] Context sensitive PGO (part 3)
Part 3 of CSPGO changes (mostly related to PassMananger).

Differential Revision: https://reviews.llvm.org/D54175


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355330 91177308-0d34-0410-b5e6-96231b3b80d8
2019-03-04 20:21:27 +00:00
Teresa Johnson
c415237177 [ThinLTO] Detect partially split modules during the thin link
Summary:
The changes to disable LTO unit splitting by default (r350949) and
detect inconsistently split LTO units (r350948) are causing some crashes
when the inconsistency is detected in multiple threads simultaneously.
Fix that by having the code always look for the inconsistently split
LTO units during the thin link, by checking for the presence of type
tests recorded in the summaries.

Modify test added in r350948 to remove single threading required to fix
a bot failure due to this issue (and some debugging options added in the
process of diagnosing it).

Reviewers: pcc

Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D57561

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354062 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-14 21:22:50 +00:00
Chandler Carruth
6b547686c5 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@351636 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-19 08:50:56 +00:00
Teresa Johnson
e39377580b [LTO] Record whether LTOUnit splitting is enabled in index
Summary:
Records in the module summary index whether the bitcode was compiled
with the option necessary to enable splitting the LTO unit
(e.g. -fsanitize=cfi, -fwhole-program-vtables, or -fsplit-lto-unit).

The information is passed down to the ModuleSummaryIndex builder via a
new module flag "EnableSplitLTOUnit", which is propagated onto a flag
on the summary index.

This is then used during the LTO link to check whether all linked
summaries were built with the same value of this flag. If not, an error
is issued when we detect a situation requiring whole program visibility
of the class hierarchy. This is the case when both of the following
conditions are met:
1) We are performing LowerTypeTests or Whole Program Devirtualization.
2) There are type tests or type checked loads in the code.

Note I have also changed the ThinLTOBitcodeWriter to also gate the
module splitting on the value of this flag.

Reviewers: pcc

Subscribers: ormris, mehdi_amini, Prazek, inglorion, eraman, steven_wu, dexonsmith, arphaman, dang, llvm-commits

Differential Revision: https://reviews.llvm.org/D53890

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@350948 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-11 18:31:57 +00:00
Easwaran Raman
116e08026c Refactor synthetic profile count computation. NFC.
Summary:
Instead of using two separate callbacks to return the entry count and the
relative block frequency, use a single callback to return callsite
count. This would allow better supporting hybrid mode in the future as
the count of callsite need not always be derived from entry count (as in
sample PGO).

Reviewers: davidxl

Subscribers: mehdi_amini, steven_wu, dexonsmith, dang, llvm-commits

Differential Revision: https://reviews.llvm.org/D56464

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@350755 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-09 20:10:27 +00:00
Matthew Voss
dc8dd58c4c [ThinLTO] Remove dllimport attribute from locally defined symbols
Summary:
The LTO/ThinLTO driver currently creates invalid bitcode by setting 
symbols marked dllimport as dso_local. The compiler often has access 
to the definition (often dllexport) and the declaration (often 
dllimport) of an object at link-time, leading to a conflicting 
declaration. This patch resolves the inconsistency by removing the
dllimport attribute.

Reviewers: tejohnson, pcc, rnk, echristo

Reviewed By: rnk

Subscribers: dmikulin, wristow, mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, dang, llvm-commits

Differential Revision: https://reviews.llvm.org/D55627

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@349667 91177308-0d34-0410-b5e6-96231b3b80d8
2018-12-19 19:07:45 +00:00
Easwaran Raman
f1f1adc004 [ThinLTO] Compute synthetic function entry count
Summary:
This patch computes the synthetic function entry count on the whole
program callgraph (based on module summary) and writes the entry counts
to the summary. After function importing, this count gets attached to
the IR as metadata. Since it adds a new field to the summary, this bumps
up the version.

Reviewers: tejohnson

Subscribers: mehdi_amini, inglorion, llvm-commits

Differential Revision: https://reviews.llvm.org/D43521

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@349076 91177308-0d34-0410-b5e6-96231b3b80d8
2018-12-13 19:54:27 +00:00
Peter Collingbourne
2e4508a5ad LTO: Don't internalize available_externally globals.
This breaks C and C++ semantics because it can cause the address
of the global inside the module to differ from the address outside
of the module.

Differential Revision: https://reviews.llvm.org/D55237

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@348321 91177308-0d34-0410-b5e6-96231b3b80d8
2018-12-05 00:09:36 +00:00
George Burgess IV
3982fbc2ec [ThinLTO] Look through aliases when computing hash keys
Without this, we don't consider types used by aliasees in our cache key.
This caused issues when using the same cache for thin-linking the same
TU with different sets of virtual call candidates for a virtual call
inside of a constructor. That's sort of a mouthful. :)

Differential Revision: https://reviews.llvm.org/D55060


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@348216 91177308-0d34-0410-b5e6-96231b3b80d8
2018-12-04 00:02:33 +00:00
Teresa Johnson
9a1f991fa8 [ThinLTO] Import local variables from the same module as caller
Summary:
We can sometimes end up with multiple copies of a local variable that
have the same GUID in the index. This happens when there are local
variables with the same name that are in different source files having the
same name/path at compile time (but compiled into different bitcode objects).

In this case make sure we import the copy in the caller's module.
This enables importing both of the variables having the same GUID
(but which will have different promoted names since the module paths,
and therefore the module hashes, will be distinct).

Importing the wrong copy is particularly problematic for read only
variables, since we must import them as a local copy whenever
referenced. Otherwise we get undefs at link time.

Note that the llvm-lto.cpp and ThinLTOCodeGenerator changes are needed
for testing the distributed index case via clang, which will be sent as
a separate clang-side patch shortly. We were previously not doing the
dead code/read only computation before computing imports when testing
distributed index generation (like it was for testing importing and
other ThinLTO mechanisms alone).

Reviewers: evgeny777

Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, dang, llvm-commits

Differential Revision: https://reviews.llvm.org/D55047

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@347886 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-29 17:02:42 +00:00
Teresa Johnson
e7e4f1f171 [ThinLTO] Consolidate cache key computation between new/old LTO APIs
Summary:
The old legacy LTO API had a separate cache key computation, which was
a subset of the cache key computation in the new LTO API (from what I
can tell this is largely just because certain features such as CFI,
dsoLocal, etc are only utilized via the new LTO API). However, having
separate computations is unnecessary (much of the code is duplicated),
and can lead to bugs when adding new optimizations if both cache
computation algorithms aren't updated properly - it's much easier to
maintain if we have a single facility.

This patch refactors the old LTO API code to use the cache key
computation from the new LTO API. To do this, we set up an lto::Config
object and fill in the fields that the old LTO was hashing (the others
will just use the defaults).

There are two notable changes:
- I added a Freestanding flag to the LTO Config. Currently this is only
used by the legacy LTO API. In the patch that added it (D30791) I had
asked about adding it to the new LTO API, but it looks like that was not
addressed. This should probably be discussed as a follow up to this
change, as it is orthogonal.
- The legacy LTO API had some code that was hashing the GUID of all
preserved symbols defined in the module. I looked back at the history of
this (which was added with the original hashing in the legacy LTO API in
D18494), and there is a comment in the review thread that it was added
in preparation for future internalization. We now do the internalization
of course, and that is handled in the new LTO API cache key computation
by hashing the recorded linkage type of all defined globals. Therefore I
didn't try to move over and keep the preserved symbols handling.

Reviewers: steven_wu, pcc

Subscribers: mehdi_amini, inglorion, eraman, dexonsmith, dang, llvm-commits

Differential Revision: https://reviews.llvm.org/D54635

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@347592 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-26 20:40:37 +00:00
Eugene Leviant
a3f1de9121 [ThinLTO] Internalize readonly globals
An attempt to recommit r346584 after failure on OSX build bot.
Fixed cache key computation in ThinLTOCodeGenerator and added
test case


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@347033 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-16 07:08:00 +00:00
Steven Wu
e0fbc0969c Revert "[ThinLTO] Internalize readonly globals"
This reverts commit 10c84a8f35.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@346768 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-13 17:35:04 +00:00
Jonas Devlieghere
686dfe3676 [Support] Make error banner optional in logAllUnhandledErrors
In a lot of places an empty string was passed as the ErrorBanner to
logAllUnhandledErrors. This patch makes that argument optional to
simplify the call sites.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@346604 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-11 01:46:03 +00:00
Eugene Leviant
10c84a8f35 [ThinLTO] Internalize readonly globals
This patch allows internalising globals if all accesses to them
(from live functions) are from non-volatile load instructions

Differential revision: https://reviews.llvm.org/D49362


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@346584 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-10 08:31:21 +00:00
Pirama Arumuga Nainar
32530669f1 [LTO] Drop non-prevailing definitions only if linkage is not local or appending
Summary:
This fixes PR 37422

In ELF, non-weak symbols can also be non-prevailing.  In this particular
PR, the __llvm_profile_* symbols are non-prevailing but weren't getting
dropped - causing multiply-defined errors with lld.

Also add a test, strong_non_prevailing.ll, to ensure that multiple
copies of a strong symbol are dropped.

To fix the test regressions exposed by this fix,
- do not mark prevailing copies for symbols with 'appending' linkage.
There's no one prevailing copy for such symbols.
- fix the prevailing version in dead-strip-fulllto.ll
- explicitly pass exported symbols to llvm-lto in fumcimport.ll and
funcimport_var.ll

Reviewers: tejohnson, pcc

Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith,
dang, srhines, llvm-commits

Differential Revision: https://reviews.llvm.org/D54125

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@346436 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-08 20:10:07 +00:00
Xin Tong
fbe10782a9 [ThinLTO] Add an option to disable (thin)lto internalization.
Summary:
LTO and ThinLTO optimizes the IR differently.

One source of differences is the amount of internalizations that
can happen.

Add an option to enable/disable internalization so that other
differences can be studied in isolation. e.g. inlining.

There are other things lto and thinlto do differently, I will add
flags to enable/disable them as needed.

Reviewers: tejohnson, pcc, steven_wu

Subscribers: mehdi_amini, inglorion, steven_wu, dexonsmith, dang, llvm-commits

Differential Revision: https://reviews.llvm.org/D53294

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@346140 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-05 15:49:46 +00:00
Fedor Sergeev
ff88649977 [NewPM] teach -passes= to emit meaningful error messages
All the PassBuilder::parse interfaces now return descriptive StringError
instead of a plain bool. It allows to make -passes/aa-pipeline parsing
errors context-specific and thus less confusing.

TODO: ideally we should also make suggestions for misspelled pass names,
but that requires some extensions to PassBuilder.

Reviewed By: philip.pfaffe, chandlerc
Differential Revision: https://reviews.llvm.org/D53246

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@344685 91177308-0d34-0410-b5e6-96231b3b80d8
2018-10-17 10:36:23 +00:00
Fedor Sergeev
dea3b338ac Revert "[NewPM] teach -passes= to emit meaningful error messages"
This reverts r344519 due to failures in pipeline-parsing test.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@344524 91177308-0d34-0410-b5e6-96231b3b80d8
2018-10-15 15:36:08 +00:00
Fedor Sergeev
bce0a9abff [NewPM] teach -passes= to emit meaningful error messages
Summary:
All the PassBuilder::parse interfaces now return descriptive StringError
instead of a plain bool. It allows to make -passes/aa-pipeline parsing
errors context-specific and thus less confusing.

TODO: ideally we should also make suggestions for misspelled pass names,
but that requires some extensions to PassBuilder.

Reviewed By: philip.pfaffe, chandlerc
Differential Revision: https://reviews.llvm.org/D53246

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@344519 91177308-0d34-0410-b5e6-96231b3b80d8
2018-10-15 15:00:18 +00:00
Richard Smith
63ec2563a9 Add a flag to remap manglings when reading profile data information.
This can be used to preserve profiling information across codebase
changes that have widespread impact on mangled names, but across which
most profiling data should still be usable. For example, when switching
from libstdc++ to libc++, or from the old libstdc++ ABI to the new ABI,
or even from a 32-bit to a 64-bit build.

The user can provide a remapping file specifying parts of mangled names
that should be treated as equivalent (eg, std::__1 should be treated as
equivalent to std::__cxx11), and profile data will be treated as
applying to a particular function if its name is equivalent to the name
of a function in the profile data under the provided equivalences. See
the documentation change for a description of how this is configured.

Remapping is supported for both sample-based profiling and instruction
profiling. We do not support remapping indirect branch target
information, but all other profile data should be remapped
appropriately.

Support is only added for the new pass manager. If someone wants to also
add support for this for the old pass manager, doing so should be
straightforward.

This is the LLVM side of Clang r344199.

Reviewers: davidxl, tejohnson, dlj, erik.pilkington

Subscribers: mehdi_amini, steven_wu, dexonsmith, llvm-commits

Differential Revision: https://reviews.llvm.org/D51249

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@344200 91177308-0d34-0410-b5e6-96231b3b80d8
2018-10-10 23:13:47 +00:00
Warren Ristow
7f7ab9f57f [LTO] Account for overriding lib calls via the alias attribute
Given a library call that is represented as an llvm intrinsic call, but
later transformed to an actual call, if an overriding definition of that
library routine is provided indirectly via an alias, prevent LTO from
eliminating the definition.

This is a fix for PR38547.

Differential Revision: https://reviews.llvm.org/D52836


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@344198 91177308-0d34-0410-b5e6-96231b3b80d8
2018-10-10 22:54:31 +00:00
Fangrui Song
3b35e17b21 llvm::sort(C.begin(), C.end(), ...) -> llvm::sort(C, ...)
Summary: The convenience wrapper in STLExtras is available since rL342102.

Reviewers: dblaikie, javed.absar, JDevlieghere, andreadb

Subscribers: MatzeB, sanjoy, arsenm, dschuff, mehdi_amini, sdardis, nemanjai, jvesely, nhaehnle, sbc100, jgravelle-google, eraman, aheejin, kbarton, JDevlieghere, javed.absar, gbedwell, jrtc27, mgrang, atanasyan, steven_wu, george.burgess.iv, dexonsmith, kristina, jsji, llvm-commits

Differential Revision: https://reviews.llvm.org/D52573

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343163 91177308-0d34-0410-b5e6-96231b3b80d8
2018-09-27 02:13:45 +00:00
Fedor Sergeev
b8bd68f481 [PassTiming] cleaning up legacy PassTimingInfo interface. NFCI.
During D51276 discussion it was decided that legacy PassTimingInfo
interface can not be reused for new pass manager's implementation
of -time-passes.

This is a cleanup in preparation for D51276 to make legacy interface
as concise as possible, moving the PassTimingInfo from the header
into the anonymous legacy namespace in .cpp.

It is rather close to a revert of rL340872 in a sense that it hides
the interface and gets rid of templates. However as compared to
a complete revert it resides in a different translation unit and has
an additional pass-instance counting funcitonality (PassIDCountMap).

Reviewers: philip.pfaffe
Differential Revision: https://reviews.llvm.org/D52356

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343104 91177308-0d34-0410-b5e6-96231b3b80d8
2018-09-26 13:01:43 +00:00
Teresa Johnson
0d4dfd227e [ThinLTO] Efficiency fix for writing type id records in per-module indexes
Summary:
In D49565/r337503, the type id record writing was fixed so that only
referenced type ids were emitted into each per-module index for ThinLTO
distributed builds. However, this still left an efficiency issue: each
per-module index checked all type ids for membership in the referenced
set, yielding O(M*N) performance (M indexes and N type ids).

Change the TypeIdMap in the summary to be indexed by GUID, to facilitate
correlating with type identifier GUIDs referenced in the function
summary TypeIdInfo structures. This allowed simplifying other
places where a map from type id GUID to type id map entry was previously
being used to aid this correlation.

Also fix AsmWriter code to handle the rare case of type id GUID
collision.

For a large internal application, this reduced the thin link time by
almost 15%.

Reviewers: pcc, vitalybuka

Subscribers: mehdi_amini, inglorion, steven_wu, dexonsmith, llvm-commits

Differential Revision: https://reviews.llvm.org/D51330

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343021 91177308-0d34-0410-b5e6-96231b3b80d8
2018-09-25 20:14:40 +00:00
Caroline Tice
a53c520ec5 Pass code-model through Module IR to LTO which will use it.
Currently the code-model does not get saved in the module IR,
so if a code model is specified when compiling with LTO,
it gets lost and is not propagated properly to LTO. This patch,
along with one for the front end, fixes that.

Differential Revision: https://reviews.llvm.org/D52322



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@342760 91177308-0d34-0410-b5e6-96231b3b80d8
2018-09-21 18:41:31 +00:00