Commit Graph

53 Commits

Author SHA1 Message Date
Teresa Johnson
af76bb65e2 [LTO] Allow pass remarks with hotness to be set when emitting to stderr
Summary:
Set setDiagnosticsHotnessRequested before the early exit check for a
diagnostic output file, so that pass remarks with hotness works when
emitting pass remarks to stderr (e.g. via -pass-remarks=.).

Also fix the llvm-lto2 diagnistic handler so that it only calls exit(1)
when the diagnistic is an error type. Otherwise the new test invocation
of llvm-lto2 with -pass-remarks causes it to fail. The new code is
consistent with the diagnostic handler elsewhere (e.g. on the
LLVMContext).

Reviewers: pcc, davide

Subscribers: fhahn, mehdi_amini, llvm-commits, inglorion

Differential Revision: https://reviews.llvm.org/D46387

llvm-svn: 331569
2018-05-04 23:59:34 +00:00
Florian Hahn
dd48f29ba7 [LTO] Add stats-file option to LTO/Config.h.
This patch adds a StatsFile option to LTO/Config.h and updates both
LLVMGold and llvm-lto2 to set it.

Reviewers: MatzeB, tejohnson, espindola

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D45531

llvm-svn: 330411
2018-04-20 10:18:36 +00:00
David Blaikie
5457a86a41 Rename *CommandFlags.def to *CommandFlags.inc
These aren't the .def style files used in LLVM that require a macro
defined before their inclusion - they're just basic non-modular includes
to stamp out command line flag variables.

llvm-svn: 329840
2018-04-11 18:49:37 +00:00
Vitaly Buka
b49050d555 [ThinLTO] Always create linked objects file for --thinlto-index-only=
Summary:
ThinLTO indexing may decide to skip all objects. If we don't write something to
the list build system may consider this as failure or linker can reuse a file
from the previews build.

Reviewers: pcc, tejohnson

Subscribers: mehdi_amini, inglorion, eraman, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D43415

llvm-svn: 325819
2018-02-22 19:06:15 +00:00
Teresa Johnson
b335ddcfcc [LTO] Remove unused Path parameter to AddBufferFn
Summary:
With D43396, no clients use the Path parameter anymore.

Depends on D43396.

Reviewers: pcc

Subscribers: mehdi_amini, inglorion, llvm-commits

Differential Revision: https://reviews.llvm.org/D43400

llvm-svn: 325619
2018-02-20 20:21:53 +00:00
Vitaly Buka
b9e1305d4a [ThinLTO/gold] Write empty imports even for modules with symbols
Summary: ThinLTO may skip object for other reasons, e.g. if there is no summary.

Reviewers: pcc, eugenis

Subscribers: mehdi_amini, inglorion, eraman, llvm-commits

Differential Revision: https://reviews.llvm.org/D42514

llvm-svn: 323818
2018-01-30 21:19:26 +00:00
David Blaikie
1aae789bfe Rename CommandFlags.h -> CommandFlags.def
Since this isn't a real header - it includes static functions and had
external linkage variables (though this change makes them static, since
that's what they should be) so can't be included more than once in a
program.

llvm-svn: 319082
2017-11-27 19:43:58 +00:00
Peter Collingbourne
0ec5439ada LTO: Try to open cache files before renaming them.
It appears that a potential race between the cache client and the cache
pruner that I thought was unlikely actually happened in practice [1].
Try to avoid the race condition by opening the temporary file before
renaming it. Do this only on non-Windows platforms because we cannot
rename open files on Windows using the sys::fs::rename function.

[1] https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Fchromium.memory%2FLinux_CFI%2F1610%2F%2B%2Frecipes%2Fsteps%2Fcompile%2F0%2Fstdout

Differential Revision: https://reviews.llvm.org/D37410

llvm-svn: 312567
2017-09-05 19:51:38 +00:00
Rafael Espindola
f2011a3ae7 Delete Default and JITDefault code models
IMHO it is an antipattern to have a enum value that is Default.

At any given piece of code it is not clear if we have to handle
Default or if has already been mapped to a concrete value. In this
case in particular, only the target can do the mapping and it is nice
to make sure it is always done.

This deletes the two default enum values of CodeModel and uses an
explicit Optional<CodeModel> when it is possible that it is
unspecified.

llvm-svn: 309911
2017-08-03 02:16:21 +00:00
Dehao Chen
520d826c62 Fix the bug that parseAAPipeline is not invoked in runNewPMPasses in release compiler.
Summary: The logic is guarded by "assert".

Reviewers: davidxl, davide, chandlerc

Reviewed By: davide, chandlerc

Subscribers: sanjoy, llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D36195

llvm-svn: 309787
2017-08-02 03:03:19 +00:00
Tobias Edler von Koch
90d8eae713 [LTO] llvm-lto2: Add option to load sample profile
Summary:
This exposes LTO's Conf.SampleProfile as a command line option
(-lto-sample-profile-file) for testing via the llvm-lto2 utility.

Reviewers: pcc, danielcdh

Subscribers: mehdi_amini, inglorion, llvm-commits

Differential Revision: https://reviews.llvm.org/D36030

llvm-svn: 309456
2017-07-28 23:43:22 +00:00
Teresa Johnson
411e4731ce [LTO] Prevent dead stripping and internalization of symbols with sections
Summary:
ELF linkers generate __start_<secname> and __stop_<secname> symbols
when there is a value in a section <secname> where the name is a valid
C identifier.  If dead stripping determines that the values declared
in section <secname> are dead, and we then internalize (and delete)
such a symbol, programs that reference the corresponding start and end
section symbols will get undefined reference linking errors.

To fix this, add the section name to the IRSymtab entry when a symbol is
defined in a specific section. Then use this in the gold-plugin to mark
the symbol as external and visible from outside the summary when the
section name is a valid C identifier.

Reviewers: pcc

Subscribers: mehdi_amini, inglorion, eraman, llvm-commits

Differential Revision: https://reviews.llvm.org/D35639

llvm-svn: 309009
2017-07-25 19:42:32 +00:00
Hiroshi Inoue
11f0187c4e Add missing library dependency to fix build break in llvm-lto2
error message
CMakeFiles/llvm-lto2.dir/llvm-lto2.cpp.o: In function `dumpSymtab(int, char**)':
llvm-lto2.cpp:(.text._ZL10dumpSymtabiPPc+0x238): undefined reference to `llvm::getBitcodeFileContents(llvm::MemoryBufferRef)'
collect2: error: ld returned 1 exit status

llvm-svn: 306507
2017-06-28 06:14:30 +00:00
Peter Collingbourne
8c0398b976 Object: Teach irsymtab::read() to try to use the irsymtab that we wrote to disk.
Fixes PR27551.

Differential Revision: https://reviews.llvm.org/D33974

llvm-svn: 306488
2017-06-27 23:50:24 +00:00
Dmitry Mikulin
c325036358 Symbols re-defined with -wrap and -defsym need to be excluded from inter-
procedural optimizations to prevent dropping symbols and allow the linker
to process re-directs.

PR33145: --wrap doesn't work with lto.
Differential Revision: https://reviews.llvm.org/D33621

llvm-svn: 304719
2017-06-05 16:24:25 +00:00
Tim Shen
c432c72577 [ThinLTO] Move -lto-use-new-pm to llvm-lto2, and change it to -use-new-pm.
Summary:
As we teach Clang to use ThinkLTO + new PM, it's good for the users to
inject through Config, instead of setting a flag in the LTOBackend
library. Move the flag to llvm-lto2.

As it moves to llvm-lto2, a new name -use-new-pm seems simpler and as
clear.

Reviewers: davide, tejohnson

Subscribers: mehdi_amini, Prazek, inglorion, eraman, chandlerc, llvm-commits

Differential Revision: https://reviews.llvm.org/D33799

llvm-svn: 304492
2017-06-01 23:13:44 +00:00
Peter Collingbourne
5d0d01d99f Object, LTO: Add target triple to irsymtab and LTO API.
Start using it in LLD to avoid needing to read bitcode again just to get the
target triple, and in llvm-lto2 to avoid printing symbol table information
that is inappropriate for the target.

Differential Revision: https://reviews.llvm.org/D32038

llvm-svn: 300300
2017-04-14 02:55:06 +00:00
Tobias Edler von Koch
80cff62afe LTO: Pass SF_Executable flag through to InputFile::Symbol
Summary:
The linker needs to be able to determine whether a symbol is text or data to
handle the case of a common being overridden by a strong definition in an
archive. If the archive contains a text member of the same name as the common,
that function is discarded. However, if the archive contains a data member of
the same name, that strong definition overrides the common. This is a behavior
of ld.bfd, which the Qualcomm linker also supports in LTO.

Here's a test case to illustrate:

####

cat > 1.c << \!
int blah;
!

cat > 2.c << \!
int blah() {
  return 0;
}
!

cat > 3.c << \!
int blah = 20;
!

clang -c 1.c
clang -c 2.c
clang -c 3.c

ar cr lib.a 2.o 3.o
ld 1.o lib.a -t

####

The correct output is:

1.o
(lib.a)3.o

Thanks to Shankar Easwaran and Hemant Kulkarni for the test case!

Reviewers: mehdi_amini, rafael, pcc, davide

Reviewed By: pcc

Subscribers: davide, llvm-commits, inglorion

Differential Revision: https://reviews.llvm.org/D31901

llvm-svn: 300205
2017-04-13 16:24:14 +00:00
Peter Collingbourne
47c4d3700d llvm-lto2: Add a dump-symtab subcommand.
This allows us to test the symbol table APIs for LTO input files.

Differential Revision: https://reviews.llvm.org/D31920

llvm-svn: 300086
2017-04-12 18:27:00 +00:00
Peter Collingbourne
6517513ff1 llvm-lto2: Move the LTO::run() action behind a subcommand.
Move LTO::run() to a "run" subcommand so that we can introduce new subcommands
for testing different parts of the LTO implementation.

This doesn't use llvm::cl subcommands because it doesn't appear to be currently
possible to pass an argument not associated with a subcommand to a subcommand
(e.g. -lto-use-new-pm, -mcpu=yonah).

Differential Revision: https://reviews.llvm.org/D31410

llvm-svn: 299967
2017-04-11 18:12:00 +00:00
Peter Collingbourne
3b3c09c504 More accurate header inclusions. NFC.
llvm-svn: 298960
2017-03-28 23:35:34 +00:00
Peter Collingbourne
ff87ac19c3 LTO: Fix a potential race condition in the caching API.
After the call to sys::fs::exists succeeds, indicating a cache hit, we call
AddFile and the client will open the file using the supplied path. If the
client is using cache pruning, there is a potential race between the pruner
and the client. To avoid this, change the caching API so that it provides
a MemoryBuffer to the client, and have clients use that MemoryBuffer where
possible.

This scheme won't work with the gold plugin because the plugin API expects a
file path. So we have the gold plugin use the buffer identifier as a path and
live with the race for now. (Note that the gold plugin isn't actually affected
by the problem at the moment because it doesn't support cache pruning.)

This effectively reverts r279883 modulo the change to use the existing path
in the gold plugin.

Differential Revision: https://reviews.llvm.org/D31063

llvm-svn: 298020
2017-03-17 00:34:07 +00:00
Teresa Johnson
49123ecd70 Fix test and add missing return for llvm-lto2 error case
Summary:
This test was missing the target triple.
Once I fixed that, the case with the invalid character error stopped
returning 1 from llvm-lto2 and the test reported a failure. Fixed by
adding the missing return from llvm-lto2. Apparently we were failing
when we eventually tried to get the target.

Reviewers: pcc

Subscribers: mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D30585

llvm-svn: 297173
2017-03-07 18:15:13 +00:00
Peter Collingbourne
dfbe8f3dfb LTO: When creating a local cache, create the cache directory if it does not already exist.
Differential Revision: https://reviews.llvm.org/D30519

llvm-svn: 296726
2017-03-02 02:02:38 +00:00
Tobias Edler von Koch
fddccdb0f9 [LTO] Add ability to emit assembly to new LTO API
Summary:
Add a field to LTO::Config, CGFileType, to select the file type to emit (object
or assembly). This is useful for testing and to implement -save-temps.

Reviewers: tejohnson, mehdi_amini, pcc

Reviewed By: mehdi_amini

Subscribers: davide, llvm-commits

Differential Revision: https://reviews.llvm.org/D29475

llvm-svn: 295226
2017-02-15 20:36:36 +00:00
Davide Italiano
4a2f3539a0 [llvm-lto2] Fix typo spotted by Teresa (r294885 post-commit review).
llvm-svn: 294962
2017-02-13 16:08:36 +00:00
Davide Italiano
929e2c167e [lib/LTO] Add support for hotness optremarks in the new API.
llvm-svn: 294885
2017-02-12 05:05:35 +00:00
Davide Italiano
4e84a97600 [llvm-lto2] Fix typo in error message.
llvm-svn: 294883
2017-02-12 03:42:09 +00:00
Davide Italiano
9f5a87c088 [lib/LTO] Initial support for optimization remarks in the new API.
llvm-svn: 294882
2017-02-12 03:31:30 +00:00
Mehdi Amini
eef0435aa7 llvm-lto2: remove leading '-' for cl::opt declaration
llvm-svn: 290464
2016-12-23 23:54:34 +00:00
Mehdi Amini
22aacdc951 llvm-lto2: Print diagnostics before exiting (NFC)
llvm-svn: 290463
2016-12-23 23:54:17 +00:00
Peter Collingbourne
50a8d23367 LTO: Hash the parts of the LTO configuration that affect code generation.
Most importantly, we need to hash the relocation model, otherwise we can
end up trying to link non-PIC object files into PIEs or DSOs.

Differential Revision: https://reviews.llvm.org/D27556

llvm-svn: 289024
2016-12-08 05:28:30 +00:00
Peter Collingbourne
0d889ceff3 llvm-lto2: Simpler workaround for PR30396.
Maintain the command line resolutions as a map to a list of resolutions
rather than a single resolution, and apply the resolutions in the order
observed. This is not only simpler but allows us to test the scenario where
the two symbols have different resolutions.

Differential Revision: https://reviews.llvm.org/D27285

llvm-svn: 288288
2016-11-30 23:19:05 +00:00
Chris Bieneman
68d2cdc808 [CMake] llvm-lto2 depends on intrinsics_gen
llvm-lto2.cpp has the following include chain:

llvm/LTO/Caching.h
llvm/LTO/LTO.h
llvm/CodeGen/Analysis.h
llvm/IR/CallSite.h
llvm/IR/Attributes.h
llvm/IR/Attributes.gen

This means llvm-lto2 needs to depend on intrinsics_gen.

llvm-svn: 287434
2016-11-19 03:19:58 +00:00
Teresa Johnson
440d83db71 [ThinLTO] Disable importing and other cross-module optis at -O0
Summary:
There is no point to importing at -O0, since we won't inline. We should
also disable other cross-module optimizations.

(Plan to backport this fix to the 3.9 branch to fix PR30774)

Reviewers: pcc

Subscribers: johanengelen, mehdi_amini

Differential Revision: https://reviews.llvm.org/D25918

llvm-svn: 285648
2016-10-31 22:12:21 +00:00
Teresa Johnson
73f09465fc [ThinLTO] Default backend threads to heavyweight_hardware_concurrency
Summary:
Changes default backend parallelism from thread::hardware_concurrency to
the new llvm::heavyweight_hardware_concurrency, which for X86 Linux
defaults to the number of physical cores (and will fall back to
thread::hardware_concurrency otherwise). This avoid oversubscribing
the physical cores using hyperthreading.

Reviewers: mehdi_amini, pcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D25775

llvm-svn: 284618
2016-10-19 17:35:01 +00:00
Teresa Johnson
c8d516dcb9 [ThinLTO] Fix bot failure due to unused variable with NDEBUG
Put variable only used in assert under #ifndef NDEBUG.

This should fix bot failure at
http://lab.llvm.org:8011/builders/lld-x86_64-darwin13/builds/28537

llvm-svn: 284039
2016-10-12 20:06:02 +00:00
Teresa Johnson
de773ccc50 [ThinLTO] Don't link module level assembly when importing
Module inline asm was always being linked/concatenated
when running the IRLinker. This is correct for full LTO but not when
we are importing for ThinLTO, as it can result in multiply defined
symbols when the module asm defines a global symbol.

In order to test with llvm-lto2, I had to work around PR30396,
where a symbol that is defined in module assembly but defined in the
LLVM IR appears twice. Added workaround to llvm-lto2 with a FIXME.

Fixes PR30610.

Reviewers: mehdi_amini

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D25359

llvm-svn: 284030
2016-10-12 18:39:29 +00:00
Peter Collingbourne
cef2ce70e8 LTO: Simplify caching interface.
The NativeObjectOutput class has a design problem: it mixes up the caching
policy with the interface for output streams, which makes the client-side
code hard to follow and would for example make it harder to replace the
cache implementation in an arbitrary client.

This change separates the two aspects by moving the caching policy
to a separate field in Config, replacing NativeObjectOutput with a
NativeObjectStream class which only deals with streams and does not need to
be overridden by most clients and introducing an AddFile callback for adding
files (e.g. from the cache) to the link.

Differential Revision: https://reviews.llvm.org/D24622

llvm-svn: 282299
2016-09-23 21:33:43 +00:00
Davide Italiano
b91039268d [LTO] Add ability to parse AA pipelines.
This is supposed to be a drop in replacement for what lld
provides via --lto-newpm-aa-pipeline.

llvm-svn: 281774
2016-09-16 21:03:21 +00:00
Davide Italiano
9cc7f6f6fa [lib/LTO] Add a way to run a custom pipeline
Differential Revision:  https://reviews.llvm.org/D24095

llvm-svn: 280830
2016-09-07 17:46:16 +00:00
Teresa Johnson
048eeff265 [ThinLTO] Move loading of cache entry to client
Summary:
Have the cache pass back the path to the cache entry when it
is ready to be loaded, instead of a buffer.

For gold-plugin we can simply pass this file back to gold directly,
which avoids expensive writing of a separate tmp file. Ensure
the cache entry is not deleted on cleanup by adjusting the setting
of the IsTemporary flags.

Moved the loading of the buffer into llvm-lto2 to maintain current
behavior.

Reviewers: mehdi_amini

Subscribers: llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D23946

llvm-svn: 279883
2016-08-26 23:29:14 +00:00
Mehdi Amini
54cea79c7b [ThinLTO] Add caching to the new LTO API
Add the ability to plug a cache on the LTO API.
I tried to write such that a linker implementation can
control the cache backend. This is intrusive and I'm
not totally happy with it, but I can't figure out a
better design right now.

Differential Revision: https://reviews.llvm.org/D23599

llvm-svn: 279576
2016-08-23 21:30:12 +00:00
Mehdi Amini
c319f0c436 [LTO] Add the ability to test -thinlto-emit-imports-files through llvm-lto2
Summary:
Start bringing llvm-lto2 to a level where we can test the LTO API
a bit deeper.

Reviewers: tejohnson

Subscribers: llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D23681

llvm-svn: 279349
2016-08-19 23:54:40 +00:00
Mehdi Amini
ebde3ba665 [LTO] Change addSaveTemps API: do not add dot to the supplied prefix path
Summary:
It does not play well with directories (end up with a bunch of hidden
files).
Also, do not strip the 0 suffix for the first task, especially since
0 can be used by ThinLTO as well now.

Reviewers: tejohnson

Subscribers: mehdi_amini, pcc, llvm-commits

Differential Revision: https://reviews.llvm.org/D23612

llvm-svn: 279014
2016-08-18 00:12:33 +00:00
Chandler Carruth
b4447ed5f1 [LTO] Fix a use-after-free introduced in r278907 and caught by ASan.
The ASan build bot caught this right away:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/15580/steps/check-llvm%20asan/logs/stdio

This was also breaking a Windows build bot I'm pretty sure.

llvm-svn: 278912
2016-08-17 07:48:34 +00:00
Mehdi Amini
e80ee44a77 [LTO] Introduce an Output class to wrap the output stream creation (NFC)
Summary:
While NFC for now, this will allow more flexibility on the client side
to hold state necessary to back up the stream.
Also when adding caching, this class will grow in complexity.

Note I blindly modified the gold-plugin as I can't compile it.

Reviewers: tejohnson

Subscribers: mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D23542

llvm-svn: 278907
2016-08-17 06:23:09 +00:00
Teresa Johnson
84d0e6d6a5 Add (hopefully last) remaining missing dependences to llvm-lto2
There are still a few missing symbols reported by:
http://bb.pgr.jp/builders/i686-mingw32-RA-on-linux/builds/15535/steps/build_llvmclang/logs/stdio
http://bb.pgr.jp/builders/i686-mingw32-RA-on-linux/builds/15535/steps/build_llvmclang/logs/stdio

This should hopefully take care of them.

llvm-svn: 278353
2016-08-11 16:29:47 +00:00
Teresa Johnson
0284619fa1 More missing llvm-lto2 dependencies
Follow-on to r278341: Update CMakeLists.txt to match LLVMBuild.txt

llvm-svn: 278349
2016-08-11 15:58:49 +00:00
Teresa Johnson
d5763f35d0 Fix bot failure from r278338 due to missing dependences
Add some missing dependences to the llvm-lto2 tool to attempt to appease
missing symbols in link from bot:
http://bb.pgr.jp/builders/i686-mingw32-RA-on-linux/builds/15527

llvm-svn: 278341
2016-08-11 15:23:05 +00:00