1148 Commits

Author SHA1 Message Date
Amir Ayupov
5790441c45 [BOLT][NFC] Use getShortOpcodeArith in X86MCPlusBuilder
Unify `llvm::X86::getRelaxedOpcodeArith` and `getShortArithOpcode` in
X86MCPlusBuilder.cpp.

Addresses https://lists.llvm.org/pipermail/llvm-dev/2022-January/154526.html

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D121404
2022-03-12 09:07:28 -08:00
Petr Hosek
0c0f6cfb7b [CMake] Rename TARGET_TRIPLE to LLVM_TARGET_TRIPLE
This clarifies that this is an LLVM specific variable and avoids
potential conflicts with other projects.

Differential Revision: https://reviews.llvm.org/D119918
2022-03-11 15:43:01 -08:00
Elvina Yakubova
db65429db5 [BOLT] Divide RegularPageSize for X86 and AArch64 cases
For AArch64 in some cases/some distributions ld uses 64K alignment of LOAD segments by default.

Reviewed By: yota9, maksfb

Differential Revision: https://reviews.llvm.org/D119267
2022-03-10 23:09:50 +03:00
Vladislav Khmelevsky
04b87cf0e7 [BOLT] LongJmp: Use per-function alignment values
The per-function alignment values must be used in order to create
tentative layout.

Differential Revision: https://reviews.llvm.org/D121298
2022-03-10 19:48:48 +03:00
Amir Ayupov
d1638cb0b5 [BOLT][NFC] Fix print-cfg data race
Addresses ThreadSanitizer warning

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D121337
2022-03-09 20:28:06 -08:00
Amir Ayupov
d16bbc5340 [BOLT][NFC] Check errors from Obj.dynamicEntries
Addresses fuzzer crash

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D121336
2022-03-09 20:24:44 -08:00
Vladislav Khmelevsky
506a91c089 [BOLT] Move some of the tests to common directory
Some of the tests are not x86-specific, move them to common directory.

Differential Revision: https://reviews.llvm.org/D121261
2022-03-09 14:51:17 +03:00
Vladislav Khmelevsky
8bdbcfe7d8 [BOLT] Handle ifuncs trampolines for aarch64
The aarch64 uses the trampolines located in .iplt section, which
contains plt-like trampolines on the value stored in .got. In this case
we don't have JUMP_SLOT relocation, but we have a symbol that belongs to
ifunc trampoline, so use it and set set plt symbol for such functions.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Differential Revision: https://reviews.llvm.org/D120850
2022-03-09 01:38:19 +03:00
Amir Ayupov
1e016c3bd5 [BOLT][NFC] Handle "dynamic section sizes should match"
Address fuzzer crash on malformed input

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D121070
2022-03-08 13:03:05 -08:00
Amir Ayupov
687e4af1c0 [BOLT] CMOVConversion pass
Convert simple hammocks into cmov based on misprediction rate.

Test Plan:
- Assembly test: `cmov-conversion.s`
- Testing on a binary:
  # Bootstrap clang with `-x86-cmov-converter-force-all` and `-Wl,--emit-relocs`
  (Release build)
  # Collect perf.data:

    - `clang++ <opts> bolt/lib/Core/BinaryFunction.cpp -E > bf.cpp`
    - `perf record -e cycles:u -j any,u -- clang-15 bf.cpp -O2 -std=c++14 -c -o bf.o`
  # Optimize clang-15 with and w/o -cmov-conversion:
    - `llvm-bolt clang-15 -p perf.data -o clang-15.bolt`
    - `llvm-bolt clang-15 -p perf.data -cmov-conversion -o clang-15.bolt.cmovconv`
  # Run perf experiment:
    - test: `clang-15.bolt.cmovconv`,
    - control: `clang-15.bolt`,
    - workload (clang options): `bf.cpp -O2 -std=c++14 -c -o bf.o`
Results:
```
  task-clock [delta: -360.21 ± 356.75, delta(%): -1.7760 ± 1.7589, p-value: 0.047951, balance: -6]
  instructions  [delta: 44061118 ± 13246382, delta(%): 0.0690 ± 0.0207, p-value: 0.000001, balance: 50]
  icache-misses [delta: -5534468 ± 2779620, delta(%): -0.4331 ± 0.2175, p-value: 0.028014, balance: -28]
  branch-misses [delta: -1624270 ± 1113244, delta(%): -0.3456 ± 0.2368, p-value: 0.030300, balance: -22]
```

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D120177
2022-03-08 10:44:31 -08:00
Amir Ayupov
ced5472e09 [BOLT][NFC] Check section contents before registering it
Address fuzzer crash on malformed input:
```
BOLT-ERROR: cannot get section contents for .dynsym: The end of the file was unexpectedly encountered.
```

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D121068
2022-03-08 09:13:01 -08:00
Amir Ayupov
018ad03efa [BOLT][CMAKE] Remove CMake 3.13.4 incompatible parameter
Remove `TYPE BIN` parameter that is introduced in CMake 3.14 and revert back to
the equivalent compatible form `DESTINATION ${CMAKE_INSTALL_BINDIR}`.

Addresses https://github.com/llvm/llvm-project/issues/54099

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D121012
2022-03-07 18:42:00 -08:00
Yi Kong
8142ace0a7 Revert "Add CMake option not to build BOLT tests"
This reverts commit d8f4d54664171bac46e6c8446df04850b40cc4f7.

Merged by accident.
2022-03-08 02:01:15 +08:00
Yi Kong
d8f4d54664 Add CMake option not to build BOLT tests 2022-03-08 01:59:44 +08:00
Maksim Panchenko
fada230920 [BOLT][NFC] Return MCRegister::NoRegister from MCPlusBuilder::getNoRegister()
Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D120863
2022-03-03 13:25:13 -08:00
Vladislav Khmelevsky
00b6efc830 [BOLT] Enable PLT analysis for aarch64
This patch enables PLT analysis for aarch64. It is used by the static
relocations in order to provide final symbol address of PLT entry for some
instructions like ADRP.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Differential Revision: https://reviews.llvm.org/D118088
2022-03-02 22:14:48 +03:00
Maksim Panchenko
ae87445c25 [BOLT][test] Fix function size in test case 2022-03-01 17:53:41 -08:00
Amir Ayupov
08dcbed92f [BOLT] Fix X86MCPlusBuilder::replaceRegWithImm
Reassigning the operand didn't update the operand type which resulted in an
assertion (`Assertion `isReg() && "This is not a register operand!"' failed.`)
Reset the instruction instead.

Test Plan:
```
ninja check-bolt
...
PASS: BOLT-Unit :: Core/./CoreTests/X86/MCPlusBuilderTester.ReplaceRegWithImm/0 (90 of 136)
```

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D120263
2022-02-28 19:24:46 -08:00
Alexander Yermolovich
a44fe31977 [BOLT][DWARF] Fix how DW_AT_high_pc [DW_FORM_udata] is handled
We were not handling correctly conversion from DW_AT_high_pc into DW_AT_ranges,
when size of DW_AT_high_pc is not 4/8 bytes.

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D120528
2022-02-25 10:32:05 -08:00
Maksim Panchenko
4101aa130a [BOLT] Support PC-relative relocations with addends
PC-relative memory operand could reference a different object from
the one located at the target address, e.g. when a negative offset
is used. Check relocations for the real referenced object.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D120379
2022-02-23 22:54:42 -08:00
Amir Ayupov
af6e66f44c [BOLT][NFC] Report errors from RewriteInstance discoverStorage and run
Further improve error handling in BOLT by reporting `RewriteInstance` errors in
a library and fuzzer-friendly way instead of exiting.

Follow-up to D119658

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D120224
2022-02-23 20:42:39 -08:00
Amir Ayupov
454c149898 [BOLT][NFC] Fix undefined behavior in encodeAnnotationImm
Fix UBSan-reported issue in MCPlusBuilder::encodeAnnotationImm (left shift of a
negative value).

Test Plan:
```
ninja check-bolt
...
PASS: BOLT-Unit :: Core/./CoreTests/AArch64/MCPlusBuilderTester.Annotation/0 (1 of 140)
PASS: BOLT-Unit :: Core/./CoreTests/X86/MCPlusBuilderTester.Annotation/0 (131 of 134)
```

Reviewed By: maksfb, yota9

Differential Revision: https://reviews.llvm.org/D120260
2022-02-23 16:02:49 -08:00
Alexander Yermolovich
210bb04e23 [BOLT][DWARF] Remove patchLowHigh unused function.
Cleanup after removing caching mechanims for ranges/abbrevs.

Reviewed By: rafauler, yota9

Differential Revision: https://reviews.llvm.org/D120174
2022-02-22 13:28:15 -08:00
Amir Ayupov
d44f99c748 [BOLT] Added fuzzer target (llvm-bolt-fuzzer)
This adds a target that would consume random binary as an
input ELF file.
TBD: add structured input support (ELF).

Build:
```
cmake /path/to/llvm-project/llvm -GNinja \
-DLLVM_TARGETS_TO_BUILD="X86;AArch64" \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_ASSERTIONS=1 \
-DCMAKE_C_COMPILER=<sanitizer-capable clang> \
-DCMAKE_CXX_COMPILER=<sanitizer-capable clang++> \
-DLLVM_ENABLE_PROJECTS="bolt"  \
-DLLVM_USE_SANITIZER=Address \
-DLLVM_USE_SANITIZE_COVERAGE=On
ninja llvm-bolt-fuzzer
```

Test Plan: ninja llvm-bolt-fuzzer

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D120016
2022-02-20 17:24:16 -08:00
Amir Ayupov
36ada32727 [BOLT][NFC] Fix data race in ShrinkWrapping stats
Fix data race reported by ThreadSanitizer in clang.test:
```
ThreadSanitizer: data race /data/llvm-project/bolt/lib/Passes/ShrinkWrapping.cpp:1359:28
in llvm::bolt::ShrinkWrapping::moveSaveRestores()
```

The issue is with incrementing global counters from multiple threads.

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D120218
2022-02-20 17:21:58 -08:00
Amir Ayupov
32d2473a5d [BOLT][NFC] Report errors from createBinaryContext and RewriteInstance ctor
Refactor createBinaryContext and RewriteInstance/MachORewriteInstance
constructors to report an error in a library and fuzzer-friendly way instead of
returning a nullptr or exiting.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D119658
2022-02-17 00:50:52 -08:00
Vladislav Khmelevsky
729d29e167 [BOLT] Update dynamic relocations from section relocations
This patch changes patchELFAllocatableRelaSections from going through
old relocations sections and update the relocation offsets to emitting
the relocations stored in binary sections. This is needed in case we
would like to remove and add dynamic relocations during BOLT work and it
is used by golang support pass. Note: Currently we emit relocations in
the old sections, so the total number of them should be equal or less
of old number.

Testing: No special tests are neeeded, since this patch does not fix
anything or add new functionality (it only prepares to add). Every
PIC-compiled test binary will use this code and thus become a test.
But just in case the aarch64 dynamic relocations tests were added.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D117612
2022-02-16 18:40:54 +03:00
Shao-Ce SUN
2aed07e96c [NFC][MC] remove unused argument MCRegisterInfo in MCCodeEmitter
Reviewed By: skan

Differential Revision: https://reviews.llvm.org/D119846
2022-02-16 13:10:09 +08:00
Shao-Ce SUN
9cc49c1951 Revert "[NFC][MC] remove unused argument MCRegisterInfo in MCCodeEmitter"
This reverts commit fe25c06cc5bdc2ef9427309f8ec1434aad69dc7a.
2022-02-16 11:57:49 +08:00
Shao-Ce SUN
fe25c06cc5 [NFC][MC] remove unused argument MCRegisterInfo in MCCodeEmitter
For ten years, it seems that `MCRegisterInfo` is not used by any target.

Reviewed By: skan

Differential Revision: https://reviews.llvm.org/D119846
2022-02-16 11:47:17 +08:00
Alexander Yermolovich
bd1ebe9d04 [BOLT][DWARF] Add ability to insert new entries in to DIE
Added ability to append new entries to DIE. This is useful to standadize DWARF4
Split Dwarf, and simplify implementation of DWARF5.
Multiple DIEs can share an abbrev. So currently limitation is that only unique
Attributes can be added.

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D119577
2022-02-15 18:07:19 -08:00
Vladislav Khmelevsky
eccdf2d9b1 [BOLT] Fix aarch64 dwarf test
After "Remove caching of ranges/abbrevs" patch the dwarf offsets are a
bit changed and the subprograms high pc is replaced with AT_RANGES.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D119733
2022-02-15 13:59:38 +03:00
serge-sans-paille
290e482342 Cleanup LLVMDWARFDebugInfo
As usual with that header cleanup series, some implicit dependencies now need to
be explicit:

llvm/DebugInfo/DWARF/DWARFContext.h no longer includes:
- "llvm/DebugInfo/DWARF/DWARFAcceleratorTable.h"
- "llvm/DebugInfo/DWARF/DWARFCompileUnit.h"
- "llvm/DebugInfo/DWARF/DWARFDebugAbbrev.h"
- "llvm/DebugInfo/DWARF/DWARFDebugAranges.h"
- "llvm/DebugInfo/DWARF/DWARFDebugFrame.h"
- "llvm/DebugInfo/DWARF/DWARFDebugLoc.h"
- "llvm/DebugInfo/DWARF/DWARFDebugMacro.h"
- "llvm/DebugInfo/DWARF/DWARFGdbIndex.h"
- "llvm/DebugInfo/DWARF/DWARFSection.h"
- "llvm/DebugInfo/DWARF/DWARFTypeUnit.h"
- "llvm/DebugInfo/DWARF/DWARFUnitIndex.h"

Plus llvm/Support/Errc.h not included by a bunch of llvm/DebugInfo/DWARF/DWARF*.h files

Preprocessed lines to build llvm on my setup:
after: 1065629059
before: 1066621848

Which is a great diff!

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D119723
2022-02-15 09:16:03 +01:00
Maksim Panchenko
5a343994c3 [BOLT] Make order of jump table successors deterministic
When a jump table is recovered in postProcessIndirectBranches(),
successors for the containing basic block are added in random order.
Make the order deterministic.

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D119672
2022-02-14 10:37:20 -08:00
Maksim Panchenko
641e92d46b [BOLT] Skip warning message if no functions were ignored
Reviewed By: yota9, Amir

Differential Revision: https://reviews.llvm.org/D119673
2022-02-14 10:31:43 -08:00
serge-sans-paille
e72c195fdc Cleanup LLVMObject headers
Most notably,

llvm/Object/Binary.h no longer includes llvm/Support/MemoryBuffer.h
llvm/Object/MachOUniversal*.h no longer include llvm/Object/Archive.h
llvm/Object/TapiUniversal.h no longer includes llvm/Object/TapiFile.h

llvm-project preprocessed size:
before: 1068185081
after:  1068324320

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D119457
2022-02-10 21:13:44 +01:00
Louis Dionne
4ae83bb2b1 Update all LLVM documentation mentioning runtimes in LLVM_ENABLE_PROJECTS
We are moving away from building the runtimes with LLVM_ENABLE_PROJECTS,
however the documentation was largely outdated. This commit updates all
the documentation I could find to use LLVM_ENABLE_RUNTIMES instead of
LLVM_ENABLE_PROJECTS for building runtimes.

Note that in the near future, libcxx, libcxxabi and libunwind will stop
supporting being built with LLVM_ENABLE_PROJECTS altogether. I don't know
what the plans are for other runtimes like libc, openmp and compiler-rt,
so I didn't make any changes to the documentation that would imply
something for those projects.

Once this lands, I will also cherry-pick this on the release/14.x branch
to make sure that LLVM's documentation is up-to-date and reflects what
we intend to support in the future.

Differential Revision: https://reviews.llvm.org/D119351
2022-02-10 15:05:23 -05:00
serge-sans-paille
57f7c7d90e Add missing MC includes in bolt/
Changes needed after ef736a1c39f27ef4 that removes some implicit
dependencies from MrCV headers.
2022-02-09 08:28:34 -05:00
Alexander Yermolovich
0d9921daad [BOLT][DWARF] Remove caching of ranges/abbrevs
Removing caching of ranges/abbrevs to simplify the code.
Before we were doing it to get around a gdb limitation.
FBD34015613

Reviewed By: Amir, maksfb

Differential Revision: https://reviews.llvm.org/D119276
2022-02-08 16:37:40 -08:00
Amir Ayupov
9be6e40d1a [BOLT][TEST] Add .so instrumentation test
Summary: Shared object instrumentation test

Test Plan: bin/llvm-lit -a bolt/test/X86/internal-call-instrument-so.s

Reviewers: rafauler

FBD34064557
2022-02-08 12:43:06 -08:00
Amir Ayupov
c840047c38 [BOLT][CMAKE][NFC] Update runtime/CMakeLists.txt
Summary:
- Specify compiler flags for runtime libraries as BOLT_RT_FLAGS,
- Remove redundant CMake definitions.

Reviewers: maksfb

FBD34048561
2022-02-07 21:04:41 -08:00
Amir Ayupov
03f014c9ae [BOLT] Add ld.lld substitution
Register ld.lld substition (tests were failing)

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D118738
2022-02-07 20:59:18 -08:00
Vladislav Khmelevsky
823ebcc7a8 [BOLT] Fix runtime osx cross-compile build
Place include elf.h under !apple condition

Differential Revision: https://reviews.llvm.org/D119038
2022-02-08 03:42:47 +03:00
Vladislav Khmelevsky
19fb5a210d [BOLT] Add aarch64 support for peephole passes
Enable peephole optimizations for aarch64.
Also small code refactoring - add PeepholeOpts under Peepholes class.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Differential Revision: https://reviews.llvm.org/D118732
2022-02-08 03:04:40 +03:00
Vladislav Khmelevsky
5c2ae5f454 [BOLT] Refactor heatmap to be standalone tool
Separate heatmap from bolt and build it as standalone tool.

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D118946
2022-02-07 22:00:44 +03:00
Amir Ayupov
7f928cbac2 [BOLT] Add nfc-check-setup script
Add the script to set up llvm-bolt-wrapper. The intended use is to run NFC
checks manually and automatically on a buildbot.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D118516
2022-02-04 18:03:36 -08:00
Amir Ayupov
eddf384965 [BOLT-UnitTests] Fix shared libraries build
Fix build with `-DBUILD_SHARED_LIBS=ON` (add explicit deps).

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D119042
2022-02-04 18:01:15 -08:00
Amir Ayupov
194b164eb5 [BOLT][NFC] Fix compiler warnings
Summary:
- variable 'TotalSize' set but not used
- variable 'TotalCallsTopN' set but not used
- use of bitwise '|' with boolean operands

Reviewed By: maksfb

FBD33911129
2022-02-04 15:57:33 -08:00
Amir Ayupov
d36bd08572 [BOLT][TEST] Add section flags for .gcc_except_table
clang-10 complains about changed section flags in two tests:
- X86/shrinkwrapping.test
- X86/exceptions-args.test

Fix that by adding the missing flags.

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D119014
2022-02-04 09:48:22 -08:00
Amir Ayupov
167b623a6a [BOLT][NFC] Use isInt<> instead of range checks
Summary: Reuse LLVM isInt check

Reviewed By: maksfb

FBD33945182
2022-02-02 20:32:05 -08:00