1191 Commits

Author SHA1 Message Date
Vladislav Khmelevsky
2d84e344d9 [PR][BOLT] Check for end iterator in LongJmp stub lookup
Summary:
The lower_bound might return the end iterator, the ignoring of which will
cause memory corruption.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD33307803)
2021-11-28 02:56:30 +03:00
Maksim Panchenko
e1eeef5b90 [BOLT][RFC] Use new LLVM license for ADRRelaxationPass
Summary: Fixes facebookincubator/BOLT#271

(cherry picked from FBD33299273)
2021-12-23 10:49:37 -08:00
Rafael Auler
b392ec696b Re-enable Windows build and fix issues
Summary:
Fix missing string header file inclusion and link_fdata find
problem in lit tests. Change root-level tests to require
linux. Re-enable Windows in our root CMakeLists.txt.

(cherry picked from FBD33296290)
2021-12-23 05:59:35 -08:00
Rafael Auler
3652483c8e [BOLTCore] [NFC] Fix braces usages according to LLVM
Summary:
Fix according to Coding Standards doc, section Don't Use
Braces on Simple Single-Statement Bodies of if/else/loop Statements.
This set of changes applies to lib Core only.

(cherry picked from FBD33240028)
2021-12-20 11:07:46 -08:00
Maksim Panchenko
9a689946f2 [bughunter.sh][NFC] Fix license and file description
Summary:
Convert bughunter.sh to the new LLVM license and fix file-description
comments.

(cherry picked from FBD33298823)
2021-12-23 10:14:27 -08:00
Maksim Panchenko
2f09f445b2 [BOLT][NFC] Fix file-description comments
Summary: Fix comments at the start of source files.

(cherry picked from FBD33274597)
2021-12-21 10:21:41 -08:00
Maksim Panchenko
0f32a5e5e1 [BOLT][NFC] Clear HFSort copyright/license
Summary:
Remove the copyright/license message for the code originated from
Facebook.

(cherry picked from FBD33293101)
2021-12-22 23:34:49 -08:00
Amir Ayupov
4eb237e45d [BOLT][DOCS] Updated clang build instructions in OptimizingClang.md
Summary:
Addressing the feedback in https://lists.llvm.org/pipermail/llvm-dev/2021-December/154420.html:
Updated the instructions to reflect LLVM monorepo layout.
Resolved build issues in compiler-rt.

(cherry picked from FBD33242317)
2021-12-20 12:58:53 -08:00
Rafael Auler
07d9e014ed [BOLT] Don't use ld.lld in tests
Summary: Addressing issue 270.

(cherry picked from FBD33255608)
2021-12-21 07:36:35 -08:00
Maksim Panchenko
226c973280 [BOLT][NFC] Remove another unused function
Summary: Remove DataReader::getBranchRange().

(cherry picked from FBD32810933)
2021-12-02 13:41:59 -08:00
Maksim Panchenko
cf624f2e63 [BOLT][NFC] Remove unused function
Summary: Remove BinaryFunction::getBasicBlockOriginalAddress().

(cherry picked from FBD32810127)
2021-12-02 13:14:02 -08:00
Maksim Panchenko
ccb99dd126 [BOLT] Fix profile and tests for nop-removal pass
Summary:
Since nops are now removed in a separate pass, the profile is consumed
on a CFG with nops. If previously a profile was generated without nops,
the offsets in the profile could be different if branches included nops
either as a source or a destination.

This diff adjust offsets to make the profile reading backwards
compatible.

(cherry picked from FBD33231254)
2021-12-18 17:05:00 -08:00
Vladislav Khmelevsky
08f56926c2 [BOLT] Move disassemble optimizations to optimization passes
Summary:
The patch moves the shortenInstructions and nop remove to separate binary
passes. As a result when llvm-bolt optimizations stage will begin the
instructions of the binary functions will be absolutely the same as it
was in the binary. This is needed for the golang support by llvm-bolt.
Some of the tests must be changed, since bb alignment nops might create
unreachable BBs in original functions.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD32896517)
2021-12-18 17:03:35 -08:00
Rafael Auler
46e93fb427 Fix frameopt crash when processing POPF
Summary: POPF instruction was triggering an assertion in our analysis.

(cherry picked from FBD33141809)
2021-12-15 13:29:46 -08:00
Elvina Yakubova
4a4045f740 [PR] Fix update-debug-sections for AArch64
Summary:
This patch adds AArch64 relocations handling in case updating of
debug sections is enabled

Elvina Yakubova,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD33077609)
2021-12-08 16:53:38 +03:00
Rafael Auler
283a87743e Fix install-bolt_rt dependencies
Summary:
This rule depends on bolt target, since it will run
install-bolt as well.

(cherry picked from FBD33152071)
2021-12-15 18:53:52 -08:00
Maksim Panchenko
40c2e0fafe [BOLT][NFC] Reformat with clang-format
Summary: Selectively apply clang-format to BOLT code base.

(cherry picked from FBD33119052)
2021-12-14 16:52:51 -08:00
Rafael Auler
5fc8adb529 Add bolt target to cmake
Summary:
Create a new high-level target named bolt that builds all
BOLT artifacts, as well as a install-bolt target that installs them.

(cherry picked from FBD33133002)
2021-12-15 10:16:58 -08:00
Amir Ayupov
6aa735ceaf [BOLT] Split functions: support fragments with multiple parents
Summary:
Gracefully handle binaries with split functions where two fragments are folded
into one, resulting in a fragment with two parent functions.

This behavior is expected in GCC8+ with -O2 optimization level, where both
function splitting and ICF are enabled by default.

On the BOLT side, the changes are:
- BinaryFunction: allow multiple parent fragments:
  - `ParentFragment` --> `ParentFragments`,
  - `setParentFragment` --> `addParentFragment`.
- BinaryContext:
  - `populateJumpTables`: mark fragments to be skipped later,
  - `registerFragment`: add a name heuristic check, return false if it failed,
  - `processInterproceduralReferences`: check if `registerFragment`
succeeded, otherwise issue a warning,
  - `skipMarkedFragments`: move out fragment traversal and skipping from
  `populateJumpTables` into a separate function.

This change fixes an issue where unrelated functions might be registered
as fragments:

```
BOLT-WARNING: interprocedural reference between unrelated fragments:
bad_gs/1(*2) and amd_decode_mce.cold.27/1(*2)
```

(Linux kernel binary)

(cherry picked from FBD32786688)
2021-12-01 21:14:56 -08:00
Maksim Panchenko
69706eafab [BOLT] Refactor BinaryBasicBlock to use ADT
Summary:
Refactor members of BinaryBasicBlock. Replace some std containers with
ADT equivalents. The size of BinaryBasicBlock on x86-64 Linux is reduced
from 232 bytes to 192 bytes.

(cherry picked from FBD33081850)
2021-12-09 11:53:12 -08:00
Rafael Auler
bb201ca3e8 Disable Windows build
Summary:
Build is already broken because VS fails to locate
llvm-boltdiff when running tests, and VS also complains that
include/bolt/Passes/InstrumentationSummary.h is lacking an include
string header. Disable this until we have a Windows buildbot to make
sure this build is sane.

(cherry picked from FBD33039972)
2021-12-10 21:19:18 -08:00
Maksim Panchenko
ebe51c4d23 [BOLT] Use more ADT data structures for BinaryFunction
Summary:
Switched members of BinaryFunction to ADT where it was possible and
made sense. As a result, the size of BinaryFunction on x86-64 Linux
reduced from 1624 bytes to 1448.

(cherry picked from FBD32981555)
2021-12-08 22:59:09 -08:00
Maksim Panchenko
a73b1b7289 [BOLT][NFC] Clear HFSort copyright/license
Summary:
Remove the copyright/license message for the code originated from
Facebook.

(cherry picked from FBD32998404)
2021-12-09 12:24:16 -08:00
Alexander Yermolovich
1417f607bd [BOLT][DWARF] Fix for abbrev check in DWP case
Summary:
For DWP case the AbbreviationsOffset is the offset of the abbrev
 contribution in the DWP file, so can be none zero.

(cherry picked from FBD32961240)
2021-12-08 12:04:45 -08:00
Rafael Auler
a2a8fb10a7 Add code owners file
Summary: Make explicit the contact of component owners.

(cherry picked from FBD32904304)
2021-12-06 17:54:10 -08:00
Maksim Panchenko
b73c87bc4f [BOLT][DWARF] Force allocation of debug_line in RuntimeDyld
Summary:
Currently, RuntimeDyld will not allocate a section without relocations
even if such a section is marked allocatable and defines symbols.

When we emit .debug_line for compile units with unchanged code, we
output original (input) data, without relocations. If all units are
emitted in this way, we will have no relocations in the emitted
.debug_line. RuntimeDyld will not allocate the section and as a result
we will write an empty .debug_line section.

To workaround the issue, always emit a relocation of RELOC_NONE type
when emitting raw contents to debug_line.

(cherry picked from FBD32909869)
2021-12-06 23:32:40 -08:00
Maksim Panchenko
cbf530bf41 [BOLT] Add pass to normalize CFG
Summary:
Some optimizations may remove all instructions in a basic block.

The pass will cleanup the CFG afterwards by removing empty basic
blocks and merging duplicate CFG edges.

The normalized CFG is printed under '-print-normalized' option.

(cherry picked from FBD32774360)
2021-12-01 13:57:50 -08:00
Amir Ayupov
fd71cc5163 [BOLT][TESTS] Move debugTypesBug.s test into binary tests
Summary:
Remove the test and its inputs.
(cherry picked from FBD32855788)
2021-12-03 16:57:24 -08:00
Amir Ayupov
b69d487a62 [BOLT][NFC] Remove unused MCPlusBuilder::isEnter
Summary: Remove unused code identified via coverage report.

(cherry picked from FBD32818608)
2021-12-02 17:23:58 -08:00
Amir Ayupov
8e632eae56 [BOLT][NFC] Remove unused MCPlusBuilder::createIndirectCall method
Summary: Remove unused code identified via coverage report.

(cherry picked from FBD32818329)
2021-12-02 17:19:33 -08:00
Amir Ayupov
02145d20ab [BOLT] Tail duplication: disable const/copy propagation by default as a workaround
Summary:
Disable const/copy propagation as a bug workaround.
Also add the debug logging in aggressive duplication.

(cherry picked from FBD32774744)
2021-12-01 14:05:05 -08:00
Maksim Panchenko
4f91538f57 [BOLT][NFC] Remove misleading debug message
Summary:
The debug message for the last fall-through block was printed under the
reverse condition, i.e. when the block was not a fall-through. Remove
the debug message. If we'll need such information, we can add a pass
with more analysis, i.e. checking the last instruction, if the block is
reachable, etc.

(cherry picked from FBD32670816)
2021-11-25 13:14:16 -08:00
Amir Ayupov
eb9f4eb6ab [BOLT][NFC] Better diagnostics for unsupported relocation types
Summary: Print the relocation name instead of just the number.

(cherry picked from FBD32704832)
2021-11-29 13:20:03 -08:00
Amir Ayupov
76cd07f9e4 [BOLT] Tail Duplication: fix jump table check
Summary: The intent is clearly to check the current basic block.

(cherry picked from FBD32658103)
2021-11-24 15:39:24 -08:00
Amir Ayupov
7261655d2c [BOLT] Tail Duplication: skip unreachable blocks
Summary:
TailDuplication::isInCacheLine makes the assumption that the block
has a valid layout index, which is not the case for unreachable blocks.
Add a check for a valid layout index.

(cherry picked from FBD32659755)
2021-11-24 16:13:42 -08:00
Vladislav Khmelevsky
4e4ef2f3e7 [PR] Disable stack protection in runtime libraries
Summary:
In some of the system stack protection is enabled by default, which will
lead in extra symbols dependencies, which we want to avoid.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD32478426)
2021-11-16 16:34:33 +03:00
Rafael Auler
a23726bb33 [BOLT] Fix crash when trying to resolve external symbols for runtime libs
Summary:
As pointed out by Vladislav in issue 217, if our RTDyld-based
linker fails to locate a symbol, it will crash with segfault. Fix that.

(cherry picked from FBD32481543)
2021-11-16 16:47:02 -08:00
Amir Ayupov
e4ccdfe84f [BOLT] Import bughunter script
Summary: Import the script and update it to use `-funcs-file-no-regex`

(cherry picked from FBD32636025)
2021-11-23 20:48:50 -08:00
Amir Ayupov
d474dbdfcb [BOLT][NFC] Use function names passed in -funcs-no-regex as-is
Summary:
Currently there are two issues rendering the use of bughunter/BOLT on a binary
with a large number of functions (100k) impossible:
1) `selectFunctionsToProcess` has O(binary_fn * force_fn) run-time, which is up
to quadratic with the number of functions in the binary.
2) It unnecessarily treats supplied function names as regexes.

This diff proposes the following changes to address the issue:
1. Add two options that treat function names as is, not as regexes, matching
bughunter usage model: `-funcs-no-regex`/`-funcs-file-no-regex`.
These options are complementary to `-funcs`/`-funcs-file` and `-skip-funcs`/
`-skip-funcs-file`. `funcs` takes precedence over `funcs-no-regex`.
2. Use string set to speed up function eligibility checking with
`-funcs-file-no-regex` to O(binary_fn * log force_fn).

(cherry picked from FBD28917225)
2021-06-04 18:49:29 -07:00
Amir Ayupov
e9ee2ca1fa [BOLT][TEST] Fix runtime/X86/retpoline-synthetic.test
Summary:
Restructure the test to prevent command echo from getting to check
statements.

(cherry picked from FBD32635888)
2021-11-23 20:33:50 -08:00
Vladislav Khmelevsky
a944a487ae [PR] Fix ShrinkWrapping pop order
Summary:
The push and pop instructions might have wrong reorder due to this
error. Thanks rafaelauler for the provided test case.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD32478348)
2021-11-14 02:23:20 +03:00
Alexander Yermolovich
68b0003ee3 [BOLT][DWARF] Fix for Unsupported Debug section: debug_line.dwo warning
Summary: Probably copy and paste mistake or something.

(cherry picked from FBD32625751)
2021-11-23 11:52:25 -08:00
Maksim Panchenko
63bebdd813 [BOLT][NFC] Change guard macros in headers
Summary: Change macro names to match the new directory structure.

(cherry picked from FBD32540857)
2021-11-18 12:21:38 -08:00
Rafael Auler
2ccea6eac3 Fix shared build
Summary:
We were not tracking -DBUILD_SHARED_LIBS=ON and we introduced
some commits that break that by not specifying the correct dependencies.
Fix that.

(cherry picked from FBD32453377)
2021-11-15 20:01:48 -08:00
Rafael Auler
2f3285989e [BOLT] Fix tailcall-traps and basic-instr tests on ubuntu
Summary: These tests are failing on opensource ubuntu.

(cherry picked from FBD32514489)
2021-11-17 15:38:36 -08:00
Amir Ayupov
c7f8adb87f [BOLT][TEST] Add llvm-boltdiff to build/test requirements
Summary: llvm-boltdiff is required for `runtime/meta-merge-fdata.test`

(cherry picked from FBD32442220)
2021-11-15 13:53:18 -08:00
Amir Ayupov
1d0a276c72 [BOLT][TEST] Import small tests
Summary: Imported small internal tests.

(cherry picked from FBD32405870)
2021-11-12 15:38:45 -08:00
Amir Ayupov
7ea61dab03 [BOLT][TEST] Reduce vararg.test
Summary: Reduce assembly inputs to vararg.test using CReduce

(cherry picked from FBD32405869)
2021-11-11 20:05:09 -08:00
Marius Wachtler
3b00a3a2f8 [PR] instr: change assert to allow FD 0 return by __open()
Summary: In some cases __open() is returning 0 for me.  The open
syscall will return a negative number (-1) on error so 0 should be
valid.

This happens when one compiles a BOLT instrumented executable of
pyston (python implementation) and afterwards pip installs a package
which needs to be compiled and sets CFLAGS=-pipe.  I could not reduce
it down to a small testcase but I guess it one can trigger when
manually closing std{in, out, err}.  Everything seems to work
normally when disabling the assert for 0 in getBinaryPath() - I
decided to also modify the second case in readDescriptions() even
though I did not run into that one yet.

(cherry picked from FBD32409548)
2021-11-05 16:47:30 +01:00
Rafael Auler
ae585be11c [BOLT] Fix Windows build
Summary:
Make BOLT build in VisualStudio compiler and run without
crashing on a simple test. Other tests are not running.

(cherry picked from FBD32378736)
2021-11-11 18:14:53 -08:00