This patch refactors the DWARF section dumpers. When dumping a DWARF
section, if the DWARF parser fails to parse the section, we will dump it
as a raw content section. This patch also fixes a bug in
DWARFYAML::Data::isEmpty(). Finally, a test case that tests dumping the
__debug_aranges section is added.
Reviewed By: jhenderson, grimar
Differential Revision: https://reviews.llvm.org/D85506
The switch from llvm::cl to OptTable (D83530) dropped --version, which
is needed by some users.
This patch also adds a -v alias, which is available in GNU addr2line.
The version dumping is similar to llvm-objcopy --version (exotic):
```
llvm-symbolizer
LLVM (http://llvm.org/):
LLVM version 12.0.0git
Optimized build with assertions.
Default target: x86_64-unknown-linux-gnu
Host CPU: skylake-avx512
```
Reviewed By: dyung, jhenderson
Differential Revision: https://reviews.llvm.org/D85624
Although the DWARF specification states that .debug_aranges entries
can't have length zero, these can occur in the wild. There's no
particular reason to enforce this part of the spec, since functionally
they have no impact. The patch removes the error and introduces a new
warning for premature terminator entries which does not stop parsing.
This is a relanding of cb3a598c87db, adding the missing obj2yaml part
that was needed.
Fixes https://bugs.llvm.org/show_bug.cgi?id=46805. See also
https://reviews.llvm.org/D71932 which originally introduced the error.
Reviewed by: ikudrin, dblaikie, Higuoxing
Differential Revision: https://reviews.llvm.org/D85313
It adds the proper warnings reporting and updates the mips-reginfo.test to
remove using of the precompiled binary.
Differential revision: https://reviews.llvm.org/D85511
Implemented the `llvm-profdata overlap` feature for sample profiles. It reports weighted //similarity// and unweighted //overlap// metrics at program and function level for two input profiles. Similarity metrics are symmetric with regards to the order of two input profiles. By default, the tool only reports program-level summary. Users can look into function-level details via additional options `--function`, `--similarity-cutoff`, and `--value-cutoff`.
The similarity metrics are designed as follows:
* Program-level summary
* Whole program profile similarity is an aggregate over function-level similarity `FS`: `PS = sum(FS(A) * avg_weight(A))` for all function `A`.
* Whole program sample overlap: `PSO = common_samples / total_samples`.
* Function overlap: `FO = #common_function / #total_function`.
* Hot-function overlap: `HFO = #common_hot_function / #total_hot_function`.
* Hot-block overlap: `HBO = #common_hot_block / #total_hot_block`.
* Function-level details
* Function-level similarity is an aggregate over line/block-level similarities `BS` of all sample lines/blocks in the function, weighted by the closeness of the function's weights in two profiles: `FS = sum(BS(i)) * (1 - weight_distance(A))`.
* Function-level sample overlap: `FSO = common_samples / total_samples` for samples in the function.
Reviewed By: wenlei, hoyFB, wmi
Differential Revision: https://reviews.llvm.org/D83852
It is not enough to replace all uses of users of the function with undef,
the users, we only drop instruction users, so they may stick around.
Let's try different approach - first drop bodies for all the functions
we will drop, which should take care of blockaddress issue the previous
rewrite was dealing with; then, after dropping *all* such bodies,
replace remaining uses with undef (thus all the uses are either
outside of functions, or are in kept functions)
and then finally drop functions.
This seems to work, and passes the *existing* test coverage,
but it is possible that a new issue will be discovered later :)
A new (previously crashing) test added.
Add support for `-D` and `-U` options for llvm-libtool-darwin. `-D`
allows for using zero for timestamps and UIDs/GIDs. `-U` allows for
using actual timestamps and UIDs/GIDs.
Reviewed by jhenderson, smeenai
Differential Revision: https://reviews.llvm.org/D84209
Add support for `-filelist` option for llvm-libtool-darwin. `-filelist`
option allows for passing in a file containing a list of filenames.
Reviewed by jhenderson, smeenai
Differential Revision: https://reviews.llvm.org/D84206
Add support for constant MachO::CPU_SUBTYPE_ARM64_V8. This constant is
needed so as to match `llvm-libtool-darwin`'s behavior to that of
cctools' libtool when `-arch_only` flag is passed in on command line.
Reviewed by jhenderson, alexshap, smeenai
Differential Revision: https://reviews.llvm.org/D85041
Add symlinks for `llvm-libtool-darwin` and
`llvm-install-name-tool`.
Reviewed by jhenderson, smeenai
Differential Revision: https://reviews.llvm.org/D85054
dumpDebugStrings() and dumpDebugAbbrev() are no longer used in
macho2yaml. This patch helps remove them.
Reviewed By: grimar
Differential Revision: https://reviews.llvm.org/D85496
Unit.Format, Unit.Version and Unit.AddrSize are replaced with
dwarf::FormParams in D84496 to get rid of unnecessary functions
getOffsetSize() and getRefSize(). However, that change makes it
difficult to make AddrSize optional (Optional<uint8_t>). This change
pulls out dwarf::FormParams from DWARFYAML::Unit and use it as a helper
struct in DWARFYAML::emitDebugInfo().
Reviewed By: jhenderson, MaskRay
Differential Revision: https://reviews.llvm.org/D85296
The `decode_relrs` helper is declared as:
`Expected<std::vector<Elf_Rel>> decode_relrs(Elf_Relr_Range relrs) const;`
it never returns an error though and hence can be simplified to return
a vector.
Differential revision: https://reviews.llvm.org/D85302
This patch adds support for dumping DWARF sections to obj2yaml. The
.debug_aranges section is used to illustrate the basic idea.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D85094
Currently, we only test the `--stackmap` option here:
https://github.com/llvm/llvm-project/blob/master/llvm/test/Object/stackmap-dump.test
it uses a precompiled MachO binary currently and I've found no tests for this option for ELF.
The implementation also has issues. For example, it might assert on a wrong version
of the .llvm-stackmaps section. Or it might crash on an empty or truncated section.
This patch introduces a new tools/llvm-readobj/ELF test file as well as implements a few
basic checks to catch simple crashes/issues
It also eliminates `unwrapOrError` calls in `printStackMap()`.
Differential revision: https://reviews.llvm.org/D85208
This can practically easily be a product of combining strings with
macros in resource files.
This fixes https://github.com/mstorsjo/llvm-mingw/issues/140.
As string literals within llvm-rc are handled as StringRefs, each
referencing an uninterpreted slice of the input file, with actual
interpretation of the input string (codepage handling, unescaping etc)
done only right before writing them out to disk, it's hard to
concatenate them other than just bundling them up in a vector,
without rearchitecting a large part of llvm-rc.
This matches how the same already is supported in VersionInfoValue,
with a std::vector<IntOrString> Values.
MS rc.exe only supports concatenated string literals in version info
values (already supported), string tables (implemented in this patch)
and user data resources (easily implemented in a separate patch, but
hasn't been requested by any end user yet), while GNU windres supports
string immediates split into multiple strings anywhere (e.g. like
(100 ICON "myicon" ".ico"). Not sure if concatenation in other
statements actually is used in the wild though, in resource files
normally built by GNU windres.
Differential Revision: https://reviews.llvm.org/D85183
D83530 removed --inlining={true,false} which were used by old asan_symbolize.py script.
Add compatibility aliases so that old asan_symbolize.py and sanitizer
binaries can work with new llvm-symbolizer.
Reviewed By: thakis
Differential Revision: https://reviews.llvm.org/D85228
for the advantage outlined by D83639 ([OptTable] Support grouped short options)
Some behavior changes:
* -i={0,false} is removed. Use --no-inlines instead.
* --demangle={0,false} is removed. Use --no-demangle instead
* -untag-addresses={0,false} is removed. Use --no-untag-addresses instead
Added a higher level API OptTable::parseArgs which handles optional
initial options populated from an environment variable, expands response
files recursively, and parses options.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D83530
http://lab.llvm.org:8011/builders/clang-cmake-x86_64-avx2-linux/builds/15718/steps/build%20stage%201/logs/stdio:
FAILED: /usr/bin/c++ -DGTEST_HAS_RTTI=0 -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/llvm-readobj -I/home/ssglocal/clang-cmake-x86_64-avx2-linux/clang-cmake-x86_64-avx2-linux/llvm/llvm/tools/llvm-readobj -Iinclude -I/home/ssglocal/clang-cmake-x86_64-avx2-linux/clang-cmake-x86_64-avx2-linux/llvm/llvm/include -march=broadwell -fPIC -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -ffunction-sections -fdata-sections -O3 -fno-exceptions -fno-rtti -UNDEBUG -std=c++14 -MD -MT tools/llvm-readobj/CMakeFiles/llvm-readobj.dir/ELFDumper.cpp.o -MF tools/llvm-readobj/CMakeFiles/llvm-readobj.dir/ELFDumper.cpp.o.d -o tools/llvm-readobj/CMakeFiles/llvm-readobj.dir/ELFDumper.cpp.o -c /home/ssglocal/clang-cmake-x86_64-avx2-linux/clang-cmake-x86_64-avx2-linux/llvm/llvm/tools/llvm-readobj/ELFDumper.cpp
/home/ssglocal/clang-cmake-x86_64-avx2-linux/clang-cmake-x86_64-avx2-linux/llvm/llvm/tools/llvm-readobj/ELFDumper.cpp: In function ‘llvm::Expected<const llvm::object::Elf_Mips_Options<ELFT>*> readMipsOptions(const uint8_t*, llvm::ArrayRef<unsigned char>&, bool&)’:
/home/ssglocal/clang-cmake-x86_64-avx2-linux/clang-cmake-x86_64-avx2-linux/llvm/llvm/tools/llvm-readobj/ELFDumper.cpp:3374:12: error: parse error in template argument list
if (O->size < ExpectedSize)
Note: I played with godbolt.org and was able to catch the similar "error in template argument list" error when used gcc 4.9.0 with this code.
Fix: try to introduce a variable to store `O->size`, it helped to me in godbolt.
http://lab.llvm.org:8011/builders/clang-cmake-x86_64-avx2-linux/builds/15710/steps/build%20stage%201/logs/stdio
fails with:
/home/ssglocal/clang-cmake-x86_64-avx2-linux/clang-cmake-x86_64-avx2-linux/llvm/llvm/tools/llvm-readobj/ELFDumper.cpp: In function ‘llvm::Expected<const llvm::object::Elf_Mips_Options<ELFT>*> readMipsOptions(const uint8_t*, llvm::ArrayRef<unsigned char>&, bool&)’:
/home/ssglocal/clang-cmake-x86_64-avx2-linux/clang-cmake-x86_64-avx2-linux/llvm/llvm/tools/llvm-readobj/ELFDumper.cpp:3373:19: error: the value of ‘ExpectedSize’ is not usable in a constant expression
if (O->size < ExpectedSize)
^
/home/ssglocal/clang-cmake-x86_64-avx2-linux/clang-cmake-x86_64-avx2-linux/llvm/llvm/tools/llvm-readobj/ELFDumper.cpp:3369:10: note: ‘size_t ExpectedSize’ is not const
size_t ExpectedSize =
^
/home/ssglocal/clang-cmake-x86_64-avx2-linux/clang-cmake-x86_64-avx2-linux/llvm/llvm/tools/llvm-readobj/ELFDumper.cpp:3373:12: error: parse error in template argument list
if (O->size < ExpectedSize)
^
/home/ssglocal/clang-cmake-x86_64-avx2-linux/clang-cmake-x86_64-avx2-linux/llvm/llvm/tools/llvm-readobj/ELFDumper.cpp: In instantiation of ‘llvm::Expected<const llvm::object::Elf_Mips_Options<ELFT>*> readMipsOptions(const uint8_t*, llvm::ArrayRef<unsigned char>&, bool&) [with ELFT = llvm::object::ELFType<(llvm::support::endianness)0u, true>; uint8_t = unsigned char]’:
/home/ssglocal/clang-cmake-x86_64-avx2-linux/clang-cmake-x86_64-avx2-linux/llvm/llvm/tools/llvm-readobj/ELFDumper.cpp:3400:30: required from ‘void {anonymous}::ELFDumper<ELFT>::printMipsOptions() [with ELFT = llvm::object::ELFType<(llvm::support::endianness)0u, true>]’
/home/ssglocal/clang-cmake-x86_64-avx2-linux/clang-cmake-x86_64-avx2-linux/llvm/llvm/tools/llvm-readobj/ELFDumper.cpp:2878:21: required from ‘void {anonymous}::ELFDumper<ELFT>::printArchSpecificInfo() [with ELFT = llvm::object::ELFType<(llvm::support::endianness)0u, true>]’
/home/ssglocal/clang-cmake-x86_64-avx2-linux/clang-cmake-x86_64-avx2-linux/llvm/llvm/tools/llvm-readobj/ELFDumper.cpp:6999:1: required from here
/home/ssglocal/clang-cmake-x86_64-avx2-linux/clang-cmake-x86_64-avx2-linux/llvm/llvm/tools/llvm-readobj/ELFDumper.cpp:3373:5: error: ‘size’ is not a member template function
Fix: add 2 `const` words to variables.
`printMipsOptions()` and the test related has the following issues currently:
1) It does not check the value of Elf_Mips_Options<ELFT>::size field.
2) For ODK_REGINFO options it is possible to read past the end of buffer,
because there is no check against the `sizeof(Elf_Mips_RegInfo<ELFT>)`.
3) The error about the broken size is just printed to the standard output.
4) The binary input is used for the test.
5) There is no testing for multiple options in the .MIPS.options section,
though the code supports it.
6) Only llvm-readobj is tested, but not llvm-readelf.
7) "Unsupported MIPS options tag" message does not reveal the tag ID/name.
This patch fixes all of these points.
Differential revision: https://reviews.llvm.org/D84854
It turns out that findSectionByName can return
const Elf_Shdr * instead of Expected<>, because its
code never returns an error currently (it reports warnings instead).
Differential revision: https://reviews.llvm.org/D85135
See https://lists.llvm.org/pipermail/llvm-dev/2020-July/143373.html
"[llvm-dev] Multiple documents in one test file" for some discussions.
This patch has explored several alternatives. The current semantics are similar to
what @dblaikie proposed.
`split-file filename output` splits the input file into multiple parts separated by
regex `^(.|//)--- filename` and write each part to the file `output/filename`
(`filename` can include path separators).
Use case A (organizing input of different formats (e.g. linker
script+assembly) in one file).
```
# RUN: split-file %s %t
# RUN: llvm-mc %t/asm -o %t.o
# RUN: ld.lld -T %t/lds %t.o -o %t
This is sometimes better than the %S/Inputs/ approach because the user
can see the auxiliary files immediately and don't have to open another file.
# asm
...
# lds
...
```
Use case B (for utilities which don't have built-in input splitting
feature):
```
// RUN: split-file %s %t
// RUN: llc < %t/1.ll | FileCheck %s --check-prefix=CASE1
// RUN: llc < %t/2.ll | FileCheck %s --check-prefix=CASE2
Combing tests prudently can improve readability.
For example, when testing parsing errors if the recovery mechanism isn't possible,
grouping the tests in one file can more readily see test coverage/strategy.
//--- 1.ll
...
//--- 2.ll
...
```
Since this is a new utility, there is no git history concerns for
UpperCase variable names. I use lowerCase variable names like mlir/lld.
Reviewed By: jhenderson, lattner
Differential Revision: https://reviews.llvm.org/D83834
Archives can now be specified as input files the same way that object
files are. Archives will always be linked after all objects (regardless
of the relative order of the inputs) but before any dynamic libraries or
process symbols.
This patch also relaxes matching for slice triples in
StaticLibraryDefinitionGenerator in order to support this feature:
Vendors need not match if the source vendor is unknown.
We have a `findSectionByName` helper that tries to find a section
by it name. It is used in a few places, but never tested.
I'd like to reuse this helper for a different place.
For this, I've changed it to return Expected<> and now it
doesn't use `unwrapOrErr` anymore. It also now a member of
Dumper class and might report warnings.
Differential revision: https://reviews.llvm.org/D84651
The -phony-externals option adds a generator which explicitly defines any
otherwise unresolved externals as null. This transforms link-time
unresolved-symbol errors into potential runtime null pointer accesses
(if an unresolved external is actually accessed during execution).
This option can be useful in -harness mode to avoid having to mock a
large number of symbols that are not reachable at runtime (e.g. unused
methods referenced by a class vtable).
This prevents weak symbols from being immediately dead-stripped when not
directly referenced from the test harneess, enabling use of weak symbols
from the code under test.
Refactoring `Slice` class and function `createUniversalBinary` from
`llvm-lipo` into MachOUniversalWriter. This refactoring is necessary so
as to use the refactored code for creating universal binaries under
llvm-libtool-darwin.
Reviewed by alexshap, smeenai
Differential Revision: https://reviews.llvm.org/D84662
This patch makes the 'debug_aranges' entry optional. If the entry is
empty, yaml2obj will only emit the header for it.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D84921
The -harness option enables new testing use-cases for llvm-jitlink. It takes a
list of objects to treat as a test harness for any regular objects passed to
llvm-jitlink.
If any files are passed using the -harness option then the following
transformations are applied to all other files:
(1) Symbols definitions that are referenced by the harness files are promoted
to default scope. (This enables access to statics from test harness).
(2) Symbols definitions that clash with definitions in the harness files are
deleted. (This enables interposition by test harness).
(3) All other definitions in regular files are demoted to local scope.
(This causes untested code to be dead stripped, reducing memory cost and
eliminating spurious unresolved symbol errors from untested code).
These transformations allow the harness files to reference and interpose
symbols in the regular object files, which can be used to support execution
tests (including fuzz tests) of functions in relocatable objects produced by a
build.
This reduces the number of check-llvm failures by 500.
Ideally we'd have a codegen version of PassRegistry.def, or have all the
codegen passes ported and put into PassRegistry.def. But since that
doesn't exist yet, hardcode the list of codegen IR passes.
There are still codegen passes missing from this list, I'll add them
later as I stumble upon them.
Reviewed By: asbirlea, ychen
Differential Revision: https://reviews.llvm.org/D84872
This introduces the printRelocationsHelper() which now contains the common
code used by both GNU and LLVM output styles.
Differential revision: https://reviews.llvm.org/D83935
This reverts commit d054c7ee2e9f4f98af7f22a5b00a941eb919bd59.
There are discussions about the utility name, its functionality and user interface.
Revert before we reach consensus.
This patch refactors the llvm tools namely, llvm-stress and sancov,
as well as the llvm TableGen utility, to use the new InitLLVM
interface which encapsulates PrettyStackTrace.
This is from https://reviews.llvm.org/D70702, but only for LLVM.
Reviewed-by: Kai
Differential Revision: https://reviews.llvm.org/D83484
Currently, when dumping section headers, llvm-readelf
prints "RELR" for SHT_ANDROID_RELR/SHT_RELR sections.
The behavior was introduced in D47919 and revealed in D84330.
But "SHT_ANDROID_RELR" has a different value from "SHT_RELR".
Also, "SHT_ANDROID_REL/SHT_ANDROID_RELA" are printed as "ANDROID_REL/ANDROID_RELA",
what makes the handling of the "SHT_ANDROID_RELR" inconsistent.
This patch makes llvm-readelf to print "ANDROID_RELR" instead of "RELR".
Differential revision: https://reviews.llvm.org/D84393
PGO profile is usually more precise than sample profile. However, PGO profile
needs to be collected from loadtest and loadtest may not be representative
enough to the production workload. Sample profile collected from production
can be used as a supplement -- for functions cold in loadtest but warm/hot
in production, we can scale up the related function in PGO profile if the
function is warm or hot in sample profile.
The implementation contains changes in compiler side and llvm-profdata side.
Given an instr profile and a sample profile, for a function cold in PGO
profile but warm/hot in sample profile, llvm-profdata will either mark
all the counters in the profile to be -1 or scale up the max count in the
function to be above hot threshold, depending on the zero counter ratio in
the profile. The assumption is if there are too many counters being zero
in the function profile, the profile is more likely to cause harm than good,
then llvm-profdata will mark all the counters to be -1 indicating the
function is hot but the profile is unaccountable. In compiler side, if a
function profile with all -1 counters is seen, the function entry count will
be set to be above hot threshold but its internal profile will be dropped.
In the long run, it may be useful to let compiler support using PGO profile
and sample profile at the same time, but that requires more careful design
and more substantial changes to make two profiles work seamlessly. The patch
here serves as a simple intermediate solution.
Differential Revision: https://reviews.llvm.org/D81981
This patch helps teach llvm-readelf to emit a correct number spaces when
dumping in hex format.
Before this patch, when the hex data doesn't fill the 4th column, some
spaces are missing.
```
Hex dump of section '.sec':
0x00000000 00000000 00000000 00000000 00000000 ................
0x00000010 00000000 00000000 00000000 0000 ..............
```
After this patch:
```
Hex dump of section '.sec':
0x00000000 00000000 00000000 00000000 00000000 ................
0x00000010 00000000 00000000 00000000 0000 ..............
```
Reviewed By: grimar
Differential Revision: https://reviews.llvm.org/D84640
Starting with Skylake, the LBR contains the precise number of cycles between the two
consecutive branches.
Making use of this will hopefully make the measurements more precise than the
existing methods of using RDTSC.
Differential Revision: https://reviews.llvm.org/D77422
New change: check for existence of field `cycles` in perf_branch_entry before enabling this mode.
This should prevent compilation errors when building for older kernel whose headers don't support it.
