Commit Graph

134 Commits

Author SHA1 Message Date
Diego Novillo
aa097db5de Sample profiles - Re-arrange binary format to emit head samples only on top functions.
The number of samples collected at the head of a function only make
sense for top-level functions (i.e., those actually called as opposed to
being inlined inside another).

Head samples essentially count the time spent inside the function's
prologue.  This clearly doesn't make sense for inlined functions, so we
were always emitting 0 in those.

llvm-svn: 250539
2015-10-16 18:54:35 +00:00
Diego Novillo
52c1392c42 Sample Profiles - Adjust integer types. Mostly NFC.
This adjusts all integers in the reader/writer to reflect the types
stored on profile files. They should all be unsigned 32-bit or 64-bit
values. Changed all associated internal types to be uint32_t or
uint64_t.

The only place that needed some adjustments is in the sample profile
transformation. Altough the weight read from the profile are 64-bit
values, the internal API for branch weights only accepts 32-bit values.
The pass now saturates weights that overflow uint32_t.

llvm-svn: 250427
2015-10-15 16:36:21 +00:00
Diego Novillo
4fd12b32db Sample profiles - Add documentation for binary profile encoding. NFC.
This adds documentation for the binary profile encoding and moves the
documentation for the text encoding into the header file
SampleProfReader.h.

llvm-svn: 250309
2015-10-14 18:36:30 +00:00
Diego Novillo
5316879830 Sample profile reader - remove dead code. NFC.
This removes old remnants from the gcov reader. I missed these when I
re-wrote it recently.

llvm-svn: 250242
2015-10-13 22:48:48 +00:00
Diego Novillo
c53fb25fe1 Sample profiles - Add a name table to the binary encoding.
Binary encoded profiles used to encode all function names inline at
every reference.  This is clearly suboptimal in terms of space.  This
patch fixes this by adding a name table to the header of the file.

llvm-svn: 250241
2015-10-13 22:48:46 +00:00
Xinliang David Li
3661a98f5d Fix a couple of comments; NFC
llvm-svn: 250177
2015-10-13 16:35:59 +00:00
Nathan Slingerland
3dad6f16f9 [ProfileData] Test commit for slingn
This is a test of the LLVM commit system. In the event of a real commit there would be some useful code changes.

llvm-svn: 249972
2015-10-11 13:30:56 +00:00
Diego Novillo
550aed332d Add inline stack streaming to binary sample profiles.
With this patch we can now read and write inline stacks in sample
profiles to the binary encoded profiles.

In a subsequent patch, I will add a string table to the binary encoding.
Right now function names are emitted as strings every time we find them.
This is too bloated and will produce large files in applications with
lots of inlining.

llvm-svn: 249861
2015-10-09 17:54:24 +00:00
Diego Novillo
c49885ed99 Re-apply r249644: Handle inline stacks in gcov-encoded sample profiles.
This fixes memory allocation problems by making the merge operation keep
the profile readers around until the merged profile has been emitted.
This is needed to prevent the inlined function names to disappear from
the function profiles. Since all the names are kept as references, once
the reader disappears, the names are also deallocated.

Additionally, XFAIL on big-endian architectures. The test case uses a
gcov file generated on a little-endian system.

llvm-svn: 249724
2015-10-08 19:40:37 +00:00
Diego Novillo
cd224d43cd Revert "Handle inline stacks in gcov-encoded sample profiles."
This reverts commit r249644.

The buildbots are failing the new test I added. Investigating.

llvm-svn: 249648
2015-10-08 01:17:26 +00:00
Diego Novillo
87a89a64fa Handle inline stacks in gcov-encoded sample profiles.
This patch adds support for reading sample profiles with inline stacks.
Inline stacks in a profile are generated when the sampled binary has
samples in inlined functions.

For instance, if main() calls foo() and foo() calls bar(), and bar() is
inlined into foo() and foo() inlined into main(), the profile may look
something like:

main total:364084 head:0
  [ ... ]
  2.3: _Z3fool total:243786
    1: 60149
    1.2: 38568
    1.4: 46511
    1.7: _Z3bari total:98558
      1.1: 52672
      1.2: 45886

At line 2, discriminator 3, main() calls foo(). In turn, foo() calls
bar() at line 1, discriminator 7.

In the textual format, this stacking of inline calls is represented
with indentation.

With this change, LLVM can now read sample profile files generated by
the create_gcov tool from https://github.com/google/autofdo.

llvm-svn: 249644
2015-10-08 00:39:11 +00:00
Diego Novillo
4cb50fee06 Remove AutoFDO profile handling for GCC's LIPO. NFC.
Given the work we are doing on ThinLTO, we will never need to support
module groups and working sets in GCC's implementation of LIPO. These
are currently dead code, and will continue to be so.

llvm-svn: 249351
2015-10-05 21:08:05 +00:00
Justin Bogner
894a1720db InstrProf: Don't call std::unique twice here
llvm-svn: 248872
2015-09-30 02:02:08 +00:00
Dehao Chen
0024ada446 http://reviews.llvm.org/D13145
Support hierarachical sample profile format.

llvm-svn: 248865
2015-09-30 00:42:46 +00:00
Justin Bogner
b34b76538a InstrProf: Support for value profiling in the indexed profile format
Add support to the indexed instrprof reader and writer for the format
that will be used for value profiling.

Patch by Betul Buyukkurt, with minor modifications.

llvm-svn: 248833
2015-09-29 22:13:58 +00:00
Diego Novillo
3ccc9d9f6a GCC AutoFDO profile reader - Initial support.
This adds enough machinery to support reading simple GCC AutoFDO
profiles. It now supports reading flat profiles (no function calls).
Subsequent patches will add support for:

- Inlined calls (in particular, the inline call stack is not traversed
  to accumulate samples).

- Working sets and modules. These are used mostly for GCC's LIPO
  optimizations, so they're not needed in LLVM atm. I'm not sure that
  we will ever need them. For now, I've if0'd around the calls.

The patch also adds support in GCOV.h for gcov version V704 (generated
by GCC's profile conversion tool).

llvm-svn: 247874
2015-09-17 00:17:24 +00:00
Alexander Kornienko
f993659b8f Revert r240137 (Fixed/added namespace ending comments using clang-tidy. NFC)
Apparently, the style needs to be agreed upon first.

llvm-svn: 240390
2015-06-23 09:49:53 +00:00
Justin Bogner
bc637e62b6 Re-apply "InstrProf: When reading, copy the data instead of taking a reference. NFC"
This version fixes a missing include that MSVC noticed and
clarifies the ownership of the counter buffer that's passed to
InstrProfRecord.

This restores r240206, which was reverted in r240208.

Patch by Betul Buyukkurt.

llvm-svn: 240360
2015-06-22 23:58:05 +00:00
Frederic Riss
0c3b9888f7 [Object] Search for architecures by name in MachOUniversalBinary::getObjectForArch()
The reason we need to search by name rather than by Triple::ArchType
is to handle subarchitecture correclty. There is no different ArchType
for the x86_64h architecture (it identifies itself as x86_64), or for
the various ARM subarches. The only way to get to the subarch slice
in an universal binary is to search by name.

This issue led to hard to debug and transient symbolication failures
in Asan tests (it mostly works, because the files are very similar).

This also affects the Profiling infrastucture as it is the other user
of that API.

Reviewers: samsonov, bogner

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D10604

llvm-svn: 240339
2015-06-22 21:33:24 +00:00
Justin Bogner
5391d5c278 Revert "InstrProf: When reading, copy the data instead of taking a reference. NFC"
Seems like MSVC doesn't like this:

  InstrProf.h(49) : error C2614: 'llvm::InstrProfRecord' : illegal member initialization: 'Hash' is not a base or member

This reverts r240206.

llvm-svn: 240208
2015-06-20 01:37:56 +00:00
Justin Bogner
f4084a2560 InstrProf: When reading, copy the data instead of taking a reference. NFC
This consolidates the logic to read instrprof records into the on disk
hash table's lookup trait and makes us copy the counter data instead
of taking references to it as we read. This will simplify further
changes to the format.

Patch by Betul Buyukkurt.

llvm-svn: 240206
2015-06-20 01:26:04 +00:00
Alexander Kornienko
40cb19d802 Fixed/added namespace ending comments using clang-tidy. NFC
The patch is generated using this command:

tools/clang/tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
  -checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \
  llvm/lib/


Thanks to Eugene Kosov for the original patch!

llvm-svn: 240137
2015-06-19 15:57:42 +00:00
Duncan P. N. Exon Smith
28058eb1bf modules: Add explicit dependency on intrinsics_gen
`LLVM_ENABLE_MODULES` builds sometimes fail because `Intrinsics.td`
needs to regenerate `Instrinsics.h` before anyone can include anything
from the LLVM_IR module.  Represent the dependency explicitly to prevent
that.

llvm-svn: 239796
2015-06-16 00:44:12 +00:00
Rafael Espindola
da86c89e4a Don't use std::errc.
As noted on Errc.h:

// * std::errc is just marked with is_error_condition_enum. This means that
//   common patters like AnErrorCode == errc::no_such_file_or_directory take
//   4 virtual calls instead of two comparisons.

And on some libstdc++ those virtual functions conclude that

------------------------
int main() {
  std::error_code foo = std::make_error_code(std::errc::no_such_file_or_directory);
  return foo == std::errc::no_such_file_or_directory;
}
-------------------------

should exit with 0.

llvm-svn: 239683
2015-06-13 17:23:04 +00:00
Justin Bogner
8bdca83c33 InstrProf: Fix reading of consecutive 32 bit coverage maps
When we generate coverage data, we explicitly set each coverage map's
alignment to 8 (See InstrProfiling::lowerCoverageData), but when we
read the coverage data, we assume consecutive maps are exactly
adjacent. When we're dealing with 32 bit, maps can end on a 4 byte
boundary, causing us to think the padding is part of the next record.

Fix this by adjusting the buffer to an appropriately aligned address
between records.

This is pretty awkward to test, as it requires a binary with multiple
coverage maps to hit, so we'd need to check in multiple source files
and a binary blob as inputs.

llvm-svn: 239129
2015-06-05 01:23:42 +00:00
Justin Bogner
ca8e06d93f InstrProf: Treat functions with a coverage map but no profile as unreached
If we have a coverage mapping but no profile data for a function,
calling it mismatched is misleading. This can just as easily be
unreachable code that was stripped from the binary. Instead, treat
these the same as functions where we have an explicit "zero" coverage
map by setting the count to zero for each mapped region.

llvm-svn: 237298
2015-05-13 22:03:04 +00:00
Justin Bogner
4104eb5169 InstrProf: Simplify looking up sections for coverage data
llvm-svn: 236685
2015-05-07 00:31:58 +00:00
Justin Bogner
aa99b531f2 InstrProf: Give coverage its own errors instead of piggy backing on instrprof
Since the coverage mapping reader and the instrprof reader were
emitting a shared set of error codes, the error messages you'd get
back from llvm-cov were ambiguous about what was actually wrong. Add
another error category to fix this.

I've also improved the wording on a couple of the instrprof errors,
for consistency.

llvm-svn: 236665
2015-05-06 23:19:35 +00:00
Justin Bogner
a01489eaf6 InstrProf: Remove a function that just returns its argument (NFC)
llvm-svn: 236664
2015-05-06 23:15:55 +00:00
Justin Bogner
8d160aaf1d InstrProf: Strip filename prefixes from the names we display for coverage
For consumers of coverage data, any filename prefixes we store in the
profile data are just noise. Strip this prefix if it exists.

llvm-svn: 236558
2015-05-05 23:44:48 +00:00
Benjamin Kramer
6a9aa608f1 Re-sort includes with sort-includes.py and insert raw_ostream.h where it's used.
llvm-svn: 232998
2015-03-23 19:32:43 +00:00
Justin Bogner
5878b9185f InstrProf: Fix CoverageMappingReader on big endian
This makes the reader check the endianness of the object file its
given and behave appropriately. For the test I dug up a really old
linker and created a ppc-apple-darwin file for llvm-cov to read.

llvm-svn: 232422
2015-03-16 21:40:18 +00:00
Justin Bogner
6484f6de8c InstrProf: Do a better job of reading coverage mapping data.
This code was casting regions of a memory buffer to a couple of
different structs. This is wrong in a few ways:

1. It breaks aliasing rules.
2. If the buffer isn't aligned, it hits undefined behaviour.
3. It completely ignores endianness differences.
4. The structs being defined for this aren't specifying their padding
   properly, so this doesn't even represent the data properly on some
   platforms.

This commit is mostly NFC, except that it fixes reading coverage for
32 bit binaries as a side effect of getting rid of the mispadded
structs. I've included a test for that.

I've also baked in that we only handle little endian more explicitly,
since that was true in practice already. I'll fix this to handle
endianness properly in a followup commit.

llvm-svn: 232346
2015-03-16 06:55:45 +00:00
Justin Bogner
ba70a38423 InstrProf: Teach llvm-cov to handle universal binaries when given -arch
llvm-svn: 231902
2015-03-11 02:30:51 +00:00
Justin Bogner
cdce389220 InstrProf: Allow hexadecimal function hashes in proftext format
llvm-svn: 231685
2015-03-09 18:54:49 +00:00
Benjamin Kramer
ce7baceeed Add missing includes. make_unique proliferated everywhere.
llvm-svn: 230909
2015-03-01 21:28:53 +00:00
Justin Bogner
a0537de57c InstrProf: Simplify the construction of BinaryCoverageReader
Creating BinaryCoverageReader is a strange and complicated dance where
the constructor sets error codes that member functions will later
read, and the object is in an invalid state if readHeader isn't
immediately called after construction.

Instead, make the constructor private and add a static create method
to do the construction properly. This also has the benefit of removing
readHeader completely and simplifying the interface of the object.

llvm-svn: 230676
2015-02-26 20:06:28 +00:00
Justin Bogner
fd3c2961be InstrProf: Rename ObjectFileCoverageMappingReader to BinaryCoverageReader
The current name is long and confusing. A shorter one is both easier
to understand and easier to work with.

llvm-svn: 230675
2015-02-26 20:06:24 +00:00
Justin Bogner
2218fa75da InstrProf: Remove dead code in CoverageMappingReader
Remove a default argument that's never passed and a constructor that's
never called.

llvm-svn: 230563
2015-02-25 22:44:50 +00:00
Justin Bogner
720e999911 Disallow implicit conversions from None to integer types
This fixes an error introduced in r228934 where None was converted to
an int instead of the int being converted to an Optional as intended.
We make that sort of mistake a compile error by changing NoneType into
a scoped enum.

Finally, provide a static NoneType called None to avoid forcing all
users to spell it NoneType::None.

llvm-svn: 229980
2015-02-20 07:28:28 +00:00
Justin Bogner
485508085b InstrProf: Don't combine expansion regions with code regions
This was leading to duplicate counts when a code region happened to
overlap exactly with an expansion. The combining behaviour only makes
sense for code regions.

llvm-svn: 229723
2015-02-18 19:01:06 +00:00
Justin Bogner
8ab6d199d1 InstrProf: Handle unknown functions if they consist only of zero-regions
This comes up when we generate coverage for a function but don't end
up emitting the function at all - dead static functions or inline
functions that aren't referenced in a particular TU, for example. In
these cases we'd like to show that the function was never called,
which is trivially true.

llvm-svn: 229717
2015-02-18 18:40:46 +00:00
Justin Bogner
dfdc3baaba InstrProf: Make CoverageMapping testable and add a basic unit test
Make CoverageMapping easier to create, so that we can write targeted
unit tests for its internals, and add a some infrastructure to write
these tests. Finally, add a simple unit test for basic functionality.

llvm-svn: 229709
2015-02-18 18:01:14 +00:00
Justin Bogner
a033796bf5 Re-apply "InstrProf: Add unit tests for the profile reader and writer"
Have the InstrProfWriter return a MemoryBuffer instead of a
std::string. This fixes the alignment issues the reader would hit, and
it's a more appropriate type for this anyway.

I've also removed an ugly helper function that's not needed since
we're allowing initializer lists now, and updated some error code
checks based on MSVC's issues with r229473.

This reverts r229483, reapplying r229478.

llvm-svn: 229602
2015-02-18 01:58:17 +00:00
Benjamin Kramer
21bee91af5 Prefer SmallVector::append/insert over push_back loops.
Same functionality, but hoists the vector growth out of the loop.

llvm-svn: 229500
2015-02-17 15:29:18 +00:00
Justin Bogner
892964de44 Revert "InstrProf: Add unit tests for the profile reader and writer"
This added API to the InstrProfWriter to write to a string so I could
write unittests without using temp files. This doesn't really work,
since the format has tighter alignment requirements than a char.

This reverts r229478 and its follow-up, r229481.

llvm-svn: 229483
2015-02-17 09:21:43 +00:00
Justin Bogner
5a941203c6 Re-apply "InstrProf: Add unit tests for the profile reader and writer"
Add these tests again, but use va_list instead of initializer lists.

This reverts r229456, reapplying r229455.

llvm-svn: 229478
2015-02-17 07:50:59 +00:00
Justin Bogner
7e3c504766 Revert "InstrProf: Add unit tests for the profile reader and writer"
Looks like the bots don't like my initializer lists.

This reverts r229455

llvm-svn: 229456
2015-02-16 23:31:07 +00:00
Justin Bogner
116967eecd InstrProf: Add unit tests for the profile reader and writer
This required some minor API to be added to these types to avoid
needing temp files.

Also, I've used initializer lists in the tests, as MSVC 2013 claims to
support them. I'll redo this without them if the bots complain.

llvm-svn: 229455
2015-02-16 23:27:48 +00:00
Justin Bogner
bb1f720b6c InstrProf: Use ErrorOr for IndexedInstrProfReader::create (NFC)
The other InstrProfReader::create factories were updated to return
ErrorOr in r221120, and it's odd for these APIs not to match.

llvm-svn: 229433
2015-02-16 21:28:58 +00:00