Summary:
Adds support for in-memory round-trip of sample profile data along with basic
round trip unit tests. This will also make it easier to include unit tests for
future changes to sample profiling.
Reviewers: davidxl, dnovillo, silvas
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D15211
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@255264 91177308-0d34-0410-b5e6-96231b3b80d8
Different version of indexed format may use different
name uniquing schemes for static functions. Pass the
version info to the name interface so that different
schmes can be picked (for profile lookup).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254838 91177308-0d34-0410-b5e6-96231b3b80d8
With the latest refactoring and code sharing patches landed,
it is possible to unify the value profile implementation between
raw and indexed profile. This is the patch in raw profile reader
that uses the common interface.
Differential Revision: http://reviews.llvm.org/D15056
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254677 91177308-0d34-0410-b5e6-96231b3b80d8
This change adds support for an optional weight when merging profile data with the llvm-profdata tool.
Weights are specified by adding an option ':<weight>' suffix to the input file names.
Adding support for arbitrary weighting of input profile data allows for relative importance to be placed on the
input data from multiple training runs.
Both sampled and instrumented profiles are supported.
Reviewers: dnovillo, bogner, davidxl
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D14547
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254669 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: This changes overflow handling during instrumentation profile merge. Rathar than throwing away records that would result in counter overflow, merged counts are instead clamped to the maximum representable value. A warning about counter overflow is still surfaced to the user as before.
Reviewers: dnovillo, davidxl, silvas
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D14893
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254525 91177308-0d34-0410-b5e6-96231b3b80d8
This is the last step to enable profile runtime to share the same value prof
data format and reader/writer code with llvm host tools. The VP related
data structures are moved to a section in InstrProfData.inc enabled with macro
INSTR_PROF_VALUE_PROF_DATA, and common API implementations are enabled with
INSTR_PROF_COMMON_API_IMPL. There should be no functional change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254235 91177308-0d34-0410-b5e6-96231b3b80d8
Raw profile writer needs to write all data of one kind in one continuous block,
so the buffer needs to be pre-allocated and passed to the writer method in
pieces for function profile data. The change adds the support for raw value data
writing.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254219 91177308-0d34-0410-b5e6-96231b3b80d8
This is one of the many steps to commonize value profiling support between profile
runtime and compiler/llvm tools.
After this change, profiler runtime now can share the same C APIs to do VP
serialization/deseriazation with LLVM host tools (and produces value data
in identical format between indexed and raw profile).
It is not yet enabled in profiler runtime yet.
Also added a unit test case to test runtime profile data serialization/deserialization
interfaces implemented using common closure code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254110 91177308-0d34-0410-b5e6-96231b3b80d8
1. Convert serialization methods using InstrProfRecord as source into C (impl)
interfaces using Closure.
2. Reimplement InstrProfRecord serialization method to use new C interface
as dummy wrapper.
Now it is ready to implement wrapper for runtime value profile data.
(The new code need better source location -- but not changed in this patch to
minimize diffs. )
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254057 91177308-0d34-0410-b5e6-96231b3b80d8
Convert two C++ static member functions to be C APIs. This
is one of the many steps to get ready to share VP writer code
with profiler runtime.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@253999 91177308-0d34-0410-b5e6-96231b3b80d8
The new option is similar to the SampleProfile dump option.
- dump raw/indexed format into text profile format
- merge the profile and output into text profile format.
Note that Value Profiling data text format is not yet designed.
That functionality will be added later.
Differential Revision: http://reviews.llvm.org/D14894
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@253913 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This change refactors two aspects of InstrProfRecord:
1) Add a merge() method to InstrProfRecord (previously InstrProfWriter combineInstrProfRecords()) in order to better encapsulate this functionality and to make the InstrProfRecord and SampleRecord APIs more consistent.
2) Make InstrProfRecord mergeValueProfData() a private method since it is only ever called internally by merge().
Reviewers: dnovillo, bogner, davidxl
Subscribers: silvas, vsk, llvm-commits
Differential Revision: http://reviews.llvm.org/D14786
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@253695 91177308-0d34-0410-b5e6-96231b3b80d8
When dumping function samples or writing them out as text format, it
helps if the samples are emitted sorted by source location. The sorting
of the maps is a bit slow, so we only do it on demand.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@253568 91177308-0d34-0410-b5e6-96231b3b80d8
This change introduces an instrumentation intrinsic instruction for
value profiling purposes, the lowering of the instrumentation intrinsic
and raw reader updates. The raw profile data files for llvm-profdata
testing are updated.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@253484 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This change tries to make the root cause of instrumented profile data merge failures clearer.
Previous:
$ llvm-profdata merge test_0.profraw test_1.profraw -o test_merged.profdata
test_1.profraw: foo: Function count mismatch
test_1.profraw: bar: Function count mismatch
test_1.profraw: baz: Function count mismatch
...
Changed:
$ llvm-profdata merge test_0.profraw test_1.profraw -o test_merged.profdata
test_1.profraw: foo: Function basic block count change detected (counter mismatch)
Make sure that all profile data to be merged is generated from the same binary.
test_1.profraw: bar: Function basic block count change detected (counter mismatch)
test_1.profraw: baz: Function basic block count change detected (counter mismatch)
...
Reviewers: dnovillo, davidxl, bogner
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D14739
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@253384 91177308-0d34-0410-b5e6-96231b3b80d8
Indexed profile data as designed today does not guarantee
counter data to be well aligned, so reading needs to use
the slower form (with memcpy). This is less than ideal and
should be improved in the future (i.e., with fixed length
function key instead of variable length name key).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@253309 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This change addresses two possible instances of user error / confusion when
merging sampled profile data.
Previously any input that didn't match the raw or processed instrumented format
would automatically be interpreted as instrumented profile text format data.
No error would be reported during the merge.
Example:
If foo-sampled.profdata and bar-sampled.profdata are binary sampled profiles:
Old behavior:
$ llvm-profdata merge foo-sampled.profdata bar-sampled.profdata -output foobar-sampled.profdata
$ llvm-profdata show -sample foobar-sampled.profdata
error: foobar-sampled.profdata:1: Expected 'mangled_name:NUM:NUM', found lprofi
This change adds basic checks for valid input data when assuming text input.
It also makes error messages related to file format validity more specific about
the assumbed profile data type.
New behavior:
$ llvm-profdata merge foo-sampled.profdata bar-sampled.profdata -o foobar-sampled.profdata
error: foo.profdata: Unrecognized instrumentation profile encoding format
Perhaps you forgot to use the -sample option?
Reviewers: bogner, davidxl, dnovillo
Subscribers: davidxl, llvm-commits
Differential Revision: http://reviews.llvm.org/D14558
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@253009 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This change addresses two possible instances of user error / confusion when
merging sampled profile data.
Previously any input that didn't match the raw or processed instrumented format
would automatically be interpreted as instrumented profile text format data.
No error would be reported during the merge.
Example:
If foo-sampled.profdata and bar-sampled.profdata are binary sampled profiles:
Old behavior:
$ llvm-profdata merge foo-sampled.profdata bar-sampled.profdata -output foobar-sampled.profdata
$ llvm-profdata show -sample foobar-sampled.profdata
error: foobar-sampled.profdata:1: Expected 'mangled_name:NUM:NUM', found lprofi
This change adds basic checks for valid input data when assuming text input.
It also makes error messages related to file format validity more specific about
the assumbed profile data type.
New behavior:
$ llvm-profdata merge foo-sampled.profdata bar-sampled.profdata -o foobar-sampled.profdata
error: foo.profdata: Unrecognized instrumentation profile encoding format
Perhaps you forgot to use the -sample option?
Reviewers: bogner, davidxl, dnovillo
Subscribers: davidxl, llvm-commits
Differential Revision: http://reviews.llvm.org/D14558
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@252916 91177308-0d34-0410-b5e6-96231b3b80d8
- Make indexed value profile data more compact by peeling out
the per-site value count field into its own smaller sized array.
- Introduced formal data structure definitions to specify value
profile data layout in indexed format. Previously the layout
of the data is only assumed in the client code (scattered in
three different places : size computation, EmitData, and ReadData
- The new data structure serves as a central place for layout documentation.
- Add interfaces to force BE output for value profile data (testing purpose)
- Add byte swap unit tests
Differential Revision: http://reviews.llvm.org/D14401
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@252563 91177308-0d34-0410-b5e6-96231b3b80d8
With this change, instrumentation code and reader/write
code related to profile data structs are kept strictly
in-sync. THis will be extended to cfe and compile-rt
references as well.
Differential Revision: http://reviews.llvm.org/D13843
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@252113 91177308-0d34-0410-b5e6-96231b3b80d8
1. Added a set of public interfaces in InstrProfRecord
class to access (read/write) value profile data.
2. Changed IndexedProfile reader and writer code to
use the newly defined interfaces and hide implementation
details.
3. Added a couple of unittests for value profiling:
- Test new interfaces to get and set value profile data
- Test value profile data merging with various scenarios.
No functional change is expected. The new interfaces will also
make it possible to change on-disk format of value prof data
to be more compact (to be submitted).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@251771 91177308-0d34-0410-b5e6-96231b3b80d8
Add a couple of helper methods to make the primary
raw profile reader interface's implementation more
readable. It also hides more format details. This
patch has no functional change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@251546 91177308-0d34-0410-b5e6-96231b3b80d8
Change InstrProfReaderIndex from typedef into a wrapper
class with helper methods. This makes the index profile
reader code more readable. It also hides the implementation
detail of the index format and make it more flexible to allow
support different (or more than one) format in the future.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@251491 91177308-0d34-0410-b5e6-96231b3b80d8
This is a clean up patch that defines instr prof section and variable
name prefixes in a common header with access helper functions.
clang FE change will be done as a follow up once this patch is in.
Differential Revision: http://reviews.llvm.org/D13919
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@251058 91177308-0d34-0410-b5e6-96231b3b80d8
In some cases (as illustrated in the unittest), lineno can be less than the heade_lineno because the function body are included from some other files. In this case, offset will be negative. This patch makes clang still able to match the profile to IR in this situation.
http://reviews.llvm.org/D13914
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@250873 91177308-0d34-0410-b5e6-96231b3b80d8
1. Key constant values (version, magic) and data structures related to raw and
indexed profile format are moved into one centralized file: InstrProf.h.
2. Utility function such as MD5Hash computation is also moved to the common
header to allow sharing with other components in the future.
3. A header data structure is introduced for Indexed format so that the reader
and writer can always be in sync.
4. Added some comments to document different places where multiple definition
of the data structure must be kept in sync (reader/writer, runtime, lowering
etc). No functional change is intended.
Differential Revision: http://reviews.llvm.org/D13758
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@250638 91177308-0d34-0410-b5e6-96231b3b80d8
The number of samples collected at the head of a function only make
sense for top-level functions (i.e., those actually called as opposed to
being inlined inside another).
Head samples essentially count the time spent inside the function's
prologue. This clearly doesn't make sense for inlined functions, so we
were always emitting 0 in those.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@250539 91177308-0d34-0410-b5e6-96231b3b80d8
This adjusts all integers in the reader/writer to reflect the types
stored on profile files. They should all be unsigned 32-bit or 64-bit
values. Changed all associated internal types to be uint32_t or
uint64_t.
The only place that needed some adjustments is in the sample profile
transformation. Altough the weight read from the profile are 64-bit
values, the internal API for branch weights only accepts 32-bit values.
The pass now saturates weights that overflow uint32_t.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@250427 91177308-0d34-0410-b5e6-96231b3b80d8
This adds documentation for the binary profile encoding and moves the
documentation for the text encoding into the header file
SampleProfReader.h.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@250309 91177308-0d34-0410-b5e6-96231b3b80d8
Binary encoded profiles used to encode all function names inline at
every reference. This is clearly suboptimal in terms of space. This
patch fixes this by adding a name table to the header of the file.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@250241 91177308-0d34-0410-b5e6-96231b3b80d8
This is a test of the LLVM commit system. In the event of a real commit there would be some useful code changes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249972 91177308-0d34-0410-b5e6-96231b3b80d8
With this patch we can now read and write inline stacks in sample
profiles to the binary encoded profiles.
In a subsequent patch, I will add a string table to the binary encoding.
Right now function names are emitted as strings every time we find them.
This is too bloated and will produce large files in applications with
lots of inlining.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249861 91177308-0d34-0410-b5e6-96231b3b80d8
This fixes memory allocation problems by making the merge operation keep
the profile readers around until the merged profile has been emitted.
This is needed to prevent the inlined function names to disappear from
the function profiles. Since all the names are kept as references, once
the reader disappears, the names are also deallocated.
Additionally, XFAIL on big-endian architectures. The test case uses a
gcov file generated on a little-endian system.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249724 91177308-0d34-0410-b5e6-96231b3b80d8
This patch adds support for reading sample profiles with inline stacks.
Inline stacks in a profile are generated when the sampled binary has
samples in inlined functions.
For instance, if main() calls foo() and foo() calls bar(), and bar() is
inlined into foo() and foo() inlined into main(), the profile may look
something like:
main total:364084 head:0
[ ... ]
2.3: _Z3fool total:243786
1: 60149
1.2: 38568
1.4: 46511
1.7: _Z3bari total:98558
1.1: 52672
1.2: 45886
At line 2, discriminator 3, main() calls foo(). In turn, foo() calls
bar() at line 1, discriminator 7.
In the textual format, this stacking of inline calls is represented
with indentation.
With this change, LLVM can now read sample profile files generated by
the create_gcov tool from https://github.com/google/autofdo.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249644 91177308-0d34-0410-b5e6-96231b3b80d8
Given the work we are doing on ThinLTO, we will never need to support
module groups and working sets in GCC's implementation of LIPO. These
are currently dead code, and will continue to be so.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249351 91177308-0d34-0410-b5e6-96231b3b80d8
Add support to the indexed instrprof reader and writer for the format
that will be used for value profiling.
Patch by Betul Buyukkurt, with minor modifications.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248833 91177308-0d34-0410-b5e6-96231b3b80d8
This adds enough machinery to support reading simple GCC AutoFDO
profiles. It now supports reading flat profiles (no function calls).
Subsequent patches will add support for:
- Inlined calls (in particular, the inline call stack is not traversed
to accumulate samples).
- Working sets and modules. These are used mostly for GCC's LIPO
optimizations, so they're not needed in LLVM atm. I'm not sure that
we will ever need them. For now, I've if0'd around the calls.
The patch also adds support in GCOV.h for gcov version V704 (generated
by GCC's profile conversion tool).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@247874 91177308-0d34-0410-b5e6-96231b3b80d8
This version fixes a missing include that MSVC noticed and
clarifies the ownership of the counter buffer that's passed to
InstrProfRecord.
This restores r240206, which was reverted in r240208.
Patch by Betul Buyukkurt.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@240360 91177308-0d34-0410-b5e6-96231b3b80d8
The reason we need to search by name rather than by Triple::ArchType
is to handle subarchitecture correclty. There is no different ArchType
for the x86_64h architecture (it identifies itself as x86_64), or for
the various ARM subarches. The only way to get to the subarch slice
in an universal binary is to search by name.
This issue led to hard to debug and transient symbolication failures
in Asan tests (it mostly works, because the files are very similar).
This also affects the Profiling infrastucture as it is the other user
of that API.
Reviewers: samsonov, bogner
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D10604
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@240339 91177308-0d34-0410-b5e6-96231b3b80d8
Seems like MSVC doesn't like this:
InstrProf.h(49) : error C2614: 'llvm::InstrProfRecord' : illegal member initialization: 'Hash' is not a base or member
This reverts r240206.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@240208 91177308-0d34-0410-b5e6-96231b3b80d8
This consolidates the logic to read instrprof records into the on disk
hash table's lookup trait and makes us copy the counter data instead
of taking references to it as we read. This will simplify further
changes to the format.
Patch by Betul Buyukkurt.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@240206 91177308-0d34-0410-b5e6-96231b3b80d8
The patch is generated using this command:
tools/clang/tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
-checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \
llvm/lib/
Thanks to Eugene Kosov for the original patch!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@240137 91177308-0d34-0410-b5e6-96231b3b80d8
`LLVM_ENABLE_MODULES` builds sometimes fail because `Intrinsics.td`
needs to regenerate `Instrinsics.h` before anyone can include anything
from the LLVM_IR module. Represent the dependency explicitly to prevent
that.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@239796 91177308-0d34-0410-b5e6-96231b3b80d8
As noted on Errc.h:
// * std::errc is just marked with is_error_condition_enum. This means that
// common patters like AnErrorCode == errc::no_such_file_or_directory take
// 4 virtual calls instead of two comparisons.
And on some libstdc++ those virtual functions conclude that
------------------------
int main() {
std::error_code foo = std::make_error_code(std::errc::no_such_file_or_directory);
return foo == std::errc::no_such_file_or_directory;
}
-------------------------
should exit with 0.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@239683 91177308-0d34-0410-b5e6-96231b3b80d8
When we generate coverage data, we explicitly set each coverage map's
alignment to 8 (See InstrProfiling::lowerCoverageData), but when we
read the coverage data, we assume consecutive maps are exactly
adjacent. When we're dealing with 32 bit, maps can end on a 4 byte
boundary, causing us to think the padding is part of the next record.
Fix this by adjusting the buffer to an appropriately aligned address
between records.
This is pretty awkward to test, as it requires a binary with multiple
coverage maps to hit, so we'd need to check in multiple source files
and a binary blob as inputs.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@239129 91177308-0d34-0410-b5e6-96231b3b80d8
If we have a coverage mapping but no profile data for a function,
calling it mismatched is misleading. This can just as easily be
unreachable code that was stripped from the binary. Instead, treat
these the same as functions where we have an explicit "zero" coverage
map by setting the count to zero for each mapped region.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@237298 91177308-0d34-0410-b5e6-96231b3b80d8
Since the coverage mapping reader and the instrprof reader were
emitting a shared set of error codes, the error messages you'd get
back from llvm-cov were ambiguous about what was actually wrong. Add
another error category to fix this.
I've also improved the wording on a couple of the instrprof errors,
for consistency.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236665 91177308-0d34-0410-b5e6-96231b3b80d8
For consumers of coverage data, any filename prefixes we store in the
profile data are just noise. Strip this prefix if it exists.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236558 91177308-0d34-0410-b5e6-96231b3b80d8
This makes the reader check the endianness of the object file its
given and behave appropriately. For the test I dug up a really old
linker and created a ppc-apple-darwin file for llvm-cov to read.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@232422 91177308-0d34-0410-b5e6-96231b3b80d8
This code was casting regions of a memory buffer to a couple of
different structs. This is wrong in a few ways:
1. It breaks aliasing rules.
2. If the buffer isn't aligned, it hits undefined behaviour.
3. It completely ignores endianness differences.
4. The structs being defined for this aren't specifying their padding
properly, so this doesn't even represent the data properly on some
platforms.
This commit is mostly NFC, except that it fixes reading coverage for
32 bit binaries as a side effect of getting rid of the mispadded
structs. I've included a test for that.
I've also baked in that we only handle little endian more explicitly,
since that was true in practice already. I'll fix this to handle
endianness properly in a followup commit.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@232346 91177308-0d34-0410-b5e6-96231b3b80d8
Creating BinaryCoverageReader is a strange and complicated dance where
the constructor sets error codes that member functions will later
read, and the object is in an invalid state if readHeader isn't
immediately called after construction.
Instead, make the constructor private and add a static create method
to do the construction properly. This also has the benefit of removing
readHeader completely and simplifying the interface of the object.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@230676 91177308-0d34-0410-b5e6-96231b3b80d8
The current name is long and confusing. A shorter one is both easier
to understand and easier to work with.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@230675 91177308-0d34-0410-b5e6-96231b3b80d8
This fixes an error introduced in r228934 where None was converted to
an int instead of the int being converted to an Optional as intended.
We make that sort of mistake a compile error by changing NoneType into
a scoped enum.
Finally, provide a static NoneType called None to avoid forcing all
users to spell it NoneType::None.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229980 91177308-0d34-0410-b5e6-96231b3b80d8
This was leading to duplicate counts when a code region happened to
overlap exactly with an expansion. The combining behaviour only makes
sense for code regions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229723 91177308-0d34-0410-b5e6-96231b3b80d8
This comes up when we generate coverage for a function but don't end
up emitting the function at all - dead static functions or inline
functions that aren't referenced in a particular TU, for example. In
these cases we'd like to show that the function was never called,
which is trivially true.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229717 91177308-0d34-0410-b5e6-96231b3b80d8
Make CoverageMapping easier to create, so that we can write targeted
unit tests for its internals, and add a some infrastructure to write
these tests. Finally, add a simple unit test for basic functionality.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229709 91177308-0d34-0410-b5e6-96231b3b80d8
Have the InstrProfWriter return a MemoryBuffer instead of a
std::string. This fixes the alignment issues the reader would hit, and
it's a more appropriate type for this anyway.
I've also removed an ugly helper function that's not needed since
we're allowing initializer lists now, and updated some error code
checks based on MSVC's issues with r229473.
This reverts r229483, reapplying r229478.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229602 91177308-0d34-0410-b5e6-96231b3b80d8
This added API to the InstrProfWriter to write to a string so I could
write unittests without using temp files. This doesn't really work,
since the format has tighter alignment requirements than a char.
This reverts r229478 and its follow-up, r229481.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229483 91177308-0d34-0410-b5e6-96231b3b80d8
Add these tests again, but use va_list instead of initializer lists.
This reverts r229456, reapplying r229455.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229478 91177308-0d34-0410-b5e6-96231b3b80d8
This required some minor API to be added to these types to avoid
needing temp files.
Also, I've used initializer lists in the tests, as MSVC 2013 claims to
support them. I'll redo this without them if the bots complain.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229455 91177308-0d34-0410-b5e6-96231b3b80d8
The other InstrProfReader::create factories were updated to return
ErrorOr in r221120, and it's odd for these APIs not to match.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229433 91177308-0d34-0410-b5e6-96231b3b80d8
Keeping regions that start at the same location in insertion order
makes this logic easier to test / more deterministic.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228083 91177308-0d34-0410-b5e6-96231b3b80d8
Creating empty and expansion regions is awkward with the current API.
Expose static methods to make this simpler.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228075 91177308-0d34-0410-b5e6-96231b3b80d8
This is still kind of a weird API, but dropping the (partial) update
of the passed in CoverageMappingRecord makes it a little easier to
understand and use.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227900 91177308-0d34-0410-b5e6-96231b3b80d8
If two coverage segments cover the same area we need to combine them,
as per r218432. OTOH, just because they start at the same place
doesn't mean they cover the same area. This fixes the check to be more
exact about this.
This is pretty hard to test right now. The frontend doesn't currently
emit regions that start at the same place but don't overlap, but some
upcoming work changes this.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227017 91177308-0d34-0410-b5e6-96231b3b80d8
This patch was generated by a clang tidy checker that is being open sourced.
The documentation of that checker is the following:
/// The emptiness of a container should be checked using the empty method
/// instead of the size method. It is not guaranteed that size is a
/// constant-time function, and it is generally more efficient and also shows
/// clearer intent to use empty. Furthermore some containers may implement the
/// empty method but not implement the size method. Using empty whenever
/// possible makes it easier to switch to another container in the future.
Patch by Gábor Horváth!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226161 91177308-0d34-0410-b5e6-96231b3b80d8
utils/sort_includes.py.
I clearly haven't done this in a while, so more changed than usual. This
even uncovered a missing include from the InstrProf library that I've
added. No functionality changed here, just mechanical cleanup of the
include order.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225974 91177308-0d34-0410-b5e6-96231b3b80d8
This teaches CoverageMapping::getCoveredFunctions to filter to a
particular file and uses that to replace most of the logic found in
llvm-cov report.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@221962 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch extends the 'show' and 'merge' commands in llvm-profdata to handle
sample PGO formats. Using the 'merge' command it is now possible to convert
one sample PGO format to another.
The only format that is currently not working is 'gcc'. I still need to
implement support for it in lib/ProfileData.
The changes in the sample profile support classes are needed for the
merge operation.
Reviewers: bogner
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D6065
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@221032 91177308-0d34-0410-b5e6-96231b3b80d8
The getBinary and getBuffer method now return ordinary pointers of appropriate
const-ness. Ownership is transferred by calling takeBinary(), which returns a
pair of the Binary and a MemoryBuffer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@221003 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch finishes up support for handling sampling profiles in both
text and binary formats. The new binary format uses uleb128 encoding to
represent numeric values. This makes profiles files about 25% smaller.
The profile writer class can write profiles in the existing text and the
new binary format. In subsequent patches, I will add the capability to
read (and perhaps write) profiles in the gcov format used by GCC.
Additionally, I will be adding support in llvm-profdata to manipulate
sampling profiles.
There was a bit of refactoring needed to separate some code that was in
the reader files, but is actually common to both the reader and writer.
The new test checks that reading the same profile encoded as text or
raw, produces the same results.
Reviewers: bogner, dexonsmith
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D6000
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@220915 91177308-0d34-0410-b5e6-96231b3b80d8
There are two methods in SectionRef that can fail:
* getName: The index into the string table can be invalid.
* getContents: The section might point to invalid contents.
Every other method will always succeed and returning and std::error_code just
complicates the code. For example, a section can have an invalid alignment,
but if we are able to get to the section structure at all and create a
SectionRef, we will always be able to read that invalid alignment.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@219314 91177308-0d34-0410-b5e6-96231b3b80d8
Every time we were adding or removing an expression when generating a
coverage mapping we were doing a linear search to try and deduplicate
the list. The indices in the list are important, so we can't just
replace it by a DenseMap entirely, but an auxilliary DenseMap for fast
lookup massively improves the performance issues I was seeing here.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218892 91177308-0d34-0410-b5e6-96231b3b80d8
When I was preparing r218879 for commit, I removed an early return
that I decided was just noise. It wasn't. This is r218879 no-crash
edition.
This reverts commit r218881, reapplying r218879.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218887 91177308-0d34-0410-b5e6-96231b3b80d8
The Terms vector here represented a polynomial of of all possible
counters, and is used to simplify expressions when generating coverage
mapping. There are a few problems with this:
1. Keeping the vector as a member is wasteful, since we clear it every
time we use it.
2. Most expressions refer to a subset of the counters, so we end up
iterating over a large number of zeros doing nothing a lot of the
time.
This updates the user of the vector to store the terms locally, and
uses a sort and combine approach so that we only operate on counters
that are actually used in a given expression. For small cases this
makes very little difference, but in cases with a very large number of
counted regions this is a significant performance fix.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218879 91177308-0d34-0410-b5e6-96231b3b80d8
When writing a coverage mapping we iterate through the mapping regions
in order of FileID, but we were then repeatedly searching from the
beginning of the list to count the number of regions with a given
FileID.
It is simpler and more efficient to search forward from the current
iterator to find the number of regions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218842 91177308-0d34-0410-b5e6-96231b3b80d8
If we have multiple coverage counts for the same segment, we need to
add them up rather than arbitrarily choosing one. This fixes that and
adds a test with template instantiations to exercise it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218432 91177308-0d34-0410-b5e6-96231b3b80d8
This splits the logic for actually looking up coverage information
from the logic that displays it. These were tangled rather thoroughly
so this change is a bit large, but it mostly consists of moving things
around. The coverage lookup logic itself now lives in the library,
rather than being spread between the library and the tool.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218184 91177308-0d34-0410-b5e6-96231b3b80d8
This debug output is really for testing CoverageMappingReader, not the
llvm-cov tool. Move it to where it can be more useful.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218183 91177308-0d34-0410-b5e6-96231b3b80d8
It isn't always useful to skip blank lines, as evidenced by the
somewhat awkward use of line_iterator in llvm-cov. This adds a knob to
control whether or not to skip blanks.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217960 91177308-0d34-0410-b5e6-96231b3b80d8
The raw profiles that are generated in compiler-rt always add padding
so that each profile is aligned, so we can simply treat files that
don't have this property as malformed.
Caught by Alexey's new ubsan bot. Thanks!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217708 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch moves the profile reading logic out of the Sample Profile
transformation into a generic profile reader facility in
lib/ProfileData.
The intent is to use this new reader to implement a sample profile
reader/writer that can be used to convert sample profiles from external
sources into LLVM.
This first patch introduces no functional changes. It moves the profile
reading code from lib/Transforms/SampleProfile.cpp into
lib/ProfileData/SampleProfReader.cpp.
In subsequent patches I will:
- Add a bitcode format for sample profiles to allow for more efficient
encoding of the profile.
- Add a writer for both text and bitcode format profiles.
- Add a 'convert' command to llvm-profdata to be able to convert between
the two (and serve as entry point for other sample profile formats).
Reviewers: bogner, echristo
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D5250
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217437 91177308-0d34-0410-b5e6-96231b3b80d8
This commit expands llvm-cov's functionality by adding support for a new code coverage
tool that uses LLVM's coverage mapping format and clang's instrumentation based profiling.
The gcov compatible tool can be invoked by supplying the 'gcov' command as the first argument,
or by modifying the tool's name to end with 'gcov'.
Differential Revision: http://reviews.llvm.org/D4445
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216300 91177308-0d34-0410-b5e6-96231b3b80d8
The profile data format was recently updated and the new indexing api
requires the code coverage tool to know the function's hash as well
as the function's name to get the execution counts for a function.
Differential Revision: http://reviews.llvm.org/D4994
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216207 91177308-0d34-0410-b5e6-96231b3b80d8
Owning the buffer is somewhat inflexible. Some Binaries have sub Binaries
(like Archive) and we had to create dummy buffers just to handle that. It is
also a bad fit for IRObjectFile where the Module wants to own the buffer too.
Keeping this ownership would make supporting IR inside native objects
particularly painful.
This patch focuses in lib/Object. If something elsewhere used to own an Binary,
now it also owns a MemoryBuffer.
This patch introduces a few new types.
* MemoryBufferRef. This is just a pair of StringRefs for the data and name.
This is to MemoryBuffer as StringRef is to std::string.
* OwningBinary. A combination of Binary and a MemoryBuffer. This is needed
for convenience functions that take a filename and return both the
buffer and the Binary using that buffer.
The C api now uses OwningBinary to avoid any change in semantics. I will start
a new thread to see if we want to change it and how.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216002 91177308-0d34-0410-b5e6-96231b3b80d8
Add header guards to files that were missing guards. Remove #endif comments
as they don't seem common in LLVM (we can easily add them back if we decide
they're useful)
Changes made by clang-tidy with minor tweaks.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215558 91177308-0d34-0410-b5e6-96231b3b80d8
This updates the instrumentation based profiling format so that when
we have multiple functions with the same name (but different function
hashes) we keep all of them instead of rejecting the later ones.
There are a number of scenarios where this can come up where it's more
useful to keep multiple function profiles:
* Name collisions in unrelated libraries that are profiled together.
* Multiple "main" functions from multiple tools built against a common
library.
* Combining profiles from different build configurations (ie, asserts
and no-asserts)
The profile format now stores the number of counters between the hash
and the counts themselves, so that multiple sets of counts can be
stored. Since this is backwards incompatible, I've bumped the format
version and added some trivial logic to skip this when reading the old
format.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214585 91177308-0d34-0410-b5e6-96231b3b80d8
This moves some tests around to make it clearer what's being tested,
and adds very rudimentary comment syntax to the text input format to
make specifying this kind of test a little bit simpler.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214235 91177308-0d34-0410-b5e6-96231b3b80d8
This patch reduces the complexity of the two inner loops in order to speed up
the loading of coverage data for very large functions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214228 91177308-0d34-0410-b5e6-96231b3b80d8
This patch removes the empty coverage mapping regions.
Those regions were produced by clang's old mapping region generation
algorithm, but the new algorithm doesn't generate them.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213981 91177308-0d34-0410-b5e6-96231b3b80d8
This patch implements the data structures, the reader and
the writers for the new code coverage mapping system.
The new code coverage mapping system uses the instrumentation
based profiling to provide code coverage analysis.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213910 91177308-0d34-0410-b5e6-96231b3b80d8
This patch implements the data structures, the reader and
the writers for the new code coverage mapping system.
The new code coverage mapping system uses the instrumentation
based profiling to provide code coverage analysis.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213909 91177308-0d34-0410-b5e6-96231b3b80d8
This code was never being used and any use of it would look fairly strange.
For example, it would try to map a object_error::parse_failed to
std::errc::invalid_argument.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210912 91177308-0d34-0410-b5e6-96231b3b80d8
The idea of this patch is to turn llvm/Support/system_error.h into a
transitional header that just brings in the erorr_code api to the llvm
namespace. I will remove it shortly afterwards.
The cases where the general idea needed some tweaking:
* std::errc is a namespace in msvc, so we cannot use "using std::errc". I could
add an #ifdef, but there were not that many uses, so I just added std:: to
them in this patch.
* Template specialization had to be moved to the std namespace in this
patch set already.
* The msvc implementation of default_error_condition doesn't seem to
provide the same transformations as we need. Not too surprising since
the standard doesn't actually say what "equivalent" means. I fixed the
problem by keeping our old mapping and using it at error_code
construction time.
Despite these shortcomings I think this is still a good thing. Some reasons:
* The different implementations of system_error might improve over time.
* It removes 925 lines of code from llvm already.
* It removes 6313 bytes from the text segment of the clang binary when
it is built with gcc and 2816 bytes when building with clang and
libstdc++.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210687 91177308-0d34-0410-b5e6-96231b3b80d8
Allow multiple raw profiles to coexist in a single .profraw file,
given the following conditions:
- Zero padding at the end of or between profiles will be skipped.
- Each profile must start with a valid header.
- Mixing endianness or pointer sizes in concatenated profiles files is
not allowed.
This is needed to handle cases where a program's shared libraries are
profiled as well as the main executable itself, as we'll need to emit
each executable's counters. Combining the tables in the runtime would
be expensive for the instrumented program.
rdar://16918688
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@208938 91177308-0d34-0410-b5e6-96231b3b80d8
We're currently copying CounterData from InstrProfWriter into the
OnDiskHashTable, even though we don't need to, and then carelessly
leaking those copies. A const pointer is much better here.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207009 91177308-0d34-0410-b5e6-96231b3b80d8
This adds support for an indexed instrumentation based profiling
format, which is just a small header and an on disk hash table. This
format will be used by clang's -fprofile-instr-use= for PGO.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206656 91177308-0d34-0410-b5e6-96231b3b80d8
Since the profile can come from 32-bit machines, we need to check the
pointer size. Change the magic number to facilitate this.
Adds tests for reading 32-bit and 64-bit binaries (both big- and
little-endian). The tests write a binary using printf in RUN lines
(like raw-magic-but-no-header.test). Assuming the bots don't complain,
this seems like a better way forward for testing RawInstrProfReader than
committing binary files.
<rdar://problem/16400648>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204557 91177308-0d34-0410-b5e6-96231b3b80d8
Include non-text characters in the magic number so that text files can't
match.
<rdar://problem/15950346>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204513 91177308-0d34-0410-b5e6-96231b3b80d8
Read a raw binary profile that corresponds to a memory dump from the
runtime profile.
The test is a binary file generated from
cfe/trunk/test/Profile/c-general.c with the new compiler-rt runtime and
the matching text version of the input. It includes instructions on how
to regenerate.
<rdar://problem/15950346>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204496 91177308-0d34-0410-b5e6-96231b3b80d8
This isn't a format we'll want to write out in practice, but moving it
to the writer library simplifies llvm-profdata and isolates it from
further changes to the format.
This also allows us to update the tests to not rely on the text output
format.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204489 91177308-0d34-0410-b5e6-96231b3b80d8
This introduces the ProfileData library and updates llvm-profdata to
use this library for reading profiles. InstrProfReader is an abstract
base class that will be subclassed for both the raw instrprof data
from compiler-rt and the efficient instrprof format that will be used
for PGO.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204482 91177308-0d34-0410-b5e6-96231b3b80d8
