Add a "-j" option to llvm-profdata to control the number of threads used.
Auto-detect NumThreads when it isn't specified, and avoid spawning threads when
they wouldn't be beneficial.
I tested this patch using a raw profile produced by clang (147MB). Here is the
time taken to merge 4 copies together on my laptop:
No thread pool: 112.87s user 5.92s system 97% cpu 2:01.08 total
With 2 threads: 134.99s user 26.54s system 164% cpu 1:33.31 total
Changes since the initial commit:
- When handling odd-length inputs, call ThreadPool::wait() before merging the
last profile. Should fix a race/off-by-one (see r275937).
Differential Revision: https://reviews.llvm.org/D22438
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275938 91177308-0d34-0410-b5e6-96231b3b80d8
Add a "-j" option to llvm-profdata to control the number of threads
used. Auto-detect NumThreads when it isn't specified, and avoid spawning
threads when they wouldn't be beneficial.
I tested this patch using a raw profile produced by clang (147MB). Here is the
time taken to merge 4 copies together on my laptop:
No thread pool: 112.87s user 5.92s system 97% cpu 2:01.08 total
With 2 threads: 134.99s user 26.54s system 164% cpu 1:33.31 total
Differential Revision: https://reviews.llvm.org/D22438
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275921 91177308-0d34-0410-b5e6-96231b3b80d8
Changes since the initial commit:
- Normalize file paths read from the file to prevent Windows path
separators from escaping parts of the path.
- Since we need to store the normalized file paths in WeightedFile,
don't do tricky things to keep the source MemoryBuffer alive.
- Don't use list-initialization for a std::string in WeightedFile.
Differential Revision: http://reviews.llvm.org/D20980
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271953 91177308-0d34-0410-b5e6-96231b3b80d8
Changes since the initial commit:
- Normalize file paths read from the file to prevent Windows path
separators from escaping parts of the path.
- Since we need to store the normalized file paths in WeightedFile,
don't do tricky things to keep the source MemoryBuffer alive.
Differential Revision: http://reviews.llvm.org/D20980
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271949 91177308-0d34-0410-b5e6-96231b3b80d8
and/or tests aren't working on Windows currently.
There seems to be some problem with quoting the file paths. I don't
understand the test structure here or the code well enough to try to
come up with a way to correctly handle paths with back slashes in them,
and this has caused the Windows builds to be failing for 7 hours now, so
I'm reverting the whole thing to bring them back to life. Sorry for the
disruption, but a couple of these were bug fixes anyways that can be
folded into a fresh commit.
Reverts the following patches:
r271756: Clean up the way we create the input filenames buffer (NFC)
r271748: Fix use-after-free from discarded MemoryBuffer (NFC)
r271710: Fix option description (NFC)
r271709: Add option to ingest filepaths from a file
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271760 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This change adds support for specifying a weight when merging profile data with the llvm-profdata tool.
Weights are specified by using the --weighted-input=<weight>,<filename> option. Input files not specified
with this option (normal positional list after options) are given a default weight of 1.
Adding support for arbitrary weighting of input profile data allows for relative importance to be placed on the
input data from multiple training runs.
Both sampled and instrumented profiles are supported.
Reviewers: davidxl, dnovillo, bogner, silvas
Subscribers: silvas, davidxl, llvm-commits
Differential Revision: http://reviews.llvm.org/D15306
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@255659 91177308-0d34-0410-b5e6-96231b3b80d8
This reverts commit b7250858d96b8ce567681214273ac0e62713c661.
Reverting in order to investigate Windows test failure.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254687 91177308-0d34-0410-b5e6-96231b3b80d8
This change adds support for an optional weight when merging profile data with the llvm-profdata tool.
Weights are specified by adding an option ':<weight>' suffix to the input file names.
Adding support for arbitrary weighting of input profile data allows for relative importance to be placed on the
input data from multiple training runs.
Both sampled and instrumented profiles are supported.
Reviewers: dnovillo, bogner, davidxl
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D14547
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254669 91177308-0d34-0410-b5e6-96231b3b80d8
Consolidate the description of -binary/-text option description
to avoid duplicate ID error by sphinux-build.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254018 91177308-0d34-0410-b5e6-96231b3b80d8
The new option is similar to the SampleProfile dump option.
- dump raw/indexed format into text profile format
- merge the profile and output into text profile format.
Note that Value Profiling data text format is not yet designed.
That functionality will be added later.
Differential Revision: http://reviews.llvm.org/D14894
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@253913 91177308-0d34-0410-b5e6-96231b3b80d8
These links seem broken on llvm.org/docs. Change them to use the
sphinx-recommended style to see if that helps.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@232001 91177308-0d34-0410-b5e6-96231b3b80d8
Introducing llvm-profdata, a tool for merging profile data generated by
PGO instrumentation in clang.
- The name indicates a file extension of <name>.profdata. Eventually
profile data output by clang should be changed to that extension.
- llvm-profdata merges two profiles. However, the name is more general,
since it will likely pick up more tasks (such as summarizing a single
profile).
- llvm-profdata parses the current text-based format, but will be
updated once we settle on a binary format.
<rdar://problem/15949645>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201535 91177308-0d34-0410-b5e6-96231b3b80d8
