Some out-of-tree targets depend on the enum name SPF_Binary. Keep the name
can avoid unnecessary churn to those targets.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334476 91177308-0d34-0410-b5e6-96231b3b80d8
Name table occupies a big chunk of size in current binary format sample profile.
In order to reduce its size, the patch changes the sample writer/reader to
save/restore MD5Hash of names in the name table. Sample annotation phase will
also use MD5Hash of name to query samples accordingly.
Experiment shows compact binary format can reduce the size of sample profile by
2/3 compared with binary format generally.
Differential Revision: https://reviews.llvm.org/D47955
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334447 91177308-0d34-0410-b5e6-96231b3b80d8
We have a few functions that virtually all command wants to run on
process startup/shutdown. This patch adds InitLLVM class to do that
all at once, so that we don't need to copy-n-paste boilerplate code
to each llvm command's main() function.
Differential Revision: https://reviews.llvm.org/D45602
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330046 91177308-0d34-0410-b5e6-96231b3b80d8
But now include a check for CPU_COUNT so we still build on 10 year old
versions of glibc.
Original message:
Use sched_getaffinity instead of std:🧵:hardware_concurrency.
The issue with std:🧵:hardware_concurrency is that it forwards
to libc and some implementations (like glibc) don't take thread
affinity into consideration.
With this change a llvm program that can execute in only 2 cores will
use 2 threads, even if the machine has 32 cores.
This makes benchmarking a lot easier, but should also help if someone
doesn't want to use all cores for compilation for example.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@314931 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This reverts D38481. The change breaks systems with older versions of glibc. It
injects a use of CPU_COUNT() from sched.h without checking to ensure that the
function exists first.
Reviewers:
Subscribers:
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@314922 91177308-0d34-0410-b5e6-96231b3b80d8
The issue with std:🧵:hardware_concurrency is that it forwards
to libc and some implementations (like glibc) don't take thread
affinity into consideration.
With this change a llvm program that can execute in only 2 cores will
use 2 threads, even if the machine has 32 cores.
This makes benchmarking a lot easier, but should also help if someone
doesn't want to use all cores for compilation for example.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@314809 91177308-0d34-0410-b5e6-96231b3b80d8
Reduces llvm-profdata memory usage on a large profile from 7.8GB to 5.1GB.
The ProfData API now supports reporting all the errors/warnings rather
than only the first, though llvm-profdata ignores everything after the
first for now to preserve existing behavior. (if there's a desire for
other behavior, happy to implement that - but might be as well left for
a separate patch)
Reviewers: davidxl
Differential Revision: https://reviews.llvm.org/D35149
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@307516 91177308-0d34-0410-b5e6-96231b3b80d8
The InstrProfWriter already stores the name and hash of the record in
the nested maps it uses for lookup while merging - this data is
duplicated in the value within the maps.
Refactor the InstrProfRecord to use a nested struct for the counters
themselves so that InstrProfWriter can use this nested struct alone
without the name or hash duplicated there.
This work is incomplete, but enough to demonstrate the value (around a
50% decrease in memory usage for a large test case (10GB -> 5GB)).
Though most of that decrease is probably from removing the
SoftInstrProfError as well, but I haven't implemented a replacement for
it yet. (it needs to go with the counters, because the operations on the
counters - merging, etc, are where the failures are - unlike the
name/hash which are totally unused by those counter-related operations
and thus easy to split out)
Ongoing discussion about removing SoftInstrProfError as a field of the
InstrProfRecord is happening on the thread that added it - including
the possibility of moving back towards an earlier version of that
proposed patch that passed SoftInstrProfError through the various APIs,
rather than as a member of InstrProfRecord.
Reviewers: davidxl
Differential Revision: https://reviews.llvm.org/D34838
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@307298 91177308-0d34-0410-b5e6-96231b3b80d8
With PR33517, it became apparent that symbol table creation can fail
when presented with malformed inputs. This patch makes that sort of
error detectable, so llvm-cov etc. can fail more gracefully.
Specifically, we now check that function names within the symbol table
aren't empty.
Testing: check-{llvm,clang,profile}, some unit test updates.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@305765 91177308-0d34-0410-b5e6-96231b3b80d8
llvm-profdata.cpp has the following include chain:
llvm/ProfileData/SampleProfReader.h
llvm/IR/Function.h
llvm/IR/Argument.h
llvm/IR/Attributes.h
llvm/IR/Attributes.gen
This means llvm-profdata needs to depend on intrinsics_gen.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@287394 91177308-0d34-0410-b5e6-96231b3b80d8
Profile runtime can generate an empty raw profile (when there is no function in
the shared library). This empty profile is treated as a text format profile. A
test format profile without the flag of "#IR" is thought to be a clang
generated profile. So in llvm profile merging, we will get a bogus warning of
"Merge IR generated profile with Clang generated profile."
The fix here is to skip the empty profile (when the buffer size is 0) for
profile merge.
Reviewers: vsk, davidxl
Differential Revision: http://reviews.llvm.org/D25687
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284659 91177308-0d34-0410-b5e6-96231b3b80d8
The core of the change is supposed to be NFC, however it also fixes
what I believe was an undefined behavior when calling:
va_start(ValueArgs, Desc);
with Desc being a StringRef.
Differential Revision: https://reviews.llvm.org/D25342
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283671 91177308-0d34-0410-b5e6-96231b3b80d8
Add a "-j" option to llvm-profdata to control the number of threads used.
Auto-detect NumThreads when it isn't specified, and avoid spawning threads when
they wouldn't be beneficial.
I tested this patch using a raw profile produced by clang (147MB). Here is the
time taken to merge 4 copies together on my laptop:
No thread pool: 112.87s user 5.92s system 97% cpu 2:01.08 total
With 2 threads: 134.99s user 26.54s system 164% cpu 1:33.31 total
Changes since the initial commit:
- When handling odd-length inputs, call ThreadPool::wait() before merging the
last profile. Should fix a race/off-by-one (see r275937).
Differential Revision: https://reviews.llvm.org/D22438
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275938 91177308-0d34-0410-b5e6-96231b3b80d8
Add a "-j" option to llvm-profdata to control the number of threads
used. Auto-detect NumThreads when it isn't specified, and avoid spawning
threads when they wouldn't be beneficial.
I tested this patch using a raw profile produced by clang (147MB). Here is the
time taken to merge 4 copies together on my laptop:
No thread pool: 112.87s user 5.92s system 97% cpu 2:01.08 total
With 2 threads: 134.99s user 26.54s system 164% cpu 1:33.31 total
Differential Revision: https://reviews.llvm.org/D22438
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275921 91177308-0d34-0410-b5e6-96231b3b80d8
looking for it along $PATH. This allows installs of LLVM tools outside of
$PATH to find the symbolizer and produce pretty backtraces if they crash.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272232 91177308-0d34-0410-b5e6-96231b3b80d8
Changes since the initial commit:
- Normalize file paths read from the file to prevent Windows path
separators from escaping parts of the path.
- Since we need to store the normalized file paths in WeightedFile,
don't do tricky things to keep the source MemoryBuffer alive.
- Don't use list-initialization for a std::string in WeightedFile.
Differential Revision: http://reviews.llvm.org/D20980
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271953 91177308-0d34-0410-b5e6-96231b3b80d8
Changes since the initial commit:
- Normalize file paths read from the file to prevent Windows path
separators from escaping parts of the path.
- Since we need to store the normalized file paths in WeightedFile,
don't do tricky things to keep the source MemoryBuffer alive.
Differential Revision: http://reviews.llvm.org/D20980
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271949 91177308-0d34-0410-b5e6-96231b3b80d8
and/or tests aren't working on Windows currently.
There seems to be some problem with quoting the file paths. I don't
understand the test structure here or the code well enough to try to
come up with a way to correctly handle paths with back slashes in them,
and this has caused the Windows builds to be failing for 7 hours now, so
I'm reverting the whole thing to bring them back to life. Sorry for the
disruption, but a couple of these were bug fixes anyways that can be
folded into a fresh commit.
Reverts the following patches:
r271756: Clean up the way we create the input filenames buffer (NFC)
r271748: Fix use-after-free from discarded MemoryBuffer (NFC)
r271710: Fix option description (NFC)
r271709: Add option to ingest filepaths from a file
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271760 91177308-0d34-0410-b5e6-96231b3b80d8
Create the buffer before calling parseInputFilenamesFile(), and add a
comment explaining why this is done.
Thanks to David Li for the suggestion!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271756 91177308-0d34-0410-b5e6-96231b3b80d8
This splits ProfileSummary into two classes: a ProfileSummary class that has methods to convert from/to metadata and a ProfileSummaryBuilder class that computes the profiles summary which is in ProfileData.
Differential Revision: http://reviews.llvm.org/D20314
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270136 91177308-0d34-0410-b5e6-96231b3b80d8
