Using GNU diff, `--strip-trailing-cr` removes a `\r` appearing before
a `\n` at the end of a line. Without this patch, lit's internal diff
only removes `\r` if it appears as the last character. That seems
useless. This patch fixes that.
This patch also adds `--strip-trailing-cr` to some tests that fail on
Windows bots when D68664 is applied. Based on what I see in the bot
logs, I think the following is happening. In each test there, lit
diff is comparing a file with `\r\n` line endings to a file with `\n`
line endings. Without D68664, lit diff reads those files in text
mode, which in Windows causes `\r\n` to be replaced with `\n`.
However, with D68664, lit diff reads the files in binary mode instead
and thus reports that every line is different, just as GNU diff does
(at least under Ubuntu). Adding `--strip-trailing-cr` to those tests
restores the previous behavior while permitting the behavior of lit
diff to be more like GNU diff.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D68839
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@375020 91177308-0d34-0410-b5e6-96231b3b80d8
I removed this test to unblock the ARM bots while looking into failures
(r374915), and am reinstating it now with a fix.
I believe the problem was that counter ptr address I used,
'\0\0\6\0\1\0\0\1', set the high bits of the pointer, not the low bits
like I wanted. On x86_64 this superficially looks like it tests r370826,
but it doesn't, as it would have been caught before r370826. However, on
ARM (or, 32-bit hosts more generally), I suspect the high bits were
cleared, and you get a 'valid' profile.
I verified that setting the *low* bits of the pointer does trigger the
new condition:
-// Note: The CounterPtr here is off-by-one. This should trigger a malformed profile error.
-RUN: printf '\0\0\6\0\1\0\0\1' >> %t.profraw
+// Note: The CounterPtr here is off-by-one.
+//
+// Octal '\11' is 9 in decimal: this should push CounterOffset to 1. As there are two counters,
+// the profile reader should error out.
+RUN: printf '\11\0\6\0\1\0\0\0' >> %t.profraw
This reverts commit c7cf5b3e4b918c9769fd760f28485b8d943ed968.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374927 91177308-0d34-0410-b5e6-96231b3b80d8
There are a number arm bots failing after r374617 landed, and I'm not
sure why. It looks a bit like the error message llvm-profdata is
expected to print to stderr isn't flushed.
Weaken the test in an attempt to appease the arm bots: if this doesn't
work, that means that llvm-profdata is actually *not failing*, and that
will be a clear indication that some logic error is actually happening.
http://lab.llvm.org:8011/builders/clang-cmake-armv7-global-isel/builds/5604/
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374792 91177308-0d34-0410-b5e6-96231b3b80d8
Using GNU diff, `--strip-trailing-cr` removes a `\r` appearing before
a `\n` at the end of a line. Without this patch, lit's internal diff
only removes `\r` if it appears as the last character. That seems
useless. This patch fixes that.
This patch also adds `--strip-trailing-cr` to some tests that fail on
Windows bots when D68664 is applied. Based on what I see in the bot
logs, I think the following is happening. In each test there, lit
diff is comparing a file with `\r\n` line endings to a file with `\n`
line endings. Without D68664, lit diff reads those files with
Python's universal newlines support activated, causing `\r` to be
dropped. However, with D68664, lit diff reads the files in binary
mode instead and thus reports that every line is different, just as
GNU diff does (at least under Ubuntu). Adding `--strip-trailing-cr`
to those tests restores the previous behavior while permitting the
behavior of lit diff to be more like GNU diff.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D68839
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374652 91177308-0d34-0410-b5e6-96231b3b80d8
Previously ExtBinary profile format only supports compression using zlib for
profile symbol list. In this patch, we extend the compression support to any
section. User can select some or all of the sections to compress. In an
experiment, for a 45M profile in ExtBinary format, compressing name table
reduced its size to 24M, and compressing all the sections reduced its size
to 11M.
Differential Revision: https://reviews.llvm.org/D68253
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@373914 91177308-0d34-0410-b5e6-96231b3b80d8
or the size of the profile for profile in ExtBinary format.
Fix a test failure on Mac.
[SampleFDO] Expose an interface to return the size of a section or the
size of the profile for profile in ExtBinary format.
Sometimes we want to limit the size of the profile by stripping some functions
with low sample count or by stripping some function names with small text size
from profile symbol list. That requires the profile reader to have the
interfaces returning the size of a section or the size of total profile. The
patch add those interfaces.
At the same time, add some dump facility to show the size of each section.
Differential revision: https://reviews.llvm.org/D67726
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@372478 91177308-0d34-0410-b5e6-96231b3b80d8
of the profile for profile in ExtBinary format.
Sometimes we want to limit the size of the profile by stripping some functions
with low sample count or by stripping some function names with small text size
from profile symbol list. That requires the profile reader to have the
interfaces returning the size of a section or the size of total profile. The
patch add those interfaces.
At the same time, add some dump facility to show the size of each section.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@372439 91177308-0d34-0410-b5e6-96231b3b80d8
Add a mode in which profile read errors are not immediately treated as
fatal. In this mode, merging makes forward progress and reports failure
only if no inputs can be read.
Differential Revision: https://reviews.llvm.org/D66985
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@370827 91177308-0d34-0410-b5e6-96231b3b80d8
The check needs to validate a counter offset before performing pointer
arithmetic with the (potentially corrupt) offset.
Found by UBSan's pointer overflow check.
rdar://54843625
Differential Revision: https://reviews.llvm.org/D66979
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@370826 91177308-0d34-0410-b5e6-96231b3b80d8
cold versus function being newly added.
This is the second half of https://reviews.llvm.org/D66374.
Profile symbol list is the collection of function symbols showing up in
the binary which generates the current profile. It is used to discriminate
function being cold versus function being newly added. Profile symbol list
is only added for profile with ExtBinary format.
During profile use compilation, when profile-sample-accurate is enabled,
a function without profile will be regarded as cold only when it is
contained in that list.
Differential Revision: https://reviews.llvm.org/D66766
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@370563 91177308-0d34-0410-b5e6-96231b3b80d8
This is a patch split from https://reviews.llvm.org/D66374. It tries to add
a new format of profile called ExtBinary. The format adds a section header
table to the profile and organize the profile in sections, so the future
extension like adding a new section or extending an existing section will be
easier while keeping backward compatiblity feasible.
Differential Revision: https://reviews.llvm.org/D66513
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@369798 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
StringMap is used for storing call target to frequency map for AutoFDO. However the iterating order of StringMap is non-deterministic, which leads to non-determinism in AutoFDO profile output. Now new API getSortedCallTargets and SortCallTargets are added for deterministic ordering and output.
Roundtrip test for text profile and binary profile is added.
Reviewers: wmi, davidxl, danielcdh
Subscribers: hiraditya, mgrang, llvm-commits, twoh
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D66191
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@369440 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: Fix "llvm-profdata show" so it can work with compact binary format profile. The change is to mark all functions "used" so SampleProfileReaderCompactBinary::read will read in all profiles available for dumping. The function names will be MD5 hash for compact binary format.
Reviewers: wmi, davidxl, danielcdh
Subscribers: hiraditya, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D65162
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@368731 91177308-0d34-0410-b5e6-96231b3b80d8
Some file systems may not allow this behavior, the test fails on our internal
system ("Permission denied").
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@365450 91177308-0d34-0410-b5e6-96231b3b80d8
Currently llvm-profdata does not expect the same file name for the input profile
and the output profile.
>llvm-profdata merge A.profraw B.profraw -o B.profraw
The above command runs successfully but the resulted B.profraw is not correct.
This patch fixes the issue by moving the initialization of writer after loading
the profile.
For the show command, the following will report a confusing error of
"Empty raw profile file":
>llvm-profdata show B.profraw -o B.profraw
It's harder to fix as we need to output something before loading the input profile.
I don't think that a fix for this is worth the effort. I just make the error explicit for
the show command.
Differential Revision: https://reviews.llvm.org/D64360
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@365386 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Triple components in `XFAIL` lines are tested against the target triple.
Various tests that are expected to fail on big-endian hosts are marked
as being `XFAIL` for big-endian targets. This patch corrects these tests
by having them test against a new `host-byteorder-big-endian` feature.
Reviewers: xingxue, sfertile, jasonliu
Reviewed By: xingxue
Subscribers: jvesely, nhaehnle, fedor.sergeev, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D60551
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@359689 91177308-0d34-0410-b5e6-96231b3b80d8
This patch improves llvm-profdata show command:
(1) add -value-cutoff=<N> option: Show only those functions whose max count
values are greater or equal to N.
(2) add -list-below-cutoff option: Only output names of functions whose max
count value are below the cutoff.
(3) formats value-profile counts and prints out percentage.
Differential Revision: https://reviews.llvm.org/D56342
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@350673 91177308-0d34-0410-b5e6-96231b3b80d8
In LTO or Thin-lto mode (though linker plugin), the module
names are of temp file names which are different for
different compilations. Using SourceFileName avoids the issue.
This should not change any functionality for current PGO as
all the current callers of getPGOFuncName() is before LTO.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@350579 91177308-0d34-0410-b5e6-96231b3b80d8
add a tool to generate symbol remapping files.
Summary:
The new tool llvm-cxxmap builds a symbol mapping table from a file containing
a description of partial equivalences to apply to mangled names and files
containing old and new symbol tables.
Reviewers: davidxl
Subscribers: mgorny, llvm-commits
Differential Revision: https://reviews.llvm.org/D51470
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@342168 91177308-0d34-0410-b5e6-96231b3b80d8
Per LLVM's CommandGuide, llvm-profdata show -text is supposed to produce
textual output that can be passed as input to further llvm-profdata
invocations. This previously didn't work for two reasons:
1) -text was not sufficient to enable the machine-readable text format output;
instead, -text was effectively ignored if -counts was not also specified. (With
this patch, -counts is instead ignored if -text is specified, because the
machine-readable text format always includes counts.)
2) When the input data was an IR-level profile, the :ir marker was missing from
the output, resulting in a text format output that would not be usable as
profiling data due to function hash mismatches.
Differential Revision: https://reviews.llvm.org/D51188
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@340592 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This reverts commit r328596.
Checking if the arguments are strings before testing if they contain "/dev/null".
Reviewers: rnk
Reviewed By: rnk
Subscribers: delcypher, llvm-commits
Differential Revision: https://reviews.llvm.org/D44914
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@328603 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This reverts commit 364eb09576 and separates out
the portion that was fixing binary reader error propagation - turns out, there
are production cases where that causes a regression.
Will re-introduce the error propagation fix separately.
The fix to the text reader error propagation is still "in".
Reviewers: bkramer
Reviewed By: bkramer
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D44807
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@328244 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
External functions appearing as indirect call targets could not be
found in the SymTab, and the value:counter record was represented,
in the text format, using an empty string for the name. This would
then cause a silent parsing error when reading.
This CL:
- adds explicit support for such functions
- fixes the places where we would not propagate errors when reading
- addresses a performance issue due to eager resorting of the SymTab.
Reviewers: xur, eraman, davidxl
Reviewed By: davidxl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D44717
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@328132 91177308-0d34-0410-b5e6-96231b3b80d8
Add powerpc- (32-bit) as XFAIL for tests that are documented either in-
line or via commit messages as expected to fail on big-endian systems.
Tests not documented in-line are documented in commit messages as
follows:
r211172 - test/tools/llvm-cov/llvm-cov.test
r247920 - test/Transforms/SampleProfile/gcc-simple.ll
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@322114 91177308-0d34-0410-b5e6-96231b3b80d8
There is data racing to the static variable RecordIndex in index profile reader
when merging in multiple threads. Make it a member variable in
IndexedInstrProfReader to fix this.
Differential Revision: https://reviews.llvm.org/D38431
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@314990 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: As we support multiple callsites for the same location, we need to traverse all locations to get the number of callsites.
Reviewers: davidxl
Reviewed By: davidxl
Subscribers: sanjoy, llvm-commits
Differential Revision: https://reviews.llvm.org/D36246
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309907 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: This patch changes the function profile output order to be deterministic. In order to make it easier to understand, hottest functions (with most total samples) is ordered first.
Reviewers: dnovillo, davidxl
Reviewed By: dnovillo
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D33111
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@302851 91177308-0d34-0410-b5e6-96231b3b80d8
The SampleProfWriter emits function information in an order determined
by the string hash function. The situation is a bit brittle, because
changing the hash function can break the tests.
Instead of sorting the function samples to get a relaible ordering (that
might be too expensive), make the tests not depend on a particular
ordering of function samples.
Differential Revision: https://reviews.llvm.org/D32516
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@301419 91177308-0d34-0410-b5e6-96231b3b80d8
Joel E. Denny