Summary:
llvm-xray changes:
- account-mode - process-id {...} shows after thread-id
- convert-mode - process {...} shows after thread
- parses FDR and basic mode pid entries
- Checks version number for FDR log parsing.
Basic logging changes:
- Update header version from 2 -> 3
FDR logging changes:
- Update header version from 2 -> 3
- in writeBufferPreamble, there is an additional PID Metadata record (after thread id record and tsc record)
Test cases changes:
- fdr-mode.cc, fdr-single-thread.cc, fdr-thread-order.cc modified to catch process id output in the log.
Reviewers: dberris
Reviewed By: dberris
Subscribers: hiraditya, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D49153
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@336974 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
When there's a mismatch of a function argument being right after the
wrong function, print an offset into the file where that happened, to
ease further debugging.
Reviewers: dberris, eizan, kpw
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D42492
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@323758 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Before this change, the FDR mode implementation relied on at thread-exit
handling to return buffers back to the (global) buffer queue. This
introduces issues with the initialisation of the thread_local objects
which, even through the use of pthread_setspecific(...) may eventually
call into an allocation function. Similar to previous changes in this
line, we're finding that there is a huge potential for deadlocks when
initialising these thread-locals when the memory allocation
implementation is also xray-instrumented.
In this change, we limit the call to pthread_setspecific(...) to provide
a non-null value to associate to the key created with
pthread_key_create(...). While this doesn't completely eliminate the
potential for the deadlock(s), it does allow us to still clean up at
thread exit when we need to. The change is that we don't need to do more
work when starting and ending a thread's lifetime. We also have a test
to make sure that we actually can safely recycle the buffers in case we
end up re-using the buffer(s) available from the queue on multiple
thread entry/exits.
This change cuts across both LLVM and compiler-rt to allow us to update
both the XRay runtime implementation as well as the library support for
loading these new versions of the FDR mode logging. Version 2 of the FDR
logging implementation makes the following changes:
* Introduction of a new 'BufferExtents' metadata record that's outside
of the buffer's contents but are written before the actual buffer.
This data is associated to the Buffer handed out by the BufferQueue
rather than a record that occupies bytes in the actual buffer.
* Removal of the "end of buffer" records. This is in-line with the
changes we described above, to allow for optimistic logging without
explicit record writing at thread exit.
The optimistic logging model operates under the following assumptions:
* Threads writing to the buffers will potentially race with the thread
attempting to flush the log. To avoid this situation from occuring,
we make sure that when we've finalized the logging implementation,
that threads will see this finalization state on the next write, and
either choose to not write records the thread would have written or
write the record(s) in two phases -- first write the record(s), then
update the extents metadata.
* We change the buffer queue implementation so that once it's handed
out a buffer to a thread, that we assume that buffer is marked
"used" to be able to capture partial writes. None of this will be
safe to handle if threads are racing to write the extents records
and the reader thread is attempting to flush the log. The optimism
comes from the finalization routine being required to complete
before we attempt to flush the log.
This is a fairly significant semantics change for the FDR
implementation. This is why we've decided to update the version number
for FDR mode logs. The tools, however, still need to be able to support
older versions of the log until we finally deprecate those earlier
versions.
Reviewers: dblaikie, pelikan, kpw
Subscribers: llvm-commits, hiraditya
Differential Revision: https://reviews.llvm.org/D39526
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@318733 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
The arg1 logging handler changed in compiler-rt to start writing a
different type for entries encountered when logging the first argument
of XRay-instrumented functions. This change allows the trace loader to
support reading these record types as well as prepare for when the
basic (naive) mode implementation starts writing down the argument
payloads.
Without this change, binaries with arg1 logging support enabled start
writing unreadable logs for any of the XRay tracing tools.
Reviewers: pelikan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D38550
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@314967 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
A new FDR metadata record will support logging a function call argument;
appending multiple metadata records will represent a sequence of arguments
meaning that "holes" are not representable by the buffer format. Each
call argument is currently a 64-bit value (useful for "this" pointers and
synchronization objects).
If present, we put this argument to the function call "entry" record it
belongs to, and alter its type to notify the user of its presence.
Reviewers: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D32840
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@314269 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This change adds support for explicit tail-exit records to be written by
the XRay runtime. This lets us differentiate the tail exit
records/events in the log, and allows us to treat those exit events
especially in the future. For now we allow printing those out in YAML
(and reading them in).
Reviewers: kpw, pelikan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D37964
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313514 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
For readers unfamiliar with the XRay code base, reference the compiler-rt
implementation even though we're not allowed to share any code and explain
our little-endian views more clearly.
For code clarity either get rid of obvious comments or explain their
intentions, fix typos, correct coding style according to LLVM's standards
and manually CSE long expressions to point out it is the same expression.
Reviewers: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D34339
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313340 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Adding a new restructuredText file to document the trace format produced with
an FDR mode handler and read by llvm-xray toolset.
Fixed two problems in the documentation from differential review. One bad table
and a missing link in the toc.
Original commit was e97c5836a77db803fe53319c53f3bf8e8b26d2b7.
Reviewers: dberris, pelikan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D36041
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309891 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Adding a new restructuredText file to document the trace format produced with
an FDR mode handler and read by llvm-xray toolset.
Reviewers: dberris, pelikan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D36041
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309836 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
In D30630 we will start writing custom event records. To avoid breaking
the tools that read the FDR mode records, we skip over these records.
To support these custom event records more effectively, we will have to
expose them in the trace loading API. Those changes will be forthcoming.
Reviewers: kpw, pelikan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D33032
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@302856 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
It is problematic for this reader that it expects to read data from
several threads, but the header or message format does not define
framing. Since the buffers are reused, we can't rely on skipping
zeroed out data as a synchronization method either.
There is an argument that this is not version compatible with the format
the reader expected previously. I argue that since the writer wrote garbage
past the end of buffer record, there is no currently working reader to
compromise.
The corresponding writer change is posted to D31384.
Reviewers: dberris, pelikan
Reviewed By: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31385
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@298983 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
The file type packs function trace data onto disk from potentially multiple
threads that are aggregated and flushed during the course of an instrumented
program's runtime.
It is named FDR mode or Flight Data recorder as an analogy to plane
blackboxes, which instrument a running system without access to IO.
The writer code is defined in compiler-rt in xray_fdr_logging.h/cc
Reviewers: rSerge, kcc, dberris
Reviewed By: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D29697
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295397 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
In this change we move the definition of the log reading routines from
the tools directory in LLVM to {include/llvm,lib}/XRay. We improve the
documentation a little bit for the publicly accessible headers, and
adjust the top-matter. This also leads to some refactoring and cleanup
in the tooling code.
In particular, we do the following:
- Rename the class from LogReader to Trace, as it better represents
the logical set of records as opposed to a log.
- Use file type detection instead of asking the user to say what
format the input file is. This allows us to keep the interface
simple and encapsulate the logic of loading the data appropriately.
In future changes we increase the API surface and write dedicated unit
tests for the XRay library.
Depends on D24376.
Reviewers: dblaikie, echristo
Subscribers: mehdi_amini, mgorny, llvm-commits, varno
Differential Revision: https://reviews.llvm.org/D28345
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291652 91177308-0d34-0410-b5e6-96231b3b80d8
