`BaseProfilerMaybeMutex` wraps a `BaseProfilerMutex` inside a `Maybe`.
The decision to use a mutex or not is set at construction time.
If there is no mutex, all operations do nothing (at the small cost of checking
if the mutex is present.)
`BaseProfilerMaybeAutoLock` is the recommented RAII object to lock and
automatically unlock a `BaseProfilerMaybeMutex` until the end of a block scope.
Differential Revision: https://phabricator.services.mozilla.com/D45304
--HG--
extra : moz-landing-system : lando
Instead of copying `BlocksRingBuffer` data byte-by-byte (using iterators byte
dereferencers), we can now use `ModuloBuffer::Iterator::ReadInto(Iterator&)` to
copy them using a small number of `memcpy`s.
Differential Revision: https://phabricator.services.mozilla.com/D45839
--HG--
extra : moz-landing-system : lando
Some objects are copied byte-by-byte to/from `ModuloBuffer`s.
E.g., serialized `BlocksRingBuffer`s, or duplicate stacks. (And more to come.)
`Iterator::ReadInto(Iterator&)` optimizes these copies by using the minimum
number of `memcpy`s possible.
Differential Revision: https://phabricator.services.mozilla.com/D45838
--HG--
extra : moz-landing-system : lando
profiler_can_accept_markers() is a fast and racy check that markers would
currently be stored, it should be used around potentially-expensive calls to
add markers.
And now markers are no longer stored when the profiler is paused. (Note that the
profiler is paused when a profile is being stored, this will help make this
operation faster.)
Differential Revision: https://phabricator.services.mozilla.com/D44434
--HG--
extra : moz-landing-system : lando
Since all profiler data is now stored inside ProfileBuffer, there is no real
need to continuously discard old data during sampling (this was particularly
useful to reclaim memory taken by old markers&payloads).
Instead, we can now just discard the old data once, just before starting to
stream it to JSON.
Differential Revision: https://phabricator.services.mozilla.com/D44433
--HG--
extra : moz-landing-system : lando
Now that what was in ProfilerMarker is stored directly in `BlocksRingBuffer`,
there is no need for this class anymore!
This also removes all the pointer management around it (when added to a TLS
list, moved during sampling, deleted when expired).
Differential Revision: https://phabricator.services.mozilla.com/D43431
--HG--
extra : moz-landing-system : lando
Since payloads are not kept alive long after they have been serialized, we can
just create them on the stack and pass a reference to their base (or pointer,
`nullptr` representing "no payloads") to `profiler_add_marker()`.
Differential Revision: https://phabricator.services.mozilla.com/D43430
--HG--
extra : moz-landing-system : lando
Markers and their payloads can now be serialized straight into the profiler's
`BlocksRingBuffer`, which is now thread-safe to allow such concurrent accesses
(even when gPSMutex is not locked).
This already saves us from having to allocate a `PayloadMarker` on the heap, and
from having to manage it in different lists.
The now-thread-safe `BlocksRingBuffer` in `CorePS` cannot be used inside the
critical section of `SamplerThread::Run`, because any mutex function could hang
because of the suspended thread (even though they functionally don't appear to
interact). So the sampler now uses a local `BlocksRingBuffer` without mutex.
As a bonus, the separate buffer helps reduce the number of concurrent locking
operations needed when capturing the stack.
Differential Revision: https://phabricator.services.mozilla.com/D43429
--HG--
extra : moz-landing-system : lando
Copy the full contents of BlocksRingBuffer into another one.
This is mainly useful to use a temporary buffer to store some data without
contentions, and then integrate the temporary buffer into the main one.
Differential Revision: https://phabricator.services.mozilla.com/D45306
--HG--
extra : moz-landing-system : lando
Payloads will serialize themselves into a `BlocksRingBuffer` entry when first
captured.
Later they will be deserialized, to stream JSON for the output profile.
Differential Revision: https://phabricator.services.mozilla.com/D43428
--HG--
extra : moz-landing-system : lando
The common data members stored in the ProfilerMarkerPayload base class can be
gathered into a struct, which will make it easier to pass around, especially
when a derived object is constructed with these common properties.
Differential Revision: https://phabricator.services.mozilla.com/D43427
--HG--
extra : moz-landing-system : lando
Markers may contain backtraces, so we need to be able to de/serialize them.
This involves de/serializing the underyling `BlocksRingBuffer`.
Differential Revision: https://phabricator.services.mozilla.com/D43426
--HG--
extra : moz-landing-system : lando
Add basic testing of `profiler_get_backtrace` before working on serializing it.
Differential Revision: https://phabricator.services.mozilla.com/D43424
--HG--
extra : moz-landing-system : lando
The main `BlocksRingBuffer`s will be stored in `CorePS` (outside of
`ProfileBuffer`s), as we need to be able to safely access the underlying buffers
when profilers are not enabled.
Also `ProfilerBacktrace` will own the `BlocksRingBuffer` that its captured
`ProfileBuffer` uses.
Taking this opportunity to rename the different `mBuffer`s to more useful names.
Differential Revision: https://phabricator.services.mozilla.com/D43422
--HG--
extra : moz-landing-system : lando
Now that we are using a byte-oriented `BlocksRingBuffer` instead of an array of
9-byte `ProfileBufferEntry`'s, internally the profiler sets the buffer size in
bytes. However all external users (popup, tests, etc.) still assume that the
requested capacity is in entries!
To limit the amount of changes, we will keep assuming externally-visible
capacities are in entries, and convert them to bytes.
Even though entries used to be 9 bytes each, and `BlocksRingBuffer` adds 1 byte
for the entry size, some entries actually need less space (e.g., 32-bit numbers
now take 6 bytes instead of 9), so converting to less than 9 bytes per entry is
acceptable.
We are settling on 8 bytes per entry: It's close to 9, and it's a power of two;
since the effective number of entries was a power of two, and `BlocksRingBuffer`
also uses a power of two size in bytes, this convertion keeps sizes in powers of
two.
Differential Revision: https://phabricator.services.mozilla.com/D44953
--HG--
extra : moz-landing-system : lando
This just replaces `ProfileBuffer`'s self-managed circular buffer with a
`BlocksRingBuffer`.
That `BlocksRingBuffer` does not need its own mutex (yet), because all uses go
through gPSMutex-guarded code.
`ProfileBuffer` now also pre-allocates a small buffer for use in
`DuplicateLastSample()`, this avoids multiple mallocs at each sleeping thread
stack duplication.
Note: Internal "magic" sizes have been multiplied by 8 (and tweaked upwards, to
handle bigger stacks), because they originally were the number of 9-byte
entries, but now it's the buffer size in bytes. (And entries can now be smaller
than 9 bytes, so overall the capacity in entries should be similar or better.)
However, external calls still think they are giving a number of "entries", this
will be handled in the next patch.
Differential Revision: https://phabricator.services.mozilla.com/D43421
--HG--
extra : moz-landing-system : lando
Add some stats (off by default) around streaming JSON, as the following patches
may affect it.
Differential Revision: https://phabricator.services.mozilla.com/D44952
--HG--
extra : moz-landing-system : lando
In some situations we will *need* to use a `BlocksRingBuffer` that absolutely
does not use a mutex -- In particular in the critical section of
`SamplerThread::Run()`, when a thread is suspended, because any action on any
mutex (even a private one that no-one else interacts with) can result in a hang.
As a bonus, `BlocksRingBuffer`s that are known not to be used in multi-thread
situations (e.g., backtraces, extracted stacks for duplication, etc.) will waste
less resources trying to lock/unlock their mutex.
Differential Revision: https://phabricator.services.mozilla.com/D45305
--HG--
extra : moz-landing-system : lando
`BaseProfilerMaybeMutex` wraps a `BaseProfilerMutex` inside a `Maybe`.
The decision to use a mutex or not is set at construction time.
If there is no mutex, all operations do nothing (at the small cost of checking
if the mutex is present.)
`BaseProfilerMaybeAutoLock` is the recommented RAII object to lock and
automatically unlock a `BaseProfilerMaybeMutex` until the end of a block scope.
Differential Revision: https://phabricator.services.mozilla.com/D45304
--HG--
extra : moz-landing-system : lando
Having a full VPATH for the srcdir sometimes causes make to grab the
wrong prerequisite for a rule, in particular if we have a file in the
srcdir and also generate a file of the same name in the objdir. We don't
really need VPATH anymore though, since most of the information comes
from mozbuild, where we can explicitly list the path to the srcdir or
objdir as necessary.
Differential Revision: https://phabricator.services.mozilla.com/D42968
--HG--
extra : moz-landing-system : lando
`WindowsErrorResult` is a class to hold either a value or a Windows error
code based on the `Result` template. We also have `LauncherResult` for the
same purpose, which was introduced as a part of the launcher process feature
afterward. The difference is `LauncherResult` holds a filename and line
number along with an error code.
This patch integrates LauncherResult.h into WinHeaderOnlyUtils.h so that we
can use `LauncherResult` more broadly.
Differential Revision: https://phabricator.services.mozilla.com/D44512
--HG--
extra : moz-landing-system : lando
Gather stats for most calls to `profiler_add_marker()`, including the time to
allocate payloads if any.
Differential Revision: https://phabricator.services.mozilla.com/D43420
--HG--
extra : moz-landing-system : lando
All calls to `profiler_add_marker()` (outside of the profilers code) are
now replaced by either:
- `PROFILER_ADD_MARKER(name, categoryPair)`
- `PROFILER_ADD_MARKER_WITH_PAYLOAD(name, categoryPair, TypeOfMarkerPayload,
(payload, ..., arguments))`
This makes all calls consistent, and they won't need to prefix the category pair
with `JS::ProfilingCategoryPair::`.
Also it will make it easier to add (and later remove) internal-profiling
instrumentation (bug 1576550), and to replace heap-allocated payloads with
stack-allocated ones (bug 1576555).
Differential Revision: https://phabricator.services.mozilla.com/D43588
--HG--
extra : moz-landing-system : lando
Bug 1556993 fixed the crash in PoisonIOInterposer (due to missing stdout&stderr
fd's), bug 1559379 fixed the mozglue memory issue that the unit test
experienced, and this patch also includes a fix for the test failure in
browser_checkdllblockliststate.js.
So now Base Profiler can be enabled by default on Windows, still using
`MOZ_BASE_PROFILER_...` env-vars for now (upcoming bug will merge these with
non-BASE env-vars soon).
Differential Revision: https://phabricator.services.mozilla.com/D44091
--HG--
extra : moz-landing-system : lando
Whereas previously MozDescribeCodeAddress would have handled demangling,
we need to explicitly do that from our new GetFunction method. The string we
generate is now more useful for the profiler to merge -- having dropped the
address in the previous patch, and the file & line number and library in this
patch.
While we're at it, try to demangle Rust symbols too.
Ideally we'd add Rust symbol handling to DemangleSymbol in
StackWalk.cpp, but that lives in mozglue, which currently cannot have
any Rust crate dependencies.
Differential Revision: https://phabricator.services.mozilla.com/D43142
--HG--
extra : moz-landing-system : lando
Use AUTO_PROFILER_STATS in both profilers, in:
- SamplerThread::Run() calling DoPeriodicSample()
- racy_profiler_add_marker
- ProfileBuffer::DeleteExpiredStoredMarkers()
This should cover all areas affected by the upcoming changes to the
ProfileBuffer storage, and how markers are stored.
Differential Revision: https://phabricator.services.mozilla.com/D42826
--HG--
extra : moz-landing-system : lando
AUTO_PROFILER_STATS(name) can be used to time a {block}.
Statistics are gathered in a function-static variable, and printf'd when the
program ends.
Differential Revision: https://phabricator.services.mozilla.com/D42825
--HG--
extra : moz-landing-system : lando
Some users will want to lock the buffer but not do any specific operation with
it.
Differential Revision: https://phabricator.services.mozilla.com/D42824
--HG--
extra : moz-landing-system : lando
Backtraces (that are kept in some marker payloads) are stored in a small
ProfileBuffer, we will need to store that data, which will happen to be inside
a BlockRingBuffer, so BlockRingBuffer needs to be able to (de)serialize itself!
This is done by storing the contents in the active buffer range, and some extra
data, to later reconstruct a BlocksRingBuffer that looks like the original.
Depends on D42496
Differential Revision: https://phabricator.services.mozilla.com/D42634
--HG--
extra : moz-landing-system : lando
Markers and their payloads contain all kinds of objects that we'll need to
serialize into a BlocksRingBuffer (new ProfileBuffer storage).
This patch will add functions to:
- Compute the size needed to store objects,
- Write multiple objects into a BlockRingBuffer entry,
- Read objects back from an entry.
And it will provide a number of useful de/serialization helpers for:
- Trivially-copyable objects,
- Strings of different types,
- Raw pointers (with some safety guards to avoid surprises),
- Tuples (to store multiple sub-objects),
- Spans,
- Maybe (for optional objects),
- Variant.
This should be enough to store most kinds of data. Further specializations
can&will be written as necessary for more complex or obscure types.
Differential Revision: https://phabricator.services.mozilla.com/D42496
--HG--
extra : moz-landing-system : lando
This allows its use in std algorithms and types that require a real iterator
(like `template <typename InputIt> std::string::string(InputIt, InputIt)`).
Differential Revision: https://phabricator.services.mozilla.com/D42452
--HG--
extra : moz-landing-system : lando
`Reader::At()` can be used to get a `BlockIterator` at a given `BlockIndex`,
clamped between `begin()` and `end()`.
This will be useful when we want to iterate starting at a given index, e.g.,
when duplicating stacks.
Differential Revision: https://phabricator.services.mozilla.com/D42449
--HG--
extra : moz-landing-system : lando
The main goal of these bugs is to move markers to a new storage, so I'm adding
lots of markers to TestBaseProfiler.
Also adding labels, easier to read unsymbolicated profiles, and gives a bit more
coverage too.
And adding a separate "fibonacci canceller" thread, which is needed on some
slower platforms (e.g., Linux 64 ASAN times out otherwise); as a bonus, this
tests AUTO_BASE_PROFILER_REGISTER_THREAD.
Differential Revision: https://phabricator.services.mozilla.com/D42448
--HG--
extra : moz-landing-system : lando
(Also MOZ_BASE_PROFILER_STARTUP.)
This makes it easier to keep that variable setup in the environment, and change
its value to switch between enabling and disabling the profiler.
Differential Revision: https://phabricator.services.mozilla.com/D42447
--HG--
extra : moz-landing-system : lando
In practice the Reader doesn't need to be copied/moved/reassign.
BlocksRingBuffer::Read() can just instantiate one on the stack, and pass it by
reference to callbacks.
Differential Revision: https://phabricator.services.mozilla.com/D42118
--HG--
extra : moz-landing-system : lando
The point of the EntryReserver was mainly to have an object that represented a
writing lock on BlocksRingBuffer, so potentially perform multiple consecutive
writes.
After some experience implementing bug 1562604, there's actually no need for it.
So instead of having `Put()` create an `EntryReserver`, we now have
`ReserveAndPut()` that does the whole work in one function.
Differential Revision: https://phabricator.services.mozilla.com/D42116
--HG--
extra : moz-landing-system : lando
EntryWriter doesn't even need to be moveable, as BlocksRingBuffer can just
create one on the stack, and pass it by reference to callbacks.
This removes risks, and potential data copies.
Differential Revision: https://phabricator.services.mozilla.com/D42115
--HG--
extra : moz-landing-system : lando
Actually, we use clang for all Linux and Android platform, so it is no reason to use frame pointer based stack walker for LUL on Android/x86 and Android/x86-64 if no DWARF rule.
Differential Revision: https://phabricator.services.mozilla.com/D41320
--HG--
extra : moz-landing-system : lando
While mozglue continues to be the correct location for calling the affected
code in this patch, the calls requiring profiler labels will soon be
originating from firefox.exe via the launcher process.
mozglue will be supplying the launcher process with an interface that consists
of what are effectively "OnBeginDllLoad" and "OnEndDllLoad" callback
notifications; obviously an RAII class is not going to be useful for that case.
We still want to keep the RAII stuff around, however, since we still need it
for cases where we need to fall back to using the legacy DLL blocklist.
Differential Revision: https://phabricator.services.mozilla.com/D41807
--HG--
extra : moz-landing-system : lando
While mozglue continues to be the correct location for calling the affected
code in this patch, the calls requiring stackwalk suppression will soon be
originating from firefox.exe via the launcher process.
mozglue will be supplying the launcher process with an interface that consists
of what are effectively "OnBeginDllLoad" and "OnEndDllLoad" callback
notifications; obviously an RAII class is not going to be useful for that case.
We still want to keep the RAII stuff around, however, since we still need it
for cases where we need to fall back to using the legacy DLL blocklist.
Differential Revision: https://phabricator.services.mozilla.com/D41808
--HG--
extra : moz-landing-system : lando
`BlocksRingBuffer` will be used both inside and outside `ProfileBuffer`:
- Inside to serve as `ProfileBuffer`'s main storage for stack traces,
- Outside to allow marker storage even when `ProfileBuffer` is locked during
stack sampling.
`ProfileBuffer` only exists while `ActivePS` is alive, but because of the
potential outside accesses above (due to small races between ProfileBuffer
shutdown, and thread-local IsBeingProfiled() flags), we cannot just do the same
for BlocksRingBuffer, and it must remain alive to gracefully deny these accesses
around the profiler startup and shutdown times.
To accomplish this, `BlocksRingBuffer` may be in different states:
- "In-session", we have a real buffer to write to and read from,
- "Out-of-session", without buffer so reads&writes do nothing.
This is implemented by enclosing the underlying `ModuloBuffer` and the entry
deleter in a `Maybe`, which may be `Nothing` when the profiler is not running
and the `ProfileBuffer`'s `BlocksRingBuffer` is out-of-session.
Differential Revision: https://phabricator.services.mozilla.com/D41519
--HG--
extra : moz-landing-system : lando
