This patch rewrites some parts of the GeckoProfiler code to make it clearer and easier to maintain. It also changes how the profiles get organized into separate folders for each type. Furthermore, the archives no longer have the full directory path in them. To do this, we also have to update browsertime.
Differential Revision: https://phabricator.services.mozilla.com/D102043
mPostMeasurementTimeStamp records the time right after CPU measurements (point-based or interval) ended.
It is then used as the main sample timestamp, to both avoid another TimeStamp::Now() call, and to keep measurements and timestamp as close together as possible (and even closer in the next patch).
Differential Revision: https://phabricator.services.mozilla.com/D101545
This handles the conversion (from TimeStamp to number of milliseconds since process start) once and gives it to subroutines.
It will also help in a following patch where this value will be more closely tied with the CPU usage value, so we need to make sure the sample timestamp is taken at a single point and then forwarded wherever it's needed, be it a duplicate or a real sample.
While here, the nested `delta` variables in the Sampler have been disambiguated for better clarity:
- `sampleStartDeltaMs` is at the start of each sampling loop,
- `threadSampleDeltaMs` is associated to one thread being sampled during that loop.
Differential Revision: https://phabricator.services.mozilla.com/D101544
Now that we use an external dump_syms, we don't need to build
breakpad's.
This means we also don't need the dump_syms_rust_demangle crate anymore.
Differential Revision: https://phabricator.services.mozilla.com/D101865
This patch converts `GeckoChildProcessServices.java` into a jinja template.
We also add an overlay generated from a jinja template for `AndroidManifest.xml`
that provides the definitions for content process services.
Note that even though Gradle supports simple substitution of variables in
manifests, I opted not to use that functionality. Since we need the more
powerful template functionality that jinja provides, I felt that having multiple
ways to substitute information into the manifest would be confusing, so we're
using jinja exclusively.
Differential Revision: https://phabricator.services.mozilla.com/D82578
This patch converts `GeckoChildProcessServices.java` into a jinja template.
We also add an overlay generated from a jinja template for `AndroidManifest.xml`
that provides the definitions for content process services.
Note that even though Gradle supports simple substitution of variables in
manifests, I opted not to use that functionality. Since we need the more
powerful template functionality that jinja provides, I felt that having multiple
ways to substitute information into the manifest would be confusing, so we're
using jinja exclusively.
Differential Revision: https://phabricator.services.mozilla.com/D82578
pylint_requirements.txt fail to install with the new pip resolver due
to a conflict between astroid and lazy-object-proxy.
Rather than bumping those packages and handling the potential fallout,
the package-upgrade has been deferred and we will use the legacy
resolver in the interrim.
Differential Revision: https://phabricator.services.mozilla.com/D99940
In addition to the usual dot-release type of fixes, this also lets us drop a good amount of code that we had patched into our clang 11.
Differential Revision: https://phabricator.services.mozilla.com/D100959
The `RunningTimes` class stores CPU measurements. It may seem overkill for only one value, but in the future more measurements will be added.
During sampling, CPU measurements are collected by platform-specific code. This patch doesn't produce anything yet, see later patches.
These are stored with the samples.
Note that for duplicated samples (when a thread is known to be "asleep"), we still need to collect new measurements, because there could potentially be some activity happening, e.g. in system calls.
Finally the measurements are output as extra "samples" values.
Units for these values may platform-specific, so they are stored in the top-level JSON "meta" object.
We don't collect running times in the Base Profiler (yet), but we still need to add the appropriate field names in the samples' "schema", as expected by profiler.firefox.com.
Differential Revision: https://phabricator.services.mozilla.com/D99413
This patch adds "CPU Utilization" ("cpu" for short) as a new feature that will control the upcoming still-experimental CPU measurements.
Differential Revision: https://phabricator.services.mozilla.com/D99054
Instead of only capturing one feature (NoStackSampling), the sampler thread now stores all features so that any feature can be quickly looked at during sampling.
Currently this is still limited to NoStackSampling, a later patch will start using another feature.
Differential Revision: https://phabricator.services.mozilla.com/D99053
pylint_requirements.txt fail to install with the new pip resolver due
to a conflict between astroid and lazy-object-proxy.
Rather than bumping those packages and handling the potential fallout,
the package-upgrade has been deferred and we will use the legacy
resolver in the interrim.
Differential Revision: https://phabricator.services.mozilla.com/D99940
We need it from both FormAutofillHeuristics and CreditCardRuleset, and it would make a circular import otherwise: FormAutofillHeuristics -> CreditCardRuleset -> FormAutofillHeuristics.
Differential Revision: https://phabricator.services.mozilla.com/D100140
The "expected maximum stack size" (currently 64KiB) value was present in multiple places.
Now it's accessible from everywhere as ProfileBufferChunkManager::scExpectedMaximumStackSize, so it's easier to modify as needed.
Differential Revision: https://phabricator.services.mozilla.com/D100222
The profiler sampler was using the "cleared block count" as indication that some data could not be stored.
But this was incorrect, because cleared blocks only account for blocks in chunks that have been destroyed or recycled, but this missed data that could not be stored due to lack of space.
In particular, `ProfileBufferChunkManagerSingle` only provides one chunk, so when it gets full, a new chunk cannot be allocated, and the data is lost; but the "cleared block count" is left unchanged (the one chunk is only released, but not destroyed nor recycled).
Now the sampler correctly looks at the "failed puts bytes", and can report precisely how many bytes could not be stored.
Differential Revision: https://phabricator.services.mozilla.com/D99978
This gtest failure was useful during early development of markers 2.0 schema, to catch all known (at the time) marker types, so they could be tested.
But now that most marker types live locally where they're used, they cannot easily be unit-tested here, so it's easy to miss some of those types. And more of them will be added in the future, potentially causing more failures here.
At the same time, these Profiler tests run along with lots of other tests, which may "naturally" produce some of those marker types, in which case the corresponding schema will be present in output profiles when running Profiler tests.
So we shouldn't make our tests fail anymore when encountering unknown marker types. They may still be verified in other tests related to the code they live in (e.g., an XPCShell test could exercise some Firefox functionality that generates markers when profiling.)
There is still a benign `printf` message, which may be useful during development, but shouldn't appear as a failure (to be fixed) in CI.
Differential Revision: https://phabricator.services.mozilla.com/D100221
