Commit Graph

2563 Commits

Author SHA1 Message Date
Kartik Gautam
7ae6aea145 Bug 1684173 - Add newline character at end of files when missing r=sylvestre,geckoview-reviewers,agi
Differential Revision: https://phabricator.services.mozilla.com/D100484
2021-01-07 08:53:08 +00:00
Gerald Squelart
a82ad0bb16 Bug 1329600 - Capture CPU usage on Linux - r=canaltinova,gcp
Differential Revision: https://phabricator.services.mozilla.com/D99416
2021-01-04 11:37:46 +00:00
Gerald Squelart
8ae4195eeb Bug 1329600 - Capture CPU usage on macos - r=canaltinova
Differential Revision: https://phabricator.services.mozilla.com/D99415
2021-01-04 07:22:59 +00:00
Gerald Squelart
6b6370fc4a Bug 1329600 - Capture CPU usage on Windows - r=canaltinova
Differential Revision: https://phabricator.services.mozilla.com/D99414
2021-01-04 07:19:18 +00:00
Gerald Squelart
ecaf794c38 Bug 1329600 - Scaffolding to output CPU measurements in samples - r=canaltinova
The `RunningTimes` class stores CPU measurements. It may seem overkill for only one value, but in the future more measurements will be added.

During sampling, CPU measurements are collected by platform-specific code. This patch doesn't produce anything yet, see later patches.
These are stored with the samples.
Note that for duplicated samples (when a thread is known to be "asleep"), we still need to collect new measurements, because there could potentially be some activity happening, e.g. in system calls.
Finally the measurements are output as extra "samples" values.

Units for these values may platform-specific, so they are stored in the top-level JSON "meta" object.

We don't collect running times in the Base Profiler (yet), but we still need to add the appropriate field names in the samples' "schema", as expected by profiler.firefox.com.

Differential Revision: https://phabricator.services.mozilla.com/D99413
2021-01-04 07:19:16 +00:00
Gerald Squelart
484485975e Bug 1679930 - Add ActivePS experimental feature "CPU Utilization" - r=canaltinova
This patch adds "CPU Utilization" ("cpu" for short) as a new feature that will control the upcoming still-experimental CPU measurements.

Differential Revision: https://phabricator.services.mozilla.com/D99054
2021-01-04 07:17:38 +00:00
Gerald Squelart
581ab50327 Bug 1679930 - When the sampler thread function starts, it can store all ActivePS features - r=canaltinova
Instead of only capturing one feature (NoStackSampling), the sampler thread now stores all features so that any feature can be quickly looked at during sampling.

Currently this is still limited to NoStackSampling, a later patch will start using another feature.

Differential Revision: https://phabricator.services.mozilla.com/D99053
2021-01-04 07:16:59 +00:00
Cosmin Sabou
2978aa00a3 Backed out changeset dbed1cdf588f (bug 1684173) for mochitest plain and devtools failures. a=backout DONTBUILD 2020-12-28 00:43:51 +02:00
Kartik Gautam
775cdec032 Bug 1684173 - Add newline character at end of files when missing r=sylvestre
Depends on D100443

Differential Revision: https://phabricator.services.mozilla.com/D100484
2020-12-27 11:43:41 +00:00
Gerald Squelart
f1f98f5879 Bug 1678651 - Centralize scExpectedMaximumStackSize in ProfileBufferChunkManager - r=canaltinova
The "expected maximum stack size" (currently 64KiB) value was present in multiple places.
Now it's accessible from everywhere as ProfileBufferChunkManager::scExpectedMaximumStackSize, so it's easier to modify as needed.

Differential Revision: https://phabricator.services.mozilla.com/D100222
2020-12-22 10:50:54 +00:00
Gerald Squelart
5e0acb08c3 Bug 1678651 - Fixed sampler's detection of too-big stacks - r=canaltinova
The profiler sampler was using the "cleared block count" as indication that some data could not be stored.
But this was incorrect, because cleared blocks only account for blocks in chunks that have been destroyed or recycled, but this missed data that could not be stored due to lack of space.
In particular, `ProfileBufferChunkManagerSingle` only provides one chunk, so when it gets full, a new chunk cannot be allocated, and the data is lost; but the "cleared block count" is left unchanged (the one chunk is only released, but not destroyed nor recycled).

Now the sampler correctly looks at the "failed puts bytes", and can report precisely how many bytes could not be stored.

Differential Revision: https://phabricator.services.mozilla.com/D99978
2020-12-22 10:49:30 +00:00
Gerald Squelart
c50f75d970 Bug 1683314 - Convert unknown-marker failure into benign message - r=canaltinova
This gtest failure was useful during early development of markers 2.0 schema, to catch all known (at the time) marker types, so they could be tested.

But now that most marker types live locally where they're used, they cannot easily be unit-tested here, so it's easy to miss some of those types. And more of them will be added in the future, potentially causing more failures here.
At the same time, these Profiler tests run along with lots of other tests, which may "naturally" produce some of those marker types, in which case the corresponding schema will be present in output profiles when running Profiler tests.

So we shouldn't make our tests fail anymore when encountering unknown marker types. They may still be verified in other tests related to the code they live in (e.g., an XPCShell test could exercise some Firefox functionality that generates markers when profiling.)
There is still a benign `printf` message, which may be useful during development, but shouldn't appear as a failure (to be fixed) in CI.

Differential Revision: https://phabricator.services.mozilla.com/D100221
2020-12-21 11:51:52 +00:00
Gerald Squelart
bd9778b416 Bug 1682840 - Remove timelineOverview from expected display locations in BHR marker schema gtest - r=julienw
This is to match the trivial change in bug 1682349.

The failure was intermittent because we only test the BHR marker schema if that marker was actually used, and there's no easy way to force it during our tests; however while running the full gtest suite, it's possible that a previous test triggered that marker and failed in the non-updated test.

Differential Revision: https://phabricator.services.mozilla.com/D99973
2020-12-17 14:18:58 +00:00
Gerald Squelart
08f7a701ea Bug 1681738 - GeckoProfiler.GetProfile gtest: Only check JSON for features that are actually active - r=canaltinova
On some platforms, features may be unavailable. E.g.: StackWalk on Mac non-Nightly.

Differential Revision: https://phabricator.services.mozilla.com/D99412
2020-12-11 09:23:42 +00:00
Gerald Squelart
062e542f6f Bug 1680995 - Profiler gtests: Use JSONOutputCheck in GeckoProfiler.Markers gtest - r=canaltinova
Consolidate all JSON checks to go through JSONOutputCheck.

Differential Revision: https://phabricator.services.mozilla.com/D98893
2020-12-09 09:10:48 +00:00
Gerald Squelart
d336e4f722 Bug 1680995 - Profiler gtests: Use JSONOutputCheck on profiler_get_profile() output - r=canaltinova
Differential Revision: https://phabricator.services.mozilla.com/D98892
2020-12-09 09:10:31 +00:00
Gerald Squelart
275336b264 Bug 1680995 - Profiler gtests: Add some general checks - r=canaltinova
JSONOutputCheck now checks a few more properties, including thread contents, and sub-process profiles.

Differential Revision: https://phabricator.services.mozilla.com/D98891
2020-12-09 09:10:13 +00:00
Gerald Squelart
24b2fcb59f Bug 1680995 - Profiler gtests: JSONOutputCheck now parses profile JSON and its properties - r=canaltinova
JSONOutputCheck used to only check the profile output for the presence of some strings.
Now it parses the output as JSON, and navigates the JSON data to check expected properties, including their types, and values as needed.

Differential Revision: https://phabricator.services.mozilla.com/D98890
2020-12-09 09:09:55 +00:00
Csoregi Natalia
5b6d74bd1d Backed out 4 changesets (bug 1680995) for hazard failures on GeckoProfiler.cpp. CLOSED TREE
Backed out changeset b802e2d38707 (bug 1680995)
Backed out changeset 32045ed29d9e (bug 1680995)
Backed out changeset ff0701c57f86 (bug 1680995)
Backed out changeset 5af1708f901a (bug 1680995)
2020-12-09 10:37:12 +02:00
Barret Rennie
e2fce2bed4 Bug 1677000 - Rename IOUtils.writeAtomic{,UTF8} to IOUtils.write{,UTF8} r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D99001
2020-12-09 05:48:46 +00:00
Razvan Maries
9614a08d73 Backed out 7 changesets (bug 1677000) for perma failures on test_ioutils_read_write.html. CLOSED TREE
Backed out changeset 3521334cfc38 (bug 1677000)
Backed out changeset 737f380c2cf2 (bug 1677000)
Backed out changeset 9dba15fdfe1f (bug 1677000)
Backed out changeset b2fa1a061b15 (bug 1677000)
Backed out changeset a5d7845d3c0c (bug 1677000)
Backed out changeset 8e6081b0c07d (bug 1677000)
Backed out changeset 6592577cf0fa (bug 1677000)
2020-12-09 07:34:21 +02:00
Barret Rennie
0dc367a38f Bug 1677000 - Rename IOUtils.writeAtomic{,UTF8} to IOUtils.write{,UTF8} r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D99001
2020-12-09 04:31:40 +00:00
Gerald Squelart
1bd03378a6 Bug 1680995 - Profiler gtests: Use JSONOutputCheck in GeckoProfiler.Markers gtest - r=canaltinova
Consolidate all JSON checks to go through JSONOutputCheck.

Differential Revision: https://phabricator.services.mozilla.com/D98893
2020-12-09 04:14:56 +00:00
Gerald Squelart
7961b8c71d Bug 1680995 - Profiler gtests: Use JSONOutputCheck on profiler_get_profile() output - r=canaltinova
Differential Revision: https://phabricator.services.mozilla.com/D98892
2020-12-09 04:14:43 +00:00
Gerald Squelart
90f7b56bf6 Bug 1680995 - Profiler gtests: Add some general checks - r=canaltinova
JSONOutputCheck now checks a few more properties, including thread contents, and sub-process profiles.

Differential Revision: https://phabricator.services.mozilla.com/D98891
2020-12-09 04:14:20 +00:00
Gerald Squelart
fba8be0f11 Bug 1680995 - Profiler gtests: JSONOutputCheck now parses profile JSON and its properties - r=canaltinova
JSONOutputCheck used to only check the profile output for the presence of some strings.
Now it parses the output as JSON, and navigates the JSON data to check expected properties, including their types, and values as needed.

Differential Revision: https://phabricator.services.mozilla.com/D98890
2020-12-09 04:14:02 +00:00
Simon Giesecke
1c53236b70 Bug 1679272 - Include ScopeExit.h exactly where used. r=andi
Differential Revision: https://phabricator.services.mozilla.com/D98888
2020-12-07 14:25:59 +00:00
Simon Giesecke
91331998d6 Bug 1679763 - Explicitly instantiate often-used profiler templates. r=gerald
Differential Revision: https://phabricator.services.mozilla.com/D98187
2020-12-04 14:21:57 +00:00
Bogdan Tara
24ca702e18 Backed out changeset fdd4de984fd3 (bug 1679763) for profiler related bustage CLOSED TREE 2020-12-02 21:10:06 +02:00
Simon Giesecke
e74ea6985a Bug 1679763 - Explicitly instantiate often-used profiler templates. r=gerald
Differential Revision: https://phabricator.services.mozilla.com/D98187
2020-12-02 17:00:06 +00:00
Christian Holler
894124e3cf Bug 1677049 - Skip problematic mochitest-bc tests in TSan. r=Gankro,preferences-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D98365
2020-12-01 20:21:20 +00:00
Gerald Squelart
2893c11987 Bug 1676079 - For consistency and clarity, add Marker suffix to all marker types - r=gregtatum
Differential Revision: https://phabricator.services.mozilla.com/D98123
2020-11-30 22:52:44 +00:00
Gerald Squelart
b9191bb2ce Bug 1646613 - Rename profiler overhead variables to include their true unit - r=canaltinova
Differential Revision: https://phabricator.services.mozilla.com/D97993
2020-11-30 10:44:13 +00:00
Sylvestre Ledru
5079cc5b3f Bug 1588710 - Do not fail on stack protector on some asm chromium & breakpad sandboxing code r=mhentges
caused:
[task 2020-11-05T10:14:26.012Z] 10:14:26     INFO -  In file included from Unified_cpp_sandbox_linux2.cpp:137:
[task 2020-11-05T10:14:26.012Z] 10:14:26    ERROR -  /builds/worker/checkouts/gecko/security/sandbox/chromium/sandbox/linux/seccomp-bpf/syscall.cc:369:3: error: Unable to protect inline asm that clobbers stack pointer against stack clash [-Werror,-Wstack-protector]
[task 2020-11-05T10:14:26.012Z] 10:14:26     INFO -    asm volatile(
[task 2020-11-05T10:14:26.013Z] 10:14:26     INFO -    ^
[task 2020-11-05T10:14:26.013Z] 10:14:26     INFO -  1 error generated.

Differential Revision: https://phabricator.services.mozilla.com/D97567
2020-11-27 10:14:01 +00:00
Simon Giesecke
dd80614fa0 Bug 1678062 - Remove unnecessary includes. r=andi
Differential Revision: https://phabricator.services.mozilla.com/D97467

Depends on D96561
2020-11-23 16:12:02 +00:00
Simon Giesecke
2499964dc8 Bug 1676361 - Clean up profiler includes. r=gerald
Differential Revision: https://phabricator.services.mozilla.com/D96539

Depends on D96538
2020-11-23 16:09:13 +00:00
Simon Giesecke
ae75be244a Bug 1677466 - Split Endpoint.h and ProtocolMessageUtils.h from ProtocolUtils.h. r=mccr8
Differential Revision: https://phabricator.services.mozilla.com/D93568

Depends on D93567
2020-11-23 16:06:42 +00:00
Simon Giesecke
971b645fe3 Bug 1660470 - Add missing include directives/forward declarations. r=nika
Differential Revision: https://phabricator.services.mozilla.com/D87865
2020-11-23 16:21:38 +00:00
smolnar
ecaaa56bed Bug 1641000 - Fix linting error on buffer.rst r=lint-fix 2020-11-20 21:38:21 +02:00
Greg Tatum
dbae6744d9 Bug 1641000 - Start documenting the Gecko Profiler; r=gerald
This patch represents joint authoriship by Greg Tatum and Gerald Squelart.
This is a work in progress start to documenting the Gecko Profiler. We
collaborated on this on GitHub.

https://github.com/gregtatum/gecko-dev/tree/profiler-docs/tools/profiler/docs

Differential Revision: https://phabricator.services.mozilla.com/D97655
2020-11-20 18:55:26 +00:00
Florian Quèze
0d164d3a9f Bug 1674476 - ChromeUtils.addProfilerMarker should support capturing a stack or setting the category, r=gerald,gregtatum.
Differential Revision: https://phabricator.services.mozilla.com/D95406
2020-11-20 15:41:16 +00:00
Gerald Squelart
a56d00ddb7 Bug 1678159 - Move WritePropertyTime to SpliceableJSONWriter::TimeProperty - r=gregtatum
TimeStamps in markers must now be streamed through `SpliceableJSONWriter::TimeProperty(name, timestamp)`.
This is consistent with all other JSON-writing functions being in `SpliceableJSONWriter` (and base class `JSONWriter`).

Depends on D97556

Differential Revision: https://phabricator.services.mozilla.com/D97557
2020-11-20 02:23:08 +00:00
Gerald Squelart
7c60972925 Bug 1678181 - Output chartLabel and tableLabel in marker schema - r=gregtatum
Depends on D97555

Differential Revision: https://phabricator.services.mozilla.com/D97556
2020-11-20 02:19:57 +00:00
Gerald Squelart
202a726127 Bug 1678181 - Add thorough tests for custom marker definitions - r=gregtatum
This adds tests for custom marker types, exercizing most streaming and schema functions.

Depends on D96971

Differential Revision: https://phabricator.services.mozilla.com/D97555
2020-11-20 02:18:53 +00:00
Gerald Squelart
11a06b90fa Bug 1671403 - Also prevent re-entrant operations when a mutex is locked in the ProfilerParent - r=canaltinova
Expand `profiler_is_locked_on_current_thread()` to also return true if the ProfileBufferGlobalController mutex is locked on the current thread.
This will prevent some profiler-re-entrant operations (like native allocation markers) from running.

In particular, this removes potential deadlocks similar to the one found in bug 1671403:
- SamplerThread: In the sampling loop, lock the main profiler mutex, run a local update, which attempts to lock the ProfileBufferGlobalController mutex.
- ProfilerChild thread: While processing an IPC message with an update, lock the ProfileBufferGlobalController mutex, then resolve the update, which records a native allocation with a backtrace that attempts to lock the main profiler mutex.
With this patch, the native allocation won't record a marker while the ProfileBufferGlobalController mutex is locked.

Differential Revision: https://phabricator.services.mozilla.com/D96970
2020-11-19 11:48:14 +00:00
Gerald Squelart
d225beb0eb Bug 1671403 - Also prevent re-entrant operations when the ProfilerChild mutex is locked on the current thread - r=canaltinova
Expand `profiler_is_locked_on_current_thread()` to also return true if the ProfilerChild mutex is locked on the current thread.
This will prevent some profiler-re-entrant operations (like native allocation markers) from running.

In particular, this removes the potential deadlock found in bug 1671403:
- SamplerThread: In the sampling loop, lock the main profiler mutex, run a ProfilerChild update, which attempts to lock the ProfilerChild mutex.
- ProfilerChild thread: While processing an IPC message with an update, lock the ProfilerChild mutex, then resolve the update, which records a native allocation with a backtrace that attempts to lock the main profiler mutex.
With this patch, the native allocation won't record a marker while the ProfilerChild mutex is locked.

Differential Revision: https://phabricator.services.mozilla.com/D96969
2020-11-19 11:47:46 +00:00
Markus Stange
8b72e2eb8d Bug 1677868 - Rename "Java Main Thread" to "AndroidUI (Java)" in the profiler. r=canaltinova
Differential Revision: https://phabricator.services.mozilla.com/D97378
2020-11-18 23:15:27 +00:00
Gerald Squelart
c27bc89bad Bug 1666566 - Bump Gecko profile version to 22, to use marker schema - r=gregtatum
Differential Revision: https://phabricator.services.mozilla.com/D96684
2020-11-18 21:57:15 +00:00
Gerald Squelart
b3314a1582 Bug 1675409 - Removed now-unused ProfilerMarkerPayload and all dependencies - r=gregtatum,necko-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D96054
2020-11-18 21:56:57 +00:00
Gerald Squelart
a10f7166db Bug 1675409 - Migrated TracingMarkerPayload to Markers 2.0 API - r=gregtatum
Differential Revision: https://phabricator.services.mozilla.com/D96053
2020-11-18 21:55:51 +00:00