This patch rolls up all the required changes for this purpose. Since the
whole crash reporting flow must understand the new format it's not possible
to land this as separate patches as individually they would be broken. This
patch includes the following changes:
* Changes to the crash reporting machinery to write out annotations as JSON,
these includes changes to the DLL blocklist code that must be run at crash
time.
* Modifications to the crash reporter client so that it can read and
submit the new format; this includes platform-specific changes to the
Breakpad libraries it uses for submitting crashes.
* Modifications to the minidump-analyzer to understand and process the new
format correctly.
* Modifications to the crash manager to understand and process the new format
correctly.
* Modifications to GeckoView's crash handler to understand and submit the
new format correctly.
* Added new tests to cover the new format and modified existing ones to
accomodate the new one.
Differential Revision: https://phabricator.services.mozilla.com/D46848
--HG--
extra : moz-landing-system : lando
This is especially useful when debugging both Base Profiler and Gecko Profiler,
so that their messages are correctly sequenced.
Differential Revision: https://phabricator.services.mozilla.com/D54447
--HG--
extra : moz-landing-system : lando
The Base Profiler originally named the main thread "Main Thread", which is
friendlier than "GeckoMain". However this makes it more difficult to combine the
controls for both profilers if they use different names.
So now both profilers use "GeckoMain", so filters can be exactly the same.
Base Profiler adds "(pre-xul)" to the name to distinguish tracks in the
frontend -- This distinction is actually necessary so the frontend doesn't get
confused by threads with the exact same name, but eventually tracks will get
combined in the frontend as well.
Differential Revision: https://phabricator.services.mozilla.com/D54444
--HG--
extra : moz-landing-system : lando
This is especially useful when debugging both Base Profiler and Gecko Profiler,
so that their messages are correctly sequenced.
Differential Revision: https://phabricator.services.mozilla.com/D54447
--HG--
extra : moz-landing-system : lando
The Base Profiler originally named the main thread "Main Thread", which is
friendlier than "GeckoMain". However this makes it more difficult to combine the
controls for both profilers if they use different names.
So now both profilers use "GeckoMain", so filters can be exactly the same.
Base Profiler adds "(pre-xul)" to the name to distinguish tracks in the
frontend -- This distinction is actually necessary so the frontend doesn't get
confused by threads with the exact same name, but eventually tracks will get
combined in the frontend as well.
Differential Revision: https://phabricator.services.mozilla.com/D54444
--HG--
extra : moz-landing-system : lando
Our detour cannot handle assembly patterns which is injected by the code coverage
instrumentation. We need to skip them in CCov build.
Differential Revision: https://phabricator.services.mozilla.com/D54745
--HG--
extra : moz-landing-system : lando
For now, there is no flag to actually allow it, but this is the
code-side changes to allow the linker being disabled.
Differential Revision: https://phabricator.services.mozilla.com/D54074
--HG--
extra : moz-landing-system : lando
The custom linker used on Android (FaultyLib), can be enabled on
non-Android, and in the near future, may not always be enabled on
Android, so the FaultyLib-specific parts of the profile need to not be
specific to Android, but to the linker being enabled instead.
Differential Revision: https://phabricator.services.mozilla.com/D54073
--HG--
extra : moz-landing-system : lando
This function works on all GeckoProcessTypes, not just those for child
processes.
Differential Revision: https://phabricator.services.mozilla.com/D54375
--HG--
extra : moz-landing-system : lando
This patch adds the following pattern to our x64 detour so that we can hook APIs
even though a target is already detoured by another application.
```
mov rax, imm64
push rax
ret
```
We already have `PatchIfTargetIsRecognizedTrampoline` to detour the pattern
`mov; jmp`. There is another variation using `push rax;ret` to jump.
Differential Revision: https://phabricator.services.mozilla.com/D53877
--HG--
extra : moz-landing-system : lando
Passing a dummy object to `NtQueryFullAttributesFile` to avoid AV on WOW64.
Differential Revision: https://phabricator.services.mozilla.com/D53876
--HG--
extra : moz-landing-system : lando
We want the profiler UI to be able to know if the data can be used for
reconstructing the event delays, since it measures something different
from the old 16ms event injection.
Differential Revision: https://phabricator.services.mozilla.com/D52534
--HG--
extra : moz-landing-system : lando
This patch rolls up all the required changes for this purpose. Since the
whole crash reporting flow must understand the new format it's not possible
to land this as separate patches as individually they would be broken. This
patch includes the following changes:
* Changes to the crash reporting machinery to write out annotations as JSON,
these includes changes to the DLL blocklist code that must be run at crash
time.
* Modifications to the crash reporter client so that it can read and
submit the new format; this includes platform-specific changes to the
Breakpad libraries it uses for submitting crashes.
* Modifications to the minidump-analyzer to understand and process the new
format correctly.
* Modifications to the crash manager to understand and process the new format
correctly.
* Modifications to GeckoView's crash handler to understand and submit the
new format correctly.
* Added new tests to cover the new format and modified existing ones to
accomodate the new one.
Differential Revision: https://phabricator.services.mozilla.com/D46848
--HG--
extra : moz-landing-system : lando
`patched_NtMapViewOfSection` uses the process default heap to copy a string.
However, `patched_NtMapViewOfSection` can be invoked even before the process
heap is initialized. One example we found is Windows Defender's EAF, with
which "verifier.dll" is loaded before the process heap is initialized.
This patch adds a check whether the heap is initialized or not in
`patched_NtMapViewOfSection` and `NativeNtBlockSet::Add`. This also minimizes
the usage of the heap, i.e. not copying a string when we block a dll.
Differential Revision: https://phabricator.services.mozilla.com/D51028
--HG--
extra : moz-landing-system : lando
This moves the parts of toolkit/library/rust/shared/lib.rs related to
panic hooking to a new mozglue subdirectory, which will be used for
things that can be statically linked to e.g. libxul, rather than in
a "shared library".
The panic hook is disabled when building spidermonkey via the mozjs_sys
crate.
Differential Revision: https://phabricator.services.mozilla.com/D52793
--HG--
extra : moz-landing-system : lando
This moves the parts of toolkit/library/rust/shared/lib.rs related to
panic hooking to a new mozglue subdirectory, which will be used for
things that can be statically linked to e.g. libxul, rather than in
a "shared library".
The panic hook is disabled when building spidermonkey via the mozjs_sys
crate.
Differential Revision: https://phabricator.services.mozilla.com/D52793
--HG--
extra : moz-landing-system : lando
This moves the parts of toolkit/library/rust/shared/lib.rs related to
panic hooking to a new mozglue subdirectory, which will be used for
things that can be statically linked to e.g. libxul, rather than in
a "shared library".
The panic hook is disabled when building spidermonkey via the mozjs_sys
crate.
Differential Revision: https://phabricator.services.mozilla.com/D52793
--HG--
extra : moz-landing-system : lando
We want the profiler UI to be able to know if the data can be used for
reconstructing the event delays, since it measures something different
from the old 16ms event injection.
Differential Revision: https://phabricator.services.mozilla.com/D52534
--HG--
extra : moz-landing-system : lando
Avoids deadlocks on Windows due to Now() taking a lock; if done while we've
paused a thread that holds the lock we will deadlock.
Differential Revision: https://phabricator.services.mozilla.com/D52392
--HG--
extra : moz-landing-system : lando
Some applications tamper Import Directory entry of a loaded executable image
to pretend static dependency on their module. We have `RestoreImportDirectory`
to revert it in the browser process. If tampering happened in the launcher
process, however, we failed to get an IAT thunk address via `GetIATThunksForModule`
because it could be located outside the mapped image.
With this patch, we skip bounds check in `GetIATThunksForModule` if we detect
tampering in the launcher process. We can proceed safely because it's expected
that Import Directory is still valid though it's located outside.
Differential Revision: https://phabricator.services.mozilla.com/D49690
--HG--
extra : moz-landing-system : lando
profile.counters[n].sample_groups was mistakenly streamed as an object, which
prevents having more than one, and goes against the published format
documentation.
The front-end was implemented to process the incorrect format, so it will need
to be updated as well; hence the version change to 18.
Differential Revision: https://phabricator.services.mozilla.com/D49867
--HG--
extra : moz-landing-system : lando
In Android gtest, instead of pushing the gtest libxul.so to the
(assumed and potentially incorrect) location of the test application's
libraries, push libxul.so to /data/local/gtest, and direct the linker
to use that instead.
Differential Revision: https://phabricator.services.mozilla.com/D34942
--HG--
extra : moz-landing-system : lando
Having `mProfileBuffer` be a pointer is not really helpful:
- The pointer is never null (It's allocated on ActivePS construction, and
implicitly deleted on ActivePS destruction); it's never moved-from.
- It requires an extra `new` and an extra `delete`.
Differential Revision: https://phabricator.services.mozilla.com/D48650
--HG--
extra : moz-landing-system : lando
Add assertions that all `sInstance` pointers (from both `CorePS` and `ActivePS`)
are not null before being dereferenced.
This is probably more than needed, but it's only `MOZ_ASSERT`s limited to
Nightly, and it should give better feedback in case something goes wrong.
Eventually, I think it would be better to make most methods non-static, and have
a checked reference-to-instance getter.
Differential Revision: https://phabricator.services.mozilla.com/D48649
--HG--
extra : moz-landing-system : lando
We are not simply excluding all about:blanks because there might be some
about:blank that user really visits. But for others we don't want to include
the first about:blank because when a BrowsingContext is loaded, and if the
principal matches, the first document loaded in it will share the inner window.
Differential Revision: https://phabricator.services.mozilla.com/D47067
--HG--
extra : moz-landing-system : lando
We were keeping nsDocShell::mHistoryId and nsDocShell::mOSHE as keys. They
weren't quite good because:
1. While loading an iframe, they were being registered twice with the same
ids(for about:blank and the real URL) sometimes.
2. It wasn't possible to access to the parent mHistoryId and mOSHE from a child
processes if the parent is in a different process. That may not be the case for
now, but it will be after fission.
So we had to find other IDs to:
1. Determine the Tab of the frames.
2. Determine the URLs of the frames.
For the first use case, we were using nsDocShell::mHistoryId for that purpose
but that was wrong. The closest thing that we can get to a tab ID is
BrowsingContext ID because they don't change after a navigation. But iframes
have different BrowsingContext's, so we still need to create a tree to
construct a tab content. That can be either in the front-end or capture time.
For the second use case, we were using a key pair of mHistoryId and mOSHE. We
now chose to keep inner window IDs for that purpose. Inner window IDs are
unique for each navigation loads because inner window correspond to each JS
window global objects. That's why we can use that without any problem. But one
problem is that we cannot handle `history.pushState` and `history.replaceState`
changes with that change since window global objects won't change during those.
But that was the best thing we can do after fission. So this will be a small
sacrifice for us to keep that functionality working after fission.
In that patch we also remove the registration/unregistration calls. We are
going to add those calls in the next patch.
Differential Revision: https://phabricator.services.mozilla.com/D47065
--HG--
extra : moz-landing-system : lando
Previously, the absence of "stackwalk", "leaf", and "javascript" implied that
the test/user didn't want any sampling, but this caused issues in some tests
that enabled "stackwalk" on platforms that didn't support stack-walking, which
ended up suppressing label-only stacks that the test expected.
we now have an explicit feature "nostacksampling" that disables backtraces from
the samplers in both profilers. This effectively cancels "stackwalk", "leaf",
and "javascript" if present.
Differential Revision: https://phabricator.services.mozilla.com/D47731
--HG--
extra : moz-landing-system : lando
Previously, the absence of "stackwalk", "leaf", and "javascript" implied that
the test/user didn't want any sampling, but this caused issues in some tests
that enabled "stackwalk" on platforms that didn't support stack-walking, which
ended up suppressing label-only stacks that the test expected.
we now have an explicit feature "nostacksampling" that disables backtraces from
the samplers in both profilers. This effectively cancels "stackwalk", "leaf",
and "javascript" if present.
Differential Revision: https://phabricator.services.mozilla.com/D47731
--HG--
extra : moz-landing-system : lando
I also fixed a couple of typos in comments in the python script that didn't
seem to warrant their own bug.
Differential Revision: https://phabricator.services.mozilla.com/D47455
--HG--
extra : moz-landing-system : lando
Defines an android-only version of nsTraceRefcnt::WalkTheStack that takes a function callback, which outputs the stack frame buffer to `__android_log_print`. Also uses `__wrap_dladdr` in MozDescribeCodeAddress, which outputs slightly more informative data for the stack trace (instead of instances of '???/??? [???]').
Differential Revision: https://phabricator.services.mozilla.com/D46868
--HG--
extra : moz-landing-system : lando
Defines an android-only version of nsTraceRefcnt::WalkTheStack that takes a function callback, which outputs the stack frame buffer to `__android_log_print`. Also uses `__wrap_dladdr` in MozDescribeCodeAddress, which outputs slightly more informative data for the stack trace (instead of instances of '???/??? [???]').
Differential Revision: https://phabricator.services.mozilla.com/D46868
--HG--
extra : moz-landing-system : lando
This assertion could fire if the profiler state changes between beginning and
ending a DLL load. That's less than ideal, so let's just get rid of it.
Differential Revision: https://phabricator.services.mozilla.com/D46897
--HG--
extra : moz-landing-system : lando
`BlocksRingBuffer` had an "entry destructor" to make it a more generic
container, and it was useful during early prototyping of the new profiler
storage (so that we could store owning pointers).
But this entry destructor is stored in an `std::function`, which gets marked as
a potential GC caller by the js rooting hazard analyzer; and as this bug found,
it's not obvious where to place `JS::AutoSuppressGCAnalysis`, because profiler
entries (including stacks) could be added on one thread while GC happens
elsewhere, which triggers the embedded `AutoAssertNoGC` check.
Since we don't actually use the entry destructor facility in the profiler, it's
easier to just get rid of it. As a bonus, it's a small optimization.
Tests that were using an entry destructor now use the `State` instead, to verify
that entries are pushed and cleared as expected.
If needed in the future outside of the profiler, `BlocksRingBuffer` could again
include an entry destructor, but it would have to be through templating, so that
the class used in the profiler wouldn't contain an `std::function`.
Differential Revision: https://phabricator.services.mozilla.com/D46738
--HG--
extra : moz-landing-system : lando
* At this point our DLL blocking infra is complicated enough that I decided to
bite the bullet and move all of this code out of `mozglue/build` and into its
own subdirectory, `mozglue/dllservices`.
* We delete the original `UntrustedDllsHandler` code which is now obsolete.
* We implement mozglue's `LoaderObserver`:
** When this observer registers itself with the launcher process API, it
receives a vector containing all saved records of loaded DLLs that happened
until that moment.
** This code handles profiler labels and stackwalking suppression.
** Once a load has completed, we either pass the load on to XUL for further
processing, or save it for later if XUL is not initialized yet.
* mozglue has its own `ModuleLoadFrame` implementation for the legacy blocklist.
* `DllServicesBase` is updated to support the new interfaces.
* We implement `FallbackLoaderAPI` for `plugin-container`, `xpcshell`, and
any other non-`firefox` processes that do not have a launcher process
providing a loader API.
* We add some wide to UTF8 conversion functions.
Depends on D43157
Differential Revision: https://phabricator.services.mozilla.com/D43158
--HG--
rename : mozglue/build/Authenticode.cpp => mozglue/dllservices/Authenticode.cpp
rename : mozglue/build/Authenticode.h => mozglue/dllservices/Authenticode.h
rename : browser/app/winlauncher/freestanding/LoaderAPIInterfaces.h => mozglue/dllservices/LoaderAPIInterfaces.h
rename : browser/app/winlauncher/freestanding/ModuleLoadInfo.h => mozglue/dllservices/ModuleLoadInfo.h
rename : browser/app/winlauncher/NtLoaderAPI.h => mozglue/dllservices/NtLoaderAPI.h
rename : mozglue/build/WindowsDllBlocklist.cpp => mozglue/dllservices/WindowsDllBlocklist.cpp
rename : mozglue/build/WindowsDllBlocklist.h => mozglue/dllservices/WindowsDllBlocklist.h
rename : mozglue/build/WindowsDllBlocklistCommon.h => mozglue/dllservices/WindowsDllBlocklistCommon.h
rename : mozglue/build/WindowsDllBlocklistDefs.in => mozglue/dllservices/WindowsDllBlocklistDefs.in
rename : mozglue/build/WindowsDllServices.h => mozglue/dllservices/WindowsDllServices.h
rename : mozglue/build/gen_dll_blocklist_defs.py => mozglue/dllservices/gen_dll_blocklist_defs.py
rename : mozglue/build/moz.build => mozglue/dllservices/moz.build
rename : mozglue/build/MozglueUtils.h => mozglue/misc/WinUtils.h
extra : moz-landing-system : lando
This patch adds the following:
* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
owns its buffer. The buffers are null-terminated so that they may be used as
C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
current thread ID directly from the `TEB` when we do not yet have access to
kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
MFBT `Vector` in code that might not yet have access to Win32 heap functions.
Differential Revision: https://phabricator.services.mozilla.com/D43155
--HG--
extra : moz-landing-system : lando
* At this point our DLL blocking infra is complicated enough that I decided to
bite the bullet and move all of this code out of `mozglue/build` and into its
own subdirectory, `mozglue/dllservices`.
* We delete the original `UntrustedDllsHandler` code which is now obsolete.
* We implement mozglue's `LoaderObserver`:
** When this observer registers itself with the launcher process API, it
receives a vector containing all saved records of loaded DLLs that happened
until that moment.
** This code handles profiler labels and stackwalking suppression.
** Once a load has completed, we either pass the load on to XUL for further
processing, or save it for later if XUL is not initialized yet.
* mozglue has its own `ModuleLoadFrame` implementation for the legacy blocklist.
* `DllServicesBase` is updated to support the new interfaces.
* We implement `FallbackLoaderAPI` for `plugin-container`, `xpcshell`, and
any other non-`firefox` processes that do not have a launcher process
providing a loader API.
* We add some wide to UTF8 conversion functions.
Differential Revision: https://phabricator.services.mozilla.com/D43158
--HG--
rename : mozglue/build/Authenticode.cpp => mozglue/dllservices/Authenticode.cpp
rename : mozglue/build/Authenticode.h => mozglue/dllservices/Authenticode.h
rename : mozglue/build/WindowsDllBlocklist.cpp => mozglue/dllservices/WindowsDllBlocklist.cpp
rename : mozglue/build/WindowsDllBlocklist.h => mozglue/dllservices/WindowsDllBlocklist.h
rename : mozglue/build/WindowsDllBlocklistCommon.h => mozglue/dllservices/WindowsDllBlocklistCommon.h
rename : mozglue/build/WindowsDllBlocklistDefs.in => mozglue/dllservices/WindowsDllBlocklistDefs.in
rename : mozglue/build/WindowsDllServices.h => mozglue/dllservices/WindowsDllServices.h
rename : mozglue/build/gen_dll_blocklist_defs.py => mozglue/dllservices/gen_dll_blocklist_defs.py
rename : mozglue/build/moz.build => mozglue/dllservices/moz.build
rename : mozglue/build/MozglueUtils.h => mozglue/misc/WinUtils.h
extra : moz-landing-system : lando
This patch adds the following:
* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
owns its buffer. The buffers are null-terminated so that they may be used as
C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
current thread ID directly from the `TEB` when we do not yet have access to
kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
MFBT `Vector` in code that might not yet have access to Win32 heap functions.
Differential Revision: https://phabricator.services.mozilla.com/D43155
--HG--
extra : moz-landing-system : lando
* At this point our DLL blocking infra is complicated enough that I decided to
bite the bullet and move all of this code out of `mozglue/build` and into its
own subdirectory, `mozglue/dllservices`.
* We delete the original `UntrustedDllsHandler` code which is now obsolete.
* We implement mozglue's `LoaderObserver`:
** When this observer registers itself with the launcher process API, it
receives a vector containing all saved records of loaded DLLs that happened
until that moment.
** This code handles profiler labels and stackwalking suppression.
** Once a load has completed, we either pass the load on to XUL for further
processing, or save it for later if XUL is not initialized yet.
* mozglue has its own `ModuleLoadFrame` implementation for the legacy blocklist.
* `DllServicesBase` is updated to support the new interfaces.
* We implement `FallbackLoaderAPI` for `plugin-container`, `xpcshell`, and
any other non-`firefox` processes that do not have a launcher process
providing a loader API.
* We add some wide to UTF8 conversion functions.
Differential Revision: https://phabricator.services.mozilla.com/D43158
--HG--
rename : mozglue/build/Authenticode.cpp => mozglue/dllservices/Authenticode.cpp
rename : mozglue/build/Authenticode.h => mozglue/dllservices/Authenticode.h
rename : mozglue/build/WindowsDllBlocklist.cpp => mozglue/dllservices/WindowsDllBlocklist.cpp
rename : mozglue/build/WindowsDllBlocklist.h => mozglue/dllservices/WindowsDllBlocklist.h
rename : mozglue/build/WindowsDllBlocklistCommon.h => mozglue/dllservices/WindowsDllBlocklistCommon.h
rename : mozglue/build/WindowsDllBlocklistDefs.in => mozglue/dllservices/WindowsDllBlocklistDefs.in
rename : mozglue/build/WindowsDllServices.h => mozglue/dllservices/WindowsDllServices.h
rename : mozglue/build/gen_dll_blocklist_defs.py => mozglue/dllservices/gen_dll_blocklist_defs.py
rename : mozglue/build/moz.build => mozglue/dllservices/moz.build
rename : mozglue/build/MozglueUtils.h => mozglue/misc/WinUtils.h
extra : moz-landing-system : lando
This patch adds the following:
* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
owns its buffer. The buffers are null-terminated so that they may be used as
C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
current thread ID directly from the `TEB` when we do not yet have access to
kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
MFBT `Vector` in code that might not yet have access to Win32 heap functions.
Differential Revision: https://phabricator.services.mozilla.com/D43155
--HG--
extra : moz-landing-system : lando
This is showing up with hooks that are set both by the launcher process and by
the browser process when starting early DLL blocklist init on content processes:
* The browser's copy of mOrigFunc was set by the launcher process.
* The browser is setting a hook in the new child process, which writes to the
child's mOrigFunc.
* But FuncHookCrossProcess also writes that pointer to the browser's mOrigFunc,
thus corrupting the browser process's copy of the pointer.
For in-process hooks, we want to immediately write the stub pointer to its final
location; this is not an issue for cross-process hooks since the child process
is suspended when we do this and the parent process can't call the stub; there
is no possibility of a race.
Differential Revision: https://phabricator.services.mozilla.com/D46615
--HG--
extra : moz-landing-system : lando
* At this point our DLL blocking infra is complicated enough that I decided to
bite the bullet and move all of this code out of `mozglue/build` and into its
own subdirectory, `mozglue/dllservices`.
* We delete the original `UntrustedDllsHandler` code which is now obsolete.
* We implement mozglue's `LoaderObserver`:
** When this observer registers itself with the launcher process API, it
receives a vector containing all saved records of loaded DLLs that happened
until that moment.
** This code handles profiler labels and stackwalking suppression.
** Once a load has completed, we either pass the load on to XUL for further
processing, or save it for later if XUL is not initialized yet.
* mozglue has its own `ModuleLoadFrame` implementation for the legacy blocklist.
* `DllServicesBase` is updated to support the new interfaces.
* We implement `FallbackLoaderAPI` for `plugin-container`, `xpcshell`, and
any other non-`firefox` processes that do not have a launcher process
providing a loader API.
* We add some wide to UTF8 conversion functions.
Differential Revision: https://phabricator.services.mozilla.com/D43158
--HG--
rename : mozglue/build/Authenticode.cpp => mozglue/dllservices/Authenticode.cpp
rename : mozglue/build/Authenticode.h => mozglue/dllservices/Authenticode.h
rename : mozglue/build/WindowsDllBlocklist.cpp => mozglue/dllservices/WindowsDllBlocklist.cpp
rename : mozglue/build/WindowsDllBlocklist.h => mozglue/dllservices/WindowsDllBlocklist.h
rename : mozglue/build/WindowsDllBlocklistCommon.h => mozglue/dllservices/WindowsDllBlocklistCommon.h
rename : mozglue/build/WindowsDllBlocklistDefs.in => mozglue/dllservices/WindowsDllBlocklistDefs.in
rename : mozglue/build/WindowsDllServices.h => mozglue/dllservices/WindowsDllServices.h
rename : mozglue/build/gen_dll_blocklist_defs.py => mozglue/dllservices/gen_dll_blocklist_defs.py
rename : mozglue/build/moz.build => mozglue/dllservices/moz.build
rename : mozglue/build/MozglueUtils.h => mozglue/misc/WinUtils.h
extra : moz-landing-system : lando
This patch adds the following:
* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
owns its buffer. The buffers are null-terminated so that they may be used as
C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
current thread ID directly from the `TEB` when we do not yet have access to
kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
MFBT `Vector` in code that might not yet have access to Win32 heap functions.
Differential Revision: https://phabricator.services.mozilla.com/D43155
--HG--
extra : moz-landing-system : lando
In some situations, entries may in fact take more than half the buffer size
(e.g., when duplicating a stack into a small temporary buffer).
So we now allow blocks to take the full buffer size -- but not more, as they
would start overwriting themselves!
Differential Revision: https://phabricator.services.mozilla.com/D46453
--HG--
extra : moz-landing-system : lando
Instead of copying `BlocksRingBuffer` data byte-by-byte (using iterators byte
dereferencers), we can now use `ModuloBuffer::Iterator::ReadInto(Iterator&)` to
copy them using a small number of `memcpy`s.
Differential Revision: https://phabricator.services.mozilla.com/D45839
--HG--
extra : moz-landing-system : lando
Some objects are copied byte-by-byte to/from `ModuloBuffer`s.
E.g., serialized `BlocksRingBuffer`s, or duplicate stacks. (And more to come.)
`Iterator::ReadInto(Iterator&)` optimizes these copies by using the minimum
number of `memcpy`s possible.
Differential Revision: https://phabricator.services.mozilla.com/D45838
--HG--
extra : moz-landing-system : lando
profiler_can_accept_markers() is a fast and racy check that markers would
currently be stored, it should be used around potentially-expensive calls to
add markers.
And now markers are no longer stored when the profiler is paused. (Note that the
profiler is paused when a profile is being stored, this will help make this
operation faster.)
Differential Revision: https://phabricator.services.mozilla.com/D44434
--HG--
extra : moz-landing-system : lando
Since all profiler data is now stored inside ProfileBuffer, there is no real
need to continuously discard old data during sampling (this was particularly
useful to reclaim memory taken by old markers&payloads).
Instead, we can now just discard the old data once, just before starting to
stream it to JSON.
Differential Revision: https://phabricator.services.mozilla.com/D44433
--HG--
extra : moz-landing-system : lando
Now that what was in ProfilerMarker is stored directly in `BlocksRingBuffer`,
there is no need for this class anymore!
This also removes all the pointer management around it (when added to a TLS
list, moved during sampling, deleted when expired).
Differential Revision: https://phabricator.services.mozilla.com/D43431
--HG--
extra : moz-landing-system : lando
Since payloads are not kept alive long after they have been serialized, we can
just create them on the stack and pass a reference to their base (or pointer,
`nullptr` representing "no payloads") to `profiler_add_marker()`.
Differential Revision: https://phabricator.services.mozilla.com/D43430
--HG--
extra : moz-landing-system : lando
Markers and their payloads can now be serialized straight into the profiler's
`BlocksRingBuffer`, which is now thread-safe to allow such concurrent accesses
(even when gPSMutex is not locked).
This already saves us from having to allocate a `PayloadMarker` on the heap, and
from having to manage it in different lists.
The now-thread-safe `BlocksRingBuffer` in `CorePS` cannot be used inside the
critical section of `SamplerThread::Run`, because any mutex function could hang
because of the suspended thread (even though they functionally don't appear to
interact). So the sampler now uses a local `BlocksRingBuffer` without mutex.
As a bonus, the separate buffer helps reduce the number of concurrent locking
operations needed when capturing the stack.
Differential Revision: https://phabricator.services.mozilla.com/D43429
--HG--
extra : moz-landing-system : lando
Copy the full contents of BlocksRingBuffer into another one.
This is mainly useful to use a temporary buffer to store some data without
contentions, and then integrate the temporary buffer into the main one.
Differential Revision: https://phabricator.services.mozilla.com/D45306
--HG--
extra : moz-landing-system : lando
Payloads will serialize themselves into a `BlocksRingBuffer` entry when first
captured.
Later they will be deserialized, to stream JSON for the output profile.
Differential Revision: https://phabricator.services.mozilla.com/D43428
--HG--
extra : moz-landing-system : lando
The common data members stored in the ProfilerMarkerPayload base class can be
gathered into a struct, which will make it easier to pass around, especially
when a derived object is constructed with these common properties.
Differential Revision: https://phabricator.services.mozilla.com/D43427
--HG--
extra : moz-landing-system : lando
Markers may contain backtraces, so we need to be able to de/serialize them.
This involves de/serializing the underyling `BlocksRingBuffer`.
Differential Revision: https://phabricator.services.mozilla.com/D43426
--HG--
extra : moz-landing-system : lando
Add basic testing of `profiler_get_backtrace` before working on serializing it.
Differential Revision: https://phabricator.services.mozilla.com/D43424
--HG--
extra : moz-landing-system : lando
The main `BlocksRingBuffer`s will be stored in `CorePS` (outside of
`ProfileBuffer`s), as we need to be able to safely access the underlying buffers
when profilers are not enabled.
Also `ProfilerBacktrace` will own the `BlocksRingBuffer` that its captured
`ProfileBuffer` uses.
Taking this opportunity to rename the different `mBuffer`s to more useful names.
Differential Revision: https://phabricator.services.mozilla.com/D43422
--HG--
extra : moz-landing-system : lando
Now that we are using a byte-oriented `BlocksRingBuffer` instead of an array of
9-byte `ProfileBufferEntry`'s, internally the profiler sets the buffer size in
bytes. However all external users (popup, tests, etc.) still assume that the
requested capacity is in entries!
To limit the amount of changes, we will keep assuming externally-visible
capacities are in entries, and convert them to bytes.
Even though entries used to be 9 bytes each, and `BlocksRingBuffer` adds 1 byte
for the entry size, some entries actually need less space (e.g., 32-bit numbers
now take 6 bytes instead of 9), so converting to less than 9 bytes per entry is
acceptable.
We are settling on 8 bytes per entry: It's close to 9, and it's a power of two;
since the effective number of entries was a power of two, and `BlocksRingBuffer`
also uses a power of two size in bytes, this convertion keeps sizes in powers of
two.
Differential Revision: https://phabricator.services.mozilla.com/D44953
--HG--
extra : moz-landing-system : lando
This just replaces `ProfileBuffer`'s self-managed circular buffer with a
`BlocksRingBuffer`.
That `BlocksRingBuffer` does not need its own mutex (yet), because all uses go
through gPSMutex-guarded code.
`ProfileBuffer` now also pre-allocates a small buffer for use in
`DuplicateLastSample()`, this avoids multiple mallocs at each sleeping thread
stack duplication.
Note: Internal "magic" sizes have been multiplied by 8 (and tweaked upwards, to
handle bigger stacks), because they originally were the number of 9-byte
entries, but now it's the buffer size in bytes. (And entries can now be smaller
than 9 bytes, so overall the capacity in entries should be similar or better.)
However, external calls still think they are giving a number of "entries", this
will be handled in the next patch.
Differential Revision: https://phabricator.services.mozilla.com/D43421
--HG--
extra : moz-landing-system : lando
Add some stats (off by default) around streaming JSON, as the following patches
may affect it.
Differential Revision: https://phabricator.services.mozilla.com/D44952
--HG--
extra : moz-landing-system : lando
In some situations we will *need* to use a `BlocksRingBuffer` that absolutely
does not use a mutex -- In particular in the critical section of
`SamplerThread::Run()`, when a thread is suspended, because any action on any
mutex (even a private one that no-one else interacts with) can result in a hang.
As a bonus, `BlocksRingBuffer`s that are known not to be used in multi-thread
situations (e.g., backtraces, extracted stacks for duplication, etc.) will waste
less resources trying to lock/unlock their mutex.
Differential Revision: https://phabricator.services.mozilla.com/D45305
--HG--
extra : moz-landing-system : lando
`BaseProfilerMaybeMutex` wraps a `BaseProfilerMutex` inside a `Maybe`.
The decision to use a mutex or not is set at construction time.
If there is no mutex, all operations do nothing (at the small cost of checking
if the mutex is present.)
`BaseProfilerMaybeAutoLock` is the recommented RAII object to lock and
automatically unlock a `BaseProfilerMaybeMutex` until the end of a block scope.
Differential Revision: https://phabricator.services.mozilla.com/D45304
--HG--
extra : moz-landing-system : lando
Instead of copying `BlocksRingBuffer` data byte-by-byte (using iterators byte
dereferencers), we can now use `ModuloBuffer::Iterator::ReadInto(Iterator&)` to
copy them using a small number of `memcpy`s.
Differential Revision: https://phabricator.services.mozilla.com/D45839
--HG--
extra : moz-landing-system : lando
Some objects are copied byte-by-byte to/from `ModuloBuffer`s.
E.g., serialized `BlocksRingBuffer`s, or duplicate stacks. (And more to come.)
`Iterator::ReadInto(Iterator&)` optimizes these copies by using the minimum
number of `memcpy`s possible.
Differential Revision: https://phabricator.services.mozilla.com/D45838
--HG--
extra : moz-landing-system : lando
profiler_can_accept_markers() is a fast and racy check that markers would
currently be stored, it should be used around potentially-expensive calls to
add markers.
And now markers are no longer stored when the profiler is paused. (Note that the
profiler is paused when a profile is being stored, this will help make this
operation faster.)
Differential Revision: https://phabricator.services.mozilla.com/D44434
--HG--
extra : moz-landing-system : lando
Since all profiler data is now stored inside ProfileBuffer, there is no real
need to continuously discard old data during sampling (this was particularly
useful to reclaim memory taken by old markers&payloads).
Instead, we can now just discard the old data once, just before starting to
stream it to JSON.
Differential Revision: https://phabricator.services.mozilla.com/D44433
--HG--
extra : moz-landing-system : lando
Now that what was in ProfilerMarker is stored directly in `BlocksRingBuffer`,
there is no need for this class anymore!
This also removes all the pointer management around it (when added to a TLS
list, moved during sampling, deleted when expired).
Differential Revision: https://phabricator.services.mozilla.com/D43431
--HG--
extra : moz-landing-system : lando
Since payloads are not kept alive long after they have been serialized, we can
just create them on the stack and pass a reference to their base (or pointer,
`nullptr` representing "no payloads") to `profiler_add_marker()`.
Differential Revision: https://phabricator.services.mozilla.com/D43430
--HG--
extra : moz-landing-system : lando
Markers and their payloads can now be serialized straight into the profiler's
`BlocksRingBuffer`, which is now thread-safe to allow such concurrent accesses
(even when gPSMutex is not locked).
This already saves us from having to allocate a `PayloadMarker` on the heap, and
from having to manage it in different lists.
The now-thread-safe `BlocksRingBuffer` in `CorePS` cannot be used inside the
critical section of `SamplerThread::Run`, because any mutex function could hang
because of the suspended thread (even though they functionally don't appear to
interact). So the sampler now uses a local `BlocksRingBuffer` without mutex.
As a bonus, the separate buffer helps reduce the number of concurrent locking
operations needed when capturing the stack.
Differential Revision: https://phabricator.services.mozilla.com/D43429
--HG--
extra : moz-landing-system : lando
Copy the full contents of BlocksRingBuffer into another one.
This is mainly useful to use a temporary buffer to store some data without
contentions, and then integrate the temporary buffer into the main one.
Differential Revision: https://phabricator.services.mozilla.com/D45306
--HG--
extra : moz-landing-system : lando
Payloads will serialize themselves into a `BlocksRingBuffer` entry when first
captured.
Later they will be deserialized, to stream JSON for the output profile.
Differential Revision: https://phabricator.services.mozilla.com/D43428
--HG--
extra : moz-landing-system : lando
The common data members stored in the ProfilerMarkerPayload base class can be
gathered into a struct, which will make it easier to pass around, especially
when a derived object is constructed with these common properties.
Differential Revision: https://phabricator.services.mozilla.com/D43427
--HG--
extra : moz-landing-system : lando
Markers may contain backtraces, so we need to be able to de/serialize them.
This involves de/serializing the underyling `BlocksRingBuffer`.
Differential Revision: https://phabricator.services.mozilla.com/D43426
--HG--
extra : moz-landing-system : lando
