Some of the BaseProfile headers are always available, even when BaseProfiler
itself is not built, so we should test these in all cases.
Differential Revision: https://phabricator.services.mozilla.com/D63230
--HG--
extra : moz-landing-system : lando
Some of the BaseProfile headers are always available, even when BaseProfiler
itself is disabled, so we should test these in all cases.
Differential Revision: https://phabricator.services.mozilla.com/D63230
--HG--
extra : moz-landing-system : lando
GENERATED_FILES now defaults to python3 unless py2=True is specified as
an argument. All existing GENERATED_FILES scripts and GeneratedFile
templates have the py2=True attribute added, so this patch should
effectively be a no-op.
Going forward, individual scripts can be converted to python3 and their
corresponding py2=True attribute can be deleted. In effect, this patch
will be backed out in pieces until all scripts run in python3, at which
point the py2 attribute itself can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D60919
--HG--
extra : moz-landing-system : lando
GENERATED_FILES now defaults to python3 unless py2=True is specified as
an argument. All existing GENERATED_FILES scripts and GeneratedFile
templates have the py2=True attribute added, so this patch should
effectively be a no-op.
Going forward, individual scripts can be converted to python3 and their
corresponding py2=True attribute can be deleted. In effect, this patch
will be backed out in pieces until all scripts run in python3, at which
point the py2 attribute itself can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D60919
--HG--
extra : moz-landing-system : lando
Apps targeting SDK 29 are not allowed to open /dev/ashmem directly, and
instead must use NDK functions. Those functions are only available in
SDK 26 and higher, so we need this shim to use the functions if they
are available, else fallback to opening /dev/ashmem directly.
Differential Revision: https://phabricator.services.mozilla.com/D61012
--HG--
extra : moz-landing-system : lando
GENERATED_FILES now defaults to python3 unless py2=True is specified as
an argument. All existing GENERATED_FILES scripts and GeneratedFile
templates have the py2=True attribute added, so this patch should
effectively be a no-op.
Going forward, individual scripts can be converted to python3 and their
corresponding py2=True attribute can be deleted. In effect, this patch
will be backed out in pieces until all scripts run in python3, at which
point the py2 attribute itself can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D60919
--HG--
extra : moz-landing-system : lando
This patch adds a function to get an exported function in a remote process.
We need this implementation to address Bug 1604008, Bug 1608645, and Bug 1610790.
When `WindowsDllInterceptor` detours a function in a remote process, we used the
native `GetProcAddress` locally, and then detours the returned address in the
target process. The problem is if the caller's export table was modified, the
address returned from `GetProcAddress` might be invalid in the target process,
which is Bug 1604008.
I implemented `GetProcAddress` depending on both local and remote process image,
but it caused two regressions Bug 1608645 and Bug 1610790 because multiple
applications modify firefox's export table in multiple ways, such as replacing
an entry of EAT, replacing an RVA to Export section, or etc.
With this patch, we can use `PEExportSection<MMPolicy>::GetProcAddress` to get
an exported function in a remote process without relying on any local data so
that it's not impacted by modification of the local export table.
Differential Revision: https://phabricator.services.mozilla.com//D62315
Depends on D62314
This patch changes the entrypoint of test programs under mozglue/tests so that
a coming test program can handle a command string easily.
Differential Revision: https://phabricator.services.mozilla.com//D62314
This patch adds a function to get an exported function in a remote process.
We need this implementation to address Bug 1604008, Bug 1608645, and Bug 1610790.
When `WindowsDllInterceptor` detours a function in a remote process, we used the
native `GetProcAddress` locally, and then detours the returned address in the
target process. The problem is if the caller's export table was modified, the
address returned from `GetProcAddress` might be invalid in the target process,
which is Bug 1604008.
I implemented `GetProcAddress` depending on both local and remote process image,
but it caused two regressions Bug 1608645 and Bug 1610790 because multiple
applications modify firefox's export table in multiple ways, such as replacing
an entry of EAT, replacing an RVA to Export section, or etc.
With this patch, we can use `PEExportSection<MMPolicy>::GetProcAddress` to get
an exported function in a remote process without relying on any local data so
that it's not impacted by modification of the local export table.
Differential Revision: https://phabricator.services.mozilla.com/D62315
Depends on D62314
--HG--
extra : rebase_source : 3088f5997a2097ef22ce8567783375e5f7866ab2
This patch changes the entrypoint of test programs under mozglue/tests so that
a coming test program can handle a command string easily.
Differential Revision: https://phabricator.services.mozilla.com/D62314
--HG--
extra : rebase_source : a180de844700bbee60a6491a35da33da84aa12ed
We had a thread-local varialbe `ModuleLoadFrame::sTopFrame` to track the topmost
stack frame of `LdrLoadDll`. However, our hook function `patched_LdrLoadDll` can
be called even before TLS is initialized. In such a case, accessing `sTopFrame`
causes AV.
This patch introduces `SafeThreadLocal` to safely access a thread-local varialbe.
If TLS is not initialized, it falls back to a global variable because in that
early stage there is only a single thread running.
Differential Revision: https://phabricator.services.mozilla.com/D55870
--HG--
extra : moz-landing-system : lando
We had a thread-local varialbe `ModuleLoadFrame::sTopFrame` to track the topmost
stack frame of `LdrLoadDll`. However, our hook function `patched_LdrLoadDll` can
be called even before TLS is initialized. In such a case, accessing `sTopFrame`
causes AV.
This patch introduces `SafeThreadLocal` to safely access a thread-local varialbe.
If TLS is not initialized, it falls back to a global variable because in that
early stage there is only a single thread running.
Differential Revision: https://phabricator.services.mozilla.com/D55870
--HG--
extra : moz-landing-system : lando
Changes:
Whitelist the `RegisteredThread::RegisteredThread` failures for linux1804-64-asan/opt temporarily while the parent bug is left open, to support `mochitest-browser-chrome` migration to linux1804-64.
Differential Revision: https://phabricator.services.mozilla.com/D61224
--HG--
extra : moz-landing-system : lando
The "(null)" printing behavior is a glibc extension and the specified behavior
in the standard is undefined
Differential Revision: https://phabricator.services.mozilla.com/D61071
--HG--
extra : moz-landing-system : lando
Under the stronger Control Flow Guard scheme coming in clang 10, when a nop-space hook jumps back to the original API, at `ntdll!Whatever+2`, that address is not a registered jump target, so we crash with a CFG failure. Since this is a deliberate violation of the rules, let's disable CFG for these calls.
Based on my testing, this is the only place we need to use this attribute, so I placed its definition close to the use. (Had we needed more of these, I would have put it in mfbt/.)
Differential Revision: https://phabricator.services.mozilla.com/D59728
--HG--
extra : moz-landing-system : lando
A third-party application can modify the export directory, the export address/name/ordinal
tables, or an entry in those tables. If that happens, we will see an RVA is located outside
the mapped image and `RVAToPtr` returns null. This patch makes sure we don't hit null AV
when modification is detected.
`FindExportAddressTableEntry` should not return a pointer to the modified table entry because
we dereference it in another process to cross-process detour.
Differential Revision: https://phabricator.services.mozilla.com/D59738
--HG--
extra : moz-landing-system : lando
This was done by:
This was done by applying:
```
diff --git a/python/mozbuild/mozbuild/code-analysis/mach_commands.py b/python/mozbuild/mozbuild/code-analysis/mach_commands.py
index 789affde7bbf..fe33c4c7d4d1 100644
--- a/python/mozbuild/mozbuild/code-analysis/mach_commands.py
+++ b/python/mozbuild/mozbuild/code-analysis/mach_commands.py
@@ -2007,7 +2007,7 @@ class StaticAnalysis(MachCommandBase):
from subprocess import Popen, PIPE, check_output, CalledProcessError
diff_process = Popen(self._get_clang_format_diff_command(commit), stdout=PIPE)
- args = [sys.executable, clang_format_diff, "-p1", "-binary=%s" % clang_format]
+ args = [sys.executable, clang_format_diff, "-p1", "-binary=%s" % clang_format, '-sort-includes']
if not output_file:
args.append("-i")
```
Then running `./mach clang-format -c <commit-hash>`
Then undoing that patch.
Then running check_spidermonkey_style.py --fixup
Then running `./mach clang-format`
I had to fix four things:
* I needed to move <utility> back down in GuardObjects.h because I was hitting
obscure problems with our system include wrappers like this:
0:03.94 /usr/include/stdlib.h:550:14: error: exception specification in declaration does not match previous declaration
0:03.94 extern void *realloc (void *__ptr, size_t __size)
0:03.94 ^
0:03.94 /home/emilio/src/moz/gecko-2/obj-debug/dist/include/malloc_decls.h:53:1: note: previous declaration is here
0:03.94 MALLOC_DECL(realloc, void*, void*, size_t)
0:03.94 ^
0:03.94 /home/emilio/src/moz/gecko-2/obj-debug/dist/include/mozilla/mozalloc.h:22:32: note: expanded from macro 'MALLOC_DECL'
0:03.94 MOZ_MEMORY_API return_type name##_impl(__VA_ARGS__);
0:03.94 ^
0:03.94 <scratch space>:178:1: note: expanded from here
0:03.94 realloc_impl
0:03.94 ^
0:03.94 /home/emilio/src/moz/gecko-2/obj-debug/dist/include/mozmemory_wrap.h:142:41: note: expanded from macro 'realloc_impl'
0:03.94 #define realloc_impl mozmem_malloc_impl(realloc)
Which I really didn't feel like digging into.
* I had to restore the order of TrustOverrideUtils.h and related files in nss
because the .inc files depend on TrustOverrideUtils.h being included earlier.
* I had to add a missing include to RollingNumber.h
* Also had to partially restore include order in JsepSessionImpl.cpp to avoid
some -WError issues due to some static inline functions being defined in a
header but not used in the rest of the compilation unit.
Differential Revision: https://phabricator.services.mozilla.com/D60327
--HG--
extra : moz-landing-system : lando
This makes Gecko build again. I'll squash this into the previous commit anyhow.
Differential Revision: https://phabricator.services.mozilla.com/D60324
--HG--
extra : moz-landing-system : lando
rg -l 'mozilla/Move.h' | xargs sed -i 's/#include "mozilla\/Move.h"/#include <utility>/g'
Further manual fixups and cleanups to the include order incoming.
Differential Revision: https://phabricator.services.mozilla.com/D60323
--HG--
extra : moz-landing-system : lando
Now both profilers are controlled by the same environment variables, e.g.
"MOZ_PROFILER_STARTUP" to run both profilers at startup.
The only remaining Base Profiler-specific commands are:
- MOZ_BASE_PROFILER_HELP, because the Base Profiler doesn't have the same
capabilities as Gecko, so it may display different available features.
- MOZ_BASE_PROFILER_LOGGING, because the Base Profiler doesn't have access to
MOZ_LOG (both the env-var and the C++ macros).
Differential Revision: https://phabricator.services.mozilla.com/D54449
--HG--
extra : moz-landing-system : lando
This env-vars were not processed because of some needed special processing, and
they were not critical.
But now we want them to be correctly read, to match the Gecko Profiler features.
Differential Revision: https://phabricator.services.mozilla.com/D54448
--HG--
extra : moz-landing-system : lando
When `WindowsDllInterceptor` detours a function in a remote process, it calculates
a target address via `GetProcAddress` in the caller's process first, and detours
that address in the target process. If the caller's export table was modified, the
target address might be invalid in the target process.
With this patch, `WindowsDllInterceptor` uses the target process's export table to
calculate a target function address.
Differential Revision: https://phabricator.services.mozilla.com/D58305
--HG--
extra : moz-landing-system : lando
`GetProcessTimes` is based on QPC, while `GetSystemTime` is based on clock
interruption whose accuracy is lower than QPC. This means in a process's early
stage, `GetSystemTime` may return a timestamp earlier than creation timestamp.
If this happens we'll keep a negative process uptime which causes overflow in
telemetry processing.
Win8+ has a handy API `GetSystemTimePreciseAsFileTime` that solves everything.
On Win7, `GetSystemTimeAsFileTime` still solves this issue. In the worst case,
it returns the exact same timestamp as process creation, but it's ok.
Because the system time is stored as a `FILETIME` in `KUSER`, converting it to
a `SYSTEMTIME` with `GetSystemTime` drops accuracy. We should avoid it unless
needed.
This patch also moves the call to `GetProcessTimes` before getting the current
timestamp in case clock interruption happens in between those two function calls.
Differential Revision: https://phabricator.services.mozilla.com/D56273
--HG--
extra : moz-landing-system : lando
In bug 1587332, ActivePS::mProfileBuffer was changed from
`const UniquePtr<ProfileBuffer>` to just `ProfileBuffer`, so its own object size
is now counted as part of `sizeof(ActivePS)`, and we should only count its
dependencies with `SizeOfExcludingThis`.
Differential Revision: https://phabricator.services.mozilla.com/D58288
--HG--
extra : moz-landing-system : lando
Our detour allocates a trampoline with `PAGE_EXECUTE_READ` first, and then makes
it writable before use. If the dynamic code policy is enabled after allocation,
we fail to change the attribute, and crash the process because we try to write
data into a readonly page. We need to check the validity of a trampoline before
writing data.
Differential Revision: https://phabricator.services.mozilla.com/D56983
--HG--
extra : moz-landing-system : lando
This is a pretty straightforward patch; we add `WindowsIATPatcher` to
implement the actual IAT patching, and use a partial specialization of
`FuncHook` to account for the underlying differences in implementation vs our
existing interceptor code.
Differential Revision: https://phabricator.services.mozilla.com/D57831
--HG--
extra : moz-landing-system : lando
Strictly speaking, the import descriptor includes more than just the IAT,
so renaming this method to reflect that.
Differential Revision: https://phabricator.services.mozilla.com/D57830
--HG--
extra : moz-landing-system : lando
This is a pretty straightforward patch; we add `WindowsIATPatcher` to
implement the actual IAT patching, and use a partial specialization of
`FuncHook` to account for the underlying differences in implementation vs our
existing interceptor code.
Differential Revision: https://phabricator.services.mozilla.com/D57831
--HG--
extra : moz-landing-system : lando
Strictly speaking, the import descriptor includes more than just the IAT,
so renaming this method to reflect that.
Differential Revision: https://phabricator.services.mozilla.com/D57830
--HG--
extra : moz-landing-system : lando
This changeset is a simple find and replace of `MOZ_FALLTHROUGH` and `[[fallthrough]]`.
Unfortunately, the MOZ_FALLTHROUGH_ASSERT macro (to assert on case fallthrough in debug builds) is still necessary after switching from [[clang::fallthrough]] to [[fallthrough]] because:
* MOZ_ASSERT(false) followed by [[fallthrough]] triggers a -Wunreachable-code warning in DEBUG builds
* but MOZ_ASSERT(false) without [[fallthrough]] triggers a -Wimplicit-fallthrough warning in NDEBUG builds.
Differential Revision: https://phabricator.services.mozilla.com/D56440
--HG--
extra : moz-landing-system : lando
We rename `LoaderObserver::Clear` to `LoaderObserver::Disable` to more accurately
reflect the following behaviour change:
Not only does the `Disable` call free any enqueued module load events, it also
ensures that no further module loads will be saved in the future. This reflects
the reality that any `mozglue` client that calls `Disable` has no intention of
ever processing these events.
Differential Revision: https://phabricator.services.mozilla.com/D57897
--HG--
extra : moz-landing-system : lando
The only way to select all registered threads was to have an empty entry in the
filter list, usually by just having "," (a single comma).
The popup removes spurious empty items, making this impossible for most users.
It is now possible to just have "*" (as the whole filter string, or at least one
item in the list) to select all registered threads.
Differential Revision: https://phabricator.services.mozilla.com/D57594
--HG--
extra : moz-landing-system : lando
Changing the OS we run Linux ASan on from ubuntu1604 to ubuntu1804
causes 820800 bytes of leaks with libfreetype.so in the stack in
almost every test suite. Presumably this is just a variation on
existing leaks we get because we don't shut down fonts properly, so
just whitelist it.
Differential Revision: https://phabricator.services.mozilla.com/D56511
--HG--
extra : moz-landing-system : lando
This patch contains the core changes to make this all work across e10s:
* We clarify the naming of path variables to be more specific as to whether they are NT paths or DOS paths;
* We add IPC `ParamTraits` that are necessary for `UntrustedModulesData` types;
* We implement `ProcessModuleLoadQueue` for child processes. Because of sandboxing, we need to split this sequence into multiple async operations:
** Initial queue processing;
** Sending the list of modules to the parent process to determine trustworthiness (via `GetModulesTrust`);
** Receiving the results from the parent process and producing a final result (via `CompleteProcessing`).
* We implement the `GetModulesTrust` function for the parent process, which evaluates the trust of child process modules;
* We change all hash tables to be keyed using NT paths. Because resolving DOS paths may not be permitted in sandboxed processes,
we need to standardize on NT paths as the "universal path" across processes.
* We add `WinDllServices::StartUntrustedModulesProcessor` to separate untrusted modules startup from `WinDllServices` construction:
** While we now start `WinDllServices` across all child process types, only specific process types will support untrusted modules.
** Furthermore, untrusted modules must be started at a very specific point that is dependent on the type of child process.
** We add those calls to `StartUntrustedModulesProcessor` in subsequent patches.
Differential Revision: https://phabricator.services.mozilla.com/D53680
--HG--
extra : moz-landing-system : lando
We need a way for the sandbox broker to be able to initialize the launcher
DLL blocklist when starting a new content process.
This patch adds the ability to resolve the initialization function through
DLL services.
Differential Revision: https://phabricator.services.mozilla.com/D53678
--HG--
extra : moz-landing-system : lando
Now that the launcher blocklist will support child processes, we need to add
them to the launcher blocklist. The revised criteria the `Launcher` blocklist
matches the criteria already in use by the `Legacy` blocklist.
Differential Revision: https://phabricator.services.mozilla.com/D53675
--HG--
extra : moz-landing-system : lando
* We change `InitializeDllBlocklistOOP` to be able to set the correct flags
when initializing a sandbox child process.
* We change the freestanding DLL blocklist code to be sensitive to the
`CHILD_PROCESSES_ONLY` flag;
* We move the declaration of `gBlocklistInitFlags` to `WindowsDllBlocklist.h`
so that it is visible to more code.
Differential Revision: https://phabricator.services.mozilla.com/D53674
--HG--
extra : moz-landing-system : lando
This patch contains the core changes to make this all work across e10s:
* We clarify the naming of path variables to be more specific as to whether they are NT paths or DOS paths;
* We add IPC `ParamTraits` that are necessary for `UntrustedModulesData` types;
* We implement `ProcessModuleLoadQueue` for child processes. Because of sandboxing, we need to split this sequence into multiple async operations:
** Initial queue processing;
** Sending the list of modules to the parent process to determine trustworthiness (via `GetModulesTrust`);
** Receiving the results from the parent process and producing a final result (via `CompleteProcessing`).
* We implement the `GetModulesTrust` function for the parent process, which evaluates the trust of child process modules;
* We change all hash tables to be keyed using NT paths. Because resolving DOS paths may not be permitted in sandboxed processes,
we need to standardize on NT paths as the "universal path" across processes.
* We add `WinDllServices::StartUntrustedModulesProcessor` to separate untrusted modules startup from `WinDllServices` construction:
** While we now start `WinDllServices` across all child process types, only specific process types will support untrusted modules.
** Furthermore, untrusted modules must be started at a very specific point that is dependent on the type of child process.
** We add those calls to `StartUntrustedModulesProcessor` in subsequent patches.
Differential Revision: https://phabricator.services.mozilla.com/D53680
--HG--
extra : moz-landing-system : lando
We need a way for the sandbox broker to be able to initialize the launcher
DLL blocklist when starting a new content process.
This patch adds the ability to resolve the initialization function through
DLL services.
Differential Revision: https://phabricator.services.mozilla.com/D53678
--HG--
extra : moz-landing-system : lando
Now that the launcher blocklist will support child processes, we need to add
them to the launcher blocklist. The revised criteria the `Launcher` blocklist
matches the criteria already in use by the `Legacy` blocklist.
Differential Revision: https://phabricator.services.mozilla.com/D53675
--HG--
extra : moz-landing-system : lando
* We change `InitializeDllBlocklistOOP` to be able to set the correct flags
when initializing a sandbox child process.
* We change the freestanding DLL blocklist code to be sensitive to the
`CHILD_PROCESSES_ONLY` flag;
* We move the declaration of `gBlocklistInitFlags` to `WindowsDllBlocklist.h`
so that it is visible to more code.
Differential Revision: https://phabricator.services.mozilla.com/D53674
--HG--
extra : moz-landing-system : lando
mov byte ptr support was added in bug 1382251 but did not properly count the instruction size. It was missing the 1-byte operand, which causes the rest of the trampoline to be garbage.
Differential Revision: https://phabricator.services.mozilla.com/D55744
--HG--
extra : moz-landing-system : lando
SetWindowLong*/SetWindowLongPtr* was being intercepted so that we could override windowprocs in windowed plugins on Windows. We no longer support windowed plugins so these functions are never intercepted.
Differential Revision: https://phabricator.services.mozilla.com/D55536
--HG--
extra : moz-landing-system : lando
CloseHandle has a jump followed by enough `nop`s to fit a 10-byte patch but not enough to fit the default 13-byte patch when running Windows 8 or 8.1. This patch tells the interceptor to use a 10-byte patch on those OSs.
Differential Revision: https://phabricator.services.mozilla.com/D55535
--HG--
extra : moz-landing-system : lando
This patch rolls up all the required changes for this purpose. Since the
whole crash reporting flow must understand the new format it's not possible
to land this as separate patches as individually they would be broken. This
patch includes the following changes:
* Changes to the crash reporting machinery to write out annotations as JSON,
these includes changes to the DLL blocklist code that must be run at crash
time.
* Modifications to the crash reporter client so that it can read and
submit the new format; this includes platform-specific changes to the
Breakpad libraries it uses for submitting crashes.
* Modifications to the minidump-analyzer to understand and process the new
format correctly.
* Modifications to the crash manager to understand and process the new format
correctly.
* Modifications to GeckoView's crash handler to understand and submit the
new format correctly.
* Added new tests to cover the new format and modified existing ones to
accomodate the new one.
Differential Revision: https://phabricator.services.mozilla.com/D46848
--HG--
extra : moz-landing-system : lando
This is especially useful when debugging both Base Profiler and Gecko Profiler,
so that their messages are correctly sequenced.
Differential Revision: https://phabricator.services.mozilla.com/D54447
--HG--
extra : moz-landing-system : lando
The Base Profiler originally named the main thread "Main Thread", which is
friendlier than "GeckoMain". However this makes it more difficult to combine the
controls for both profilers if they use different names.
So now both profilers use "GeckoMain", so filters can be exactly the same.
Base Profiler adds "(pre-xul)" to the name to distinguish tracks in the
frontend -- This distinction is actually necessary so the frontend doesn't get
confused by threads with the exact same name, but eventually tracks will get
combined in the frontend as well.
Differential Revision: https://phabricator.services.mozilla.com/D54444
--HG--
extra : moz-landing-system : lando
This is especially useful when debugging both Base Profiler and Gecko Profiler,
so that their messages are correctly sequenced.
Differential Revision: https://phabricator.services.mozilla.com/D54447
--HG--
extra : moz-landing-system : lando
The Base Profiler originally named the main thread "Main Thread", which is
friendlier than "GeckoMain". However this makes it more difficult to combine the
controls for both profilers if they use different names.
So now both profilers use "GeckoMain", so filters can be exactly the same.
Base Profiler adds "(pre-xul)" to the name to distinguish tracks in the
frontend -- This distinction is actually necessary so the frontend doesn't get
confused by threads with the exact same name, but eventually tracks will get
combined in the frontend as well.
Differential Revision: https://phabricator.services.mozilla.com/D54444
--HG--
extra : moz-landing-system : lando
Our detour cannot handle assembly patterns which is injected by the code coverage
instrumentation. We need to skip them in CCov build.
Differential Revision: https://phabricator.services.mozilla.com/D54745
--HG--
extra : moz-landing-system : lando
For now, there is no flag to actually allow it, but this is the
code-side changes to allow the linker being disabled.
Differential Revision: https://phabricator.services.mozilla.com/D54074
--HG--
extra : moz-landing-system : lando
The custom linker used on Android (FaultyLib), can be enabled on
non-Android, and in the near future, may not always be enabled on
Android, so the FaultyLib-specific parts of the profile need to not be
specific to Android, but to the linker being enabled instead.
Differential Revision: https://phabricator.services.mozilla.com/D54073
--HG--
extra : moz-landing-system : lando
This function works on all GeckoProcessTypes, not just those for child
processes.
Differential Revision: https://phabricator.services.mozilla.com/D54375
--HG--
extra : moz-landing-system : lando
This patch adds the following pattern to our x64 detour so that we can hook APIs
even though a target is already detoured by another application.
```
mov rax, imm64
push rax
ret
```
We already have `PatchIfTargetIsRecognizedTrampoline` to detour the pattern
`mov; jmp`. There is another variation using `push rax;ret` to jump.
Differential Revision: https://phabricator.services.mozilla.com/D53877
--HG--
extra : moz-landing-system : lando
Passing a dummy object to `NtQueryFullAttributesFile` to avoid AV on WOW64.
Differential Revision: https://phabricator.services.mozilla.com/D53876
--HG--
extra : moz-landing-system : lando
We want the profiler UI to be able to know if the data can be used for
reconstructing the event delays, since it measures something different
from the old 16ms event injection.
Differential Revision: https://phabricator.services.mozilla.com/D52534
--HG--
extra : moz-landing-system : lando
This patch rolls up all the required changes for this purpose. Since the
whole crash reporting flow must understand the new format it's not possible
to land this as separate patches as individually they would be broken. This
patch includes the following changes:
* Changes to the crash reporting machinery to write out annotations as JSON,
these includes changes to the DLL blocklist code that must be run at crash
time.
* Modifications to the crash reporter client so that it can read and
submit the new format; this includes platform-specific changes to the
Breakpad libraries it uses for submitting crashes.
* Modifications to the minidump-analyzer to understand and process the new
format correctly.
* Modifications to the crash manager to understand and process the new format
correctly.
* Modifications to GeckoView's crash handler to understand and submit the
new format correctly.
* Added new tests to cover the new format and modified existing ones to
accomodate the new one.
Differential Revision: https://phabricator.services.mozilla.com/D46848
--HG--
extra : moz-landing-system : lando
`patched_NtMapViewOfSection` uses the process default heap to copy a string.
However, `patched_NtMapViewOfSection` can be invoked even before the process
heap is initialized. One example we found is Windows Defender's EAF, with
which "verifier.dll" is loaded before the process heap is initialized.
This patch adds a check whether the heap is initialized or not in
`patched_NtMapViewOfSection` and `NativeNtBlockSet::Add`. This also minimizes
the usage of the heap, i.e. not copying a string when we block a dll.
Differential Revision: https://phabricator.services.mozilla.com/D51028
--HG--
extra : moz-landing-system : lando
This moves the parts of toolkit/library/rust/shared/lib.rs related to
panic hooking to a new mozglue subdirectory, which will be used for
things that can be statically linked to e.g. libxul, rather than in
a "shared library".
The panic hook is disabled when building spidermonkey via the mozjs_sys
crate.
Differential Revision: https://phabricator.services.mozilla.com/D52793
--HG--
extra : moz-landing-system : lando
This moves the parts of toolkit/library/rust/shared/lib.rs related to
panic hooking to a new mozglue subdirectory, which will be used for
things that can be statically linked to e.g. libxul, rather than in
a "shared library".
The panic hook is disabled when building spidermonkey via the mozjs_sys
crate.
Differential Revision: https://phabricator.services.mozilla.com/D52793
--HG--
extra : moz-landing-system : lando
This moves the parts of toolkit/library/rust/shared/lib.rs related to
panic hooking to a new mozglue subdirectory, which will be used for
things that can be statically linked to e.g. libxul, rather than in
a "shared library".
The panic hook is disabled when building spidermonkey via the mozjs_sys
crate.
Differential Revision: https://phabricator.services.mozilla.com/D52793
--HG--
extra : moz-landing-system : lando
We want the profiler UI to be able to know if the data can be used for
reconstructing the event delays, since it measures something different
from the old 16ms event injection.
Differential Revision: https://phabricator.services.mozilla.com/D52534
--HG--
extra : moz-landing-system : lando
Avoids deadlocks on Windows due to Now() taking a lock; if done while we've
paused a thread that holds the lock we will deadlock.
Differential Revision: https://phabricator.services.mozilla.com/D52392
--HG--
extra : moz-landing-system : lando
Some applications tamper Import Directory entry of a loaded executable image
to pretend static dependency on their module. We have `RestoreImportDirectory`
to revert it in the browser process. If tampering happened in the launcher
process, however, we failed to get an IAT thunk address via `GetIATThunksForModule`
because it could be located outside the mapped image.
With this patch, we skip bounds check in `GetIATThunksForModule` if we detect
tampering in the launcher process. We can proceed safely because it's expected
that Import Directory is still valid though it's located outside.
Differential Revision: https://phabricator.services.mozilla.com/D49690
--HG--
extra : moz-landing-system : lando
profile.counters[n].sample_groups was mistakenly streamed as an object, which
prevents having more than one, and goes against the published format
documentation.
The front-end was implemented to process the incorrect format, so it will need
to be updated as well; hence the version change to 18.
Differential Revision: https://phabricator.services.mozilla.com/D49867
--HG--
extra : moz-landing-system : lando
In Android gtest, instead of pushing the gtest libxul.so to the
(assumed and potentially incorrect) location of the test application's
libraries, push libxul.so to /data/local/gtest, and direct the linker
to use that instead.
Differential Revision: https://phabricator.services.mozilla.com/D34942
--HG--
extra : moz-landing-system : lando
Having `mProfileBuffer` be a pointer is not really helpful:
- The pointer is never null (It's allocated on ActivePS construction, and
implicitly deleted on ActivePS destruction); it's never moved-from.
- It requires an extra `new` and an extra `delete`.
Differential Revision: https://phabricator.services.mozilla.com/D48650
--HG--
extra : moz-landing-system : lando
Add assertions that all `sInstance` pointers (from both `CorePS` and `ActivePS`)
are not null before being dereferenced.
This is probably more than needed, but it's only `MOZ_ASSERT`s limited to
Nightly, and it should give better feedback in case something goes wrong.
Eventually, I think it would be better to make most methods non-static, and have
a checked reference-to-instance getter.
Differential Revision: https://phabricator.services.mozilla.com/D48649
--HG--
extra : moz-landing-system : lando
We are not simply excluding all about:blanks because there might be some
about:blank that user really visits. But for others we don't want to include
the first about:blank because when a BrowsingContext is loaded, and if the
principal matches, the first document loaded in it will share the inner window.
Differential Revision: https://phabricator.services.mozilla.com/D47067
--HG--
extra : moz-landing-system : lando
We were keeping nsDocShell::mHistoryId and nsDocShell::mOSHE as keys. They
weren't quite good because:
1. While loading an iframe, they were being registered twice with the same
ids(for about:blank and the real URL) sometimes.
2. It wasn't possible to access to the parent mHistoryId and mOSHE from a child
processes if the parent is in a different process. That may not be the case for
now, but it will be after fission.
So we had to find other IDs to:
1. Determine the Tab of the frames.
2. Determine the URLs of the frames.
For the first use case, we were using nsDocShell::mHistoryId for that purpose
but that was wrong. The closest thing that we can get to a tab ID is
BrowsingContext ID because they don't change after a navigation. But iframes
have different BrowsingContext's, so we still need to create a tree to
construct a tab content. That can be either in the front-end or capture time.
For the second use case, we were using a key pair of mHistoryId and mOSHE. We
now chose to keep inner window IDs for that purpose. Inner window IDs are
unique for each navigation loads because inner window correspond to each JS
window global objects. That's why we can use that without any problem. But one
problem is that we cannot handle `history.pushState` and `history.replaceState`
changes with that change since window global objects won't change during those.
But that was the best thing we can do after fission. So this will be a small
sacrifice for us to keep that functionality working after fission.
In that patch we also remove the registration/unregistration calls. We are
going to add those calls in the next patch.
Differential Revision: https://phabricator.services.mozilla.com/D47065
--HG--
extra : moz-landing-system : lando
Previously, the absence of "stackwalk", "leaf", and "javascript" implied that
the test/user didn't want any sampling, but this caused issues in some tests
that enabled "stackwalk" on platforms that didn't support stack-walking, which
ended up suppressing label-only stacks that the test expected.
we now have an explicit feature "nostacksampling" that disables backtraces from
the samplers in both profilers. This effectively cancels "stackwalk", "leaf",
and "javascript" if present.
Differential Revision: https://phabricator.services.mozilla.com/D47731
--HG--
extra : moz-landing-system : lando
Previously, the absence of "stackwalk", "leaf", and "javascript" implied that
the test/user didn't want any sampling, but this caused issues in some tests
that enabled "stackwalk" on platforms that didn't support stack-walking, which
ended up suppressing label-only stacks that the test expected.
we now have an explicit feature "nostacksampling" that disables backtraces from
the samplers in both profilers. This effectively cancels "stackwalk", "leaf",
and "javascript" if present.
Differential Revision: https://phabricator.services.mozilla.com/D47731
--HG--
extra : moz-landing-system : lando
I also fixed a couple of typos in comments in the python script that didn't
seem to warrant their own bug.
Differential Revision: https://phabricator.services.mozilla.com/D47455
--HG--
extra : moz-landing-system : lando
Defines an android-only version of nsTraceRefcnt::WalkTheStack that takes a function callback, which outputs the stack frame buffer to `__android_log_print`. Also uses `__wrap_dladdr` in MozDescribeCodeAddress, which outputs slightly more informative data for the stack trace (instead of instances of '???/??? [???]').
Differential Revision: https://phabricator.services.mozilla.com/D46868
--HG--
extra : moz-landing-system : lando
Defines an android-only version of nsTraceRefcnt::WalkTheStack that takes a function callback, which outputs the stack frame buffer to `__android_log_print`. Also uses `__wrap_dladdr` in MozDescribeCodeAddress, which outputs slightly more informative data for the stack trace (instead of instances of '???/??? [???]').
Differential Revision: https://phabricator.services.mozilla.com/D46868
--HG--
extra : moz-landing-system : lando
This assertion could fire if the profiler state changes between beginning and
ending a DLL load. That's less than ideal, so let's just get rid of it.
Differential Revision: https://phabricator.services.mozilla.com/D46897
--HG--
extra : moz-landing-system : lando
`BlocksRingBuffer` had an "entry destructor" to make it a more generic
container, and it was useful during early prototyping of the new profiler
storage (so that we could store owning pointers).
But this entry destructor is stored in an `std::function`, which gets marked as
a potential GC caller by the js rooting hazard analyzer; and as this bug found,
it's not obvious where to place `JS::AutoSuppressGCAnalysis`, because profiler
entries (including stacks) could be added on one thread while GC happens
elsewhere, which triggers the embedded `AutoAssertNoGC` check.
Since we don't actually use the entry destructor facility in the profiler, it's
easier to just get rid of it. As a bonus, it's a small optimization.
Tests that were using an entry destructor now use the `State` instead, to verify
that entries are pushed and cleared as expected.
If needed in the future outside of the profiler, `BlocksRingBuffer` could again
include an entry destructor, but it would have to be through templating, so that
the class used in the profiler wouldn't contain an `std::function`.
Differential Revision: https://phabricator.services.mozilla.com/D46738
--HG--
extra : moz-landing-system : lando
* At this point our DLL blocking infra is complicated enough that I decided to
bite the bullet and move all of this code out of `mozglue/build` and into its
own subdirectory, `mozglue/dllservices`.
* We delete the original `UntrustedDllsHandler` code which is now obsolete.
* We implement mozglue's `LoaderObserver`:
** When this observer registers itself with the launcher process API, it
receives a vector containing all saved records of loaded DLLs that happened
until that moment.
** This code handles profiler labels and stackwalking suppression.
** Once a load has completed, we either pass the load on to XUL for further
processing, or save it for later if XUL is not initialized yet.
* mozglue has its own `ModuleLoadFrame` implementation for the legacy blocklist.
* `DllServicesBase` is updated to support the new interfaces.
* We implement `FallbackLoaderAPI` for `plugin-container`, `xpcshell`, and
any other non-`firefox` processes that do not have a launcher process
providing a loader API.
* We add some wide to UTF8 conversion functions.
Depends on D43157
Differential Revision: https://phabricator.services.mozilla.com/D43158
--HG--
rename : mozglue/build/Authenticode.cpp => mozglue/dllservices/Authenticode.cpp
rename : mozglue/build/Authenticode.h => mozglue/dllservices/Authenticode.h
rename : browser/app/winlauncher/freestanding/LoaderAPIInterfaces.h => mozglue/dllservices/LoaderAPIInterfaces.h
rename : browser/app/winlauncher/freestanding/ModuleLoadInfo.h => mozglue/dllservices/ModuleLoadInfo.h
rename : browser/app/winlauncher/NtLoaderAPI.h => mozglue/dllservices/NtLoaderAPI.h
rename : mozglue/build/WindowsDllBlocklist.cpp => mozglue/dllservices/WindowsDllBlocklist.cpp
rename : mozglue/build/WindowsDllBlocklist.h => mozglue/dllservices/WindowsDllBlocklist.h
rename : mozglue/build/WindowsDllBlocklistCommon.h => mozglue/dllservices/WindowsDllBlocklistCommon.h
rename : mozglue/build/WindowsDllBlocklistDefs.in => mozglue/dllservices/WindowsDllBlocklistDefs.in
rename : mozglue/build/WindowsDllServices.h => mozglue/dllservices/WindowsDllServices.h
rename : mozglue/build/gen_dll_blocklist_defs.py => mozglue/dllservices/gen_dll_blocklist_defs.py
rename : mozglue/build/moz.build => mozglue/dllservices/moz.build
rename : mozglue/build/MozglueUtils.h => mozglue/misc/WinUtils.h
extra : moz-landing-system : lando
This patch adds the following:
* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
owns its buffer. The buffers are null-terminated so that they may be used as
C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
current thread ID directly from the `TEB` when we do not yet have access to
kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
MFBT `Vector` in code that might not yet have access to Win32 heap functions.
Differential Revision: https://phabricator.services.mozilla.com/D43155
--HG--
extra : moz-landing-system : lando
* At this point our DLL blocking infra is complicated enough that I decided to
bite the bullet and move all of this code out of `mozglue/build` and into its
own subdirectory, `mozglue/dllservices`.
* We delete the original `UntrustedDllsHandler` code which is now obsolete.
* We implement mozglue's `LoaderObserver`:
** When this observer registers itself with the launcher process API, it
receives a vector containing all saved records of loaded DLLs that happened
until that moment.
** This code handles profiler labels and stackwalking suppression.
** Once a load has completed, we either pass the load on to XUL for further
processing, or save it for later if XUL is not initialized yet.
* mozglue has its own `ModuleLoadFrame` implementation for the legacy blocklist.
* `DllServicesBase` is updated to support the new interfaces.
* We implement `FallbackLoaderAPI` for `plugin-container`, `xpcshell`, and
any other non-`firefox` processes that do not have a launcher process
providing a loader API.
* We add some wide to UTF8 conversion functions.
Differential Revision: https://phabricator.services.mozilla.com/D43158
--HG--
rename : mozglue/build/Authenticode.cpp => mozglue/dllservices/Authenticode.cpp
rename : mozglue/build/Authenticode.h => mozglue/dllservices/Authenticode.h
rename : mozglue/build/WindowsDllBlocklist.cpp => mozglue/dllservices/WindowsDllBlocklist.cpp
rename : mozglue/build/WindowsDllBlocklist.h => mozglue/dllservices/WindowsDllBlocklist.h
rename : mozglue/build/WindowsDllBlocklistCommon.h => mozglue/dllservices/WindowsDllBlocklistCommon.h
rename : mozglue/build/WindowsDllBlocklistDefs.in => mozglue/dllservices/WindowsDllBlocklistDefs.in
rename : mozglue/build/WindowsDllServices.h => mozglue/dllservices/WindowsDllServices.h
rename : mozglue/build/gen_dll_blocklist_defs.py => mozglue/dllservices/gen_dll_blocklist_defs.py
rename : mozglue/build/moz.build => mozglue/dllservices/moz.build
rename : mozglue/build/MozglueUtils.h => mozglue/misc/WinUtils.h
extra : moz-landing-system : lando
This patch adds the following:
* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
owns its buffer. The buffers are null-terminated so that they may be used as
C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
current thread ID directly from the `TEB` when we do not yet have access to
kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
MFBT `Vector` in code that might not yet have access to Win32 heap functions.
Differential Revision: https://phabricator.services.mozilla.com/D43155
--HG--
extra : moz-landing-system : lando
* At this point our DLL blocking infra is complicated enough that I decided to
bite the bullet and move all of this code out of `mozglue/build` and into its
own subdirectory, `mozglue/dllservices`.
* We delete the original `UntrustedDllsHandler` code which is now obsolete.
* We implement mozglue's `LoaderObserver`:
** When this observer registers itself with the launcher process API, it
receives a vector containing all saved records of loaded DLLs that happened
until that moment.
** This code handles profiler labels and stackwalking suppression.
** Once a load has completed, we either pass the load on to XUL for further
processing, or save it for later if XUL is not initialized yet.
* mozglue has its own `ModuleLoadFrame` implementation for the legacy blocklist.
* `DllServicesBase` is updated to support the new interfaces.
* We implement `FallbackLoaderAPI` for `plugin-container`, `xpcshell`, and
any other non-`firefox` processes that do not have a launcher process
providing a loader API.
* We add some wide to UTF8 conversion functions.
Differential Revision: https://phabricator.services.mozilla.com/D43158
--HG--
rename : mozglue/build/Authenticode.cpp => mozglue/dllservices/Authenticode.cpp
rename : mozglue/build/Authenticode.h => mozglue/dllservices/Authenticode.h
rename : mozglue/build/WindowsDllBlocklist.cpp => mozglue/dllservices/WindowsDllBlocklist.cpp
rename : mozglue/build/WindowsDllBlocklist.h => mozglue/dllservices/WindowsDllBlocklist.h
rename : mozglue/build/WindowsDllBlocklistCommon.h => mozglue/dllservices/WindowsDllBlocklistCommon.h
rename : mozglue/build/WindowsDllBlocklistDefs.in => mozglue/dllservices/WindowsDllBlocklistDefs.in
rename : mozglue/build/WindowsDllServices.h => mozglue/dllservices/WindowsDllServices.h
rename : mozglue/build/gen_dll_blocklist_defs.py => mozglue/dllservices/gen_dll_blocklist_defs.py
rename : mozglue/build/moz.build => mozglue/dllservices/moz.build
rename : mozglue/build/MozglueUtils.h => mozglue/misc/WinUtils.h
extra : moz-landing-system : lando
This patch adds the following:
* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
owns its buffer. The buffers are null-terminated so that they may be used as
C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
current thread ID directly from the `TEB` when we do not yet have access to
kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
MFBT `Vector` in code that might not yet have access to Win32 heap functions.
Differential Revision: https://phabricator.services.mozilla.com/D43155
--HG--
extra : moz-landing-system : lando
This is showing up with hooks that are set both by the launcher process and by
the browser process when starting early DLL blocklist init on content processes:
* The browser's copy of mOrigFunc was set by the launcher process.
* The browser is setting a hook in the new child process, which writes to the
child's mOrigFunc.
* But FuncHookCrossProcess also writes that pointer to the browser's mOrigFunc,
thus corrupting the browser process's copy of the pointer.
For in-process hooks, we want to immediately write the stub pointer to its final
location; this is not an issue for cross-process hooks since the child process
is suspended when we do this and the parent process can't call the stub; there
is no possibility of a race.
Differential Revision: https://phabricator.services.mozilla.com/D46615
--HG--
extra : moz-landing-system : lando
* At this point our DLL blocking infra is complicated enough that I decided to
bite the bullet and move all of this code out of `mozglue/build` and into its
own subdirectory, `mozglue/dllservices`.
* We delete the original `UntrustedDllsHandler` code which is now obsolete.
* We implement mozglue's `LoaderObserver`:
** When this observer registers itself with the launcher process API, it
receives a vector containing all saved records of loaded DLLs that happened
until that moment.
** This code handles profiler labels and stackwalking suppression.
** Once a load has completed, we either pass the load on to XUL for further
processing, or save it for later if XUL is not initialized yet.
* mozglue has its own `ModuleLoadFrame` implementation for the legacy blocklist.
* `DllServicesBase` is updated to support the new interfaces.
* We implement `FallbackLoaderAPI` for `plugin-container`, `xpcshell`, and
any other non-`firefox` processes that do not have a launcher process
providing a loader API.
* We add some wide to UTF8 conversion functions.
Differential Revision: https://phabricator.services.mozilla.com/D43158
--HG--
rename : mozglue/build/Authenticode.cpp => mozglue/dllservices/Authenticode.cpp
rename : mozglue/build/Authenticode.h => mozglue/dllservices/Authenticode.h
rename : mozglue/build/WindowsDllBlocklist.cpp => mozglue/dllservices/WindowsDllBlocklist.cpp
rename : mozglue/build/WindowsDllBlocklist.h => mozglue/dllservices/WindowsDllBlocklist.h
rename : mozglue/build/WindowsDllBlocklistCommon.h => mozglue/dllservices/WindowsDllBlocklistCommon.h
rename : mozglue/build/WindowsDllBlocklistDefs.in => mozglue/dllservices/WindowsDllBlocklistDefs.in
rename : mozglue/build/WindowsDllServices.h => mozglue/dllservices/WindowsDllServices.h
rename : mozglue/build/gen_dll_blocklist_defs.py => mozglue/dllservices/gen_dll_blocklist_defs.py
rename : mozglue/build/moz.build => mozglue/dllservices/moz.build
rename : mozglue/build/MozglueUtils.h => mozglue/misc/WinUtils.h
extra : moz-landing-system : lando
This patch adds the following:
* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
owns its buffer. The buffers are null-terminated so that they may be used as
C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
current thread ID directly from the `TEB` when we do not yet have access to
kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
MFBT `Vector` in code that might not yet have access to Win32 heap functions.
Differential Revision: https://phabricator.services.mozilla.com/D43155
--HG--
extra : moz-landing-system : lando
In some situations, entries may in fact take more than half the buffer size
(e.g., when duplicating a stack into a small temporary buffer).
So we now allow blocks to take the full buffer size -- but not more, as they
would start overwriting themselves!
Differential Revision: https://phabricator.services.mozilla.com/D46453
--HG--
extra : moz-landing-system : lando
