With this new approach, Shmem instances will now have their handles
transferred inline within messages as attachments, rather than being
associated with their actors and sent in separate messages.
This has a few advantages:
* The implementation is much simpler
* Releasing all references to a Shmem will automatically destroy it by
RAII, rather than leaking the shared memory region until the toplevel
actor is destroyed, removing the need for types like RaiiShmem.
* This allows re-transmitting Shmem instances to another process, as we
don't close the shared memory region handle upon receiving it.
But also has a disadvantage that because we keep alive the shared memory
region's handle until the shmem is destroyed, so that it can be
re-transmitted, we may end up using more FDs or HANDLEs while running.
This patch intentionally doesn't change or simplify callsites, removing
APIs like RaiiShmem, in order to make it easier to revert if this causes
issues on platforms like Linux due to FD exhaustion. If we don't run
into increased resource exhaustion problems, we can make these changes
in a follow-up.
Differential Revision: https://phabricator.services.mozilla.com/D140211
Previously, when using the system Python packages, Mach would reuse the
values already existing in the `sys.path`. This had two benefits:
1. We didn't have to do work to calculate which paths the "system
Python" specifically referred to.
2. This allowed us to support nested Mach calls
(such as `./mach --virtualenv psutil ./mach build`).
However, it came with its own headaches, specifically around "consistent
imports" and Python subprocesses, such as in the following example
1. Mach runs "using system Python"
2. The "build" site (for example) would be activated. The current
`sys.path` is included in the virtualenv's `mach.pth` file so that
subprocesses will have access to the same packages as the primary
Mach process
3. //Something// adds something to the `sys.path`
4. The build virtualenv is redundantly re-activated (such as due to
generic setup code for some module). Though we don't physically
re-activate the virtualenv, we //do// check that it's up-to-date to
validate assumptions - **and it's not!** It's missing the
//something// that was added to the `sys.path`.
5. We can't re-create the build virtualenv because it's already active,
so:
6. An error is raised so that unexpected out-of-date-ness doesn't fly
under the radar.
-----
This patch solves this by calculating the "system Python" paths in a
deterministic way. Then, even when using the system Python, the Mach and
command sites use define themselves in a consistent way.
This means that we can't do `./mach --virtualenv <venv> ./mach ...` to
shares `venv`'s packages with the whole new Mach process. Fortunately,
this has been replaced with moving such packages into the nested Mach
command's requirements definition instead.
TL;DR: more consistent, less failures. 👍
-----
One more detail, this time around the `sys_path()` function:
Ideally, we could calculate the standard library paths *and* the
"system" paths with only one Python subprocess. Unfortunately,
that's not the case:
* If `-S` isn't provided, then it's tricky to separate the standard
library paths from the system paths
* If `-S` is provided, then doing `site.getsitepackages()` in a
virtualenv returns the *system* site packages, not the virtualenv's.
To work around this, we call the external python twice, and in
parallel. This allows resolving the information we need while avoiding
performance costs.
Differential Revision: https://phabricator.services.mozilla.com/D143200
Cleans up the interface to AudioSession and brings in line with COM best practices. Uses background threads that are implicitly MTA and asserts proper thread behavior. This also removes AudioSession's Co[Un]Initialize static analysis violations.
Differential Revision: https://phabricator.services.mozilla.com/D140741
This metadata is based on what I thik we should include. We don't need most
of the documentation section (category, description) since they are specific
to ESLint. The url is useful, as it will show up in VSCode and other editors
for developers wanting more information.
The type of rule probably isn't important, but is kept as a general guide.
Differential Revision: https://phabricator.services.mozilla.com/D142286
ESLint is only documenting the new module style of meta data and returning a create function.
Using this format allows us to easily specify extra meta data, e.g. documentation urls.
Hence, lets move all our rules across to the newer style so that any copy and paste will use
the new style, and we'll also be set up if ESLint requires it in the future.
Differential Revision: https://phabricator.services.mozilla.com/D142284
This is code written by Mozilla employees but licensed under dual Apache2 / MIT
license and originally hosted on GitHub. Rather than try to create a text that
will pass the license lint, just exclude it from linting.
Differential Revision: https://phabricator.services.mozilla.com/D139865
* Remove mention of --enable-address-sanitizer, since it's not at all sufficient on its own. (Leave link to asan docs though)
* Clarify that ./mach gtest dontruntests is only needed for gtests. (I didn't need it for grizzly replays)
Differential Revision: https://phabricator.services.mozilla.com/D141542
The regular xpcshell test don't appear to fail, so this additional patch ensures that only the TV (test verify) runs of test_feature_nativeallocations.js are skipped.
Differential Revision: https://phabricator.services.mozilla.com/D141814
All commands declaring a virtualenv will have them activated before the
command executes. Removes all now-redundant manual activations of
declared virtualenvs.
Commands that don't declare a virtualenv will still implicitly be
associated with the "common" virtualenv, but unlike explicit
virtualenv declarations it'll have to be activated manually, just
like it was before this patch.
To smooth the migration with existing usages, virtualenv activation
behaviour was changed slightly: if attempting to activate a new
virtualenv, but the source venv is already command venv, then raise an
exception. (In the future, we should improve testability of
virtualenv scaffolding logic so that tests can be added for this
sort of thing.) This did cause some issues with some tests, which
will be solved more cleanly with bug 1724273. In the meantime,
minimal modifications were made to failing tests to keep them green:
* `test_command_line.py` was activating the `common` virtualenv so
that it could install `mozproxy`, and use its CLI. Instead, I
modified the test to use `mozproxy` using the "module" interface
(`python -m mozproxy ...`). At that point, `MozbuildObject` was
unnecessary and usages were replaced with simpler variants.
* `test_vendor.py` needed its explicit `activate_virtualenv()` call
patched out. It still needs to use a virtualenv's Python
executable, but due to `sys.executable` now being kept up-to-date
as of bug 1717051, it could be used directly.
Differential Revision: https://phabricator.services.mozilla.com/D122892
Creates/updates virtualenvs for some mach commands, replacing
their ad-hoc usage of `install_pip_package()`, `pip install`,
and `sys.path` modifications.
Note: The `docs` virtualenv has `Sphinx==1.1.3` installed, even
though a more modern version of `Sphinx` is used when
`./mach doc` is run. This is ok for now, since `./mach doc` will
just install the newer `Sphinx` over top of the old one. Secondarily,
when we port `./mach doc` to use the centralized system, we'll
be incentivized to make the different `doc` commands use synchonized
versions of the same packages. Success!
Also, note that manual installation of `html5lib` and `requests`
isn't ported to the `wpt` site: this is because they're already
provided by the inherited Mach site.
Differential Revision: https://phabricator.services.mozilla.com/D122902
The native allocations feature is crashing tests on Android in the watchdog thread, possibly due to its CPU overhead.
This feature is rarely used, and mostly by Firefox developers, so it's okay to keep it available without tests.
Differential Revision: https://phabricator.services.mozilla.com/D141568
