We recently switched make check to call into |mach python-test| rather than invoking python
itself for each test file. But this ended up slowing down the tests as they were no longer
being run in parallel. This patch adds a --jobs flag to python-tests and runs test files in
parallel.
Note: if more than one job is used, output per test will be buffered and printed at the end
to avoid interleaving. This has the unfortunate side effect of making |mach python-test| look
like it is hanging, especially if running a very long file like mozbase's test.py. For this
reason, we still use -j1 by default so output will continue to be streamed. In automation we
will use multiple processes though.
MozReview-Commit-ID: 3u0wOFmyQLI
--HG--
extra : rebase_source : d08ac412023731c46226c7adbf5f6e798b9a345a
We need the fix from https://github.com/agronholm/pythonfutures/issues/25
to allow sending KeyboardInterrupts to thread pools.
MozReview-Commit-ID: 5VfBttLbKOr
--HG--
extra : rebase_source : 8e3aa7a1d6fbbaa7b94cea35b196b35e103a1e33
Some mach commands may want to re-use a requirements.txt file rather than installing packages
individually. This enables --require-hashes which means all packages and dependencies must be
listed with their hashes. For more details, see:
https://pip.pypa.io/en/stable/reference/pip_install/#hash-checking-mode
MozReview-Commit-ID: 3lOutbcSzIY
--HG--
extra : rebase_source : d07ac21bd1f2f0009465f9e004208464b22de01b
The current state of python configure sandbox execution is that if a
template imports a module, and a function defined in the template tries
to use the module, it doesn't work. Ideally, it would, but rather than
try to fix this corner case, we remove the unit tests that assume it
works (and consequently pass for half bad reasons), and add a unit test
so that the behavior doesn't change unwillingly.
--HG--
extra : rebase_source : 579ba2bc7c19d4fe7df11bbdb1ceb6171a1ee857
The way it works now, `mach` commands often invoke subprocesses where
the subprocesses' stdio file descriptors are pipes so the mach command
can e.g. parse output.
Processes like clang, gcc, and cargo determine if they can send color
codes to {stderr, stdout} by seeing if those file descriptors are TTYs.
When e.g. `make` is executed via `mach`, this test fails because those
descriptors are pipes (even though they eventually end up on a TTY).
We can't wire the file descriptors to the TTY because `mach` needs
to analyze output. We don't want users defining process flags to force
color in their mozconfigs because color codes would still be sent
if stdout was not a TTY.
This patch sets the MACH_STDOUT_ISATTY environment variable in all mach
commands when stdout is a TTY. Subsequent processes can then look for
this variable to determine whether to override color settings, print
terminal control codes, etc.
MozReview-Commit-ID: GxXP2mQssjC
--HG--
extra : rebase_source : 4b99547b453cb7dd5cb590a71ed554ce2bc4759d
Currently, environment variables set when running mach commands will
propagate after the command is finished. This can allow unwanted state
to bleed through.
This likely isn't an issue today, but isolating state during code
execution is generally a good practice. So do that.
MozReview-Commit-ID: AdaomGub5EF
--HG--
extra : rebase_source : ce81987a1f6de3a16bce6a9e45b9dc8e8eb29b4b
Also allow when=True/False to avoid the chicken-egg problem of using
a generic `when` to use in replacement of @depends('--help') for things
like @dependable.
--HG--
extra : rebase_source : f1571a5b904efb66a361b90f3b7e1edbaa75772e
--help dependencies currently help identify functions that will run when
running configure --help, which we don't want to have spreading too
much. OTOH, when such functions have no side effect, it's not really
that important to have them explicitly marked.
So, allow missing --help dependencies for functions that:
- don't use @imports
- don't have a closure
- don't use global variables
This is a first step towards entirely removing the --help markings (the
end goal being that --help dependencies will indicate actual --help
dependencies). As such, we don't really care about updating the lint
error message.
--HG--
extra : rebase_source : e81ec9b51ff01c2ee75722904e551286aa0b2bec
We want functions without an @imports to not have any side effects, and
to not use external resources. So remove the few functions we expose from
os.path without @imports('os') that do.
--HG--
extra : rebase_source : a9485ec269d4de5785d66d7772eda4fae5a84b4a
Missing such dependencies shouldn't impair running configure itself
after local modifications, but they are currently required for
(mostly) documentation purpose. Which means they are better done in
the linter.
--HG--
extra : rebase_source : 6bfff2342cda2ed1351f561c9eb9623f1fb4e4c4
Previously if an Arch Linux user had a different package extension configured
in `/etc/makepkg.conf` building AUR packages during bootstrap would fail.
Forcing the extension by providing it as an environment variable makes sure
building doesn't fail regarding of a user's configuration.
MozReview-Commit-ID: 4aryYS0XVr7
--HG--
extra : rebase_source : 4c466e49f729de625e814a92325c6d38e6d1e0b4
