In contrast to Posix the returncodes on Windows have positive
and not negative numbers.
MozReview-Commit-ID: 4foHWf9RR0B
--HG--
extra : rebase_source : 8e06cbb3e669fea7abe46cd8b53386b56030574d
Currently the returncode gets set immediately after the process
has been terminated via TerminateJobObject() or TerminateProcess().
Given that in both cases the process has not been quit yet, but
still waits for all streams to be closed, the returncode has to
be set by via wait().
Also in case of TerminateJobObject() the _cleanup method is never
called if an exception occurs.
MozReview-Commit-ID: 4NEyqafN0DD
--HG--
extra : rebase_source : ae176d5e052785cc77865e1bf220013e87d7a3f0
The psutil package has only been used to check for the existence
of a given pid. Given the troubles with getting psutil compiled
on Windows, or by supplying the correct wheel, it has been decided
to get rid of this dependency.
Instead the ProcessHandler class itself now got the feature to
determine the existence of a pid by using ctypes to do the
necessary Windows API calls.
MozReview-Commit-ID: KAiSv0AH8HZ
--HG--
extra : rebase_source : 55e9ecac6ce12b0abcbaceb9aa385100744b16dd
Only test_mozprocess.py was still using the C implementation but is
disabled since ages.
Given that the proclaunch script as written in Python replaced the
C implemenation lets remove all the old unused code.
MozReview-Commit-ID: J4izHz5ljtO
--HG--
extra : rebase_source : e33720aa3a6734fa3dd5fc082441ef54d515e75f
This is a new issue that gets linted with flake8 3.5.0. Basically you should
never use a blank except: statement.
This will catch all exceptions, including KeyboardInterrupt and SystemExit
(which is likely not intended). If a catch all is needed, use
`except: Exception`. If you *really* mean to also catch KeyboardInterrupt et
al, use `except: BaseException`.
Of course, being specific is often better than a catch all.
MozReview-Commit-ID: FKx80MLO4RN
--HG--
extra : rebase_source : 7c74a7d0d81f2c984b47aff3a0ee3448b791177b
This is needed before we can upgrade to flake8 3.3.0, as that version starts flagging these errors.
These files were modified by running:
autopep8 --select E305 --in-place -r <dir>
on the affected directories. I did it one dir at a time and verified the result after each.
MozReview-Commit-ID: FmlsfiKIbtr
--HG--
extra : rebase_source : 9df32258cadff5d27a0e72113c57f782756c0b18
The subsuite is added conditionally because we only have the capability of
running source-check tasks on linux at the moment. Once taskcluster support
for windows and mac has matured a bit and the taskcluster configs support
source-check there, we should apply the subuite unconditionally.
MozReview-Commit-ID: Kk9Irz3fn14
--HG--
extra : rebase_source : b9266a06583083c36477d4e93f5462ee614cdb71
The mozbase unittests don't use mozunit, so their output is confusing in the log.
This makes mozbase output consistent with the rest of the python unittests.
MozReview-Commit-ID: AIs5mza8Rn6
--HG--
extra : rebase_source : 10f65e612f5b3cebb921c47699f5a8be7cd2ba5a
For some reason calling os.getpgid(proc.pid) in this bug results in an OSError "No such process"
on OSX. This bug is starting the ProcessHandler from a concurrent.futures Thread, that must be
somehow related.
I tried debugging this, but couldn't figure out why this is happening. However, the pgid is not
needed for this use case, and simply ignoring the error works. We also ignore this very same
exception when calling os.getpgid elsewhere in mozprocess, so there must be some weird OSXism
happening.
MozReview-Commit-ID: 2YXhBaORC5s
--HG--
extra : rebase_source : 120e4bff7ef29d2a0ad1e3bdd2df11b8b682d981
added testing/mozbase to tools/lint/flake8.lint
fixed a first batch of PEP8 errors/warnings
at first the commad autopep8 -i --max-line-length 99 -r -j 8 .
has been used to fix simpler problems, run from testing/mozbase
some of the issues can not easily fixed :
- undefined 'names' in code for example isLinux - isLinux and isBsd "fixed" with # noqa
- undefined 'message' resolved with return fmt.format(...
- undefined 'structured' resolved replacing those with mozlog
- long comments - some remaining - addressed with # noqa
- package level import everything - addressed with # flake8: noqa
restored testing/mozbase/mozdevice/mozdevice/Zeroconf.py
fixed issues reported on mozreview
fixed ')' in testing/mozbase/mozprocess/mozprocess/qijo.py imports
finally fixed multiline string at testing/mozbase/manifestparser/tests/test_manifestparser.py:114
^^^ and again, but now with ./mach python-test --path-only testing/mozbase/manifestparser/tests/test_manifestparser.py passing
fixed testing/mozbase/manifestparser/tests/test_convert_directory.py assert
fixed this error:
10:15:21 INFO - return lambda line: stack_fixer_module.fixSymbols(line)
10:15:21 INFO - TypeError: fixSymbols() takes exactly 2 arguments (1 given)
fixed two spaces lint error even of # noqa comments
restored assignement to lambda with # noqa to silence the lint error
global noqa for testing/mozbase/manifestparser/tests/test_filters.py
stupid is/is not error...
MozReview-Commit-ID: 1FpJF54GqIi
--HG--
extra : rebase_source : 3cf0277fb36a296e3506aeacc2ff05e1b03f9eac
MozReview-Commit-ID: 9e6HNAaF0Yo
In case of in-process restarts it can happen that the new process gets forked into a new process group.
When that happens we loose the capability to kill the process. To prevent a hang when joining the output
reader threads in wait(), we simply skip that call by passing-through the IO error.
--HG--
extra : rebase_source : 702dfec407ed13114f59fa6ccb0d82c5b0790550
Sometimes the IO completion port doesn't shutdown child processes. When this happens,
mozprocess will attempt to force kill the child processes manually. However, there is
a bug here which causes the OSError to get raised.
Although this fixes that bug, the original issue(s) which prevented the IOC port
from signaling shutdown remain and are still undiagnosed.
MozReview-Commit-ID: L3DQPW0Is5v
--HG--
extra : rebase_source : cf6320cffea5a4c8fb5d62861c41065d9dcefa52
We can set MOZPROCESS_DEBUG to help debug windows process code. However on try
it is unreadable as there are multiple things using mozprocess, and each process
has multiple threads. It's impossible to tell which log message comes from where.
This improves the debug logs a bit by always specifying the PID and thread name.
There are a few other drive-by cleanups in this thread. The only one of note is
removing a python 2.5 only code path.
MozReview-Commit-ID: L3DQPW0Is5v
--HG--
extra : rebase_source : f07c07f53b06b1160cd3e70cb06b8dc12a89c3ab
This now check process status based on pids, not anymore
on an arbitrary string given in the command line.
Some cleanup in there, including removing code duplicate.
This patch detects when breaking away from the parent job isn't strictly necessary to manage processses in a new job to allow using nested job objects to manage processes and their children. Loss of job object functionality is handled in mozprocess as non-fatal, however mozrunner and others doing things like restarting firefox require it.
A new parameter called 'processStderrLine' is added. When specified, stdout gets processed by the
'processOutputLine' callbacks and stderr is processed by the 'processStderrLine' callbacks. When
not specified, stderr is redirected to stdout which is the same default behaviour.
A side effect of this is that mozprocess now uses three threads to process output. One thread each
for stdout and stderr that reads output lines and stores them in a Queue as fast as possible, this
makes sure there is no blocking in stdout.write(). A third thread executes the callbacks.
