gecko-dev/taskcluster
Mike Hommey 16c5f4c5f4 Bug 1432390 - Use zstandard and requests modules instead of spawning curl | zstd in docker.load_image. r=dustin
The zstd command we spawn, if available at all, might be the wrong
version: zstd changed its stream format in an incompatible way at some
point, and the version shipped in e.g. Ubuntu 16.04 uses the old format,
while the version taskcluster relies on uses the new format.

Relying on gps's zstandard library allows to ensure we use the right
version. Another advantage is that we can trivially pip install it in a
virtualenv if it isn't available on the system running the command.

If we're ridding ourselves of the subprocess spawning for zstd, we might
as well cover curl as well. Especially considering the error handling
when subprocesses are involved is not trivial, such that the current
error handling code is actually broken and leads to dead-lock
conditions, when, for example, curl is still waiting for the python side
to read data, but the python side is not reading data anymore because
an exception was thrown in the tar reading loop.

--HG--
extra : rebase_source : 054c37cfaa68bf475b37545ebaa99144584b93d4
2018-01-24 11:18:13 +09:00
..
ci Bug 1433185 - Require Rust 1.23. r=nalexander 2018-01-25 10:29:11 -08:00
docker Merge mozilla-central to mozilla-inbound. a=merge on a CLOSED TREE 2018-01-25 00:07:47 +02:00
docs Bug 1431397 - Pass release_eta down from action task input r=bhearsum 2018-01-18 15:10:53 +01:00
scripts Bug 1431809 Bump MinGW version for new Product Constants r=froydnj 2018-01-23 17:57:08 -06:00
taskgraph Bug 1432390 - Use zstandard and requests modules instead of spawning curl | zstd in docker.load_image. r=dustin 2018-01-24 11:18:13 +09:00
.yamllint Bug 1414824 - Enable stricter linting in taskcluster yaml files. r=Callek 2017-11-11 00:36:41 -05:00
mach_commands.py Bug 1432390 - Use zstandard and requests modules instead of spawning curl | zstd in docker.load_image. r=dustin 2018-01-24 11:18:13 +09:00
moz.build Backed out 2 changesets (bug 1403519) for bustage on \python\mozbuild\mozbuild\test\frontend\test_reader.py on a CLOSED TREE 2018-01-17 07:23:40 +02:00