Also adjust 'basestring' usage in some places.
Differential Revision: https://phabricator.services.mozilla.com/D28102
--HG--
extra : rebase_source : f0fecd6b040336b0cf2f055e266fe37eea5a9a48
extra : intermediate-source : 712718250f0ba5c286c1fb6b6c7f370b713c7225
extra : source : 6b24ce2da518d8158e13627adb11c9fcad19c92a
shellutil.py needed special handling since it is invoked earlier than the
vendor directory adding we do, so it does not have six available.
Differential Revision: https://phabricator.services.mozilla.com/D28096
--HG--
extra : rebase_source : c5c611d280dc821747c9db736e4943c2cc421557
extra : intermediate-source : 30fb268b41fdc45937a22ab446a8805fb099c1f0
extra : source : d102292b6b5b812e7df165116410488d18042bd3
Lint python/mozbuild/{mozbuild,mozpack}.
This makes sure we use byte strings (type(b'')) in many places.
This also has a few places where we know we want unicode, by enforcing it.
This code also has a few places where we call into windows API's which expect either all args to be unicode,
or all args to be bytestrings (not-unicode) so make sure those calls align.
This is the most risky of the stack, since there are some mach commands and codepaths that are neither excercised in automation
nor excercised in tests.
Differential Revision: https://phabricator.services.mozilla.com/D26645
--HG--
extra : rebase_source : 1ee367576bede0318a2db05644693d4f440a250a
extra : intermediate-source : 2516c350cba6f27eb204f0087728721abbb0e085
extra : source : 6351a29c5765c13c39ef6186f9e5e911e00420e7
Leaving one fix for an explicit review.
Lint python/mozbuild/{mozbuild,mozpack}. r=#build
Differential Revision: https://phabricator.services.mozilla.com/D26641
--HG--
extra : rebase_source : e14d7762c7802d98e24ee8f86b0167d96666d288
extra : intermediate-source : 79b0d1678dd2f6aadf60f480d902ddeca4a52c3a
extra : source : faadf440e7d7b0a8c473e2dee3fc90b27023814a
Lint python/mozbuild/{mozbuild,mozpack}.
ran './mach lint -l py2 -l flake8 -l shellcheck -l codespell -l yaml python/mozbuild/{mozbuild,mozpack}/ --fix' in order
to undo some black changes and get closer to making this folder able to be validated on every lint run
Differential Revision: https://phabricator.services.mozilla.com/D26640
--HG--
extra : rebase_source : 6b69b6ebcac73835b752607f7b5b4429de7f95cf
extra : intermediate-source : 114355f83d36188fac592c8c6497242c71b27ad6
extra : source : 51fbdf63bbce8f75fac407de305f826dc775fcb5
Having `rustc` be `rustup`'s wrapper for `rustc` means that we may
silently honor `rustup`'s override mechanisms. We noticed this first on
OS X, where we use the "real" `cargo` but `rustup`'s `rustc` wrapper,
and problems ensued when `cargo` thought it was using one version of
`rustc`, but actually wound up using something different.
It seems better to avoid silently interposing `rustup`'s toolchain
override mechanisms everywhere, rather than having to special-case OS
X. So let's factor out a general mechanism for removing the wrappers
`rustup` provides and use that for both `rustc` and `cargo`. The tests
need adjusting because we weren't triggering the unwrapping cases
before; we don't yet test the case where we really do need to unwrap.
That test can be left for a future patch.
Differential Revision: https://phabricator.services.mozilla.com/D29531
--HG--
extra : moz-landing-system : lando
We've not been checking the clang-cl version in use. This lack of
checking is bad, for a couple of reasons:
* Released versions of clang-cl differ drastically in their robustness;
* Only the most recent version of clang-cl supports aarch64.
We should check for a minimum version of clang-cl, just like our other
supported compilers. As a bonus, we can then start depending on
features that we know appear in the particular minimum clang-cl
version. (The current patch is motivated by `/clang:` command-line
support, but one could pick other things.)
Differential Revision: https://phabricator.services.mozilla.com/D30723
--HG--
extra : moz-landing-system : lando
Also for checker `modernize-avoid-bind` export the respective reliability index from config.yaml
Differential Revision: https://phabricator.services.mozilla.com/D30904
--HG--
extra : moz-landing-system : lando
0.1.21 mishandles cargo package renames, which are a required
feature for Rust 2018 support. The latest version fixes this.
Differential Revision: https://phabricator.services.mozilla.com/D29946
--HG--
extra : moz-landing-system : lando
Under the hood, browsertime invokes a certain `visualmetrics.py`
script. That script depends on `ffmpeg` and ImageMagick's `convert`,
`compare`, and `mogrify` commands. It also depends on certain Python
packages.
So this installs those dependencies, and then wires up the evaluation
environment such that `./mach browsertime` can find the dependencies.
It also adds a `./mach visualmetrics` command for processing a
captured MP4 file in the same way that browsertime processes such a
file.
In order to avoid downloading dependencies multiple time, the existing
artifact cache is extracted. This is a small first step towards [Bug
1526021](https://bugzilla.mozilla.org/show_bug.cgi?id=1526021), which
might want to use this artifact cache as well.
At this time, hashes and filesizes are not verified. During
development, the upstream files changed multiple times, and it's not
worth being completely locked down while experimenting with this
functionality. If we start running this code in automation or in more
sensitive environments, we can build fetch tasks and TC indexes to
streamline the artifact gathering process.
It is expected that a future mach command will want to invoke
browsertime without suffering the overhead of invoking Python (and
mach, which is itself bulky) so a nod is given to exposing the
relevant environment pieces.
During testing, it was discovered that [MozillaBuild doesn't ship
git](https://bugzilla.mozilla.org/show_bug.cgi?id=1503028), so that
git repositories can't be used out-of-the-box on Windows. So instead
we use a [tarball link from github.com/$USER/$REPO/tarball/$COMMIT-LIKE](https://github.blog/2008-03-03-tarball-downloads/).
Differential Revision: https://phabricator.services.mozilla.com/D29442
--HG--
rename : python/mozbuild/mozbuild/test/test_artifacts.py => python/mozbuild/mozbuild/test/test_artifact_cache.py
extra : moz-landing-system : lando
Some media libraries use gas syntax in their assembly files. Rather than
converting these arm assembly syntax files for aarch64, we can use clang-cl
to build them directly.
Differential Revision: https://phabricator.services.mozilla.com/D27785
--HG--
extra : moz-landing-system : lando
With this change tooltool.py also supports taskcluster credentials to be passed
(in json format) to --authentication-file option. RelengAPI tokens are still
working with this patch, just additional authentication is added.
Differential Revision: https://phabricator.services.mozilla.com/D27881
--HG--
extra : moz-landing-system : lando
The noatime option doesn't appear to have a significant impact on no-op
build times, even though the build system does a significant number of
file reads. Since macs have atime enabled by default, we should just
disable this check rather than give developers unhelpful error messages.
Differential Revision: https://phabricator.services.mozilla.com/D28906
--HG--
extra : moz-landing-system : lando
For some reason, using `tar -Jxf` with Windows paths fails, and that makes
`mach artifact toolchain` unable to pull .tar.xz archives on Windows.
However, executing `xz -d -c` works, and we can feed its output to
python's tarfile.
Differential Revision: https://phabricator.services.mozilla.com/D28779
--HG--
extra : moz-landing-system : lando
With this change tooltool.py also supports taskcluster credentials to be passed
(in json format) to --authentication-file option. RelengAPI tokens are still
working with this patch, just additional authentication is added.
Differential Revision: https://phabricator.services.mozilla.com/D27881
--HG--
extra : moz-landing-system : lando
Add basic support for 'mach gtest' on Android.
Handling of Android-only and desktop-only options is awkward; I hope to
re-visit this after bug 1519369.
Differential Revision: https://phabricator.services.mozilla.com/D28129
--HG--
extra : moz-landing-system : lando
