This fixes a regression from bug 1408365. Ideally these imports would
be defined at the top of the file, then the py2 linter would have caught
them. Even more ideally, mozboot would have some tests that hit these
code paths.
MozReview-Commit-ID: BWvIwAdUBF4
--HG--
extra : rebase_source : 6472f730a5cd12aa98af7a21f958d1ad2400f995
I think we're approaching an inflection point for the bootstrapper,
one where it no longer is possible to bootstrap without a source
checkout. For now, however, let's just do the simplest thing and
install the Proguard JAR along the happy path.
MozReview-Commit-ID: xUY37eE6oR
--HG--
extra : rebase_source : 31549ab3b6d662d84761c2a260cd236a5809c8ac
The goal is to use a newer Android-Gradle build plugin version (2.3.3
is latest stable). That requires a modern Gradle (anything 3.3+, but
3.4.1 is the default from my Android Studio), and also a newer
build-tools (25.0.3 is latest stable).
The locations of lint output changed, and we want to use the standard
output location because it's difficult to accommodate variant details
in custom names. We change the location of findbugs output to follow
suit.
This requires either:
- fixing lint errors
- adding to the lint whitelist
- using the new lint baseline
It's best to use the new lint baseline, which will happen in the next commit.
MozReview-Commit-ID: D19FzIDCJrE
--HG--
extra : rebase_source : 12d132c0c3e0dbe2b8873b31360ea96d612de44c
We expect to start requiring rust 1.21 soon, so have
./mach bootstrap upgrade users to it to consolidate
churn between now and then.
MozReview-Commit-ID: HEnXm25duUr
--HG--
extra : rebase_source : d05071ee5c2852eb69da22e80623f21ca8c6f7cb
The old system was simply in place because I couldn't figure out how
to pipe `yes | ...` in Python. This is good enough to replace it, and
much less fragile since the license hashes change frequently.
MozReview-Commit-ID: AhJJPqMKfUh
--HG--
extra : rebase_source : 86289e692d646181d545457fc953610e165ee2df
GCC isn't safe to use on architectures that switched to Clang because
libstdc++ and libc++ aren't very compatible. Newer LLVM and Clang are
often already installed as a dependency for Mesa packages. So, always
require llvm* package.
MozReview-Commit-ID: 8651mz5tiIp
--HG--
extra : rebase_source : 7713e167b34f14a18fd5bf9c5ec33e926b2b929c
The MozillaBuildBootstrapper specific rust install code in not needed as
mozbase already includes genertic code to achieve the same outcome. The
mozilla-build specific code also leads to issues where it tries to add already
existing targets and fails the bootstrap. This changeset removes the
mozilla-build specific step.
MozReview-Commit-ID: G0BqKZrF40A
--HG--
extra : rebase_source : 60e9638afff744c937a9665d6fd5830187835ea4
This also passes the '--noreplace' option to all the emerge invocations thus
preventing already installed packages from being rebuilt from scratch.
MozReview-Commit-ID: 4JBuptmgS3Y
--HG--
extra : rebase_source : e581607d4a2e997e7d79c7c4496d13b8e9b10e50
This passes `python3 -mcompileall`. Changes:
* use `0o` prefix for octal literals
* print as a function
* except .. as
* use six.reraise to replace a multi-argument raise statement
* use six.string_types and six.moves.configparser
* remove uses of `L` suffix for long integers
MozReview-Commit-ID: KLaYRNHGpay
--HG--
extra : rebase_source : 6ca1b5447cd28eff8d9f2805add6a0f07e8b4c63
We correctly worked around problems for `hg pull` but not for
`hg update`. Make the workaround consistent.
MozReview-Commit-ID: 7A4dgANO0ip
--HG--
extra : rebase_source : fd4b90200ab3ce07a3921a02ae28f5a7a0c3d83c
Hacky, but it works -- until Google updates its license hashes. This
looks ahead to using |mach bootstrap| to build docker images.
MozReview-Commit-ID: DF23v8tr8SW
--HG--
extra : rebase_source : 5a80bdd5ddfb551b374464f42c3783fef5a71fc3
This looks ahead to using |mach bootstrap| when building docker
images.
MozReview-Commit-ID: LoMU7ji5T0x
--HG--
extra : rebase_source : 2885195855497d69b1cbba02a943964dd3286b93
This was a regression introduced in Bug 1344244. Google changed the
layout of their packages. It used to be that the Android SDK archive
had a top-level 'android-sdk-$OS_NAME' directory; that's no longer the
case. It would be cleaner to unpack to 'android-sdk' without the OS
name, but that complicates the logic that detects existing installs.
MozReview-Commit-ID: 4B2Rt1AM2ky
--HG--
extra : rebase_source : 3cd1fef88cbb47fc9cd4e47a2a4bc2277c576c58
Consider Rust toolchains prior to 1.19.0 old enough to update
by the `mach bootstrap` command, in preparation for requiring
that release during Firefox 57 nightly development.
MozReview-Commit-ID: JaaM3sPDmzn
--HG--
extra : rebase_source : 3835103a822cfc05bde92ee920b344fe7b68d61c
There's no need to install the caskroom bits for Homebrew now,
allowing to fix Java installation on Mac OS X and simplify the code at
the same time.
MozReview-Commit-ID: 1Ssjm4YRrPQ
--HG--
extra : rebase_source : 3414d1fbe2bdb693cae1f5b1379d8d9335f1e7f4
This is the real fix. Google has replaced the |android --no-ui ...|
tool with a simpler |sdkmanager| tool, which makes it easier to
install packages with particular major versions. (Minor versions
still can't be controlled; for example, the m2repository extras are
constantly rolling forward.)
|sdkmanager| fails if the required packages aren't installed and can't
be installed, so there's no need to search for missing packages, etc,
simplifying the code considerably.
I don't see an easy way to upgrade outdated Android SDK installations
-- it's not clear that unpacking over top of an existing SDK
installation succeeds -- so I've included a message about moving or
removing outdated installations. This will punish folks who have
added additional Android platforms, or download emulator images using
the Android toolchain (but not those downloaded using |mach
emulator|). C'est la vie.
MozReview-Commit-ID: GLhKyuq701k
--HG--
extra : rebase_source : 26578c5ef4dcc6a29809630add2232a98407ec54
This refactoring unifies similar code that has been copy-pasted and
subsequently diverged.
MozReview-Commit-ID: EuVQBR4gsDo
--HG--
extra : rebase_source : bda66ef9001a1cddf75417aaeebd9dcecd05a6b7
With configure now being able to auto-detect the presence of a `mach
bootstrap`-installed clang and libclang and the upcoming
build-by-defaultness of Stylo, we can start downloading these packages
all the time.
LLVM/clang is needed for Stylo's bindgen, and Apple's clang is unusable
for such purposes. For other platforms, we have installed LLVM/clang
from our tooltool archive on the supposition that those packages are
definitely known to work, as we use said packages in automation. For
Mac, however, we haven't been able to generate packages for tooltool
that successfully build Stylo, and even if we had, those packages would
solely be used for developer builds of Stylo and would not be used in
automation. The case for downloading LLVM/clang for Mac from tooltool,
therefore, is not as strong as for other platforms.
As a result, we'll rely on the installed package manager for LLVM/clang,
which many people may have installed anyway.
In passing, also delete some old code for OS X versions < 10.7; such
platforms are no longer supported for running or building Firefox.
When determining the path of a possible mozilla-central checkout,
bootstrap currently considers the existence of a `.git` directory as
sufficient proof that the containing directory is a git checkout.
Unfortunately, if you happen to execute standalone bootstrap from a git
checkout of something else, you're gonna have a bad time.
To prevent this, check for the existence of a moz.configure file. This
is not an ideal proof, but it is better than what we currently have.
Bump the target version and checksums for the rustup installers
we use for the latest release so reduce the variance with manual
installs.
MozReview-Commit-ID: E5O4UOu1wLr
--HG--
extra : rebase_source : a0745515957667787929bc5df05a66adb29cbd66
libGL package was consolidated into mesa-libs but quarterly set still
uses the old name. Since gtk3 (via libepoxy) and gtk2 (via cairo)
already indirectly depend on mesa-libs take advantage of it to avoid
churn on updates.
MozReview-Commit-ID: F5LBOOthAMc
--HG--
extra : rebase_source : 4e3e30e817187c3ffef30e280554b12b02f44568
This version offers a 15-20% improvement to compilation
speed, so we will likely require it for developers soon.
Begin installing it now to reduce the frequence of double
migrations.
MozReview-Commit-ID: JZRssgOjkF7
--HG--
extra : rebase_source : 7d2ef10315c570e21d609f713391be3e82a88383
We install the tooltool packages to ${HOME}/.mozbuild by default, and
${HOME} may have spaces on Windows. The arguments to the command-line
options need to be quoted appropriately.
DONTBUILD because NPOTB
We'll need this information to be able to locate mach later, since we
can't assume that mach is just several directories above the script
we're currently running.
If we're running bootstrap from inside a mozilla-central checkout, we'll
have all the necessary files that we need to install tooltool packages
via `mach artifact toolchain`. If we're running bootstrap from a
downloaded bootstrap.py, however, it's possible that we failed to clone
the repository. We need to account for that situation and provide
instructions on how to deal with it.
A consequence of these two mechanisms for bootstrap is that we'll have
to make sure we're invoking `mach artifact toolchain` correctly in both
cases, which we'll handle in subsequent commits.
Now that we're installing Stylo packages through `mach`, we need to have
`mach` available, which is only true after we've checked for a clone of
the tree. We move the block performing the installation in this commit
and then fixup resulting problems in future commits.
Using this command is more robust than our current method, and brings
several benefits, such as smart caching of the downloaded toolchain. We
change the clang package downloaded for Windows with this change, but
bindgen has been updated to work well with LLVM 5.0, so there should be
no problems.
The base bootstrap class has code to conditionally upgrade Mercurial
depending on its version. It is kinda broken. This commit overrides
that code in the MozillaBuild bootstrapper to always run `pip`, which
will ensure the latest stable Mercurial release is installed.
MozReview-Commit-ID: 2O1Ff7dAp4o
--HG--
extra : rebase_source : 98cba0abbebf13e8fe93673d5b46eb16e6a3c147
Previously, `mach bootstrap` would unconditionally prompt to help
configure Mercurial in most scenarios. I agree with Ehsan's observation
in a mailing list post that this behavior doesn't make sense when
running from a Git checkout, as the user probably doesn't care about
Mercurial if they are using Git.
This change doesn't completely ignore Mercurial for Git users. For
example, we still unconditionally run code that verifies that Mercurial
is installed and reasonably up to date. Changing this would be a bit
of work. But even if we wanted to change it, git-cinnabar users
would benefit from having a modern Mercurial installed. So it isn't
straightforward for Git users to ignore Mercurial completely.
MozReview-Commit-ID: 8ncHRgCsjz
--HG--
extra : rebase_source : 7945e3bf3d5283105bac517885f794fc5d7bba6d
Fixes a 403 error running `./mach bootstrap` with stylo enabled.
Bug 1359968 updated the tooltool manifests to point to a
publicly-available clang package. Update the macOS hash
used by mozboot to install the same package for local
use by developers who need rust-bindgen.
MozReview-Commit-ID: FtkATaLwc7n
--HG--
extra : rebase_source : 61390c729e694589b0351026aac0263440a9a3c5
The Python print() function's does not take a format string, so the extraneous %s here inadvertently prints "Downloading clang package from %s https://..." instead of "Downloading clang package from https://...".
MozReview-Commit-ID: 5Is5TK3X34e
--HG--
extra : rebase_source : 02e625fbdaab536d1a787c33edd6dfda9c1039c6
For Stylo development, LLVM packages are required due to Stylo's
extensive use of bindgen--generating Rust bindings to Gecko's C++ code.
While people can install LLVM via their system package manager, we've
opted to download the LLVM packages used on Mozilla infrastructure for
building Gecko. Using Mozilla's packages for LLVM ensures that they
work, and also makes it easier/trivial to integrate support for other
things (e.g. Mozilla's static checkers) into `mach bootstrap`.
When downloading rust manually, it's mandatory to restart the shell,
as the script does not export the PATH environment variable.
This fix also ensures that the rust version in PATH is modern enough for a successful build.
MozReview-Commit-ID: HdLpiLPBLW7
--HG--
extra : rebase_source : dc087c36e89fd6d2957fd1e03af42229de914d5a
The FileNotFoundError built-in exception is only present in
python 3. Emulate its behaviour in python 2 with a conditional
OSError.
MozReview-Commit-ID: 4b8THPG7jph
--HG--
extra : rebase_source : 718bf3659f14bd349d052d43bf3197dfbb4a016f
Bump the minimum version of the rust toolchain we require to
build. The 1.15 release includes support for custom #[derive]
directives, letting us use the serde serialization crate without
checking in a lot of generated code.
This is primarily motivated by webrender and the audio remoting
work, and lets us drop the heavy syntex dependency.
MozReview-Commit-ID: 6IObHhouPAn
--HG--
extra : rebase_source : 4be8b148fb653a48f6df4309811ab1d8755f7edf
If we have rustup installed, use it to check the available
target platforms and install 32-bit windows support if we're
on the (default 64-bit) windows platform.
This catches systems where the mozillabuild bootstrapper was
run before it installed this, so rustup is available, but the
i686 target library isn't.
MozReview-Commit-ID: 9bE2OQnmvxs
--HG--
extra : rebase_source : 0915fef85755718c5524b13b6b7d0d8b6dbe05b4
Windows devs often want to target 32-bit windows. Make this
easier by installing the target for them at mozboot time.
MozReview-Commit-ID: 6gFbFBOqMz8
--HG--
extra : rebase_source : 12933c6cb7895cef859c9022efa87c62f321219e
We need to mangle the path for both parts of the text printed
but rust_path_advice on windows. Otherwise the report of where
the rust binary was found ends up a mixed pathname even when
the suggested shell command is correct.
MozReview-Commit-ID: FDtP5HY8tJ1
--HG--
extra : rebase_source : ad3671be04751ece8966fa19267239b9f1614551
Adjust newlines so prompt and result information is easier to read.
MozReview-Commit-ID: BbJldqZ6G4
--HG--
extra : rebase_source : 50781f672c452e97c47d4ae07aa7f8b67a96ec50
Work around missing redhat-hardened-cc1 when building psutil_linux
on Fedora by installing the redhat-rpm-config package.
This fixed build warnings with several mach commands, and errors
with others like `./mach mochitest` and `./mach install` for fennec
builds.
MozReview-Commit-ID: G9jn4abUEtp
--HG--
extra : rebase_source : 98337b820fff52e2efd2368e89f7ff488b36f1ff
Should be pretty safe as Java compa is usually very good.
This will fix the issue on Debian testing not having openjdk-7-jdk and
current Debian stable having it.
(same with Ubuntu)
MozReview-Commit-ID: Alxg4K4PwQ4
--HG--
extra : rebase_source : 920cdb1618ba587a4776e33ef7857415a59c53b9
On windows, python generally returns windows-style paths,
with drive letters and backslash for a separator. However,
when we offer advice for updating the PATH variable, we're
talking about the msys environment which uses unix-style
paths. Convert to avoid confusion.
This is intended to turn c:\Users\mozilla\ into /c/Users/mozilla/.
MozReview-Commit-ID: FdB8FvjeCV1
--HG--
extra : rebase_source : 6d9e87b460417254bbe2eb5af3813e22f2126fb1
This provides build-completion notification from mach.
We already do this on debian-based distros.
MozReview-Commit-ID: Jl3OWa9MpZ6
--HG--
extra : rebase_source : e97c02d2924f888b33593ad5209cedaccceba633
Update the version number and checksums of the rustup
installer we download to 1.0.0. This had a first stable
release alongside rust 1.15.
This is the result of running `python mozboot/rust.py --update`
and applying the resulting output.
MozReview-Commit-ID: 1gzMLHZuhNx
--HG--
extra : rebase_source : b9d0f95f17e76a32e17e82d05505cf07a21c5e66
It seems older Python (e.g. 2.7 from Ubuntu 14.04) doesn't
support SNI, so we get a TLS error with the canonical
https://static.rust-lang.org/ url even when using the
`requests` module.
Fall back to the no-CNAME host instead which is ugly but works.
Thanks to Simon Sapin for the suggestion.
MozReview-Commit-ID: I6V5ASijuKi
--HG--
extra : rebase_source : 2e2a449fbb3011b51207f1c6baa3d288d0dc774c
On Windows, the rustup installer doesn't create ~/.cargo/env
but instead pokes .cargo/bin into the path in the Windows
registry. This doesn't necessarily propagate to the msys
evironment however, so some PATH manipulation may still
be necessary.
Split our path advice to be clear in both the new install
and unconfigured path cases, and amend our path update
advice to not mention .cargo/env if it isn't present.
MozReview-Commit-ID: 9IHhS6UYCqq
--HG--
extra : rebase_source : 898615106078882f335385ac0b50eff1612377f0
Always report if we found an out-of-date rustc.
This is better when an older rustc is installed, but not rustup,
and it's shadowing the version we install in $CARGO_HOME.
MozReview-Commit-ID: 43io6uISMNI
--HG--
extra : rebase_source : f02e36e0c0c0e5380b3f511852b29a77622165c5
This is needed on Fedora as well as CentOS, at least on Fedora 25
where it's evidently not part of the "GNOME Software Development"
group.
MozReview-Commit-ID: KMW8FUsvJcv
--HG--
extra : rebase_source : 07acb80148409cc1d2918900a56e5d0210c157e1
We were checking for success installing rust with the same code
we checked for upgrade success, but in the case of a clean install
this will likely fail because the binaries installed by rustup
won't be in path. Instead, print the help message about adding
them after installation completes.
MozReview-Commit-ID: xa5PKIDKzZ
WindowsBootstrapper overrides BaseBootstrapper.which() to append
the Windows 'exe' filename extension, so which() finds rustc.exe
and rustup.exe properly. However, our other code which constructs
the program name in _parse_version or looks for the files in
CARGO_HOME didn't take this into account, making the script think
it couldn't find rust.
Don't use os.path.join to construct urls, since on windows this
inserts the backslash '\' path separator instead of the normal
forward slash.
MozReview-Commit-ID: HWJjwCDHuNK
--HG--
extra : rebase_source : c9a295a22c06dcbfa60637ff6d56d6f1ca269e83
Python urllib2 doesn't validate https origins in all versions.
During actual bootstrap the static hash values act as an out-of-bound
validatation channel.
However, that doesn't help when doing the initial download and hash
generation when invoked as `python rust.py [--update]`. Fortunately
we don't expect to be called this way in standalone mode, so we can
use the in-tree requests module to fetch things properly.
MozReview-Commit-ID: KZTtIXDfWTB
--HG--
extra : rebase_source : 14c505797a74de16a7e9bec1f791c0b4659d2932
Reopen sys.stdout in unbuffered mode so we can correctly print
'Checking foo... Result' in two parts without calling sys.stdout.flush()
everywhere.
Although we import print_function from the future, the python 2 version
does not support the python 3 flush=True argument.
MozReview-Commit-ID: SjliWeoSa3
--HG--
extra : rebase_source : e16905ac4045e80060d6f248cae0ec731dd0d1c5
Make the mozboot.rust module invokable as a utility. E.g.
python rust.py --update
When called with the --update option it downloads and generates
checksums for the latest version of the installer on supported
platforms, suitable for updating the values coded in the script.
When invoked without the --update option, it verifies the
current version and checksums against the server.
MozReview-Commit-ID: 2NVFf0ptvbM
--HG--
extra : rebase_source : 5e8b650a9b3c6e1d2b8bfdc90d02faa194f1aa04
Download and run a known-good rustup-init installer for the host
system. Once installed, use it to upgrade the latest toolchain.
NB: I expect the MozillaBuildBootstrapper to run its installer
first, but this will take care of Mac, Linux, and FreeBSD.
MozReview-Commit-ID: BKDm1UcLxQS
--HG--
extra : rebase_source : 4e4489d55ad658166a7e4b20c53185532c041204
Add a check to `mach bootstrap` that a compatible version of
the rust toolchain is installed and in path. Note that we use
a separate minimum version from the one checked at configure
time, defined in build/moz.configure/rust.configure, because
this script may be running stand-alone.
If we don't find `rustc` in PATH, we check for it in CARGO_HOME
and suggest adding that to PATH.
If we don't find `rustc` but find a `rustup`, we call it with
the --version switch and report if that fails. This will detect
the older `rustup.rs` script.
MozReview-Commit-ID: EPfQhLz4Dvo
--HG--
extra : rebase_source : 2ea4acdfbfdc2a436f386eae7bc3cbcbc485aa1b
Also use the abstracted helper method for reading the current
mercurial version too. This changes the regex from what was in
use before, but should work for normal version numbers.
MozReview-Commit-ID: IZfC65Jg6T8
--HG--
extra : rebase_source : d61a73b7b1b438d8c846523e5e1f639950fe5473
Move version parsing to a helper method so it can be used
for more than one executable.
MozReview-Commit-ID: 4gOgdgYFbFx
--HG--
extra : rebase_source : 944f562c0d5a6a105a0c27af6f4d7dfc214f3c01
Previously if an Arch Linux user had a different package extension configured
in `/etc/makepkg.conf` building AUR packages during bootstrap would fail.
Forcing the extension by providing it as an environment variable makes sure
building doesn't fail regarding of a user's configuration.
MozReview-Commit-ID: 4aryYS0XVr7
--HG--
extra : rebase_source : 4c466e49f729de625e814a92325c6d38e6d1e0b4
