MacOS doesn't include openssl any more, but the cargo-vendor
build expects a system version. Special case using a copy
in /usr/local/opt/openssl, where homebrew typically installs it.
MozReview-Commit-ID: DbdnfVdEhWV
--HG--
extra : rebase_source : c793cf239c238f817ba43798e759b8afe0fd5b9c
The sgr0 terminal capabity defines the byte sequence needed to reset
terminal text to its default state. For reasons documented inline in
this commit, we now print this sequence after every line printed by
mach's terminal logger.
MozReview-Commit-ID: 3RukP0QXtqy
--HG--
extra : rebase_source : 5e4b7d001300ec1059b53423b310ac9fdd514c72
This was a mistake from the beginning. I'm removing it now so that I
can easily generate across objdirs. While we transition from
moz.build to Gradle, I want all the build logic to be in
mobile/android/base but the outputs to be split across Gradle project
locations. That's hard to do when generated/ is automatically
prepended to generated_sources paths.
MozReview-Commit-ID: L07ZZBTsNw5
--HG--
extra : rebase_source : 1fbd131f1f28dfd67465f7ecda40985abd530ae0
Make the symlinks in the dist directory relative instead of absolute.
MozReview-Commit-ID: HS7KL4JwSbV
--HG--
extra : rebase_source : 5dca673cc17423d47e6707d8800f7ee9693a9c48
It seems older Python (e.g. 2.7 from Ubuntu 14.04) doesn't
support SNI, so we get a TLS error with the canonical
https://static.rust-lang.org/ url even when using the
`requests` module.
Fall back to the no-CNAME host instead which is ugly but works.
Thanks to Simon Sapin for the suggestion.
MozReview-Commit-ID: I6V5ASijuKi
--HG--
extra : rebase_source : 2e2a449fbb3011b51207f1c6baa3d288d0dc774c
The test results were updated to match current behaviour. The
TestDummyAudioWithTransport and TestDummyVideoWithTransports are disabled due
to shutdown crashes and intermittent failures that show up in automation.
A follow up bug has been filed to fix these. The GMP test was removed
completely as it seems unlikely that it will be practical to test that from a
gtest.
MozReview-Commit-ID: 2pOb7u2Qp7v
--HG--
rename : media/webrtc/signaling/test/mediaconduit_unittests.cpp => media/webrtc/signaling/gtest/mediaconduit_unittests.cpp
extra : rebase_source : 992330f83e0a6a57810f1c5f0b4ea77f2512cd92
This commit also adds an overdue test for plain Rust libraries in the
recursivemake backend, but said test also serves to ensure that we don't
emit features for a library if none were defined in moz.build.
It turns out that, in practice, the LD variable is not used by the build
system, except on Windows, where it's used to feed the default for LINK,
which is then re-injected as LD.
The upcoming changes are going to normalize the use of LD/LINK.
--HG--
extra : rebase_source : 2a1a9924e963e082e119ff3874e8ff24247f4f94
We're going to need this functionality for Rust programs as well as Rust
libraries, so we might as well move the functionality out of RustLibrary
into a separate function.
We were checking for success installing rust with the same code
we checked for upgrade success, but in the case of a clean install
this will likely fail because the binaries installed by rustup
won't be in path. Instead, print the help message about adding
them after installation completes.
MozReview-Commit-ID: xa5PKIDKzZ
Always report if we found an out-of-date rustc.
This is better when an older rustc is installed, but not rustup,
and it's shadowing the version we install in $CARGO_HOME.
MozReview-Commit-ID: 43io6uISMNI
--HG--
extra : rebase_source : f02e36e0c0c0e5380b3f511852b29a77622165c5
The pip function `check_if_exists` returns True if any version of the named
package exists, and sets attributes on the requirement object based on
whether the correct version exists. Prior to this patch, we were interpreting
a `True` result from `check_if_exists` to mean the named package did not need
to be installed or updated at all, causing the build system virtualenv to use
incorrect dependency versions and fail in some cases. This was found in the
case of attempting an artifact build on a system where mozregression was
installed locally, resulting in an incompatible version of the taskcluster
client being used by the artifact code.
MozReview-Commit-ID: 5Q54PsUIrKR
--HG--
extra : rebase_source : f6ae53957401af13cf2eae2780783094f99f8136
This is needed on Fedora as well as CentOS, at least on Fedora 25
where it's evidently not part of the "GNOME Software Development"
group.
MozReview-Commit-ID: KMW8FUsvJcv
--HG--
extra : rebase_source : 07acb80148409cc1d2918900a56e5d0210c157e1
Ideally, we'd shell-quote more arguments, but that would also require
different quoting for the make dependencies and the command line
arguments, so go for the immediate need.
--HG--
extra : rebase_source : c24c782e4faec41ccb18575dce4d0cf513f8649a
While the recursive make backend needs to handle ObjDir*Files separately
for now, other backends can handle them just as FinalTarget*Files instead,
simplifying their implementation.
--HG--
extra : rebase_source : 3201fc755e07489b0f440839f02e4052fb3dff4b
This change is mostly straightforward, except for the following.
- It removes all the printing from the do_check_* macros because gtest macros
do appropriate printing.
- test_StatementCache.cpp needs some special gtest magic for the type
parameterization.
- It merges the four tests in test_unlock_notify.cpp because they rely on being
executed in order, and so aren't independent.
- storage_test_harness_tail.h is no longer necessary because gtest provides the
test looping functionality.
- It uses #include and the preprocessor to remove the duplication between
test_deadlock_detector.cpp and xpcom/tests/DeadlockDetector.cpp.
- It makes the test in test_service_init_background_thread.cpp a death test to
force it to be the first storage gtest, because it fails otherwise.
- It adds code to undo the SQLite mutex hooking as necessary, so that tests
don't interfere with each other.
- It de-virtualizes Spinner's destructor (as identified in bug 1318282).
--HG--
rename : storage/test/storage_test_harness.h => storage/test/gtest/storage_test_harness.h
rename : storage/test/test_AsXXX_helpers.cpp => storage/test/gtest/test_AsXXX_helpers.cpp
rename : storage/test/test_StatementCache.cpp => storage/test/gtest/test_StatementCache.cpp
rename : storage/test/test_asyncStatementExecution_transaction.cpp => storage/test/gtest/test_asyncStatementExecution_transaction.cpp
rename : storage/test/test_async_callbacks_with_spun_event_loops.cpp => storage/test/gtest/test_async_callbacks_with_spun_event_loops.cpp
rename : storage/test/test_binding_params.cpp => storage/test/gtest/test_binding_params.cpp
rename : storage/test/test_deadlock_detector.cpp => storage/test/gtest/test_deadlock_detector.cpp
rename : storage/test/test_file_perms.cpp => storage/test/gtest/test_file_perms.cpp
rename : storage/test/test_mutex.cpp => storage/test/gtest/test_mutex.cpp
rename : storage/test/test_service_init_background_thread.cpp => storage/test/gtest/test_service_init_background_thread.cpp
rename : storage/test/test_statement_scoper.cpp => storage/test/gtest/test_statement_scoper.cpp
rename : storage/test/test_transaction_helper.cpp => storage/test/gtest/test_transaction_helper.cpp
rename : storage/test/test_true_async.cpp => storage/test/gtest/test_true_async.cpp
rename : storage/test/test_unlock_notify.cpp => storage/test/gtest/test_unlock_notify.cpp
extra : rebase_source : dbb695c112564efa1945116be1a8435988982e74
The current way they are generated makes it so that it's not trivial for
the backends to figure that one depends on the other. While we should
eventually make things such that using FINAL_TARGET_PP_FILES works for
that, it's currently simpler to just use GENERATED_FILES.
--HG--
rename : build/application.ini => build/application.ini.in
extra : rebase_source : cad3460b65d99bdae3e6f1d9dd611e0a602ebfed
WindowsBootstrapper overrides BaseBootstrapper.which() to append
the Windows 'exe' filename extension, so which() finds rustc.exe
and rustup.exe properly. However, our other code which constructs
the program name in _parse_version or looks for the files in
CARGO_HOME didn't take this into account, making the script think
it couldn't find rust.
Don't use os.path.join to construct urls, since on windows this
inserts the backslash '\' path separator instead of the normal
forward slash.
MozReview-Commit-ID: HWJjwCDHuNK
--HG--
extra : rebase_source : c9a295a22c06dcbfa60637ff6d56d6f1ca269e83
Python urllib2 doesn't validate https origins in all versions.
During actual bootstrap the static hash values act as an out-of-bound
validatation channel.
However, that doesn't help when doing the initial download and hash
generation when invoked as `python rust.py [--update]`. Fortunately
we don't expect to be called this way in standalone mode, so we can
use the in-tree requests module to fetch things properly.
MozReview-Commit-ID: KZTtIXDfWTB
--HG--
extra : rebase_source : 14c505797a74de16a7e9bec1f791c0b4659d2932
Reopen sys.stdout in unbuffered mode so we can correctly print
'Checking foo... Result' in two parts without calling sys.stdout.flush()
everywhere.
Although we import print_function from the future, the python 2 version
does not support the python 3 flush=True argument.
MozReview-Commit-ID: SjliWeoSa3
--HG--
extra : rebase_source : e16905ac4045e80060d6f248cae0ec731dd0d1c5
Make the mozboot.rust module invokable as a utility. E.g.
python rust.py --update
When called with the --update option it downloads and generates
checksums for the latest version of the installer on supported
platforms, suitable for updating the values coded in the script.
When invoked without the --update option, it verifies the
current version and checksums against the server.
MozReview-Commit-ID: 2NVFf0ptvbM
--HG--
extra : rebase_source : 5e8b650a9b3c6e1d2b8bfdc90d02faa194f1aa04
Download and run a known-good rustup-init installer for the host
system. Once installed, use it to upgrade the latest toolchain.
NB: I expect the MozillaBuildBootstrapper to run its installer
first, but this will take care of Mac, Linux, and FreeBSD.
MozReview-Commit-ID: BKDm1UcLxQS
--HG--
extra : rebase_source : 4e4489d55ad658166a7e4b20c53185532c041204
Add a check to `mach bootstrap` that a compatible version of
the rust toolchain is installed and in path. Note that we use
a separate minimum version from the one checked at configure
time, defined in build/moz.configure/rust.configure, because
this script may be running stand-alone.
If we don't find `rustc` in PATH, we check for it in CARGO_HOME
and suggest adding that to PATH.
If we don't find `rustc` but find a `rustup`, we call it with
the --version switch and report if that fails. This will detect
the older `rustup.rs` script.
MozReview-Commit-ID: EPfQhLz4Dvo
--HG--
extra : rebase_source : 2ea4acdfbfdc2a436f386eae7bc3cbcbc485aa1b
Also use the abstracted helper method for reading the current
mercurial version too. This changes the regex from what was in
use before, but should work for normal version numbers.
MozReview-Commit-ID: IZfC65Jg6T8
--HG--
extra : rebase_source : d61a73b7b1b438d8c846523e5e1f639950fe5473
Move version parsing to a helper method so it can be used
for more than one executable.
MozReview-Commit-ID: 4gOgdgYFbFx
--HG--
extra : rebase_source : 944f562c0d5a6a105a0c27af6f4d7dfc214f3c01
This will allow tests, etc to conditionally run depending on Stylo's
presence.
MozReview-Commit-ID: 3WHxNawP1qC
--HG--
extra : rebase_source : d4d983d37d3bfac1f9d465a3023b5bd5df52cd56
extra : source : 6baa88f7b1f4183ea57b9f8928556975042cad29
The build system's TestResolver does a pretty good job of getting the right manifestparser
based tests to run, but it isn't perfect. Notably, it ignores the 'disabled' key. We filter
the tests through manifestparser here to make sure the build system didn't miss anything.
For context, this is also what the other test harnesses (e.g mochitest) do as well.
MozReview-Commit-ID: FaHb4nvuoK9
--HG--
extra : rebase_source : 476b7068c3ddf7d9757d605de5843e21547c6c33
This deprecates PYTHON_UNIT_TESTS and replaces it with PYTHON_UNITTEST_MANIFESTS.
In the build system, this means python unittests will be treated the same as all
other test suites that use manifestparser. New manifests called 'python.ini' have
been created for all test directories containing python unittests.
MozReview-Commit-ID: IBHG7Thif2D
--HG--
extra : rebase_source : 11a92a2bc544d067946bbd774975140147458caa
This fixes a UnicodeDecodeError when sys.stdout's encoding can't handle unicode.
MozReview-Commit-ID: 3INna8MRje5
--HG--
extra : rebase_source : bb8f3b5cbdbb6601b6946d5fbb7396848f22b5c6
This patch does a few things:
1) Change all the in-tree tooltool manifests to contain sccache2 instead of the existing Python sccache
2) Change mozconfig.cache to point at sccache.
3) Lightly tweak the --with-cccache configure option to support sccache, and detect whether we're using ccache or sccache and set an option appropriately.
4) Add a MOZ_SCCACHE_VERBOSE_STATS option, and add a target in the top-level Makefile to make sccache spit out its stats at the end of the build. This is useful to see the cache hits/errors until we get something better.
5) Add MOZ_USING_SCCACHE to the build telemetry. Not that I think it will be immediately useful, but for future use.
MozReview-Commit-ID: 9lrdLwNj5Bm
--HG--
extra : rebase_source : d323457df10d0ee0ac5811940e518d9422a7e070
In order for VS2017 to recognize solution and project files as
belonging to VS2017 and not some earlier version, we need to bump
a few versions strings.
The added code path can't be hit yet because MSVS_VERSION cannot
be "2017" until configure detects VS2017. I hacked up MSVS_VERSION
locally and verified that solutions produced with the new code load
in VS2017.
FWIW, the solution loads significantly faster in VS2017 compared to
VS2015!
MozReview-Commit-ID: 2vYAgd6wOsG
--HG--
extra : rebase_source : a29b956bb6eef9071cb46d30460a42ef17658026
This replaces the "return_code" property on the LintRoller object with a list of "failed"
linters. This is a bit more useful as it lets us know exactly which linters had a problem
(whereas previously we just knew *something* went wrong). This patch pushes determining
the return code back into cli.py, which I think is fine.
In addition, we now pass the list of failed linters into the formatter. This allows us to
clarify exactly how many linters hit a failure which is a lot better than a seemingly
"successful" summary message.
Finally I also removed the "no files to lint" message because I've seen several people
confuse it for an error. I'll probably add it back as a debug log message when we switch
to using mozlog for output.
MozReview-Commit-ID: 4wyCeOZdOf8
--HG--
extra : rebase_source : 23810a8ab8dd9cbbad6b9e965ccff7214f947fbc
This patch contains a number of changes to the gyp_reader code:
* Add three new flags to GYP_DIRS:
** no_chromium, to skip forcing the includes/etc needed for chromium gyp files
** no_unified, to force building all sources without unification
** action_overrides, to pass scripts used when mapping gyp actions to moz.build GENERATED_FILES
* Handle the flags mentioned above in read_from_gyp
* Handle actions in gyp targets by mapping them to GENERATED_FILES, using scripts specified in the action_overrides flag. We don't try to handle the generic action case, we require special-casing for each action.
* Handle a subset of copies in gyp targets by mapping them to EXPORTS, just enough to handle the use of them for NSS exports.
* Handle shared_library and executable gyp targets
* Handle gyp target dependencies/libraries as USE_LIBS/OS_LIBS
* Handle generated source files
* Handle .def files in sources by mapping them to SYMBOLS_FILE
* Special-case some include_dirs:
** Map `<(PRODUCT_DIR)/dist/` to $DIST/include (to handle include paths for NSS exports)
** Map include_dirs starting with topobjdir to objdir-relative paths, to handle passing the NSPR include path to NSS
* split /build/gyp.mozbuild into two parts, with gyp_base.mozbuild containing generic bits, and gyp.mozbuild containing chromium-specific bits
MozReview-Commit-ID: FbDmlqDjRp4
--HG--
extra : rebase_source : d3fb470c589f92d8c956b9ecd550fb8df79ff5bc
We recently switched make check to call into |mach python-test| rather than invoking python
itself for each test file. But this ended up slowing down the tests as they were no longer
being run in parallel. This patch adds a --jobs flag to python-tests and runs test files in
parallel.
Note: if more than one job is used, output per test will be buffered and printed at the end
to avoid interleaving. This has the unfortunate side effect of making |mach python-test| look
like it is hanging, especially if running a very long file like mozbase's test.py. For this
reason, we still use -j1 by default so output will continue to be streamed. In automation we
will use multiple processes though.
MozReview-Commit-ID: 3u0wOFmyQLI
--HG--
extra : rebase_source : d08ac412023731c46226c7adbf5f6e798b9a345a
We need the fix from https://github.com/agronholm/pythonfutures/issues/25
to allow sending KeyboardInterrupts to thread pools.
MozReview-Commit-ID: 5VfBttLbKOr
--HG--
extra : rebase_source : 8e3aa7a1d6fbbaa7b94cea35b196b35e103a1e33
Some mach commands may want to re-use a requirements.txt file rather than installing packages
individually. This enables --require-hashes which means all packages and dependencies must be
listed with their hashes. For more details, see:
https://pip.pypa.io/en/stable/reference/pip_install/#hash-checking-mode
MozReview-Commit-ID: 3lOutbcSzIY
--HG--
extra : rebase_source : d07ac21bd1f2f0009465f9e004208464b22de01b
This is a most minimal gtest conversion possible. It leaves in place
significant amounts of non-typical-for-gtest code.
Notable changes:
- All the mock Link and URLSearchParams method definitions are no longer
needed.
- The changes adds a new constructor for Link that doesn't set mHistory.
Without that, leaked URLs occur at shutdown.
- The output printed by the test is slightly streamlined, mostly by omitting
the test filename.
- It disables TestMediaFormatReader.cpp, which was causing problems. That test
is slated for removal in bug 1318225 anyway.
--HG--
rename : toolkit/components/places/tests/cpp/mock_Link.h => toolkit/components/places/tests/gtest/mock_Link.h
rename : toolkit/components/places/tests/cpp/moz.build => toolkit/components/places/tests/gtest/moz.build
rename : toolkit/components/places/tests/cpp/places_test_harness.h => toolkit/components/places/tests/gtest/places_test_harness.h
rename : toolkit/components/places/tests/cpp/places_test_harness_tail.h => toolkit/components/places/tests/gtest/places_test_harness_tail.h
rename : toolkit/components/places/tests/cpp/test_IHistory.cpp => toolkit/components/places/tests/gtest/test_IHistory.cpp
extra : rebase_source : b7def3f9afce3a44e99f5ed35cb220f7814551cd
On Windows, the rustup installer doesn't create ~/.cargo/env
but instead pokes .cargo/bin into the path in the Windows
registry. This doesn't necessarily propagate to the msys
evironment however, so some PATH manipulation may still
be necessary.
Split our path advice to be clear in both the new install
and unconfigured path cases, and amend our path update
advice to not mention .cargo/env if it isn't present.
MozReview-Commit-ID: 9IHhS6UYCqq
--HG--
extra : rebase_source : 898615106078882f335385ac0b50eff1612377f0
The current state of python configure sandbox execution is that if a
template imports a module, and a function defined in the template tries
to use the module, it doesn't work. Ideally, it would, but rather than
try to fix this corner case, we remove the unit tests that assume it
works (and consequently pass for half bad reasons), and add a unit test
so that the behavior doesn't change unwillingly.
--HG--
extra : rebase_source : 579ba2bc7c19d4fe7df11bbdb1ceb6171a1ee857
The way it works now, `mach` commands often invoke subprocesses where
the subprocesses' stdio file descriptors are pipes so the mach command
can e.g. parse output.
Processes like clang, gcc, and cargo determine if they can send color
codes to {stderr, stdout} by seeing if those file descriptors are TTYs.
When e.g. `make` is executed via `mach`, this test fails because those
descriptors are pipes (even though they eventually end up on a TTY).
We can't wire the file descriptors to the TTY because `mach` needs
to analyze output. We don't want users defining process flags to force
color in their mozconfigs because color codes would still be sent
if stdout was not a TTY.
This patch sets the MACH_STDOUT_ISATTY environment variable in all mach
commands when stdout is a TTY. Subsequent processes can then look for
this variable to determine whether to override color settings, print
terminal control codes, etc.
MozReview-Commit-ID: GxXP2mQssjC
--HG--
extra : rebase_source : 4b99547b453cb7dd5cb590a71ed554ce2bc4759d
Currently, environment variables set when running mach commands will
propagate after the command is finished. This can allow unwanted state
to bleed through.
This likely isn't an issue today, but isolating state during code
execution is generally a good practice. So do that.
MozReview-Commit-ID: AdaomGub5EF
--HG--
extra : rebase_source : ce81987a1f6de3a16bce6a9e45b9dc8e8eb29b4b
Also allow when=True/False to avoid the chicken-egg problem of using
a generic `when` to use in replacement of @depends('--help') for things
like @dependable.
--HG--
extra : rebase_source : f1571a5b904efb66a361b90f3b7e1edbaa75772e
--help dependencies currently help identify functions that will run when
running configure --help, which we don't want to have spreading too
much. OTOH, when such functions have no side effect, it's not really
that important to have them explicitly marked.
So, allow missing --help dependencies for functions that:
- don't use @imports
- don't have a closure
- don't use global variables
This is a first step towards entirely removing the --help markings (the
end goal being that --help dependencies will indicate actual --help
dependencies). As such, we don't really care about updating the lint
error message.
--HG--
extra : rebase_source : e81ec9b51ff01c2ee75722904e551286aa0b2bec
We want functions without an @imports to not have any side effects, and
to not use external resources. So remove the few functions we expose from
os.path without @imports('os') that do.
--HG--
extra : rebase_source : a9485ec269d4de5785d66d7772eda4fae5a84b4a
Missing such dependencies shouldn't impair running configure itself
after local modifications, but they are currently required for
(mostly) documentation purpose. Which means they are better done in
the linter.
--HG--
extra : rebase_source : 6bfff2342cda2ed1351f561c9eb9623f1fb4e4c4
Previously if an Arch Linux user had a different package extension configured
in `/etc/makepkg.conf` building AUR packages during bootstrap would fail.
Forcing the extension by providing it as an environment variable makes sure
building doesn't fail regarding of a user's configuration.
MozReview-Commit-ID: 4aryYS0XVr7
--HG--
extra : rebase_source : 4c466e49f729de625e814a92325c6d38e6d1e0b4
Both BuildMonitor and BuildOutputManager are doing similar things and
wrapping the build with forms of monitoring. These classes are only
used in the `mach build` command.
Since the BuildOutputManager takes an instance of BuildMonitor as an
argument and since BuildOutputManager behaves as a context manager,
it makes sense for BuildOutputManager to control the lifetime of
BuildMonitor's collection.
So, we teach BuildOutputManager's context manager to start the
BuildMonitor and ensure its resource collection is stopped when it
exits.
MozReview-Commit-ID: 3l9Hwz0Xe7o
--HG--
extra : rebase_source : 600b0231fce050465bdccb2bc11680f77bbc35d9
r?gps for the build changes and ochameau for the rest.
This results in about a 28% speed-up for Jetpack mochitest runs, for me.
MozReview-Commit-ID: K30q7BfvTLs
--HG--
extra : rebase_source : 8074947062c73a329f1d8a255a9601147ba1ba0c
For devtools-html, devtools will progressively stop using DTD files and will use only properties
files. To avoid retranslating already localized strings in every locale, this script can be used
by localization teams to automatically migrate strings moved in the scope of the devtools-html
project.
MozReview-Commit-ID: KNDfCoPXOM9
--HG--
extra : rebase_source : 48fbf2ae3daab89891262e460c979b6c141a2caa