Commit Graph

5182 Commits

Author SHA1 Message Date
Chris Manchester
0bb09fc35d Bug 1578596 - Require rustc 1.37. r=firefox-build-system-reviewers,mshal
Differential Revision: https://phabricator.services.mozilla.com/D44699

--HG--
extra : moz-landing-system : lando
2019-09-04 19:08:28 +00:00
Andrew Halberstadt
b491b45f70 Bug 1577517 - [mozbuild] Fix bug in error handling case when 'terminal-notifier' is not installed, r=firefox-build-system-reviewers,mshal
Differential Revision: https://phabricator.services.mozilla.com/D44156

--HG--
extra : moz-landing-system : lando
2019-09-04 17:50:56 +00:00
Andrew Halberstadt
41d9e6e547 Bug 1578198 - [configure] Ensure wrapped_subprocess uses text env with Python 3 r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D44483

--HG--
extra : moz-landing-system : lando
2019-09-04 02:37:48 +00:00
Francesco Lodolo (:flod)
1c3c190bfe Bug 1578163 - Remove Fluent migration recipes for Firefox 69 r=Pike
Differential Revision: https://phabricator.services.mozilla.com/D44357

--HG--
extra : moz-landing-system : lando
2019-09-03 07:24:32 +00:00
Andrew Halberstadt
8c97bc7a17 Bug 1577908 - [mach] Disable suggested command execution until Python 3 migration is finished, r=glandium
The mach driver will now run all misspelled commands with Python 3. That means
we can't automatically execute the suggested command anymore, as it may need to
run against Python 2.

Ideally we could figure out a way to check the command against the 'mach'
whitelist, but until then, let's just disable automatic execution. Worst case
scenario we can turn it back on after the migration has finished.

Differential Revision: https://phabricator.services.mozilla.com/D44282

--HG--
extra : moz-landing-system : lando
2019-09-03 03:46:16 +00:00
Andrew Halberstadt
23aca97aa6 Bug 1577501 - [mach] Make sure ./mach help <command> uses the correct Python, r=glandium
Some commands use external argument parsers, so invoking |mach help <command>| will import
external modules (which may only be Python 2 compatible).

This makes sure that we detect the actual subcommand we're generating help for
and use the proper Python.

A much simpler solution would have been to run |mach help| with Python 2 all
the time. However, as we convert things to Python 3 this would have meant that
Python 3 only code would blow up. This would have forced us to continue
supporting Python 2, even for Python 3-only commands.

Differential Revision: https://phabricator.services.mozilla.com/D43989

--HG--
extra : moz-landing-system : lando
2019-09-03 00:39:22 +00:00
Andrew Halberstadt
f2210ba1ed Bug 1568638 - [build] Create objdir in test_manifest backend if it doesn't exist r=mshal
This fixes an edge case where a task needs to invoke the TestManifest backend from a source dir that doesn't have an objdir created yet.

Depends on D43513

Differential Revision: https://phabricator.services.mozilla.com/D43514

--HG--
extra : moz-landing-system : lando
2019-08-30 17:01:12 +00:00
Edwin Gao
a0497df342 Bug 1577603 - convert mach clobber to run with python 3.5 r=nalexander
Changes:
- remove binary encoding when the subdirectories are being collected
- do not open file in binary mode when reading from `telemetry_client_id.json`

Differential Revision: https://phabricator.services.mozilla.com/D44057

--HG--
extra : moz-landing-system : lando
2019-08-30 17:25:47 +00:00
Nick Thomas
f03bae30af Bug 1577664 - pass product version to the complete mar generation script, r=Callek
MOZ_FULL_PRODUCT_VERSION is only used here, while tools/update-packaging/make_full_update.sh expects MOZ_PRODUCT_VERSION.

Depends on D44089

Differential Revision: https://phabricator.services.mozilla.com/D44090

--HG--
extra : moz-landing-system : lando
2019-08-30 01:35:08 +00:00
Mark Striemer
9aff921500 Bug 1533863 - Use icons instead of text for buttons in about:config r=fluent-reviewers,jaws,flod
Differential Revision: https://phabricator.services.mozilla.com/D43627

--HG--
rename : browser/components/aboutlogins/content/icons/edit.svg => browser/themes/shared/icons/edit.svg
extra : moz-landing-system : lando
2019-08-29 21:03:13 +00:00
Dan Minor
8d11bb48b4 Bug 1554976 - Allow network functions needed by Rust mDNS responder; r=tjr
This removes these functions: bind, getaddrinfo, recvfrom, sendto, setsockopt,
socket from the check_networking test to allow for their use by the Rust mDNS
responder.

Differential Revision: https://phabricator.services.mozilla.com/D38488

--HG--
extra : moz-landing-system : lando
2019-08-28 19:48:38 +00:00
Tom Ritter
004d9a25d7 Bug 1570597 - Pass the -delayload flag to lld correctly for MinGW builds r=firefox-build-system-reviewers,mshal
Differential Revision: https://phabricator.services.mozilla.com/D43455

--HG--
extra : moz-landing-system : lando
2019-08-28 14:18:29 +00:00
Chris Manchester
1dcf500d9c Bug 1577049 - Provide sccache and dist-toolchains during bootstrap on Windows. r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D43700

--HG--
extra : moz-landing-system : lando
2019-08-28 18:31:23 +00:00
Gurzau Raul
a6c5c5e3ac Backed out 14 changesets (bug 1554976) for Windows build bustages on a CLOSED TREE.
Backed out changeset a9b209d9d880 (bug 1554976)
Backed out changeset 27b4dddf9597 (bug 1554976)
Backed out changeset 3e16c10bb966 (bug 1554976)
Backed out changeset 6a404fca61dc (bug 1554976)
Backed out changeset 77c4e76c8130 (bug 1554976)
Backed out changeset 24f146b86cc4 (bug 1554976)
Backed out changeset b49b4326dcfd (bug 1554976)
Backed out changeset e2c837d1e0a0 (bug 1554976)
Backed out changeset 24728144c263 (bug 1554976)
Backed out changeset a099e69241a0 (bug 1554976)
Backed out changeset 0e34595c2680 (bug 1554976)
Backed out changeset a506bb40047e (bug 1554976)
Backed out changeset 513026415092 (bug 1554976)
Backed out changeset e0fc6a1d4332 (bug 1554976)
2019-08-28 18:10:11 +03:00
Dan Minor
023252b4a9 Bug 1554976 - Allow network functions needed by Rust mDNS responder; r=tjr
This removes these functions: bind, getaddrinfo, recvfrom, sendto, setsockopt,
socket from the check_networking test to allow for their use by the Rust mDNS
responder.

Differential Revision: https://phabricator.services.mozilla.com/D38488

--HG--
extra : moz-landing-system : lando
2019-08-28 13:09:50 +00:00
Māris Fogels
a445a87248 Bug 1210157 - Fix Python 3 issues in mozboot.gentoo and mozboot.archlinux r=firefox-build-system-reviewers,mshal
Fix some library calls and syntax that are required to support Python 3.

Depends on D39364

Differential Revision: https://phabricator.services.mozilla.com/D39365

--HG--
extra : moz-landing-system : lando
2019-08-20 21:19:18 +00:00
Māris Fogels
9e273f1008 Bug 1210157 - Make bootstrap.py work with Python 3.5+ and Python 2.7 r=firefox-build-system-reviewers,mshal
Make bootstrap.py compatible with both Python 3.5+ and Python 2.7. Remove
Python 2.6 support as Python 2.6 is no longer supported in the Firefox source
tree.

Differential Revision: https://phabricator.services.mozilla.com/D39364

--HG--
extra : moz-landing-system : lando
2019-08-20 21:19:18 +00:00
Māris Fogels
6aec656a0f Bug 1210157 - Add a --debug option to bootstrap.py r=nalexander
Add an option to run bootstrap.py with '--debug'. This will print full
tracebacks if the bootstrap process encounters an uncaught exception. It should
be useful for developers who are working on the bootstrap code as well as users
when writing bug reports.

Depends on D39362

Differential Revision: https://phabricator.services.mozilla.com/D39363

--HG--
extra : moz-landing-system : lando
2019-08-20 21:19:17 +00:00
Māris Fogels
bc36ca8edd Bug 1569046 - Add Python 3 support to mozboot.osx r=firefox-build-system-reviewers,mshal
Make the mozboot.osx module work with Python 3 in addition to
Python 2.7.

Differential Revision: https://phabricator.services.mozilla.com/D39560

--HG--
extra : moz-landing-system : lando
2019-08-20 21:19:48 +00:00
Māris Fogels
9399eb1a1a Bug 1210157 - Add Python 3 support to mozboot Android toolchain bootstrap r=nalexander
Support installing the android development toolchain with Python 3 as well as
Python 2.7.

Depends on D39361

Differential Revision: https://phabricator.services.mozilla.com/D39362

--HG--
extra : moz-landing-system : lando
2019-08-20 21:19:17 +00:00
Māris Fogels
6cc808b85a Bug 1210157 - Add Python 3 support to the mozboot Debian bootstrapper r=firefox-build-system-reviewers,mshal
Add support for Python 3 and Python 2.7 to the Debian-based linux distro
bootstrap routines.

Differential Revision: https://phabricator.services.mozilla.com/D39361

--HG--
extra : moz-landing-system : lando
2019-08-20 21:19:17 +00:00
Māris Fogels
fef8423731 Bug 1210157 - Add Python 3 support to mozboot.base and mozboot.bootstrap r=firefox-build-system-reviewers,mshal
Add support for both Python 3 and Python 2.7 to the mozboot.base and
mozboot.bootstrap modules.  Remove legacy Python 2.6 code or mark it for later
removal.

Depends on D39359

Differential Revision: https://phabricator.services.mozilla.com/D39360

--HG--
extra : moz-landing-system : lando
2019-08-22 22:42:50 +00:00
Māris Fogels
15db7ec0c8 Bug 1210157 - Use unicode_literals in all mozboot modules r=firefox-build-system-reviewers,mshal
Add unicode_literals to all mozboot module __future__ statements to support
running the modules under Python 3.  Remove comments about unicode_literals and
Python 2.6 support as Python 2.6 is no longer supported in tree.

Differential Revision: https://phabricator.services.mozilla.com/D39359

--HG--
extra : moz-landing-system : lando
2019-08-20 21:19:16 +00:00
Andrew Halberstadt
333ffc4bcf Bug 1473498 - [mach] Support running mach commands with python 3 r=glandium,mars
Credit: mars for making the shell POSIX compliant

This embeds a blacklist of every mach command that needs to run with Python 2
directly in the mach driver itself. Initially this is every mach command. We
then use a bit of shell to determine whether the command being run needs Python
2 or 3.

While this approach may seem a bit hacky, it has several benefits:

1. No need to add complex machinery in mach's registration code.
2. No need to spawn two separate Python interpreters in the event a different
   Python from the original interpreter is needed.
3. Perf impact is negligible.
4. New commands are Python 3 by default.

It is also only a temporary hack. Once all commands are running with Python 3,
we can revert back to the original mach driver.

Differential Revision: https://phabricator.services.mozilla.com/D36103

--HG--
extra : moz-landing-system : lando
2019-08-27 03:08:08 +00:00
Andrew Halberstadt
ec79d89147 Bug 1473498 - [mozbuild.util] Don't use 'patch_main' hack with Python 3.4+ r=firefox-build-system-reviewers,mshal
Differential Revision: https://phabricator.services.mozilla.com/D42016

--HG--
extra : moz-landing-system : lando
2019-08-27 02:55:18 +00:00
Andrew Halberstadt
8710cf7cfc Bug 1473498 - [mozbuild.base] Fix Python 3 incompatibilities in mozconfig error handling, r=glandium
Use of 'BaseException.message` was deprecated in Python 2.6 (and removed in
Python 3). We should rely on the exception's '__str__' instead.

We also need to ensure the mozconfig subprocess' output is text when formatting
it into the message with Py3.

Differential Revision: https://phabricator.services.mozilla.com/D42843

--HG--
extra : moz-landing-system : lando
2019-08-27 03:05:59 +00:00
Andrew Halberstadt
9ec9b5b7ab Bug 1473498 - [mozbuild.mozconfig] Improve error message in MozconfigLoadException r=glandium
Without the output, this exception wasn't being very helpful.

Differential Revision: https://phabricator.services.mozilla.com/D42015

--HG--
extra : moz-landing-system : lando
2019-08-27 03:03:49 +00:00
Mike Hommey
8afaba2056 Bug 1575760 - Make mach vendor rust create a .cargo/config and check it in the tree. r=nalexander
Maybe back when .cargo/config.in was added, the directory indicated for
vendored crates needed to be absolute. That is at least not the case
with the current supported versions of rust.

The current setup has a few caveats:
- .cargo/config.in has shown to become stale (it currently contains
  multiple unused entries)
- non-gecko build tasks have to generate a .cargo/config on their own if
  they want to use vendored crates
- in turn, non-gecko build tasks that don't, may unknowingly get their
  dependencies from crates.io (see the recent attempt at moving
  geckodriver builds to a separate task).

By checking in a .cargo/config file, we can alleviate the last two, but
that comes at the price of `cargo update` not wanting to act when
.cargo/config exists, because of the source replacement configuration.

But rust vendor gently generates a suitable configuration on its own, so
we can use that to generate a .cargo/config automatically. Which
addresses the first caveat of the current setup. That leaves us with
`cargo update` not working out of the box, but that just requires people
running it to manually remove .cargo/config first. Which is arguably
what rust wants you to do in the first place. It's kind of incidental
that we started with a .cargo/config.in rather than .cargo/config.

Now, while a simple .cargo/config works, that's not enough for the case
where the objdir doesn't live inside the source directory. In that case
cargo looks for the configuration from the objdir, and fails to find it.
So we still need a .cargo/config.in, which we generate with a little
trick.

Differential Revision: https://phabricator.services.mozilla.com/D43012

--HG--
rename : .cargo/config.in => .cargo/config
extra : moz-landing-system : lando
2019-08-26 22:20:32 +00:00
Mike Hommey
2c49da7d67 Bug 1575760 - Generate .cargo/config as part of the build rather than as a configure subst file. r=firefox-build-system-reviewers,chmanchester
We'll require preprocessing that configure subst files don't allow in
the next change, so prepare for that.

Differential Revision: https://phabricator.services.mozilla.com/D43011

--HG--
extra : moz-landing-system : lando
2019-08-22 21:27:24 +00:00
Geoff Brown
7ba1b41441 Bug 1575667 - Add --no-install option for android mach reftest and gtest; r=bc
Straight-forward addition of the --no-install option for reftest, just like
the existing option for mochitest. For gtest, I alos noticed the mach command
help needed some cleanup.

Differential Revision: https://phabricator.services.mozilla.com/D43299

--HG--
extra : moz-landing-system : lando
2019-08-26 13:45:10 +00:00
Andi-Bogdan Postelnicu
7b5552dee0 Bug 1575659 - Fixed crashing of mach static-analisys when no arguments where passed. r=bastien
Differential Revision: https://phabricator.services.mozilla.com/D43393

--HG--
extra : moz-landing-system : lando
2019-08-26 08:57:49 +00:00
Mike Hommey
aff45da04a Bug 1576097 - Remove mozbuild.util.encode again. r=nalexander
This time, existing config.status trying to import it will throw an exception
that will trigger a re-configure.

Differential Revision: https://phabricator.services.mozilla.com/D43215

--HG--
extra : moz-landing-system : lando
2019-08-23 16:43:58 +00:00
Mike Hommey
9f1da4f63d Bug 1576097 - Deal with config.status breakage more gracefully. r=nalexander
As seen in bug 1575774/1575959, things can go badly when config.status
doesn't match expectations, especially when most mach commands try to
read it, starting with mach clobber, which is supposed to be the one
commant to get away from most problems.

The idea here is to throw a recognizable exception so that callers can
react accordingly. While not obvious from the patch, the result is that
running e.g. mach build with a broken config.status will automatically
run configure (because the relevant caller actually handles the rethrown
exception that way).

There are other calls to from_config_status in the mozbuild reader, but
that can't be reached before MozbuildObject.config_environment.

Differential Revision: https://phabricator.services.mozilla.com/D43214

--HG--
extra : moz-landing-system : lando
2019-08-23 22:26:02 +00:00
Dustin J. Mitchell
23d547abce Bug 1562686 - revert remaining unnecessary bit of bug 1187245; r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D42644

--HG--
extra : moz-landing-system : lando
2019-08-23 12:32:02 +00:00
Mihai Alexandru Michis
aeeeb2bb05 Merge mozilla-central to autoland. CLOSED TREE
--HG--
extra : rebase_source : b8f2ddf1c4cafdaa69093d306f4f3922bfd2acbd
extra : amend_source : e1568da3f97fd327d91a646913a3d7ab5241638a
2019-08-23 01:14:01 +03:00
Mike Hommey
720d497cc2 Bug 1575959 - Restore the mozbuild.util.encode function. a=glandium
Older config.status laying around in old trees are read by mach whenever
it runs, including when running mach clobber. Those files still try to
include mozbuild.util.encode, which is not here anymore. So we restore
the function for now to unbreak those.

Differential Revision: https://phabricator.services.mozilla.com//D43132

--HG--
extra : histedit_source : 3eb10cc8e18cc9094887061b00705afb6e705b54
2019-08-22 23:58:00 +03:00
Mike Hommey
f7da81f01b Bug 1574056 - Make mach vendor rust use now native cargo vendor. r=chmanchester
Note that cargo vendor always uses the native version even when
cargo-vendor was already installed, so there is no concern wrt that.

Differential Revision: https://phabricator.services.mozilla.com/D43007

--HG--
extra : moz-landing-system : lando
2019-08-22 21:46:40 +00:00
Mike Hommey
20f01277a6 Bug 1575375 - Pull mozbuild tests out of make check. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D42759

--HG--
extra : moz-landing-system : lando
2019-08-21 05:00:50 +00:00
Mike Hommey
bfa5918bf6 Bug 1575375 - Always include mozbuild/mozpack tests. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D42757

--HG--
extra : moz-landing-system : lando
2019-08-21 03:05:09 +00:00
Mike Hommey
6b7c48eb98 Bug 1575375 - Allow nodeutil to find node on its own on automation. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D42756

--HG--
extra : moz-landing-system : lando
2019-08-21 03:18:55 +00:00
Mike Hommey
ab23d101a0 Bug 1575375 - Use MozbuildObject.resolve_config_guess instead of invoking config.guess manually in test_base. r=nalexander
Bug 1522788 changed how MozbuildObject gets a value for @CONFIG_GUESS@
in mozconfigs, and it's not entirely reliable to manually call
config.guess in test_base. As a matter of fact, in some cases,
config.guess is not even invoked by MozbuildObject (like, on Windows).

Plus, the config.guess invocation in the test currently doesn't work on
Windows on automation, for some reason.

Differential Revision: https://phabricator.services.mozilla.com/D42755

--HG--
extra : moz-landing-system : lando
2019-08-21 03:02:29 +00:00
Mike Hommey
7af747540d Bug 1575375 - Make test_config_environment work without an objdir. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D42754

--HG--
extra : moz-landing-system : lando
2019-08-21 03:00:11 +00:00
Mike Hommey
843c5016d9 Bug 1575375 - Make mozpack.path.normsep work with both bytes and unicode strings in python 3. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D42753

--HG--
extra : moz-landing-system : lando
2019-08-21 03:03:53 +00:00
Mike Hommey
86dcd49bdf Bug 1575375 - Make test_lcov_rewrite use default values for e.g. MOZ_APP_NAME when not running from an objdir. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D42752

--HG--
extra : moz-landing-system : lando
2019-08-21 02:57:32 +00:00
Mike Hommey
e408798c4c Bug 1575375 - Make node test find node on its own when not running in an objdir. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D42751

--HG--
extra : moz-landing-system : lando
2019-08-21 02:56:33 +00:00
Mike Hommey
ade6505592 Bug 1575375 - Fix mozpath.match with python 3.7. r=nalexander
Python 3.7 changed what it escapes with re.escape. Adapt mozpath.match
to account for this.

Differential Revision: https://phabricator.services.mozilla.com/D42750

--HG--
extra : moz-landing-system : lando
2019-08-21 03:12:36 +00:00
Mike Hommey
04e3b11dcf Bug 844509 - Don't create a separate unicode version of the build config. r=nalexander
Now that the configuration comes in without bytes strings, there is no
need to convert it anymore.

Differential Revision: https://phabricator.services.mozilla.com/D42631

--HG--
extra : moz-landing-system : lando
2019-08-20 22:05:24 +00:00
Mike Hommey
621ee253e1 Bug 844509 - Don't encode the configuration that configure passes to config_status. r=nalexander
With bug 1575135, we ensure nothing gets out of the configure sandbox
as a bytes string. We can thus now avoid the encode() pass in
configure.py. We still need, however, to normalize the configuration
so that it doesn't contain unexpected types, and conformning to what
indented_repr does to the configuration in config.status.

While here, convert some obj.iteritems() to six.iteritems(obj).
And remove the now unused encode function.

Differential Revision: https://phabricator.services.mozilla.com/D42630

--HG--
extra : moz-landing-system : lando
2019-08-20 22:22:14 +00:00
Mike Hommey
250c3cbc9a Bug 844509 - Don't encode environment in get_cmd_output and old_configure. r=nalexander
The configure sandbox has wrapped subprocess methods to add its own
encoded environment if none is provided, since bug 1520394. It only
makes sense that it normalizes the environment that comes in too,
avoiding caller in the configure sandbox to have to do it themselves.

OTOH, and while we're here, none of get_cmd_output, old_configure or the
sandbox were actually using the right encoding for this conversion, so
fix the configure sandbox to use the right one, and make it stop using
encode(), which does deep recursion that is not necessary here, and that
I'm trying to remove entirely.

Also while here, remove an unused import of encode().

Differential Revision: https://phabricator.services.mozilla.com/D42608

--HG--
extra : moz-landing-system : lando
2019-08-20 22:12:23 +00:00
Razvan Maries
3d74e341f9 Backed out 2 changesets (bug 1562686) as per glandium's request. CLOSED TREE
Backed out changeset 0ce37eda652a (bug 1562686)
Backed out changeset fe7b9445e1d3 (bug 1562686)
2019-08-22 02:43:16 +03:00