Commit Graph

166 Commits

Author SHA1 Message Date
championshuttler
1a42e404f2 Bug 1700419 - Remove py2commands logic from mach driver.r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D109654
2021-03-26 05:26:47 +00:00
Andrew Halberstadt
2cdfb630ea Bug 1638991 - [mach] Run ./mach telemetry-tests-client with Python 3, r=raphael
Differential Revision: https://phabricator.services.mozilla.com/D109082
2021-03-23 17:24:43 +00:00
Andrew Halberstadt
afc041270e Bug 1520458 - Run ./mach raptor with Python 3, r=perftest-reviewers,Bebe,sparky
Differential Revision: https://phabricator.services.mozilla.com/D108828
2021-03-22 14:55:18 +00:00
Andrew Halberstadt
e9d6672ac4 Bug 1638992 - Run ./mach test with Python 3, r=releng-reviewers,jmaher
Differential Revision: https://phabricator.services.mozilla.com/D108257
2021-03-12 21:10:31 +00:00
Mitchell Hentges
b7e0dfe1d1 Bug 1695272: Move --no-interactive to global mach args r=firefox-build-system-reviewers,glandium
Updates `./mach bootstrap` to use `--no-interactive` from global args.
Ensures all bootstrap prompts have a default option.

Differential Revision: https://phabricator.services.mozilla.com/D106814
2021-03-12 16:07:11 +00:00
Andrew Halberstadt
22417e814d Bug 1639024 - Run |mach mozharness| with Python 3, r=releng-reviewers,aki
Differential Revision: https://phabricator.services.mozilla.com/D108111
2021-03-12 14:27:29 +00:00
Andrew Halberstadt
659a890219 Bug 1638970 - Run 'mach jsshell-bench' with Python 3 r=releng-reviewers,mtabara,bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D94882
2021-03-11 17:52:59 +00:00
James Graham
caa4eff9e7 Bug 1639003 - Convert mach wpt-metadata-merge to Python 3, r=ahal
This code is generally missing tests, but I tried it out on the wpt-sync
where it's used and with this fix it worked (previously it had broken
because the wpt libraries started to assume Py 3 semantics)

Differential Revision: https://phabricator.services.mozilla.com/D107992
2021-03-11 15:58:51 +00:00
Joel Maher
6bc1ed63b7 Bug 1697048 - run awsy in py3. r=ahal
run awsy in py3

Differential Revision: https://phabricator.services.mozilla.com/D107551
2021-03-09 15:13:46 +00:00
Henrik Skupin
7b8f1c0f27 Bug 1638960 - Run 'mach firefox-ui-functional' with Python 3. r=jgraham
Differential Revision: https://phabricator.services.mozilla.com/D105002
2021-02-13 20:26:14 +00:00
Henrik Skupin
d6d841ad6d Bug 1638973 - Run 'mach marionette-test' with Python 3. r=jgraham
Differential Revision: https://phabricator.services.mozilla.com/D105000
2021-02-13 20:26:13 +00:00
Jan de Mooij
c692200e2c Bug 1691923 - Don't require Python 2 for mach jstests. r=sfink,gbrown
This fixes a regression from bug 1690784 and is a good idea anyway.

Differential Revision: https://phabricator.services.mozilla.com/D104667
2021-02-11 06:56:43 +00:00
Mike Hommey
2eeafe257c Bug 1687595 - Always get mach modules from subdirectories of where mach is. r=firefox-build-system-reviewers,mhentges
Shortly after mach was introduced, bug 840588 turned mach into a small
wrapper that could be copied in a directory part of $PATH, making that
wrapper load the right thing from the right source directory.

Years later, that setup is not really supported, and it's extra
complexity that can lead to unpleasant surprises.

Differential Revision: https://phabricator.services.mozilla.com/D102376
2021-01-21 01:39:56 +00:00
Mitchell Hentges
d1929ccc4a Bug 1686168: Immediately unset __PYVENV_LAUNCHER__ when mach starts r=firefox-build-system-reviewers,sheehan,glandium
Catch additional cases where both __PYVENV_LAUNCHER__ is set and we invoke a
virtualenv python by unsetting the variable in a prominent location.
This will specifically catch the `./mach bootstrap` => `./mach artifact` case.

Differential Revision: https://phabricator.services.mozilla.com/D101809
2021-01-20 16:27:09 +00:00
Henrik Skupin
8f891b15aa Bug 1638962 - [geckodriver] Remove broken geckodriver-test mach command. r=webdriver-reviewers,jgraham
Differential Revision: https://phabricator.services.mozilla.com/D100901
2021-01-06 12:36:17 +00:00
James Graham
5f8380513a Bug 1638961 - Run mach geckodriver under Py 3, r=whimboo
Differential Revision: https://phabricator.services.mozilla.com/D100253
2020-12-21 19:08:56 +00:00
Kagami Sascha Rosylight
c29f516191 Bug 1639008 - Run wpt-update and wpt-serve on Python 3 r=jgraham,firefox-build-system-reviewers,dmajor
There have been Py3 support patches on WPT. I've used them on Python 3 and haven't seen any unfixed bugs so far.

Differential Revision: https://phabricator.services.mozilla.com/D99292
2020-12-15 14:11:42 +00:00
James Graham
fae57b8a1c Bug 1638996 - Run wpt mach commands under Python 3, r=karlcow
Differential Revision: https://phabricator.services.mozilla.com/D99249
2020-12-14 14:11:15 +00:00
Geoff Brown
8a52920aa9 Bug 1638947 - Run 'mach android' with python 3; r=nalexander
Switch 'mach android' to python 3. As far as I can tell, this just works.

Differential Revision: https://phabricator.services.mozilla.com/D96275
2020-11-09 17:54:33 +00:00
Victor Porof
8c1171c7f8 Bug 1638977 - Convert 'mach prettier-format' to run with Python 3, r=sylvestre
Differential Revision: https://phabricator.services.mozilla.com/D93350
2020-11-05 09:45:14 +00:00
Geoff Brown
17cd08802e Bug 1638981 - Run 'mach reftest', 'mach crashtest', and 'mach jsreftest' with python 3; r=bc
These work for me now, thanks to your earlier work on reftest and my earlier
work on mochitest.

Differential Revision: https://phabricator.services.mozilla.com/D94892
2020-10-29 13:51:44 +00:00
Geoff Brown
e83b64c0b7 Bug 1638986 - Run 'mach talos-test' with python 3; r=sparky,perftest-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D94770
2020-10-29 00:23:51 +00:00
Cosmin Sabou
f654e61a4f Backed out changeset 86e3c90d221f (bug 1638970) for causing Bug 1673914.
CLOSED TREE DONTBUILD
2020-10-28 23:41:26 +02:00
Ben Hearsum
0f48a58046 Bug 1638970: run 'mach jsshell-bench with python 3' r=releng-reviewers,mtabara DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D94882
2020-10-28 13:05:49 +00:00
James Graham
2aec04a4c0 Bug 1639006 - Make mach wpt-test-paths run in Python 3, r=karlcow DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D92935
2020-10-26 13:14:52 +00:00
James Graham
5af13e9963 Bug 1639002 - Make mach wpt-manifest-update run in Python 3, r=karlcow
Differential Revision: https://phabricator.services.mozilla.com/D92932
2020-10-26 13:14:13 +00:00
James Graham
904eff9577 Bug 1639007 - Make mach wpt-unittest run in Python 3, r=karlcow DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D92930
2020-10-26 13:13:29 +00:00
James Graham
adcdde0bf9 Bug 1639004 - Make mach wpt-metadata-summary run in Python 3, r=karlcow
Differential Revision: https://phabricator.services.mozilla.com/D92915
2020-10-26 13:12:15 +00:00
Ben Hearsum
427616a2a1 Bug 1598823: remove visualmetrics mach command r=sparky,releng-reviewers,perftest-reviewers,aki DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D94160
2020-10-22 15:20:20 +00:00
Ben Hearsum
b81bccea51 Bug 1638952: remove check-spidermonkey mach command. r=sfink,releng-reviewers,aki DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D94159
2020-10-22 13:12:32 +00:00
Ben Hearsum
6559aa4480 Bug 1638968: run 'mach jsapi-tests' with python 3 r=releng-reviewers,mtabara DONTBUILD
This already seems to work fine with python 3

Differential Revision: https://phabricator.services.mozilla.com/D94336
2020-10-21 17:17:59 +00:00
Ben Hearsum
5d20c1125d Bug 1638994: run 'mach tps-build' with python 3 r=releng-reviewers,aki DONTBUILD
This appears to work fine already with python 3

Differential Revision: https://phabricator.services.mozilla.com/D94311
2020-10-21 16:38:00 +00:00
Ben Hearsum
d684a58597 Bug 1638989: run 'mach taskcluster-load-image' with python 3 r=releng-reviewers,aki DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D94188
2020-10-21 14:04:50 +00:00
Ben Hearsum
2167d1b284 Bug 1638957: run 'mach devtools-css-db' under python 3. r=releng-reviewers,aki DONTBUILD
It's not 100% clear how long this command will live, but it doesn't look
immediately removable, and it was easy to add python3 support.

Differential Revision: https://phabricator.services.mozilla.com/D94179
2020-10-21 14:04:17 +00:00
Ben Hearsum
d28c2049f7 Bug 1638955: run 'mach cramtest' under python 3. r=releng-reviewers,aki DONTBUILD
This command appears to work just fine under python3 already -- I removed it from the py2 list and ran it with various options, and they all worked fine.

Differential Revision: https://phabricator.services.mozilla.com/D94174
2020-10-20 23:39:35 +00:00
Mike Hommey
e469210428 Bug 1669874 - Use command -v instead of which in mach. r=firefox-build-system-reviewers,andi,rstewart
The former is a POSIX shell construct, usually a builtin, and the latter
is an external program that is not guaranteed to be available.

When `which` is not available, a full ./mach build succeeds with this
change.

Differential Revision: https://phabricator.services.mozilla.com/D92864
2020-10-08 14:58:43 +00:00
Geoff Brown
7d74529955 Bug 1638974 - Run 'mach mochitest' with Python 3; r=bc
Minor fix to remoteautomation.py: Increment stdoutlen before any type conversions,
to ensure that it accurately reflects the byte offset in the file.

With this last change, 'mach mochitest' appears to run correctly on Android with Python 3:
switch it over to Python 3.

Differential Revision: https://phabricator.services.mozilla.com/D91586
2020-09-28 16:30:33 +00:00
Mitchell Hentges
1de0840710 Bug 1665445: install-moz-phab installs the Phabricator credentials r=firefox-build-system-reviewers,rstewart
Since `install-moz-phab` is meant to simplify the moz-phab setup flow,
automatically prompting for Phabricator credentials removes an otherwise
manual step.

Detecting the "console_script" location of a package in a
cross-platform, virtualenv-supporting and "--user"-supporting way is
tough, and the most consistent solution seems to be to list the package
contents of moz-phab and look for the one that seems to be the entry
point.

Differential Revision: https://phabricator.services.mozilla.com/D90642
2020-09-22 20:44:32 +00:00
Steve Fink
038a3a2fc2 Bug 1638966 - Run mach hazards under py3 r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D90483
2020-09-17 19:57:18 +00:00
Ricky Stewart
1a65ffc115 Bug 1662787 - Provide an environment variable to force running mach python-test command with Python 2 r=froydnj,firefox-build-system-reviewers,mhentges
This is only useful for `mach` commands that we want to run with Python 3 by default, but for which running with Python 2 is still useful. We now have one such command: `python-test`.

In `mach`, switch on the presence of the `MACH_PY2` environment variable. We only want to allow this for `python-test`, so do that sanity check in `mach` as well.

Differential Revision: https://phabricator.services.mozilla.com/D89162
2020-09-02 16:21:42 +00:00
Ricky Stewart
bbfa258584 Bug 1656993: Create and require by default global virtualenvs in ~/.mozbuild for mach r=mhentges,ahal
In two different places we've been encountering issues regarding 1) how we configure the system Python environment and 2) how the system Python environment relates to the `virtualenv`s that we use for building, testing, and other dev tasks. Specifically:

1. With the push to use `glean` for telemetry in `mach`, we are requiring (or rather, strongly encouraging) the `glean_sdk` Python package to be installed with bug 1651424. `mach bootstrap` upgrades the library using your system Python 3 in bug 1654607. We can't vendor it due to the package containing native code. Since we generally vendor all code required for `mach` to function, requiring that the system Python be configured with a certain version of `glean` is an unfortunate change.

2. The build uses the vendored `glean_parser` for a number of build tasks. Since the vendored `glean_parser` conflicts with the globally-installed `glean_sdk` package, we had to add special ad-hoc handling to allow us to circumvent this conflict in bug 1655781.

3. We begin to rely more and more on the `zstandard` package during build tasks, this package again being one that we can't vendor due to containing native code. Bug 1654994 contained more ad-hoc code which subprocesses out from the build system's `virtualenv` to the SYSTEM `python3` binary, assuming that the system `python3` has `zstandard` installed.

As we rely more on `glean_sdk`, `zstandard`, and other packages that are not vendorable, we need to settle on a standard model for how `mach`, the build process, and other `mach` commands that may make their own `virtualenv`s work in the presence of unvendorable packages.

With that in mind, this patch does all the following:

1. Separate out the `mach` `virtualenv_packages` from the in-build `virtualenv_packages`. Refactor the common stuff into `common_virtualenv_packages.txt`. Add functionality to the `virtualenv_packages` manifest parsing to allow the build `virtualenv` to "inherit" from the parent by pointing to the parent's `site-packages`. The `in-virtualenv` feature from bug 1655781 is no longer necessary, so delete it.

2. Add code to `bootstrap`, as well as a new `mach` command `create-mach-environment` to create `virtualenv`s in `~/.mozbuild`.

3. Add code to `mach` to dispatch either to the in-`~/.mozbuild` `virtualenv`s (or to the system Python 3 for commands which cannot run in the `virtualenv`s, namely `bootstrap` and `create-mach-environment`).

4. Remove the "add global argument" feature from `mach`. It isn't used and conflicts with (3).

5. Remove the `--print-command` feature from `mach` which is obsoleted by these changes.

This has the effect of allowing us to install packages that cannot be vendored into a "common" place (namely the global `~/.mozbuild` `virtualenv`s) and use those from the build without requiring us to hit the network. Miscellaneous implementation notes:

1. We allow users to force running `mach` with the system Python if they like. For now it doesn't make any sense to require 100% of people to create these `virtualenv`s when they're allowed to continue on with the old behavior if they like. We also skip this in CI.

2. We needed to duplicate the global-argument logic into the `mach` script to allow for the dispatch behavior. This is something we avoided with the Python 2 -> Python 3 migration with the `--print-command` feature, justifying its use by saying it was only temporarily required until all `mach` commands were running with Python 3. With this change, we'll need to be able to determine the `mach` command from the shell script for the forseeable future, and committing to this forever with the cost that `--print-command` incurs (namely `mach` startup time, an additional .4s on my machine) didn't seem worth it to me. It's not a ton of duplicated code.

Differential Revision: https://phabricator.services.mozilla.com/D85916
2020-08-17 17:21:02 +00:00
Hamzah
b7e6b41fc5 Bug 1594914 - Convert mach python-test to run with python 3 r=gbrown
Differential Revision: https://phabricator.services.mozilla.com/D77872
2020-08-04 17:40:58 +00:00
Tom Prince
1501888375 Bug 1638990: [taskgraph] Run mach taskgraph with python 3; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D84087
2020-07-23 03:48:56 +00:00
Geoff Brown
8731a1e19e Bug 1638963 - Run 'mach geckoview-junit' with py3; r=jmaher
Differential Revision: https://phabricator.services.mozilla.com/D84560
2020-07-22 18:08:02 +00:00
Hamzah
570e9a50fc Bug 1606475 - Move mach clang-format to Python 3 r=andi
Manually tested various combination of commands for `./mach clang-format`. It works fine on both py2 as well as py3

Differential Revision: https://phabricator.services.mozilla.com/D79330
2020-07-20 12:12:28 +00:00
Tom Prince
7524e40b38 Bug 1653662: [mozrelease] Update mozrelease subcommands to python 3; r=aki
Differential Revision: https://phabricator.services.mozilla.com/D84042
2020-07-18 01:36:36 +00:00
Edgar Chen
122e54cf20 Bug 1638999 - Convert 'mach webidl-parser-test' to run with Python 3; r=peterv
Differential Revision: https://phabricator.services.mozilla.com/D83416
2020-07-15 20:28:16 +00:00
Edgar Chen
c2f8b06f06 Bug 1638998 - Convert 'mach webidl-example' to run with Python 3; r=peterv
Differential Revision: https://phabricator.services.mozilla.com/D83409
2020-07-15 20:28:03 +00:00
Hamzah
6802583a06 Bug 1639009 - Convert 'mach xpcshell-test' to run with Python 3 r=gbrown
Differential Revision: https://phabricator.services.mozilla.com/D78522
2020-07-09 16:49:48 +00:00
Mihai Alexandru Michis
53a4dbf91a Backed out changeset 19269e470c71 (bug 1594914) for causing bustages.
CLOSED TREE
2020-07-09 01:36:29 +03:00