Commit Graph
205 Commits
Author SHA1 Message Date
ahochheiden a7840606de Bug 1843209 - Bump Mach's minimum Python version to 3.8 r=firefox-build-system-reviewers,glandium
Differential Revision: https://phabricator.services.mozilla.com/D183449
2023-12-12 03:52:00 +00:00
serge-sans-paille bf920c1b96 Bug 1866760 - Do not handle SystemExit and KeyboardInterrupt as generic exceptions during error handling r=ahochheiden,glandium
Differential Revision: https://phabricator.services.mozilla.com/D194749
2023-11-30 08:04:28 +00:00
Cosmin Sabou 4d28b93ff6 Backed out 2 changesets (bug 1843209) for python3 related browser accessible failures. CLOSED TREE
Backed out changeset 052214d67a99 (bug 1843209)
Backed out changeset 36c5e6c2819f (bug 1843209)
2023-11-23 02:41:22 +02:00
ahochheiden 90004e9277 Bug 1843209 - Bump Mach's minimum Python version to 3.8 r=firefox-build-system-reviewers,glandium
Differential Revision: https://phabricator.services.mozilla.com/D183449
2023-11-22 20:07:41 +00:00
serge-sans-paille 37e171f844 Bug 1865672 - Format mach with black r=ahochheiden
To do so we allow straight filenames in the include section and consider
them as automatically-included files.

Differential Revision: https://phabricator.services.mozilla.com/D194093
2023-11-21 08:10:29 +00:00
serge-sans-paille 81e011d33c Bug 1859692 - Display a warning when running a bleeding edge Python r=glandium,firefox-build-system-reviewers,afranchuk,ahochheiden
Do not try to outsmart the user and pick an older Python version there:
first because it may work, second because when you leave on the bleeding
edge, you may want to fix the portability issue, and should be skilled
enough to invoke another Python interpreter on your own.

Still give an advice if we can find an alternate python version that
should work.

Differential Revision: https://phabricator.services.mozilla.com/D191266
2023-11-07 07:42:52 +00:00
ahochheiden e05546cf6e Bug 1695312 - Activate the virtualenv associated with a mach command much earlier r=firefox-build-system-reviewers,glandium
This activated virtualenv for a command is managed
`CommandSiteManager` and it is passed down to where it was activated
before to prevent a second, redundant, activation.

Differential Revision: https://phabricator.services.mozilla.com/D180499
2023-07-25 00:24:53 +00:00
Natalia Csoregi 72be9af4a3 Backed out 4 changesets (bug 1695312) as per request. a=backout
Backed out changeset e5687aa10124 (bug 1695312)
Backed out changeset 6123c201f979 (bug 1695312)
Backed out changeset 06aefe3418e1 (bug 1695312)
Backed out changeset 56f3d3fc806f (bug 1695312)
2023-07-19 17:31:51 +03:00
ahochheiden 79b3316325 Bug 1695312 - Activate the virtualenv associated with a mach command much earlier r=firefox-build-system-reviewers,glandium
This activated virtualenv for a command is managed
`CommandSiteManager` and it is passed down to where it was activated
before to prevent a second, redundant, activation.

Differential Revision: https://phabricator.services.mozilla.com/D180499
2023-07-19 05:09:01 +00:00
Sandor Molnar 28a570d783 Backed out 6 changesets (bug 1842700, bug 1695312, bug 1808732) for causing lint failures. CLOSED TREE
Backed out changeset 0413a48d4f03 (bug 1808732)
Backed out changeset 2839c2d35459 (bug 1695312)
Backed out changeset eab5cdaf4bc9 (bug 1695312)
Backed out changeset a35da3fce458 (bug 1695312)
Backed out changeset 6d2a2dd4790b (bug 1695312)
Backed out changeset b8cf086066cf (bug 1842700)
2023-07-13 15:01:55 +03:00
ahochheiden c9bb5d3abd Bug 1695312 - Activate the virtualenv associated with a mach command much earlier r=firefox-build-system-reviewers,glandium
This activated virtualenv for a command is managed
`CommandSiteManager` and it is passed down to where it was activated
before to prevent a second, redundant, activation.

Differential Revision: https://phabricator.services.mozilla.com/D180499
2023-07-13 05:41:21 +00:00
Alex Hochheiden d15e04fbb3 Bug 1820915 - Attempt to re-launch Mach with a compatible python3 version if one is available r=firefox-build-system-reviewers,andi
Differential Revision: https://phabricator.services.mozilla.com/D172101
2023-03-09 10:53:16 +00:00
Alex Hochheiden 4213dab842 Bug 1734402 - Bump Mach's minimum Python version to 3.7 r=firefox-build-system-reviewers,glandium
Differential Revision: https://phabricator.services.mozilla.com/D171151
2023-03-04 01:21:57 +00:00
Marian-Vasile Laza bfb110fb15 Backed out 4 changesets (bug 1732228, bug 1734402) for multiple browsertime and talos failures. CLOSED TREE
Backed out changeset 266c417563fe (bug 1732228)
Backed out changeset ad9c880233ed (bug 1732228)
Backed out changeset dbab8862bb8a (bug 1734402)
Backed out changeset 2b233ff8eee7 (bug 1734402)
2023-03-01 06:34:22 +02:00
Alex Hochheiden f73427d593 Bug 1734402 - Bump Mach's minimum Python version to 3.7 r=firefox-build-system-reviewers,glandium
Differential Revision: https://phabricator.services.mozilla.com/D171151
2023-03-01 00:39:15 +00:00
Marian-Vasile Laza 6c9f170aa9 Backed out 4 changesets (bug 1734402, bug 1732228) for build bustages when upgrade pip-23.0.1-py3-none-any.whl. CLOSED TREE
Backed out changeset 675f6a2134ac (bug 1732228)
Backed out changeset 37b904639717 (bug 1732228)
Backed out changeset 9eaec62e8473 (bug 1734402)
Backed out changeset 2ad732e17790 (bug 1734402)
2023-02-28 23:53:18 +02:00
Alex Hochheiden da1416eae5 Bug 1734402 - Bump Mach's minimum Python version to 3.7 r=firefox-build-system-reviewers,glandium
Differential Revision: https://phabricator.services.mozilla.com/D171151
2023-02-28 21:25:01 +00:00
Stanca Serban d5bdfc5744 Backed out 4 changesets (bug 1732228, bug 1734402) for causing mda failures. CLOSED TREE
Backed out changeset fe7621bd33fe (bug 1732228)
Backed out changeset 61a521ac78e9 (bug 1732228)
Backed out changeset abf785b6ffe4 (bug 1734402)
Backed out changeset e1f91aaed4c3 (bug 1734402)
2023-02-28 11:33:41 +02:00
Alex Hochheiden 771f7a6481 Bug 1734402 - Bump Mach's minimum Python version to 3.7 r=firefox-build-system-reviewers,glandium
Differential Revision: https://phabricator.services.mozilla.com/D171151
2023-02-28 08:15:45 +00:00
serge-sans-paille 9163ec0b65 Bug 1814369 - Properly format Python2 detection message r=ahochheiden
Do not rely on non-portable print formatting.

Differential Revision: https://phabricator.services.mozilla.com/D168531
2023-02-01 16:22:39 +00:00
Marco Castelluccio ba39d286d5 Bug 1801836 - Remove no longer necessary 'from __future__' imports. r=linter-reviewers,glandium,webdriver-reviewers,perftest-reviewers,geckoview-reviewers,jld,ahal,owlish,afinder DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D165395
2022-12-23 22:45:46 +00:00
ahochheiden 74b7904231 Bug 1766497 - Remove early exit on Homebrew Python3.10 now that we're using venv instead of virtualenv r=firefox-build-system-reviewers,nalexander
Depends on D158922

Differential Revision: https://phabricator.services.mozilla.com/D159196
2022-11-01 07:48:06 +00:00
Cristian Tuns 3e54e59e0c Backed out 5 changesets (bug 1766497) for causing build bustages with "No module named 'encodings'" CLOSED TREE
Backed out changeset 09f7f3d1c6dd (bug 1766497)
Backed out changeset 6bb5f754957a (bug 1766497)
Backed out changeset aedf2d7af408 (bug 1766497)
Backed out changeset 394555265c86 (bug 1766497)
Backed out changeset 41be40696e00 (bug 1766497)
2022-10-12 22:51:51 -04:00
ahochheiden 1e841436a8 Bug 1766497 - Remove early exit on Homebrew Python3.10 now that we're using venv instead of virtualenv r=firefox-build-system-reviewers,nalexander
Differential Revision: https://phabricator.services.mozilla.com/D159196
2022-10-13 00:40:13 +00:00
Mike Hommey 5036ffbd3a Bug 1793550 - Make the homebrew python 3.10 problem immediately visible. r=firefox-build-system-reviewers,andi
The problem only happens with homebrew (python.org's build doesn't have
the problem). Many people are hitting it, and we might as well avoid
them the trouble to have to find the documentation and just fail
directly.

Differential Revision: https://phabricator.services.mozilla.com/D158546
2022-10-04 06:50:17 +00:00
Mitchell Hentges 8c5adeeec3 Bug 1752927: Avoid distutils deprecation warning on every Mach call r=ahal
Python 3.10 prints a warning when it sees `distutils` used, so let's
avoid using `distutils` in the "common Mach code" that runs on every
Mach call.

Note that this change will also fix the `SetuptoolsDeprecationWarning`
that shipped on `setuptools>=58.3.0`, since our usage of `distutils` was
calling a `setup.py install` under the hood.

Differential Revision: https://phabricator.services.mozilla.com/D137499
2022-02-04 21:49:15 +00:00
Mitchell Hentges 41c79bfb04 Bug 1751284: Assert for distutils during Mach initialization r=ahal
Our platform-specific `bootstrap` code transitively depends on
the standard library's `distutils` module, but Debian Linux and derived
distros generally split `distutils` into a separate installable OS
package from `python3` itself.

So, Python 3 being able to run isn't a sufficient guarantee that
`distutils` is available.

To catch this case, add a `distutils` check at the very beginning of
Mach initialization.

-----

Also remove an obsolete comment claiming that `mach bootstrap` doesn't
need `distutils`, which was obsoleted by Bug 1717051.

Differential Revision: https://phabricator.services.mozilla.com/D136639
2022-01-24 22:35:25 +00:00
Mitchell Hentges ddbb742a6c Bug 1740123: Allow invoking Mach outside of MozillaBuild r=glandium
Assuming that the `MOZILLABUILD` environment variable is set, allow
invoking Mach from non-MozillaBuild terminals.

Note that MozillaBuild still needs to be installed, and the
`MOZILLABUILD` environment variable will have to be set.

For future reference: when I tried setting this up with Windows
Store's Python 3.9, I encountered issues when running binaries installed
via `pip`: it would fail with `abort: failed to load Python DLL
python3x.dll`.

Differential Revision: https://phabricator.services.mozilla.com/D133936
2022-01-06 06:49:47 +00:00
Mitchell Hentges 39d5270721 Bug 1740123: Move Python version check earlier r=ahal
Move Python version check as early as possible so that more code can
safely depend on modern behaviour while out-of-date Python versions still
get graceful error messages.

Without this change, Python 2 usages fail on importing `importlib.util`
before the nice "out-of-date version" warning is printed.

Differential Revision: https://phabricator.services.mozilla.com/D134185
2022-01-06 00:32:49 +00:00
Mitchell Hentges fdc33dac57 Bug 1725895: Port --profile-command to pure-Python r=nalexander,glandium
As part of this, the shell-script part of `./mach` can be removed,
making it pure Python.

There's a change in `--profile-command` behaviour, though: it now only
profiles the specific command, rather than all of Mach.
This is because _so much of Mach_ has already been run before
CLI arguments are parsed in the Python process.

If a developer wants to profile Mach itself, they can manually run
`python3 -m cProfile -o <file> ./mach ...`

Differential Revision: https://phabricator.services.mozilla.com/D133928
2022-01-06 00:32:48 +00:00
Marian-Vasile Laza 5e6f31856a Backed out 2 changesets (bug 1725895, bug 1740123) for causing build bustages.
Backed out changeset a6eabd95e31d (bug 1740123)
Backed out changeset bfd5211060b4 (bug 1725895)
2022-01-05 03:46:17 +02:00
Mitchell Hentges 59f01ec9cc Bug 1740123: Move Python version check earlier r=ahal
Move Python version check as early as possible so that more code can
safely depend on modern behaviour while out-of-date Python versions still
get graceful error messages.

Without this change, Python 2 usages fail on importing `importlib.util`
before the nice "out-of-date version" warning is printed.

Differential Revision: https://phabricator.services.mozilla.com/D134185
2022-01-04 21:07:32 +00:00
Mitchell Hentges 6d1070bc8e Bug 1725895: Port --profile-command to pure-Python r=nalexander,glandium
As part of this, the shell-script part of `./mach` can be removed,
making it pure Python.

There's a change in `--profile-command` behaviour, though: it now only
profiles the specific command, rather than all of Mach.
This is because _so much of Mach_ has already been run before
CLI arguments are parsed in the Python process.

If a developer wants to profile Mach itself, they can manually run
`python3 -m cProfile -o <file> ./mach ...`

Differential Revision: https://phabricator.services.mozilla.com/D133928
2022-01-04 21:07:32 +00:00
Rob Lemley 50ca4555e1 Bug 1731158 - Allow initializing mach from a Thunderbird source repo. r=firefox-build-system-reviewers,mhentges
mach will initialize from comm/build/mach_initialize.py if it's present.
See bug 1731160. The initialization function wraps the one in build/mach_initialize.py,
then extends sys.path and loads more mach_commands.py files

Differential Revision: https://phabricator.services.mozilla.com/D131869
2021-11-29 18:34:58 +00:00
Mitchell Hentges 6706092fe6 Bug 1717051: Automatically create and activate Mach virtualenv r=ahal
Consolidate Mach virtualenv management to the front of the
Mach process. This obsoletes `./mach create-mach-environment`
and simplifies the `sh` portion of the top-level `./mach` script.

This helps ensure that the Mach virtualenv doesn't become
out-of-sync and simplifies the mental model of the Mach
virtualenv situation.

Differential Revision: https://phabricator.services.mozilla.com/D120401
2021-11-24 20:06:33 +00:00
Mitchell Hentges 1c5ea36723 Bug 1717051: Remove "install-moz-phab" from nativecmds r=ahal
It was in `nativecmds` to make finding `pip3` easier.
However, since we're removing the distinction between "regular" and
"native" cmds (Mach will create and activate its venv naturally during
initialization), we need to start pruning the `nativecmds` list.
Fortunately, we can use the stored "external_python" value to decide
where to install "moz-phab" to.

The new behaviour continues to support the following use cases:
* Using Mach with system Python in the `$PATH`.
* Using Mach from within a human-managed virtualenv.
* Either of the above^ in combination with `MACH_USE_SYSTEM_PYTHON`.

Differential Revision: https://phabricator.services.mozilla.com/D120397
2021-11-24 20:06:33 +00:00
Mitchell Hentges 7753b911e2 Bug 1717051: Rename "mach_bootstrap.py" to "mach_initialize.py" r=firefox-build-system-reviewers,glandium
We've overloaded "bootstrap" to mean three different things:
* The "standalone bootstrap script": `python/mozboot/bin/bootstrap.py`.
  This is to freshly clone a new repo, then run `./mach bootstrap`.
* `./mach bootstrap`: Install necessary dependencies and set up the
  system for development.
* "Mach bootstrap": do the in-process initialization work Mach needs
  before it can run commands.

By using the term "initialize" instead, perhaps we can remove
ambiguity when discussing Mach.

I'm not attached to the name (or this change at all), but I'm interested
in reviewer thoughts :)

Differential Revision: https://phabricator.services.mozilla.com/D120410
2021-09-03 20:46:22 +00:00
Mitchell Hentges 0dbb09344f Bug 1708260: Remove "imp" usage when loading Mach r=firefox-build-system-reviewers,glandium
We always run on Python 3.6+, so the `imp` usage was dead code.

Differential Revision: https://phabricator.services.mozilla.com/D120398
2021-09-03 20:46:21 +00:00
Nick Alexander a6d8fb5bfc Bug 1713173 - Add --profile-command flag for profiling mach commands. r=firefox-build-system-reviewers,mhentges
I chose to do this at the level of the outer Python invocation because:

1. `python -m cProfile ...` handles writing the file and some other
   details.  It's possible to rebuild the functionality -- the tools
   are there -- but the APIs are awkward.

2. this allows to profile `mach` internals, instead of just the
   invoked command's implementation.

This uses the return code of the `get_command` subshell to transmit
the single bit of information "is the flag present".

The Python-level argument is required in order to have `--help` know
about the option and to avoid the `mach` shell script having to filter
arguments.

Differential Revision: https://phabricator.services.mozilla.com/D116151
2021-06-02 21:28:56 +00:00
championshuttler 188170a8fe Bug 1700419 - Remove py2commands logic from mach driver.r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D109654
2021-03-26 05:26:47 +00:00
Andrew Halberstadt 29d4d60453 Bug 1638991 - [mach] Run ./mach telemetry-tests-client with Python 3, r=raphael
Differential Revision: https://phabricator.services.mozilla.com/D109082
2021-03-23 17:24:43 +00:00
Andrew Halberstadt 1612bc1db5 Bug 1520458 - Run ./mach raptor with Python 3, r=perftest-reviewers,Bebe,sparky
Differential Revision: https://phabricator.services.mozilla.com/D108828
2021-03-22 14:55:18 +00:00
Andrew Halberstadt a32b73e159 Bug 1638992 - Run ./mach test with Python 3, r=releng-reviewers,jmaher
Differential Revision: https://phabricator.services.mozilla.com/D108257
2021-03-12 21:10:31 +00:00
Mitchell Hentges 48a2490eae Bug 1695272: Move --no-interactive to global mach args r=firefox-build-system-reviewers,glandium
Updates `./mach bootstrap` to use `--no-interactive` from global args.
Ensures all bootstrap prompts have a default option.

Differential Revision: https://phabricator.services.mozilla.com/D106814
2021-03-12 16:07:11 +00:00
Andrew Halberstadt 2a1b4db04f Bug 1639024 - Run |mach mozharness| with Python 3, r=releng-reviewers,aki
Differential Revision: https://phabricator.services.mozilla.com/D108111
2021-03-12 14:27:29 +00:00
Andrew Halberstadt 29d4a8153b Bug 1638970 - Run 'mach jsshell-bench' with Python 3 r=releng-reviewers,mtabara,bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D94882
2021-03-11 17:52:59 +00:00
James Graham d62e0771c7 Bug 1639003 - Convert mach wpt-metadata-merge to Python 3, r=ahal
This code is generally missing tests, but I tried it out on the wpt-sync
where it's used and with this fix it worked (previously it had broken
because the wpt libraries started to assume Py 3 semantics)

Differential Revision: https://phabricator.services.mozilla.com/D107992
2021-03-11 15:58:51 +00:00
Joel Maher 2ea727c24d Bug 1697048 - run awsy in py3. r=ahal
run awsy in py3

Differential Revision: https://phabricator.services.mozilla.com/D107551
2021-03-09 15:13:46 +00:00
Henrik Skupin 27e7ddb33f Bug 1638960 - Run 'mach firefox-ui-functional' with Python 3. r=jgraham
Differential Revision: https://phabricator.services.mozilla.com/D105002
2021-02-13 20:26:14 +00:00
Henrik Skupin e3eefc4a89 Bug 1638973 - Run 'mach marionette-test' with Python 3. r=jgraham
Differential Revision: https://phabricator.services.mozilla.com/D105000
2021-02-13 20:26:13 +00:00