80 Commits

Author SHA1 Message Date
Mike Hommey
f805480ab5 Bug 1627163 - Switch python configure to python 3. r=firefox-build-system-reviewers,rstewart
This also does a few remaining python 2 incompatible changes to
.configure files.

Differential Revision: https://phabricator.services.mozilla.com/D69538

--HG--
extra : moz-landing-system : lando
2020-04-07 18:31:56 +00:00
Mike Hommey
5e73bfc93a Bug 1627163 - Use POINTER from ctypes rather than wintypes. r=firefox-build-system-reviewers,rstewart
It's in both in python 2, but only in the former in python 3.

Differential Revision: https://phabricator.services.mozilla.com/D69532

--HG--
extra : moz-landing-system : lando
2020-04-05 08:51:21 +00:00
Ricky Stewart
232331fdb4 Bug 1610944 - configure/test_toolchain_configure.py and configure/test_toolchain_helpers.py support Python 3 r=firefox-build-system-reviewers,mshal CLOSED TREE
Differential Revision: https://phabricator.services.mozilla.com/D60743

--HG--
extra : source : f5b4973be6dac983ff87add5d198f959de2cec78
extra : amend_source : 30b53ec7c0a22138d43d9c7e85c6e43c9dea84b1
2020-01-23 17:15:38 +00:00
Csoregi Natalia
c12e02a6be Backed out changeset f5b4973be6da (bug 1610944) for nightly bustages. a=backout 2020-01-24 13:40:32 +02:00
Ricky Stewart
e82e92605b Bug 1610944 - configure/test_toolchain_configure.py and configure/test_toolchain_helpers.py support Python 3 r=firefox-build-system-reviewers,mshal
Differential Revision: https://phabricator.services.mozilla.com/D60743

--HG--
extra : moz-landing-system : lando
2020-01-23 17:15:38 +00:00
Cosmin Sabou
7b76379364 Backed out changeset 7c2048122a9f (bug 1610944) for causing L10n bustages. 2020-01-23 07:21:28 +02:00
Ricky Stewart
3003ade6dd Bug 1610944 - configure/test_toolchain_configure.py and configure/test_toolchain_helpers.py support Python 3 r=firefox-build-system-reviewers,mshal
Differential Revision: https://phabricator.services.mozilla.com/D60743

--HG--
extra : moz-landing-system : lando
2020-01-22 23:46:36 +00:00
Ricky Stewart
6a7f736bdb Bug 1607284 - mozbuild/configure/util.py supports Python 3 r=ahal,mars
Differential Revision: https://phabricator.services.mozilla.com/D58834

--HG--
extra : moz-landing-system : lando
2020-01-10 22:09:56 +00:00
Andrew Halberstadt
2fd209f77e Bug 1563797 - Use 'backports.shutil_which' instead of 'which' in configure r=glandium
This gets rid of the last use of the 'which' module left in the tree. So not
only will this help 'configure' become a little more Python 3 compatible, but
we can now remove 'third_party/python/which'.

Differential Revision: https://phabricator.services.mozilla.com/D37427

--HG--
extra : moz-landing-system : lando
2020-01-10 21:04:10 +00:00
Coroiu Cristina
c9e1368a0f Backed out 5 changesets (bug 1607345, bug 1607284, bug 1607512, bug 1563797, bug 1607503) for flake8 failure at python/mozbuild/mozbuild/configure/__init__.py on a CLOSED TREE
Backed out changeset 8dab339bab78 (bug 1607512)
Backed out changeset 1c5880c1c459 (bug 1607503)
Backed out changeset 42a69c5056b0 (bug 1607345)
Backed out changeset 431cb34ec6b0 (bug 1607284)
Backed out changeset 9837631a9c71 (bug 1563797)
2020-01-10 22:59:13 +02:00
Ricky Stewart
39953e00e6 Bug 1607284 - mozbuild/configure/util.py supports Python 3 r=ahal,mars
Differential Revision: https://phabricator.services.mozilla.com/D58834

--HG--
extra : moz-landing-system : lando
2020-01-08 16:17:58 +00:00
Andrew Halberstadt
ab88ef1f24 Bug 1563797 - Use 'backports.shutil_which' instead of 'which' in configure r=glandium
This gets rid of the last use of the 'which' module left in the tree. So not
only will this help 'configure' become a little more Python 3 compatible, but
we can now remove 'third_party/python/which'.

Differential Revision: https://phabricator.services.mozilla.com/D37427

--HG--
extra : moz-landing-system : lando
2020-01-10 07:09:06 +00:00
Andrew Halberstadt
80160f742f Bug 1473498 - [configure] Fix ImportError in util.configure under Python 3, r=glandium
This import is actually unused as of bug 844509.

Differential Revision: https://phabricator.services.mozilla.com/D42869

--HG--
extra : moz-landing-system : lando
2019-08-27 02:55:18 +00:00
Mike Hommey
250c3cbc9a Bug 844509 - Don't encode environment in get_cmd_output and old_configure. r=nalexander
The configure sandbox has wrapped subprocess methods to add its own
encoded environment if none is provided, since bug 1520394. It only
makes sense that it normalizes the environment that comes in too,
avoiding caller in the configure sandbox to have to do it themselves.

OTOH, and while we're here, none of get_cmd_output, old_configure or the
sandbox were actually using the right encoding for this conversion, so
fix the configure sandbox to use the right one, and make it stop using
encode(), which does deep recursion that is not necessary here, and that
I'm trying to remove entirely.

Also while here, remove an unused import of encode().

Differential Revision: https://phabricator.services.mozilla.com/D42608

--HG--
extra : moz-landing-system : lando
2019-08-20 22:12:23 +00:00
Mike Hommey
16c5b7db29 Bug 1575135 - Change get_cmd_output to emit unicode strings. r=nalexander
Because most calling places in python configure don't actually want to
deal with encodings, although in practical terms they should, make
get_cmd_output handle it itself.

Places that explicitly do want bytes can keep using subprocess directly.

Differential Revision: https://phabricator.services.mozilla.com/D42604

--HG--
extra : moz-landing-system : lando
2019-08-20 16:43:15 +00:00
Mike Hommey
76e02a9e5d Bug 1575135 - Remove unused subprocess imports. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D42600

--HG--
extra : moz-landing-system : lando
2019-08-20 16:39:30 +00:00
Mike Hommey
ef876da54d Bug 1526062 - Fix miscellaneous NameErrors lingering in the python configure code. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D19109

--HG--
extra : moz-landing-system : lando
2019-02-08 16:56:13 +00:00
Mike Hommey
a7be46c489 Bug 1523153 - Don't rely on which.which's os.environ in python configure. r=chmanchester
Bug 1520394 changed things such that the configure sandbox is using a
copy of os.environ. So when mozconfig injects environment changes, they
only affect the sandbox. Which means when the which module uses
os.environ to get PATH, it gets the original unmodified environment.

So instead of relying on the which module getting PATH itself, we feed
it with it. It's worth noting that the which module adds `.` on Windows,
but we don't copy this behavior, because in the context of configure,
it's actually not important (`.` would be the topobjdir).

Differential Revision: https://phabricator.services.mozilla.com/D17771

--HG--
extra : moz-landing-system : lando
2019-01-29 01:17:43 +00:00
Mike Hommey
c374d6cb17 Bug 1520340 - Move old_configure_assignments formatting to prepare_configure. r=froydnj
Depends on D16619

Differential Revision: https://phabricator.services.mozilla.com/D16620

--HG--
extra : moz-landing-system : lando
2019-01-16 23:06:41 +00:00
Kris Maglione
f07a2d0fb0 Bug 1497976: Pass close_fds when running commands for configure. r=glandium
Running rustc with an open fd 4 sometimes causes it to fail with an obscure
error when it tries to connect to a job server on that fd. Closing the fd
solves the problem.

close_fds does what we need, and is the default value in Python 3, but not
in Python 2.

Differential Revision: https://phabricator.services.mozilla.com/D8167

--HG--
extra : rebase_source : d94447019e3e698d7cf2c293b0a9b99769cf316a
2018-10-09 16:15:13 -07:00
Mike Hommey
0580b59e77 Bug 1496327 - Remove --without-x r=froydnj
Bug 1282866 removed the only configuration where it did something.
Since then, using it only leads to a configure error.

Differential Revision: https://phabricator.services.mozilla.com/D7691

--HG--
extra : moz-landing-system : lando
2018-10-04 13:54:18 +00:00
Ivan Pozdeev
319f5fcf1d Bug 1323381 - Make 8dot3 error message useful for fixing; r=gps
--HG--
extra : amend_source : 72f43dbe21c9b0af738dccda80dff3b24da8e481
2018-07-23 18:09:15 -07:00
Mike Hommey
1cf4ac0f2e Bug 1469088 - Relax the assumptions of --enable-lto, and make it work for macOS builds. r=ted
Currently, --enable-lto just implies --enable-linker=lld, which
essentially only works on Linux, and assumes one can't do lto with
anything other than lld. Which is not true. As a matter of fact, even
ld.bfd can do lto, as long as the gold plugin for llvm is available,
or when doing lto with GCC instead of clang.

Anyways, to allow more lto setups, we adapt the --enable-linker setup
to:
- work on macOS, which it currently doesn't, and add support for the mac
  linker (ld64), which, unfortunately, doesn't have a clean way to be
  detected, so work around that.
- default to lld if lto is enable, no linker was explicitly given, the
  compiler is clang *and* the build target is not macOS.

--HG--
extra : rebase_source : 1dab2ad6230d18e7f4285943e76352e23b987d4e
2018-07-05 11:35:31 +09:00
Gregory Szorc
f818d2620e Bug 1411994 - Ignore Unicode errors when logging; r=glandium
LineIO currently attempts to convert bytes to unicode using the
system preferred encoding. This can obviously fail for some byte
sequences.

In many cases, the conversion is happening for purposes of logging.
In these cases, Unicode validation shouldn't be that important to
us.

So, this commit teaches LineIO to accept an argument defining its
Unicode error mode. We also teach relevant users of LineIO doing
logging to switch the default mode from "strict" to "replace" so
invalid Unicode byte sequences can be logged with the Unicode
replacement character.

During review, glandium pointed out that it may make better sense
to defer Unicode decoding to the logging layer instead of in LineIO.
I agree with this idea. But it can be implemented in another bug:
for now we should unbust the build system.

MozReview-Commit-ID: 7miuSDY8Xzv

--HG--
extra : rebase_source : 40d5d97d908d00b410b49d03729a34568e5bfb1a
2017-10-26 15:27:01 -07:00
Sylvestre Ledru
816ba0c177 Bug 1406668 - Make build/moz.configure/*.configure compliant to the pep8 format + add to the list of directories to check r=glandium
MozReview-Commit-ID: 89NxxGUVjHV

--HG--
extra : rebase_source : 278ceb2d7d31da086216288bd54688fe25faa03b
2017-10-12 15:22:59 +02:00
Sebastian Hengst
3980c1732a Backed out changeset 4dc78384cb58 (bug 1406668) for flake8 linting failures. r=backout
--HG--
extra : rebase_source : 20f93207038c76e2804f255e4ebfdd5e03a8f7cc
2017-10-12 15:08:35 +02:00
Sylvestre Ledru
a102c9f46a Bug 1406668 - Make build/moz.configure/*.configure compliant to the pep8 format + add to the list of directories to check r=glandium
MozReview-Commit-ID: 89NxxGUVjHV

--HG--
extra : rebase_source : 658301cb32e9c042cd1931456e4e7b8f475900bd
2017-10-11 16:28:29 +02:00
Sebastian Hengst
7a40cdf1bb Backed out changeset b526011049ee (bug 1406668) for linting failures. r=backout
--HG--
extra : amend_source : fe8373f9572300c7fd090f529c6957fbf905d9ef
2017-10-12 14:41:28 +02:00
Sylvestre Ledru
4e4de3339f Bug 1406668 - Make build/moz.configure/*.configure compliant to the pep8 format + add to the list of directories to check r=glandium
MozReview-Commit-ID: 89NxxGUVjHV

--HG--
extra : rebase_source : 3d22007ea6793961e7031603efe58cdc66e3c6c9
2017-10-11 16:28:29 +02:00
Gregory Szorc
f6426ce5ad Bug 1384396 - Add a @depends_all utility function; r=nalexander
Often you only want to evaluate a function if all its dependencies
are true. Expressing this in a "when" can be difficult. So let's
add a convenience decorator for it.

The existing code for @depends_if() was refactored to take an
evaluation function as its first argument. This prevents some
duplicate code and turns @depends_if() and @depends_all() into
one-liners.

MozReview-Commit-ID: Jbugvf0lioM

--HG--
extra : rebase_source : 177741b80ac4fbfb547d6b36a6f5777fe514d91a
extra : source : 0c2bc12f4ebe44428385745266d2fd158e0c3382
2017-07-25 22:00:38 -07:00
Sebastian Hengst
75704b249a Backed out changeset 0c2bc12f4ebe (bug 1384396) for breaking builds when hg username has non-ascii characters. r=backout a=backout on a CLOSED TREE
MozReview-Commit-ID: 2mqBxbDHsH0
2017-07-27 14:56:29 +02:00
Gregory Szorc
869db005b9 Bug 1384396 - Add a @depends_all utility function; r=nalexander
Often you only want to evaluate a function if all its dependencies
are true. Expressing this in a "when" can be difficult. So let's
add a convenience decorator for it.

The existing code for @depends_if() was refactored to take an
evaluation function as its first argument. This prevents some
duplicate code and turns @depends_if() and @depends_all() into
one-liners.

MozReview-Commit-ID: Jbugvf0lioM

--HG--
extra : rebase_source : 38b4af7b668830589126e8a83f5d5ab73914d07c
2017-07-25 22:00:38 -07:00
Bob Owen
847e4be624 Bug 1364137: Get both 32-bit and 64-bit registry values when searching for the Windows SDK. r=glandium
The values that we need to find in the registry can be inconsistent across
different installations, so we retrieve values from both views in our search
for a valid SDK. This also ensures this works for 32-bit and 64-bit python.
2017-07-22 11:32:24 +01:00
Gregory Szorc
b47fb974f1 Bug 1377216 - Accept environment variables to check_cmd_output; r=glandium
And include code to work around a bug on older Python versions.

MozReview-Commit-ID: 4pBnMQQJOGB

--HG--
extra : rebase_source : 6f7c5784230bd37b3496b9bb1781e8d342f741b4
2017-07-18 18:06:03 -07:00
Ryan VanderMeulen
741a7a5ea3 Backed out 6 changesets (bug 1377216) for SM(pkg) bustage on a CLOSED TREE.
Backed out changeset b5701f3ce506 (bug 1377216)
Backed out changeset 1f37aebe0f43 (bug 1377216)
Backed out changeset c8931588f674 (bug 1377216)
Backed out changeset ded5328ba74c (bug 1377216)
Backed out changeset 6cb234539746 (bug 1377216)
Backed out changeset ce836965fd58 (bug 1377216)
2017-07-19 22:14:03 -04:00
Gregory Szorc
c29efbaf05 Bug 1377216 - Accept environment variables to check_cmd_output; r=glandium
And include code to work around a bug on older Python versions.

MozReview-Commit-ID: 4pBnMQQJOGB

--HG--
extra : rebase_source : 6f7c5784230bd37b3496b9bb1781e8d342f741b4
2017-07-18 18:06:03 -07:00
Mike Hommey
f10295dbae Bug 1363811 - Replace all uses of delayed_getattr(a, 'b') with a.b. r=cmanchester+432261
And remove delayed_getattr.
2017-05-19 07:03:40 +09:00
Mike Hommey
6ddaf5ad21 Bug 1316956 - Add a when argument to check_prog. r=chmanchester
--HG--
extra : rebase_source : ce5167bbceb31e554857cba940e72181a5fa361b
2017-05-13 07:12:56 +09:00
Mike Hommey
9de779f50a Bug 1335666 - Remove @depends_when. r=chmanchester
In bug 1296530, we made @depends take a when argument, it can now replace
all uses of @depends_when.

--HG--
extra : rebase_source : d090723fcbf3a56e06bd9c2defc3301cac04d8b0
2017-02-01 16:25:29 +09:00
Carsten "Tomcat" Book
5615a8aaba Bug 1323901 - Bustage fix from glandium 2016-12-28 09:29:31 +01:00
Mike Hommey
74f8bb6e5e Bug 1323901 - Force which to use lowercase extensions on Windows. r=cmanchester+432261
--HG--
extra : rebase_source : 1932e63f7e02a80e5c83ee6bc0963d2c98d7fdd9
2016-12-23 06:28:03 +09:00
Mike Hommey
5fd2ce5f30 Bug 1317359 - Move add_old_configure_* functions to util.configure. r=chmanchester
--HG--
extra : rebase_source : 240fd42920ed4a17592c368ab764672cac01ca31
2016-11-15 14:35:22 +09:00
Mike Hommey
900e53b021 Bug 1293675 - Error out when GetShortPathName returns a long path name. r=gps
--HG--
extra : rebase_source : fa95f8a1356009c44b33701b76d878e75fba7324
2016-11-01 08:06:40 +09:00
Mike Hommey
be15eb6990 Bug 1313306 - Remove --help dependencies from @dependable and delayed_getattr. r=chmanchester
This sadly requires to replace a few @dependendable to pass the lint
check.

--HG--
extra : rebase_source : e0fd1c6f104076ea9f2c3474aadb7bebdf9a7fd8
2016-10-27 15:00:51 +09:00
Phil Ringnalda
4880541dd8 Backed out changeset 6c105cd2bf71 (bug 1293675) for Windows failures in test_checks_configure.py 2016-10-31 18:46:00 -07:00
Mike Hommey
bd70f6640e Bug 1293675 - Error out when GetShortPathName returns a long path name. r=gps
--HG--
extra : rebase_source : b871d4527cdba54ec1764fe99cef1261bf09d868
2016-11-01 08:06:40 +09:00
Mike Hommey
e5ba3fd67a Bug 1296530 - Replace include_when with include, and remove include_when. r=chmanchester
--HG--
extra : rebase_source : e2efd38b3875dde3dae996405838e9ca20475c32
2016-10-13 17:15:24 +09:00
Mike Hommey
caeb1547de Bug 1296530 - Move more things in the new DependsFunction and add a repr() for it. r=chmanchester
--HG--
extra : rebase_source : 739ff145c9f50438d9b7997ab12a7270d4c1b4e5
2016-10-13 13:28:06 +09:00
Mike Hommey
359529b830 Bug 1296530 - Store DependsFunction information for the sandbox as class instances instead of tuples. r=chmanchester
--HG--
extra : rebase_source : 5b05df452074180669bc9cc16501acfe7be7be50
2016-10-13 13:04:18 +09:00
Mike Hommey
16f5c51446 Bug 1296530 - Rename DependsFunction to SandboxDependsFunction. r=chmanchester
--HG--
extra : rebase_source : 60d2ef6f9d8eda36a9f9e3fb82c017958eab18a3
2016-10-13 12:45:24 +09:00