Commit Graph

4202 Commits

Author SHA1 Message Date
Mitchell Hentges
38e0a4b340 Bug 1550832 - synchronize makefile blacklist with mozbuild variables r=rstewart
As we migrate from makefiles to moz.build, we want to warn developers if
variables only used by moz.build are accidentally defined in a makefile.

This was the purpose of `MOZBUILD_VARIABLES` in recursivemake.py, though it became out-of-date.

This patch defines `MOZBUILD_VARIABLES` off the official list from `mozbuild.frontend.context.VARIABLES`, and
removes unused (?) code from makefiles accordingly

Note that the following variables use to be in `MOZBUILD_VARIABLES`, but aren't there any more because
they aren't in `mozbuild.frontend.context.VARIABLES`:
* CMSRCS
* CMMSRCS
* EXTRA_JS_MODULES
* EXTRA_PP_COMPONENTS
* EXTRA_PP_JS_MODULES
* HOST_CSRCS
* HOST_CMMSRCS
* HOST_EXTRA_LIBS
* JAVA_JAR_TARGETS
* LIBS
* MAKE_FRAMEWORK
* MODULE
* NO_DIST_INSTALL
* NO_INTERFACES_MANIFEST
* PARALLEL_DIRS
* PREF_JS_EXPORTS
* RESOURCE_FILES
* SHARED_LIBRARY_LIBS
* TEST_DIRS
* TOOL_DIRS
* XPCSHELL_TESTS

Differential Revision: https://phabricator.services.mozilla.com/D72076
2020-04-24 14:47:04 +00:00
Cosmin Sabou
f56aee4c5c Backed out changeset 022f4f67d388 (bug 1550832) for causing SM bustages. CLOSED TREE 2020-04-24 19:22:27 +03:00
Mitchell Hentges
373ffb5d65 Bug 1550832 - synchronize makefile blacklist with mozbuild variables r=rstewart
As we migrate from makefiles to moz.build, we want to warn developers if
variables only used by moz.build are accidentally defined in a makefile.

This was the purpose of `MOZBUILD_VARIABLES` in recursivemake.py, though it became out-of-date.

This patch defines `MOZBUILD_VARIABLES` off the official list from `mozbuild.frontend.context.VARIABLES`, and
removes unused (?) code from makefiles accordingly

Note that the following variables use to be in `MOZBUILD_VARIABLES`, but aren't there any more because
they aren't in `mozbuild.frontend.context.VARIABLES`:
* CMSRCS
* CMMSRCS
* EXTRA_JS_MODULES
* EXTRA_PP_COMPONENTS
* EXTRA_PP_JS_MODULES
* HOST_CSRCS
* HOST_CMMSRCS
* HOST_EXTRA_LIBS
* JAVA_JAR_TARGETS
* LIBS
* MAKE_FRAMEWORK
* MODULE
* NO_DIST_INSTALL
* NO_INTERFACES_MANIFEST
* PARALLEL_DIRS
* PREF_JS_EXPORTS
* RESOURCE_FILES
* SHARED_LIBRARY_LIBS
* TEST_DIRS
* TOOL_DIRS
* XPCSHELL_TESTS

Differential Revision: https://phabricator.services.mozilla.com/D72076
2020-04-24 14:47:04 +00:00
Bogdan Tara
a5112d03db Backed out changeset 97280448a042 (bug 1621095) for test_ext_test.html failures CLOSED TREE 2020-04-24 17:09:37 +03:00
Joel Maher
b9a6e9d31c Bug 1621095 - add reftest-qr to run focused webrender reftests on android phones. r=bc
add reftest-qr to run focused webrender reftests on android phones.

Differential Revision: https://phabricator.services.mozilla.com/D72312
2020-04-24 12:35:12 +00:00
Mike Hommey
5e8b3e9559 Bug 1632353 - Convert mach resource-usage to python3. r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D72117
2020-04-23 22:56:37 +00:00
Mike Hommey
56ca1ef0e7 Bug 1631211 - In configure, pass extra compiler flags after source path. r=dmajor,rstewart
When running e.g. check_symbols with extra flags like when checking
for vpx_codec_dec_init_ver when building against system libvpx, in some
configurations, the test can fail when the library flags (-l) appear
before the source file path.

The reason is that in some configurations, the compiler passes
--as-needed to the linker before both the flags and the object file
path, and the object file path is in the same position as the source
file path was. With --as-needed, -l flags are dropped if the library
wasn't needed for any of the linked code that appears *before* the flag.
So linking with `--as-needed -lfoo foo.o`, is equivalent to linking with
`foo.o` only in practice, while `--as-needed foo.o -lfoo` is equivalent
to `foo.o -lfoo`.

Differential Revision: https://phabricator.services.mozilla.com/D71456
2020-04-23 21:46:01 +00:00
shravanrn@gmail.com
e0273c024b Bug 1626174 - Enable use of wasm sandboxed libOgg in the OggDemuxer in linux, mac, try servers r=padenot,erahm,dmajor,firefox-build-system-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D70652
2020-04-22 11:16:10 +00:00
Brindusan Cristian
c4fd863aaa Backed out 2 changesets (bug 1626174, bug 1625876) for build bustages at LibrarySandboxPreload.cpp and OggDemuxer.cpp. CLOSED TREE
Backed out changeset 40fea0f3ab6c (bug 1626174)
Backed out changeset a3117fce845d (bug 1625876)
2020-04-21 19:29:02 +03:00
shravanrn@gmail.com
f0399f4146 Bug 1626174 - Enable use of wasm sandboxed libOgg in the OggDemuxer in linux, mac, try servers r=padenot,erahm,dmajor,firefox-build-system-reviewers
Depends on D68764

Differential Revision: https://phabricator.services.mozilla.com/D70652
2020-04-21 15:30:37 +00:00
Tooru Fujisawa
2108f90fcf Bug 1623965 - Improve error message when binary not found while testing. r=glandium,remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers,Bebe,whimboo
Differential Revision: https://phabricator.services.mozilla.com/D67726
2020-04-21 11:58:04 +00:00
Masatoshi Kimura
8d036ee03e Bug 1631199 - Specify the encoding explicitly when opening backend files to be deleted. r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D71459
2020-04-20 15:25:54 +00:00
Tom Prince
01409a49a6 Bug 1630047 - Allow mozversioncontrol to add or remove multiple files at once; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D70938

--HG--
extra : moz-landing-system : lando
2020-04-15 22:16:49 +00:00
Kartikaya Gupta
e9cc085269 Bug 1629595 - Avoid rewriting .mozconfig.json in mixed-python environments. r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D70708

--HG--
extra : moz-landing-system : lando
2020-04-14 17:47:48 +00:00
Nicholas Nethercote
8139b4051e Bug 1619840 - Remove fix_{linux,macosx}_stack.py and fix_stack_using_bpsyms.py. r=erahm
This commit removes `test_fix_stack_using_bpsyms.py`. That test can't easily be
modified to work with `fix_stacks.py` because it relies on internal
implementation details of `fix_stack_using_bpsym.py`. The unit testing done in
the `fix-stacks` repo provides test coverage that is as good or better.

Differential Revision: https://phabricator.services.mozilla.com/D66924

--HG--
extra : moz-landing-system : lando
2020-04-08 06:55:54 +00:00
Geoff Brown
9ccbc3975a Bug 1457057 - Exit 'mach gtest' with error if debugger not found; r=bc
Simple update to gtest mach command handling of debugger arguments.

Differential Revision: https://phabricator.services.mozilla.com/D70443

--HG--
extra : moz-landing-system : lando
2020-04-13 13:57:00 +00:00
Mike Hommey
0e36d45b8a Bug 1628927 - Convert symbolstore.py to python 3. r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D70467

--HG--
extra : moz-landing-system : lando
2020-04-12 00:03:54 +00:00
Csoregi Natalia
8ed108064b Backed out changeset 789cef4e9c6d (bug 1628927) for symbol related bustage. CLOSED TREE 2020-04-11 04:31:34 +03:00
Mike Hommey
e62d5e9a93 Bug 1628927 - Convert symbolstore.py to python 3. r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D70467

--HG--
extra : moz-landing-system : lando
2020-04-10 21:42:18 +00:00
Mike Hommey
8403480b31 Bug 1628498 - Fix corner-case in virtualenv creation. r=rstewart
This also reverts the previous hack around the issue.

Differential Revision: https://phabricator.services.mozilla.com/D70469

--HG--
extra : moz-landing-system : lando
2020-04-10 21:40:25 +00:00
Ricky Stewart
50c061e425 Bug 1628683 - Update chrome_map.py to support Python 3 r=dmajor
Differential Revision: https://phabricator.services.mozilla.com/D70376

--HG--
extra : moz-landing-system : lando
2020-04-09 20:49:42 +00:00
Ricky Stewart
c8269c0f4d Bug 1628131 - Import WINFUNCTYPE from the correct module to preserve compatibility across py2/py3 r=dmajor
Differential Revision: https://phabricator.services.mozilla.com/D70430

--HG--
extra : moz-landing-system : lando
2020-04-09 19:13:44 +00:00
Andi-Bogdan Postelnicu
8d21444c67 Bug 1628748 - Switch to python3 seems to have broken the Compiler database backend. r=sylvestre
Differential Revision: https://phabricator.services.mozilla.com/D70394

--HG--
extra : moz-landing-system : lando
2020-04-09 15:54:33 +00:00
Emilio Cobos Álvarez
587d6bcab6 Bug 1628498 - Hackaround: Try to build virtualenv from scratch if there was a previous one. r=rstewart
This works for me and should allow trees to reopen.

Differential Revision: https://phabricator.services.mozilla.com/D70415

--HG--
extra : histedit_source : 529e1cb5cdb3030f145f44fc1ebf0ea9ae00aa8f
2020-04-09 17:50:59 +00:00
Benjamin Bouvier
0761d2bc71 Bug 1628641: Use OrderedDict.items() in place of Python 2's iteritems; r=sylvestre
Differential Revision: https://phabricator.services.mozilla.com/D70329

--HG--
extra : moz-landing-system : lando
2020-04-09 10:13:49 +00:00
Mike Hommey
0fb4a38faf Bug 1621436 - Run process_install_manifest with python3. r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D70156

--HG--
extra : moz-landing-system : lando
2020-04-09 00:21:12 +00:00
Geoff Brown
dc33776dd5 Bug 1621244 - Ensure all test files are installed, even when running only a subset; r=bc
Most (all?) mach xpcshell-test calls result in calling install_tests(None) while
most mach test calls for the same test/directory result in calling install_tests()
with a collection of test objects. Providing test objects allows install_tests()
to optimize which tests are installed, but there have been several recent bugs
that appear to be related to that optimization. Let's rely less on that optimization
and make things consistent between test/xpcshell-test. (There's a parallel
consideration for test vs mochitest.)

Differential Revision: https://phabricator.services.mozilla.com/D70052

--HG--
extra : moz-landing-system : lando
2020-04-08 18:30:14 +00:00
Ricky Stewart
6805c2f318 Bug 1625285 - A few small fixes to prevent bustage under Python 3 in lint/resource monitor/testing r=marionette-reviewers,glandium,whimboo
Update code in a few places to work with Python 3, especially with regard to how we invoke subprocesses with text output. Note that the Marionette test harness code needs to be updated so that text is written out in the correct encoding (see bug 1615748).

Differential Revision: https://phabricator.services.mozilla.com/D69012

--HG--
extra : moz-landing-system : lando
2020-04-08 11:42:52 +00:00
Mike Hommey
f805480ab5 Bug 1627163 - Switch python configure to python 3. r=firefox-build-system-reviewers,rstewart
This also does a few remaining python 2 incompatible changes to
.configure files.

Differential Revision: https://phabricator.services.mozilla.com/D69538

--HG--
extra : moz-landing-system : lando
2020-04-07 18:31:56 +00:00
Mike Hommey
848d849a24 Bug 1627163 - Fix a few python 3 issues in mozbuild.backend.visualstudio. r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D69950

--HG--
extra : moz-landing-system : lando
2020-04-07 15:39:58 +00:00
Mike Hommey
1ca5fc294d Bug 1627163 - Avoid a comparison between a str and an int in gyp. r=firefox-build-system-reviewers,rstewart
Python 3 doesn't like that.

Differential Revision: https://phabricator.services.mozilla.com/D69537

--HG--
extra : moz-landing-system : lando
2020-04-07 09:10:42 +00:00
Mike Hommey
6f5d58ae41 Bug 1627163 - Avoid scanning ConfigureSandbox._implied_options when handling implied options. r=firefox-build-system-reviewers,rstewart
ConfigureSandbox._implied_options is a list of ReadOnlyNamespaces.
In python 3.5, ReadOnlyNamespaces end up with no guarantee in the order
of their __dict__. So when comparing

   ReadOnlyNamespace(a=1, b=2)

and

   ReadOnlyNamespace(a=3, b=4)

It's not guaranteed that the a's are compared before the b's.

In ConfigureSandbox._implied_options, some of those ReadOnlyNamespace
fields are SandboxDependsFunctions, which actually raise an error when
they're being compared, because we don't want that to happen in the
sandbox.

So when using python 3, configure would randomly fail when trying to
remove items from the ConfigureSandbox._implied_options list because
removing an item from a list scans the list to find the first element
that matches.

And ConfigureSandbox._implied_options needs to be ordered, which is why
it's currently a list.

So instead of removing by value, we create a new list with the remaining
values. But because the loop recurses, and needs the updated list, we
filter first.

Differential Revision: https://phabricator.services.mozilla.com/D69535

--HG--
extra : moz-landing-system : lando
2020-04-07 15:38:29 +00:00
Mike Hommey
a83c3a6e86 Bug 1624670 - Cap ProcessPoolExecutor's max_workers to 60 on Windows. r=firefox-build-system-reviewers,rstewart
See https://bugs.python.org/issue26903#msg365886.

Differential Revision: https://phabricator.services.mozilla.com/D69921

--HG--
extra : moz-landing-system : lando
2020-04-07 15:55:09 +00:00
Stefan Hindli
a34a2e6240 Backed out changeset 1687bce0a804 (bug 1621244) per gbrown's request on riot CLOSED TREE 2020-04-08 00:18:01 +03:00
Geoff Brown
841cd4f179 Bug 1621244 - Ensure test head files are installed, even when in another directory; r=bc
The effect of the deleted code has been to not install required head files
when those head files are not in the same directory as the test's manifest;
that seems wrong. I am slightly concerned about the comment justifying the
short-cut, but I cannot find any ill effect from removing the code, and doing
so allows test_attachments_downloader.js to run with either xpcshell-test or test.

Differential Revision: https://phabricator.services.mozilla.com/D70051

--HG--
extra : moz-landing-system : lando
2020-04-07 19:08:57 +00:00
Mike Hommey
832881bcbd Bug 1627163 - Add support for python 3 to mozbuild.configure.libstdcxx. r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D69534

--HG--
extra : moz-landing-system : lando
2020-04-05 08:51:21 +00:00
Mike Hommey
cc59888760 Bug 1606703 - Switch python configure lint to python 3. r=rstewart
Support for python 2 is completely dropped. It wouldn't be too much code
to support both python 2 and 3 but since configure is going to switch to
python 3 shortly after, it's not worth the effort.

Differential Revision: https://phabricator.services.mozilla.com/D69525

--HG--
extra : moz-landing-system : lando
2020-04-07 01:19:41 +00:00
Tarek Ziadé
5c0481ca70 Bug 1623321 - Add perftest r=sparky,perftest-reviewers,marionette-reviewers,whimboo,ahal
mach perftest

Differential Revision: https://phabricator.services.mozilla.com/D67342

--HG--
extra : moz-landing-system : lando
2020-04-02 13:04:41 +00:00
Andi-Bogdan Postelnicu
53b98125ff Bug 1626640 - For modules that are compatible with the non unified build environment stop using the unified files for Coverity Analysis. r=marco
Differential Revision: https://phabricator.services.mozilla.com/D69211

--HG--
extra : moz-landing-system : lando
2020-04-01 16:46:08 +00:00
Mike Hommey
65f99a7349 Bug 1626163 - Add rust-generated C/C++ files to generated-files. r=dmajor
Differential Revision: https://phabricator.services.mozilla.com/D68964

--HG--
extra : moz-landing-system : lando
2020-03-31 21:08:37 +00:00
Edwin Takahashi
a853d06380 Bug 1608837 - fix incorrect names of reftest/crashtest variants of web-platform-tests r=ahal
The definition for reftest/crashtest variants of web-platform-tests in web-platform.yml has a trailing `s` which is technically incorrect.

This causes problems when querying ActiveData for the runtimes for these subsuites since the suite name recorded in ActiveData has the trailing s.

Changes:

  - remove the trailing `s` from the definitions

Differential Revision: https://phabricator.services.mozilla.com/D68069

--HG--
extra : moz-landing-system : lando
2020-03-31 20:41:17 +00:00
Andi-Bogdan Postelnicu
db9f108877 Bug 1626190 - Add mach command that checks syntax for C/C++ files. r=bastien
This was designed to be used outside of the `unified build system` in order to keep
individual files syntax sane and to use the `compile_commands.json` outside of the
`unified build` environment.

Differential Revision: https://phabricator.services.mozilla.com/D68968

--HG--
extra : moz-landing-system : lando
2020-03-31 15:02:01 +00:00
Michal Novotny
e3d86ceafe Bug 1587353 - Add http3 tests r=valentin
Differential Revision: https://phabricator.services.mozilla.com/D48670

--HG--
extra : moz-landing-system : lando
2020-03-25 13:36:00 +00:00
Paul Morris
8f4c70e8a1 Bug 1612198 - Remove last remnants of NO_COMPONENTS_MANIFEST build variable. r=dmajor
Differential Revision: https://phabricator.services.mozilla.com/D65584

--HG--
extra : moz-landing-system : lando
2020-03-27 12:36:03 +00:00
Ricky Stewart
3a63c03607 Bug 1624740 - Use io.StringIO instead of io.BytesIO to capture text in building.py r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D68201

--HG--
extra : moz-landing-system : lando
2020-03-26 12:13:25 +00:00
Ricky Stewart
e52b331537 Bug 1623982 - Fix performance of indented_repr in Python 3 r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D67688

--HG--
extra : moz-landing-system : lando
2020-03-25 19:40:06 +00:00
Nathan Froyd
98d5951dcc Bug 1624670 - remove max_workers argument for ProcessPoolExecutor; r=dmajor
`ProcessPoolExecutor` will naturally default to the number of CPUs on
the machine and will also handle edge cases on Windows.

Differential Revision: https://phabricator.services.mozilla.com/D68185

--HG--
extra : moz-landing-system : lando
2020-03-25 15:20:37 +00:00
Anthony Ramine
4bbeb892c3 Bug 1624190 - Use time.process_time() on Python 3; r=rstewart
time.clock() is deprecated since Python 3.3 and gone in Python 3.8.

Depends on D67773

Differential Revision: https://phabricator.services.mozilla.com/D67774

--HG--
extra : moz-landing-system : lando
2020-03-23 16:32:07 +00:00
Geoff Brown
d46135408b Bug 1598746 - Adjust some mochitest and reftest SCHEDULES.exclusive entries; r=ahal
Use finer granularity for some reftest/mochitest SCHEDULES.exclusive entries,
so that reftest-plain does not run when only crashtests are modified, and
vice versa; similarly, break up mochitest into mochitest/browser-chrome/chrome/
a11y. Use schedules-component instead of category.

Differential Revision: https://phabricator.services.mozilla.com/D60085

--HG--
extra : moz-landing-system : lando
2020-03-24 14:55:28 +00:00
Ricky Stewart
c1c042cab7 Bug 1623765 - mach watch supports Python 3 r=dmajor
Differential Revision: https://phabricator.services.mozilla.com/D67927

--HG--
extra : moz-landing-system : lando
2020-03-23 21:34:16 +00:00