If this is not done, the next person to vendor will also make all of the changes in this commit, which is unnecessary noise.
Differential Revision: https://phabricator.services.mozilla.com/D183450
Also remove dependencies that we don't depend on directly. I assume these were
previously added to get a lockfile of sorts, but maintaining our dependencies'
dependencies manually is an excercise in futility.
Let's forget about locking, and if it becomes a problem implement locking into
mach's sites logic.
Differential Revision: https://phabricator.services.mozilla.com/D195300
If this is not done, the next person to vendor will also make all of the changes in this commit, which is unnecessary noise.
Differential Revision: https://phabricator.services.mozilla.com/D183450
D180501 made the original change that this patch undoes. An earlier
change in that stack inadvertently silently disabled Telemetry, which
hid this issue.
The problem is that `glean` (which we optionally install in the `mach`
site also needs `jsonschema` (and `jsonschema` depends on `pyrsistent`
and `pkgutil_resolve_name`). Since our vendored version was no longer
specified in the `mach.txt` site, it pulled a different version
(`4.19.1`) from the vendored version (`4.17.3`) which causes issues in
the `wpt` site.
Since glean explicitly needs `jsonschema` it never should have been
taken out of the `mach` site, so this patch corrects that mistake.
Differential Revision: https://phabricator.services.mozilla.com/D189553
Update `aiohttp` to version 3.8.5 and `requests` to version 2.31.0,
and vendor their respective dependencies. Add all the new dependencies
to the various required site virtualenv requirements files.
Differential Revision: https://phabricator.services.mozilla.com/D188904
ManifestParser will read TOML files, if present, when use_toml=True
Added tomlkit as a third_party python package
Added poetry-core and tomlkit to pypi (separately as Bug 1845383, Bug 1844787)
Adds TOML test coverage
Adds tomlkit as a dependency of mozharness (in test_archive.py)
Added tomlkit to virtualenv_modules in testing/mozharness/configs/unittests
Removes dependency on six
testing/tools/mach_test_package_initialize.py
- Corrected SEARCH_PATHS
testing/mozharness/mozharness/mozilla/testing/per_test_base.py
- moved `from manifestparser import TestManifest` into function call
to avoid harness inability to locate the internal artifact
- Removed linter warnings
testing/mozbase/manifestparser/manifestparser/manifestparser.py
- Removed linter warnings
- Updated logger usage pattern
- Simplifed _read logic, refactored get_fp_filename()
- Improve context for `include:` logging message
- Defer `import mozlog` until the point of use
testing/mozbase/manifestparser/manifestparser/toml.py
- Removed linter warnings
- Removed unused logger
- Improved readability of read_toml()
testing/mozbase/manifestparser/manifestparser/ini.py
- Removed linter warnings
- Removed unused logger
testing/mozbase/manifestparser/manifestparser/filters.py
- Removed linter warnings
testing/mozbase/manifestparser/tests/test_chunking.py
- Removed linter warnings
Bumped manifestparser version to 2.2.31
Differential Revision: https://phabricator.services.mozilla.com/D184020
Now that we selectively load command modules, and activate the command
virtualenv much earlier in the mach process, a lot of the module
dependencies specified in `mach.txt` are no longer necessary there. With
their removal from `mach.txt` they will no longer be automatically
inherited by every site, which reduces potential dependency conflicts
for specific sites.
The `common` site still effectively has the same set of dependencies.
This is the default site that all commands use unless otherwise
specified. Most commands use this site, and going through every command
and seeing if a dependency is or isn't needed, then deciding if or if
not to create a new site for that command made sense was too time
consuming to do here.
Essentially the idea here going forward is that if you're trying to
add/update a new dependency to a command that is currently defaulting to
the `common` site and there is a conflict with one of the dependencies
in `common` you can move your command to a new site specifically for
your command, and you will have the minimal possible set of dependencies
a mach command can have, improving the odds that you can add the
module(s) you need for your command.
Depends on D180500
Differential Revision: https://phabricator.services.mozilla.com/D180501
Now that we selectively load command modules, and activate the command
virtualenv much earlier in the mach process, a lot of the module
dependencies specified in `mach.txt` are no longer necessary there. With
their removal from `mach.txt` they will no longer be automatically
inherited by every site, which reduces potential dependency conflicts
for specific sites.
The `common` site still effectively has the same set of dependencies.
This is the default site that all commands use unless otherwise
specified. Most commands use this site, and going through every command
and seeing if a dependency is or isn't needed, then deciding if or if
not to create a new site for that command made sense was too time
consuming to do here.
Essentially the idea here going forward is that if you're trying to
add/update a new dependency to a command that is currently defaulting to
the `common` site and there is a conflict with one of the dependencies
in `common` you can move your command to a new site specifically for
your command, and you will have the minimal possible set of dependencies
a mach command can have, improving the odds that you can add the
module(s) you need for your command.
Depends on D180500
Differential Revision: https://phabricator.services.mozilla.com/D180501
Now that we selectively load command modules, and activate the command
virtualenv much earlier in the mach process, a lot of the module
dependencies specified in `mach.txt` are no longer necessary there. With
their removal from `mach.txt` they will no longer be automatically
inherited by every site, which reduces potential dependency conflicts
for specific sites.
The `common` site still effectively has the same set of dependencies.
This is the default site that all commands use unless otherwise
specified. Most commands use this site, and going through every command
and seeing if a dependency is or isn't needed, then deciding if or if
not to create a new site for that command made sense was too time
consuming to do here.
Essentially the idea here going forward is that if you're trying to
add/update a new dependency to a command that is currently defaulting to
the `common` site and there is a conflict with one of the dependencies
in `common` you can move your command to a new site specifically for
your command, and you will have the minimal possible set of dependencies
a mach command can have, improving the odds that you can add the
module(s) you need for your command.
Depends on D180500
Differential Revision: https://phabricator.services.mozilla.com/D180501
We are using features from modern JSONSchema drafts, like dependantRequired, in
the Firefox Messaging System schemas. jsonschema 3.x does not support these
features. Experimenter is also being updated to 4.17.3 and we want to keep our
validation tests in sync with Experimenter.
Differential Revision: https://phabricator.services.mozilla.com/D172457
We are using features from modern JSONSchema drafts, like dependantRequired, in
the Firefox Messaging System schemas. jsonschema 3.x does not support these
features. Experimenter is also being updated to 4.17.3 and we want to keep our
validation tests in sync with Experimenter.
Differential Revision: https://phabricator.services.mozilla.com/D172457
We are using features from modern JSONSchema drafts, like dependantRequired, in
the Firefox Messaging System schemas. jsonschema 3.x does not support these
features. Experimenter is also being updated to 4.17.3 and we want to keep our
validation tests in sync with Experimenter.
Differential Revision: https://phabricator.services.mozilla.com/D172457
`Poetry` version `1.4` depends on `jsonschema>=4.10.0`. Unfortunately, our current implementation of `./mach vendor python` cannot see that specific dependency, because it specific to the `vendor` site.
`glean-parser` depends on `jsonschema>=3.2`, and when the lockfile is generated, it pulls in `jsonschema==3.2`, which causes an incompatibility with `poetry` in the `vendor` site.
Fortunately the loose dependencies of `glean-parser` and `poetry` are compatible, the lockfile just doesn't know it. As such, if we explicitly specify `jsonschema==4.10.0`, the compatibility in the `vendor` site is resolved.
This is a temporary workaround and will be resolved by either vendoring `poetry` itself, or adding `lockfiles` to every site.
Differential Revision: https://phabricator.services.mozilla.com/D171760
Sylvestre Ledru