This patch adds some documentation about the mach perftest alert layer, and how to use it. This includes the additional options that it has.
Differential Revision: https://phabricator.services.mozilla.com/D224006
We can continue using `-o` to generate the `requiriements.txt` file by just using the filename instead of the absolute path. This works because we're already setting the `cwd` to be where it needs to be generated. This makes the comment at the top of the generated `requirements.txt` file always be the same, regardless of who ran the vendor.
Differential Revision: https://phabricator.services.mozilla.com/D224508
googlesource only has a base64 encoded raw form, so this patch adds the facility
for a host to specify a function for the copy and decode from the temp download
file. It also allows the individual-files-default-upstream to be an empty str,
because it seems reasonable for a file to be relative to the base vendoring url.
Differential Revision: https://phabricator.services.mozilla.com/D223145
Currently the list of patches are sorted after matching, but the documentation
suggests that they should be in the order specified first and only sorted within
a glob.
The flexibility stated in the documentation seems reasonable.
This affects the sorting of other lists as they are converted to paths, but the
order does not matter in the way that they are used.
Differential Revision: https://phabricator.services.mozilla.com/D223129
We can continue using `-o` to generate the `requiriements.txt` file by just using the filename instead of the absolute path. This works because we're already setting the `cwd` to be where it needs to be generated. This makes the comment at the top of the generated `requirements.txt` file always be the same, regardless of who ran the vendor.
Differential Revision: https://phabricator.services.mozilla.com/D224508
Currently, running running `./mach vendor python --upgrade` will not do
anything different, because all dependencies are pinned. In later
patches, we can unpin dependencies and that will make any future runs
with `--upgrade` added pull the newest versions from PyPi that are
universally compatible.
Differential Revision: https://phabricator.services.mozilla.com/D223430
- Added argument "--add", which allows a user to specify a dependency to add via the command line, rather than editing the 'pyproject.toml' directly.
- Added argument "--remove", which is the opposite of add, with slightly different syntax (This does not require the "==<major.minor.patch>" portion of a dependency.
(The above arguments are both features of uv, and we're essentially just passing them through from `mach` to `uv`)
- Added a hash of the lockfile, so that if you run './mach vendor python', but the lockfile has not changed, there's nothing to do, so the rest of vendoring is skipped.
- Added "--force" to force a vendor, which essentially just skips the lockfile hashing steps, and always forces a vendor, regardless of the change status.
Differential Revision: https://phabricator.services.mozilla.com/D222993
The dependency resolution in 'poetry' was 'lowest', the default
dependency resolution in 'uv' is 'highest'. This means when we vendor,
we will vendor the newest/highest version possible of all transitive
dependencies that are universally/cross-platform compatible.
We could change from the default of 'highest', but I think this is
better. Various packages have been upgraded, and some have been
removed/replaced by others, and that's why there's various changes in
the <site>.txt files.
Differential Revision: https://phabricator.services.mozilla.com/D222992
The vendoring steps are the same. Instead of using 'poetry' to handle
dependency resolution and creating a lock file, we now use 'uv'. That
lock file is then used to create a cross-platform 'requirements.txt'
file by 'uv'.
How the 'requirements.txt' file is consumed by 'pip' to download the
dependencies is unchanged.
Differential Revision: https://phabricator.services.mozilla.com/D222989
Before bug 1915568, we didn't have symbolic links in the clang artifact.
That was an unexpected side effect, but supporting unpacking toolchain
artifacts with symbolic links has been a longstanding thing I've wanted
to fix, unlocking Linux and Mac cross-compiles on Windows (e.g. sysroot
artifacts contain symbolic links).
The clang situation is a good occasion to finally do it. It's also
somehow desirable to keep clang with symbolic links, as it makes the
toolchain artifact smaller.
When a tarball contains symbolic links, the python tarfile module ends
up trying to seek in the tar, which doesn't quite work when the tar is
a stream (which it is when decompressing at the same time).
We wrap TarFile so as to handle the symbolic link situation ourselves,
and when we can't create symbolic links, we instead create a copy of
the file the link points to.
Differential Revision: https://phabricator.services.mozilla.com/D221282
Before bug 1915568, we didn't have symbolic links in the clang artifact.
That was an unexpected side effect, but supporting unpacking toolchain
artifacts with symbolic links has been a longstanding thing I've wanted
to fix, unlocking Linux and Mac cross-compiles on Windows (e.g. sysroot
artifacts contain symbolic links).
The clang situation is a good occasion to finally do it. It's also
somehow desirable to keep clang with symbolic links, as it makes the
toolchain artifact smaller.
When a tarball contains symbolic links, the python tarfile module ends
up trying to seek in the tar, which doesn't quite work when the tar is
a stream (which it is when decompressing at the same time).
We wrap TarFile so as to handle the symbolic link situation ourselves,
and when we can't create symbolic links, we instead create a copy of
the file the link points to.
Differential Revision: https://phabricator.services.mozilla.com/D221282
Still missing:
cargo update -p glean
mach vendor rust
These steps should be done once application-services is updated to pull
in a single version of UniFFI.
Differential Revision: https://phabricator.services.mozilla.com/D219729
The product name (minus any qualifiers such as "beta", "nightly", "devedition", etc.) is needed for Thunderbird to use a channel-agnostic desktop file template. Currently our deb desktop file is broken because the deb repackage looks for a desktop file template based on the release_product.
I.e. release_product is sometimes thunderbird and sometimes thunderbird-beta/thunderbird-nightly. Because of this, it can't be used to find thunderbird.desktop in the debian templates directory.
Differential Revision: https://phabricator.services.mozilla.com/D219114
