The previous implementation used days since Jan 1 2000 for the last
16-bit segment. This was not unique enough and caused issues with
Antivirus software if two different channels were built on the same day.
The new approach uses hours since the last milestone bump and uses the
VCS to determine how long ago that was relative to the build time. This
means it will always reset when a new cycle begins, but still be unique
since the digits in the first 3 segments have incremented.
We also now use two of the 16-bits to encode the channel (nightly, beta,
ESR, and release). So two channels built within the same hour will still
be unique.
Using only 14-bits to store the 'hours since version bump', we have
about ~682 days from a version bump before we reach the maximum value we
can store. If a build is done after that point, the segment value will
always be the maximum value for that channel.
Differential Revision: https://phabricator.services.mozilla.com/D200989
The previous implementation used days since Jan 1 2000 for the last
16-bit segment. This was not unique enough and caused issues with
Antivirus software if two different channels were built on the same day.
The new approach uses hours since the last milestone bump and uses the
VCS to determine how long ago that was relative to the build time. This
means it will always reset when a new cycle begins, but still be unique
since the digits in the first 3 segments have incremented.
We also now use two of the 16-bits to encode the channel (nightly, beta,
ESR, and release). So two channels built within the same hour will still
be unique.
Using only 14-bits to store the 'hours since version bump', we have
about ~682 days from a version bump before we reach the maximum value we
can store. If a build is done after that point, the segment value will
always be the maximum value for that channel.
Differential Revision: https://phabricator.services.mozilla.com/D200989
The previous implementation used days since Jan 1 2000 for the last
16-bit segment. This was not unique enough and caused issues with
Antivirus software if two different channels were built on the same day.
The new approach uses hours since the last milestone bump and uses the
VCS to determine how long ago that was relative to the build time. This
means it will always reset when a new cycle begins, but still be unique
since the digits in the first 3 segments have incremented.
We also now use two of the 16-bits to encode the channel (nightly, beta,
ESR, and release). So two channels built within the same hour will still
be unique.
Using only 14-bits to store the 'hours since version bump', we have
about ~682 days from a version bump before we reach the maximum value we
can store. If a build is done after that point, the segment value will
always be the maximum value for that channel.
Differential Revision: https://phabricator.services.mozilla.com/D200989
Adds a CLI option `--l10n-git` that uses the firefox-l10n-source git repo instead of the gecko-strings hg repo when linting localization files.
When we complete the migration, the option and mercurial repo support should be removed.
Differential Revision: https://phabricator.services.mozilla.com/D199866
A previous patch added parsing `git remote -v`, preferring
`mozilla-unified` as the official upstream remote and falling
back to the first official-looking remote if unified was not
found. We assumed that a developer is either using mozilla-unified,
or is using a single-headed repo like central. This overlooks
the fact that it is possible to clone from central, and then
pull other repos in, creating a repo with multiple official
remotes.
Update the `get_upstream_remote` function to find all official
looking remotes instead of a single official remote, and change
the `get_remote_arg` function to return a list of `--remotes`
arguments to be passed to various Git commands. This allows
Git to take all official remotes into consideration and more
precisely find the commits which are not present on any official
upsteams.
Add a test for `get_mozilla_remote_args` while we are here.
Differential Revision: https://phabricator.services.mozilla.com/D199637
In bug 1866395 I added a function to return the canonical Firefox
remote and use it when determining the base ref. I overlooked
that this also needs to be used for `get_branch_nodes` to avoid
returning a set of branch nodes that doesn't begin from the same
base ref. Move the building of the `--remotes` arg to a separate
function and use it in `get_branch_nodes`.
Differential Revision: https://phabricator.services.mozilla.com/D198032
Add a `get_mozilla_upstream_remote` method for `GitRepository` that
scans the output of `git remote -v` for an official Mozilla URL. Use
this URL in `base_ref` to limit the remotes considered for remote
branches, defaulting to `--remotes` without an argument if one can't
be found.
Differential Revision: https://phabricator.services.mozilla.com/D194881
Git versions before 2.41 don't display headers for empty commits
in `git format-patch`. Force headers to display with `--always`,
so Lando can properly apply the patch when pushing to try.
Differential Revision: https://phabricator.services.mozilla.com/D194911
`mozversioncontrol` runs currently use `universal_newlines`, which
is an alias for `text`. This forces the command output file object
to be opened in text mode using the platform's default encoding. On
Windows the default encoding is usually a single-byte encoding such
as `cp-1252`, which can cause issues when parsing patches that include
multi-byte sequences. Add an `encoding` option to `_run` with `utf-8`
as the default option and change the `check_output` call to use it.
Callers can revert to the previous behaviour quite easily by setting
`encoding=None` if necessary.
Differential Revision: https://phabricator.services.mozilla.com/D190710
Add support for submitting stacks of commits to Lando for queueing
on Try, instead of pushing to hg.mozilla.org directly. This patch
implements the Device Code Authorization flow for Auth0, simple changeset
discovery and patch gathering, and submissing to Lando via HTTP POST.
Add a `try.txt` virtualenv site that contains packages from the common
virtualenv as well as the `auth0-python` package for verifying Auth0
JWTs. Use this new virtualenv for `mach try` and related subcommands.
Add a `--push-to-lando` flag that controls whether the push will be made
via the Lando API or using the VCS to hg.mozilla.org directly.
Create a `lando.py` module in the `tryselect` package that handles the
details around submitting to Lando. Authentication is handled by the
Device Code Authorization flow, and the returned access token is saved
to the mozbuild state directory. Auth0 details are added to the `.lando.ini`
file in the repo root, and a `LANDO_TRY_USE_DEV` environment variable can
be set to control submitting to the prod or dev Lando environments.
This module also includes patch stack gathering and discovery via `mozversioncontrol`.
mozversioncontrol's `Repository` subclass is extended with helper functions
to gather patch files from Mercurial and Git. We also add a `try_config_commit`
context manager that creates a temporary commit to hold try syntax commit
messages and/or a `try_task_config.json` file, which is removed from version
control on submission.
The `mach try` cram tests now use a separate virtualenv that must be built
before running, causing unexpected output in the tests. Run `mach try --help`
in the test setup to force the virtualenv to be built before running any
test.
`mach try chooser` would previously install packages for a small web application
from a `requirements.txt` file when run. Install these via the new `try.txt`
site environment instead. The required dependencies are also added to the
`python-test.txt` site since they will cause some tests to fail due to the
missing dependencies.
Differential Revision: https://phabricator.services.mozilla.com/D187641
This results in some changes from our current `isort` configuration. I'm
unclear if it's because ruff isn't at 100% parity with isort, they choose
different defaults or if I missed some configuration.
Either way, the changes all look reasonable to me (see child commit), so I'm
inclined to just accept the new import format it imposes.
Differential Revision: https://phabricator.services.mozilla.com/D172348
This results in some changes from our current `isort` configuration. I'm
unclear if it's because ruff isn't at 100% parity with isort, they choose
different defaults or if I missed some configuration.
Either way, the changes all look reasonable to me (see child commit), so I'm
inclined to just accept the new import format it imposes.
Differential Revision: https://phabricator.services.mozilla.com/D172348
This patch adds a `capture_log` flag that will allow a user to capture the logs produced when running the `push_to_try` methods. Currently, we use `subprocess.check_call` which causes hg to hang when we attempt to redirect and gather stdout. Using `subprocess.run` is better as it lets us capture the logs, but the log output is very slow and can easily lead people to believe that hg is hanging when it's not. This results in corrupted repos. Using Popen, the logs are output slower than `check_call`, but faster than `run` so you know something is happening. The speed at which these logs are printed is also why I have this log capturing behind a flag.
This functionality will be used in the `./mach try perf` selector to capture the child revisions produced for the try task config file changes and allow us to produce a PerfCompare link to provide the user.
Differential Revision: https://phabricator.services.mozilla.com/D155980
Following the pytest "deprecations and removals" docs [1], this patch:
* Replaces `@pytest.yield_fixture` with `@pytest.fixture`.
* Replaces `.funcargnames` with `.fixturenames`.
* Uses `Pathlib` parameter instead of the `py.path.local` one in
associated `pytest_*` hooks.
* Replaces `--strict` with `--strict-markers`
[1] https://docs.pytest.org/en/latest/deprecations.html
Differential Revision: https://phabricator.services.mozilla.com/D138815
Though not a panacea, this patch addresses existing usages of
`get_repository_from_build_config()` to fall back to
`get_repository_object()` if `MissingVCSTool` was encountered.
Ideally, we'd update `substs["HG"]` when we detect that it's changed,
but we don't have infrastructure to incrementally change configure
output.
So, other issues may creep up that will still warrant a fresh
`./mach configure`.
Differential Revision: https://phabricator.services.mozilla.com/D137675
The py.path allowed adding arbitrary fields to it, while a pathlib.Path
does not. To replicate the behavior with as few changes as possible, I
made a test fixture wrapper class (RepoTestFixture) to add the same
arbitrary fields to, while now containing a Pathlib path instead.
Also added a utility functions to make the running of tests a
bit cleaner ('execute_next_step')
Differential Revision: https://phabricator.services.mozilla.com/D136302
Note: In `test_commit.py` For date formatting, the `%s` did not work on Windows.
On Linux it gives the time in seconds. The switch to the rfc822date format for
both hg and git `%aD` made the expected output in the check later work for both
git and hg, and made it platform agnostic.
Differential Revision: https://phabricator.services.mozilla.com/D136209
If watchman is invoked (as part of hg's fsmonitor) with a path whose case does
not match the on-disk case (e.g., for case-insensitive file systems like NTFS),
it will get angry and throw a RootResolveError (although it will still work).
When doing `./mach try`, we end up invoking watchman several times which leads
to it getting very angry and throwing lots of errors.
This is all due to how we compute virtualenv paths -- by normcase()-ing them in
`mach.pth` -- which results in `push.py` having a lowercase path *even if* your
mozilla-central checkout has a non-lowercase on-disk path.
Calling Path.resolve() on the resulting path returns it to its on-disk
representation and no longer angers watchman.
Differential Revision: https://phabricator.services.mozilla.com/D133397
Unless an upstream is specified, we should be able to detect which files
are changed on our local branch without hitting the network.
Differential Revision: https://phabricator.services.mozilla.com/D130684
To track whether issues being reported are for old versions, we attach
an hg revision hash as a release version with each report.
We use the base_ref because:
* It's a public revision that we can look up, and
* Since we only submit Sentry events when mach is unmodified,
it's essentially acting as a version identifier.
Differential Revision: https://phabricator.services.mozilla.com/D108307
ahochheiden