mirror of
https://github.com/horizon-rust/horizon.git
synced 2026-11-11 23:55:37 -10:00
Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
42ee9ad644 |
No files matched your search
+32
-18
@@ -1,5 +1,5 @@
|
||||
name: checks
|
||||
# verify: every pull request and every push to main, on GitHub's runners (the repository is public). It is the
|
||||
# verify: every pull request, every merge queue entry and every push to main (runners: see the job). It is the
|
||||
# check main requires, so it never has a condition: a skipped job counts as passed, and every run of this workflow
|
||||
# must report a real verify (a pull request that changes no Rust finishes it in seconds instead).
|
||||
# parity: the vanilla harnesses a pull request's changes can affect, on the Mac mini's runner, which holds the
|
||||
@@ -7,11 +7,15 @@ name: checks
|
||||
# request that changes a harness, oracle, probe or the runner: the rest are checked on main by parity-main. A fork's pull
|
||||
# request runs it only when a maintainer adds the run-parity label (.github/workflows/run-parity.yml), so labels
|
||||
# never start this workflow. See docs/release-flow.md.
|
||||
# merge_group: the merge queue's commit (main's tip, the pull requests ahead in the queue and this one) runs verify
|
||||
# before it lands, on GitHub's runners; parity and launcher wait for main's push (docs/release-flow.md#merge-queue).
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
types: [opened, synchronize, reopened]
|
||||
merge_group:
|
||||
types: [checks_requested]
|
||||
workflow_dispatch:
|
||||
permissions:
|
||||
contents: read
|
||||
@@ -19,7 +23,8 @@ concurrency:
|
||||
# A new push to a pull request replaces its running checks. On main, the running check finishes and only the
|
||||
# newest push waits behind it: GitHub cancels a waiting run when a newer one joins the group. A burst of merges
|
||||
# queued ~100 main runs on 2026-11-09; the release follows the newest finished main run (docs/release-flow.md).
|
||||
group: checks-${{ github.event.pull_request.number || (github.event_name == 'push' && 'main') || github.sha }}
|
||||
# Each merge queue entry has a group of its own (its temporary branch), and nothing cancels it: the queue does.
|
||||
group: checks-${{ github.event.pull_request.number || github.event.merge_group.head_ref || (github.event_name == 'push' && 'main') || github.sha }}
|
||||
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
||||
env:
|
||||
# Test runs never reach the telemetry server: no live-count ping, no uploads (docs/telemetry.md).
|
||||
@@ -34,11 +39,14 @@ jobs:
|
||||
# forks' pull requests and main's pushes (which also save the Rust cache GitHub's pull request runs read), so both
|
||||
# pools work at once. The repository variable SELF_HOSTED_CI moves them: `all` every job to ours, `light` (the
|
||||
# desktop paused) or `off` every job to GitHub's. A fork's pull request sees no repository variable, so no switch
|
||||
# could move it off a machine that is off: it always runs on GitHub's. On ours the build folder persists
|
||||
# (CARGO_TARGET_DIR, outside the checkout), so the apt packages, Python and the Rust cache steps are GitHub's only.
|
||||
# could move it off a machine that is off: it always runs on GitHub's. So does a merge queue's run, whatever
|
||||
# SELF_HOSTED_CI says: its commit can hold a fork's code, which job-started.sh refuses on ours. On ours the build
|
||||
# folder persists (CARGO_TARGET_DIR, outside the checkout), so the apt packages, Python and the Rust cache steps are
|
||||
# GitHub's only.
|
||||
verify:
|
||||
runs-on: >-
|
||||
${{ vars.SELF_HOSTED_CI != 'off' && vars.SELF_HOSTED_CI != 'light' && (vars.SELF_HOSTED_CI == 'all' ||
|
||||
${{ vars.SELF_HOSTED_CI != 'off' && vars.SELF_HOSTED_CI != 'light' && github.event_name != 'merge_group' &&
|
||||
(vars.SELF_HOSTED_CI == 'all' ||
|
||||
github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository &&
|
||||
(endsWith(format('{0}', github.event.pull_request.number), '0') ||
|
||||
endsWith(format('{0}', github.event.pull_request.number), '1'))) &&
|
||||
@@ -52,22 +60,24 @@ jobs:
|
||||
if: runner.environment == 'github-hosted'
|
||||
with:
|
||||
python-version: '3.12'
|
||||
# A merge queue's run reads its commit against main's tip (merge_group's base and head).
|
||||
- name: Does this pull request need Rust checks?
|
||||
if: github.event_name == 'pull_request'
|
||||
if: github.event_name == 'pull_request' || github.event_name == 'merge_group'
|
||||
id: rust
|
||||
env:
|
||||
BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.merge_group.base_sha }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha || github.event.merge_group.head_sha }}
|
||||
run: python3 tools/ci/rust_changes.py "$BASE_SHA" "$HEAD_SHA" >> "$GITHUB_OUTPUT"
|
||||
# A pull request adds only its own commits to main, so it reads those; main's pushes read every ref's history.
|
||||
# A pull request (or a merge queue entry) adds only its own commits to main, so it reads those; main's pushes
|
||||
# read every ref's history.
|
||||
- name: Keep game artifacts out of Git
|
||||
env:
|
||||
EVENT: ${{ github.event_name }}
|
||||
BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.merge_group.base_sha }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha || github.event.merge_group.head_sha }}
|
||||
run: |
|
||||
python tools/check_public_tree.py
|
||||
if [ "$EVENT" = pull_request ]; then
|
||||
if [ "$EVENT" = pull_request ] || [ "$EVENT" = merge_group ]; then
|
||||
python tools/check_public_tree.py --range "$BASE_SHA..$HEAD_SHA"
|
||||
else
|
||||
python tools/check_public_tree.py --history
|
||||
@@ -84,21 +94,22 @@ jobs:
|
||||
echo "WGPU_BACKEND=dx12" >> "$GITHUB_ENV"
|
||||
# On a pull request, the next three checks compare with HEAD^1: the main that GitHub's merge commit puts the pull
|
||||
# request on. pull_request.base.sha is main at the pull request's last push and falls behind, so main's own
|
||||
# changes since then would count as the pull request's (#13874).
|
||||
# changes since then would count as the pull request's (#13874). A merge queue's entry compares with main's tip
|
||||
# (merge_group.base_sha), whatever the queue's merge method makes of HEAD^1.
|
||||
# docs/layers.md: a crate depends only on its own layer or a lower one; today's exceptions only shrink.
|
||||
- name: Crates follow the layers
|
||||
env:
|
||||
BASE: ${{ github.event_name == 'pull_request' && 'HEAD^1' || '' }}
|
||||
BASE: ${{ github.event_name == 'pull_request' && 'HEAD^1' || github.event.merge_group.base_sha || '' }}
|
||||
run: python3 tools/check_layers.py ${BASE:+--base "$BASE"}
|
||||
# docs/layers.md#built-in-addresses: every built-in outside address in the Horizon layer's home; the rest only shrinks.
|
||||
- name: Built-in addresses stay in their home
|
||||
env:
|
||||
BASE: ${{ github.event_name == 'pull_request' && 'HEAD^1' || '' }}
|
||||
BASE: ${{ github.event_name == 'pull_request' && 'HEAD^1' || github.event.merge_group.base_sha || '' }}
|
||||
run: python3 tools/check_addresses.py ${BASE:+--base "$BASE"}
|
||||
# docs/layers.md#file-size: a Rust file over 3000 lines may not grow (pull requests; main has no base to compare).
|
||||
- name: Big files don't grow
|
||||
env:
|
||||
BASE: ${{ github.event_name == 'pull_request' && 'HEAD^1' || '' }}
|
||||
BASE: ${{ github.event_name == 'pull_request' && 'HEAD^1' || github.event.merge_group.base_sha || '' }}
|
||||
run: python3 tools/check_file_size.py ${BASE:+--base "$BASE"}
|
||||
- name: Test offline research tooling
|
||||
run: python -m unittest discover -s tests -v
|
||||
@@ -139,10 +150,13 @@ jobs:
|
||||
cargo +stable test --locked --workspace --doc
|
||||
# The launcher is its own workspace (launcher/Cargo.toml), so verify's cargo runs never build it. A pull request that
|
||||
# changes it or a crate it builds is checked here, and so is every push to main: two pull requests can each pass
|
||||
# alone and break it together (#2059 and the registry ratings did, 2026-10-26).
|
||||
# alone and break it together (#2059 and the registry ratings did, 2026-10-26). Not in the merge queue: no check of
|
||||
# its is required, so the queue wouldn't wait for it, and main's push runs it anyway.
|
||||
launcher:
|
||||
if: github.event_name != 'merge_group'
|
||||
runs-on: >-
|
||||
${{ vars.SELF_HOSTED_CI != 'off' && vars.SELF_HOSTED_CI != 'light' && (vars.SELF_HOSTED_CI == 'all' ||
|
||||
${{ vars.SELF_HOSTED_CI != 'off' && vars.SELF_HOSTED_CI != 'light' && github.event_name != 'merge_group' &&
|
||||
(vars.SELF_HOSTED_CI == 'all' ||
|
||||
github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository &&
|
||||
(endsWith(format('{0}', github.event.pull_request.number), '0') ||
|
||||
endsWith(format('{0}', github.event.pull_request.number), '1'))) &&
|
||||
|
||||
+3
-1
@@ -35,6 +35,8 @@ from a `ACTIONS_RUNNER_HOOK_JOB_STARTED` hook in the NixOS configuration.
|
||||
minutes of work on GitHub, 11–13 on a desktop runner with a cold build folder) and `launcher` (1–11). Three
|
||||
runners give ~180 runner-minutes an hour: about the fifth of the pull requests they take. Main's pushes also save
|
||||
the Rust cache GitHub's runs of pull requests read. `SELF_HOSTED_CI=all` sends them all to ours.
|
||||
- **The merge queue's `verify`**, whatever `SELF_HOSTED_CI` says: the queue's commit can hold a fork's code, which
|
||||
`job-started.sh` refuses on ours ([release flow](release-flow.md#merge-queue)).
|
||||
|
||||
## Which jobs, and when
|
||||
|
||||
@@ -44,7 +46,7 @@ for):
|
||||
| `SELF_HOSTED_CI` | Bots (`horizon-light`) | `checks`, `mod-registry`, `telemetry-server`, `pack load bench` (`horizon-checks`) |
|
||||
| --- | --- | --- |
|
||||
| unset (default) | ours | `verify`, `launcher`: ours for this repository's own pull requests ending in 0 or 1, GitHub's for the others, forks' and main's pushes; the rest ours, forks' pull requests excepted |
|
||||
| `all` | ours | ours, every pull request |
|
||||
| `all` | ours | ours, every pull request and main's pushes (not the merge queue's runs) |
|
||||
| `light` | ours | GitHub's: the desktop is paused |
|
||||
| `off` | GitHub's | GitHub's: everything back, in one switch |
|
||||
|
||||
|
||||
+39
-2
@@ -16,7 +16,8 @@ reopened; labels never start it):
|
||||
| `verify` | Our checks runners (label `horizon-checks`, the desktop's) for this repository's own pull requests whose number ends in 0 or 1; GitHub `ubuntu-latest` for the others, forks' pull requests (they see no repository variable, so no switch could move them off a machine that is off) and main's pushes. `SELF_HOSTED_CI` moves them: `all` all to ours, `light` or `off` all to GitHub ([CI runners](ci-runners.md)) | Public-tree check, Python tests, rustfmt, clippy with warnings denied, Rust tests, one after another in one job. Dependencies and the workspace build come from a cache saved by main's runs (GitHub) or the runner's own build folder (ours) | ~8 min |
|
||||
| `parity / parity` | A runner with the label `horizon-game`: the Mac mini's and playhorizon's (below) | Only for a change to a harness, an oracle, a probe or the runner (or the `run-parity` label): the vanilla harnesses the change can affect (`tools/parity.py --changed <base>`). Other pull requests skip it: [parity-main](#parity-on-main) checks them on main | seconds when nothing is affected; minutes per affected harness |
|
||||
|
||||
Pushes to main run `verify` only; the nightly audit covers main's harnesses.
|
||||
Pushes to main run `verify` and `launcher`; parity-main covers main's harnesses. The merge queue, once an owner
|
||||
turns it on, runs `verify` alone ([merge queue](#merge-queue)).
|
||||
|
||||
`verify` is the check main requires, so it has no condition and runs in every run of `checks`: GitHub counts a
|
||||
*skipped* required check as passed, and labels used to start runs whose `verify` was skipped, which let
|
||||
@@ -123,7 +124,7 @@ run there (parity has none). It is also a parity host (`playhorizon` in `tools/p
|
||||
Pull requests merge with a merge commit, by hand: never `gh pr merge --auto`. Branch protection on main
|
||||
requires `verify` for everyone, and Ludovic's review for the files `.github/CODEOWNERS` lists (what builds, signs
|
||||
and ships releases). Branches don't have to be up to date with main: main's own `checks` run catches the rare
|
||||
semantic conflict.
|
||||
semantic conflict, and the [merge queue](#merge-queue), once on, before it lands.
|
||||
|
||||
**Who merges.** The Horizon Team (GitHub team `horizon-team`: kacpherek, levzzz5154, rdbtCVS, Online5277, dezer-x) and
|
||||
Ludovic. The orchestrator's agents prove only the risky pull requests (below), so a contributor's ordinary pull
|
||||
@@ -154,6 +155,42 @@ paths, `lan_e2e`, a screenshot for anything visible) before it merges. Risky mea
|
||||
|
||||
Unsure? Add `needs-proof`: it costs a few hours, a broken release costs more.
|
||||
|
||||
### Merge queue
|
||||
|
||||
The Council decided on a merge queue (poll P43, decision D17, 2026-11-12). With it, merging a pull request adds it to
|
||||
main's queue; GitHub builds a temporary branch (`gh-readonly-queue/main/...`) of main's tip, the entries ahead of it
|
||||
and this one, starts `checks` on it (`merge_group`), and moves main there only once the required checks pass on it.
|
||||
Two pull requests that each pass alone but break main together no longer both land.
|
||||
|
||||
In a queue run (`.github/workflows/ci.yml`):
|
||||
|
||||
- `verify` runs as on a pull request, on the queue's commit against main's tip (`merge_group.base_sha`): the
|
||||
public-tree check reads only the new commits, and a change with no Rust runs only the quick steps.
|
||||
- It always runs on GitHub's runners, whatever `SELF_HOSTED_CI` says: the commit can hold a fork's code, which
|
||||
`job-started.sh` refuses on ours ([CI runners](ci-runners.md)).
|
||||
- Each entry has its own concurrency group and nothing in the workflow cancels it; the queue does when it drops
|
||||
or rebuilds an entry.
|
||||
- `parity` and `launcher` are skipped (a skipped job counts as passed): parity-main and `launcher` run on main's push
|
||||
that follows, as today. Main's push still runs `checks` and parity-main, so releases change nothing.
|
||||
|
||||
Turning it on is an organisation owner's step; the workflow side is in place:
|
||||
|
||||
1. Settings → Rules → Rulesets → New branch ruleset: target `main` (Default branch), enforcement Active, rule
|
||||
**Require merge queue** with merge method **Merge commit** (the release train and parity-main read `Merge pull
|
||||
request #N` first-parent merges), build concurrency 5, minimum group size 1 (no wait), maximum group size 5,
|
||||
"Require all queue entries to pass required checks" on, status check timeout 60 minutes (`verify` takes ~15 at
|
||||
most). Add **Require status checks to pass** with `verify` (source GitHub Actions) in the same ruleset. Bypass
|
||||
list: only whoever may still merge past the queue (`gh pr merge --admin`).
|
||||
2. Classic branch protection on main keeps requiring `verify`: the queue's `merge_group` run reports `verify` on the
|
||||
queue's commit, which satisfies it. Before requiring any other check in either place (`parity / parity` below
|
||||
included), make sure queue runs report it under that name, or every entry waits for it until the timeout: a
|
||||
skipped reusable-workflow call reports only `parity`.
|
||||
|
||||
After that, `gh pr merge N --match-head-commit SHA` queues the pull request instead of merging it: the command
|
||||
returns at once and the pull request is merged later, or dropped from the queue (its timeline says why). Scripts that
|
||||
treat its success as "merged" have to check the pull request's state afterwards. `--auto` stays forbidden;
|
||||
`--admin` merges directly, past the queue, only for someone on the ruleset's bypass list.
|
||||
|
||||
`parity` becomes required once it has run green on pull requests for a while (a week without a false failure):
|
||||
|
||||
```sh
|
||||
|
||||
@@ -127,6 +127,25 @@ class OwnRunnerTests(unittest.TestCase):
|
||||
if "horizon-checks" in value:
|
||||
self.assertIn(FORK_GUARD, value, f"{path.name} {name}")
|
||||
|
||||
def test_the_merge_queue_runs_the_required_verify_on_githubs_runners(self):
|
||||
# main requires verify; with a merge queue on, every entry waits for a verify of its merge_group run. A skipped
|
||||
# job counts as passed, so verify has no condition. The queue's commit can hold a fork's code, which
|
||||
# job-started.sh refuses on ours: SELF_HOSTED_CI=all must not send it there.
|
||||
ci = (WORKFLOWS / "ci.yml").read_text()
|
||||
self.assertIn("merge_group", triggers(ci))
|
||||
verify = dict(jobs(ci))["verify"]
|
||||
self.assertFalse([l for l in verify.splitlines() if l.startswith(" if:")])
|
||||
for path in sorted(WORKFLOWS.glob("*.yml")):
|
||||
text = path.read_text()
|
||||
if "merge_group" not in triggers(text):
|
||||
continue
|
||||
for name, block in jobs(text):
|
||||
value = runs_on(block)
|
||||
job_if = next((l for l in block.splitlines() if l.startswith(" if:")), "")
|
||||
# A job that runs only for another event never sees the queue.
|
||||
if "self-hosted" in value and not re.search(r"github\.event_name (== '(?!merge_group)|!= 'merge_group')", job_if):
|
||||
self.assertIn("github.event_name != 'merge_group'", value, f"{path.name} {name}")
|
||||
|
||||
def test_the_light_runners_refuse_forks_themselves(self):
|
||||
script = (ROOT / "tools/ci/checks-runners.sh").read_text()
|
||||
self.assertIn("ACTIONS_RUNNER_HOOK_JOB_STARTED=$shared/job-started.sh", script)
|
||||
|
||||
Reference in new issue
Block a user