Files
AutoGPT/.github/scripts/single_container_cache.py
c23b67a643 ci(platform): let single-container PR builds read the GHCR layer cache (#15013)
### Why / What / How

**Why:** Single-container PR builds run cold every time: 386-462 s for
the image build on recent PR runs, against 151-177 s for a warm `dev`
push. #14969 moved the dependency layers (apt, Node, RabbitMQ/Erlang,
FalkorDB, frontend) under the code layers, so a PR that only touches
code could reuse almost all of them. PR runs just never read the cache.

**What:** PR runs now read the shared `dev` GHCR layer cache
(`v4-{arch}`), the same trick as the e2e job in #14892. They still never
write it.

**How:** Stacked on #14969, which has now merged into `dev`.
- `single_container_cache.py` no longer returns early for
`pull_request`. PR runs get the three `cache-from` lines
(`backend-server-base`, `backend-server`, `single-container`) for
`v4-{arch}` and no `cache-to`. Only a push to `dev` and
`workflow_dispatch` export, as before.
- The workflow's "Authenticate to GHCR build cache" step now runs on PRs
too, with `continue-on-error: ${{ github.event_name == 'pull_request'
}}`. On a PR, a failed login (for example a fork's read-only token
without package access) gives a cold build, not a red job. On a `dev`
push or a dispatch, a failed login still fails loudly.

### Changes 🏗️

- `.github/scripts/single_container_cache.py`: drop the `pull_request`
early return.
- `.github/workflows/platform-single-container-docker.yml`: the GHCR
login step drops `if: github.event_name != 'pull_request'` and gains
PR-only `continue-on-error`.
- `autogpt_platform/single-container/tests/test_ci_registry_cache.py`:
- `test_pull_requests_never_read_or_write_registry_cache` becomes
`test_pull_requests_read_but_never_write_registry_cache`: exactly the
three `cache-from ...:v4-amd64` lines, and nothing with `cache-to`
(arm64 too).
- New `test_only_pull_requests_tolerate_a_failed_cache_login`: the login
step has no `if:`, and its `continue-on-error` is PR-only.
- `docs/platform/contributing/tests.md`: the Caching section said PRs
don't read or write registry caches. It now says they read but never
write, and that on a PR a failed login gives a cold build.

No `.env`, compose or permission changes. The job keeps `packages:
write`: PR runs never get a `cache-to` line, and fork PRs get a
read-only token anyway, the same shape as the e2e job.

### Evidence

Collected on a tester VM against `dev` 6b97e95 (#14969 merged) plus this
branch:

| | `dev` | this branch |
|---|---|---|
| `single_container_cache.py`, `event=pull_request` | empty set block |
3 `cache-from ...:v4-{arch}` lines, no `cache-to` |
| `single_container_cache.py`, `event=push` to `dev` | 3 `cache-from` +
1 `cache-to` | unchanged |
| new tests on `dev`'s script/workflow | 2 of 7 FAIL | - |
| login test with plain `continue-on-error: true` | FAIL | - |
| `autogpt_platform/single-container/tests` (whole suite) | - | 173 run,
OK (1 allowed skip) |

- **Unreadable cache means a cold build, not an error.** On the VM, with
no GHCR credentials, a `docker buildx bake` using the PR-run
`cache-from` lines (docker-container driver, buildkit v0.32.0 as in CI)
logged `ERROR: failed to configure registry cache importer ... 401
Unauthorized`. It then kept building cold: 58 steps finished before I
stopped it at 300 s.
- **This repo's token can read the package.** `dev` push run 36479435743
(linux/amd64) logs in with `GITHUB_TOKEN`, runs `importing cache
manifest from
ghcr.io/significant-gravitas/autogpt-single-container-buildcache:v4-amd64`
(55 `CACHED` steps), then runs `exporting cache to registry`. The
package is private (anonymous pull gets 401).

### Caveats

- **Fork and Dependabot PRs are unproven.** They read the cache only if
the `autogpt-single-container-buildcache` package gives this repo
Actions read access. The `dev` push run above shows this repo's token
reading and writing the package, but I could not read the package
settings: my token lacks `read:packages`. This PR is a same-repo branch,
so its own run does not exercise the read-only token. If access is
missing, those PRs log the 401 and build cold, as they do today.
- The tester VM had no GHCR credentials, so it did not show a warm cache
hit. This PR's own `pull_request` run is the live proof of the warm
path.

### Agents and large language models used

- Claude Code with Claude Opus 5.5

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] `single_container_cache.py` output for `pull_request` and `push`,
before and after
- [x] The changed and new tests fail on `dev` and pass with the change.
Whole single-container suite: 173 OK
- [x] An unauthenticated bake with the PR `cache-from` lines builds cold
instead of failing
- [ ] This PR's own single-container run shows cache hits and a shorter
build than the 386-462 s PR baseline

#### For configuration changes:

- [x] `.env.default` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)

---------

Co-authored-by: Nicholas Tindle <[email protected]>
Co-authored-by: Claude Opus 5.5 (Claude Code) <[email protected]>
2026-09-28 23:30:16 +00:00

41 lines
1.4 KiB
Python

#!/usr/bin/env python3
import hashlib
import os
def cache_settings(event: str, ref: str, image: str, arch: str) -> list[str]:
if arch not in {"amd64", "arm64"}:
raise ValueError(f"Unsupported cache architecture: {arch}")
shared = f"{image}:v4-{arch}"
imports = [shared]
export = None
if event == "workflow_dispatch":
branch_digest = hashlib.sha256(ref.encode()).hexdigest()[:20]
export = f"{image}:v4-branch-{branch_digest}-{arch}"
imports.insert(0, export)
elif event == "push" and ref == "refs/heads/dev":
export = shared
settings = [
f"{target}.cache-from=type=registry,ref={cache_ref}"
for target in ("backend-server-base", "backend-server", "single-container")
for cache_ref in imports
]
if export:
settings.append(
f"single-container.cache-to=type=registry,ref={export},mode=max,"
"oci-mediatypes=true,image-manifest=false,ignore-error=true"
)
return settings
if __name__ == "__main__":
settings = cache_settings(
os.environ["GITHUB_EVENT_NAME"],
os.environ["GITHUB_REF"],
os.environ["BUILD_CACHE_IMAGE"],
os.environ["BUILD_CACHE_ARCH"],
)
with open(os.environ["GITHUB_OUTPUT"], "a", encoding="utf-8") as output:
output.write("set<<CACHE_EOF\n" + "\n".join(settings) + "\nCACHE_EOF\n")