mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-07 06:16:06 -10:00
fix(runner-e2e): drop the Rust dependency layer that filled the image builder disk (#19669)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Runner Full-Stack E2E runs Daytona cells from a signed, content-addressed runner image > - #19545 made the image build faster. One part of it added a Rust dependency layer that keeps Cargo's target and registry trees in the builder > - The first default-branch build after the merge ran out of builder disk while it exported the registry cache > - That run pushed the image tag before the failure, so the tag has no signature, and campaigns that reuse it fail `cosign verify` > - This pull request removes only the dependency layer and keeps the parts of #19545 that add no disk > - The benefit is a working default-branch image build again, a new content ID that replaces the unsigned tag, and the measured 16.5 to 11.6 minute improvement ## Linked Issues or Issue Description Refs #19545. Related #18494 (the same builder disk limit). **What happened?** The default-branch `image_only` dispatch, run 42286903290 (job 128202722275), built and pushed `ghcr.io/paperclipai/paperclip-daytona-runner:e2e-content-7154c0dfc9abb297d8d638a982326c1435dc4440c25112afacc559712d4430b3` (`sha256:2767e91c5279...`). The registry cache export then failed with `ResourceExhausted: ... no space left on device` (exit 102). `preparing build cache for export` took 182 s, against 87 s before #19545. The job stopped before `cosign sign`. The dependency layer retained `target` 852 MB and Cargo registry 333 MB (measured in run 42271236140). **Expected behavior** Default-branch image builds publish the image, write the cache, and sign the digest within the builder disk. **Steps to reproduce** Dispatch `runner-full-stack-e2e.yml` from master with `-f image_only=true` and a Daytona selector on `840838476a`. Read the `Reuse or publish immutable image` step log. ## What Changed - `docker/daytona-runner/Dockerfile`: remove the placeholder-target dependency layer. The runnerd stage is again the single build layer from before #19545, so its cache key matches the earlier default-branch cache. Restore and extend the disk comment. - Keep the content-ID check above the `COPY --from` steps and keep `COPY --link`. They add no retained data. - `tests/runner-e2e/README.md`: describe the two remaining layout rules and why there is no dependency layer. - The Dockerfile is a content-ID input, so the next build gets a new tag. No campaign reuses the unsigned tag. ## Verification - `vitest run tests/runner-e2e/daytona-image.test.ts tests/runner-e2e/workflow-image-only.test.ts`: 20/20 pass. - Branch evidence for the kept parts: job 128077687214 took 11.6 min, against 16.5 min before. The cached-layer download overlapped the compile. - After merge, an `image_only` dispatch from master must publish, write the cache and sign. I did not use a dispatch before merge, because only a default-branch build writes the cache, and the cache export is what failed. ## Risks - Runner source changes again recompile the full Rust dependency graph (about 5 min). To get that saving back, the builder needs more free disk first (for example, remove unused preinstalled toolchains in the image job, or use a larger RunsOn volume). That is a separate change. - The unsigned tag stays in GHCR. Nothing selects it after this change, because the content ID changes. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (shell, GitHub CLI). Extended reasoning on. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <[email protected]>
This commit is contained in:
1 parent
07a548073a
commit
fb1b458409
2 files changed
+11
-29
No files matched your search
@@ -7,32 +7,14 @@ FROM rust:1.97-bookworm@sha256:408fe88047cef61a2087653b0c5255fa51c0f2d6d94ddedd7
|
||||
|
||||
WORKDIR /workspace/packages/paperclip-runner/runner
|
||||
COPY packages/paperclip-runner/runner/Cargo.toml packages/paperclip-runner/runner/Cargo.lock ./
|
||||
COPY packages/paperclip-runner/runner/crates/runner-core/Cargo.toml ./crates/runner-core/Cargo.toml
|
||||
# Compile the locked third-party dependency graph against placeholder targets
|
||||
# so this layer depends only on the manifests and lockfile. Runner source
|
||||
# changes then reuse it from the registry cache and recompile one crate instead
|
||||
# of the whole AWS/TLS graph (about five of the build's six and a half minutes).
|
||||
# The placeholder targets' outputs and fingerprints are deleted, so the next
|
||||
# step always compiles the real runner sources against the same lockfile.
|
||||
RUN set -eu; \
|
||||
mkdir -p crates/runner-core/src; \
|
||||
: > crates/runner-core/src/lib.rs; \
|
||||
for target in $(grep -Eo 'src/bin/[A-Za-z0-9_-]+\.rs' crates/runner-core/Cargo.toml); do \
|
||||
mkdir -p "crates/runner-core/$(dirname "$target")"; \
|
||||
printf 'fn main() {}\n' > "crates/runner-core/$target"; \
|
||||
done; \
|
||||
cargo build --locked --release -j 2 -p paperclip-runner-core --bin paperclip-runnerd; \
|
||||
rm -rf crates/runner-core/src \
|
||||
target/release/.fingerprint/paperclip-runner-core-* \
|
||||
target/release/deps/*paperclip_runner* \
|
||||
target/release/paperclip-runnerd target/release/paperclip-runnerd.d; \
|
||||
du -sk target /usr/local/cargo/registry
|
||||
COPY packages/paperclip-runner/runner/crates ./crates
|
||||
COPY packages/paperclip-runner/protocol ../protocol
|
||||
COPY packages/paperclip-runner/acpx-profiles.json packages/paperclip-runner/antigravity-distributions.json ../
|
||||
# Only the finished executable leaves this stage. The final image never carries
|
||||
# Cargo's target or download trees; the dependency layer above keeps them only
|
||||
# in this builder stage and the trusted build cache.
|
||||
# Retain only the finished executable in this layer. Keeping Cargo's target and
|
||||
# download trees makes BuildKit run out of disk while exporting the final image
|
||||
# and the default-branch build cache. A separate dependency-precompile layer
|
||||
# (#19545) did exactly that: about 1.2 GB of retained Cargo trees exhausted the
|
||||
# builder disk during the registry cache export.
|
||||
RUN cargo build --locked --release -j 2 -p paperclip-runner-core --bin paperclip-runnerd \
|
||||
&& strip /workspace/packages/paperclip-runner/runner/target/release/paperclip-runnerd \
|
||||
&& install -m 0755 target/release/paperclip-runnerd /usr/local/bin/paperclip-runnerd \
|
||||
|
||||
@@ -1208,12 +1208,12 @@ changes can repeat the dependency install; this bounds retained builder disk
|
||||
at the cost of install-cache granularity. The build reports workspace and pack
|
||||
sizes, and final runtime and CLI checks still execute in the published image.
|
||||
|
||||
Build speed comes from three layout rules. The Rust third-party dependency
|
||||
graph compiles in its own layer, keyed only by the Cargo manifests and lock, so
|
||||
a Runner source change recompiles one crate from the default-branch registry
|
||||
cache. The revision-metadata step sits above both `COPY --from` steps, so the
|
||||
cached base and CLI layers download while the builder stages compile. Both
|
||||
copies use `--link`, so they do not wait for that download.
|
||||
Build speed comes from two layout rules. The revision-metadata step sits above
|
||||
both `COPY --from` steps, so the cached base and CLI layers download while the
|
||||
builder stages compile. Both copies use `--link`, so they do not wait for that
|
||||
download. The Rust build keeps no dependency layer: retaining Cargo's target
|
||||
and registry trees (about 1.2 GB) exhausted the builder disk during the
|
||||
default-branch cache export.
|
||||
|
||||
To publish only the image for a branch, without a paid gate, dispatch with
|
||||
`image_only`. Authorization, target resolution, catalog validation and the
|
||||
|
||||
Reference in new issue
Block a user