perf(ci): let the public-install registry reuse warmed scoped metadata (#16838)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Every pull request runs the `ci / Build` check. This check was the
slowest check in recent green runs: 432s in run 39254079691, 480s in run
39251304886 and 472s in run 39245233967.
> - The slowest step in Build is "Verify built-in Grok from a clean
public npm install" (162s and 207s). Its consumer `npm install` took 69s
and 75s, even though a background step fills the npm cache first.
> - The loopback registry in `scripts/public-install-registry.mjs` sends
packages that it does not serve to npmjs with a 302 redirect. It escaped
scoped names with `encodeURIComponent` (`%40scope%2Fname`). npm requests
and caches the same metadata as `@scope%2fname`, and npm keys cached
metadata by URL.
> - Thus the consumer install could not use the scoped metadata that the
warm-up had just put in the cache. It downloaded and parsed all 291
scoped packuments again (about 285 MB of full packument JSON) inside the
4-vCPU container.
> - This pull request makes the redirect use the same escaping as npm.
> - The benefit is a shorter Build check on every pull request. The
installed bytes do not change.

## Linked Issues or Issue Description

**Describe the bug**
The consumer install in the public-install verifier does not use cached
scoped package metadata. The warm-up step
(`scripts/warm-public-install-npm-cache.mjs`) caches this metadata
through direct npm requests.

**Steps to reproduce**
1. Run `PAPERCLIP_PUBLIC_INSTALL_NPM_CACHE=/tmp/c node
scripts/warm-public-install-npm-cache.mjs`.
2. Install the same dependency set through the loopback registry with
`npm_config_cache=/tmp/c` and `--loglevel http`.
3. Count the `(cache miss)` lines.

**Expected behavior**
No cache misses for metadata that the warm-up already fetched.

**Actual behavior**
291 cache misses. All of them are scoped packages that are requested as
`https://registry.npmjs.org/%40scope%2Fname`. The cache holds them as
`https://registry.npmjs.org/@scope%2fname`.

Related: #16513 changes when the verifier waits for the warm-up. This
pull request does not overlap with it, and the two changes add together.

## What Changed

- `scripts/public-install-registry.mjs`: the 302 redirect for packages
that the registry does not serve now uses npm's own escaping
(`name.replace('/', '%2f')`, the same as npm-package-arg `escapedName`).
The `packageName` check already allows only characters that need no
other escaping.
- `scripts/__tests__/public-install-registry.test.mjs`: the test now
expects `@other%2fpkg` for a scoped redirect. A new assertion covers an
unscoped redirect.

## Verification

- `node --test scripts/__tests__/public-install-registry.test.mjs`: 3 of
3 pass. This includes the real npm install test that compares installed
bytes from file installs and registry installs.
- Local replay (Linux, npm 11.12.1). I filled a fresh cache with the
warm-up script, then installed the same 62-dependency set (627 packages)
through the registry:
  - Before: 291 cache misses, 28.3s.
  - After: 0 cache misses, 19.8s.
- I also ran the real `scripts/public-install-registry.mjs` with a
packed root package that depends on the full public dependency set.
Result: 2 misses (only the loopback package and its tarball, as
expected) and 1438 cache hits.
- To compare on CI, look at `consumer_install_ms` in the
`paperclip.public-npm-install.timing.v1` line of the Build log.
Baseline: 68822 ms (run 39254079691) and 75175 ms (run 39251304886).

## Risks

- Low risk. The redirect target is the same npmjs document. Only the URL
escaping changes. npm still checks every tarball against the integrity
in the packument.
- If the npm cache holds stale scoped metadata, npm revalidates it the
same way as it does for direct requests (`cache-control: max-age=300`).
- The expected time saving comes from a local measurement. The real
Build saving depends on the network and CPU of the runner. I expect
about 10–20s per Build run. The CI timing line will confirm the number.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code as a Paperclip agent with tool use (shell, file edit). I did the
root-cause analysis, the local replays and the code change with this
model.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: devinfoley <[email protected]>
Co-authored-by: Paperclip <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
This commit is contained in:
authored and GitHub committed 2026-11-04 12:10:00 -08:00
1 parent ff7e7b4963
commit c3715e74a6
2 files changed
+4 -2

No files matched your search

@@ -124,7 +124,9 @@ test('the registry exposes only known tarballs, redirects other packages to npm
assert.equal(await (await fetch(metadata.versions['1.0.0'].dist.tarball)).text(), 'packed bytes');
const external = await fetch(`${registry.url}/@other%2fpkg`, { redirect: 'manual' });
assert.equal(external.status, 302);
assert.equal(external.headers.get('location'), 'https://registry.npmjs.org/%40other%2Fpkg');
assert.equal(external.headers.get('location'), 'https://registry.npmjs.org/@other%2fpkg');
const unscoped = await fetch(`${registry.url}/other-pkg`, { redirect: 'manual' });
assert.equal(unscoped.headers.get('location'), 'https://registry.npmjs.org/other-pkg');
assert.equal((await fetch(`${registry.url}/-/900.tgz`)).status, 404);
assert.equal((await fetch(`${registry.url}/some%2foutside%2ffile`)).status, 404);
assert.equal((await fetch(`${registry.url}/@ci%2fpkg`, { method: 'POST' })).status, 405);
+1 -1
View File
@@ -54,7 +54,7 @@ export async function startPackedRegistry(directory) {
if (!packageName.test(path)) { response.writeHead(404).end(); return; }
const entry = packages.get(path);
if (!entry) {
response.writeHead(302, { location: `https://registry.npmjs.org/${encodeURIComponent(path)}` }).end();
response.writeHead(302, { location: `https://registry.npmjs.org/${path.replace('/', '%2f')}` }).end();
return;
}
const { manifest, integrity } = entry;