mirror of
https://github.com/paperclipai/paperclip-docs.git
synced 2026-11-07 06:16:07 -10:00
Replace stale LOOA-931 receipt with LOOA-1753 runbook
The old maintenance doc recorded the obsolete 184-URL baseline and the superseded in-build unshallow patch. The new runbook records the Cloudflare Pages config surface, the repo/operator split, verification receipts against current main (193 URLs), the uniform-corpus finding from PR #115, the metric plan, and the rollback path. Co-Authored-By: Claude Fable 5 <[email protected]>
This commit is contained in:
Michael NguyenandClaude Fable 5 committed
1 parent
af82f62417
commit
bf94a1b960
2 files changed
+122
-180
No files matched your search
@@ -0,0 +1,122 @@
|
||||
# LOOA-1753: Docs sitemap `lastmod` — reconciled repair and deployment runbook
|
||||
|
||||
**Date:** 2026-08-26 (UTC 2026-08-27). Supersedes
|
||||
`maintenance/LOOA-931-production-verification.md`, whose 184-URL baseline and
|
||||
in-build unshallow patch are obsolete.
|
||||
|
||||
## What this change is
|
||||
|
||||
Production (`https://docs.paperclip.ing/sitemap.xml`) serves 193 URLs with zero
|
||||
`<lastmod>` values. The build's fail-closed gates from PR #113
|
||||
(`site/build-release.mjs`) omit dates when the checkout is shallow or when
|
||||
every page collapses to one date. Those gates are correct and are kept
|
||||
unchanged. This change adds the missing half: supplying complete Git history to
|
||||
the Cloudflare Pages build so the gates can pass honestly.
|
||||
|
||||
- `scripts/ensure-full-git-history.mjs` — preflight that deepens a shallow
|
||||
checkout (`git fetch --unshallow` from `origin`, falling back to the public
|
||||
repository URL), verifies the result, and always exits 0. The build's own
|
||||
fail-closed gate stays authoritative.
|
||||
- `package.json` — new `docs:build:cloudflare` script:
|
||||
`node scripts/ensure-full-git-history.mjs && npm run docs:build`.
|
||||
- `site/build-release.mjs` — generated deploy guide now recommends
|
||||
`npm run docs:build:cloudflare` as the Pages build command (text only; no
|
||||
build-logic change).
|
||||
- `scripts/audit-sitemap-seo.mjs` — reproducible URL-level production audit
|
||||
(generic; takes sitemap URLs as arguments, asserts no fixed counts).
|
||||
|
||||
## Cloudflare Pages configuration surface and owner
|
||||
|
||||
- Project: `paperclip-docs` (deployment URLs `*.paperclip-docs-74t.pages.dev`),
|
||||
git-connected to `paperclipai/paperclip-docs`; pushes to `main` deploy
|
||||
production at `docs.paperclip.ing`, other branches get preview deployments.
|
||||
- The **build command is dashboard state**, not repository state. The repo's
|
||||
`wrangler.jsonc` carries only `pages_build_output_dir: ".site"`; there is no
|
||||
repository file that sets the Pages build command. It is edited at
|
||||
**Workers & Pages → paperclip-docs → Settings → Build & deployments** by an
|
||||
authorized Cloudflare operator.
|
||||
- Required operator action (one line): set the production build command to
|
||||
`npm run docs:build:cloudflare`. No secrets, tokens, or paid features are
|
||||
involved; the preflight fetches over the clone's existing remote or the
|
||||
public GitHub URL.
|
||||
- Current build command is presumed `npm run docs:build`; this cannot be read
|
||||
from the repository and must be confirmed by the operator in the dashboard.
|
||||
- Split of responsibility: the preflight and build entrypoint are
|
||||
**repository-managed** (this PR); the build-command switch **must be
|
||||
performed in Cloudflare** by an authorized operator. Agents do not hold
|
||||
Cloudflare credentials.
|
||||
|
||||
## Finding: at current main, complete history still yields zero dates — honestly
|
||||
|
||||
Commit `325c9eb` ("seo: hand-write titles and descriptions for all 192 pages",
|
||||
PR #115) edited every docs page, so every page's last-commit date is
|
||||
legitimately `2026-08-25`. The uniform-date gate (`dropUniformLastmod`)
|
||||
therefore fires even on a complete-history build. Consequences to expect:
|
||||
|
||||
- The first full-history production deploy at or near current main will still
|
||||
publish **zero** dates. That is correct behavior, not a failed change.
|
||||
- Dates appear at the first subsequent docs commit that touches a subset of
|
||||
pages: the sitemap then flips to one `<lastmod>` per URL with more than one
|
||||
distinct date, exactly as verified below.
|
||||
|
||||
## Verification receipts (2026-08-26, local)
|
||||
|
||||
Branch `LOOA-931-verify-post-merge-production-and-repair-docs-sitemap-lastmod`
|
||||
at merge of `main@325c9eb`; expected URL count derived from the manifest:
|
||||
192 routes + root = **193**, matching production's 193.
|
||||
|
||||
| Build | URLs | lastmods | Distinct dates | Range | sitemap.xml SHA-256 |
|
||||
| --- | ---: | ---: | ---: | --- | --- |
|
||||
| Production fetch, 2026-08-27T06:54Z (HTTP 200) | 193 | 0 | 0 | — | `27ab10b6c4d8633dddf9f6e0eab7b5428cea802cedde0d09e0c0aa12e4fab11a` |
|
||||
| Full-history build at branch head | 193 | 0 | 0 | — | `27ab10b6…fab11a` (byte-identical to production) |
|
||||
| + one probe commit editing a single page | 193 | 193 | 2 | 2026-08-25 → 2026-08-26 | `90e358d4cd84eefb04cd6c6f6933b66bb3f58b14b82bd017bb3b0020797c4261` |
|
||||
| Same commit rebuilt (stability) | 193 | 193 | 2 | identical | identical SHA |
|
||||
| + second probe commit editing one other page | 193 | 193 | 2 | identical | exactly one `<lastmod>` line changed in the diff |
|
||||
| Forced `--depth 1` clone, no preflight | 193 | 0 | 0 | — | `27ab10b6…fab11a`; warned "Shallow git checkout detected" |
|
||||
| Forced `--depth 1` clone, `docs:build:cloudflare` | 193 | 193 | 2 | 2026-08-25 → 2026-08-26 | checkout deepened from origin, then dated build |
|
||||
|
||||
Preflight unit behavior: on a complete checkout it logs
|
||||
"already has complete history" and exits 0; on an undeepenable checkout it
|
||||
warns and exits 0, leaving the fail-closed gate to omit dates.
|
||||
|
||||
Commands used:
|
||||
|
||||
```bash
|
||||
npm run docs:build # full-history baseline
|
||||
npm run docs:build:cloudflare # preflight + build (deployment path)
|
||||
git clone --depth 1 file://<repo> lab # forced-shallow reproduction
|
||||
npm run docs:test # static-routes, asset-fingerprints,
|
||||
# skills-nav, skill-source-blocks,
|
||||
# crawlable-links, seo-metadata — all passed
|
||||
node scripts/audit-sitemap-seo.mjs --sitemap https://docs.paperclip.ing/sitemap.xml --out audit.json
|
||||
```
|
||||
|
||||
Sitemap metrics were computed by counting `<loc>`/`<lastmod>` elements and
|
||||
hashing the exact response bytes (SHA-256).
|
||||
|
||||
## Metric plan (preview, then production)
|
||||
|
||||
On the candidate preview and on production after each of the next two deploys,
|
||||
record: HTTP status, URL count, `lastmod` count, distinct-date count, min/max
|
||||
date, sitemap SHA-256, and the diff of `<lastmod>` lines for source files
|
||||
unchanged between the deploys (expected: empty). Any ranking or traffic effect
|
||||
is tracked separately in Search Console and is **speculative**.
|
||||
|
||||
## Rollback
|
||||
|
||||
- Repository: revert the PR merge commit and rebuild; behavior returns to
|
||||
today's (fail-closed, no dates).
|
||||
- Cloudflare: set the build command back to `npm run docs:build`. Either
|
||||
rollback alone is safe — the preflight without the dashboard change is
|
||||
inert, and the dashboard change without the preflight fails closed to a
|
||||
dateless sitemap. Wrong dates cannot be produced by any combination.
|
||||
|
||||
## Approval path
|
||||
|
||||
- Pull request: paperclip-docs PR #85 (this branch), reviewer `cryppadotta`
|
||||
(existing review request). Maintainer merges after approval.
|
||||
- Deployment: after merge, the authorized Cloudflare operator flips the build
|
||||
command; production verification per the metric plan above.
|
||||
- Paperclip: fresh target-bound approval staged on LOOA-1753 referencing this
|
||||
runbook revision and the PR head SHA. No merge, deploy, or Cloudflare
|
||||
mutation before that named human approval.
|
||||
@@ -1,180 +0,0 @@
|
||||
# LOOA-931: Production crawl and docs sitemap `lastmod` repair
|
||||
|
||||
**Audit date:** 2026-08-05
|
||||
|
||||
**Scope:** all 60 URLs in `https://paperclip.ing/sitemap.xml` and all 184
|
||||
URLs in `https://docs.paperclip.ing/sitemap.xml`, plus the docs release build.
|
||||
|
||||
## Decision
|
||||
|
||||
Repair the docs release build so it does not derive `lastmod` from a shallow
|
||||
Git boundary commit. A shallow checkout now fetches the complete current-branch
|
||||
history once before computing per-document dates. If complete history cannot be
|
||||
confirmed, the build omits `lastmod` rather than publishing a deployment-wide
|
||||
date as if every document changed.
|
||||
|
||||
Google says sitemap `lastmod` should represent the page's last significant
|
||||
update and that the value is useful only when it consistently matches reality:
|
||||
|
||||
- https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap
|
||||
- https://developers.google.com/search/blog/2023/06/sitemaps-lastmod-ping
|
||||
|
||||
Git documents `rev-parse --is-shallow-repository` as the repository-depth
|
||||
check and `fetch --unshallow` as the operation that converts a shallow clone to
|
||||
a complete one when its source is complete:
|
||||
|
||||
- https://git-scm.com/docs/git-rev-parse#Documentation/git-rev-parse.txt---is-shallow-repository
|
||||
- https://git-scm.com/docs/git-fetch#Documentation/git-fetch.txt---unshallow
|
||||
|
||||
## Production commit verification
|
||||
|
||||
The deployed sites do not expose a trustworthy commit-SHA header, so the
|
||||
merged commits were verified by public behavioral fingerprints rather than an
|
||||
invented deploy receipt.
|
||||
|
||||
- Website PR [#67](https://github.com/paperclipai/paperclip-website/pull/67)
|
||||
merged as `64ff9e9c118e67a1cba645f5246441213113217e`.
|
||||
Production contains all sampled unique markers: the crawlable Solutions
|
||||
overview footer link, canonical `/brand/` footer path, visible `Paperclip
|
||||
blog` H1, and repaired links in the Hermes, release-announcement, and
|
||||
v2026.626 posts.
|
||||
- Docs PR [#77](https://github.com/paperclipai/paperclip-docs/pull/77)
|
||||
merged as `f80e8a72897784c6c1bfcb1699ae9d6c9def5d71`.
|
||||
Production docs HTML contains crawlable previous and next anchors on the
|
||||
sampled Skills reference route.
|
||||
|
||||
This proves the changes are live at the behavior level. It does not prove the
|
||||
current deployments are still byte-for-byte those merge commits; later
|
||||
production changes may also be present.
|
||||
|
||||
## URL-level production baseline
|
||||
|
||||
The reproducible crawler is `scripts/audit-sitemap-seo.mjs`:
|
||||
|
||||
```bash
|
||||
node scripts/audit-sitemap-seo.mjs \
|
||||
--sitemap https://paperclip.ing/sitemap.xml \
|
||||
--sitemap https://docs.paperclip.ing/sitemap.xml \
|
||||
--concurrency 8 \
|
||||
--out LOOA-931-production-crawl.json
|
||||
```
|
||||
|
||||
| Signal | Marketing | Docs | Combined |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| Submitted URLs | 60 | 184 | 244 |
|
||||
| Fetch errors | 0 | 0 | 0 |
|
||||
| HTTP 200 | 60 | 184 | 244 |
|
||||
| Indexable | 60 | 184 | 244 |
|
||||
| Self-canonical | 60 | 184 | 244 |
|
||||
| Redirecting sitemap URLs | 0 | 0 | 0 |
|
||||
| Soft-404 candidates | 0 | 0 | 0 |
|
||||
| Server-rendered inbound-link candidates with zero links | 1 | 0 | 1 |
|
||||
| Pages with multiple H1s | 0 | 183 | 183 |
|
||||
|
||||
Additional findings:
|
||||
|
||||
- **Proven:** the docs sitemap has 184/184 `lastmod` values but only one
|
||||
distinct date, `2026-07-30`.
|
||||
- **Proven:** the marketing sitemap has 36 `lastmod` values across 60 URLs and
|
||||
29 distinct dates. Missing `lastmod` is not itself an error when a reliable
|
||||
date is unavailable.
|
||||
- **Proven:** 183 docs article routes render the landing headline plus the
|
||||
article headline as two H1s. This is an existing template issue, not changed
|
||||
here; it is evidence for [LOOA-937](/LOOA/issues/LOOA-937).
|
||||
- **Proven:** 11 duplicate-title groups exist in docs, including generic
|
||||
`Overview`, `Issues`, `Approvals`, and `Agents` titles. This is recorded for
|
||||
the opportunity/technical-design work and is not silently expanded into this
|
||||
repair.
|
||||
- **Plausible:** `https://paperclip.ing/waitlist/` is an orphan candidate. It
|
||||
is indexable and in the sitemap but received zero server-rendered inbound
|
||||
links from the other 243 submitted URLs. Links from nonsitemap pages or
|
||||
non-anchor interactions could overturn that classification.
|
||||
|
||||
The complete URL-level JSON is the acceptance artifact; its SHA-256 is
|
||||
`047d58a50844145e97f3603c7b92c9eaa7c0c23e850c8e7ee498ace74aed8fa4`.
|
||||
|
||||
## Root cause and verification
|
||||
|
||||
**Root cause — proven by reproduction.** In a depth-1 clone at the production
|
||||
docs head, `git log -1 --follow --format=%cs -- <path>` sees the shallow
|
||||
boundary commit as the newest available history for every tracked document.
|
||||
The unpatched build therefore emits one deployment-commit date for every URL,
|
||||
matching production exactly.
|
||||
|
||||
| Build | URLs / `lastmod` | Distinct dates | Range |
|
||||
| --- | ---: | ---: | --- |
|
||||
| Production before repair | 184 / 184 | 1 | 2026-07-30 only |
|
||||
| Patched depth-1 clone | 184 / 184 | 24 | 2026-04-22 through 2026-07-30 |
|
||||
|
||||
Verification performed:
|
||||
|
||||
- `npm run docs:test:static-routes` — passed.
|
||||
- `npm run sync:lint-links` — passed; 186 Markdown files, no broken internal
|
||||
links.
|
||||
- `npm run docs:build` — passed; 183 crawlable route pages generated.
|
||||
- Patched depth-1 clone — repository changed from shallow to complete during
|
||||
the build; 184 dates across 24 distinct values.
|
||||
- `git diff --check` — passed.
|
||||
|
||||
## Branch preview verification
|
||||
|
||||
Cloudflare deployed commit `e6e084ccdadc082653663be394ee4b97594c92a6` at
|
||||
`https://9701e6d9.paperclip-docs-74t.pages.dev`. The preview check passed and
|
||||
the fetched preview artifact showed:
|
||||
|
||||
- 184 sitemap URLs and 184 `lastmod` values;
|
||||
- 24 distinct dates from 2026-04-22 through 2026-07-30;
|
||||
- byte-for-byte parity with the local full-history sitemap build;
|
||||
- the sampled Skills route retained its production canonical, `index, follow`
|
||||
robots rule, and crawlable previous/next anchors.
|
||||
|
||||
The preview sitemap SHA-256 is
|
||||
`51a37a7922d112b143a2cb642ce938a572f378a01c85ced95ee6ba2824c45ff5`.
|
||||
|
||||
## Prior, hypothesis, metric
|
||||
|
||||
**Prior — strong.** A per-file Git commit date is a defensible approximation
|
||||
for significant document change when complete history is available. A shallow
|
||||
boundary date is not.
|
||||
|
||||
**Hypothesis.** If the deployment build confirms complete current-branch Git
|
||||
history before computing sitemap metadata, then the branch preview and next
|
||||
production deploy will publish multiple stable per-document dates, while a
|
||||
build that cannot obtain complete history will publish no false `lastmod`
|
||||
values.
|
||||
|
||||
**Metric.** On preview and production: 184/184 current sitemap URLs retain HTTP
|
||||
200/indexable/self-canonical status; the sitemap has one `lastmod` per URL; the
|
||||
distinct-date count is greater than one; unchanged source files retain their
|
||||
historic dates across consecutive deploys. The current patched baseline is 24
|
||||
distinct dates. Search outcome is monitored separately in Search Console;
|
||||
ranking or traffic movement from this metadata correction is **speculative**.
|
||||
|
||||
## Changed files and URLs
|
||||
|
||||
- `site/build-release.mjs` — detect and complete shallow history once; omit
|
||||
dates if completeness cannot be confirmed.
|
||||
- `scripts/verify-static-routes.mjs` — require one date per URL in a complete
|
||||
checkout and more than one distinct date.
|
||||
- `scripts/audit-sitemap-seo.mjs` — reproducible URL-level production audit.
|
||||
- `maintenance/LOOA-931-production-verification.md` — evidence, hypothesis,
|
||||
metrics, and rollback.
|
||||
- Public URL changes: none.
|
||||
|
||||
## Expected movement and measurement dates
|
||||
|
||||
Immediate expected movement after an approved deploy: docs sitemap distinct
|
||||
`lastmod` values move from 1 to greater than 1; the local depth-1 baseline is
|
||||
24. No status, canonical, robots, sitemap-membership, or visible-page movement
|
||||
is expected.
|
||||
|
||||
If production deploys on 2026-08-05, verify the sitemap immediately and compare
|
||||
Search Console at 7 days (2026-08-12), 14 days (2026-08-19), and 28 days
|
||||
(2026-09-02). Shift those dates with the actual deployment date.
|
||||
|
||||
## Rollback
|
||||
|
||||
Revert the repair commit and rebuild. That restores the prior build behavior;
|
||||
because the prior behavior publishes false deployment-wide dates in shallow
|
||||
clones, the safer emergency fallback is to omit `lastmod` entirely until full
|
||||
history is available. No URL or content restoration is required.
|
||||
Reference in new issue
Block a user