The e2e suite and CI gates could report green without checking anything: the
lint/codegen gate never ran on fork PRs, the codegen freshness check diffed the
wrong range, the sticky comment claimed passes it never read, and several
visual/palette/contrast/a11y gates were written so they could not fail. Closing
those gates then exposed real defects.
**E2E and CI.** New coverage for the assistant lifecycle, flow interrupt, live
panels, tab deep links, the report route and its exports, session expiry, and
uploads/downloads (59 → 125 spec cases). Gates now fail on real regressions:
absolute pixel budget, per-tab panel sweeps, scoped a11y waivers, page-error
assertions. CI runs on `pull_request`, ranges from the PR base, routes comments
by PR identity, and redacts stand secrets from the public artifact.
**Markdown editor.** ~20 round-trip and toolbar fixes: table pipe escaping
(nested lists, Go template pipelines, headerless tables), fence validity,
setext and line-leading escapes, inline code containing a backtick, block
toggles under select-all, controls disabled where the command is unavailable or
a table cell cannot hold the block, theme-aware code surface. Two perf fixes cut
1 MB document load from 69s to 4s.
**Accessibility and UI.** Focus returns to whatever opened a dialog, sheet or
menu; icon-only buttons named; `aria-sort` on table columns; syntax tokens
retuned to clear AA in both themes; a dedicated link token. Detail and settings
pages now distinguish authz denials and partial errors from missing records,
survive background refetches without blanking or discarding unsaved edits, and
stop naming exported PDFs `*.pdf.pdf`.
**Backend.** Passwords capped at the 72 bytes bcrypt can hash (over-long ones
previously 500'd after passing validation); four agent options the GraphQL round
trip silently dropped are carried through; a flow whose worker fails to start is
no longer left in the listing; the Graphiti health check retries before
permanently disabling the client.
---------
Co-authored-by: Sergey Kozyrenko <sirozha_blr@tut.by>
The comment on AppHeaderActions claimed the component pins itself to the right.
It does not — the sibling content block takes `flex-1` and pushes it there — and
the rule it stated was an instruction to call sites, not a fact about the twelve
characters below it. The convention now lives in docs/list_detail_pages.md, next
to the detail-page recipe that needs it.
Two more went the same way, by making the code carry the point instead: the
knowledge header's two ids are now `documentId` and `routeKnowledgeId`, so
there is nothing to warn about, and the separator that doubles up is gated on a
named `hasViewRow`. In the pager spec the injected delay is held by an assertion
that counts the samples taken before the sibling appears — removing the delay
now fails the test instead of quietly emptying it.
Across the e2e specs, six comments that only explained why a test exists are
gone and five more are down to one line.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The intro left /settings/prompts out of the list pages built on these
pieces and implied every one of them has a detail page; api-tokens edits
in place. The removed/renamed table pointed at stale mentions of the
deleted writer hook in two files that no longer contain any.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The stand tier's fork/secret protection is the protected Environment's reviewer
gate, not the job's label condition — a fork PR can carry the label but blocks
on a human before any secret. Say so, note the tier now lives in its own
workflow, and document the E2E_STAND_* secrets and their E2E_* tool names.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The probes carry `transition-colors`, so reading the computed background right
after hover() sampled a colour still interpolating away from the rest state —
the half of the gate that exists to catch hover regressions could not catch one.
Disabling the transition on the probes fixes it: verified by giving one variant a
deliberately unreadable hover fill, which the assertion now reports as "badge
blue on hover" and previously let through.
Key the probe list off the variant union so a new badge variant fails to compile
until it is probed, rather than silently going unmeasured.
Also correct two route-manifest ownerships that misdirect diff-scoping: the flows
provider is mounted by the detail route as well as the list, and the provider
detail page was attributed to the list route that never renders it.
Doc fixes: the route sweep is tagged @cross, not @coverage; the licenses link in
CONTRIBUTING resolved outside the repository; and the frontend prettierignore
duplicated entries prettier already reads from gitignore.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The suite tags every spec but the docs only hinted at "@smoke, …". List the real
tags (@smoke/@flows/@crud/@coverage/@cross/@visual/@real/@stand) with their meaning
and how to filter, and spell out the two conventions the mock gate reserves:
@quarantine to isolate a flaky spec (excluded via --grep-invert) and @generated for
recorder-derived cassettes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The base compose file pins fixed network names, so any network left un-overridden
in this e2e override joins the stack to the developer's live observability and
langfuse networks. Give all three networks e2e-scoped names. Add a healthcheck to
the mock-LLM service and make pentagi depend on service_healthy, so a crash-looping
mock can never read as a started dependency and silently mask itself.
Update frontend/docs/e2e.md to match reality: the stand tier runs @stand only
(flow-run drives a real paid agent and stays local), every mock run rebuilds its
own bundle by design, and the trend/affected-routes notes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The co-located components/shared/README.md documented hooks that no longer
exist. `usePagination` was never a hook, and the writer `useTableQueryFilter`
was replaced by the unified `useTableState`, so the copy-paste recipe it shipped
would not compile. Detail pages had likewise moved from spreading a `toolbarProps`
object to passing a `controller` prop.
Rewrite it as frontend/docs/list_detail_pages.md, matching the flat
per-subsystem layout backend/docs already uses: exact signatures for every
hook, component and storage helper, the wiring the five live list pages
actually use, and an explicit table of removed names so they are not
reintroduced. Drop the source-adjacent README — no other directory under
frontend/src or backend/pkg carries one.
Fix the comments that still named the removed hook, plus two claims in them
that contradict the code: the controlled filter is URL-backed only (DataTable
persists sorting, column visibility, page size and search columns, never the
filter), and `busy` on InlineEditInput disables the Save and Cancel buttons
while leaving the input editable.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
schema-compat set NODE_TLS_REJECT_UNAUTHORIZED=0 process-wide (CodeQL
js/disabling-certificate-validation, high). A real stand has a valid
cert; only the local self-signed Tier-2 stack needs it, so the operator
now opts in via their own shell env, never in code.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Lays the Phase-3 substrate on top of the three tiers.
- schema-compat pre-flight (e2e/tools/schema-compat.mjs): introspects a
target backend's live GraphQL schema and validates every frontend
operation against it, so deploy skew (a renamed/removed field) fails
once, readably, instead of as dozens of red specs. Verified against the
live Tier-2 backend both ways: 105 operations pass, an injected bogus
field is caught with the exact location.
- trend aggregation (trend.mjs): turns a run's results.json into one JSONL
record (p50/p95 spec duration, slowest three, pass/flaky/fail) so slow
regressions are visible, not just green/red; CI appends it to a
90-day-retained artifact.
- diff-scoping (affected-routes.ts + affected.ts): maps a diff to the
manifest routes it touches via each route's owning sources — the
substrate for selective runs and for scoping the exploratory agent.
Pure mapping fn, unit-tested (backend-only → none; shared infra → all;
owned → that route).
- CI: an e2e-stand job (label-gated + a protected Environment whose
reviewers approve before secrets are exposed; schema-compat runs first),
a trend step, and an LLM-independent @stand smoke.
- docs: the stand tier, trend/affected tools, and the LLM advisory recipe
(playwright-mcp + init-agents + guardrails) — the recipe the
deterministic tiers plug into, not a bespoke bot.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
18 baselines (9 manifest routes × light/dark) compared only inside
mcr.microsoft.com/playwright:v<version>-noble, so pixels are identical on
every machine and CI; the wrapper derives the tag from the installed
@playwright/test version, making the pin drift-proof, and a CI guard
fails loudly if the workflow's container tag falls behind.
macOS hosts cannot run the visual project directly (parallel darwin
baselines) nor mount their node_modules into the container (native vite
binaries): the wrapper builds dist on the host and the container serves
it with a dependency-free static server — route mocks intercept API
calls before the network, so no proxy is needed. The xterm canvas is
masked (SwiftShader pixels are driver-dependent). Determinism proven by
back-to-back container runs. The e2e-visual CI job is advisory, never a
required check.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
specs/real/** now exercise the actual agent loop end to end: create flow
-> image/language/title -> tool-call-ID sampling -> subtask_list plan ->
done barrier -> subtask_patch refine -> report_result — with messages
streamed over the real GraphQL websocket and the flow settling in
Waiting. The mock LLM is ~150 lines of Node driven by a deterministic
first-match transcript; the custom provider env seam means zero backend
changes.
The stack is fully isolated from a developer machine:
- own compose project/network/ports (8444/5433), coexists with a dev stack
- --env-file /dev/null so the developer's .env (live keys, DOCKER_HOST,
config paths) never leaks into the e2e backend
- flow ids seeded from 90001: sandbox containers are named
pentagi-terminal-<flowId> on the shared docker daemon, so the range
keeps them clear of dev flows and makes runner cleanup unambiguous
(down -v cannot remove them — the backend spawns them outside compose)
- pentagi healthcheck via busybox wget (the alpine image has no curl),
since the base service has none and up --wait returns too early
Auth for live tiers moved to the canonical setup-project + storageState;
the localStorage seed is now mock-tier-only (a forged client session has
no cookie behind it and the first 401 wipes it). CI gains a nightly/
dispatch-only e2e-local job.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The default tier runs the production bundle (vite build + preview)
against a cassette-driven mock of the whole API surface — GraphQL over
HTTP, graphql-transport-ws subscriptions, and REST — so the suite needs
no backend, no keys, and no secrets, and fork PRs can run it.
- mock engine: (operationName, variables) matching with sequenced
entries and world flags (login flips /info guest->user without
call-order coupling); unmatched calls answer 501 and fail the test,
so nothing leaks through the vite preview proxy to a live backend
- ws mock follows the graphql-transport-ws contract the app's client
needs: immediate ack, nothing before ack, streams stay open, delta
cursors survive reconnects, drops use retryable close codes
- clock and timezone pinned on the mock tier: formatDate branches on
isToday/isThisYear, so unpinned cassette dates rot within a day
- cassettes are TS modules typed against the generated GraphQL types,
so schema/operation drift fails the existing tsc gate at compile time
- CI: fork-safe e2e.yml (read-only token, no secrets, no write steps)
plus e2e-report.yml posting a sticky PR comment via workflow_run,
resolving fork PRs by head SHA
- smoke specs (login redirect, form login, authenticated /flows render)
green on the mock tier; e2e/ wired into tsc, eslint, and prettier
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dmitry Ng