From bc3692ee69dd2151c144fa2cda09f5371b60e99a Mon Sep 17 00:00:00 2001 From: MervinPraison <454862+MervinPraison@users.noreply.github.com> Date: Mon, 26 Oct 2026 11:36:14 +0000 Subject: [PATCH] docs(replay): user guide and clean-room ADR docs/features/replay.md describes the page, keys, approvals, export and HTTP API. docs/adr/0001-replay-view-clean-room.md records the ideas we took (graph plus trace, score-like swimlanes, OTel GenAI vocabulary), that no third-party UI code, CSS, icons, layouts or screenshots were used, and the palette, accessibility and licence reasoning. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/adr/0001-replay-view-clean-room.md | 92 +++++++++++++++++++++++ docs/features/replay.md | 97 +++++++++++++++++++++++++ mkdocs.yml | 1 + 3 files changed, 190 insertions(+) create mode 100644 docs/adr/0001-replay-view-clean-room.md create mode 100644 docs/features/replay.md diff --git a/docs/adr/0001-replay-view-clean-room.md b/docs/adr/0001-replay-view-clean-room.md new file mode 100644 index 0000000..bb76490 --- /dev/null +++ b/docs/adr/0001-replay-view-clean-room.md @@ -0,0 +1,92 @@ +# ADR 0001: PraisonAI Replay view, a clean-room design + +- **Status:** accepted +- **Date:** 2026-10-26 +- **Scope:** `src/praisonaiui/features/replay.py`, `features/_trace_jsonl.py`, + `templates/frontend/plugins/views/replay-core.js`, `templates/frontend/plugins/views/replay.js` + +## Context + +PraisonAI records agent runs as context-trace JSONL (one `ContextEvent.to_dict()` per +line). People need to see what a team or flow did: who did what, when, why, and at what +cost, both after the fact and live. Other tools have shown that a graph plus a trace is a +useful way to look at agent runs. We want that value without copying anyone's product, +code or look, and without adding dependencies. + +## Decision + +Build our own view, in plain DOM and SVG, with our own layout code, palette and words. + +### Ideas we took (ideas only) + +| Idea | Where it came from | What we did with it | +|---|---|---| +| Show the graph of a multi-agent program next to the trace of a run | The general concept behind LangGraph Studio's graph-plus-trace view, known from public talks and articles | A **Map** view drawn by our own layered layout (`layoutMap`); the default view is a **Timeline**, which Studio does not have | +| Parallel voices on shared time, read left to right | Musical scores and swimlane diagrams | One lane per agent plus Manager and You; parallel branches as sub-lanes; tasks and steps as "chapters" across the top | +| Vocabulary for LLM and tool spans, tokens and cost | OpenTelemetry GenAI semantic conventions (public specification) | Block kinds (LLM, tool), token and cost totals; the wrapper's OTLP export uses the conventions, the UI only borrows the concepts | + +### What we did not use + +- **No third-party UI code, CSS, icons, layouts or screenshots** were used to build this + view. Shapes, palette, spacing, wording ("Brief", "Delivered", "Waiting for you") and + the layout algorithm are our own. +- **LangGraph's server (Elastic License 2.0) and the proprietary LangGraph Studio were + not consulted for code**, and none of their assets, markup or styles were copied. We + do not use the name "Studio", any "Lang…" naming, or `__start__`/`__end__` pills. +- No Mermaid, D3, React Flow, dagre, ELK or other graph libraries. The dashboard's + existing Mermaid CDN plugin is not used by Replay. + +### How it is built + +- `replay-core.js`: pure functions (`buildModel`, `narrate`, `layoutMap`, + `deriveGraph`). No DOM, no network. Tested with `node --test`. +- `replay.js`: DOM/SVG rendering, keyboard handling, SSE follow and approvals. It also + boots by itself when `window.__REPLAY_DATA__ = {events, graph}` is set, which is how + the wrapper's single-file HTML export works (`standalone_html()` inlines both files). +- `layoutMap` is a textbook layered layout written from scratch: depth-first search to + find back edges (loops), longest-path layering, one virtual waypoint per crossed layer + for long edges, and a few barycentre sweeps to order each layer. It is deterministic. +- Narration is template text, chosen per event type. No LLM, no randomness. + +## Palette and accessibility rationale + +| Role | Fill | Shape | Why | +|---|---|---|---| +| Agent | `#8B0000` | circle | PraisonAI's brand red; white text on it is 10.0:1 | +| Tool | `#0F7287` | rounded rectangle | Teal kept from the brand but darkened from `#189AB4` (white text on `#189AB4` fails AA); white on `#0F7287` is 5.6:1 | +| You (human) | `#8A5A00` | diamond | Amber distinct from red and teal for common colour-vision deficiencies; white on it is 5.9:1 | +| Manager | `#3C4A5C` | square | Neutral slate, so the manager reads as coordination, not work; white on it is 9.0:1 | + +- **Never colour alone.** Every lane, block, node and step has a shape and a text label; + arrows differ by dash pattern and carry words ("hands off", "delegates", "routes"). + Heat in the cost/latency overlay is a bar whose height and shade both grow, plus the + value in text. +- **Light theme first.** Text tokens are at least 6.9:1 on their backgrounds in light mode + and 7.7:1 in dark mode. In dark mode the palette fills keep white text at AA, and get a + light 2 px outline so each shape's boundary stays above 3:1 against the dark page. +- **Keyboard throughout.** ← → (and ↑ ↓ in the step list) move the replay cursor, Home + and End jump to the ends, tabs follow the ARIA tabs pattern, map nodes use a roving + tab stop, and every action has a real button. +- **Screen readers.** The narration column is a real ordered list and is the main way + through a run. Timeline blocks and map nodes are named buttons whose names start with + their visible text (WCAG 2.5.3). Live updates are announced through a polite live + region, throttled. +- **Motion.** The only animation (the live dot) runs only under + `prefers-reduced-motion: no-preference`; scrolling is instant when motion is reduced. +- Checked with axe-core against WCAG 2.0, 2.1 and 2.2 A/AA in both themes (dev-only test + in `tests/browser/`). + +## Dependency licences + +No new runtime or build dependencies. The Python side uses only the standard library and +Starlette, which aiui already depends on. The browser side is plain ES modules with no +imports beyond our own file. Dev-only test tools that are not shipped: Playwright +(Apache-2.0, already used in this environment) and axe-core (MPL-2.0, loaded from a path +given by `AXE_CORE_JS`; not vendored). + +## Consequences + +- We own the look and the layout code, and can change them without licence questions. +- The layout is simpler than a dedicated graph engine. It is fine for the team and flow + sizes we see (tens of nodes). Very large graphs may need a better crossing-reduction + step later. diff --git a/docs/features/replay.md b/docs/features/replay.md new file mode 100644 index 0000000..8ad478b --- /dev/null +++ b/docs/features/replay.md @@ -0,0 +1,97 @@ +# Replay + +**PraisonAI Replay** shows a recorded agent run as a story: who did what, when, why, and +at what cost. It works for finished runs and for runs that are still going. + +## Record a run, then open it + +```bash +praisonai replay view agents.yaml # records the run and opens Replay on it +``` + +or from Python: + +```python +from praisonai.replay import record + +with record(live=True): + team.start() +``` + +Every recorded run is a file `.jsonl` in the traces folder. Replay reads it from +`$PRAISONAI_TRACES_DIR`, else `$PRAISON_HOME/traces`, else `~/.praison/traces`. + +To open aiui on Replay yourself: + +```bash +aiui run "$(python -c 'import praisonaiui.replay_app as m; print(m.__file__)')" \ + --style dashboard --host 127.0.0.1 --port 8090 +# then open http://127.0.0.1:8090/replay?run= +``` + +`/#/replay?run=` works too. In a full dashboard, Replay is in the sidebar under +**Control**. + +## What you see + +| Part | What it shows | +|---|---| +| **Runs** | Recorded runs, newest first. A run that is still being written has a **Live** badge. | +| **Timeline** (default) | One lane per agent, plus **Manager** and **You**. Time runs left to right. LLM calls and tool calls are blocks in their agent's lane. Arrows show hand-offs, delegation, routing and approval requests. Parallel branches get their own sub-lanes. A call or step repeated back to back is one block marked ×N. Tasks and steps run across the top. | +| **Map** | The team or flow as a graph, drawn from the graph recorded at the start of the run. Older recordings without a graph get a "who talked to whom" map built from the events. Nodes you have not reached yet have a dashed outline; the node the cursor is on has a bold one. | +| **What happened** | One plain sentence per step, for example *"Researcher asked web_search for 'solar sail facts' — 1.2 s, $0.002."* Hovering or focusing a sentence highlights its block, and the other way round. | +| **Details** | Three tabs for the selected step. **What happened**: when, who, how long, cost, tokens, output. **Why**: only what the recording holds, such as the input given to a tool, the context passed in a hand-off and its reason, the manager's task snapshot, the route a step chose, and an approval's decision and where an automatic approval came from. **Raw JSON**: the events themselves. | +| **Cost & latency** | Adds a bar under each block (taller and darker means more cost, or more time) and per-agent totals in the lane labels. | + +Shapes always carry meaning as well as colour: agents are circles, tools are rounded +rectangles, you are a diamond and the manager is a square. + +## Moving through a run + +Replay opens at the end of the run, so you see everything. Move the cursor to replay it: +anything after the cursor is drawn as an outline. + +| Key | Action | +|---|---| +| ← → | Previous / next step | +| ↑ ↓ | Previous / next step in the step list; previous / next node in the map | +| Home / End | First / last step | +| Enter or Space | Open the focused block or node | +| ← → in the tabs | Switch between What happened, Why and Raw JSON | + +The scrubber under the timeline does the same with a mouse or touch. On a live run the +cursor follows new events; if you step back, **Jump to live** takes you to the latest. + +## Approving from Replay + +When an agent asks for approval and the run uses aiui's approval store, a **Waiting for +you** card appears above the timeline: + +- **Approve** or **Deny**, with an optional reason. +- **Edit arguments…** opens the arguments as JSON (taken from the server, not the redacted + recording when the server has them). The JSON must be an object; mistakes are shown + as you type. **Approve with these arguments** sends them as `modified_args`. + +These buttons call `POST /api/approvals/{id}/approve` and `/deny`. The agent waits until +you answer, and is denied if nobody answers before the approval timeout. + +## Sharing a run as one file + +`praisonai replay export RUN --format html` writes a single HTML file with the run and +this viewer inside. It needs no server and no network. Approvals cannot be answered from +an exported file. + +## HTTP API + +| Endpoint | Returns | +|---|---| +| `GET /api/replay/runs` | `[{run_id, session_id, path, started_at, ended_at?, status, kind, name, agents, events, cost_usd, live}]`, newest first | +| `GET /api/replay/runs/{id}/events?after_seq=N` | `{events, graph, run}`; `after_seq` keeps only events with a larger `sequence_num` | +| `GET /api/replay/runs/{id}/stream` | Server-sent events: `event: ctx` per trace event (with `id:` = `sequence_num`), `ping` every 15 s, `end` once the run has finished and gone quiet for 2 s. Honours `after_seq` and `Last-Event-ID`. | + +Run ids must match `^[A-Za-z0-9._-]+$`. Replay is read-only: it never writes trace +files. Half-written last lines, unknown fields and unknown event types are skipped +safely, and schema 1.0 recordings still open. + +The **Traces** page reads the same files: each run is a trace, and its spans come from +matching start and end events. diff --git a/mkdocs.yml b/mkdocs.yml index f0257fd..3662bb3 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -100,6 +100,7 @@ nav: - BeautifulUI Upstream Pin: frontend/BEAUTIFULUI_UPSTREAM.md - Features: - Dashboard: features/dashboard.md + - Replay: features/replay.md - Gateway Chat: features/gateway-chat.md - Attachments: features/attachments.md - Theme System: features/theme-system.md