Files
paperclip/ui/src/components/task-chat/TaskChatRunnerTurn.tsx
T
DottaandPaperclip a0ac81e114 fix(ui): show task image results in compact thumbnail grids (#19626)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Task feeds show agent messages, documents, and files in time order.
> - Each image currently uses a full feed card.
> - A turn with seven screenshots makes the conversation hard to scan.
> - This pull request groups adjacent images from the same run in a
compact grid.
> - Users can scan the results and open each image in the existing
gallery.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Image results in task history and the live task feed.

**Current behavior**

Each image uses a separate full-width card. Native image upload receipt
messages can also separate related screenshots.

**Proposed behavior**

Adjacent images from one run share a wrapping thumbnail grid. New image
results join the grid. Documents, video, and authored messages keep
their positions. The grid opens the existing gallery.

**Reason and benefit**

Users can see a batch of screenshots without scrolling past one large
card for each file.

**Breaking changes**

The feed layout changes. The stored comments, files, APIs, and
permissions do not change.

Related public work: Refs #17228. Refs #8785. Refs #7667. These changes
supplied the artifact cards and media gallery that this pull request
reuses.

## What Changed

- Add one image grouping function for settled history, turn history, and
live results. Group only images with the same known run. Keep the first
row key and each image key when results arrive.
- Add a token-based grid that reuses the media artifact cards and image
gallery.
- Add an optional typed attachment-preparation marker to file
registration and chat-file reuse. Preserve the marker through comment
normalization. Use it with matching image, author, and run links to hide
generated receipts. Keep the exact historical sentence as a fallback for
old receipts.
- Add nine production component stories with seven sample-account
screenshots. Cover mobile, Classic, mixed results, single images,
missing images, streaming, gallery navigation, and the complete feed
projection.
- Add grouping and production-feed regressions. Add 18 browser cases
across light and dark themes.
- Improve contrast for the shared fallback label and gallery counter
after the accessibility checks found low contrast.
- Match alternate original-image paths when opening the gallery. Keep a
separate Open original link beside Download in the gallery.
- Record the feed layout in DESIGN.md.

## Verification

- Final qualification at `09b12381ce85bee6593050bd8a2fa18cbe2c45a6`:
[required CI
passed](https://github.com/paperclipai/paperclip/actions/runs/42317080212/job/128237094513),
including typecheck, build, all test lanes, E2E, and verify. The
snapshot has 58 successful check runs, three conditional skips, one
neutral advisory job, and a successful legacy Snyk status. No pending or
failed checks remain. Greptile and Paperclip Review both score this
exact head 5/5; all review threads are resolved.

- Current-head local checks: 544 focused UI tests passed across 11
files, with one existing skip. UI/shared typechecks and token gates
pass. All 18 focused Storybook browser cases passed after the CSS merge,
with retries disabled.
- Two database integration cases verify file registration and chat-file
reuse receipt markers. All 52 issue-validator tests and six upstream
default-model tests passed.
- Regressions failed before the fixes for unknown-run grouping, receipt
wording, and authored progress text with an image. The repair uses a
typed receipt marker and requires matching author, attachment, image,
and run links before hiding a generated receipt. All five review
findings are resolved.
- Browser checks cover loaded images, wrapping columns, overflow,
missing-image fallback, streaming stability, keyboard gallery
navigation, original-file links, and downloads. Fixtures use production
renderers. These checks do not prove deployed persistence or provider
behavior and do not compare visual baselines.
- Local limits: the preflight helper could not finish because Cargo was
absent. The earlier low-memory server typecheck and dependency-free
selector did not replace final-head CI. The now-passing final CI
supplies broad typecheck, build, server, Runner, and E2E coverage.
- Prior independent CI failures remain recorded: one Slack timer case
passed a single diagnosed rerun; its exact timer cause was not proved. A
later chat shard lost its self-hosted runner without a test assertion
failure. The current revision passes that shard and all applicable
gates. Master conflict resolution retained both image-grid and patch CSS
rules.

## Risks

- Images without a run ID remain separate. Messages and other resource
types end each batch.
- Receipt suppression requires a matching preparation marker or the
exact legacy sentence, plus matching attachment, image work product,
run, and author. Tests preserve human messages and native progress text
with attachments. The optional marker uses existing JSON metadata and
needs no database migration.
- Shared gallery and fallback text now have stronger contrast.

## Model Used

OpenAI GPT-6, through Codex. The agent used reasoning, repository tools,
code execution, and browser tests. The runtime did not expose a more
specific model ID or context window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <[email protected]>
2026-11-06 18:42:22 -06:00

327 lines
12 KiB
TypeScript

import { groupImageResources } from "./image-resources";
import { useRef, type ReactNode } from "react";
import type { ExecutionProjection } from "@paperclipai/shared";
import { useSecondTick } from "@/hooks/useSecondTick";
import { cn } from "@/lib/utils";
import type {
TaskChatItem,
TaskChatMessageItem,
TaskChatRuntimeRequestDecision,
TaskChatRuntimeRequestItem,
} from "./task-chat-model";
import { TaskChatAgentIdentity, TaskChatBubble } from "./TaskChatBubble";
import { TaskChatBubbleActions } from "./TaskChatBubbleActions";
import { formatTaskChatTimestamp } from "./task-chat-adapter";
import { TaskChatRunnerActivityGroup } from "./TaskChatRunnerActivityGroup";
import { TaskChatSubagentTree } from "./TaskChatSubagentCard";
import { TaskChatRunHealthBanner, partitionRunHealthItems } from "./TaskChatRunHealthBanner";
import { TaskChatProtocolCard } from "./TaskChatProtocolCard";
import { TaskChatPlanPreviewCard } from "./TaskChatPlanPreviewCard";
import { TaskChatMarker } from "./TaskChatMarker";
import { TaskChatShimmerScope, useTaskChatShimmer } from "./TaskChatShimmer";
import {
buildTurnTimelineRows,
isTerminalRunStatus,
omitProgressRepeatedByResponse,
paperclipRunnerFinalResponse,
paperclipRunnerTimelineItems,
} from "./transcript-adapter";
function currentActivityStatusItems(
items: readonly TaskChatItem[],
): readonly TaskChatItem[] {
let boundaryIndex = -1;
for (let index = items.length - 1; index >= 0; index -= 1) {
const item = items[index];
if (
item.kind === "message" ||
item.kind === "plan_document" ||
(item.kind === "protocol" &&
(item.surface === "runtime_request" ||
item.surface === "run_result" ||
item.surface === "run_terminal"))
) {
boundaryIndex = index;
break;
}
}
return items.slice(boundaryIndex + 1);
}
function formatCompactDuration(ms: number | null): string | null {
if (ms == null || !Number.isFinite(ms)) return null;
const totalSeconds = Math.max(0, Math.floor(ms / 1000));
if (totalSeconds < 60) return `${totalSeconds}s`;
const minutes = Math.floor(totalSeconds / 60);
const seconds = totalSeconds % 60;
return seconds === 0 ? `${minutes}m` : `${minutes}m ${seconds}s`;
}
function terminalStatusFailed(status: string): boolean {
return (
status === "failed" ||
status === "cancelled" ||
status === "timed_out" ||
status === "interrupted"
);
}
function RunnerTurnStatus({
status,
startedAtMs,
finishedAtMs,
continuedAfterSteering = false,
waitingLabel,
}: {
status: string;
startedAtMs: number | null;
finishedAtMs?: number | null;
continuedAfterSteering?: boolean;
waitingLabel?: string | null;
}) {
const terminal = isTerminalRunStatus(status);
useSecondTick(!terminal && startedAtMs != null);
const elapsedMs =
startedAtMs == null
? null
: Math.max(
0,
(terminal ? (finishedAtMs ?? Date.now()) : Date.now()) - startedAtMs,
);
const elapsed = formatCompactDuration(elapsedMs);
const failed = terminalStatusFailed(status);
const label = terminal ? (failed ? "Stopped" : "Worked") : waitingLabel ?? "Working";
const suffix = terminal
? elapsed ? ` ${failed ? "after" : "for"} ${elapsed}` : ""
: waitingLabel ? elapsed ? ` · ${elapsed}` : "" : ` for ${elapsed ?? "0s"}`;
return (
<span
className="min-w-0 truncate text-sm font-normal text-muted-foreground"
data-testid="task-chat-turn-status-header"
data-turn-position="identity"
aria-live="polite"
aria-atomic="true"
>
{continuedAfterSteering ? <span data-conversation-meta="">Continued after steering · </span> : null}
{label}<span data-conversation-meta="">{suffix}</span>
</span>
);
}
function RunnerCurrentActivityTail({ status, waitingLabel }: { status: string; waitingLabel?: string | null }) {
const shimmer = useTaskChatShimmer("working");
if (isTerminalRunStatus(status)) return null;
return <div className="mt-2 flex min-h-8 min-w-0 items-center gap-2 px-1 py-1 text-xs text-muted-foreground" data-testid="task-chat-current-activity" data-turn-position="tail">
<span className={cn(!waitingLabel && shimmer && "shimmer-text shimmer-text-muted")} aria-live="polite" data-testid="task-chat-current-activity-label">{waitingLabel ?? "Thinking"}</span>
</div>;
}
interface RunnerHeaderProps {
runId?: string | null;
agentName?: string | null;
agentIcon?: string | null;
agent?: import("../AgentAvatar").AvatarAgent;
status: string;
execution?: ExecutionProjection | null;
preparationLabel?: string;
startedAtMs: number | null;
finishedAtMs?: number | null;
continuedAfterSteering?: boolean;
}
function runnerWaitingLabel({ runId, status, execution, preparationLabel }: RunnerHeaderProps) {
if (isTerminalRunStatus(status)) return null;
if (execution?.phase === "waiting_for_provider") return execution.label;
if (!runId && preparationLabel) return preparationLabel;
return status === "queued" ? execution?.label ?? "Queued" : null;
}
/** One identity stays mounted while admission discovers the run's renderer. */
export function TaskChatRunnerHeader({ statusContent, ...props }: RunnerHeaderProps & { statusContent?: ReactNode }) {
return <div className="flex min-h-8 min-w-0 items-center gap-2 pb-1 pt-2" data-testid="task-chat-runner-identity-row">
{props.agentName ? <TaskChatAgentIdentity agentName={props.agentName} agentIcon={props.agentIcon} agent={props.agent} /> : null}
{statusContent ?? <RunnerTurnStatus status={props.status} startedAtMs={props.startedAtMs} finishedAtMs={props.finishedAtMs}
continuedAfterSteering={props.continuedAfterSteering} waitingLabel={runnerWaitingLabel(props)} />}
</div>;
}
export function TaskChatRunnerTurn({
runId,
agentName,
agentIcon,
agent,
items,
status,
execution,
preparationLabel,
startedAtMs,
finishedAtMs,
activityUnavailable = false,
allowSummaryFallback = true,
awaitingContinuation = false,
waitSummary = null,
continuedAfterSteering = false,
showHeader = true,
onRuntimeRequestDecision,
}: {
/** Stable identity used to clear replay-latched final text for the next turn. */
runId?: string | null;
agentName?: string | null;
agentIcon?: string | null;
agent?: import("../AgentAvatar").AvatarAgent;
items: readonly TaskChatItem[];
status: string;
execution?: ExecutionProjection | null;
/** Presentation only while Send/Retry admission has no run yet. */
preparationLabel?: string;
startedAtMs: number | null;
finishedAtMs?: number | null;
activityUnavailable?: boolean;
allowSummaryFallback?: boolean;
/**
* Accepted wait/interaction authority: the turn is waiting, so an early
* provider final that only restates the wait is not presented as a reply.
* Prose the agent wrote for the user stays visible.
*/
awaitingContinuation?: boolean;
/** The accepted yielded result's wait summary, when known. */
waitSummary?: string | null;
/** The visible tail resumes the same native run after an accepted steer. */
continuedAfterSteering?: boolean;
/** The task thread keeps a shared header across native and legacy discovery. */
showHeader?: boolean;
onRuntimeRequestDecision?: (
item: TaskChatRuntimeRequestItem,
decision: TaskChatRuntimeRequestDecision,
) => void | Promise<void>;
}) {
const { failures: mcpStartupFailures, rest: turnItems } = partitionRunHealthItems(items);
const terminal = isTerminalRunStatus(status);
const waitingLabel = runnerWaitingLabel({ runId, status, execution, preparationLabel, startedAtMs });
const yielded = turnItems.some(
(item) =>
item.kind === "protocol" &&
item.surface === "run_result" &&
item.disposition === "yielded",
);
const observedFinal = paperclipRunnerFinalResponse(turnItems, awaitingContinuation
? { allowFallback: false, yielded: true, waitSummary }
: { allowFallback: terminal && allowSummaryFallback });
const observedProviderText = Boolean(
observedFinal &&
turnItems.some(
(item) =>
item.kind === "message" &&
item.id === observedFinal.id &&
item.channel !== "progress",
),
);
// A reconnect/replay can briefly rebuild the transcript without the final
// item (or with an earlier, shorter prefix). Provider-authored final text
// always replaces a structured summary fallback, even when it is shorter;
// within either class, displayed answer text remains monotonic.
const finalRef = useRef<{
runId?: string | null;
item?: TaskChatMessageItem;
providerText?: boolean;
}>({ runId });
if (finalRef.current.runId !== runId) finalRef.current = { runId };
// A provider final can arrive before the accepted yielded result. Clear any
// replay latch once the control plane establishes that this turn is waiting
// for continuation; only an authored (non-wait) final is re-latched below.
if (yielded || awaitingContinuation) finalRef.current = { runId };
if (
observedFinal &&
(!finalRef.current.item ||
(observedProviderText && !finalRef.current.providerText) ||
(observedProviderText === Boolean(finalRef.current.providerText) &&
observedFinal.text.length >= finalRef.current.item.text.length))
) {
finalRef.current.item = observedFinal;
finalRef.current.providerText = observedProviderText;
}
const final = finalRef.current.item;
const timelineItems = paperclipRunnerTimelineItems(turnItems);
const currentActivityItems = currentActivityStatusItems(timelineItems);
const timelineRows = groupImageResources(buildTurnTimelineRows(
omitProgressRepeatedByResponse(timelineItems, final?.text),
!terminal,
));
return (
<TaskChatShimmerScope items={items}><div
className="flex min-w-0 flex-col"
data-testid="task-chat-runner-turn"
data-phase={status === "queued" ? "startup" : undefined}
>
{showHeader ? <TaskChatRunnerHeader runId={runId} agentName={agentName} agentIcon={agentIcon} agent={agent}
status={status} execution={execution} preparationLabel={preparationLabel} startedAtMs={startedAtMs}
finishedAtMs={finishedAtMs} continuedAfterSteering={continuedAfterSteering} /> : null}
<TaskChatRunHealthBanner failures={mcpStartupFailures} />
{activityUnavailable ? (
<div
className="px-1 py-1 text-xs text-muted-foreground"
role="status"
data-testid="task-chat-activity-unavailable"
>
Live runner activity is temporarily unavailable. Retrying…
</div>
) : null}
{timelineRows.length > 0 ? (
<div
className="flex min-w-0 flex-col gap-2 py-1"
data-testid="task-chat-turn-timeline"
>
{timelineRows.map((row) => (
<div
className="min-w-0"
key={`${runId ?? "run"}:${row.id}`}
data-testid="task-chat-turn-timeline-row"
data-timeline-row-id={row.id}
data-thread-anchor={row.id}
>
{row.kind === "activity_phase" ? (
<TaskChatRunnerActivityGroup item={row} />
) : row.kind === "plan_document" ? (
<TaskChatPlanPreviewCard
source={{ kind: "saved", document: row.document }}
testId={
row.placement === "fallback"
? "task-chat-plan-preview-fallback"
: "task-chat-plan-preview"
}
/>
) : row.kind === "marker" ? (
<TaskChatMarker item={row} />
) : row.kind === "subagent_group" ? (
<TaskChatSubagentTree subagents={row.subagents} />
) : row.kind === "protocol" ? (
<TaskChatProtocolCard
item={row}
onRuntimeRequestDecision={onRuntimeRequestDecision}
/>
) : null}
</div>
))}
</div>
) : null}
{final ? (
<div
className="w-full"
data-testid="task-chat-final-response"
>
<TaskChatBubble
item={{ ...final, authorName: agentName ?? undefined, agentIcon, agent, timestamp: final.timestamp ?? formatTaskChatTimestamp(final.atMs) }}
animateEntry={false}
hideAgentIdentity={!continuedAfterSteering}
actions={<TaskChatBubbleActions copyText={final.text} />}
/>
</div>
) : null}
{!final && currentActivityItems.length === 0 ? <RunnerCurrentActivityTail status={status} waitingLabel={waitingLabel} /> : null}
</div></TaskChatShimmerScope>
);
}