mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-07 06:16:06 -10:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task feeds show agent messages, documents, and files in time order. > - Each image currently uses a full feed card. > - A turn with seven screenshots makes the conversation hard to scan. > - This pull request groups adjacent images from the same run in a compact grid. > - Users can scan the results and open each image in the existing gallery. ## Linked Issues or Issue Description **What existing behavior does this improve?** Image results in task history and the live task feed. **Current behavior** Each image uses a separate full-width card. Native image upload receipt messages can also separate related screenshots. **Proposed behavior** Adjacent images from one run share a wrapping thumbnail grid. New image results join the grid. Documents, video, and authored messages keep their positions. The grid opens the existing gallery. **Reason and benefit** Users can see a batch of screenshots without scrolling past one large card for each file. **Breaking changes** The feed layout changes. The stored comments, files, APIs, and permissions do not change. Related public work: Refs #17228. Refs #8785. Refs #7667. These changes supplied the artifact cards and media gallery that this pull request reuses. ## What Changed - Add one image grouping function for settled history, turn history, and live results. Group only images with the same known run. Keep the first row key and each image key when results arrive. - Add a token-based grid that reuses the media artifact cards and image gallery. - Add an optional typed attachment-preparation marker to file registration and chat-file reuse. Preserve the marker through comment normalization. Use it with matching image, author, and run links to hide generated receipts. Keep the exact historical sentence as a fallback for old receipts. - Add nine production component stories with seven sample-account screenshots. Cover mobile, Classic, mixed results, single images, missing images, streaming, gallery navigation, and the complete feed projection. - Add grouping and production-feed regressions. Add 18 browser cases across light and dark themes. - Improve contrast for the shared fallback label and gallery counter after the accessibility checks found low contrast. - Match alternate original-image paths when opening the gallery. Keep a separate Open original link beside Download in the gallery. - Record the feed layout in DESIGN.md. ## Verification - Final qualification at `09b12381ce85bee6593050bd8a2fa18cbe2c45a6`: [required CI passed](https://github.com/paperclipai/paperclip/actions/runs/42317080212/job/128237094513), including typecheck, build, all test lanes, E2E, and verify. The snapshot has 58 successful check runs, three conditional skips, one neutral advisory job, and a successful legacy Snyk status. No pending or failed checks remain. Greptile and Paperclip Review both score this exact head 5/5; all review threads are resolved. - Current-head local checks: 544 focused UI tests passed across 11 files, with one existing skip. UI/shared typechecks and token gates pass. All 18 focused Storybook browser cases passed after the CSS merge, with retries disabled. - Two database integration cases verify file registration and chat-file reuse receipt markers. All 52 issue-validator tests and six upstream default-model tests passed. - Regressions failed before the fixes for unknown-run grouping, receipt wording, and authored progress text with an image. The repair uses a typed receipt marker and requires matching author, attachment, image, and run links before hiding a generated receipt. All five review findings are resolved. - Browser checks cover loaded images, wrapping columns, overflow, missing-image fallback, streaming stability, keyboard gallery navigation, original-file links, and downloads. Fixtures use production renderers. These checks do not prove deployed persistence or provider behavior and do not compare visual baselines. - Local limits: the preflight helper could not finish because Cargo was absent. The earlier low-memory server typecheck and dependency-free selector did not replace final-head CI. The now-passing final CI supplies broad typecheck, build, server, Runner, and E2E coverage. - Prior independent CI failures remain recorded: one Slack timer case passed a single diagnosed rerun; its exact timer cause was not proved. A later chat shard lost its self-hosted runner without a test assertion failure. The current revision passes that shard and all applicable gates. Master conflict resolution retained both image-grid and patch CSS rules. ## Risks - Images without a run ID remain separate. Messages and other resource types end each batch. - Receipt suppression requires a matching preparation marker or the exact legacy sentence, plus matching attachment, image work product, run, and author. Tests preserve human messages and native progress text with attachments. The optional marker uses existing JSON metadata and needs no database migration. - Shared gallery and fallback text now have stronger contrast. ## Model Used OpenAI GPT-6, through Codex. The agent used reasoning, repository tools, code execution, and browser tests. The runtime did not expose a more specific model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <[email protected]>
327 lines
12 KiB
TypeScript
327 lines
12 KiB
TypeScript
import { groupImageResources } from "./image-resources";
|
|
import { useRef, type ReactNode } from "react";
|
|
import type { ExecutionProjection } from "@paperclipai/shared";
|
|
import { useSecondTick } from "@/hooks/useSecondTick";
|
|
import { cn } from "@/lib/utils";
|
|
import type {
|
|
TaskChatItem,
|
|
TaskChatMessageItem,
|
|
TaskChatRuntimeRequestDecision,
|
|
TaskChatRuntimeRequestItem,
|
|
} from "./task-chat-model";
|
|
import { TaskChatAgentIdentity, TaskChatBubble } from "./TaskChatBubble";
|
|
import { TaskChatBubbleActions } from "./TaskChatBubbleActions";
|
|
import { formatTaskChatTimestamp } from "./task-chat-adapter";
|
|
import { TaskChatRunnerActivityGroup } from "./TaskChatRunnerActivityGroup";
|
|
import { TaskChatSubagentTree } from "./TaskChatSubagentCard";
|
|
import { TaskChatRunHealthBanner, partitionRunHealthItems } from "./TaskChatRunHealthBanner";
|
|
import { TaskChatProtocolCard } from "./TaskChatProtocolCard";
|
|
import { TaskChatPlanPreviewCard } from "./TaskChatPlanPreviewCard";
|
|
import { TaskChatMarker } from "./TaskChatMarker";
|
|
import { TaskChatShimmerScope, useTaskChatShimmer } from "./TaskChatShimmer";
|
|
import {
|
|
buildTurnTimelineRows,
|
|
isTerminalRunStatus,
|
|
omitProgressRepeatedByResponse,
|
|
paperclipRunnerFinalResponse,
|
|
paperclipRunnerTimelineItems,
|
|
} from "./transcript-adapter";
|
|
|
|
function currentActivityStatusItems(
|
|
items: readonly TaskChatItem[],
|
|
): readonly TaskChatItem[] {
|
|
let boundaryIndex = -1;
|
|
for (let index = items.length - 1; index >= 0; index -= 1) {
|
|
const item = items[index];
|
|
if (
|
|
item.kind === "message" ||
|
|
item.kind === "plan_document" ||
|
|
(item.kind === "protocol" &&
|
|
(item.surface === "runtime_request" ||
|
|
item.surface === "run_result" ||
|
|
item.surface === "run_terminal"))
|
|
) {
|
|
boundaryIndex = index;
|
|
break;
|
|
}
|
|
}
|
|
return items.slice(boundaryIndex + 1);
|
|
}
|
|
|
|
function formatCompactDuration(ms: number | null): string | null {
|
|
if (ms == null || !Number.isFinite(ms)) return null;
|
|
const totalSeconds = Math.max(0, Math.floor(ms / 1000));
|
|
if (totalSeconds < 60) return `${totalSeconds}s`;
|
|
const minutes = Math.floor(totalSeconds / 60);
|
|
const seconds = totalSeconds % 60;
|
|
return seconds === 0 ? `${minutes}m` : `${minutes}m ${seconds}s`;
|
|
}
|
|
|
|
function terminalStatusFailed(status: string): boolean {
|
|
return (
|
|
status === "failed" ||
|
|
status === "cancelled" ||
|
|
status === "timed_out" ||
|
|
status === "interrupted"
|
|
);
|
|
}
|
|
|
|
function RunnerTurnStatus({
|
|
status,
|
|
startedAtMs,
|
|
finishedAtMs,
|
|
continuedAfterSteering = false,
|
|
waitingLabel,
|
|
}: {
|
|
status: string;
|
|
startedAtMs: number | null;
|
|
finishedAtMs?: number | null;
|
|
continuedAfterSteering?: boolean;
|
|
waitingLabel?: string | null;
|
|
}) {
|
|
const terminal = isTerminalRunStatus(status);
|
|
useSecondTick(!terminal && startedAtMs != null);
|
|
const elapsedMs =
|
|
startedAtMs == null
|
|
? null
|
|
: Math.max(
|
|
0,
|
|
(terminal ? (finishedAtMs ?? Date.now()) : Date.now()) - startedAtMs,
|
|
);
|
|
const elapsed = formatCompactDuration(elapsedMs);
|
|
|
|
const failed = terminalStatusFailed(status);
|
|
const label = terminal ? (failed ? "Stopped" : "Worked") : waitingLabel ?? "Working";
|
|
const suffix = terminal
|
|
? elapsed ? ` ${failed ? "after" : "for"} ${elapsed}` : ""
|
|
: waitingLabel ? elapsed ? ` · ${elapsed}` : "" : ` for ${elapsed ?? "0s"}`;
|
|
|
|
return (
|
|
<span
|
|
className="min-w-0 truncate text-sm font-normal text-muted-foreground"
|
|
data-testid="task-chat-turn-status-header"
|
|
data-turn-position="identity"
|
|
aria-live="polite"
|
|
aria-atomic="true"
|
|
>
|
|
{continuedAfterSteering ? <span data-conversation-meta="">Continued after steering · </span> : null}
|
|
{label}<span data-conversation-meta="">{suffix}</span>
|
|
</span>
|
|
);
|
|
}
|
|
|
|
function RunnerCurrentActivityTail({ status, waitingLabel }: { status: string; waitingLabel?: string | null }) {
|
|
const shimmer = useTaskChatShimmer("working");
|
|
if (isTerminalRunStatus(status)) return null;
|
|
return <div className="mt-2 flex min-h-8 min-w-0 items-center gap-2 px-1 py-1 text-xs text-muted-foreground" data-testid="task-chat-current-activity" data-turn-position="tail">
|
|
<span className={cn(!waitingLabel && shimmer && "shimmer-text shimmer-text-muted")} aria-live="polite" data-testid="task-chat-current-activity-label">{waitingLabel ?? "Thinking"}</span>
|
|
</div>;
|
|
}
|
|
|
|
interface RunnerHeaderProps {
|
|
runId?: string | null;
|
|
agentName?: string | null;
|
|
agentIcon?: string | null;
|
|
agent?: import("../AgentAvatar").AvatarAgent;
|
|
status: string;
|
|
execution?: ExecutionProjection | null;
|
|
preparationLabel?: string;
|
|
startedAtMs: number | null;
|
|
finishedAtMs?: number | null;
|
|
continuedAfterSteering?: boolean;
|
|
}
|
|
|
|
function runnerWaitingLabel({ runId, status, execution, preparationLabel }: RunnerHeaderProps) {
|
|
if (isTerminalRunStatus(status)) return null;
|
|
if (execution?.phase === "waiting_for_provider") return execution.label;
|
|
if (!runId && preparationLabel) return preparationLabel;
|
|
return status === "queued" ? execution?.label ?? "Queued" : null;
|
|
}
|
|
|
|
/** One identity stays mounted while admission discovers the run's renderer. */
|
|
export function TaskChatRunnerHeader({ statusContent, ...props }: RunnerHeaderProps & { statusContent?: ReactNode }) {
|
|
return <div className="flex min-h-8 min-w-0 items-center gap-2 pb-1 pt-2" data-testid="task-chat-runner-identity-row">
|
|
{props.agentName ? <TaskChatAgentIdentity agentName={props.agentName} agentIcon={props.agentIcon} agent={props.agent} /> : null}
|
|
{statusContent ?? <RunnerTurnStatus status={props.status} startedAtMs={props.startedAtMs} finishedAtMs={props.finishedAtMs}
|
|
continuedAfterSteering={props.continuedAfterSteering} waitingLabel={runnerWaitingLabel(props)} />}
|
|
</div>;
|
|
}
|
|
|
|
export function TaskChatRunnerTurn({
|
|
runId,
|
|
agentName,
|
|
agentIcon,
|
|
agent,
|
|
items,
|
|
status,
|
|
execution,
|
|
preparationLabel,
|
|
startedAtMs,
|
|
finishedAtMs,
|
|
activityUnavailable = false,
|
|
allowSummaryFallback = true,
|
|
awaitingContinuation = false,
|
|
waitSummary = null,
|
|
continuedAfterSteering = false,
|
|
showHeader = true,
|
|
onRuntimeRequestDecision,
|
|
}: {
|
|
/** Stable identity used to clear replay-latched final text for the next turn. */
|
|
runId?: string | null;
|
|
agentName?: string | null;
|
|
agentIcon?: string | null;
|
|
agent?: import("../AgentAvatar").AvatarAgent;
|
|
items: readonly TaskChatItem[];
|
|
status: string;
|
|
execution?: ExecutionProjection | null;
|
|
/** Presentation only while Send/Retry admission has no run yet. */
|
|
preparationLabel?: string;
|
|
startedAtMs: number | null;
|
|
finishedAtMs?: number | null;
|
|
activityUnavailable?: boolean;
|
|
allowSummaryFallback?: boolean;
|
|
/**
|
|
* Accepted wait/interaction authority: the turn is waiting, so an early
|
|
* provider final that only restates the wait is not presented as a reply.
|
|
* Prose the agent wrote for the user stays visible.
|
|
*/
|
|
awaitingContinuation?: boolean;
|
|
/** The accepted yielded result's wait summary, when known. */
|
|
waitSummary?: string | null;
|
|
/** The visible tail resumes the same native run after an accepted steer. */
|
|
continuedAfterSteering?: boolean;
|
|
/** The task thread keeps a shared header across native and legacy discovery. */
|
|
showHeader?: boolean;
|
|
onRuntimeRequestDecision?: (
|
|
item: TaskChatRuntimeRequestItem,
|
|
decision: TaskChatRuntimeRequestDecision,
|
|
) => void | Promise<void>;
|
|
}) {
|
|
const { failures: mcpStartupFailures, rest: turnItems } = partitionRunHealthItems(items);
|
|
const terminal = isTerminalRunStatus(status);
|
|
const waitingLabel = runnerWaitingLabel({ runId, status, execution, preparationLabel, startedAtMs });
|
|
const yielded = turnItems.some(
|
|
(item) =>
|
|
item.kind === "protocol" &&
|
|
item.surface === "run_result" &&
|
|
item.disposition === "yielded",
|
|
);
|
|
const observedFinal = paperclipRunnerFinalResponse(turnItems, awaitingContinuation
|
|
? { allowFallback: false, yielded: true, waitSummary }
|
|
: { allowFallback: terminal && allowSummaryFallback });
|
|
const observedProviderText = Boolean(
|
|
observedFinal &&
|
|
turnItems.some(
|
|
(item) =>
|
|
item.kind === "message" &&
|
|
item.id === observedFinal.id &&
|
|
item.channel !== "progress",
|
|
),
|
|
);
|
|
// A reconnect/replay can briefly rebuild the transcript without the final
|
|
// item (or with an earlier, shorter prefix). Provider-authored final text
|
|
// always replaces a structured summary fallback, even when it is shorter;
|
|
// within either class, displayed answer text remains monotonic.
|
|
const finalRef = useRef<{
|
|
runId?: string | null;
|
|
item?: TaskChatMessageItem;
|
|
providerText?: boolean;
|
|
}>({ runId });
|
|
if (finalRef.current.runId !== runId) finalRef.current = { runId };
|
|
// A provider final can arrive before the accepted yielded result. Clear any
|
|
// replay latch once the control plane establishes that this turn is waiting
|
|
// for continuation; only an authored (non-wait) final is re-latched below.
|
|
if (yielded || awaitingContinuation) finalRef.current = { runId };
|
|
if (
|
|
observedFinal &&
|
|
(!finalRef.current.item ||
|
|
(observedProviderText && !finalRef.current.providerText) ||
|
|
(observedProviderText === Boolean(finalRef.current.providerText) &&
|
|
observedFinal.text.length >= finalRef.current.item.text.length))
|
|
) {
|
|
finalRef.current.item = observedFinal;
|
|
finalRef.current.providerText = observedProviderText;
|
|
}
|
|
const final = finalRef.current.item;
|
|
const timelineItems = paperclipRunnerTimelineItems(turnItems);
|
|
const currentActivityItems = currentActivityStatusItems(timelineItems);
|
|
const timelineRows = groupImageResources(buildTurnTimelineRows(
|
|
omitProgressRepeatedByResponse(timelineItems, final?.text),
|
|
!terminal,
|
|
));
|
|
|
|
return (
|
|
<TaskChatShimmerScope items={items}><div
|
|
className="flex min-w-0 flex-col"
|
|
data-testid="task-chat-runner-turn"
|
|
data-phase={status === "queued" ? "startup" : undefined}
|
|
>
|
|
{showHeader ? <TaskChatRunnerHeader runId={runId} agentName={agentName} agentIcon={agentIcon} agent={agent}
|
|
status={status} execution={execution} preparationLabel={preparationLabel} startedAtMs={startedAtMs}
|
|
finishedAtMs={finishedAtMs} continuedAfterSteering={continuedAfterSteering} /> : null}
|
|
<TaskChatRunHealthBanner failures={mcpStartupFailures} />
|
|
{activityUnavailable ? (
|
|
<div
|
|
className="px-1 py-1 text-xs text-muted-foreground"
|
|
role="status"
|
|
data-testid="task-chat-activity-unavailable"
|
|
>
|
|
Live runner activity is temporarily unavailable. Retrying…
|
|
</div>
|
|
) : null}
|
|
{timelineRows.length > 0 ? (
|
|
<div
|
|
className="flex min-w-0 flex-col gap-2 py-1"
|
|
data-testid="task-chat-turn-timeline"
|
|
>
|
|
{timelineRows.map((row) => (
|
|
<div
|
|
className="min-w-0"
|
|
key={`${runId ?? "run"}:${row.id}`}
|
|
data-testid="task-chat-turn-timeline-row"
|
|
data-timeline-row-id={row.id}
|
|
data-thread-anchor={row.id}
|
|
>
|
|
{row.kind === "activity_phase" ? (
|
|
<TaskChatRunnerActivityGroup item={row} />
|
|
) : row.kind === "plan_document" ? (
|
|
<TaskChatPlanPreviewCard
|
|
source={{ kind: "saved", document: row.document }}
|
|
testId={
|
|
row.placement === "fallback"
|
|
? "task-chat-plan-preview-fallback"
|
|
: "task-chat-plan-preview"
|
|
}
|
|
/>
|
|
) : row.kind === "marker" ? (
|
|
<TaskChatMarker item={row} />
|
|
) : row.kind === "subagent_group" ? (
|
|
<TaskChatSubagentTree subagents={row.subagents} />
|
|
) : row.kind === "protocol" ? (
|
|
<TaskChatProtocolCard
|
|
item={row}
|
|
onRuntimeRequestDecision={onRuntimeRequestDecision}
|
|
/>
|
|
) : null}
|
|
</div>
|
|
))}
|
|
</div>
|
|
) : null}
|
|
{final ? (
|
|
<div
|
|
className="w-full"
|
|
data-testid="task-chat-final-response"
|
|
>
|
|
<TaskChatBubble
|
|
item={{ ...final, authorName: agentName ?? undefined, agentIcon, agent, timestamp: final.timestamp ?? formatTaskChatTimestamp(final.atMs) }}
|
|
animateEntry={false}
|
|
hideAgentIdentity={!continuedAfterSteering}
|
|
actions={<TaskChatBubbleActions copyText={final.text} />}
|
|
/>
|
|
</div>
|
|
) : null}
|
|
{!final && currentActivityItems.length === 0 ? <RunnerCurrentActivityTail status={status} waitingLabel={waitingLabel} /> : null}
|
|
</div></TaskChatShimmerScope>
|
|
);
|
|
}
|