mirror of
https://github.com/expo/explore.git
synced 2026-11-15 13:18:16 -10:00
Compare commits
6
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a39bf954c4 | ||
|
|
47cb6e51fb | ||
|
|
f057a9dc18 | ||
|
|
86b7fab2bd | ||
|
|
3168bab03b | ||
|
|
79feb5441b |
No files matched your search
@@ -28,7 +28,7 @@ Explore maps your app: a screenshot of every screen, connected by the taps betwe
|
||||
<td width="50%" valign="top">
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="docs/screenshots/readme-pr-comment-dark.png">
|
||||
<img width="100%" alt="Explore's GitHub PR comment on a euxy PR: the heading links to the iOS and Android reports, a table with one row per platform giving device, result and an Open link, the Added, Removed, Modified, Needs review and Source-affected screens as collapsed lists, fading out below them" src="docs/screenshots/readme-pr-comment-light.png">
|
||||
<img width="100%" alt="Explore's GitHub PR comment on a euxy PR: a heading with how many screens changed and a Review in Explore link, a line naming each device with a link to its report, and the changed screens as a short list, fading out below them" src="docs/screenshots/readme-pr-comment-light.png">
|
||||
</picture>
|
||||
<strong>A comment on every PR</strong>: with <a href="#set-up-ci">CI</a>, the report links and the added, removed and modified screens.
|
||||
</td>
|
||||
|
||||
@@ -360,7 +360,7 @@ Create an Expo access token for an account with access to the app's EAS project.
|
||||
|
||||
Capture uses the automatic GitHub token with `pull-requests: write`, so it can keep the sticky comment current while the run works; publication rewrites that comment from a separate job. `GITHUB_TOKEN_MISSING` means the workflow environment is missing the automatic token mapping.
|
||||
|
||||
The sticky comment is one comment for the whole run: a header with the base and head commits, one row per captured platform (device, result, **Open** / **zip** / **run** links), every changed screen listed once with a link per platform, and a collapsed validation table with one column per platform. Each platform's job updates only its own hidden record in the comment, so iOS and Android jobs never overwrite each other. When a new run starts (a push, a rerun), the earlier run moves to a collapsed **Previous runs** list that keeps its head commit, result, report and run links; the last five runs are kept. A run that finishes after the PR moved on is still published, under previous runs, instead of failing. The failure job posts through the same renderer (`ci:failure`), so a crash before the pipeline's own comment still lands in that platform's row.
|
||||
The sticky comment is one comment for the whole run: a heading with what changed (**17 screens changed · Review in Explore**), a line naming every target with a link to its report plus the compared commits and when the comment was last updated, one line for any target that is still working, failed, or finished with captures missing, then the changed screens as a short list (the first ten, each linking its Before/After in the report, with a link to the rest). Capture problems, when there are any, sit in a collapsed **Capture issues** block; the footer links the workflow run and the config. Each target's job updates only its own hidden record in the comment, so iOS and Android jobs never overwrite each other. When a new run starts (a push, or a rerun's new attempt), the comment keeps the last review under a **New run in progress** line until the new run has a ready target of its own; a target of the new run that fails is named on that line, which reads **New run failed** once nothing is still working. A push also moves the earlier run to a collapsed **Previous runs** list that keeps its head commit, result, report and run links; the last five runs are kept. A run that finishes after the PR moved on is still published, under previous runs, instead of failing. The failure job posts through the same renderer (`ci:failure`), so a crash before the pipeline's own comment still lands in that target's line.
|
||||
|
||||
The comment can also draw a map of the change: the added, removed and visually changed screens, the screens one step before them, and the transitions among them, as a collapsed Mermaid flowchart that GitHub draws in place. It is off by default; turn it on with:
|
||||
|
||||
|
||||
@@ -100,17 +100,18 @@ try {
|
||||
const context = await browser.newContext({ viewport: { width: 1000, height: 1400 }, deviceScaleFactor: 2, colorScheme: theme });
|
||||
const page = await context.newPage();
|
||||
await page.goto(args.pr, { waitUntil: "domcontentloaded" });
|
||||
const comment = page.locator(".timeline-comment").filter({ has: page.locator("h3", { hasText: /Explore review/ }) }).first();
|
||||
const comment = page.locator(".timeline-comment").filter({ has: page.locator("h3", { hasText: /Review in Explore|Explore review/ }) }).first();
|
||||
await comment.waitFor({ timeout: 30_000 });
|
||||
await comment.scrollIntoViewIfNeeded();
|
||||
// The comment's pointer toward the avatar would show as a stray tick above the card.
|
||||
await page.addStyleTag({ content: ".timeline-comment::before, .timeline-comment::after { display: none !important; }" });
|
||||
await page.waitForTimeout(1000);
|
||||
// Cut after the list of changed screens: the launch timings, validation, and footer below it are detail.
|
||||
// Cut after the list of changed screens: the collapsed blocks and footer below it are detail.
|
||||
const clip = await comment.evaluate((card) => {
|
||||
const box = card.getBoundingClientRect();
|
||||
const after = [...card.querySelectorAll("p")].find(p => /Launch:/.test(p.textContent ?? ""));
|
||||
const bottom = after ? after.getBoundingClientRect().top : box.bottom;
|
||||
const list = [...card.querySelectorAll("ul")].find(ul => ul.querySelector(":scope > li code"));
|
||||
const after = list?.nextElementSibling;
|
||||
const bottom = after ? after.getBoundingClientRect().top : list ? list.getBoundingClientRect().bottom : box.bottom;
|
||||
return { x: box.left + window.scrollX, y: box.top + window.scrollY, width: box.width, height: Math.round(bottom - box.top) };
|
||||
});
|
||||
const shot = await page.screenshot({ clip, fullPage: true });
|
||||
|
||||
+188
-229
@@ -2,7 +2,7 @@ import type { PublicReview } from "../contracts/capture.js";
|
||||
import type { RunTarget } from "../types.js";
|
||||
import type { SimulatorRuntimeMetadata } from "../capture/run.js";
|
||||
import { CI_COMMENT_MARKER } from "./pr-comment.js";
|
||||
import { CHANGE_GROUPS, MAX_COMMENT_TARGETS, MAX_NAVIGATION_EDGES, MAX_NAVIGATION_NODES, changeCountText, compareTargets, defaultTargetLabel, foldRecord, strongerGroup, historyLine, orderedRecords, readCommentRecords, recordLabel, recordLine, recordSummary, recordTarget,
|
||||
import { CHANGE_GROUPS, MAX_COMMENT_TARGETS, MAX_NAVIGATION_EDGES, MAX_NAVIGATION_NODES, compareTargets, defaultTargetLabel, foldRecord, strongerGroup, historyLine, orderedPrevious, orderedRecords, previousLine, readCommentRecords, recordLabel, recordLine, recordSummary,
|
||||
type ChangeGroup, type CommentNavigation, type CommentTarget, type CommentRecords, type HistoryEntry, type NavigationGroup, type NavigationKind, type PlatformRecord } from "./pr-comment-record.js";
|
||||
import { isAccessibilityOnlyChange } from "../contracts/review-validation.js";
|
||||
import type { PublicCaptureHealth, PublicEvidenceSummary } from "../review/evidence-assessment.js";
|
||||
@@ -64,13 +64,6 @@ export type PublicCommentState = {
|
||||
type ReviewGroup = ChangeGroup;
|
||||
|
||||
const GROUP_ORDER: ReviewGroup[] = CHANGE_GROUPS;
|
||||
const GROUP_LABEL: Record<ReviewGroup, string> = {
|
||||
added: "🟩 Added",
|
||||
modified: "🟧 Modified",
|
||||
removed: "🟥 Removed",
|
||||
"accessibility-only": "🟦 Accessibility only",
|
||||
"needs-review": "🟨 Needs review",
|
||||
};
|
||||
|
||||
/** The body is HTML, so no blank lines are needed and the block keeps its natural spacing. */
|
||||
function details(summary: string, content: string[]): string[] {
|
||||
@@ -115,14 +108,6 @@ function link(label: string, destination: string): string {
|
||||
|
||||
const SUPERSEDED_NOTE = "The pull request moved while this run worked; its report is kept here until the current run replaces it.";
|
||||
|
||||
function duration(startedAt: number, updatedAt: number): string {
|
||||
const total = Math.max(0, Math.round((updatedAt - startedAt) / 1000));
|
||||
if (startedAt <= 0 || total > 24 * 3600) return "";
|
||||
const minutes = Math.floor(total / 60);
|
||||
const seconds = total % 60;
|
||||
return minutes > 0 ? `${minutes}m ${String(seconds).padStart(2, "0")}s` : `${seconds}s`;
|
||||
}
|
||||
|
||||
function plural(count: number, noun: string): string {
|
||||
return `${count} ${noun}${count === 1 ? "" : "s"}`;
|
||||
}
|
||||
@@ -191,12 +176,7 @@ function previewExpiry(expiresAt: number): string {
|
||||
return new Date(expiresAt).toLocaleDateString("en-US", { month: "short", day: "numeric", year: "numeric", timeZone: "UTC" }) + " (UTC)";
|
||||
}
|
||||
|
||||
/**
|
||||
* Differences that are not review targets, stated once instead of per screen: screen variants
|
||||
* (source-unchanged screens captured in another runtime state) and app-wide source changes that
|
||||
* left some screens without a visible difference.
|
||||
*/
|
||||
/** "Launch: splash unchanged for 3.1 s" — the app launch per appearance, kept apart from the screen rows. */
|
||||
/** "Launch: splash unchanged for 3.1 s" — the app launch per appearance, for the local text summary. */
|
||||
function launchLine(review: PublicReview): string | undefined {
|
||||
const launch = review.launch ?? [];
|
||||
if (!launch.length) return undefined;
|
||||
@@ -220,6 +200,16 @@ function launchLine(review: PublicReview): string | undefined {
|
||||
return `Launch: ${parts.join("; ")}.`;
|
||||
}
|
||||
|
||||
/** "Splash screen changed": the launch comparison when it found a difference; a steady splash is not news for the comment. */
|
||||
function splashLines(review: PublicReview): string[] {
|
||||
const launch = review.launch ?? [];
|
||||
const changed = launch.filter(entry => entry.visual && entry.visual.verdict !== "unchanged");
|
||||
if (!changed.length) return [];
|
||||
const verdict = changed.every(entry => entry.visual!.verdict === "needs-review") ? "needs review" : "changed";
|
||||
const schemes = changed.map(entry => entry.colorScheme).filter((scheme): scheme is NonNullable<typeof scheme> => Boolean(scheme));
|
||||
return [`Splash screen ${verdict}${schemes.length && changed.length < launch.length ? ` in ${schemes.join(" and ")} mode` : ""}`];
|
||||
}
|
||||
|
||||
function contextLines(review: PublicReview): string[] {
|
||||
const variants = review.surfaces.filter(surface => surface.review?.status === "variant").length;
|
||||
const appWide = review.surfaces.filter(surface => surface.sourceScope === "app-wide");
|
||||
@@ -236,7 +226,6 @@ function contextLines(review: PublicReview): string[] {
|
||||
}
|
||||
|
||||
const SURFACE_ID = /^[A-Za-z0-9._-]{1,200}(@(light|dark))?$/;
|
||||
const MAX_ROWS_PER_GROUP = 200;
|
||||
|
||||
/** The run id and attempt the run link names (`…/actions/runs/<id>/attempts/<n>`). */
|
||||
function runIdentity(run: string): { runId: string; runAttempt: string } {
|
||||
@@ -316,7 +305,7 @@ export function platformRecordOf(state: PublicCommentState, target?: CommentTarg
|
||||
if (navigation) record.navigation = navigation;
|
||||
const notCompared = state.review.surfaces.filter(notComparedChange).length;
|
||||
if (notCompared) record.notCompared = notCompared;
|
||||
record.context = contextLines(state.review).filter(line => line.length);
|
||||
record.context = splashLines(state.review);
|
||||
const diagnostics = [...(state.evidence?.diagnostics ?? [])];
|
||||
for (const surface of state.review.surfaces) {
|
||||
for (const side of ["base", "head"] as const) {
|
||||
@@ -447,214 +436,185 @@ function surfaceLink(report: string, id: string): string | undefined {
|
||||
return url.href;
|
||||
}
|
||||
|
||||
function resultCell(record: PlatformRecord): string {
|
||||
switch (record.phase) {
|
||||
case "pending": return "⏳ starting";
|
||||
case "capturing": {
|
||||
const capture = record.capture;
|
||||
if (!capture) return "⏳ capturing";
|
||||
const side = capture.current ?? (capture.done.head > 0 || capture.baseFromCache ? "head" : "base");
|
||||
return `⏳ capturing ${side} ${capture.done[side]}/${capture.planned[side]}${capture.baseFromCache && side === "head" ? " · base reused" : ""}`;
|
||||
}
|
||||
case "failed": return `🟥 failed · ${inlineText(record.failure?.step ?? "capture", 48)}`;
|
||||
case "ready": {
|
||||
const unavailable = record.health ? record.health.base.unavailable + record.health.head.unavailable : 0;
|
||||
const unverified = record.health ? record.health.base.unverified + record.health.head.unverified : 0;
|
||||
const failed = (record.validation?.visual?.failed ?? 0) + (record.validation?.accessibility?.failed ?? 0);
|
||||
const notCompared = record.notCompared ?? 0;
|
||||
const icon = unavailable || unverified || failed || notCompared ? "🟨" : "🟩";
|
||||
return [`${icon} ${changeCountText(record.changes?.map(change => change.group) ?? [])}`, notCompared ? `${notCompared} not compared` : undefined, unavailable ? `${unavailable} unavailable` : undefined, unverified ? `${unverified} unattributed` : undefined,
|
||||
failed ? `${failed} comparison${failed === 1 ? "" : "s"} failed` : undefined, duration(record.startedAt, record.updatedAt)].filter(Boolean).join(" · ");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** The report link alone; the workflow run and the report zip sit in the footer, where someone debugging looks for them. */
|
||||
function reportCell(record: PlatformRecord): string {
|
||||
const run = link("run", record.links.run);
|
||||
if (record.phase === "failed") return `${run} · ${link("evidence", record.links.artifacts)}`;
|
||||
if (record.phase !== "ready") return run;
|
||||
return record.links.report ? link(record.links.report.includes(".expo.app/") ? "Open" : "Viewer", record.links.report) : record.links.deploying ? "deploying…" : run;
|
||||
}
|
||||
const ROW_LIMIT = 10;
|
||||
const MAX_FAILURE_MESSAGE = 600;
|
||||
|
||||
/** A target's label as Markdown text. */
|
||||
const labelOf = (record: PlatformRecord) => inlineText(recordLabel(record), 80);
|
||||
const finished = (record: PlatformRecord) => record.phase === "ready" || record.phase === "failed";
|
||||
const firstReport = (records: PlatformRecord[]) => records.find(record => record.phase === "ready" && record.links.report)?.links.report;
|
||||
|
||||
/** "platform" for the per-platform targets of a config without targets, "device" once a config names its own. */
|
||||
function targetNoun(records: PlatformRecord[]): "platform" | "device" {
|
||||
return records.every(record => !record.target && record.label === undefined) ? "platform" : "device";
|
||||
}
|
||||
|
||||
/** "Workflow iOS run · zip; Android run · zip" for the ready targets, with the target named only when there are several. */
|
||||
function artifactLinks(records: PlatformRecord[]): string | undefined {
|
||||
function updatedText(records: PlatformRecord[]): string {
|
||||
const at = Math.max(...records.map(record => record.updatedAt));
|
||||
return `${new Date(at).toLocaleString("en-US", { month: "short", day: "numeric", hour: "numeric", minute: "2-digit", timeZone: "UTC" })} UTC`;
|
||||
}
|
||||
|
||||
/** "17 screens changed · Review in Explore": what the run found, with the report as the heading's link; until a target finished, what it is doing. */
|
||||
function title(records: PlatformRecord[], rows: ChangeRow[]): string {
|
||||
const ready = records.filter(record => record.phase === "ready");
|
||||
if (!ready.length) return undefined;
|
||||
const parts = ready.map(record => {
|
||||
const links = [link("run", record.links.run), record.links.download ? link("zip", record.links.download) : undefined].filter(Boolean).join(" · ");
|
||||
return ready.length > 1 ? `${labelOf(record)} ${links}` : links;
|
||||
});
|
||||
return `Workflow ${parts.join("; ")}`;
|
||||
}
|
||||
|
||||
function deviceCell(record: PlatformRecord): string {
|
||||
const parts = [record.device ? inlineText(record.device, 80) : undefined,
|
||||
record.app?.version ? inlineText(`${record.app.name ? `${record.app.name} ` : ""}${record.app.version}${record.app.build ? ` (${record.app.build})` : ""}`, 80) : undefined].filter(Boolean);
|
||||
return parts.length ? parts.join(" · ") : "—";
|
||||
}
|
||||
|
||||
type ChangeRow = { id: string; route: string; scheme?: string; group: ReviewGroup; description: string; perTarget: Partial<Record<CommentTarget, { group: ReviewGroup; href?: string }>> };
|
||||
|
||||
function changeRows(records: PlatformRecord[]): Map<ReviewGroup, ChangeRow[]> {
|
||||
const rows = new Map<string, ChangeRow>();
|
||||
for (const record of records) {
|
||||
if (record.phase !== "ready") continue;
|
||||
for (const change of record.changes ?? []) {
|
||||
const href = record.links.report ? surfaceLink(record.links.report, change.id) : undefined;
|
||||
const row = rows.get(change.id) ?? { id: change.id, route: change.route, group: change.group, description: change.description, perTarget: {}, ...(change.scheme ? { scheme: change.scheme } : {}) };
|
||||
// A screen accessibility-only on one target and visibly changed on another is listed as the visible change.
|
||||
if (strongerGroup(row.group, change.group) !== row.group) Object.assign(row, { group: change.group, description: change.description });
|
||||
row.perTarget[recordTarget(record)] = { group: change.group, ...(href ? { href } : {}) };
|
||||
rows.set(change.id, row);
|
||||
}
|
||||
}
|
||||
const groups = new Map<ReviewGroup, ChangeRow[]>();
|
||||
for (const row of rows.values()) groups.set(row.group, [...(groups.get(row.group) ?? []), row]);
|
||||
return groups;
|
||||
}
|
||||
|
||||
function renderChangeRow(row: ChangeRow, records: PlatformRecord[]): string {
|
||||
const where = records.flatMap(record => {
|
||||
const entry = row.perTarget[recordTarget(record)];
|
||||
if (!entry) return [];
|
||||
const label = labelOf(record) + (entry.group !== row.group ? ` (${GROUP_LABEL[entry.group].replace(/^\S+ /, "").toLowerCase()})` : "");
|
||||
return [entry.href ? link(label, entry.href) : label];
|
||||
});
|
||||
const route = `\`${plainText(row.route, MAX_ROUTE_LENGTH).replace(/`/g, "")}\`${row.scheme ? ` (${inlineText(row.scheme, 20)})` : ""}`;
|
||||
return `- ${route} — ${inlineText(row.description, MAX_DESCRIPTION_LENGTH)}${where.length ? ` · ${where.join(" · ")}` : ""}`;
|
||||
}
|
||||
|
||||
function changesSection(records: PlatformRecord[], rowCap: number, withMap: boolean): string[] {
|
||||
const ready = records.filter(record => record.phase === "ready");
|
||||
if (!ready.length) return [];
|
||||
const groups = changeRows(ready);
|
||||
const lines: string[] = [];
|
||||
const noun = targetNoun(records);
|
||||
const total = [...groups.values()].reduce((sum, rows) => sum + rows.length, 0);
|
||||
lines.push(`**Changes** · ${plural(total, "screen")}${records.length > 1 && ready.length > 1 ? ` · a screen is listed once, with a link per ${noun}` : ""}`, "");
|
||||
// With screens left uncompared, nothing changed only among the ones compared.
|
||||
if (total === 0) lines.push(ready.some(record => record.notCompared) ? `No compared screen changed on the reviewed ${noun}s.` : `No screen changed on the reviewed ${noun}s.`, "");
|
||||
for (const group of GROUP_ORDER) {
|
||||
const rows = groups.get(group) ?? [];
|
||||
if (!rows.length) continue;
|
||||
const shown = rows.slice(0, rowCap);
|
||||
const content = ["", ...(group === "accessibility-only" ? ["Before and After look the same. What screen readers and voice control read changed.", ""] : []),
|
||||
...shown.map(row => renderChangeRow(row, records))];
|
||||
if (rows.length > shown.length) {
|
||||
const report = ready.find(record => record.links.report)?.links.report;
|
||||
content.push(`- ${rows.length - shown.length} more${report ? ` — ${link("Explore report", report)}` : ""}`);
|
||||
}
|
||||
content.push("");
|
||||
lines.push(...details(`${GROUP_LABEL[group]} (${rows.length})`, content));
|
||||
}
|
||||
if (withMap) lines.push(...mapSection(ready));
|
||||
const seen = new Set<string>();
|
||||
for (const record of ready) for (const line of record.context ?? []) {
|
||||
if (seen.has(line)) continue;
|
||||
seen.add(line);
|
||||
const everywhere = ready.every(other => other.context?.includes(line));
|
||||
lines.push(everywhere ? line : `${labelOf(record)} · ${line}`, "");
|
||||
}
|
||||
return lines;
|
||||
}
|
||||
|
||||
/**
|
||||
* The heading carries the report links: on a phone the table's last column is clipped, and the
|
||||
* commits it reviews are a subtitle under it. A target still capturing adds its link when it is ready.
|
||||
*/
|
||||
function heading(records: PlatformRecord[]): string {
|
||||
const ready = records.filter(record => record.phase === "ready" && record.links.report);
|
||||
if (!ready.length) {
|
||||
return records.some(record => record.phase === "ready" && record.links.deploying) ? "### Explore review · report deploying, not ready yet" : "### Explore review";
|
||||
if (records.every(record => record.phase === "failed")) return "### Explore review · capture failed";
|
||||
const active = records.filter(record => !finished(record));
|
||||
const verb = active.some(record => record.phase === "capturing") ? "capturing" : "starting";
|
||||
return `### Explore review · ${verb}${records.length > 1 ? ` on ${plural(active.length, targetNoun(records))}` : ""}`;
|
||||
}
|
||||
const links = ready.length === 1 ? [link("Open report", ready[0]!.links.report!)]
|
||||
: ready.map(record => link(`${labelOf(record)} report`, record.links.report!));
|
||||
return `### Explore review · ${links.join(" · ")}`;
|
||||
// With screens left uncompared, nothing changed only among the ones compared.
|
||||
const summary = rows.length ? `${plural(rows.length, "screen")} changed` : ready.some(record => gaps(record).length) ? "No compared screens changed" : "No screens changed";
|
||||
const report = firstReport(ready);
|
||||
const open = report ? link("Review in Explore", report) : ready.some(record => record.links.deploying) ? "report deploying…" : undefined;
|
||||
return `### ${summary}${open ? ` · ${open}` : ""}`;
|
||||
}
|
||||
|
||||
function validationCell(record: PlatformRecord, check: "capture" | "routes" | "visual" | "accessibility" | "inventory"): string {
|
||||
if (record.phase === "pending" || record.phase === "capturing") return "⏳";
|
||||
if (record.phase === "failed") {
|
||||
if (check !== "capture") return "—";
|
||||
const capture = record.capture;
|
||||
return capture ? `🟥 ${capture.done.base + capture.done.head}/${capture.planned.base + capture.planned.head} surfaces` : "🟥 not started";
|
||||
}
|
||||
const health = record.health;
|
||||
switch (check) {
|
||||
case "capture": {
|
||||
const planned = health ? health.base.planned + health.head.planned : (record.capture?.planned.base ?? 0) + (record.capture?.planned.head ?? 0);
|
||||
const done = health ? health.base.captured + health.head.captured : (record.capture?.done.base ?? 0) + (record.capture?.done.head ?? 0);
|
||||
const cache = record.capture?.baseFromCache ? " · base cache reused" : "";
|
||||
return done < planned ? `🟨 ${done}/${planned} surfaces · ${planned - done} unavailable${cache}` : `🟩 ${done}/${planned} surfaces${cache}`;
|
||||
}
|
||||
case "routes": {
|
||||
if (!health) return "—";
|
||||
const verified = health.base.verified + health.head.verified, planned = health.base.planned + health.head.planned;
|
||||
const unverified = health.base.unverified + health.head.unverified, unavailable = health.base.unavailable + health.head.unavailable;
|
||||
return `${verified === planned ? "🟩" : "🟨"} ${verified}/${planned} verified${unverified ? ` · ${unverified} unattributed` : ""}${unavailable ? ` · ${unavailable} unavailable` : ""}`;
|
||||
}
|
||||
case "visual": {
|
||||
const v = record.validation?.visual;
|
||||
if (!v) return "🟨 not run";
|
||||
const mark = v.compared === 0 || v.failed > 0 || v.skipped ? "🟨" : "🟩";
|
||||
return `${mark} ${v.compared} compared · threshold ${v.threshold}${v.failed ? ` · ${v.failed} failed` : ""}${v.skipped ? ` · ${v.skipped} skipped` : ""}`;
|
||||
}
|
||||
case "accessibility": {
|
||||
const a = record.validation?.accessibility;
|
||||
if (!a) return "🟨 not run";
|
||||
return `${a.compared === 0 || a.failed > 0 ? "🟨" : "🟩"} ${a.compared} compared · ${plural(a.described, "description")}${a.failed ? ` · ${a.failed} failed` : ""}`;
|
||||
}
|
||||
case "inventory": return record.validation?.routes !== undefined ? `🟩 ${plural(record.validation.routes, "route")}` : "🟩 analyzed";
|
||||
}
|
||||
/** Every target by name, each linking its own report, then the commits and when the comment last changed. */
|
||||
function subtitle(records: PlatformRecord[], repository: string): string {
|
||||
const first = records[0]!;
|
||||
const targets = records.map(record => record.phase === "ready" && record.links.report ? link(labelOf(record), record.links.report) : labelOf(record));
|
||||
return `<sub>${[...targets, `${commitLink(repository, first.headSha)} vs ${commitLink(repository, first.baseSha)}`, `updated ${updatedText(records)}`].join(" · ")}</sub>`;
|
||||
}
|
||||
|
||||
function validationSummary(record: PlatformRecord): string {
|
||||
/** One line for a target that is not simply done: still working, failed, or finished with captures missing. */
|
||||
function statusLine(record: PlatformRecord): string | undefined {
|
||||
const label = labelOf(record), run = link("run", record.links.run);
|
||||
switch (record.phase) {
|
||||
case "failed": return "🟥 failed";
|
||||
case "pending": return `${label} · starting · ${run}`;
|
||||
case "capturing": {
|
||||
const capture = record.capture;
|
||||
if (!capture) return `${label} · capturing · ${run}`;
|
||||
const side = capture.current ?? (capture.done.head > 0 || capture.baseFromCache ? "head" : "base");
|
||||
return `${label} · capturing ${side} ${capture.done[side]}/${capture.planned[side]}${capture.baseFromCache && side === "head" ? " · base reused" : ""} · ${run}`;
|
||||
}
|
||||
case "failed": return [`**${label} failed**`, inlineText(record.failure?.step ?? "capture", 48), record.failure?.message ? inlineText(record.failure.message, MAX_FAILURE_MESSAGE) : undefined,
|
||||
run, link("evidence", record.links.artifacts)].filter(Boolean).join(" · ");
|
||||
case "ready": {
|
||||
const health = record.health;
|
||||
if (!health) return "🟩 reviewed";
|
||||
const captured = health.base.captured + health.head.captured, planned = health.base.planned + health.head.planned;
|
||||
return `${captured < planned ? "🟨" : "🟩"} ${captured}/${planned} captured`;
|
||||
const parts = gaps(record);
|
||||
return parts.length ? `${label} · ${parts.join(" · ")} · ${run}` : undefined;
|
||||
}
|
||||
default: return "⏳ in progress";
|
||||
}
|
||||
}
|
||||
|
||||
function validationSection(byTarget: PlatformRecord[]): string[] {
|
||||
if (!byTarget.some(record => record.phase === "ready" || record.phase === "failed")) return [];
|
||||
const summary = `Validation · ${byTarget.map(record => `${labelOf(record)} ${validationSummary(record)}`).join(" · ")}`;
|
||||
const header = `| Check | ${byTarget.map(labelOf).join(" | ")} |`;
|
||||
const checks: [string, Parameters<typeof validationCell>[1]][] = [["Capture", "capture"], ["Route verification", "routes"], ["Visual comparison", "visual"], ["Accessibility comparison", "accessibility"], ["Source inventory", "inventory"]];
|
||||
const content = ["", header, `|---|${byTarget.map(() => "---").join("|")}|`,
|
||||
...checks.map(([label, check]) => `| ${label} | ${byTarget.map(record => validationCell(record, check)).join(" | ")} |`), ""];
|
||||
for (const record of byTarget) {
|
||||
if (record.phase === "failed" && record.failure) {
|
||||
content.push(`**${labelOf(record)} failed** — ${inlineText(record.failure.message, 1200)} ${link("Open the failed run", record.links.run)} · ${link("Evidence artifact", record.links.artifacts)}`, "");
|
||||
}
|
||||
if (record.phase !== "ready") continue;
|
||||
const health = record.health;
|
||||
const gaps = health ? health.base.unavailable + health.base.unverified + health.head.unavailable + health.head.unverified : 0;
|
||||
if (!gaps && !record.diagnostics?.length) continue;
|
||||
const sideLine = (side: "base" | "head") => { const h = health![side]; return `${side} ${h.verified}/${h.planned} verified${h.unverified ? ` · ${h.unverified} unattributed` : ""}${h.unavailable ? ` · ${h.unavailable} unavailable` : ""}`; };
|
||||
content.push(health ? `**${labelOf(record)}** · ${sideLine("base")} · ${sideLine("head")}` : `**${labelOf(record)}**`, "");
|
||||
/** What a finished target has no verdict on: captures missing or unattributed, comparisons that did not run or failed. */
|
||||
function gaps(record: PlatformRecord): string[] {
|
||||
const unavailable = record.health ? record.health.base.unavailable + record.health.head.unavailable : 0;
|
||||
const unverified = record.health ? record.health.base.unverified + record.health.head.unverified : 0;
|
||||
const failed = (record.validation?.visual?.failed ?? 0) + (record.validation?.accessibility?.failed ?? 0);
|
||||
return [unavailable ? `${plural(unavailable, "capture")} unavailable` : undefined, unverified ? `${unverified} unattributed` : undefined,
|
||||
record.notCompared ? `${record.notCompared} not compared` : undefined, failed ? `${plural(failed, "comparison")} failed` : undefined]
|
||||
.filter((part): part is string => Boolean(part));
|
||||
}
|
||||
|
||||
/** The per-screen capture problems, collapsed: the status line already counts them. */
|
||||
function diagnosticsSection(records: PlatformRecord[]): string[] {
|
||||
const flagged = records.filter(record => record.phase === "ready" && ((record.diagnostics?.length ?? 0) > 0 || (record.diagnosticCount ?? 0) > 0));
|
||||
if (!flagged.length) return [];
|
||||
const content: string[] = [""];
|
||||
for (const record of flagged) {
|
||||
if (records.length > 1) content.push(`**${labelOf(record)}**`, "");
|
||||
for (const diagnostic of record.diagnostics ?? []) {
|
||||
content.push(`- ${diagnostic.side ? `${diagnostic.side === "base" ? "Base" : "Head"} · ` : ""}${inlineText(diagnostic.route, MAX_ROUTE_LENGTH)} · ${inlineText(diagnostic.code, 80)}: ${inlineText(diagnostic.message, 160)} Next action: ${inlineText(diagnostic.nextAction, 240)}`);
|
||||
}
|
||||
const more = (record.diagnosticCount ?? 0) - (record.diagnostics?.length ?? 0);
|
||||
if (more > 0) content.push(`- ${more} more — ${link("View run", record.links.run)}`);
|
||||
if (record.diagnostics?.length) content.push("");
|
||||
if (more > 0) content.push(`- ${more} more — ${link("run", record.links.run)}`);
|
||||
content.push("");
|
||||
}
|
||||
return details(summary, content);
|
||||
const total = flagged.reduce((sum, record) => sum + Math.max(record.diagnosticCount ?? 0, record.diagnostics?.length ?? 0), 0);
|
||||
return details(`Capture issues (${total})`, content);
|
||||
}
|
||||
|
||||
type ChangeRow = { id: string; route: string; scheme?: string; group: ReviewGroup; description: string; href?: string; targets: PlatformRecord[] };
|
||||
|
||||
/** Every changed screen once, in group order (added, removed, modified, …), linking the first target in config order that shows its group. */
|
||||
function changeRows(ready: PlatformRecord[]): ChangeRow[] {
|
||||
const rows = new Map<string, ChangeRow>();
|
||||
for (const record of ready) {
|
||||
for (const change of record.changes ?? []) {
|
||||
const href = record.links.report ? surfaceLink(record.links.report, change.id) : undefined;
|
||||
const row = rows.get(change.id) ?? { id: change.id, route: change.route, group: change.group, description: change.description, targets: [], ...(change.scheme ? { scheme: change.scheme } : {}) };
|
||||
// A screen accessibility-only on one target and visibly changed on another is listed as the visible change.
|
||||
// The link opens the target the row describes.
|
||||
if (strongerGroup(row.group, change.group) !== row.group) Object.assign(row, { group: change.group, description: change.description, href });
|
||||
else if (!row.href && href && change.group === row.group) row.href = href;
|
||||
row.targets.push(record);
|
||||
rows.set(change.id, row);
|
||||
}
|
||||
}
|
||||
return [...rows.values()].sort((a, b) => GROUP_ORDER.indexOf(a.group) - GROUP_ORDER.indexOf(b.group));
|
||||
}
|
||||
|
||||
function renderChangeRow(row: ChangeRow, ready: PlatformRecord[]): string {
|
||||
const code = `\`${plainText(row.route, MAX_ROUTE_LENGTH).replace(/`/g, "")}\``;
|
||||
const route = `${row.href ? link(code, row.href) : code}${row.scheme ? ` (${inlineText(row.scheme, 20)})` : ""}`;
|
||||
const description = inlineText(row.description.replace(/\.$/, ""), MAX_DESCRIPTION_LENGTH);
|
||||
let text: string;
|
||||
if (row.group === "added" || row.group === "removed") {
|
||||
const verb = row.group === "added" ? "Added" : "Removed";
|
||||
// The default description repeats the verb and route; an agent's note does not.
|
||||
text = row.description === `${verb} ${row.route} screen` ? `${verb} ${route}` : `${verb} ${route} · ${description}`;
|
||||
} else {
|
||||
text = `${route} · ${description}`;
|
||||
}
|
||||
const tag = row.group === "accessibility-only" ? "accessibility only" : row.group === "needs-review" ? "needs review" : undefined;
|
||||
// A target with screens it has no verdict on may have changed this one too: name where it changed, not where it did not.
|
||||
const where = ready.some(record => !row.targets.includes(record) && gaps(record).length)
|
||||
? `on ${row.targets.map(labelOf).join(", ")}` : targetsNote(row.targets, ready);
|
||||
return `- ${[text, tag, where].filter(Boolean).join(" · ")}`;
|
||||
}
|
||||
|
||||
/** "Android only" / "not on Android": the targets a screen changed on, when not all of them, named the shorter way. */
|
||||
function targetsNote(on: PlatformRecord[], ready: PlatformRecord[]): string | undefined {
|
||||
if (ready.length < 2 || on.length >= ready.length) return undefined;
|
||||
const missing = ready.filter(record => !on.includes(record));
|
||||
return missing.length < on.length ? `not on ${missing.map(labelOf).join(", ")}` : `${on.map(labelOf).join(", ")} only`;
|
||||
}
|
||||
|
||||
function changesSection(ready: PlatformRecord[], rows: ChangeRow[], rowCap: number, withMap: boolean): string[] {
|
||||
if (!ready.length) return [];
|
||||
const lines: string[] = [];
|
||||
const shown = rows.slice(0, rowCap);
|
||||
lines.push(...shown.map(row => renderChangeRow(row, ready)));
|
||||
if (rows.length > shown.length) {
|
||||
const report = firstReport(ready);
|
||||
const more = `${plural(rows.length - shown.length, "more screen")} in the report`;
|
||||
lines.push(`- ${report ? link(more, report) : more}`);
|
||||
}
|
||||
if (lines.length) lines.push("");
|
||||
if (withMap) lines.push(...mapSection(ready));
|
||||
const byLine = new Map<string, PlatformRecord[]>();
|
||||
for (const record of ready) for (const line of record.context ?? []) byLine.set(line, [...(byLine.get(line) ?? []), record]);
|
||||
for (const [line, records] of byLine) lines.push(records.length === ready.length ? line : `${line} · ${targetsNote(records, ready)}`, "");
|
||||
return lines;
|
||||
}
|
||||
|
||||
/**
|
||||
* The previous run stays on the comment while a new one works or has only failed; this line, above it, says which,
|
||||
* with each failure's step and message.
|
||||
*/
|
||||
function newRunLine(current: PlatformRecord[], shown: PlatformRecord[], repository: string): string {
|
||||
const first = current[0]!;
|
||||
const active = current.filter(record => !finished(record));
|
||||
const failed = current.filter(record => record.phase === "failed");
|
||||
const named = current.length > 1 || shown.length > 1 || Boolean(first.target) || first.label !== undefined;
|
||||
const verb = active.some(record => record.phase === "capturing") ? "capturing" : "starting";
|
||||
const working = active.length ? [`${verb}${named ? ` on ${active.map(labelOf).join(", ")}` : ""}`] : [];
|
||||
const failures = failed.map(record => [named ? `**${labelOf(record)} failed**` : undefined, inlineText(record.failure?.step ?? "capture", 48),
|
||||
record.failure?.message ? inlineText(record.failure.message, MAX_FAILURE_MESSAGE) : undefined].filter(Boolean).join(" · "));
|
||||
const links = [link("run", first.links.run), ...(failed.length ? [link("evidence", failed[0]!.links.artifacts)] : [])];
|
||||
return `> **${active.length ? "New run in progress" : "New run failed"}** for ${commitLink(repository, first.headSha)} · ${[...working, ...failures, ...links].join(" · ")}`;
|
||||
}
|
||||
|
||||
function footer(records: PlatformRecord[]): string {
|
||||
const parts: string[] = [];
|
||||
const expiries = records.filter(record => record.phase === "ready" && record.links.expiresAt !== undefined).map(record => record.links.expiresAt);
|
||||
if (expiries.length) parts.push(expiries.some(at => at === null) ? "Preview links work for anyone who has them until the deployment is deleted"
|
||||
: `Preview links work for anyone who has them until ${previewExpiry(Math.min(...expiries.filter((at): at is number => at !== null)))}`);
|
||||
const runs = records.filter((record, index, all) => all.findIndex(other => other.links.run === record.links.run) === index);
|
||||
parts.push(runs.length === 1 ? link("Workflow run", runs[0]!.links.run) : runs.map(record => link(`${labelOf(record)} run`, record.links.run)).join(" · "));
|
||||
parts.push(link("Config", records[0]!.links.config));
|
||||
return `<sub>${parts.join(" · ")}</sub>`;
|
||||
}
|
||||
|
||||
function historySection(repository: string, history: HistoryEntry[]): string[] {
|
||||
@@ -683,44 +643,47 @@ function deploymentOf(url: string | undefined): string | undefined {
|
||||
}
|
||||
function deploymentMarkerOf(records: CommentRecords): string | undefined {
|
||||
const names = new Set<string>();
|
||||
for (const record of Object.values(records.current)) { const name = deploymentOf(record?.links.report); if (name) names.add(name); }
|
||||
for (const record of [...Object.values(records.current), ...Object.values(records.previous ?? {})]) { const name = deploymentOf(record?.links.report); if (name) names.add(name); }
|
||||
for (const entry of records.history) for (const item of Object.values(entry.platforms)) { const name = deploymentOf(item?.report); if (name) names.add(name); }
|
||||
return names.size ? `<!-- expo-explore-deployments:${[...names].sort().join(",")} -->` : undefined;
|
||||
}
|
||||
|
||||
/** The whole comment from its records: hidden lines first, then one review for every target of the current run, in config order. */
|
||||
/**
|
||||
* The whole comment from its records: hidden lines first, then the review. Until a target of the current run is ready,
|
||||
* the previous run's review stays up under a line that says a new run is working or failed.
|
||||
*/
|
||||
export function renderCommentRecords(records: CommentRecords, existing?: string, budget = PUBLIC_COMMENT_BODY_BUDGET): string {
|
||||
const current = orderedRecords(records);
|
||||
const previous = orderedPrevious(records);
|
||||
const hidden = [CI_COMMENT_MARKER, ...[deploymentMarkerOf(records)].filter((line): line is string => Boolean(line)),
|
||||
...current.map(recordLine), ...[historyLine(records.history)].filter((line): line is string => Boolean(line))];
|
||||
...current.map(recordLine), ...previous.map(previousLine), ...[historyLine(records.history)].filter((line): line is string => Boolean(line))];
|
||||
if (!current.length) return [...hidden, "### Explore review", "", ...(records.history.some(entry => Object.values(entry.platforms).some(item => item?.superseded)) ? [SUPERSEDED_NOTE, ""] : []),
|
||||
...historySection("", records.history)].join("\n");
|
||||
const first = current[0]!;
|
||||
const shown = current.some(record => record.phase === "ready") || !previous.length ? current : previous;
|
||||
const first = shown[0]!;
|
||||
const repository = repositoryOf(first.links.run);
|
||||
const ready = shown.filter(record => record.phase === "ready");
|
||||
const rows = changeRows(ready);
|
||||
const render = (rowCap: number, withMap: boolean): string => {
|
||||
const lines: string[] = [...hidden, heading(current), "",
|
||||
`<sub>base ${commitLink(repository, first.baseSha)} → ${commitLink(repository, first.headSha)}</sub>`, ""];
|
||||
const lines: string[] = [...hidden];
|
||||
if (shown !== current) lines.push(newRunLine(current, shown, repository), "");
|
||||
lines.push(title(shown, rows), "", subtitle(shown, repository), "");
|
||||
if (first.superseded) lines.push(SUPERSEDED_NOTE, "");
|
||||
lines.push(`| | Device | Result | Report |`, `|---|---|---|---|`,
|
||||
...current.map(record => `| ${labelOf(record)} | ${deviceCell(record)} | ${resultCell(record)} | ${reportCell(record)} |`), "");
|
||||
lines.push(...changesSection(current, rowCap, withMap));
|
||||
lines.push(...validationSection(current));
|
||||
for (const line of shown.map(statusLine)) if (line) lines.push(line, "");
|
||||
lines.push(...changesSection(ready, rows, rowCap, withMap));
|
||||
lines.push(...diagnosticsSection(shown));
|
||||
lines.push(...historySection(repository, records.history));
|
||||
const expiries = current.filter(record => record.phase === "ready" && record.links.expiresAt !== undefined).map(record => record.links.expiresAt);
|
||||
const footer: string[] = [];
|
||||
if (expiries.length) footer.push(expiries.some(at => at === null) ? "Preview links work for anyone who has them until the deployment is deleted."
|
||||
: `Preview links work until ${previewExpiry(Math.min(...expiries.filter((at): at is number => at !== null)))} for anyone who has them.`);
|
||||
footer.push(`Managed via ${link("`" + plainText(configPathOf(first.links.config), 120).replace(/`/g, "") + "`", first.links.config)}`);
|
||||
const artifacts = artifactLinks(current);
|
||||
if (artifacts) footer.push(artifacts);
|
||||
lines.push(footer.join(" · "));
|
||||
lines.push(footer(shown));
|
||||
return lines.join("\n");
|
||||
};
|
||||
// The map goes first when the body is over budget, then the change lists shrink.
|
||||
let rowCap = MAX_ROWS_PER_GROUP;
|
||||
// The map goes first when the body is over budget, then the list shrinks, then the previous run's records go; once the
|
||||
// current run shows, nothing renders them, so they go first.
|
||||
let rowCap = ROW_LIMIT;
|
||||
let body = render(rowCap, true);
|
||||
if (body.length > budget && shown === current && previous.length) return renderCommentRecords({ ...records, previous: {} }, existing, budget);
|
||||
if (body.length > budget) body = render(rowCap, false);
|
||||
while (body.length > budget && rowCap > 1) { rowCap = Math.max(1, Math.floor(rowCap / 2)); body = render(rowCap, false); }
|
||||
if (body.length > budget && previous.length) return renderCommentRecords({ ...records, previous: {} }, existing, budget);
|
||||
return body;
|
||||
}
|
||||
|
||||
@@ -728,10 +691,6 @@ function repositoryOf(run: string): string {
|
||||
const match = /^https:\/\/github\.com\/([^/]+)\/([^/]+)\/actions\//.exec(run);
|
||||
return match ? `${decodeURIComponent(match[1]!)}/${decodeURIComponent(match[2]!)}` : "";
|
||||
}
|
||||
function configPathOf(config: string): string {
|
||||
const match = /\/blob\/[0-9a-f]{40}\/(.+)$/.exec(config);
|
||||
return match ? decodeURIComponent(match[1]!) : "config";
|
||||
}
|
||||
|
||||
/**
|
||||
* Render the sticky pull-request comment for any phase of a public review run, folding this writer into the existing
|
||||
|
||||
@@ -127,7 +127,12 @@ export type HistoryEntry = {
|
||||
};
|
||||
export type HistoryItem = { phase: RecordPhase; summary: string; run: string; report?: string; download?: string; superseded?: boolean; label?: string; order?: number };
|
||||
|
||||
export type CommentRecords = { current: Partial<Record<CommentTarget, PlatformRecord>>; history: HistoryEntry[] };
|
||||
export type CommentRecords = {
|
||||
current: Partial<Record<CommentTarget, PlatformRecord>>;
|
||||
/** The last finished run's records, shown while the current run has no finished record of its own. */
|
||||
previous?: Partial<Record<CommentTarget, PlatformRecord>>;
|
||||
history: HistoryEntry[];
|
||||
};
|
||||
|
||||
/** The id a record is keyed by. */
|
||||
export function recordTarget(record: Pick<PlatformRecord, "platform" | "target">): CommentTarget {
|
||||
@@ -146,12 +151,22 @@ export function compareTargets(a: { id: CommentTarget; order?: number | undefine
|
||||
|
||||
/** The current records in display order. */
|
||||
export function orderedRecords(records: CommentRecords): PlatformRecord[] {
|
||||
return Object.values(records.current).filter((record): record is PlatformRecord => Boolean(record))
|
||||
return ordered(records.current);
|
||||
}
|
||||
|
||||
/** The previous run's records in display order. */
|
||||
export function orderedPrevious(records: CommentRecords): PlatformRecord[] {
|
||||
return ordered(records.previous ?? {});
|
||||
}
|
||||
|
||||
function ordered(records: Partial<Record<CommentTarget, PlatformRecord>>): PlatformRecord[] {
|
||||
return Object.values(records).filter((record): record is PlatformRecord => Boolean(record))
|
||||
.sort((a, b) => compareTargets({ id: recordTarget(a), order: a.order }, { id: recordTarget(b), order: b.order }));
|
||||
}
|
||||
|
||||
export const HISTORY_LIMIT = 5;
|
||||
const RECORD_LINE = /^<!-- expo-explore-record:([a-z][a-z0-9-]{0,31}):([A-Za-z0-9_-]+) -->$/gm;
|
||||
const PREVIOUS_LINE = /^<!-- expo-explore-previous:([a-z][a-z0-9-]{0,31}):([A-Za-z0-9_-]+) -->$/gm;
|
||||
const HISTORY_LINE = /^<!-- expo-explore-history:([A-Za-z0-9_-]+) -->$/m;
|
||||
const MAX_ENCODED = 24_000;
|
||||
const MAX_DECODED = 256_000;
|
||||
@@ -176,6 +191,10 @@ export function recordLine(record: PlatformRecord): string {
|
||||
return `<!-- expo-explore-record:${recordTarget(record)}:${encodeRecordPayload(record)} -->`;
|
||||
}
|
||||
|
||||
export function previousLine(record: PlatformRecord): string {
|
||||
return `<!-- expo-explore-previous:${recordTarget(record)}:${encodeRecordPayload(record)} -->`;
|
||||
}
|
||||
|
||||
export function historyLine(history: HistoryEntry[]): string | undefined {
|
||||
return history.length ? `<!-- expo-explore-history:${encodeRecordPayload(history)} -->` : undefined;
|
||||
}
|
||||
@@ -312,12 +331,18 @@ export function parseHistory(value: unknown): HistoryEntry[] {
|
||||
export function readCommentRecords(body: string | undefined): CommentRecords {
|
||||
const records: CommentRecords = { current: {}, history: [] };
|
||||
if (!body) return records;
|
||||
for (const match of body.matchAll(RECORD_LINE)) {
|
||||
const record = parsePlatformRecord(decodePayload(match[2]!));
|
||||
const target = record && recordTarget(record);
|
||||
if (!record || target !== match[1] || !isTargetId(target)) continue;
|
||||
if (records.current[target] || Object.keys(records.current).length < MAX_COMMENT_TARGETS) records.current[target] = record;
|
||||
}
|
||||
const read = (pattern: RegExp, into: Partial<Record<CommentTarget, PlatformRecord>>) => {
|
||||
for (const match of body.matchAll(pattern)) {
|
||||
const record = parsePlatformRecord(decodePayload(match[2]!));
|
||||
const target = record && recordTarget(record);
|
||||
if (!record || target !== match[1] || !isTargetId(target)) continue;
|
||||
if (into[target] || Object.keys(into).length < MAX_COMMENT_TARGETS) into[target] = record;
|
||||
}
|
||||
};
|
||||
read(RECORD_LINE, records.current);
|
||||
const previous: Partial<Record<CommentTarget, PlatformRecord>> = {};
|
||||
read(PREVIOUS_LINE, previous);
|
||||
if (Object.keys(previous).length) records.previous = previous;
|
||||
const history = body.match(HISTORY_LINE);
|
||||
if (history) records.history = parseHistory(decodePayload(history[1]!));
|
||||
return records;
|
||||
@@ -371,14 +396,43 @@ export function foldRecord(existing: CommentRecords, incoming: PlatformRecord):
|
||||
const record = withKnownTarget(incoming, existing);
|
||||
const target = recordTarget(record);
|
||||
const currentRun = current[0];
|
||||
if (!currentRun && record.superseded) return { current: {}, history: pushHistory(existing.history, historyEntryOf([record])) };
|
||||
const previous = existing.previous ?? {};
|
||||
if (!currentRun && record.superseded) return { current: {}, previous, history: pushHistory(existing.history, historyEntryOf([record])) };
|
||||
if (!currentRun || currentRun.runId === record.runId) {
|
||||
return { current: { ...existing.current, [target]: record }, history: existing.history };
|
||||
const next = { ...existing.current, [target]: record };
|
||||
// A rerun's new attempt replaces a ready record with one that is not; the ready one stays readable meanwhile.
|
||||
const replaced = existing.current[target];
|
||||
const rerun = replaced && isReady(replaced) && !isReady(record) && Number(record.runAttempt) > Number(replaced.runAttempt);
|
||||
return { current: next, previous: trimPrevious(rerun ? { ...ofRun(previous, record.runId), [target]: replaced } : previous, next), history: existing.history };
|
||||
}
|
||||
if (runOrder(currentRun.runId) > runOrder(record.runId) || record.superseded) {
|
||||
return { current: existing.current, history: pushHistory(existing.history, historyEntryOf([{ ...record, superseded: true }])) };
|
||||
return { current: existing.current, previous, history: pushHistory(existing.history, historyEntryOf([{ ...record, superseded: true }])) };
|
||||
}
|
||||
return { current: { [target]: record }, history: pushHistory(existing.history, historyEntryOf(current)) };
|
||||
// The review of the run being replaced stays readable until the new run has a ready target. It replaces older
|
||||
// previous records rather than joining them: what the comment shows as one review comes from one run. A run without a
|
||||
// ready target has no review, so the one before it stays.
|
||||
const kept = current.some(isReady) ? Object.fromEntries(current.filter(isFinished).map(item => [recordTarget(item), item])) : previous;
|
||||
const next = { [target]: record };
|
||||
return { current: next, previous: trimPrevious(kept, next), history: pushHistory(existing.history, historyEntryOf(current)) };
|
||||
}
|
||||
|
||||
const isFinished = (record: PlatformRecord) => record.phase === "ready" || record.phase === "failed";
|
||||
const isReady = (record: PlatformRecord) => record.phase === "ready";
|
||||
|
||||
function ofRun(records: Partial<Record<CommentTarget, PlatformRecord>>, runId: string): Partial<Record<CommentTarget, PlatformRecord>> {
|
||||
return Object.fromEntries(Object.entries(records).filter(([, record]) => record?.runId === runId));
|
||||
}
|
||||
|
||||
/**
|
||||
* A target's previous record goes once its current one is ready; all of them go once every current record finished
|
||||
* and one is ready. A failure is no review: the previous one stays up beside it.
|
||||
*/
|
||||
function trimPrevious(previous: Partial<Record<CommentTarget, PlatformRecord>>, current: Partial<Record<CommentTarget, PlatformRecord>>): Partial<Record<CommentTarget, PlatformRecord>> {
|
||||
const records = Object.values(current).filter((record): record is PlatformRecord => Boolean(record));
|
||||
if (records.every(isFinished) && records.some(isReady)) return {};
|
||||
const trimmed = { ...previous };
|
||||
for (const record of records) if (isReady(record)) delete trimmed[recordTarget(record)];
|
||||
return trimmed;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -388,7 +442,7 @@ export function foldRecord(existing: CommentRecords, incoming: PlatformRecord):
|
||||
function withKnownTarget(record: PlatformRecord, existing: CommentRecords): PlatformRecord {
|
||||
const target = recordTarget(record);
|
||||
if (!record.target || record.label !== undefined || record.order !== undefined) return record;
|
||||
const known = existing.current[target];
|
||||
const known = existing.current[target] ?? existing.previous?.[target];
|
||||
if (known) return { ...record, platform: known.platform, ...(known.label !== undefined ? { label: known.label } : {}), ...(known.order !== undefined ? { order: known.order } : {}) };
|
||||
const item = existing.history.find(entry => entry.platforms[target])?.platforms[target];
|
||||
return item ? { ...record, ...(item.label !== undefined ? { label: item.label } : {}), ...(item.order !== undefined ? { order: item.order } : {}) } : record;
|
||||
|
||||
@@ -87,7 +87,7 @@ test("the Android controller open response is accepted only for the Android plat
|
||||
assert.throws(() => openedSimulatorDevice(response, "dev.brent.euxy", "open", "ios"), /full ios device shape/);
|
||||
});
|
||||
|
||||
test("the PR comment names Android builds and runtimes", () => {
|
||||
test("the PR comment names the Android target while it captures", () => {
|
||||
const pullRequest = { repositoryId: "1", repository: "expo/explore-playground", number: 1, title: "Android", url: "https://github.com/expo/explore-playground/pull/1",
|
||||
base: { ref: "main", sha: "a".repeat(40) }, head: { ref: "android", sha: "b".repeat(40) } };
|
||||
const body = renderPublicComment({
|
||||
@@ -99,7 +99,8 @@ test("the PR comment names Android builds and runtimes", () => {
|
||||
deviceModel: "EasAndroidDevice01", osVersion: "35", locale: "en-US", orientation: "portrait", colorScheme: "light" },
|
||||
capture: { planned: { base: 3, head: 3 }, done: { base: 0, head: 1 }, baseFromCache: false },
|
||||
});
|
||||
assert.match(body, /\| Android \| EasAndroidDevice01 35 · euxy 1\\\.2\\\.0 \\\(1\\\) \| ⏳ capturing head 1\/3 \|/);
|
||||
assert.match(body, /^### Explore review · capturing$/m);
|
||||
assert.match(body, /^Android · capturing head 1\/3 · \[run\]\(<[^>]+>\)$/m);
|
||||
assert.doesNotMatch(body, /iOS/);
|
||||
});
|
||||
|
||||
|
||||
@@ -203,8 +203,8 @@ test("ci:deploy seals the exact report, deploys it with the preview export, veri
|
||||
assert.ok(eas.requests.every(request => request.url.origin !== "https://acme-explore--1lcbdcpwn3.expo.app" || request.options.cache === "no-store"));
|
||||
const posted = eas.requests.find(request => request.options.method === "POST");
|
||||
const body = JSON.parse(posted.options.body).body;
|
||||
assert.match(body, new RegExp(`\\| \\[Open\\]\\(<https://acme-explore--1lcbdcpwn3\\.expo\\.app/p/${token}>\\) \\|`));
|
||||
assert.match(body, /Preview links work for anyone who has them until the deployment is deleted\./);
|
||||
assert.match(body, new RegExp(`^### [^\\n]*\\[Review in Explore\\]\\(<https://acme-explore--1lcbdcpwn3\\.expo\\.app/p/${token}>\\)$`, 'm'));
|
||||
assert.match(body, /Preview links work for anyone who has them until the deployment is deleted · /);
|
||||
assert.doesNotMatch(body, /explore-preview\.expo\.app\/\?/);
|
||||
assert.ok(eas.requests.indexOf(posted) > eas.requests.findIndex(request => request.url.pathname === `/p/${token}/viewer/index.html`), "publish after verification");
|
||||
});
|
||||
@@ -359,11 +359,10 @@ test("ci:deploy fans several platforms' reports into one sealed deployment, veri
|
||||
"/", `/p/${wrong}`, `/p/${wrong}/manifest.json`, `/p/${token}/viewer/index.html`]);
|
||||
const posted = eas.requests.filter(request => request.options.method === "POST" || request.options.method === "PATCH").map(request => JSON.parse(request.options.body).body);
|
||||
assert.equal(posted.length, 2, "the second platform updates the comment the first created");
|
||||
assert.match(posted[0], new RegExp(`\\| iOS \\| [^|]+\\| 🟩 [^|]+\\| \\[Open\\]\\(<https://acme-explore--1lcbdcpwn3\\.expo\\.app/p/${token}\\?platform=ios>\\)`));
|
||||
assert.doesNotMatch(posted[0], /\| Android \|/);
|
||||
assert.match(posted[1], new RegExp(`\\| iOS \\| [^|]+\\| 🟩 [^|]+\\| \\[Open\\]\\(<[^>]+/p/${token}\\?platform=ios>\\)`), 'the Android writer keeps the iOS row');
|
||||
assert.match(posted[1], new RegExp(`\\| Android \\| [^|]+\\| 🟩 [^|]+\\| \\[Open\\]\\(<[^>]+/p/${token}\\?platform=android>\\)`));
|
||||
assert.equal(posted[1].split("<summary>Validation").length, 2, 'one validation block for both platforms');
|
||||
assert.match(posted[0], new RegExp(`^<sub>\\[iOS\\]\\(<https://acme-explore--1lcbdcpwn3\\.expo\\.app/p/${token}\\?platform=ios>\\) · \\[`, 'm'));
|
||||
assert.doesNotMatch(posted[0], /Android/);
|
||||
assert.match(posted[1], new RegExp(`^<sub>\\[iOS\\]\\(<[^>]+/p/${token}\\?platform=ios>\\) · \\[Android\\]\\(<[^>]+/p/${token}\\?platform=android>\\) · \\[`, 'm'), 'the Android writer keeps the iOS link');
|
||||
assert.equal(posted[1].split("### ").length, 2, 'one heading for both platforms');
|
||||
});
|
||||
|
||||
function captureListing(listing) {
|
||||
@@ -422,7 +421,7 @@ test("a preview sets no expiry of its own: an old config's expiresAfter is noted
|
||||
assert.equal("EXPLORE_PREVIEW_EXPIRES_AT" in eas.captured.env, false);
|
||||
assert.equal(result.expiresAt, null);
|
||||
const body = JSON.parse(eas.requests.find(request => request.options.method === "PATCH" || request.options.method === "POST").options.body).body;
|
||||
assert.match(body, /^Preview links work for anyone who has them until the deployment is deleted\. · Managed via /m);
|
||||
assert.match(body, /^<sub>Preview links work for anyone who has them until the deployment is deleted · \[Workflow run\]/m);
|
||||
assert.doesNotMatch(body, /works until/);
|
||||
assert.ok(warnings.some(line => line.includes("report.preview.expiresAfter no longer applies: a hosted preview lasts as long as its EAS Hosting deployment")), warnings.join(""));
|
||||
|
||||
|
||||
@@ -78,8 +78,7 @@ for (const provider of ['expo-router', 'react-navigation']) {
|
||||
const published = await publishCiReportComment({ inputPath: result.inputPath,
|
||||
artifactUrl: 'https://github.com/acme/example/actions/runs/1234/artifacts/900', environment: f.environment, transport: f.transport });
|
||||
assert.equal(new URL(published.artifactUrl).origin, 'https://explore-preview.expo.app', 'CI always uses the hosted viewer, even with a legacy origin override');
|
||||
assert.match(f.comments.at(-1), /\[Open\]\(<https:\/\/explore-preview\.expo\.app\/\?[^>]+>\)/);
|
||||
assert.match(f.comments.at(-1), /\[zip\]\(<https:\/\/github\.com\/acme\/example\/actions\/runs\/1234\/artifacts\/900>\)/);
|
||||
assert.match(f.comments.at(-1), /\[Review in Explore\]\(<https:\/\/explore-preview\.expo\.app\/\?[^>]+artifact=900>\)/);
|
||||
assert.match(f.comments.at(-1), /explore-preview\.expo\.app/);
|
||||
assert.doesNotMatch(f.comments.at(-1), /viewer\.example/);
|
||||
// A matrix job's report artifact carries its target id: an unsuffixed artifact is not target duo-open's.
|
||||
@@ -132,12 +131,12 @@ test('unreachable screens do not fail CI: both revisions, the report and an Inco
|
||||
await publishCiReportComment({ inputPath: result.inputPath,
|
||||
artifactUrl: 'https://github.com/acme/example/actions/runs/1234/artifacts/900', environment: f.environment, transport: f.transport });
|
||||
const comment = f.comments.at(-1).replaceAll('\\', '');
|
||||
assert.match(comment, /\| iOS \| [^|]+\| 🟨 [^|]*unavailable[^|]*\| \[Open\]\(<https:\/\/explore-preview\.expo\.app/, 'the report link is published even though a screen is unavailable');
|
||||
assert.match(comment, /^### \d+ screens? changed · \[Review in Explore\]\(<https:\/\/explore-preview\.expo\.app/m, 'the report link is published even though a screen is unavailable');
|
||||
assert.match(comment, /^iOS · 1 capture unavailable · 1 unattributed · \[run\]/m);
|
||||
assert.doesNotMatch(comment, /🟥 failed/);
|
||||
assert.match(comment, /Head · \/home · CAPTURE_OBSERVER_NOT_READY/);
|
||||
assert.match(comment, /Base · \/home · CAPTURE_OBSERVER_MISSING/);
|
||||
assert.match(comment, /1 unavailable/);
|
||||
assert.match(comment, /1 unattributed/);
|
||||
assert.match(comment, /<summary>Capture issues \(2\)<\/summary>/);
|
||||
assert.equal(f.requests.filter(request => request.args[0] === 'simulator:stop').length, 2);
|
||||
assert.equal(git(f.baseProject, 'status', '--porcelain'), '');
|
||||
assert.equal(git(f.headProject, 'status', '--porcelain'), '');
|
||||
@@ -290,7 +289,7 @@ test('a PR that moved during a verification retry still publishes, as a previous
|
||||
assert.ok(f.comments.length > 0, 'the run keeps posting; its report is kept under previous runs');
|
||||
assert.match(f.comments.at(-1), /Previous runs \(1\)/);
|
||||
assert.match(f.comments.at(-1), /The pull request moved while this run worked/);
|
||||
assert.doesNotMatch(f.comments.at(-1), /^<sub>base \[/m, 'nothing is presented as the current review');
|
||||
assert.match(f.comments.at(-1), /^### Explore review$/m, 'nothing is presented as the current review');
|
||||
});
|
||||
|
||||
test('authorization and other non-server PR verification failures are never retried', async t => {
|
||||
|
||||
+329
-183
@@ -88,60 +88,61 @@ function reviewOnlyState(review) {
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
const visible = body => body.split('\n').filter(line => !line.startsWith('<!-- expo-explore-record') && !line.startsWith('<!-- expo-explore-history')).join('\n');
|
||||
const visible = body => body.split('\n').filter(line => !line.startsWith('<!-- expo-explore-record') && !line.startsWith('<!-- expo-explore-previous') && !line.startsWith('<!-- expo-explore-history')).join('\n');
|
||||
const summaries = body => [...body.matchAll(/<summary>([^<]+)<\/summary>/g)].map(match => match[1]);
|
||||
const platformRow = (body, platform) => body.split('\n').find(line => line.startsWith(`| ${platform} |`));
|
||||
const checkRow = (body, check) => body.split('\n').find(line => line.startsWith(`| ${check} |`));
|
||||
const titleOf = body => body.split('\n').find(line => line.startsWith('### '));
|
||||
/** The `<sub>` two lines under the heading; the footer is the last line. */
|
||||
const subtitleOf = body => { const lines = body.split('\n'); return lines[lines.findIndex(line => line.startsWith('### ')) + 2]; };
|
||||
const statusOf = (body, label) => body.split('\n').find(line => line.startsWith(`${label} · `) || line.startsWith(`**${label} failed**`));
|
||||
/** The changed-screen rows: the list before any collapsed block. */
|
||||
const listOf = body => visible(body).split('\n<details>')[0].split('\n').filter(line => line.startsWith('- '));
|
||||
const runLink = '[run](<https://github.com/expo/explore-playground/actions/runs/42/attempts/2>)';
|
||||
const workflowRun = '[Workflow run](<https://github.com/expo/explore-playground/actions/runs/42/attempts/2>)';
|
||||
const evidenceLink = '[evidence](<https://github.com/expo/explore-playground/actions/runs/42#artifacts>)';
|
||||
const reportUrl = 'https://explore-preview.expo.app/?repo=expo%2Fexplore-playground&run=42&attempt=2&artifact=7654';
|
||||
const commits = `[bbbbbbbb](<https://github.com/expo/explore-playground/commit/${headSha}>) vs [aaaaaaaa](<https://github.com/expo/explore-playground/commit/${baseSha}>)`;
|
||||
const configLink = `[Config](<https://github.com/expo/explore-playground/blob/${headSha}/.expo-explore/config.yaml>)`;
|
||||
|
||||
test('the ready comment is one header, a platform table, the changes listed once, and a collapsed validation block', () => {
|
||||
test('the ready comment is a heading with the count and the report, a subtitle, the changed screens, and a footer', () => {
|
||||
const body = renderPublicComment(state({ artifactOnly: true }));
|
||||
const lines = body.split('\n');
|
||||
assert.equal(lines[0], CI_COMMENT_MARKER);
|
||||
assert.match(lines[1], /^<!-- expo-explore-record:ios:[A-Za-z0-9_-]+ -->$/);
|
||||
assert.match(lines[2], /^### Explore review · \[Open report\]\(<https:\/\/explore-preview\.expo\.app\/\?repo=expo%2Fexplore-playground&run=42&attempt=2[^>]*>\)$/, 'the report link is the heading, where a phone shows it first');
|
||||
assert.equal(lines[4], '<sub>base [aaaaaaaa](<https://github.com/expo/explore-playground/commit/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa>) → [bbbbbbbb](<https://github.com/expo/explore-playground/commit/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb>)</sub>');
|
||||
assert.deepEqual([lines[3], ...lines.slice(5, 8)], ['', '', '| | Device | Result | Report |', '|---|---|---|---|']);
|
||||
assert.equal(lines[8], `| iOS | iPhone 17 26\\.5 · Example 1\\.4\\.0 \\(27\\) | 🟩 3 changes · 6m 12s | [Open](<https://explore-preview.expo.app/?repo=expo%2Fexplore-playground&run=42&attempt=2&artifact=7654>) |`, 'the Report cell is the report link alone');
|
||||
assert.deepEqual(summaries(body), ['🟩 Added (1)', '🟥 Removed (1)', '🟧 Modified (1)', 'Validation · iOS 🟩 6/6 captured']);
|
||||
assert.match(body, /\n\*\*Changes\*\* · 3 screens\n/);
|
||||
assert.match(body, /\n- `\/settings` — Changed button title from “Share” to “Share Example”\\\. · \[iOS\]\(<https:\/\/explore-preview\.expo\.app\/\?repo=expo%2Fexplore-playground&run=42&attempt=2&artifact=7654&view=compare&screen=settings>\)\n/);
|
||||
assert.match(body, /\n- `\/cats\/care-guide` — Added \/cats\/care\\-guide screen · \[iOS\]\(<[^>]+screen=care-guide>\)\n/);
|
||||
assert.match(body, /\n- `\/cats\/sweet-bee` — Removed \/cats\/sweet\\-bee screen · \[iOS\]\(<[^>]+screen=sweet-bee>\)\n/);
|
||||
assert.equal(checkRow(body, 'Capture'), '| Capture | 🟩 6/6 surfaces · base cache reused |');
|
||||
assert.equal(checkRow(body, 'Route verification'), '| Route verification | 🟩 6/6 verified |');
|
||||
assert.equal(checkRow(body, 'Visual comparison'), '| Visual comparison | 🟩 2 compared · threshold 0.1 |');
|
||||
assert.equal(checkRow(body, 'Accessibility comparison'), '| Accessibility comparison | 🟩 2 compared · 1 description |');
|
||||
assert.equal(checkRow(body, 'Source inventory'), '| Source inventory | 🟩 54 routes |');
|
||||
assert.equal(lines.at(-1), `Managed via [\`.expo-explore/config.yaml\`](<https://github.com/expo/explore-playground/blob/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb/.expo-explore/config.yaml>) · Workflow ${runLink} · [zip](<https://github.com/expo/explore-playground/actions/runs/42/artifacts/7654>)`, 'the run and the report zip sit in the footer');
|
||||
assert.doesNotMatch(body, /<img|### Review target|### What changed|### Validation|Unchanged|screen=cats|\/home|Reviewed at|Open Explore|Download report|View run/);
|
||||
assert.ok(body.indexOf('**Changes**') < body.indexOf('<summary>Validation'));
|
||||
assert.equal(lines[2], `### 3 screens changed · [Review in Explore](<${reportUrl}>)`, 'the report link is the heading, where a phone shows it first');
|
||||
assert.equal(lines[4], `<sub>[iOS](<${reportUrl}>) · ${commits} · updated Sep 11, 3:07 AM UTC</sub>`);
|
||||
assert.deepEqual(lines.slice(5, 10), ['',
|
||||
`- Added [\`/cats/care-guide\`](<${reportUrl}&view=compare&screen=care-guide>)`,
|
||||
`- Removed [\`/cats/sweet-bee\`](<${reportUrl}&view=compare&screen=sweet-bee>)`,
|
||||
`- [\`/settings\`](<${reportUrl}&view=compare&screen=settings>) · Changed button title from “Share” to “Share Example”`, '']);
|
||||
assert.equal(lines.at(-1), `<sub>${workflowRun} · ${configLink}</sub>`, 'the run and the config sit in the footer');
|
||||
assert.equal(lines.length, 11);
|
||||
assert.deepEqual(summaries(body), []);
|
||||
assert.doesNotMatch(body, /<img|\*\*Changes\*\*|Validation|Capture issues|🟩|🟧|🟨|Managed via|zip|6m 12s|iPhone 17|Example 1|Unchanged|screen=cats|\/home|Reviewed at|Open Explore|Download report|View run|^\|/m);
|
||||
// Hidden state is the only thing the comment carries besides markdown.
|
||||
assert.equal(lines.filter(line => line.startsWith('<!--')).length, 2);
|
||||
});
|
||||
|
||||
test('report links appear only once a report exists; a failed run links the run and its evidence', () => {
|
||||
test('report links appear only once a report exists; a failed target says so with its run and evidence', () => {
|
||||
const beforeUpload = viewerLinks({ repository: pullRequest.repository, headSha, runId: '42', runAttempt: '2', configPath: '.expo-explore/config.yaml' });
|
||||
const ready = renderPublicComment(state({ links: beforeUpload }));
|
||||
assert.match(platformRow(ready, 'iOS'), /\| 🟩 3 changes · 6m 12s \| \[run\]\(<[^>]+>\) \|$/, 'before the artifact exists there is nothing to open');
|
||||
assert.doesNotMatch(ready, /\[zip\]|\[Open\]/);
|
||||
assert.equal(titleOf(ready), '### 3 screens changed', 'before the artifact exists there is nothing to open');
|
||||
assert.match(subtitleOf(ready), /^<sub>iOS · \[bbbbbbbb\]/);
|
||||
assert.match(ready, /\n- Added `\/cats\/care-guide`\n/);
|
||||
assert.doesNotMatch(ready, /\[zip\]|\[Open\]|Review in Explore|explore-preview/);
|
||||
const failed = renderPublicComment(state({ phase: 'failed', review: undefined, evidence: undefined, failure: { step: 'capture-head', message: 'head capture did not complete; token=abc123 leaked' } }));
|
||||
assert.equal(platformRow(failed, 'iOS'), `| iOS | iPhone 17 26\\.5 · Example 1\\.4\\.0 \\(27\\) | 🟥 failed · capture\\-head | ${runLink} · [evidence](<https://github.com/expo/explore-playground/actions/runs/42#artifacts>) |`);
|
||||
assert.deepEqual(summaries(failed), ['Validation · iOS 🟥 failed']);
|
||||
assert.match(failed, /\*\*iOS failed\*\* — head capture did not complete; token=\\\[redacted\\\] leaked \[Open the failed run\]\(<[^>]+>\) · \[Evidence artifact\]\(<[^>]+>\)/);
|
||||
assert.equal(checkRow(failed, 'Capture'), '| Capture | 🟥 6/6 surfaces |');
|
||||
assert.equal(checkRow(failed, 'Visual comparison'), '| Visual comparison | — |');
|
||||
assert.doesNotMatch(failed, /\*\*Changes\*\*|\[Open\]|\[zip\]|abc123/);
|
||||
assert.equal(titleOf(failed), '### Explore review · capture failed');
|
||||
assert.equal(statusOf(failed, 'iOS'), `**iOS failed** · capture\\-head · head capture did not complete; token=\\[redacted\\] leaked · ${runLink} · ${evidenceLink}`);
|
||||
assert.deepEqual(listOf(failed), []);
|
||||
assert.doesNotMatch(failed, /\[Open\]|\[zip\]|abc123|Review in Explore|Validation/);
|
||||
});
|
||||
|
||||
test('a secondary appearance surface (<route>@<scheme>) is listed with its scheme and deep-linked like any other route', () => {
|
||||
const dark = surface('settings@dark', 'added', { route: '/settings', extra: { colorScheme: 'dark', review: { status: 'added', description: 'Added the dark appearance of /settings.' } } });
|
||||
const body = renderPublicComment(state({ review: review([dark]) }));
|
||||
assert.match(body, /\n- `\/settings` \(dark\) — Added [^·]+ · \[iOS\]\(<[^>]+&view=compare&screen=settings%40dark>\)\n/);
|
||||
assert.match(body, /\n- Added \[`\/settings`\]\(<[^>]+&view=compare&screen=settings%40dark>\) \(dark\)\n/, 'an added screen needs no description beyond its route');
|
||||
});
|
||||
|
||||
test('agent notes replace descriptions without changing groups, counts, or the rest of the comment', () => {
|
||||
test('agent notes replace descriptions without changing counts or the rest of the comment', () => {
|
||||
const original = evidenceReview();
|
||||
const annotated = structuredClone(original);
|
||||
for (const [index, text] of [[0, 'Added care guidance and a daily reminder action.'], [1, 'Removed the Sweet Bee activity screen.'], [2, 'The settings share action now reads “Share Example”.']]) {
|
||||
@@ -149,83 +150,78 @@ test('agent notes replace descriptions without changing groups, counts, or the r
|
||||
}
|
||||
const before = visible(renderPublicComment(state({ review: original })));
|
||||
const after = visible(renderPublicComment(state({ review: annotated })));
|
||||
assert.deepEqual(summaries(after), summaries(before));
|
||||
assert.equal(after.slice(after.indexOf('<summary>Validation')), before.slice(before.indexOf('<summary>Validation')));
|
||||
assert.equal(after.slice(0, after.indexOf('**Changes**')), before.slice(0, before.indexOf('**Changes**')));
|
||||
assert.match(after, /Added care guidance and a daily reminder action/);
|
||||
assert.match(after, /The settings share action now reads “Share Example”/);
|
||||
assert.equal(titleOf(after), titleOf(before));
|
||||
assert.equal(subtitleOf(after), subtitleOf(before));
|
||||
assert.equal(after.split('\n').at(-1), before.split('\n').at(-1));
|
||||
assert.equal(listOf(after).length, listOf(before).length);
|
||||
assert.match(after, /\n- Added \[`\/cats\/care-guide`\]\(<[^>]+>\) · Added care guidance and a daily reminder action\n/, 'a note on an added screen follows the route');
|
||||
assert.match(after, /\n- \[`\/settings`\]\(<[^>]+>\) · The settings share action now reads “Share Example”\n/);
|
||||
assert.doesNotMatch(after, /Changed button title from|evidenceDigest|Model|Tokens/);
|
||||
});
|
||||
|
||||
test('a long group is capped before the body budget and the comment stays under GitHub\'s limit', () => {
|
||||
const many = review(Array.from({ length: 40 }, (_, index) => surface(`screen-${index}`, 'modified', { extra: {
|
||||
review: { status: 'modified', description: `Changed ${index}` },
|
||||
test('the list shows ten screens and links the rest; the comment stays under GitHub\'s limit', () => {
|
||||
const screens = (count, description) => review(Array.from({ length: count }, (_, index) => surface(`screen-${index}`, 'modified', { extra: {
|
||||
review: { status: 'modified', description: description(index) },
|
||||
visual: { verdict: 'modified', reason: 'repeated-cross-build-delta', mismatchPercentage: 1, threshold: 0.1, tool: '[email protected]' },
|
||||
} })));
|
||||
const body = renderPublicComment(state({ review: many, capture: { planned: { base: 40, head: 40 }, done: { base: 40, head: 40 }, baseFromCache: false } }));
|
||||
assert.match(body, /<summary>🟧 Modified \(40\)<\/summary>/);
|
||||
assert.equal((body.match(/\n- `\/screen-\d+`/g) ?? []).length, 40);
|
||||
const body = renderPublicComment(state({ review: screens(40, index => `Changed ${index}`), capture: { planned: { base: 40, head: 40 }, done: { base: 40, head: 40 }, baseFromCache: false } }));
|
||||
assert.equal(titleOf(body), `### 40 screens changed · [Review in Explore](<${reportUrl}>)`);
|
||||
assert.equal((body.match(/\n- \[`\/screen-\d+`\]/g) ?? []).length, 10);
|
||||
assert.equal(listOf(body).at(-1), `- [30 more screens in the report](<${reportUrl}>)`);
|
||||
assert.ok(body.length < PUBLIC_COMMENT_BODY_BUDGET);
|
||||
const huge = review(Array.from({ length: 900 }, (_, index) => surface(`s${index}`, 'modified', { extra: { review: { status: 'modified', description: `Changed ${index} in a way that takes a sentence to describe for the budget check.` } } })));
|
||||
const long = renderPublicComment(state({ review: huge }));
|
||||
assert.equal(listOf(renderPublicComment(state({ review: screens(11, () => 'Changed') }))).at(-1), `- [1 more screen in the report](<${reportUrl}>)`);
|
||||
assert.equal(listOf(renderPublicComment(state({ review: screens(10, () => 'Changed') }))).length, 10, 'ten screens need no link to the rest');
|
||||
const long = renderPublicComment(state({ review: screens(900, index => `Changed ${index} in a way that takes a sentence to describe for the budget check.`) }));
|
||||
assert.ok(long.length < PUBLIC_COMMENT_BODY_BUDGET, String(long.length));
|
||||
assert.match(long, /<summary>🟧 Modified \(900\)<\/summary>/);
|
||||
assert.match(long, /\n- \d+ more — \[Explore report\]\(<[^>]+>\)\n/);
|
||||
assert.match(titleOf(long), /^### 900 screens changed · /);
|
||||
assert.match(long, /\n- \[890 more screens in the report\]\(<[^>]+>\)\n/);
|
||||
});
|
||||
|
||||
test('pending and capturing runs show progress in the platform table and no change lists', () => {
|
||||
const links = viewerLinks({ viewerUrl: 'https://explore-preview.expo.app/map', repository: pullRequest.repository, headSha, runId: '42', runAttempt: '2', configPath: '.expo-explore/config.yaml' });
|
||||
test('pending and capturing runs say what they are doing in the heading and a line per target, with no list', () => {
|
||||
const links = viewerLinks({ repository: pullRequest.repository, headSha, runId: '42', runAttempt: '2', configPath: '.expo-explore/config.yaml' });
|
||||
const pending = renderPublicComment(state({ phase: 'pending', links, review: undefined, evidence: undefined, runtime: undefined, capture: { planned: { base: 3, head: 3 }, done: { base: 0, head: 0 }, baseFromCache: false } }));
|
||||
assert.equal(platformRow(pending, 'iOS'), `| iOS | Example 1\\.4\\.0 \\(27\\) | ⏳ starting | ${runLink} |`);
|
||||
assert.equal(titleOf(pending), '### Explore review · starting');
|
||||
assert.equal(subtitleOf(pending), `<sub>iOS · ${commits} · updated Sep 11, 3:07 AM UTC</sub>`);
|
||||
assert.equal(statusOf(pending, 'iOS'), `iOS · starting · ${runLink}`);
|
||||
assert.deepEqual(listOf(pending), []);
|
||||
assert.deepEqual(summaries(pending), []);
|
||||
assert.doesNotMatch(pending, /\*\*Changes\*\*|Validation|Waiting|Open|zip/);
|
||||
const progress = (current, baseFromCache) => platformRow(renderPublicComment(state({ phase: 'capturing', links, review: undefined, evidence: undefined,
|
||||
assert.doesNotMatch(pending, /changed|Review in Explore|Validation|Waiting|Open|zip/);
|
||||
const progress = (current, baseFromCache) => statusOf(renderPublicComment(state({ phase: 'capturing', links, review: undefined, evidence: undefined,
|
||||
capture: { planned: { base: 3, head: 3 }, done: { base: 2, head: 1 }, baseFromCache, current } })), 'iOS');
|
||||
assert.match(progress('base', false), /\| ⏳ capturing base 2\/3 \| /);
|
||||
assert.match(progress('head', false), /\| ⏳ capturing head 1\/3 \| /);
|
||||
assert.match(progress('head', true), /\| ⏳ capturing head 1\/3 · base reused \| /);
|
||||
assert.match(progress(undefined, false), /\| ⏳ capturing head 1\/3 \| /);
|
||||
assert.equal(progress('base', false), `iOS · capturing base 2/3 · ${runLink}`);
|
||||
assert.equal(progress('head', false), `iOS · capturing head 1/3 · ${runLink}`);
|
||||
assert.equal(progress('head', true), `iOS · capturing head 1/3 · base reused · ${runLink}`);
|
||||
assert.equal(progress(undefined, false), `iOS · capturing head 1/3 · ${runLink}`);
|
||||
assert.equal(titleOf(renderPublicComment(state({ phase: 'capturing', links, review: undefined, evidence: undefined }))), '### Explore review · capturing');
|
||||
const none = renderPublicComment(state({ review: review([surface('cats', 'modified', { extra: { review: { status: 'unchanged', description: 'No visible change' } } })]) }));
|
||||
assert.deepEqual(summaries(none), ['Validation · iOS 🟩 6/6 captured'], 'unchanged surfaces produce no group');
|
||||
assert.match(none, /\*\*Changes\*\* · 0 screens\n\nNo screen changed on the reviewed platforms\./);
|
||||
assert.equal(titleOf(none), `### No screens changed · [Review in Explore](<${reportUrl}>)`, 'unchanged surfaces are no change');
|
||||
assert.deepEqual(listOf(none), []);
|
||||
});
|
||||
|
||||
test('an OS version the simulator did not report is omitted, not shown as unknown', () => {
|
||||
const body = renderPublicComment(state({ runtime: { ...state().runtime, osVersion: 'unknown' } }));
|
||||
assert.match(platformRow(body, 'iOS'), /^\| iOS \| iPhone 17 · Example 1\\\.4\\\.0 \\\(27\\\) \|/);
|
||||
assert.doesNotMatch(body, /unknown/i);
|
||||
});
|
||||
|
||||
test('without evidence the comment lists only added and removed screens and says comparisons did not run', () => {
|
||||
test('without evidence the comment lists only added and removed screens and counts the uncompared one', () => {
|
||||
const plain = review([surface('care-guide', 'added'), surface('sweet-bee', 'removed'), surface('settings', 'modified'), surface('cats', 'unchanged')]);
|
||||
const body = renderPublicComment(reviewOnlyState(plain));
|
||||
// A source change nobody compared is not a change the captures show.
|
||||
assert.deepEqual(summaries(body).slice(0, 2), ['🟩 Added (1)', '🟥 Removed (1)']);
|
||||
assert.doesNotMatch(body, /Source-affected|screen=settings/);
|
||||
assert.equal(checkRow(body, 'Visual comparison'), '| Visual comparison | 🟨 not run |');
|
||||
assert.equal(checkRow(body, 'Accessibility comparison'), '| Accessibility comparison | 🟨 not run |');
|
||||
assert.match(platformRow(body, 'iOS'), /^\| iOS \| Example 1\\\.4\\\.0 \\\(27\\\) \| 🟨 2 changes · 1 not compared · 0s \|/, 'the uncompared /settings is counted, not hidden');
|
||||
assert.doesNotMatch(body, /Needs review|Unchanged|screen=cats/);
|
||||
assert.match(titleOf(body), /^### 2 screens changed · /);
|
||||
assert.deepEqual(listOf(body).map(line => /^- (Added|Removed) /.exec(line)?.[1]), ['Added', 'Removed']);
|
||||
assert.equal(statusOf(body, 'iOS'), `iOS · 1 not compared · ${runLink}`, 'the uncompared /settings is counted, not hidden');
|
||||
assert.doesNotMatch(body, /Source-affected|screen=settings|Needs review|Unchanged|screen=cats|not run/);
|
||||
});
|
||||
|
||||
test('the completed duration stays tied to the captured review when the comment is republished', () => {
|
||||
const original = platformRow(renderPublicComment(state()), 'iOS');
|
||||
assert.equal(platformRow(renderPublicComment(state({ updatedAt: Date.parse('2026-09-12T09:42:00Z') })), 'iOS'), original);
|
||||
assert.match(original, /🟩 3 changes · 6m 12s/);
|
||||
assert.doesNotMatch(renderPublicComment(state({ phase: 'capturing', review: undefined })), /6m 12s/);
|
||||
test('the subtitle says when the comment last changed: the review\'s time once ready, the writer\'s before', () => {
|
||||
assert.match(subtitleOf(renderPublicComment(state({ updatedAt: Date.parse('2026-09-12T09:42:00Z') }))), /· updated Sep 11, 3:07 AM UTC<\/sub>$/, 'a republished review keeps its time');
|
||||
assert.match(subtitleOf(renderPublicComment(state({ phase: 'capturing', review: undefined, updatedAt: Date.parse('2026-09-12T09:42:00Z') }))), /· updated Sep 12, 9:42 AM UTC<\/sub>$/);
|
||||
});
|
||||
|
||||
test('needs-review surfaces keep their group and validation status without capture warnings', () => {
|
||||
test('needs-review surfaces are tagged on their row; screen variants and capture warnings stay out', () => {
|
||||
const flagged = evidenceReview();
|
||||
flagged.surfaces[2].review = { status: 'needs-review', description: 'Screenshots differ (2.1%) without a source change' };
|
||||
flagged.surfaces.push(surface('coins', 'unchanged', { extra: { sourceScope: 'none', review: { status: 'variant', description: 'Runtime state differs: Removed text “84”.' },
|
||||
visual: { verdict: 'modified', reason: 'repeated-cross-build-delta', mismatchPercentage: 0.2, threshold: 0.1, tool: '[email protected]' } } }));
|
||||
const body = renderPublicComment(state({ review: flagged, warnings: ['src/app/orphan.tsx has unresolved navigation'], inventory: { routes: 54 } }));
|
||||
assert.match(body, /<summary>🟨 Needs review \(1\)<\/summary>/);
|
||||
assert.doesNotMatch(body, /<summary>🟧 Modified/);
|
||||
assert.match(body, /\n- `\/settings` — Screenshots differ \\\(2\\\.1%\\\) without a source change · \[iOS\]/);
|
||||
assert.equal(checkRow(body, 'Source inventory'), '| Source inventory | 🟩 54 routes |');
|
||||
assert.doesNotMatch(body, /Capture warnings|orphan/);
|
||||
assert.match(titleOf(body), /^### 3 screens changed · /);
|
||||
assert.match(body, /\n- \[`\/settings`\]\(<[^>]+>\) · Screenshots differ \\\(2\\\.1%\\\) without a source change · needs review\n/);
|
||||
assert.doesNotMatch(body, /coins|Capture warnings|orphan|Needs review \(|54 routes/);
|
||||
});
|
||||
|
||||
test('text is redacted and escaped before it reaches the comment, and credentialed links are refused', () => {
|
||||
@@ -236,19 +232,18 @@ test('text is redacted and escaped before it reaches the comment, and credential
|
||||
assert.throws(() => renderPublicComment(state({ links: { ...state().links, run: 'https://user:[email protected]/x' } })), /credentials/);
|
||||
});
|
||||
|
||||
test('completed partial capture shows unavailable targets rather than ongoing work', () => {
|
||||
test('a completed partial capture gets one line with the count of missing captures, not ongoing work', () => {
|
||||
const capture = { planned: { base: 34, head: 34 }, done: { base: 21, head: 34 }, baseFromCache: false };
|
||||
const ready = renderPublicComment(state({ capture }));
|
||||
assert.equal(checkRow(ready, 'Capture'), '| Capture | 🟨 55/68 surfaces · 13 unavailable |');
|
||||
assert.match(platformRow(ready, 'iOS'), /\| 🟨 3 changes · 13 unavailable · 6m 12s \|/);
|
||||
assert.doesNotMatch(ready, /⏳/);
|
||||
assert.equal(checkRow(renderPublicComment(state({ phase: 'failed', capture, failure: { step: 'capture-base', message: 'failed' } })), 'Capture'), '| Capture | 🟥 55/68 surfaces |');
|
||||
assert.equal(checkRow(renderPublicComment(state({ capture: { ...capture, done: { base: 34, head: 34 } } })), 'Capture'), '| Capture | 🟩 68/68 surfaces |');
|
||||
assert.equal(checkRow(renderPublicComment(state({ capture: { ...capture, done: { base: 0, head: 0 } } })), 'Capture'), '| Capture | 🟨 6/68 surfaces · 62 unavailable |', 'the review’s own frames count as captured');
|
||||
assert.equal(checkRow(renderPublicComment(state({ capture: { ...capture, baseFromCache: true } })), 'Capture'), '| Capture | 🟨 55/68 surfaces · 13 unavailable · base cache reused |');
|
||||
assert.equal(statusOf(ready, 'iOS'), `iOS · 13 captures unavailable · ${runLink}`);
|
||||
assert.match(titleOf(ready), /^### 3 screens changed · /);
|
||||
assert.doesNotMatch(ready, /⏳|Capture issues|surfaces/);
|
||||
assert.equal(statusOf(renderPublicComment(state({ phase: 'failed', capture, failure: { step: 'capture-base', message: 'failed' } })), 'iOS'), `**iOS failed** · capture\\-base · failed · ${runLink} · ${evidenceLink}`);
|
||||
assert.equal(statusOf(renderPublicComment(state({ capture: { ...capture, done: { base: 34, head: 34 } } })), 'iOS'), undefined, 'a complete capture needs no line');
|
||||
assert.equal(statusOf(renderPublicComment(state({ capture: { ...capture, done: { base: 0, head: 0 } } })), 'iOS'), `iOS · 62 captures unavailable · ${runLink}`, 'the review’s own frames count as captured');
|
||||
});
|
||||
|
||||
test('capture problems are listed as diagnostics inside the validation block with both sides', () => {
|
||||
test('capture problems are counted on the target\'s line and listed, collapsed, with both sides', () => {
|
||||
const flagged = evidenceReview();
|
||||
const failure = side => ({ status: 'failed', attempts: 1, landing: 'unverified', error: {
|
||||
code: `${side.toUpperCase()}_SESSION_FAILED`, stage: 'capture', message: `${side} simulator disconnected; token=private`,
|
||||
@@ -262,17 +257,18 @@ test('capture problems are listed as diagnostics inside the validation block wit
|
||||
flagged.surfaces[2].headOutcome.landing = 'unverified';
|
||||
flagged.surfaces[2].review = { status: 'needs-review', description: 'old destination failure' };
|
||||
const body = renderPublicComment(state({ review: flagged, capture: { planned: { base: 3, head: 3 }, done: { base: 1, head: 2 }, baseFromCache: false } }));
|
||||
// /settings could not be compared, so it is no change: its failures are named under Validation.
|
||||
assert.deepEqual(summaries(body), ['🟩 Added (1)', '🟥 Removed (1)', 'Validation · iOS 🟨 3/6 captured']);
|
||||
assert.match(platformRow(body, 'iOS'), /\| 🟨 2 changes · 3 unavailable · 1 unattributed · 6m 12s \|/);
|
||||
// /settings could not be compared, so it is no change: its failures are named under Capture issues.
|
||||
assert.match(titleOf(body), /^### 2 screens changed · /);
|
||||
assert.equal(statusOf(body, 'iOS'), `iOS · 3 captures unavailable · 1 unattributed · ${runLink}`);
|
||||
assert.deepEqual(summaries(body), ['Capture issues (4)']);
|
||||
assert.ok(body.indexOf('<summary>Capture issues') > body.indexOf('\n- Removed '), 'the list comes first');
|
||||
assert.match(body, /\n- Base · \/settings · /);
|
||||
assert.match(body, /\*\*iOS\*\* · base 1\/3 verified · 2 unavailable · head 1\/3 verified · 1 unattributed · 1 unavailable/);
|
||||
assert.match(body, /\n- Base · \/cats\/sweet\\-bee · BASE\\_SESSION\\_FAILED: base simulator disconnected; token=\\\[redacted\\\] Next action: Restart the base simulator session and rerun capture\\\./);
|
||||
assert.match(body, /\n- Head · \/cats\/care\\-guide · HEAD\\_SESSION\\_FAILED/);
|
||||
assert.doesNotMatch(body, /Needs review|Modified|old capture|old destination|token=private/);
|
||||
assert.doesNotMatch(body, /needs review|Modified|old capture|old destination|token=private|\*\*iOS\*\*|Validation/);
|
||||
});
|
||||
|
||||
test('zero eligible pairs are visibly skipped instead of successful comparisons', () => {
|
||||
test('zero eligible pairs are counted as unattributed instead of successful comparisons', () => {
|
||||
const flagged = evidenceReview();
|
||||
for (const surface of flagged.surfaces) {
|
||||
for (const side of ['base', 'head']) if (surface[`${side}Outcome`]) surface[`${side}Outcome`].landing = 'unverified';
|
||||
@@ -284,30 +280,29 @@ test('zero eligible pairs are visibly skipped instead of successful comparisons'
|
||||
visual: { compared: 0, modified: 0, unchanged: 0, needsReview: 0, failed: 0, errors: [], threshold: 0.1, tool: '[email protected]' },
|
||||
accessibility: { compared: 0, described: 0, failed: 0, skipped: 0, errors: [] },
|
||||
} }));
|
||||
assert.equal(checkRow(body, 'Route verification'), '| Route verification | 🟨 0/6 verified · 6 unattributed |');
|
||||
assert.equal(checkRow(body, 'Visual comparison'), '| Visual comparison | 🟨 0 compared · threshold 0.1 · 2 skipped |');
|
||||
assert.equal(checkRow(body, 'Accessibility comparison'), '| Accessibility comparison | 🟨 0 compared · 0 descriptions |');
|
||||
assert.match(platformRow(body, 'iOS'), /\| 🟨 2 changes · 6 unattributed · 6m 12s \|/, 'the uncompared modified screens are not changes');
|
||||
assert.match(titleOf(body), /^### 2 screens changed · /, 'the uncompared modified screens are not changes');
|
||||
assert.equal(statusOf(body, 'iOS'), `iOS · 6 unattributed · ${runLink}`);
|
||||
});
|
||||
|
||||
test('the review-only comment counts failed outcomes in planned coverage and lists the reason', () => {
|
||||
test('the review-only comment counts failed outcomes as unavailable and lists the reason', () => {
|
||||
const added = surface('added', 'added');
|
||||
delete added.headImage;
|
||||
added.headOutcome = { status: 'missing-params', attempts: 0, landing: 'unverified', reason: 'Provide the item ID.' };
|
||||
const body = renderPublicComment(reviewOnlyState(review([added])));
|
||||
assert.equal(checkRow(body, 'Capture'), '| Capture | 🟨 0/1 surfaces · 1 unavailable |');
|
||||
assert.match(body, /<summary>🟩 Added \(1\)<\/summary>/);
|
||||
assert.equal(statusOf(body, 'iOS'), `iOS · 1 capture unavailable · ${runLink}`);
|
||||
assert.match(body, /\n- Added \[`\/added`\]\(<[^>]+screen=added>\)\n/);
|
||||
assert.deepEqual(summaries(body), ['Capture issues (1)']);
|
||||
assert.match(body, /CAPTURE\\_UNAVAILABLE: Provide the item ID\\\./);
|
||||
});
|
||||
|
||||
test('a failed run keeps the source additions and removals of its partial report', () => {
|
||||
test('a failed run keeps its message on the target\'s line and lists nothing', () => {
|
||||
const body = renderPublicComment(state({ phase: 'failed', failure: { step: 'capture', message: 'Capture coverage is incomplete.' } }));
|
||||
assert.match(platformRow(body, 'iOS'), /\| 🟥 failed · capture \|/);
|
||||
assert.match(body, /\*\*iOS failed\*\* — Capture coverage is incomplete\\\./);
|
||||
assert.doesNotMatch(body, /\*\*Changes\*\*|🟩 Added/, 'a failed run has no review to list');
|
||||
assert.equal(titleOf(body), '### Explore review · capture failed');
|
||||
assert.equal(statusOf(body, 'iOS'), `**iOS failed** · capture · Capture coverage is incomplete\\. · ${runLink} · ${evidenceLink}`);
|
||||
assert.deepEqual(listOf(body), [], 'a failed run has no review to list');
|
||||
});
|
||||
|
||||
test('screen variants and app-wide changes are stated once, not listed as review targets', () => {
|
||||
test('screen variants and app-wide source changes are left to the report', () => {
|
||||
const variant = surface('coins', 'unchanged', { extra: { sourceScope: 'none',
|
||||
review: { status: 'variant', description: 'Runtime state differs: Removed text “84”.' },
|
||||
visual: { verdict: 'modified', reason: 'repeated-cross-build-delta', mismatchPercentage: 0.2, threshold: 0.1, tool: '[email protected]' } } });
|
||||
@@ -315,29 +310,40 @@ test('screen variants and app-wide changes are stated once, not listed as review
|
||||
review: { status: 'unchanged', description: 'No visible change' },
|
||||
sourceChange: { ...sourceChange('Dependency update.', 'global'), files: [{ path: 'package.json', status: 'modified', relation: 'global', hunks: [], facts: [], truncated: false }] } } });
|
||||
const body = renderPublicComment(state({ review: review([...evidenceReview().surfaces, variant, appWide]) }));
|
||||
assert.doesNotMatch(body, /\/coins|\/profile|Needs review \(/);
|
||||
assert.match(body, /1 screen differs only in runtime state \(screen variants\); no source change reaches it\./);
|
||||
assert.match(body, /App-wide source changes \(`package\.json`\) reach every screen; screens without a visible difference are not listed\./);
|
||||
assert.match(titleOf(body), /^### 3 screens changed · /);
|
||||
assert.doesNotMatch(body, /\/coins|\/profile|runtime state|App-wide|package\.json/);
|
||||
});
|
||||
|
||||
test('a hosted preview is the Open link, the footer keeps the zip and run and says how long the link works', () => {
|
||||
test('a changed splash screen is one line after the list; an unchanged one is not mentioned', () => {
|
||||
const launch = verdict => ({ ...evidenceReview(), launch: [
|
||||
{ colorScheme: 'light', head: { status: 'observed', source: 'recording', splashVisibleMs: 900, firstScreenMs: 1900 }, base: { status: 'observed' }, visual: { verdict } },
|
||||
{ colorScheme: 'dark', head: { status: 'observed', source: 'recording', splashVisibleMs: 900, firstScreenMs: 1900 }, base: { status: 'observed' }, visual: { verdict: 'unchanged' } }] });
|
||||
const changed = renderPublicComment(state({ review: launch('modified'), artifactOnly: true }));
|
||||
assert.match(changed, /\n- \[`\/settings`\][^\n]+\n\nSplash screen changed in light mode\n\n<sub>/);
|
||||
assert.match(renderPublicComment(state({ review: launch('needs-review'), artifactOnly: true })), /\nSplash screen needs review in light mode\n/);
|
||||
assert.doesNotMatch(renderPublicComment(state({ review: launch('unchanged'), artifactOnly: true })), /Splash|Launch|first screen/);
|
||||
const both = { ...evidenceReview(), launch: [{ head: { status: 'observed', source: 'recording' }, base: { status: 'observed' }, visual: { verdict: 'modified' } }] };
|
||||
assert.match(renderPublicComment(state({ review: both, artifactOnly: true })), /\nSplash screen changed\n/);
|
||||
});
|
||||
|
||||
test('a hosted preview is the report link, and the footer says how long the links work', () => {
|
||||
const token = 'abcdefghijklmnopqrstuvwxyz234567';
|
||||
const preview = previewLinks(`https://acme-explore--1lcbdcpwn3.expo.app/p/${token}`, Date.parse('2026-09-18T03:07:00Z'));
|
||||
const links = { ...viewerLinks({ repository: pullRequest.repository, headSha, runId: '42', runAttempt: '2', configPath: '.expo-explore/config.yaml', artifactUrl }), preview };
|
||||
const body = renderPublicComment(state({ links, artifactOnly: true }));
|
||||
const lines = body.split('\n');
|
||||
assert.equal(lines[1], '<!-- expo-explore-deployments:acme-explore--1lcbdcpwn3 -->');
|
||||
assert.match(platformRow(body, 'iOS'), new RegExp(`\\| \\[Open\\]\\(<https://acme-explore--1lcbdcpwn3\\.expo\\.app/p/${token}>\\) \\|$`));
|
||||
assert.match(lines.at(-1), /· Workflow \[run\]\(<[^>]+>\) · \[zip\]\(<[^>]+>\)$/);
|
||||
assert.match(body, new RegExp(`\\[iOS\\]\\(<https://acme-explore--1lcbdcpwn3\\.expo\\.app/p/${token}\\?view=compare&screen=settings>\\)`));
|
||||
assert.match(lines.at(-1), /^Preview links work until Sep 18, 2026 \(UTC\) for anyone who has them\. · Managed via /);
|
||||
assert.doesNotMatch(body, /Open Explore|explore-viewer\.expo\.app|explore-preview\.expo\.app/);
|
||||
assert.equal(titleOf(body), `### 3 screens changed · [Review in Explore](<https://acme-explore--1lcbdcpwn3.expo.app/p/${token}>)`);
|
||||
assert.equal(subtitleOf(body), `<sub>[iOS](<https://acme-explore--1lcbdcpwn3.expo.app/p/${token}>) · ${commits} · updated Sep 11, 3:07 AM UTC</sub>`);
|
||||
assert.match(body, new RegExp(`\\n- \\[\`/settings\`\\]\\(<https://acme-explore--1lcbdcpwn3\\.expo\\.app/p/${token}\\?view=compare&screen=settings>\\) · `));
|
||||
assert.equal(lines.at(-1), `<sub>Preview links work for anyone who has them until Sep 18, 2026 (UTC) · ${workflowRun} · ${configLink}</sub>`);
|
||||
assert.doesNotMatch(body, /Open Explore|explore-viewer\.expo\.app|explore-preview\.expo\.app|zip/);
|
||||
assert.doesNotMatch(renderPublicComment(state({ links, phase: 'failed', review: undefined, failure: { step: 'capture-head', message: 'boom' } })), /acme-explore--1lcbdcpwn3\.expo\.app\/p|for anyone who has/);
|
||||
const forever = renderPublicComment(state({ links: { ...links, preview: previewLinks(`https://acme-explore--1lcbdcpwn3.expo.app/p/${token}`, null) }, artifactOnly: true }));
|
||||
assert.match(forever, /\nPreview links work for anyone who has them until the deployment is deleted\. · Managed via /);
|
||||
assert.match(forever, /\n<sub>Preview links work for anyone who has them until the deployment is deleted · \[Workflow run\]/);
|
||||
});
|
||||
|
||||
test('two platforms share one comment: one table row each, every changed screen listed once with a link per platform, one validation table', async () => {
|
||||
test('two platforms share one comment: both named in the subtitle, every changed screen listed once, a line only for the one still working', async () => {
|
||||
const { platformSectionMerge, readCommentRecords } = await import('../../dist/report/pr-comment-body.js');
|
||||
const ready = state({ artifactOnly: true });
|
||||
const single = renderPublicComment(ready);
|
||||
@@ -347,29 +353,24 @@ test('two platforms share one comment: one table row each, every changed screen
|
||||
runtime: { ...state().runtime, platform: 'android', deviceModel: 'EasAndroidDevice01', osVersion: '15.0' }, ...overrides });
|
||||
const capturing = platformSectionMerge(androidState({ phase: 'capturing', review: undefined, evidence: undefined, capture: { planned: { base: 3, head: 3 }, done: { base: 3, head: 1 }, baseFromCache: false, current: 'head' } }), 'android').merge(single);
|
||||
assert.deepEqual(Object.keys(readCommentRecords(capturing).current), ['ios', 'android']);
|
||||
assert.equal(platformRow(capturing, 'iOS'), platformRow(single, 'iOS'));
|
||||
assert.equal(platformRow(capturing, 'Android'), `| Android | EasAndroidDevice01 15\\.0 · Example 1\\.4\\.0 \\(27\\) | ⏳ capturing head 1/3 | ${runLink} |`);
|
||||
assert.deepEqual(summaries(capturing), ['🟩 Added (1)', '🟥 Removed (1)', '🟧 Modified (1)', 'Validation · iOS 🟩 6/6 captured · Android ⏳ in progress']);
|
||||
assert.match(capturing, /\| Check \| iOS \| Android \|\n\|---\|---\|---\|\n\| Capture \| 🟩 6\/6 surfaces · base cache reused \| ⏳ \|/);
|
||||
assert.match(capturing, /\n- `\/settings` — Changed button title from “Share” to “Share Example”\\\. · \[iOS\]\(<[^>]+>\)\n/, 'a platform still capturing adds no link');
|
||||
assert.equal(titleOf(capturing), titleOf(single), 'a platform still capturing has no report to link yet');
|
||||
assert.equal(subtitleOf(capturing), `<sub>[iOS](<${reportUrl}>) · Android · ${commits} · updated Sep 11, 3:07 AM UTC</sub>`);
|
||||
assert.equal(statusOf(capturing, 'Android'), `Android · capturing head 1/3 · ${runLink}`);
|
||||
assert.equal(statusOf(capturing, 'iOS'), undefined);
|
||||
assert.deepEqual(listOf(capturing), listOf(single), 'a platform still capturing adds nothing to the list');
|
||||
const flagged = evidenceReview();
|
||||
flagged.surfaces[2].review = { status: 'needs-review', description: 'Screenshots differ (2.1%) without a source change' };
|
||||
flagged.surfaces.push(surface('coins', 'unchanged', { extra: { sourceScope: 'none', review: { status: 'variant', description: 'Runtime state differs: Removed text “84”.' },
|
||||
visual: { verdict: 'modified', reason: 'repeated-cross-build-delta', mismatchPercentage: 0.2, threshold: 0.1, tool: '[email protected]' } } }));
|
||||
const both = platformSectionMerge(androidState({ review: flagged }), 'android').merge(capturing);
|
||||
assert.deepEqual(Object.keys(readCommentRecords(both).current), ['ios', 'android']);
|
||||
assert.equal(platformRow(both, 'iOS'), platformRow(single, 'iOS'));
|
||||
assert.match(platformRow(both, 'Android'), /^\| Android \| EasAndroidDevice01 15\\\.0 · Example 1\\\.4\\\.0 \\\(27\\\) \| 🟩 3 changes · 6m 12s \| \[Open\]/);
|
||||
assert.match(both, /\*\*Changes\*\* · 3 screens · a screen is listed once, with a link per platform\n/);
|
||||
assert.match(capturing, /^### Explore review · \[Open report\]\(<[^>]+>\)$/m, 'a platform still capturing has no report to link yet');
|
||||
assert.match(both, /^### Explore review · \[iOS report\]\(<[^>]+>\) · \[Android report\]\(<[^>]+>\)$/m, 'with two platforms the heading links each report');
|
||||
assert.match(both, /\nAndroid · 1 screen differs only in runtime state \(screen variants\); no source change reaches it\.\n/, 'a context line from one platform names it');
|
||||
assert.doesNotMatch(both, /iOS · /);
|
||||
assert.match(both, /\n- `\/cats\/care-guide` — Added \/cats\/care\\-guide screen · \[iOS\]\(<[^>]+screen=care-guide>\) · \[Android\]\(<[^>]+screen=care-guide>\)\n/);
|
||||
assert.match(both, /\n- `\/settings` — Changed button title from “Share” to “Share Example”\\\. · \[iOS\]\(<[^>]+>\) · \[Android \(needs review\)\]\(<[^>]+>\)\n/, 'a platform whose verdict differs says so on the row');
|
||||
assert.deepEqual(summaries(both), ['🟩 Added (1)', '🟥 Removed (1)', '🟧 Modified (1)', 'Validation · iOS 🟩 6/6 captured · Android 🟩 8/8 captured']);
|
||||
assert.equal((both.match(/\n- `/g) ?? []).length, 3, 'no screen is listed twice');
|
||||
assert.equal((both.match(/\| Capture \|/g) ?? []).length, 1);
|
||||
assert.equal(titleOf(both), titleOf(single), 'the heading links the first platform’s report');
|
||||
assert.match(subtitleOf(both), /^<sub>\[iOS\]\(<[^>]+artifact=7654>\) · \[Android\]\(<[^>]+artifact=7654>\) · \[bbbbbbbb\]/);
|
||||
assert.equal(statusOf(both, 'Android'), undefined);
|
||||
assert.equal(listOf(both).length, 3, 'no screen is listed twice');
|
||||
assert.match(both, /\n- Added \[`\/cats\/care-guide`\]\(<[^>]+screen=care-guide>\)\n/);
|
||||
assert.match(both, /\n- \[`\/settings`\]\(<[^>]+>\) · Changed button title from “Share” to “Share Example”\n/, 'a platform whose verdict differs does not change the row');
|
||||
assert.doesNotMatch(both, / only\n|not on |coins|runtime state|iOS · /);
|
||||
assert.ok(both.length < 8_000, String(both.length));
|
||||
// A writer that finds no records (a legacy or hand-edited comment) starts over rather than failing.
|
||||
const legacy = `${CI_COMMENT_MARKER}\nExplore review for x\n<!-- expo-explore-platform:ios -->\n## iOS · 3 changes\n<!-- /expo-explore-platform:ios -->`;
|
||||
@@ -378,32 +379,68 @@ test('two platforms share one comment: one table row each, every changed screen
|
||||
assert.equal(platformSectionMerge(ready, 'ios').merge(tampered), single);
|
||||
});
|
||||
|
||||
test('a newer run moves the current run into Previous runs; a run the PR moved past joins history without displacing the current review', async () => {
|
||||
test('a screen that changed on some devices only says which, the shorter way', async () => {
|
||||
const { platformSectionMerge } = await import('../../dist/report/pr-comment-body.js');
|
||||
const [iphone, closed, open, android] = duoTargets;
|
||||
const settingsOnly = review([evidenceReview().surfaces[2]]);
|
||||
let body;
|
||||
for (const target of [iphone, closed, open]) body = platformSectionMerge(targetState(target), target.id).merge(body);
|
||||
body = platformSectionMerge(targetState(android, { review: settingsOnly }), 'android').merge(body);
|
||||
assert.match(body, /\n- Added \[`\/cats\/care-guide`\]\(<[^>]+>\) · not on Pixel 7\n/);
|
||||
assert.match(body, /\n- \[`\/settings`\]\(<[^>]+>\) · Changed button title from “Share” to “Share Example”\n/, 'a screen changed everywhere names no device');
|
||||
const two = platformSectionMerge(targetState(android, { review: settingsOnly }), 'android').merge(platformSectionMerge(targetState(iphone), 'iphone').merge(undefined));
|
||||
assert.match(two, /\n- Added \[`\/cats\/care-guide`\]\(<[^>]+>\) · iPhone 17 only\n/);
|
||||
// A device missing a capture may have changed the screen too: the row says where it changed, not where it did not.
|
||||
const missing = platformSectionMerge(targetState(android, { review: settingsOnly, capture: { planned: { base: 3, head: 3 }, done: { base: 3, head: 2 }, baseFromCache: false } }), 'android')
|
||||
.merge(platformSectionMerge(targetState(iphone), 'iphone').merge(undefined));
|
||||
assert.match(missing, /\nPixel 7 · 1 capture unavailable · /);
|
||||
assert.match(missing, /\n- Added \[`\/cats\/care-guide`\]\(<[^>]+>\) · on iPhone 17\n/);
|
||||
assert.doesNotMatch(missing, / only\n|not on /);
|
||||
});
|
||||
|
||||
test('a new run keeps the last finished review up, under a line saying it is working, until it has a finished target; the old run joins Previous runs', async () => {
|
||||
const { platformSectionMerge, readCommentRecords } = await import('../../dist/report/pr-comment-body.js');
|
||||
const first = platformSectionMerge(state({ artifactOnly: true }), 'ios').merge(undefined);
|
||||
const androidReady = state({ artifactOnly: true, build: { ...state().build, platform: 'android' }, runtime: { ...state().runtime, platform: 'android', deviceModel: 'Pixel 9' } });
|
||||
const both = platformSectionMerge(androidReady, 'android').merge(first);
|
||||
const nextHead = 'd'.repeat(40);
|
||||
const nextLinks = runId => viewerLinks({ repository: pullRequest.repository, headSha: nextHead, runId, runAttempt: '1', configPath: '.expo-explore/config.yaml' });
|
||||
const nextLinks = (runId, artifact) => viewerLinks({ ...(artifact ? { artifactUrl: `https://github.com/expo/explore-playground/actions/runs/${runId}/artifacts/${artifact}` } : {}),
|
||||
repository: pullRequest.repository, headSha: nextHead, runId, runAttempt: '1', configPath: '.expo-explore/config.yaml' });
|
||||
const nextState = overrides => state({ phase: 'capturing', review: undefined, evidence: undefined, links: nextLinks('43'), pullRequest: { ...pullRequest, head: { ref: 'feature', sha: nextHead } },
|
||||
startedAt: Date.parse('2026-09-11T04:00:00Z'), updatedAt: Date.parse('2026-09-11T04:01:00Z'), capture: { planned: { base: 3, head: 3 }, done: { base: 3, head: 0 }, baseFromCache: true, current: 'head' }, ...overrides });
|
||||
const next = platformSectionMerge(nextState(), 'ios').merge(both);
|
||||
assert.match(next.split('\n').find(line => line.startsWith('<sub>')), /^<sub>base \[aaaaaaaa\]\(<[^>]+>\) → \[dddddddd\]/);
|
||||
assert.equal(visible(next).split('\n')[1], `> **New run in progress** for [dddddddd](<https://github.com/expo/explore-playground/commit/${nextHead}>) · capturing on iOS · [run](<https://github.com/expo/explore-playground/actions/runs/43/attempts/1>)`);
|
||||
assert.equal(titleOf(next), titleOf(both), 'the previous review stays up');
|
||||
assert.equal(subtitleOf(next), subtitleOf(both));
|
||||
assert.deepEqual(listOf(next), listOf(both));
|
||||
assert.deepEqual(Object.keys(readCommentRecords(next).current), ['ios']);
|
||||
assert.deepEqual(Object.keys(readCommentRecords(next).previous), ['ios', 'android']);
|
||||
assert.deepEqual(summaries(next), ['Previous runs (1)']);
|
||||
assert.match(next, /\n- \[bbbbbbbb\]\(<https:\/\/github\.com\/expo\/explore-playground\/commit\/b{40}>\) · iOS 🟩 3 changes \(\[report\]\(<https:\/\/explore-preview\.expo\.app\/\?repo=[^>]+&artifact=7654>\) · \[run\]\(<[^>]+runs\/42\/attempts\/2>\)\) · Android 🟩 3 changes \(\[report\]\(<[^>]+>\) · \[run\]\(<[^>]+>\)\)\n/);
|
||||
assert.doesNotMatch(next, /\*\*Changes\*\*|screen=settings/);
|
||||
// The previous run's Android report job finishes after the new run started: it is superseded and lands in history.
|
||||
const late = platformSectionMerge(androidReady, 'android').merge(next);
|
||||
assert.deepEqual(Object.keys(readCommentRecords(late).current), ['ios']);
|
||||
assert.equal(readCommentRecords(late).history.length, 1);
|
||||
assert.match(late, /^<sub>base \[aaaaaaaa\]\(<[^>]+>\) → \[dddddddd\]/m);
|
||||
// The new run's Android writer joins the current run, and the previous run stays listed.
|
||||
assert.equal(titleOf(late), titleOf(both));
|
||||
// The new run's Android writer joins; nothing finished yet, so the previous review stays.
|
||||
const joined = platformSectionMerge(nextState({ build: { ...state().build, platform: 'android' } }), 'android').merge(late);
|
||||
assert.deepEqual(Object.keys(readCommentRecords(joined).current), ['ios', 'android']);
|
||||
assert.match(visible(joined).split('\n')[1], /^> \*\*New run in progress\*\* for \[dddddddd\]\(<[^>]+>\) · capturing on iOS, Android · \[run\]/);
|
||||
assert.deepEqual(listOf(joined), listOf(both));
|
||||
assert.deepEqual(summaries(joined), ['Previous runs (1)']);
|
||||
// The new run's iOS finishes: its review takes over; the previous Android record waits for the new one.
|
||||
const iosDone = platformSectionMerge(nextState({ phase: 'ready', review: evidenceReview(), links: nextLinks('43', '7700') }), 'ios').merge(joined);
|
||||
assert.doesNotMatch(iosDone, /New run in progress/);
|
||||
assert.match(titleOf(iosDone), /^### 3 screens changed · \[Review in Explore\]\(<[^>]+artifact=7700>\)$/);
|
||||
assert.match(subtitleOf(iosDone), /^<sub>\[iOS\]\(<[^>]+artifact=7700>\) · Android · \[dddddddd\]/);
|
||||
assert.equal(statusOf(iosDone, 'Android'), 'Android · capturing head 0/3 · base reused · [run](<https://github.com/expo/explore-playground/actions/runs/43/attempts/1>)');
|
||||
assert.deepEqual(Object.keys(readCommentRecords(iosDone).previous), ['android']);
|
||||
const androidDone = platformSectionMerge(nextState({ phase: 'ready', review: evidenceReview(), links: nextLinks('43', '7700'), build: { ...state().build, platform: 'android' } }), 'android').merge(iosDone);
|
||||
assert.equal(readCommentRecords(androidDone).previous, undefined, 'once every target finished nothing previous is kept');
|
||||
assert.doesNotMatch(androidDone, /expo-explore-previous/);
|
||||
assert.equal(statusOf(androidDone, 'Android'), undefined);
|
||||
// Each later run pushes one more entry; history is capped.
|
||||
let body = joined;
|
||||
let body = androidDone;
|
||||
for (let run = 44; run < 52; run += 1) {
|
||||
const sha = run.toString(16).padStart(40, '0');
|
||||
body = platformSectionMerge(state({ links: viewerLinks({ repository: pullRequest.repository, headSha: sha, runId: String(run), runAttempt: '1', configPath: '.expo-explore/config.yaml' }),
|
||||
@@ -414,13 +451,112 @@ test('a newer run moves the current run into Previous runs; a run the PR moved p
|
||||
assert.deepEqual(history.map(entry => entry.runId), ['50', '49', '48', '47', '46']);
|
||||
assert.match(body, /<summary>Previous runs \(5\)<\/summary>/);
|
||||
assert.ok(body.length < PUBLIC_COMMENT_BODY_BUDGET);
|
||||
assert.doesNotMatch(body, /expo-explore-previous|New run in progress/);
|
||||
// A state marked superseded by its own writer goes straight to history even when the comment has nothing newer yet.
|
||||
const superseded = platformSectionMerge(state({ artifactOnly: true, superseded: true }), 'ios').merge(undefined);
|
||||
assert.match(superseded, /The pull request moved while this run worked; its report is kept here until the current run replaces it\./);
|
||||
assert.match(superseded, /^### Explore review\n\nThe pull request moved while this run worked; its report is kept here until the current run replaces it\./m);
|
||||
const replaced = platformSectionMerge(nextState(), 'ios').merge(superseded);
|
||||
assert.equal(titleOf(replaced), '### Explore review · capturing');
|
||||
assert.match(replaced, /Previous runs \(1\)/);
|
||||
});
|
||||
|
||||
test('the review a new run keeps up comes from one run: a target the last run never finished does not bring back an older run', async () => {
|
||||
const { platformSectionMerge, readCommentRecords } = await import('../../dist/report/pr-comment-body.js');
|
||||
const androidOf = overrides => state({ artifactOnly: true, build: { ...state().build, platform: 'android' }, runtime: { ...state().runtime, platform: 'android', deviceModel: 'Pixel 9' }, ...overrides });
|
||||
const run = (runId, head, artifact) => ({ pullRequest: { ...pullRequest, head: { ref: 'feature', sha: head } },
|
||||
links: viewerLinks({ ...(artifact ? { artifactUrl: `https://github.com/expo/explore-playground/actions/runs/${runId}/artifacts/${artifact}` } : {}),
|
||||
repository: pullRequest.repository, headSha: head, runId, runAttempt: '1', configPath: '.expo-explore/config.yaml' }) });
|
||||
const working = { phase: 'capturing', review: undefined, evidence: undefined };
|
||||
const [c, d] = ['c'.repeat(40), 'd'.repeat(40)];
|
||||
let body = platformSectionMerge(state({ artifactOnly: true }), 'ios').merge(undefined);
|
||||
body = platformSectionMerge(androidOf({}), 'android').merge(body);
|
||||
// Run 43 finishes Android only; its iOS job never reports (cancelled).
|
||||
body = platformSectionMerge(state({ ...working, ...run('43', c) }), 'ios').merge(body);
|
||||
body = platformSectionMerge(androidOf({ ...run('43', c, '7800'), review: review([evidenceReview().surfaces[2]]) }), 'android').merge(body);
|
||||
body = platformSectionMerge(state({ ...working, ...run('44', d) }), 'ios').merge(body);
|
||||
const previous = readCommentRecords(body).previous;
|
||||
assert.deepEqual(Object.keys(previous), ['android']);
|
||||
assert.equal(previous.android.runId, '43');
|
||||
assert.match(subtitleOf(body), /^<sub>\[Android\]\(<[^>]+artifact=7800>\) · \[cccccccc\]/);
|
||||
assert.equal(titleOf(body), '### 1 screen changed · [Review in Explore](<https://explore-preview.expo.app/?repo=expo%2Fexplore-playground&run=43&attempt=1&artifact=7800>)');
|
||||
assert.doesNotMatch(visible(body).split('<details>')[0], /artifact=7654/, 'run 42 is only under Previous runs');
|
||||
});
|
||||
|
||||
test('a new run\'s failure does not replace the previous review: it is named on the new-run line until a target is ready', async () => {
|
||||
const { platformSectionMerge, readCommentRecords } = await import('../../dist/report/pr-comment-body.js');
|
||||
const androidOf = overrides => state({ artifactOnly: true, build: { ...state().build, platform: 'android' }, runtime: { ...state().runtime, platform: 'android', deviceModel: 'Pixel 9' }, ...overrides });
|
||||
const both = platformSectionMerge(androidOf({}), 'android').merge(platformSectionMerge(state({ artifactOnly: true }), 'ios').merge(undefined));
|
||||
const nextHead = 'd'.repeat(40);
|
||||
const next = { pullRequest: { ...pullRequest, head: { ref: 'feature', sha: nextHead } },
|
||||
links: viewerLinks({ repository: pullRequest.repository, headSha: nextHead, runId: '43', runAttempt: '1', configPath: '.expo-explore/config.yaml' }) };
|
||||
const commit = `[dddddddd](<https://github.com/expo/explore-playground/commit/${nextHead}>)`;
|
||||
const run43 = '[run](<https://github.com/expo/explore-playground/actions/runs/43/attempts/1>) · [evidence](<https://github.com/expo/explore-playground/actions/runs/43#artifacts>)';
|
||||
const failure = { phase: 'failed', review: undefined, evidence: undefined, failure: { step: 'capture-head', message: 'boom' } };
|
||||
// Android fails before iOS posts anything.
|
||||
const failed = platformSectionMerge(androidOf({ ...next, ...failure }), 'android').merge(both);
|
||||
assert.equal(visible(failed).split('\n')[1], `> **New run failed** for ${commit} · **Android failed** · capture\\-head · boom · ${run43}`);
|
||||
assert.equal(titleOf(failed), titleOf(both));
|
||||
assert.deepEqual(listOf(failed), listOf(both));
|
||||
assert.deepEqual(Object.keys(readCommentRecords(failed).previous), ['ios', 'android']);
|
||||
const working = platformSectionMerge(state({ ...next, phase: 'capturing', review: undefined, evidence: undefined, capture: undefined }), 'ios').merge(failed);
|
||||
assert.equal(visible(working).split('\n')[1], `> **New run in progress** for ${commit} · capturing on iOS · **Android failed** · capture\\-head · boom · ${run43}`);
|
||||
assert.deepEqual(listOf(working), listOf(both));
|
||||
// iOS is ready: its review takes over, and Android's failure is its own line.
|
||||
const ready = platformSectionMerge(state({ ...next, artifactOnly: true }), 'ios').merge(working);
|
||||
assert.doesNotMatch(ready, /New run|expo-explore-previous/);
|
||||
assert.match(statusOf(ready, 'Android'), /^\*\*Android failed\*\* · capture\\-head · boom · /);
|
||||
// A single target that failed keeps the review before it, and the run after it still has that review to keep.
|
||||
const single = platformSectionMerge(state({ ...next, ...failure }), 'ios').merge(platformSectionMerge(state({ artifactOnly: true }), 'ios').merge(undefined));
|
||||
assert.equal(visible(single).split('\n')[1], `> **New run failed** for ${commit} · capture\\-head · boom · ${run43}`);
|
||||
assert.match(titleOf(single), /^### 3 screens changed · /);
|
||||
const later = platformSectionMerge(state({ phase: 'capturing', review: undefined, evidence: undefined, capture: undefined,
|
||||
links: viewerLinks({ repository: pullRequest.repository, headSha: 'e'.repeat(40), runId: '44', runAttempt: '1', configPath: '.expo-explore/config.yaml' }),
|
||||
pullRequest: { ...pullRequest, head: { ref: 'feature', sha: 'e'.repeat(40) } } }), 'ios').merge(single);
|
||||
assert.equal(readCommentRecords(later).previous.ios.runId, '42');
|
||||
assert.match(titleOf(later), /^### 3 screens changed · /);
|
||||
});
|
||||
|
||||
test('a rerun keeps the attempt it replaces up until the new attempt finishes', async () => {
|
||||
const { platformSectionMerge, readCommentRecords } = await import('../../dist/report/pr-comment-body.js');
|
||||
const ready = platformSectionMerge(state({ artifactOnly: true }), 'ios').merge(undefined);
|
||||
const attempt3 = viewerLinks({ repository: pullRequest.repository, headSha, runId: '42', runAttempt: '3', configPath: '.expo-explore/config.yaml' });
|
||||
const rerun = platformSectionMerge(state({ phase: 'capturing', review: undefined, evidence: undefined, links: attempt3 }), 'ios').merge(ready);
|
||||
assert.equal(visible(rerun).split('\n')[1], `> **New run in progress** for [bbbbbbbb](<https://github.com/expo/explore-playground/commit/${headSha}>) · capturing · [run](<https://github.com/expo/explore-playground/actions/runs/42/attempts/3>)`);
|
||||
assert.equal(titleOf(rerun), titleOf(ready));
|
||||
assert.deepEqual(listOf(rerun), listOf(ready));
|
||||
assert.equal(readCommentRecords(rerun).previous.ios.runAttempt, '2');
|
||||
// The same attempt writing again (capture progress, a republish) keeps nothing.
|
||||
assert.equal(readCommentRecords(platformSectionMerge(state({ phase: 'capturing', review: undefined, evidence: undefined }), 'ios').merge(ready)).previous, undefined);
|
||||
const done = platformSectionMerge(state({ artifactOnly: true, links: { ...attempt3, viewer: state().links.viewer } }), 'ios').merge(rerun);
|
||||
assert.equal(readCommentRecords(done).previous, undefined);
|
||||
assert.doesNotMatch(done, /New run in progress/);
|
||||
});
|
||||
|
||||
test('over budget, the previous run\'s records go before the hidden state is cut', async () => {
|
||||
const { readCommentRecords, renderCommentRecords } = await import('../../dist/report/pr-comment-body.js');
|
||||
const previous = platformRecordOf(state({ artifactOnly: true }));
|
||||
const nextHead = 'd'.repeat(40);
|
||||
const current = platformRecordOf(state({ phase: 'capturing', review: undefined, evidence: undefined, pullRequest: { ...pullRequest, head: { ref: 'feature', sha: nextHead } },
|
||||
links: viewerLinks({ repository: pullRequest.repository, headSha: nextHead, runId: '43', runAttempt: '1', configPath: '.expo-explore/config.yaml' }) }));
|
||||
const records = { current: { ios: current }, previous: { ios: previous }, history: [] };
|
||||
const full = renderCommentRecords(records);
|
||||
assert.match(full, /New run in progress/);
|
||||
assert.deepEqual(Object.keys(readCommentRecords(full).previous), ['ios']);
|
||||
const without = renderCommentRecords({ current: { ios: current }, history: [] });
|
||||
assert.equal(renderCommentRecords(records, undefined, without.length + 10), without);
|
||||
// Once the new run shows a finished target, the previous records it still carries are hidden: they go before the list shrinks.
|
||||
const twelve = review(Array.from({ length: 12 }, (_, index) => surface(`screen-${index}`, 'modified', { extra: {
|
||||
review: { status: 'modified', description: `Changed ${index} in a way that takes a sentence to describe for the budget check.` },
|
||||
visual: { verdict: 'modified', reason: 'repeated-cross-build-delta', mismatchPercentage: 1, threshold: 0.1, tool: '[email protected]' } } })));
|
||||
const finished = platformRecordOf(state({ artifactOnly: true, review: twelve, pullRequest: { ...pullRequest, head: { ref: 'feature', sha: nextHead } },
|
||||
links: viewerLinks({ artifactUrl: 'https://github.com/expo/explore-playground/actions/runs/43/artifacts/7700', repository: pullRequest.repository, headSha: nextHead, runId: '43', runAttempt: '1', configPath: '.expo-explore/config.yaml' }) }));
|
||||
const android = { ...current, platform: 'android' };
|
||||
const hidden = { current: { ios: finished, android }, previous: { android: { ...previous, platform: 'android' } }, history: [] };
|
||||
const lean = renderCommentRecords({ ...hidden, previous: {} });
|
||||
assert.equal(renderCommentRecords(hidden, undefined, lean.length + 10), lean);
|
||||
assert.equal(listOf(lean).length, 11, 'ten screens and the link to the rest');
|
||||
});
|
||||
|
||||
test('the deployments line names every hosted preview the comment links right now, current and previous', async () => {
|
||||
const { platformSectionMerge, existingDeploymentMarker } = await import('../../dist/report/pr-comment-body.js');
|
||||
const marker = '<!-- expo-explore-deployments:acme-explore--abc123 -->';
|
||||
@@ -437,24 +573,28 @@ test('the deployments line names every hosted preview the comment links right no
|
||||
const next = state({ links: { ...viewerLinks({ repository: pullRequest.repository, headSha: nextHead, runId: '43', runAttempt: '1', configPath: '.expo-explore/config.yaml' }),
|
||||
preview: previewLinks('https://acme-explore--def456.expo.app/p/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb', Date.now() + 86_400_000) }, pullRequest: { ...pullRequest, head: { ref: 'feature', sha: nextHead } } });
|
||||
assert.equal(platformSectionMerge(next, 'ios').merge(single).split('\n')[1], '<!-- expo-explore-deployments:acme-explore--abc123,acme-explore--def456 -->');
|
||||
// While the new run works the previous run's preview is still the one on show.
|
||||
const working = platformSectionMerge({ ...next, phase: 'capturing', review: undefined, evidence: undefined, links: viewerLinks({ repository: pullRequest.repository, headSha: nextHead, runId: '43', runAttempt: '1', configPath: '.expo-explore/config.yaml' }) }, 'ios').merge(single);
|
||||
assert.equal(working.split('\n')[1], marker);
|
||||
assert.equal(existingDeploymentMarker(`${CI_COMMENT_MARKER}\n<!-- expo-explore-deployments:Bad Host -->\nx`), undefined);
|
||||
});
|
||||
|
||||
test('a report whose hosted preview is still deploying says so instead of linking the central Viewer', () => {
|
||||
const deploying = renderPublicComment(state({ links: { ...state().links, deploying: true }, artifactOnly: true }));
|
||||
assert.match(deploying, /^### Explore review · report deploying, not ready yet$/m, 'the heading says the report is not ready');
|
||||
assert.equal(titleOf(deploying), '### 3 screens changed · report deploying…', 'the heading says the report is not ready');
|
||||
assert.ok(!deploying.includes('explore-preview.expo.app'), 'no download-and-drop Viewer link while the hosted report is on its way');
|
||||
assert.match(deploying, /\| deploying… \|/, 'the Report cell says so');
|
||||
assert.match(deploying, /· Workflow \[run\]\(<[^>]+>\) · \[zip\]\(<[^>]+>\)$/, 'the footer keeps the zip and run');
|
||||
assert.match(subtitleOf(deploying), /^<sub>iOS · \[bbbbbbbb\]/, 'nothing to link the platform to yet');
|
||||
assert.match(deploying, /\n- Added `\/cats\/care-guide`\n/);
|
||||
assert.equal(deploying.split('\n').at(-1), `<sub>${workflowRun} · ${configLink}</sub>`);
|
||||
const deployed = renderPublicComment(state({ links: { ...state().links, deploying: true,
|
||||
preview: previewLinks('https://acme-explore--1lcbdcpwn3.expo.app/p/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb', null) } }));
|
||||
assert.match(deployed, /^### Explore review · \[Open report\]\(<https:\/\/acme-explore--1lcbdcpwn3\.expo\.app[^>]*>\)$/m, 'once deployed the hosted report is the heading');
|
||||
assert.match(titleOf(deployed), /^### 3 screens changed · \[Review in Explore\]\(<https:\/\/acme-explore--1lcbdcpwn3\.expo\.app[^>]*>\)$/, 'once deployed the hosted report is the heading');
|
||||
const record = JSON.parse(JSON.stringify(platformRecordOf(state({ links: { ...state().links, deploying: true }, artifactOnly: true }))));
|
||||
assert.equal(record.links.report, undefined, 'the record carries no report link while deploying');
|
||||
assert.equal(parsePlatformRecord(record).links.deploying, true, 'the deploying flag survives the comment round trip');
|
||||
});
|
||||
|
||||
test('a screen whose screenshots match but whose accessibility tree changed is listed as accessibility only, apart from the visual changes', () => {
|
||||
test('a screen whose screenshots match but whose accessibility tree changed is listed after the visual changes, tagged accessibility only', () => {
|
||||
const accessibilityOnly = (id) => surface(id, 'modified', { extra: {
|
||||
review: { status: 'modified', description: 'Added button “Edit tempo”.', facts: [
|
||||
{ category: 'element', templateId: 'element.added', bullet: 'Added button “Edit tempo”.' },
|
||||
@@ -465,27 +605,28 @@ test('a screen whose screenshots match but whose accessibility tree changed is l
|
||||
} });
|
||||
const withAccessibility = review([...evidenceReview().surfaces, accessibilityOnly('device-picker'), accessibilityOnly('enable-midi')]);
|
||||
const body = renderPublicComment(state({ review: withAccessibility, artifactOnly: true }));
|
||||
assert.match(platformRow(body, 'iOS'), /\| 🟩 3 visual changes · 2 accessibility only · 6m 12s \|/);
|
||||
assert.deepEqual(summaries(body), ['🟩 Added (1)', '🟥 Removed (1)', '🟧 Modified (1)', '🟦 Accessibility only (2)', 'Validation · iOS 🟩 10/10 captured']);
|
||||
assert.match(body, /\n\*\*Changes\*\* · 5 screens\n/);
|
||||
assert.match(body, /<summary>🟦 Accessibility only \(2\)<\/summary>\n\nBefore and After look the same\. What screen readers and voice control read changed\.\n\n- `\/device-picker` — Added button “Edit tempo”\\\. · \[iOS\]/);
|
||||
assert.match(titleOf(body), /^### 5 screens changed · /);
|
||||
assert.deepEqual(listOf(body).slice(3), [
|
||||
`- [\`/device-picker\`](<${reportUrl}&view=compare&screen=device-picker>) · Added button “Edit tempo” · accessibility only`,
|
||||
`- [\`/enable-midi\`](<${reportUrl}&view=compare&screen=enable-midi>) · Added button “Edit tempo” · accessibility only`]);
|
||||
assert.doesNotMatch(body, /Before and After look the same/);
|
||||
const record = parsePlatformRecord(JSON.parse(JSON.stringify(platformRecordOf(state({ review: withAccessibility, artifactOnly: true })))));
|
||||
assert.deepEqual(record.changes.filter(change => change.group === 'accessibility-only').map(change => change.id), ['device-picker', 'enable-midi'],
|
||||
'the group survives the comment round trip');
|
||||
// A comment with no accessibility-only screen keeps its plain count.
|
||||
assert.match(platformRow(renderPublicComment(state({ artifactOnly: true })), 'iOS'), /\| 🟩 3 changes · 6m 12s \|/);
|
||||
});
|
||||
|
||||
test('a comment whose only changes are for assistive technology counts just those', () => {
|
||||
test('a comment whose only change is for assistive technology counts it like any other', () => {
|
||||
const only = review([surface('device-picker', 'modified', { extra: {
|
||||
review: { status: 'modified', description: 'Changed “Play” from a text to a button.', facts: [{ category: 'accessibility', templateId: 'accessibility.role-changed', bullet: 'Changed “Play” from a text to a button.' }] },
|
||||
visual: { verdict: 'unchanged', reason: 'matched', mismatchPercentage: 0, threshold: 0.1, tool: '[email protected]' },
|
||||
sourceChange: sourceChange('Made Play a button.'),
|
||||
} })]);
|
||||
assert.match(platformRow(renderPublicComment(state({ review: only, artifactOnly: true })), 'iOS'), /\| 🟩 1 accessibility only · 6m 12s \|/);
|
||||
const body = renderPublicComment(state({ review: only, artifactOnly: true }));
|
||||
assert.match(titleOf(body), /^### 1 screen changed · /);
|
||||
assert.deepEqual(listOf(body), [`- [\`/device-picker\`](<${reportUrl}&view=compare&screen=device-picker>) · Changed “Play” from a text to a button · accessibility only`]);
|
||||
});
|
||||
|
||||
test('a screen accessibility-only on iOS and visibly modified on Android is listed as Modified, whichever platform comes first', async () => {
|
||||
test('a screen accessibility-only on iOS and visibly modified on Android is listed as the visible change, whichever platform comes first', async () => {
|
||||
const { platformSectionMerge } = await import('../../dist/report/pr-comment-body.js');
|
||||
const settings = (verdict) => {
|
||||
const changed = evidenceReview();
|
||||
@@ -494,25 +635,28 @@ test('a screen accessibility-only on iOS and visibly modified on Android is list
|
||||
};
|
||||
const androidState = overrides => state({ artifactOnly: true, build: { ...state().build, platform: 'android' },
|
||||
runtime: { ...state().runtime, platform: 'android', deviceModel: 'EasAndroidDevice01', osVersion: '15.0' }, ...overrides });
|
||||
const androidLinks = viewerLinks({ artifactUrl: 'https://github.com/expo/explore-playground/actions/runs/42/artifacts/7700', repository: pullRequest.repository, headSha, runId: '42', runAttempt: '2', configPath: '.expo-explore/config.yaml' });
|
||||
const ios = renderPublicComment(state({ review: settings('unchanged'), artifactOnly: true }));
|
||||
assert.ok(summaries(ios).includes('🟦 Accessibility only (1)'), 'iOS alone: the screenshots match');
|
||||
const both = platformSectionMerge(androidState({ review: settings('modified') }), 'android').merge(ios);
|
||||
assert.deepEqual(summaries(both).slice(0, 3), ['🟩 Added (1)', '🟥 Removed (1)', '🟧 Modified (1)']);
|
||||
assert.doesNotMatch(both, /Accessibility only \(|Before and After look the same/);
|
||||
assert.match(both, /\n- `\/settings` — [^\n]+ · \[iOS \(accessibility only\)\]\(<[^>]+>\) · \[Android\]\(<[^>]+>\)\n/);
|
||||
assert.match(ios, /\n- \[`\/settings`\]\(<[^>]+>\) · [^\n]+ · accessibility only\n/, 'iOS alone: the screenshots match');
|
||||
const both = platformSectionMerge(androidState({ review: settings('modified'), links: androidLinks }), 'android').merge(ios);
|
||||
assert.match(both, /\n- \[`\/settings`\]\(<[^>]+artifact=7700&view=compare&screen=settings>\) · Changed button title from “Share” to “Share Example”\n/, 'the link opens Android, where it changed visibly');
|
||||
assert.doesNotMatch(both, /accessibility only/);
|
||||
const androidFirst = platformSectionMerge(state({ review: settings('unchanged'), artifactOnly: true }), 'ios')
|
||||
.merge(platformSectionMerge(androidState({ review: settings('modified'), links: androidLinks }), 'android').merge(undefined));
|
||||
assert.match(androidFirst, /\n- \[`\/settings`\]\(<[^>]+artifact=7700&view=compare&screen=settings>\) · Changed button title from “Share” to “Share Example”\n/);
|
||||
});
|
||||
|
||||
test('screens whose comparison did not run are counted as not compared, never as a green "no change"', () => {
|
||||
const uncompared = review([surface('settings', 'modified'), surface('tempo', 'modified')]);
|
||||
const body = renderPublicComment(state({ review: uncompared, artifactOnly: true }));
|
||||
assert.match(platformRow(body, 'iOS'), /\| 🟨 0 changes · 2 not compared · 6m 12s \|/);
|
||||
assert.match(body, /\nNo compared screen changed on the reviewed platforms\.\n/);
|
||||
assert.equal(titleOf(body), `### No compared screens changed · [Review in Explore](<${reportUrl}>)`);
|
||||
assert.equal(statusOf(body, 'iOS'), `iOS · 2 not compared · ${runLink}`);
|
||||
// A screen that was never captured (outside the capture scope) stays out of the count.
|
||||
const outOfScope = surface('channel', 'modified');
|
||||
for (const key of ['baseImage', 'headImage', 'baseOutcome', 'headOutcome']) delete outOfScope[key];
|
||||
const quiet = renderPublicComment(state({ review: review([outOfScope]), artifactOnly: true, evidence: undefined, capture: { planned: { base: 0, head: 0 }, done: { base: 0, head: 0 }, baseFromCache: false } }));
|
||||
assert.match(platformRow(quiet, 'iOS'), /\| 🟩 0 changes · 6m 12s \|/);
|
||||
assert.match(quiet, /\nNo screen changed on the reviewed platforms\.\n/);
|
||||
assert.equal(titleOf(quiet), `### No screens changed · [Review in Explore](<${reportUrl}>)`);
|
||||
assert.equal(statusOf(quiet, 'iOS'), undefined);
|
||||
});
|
||||
|
||||
const duoTargets = [{ id: 'iphone', platform: 'ios', label: 'iPhone 17' }, { id: 'duo-closed', platform: 'ios', label: 'iPhone Duo (closed)' },
|
||||
@@ -535,7 +679,7 @@ test('a config without targets writes the same records and comment as before tar
|
||||
assert.deepEqual(Object.keys(readCommentRecords(after).current), ['ios', 'android']);
|
||||
});
|
||||
|
||||
test('several targets share a comment: one row each in config order, labelled by target, keyed by id in the records and history', async () => {
|
||||
test('several targets share a comment: named in config order in the subtitle, each linking its report, keyed by id in the records and history', async () => {
|
||||
const { platformSectionMerge, readCommentRecords } = await import('../../dist/report/pr-comment-body.js');
|
||||
const [iphone, closed, open, android] = duoTargets;
|
||||
// Report jobs finish in any order; the comment follows config order.
|
||||
@@ -548,19 +692,19 @@ test('several targets share a comment: one row each in config order, labelled by
|
||||
assert.deepEqual(Object.keys(records).sort(), ['android', 'duo-closed', 'duo-open', 'iphone']);
|
||||
assert.deepEqual([records['duo-open'].platform, records['duo-open'].target, records['duo-open'].label, records['duo-open'].order], ['ios', 'duo-open', 'iPhone Duo (open)', 2]);
|
||||
assert.deepEqual([records.android.platform, records.android.target, records.android.label, records.android.order], ['android', undefined, 'Pixel 7', 3]);
|
||||
const lines = body.split('\n'), header = lines.indexOf('| | Device | Result | Report |');
|
||||
assert.deepEqual(lines.slice(header + 2, lines.indexOf('', header)).map(line => line.split(' | ')[0]),
|
||||
['| iPhone 17', '| iPhone Duo \\(closed\\)', '| iPhone Duo \\(open\\)', '| Pixel 7']);
|
||||
assert.match(body, /^### Explore review · \[iPhone 17 report\]\(<[^>]+>\) · \[iPhone Duo \\\(open\\\) report\]\(<[^>]+>\) · \[Pixel 7 report\]\(<[^>]+>\)$/m);
|
||||
assert.match(body, /\*\*Changes\*\* · 3 screens · a screen is listed once, with a link per device\n/);
|
||||
assert.match(body, /\n- `\/cats\/care-guide` — Added \/cats\/care\\-guide screen · \[iPhone 17\]\(<[^>]+>\) · \[iPhone Duo \\\(open\\\)\]\(<[^>]+>\) · \[Pixel 7\]\(<[^>]+>\)\n/);
|
||||
assert.match(body, /\| Check \| iPhone 17 \| iPhone Duo \\\(closed\\\) \| iPhone Duo \\\(open\\\) \| Pixel 7 \|\n\|---\|---\|---\|---\|---\|/);
|
||||
assert.deepEqual(summaries(body).at(-1), 'Validation · iPhone 17 🟩 6/6 captured · iPhone Duo \\(closed\\) ⏳ in progress · iPhone Duo \\(open\\) 🟩 6/6 captured · Pixel 7 🟩 6/6 captured');
|
||||
// A newer run moves them all into history, still labelled and ordered by target.
|
||||
assert.equal(titleOf(body), `### 3 screens changed · [Review in Explore](<${reportUrl}>)`);
|
||||
assert.equal(subtitleOf(body), `<sub>[iPhone 17](<${reportUrl}>) · iPhone Duo \\(closed\\) · [iPhone Duo \\(open\\)](<${reportUrl}>) · [Pixel 7](<${reportUrl}>) · ${commits} · updated Sep 11, 3:07 AM UTC</sub>`);
|
||||
assert.equal(statusOf(body, 'iPhone Duo \\(closed\\)'), `iPhone Duo \\(closed\\) · capturing head 1/3 · ${runLink}`);
|
||||
assert.equal(listOf(body).length, 3);
|
||||
assert.doesNotMatch(body, / only\n|not on /);
|
||||
// A newer run moves them all into history, still labelled and ordered by target, and keeps their review up while it works.
|
||||
const nextHead = 'd'.repeat(40);
|
||||
const next = platformSectionMerge(targetState(open, { phase: 'capturing', review: undefined, evidence: undefined, pullRequest: { ...pullRequest, head: { ref: 'feature', sha: nextHead } },
|
||||
links: viewerLinks({ repository: pullRequest.repository, headSha: nextHead, runId: '43', runAttempt: '1', configPath: '.expo-explore/config.yaml' }) }), 'duo-open').merge(body);
|
||||
assert.deepEqual(Object.keys(readCommentRecords(next).current), ['duo-open']);
|
||||
assert.deepEqual(Object.keys(readCommentRecords(next).previous).sort(), ['android', 'duo-open', 'iphone'], 'the target still capturing had nothing to keep');
|
||||
assert.match(visible(next).split('\n')[1], /^> \*\*New run in progress\*\* for \[dddddddd\]\(<[^>]+>\) · capturing on iPhone Duo \\\(open\\\) · \[run\]/);
|
||||
assert.equal(subtitleOf(next), `<sub>[iPhone 17](<${reportUrl}>) · [iPhone Duo \\(open\\)](<${reportUrl}>) · [Pixel 7](<${reportUrl}>) · ${commits} · updated Sep 11, 3:07 AM UTC</sub>`);
|
||||
const history = readCommentRecords(next).history;
|
||||
assert.deepEqual(Object.keys(history[0].platforms).sort(), ['android', 'duo-closed', 'duo-open', 'iphone']);
|
||||
assert.match(next, /\n- \[bbbbbbbb\]\(<[^>]+>\) · iPhone 17 🟩 3 changes \(.*?\) · iPhone Duo \\\(closed\\\) ⏳ did not finish \(.*?\) · iPhone Duo \\\(open\\\) 🟩 3 changes \(.*?\) · Pixel 7 🟩 3 changes /);
|
||||
@@ -587,6 +731,8 @@ test('a comment written by a per-platform version keeps parsing, and malformed t
|
||||
assert.deepEqual(parse(line('duo-open', { ...valid, order: 8 })), []);
|
||||
const many = Array.from({ length: 10 }, (_, index) => line(`t${index}`, { ...valid, target: `t${index}` }));
|
||||
assert.equal(parse(...many).length, 8, 'a comment holds at most one record per configurable target');
|
||||
const previous = readCommentRecords([CI_COMMENT_MARKER, line('duo-open', valid), `<!-- expo-explore-previous:iphone:${encodeRecordPayload({ ...valid, target: 'iphone' })} -->`, '<!-- expo-explore-previous:iphone:AAAA -->'].join('\n')).previous;
|
||||
assert.deepEqual(Object.keys(previous), ['iphone'], 'previous records parse the same way');
|
||||
});
|
||||
|
||||
test('with report.comment.map, the map draws the added, removed and visually changed screens, the screens one step before them, and tabs as a group', async () => {
|
||||
@@ -604,13 +750,13 @@ test('with report.comment.map, the map draws the added, removed and visually cha
|
||||
assert.ok(!/<summary>Map/.test(renderPublicComment(state({ artifactOnly: true, review: withNavigation }))));
|
||||
assert.equal(platformRecordOf(state({ review: withNavigation })).navigation, undefined);
|
||||
const body = renderPublicComment(state({ artifactOnly: true, map: true, review: withNavigation }));
|
||||
assert.deepEqual(summaries(body), ['🟩 Added (1)', '🟥 Removed (1)', '🟧 Modified (2)', 'Map (4 screens)', 'Validation · iOS 🟩 8/8 captured']);
|
||||
assert.deepEqual(summaries(body), ['Map (4 screens)']);
|
||||
const map = body.slice(body.indexOf('```mermaid'), body.indexOf('```\n', body.indexOf('```mermaid')) + 3);
|
||||
assert.equal(map, ['```mermaid', 'flowchart LR', ' subgraph tabs["Tabs"]', ' n0["/cats"]', ' n3["/settings"]', ' end',
|
||||
' n1["/cats/care-guide"]', ' n2["/cats/sweet-bee"]', ' n0 -->|push| n1', ' n0 -.->|push| n2',
|
||||
' classDef added stroke:#1a7f37,stroke-width:3px', ' class n1 added', ' classDef modified stroke:#bc4c00,stroke-width:3px', ' class n3 modified',
|
||||
' classDef removed stroke:#cf222e,stroke-width:2px,stroke-dasharray:5 4', ' class n2 removed', '```'].join('\n'));
|
||||
assert.ok(body.indexOf('<summary>🟧 Modified') < body.indexOf('<summary>Map') && body.indexOf('<summary>Map') < body.indexOf('<summary>Validation'));
|
||||
assert.ok(body.indexOf('\n- [`/settings`]') < body.indexOf('<summary>Map'), 'the map follows the list');
|
||||
assert.match(body, /<summary>Map \(4 screens\)<\/summary>\n\n```mermaid\n/, 'a blank line opens the fence inside the collapsible');
|
||||
assert.deepEqual(readCommentRecords(body).current.ios.navigation, platformRecordOf(state({ map: true, review: withNavigation })).navigation, 'the map is carried in the record');
|
||||
|
||||
@@ -624,7 +770,7 @@ test('with report.comment.map, the map draws the added, removed and visually cha
|
||||
assert.ok(!/<summary>Map/.test(renderPublicComment(state({ map: true, review: unchanged }))));
|
||||
});
|
||||
|
||||
test('the map is dropped before the change lists shrink, and a malformed map is dropped from the record', async () => {
|
||||
test('the map is dropped before the list shrinks, and a malformed map is dropped from the record', async () => {
|
||||
const { encodeRecordPayload } = await import('../../dist/report/pr-comment-record.js');
|
||||
const { readCommentRecords, renderCommentRecords } = await import('../../dist/report/pr-comment-body.js');
|
||||
const record = { ...platformRecordOf(state()), navigation: { nodes: [{ route: '/a', group: 'modified' }, { route: '/b' }], edges: [{ from: 1, to: 0, kinds: ['push'] }] } };
|
||||
@@ -632,7 +778,7 @@ test('the map is dropped before the change lists shrink, and a malformed map is
|
||||
assert.match(renderCommentRecords(records), /<summary>Map \(2 screens\)<\/summary>/);
|
||||
const tight = renderCommentRecords(records, undefined, renderCommentRecords(records).length - 1);
|
||||
assert.ok(!tight.includes('```mermaid'));
|
||||
assert.match(tight, /<summary>🟧 Modified \(1\)<\/summary>/);
|
||||
assert.equal(listOf(tight).length, 3, 'the list is intact');
|
||||
const parse = navigation => readCommentRecords(`<!-- expo-explore-record:ios:${encodeRecordPayload({ ...record, navigation })} -->`).current.ios;
|
||||
assert.deepEqual(parse(record.navigation).navigation, record.navigation);
|
||||
for (const navigation of [{ ...record.navigation, edges: [{ from: 0, to: 2, kinds: ['push'] }] }, { ...record.navigation, edges: [{ from: 0, to: 1, kinds: ['teleport'] }] },
|
||||
|
||||
@@ -16,7 +16,8 @@ test('round trip: light + dark on both sides with a fully reused baseline', asyn
|
||||
const inconclusiveDark = report.review.surfaces.filter(surface => surface.id.endsWith('@dark') && surface.review?.status === 'needs-review');
|
||||
assert.deepEqual(inconclusiveDark.map(surface => `${surface.id}: ${surface.review.description}`), [], 'an unchanged dark pair compares conclusively');
|
||||
assert.ok(!progress.some(line => line.startsWith('::warning')), progress.filter(line => line.startsWith('::warning')).join('\n'));
|
||||
assert.match(comment, /\| iOS \| [^|]*light \+ dark[^|]*\| 🟩 /);
|
||||
assert.match(comment, /^### (\d+ screens?|No screens) changed · \[Review in Explore\]/m);
|
||||
assert.doesNotMatch(comment, /^iOS · /m, 'nothing is missing on iOS');
|
||||
assert.equal(records.current.ios?.phase, 'ready');
|
||||
assert.ok(records.current.ios?.capture, 'the final record keeps the capture counts');
|
||||
assert.deepEqual(records.current.ios.capture.done, records.current.ios.capture.planned, 'progress counts surfaces (routes), not light + dark images');
|
||||
@@ -33,7 +34,8 @@ test('round trip: a dark-only base against a light + dark head (the first PR tha
|
||||
const lightOnly = report.review.surfaces.filter(surface => surface.head && !surface.base && !surface.id.endsWith('@dark') && surface.head.route !== '/added');
|
||||
assert.ok(lightOnly.length > 0, 'the light appearance is an added surface, not a modified one');
|
||||
assert.ok(!progress.some(line => line.startsWith('::warning')), progress.filter(line => line.startsWith('::warning')).join('\n'));
|
||||
assert.match(comment, /\| iOS \| [^|]*light \+ dark[^|]*\| 🟩 /);
|
||||
assert.match(comment, /^### (\d+ screens?|No screens) changed · \[Review in Explore\]/m);
|
||||
assert.doesNotMatch(comment, /^iOS · /m, 'nothing is missing on iOS');
|
||||
assert.equal(records.current.ios?.phase, 'ready');
|
||||
});
|
||||
|
||||
@@ -41,14 +43,16 @@ test('round trip: light + dark with a freshly captured base', async t => {
|
||||
const { result, bundles, comment } = await roundTrip(t, { userInterfaceStyle: 'automatic' }, { cachedBase: false });
|
||||
assert.deepEqual(schemesOf(bundles.base), ['light', 'dark']);
|
||||
assert.equal(result.evidence.summary.visual.failed, 0);
|
||||
assert.match(comment, /\| iOS \| [^|]*light \+ dark[^|]*\| 🟩 /);
|
||||
assert.match(comment, /^### (\d+ screens?|No screens) changed · \[Review in Explore\]/m);
|
||||
assert.doesNotMatch(comment, /^iOS · /m, 'nothing is missing on iOS');
|
||||
});
|
||||
|
||||
test('round trip: Android light + dark', async t => {
|
||||
const { result, bundles, comment, records } = await roundTrip(t, { userInterfaceStyle: 'automatic', platform: 'android' });
|
||||
assert.deepEqual(schemesOf(bundles.head), ['light', 'dark']);
|
||||
assert.equal(result.evidence.summary.visual.failed, 0);
|
||||
assert.match(comment, /\| Android \| [^|]*light \+ dark[^|]*\| 🟩 /);
|
||||
assert.match(comment, /^### (\d+ screens?|No screens) changed · \[Review in Explore\]/m);
|
||||
assert.doesNotMatch(comment, /^Android · /m, 'nothing is missing on Android');
|
||||
assert.equal(records.current.android?.phase, 'ready');
|
||||
});
|
||||
|
||||
@@ -57,7 +61,8 @@ test('round trip: a single-scheme app (no userInterfaceStyle) captures light onl
|
||||
assert.deepEqual(bundleColorSchemes(bundles.head), ['light']);
|
||||
assert.ok(!report.review.surfaces.some(surface => /@(light|dark)$/.test(surface.id)), 'no secondary appearance surfaces');
|
||||
assert.equal(report.review.runtime.colorSchemes, undefined);
|
||||
assert.match(comment, /\| iOS \| [^|]*\| 🟩 /);
|
||||
assert.match(comment, /^### (\d+ screens?|No screens) changed · \[Review in Explore\]/m);
|
||||
assert.doesNotMatch(comment, /^iOS · /m);
|
||||
assert.doesNotMatch(comment, /light \+ dark/);
|
||||
assert.equal(records.current.ios.phase, 'ready');
|
||||
});
|
||||
|
||||
@@ -11,7 +11,7 @@ test('round trip: light + dark with an unavailable screen publishes an incomplet
|
||||
assert.ok(unattributed.length >= 1);
|
||||
assert.ok(unattributed.every(surface => surface.headImage && surface.headOutcome.observed?.reason === 'observer-not-ready'));
|
||||
assert.ok(progress.some(line => line.startsWith('::warning title=Explore head capture unavailable')), progress.filter(line => line.startsWith('::warning')).join('\n'));
|
||||
assert.match(comment, /\| iOS \| [^|]*\| 🟨 [^|]*unattributed/);
|
||||
assert.match(comment, /^iOS · [^\n]*unattributed · \[run\]/m);
|
||||
assert.doesNotMatch(comment, /🟥 failed/);
|
||||
assert.equal(records.current.ios?.phase, 'ready');
|
||||
});
|
||||
@@ -33,7 +33,7 @@ test('round trip: a failed screenshot comparison is a named gap, not a failed ru
|
||||
assert.ok(surface, `${id} stays in the report`);
|
||||
assert.notEqual(surface.review?.status, 'modified', 'a failed comparison is not reported as a change');
|
||||
}
|
||||
assert.match(comment, /\| iOS \| [^|]*\| 🟨 [^|]*2 comparisons failed/);
|
||||
assert.match(comment, /^iOS · [^\n]*2 comparisons failed · \[run\]/m);
|
||||
assert.doesNotMatch(comment, /🟥 failed/);
|
||||
assert.equal(records.current.ios?.phase, 'ready');
|
||||
});
|
||||
@@ -44,7 +44,7 @@ test('round trip: a baseline with an unavailable screen is still cached, and the
|
||||
assert.ok(progress.some(line => line.startsWith('baseline: ::warning title=Explore head capture unavailable')));
|
||||
assert.ok(progress.some(line => !line.startsWith('baseline:') && /reused from the baseline, \d+ to capture/.test(line)), progress.filter(line => line.includes('capture')).join('\n'));
|
||||
assert.ok(result.evidence.summary.visual.compared > 0);
|
||||
assert.match(comment, /\| iOS \| [^|]*\| 🟨 /);
|
||||
assert.match(comment, /^iOS · [^\n]*· \[run\]/m, 'the gap is named on the iOS line');
|
||||
assert.equal(records.current.ios?.phase, 'ready');
|
||||
});
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@ test('round trip: a push mid-review publishes the finished run under previous ru
|
||||
assert.equal(records.current.ios.headSha, newHead);
|
||||
assert.equal(records.history.length, 1);
|
||||
assert.equal(records.history[0].headSha, f.headSha);
|
||||
assert.match(comment, new RegExp(`base \\[${f.baseSha.slice(0, 8)}\\][^\\n]*\\[${newHead.slice(0, 8)}\\]`));
|
||||
assert.match(comment, new RegExp(`^<sub>\\[iOS\\]\\(<[^>]+>\\) · \\[${newHead.slice(0, 8)}\\]\\(<[^>]+>\\) vs \\[${f.baseSha.slice(0, 8)}\\]`, 'm'));
|
||||
assert.match(comment, /Previous runs \(1\)/);
|
||||
assert.ok(comment.includes(f.headSha.slice(0, 8)), 'the previous head is named');
|
||||
assert.ok(comment.includes('/actions/runs/1234/'), 'the previous run stays linked');
|
||||
@@ -46,13 +46,12 @@ test('round trip: iOS and Android jobs of one run share a single comment: one ro
|
||||
assert.equal(records.current.ios.phase, 'ready');
|
||||
assert.equal(records.current.android.phase, 'ready');
|
||||
assert.equal(records.current.ios.runId, records.current.android.runId);
|
||||
assert.equal((comment.match(/^\| iOS \|/gm) ?? []).length, 1);
|
||||
assert.equal((comment.match(/^\| Android \|/gm) ?? []).length, 1);
|
||||
assert.equal(comment.match(/^### Explore review/gm).length, 1, 'one header');
|
||||
const routeLines = comment.split('\n').filter(line => /^- `\/(added|removed|home)`/.test(line));
|
||||
assert.equal(comment.match(/^### /gm).length, 1, 'one header');
|
||||
assert.match(comment, /^<sub>\[iOS\]\(<[^>]+>\) · \[Android\]\(<[^>]+>\) · /m, 'the subtitle links each platform\'s report');
|
||||
const routeLines = comment.split('\n').filter(line => /^- (Added |Removed )?\[`\/(added|removed|home)`\]\(</.test(line));
|
||||
assert.ok(routeLines.length >= 1, `the comment lists changed routes once:\n${comment}`);
|
||||
assert.equal(routeLines.length, 6, `each changed screen is listed once:\n${comment}`);
|
||||
for (const line of routeLines) assert.ok(/\[iOS\]\(/.test(line) && /\[Android\]\(/.test(line), `each changed route carries both platforms once: ${line}`);
|
||||
for (const line of routeLines) assert.doesNotMatch(line, / only$|not on /, `a screen changed on both platforms names neither: ${line}`);
|
||||
const projection = publicReviewProjection(android.report.review);
|
||||
for (const surface of android.report.review.surfaces) assert.ok(projection.surfaces[surface.id], `the Viewer projects the Android surface ${surface.id}`);
|
||||
assert.ok(ios.projection.surfaces[ios.report.review.surfaces[0].id], 'the iOS report still opens');
|
||||
@@ -82,5 +81,5 @@ test('round trip: a run on an old head that finishes after the new head\'s run f
|
||||
assert.ok(late.records.history.some(entry => entry.headSha === f.headSha && entry.platforms?.ios?.superseded === true || entry.superseded === true), JSON.stringify(late.records.history));
|
||||
assert.match(late.comment, /Previous runs \(2\)/);
|
||||
assert.ok(late.comment.includes('/actions/runs/1236/'), 'the late report stays reachable');
|
||||
assert.match(late.comment, new RegExp(`base \\[${f.baseSha.slice(0, 8)}\\][^\\n]*\\[${newHead.slice(0, 8)}\\]`));
|
||||
assert.match(late.comment, new RegExp(`^<sub>\\[iOS\\]\\(<[^>]+>\\) · \\[${newHead.slice(0, 8)}\\]\\(<[^>]+>\\) vs \\[${f.baseSha.slice(0, 8)}\\]`, 'm'));
|
||||
});
|
||||
@@ -155,14 +155,14 @@ test("only a live uploaded final artifact becomes Ready with a central Viewer de
|
||||
const posted = calls.find(call => call.options.method === "POST");
|
||||
assert.ok(posted);
|
||||
const body = JSON.parse(posted.options.body).body;
|
||||
assert.match(body, /\| iOS \| [^|]*\| 🟩 /);
|
||||
assert.match(body, /^### (\d+ screens?|No screens) changed · \[Review in Explore\]\(<https:\/\/explore-preview/m);
|
||||
const viewer = new URL(result.artifactUrl);
|
||||
assert.equal(viewer.origin, "https://explore-preview.expo.app");
|
||||
assert.equal(viewer.pathname, "/");
|
||||
assert.deepEqual(Object.fromEntries(viewer.searchParams), { repo: pr.repository, run: "10", attempt: "1", artifact: "20" });
|
||||
assert.ok(body.includes(result.artifactUrl));
|
||||
assert.match(body, new RegExp(`\\| iOS \\| [^|]+\\| 🟩 [^|]+\\| \\[Open\\]\\(<${result.artifactUrl.replace(/[.?+*]/g, "\\$&")}>\\) \\|$`, "m"), "the Report cell is the report link alone");
|
||||
assert.match(body, new RegExp(`· Workflow \\[run\\]\\(<https://github\\.com/expo/explore-playground/actions/runs/10/attempts/1>\\) · \\[zip\\]\\(<${artifactUrl.replace(/[.?+*]/g, "\\$&")}>\\)$`, "m"), "the run and zip links sit in the footer");
|
||||
assert.match(body, new RegExp(`^### (\\d+ screens?|No screens) changed · \\[Review in Explore\\]\\(<${result.artifactUrl.replace(/[.?+*]/g, "\\$&")}>\\)$`, "m"), "the report link is the heading");
|
||||
assert.match(body, /^<sub>\[Workflow run\]\(<https:\/\/github\.com\/expo\/explore-playground\/actions\/runs\/10\/attempts\/1>\) · \[Config\]\(<[^>]+>\)<\/sub>$/m, "the run sits in the footer");
|
||||
assert.doesNotMatch(body, /embedded\.expo-explore\.invalid|<img|convex/);
|
||||
assert.ok(calls.some(call => call.url.pathname.endsWith("/actions/runs/10/attempts/1")));
|
||||
});
|
||||
@@ -172,10 +172,10 @@ test("an uploaded diagnostic artifact with unverified landings remains Incomplet
|
||||
const { transport, calls } = transportFixture();
|
||||
await publishCiReportComment({ inputPath, artifactUrl, environment, transport });
|
||||
const body = JSON.parse(calls.find(call => call.options.method === "POST").options.body).body;
|
||||
assert.match(body, /\| iOS \| [^|]+\| 🟨 [^|]*2 unattributed/);
|
||||
assert.match(body, /\| Route verification \| 🟨 0\/2 verified · 2 unattributed \|/);
|
||||
assert.match(body, /\*\*iOS\*\* · base 0\/1 verified · 1 unattributed · head 0\/1 verified · 1 unattributed/);
|
||||
assert.doesNotMatch(body, /\| iOS \| [^|]+\| 🟩|⏳|Needs review/);
|
||||
assert.match(body, /^iOS · 2 unattributed · \[run\]/m);
|
||||
assert.match(body, /<summary>Capture issues \(2\)<\/summary>/);
|
||||
assert.match(body, /\n- Base · [^\n]+ · CAPTURE\\_LANDING\\_UNVERIFIED/);
|
||||
assert.doesNotMatch(body, /⏳|capturing|Needs review/);
|
||||
});
|
||||
|
||||
test("report publication separates credential, permission, rate limit, and service failures", async t => {
|
||||
@@ -209,7 +209,7 @@ test("stale PRs, expired artifacts, foreign artifacts, and replayed attempts can
|
||||
await publishCiReportComment({ inputPath, artifactUrl, environment, transport: moved.transport });
|
||||
const movedBody = JSON.parse(moved.calls.find(call => call.options.method === "POST").options.body).body;
|
||||
assert.match(movedBody, /Previous runs \(1\)/, "a PR that moved on still gets its report, under previous runs");
|
||||
assert.doesNotMatch(movedBody, /^<sub>base \[/m);
|
||||
assert.match(movedBody, /^### Explore review$/m, "nothing is presented as the current review");
|
||||
for (const configuration of [{ expired: true }, { artifactName: "explore-capture-evidence-1" },
|
||||
{ artifactRun: 11 }, { attempt: 2 }]) {
|
||||
const { transport, calls } = transportFixture(configuration);
|
||||
|
||||
@@ -175,7 +175,7 @@ test("failure comments use actionable capture diagnostics and update only the bo
|
||||
assert.equal(writes.length, 1);
|
||||
assert.equal(writes[0].action, "update");
|
||||
assert.equal(writes[0].comment_id, 3);
|
||||
assert.match(writes[0].body, /\| iOS \| [^|]*\| 🟥 failed · capture\\-head \|/);
|
||||
assert.match(writes[0].body, /^\*\*iOS failed\*\* · capture\\-head · /m);
|
||||
assert.match(writes[0].body, /APP\\_INPUT\\_MISSING/);
|
||||
assert.match(writes[0].body, /Required app input is missing/);
|
||||
assert.match(writes[0].body, /Next action\\: Set APP\\_REVIEW\\_PASSWORD/);
|
||||
@@ -232,7 +232,7 @@ test("closed, draft, and forked PRs receive no failure comment; a PR that moved
|
||||
const [write] = Object.assign([], await reportFailure(t, { capture: sampleDiagnostic(), pr: moved }));
|
||||
assert.match(write.body, /<summary>Previous runs \(1\)<\/summary>/);
|
||||
assert.match(write.body, /bbbbbbbb.*iOS 🟥 failed · capture\\-head/);
|
||||
assert.doesNotMatch(write.body, /^<sub>base \[/m);
|
||||
assert.match(write.body, /^### Explore review$/m);
|
||||
const invalid = await reportFailure(t, { capture: sampleDiagnostic(), head: "not-a-sha" });
|
||||
assert.deepEqual([...invalid], []);
|
||||
});
|
||||
@@ -249,14 +249,14 @@ test("a failed platform joins the other platform's row in a multi-platform comme
|
||||
const comments = [{ id: 7, user: { login: "github-actions[bot]" }, body: existing }];
|
||||
const [write] = await reportFailure(t, { capture: sampleDiagnostic(), platform: "android", comments });
|
||||
assert.equal(write.comment_id, 7);
|
||||
assert.match(write.body, /\| iOS \| [^|]*\| ⏳ capturing head 1\/2 \|/);
|
||||
assert.match(write.body, /\| Android \| [^|]*\| 🟥 failed · capture\\-head \|/);
|
||||
assert.match(write.body, /^iOS · capturing head 1\/2 · \[run\]/m);
|
||||
assert.match(write.body, /^\*\*Android failed\*\* · capture\\-head · /m);
|
||||
assert.match(write.body, /APP\\_INPUT\\_MISSING/);
|
||||
assert.equal(write.body.split("<!-- expo-explore-ci -->").length, 2);
|
||||
// A comment without records (a legacy layout) is replaced by the failure alone.
|
||||
const [whole] = await reportFailure(t, { capture: sampleDiagnostic(), platform: "android", comments: [{ id: 8, user: { login: "github-actions[bot]" }, body: "<!-- expo-explore-ci -->\nPrevious run" }] });
|
||||
assert.match(whole.body, /^<!-- expo-explore-ci -->\n<!-- expo-explore-record:android:/);
|
||||
assert.doesNotMatch(whole.body, /\| iOS \|/);
|
||||
assert.doesNotMatch(whole.body, /iOS/);
|
||||
});
|
||||
|
||||
test("a failed target is named by its artifact and keeps the label and config order its capture record posted", async t => {
|
||||
@@ -276,14 +276,13 @@ test("a failed target is named by its artifact and keeps the label and config or
|
||||
const [write] = await reportFailure(t, { capture: sampleDiagnostic(), platform: "duo-open", comments: [{ id: 7, user: { login: "github-actions[bot]" }, body: existing }] });
|
||||
assert.equal(write.comment_id, 7);
|
||||
assert.match(write.body, /^<!-- expo-explore-record:duo-open:/m);
|
||||
const lines = write.body.split("\n"), header = lines.indexOf("| | Device | Result | Report |");
|
||||
const rows = lines.slice(header + 2, lines.indexOf("", header));
|
||||
assert.deepEqual(rows.map(line => line.split(" | ")[0].slice(2)), ["iPhone 17", "iPhone Duo \\(open\\)", "Pixel 7"], "config order, each target's own label");
|
||||
assert.match(write.body, /\| iPhone Duo \\\(open\\\) \| [^|]*\| 🟥 failed · capture\\-head \|/);
|
||||
const subtitle = write.body.split("\n").find(line => line.startsWith("<sub>"));
|
||||
assert.deepEqual([...subtitle.matchAll(/(?:^<sub>|· )\[?(iPhone 17|iPhone Duo \\\(open\\\)|Pixel 7)/g)].map(match => match[1]), ["iPhone 17", "iPhone Duo \\(open\\)", "Pixel 7"], "config order, each target's own label");
|
||||
assert.match(write.body, /^\*\*iPhone Duo \\\(open\\\) failed\*\* · capture\\-head · /m);
|
||||
// A target the comment has not seen yet is labelled by its id.
|
||||
const [fresh] = await reportFailure(t, { capture: sampleDiagnostic(), platform: "duo-closed" });
|
||||
assert.match(fresh.body, /^<!-- expo-explore-record:duo-closed:/m);
|
||||
assert.match(fresh.body, /\| duo\\-closed \| [^|]*\| 🟥 failed/);
|
||||
assert.match(fresh.body, /^\*\*duo\\-closed failed\*\* · /m);
|
||||
// Artifact directories: any valid id, never a reserved or malformed one.
|
||||
const root = await mkdtemp(join(tmpdir(), "explore-failure-targets-"));
|
||||
t.after(() => rm(root, { recursive: true, force: true }));
|
||||
|
||||
Reference in new issue
Block a user