Compare commits

...
Author SHA1 Message Date
Brent Vatne aa3665cb51 Merge remote-tracking branch 'origin/main' into devin/1792350000-max-sessions 2026-10-18 21:34:53 -07:00
Brent Vatne 2701d9bb0e capture.maxSessions defaults to 5, a typical plan's simulators, and may sit below the number of targets
Two per target asked an account for ten simulators at once for five targets. Unset, the ceiling is now 5 and each
target's share is floor(5 / targets), leftovers to foldables only: one target 5, two 2 each, five 1 each. A ceiling
below the number of targets (2 on a starter plan with an iOS and an Android target) gives each target one session,
and only that many target jobs run at once: config:targets reports maxParallel, the review, compare and baseline
workflows set their capture matrix's max-parallel from it, and a local review limits its targets the same way.
2026-10-18 21:34:48 -07:00
Brent VatneandDevin AI a90ed766df capture.maxSessions: a review-wide session ceiling, spent on separate light/dark sessions when that pays
- capture.maxSessions: a review-wide session ceiling and how a target's job shares it
- runCiMap: appearancePass names one session's share of the appearance passes
- appearancePass keeps its pass slots aligned and skips the health check of a secondary-only run
- A secondary-only appearancePass run is complete when its own @scheme frames are
- Merge a side's appearance-pass exports; after-capture stops per-pass sessions too
- capture:review divides capture.maxSessions over the targets and splits a side's appearance passes across sessions
- Test the export merge; document capture.maxSessions
- capture:baseline shares capture.maxSessions too; one helper runs a side's passes
- capture:baseline takes its target's share of capture.maxSessions
- Session state files stop in head, base order as before
- Appearance passes of one side share a build and repack
- Merged appearance exports recombine outcomes as one run's passes do
- capture.maxSessions is at least one session per target; docs carry the measured split
- Test two iOS targets for the maxSessions floor
- A capture waiting on a shared build starts its EAS session meanwhile
- Docs: the measured split side is 4:26 with the shared repack
- Plan a side's appearance sessions from eas config in a child, not the review's process
- A dark-only session keeps its pass's outcomes for the merge, which keeps open gaps partial
- A capture waiting on a shared build starts its session once the build is repacking
- ci:after-capture and the composite action stop per-appearance sessions; platforms count toward maxSessions
- capture.maxSessions stays out of the capture recipe: how many sessions capture the evidence does not change it
- Test that a split side merges into the export one run makes, each appearance a verified sibling

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-18 21:23:48 -07:00
Brent VatneandDevin AI 69d52661c1 Faster capture: parallel compare with early-abort region search, one Argent bridge thread per capture
- Compare screenshots in parallel and stop region-shift searches early
- Keep one Argent bridge on a worker thread for the whole capture
- Region shifts: stop a whole-pixel fit only past the margin of the fit a tie chooses, so the search finds what the exhaustive one did
- Compare: a pair whose turn comes after the budget is not started, every review in a process shares one limit, and a region-shift worker that fails measures on the calling thread
- Argent bridge thread: start again when the session's environment changes, stop serving when it crashes, and wait on the value it compared
- Argent bridge thread: recordings run per command in the session's directory, and a screenshot is taken only from a file that is a PNG or JPEG

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-18 21:23:48 -07:00
28 changed files with 754 additions and 60 deletions

No files matched your search

+7 -3
View File
@@ -157,6 +157,10 @@ runs:
EXPO_TOKEN: ${{ inputs.expo_token }}
run: |
set -euo pipefail
if [[ ! -e "$OUTPUT/head-session" && ! -L "$OUTPUT/head-session" && ! -e "$OUTPUT/base-session" && ! -L "$OUTPUT/base-session" ]]; then exit 0; fi
node "$TOOLS/scripts/cleanup-ci-sessions.mjs" --eas "$TOOLS/node_modules/.bin/eas" \
--session-file "$OUTPUT/head-session" --session-file "$OUTPUT/base-session"
# A side that captures an appearance in a session of its own records it as <side>-session-<scheme>.
files=()
for name in head-session base-session head-session-light head-session-dark base-session-light base-session-dark; do
if [[ -e "$OUTPUT/$name" || -L "$OUTPUT/$name" ]]; then files+=(--session-file "$OUTPUT/$name"); fi
done
if (( ${#files[@]} == 0 )); then exit 0; fi
node "$TOOLS/scripts/cleanup-ci-sessions.mjs" --eas "$TOOLS/node_modules/.bin/eas" "${files[@]}"
+2
View File
@@ -103,6 +103,8 @@ exploration:
capture:
maxMinutes: 30
maxActions: 500
# maxSessions: 5 # the most EAS Simulator sessions one review runs at once, across all its targets and sides (default: 5, a typical plan's concurrent simulator limit). Each target's job gets floor(maxSessions / targets) sessions, at least one, and sessions left over go to foldable targets (pose): one target gets 5 (head and base at once, light and dark in their own sessions when that pays), two get 2 each, five get 1 each (head, then base). Below the number of targets, only that many target jobs run at once (the workflow's max-parallel, and a local review's targets), one session each. A side spreads its light and dark passes over separate sessions only when that saves it at least a minute, so the ceiling is a limit, not a target: a 3-route target stays on two sessions. euxy's four targets: 8 is two per target, 16 captures each appearance in its own session (measured: an iPhone side 6:12 → 4:26, one repack shared by its sessions, ~3.5 session-minutes more per side).
# On a starter plan (one simulator per platform), set maxSessions to the plan's limit, e.g. 2 for an iOS and an Android target.
repack: eas # eas (hosted workflow job; the default on EAS Simulator) | local (repack-app on the runner; the default with simulator: local)
appearance: auto # auto (follow the app's userInterfaceStyle: automatic captures light and dark) | dark | light | [dark, light]
launch: screenshot # screenshot (relaunch the app per appearance and keep its splash and first settled frame, default) | video (also keep a launch recording in the report) | off
+2
View File
@@ -109,6 +109,8 @@ capture:
# simulator: eas # eas (EAS Simulator, default) | local (a simulator or emulator Explore starts on this machine with Xcode or the Android SDK; --simulator overrides it for one run)
maxMinutes: 30
maxActions: 500
# maxSessions: 5 # the most EAS Simulator sessions one review runs at once, across all its targets and sides (default: 5, a typical plan's concurrent simulator limit). Each target's job gets floor(maxSessions / targets) sessions, at least one, and sessions left over go to foldable targets (pose): one target gets 5 (head and base at once, light and dark in their own sessions when that pays), two get 2 each, five get 1 each (head, then base). Below the number of targets, only that many target jobs run at once (the workflow's max-parallel, and a local review's targets), one session each. A side spreads its light and dark passes over separate sessions only when that saves it at least a minute, so the ceiling is a limit, not a target: a 3-route target stays on two sessions. euxy's four targets: 8 is two per target, 16 captures each appearance in its own session (measured: an iPhone side 6:12 → 4:26, one repack shared by its sessions, ~3.5 session-minutes more per side).
# On a starter plan (one simulator per platform), set maxSessions to the plan's limit, e.g. 2 for an iOS and an Android target.
repack: eas # eas (hosted EAS workflow job, the default on EAS Simulator) | local (repack-app on the runner, the default with simulator: local; --repack overrides it for one run)
appearance: auto # auto (follow the app's userInterfaceStyle: automatic captures light and dark) | dark | light | [dark, light]
launch: screenshot # screenshot (relaunch the app per appearance and keep its splash and first settled frame, default) | video (also keep a launch recording in the report) | off
+2 -1
View File
@@ -10,7 +10,8 @@ const SESSION_ID = /^[A-Za-z0-9][A-Za-z0-9._-]{0,199}$/;
/** Stop only exact sessions whose ownership the capture coordinator already verified. */
export async function cleanupCiSessions({ sessionFiles, easCommand, environment = process.env, runCommand = spawnSync }) {
if (!Array.isArray(sessionFiles) || sessionFiles.length > 3) throw new Error('Provide at most three explicit session handoff files.');
// head, base and baseline, each with a session per appearance pass (light, dark) beside its own.
if (!Array.isArray(sessionFiles) || sessionFiles.length > 9) throw new Error('Provide at most nine explicit session handoff files.');
const temporary = await realpath(environment.RUNNER_TEMP || tmpdir());
const result = { stopped: [], failures: [] };
let project;
+37 -11
View File
@@ -1,4 +1,4 @@
import { appearanceArgs, argentColorSchemes, bundleColorSchemes, mergeSchemeOutcomes, resolveColorSchemes, routeIdOfScreen, type AppearanceSetting, type ColorScheme } from "./appearance.js";
import { appearanceArgs, argentColorSchemes, bundleColorSchemes, mergeSchemeOutcomes, resolveColorSchemes, routeIdOfScreen, schemeScreenId, type AppearanceSetting, type ColorScheme } from "./appearance.js";
import { assertNoCaptureHealthIssues, captureHealthIssues, partitionCaptureHealthIssues } from "./health.js";
import { type RouteResetSetting } from "./route-reset.js";
import { defaultReadinessSeconds } from "./readiness.js";
@@ -73,6 +73,12 @@ export type CiMapOptions = {
buildShare?: string;
/** Which appearances to capture; `auto` (default) follows the app's `userInterfaceStyle`. */
appearance?: AppearanceSetting;
/**
* One session's share of the appearance passes when a side splits them across sessions: `capture` names the passes
* this run makes, out of the `all` it would make alone, so frames keep the ids of a single run (`<id>@dark`) and the
* exports merge. Absent: every resolved appearance.
*/
appearancePass?: { all: readonly ColorScheme[]; capture: readonly ColorScheme[] };
/** Whether to observe the app launch (splash screen) per appearance; `video` also keeps the recording. Defaults to `off`. */
launch?: LaunchSetting;
/** Decodes launch recordings; defaults to ffmpeg when installed. */
@@ -354,7 +360,11 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
let captureFailure: unknown;
const finish = async (): Promise<CiMapResult> => {
if (!(await store.exists())) await store.initialize(map, false);
const incomplete = captureFailure !== undefined || selectedIds.some(id => outcomes[id]?.status !== "captured" || outcomes[id]?.landing !== "verified")
// A session making only secondary passes owns the `<id>@<scheme>` frames of those passes, not the primary ones.
const ownFrameIds = options.appearancePass && !options.appearancePass.capture.includes(colorSchemes[0]!)
? selectedIds.flatMap(id => colorSchemes.filter(scheme => options.appearancePass!.capture.includes(scheme)).map(scheme => schemeScreenId(id, scheme)))
: selectedIds;
const incomplete = captureFailure !== undefined || ownFrameIds.some(id => outcomes[id]?.status !== "captured" || outcomes[id]?.landing !== "verified")
|| map.gaps.some(gap => gap.status === "open");
map.run.status = incomplete || map.screens.length === 0 ? "partial" : "complete";
if (routes.length === 0) map.run.stopReason = "No source targets require capture.";
@@ -423,7 +433,9 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
if (options.agent) assertAgentAcceptance(bundle, map, { required: true });
// The side that produces the evidence checks it the way the review will: an integrity defect fails here, naming
// its codes, instead of on the consuming side after both captures. Unavailable screens are not defects.
assertNoCaptureHealthIssues(partitionCaptureHealthIssues(captureHealthIssues(bundle, selectedIds)).integrity);
// A session capturing only a secondary appearance has no primary frames by design; the merged export is checked instead.
const partialPass = options.appearancePass !== undefined && !options.appearancePass.capture.includes(colorSchemes[0]!);
if (!partialPass) assertNoCaptureHealthIssues(partitionCaptureHealthIssues(captureHealthIssues(bundle, selectedIds)).integrity);
await writeFile(join(outputDirectory, "bundle.json"), `${JSON.stringify(bundle, null, 2)}\n`, { flag: "wx" });
if (metadata) await writeFile(join(outputDirectory, "execution.json"), `${JSON.stringify(metadata, null, 2)}\n`, { flag: "wx" });
assertCheckout();
@@ -565,7 +577,8 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
const appDirectory = !options.buildShare ? await realpath(await mkdtemp(join(tmpdir(), "expo-explore-app-"))) : undefined;
let localApp: string | undefined;
type BuiltApp = { fingerprint: string; nativeEasBuildId: string | null; easBuildId: string | null; app?: string; reused: boolean; version?: string; buildNumber?: string };
const buildApp = async (keepApp: string): Promise<BuiltApp> => {
// `repacking` tells the captures sharing this build that their sessions can start too (sharedBuild).
const buildApp = async (keepApp: string, repacking?: () => Promise<void>): Promise<BuiltApp> => {
options.signal?.throwIfAborted();
say("computing the native fingerprint");
const fingerprintResult = eas(["fingerprint:generate", "--platform", options.platform, "--build-profile", options.easProfile, "--json", "--non-interactive"]);
@@ -575,7 +588,7 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
"--fingerprint-hash", sourceFingerprint, ...(options.platform === "ios" ? ["--simulator"] : []), "--limit", "1", "--json", "--non-interactive"]).stdout);
if (reusable) {
say(`reusing EAS build ${reusable.id} (fingerprint ${sourceFingerprint.slice(0, 12)}); repacking it with this commit's JavaScript and the route observer`);
startSession();
startSession(); await repacking?.();
const repacked = await prepareExactSourceBuild({ platform: options.platform, runner, easCommand: options.easCommand, repackCommand: options.repackCommand,
project, nativeBuildId: reusable.id, fingerprint: sourceFingerprint, repackProfile: profile, easEnvironment, repackEnvironment: restrictedEnvironment, routes: inventory, repack: options.repack, easProfile: options.easProfile, keepApp });
if (!repacked) throw new CiError({ code: "CAPTURE_OBSERVER_BUILD_UNSUPPORTED", stage: "instrumentation",
@@ -589,7 +602,7 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
: `no finished simulator build has fingerprint ${sourceFingerprint.slice(0, 12)}; building on EAS and waiting for it to finish`);
const built = nativeBuildFrom(eas(["build", "--platform", options.platform, "--profile", options.easProfile, "--wait", "--json", "--non-interactive"], 45 * 60_000).stdout);
say(`EAS build ${built.id} finished; repacking it with the route observer`);
startSession();
startSession(); await repacking?.();
const repacked = await prepareExactSourceBuild({ platform: options.platform, runner, easCommand: options.easCommand, repackCommand: options.repackCommand,
project, nativeBuildId: built.id, fingerprint: built.fingerprint, repackProfile: profile, easEnvironment, repackEnvironment: restrictedEnvironment, routes: inventory, repack: options.repack, easProfile: options.easProfile, keepApp });
if (!repacked) throw new CiError({ code: "CAPTURE_OBSERVER_BUILD_UNSUPPORTED", stage: "instrumentation",
@@ -605,9 +618,11 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
? await sharedBuild({ directory: options.buildShare, appName, ...(options.signal ? { signal: options.signal } : {}), build: buildApp, owner: options.outputDirectory,
key: { platform: options.platform, easProfile: options.easProfile, bundleId: options.bundleId, gitCommit: options.gitCommit, repack: options.repack ?? "eas", local,
routes: inventory.map(route => route.id) },
waiting: () => say("using the build another target of this review is preparing for this commit") })
waiting: () => say("using the build another target of this review is preparing for this commit"),
// The session is named before its app exists, so a waiter's session boots while the other capture repacks.
repacking: () => startSession() })
: await buildApp(join(appDirectory!, appName));
// A capture that used another target's build starts its session now.
// A capture that used another capture's build has its session started above; this starts the rest.
startSession();
fingerprint = built.fingerprint; nativeEasBuildId = built.nativeEasBuildId; easBuildId = built.easBuildId; localApp = built.app; reusedEasBuild = built.reused;
if (!app.version && built.version) app.version = built.version;
@@ -921,7 +936,10 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
// Every pass sets the appearance explicitly, so a capture does not inherit whatever the simulator last used.
const passes: CaptureBundle["outcomes"][] = [];
const recordingTools = launchSetting === "off" ? undefined : options.recordingTools === undefined ? ffmpegRecordingTools() : options.recordingTools ?? undefined;
const capturedSchemes = options.appearancePass ? colorSchemes.filter(scheme => options.appearancePass!.capture.includes(scheme)) : colorSchemes;
for (const [index, colorScheme] of colorSchemes.entries()) {
// A pass another session makes leaves its slot empty, so the passes stay aligned with the appearances they merge by.
if (!capturedSchemes.includes(colorScheme)) { passes.push({}); continue; }
// Argent cannot switch the appearance yet (software-mansion/argent#1509): the host does, simctl or adb on this
// machine and serve-sim on an EAS iOS session; elsewhere its single pass runs as the device starts.
if (controller !== "argent") runner({ command: controllerCommand, args: [...appearanceArgs(colorScheme), "--session", CONTROLLER_SESSION, ...deviceArgs(runtime), "--platform", options.platform],
@@ -956,7 +974,12 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
...(colorSchemes.length > 1 ? { appearance: { scheme: colorScheme, primary: index === 0 } } : {}) }));
}
runtime.colorScheme = colorSchemes[0]!;
Object.assign(outcomes, mergeSchemeOutcomes(passes, colorSchemes));
// A session without the primary pass keeps each outcome as its pass recorded it, under `<id>@<scheme>`: the export
// merge recombines them with the primary pass's (mergeSchemeOutcomes), which only then can tell an unattributed
// frame beside a primary one from a lost route.
if (options.appearancePass && !capturedSchemes.includes(colorSchemes[0]!)) {
for (const [index, pass] of passes.entries()) for (const [id, outcome] of Object.entries(pass)) outcomes[schemeScreenId(id, colorSchemes[index])] = outcome;
} else Object.assign(outcomes, mergeSchemeOutcomes(passes, colorSchemes));
} catch (error) { primaryError = toCiError(error, { code: `CAPTURE_${runtimeStage.toUpperCase()}_FAILED`, stage: runtimeStage,
nextAction: "Inspect the capture diagnostic, correct the failing prerequisite, and rerun Explore." }); }
finally {
@@ -1025,7 +1048,10 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
throw primaryError;
}
setStage("export");
const captured = selectedIds.filter(id => outcomes[id]?.status === "captured").length;
say(`${captured} of ${selectedIds.length} ${selectedIds.length === 1 ? "route" : "routes"} captured`);
const ownSchemes = options.appearancePass && !options.appearancePass.capture.includes(colorSchemes[0]!)
? colorSchemes.filter(scheme => options.appearancePass!.capture.includes(scheme)) : undefined;
const captured = selectedIds.filter(id => ownSchemes
? ownSchemes.every(scheme => outcomes[schemeScreenId(id, scheme)]?.status === "captured") : outcomes[id]?.status === "captured").length;
say(`${captured} of ${selectedIds.length} ${selectedIds.length === 1 ? "route" : "routes"} captured${ownSchemes ? ` in ${ownSchemes.join(", ")}` : ""}`);
return finish();
}
+8 -4
View File
@@ -19,21 +19,23 @@ const POLL_MS = 2_000;
/**
* Runs `build` once per key across every capture sharing `directory`; the app it leaves goes in the build's folder
* (`keepApp`), which outlives each capture. A waiter is told so it can say whose build it uses.
* (`keepApp`), which outlives each capture. A waiter is told so it can say whose build it uses, and told again
* (`repacking`) once the builder calls its `repacking` (the native build is known and is being repacked), or the
* build is done: a waiter starts its session then, not while a native build may run for most of an hour.
*/
export async function sharedBuild<T extends object>(options: { directory: string; key: SharedBuildKey; appName: string; signal?: AbortSignal;
build: (keepApp: string) => Promise<T>; waiting?: () => void;
build: (keepApp: string, repacking: () => Promise<void>) => Promise<T>; waiting?: () => void; repacking?: () => void;
/** Who builds it (the capture's output directory), so a review can fail its waiters if that capture dies (abandonSharedBuilds). */
owner?: string }): Promise<T> {
const folder = join(options.directory, sharedBuildId(options.key));
const result = join(folder, "result.json"), failure = join(folder, "failure.json");
const result = join(folder, "result.json"), failure = join(folder, "failure.json"), repack = join(folder, "repacking");
await mkdir(options.directory, { recursive: true });
let owner = true;
try { await mkdir(folder); } catch (error) { if ((error as NodeJS.ErrnoException).code !== "EEXIST") throw error; owner = false; }
if (owner) {
if (options.owner) await writeFile(join(folder, "owner"), options.owner);
try {
const built = await options.build(join(folder, options.appName));
const built = await options.build(join(folder, options.appName), () => writeFile(repack, "").catch(() => undefined));
await writeFile(`${result}.tmp`, JSON.stringify(built));
await rename(`${result}.tmp`, result);
return built;
@@ -44,9 +46,11 @@ export async function sharedBuild<T extends object>(options: { directory: string
}
}
options.waiting?.();
let repackSeen = false;
for (;;) {
options.signal?.throwIfAborted();
const done = await readFile(result, "utf8").catch(() => undefined);
if (!repackSeen && (done !== undefined || existsSync(repack))) { repackSeen = true; options.repacking?.(); }
if (done !== undefined) return JSON.parse(done) as T;
const failed = await readFile(failure, "utf8").catch(() => undefined);
if (failed !== undefined) {
+4 -1
View File
@@ -28,6 +28,9 @@ const recipeKey = ({ inventory, profile, config, toolsCommit }: Recipe) => {
/** Only settings that can change captured runtime evidence belong in the recipe. */
export function captureRecipeConfig(config: ExploreConfig, instructionsSha256?: string): string {
// How many sessions capture the evidence (capture.maxSessions) does not change it.
const { maxSessions: _maxSessions, ...capture } = config.capture;
void _maxSessions;
return canonicalJson({
observer: CAPTURE_OBSERVER_HASH,
mode: config.mode,
@@ -36,7 +39,7 @@ export function captureRecipeConfig(config: ExploreConfig, instructionsSha256?:
...(config.app.scheme === undefined ? {} : { scheme: config.app.scheme }) },
device: config.device, parameters: config.parameters, setup: config.setup ?? [],
// The repack as it resolves, so a config that leaves it to the default keeps the recipe it had when the default was written out.
capture: { ...config.capture, repack: captureRepack(config) },
capture: { ...capture, repack: captureRepack(config) },
// agent-device recipes stay as they were; an Argent capture is keyed to the Argent release that drove it.
...(config.capture.controller === "argent" ? { controllerVersion: ARGENT_VERSION } : {}),
});
+83
View File
@@ -0,0 +1,83 @@
import { copyFile, mkdir, readdir, readFile, writeFile, appendFile, lstat } from "node:fs/promises";
import { dirname, join, relative } from "node:path";
import { bundleColorSchemes, mergeSchemeOutcomes, routeIdOfScreen, type ColorScheme } from "../capture/appearance.js";
import type { CaptureOutcome } from "../contracts/capture.js";
import type { CaptureBundle } from "../contracts/capture.js";
import type { ExploreMap } from "../types.js";
const LIST_FIELDS = ["artifacts", "screens", "variants", "observations", "elements", "actionAttempts", "transitions", "journeys", "frontier", "gaps"] as const;
async function readJson<T>(path: string): Promise<T> { return JSON.parse(await readFile(path, "utf8")) as T; }
async function copyTree(source: string, destination: string, skip: (relativePath: string) => boolean, root = source): Promise<void> {
for (const entry of await readdir(source, { withFileTypes: true })) {
const from = join(source, entry.name), to = join(destination, entry.name), rel = relative(root, from);
if (skip(rel)) continue;
if (entry.isDirectory()) { await mkdir(to, { recursive: true }); await copyTree(from, to, skip, root); continue; }
if (!entry.isFile()) continue;
if ((await lstat(to).catch(() => undefined))) continue;
await mkdir(dirname(to), { recursive: true });
await copyFile(from, to);
}
}
/**
* Fold the exports of a side's secondary appearance sessions into its primary export, in place: the frames, outcomes,
* launch evidence and map records of each `<id>@<scheme>` screen, and their files. Each session captured its passes as
* one run would (`appearancePass`), so records only append; a secondary export never replaces a primary record.
*/
export async function mergeCaptureExports(primary: string, secondaries: readonly { directory: string; scheme: ColorScheme }[]): Promise<void> {
if (secondaries.length === 0) return;
const bundle = await readJson<CaptureBundle>(join(primary, "bundle.json"));
const map = await readJson<ExploreMap>(join(primary, "screen-map.json"));
const known = { screens: new Set(map.screens.map(screen => screen.id)), artifacts: new Set(map.artifacts.map(artifact => artifact.id)) };
// The outcomes recombine as one run's passes do (mergeSchemeOutcomes): a secondary pass records a verified frame under
// `<id>@<scheme>` and a lost route under `<id>`, which the merge folds into the primary record the same way.
const schemes = bundleColorSchemes(bundle);
const passes: Record<string, CaptureOutcome>[] = schemes.map(() => ({}));
for (const [id, outcome] of Object.entries(bundle.outcomes)) if (!routeIdOfScreen(id).scheme) passes[0]![id] = outcome;
for (const { directory: secondary, scheme } of secondaries) {
const other = await readJson<CaptureBundle>(join(secondary, "bundle.json"));
const pass = passes[schemes.indexOf(scheme)];
// A session that could not show its appearance (Argent without appearance.set captures one) has nothing to add, as
// one run would have no pass for it.
if (!pass && !bundleColorSchemes(other).includes(scheme)) continue;
if (!pass) throw new Error(`The ${scheme} session's export is not an appearance of the primary export (${schemes.join(", ")})`);
for (const [id, outcome] of Object.entries(other.outcomes)) pass[routeIdOfScreen(id).routeId] = outcome;
const otherMap = await readJson<ExploreMap>(join(secondary, "screen-map.json"));
for (const [id, image] of Object.entries(other.images)) if (routeIdOfScreen(id).scheme && !bundle.images[id]) bundle.images[id] = image;
for (const [scheme, record] of Object.entries(other.launch ?? {})) { bundle.launch ??= {}; bundle.launch[scheme] ??= record; }
for (const [id, context] of Object.entries(other.captureContexts ?? {})) { bundle.captureContexts ??= {}; bundle.captureContexts[id] ??= context; }
for (const field of LIST_FIELDS) {
const records = (otherMap[field] ?? []) as Array<Record<string, unknown>>;
const target = ((map as unknown as Record<string, unknown[]>)[field] ??= []);
for (const record of records) {
if (field === "screens" && known.screens.has(record.id as string)) continue;
if (field === "artifacts" && known.artifacts.has(record.id as string)) continue;
if (field === "screens") known.screens.add(record.id as string);
if (field === "artifacts") known.artifacts.add(record.id as string);
target.push(record);
}
}
// Files: artifacts and launch evidence are new paths; diagnostics append to the primary's lines.
await copyTree(secondary, primary, rel => rel === "bundle.json" || rel === "screen-map.json" || rel === "screen-map.schema.json" || rel === "manifest.json"
|| rel === "execution.json" || rel.startsWith("capture-diagnostics"));
const diagnostics = join(secondary, "capture-diagnostics");
for (const entry of await readdir(diagnostics, { withFileTypes: true }).catch(() => [])) {
if (!entry.isFile()) continue;
const to = join(primary, "capture-diagnostics", entry.name);
await mkdir(dirname(to), { recursive: true });
if (entry.name.endsWith(".jsonl")) await appendFile(to, await readFile(join(diagnostics, entry.name)));
else if (!(await lstat(to).catch(() => undefined))) await copyFile(join(diagnostics, entry.name), to);
}
}
// An open gap a secondary pass left makes the run partial, as it does one run's (a complete run has none).
if (map.gaps.some(gap => gap.status === "open")) map.run.status = "partial";
const manifest = await readJson<{ files: unknown[] }>(join(primary, "manifest.json"));
manifest.files = map.artifacts.map(artifact => ({ path: artifact.path, sha256: artifact.sha256, bytes: artifact.bytes, mimeType: artifact.mimeType }));
await writeFile(join(primary, "screen-map.json"), JSON.stringify(map));
bundle.outcomes = mergeSchemeOutcomes(passes, schemes);
await writeFile(join(primary, "bundle.json"), JSON.stringify(bundle));
await writeFile(join(primary, "manifest.json"), `${JSON.stringify(manifest, null, 2)}\n`);
}
+16 -7
View File
@@ -1,14 +1,14 @@
import type { RunTarget } from "../types.js";
import { spawnSync } from "node:child_process";
import { randomBytes } from "node:crypto";
import { readFileSync } from "node:fs";
import { readFileSync, readdirSync } from "node:fs";
import { copyFile, cp, lstat, mkdir, mkdtemp, readdir, readFile, realpath, rename, rm, stat, utimes, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join, relative, resolve } from "node:path";
import { runCiReview } from "./pipeline.js";
import { resolveCommit } from "./workflow-steps.js";
import { ciLayout, installCommand, type PackageManager } from "./workflow-install.js";
import { configPlatforms, readExploreConfig, resolveTargetConfig, configTargets, withRunOverrides, DEFAULT_EXPLORE_CONFIG_PATH, type ExploreConfig, type ExplorePlatform } from "../config/explore-config.js";
import { configPlatforms, readExploreConfig, resolveTargetConfig, configTargets, posedTargets, withRunOverrides, DEFAULT_EXPLORE_CONFIG_PATH, type ExploreConfig, type ExplorePlatform } from "../config/explore-config.js";
import { AGENT_PROVIDERS, defaultExploreConfig } from "../config/setup.js";
import type { PrepareCiExplorationOptions } from "../capture/agent/exploration.js";
import { createCiAgentSession } from "../capture/agent/session.js";
@@ -26,6 +26,7 @@ import { abandonSharedBuilds } from "../capture/shared-build.js";
import { exploreCacheRoot } from "../shared/cache-root.js";
import { packageBin } from "../shared/package-paths.js";
import { silentProgress, type Progress } from "../shared/progress.js";
import { maxParallelTargets } from "./session-plan.js";
/**
* `capture:review --base REF [--head REF]` on a laptop: the same review CI runs (`runCiReview`), on two commits of the
@@ -44,7 +45,7 @@ export type LocalReviewOptions = {
platform?: ExplorePlatform | undefined;
/** `--target`: capture this config.yaml target alone; with `platform`, the targets of that platform. */
target?: string | undefined;
/** `--parallel`: how many targets capture at once (each runs a simulator or emulator per side); all of them by default. */
/** `--parallel`: how many targets capture at once (each runs a simulator or emulator per side); by default all of them, up to capture.maxSessions. */
parallel?: number | undefined;
scope?: string | undefined;
expiresIn?: string | undefined;
@@ -285,8 +286,15 @@ export async function keepCapture(exported: string, target: string): Promise<boo
}
}
/** The session state files a review writes: `<side>-session`, and `<side>-session-<scheme>` for appearance passes in their own sessions. */
function sessionStateFiles(output: string): string[] {
const rank = (name: string) => ["head", "base", "baseline"].indexOf(name.split("-")[0]!);
try { return readdirSync(output).filter(name => /^(head|base|baseline)-session(-[a-z]+)?$/.test(name)).sort((a, b) => rank(a) - rank(b) || a.localeCompare(b)); }
catch { return []; }
}
function recordedSessionIds(outputs: string[]): string[] {
return outputs.flatMap(output => ["head-session", "base-session"].flatMap(name => {
return outputs.flatMap(output => sessionStateFiles(output).flatMap(name => {
try { const id = readFileSync(join(output, name), "utf8").trim(); return /^[A-Za-z0-9][A-Za-z0-9._-]{0,199}$/.test(id) ? [id] : []; } catch { return []; }
}));
}
@@ -307,7 +315,7 @@ export async function stopRecordedSessions(outputs: string[], environment: NodeJ
simctl: (args: string[]) => { status: number | null } = args => spawnSync("xcrun", ["simctl", ...args], { env: environment, encoding: "utf8", timeout: 120_000 }),
adb: (args: string[]) => { status: number | null } = args => spawnSync(androidTools(environment).adb, args, { env: environment, encoding: "utf8", timeout: 60_000 })): Promise<string[]> {
const stopped: string[] = [];
for (const output of outputs) for (const name of ["head-session", "base-session"]) {
for (const output of outputs) for (const name of sessionStateFiles(output)) {
const file = join(output, name);
const id = (await readFile(file, "utf8").catch(() => "")).trim();
if (!/^[A-Za-z0-9][A-Za-z0-9._-]{0,199}$/.test(id)) continue;
@@ -479,7 +487,7 @@ export async function runLocalReview(options: LocalReviewOptions): Promise<Local
target, ...(targets.length > 1 ? { targets } : {}) };
progress(`${target.id}: reviewing${target.id === platform ? "" : ` (${target.label})`}`);
const result = await runCiReview({ projects, output, config, presentation, runId, runAttempt: "1", state, environment: platformEnvironment,
...(platforms.length > 1 ? { platforms } : {}), ...(targets.length > 1 ? { targets } : {}),
...(platforms.length > 1 ? { platforms } : {}), ...(targets.length > 1 ? { targets } : {}), posedTargets: posedTargets(rawConfig),
// Targets of one platform capture the same commits from the same build; it is built and repacked once.
...(targets.length > 1 ? { buildShare: join(work, ".builds") } : {}),
subscription, useLocalEasSession, progress: message => progress(`${target.id}: ${message}`),
@@ -537,7 +545,8 @@ export async function runLocalReview(options: LocalReviewOptions): Promise<Local
await abandonSharedBuilds(join(work, ".builds"), owner => owner === join(work, target.id, `${side}-export`), failure.code);
}
};
const settled = await settleLimited(targets, options.parallel ?? targets.length,
// Each target needs a simulator session: no more run at once than capture.maxSessions, unless --parallel says.
const settled = await settleLimited(targets, options.parallel ?? maxParallelTargets(rawConfig.capture.maxSessions, targets.length),
target => captureTarget(target).catch(async (error: unknown) => { await releaseBuilds(target, error).catch(() => undefined); throw error; }));
// One target that fails costs that target's report, not the others': they are kept, and the failure is named.
// Only an interrupted review, or one where every target failed, fails as a whole.
+125 -10
View File
@@ -1,13 +1,18 @@
import { systemRegions } from "../contracts/capture.js";
import type { RunTarget } from "../types.js";
import { spawnSync } from "node:child_process";
import { mkdir, open, readFile, realpath, writeFile } from "node:fs/promises";
import { execFile, spawnSync } from "node:child_process";
import { mkdir, mkdtemp, open, readFile, realpath, rm, writeFile } from "node:fs/promises";
import { join, relative, resolve, isAbsolute } from "node:path";
import { tmpdir } from "node:os";
import { setTimeout as delay } from "node:timers/promises";
import { promisify } from "node:util";
import { buildVerifiedPlan } from "./verified-plan.js";
import { runCiMap, type CommandRunner } from "../capture/run.js";
import { routeIdOfScreen } from "../capture/appearance.js";
import { runCiMap, type CiMapOptions, type CommandRunner } from "../capture/run.js";
import { resolveColorSchemes, routeIdOfScreen, type ColorScheme } from "../capture/appearance.js";
import { baseEnvironment } from "../capture/common.js";
import { DEFAULT_MAX_SESSIONS, planSideSessions, sessionShare } from "./session-plan.js";
import { mergeCaptureExports } from "./export-merge.js";
import { abandonSharedBuilds } from "../capture/shared-build.js";
import { prepareCaptureReview, readCaptureBundle } from "../review/capture-review.js";
import { quarantineReviewContract } from "../contracts/review-validation.js";
import { agentCaptureConstraints } from "../capture/agent/source-context.js";
@@ -64,6 +69,8 @@ async function recipe(config: ExploreConfig, plan: SourceCapturePlan, side: "bas
/** The capture's clock and host-side pause; tests pair them with an injected runner so readiness budgets are not real time. */
type CaptureClock = { now?: (() => number) | undefined; sleep?: ((milliseconds: number) => void) | undefined };
const execFileAsync = promisify(execFile);
function captureOptions(config: ExploreConfig, environment: NodeJS.ProcessEnv, runner?: CommandRunner, clock: CaptureClock = {}) {
return { ...(runner && clock.now ? { now: clock.now } : {}), ...(runner && clock.sleep ? { sleep: clock.sleep } : {}), platform: config.app.platform, bundleId: config.app.bundleId,
...(config.app.scheme ? { scheme: config.app.scheme } : {}), device: config.device.name, ...(config.device.pose ? { devicePose: config.device.pose } : {}),
@@ -85,6 +92,87 @@ async function prefetchControllers(config: ExploreConfig, environment: NodeJS.Pr
if (config.capture.controller === "argent" && config.capture.simulator === "local") await ensureArgentToolServer({ environment, progress });
}
/**
* The appearances a capture of this side renders, read before any session starts so the sessions can divide them. The
* app config comes from `eas config` in a child process, as the capture reads it, with the capture's restricted
* environment: never evaluated in this process, which holds the review's secrets. Unknown: undefined.
*/
async function expectedColorSchemes(config: ExploreConfig, project: string, environment: NodeJS.ProcessEnv, easCommand: string): Promise<ColorScheme[] | undefined> {
// Argent switches the appearance only through serve-sim on an EAS iOS session; elsewhere it captures one.
if (config.capture.controller === "argent" && config.capture.simulator !== "local" && config.app.platform !== "ios") return undefined;
if (config.capture.appearance !== "auto") return resolveColorSchemes(config.capture.appearance, undefined, config.app.platform);
const easEnvironment = { ...baseEnvironment(environment), ...(environment.EXPO_TOKEN ? { EXPO_TOKEN: environment.EXPO_TOKEN } : {}) };
try {
const { stdout } = await execFileAsync(easCommand, ["config", "--platform", config.app.platform, "--profile", config.app.easProfile, "--json", "--non-interactive"],
{ cwd: project, env: easEnvironment, timeout: 5 * 60_000, maxBuffer: 16 * 1024 * 1024 });
const app = (JSON.parse(stdout) as { appConfig?: Record<string, unknown> }).appConfig;
return app && typeof app === "object" ? resolveColorSchemes("auto", app, config.app.platform) : undefined;
} catch {
// The capture reads the same config and reports why it cannot; until then the side keeps one session.
return undefined;
}
}
/**
* Run one side's capture: in one session, or when `passes` splits its appearances, one worker session per pass. The
* primary pass exports to `mapOptions.outputDirectory`; the others beside it, named by their appearances, then merge.
*/
async function captureSideInSessions(mapOptions: CiMapOptions, passes: readonly (readonly ColorScheme[])[] | undefined,
progress: Progress, signal: AbortSignal | undefined, worker: boolean): Promise<Awaited<ReturnType<typeof runCiMap>>> {
const runOnWorker = (passOptions: CiMapOptions) =>
runCiMapOnWorker((({ progress: _progress, runner: _runner, agent: _agent, signal: _signal, ...rest }) => rest)(passOptions), progress, signal);
if (!passes || passes.length <= 1) return worker ? runOnWorker(mapOptions) : runCiMap(mapOptions);
const schemes = passes.flat();
progress(`${mapOptions.side}: capturing ${passes.map(pass => pass.join("+")).join(" and ")} in separate sessions`);
// The passes install the same app: they share one build and repack, as the targets of a local review do.
const ownBuildShare = mapOptions.buildShare ? undefined : await mkdtemp(join(tmpdir(), "expo-explore-builds-"));
const buildShare = mapOptions.buildShare ?? ownBuildShare!;
const settled = await Promise.allSettled(passes.map(async (pass, index) => {
const suffix = index === 0 ? "" : `-${pass.join("-")}`;
const outputDirectory = `${mapOptions.outputDirectory}${suffix}`;
try {
return await runOnWorker({ ...mapOptions, mapDirectory: `${mapOptions.mapDirectory}${suffix}`, outputDirectory, buildShare,
...(mapOptions.sessionStatePath ? { sessionStatePath: `${mapOptions.sessionStatePath}${suffix}` } : {}), appearancePass: { all: schemes, capture: pass } });
} catch (error) {
// A pass that dies mid-build must not leave the others polling for its result.
await abandonSharedBuilds(buildShare, owner => owner === outputDirectory, toCiError(error).code).catch(() => undefined);
throw error;
}
}));
// The build folder holds the repacked app, which the sessions have installed.
if (ownBuildShare) await rm(ownBuildShare, { recursive: true, force: true });
for (const outcome of settled) if (outcome.status === "rejected") throw outcome.reason;
await mergeCaptureExports(mapOptions.outputDirectory, passes.slice(1).map(pass => ({ directory: `${mapOptions.outputDirectory}-${pass.join("-")}`, scheme: pass[0]! })));
return (settled[0] as PromiseFulfilledResult<Awaited<ReturnType<typeof runCiMap>>>).value;
}
/**
* This target's share of `capture.maxSessions`: the targets are this review's (`targets`), or the config's one, and
* `posedTargets` (foldables) take the sessions that do not divide evenly.
*/
function targetSessionShare(config: ExploreConfig, targets: readonly RunTarget[] | undefined, posedTargets: readonly string[] | undefined): number {
const targetIds: string[] = (targets ?? []).map(target => target.id);
if (targetIds.length === 0) targetIds.push(...(config.targets ? Object.keys(config.targets) : [config.target?.id ?? "default"]));
return sessionShare({ maxSessions: config.capture.maxSessions, targets: targetIds, target: config.target?.id ?? targetIds[0]!,
prefer: id => posedTargets?.includes(id) ? 1 : 0 });
}
/**
* The appearance passes one side spreads over sessions of its own, out of its target's share of `capture.maxSessions`;
* undefined keeps the side in one session. The side's app config is read only when its share and routes could pay for
* a split.
*/
export async function sideAppearancePasses(config: ExploreConfig, environment: NodeJS.ProcessEnv,
input: { share: number; sides: 1 | 2; routes: number; project: string; easCommand?: string }): Promise<ColorScheme[][] | undefined> {
// A pass observes its launch only when capture.launch asks for it.
const plan = (schemes: readonly ColorScheme[]) => planSideSessions({ share: input.share, sides: input.sides, routes: input.routes, schemes,
...(config.capture.launch === "off" ? { launchSeconds: 0 } : {}) }).passes;
if (plan(["light", "dark"]).length < 2) return undefined;
const schemes = await expectedColorSchemes(config, input.project, environment, input.easCommand ?? packageBin("eas-cli", "eas"));
const passes = schemes ? plan(schemes) : [];
return passes.length > 1 ? passes.map(pass => [...pass]) : undefined;
}
export function captureSidesConcurrently(input: { plan: SourceCapturePlan; agent: boolean; injectedRunner: boolean; baselineMayExist: boolean }): boolean {
return !input.agent && !input.injectedRunner && !input.baselineMayExist
&& input.plan.headCaptureSurfaceIds.length > 0 && input.plan.baseCaptureSurfaceIds.length > 0;
@@ -112,6 +200,8 @@ export type PublicCiOptions = {
platforms?: ExplorePlatform[];
/** Every target config.yaml captures, in config order; recorded in the report so the Viewer links the sibling targets' reports. */
targets?: RunTarget[];
/** The targets with a device pose (foldables), which take the sessions `capture.maxSessions` does not divide evenly. */
posedTargets?: readonly string[];
baseProject: string; headProject: string; pullRequestPath: string; output: string;
config: ExploreConfig; configPath?: string; environment?: NodeJS.ProcessEnv; transport?: typeof fetch;
now?: () => number; sleep?: (milliseconds: number) => void; onComment?: (state: PublicCommentState, body: string) => void;
@@ -254,6 +344,8 @@ export async function runCiReview(options: {
platforms?: ExplorePlatform[];
/** Every target config.yaml captures; `config.target` is the one this review captures. */
targets?: RunTarget[];
/** The targets with a device pose (foldables), which take the sessions `capture.maxSessions` does not divide evenly. */
posedTargets?: readonly string[];
/** A folder shared with the other targets' reviews, so the targets of one platform build each commit once. */
buildShare?: string;
measure?: Parameters<typeof assessPublicEvidence>[0]["measure"];
@@ -309,7 +401,20 @@ export async function runCiReview(options: {
// A baseline that may be restored needs the head runtime profile first; otherwise both sides capture at once.
const baselineMayExist = plan.baseCaptureSurfaceIds.length > 0 && await captureArtifactCacheMayMatch({ ...cacheRecipes.base,
...(environment.EXPLORE_CAPTURE_CACHE ? { sourceDirectory: environment.EXPLORE_CAPTURE_CACHE } : {}), commit: pr.base.sha });
const concurrent = captureSidesConcurrently({ plan, agent: !!exploration.agent, injectedRunner: !!options.runner, baselineMayExist });
const share = targetSessionShare(config, options.targets, options.posedTargets);
// A share of one session captures the sides in turn.
const concurrent = captureSidesConcurrently({ plan, agent: !!exploration.agent, injectedRunner: !!options.runner, baselineMayExist }) && share >= 2;
// Each side's appearances split by its own app config: a pull request can change userInterfaceStyle. Sides taken in
// turn (a baseline may be restored) each have the whole share. Passes run on worker threads, which cannot take an
// injected runner or an agent session.
const sidePasses = { head: undefined as ColorScheme[][] | undefined, base: undefined as ColorScheme[][] | undefined };
if (!exploration.agent && !options.runner) await Promise.all((["head", "base"] as const).map(async side => {
sidePasses[side] = await sideAppearancePasses(config, environment, { share, sides: concurrent ? 2 : 1, routes: plan[`${side}CaptureSurfaceIds`].length, project: projects[side] });
}));
const split = (["head", "base"] as const).filter(side => sidePasses[side]);
progress(`sessions: ${share} of ${config.capture.maxSessions ?? DEFAULT_MAX_SESSIONS} for this target; `
+ (split.length ? `${split.join(" and ")} ${split.length === 1 ? "captures its" : "capture their"} appearances in separate sessions`
: concurrent ? "one session per side" : "the sides in turn"));
const captureSide = async (side: "head" | "base") => {
const directory = join(output, `${side}-export`);
const selectedIds = plan[`${side}CaptureSurfaceIds`];
@@ -329,9 +434,8 @@ export async function runCiReview(options: {
name: config.app.name ?? pr.repository, buildId: pr.origin === "local" ? `local-${pr.runId}-${side}` : pr.origin === "compare" ? `compare-${options.runId}-${side}` : `pr-${pr.number}-${side}`, buildVersion: pr[side].sha.slice(0, 8), gitCommit: pr[side].sha,
planPath: join(output, "execution-plan.json"), side, sessionStatePath: join(output, `${side}-session`), ...(options.buildShare ? { buildShare: options.buildShare } : {}),
...(reuseDirectory ? { reuseDirectory } : {}), ...(options.signal ? { signal: options.signal } : {}) };
const captured = concurrent
? await runCiMapOnWorker((({ progress: _progress, runner: _runner, agent: _agent, signal: _signal, ...rest }) => rest)(mapOptions), progress, options.signal)
: await runCiMap(mapOptions);
// A restored baseline already holds every appearance pass, so it never splits.
const captured = await captureSideInSessions(mapOptions, reuseDirectory ? undefined : sidePasses[side], progress, options.signal, concurrent);
const bundle = await readCaptureBundle(directory);
bundle.profile = cacheRecipe.profile;
await writeFile(join(directory, "bundle.json"), JSON.stringify(bundle));
@@ -464,6 +568,10 @@ export async function runCiReview(options: {
}
export async function runBaselineCi(options: { platform?: ExplorePlatform; project: string; sha: string; output: string; config: ExploreConfig; environment?: NodeJS.ProcessEnv; runner?: CommandRunner;
/** Every target config.yaml captures, so this baseline takes its target's share of `capture.maxSessions`. */
targets?: RunTarget[];
/** The targets with a device pose (foldables), which take the sessions `capture.maxSessions` does not divide evenly. */
posedTargets?: readonly string[];
now?: () => number; sleep?: (milliseconds: number) => void;
agentSessionFactory?: PrepareCiExplorationOptions["createSession"]; progress?: Progress;
/** Use the local `eas login` session instead of EXPO_TOKEN (expo-explore doctor --capture). */
@@ -489,12 +597,19 @@ export async function runBaselineCi(options: { platform?: ExplorePlatform; proje
onUsage: recordAgentUsage(output, () => usagePhase), ...(options.subscription ? { subscription: true } : {}) });
const directory = join(output, "baseline-export");
usagePhase = "capture-baseline";
const result = await runCiMap({ ...captureOptions(options.config, environment, options.runner, { now: options.now, sleep: options.sleep }), progress,
const baselineOptions: CiMapOptions = { ...captureOptions(options.config, environment, options.runner, { now: options.now, sleep: options.sleep }), progress,
...(options.useLocalEasSession ? { useLocalEasSession: true } : {}),
...(exploration.agent ? { agent: exploration.agent } : {}), project: options.project,
mapDirectory: join(output, "baseline-map"), outputDirectory: directory, name: options.config.app.name ?? options.config.app.bundleId,
buildId: `baseline-${options.sha.slice(0, 12)}`, buildVersion: options.sha.slice(0, 8), gitCommit: options.sha,
side: "head", planPath: join(output, "execution-plan.json"), sessionStatePath: join(output, "baseline-session") });
side: "head", planPath: join(output, "execution-plan.json"), sessionStatePath: join(output, "baseline-session") };
const share = targetSessionShare(options.config, options.targets, options.posedTargets);
// Passes on worker threads cannot take an injected runner or an agent session.
const passes = !options.runner && !exploration.agent
? await sideAppearancePasses(options.config, environment, { share, sides: 1, routes: plan.headCaptureSurfaceIds.length, project: options.project }) : undefined;
progress(`sessions: ${share} of ${options.config.capture.maxSessions ?? DEFAULT_MAX_SESSIONS} for this target; `
+ (passes ? `capturing ${passes.map(pass => pass.join("+")).join(" and ")} in separate sessions` : "one session"));
const result = await captureSideInSessions(baselineOptions, passes, progress, undefined, passes !== undefined);
const cacheRecipe = await recipe(options.config, plan, "head", environment, exploration.instructionsSha256);
if (options.subscription) cacheRecipe.toolsCommit += ":local-subscription";
const bundle = await readCaptureBundle(directory); bundle.profile = cacheRecipe.profile;
+70
View File
@@ -0,0 +1,70 @@
import type { ColorScheme } from "../capture/appearance.js";
/** One route capture on a remote session, with the bridge thread (observed ~5 s; 9.4 s before it). */
export const ROUTE_SECONDS = 5;
/** Observing the app launch once per appearance. */
export const LAUNCH_SECONDS = 28;
/** A pass moves to its own session only when that shortens the side by at least this much. */
export const SPLIT_MIN_SAVING_SECONDS = 60;
/**
* `capture.maxSessions` when the config sets none: the simulators a typical EAS plan runs at once. A starter plan runs
* one per platform, so its config sets the ceiling lower (2 for an iOS and an Android target).
*/
export const DEFAULT_MAX_SESSIONS = 5;
export type SessionShareInput = {
/** `capture.maxSessions`: the review-wide ceiling; absent means DEFAULT_MAX_SESSIONS. */
maxSessions: number | undefined;
/** The review's target ids in config order, and the one this job captures. */
targets: readonly string[];
target: string;
/** Targets a split pays off most for take the sessions that do not divide evenly: foldables and large screens first. */
prefer?: (target: string) => number;
};
/**
* This job's share of the review's session ceiling: the ceiling divided over the targets, the remainder going to the
* preferred targets (foldables) in config order, and unused when there are none. Never below 1 (a side then captures after the other): under a ceiling below the
* number of targets, at most that many target jobs run at once (maxParallelTargets).
*/
export function sessionShare(input: SessionShareInput): number {
const count = Math.max(1, input.targets.length);
const ceiling = input.maxSessions ?? DEFAULT_MAX_SESSIONS;
const base = Math.floor(ceiling / count);
let remainder = ceiling - base * count;
// A session left over goes to a preferred target only; without one it stays unused (two targets of five: two each).
const ranked = input.targets.filter(id => (input.prefer?.(id) ?? 0) > 0)
.sort((a, b) => input.prefer!(b) - input.prefer!(a) || input.targets.indexOf(a) - input.targets.indexOf(b));
const extra = new Set<string>();
for (const id of ranked) { if (remainder <= 0) break; extra.add(id); remainder -= 1; }
return Math.max(1, base + (extra.has(input.target) ? 1 : 0));
}
/** How many target jobs one review runs at once: each needs a session, so never more than the ceiling. */
export function maxParallelTargets(maxSessions: number | undefined, targets: number): number {
return Math.max(1, Math.min(targets, maxSessions ?? DEFAULT_MAX_SESSIONS));
}
export type SidePlan = {
/** Whether head and base capture at the same time. */
concurrentSides: boolean;
/** The appearance passes of one side, each on its own session; one entry means the side keeps a single session. */
passes: readonly (readonly ColorScheme[])[];
};
/**
* How one target's job spends its share of sessions: both sides at once when it has two, and a side's appearance passes
* in separate sessions only when the pass moved out shortens the side by at least a minute (a session costs its fixed
* boot and install time, so a few routes are not worth one). Sessions the work cannot fill stay unused: the ceiling is
* a limit, not a target.
*/
export function planSideSessions(input: { share: number; sides: 1 | 2; routes: number; schemes: readonly ColorScheme[];
minSavingSeconds?: number; routeSeconds?: number; launchSeconds?: number }): SidePlan {
const minSaving = input.minSavingSeconds ?? SPLIT_MIN_SAVING_SECONDS, perRoute = input.routeSeconds ?? ROUTE_SECONDS, launch = input.launchSeconds ?? LAUNCH_SECONDS;
const concurrentSides = input.sides === 1 || input.share >= 2;
const perSide = Math.floor(input.share / (concurrentSides ? input.sides : 1));
const schemes = [...input.schemes];
const passSeconds = input.routes * perRoute + launch;
const splitAppearances = schemes.length > 1 && perSide >= schemes.length && passSeconds >= minSaving;
return { concurrentSides, passes: splitAppearances ? schemes.map(scheme => [scheme]) : [schemes] };
}
+2 -1
View File
@@ -196,7 +196,8 @@ export async function cleanupAfterCapture(options: { output: string; checkouts:
const environment = options.environment ?? process.env;
const result: { stopped: string[]; failures: Array<{ code: string; message: string; nextAction: string }> } = { stopped: [], failures: [] };
const sessionFiles: string[] = [];
for (const name of ["head-session", "base-session", "baseline-session"]) {
// A side that captures an appearance in a session of its own (capture.maxSessions) records it as `<side>-session-<scheme>`.
for (const name of ["head", "base", "baseline"].flatMap(side => ["", "-light", "-dark"].map(scheme => `${side}-session${scheme}`))) {
const path = resolve(options.output, name);
try { await lstat(path); sessionFiles.push(path); } catch { /* That side never started a session. */ }
}
+12 -6
View File
@@ -15,12 +15,13 @@ import { installCiFiles } from "../ci/workflow-install.js";
import { readRouteEntry } from "../ci/route-entry.js";
import { buildVerifiedPlan } from "../ci/verified-plan.js";
import { formatCaptureCheck, formatDoctorReport, referencedSecretNames, runCaptureCheck, runDoctor } from "../ci/doctor.js";
import { readExploreConfig, readExploreInstructions, resolveTargetConfig, configTargets, isTargetId, withRunOverrides, warnIgnoredExpiry, configPlatforms as listPlatforms, EXPLORE_PLATFORMS, DEFAULT_EXPLORE_CONFIG_PATH, type ExplorePlatform } from "../config/explore-config.js";
import { readExploreConfig, readExploreInstructions, resolveTargetConfig, configTargets, posedTargets, isTargetId, withRunOverrides, warnIgnoredExpiry, configPlatforms as listPlatforms, EXPLORE_PLATFORMS, DEFAULT_EXPLORE_CONFIG_PATH, type ExplorePlatform } from "../config/explore-config.js";
import { argumentError, output, required, has, flag, flags, nonNegativeInteger, oneOf, positiveInteger } from "./args.js";
import { help } from "./help.js";
import { CiError } from "../shared/ci-errors.js";
import { cleanupAfterCapture, findOptionalBaseline, resolveCompareRefs, resolvePullRequest, writeCaptureOutputs, writeStepOutputs } from "../ci/workflow-steps.js";
import { stderrProgress } from "../shared/progress.js";
import { maxParallelTargets } from "../ci/session-plan.js";
export async function doctor(): Promise<void> {
if (has("help")) return help();
@@ -159,10 +160,14 @@ export async function configPlatforms(): Promise<void> {
output({ ok: true, platforms: configTargets(config).map(target => target.id) });
}
/** The targets config.yaml captures, in order, with their platforms and labels: the review workflow's matrix. */
/**
* The targets config.yaml captures, in order, with their platforms and labels: the review workflow's matrix, and how
* many of its jobs run at once (`maxParallel`), since each needs a session of capture.maxSessions.
*/
export async function configTargetList(): Promise<void> {
const config = await readExploreConfig(flag("project") ?? process.cwd(), flag("config") ?? DEFAULT_EXPLORE_CONFIG_PATH);
output({ ok: true, targets: configTargets(config) });
const targets = configTargets(config);
output({ ok: true, targets, maxParallel: maxParallelTargets(config.capture.maxSessions, targets.length) });
}
async function exists(path: string): Promise<boolean> {
@@ -229,7 +234,7 @@ export async function captureReview(): Promise<void> {
// A matrix job names its target; its artifacts, report and comment section carry the target's id.
const named = selection.target !== undefined || selection.platform !== undefined;
const common = { ...(named ? { target: config.target!.id } : {}), ...(selection.platform ? { platform: selection.platform } : {}),
platforms: listPlatforms(rawConfig), targets: configTargets(rawConfig), baseProject: required("base-project"), headProject,
platforms: listPlatforms(rawConfig), targets: configTargets(rawConfig), posedTargets: posedTargets(rawConfig), baseProject: required("base-project"), headProject,
output: required("output"), config, configPath, progress: stderrProgress() };
const result = has("compare") ? await runCompareCi({ ...common, comparePath: required("compare") })
: await runPublicCi({ ...common, pullRequestPath: required("pull-request") });
@@ -240,8 +245,9 @@ export async function captureReview(): Promise<void> {
export async function captureBaseline(): Promise<void> {
const project = required("project");
const config = withRunOverrides(resolveTargetConfig(await readExploreConfig(project, flag("config") ?? DEFAULT_EXPLORE_CONFIG_PATH), targetSelection()), { controller: flag("controller"), simulator: flag("simulator"), repack: flag("repack") });
output({ ok: true, ...await runBaselineCi({ platform: config.app.platform, project, sha: required("sha"), output: required("output"), config, progress: stderrProgress() }) });
const rawConfig = await readExploreConfig(project, flag("config") ?? DEFAULT_EXPLORE_CONFIG_PATH);
const config = withRunOverrides(resolveTargetConfig(rawConfig, targetSelection()), { controller: flag("controller"), simulator: flag("simulator"), repack: flag("repack") });
output({ ok: true, ...await runBaselineCi({ platform: config.app.platform, project, sha: required("sha"), output: required("output"), config, targets: configTargets(rawConfig), posedTargets: posedTargets(rawConfig), progress: stderrProgress() }) });
}
/** `capture` (or `capture:review`) `[--base REF] [--head REF | --committed]`: a local review, opened in the Viewer or hosted with --upload. */
+1 -1
View File
@@ -31,7 +31,7 @@ const GET_STARTED = `Get started (run these in the Expo app's directory):
machine, instead of EAS Simulator, and removes it afterwards (or set capture.simulator in config.yaml); builds
still run on EAS, and the host switches appearance for either controller; the app is repacked with the route
observer on this machine too (--repack eas, or capture.repack, keeps the hosted EAS job);
with targets in config.yaml it reviews them at once (--parallel N at most N at a time; the targets of a platform
with targets in config.yaml it reviews them at once, up to capture.maxSessions (--parallel N at most N at a time; the targets of a platform
share each commit's build), --target ID one of them, --platform those of a platform)
view [--catalog FILE] [--port N] [--no-open]
(opens a report in the Viewer)
+12 -1
View File
@@ -94,6 +94,11 @@ export type ExploreConfig = {
/** `exclude` names routes, as `requiresRoute` does, that stay in the map but are never captured. */
exploration: { scope: "changes" | "all"; exclude?: string[] };
capture: { maxMinutes: number; maxActions: number;
/**
* The most EAS Simulator sessions one review may run at once, across its targets and sides; absent:
* DEFAULT_MAX_SESSIONS (5). Below the number of targets, that many target jobs run at once, one session each.
*/
maxSessions?: number;
/** Where the route observer is bundled into the app, when the config chooses; captureRepack gives the default. */
repack?: "eas" | "local"; appearance: AppearanceSetting; launch: LaunchSetting; reset: RouteResetSetting; unattributed: UnattributedSetting; readinessSeconds?: number;
/** The device controller; absent means agent-device, so configs that never chose one keep their cache recipes. */
@@ -295,7 +300,7 @@ export function validateExploreConfig(value: unknown): ExploreConfig {
const exploration = optionalRecord(root.exploration, "exploration");
exactKeys(exploration, ["scope", "exclude"], "exploration");
const capture = optionalRecord(root.capture, "capture");
exactKeys(capture, ["maxMinutes", "maxActions", "repack", "appearance", "launch", "reset", "unattributed", "readinessSeconds", "controller", "simulator"], "capture");
exactKeys(capture, ["maxMinutes", "maxActions", "maxSessions", "repack", "appearance", "launch", "reset", "unattributed", "readinessSeconds", "controller", "simulator"], "capture");
const controller = literal(capture.controller, ["agent-device", "argent"] as const, "capture.controller", "agent-device");
const simulator = literal(capture.simulator, ["eas", "local"] as const, "capture.simulator", "eas");
const mode = literal(root.mode, ["deterministic", "agent"] as const, "mode", "deterministic");
@@ -342,6 +347,7 @@ export function validateExploreConfig(value: unknown): ExploreConfig {
capture: {
maxMinutes: positiveInteger(capture.maxMinutes, "capture.maxMinutes", 30),
maxActions: positiveInteger(capture.maxActions, "capture.maxActions", 500),
...(capture.maxSessions === undefined ? {} : { maxSessions: positiveInteger(capture.maxSessions, "capture.maxSessions", 2) }),
...(capture.repack === undefined ? {} : { repack: literal(capture.repack, ["eas", "local"] as const, "capture.repack") }),
appearance: parseAppearance(capture.appearance, "capture.appearance"),
launch: parseLaunchSetting(capture.launch, "capture.launch"),
@@ -440,6 +446,11 @@ export function configTargets(config: ExploreConfig): RunTarget[] {
label: target.label ?? `${target.device ?? (target.platform === config.app.platform ? config.device.name : DEFAULT_ANDROID_DEVICE)}${target.pose ? ` (${target.pose})` : ""}` }));
}
/** The targets with a device pose (foldables): they take the sessions `capture.maxSessions` does not divide evenly. */
export function posedTargets(config: ExploreConfig): string[] {
return Object.entries(config.targets ?? {}).filter(([, target]) => target.pose).map(([id]) => id);
}
/**
* The flat configuration capture consumes for one target, with `target` naming it. `--target` names one; `--platform`
* names the target of that platform, when only one has it; with neither, the config must have a single target.
+6 -2
View File
@@ -25,6 +25,7 @@ jobs:
contents: read
outputs:
targets: ${{ steps.targets.outputs.targets }}
max_parallel: ${{ steps.targets.outputs.max_parallel }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: { ref: '${{ github.sha }}', path: app, persist-credentials: false }
@@ -51,12 +52,15 @@ jobs:
shell: bash
run: |
set -euo pipefail
targets="$(node "$TOOLS/dist/cli.js" config:targets --project "$GITHUB_WORKSPACE/app/$APP_PROJECT_DIR" --config .expo-explore/config.yaml | jq -c '[.targets[].id]')"
echo "targets=$targets" >> "$GITHUB_OUTPUT"
listed="$(node "$TOOLS/dist/cli.js" config:targets --project "$GITHUB_WORKSPACE/app/$APP_PROJECT_DIR" --config .expo-explore/config.yaml)"
echo "targets=$(jq -c '[.targets[].id]' <<< "$listed")" >> "$GITHUB_OUTPUT"
# Each target job needs a simulator session: at most capture.maxSessions of them run at once.
echo "max_parallel=$(jq '.maxParallel // (.targets | length)' <<< "$listed")" >> "$GITHUB_OUTPUT"
baseline:
needs: targets
strategy:
fail-fast: false
max-parallel: ${{ fromJSON(needs.targets.outputs.max_parallel) }}
matrix:
target: ${{ fromJSON(needs.targets.outputs.targets) }}
runs-on: ubuntu-22.04
+6 -2
View File
@@ -48,6 +48,7 @@ jobs:
contents: read
outputs:
targets: ${{ steps.targets.outputs.targets }}
max_parallel: ${{ steps.targets.outputs.max_parallel }}
base: ${{ steps.refs.outputs.base }}
head: ${{ steps.refs.outputs.head }}
compare: ${{ steps.refs.outputs.compare }}
@@ -89,12 +90,15 @@ jobs:
shell: bash
run: |
set -euo pipefail
targets="$(node "$TOOLS/dist/cli.js" config:targets --project "$GITHUB_WORKSPACE/app-head/$APP_PROJECT_DIR" --config .expo-explore/config.yaml | jq -c '[.targets[].id]')"
echo "targets=$targets" >> "$GITHUB_OUTPUT"
listed="$(node "$TOOLS/dist/cli.js" config:targets --project "$GITHUB_WORKSPACE/app-head/$APP_PROJECT_DIR" --config .expo-explore/config.yaml)"
echo "targets=$(jq -c '[.targets[].id]' <<< "$listed")" >> "$GITHUB_OUTPUT"
# Each target job needs a simulator session: at most capture.maxSessions of them run at once.
echo "max_parallel=$(jq '.maxParallel // (.targets | length)' <<< "$listed")" >> "$GITHUB_OUTPUT"
capture:
needs: targets
strategy:
fail-fast: false
max-parallel: ${{ fromJSON(needs.targets.outputs.max_parallel) }}
matrix:
target: ${{ fromJSON(needs.targets.outputs.targets) }}
runs-on: ubuntu-22.04
+6 -2
View File
@@ -33,6 +33,7 @@ jobs:
pull-requests: read
outputs:
targets: ${{ steps.targets.outputs.targets }}
max_parallel: ${{ steps.targets.outputs.max_parallel }}
# The failure job comments with these when a capture job fails before it resolves the PR itself.
pr_number: ${{ steps.pr.outputs.pr_number }}
base: ${{ steps.pr.outputs.base }}
@@ -69,12 +70,15 @@ jobs:
shell: bash
run: |
set -euo pipefail
targets="$(node "$TOOLS/dist/cli.js" config:targets --project "$GITHUB_WORKSPACE/app-head/$APP_PROJECT_DIR" --config .expo-explore/config.yaml | jq -c '[.targets[].id]')"
echo "targets=$targets" >> "$GITHUB_OUTPUT"
listed="$(node "$TOOLS/dist/cli.js" config:targets --project "$GITHUB_WORKSPACE/app-head/$APP_PROJECT_DIR" --config .expo-explore/config.yaml)"
echo "targets=$(jq -c '[.targets[].id]' <<< "$listed")" >> "$GITHUB_OUTPUT"
# Each target job needs a simulator session: at most capture.maxSessions of them run at once.
echo "max_parallel=$(jq '.maxParallel // (.targets | length)' <<< "$listed")" >> "$GITHUB_OUTPUT"
capture:
needs: targets
strategy:
fail-fast: false
max-parallel: ${{ fromJSON(needs.targets.outputs.max_parallel) }}
matrix:
target: ${{ fromJSON(needs.targets.outputs.targets) }}
runs-on: ubuntu-22.04
+2 -2
View File
@@ -54,7 +54,7 @@ export async function readUsage(output) {
export async function fixture(t, provider, { docsOnly = false, unresolvedParameters = false, observerFailurePath, agentMode = false,
instructions = 'Explore selected routes in a stable, comparable state.', recoverableOpenFailure = false,
userInterfaceStyle, headUserInterfaceStyle, platform: initialPlatform = 'ios', targets } = {}) {
userInterfaceStyle, headUserInterfaceStyle, platform: initialPlatform = 'ios', targets, maxSessions } = {}) {
let platform = initialPlatform;
const root = await realpath(await mkdtemp(join(tmpdir(), 'explore-ci-integration-')));
t.after(() => rm(root, { recursive: true, force: true }));
@@ -67,7 +67,7 @@ export async function fixture(t, provider, { docsOnly = false, unresolvedParamet
const commit = message => { git(headProject, 'add', '.'); git(headProject, 'commit', '-m', message); return git(headProject, 'rev-parse', 'HEAD'); };
const config = { mode: 'deterministic', app: { name: 'Example', platform, bundleId: 'com.example.app', easProfile: 'capture', scheme: 'sample' },
device: { name: 'iPhone 17 Pro' }, navigation: { provider, ...(provider === 'react-navigation' ? { entry: 'src/App.tsx' } : {}) },
exploration: { scope: 'changes' }, capture: { maxMinutes: 30, maxActions: 500, repack: 'eas' },
exploration: { scope: 'changes' }, capture: { maxMinutes: 30, maxActions: 500, repack: 'eas', ...(maxSessions ? { maxSessions } : {}) },
parameters: provider === 'expo-router' && !unresolvedParameters ? { '/item/[id]': { id: 'example-item' } } : {} };
if (targets) config.targets = targets;
if (agentMode) { config.mode = 'agent'; config.agent = { provider: 'anthropic', model: 'fixture-model', instructions: '.expo-explore/instructions.md' }; }
+35 -5
View File
@@ -138,9 +138,10 @@ test("baseline captures a complete single default-branch commit without a PR", (
const job = baseline.jobs.baseline;
assert.match(baseline.jobs.targets.if, /github\.event\.repository\.default_branch/);
assert.deepEqual(job.needs, "targets");
assert.deepEqual(job.strategy, { "fail-fast": false, matrix: { target: "${{ fromJSON(needs.targets.outputs.targets) }}" } });
assert.deepEqual(job.strategy, { "fail-fast": false, "max-parallel": "${{ fromJSON(needs.targets.outputs.max_parallel) }}",
matrix: { target: "${{ fromJSON(needs.targets.outputs.targets) }}" } });
assert.equal(job.env.TARGET, "${{ matrix.target }}");
assert.match(baseline.jobs.targets.steps.at(-1).run, /config:targets .* \| jq -c '\[\.targets\[\]\.id\]'/);
assert.match(baseline.jobs.targets.steps.at(-1).run, /config:targets --project "\$GITHUB_WORKSPACE\/app\/\$APP_PROJECT_DIR"/);
const verify = job.steps.find(step => step.name === "Verify deterministic source planning");
assert.match(verify.run, /--base-sha "\$GITHUB_SHA" --head-sha "\$GITHUB_SHA" --scope all/);
const capture = job.steps.find(step => step.name === "Capture complete single-commit baseline");
@@ -325,16 +326,21 @@ test("review captures and reports each configured target in its own matrix job,
assert.deepEqual(targets.permissions, { contents: "read", "pull-requests": "read" });
assert.equal(targets.outputs.targets, "${{ steps.targets.outputs.targets }}");
// config:targets lists ids in config order; a config without targets: lists ["ios"] / ["ios","android"], as config:platforms did.
assert.match(targets.steps.at(-1).run, /config:targets --project "\$GITHUB_WORKSPACE\/app-head\/\$APP_PROJECT_DIR" --config \.expo-explore\/config\.yaml \| jq -c '\[\.targets\[\]\.id\]'/);
assert.match(targets.steps.at(-1).run, /config:targets --project "\$GITHUB_WORKSPACE\/app-head\/\$APP_PROJECT_DIR" --config \.expo-explore\/config\.yaml/);
assert.equal(targets.outputs.max_parallel, "${{ steps.targets.outputs.max_parallel }}");
assert.doesNotMatch(JSON.stringify(targets), /EXPO_TOKEN|OPENAI_API_KEY|ANTHROPIC_API_KEY|APP_REVIEW/);
for (const workflow of [review, compare, baseline]) {
assert.doesNotMatch(JSON.stringify(workflow), /matrix\.platform|config:platforms|\$PLATFORM|needs\.platforms/, "every job is keyed by target, never by platform");
assert.match(JSON.stringify(workflow), /matrix\.target/, "doctor looks for matrix.target in an installed workflow");
}
for (const job of [review.jobs.capture, review.jobs.report]) {
assert.deepEqual(job.strategy, { "fail-fast": false, matrix: { target: "${{ fromJSON(needs.targets.outputs.targets) }}" } });
// Each capture job needs a simulator session, so at most capture.maxSessions of them run at once; a report needs none.
for (const job of [review.jobs.capture, compare.jobs.capture]) {
assert.deepEqual(job.strategy, { "fail-fast": false, "max-parallel": "${{ fromJSON(needs.targets.outputs.max_parallel) }}",
matrix: { target: "${{ fromJSON(needs.targets.outputs.targets) }}" } });
assert.equal(job.env.TARGET, "${{ matrix.target }}");
}
assert.deepEqual(review.jobs.report.strategy, { "fail-fast": false, matrix: { target: "${{ fromJSON(needs.targets.outputs.targets) }}" } });
assert.equal(review.jobs.report.env.TARGET, "${{ matrix.target }}");
assert.equal(review.jobs.capture.needs, "targets");
assert.deepEqual(review.jobs.report.needs, ["targets", "capture"]);
const runs = [review.jobs.capture, review.jobs.report, baseline.jobs.baseline].flatMap(job => job.steps.filter(step => step.run).map(step => step.run));
@@ -378,3 +384,27 @@ test("compare reviews any two refs from a manual run or another workflow, with n
assert.match(compare.jobs.capture.steps.find(step => step.id === "capture").run, /printf '%s' "\$COMPARE" > "\$RUNNER_TEMP\/explore-compare\.json"/);
assert.match(compare.jobs.deploy.steps.at(-1).run, /ci:deploy --inputs/);
});
test("the targets job lists the target ids and how many capture jobs run at once: no more than capture.maxSessions", async t => {
const root = await mkdtemp(join(tmpdir(), "explore-targets-step-"));
t.after(() => rm(root, { recursive: true, force: true }));
const tools = new URL("../..", import.meta.url).pathname;
const run = async (workflow, checkout, config) => {
const project = join(root, checkout); await mkdir(join(project, ".expo-explore"), { recursive: true });
await writeFile(join(project, ".expo-explore", "config.yaml"), config);
const output = join(root, "output"); await writeFile(output, "");
const step = workflow.jobs.targets.steps.find(entry => entry.id === "targets");
const result = spawnSync("bash", ["-c", step.run], { encoding: "utf8",
env: { PATH: process.env.PATH, HOME: root, TOOLS: tools, GITHUB_WORKSPACE: root, APP_PROJECT_DIR: ".", GITHUB_OUTPUT: output } });
assert.equal(result.status, 0, result.stderr);
const outputs = Object.fromEntries((await readFile(output, "utf8")).trim().split("\n").map(line => line.split(/=(.*)/s).slice(0, 2)));
return { targets: JSON.parse(outputs.targets), maxParallel: JSON.parse(outputs.max_parallel) };
};
const app = "app:\n bundleId: com.example.app\n easProfile: explore\ndevice:\n name: iPhone 17\n";
const five = "targets:\n" + ["iphone", "ipad", "duo-closed", "duo-open"].map(id => ` ${id}: { platform: ios }\n`).join("") + " pixel: { platform: android, bundleId: com.example.app, easProfile: explore }\n";
for (const [workflow, checkout] of [[review, "app-head"], [compare, "app-head"], [baseline, "app"]]) {
assert.deepEqual(await run(workflow, checkout, app + five), { targets: ["iphone", "ipad", "duo-closed", "duo-open", "pixel"], maxParallel: 5 }, "unset: five");
assert.deepEqual((await run(workflow, checkout, app + "capture:\n maxSessions: 2\n" + five)).maxParallel, 2, "a starter plan's two");
assert.deepEqual(await run(workflow, checkout, app), { targets: ["ios"], maxParallel: 1 });
}
});
@@ -135,6 +135,9 @@ test("capture recipes exclude display settings but bind agent instructions and c
assert.equal(JSON.parse(key).observer, CAPTURE_OBSERVER_HASH);
assert.match(JSON.parse(key).observer, /^[a-f0-9]{64}$/);
assert.equal(key,captureRecipeConfig({...config,app:{...config.app,name:"Renamed display"}}));
// How many sessions capture it does not change the evidence: a baseline captured under any ceiling serves the review.
assert.equal(key,captureRecipeConfig({...config,capture:{...config.capture,maxSessions:16}}));
assert.equal(captureRecipeConfig({...config,capture:{...config.capture,maxSessions:4}}),captureRecipeConfig({...config,capture:{...config.capture,maxSessions:16}}));
// An unset repack is keyed as it resolves, so configs that never chose one keep the recipe of the old written-out default.
assert.equal(key,captureRecipeConfig({...config,capture:{...config.capture,repack:"eas"}}));
assert.equal(JSON.parse(key).capture.repack, "eas");
+58 -1
View File
@@ -14,7 +14,10 @@ import { AGENT_DEVICE_VERSION, captureDeepLink, controllerCommandFailure, revisi
import { CiError, serializeCiError } from '../../dist/shared/ci-errors.js';
import { createSourceCapturePlan } from '../../dist/source/capture-plan.js';
import { CAPTURE_OBSERVER_IDENTIFIER } from '../../dist/capture/observer.js';
import { parseRuntimeExecutionContext } from '../../dist/review/runtime-adapter.js';
import { parseRuntimeExecutionContext, runtimeEvidenceInputFromMap } from '../../dist/review/runtime-adapter.js';
import { readExportedMap } from '../../dist/review/evidence-assessment.js';
import { projectRuntimeEvidence } from '../../dist/review/runtime-evidence.js';
import { mergeCaptureExports } from '../../dist/ci/export-merge.js';
import { assertCompleteCaptures, hasVerifiedCapture } from '../../dist/capture/health.js';
import { readCaptureBundle } from '../../dist/review/capture-review.js';
import { writeCaptureObserverPackages } from '../helpers/capture-observer-fixture.mjs';
@@ -3557,3 +3560,57 @@ test('a verified label is located on the image\'s axes when the screenshot is th
// A 669 x 951 portrait panel captured upright as a 951 x 669 image: the button near its bottom-left is near the image's top-left.
assert.deepEqual(verifiedCaptureLabels(nodes, ['Done'], { height: 951 })[0].rect, { x: 11, y: 10, width: 40, height: 100 });
});
test('a session given only the dark pass (appearancePass) captures @dark screens shaped as in a single run', async t => {
const f = await fixture(t, { userInterfaceStyle: 'automatic' });
const result = await runCiMap({ ...f.options, appearancePass: { all: ['light', 'dark'], capture: ['dark'] } });
assert.equal(result.runStatus, 'complete');
assert.deepEqual(f.requests.filter(request => request.args[0] === 'settings').map(request => request.args.slice(1, 3)), [['appearance', 'dark']]);
assert.equal(f.requests.filter(request => request.args[0] === 'screenshot').length, 2);
const map = JSON.parse(await readFile(join(f.options.outputDirectory, 'screen-map.json'), 'utf8'));
assert.deepEqual(map.run.environment.colorSchemes, ['light', 'dark']);
assert.deepEqual(map.screens.map(screen => [screen.id, screen.colorScheme]), [[`${f.ids.home}@dark`, 'dark'], [`${f.ids.settings}@dark`, 'dark']]);
const bundle = JSON.parse(await readFile(join(f.options.outputDirectory, 'bundle.json'), 'utf8'));
assert.deepEqual(Object.keys(bundle.images).sort(), [`${f.ids.home}@dark`, `${f.ids.settings}@dark`]);
assert.deepEqual(Object.keys(bundle.outcomes).sort(), [`${f.ids.home}@dark`, `${f.ids.settings}@dark`]);
assert.equal(bundle.runtimeProfile.colorSchemes, 'light,dark');
});
test('a session given only the dark pass keeps a wrong landing as the unattributed frame its pass recorded, for the export merge', async t => {
const f = await fixture(t, { userInterfaceStyle: 'automatic', markerMismatchAt: { path: '/settings', scheme: 'dark' } });
const result = await runCiMap({ ...f.options, appearancePass: { all: ['light', 'dark'], capture: ['dark'] } });
const dark = `${f.ids.settings}@dark`;
// Without the light pass beside it, the frame is not demoted to a lost route: the merge decides, as one run's passes do.
assert.equal(result.outcomes[dark].status, 'captured');
assert.equal(result.outcomes[dark].landing, 'unverified');
assert.equal(result.outcomes[f.ids.settings], undefined);
assert.equal(result.runStatus, 'partial');
const bundle = JSON.parse(await readFile(join(f.options.outputDirectory, 'bundle.json'), 'utf8'));
assert.equal(bundle.images[dark].sha256, result.outcomes[dark].screenshotSha256);
assert.deepEqual(Object.keys(bundle.outcomes).sort(), [`${f.ids.home}@dark`, dark].sort());
});
test('a split side merges into the export one run makes: each appearance a verified sibling, its runtime evidence resolvable', async t => {
const f = await fixture(t, { userInterfaceStyle: 'automatic' });
const at = name => ({ mapDirectory: join(f.root, `map${name}`), outputDirectory: join(f.root, `export${name}`) });
await runCiMap({ ...f.options, ...at('-single') });
await runCiMap({ ...f.options, ...at(''), appearancePass: { all: ['light', 'dark'], capture: ['light'] } });
await runCiMap({ ...f.options, ...at('-dark'), appearancePass: { all: ['light', 'dark'], capture: ['dark'] } });
await mergeCaptureExports(join(f.root, 'export'), [{ directory: join(f.root, 'export-dark'), scheme: 'dark' }]);
const [single, merged] = await Promise.all(['export-single', 'export'].map(name => readCaptureBundle(join(f.root, name))));
assert.deepEqual(Object.keys(merged.images).sort(), Object.keys(single.images).sort());
assert.deepEqual(merged.outcomes, single.outcomes);
// What a review needs of a sibling appearance (#28): its own verified frame and a map that resolves it as one run's would.
const map = await readExportedMap(join(f.root, 'export'));
const execution = parseRuntimeExecutionContext(JSON.parse(await readFile(join(f.root, 'export', 'execution.json'), 'utf8')));
for (const [route, id] of Object.values(f.ids).flatMap(id => [[id, id], [id, `${id}@dark`]])) {
// A verified secondary frame has no outcome of its own: it is verified under its route's, as the review reads it.
const outcome = merged.outcomes[id] ?? merged.outcomes[route];
assert.equal(outcome.status, 'captured', id); assert.equal(outcome.landing, 'verified', id);
assert.ok(existsSync(join(f.root, 'export', merged.images[id].path)), id);
const screen = map.screens.find(entry => entry.id === id);
assert.equal(screen.observationIds.length, 1, id);
projectRuntimeEvidence(runtimeEvidenceInputFromMap({ side: 'head', map, screenId: id, representativeObservationId: screen.observationIds[0],
sampleObservationIds: screen.observationIds, expectedSampleCount: 1, profile: 'capture:com.example.app', execution }));
}
});
+9
View File
@@ -346,6 +346,15 @@ test("target ids and entries are validated", () => {
assert.throws(() => validateExploreConfig({ ...base(), targets }), error => error.code === "CONFIG_INVALID", JSON.stringify(targets));
}
assert.throws(() => validateExploreConfig({ ...base(), platforms: { ios: {} }, targets: { iphone: { platform: "ios" } } }), /targets or platforms, not both/);
const twoTargets = { iphone: { platform: "ios" }, ipad: { platform: "ios" } };
assert.equal(validateExploreConfig({ ...base(), targets: twoTargets, capture: { maxSessions: 2 } }).capture.maxSessions, 2);
assert.equal("maxSessions" in validateExploreConfig({ ...base(), targets: twoTargets }).capture, false);
// A ceiling below the number of targets runs that many target jobs at once (a starter plan's one simulator per platform).
assert.equal(validateExploreConfig({ ...base(), targets: twoTargets, capture: { maxSessions: 1 } }).capture.maxSessions, 1);
assert.throws(() => validateExploreConfig({ ...base(), capture: { maxSessions: 0 } }), /capture.maxSessions must be a positive integer/);
assert.throws(() => validateExploreConfig({ ...base(), capture: { maxSessions: 1.5 } }), /capture.maxSessions must be a positive integer/);
const platforms = { ios: {}, android: { bundleId: "com.example.app", easProfile: "explore" } };
assert.equal(validateExploreConfig({ ...base(), platforms, capture: { maxSessions: 1 } }).capture.maxSessions, 1);
});
test("a target can choose its own controller and simulator over capture's", () => {
+96
View File
@@ -0,0 +1,96 @@
import assert from "node:assert/strict";
import { mkdtemp, mkdir, readFile, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import test from "node:test";
import { mergeSchemeOutcomes } from "../../dist/capture/appearance.js";
import { mergeCaptureExports } from "../../dist/ci/export-merge.js";
async function writeExport(root, { screens, images, outcomes, launch, diagnostics }) {
await mkdir(join(root, "artifacts"), { recursive: true });
await mkdir(join(root, "capture-diagnostics"), { recursive: true });
const artifacts = [];
for (const screen of screens) {
const path = `artifacts/${screen}.png`;
await writeFile(join(root, path), screen);
artifacts.push({ id: `artifact-${screen}`, path, sha256: screen, bytes: screen.length, mimeType: "image/png" });
}
const map = { contract: "explore.map", revision: 1, run: { id: "run", environment: { colorSchemes: ["light", "dark"] } }, artifacts,
screens: screens.map(id => ({ id, colorScheme: id.endsWith("@dark") ? "dark" : "light" })), variants: [], observations: screens.map(id => ({ screenId: id })),
elements: [], actionAttempts: [], transitions: [], journeys: [], frontier: [], gaps: [] };
await writeFile(join(root, "screen-map.json"), JSON.stringify(map));
await writeFile(join(root, "bundle.json"), JSON.stringify({ contract: "explore.bundle", side: "head", runtimeProfile: { colorSchemes: "light,dark" }, images, outcomes, ...(launch ? { launch } : {}) }));
await writeFile(join(root, "manifest.json"), JSON.stringify({ contract: "explore.map", runId: "run", revision: 1, files: artifacts.map(({ id: _id, ...file }) => file) }));
await writeFile(join(root, "capture-diagnostics", "controller-timing.jsonl"), diagnostics);
return root;
}
test("a dark session's export folds into the light session's: @dark frames, outcomes, launch, map records, files and diagnostics", async () => {
const root = await mkdtemp(join(tmpdir(), "explore-merge-"));
const primary = await writeExport(join(root, "head-export"), { screens: ["home", "settings"],
images: { home: { path: "artifacts/home.png" }, settings: { path: "artifacts/settings.png" } },
outcomes: { home: { status: "captured", landing: "verified", attempts: 1 }, settings: { status: "captured", landing: "verified", attempts: 1 } },
launch: { light: { splash: "l" } }, diagnostics: "{\"pass\":\"light\"}\n" });
const dark = await writeExport(join(root, "head-export-dark"), { screens: ["home@dark", "settings@dark"],
images: { "home@dark": { path: "artifacts/[email protected]" }, "settings@dark": { path: "artifacts/[email protected]" } },
// A secondary-only run reports a route it lost under the route id, as a run's dark pass does before the passes merge.
outcomes: { "home@dark": { status: "captured", landing: "verified", attempts: 2 }, settings: { status: "failed", landing: "unverified", attempts: 3 } },
launch: { dark: { splash: "d" } }, diagnostics: "{\"pass\":\"dark\"}\n" });
await mergeCaptureExports(primary, [{ directory: dark, scheme: "dark" }]);
const bundle = JSON.parse(await readFile(join(primary, "bundle.json"), "utf8"));
assert.deepEqual(Object.keys(bundle.images).sort(), ["home", "home@dark", "settings", "settings@dark"]);
// As in one run: a dark frame verified beside its light one needs no record of its own; a lost one is the scheme's own outcome.
assert.deepEqual(bundle.outcomes, { home: { status: "captured", landing: "verified", attempts: 2 }, settings: { status: "captured", landing: "verified", attempts: 3 },
"settings@dark": { status: "failed", landing: "unverified", attempts: 3 } });
assert.deepEqual(bundle.launch, { light: { splash: "l" }, dark: { splash: "d" } });
const map = JSON.parse(await readFile(join(primary, "screen-map.json"), "utf8"));
assert.deepEqual(map.screens.map(screen => screen.id), ["home", "settings", "home@dark", "settings@dark"]);
assert.equal(map.observations.length, 4);
assert.equal(map.artifacts.length, 4);
const manifest = JSON.parse(await readFile(join(primary, "manifest.json"), "utf8"));
assert.deepEqual(manifest.files.map(file => file.path).sort(), ["artifacts/home.png", "artifacts/[email protected]", "artifacts/settings.png", "artifacts/[email protected]"]);
assert.equal(await readFile(join(primary, "artifacts/[email protected]"), "utf8"), "home@dark");
assert.equal(await readFile(join(primary, "capture-diagnostics/controller-timing.jsonl"), "utf8"), "{\"pass\":\"light\"}\n{\"pass\":\"dark\"}\n");
// Merging is idempotent for records already present.
await mergeCaptureExports(primary, [{ directory: dark, scheme: "dark" }]);
assert.equal(JSON.parse(await readFile(join(primary, "screen-map.json"), "utf8")).screens.length, 4);
});
test("merged passes recombine as one run's: an unattributed dark frame beside a verified light one is kept", async () => {
const root = await mkdtemp(join(tmpdir(), "explore-merge-"));
const light = { status: "captured", landing: "verified", attempts: 1 };
const wrong = { status: "captured", landing: "unverified", attempts: 2, screenshotSha256: "x", observed: { reason: "landing-mismatch", route: "/login", attempts: 2 } };
const primary = await writeExport(join(root, "head-export"), { screens: ["settings"], images: { settings: { path: "artifacts/settings.png" } },
outcomes: { settings: light }, diagnostics: "" });
const dark = await writeExport(join(root, "head-export-dark"), { screens: ["settings@dark"], images: { "settings@dark": { path: "artifacts/[email protected]" } },
outcomes: { "settings@dark": wrong }, diagnostics: "" });
await mergeCaptureExports(primary, [{ directory: dark, scheme: "dark" }]);
const bundle = JSON.parse(await readFile(join(primary, "bundle.json"), "utf8"));
assert.deepEqual(bundle.outcomes, mergeSchemeOutcomes([{ settings: light }, { settings: wrong }], ["light", "dark"]));
assert.deepEqual(bundle.outcomes["settings@dark"], wrong);
});
test("a dark session that could not show dark (Argent without appearance.set) adds nothing, and an open gap leaves the run partial", async () => {
const root = await mkdtemp(join(tmpdir(), "explore-merge-"));
const primary = await writeExport(join(root, "head-export"), { screens: ["home"], images: { home: { path: "artifacts/home.png" } },
outcomes: { home: { status: "captured", landing: "verified", attempts: 1 } }, diagnostics: "" });
const bundlePath = join(primary, "bundle.json"), mapPath = join(primary, "screen-map.json");
// The primary session could not switch either: it captured light only.
await writeFile(bundlePath, JSON.stringify({ ...JSON.parse(await readFile(bundlePath, "utf8")), runtimeProfile: { colorScheme: "light" } }));
await writeFile(mapPath, JSON.stringify({ ...JSON.parse(await readFile(mapPath, "utf8")), run: { id: "run", status: "complete", environment: {} } }));
const narrowed = await writeExport(join(root, "head-export-dark"), { screens: [], images: {}, outcomes: {}, diagnostics: "" });
const narrowedBundle = join(narrowed, "bundle.json");
await writeFile(narrowedBundle, JSON.stringify({ ...JSON.parse(await readFile(narrowedBundle, "utf8")), runtimeProfile: { colorScheme: "light" } }));
await mergeCaptureExports(primary, [{ directory: narrowed, scheme: "dark" }]);
assert.deepEqual(Object.keys(JSON.parse(await readFile(bundlePath, "utf8")).outcomes), ["home"]);
assert.equal(JSON.parse(await readFile(mapPath, "utf8")).run.status, "complete");
const gapped = await writeExport(join(root, "other-export-dark"), { screens: ["home@dark"], images: { "home@dark": { path: "artifacts/[email protected]" } },
outcomes: { "home@dark": { status: "captured", landing: "verified", attempts: 1 } }, diagnostics: "" });
const gappedMap = join(gapped, "screen-map.json");
await writeFile(gappedMap, JSON.stringify({ ...JSON.parse(await readFile(gappedMap, "utf8")), gaps: [{ id: "gap-1", status: "open" }] }));
await writeFile(bundlePath, JSON.stringify({ ...JSON.parse(await readFile(bundlePath, "utf8")), runtimeProfile: { colorSchemes: "light,dark" } }));
await mergeCaptureExports(primary, [{ directory: gapped, scheme: "dark" }]);
assert.equal(JSON.parse(await readFile(mapPath, "utf8")).run.status, "partial", "a complete run has no open gaps");
});
+24
View File
@@ -437,6 +437,30 @@ test('a review runs at most --parallel targets at once, and every target settles
assert.deepEqual(results.map(result => result.status === 'fulfilled' ? result.value : result.reason.message), [10, 20, 'three', 40, 50]);
});
test('a local review runs no more targets at once than capture.maxSessions', async t => {
const targets = { phone: { platform: 'ios' }, tablet: { platform: 'ios', device: 'iPad Air' } };
// Whether one target's review began before the other's ended, from the order of their progress lines.
const overlapped = async (maxSessions, parallel) => {
const f = await fixture(t, 'expo-router', { targets, maxSessions });
t.mock.method(globalThis, 'fetch', async input => assert.fail(`Unmocked network request: ${input}`));
const environment = { ...f.environment, EXPLORE_CACHE_DIR: await scratch(t, 'explore-local-cache-'), GITHUB_RUN_ID: '', GITHUB_RUN_ATTEMPT: '', GITHUB_TOKEN: '' };
const lines = [];
const result = await runLocalReview({ project: f.headProject, base: f.baseSha, head: f.headSha, environment, cache: false, ...(parallel ? { parallel } : {}),
install: async checkout => { await writeCaptureObserverPackages(checkout, { provider: 'expo-router' }); }, runner: f.runner, now: f.now, sleep: f.sleep,
progress: message => lines.push(message) });
t.after(() => rm(result.work, { recursive: true, force: true }));
assert.deepEqual(result.reports.map(report => report.target.id).sort(), ['phone', 'tablet']);
const span = id => [lines.findIndex(line => line.startsWith(`${id}: `)), lines.findLastIndex(line => line.startsWith(`${id}: `))];
const [phone, tablet] = [span('phone'), span('tablet')];
return phone[0] < tablet[1] && tablet[0] < phone[1];
};
// One simulator at a time: the targets take turns.
assert.equal(await overlapped(1), false);
// The default ceiling (five) runs both at once, and --parallel still chooses for itself.
assert.equal(await overlapped(undefined), true);
assert.equal(await overlapped(1, 2), true);
});
test('a target that fails outright costs only its own report', async t => {
const targets = { phone: { platform: 'ios' }, tablet: { platform: 'ios', device: 'iPad Air' } };
const f = await fixture(t, 'expo-router', { targets });
+86
View File
@@ -0,0 +1,86 @@
import assert from "node:assert/strict";
import { chmod, mkdir, mkdtemp, readFile, realpath, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import test from "node:test";
import { sideAppearancePasses } from "../../dist/ci/pipeline.js";
import { DEFAULT_MAX_SESSIONS, maxParallelTargets, planSideSessions, sessionShare } from "../../dist/ci/session-plan.js";
import { validateExploreConfig } from "../../dist/config/explore-config.js";
test("the review-wide session ceiling is divided over the targets, the remainder going to the preferred ones", () => {
const targets = ["iphone", "duo-closed", "duo-open", "android"];
const prefer = target => target.startsWith("duo") ? 1 : 0;
assert.equal(sessionShare({ maxSessions: 8, targets, target: "android" }), 2);
assert.equal(sessionShare({ maxSessions: 16, targets, target: "iphone" }), 4);
assert.equal(sessionShare({ maxSessions: 10, targets, target: "duo-closed", prefer }), 3, "remainder goes to the foldable first");
assert.equal(sessionShare({ maxSessions: 10, targets, target: "duo-open", prefer }), 3);
assert.equal(sessionShare({ maxSessions: 10, targets, target: "iphone", prefer }), 2);
assert.equal(sessionShare({ maxSessions: 2, targets, target: "iphone" }), 1, "never below one");
});
test("unset, the ceiling is a typical plan's five simulators; below the number of targets, that many target jobs run at once", () => {
assert.equal(DEFAULT_MAX_SESSIONS, 5);
// One target: head and base at once, with sessions left to split light and dark.
assert.equal(sessionShare({ maxSessions: undefined, targets: ["iphone"], target: "iphone" }), 5);
assert.deepEqual(planSideSessions({ share: 5, sides: 2, routes: 16, schemes: ["light", "dark"] }), { concurrentSides: true, passes: [["light"], ["dark"]] });
// Two targets: two each, head and base at once, as before the option.
for (const target of ["iphone", "android"]) assert.equal(sessionShare({ maxSessions: undefined, targets: ["iphone", "android"], target }), 2);
// Five targets: one each, head then base.
const five = ["iphone", "ipad", "duo-closed", "duo-open", "android"];
for (const target of five) assert.equal(sessionShare({ maxSessions: undefined, targets: five, target }), 1);
assert.equal(planSideSessions({ share: 1, sides: 2, routes: 16, schemes: ["light", "dark"] }).concurrentSides, false);
assert.deepEqual([maxParallelTargets(undefined, 1), maxParallelTargets(undefined, 2), maxParallelTargets(undefined, 5), maxParallelTargets(undefined, 7)], [1, 2, 5, 5]);
// A starter plan, one simulator per platform: an iOS and an Android target, one session each, both jobs at once.
for (const target of ["iphone", "android"]) assert.equal(sessionShare({ maxSessions: 2, targets: ["iphone", "android"], target }), 1);
assert.equal(maxParallelTargets(2, 2), 2);
// A ceiling below the targets: one session each, and only that many target jobs at a time.
for (const target of five) assert.equal(sessionShare({ maxSessions: 2, targets: five, target }), 1);
assert.equal(maxParallelTargets(2, 5), 2);
assert.equal(maxParallelTargets(16, 4), 4);
});
test("a side splits its appearance passes only when the pass moved out saves at least a minute", () => {
const euxy = { sides: 2, routes: 16, schemes: ["light", "dark"] };
assert.deepEqual(planSideSessions({ share: 2, ...euxy }), { concurrentSides: true, passes: [["light", "dark"]] });
assert.deepEqual(planSideSessions({ share: 4, ...euxy }), { concurrentSides: true, passes: [["light"], ["dark"]] });
assert.deepEqual(planSideSessions({ share: 3, ...euxy }), { concurrentSides: true, passes: [["light", "dark"]] }, "one spare session cannot host a pass for both sides");
assert.deepEqual(planSideSessions({ share: 8, ...euxy, routes: 3 }), { concurrentSides: true, passes: [["light", "dark"]] }, "three routes would save under a minute");
assert.deepEqual(planSideSessions({ share: 8, ...euxy, schemes: ["light"] }), { concurrentSides: true, passes: [["light"]] }, "one appearance never splits");
assert.deepEqual(planSideSessions({ share: 1, ...euxy }), { concurrentSides: false, passes: [["light", "dark"]] }, "one session: the sides take turns");
assert.deepEqual(planSideSessions({ share: 2, sides: 1, routes: 16, schemes: ["light", "dark"] }), { concurrentSides: true, passes: [["light"], ["dark"]] }, "a baseline has one side");
});
test("a side's appearances come from eas config in a child with the capture's environment, read only when a split could pay", async t => {
const root = await mkdtemp(join(tmpdir(), "explore-session-plan-"));
t.after(() => rm(root, { recursive: true, force: true }));
// The app config would leak this process's environment if it were evaluated here.
await writeFile(join(root, "app.config.js"), `module.exports = () => { require("fs").writeFileSync(${JSON.stringify(join(root, "evaluated"))}, "yes"); return {}; };\n`);
const eas = join(root, "eas");
await writeFile(eas, `#!/usr/bin/env node
require("fs").appendFileSync(${JSON.stringify(join(root, "calls"))}, JSON.stringify({ args: process.argv.slice(2), cwd: process.cwd(), env: process.env }) + "\\n");
process.stdout.write(JSON.stringify({ appConfig: { userInterfaceStyle: require("fs").existsSync("light-only") ? "light" : "automatic" }, buildProfile: {} }));
`);
await chmod(eas, 0o755);
const calls = async () => (await readFile(join(root, "calls"), "utf8").catch(() => "")).split("\n").filter(Boolean).map(line => JSON.parse(line));
const config = maxSessions => validateExploreConfig({ app: { bundleId: "com.example.app", easProfile: "explore" }, device: { name: "iPhone 17" },
...(maxSessions === undefined ? {} : { capture: { maxSessions } }) });
const environment = { PATH: process.env.PATH, HOME: process.env.HOME, EXPO_TOKEN: "expo-token", GITHUB_TOKEN: "github-token", ANTHROPIC_API_KEY: "agent-key" };
const side = { share: 2, sides: 1, routes: 16, project: root, easCommand: eas };
assert.equal(await sideAppearancePasses(config(8), environment, { ...side, routes: 3 }), undefined, "three routes would save under a minute");
assert.equal(await sideAppearancePasses(config(1), environment, { ...side, share: 1 }), undefined);
assert.deepEqual(await calls(), [], "no app config is read when no split could pay");
assert.deepEqual(await sideAppearancePasses(config(2), environment, side), [["light"], ["dark"]]);
const [call] = await calls();
assert.deepEqual(call.args, ["config", "--platform", "ios", "--profile", "explore", "--json", "--non-interactive"]);
assert.equal(await realpath(call.cwd), await realpath(root));
assert.equal(call.env.EXPO_TOKEN, "expo-token");
assert.equal(call.env.GITHUB_TOKEN, undefined);
assert.equal(call.env.ANTHROPIC_API_KEY, undefined);
await assert.rejects(readFile(join(root, "evaluated")), "the app config is never evaluated in the review's process");
assert.deepEqual(await sideAppearancePasses(config(undefined), environment, side), [["light"], ["dark"]], "unset, the default ceiling's share splits too");
// Each side reads its own app config: one that declares a single appearance keeps one session.
const lightOnly = join(root, "base"); await mkdir(lightOnly); await writeFile(join(lightOnly, "light-only"), "");
assert.equal(await sideAppearancePasses(config(2), environment, { ...side, project: lightOnly }), undefined);
assert.equal(await sideAppearancePasses(config(2), environment, { ...side, easCommand: join(root, "missing-eas") }), undefined, "an unreadable config keeps one session");
});
+30
View File
@@ -59,3 +59,33 @@ test("a capture that ends without finishing its shared build releases the target
assert.equal(error.code, "CAPTURE_BUILD_FAILED");
assert.match(error.message, /the capture building it stopped \(CAPTURE_WORKER_EXITED\)/);
});
test("a waiter is told the build is repacking only when its builder says so, not while a native build may run", async t => {
const directory = await scratch(t);
let started, repack, finish;
const building = new Promise(resolve => { started = resolve; });
const owner = sharedBuild({ directory, key, appName: "App.app", build: (_keepApp, repacking) => new Promise(resolve => {
started(); repack = repacking; finish = () => resolve({ buildId: "native" }); }) });
await building;
const told = [];
const waiter = sharedBuild({ directory, key, appName: "App.app", build: async () => assert.fail("only the owner builds"),
waiting: () => told.push("waiting"), repacking: () => told.push("repacking") });
// The owner is still in its native build: two polls go by without the waiter's session starting.
await new Promise(resolve => setTimeout(resolve, 4_500));
assert.deepEqual(told, ["waiting"]);
await repack();
await new Promise(resolve => setTimeout(resolve, 2_500));
assert.deepEqual(told, ["waiting", "repacking"]);
finish();
await owner; await waiter;
assert.deepEqual(told, ["waiting", "repacking"]);
});
test("a waiter whose build finished before it looked is told it is repacking once, with the result", async t => {
const directory = await scratch(t);
await sharedBuild({ directory, key, appName: "App.app", build: async () => ({ buildId: "native" }) });
const told = [];
assert.deepEqual(await sharedBuild({ directory, key, appName: "App.app", build: async () => assert.fail("only the owner builds"),
waiting: () => told.push("waiting"), repacking: () => told.push("repacking") }), { buildId: "native" });
assert.deepEqual(told, ["waiting", "repacking"]);
});
+10
View File
@@ -93,3 +93,13 @@ test("doctor names a workflow that sets the app directory in one job but not ano
assert.deepEqual(issues.map(issue => issue.code), ["WORKFLOW_APP_DIR_MISMATCH"]);
assert.match(issues[0].message, /APP_PROJECT_DIR is \. in one of its jobs/);
});
test("ci:after-capture stops the sessions a side's appearance passes recorded, not only the side's own", async t => {
const output = await mkdtemp(join(tmpdir(), "explore-steps-sessions-"));
t.after(() => rm(output, { recursive: true, force: true }));
// Unreadable ids make the cleanup report each file it was handed without calling EAS.
for (const name of ["head-session-dark", "base-session-dark", "baseline-session-light"]) await writeFile(join(output, name), "not a session id\n");
await writeFile(join(output, "head-session-sepia"), "not a session id\n");
const result = await cleanupAfterCapture({ output, checkouts: [], environment: { RUNNER_TEMP: output, EXPO_TOKEN: "token" } });
assert.deepEqual(result.failures.map(failure => failure.message), Array(3).fill("Refusing an invalid session ID."));
});