mirror of
https://github.com/expo/explore.git
synced 2026-11-15 17:32:23 -10:00
Compare commits
7
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5845e7cfa9 | ||
|
|
d992899292 | ||
|
|
8e86e6a14e | ||
|
|
ebbcbb62c4 | ||
|
|
29f1f0721f | ||
|
|
f620577e30 | ||
|
|
6c927b1f22 |
No files matched your search
+79
-6
@@ -14,7 +14,7 @@ Source of truth: `templates/explore-review.yml`, `templates/explore-baseline.yml
|
||||
| Pipeline | `capture:review` → `runCiReview` | The same `runCiReview`, called directly |
|
||||
| EAS credentials | `EXPO_TOKEN` secret | Local `eas login` session or `EXPO_TOKEN` |
|
||||
| Model credentials (agent mode) | `ANTHROPIC_API_KEY` / `OPENAI_API_KEY` | Claude Code or Codex subscription login; API keys are removed from the subprocess |
|
||||
| Base cache | A baseline workflow artifact for the exact base SHA, when one exists | None; both sides are captured |
|
||||
| Base cache | A baseline workflow artifact for the exact base SHA, when one exists, or the baseline run still capturing it, waited for alongside head | None; both sides are captured |
|
||||
| Output | Report artifact + sticky PR comment with **Open** / **zip** | `runs/<id>/artifacts/report.zip` + `PR-comment.md`, opened with `ci:local:view` |
|
||||
|
||||
## 1. Installing Explore in an app repository
|
||||
@@ -49,8 +49,8 @@ Each workflow starts with a `targets` job: it installs the pinned CLI and runs `
|
||||
| | checkout `app-base`, `app-head` | Checks out the exact base and head SHAs with full history and without persisted credentials. |
|
||||
| | Verify deterministic source planning | `source:verify` builds the capture plan from source alone in two fresh processes and compares the bytes. Runs before any build or simulator work and needs no credentials. |
|
||||
| | Install app dependencies | `npm ci --ignore-scripts` in both checkouts (`APP_INSTALL_DIR` for monorepos). |
|
||||
| | Find / Download optional exact-base evidence | `ci:find-baseline` looks for a finished run's `explore-baseline-<base sha>-<target>` artifact (a run another target's baseline failed in counts: a baseline job uploads its artifact only after its own steps passed) (target `ios` also accepts the older unsuffixed `explore-baseline-<base sha>`) from the baseline workflow on the default branch and downloads it as `EXPLORE_CAPTURE_CACHE`. Missing evidence is not an error. |
|
||||
| | Capture review | `capture:review` (§3). `EXPO_TOKEN`, the provider key and the app's `{{secret:NAME}}` values reach only this step. The sticky PR comment is updated while it runs. It writes `preview=<hosting>` to the step outputs, which decides whether the deploy job runs. |
|
||||
| | Find / Download optional exact-base evidence | `ci:find-baseline` looks for a finished run's `explore-baseline-<base sha>-<target>` artifact (a run another target's baseline failed in counts: a baseline job uploads its artifact only after its own steps passed) (target `ios` also accepts the older unsuffixed `explore-baseline-<base sha>`) from the baseline workflow on the default branch and downloads it as `EXPLORE_CAPTURE_CACHE`. With none, it names a baseline run of the base commit still queued or capturing (`pending`), held to the same checks: the baseline workflow, the default branch of this repository, a `push` or `workflow_dispatch`, exactly the base SHA; nothing from the PR head selects it. Missing evidence and a failed lookup are not errors. |
|
||||
| | Capture review | `capture:review` (§3). With `EXPLORE_BASELINE_RUN` (the find step's `pending`) it waits for that run's baseline of this target (§3.1). `EXPO_TOKEN`, the provider key and the app's `{{secret:NAME}}` values reach only this step. The sticky PR comment is updated while it runs. It writes `preview=<hosting>` to the step outputs, which decides whether the deploy job runs. |
|
||||
| | Stop capture sessions and check the checkouts | `ci:after-capture`, whenever the PR was resolved, success or not: stops the EAS Simulator sessions named in `head-session` / `base-session`, nothing else, then fails with `CHECKOUT_MODIFIED` if capture modified an app checkout. |
|
||||
| | Upload captured review / diagnostics | `explore-capture-<run>-<attempt>-<target>` feeds the report job. On failure `explore-diagnostics-<run>-<attempt>-<target>` keeps plans, exports and agent diagnostics for 7 days. |
|
||||
| report (matrix: target) | Download the capture, install Explore | Runs without Expo or model credentials, after every capture job, whether or not each succeeded (`!cancelled()`). Each leg downloads its own target's capture by an exact-name pattern: a target whose capture failed uploaded none, so its leg packages and publishes nothing, and the other targets' reports still reach the comment. |
|
||||
@@ -68,7 +68,7 @@ Validate baseline source → Install Explore → Verify deterministic source pla
|
||||
|
||||
### 2.4 `explore-compare.yml`
|
||||
|
||||
The same jobs as a PR review, for any `base` and `head` without a pull request. The `targets` job resolves both refs once with `ci:compare-refs` (a tag wins over a branch of the same name; the same commit twice fails with `COMPARE_SAME_COMMIT`) and passes the comparison to the capture jobs, which run `capture:review --compare` with no GitHub requests. The optional `scope` input overrides `exploration.scope` for that run; `expires-in` is accepted and ignored. The report job's `ci:publish` writes each target's **Open** and **zip** links to the run summary, and the deploy job adds the hosted link there; nothing comments on a pull request.
|
||||
The same jobs as a PR review, for any `base` and `head` without a pull request. The `targets` job resolves both refs once with `ci:compare-refs` (a tag wins over a branch of the same name; the same commit twice fails with `COMPARE_SAME_COMMIT`) and passes the comparison to the capture jobs, which run `capture:review --compare`; its only GitHub requests are the wait for a baseline run still capturing the base (§3.1), with the job's `actions: read` token. The optional `scope` input overrides `exploration.scope` for that run; `expires-in` is accepted and ignored. The report job's `ci:publish` writes each target's **Open** and **zip** links to the run summary, and the deploy job adds the hosted link there; nothing comments on a pull request.
|
||||
|
||||
## 3. Inside `capture:review` (`runCiReview`)
|
||||
|
||||
@@ -77,7 +77,7 @@ The same jobs as a PR review, for any `base` and `head` without a pull request.
|
||||
| 1 | `plan` | `buildVerifiedPlan`: snapshot both sources, compute the two-dot diff, run `plan-worker` twice with reversed input order and compare the canonical plans byte for byte. Summarises `sourceChange` per surface into `execution-plan.json`; the PR comment shows the planned capture count and warnings. | `SOURCE_ANALYSIS_FAILED`, inconsistent plan |
|
||||
| 2 | `preflight` | Checks `EXPO_TOKEN`. `prepareCiExploration`: nothing for deterministic mode; for agent mode, confirms the instructions file is committed, creates the provider session and makes one short tool-free model request. | `AGENT_API_KEY_MISSING`, `AGENT_INSTRUCTIONS_UNCOMMITTED`, `AGENT_MODEL_NOT_FOUND` |
|
||||
| 3 | `capture-head` | `runCiMap(head)` (§4). Records head's runtime profile and build. | `CAPTURE_*`, `SETUP_*`, `EAS_*` |
|
||||
| 4 | `capture-base` | Tries the cache first: a baseline is reused only when its recipe matches (tools, config, instructions, device, profile, observer hash, runtime profile). Only routes missing from the cache run through `runCiMap(base)`. | same |
|
||||
| 4 | `capture-base` | Tries the cache first: a baseline is reused only when its recipe matches (tools, config, instructions, device, profile, observer hash, runtime profile). Only routes missing from the cache run through `runCiMap(base)`. With no cache but a baseline run still capturing the base, base starts only once the wait for it ends (§3.1). | same |
|
||||
| 5 | `compare` | `assessPublicEvidence` compares pixels and accessibility for route pairs verified on both sides; `prepareCaptureReview` assigns verdicts (§6). | |
|
||||
| 6 | `capture-validation` | Every selected route was captured on both sides. | `CAPTURE_INCOMPLETE` |
|
||||
| 7 | `comparison-validation` | Every pair was compared visually and for accessibility. | `COMPARISON_INCOMPLETE` |
|
||||
@@ -86,6 +86,17 @@ The same jobs as a PR review, for any `base` and `head` without a pull request.
|
||||
|
||||
A failure at any stage is recorded in `state.failure`, a partial report is written when a plan exists, and the PR comment switches to the failed state.
|
||||
|
||||
### 3.1 Waiting for a baseline still being captured
|
||||
|
||||
Right after a merge, the next pull request's review starts while the baseline workflow is still capturing the new base: in brentvatne/euxy's review run 39348956443 it had started 15 s before, and every target captured its base too (10–14 minutes per target instead of about 6, on twice the EAS Simulator sessions). A baseline job takes 6–8 minutes and head capture 5–8, so `capture:review` waits for the baseline beside head instead of before it (`src/ci/baseline-wait.ts`):
|
||||
|
||||
1. The wait starts after preflight, only when `EXPLORE_BASELINE_RUN` names a run, no downloaded baseline matches, base has routes to capture, and `capture.baselineWaitMinutes` (default 15, 0 never waits) is not 0. Head starts capturing at once.
|
||||
2. Every 20 s it reads the run (checked again by the same trust rules as `ci:find-baseline`), this target's artifact (`explore-baseline-<sha>-<target>`) in that run, and the run's jobs, logging `waiting for main's baseline run <id> of <sha> (<target>: in progress, 3 min)` once a minute or when the state changes. Each read retries GitHub's 5xx and network failures (`retryGitHubLookup`).
|
||||
3. When the artifact is listed, it is downloaded in-process (the token goes to GitHub, never to the storage URL it redirects to), unpacked into `<output>/baseline-download` within the cache's size limits, and restored as a downloaded baseline once head has its runtime profile. Another target's failed job does not matter; each target's job uploads its own artifact.
|
||||
4. The wait ends without a baseline when this target's job fails, is cancelled or is skipped, the run ends without listing the artifact for a minute, the time runs out, GitHub refuses the token (`actions: read` missing) or keeps failing, or head fails. Base then captures at once, beside head when the sides capture concurrently. The wait never fails the review.
|
||||
|
||||
Head capture is never delayed. When the baseline arrives while head captures, base costs only its restore; when it does not, base starts the moment the wait gives up, so a failed baseline delays base only by the time it took to fail. Sessions are planned as without a wait: with a share of two or more, head keeps its half while base waits, so base can start in the rest; with one session, base follows head as before, and the wait may go on past head, up to its limit.
|
||||
|
||||
## 4. Capturing one side (`runCiMap`)
|
||||
|
||||
| # | Stage | Work |
|
||||
@@ -93,11 +104,73 @@ A failure at any stage is recorded in `state.failure`, a partial report is writt
|
||||
| 1 | cache | If a reuse directory exists, its bundle and map are validated and copied. With no routes left, skip straight to export. |
|
||||
| 2 | `configuration` | `eas config --profile` confirms the app identity and scheme, `ios.simulator: true` and the bundle ID; `eas simulator:availability`. |
|
||||
| 3 | `build` | `eas fingerprint:generate`; a finished simulator build with the same fingerprint is reused, otherwise `eas build --wait` (up to 45 minutes). |
|
||||
| 4 | `instrumentation` | A hosted EAS `repack` workflow job (`eas workflow:run --wait --json` from a staged copy of the checkout) rebundles the current JavaScript with a read-only navigation observer (the same observer on both sides), and the repacked app is downloaded and installed into the session; `capture.repack: local` (the default with `capture.simulator: local`) runs `@expo/repack-app` on the runner, and its app is installed without an upload. Generated files stay outside the checkout. |
|
||||
| 4 | `instrumentation` | A hosted EAS `repack` workflow job (`eas workflow:run --json` from a staged copy of the checkout, then its run read until it ends) rebundles the current JavaScript with a read-only navigation observer (the same observer on both sides), and the repacked app is downloaded and installed into the session; `capture.repack: local` (the default with `capture.simulator: local`) runs `@expo/repack-app` on the runner, and its app is installed without an upload. Generated files stay outside the checkout. Preparing the app has one 30-minute budget, a failed hosted repack starts again, and a run attempt's matrix jobs make one repack of each commit between them (§4.1). |
|
||||
| 5 | `session` | `eas simulator:start`; the session ID is written to `<side>-session`; the app is installed. |
|
||||
| 6 | route capture | Mode-specific (§5). `agent-device` drives the simulator and records a screenshot plus the accessibility tree. A capture is accepted only when the observer's reported destination matches the requested route. |
|
||||
| 7 | `export` / `cleanup` | Validate the map, write `<side>-export/` (`bundle.json`, `execution.json`, PNGs, `capture-diagnostics/`), stop the session. |
|
||||
|
||||
### 4.1 The hosted repack: one budget, retries, and one repack per commit for a run's jobs
|
||||
|
||||
`prepareHostedRepack` (`src/capture/eas-build.ts`) runs the repack job. Three things keep it from failing a target, running long, or repeating work.
|
||||
|
||||
**One budget.** Preparing the app gets 30 minutes (`HOSTED_REPACK_BUDGET_MS`), counted from the moment the EAS session starts beside the repack. That covers waiting for another job's repack, following it, and every repack this job starts, including each run's wait. Before this, each run had a 30-minute wait of its own, so a share wait plus retries could prepare for over an hour. The rules:
|
||||
|
||||
- A repack starts only when at least 5 minutes of the budget are left.
|
||||
- A run still going at the end of the budget is cancelled, and the target fails with `EAS_REPACK_FAILED` (`failure: timeout`).
|
||||
- The session was asked for the capture's minutes plus a 20-minute allowance (`SESSION_PREPARATION_MINUTES`). If the app took longer than the allowance, the session is stopped and a fresh one started before the install, so the capture keeps all its minutes. This counts as one of the session's replacements.
|
||||
|
||||
**Explore waits on the run itself.** `workflow:run` runs without `--wait`. Explore reads the run's id from the JSON, or from the "See logs:" link if the JSON is missing, then reads `eas workflow:view <id> --json` every 20 s. It does not use `--wait` because eas-cli 25.2.0's `--wait` exits with code 13 after six failed status reads and leaves the run going, and Explore then started another one beside it. Now:
|
||||
|
||||
- A run whose state stays unreadable six times in a row is cancelled (`eas workflow:cancel`). So is a run that outlasts the budget, or whose capture is stopped. None of them is replaced.
|
||||
- If the start times out, `defaultRunner` keeps what it printed. When that names the run, Explore waits on it. When it doesn't, Explore starts no other, because one may be running.
|
||||
- `capture:review` and `capture:baseline` turn the runner's SIGINT and SIGTERM, which GitHub sends to a cancelled job, into an abort. The capture then stops its session and cancels the run it waits on. A second signal quits at once.
|
||||
|
||||
**A failed repack starts again.** In brentvatne/euxy#176 (review run 39506405687), EAS ended a repack's build while its "Upload artifact" step was saving the app ("Turtle build is not in progress", reported as `Failed to upload application archive.`), and the target failed with `EAS_REPACK_FAILED`. Only a run EAS reports finished can be followed by another. `hostedRepackFailure` reads the build's error, the job's errors, the failed step's logs (`eas workflow:logs <job> --json`) and the CLI's output, in this order:
|
||||
|
||||
| Failure | Read from | Fresh repacks |
|
||||
| --- | --- | --- |
|
||||
| `canceled` | a run someone cancelled | 0 |
|
||||
| `configuration` | `Not logged in`, `Unauthorized`, `not authorized`, `E401`/`E403`, `401 Unauthorized`/`403 Forbidden` (a private registry's included) | 0 |
|
||||
| `infrastructure` | a file of EAS's worker missing (`'/opt/eas/…'`) | 2 |
|
||||
| `app` | the `Repack` / `Repack app` step failed, or Metro, Babel or Expo named the app's code or files: `Unable to resolve`, `SyntaxError`, `TransformError`, `Cannot find module`, `could not read ./…` | 0 |
|
||||
| `timeout` | `EAS_BUILD_TIMEOUT_ERROR`: a rerun would run out its time again (a bundling hang, say) | 0 |
|
||||
| `infrastructure` | a failed `Spin up…`, `Set up…`, `Download…` or `Upload…` step; EAS's or eas-cli's exact words (`Turtle build is not in progress`, `Failed to upload application archive`, `SERVER_ERROR`, `Failed to upload project sources`, `Failed to start the workflow with the API`, `Unexpected response from server (5xx)`, `ECONNRESET`, `ETIMEDOUT`, `ENOTFOUND`, `EAI_AGAIN`, `socket hang up`, `fetch failed`) | 2 |
|
||||
| `unknown` | nothing above | 1 |
|
||||
|
||||
Each new start logs the failed run's link and reason, for example `the hosted EAS repack https://expo.dev/…/workflows/<id> failed (…); starting another repack (1/2)`. The session is started once, beside the first repack, and installs whichever repack succeeds.
|
||||
|
||||
**One repack per commit for a run attempt's matrix jobs.** Captures that run in one process already share a build in a directory (`src/capture/shared-build.ts`): the appearance passes of one side, which run in worker threads, and the targets of a local review. The first to claim a key builds and repacks, and the others wait for its `result.json` ("using the build another capture of this run is preparing for this commit"). A GitHub matrix job is a runner of its own, with no directory in common, so in #176 the four iOS jobs each logged `reusing EAS build a81d5d80 … repacking it with this commit's JavaScript` and made four identical repacks. The jobs of one run attempt meet on EAS instead:
|
||||
|
||||
1. `hostedRepackPeers` lists, in config order, the targets that repack the same app on EAS: one platform, EAS profile and bundle id. `capture:review`, `capture:review --compare` and `capture:baseline` pass them on.
|
||||
- A capture gets a `RepackShare` when it has two or more peers and `GITHUB_RUN_ID`, `GITHUB_RUN_ATTEMPT` and `GITHUB_REPOSITORY` are set. Its scope is the run attempt, its leader is the first peer, and its rank is its place among the peers.
|
||||
- A single target, a local review, `capture.repack: local`, and a native build this job made itself behave as before. Nobody else's key can match a native build this job made.
|
||||
2. The build to repack is the newest finished build with the fingerprint that Explore did not repack itself (`existingBuild`, from 20 builds). If repacks carry their source's fingerprint, every job still picks the same build.
|
||||
3. Once the copy is staged, the capture takes a key (`hostedRepackShareKey`) from:
|
||||
- the run attempt, the platform and the native build id;
|
||||
- the EAS profile and environment name;
|
||||
- a snapshot of the environment's variables: `eas env:list --format long` for the project and the account, which names each variable with its value (masked when sensitive or secret) and when it was last updated;
|
||||
- the Explore release;
|
||||
- a digest of every staged file with its path, executable bit and content (`stagedProjectDigest`), which covers the commit's JavaScript and the generated observer.
|
||||
|
||||
If either variable list fails, the capture repacks without sharing. Its own repack's message ends with `[explore share <key>]`.
|
||||
4. It reads the platform's 50 most recent builds (`eas build:list --limit 50 --json`, a single page) for one whose message ends with that marker, and joins any it finds, whichever job started it. A finished one is downloaded instead of repacking. A queued or running one is then read by its id (`eas build:view <id> --json`) every 30 s, so newer builds on a busy project cannot push it off the list.
|
||||
5. How long a capture looks is staggered (`sharedRepackWaitMs`). The leader looks once and repacks if it finds nothing. The next target looks every 15 s for 3 minutes, and each target after it a minute longer. When the leader makes no repack (its baseline covered every route, or `max-parallel` kept it out of the first jobs), the next target's repack appears before the one after it would start another, so they all join one. In a re-run attempt, which runs only the jobs it re-runs, the stagger starts at once (1, 2, 3 minutes).
|
||||
6. When the repack it follows fails, is cancelled or loses the key, the capture looks for another with the same key for 60 s, staggered the same way, before it repacks itself. That catches the leader's retry, which starts at once (#176).
|
||||
7. It also repacks itself, with the retries above, when nothing appears in its wait, or when EAS fails to answer three times in a row. Looking stops while the budget still has 5 minutes for a repack of its own. A followed repack that does not finish within the budget fails the target. No capture waits on a job that cannot answer, and none waits past its budget.
|
||||
|
||||
Only a build whose marker is this capture's own key is used, and every input to the key is what this capture would itself have uploaded and built with. So two jobs share a repack only when their uploads, native build, profile, environment variables and Explore release are the same, in the same run attempt. The run attempt is in the scope, and a review's or a first attempt's repacks are never used by another run or attempt. That covers the baseline workflow, whose artifacts later reviews reuse, and "Re-run failed jobs" after an `EXPO_PUBLIC_*` value was fixed, which rebundles with the new value.
|
||||
|
||||
What the key does not protect against: every input to it is public or predictable, and the pull request's head code runs in the capture job with `EXPO_TOKEN` (`app.config.*` under `eas config` and `eas fingerprint:generate`, and the app's build code on EAS). Head code could therefore make a build whose message carries the base side's marker, and base captures of the same run attempt in other matrix jobs would install it. That is within the trust the capture already gives EAS: the same credentials could publish a simulator build with a reused fingerprint, which every capture already reuses by fingerprint (the `build` stage above), and head code in one job can already tamper with that job's own base side on the runner. The run attempt in the scope keeps such a build out of every other run, including the baseline. Requiring the build's `initiatingActor` to be this job's own credentials would not exclude head code, which runs with the same token. Explore cannot tell offline which actor EAS records for a workflow's repack build, so it does not check it.
|
||||
|
||||
Cost, per side and job:
|
||||
|
||||
- The leader makes 1 `build:list` and 2 `env:list`.
|
||||
- A follower makes up to 13 `build:list` while the next target waits, plus 4 more per target after it.
|
||||
- About 2 `build:view` a minute while it follows a repack: about 10 for a 5-minute repack.
|
||||
- Each repack this job runs reads its run every 20 s: about 15 `workflow:view` for 5 minutes, and one `workflow:logs` if it fails.
|
||||
|
||||
In #176's run that is about 20 calls per side for each of the three iOS targets that would otherwise have repacked, in place of three repacks. A follower whose leader and earlier peers make no repack of this side loses its wait, 3 to 5 minutes, before it repacks itself. Native builds are not shared across jobs: a fingerprint EAS has no build for is built by every job that needs it.
|
||||
|
||||
## 5. Deterministic and agent modes
|
||||
|
||||
| | Deterministic | Agent |
|
||||
|
||||
@@ -492,7 +492,7 @@ jobs:
|
||||
- The run summary links each target's report (**Open** and **zip**, like a PR comment) and, with hosting, the hosted report.
|
||||
- `setup --workflows` installs it from 0.19.0; `npx @expo/explore@latest setup --workflows` adds it to an existing setup. The caller grants `contents: read` and `actions: read`; `secrets: inherit` passes `EXPO_TOKEN` and any app-test secrets.
|
||||
- Call it once per workflow run. A called workflow runs inside its caller's run, so a second call in the same run would reuse the first one's artifact names; use separate workflows instead.
|
||||
- When the baseline workflow captured `base` in the last 30 days, with the same config and Explore version, that capture is reused and only `head` is captured. The report artifact is kept 30 days; a hosted link lasts as long as its EAS Hosting deployment (an aliased one until the alias moves).
|
||||
- When the baseline workflow captured `base` in the last 30 days, with the same config and Explore version, that capture is reused and only `head` is captured. When it is still capturing `base` (right after a merge), the review waits for it while `head` captures, up to `capture.baselineWaitMinutes` (15 by default). The report artifact is kept 30 days; a hosted link lasts as long as its EAS Hosting deployment (an aliased one until the alias moves).
|
||||
|
||||
</details>
|
||||
|
||||
|
||||
@@ -105,6 +105,7 @@ capture:
|
||||
maxActions: 500
|
||||
# maxSessions: 5 # the most EAS Simulator sessions one review runs at once, across all its targets and sides (default: 5, a typical plan's concurrent simulator limit). Each target's job gets floor(maxSessions / targets) sessions, at least one, and sessions left over go to foldable targets (pose): one target gets 5 (head and base at once, light and dark in their own sessions when that pays), two get 2 each, five get 1 each (head, then base). Below the number of targets, only that many target jobs run at once (the workflow's max-parallel, and a local review's targets), one session each. A side spreads its light and dark passes over separate sessions only when that saves it at least a minute, so the ceiling is a limit, not a target: a 3-route target stays on two sessions. euxy's four targets: 8 is two per target, 16 captures each appearance in its own session (measured: an iPhone side 6:12 → 4:26, one repack shared by its sessions, ~3.5 session-minutes more per side).
|
||||
# On a starter plan (one simulator per platform), set maxSessions to the plan's limit, e.g. 2 for an iOS and an Android target.
|
||||
# baselineWaitMinutes: 15 # a review that starts while the default branch's baseline run of its base commit is still capturing (right after a merge) waits up to this long for that target's baseline instead of capturing the base too (default 15; 0 never waits). Head captures meanwhile; base waits only where it would start, and captures itself if the baseline job fails or is cancelled, or the time runs out.
|
||||
repack: eas # eas (hosted workflow job; the default on EAS Simulator) | local (repack-app on the runner; the default with simulator: local)
|
||||
appearance: auto # auto (follow the app's userInterfaceStyle: automatic captures light and dark) | dark | light | [dark, light]
|
||||
launch: screenshot # screenshot (relaunch the app per appearance and keep its splash and first settled frame, default) | video (also keep a launch recording in the report) | off
|
||||
|
||||
@@ -111,6 +111,7 @@ capture:
|
||||
maxActions: 500
|
||||
# maxSessions: 5 # the most EAS Simulator sessions one review runs at once, across all its targets and sides (default: 5, a typical plan's concurrent simulator limit). Each target's job gets floor(maxSessions / targets) sessions, at least one, and sessions left over go to foldable targets (pose): one target gets 5 (head and base at once, light and dark in their own sessions when that pays), two get 2 each, five get 1 each (head, then base). Below the number of targets, only that many target jobs run at once (the workflow's max-parallel, and a local review's targets), one session each. A side spreads its light and dark passes over separate sessions only when that saves it at least a minute, so the ceiling is a limit, not a target: a 3-route target stays on two sessions. euxy's four targets: 8 is two per target, 16 captures each appearance in its own session (measured: an iPhone side 6:12 → 4:26, one repack shared by its sessions, ~3.5 session-minutes more per side).
|
||||
# On a starter plan (one simulator per platform), set maxSessions to the plan's limit, e.g. 2 for an iOS and an Android target.
|
||||
# baselineWaitMinutes: 15 # a review that starts while the default branch's baseline run of its base commit is still capturing (right after a merge) waits up to this long for that target's baseline instead of capturing the base too (default 15; 0 never waits). Head captures meanwhile; base waits only where it would start, and captures itself if the baseline job fails or is cancelled, or the time runs out.
|
||||
repack: eas # eas (hosted EAS workflow job, the default on EAS Simulator) | local (repack-app on the runner, the default with simulator: local; --repack overrides it for one run)
|
||||
appearance: auto # auto (follow the app's userInterfaceStyle: automatic captures light and dark) | dark | light | [dark, light]
|
||||
launch: screenshot # screenshot (relaunch the app per appearance and keep its splash and first settled frame, default) | video (also keep a launch recording in the report) | off
|
||||
@@ -391,7 +392,7 @@ The review workflow resolves the current PR through GitHub, checks out its exact
|
||||
| Review packaging | The bundle contains metadata and assets, not a standalone HTML page |
|
||||
| Publication | The artifact is uploaded before current-PR identity is checked again and the **Open** and **zip** links are posted |
|
||||
|
||||
The baseline workflow runs on default-branch pushes or manual default-branch dispatch. It captures one complete commit without needing a PR. Reports and baselines are retained for 30 days; intermediate evidence and diagnostics for 7 days.
|
||||
The baseline workflow runs on default-branch pushes or manual default-branch dispatch. It captures one complete commit without needing a PR. A review that starts while the baseline of its base commit is still being captured, as the next pull request's review does right after a merge, waits for it (`capture.baselineWaitMinutes`, 15 minutes by default) while head captures, instead of capturing the base too. Reports and baselines are retained for 30 days; intermediate evidence and diagnostics for 7 days.
|
||||
|
||||
`exploration.scope: changes` includes source-affected routes and the routes one navigation step before or after them, so a reviewer sees how each change is reached and where it leads. The context is not transitive: in most apps tabs, guards and Home connect every screen, so all ancestors and descendants would be the whole app. A layout or screen guard that sends every signed-out screen to sign-in is not a step. Forwarding routes have no screenshot of their own, so their redirect chains are followed to the first screen. Unresolved navigation still selects every route. `all` captures the full inventory.
|
||||
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "@expo/explore",
|
||||
"version": "0.20.3",
|
||||
"version": "0.20.6",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "@expo/explore",
|
||||
"version": "0.20.3",
|
||||
"version": "0.20.6",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@expo/repack-app": "0.10.1",
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@expo/explore",
|
||||
"version": "0.20.3",
|
||||
"version": "0.20.6",
|
||||
"description": "Source-driven mobile app maps, deterministic CI capture, and visual reviews",
|
||||
"license": "MIT",
|
||||
"type": "module",
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import type { CaptureOutcome } from "../contracts/capture.js";
|
||||
import type { CaptureBundle, CaptureOutcome } from "../contracts/capture.js";
|
||||
|
||||
export type ColorScheme = "light" | "dark";
|
||||
/** `auto` follows the app's `userInterfaceStyle`; a list captures exactly those schemes, first one primary. */
|
||||
@@ -118,3 +118,11 @@ export function appearancePairs(routeId: string, headSchemes: readonly ColorSche
|
||||
baseId: baseSchemes.includes(scheme) ? schemeScreenId(routeId, scheme === baseSchemes[0] ? undefined : scheme) : null,
|
||||
}));
|
||||
}
|
||||
|
||||
/** A `<id>@<scheme>` outcome recording that its whole appearance pass failed (`sessionFailures`), whatever the primary did. */
|
||||
export function failedPassOutcome(bundle: Pick<CaptureBundle, "outcomes" | "sessionFailures">, id: string): boolean {
|
||||
const { scheme } = routeIdOfScreen(id);
|
||||
const own = bundle.outcomes[id];
|
||||
return scheme !== undefined && own?.status === "failed" && (bundle.sessionFailures ?? []).some(failure => failure.scheme === scheme
|
||||
&& failure.error.code === own.error?.code && failure.error.message === own.error?.message);
|
||||
}
|
||||
@@ -92,7 +92,9 @@ export function startArgentBridgeRunner(options: {
|
||||
stop(`the Argent bridge thread is unavailable (${text}); running the bridge per command`);
|
||||
return options.fallback(request);
|
||||
}
|
||||
const result = JSON.parse(text) as { exitCode: number; stdout: string };
|
||||
const result = JSON.parse(text) as { exitCode: number; stdout: string; notes?: string[] };
|
||||
// The thread cannot reach the capture's log while this one waits for it, so its lines come with the answer.
|
||||
for (const note of result.notes ?? []) options.progress?.(note);
|
||||
return commandOutcome(request, { status: result.exitCode, stdout: result.stdout, stderr: "" });
|
||||
};
|
||||
return { runner, get active() { return worker !== undefined; }, close: () => { if (worker) { void worker.terminate(); worker = undefined; } } };
|
||||
|
||||
@@ -61,6 +61,20 @@ async function materializedFile(data: unknown): Promise<string | undefined> {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* An error's message with its cause's, as Argent's client words a failed fetch: "fetch failed: connect ECONNREFUSED …".
|
||||
* A host that resolves to several addresses (localhost: ::1 and 127.0.0.1) fails with an AggregateError whose message is
|
||||
* empty, so the causes' codes are named too.
|
||||
*/
|
||||
export function failureText(error: unknown): string {
|
||||
if (!(error instanceof Error)) return String(error);
|
||||
const cause = error.cause;
|
||||
if (!(cause instanceof Error)) return error.message;
|
||||
const codes = [cause, ...(cause instanceof AggregateError ? cause.errors : [])]
|
||||
.map(item => (item as { code?: unknown } | undefined)?.code).filter((code): code is string => typeof code === "string" && !cause.message.includes(code));
|
||||
return [`${error.message}:`, cause.message, ...(codes.length ? [`(${[...new Set(codes)].join(", ")})`] : [])].filter(Boolean).join(" ");
|
||||
}
|
||||
|
||||
async function loadClient(environment: NodeJS.ProcessEnv, clientModule?: string): Promise<ArgentClient> {
|
||||
const module = clientModule ?? join(dirname(await ensureController("argent", { environment })), "client.js");
|
||||
const loaded = await import(pathToFileURL(module).href) as { createArgentClient?: () => ArgentClient };
|
||||
@@ -74,8 +88,9 @@ export function argentClientCall(client: ArgentClient, signal: () => AbortSignal
|
||||
const active = signal();
|
||||
if (active.aborted) throw new ArgentCallError(`argent run ${tool} was not sent: the command was given up.`, false);
|
||||
let answer: { data: unknown };
|
||||
try { answer = await client.callTool(tool, args, { signal: AbortSignal.any([active, AbortSignal.timeout(ARGENT_CALL_TIMEOUT_MS)]) }); }
|
||||
catch (error) { throw new ArgentCallError(`argent run ${tool} failed: ${(error instanceof Error ? error.message : String(error)).slice(0, 800)}`, true); }
|
||||
const timeoutMs = Math.max(1, Math.ceil(Math.min(ARGENT_CALL_TIMEOUT_MS, options?.timeoutMs ?? ARGENT_CALL_TIMEOUT_MS)));
|
||||
try { answer = await client.callTool(tool, args, { signal: AbortSignal.any([active, AbortSignal.timeout(timeoutMs)]) }); }
|
||||
catch (error) { throw new ArgentCallError(`argent run ${tool} failed: ${failureText(error).slice(0, 800)}`, true); }
|
||||
if (!options?.out) return answer.data;
|
||||
const file = await materializedFile(answer.data);
|
||||
if (!file) {
|
||||
|
||||
@@ -57,9 +57,23 @@ function summarize(tool: string, value: unknown): unknown {
|
||||
screen: tree.screen, nodes, roots: (tree.roots ?? []).map(root => [root.type ?? null, root.bundleId ?? null, (root.children ?? []).length]) };
|
||||
}
|
||||
|
||||
/**
|
||||
* What a failed `argent run` printed, for its error: the last few lines, and ahead of them the line saying the
|
||||
* tool-server is not serving its tools yet, which the bridge waits out (argent-driver's TOOLS_NOT_SERVED) and which a
|
||||
* stack trace can push out of the last lines.
|
||||
*/
|
||||
export function argentRunFailureDetail(stderr: string, stdout: string): string {
|
||||
const lines = `${stderr}\n${stdout}`.trim().split("\n").filter(Boolean);
|
||||
const notServed = lines.find(line => /GET \/tools failed: \d{3}|ECONNREFUSED/.test(line));
|
||||
return [...new Set([...(notServed ? [notServed] : []), ...lines.slice(-4)])].join(" | ").slice(0, 800);
|
||||
}
|
||||
|
||||
/** A command's answer, with the lines it has for the capture's log (`notes`), which its host prints. */
|
||||
export type ArgentBridgeResult = BridgeResult & { notes?: string[] };
|
||||
|
||||
export type ArgentBridge = {
|
||||
/** Runs one agent-device-shaped command against the session. */
|
||||
run(argv: readonly string[]): Promise<BridgeResult>;
|
||||
run(argv: readonly string[]): Promise<ArgentBridgeResult>;
|
||||
};
|
||||
|
||||
/**
|
||||
@@ -77,8 +91,10 @@ export function createArgentBridge(settings: ArgentBridgeSettings, call: ArgentC
|
||||
await appendFile(logPath, `${JSON.stringify({ at: new Date().toISOString(), host, command: argv[0], ...entry })}\n`, { mode: 0o600 }).catch(() => undefined);
|
||||
};
|
||||
const state = await readState(statePath);
|
||||
const notes: string[] = [];
|
||||
const result = await runBridgeCommand(argv, {
|
||||
state, platform, version: ARGENT_VERSION,
|
||||
progress: message => { notes.push(message); void log({ note: message }); },
|
||||
...(environment.EXPLORE_ARGENT_BUNDLE_ID ? { bundleId: environment.EXPLORE_ARGENT_BUNDLE_ID } : {}),
|
||||
...(environment.EXPLORE_ARGENT_DEVICE ? { device: environment.EXPLORE_ARGENT_DEVICE } : {}),
|
||||
// An opened foldable's inner panel shows its UI in landscape (Argent's fold: "Unfolded, the UI is landscape").
|
||||
@@ -101,7 +117,8 @@ export function createArgentBridge(settings: ArgentBridgeSettings, call: ArgentC
|
||||
// The same URL Argent's own client checks: ARGENT_TOOLS_URL may carry a path, which /tools is appended to.
|
||||
const response = await fetch(`${(toolsUrl ?? "").replace(/\/+$/, "")}/tools`, { headers: token ? { authorization: `Bearer ${token}` } : {}, signal: AbortSignal.timeout(5_000) });
|
||||
await response.body?.cancel();
|
||||
ok = response.status < 500;
|
||||
// A tool-server still starting can answer 404 to /tools (euxy main baseline run 39506374824): it is not serving yet.
|
||||
ok = response.status < 500 && response.status !== 404;
|
||||
} catch { ok = false; }
|
||||
if (!ok) await log({ tool: "reachable", ok });
|
||||
return ok;
|
||||
@@ -131,7 +148,7 @@ export function createArgentBridge(settings: ArgentBridgeSettings, call: ArgentC
|
||||
});
|
||||
await writeState(statePath, state);
|
||||
await log({ result: result.exitCode === 0 ? "ok" : result.stdout.slice(0, 300) });
|
||||
return result;
|
||||
return notes.length ? { ...result, notes } : result;
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -8,7 +8,7 @@
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { stat } from "node:fs/promises";
|
||||
import { ArgentCallError } from "./argent-driver.js";
|
||||
import { argentBridgeSettings, createArgentBridge } from "./argent-bridge.js";
|
||||
import { argentBridgeSettings, argentRunFailureDetail, createArgentBridge } from "./argent-bridge.js";
|
||||
import { ensureController } from "../shared/controller-cache.js";
|
||||
|
||||
const ARGENT_CALL_TIMEOUT_MS = 180_000;
|
||||
@@ -34,13 +34,12 @@ async function main(): Promise<number> {
|
||||
return 1;
|
||||
}
|
||||
|
||||
const callArgent = async (tool: string, args: Record<string, unknown>, options?: { out?: string }): Promise<unknown> => {
|
||||
const callArgent = async (tool: string, args: Record<string, unknown>, options?: { out?: string; timeoutMs?: number }): Promise<unknown> => {
|
||||
const run = spawnSync(process.execPath, [cli, "run", tool, "--args", JSON.stringify(args), ...(options?.out ? ["--out", options.out] : [])],
|
||||
{ env: environment, encoding: "utf8", maxBuffer: 64 * 1024 * 1024, timeout: ARGENT_CALL_TIMEOUT_MS });
|
||||
{ env: environment, encoding: "utf8", maxBuffer: 64 * 1024 * 1024, timeout: Math.min(ARGENT_CALL_TIMEOUT_MS, options?.timeoutMs ?? ARGENT_CALL_TIMEOUT_MS) });
|
||||
if (run.error) throw new ArgentCallError(`argent run ${tool}: ${run.error.message}`, true);
|
||||
if (run.status !== 0) {
|
||||
const detail = `${run.stderr ?? ""}\n${run.stdout ?? ""}`.trim().split("\n").filter(Boolean).slice(-4).join(" | ").slice(0, 800);
|
||||
throw new ArgentCallError(`argent run ${tool} failed: ${detail}`, true);
|
||||
throw new ArgentCallError(`argent run ${tool} failed: ${argentRunFailureDetail(run.stderr ?? "", run.stdout ?? "")}`, true);
|
||||
}
|
||||
// With --out the client saves the tool's file and prints where ("Wrote: <path>"), not the tool's JSON.
|
||||
if (options?.out) {
|
||||
@@ -53,6 +52,7 @@ async function main(): Promise<number> {
|
||||
};
|
||||
|
||||
const result = await createArgentBridge(settings, callArgent, "process").run(argv);
|
||||
for (const note of result.notes ?? []) process.stderr.write(`[explore] ${note}\n`);
|
||||
process.stdout.write(result.stdout);
|
||||
return result.exitCode;
|
||||
}
|
||||
|
||||
@@ -15,7 +15,8 @@ export class ArgentCallError extends Error {
|
||||
constructor(message: string, readonly sent: boolean) { super(message); }
|
||||
}
|
||||
|
||||
export type ArgentCall = (tool: string, args: Record<string, unknown>, options?: { out?: string }) => Promise<unknown>;
|
||||
/** `timeoutMs` shortens the call's own timeout, so a wait for a tool-server still starting stays within its window. */
|
||||
export type ArgentCall = (tool: string, args: Record<string, unknown>, options?: { out?: string; timeoutMs?: number }) => Promise<unknown>;
|
||||
|
||||
type Rect = { x: number; y: number; width: number; height: number };
|
||||
|
||||
@@ -31,6 +32,8 @@ export type ArgentBridgeState = {
|
||||
/** iOS asks "Open in “<app>”?" for the first custom-scheme link of a session; later links open directly. */
|
||||
deepLinkPromptAccepted?: boolean;
|
||||
recording?: string;
|
||||
/** The tool-server has answered a call this session, so a GET /tools it fails is no longer taken for it still starting. */
|
||||
toolsServed?: boolean;
|
||||
};
|
||||
|
||||
export type ArgentBridgeContext = {
|
||||
@@ -43,6 +46,8 @@ export type ArgentBridgeContext = {
|
||||
device?: string;
|
||||
version: string;
|
||||
sleep: (milliseconds: number) => Promise<void>;
|
||||
/** A monotonic clock in milliseconds (performance.now when absent), which bounds the wait for a tool-server to start. */
|
||||
now?: () => number;
|
||||
/** Moves a file the Argent client downloaded (a screen recording) to where Explore asked for it. */
|
||||
moveFile: (from: string, to: string) => Promise<void>;
|
||||
/** Whether the tool-server answers, checked before a UI action is sent (see `remoteUnavailable`). */
|
||||
@@ -57,6 +62,8 @@ export type ArgentBridgeContext = {
|
||||
landscape?: boolean;
|
||||
/** Turns a portrait screen recording a quarter clockwise, to the landscape UI it shows. */
|
||||
orientVideo?: (path: string) => Promise<void>;
|
||||
/** A line for the capture's log: the bridge waiting on a tool-server that is still starting. */
|
||||
progress?: (message: string) => void;
|
||||
};
|
||||
|
||||
export type BridgeResult = { exitCode: number; stdout: string };
|
||||
@@ -259,6 +266,20 @@ const SERVICE_DEPENDENCY_FAILED = "Service dependency failed:";
|
||||
* comes after a pause, so a device still coming back from the restart has the time it needs.
|
||||
*/
|
||||
const DEPENDENCY_RETRY_DELAYS_MS = [3_000, 6_000] as const;
|
||||
/**
|
||||
* A tool-server that is not serving its tools yet. Argent's client asks GET /tools before every tool call and stops with
|
||||
* "GET /tools failed: <status>" when that is not answered, so the tool itself was never sent; a refused connection
|
||||
* reached nothing at all. Either is safe to send again, a UI action too. In brentvatne/euxy's main baseline run
|
||||
* 39506374824 a replacement EAS session's tool-server answered 404 to /tools for the first list-devices of its install,
|
||||
* and the whole target failed. An unknown tool's 404 comes from POST /tools/<name> instead, and still fails at once.
|
||||
*/
|
||||
const TOOLS_NOT_SERVED = /GET \/tools failed: (?:404|503)\b|ECONNREFUSED/;
|
||||
/**
|
||||
* How long a tool-server that has not answered yet gets to start serving its tools, requests included: a deadline, not
|
||||
* a number of tries, since each try can itself take until the call's timeout. The pauses between tries grow to 10 s.
|
||||
*/
|
||||
const STARTUP_WINDOW_MS = 75_000;
|
||||
const STARTUP_RETRY_DELAYS_MS = [1_000, 2_000, 4_000, 8_000, 10_000] as const;
|
||||
const SPRINGBOARD = "com.apple.springboard";
|
||||
const ANDROID_SYSTEM_UI = "com.android.systemui";
|
||||
const OPEN_IN_APP_PROMPT = /^Open in “.+”\?$/;
|
||||
@@ -345,14 +366,40 @@ export async function runBridgeCommand(argv: readonly string[], context: ArgentB
|
||||
/**
|
||||
* Reads are resent after a transport failure (they change nothing); a UI action is sent once, after a health check.
|
||||
* Any tool, a UI action too, is resent when Argent could not start a service it depends on, which it raises before
|
||||
* the tool runs (see SERVICE_DEPENDENCY_FAILED).
|
||||
* the tool runs (see SERVICE_DEPENDENCY_FAILED), and while a tool-server that has not answered yet is still starting
|
||||
* (see TOOLS_NOT_SERVED).
|
||||
*/
|
||||
const call = async (tool: string, args: Record<string, unknown> = {}, options?: { out?: string }): Promise<Record<string, unknown>> => {
|
||||
// Tools that name devices their own way (or none) take no udid.
|
||||
const withDevice = tool === "list-devices" || tool === "stop-all-simulator-servers" ? args : { udid: state.udid, ...args };
|
||||
for (let readRetry = 0, dependencyRetry = 0; ;) {
|
||||
try { return record(await context.call(tool, withDevice, options)); }
|
||||
catch (error) {
|
||||
const clock = context.now ?? (() => performance.now());
|
||||
// The window opens with the first call to a tool-server that has not answered yet, and closes 75 s later. A device
|
||||
// already named came from the tool-server's own answer.
|
||||
const served = () => state.toolsServed === true || state.udid !== undefined;
|
||||
const startupDeadline = served() ? undefined : clock() + STARTUP_WINDOW_MS;
|
||||
for (let readRetry = 0, dependencyRetry = 0, startupRetry = 0; ;) {
|
||||
const starting = !served() && startupDeadline !== undefined;
|
||||
try {
|
||||
// A call to a tool-server still starting gets only the time left in the window.
|
||||
const answer = record(await context.call(tool, withDevice, starting ? { ...options, timeoutMs: Math.max(1, Math.ceil(startupDeadline - clock())) } : options));
|
||||
state.toolsServed = true;
|
||||
return answer;
|
||||
} catch (error) {
|
||||
// Until the session's tool-server has answered once, a GET /tools it does not serve is it still starting.
|
||||
if (starting && error instanceof ArgentCallError) {
|
||||
const notServed = TOOLS_NOT_SERVED.test(error.message);
|
||||
const left = startupDeadline - clock();
|
||||
// A try the window's end cut short is the tool-server not starting in time, whatever the request reported.
|
||||
if (left <= 0) {
|
||||
throw new BridgeFailure("TOOL_SERVER_NOT_READY", `Argent's tool-server did not start serving its tools within ${STARTUP_WINDOW_MS / 1_000} s: ${error.message}`);
|
||||
}
|
||||
if (notServed) {
|
||||
const delay = Math.min(STARTUP_RETRY_DELAYS_MS[Math.min(startupRetry++, STARTUP_RETRY_DELAYS_MS.length - 1)]!, left);
|
||||
context.progress?.(`Argent's tool-server is not serving its tools yet (${error.message}); sending ${tool} again in ${Math.ceil(delay / 1_000)} s (${Math.ceil(left / 1_000)} s left of ${STARTUP_WINDOW_MS / 1_000} s)`);
|
||||
await context.sleep(delay);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
if (error instanceof ArgentCallError && error.message.includes(`[Tool:${tool}] ${SERVICE_DEPENDENCY_FAILED}`)
|
||||
&& dependencyRetry < DEPENDENCY_RETRY_DELAYS_MS.length) {
|
||||
await context.sleep(DEPENDENCY_RETRY_DELAYS_MS[dependencyRetry++]!);
|
||||
@@ -417,8 +464,9 @@ export async function runBridgeCommand(argv: readonly string[], context: ArgentB
|
||||
return target;
|
||||
};
|
||||
/**
|
||||
* A UI action is only sent when the tool-server answers first, so a failure after this point is never resent. A
|
||||
* session's tool-server can still be starting when EAS reports the session ready, so it is given about 25 s to answer.
|
||||
* A UI action is only sent when the tool-server answers first, so a failure after this point is never resent unless
|
||||
* the tool provably never ran (see call). A session's tool-server can still be starting when EAS reports the session
|
||||
* ready, so it is given about 25 s to answer.
|
||||
*/
|
||||
const beforeAction = async () => {
|
||||
for (const delay of [1_000, 2_000, 3_000, 4_000, 5_000, 5_000, 5_000]) {
|
||||
|
||||
@@ -13,7 +13,8 @@ export function defaultRunner(request: CommandRequest): CommandResult {
|
||||
maxBuffer: 50 * 1024 * 1024,
|
||||
...(request.timeoutMs ? { timeout: request.timeoutMs } : {}),
|
||||
});
|
||||
if (result.error) throw result.error;
|
||||
// A command that timed out or could not run keeps what it printed on its error: a hosted repack's start names its run there.
|
||||
if (result.error) throw Object.assign(result.error, { stdout: result.stdout ?? "", stderr: result.stderr ?? "" });
|
||||
return commandOutcome(request, { status: result.status, stdout: result.stdout ?? "", stderr: result.stderr ?? "" });
|
||||
}
|
||||
|
||||
|
||||
+474
-24
@@ -1,8 +1,11 @@
|
||||
import { cp, lstat, realpath, mkdtemp, mkdir, writeFile, readFile, rename, rm } from "node:fs/promises";
|
||||
import { createHash } from "node:crypto";
|
||||
import { cp, lstat, realpath, mkdtemp, mkdir, writeFile, readFile, readdir, readlink, rename, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { isAbsolute, basename, dirname, resolve, join } from "node:path";
|
||||
import { isAbsolute, basename, dirname, resolve, join, relative, sep } from "node:path";
|
||||
import { setTimeout as delay } from "node:timers/promises";
|
||||
import type { SourceRoute } from "../contracts/source-route.js";
|
||||
import { CiError } from "../shared/ci-errors.js";
|
||||
import { EXPLORE_VERSION } from "../shared/package-paths.js";
|
||||
import { prepareCaptureObserver, captureObserverParameterKeys } from "./observer.js";
|
||||
import { isRecord, parseJson, nonEmpty } from "./common.js";
|
||||
import { isWithin } from "./fs-safety.js";
|
||||
@@ -110,12 +113,22 @@ export function nativeBuildFrom(stdout: string): EasBuildDescriptor & { fingerpr
|
||||
};
|
||||
}
|
||||
|
||||
/** The message Explore's hosted repacks give their builds (hostedRepackWorkflow), with or without a share marker. */
|
||||
const EXPLORE_REPACK_MESSAGE = /^expo-explore (?:ios|android) capture: route observer repack\b/;
|
||||
|
||||
/**
|
||||
* The finished build to repack from `eas build:list --fingerprint-hash`: the newest one Explore did not repack itself,
|
||||
* else the newest. A repack keeps its source build's native code, and may carry its fingerprint; preferring the build
|
||||
* it came from has every job of a run pick the same one, which a shared repack's key needs (RepackShare), instead of a
|
||||
* repack another job finished a minute earlier.
|
||||
*/
|
||||
export function existingBuild(stdout: string): EasBuildDescriptor | undefined {
|
||||
const value = parseJson(stdout, "eas build:list");
|
||||
if (!Array.isArray(value)) throw new Error("eas build:list did not return a build list");
|
||||
if (value.length === 0) return undefined;
|
||||
const id = buildIdFrom(value, "eas build:list");
|
||||
const candidate = value[0];
|
||||
const index = Math.max(0, value.findIndex(build => !(isRecord(build) && typeof build.message === "string" && EXPLORE_REPACK_MESSAGE.test(build.message))));
|
||||
const candidate = value[index];
|
||||
const id = buildIdFrom([candidate], "eas build:list");
|
||||
return isRecord(candidate) ? easBuildDescriptor(candidate, id, "eas build:list") : { id };
|
||||
}
|
||||
|
||||
@@ -395,48 +408,438 @@ export function hostedRepackWorkflow(input: {
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
/** The repacked build id from `eas workflow:run --json`, or the job's failure as a readable error. */
|
||||
export function hostedRepackBuildId(stdout: string, stderr: string, platform: "ios" | "android"): string {
|
||||
const logs = /See logs: (https:\/\/expo\.dev\/\S+)/.exec(stderr)?.[1];
|
||||
const where = logs ? ` Logs: ${logs}` : "";
|
||||
/**
|
||||
* Why a hosted repack failed, as far as its run tells:
|
||||
* - `app`: the app's own code broke it: the "Repack" step failed, or the logs show a bundling error. A rerun repeats it.
|
||||
* - `configuration`: EAS or a package registry refused a credential (not logged in, unauthorized, a 401 or 403).
|
||||
* - `canceled`: someone cancelled the run.
|
||||
* - `timeout`: the build ran out its EAS build time, which a rerun would wait out again, or Explore's time for the repack ran out.
|
||||
* - `unconfirmed`: Explore could not learn whether the run ended; it asked EAS to cancel it and starts no other beside it.
|
||||
* - `infrastructure`: EAS failed around the app's code: a step that sets up the job, downloads or uploads failed, or EAS
|
||||
* ended the build (brentvatne/euxy#176).
|
||||
* - `unknown`: nothing says why.
|
||||
*/
|
||||
export type HostedRepackFailure = "app" | "configuration" | "canceled" | "timeout" | "unconfirmed" | "infrastructure" | "unknown";
|
||||
|
||||
const APP_REPACK_STEP = /^Repack(?: app)?$/i;
|
||||
// Metro's, Babel's and Expo's words for the app's own code or files: `Unable to resolve "react-native-reanimated" from
|
||||
// "index.js"`, a missing module, a file an asset step could not read.
|
||||
const APP_REPACK_FAILURE = /Unable to resolve\b|SyntaxError|TransformError|Error while bundling|Bundling failed|Cannot find module|could not read \.{0,2}\/|no such file or directory, open '\.{0,2}\//i;
|
||||
// A file of EAS's own worker missing (`Cannot find module '/opt/eas/...'`) is EAS's failure, whatever the step.
|
||||
const WORKER_REPACK_FAILURE = /['"]\/opt\/eas\//;
|
||||
const CONFIGURATION_REPACK_FAILURE = /Not logged in|\bunauthori[sz]ed\b|\bnot authorized\b|\bE40[13]\b|\b40[13] (?:Unauthorized|Forbidden)\b|authentication (?:failed|required)/i;
|
||||
const TIMEOUT_REPACK_FAILURE = /\bEAS_BUILD_TIMEOUT_ERROR\b/;
|
||||
// The steps EAS runs around the app's own: setting up the job, fetching the build to repack, saving the result.
|
||||
const INFRASTRUCTURE_REPACK_STEP = /^(?:Spin up|Set up|Download|Upload)\b/i;
|
||||
// EAS's and eas-cli's own words for a failure on their side. In brentvatne/euxy#176 (review run 39506405687) EAS refused
|
||||
// the "Upload artifact" step with "Turtle build is not in progress", and the build reported "Failed to upload application archive.".
|
||||
const INFRASTRUCTURE_REPACK_FAILURE = /Turtle build is not in progress|Failed to upload application archive|\bSERVER_ERROR\b|Failed to upload project sources|Failed to start the workflow with the API|Unexpected response from server \(5\d\d\)|\b(?:ECONNRESET|ECONNREFUSED|ETIMEDOUT|ENOTFOUND|EAI_AGAIN)\b|socket hang up|fetch failed/;
|
||||
|
||||
/** How a failure reads: from the build's error, the job's errors, the failed job's logs and the CLI's output. */
|
||||
export function hostedRepackFailure(evidence: string, status?: unknown): HostedRepackFailure {
|
||||
if (status === "CANCELED") return "canceled";
|
||||
if (CONFIGURATION_REPACK_FAILURE.test(evidence)) return "configuration";
|
||||
if (WORKER_REPACK_FAILURE.test(evidence)) return "infrastructure";
|
||||
const step = [...evidence.matchAll(/Build step "([^"]+)" failed/g)].at(-1)?.[1];
|
||||
if ((step && APP_REPACK_STEP.test(step)) || APP_REPACK_FAILURE.test(evidence)) return "app";
|
||||
if (TIMEOUT_REPACK_FAILURE.test(evidence)) return "timeout";
|
||||
if ((step && INFRASTRUCTURE_REPACK_STEP.test(step)) || INFRASTRUCTURE_REPACK_FAILURE.test(evidence)) return "infrastructure";
|
||||
return "unknown";
|
||||
}
|
||||
|
||||
/** How many fresh repacks a failure earns: two after EAS's own failure, one after one nothing explains, none otherwise. */
|
||||
export function hostedRepackRetries(error: unknown): number {
|
||||
const failure = error instanceof CiError && error.code === "EAS_REPACK_FAILED" ? error.details?.failure : undefined;
|
||||
return failure === "infrastructure" ? 2 : failure === "unknown" ? 1 : 0;
|
||||
}
|
||||
|
||||
const EXPO_URL = /^https:\/\/expo\.dev\/\S+$/;
|
||||
const EAS_ID = /^[A-Za-z0-9-]{1,128}$/;
|
||||
|
||||
/**
|
||||
* The repacked build id from a finished run's JSON (`eas workflow:view --json`), or the job's failure as a readable
|
||||
* error. `evidence` is the rest of what the run said: the CLI's stderr and the failed job's logs.
|
||||
*/
|
||||
export function hostedRepackBuildId(stdout: string, evidence: string, platform: "ios" | "android"): string {
|
||||
let logs = /See logs: (https:\/\/expo\.dev\/\S+)/.exec(evidence)?.[1];
|
||||
let value: unknown;
|
||||
try {
|
||||
value = parseJson(stdout, "eas workflow:run");
|
||||
} catch (error) {
|
||||
const tail = evidence.trim().split("\n").slice(-3).join(" | ").slice(0, 600);
|
||||
throw new CiError({
|
||||
code: "EAS_REPACK_FAILED",
|
||||
stage: "instrumentation",
|
||||
message: `eas workflow:run did not report the hosted ${platform} repack: ${stderr.trim().split("\n").slice(-3).join(" | ").slice(0, 600) || (error instanceof Error ? error.message : String(error))}`,
|
||||
nextAction: `Check that EXPO_TOKEN can run workflows for this EAS project and retry.${where}`,
|
||||
message: `eas workflow:run did not report the hosted ${platform} repack: ${tail || (error instanceof Error ? error.message : String(error))}`,
|
||||
nextAction: `Check that EXPO_TOKEN can run workflows for this EAS project and retry.${logs ? ` Logs: ${logs}` : ""}`,
|
||||
details: { failure: hostedRepackFailure(evidence), ...(logs ? { logs } : {}) },
|
||||
cause: error,
|
||||
});
|
||||
}
|
||||
for (const field of ["url", "logURL"]) {
|
||||
const link = isRecord(value) ? value[field] : undefined;
|
||||
if (typeof link === "string" && EXPO_URL.test(link)) logs = link;
|
||||
}
|
||||
const where = logs ? ` Logs: ${logs}` : "";
|
||||
const jobs = isRecord(value) && Array.isArray(value.jobs) ? value.jobs.filter(isRecord) : [];
|
||||
const job = jobs.find(entry => entry.key === "repack") ?? jobs[0];
|
||||
if (!job || job.status !== "SUCCESS") {
|
||||
const build = job && isRecord(job.turtleBuild) ? job.turtleBuild : undefined;
|
||||
const detail = build && isRecord(build.error) && typeof build.error.message === "string" ? build.error.message.trim() : undefined;
|
||||
const planLimit = easPlanLimitError(new Error(`${detail ?? ""}\n${stderr}`), platform);
|
||||
const buildError = build && isRecord(build.error) ? build.error : undefined;
|
||||
const detail = typeof buildError?.message === "string" ? buildError.message.trim() : undefined;
|
||||
const planLimit = easPlanLimitError(new Error(`${detail ?? ""}\n${evidence}`), platform);
|
||||
if (planLimit) throw planLimit;
|
||||
const jobErrors = Array.isArray(job?.errors) ? job.errors.filter(isRecord).map(entry => `${String(entry.title ?? "")} ${String(entry.message ?? "")}`) : [];
|
||||
const failure = hostedRepackFailure([detail ?? "", typeof buildError?.errorCode === "string" ? buildError.errorCode : "", ...jobErrors, evidence].join("\n"),
|
||||
job?.status ?? (isRecord(value) ? value.status : undefined));
|
||||
throw new CiError({
|
||||
code: "EAS_REPACK_FAILED",
|
||||
stage: "instrumentation",
|
||||
message: `The hosted EAS repack of the ${platform} build ${job ? `finished with status ${String(job.status)}` : "did not run"}${detail ? `: ${detail.slice(0, 600)}` : "."}`,
|
||||
nextAction: `Open the EAS workflow run and fix the repack job; a broken app build environment usually surfaces in its "Repack" step.${where}`,
|
||||
details: { failure, ...(logs ? { logs } : {}) },
|
||||
});
|
||||
}
|
||||
const outputs = isRecord(job.outputs) ? job.outputs : {};
|
||||
const buildId = outputs.build_id;
|
||||
if (typeof buildId !== "string" || !/^[A-Za-z0-9-]{1,128}$/.test(buildId)) {
|
||||
if (typeof buildId !== "string" || !EAS_ID.test(buildId)) {
|
||||
throw new Error("eas workflow:run did not return the repacked build id");
|
||||
}
|
||||
return buildId;
|
||||
}
|
||||
|
||||
/**
|
||||
* The run `eas workflow:run --json` (without `--wait`) started: its id and link, from its JSON or, when the JSON is
|
||||
* missing, from the "See logs:" link it printed once the run existed. With neither, no run started, and the failure
|
||||
* says why.
|
||||
*/
|
||||
export function hostedRepackRunStart(stdout: string, stderr: string, platform: "ios" | "android"): { id: string; url?: string } {
|
||||
const printed = /See logs: (https:\/\/expo\.dev\/\S+)/.exec(stderr)?.[1];
|
||||
let value: unknown;
|
||||
try { value = JSON.parse(stdout); } catch { value = undefined; }
|
||||
const url = isRecord(value) && typeof value.url === "string" && EXPO_URL.test(value.url) ? value.url : printed;
|
||||
const id = isRecord(value) && typeof value.id === "string" ? value.id : url ? /\/workflows\/([^/?#\s]+)$/.exec(url)?.[1] : undefined;
|
||||
if (id && EAS_ID.test(id)) return { id, ...(url ? { url } : {}) };
|
||||
const tail = stderr.trim().split("\n").slice(-3).join(" | ").slice(0, 600);
|
||||
throw new CiError({ code: "EAS_REPACK_FAILED", stage: "instrumentation",
|
||||
message: `eas workflow:run did not start the hosted ${platform} repack: ${tail || "it printed no run"}`,
|
||||
nextAction: "Check that EXPO_TOKEN can run workflows for this EAS project and retry.",
|
||||
details: { failure: hostedRepackFailure(stderr) } });
|
||||
}
|
||||
|
||||
/**
|
||||
* The most time preparing the app may take once the build to repack is known: waiting for another job's repack,
|
||||
* following it, and every repack this job starts, each run's wait included. It replaces a 30-minute wait per run, which
|
||||
* with a share wait and two retries let one capture prepare for over an hour; a capture whose app takes longer than
|
||||
* the session's allowance (SESSION_PREPARATION_MINUTES in run.ts) gets a fresh session.
|
||||
*/
|
||||
export const HOSTED_REPACK_BUDGET_MS = 30 * 60_000;
|
||||
/** A repack starts only with this much of the budget left: a typical one takes about five minutes. */
|
||||
const HOSTED_REPACK_MIN_ATTEMPT_MS = 5 * 60_000;
|
||||
/** How often a running repack's state is read, and how many unreadable reads in a row mean Explore cannot tell. */
|
||||
const HOSTED_REPACK_POLL_MS = 20_000;
|
||||
const HOSTED_REPACK_UNREADABLE_READS = 6;
|
||||
const TERMINAL_RUN_STATUSES = new Set(["SUCCESS", "FAILURE", "CANCELED"]);
|
||||
|
||||
/** What the hosted repack waits and reports with: the end of its budget, the clock and pause (tests replace both), and an abort. */
|
||||
type RepackClock = { deadline: number; now: () => number; sleep?: ((milliseconds: number) => void) | undefined; signal?: AbortSignal | undefined };
|
||||
|
||||
async function pause(milliseconds: number, clock: RepackClock): Promise<void> {
|
||||
if (clock.sleep) clock.sleep(milliseconds);
|
||||
else {
|
||||
try { await delay(milliseconds, undefined, clock.signal ? { signal: clock.signal } : {}); }
|
||||
catch (error) { clock.signal?.throwIfAborted(); throw error; }
|
||||
}
|
||||
clock.signal?.throwIfAborted();
|
||||
}
|
||||
|
||||
function timeoutError(platform: "ios" | "android", detail: string, logs?: string): CiError {
|
||||
return new CiError({ code: "EAS_REPACK_FAILED", stage: "instrumentation",
|
||||
message: `The hosted EAS repack of the ${platform} build did not finish within the ${HOSTED_REPACK_BUDGET_MS / 60_000} minutes a capture has to prepare its app: ${detail}.`,
|
||||
nextAction: `Check the EAS workflow run for a long queue or a stuck step, then rerun Explore.${logs ? ` Logs: ${logs}` : ""}`,
|
||||
details: { failure: "timeout", ...(logs ? { logs } : {}) } });
|
||||
}
|
||||
|
||||
/**
|
||||
* The failed step's logs from `eas workflow:logs --json` (the step that logged `Build step "…" failed`, else the last
|
||||
* with any lines), so that a warning another step printed does not decide the failure; nothing when unreadable.
|
||||
*/
|
||||
function workflowLogText(stdout: string): string {
|
||||
try {
|
||||
const value: unknown = JSON.parse(stdout);
|
||||
if (!isRecord(value)) return "";
|
||||
const steps = Object.values(value).map(lines => Array.isArray(lines) ? lines.filter(isRecord).map(line => String(line.msg ?? "")) : []).filter(lines => lines.length);
|
||||
const failed = [...steps].reverse().find(lines => lines.some(line => /Build step "[^"]+" failed/.test(line))) ?? steps.at(-1) ?? [];
|
||||
return failed.slice(-400).join("\n");
|
||||
} catch { return ""; }
|
||||
}
|
||||
|
||||
type EasCall = (args: string[], timeoutMs: number, options?: { cwd?: string; strict?: boolean }) => { stdout: string; stderr: string };
|
||||
|
||||
/** What a command that threw still printed: defaultRunner keeps a timed-out or signalled command's output on its error. */
|
||||
function printedBy(error: unknown): { stdout: string; stderr: string } {
|
||||
const value = error as { stdout?: unknown; stderr?: unknown } | undefined;
|
||||
return { stdout: typeof value?.stdout === "string" ? value.stdout : "", stderr: typeof value?.stderr === "string" ? value.stderr : "" };
|
||||
}
|
||||
|
||||
/**
|
||||
* One hosted repack run: started without `--wait`, so Explore always knows the run and waits on it within its own
|
||||
* budget. eas-cli's `--wait` gives up after six failed status reads with exit code 13 and leaves the run going, and
|
||||
* Explore then started another beside it. A run whose state stays unreadable, that outlasts the budget, or whose
|
||||
* capture is stopped is cancelled (`eas workflow:cancel`) and not replaced: only a run EAS reports finished is
|
||||
* followed by another. A start that timed out or was stopped is waited on when it printed its run, and otherwise
|
||||
* not replaced, since it may have started one.
|
||||
*/
|
||||
async function hostedRepackRun(options: { eas: EasCall; staged: string; platform: "ios" | "android"; clock: RepackClock }): Promise<string> {
|
||||
const { clock, platform } = options;
|
||||
let started: { stdout: string; stderr: string };
|
||||
let startFailure: unknown;
|
||||
try { started = options.eas(["workflow:run", HOSTED_REPACK_WORKFLOW, "--json", "--non-interactive"], Math.min(10 * 60_000, clock.deadline - clock.now()), { cwd: options.staged }); }
|
||||
catch (error) { started = printedBy(error); startFailure = error; }
|
||||
let run: { id: string; url?: string };
|
||||
try { run = hostedRepackRunStart(started.stdout, started.stderr, platform); }
|
||||
catch (error) {
|
||||
if (startFailure === undefined) throw error;
|
||||
throw new CiError({ code: "EAS_REPACK_FAILED", stage: "instrumentation",
|
||||
message: `eas workflow:run did not finish starting the hosted ${platform} repack (${startFailure instanceof Error ? startFailure.message : String(startFailure)}), and printed no run: one may have started, so Explore does not start another.`,
|
||||
nextAction: "Check the project's workflow runs on expo.dev, cancel a repack still running, and rerun Explore.",
|
||||
details: { failure: "unconfirmed" }, cause: startFailure });
|
||||
}
|
||||
const cancel = (): boolean => {
|
||||
try { options.eas(["workflow:cancel", run.id, "--non-interactive"], 2 * 60_000, { strict: true }); return true; } catch { return false; }
|
||||
};
|
||||
try {
|
||||
let unreadable = 0;
|
||||
for (;;) {
|
||||
clock.signal?.throwIfAborted();
|
||||
let view = { stdout: "", stderr: "" };
|
||||
try { view = options.eas(["workflow:view", run.id, "--json", "--non-interactive"], 2 * 60_000); } catch { /* Counted as an unreadable read below. */ }
|
||||
let value: unknown;
|
||||
try { value = JSON.parse(view.stdout); } catch { value = undefined; }
|
||||
const status = isRecord(value) && typeof value.status === "string" ? value.status : undefined;
|
||||
if (status && TERMINAL_RUN_STATUSES.has(status)) {
|
||||
const jobs = isRecord(value) && Array.isArray(value.jobs) ? value.jobs.filter(isRecord) : [];
|
||||
const job = jobs.find(entry => entry.key === "repack") ?? jobs[0];
|
||||
let logs = "";
|
||||
if (status === "FAILURE" && typeof job?.id === "string" && EAS_ID.test(job.id)) {
|
||||
try { logs = workflowLogText(options.eas(["workflow:logs", job.id, "--json", "--non-interactive"], 2 * 60_000).stdout); } catch { /* The build's error still classifies it. */ }
|
||||
}
|
||||
return hostedRepackBuildId(view.stdout, `${started.stderr}\n${logs}`, platform);
|
||||
}
|
||||
unreadable = status ? 0 : unreadable + 1;
|
||||
if (unreadable >= HOSTED_REPACK_UNREADABLE_READS) {
|
||||
const cancelled = cancel();
|
||||
throw new CiError({ code: "EAS_REPACK_FAILED", stage: "instrumentation",
|
||||
message: `EAS did not report the state of the hosted ${platform} repack ${HOSTED_REPACK_UNREADABLE_READS} times in a row; ${cancelled ? "Explore cancelled the run" : "Explore could not cancel the run"} and did not start another beside it.`,
|
||||
nextAction: `Check the EAS status and the run${cancelled ? "" : ", and cancel it if it is still going"}, then rerun Explore.${run.url ? ` Logs: ${run.url}` : ""}`,
|
||||
details: { failure: "unconfirmed", ...(run.url ? { logs: run.url } : {}) } });
|
||||
}
|
||||
const left = clock.deadline - clock.now();
|
||||
if (left <= 0) {
|
||||
const cancelled = cancel();
|
||||
throw timeoutError(platform, `run ${run.id} was still ${status ?? "unreadable"}; ${cancelled ? "it was cancelled" : "it could not be cancelled"}`, run.url);
|
||||
}
|
||||
await pause(Math.min(HOSTED_REPACK_POLL_MS, left), clock);
|
||||
}
|
||||
} catch (error) {
|
||||
// A capture stopped while its repack runs (a cancelled job, Ctrl+C) leaves no run behind it.
|
||||
if (clock.signal?.aborted) cancel();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Where the matrix jobs of one GitHub run attempt meet to make one hosted repack of an app instead of one each: `scope`
|
||||
* names the run attempt (repository, run id and attempt), `leader` the first target in config order that repacks this
|
||||
* app, `target` this one, `rank` its place among them (the leader's is 0), and `rerun` an attempt after the first, which
|
||||
* runs only the jobs it re-runs. In brentvatne/euxy#176 (review run 39506405687) four iOS targets each repacked EAS
|
||||
* build a81d5d80 with the same commit's JavaScript at the same time, and one of the four failed.
|
||||
*/
|
||||
export type RepackShare = { scope: string; leader: string; target: string; rank: number; rerun?: boolean };
|
||||
|
||||
/** How long the target after the leader waits for a repack to appear on EAS before it repacks itself. */
|
||||
const SHARED_REPACK_APPEAR_MS = 3 * 60_000;
|
||||
/**
|
||||
* Each later target waits this much longer, so that when the leader makes no repack (its baseline covered every route,
|
||||
* or `max-parallel` kept it out of the first jobs) the next target's repack has appeared before the one after it would
|
||||
* start another: the targets join one repack instead of all starting their own at the same moment.
|
||||
*/
|
||||
const SHARED_REPACK_STAGGER_MS = 60_000;
|
||||
/** How long a target whose followed repack failed looks for the repack that replaces it (the leader's retry) before repacking itself. */
|
||||
const SHARED_REPACK_GRACE_MS = 60_000;
|
||||
/** Polls while no repack is followed, and while one is. */
|
||||
const SHARED_REPACK_APPEAR_POLL_MS = 15_000;
|
||||
const SHARED_REPACK_FOLLOW_POLL_MS = 30_000;
|
||||
/** The recent builds of the platform a look reads (eas-cli's most): one this run made minutes ago is among them. */
|
||||
const SHARED_REPACK_LIST_LIMIT = 50;
|
||||
|
||||
/**
|
||||
* Every byte the hosted repack would upload, as one digest: the staged checkout's tracked files and the generated
|
||||
* observer, path by path, with their kind, executable bit and content (a symlink by its target).
|
||||
*/
|
||||
export async function stagedProjectDigest(root: string): Promise<string> {
|
||||
const hash = createHash("sha256");
|
||||
const entries = (await readdir(root, { recursive: true, withFileTypes: true }))
|
||||
.map(entry => ({ entry, path: relative(root, join(entry.parentPath, entry.name)).split(sep).join("/") }))
|
||||
.sort((a, b) => a.path < b.path ? -1 : a.path > b.path ? 1 : 0);
|
||||
for (const { entry, path } of entries) {
|
||||
const full = join(root, path);
|
||||
if (entry.isSymbolicLink()) hash.update(`link\0${path}\0${await readlink(full)}\0`);
|
||||
else if (entry.isDirectory()) hash.update(`directory\0${path}\0`);
|
||||
else if (entry.isFile()) {
|
||||
const executable = ((await lstat(full)).mode & 0o111) !== 0;
|
||||
hash.update(`file\0${path}\0${executable ? "x" : "-"}\0${createHash("sha256").update(await readFile(full)).digest("hex")}\0`);
|
||||
} else throw new Error(`The staged copy holds ${path}, which is neither a file, a directory nor a symlink`);
|
||||
}
|
||||
return hash.digest("hex");
|
||||
}
|
||||
|
||||
/**
|
||||
* The key a shared repack's build carries in its message: everything the repack is made from, so two jobs share one only
|
||||
* when each would have uploaded the same bytes for the same native build, profile, EAS environment (by name and by a
|
||||
* snapshot of its variables) and Explore release, in the same run attempt.
|
||||
*/
|
||||
export function hostedRepackShareKey(input: { scope: string; platform: "ios" | "android"; nativeBuildId: string; easProfile: string;
|
||||
environment: string; environmentVariables: string; project: string }): string {
|
||||
return createHash("sha256").update(JSON.stringify({ contract: "expo-explore.repack-share.v2", explore: EXPLORE_VERSION, scope: input.scope,
|
||||
platform: input.platform, nativeBuildId: input.nativeBuildId, easProfile: input.easProfile, environment: input.environment,
|
||||
environmentVariables: input.environmentVariables, project: input.project })).digest("hex").slice(0, 32);
|
||||
}
|
||||
|
||||
/** The marker a shared repack's build message ends with. */
|
||||
export function hostedRepackShareMarker(key: string): string {
|
||||
return `[explore share ${key}]`;
|
||||
}
|
||||
|
||||
type ListedBuild = { id: string; status: string };
|
||||
|
||||
function sharedRepackBuild(build: unknown, marker: string, platform: "ios" | "android"): ListedBuild | undefined {
|
||||
return isRecord(build) && typeof build.id === "string" && EAS_ID.test(build.id) && typeof build.status === "string"
|
||||
&& typeof build.message === "string" && build.message.endsWith(marker) && String(build.platform).toLowerCase() === platform
|
||||
? { id: build.id, status: build.status } : undefined;
|
||||
}
|
||||
|
||||
/** The builds `eas build:list --json` lists that carry `marker` for `platform`, or undefined when the list is unreadable. */
|
||||
export function sharedRepackBuilds(stdout: string, marker: string, platform: "ios" | "android"): ListedBuild[] | undefined {
|
||||
let value: unknown;
|
||||
try { value = JSON.parse(stdout); } catch { return undefined; }
|
||||
if (!Array.isArray(value)) return undefined;
|
||||
return value.flatMap(build => sharedRepackBuild(build, marker, platform) ?? []);
|
||||
}
|
||||
|
||||
/** One build from `eas build:view --json`: null when it no longer carries `marker` for `platform`, undefined when unreadable. */
|
||||
export function sharedRepackBuildView(stdout: string, marker: string, platform: "ios" | "android"): ListedBuild | null | undefined {
|
||||
let value: unknown;
|
||||
try { value = JSON.parse(stdout); } catch { return undefined; }
|
||||
return isRecord(value) ? sharedRepackBuild(value, marker, platform) ?? null : undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* A snapshot of the EAS environment's variables the repack job reads: `eas env:list --format long` of the project's
|
||||
* and the account's variables, which names each with its value (masked when sensitive or secret) and when it was last
|
||||
* updated, so an edited value changes it. Undefined when either list fails.
|
||||
*/
|
||||
function easEnvironmentSnapshot(list: (scope: "project" | "account") => string | undefined): string | undefined {
|
||||
const project = list("project"), account = list("account");
|
||||
if (project === undefined || account === undefined) return undefined;
|
||||
return createHash("sha256").update(JSON.stringify([project, account])).digest("hex");
|
||||
}
|
||||
|
||||
const LIVE_BUILD_STATUSES = new Set(["NEW", "IN_QUEUE", "IN_PROGRESS"]);
|
||||
|
||||
/**
|
||||
* How long a target looks for another's repack before it repacks itself: the leader not at all, and each target after it
|
||||
* SHARED_REPACK_STAGGER_MS longer than the one before, from SHARED_REPACK_APPEAR_MS. A re-run attempt's leader may not
|
||||
* run again, so there the stagger starts at once. After a followed repack failed, the wait is SHARED_REPACK_GRACE_MS,
|
||||
* staggered the same way.
|
||||
*/
|
||||
export function sharedRepackWaitMs(share: Pick<RepackShare, "rank" | "rerun">, afterFailure = false): number {
|
||||
const later = Math.max(0, share.rank - 1) * SHARED_REPACK_STAGGER_MS;
|
||||
if (afterFailure) return SHARED_REPACK_GRACE_MS + later;
|
||||
if (share.rank === 0) return 0;
|
||||
return share.rerun ? share.rank * SHARED_REPACK_STAGGER_MS : SHARED_REPACK_APPEAR_MS + later;
|
||||
}
|
||||
|
||||
/**
|
||||
* A repack another job of this run attempt made, or is making, of exactly this app: its EAS build id once it finished,
|
||||
* or undefined when this job should repack itself.
|
||||
* - The leader looks once. Another target looks every 15 s for as long as sharedRepackWaitMs gives it, and joins any
|
||||
* repack with the key it finds, whichever job started it.
|
||||
* - A repack it finds is then read by its id every 30 s (a list of recent builds can lose it on a busy project).
|
||||
* - One that fails, or no longer carries the key, is set aside, and for SHARED_REPACK_GRACE_MS the target looks for
|
||||
* another with the key (the leader's retry, at once in brentvatne/euxy#176) before it repacks itself.
|
||||
* - Looking stops while the budget still has time for a repack of this job's own, and following stops at the end of
|
||||
* the budget. Three unreadable reads in a row also end the wait: it never waits on a job that cannot answer.
|
||||
*/
|
||||
async function sharedHostedRepack(options: { share: RepackShare; marker: string; platform: "ios" | "android"; list: () => string;
|
||||
view: (id: string) => string; say: (message: string) => void; clock: RepackClock }): Promise<string | undefined> {
|
||||
const { clock, say } = options;
|
||||
const start = clock.now();
|
||||
const lookLatest = clock.deadline - HOSTED_REPACK_MIN_ATTEMPT_MS;
|
||||
const firstWait = sharedRepackWaitMs(options.share);
|
||||
let lookUntil = Math.min(start + firstWait, lookLatest);
|
||||
let followed: string | undefined;
|
||||
const setAside = new Set<string>();
|
||||
let unreadable = 0;
|
||||
let announced = false;
|
||||
for (;;) {
|
||||
clock.signal?.throwIfAborted();
|
||||
let read: ListedBuild[] | null | undefined;
|
||||
try {
|
||||
if (followed) {
|
||||
const build = sharedRepackBuildView(options.view(followed), options.marker, options.platform);
|
||||
read = build === undefined ? undefined : build === null ? [] : [build];
|
||||
} else read = sharedRepackBuilds(options.list(), options.marker, options.platform);
|
||||
} catch { read = undefined; }
|
||||
if (!read) {
|
||||
unreadable += 1;
|
||||
if (unreadable >= 3) { say("EAS did not answer three times in a row; repacking here instead of waiting for another target's repack"); return undefined; }
|
||||
} else {
|
||||
unreadable = 0;
|
||||
const builds = read.filter(build => !setAside.has(build.id));
|
||||
const finished = builds.find(build => build.status === "FINISHED");
|
||||
if (finished) {
|
||||
say(`using EAS build ${finished.id}, the hosted repack of this commit another target of this run made, instead of repacking it again`);
|
||||
return finished.id;
|
||||
}
|
||||
const live = builds.find(build => LIVE_BUILD_STATUSES.has(build.status));
|
||||
if (followed && !live) {
|
||||
say(`the shared repack (EAS build ${followed}) ${read[0] ? `ended ${read[0].status}` : "no longer carries this commit's key"}; looking for another for up to ${sharedRepackWaitMs(options.share, true) / 1000} s before repacking here`);
|
||||
setAside.add(followed);
|
||||
followed = undefined;
|
||||
lookUntil = Math.min(clock.now() + sharedRepackWaitMs(options.share, true), lookLatest);
|
||||
} else if (!followed && live) {
|
||||
followed = live.id;
|
||||
say(`waiting for EAS build ${live.id}, the hosted repack of this commit another target of this run started, instead of repacking it again`);
|
||||
}
|
||||
}
|
||||
const now = clock.now();
|
||||
if (followed && now >= clock.deadline) {
|
||||
say(`the shared repack (EAS build ${followed}) did not finish in the time this capture has to prepare its app`);
|
||||
return undefined;
|
||||
}
|
||||
if (!followed && now >= lookUntil) {
|
||||
if (setAside.size) say("no other repack of this commit appeared; repacking here");
|
||||
else if (firstWait) say(`no repack of this commit from ${options.share.leader} or another target appeared within ${Math.round((now - start) / 60_000)} minutes; repacking here`);
|
||||
return undefined;
|
||||
}
|
||||
if (!followed && !announced && firstWait) {
|
||||
announced = true;
|
||||
say(`waiting up to ${Math.round(firstWait / 60_000)} minutes for the hosted repack ${options.share.leader} or another target makes of this commit, instead of repacking it again`);
|
||||
}
|
||||
const until = followed ? clock.deadline : lookUntil;
|
||||
await pause(Math.max(0, Math.min(followed ? SHARED_REPACK_FOLLOW_POLL_MS : SHARED_REPACK_APPEAR_POLL_MS, until - now)), clock);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Repack on EAS instead of the runner: the tracked files of the clean checkout are staged beside the
|
||||
* observer and a one-job workflow, uploaded with `eas workflow:run`, and the job's `build_id` comes back.
|
||||
* No build tools, no `eas upload`, and the profile's EAS environment applies as it does for any build.
|
||||
* With a `share`, a repack another job of this run attempt makes of the same staged bytes is used instead
|
||||
* (sharedHostedRepack), and this job's own repack carries the key that lets the others use it. A repack that fails
|
||||
* for a reason a rerun can fix starts again (hostedRepackRetries): in brentvatne/euxy#176 EAS ended a build that had
|
||||
* repacked before it could save the app. All of it stays within the capture's budget (`clock.deadline`).
|
||||
*/
|
||||
async function prepareHostedRepack(options: {
|
||||
platform: "ios" | "android";
|
||||
@@ -451,7 +854,11 @@ async function prepareHostedRepack(options: {
|
||||
provider: SourceRoute["provider"];
|
||||
parameterKeys: readonly string[];
|
||||
root: string;
|
||||
share?: RepackShare | undefined;
|
||||
say: (message: string) => void;
|
||||
clock: RepackClock;
|
||||
}): Promise<string> {
|
||||
const { clock, platform } = options;
|
||||
const staged = join(options.root, "project");
|
||||
await mkdir(staged, { mode: 0o700 });
|
||||
// Paths relative to the app directory, which is what is staged. With --full-name they were relative to the
|
||||
@@ -481,19 +888,51 @@ async function prepareHostedRepack(options: {
|
||||
message: error instanceof Error ? error.message : "The route observer could not be prepared.",
|
||||
nextAction: "Check the installed Expo and navigation package compatibility for automatic capture instrumentation.", cause: error });
|
||||
}
|
||||
const privateEnvironment = { ...options.easEnvironment, TMPDIR: options.root, TMP: options.root, TEMP: options.root };
|
||||
// Reads may fail and say so in their output; `strict` makes a failed command throw (a cancel that did not happen).
|
||||
const eas: EasCall = (args, timeoutMs, { cwd = options.project, strict = false } = {}) => options.runner({ command: options.easCommand, args, cwd,
|
||||
environment: cwd === staged ? { ...privateEnvironment, EAS_NO_VCS: "1" } : privateEnvironment, timeoutMs, ...(strict ? {} : { allowFailure: true }) });
|
||||
let message = `expo-explore ${platform} capture: route observer repack`;
|
||||
if (options.share) {
|
||||
const environment = options.repackProfile.easEnvironment;
|
||||
const variables = easEnvironmentSnapshot(scope => {
|
||||
try {
|
||||
const listedVariables = options.runner({ command: options.easCommand, args: ["env:list", "--environment", environment, "--scope", scope, "--format", "long"],
|
||||
cwd: options.project, environment: privateEnvironment, timeoutMs: 2 * 60_000 });
|
||||
return listedVariables.stdout;
|
||||
} catch { return undefined; }
|
||||
});
|
||||
if (variables === undefined) options.say(`EAS did not list the ${environment} environment's variables; repacking here without sharing the repack`);
|
||||
else {
|
||||
// The digest is taken before the workflow file is written: the file's message carries the key made from it.
|
||||
const marker = hostedRepackShareMarker(hostedRepackShareKey({ scope: options.share.scope, platform, nativeBuildId: options.nativeBuildId,
|
||||
easProfile: options.easProfile, environment, environmentVariables: variables, project: await stagedProjectDigest(staged) }));
|
||||
const shared = await sharedHostedRepack({ share: options.share, marker, platform, say: options.say, clock,
|
||||
list: () => eas(["build:list", "--platform", platform, "--limit", String(SHARED_REPACK_LIST_LIMIT), "--json", "--non-interactive"], 2 * 60_000).stdout,
|
||||
view: id => eas(["build:view", id, "--json"], 2 * 60_000).stdout });
|
||||
if (shared) return shared;
|
||||
message = `${message} ${marker}`;
|
||||
}
|
||||
}
|
||||
const workflowPath = join(staged, HOSTED_REPACK_WORKFLOW);
|
||||
await mkdir(dirname(workflowPath), { recursive: true });
|
||||
await writeFile(workflowPath, hostedRepackWorkflow({ buildId: options.nativeBuildId, profile: options.easProfile,
|
||||
environment: options.repackProfile.easEnvironment, message: `expo-explore ${options.platform} capture: route observer repack` }), { flag: "wx", mode: 0o600 });
|
||||
const run = options.runner({
|
||||
command: options.easCommand,
|
||||
args: ["workflow:run", HOSTED_REPACK_WORKFLOW, "--wait", "--json", "--non-interactive"],
|
||||
cwd: staged,
|
||||
environment: { ...options.easEnvironment, EAS_NO_VCS: "1", TMPDIR: options.root, TMP: options.root, TEMP: options.root },
|
||||
timeoutMs: 30 * 60_000,
|
||||
allowFailure: true,
|
||||
});
|
||||
return hostedRepackBuildId(run.stdout, run.stderr, options.platform);
|
||||
environment: options.repackProfile.easEnvironment, message }), { flag: "wx", mode: 0o600 });
|
||||
let retries = 0;
|
||||
for (;;) {
|
||||
if (clock.deadline - clock.now() < HOSTED_REPACK_MIN_ATTEMPT_MS) throw timeoutError(platform, retries ? "too little time was left for another repack" : "too little time was left for a repack of its own");
|
||||
try {
|
||||
return await hostedRepackRun({ eas, staged, platform, clock });
|
||||
} catch (error) {
|
||||
// Only a run EAS reported finished gets here with a failure that earns another (hostedRepackRun), so none is left beside it.
|
||||
const allowed = hostedRepackRetries(error);
|
||||
if (retries >= allowed || clock.deadline - clock.now() < HOSTED_REPACK_MIN_ATTEMPT_MS) throw error;
|
||||
retries += 1;
|
||||
const logs = error instanceof CiError && typeof error.details?.logs === "string" ? error.details.logs : undefined;
|
||||
options.say(`the hosted EAS repack${logs ? ` ${logs}` : ""} failed (${error instanceof Error ? error.message : String(error)}); starting another repack (${retries}/${allowed})`);
|
||||
clock.signal?.throwIfAborted();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -520,6 +959,16 @@ export async function prepareExactSourceBuild(options: {
|
||||
easProfile?: string | undefined;
|
||||
/** Where to leave the instrumented app on this machine, for the capture to install; it must not exist yet. */
|
||||
keepApp: string;
|
||||
/** The other matrix jobs of this run that repack the same app, to make one hosted repack with (RepackShare). */
|
||||
share?: RepackShare | undefined;
|
||||
/** Says what the hosted repack is doing when it waits for another job's repack or starts another one. */
|
||||
progress?: ((message: string) => void) | undefined;
|
||||
/** When preparing the app must be done by (HOSTED_REPACK_BUDGET_MS after the session started beside it), on `now`'s clock. */
|
||||
deadline?: number | undefined;
|
||||
/** The clock and pause the waits and retries use; tests replace them so that minutes are not real time. */
|
||||
now?: (() => number) | undefined;
|
||||
sleep?: ((milliseconds: number) => void) | undefined;
|
||||
signal?: AbortSignal | undefined;
|
||||
}): Promise<PreparedBuild | undefined> {
|
||||
const platform = options.platform ?? "ios";
|
||||
const root = await mkdtemp(join(tmpdir(), "expo-explore-repack-"));
|
||||
@@ -535,7 +984,8 @@ export async function prepareExactSourceBuild(options: {
|
||||
const hosted = await prepareHostedRepack({ platform, runner: options.runner, easCommand: options.easCommand, project: options.project,
|
||||
nativeBuildId: options.nativeBuildId, easProfile: options.easProfile, repackProfile: options.repackProfile,
|
||||
easEnvironment: options.easEnvironment, restrictedEnvironment: options.repackEnvironment, provider: providers[0]!,
|
||||
parameterKeys: captureObserverParameterKeys(options.routes), root });
|
||||
parameterKeys: captureObserverParameterKeys(options.routes), root, share: options.share, say: options.progress ?? (() => undefined),
|
||||
clock: { deadline: options.deadline ?? (options.now ?? Date.now)() + HOSTED_REPACK_BUDGET_MS, now: options.now ?? Date.now, sleep: options.sleep, signal: options.signal } });
|
||||
{
|
||||
// The hosted job leaves its app on EAS; the capture installs a copy downloaded here.
|
||||
const downloaded = await downloadedAppPath(options.runner({ command: options.easCommand,
|
||||
|
||||
+12
-2
@@ -1,5 +1,5 @@
|
||||
import type { CaptureBundle, CaptureOutcome } from "../contracts/capture.js";
|
||||
import { bundleColorSchemes, routeIdOfScreen, schemeScreenId } from "./appearance.js";
|
||||
import { bundleColorSchemes, failedPassOutcome, routeIdOfScreen, schemeScreenId } from "./appearance.js";
|
||||
import { CiError } from "../shared/ci-errors.js";
|
||||
|
||||
type CaptureSide = CaptureBundle["side"];
|
||||
@@ -59,8 +59,10 @@ export function captureHealthIssues(bundle: CaptureBundle, expectedIds: readonly
|
||||
for (const id of [...extras].sort()) {
|
||||
// A secondary frame is expected evidence; its own outcome is the failure of that appearance pass, or its verified
|
||||
// frame when the primary pass has none (the route is then published from this appearance alone).
|
||||
// A pass that failed as a whole (sessionFailures) records its failure under every route's `<target>@<scheme>`.
|
||||
const primaryCaptured = bundle.outcomes[routeIdOfScreen(id).routeId]?.status === "captured";
|
||||
if (!expected.has(id) && !(expectedSecondary.has(id) && !selected.has(id) && hasVerifiedOutcome(bundle.outcomes[id]) !== primaryCaptured)) {
|
||||
if (!expected.has(id) && !(expectedSecondary.has(id) && !selected.has(id)
|
||||
&& (hasVerifiedOutcome(bundle.outcomes[id]) !== primaryCaptured || failedPassOutcome(bundle, id)))) {
|
||||
add(id, "CAPTURE_TARGET_UNEXPECTED", "Capture evidence contains a target absent from the source plan.", inventoryAction);
|
||||
}
|
||||
}
|
||||
@@ -121,6 +123,14 @@ export function captureHealthIssues(bundle: CaptureBundle, expectedIds: readonly
|
||||
const INTEGRITY_CODES = new Set(["CAPTURE_SIDE_MISMATCH", "CAPTURE_COMMIT_INVALID", "CAPTURE_SELECTION_DUPLICATE", "CAPTURE_TARGET_UNEXPECTED",
|
||||
"CAPTURE_TARGET_MISSING", "CAPTURE_OUTCOME_MISSING", "CAPTURE_IMAGE_BINDING_INVALID", "CAPTURE_IMAGE_PROVENANCE_INVALID"]);
|
||||
|
||||
/**
|
||||
* Whether a failure casts doubt on everything a capture took, so its other appearances cannot be kept as a partial
|
||||
* report: a checkout that changed under it, or evidence that failed its integrity checks.
|
||||
*/
|
||||
export function capturedEvidenceInDoubt(code: string): boolean {
|
||||
return /^(?:SOURCE|CHECKOUT)_/.test(code) || code === "CAPTURE_INCOMPLETE" || INTEGRITY_CODES.has(code);
|
||||
}
|
||||
|
||||
export function sortedCaptureHealthIssues(inputs: readonly CaptureHealthInput[]): CaptureHealthIssue[] {
|
||||
const compare = (a: string, b: string) => a < b ? -1 : a > b ? 1 : 0;
|
||||
return inputs.flatMap(({ bundle, selectedIds, side }) => captureHealthIssues(bundle, selectedIds, side))
|
||||
|
||||
+110
-44
@@ -1,5 +1,5 @@
|
||||
import { appearanceArgs, argentColorSchemes, bundleColorSchemes, mergeSchemeOutcomes, resolveColorSchemes, routeIdOfScreen, schemeScreenId, type AppearanceSetting, type ColorScheme } from "./appearance.js";
|
||||
import { assertNoCaptureHealthIssues, captureHealthIssues, partitionCaptureHealthIssues } from "./health.js";
|
||||
import { assertNoCaptureHealthIssues, capturedEvidenceInDoubt, captureHealthIssues, partitionCaptureHealthIssues } from "./health.js";
|
||||
import { type RouteResetSetting } from "./route-reset.js";
|
||||
import { defaultReadinessSeconds } from "./readiness.js";
|
||||
import { captureLaunch, ffmpegRecordingTools, launchDirectory, type LaunchSetting, type RecordingTools } from "./launch-capture.js";
|
||||
@@ -13,7 +13,7 @@ import { sourceRouteNames } from "../contracts/source-route.js";
|
||||
import type { SourceCapturePlan } from "../source/capture-plan.js";
|
||||
import { CONTRACT, type ExploreMap, type Platform, type RunStatus } from "../types.js";
|
||||
import { ExploreStore, validateMap } from "../map/store.js";
|
||||
import { CiError, serializeCiError, toCiError } from "../shared/ci-errors.js";
|
||||
import { CiError, environmentSecrets, serializeCiError, toCiError } from "../shared/ci-errors.js";
|
||||
import type { CaptureSetupAction, DevicePose } from "../config/explore-config.js";
|
||||
import { type CiAgentRuntimeOptions } from "./agent/runtime.js";
|
||||
import { EAS_CLI_VERSION, CONTROLLER_SESSION, isRecord, parseJson, baseEnvironment, nonEmpty } from "./common.js";
|
||||
@@ -21,7 +21,7 @@ import { lstatIfExists, isWithin, temporaryCaptureDirectory, absoluteContractFil
|
||||
import { loadAppIcon, loadCachedAppIcon, storeAppIcon } from "./app-icon.js";
|
||||
import { defaultRunner, runAgentControllerCommand } from "./controller.js";
|
||||
import { silentProgress, type Progress } from "../shared/progress.js";
|
||||
import { assertCaptureEasProfile, assertSimulatorAvailable, fingerprintFrom, hasEasEnvironmentFetchFailure, nativeBuildFrom, existingBuild, repackProfileFrom, prepareExactSourceBuild, type RepackMode } from "./eas-build.js";
|
||||
import { assertCaptureEasProfile, assertSimulatorAvailable, fingerprintFrom, hasEasEnvironmentFetchFailure, nativeBuildFrom, existingBuild, repackProfileFrom, prepareExactSourceBuild, HOSTED_REPACK_BUDGET_MS, type RepackMode, type RepackShare } from "./eas-build.js";
|
||||
import { SIMULATOR_DOTENV_FILE_NAME, SIMULATOR_DOTENV_HEADER, endedOwnSimulatorSession, liveSimulatorSessionsNamed, simulatorSessionName, simulatorStartFailure, readSimulatorDotenvHandoff, assertSimulatorSessionIdentity, assertSimulatorStopped, collectRuntime, controllerVersion, simulatorOsVersion, type ActiveSimulatorRuntime, type SimulatorConnection } from "./simulator.js";
|
||||
import { argentControllerInvocation } from "./controller-invocation.js";
|
||||
import { startArgentBridgeRunner, type ArgentBridgeRunner } from "./argent-bridge-host.js";
|
||||
@@ -71,6 +71,8 @@ export type CiMapOptions = {
|
||||
repack?: RepackMode;
|
||||
/** A folder the review's captures share builds through: targets of one platform, profile and commit build and repack once. */
|
||||
buildShare?: string;
|
||||
/** The matrix jobs of this GitHub run that repack the same app: they make one hosted repack per commit between them. */
|
||||
repackShare?: RepackShare;
|
||||
/** Which appearances to capture; `auto` (default) follows the app's `userInterfaceStyle`. */
|
||||
appearance?: AppearanceSetting;
|
||||
/**
|
||||
@@ -358,6 +360,8 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
|
||||
+ (routes.length - missingRoutes.length ? `, ${routes.length - missingRoutes.length} reused from the baseline` : "")
|
||||
+ (missingRoutes.length ? `, ${missingRoutes.length} to capture` : ""));
|
||||
let captureFailure: unknown;
|
||||
// Later appearance passes that failed after the primary one captured (see the pass loop).
|
||||
const sessionFailures: NonNullable<CaptureBundle["sessionFailures"]> = [];
|
||||
const finish = async (): Promise<CiMapResult> => {
|
||||
if (!(await store.exists())) await store.initialize(map, false);
|
||||
// A session making only secondary passes owns the `<id>@<scheme>` frames of those passes, not the primary ones, even
|
||||
@@ -366,10 +370,11 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
|
||||
? selectedIds.flatMap(id => options.appearancePass!.capture.filter(scheme => scheme !== colorSchemes[0]).map(scheme => schemeScreenId(id, scheme)))
|
||||
: selectedIds;
|
||||
const incomplete = captureFailure !== undefined || ownFrameIds.some(id => outcomes[id]?.status !== "captured" || outcomes[id]?.landing !== "verified")
|
||||
|| map.gaps.some(gap => gap.status === "open");
|
||||
|| map.gaps.some(gap => gap.status === "open") || sessionFailures.length > 0;
|
||||
map.run.status = incomplete || map.screens.length === 0 ? "partial" : "complete";
|
||||
if (routes.length === 0) map.run.stopReason = "No source targets require capture.";
|
||||
else if (captureFailure) map.run.stopReason = "Agent exploration did not finish; see capture diagnostics and bundle outcomes.";
|
||||
else if (sessionFailures.length) map.run.stopReason = sessionFailures.map(failure => `The ${failure.scheme} pass failed (${failure.error.code}): ${failure.error.message.replace(/\.+$/, "")}.`).join(" ");
|
||||
else if (incomplete) map.run.stopReason = "Some source targets could not be captured; see bundle outcomes.";
|
||||
map.run.completedAt = new Date((options.now ?? Date.now)()).toISOString();
|
||||
await store.write(map);
|
||||
@@ -430,7 +435,8 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
|
||||
...(Object.keys(launchRecords).length ? { launch: launchRecords } : {}),
|
||||
...(options.agent ? { captureContexts: Object.fromEntries(selectedIds.filter(id => outcomes[id]?.status === "captured" && options.agent!.contexts[id])
|
||||
.map(id => [id, options.agent!.contexts[id]!])), agentAcceptance: { policy: AGENT_ACCEPTANCE_POLICY,
|
||||
frames: Object.fromEntries(Object.keys(images).map(id => [id, acceptedFrames[id]!])) } } : {}) };
|
||||
frames: Object.fromEntries(Object.keys(images).map(id => [id, acceptedFrames[id]!])) } } : {}),
|
||||
...(sessionFailures.length ? { sessionFailures } : {}) };
|
||||
if (options.agent) assertAgentAcceptance(bundle, map, { required: true });
|
||||
// The side that produces the evidence checks it the way the review will: an integrity defect fails here, naming
|
||||
// its codes, instead of on the consuming side after both captures. Unavailable screens are not defects.
|
||||
@@ -504,6 +510,8 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
|
||||
const localName = `Explore ${options.device} (${side} ${options.gitCommit.slice(0, 8)})`;
|
||||
// An EAS session's start, once begun (startSession): it resolves with the start command's failure, if any.
|
||||
let sessionStart: Promise<unknown> | undefined;
|
||||
// When the current session was asked for, so a session whose app took longer than its allowance can be replaced.
|
||||
let sessionStartedAt: number | undefined;
|
||||
const stopEarlyDevice = async () => {
|
||||
// An EAS session started beside a repack that then failed: stop the exact session its handoff names.
|
||||
if (sessionStart) {
|
||||
@@ -517,6 +525,10 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
|
||||
await rm(simulatorProject, { recursive: true, force: true });
|
||||
await localRecord?.close();
|
||||
};
|
||||
const earlyCleanupFailure = (original: unknown, cleanup: unknown) => new CiError({ code: "SESSION_CLEANUP_FAILED", stage: "cleanup",
|
||||
message: "A simulator Explore started before the app was ready could not be confirmed stopped.",
|
||||
nextAction: "Stop the EAS Simulator session or local device this capture started (eas simulator:list), and verify its status.",
|
||||
details: { originalFailure: serializeCiError(original, { secrets: [options.expoToken] }) }, cause: cleanup });
|
||||
// What this side starts on this machine, so a later review cleans up after it if this process is stopped hard.
|
||||
const localRecord = local ? await openLocalCaptureRecord(sourceEnvironment, simulatorProject) : undefined;
|
||||
if (local) try {
|
||||
@@ -542,7 +554,7 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
|
||||
} });
|
||||
localOsVersion = created.osVersion;
|
||||
}
|
||||
} catch (error) { await stopEarlyDevice().catch(() => undefined); throw error; }
|
||||
} catch (error) { await stopEarlyDevice().catch(cleanup => { throw earlyCleanupFailure(error, cleanup); }); throw error; }
|
||||
// The session is named before its app exists: an EAS session starts while the app is repacked (startSession).
|
||||
// A session EAS failed to start is replaced once (see the session block), under a name of its own.
|
||||
const sessionNameFor = (retry: number) => simulatorSessionName({ mapName: options.name, mapId: options.buildId, gitCommit: options.gitCommit, easBuildId: local ? "local" : "installed",
|
||||
@@ -557,6 +569,7 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
|
||||
const startSession = (retry?: { timeoutMs: number }) => {
|
||||
if (local || sessionStart) return;
|
||||
if (!retry) say(`starting an EAS Simulator session (${options.device}) while the app is repacked`);
|
||||
sessionStartedAt = (options.now ?? Date.now)();
|
||||
const request = (minutes: number) => ({ command: options.easCommand, args: ["simulator:start", "--name", sessionName, "--platform", options.platform, "--device", options.device,
|
||||
"--type", controller, "--package-version", controllerVersion(controller), "--public-link",
|
||||
"--max-duration-minutes", String(minutes), "--out-config-type", "dotenv", "--non-interactive"], cwd: simulatorProject, environment: easEnvironment,
|
||||
@@ -580,6 +593,11 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
|
||||
const appDirectory = !options.buildShare ? await realpath(await mkdtemp(join(tmpdir(), "expo-explore-app-"))) : undefined;
|
||||
let localApp: string | undefined;
|
||||
type BuiltApp = { fingerprint: string; nativeEasBuildId: string | null; easBuildId: string | null; app?: string; reused: boolean; version?: string; buildNumber?: string };
|
||||
// What a hosted repack waits with and reports through: one deadline from the moment the session starts beside it, for
|
||||
// the wait for another job's repack and every repack of its own. The repack is shared with this run's other matrix
|
||||
// jobs only when it repacks a build EAS already had: a native build this job made is its own, so nobody else's key matches it.
|
||||
const repackWaits = (shared: boolean) => ({ ...(shared ? { share: options.repackShare } : {}), progress: say, now: options.now, sleep: options.sleep,
|
||||
signal: options.signal, deadline: (options.now ?? Date.now)() + HOSTED_REPACK_BUDGET_MS });
|
||||
// `repacking` tells the captures sharing this build that their sessions can start too (sharedBuild).
|
||||
const buildApp = async (keepApp: string, repacking?: () => Promise<void>): Promise<BuiltApp> => {
|
||||
options.signal?.throwIfAborted();
|
||||
@@ -588,12 +606,12 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
|
||||
const sourceFingerprint = fingerprintFrom(fingerprintResult.stdout);
|
||||
const reusable = options.forceFreshBuild || hasEasEnvironmentFetchFailure(configResult.stderr) || hasEasEnvironmentFetchFailure(fingerprintResult.stderr)
|
||||
? undefined : existingBuild(eas(["build:list", "--platform", options.platform, "--status", "finished", "--build-profile", options.easProfile,
|
||||
"--fingerprint-hash", sourceFingerprint, ...(options.platform === "ios" ? ["--simulator"] : []), "--limit", "1", "--json", "--non-interactive"]).stdout);
|
||||
"--fingerprint-hash", sourceFingerprint, ...(options.platform === "ios" ? ["--simulator"] : []), "--limit", "20", "--json", "--non-interactive"]).stdout);
|
||||
if (reusable) {
|
||||
say(`reusing EAS build ${reusable.id} (fingerprint ${sourceFingerprint.slice(0, 12)}); repacking it with this commit's JavaScript and the route observer`);
|
||||
startSession(); await repacking?.();
|
||||
const repacked = await prepareExactSourceBuild({ platform: options.platform, runner, easCommand: options.easCommand, repackCommand: options.repackCommand,
|
||||
project, nativeBuildId: reusable.id, fingerprint: sourceFingerprint, repackProfile: profile, easEnvironment, repackEnvironment: restrictedEnvironment, routes: inventory, repack: options.repack, easProfile: options.easProfile, keepApp });
|
||||
project, nativeBuildId: reusable.id, fingerprint: sourceFingerprint, repackProfile: profile, easEnvironment, repackEnvironment: restrictedEnvironment, routes: inventory, repack: options.repack, easProfile: options.easProfile, keepApp, ...repackWaits(true) });
|
||||
if (!repacked) throw new CiError({ code: "CAPTURE_OBSERVER_BUILD_UNSUPPORTED", stage: "instrumentation",
|
||||
message: "The build environment cannot be reproduced for automatic capture instrumentation.",
|
||||
nextAction: "Use a simulator build profile whose required bundle environment can be pulled for local repacking; an uninstrumented build cannot verify captures." });
|
||||
@@ -607,7 +625,7 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
|
||||
say(`EAS build ${built.id} finished; repacking it with the route observer`);
|
||||
startSession(); await repacking?.();
|
||||
const repacked = await prepareExactSourceBuild({ platform: options.platform, runner, easCommand: options.easCommand, repackCommand: options.repackCommand,
|
||||
project, nativeBuildId: built.id, fingerprint: built.fingerprint, repackProfile: profile, easEnvironment, repackEnvironment: restrictedEnvironment, routes: inventory, repack: options.repack, easProfile: options.easProfile, keepApp });
|
||||
project, nativeBuildId: built.id, fingerprint: built.fingerprint, repackProfile: profile, easEnvironment, repackEnvironment: restrictedEnvironment, routes: inventory, repack: options.repack, easProfile: options.easProfile, keepApp, ...repackWaits(false) });
|
||||
if (!repacked) throw new CiError({ code: "CAPTURE_OBSERVER_BUILD_UNSUPPORTED", stage: "instrumentation",
|
||||
message: "The fresh native build cannot be instrumented with the required route observer.",
|
||||
nextAction: "Use a simulator profile with a reproducible bundle environment so both revisions can be repacked with the same observer." });
|
||||
@@ -621,19 +639,36 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
|
||||
? await sharedBuild({ directory: options.buildShare, appName, ...(options.signal ? { signal: options.signal } : {}), build: buildApp, owner: options.outputDirectory,
|
||||
key: { platform: options.platform, easProfile: options.easProfile, bundleId: options.bundleId, gitCommit: options.gitCommit, repack: options.repack ?? "eas", local,
|
||||
routes: inventory.map(route => route.id) },
|
||||
waiting: () => say("using the build another target of this review is preparing for this commit"),
|
||||
waiting: () => say("using the build another capture of this run is preparing for this commit"),
|
||||
// The session is named before its app exists, so a waiter's session boots while the other capture repacks.
|
||||
repacking: () => startSession() })
|
||||
: await buildApp(join(appDirectory!, appName));
|
||||
// A capture that used another capture's build has its session started above; this starts the rest.
|
||||
startSession();
|
||||
// A session started beside an app that took longer to prepare than its allowance has less than the capture's minutes
|
||||
// left, so it is replaced. One that named no session is left to the session block below, which finds out why.
|
||||
const clock = options.now ?? Date.now;
|
||||
if (!local && sessionStart && sessionStartedAt !== undefined && clock() - sessionStartedAt > SESSION_PREPARATION_MINUTES * 60_000) {
|
||||
await sessionStart;
|
||||
const handoff = await readSimulatorDotenvHandoff(dotenvPath).catch(() => undefined);
|
||||
if (handoff?.sessionId) {
|
||||
say(`the app took ${Math.round((clock() - sessionStartedAt) / 60_000)} minutes to prepare, more than the ${SESSION_PREPARATION_MINUTES} session ${handoff.sessionId} was started with for it; replacing it so the capture keeps its ${options.maxDurationMinutes} minutes`);
|
||||
assertSimulatorStopped(eas(["simulator:stop", "--id", handoff.sessionId, "--json", "--non-interactive"], 2 * 60_000, simulatorProject).stdout, handoff.sessionId);
|
||||
await rm(dotenvPath, { force: true });
|
||||
sessionRetries += 1;
|
||||
sessionName = sessionNameFor(sessionRetries);
|
||||
sessionStart = undefined;
|
||||
startSession({ timeoutMs: SESSION_START_TIMEOUT_MS });
|
||||
}
|
||||
}
|
||||
fingerprint = built.fingerprint; nativeEasBuildId = built.nativeEasBuildId; easBuildId = built.easBuildId; localApp = built.app; reusedEasBuild = built.reused;
|
||||
if (!app.version && built.version) app.version = built.version;
|
||||
if (!app.buildNumber && built.buildNumber) app.buildNumber = built.buildNumber;
|
||||
options.signal?.throwIfAborted();
|
||||
} catch (error) {
|
||||
// A build or repack that fails leaves nothing running: a device started for it goes, with the session's files and the app.
|
||||
await stopEarlyDevice().catch(() => undefined);
|
||||
// One that does not go is the failure to report: a session left running is never hidden behind the build's error.
|
||||
await stopEarlyDevice().catch(cleanup => { throw earlyCleanupFailure(error, cleanup); });
|
||||
if (appDirectory) await rm(appDirectory, { recursive: true, force: true });
|
||||
throw error;
|
||||
}
|
||||
@@ -688,7 +723,14 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
|
||||
const live = liveSimulatorSessionsNamed(easRead(["simulator:list", "--name", sessionName, "--status", "new", "--status", "queued", "--status", "starting",
|
||||
"--status", "in-progress", "--limit", "100", "--json", "--non-interactive"]), sessionName, profile.projectIdentity);
|
||||
if (live) {
|
||||
for (const id of live) assertSimulatorStopped(eas(["simulator:stop", "--id", id, "--json", "--non-interactive"], 2 * 60_000).stdout, id);
|
||||
// No session file names these sessions, so one that does not stop is reported as left running, never as a
|
||||
// capture failure a split side could keep as a partial report.
|
||||
for (const id of live) {
|
||||
try { assertSimulatorStopped(eas(["simulator:stop", "--id", id, "--json", "--non-interactive"], 2 * 60_000).stdout, id); }
|
||||
catch (error) { throw new CiError({ code: "SESSION_CLEANUP_FAILED", stage: "cleanup",
|
||||
message: `EAS Simulator session ${id}, started by a request that failed, could not be confirmed stopped.`,
|
||||
nextAction: `Stop EAS Simulator session ${id} and verify its status.`, details: { sessionId: id }, cause: error }); }
|
||||
}
|
||||
reason = `EAS could not start the simulator session (${simulatorStartFailure(startError, [options.expoToken]).message})`;
|
||||
}
|
||||
}
|
||||
@@ -965,41 +1007,60 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
|
||||
const passes: CaptureBundle["outcomes"][] = [];
|
||||
const recordingTools = launchSetting === "off" ? undefined : options.recordingTools === undefined ? ffmpegRecordingTools() : options.recordingTools ?? undefined;
|
||||
const capturedSchemes = options.appearancePass ? colorSchemes.filter(scheme => options.appearancePass!.capture.includes(scheme)) : colorSchemes;
|
||||
const activeRuntime = runtime;
|
||||
for (const [index, colorScheme] of colorSchemes.entries()) {
|
||||
// A pass another session makes leaves its slot empty, so the passes stay aligned with the appearances they merge by.
|
||||
if (!capturedSchemes.includes(colorScheme)) { passes.push({}); continue; }
|
||||
// Argent cannot switch the appearance yet (software-mansion/argent#1509): the host does, simctl or adb on this
|
||||
// machine and serve-sim on an EAS iOS session; elsewhere its single pass runs as the device starts.
|
||||
if (controller !== "argent") runner({ command: controllerCommand, args: [...appearanceArgs(colorScheme), "--session", CONTROLLER_SESSION, ...deviceArgs(runtime), "--platform", options.platform],
|
||||
cwd: simulatorProject, environment: controllerEnvironment, timeoutMs: 60_000 });
|
||||
else if (sessionId && controlActions.has("appearance.set")) simulatorControl(sessionEas, sessionId, "appearance.set", [colorScheme], controlOptions);
|
||||
else if (localSimulator) setLocalSimulatorAppearance({ runner, udid: localSimulator, scheme: colorScheme, cwd: simulatorProject, environment: restrictedEnvironment });
|
||||
else if (localEmulator) setLocalEmulatorAppearance({ runner, serial: localEmulator, scheme: colorScheme, cwd: simulatorProject, environment: restrictedEnvironment });
|
||||
// A scheme change recreates the Android activity in place, and the sheets it re-presents lose their inset; a
|
||||
// relaunch starts the app in the new scheme, the same way the first pass did.
|
||||
if (launchSetting !== "off") {
|
||||
// Observing the launch relaunches the app before setup runs, so no UI-created state is lost.
|
||||
say(`observing the app launch${colorSchemes.length > 1 ? ` (${colorScheme})` : ""}`);
|
||||
try {
|
||||
// A restored baseline's launch evidence is replaced only by a fresh observation.
|
||||
const record = await captureLaunch({ bundleId: options.bundleId, keepVideo: launchSetting === "video", platform: options.platform,
|
||||
directory: join(launchWorkDirectory, launchDirectory(colorScheme)), relativeDirectory: launchDirectory(colorScheme), ...(options.now ? { now: options.now } : {}),
|
||||
...(recordingTools ? { recordingTools } : {}),
|
||||
controller: (args, controllerOptions) => runner({ command: controllerCommand, args: [...args, "--session", CONTROLLER_SESSION, ...deviceArgs(runtime!), "--platform", options.platform],
|
||||
cwd: simulatorProject, environment: controllerEnvironment, timeoutMs: controllerOptions?.timeoutMs ?? 60_000, ...(controllerOptions?.allowFailure ? { allowFailure: true } : {}) }) });
|
||||
launch[colorScheme] = { root: launchWorkDirectory, record };
|
||||
} catch (error) {
|
||||
// The launch is supplementary evidence: a failure to observe it leaves the route capture intact.
|
||||
say(`the app launch could not be observed (${(error as Error).message}); the report has no launch evidence for ${colorScheme}`);
|
||||
await rm(join(launchWorkDirectory, launchDirectory(colorScheme)), { recursive: true, force: true });
|
||||
}
|
||||
} else if (index > 0) runner({ command: controllerCommand, args: ["open", options.bundleId, "--relaunch", "--session", CONTROLLER_SESSION,
|
||||
...deviceArgs(runtime), "--platform", options.platform, "--level", "full", "--json"], cwd: simulatorProject, environment: controllerEnvironment, timeoutMs: 60_000 });
|
||||
runtime.colorScheme = colorScheme;
|
||||
if (colorSchemes.length > 1) say(`appearance ${colorScheme} (${index + 1}/${colorSchemes.length})`);
|
||||
passes.push(await capturePlannedRoutes({ capture: { ...options, ...(argentBridge && !options.agent ? { runner: argentBridge.runner } : {}), agentDeviceCommand: controllerCommand, setup, progress: say }, routes: missingRoutes, inventory, scheme, appName: app.name, runtime,
|
||||
connection, environment: controllerEnvironment, store, map, acceptedFrames, outcomes: index === 0 ? outcomes : {},
|
||||
...(colorSchemes.length > 1 ? { appearance: { scheme: colorScheme, primary: index === 0 } } : {}) }));
|
||||
const passOutcomes: CaptureBundle["outcomes"] = index === 0 ? outcomes : {};
|
||||
const capturePass = async () => {
|
||||
// Argent cannot switch the appearance yet (software-mansion/argent#1509): the host does, simctl or adb on this
|
||||
// machine and serve-sim on an EAS iOS session; elsewhere its single pass runs as the device starts.
|
||||
if (controller !== "argent") runner({ command: controllerCommand, args: [...appearanceArgs(colorScheme), "--session", CONTROLLER_SESSION, ...deviceArgs(activeRuntime), "--platform", options.platform],
|
||||
cwd: simulatorProject, environment: controllerEnvironment, timeoutMs: 60_000 });
|
||||
else if (sessionId && controlActions.has("appearance.set")) simulatorControl(sessionEas, sessionId, "appearance.set", [colorScheme], controlOptions);
|
||||
else if (localSimulator) setLocalSimulatorAppearance({ runner, udid: localSimulator, scheme: colorScheme, cwd: simulatorProject, environment: restrictedEnvironment });
|
||||
else if (localEmulator) setLocalEmulatorAppearance({ runner, serial: localEmulator, scheme: colorScheme, cwd: simulatorProject, environment: restrictedEnvironment });
|
||||
// A scheme change recreates the Android activity in place, and the sheets it re-presents lose their inset; a
|
||||
// relaunch starts the app in the new scheme, the same way the first pass did.
|
||||
if (launchSetting !== "off") {
|
||||
// Observing the launch relaunches the app before setup runs, so no UI-created state is lost.
|
||||
say(`observing the app launch${colorSchemes.length > 1 ? ` (${colorScheme})` : ""}`);
|
||||
try {
|
||||
// A restored baseline's launch evidence is replaced only by a fresh observation.
|
||||
const record = await captureLaunch({ bundleId: options.bundleId, keepVideo: launchSetting === "video", platform: options.platform,
|
||||
directory: join(launchWorkDirectory, launchDirectory(colorScheme)), relativeDirectory: launchDirectory(colorScheme), ...(options.now ? { now: options.now } : {}),
|
||||
...(recordingTools ? { recordingTools } : {}),
|
||||
controller: (args, controllerOptions) => runner({ command: controllerCommand, args: [...args, "--session", CONTROLLER_SESSION, ...deviceArgs(activeRuntime), "--platform", options.platform],
|
||||
cwd: simulatorProject, environment: controllerEnvironment, timeoutMs: controllerOptions?.timeoutMs ?? 60_000, ...(controllerOptions?.allowFailure ? { allowFailure: true } : {}) }) });
|
||||
launch[colorScheme] = { root: launchWorkDirectory, record };
|
||||
} catch (error) {
|
||||
// The launch is supplementary evidence: a failure to observe it leaves the route capture intact.
|
||||
say(`the app launch could not be observed (${(error as Error).message}); the report has no launch evidence for ${colorScheme}`);
|
||||
await rm(join(launchWorkDirectory, launchDirectory(colorScheme)), { recursive: true, force: true });
|
||||
}
|
||||
} else if (index > 0) runner({ command: controllerCommand, args: ["open", options.bundleId, "--relaunch", "--session", CONTROLLER_SESSION,
|
||||
...deviceArgs(activeRuntime), "--platform", options.platform, "--level", "full", "--json"], cwd: simulatorProject, environment: controllerEnvironment, timeoutMs: 60_000 });
|
||||
activeRuntime.colorScheme = colorScheme;
|
||||
if (colorSchemes.length > 1) say(`appearance ${colorScheme} (${index + 1}/${colorSchemes.length})`);
|
||||
return await capturePlannedRoutes({ capture: { ...options, ...(argentBridge && !options.agent ? { runner: argentBridge.runner } : {}), agentDeviceCommand: controllerCommand, setup, progress: say }, routes: missingRoutes, inventory, scheme, appName: app.name, runtime: activeRuntime,
|
||||
connection, environment: controllerEnvironment, store, map, acceptedFrames, outcomes: passOutcomes,
|
||||
...(colorSchemes.length > 1 ? { appearance: { scheme: colorScheme, primary: index === 0 } } : {}) });
|
||||
};
|
||||
// A later pass of a session whose primary pass captured fails only its own appearance, as a secondary session does
|
||||
// when the passes are split over sessions (captureSideInSessions): in brentvatne/euxy's main baseline run
|
||||
// 39506374824 a failed dark session failed a target whose light session had captured 16 of 16 routes.
|
||||
if (index === 0 || !capturedSchemes.includes(colorSchemes[0]!) || options.agent) { passes.push(await capturePass()); continue; }
|
||||
try { passes.push(await capturePass()); }
|
||||
catch (error) {
|
||||
const failure = toCiError(error, { code: `CAPTURE_${runtimeStage.toUpperCase()}_FAILED`, stage: runtimeStage,
|
||||
nextAction: "Inspect the capture diagnostic, correct the failing prerequisite, and rerun Explore." });
|
||||
if (options.signal?.aborted || capturedEvidenceInDoubt(failure.code)) throw failure;
|
||||
const { code, stage, message, nextAction, retryable } = serializeCiError(failure, { secrets: [options.expoToken, ...environmentSecrets(sourceEnvironment)] });
|
||||
say(`the ${colorScheme} pass failed (${code}: ${message}); keeping the ${colorSchemes[0]} captures, and the report marks every ${colorScheme} frame it did not take failed`);
|
||||
sessionFailures.push({ scheme: colorScheme, error: { code, stage, message, nextAction, retryable } });
|
||||
// The routes the pass finished keep what it recorded.
|
||||
passes.push(passOutcomes);
|
||||
}
|
||||
}
|
||||
runtime.colorScheme = colorSchemes[0]!;
|
||||
// A session without the primary pass keeps each outcome as its pass recorded it, under `<id>@<scheme>`: the export
|
||||
@@ -1008,6 +1069,11 @@ async function executeCiMap(options: CiMapOptions, setStage: (stage: string) =>
|
||||
if (options.appearancePass && !capturedSchemes.includes(colorSchemes[0]!)) {
|
||||
for (const [index, pass] of passes.entries()) for (const [id, outcome] of Object.entries(pass)) outcomes[schemeScreenId(id, colorSchemes[index])] = outcome;
|
||||
} else Object.assign(outcomes, mergeSchemeOutcomes(passes, colorSchemes));
|
||||
// A failed pass records its failure under each route it did not finish, as the export merge does for a failed session.
|
||||
for (const { scheme: failedScheme, error } of sessionFailures) {
|
||||
const pass = passes[colorSchemes.indexOf(failedScheme)] ?? {};
|
||||
for (const id of selectedIds) if (!pass[id]) outcomes[schemeScreenId(id, failedScheme)] = { status: "failed", attempts: 0, landing: "unverified", reason: error.message, error };
|
||||
}
|
||||
} catch (error) { primaryError = toCiError(error, { code: `CAPTURE_${runtimeStage.toUpperCase()}_FAILED`, stage: runtimeStage,
|
||||
nextAction: "Inspect the capture diagnostic, correct the failing prerequisite, and rerun Explore." }); }
|
||||
finally {
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { routeIdOfScreen } from "../capture/appearance.js";
|
||||
import { bundleColorSchemes, routeIdOfScreen, schemeScreenId } from "../capture/appearance.js";
|
||||
import { createHash } from "node:crypto";
|
||||
import { cp, lstat, mkdtemp, readFile, readdir, realpath, rm, writeFile } from "node:fs/promises";
|
||||
import { isAbsolute, join, relative, resolve, sep } from "node:path";
|
||||
@@ -15,7 +15,10 @@ import { ARGENT_VERSION } from "../shared/argent-version.js";
|
||||
|
||||
const METADATA = "cache-metadata.json";
|
||||
const CONTRACT = "explore.capture-cache";
|
||||
const MAX_BYTES = 250 * 1024 * 1024;
|
||||
/** The most evidence a capture cache holds; a downloaded baseline is refused past it before it is unpacked. */
|
||||
export const CAPTURE_CACHE_MAX_BYTES = 250 * 1024 * 1024;
|
||||
/** The most files a capture cache holds, besides its metadata. */
|
||||
export const CAPTURE_CACHE_MAX_FILES = 10_000;
|
||||
type Recipe = { inventory: unknown; profile: string; config: string; toolsCommit: string };
|
||||
type FileRecord = { path: string; bytes: number; sha256: string };
|
||||
type CacheMetadata = { contract: typeof CONTRACT; recipe: string; commit: string; files: FileRecord[] };
|
||||
@@ -28,9 +31,9 @@ const recipeKey = ({ inventory, profile, config, toolsCommit }: Recipe) => {
|
||||
|
||||
/** Only settings that can change captured runtime evidence belong in the recipe. */
|
||||
export function captureRecipeConfig(config: ExploreConfig, instructionsSha256?: string): string {
|
||||
// How many sessions capture the evidence (capture.maxSessions) does not change it.
|
||||
const { maxSessions: _maxSessions, ...capture } = config.capture;
|
||||
void _maxSessions;
|
||||
// How many sessions capture the evidence (capture.maxSessions), and how long a review waits for a baseline, do not change it.
|
||||
const { maxSessions: _maxSessions, baselineWaitMinutes: _baselineWaitMinutes, ...capture } = config.capture;
|
||||
void _maxSessions; void _baselineWaitMinutes;
|
||||
return canonicalJson({
|
||||
observer: CAPTURE_OBSERVER_HASH,
|
||||
mode: config.mode,
|
||||
@@ -51,7 +54,8 @@ function requiresAgentAcceptance(recipe: Recipe, bundle: CaptureBundle): boolean
|
||||
catch { return false; }
|
||||
}
|
||||
|
||||
async function validateEvidence(directory: string, bundle: CaptureBundle): Promise<void> {
|
||||
/** Checks an export the way a cache entry is checked before it is written or restored: its provenance, profile and frames. */
|
||||
export async function validateEvidence(directory: string, bundle: CaptureBundle): Promise<void> {
|
||||
const map = await readExportedMap(directory, { checkArtifacts: true });
|
||||
const execution = parseRuntimeExecutionContext(JSON.parse(await readFile(join(directory, "execution.json"), "utf8")));
|
||||
if (!bundle.runtimeProfile || !/^[a-f0-9]{40}$/.test(bundle.gitCommit)
|
||||
@@ -104,12 +108,37 @@ async function fileInventory(directory: string, path = ""): Promise<FileRecord[]
|
||||
else if (info.isFile()) files.push({ path: child, bytes: info.size, sha256: hash(await readFile(join(directory, child))) });
|
||||
else throw new Error("Capture cache contains an unsupported file");
|
||||
}
|
||||
if (files.length > 10_000 || files.reduce((sum, file) => sum + file.bytes, 0) > MAX_BYTES) {
|
||||
if (files.length > CAPTURE_CACHE_MAX_FILES || files.reduce((sum, file) => sum + file.bytes, 0) > CAPTURE_CACHE_MAX_BYTES) {
|
||||
throw new Error("Capture cache exceeds the evidence size limit");
|
||||
}
|
||||
return files;
|
||||
}
|
||||
|
||||
/**
|
||||
* The routes a review can reuse from a capture: a verified frame in every appearance it renders. A baseline whose dark
|
||||
* session failed (brentvatne/euxy's main baseline run 39506374824) holds its light frames and a failed `<route>@dark`;
|
||||
* reusing the light frame alone would leave the review's base with no dark frame, and dark would read as unavailable
|
||||
* because of the base. Such a route is captured by the review's base instead, in every appearance.
|
||||
*/
|
||||
export function reusableSurfaceIds(bundle: CaptureBundle): string[] {
|
||||
const secondarySchemes = bundleColorSchemes(bundle).slice(1);
|
||||
return bundle.selectedSurfaceIds.filter(id => hasVerifiedCapture(bundle, id)
|
||||
&& secondarySchemes.every(scheme => bundle.images[schemeScreenId(id, scheme)] && bundle.outcomes[schemeScreenId(id, scheme)] === undefined));
|
||||
}
|
||||
|
||||
/**
|
||||
* Why a capture of `commit` in `sourceDirectory` is not offered for reuse, when it is there and captured some routes:
|
||||
* none has every appearance (its dark session failed). Undefined otherwise, the cache's usual misses included.
|
||||
*/
|
||||
export async function incompleteCaptureReason(sourceDirectory: string, commit: string): Promise<string | undefined> {
|
||||
try {
|
||||
const bundle = await readCaptureBundle(sourceDirectory);
|
||||
if (bundle.gitCommit !== commit || reusableSurfaceIds(bundle).length || !bundle.selectedSurfaceIds.some(id => hasVerifiedCapture(bundle, id))) return undefined;
|
||||
const failed = (bundle.sessionFailures ?? []).map(failure => failure.scheme);
|
||||
return failed.length ? `its ${failed.join(" and ")} session failed, so no route has every appearance` : "no route was captured in every appearance";
|
||||
} catch { return undefined; }
|
||||
}
|
||||
|
||||
/** Cache full exported evidence, including accessibility maps and all referenced screenshots. */
|
||||
export async function writeCaptureArtifactCache(directory: string, recipe: Recipe): Promise<boolean> {
|
||||
try {
|
||||
@@ -119,6 +148,9 @@ export async function writeCaptureArtifactCache(directory: string, recipe: Recip
|
||||
if (!bundle.runtimeProfile) return false;
|
||||
assertAgentAcceptance(bundle, undefined, { required: requiresAgentAcceptance(recipe, bundle) });
|
||||
await validateEvidence(directory, bundle);
|
||||
// A capture with no route a review could reuse is not offered: a review that found it would wait for it, and capture
|
||||
// its sides in turn instead of together, only to capture every route of the base itself.
|
||||
if (!reusableSurfaceIds(bundle).length) return false;
|
||||
const metadata: CacheMetadata = {
|
||||
contract: CONTRACT, recipe: recipeKey(recipe), commit: bundle.gitCommit, files: await fileInventory(directory),
|
||||
};
|
||||
@@ -178,7 +210,8 @@ export async function restoreCaptureArtifactCache(options: Recipe & {
|
||||
stage = await mkdtemp(`${options.directory}.cache-`);
|
||||
const stagedEvidence = join(stage, "evidence");
|
||||
await cp(source, stagedEvidence, { recursive: true, force: false, errorOnExist: true });
|
||||
const available = options.selectedIds.filter(id => hasVerifiedCapture(bundle, id)
|
||||
const reusable = new Set(reusableSurfaceIds(bundle));
|
||||
const available = options.selectedIds.filter(id => reusable.has(id)
|
||||
&& (!options.captureContexts || options.captureContexts[id] && bundle.captureContexts?.[id]
|
||||
&& canonicalJson(options.captureContexts[id]) === canonicalJson(bundle.captureContexts[id])));
|
||||
if (!available.length) return false;
|
||||
@@ -189,7 +222,8 @@ export async function restoreCaptureArtifactCache(options: Recipe & {
|
||||
const captureContexts = bundle.captureContexts ? Object.fromEntries(available.map(id => [id, bundle.captureContexts![id]])) : undefined;
|
||||
const agentAcceptance = bundle.agentAcceptance ? { policy: bundle.agentAcceptance.policy,
|
||||
frames: Object.fromEntries(available.map(id => [id, bundle.agentAcceptance!.frames[id]!])) } : undefined;
|
||||
await writeFile(join(stagedEvidence, "bundle.json"), JSON.stringify({ ...bundle, side: "base", selectedSurfaceIds: available, images, outcomes,
|
||||
// Only complete routes are restored, so a failed pass of the baseline's is no part of the restored bundle.
|
||||
await writeFile(join(stagedEvidence, "bundle.json"), JSON.stringify({ ...bundle, side: "base", selectedSurfaceIds: available, images, outcomes, sessionFailures: undefined,
|
||||
...(captureContexts ? { captureContexts } : {}), ...(agentAcceptance ? { agentAcceptance } : {}) }));
|
||||
await cp(stagedEvidence, options.directory, { recursive: true, force: false, errorOnExist: true });
|
||||
return true;
|
||||
|
||||
@@ -0,0 +1,203 @@
|
||||
import { openAsBlob } from "node:fs";
|
||||
import { mkdir, mkdtemp, open, rm, writeFile } from "node:fs/promises";
|
||||
import { dirname, join, relative, resolve } from "node:path";
|
||||
import { setTimeout as delay } from "node:timers/promises";
|
||||
import { BlobReader, Uint8ArrayWriter, ZipReader, configure } from "@zip.js/zip.js";
|
||||
import { CiError } from "../shared/ci-errors.js";
|
||||
import { silentProgress, type Progress } from "../shared/progress.js";
|
||||
import { retryGitHubLookup } from "../report/github-lookup.js";
|
||||
import { safeRelativePath } from "../binary/archive.js";
|
||||
import { CAPTURE_CACHE_MAX_BYTES, CAPTURE_CACHE_MAX_FILES } from "./artifact-cache.js";
|
||||
import { baselineLookup, githubContext, trustedBaselineRun, type GithubArtifact, type GithubRun } from "./workflow-steps.js";
|
||||
|
||||
/**
|
||||
* A review that waits for main's baseline of its base commit instead of capturing the base itself. Right after a
|
||||
* merge, the next pull request's review starts while the baseline workflow is still capturing the new base: in
|
||||
* brentvatne/euxy's review run 39348956443 it had started 15 s before, so every target captured its base too, about
|
||||
* 10–14 minutes per target instead of 6, on twice the EAS Simulator sessions. A baseline job takes 6–8 minutes and
|
||||
* head capture 5–8, so `runCiReview` starts head at once and waits here only where base would start.
|
||||
*/
|
||||
|
||||
/** `capture.baselineWaitMinutes` when the config sets none: two baseline jobs' worth, so one queued behind another still counts. */
|
||||
export const DEFAULT_BASELINE_WAIT_MINUTES = 15;
|
||||
const POLL_MS = 20_000;
|
||||
/** GitHub's Actions API lists an artifact a little after its upload (#47); a finished job gets this long for its artifact to appear. */
|
||||
const ARTIFACT_LISTING_GRACE_MS = 60_000;
|
||||
/** A zip of the largest capture cache, stored rather than deflated, plus its headers. */
|
||||
const MAX_ARTIFACT_ZIP_BYTES = CAPTURE_CACHE_MAX_BYTES + 16 * 1024 * 1024;
|
||||
|
||||
type GithubJob = { name: string; status?: string; conclusion?: string | null };
|
||||
|
||||
export type BaselineWaitOptions = {
|
||||
/** The baseline run `ci:find-baseline` named; it is checked again here, by the same trust rules. */
|
||||
run: number; sha: string; target: string;
|
||||
/** The longest the review waits, from its first poll (`capture.baselineWaitMinutes`). */
|
||||
timeoutMs: number; pollMs?: number;
|
||||
environment?: NodeJS.ProcessEnv; transport?: typeof fetch; progress?: Progress; signal?: AbortSignal;
|
||||
now?: () => number; sleep?: (milliseconds: number, signal?: AbortSignal) => Promise<void>;
|
||||
};
|
||||
|
||||
/** A failed GitHub read that may succeed later: no answer or a 5xx. A bad token or a missing permission is not. */
|
||||
const transientGitHubFailure = (error: unknown) => error instanceof CiError && error.retryable;
|
||||
const describe = (error: unknown) => error instanceof CiError ? `${error.code}: ${error.message}` : error instanceof Error ? error.message : String(error);
|
||||
const statusText = (status: string | undefined) => (status ?? "queued").replaceAll("_", " ");
|
||||
|
||||
/**
|
||||
* Polls one baseline run until this target's baseline artifact is listed, and answers with it. The run must pass the
|
||||
* trust check a finished baseline passes. Each target's job uploads its own artifact when it finishes, so a target is
|
||||
* reused even when another target's job fails (#48); this target's job failing or being cancelled ends the wait, as do
|
||||
* the time limit, a missing permission and GitHub failures that outlast their retries. The reason says which.
|
||||
*/
|
||||
export async function waitForBaselineArtifact(options: BaselineWaitOptions): Promise<{ artifact: { id: number; run: number } } | { reason: string }> {
|
||||
const now = options.now ?? Date.now;
|
||||
const sleep = options.sleep ?? ((milliseconds: number, signal?: AbortSignal) => delay(milliseconds, undefined, signal ? { signal } : {}));
|
||||
const progress = options.progress ?? silentProgress;
|
||||
const started = now();
|
||||
const short = options.sha.slice(0, 8);
|
||||
try {
|
||||
const lookup = await baselineLookup({ sha: options.sha, target: options.target, ...(options.environment ? { environment: options.environment } : {}),
|
||||
...(options.transport ? { transport: options.transport } : {}),
|
||||
retry: read => retryGitHubLookup(read, transientGitHubFailure, milliseconds => sleep(milliseconds, options.signal)) });
|
||||
if (!lookup) return { reason: "the repository has no baseline workflow" };
|
||||
const { context, get, target } = lookup;
|
||||
// The matrix names each baseline job after its target; an older single-platform workflow's one job is `baseline`.
|
||||
const jobNames = [`baseline (${target})`, ...(target === "ios" ? ["baseline"] : [])];
|
||||
let finishedAt: number | undefined;
|
||||
let logged = "";
|
||||
for (;;) {
|
||||
const run = await get<GithubRun>(`repos/${context.repository}/actions/runs/${options.run}`);
|
||||
if (!trustedBaselineRun(run, lookup)) return { reason: `run ${options.run} is not a baseline run of ${short} on ${lookup.defaultBranch}` };
|
||||
for (const name of lookup.names) {
|
||||
const listed = await get<{ artifacts?: GithubArtifact[] }>(`repos/${context.repository}/actions/runs/${run.id}/artifacts?name=${encodeURIComponent(name)}&per_page=100`);
|
||||
const artifact = (listed?.artifacts ?? []).filter(candidate => !candidate.expired && (candidate.name ?? name) === name
|
||||
&& Number.isSafeInteger(candidate.id) && candidate.id > 0).sort((a, b) => b.id - a.id)[0];
|
||||
if (artifact) return { artifact: { id: artifact.id, run: run.id } };
|
||||
}
|
||||
const jobs = await get<{ jobs?: GithubJob[] }>(`repos/${context.repository}/actions/runs/${run.id}/jobs?per_page=100`);
|
||||
const job = jobs?.jobs?.find(candidate => jobNames.includes(candidate.name));
|
||||
if (job?.status === "completed" && job.conclusion !== "success") {
|
||||
return { reason: `its ${target} job ${job.conclusion === "cancelled" ? "was cancelled" : job.conclusion === "failure" ? "failed" : `ended (${job.conclusion ?? "no conclusion"})`}` };
|
||||
}
|
||||
if (job?.status === "completed" || run.status === "completed") {
|
||||
finishedAt ??= now();
|
||||
if (now() - finishedAt >= ARTIFACT_LISTING_GRACE_MS) return { reason: job ? `its ${target} job finished, but GitHub lists no ${lookup.names[0]} artifact`
|
||||
: `the run ended (${run.conclusion ?? "no conclusion"}) without a ${target} baseline job` };
|
||||
}
|
||||
const waited = now() - started;
|
||||
const state = job ? statusText(job.status) : run.status === "completed" ? "finished" : "not started";
|
||||
if (waited >= options.timeoutMs) return { reason: `its ${target} job is still ${state} after ${Math.round(waited / 60_000)} min` };
|
||||
// A line when the state changes and once a minute, not on every poll.
|
||||
const line = `waiting for ${lookup.defaultBranch}'s baseline run ${run.id} of ${short} (${target}: ${state}, ${Math.floor(waited / 60_000)} min)`;
|
||||
if (line !== logged) { progress(line); logged = line; }
|
||||
await sleep(Math.min(options.pollMs ?? POLL_MS, options.timeoutMs - waited), options.signal);
|
||||
}
|
||||
} catch (error) {
|
||||
return { reason: options.signal?.aborted ? "the review stopped waiting" : describe(error) };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Downloads a baseline artifact and unpacks it into `directory`, which must not exist yet: what
|
||||
* actions/download-artifact would have put in EXPLORE_CAPTURE_CACHE. The token goes to GitHub only, never to the
|
||||
* storage URL GitHub redirects to. Paths are confined to the directory and the cache's size limits apply before
|
||||
* anything is written; restoreCaptureArtifactCache then checks the evidence as it checks a downloaded one.
|
||||
*/
|
||||
export async function downloadBaselineArtifact(options: { artifact: number; directory: string; environment?: NodeJS.ProcessEnv; transport?: typeof fetch;
|
||||
signal?: AbortSignal; sleep?: (milliseconds: number, signal?: AbortSignal) => Promise<void> }): Promise<void> {
|
||||
const environment = options.environment ?? process.env; const transport = options.transport ?? fetch;
|
||||
const context = githubContext(environment);
|
||||
const sleep = options.sleep ?? ((milliseconds: number, signal?: AbortSignal) => delay(milliseconds, undefined, signal ? { signal } : {}));
|
||||
const signal = (milliseconds: number) => options.signal ? AbortSignal.any([options.signal, AbortSignal.timeout(milliseconds)]) : AbortSignal.timeout(milliseconds);
|
||||
const path = `repos/${context.repository}/actions/artifacts/${options.artifact}/zip`;
|
||||
// A 404 just after the artifact was listed is the Actions API catching up (#47), as a 5xx or no answer may be.
|
||||
const located = await retryGitHubLookup(async () => {
|
||||
let response: Response;
|
||||
try {
|
||||
response = await transport(`https://api.github.com/${path}`, { redirect: "manual", signal: signal(60_000),
|
||||
headers: { authorization: `Bearer ${context.token}`, accept: "application/vnd.github+json", "x-github-api-version": "2022-11-28" } });
|
||||
} catch (error) {
|
||||
throw new CiError({ code: "GITHUB_REQUEST_FAILED", stage: "capture-base", message: `GitHub could not be reached (${path}).`,
|
||||
nextAction: "Check GitHub availability.", retryable: true, cause: error });
|
||||
}
|
||||
if (response.status >= 300 && response.status < 400) {
|
||||
await response.body?.cancel().catch(() => {});
|
||||
return { location: response.headers.get("location") ?? "" };
|
||||
}
|
||||
if (response.ok) return { response };
|
||||
await response.body?.cancel().catch(() => {});
|
||||
throw new CiError({ code: response.status === 403 ? "GITHUB_PERMISSION_DENIED" : "GITHUB_REQUEST_FAILED", stage: "capture-base",
|
||||
message: `GitHub answered HTTP ${response.status} for ${path}.`, retryable: response.status === 404 || response.status >= 500,
|
||||
nextAction: response.status === 403 ? "Grant the capture job actions: read." : "Rerun the workflow; check GitHub's status if it keeps failing." });
|
||||
}, transientGitHubFailure, milliseconds => sleep(milliseconds, options.signal));
|
||||
let response = located.response;
|
||||
if (!response) {
|
||||
const location = URL.canParse(located.location ?? "") ? new URL(located.location!) : undefined;
|
||||
if (location?.protocol !== "https:") throw new Error("GitHub redirected the artifact download to no HTTPS address");
|
||||
response = await transport(location.href, { signal: signal(10 * 60_000) });
|
||||
if (!response.ok || !response.body) throw new Error(`The artifact download answered HTTP ${response.status}`);
|
||||
}
|
||||
await mkdir(dirname(resolve(options.directory)), { recursive: true });
|
||||
const stage = await mkdtemp(`${resolve(options.directory)}.download-`);
|
||||
try {
|
||||
const archive = join(stage, "artifact.zip");
|
||||
const file = await open(archive, "wx", 0o600);
|
||||
try {
|
||||
let bytes = 0;
|
||||
for await (const chunk of response.body as unknown as AsyncIterable<Uint8Array>) {
|
||||
bytes += chunk.byteLength;
|
||||
if (bytes > MAX_ARTIFACT_ZIP_BYTES) throw new Error("The baseline artifact is larger than a capture cache can be");
|
||||
await file.write(chunk);
|
||||
}
|
||||
} finally { await file.close(); }
|
||||
await unpackArtifact(archive, options.directory);
|
||||
} finally {
|
||||
await rm(stage, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
async function unpackArtifact(archive: string, directory: string): Promise<void> {
|
||||
configure({ useWebWorkers: false });
|
||||
const zip = new ZipReader(new BlobReader(await openAsBlob(archive)), { useWebWorkers: false });
|
||||
try {
|
||||
const entries = await zip.getEntries();
|
||||
const files = entries.filter(entry => !entry.directory);
|
||||
// The cache's own metadata file is the one file beyond its evidence.
|
||||
if (files.length > CAPTURE_CACHE_MAX_FILES + 1 || files.reduce((sum, entry) => sum + entry.uncompressedSize, 0) > CAPTURE_CACHE_MAX_BYTES + 2 * 1024 * 1024) {
|
||||
throw new Error("The baseline artifact exceeds the capture cache's limits");
|
||||
}
|
||||
const root = resolve(directory);
|
||||
await mkdir(root);
|
||||
for (const entry of files) {
|
||||
const path = safeRelativePath(entry.filename);
|
||||
const target = path ? resolve(root, path) : undefined;
|
||||
if (!target || relative(root, target).startsWith("..") || !entry.getData) throw new Error(`The baseline artifact holds an unsafe path: ${entry.filename}`);
|
||||
await mkdir(dirname(target), { recursive: true });
|
||||
await writeFile(target, await entry.getData(new Uint8ArrayWriter(), { checkSignature: true, useWebWorkers: false }), { flag: "wx", mode: 0o600 });
|
||||
}
|
||||
} finally {
|
||||
await zip.close();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The wait as `capture:review` runs it: the downloaded baseline's directory, or undefined, and then base captures as it
|
||||
* would have. It never fails the review; each outcome is one progress line.
|
||||
*/
|
||||
export async function awaitPendingBaseline(options: BaselineWaitOptions & { directory: string }): Promise<string | undefined> {
|
||||
const progress = options.progress ?? silentProgress;
|
||||
const short = options.sha.slice(0, 8);
|
||||
const outcome = await waitForBaselineArtifact(options);
|
||||
if ("reason" in outcome) {
|
||||
progress(`base: stopped waiting for the baseline of ${short} (${outcome.reason}); capturing the base too`);
|
||||
return undefined;
|
||||
}
|
||||
try {
|
||||
await downloadBaselineArtifact({ ...options, artifact: outcome.artifact.id });
|
||||
progress(`base: downloaded the baseline of ${short} that run ${outcome.artifact.run} captured`);
|
||||
return options.directory;
|
||||
} catch (error) {
|
||||
await rm(options.directory, { recursive: true, force: true }).catch(() => {});
|
||||
progress(`base: the baseline of ${short} from run ${outcome.artifact.run} could not be downloaded (${options.signal?.aborted ? "the review stopped waiting" : describe(error)}); capturing the base too`);
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
import { Worker, isMainThread, parentPort, workerData } from "node:worker_threads";
|
||||
import { runCiMap, type CiMapOptions, type CiMapResult } from "../capture/run.js";
|
||||
import { CiError, serializeCiError, type CiErrorData } from "../shared/ci-errors.js";
|
||||
import { CiError, environmentSecrets, serializeCiError, type CiErrorData } from "../shared/ci-errors.js";
|
||||
import type { Progress } from "../shared/progress.js";
|
||||
|
||||
type WorkerMessage = { type: "progress"; message: string } | { type: "done"; result: CiMapResult } | { type: "error"; error: CiErrorData };
|
||||
@@ -41,5 +41,5 @@ if (!isMainThread && parentPort) {
|
||||
port.unref();
|
||||
runCiMap({ ...options, signal: controller.signal, progress: message => port.postMessage({ type: "progress", message } satisfies WorkerMessage) })
|
||||
.then(result => port.postMessage({ type: "done", result } satisfies WorkerMessage))
|
||||
.catch((error: unknown) => port.postMessage({ type: "error", error: serializeCiError(error, { secrets: [options.expoToken] }) } satisfies WorkerMessage));
|
||||
.catch((error: unknown) => port.postMessage({ type: "error", error: serializeCiError(error, { secrets: [options.expoToken, ...environmentSecrets(options.environment ?? {})] }) } satisfies WorkerMessage));
|
||||
}
|
||||
+38
-11
@@ -1,7 +1,7 @@
|
||||
import { copyFile, mkdir, readdir, readFile, writeFile, appendFile, lstat } from "node:fs/promises";
|
||||
import { dirname, join, relative } from "node:path";
|
||||
|
||||
import { bundleColorSchemes, mergeSchemeOutcomes, routeIdOfScreen, type ColorScheme } from "../capture/appearance.js";
|
||||
import { bundleColorSchemes, mergeSchemeOutcomes, routeIdOfScreen, schemeScreenId, type ColorScheme } from "../capture/appearance.js";
|
||||
import type { CaptureOutcome } from "../contracts/capture.js";
|
||||
import type { CaptureBundle } from "../contracts/capture.js";
|
||||
import type { ExploreMap } from "../types.js";
|
||||
@@ -22,12 +22,21 @@ async function copyTree(source: string, destination: string, skip: (relativePath
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* A secondary appearance session's result: its export, or the error it failed with, when it left none. In brentvatne/euxy's
|
||||
* main baseline run 39506374824 the dark session's install failed after the light session had captured 16 of 16 routes.
|
||||
*/
|
||||
export type SecondaryCaptureExport = { directory: string; scheme: ColorScheme } | { scheme: ColorScheme; error: NonNullable<CaptureOutcome["error"]> };
|
||||
|
||||
/**
|
||||
* Fold the exports of a side's secondary appearance sessions into its primary export, in place: the frames, outcomes,
|
||||
* launch evidence and map records of each `<id>@<scheme>` screen, and their files. Each session captured its passes as
|
||||
* one run would (`appearancePass`), so records only append; a secondary export never replaces a primary record.
|
||||
* one run would (`appearancePass`), so records only append; a secondary export never replaces a primary record. A session
|
||||
* that failed is listed in the bundle's `sessionFailures`, adds a failed `<id>@<scheme>` outcome carrying its error for
|
||||
* every selected route, and leaves the run partial, so its appearance reads as unavailable with its own reason (a route
|
||||
* the primary lost too would otherwise show the primary's) instead of going missing.
|
||||
*/
|
||||
export async function mergeCaptureExports(primary: string, secondaries: readonly { directory: string; scheme: ColorScheme }[]): Promise<void> {
|
||||
export async function mergeCaptureExports(primary: string, secondaries: readonly SecondaryCaptureExport[]): Promise<void> {
|
||||
if (secondaries.length === 0) return;
|
||||
const bundle = await readJson<CaptureBundle>(join(primary, "bundle.json"));
|
||||
const map = await readJson<ExploreMap>(join(primary, "screen-map.json"));
|
||||
@@ -37,15 +46,28 @@ export async function mergeCaptureExports(primary: string, secondaries: readonly
|
||||
const schemes = bundleColorSchemes(bundle);
|
||||
const passes: Record<string, CaptureOutcome>[] = schemes.map(() => ({}));
|
||||
for (const [id, outcome] of Object.entries(bundle.outcomes)) if (!routeIdOfScreen(id).scheme) passes[0]![id] = outcome;
|
||||
for (const { directory: secondary, scheme } of secondaries) {
|
||||
const other = await readJson<CaptureBundle>(join(secondary, "bundle.json"));
|
||||
const failedSessions: string[] = [];
|
||||
const failedOutcomes: Record<string, CaptureOutcome> = {};
|
||||
for (const secondary of secondaries) {
|
||||
const { scheme } = secondary;
|
||||
if ("error" in secondary) {
|
||||
failedSessions.push(`The ${scheme} session failed (${secondary.error.code}): ${secondary.error.message.replace(/\.+$/, "")}.`);
|
||||
(bundle.sessionFailures ??= []).push({ scheme, error: secondary.error });
|
||||
// A primary that renders no such appearance has no slot for its frames; the failure stays on the bundle alone.
|
||||
if (!schemes.includes(scheme)) continue;
|
||||
for (const id of bundle.selectedSurfaceIds) {
|
||||
failedOutcomes[schemeScreenId(id, scheme)] = { status: "failed", attempts: 0, landing: "unverified", reason: secondary.error.message, error: secondary.error };
|
||||
}
|
||||
continue;
|
||||
}
|
||||
const other = await readJson<CaptureBundle>(join(secondary.directory, "bundle.json"));
|
||||
const pass = passes[schemes.indexOf(scheme)];
|
||||
// A session that could not show its appearance (Argent without appearance.set captures one) has nothing to add, as
|
||||
// one run would have no pass for it.
|
||||
if (!pass && !bundleColorSchemes(other).includes(scheme)) continue;
|
||||
if (!pass) throw new Error(`The ${scheme} session's export is not an appearance of the primary export (${schemes.join(", ")})`);
|
||||
for (const [id, outcome] of Object.entries(other.outcomes)) pass[routeIdOfScreen(id).routeId] = outcome;
|
||||
const otherMap = await readJson<ExploreMap>(join(secondary, "screen-map.json"));
|
||||
const otherMap = await readJson<ExploreMap>(join(secondary.directory, "screen-map.json"));
|
||||
for (const [id, image] of Object.entries(other.images)) if (routeIdOfScreen(id).scheme && !bundle.images[id]) bundle.images[id] = image;
|
||||
for (const [scheme, record] of Object.entries(other.launch ?? {})) { bundle.launch ??= {}; bundle.launch[scheme] ??= record; }
|
||||
for (const [id, context] of Object.entries(other.captureContexts ?? {})) { bundle.captureContexts ??= {}; bundle.captureContexts[id] ??= context; }
|
||||
@@ -61,9 +83,9 @@ export async function mergeCaptureExports(primary: string, secondaries: readonly
|
||||
}
|
||||
}
|
||||
// Files: artifacts and launch evidence are new paths; diagnostics append to the primary's lines.
|
||||
await copyTree(secondary, primary, rel => rel === "bundle.json" || rel === "screen-map.json" || rel === "screen-map.schema.json" || rel === "manifest.json"
|
||||
await copyTree(secondary.directory, primary, rel => rel === "bundle.json" || rel === "screen-map.json" || rel === "screen-map.schema.json" || rel === "manifest.json"
|
||||
|| rel === "execution.json" || rel.startsWith("capture-diagnostics"));
|
||||
const diagnostics = join(secondary, "capture-diagnostics");
|
||||
const diagnostics = join(secondary.directory, "capture-diagnostics");
|
||||
for (const entry of await readdir(diagnostics, { withFileTypes: true }).catch(() => [])) {
|
||||
if (!entry.isFile()) continue;
|
||||
const to = join(primary, "capture-diagnostics", entry.name);
|
||||
@@ -72,12 +94,17 @@ export async function mergeCaptureExports(primary: string, secondaries: readonly
|
||||
else if (!(await lstat(to).catch(() => undefined))) await copyFile(join(diagnostics, entry.name), to);
|
||||
}
|
||||
}
|
||||
// An open gap a secondary pass left makes the run partial, as it does one run's (a complete run has none).
|
||||
if (map.gaps.some(gap => gap.status === "open")) map.run.status = "partial";
|
||||
// An open gap a secondary pass left makes the run partial, as it does one run's (a complete run has none), and so does
|
||||
// a secondary session that failed. A partial run names why, as the map's schema requires.
|
||||
if (map.gaps.some(gap => gap.status === "open") || failedSessions.length) {
|
||||
map.run.status = "partial";
|
||||
map.run.stopReason = [map.run.stopReason ?? (failedSessions.length ? undefined : "Some source targets could not be captured; see bundle outcomes."),
|
||||
...failedSessions].filter(Boolean).join(" ");
|
||||
}
|
||||
const manifest = await readJson<{ files: unknown[] }>(join(primary, "manifest.json"));
|
||||
manifest.files = map.artifacts.map(artifact => ({ path: artifact.path, sha256: artifact.sha256, bytes: artifact.bytes, mimeType: artifact.mimeType }));
|
||||
await writeFile(join(primary, "screen-map.json"), JSON.stringify(map));
|
||||
bundle.outcomes = mergeSchemeOutcomes(passes, schemes);
|
||||
bundle.outcomes = { ...mergeSchemeOutcomes(passes, schemes), ...failedOutcomes };
|
||||
await writeFile(join(primary, "bundle.json"), JSON.stringify(bundle));
|
||||
await writeFile(join(primary, "manifest.json"), `${JSON.stringify(manifest, null, 2)}\n`);
|
||||
}
|
||||
+12
-2
@@ -293,6 +293,17 @@ function sessionStateFiles(output: string): string[] {
|
||||
catch { return []; }
|
||||
}
|
||||
|
||||
/**
|
||||
* A finished target keeps only its report: the captures, maps and checkouts are the bulk of the run. A session file
|
||||
* stays, since it names a session no capture confirmed stopped; the review stops it on its way out (stopRecordedSessions).
|
||||
*/
|
||||
export async function pruneTargetOutput(output: string): Promise<void> {
|
||||
const sessions = new Set(sessionStateFiles(output));
|
||||
for (const entry of await readdir(output)) {
|
||||
if (entry !== "report" && entry !== "report.zip" && !sessions.has(entry)) await rm(join(output, entry), { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
function recordedSessionIds(outputs: string[]): string[] {
|
||||
return outputs.flatMap(output => sessionStateFiles(output).flatMap(name => {
|
||||
try { const id = readFileSync(join(output, name), "utf8").trim(); return /^[A-Za-z0-9][A-Za-z0-9._-]{0,199}$/.test(id) ? [id] : []; } catch { return []; }
|
||||
@@ -526,8 +537,7 @@ export async function runLocalReview(options: LocalReviewOptions): Promise<Local
|
||||
if (side === "head" && head.uncommitted) continue;
|
||||
await keepCapture(join(output, `${side}-export`), captureCacheDirectory(environment, target.id, side === "base" ? base.sha : head.sha));
|
||||
}
|
||||
// The report is all that is kept; the captures, maps and checkouts are the bulk of the run.
|
||||
for (const entry of await readdir(output)) if (entry !== "report" && entry !== "report.zip") await rm(join(output, entry), { recursive: true, force: true });
|
||||
await pruneTargetOutput(output);
|
||||
abort.signal.throwIfAborted();
|
||||
return report;
|
||||
};
|
||||
|
||||
+160
-26
@@ -1,23 +1,25 @@
|
||||
import { systemRegions } from "../contracts/capture.js";
|
||||
import type { RunTarget } from "../types.js";
|
||||
import { execFile, spawnSync } from "node:child_process";
|
||||
import { mkdir, mkdtemp, open, readFile, realpath, rm, writeFile } from "node:fs/promises";
|
||||
import { lstat, mkdir, mkdtemp, open, readFile, realpath, rm, writeFile } from "node:fs/promises";
|
||||
import { join, relative, resolve, isAbsolute } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { setTimeout as delay } from "node:timers/promises";
|
||||
import { promisify } from "node:util";
|
||||
import { buildVerifiedPlan } from "./verified-plan.js";
|
||||
import { runCiMap, type CiMapOptions, type CommandRunner } from "../capture/run.js";
|
||||
import type { RepackShare } from "../capture/eas-build.js";
|
||||
import { resolveColorSchemes, routeIdOfScreen, type ColorScheme } from "../capture/appearance.js";
|
||||
import { baseEnvironment } from "../capture/common.js";
|
||||
import { DEFAULT_MAX_SESSIONS, planSideSessions, sessionShare } from "./session-plan.js";
|
||||
import { mergeCaptureExports } from "./export-merge.js";
|
||||
import { mergeCaptureExports, type SecondaryCaptureExport } from "./export-merge.js";
|
||||
import { abandonSharedBuilds } from "../capture/shared-build.js";
|
||||
import { prepareCaptureReview, readCaptureBundle } from "../review/capture-review.js";
|
||||
import { quarantineReviewContract } from "../contracts/review-validation.js";
|
||||
import { agentCaptureConstraints } from "../capture/agent/source-context.js";
|
||||
import { captureArtifactCacheMayMatch, captureRecipeConfig, restoreCaptureArtifactCache, writeCaptureArtifactCache } from "./artifact-cache.js";
|
||||
import { captureArtifactCacheMayMatch, captureRecipeConfig, incompleteCaptureReason, restoreCaptureArtifactCache, writeCaptureArtifactCache } from "./artifact-cache.js";
|
||||
import { runCiMapOnWorker } from "./capture-worker.js";
|
||||
import { DEFAULT_BASELINE_WAIT_MINUTES, awaitPendingBaseline } from "./baseline-wait.js";
|
||||
import { writeCiReportInput, readAppDirectory } from "../report/artifact.js";
|
||||
import { assessPublicEvidence, readExportedMap, type PublicEvidenceSide } from "../review/evidence-assessment.js";
|
||||
import { parseRuntimeExecutionContext } from "../review/runtime-adapter.js";
|
||||
@@ -28,10 +30,10 @@ import { viewerLinks } from "../report/viewer-links.js";
|
||||
import { DEFAULT_EXPLORE_CONFIG_PATH, captureRepack, type ExploreConfig, type ExplorePlatform } from "../config/explore-config.js";
|
||||
import { prepareCiExploration, type PrepareCiExplorationOptions } from "../capture/agent/exploration.js";
|
||||
import { generateAgentChangeNotes } from "../capture/agent/change-notes.js";
|
||||
import { CiError, serializeCiError, toCiError } from "../shared/ci-errors.js";
|
||||
import { CiError, environmentSecrets, serializeCiError, toCiError } from "../shared/ci-errors.js";
|
||||
import type { SourceCapturePlan } from "../source/capture-plan.js";
|
||||
import { createSourceChangeSummaries } from "../source/change-summary.js";
|
||||
import { assertNoCaptureHealthIssues, partitionCaptureHealthIssues, renderCaptureIssueAnnotation, renderComparisonDiagnosticAnnotation, renderNotesUnavailableAnnotation, renderReviewGapAnnotation, sortedCaptureHealthIssues } from "../capture/health.js";
|
||||
import { assertNoCaptureHealthIssues, capturedEvidenceInDoubt, partitionCaptureHealthIssues, renderCaptureIssueAnnotation, renderComparisonDiagnosticAnnotation, renderNotesUnavailableAnnotation, renderReviewGapAnnotation, sortedCaptureHealthIssues } from "../capture/health.js";
|
||||
import { EXPLORE_PACKAGE, packageBin } from "../shared/package-paths.js";
|
||||
import { agentDeviceLauncher, ensureArgentToolServer, ensureController } from "../shared/controller-cache.js";
|
||||
import { silentProgress, type Progress } from "../shared/progress.js";
|
||||
@@ -80,6 +82,21 @@ function captureOptions(config: ExploreConfig, environment: NodeJS.ProcessEnv, r
|
||||
...(config.capture.simulator ? { simulator: config.capture.simulator } : {}),
|
||||
environment, ...(runner ? { runner } : {}), easCommand: packageBin("eas-cli", "eas"), agentDeviceCommand: agentDeviceLauncher(environment), repackCommand: packageBin("@expo/repack-app", "repack-app") };
|
||||
}
|
||||
/**
|
||||
* The hosted repack this capture shares with the other matrix jobs of its GitHub run attempt (RepackShare): only inside
|
||||
* a GitHub Actions run, whose jobs are the only ones that meet, and only when another target repacks the same app
|
||||
* (`peers`, in config order, from hostedRepackPeers). A run attempt's repacks are never used by another attempt or run,
|
||||
* such as the baseline workflow's: "Re-run failed jobs" after an EAS environment value was fixed rebundles with it.
|
||||
* Each target's `rank` (its place among the peers) staggers how long it waits for another's repack (sharedRepackWaitMs);
|
||||
* a re-run attempt runs only the jobs it re-runs, so its leader may not be one of them, and its stagger starts at once.
|
||||
*/
|
||||
export function repackShareOption(config: ExploreConfig, environment: NodeJS.ProcessEnv, peers: readonly string[] | undefined): { repackShare?: RepackShare } {
|
||||
const target = config.target?.id;
|
||||
if (!peers || peers.length < 2 || !target || !peers.includes(target) || captureRepack(config) !== "eas") return {};
|
||||
const run = environment.GITHUB_RUN_ID ?? "", attempt = environment.GITHUB_RUN_ATTEMPT ?? "1", repository = environment.GITHUB_REPOSITORY ?? "";
|
||||
if (!/^[1-9][0-9]*$/.test(run) || !/^[1-9][0-9]*$/.test(attempt) || !/^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/.test(repository)) return {};
|
||||
return { repackShare: { scope: `${repository}#${run}.${attempt}`, leader: peers[0]!, target, rank: peers.indexOf(target), ...(attempt === "1" ? {} : { rerun: true }) } };
|
||||
}
|
||||
/**
|
||||
* Head and base run in parallel worker threads when nothing ties them together: an agent session is
|
||||
* shared, an injected runner belongs to this thread, and a plausible baseline needs head's runtime profile.
|
||||
@@ -113,14 +130,37 @@ async function expectedColorSchemes(config: ExploreConfig, project: string, envi
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Failures that fail the side even when only a secondary appearance session had them. A session Explore could not
|
||||
* confirm stopped, or whose identity it could not verify, must never be hidden behind a partial report; a checkout that
|
||||
* changed, evidence that failed its integrity checks, or a failure Explore did not classify means the primary's frames
|
||||
* may not be what they claim either.
|
||||
*/
|
||||
const SIDE_FAILURE_CODES = new Set(["SESSION_CLEANUP_FAILED", "SESSION_HANDOFF_UNVERIFIED", "CAPTURE_WORKER_EXITED", "EXPLORE_FAILED"]);
|
||||
|
||||
/**
|
||||
* Whether a secondary appearance session's failure can be reported as that appearance's frames failing, keeping the
|
||||
* primary's: only when the capture reported it itself (a worker that died before its cleanup ran rejects otherwise), its
|
||||
* session is proven stopped (the capture removes its session file only after a confirmed stop), and nothing in it
|
||||
* casts doubt on the primary's evidence.
|
||||
*/
|
||||
async function secondaryFailureKept(error: unknown, sessionStatePath: string | undefined): Promise<boolean> {
|
||||
if (!(error instanceof CiError)) return false;
|
||||
if (SIDE_FAILURE_CODES.has(error.code) || capturedEvidenceInDoubt(error.code)
|
||||
|| (error.details as { allocation?: unknown } | undefined)?.allocation === "unknown") return false;
|
||||
return !(sessionStatePath && await lstat(sessionStatePath).then(() => true, () => false));
|
||||
}
|
||||
|
||||
/**
|
||||
* Run one side's capture: in one session, or when `passes` splits its appearances, one worker session per pass. The
|
||||
* primary pass exports to `mapOptions.outputDirectory`; the others beside it, named by their appearances, then merge.
|
||||
* `runPass` replaces the worker capture of each pass (tests).
|
||||
*/
|
||||
async function captureSideInSessions(mapOptions: CiMapOptions, passes: readonly (readonly ColorScheme[])[] | undefined,
|
||||
progress: Progress, signal: AbortSignal | undefined, worker: boolean): Promise<Awaited<ReturnType<typeof runCiMap>>> {
|
||||
const runOnWorker = (passOptions: CiMapOptions) =>
|
||||
runCiMapOnWorker((({ progress: _progress, runner: _runner, agent: _agent, signal: _signal, ...rest }) => rest)(passOptions), progress, signal);
|
||||
export async function captureSideInSessions(mapOptions: CiMapOptions, passes: readonly (readonly ColorScheme[])[] | undefined,
|
||||
progress: Progress, signal: AbortSignal | undefined, worker: boolean,
|
||||
runPass?: (passOptions: CiMapOptions) => Promise<Awaited<ReturnType<typeof runCiMap>>>): Promise<Awaited<ReturnType<typeof runCiMap>>> {
|
||||
const runOnWorker = runPass ?? ((passOptions: CiMapOptions) =>
|
||||
runCiMapOnWorker((({ progress: _progress, runner: _runner, agent: _agent, signal: _signal, ...rest }) => rest)(passOptions), progress, signal));
|
||||
if (!passes || passes.length <= 1) return worker ? runOnWorker(mapOptions) : runCiMap(mapOptions);
|
||||
const schemes = passes.flat();
|
||||
progress(`${mapOptions.side}: capturing ${passes.map(pass => pass.join("+")).join(" and ")} in separate sessions`);
|
||||
@@ -141,9 +181,29 @@ async function captureSideInSessions(mapOptions: CiMapOptions, passes: readonly
|
||||
}));
|
||||
// The build folder holds the repacked app, which the sessions have installed.
|
||||
if (ownBuildShare) await rm(ownBuildShare, { recursive: true, force: true });
|
||||
for (const outcome of settled) if (outcome.status === "rejected") throw outcome.reason;
|
||||
await mergeCaptureExports(mapOptions.outputDirectory, passes.slice(1).map(pass => ({ directory: `${mapOptions.outputDirectory}-${pass.join("-")}`, scheme: pass[0]! })));
|
||||
return (settled[0] as PromiseFulfilledResult<Awaited<ReturnType<typeof runCiMap>>>).value;
|
||||
// Only the primary appearance's session fails the side, besides a stopped review and a secondary failure that cannot be
|
||||
// kept (secondaryFailureKept). In brentvatne/euxy's main baseline run 39506374824 the light session captured 16 of 16
|
||||
// routes and the dark session's install failed, which failed the whole target and lost its baseline. Such a failure is
|
||||
// kept in the report instead: every frame of its appearance is failed with its error, and the run is partial.
|
||||
const suffixOf = (pass: readonly ColorScheme[]) => `-${pass.join("-")}`;
|
||||
for (const [index, outcome] of settled.entries()) {
|
||||
if (outcome.status !== "rejected" || index === 0) continue;
|
||||
const statePath = mapOptions.sessionStatePath ? `${mapOptions.sessionStatePath}${suffixOf(passes[index]!)}` : undefined;
|
||||
if (signal?.aborted || !await secondaryFailureKept(outcome.reason, statePath)) throw outcome.reason;
|
||||
}
|
||||
const primary = settled[0]!;
|
||||
if (primary.status === "rejected") throw primary.reason;
|
||||
const secrets = [mapOptions.expoToken, ...environmentSecrets(mapOptions.environment ?? {})];
|
||||
const secondaries = passes.slice(1).map((pass, index): SecondaryCaptureExport => {
|
||||
const outcome = settled[index + 1]!;
|
||||
if (outcome.status === "fulfilled") return { directory: `${mapOptions.outputDirectory}${suffixOf(pass)}`, scheme: pass[0]! };
|
||||
const error = serializeCiError(outcome.reason, { secrets });
|
||||
progress(`${mapOptions.side}: the ${pass.join("+")} session failed (${error.code}: ${error.message}); keeping the ${passes[0]!.join("+")} captures, and the report marks every ${pass.join("+")} frame failed`);
|
||||
return { scheme: pass[0]!, error: { code: error.code, stage: error.stage, message: error.message, nextAction: error.nextAction, retryable: error.retryable } };
|
||||
});
|
||||
await mergeCaptureExports(mapOptions.outputDirectory, secondaries);
|
||||
const result = primary.value;
|
||||
return secondaries.some(secondary => "error" in secondary) ? { ...result, runStatus: "partial" } : result;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -173,6 +233,24 @@ export async function sideAppearancePasses(config: ExploreConfig, environment: N
|
||||
return passes.length > 1 ? passes.map(pass => [...pass]) : undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Base's start in a review that waits for a baseline still being captured; head has already started. Base captures as
|
||||
* soon as the wait gives up, beside head when the sides capture concurrently, and a downloaded baseline is restored
|
||||
* once head finishes, since restoring checks it against head's runtime profile. A head that fails ends the wait, and
|
||||
* base is never started for a review that has already failed.
|
||||
*/
|
||||
export async function captureBaseAfterWait(input: { waited: Promise<string | undefined> | undefined; head: Promise<void>; stopWaiting: () => void;
|
||||
captureBase: (cacheDirectory?: string) => Promise<void> }): Promise<void> {
|
||||
if (!input.waited) return input.captureBase();
|
||||
let headFailed = false;
|
||||
input.head.catch(() => { headFailed = true; input.stopWaiting(); });
|
||||
const waited = await input.waited;
|
||||
if (headFailed) return;
|
||||
if (!waited) return input.captureBase();
|
||||
await input.head;
|
||||
return input.captureBase(waited);
|
||||
}
|
||||
|
||||
export function captureSidesConcurrently(input: { plan: SourceCapturePlan; agent: boolean; injectedRunner: boolean; baselineMayExist: boolean }): boolean {
|
||||
return !input.agent && !input.injectedRunner && !input.baselineMayExist
|
||||
&& input.plan.headCaptureSurfaceIds.length > 0 && input.plan.baseCaptureSurfaceIds.length > 0;
|
||||
@@ -202,15 +280,40 @@ export type PublicCiOptions = {
|
||||
targets?: RunTarget[];
|
||||
/** The targets with a device pose (foldables), which take the sessions `capture.maxSessions` does not divide evenly. */
|
||||
posedTargets?: readonly string[];
|
||||
/** The targets whose jobs repack the same app as this one, in config order (hostedRepackPeers): they share one hosted repack per commit. */
|
||||
repackPeers?: readonly string[];
|
||||
baseProject: string; headProject: string; pullRequestPath: string; output: string;
|
||||
config: ExploreConfig; configPath?: string; environment?: NodeJS.ProcessEnv; transport?: typeof fetch;
|
||||
now?: () => number; sleep?: (milliseconds: number) => void; onComment?: (state: PublicCommentState, body: string) => void;
|
||||
runner?: CommandRunner;
|
||||
/** Internal dependency injection; configuration cannot replace the authenticated CLI session. */
|
||||
agentSessionFactory?: PrepareCiExplorationOptions["createSession"];
|
||||
/** Internal dependency injection for tests: replaces the wait for a baseline run that EXPLORE_BASELINE_RUN names. */
|
||||
awaitBaseline?: BaselineWait;
|
||||
progress?: Progress;
|
||||
/** Stops the captures between steps (a cancelled job): a session is still stopped, and a hosted repack run cancelled. */
|
||||
signal?: AbortSignal;
|
||||
};
|
||||
|
||||
/** Waits for a baseline of the base commit still being captured: its downloaded export's directory, or undefined. */
|
||||
type BaselineWait = (signal: AbortSignal) => Promise<string | undefined>;
|
||||
|
||||
/**
|
||||
* The wait for a baseline still capturing the base commit: for the run `ci:find-baseline` named in
|
||||
* EXPLORE_BASELINE_RUN, which the wait checks again by the same trust rules, for up to capture.baselineWaitMinutes. A
|
||||
* workflow that names no run, or a config that sets 0, captures the base as before.
|
||||
*/
|
||||
function pendingBaselineWait(options: Pick<PublicCiOptions, "awaitBaseline" | "config" | "target" | "platform" | "output" | "progress">,
|
||||
environment: NodeJS.ProcessEnv, transport: typeof fetch, baseSha: string): BaselineWait | undefined {
|
||||
if (options.awaitBaseline) return options.awaitBaseline;
|
||||
const run = Number(environment.EXPLORE_BASELINE_RUN);
|
||||
const minutes = options.config.capture.baselineWaitMinutes ?? DEFAULT_BASELINE_WAIT_MINUTES;
|
||||
if (!Number.isSafeInteger(run) || run <= 0 || minutes === 0) return undefined;
|
||||
const target = options.target ?? options.platform ?? options.config.target?.id ?? "ios";
|
||||
return signal => awaitPendingBaseline({ run, sha: baseSha, target, timeoutMs: minutes * 60_000, directory: join(resolve(options.output), "baseline-download"),
|
||||
environment, transport, signal, ...(options.progress ? { progress: options.progress } : {}) });
|
||||
}
|
||||
|
||||
function assertRunIdentity(environment: NodeJS.ProcessEnv): void {
|
||||
if (!/^[1-9][0-9]*$/.test(environment.GITHUB_RUN_ID ?? "") || !/^[1-9][0-9]*$/.test(environment.GITHUB_RUN_ATTEMPT ?? "")) {
|
||||
throw new CiError({ code: "CI_RUN_IDENTITY_MISSING", stage: "preflight", message: "The GitHub run ID or attempt is missing.",
|
||||
@@ -247,13 +350,14 @@ async function prepareCiRun(options: Pick<PublicCiOptions, "baseProject" | "head
|
||||
}
|
||||
|
||||
/** A review of any two commits (`capture:review --compare FILE`): the same capture as a PR's, with no PR to verify or comment on. */
|
||||
export async function runCompareCi(options: Omit<PublicCiOptions, "pullRequestPath" | "transport" | "onComment"> & { comparePath: string }) {
|
||||
export async function runCompareCi(options: Omit<PublicCiOptions, "pullRequestPath" | "onComment"> & { comparePath: string }) {
|
||||
const environment = options.environment ?? process.env;
|
||||
assertRunIdentity(environment);
|
||||
const comparison = projectComparePresentation(JSON.parse(await readFile(options.comparePath, "utf8")));
|
||||
const { projects, output, state } = await prepareCiRun(options, environment, comparison);
|
||||
const assertCurrentHead = async () => { for (const side of ["base", "head"] as const) assertCheckout(projects[side], comparison[side].sha); };
|
||||
return runCiReview({ ...options, projects, output, state, presentation: comparison,
|
||||
const awaitBaseline = pendingBaselineWait(options, environment, options.transport ?? fetch, comparison.base.sha);
|
||||
return runCiReview({ ...options, projects, output, state, presentation: comparison, ...(awaitBaseline ? { awaitBaseline } : {}),
|
||||
environment, runId: environment.GITHUB_RUN_ID!, runAttempt: environment.GITHUB_RUN_ATTEMPT!,
|
||||
assertCurrentHead, postComment: async () => {} });
|
||||
}
|
||||
@@ -324,7 +428,8 @@ export async function runPublicCi(options: PublicCiOptions) {
|
||||
for (const side of ["base", "head"] as const) assertCheckout(projects[side], pr[side].sha);
|
||||
await assertCurrentPr();
|
||||
};
|
||||
return runCiReview({ ...options, projects, output, state, presentation: pr,
|
||||
const awaitBaseline = pendingBaselineWait(options, environment, transport, pr.base.sha);
|
||||
return runCiReview({ ...options, projects, output, state, presentation: pr, ...(awaitBaseline ? { awaitBaseline } : {}),
|
||||
environment, runId: environment.GITHUB_RUN_ID!, runAttempt: environment.GITHUB_RUN_ATTEMPT!,
|
||||
assertCurrentHead, postComment, canPublish: () => !prVerificationFailed });
|
||||
}
|
||||
@@ -348,12 +453,19 @@ export async function runCiReview(options: {
|
||||
posedTargets?: readonly string[];
|
||||
/** A folder shared with the other targets' reviews, so the targets of one platform build each commit once. */
|
||||
buildShare?: string;
|
||||
/** The targets whose matrix jobs repack the same app as this one, in config order: they share one hosted repack per commit. */
|
||||
repackPeers?: readonly string[];
|
||||
measure?: Parameters<typeof assessPublicEvidence>[0]["measure"];
|
||||
state: PublicCommentState; environment: NodeJS.ProcessEnv;
|
||||
assertCurrentHead: () => Promise<void>; postComment: () => Promise<void>; canPublish?: () => boolean;
|
||||
now?: () => number; sleep?: (milliseconds: number) => void; runner?: CommandRunner; subscription?: boolean; useLocalEasSession?: boolean;
|
||||
agentSessionFactory?: PrepareCiExplorationOptions["createSession"];
|
||||
prepareCapture?: () => Promise<void>;
|
||||
/**
|
||||
* Waits for a baseline of the base commit that is still being captured, when no downloaded baseline matches. It runs
|
||||
* alongside head, and base waits for it only where base would start capturing.
|
||||
*/
|
||||
awaitBaseline?: BaselineWait;
|
||||
progress?: Progress;
|
||||
/** Stops the captures between steps; model requests take it through agentSessionFactory. */
|
||||
signal?: AbortSignal;
|
||||
@@ -366,11 +478,13 @@ export async function runCiReview(options: {
|
||||
if (!options.runner) await prefetchControllers(config, environment, progress);
|
||||
const pr = options.presentation;
|
||||
const now = options.now ?? Date.now;
|
||||
const secretValues = Object.entries(environment).filter(([key]) => /(?:TOKEN|SECRET|PASSWORD|API_KEY)/i.test(key)).flatMap(([, value]) => value ? [value] : []);
|
||||
const secretValues = environmentSecrets(environment);
|
||||
let stage = "plan";
|
||||
let usagePhase: AgentUsagePhase = "preflight";
|
||||
let preserveReport: (() => Promise<void>) | undefined;
|
||||
let reportWritten = false;
|
||||
// Stops a baseline wait still polling once the review no longer needs it: done, failed, or stopped.
|
||||
const stopWaiting = new AbortController();
|
||||
try {
|
||||
progress(`planning the review of ${pr.head.sha.slice(0, 8)} against base ${pr.base.sha.slice(0, 8)} (scope ${config.exploration.scope})`);
|
||||
const plan = await buildVerifiedPlan({ baseProject: projects.base, headProject: projects.head,
|
||||
@@ -401,6 +515,10 @@ export async function runCiReview(options: {
|
||||
// A baseline that may be restored needs the head runtime profile first; otherwise both sides capture at once.
|
||||
const baselineMayExist = plan.baseCaptureSurfaceIds.length > 0 && await captureArtifactCacheMayMatch({ ...cacheRecipes.base,
|
||||
...(environment.EXPLORE_CAPTURE_CACHE ? { sourceDirectory: environment.EXPLORE_CAPTURE_CACHE } : {}), commit: pr.base.sha });
|
||||
// With no finished baseline to reuse, one still being captured is waited for: from now, alongside head, and base
|
||||
// waits for it only where it would start. Head is never delayed by it (brentvatne/euxy's review run 39348956443).
|
||||
const waitedBaseline = !baselineMayExist && plan.baseCaptureSurfaceIds.length > 0 && options.awaitBaseline
|
||||
? options.awaitBaseline(options.signal ? AbortSignal.any([stopWaiting.signal, options.signal]) : stopWaiting.signal).catch(() => undefined) : undefined;
|
||||
const share = targetSessionShare(config, options.targets, options.posedTargets);
|
||||
// A share of one session captures the sides in turn.
|
||||
const concurrent = captureSidesConcurrently({ plan, agent: !!exploration.agent, injectedRunner: !!options.runner, baselineMayExist }) && share >= 2;
|
||||
@@ -415,17 +533,20 @@ export async function runCiReview(options: {
|
||||
progress(`sessions: ${share} of ${config.capture.maxSessions ?? DEFAULT_MAX_SESSIONS} for this target; `
|
||||
+ (split.length ? `${split.join(" and ")} ${split.length === 1 ? "captures its" : "capture their"} appearances in separate sessions`
|
||||
: concurrent ? "one session per side" : "the sides in turn"));
|
||||
const captureSide = async (side: "head" | "base") => {
|
||||
const captureSide = async (side: "head" | "base", cacheDirectory = environment.EXPLORE_CAPTURE_CACHE) => {
|
||||
const directory = join(output, `${side}-export`);
|
||||
const selectedIds = plan[`${side}CaptureSurfaceIds`];
|
||||
const cacheRecipe = cacheRecipes[side];
|
||||
let reuseDirectory: string | undefined;
|
||||
if (side === "base" && selectedIds.length && runtimeProfile && environment.EXPLORE_CAPTURE_CACHE) {
|
||||
const restored = await restoreCaptureArtifactCache({ ...cacheRecipe, sourceDirectory: environment.EXPLORE_CAPTURE_CACHE,
|
||||
if (side === "base" && selectedIds.length && runtimeProfile && cacheDirectory) {
|
||||
const restored = await restoreCaptureArtifactCache({ ...cacheRecipe, sourceDirectory: cacheDirectory,
|
||||
commit: pr.base.sha, selectedIds, directory: join(output, "base-cache"), runtimeProfile,
|
||||
...(exploration.agent ? { captureContexts: exploration.agent.contexts } : {}) });
|
||||
if (restored) reuseDirectory = join(output, "base-cache");
|
||||
progress(restored ? `base: found the baseline captures of ${pr.base.sha.slice(0, 8)}` : `base: no compatible baseline of ${pr.base.sha.slice(0, 8)}; capturing the base too`);
|
||||
const incomplete = restored ? undefined : await incompleteCaptureReason(cacheDirectory, pr.base.sha);
|
||||
progress(restored ? `base: found the baseline captures of ${pr.base.sha.slice(0, 8)}`
|
||||
: incomplete ? `base: the baseline of ${pr.base.sha.slice(0, 8)} has nothing to reuse (${incomplete}); capturing the base too`
|
||||
: `base: no compatible baseline of ${pr.base.sha.slice(0, 8)}; capturing the base too`);
|
||||
}
|
||||
const mapOptions = { ...captureOptions(config, environment, options.runner, { now: options.now, sleep: options.sleep }), progress,
|
||||
...(options.useLocalEasSession ? { useLocalEasSession: true } : {}),
|
||||
@@ -433,6 +554,7 @@ export async function runCiReview(options: {
|
||||
mapDirectory: join(output, `${side}-map`), outputDirectory: directory,
|
||||
name: config.app.name ?? pr.repository, buildId: pr.origin === "local" ? `local-${pr.runId}-${side}` : pr.origin === "compare" ? `compare-${options.runId}-${side}` : `pr-${pr.number}-${side}`, buildVersion: pr[side].sha.slice(0, 8), gitCommit: pr[side].sha,
|
||||
planPath: join(output, "execution-plan.json"), side, sessionStatePath: join(output, `${side}-session`), ...(options.buildShare ? { buildShare: options.buildShare } : {}),
|
||||
...repackShareOption(config, environment, options.repackPeers),
|
||||
...(reuseDirectory ? { reuseDirectory } : {}), ...(options.signal ? { signal: options.signal } : {}) };
|
||||
// A restored baseline already holds every appearance pass, so it never splits.
|
||||
const captured = await captureSideInSessions(mapOptions, reuseDirectory ? undefined : sidePasses[side], progress, options.signal, concurrent);
|
||||
@@ -453,12 +575,17 @@ export async function runCiReview(options: {
|
||||
}
|
||||
await writeCaptureArtifactCache(directory, cacheRecipe);
|
||||
};
|
||||
const captureBase = (head: Promise<void>) => captureBaseAfterWait({ waited: waitedBaseline, head, stopWaiting: () => stopWaiting.abort(),
|
||||
captureBase: cacheDirectory => captureSide("base", cacheDirectory) });
|
||||
if (waitedBaseline) progress(`base waits for the baseline run of ${pr.base.sha.slice(0, 8)} still being captured; head captures meanwhile`);
|
||||
if (concurrent) {
|
||||
stage = "capture-head"; state.phase = "capturing"; state.capture.current = "head"; usagePhase = "capture-head";
|
||||
progress("capturing head and base at the same time in two simulator sessions");
|
||||
progress(waitedBaseline ? "capturing head; base captures in a second simulator session if the baseline cannot be reused"
|
||||
: "capturing head and base at the same time in two simulator sessions");
|
||||
await postComment();
|
||||
// Let both sides finish or fail on their own so every started session is stopped by its owner.
|
||||
const settled = await Promise.allSettled([captureSide("head"), captureSide("base")]);
|
||||
const head = captureSide("head");
|
||||
const settled = await Promise.allSettled([head, captureBase(head)]);
|
||||
for (const [index, outcome] of settled.entries()) {
|
||||
if (outcome.status === "rejected") { stage = index === 0 ? "capture-head" : "capture-base"; throw outcome.reason; }
|
||||
}
|
||||
@@ -467,7 +594,7 @@ export async function runCiReview(options: {
|
||||
stage = `capture-${side}`; state.phase = "capturing"; state.capture.current = side;
|
||||
usagePhase = `capture-${side}`;
|
||||
await postComment();
|
||||
await captureSide(side);
|
||||
await (side === "head" ? captureSide("head") : captureBase(Promise.resolve()));
|
||||
}
|
||||
}
|
||||
delete state.capture.current;
|
||||
@@ -564,6 +691,8 @@ export async function runCiReview(options: {
|
||||
try { await postComment(); } catch { /* A stale PR or inaccessible GitHub must not replace the original failure. */ }
|
||||
}
|
||||
throw error;
|
||||
} finally {
|
||||
stopWaiting.abort();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -572,12 +701,16 @@ export async function runBaselineCi(options: { platform?: ExplorePlatform; proje
|
||||
targets?: RunTarget[];
|
||||
/** The targets with a device pose (foldables), which take the sessions `capture.maxSessions` does not divide evenly. */
|
||||
posedTargets?: readonly string[];
|
||||
/** The targets whose baseline jobs repack the same app as this one, in config order: they share one hosted repack. */
|
||||
repackPeers?: readonly string[];
|
||||
now?: () => number; sleep?: (milliseconds: number) => void;
|
||||
agentSessionFactory?: PrepareCiExplorationOptions["createSession"]; progress?: Progress;
|
||||
/** Use the local `eas login` session instead of EXPO_TOKEN (expo-explore doctor --capture). */
|
||||
useLocalEasSession?: boolean;
|
||||
/** Agent mode with the local claude or codex login instead of the provider's API key. */
|
||||
subscription?: boolean }) {
|
||||
subscription?: boolean;
|
||||
/** Stops the capture between steps (a cancelled job): its session is still stopped, and a hosted repack run cancelled. */
|
||||
signal?: AbortSignal }) {
|
||||
const environment = options.environment ?? process.env;
|
||||
assertCheckout(options.project, options.sha);
|
||||
if (!options.runner) await prefetchControllers(options.config, environment, options.progress ?? silentProgress);
|
||||
@@ -602,14 +735,15 @@ export async function runBaselineCi(options: { platform?: ExplorePlatform; proje
|
||||
...(exploration.agent ? { agent: exploration.agent } : {}), project: options.project,
|
||||
mapDirectory: join(output, "baseline-map"), outputDirectory: directory, name: options.config.app.name ?? options.config.app.bundleId,
|
||||
buildId: `baseline-${options.sha.slice(0, 12)}`, buildVersion: options.sha.slice(0, 8), gitCommit: options.sha,
|
||||
side: "head", planPath: join(output, "execution-plan.json"), sessionStatePath: join(output, "baseline-session") };
|
||||
side: "head", planPath: join(output, "execution-plan.json"), sessionStatePath: join(output, "baseline-session"),
|
||||
...repackShareOption(options.config, environment, options.repackPeers), ...(options.signal ? { signal: options.signal } : {}) };
|
||||
const share = targetSessionShare(options.config, options.targets, options.posedTargets);
|
||||
// Passes on worker threads cannot take an injected runner or an agent session.
|
||||
const passes = !options.runner && !exploration.agent
|
||||
? await sideAppearancePasses(options.config, environment, { share, sides: 1, routes: plan.headCaptureSurfaceIds.length, project: options.project }) : undefined;
|
||||
progress(`sessions: ${share} of ${options.config.capture.maxSessions ?? DEFAULT_MAX_SESSIONS} for this target; `
|
||||
+ (passes ? `capturing ${passes.map(pass => pass.join("+")).join(" and ")} in separate sessions` : "one session"));
|
||||
const result = await captureSideInSessions(baselineOptions, passes, progress, undefined, passes !== undefined);
|
||||
const result = await captureSideInSessions(baselineOptions, passes, progress, options.signal, passes !== undefined);
|
||||
const cacheRecipe = await recipe(options.config, plan, "head", environment, exploration.instructionsSha256);
|
||||
if (options.subscription) cacheRecipe.toolsCommit += ":local-subscription";
|
||||
const bundle = await readCaptureBundle(directory); bundle.profile = cacheRecipe.profile;
|
||||
|
||||
+70
-21
@@ -30,7 +30,7 @@ export async function writeCaptureOutputs(inputPath: string, environment: Enviro
|
||||
await writeStepOutputs(environment, { preview: (await readCiReportInput(inputPath)).preview?.hosting ?? "" });
|
||||
}
|
||||
|
||||
function githubContext(environment: Environment) {
|
||||
export function githubContext(environment: Environment) {
|
||||
const repository = environment.GITHUB_REPOSITORY ?? "";
|
||||
const token = environment.GITHUB_TOKEN?.trim();
|
||||
if (!/^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/.test(repository)) throw new CiError({ code: "CI_RUN_IDENTITY_MISSING", stage: "preflight",
|
||||
@@ -41,7 +41,7 @@ function githubContext(environment: Environment) {
|
||||
}
|
||||
|
||||
/** One GitHub REST read; 404 is an answer (undefined), every other failure stops the step with what to do. */
|
||||
async function githubGet<T>(path: string, context: { token: string }, transport: typeof fetch): Promise<T | undefined> {
|
||||
export async function githubGet<T>(path: string, context: { token: string }, transport: typeof fetch): Promise<T | undefined> {
|
||||
let response: Response;
|
||||
try {
|
||||
response = await transport(`https://api.github.com/${path}`, { redirect: "error", signal: AbortSignal.timeout(30_000),
|
||||
@@ -102,60 +102,109 @@ async function trustedAuthors(context: { repository: string; token: string }, tr
|
||||
catch (error) { return { logins: [], invalid: error instanceof Error ? error.message : String(error) }; }
|
||||
}
|
||||
|
||||
type GithubArtifact = { id: number; expired: boolean; workflow_run?: { id?: number } };
|
||||
type GithubRun = { conclusion: string | null; workflow_id: number; head_sha: string; head_branch: string; event: string; head_repository?: { full_name?: string } };
|
||||
export type GithubArtifact = { id: number; name?: string; expired: boolean; workflow_run?: { id?: number } };
|
||||
export type GithubRun = { id: number; status?: string; conclusion: string | null; workflow_id: number; head_sha: string; head_branch: string; event: string;
|
||||
head_repository?: { full_name?: string } };
|
||||
|
||||
/**
|
||||
* `ci:find-baseline`: the newest baseline-workflow artifact of exactly this commit, uploaded from the default branch of
|
||||
* this repository by a finished run. Its id and run go to the step outputs for actions/download-artifact. A run that
|
||||
* failed counts: a target's baseline job uploads its artifact only after every step before it passed, so another
|
||||
* target's failure (one flaky simulator session, as in brentvatne/euxy#172) does not make this target's baseline
|
||||
* unusable. A run still in progress or cancelled does not.
|
||||
* What every baseline lookup starts from: the GitHub context, a checked commit and target, the default branch and the
|
||||
* baseline workflow's id, and the artifact names a baseline of this target may carry. Undefined when the repository
|
||||
* has no baseline workflow.
|
||||
*/
|
||||
export async function findBaselineArtifact(options: { sha: string; target?: string;
|
||||
/** What callers from before targets pass: a platform is the id of its default target. */ platform?: string; environment?: Environment; transport?: typeof fetch }): Promise<{ id: number; run: number } | undefined> {
|
||||
export async function baselineLookup(options: { sha: string; target?: string; platform?: string; environment?: Environment; transport?: typeof fetch;
|
||||
/** Wraps each read, as a wait that polls for minutes retries GitHub's transient failures. */ retry?: <T>(read: () => Promise<T>) => Promise<T> }) {
|
||||
const environment = options.environment ?? process.env; const transport = options.transport ?? fetch;
|
||||
const context = githubContext(environment);
|
||||
if (!SHA.test(options.sha)) throw new CiError({ code: "SHA_INVALID", stage: "preflight", message: "The base commit must be a full SHA.", nextAction: "Pass --sha with a 40-character commit SHA." });
|
||||
const target = options.target ?? options.platform ?? "ios";
|
||||
if (!isTargetId(target)) throw new CiError({ code: "TARGET_INVALID", stage: "preflight", message: `${target} is not a target id.`,
|
||||
nextAction: "Pass --target with an id from targets: in .expo-explore/config.yaml (ios or android without targets:)." });
|
||||
const repository = await githubGet<{ default_branch: string }>(`repos/${context.repository}`, context, transport);
|
||||
const workflow = await githubGet<{ id: number }>(`repos/${context.repository}/actions/workflows/explore-baseline.yml`, context, transport);
|
||||
const read = <T>(path: string) => githubGet<T>(path, context, transport);
|
||||
const get = <T>(path: string) => options.retry ? options.retry(() => read<T>(path)) : read<T>(path);
|
||||
const repository = await get<{ default_branch: string }>(`repos/${context.repository}`);
|
||||
const workflow = await get<{ id: number }>(`repos/${context.repository}/actions/workflows/explore-baseline.yml`);
|
||||
if (!repository || !workflow) return undefined;
|
||||
// Baselines carry the target suffix; target ios also accepts the unsuffixed name older single-platform baseline workflows uploaded.
|
||||
const names = [`explore-baseline-${options.sha}-${target}`, ...(target === "ios" ? [`explore-baseline-${options.sha}`] : [])];
|
||||
return { context, get, target, sha: options.sha, defaultBranch: repository.default_branch, workflow: workflow.id, names };
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a run may supply this commit's baseline: a run of the baseline workflow for exactly this commit, on the
|
||||
* default branch of this repository, started by a push or a manual dispatch. A finished run's artifact is reused, and
|
||||
* a run still capturing is waited for, only when it passes this check; nothing from a pull request's head selects it.
|
||||
*/
|
||||
export function trustedBaselineRun(run: GithubRun | undefined, lookup: { workflow: number; sha: string; defaultBranch: string; context: { repository: string } }): run is GithubRun {
|
||||
return !!run && run.workflow_id === lookup.workflow && run.head_sha === lookup.sha && run.head_branch === lookup.defaultBranch
|
||||
&& ["push", "workflow_dispatch"].includes(run.event) && run.head_repository?.full_name === lookup.context.repository;
|
||||
}
|
||||
|
||||
/**
|
||||
* `ci:find-baseline`: the newest baseline-workflow artifact of exactly this commit, uploaded from the default branch of
|
||||
* this repository by a finished run. Its id and run go to the step outputs for actions/download-artifact. A run that
|
||||
* failed counts: a target's baseline job uploads its artifact only after every step before it passed, so another
|
||||
* target's failure (one flaky simulator session, as in brentvatne/euxy#172) does not make this target's baseline
|
||||
* unusable. A run still in progress or cancelled does not; findPendingBaselineRun finds one still capturing.
|
||||
*/
|
||||
export async function findBaselineArtifact(options: { sha: string; target?: string;
|
||||
/** What callers from before targets pass: a platform is the id of its default target. */ platform?: string; environment?: Environment; transport?: typeof fetch }): Promise<{ id: number; run: number } | undefined> {
|
||||
const lookup = await baselineLookup(options);
|
||||
if (!lookup) return undefined;
|
||||
const { context, get } = lookup;
|
||||
const artifacts: GithubArtifact[] = [];
|
||||
for (const name of names) for (let page = 1; page <= 10; page++) {
|
||||
const listed = await githubGet<{ artifacts: GithubArtifact[] }>(`repos/${context.repository}/actions/artifacts?name=${encodeURIComponent(name)}&per_page=100&page=${page}`, context, transport);
|
||||
for (const name of lookup.names) for (let page = 1; page <= 10; page++) {
|
||||
const listed = await get<{ artifacts: GithubArtifact[] }>(`repos/${context.repository}/actions/artifacts?name=${encodeURIComponent(name)}&per_page=100&page=${page}`);
|
||||
artifacts.push(...listed?.artifacts ?? []);
|
||||
if ((listed?.artifacts.length ?? 0) < 100) break;
|
||||
}
|
||||
for (const artifact of artifacts.filter(candidate => !candidate.expired).sort((a, b) => b.id - a.id)) {
|
||||
if (!artifact.workflow_run?.id) continue;
|
||||
const run = await githubGet<GithubRun>(`repos/${context.repository}/actions/runs/${artifact.workflow_run.id}`, context, transport);
|
||||
if (!run || (run.conclusion !== "success" && run.conclusion !== "failure") || run.workflow_id !== workflow.id || run.head_sha !== options.sha
|
||||
|| run.head_branch !== repository.default_branch || !["push", "workflow_dispatch"].includes(run.event)
|
||||
|| run.head_repository?.full_name !== context.repository) continue;
|
||||
const run = await get<GithubRun>(`repos/${context.repository}/actions/runs/${artifact.workflow_run.id}`);
|
||||
if (!trustedBaselineRun(run, lookup) || (run.conclusion !== "success" && run.conclusion !== "failure")) continue;
|
||||
return { id: artifact.id, run: artifact.workflow_run.id };
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* A baseline run of exactly this commit that has not finished yet (queued, waiting or in progress), passing the same
|
||||
* trust check as a finished one. In brentvatne/euxy's review run 39348956443 main's baseline of the base commit had
|
||||
* started 15 s before the review, so no artifact existed yet. A run already capturing is preferred to one queued
|
||||
* behind it in the baseline's concurrency group, since it finishes first; then the newest.
|
||||
*/
|
||||
export async function findPendingBaselineRun(options: Parameters<typeof baselineLookup>[0]): Promise<{ run: number } | undefined> {
|
||||
const lookup = await baselineLookup(options);
|
||||
if (!lookup) return undefined;
|
||||
const listed = await lookup.get<{ workflow_runs?: GithubRun[] }>(`repos/${lookup.context.repository}/actions/workflows/${lookup.workflow}/runs?head_sha=${lookup.sha}`
|
||||
+ `&branch=${encodeURIComponent(lookup.defaultBranch)}&per_page=100`);
|
||||
const pending = (listed?.workflow_runs ?? []).filter(run => run.status !== "completed" && Number.isSafeInteger(run.id) && run.id > 0 && trustedBaselineRun(run, lookup))
|
||||
.sort((a, b) => Number(b.status === "in_progress") - Number(a.status === "in_progress") || b.id - a.id);
|
||||
return pending[0] ? { run: pending[0].id } : undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* `ci:find-baseline` as the workflow runs it: a baseline is optional, so a lookup that fails (a GitHub 5xx, a rate
|
||||
* limit) is a warning and no baseline, never an error. A `ci:*` error would write explore-diagnostics/error.json,
|
||||
* which the failure comment would then report as the cause of an unrelated failure, possibly on another target.
|
||||
* With no finished baseline, it names a trusted baseline run still capturing the commit as `pending`, which
|
||||
* `capture:review` waits for alongside head.
|
||||
*/
|
||||
export async function findOptionalBaseline(options: Parameters<typeof findBaselineArtifact>[0]):
|
||||
Promise<{ baseline?: { id: number; run: number }; warning?: { code: string; message: string } }> {
|
||||
Promise<{ baseline?: { id: number; run: number }; pending?: { run: number }; warning?: { code: string; message: string } }> {
|
||||
try {
|
||||
const baseline = await findBaselineArtifact(options);
|
||||
return baseline ? { baseline } : {};
|
||||
if (baseline) return { baseline };
|
||||
} catch (error) {
|
||||
return { warning: { code: error instanceof CiError ? error.code : "BASELINE_LOOKUP_FAILED",
|
||||
message: `No baseline is reused, so capture captures the base too: ${error instanceof Error ? error.message : String(error)}` } };
|
||||
}
|
||||
try {
|
||||
const pending = await findPendingBaselineRun(options);
|
||||
return pending ? { pending } : {};
|
||||
} catch (error) {
|
||||
return { warning: { code: error instanceof CiError ? error.code : "BASELINE_LOOKUP_FAILED",
|
||||
message: `No baseline run in progress is waited for, so capture captures the base too: ${error instanceof Error ? error.message : String(error)}` } };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
+37
-9
@@ -15,7 +15,7 @@ import { installCiFiles } from "../ci/workflow-install.js";
|
||||
import { readRouteEntry } from "../ci/route-entry.js";
|
||||
import { buildVerifiedPlan } from "../ci/verified-plan.js";
|
||||
import { formatCaptureCheck, formatDoctorReport, referencedSecretNames, runCaptureCheck, runDoctor } from "../ci/doctor.js";
|
||||
import { readExploreConfig, readExploreInstructions, resolveTargetConfig, configTargets, posedTargets, isTargetId, withRunOverrides, warnIgnoredExpiry, configPlatforms as listPlatforms, EXPLORE_PLATFORMS, DEFAULT_EXPLORE_CONFIG_PATH, type ExplorePlatform } from "../config/explore-config.js";
|
||||
import { readExploreConfig, readExploreInstructions, resolveTargetConfig, configTargets, posedTargets, hostedRepackPeers, isTargetId, withRunOverrides, warnIgnoredExpiry, configPlatforms as listPlatforms, EXPLORE_PLATFORMS, DEFAULT_EXPLORE_CONFIG_PATH, type ExplorePlatform } from "../config/explore-config.js";
|
||||
import { argumentError, output, required, has, flag, flags, nonNegativeInteger, oneOf, positiveInteger } from "./args.js";
|
||||
import { help } from "./help.js";
|
||||
import { CiError } from "../shared/ci-errors.js";
|
||||
@@ -110,13 +110,20 @@ export async function ciPullRequest(): Promise<void> {
|
||||
output({ ok: true, pullRequest: presentation, author });
|
||||
}
|
||||
|
||||
/** `ci:find-baseline --sha SHA [--platform P]`: a reusable baseline artifact of the base commit, if one exists. */
|
||||
/**
|
||||
* `ci:find-baseline --sha SHA [--platform P]`: a reusable baseline artifact of the base commit, if one exists, or else
|
||||
* the baseline run still capturing it (`pending`), which `capture:review` waits for alongside head.
|
||||
*/
|
||||
export async function ciFindBaseline(): Promise<void> {
|
||||
const platform = platformFlag(), target = targetFlag();
|
||||
const { baseline, warning } = await findOptionalBaseline({ sha: required("sha"), ...(platform ? { platform } : {}), ...(target ? { target } : {}) });
|
||||
const { baseline, pending, warning } = await findOptionalBaseline({ sha: required("sha"), ...(platform ? { platform } : {}), ...(target ? { target } : {}) });
|
||||
if (baseline) await writeStepOutputs(process.env, { id: String(baseline.id), run: String(baseline.run) });
|
||||
if (pending) {
|
||||
await writeStepOutputs(process.env, { pending: String(pending.run) });
|
||||
console.error(`No finished baseline of ${required("sha").slice(0, 8)} yet; baseline run ${pending.run} is still capturing it; capture:review waits for it while head captures, up to capture.baselineWaitMinutes.`);
|
||||
}
|
||||
if (warning && process.env.GITHUB_ACTIONS) console.log(`::warning title=${warning.code}::${warning.message.replace(/[\r\n]+/g, " ")}`);
|
||||
output({ ok: true, baseline: baseline ?? null, ...(warning ? { warning } : {}) });
|
||||
output({ ok: true, baseline: baseline ?? null, ...(pending ? { pending } : {}), ...(warning ? { warning } : {}) });
|
||||
}
|
||||
|
||||
/** `ci:compare-refs --project DIR --base REF --head REF [--output FILE]`: the two commits a comparison captures. */
|
||||
@@ -220,6 +227,23 @@ export async function sourceVerify(): Promise<void> {
|
||||
output({ ok: true, deterministic: true, routes: plan.inventorySurfaceIds.length });
|
||||
}
|
||||
|
||||
/**
|
||||
* Stops a CI capture GitHub cancels: the runner sends SIGINT, then SIGTERM. The first stops the captures between steps,
|
||||
* so each stops its session and cancels a hosted repack run it waits on (a blocked eas-cli command gets the signal too
|
||||
* and returns); the second quits at once. Without it a cancelled job left its EAS workflow run repacking.
|
||||
*/
|
||||
function ciInterrupt(): AbortSignal {
|
||||
const controller = new AbortController();
|
||||
const interrupt = (signal: NodeJS.Signals) => {
|
||||
if (controller.signal.aborted) process.exit(130);
|
||||
process.stderr.write(`\nStopping the capture (${signal}): its sessions are stopped and its hosted repack cancelled. A second signal quits now.\n`);
|
||||
controller.abort(new CiError({ code: "CAPTURE_CANCELLED", stage: "capture", message: `The capture was stopped (${signal}).`, nextAction: "Rerun the workflow when you are ready." }));
|
||||
};
|
||||
process.on("SIGINT", interrupt);
|
||||
process.on("SIGTERM", interrupt);
|
||||
return controller.signal;
|
||||
}
|
||||
|
||||
export async function captureReview(): Promise<void> {
|
||||
// Refs without the CI flags: a review on this machine, with your own logins (see runLocalReview).
|
||||
if (!has("base-project") && !has("head-project") && !has("pull-request") && !has("compare")) return localReview();
|
||||
@@ -229,13 +253,15 @@ export async function captureReview(): Promise<void> {
|
||||
const rawConfig = await readExploreConfig(headProject, configPath);
|
||||
// A release comparison's workflow may still pass --expires-in; ci:deploy says once that report.preview.expiresAfter is ignored.
|
||||
if (has("expires-in")) warnIgnoredExpiry("--expires-in");
|
||||
const config = withRunOverrides(resolveTargetConfig(rawConfig, selection), { scope: flag("scope"), expiresIn: flag("expires-in"), controller: flag("controller"), simulator: flag("simulator"), repack: flag("repack") });
|
||||
const overrides = { scope: flag("scope"), expiresIn: flag("expires-in"), controller: flag("controller"), simulator: flag("simulator"), repack: flag("repack") };
|
||||
const config = withRunOverrides(resolveTargetConfig(rawConfig, selection), overrides);
|
||||
if (has("pull-request") === has("compare")) argumentError("Pass --pull-request FILE for a pull request, or --compare FILE for any two commits.", has("compare") ? "compare" : "pull-request");
|
||||
// A matrix job names its target; its artifacts, report and comment section carry the target's id.
|
||||
const named = selection.target !== undefined || selection.platform !== undefined;
|
||||
const common = { ...(named ? { target: config.target!.id } : {}), ...(selection.platform ? { platform: selection.platform } : {}),
|
||||
platforms: listPlatforms(rawConfig), targets: configTargets(rawConfig), posedTargets: posedTargets(rawConfig), baseProject: required("base-project"), headProject,
|
||||
output: required("output"), config, configPath, progress: stderrProgress() };
|
||||
platforms: listPlatforms(rawConfig), targets: configTargets(rawConfig), posedTargets: posedTargets(rawConfig), repackPeers: hostedRepackPeers(rawConfig, config, overrides),
|
||||
baseProject: required("base-project"), headProject,
|
||||
output: required("output"), config, configPath, progress: stderrProgress(), signal: ciInterrupt() };
|
||||
const result = has("compare") ? await runCompareCi({ ...common, comparePath: required("compare") })
|
||||
: await runPublicCi({ ...common, pullRequestPath: required("pull-request") });
|
||||
// The deploy job runs only when the reviewed config asks for a hosted report.
|
||||
@@ -246,8 +272,10 @@ export async function captureReview(): Promise<void> {
|
||||
export async function captureBaseline(): Promise<void> {
|
||||
const project = required("project");
|
||||
const rawConfig = await readExploreConfig(project, flag("config") ?? DEFAULT_EXPLORE_CONFIG_PATH);
|
||||
const config = withRunOverrides(resolveTargetConfig(rawConfig, targetSelection()), { controller: flag("controller"), simulator: flag("simulator"), repack: flag("repack") });
|
||||
output({ ok: true, ...await runBaselineCi({ platform: config.app.platform, project, sha: required("sha"), output: required("output"), config, targets: configTargets(rawConfig), posedTargets: posedTargets(rawConfig), progress: stderrProgress() }) });
|
||||
const overrides = { controller: flag("controller"), simulator: flag("simulator"), repack: flag("repack") };
|
||||
const config = withRunOverrides(resolveTargetConfig(rawConfig, targetSelection()), overrides);
|
||||
output({ ok: true, ...await runBaselineCi({ platform: config.app.platform, project, sha: required("sha"), output: required("output"), config, targets: configTargets(rawConfig), posedTargets: posedTargets(rawConfig),
|
||||
repackPeers: hostedRepackPeers(rawConfig, config, overrides), progress: stderrProgress(), signal: ciInterrupt() }) });
|
||||
}
|
||||
|
||||
/** `capture` (or `capture:review`) `[--base REF] [--head REF | --committed]`: a local review, opened in the Viewer or hosted with --upload. */
|
||||
|
||||
@@ -95,6 +95,7 @@ const CI = `CI (the installed workflows run these; they manage remote EAS Simula
|
||||
(resolves the live PR and refuses it unless it is open, from this repository, and by a trusted author)
|
||||
ci:compare-refs --project DIRECTORY --base REF --head REF [--output FILE]
|
||||
ci:find-baseline --sha COMMIT [--target ID | --platform ios|android]
|
||||
(a finished baseline's artifact of the commit, or else the default branch's baseline run still capturing it)
|
||||
ci:after-capture --output TEMP_DIRECTORY --checkout DIRECTORY...
|
||||
(stops the capture's EAS Simulator sessions and checks capture left each checkout unchanged)
|
||||
(the ci:* commands write their results to $GITHUB_OUTPUT for the next steps)`;
|
||||
|
||||
@@ -99,6 +99,11 @@ export type ExploreConfig = {
|
||||
* DEFAULT_MAX_SESSIONS (5). Below the number of targets, that many target jobs run at once, one session each.
|
||||
*/
|
||||
maxSessions?: number;
|
||||
/**
|
||||
* How many minutes a review waits for the default branch's baseline run of its base commit, still capturing when
|
||||
* the review starts, before it captures the base itself; 0 never waits. Absent: DEFAULT_BASELINE_WAIT_MINUTES (15).
|
||||
*/
|
||||
baselineWaitMinutes?: number;
|
||||
/** Where the route observer is bundled into the app, when the config chooses; captureRepack gives the default. */
|
||||
repack?: "eas" | "local"; appearance: AppearanceSetting; launch: LaunchSetting; reset: RouteResetSetting; unattributed: UnattributedSetting; readinessSeconds?: number;
|
||||
/** The device controller; absent means agent-device, so configs that never chose one keep their cache recipes. */
|
||||
@@ -194,6 +199,11 @@ function positiveInteger(value: unknown, label: string, fallback?: number): numb
|
||||
return value;
|
||||
}
|
||||
|
||||
function boundedInteger(value: unknown, label: string, minimum: number, maximum: number): number {
|
||||
if (typeof value !== "number" || !Number.isSafeInteger(value) || value < minimum || value > maximum) fail(`${label} must be a whole number from ${minimum} to ${maximum}`, label);
|
||||
return value;
|
||||
}
|
||||
|
||||
function customScheme(value: unknown, label: string): string {
|
||||
const result = text(value, label, 128).toLowerCase();
|
||||
if (!/^[a-z][a-z0-9+.-]*$/.test(result) || RESERVED_SCHEMES.has(result)) {
|
||||
@@ -300,7 +310,7 @@ export function validateExploreConfig(value: unknown): ExploreConfig {
|
||||
const exploration = optionalRecord(root.exploration, "exploration");
|
||||
exactKeys(exploration, ["scope", "exclude"], "exploration");
|
||||
const capture = optionalRecord(root.capture, "capture");
|
||||
exactKeys(capture, ["maxMinutes", "maxActions", "maxSessions", "repack", "appearance", "launch", "reset", "unattributed", "readinessSeconds", "controller", "simulator"], "capture");
|
||||
exactKeys(capture, ["maxMinutes", "maxActions", "maxSessions", "baselineWaitMinutes", "repack", "appearance", "launch", "reset", "unattributed", "readinessSeconds", "controller", "simulator"], "capture");
|
||||
const controller = literal(capture.controller, ["agent-device", "argent"] as const, "capture.controller", "agent-device");
|
||||
const simulator = literal(capture.simulator, ["eas", "local"] as const, "capture.simulator", "eas");
|
||||
const mode = literal(root.mode, ["deterministic", "agent"] as const, "mode", "deterministic");
|
||||
@@ -348,6 +358,7 @@ export function validateExploreConfig(value: unknown): ExploreConfig {
|
||||
maxMinutes: positiveInteger(capture.maxMinutes, "capture.maxMinutes", 30),
|
||||
maxActions: positiveInteger(capture.maxActions, "capture.maxActions", 500),
|
||||
...(capture.maxSessions === undefined ? {} : { maxSessions: positiveInteger(capture.maxSessions, "capture.maxSessions", 2) }),
|
||||
...(capture.baselineWaitMinutes === undefined ? {} : { baselineWaitMinutes: boundedInteger(capture.baselineWaitMinutes, "capture.baselineWaitMinutes", 0, 60) }),
|
||||
...(capture.repack === undefined ? {} : { repack: literal(capture.repack, ["eas", "local"] as const, "capture.repack") }),
|
||||
appearance: parseAppearance(capture.appearance, "capture.appearance"),
|
||||
launch: parseLaunchSetting(capture.launch, "capture.launch"),
|
||||
@@ -560,6 +571,19 @@ export function captureRepack(config: ExploreConfig): "eas" | "local" {
|
||||
return config.capture.repack ?? (config.capture.simulator === "local" ? "local" : "eas");
|
||||
}
|
||||
|
||||
/**
|
||||
* The targets, in config order, whose captures repack the same app as `selected` on EAS: one platform, EAS profile and
|
||||
* bundle id. Their matrix jobs make one hosted repack per commit between them, led by the first (RepackShare); a target
|
||||
* no other one shares an app with gets just itself.
|
||||
*/
|
||||
export function hostedRepackPeers(config: ExploreConfig, selected: ExploreConfig, overrides: Parameters<typeof withRunOverrides>[1] = {}): string[] {
|
||||
if (captureRepack(selected) !== "eas" || !selected.target) return [];
|
||||
const sameApp = (other: ExploreConfig) => other.app.platform === selected.app.platform && other.app.easProfile === selected.app.easProfile
|
||||
&& other.app.bundleId === selected.app.bundleId && captureRepack(other) === "eas";
|
||||
return configTargets(config).filter(target => target.id === selected.target!.id || sameApp(withRunOverrides(resolveTargetConfig(config, { target: target.id }), overrides)))
|
||||
.map(target => target.id);
|
||||
}
|
||||
|
||||
const GITHUB_LOGIN = /^[A-Za-z0-9](?:[A-Za-z0-9]|-(?=[A-Za-z0-9])){0,38}(?:\[bot\])?$/;
|
||||
export const TRUSTED_AUTHORS_MAX = 20;
|
||||
|
||||
|
||||
@@ -93,6 +93,12 @@ export type CaptureBundle = {
|
||||
}>;
|
||||
/** The exact observed frames explicitly accepted by the agent, never replacement screenshots. */
|
||||
agentAcceptance?: { policy: typeof AGENT_ACCEPTANCE_POLICY; frames: Record<string, AgentAcceptedFrame> };
|
||||
/**
|
||||
* The appearance passes that failed while the primary one captured (a secondary session that failed, or a later pass of
|
||||
* one session), each with its error. Every selected route then has a failed `<id>@<scheme>` outcome carrying it, when
|
||||
* the bundle renders that appearance. Absent when every pass finished.
|
||||
*/
|
||||
sessionFailures?: Array<{ scheme: "light" | "dark"; error: NonNullable<CaptureOutcome["error"]> }>;
|
||||
};
|
||||
|
||||
/** Bind acceptance receipts to captured images and, when supplied, their exported observations. */
|
||||
|
||||
@@ -321,21 +321,34 @@ export function platformRecordOf(state: PublicCommentState, target?: CommentTarg
|
||||
if (notCompared) record.notCompared = notCompared;
|
||||
record.context = splashLines(state.review);
|
||||
const diagnostics = [...(state.evidence?.diagnostics ?? [])];
|
||||
// A failed appearance pass is one issue, named first, rather than one per frame it left unavailable: in brentvatne/
|
||||
// euxy's main baseline run 39506374824 the dark session's failure would have listed all 16 routes, crowding out the rest.
|
||||
const sessionFailures = state.evidence?.sessionFailures ?? [];
|
||||
if (sessionFailures.length) record.sessionFailures = sessionFailures.map(({ side, scheme, frames }) => ({ side, scheme, frames }));
|
||||
const sessionLines = sessionFailures.map(failure => ({ side: failure.side, route: `${failure.scheme} session`, code: failure.code,
|
||||
message: `${failure.message.replace(/\.+$/, "")}.${failure.frames ? ` ${plural(failure.frames, `${failure.scheme} frame`)} ${failure.frames === 1 ? "is" : "are"} unavailable.` : ""}`,
|
||||
nextAction: failure.nextAction }));
|
||||
const ofFailedPass = (surfaceId: string, side: "base" | "head", outcome: NonNullable<Surface["headOutcome"]>) => sessionFailures.some(failure => failure.side === side
|
||||
&& schemeOf(surfaceId) === failure.scheme && outcome.error?.code === failure.code && outcome.error.message === failure.message);
|
||||
for (const surface of state.review.surfaces) {
|
||||
for (const side of ["base", "head"] as const) {
|
||||
const outcome = surface[`${side}Outcome`];
|
||||
if (!outcome || (outcome.status === "captured" && outcome.landing === "verified") || diagnostics.some(item => item.surfaceId === surface.id && item.side === side)) continue;
|
||||
if (ofFailedPass(surface.id, side, outcome)) continue;
|
||||
diagnostics.push({ surfaceId: surface.id, side, code: outcome.error?.code ?? (outcome.status === "captured" ? "CAPTURE_LANDING_UNVERIFIED" : "CAPTURE_UNAVAILABLE"),
|
||||
stage: outcome.error?.stage ?? "capture", message: outcome.error?.message ?? outcome.reason ?? (outcome.status === "captured" ? "Destination route was not verified." : `Capture ${outcome.status}.`),
|
||||
nextAction: outcome.error?.nextAction ?? "Check this side's capture diagnostics, correct the route configuration or capture failure, and rerun capture.", retryable: outcome.error?.retryable ?? false });
|
||||
}
|
||||
}
|
||||
if (diagnostics.length) {
|
||||
record.diagnostics = diagnostics.slice(0, 20).map(diagnostic => {
|
||||
if (diagnostics.length || sessionLines.length) {
|
||||
record.diagnostics = [...sessionLines, ...diagnostics.map(diagnostic => {
|
||||
const surface = state.review!.surfaces.find(surface => surface.id === diagnostic.surfaceId);
|
||||
return { ...(diagnostic.side ? { side: diagnostic.side } : {}), route: surface ? routeOf(surface) : diagnostic.surfaceId, code: diagnostic.code, message: diagnostic.message, nextAction: diagnostic.nextAction };
|
||||
});
|
||||
record.diagnosticCount = diagnostics.length;
|
||||
// A secondary appearance's issue names its appearance, as its change row does: its light frame may be fine.
|
||||
const scheme = schemeOf(diagnostic.surfaceId);
|
||||
return { ...(diagnostic.side ? { side: diagnostic.side } : {}), route: `${surface ? routeOf(surface) : diagnostic.surfaceId}${surface && scheme ? ` (${scheme})` : ""}`,
|
||||
code: diagnostic.code, message: diagnostic.message, nextAction: diagnostic.nextAction };
|
||||
})].slice(0, 20);
|
||||
record.diagnosticCount = sessionLines.length + diagnostics.length;
|
||||
}
|
||||
}
|
||||
if (state.failure) record.failure = { step: state.failure.step, message: state.failure.message };
|
||||
@@ -480,9 +493,11 @@ function title(records: PlatformRecord[], rows: ChangeRow[]): string {
|
||||
}
|
||||
// With screens left uncompared, nothing changed only among the ones compared.
|
||||
const summary = rows.length ? `${plural(rows.length, "screen")} changed` : ready.some(record => gaps(record).length) ? "No compared screens changed" : "No screens changed";
|
||||
// An appearance a pass never captured is said in the heading too: the verdict above says nothing about it.
|
||||
const missing = [...new Set(ready.flatMap(record => (record.sessionFailures ?? []).map(failure => failure.scheme)))].map(scheme => `${scheme} not captured`);
|
||||
const report = firstReport(ready);
|
||||
const open = report ? link("Review in Explore", report) : ready.some(record => record.links.deploying) ? "report deploying…" : undefined;
|
||||
return `### ${summary}${open ? ` · ${open}` : ""}`;
|
||||
return `### ${[summary, ...missing, ...(open ? [open] : [])].join(" · ")}`;
|
||||
}
|
||||
|
||||
/** Every target by name, each linking its own report, then the commits and when the comment last changed. */
|
||||
@@ -512,12 +527,21 @@ function statusLine(record: PlatformRecord): string | undefined {
|
||||
}
|
||||
}
|
||||
|
||||
/** "dark not captured" for an appearance whose pass failed on head (or both sides), "… on base" when only the base's did. */
|
||||
function missingAppearances(record: PlatformRecord): string[] {
|
||||
const failures = record.sessionFailures ?? [];
|
||||
return [...new Set(failures.map(failure => failure.scheme))].map(scheme => {
|
||||
const sides = new Set(failures.filter(failure => failure.scheme === scheme).map(failure => failure.side));
|
||||
return `${scheme} not captured${sides.size === 1 && sides.has("base") ? " on base" : ""}`;
|
||||
});
|
||||
}
|
||||
|
||||
/** What a finished target has no verdict on: captures missing or unattributed, comparisons that did not run or failed. */
|
||||
function gaps(record: PlatformRecord): string[] {
|
||||
const unavailable = record.health ? record.health.base.unavailable + record.health.head.unavailable : 0;
|
||||
const unverified = record.health ? record.health.base.unverified + record.health.head.unverified : 0;
|
||||
const failed = (record.validation?.visual?.failed ?? 0) + (record.validation?.accessibility?.failed ?? 0);
|
||||
return [unavailable ? `${plural(unavailable, "capture")} unavailable` : undefined, unverified ? `${unverified} unattributed` : undefined,
|
||||
return [...missingAppearances(record), unavailable ? `${plural(unavailable, "capture")} unavailable` : undefined, unverified ? `${unverified} unattributed` : undefined,
|
||||
record.notCompared ? `${record.notCompared} not compared` : undefined, failed ? `${plural(failed, "comparison")} failed` : undefined]
|
||||
.filter((part): part is string => Boolean(part));
|
||||
}
|
||||
|
||||
@@ -115,6 +115,8 @@ export type PlatformRecord = {
|
||||
navigation?: CommentNavigation;
|
||||
diagnostics?: { side?: "base" | "head"; route: string; code: string; message: string; nextAction: string }[];
|
||||
diagnosticCount?: number;
|
||||
/** Appearance passes that failed while the primary one captured: that appearance was not captured on that side. */
|
||||
sessionFailures?: { side: "base" | "head"; scheme: "light" | "dark"; frames: number }[];
|
||||
failure?: { step: string; message: string };
|
||||
/** The PR moved while this run worked; its report stays reachable but it is not the current review. */
|
||||
superseded?: boolean;
|
||||
@@ -287,6 +289,11 @@ export function parsePlatformRecord(value: unknown): PlatformRecord | undefined
|
||||
}
|
||||
if (isCount(value.notCompared) && value.notCompared > 0) record.notCompared = value.notCompared;
|
||||
if (isCount(value.diagnosticCount)) record.diagnosticCount = value.diagnosticCount;
|
||||
if (Array.isArray(value.sessionFailures)) {
|
||||
const failures = value.sessionFailures.flatMap((entry): NonNullable<PlatformRecord["sessionFailures"]> => isRecord(entry) && isSide(entry.side)
|
||||
&& (entry.scheme === "light" || entry.scheme === "dark") && isCount(entry.frames) ? [{ side: entry.side, scheme: entry.scheme, frames: entry.frames }] : []).slice(0, 4);
|
||||
if (failures.length) record.sessionFailures = failures;
|
||||
}
|
||||
if (isRecord(value.failure) && isString(value.failure.step, 200) && isString(value.failure.message)) record.failure = { step: value.failure.step, message: value.failure.message };
|
||||
if (value.superseded === true) record.superseded = true;
|
||||
return record;
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import type { RunTarget } from "../types.js";
|
||||
import { appearancePairs, bundleColorSchemes, routeIdOfScreen } from "../capture/appearance.js";
|
||||
import { appearancePairs, bundleColorSchemes, failedPassOutcome, routeIdOfScreen } from "../capture/appearance.js";
|
||||
import { createHash } from "node:crypto";
|
||||
import { lstat, readFile, realpath } from "node:fs/promises";
|
||||
import { isAbsolute, relative, resolve, sep } from "node:path";
|
||||
@@ -95,9 +95,11 @@ function secondaryPassOutcome(bundle: CaptureBundle, selected: ReadonlySet<strin
|
||||
const { routeId, scheme } = routeIdOfScreen(id);
|
||||
const own = bundle.outcomes[id];
|
||||
const verified = own?.status === "captured" && own.landing === "verified";
|
||||
return scheme !== undefined && selected.has(routeId) && verified !== (bundle.outcomes[routeId]?.status === "captured");
|
||||
return scheme !== undefined && selected.has(routeId)
|
||||
&& (verified !== (bundle.outcomes[routeId]?.status === "captured") || failedPassOutcome(bundle, id));
|
||||
}
|
||||
|
||||
|
||||
export async function readCaptureBundle(directory: string): Promise<CaptureBundle> {
|
||||
const bundle = JSON.parse(await readFile(resolve(await realpath(directory), "bundle.json"), "utf8")) as CaptureBundle;
|
||||
if (!bundle || bundle.contract !== "explore.bundle" || typeof bundle.gitCommit !== "string"
|
||||
@@ -110,6 +112,11 @@ export async function readCaptureBundle(directory: string): Promise<CaptureBundl
|
||||
throw new Error("Capture bundle completion timestamp is invalid");
|
||||
}
|
||||
if (bundle.appIcon !== undefined) assertAppIconRecord(bundle.appIcon);
|
||||
if (bundle.sessionFailures !== undefined && (!Array.isArray(bundle.sessionFailures) || bundle.sessionFailures.some(failure => !failure
|
||||
|| !["light", "dark"].includes(failure.scheme) || !failure.error || typeof failure.error.code !== "string" || typeof failure.error.message !== "string"
|
||||
|| typeof failure.error.stage !== "string" || typeof failure.error.nextAction !== "string" || typeof failure.error.retryable !== "boolean"))) {
|
||||
throw new Error("Capture bundle session failures are invalid");
|
||||
}
|
||||
if (bundle.launch !== undefined) assertLaunchRecords(bundle.launch);
|
||||
const selected = new Set(bundle.selectedSurfaceIds);
|
||||
if (selected.size !== bundle.selectedSurfaceIds.length
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { unattributedIssue } from "../capture/health.js";
|
||||
import { appearancePairs, bundleColorSchemes, type ColorScheme } from "../capture/appearance.js";
|
||||
import { appearancePairs, bundleColorSchemes, failedPassOutcome, type ColorScheme } from "../capture/appearance.js";
|
||||
import { launchSurfaceId, systemRegions as defaultSystemRegions, type SystemRegion } from "../contracts/capture.js";
|
||||
import { readFileSync, statSync } from "node:fs";
|
||||
import { stat } from "node:fs/promises";
|
||||
@@ -79,8 +79,16 @@ export type PublicEvidenceSummary = {
|
||||
comparison: { planned: number; eligible: number; skipped: number };
|
||||
/** Every unavailable or unverified requested side, plus operational comparison errors. */
|
||||
diagnostics: PublicEvidenceDiagnostic[];
|
||||
/**
|
||||
* Each appearance pass of a side that failed while its primary one captured (the bundle's `sessionFailures`), with how
|
||||
* many of its frames the failure left unavailable: the comment names it once, not once per frame.
|
||||
*/
|
||||
sessionFailures?: PublicSessionFailure[];
|
||||
};
|
||||
|
||||
export type PublicSessionFailure = { side: "base" | "head"; scheme: ColorScheme; frames: number;
|
||||
code: string; stage: string; message: string; nextAction: string; retryable: boolean };
|
||||
|
||||
const MAX_SUMMARY_ERRORS = 5;
|
||||
|
||||
function recordError(errors: string[], surfaceId: string, error: unknown): void {
|
||||
@@ -756,6 +764,9 @@ export async function assessPublicEvidence(options: {
|
||||
}
|
||||
}
|
||||
|
||||
const sessionFailures = (["base", "head"] as const).flatMap(side => (options[side].bundle.sessionFailures ?? []).map(({ scheme, error }): PublicSessionFailure => ({
|
||||
side, scheme, frames: Object.keys(options[side].bundle.outcomes).filter(id => id.endsWith(`@${scheme}`) && failedPassOutcome(options[side].bundle, id)
|
||||
&& options[side].bundle.outcomes[id]!.error?.message === error.message).length, ...error })));
|
||||
return {
|
||||
surfaces,
|
||||
summary: {
|
||||
@@ -763,6 +774,7 @@ export async function assessPublicEvidence(options: {
|
||||
accessibility: accessibilityAvailable ? accessibilityCounts : null,
|
||||
review: reviewCounts,
|
||||
capture, comparison, diagnostics,
|
||||
...(sessionFailures.length ? { sessionFailures } : {}),
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -55,6 +55,11 @@ export function ciErrorForCommand(error: unknown, command: string): CiError {
|
||||
|
||||
export type CiErrorRenderOptions = { secrets?: readonly string[] };
|
||||
|
||||
/** The values of every variable that names a secret (a token, secret, password or API key), to redact from what Explore writes. */
|
||||
export function environmentSecrets(environment: NodeJS.ProcessEnv): string[] {
|
||||
return Object.entries(environment).filter(([key]) => /(?:TOKEN|SECRET|PASSWORD|API_KEY)/i.test(key)).flatMap(([, value]) => value ? [value] : []);
|
||||
}
|
||||
|
||||
function redact(value: string, secrets: readonly string[]): string {
|
||||
let result = value;
|
||||
// Controller failures may embed JSON or URLs inside their message rather than
|
||||
|
||||
@@ -143,6 +143,10 @@ jobs:
|
||||
set -euo pipefail
|
||||
npm ci --ignore-scripts --prefix "app-base/$APP_INSTALL_DIR"
|
||||
npm ci --ignore-scripts --prefix "app-head/$APP_INSTALL_DIR"
|
||||
# A finished baseline of the base commit is downloaded here. With none, ci:find-baseline names a baseline run still
|
||||
# capturing it (pending): right after a merge the next review starts while main's baseline runs. Capture review
|
||||
# checks that run again (this repository's baseline workflow, the default branch, a push or manual dispatch, exactly
|
||||
# the base commit) and waits for its artifact while head captures, for up to capture.baselineWaitMinutes.
|
||||
- name: Find optional exact-base evidence
|
||||
id: cache
|
||||
continue-on-error: true
|
||||
@@ -164,6 +168,8 @@ jobs:
|
||||
id: capture
|
||||
env:
|
||||
COMPARE: ${{ needs.targets.outputs.compare }}
|
||||
# Reads the pending baseline run's jobs and artifact (actions: read); a comparison makes no other GitHub request.
|
||||
GITHUB_TOKEN: ${{ github.token }}
|
||||
SCOPE: ${{ inputs.scope }}
|
||||
EXPIRES_IN: ${{ inputs.expires-in }}
|
||||
EXPO_TOKEN: ${{ secrets.EXPO_TOKEN }}
|
||||
@@ -172,6 +178,7 @@ jobs:
|
||||
ANTHROPIC_WORKSPACE_ID: ${{ vars.ANTHROPIC_WORKSPACE_ID }}
|
||||
# App-test secrets: one NAME: line per {{secret:NAME}} input in .expo-explore (expo-explore setup adds them).
|
||||
EXPLORE_CAPTURE_CACHE: ${{ runner.temp }}/explore-cache
|
||||
EXPLORE_BASELINE_RUN: ${{ steps.cache.outputs.pending }}
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
@@ -133,6 +133,10 @@ jobs:
|
||||
set -euo pipefail
|
||||
npm ci --ignore-scripts --prefix "app-base/$APP_INSTALL_DIR"
|
||||
npm ci --ignore-scripts --prefix "app-head/$APP_INSTALL_DIR"
|
||||
# A finished baseline of the base commit is downloaded here. With none, ci:find-baseline names a baseline run still
|
||||
# capturing it (pending): right after a merge the next review starts while main's baseline runs. Capture review
|
||||
# checks that run again (this repository's baseline workflow, the default branch, a push or manual dispatch, exactly
|
||||
# the base commit) and waits for its artifact while head captures, for up to capture.baselineWaitMinutes.
|
||||
- name: Find optional exact-base evidence
|
||||
id: cache
|
||||
continue-on-error: true
|
||||
@@ -161,6 +165,7 @@ jobs:
|
||||
ANTHROPIC_WORKSPACE_ID: ${{ vars.ANTHROPIC_WORKSPACE_ID }}
|
||||
# App-test secrets: one NAME: line per {{secret:NAME}} input in .expo-explore (expo-explore setup adds them).
|
||||
EXPLORE_CAPTURE_CACHE: ${{ runner.temp }}/explore-cache
|
||||
EXPLORE_BASELINE_RUN: ${{ steps.cache.outputs.pending }}
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
@@ -159,7 +159,12 @@ const Navigation=createStaticNavigation(Root); export default function App(){ret
|
||||
assert.ok(existsSync(join(request.cwd, '.expo-explore-observer/metro.config.cjs')), 'the staged copy carries the observer');
|
||||
assert.ok(!existsSync(join(request.cwd, '.git')), 'the staged copy has no repository');
|
||||
repackedSha = /build_id: "build-([0-9a-f]+)"/.exec(workflow)[1];
|
||||
return json({ jobs: [{ key: 'repack', status: 'SUCCESS', outputs: { build_id: `repacked-${repackedSha}`, platform } }] });
|
||||
// Explore waits on the run itself; the run's id carries the commit its repack is of.
|
||||
return json({ id: `run-${repackedSha}`, url: `https://expo.dev/accounts/acme/projects/example/workflows/run-${repackedSha}` });
|
||||
}
|
||||
if (command === 'workflow:view') {
|
||||
const sha = /^run-([0-9a-f]+)$/.exec(request.args[1])[1];
|
||||
return json({ id: request.args[1], status: 'SUCCESS', jobs: [{ id: `job-${sha}`, key: 'repack', status: 'SUCCESS', outputs: { build_id: `repacked-${sha}`, platform } }] });
|
||||
}
|
||||
if (command === 'simulator:start') {
|
||||
const id = `session-${++sessionCounter}`; currentSession = id; openFailed = false;
|
||||
|
||||
@@ -403,6 +403,24 @@ test("review captures and reports each configured target in its own matrix job,
|
||||
}
|
||||
});
|
||||
|
||||
test("a review waits for the baseline run ci:find-baseline names, with the token's Actions read and nothing the PR head chooses", () => {
|
||||
for (const workflow of [review, compare]) {
|
||||
const steps = workflow.jobs.capture.steps;
|
||||
const find = steps.find(step => step.name === "Find optional exact-base evidence");
|
||||
const capture = steps.find(step => step.name === "Capture review");
|
||||
assert.equal(find["continue-on-error"], true, "a failed lookup never fails the review");
|
||||
assert.ok(steps.indexOf(find) < steps.indexOf(capture));
|
||||
// The run is the find step's output, chosen by the base commit and the default branch's baseline workflow; capture
|
||||
// checks it again before it waits.
|
||||
assert.equal(capture.env.EXPLORE_BASELINE_RUN, "${{ steps.cache.outputs.pending }}");
|
||||
assert.equal(capture.env.GITHUB_TOKEN, "${{ github.token }}", "the wait reads the run, its jobs and its artifact");
|
||||
assert.equal(workflow.jobs.capture.permissions.actions, "read");
|
||||
assert.equal(workflow.jobs.capture.permissions.contents, "read");
|
||||
}
|
||||
assert.deepEqual(compare.jobs.capture.permissions, { contents: "read", actions: "read" }, "the wait needs no permission a comparison did not have");
|
||||
assert.deepEqual(review.jobs.capture.permissions, { contents: "read", actions: "read", "pull-requests": "write" });
|
||||
});
|
||||
|
||||
test("compare reviews any two refs from a manual run or another workflow, with no pull request access", () => {
|
||||
for (const trigger of ["workflow_dispatch", "workflow_call"]) {
|
||||
assert.deepEqual(Object.keys(compare.on[trigger].inputs).sort(), ["base", "expires-in", "head", "scope"]);
|
||||
|
||||
@@ -4,7 +4,11 @@ import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import test from "node:test";
|
||||
|
||||
import { createServer } from "node:http";
|
||||
import { startArgentBridgeRunner } from "../../dist/capture/argent-bridge-host.js";
|
||||
import { argentClientCall } from "../../dist/capture/argent-bridge-worker.js";
|
||||
import { argentRunFailureDetail } from "../../dist/capture/argent-bridge.js";
|
||||
import { runBridgeCommand } from "../../dist/capture/argent-driver.js";
|
||||
import { ARGENT_VERSION } from "../../dist/shared/argent-version.js";
|
||||
|
||||
const PNG = Buffer.from("iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=", "base64");
|
||||
@@ -133,3 +137,48 @@ test("a recording runs per command, and a screenshot is taken only from a file t
|
||||
assert.match(failed.error.message, /did not write/);
|
||||
await assert.rejects(readFile(join(root, "out", "two.png")), { code: "ENOENT" });
|
||||
});
|
||||
|
||||
test("the thread's lines about a tool-server still starting reach the capture's log with the command's answer", async (t) => {
|
||||
const root = await mkdtemp(join(tmpdir(), "explore-argent-bridge-")); t.after(() => rm(root, { recursive: true, force: true }));
|
||||
// Argent's client asks GET /tools before each call; the first one is answered 404, as on euxy's main baseline run 39506374824.
|
||||
const clientModule = await fakeClient(root, `
|
||||
if (name === "list-devices" && calls.filter(call => call.name === "list-devices").length === 1) throw new Error("GET /tools failed: 404 Not Found");
|
||||
if (name === "list-devices") return { data: { devices: [{ udid: "UDID-1", name: "iPhone 17", state: "Booted", platform: "ios", runtime: "iOS 26.0" }] } };
|
||||
if (name === "await-screen-idle") return { data: { settled: true } };
|
||||
throw new Error("unexpected tool " + name);`);
|
||||
const messages = [];
|
||||
const environment = { EXPLORE_ARGENT_STATE: join(root, "state.json"), EXPLORE_ARGENT_PLATFORM: "ios", ARGENT_TOOLS_URL: "http://127.0.0.1:9/tools", HOME: root, PATH: process.env.PATH };
|
||||
const bridge = startArgentBridgeRunner({ command: "/opt/argent-controller", environment, clientModule, progress: message => messages.push(message),
|
||||
fallback: () => assert.fail("the thread serves every command") });
|
||||
t.after(() => bridge.close());
|
||||
const waited = JSON.parse(bridge.runner({ command: "/opt/argent-controller", args: ["wait", "stable", "--session", "explore", "--platform", "ios", "--json"], cwd: root, environment, timeoutMs: 20_000 }).stdout);
|
||||
assert.deepEqual(waited, { success: true, data: { stable: true } });
|
||||
assert.deepEqual(messages, ["Argent's tool-server is not serving its tools yet (argent run list-devices failed: GET /tools failed: 404 Not Found); sending list-devices again in 1 s (75 s left of 75 s)"]);
|
||||
});
|
||||
|
||||
test("a refused connection to a localhost tool-server, whose cause is an AggregateError with no message, is the tool-server still starting", async () => {
|
||||
// A port nothing listens on: bound, then closed.
|
||||
const port = await new Promise(resolve => { const server = createServer().listen(0, "127.0.0.1", () => { const { port } = server.address(); server.close(() => resolve(port)); }); });
|
||||
const client = { async callTool() { const response = await fetch(`http://localhost:${port}/tools`); return { data: await response.json() }; } };
|
||||
const call = argentClientCall(client, () => new AbortController().signal);
|
||||
const refused = await call("list-devices", {}).then(() => assert.fail("nothing listens"), error => error);
|
||||
assert.match(refused.message, /^argent run list-devices failed: fetch failed: .*ECONNREFUSED/);
|
||||
// Through the bridge, the refusal is waited out like a 404 from /tools, until the window ends.
|
||||
let now = 0, tries = 0;
|
||||
const result = await runBridgeCommand(["snapshot", "--platform", "ios", "--json"], { state: { generation: 0 }, platform: "ios", version: "0.29.0",
|
||||
now: () => now, sleep: async milliseconds => { now += milliseconds; }, reachable: async () => true, moveFile: async () => {},
|
||||
call: async (tool, args, options) => { tries++; return call(tool, args, options); } });
|
||||
assert.equal(JSON.parse(result.stdout).error.code, "TOOL_SERVER_NOT_READY");
|
||||
assert.equal(tries, 11);
|
||||
});
|
||||
|
||||
test("a failed argent run keeps the line saying /tools is not served ahead of the stack trace that follows it", () => {
|
||||
const stderr = ["[argent] Unhandled rejection: Error: GET /tools failed: 404 Not Found", " at fetchTools (file:///argent/dist/cli-cmds.mjs:10065:21)",
|
||||
" at async fetchTool (file:///argent/dist/cli-cmds.mjs:10070:19)", " at async run (file:///argent/dist/cli-cmds.mjs:31166:17)",
|
||||
" at async main (file:///argent/dist/cli.js:12:3)"].join("\n");
|
||||
const detail = argentRunFailureDetail(stderr, "");
|
||||
assert.ok(detail.startsWith("[argent] Unhandled rejection: Error: GET /tools failed: 404 Not Found | at fetchTools "), detail);
|
||||
assert.ok(detail.endsWith("at async main (file:///argent/dist/cli.js:12:3)"), "the last frames follow it");
|
||||
assert.equal(argentRunFailureDetail("Tool \"gesture-tap\" not found. Run `argent tools` to list available tools.\n", ""), "Tool \"gesture-tap\" not found. Run `argent tools` to list available tools.");
|
||||
assert.equal(argentRunFailureDetail("GET /tools failed: 503 Service Unavailable\n", ""), "GET /tools failed: 503 Service Unavailable", "a line named once");
|
||||
});
|
||||
@@ -302,6 +302,87 @@ test("an unreachable tool-server before a UI action is the retryable 'Remote dae
|
||||
assert.ok(!calls.some(call => call.tool === "gesture-tap"));
|
||||
});
|
||||
|
||||
// brentvatne/euxy's main baseline run 39506374824: a replacement session's tool-server answered 404 to /tools for the
|
||||
// first list-devices of its install, before it served its tools.
|
||||
const TOOLS_404 = "argent run list-devices failed: GET /tools failed: 404 Not Found";
|
||||
|
||||
/** A clock the harness's pauses (and, given `requestMs`, its requests) move forward. */
|
||||
function withClock(context, requestMs = () => 0) {
|
||||
const clock = { now: 0, pauses: [], timeouts: [] };
|
||||
context.now = () => clock.now;
|
||||
context.sleep = async milliseconds => { clock.pauses.push(milliseconds); clock.now += milliseconds; };
|
||||
const call = context.call;
|
||||
context.call = async (tool, args, options) => {
|
||||
if (options?.timeoutMs !== undefined) clock.timeouts.push(options.timeoutMs);
|
||||
clock.now += Math.min(requestMs(tool), options?.timeoutMs ?? Infinity);
|
||||
return call(tool, args, options);
|
||||
};
|
||||
return clock;
|
||||
}
|
||||
|
||||
test("a tool-server that does not serve /tools yet is asked again, with a log line, until it does", async () => {
|
||||
const { run, calls, context } = harness("ios", { "list-devices": [new ArgentCallError(TOOLS_404, true), new ArgentCallError(TOOLS_404, true), IOS_DEVICES], "reinstall-app": {} }, { bundleId: BUNDLE });
|
||||
const clock = withClock(context);
|
||||
const lines = [];
|
||||
context.progress = line => lines.push(line);
|
||||
assert.deepEqual((await run(["install", "/work/bundle/euxy.app", ...flags("ios")])).json.data, { installed: true, appBundleId: BUNDLE });
|
||||
assert.equal(calls.filter(call => call.tool === "list-devices").length, 3);
|
||||
assert.deepEqual(clock.pauses, [1_000, 2_000]);
|
||||
assert.deepEqual(lines, [
|
||||
"Argent's tool-server is not serving its tools yet (argent run list-devices failed: GET /tools failed: 404 Not Found); sending list-devices again in 1 s (75 s left of 75 s)",
|
||||
"Argent's tool-server is not serving its tools yet (argent run list-devices failed: GET /tools failed: 404 Not Found); sending list-devices again in 2 s (74 s left of 75 s)"]);
|
||||
assert.equal(context.state.toolsServed, true);
|
||||
assert.deepEqual(clock.timeouts, [75_000, 74_000, 72_000], "each try gets only what is left of the window");
|
||||
assert.equal(calls.filter(call => call.tool === "reinstall-app")[0].options, undefined, "a served tool-server's calls keep their own timeout");
|
||||
});
|
||||
|
||||
test("a refused connection or a 503 from /tools is the tool-server still starting too", async () => {
|
||||
for (const failure of ["fetch failed: connect ECONNREFUSED 127.0.0.1:9", "GET /tools failed: 503 Service Unavailable"]) {
|
||||
const starting = new ArgentCallError(`argent run list-devices failed: ${failure}`, true);
|
||||
const { run, calls, context } = harness("ios", { "list-devices": [starting, starting, starting, IOS_DEVICES], "ui-tree": IOS_HOME });
|
||||
withClock(context);
|
||||
assert.equal((await run(["snapshot", ...flags("ios")])).exitCode, 0, failure);
|
||||
assert.equal(calls.filter(call => call.tool === "list-devices").length, 4, "asked past the two read retries");
|
||||
}
|
||||
});
|
||||
|
||||
test("a tool-server that never serves /tools fails with a clear error 75 s after the first try", async () => {
|
||||
const { run, calls, context } = harness("ios", { "list-devices": new ArgentCallError(TOOLS_404, true), "reinstall-app": {} }, { bundleId: BUNDLE });
|
||||
const clock = withClock(context);
|
||||
const failed = await run(["install", "/work/bundle/euxy.app", ...flags("ios")]);
|
||||
assert.deepEqual(failed.json.error, { code: "TOOL_SERVER_NOT_READY", message: `Argent's tool-server did not start serving its tools within 75 s: ${TOOLS_404}` });
|
||||
assert.equal(clock.now, 75_000);
|
||||
assert.equal(calls.filter(call => call.tool === "list-devices").length, 11);
|
||||
assert.ok(!calls.some(call => call.tool === "reinstall-app"));
|
||||
});
|
||||
|
||||
test("slow answers count against the 75 s too: each try is cut to the time left, and the wait ends on time", async () => {
|
||||
const slow = new ArgentCallError("argent run list-devices failed: GET /tools failed: 503 Service Unavailable", true);
|
||||
const { run, calls, context } = harness("ios", { "list-devices": slow, "reinstall-app": {} }, { bundleId: BUNDLE });
|
||||
// Each 503 takes 30 s to come back, as a gateway's might.
|
||||
const clock = withClock(context, () => 30_000);
|
||||
assert.equal((await run(["install", "/work/bundle/euxy.app", ...flags("ios")])).json.error.code, "TOOL_SERVER_NOT_READY");
|
||||
assert.equal(clock.now, 75_000, "not eleven 30 s tries");
|
||||
assert.deepEqual(clock.timeouts, [75_000, 44_000, 12_000]);
|
||||
assert.equal(calls.filter(call => call.tool === "list-devices").length, 3);
|
||||
});
|
||||
|
||||
test("an unknown tool's 404 after the tool-server is up fails at once, and a later /tools 404 keeps only the read retries", async () => {
|
||||
const unknown = 'argent run gesture-tap failed: Tool "gesture-tap" not found';
|
||||
const { run, calls, context } = harness("ios", { "list-devices": IOS_DEVICES, "ui-tree": IOS_HOME, "gesture-tap": new ArgentCallError(unknown, true) });
|
||||
const pauses = [];
|
||||
context.sleep = async milliseconds => { pauses.push(milliseconds); };
|
||||
assert.deepEqual((await run(["press", "10", "10", ...flags("ios")])).json.error, { code: "ARGENT_TOOL_FAILED", message: unknown });
|
||||
assert.equal(calls.filter(call => call.tool === "gesture-tap").length, 1);
|
||||
assert.deepEqual(pauses, []);
|
||||
const served = harness("ios", { "list-devices": IOS_DEVICES, "ui-tree": new ArgentCallError("argent run ui-tree failed: GET /tools failed: 404 Not Found", true) },
|
||||
{ state: { generation: 0, udid: UDID, toolsServed: true } });
|
||||
served.context.sleep = async milliseconds => { pauses.push(milliseconds); };
|
||||
assert.equal((await served.run(["snapshot", ...flags("ios")])).json.error.code, "ARGENT_TOOL_FAILED");
|
||||
assert.equal(served.calls.filter(call => call.tool === "ui-tree").length, 3);
|
||||
assert.deepEqual(pauses, [1_000, 2_000]);
|
||||
});
|
||||
|
||||
test("a UI action whose answer is lost is reported once and never retried", async () => {
|
||||
const { run, calls } = harness("ios", { "list-devices": IOS_DEVICES, "ui-tree": IOS_HOME, "gesture-tap": new ArgentCallError("argent run gesture-tap failed: socket hang up", true) });
|
||||
const pressed = await run(["press", "10", "10", ...flags("ios")]);
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { readFile, readdir } from 'node:fs/promises';
|
||||
import { join } from 'node:path';
|
||||
import { Uint8ArrayReader, Uint8ArrayWriter, ZipWriter, configure } from '@zip.js/zip.js';
|
||||
import { runPublicCi, runBaselineCi, captureBaseAfterWait } from '../../dist/ci/pipeline.js';
|
||||
import { readCaptureBundle } from '../../dist/review/capture-review.js';
|
||||
import { fixture } from '../helpers/public-ci-fixture.mjs';
|
||||
|
||||
const deferred = () => { let resolve, reject; const promise = new Promise((done, fail) => { resolve = done; reject = fail; }); return { promise, resolve, reject }; };
|
||||
const tick = () => new Promise(done => setImmediate(done));
|
||||
|
||||
/** The files of a baseline export, zipped as actions/upload-artifact uploads them. */
|
||||
async function zipDirectory(directory) {
|
||||
configure({ useWebWorkers: false });
|
||||
const writer = new ZipWriter(new Uint8ArrayWriter(), { useWebWorkers: false });
|
||||
for (const entry of await readdir(directory, { recursive: true, withFileTypes: true })) {
|
||||
if (!entry.isFile()) continue;
|
||||
const path = join(entry.parentPath, entry.name);
|
||||
await writer.add(path.slice(directory.length + 1), new Uint8ArrayReader(await readFile(path)));
|
||||
}
|
||||
return writer.close();
|
||||
}
|
||||
|
||||
test('base starts the moment the wait gives up, beside head, and a waited baseline is restored only once head has finished', async () => {
|
||||
const events = [];
|
||||
const run = ({ head, waited }) => captureBaseAfterWait({ waited: waited?.promise, head: head.promise, stopWaiting: () => events.push('stop waiting'),
|
||||
captureBase: async cache => { events.push(cache ? `base from ${cache}` : 'base captures'); } });
|
||||
// The baseline job failed while head was still capturing: base captures at once, in its own session.
|
||||
let head = deferred(), waited = deferred();
|
||||
let base = run({ head, waited });
|
||||
waited.resolve(undefined); await tick();
|
||||
assert.deepEqual(events, ['base captures'], 'base did not wait for head');
|
||||
head.resolve(); await base;
|
||||
// The baseline arrived first: restoring it needs head's runtime profile, so base starts after head.
|
||||
events.length = 0; head = deferred(); waited = deferred();
|
||||
base = run({ head, waited });
|
||||
waited.resolve('/runner/baseline-download'); await tick();
|
||||
assert.deepEqual(events, []);
|
||||
head.resolve(); await base;
|
||||
assert.deepEqual(events, ['base from /runner/baseline-download']);
|
||||
// Head failed while base still waited: the wait stops, and base never starts for a failed review.
|
||||
events.length = 0; head = deferred(); waited = deferred();
|
||||
base = run({ head, waited });
|
||||
head.reject(new Error('head capture failed')); await tick();
|
||||
assert.deepEqual(events, ['stop waiting']);
|
||||
waited.resolve(undefined); await base;
|
||||
assert.deepEqual(events, ['stop waiting'], 'base was not captured');
|
||||
// Without a wait, base starts as before.
|
||||
events.length = 0;
|
||||
await captureBaseAfterWait({ waited: undefined, head: new Promise(() => {}), stopWaiting: () => events.push('stop waiting'), captureBase: async () => { events.push('base captures'); } });
|
||||
assert.deepEqual(events, ['base captures']);
|
||||
});
|
||||
|
||||
test('head capture starts while the review waits for the baseline, and the waited baseline is restored for base', async t => {
|
||||
const f = await fixture(t, 'expo-router');
|
||||
t.mock.method(globalThis, 'fetch', async input => assert.fail(`Unmocked network request: ${input}`));
|
||||
const baseline = await runBaselineCi({ project: f.baseProject, sha: f.baseSha, output: join(f.root, 'baseline'), config: f.config, environment: f.environment, runner: f.runner, now: f.now, sleep: f.sleep });
|
||||
const before = f.requests.length;
|
||||
// The baseline finishes only once head capture has started a simulator session; were head held back by the wait,
|
||||
// the fallback would end it with no baseline after 20 s, and base would capture.
|
||||
const finished = deferred();
|
||||
let finishedBy;
|
||||
const fallback = setTimeout(() => { finishedBy ??= 'timeout'; finished.resolve(undefined); }, 20_000);
|
||||
t.after(() => clearTimeout(fallback));
|
||||
const runner = request => {
|
||||
if (request.args[0] === 'simulator:start' && !finishedBy) { finishedBy = 'head started'; setImmediate(() => finished.resolve(baseline.directory)); }
|
||||
return f.runner(request);
|
||||
};
|
||||
let waitSignal;
|
||||
const progress = [];
|
||||
const output = join(f.root, 'review');
|
||||
await runPublicCi({ baseProject: f.baseProject, headProject: f.headProject, pullRequestPath: f.pullRequestPath, output, config: f.config,
|
||||
environment: f.environment, transport: f.transport, runner, now: f.now, sleep: f.sleep, progress: line => progress.push(line),
|
||||
awaitBaseline: signal => { waitSignal = signal; return finished.promise; } });
|
||||
assert.equal(finishedBy, 'head started');
|
||||
assert.equal(f.requests.slice(before).filter(request => request.args[0] === 'simulator:start').length, 1, 'base reused the baseline instead of a second session');
|
||||
assert.ok(progress.includes(`base waits for the baseline run of ${f.baseSha.slice(0, 8)} still being captured; head captures meanwhile`));
|
||||
assert.ok(progress.includes(`base: found the baseline captures of ${f.baseSha.slice(0, 8)}`));
|
||||
const baselineBundle = await readCaptureBundle(baseline.directory);
|
||||
for (const [id, image] of Object.entries((await readCaptureBundle(join(output, 'base-export'))).images)) assert.deepEqual(image, baselineBundle.images[id]);
|
||||
assert.equal(JSON.parse(await readFile(join(output, 'execution.json'), 'utf8')).capture.baseFromCache, true);
|
||||
assert.equal(waitSignal.aborted, true, 'the wait is told to stop once the review is done');
|
||||
});
|
||||
|
||||
test('capture:review waits for the baseline run ci:find-baseline named, downloads its artifact, and restores base from it', async t => {
|
||||
const f = await fixture(t, 'expo-router');
|
||||
t.mock.method(globalThis, 'fetch', async input => assert.fail(`Unmocked network request: ${input}`));
|
||||
const baseline = await runBaselineCi({ project: f.baseProject, sha: f.baseSha, output: join(f.root, 'baseline'), config: f.config, environment: f.environment, runner: f.runner, now: f.now, sleep: f.sleep });
|
||||
const zip = await zipDirectory(baseline.directory);
|
||||
const actions = [];
|
||||
const transport = async (input, init = {}) => {
|
||||
const url = new URL(input);
|
||||
if (url.origin === 'https://blob.example') { actions.push('download'); assert.equal(init.headers?.authorization, undefined); return new Response(zip); }
|
||||
const path = url.pathname.replace('/repos/acme/example', '');
|
||||
if (path === '' || path.startsWith('/actions/runs/40') || path.startsWith('/actions/workflows/') || path.startsWith('/actions/artifacts/77')) actions.push(path);
|
||||
if (path === '') return Response.json({ default_branch: 'main' });
|
||||
if (path === '/actions/workflows/explore-baseline.yml') return Response.json({ id: 5 });
|
||||
if (path === '/actions/runs/40') return Response.json({ id: 40, status: 'in_progress', conclusion: null, workflow_id: 5, head_sha: f.baseSha, head_branch: 'main', event: 'push', head_repository: { full_name: 'acme/example' } });
|
||||
if (path === '/actions/runs/40/artifacts') return Response.json({ artifacts: [{ id: 77, name: url.searchParams.get('name'), expired: false, workflow_run: { id: 40 } }] });
|
||||
if (path === '/actions/artifacts/77/zip') return new Response(null, { status: 302, headers: { location: 'https://blob.example/77.zip' } });
|
||||
return f.transport(input, init);
|
||||
};
|
||||
const review = async (name, config) => {
|
||||
const before = f.requests.length; actions.length = 0;
|
||||
const output = join(f.root, name);
|
||||
const progress = [];
|
||||
await runPublicCi({ baseProject: f.baseProject, headProject: f.headProject, pullRequestPath: f.pullRequestPath, output, config,
|
||||
environment: { ...f.environment, EXPLORE_BASELINE_RUN: '40' }, transport, runner: f.runner, now: f.now, sleep: f.sleep, progress: line => progress.push(line) });
|
||||
return { output, progress, sessions: f.requests.slice(before).filter(request => request.args[0] === 'simulator:start').length };
|
||||
};
|
||||
const waited = await review('review', f.config);
|
||||
assert.deepEqual(actions, ['', '/actions/workflows/explore-baseline.yml', '/actions/runs/40', '/actions/runs/40/artifacts', '/actions/artifacts/77/zip', 'download']);
|
||||
assert.ok(waited.progress.includes(`base: downloaded the baseline of ${f.baseSha.slice(0, 8)} that run 40 captured`));
|
||||
assert.equal(waited.sessions, 1, 'base was restored from the downloaded baseline');
|
||||
assert.equal(JSON.parse(await readFile(join(waited.output, 'execution.json'), 'utf8')).capture.baseFromCache, true);
|
||||
const off = await review('review-no-wait', { ...f.config, capture: { ...f.config.capture, baselineWaitMinutes: 0 } });
|
||||
assert.deepEqual(actions, [], 'capture.baselineWaitMinutes: 0 never waits');
|
||||
assert.equal(off.sessions, 2, 'base captured itself');
|
||||
});
|
||||
@@ -0,0 +1,203 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtemp, readFile, readdir, rm, stat } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import test from "node:test";
|
||||
import { Uint8ArrayReader, Uint8ArrayWriter, ZipWriter, configure } from "@zip.js/zip.js";
|
||||
|
||||
const { waitForBaselineArtifact, awaitPendingBaseline } = await import("../../dist/ci/baseline-wait.js");
|
||||
const { findPendingBaselineRun, findOptionalBaseline } = await import("../../dist/ci/workflow-steps.js");
|
||||
|
||||
const sha = "a".repeat(40);
|
||||
const environment = { GITHUB_REPOSITORY: "acme/app", GITHUB_TOKEN: "test-token" };
|
||||
const trustedRun = { id: 20, workflow_id: 77, head_sha: sha, head_branch: "main", event: "push", head_repository: { full_name: "acme/app" } };
|
||||
const job = (status, conclusion = null, target = "iphone") => ({ name: `baseline (${target})`, status, conclusion });
|
||||
|
||||
async function zipFiles(files) {
|
||||
configure({ useWebWorkers: false });
|
||||
const writer = new ZipWriter(new Uint8ArrayWriter(), { useWebWorkers: false });
|
||||
for (const [name, content] of Object.entries(files)) await writer.add(name, new Uint8ArrayReader(new TextEncoder().encode(content)));
|
||||
return writer.close();
|
||||
}
|
||||
|
||||
/**
|
||||
* GitHub as a review's wait sees it: baseline run 20 of `sha`, whose state moves to the next of `states` each time the
|
||||
* run is read (the last one stays), and artifact 500 of the iphone target, served from a storage URL GitHub redirects to.
|
||||
*/
|
||||
function fakeGitHub({ states = [{}], run = {}, zip, runFailures = [], runs } = {}) {
|
||||
let poll = -1;
|
||||
const requests = [];
|
||||
const state = () => states[Math.min(Math.max(poll, 0), states.length - 1)];
|
||||
const transport = async (input, init = {}) => {
|
||||
const url = new URL(input);
|
||||
requests.push(`${url.pathname}${url.search}`);
|
||||
if (url.origin === "https://blob.example") {
|
||||
assert.equal(init.headers?.authorization, undefined, "the token never reaches the storage URL GitHub redirects to");
|
||||
return new Response(zip);
|
||||
}
|
||||
assert.equal(url.origin, "https://api.github.com");
|
||||
assert.equal(init.headers?.authorization, "Bearer test-token");
|
||||
const path = url.pathname;
|
||||
if (path === "/repos/acme/app") return Response.json({ default_branch: "main" });
|
||||
if (path === "/repos/acme/app/actions/workflows/explore-baseline.yml") return Response.json({ id: 77 });
|
||||
if (path === "/repos/acme/app/actions/artifacts") return Response.json({ artifacts: [] });
|
||||
if (path === "/repos/acme/app/actions/workflows/77/runs") {
|
||||
assert.equal(url.searchParams.get("head_sha"), sha);
|
||||
assert.equal(url.searchParams.get("branch"), "main");
|
||||
return Response.json({ workflow_runs: runs ?? [] });
|
||||
}
|
||||
if (path === "/repos/acme/app/actions/runs/20") {
|
||||
poll += 1;
|
||||
const failure = runFailures.shift();
|
||||
if (failure) return new Response("unavailable", { status: failure });
|
||||
return Response.json({ ...trustedRun, status: state().runStatus ?? "in_progress", conclusion: state().conclusion ?? null, ...run });
|
||||
}
|
||||
if (path === "/repos/acme/app/actions/runs/20/artifacts") {
|
||||
const name = url.searchParams.get("name");
|
||||
return Response.json({ artifacts: state().artifact && name === `explore-baseline-${sha}-iphone` ? [{ id: 500, name, expired: false, workflow_run: { id: 20 } }] : [] });
|
||||
}
|
||||
if (path === "/repos/acme/app/actions/runs/20/jobs") return Response.json({ jobs: [{ name: "targets", status: "completed", conclusion: "success" }, ...state().jobs ?? []] });
|
||||
if (path === "/repos/acme/app/actions/artifacts/500/zip") {
|
||||
assert.equal(init.redirect, "manual", "the redirect is followed by hand, without the token");
|
||||
return new Response(null, { status: 302, headers: { location: "https://blob.example/artifact.zip?sig=1" } });
|
||||
}
|
||||
throw new Error(`unexpected request ${url}`);
|
||||
};
|
||||
return { transport, requests, polls: () => poll + 1 };
|
||||
}
|
||||
|
||||
/** A clock the wait's pauses advance, so fifteen minutes of polling take no real time. */
|
||||
function clock() {
|
||||
let milliseconds = 0;
|
||||
return { now: () => milliseconds, sleep: async pause => { milliseconds += pause; }, elapsed: () => milliseconds };
|
||||
}
|
||||
|
||||
const waitOptions = (github, time, extra = {}) => ({ run: 20, sha, target: "iphone", timeoutMs: 15 * 60_000, environment, transport: github.transport,
|
||||
now: time.now, sleep: time.sleep, ...extra });
|
||||
|
||||
test("a baseline job in progress, then finished: the review reuses its artifact, downloaded and unpacked as download-artifact would", async t => {
|
||||
const root = await mkdtemp(join(tmpdir(), "explore-baseline-wait-"));
|
||||
t.after(() => rm(root, { recursive: true, force: true }));
|
||||
const zip = await zipFiles({ "bundle.json": "{\"side\":\"head\"}", "cache-metadata.json": "{}", "screenshots/home.png": "png" });
|
||||
const github = fakeGitHub({ zip, states: [{ jobs: [] }, { jobs: [job("queued")] }, { jobs: [job("in_progress")] }, { jobs: [job("in_progress")] },
|
||||
{ jobs: [job("completed", "success")], artifact: true }] });
|
||||
const time = clock(); const lines = [];
|
||||
const directory = join(root, "review", "baseline-download");
|
||||
assert.equal(await awaitPendingBaseline({ ...waitOptions(github, time, { progress: line => lines.push(line) }), directory }), directory);
|
||||
assert.deepEqual((await readdir(directory, { recursive: true })).sort(), ["bundle.json", "cache-metadata.json", "screenshots", "screenshots/home.png"]);
|
||||
assert.equal(await readFile(join(directory, "screenshots/home.png"), "utf8"), "png");
|
||||
assert.equal(github.polls(), 5);
|
||||
assert.deepEqual(lines, [
|
||||
"waiting for main's baseline run 20 of aaaaaaaa (iphone: not started, 0 min)",
|
||||
"waiting for main's baseline run 20 of aaaaaaaa (iphone: queued, 0 min)",
|
||||
"waiting for main's baseline run 20 of aaaaaaaa (iphone: in progress, 0 min)",
|
||||
"waiting for main's baseline run 20 of aaaaaaaa (iphone: in progress, 1 min)",
|
||||
"base: downloaded the baseline of aaaaaaaa that run 20 captured",
|
||||
]);
|
||||
assert.ok(github.requests.every(request => !request.startsWith("/repos/acme/app/actions/runs/") || request.startsWith("/repos/acme/app/actions/runs/20")),
|
||||
"only the run ci:find-baseline named is read");
|
||||
assert.deepEqual((await readdir(join(root, "review"))).sort(), ["baseline-download"], "the downloaded zip is removed once unpacked");
|
||||
});
|
||||
|
||||
test("this target's artifact is reused although another target's baseline job failed (#48)", async () => {
|
||||
const github = fakeGitHub({ states: [{ jobs: [job("completed", "failure", "duo-open"), job("in_progress")] },
|
||||
{ runStatus: "completed", conclusion: "failure", jobs: [job("completed", "failure", "duo-open"), job("completed", "success")], artifact: true }] });
|
||||
assert.deepEqual(await waitForBaselineArtifact(waitOptions(github, clock())), { artifact: { id: 500, run: 20 } });
|
||||
});
|
||||
|
||||
test("a baseline job that fails or is cancelled ends the wait, and the review captures the base itself", async t => {
|
||||
for (const [conclusion, reason] of [["failure", "its iphone job failed"], ["cancelled", "its iphone job was cancelled"], ["skipped", "its iphone job ended (skipped)"]]) {
|
||||
const github = fakeGitHub({ states: [{ jobs: [job("in_progress")] }, { jobs: [job("completed", conclusion)] }] });
|
||||
assert.deepEqual(await waitForBaselineArtifact(waitOptions(github, clock())), { reason }, conclusion);
|
||||
}
|
||||
const root = await mkdtemp(join(tmpdir(), "explore-baseline-wait-"));
|
||||
t.after(() => rm(root, { recursive: true, force: true }));
|
||||
const github = fakeGitHub({ states: [{ jobs: [job("in_progress")] }, { jobs: [job("completed", "failure")] }] });
|
||||
const lines = [];
|
||||
assert.equal(await awaitPendingBaseline({ ...waitOptions(github, clock(), { progress: line => lines.push(line) }), directory: join(root, "baseline-download") }), undefined);
|
||||
assert.equal(lines.at(-1), "base: stopped waiting for the baseline of aaaaaaaa (its iphone job failed); capturing the base too");
|
||||
assert.ok(!github.requests.some(request => request.includes("/zip")), "nothing is downloaded");
|
||||
await assert.rejects(stat(join(root, "baseline-download")), { code: "ENOENT" });
|
||||
});
|
||||
|
||||
test("a run that ends without this target's job, or a job whose artifact is never listed, ends the wait after a minute's grace", async () => {
|
||||
const ended = fakeGitHub({ states: [{ runStatus: "completed", conclusion: "failure", jobs: [] }] });
|
||||
const time = clock();
|
||||
assert.deepEqual(await waitForBaselineArtifact(waitOptions(ended, time)), { reason: "the run ended (failure) without a iphone baseline job" });
|
||||
assert.equal(time.elapsed(), 60_000);
|
||||
const unlisted = fakeGitHub({ states: [{ jobs: [job("completed", "success")] }] });
|
||||
assert.deepEqual(await waitForBaselineArtifact(waitOptions(unlisted, clock())), { reason: `its iphone job finished, but GitHub lists no explore-baseline-${sha}-iphone artifact` });
|
||||
});
|
||||
|
||||
test("the wait gives up at its time limit, polling every 20 s", async () => {
|
||||
const github = fakeGitHub({ states: [{ jobs: [job("in_progress")] }] });
|
||||
const time = clock(); const lines = [];
|
||||
assert.deepEqual(await waitForBaselineArtifact(waitOptions(github, time, { progress: line => lines.push(line) })), { reason: "its iphone job is still in progress after 15 min" });
|
||||
assert.equal(time.elapsed(), 15 * 60_000);
|
||||
assert.equal(github.polls(), 15 * 3 + 1);
|
||||
assert.equal(lines.length, 15, "one line a minute, not one a poll");
|
||||
assert.deepEqual(await waitForBaselineArtifact(waitOptions(fakeGitHub({ states: [{ jobs: [job("in_progress")] }] }), clock(), { timeoutMs: 50_000, pollMs: 30_000 })),
|
||||
{ reason: "its iphone job is still in progress after 1 min" }, "the last pause ends at the limit");
|
||||
});
|
||||
|
||||
test("only a baseline run of exactly the base commit, on the default branch of this repository, from a push or dispatch, is waited for", async () => {
|
||||
for (const mutation of [{ head_branch: "feature" }, { event: "pull_request_target" }, { event: "pull_request" }, { workflow_id: 99 },
|
||||
{ head_sha: "b".repeat(40) }, { head_repository: { full_name: "fork/app" } }]) {
|
||||
const github = fakeGitHub({ run: mutation, states: [{ jobs: [job("completed", "success")], artifact: true }] });
|
||||
assert.deepEqual(await waitForBaselineArtifact(waitOptions(github, clock())), { reason: "run 20 is not a baseline run of aaaaaaaa on main" }, JSON.stringify(mutation));
|
||||
assert.ok(!github.requests.some(request => request.startsWith("/repos/acme/app/actions/runs/20/")), "nothing of an untrusted run is read");
|
||||
}
|
||||
const dispatched = fakeGitHub({ run: { event: "workflow_dispatch" }, states: [{ artifact: true }] });
|
||||
assert.deepEqual(await waitForBaselineArtifact(waitOptions(dispatched, clock())), { artifact: { id: 500, run: 20 } });
|
||||
});
|
||||
|
||||
test("ci:find-baseline names a trusted baseline run still capturing the base commit, preferring the one already capturing", async () => {
|
||||
const run = (id, status, mutation = {}) => ({ ...trustedRun, id, status, conclusion: status === "completed" ? "success" : null, ...mutation });
|
||||
const lookup = runs => findPendingBaselineRun({ sha, target: "iphone", environment, transport: fakeGitHub({ runs }).transport });
|
||||
assert.deepEqual(await lookup([run(30, "completed"), run(31, "in_progress", { event: "pull_request_target" }), run(33, "queued"), run(32, "in_progress"),
|
||||
run(34, "in_progress", { head_branch: "feature" }), run(35, "in_progress", { head_repository: { full_name: "fork/app" } })]), { run: 32 });
|
||||
assert.deepEqual(await lookup([run(33, "queued"), run(36, "waiting")]), { run: 36 }, "otherwise the newest");
|
||||
assert.equal(await lookup([run(30, "completed"), run(31, "in_progress", { workflow_id: 99 })]), undefined);
|
||||
assert.deepEqual(await findOptionalBaseline({ sha, target: "iphone", environment, transport: fakeGitHub({ runs: [run(32, "in_progress")] }).transport }), { pending: { run: 32 } },
|
||||
"with no finished baseline to download");
|
||||
const outage = async input => new URL(input).pathname.endsWith("/77/runs") ? new Response("bad gateway", { status: 502 }) : fakeGitHub().transport(input, { headers: { authorization: "Bearer test-token" } });
|
||||
const failed = await findOptionalBaseline({ sha, target: "iphone", environment, transport: outage });
|
||||
assert.equal(failed.pending, undefined);
|
||||
assert.match(failed.warning.message, /^No baseline run in progress is waited for, so capture captures the base too: GitHub answered HTTP 502/);
|
||||
});
|
||||
|
||||
test("a GitHub 5xx while polling is retried, and a missing permission ends the wait without failing the review", async t => {
|
||||
const github = fakeGitHub({ runFailures: [0, 502, 503], states: [{ jobs: [job("in_progress")] }, { jobs: [job("in_progress")] }, { jobs: [job("in_progress")] }, { artifact: true }] });
|
||||
t.mock.method(process.stderr, "write", () => true);
|
||||
assert.deepEqual(await waitForBaselineArtifact(waitOptions(github, clock())), { artifact: { id: 500, run: 20 } });
|
||||
const outage = fakeGitHub({ runFailures: [502, 502, 502, 502, 502] });
|
||||
assert.match((await waitForBaselineArtifact(waitOptions(outage, clock()))).reason, /^GITHUB_REQUEST_FAILED: GitHub answered HTTP 502/, "an outage that outlasts its retries ends the wait");
|
||||
const denied = fakeGitHub({ runFailures: [403] });
|
||||
assert.deepEqual(await waitForBaselineArtifact(waitOptions(denied, clock())),
|
||||
{ reason: "GITHUB_PERMISSION_DENIED: GitHub answered HTTP 403 for repos/acme/app/actions/runs/20." });
|
||||
assert.equal(denied.polls(), 1, "a missing permission is not retried");
|
||||
const tokenless = await waitForBaselineArtifact(waitOptions(fakeGitHub(), clock(), { environment: { GITHUB_REPOSITORY: "acme/app" } }));
|
||||
assert.match(tokenless.reason, /^GITHUB_TOKEN_MISSING/);
|
||||
});
|
||||
|
||||
test("an artifact with a path outside its directory is refused before anything is written, and base captures as before", async t => {
|
||||
const root = await mkdtemp(join(tmpdir(), "explore-baseline-wait-"));
|
||||
t.after(() => rm(root, { recursive: true, force: true }));
|
||||
const github = fakeGitHub({ zip: await zipFiles({ "bundle.json": "{}", "../escape.txt": "outside" }), states: [{ artifact: true }] });
|
||||
const lines = [];
|
||||
assert.equal(await awaitPendingBaseline({ ...waitOptions(github, clock(), { progress: line => lines.push(line) }), directory: join(root, "review", "baseline-download") }), undefined);
|
||||
// zip.js refuses the entry as it reads the archive's directory, before the unpacking's own path check would.
|
||||
assert.equal(lines.at(-1), "base: the baseline of aaaaaaaa from run 20 could not be downloaded (Unsafe filename); capturing the base too");
|
||||
assert.deepEqual(await readdir(join(root, "review")), [], "neither the download nor a partial directory is left");
|
||||
await assert.rejects(stat(join(root, "escape.txt")), { code: "ENOENT" });
|
||||
});
|
||||
|
||||
test("a review that no longer needs the baseline stops waiting at once", async () => {
|
||||
const github = fakeGitHub({ states: [{ jobs: [job("in_progress")] }] });
|
||||
const controller = new AbortController();
|
||||
const waiting = waitForBaselineArtifact({ run: 20, sha, target: "iphone", timeoutMs: 15 * 60_000, environment, transport: github.transport, signal: controller.signal });
|
||||
setTimeout(() => controller.abort(), 50);
|
||||
const started = Date.now();
|
||||
assert.deepEqual(await waiting, { reason: "the review stopped waiting" });
|
||||
assert.ok(Date.now() - started < 5_000);
|
||||
});
|
||||
@@ -9,7 +9,9 @@ import { AGENT_DEVICE_VERSION, EAS_CLI_VERSION } from "../../dist/capture/run.js
|
||||
import { readExportedMap } from "../../dist/review/evidence-assessment.js";
|
||||
import { ExploreStore, recordObservation } from "../../dist/map/store.js";
|
||||
|
||||
const { captureRecipeConfig, restoreCaptureArtifactCache, writeCaptureArtifactCache } = await tsImport("../../src/ci/artifact-cache.ts", import.meta.url);
|
||||
const { captureRecipeConfig, incompleteCaptureReason, restoreCaptureArtifactCache, validateEvidence, writeCaptureArtifactCache } = await tsImport("../../src/ci/artifact-cache.ts", import.meta.url);
|
||||
const { mergeCaptureExports } = await tsImport("../../src/ci/export-merge.ts", import.meta.url);
|
||||
const { captureHealthIssues, partitionCaptureHealthIssues } = await tsImport("../../src/capture/health.ts", import.meta.url);
|
||||
const { readCaptureBundle } = await tsImport("../../src/review/capture-review.ts", import.meta.url);
|
||||
const { CAPTURE_OBSERVER_HASH } = await tsImport("../../src/capture/observer.ts", import.meta.url);
|
||||
const { AGENT_ACCEPTANCE_POLICY } = await tsImport("../../src/contracts/capture.ts", import.meta.url);
|
||||
@@ -36,7 +38,7 @@ function acceptedFrames(bundle) {
|
||||
}])) };
|
||||
}
|
||||
|
||||
async function fixture(t, { missingAccessibility = false, partial = false } = {}) {
|
||||
async function fixture(t, { missingAccessibility = false, partial = false, appearances = false, darkFrames = appearances } = {}) {
|
||||
const root = await mkdtemp(join(tmpdir(), "explore-capture-artifact-cache-"));
|
||||
t.after(() => rm(root, { recursive: true, force: true }));
|
||||
const store = new ExploreStore(join(root, "active"));
|
||||
@@ -47,7 +49,7 @@ async function fixture(t, { missingAccessibility = false, partial = false } = {}
|
||||
id: "capture-run", name: "Cache evidence", platform: "ios", controller: "agent-device", status: "draft",
|
||||
createdAt: timestamp, updatedAt: timestamp,
|
||||
build: { id: "eas-build", gitCommit: COMMIT, source: "eas" },
|
||||
environment: { ...RUNTIME, authState: "logged-out" },
|
||||
environment: { ...RUNTIME, authState: "logged-out", ...(appearances ? { colorSchemes: ["light", "dark"] } : {}) },
|
||||
limits: { maxScreens: 20, maxActions: 100, maxDurationMinutes: 60 },
|
||||
},
|
||||
screens: [], variants: [], observations: [], elements: [], artifacts: [],
|
||||
@@ -58,7 +60,7 @@ async function fixture(t, { missingAccessibility = false, partial = false } = {}
|
||||
const snapshotPath = join(root, "sample.json");
|
||||
await writeFile(screenshotPath, PNG);
|
||||
const images = {};
|
||||
for (const id of ["home", "settings"]) {
|
||||
for (const id of darkFrames ? ["home", "settings", "home@dark", "settings@dark"] : ["home", "settings"]) {
|
||||
const snapshotText = JSON.stringify({ nodes: [{ type: "text", label: id }] });
|
||||
await writeFile(snapshotPath, snapshotText);
|
||||
const result = await recordObservation({
|
||||
@@ -81,15 +83,17 @@ async function fixture(t, { missingAccessibility = false, partial = false } = {}
|
||||
await store.exportTo(sourceDirectory, map);
|
||||
const bundle = {
|
||||
contract: "explore.bundle", side: "head", gitCommit: COMMIT,
|
||||
easBuildId: "eas-build", easProjectId: "eas-project", profile: RECIPE.profile, runtimeProfile: RUNTIME,
|
||||
easBuildId: "eas-build", easProjectId: "eas-project", profile: RECIPE.profile, runtimeProfile: appearances ? { ...RUNTIME, colorSchemes: "light,dark" } : RUNTIME,
|
||||
selectedSurfaceIds: ["home", "settings"], images,
|
||||
outcomes: Object.fromEntries(Object.entries(images).map(([id, image]) => [id, { status: "captured", attempts: 1, landing: "verified", screenshotSha256: image.sha256 }])),
|
||||
// A dark frame verified beside its light one has no outcome of its own, as the passes merge them.
|
||||
outcomes: Object.fromEntries(Object.entries(images).filter(([id]) => !id.endsWith("@dark"))
|
||||
.map(([id, image]) => [id, { status: "captured", attempts: 1, landing: "verified", screenshotSha256: image.sha256 }])),
|
||||
};
|
||||
await writeFile(join(sourceDirectory, "bundle.json"), JSON.stringify(bundle));
|
||||
await writeFile(join(sourceDirectory, "execution.json"), JSON.stringify(EXECUTION));
|
||||
// Construct actual exported, validated evidence; this fixture cannot pass with stub JSON maps.
|
||||
assert.equal((await readExportedMap(sourceDirectory)).observations.length, 2);
|
||||
const options = { ...RECIPE, sourceDirectory, directory: join(root, "restored"), commit: COMMIT, selectedIds: ["home", "settings"], runtimeProfile: RUNTIME };
|
||||
assert.equal((await readExportedMap(sourceDirectory)).observations.length, darkFrames ? 4 : 2);
|
||||
const options = { ...RECIPE, sourceDirectory, directory: join(root, "restored"), commit: COMMIT, selectedIds: ["home", "settings"], runtimeProfile: bundle.runtimeProfile };
|
||||
return { root, sourceDirectory, map, bundle, options };
|
||||
}
|
||||
|
||||
@@ -138,6 +142,8 @@ test("capture recipes exclude display settings but bind agent instructions and c
|
||||
// How many sessions capture it does not change the evidence: a baseline captured under any ceiling serves the review.
|
||||
assert.equal(key,captureRecipeConfig({...config,capture:{...config.capture,maxSessions:16}}));
|
||||
assert.equal(captureRecipeConfig({...config,capture:{...config.capture,maxSessions:4}}),captureRecipeConfig({...config,capture:{...config.capture,maxSessions:16}}));
|
||||
// Nor does how long a review waits for a baseline still being captured.
|
||||
assert.equal(key,captureRecipeConfig({...config,capture:{...config.capture,baselineWaitMinutes:0}}));
|
||||
// An unset repack is keyed as it resolves, so configs that never chose one keep the recipe of the old written-out default.
|
||||
assert.equal(key,captureRecipeConfig({...config,capture:{...config.capture,repack:"eas"}}));
|
||||
assert.equal(JSON.parse(key).capture.repack, "eas");
|
||||
@@ -300,6 +306,62 @@ test("partial captures reuse only the screens with complete successful evidence"
|
||||
assert.equal(await restoreCaptureArtifactCache({ ...f.options, directory: join(f.root, "failed-only"), selectedIds: ["settings"] }), false);
|
||||
});
|
||||
|
||||
test("a baseline whose dark session failed is cached, and a review captures the routes without a dark frame itself", async t => {
|
||||
// brentvatne/euxy's main baseline run 39506374824: the light session captured every route and the dark session failed.
|
||||
const failedDark = { status: "failed", attempts: 0, landing: "unverified", reason: "install: argent run list-devices failed: GET /tools failed: 404 Not Found",
|
||||
error: { code: "CAPTURE_SESSION_FAILED", stage: "session", message: "install: argent run list-devices failed: GET /tools failed: 404 Not Found", nextAction: "Rerun the capture.", retryable: false } };
|
||||
const f = await fixture(t, { partial: true, appearances: true });
|
||||
const complete = structuredClone(f.bundle);
|
||||
assert.equal(await writeCaptureArtifactCache(f.sourceDirectory, RECIPE), true);
|
||||
assert.equal(await restoreCaptureArtifactCache(f.options), true, "a baseline with every appearance is reused whole");
|
||||
assert.deepEqual(Object.keys((await readCaptureBundle(f.options.directory)).images).sort(), ["home", "home@dark", "settings", "settings@dark"]);
|
||||
|
||||
for (const id of ["home", "settings"]) { delete f.bundle.images[`${id}@dark`]; f.bundle.outcomes[`${id}@dark`] = failedDark; }
|
||||
f.bundle.sessionFailures = [{ scheme: "dark", error: failedDark.error }];
|
||||
await writeFile(join(f.sourceDirectory, "bundle.json"), JSON.stringify(f.bundle));
|
||||
// No route has its dark frame: offering it would make a review wait for it and capture its sides in turn, only to
|
||||
// capture every route of the base itself.
|
||||
assert.equal(await writeCaptureArtifactCache(f.sourceDirectory, RECIPE), false, "a baseline with nothing to reuse is not offered");
|
||||
assert.equal(await restoreCaptureArtifactCache({ ...f.options, directory: join(f.root, "no-dark") }), false);
|
||||
assert.equal(await incompleteCaptureReason(f.sourceDirectory, COMMIT), "its dark session failed, so no route has every appearance");
|
||||
assert.equal(await incompleteCaptureReason(f.sourceDirectory, "b".repeat(40)), undefined, "another commit's baseline is the usual miss");
|
||||
|
||||
const mixed = structuredClone(complete);
|
||||
delete mixed.images["settings@dark"]; mixed.outcomes["settings@dark"] = failedDark; delete mixed.sessionFailures;
|
||||
await writeFile(join(f.sourceDirectory, "bundle.json"), JSON.stringify(mixed));
|
||||
assert.equal(await writeCaptureArtifactCache(f.sourceDirectory, RECIPE), true);
|
||||
assert.equal(await restoreCaptureArtifactCache({ ...f.options, directory: join(f.root, "mixed") }), true);
|
||||
const restored = await readCaptureBundle(join(f.root, "mixed"));
|
||||
assert.deepEqual(restored.selectedSurfaceIds, ["home"], "settings has no dark frame, so the review's base captures it in both appearances");
|
||||
assert.deepEqual(Object.keys(restored.images).sort(), ["home", "home@dark"]);
|
||||
assert.deepEqual(Object.keys(restored.outcomes), ["home"]);
|
||||
});
|
||||
|
||||
test("a light session's real export merged with a failed dark session is valid evidence, a valid map, and healthy but for dark", async t => {
|
||||
// The light session of a side split over sessions: it renders light and dark, and took light alone.
|
||||
const f = await fixture(t, { appearances: true, darkFrames: false });
|
||||
const error = { code: "CAPTURE_SESSION_FAILED", stage: "session", message: "install: argent run list-devices failed: GET /tools failed: 404 Not Found",
|
||||
nextAction: "Rerun the capture.", retryable: true };
|
||||
// The light session lost /settings.
|
||||
delete f.bundle.images.settings;
|
||||
f.bundle.outcomes.settings = { status: "failed", attempts: 2, landing: "unverified", reason: "The route observer never answered." };
|
||||
await writeFile(join(f.sourceDirectory, "bundle.json"), JSON.stringify(f.bundle));
|
||||
await mergeCaptureExports(f.sourceDirectory, [{ scheme: "dark", error }]);
|
||||
const map = await readExportedMap(f.sourceDirectory);
|
||||
assert.equal(map.run.status, "partial");
|
||||
assert.equal(map.run.stopReason, `The dark session failed (CAPTURE_SESSION_FAILED): ${error.message}.`);
|
||||
const bundle = await readCaptureBundle(f.sourceDirectory);
|
||||
assert.deepEqual(Object.keys(bundle.outcomes).sort(), ["home", "home@dark", "settings", "settings@dark"]);
|
||||
// Each dark frame carries the dark session's reason, the one light lost too: the review shows it, not light's.
|
||||
assert.equal(bundle.outcomes["settings@dark"].reason, error.message);
|
||||
assert.equal(bundle.outcomes.settings.reason, "The route observer never answered.");
|
||||
await validateEvidence(f.sourceDirectory, bundle);
|
||||
const issues = partitionCaptureHealthIssues(captureHealthIssues(bundle, ["home", "settings"]));
|
||||
assert.deepEqual(issues.integrity, []);
|
||||
assert.deepEqual(issues.unavailable.map(issue => [issue.surfaceId, issue.code]), [["home@dark", "CAPTURE_SESSION_FAILED"], ["settings", "CAPTURE_FAILED"]]);
|
||||
assert.equal(await writeCaptureArtifactCache(f.sourceDirectory, RECIPE), false, "valid, but nothing in it a review could reuse");
|
||||
});
|
||||
|
||||
test("partial caches retain verified targets without restoring captured but unverified screenshots", async t => {
|
||||
const f = await fixture(t, { partial: true });
|
||||
f.bundle.outcomes.settings.landing = "unverified";
|
||||
|
||||
+431
-5
@@ -128,7 +128,7 @@ async function fixture(t, settings = {}) {
|
||||
let current = '/'; let snapshots = 0; let sessionName; let screenshotAttempts = 0; let splashLeft = 0; let recordingPath = ''; let alert = null; let appearance = 'light'; let alertReads = 0;
|
||||
let enteredName = false; let scrolledToContinue = false; let finishHidden = false; let wedgeCleared = false; let deviceLost = false; let relaunchedAfterForeign = false; let serverStarts = 0; let unreadableLeft = settings.unreadableAfterFold ?? 0;
|
||||
// startsErrored / startsUnnamed: how many session starts EAS fails, after naming the session (it ERRORS) or before.
|
||||
let starts = 0; let currentSession = 'session-1'; const erroredSessions = new Set(); let foldUnverifiedLeft = settings.foldUnverified ?? 0;
|
||||
let starts = 0; let currentSession = 'session-1'; let workflowRuns = 0; const workflowRunRecords = new Map(); const cancelledRuns = []; const erroredSessions = new Set(); let foldUnverifiedLeft = settings.foldUnverified ?? 0;
|
||||
// listFailures: the first so many `simulator:control --list` calls fail, as a host that drops the connection does.
|
||||
let listFailuresLeft = settings.listFailures ?? 0;
|
||||
// focusedField: the screen at `path` focuses its text field, and the keyboard is up from the `keyboardFrom`th read of it.
|
||||
@@ -219,6 +219,30 @@ async function fixture(t, settings = {}) {
|
||||
}
|
||||
if (command === 'fingerprint:generate') return json({ hash: 'source-fingerprint' });
|
||||
const buildMetadata = settings.buildMetadata ? { appVersion: '2.1.0', appBuildVersion: '42' } : {};
|
||||
// eas: the project's builds on EAS, shared by the fixtures of one test as the matrix jobs of one run share EAS.
|
||||
// A list without a fingerprint is a target looking for a repack another job of its run made (onList moves it on).
|
||||
if (command === 'build:list' && !request.args.includes('--fingerprint-hash')) {
|
||||
assert.ok(settings.eas, 'only a shared repack lists builds without a fingerprint');
|
||||
const limit = Number(value(request, '--limit'));
|
||||
assert.ok(Number.isInteger(limit) && limit >= 1 && limit <= 50, 'eas-cli lists at most 50 builds');
|
||||
settings.eas.lists += 1; settings.eas.onList?.(settings.eas);
|
||||
if (settings.eas.unreadableLists && settings.eas.lists <= settings.eas.unreadableLists) return { stdout: '', stderr: 'Error: GraphQL request failed.\n' };
|
||||
return json(settings.eas.builds.slice(0, limit));
|
||||
}
|
||||
// A target following another job's repack reads it by its id (onView moves it on).
|
||||
if (command === 'build:view') {
|
||||
assert.ok(settings.eas, 'only a shared repack views a build');
|
||||
settings.eas.views += 1; settings.eas.onView?.(settings.eas, request.args[1]);
|
||||
const build = settings.eas.builds.find(candidate => candidate.id === request.args[1]);
|
||||
return build ? json(build) : { stdout: '', stderr: `Error: build ${request.args[1]} not found\n` };
|
||||
}
|
||||
// envList: what eas env:list prints for each scope, the snapshot a shared repack's key includes.
|
||||
if (command === 'env:list') {
|
||||
assert.deepEqual([value(request, '--environment'), value(request, '--format')], ['preview', 'long']);
|
||||
const listed = settings.envList?.[value(request, '--scope')];
|
||||
if (listed === null) throw new Error('eas env:list exited with status 1: GraphQL request failed.');
|
||||
return { stdout: listed ?? 'Environment: preview\nNo variables found for this environment.\n', stderr: '' };
|
||||
}
|
||||
if (command === 'build:list') return json(settings.reuse ? [{ id: 'native-build', ...buildMetadata }] : []);
|
||||
if (command === 'build') return json([{ id: 'built-app', fingerprint: { hash: 'source-fingerprint' }, ...buildMetadata }]);
|
||||
if (command === 'env:pull') { writeFileSync(value(request, '--path'), '# Environment: preview\nEXPO_PUBLIC_ENDPOINT=sample\n'); return { stdout: '', stderr: '' }; }
|
||||
@@ -237,9 +261,52 @@ async function fixture(t, settings = {}) {
|
||||
assert.match(readFileSync(join(request.cwd, '.expo-explore-observer/metro.config.cjs'), 'utf8'), /PROJECT_RELATIVE = "\.\."/);
|
||||
assert.ok(existsSync(join(request.cwd, 'package.json')), 'tracked files are staged');
|
||||
assert.ok(!existsSync(join(request.cwd, 'node_modules')), 'untracked files are not staged');
|
||||
if (settings.workflowPlanLimit) return json({ jobs: [{ key: 'repack', status: 'FAILURE', turtleBuild: { status: 'ERRORED', error: { message: 'This account has used its local builds from the production_plus plan this month, which will reset in 13 days (on Sat Oct 24 2026). | Request ID: 1234 | Error: GraphQL request failed.' } }, outputs: {} }] });
|
||||
if (settings.workflowFailure) return { stdout: json({ jobs: [{ key: 'repack', status: 'FAILURE', turtleBuild: { status: 'ERRORED', error: { errorCode: 'UNKNOWN_CUSTOM_BUILD_ERROR', message: 'Build step "Repack" failed.' } }, outputs: {} }] }).stdout, stderr: 'See logs: https://expo.dev/accounts/acme/projects/example/workflows/run-1\n' };
|
||||
return json({ jobs: [{ key: 'repack', status: 'SUCCESS', outputs: { build_id: 'repacked-app', platform: settings.android ? 'android' : 'ios' } }] });
|
||||
assert.equal(request.args.includes('--wait'), false, 'Explore waits on the run itself');
|
||||
// repackFailures: how EAS ends the first runs, as { status, message, logs } of the repack job, its build and its logs.
|
||||
// repackStart: what workflow:run prints instead of the run's JSON (a start that failed, or a run with no JSON).
|
||||
workflowRuns += 1;
|
||||
const id = `run-${workflowRuns}`, url = `https://expo.dev/accounts/acme/projects/example/workflows/${id}`;
|
||||
const start = settings.repackStart?.[workflowRuns - 1];
|
||||
if (start?.noRun) return { stdout: '', stderr: start.stderr };
|
||||
const failure = settings.repackFailures?.[workflowRuns - 1];
|
||||
const buildId = failure || settings.workflowFailure || settings.workflowPlanLimit ? `repack-build-${workflowRuns}` : 'repacked-app';
|
||||
const build = { id: buildId, status: 'IN_PROGRESS', platform: settings.android ? 'ANDROID' : 'IOS', message: JSON.parse(/^ {6}message: (.*)$/m.exec(workflow)[1]) };
|
||||
if (settings.eas) settings.eas.builds.unshift(build);
|
||||
const finalJob = settings.workflowPlanLimit ? { key: 'repack', status: 'FAILURE', turtleBuild: { status: 'ERRORED', error: { message: 'This account has used its local builds from the production_plus plan this month, which will reset in 13 days (on Sat Oct 24 2026). | Request ID: 1234 | Error: GraphQL request failed.' } }, outputs: {} }
|
||||
: settings.workflowFailure ? { key: 'repack', status: 'FAILURE', turtleBuild: { status: 'ERRORED', error: { errorCode: 'UNKNOWN_CUSTOM_BUILD_ERROR', message: 'Build step "Repack" failed.' } }, outputs: {} }
|
||||
: failure ? { key: 'repack', status: failure.status, turtleBuild: { status: 'ERRORED', ...(failure.message ? { error: { errorCode: failure.errorCode ?? 'UNKNOWN_CUSTOM_BUILD_ERROR', message: failure.message } } : {}) }, outputs: {} }
|
||||
: { key: 'repack', status: 'SUCCESS', outputs: { build_id: 'repacked-app', platform: settings.android ? 'android' : 'ios' } };
|
||||
workflowRunRecords.set(id, { id, url, views: 0, build, failure, job: { id: `job-${workflowRuns}`, ...finalJob } });
|
||||
// A start that ran out its time throws, keeping what it printed (defaultRunner); `printed` says whether that named the run.
|
||||
if (start?.throws) throw Object.assign(new Error('spawnSync fake-eas ETIMEDOUT'), { stdout: '', stderr: start.printed ? `See logs: ${url}\n` : 'Uploading project sources...\n' });
|
||||
if (start) return { stdout: '', stderr: `${start.stderr}See logs: ${url}\n` };
|
||||
return { stdout: JSON.stringify({ id, url }), stderr: `See logs: ${url}\n` };
|
||||
}
|
||||
// A run is read until it ends: repackViews reads of IN_PROGRESS first (repackMs pass on the first), unreadableViews
|
||||
// reads that print no JSON, and then the end repackFailures and the other settings chose.
|
||||
if (command === 'workflow:view') {
|
||||
const run = workflowRunRecords.get(request.args[1]);
|
||||
assert.ok(run, 'only a run Explore started is viewed');
|
||||
run.views += 1;
|
||||
if (run.views === 1) clockMs += settings.repackMs ?? 0;
|
||||
if (run.views <= (settings.unreadableViews ?? 0)) return { stdout: '', stderr: 'Failed to fetch the workflow run.\n' };
|
||||
if (run.views <= (settings.unreadableViews ?? 0) + (settings.repackViews ?? 0) && !run.cancelled) return json({ id: run.id, status: 'IN_PROGRESS', logURL: run.url, jobs: [{ ...run.job, status: 'IN_PROGRESS', outputs: {} }] });
|
||||
const status = run.cancelled ? 'CANCELED' : run.job.status === 'SUCCESS' ? 'SUCCESS' : run.job.status === 'CANCELED' ? 'CANCELED' : 'FAILURE';
|
||||
run.build.status = status === 'SUCCESS' ? 'FINISHED' : status === 'CANCELED' ? 'CANCELED' : 'ERRORED';
|
||||
return json({ id: run.id, status, logURL: run.url, jobs: [run.cancelled ? { ...run.job, status: 'CANCELED' } : run.job] });
|
||||
}
|
||||
if (command === 'workflow:logs') {
|
||||
const run = [...workflowRunRecords.values()].find(candidate => candidate.job.id === request.args[1]);
|
||||
assert.ok(run, 'only a job of a run Explore started has its logs read');
|
||||
return json({ [run.failure?.step ?? 'Repack app']: (run.failure?.logs ?? []).map(msg => ({ time: '2026-10-20T00:00:00.000Z', msg })) });
|
||||
}
|
||||
if (command === 'workflow:cancel') {
|
||||
const run = workflowRunRecords.get(request.args[1]);
|
||||
assert.ok(run, 'only a run Explore started is cancelled');
|
||||
cancelledRuns.push(run.id);
|
||||
if (settings.cancelFails) throw new Error('eas workflow:cancel exited with status 1');
|
||||
run.cancelled = true;
|
||||
return { stdout: '', stderr: '' };
|
||||
}
|
||||
if (command === 'upload') {
|
||||
return json({ url: 'https://expo.dev/accounts/acme/projects/example/builds/repacked-app' });
|
||||
@@ -442,7 +509,7 @@ async function fixture(t, settings = {}) {
|
||||
environment: { PATH: process.env.PATH, HOME: root, ...(settings.local && settings.android ? { ANDROID_HOME: sdk, TMPDIR: root } : {}), GITHUB_TOKEN: 'github-private', ANTHROPIC_API_KEY: 'must-not-be-used', AGENT_DEVICE_DAEMON_AUTH_TOKEN: 'wrong-token', ...settings.environment },
|
||||
// Every controller round trip advances the fixture clock by one poll interval, as a device snapshot would.
|
||||
runner: request => { clockMs += 600; return runner(request); }, now: () => clockMs, sleep: milliseconds => { clockMs += milliseconds; }, fetch: catalog };
|
||||
return { root, options, requests, catalogRequests, routes, ids, plan, operations: () => requests.map(request => request.args[0]) };
|
||||
return { root, options, requests, catalogRequests, routes, ids, plan, cancelledRuns, operations: () => requests.map(request => request.args[0]) };
|
||||
}
|
||||
|
||||
test('source plan captures the configured device without app fixtures or model execution', async t => {
|
||||
@@ -495,6 +562,36 @@ test('an app that follows the system appearance is captured light then dark, dar
|
||||
assert.equal(execution.colorScheme, 'light');
|
||||
});
|
||||
|
||||
test('a single session whose dark pass fails keeps its light captures and marks every dark frame failed, as split sessions do', async t => {
|
||||
// brentvatne/euxy's main baseline run 39506374824 split the appearances over two sessions; one session must read the same.
|
||||
const f = await fixture(t, { userInterfaceStyle: 'automatic' });
|
||||
const lines = [];
|
||||
const result = await runCiMap({ ...f.options, progress: line => lines.push(line), runner: request => {
|
||||
if (request.args[0] === 'settings' && request.args[2] === 'dark') throw new Error('Controller settings exited with status 1: the device stopped answering');
|
||||
return f.options.runner(request);
|
||||
} });
|
||||
assert.equal(result.runStatus, 'partial');
|
||||
const bundle = JSON.parse(await readFile(join(f.options.outputDirectory, 'bundle.json'), 'utf8'));
|
||||
assert.deepEqual(Object.keys(bundle.images).sort(), [f.ids.home, f.ids.settings].sort());
|
||||
for (const id of Object.values(f.ids)) {
|
||||
assert.equal(bundle.outcomes[id].status, 'captured');
|
||||
assert.equal(bundle.outcomes[`${id}@dark`].status, 'failed');
|
||||
assert.match(bundle.outcomes[`${id}@dark`].error.message, /the device stopped answering/);
|
||||
}
|
||||
assert.deepEqual(bundle.sessionFailures.map(failure => failure.scheme), ['dark']);
|
||||
const map = JSON.parse(await readFile(join(f.options.outputDirectory, 'screen-map.json'), 'utf8'));
|
||||
assert.equal(map.run.status, 'partial');
|
||||
assert.match(map.run.stopReason, /^The dark pass failed \(/);
|
||||
assert.ok(lines.some(line => /the dark pass failed .*keeping the light captures/.test(line)), lines.join('\n'));
|
||||
assert.equal(f.operations().filter(operation => operation === 'simulator:stop').length, 1);
|
||||
// A checkout that changed under the dark pass casts doubt on the light frames too: the capture fails.
|
||||
const g = await fixture(t, { userInterfaceStyle: 'automatic' });
|
||||
await assert.rejects(runCiMap({ ...g.options, runner: request => {
|
||||
if (request.args[0] === 'settings' && request.args[2] === 'dark') throw new CiError({ code: 'SOURCE_CHECKOUT_CHANGED', stage: 'source', message: 'The application checkout differs from its recorded commit.', nextAction: 'Rerun.' });
|
||||
return g.options.runner(request);
|
||||
} }), error => error.code === 'SOURCE_CHECKOUT_CHANGED');
|
||||
});
|
||||
|
||||
test('an app pinned to one appearance is captured once, in that appearance', async t => {
|
||||
const f = await fixture(t, { userInterfaceStyle: 'dark' });
|
||||
const result = await runCiMap(f.options);
|
||||
@@ -713,6 +810,15 @@ test('a start that fails before naming a session stops any session of that name
|
||||
assert.equal(always.requests.filter(request => request.args[0] === 'simulator:stop' && value(request, '--id') === 'orphan-1').length, 6);
|
||||
});
|
||||
|
||||
test('a session a failed start left that does not stop is reported as left running, not as a start failure', async t => {
|
||||
// No session file names it, so a split side would otherwise keep its other appearance and hide it.
|
||||
const f = await fixture(t, { startsUnnamed: 1, orphanFromFailedStart: true });
|
||||
await assert.rejects(runCiMap({ ...f.options, sleep: () => {}, runner: request => {
|
||||
if (request.args[0] === 'simulator:stop' && value(request, '--id') === 'orphan-1') throw new Error('Controller simulator:stop exited with status 1: network unreachable');
|
||||
return f.options.runner(request);
|
||||
} }), error => error.code === 'SESSION_CLEANUP_FAILED' && /orphan-1/.test(error.message));
|
||||
});
|
||||
|
||||
test('a session of that name Explore cannot prove is its own is never stopped or replaced', async t => {
|
||||
const f = await fixture(t, { startsUnnamed: 1, orphanFromFailedStart: true, orphanUnproven: true });
|
||||
await assert.rejects(runCiMap({ ...f.options, sleep: () => {} }), error => error.code === 'SIMULATOR_START_FAILED');
|
||||
@@ -1876,6 +1982,326 @@ test('a failed hosted repack job stops the capture as EAS_REPACK_FAILED with the
|
||||
assert.deepEqual(f.operations().filter(operation => operation.startsWith('simulator:')).filter(operation => operation !== 'simulator:availability'), ['simulator:start', 'simulator:stop']);
|
||||
});
|
||||
|
||||
test('a session started beside a repack that failed, and that does not stop, is the failure reported', async t => {
|
||||
const f = await fixture(t, { workflowFailure: true });
|
||||
await assert.rejects(runCiMap({ ...f.options, runner: request => {
|
||||
if (request.args[0] === 'simulator:stop') throw new Error('Controller simulator:stop exited with status 1: network unreachable');
|
||||
return f.options.runner(request);
|
||||
} }), error => error.code === 'SESSION_CLEANUP_FAILED' && error.details.originalFailure.code === 'EAS_REPACK_FAILED');
|
||||
});
|
||||
|
||||
const runs = f => f.operations().filter(operation => operation === 'workflow:run').length;
|
||||
const sessionOperations = f => f.operations().filter(operation => /^simulator:(start|stop)$/.test(operation));
|
||||
// The upload step of brentvatne/euxy#176 (review run 39506405687), refused once the build had repacked.
|
||||
const uploadRefused = { status: 'FAILURE', message: 'Failed to upload application archive.', step: 'Upload artifact',
|
||||
logs: ['Unexpected response from server (400): {"errors":[{"code":"VALIDATION_ERROR","type":"USER","message":"Turtle build is not in progress, it cannot save artifacts.","isTransient":false}]}',
|
||||
'Failed to upload application archive.', 'Build step "Upload artifact" failed'] };
|
||||
|
||||
test('a repack the app broke, a refused credential, a build timeout or a cancelled run is not started again', async t => {
|
||||
const cases = {
|
||||
'the Repack app step': { status: 'FAILURE', message: 'Build step "Repack app" failed.' },
|
||||
'a bundling error in the logs': { status: 'FAILURE', message: 'Unknown custom build error. See logs for more information.',
|
||||
logs: ['SyntaxError: app/home.tsx: Unexpected token (4:2)', 'Build step "Repack app" failed'] },
|
||||
"a private registry's 401": { status: 'FAILURE', message: 'Unknown custom build error. See logs for more information.', step: 'Install dependencies',
|
||||
logs: ['npm error code E401', 'npm error 401 Unauthorized - GET https://npm.example.com/download/@acme/ui-1.0.0.tgz', 'Build step "Install dependencies" failed'] },
|
||||
'a bundling hang that ran out the build time': { status: 'FAILURE', message: 'Build timed out.', errorCode: 'EAS_BUILD_TIMEOUT_ERROR' },
|
||||
'a cancelled run': { status: 'CANCELED' },
|
||||
};
|
||||
for (const [name, failure] of Object.entries(cases)) {
|
||||
const f = await fixture(t, { repackFailures: [failure] });
|
||||
await assert.rejects(runCiMap(f.options), error => error.code === 'EAS_REPACK_FAILED', name);
|
||||
assert.equal(runs(f), 1, name);
|
||||
}
|
||||
});
|
||||
|
||||
test('a hosted repack EAS ended before it saved the app starts again, and the capture goes on (brentvatne/euxy#176)', async t => {
|
||||
const f = await fixture(t, { repackFailures: [uploadRefused] });
|
||||
const lines = [];
|
||||
const result = await runCiMap({ ...f.options, progress: line => lines.push(line) });
|
||||
assert.equal(result.runStatus, 'complete');
|
||||
assert.equal(result.easBuildId, 'repacked-app');
|
||||
assert.equal(runs(f), 2);
|
||||
assert.ok(lines.some(line => /the hosted EAS repack https:\/\/expo\.dev\/accounts\/acme\/projects\/example\/workflows\/run-1 failed \(The hosted EAS repack of the ios build finished with status FAILURE: Failed to upload application archive\.\); starting another repack \(1\/2\)/.test(line)), lines.join('\n'));
|
||||
// Explore waits on each run itself, and reads the failed job's logs once.
|
||||
assert.equal(f.requests.some(request => request.args[0] === 'workflow:run' && request.args.includes('--wait')), false);
|
||||
assert.equal(f.operations().filter(operation => operation === 'workflow:logs').length, 1);
|
||||
// One session, started beside the first repack, installs the second one's app.
|
||||
assert.deepEqual(sessionOperations(f), ['simulator:start', 'simulator:stop']);
|
||||
});
|
||||
|
||||
test('a repack EAS keeps failing is started twice more, one nothing explains once more, and one near the end of the budget not again', async t => {
|
||||
const cases = [{ failures: [uploadRefused, uploadRefused, uploadRefused], runs: 3 }, { failures: [{ status: 'ERRORED' }, { status: 'ERRORED' }], runs: 2 },
|
||||
// A failure 26 minutes in leaves less of the 30-minute budget than another repack needs.
|
||||
{ failures: [uploadRefused], runs: 1, repackMs: 26 * 60_000 }];
|
||||
for (const { failures, runs: expected, repackMs } of cases) {
|
||||
const f = await fixture(t, { repackFailures: failures, ...(repackMs ? { repackMs } : {}) });
|
||||
await assert.rejects(runCiMap(f.options), error => error.code === 'EAS_REPACK_FAILED' && new RegExp(`workflows/run-${expected}`).test(error.nextAction));
|
||||
assert.equal(runs(f), expected);
|
||||
assert.deepEqual(sessionOperations(f), ['simulator:start', 'simulator:stop']);
|
||||
}
|
||||
});
|
||||
|
||||
test('a run whose state EAS keeps failing to report is cancelled, and no other is started beside it', async t => {
|
||||
const f = await fixture(t, { unreadableViews: 6 });
|
||||
await assert.rejects(runCiMap(f.options), error => error.code === 'EAS_REPACK_FAILED' && error.details.failure === 'unconfirmed'
|
||||
&& /did not report the state of the hosted ios repack 6 times in a row; Explore cancelled the run/.test(error.message));
|
||||
assert.equal(runs(f), 1);
|
||||
assert.deepEqual(f.cancelledRuns, ['run-1']);
|
||||
// Five unreadable reads are waited out.
|
||||
const recovered = await fixture(t, { unreadableViews: 5 });
|
||||
assert.equal((await runCiMap(recovered.options)).easBuildId, 'repacked-app');
|
||||
assert.deepEqual(recovered.cancelledRuns, []);
|
||||
});
|
||||
|
||||
test('a start that printed its run but no JSON is waited on, and one that started no run is started again', async t => {
|
||||
const printed = await fixture(t, { repackStart: [{ stderr: 'Uploading project sources...\n' }] });
|
||||
assert.equal((await runCiMap(printed.options)).easBuildId, 'repacked-app');
|
||||
assert.equal(runs(printed), 1, 'the run it printed is followed, not replaced');
|
||||
assert.ok(printed.requests.some(request => request.args[0] === 'workflow:view' && request.args[1] === 'run-1'));
|
||||
const none = await fixture(t, { repackStart: [{ noRun: true, stderr: 'Failed to upload project sources.\nError: socket hang up\n' }] });
|
||||
assert.equal((await runCiMap(none.options)).easBuildId, 'repacked-app');
|
||||
assert.equal(runs(none), 2);
|
||||
});
|
||||
|
||||
test('a run that outlasts the 30 minutes a capture has to prepare its app is cancelled and fails the target', async t => {
|
||||
const f = await fixture(t, { repackViews: 10_000 });
|
||||
await assert.rejects(runCiMap(f.options), error => error.code === 'EAS_REPACK_FAILED' && error.details.failure === 'timeout'
|
||||
&& /did not finish within the 30 minutes a capture has to prepare its app: run run-1 was still IN_PROGRESS; it was cancelled/.test(error.message));
|
||||
assert.equal(runs(f), 1);
|
||||
assert.deepEqual(f.cancelledRuns, ['run-1']);
|
||||
const views = f.operations().filter(operation => operation === 'workflow:view').length;
|
||||
assert.ok(views >= 85 && views <= 92, `one read every 20 s: ${views}`);
|
||||
});
|
||||
|
||||
test('a capture stopped while its repack runs cancels the run, and a start that timed out is waited on only when it named its run', async t => {
|
||||
const stop = new AbortController();
|
||||
const f = await fixture(t, { repackViews: 10 });
|
||||
const sleep = milliseconds => { clockMs += milliseconds;
|
||||
if (f.operations().filter(operation => operation === 'workflow:view').length >= 2) stop.abort(new CiError({ code: 'CAPTURE_CANCELLED', stage: 'capture', message: 'Stopped.', nextAction: 'Rerun.' })); };
|
||||
await assert.rejects(runCiMap({ ...f.options, signal: stop.signal, sleep }), error => error.code === 'CAPTURE_CANCELLED');
|
||||
assert.deepEqual(f.cancelledRuns, ['run-1']);
|
||||
assert.deepEqual(sessionOperations(f), ['simulator:start', 'simulator:stop']);
|
||||
const named = await fixture(t, { repackStart: [{ throws: true, printed: true }] });
|
||||
assert.equal((await runCiMap(named.options)).easBuildId, 'repacked-app');
|
||||
assert.equal(runs(named), 1);
|
||||
const unnamed = await fixture(t, { repackStart: [{ throws: true }] });
|
||||
await assert.rejects(runCiMap(unnamed.options), error => error.code === 'EAS_REPACK_FAILED' && error.details.failure === 'unconfirmed' && /one may have started, so Explore does not start another/.test(error.message));
|
||||
assert.equal(runs(unnamed), 1);
|
||||
// A run whose cancel EAS refuses says so.
|
||||
const refused = await fixture(t, { repackViews: 10_000, cancelFails: true });
|
||||
await assert.rejects(runCiMap(refused.options), error => /it could not be cancelled/.test(error.message));
|
||||
});
|
||||
|
||||
test('a session whose app took longer than its allowance is replaced, so the capture keeps its minutes', async t => {
|
||||
const f = await fixture(t, { repackMs: 25 * 60_000 });
|
||||
const lines = [];
|
||||
const result = await runCiMap({ ...f.options, progress: line => lines.push(line) });
|
||||
assert.equal(result.runStatus, 'complete');
|
||||
assert.equal(result.simulatorSessionId, 'session-2');
|
||||
assert.deepEqual(sessionOperations(f), ['simulator:start', 'simulator:stop', 'simulator:start', 'simulator:stop']);
|
||||
const stop = f.requests.find(request => request.args[0] === 'simulator:stop');
|
||||
assert.equal(value(stop, '--id'), 'session-1');
|
||||
assert.ok(lines.some(line => /the app took 25 minutes to prepare, more than the 20 session session-1 was started with for it; replacing it/.test(line)), lines.join('\n'));
|
||||
// Twenty minutes or less keeps the session.
|
||||
const quick = await fixture(t, { repackMs: 19 * 60_000 });
|
||||
assert.equal((await runCiMap(quick.options)).simulatorSessionId, 'session-1');
|
||||
});
|
||||
|
||||
// Two matrix jobs of one GitHub run attempt: iphone leads, duo-open follows, as hostedRepackPeers and repackShareOption give them.
|
||||
const SCOPE = 'acme/example#39506405687.1';
|
||||
const share = (target, scope = SCOPE) => ({ repackShare: { scope, leader: 'iphone', target, rank: target === 'iphone' ? 0 : 1 } });
|
||||
const shareLists = f => f.requests.filter(request => request.args[0] === 'build:list' && !request.args.includes('--fingerprint-hash')).length;
|
||||
const downloaded = f => f.requests.filter(request => request.args[0] === 'build:download').map(request => value(request, '--build-id'));
|
||||
const easBuilds = (builds = []) => ({ builds, lists: 0, views: 0 });
|
||||
const shared = message => message.match(/\[explore share [0-9a-f]{32}\]$/)?.[0];
|
||||
/** The message the leader's repack of this fixture's app gives its build, with the key the others look for. */
|
||||
async function leaderMessage(t, scope, settings = {}) {
|
||||
const eas = easBuilds();
|
||||
const leader = await fixture(t, { reuse: true, eas, ...settings });
|
||||
await runCiMap({ ...leader.options, ...share('iphone', scope) });
|
||||
return eas.builds[0].message;
|
||||
}
|
||||
async function follow(t, eas, settings = {}, options = {}) {
|
||||
const f = await fixture(t, { reuse: true, eas, ...settings });
|
||||
const lines = [];
|
||||
const run = runCiMap({ ...f.options, ...share('duo-open'), progress: line => lines.push(line), ...options });
|
||||
return { f, lines, run };
|
||||
}
|
||||
|
||||
test('two jobs of one run attempt that repack the same app share one hosted repack', async t => {
|
||||
const eas = easBuilds();
|
||||
const leader = await fixture(t, { reuse: true, eas });
|
||||
await runCiMap({ ...leader.options, ...share('iphone') });
|
||||
// The leader looks once, finds nothing, and repacks with the key in its build's message.
|
||||
assert.equal(shareLists(leader), 1);
|
||||
assert.equal(runs(leader), 1);
|
||||
assert.deepEqual(leader.requests.filter(request => request.args[0] === 'env:list').map(request => value(request, '--scope')), ['project', 'account']);
|
||||
assert.match(eas.builds[0].message, /^expo-explore ios capture: route observer repack \[explore share [0-9a-f]{32}\]$/);
|
||||
const { f, lines, run } = await follow(t, eas);
|
||||
const result = await run;
|
||||
assert.equal(runs(f), 0, 'the follower does not repack');
|
||||
assert.deepEqual(downloaded(f), ['repacked-app']);
|
||||
assert.equal(result.easBuildId, 'repacked-app');
|
||||
assert.equal(result.runStatus, 'complete');
|
||||
assert.ok(lines.some(line => /using EAS build repacked-app, the hosted repack of this commit another target of this run made/.test(line)), lines.join('\n'));
|
||||
});
|
||||
|
||||
test("a target follows the leader's repack by its id while it runs, even once newer builds push it off the list", async t => {
|
||||
const message = await leaderMessage(t);
|
||||
const eas = easBuilds([{ id: 'leader-repack', status: 'IN_QUEUE', platform: 'IOS', message }]);
|
||||
eas.onView = state => {
|
||||
const build = state.builds.find(candidate => candidate.id === 'leader-repack');
|
||||
if (state.views === 1) state.builds.unshift(...Array.from({ length: 60 }, (_, index) => ({ id: `other-${index}`, status: 'FINISHED', platform: 'IOS', message: 'eas build' })));
|
||||
build.status = state.views >= 3 ? 'FINISHED' : 'IN_PROGRESS';
|
||||
};
|
||||
const { f, lines, run } = await follow(t, eas);
|
||||
const result = await run;
|
||||
assert.equal(runs(f), 0);
|
||||
assert.deepEqual(downloaded(f), ['leader-repack']);
|
||||
assert.equal(result.easBuildId, 'leader-repack');
|
||||
assert.deepEqual([eas.lists, eas.views], [1, 3]);
|
||||
assert.ok(lines.some(line => /waiting for EAS build leader-repack, the hosted repack of this commit another target of this run started/.test(line)), lines.join('\n'));
|
||||
});
|
||||
|
||||
test("a target whose leader's repack failed follows the leader's retry", async t => {
|
||||
const message = await leaderMessage(t);
|
||||
const eas = easBuilds([{ id: 'leader-repack', status: 'IN_PROGRESS', platform: 'IOS', message }]);
|
||||
eas.onView = (state, id) => {
|
||||
const build = state.builds.find(candidate => candidate.id === id);
|
||||
if (id === 'leader-repack') build.status = 'ERRORED';
|
||||
else if (state.views >= 3) build.status = 'FINISHED';
|
||||
};
|
||||
// The leader starts its retry while the follower looks again.
|
||||
eas.onList = state => { if (state.lists === 2) state.builds.unshift({ id: 'leader-retry', status: 'IN_QUEUE', platform: 'IOS', message }); };
|
||||
const { f, lines, run } = await follow(t, eas);
|
||||
const result = await run;
|
||||
assert.equal(runs(f), 0, 'no fourth repack beside the retry');
|
||||
assert.deepEqual(downloaded(f), ['leader-retry']);
|
||||
assert.equal(result.easBuildId, 'leader-retry');
|
||||
assert.ok(lines.some(line => /the shared repack \(EAS build leader-repack\) ended ERRORED; looking for another for up to 60 s before repacking here/.test(line)), lines.join('\n'));
|
||||
});
|
||||
|
||||
test("a target whose leader's repack failed and was not retried repacks itself, with the same key for the others", async t => {
|
||||
const message = await leaderMessage(t);
|
||||
const eas = easBuilds([{ id: 'leader-repack', status: 'IN_PROGRESS', platform: 'IOS', message }]);
|
||||
eas.onView = state => { if (state.views === 2) state.builds[0].status = 'ERRORED'; };
|
||||
const { f, lines, run } = await follow(t, eas);
|
||||
const result = await run;
|
||||
assert.equal(runs(f), 1);
|
||||
assert.equal(result.easBuildId, 'repacked-app');
|
||||
// One look at once, then one every 15 s for the minute after the failure.
|
||||
assert.ok(eas.lists >= 5 && eas.lists <= 6, `${eas.lists} lists`);
|
||||
assert.equal(eas.builds[0].message, message, "the follower's own repack carries the same key");
|
||||
assert.ok(lines.some(line => /no other repack of this commit appeared; repacking here/.test(line)), lines.join('\n'));
|
||||
});
|
||||
|
||||
test('a repack of other inputs is never used, and the wait for one ends after three minutes of modest polling', async t => {
|
||||
const message = await leaderMessage(t);
|
||||
const otherAttempt = await leaderMessage(t, 'acme/example#39506405687.2');
|
||||
// Another value in the EAS environment, and another commit of the app.
|
||||
const otherVariables = await leaderMessage(t, SCOPE, { envList: { project: 'Environment: preview\nVariables for this project:\nName EXPO_PUBLIC_API\nValue https://api.example.com\nUpdated at Oct 20 10:00:00\n' } });
|
||||
const changedEas = easBuilds();
|
||||
const changedLeader = await fixture(t, { reuse: true, eas: changedEas });
|
||||
writeFileSync(join(changedLeader.options.project, 'app.json'), JSON.stringify({ expo: { name: 'Example', slug: 'example', version: '2' } }));
|
||||
await runCiMap({ ...changedLeader.options, ...share('iphone') });
|
||||
const otherJavaScript = changedEas.builds[0].message;
|
||||
assert.equal(new Set([message, otherAttempt, otherVariables, otherJavaScript].map(shared)).size, 4, 'each input has its own key');
|
||||
const eas = easBuilds([
|
||||
{ id: 'other-attempt', status: 'FINISHED', platform: 'IOS', message: otherAttempt },
|
||||
{ id: 'other-variables', status: 'FINISHED', platform: 'IOS', message: otherVariables },
|
||||
{ id: 'other-javascript', status: 'FINISHED', platform: 'IOS', message: otherJavaScript },
|
||||
{ id: 'other-platform', status: 'FINISHED', platform: 'ANDROID', message },
|
||||
{ id: 'not-the-key', status: 'FINISHED', platform: 'IOS', message: `${message} (edited)` },
|
||||
]);
|
||||
const { f, lines, run } = await follow(t, eas);
|
||||
const result = await run;
|
||||
assert.equal(result.easBuildId, 'repacked-app', 'the follower repacked its own app');
|
||||
assert.equal(runs(f), 1);
|
||||
// One list every 15 s for three minutes, and the first at once.
|
||||
assert.ok(eas.lists >= 12 && eas.lists <= 13, `${eas.lists} lists`);
|
||||
assert.ok(lines.some(line => /no repack of this commit from iphone or another target appeared within 3 minutes; repacking here/.test(line)), lines.join('\n'));
|
||||
});
|
||||
|
||||
test('three unreadable lists in a row end the wait, and so does an abort', async t => {
|
||||
const unreadable = easBuilds(); unreadable.unreadableLists = 3;
|
||||
const { f, lines, run } = await follow(t, unreadable);
|
||||
assert.equal((await run).easBuildId, 'repacked-app');
|
||||
assert.deepEqual([unreadable.lists, runs(f)], [3, 1]);
|
||||
assert.ok(lines.some(line => /EAS did not answer three times in a row; repacking here/.test(line)), lines.join('\n'));
|
||||
const stop = new AbortController();
|
||||
const waiting = easBuilds(); waiting.onList = state => { if (state.lists === 2) stop.abort(new CiError({ code: 'CAPTURE_CANCELLED', stage: 'build', message: 'Stopped.', nextAction: 'Rerun.' })); };
|
||||
const aborted = await follow(t, waiting, {}, { signal: stop.signal });
|
||||
await assert.rejects(aborted.run, error => error.code === 'CAPTURE_CANCELLED');
|
||||
assert.equal(runs(aborted.f), 0);
|
||||
assert.deepEqual(sessionOperations(aborted.f), ['simulator:start', 'simulator:stop'], 'the session started beside the wait is stopped');
|
||||
});
|
||||
|
||||
test("a followed repack that outlasts the budget fails the target, and a wait that leaves too little for a retry does not retry", async t => {
|
||||
const message = await leaderMessage(t);
|
||||
const stuck = easBuilds([{ id: 'leader-repack', status: 'IN_PROGRESS', platform: 'IOS', message }]);
|
||||
const { f, run } = await follow(t, stuck);
|
||||
await assert.rejects(run, error => error.code === 'EAS_REPACK_FAILED' && error.details.failure === 'timeout' && /too little time was left for a repack of its own/.test(error.message));
|
||||
assert.equal(runs(f), 0);
|
||||
assert.ok(stuck.views >= 55 && stuck.views <= 61, `one read every 30 s: ${stuck.views}`);
|
||||
// The leader's repack fails 21 minutes in: this target looks for a minute, repacks, and its own refused upload 4
|
||||
// minutes later leaves too little of the budget for another.
|
||||
const late = easBuilds([{ id: 'leader-repack', status: 'IN_PROGRESS', platform: 'IOS', message }]);
|
||||
late.onView = state => { clockMs += 21 * 60_000; state.builds[0].status = 'ERRORED'; };
|
||||
const budget = await follow(t, late, { repackFailures: [uploadRefused], repackMs: 4 * 60_000 });
|
||||
const started = clockMs;
|
||||
await assert.rejects(budget.run, error => error.code === 'EAS_REPACK_FAILED' && /Failed to upload application archive/.test(error.message));
|
||||
assert.equal(runs(budget.f), 1);
|
||||
assert.ok(clockMs - started < 30 * 60_000, 'everything stayed in one 30-minute budget');
|
||||
});
|
||||
|
||||
test("a re-run attempt's target does not wait 3 minutes for a leader that may not run again, nor use the first attempt's repack", async t => {
|
||||
const eas = easBuilds();
|
||||
const first = await fixture(t, { reuse: true, eas });
|
||||
await runCiMap({ ...first.options, ...share('iphone') });
|
||||
// In a re-run the second target waits one stagger (60 s) instead of three minutes.
|
||||
const rerun = await fixture(t, { reuse: true, eas });
|
||||
await runCiMap({ ...rerun.options, repackShare: { scope: 'acme/example#39506405687.2', leader: 'iphone', target: 'duo-open', rank: 1, rerun: true } });
|
||||
assert.ok(shareLists(rerun) >= 4 && shareLists(rerun) <= 5, `${shareLists(rerun)} lists`);
|
||||
assert.equal(runs(rerun), 1);
|
||||
assert.notEqual(shared(eas.builds[0].message), shared(eas.builds[1].message));
|
||||
});
|
||||
|
||||
test('targets whose leader makes no repack join the first of them to start one instead of all repacking at once', async t => {
|
||||
const message = await leaderMessage(t);
|
||||
// duo-closed (second after the leader) waits a minute longer than duo-half-open; the latter's repack appears meanwhile.
|
||||
const eas = easBuilds();
|
||||
eas.onList = state => { if (state.lists === 14) state.builds.unshift({ id: 'half-open-repack', status: 'IN_QUEUE', platform: 'IOS', message }); };
|
||||
eas.onView = state => { if (state.views >= 2) state.builds[0].status = 'FINISHED'; };
|
||||
const f = await fixture(t, { reuse: true, eas });
|
||||
const result = await runCiMap({ ...f.options, repackShare: { scope: SCOPE, leader: 'iphone', target: 'duo-closed', rank: 2 } });
|
||||
assert.equal(runs(f), 0);
|
||||
assert.equal(result.easBuildId, 'half-open-repack');
|
||||
});
|
||||
|
||||
test('a native build this job made, or an environment EAS cannot list, is repacked here without sharing', async t => {
|
||||
const fresh = await fixture(t, { eas: easBuilds() });
|
||||
await runCiMap({ ...fresh.options, ...share('duo-open') });
|
||||
assert.equal(fresh.operations().includes('build'), true);
|
||||
assert.deepEqual([shareLists(fresh), runs(fresh)], [0, 1], 'no 3-minute wait for a key nobody else can have');
|
||||
assert.equal(fresh.operations().includes('env:list'), false);
|
||||
const eas = easBuilds();
|
||||
const unlisted = await fixture(t, { reuse: true, eas, envList: { account: null } });
|
||||
const lines = [];
|
||||
await runCiMap({ ...unlisted.options, ...share('duo-open'), progress: line => lines.push(line) });
|
||||
assert.deepEqual([shareLists(unlisted), runs(unlisted)], [0, 1]);
|
||||
assert.equal(shared(eas.builds[0].message), undefined, 'a repack without a snapshot carries no key');
|
||||
assert.ok(lines.some(line => /EAS did not list the preview environment's variables; repacking here without sharing the repack/.test(line)), lines.join('\n'));
|
||||
});
|
||||
|
||||
test("the build to repack is the newest one Explore did not repack itself", async () => {
|
||||
const { existingBuild } = await import('../../dist/capture/eas-build.js');
|
||||
const listed = JSON.stringify([{ id: 'repack-1', message: 'expo-explore ios capture: route observer repack [explore share 0123456789abcdef0123456789abcdef]' },
|
||||
{ id: 'repack-2', message: 'expo-explore ios capture: route observer repack' }, { id: 'native', message: 'Release 2.1' }, { id: 'older' }]);
|
||||
assert.equal(existingBuild(listed).id, 'native');
|
||||
assert.equal(existingBuild(JSON.stringify([{ id: 'repack-1', message: 'expo-explore ios capture: route observer repack' }])).id, 'repack-1', 'a repack when nothing else has the fingerprint');
|
||||
assert.equal(existingBuild('[]'), undefined);
|
||||
});
|
||||
|
||||
test('capture.repack: local repacks on the runner and installs the app from it, with no upload', async t => {
|
||||
for (const reuse of [false, true]) {
|
||||
const f = await fixture(t, { repack: 'local', reuse, buildMetadata: true });
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { readFile, readdir } from "node:fs/promises";
|
||||
import { createRequire } from "node:module";
|
||||
import { join, relative } from "node:path";
|
||||
import test from "node:test";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
// A fake EAS accepts any flag, so a flag the pinned eas-cli does not have passes every other test and fails only on a
|
||||
// real runner: 0.20.6 sent `env:list --non-interactive`, which eas-cli 25.2.0 refuses, so every capture logged "EAS did
|
||||
// not list the development environment's variables" and never shared a repack (brentvatne/euxy review run 39517635857).
|
||||
// Every literal `eas` command in src is checked against the command manifest of the eas-cli package.json pins.
|
||||
const root = fileURLToPath(new URL("../..", import.meta.url));
|
||||
const manifest = JSON.parse(await readFile(createRequire(join(root, "package.json")).resolve("eas-cli/oclif.manifest.json"), "utf8")).commands;
|
||||
const commands = new Map();
|
||||
for (const [id, command] of Object.entries(manifest)) {
|
||||
commands.set(id, command);
|
||||
for (const alias of command.aliases ?? []) commands.set(alias, command);
|
||||
}
|
||||
|
||||
async function sources(directory) {
|
||||
const files = [];
|
||||
for (const entry of await readdir(directory, { withFileTypes: true })) {
|
||||
const path = join(directory, entry.name);
|
||||
if (entry.isDirectory()) files.push(...await sources(path));
|
||||
else if (entry.name.endsWith(".ts")) files.push(path);
|
||||
}
|
||||
return files;
|
||||
}
|
||||
|
||||
test("every literal eas command in src names a flag the pinned eas-cli has", async () => {
|
||||
const checked = new Set();
|
||||
const unknown = [];
|
||||
for (const file of await sources(join(root, "src"))) {
|
||||
const text = await readFile(file, "utf8");
|
||||
// An argument list that starts with a command id, up to its closing bracket.
|
||||
for (const match of text.matchAll(/\[\s*"([a-z]+(?::[a-z-]+)+)"([^\]]*)\]/g)) {
|
||||
const command = commands.get(match[1]);
|
||||
if (!command) continue; // Explore's own commands (source:verify, explore:binary, …) share the shape.
|
||||
checked.add(match[1]);
|
||||
const flags = command.flags ?? {};
|
||||
for (const flag of match[2].matchAll(/"--([a-z0-9-]+)(?:=[^"]*)?"/g)) {
|
||||
if (!(flag[1] in flags)) unknown.push(`${relative(root, file)}: eas ${match[1]} --${flag[1]}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
assert.deepEqual(unknown, []);
|
||||
// The scan finds the commands capture depends on, so it cannot pass by matching nothing.
|
||||
for (const id of ["env:list", "build:list", "build:view", "workflow:run", "workflow:view", "workflow:cancel", "simulator:start", "simulator:stop", "simulator:control"]) {
|
||||
assert.ok(checked.has(id), `eas ${id} was not found in src`);
|
||||
}
|
||||
});
|
||||
@@ -5,7 +5,7 @@ import { join } from "node:path";
|
||||
import test from "node:test";
|
||||
import { tsImport } from "tsx/esm/api";
|
||||
|
||||
const { DEFAULT_EXPLORE_CONFIG_PATH, assertExecutionMode, parseExploreConfig, readExploreConfig, validateExploreConfig, configPlatforms, configTargets, resolvePlatformConfig, resolveTargetConfig, withRunOverrides, captureRepack } =
|
||||
const { DEFAULT_EXPLORE_CONFIG_PATH, assertExecutionMode, parseExploreConfig, readExploreConfig, validateExploreConfig, configPlatforms, configTargets, resolvePlatformConfig, resolveTargetConfig, withRunOverrides, captureRepack, hostedRepackPeers } =
|
||||
await tsImport("../../src/config/explore-config.ts", import.meta.url);
|
||||
|
||||
const SOURCE = `app:
|
||||
@@ -47,6 +47,12 @@ test("rejects obsolete configuration and gives the exact next action", () => {
|
||||
assert.equal(validateExploreConfig({ ...base(), capture: { readinessSeconds: 12 } }).capture.readinessSeconds, 12);
|
||||
assert.equal("readinessSeconds" in validateExploreConfig(base()).capture, false);
|
||||
assert.throws(() => validateExploreConfig({ ...base(), capture: { readinessSeconds: 0 } }), /capture.readinessSeconds must be a positive integer/);
|
||||
// 0 never waits for a baseline still being captured; absent, the review waits DEFAULT_BASELINE_WAIT_MINUTES.
|
||||
for (const minutes of [0, 15, 60]) assert.equal(validateExploreConfig({ ...base(), capture: { baselineWaitMinutes: minutes } }).capture.baselineWaitMinutes, minutes);
|
||||
assert.equal("baselineWaitMinutes" in validateExploreConfig(base()).capture, false);
|
||||
for (const minutes of [-1, 61, 1.5, "15"]) {
|
||||
assert.throws(() => validateExploreConfig({ ...base(), capture: { baselineWaitMinutes: minutes } }), /capture.baselineWaitMinutes must be a whole number from 0 to 60/, String(minutes));
|
||||
}
|
||||
});
|
||||
|
||||
test("validates device, capture limits, providers, paths, and custom app schemes", () => {
|
||||
@@ -357,6 +363,33 @@ test("target ids and entries are validated", () => {
|
||||
assert.equal(validateExploreConfig({ ...base(), platforms, capture: { maxSessions: 1 } }).capture.maxSessions, 1);
|
||||
});
|
||||
|
||||
test("the targets that repack one app on EAS are its repack peers, led by the first in config order", () => {
|
||||
const config = validateExploreConfig({ ...base(), targets: {
|
||||
iphone: { platform: "ios" },
|
||||
"duo-closed": { platform: "ios", device: "iPhone Duo", pose: "closed" },
|
||||
"duo-open": { platform: "ios", device: "iPhone Duo", pose: "open" },
|
||||
local: { platform: "ios", simulator: "local" },
|
||||
android: { platform: "android", bundleId: "com.example.app", easProfile: "explore-apk" },
|
||||
} });
|
||||
const peers = (id, overrides) => hostedRepackPeers(config, withRunOverrides(resolveTargetConfig(config, { target: id }), overrides ?? {}), overrides);
|
||||
assert.deepEqual(peers("duo-open"), ["iphone", "duo-closed", "duo-open"]);
|
||||
assert.deepEqual(peers("android"), ["android"], "a target no other one shares an app with has only itself");
|
||||
assert.deepEqual(peers("local"), [], "a repack on this machine is shared with nobody");
|
||||
// A run that repacks on the runner (--repack local) shares nothing; one that repacks on EAS includes the local-simulator target.
|
||||
assert.deepEqual(peers("iphone", { repack: "local" }), []);
|
||||
assert.deepEqual(peers("iphone", { repack: "eas" }), ["iphone", "duo-closed", "duo-open", "local"]);
|
||||
// An override that makes every target run on a local simulator repacks each on its runner, unless it also asks for EAS.
|
||||
assert.deepEqual(peers("iphone", { simulator: "local" }), []);
|
||||
assert.deepEqual(peers("iphone", { simulator: "local", repack: "eas" }), ["iphone", "duo-closed", "duo-open", "local"]);
|
||||
// A config whose own repack setting makes one target local leaves it out of the others' peers under the same override.
|
||||
const mixed = validateExploreConfig({ ...base(), targets: { iphone: { platform: "ios" }, duo: { platform: "ios", device: "iPhone Duo" }, desk: { platform: "ios", simulator: "local" } } });
|
||||
const mixedPeers = (id, overrides) => hostedRepackPeers(mixed, withRunOverrides(resolveTargetConfig(mixed, { target: id }), overrides), overrides);
|
||||
assert.deepEqual(mixedPeers("duo", { controller: "argent" }), ["iphone", "duo"]);
|
||||
assert.deepEqual(mixedPeers("desk", { controller: "argent" }), []);
|
||||
const flat = validateExploreConfig(base());
|
||||
assert.deepEqual(hostedRepackPeers(flat, resolveTargetConfig(flat)), ["ios"]);
|
||||
});
|
||||
|
||||
test("a target can choose its own controller and simulator over capture's", () => {
|
||||
const config = validateExploreConfig({ ...base(), capture: { controller: "argent", simulator: "local" }, targets: {
|
||||
duo: { platform: "ios", device: "iPhone Duo" },
|
||||
|
||||
@@ -6,6 +6,8 @@ import test from "node:test";
|
||||
|
||||
import { mergeSchemeOutcomes } from "../../dist/capture/appearance.js";
|
||||
import { mergeCaptureExports } from "../../dist/ci/export-merge.js";
|
||||
import { captureSideInSessions } from "../../dist/ci/pipeline.js";
|
||||
import { CiError } from "../../dist/shared/ci-errors.js";
|
||||
|
||||
async function writeExport(root, { screens, images, outcomes, launch, diagnostics }) {
|
||||
await mkdir(join(root, "artifacts"), { recursive: true });
|
||||
@@ -20,7 +22,8 @@ async function writeExport(root, { screens, images, outcomes, launch, diagnostic
|
||||
screens: screens.map(id => ({ id, colorScheme: id.endsWith("@dark") ? "dark" : "light" })), variants: [], observations: screens.map(id => ({ screenId: id })),
|
||||
elements: [], actionAttempts: [], transitions: [], journeys: [], frontier: [], gaps: [] };
|
||||
await writeFile(join(root, "screen-map.json"), JSON.stringify(map));
|
||||
await writeFile(join(root, "bundle.json"), JSON.stringify({ contract: "explore.bundle", side: "head", runtimeProfile: { colorSchemes: "light,dark" }, images, outcomes, ...(launch ? { launch } : {}) }));
|
||||
await writeFile(join(root, "bundle.json"), JSON.stringify({ contract: "explore.bundle", side: "head", runtimeProfile: { colorSchemes: "light,dark" },
|
||||
selectedSurfaceIds: [...new Set(Object.keys(outcomes).map(id => id.replace(/@(light|dark)$/, "")))], images, outcomes, ...(launch ? { launch } : {}) }));
|
||||
await writeFile(join(root, "manifest.json"), JSON.stringify({ contract: "explore.map", runId: "run", revision: 1, files: artifacts.map(({ id: _id, ...file }) => file) }));
|
||||
await writeFile(join(root, "capture-diagnostics", "controller-timing.jsonl"), diagnostics);
|
||||
return root;
|
||||
@@ -94,3 +97,111 @@ test("a dark session that could not show dark (Argent without appearance.set) ad
|
||||
await mergeCaptureExports(primary, [{ directory: gapped, scheme: "dark" }]);
|
||||
assert.equal(JSON.parse(await readFile(mapPath, "utf8")).run.status, "partial", "a complete run has no open gaps");
|
||||
});
|
||||
|
||||
// brentvatne/euxy's main baseline run 39506374824: the light session captured every route, then the dark session's install failed.
|
||||
const DARK_FAILURE = { code: "CAPTURE_SESSION_FAILED", stage: "session", message: "install: argent run list-devices failed: GET /tools failed: 404 Not Found",
|
||||
nextAction: "Rerun the capture on a new session.", retryable: true };
|
||||
const failedDark = { status: "failed", attempts: 0, landing: "unverified", reason: DARK_FAILURE.message, error: DARK_FAILURE };
|
||||
|
||||
test("a dark session that left no export records its failure as every route's own dark outcome, and the run is partial", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "explore-merge-"));
|
||||
const lost = { status: "failed", landing: "unverified", attempts: 3, reason: "The route observer never answered." };
|
||||
const primary = await writeExport(join(root, "head-export"), { screens: ["home"], images: { home: { path: "artifacts/home.png" } },
|
||||
outcomes: { home: { status: "captured", landing: "verified", attempts: 1 }, settings: lost }, diagnostics: "" });
|
||||
const mapPath = join(primary, "screen-map.json");
|
||||
await writeFile(mapPath, JSON.stringify({ ...JSON.parse(await readFile(mapPath, "utf8")), run: { id: "run", status: "complete", environment: { colorSchemes: ["light", "dark"] } } }));
|
||||
await mergeCaptureExports(primary, [{ scheme: "dark", error: DARK_FAILURE }]);
|
||||
const bundle = JSON.parse(await readFile(join(primary, "bundle.json"), "utf8"));
|
||||
// A route the light session lost keeps its own reason, and its dark frame carries the dark session's, not the light one's.
|
||||
assert.deepEqual(bundle.outcomes, { home: { status: "captured", landing: "verified", attempts: 1 }, settings: lost, "home@dark": failedDark, "settings@dark": failedDark });
|
||||
assert.deepEqual(bundle.sessionFailures, [{ scheme: "dark", error: DARK_FAILURE }]);
|
||||
assert.deepEqual(Object.keys(bundle.images), ["home"]);
|
||||
const map = JSON.parse(await readFile(mapPath, "utf8"));
|
||||
assert.equal(map.run.status, "partial");
|
||||
assert.equal(map.run.stopReason, `The dark session failed (CAPTURE_SESSION_FAILED): ${DARK_FAILURE.message}.`);
|
||||
});
|
||||
|
||||
test("a dark session that failed beside a primary that renders no dark has no frames to fail, and stays on the bundle", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "explore-merge-"));
|
||||
const primary = await writeExport(join(root, "head-export"), { screens: ["home"], images: { home: { path: "artifacts/home.png" } },
|
||||
outcomes: { home: { status: "captured", landing: "verified", attempts: 1 } }, diagnostics: "" });
|
||||
const bundlePath = join(primary, "bundle.json");
|
||||
// The light session could not switch appearance (Argent without appearance.set): it captured light alone.
|
||||
await writeFile(bundlePath, JSON.stringify({ ...JSON.parse(await readFile(bundlePath, "utf8")), runtimeProfile: { colorScheme: "light" } }));
|
||||
await mergeCaptureExports(primary, [{ scheme: "dark", error: DARK_FAILURE }]);
|
||||
const bundle = JSON.parse(await readFile(bundlePath, "utf8"));
|
||||
assert.deepEqual(Object.keys(bundle.outcomes), ["home"]);
|
||||
assert.deepEqual(bundle.sessionFailures, [{ scheme: "dark", error: DARK_FAILURE }], "the comment still names the failed session");
|
||||
assert.equal(JSON.parse(await readFile(join(primary, "screen-map.json"), "utf8")).run.status, "partial");
|
||||
});
|
||||
|
||||
/** The side's sessions as captureSideInSessions runs them: the light one exports, the dark one does what `dark` says. */
|
||||
async function sideInSessions({ light, dark, leftSessionFile = false, environment = {}, signal } = {}) {
|
||||
const root = await mkdtemp(join(tmpdir(), "explore-sessions-"));
|
||||
const progress = [];
|
||||
const mapOptions = { side: "head", outputDirectory: join(root, "head-export"), mapDirectory: join(root, "head-map"), buildShare: join(root, "builds"),
|
||||
sessionStatePath: join(root, "head-session"), expoToken: "expo-secret-token", environment };
|
||||
const runPass = async options => {
|
||||
const scheme = options.appearancePass.capture[0];
|
||||
const outcome = scheme === "light" ? light : dark;
|
||||
// A capture removes its session file only once its session is confirmed stopped.
|
||||
if (scheme === "dark" && leftSessionFile) await writeFile(options.sessionStatePath, "01a15b61-4df9\n");
|
||||
if (outcome instanceof Error) throw outcome;
|
||||
const id = scheme === "light" ? "home" : "home@dark";
|
||||
await writeExport(options.outputDirectory, { screens: [id], images: { [id]: { path: `artifacts/${id}.png` } },
|
||||
outcomes: { [id]: { status: "captured", landing: "verified", attempts: 1 } }, diagnostics: "" });
|
||||
return { runStatus: "complete", outputDirectory: options.outputDirectory, scheme };
|
||||
};
|
||||
const run = () => captureSideInSessions(mapOptions, [["light"], ["dark"]], line => progress.push(line), signal, true, runPass);
|
||||
return { root, progress, run, bundle: async () => JSON.parse(await readFile(join(mapOptions.outputDirectory, "bundle.json"), "utf8")) };
|
||||
}
|
||||
|
||||
test("a side keeps its light captures when only its dark session fails, and the report marks every dark frame failed", async () => {
|
||||
const side = await sideInSessions({ dark: new CiError({ ...DARK_FAILURE, message: `${DARK_FAILURE.message} (tokens expo-secret-token, gh-secret)` }),
|
||||
environment: { GITHUB_TOKEN: "gh-secret", PATH: "/usr/bin" } });
|
||||
const result = await side.run();
|
||||
assert.equal(result.scheme, "light", "the primary session's result");
|
||||
assert.equal(result.runStatus, "partial");
|
||||
const bundle = await side.bundle();
|
||||
assert.deepEqual(Object.keys(bundle.images), ["home"]);
|
||||
assert.equal(bundle.outcomes["home@dark"].status, "failed");
|
||||
assert.equal(bundle.outcomes["home@dark"].error.code, "CAPTURE_SESSION_FAILED");
|
||||
assert.doesNotMatch(JSON.stringify(bundle), /expo-secret-token|gh-secret/, "the session's error is stored without any secret the environment holds");
|
||||
assert.ok(side.progress.some(line => /^head: the dark session failed \(CAPTURE_SESSION_FAILED: install: argent run list-devices failed/.test(line)
|
||||
&& line.endsWith("keeping the light captures, and the report marks every dark frame failed")), side.progress.join("\n"));
|
||||
});
|
||||
|
||||
test("a side whose light session fails still fails, whatever the dark session did", async () => {
|
||||
const light = new CiError({ ...DARK_FAILURE, code: "CAPTURE_INSTALL_FAILED" });
|
||||
await assert.rejects((await sideInSessions({ light })).run(), { code: "CAPTURE_INSTALL_FAILED" });
|
||||
await assert.rejects((await sideInSessions({ light, dark: new CiError({ ...DARK_FAILURE }) })).run(), { code: "CAPTURE_INSTALL_FAILED" });
|
||||
});
|
||||
|
||||
test("a dark session not proven stopped fails the side, so a running session is never hidden behind a partial report", async () => {
|
||||
const cleanup = new CiError({ code: "SESSION_CLEANUP_FAILED", stage: "cleanup", message: "The exact EAS session could not be confirmed stopped.",
|
||||
nextAction: "Stop EAS Simulator session 01a15b61 and verify its status.", retryable: false });
|
||||
await assert.rejects((await sideInSessions({ dark: cleanup })).run(), { code: "SESSION_CLEANUP_FAILED" });
|
||||
await assert.rejects((await sideInSessions({ light: new CiError({ ...DARK_FAILURE, code: "CAPTURE_INSTALL_FAILED" }), dark: cleanup })).run(), { code: "SESSION_CLEANUP_FAILED" },
|
||||
"the leaked session is reported before the light session's own failure");
|
||||
const handoff = new CiError({ code: "SESSION_HANDOFF_UNVERIFIED", stage: "session", message: "EAS did not provide an exact verifiable simulator session.",
|
||||
nextAction: "Verify a session's exact ID and ownership before cleanup." });
|
||||
await assert.rejects((await sideInSessions({ dark: handoff })).run(), { code: "SESSION_HANDOFF_UNVERIFIED" });
|
||||
const unverified = new CiError({ code: "SIMULATOR_START_FAILED", stage: "session", message: "EAS simulator:start failed before a session identity could be verified.",
|
||||
nextAction: "Verify a session's exact ID and ownership before cleanup.", details: { allocation: "unknown" } });
|
||||
await assert.rejects((await sideInSessions({ dark: unverified })).run(), { code: "SIMULATOR_START_FAILED" });
|
||||
// A worker that died before its capture's own cleanup ran: it exited, or failed with a raw error.
|
||||
const exited = new CiError({ code: "CAPTURE_WORKER_EXITED", stage: "capture-head", message: "The capture worker for the head side exited with code 1 before it reported a result.",
|
||||
nextAction: "Inspect the job log for the worker's failure, then rerun Explore." });
|
||||
await assert.rejects((await sideInSessions({ dark: exited })).run(), { code: "CAPTURE_WORKER_EXITED" });
|
||||
await assert.rejects((await sideInSessions({ dark: new Error("Worker terminated due to reaching memory limit: JS heap out of memory") })).run(), /memory limit/);
|
||||
// Any failure that left the session's file behind: the session was never confirmed stopped.
|
||||
await assert.rejects((await sideInSessions({ dark: new CiError({ ...DARK_FAILURE }), leftSessionFile: true })).run(), { code: "CAPTURE_SESSION_FAILED" });
|
||||
});
|
||||
|
||||
test("a dark session whose checkout changed, whose evidence failed its checks, or whose review was stopped fails the side", async () => {
|
||||
for (const code of ["SOURCE_CHECKOUT_CHANGED", "CHECKOUT_MODIFIED", "CAPTURE_INCOMPLETE", "CAPTURE_IMAGE_BINDING_INVALID", "EXPLORE_FAILED"]) {
|
||||
await assert.rejects((await sideInSessions({ dark: new CiError({ ...DARK_FAILURE, code }) })).run(), { code }, code);
|
||||
}
|
||||
const stop = new AbortController(); stop.abort(new CiError({ code: "LOCAL_REVIEW_INTERRUPTED", stage: "capture", message: "Interrupted.", nextAction: "Rerun." }));
|
||||
await assert.rejects((await sideInSessions({ dark: new CiError({ ...DARK_FAILURE }), signal: stop.signal })).run(), { code: "CAPTURE_SESSION_FAILED" });
|
||||
});
|
||||
@@ -1,6 +1,10 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { hostedRepackBuildId, hostedRepackWorkflow } from '../../dist/capture/eas-build.js';
|
||||
import { chmod, mkdir, mkdtemp, rm, symlink, writeFile } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { hostedRepackBuildId, hostedRepackFailure, hostedRepackRetries, hostedRepackRunStart, hostedRepackShareKey, hostedRepackShareMarker, hostedRepackWorkflow, sharedRepackBuildView, sharedRepackWaitMs, sharedRepackBuilds, stagedProjectDigest } from '../../dist/capture/eas-build.js';
|
||||
import { repackShareOption } from '../../dist/ci/pipeline.js';
|
||||
import { CiError } from '../../dist/shared/ci-errors.js';
|
||||
|
||||
test('the generated repack workflow pins the source build, skips VCS and points Metro at the staged observer', () => {
|
||||
@@ -42,3 +46,121 @@ test('an exhausted build plan surfaces as EAS_PLAN_LIMIT_REACHED even from the h
|
||||
test('non-JSON output from eas workflow:run is reported with the CLI tail', () => {
|
||||
assert.throws(() => hostedRepackBuildId('', 'Error: Not logged in\n', 'ios'), error => error instanceof CiError && error.code === 'EAS_REPACK_FAILED' && /Not logged in/.test(error.message));
|
||||
});
|
||||
|
||||
const failed = (job, stderr = '') => { try { hostedRepackBuildId(run(job), stderr, 'ios'); } catch (error) { return error; } assert.fail('the repack did not fail'); };
|
||||
|
||||
test('a repack EAS failed around is repacked twice more, an unexplained one once, and one the app, a credential, a timeout or a person ended never', () => {
|
||||
// brentvatne/euxy#176 (review run 39506405687): the upload step's report of "Turtle build is not in progress".
|
||||
const upload = failed({ key: 'repack', status: 'FAILURE', turtleBuild: { status: 'ERRORED', error: { errorCode: 'UNKNOWN_CUSTOM_BUILD_ERROR', message: 'Failed to upload application archive.' } } });
|
||||
assert.equal(upload.details.failure, 'infrastructure'); assert.equal(hostedRepackRetries(upload), 2);
|
||||
assert.equal(hostedRepackRetries(failed({ key: 'repack', status: 'FAILURE', turtleBuild: { error: { errorCode: 'SERVER_ERROR', message: 'Internal error.' } } })), 2);
|
||||
assert.equal(hostedRepackRetries(failed({ key: 'repack', status: 'ERRORED' })), 1, 'a run that says nothing is tried once more');
|
||||
const unexplained = { key: 'repack', status: 'FAILURE', turtleBuild: { error: { errorCode: 'UNKNOWN_CUSTOM_BUILD_ERROR', message: 'Unknown custom build error. See logs for more information.' } } };
|
||||
assert.equal(hostedRepackRetries(failed(unexplained)), 1);
|
||||
// What the CLI printed and the failed job's logs count too: a bundling error there is the app's.
|
||||
assert.equal(failed(unexplained, 'SyntaxError: app/home.tsx: Unexpected token (4:2)\n').details.failure, 'app');
|
||||
assert.equal(failed(unexplained, 'Build step "Repack app" failed\n').details.failure, 'app');
|
||||
assert.equal(failed(unexplained, 'Build step "Spin up build environment" failed\n').details.failure, 'infrastructure');
|
||||
for (const message of ['Build step "Repack app" failed', 'Unable to resolve module ./missing from app/index.tsx', 'SyntaxError: app/home.tsx: Unexpected token (4:2)']) {
|
||||
const error = failed({ key: 'repack', status: 'FAILURE', turtleBuild: { error: { errorCode: 'UNKNOWN_CUSTOM_BUILD_ERROR', message } } });
|
||||
assert.equal(error.details.failure, 'app', message); assert.equal(hostedRepackRetries(error), 0, message);
|
||||
}
|
||||
assert.equal(hostedRepackRetries(failed({ key: 'repack', status: 'CANCELED' })), 0, 'a run someone cancelled is not started again');
|
||||
// A build that ran out its time (a bundling hang, say) would run it out again.
|
||||
assert.equal(failed({ key: 'repack', status: 'FAILURE', turtleBuild: { error: { errorCode: 'EAS_BUILD_TIMEOUT_ERROR', message: 'Build timed out.' } } }).details.failure, 'timeout');
|
||||
// Before a run starts: a refused token is not retried, a failed upload of the staged copy is.
|
||||
assert.throws(() => hostedRepackBuildId('', 'Error: Not logged in\n', 'ios'), error => hostedRepackRetries(error) === 0);
|
||||
assert.throws(() => hostedRepackBuildId('', 'Failed to upload project sources.\nError: socket hang up\n', 'ios'), error => hostedRepackRetries(error) === 2);
|
||||
assert.equal(hostedRepackRetries(new Error('spawnSync eas ETIMEDOUT')), 0, 'a command that ran out its time is not repeated');
|
||||
// The run's own link names it, as eas workflow:view --json reports it.
|
||||
assert.equal(failed({ key: 'repack', status: 'ERRORED' }).details.logs, undefined);
|
||||
assert.throws(() => hostedRepackBuildId(JSON.stringify({ id: 'run-2', logURL: 'https://expo.dev/accounts/acme/projects/example/workflows/run-2', jobs: [{ key: 'repack', status: 'ERRORED' }] }), '', 'ios'),
|
||||
error => error.details.logs === 'https://expo.dev/accounts/acme/projects/example/workflows/run-2');
|
||||
});
|
||||
|
||||
test('only exact EAS words make a failure infrastructure, and a refused credential is never retried', () => {
|
||||
const cases = {
|
||||
infrastructure: ["Error: Cannot find module '/opt/eas/node_modules/@expo/build-tools/dist/index.js'", 'Turtle build is not in progress, it cannot save artifacts.', 'Unexpected response from server (503): Service Unavailable', 'Error: socket hang up',
|
||||
'getaddrinfo EAI_AGAIN api.expo.dev', 'Failed to start the workflow with the API.', 'Build step "Upload artifact" failed', 'Build step "Download build" failed'],
|
||||
app: ['Unable to resolve "react-native-reanimated" from "index.js"', 'Asset upload failed: could not read ./assets/icon.png', "Cannot find module './missing'",
|
||||
'Build step "Upload artifact" failed\nAsset upload failed: could not read ./assets/icon.png'],
|
||||
configuration: ['npm error code E401\nnpm error 401 Unauthorized - GET https://npm.example.com/download/@acme/ui-1.0.0.tgz', 'Error: Not logged in',
|
||||
'Entity not authorized: AppEntity[1234]', 'Request failed: 403 Forbidden'],
|
||||
timeout: ['EAS_BUILD_TIMEOUT_ERROR Build timed out.'],
|
||||
unknown: ['Unexpected response from server (400): bad request', 'a download of 500 files', 'Build step "Install dependencies" failed', 'the request has a timeout of 10 s',
|
||||
'Permission check finished', ''],
|
||||
};
|
||||
for (const [failure, texts] of Object.entries(cases)) for (const text of texts) assert.equal(hostedRepackFailure(text), failure, text);
|
||||
assert.equal(hostedRepackFailure('Turtle build is not in progress', 'CANCELED'), 'canceled');
|
||||
});
|
||||
|
||||
test('later targets wait a minute longer each, so a missing leader is replaced by one repack and not one per target', () => {
|
||||
assert.deepEqual([0, 1, 2, 3].map(rank => sharedRepackWaitMs({ rank })), [0, 180_000, 240_000, 300_000]);
|
||||
assert.deepEqual([0, 1, 2, 3].map(rank => sharedRepackWaitMs({ rank, rerun: true })), [0, 60_000, 120_000, 180_000]);
|
||||
assert.deepEqual([0, 1, 2, 3].map(rank => sharedRepackWaitMs({ rank }, true)), [60_000, 60_000, 120_000, 180_000]);
|
||||
});
|
||||
|
||||
test('a started run is known by its JSON or by the link it printed, and a start that printed neither started none', () => {
|
||||
const url = 'https://expo.dev/accounts/acme/projects/example/workflows/01a15b61-c4a2-7c7b-b950-9f280edea954';
|
||||
assert.deepEqual(hostedRepackRunStart(JSON.stringify({ id: '01a15b61-c4a2-7c7b-b950-9f280edea954', url }), '', 'ios'), { id: '01a15b61-c4a2-7c7b-b950-9f280edea954', url });
|
||||
assert.deepEqual(hostedRepackRunStart('', `Using workflow file from .eas/workflows/expo-explore-repack.yml\nSee logs: ${url}\n`, 'ios'), { id: '01a15b61-c4a2-7c7b-b950-9f280edea954', url });
|
||||
assert.throws(() => hostedRepackRunStart('', 'Failed to upload project sources.\nError: socket hang up\n', 'ios'),
|
||||
error => error.code === 'EAS_REPACK_FAILED' && hostedRepackRetries(error) === 2 && /did not start the hosted ios repack/.test(error.message));
|
||||
assert.throws(() => hostedRepackRunStart('', 'Error: Not logged in\n', 'ios'), error => hostedRepackRetries(error) === 0);
|
||||
});
|
||||
|
||||
test('the staged digest changes with any file, mode or link, and the share key with every input', async t => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'explore-staged-digest-'));
|
||||
t.after(() => rm(root, { recursive: true, force: true }));
|
||||
await mkdir(join(root, 'app'));
|
||||
await writeFile(join(root, 'app', 'index.js'), 'export default 1;\n');
|
||||
await writeFile(join(root, 'package.json'), '{}\n');
|
||||
await symlink('app/index.js', join(root, 'entry.js'));
|
||||
const digests = [await stagedProjectDigest(root)];
|
||||
assert.equal(await stagedProjectDigest(root), digests[0], 'the digest is stable');
|
||||
await writeFile(join(root, 'app', 'index.js'), 'export default 2;\n'); digests.push(await stagedProjectDigest(root));
|
||||
await chmod(join(root, 'package.json'), 0o755); digests.push(await stagedProjectDigest(root));
|
||||
await rm(join(root, 'entry.js')); await symlink('package.json', join(root, 'entry.js')); digests.push(await stagedProjectDigest(root));
|
||||
await mkdir(join(root, 'empty')); digests.push(await stagedProjectDigest(root));
|
||||
assert.equal(new Set(digests).size, digests.length);
|
||||
const input = { scope: 'acme/example#39506405687.1', platform: 'ios', nativeBuildId: 'a81d5d80', easProfile: 'explore', environment: 'preview', environmentVariables: 'a'.repeat(64), project: digests[0] };
|
||||
const key = hostedRepackShareKey(input);
|
||||
assert.match(key, /^[0-9a-f]{32}$/);
|
||||
for (const [name, other] of Object.entries({ scope: 'acme/example#39506405687.2', platform: 'android', nativeBuildId: 'b0000000', easProfile: 'other', environment: 'production',
|
||||
environmentVariables: 'b'.repeat(64), project: digests[1] })) {
|
||||
assert.notEqual(hostedRepackShareKey({ ...input, [name]: other }), key, name);
|
||||
}
|
||||
});
|
||||
|
||||
test("only a build whose message ends with this key, for this platform, is another job's repack", () => {
|
||||
const marker = hostedRepackShareMarker('0'.repeat(32));
|
||||
const message = `expo-explore ios capture: route observer repack ${marker}`;
|
||||
const listed = JSON.stringify([
|
||||
{ id: 'shared', status: 'IN_PROGRESS', platform: 'IOS', message },
|
||||
{ id: 'android', status: 'FINISHED', platform: 'ANDROID', message },
|
||||
{ id: 'other-key', status: 'FINISHED', platform: 'IOS', message: `expo-explore ios capture: route observer repack ${hostedRepackShareMarker('1'.repeat(32))}` },
|
||||
{ id: 'suffixed', status: 'FINISHED', platform: 'IOS', message: `${message} and more` },
|
||||
{ id: '../escape', status: 'FINISHED', platform: 'IOS', message },
|
||||
{ id: 'no-message', status: 'FINISHED', platform: 'IOS' },
|
||||
]);
|
||||
assert.deepEqual(sharedRepackBuilds(listed, marker, 'ios'), [{ id: 'shared', status: 'IN_PROGRESS' }]);
|
||||
assert.equal(sharedRepackBuilds('Error: GraphQL request failed', marker, 'ios'), undefined);
|
||||
// A followed build is read by its id: null once it no longer carries the key, undefined when unreadable.
|
||||
assert.deepEqual(sharedRepackBuildView(JSON.stringify({ id: 'shared', status: 'FINISHED', platform: 'IOS', message }), marker, 'ios'), { id: 'shared', status: 'FINISHED' });
|
||||
assert.equal(sharedRepackBuildView(JSON.stringify({ id: 'shared', status: 'FINISHED', platform: 'IOS', message: 'edited' }), marker, 'ios'), null);
|
||||
assert.equal(sharedRepackBuildView('', marker, 'ios'), undefined);
|
||||
});
|
||||
|
||||
test("a target shares its repacks only with its own GitHub run's jobs, and only when another target repacks the same app", () => {
|
||||
const config = { target: { id: 'duo-open', platform: 'ios', label: 'Duo open' }, capture: {} };
|
||||
const environment = { GITHUB_RUN_ID: '39506405687', GITHUB_REPOSITORY: 'brentvatne/euxy' };
|
||||
assert.deepEqual(repackShareOption(config, environment, ['iphone', 'duo-open']), { repackShare: { scope: 'brentvatne/euxy#39506405687.1', leader: 'iphone', target: 'duo-open', rank: 1 } });
|
||||
// "Re-run failed jobs" is an attempt of its own, whose stagger starts at once: the first attempt's leader may not run again.
|
||||
assert.deepEqual(repackShareOption(config, { ...environment, GITHUB_RUN_ATTEMPT: '2' }, ['iphone', 'duo-open']),
|
||||
{ repackShare: { scope: 'brentvatne/euxy#39506405687.2', leader: 'iphone', target: 'duo-open', rank: 1, rerun: true } });
|
||||
assert.deepEqual(repackShareOption(config, { ...environment, GITHUB_RUN_ATTEMPT: 'x' }, ['iphone', 'duo-open']), {});
|
||||
assert.deepEqual(repackShareOption(config, environment, ['duo-open']), {}, 'a single target behaves as before');
|
||||
assert.deepEqual(repackShareOption(config, environment, undefined), {});
|
||||
assert.deepEqual(repackShareOption(config, {}, ['iphone', 'duo-open']), {}, 'outside GitHub Actions there is no run to share with');
|
||||
assert.deepEqual(repackShareOption({ ...config, capture: { repack: 'local' } }, environment, ['iphone', 'duo-open']), {});
|
||||
});
|
||||
@@ -5,7 +5,7 @@ import { spawn, spawnSync } from 'node:child_process';
|
||||
import { lstat, mkdtemp, readFile, realpath, rm, writeFile } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { appEnvironment, captureCacheDirectory, defaultBase, installDependencies, keepCapture, localInstallCommand, localReviewConfig, reportRef, reviewHeadSource, runLocalReview, snapshotWorkingTree, stopRecordedSessions } from '../../dist/ci/local-review.js';
|
||||
import { appEnvironment, captureCacheDirectory, defaultBase, installDependencies, keepCapture, localInstallCommand, localReviewConfig, reportRef, pruneTargetOutput, reviewHeadSource, runLocalReview, snapshotWorkingTree, stopRecordedSessions } from '../../dist/ci/local-review.js';
|
||||
import { mkdir, readdir } from 'node:fs/promises';
|
||||
import { startLocalReportServer } from '../../dist/report/local-report-server.js';
|
||||
import { loadReport } from '../../dist/report/agent-api.js';
|
||||
@@ -364,6 +364,19 @@ test('an interrupted review deletes the local simulators it recorded and stops i
|
||||
await assert.rejects(lstat(join(output, 'head-session')));
|
||||
});
|
||||
|
||||
test('a finished target keeps its report and any session file, so the review still stops a session no capture confirmed stopped', async t => {
|
||||
const output = await realpath(await mkdtemp(join(tmpdir(), 'explore-pruned-')));
|
||||
t.after(() => rm(output, { recursive: true, force: true }));
|
||||
for (const name of ['report.zip', 'head-session-dark', 'head-export', 'base-export-dark', 'head-map']) await writeFile(join(output, name), 'x\n');
|
||||
await mkdir(join(output, 'report'));
|
||||
await pruneTargetOutput(output);
|
||||
assert.deepEqual((await readdir(output)).sort(), ['head-session-dark', 'report', 'report.zip']);
|
||||
const eas = [];
|
||||
await writeFile(join(output, 'head-session-dark'), '01a15b61-4df9\n');
|
||||
assert.deepEqual(await stopRecordedSessions([output], {}, args => { eas.push(args); return { status: 0 }; }, true), ['01a15b61-4df9']);
|
||||
assert.deepEqual(eas, [['simulator:stop', '--id', '01a15b61-4df9', '--json', '--non-interactive']]);
|
||||
});
|
||||
|
||||
test('a local review captures every target of the config at once, two of them on one platform', async t => {
|
||||
const targets = { phone: { platform: 'ios' }, 'duo-open': { platform: 'ios', device: 'iPhone Duo', pose: 'open' } };
|
||||
const f = await fixture(t, 'expo-router', { targets });
|
||||
|
||||
@@ -325,6 +325,38 @@ test('a failed dark pass on a route that captured in light reads as a valid bund
|
||||
assert.equal(dark.headOutcome?.error?.code, 'CAPTURE_OBSERVER_MISSING');
|
||||
});
|
||||
|
||||
test('a route light lost and whose dark session failed reviews dark with the dark session\'s reason, not light\'s', async t => {
|
||||
// brentvatne/euxy's main baseline run 39506374824: the dark session's install failed; here light also lost the route.
|
||||
const f = await captureFixture(t);
|
||||
const error = { code: 'CAPTURE_SESSION_FAILED', stage: 'session', message: 'install: argent run list-devices failed: GET /tools failed: 404 Not Found',
|
||||
nextAction: 'Rerun the capture on a new session.', retryable: true };
|
||||
for (const side of ['base', 'head']) {
|
||||
const bundle = f.bundles[side];
|
||||
bundle.runtimeProfile = { ...runtimeProfile, colorSchemes: 'light,dark' };
|
||||
if (side === 'base') {
|
||||
await writeFile(join(f.root, side, 'home-dark.png'), png);
|
||||
bundle.images[`${f.id}@dark`] = { ...bundle.images[f.id], path: 'home-dark.png' };
|
||||
} else {
|
||||
delete bundle.images[f.id];
|
||||
bundle.outcomes[f.id] = { status: 'failed', attempts: 2, landing: 'unverified', reason: 'The route observer never answered.' };
|
||||
bundle.outcomes[`${f.id}@dark`] = { status: 'failed', attempts: 0, landing: 'unverified', reason: error.message, error };
|
||||
bundle.sessionFailures = [{ scheme: 'dark', error }];
|
||||
}
|
||||
await writeFile(join(f.root, side, 'bundle.json'), JSON.stringify(bundle));
|
||||
}
|
||||
await readCaptureBundle(f.options.headDirectory);
|
||||
const result = await prepareCaptureReview(f.options);
|
||||
assertPublicReview(result.review);
|
||||
const [light, dark] = result.review.surfaces;
|
||||
assert.equal(light.headOutcome.reason, 'The route observer never answered.');
|
||||
assert.equal(dark.headOutcome.error.code, 'CAPTURE_SESSION_FAILED');
|
||||
assert.equal(dark.headOutcome.reason, error.message);
|
||||
// Without the session's record, a failed dark outcome beside a failed light one is not one the passes produce.
|
||||
delete f.bundles.head.sessionFailures;
|
||||
await writeFile(join(f.root, 'head', 'bundle.json'), JSON.stringify(f.bundles.head));
|
||||
await assert.rejects(readCaptureBundle(f.options.headDirectory), /inventory is inconsistent/);
|
||||
});
|
||||
|
||||
test('a head that gained a light appearance keeps its route in source navigation on the dark-only base', async t => {
|
||||
const f = await captureFixture(t);
|
||||
f.bundles.base.runtimeProfile = { ...runtimeProfile, colorSchemes: 'dark' };
|
||||
|
||||
@@ -268,6 +268,46 @@ test('capture problems are counted on the target\'s line and listed, collapsed,
|
||||
assert.doesNotMatch(body, /needs review|Modified|old capture|old destination|token=private|\*\*iOS\*\*|Validation/);
|
||||
});
|
||||
|
||||
test('a target whose dark session failed reads as ready, says dark was not captured, and names the session once', async () => {
|
||||
const { readCommentRecords } = await import('../../dist/report/pr-comment-body.js');
|
||||
// brentvatne/euxy's main baseline run 39506374824: the light session captured every route, the dark session's install failed.
|
||||
const error = { code: 'CAPTURE_SESSION_FAILED', stage: 'session', message: 'install: argent run list-devices failed: GET /tools failed: 404 Not Found',
|
||||
nextAction: 'Rerun the capture on a new session.', retryable: true };
|
||||
const unchanged = { review: { status: 'unchanged', description: 'No visible change' }, visual: { verdict: 'unchanged', reason: 'matched', mismatchPercentage: 0, threshold: 0.1, tool: '[email protected]' } };
|
||||
const surfaces = [];
|
||||
// 24 routes: more dark frames than the comment lists issues, and one light route the light session lost.
|
||||
for (let index = 0; index < 24; index++) {
|
||||
const id = `route-${index}`;
|
||||
const light = surface(id, 'modified', { extra: { colorScheme: 'light', ...unchanged } });
|
||||
if (index === 23) { delete light.headImage; delete light.review; light.headOutcome = { status: 'failed', attempts: 2, landing: 'unverified', reason: 'The route observer never answered.' }; }
|
||||
const dark = surface(`${id}@dark`, 'modified', { route: `/${id}`, extra: { colorScheme: 'dark', headOutcome: { status: 'failed', attempts: 0, landing: 'unverified', reason: error.message, error } } });
|
||||
delete dark.headImage;
|
||||
surfaces.push(light, dark);
|
||||
}
|
||||
const health = { planned: 24, captured: 24, verified: 24, unverified: 0, unavailable: 0 };
|
||||
const evidence = { visual: null, accessibility: null, capture: { base: health, head: { ...health, captured: 23, verified: 23, unavailable: 1 } },
|
||||
comparison: { planned: 24, eligible: 23, skipped: 0 }, diagnostics: [], sessionFailures: [{ side: 'head', scheme: 'dark', frames: 24, ...error }] };
|
||||
const body = renderPublicComment(state({ review: review(surfaces), evidence, capture: { planned: { base: 24, head: 24 }, done: { base: 24, head: 23 }, baseFromCache: true } }));
|
||||
assert.doesNotMatch(body, /failed\*\*|capture failed|🟥/, 'one appearance missing is not a failed review');
|
||||
assert.match(titleOf(body), /^### No compared screens changed · dark not captured · /);
|
||||
assert.equal(statusOf(body, 'iOS'), `iOS · dark not captured · 1 capture unavailable · ${runLink}`);
|
||||
assert.deepEqual(summaries(body), ['Capture issues (2)'], 'the session once, and the light route it lost');
|
||||
assert.match(body, /\n- Head · dark session · CAPTURE\\_SESSION\\_FAILED: install\\: argent run list\\-devices failed\\: GET \/tools failed\\: 404 Not Found\\. 24 dark frames are unavailable\\. Next action: Rerun the capture on a new session\\\.\n/);
|
||||
assert.match(body, /\n- Head · \/route\\-23 · CAPTURE\\_UNAVAILABLE: The route observer never answered\\. /, 'the light route is not crowded out');
|
||||
assert.doesNotMatch(body, /\(dark\\\) · CAPTURE\\_SESSION\\_FAILED/, 'no dark frame repeats the session');
|
||||
const record = readCommentRecords(body).current.ios;
|
||||
assert.deepEqual(record.sessionFailures, [{ side: 'head', scheme: 'dark', frames: 24 }], 'the record keeps it, so a republished comment says it too');
|
||||
});
|
||||
|
||||
test('a dark issue that is not the session\'s is still listed with its appearance', () => {
|
||||
const error = { code: 'CAPTURE_OBSERVER_MISSING', stage: 'capture', message: 'The capture build has not exposed its active navigation destination.', nextAction: 'Rerun capture.', retryable: true };
|
||||
const dark = surface('settings@dark', 'modified', { route: '/settings', extra: { colorScheme: 'dark', headOutcome: { status: 'failed', attempts: 2, landing: 'unverified', reason: error.message, error } } });
|
||||
delete dark.headImage;
|
||||
const body = renderPublicComment(state({ review: review([surface('settings', 'modified', { extra: { colorScheme: 'light' } }), dark]), evidence: undefined }));
|
||||
assert.match(body, /\n- Head · \/settings \\\(dark\\\) · CAPTURE\\_OBSERVER\\_MISSING/);
|
||||
assert.doesNotMatch(titleOf(body), /not captured/);
|
||||
});
|
||||
|
||||
test('zero eligible pairs are counted as unattributed instead of successful comparisons', () => {
|
||||
const flagged = evidenceReview();
|
||||
for (const surface of flagged.surfaces) {
|
||||
|
||||
@@ -684,3 +684,20 @@ test("on a foldable a region across the pill's edge is kept whole, and one acros
|
||||
assert.deepEqual([clipped.ignoredStatusBar, clipped.ignoredHomeIndicator, clipped.measurement.match, clipped.measurement.differentPixels], [0, 1, false, 80]);
|
||||
assert.deepEqual(clipped.measurement.regions.map((entry) => [entry.rect.y, entry.rect.height, entry.differentPixels, entry.shareOfDiffPercentage]), [[840, 8, 80, 100]]);
|
||||
});
|
||||
|
||||
test("a failed appearance pass is summarized once per side, with the frames it left unavailable", async (t) => {
|
||||
// brentvatne/euxy's main baseline run 39506374824: the dark session's install failed after light captured every route.
|
||||
const root = await mkdtemp(join(tmpdir(), "explore-public-evidence-")); t.after(() => rm(root, { recursive: true, force: true }));
|
||||
const base = await side(root, "base", manifests.base);
|
||||
const head = await side(root, "head", manifests.head);
|
||||
const error = { code: "CAPTURE_SESSION_FAILED", stage: "session", message: "install: argent run list-devices failed: GET /tools failed: 404 Not Found",
|
||||
nextAction: "Rerun the capture on a new session.", retryable: true };
|
||||
head.bundle.runtimeProfile = { deviceModel: "iPhone 17", osVersion: "26.0", locale: "en-US", orientation: "portrait", colorScheme: "light", colorSchemes: "light,dark" };
|
||||
for (const id of head.bundle.selectedSurfaceIds) head.bundle.outcomes[`${id}@dark`] = { status: "failed", attempts: 0, landing: "unverified", reason: error.message, error };
|
||||
head.bundle.sessionFailures = [{ scheme: "dark", error }];
|
||||
const result = await assessPublicEvidence({ plan: plan(), base, head, profile: "explore:bundle", agentDeviceCommand: "/nonexistent/agent-device", measure: measurer({}) });
|
||||
assert.deepEqual(result.summary.sessionFailures, [{ side: "head", scheme: "dark", frames: head.bundle.selectedSurfaceIds.length, ...error }]);
|
||||
delete head.bundle.sessionFailures;
|
||||
const complete = await assessPublicEvidence({ plan: plan(), base, head, profile: "explore:bundle", agentDeviceCommand: "/nonexistent/agent-device", measure: measurer({}) });
|
||||
assert.equal(complete.summary.sessionFailures, undefined);
|
||||
});
|
||||
@@ -740,6 +740,29 @@ test("a planned screen whose capture failed is unavailable, with the report's re
|
||||
assert.equal(isChangedStatus(screen.status), false);
|
||||
});
|
||||
|
||||
test("a dark frame whose session failed is unavailable with the session's error, and its light frame is unaffected", () => {
|
||||
// brentvatne/euxy's main baseline run 39506374824: the light session captured every route, the dark session's install failed.
|
||||
const review = navigationReview();
|
||||
const scan = review.surfaces.find(surface => surface.id === "scan-mood");
|
||||
const error = { code: "CAPTURE_SESSION_FAILED", stage: "session", message: "install: argent run list-devices failed: GET /tools failed: 404 Not Found",
|
||||
nextAction: "Rerun the capture on a new session.", retryable: true };
|
||||
scan.colorScheme = "light";
|
||||
review.surfaces.push({ id: "scan-mood@dark", status: "unchanged", colorScheme: "dark",
|
||||
base: entry("scan-mood@dark", scan.base.route, "mood"), head: entry("scan-mood@dark", scan.head.route, "mood"),
|
||||
baseImage: { ...scan.baseImage, url: `${scan.baseImage.url}-dark`, sha256: "f".repeat(64) }, baseOutcome: { status: "captured", attempts: 1, landing: "verified" },
|
||||
headOutcome: { status: "failed", attempts: 0, landing: "unverified", reason: error.message, error } });
|
||||
verifyImages(review);
|
||||
const diff = diffFor(review);
|
||||
const dark = diff.screens.get("scan-mood@dark");
|
||||
assert.equal(dark.status, "unavailable");
|
||||
assert.match(dark.captureNotes.after, /^Capture failed: install: argent run list-devices failed: GET \/tools failed: 404 Not Found/);
|
||||
assert.equal(dark.afterScreenshot, undefined);
|
||||
assert.notEqual(diff.screens.get("scan-mood").status, "unavailable", "the light frame keeps its own verdict");
|
||||
const capture = publicReviewProjection(review).runtimeEvidence["public:scan-mood@dark"].current.groups.capture;
|
||||
assert.equal(capture.find(row => row.label === "Capture error").value, `CAPTURE_SESSION_FAILED: ${error.message}`);
|
||||
assert.equal(capture.find(row => row.label === "Next action").value, error.nextAction);
|
||||
});
|
||||
|
||||
test("source navigation names routes, so a route whose only base frame is a secondary appearance stays known on base", () => {
|
||||
const review = navigationReview();
|
||||
const scan = review.surfaces.find(surface => surface.id === "scan-mood");
|
||||
|
||||
Reference in new issue
Block a user