Compare commits

..
Author SHA1 Message Date
Douglas LowderandClaude Opus 5.5 0f6a755640 [eas-cli] Fix workflow:validate crash on job schemas without a type
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 14:51:48 -07:00
SJ Kim 9b247a6197 [build-tools] Collect shared simulator logs (#4463)
* [build-tools] Collect shared simulator logs (ENG-26642)

* [build-tools] Link simulator log changelog entry (ENG-26642)

* [build-tools] Require user-app scope for session logs (ENG-26642)

* [build-tools] Name simulator log artifacts app-logs.ndjson (ENG-26642)

* [build-tools] Time out simulator log uploads per device (ENG-26642)

* [build-tools] Report simulator log failures to Sentry (ENG-26642)

* [build-tools] Use a type import for the log stream body (ENG-26642)

* [build-tools] Validate serve-sim log envelopes and report malformed records (ENG-26642)
2026-09-30 16:25:48 -04:00
Stanisław Chmiela 862d0a8f09 [build-tools] Clear old Maestro HTML reports before each attempt (#4409)
It seems when auto-retrying an old HTML report may be uploaded instead of a more recent one. This clears reports between attempts.

(I think behavior of auto-retry + relying on HTML reports is a bit wonky so I guess users should then use the "retry-all" retry behavior.)
2026-09-30 09:15:04 +02:00
Szymon Dziedzic 417996ade0 [eas-cli] Keep the simulator session spinner on one line (#4488) 2026-09-29 21:41:58 +02:00
Stanisław Chmiela 1c6161432a [build-tools] Add "native" TestFlight metadata step (#4475)
# Why

While working on moving off of Submission Service to job runs I found a feature discrepancy — Submission Service supported changelog and assigning to groups, but job runs did not.

# How

Added an EAS function that uses ASC API to add changelog to build and build to groups. It more or less follows what Fastlane does.

See https://github.com/expo/universe/pull/31473.

# Test Plan

- The deployed worker ran the built-in step against an existing Apple build: [staging workflow](https://staging.expo.dev/accounts/sjchmiela/projects/staging-app/workflows/01a0d929-8130-74b2-b014-0b95431907c2).
- Full end-to-end staging test passed with a [fresh signed IPA](https://staging.expo.dev/accounts/sjchmiela/projects/staging-app/builds/45c69b90-649f-4908-a50a-49044e92b18d): [workflow](https://staging.expo.dev/accounts/sjchmiela/projects/staging-app/workflows/01a0d934-b692-71ad-9aba-f9acec8d3888). `eas/upload_to_asc` returned upload ID `8ee25971-3463-4d6d-856f-947645988a7a`. The built-in step selected Apple build `3880e9fc-bf7b-430c-a865-e399a1398667`. Separate Apple API reads confirmed the exact changelog, test-group membership, and no beta review submission.
2026-09-29 13:51:40 +00:00
Stanisław Chmiela 3f667ae4a8 [build-tools] Support maestro-runner HTML and Allure reports (#4406)
We're seeing customers have good experience with `maestro-runner`. Let's add some support for exporting HTML and Allure reports from it.

Runner report generation: https://github.com/devicelab-dev/maestro-runner/blob/main/pkg/cli/test.go
2026-09-29 15:17:03 +02:00
Szymon DziedzicandClaude Opus 5.5 4fb6de564c [build-tools] Disable agent-device internal idle timeouts in remote sessions (#4485)
* [build-tools] Disable agent-device internal idle timeouts in remote sessions

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* [build-tools] Add changelog entry

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-09-29 12:29:30 +02:00
SJ Kim 975693e96e [build-tools] Detect early Argent startup exits (#4454)
* [build-tools] Detect early Argent startup exits (ENG-22026)

* [build-tools] Observe startup exit before pipe closure (ENG-22026)
2026-09-28 23:10:40 -04:00
Brent Vatne b84233bf07 [eas-cli] Recount spinner lines when the terminal resizes (#4470) 2026-09-28 18:14:11 -07:00
Stanisław Chmiela 6dec2f9a13 [eas-cli] Allow preparing submission credentials without loading app config (#4471)
<!-- You can skip the changelog check by labeling the PR with "no changelog". -->

# Why

Having to load app config requires installing node modules which takes time and requires thinking about Node.js version, images etc.

# How

Added two optional parameters to `submit:internal` — `project-id` and `application-identifier`. With them provided we don't need to load project config, so we don't need to install dependencies!

# Test Plan

I'll land this, then land https://github.com/expo/universe/pull/31467 in staging and see if it works!
2026-09-28 21:15:19 +02:00
Vojtech NovakandClaude Fable 5.1 c15e184bb8 [ci] Install ffmpeg so the recording recovery test runs (#4441)
* [ci] Install ffmpeg so the recording recovery test runs

The build-tools test that recovers unlisted recordings from a killed
host cuts a real MP4 and needs ffprobe. It skipped on every CI run.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Fail the recording recovery test loudly without ffmpeg

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [ci] Update the ffmpeg install comment

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-28 12:57:58 +02:00
Vojtech NovakandClaude Fable 5.1 89740dbbfd [build-tools] Share the urql abort-signal wrapper (#4435)
* [build-tools] Add one helper that joins a caller's signal into urql fetch

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Use the shared urql abort helper in both call sites

The artifact upload-session mutation and the sandbox-ready mutation each
wrapped fetch by hand to join a cancellation signal after urql set its own.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-28 12:57:58 +02:00
Vojtech NovakandClaude Fable 5.1 390a165c57 [build-tools] Share remote session teardown across the automation steps (#4431)
* [build-tools] Cover a teardown failure after the session body failed

The error that ended the session must reach the step, not the teardown error.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Share remote session teardown across the automation steps

The Agent Device, Appium and Argent steps each ran Promise.allSettled over
their teardown and rethrew the first rejection from a finally block, which
replaced the error that had ended the session. One helper now runs every
teardown, logs each failure, and rethrows only when the session body succeeded.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Attempt every remote session teardown step independently

A rejected event collection stop no longer leaves the automation process
running. The tunnel close, the collector-then-process pair and the host
finish are separate teardown entries; the process stop sits in a finally.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Name each remote session teardown step in its failure log

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Report teardown failures swallowed after a failed session to Sentry

Co-Authored-By: Claude Fable 5.1 <[email protected]>

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-28 12:57:57 +02:00
Vojtech NovakandClaude Fable 5.1 f7fb43b974 [build-tools] Only stop a session host that never became ready (#4430)
A readiness timeout ran the full finish pipeline: a stop request to a server
that never came up and a read of a recordings.json that did not exist, each
logging a warning unrelated to the real cause. finishAsync now skips
finalization and upload for a host that never answered /readyz. Finalization
and upload failures are reported to Sentry like the per-recording upload
failures one layer down, and the finishAsync contract states that it never
rejects.


Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-28 12:57:57 +02:00
Vojtech NovakandClaude Fable 5.1 a32080fadf [build-tools] Bound recording shutdown and artifact uploads (#4420)
* [build-tools] Bound session shutdown and recording uploads

* [build-tools] Close HTTP requests after each artifact PUT

* [build-tools] Separate artifact retries from PUT cleanup

* [build-tools] Cover a session host that ignores SIGTERM

stopAsync must not resolve while the killed process still answers kill(pid, 0).

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Wait for a killed session host to be reaped

After SIGKILL the child is a zombie until libuv reaps it on a later loop turn,
so the running check right after stopAsync always saw it alive and the
recording upload was skipped.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Cover uploads that progress past the PUT deadline

A body that keeps flowing for longer than the per-attempt cap must still upload.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Bound artifact upload stalls instead of total time

The 90 s PUT deadline now restarts on every chunk the request consumes, and the
300 s artifact and 305 s recording deadlines are gone. Upload time scales with
file size, so a whole-request cap failed every recording a slow link could not
send in time, including iOS and agent artifacts through the shared uploader.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Cover uploading the partial recording a killed Hub leaves

A killed Hub lists no recording; the host must find the fragmented partial,
skip one ffprobe cannot read, upload it flagged partial and remove the root.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Upload the partial recording a killed Hub leaves behind

When recordings.json is empty after the host stops, look for session
directories whose manifest is still at "recording" and whose fragmented
.partial ffprobe can read, then upload them through the shared recording
uploader with partial: true in the artifact metadata. The Hub writes that
manifest at the first keyframe, so the website can still place the video.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Log the Hub's reason when recording finalization is refused

The Hub answers 409 with why nothing was recorded, such as the booted-emulator
count. Read the body instead of throwing on the status so the reason reaches
the job log.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Cover recovering a failed partial recording with its reason

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Upload failed partial recordings with the Hub's reason

A Hub that failed or timed out while finalizing leaves the same fragmented
partial as a killed one, and its manifest now names the file and the error.
Recover "failed" manifests whose file decodes and pass the reason to the
website as partialReason.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Type the deadline error and simplify artifact PUT cleanup

The retry predicate matched the PUT deadline by message text and carried an
unreachable AbortError branch. A DeviceRunSessionTimeoutError replaces both.
node-fetch destroys the request body on abort and the retry loop destroys the
stream on every exit, so the extra abort listener and destroy call go. The
PUT error names the artifact and the status text again.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Do not warn about a preview tunnel that never opened

A preview whose open failed, or that its opener closed after finish began,
rejects previewTask. Retirement awaited that rejection and logged a deadline
warning for a tunnel that no longer existed.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Recover completed recordings a killed Hub never listed

recordings.json is written at finish. A recording that reached its duration
limit is complete on disk but unlisted until the session ends, so a Hub killed
in that window left a finished video the scan skipped. The scan now accepts
"complete" manifests as well and uploads them without the partial flag.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Log the session host's output when a recording is lost

The Hub reports capture failures on stderr, but the worker logged only the
stop route's one-line summary. When finalization fails or nothing is
uploaded, log the last 8 000 characters of the host's output.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Tell a Hub that never recorded apart from a failed finalization

A 409 from the stop route means nothing was captured, and the Hub says why.
Log that reason and skip the upload and the host output dump, which stay
for real capture failures.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Bound the finalization request once, through the deadline wrapper

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Ignore a deadline reset after the bounded operation settled

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Say when ffprobe is missing instead of calling a recording undecodable

Also log the recording by its session directory name rather than the full
temp path.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-28 12:57:57 +02:00
Vojtech NovakandClaude Fable 5.1 55017fbf70 [build-tools] Record Android sessions by default through a session host (#4418)
* [build-tools] Add opt-in Android session recording upload

* [build-tools] Document opt-in Android session recording

* [build-tools] Record Android device sessions by default

* [build-tools] Separate session recording from preview lifetime

* [build-tools] Stop automation alongside recording cleanup

* [build-tools] Fix session host formatting and add lifecycle release note

* [build-tools] Keep session host guidance in the README

* [build-tools] Extract session host shutdown operations

* [build-tools] Keep one changelog entry for the recording stack

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Type the session host variables with DeviceSessionHost

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Throw SystemError when recording finalization fails

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Keep the recording upload step Darwin-only

Android recordings are uploaded by the session host, and nothing on Linux
produces recordings_json.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Drop the session host section from the README

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Name the recording stop grace period after the Hub's SIGTERM deadline

Co-Authored-By: Claude Fable 5.1 <[email protected]>

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-28 12:57:56 +02:00
Byron Karlen 7fd8a3a75b [eas-cli] Remove unnecessary confirm_account query param (#4368) 2026-09-24 19:40:24 -07:00
Szymon DziedzicandClaude Opus 5.5 0f38c2ad82 [eas-cli] Include legacy sessions in the web-preview-only filter (#4466)
* [eas-cli] Include legacy sessions in the web-preview-only filter

`simulator:list --type web-preview-only` now also matches sessions with the
deprecated SERVE_SIM type. The --type flags list each value once.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* [eas-cli] Add changelog entry

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-25 00:21:04 +02:00
60 changed files with 5307 additions and 884 deletions

No files matched your search

+3
View File
@@ -26,6 +26,9 @@ jobs:
uses: ./.github/actions/setup-mise
env:
MISE_NODE_VERSION: ${{ matrix.node }}
- name: Install ffmpeg
# build-tools recording-recovery tests cut and decode real MP4 files with ffmpeg and ffprobe.
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends ffmpeg
- name: Install dependencies
run: yarn install --immutable
env:
+12 -1
View File
@@ -8,9 +8,16 @@ This is the log of notable changes to EAS CLI and related packages.
### 🎉 New features
- [eas-cli] Allow workflow submissions to skip loading app config when project and application identifiers are supplied. ([#4471](https://github.com/expo/eas-cli/pull/4471) by [@sjchmiela](https://github.com/sjchmiela))
- [build-tools] Record Android device sessions and upload the screen recording, including the partial recording a killed Device Hub leaves behind. ([#4418](https://github.com/expo/eas-cli/pull/4418) by [@vonovak](https://github.com/vonovak))
- [build-tools] Add start and collect steps that attach bounded simulator logs from serve-sim to device run sessions. ([#4463](https://github.com/expo/eas-cli/pull/4463) by [@sjkim-expo](https://github.com/sjkim-expo))
### 🐛 Bug fixes
- [eas-cli] Stop spinners from printing their text over and over after the terminal is resized. ([#4473](https://github.com/expo/eas-cli/pull/4473) by [@szdziedzic](https://github.com/szdziedzic))
- [build-tools] Clear old Maestro CLI HTML reports before each test attempt so retries cannot upload an earlier report. ([#4409](https://github.com/expo/eas-cli/pull/4409) by [@sjchmiela](https://github.com/sjchmiela))
- [eas-cli] Keep the `eas simulator` session spinner on one line, so resizing the terminal no longer leaves copies of it behind. The `eas simulator:stop` command is now printed once before the spinner starts. ([#4488](https://github.com/expo/eas-cli/pull/4488) by [@szdziedzic](https://github.com/szdziedzic))
- [build-tools] Disable agent-device's internal idle timeouts in remote sessions, so only the session's own idle and duration limits end it. ([#4485](https://github.com/expo/eas-cli/pull/4485) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] Fix `eas workflow:validate` crash when the workflow schema contains a job entry without a `type`. ([#4505](https://github.com/expo/eas-cli/pull/4505) by [@douglowder](https://github.com/douglowder))
### 🧹 Chores
@@ -21,6 +28,7 @@ This is the log of notable changes to EAS CLI and related packages.
- [eas-cli] Add `--build-fingerprint` to `eas simulator` to install the newest build with a given fingerprint. ([#4472](https://github.com/expo/eas-cli/pull/4472) by [@tchayen](https://github.com/tchayen))
- [eas-cli] Accept build IDs in `workflow:ssh <id>`. ([#4452](https://github.com/expo/eas-cli/pull/4452) by [@AHGIJMKLKKZNPJKQR](https://github.com/AHGIJMKLKKZNPJKQR))
- [eas-cli] Make `eas simulator:*` commands visible in help output. ([#4456](https://github.com/expo/eas-cli/pull/4456) by [@szdziedzic](https://github.com/szdziedzic))
- [build-tools] Add `eas/update_testflight_metadata` to set changelogs and TestFlight groups for an uploaded build with an App Store Connect API key. The step fails if a requested group does not exist. ([#4475](https://github.com/expo/eas-cli/pull/4475) by [@sjchmiela](https://github.com/sjchmiela))
- [build-tools] Pass `--share-url` when launching serve-sim so Share copies the expo.dev preview page instead of the tunnel URL. ([#4434](https://github.com/expo/eas-cli/pull/4434) by [@gwdp](https://github.com/gwdp))
- [eas-cli] Add `eas integrations:supabase:advisors` to list unresolved Supabase Security and Performance Advisor findings for the linked Supabase project. ([#4342](https://github.com/expo/eas-cli/pull/4342) by [@fiberjw](https://github.com/fiberjw))
- [eas-cli] Accept individual App Store Connect API keys for submissions, TestFlight setup, and metadata. ([#4249](https://github.com/expo/eas-cli/pull/4249) by [@sswrk](https://github.com/sswrk))
@@ -29,10 +37,13 @@ This is the log of notable changes to EAS CLI and related packages.
### 🐛 Bug fixes
- [build-tools] Report early Argent server exits during simulator startup instead of waiting for the full readiness timeout. ([#4454](https://github.com/expo/eas-cli/pull/4454) by [@sjkim-expo](https://github.com/sjkim-expo))
- [eas-cli] Stop spinners from printing a new line on every frame after the terminal gets narrower. ([#4470](https://github.com/expo/eas-cli/pull/4470) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli] Exclude time waiting for simulator concurrency from the session startup timeout and show queue progress. ([#4411](https://github.com/expo/eas-cli/pull/4411) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] List `eas simulator` only once in `eas --help`. The `sim` shorthand is now a hidden alias, so it still runs but no longer duplicates the entry. ([#4457](https://github.com/expo/eas-cli/pull/4457) by [@krystofwoldrich](https://github.com/krystofwoldrich))
- [build-tools] Start the Argent tool-server with `ARGENT_EMULATOR_NO_WINDOW=1`, so Argent can boot the Android emulator again after it stops. Without it, the windowed emulator fails on the Linux image because `libpulse.so.0` is missing. ([#4465](https://github.com/expo/eas-cli/pull/4465) by [@gwdp](https://github.com/gwdp))
- [eas-cli] Fix `eas account:view` printing `Role: undefined` for the new Release Manager role. ([#4300](https://github.com/expo/eas-cli/pull/4300) by [@byronkarlen](https://github.com/byronkarlen))
- [eas-cli] Include legacy serve-sim sessions in `eas simulator:list --type web-preview-only`, and list `web-preview-only` only once in the `--type` options. ([#4466](https://github.com/expo/eas-cli/pull/4466) by [@szdziedzic](https://github.com/szdziedzic))
### 🧹 Chores
@@ -54,11 +54,14 @@ import { createStartIosSimulatorRecordingsBuildFunction } from './functions/star
import { createStartLocalEgressBuildFunction } from './functions/startLocalEgress';
import { createStartWebPreviewRemoteSessionBuildFunction } from './functions/startWebPreviewRemoteSession';
import { createStartServeSimMetricsBuildFunction } from './functions/startServeSimMetrics';
import { createStartServeSimLogsBuildFunction } from './functions/startServeSimLogs';
import { createCollectServeSimLogsBuildFunction } from './functions/collectServeSimLogs';
import { createStartSandboxBuildFunction } from './functions/startSandbox';
import { createCollectServeSimMetricsBuildFunction } from './functions/collectServeSimMetrics';
import { createUploadArtifactBuildFunction } from './functions/uploadArtifact';
import { createUploadDeviceRunSessionScreenRecordingsBuildFunction } from './functions/uploadDeviceRunSessionScreenRecordings';
import { createUploadToAscBuildFunction } from './functions/uploadToAsc';
import { createUpdateTestFlightMetadataBuildFunction } from './functions/updateTestFlightMetadata';
import { createSetUpNpmrcBuildFunction } from './functions/useNpmToken';
import { createWaitForPosthogMetricFunction } from './functions/waitForPosthogMetric';
import { createUploadPosthogSourcemapsFunction } from './functions/uploadPosthogSourcemaps';
@@ -111,6 +114,8 @@ export function getEasFunctions(ctx: CustomBuildContext): BuildFunction[] {
createUploadDeviceRunSessionScreenRecordingsBuildFunction(ctx),
createStartWebPreviewRemoteSessionBuildFunction(ctx),
createStartServeSimMetricsBuildFunction(),
createStartServeSimLogsBuildFunction(),
createCollectServeSimLogsBuildFunction(ctx),
createStartSandboxBuildFunction(ctx),
createCollectServeSimMetricsBuildFunction(ctx),
createInstallMaestroBuildFunction(),
@@ -128,6 +133,7 @@ export function getEasFunctions(ctx: CustomBuildContext): BuildFunction[] {
createSubmissionEntityFunction(),
createUploadToAscBuildFunction(),
createUpdateTestFlightMetadataBuildFunction(),
createReportMaestroTestResultsFunction(ctx),
createMaestroTestsBuildFunction(ctx),
@@ -0,0 +1,238 @@
import { BuildRuntimePlatform, type BuildStepContext } from '@expo/steps';
import spawn from '@expo/turtle-spawn';
import fs from 'node:fs';
import type { CustomBuildContext } from '../../../customBuildContext';
import { Sentry } from '../../../sentry';
import { AndroidEmulatorUtils } from '../../../utils/AndroidEmulatorUtils';
import { turtleFetch } from '../../../utils/turtleFetch';
import { startAgentDeviceEventCollectionAsync } from '../../utils/agentDeviceEvents';
import { startAppiumEventCollectionAsync } from '../../utils/appiumEvents';
import { startDeviceSessionHostAsync } from '../../utils/deviceSessionHost';
import {
spawnDetached,
startNgrokTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
waitForFileAsync,
} from '../../utils/remoteDeviceRunSession';
import { createStartAgentDeviceRemoteSessionBuildFunction } from '../startAgentDeviceRemoteSession';
import { createStartAppiumRemoteSessionBuildFunction } from '../startAppiumRemoteSession';
jest.mock('@expo/turtle-spawn');
jest.mock('../../../sentry');
jest.mock('../../../utils/turtleFetch');
jest.mock('../../../utils/AndroidEmulatorUtils');
jest.mock('../../utils/deviceSessionHost');
jest.mock('../../utils/agentDeviceEvents');
jest.mock('../../utils/appiumEvents');
jest.mock('../../utils/agentDeviceArtifacts');
jest.mock('../../utils/remoteDeviceRunSession', () => ({
...jest.requireActual('../../utils/remoteDeviceRunSession'),
spawnDetached: jest.fn(),
startNgrokTunnelAsync: jest.fn(),
uploadRemoteSessionConfigAsync: jest.fn(),
waitForDeviceRunSessionStoppedAsync: jest.fn(),
waitForFileAsync: jest.fn(),
}));
const finishHost = jest.fn();
const closePreview = jest.fn();
const stopTool = jest.fn();
const stopTunnel = jest.fn();
const stopEvents = jest.fn();
const openPreview = jest.fn();
const logger = { info: jest.fn(), warn: jest.fn() };
beforeEach(() => {
jest.clearAllMocks();
// The daemon package is external. No installation or daemon process runs in these tests.
jest.spyOn(fs, 'existsSync').mockReturnValue(true);
jest.mocked(spawn).mockResolvedValue({ stdout: '{}' } as never);
jest
.mocked(AndroidEmulatorUtils.getAttachedDevicesAsync)
.mockResolvedValue([{ serialId: 'emulator-5554', state: 'device' } as never]);
jest.mocked(turtleFetch).mockResolvedValue({ ok: true } as never);
jest.mocked(waitForFileAsync).mockResolvedValue({ port: 4567, token: 'daemon-token' });
for (const stop of [finishHost, closePreview, stopTool, stopTunnel, stopEvents]) {
stop.mockResolvedValue(undefined);
}
openPreview.mockResolvedValue({
previewPageUrl: 'https://expo.dev/simulator-preview/preview-id',
apiUrl: 'https://preview.example.test',
closeAsync: closePreview,
});
jest
.mocked(startDeviceSessionHostAsync)
.mockResolvedValue({ openPreviewAsync: openPreview, finishAsync: finishHost });
jest.mocked(spawnDetached).mockReturnValue({
pid: undefined,
getOutput: () => '',
getExitError: () => undefined,
stopAsync: stopTool,
});
jest.mocked(startNgrokTunnelAsync).mockResolvedValue({
url: 'https://tool.example.test',
subdomainId: 'tool-id',
stopAsync: stopTunnel,
});
const events = { stopAsync: stopEvents, getLastEventObservedAt: () => undefined };
jest.mocked(startAgentDeviceEventCollectionAsync).mockResolvedValue(events);
jest.mocked(startAppiumEventCollectionAsync).mockResolvedValue(events);
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockResolvedValue(undefined);
});
afterEach(() => {
jest.restoreAllMocks();
});
describe.each([
['Appium', createStartAppiumRemoteSessionBuildFunction],
['Agent Device', createStartAgentDeviceRemoteSessionBuildFunction],
] as const)('%s host ownership', (name, createFunction) => {
async function runAsync() {
const fn = createFunction({} as CustomBuildContext);
await fn.fn!(
{
logger,
global: { runtimePlatform: BuildRuntimePlatform.LINUX },
} as unknown as BuildStepContext,
{
inputs: {
package_version: { value: undefined },
max_idle_time_minutes: { value: undefined },
},
outputs: {},
env: {
DEVICE_RUN_SESSION_ID: 'session-id',
EAS_SIMULATOR_NGROK_TUNNEL_DOMAIN: 'example.test',
NGROK_AUTHTOKEN: 'token',
},
} as never
);
}
it('holds the host until session completion', async () => {
await runAsync();
expect(openPreview).toHaveBeenCalledWith({ baseDomain: 'example.test' });
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledWith(
expect.objectContaining({
remoteConfig: expect.objectContaining({
webPreviewUrl: 'https://expo.dev/simulator-preview/preview-id',
previewApiUrl: 'https://preview.example.test',
}),
})
);
expect(finishHost).toHaveBeenCalledTimes(1);
expect(closePreview).not.toHaveBeenCalled();
expect(finishHost.mock.invocationCallOrder[0]).toBeGreaterThan(
jest.mocked(waitForDeviceRunSessionStoppedAsync).mock.invocationCallOrder[0]
);
expect(stopTool).toHaveBeenCalledTimes(1);
expect(stopEvents).toHaveBeenCalledTimes(1);
expect(stopTunnel).toHaveBeenCalledTimes(1);
});
it('stops automation while recording finalization or upload is pending', async () => {
let release!: () => void;
let stopped!: () => void;
const pendingFinish = new Promise<void>(resolve => {
release = resolve;
});
const toolStopped = new Promise<void>(resolve => {
stopped = resolve;
});
finishHost.mockReturnValueOnce(pendingFinish);
stopTool.mockImplementationOnce(async () => {
stopped();
});
let completed = false;
const running = runAsync().then(() => {
completed = true;
});
try {
await toolStopped;
expect(stopTunnel).toHaveBeenCalledTimes(1);
expect(finishHost).toHaveBeenCalledTimes(1);
expect(completed).toBe(false);
} finally {
release();
await running;
}
});
it('finalizes recording and stops automation while the tunnel close is pending', async () => {
let release!: () => void;
let finished!: () => void;
const pendingClose = new Promise<void>(resolve => {
release = resolve;
});
const hostFinished = new Promise<void>(resolve => {
finished = resolve;
});
const toolStopped = new Promise<void>(resolve => {
stopTool.mockImplementationOnce(async () => resolve());
});
stopTunnel.mockReturnValueOnce(pendingClose);
finishHost.mockImplementationOnce(async () => {
finished();
});
const running = runAsync();
try {
// Both settle while the tunnel close is still held open.
await Promise.all([hostFinished, toolStopped]);
expect(stopTunnel).toHaveBeenCalledTimes(1);
} finally {
release();
await running;
}
});
it('still stops the automation process when event collection stop rejects', async () => {
stopEvents.mockRejectedValueOnce(new Error('collector failed'));
// Agent Device wraps its collector stop in a catch-all helper; Appium does not.
const outcome = await runAsync().then(
() => 'resolved',
(err: Error) => err.message
);
expect(outcome).toBe(name === 'Appium' ? 'collector failed' : 'resolved');
expect(stopTool).toHaveBeenCalledTimes(1);
expect(stopTunnel).toHaveBeenCalledTimes(1);
expect(finishHost).toHaveBeenCalledTimes(1);
});
it('still stops automation when recording finalization rejects', async () => {
finishHost.mockRejectedValueOnce(new Error('recording cleanup failed'));
await expect(runAsync()).rejects.toThrow('recording cleanup failed');
expect(stopTool).toHaveBeenCalledTimes(1);
expect(stopTunnel).toHaveBeenCalledTimes(1);
expect(Sentry.capture).not.toHaveBeenCalled();
});
it.each(['preview', 'config', 'wait'])('finishes the host after %s fails', async phase => {
const error = new Error(`${phase} failed`);
// A teardown failure must not replace the error that ended the session.
finishHost.mockRejectedValueOnce(new Error('recording cleanup failed'));
if (phase === 'preview') {
openPreview.mockRejectedValueOnce(error);
} else if (phase === 'config') {
jest.mocked(uploadRemoteSessionConfigAsync).mockRejectedValueOnce(error);
} else {
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockRejectedValueOnce(error);
}
await expect(runAsync()).rejects.toBe(error);
expect(finishHost).toHaveBeenCalledTimes(1);
expect(stopTool).toHaveBeenCalledTimes(1);
expect(stopTunnel).toHaveBeenCalledTimes(1);
expect(logger.warn).toHaveBeenCalledWith(
{ err: expect.objectContaining({ message: 'recording cleanup failed' }) },
'Could not stop the session host during remote session teardown.'
);
expect(Sentry.capture).toHaveBeenCalledWith(
'Could not stop the session host after the remote session failed',
expect.objectContaining({ message: 'recording cleanup failed' }),
{ level: 'warning' }
);
});
});
@@ -271,20 +271,124 @@ describe('createMaestroTestsBuildFunction', () => {
expect(mockedSpawn.mock.calls[0][1]).toContain('--parallel=2');
});
it('rejects a non-junit output_format with maestro-runner', async () => {
it.each(['html', 'allure'])(
'accepts %s output_format with maestro-runner and keeps JUnit',
async format => {
mockedSpawn.mockResolvedValue(SPAWN_SUCCESS);
const copyFileSpy = jest.spyOn(fs, 'copyFile').mockResolvedValue();
const step = createStep({
flow_path: ['flows/a.yaml'],
platform: 'android',
backend: 'maestro-runner',
output_format: format,
});
await step.executeAsync();
expect(mockedSpawn.mock.calls[0][1]).toEqual(
expect.arrayContaining([
'--output=/home/expo/.maestro/tests/android-maestro-runner-attempt-0',
])
);
expect(copyFileSpy).toHaveBeenCalledWith(
'/home/expo/.maestro/tests/android-maestro-runner-attempt-0/junit-report.xml',
'/home/expo/.maestro/tests/junit-reports/android-maestro-junit-attempt-0.xml'
);
expect(parser.mergeJUnitReports).not.toHaveBeenCalled();
expect(step.getOutputValueByName('final_report_path')).toBe(
format === 'html'
? '/home/expo/.maestro/tests/android-maestro-runner-attempt-0/report.html'
: '/home/expo/.maestro/tests/android-maestro-runner-attempt-0/allure-results'
);
expect(mockedRunnerHarvest).not.toHaveBeenCalled();
expect(mockUploadArtifact).toHaveBeenCalledTimes(1);
expect(mockUploadArtifact).toHaveBeenCalledWith(
expect.objectContaining({
artifact: expect.objectContaining({
type: GenericArtifactType.OTHER,
name:
format === 'html' ? 'Maestro Runner HTML Report' : 'Maestro Runner Allure Results',
paths: [
format === 'html'
? '/home/expo/.maestro/tests/android-maestro-runner-attempt-0'
: '/home/expo/.maestro/tests/android-maestro-runner-attempt-0/allure-results',
],
}),
})
);
}
);
it('does not expose a selected runner report before an attempt starts', async () => {
const step = createStep({
flow_path: ['flows/a.yaml'],
flow_path: [],
platform: 'android',
backend: 'maestro-runner',
output_format: 'html',
});
await expect(step.executeAsync()).rejects.toThrow(UserError);
expect(step.getOutputValueByName('final_report_path')).toBeUndefined();
expect(step.getOutputValueByName('junit_report_directory')).toBe(
'/home/expo/.maestro/tests/junit-reports'
);
});
it('rejects an unknown output_format with maestro-runner', async () => {
const step = createStep({
flow_path: ['flows/a.yaml'],
platform: 'android',
backend: 'maestro-runner',
output_format: 'csv',
});
await expect(step.executeAsync()).rejects.toThrow(
'maestro-runner only supports the "junit" output_format'
'maestro-runner supports "junit", "html", and "allure" output_format values'
);
expect(mockedSpawn).not.toHaveBeenCalled();
});
it('uploads the last runner HTML report even when the test fails', async () => {
mockedSpawn.mockRejectedValue(rejectExit1());
jest.spyOn(fs, 'copyFile').mockResolvedValue();
const step = createStep({
flow_path: ['flows/a.yaml'],
platform: 'android',
backend: 'maestro-runner',
output_format: 'html',
retries: 1,
retry_failed_only: false,
});
await expect(step.executeAsync()).rejects.toThrow(UserError);
expect(mockUploadArtifact).toHaveBeenCalledWith(
expect.objectContaining({
artifact: expect.objectContaining({
name: 'Maestro Runner HTML Report',
paths: ['/home/expo/.maestro/tests/android-maestro-runner-attempt-1'],
}),
})
);
expect(step.getOutputValueByName('final_report_path')).toBe(
'/home/expo/.maestro/tests/android-maestro-runner-attempt-1/report.html'
);
});
it('does not fail a passing test when the runner report upload fails', async () => {
mockedSpawn.mockResolvedValue(SPAWN_SUCCESS);
jest.spyOn(fs, 'copyFile').mockResolvedValue();
mockUploadArtifact.mockRejectedValue(new Error('upload failed'));
const step = createStep({
flow_path: ['flows/a.yaml'],
platform: 'android',
backend: 'maestro-runner',
output_format: 'allure',
});
await expect(step.executeAsync()).resolves.toBeUndefined();
});
it('logs that maestro-runner does not support direct DADB', async () => {
mockedSpawn.mockResolvedValue(SPAWN_SUCCESS);
jest.spyOn(fs, 'copyFile').mockResolvedValue();
@@ -966,7 +1070,7 @@ describe('createMaestroTestsBuildFunction', () => {
await expect(step.executeAsync()).rejects.toThrow(UserError);
});
it('uses $HOME/.maestro/tests output path for non-junit formats (e.g. html)', async () => {
it('uses $HOME/.maestro/tests output path and uploads the Maestro CLI HTML report', async () => {
mockedSpawn.mockResolvedValue(SPAWN_SUCCESS);
const step = createStep({
@@ -977,11 +1081,119 @@ describe('createMaestroTestsBuildFunction', () => {
await step.executeAsync();
const args = mockedSpawn.mock.calls[0][1] as string[];
// Non-JUnit uses a fixed path inside $HOME/.maestro/tests so the
// whole-directory upload picks it up.
// Non-JUnit uses a fixed path inside $HOME/.maestro/tests.
const outputArg = args.find(a => a.startsWith('--output='));
expect(outputArg).toMatch(/\.maestro\/tests\/android-maestro-html\.html$/);
expect(outputArg).not.toMatch(/junit-reports/);
expect(step.getOutputValueByName('final_report_path')).toBe(
'/home/expo/.maestro/tests/android-maestro-html.html'
);
expect(mockUploadArtifact).toHaveBeenCalledWith(
expect.objectContaining({
artifact: expect.objectContaining({
type: GenericArtifactType.OTHER,
name: 'Maestro HTML Report',
paths: ['/home/expo/.maestro/tests/android-maestro-html.html'],
}),
})
);
});
it('uploads the Maestro CLI HTML report even when tests fail', async () => {
mockedSpawn.mockRejectedValue(rejectExit1());
const step = createStep({
flow_path: ['flows/a.yaml'],
output_format: 'html',
platform: 'android',
});
await expect(step.executeAsync()).rejects.toThrow(UserError);
expect(mockUploadArtifact).toHaveBeenCalledWith(
expect.objectContaining({
artifact: expect.objectContaining({
name: 'Maestro HTML Report',
paths: ['/home/expo/.maestro/tests/android-maestro-html.html'],
}),
})
);
});
it('warns that Maestro CLI does not support Allure reports', async () => {
mockedSpawn.mockRejectedValue(rejectExit1());
const logger = createMockLogger();
jest.mocked(logger.child).mockReturnValue(logger);
const step = createStep(
{
flow_path: ['flows/a.yaml'],
output_format: 'allure',
platform: 'android',
},
{ logger }
);
await expect(step.executeAsync()).rejects.toThrow(UserError);
expect(logger.warn).toHaveBeenCalledWith(
'Maestro CLI does not support Allure reports; no Allure artifact was uploaded.'
);
expect(mockUploadArtifact).not.toHaveBeenCalled();
});
it('clears the Maestro CLI HTML report before every retry attempt', async () => {
const removeSpy = jest.spyOn(fs, 'rm').mockResolvedValue();
mockedSpawn.mockRejectedValue(rejectExit1());
const step = createStep({
flow_path: ['flows/a.yaml'],
output_format: 'html',
platform: 'android',
retries: 1,
});
await expect(step.executeAsync()).rejects.toThrow(UserError);
expect(
removeSpy.mock.calls.filter(
([target]) => target === '/home/expo/.maestro/tests/android-maestro-html.html'
)
).toHaveLength(2);
expect(removeSpy).toHaveBeenNthCalledWith(
1,
'/home/expo/.maestro/tests/android-maestro-html.html',
{
force: true,
}
);
expect(removeSpy).toHaveBeenNthCalledWith(
2,
'/home/expo/.maestro/tests/android-maestro-html.html',
{
force: true,
}
);
expect(removeSpy.mock.invocationCallOrder[0]).toBeLessThan(
mockedSpawn.mock.invocationCallOrder[0]
);
expect(removeSpy.mock.invocationCallOrder[1]).toBeLessThan(
mockedSpawn.mock.invocationCallOrder[1]
);
});
it('does not run Maestro CLI or upload old HTML if the report cannot be cleared', async () => {
jest.spyOn(fs, 'rm').mockImplementation(async target => {
if (target === '/home/expo/.maestro/tests/android-maestro-html.html') {
throw new Error('permission denied');
}
});
const step = createStep({
flow_path: ['flows/a.yaml'],
output_format: 'html',
platform: 'android',
});
await expect(step.executeAsync()).rejects.toThrow(SystemError);
expect(mockedSpawn).not.toHaveBeenCalled();
expect(mockUploadArtifact).not.toHaveBeenCalled();
});
it('uses lowercase extension for non-junit formats regardless of input casing', async () => {
@@ -1216,28 +1428,31 @@ describe('createMaestroTestsBuildFunction', () => {
);
});
it('discovers maestro-runner flow results from the attempt report directory', async () => {
mockedSpawn.mockResolvedValue(SPAWN_SUCCESS);
const shot = makeShot(0);
jest.spyOn(fs, 'copyFile').mockResolvedValue();
mockedRunnerHarvest.mockResolvedValue([shot]);
const parseFlowResultsSpy = jest.spyOn(parser, 'parseMaestroRunnerReport').mockResolvedValue({
flows: [{ name: 'Login', sourceFile: 'a.yaml', status: 'failed' }],
});
it.each(['junit', undefined])(
'uploads runner screenshots with output_format=%s',
async format => {
mockedSpawn.mockResolvedValue(SPAWN_SUCCESS);
const shot = makeShot(0);
jest.spyOn(fs, 'copyFile').mockResolvedValue();
mockedRunnerHarvest.mockResolvedValue([shot]);
const parseFlowResultsSpy = jest.spyOn(parser, 'parseMaestroRunnerReport').mockResolvedValue({
flows: [{ name: 'Login', sourceFile: 'a.yaml', status: 'failed' }],
});
const step = createStep({
flow_path: ['a.yaml'],
platform: 'ios',
output_format: 'junit',
backend: 'maestro-runner',
});
await step.executeAsync();
const step = createStep({
flow_path: ['a.yaml'],
platform: 'ios',
...(format === undefined ? {} : { output_format: format }),
backend: 'maestro-runner',
});
await step.executeAsync();
expect(parseFlowResultsSpy).toHaveBeenCalledWith(
'/home/expo/.maestro/tests/ios-maestro-runner-attempt-0'
);
expect(mockUploadArtifact).toHaveBeenCalledTimes(1);
});
expect(parseFlowResultsSpy).toHaveBeenCalledWith(
'/home/expo/.maestro/tests/ios-maestro-runner-attempt-0'
);
expect(mockUploadArtifact).toHaveBeenCalledTimes(1);
}
);
it('uploads screenshots even when all attempts fail, before throwing ERR_MAESTRO_TESTS_FAILED', async () => {
mockedSpawn.mockRejectedValue(rejectExit1());
@@ -1301,14 +1516,15 @@ describe('createMaestroTestsBuildFunction', () => {
expect(mockUploadArtifact).toHaveBeenCalledTimes(30);
});
it('does not harvest or upload screenshots when output_format is not junit', async () => {
it('does not harvest or upload screenshots for Maestro CLI HTML', async () => {
mockedSpawn.mockResolvedValue(SPAWN_SUCCESS);
const step = createStep({ flow_path: ['a.yaml'], platform: 'android', output_format: 'html' });
await step.executeAsync();
expect(mockedHarvest).not.toHaveBeenCalled();
expect(mockUploadArtifact).not.toHaveBeenCalled();
expect(mockUploadArtifact).toHaveBeenCalledTimes(1);
expect(mockUploadArtifact.mock.calls[0][0].artifact.name).toBe('Maestro HTML Report');
});
it('uploads only the final attempt for a pure-failure flow, but every attempt for a flaky flow', async () => {
@@ -7,6 +7,7 @@ import { type CustomBuildContext } from '../../../customBuildContext';
import { IosSimulatorUtils } from '../../../utils/IosSimulatorUtils';
import { isProcessDescendantOfAsync } from '../../../utils/processes';
import { turtleFetch } from '../../../utils/turtleFetch';
import { startDeviceSessionHostAsync } from '../../utils/deviceSessionHost';
import {
monitorLocalEgressAsync,
readLocalEgressHandoffAsync,
@@ -14,7 +15,6 @@ import {
stopLocalEgressResourcesAsync,
} from '../../utils/localEgress';
import {
startDeviceWebPreviewWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../../utils/remoteDeviceRunSession';
@@ -27,6 +27,7 @@ jest.mock('@expo/turtle-spawn');
jest.mock('../../../utils/IosSimulatorUtils');
jest.mock('../../../utils/processes');
jest.mock('../../../utils/turtleFetch');
jest.mock('../../utils/deviceSessionHost');
jest.mock('../../utils/agentDeviceArtifacts');
jest.mock('../../utils/argentArtifacts');
jest.mock('../../utils/agentDeviceEvents', () => ({
@@ -58,7 +59,6 @@ jest.mock('../../utils/remoteDeviceRunSession', () => ({
subdomainId: 'controller-id',
stopAsync: jest.fn(),
}),
startDeviceWebPreviewWithTunnelAsync: jest.fn(),
uploadRemoteSessionConfigAsync: jest.fn(),
waitForDeviceRunSessionStoppedAsync: jest.fn(),
}));
@@ -82,6 +82,7 @@ const controllers = [
describe.each(controllers)('%s local egress', (_name, createFunction, controllerField) => {
const stopPreview = jest.fn();
const openPreview = jest.fn();
const stopEgress = jest.fn();
let lifetime: AbortSignal;
async function run(signal?: AbortSignal): Promise<void> {
@@ -119,11 +120,15 @@ describe.each(controllers)('%s local egress', (_name, createFunction, controller
jest.mocked(turtleFetch).mockResolvedValue({ ok: true } as never);
jest.mocked(readLocalEgressHandoffAsync).mockResolvedValue(handoff);
jest.mocked(monitorLocalEgressAsync).mockResolvedValue(undefined);
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
openPreview.mockResolvedValue({
previewPageUrl: 'https://expo.dev/simulator-preview/preview-id',
apiUrl: 'https://preview.test',
previewToken: 'preview-secret',
stopAsync: stopPreview,
closeAsync: jest.fn(),
});
jest.mocked(startDeviceSessionHostAsync).mockResolvedValue({
openPreviewAsync: openPreview,
finishAsync: stopPreview,
});
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockImplementation(async () => {
@@ -169,12 +174,14 @@ describe.each(controllers)('%s local egress', (_name, createFunction, controller
expect(monitorLocalEgressAsync).not.toHaveBeenCalled();
expect(stopEgress).not.toHaveBeenCalled();
});
it.each(['startup', 'report', 'wait', 'teardown'])(
it.each(['startup', 'preview', 'report', 'wait', 'teardown'])(
'releases egress after %s fails',
async phase => {
const error = new Error(`${phase} failed`);
if (phase === 'startup') {
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockRejectedValue(error);
jest.mocked(startDeviceSessionHostAsync).mockRejectedValue(error);
} else if (phase === 'preview') {
openPreview.mockRejectedValue(error);
} else if (phase === 'report') {
jest.mocked(uploadRemoteSessionConfigAsync).mockRejectedValue(error);
} else if (phase === 'wait') {
@@ -186,7 +193,7 @@ describe.each(controllers)('%s local egress', (_name, createFunction, controller
expect(lifetime.aborted).toBe(true);
expect(stopEgress).toHaveBeenCalledTimes(1);
expect(monitorLocalEgressAsync).toHaveBeenCalledTimes(
phase === 'startup' || phase === 'report' ? 0 : 1
phase === 'startup' || phase === 'preview' || phase === 'report' ? 0 : 1
);
}
);
@@ -0,0 +1,70 @@
import { type bunyan } from '@expo/logger';
import { type BuildStepContext, type BuildStepEnv } from '@expo/steps';
import { type CustomBuildContext } from '../../../customBuildContext';
import { Sentry } from '../../../sentry';
import { getDeviceRunSessionIdOrThrow } from '../../utils/remoteDeviceRunSession';
import { uploadServeSimLogsFileAsync } from '../../utils/serveSimLogsArtifacts';
import { ServeSimLogsRecorder } from '../../utils/serveSimLogsRecorder';
import { createCollectServeSimLogsBuildFunction } from '../collectServeSimLogs';
import { createStartServeSimLogsBuildFunction } from '../startServeSimLogs';
jest.mock('../../../sentry');
jest.mock('../../utils/serveSimLogsRecorder');
jest.mock('../../utils/serveSimLogsArtifacts');
jest.mock('../../utils/remoteDeviceRunSession');
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const ctx = {} as CustomBuildContext;
const step = { logger } as BuildStepContext;
const args = { inputs: {}, outputs: {}, env: {} as BuildStepEnv };
beforeEach(() => {
jest.mocked(getDeviceRunSessionIdOrThrow).mockReturnValue('session-id');
jest.mocked(ServeSimLogsRecorder.finishAsync).mockResolvedValue([]);
});
it('starts collection and tolerates startup failures', async () => {
jest.mocked(ServeSimLogsRecorder.startAsync).mockRejectedValueOnce(new Error('disk failure'));
await expect(createStartServeSimLogsBuildFunction().fn?.(step, args)).resolves.toBeUndefined();
expect(logger.warn).toHaveBeenCalled();
expect(Sentry.capture).toHaveBeenCalledWith(
'Could not start serve-sim simulator logs',
expect.any(Error)
);
});
it('uploads each device with its own bounded signal', async () => {
jest.mocked(ServeSimLogsRecorder.finishAsync).mockResolvedValue([
{ udid: 'A', filePath: '/tmp/A.ndjson' },
{ udid: 'B', filePath: '/tmp/B.ndjson' },
]);
await createCollectServeSimLogsBuildFunction(ctx).fn?.(step, args);
expect(uploadServeSimLogsFileAsync).toHaveBeenCalledTimes(2);
expect(uploadServeSimLogsFileAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({
deviceRunSessionId: 'session-id',
udid: 'A',
filePath: '/tmp/A.ndjson',
signal: expect.any(AbortSignal),
})
);
expect(jest.mocked(uploadServeSimLogsFileAsync).mock.calls[0][1].signal).not.toBe(
jest.mocked(uploadServeSimLogsFileAsync).mock.calls[1][1].signal
);
});
it('skips empty collection and warns on finalization failure', async () => {
await createCollectServeSimLogsBuildFunction(ctx).fn?.(step, args);
expect(uploadServeSimLogsFileAsync).not.toHaveBeenCalled();
jest.mocked(ServeSimLogsRecorder.finishAsync).mockRejectedValueOnce(new Error('failure'));
await expect(
createCollectServeSimLogsBuildFunction(ctx).fn?.(step, args)
).resolves.toBeUndefined();
expect(logger.warn).toHaveBeenCalled();
expect(Sentry.capture).toHaveBeenCalledWith(
'Could not finalize serve-sim simulator logs',
expect.any(Error)
);
});
@@ -8,13 +8,13 @@ import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { type CustomBuildContext } from '../../../customBuildContext';
import { pollAgentDeviceArtifactsForUploadAsync } from '../../utils/agentDeviceArtifacts';
import { startAgentDeviceEventCollectionAsync } from '../../utils/agentDeviceEvents';
import { startDeviceSessionHostAsync } from '../../utils/deviceSessionHost';
import {
getDeviceRunSessionIdOrThrow,
getNgrokAuthtokenOrThrow,
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
@@ -40,6 +40,7 @@ jest.mock('../../utils/agentDeviceArtifacts', () => ({
jest.mock('../../utils/agentDeviceEvents', () => ({
startAgentDeviceEventCollectionAsync: jest.fn(),
}));
jest.mock('../../utils/deviceSessionHost');
jest.mock('../../utils/remoteDeviceRunSession', () => ({
...jest.requireActual('../../utils/remoteDeviceRunSession'),
getDeviceRunSessionIdOrThrow: jest.fn(),
@@ -47,7 +48,6 @@ jest.mock('../../utils/remoteDeviceRunSession', () => ({
getNgrokTunnelDomainOrThrow: jest.fn(),
selectXcodeDeveloperDirectoryAsync: jest.fn(),
spawnDetached: jest.fn(),
startDeviceWebPreviewWithTunnelAsync: jest.fn(),
startNgrokTunnelAsync: jest.fn(),
uploadRemoteSessionConfigAsync: jest.fn(),
waitForDeviceRunSessionStoppedAsync: jest.fn(),
@@ -104,6 +104,7 @@ describe('createStartAgentDeviceRemoteSessionBuildFunction orchestration', () =>
jest.mocked(spawnDetached).mockReturnValue({
pid: 4242,
getOutput: () => '',
getExitError: () => undefined,
stopAsync: mockDaemonStopAsync,
});
jest.mocked(waitForFileAsync).mockResolvedValue({ port: 5678, token: 'daemon-token' });
@@ -112,10 +113,13 @@ describe('createStartAgentDeviceRemoteSessionBuildFunction orchestration', () =>
subdomainId: 'agent-device-abc',
stopAsync: mockTunnelStopAsync,
});
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
previewPageUrl: 'https://expo.dev/simulator-preview/preview-id',
apiUrl: 'https://web-preview.tunnel.example.com',
stopAsync: mockPreviewStopAsync,
jest.mocked(startDeviceSessionHostAsync).mockResolvedValue({
openPreviewAsync: jest.fn().mockResolvedValue({
previewPageUrl: 'https://expo.dev/simulator-preview/preview-id',
apiUrl: 'https://web-preview.tunnel.example.com',
closeAsync: jest.fn(),
}),
finishAsync: mockPreviewStopAsync,
});
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockResolvedValue(undefined);
@@ -134,7 +138,7 @@ describe('createStartAgentDeviceRemoteSessionBuildFunction orchestration', () =>
await runAsync(logger, BuildRuntimePlatform.LINUX);
expect(selectXcodeDeveloperDirectoryAsync).not.toHaveBeenCalled();
expect(startDeviceWebPreviewWithTunnelAsync).toHaveBeenCalledWith(
expect(startDeviceSessionHostAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({ runtimePlatform: BuildRuntimePlatform.LINUX })
);
@@ -163,7 +167,7 @@ describe('createStartAgentDeviceRemoteSessionBuildFunction orchestration', () =>
open_url: { value: 'exp://127.0.0.1:8081' },
});
expect(startDeviceWebPreviewWithTunnelAsync).toHaveBeenCalledWith(
expect(startDeviceSessionHostAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({
runtimePlatform: BuildRuntimePlatform.DARWIN,
@@ -75,6 +75,7 @@ describe(startAgentDeviceDaemonAsync, () => {
jest.mocked(spawnDetached).mockReturnValue({
pid: 1234,
getOutput: () => '',
getExitError: () => undefined,
stopAsync,
});
jest.mocked(spawn).mockImplementation((async (
@@ -108,7 +109,12 @@ describe(startAgentDeviceDaemonAsync, () => {
expect(spawnDetached).toHaveBeenCalledWith({
command: 'node',
args: [path.join(addCwd, 'node_modules/agent-device/dist/src/internal/daemon.js')],
env: expect.objectContaining({ AGENT_DEVICE_DAEMON_SERVER_MODE: 'http' }),
env: expect.objectContaining({
AGENT_DEVICE_DAEMON_SERVER_MODE: 'http',
AGENT_DEVICE_DAEMON_IDLE_TIMEOUT_MS: '0',
AGENT_DEVICE_IOS_RUNNER_IDLE_STOP_MS: '0',
AGENT_DEVICE_SESSION_IDLE_TIMEOUT_MS: '0',
}),
});
await handle.stopAsync();
@@ -165,7 +171,12 @@ describe(startAgentDeviceDaemonAsync, () => {
command: 'bun',
args: ['run', 'src/daemon.ts'],
cwd: '/tmp/agent-device-src',
env: expect.objectContaining({ AGENT_DEVICE_DAEMON_SERVER_MODE: 'http' }),
env: expect.objectContaining({
AGENT_DEVICE_DAEMON_SERVER_MODE: 'http',
AGENT_DEVICE_DAEMON_IDLE_TIMEOUT_MS: '0',
AGENT_DEVICE_IOS_RUNNER_IDLE_STOP_MS: '0',
AGENT_DEVICE_SESSION_IDLE_TIMEOUT_MS: '0',
}),
});
expect(Sentry.capture).toHaveBeenCalledWith(
'Failed to start agent-device daemon from the configured package manager; falling back to git clone',
@@ -6,6 +6,7 @@ import path from 'node:path';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { type CustomBuildContext } from '../../../customBuildContext';
import { startDeviceSessionHostAsync } from '../../utils/deviceSessionHost';
import { isProcessDescendantOfAsync } from '../../../utils/processes';
import { pollArgentArtifactsForUploadAsync } from '../../utils/argentArtifacts';
import { startArgentEventCollectionAsync } from '../../utils/argentEvents';
@@ -16,7 +17,6 @@ import {
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
@@ -42,15 +42,17 @@ jest.mock('../../utils/argentEvents', () => ({
...jest.requireActual('../../utils/argentEvents'),
startArgentEventCollectionAsync: jest.fn(),
}));
jest.mock('../../utils/deviceSessionHost');
jest.mock('../../utils/remoteDeviceRunSession', () => ({
...jest.requireActual('../../utils/remoteDeviceRunSession'),
ensureFfmpegInstalledOnceAsync: jest.fn(),
finishRemoteSessionAsync: jest.requireActual('../../utils/remoteDeviceRunSession')
.finishRemoteSessionAsync,
getDeviceRunSessionIdOrThrow: jest.fn(),
getNgrokAuthtokenOrThrow: jest.fn(),
getNgrokTunnelDomainOrThrow: jest.fn(),
selectXcodeDeveloperDirectoryAsync: jest.fn(),
spawnDetached: jest.fn(),
startDeviceWebPreviewWithTunnelAsync: jest.fn(),
startNgrokTunnelAsync: jest.fn(),
uploadRemoteSessionConfigAsync: jest.fn(),
waitForDeviceRunSessionStoppedAsync: jest.fn(),
@@ -86,6 +88,7 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
jest.mocked(spawnDetached).mockReturnValue({
pid: 4242,
getOutput: () => '',
getExitError: () => undefined,
stopAsync: jest.fn(),
});
jest.mocked(startNgrokTunnelAsync).mockResolvedValue({
@@ -93,10 +96,13 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
subdomainId: 'argent-abc',
stopAsync: mockTunnelStopAsync,
});
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
previewPageUrl: 'https://expo.dev/simulator-preview/preview-id',
apiUrl: 'https://web-preview.tunnel.example.com',
stopAsync: mockPreviewStopAsync,
jest.mocked(startDeviceSessionHostAsync).mockResolvedValue({
openPreviewAsync: jest.fn().mockResolvedValue({
previewPageUrl: 'https://expo.dev/simulator-preview/preview-id',
apiUrl: 'https://web-preview.tunnel.example.com',
closeAsync: jest.fn(),
}),
finishAsync: mockPreviewStopAsync,
});
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockResolvedValue(undefined);
@@ -113,6 +119,75 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
await fs.promises.rm(TEST_HOME, { recursive: true, force: true });
});
it('reports an early exit with output before opening tunnels or publishing readiness', async () => {
jest.mocked(spawnDetached).mockReturnValue({
pid: 4242,
getOutput: () => 'could not bind server port',
getExitError: () => new Error('process exited with code 1'),
stopAsync: jest.fn(),
});
const buildFunction = createStartArgentRemoteSessionBuildFunction({} as CustomBuildContext);
await expect(
buildFunction.fn!(
{
logger: { info: jest.fn(), warn: jest.fn() },
global: { runtimePlatform: BuildRuntimePlatform.LINUX },
} as unknown as BuildStepContext,
{
inputs: {
package_version: { value: undefined },
max_idle_time_minutes: { value: undefined },
},
outputs: {},
env: {},
} as never
)
).rejects.toThrow(
'Argent exited before becoming ready: process exited with code 1\nArgent tool-server output:\ncould not bind server port'
);
expect(startNgrokTunnelAsync).not.toHaveBeenCalled();
expect(uploadRemoteSessionConfigAsync).not.toHaveBeenCalled();
expect(startArgentEventCollectionAsync).not.toHaveBeenCalled();
});
it.each(['preview', 'config', 'wait'])(
'finishes the host and tools after %s fails',
async phase => {
const error = new Error(`${phase} failed`);
if (phase === 'preview') {
jest.mocked(startDeviceSessionHostAsync).mockResolvedValueOnce({
openPreviewAsync: jest.fn().mockRejectedValue(error),
finishAsync: mockPreviewStopAsync,
});
} else if (phase === 'config') {
jest.mocked(uploadRemoteSessionConfigAsync).mockRejectedValueOnce(error);
} else {
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockRejectedValueOnce(error);
}
const buildFunction = createStartArgentRemoteSessionBuildFunction({} as CustomBuildContext);
await expect(
buildFunction.fn!(
{
logger: { info: jest.fn(), warn: jest.fn() },
global: { runtimePlatform: BuildRuntimePlatform.LINUX },
} as unknown as BuildStepContext,
{
inputs: {
package_version: { value: undefined },
max_idle_time_minutes: { value: undefined },
},
outputs: {},
env: {},
} as never
)
).rejects.toBe(error);
expect(mockPreviewStopAsync).toHaveBeenCalledTimes(1);
expect(mockTunnelStopAsync).toHaveBeenCalledTimes(1);
expect(mockStopAsync).toHaveBeenCalledTimes(1);
expect(jest.mocked(spawnDetached).mock.results[0].value.stopAsync).toHaveBeenCalledTimes(1);
}
);
it('enables the event log flag, shares one path, and starts/stops the collector', async () => {
const ctx = {} as unknown as CustomBuildContext;
const buildFunction = createStartArgentRemoteSessionBuildFunction(ctx);
@@ -177,7 +252,7 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
jest.mocked(waitForDeviceRunSessionStoppedAsync).mock.invocationCallOrder[0]
);
expect(mockTunnelStopAsync).toHaveBeenCalledTimes(1);
expect(startDeviceWebPreviewWithTunnelAsync).toHaveBeenCalledWith(
expect(startDeviceSessionHostAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({ runtimePlatform: BuildRuntimePlatform.LINUX })
);
@@ -216,7 +291,7 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
} as never
);
expect(startDeviceWebPreviewWithTunnelAsync).toHaveBeenCalledWith(
expect(startDeviceSessionHostAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({
runtimePlatform: BuildRuntimePlatform.DARWIN,
@@ -266,4 +341,49 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
buildFunction.inputProviders?.map(provider => provider(globalCtx, 'Test step').id)
).toEqual(expect.arrayContaining(['launch_app_identifier', 'launch_args', 'open_url']));
});
it('stops automation without waiting for recording upload', async () => {
let release!: () => void;
let stopped!: () => void;
const pendingFinish = new Promise<void>(resolve => {
release = resolve;
});
const toolStopped = new Promise<void>(resolve => {
stopped = resolve;
});
mockPreviewStopAsync.mockReturnValueOnce(pendingFinish);
const stopServer = jest.fn(async () => {
stopped();
});
jest.mocked(spawnDetached).mockReturnValueOnce({
pid: 4242,
getOutput: () => '',
getExitError: () => undefined,
stopAsync: stopServer,
});
const buildFunction = createStartArgentRemoteSessionBuildFunction({} as CustomBuildContext);
const running = buildFunction.fn!(
{
logger: { info: jest.fn(), warn: jest.fn() },
global: { runtimePlatform: BuildRuntimePlatform.LINUX },
} as unknown as BuildStepContext,
{
inputs: {
package_version: { value: undefined },
max_idle_time_minutes: { value: undefined },
},
outputs: {},
env: {},
} as never
);
try {
await toolStopped;
expect(mockTunnelStopAsync).toHaveBeenCalledTimes(1);
expect(mockPreviewStopAsync).toHaveBeenCalledTimes(1);
expect(stopServer).toHaveBeenCalledTimes(1);
} finally {
release();
await running;
}
});
});
@@ -94,6 +94,59 @@ describe(waitForArgentToolServerStateAsync, () => {
return JSON.stringify({ port, pid, token });
}
it('reports process exit instead of waiting for the startup timeout', async () => {
const getExitError = jest.fn(() => new Error('server exited with code 1'));
await expect(
waitForArgentToolServerStateAsync({
stateDir,
ancestorPid: process.pid,
timeoutMs: 100,
pollIntervalMs: 1,
getExitError,
})
).rejects.toThrow('server exited with code 1');
expect(getExitError).toHaveBeenCalledTimes(1);
});
it('detects an exit after polling has begun', async () => {
const getExitError = jest
.fn<Error | undefined, []>()
.mockReturnValueOnce(undefined)
.mockReturnValueOnce(undefined)
.mockReturnValue(new Error('server stopped'));
await expect(
waitForArgentToolServerStateAsync({
stateDir,
ancestorPid: process.pid,
timeoutMs: 100,
pollIntervalMs: 1,
getExitError,
})
).rejects.toThrow('server stopped');
expect(getExitError).toHaveBeenCalledTimes(3);
});
it('rejects an exit observed while reading a matching state file', async () => {
await fs.promises.writeFile(
path.join(stateDir, 'tool-server.json'),
stateJson({ port: 4321, pid: process.pid })
);
const getExitError = jest
.fn<Error | undefined, []>()
.mockReturnValueOnce(undefined)
.mockReturnValue(new Error('server exited during startup'));
await expect(
waitForArgentToolServerStateAsync({
stateDir,
ancestorPid: process.ppid,
timeoutMs: 100,
pollIntervalMs: 1,
getExitError,
})
).rejects.toThrow('server exited during startup');
expect(getExitError).toHaveBeenCalledTimes(2);
});
it.each(['tool-server.json', 'tool-server-012345abcdef.json'])(
'reads a matching process from %s',
async fileName => {
@@ -31,7 +31,12 @@ jest.mock('../../utils/remoteDeviceRunSession', () => ({
}));
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const server = { pid: 12345, getOutput: () => '', stopAsync: jest.fn() };
const server = {
pid: 12345,
getOutput: () => '',
getExitError: () => undefined,
stopAsync: jest.fn(),
};
const tunnel = {
url: 'https://egress.example.com',
subdomainId: 'egress-id',
@@ -3,22 +3,22 @@ import { BuildRuntimePlatform, type BuildStepContext, type BuildStepEnv } from '
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { type CustomBuildContext } from '../../../customBuildContext';
import { startDeviceSessionHostAsync } from '../../utils/deviceSessionHost';
import {
getDeviceRunSessionIdOrThrow,
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
startDeviceWebPreviewWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../../utils/remoteDeviceRunSession';
import { createStartWebPreviewRemoteSessionBuildFunction } from '../startWebPreviewRemoteSession';
jest.mock('../../utils/deviceSessionHost');
jest.mock('../../utils/remoteDeviceRunSession', () => ({
...jest.requireActual('../../utils/remoteDeviceRunSession'),
getDeviceRunSessionIdOrThrow: jest.fn(),
getNgrokTunnelDomainOrThrow: jest.fn(),
selectXcodeDeveloperDirectoryAsync: jest.fn(),
startDeviceWebPreviewWithTunnelAsync: jest.fn(),
uploadRemoteSessionConfigAsync: jest.fn(),
waitForDeviceRunSessionStoppedAsync: jest.fn(),
}));
@@ -56,10 +56,13 @@ describe(createStartWebPreviewRemoteSessionBuildFunction, () => {
jest.mocked(getDeviceRunSessionIdOrThrow).mockReturnValue('device-run-session-id');
jest.mocked(getNgrokTunnelDomainOrThrow).mockReturnValue('tunnel.example.com');
jest.mocked(selectXcodeDeveloperDirectoryAsync).mockResolvedValue(undefined);
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
previewPageUrl: 'https://expo.dev/simulator-preview/preview-id',
apiUrl: 'https://web-preview.example.test',
stopAsync,
jest.mocked(startDeviceSessionHostAsync).mockResolvedValue({
openPreviewAsync: jest.fn().mockResolvedValue({
previewPageUrl: 'https://expo.dev/simulator-preview/preview-id',
apiUrl: 'https://web-preview.example.test',
closeAsync: jest.fn(),
}),
finishAsync: stopAsync,
});
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockResolvedValue(undefined);
@@ -67,11 +70,14 @@ describe(createStartWebPreviewRemoteSessionBuildFunction, () => {
});
it('reports the session token when serve-sim minted one', async () => {
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
previewPageUrl: 'https://expo.dev/simulator-preview/preview-id',
apiUrl: 'https://web-preview.example.test',
previewToken: 'tok-1',
stopAsync,
jest.mocked(startDeviceSessionHostAsync).mockResolvedValue({
openPreviewAsync: jest.fn().mockResolvedValue({
previewPageUrl: 'https://expo.dev/simulator-preview/preview-id',
apiUrl: 'https://web-preview.example.test',
previewToken: 'tok-1',
closeAsync: jest.fn(),
}),
finishAsync: stopAsync,
});
await runAsync(BuildRuntimePlatform.DARWIN);
@@ -87,6 +93,22 @@ describe(createStartWebPreviewRemoteSessionBuildFunction, () => {
);
});
it.each(['preview', 'config', 'wait'])('finishes the session after %s fails', async phase => {
const error = new Error(`${phase} failed`);
if (phase === 'preview') {
jest.mocked(startDeviceSessionHostAsync).mockResolvedValueOnce({
openPreviewAsync: jest.fn().mockRejectedValue(error),
finishAsync: stopAsync,
});
} else if (phase === 'config') {
jest.mocked(uploadRemoteSessionConfigAsync).mockRejectedValueOnce(error);
} else {
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockRejectedValueOnce(error);
}
await expect(runAsync(BuildRuntimePlatform.LINUX)).rejects.toBe(error);
expect(stopAsync).toHaveBeenCalledTimes(1);
});
it.each([
[BuildRuntimePlatform.DARWIN, true],
[BuildRuntimePlatform.LINUX, false],
@@ -94,9 +116,8 @@ describe(createStartWebPreviewRemoteSessionBuildFunction, () => {
await runAsync(runtimePlatform);
expect(selectXcodeDeveloperDirectoryAsync).toHaveBeenCalledTimes(selectsXcode ? 1 : 0);
expect(startDeviceWebPreviewWithTunnelAsync).toHaveBeenCalledWith(ctx, {
expect(startDeviceSessionHostAsync).toHaveBeenCalledWith(ctx, {
runtimePlatform,
baseDomain: 'tunnel.example.com',
env,
logger,
timeoutMs: 60_000,
@@ -139,14 +160,14 @@ describe(createStartWebPreviewRemoteSessionBuildFunction, () => {
]);
});
it('hands the launch inputs to the web preview and announces them', async () => {
it('hands the launch inputs to the session host and announces them', async () => {
await runAsync(BuildRuntimePlatform.DARWIN, {
launch_app_identifier: { value: 'host.exp.Exponent' },
launch_args: { value: ['-EXDevMenuIsOnboardingFinished', '1'] },
open_url: { value: 'exp://127.0.0.1:8081' },
});
expect(startDeviceWebPreviewWithTunnelAsync).toHaveBeenCalledWith(
expect(startDeviceSessionHostAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({
launchAppIdentifier: 'host.exp.Exponent',
@@ -166,6 +187,6 @@ describe(createStartWebPreviewRemoteSessionBuildFunction, () => {
launch_app_identifier: { value: 'host.exp.Exponent' },
})
).rejects.toThrow('runs on linux');
expect(startDeviceWebPreviewWithTunnelAsync).not.toHaveBeenCalled();
expect(startDeviceSessionHostAsync).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,293 @@
import nock from 'nock';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { createMockLogger } from '../../../__tests__/utils/logger';
import { AscApiClient } from '../../utils/ios/AscApiClient';
import {
createUpdateTestFlightMetadataBuildFunction,
updateTestFlightMetadataAsync,
} from '../updateTestFlightMetadata';
jest.unmock('node-fetch');
const client = new AscApiClient({ token: 'test-token' });
const changelog = 'Test "quotes"\n$(not-a-command)';
const options = {
client,
buildUploadId: 'upload',
changelog,
groups: [] as string[],
logger: createMockLogger(),
};
const api = () => nock('https://api.appstoreconnect.apple.com');
function mockBuild(state = 'COMPLETE', buildId: string | null = 'build'): void {
api()
.get('/v1/buildUploads/upload')
.query(true)
.reply(200, {
data: {
type: 'buildUploads',
id: 'upload',
attributes: { state: { state } },
relationships: { build: { data: buildId ? { type: 'builds', id: buildId } : null } },
},
});
if (state === 'COMPLETE' && buildId) {
api()
.get(`/v1/builds/${buildId}/app`)
.query(true)
.reply(200, {
data: { id: 'app', attributes: { primaryLocale: 'en-US' } },
});
}
}
beforeAll(() => nock.disableNetConnect());
afterAll(() => nock.enableNetConnect());
afterEach(() => {
expect(nock.pendingMocks()).toEqual([]);
nock.cleanAll();
});
it('accepts group arrays and preserves changelog input without shell parsing', async () => {
const fn = createUpdateTestFlightMetadataBuildFunction();
const spy = jest.spyOn(fn, 'fn').mockImplementation(async (_ctx, { inputs }) => {
expect(inputs.groups.value).toEqual(['A, "B"']);
expect(inputs.changelog.value).toBe(changelog);
});
const step = fn.createBuildStepFromFunctionCall(createGlobalContextMock(), {
callInputs: {
asc_api_key_path: '/tmp/key.json',
build_upload_id: 'upload',
groups: ['A, "B"'],
changelog,
},
});
await step.executeAsync();
expect(spy).toHaveBeenCalled();
});
it('uses empty metadata defaults', async () => {
const fn = createUpdateTestFlightMetadataBuildFunction();
const spy = jest.spyOn(fn, 'fn').mockImplementation(async (_ctx, { inputs }) => {
expect(inputs.groups.value).toEqual([]);
expect(inputs.changelog.value).toBe('');
});
const step = fn.createBuildStepFromFunctionCall(createGlobalContextMock(), {
callInputs: { asc_api_key_path: '/tmp/key.json', build_upload_id: 'upload' },
});
await step.executeAsync();
expect(spy).toHaveBeenCalled();
});
it('creates the primary localization when none exist', async () => {
mockBuild();
api().get('/v1/builds/build/betaBuildLocalizations').query(true).reply(200, { data: [] });
api()
.post('/v1/betaBuildLocalizations', {
data: {
type: 'betaBuildLocalizations',
attributes: { locale: 'en-US', whatsNew: changelog },
relationships: { build: { data: { type: 'builds', id: 'build' } } },
},
})
.reply(201, { data: { id: 'locale' } });
await updateTestFlightMetadataAsync(options);
});
it('updates all existing localizations without replacing them', async () => {
mockBuild();
api()
.get('/v1/builds/build/betaBuildLocalizations')
.query(true)
.reply(200, {
data: ['en-US', 'pl'].map(locale => ({ id: locale, attributes: { locale } })),
});
for (const id of ['en-US', 'pl']) {
api()
.patch(`/v1/betaBuildLocalizations/${id}`, {
data: { type: 'betaBuildLocalizations', id, attributes: { whatsNew: changelog } },
})
.reply(200, { data: { id } });
}
await updateTestFlightMetadataAsync(options);
});
it('stops before changing metadata when Apple omits a localization locale', async () => {
mockBuild();
api()
.get('/v1/builds/build/betaBuildLocalizations')
.query(true)
.reply(200, { data: [{ id: 'localization' }] });
await expect(updateTestFlightMetadataAsync(options)).rejects.toThrow(
'App Store Connect did not return a TestFlight localization locale.'
);
});
it('adds groups without changing changelog or submitting beta review', async () => {
mockBuild();
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, { data: [{ id: 'group', attributes: { name: 'A, "B"' } }] });
api()
.post('/v1/builds/build/relationships/betaGroups', {
data: [{ type: 'betaGroups', id: 'group' }],
})
.reply(204);
await updateTestFlightMetadataAsync({ ...options, changelog: '', groups: ['A, "B"', 'A, "B"'] });
});
it('fails before changing metadata when no requested group exists', async () => {
mockBuild();
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, { data: [] });
await expect(updateTestFlightMetadataAsync({ ...options, groups: ['missing'] })).rejects.toThrow(
'The following TestFlight group was not found in App Store Connect: "missing".'
);
});
it('reads all pages and adds every group with a requested name', async () => {
mockBuild();
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, {
data: [{ id: 'first', attributes: { name: 'A' } }],
links: {
next: 'https://api.appstoreconnect.apple.com/v1/betaGroups?filter%5Bapp%5D=app&limit=200&cursor=page-2',
},
});
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200', cursor: 'page-2' })
.reply(200, {
data: [
{ id: 'second', attributes: { name: 'A' } },
{ id: 'other', attributes: { name: 'B' } },
],
});
api()
.post('/v1/builds/build/relationships/betaGroups', {
data: [
{ type: 'betaGroups', id: 'first' },
{ type: 'betaGroups', id: 'second' },
],
})
.reply(204);
await updateTestFlightMetadataAsync({ ...options, changelog: '', groups: ['A'] });
});
it('fails without adding found groups when another requested name is missing', async () => {
mockBuild();
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, { data: [{ id: 'group', attributes: { name: 'A' } }] });
await expect(
updateTestFlightMetadataAsync({ ...options, changelog: '', groups: ['A', 'B', 'C'] })
).rejects.toThrow(
'The following TestFlight groups were not found in App Store Connect: "B", "C".'
);
});
it('explains when Apple rejects an internal group with automatic distribution', async () => {
mockBuild();
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, { data: [{ id: 'group', attributes: { name: 'A' } }] });
api()
.post('/v1/builds/build/relationships/betaGroups')
.reply(422, {
errors: [
{
code: 'ENTITY_UNPROCESSABLE',
title: 'Builds cannot be assigned to this internal group.',
detail: 'Cannot add internal group to a build.',
},
],
});
await expect(
updateTestFlightMetadataAsync({ ...options, changelog: '', groups: ['A'] })
).rejects.toThrow(
"App Store Connect can't add this build to a requested internal TestFlight group."
);
});
it('stops after 20 group pages', async () => {
mockBuild();
const next =
'https://api.appstoreconnect.apple.com/v1/betaGroups?filter%5Bapp%5D=app&limit=200&cursor=next';
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, { data: [], links: { next } });
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200', cursor: 'next' })
.times(19)
.reply(200, { data: [], links: { next } });
await expect(
updateTestFlightMetadataAsync({ ...options, changelog: '', groups: ['A'] })
).rejects.toThrow('more than 20 pages');
});
it('tries group assignment if the changelog update fails', async () => {
mockBuild();
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, { data: [{ id: 'group', attributes: { name: 'A' } }] });
api().get('/v1/builds/build/betaBuildLocalizations').query(true).reply(200, { data: [] });
api()
.post('/v1/betaBuildLocalizations')
.reply(403, { errors: [{ code: 'FORBIDDEN', detail: 'Not allowed' }] });
api()
.post('/v1/builds/build/relationships/betaGroups', {
data: [{ type: 'betaGroups', id: 'group' }],
})
.reply(204);
await expect(updateTestFlightMetadataAsync({ ...options, groups: ['A'] })).rejects.toThrow('403');
});
it('reports both write failures', async () => {
mockBuild();
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, { data: [{ id: 'group', attributes: { name: 'A' } }] });
api().get('/v1/builds/build/betaBuildLocalizations').query(true).reply(200, { data: [] });
api()
.post('/v1/betaBuildLocalizations')
.reply(403, { errors: [{ code: 'CHANGELOG_FAILED' }] });
api()
.post('/v1/builds/build/relationships/betaGroups')
.reply(403, { errors: [{ code: 'GROUPS_FAILED' }] });
await expect(updateTestFlightMetadataAsync({ ...options, groups: ['A'] })).rejects.toThrow(
/CHANGELOG_FAILED.*GROUPS_FAILED/
);
});
it.each(['PROCESSING', 'FAILED'])('rejects a %s upload', async state => {
mockBuild(state);
await expect(updateTestFlightMetadataAsync(options)).rejects.toThrow('not ready');
});
it('rejects a completed upload without its build relationship', async () => {
mockBuild('COMPLETE', null);
await expect(updateTestFlightMetadataAsync(options)).rejects.toThrow('not ready');
});
it('propagates Apple write failures', async () => {
mockBuild();
api().get('/v1/builds/build/betaBuildLocalizations').query(true).reply(200, { data: [] });
api()
.post('/v1/betaBuildLocalizations')
.reply(403, { errors: [{ code: 'FORBIDDEN', detail: 'Not allowed' }] });
await expect(updateTestFlightMetadataAsync(options)).rejects.toThrow('403');
});
@@ -29,7 +29,20 @@ describe(createUploadDeviceRunSessionScreenRecordingsBuildFunction, () => {
});
});
it('uploads simulator device metadata', async () => {
it.each([
{
platform: BuildRuntimePlatform.DARWIN,
udid: SIMULATOR_UDID,
name: 'iPhone 16',
runtime: 'iOS 18.6',
},
{
platform: BuildRuntimePlatform.LINUX,
udid: 'emulator-5554',
name: 'Pixel',
runtime: 'Android 16',
},
])('uploads device metadata on $platform', async ({ platform, udid, name, runtime }) => {
const recordingDirectory = await fs.mkdtemp(path.join(os.tmpdir(), 'ios-recording-test-'));
const recordingPath = path.join(recordingDirectory, 'recording.mp4');
await fs.writeFile(recordingPath, 'recording');
@@ -45,7 +58,7 @@ describe(createUploadDeviceRunSessionScreenRecordingsBuildFunction, () => {
try {
const globalContext = createGlobalContextMock({
runtimePlatform: BuildRuntimePlatform.DARWIN,
runtimePlatform: platform,
});
const buildStep = createUploadDeviceRunSessionScreenRecordingsBuildFunction(
{} as CustomBuildContext
@@ -54,9 +67,9 @@ describe(createUploadDeviceRunSessionScreenRecordingsBuildFunction, () => {
callInputs: {
recordings_json: [
{
udid: SIMULATOR_UDID,
deviceName: 'iPhone 16',
runtimeDisplayName: 'iOS 18.6',
udid,
deviceName: name,
runtimeDisplayName: runtime,
directory: recordingDirectory,
},
],
@@ -68,13 +81,13 @@ describe(createUploadDeviceRunSessionScreenRecordingsBuildFunction, () => {
expect(uploadDeviceRunSessionArtifactAsync).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({
name: 'iPhone 16 screen recording (01234567-…, started at Jul 10, 2026, 10:00:00.000 UTC)',
name: `${name} screen recording (${udid.slice(0, 8)}-…, started at Jul 10, 2026, 10:00:00.000 UTC)`,
metadata: {
__eas_type: 'screen-recording',
__eas_screen_recording: '1',
udid: SIMULATOR_UDID,
deviceName: 'iPhone 16',
runtimeDisplayName: 'iOS 18.6',
udid,
deviceName: name,
runtimeDisplayName: runtime,
firstFrameAt: '2026-07-10T10:00:00.000Z',
width: 1179,
height: 2556,
@@ -0,0 +1,46 @@
import { BuildFunction, BuildRuntimePlatform } from '@expo/steps';
import { type CustomBuildContext } from '../../customBuildContext';
import { Sentry } from '../../sentry';
import { getDeviceRunSessionIdOrThrow } from '../utils/remoteDeviceRunSession';
import { uploadServeSimLogsFileAsync } from '../utils/serveSimLogsArtifacts';
import { ServeSimLogsRecorder } from '../utils/serveSimLogsRecorder';
const UPLOAD_TIMEOUT_MS = 30_000;
export function createCollectServeSimLogsBuildFunction(ctx: CustomBuildContext): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'collect_serve_sim_logs',
name: 'Collect serve-sim simulator logs',
__metricsId: 'eas/collect_serve_sim_logs',
supportedRuntimePlatforms: [BuildRuntimePlatform.DARWIN],
fn: async ({ logger }, { env }) => {
try {
const collected = await ServeSimLogsRecorder.finishAsync({ logger });
if (collected.length === 0) {
logger.info('No simulator logs collected; skipping upload.');
return;
}
const deviceRunSessionId = getDeviceRunSessionIdOrThrow(env);
for (const { udid, filePath } of collected) {
// Each device gets its own budget so one stuck upload does not cancel the rest.
await uploadServeSimLogsFileAsync(ctx, {
deviceRunSessionId,
udid,
filePath,
logger,
signal: AbortSignal.timeout(UPLOAD_TIMEOUT_MS),
});
}
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not finalize serve-sim simulator logs', error);
logger.warn(
{ err: error },
'Could not finalize simulator logs; the session result is unchanged.'
);
}
},
});
}
@@ -107,6 +107,8 @@ function buildMaestroArgs({
return { executable: 'maestro', args };
}
case 'maestro-runner': {
// The runner writes every report format to one directory. output_format only
// selects which extra report artifact EAS uploads after the test.
const args = [`--platform=${platform}`, 'test', `--output=${output}`, '--flatten'];
if (shards !== undefined && shards > 1) {
args.push(`--parallel=${shards}`);
@@ -213,23 +215,20 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
}
const testsDirectory = path.join(home, '.maestro', 'tests');
const junitReportDirectory = path.join(testsDirectory, 'junit-reports');
const finalReportPath =
outputFormat === 'junit'
? path.join(testsDirectory, `${platform}-maestro-junit.xml`)
: undefined;
const junitFinalReportPath = path.join(testsDirectory, `${platform}-maestro-junit.xml`);
// Public docs (EAS workflows pre-packaged-jobs) document
// `${MAESTRO_TESTS_DIR}` for users to save screenshots/recordings into
// the uploaded dir.
const spawnEnv: NodeJS.ProcessEnv = { ...env, MAESTRO_TESTS_DIR: testsDirectory };
// Outputs are published BEFORE any throw below so downstream
// `if: always()` upload steps still see populated values when this
// step fails early.
// Publish known paths before backend validation so `if: always()` upload
// steps can use them after an early failure. final_report_path names the
// report selected by output_format, not every report the backend produces.
outputs.tests_directory.set(testsDirectory);
outputs.junit_report_directory.set(junitReportDirectory);
if (finalReportPath !== undefined) {
outputs.final_report_path.set(finalReportPath);
if (outputFormat === 'junit') {
outputs.final_report_path.set(junitFinalReportPath);
}
// Resolved after the output assignments above: an invalid backend input or
@@ -239,6 +238,16 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
input: inputs.backend.value,
env,
});
// The runner can write JUnit regardless of output_format; Maestro CLI only
// writes it when JUnit is selected. Either command may fail before it writes a file.
const mayHaveJUnitReports = backend === 'maestro-runner' || outputFormat === 'junit';
const maestroCliOutputPath =
backend === 'maestro' && outputFormat && outputFormat !== 'junit'
? path.join(testsDirectory, `${platform}-maestro-${outputFormat}.${outputFormat}`)
: null;
if (backend === 'maestro' && outputFormat === 'html' && maestroCliOutputPath) {
outputs.final_report_path.set(maestroCliOutputPath);
}
const flowPaths = parseInput(
FlowPathSchema,
@@ -262,10 +271,14 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
undefined,
'android_connection_mode and EAS_MAESTRO_ANDROID_CONNECTION_MODE must be either "adb" or "dadb".'
);
if (backend === 'maestro-runner' && outputFormat !== undefined && outputFormat !== 'junit') {
if (
backend === 'maestro-runner' &&
outputFormat !== undefined &&
!['junit', 'html', 'allure'].includes(outputFormat)
) {
throw new UserError(
'ERR_MAESTRO_INVALID_INPUT',
`maestro-runner only supports the "junit" output_format, but received "${outputFormat}".`
`maestro-runner supports "junit", "html", and "allure" output_format values, but received "${outputFormat}".`
);
}
const retryFailedOnly = inputs.retry_failed_only.value as boolean;
@@ -337,17 +350,14 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
}
for (let attempt = 0; attempt <= retries; attempt++) {
// maestro-runner writes its JUnit report and screenshot metadata to this directory.
// maestro-runner writes all report formats and screenshot metadata here.
const runnerOutputDirectory = path.join(
testsDirectory,
`${platform}-maestro-runner-attempt-${attempt}`
);
const outputPath =
outputFormat === 'junit'
? path.join(junitReportDirectory, `${platform}-maestro-junit-attempt-${attempt}.xml`)
: backend === 'maestro' && outputFormat
? path.join(testsDirectory, `${platform}-maestro-${outputFormat}.${outputFormat}`)
: null;
const outputPath = mayHaveJUnitReports
? path.join(junitReportDirectory, `${platform}-maestro-junit-attempt-${attempt}.xml`)
: maestroCliOutputPath;
const { executable, args: maestroArgs } = buildMaestroArgs({
backend,
platform,
@@ -371,6 +381,17 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
logger.warn({ err }, `Failed to clear ${runnerOutputDirectory} before the attempt.`);
}
}
// Maestro CLI uses the same HTML path for every retry. Remove the previous report
// before spawning so an attempt that fails before report generation cannot upload it.
if (backend === 'maestro' && outputFormat === 'html' && maestroCliOutputPath) {
try {
await fs.rm(maestroCliOutputPath, { force: true });
} catch (err) {
throw new SystemError('Failed to clear the previous Maestro HTML report', {
cause: err,
});
}
}
const attemptStartedAtMs = Date.now();
@@ -412,9 +433,8 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
reportDirectories.push(runnerOutputDirectory);
}
// Harvest this attempt's failure screenshots before any retry subsetting. Gated on
// junit: test-case-result rows (and therefore the summary icons) only exist for junit
// runs, so harvesting other formats would just create orphan artifacts the website hides.
// The pre-packaged job reports test cases only for JUnit (the default format).
// Individual screenshots need those test cases to be visible on the website.
if (outputFormat === 'junit') {
let screenshots: HarvestedScreenshot[];
switch (backend) {
@@ -504,14 +524,15 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
await sleepAsync(2000);
}
// Smart merge first; on data errors (bad XML, missing input) fall back
// to copy-latest so the caller still gets a single JUnit file.
// Only JUnit is merged into final_report_path. Runner HTML/Allure keeps its
// per-attempt JUnit files, but final_report_path names the selected report.
// On data errors (bad XML, missing input), fall back to copy-latest.
// Filesystem errors short-circuit straight to SystemError.
if (finalReportPath !== undefined) {
if (outputFormat === 'junit') {
try {
await mergeJUnitReports({
sourceDir: junitReportDirectory,
outputPath: finalReportPath,
outputPath: junitFinalReportPath,
});
} catch (mergeErr: any) {
if (isFilesystemError(mergeErr)) {
@@ -521,7 +542,7 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
try {
await copyLatestAttemptXml({
sourceDir: junitReportDirectory,
outputPath: finalReportPath,
outputPath: junitFinalReportPath,
});
} catch (copyErr: any) {
// Swallow: a copy failure here usually means maestro itself failed
@@ -529,14 +550,13 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
// the real reason and cancel billing for a user-side failure — let
// the lastAttemptExitCode check below surface ERR_MAESTRO_TESTS_FAILED.
logger.warn(
`Failed to produce final_report_path at ${finalReportPath}: ${copyErr?.message ?? copyErr}`
`Failed to produce final_report_path at ${junitFinalReportPath}: ${copyErr?.message ?? copyErr}`
);
}
}
}
// Upload before the ERR_MAESTRO_TESTS_FAILED throw below so fully-failed runs (which need
// screenshots most) still upload. Harvest only ran for junit, so guard the same way.
// Upload before the failure verdict so fully-failed runs still get screenshots.
if (outputFormat === 'junit') {
await uploadFailureScreenshotsAsync({
harvested,
@@ -547,6 +567,81 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
});
}
// Upload the selected non-JUnit report before the failure verdict. The pre-packaged
// job also uploads testsDirectory, but a named artifact makes the report easy to find.
let selectedReport:
| { name: string; artifactPath: string; finalReportPath: string }
| undefined;
switch (outputFormat) {
case 'html': {
switch (backend) {
case 'maestro': {
if (!maestroCliOutputPath) {
break;
}
selectedReport = {
name: 'Maestro HTML Report',
artifactPath: maestroCliOutputPath,
finalReportPath: maestroCliOutputPath,
};
break;
}
case 'maestro-runner': {
const reportDirectory = reportDirectories.at(-1);
if (!reportDirectory) {
break;
}
// HTML references nearby screenshots, so keep its whole attempt directory.
selectedReport = {
name: 'Maestro Runner HTML Report',
artifactPath: reportDirectory,
finalReportPath: path.join(reportDirectory, 'report.html'),
};
break;
}
}
break;
}
case 'allure': {
switch (backend) {
case 'maestro':
logger.warn(
'Maestro CLI does not support Allure reports; no Allure artifact was uploaded.'
);
break;
case 'maestro-runner': {
const reportDirectory = reportDirectories.at(-1);
if (!reportDirectory) {
break;
}
const allureResultsDirectory = path.join(reportDirectory, 'allure-results');
selectedReport = {
name: 'Maestro Runner Allure Results',
artifactPath: allureResultsDirectory,
finalReportPath: allureResultsDirectory,
};
break;
}
}
break;
}
}
if (selectedReport) {
outputs.final_report_path.set(selectedReport.finalReportPath);
try {
await ctx.runtimeApi.uploadArtifact({
artifact: {
type: GenericArtifactType.OTHER,
name: selectedReport.name,
paths: [selectedReport.artifactPath],
},
logger,
});
} catch (err: any) {
logger.warn({ err }, `Failed to upload ${selectedReport.name}.`);
}
}
// The retry loop exits via success (0), numeric status (retryable),
// or throw (infra). A non-null non-zero status means the user's tests
// failed every attempt.
@@ -17,6 +17,7 @@ import {
uploadRemoteSessionConfigWithLocalEgressAsync,
withLocalEgressSession,
} from '../utils/localEgressSession';
import { type DeviceSessionHost, startDeviceSessionHostAsync } from '../utils/deviceSessionHost';
import { Sentry } from '../../sentry';
import {
PackageManager,
@@ -30,13 +31,13 @@ import {
type DetachedProcessHandle,
createServeSimLaunchInputProviders,
describeServeSimLaunch,
finishRemoteSessionAsync,
getDeviceRunSessionIdOrThrow,
getNgrokAuthtokenOrThrow,
getNgrokTunnelDomainOrThrow,
parseServeSimLaunchInputs,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
waitForDeviceRunSessionStoppedAsync,
waitForFileAsync,
@@ -51,6 +52,13 @@ const STARTUP_TIMEOUT_MS = 60_000;
const AGENT_DEVICE_DAEMON_ENV = {
AGENT_DEVICE_DAEMON_SERVER_MODE: 'http',
AGENT_DEVICE_RETAIN_ARTIFACTS: '1',
// The session lifetime is owned by max_idle_time_minutes / max_duration_seconds
// and the device run session stop, so disable agent-device's own idle timers.
// Without this the daemon exits after 5 minutes with no open session, and the
// tunnel is left without a daemon behind it.
AGENT_DEVICE_DAEMON_IDLE_TIMEOUT_MS: '0',
AGENT_DEVICE_IOS_RUNNER_IDLE_STOP_MS: '0',
AGENT_DEVICE_SESSION_IDLE_TIMEOUT_MS: '0',
};
export function createStartAgentDeviceRemoteSessionBuildFunction(
@@ -128,18 +136,18 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
const agentDeviceRemoteSessionUrl = agentDeviceTunnel.url;
logger.info(`Tunnel is ready at ${agentDeviceRemoteSessionUrl}.`);
let webPreview: Awaited<ReturnType<typeof startDeviceWebPreviewWithTunnelAsync>> | undefined;
let sessionHost: DeviceSessionHost | undefined;
let eventCollection:
| Awaited<ReturnType<typeof startAgentDeviceEventCollectionAsync>>
| undefined;
let sessionFailed = false;
try {
const launchDescription = describeServeSimLaunch(launch);
if (launchDescription) {
logger.info(launchDescription);
}
webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
sessionHost = await startDeviceSessionHostAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
@@ -147,6 +155,7 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
launchArgs: launch.launchArgs,
openUrl: launch.openUrl,
});
const webPreview = await sessionHost.openPreviewAsync({ baseDomain: ngrokTunnelDomain });
logger.info(
`Web preview URL: ${webPreview.previewPageUrl} (server: ${webPreview.apiUrl}).`
);
@@ -193,19 +202,34 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
}
: undefined,
});
} catch (error) {
sessionFailed = true;
throw error;
} finally {
if (webPreview) {
await webPreview.stopAsync();
}
await agentDeviceTunnel.stopAsync();
if (eventCollection) {
await stopAgentDeviceEventCollectionSafelyAsync({
eventCollection,
deviceRunSessionId,
logger,
});
}
await daemonProcess.stopAsync();
await finishRemoteSessionAsync({
logger,
sessionFailed,
teardown: [
['agent-device tunnel', agentDeviceTunnel.stopAsync()],
[
'agent-device daemon',
(async () => {
try {
if (eventCollection) {
await stopAgentDeviceEventCollectionSafelyAsync({
eventCollection,
deviceRunSessionId,
logger,
});
}
} finally {
await daemonProcess.stopAsync();
}
})(),
],
['session host', sessionHost?.finishAsync()],
],
});
}
}),
});
@@ -19,6 +19,7 @@ import {
uploadRemoteSessionConfigWithLocalEgressAsync,
withLocalEgressSession,
} from '../utils/localEgressSession';
import { type DeviceSessionHost, startDeviceSessionHostAsync } from '../utils/deviceSessionHost';
import { AndroidEmulatorUtils } from '../../utils/AndroidEmulatorUtils';
import { IosSimulatorUtils } from '../../utils/IosSimulatorUtils';
import {
@@ -32,13 +33,13 @@ import { startAppiumEventCollectionAsync } from '../utils/appiumEvents';
import {
createServeSimLaunchInputProviders,
describeServeSimLaunch,
finishRemoteSessionAsync,
getDeviceRunSessionIdOrThrow,
getNgrokAuthtokenOrThrow,
getNgrokTunnelDomainOrThrow,
parseServeSimLaunchInputs,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
@@ -138,7 +139,8 @@ export function createStartAppiumRemoteSessionBuildFunction(
logger,
});
let appiumTunnel: Awaited<ReturnType<typeof startNgrokTunnelAsync>> | undefined;
let webPreview: Awaited<ReturnType<typeof startDeviceWebPreviewWithTunnelAsync>> | undefined;
let sessionHost: DeviceSessionHost | undefined;
let sessionFailed = false;
try {
appiumTunnel = await startNgrokTunnelAsync({
port: APPIUM_PORT,
@@ -154,9 +156,8 @@ export function createStartAppiumRemoteSessionBuildFunction(
if (launchDescription) {
logger.info(launchDescription);
}
webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
sessionHost = await startDeviceSessionHostAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: APPIUM_STARTUP_TIMEOUT_MS,
@@ -164,6 +165,7 @@ export function createStartAppiumRemoteSessionBuildFunction(
launchArgs: launch.launchArgs,
openUrl: launch.openUrl,
});
const webPreview = await sessionHost.openPreviewAsync({ baseDomain: ngrokTunnelDomain });
await uploadRemoteSessionConfigWithLocalEgressAsync({
env,
@@ -197,16 +199,29 @@ export function createStartAppiumRemoteSessionBuildFunction(
}
: undefined,
});
} catch (error) {
sessionFailed = true;
throw error;
} finally {
if (webPreview) {
await webPreview.stopAsync();
}
if (appiumTunnel) {
await appiumTunnel.stopAsync();
}
await eventCollection.stopAsync();
await appiumProcess.stopAsync();
await fs.promises.rm(appiumHome, { recursive: true, force: true });
await finishRemoteSessionAsync({
logger,
sessionFailed,
teardown: [
['Appium tunnel', appiumTunnel?.stopAsync()],
[
'Appium server',
(async () => {
try {
await eventCollection.stopAsync();
} finally {
await appiumProcess.stopAsync();
await fs.promises.rm(appiumHome, { recursive: true, force: true });
}
})(),
],
['session host', sessionHost?.finishAsync()],
],
});
}
}),
});
@@ -18,6 +18,7 @@ import {
uploadRemoteSessionConfigWithLocalEgressAsync,
withLocalEgressSession,
} from '../utils/localEgressSession';
import { type DeviceSessionHost, startDeviceSessionHostAsync } from '../utils/deviceSessionHost';
import { Sentry } from '../../sentry';
import {
PackageManager,
@@ -32,13 +33,13 @@ import {
createServeSimLaunchInputProviders,
describeServeSimLaunch,
ensureFfmpegInstalledOnceAsync,
finishRemoteSessionAsync,
getDeviceRunSessionIdOrThrow,
getNgrokAuthtokenOrThrow,
getNgrokTunnelDomainOrThrow,
parseServeSimLaunchInputs,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
@@ -184,6 +185,7 @@ export function createStartArgentRemoteSessionBuildFunction(
stateDir: ARGENT_STATE_DIR,
ancestorPid: argentServer.pid,
timeoutMs: STARTUP_TIMEOUT_MS,
getExitError: argentServer.getExitError,
});
toolServerPort = toolServerState.port;
toolServerToken = toolServerState.token;
@@ -216,7 +218,8 @@ export function createStartArgentRemoteSessionBuildFunction(
});
let toolsTunnel: Awaited<ReturnType<typeof startNgrokTunnelAsync>> | undefined;
let webPreview: Awaited<ReturnType<typeof startDeviceWebPreviewWithTunnelAsync>> | undefined;
let sessionHost: DeviceSessionHost | undefined;
let sessionFailed = false;
try {
toolsTunnel = await startNgrokTunnelAsync({
port: toolServerPort,
@@ -233,9 +236,8 @@ export function createStartArgentRemoteSessionBuildFunction(
if (launchDescription) {
logger.info(launchDescription);
}
webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
sessionHost = await startDeviceSessionHostAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
@@ -243,6 +245,7 @@ export function createStartArgentRemoteSessionBuildFunction(
launchArgs: launch.launchArgs,
openUrl: launch.openUrl,
});
const webPreview = await sessionHost.openPreviewAsync({ baseDomain: ngrokTunnelDomain });
logger.info(
`Web preview URL: ${webPreview.previewPageUrl} (server: ${webPreview.apiUrl}).`
);
@@ -276,23 +279,46 @@ export function createStartArgentRemoteSessionBuildFunction(
}
: undefined,
});
} catch (error) {
sessionFailed = true;
throw error;
} finally {
if (webPreview) {
await webPreview.stopAsync();
}
if (toolsTunnel) {
await toolsTunnel.stopAsync();
}
await stopArgentEventCollectionSafelyAsync({ eventCollection, deviceRunSessionId, logger });
artifactPollAbortController.abort();
try {
await artifactPollingPromise;
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not finish Argent remote session artifact polling', error);
logger.warn({ err: error }, 'Could not finish Argent remote session artifact polling.');
}
await argentServer.stopAsync();
await finishRemoteSessionAsync({
logger,
sessionFailed,
teardown: [
['Argent tunnel', toolsTunnel?.stopAsync()],
[
'Argent tool-server',
(async () => {
try {
await stopArgentEventCollectionSafelyAsync({
eventCollection,
deviceRunSessionId,
logger,
});
artifactPollAbortController.abort();
try {
await artifactPollingPromise;
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture(
'Could not finish Argent remote session artifact polling',
error
);
logger.warn(
{ err: error },
'Could not finish Argent remote session artifact polling.'
);
}
} finally {
await argentServer.stopAsync();
}
})(),
],
['session host', sessionHost?.finishAsync()],
],
});
}
}),
});
@@ -371,16 +397,26 @@ export async function waitForArgentToolServerStateAsync({
ancestorPid,
timeoutMs,
pollIntervalMs = 1_000,
getExitError,
}: {
stateDir: string;
ancestorPid: number;
timeoutMs: number;
pollIntervalMs?: number;
getExitError?: () => Error | undefined;
}): Promise<ArgentToolServerState> {
const deadline = Date.now() + timeoutMs;
let lastError: unknown;
const throwIfExited = (): void => {
const error = getExitError?.();
if (error) {
throw new SystemError(`Argent exited before becoming ready: ${error.message}`);
}
};
while (Date.now() < deadline) {
throwIfExited();
let matchingState: ArgentToolServerState | undefined;
try {
const stateFileNames = (await fs.promises.readdir(stateDir)).filter(
name => name.startsWith('tool-server') && name.endsWith('.json')
@@ -391,7 +427,8 @@ export async function waitForArgentToolServerStateAsync({
await fs.promises.readFile(path.join(stateDir, stateFileName), 'utf8')
);
if (await isProcessDescendantOfAsync(state.pid, ancestorPid)) {
return state;
matchingState = state;
break;
}
} catch (err) {
if ((err as NodeJS.ErrnoException).code !== 'ENOENT' && !(err instanceof SystemError)) {
@@ -402,6 +439,11 @@ export async function waitForArgentToolServerStateAsync({
} catch (err) {
lastError = err;
}
// Keep process failures outside the retryable state-file read errors.
throwIfExited();
if (matchingState) {
return matchingState;
}
await sleepAsync(pollIntervalMs);
}
@@ -3,6 +3,7 @@ import { BuildFunction, BuildStepInput, BuildStepInputValueTypeName } from '@exp
import { graphql } from 'gql.tada';
import { CustomBuildContext } from '../../customBuildContext';
import { graphqlAbortContext } from '../../utils/graphqlAbort';
import { withLogPhaseAsync } from '../../utils/logPhase';
import { startSandboxDaemonAsync } from '../utils/sandboxDaemon';
@@ -76,19 +77,7 @@ export async function markSandboxReadyAsync(
): Promise<void> {
signal?.throwIfAborted();
const result = await ctx.graphqlClient
.mutation(
MARK_SANDBOX_READY_MUTATION,
{ sandboxId },
signal
? {
fetch: (input, init) =>
fetch(input, {
...init,
signal: init?.signal ? AbortSignal.any([signal, init.signal]) : signal,
}),
}
: undefined
)
.mutation(MARK_SANDBOX_READY_MUTATION, { sandboxId }, graphqlAbortContext(signal))
.toPromise();
signal?.throwIfAborted();
if (result.error) {
@@ -0,0 +1,26 @@
import { BuildFunction, BuildRuntimePlatform } from '@expo/steps';
import { Sentry } from '../../sentry';
import { ServeSimLogsRecorder } from '../utils/serveSimLogsRecorder';
export function createStartServeSimLogsBuildFunction(): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'start_serve_sim_logs',
name: 'Start serve-sim simulator logs',
__metricsId: 'eas/start_serve_sim_logs',
supportedRuntimePlatforms: [BuildRuntimePlatform.DARWIN],
fn: async ({ logger }) => {
try {
await ServeSimLogsRecorder.startAsync({ logger });
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not start serve-sim simulator logs', error);
logger.warn(
{ err: error },
'Could not start simulator log collection; the session will continue.'
);
}
},
});
}
@@ -6,6 +6,7 @@ import {
} from '@expo/steps';
import { CustomBuildContext } from '../../customBuildContext';
import { startDeviceSessionHostAsync } from '../utils/deviceSessionHost';
import {
uploadRemoteSessionConfigWithLocalEgressAsync,
withLocalEgressSession,
@@ -17,7 +18,6 @@ import {
getNgrokTunnelDomainOrThrow,
parseServeSimLaunchInputs,
selectXcodeDeveloperDirectoryAsync,
startDeviceWebPreviewWithTunnelAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
@@ -69,9 +69,8 @@ export function createStartWebPreviewRemoteSessionBuildFunction(
await selectXcodeDeveloperDirectoryAsync({ env, logger });
}
const webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
const sessionHost = await startDeviceSessionHostAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
@@ -80,9 +79,10 @@ export function createStartWebPreviewRemoteSessionBuildFunction(
launchArgs: launch.launchArgs,
openUrl: launch.openUrl,
});
logger.info(`Preview URL: ${webPreview.previewPageUrl} (server: ${webPreview.apiUrl}).`);
try {
const webPreview = await sessionHost.openPreviewAsync({ baseDomain: ngrokTunnelDomain });
logger.info(`Preview URL: ${webPreview.previewPageUrl} (server: ${webPreview.apiUrl}).`);
await uploadRemoteSessionConfigWithLocalEgressAsync({
env,
signal,
@@ -104,7 +104,7 @@ export function createStartWebPreviewRemoteSessionBuildFunction(
signal,
});
} finally {
await webPreview.stopAsync();
await sessionHost.finishAsync();
}
}),
});
@@ -0,0 +1,235 @@
import { SystemError, UserError } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import { BuildFunction, BuildStepInput, BuildStepInputValueTypeName } from '@expo/steps';
import path from 'node:path';
import limitFactory from 'promise-limit';
import { z } from 'zod';
import { AscApiClient, AscApiRequestError } from '../utils/ios/AscApiClient';
import { AscApiUtils } from '../utils/ios/AscApiUtils';
export function createUpdateTestFlightMetadataBuildFunction(): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'update_testflight_metadata',
name: 'Update TestFlight metadata',
__metricsId: 'eas/update_testflight_metadata',
inputProviders: [
BuildStepInput.createProvider({
id: 'asc_api_key_path',
required: true,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
id: 'build_upload_id',
required: true,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
id: 'changelog',
required: false,
defaultValue: '',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
id: 'groups',
required: false,
defaultValue: [],
allowedValueTypeName: BuildStepInputValueTypeName.JSON,
}),
],
fn: async (ctx, { inputs }) => {
const parsedInputs = z
.object({
asc_api_key_path: z.string(),
build_upload_id: z.string(),
changelog: z.string(),
groups: z.array(z.string()),
})
.parse({
asc_api_key_path: inputs.asc_api_key_path.value,
build_upload_id: inputs.build_upload_id.value,
changelog: inputs.changelog.value,
groups: inputs.groups.value,
});
const keyPath = path.resolve(ctx.workingDirectory, parsedInputs.asc_api_key_path);
const token = await AscApiUtils.signTokenAsync({ keyPath });
await updateTestFlightMetadataAsync({
client: new AscApiClient({ token, logger: ctx.logger }),
buildUploadId: parsedInputs.build_upload_id,
changelog: parsedInputs.changelog,
groups: parsedInputs.groups,
logger: ctx.logger,
});
ctx.logger.info('TestFlight metadata updated.');
},
});
}
export async function updateTestFlightMetadataAsync({
client,
buildUploadId,
changelog,
groups,
logger,
}: {
client: AscApiClient;
buildUploadId: string;
changelog: string;
groups: string[];
logger: bunyan;
}): Promise<void> {
const { data: upload } = await client.getAsync(
'/v1/buildUploads/:id',
{
'fields[buildUploads]': ['state', 'build'],
include: ['build'],
},
{ id: buildUploadId }
);
const buildId = upload.relationships?.build?.data?.id;
if (upload.attributes?.state?.state !== 'COMPLETE' || !buildId) {
throw new UserError(
'EAS_TESTFLIGHT_BUILD_NOT_READY',
'The uploaded build is not ready for TestFlight metadata. Run eas/upload_to_asc with wait_for_processing: true first.'
);
}
logger.info(`Updating TestFlight metadata for Apple build ${buildId}...`);
const { data: app } = await client.getAsync('/v1/builds/:id/app', {}, { id: buildId });
const groupIds: string[] = [];
if (groups.length) {
const requestedNames = new Set(groups);
const foundNames = new Set<string>();
let response = await client.getAsync('/v1/betaGroups', {
'filter[app]': app.id,
limit: 200,
});
for (let page = 1; page <= 20; page++) {
for (const group of response.data) {
const name = group.attributes?.name;
if (name && requestedNames.has(name)) {
foundNames.add(name);
groupIds.push(group.id);
}
}
if (!response.links?.next) {
break;
}
if (page === 20) {
throw new SystemError('The TestFlight group list has more than 20 pages.');
}
response = await client.getNextPageAsync('/v1/betaGroups', response.links.next);
}
const missingNames = [...requestedNames].filter(name => !foundNames.has(name));
if (missingNames.length) {
throw new UserError(
'EAS_TESTFLIGHT_GROUPS_NOT_FOUND',
`The following TestFlight group${missingNames.length > 1 ? 's were' : ' was'} not found in App Store Connect: ${missingNames.map(name => `"${name}"`).join(', ')}. Check the group names and try again.`
);
}
logger.info(`Found ${groupIds.length} TestFlight group(s).`);
}
const limit = limitFactory<void>(1);
const results = await Promise.allSettled([
limit(async () => {
if (!changelog) {
return;
}
logger.info('Updating TestFlight changelog...');
const primaryLocale = app.attributes?.primaryLocale;
if (!primaryLocale) {
throw new SystemError('App Store Connect did not return the app primary locale.');
}
const response = await client.getAsync(
'/v1/builds/:id/betaBuildLocalizations',
{ limit: 200 },
{ id: buildId }
);
if (response.links?.next) {
throw new SystemError(
'Cannot update all TestFlight localizations in one page. Update the changelog in App Store Connect.'
);
}
if (response.data.some(localization => !localization.attributes?.locale)) {
throw new SystemError('App Store Connect did not return a TestFlight localization locale.');
}
for (const localization of response.data) {
await client.patchAsync(
'/v1/betaBuildLocalizations/:id',
{
data: {
type: 'betaBuildLocalizations',
id: localization.id,
attributes: { whatsNew: changelog },
},
},
{ id: localization.id }
);
}
if (!response.data.some(localization => localization.attributes?.locale === primaryLocale)) {
await client.postAsync('/v1/betaBuildLocalizations', {
data: {
type: 'betaBuildLocalizations',
attributes: { locale: primaryLocale, whatsNew: changelog },
relationships: { build: { data: { type: 'builds', id: buildId } } },
},
});
}
}),
limit(async () => {
if (!groupIds.length) {
return;
}
logger.info(`Adding Apple build to ${groupIds.length} TestFlight group(s)...`);
try {
await client.postAsync(
'/v1/builds/:id/relationships/betaGroups',
{
data: groupIds.map(id => ({ type: 'betaGroups', id })),
},
{ id: buildId }
);
} catch (error) {
if (isInternalGroupAssignmentError(error)) {
throw new UserError(
'EAS_TESTFLIGHT_INTERNAL_GROUP_ASSIGNMENT_FAILED',
"App Store Connect can't add this build to a requested internal TestFlight group. " +
"Internal groups that automatically receive new builds can't be assigned to manually. " +
'Remove the group from the list, or turn off automatic distribution in App Store Connect. ' +
`Manage groups at https://appstoreconnect.apple.com/apps/${app.id}/testflight`,
{ cause: error }
);
}
throw error;
}
}),
]);
const failures = results.filter(result => result.status === 'rejected');
if (failures.length === 1) {
throw failures[0].reason;
}
if (failures.length > 1) {
throw new AggregateError(
failures.map(failure => failure.reason),
`Failed to update the TestFlight changelog and groups: ${failures
.map(failure => String(failure.reason))
.join('; ')}`
);
}
}
// Apple returns a generic 422 code, so match the title or detail too.
function isInternalGroupAssignmentError(error: unknown): boolean {
return (
error instanceof AscApiRequestError &&
error.status === 422 &&
error.responseJson.errors.some(
({ code, title, detail }) =>
code === 'ENTITY_UNPROCESSABLE' &&
(title === 'Builds cannot be assigned to this internal group.' ||
detail === 'Cannot add internal group to a build.')
)
);
}
@@ -1,53 +1,17 @@
import { SystemError } from '@expo/eas-build-job';
import {
BuildFunction,
BuildRuntimePlatform,
BuildStepInput,
BuildStepInputValueTypeName,
} from '@expo/steps';
import { createReadStream } from 'node:fs';
import { readFile, stat } from 'node:fs/promises';
import path from 'node:path';
import limitFactory from 'promise-limit';
import { z } from 'zod';
import { type CustomBuildContext } from '../../customBuildContext';
import { Sentry } from '../../sentry';
import { formatBytes } from '../../utils/artifacts';
import { uploadDeviceRunSessionArtifactAsync } from '../utils/deviceRunSessionArtifacts';
import {
parseDeviceScreenRecordings,
uploadDeviceRunSessionScreenRecordingsAsync,
} from '../utils/deviceRunSessionScreenRecordings';
import { getDeviceRunSessionIdOrThrow } from '../utils/remoteDeviceRunSession';
const RecordingsSchema = z.array(
z.object({
udid: z.string(),
deviceName: z.string(),
runtimeDisplayName: z.string(),
directory: z.string(),
})
);
const RecordingManifestSchema = z.object({
firstFrameWallClock: z.object({
iso8601: z.string(),
}),
width: z.number().int().positive(),
height: z.number().int().positive(),
recording: z.string(),
});
const recordingStartTimeFormatter = new Intl.DateTimeFormat('en-US', {
year: 'numeric',
month: 'short',
day: 'numeric',
hour: '2-digit',
minute: '2-digit',
second: '2-digit',
fractionalSecondDigits: 3,
hourCycle: 'h23',
timeZone: 'UTC',
timeZoneName: 'short',
});
export function createUploadDeviceRunSessionScreenRecordingsBuildFunction(
ctx: CustomBuildContext
): BuildFunction {
@@ -65,69 +29,16 @@ export function createUploadDeviceRunSessionScreenRecordingsBuildFunction(
}),
],
fn: async ({ logger }, { env, inputs }) => {
const result = RecordingsSchema.safeParse(inputs.recordings_json.value ?? []);
if (!result.success) {
throw new SystemError('Invalid iOS Simulator recordings input.', {
cause: result.error,
});
}
const recordings = result.data;
const recordings = parseDeviceScreenRecordings(inputs.recordings_json.value ?? []);
if (recordings.length === 0) {
logger.info('No iOS Simulator recordings found; skipping uploads.');
logger.info('No device screen recordings found; skipping uploads.');
return;
}
const deviceRunSessionId = getDeviceRunSessionIdOrThrow(env);
const limit = limitFactory(5);
await Promise.all(
recordings.map(recording =>
limit(async () => {
try {
const metadata = RecordingManifestSchema.parse(
JSON.parse(await readFile(path.join(recording.directory, 'session.json'), 'utf-8'))
);
const startedAt = recordingStartTimeFormatter.format(
new Date(metadata.firstFrameWallClock.iso8601)
);
const shortUdid = `${recording.udid.slice(0, 8)}-…`;
const displayName = `${recording.deviceName} screen recording (${shortUdid}, started at ${startedAt})`;
const recordingPath = path.join(recording.directory, metadata.recording);
const { size } = await stat(recordingPath);
const recordingId = path.basename(recording.directory);
logger.info(
`Uploading screen recording for ${recording.deviceName} (${formatBytes(size)}).`
);
await uploadDeviceRunSessionArtifactAsync(ctx, {
deviceRunSessionId,
artifactId: recordingId,
name: displayName,
filename: `${recordingId}.mp4`,
kind: 'screen-recording',
metadata: {
__eas_type: 'screen-recording',
__eas_screen_recording: '1',
udid: recording.udid,
deviceName: recording.deviceName,
runtimeDisplayName: recording.runtimeDisplayName,
firstFrameAt: metadata.firstFrameWallClock.iso8601,
width: metadata.width,
height: metadata.height,
},
size,
stream: createReadStream(recordingPath),
});
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not upload iOS Simulator screen recording', error);
logger.warn(
{ err: error },
`Could not upload screen recording for ${recording.deviceName}.`
);
}
})
)
);
await uploadDeviceRunSessionScreenRecordingsAsync(ctx, {
logger,
deviceRunSessionId: getDeviceRunSessionIdOrThrow(env),
recordings,
});
},
});
}
@@ -7,7 +7,6 @@ import {
BuildStepOutput,
} from '@expo/steps';
import fs from 'fs-extra';
import * as jose from 'jose';
import fetch from 'node-fetch';
import path from 'node:path';
import { setTimeout } from 'node:timers/promises';
@@ -87,28 +86,7 @@ export function createUploadToAscBuildFunction(): BuildFunction {
throw new Error(`ASC API Key file not found: ${ascApiKeyPath}`);
}
const ascApiKeyJson = await fs.readJson(ascApiKeyPath);
const ascApiKey = z
.object({
issuer_id: z.string().nullish(),
key_id: z.string(),
key: z.string(),
})
.parse(ascApiKeyJson);
const privateKey = await jose.importPKCS8(ascApiKey.key, 'ES256');
const jwt = new jose.SignJWT({})
.setProtectedHeader({ alg: 'ES256', kid: ascApiKey.key_id })
.setAudience('appstoreconnect-v1')
.setExpirationTime('20m');
if (ascApiKey.issuer_id) {
jwt.setIssuer(ascApiKey.issuer_id);
} else {
// Nullish issuer_id means an individual API key
jwt.setSubject('user');
}
const token = await jwt.sign(privateKey);
const token = await AscApiUtils.signTokenAsync({ keyPath: ascApiKeyPath });
const client = new AscApiClient({ token, logger: stepsCtx.logger });
stepsCtx.logger.info(
@@ -63,7 +63,8 @@ describe(uploadDeviceRunSessionArtifactAsync, () => {
metadata: { firstFrameRecordAt: 'test-time' },
size: reportedSize,
},
})
}),
{ fetch: expect.any(Function) }
);
expect(jest.mocked(fetch)).toHaveBeenCalledWith(
'https://uploads.expo.test/artifact',
@@ -73,4 +74,52 @@ describe(uploadDeviceRunSessionArtifactAsync, () => {
})
);
});
it('forwards cancellation through upload allocation and skips PUT when allocation aborts', async () => {
const controller = new AbortController();
const mutation = jest.fn().mockReturnValue({
toPromise: async () => {
controller.abort();
return {
data: {
deviceRunSession: {
createArtifactUploadSession: {
uploadSession: {
url: 'https://uploads.expo.test/artifact',
headers: {},
},
},
},
},
};
},
});
const ctx = { graphqlClient: { mutation } } as unknown as CustomBuildContext;
await expect(
uploadDeviceRunSessionArtifactAsync(ctx, {
deviceRunSessionId: 'session',
artifactId: 'id',
name: 'Logs',
filename: 'logs.ndjson',
kind: 'simulator-log',
size: 0,
stream: Readable.from([]),
signal: controller.signal,
})
).rejects.toThrow();
expect(fetch).not.toHaveBeenCalled();
const allocationFetch = mutation.mock.calls[0][2].fetch;
const mockFetch = jest.spyOn(globalThis, 'fetch').mockRejectedValueOnce(new Error('aborted'));
try {
await expect(allocationFetch('https://api.expo.test/graphql', {})).rejects.toThrow('aborted');
expect(mockFetch).toHaveBeenCalledWith(
'https://api.expo.test/graphql',
expect.objectContaining({
signal: expect.objectContaining({ aborted: true }),
})
);
} finally {
mockFetch.mockRestore();
}
});
});
@@ -0,0 +1,80 @@
import {
DeviceRunSessionTimeoutError,
withDeviceRunSessionTimeoutAsync,
} from '../deviceRunSessionTimeout';
afterEach(() => jest.useRealTimers());
it('aborts a stalled operation when its deadline expires', async () => {
jest.useFakeTimers();
let signal: AbortSignal | undefined;
const operation = withDeviceRunSessionTimeoutAsync(
{ name: 'test operation', timeoutMs: 100 },
async value => {
signal = value;
return await new Promise<void>(() => {});
}
);
const rejected = expect(operation).rejects.toThrow('test operation timed out after 100ms');
const typed = expect(operation).rejects.toBeInstanceOf(DeviceRunSessionTimeoutError);
await jest.advanceTimersByTimeAsync(100);
await rejected;
await typed;
expect(signal?.aborted).toBe(true);
expect(jest.getTimerCount()).toBe(0);
});
it('does not start an operation with an already aborted parent', async () => {
const operation = jest.fn();
await expect(
withDeviceRunSessionTimeoutAsync(
{
name: 'test',
timeoutMs: 100,
signal: AbortSignal.abort(new Error('parent ended')),
},
operation
)
).rejects.toThrow('parent ended');
expect(operation).not.toHaveBeenCalled();
});
it('clears the deadline when an operation succeeds', async () => {
jest.useFakeTimers();
await expect(
withDeviceRunSessionTimeoutAsync({ name: 'test', timeoutMs: 100 }, async () => 42)
).resolves.toBe(42);
expect(jest.getTimerCount()).toBe(0);
});
it('ignores a deadline reset that arrives after the operation settled', async () => {
jest.useFakeTimers();
let resetDeadline!: () => void;
await expect(
withDeviceRunSessionTimeoutAsync({ name: 'test', timeoutMs: 100 }, async (_signal, reset) => {
resetDeadline = reset;
return 42;
})
).resolves.toBe(42);
resetDeadline();
expect(jest.getTimerCount()).toBe(0);
});
it('keeps an operation alive while it reports progress', async () => {
jest.useFakeTimers();
const operation = withDeviceRunSessionTimeoutAsync(
{ name: 'test operation', timeoutMs: 100 },
async (signal, resetDeadline) => {
await new Promise<void>(resolve => setTimeout(resolve, 80));
resetDeadline();
await new Promise<void>(resolve => setTimeout(resolve, 80));
resetDeadline();
await new Promise<void>(resolve => setTimeout(resolve, 80));
return signal.aborted;
}
);
const settled = expect(operation).resolves.toBe(false);
await jest.advanceTimersByTimeAsync(240);
await settled;
expect(jest.getTimerCount()).toBe(0);
});
@@ -0,0 +1,463 @@
import type { bunyan } from '@expo/logger';
import type { BuildStepEnv } from '@expo/steps';
import { Client, fetchExchange } from '@urql/core';
import { execFileSync } from 'node:child_process';
import { createReadStream } from 'node:fs';
import { copyFile, mkdir, mkdtemp, rm, stat, writeFile } from 'node:fs/promises';
import { type IncomingMessage, type ServerResponse, createServer } from 'node:http';
import type { Socket } from 'node:net';
import { tmpdir } from 'node:os';
import path from 'node:path';
import { Readable } from 'node:stream';
import { setTimeout as delay } from 'node:timers/promises';
import type { CustomBuildContext } from '../../../customBuildContext';
import { uploadDeviceRunSessionArtifactAsync } from '../deviceRunSessionArtifacts';
import {
findUnlistedDeviceScreenRecordingsAsync,
uploadDeviceRunSessionScreenRecordingsAsync,
} from '../deviceRunSessionScreenRecordings';
jest.unmock('node-fetch');
jest.unmock('node:fs');
jest.unmock('node:fs/promises');
function deferred<T>() {
let resolve!: (value: T) => void;
const promise = new Promise<T>(r => {
resolve = r;
});
return { promise, resolve };
}
const metadata = {
deviceRunSessionId: 'drs-id',
artifactId: 'recording-id',
name: 'Recording',
filename: 'recording.mp4',
kind: 'screen-recording',
size: 16,
};
let handle: (request: IncomingMessage, response: ServerResponse) => void;
let server: ReturnType<typeof createServer>;
let ctx: CustomBuildContext;
let url: string;
let directory: string;
let creates: number;
let authorization: string | undefined;
let stallCreation: boolean;
let creationStarted: ReturnType<typeof deferred<IncomingMessage>>;
let creationBodies: string[];
beforeEach(async () => {
creates = 0;
authorization = undefined;
stallCreation = false;
creationBodies = [];
creationStarted = deferred();
directory = await mkdtemp(path.join(tmpdir(), 'recording-upload-http-'));
server = createServer((request, response) => {
if (request.url === '/graphql') {
creates++;
authorization = request.headers.authorization;
creationStarted.resolve(request);
request.setEncoding('utf8');
let body = '';
request.on('data', chunk => (body += chunk));
request.on('end', () => creationBodies.push(body));
request.on('end', () => {
if (stallCreation) {
return;
}
response.setHeader('Content-Type', 'application/json');
response.end(
JSON.stringify({
data: {
deviceRunSession: {
createArtifactUploadSession: {
uploadSession: { url: `${url}/artifact`, headers: {} },
},
},
},
})
);
});
} else {
handle(request, response);
}
});
await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve));
const address = server.address();
if (!address || typeof address === 'string') {
throw new Error('Missing test server address');
}
url = `http://127.0.0.1:${address.port}`;
ctx = {
graphqlClient: new Client({
url: `${url}/graphql`,
exchanges: [fetchExchange],
fetchOptions: { headers: { Authorization: 'Bearer test-auth' } },
}),
} as CustomBuildContext;
});
afterEach(async () => {
server.closeAllConnections();
await new Promise<void>(resolve => server.close(() => resolve()));
await rm(directory, { recursive: true, force: true });
});
it('replays the full file after a transient PUT failure without recreating the upload session', async () => {
const bodies: string[] = [];
handle = (request, response) => {
const chunks: string[] = [];
request.setEncoding('utf8');
request.on('data', chunk => chunks.push(chunk));
request.on('end', () => {
bodies.push(chunks.join(''));
response.statusCode = bodies.length === 1 ? 503 : 200;
response.end();
});
};
const file = path.join(directory, 'recording.mp4');
const payload = '*'.repeat(128 * 1024);
await writeFile(file, payload);
const streams: Readable[] = [];
const open = () => {
const stream = createReadStream(file);
streams.push(stream);
return stream;
};
await uploadDeviceRunSessionArtifactAsync(ctx, {
...metadata,
size: payload.length,
stream: open(),
reopenStream: open,
});
expect(creates).toBe(1);
expect(authorization).toBe('Bearer test-auth');
expect(bodies).toEqual([payload, payload]);
expect(streams).toHaveLength(2);
expect(streams.every(stream => stream.destroyed)).toBe(true);
});
it('keeps a slow PUT alive while its body is still flowing', async () => {
jest.useFakeTimers({ doNotFake: ['nextTick', 'setImmediate', 'queueMicrotask'] });
let received = 0;
let onChunk = () => {};
handle = (request, response) => {
request.on('data', chunk => {
received += chunk.length;
onChunk();
});
request.on('end', () => {
response.statusCode = 200;
response.end();
});
};
const waitForBytes = (total: number) =>
new Promise<void>(resolve => {
onChunk = () => {
if (received >= total) {
resolve();
}
};
onChunk();
});
const stream = new Readable({ read() {} });
const upload = uploadDeviceRunSessionArtifactAsync(ctx, { ...metadata, size: 3, stream });
const settled = expect(upload).resolves.toBeUndefined();
stream.push('a');
await waitForBytes(1);
await jest.advanceTimersByTimeAsync(60_000);
stream.push('b');
await waitForBytes(2);
await jest.advanceTimersByTimeAsync(60_000);
stream.push('c');
stream.push(null);
await settled;
expect(received).toBe(3);
jest.useRealTimers();
});
it('retries a PUT that stalled past the idle deadline with a fresh stream', async () => {
jest.useFakeTimers({ doNotFake: ['nextTick', 'setImmediate', 'queueMicrotask'] });
let puts = 0;
const firstBodyRead = deferred<void>();
handle = (request, response) => {
puts++;
request.resume();
request.on('end', () => {
if (puts === 1) {
firstBodyRead.resolve();
return;
}
response.statusCode = 200;
response.end();
});
};
const streams: Readable[] = [];
const open = () => {
const stream = Readable.from('recording');
streams.push(stream);
return stream;
};
const upload = uploadDeviceRunSessionArtifactAsync(ctx, {
...metadata,
size: 9,
stream: open(),
reopenStream: open,
});
const settled = expect(upload).resolves.toBeUndefined();
await firstBodyRead.promise;
await jest.advanceTimersByTimeAsync(90_000);
await jest.advanceTimersByTimeAsync(500);
await settled;
expect(puts).toBe(2);
expect(streams).toHaveLength(2);
expect(streams.every(stream => stream.destroyed)).toBe(true);
jest.useRealTimers();
});
it('cancels an active PUT socket and destroys its source without retrying', async () => {
const started = deferred<void>();
const closed = deferred<void>();
let puts = 0;
handle = request => {
puts++;
request.socket.once('close', () => closed.resolve());
request.on('data', () => started.resolve());
};
const controller = new AbortController();
const stream = new Readable({ read() {} });
stream.push(Buffer.alloc(1024));
const reopenStream = jest.fn(() => Readable.from('unexpected retry'));
const upload = uploadDeviceRunSessionArtifactAsync(ctx, {
...metadata,
stream,
reopenStream,
signal: controller.signal,
});
const rejected = expect(upload).rejects.toThrow('test cancellation');
await started.promise;
controller.abort(new Error('test cancellation'));
await rejected;
await closed.promise;
expect(stream.destroyed).toBe(true);
expect(reopenStream).not.toHaveBeenCalled();
expect(puts).toBe(1);
});
it.each([200, 400, 503])(
'closes a stalled HTTP %s response before leaving or retrying the PUT',
async status => {
const sockets = new Set<Socket>();
let puts = 0;
let maxOpenSockets = 0;
handle = (request, response) => {
puts++;
sockets.add(request.socket);
maxOpenSockets = Math.max(maxOpenSockets, sockets.size);
request.socket.once('close', () => sockets.delete(request.socket));
request.resume();
request.on('end', () => {
response.writeHead(status, { 'Content-Length': '1000' });
response.flushHeaders();
response.write('partial response');
});
};
const upload = uploadDeviceRunSessionArtifactAsync(ctx, {
...metadata,
stream: Readable.from('recording'),
reopenStream: () => Readable.from('recording'),
});
if (status === 200) {
await expect(upload).resolves.toBeUndefined();
} else {
await expect(upload).rejects.toThrow(`HTTP ${status}`);
}
for (let i = 0; i < 100 && sockets.size > 0; i++) {
await delay(10);
}
expect(creates).toBe(1);
expect(puts).toBe(status === 503 ? 3 : 1);
expect(maxOpenSockets).toBe(1);
expect(sockets.size).toBe(0);
}
);
it('cancels urql session creation with auth intact and never starts a PUT', async () => {
stallCreation = true;
const puts = jest.fn();
handle = puts;
const controller = new AbortController();
const stream = Readable.from('recording');
const upload = uploadDeviceRunSessionArtifactAsync(ctx, {
...metadata,
stream,
signal: controller.signal,
});
const rejected = expect(upload).rejects.toThrow('test cancellation');
const request = await creationStarted.promise;
const closed = new Promise<void>(resolve => request.socket.once('close', resolve));
controller.abort(new Error('test cancellation'));
await rejected;
await closed;
expect(authorization).toBe('Bearer test-auth');
expect(creates).toBe(1);
expect(puts).not.toHaveBeenCalled();
expect(stream.destroyed).toBe(true);
});
it('says ffprobe is missing instead of calling an unlisted recording undecodable', async () => {
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const session = path.join(directory, 'session');
await mkdir(session);
await writeFile(path.join(session, 'recording.mp4.partial'), new Uint8Array(28));
await writeFile(
path.join(session, 'session.json'),
JSON.stringify({
udid: 'emulator-5554',
deviceName: 'Pixel',
runtimeDisplayName: 'Android 16',
status: 'recording',
recording: 'recording.mp4.partial',
})
);
const recordings = await findUnlistedDeviceScreenRecordingsAsync({
root: directory,
env: { ...process.env, PATH: path.join(directory, 'no-tools') } as BuildStepEnv,
logger,
});
expect(recordings).toEqual([]);
expect(logger.warn).toHaveBeenCalledWith(
'ffprobe is not available; cannot recover the unlisted recording session/recording.mp4.partial.'
);
});
it('recovers unlisted recordings from a killed host and flags the unfinished ones as partial', async () => {
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const env = process.env as BuildStepEnv;
const cut = path.join(directory, 'cut');
const empty = path.join(directory, 'empty');
const failed = path.join(directory, 'failed');
const done = path.join(directory, 'done');
await mkdir(cut);
await mkdir(empty);
await mkdir(failed);
await mkdir(done);
const manifest = {
udid: 'emulator-5554',
deviceName: 'Pixel',
runtimeDisplayName: 'Android 16',
status: 'recording',
recording: 'recording.mp4.partial',
firstFrameWallClock: { iso8601: '2026-07-10T10:00:00.000Z' },
width: 128,
height: 96,
};
execFileSync('ffmpeg', [
'-v',
'error',
'-f',
'lavfi',
'-i',
'color=c=red:s=128x96:r=1',
'-t',
'3',
'-c:v',
'libx264',
'-g',
'1',
'-pix_fmt',
'yuv420p',
'-movflags',
'frag_keyframe+empty_moov',
'-f',
'mp4',
path.join(cut, 'recording.mp4.partial'),
]);
await writeFile(path.join(cut, 'session.json'), JSON.stringify(manifest));
await writeFile(path.join(empty, 'recording.mp4.partial'), new Uint8Array(28));
await writeFile(path.join(empty, 'session.json'), JSON.stringify(manifest));
await copyFile(
path.join(cut, 'recording.mp4.partial'),
path.join(failed, 'recording.mp4.partial')
);
await writeFile(
path.join(failed, 'session.json'),
JSON.stringify({
...manifest,
status: 'failed',
error: 'Recording finalization exceeded 30000 ms.',
})
);
await copyFile(path.join(cut, 'recording.mp4.partial'), path.join(done, 'recording.mp4'));
await writeFile(
path.join(done, 'session.json'),
JSON.stringify({ ...manifest, status: 'complete', recording: 'recording.mp4' })
);
const recordings = await findUnlistedDeviceScreenRecordingsAsync({
root: directory,
env,
logger,
});
expect(recordings).toEqual([
{
udid: 'emulator-5554',
deviceName: 'Pixel',
runtimeDisplayName: 'Android 16',
directory: cut,
},
{
udid: 'emulator-5554',
deviceName: 'Pixel',
runtimeDisplayName: 'Android 16',
directory: done,
},
{
udid: 'emulator-5554',
deviceName: 'Pixel',
runtimeDisplayName: 'Android 16',
directory: failed,
},
]);
expect(logger.warn).toHaveBeenCalledWith(expect.stringContaining('empty/recording.mp4.partial'));
let putBytes = 0;
handle = (request, response) => {
request.on('data', chunk => (putBytes += chunk.length));
request.on('end', () => {
response.statusCode = 200;
response.end();
});
};
await expect(
uploadDeviceRunSessionScreenRecordingsAsync(ctx, {
logger,
deviceRunSessionId: 'drs-id',
recordings,
})
).resolves.toBe(true);
expect(putBytes).toBe(3 * (await stat(path.join(cut, 'recording.mp4.partial'))).size);
const inputs = creationBodies
.map(body => JSON.parse(body).variables.input)
.sort((a, b) => a.filename.localeCompare(b.filename));
expect(inputs.map(input => input.filename)).toEqual(['cut.mp4', 'done.mp4', 'failed.mp4']);
expect(inputs.map(input => input.name.includes(', partial)'))).toEqual([true, false, true]);
expect(inputs[1].metadata).not.toHaveProperty('partial');
expect(inputs[0].metadata).toMatchObject({
partial: true,
partialReason: 'The Device Hub stopped before it could finalize the recording.',
width: 128,
height: 96,
});
expect(inputs[2].metadata).toMatchObject({
partial: true,
partialReason: 'Recording finalization exceeded 30000 ms.',
});
});
@@ -0,0 +1,446 @@
import type { bunyan } from '@expo/logger';
import { BuildRuntimePlatform, type BuildStepEnv } from '@expo/steps';
import * as ngrok from '@ngrok/ngrok';
import { access, mkdir, rm, writeFile } from 'node:fs/promises';
import path from 'node:path';
import type { CustomBuildContext } from '../../../customBuildContext';
import { Sentry } from '../../../sentry';
import { turtleFetch } from '../../../utils/turtleFetch';
import {
findUnlistedDeviceScreenRecordingsAsync,
uploadDeviceRunSessionScreenRecordingsAsync,
} from '../deviceRunSessionScreenRecordings';
import { startDeviceSessionHostAsync } from '../deviceSessionHost';
import { spawnDetached } from '../remoteDeviceRunSession';
jest.mock('@ngrok/ngrok');
jest.mock('../../../sentry');
jest.mock('../serveSimMetricsRecorder', () => ({
readServeSimServersAsync: jest
.fn()
.mockResolvedValue([{ udid: 'emulator-5554', token: 'preview-token' }]),
}));
jest.mock('../../../utils/turtleFetch');
jest.mock('../remoteDeviceRunSession', () => ({
...jest.requireActual('../remoteDeviceRunSession'),
ensureFfmpegInstalledOnceAsync: jest.fn(),
fetchWebPreviewTurnArgsAsync: jest.fn().mockResolvedValue([]),
spawnDetached: jest.fn(),
}));
jest.mock('../deviceRunSessionScreenRecordings', () => ({
...jest.requireActual('../deviceRunSessionScreenRecordings'),
findUnlistedDeviceScreenRecordingsAsync: jest.fn(),
uploadDeviceRunSessionScreenRecordingsAsync: jest.fn(),
}));
function deferred<T>() {
let resolve!: (value: T) => void;
const promise = new Promise<T>(r => {
resolve = r;
});
return { promise, resolve };
}
const env = {
DEVICE_RUN_SESSION_ID: 'drs-id',
NGROK_AUTHTOKEN: 'token',
} as BuildStepEnv;
const ctx = {} as CustomBuildContext;
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const stopServer = jest.fn();
const closeTunnel = jest.fn();
const directories: string[] = [];
const baseDomain = 'preview.example.test';
async function startHostAsync() {
return await startDeviceSessionHostAsync(ctx, {
runtimePlatform: BuildRuntimePlatform.LINUX,
env,
logger,
timeoutMs: 10_000,
});
}
beforeEach(() => {
jest.clearAllMocks();
stopServer.mockResolvedValue(undefined);
closeTunnel.mockResolvedValue(undefined);
jest.mocked(uploadDeviceRunSessionScreenRecordingsAsync).mockReset().mockResolvedValue(false);
jest.mocked(findUnlistedDeviceScreenRecordingsAsync).mockReset().mockResolvedValue([]);
jest.mocked(spawnDetached).mockImplementation(options => {
const flag = options.args.indexOf('--android-recording-directory');
if (flag >= 0) {
directories.push(options.args[flag + 1]);
}
return {
pid: undefined,
getOutput: () => '',
getExitError: () => undefined,
stopAsync: stopServer,
};
});
jest.mocked(ngrok.forward).mockResolvedValue({
url: () => 'https://public.example.test',
close: closeTunnel,
} as never);
jest.mocked(turtleFetch).mockImplementation(
async () =>
({
ok: true,
json: async () => ({ status: 'ready', device: 'emulator-5554' }),
}) as Awaited<ReturnType<typeof turtleFetch>>
);
});
afterEach(async () => {
jest.useRealTimers();
await Promise.all(
directories.splice(0).map(directory => rm(directory, { recursive: true, force: true }))
);
});
it('records by default with no preview and finalizes before process stop, uploading exactly once', async () => {
const host = await startHostAsync();
expect(ngrok.forward).not.toHaveBeenCalled();
const directory = directories[0];
const recordings = [
{
udid: 'emulator-5554',
deviceName: 'Pixel',
runtimeDisplayName: 'Android 16',
directory: path.join(directory, 'session'),
},
];
await writeFile(path.join(directory, 'recordings.json'), JSON.stringify(recordings));
await mkdir(recordings[0].directory);
const started = deferred<void>();
const response = deferred<Awaited<ReturnType<typeof turtleFetch>>>();
jest.mocked(turtleFetch).mockImplementationOnce(async () => {
started.resolve();
return await response.promise;
});
const finishing = host.finishAsync();
expect(host.finishAsync()).toBe(finishing);
await started.promise;
expect(stopServer).not.toHaveBeenCalled();
expect(uploadDeviceRunSessionScreenRecordingsAsync).not.toHaveBeenCalled();
const token =
jest.mocked(spawnDetached).mock.calls[0][0].env.EXPO_DEVICE_HUB_RECORDING_CONTROL_TOKEN;
expect(token).toMatch(/^[a-f0-9]{64}$/);
expect(turtleFetch).toHaveBeenLastCalledWith(
expect.stringMatching(/\/_eas\/android-recording\/stop$/),
'POST',
{
headers: { Authorization: `Bearer ${token}` },
retries: 0,
shouldThrowOnNotOk: false,
signal: expect.any(AbortSignal),
}
);
response.resolve({ ok: true } as Awaited<ReturnType<typeof turtleFetch>>);
await finishing;
await host.finishAsync();
expect(stopServer).toHaveBeenCalledTimes(1);
expect(uploadDeviceRunSessionScreenRecordingsAsync).toHaveBeenCalledTimes(1);
expect(uploadDeviceRunSessionScreenRecordingsAsync).toHaveBeenCalledWith(ctx, {
logger,
deviceRunSessionId: 'drs-id',
recordings,
});
expect(stopServer.mock.invocationCallOrder[0]).toBeLessThan(
jest.mocked(uploadDeviceRunSessionScreenRecordingsAsync).mock.invocationCallOrder[0]
);
await expect(host.openPreviewAsync({ baseDomain })).rejects.toThrow(
'after session host finalization'
);
});
it('coalesces preview opens and allows close/reopen without ending capture', async () => {
const host = await startHostAsync();
const opening = host.openPreviewAsync({ baseDomain });
expect(host.openPreviewAsync({ baseDomain })).toBe(opening);
const first = await opening;
const closing = first.closeAsync();
expect(first.closeAsync()).toBe(closing);
await closing;
const secondClose = jest.fn().mockResolvedValue(undefined);
jest.mocked(ngrok.forward).mockResolvedValueOnce({
url: () => 'https://replacement.example.test',
close: secondClose,
} as never);
const second = await host.openPreviewAsync({ baseDomain });
await first.closeAsync();
expect(await host.openPreviewAsync({ baseDomain })).toBe(second);
expect(secondClose).not.toHaveBeenCalled();
expect(spawnDetached).toHaveBeenCalledTimes(1);
expect(stopServer).not.toHaveBeenCalled();
expect(uploadDeviceRunSessionScreenRecordingsAsync).not.toHaveBeenCalled();
expect(jest.mocked(turtleFetch).mock.calls.some(([, method]) => method === 'POST')).toBe(false);
await host.finishAsync();
expect(closeTunnel).toHaveBeenCalledTimes(1);
expect(secondClose).toHaveBeenCalledTimes(1);
expect(stopServer).toHaveBeenCalledTimes(1);
});
it('retries a failed tunnel without restarting the host', async () => {
const host = await startHostAsync();
jest.mocked(ngrok.forward).mockRejectedValueOnce(new Error('tunnel unavailable'));
await expect(host.openPreviewAsync({ baseDomain })).rejects.toThrow('tunnel unavailable');
expect(stopServer).not.toHaveBeenCalled();
await host.openPreviewAsync({ baseDomain });
expect(spawnDetached).toHaveBeenCalledTimes(1);
await host.finishAsync();
expect(closeTunnel).toHaveBeenCalledTimes(1);
});
it('drains an in-flight tunnel when finishing and rejects new opens', async () => {
const host = await startHostAsync();
const tunnel = deferred<Awaited<ReturnType<typeof ngrok.forward>>>();
jest.mocked(ngrok.forward).mockReturnValueOnce(tunnel.promise);
const opening = host.openPreviewAsync({ baseDomain });
const finishing = host.finishAsync();
await expect(host.openPreviewAsync({ baseDomain })).rejects.toThrow(
'after session host finalization'
);
tunnel.resolve({ url: () => 'https://late.example.test', close: closeTunnel } as never);
await expect(opening).rejects.toThrow('finalized while the preview was opening');
await finishing;
expect(closeTunnel).toHaveBeenCalledTimes(1);
expect(stopServer).toHaveBeenCalledTimes(1);
});
it('keeps cleanup and upload best-effort when tunnel close and finalization fail', async () => {
jest.mocked(spawnDetached).mockImplementationOnce(options => {
directories.push(options.args[options.args.indexOf('--android-recording-directory') + 1]);
return {
pid: undefined,
getOutput: () => '[serve-emu] emulator-5554 capture error: scrcpy exited with code 255',
getExitError: () => undefined,
stopAsync: stopServer,
};
});
const host = await startHostAsync();
await writeFile(path.join(directories[0], 'recordings.json'), '[]');
await host.openPreviewAsync({ baseDomain });
closeTunnel.mockRejectedValueOnce(new Error('tunnel close failure'));
jest.mocked(turtleFetch).mockResolvedValueOnce({
ok: false,
status: 500,
text: async () => 'Error: scrcpy exited with code 255',
} as Awaited<ReturnType<typeof turtleFetch>>);
await host.finishAsync();
expect(stopServer).toHaveBeenCalledTimes(1);
expect(logger.warn).toHaveBeenCalledWith(
{
err: expect.objectContaining({
message: expect.stringContaining('HTTP 500: Error: scrcpy exited with code 255'),
}),
},
'Could not finalize Android recording before shutdown.'
);
expect(Sentry.capture).toHaveBeenCalledWith(
'Could not finalize Android recording before shutdown',
expect.any(Error),
{ level: 'warning' }
);
expect(uploadDeviceRunSessionScreenRecordingsAsync).toHaveBeenCalledTimes(1);
expect(logger.warn).toHaveBeenCalledWith(
{ hostOutput: '[serve-emu] emulator-5554 capture error: scrcpy exited with code 255' },
'Session host output around the recording failure.'
);
});
it('logs the reason and skips upload and output dump when the Hub never recorded', async () => {
jest.mocked(spawnDetached).mockImplementationOnce(options => {
directories.push(options.args[options.args.indexOf('--android-recording-directory') + 1]);
return {
pid: undefined,
getOutput: () => '[serve-emu] Android recording skipped',
getExitError: () => undefined,
stopAsync: stopServer,
};
});
const host = await startHostAsync();
jest.mocked(turtleFetch).mockResolvedValueOnce({
ok: false,
status: 409,
text: async () => 'Android recording requires exactly one booted emulator; found 0.',
} as Awaited<ReturnType<typeof turtleFetch>>);
await host.finishAsync();
expect(stopServer).toHaveBeenCalledTimes(1);
expect(logger.warn).toHaveBeenCalledWith(
'Android recording was not captured: Android recording requires exactly one booted emulator; found 0.'
);
expect(uploadDeviceRunSessionScreenRecordingsAsync).not.toHaveBeenCalled();
expect(logger.warn).not.toHaveBeenCalledWith(
expect.anything(),
'Session host output around the recording failure.'
);
});
it('rolls back failed host startup without replacing the original error', async () => {
stopServer.mockRejectedValueOnce(new Error('stop failure'));
await expect(
startDeviceSessionHostAsync(ctx, {
runtimePlatform: BuildRuntimePlatform.LINUX,
env,
logger,
timeoutMs: 0,
})
).rejects.toThrow('Timed out waiting');
expect(stopServer).toHaveBeenCalledTimes(1);
expect(ngrok.forward).not.toHaveBeenCalled();
expect(jest.mocked(turtleFetch).mock.calls.some(([, method]) => method === 'POST')).toBe(false);
expect(uploadDeviceRunSessionScreenRecordingsAsync).not.toHaveBeenCalled();
expect(logger.warn).not.toHaveBeenCalledWith(
expect.anything(),
expect.stringMatching(/finalize|upload/)
);
});
async function writeRecordingDescriptorAsync(directory: string) {
const child = path.join(directory, 'session');
await mkdir(child);
await writeFile(
path.join(directory, 'recordings.json'),
JSON.stringify([
{
udid: 'emulator-5554',
deviceName: 'Pixel',
runtimeDisplayName: 'Android',
directory: child,
},
])
);
}
it.each([true, false])(
'removes its recording root only when every upload succeeds (%s)',
async uploaded => {
const host = await startHostAsync();
const directory = directories[0];
await writeRecordingDescriptorAsync(directory);
jest.mocked(uploadDeviceRunSessionScreenRecordingsAsync).mockResolvedValueOnce(uploaded);
await host.finishAsync();
if (uploaded) {
await expect(access(directory)).rejects.toMatchObject({ code: 'ENOENT' });
} else {
await expect(access(directory)).resolves.toBeUndefined();
}
}
);
it('finishes despite a stalled tunnel close', async () => {
const host = await startHostAsync();
await host.openPreviewAsync({ baseDomain });
const closing = deferred<void>();
closeTunnel.mockReturnValueOnce(closing.promise);
jest.useFakeTimers();
const finishing = host.finishAsync();
await jest.advanceTimersByTimeAsync(5_000);
await finishing;
expect(stopServer).toHaveBeenCalledTimes(1);
expect(logger.warn).toHaveBeenCalledWith(expect.anything(), expect.stringContaining('deadline'));
closing.resolve();
});
it('aborts a stalled finalization request before stopping the host', async () => {
const host = await startHostAsync();
const response = deferred<Awaited<ReturnType<typeof turtleFetch>>>();
jest.mocked(turtleFetch).mockReturnValueOnce(response.promise);
jest.useFakeTimers();
const finishing = host.finishAsync();
await jest.advanceTimersByTimeAsync(60_000);
await finishing;
const options = jest.mocked(turtleFetch).mock.calls.at(-1)?.[2];
expect(options?.signal?.aborted).toBe(true);
expect(stopServer).toHaveBeenCalledTimes(1);
response.resolve({ ok: true } as Awaited<ReturnType<typeof turtleFetch>>);
});
it('uploads the partial recording a killed host left behind and then removes the root', async () => {
const host = await startHostAsync();
const directory = directories[0];
const child = path.join(directory, 'session');
await mkdir(child);
await writeFile(path.join(directory, 'recordings.json'), '[]');
const partial = [
{ udid: 'emulator-5554', deviceName: 'Pixel', runtimeDisplayName: 'Android', directory: child },
];
jest.mocked(findUnlistedDeviceScreenRecordingsAsync).mockResolvedValueOnce(partial);
jest.mocked(uploadDeviceRunSessionScreenRecordingsAsync).mockResolvedValueOnce(true);
await host.finishAsync();
expect(findUnlistedDeviceScreenRecordingsAsync).toHaveBeenCalledWith({
root: directory,
env,
logger,
});
expect(uploadDeviceRunSessionScreenRecordingsAsync).toHaveBeenCalledWith(ctx, {
logger,
deviceRunSessionId: 'drs-id',
recordings: partial,
});
await expect(access(directory)).rejects.toMatchObject({ code: 'ENOENT' });
});
it('rejects descriptors outside its recording root', async () => {
const host = await startHostAsync();
const directory = directories[0];
await writeFile(
path.join(directory, 'recordings.json'),
JSON.stringify([
{
udid: 'emulator-5554',
deviceName: 'Pixel',
runtimeDisplayName: 'Android',
directory: path.dirname(directory),
},
])
);
await host.finishAsync();
expect(uploadDeviceRunSessionScreenRecordingsAsync).not.toHaveBeenCalled();
await expect(access(directory)).resolves.toBeUndefined();
});
it('retains recordings without uploading if the host cannot be stopped', async () => {
const host = await startHostAsync();
const directory = directories[0];
await writeRecordingDescriptorAsync(directory);
stopServer.mockRejectedValueOnce(new Error('process stop failed'));
await host.finishAsync();
expect(uploadDeviceRunSessionScreenRecordingsAsync).not.toHaveBeenCalled();
await expect(access(directory)).resolves.toBeUndefined();
});
it('closes a listener that arrives after finish has already timed out waiting for it', async () => {
const host = await startHostAsync();
const tunnel = deferred<Awaited<ReturnType<typeof ngrok.forward>>>();
jest.mocked(ngrok.forward).mockReturnValueOnce(tunnel.promise);
const opening = host.openPreviewAsync({ baseDomain });
const rejected = expect(opening).rejects.toThrow('finalized while the preview was opening');
jest.useFakeTimers();
const finishing = host.finishAsync();
await jest.advanceTimersByTimeAsync(5_000);
await finishing;
expect(stopServer).toHaveBeenCalledTimes(1);
tunnel.resolve({ url: () => 'https://late.example.test', close: closeTunnel } as never);
await rejected;
expect(closeTunnel).toHaveBeenCalledTimes(1);
});
it('leaves iOS recording to its existing build steps', async () => {
const host = await startDeviceSessionHostAsync(ctx, {
runtimePlatform: BuildRuntimePlatform.DARWIN,
env,
logger,
timeoutMs: 10_000,
});
const options = jest.mocked(spawnDetached).mock.calls[0][0];
expect(options.args).not.toContain('--android-recording-directory');
expect(options.env.EXPO_DEVICE_HUB_RECORDING_CONTROL_TOKEN).toBeUndefined();
await host.finishAsync();
expect(jest.mocked(turtleFetch).mock.calls.some(([, method]) => method === 'POST')).toBe(false);
expect(uploadDeviceRunSessionScreenRecordingsAsync).not.toHaveBeenCalled();
expect(stopServer).toHaveBeenCalledTimes(1);
});
@@ -454,6 +454,7 @@ describe(startChiselServerAsync, () => {
jest.mocked(spawnDetached).mockReturnValue({
pid: 12345,
getOutput: () => output,
getExitError: () => undefined,
stopAsync,
});
});
@@ -3,6 +3,9 @@ import { bunyan } from '@expo/logger';
import { BuildRuntimePlatform, BuildStepEnv, BuildStepInputValueTypeName } from '@expo/steps';
import spawn from '@expo/turtle-spawn';
import * as ngrok from '@ngrok/ngrok';
import { EventEmitter } from 'node:events';
import fs from 'node:fs/promises';
import path from 'node:path';
import {
clearTimeout as clearTimeoutCallback,
setTimeout as setTimeoutCallback,
@@ -15,24 +18,28 @@ import { Sentry } from '../../../sentry';
import { turtleFetch } from '../../../utils/turtleFetch';
import { readServeSimServersAsync } from '../serveSimMetricsRecorder';
import { sleepAsync } from '../../../utils/retry';
import { uploadDeviceRunSessionScreenRecordingsAsync } from '../deviceRunSessionScreenRecordings';
import {
createExpoDeviceHubArgs,
createServeSimArgs,
createServeSimLaunchInputProviders,
describeServeSimLaunch,
ensureFfmpegInstalledOnceAsync,
fetchWebPreviewTurnArgsAsync,
parseServeSimLaunchInputs,
simulatorPreviewPageUrl,
startDeviceWebPreviewWithTunnelAsync,
startExpoDeviceHubWithTunnelAsync,
spawnDetached,
startNgrokTunnelAsync,
turnIceServersToWebPreviewArgs,
waitForDeviceRunSessionStoppedAsync,
} from '../remoteDeviceRunSession';
import {
createExpoDeviceHubArgs,
createServeSimArgs,
simulatorPreviewPageUrl,
startDeviceSessionHostAsync,
waitForWebPreviewReadyAsync,
websiteOrigin,
websiteOriginServeSimArgs,
} from '../remoteDeviceRunSession';
} from '../deviceSessionHost';
jest.mock('@ngrok/ngrok');
jest.mock('node:timers');
@@ -41,6 +48,10 @@ jest.mock('../../../utils/turtleFetch');
jest.mock('../../../utils/retry', () => ({ sleepAsync: jest.fn() }));
jest.mock('../../../sentry');
jest.mock('@expo/turtle-spawn');
jest.mock('../deviceRunSessionScreenRecordings', () => ({
...jest.requireActual('../deviceRunSessionScreenRecordings'),
uploadDeviceRunSessionScreenRecordingsAsync: jest.fn(),
}));
// Spyable so a test can stand in for the serve-sim state directory, which a local serve-sim owns.
jest.mock('../serveSimMetricsRecorder', () => {
const actual = jest.requireActual('../serveSimMetricsRecorder');
@@ -56,6 +67,69 @@ function createLoggerMock(): bunyan {
} as unknown as bunyan;
}
describe(spawnDetached, () => {
function mockProcess(promise: Promise<unknown>): void {
jest
.mocked(spawn)
.mockReturnValue(
Object.assign(promise, { child: { pid: 1234, unref: jest.fn(), once: jest.fn() } }) as never
);
}
it('observes exit without waiting for inherited output pipes to close', () => {
const child = Object.assign(new EventEmitter(), { pid: 1234, unref: jest.fn() });
jest.mocked(spawn).mockReturnValue(Object.assign(new Promise(() => {}), { child }) as never);
const handle = spawnDetached({ command: 'server', args: [], env: {} });
child.emit('exit', 1, null);
expect(handle.getExitError()?.message).toContain('code 1');
expect(child.listenerCount('exit')).toBe(0);
});
it('does not report an exit while the process is running', () => {
mockProcess(new Promise(() => {}));
const handle = spawnDetached({ command: 'server', args: [], env: {} });
expect(handle.getExitError()).toBeUndefined();
});
it('reports a clean exit so startup does not keep waiting', async () => {
const completion = Promise.resolve();
mockProcess(completion);
const handle = spawnDetached({ command: 'server', args: [], env: {} });
await completion;
expect(handle.getExitError()?.message).toContain('code 0');
});
it.each([0, 1])('observes a real subprocess exiting with code %s', async exitCode => {
const actualSpawn =
jest.requireActual<typeof import('@expo/turtle-spawn')>('@expo/turtle-spawn').default;
let completion: ReturnType<typeof actualSpawn>;
jest.mocked(spawn).mockImplementationOnce((...args) => {
completion = actualSpawn(...args);
return completion;
});
const handle = spawnDetached({
command: process.execPath,
args: ['-e', `console.error('startup-marker'); process.exit(${exitCode});`],
env: {},
});
await completion!.catch(() => {});
expect(handle.getExitError()).toBeInstanceOf(Error);
expect(handle.getOutput()).toContain('startup-marker');
});
it.each(['spawn ENOENT', 'server exited with code 1', 'server terminated by SIGTERM'])(
'preserves the process failure: %s',
async message => {
const error = new Error(message);
const completion = Promise.reject(error);
mockProcess(completion);
const handle = spawnDetached({ command: 'server', args: [], env: {} });
await completion.catch(() => {});
expect(handle.getExitError()).toBe(error);
}
);
});
function createCtxMock(): CustomBuildContext {
return {
env: {
@@ -382,6 +456,12 @@ describe(createServeSimArgs, () => {
});
describe(createExpoDeviceHubArgs, () => {
it('opts in to recording only when a directory is provided', () => {
expect(createExpoDeviceHubArgs({ port: 4321 })).not.toContain('--android-recording-directory');
expect(
createExpoDeviceHubArgs({ port: 4321, recordingDirectory: '/tmp/recordings' }).slice(-2)
).toEqual(['--android-recording-directory', '/tmp/recordings']);
});
it('uses the latest Expo package and applies the EAS Android streaming policy', () => {
expect(
createExpoDeviceHubArgs({
@@ -531,7 +611,7 @@ describe(startNgrokTunnelAsync, () => {
});
});
describe(startDeviceWebPreviewWithTunnelAsync, () => {
describe(startDeviceSessionHostAsync, () => {
const baseDomain = 'eas-simulator.ngrok.dev';
const turnArgs = [
'--turn-url',
@@ -548,6 +628,7 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
} as unknown as BuildStepEnv;
beforeEach(() => {
jest.mocked(uploadDeviceRunSessionScreenRecordingsAsync).mockReset();
jest.mocked(spawn).mockReset();
jest.mocked(ngrok.forward).mockReset();
jest.mocked(turtleFetch).mockReset();
@@ -559,6 +640,7 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
child: {
pid: undefined,
unref: jest.fn(),
once: jest.fn(),
},
});
jest.mocked(spawn).mockReturnValue(spawnPromise as never);
@@ -600,14 +682,14 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
.mockReturnValueOnce(Promise.resolve({}) as unknown as ReturnType<typeof spawn>)
.mockReturnValueOnce(Promise.resolve({}) as unknown as ReturnType<typeof spawn>);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
const host = await startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.LINUX,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
packageVersion,
});
const preview = await host.openPreviewAsync({ baseDomain });
const spawnCalls = jest.mocked(spawn).mock.calls;
expect(spawnCalls[0]).toEqual(['ffmpeg', ['-version'], { env }]);
@@ -634,11 +716,17 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
const port = Number(args[args.indexOf('--port') + 1]);
expect(port).toBeGreaterThan(0);
expect(command).toBe('npx');
expect(args).toEqual(['--yes', ...createExpoDeviceHubArgs({ port, turnArgs, packageVersion })]);
expect(args).toContain('--android-recording-directory');
const recordingDirectory = args[args.indexOf('--android-recording-directory') + 1];
expect(args).toEqual([
'--yes',
...createExpoDeviceHubArgs({ port, turnArgs, packageVersion, recordingDirectory }),
]);
expect(ngrok.forward).toHaveBeenCalledWith(expect.objectContaining({ addr: port }));
expect(preview.apiUrl).toBe('https://android-preview.example.test');
await preview.stopAsync();
await host.finishAsync();
await fs.rm(recordingDirectory, { recursive: true, force: true });
expect(close).toHaveBeenCalledTimes(1);
});
@@ -648,13 +736,13 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
close: jest.fn().mockResolvedValue(undefined),
} as never);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
const host = await startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
});
const preview = await host.openPreviewAsync({ baseDomain });
expect(preview.previewToken).toBe('tok-1');
expect(preview.apiUrl).toBe('https://preview.example.test');
@@ -677,13 +765,13 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
close: jest.fn().mockResolvedValue(undefined),
} as never);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
const host = await startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env: { ...env, EXPO_STAGING: '1' },
logger: createLoggerMock(),
timeoutMs: 10_000,
});
const preview = await host.openPreviewAsync({ baseDomain });
expect(preview.previewPageUrl).toMatch(
/^https:\/\/staging\.expo\.dev\/simulator-preview\/[a-f0-9]{32}$/
@@ -702,9 +790,8 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
.mockResolvedValue([{ udid: 'device-id', url: 'http://127.0.0.1:1' }]);
await expect(
startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
@@ -720,13 +807,13 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
close: jest.fn().mockResolvedValue(undefined),
} as never);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
const host = await startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.LINUX,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
});
const preview = await host.openPreviewAsync({ baseDomain });
expect(preview.previewToken).toBeUndefined();
const [, args] = jest.mocked(spawn).mock.calls[0];
@@ -741,14 +828,14 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
close,
} as never);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
const host = await startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
packageVersion,
});
const preview = await host.openPreviewAsync({ baseDomain });
const [command, args] = jest.mocked(spawn).mock.calls[0];
const port = Number(args[args.indexOf('--port') + 1]);
@@ -767,33 +854,7 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
expect(ngrok.forward).toHaveBeenCalledWith(expect.objectContaining({ addr: port }));
expect(preview.apiUrl).toBe('https://ios-preview.example.test');
await preview.stopAsync();
expect(close).toHaveBeenCalledTimes(1);
});
it('does not install ffmpeg before starting expo-device-hub outside Linux', async () => {
const close = jest.fn().mockResolvedValue(undefined);
jest.mocked(ngrok.forward).mockResolvedValue({
url: () => 'https://android-preview.example.test',
close,
} as never);
const preview = await startExpoDeviceHubWithTunnelAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
});
expect(jest.mocked(spawn).mock.calls[0][0]).toBe('npx');
expect(jest.mocked(spawn)).not.toHaveBeenCalledWith(
'ffmpeg',
expect.anything(),
expect.anything()
);
await preview.stopAsync();
await host.finishAsync();
expect(close).toHaveBeenCalledTimes(1);
});
@@ -805,14 +866,14 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
} as never);
const bunEnv = { ...env, EAS_OVERRIDE_PACKAGE_MANAGER: 'bun' };
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
const host = await startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env: bunEnv,
logger: createLoggerMock(),
timeoutMs: 10_000,
packageVersion: '4.5.6',
});
const preview = await host.openPreviewAsync({ baseDomain });
const [command, args] = jest.mocked(spawn).mock.calls[0];
const port = Number(args[args.indexOf('--port') + 1]);
@@ -828,7 +889,7 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
}),
]);
await preview.stopAsync();
await host.finishAsync();
expect(close).toHaveBeenCalledTimes(1);
});
@@ -839,13 +900,13 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
close,
} as never);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
const host = await startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env: { ...env, EAS_FALLBACK_PACKAGE_MANAGER: 'bun' },
logger: createLoggerMock(),
timeoutMs: 10_000,
});
const preview = await host.openPreviewAsync({ baseDomain });
const [command, args] = jest.mocked(spawn).mock.calls[0];
const port = Number(args[args.indexOf('--port') + 1]);
@@ -860,7 +921,7 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
}),
]);
await preview.stopAsync();
await host.finishAsync();
expect(close).toHaveBeenCalledTimes(1);
});
@@ -870,9 +931,8 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
close: jest.fn().mockResolvedValue(undefined),
} as never);
await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
const host = await startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
@@ -880,6 +940,7 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
launchArgs: ['-EXDevMenuIsOnboardingFinished', '1'],
openUrl: 'exp://127.0.0.1:8081',
});
await host.finishAsync();
const [, args] = jest.mocked(spawn).mock.calls[0];
expect(args.slice(-8)).toEqual([
@@ -896,9 +957,8 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
it('refuses to launch an application on Linux, where expo-device-hub cannot', async () => {
await expect(
startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.LINUX,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
@@ -0,0 +1,25 @@
import type { BuildStepEnv } from '@expo/steps';
import { isProcessRunning, spawnDetached } from '../remoteDeviceRunSession';
jest.mock('@ngrok/ngrok');
it('resolves stopAsync only after a process that ignores SIGTERM is gone', async () => {
const handle = spawnDetached({
command: 'sh',
args: ['-c', 'trap "" TERM; echo ready; sleep 30 & wait'],
env: process.env as BuildStepEnv,
stopGracePeriodMs: 300,
});
const pid = handle.pid;
if (pid === undefined) {
throw new Error('spawn failed');
}
while (!handle.getOutput().includes('ready')) {
await new Promise(resolve => setTimeout(resolve, 20));
}
const startedAt = Date.now();
await handle.stopAsync();
expect(Date.now() - startedAt).toBeGreaterThanOrEqual(300);
expect(isProcessRunning(pid)).toBe(false);
});
@@ -0,0 +1,57 @@
import { type bunyan } from '@expo/logger';
import { Readable } from 'node:stream';
import { writeFile } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { type CustomBuildContext } from '../../../customBuildContext';
import { uploadDeviceRunSessionArtifactAsync } from '../deviceRunSessionArtifacts';
import { uploadServeSimLogsFileAsync } from '../serveSimLogsArtifacts';
jest.mock('../deviceRunSessionArtifacts');
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const ctx = {} as CustomBuildContext;
const filePath = path.join(os.tmpdir(), 'logs.ndjson');
const args = { deviceRunSessionId: 'session', udid: 'A', filePath, logger };
it('uploads NDJSON with explicit user-app scope and destroys the stream afterward', async () => {
await writeFile(filePath, '{"pid":42}\n');
await uploadServeSimLogsFileAsync(ctx, args);
expect(uploadDeviceRunSessionArtifactAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({
kind: 'simulator-log',
filename: 'app-logs.ndjson',
size: 11,
metadata: expect.objectContaining({ scope: 'user-apps', udid: 'A' }),
})
);
expect(
(jest.mocked(uploadDeviceRunSessionArtifactAsync).mock.calls[0][1].stream as Readable).destroyed
).toBe(true);
});
it('skips empty and missing files', async () => {
await uploadServeSimLogsFileAsync(ctx, args);
await writeFile(filePath, '');
await uploadServeSimLogsFileAsync(ctx, args);
expect(uploadDeviceRunSessionArtifactAsync).not.toHaveBeenCalled();
});
it('handles a file read error while upload allocation is pending without an unhandled error', async () => {
await writeFile(filePath, '{}\n');
jest
.mocked(uploadDeviceRunSessionArtifactAsync)
.mockImplementationOnce(async (_ctx, { stream, signal }) => {
const aborted = new Promise<void>((_resolve, reject) => {
signal!.addEventListener('abort', () => reject(signal!.reason), { once: true });
});
(stream as Readable).destroy(new Error('read failure'));
await aborted;
});
await expect(uploadServeSimLogsFileAsync(ctx, args)).resolves.toBeUndefined();
expect(logger.warn).toHaveBeenCalledWith(
expect.objectContaining({ err: expect.any(Error) }),
expect.stringContaining('Could not upload')
);
});
@@ -0,0 +1,204 @@
import { type bunyan } from '@expo/logger';
import { mkdtemp, readFile, rm } from 'node:fs/promises';
import { type Server, createServer } from 'node:http';
import { type AddressInfo } from 'node:net';
import os from 'node:os';
import path from 'node:path';
import { type CustomBuildContext } from '../../../customBuildContext';
import { uploadServeSimLogsFileAsync } from '../serveSimLogsArtifacts';
import { streamServeSimLogsToFileAsync } from '../serveSimLogsRecorder';
// This test exercises actual sockets, UTF-8 decoding, files, and PUT uploads.
// Only WWW's allocation response is substituted; it does not verify cloud storage.
jest.mock('../../../sentry');
jest.unmock('fs');
jest.unmock('node:fs');
jest.unmock('fs/promises');
jest.unmock('node:fs/promises');
jest.unmock('node-fetch');
const logger = {
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
debug: jest.fn(),
} as unknown as bunyan;
let directory: string;
let server: Server;
let baseUrl: string;
beforeEach(async () => {
directory = await mkdtemp(path.join(os.tmpdir(), 'simulator-logs-http-test-'));
server = createServer();
await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve));
baseUrl = `http://127.0.0.1:${(server.address() as AddressInfo).port}`;
});
afterEach(async () => {
server.closeAllConnections();
await new Promise<void>((resolve, reject) =>
server.close(error => (error ? reject(error) : resolve()))
);
await rm(directory, { recursive: true, force: true });
});
it('streams selected-device logs into NDJSON and uploads the exact bytes as a session artifact', async () => {
const record = JSON.stringify({ eventMessage: 'Hello 서울 🌲', process: 'MyApp' });
const expected = Buffer.from(record + '\n');
const envelope = JSON.stringify({ seq: 7, at: 1000, raw: record });
const wire = Buffer.from(
`: heartbeat\n\ndata: ${envelope}\n\ndata: invalid json\n\ndata: {"partial":`
);
const splitAt = wire.indexOf(Buffer.from('서울')) + 1;
let streamRequest: { url?: string; authorization?: string } | undefined;
let uploaded: Buffer | undefined;
let uploadedLength: string | undefined;
server.on('request', (request, response) => {
if (request.method === 'GET') {
streamRequest = { url: request.url, authorization: request.headers.authorization };
response.writeHead(200, {
'Content-Type': 'text/event-stream',
'X-Serve-Sim-Log-Scope': 'user-apps',
});
response.write(wire.subarray(0, splitAt));
setImmediate(() => response.end(wire.subarray(splitAt)));
return;
}
const chunks: Buffer[] = [];
request.on('data', chunk => chunks.push(Buffer.from(chunk)));
request.on('end', () => {
uploaded = Buffer.concat(chunks);
uploadedLength = request.headers['content-length'];
response.writeHead(200).end();
});
});
const filePath = path.join(directory, 'simulator.ndjson');
const result = await streamServeSimLogsToFileAsync({
serveSimUrl: baseUrl,
serveSimToken: 'local-test-token',
serveSimDevice: 'device-B',
filePath,
signal: new AbortController().signal,
logger,
});
expect(streamRequest).toEqual({
url: '/logs?envelope=true&scope=user-apps&device=device-B',
authorization: 'Bearer local-test-token',
});
expect(result).toEqual({
receivedData: true,
limitReached: false,
bytesWritten: expected.length,
lastSequence: 7,
});
expect(await readFile(filePath)).toEqual(expected);
const mutation = jest.fn().mockReturnValue({
toPromise: async () => ({
data: {
deviceRunSession: {
createArtifactUploadSession: {
uploadSession: {
url: `${baseUrl}/upload`,
headers: { 'Content-Length': String(expected.length) },
},
},
},
},
}),
});
await uploadServeSimLogsFileAsync(
{ graphqlClient: { mutation } } as unknown as CustomBuildContext,
{ deviceRunSessionId: 'session-test', udid: 'device-B', filePath, logger }
);
expect(mutation.mock.calls[0][1]).toEqual({
deviceRunSessionId: 'session-test',
input: {
name: 'App logs (device-B)',
filename: 'app-logs.ndjson',
kind: 'simulator-log',
metadata: {
__eas_type: 'simulator-log',
udid: 'device-B',
scope: 'user-apps',
source: 'serve-sim/logs',
},
size: expected.length,
},
});
expect(uploaded).toEqual(expected);
expect(uploadedLength).toBe(String(expected.length));
// Only the invalid JSON record is reported; the partial trailing record is not a record yet.
expect(logger.warn).toHaveBeenCalledTimes(1);
expect(logger.warn).toHaveBeenCalledWith(
{ malformedRecords: 1 },
expect.stringContaining('not valid JSON')
);
});
it('resumes a buffered stream without appending already persisted sequence numbers', async () => {
const urls: string[] = [];
const first = JSON.stringify({ eventMessage: 'first' });
const second = JSON.stringify({ eventMessage: 'second' });
const frame = (seq: number, raw: string): string =>
`data: ${JSON.stringify({ seq, at: seq * 1000, raw })}\n\n`;
server.on('request', (request, response) => {
urls.push(request.url!);
response.writeHead(200, {
'Content-Type': 'text/event-stream',
'X-Serve-Sim-Log-Scope': 'user-apps',
});
// Deliberately resend seq=1 on reconnect to exercise the client's guard too.
response.end(frame(1, first) + (urls.length > 1 ? frame(2, second) : ''));
});
const filePath = path.join(directory, 'resumed.ndjson');
const options = {
serveSimUrl: baseUrl,
serveSimDevice: 'device-A',
filePath,
signal: new AbortController().signal,
logger,
};
const initial = await streamServeSimLogsToFileAsync(options);
const resumed = await streamServeSimLogsToFileAsync({
...options,
since: initial.lastSequence,
});
expect(initial.lastSequence).toBe(1);
expect(resumed.lastSequence).toBe(2);
expect(resumed.bytesWritten).toBe(Buffer.byteLength(second + '\n'));
expect(urls).toEqual([
'/logs?envelope=true&scope=user-apps&device=device-A',
'/logs?envelope=true&scope=user-apps&since=1&device=device-A',
]);
expect(await readFile(filePath, 'utf8')).toBe(first + '\n' + second + '\n');
});
it('aborts a real open HTTP stream without waiting for the server to finish', async () => {
const controller = new AbortController();
let sawRequest: () => void = () => {};
const requestStarted = new Promise<void>(resolve => (sawRequest = resolve));
server.on('request', (_request, response) => {
response.writeHead(200, {
'Content-Type': 'text/event-stream',
'X-Serve-Sim-Log-Scope': 'user-apps',
});
response.write(': heartbeat\n\n');
sawRequest();
});
const streaming = streamServeSimLogsToFileAsync({
serveSimUrl: baseUrl,
filePath: path.join(directory, 'aborted.ndjson'),
signal: controller.signal,
logger,
});
await requestStarted;
controller.abort();
await expect(streaming).resolves.toEqual({
receivedData: false,
limitReached: false,
bytesWritten: 0,
});
});
@@ -0,0 +1,280 @@
import { type bunyan } from '@expo/logger';
import fetch from 'node-fetch';
import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { Readable } from 'node:stream';
import { setTimeout as delay } from 'node:timers/promises';
import { Sentry } from '../../../sentry';
import { ServeSimLogsRecorder, streamServeSimLogsToFileAsync } from '../serveSimLogsRecorder';
jest.mock('node-fetch');
jest.mock('../../../sentry');
jest.unmock('node:fs');
jest.unmock('node:fs/promises');
const { Response: FetchResponse } = jest.requireActual('node-fetch') as typeof import('node-fetch');
// A compatible serve-sim explicitly acknowledges the requested log scope.
class Response extends FetchResponse {
constructor(...args: ConstructorParameters<typeof FetchResponse>) {
super(...args);
this.headers.set('x-serve-sim-log-scope', 'user-apps');
}
}
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
let directory: string;
beforeEach(async () => {
jest.mocked(fetch).mockReset();
directory = await mkdtemp(path.join(os.tmpdir(), 'logs-test-'));
});
afterEach(async () => {
const files = await ServeSimLogsRecorder.finishAsync({ logger });
for (const file of files) {
await rm(path.dirname(file.filePath), { recursive: true, force: true });
}
await rm(directory, { recursive: true, force: true });
});
function record(options: Partial<Parameters<typeof streamServeSimLogsToFileAsync>[0]> = {}) {
return streamServeSimLogsToFileAsync({
serveSimUrl: 'http://localhost:1234',
filePath: path.join(directory, 'logs.ndjson'),
signal: new AbortController().signal,
logger,
...options,
});
}
it('preserves split UTF-8 and CRLF records, skips heartbeat and reports invalid JSON, and selects a device', async () => {
const input = Buffer.from(
':\r\n\r\ndata: {"message":"안녕"}\r\n\r\ndata: bad\n\ndata: {"pid":42}\n\n'
);
jest
.mocked(fetch)
.mockResolvedValue(new Response(Readable.from([...input].map(byte => Buffer.from([byte])))));
const result = await record({ serveSimToken: 'secret', serveSimDevice: 'device two' });
expect(result.receivedData).toBe(true);
expect(await readFile(path.join(directory, 'logs.ndjson'), 'utf8')).toBe(
'{"message":"안녕"}\n{"pid":42}\n'
);
expect(Sentry.capture).toHaveBeenCalledWith('serve-sim sent malformed simulator log records', {
extras: { malformedRecords: 1, serveSimDevice: 'device two' },
});
expect(fetch).toHaveBeenCalledWith(
'http://localhost:1234/logs?envelope=true&scope=user-apps&device=device+two',
expect.objectContaining({ headers: { Authorization: 'Bearer secret' } })
);
});
it('retains completed records when a connection errors', async () => {
const stream = Readable.from(
(async function* () {
yield Buffer.from('data: {"pid":42}\n\n');
await delay(10);
throw new Error('disconnect');
})()
);
jest.mocked(fetch).mockResolvedValue(new Response(stream));
expect(await record()).toEqual({ receivedData: true, bytesWritten: 11, limitReached: false });
expect(await readFile(path.join(directory, 'logs.ndjson'), 'utf8')).toBe('{"pid":42}\n');
});
it.each([undefined, 'all'])('refuses an unconfirmed user-app scope (%s)', async scope => {
const body = Readable.from(['data: {"eventMessage":"system noise"}\n\n']);
const response = new FetchResponse(body);
if (scope) {
response.headers.set('x-serve-sim-log-scope', scope);
}
jest.mocked(fetch).mockResolvedValue(response);
expect(await record()).toEqual({ receivedData: false, bytesWritten: 0, limitReached: false });
expect(body.destroyed).toBe(true);
await expect(readFile(path.join(directory, 'logs.ndjson'))).rejects.toMatchObject({
code: 'ENOENT',
});
expect(logger.warn).toHaveBeenCalledWith(expect.stringContaining('did not confirm user-app'));
});
it('aborts a quiet live stream and drains preceding writes before returning', async () => {
const controller = new AbortController();
const stream = new Readable({ read() {} });
stream.push('data: {"pid":42}\n\n');
jest.mocked(fetch).mockResolvedValue(new Response(stream));
const done = record({ signal: controller.signal });
await delay(20);
controller.abort();
expect((await done).receivedData).toBe(true);
expect(stream.destroyed).toBe(true);
expect(await readFile(path.join(directory, 'logs.ndjson'), 'utf8')).toBe('{"pid":42}\n');
});
it('enforces the byte limit without writing a truncated record', async () => {
jest
.mocked(fetch)
.mockResolvedValue(new Response(Readable.from(['data: {"pid":42}\n\ndata: {"pid":43}\n\n'])));
expect(await record({ maxBytes: 15 })).toEqual({
receivedData: true,
bytesWritten: 11,
limitReached: true,
});
expect(await readFile(path.join(directory, 'logs.ndjson'), 'utf8')).toBe('{"pid":42}\n');
});
it('drops an oversized partial SSE line and resumes at the next complete record', async () => {
jest
.mocked(fetch)
.mockResolvedValue(
new Response(
Readable.from(['data: ' + 'x'.repeat(1024 * 1024 + 1), 'tail\n\ndata: {"pid":42}\n\n'])
)
);
expect((await record()).bytesWritten).toBe(11);
});
it('warns and resolves on HTTP and file failures', async () => {
jest.mocked(fetch).mockResolvedValueOnce(new Response('missing', { status: 404 }));
expect((await record()).receivedData).toBe(false);
jest.mocked(fetch).mockResolvedValueOnce(new Response(Readable.from(['data: {}\n\n'])));
expect((await record({ filePath: '/missing-dir/log.ndjson' })).receivedData).toBe(false);
expect(logger.warn).toHaveBeenCalledTimes(2);
});
async function register(): Promise<string> {
const stateDir = path.join(directory, 'state');
await mkdir(stateDir);
await writeFile(
path.join(stateDir, 'server-A.json'),
JSON.stringify({ device: 'A', url: 'http://localhost:1234' })
);
return stateDir;
}
it('caps bytes across reconnects and finalizes only once even concurrently', async () => {
const stateDir = await register();
let seq = 0;
jest
.mocked(fetch)
.mockImplementation(
async () =>
new Response(
Readable.from([`data: ${JSON.stringify({ seq: ++seq, raw: '{"pid":42}' })}\n\n`])
)
);
await ServeSimLogsRecorder.startAsync({ logger, stateDir, pollIntervalMs: 5, maxBytes: 22 });
await delay(100);
expect(fetch).toHaveBeenCalledTimes(2);
const [first, second] = await Promise.all([
ServeSimLogsRecorder.finishAsync({ logger }),
ServeSimLogsRecorder.finishAsync({ logger }),
]);
expect(first).toHaveLength(1);
expect(second).toEqual([]);
expect(await readFile(first[0].filePath, 'utf8')).toBe('{"pid":42}\n{"pid":42}\n');
expect(await ServeSimLogsRecorder.finishAsync({ logger })).toEqual([]);
});
it('returns immediately before a server exists and discovers a later registration', async () => {
const stateDir = path.join(directory, 'state');
jest.mocked(fetch).mockImplementation(async () => new Response(Readable.from(['data: {}\n\n'])));
await ServeSimLogsRecorder.startAsync({ logger, stateDir, pollIntervalMs: 5 });
expect(fetch).not.toHaveBeenCalled();
await register();
await delay(30);
expect(await ServeSimLogsRecorder.finishAsync({ logger })).toHaveLength(1);
});
it('stops discovery and active streams at the session time limit', async () => {
const stateDir = await register();
const stream = new Readable({ read() {} });
stream.push('data: {}\n\n');
jest.mocked(fetch).mockResolvedValue(new Response(stream));
await ServeSimLogsRecorder.startAsync({ logger, stateDir, pollIntervalMs: 5, maxDurationMs: 30 });
await delay(80);
expect(stream.destroyed).toBe(true);
expect(fetch).toHaveBeenCalledTimes(1);
expect(await ServeSimLogsRecorder.finishAsync({ logger })).toHaveLength(1);
});
it('stops retrying after ten consecutive failures', async () => {
const stateDir = await register();
jest.mocked(fetch).mockImplementation(async () => new Response('bad', { status: 503 }));
await ServeSimLogsRecorder.startAsync({ logger, stateDir, pollIntervalMs: 5 });
await delay(150);
expect(fetch).toHaveBeenCalledTimes(10);
expect(await ServeSimLogsRecorder.finishAsync({ logger })).toEqual([]);
});
it('selects each registered device when two simulators share one server URL', async () => {
const stateDir = await register();
await writeFile(
path.join(stateDir, 'server-B.json'),
JSON.stringify({ device: 'B', url: 'http://localhost:1234' })
);
jest.mocked(fetch).mockImplementation(async () => new Response(Readable.from(['data: {}\n\n'])));
await ServeSimLogsRecorder.startAsync({ logger, stateDir, pollIntervalMs: 5, maxBytes: 3 });
await delay(50);
expect(fetch).toHaveBeenCalledWith(
'http://localhost:1234/logs?envelope=true&scope=user-apps&device=A',
expect.anything()
);
expect(fetch).toHaveBeenCalledWith(
'http://localhost:1234/logs?envelope=true&scope=user-apps&device=B',
expect.anything()
);
expect(fetch).toHaveBeenCalledTimes(2);
});
it('persists the envelope cursor only for written records and drops replayed sequences', async () => {
const frames = [1, 2, 3].map(
seq => `data: ${JSON.stringify({ seq, raw: JSON.stringify({ pid: seq }) })}\n\n`
);
jest.mocked(fetch).mockResolvedValue(new Response(Readable.from(frames)));
expect(await record({ since: 1, maxBytes: 10 })).toEqual({
receivedData: true,
bytesWritten: 10,
limitReached: true,
lastSequence: 2,
});
expect(await readFile(path.join(directory, 'logs.ndjson'), 'utf8')).toBe('{"pid":2}\n');
expect(fetch).toHaveBeenCalledWith(
'http://localhost:1234/logs?envelope=true&scope=user-apps&since=1',
expect.anything()
);
});
it('does not reconnect a legacy data stream without a replay cursor', async () => {
const stateDir = await register();
jest.mocked(fetch).mockImplementation(async () => new Response(Readable.from(['data: {}\n\n'])));
await ServeSimLogsRecorder.startAsync({ logger, stateDir, pollIntervalMs: 5 });
await delay(50);
expect(fetch).toHaveBeenCalledTimes(1);
expect(logger.warn).toHaveBeenCalledWith(expect.stringContaining('without a replay cursor'));
});
it('resets the replay cursor when a replacement server changes its token', async () => {
const stateDir = await register();
let calls = 0;
jest.mocked(fetch).mockImplementation(async (_url, options) => {
calls++;
if (calls === 1) {
await writeFile(
path.join(stateDir, 'server-A.json'),
JSON.stringify({
device: 'A',
url: 'http://localhost:1234',
token: 'new-token',
})
);
}
const seq = options?.headers ? 1 : 50;
return new Response(Readable.from([`data: ${JSON.stringify({ seq, raw: '{}' })}\n\n`]));
});
await ServeSimLogsRecorder.startAsync({ logger, stateDir, pollIntervalMs: 5, maxBytes: 6 });
await delay(50);
expect(fetch).toHaveBeenCalledTimes(2);
expect(String(jest.mocked(fetch).mock.calls[1][0])).not.toContain('since=');
expect(jest.mocked(fetch).mock.calls[1][1]?.headers).toEqual({
Authorization: 'Bearer new-token',
});
});
@@ -1,8 +1,15 @@
import { SystemError } from '@expo/eas-build-job';
import { graphql } from 'gql.tada';
import fetch, { Headers } from 'node-fetch';
import type { Readable } from 'node:stream';
import { setTimeout as delay } from 'node:timers/promises';
import { CustomBuildContext } from '../../customBuildContext';
import { graphqlAbortContext } from '../../utils/graphqlAbort';
import {
DeviceRunSessionTimeoutError,
withDeviceRunSessionTimeoutAsync,
} from './deviceRunSessionTimeout';
const CREATE_DEVICE_RUN_SESSION_ARTIFACT_UPLOAD_SESSION_MUTATION = graphql(`
mutation CreateDeviceRunSessionArtifactUploadSession(
@@ -20,82 +27,157 @@ const CREATE_DEVICE_RUN_SESSION_ARTIFACT_UPLOAD_SESSION_MUTATION = graphql(`
}
`);
type ArtifactMetadata = {
deviceRunSessionId: string;
artifactId: string;
name: string;
filename: string;
kind: string | undefined;
metadata?: Record<string, unknown>;
size: number;
};
type ArtifactSource = { stream: Readable; reopenStream?: () => Readable };
type ArtifactUploadSession = Awaited<
ReturnType<typeof createDeviceRunSessionArtifactUploadSessionAsync>
>;
export async function uploadDeviceRunSessionArtifactAsync(
ctx: CustomBuildContext,
{
deviceRunSessionId,
artifactId,
name,
filename,
kind,
metadata,
size,
stream,
}: {
deviceRunSessionId: string;
artifactId: string;
name: string;
filename: string;
kind: string | undefined;
metadata?: Record<string, unknown>;
size: number;
stream: NodeJS.ReadableStream;
}
options: ArtifactMetadata & ArtifactSource & { signal?: AbortSignal }
): Promise<void> {
const uploadSession = await createDeviceRunSessionArtifactUploadSessionAsync(ctx, {
deviceRunSessionId,
artifactId,
name,
filename,
kind,
metadata,
size,
});
const response = await fetch(uploadSession.url, {
method: 'PUT',
headers: new Headers(uploadSession.headers as Record<string, string>),
body: stream,
});
if (!response.ok) {
throw new SystemError(
`Failed to upload device run session artifact ${artifactId}: ${response.status} ${response.statusText}`,
{ cause: response }
try {
const uploadSession = await withDeviceRunSessionTimeoutAsync(
{ name: 'Artifact upload session creation', timeoutMs: 15_000, signal: options.signal },
async signal => await createDeviceRunSessionArtifactUploadSessionAsync(ctx, options, signal)
);
options.signal?.throwIfAborted();
await putArtifactWithRetriesAsync({
uploadSession,
source: options,
artifactId: options.artifactId,
signal: options.signal,
});
} finally {
// One-shot callers may have opened their stream before upload-session creation failed.
options.stream.destroy();
}
}
async function putArtifactWithRetriesAsync({
uploadSession,
source,
artifactId,
signal,
}: {
uploadSession: ArtifactUploadSession;
source: ArtifactSource;
artifactId: string;
signal?: AbortSignal;
}): Promise<void> {
const attempts = source.reopenStream ? 3 : 1;
for (let attempt = 0; attempt < attempts; attempt++) {
signal?.throwIfAborted();
const stream = attempt > 0 && source.reopenStream ? source.reopenStream() : source.stream;
try {
// Upload time scales with file size, so the deadline bounds stalls, not the whole request.
await withDeviceRunSessionTimeoutAsync(
{ name: 'Artifact PUT', timeoutMs: 90_000, signal },
async (putSignal, resetDeadline) =>
await putArtifactAsync({
uploadSession,
stream,
artifactId,
signal: putSignal,
onProgress: resetDeadline,
})
);
return;
} catch (error) {
signal?.throwIfAborted();
const retryable =
error instanceof ArtifactPutError
? [408, 429, 500, 502, 503, 504].includes(error.status)
: error instanceof DeviceRunSessionTimeoutError ||
(error instanceof Error && error.name === 'FetchError');
if (!retryable || attempt + 1 === attempts) {
throw error;
}
await delay(500 * (attempt + 1), undefined, { signal });
} finally {
stream.destroy();
}
}
}
async function putArtifactAsync({
uploadSession,
stream,
artifactId,
signal,
onProgress,
}: {
uploadSession: ArtifactUploadSession;
stream: Readable;
artifactId: string;
signal: AbortSignal;
onProgress: () => void;
}): Promise<void> {
const requestController = new AbortController();
try {
const responding = fetch(uploadSession.url, {
method: 'PUT',
headers: new Headers(uploadSession.headers as Record<string, string>),
body: stream,
signal: AbortSignal.any([signal, requestController.signal]),
});
// fetch has piped the body already, so this listener observes flow without starting it.
stream.on('data', onProgress);
const response = await responding;
signal.throwIfAborted();
if (!response.ok) {
throw new ArtifactPutError(artifactId, response);
}
} finally {
// node-fetch destroys the request body on abort, and its response stream is a PassThrough
// whose destruction leaves the socket open, so the request itself is aborted here.
requestController.abort();
}
}
class ArtifactPutError extends SystemError {
readonly status: number;
constructor(artifactId: string, response: { status: number; statusText: string }) {
super(
`Failed to upload device run session artifact ${artifactId}: HTTP ${response.status} ${response.statusText}.`
);
this.status = response.status;
}
}
async function createDeviceRunSessionArtifactUploadSessionAsync(
ctx: CustomBuildContext,
{
deviceRunSessionId,
artifactId,
name,
filename,
kind,
metadata,
size,
}: {
deviceRunSessionId: string;
artifactId: string;
name: string;
filename: string;
kind: string | undefined;
metadata?: Record<string, unknown>;
size: number;
}
{ deviceRunSessionId, artifactId, name, filename, kind, metadata, size }: ArtifactMetadata,
signal: AbortSignal
) {
const result = await ctx.graphqlClient
.mutation(CREATE_DEVICE_RUN_SESSION_ARTIFACT_UPLOAD_SESSION_MUTATION, {
deviceRunSessionId,
input: {
name,
filename,
...(kind !== undefined ? { kind } : {}),
...(metadata !== undefined ? { metadata } : {}),
size,
.mutation(
CREATE_DEVICE_RUN_SESSION_ARTIFACT_UPLOAD_SESSION_MUTATION,
{
deviceRunSessionId,
input: {
name,
filename,
...(kind !== undefined ? { kind } : {}),
...(metadata !== undefined ? { metadata } : {}),
size,
},
},
})
graphqlAbortContext(signal)
)
.toPromise();
signal.throwIfAborted();
if (result.error) {
throw new SystemError(
`Failed to create upload session for device run session artifact ${artifactId}: ${result.error.message}`,
@@ -0,0 +1,203 @@
import { SystemError } from '@expo/eas-build-job';
import type { bunyan } from '@expo/logger';
import type { BuildStepEnv } from '@expo/steps';
import spawn from '@expo/turtle-spawn';
import { createReadStream } from 'node:fs';
import { readFile, readdir, stat } from 'node:fs/promises';
import path from 'node:path';
import limitFactory from 'promise-limit';
import { z } from 'zod';
import { type CustomBuildContext } from '../../customBuildContext';
import { Sentry } from '../../sentry';
import { formatBytes } from '../../utils/artifacts';
import { uploadDeviceRunSessionArtifactAsync } from './deviceRunSessionArtifacts';
const RecordingsSchema = z.array(
z.object({
udid: z.string(),
deviceName: z.string(),
runtimeDisplayName: z.string(),
directory: z.string(),
})
);
const RecordingManifestSchema = z.object({
firstFrameWallClock: z.object({
iso8601: z.string(),
}),
width: z.number().int().positive(),
height: z.number().int().positive(),
recording: z.string(),
status: z.string().optional(),
error: z.string().optional(),
});
const UnlistedRecordingManifestSchema = z.object({
udid: z.string(),
deviceName: z.string(),
runtimeDisplayName: z.string(),
status: z.enum(['recording', 'failed', 'complete']),
recording: z.string(),
});
const recordingStartTimeFormatter = new Intl.DateTimeFormat('en-US', {
year: 'numeric',
month: 'short',
day: 'numeric',
hour: '2-digit',
minute: '2-digit',
second: '2-digit',
fractionalSecondDigits: 3,
hourCycle: 'h23',
timeZone: 'UTC',
timeZoneName: 'short',
});
export function parseDeviceScreenRecordings(input: unknown): z.infer<typeof RecordingsSchema> {
const result = RecordingsSchema.safeParse(input);
if (!result.success) {
throw new SystemError('Invalid device screen recordings input.', {
cause: result.error,
});
}
return result.data;
}
/**
* A Hub that was killed never lists its recording, even one it had already completed at the
* duration limit. session.json is at "recording" or "failed" with a fragmented .partial that plays
* up to its last keyframe, or at "complete" with the finished file. Return the ones ffprobe can read.
*/
export async function findUnlistedDeviceScreenRecordingsAsync({
root,
env,
logger,
}: {
root: string;
env: BuildStepEnv;
logger: bunyan;
}): Promise<z.infer<typeof RecordingsSchema>> {
const found: z.infer<typeof RecordingsSchema> = [];
for (const entry of await readdir(root, { withFileTypes: true })) {
if (!entry.isDirectory()) {
continue;
}
const directory = path.join(root, entry.name);
const manifest = UnlistedRecordingManifestSchema.safeParse(
await readFile(path.join(directory, 'session.json'), 'utf-8')
.then(text => JSON.parse(text))
.catch(() => null)
);
if (!manifest.success || path.basename(manifest.data.recording) !== manifest.data.recording) {
continue;
}
const file = path.join(directory, manifest.data.recording);
const name = path.join(entry.name, manifest.data.recording);
let duration = 0;
try {
const probe = await spawn(
'ffprobe',
['-v', 'error', '-show_entries', 'format=duration', '-of', 'csv=p=0', file],
{ env, stdio: 'pipe' }
);
duration = Number(probe.stdout.trim());
} catch (err) {
if ((err as NodeJS.ErrnoException).code === 'ENOENT') {
logger.warn(`ffprobe is not available; cannot recover the unlisted recording ${name}.`);
continue;
}
}
if (!(duration > 0)) {
logger.warn(`Unlisted recording ${name} does not decode; skipping it.`);
continue;
}
const { udid, deviceName, runtimeDisplayName } = manifest.data;
found.push({ udid, deviceName, runtimeDisplayName, directory });
}
return found;
}
export async function uploadDeviceRunSessionScreenRecordingsAsync(
ctx: CustomBuildContext,
{
logger,
deviceRunSessionId,
recordings,
}: {
logger: bunyan;
deviceRunSessionId: string;
recordings: z.infer<typeof RecordingsSchema>;
}
): Promise<boolean> {
if (recordings.length === 0) {
logger.info('No device screen recordings found; skipping uploads.');
return true;
}
const limit = limitFactory(5);
const uploaded = await Promise.all(
recordings.map(recording =>
limit(async () => {
try {
const metadata = RecordingManifestSchema.parse(
JSON.parse(await readFile(path.join(recording.directory, 'session.json'), 'utf-8'))
);
if (path.basename(metadata.recording) !== metadata.recording) {
throw new Error('Recording filename must not contain a directory.');
}
const startedAt = recordingStartTimeFormatter.format(
new Date(metadata.firstFrameWallClock.iso8601)
);
const partial = metadata.status !== undefined && metadata.status !== 'complete';
const shortUdid = `${recording.udid.slice(0, 8)}-…`;
const displayName = `${recording.deviceName} screen recording (${shortUdid}, started at ${startedAt}${partial ? ', partial' : ''})`;
const recordingPath = path.join(recording.directory, metadata.recording);
const { size } = await stat(recordingPath);
const recordingId = path.basename(recording.directory);
logger.info(
`Uploading screen recording for ${recording.deviceName} (${formatBytes(size)}).`
);
await uploadDeviceRunSessionArtifactAsync(ctx, {
deviceRunSessionId,
artifactId: recordingId,
name: displayName,
filename: `${recordingId}.mp4`,
kind: 'screen-recording',
metadata: {
__eas_type: 'screen-recording',
__eas_screen_recording: '1',
udid: recording.udid,
deviceName: recording.deviceName,
runtimeDisplayName: recording.runtimeDisplayName,
firstFrameAt: metadata.firstFrameWallClock.iso8601,
width: metadata.width,
height: metadata.height,
...(partial
? {
partial: true,
partialReason:
metadata.error ??
'The Device Hub stopped before it could finalize the recording.',
}
: {}),
},
size,
stream: createReadStream(recordingPath),
reopenStream: () => createReadStream(recordingPath),
});
return true;
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not upload device screen recording', error);
logger.warn(
{ err: error },
`Could not upload screen recording for ${recording.deviceName}.`
);
return false;
}
})
)
);
return uploaded.every(Boolean);
}
@@ -0,0 +1,43 @@
export class DeviceRunSessionTimeoutError extends Error {
constructor(name: string, timeoutMs: number) {
super(`${name} timed out after ${timeoutMs}ms.`);
this.name = 'DeviceRunSessionTimeoutError';
}
}
/**
* Bounds waiting and signals cancellation. Operations must check the signal before later effects.
* Calling resetDeadline restarts the timer, so the bound applies to stalls instead of total time.
*/
export async function withDeviceRunSessionTimeoutAsync<T>(
{ name, timeoutMs, signal: parent }: { name: string; timeoutMs: number; signal?: AbortSignal },
operation: (signal: AbortSignal, resetDeadline: () => void) => Promise<T>
): Promise<T> {
const controller = new AbortController();
const signal = parent ? AbortSignal.any([parent, controller.signal]) : controller.signal;
signal.throwIfAborted();
let rejectAbort: (reason: unknown) => void = () => {};
const aborted = new Promise<never>((_resolve, reject) => {
rejectAbort = reject;
});
const onAbort = () => rejectAbort(signal.reason);
signal.addEventListener('abort', onAbort, { once: true });
const expire = () => controller.abort(new DeviceRunSessionTimeoutError(name, timeoutMs));
let timer = setTimeout(expire, timeoutMs);
let settled = false;
const resetDeadline = () => {
clearTimeout(timer);
if (!settled && !signal.aborted) {
timer = setTimeout(expire, timeoutMs);
}
};
try {
const result = await Promise.race([operation(signal, resetDeadline), aborted]);
signal.throwIfAborted();
return result;
} finally {
settled = true;
clearTimeout(timer);
signal.removeEventListener('abort', onAbort);
}
}
@@ -0,0 +1,590 @@
import { SystemError, UserError } from '@expo/eas-build-job';
import type { bunyan } from '@expo/logger';
import { BuildRuntimePlatform, type BuildStepEnv } from '@expo/steps';
import { randomBytes } from 'node:crypto';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { z } from 'zod';
import type { CustomBuildContext } from '../../customBuildContext';
import { Sentry } from '../../sentry';
import {
PackageManager,
resolveConfiguredPackageManager,
resolvePackageExec,
} from '../../utils/packageManager';
import { sleepAsync } from '../../utils/retry';
import { turtleFetch } from '../../utils/turtleFetch';
import {
findUnlistedDeviceScreenRecordingsAsync,
parseDeviceScreenRecordings,
uploadDeviceRunSessionScreenRecordingsAsync,
} from './deviceRunSessionScreenRecordings';
import {
type DetachedProcessHandle,
type ServeSimLaunchOptions,
ensureFfmpegInstalledOnceAsync,
fetchWebPreviewTurnArgsAsync,
findAvailablePortAsync,
getDeviceRunSessionIdOrThrow,
getNgrokAuthtokenOrThrow,
isProcessRunning,
spawnDetached,
startNgrokTunnelAsync,
} from './remoteDeviceRunSession';
import { withDeviceRunSessionTimeoutAsync } from './deviceRunSessionTimeout';
import { SERVE_SIM_STATE_DIR, readServeSimServersAsync } from './serveSimMetricsRecorder';
const WEB_PREVIEW_HOST = '127.0.0.1';
const SERVE_SIM_PACKAGE_NAME = '@expo/serve-sim';
const SERVE_SIM_MAX_DIMENSION = '1600';
const SERVE_SIM_MJPEG_QUALITY = '0.55';
const SERVE_SIM_VIDEO_BITRATE = '10000000';
const SERVE_SIM_VIDEO_FPS = '60';
const EXPO_DEVICE_HUB_PACKAGE_NAME = 'expo-device-hub';
const EXPO_DEVICE_HUB_MAX_DIMENSION = '960';
const EXPO_DEVICE_HUB_VIDEO_BITRATE = '6000000';
const EXPO_DEVICE_HUB_VIDEO_FPS = '60';
// On SIGTERM the Hub finalizes the recording itself, with this deadline before it force-exits,
// so a stop request that failed here still gets one more chance to write the MP4.
const EXPO_DEVICE_HUB_SIGTERM_FINALIZE_DEADLINE_MS = 60_000;
const EXPO_DEVICE_HUB_EXIT_LEEWAY_MS = 10_000;
const RECORDING_STOP_GRACE_PERIOD_MS =
EXPO_DEVICE_HUB_SIGTERM_FINALIZE_DEADLINE_MS + EXPO_DEVICE_HUB_EXIT_LEEWAY_MS;
const HOST_OUTPUT_TAIL_CHARS = 8_000;
export function websiteOrigin(env: BuildStepEnv): string {
return env.EXPO_LOCAL
? 'https://expo.test'
: env.EXPO_STAGING
? 'https://staging.expo.dev'
: 'https://expo.dev';
}
export function simulatorPreviewPageUrl(env: BuildStepEnv, subdomainId: string): string {
return new URL(`/simulator-preview/${subdomainId}`, websiteOrigin(env)).toString();
}
// Website dev servers on expo.test use staging sessions, and CORS matches ports exactly.
const WEBSITE_DEV_ORIGINS = [
'https://expo.test',
'https://expo.test:13001',
...Array.from({ length: 16 }, (_, index) => `https://expo.test:${13200 + index}`),
];
export function websiteOriginServeSimArgs(env: BuildStepEnv): string[] {
const origins = new Set([websiteOrigin(env)]);
if (!env.EXPO_LOCAL && env.EXPO_STAGING) {
origins.add('https://*.expo.dev');
}
if (env.EXPO_LOCAL || env.EXPO_STAGING) {
for (const origin of WEBSITE_DEV_ORIGINS) {
origins.add(origin);
}
}
return [...origins].flatMap(origin => ['--cors-origin', origin, '--frame-ancestor', origin]);
}
function createServeSimPackageSpec(packageVersion: string | undefined): string {
return `${SERVE_SIM_PACKAGE_NAME}@${packageVersion ?? 'latest'}`;
}
function createExpoDeviceHubPackageSpec(packageVersion: string | undefined): string {
return `${EXPO_DEVICE_HUB_PACKAGE_NAME}@${packageVersion ?? 'latest'}`;
}
export function createServeSimArgs({
port,
turnArgs = [],
websiteArgs = [],
shareUrl,
packageVersion,
launchAppIdentifier,
launchArgs = [],
openUrl,
}: {
port: number;
turnArgs?: string[];
websiteArgs?: string[];
shareUrl?: string;
packageVersion?: string;
} & ServeSimLaunchOptions): string[] {
return [
createServeSimPackageSpec(packageVersion),
'--port',
String(port),
'--host',
WEB_PREVIEW_HOST,
'--require-token',
'--transport',
'webrtc',
'--webrtc-codec',
'h264',
'--max-dimension',
SERVE_SIM_MAX_DIMENSION,
'--mjpeg-quality',
SERVE_SIM_MJPEG_QUALITY,
'--video-bitrate',
SERVE_SIM_VIDEO_BITRATE,
'--video-fps',
SERVE_SIM_VIDEO_FPS,
...turnArgs,
...websiteArgs,
...(shareUrl ? ['--share-url', shareUrl] : []),
...(launchAppIdentifier ? ['--launch-app-identifier', launchAppIdentifier] : []),
...launchArgs.flatMap(argument => ['--launch-arg', argument]),
...(openUrl ? ['--open-url', openUrl] : []),
];
}
export function createExpoDeviceHubArgs({
port,
turnArgs = [],
packageVersion,
recordingDirectory,
}: {
port: number;
turnArgs?: string[];
packageVersion?: string;
recordingDirectory?: string;
}): string[] {
return [
createExpoDeviceHubPackageSpec(packageVersion),
'--port',
String(port),
'--host',
WEB_PREVIEW_HOST,
'--platform',
'android',
'--transport',
'webrtc',
'--webrtc-codec',
'h264',
'--webrtc-ice-policy',
'all',
'--max-dimension',
EXPO_DEVICE_HUB_MAX_DIMENSION,
'--video-bitrate',
EXPO_DEVICE_HUB_VIDEO_BITRATE,
'--video-fps',
EXPO_DEVICE_HUB_VIDEO_FPS,
'--hide-sidebar',
'--hide-boot-device',
...(recordingDirectory ? ['--android-recording-directory', recordingDirectory] : []),
...turnArgs,
];
}
const WebPreviewReadyResponseSchema = z.object({
status: z.literal('ready'),
device: z.string(),
});
export async function waitForWebPreviewReadyAsync({
previewServer,
serverName,
port,
timeoutMs,
}: {
previewServer: Pick<DetachedProcessHandle, 'pid' | 'getOutput'>;
serverName: string;
port: number;
timeoutMs: number;
}): Promise<string> {
const readyUrl = `http://${WEB_PREVIEW_HOST}:${port}/readyz`;
const deadline = Date.now() + timeoutMs;
let lastError: unknown;
while (Date.now() < deadline) {
if (previewServer.pid !== undefined && !isProcessRunning(previewServer.pid)) {
throw new SystemError(
`${serverName} exited before becoming ready. Last output:\n${
previewServer.getOutput() || '<empty>'
}`
);
}
try {
const response = await turtleFetch(readyUrl, 'GET', {
retries: 0,
timeout: 2_000,
});
const ready = WebPreviewReadyResponseSchema.parse(await response.json());
return ready.device;
} catch (error) {
lastError = error;
}
await sleepAsync(1_000);
}
throw new SystemError(
`Timed out waiting for ${serverName} readiness at ${readyUrl}${
lastError instanceof Error ? `: ${lastError.message}` : ''
}. Last output:\n${previewServer.getOutput() || '<empty>'}`
);
}
export async function readServeSimPreviewTokenAsync(
udid: string,
stateDir: string = SERVE_SIM_STATE_DIR
): Promise<string | undefined> {
const servers = await readServeSimServersAsync(stateDir);
return servers.find(server => server.udid === udid)?.token;
}
export type DeviceWebPreview = {
previewPageUrl: string;
apiUrl: string;
previewToken?: string;
/** Closes this tunnel only. The session host and recording remain running. */
closeAsync(): Promise<void>;
};
export type DeviceSessionHost = {
openPreviewAsync(options: { baseDomain: string }): Promise<DeviceWebPreview>;
/**
* Terminal and idempotent. Never rejects: a failed finalization, host stop or upload is logged
* and reported, so callers need no error handling around it.
*/
finishAsync(): Promise<void>;
};
type AndroidSessionRecording = {
deviceRunSessionId: string;
directory: string;
controlToken: string;
env: BuildStepEnv;
};
export async function startDeviceSessionHostAsync(
ctx: CustomBuildContext,
{
runtimePlatform,
env,
logger,
timeoutMs,
packageVersion,
launchAppIdentifier,
launchArgs,
openUrl,
}: {
runtimePlatform: BuildRuntimePlatform;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
packageVersion?: string;
} & ServeSimLaunchOptions
): Promise<DeviceSessionHost> {
const isAndroid = runtimePlatform === BuildRuntimePlatform.LINUX;
// Unreachable from the step functions, which reject a non-Darwin launch while parsing.
// Kept because this function is exported and expo-device-hub cannot launch.
if (isAndroid && launchAppIdentifier) {
throw new UserError(
'EAS_LAUNCH_APPLICATION_INVALID_INPUT',
`Cannot launch ${launchAppIdentifier}: an application launch runs through serve-sim on an iOS simulator, and this session runs expo-device-hub on ${runtimePlatform}.`
);
}
if (isAndroid) {
await ensureFfmpegInstalledOnceAsync({ runtimePlatform, env, logger });
}
const recording: AndroidSessionRecording | null = isAndroid
? {
deviceRunSessionId: getDeviceRunSessionIdOrThrow(env),
directory: await fs.promises.mkdtemp(path.join(os.tmpdir(), 'android-session-recordings-')),
controlToken: randomBytes(32).toString('hex'),
env,
}
: null;
const subdomainId = randomBytes(16).toString('hex');
const previewPageUrl = simulatorPreviewPageUrl(env, subdomainId);
const port = await findAvailablePortAsync();
const serverName = isAndroid ? 'expo-device-hub' : 'serve-sim';
const packageSpec = isAndroid
? createExpoDeviceHubPackageSpec(packageVersion)
: createServeSimPackageSpec(packageVersion);
const turnArgs = await fetchWebPreviewTurnArgsAsync(ctx, { env, logger });
const previewExec = resolvePackageExec(
resolveConfiguredPackageManager(env, PackageManager.NPM),
isAndroid
? createExpoDeviceHubArgs({
port,
turnArgs,
packageVersion,
recordingDirectory: recording?.directory,
})
: createServeSimArgs({
port,
turnArgs,
packageVersion,
websiteArgs: websiteOriginServeSimArgs(env),
shareUrl: previewPageUrl,
launchAppIdentifier,
launchArgs,
openUrl,
})
);
logger.info(
`Launching ${packageSpec} on ${WEB_PREVIEW_HOST}:${port} via ${previewExec.command}.`
);
const previewServer = spawnDetached({
command: previewExec.command,
args: previewExec.args,
env: recording
? { ...env, EXPO_DEVICE_HUB_RECORDING_CONTROL_TOKEN: recording.controlToken }
: env,
stopGracePeriodMs: recording ? RECORDING_STOP_GRACE_PERIOD_MS : undefined,
});
let previewToken: string | undefined;
let previewTask: Promise<DeviceWebPreview> | null = null;
let finishTask: Promise<void> | null = null;
let hostReady = false;
const host: DeviceSessionHost = {
openPreviewAsync({ baseDomain }) {
if (finishTask) {
return Promise.reject(
new SystemError('Cannot open a preview after session host finalization.')
);
}
if (previewTask) {
return previewTask;
}
const opening = (async (): Promise<DeviceWebPreview> => {
const tunnel = await startNgrokTunnelAsync({
port,
subdomainPrefix: 'web-preview',
subdomainId,
baseDomain,
authtoken: getNgrokAuthtokenOrThrow(env),
logger,
});
if (finishTask) {
await withDeviceRunSessionTimeoutAsync(
{ name: 'Late preview tunnel close', timeoutMs: 5_000 },
async () => await tunnel.stopAsync()
).catch(err => logger.warn({ err }, 'Could not close a late preview tunnel.'));
throw new SystemError('Session host finalized while the preview was opening.');
}
let closeTask: Promise<void> | null = null;
return {
previewPageUrl,
apiUrl: tunnel.url,
previewToken,
closeAsync() {
return (closeTask ??= (async () => {
try {
await tunnel.stopAsync();
} finally {
if (previewTask === opening) {
previewTask = null;
}
}
})());
},
};
})();
previewTask = opening;
// A failed tunnel may be retried without restarting capture.
void opening.catch(() => {
if (previewTask === opening) {
previewTask = null;
}
});
return opening;
},
finishAsync() {
return (finishTask ??= finishDeviceSessionHostAsync(ctx, {
previewTask,
previewServer,
serverName,
port,
// A host that never answered /readyz has nothing to finalize or upload.
recording: hostReady ? recording : null,
logger,
}));
},
};
try {
logger.info(`Waiting for ${serverName} to become ready.`);
const device = await waitForWebPreviewReadyAsync({
previewServer,
serverName,
port,
timeoutMs,
});
hostReady = true;
if (!isAndroid) {
previewToken = await readServeSimPreviewTokenAsync(device);
if (!previewToken) {
throw new SystemError(
`serve-sim became ready but wrote no session token for device ${device}. The preview is ` +
'on a public tunnel and would be reachable without one, so the session cannot continue. ' +
'This usually means the state file was not written as expected; retry the session, and ' +
'report it if it repeats.'
);
}
}
return host;
} catch (error) {
await host.finishAsync();
throw error;
}
}
async function finishDeviceSessionHostAsync(
ctx: CustomBuildContext,
{
previewTask,
previewServer,
serverName,
port,
recording,
logger,
}: {
previewTask: Promise<DeviceWebPreview> | null;
previewServer: DetachedProcessHandle;
serverName: string;
port: number;
recording: AndroidSessionRecording | null;
logger: bunyan;
}
): Promise<void> {
// Native ngrok operations have no scoped cancellation. Retire a late listener too.
const retirePreview = withDeviceRunSessionTimeoutAsync(
{ name: 'Preview tunnel retirement', timeoutMs: 5_000 },
async () => {
// A preview that failed to open, or was closed by its opener after finish began, has no tunnel left.
const preview = await previewTask?.catch(() => null);
await preview?.closeAsync();
}
).catch(err => {
logger.warn({ err }, `Could not close the ${serverName} preview tunnel within its deadline.`);
});
let finalization: AndroidRecordingFinalization | null = null;
if (recording) {
// stopAsync signals the whole process group, including capture's encoder.
// Finalize the MP4 first; the token protects this route on the preview server.
finalization = await finalizeAndroidRecordingAsync({
port,
controlToken: recording.controlToken,
logger,
});
}
let hostStopped = false;
try {
await previewServer.stopAsync();
if (previewServer.pid !== undefined && isProcessRunning(previewServer.pid)) {
throw new Error('Session host is still running after shutdown.');
}
hostStopped = true;
} catch (err) {
logger.warn({ err }, `Could not stop the ${serverName} session host.`);
}
await retirePreview;
// A Hub that never recorded has logged its reason and left nothing to upload.
const captured = finalization !== 'not-recording';
let uploaded = false;
if (recording && captured && hostStopped) {
uploaded = await uploadFinishedAndroidRecordingAsync(ctx, { recording, logger });
}
if (recording && captured && (finalization === 'failed' || !uploaded)) {
// The Hub reports capture failures on stderr; the stop route answers with only a summary.
logger.warn(
{ hostOutput: previewServer.getOutput().slice(-HOST_OUTPUT_TAIL_CHARS) || '<empty>' },
'Session host output around the recording failure.'
);
}
}
type AndroidRecordingFinalization = 'finalized' | 'not-recording' | 'failed';
async function finalizeAndroidRecordingAsync({
port,
controlToken,
logger,
}: {
port: number;
controlToken: string;
logger: bunyan;
}): Promise<AndroidRecordingFinalization> {
try {
const response = await withDeviceRunSessionTimeoutAsync(
{ name: 'Android recording finalization', timeoutMs: 60_000 },
async signal =>
await turtleFetch(
`http://${WEB_PREVIEW_HOST}:${port}/_eas/android-recording/stop`,
'POST',
{
headers: { Authorization: `Bearer ${controlToken}` },
retries: 0,
shouldThrowOnNotOk: false,
signal,
}
)
);
if (response.status === 409) {
// The Hub answers 409 with the reason nothing was recorded, such as the emulator count.
logger.warn(`Android recording was not captured: ${await response.text()}`);
return 'not-recording';
}
if (!response.ok) {
throw new SystemError(
`Android recording finalization returned HTTP ${response.status}: ${await response.text()}`
);
}
return 'finalized';
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not finalize Android recording before shutdown', error, {
level: 'warning',
});
logger.warn({ err: error }, 'Could not finalize Android recording before shutdown.');
return 'failed';
}
}
async function uploadFinishedAndroidRecordingAsync(
ctx: CustomBuildContext,
{ recording, logger }: { recording: AndroidSessionRecording; logger: bunyan }
): Promise<boolean> {
try {
let recordings = parseDeviceScreenRecordings(
JSON.parse(
await fs.promises.readFile(path.join(recording.directory, 'recordings.json'), 'utf8')
)
);
if (recordings.length === 0) {
// recordings.json is written at finish; a killed Hub leaves what it had on disk unlisted.
recordings = await findUnlistedDeviceScreenRecordingsAsync({
root: recording.directory,
env: recording.env,
logger,
});
}
for (const item of recordings) {
if (
path.dirname(path.resolve(item.directory)) !== recording.directory ||
(await fs.promises.lstat(item.directory)).isSymbolicLink()
) {
throw new Error('Recording directory is not an owned session child.');
}
}
const uploaded = await uploadDeviceRunSessionScreenRecordingsAsync(ctx, {
logger,
deviceRunSessionId: recording.deviceRunSessionId,
recordings,
});
if (recordings.length > 0 && uploaded) {
await fs.promises.rm(recording.directory, { recursive: true });
return true;
}
return false;
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not upload the Android session recording', error, { level: 'warning' });
logger.warn(
{ err: error, recordingDirectory: recording.directory },
'Could not upload the Android session recording.'
);
return false;
}
}
@@ -27,6 +27,48 @@ const AscErrorResponseSchema = z.object({
});
const GetApi = {
// https://developer.apple.com/documentation/appstoreconnectapi/get-v1-builds-_id_-app
'/v1/builds/:id/app': {
path: z.object({ id: z.string() }),
request: z.object({}),
response: z.object({
data: z.object({
id: z.string(),
attributes: z.object({ primaryLocale: z.string().optional() }).optional(),
}),
}),
},
// https://developer.apple.com/documentation/appstoreconnectapi/get-v1-builds-_id_-betabuildlocalizations
'/v1/builds/:id/betaBuildLocalizations': {
path: z.object({ id: z.string() }),
request: z.object({ limit: z.number().int().max(200) }),
response: z.object({
data: z.array(
z.object({
id: z.string(),
attributes: z.object({ locale: z.string().optional() }).optional(),
})
),
links: z.object({ next: z.string().nullish() }).optional(),
}),
},
// https://developer.apple.com/documentation/appstoreconnectapi/get-v1-betagroups
'/v1/betaGroups': {
path: z.object({}),
request: z.object({
'filter[app]': z.string(),
limit: z.number().int().max(200),
}),
response: z.object({
data: z.array(
z.object({
id: z.string(),
attributes: z.object({ name: z.string().optional() }).optional(),
})
),
links: z.object({ next: z.string().nullish() }).optional(),
}),
},
'/v1/apps': {
path: z.object({}),
request: z.object({
@@ -119,12 +161,40 @@ const GetApi = {
warnings: z.array(z.object({ code: z.string(), description: z.string() })).optional(),
}),
}),
relationships: z
.object({
build: z.object({
data: z.object({ type: z.literal('builds'), id: z.string() }).nullable(),
}),
})
.optional(),
}),
}),
},
} satisfies ApiSchema;
const PostApi = {
// https://developer.apple.com/documentation/appstoreconnectapi/post-v1-builds-_id_-relationships-betagroups
'/v1/builds/:id/relationships/betaGroups': {
path: z.object({ id: z.string() }),
request: z.object({
data: z.array(z.object({ type: z.literal('betaGroups'), id: z.string() })),
}),
response: z.undefined(),
},
// https://developer.apple.com/documentation/appstoreconnectapi/post-v1-betabuildlocalizations
'/v1/betaBuildLocalizations': {
request: z.object({
data: z.object({
type: z.literal('betaBuildLocalizations'),
attributes: z.object({ locale: z.string(), whatsNew: z.string().nullish() }),
relationships: z.object({
build: z.object({ data: z.object({ type: z.literal('builds'), id: z.string() }) }),
}),
}),
}),
response: z.object({ data: z.object({ id: z.string() }) }),
},
'/v1/buildUploads': {
request: z.object({
// https://developer.apple.com/documentation/appstoreconnectapi/builduploadcreaterequest/data-data.dictionary
@@ -223,6 +293,18 @@ const PostApi = {
} satisfies ApiSchema;
const PatchApi = {
// https://developer.apple.com/documentation/appstoreconnectapi/patch-v1-betabuildlocalizations-_id_
'/v1/betaBuildLocalizations/:id': {
path: z.object({ id: z.string() }),
request: z.object({
data: z.object({
type: z.literal('betaBuildLocalizations'),
id: z.string(),
attributes: z.object({ whatsNew: z.string().nullish() }).optional(),
}),
}),
response: z.object({ data: z.object({ id: z.string() }) }),
},
// https://developer.apple.com/documentation/appstoreconnectapi/patch-v1-builduploadfiles-_id_
'/v1/buildUploadFiles/:id': {
path: z.object({
@@ -334,14 +416,40 @@ export class AscApiClient {
});
}
public async getNextPageAsync<TPath extends keyof typeof GetApi>(
path: TPath,
next: string
): Promise<z.output<(typeof GetApi)[TPath]['response']>> {
const nextUrl = new URL(next, this.baseUrl);
if (nextUrl.origin !== this.baseUrl) {
throw new Error('App Store Connect returned a pagination link on another host.');
}
return await this.sendRequestAsync({
method: 'GET',
path: nextUrl.toString(),
body: {},
requestSchema: z.object({}),
responseSchema: GetApi[path].response,
});
}
public async postAsync<TPath extends keyof typeof PostApi>(
path: TPath,
body: z.input<(typeof PostApi)[TPath]['request']>
body: z.input<(typeof PostApi)[TPath]['request']>,
// Require path params if the API schema defines them.
...params: (typeof PostApi)[TPath] extends { path: infer TSchema extends z.ZodType }
? [params: z.input<TSchema>]
: []
): Promise<z.output<(typeof PostApi)[TPath]['response']>> {
const schema = PostApi[path];
let effectivePath: string = path;
const pathParams = 'path' in schema ? schema.path.parse(params[0]) : {};
for (const [key, value] of Object.entries(pathParams)) {
effectivePath = effectivePath.replaceAll(`:${key}`, encodeURIComponent(String(value)));
}
return await this.sendRequestAsync({
method: 'POST',
path,
path: effectivePath,
body,
requestSchema: schema.request,
responseSchema: schema.response,
@@ -421,6 +529,9 @@ export class AscApiClient {
});
}
if (response.status === 204) {
return responseSchema.parse(undefined);
}
const text = await response.text();
const parsedJson = await asyncResult((async () => JSON.parse(text))());
if (!parsedJson.ok) {
@@ -1,4 +1,7 @@
import { UserError } from '@expo/eas-build-job';
import fs from 'fs-extra';
import * as jose from 'jose';
import { z } from 'zod';
import {
AscApiClient,
@@ -9,6 +12,24 @@ import {
} from './AscApiClient';
export namespace AscApiUtils {
export async function signTokenAsync({ keyPath }: { keyPath: string }): Promise<string> {
const keyJson = z
.object({ issuer_id: z.string().nullish(), key_id: z.string(), key: z.string() })
.parse(await fs.readJson(keyPath));
const privateKey = await jose.importPKCS8(keyJson.key, 'ES256');
const jwt = new jose.SignJWT({})
.setProtectedHeader({ alg: 'ES256', kid: keyJson.key_id })
.setAudience('appstoreconnect-v1')
.setExpirationTime('20m');
if (keyJson.issuer_id) {
jwt.setIssuer(keyJson.issuer_id);
} else {
// An individual API key has no issuer ID.
jwt.setSubject('user');
}
return await jwt.sign(privateKey);
}
/**
* Maps a bundle's `DTPlatformName` (from its Info.plist) to the App Store
* Connect platform used for a build upload. Unknown or missing values fall
@@ -102,6 +102,13 @@ describe(AscApiClient, () => {
expect(scope.isDone()).toBeTruthy();
});
it('requires path parameters for a POST route that has them', async () => {
await expect(
// @ts-expect-error The build ID is required by this route.
client.postAsync('/v1/builds/:id/relationships/betaGroups', { data: [] })
).rejects.toThrow();
});
it('creates build upload file', async () => {
const buildUploadId = 'fdf9c476-aaa4-4ead-b91c-6e3cc3a47805';
const fileId = '5b110930-f947-4998-a129-5926ffcedde5';
@@ -245,6 +252,7 @@ describe(AscApiClient, () => {
attributes: {
state: responseFixture.data.attributes.state,
},
relationships: { build: { data: null } },
},
});
expect(scope.isDone()).toBeTruthy();
@@ -26,26 +26,10 @@ import {
parseOpenUrlInput,
} from '../functions/launchApplication';
import { Sentry } from '../../sentry';
import {
PackageManager,
resolveConfiguredPackageManager,
resolvePackageExec,
} from '../../utils/packageManager';
import { sleepAsync } from '../../utils/retry';
import { turtleFetch } from '../../utils/turtleFetch';
import { SERVE_SIM_STATE_DIR, readServeSimServersAsync } from './serveSimMetricsRecorder';
const XCODE_DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer';
const WEB_PREVIEW_HOST = '127.0.0.1';
const SERVE_SIM_PACKAGE_NAME = '@expo/serve-sim';
const SERVE_SIM_MAX_DIMENSION = '1600';
const SERVE_SIM_MJPEG_QUALITY = '0.55';
const SERVE_SIM_VIDEO_BITRATE = '10000000';
const SERVE_SIM_VIDEO_FPS = '60';
const EXPO_DEVICE_HUB_PACKAGE_NAME = 'expo-device-hub';
const EXPO_DEVICE_HUB_MAX_DIMENSION = '960';
const EXPO_DEVICE_HUB_VIDEO_BITRATE = '6000000';
const EXPO_DEVICE_HUB_VIDEO_FPS = '60';
const START_DEVICE_RUN_SESSION_MUTATION = graphql(`
mutation StartDeviceRunSession($deviceRunSessionId: ID!, $remoteConfig: JSONObject!) {
@@ -547,10 +531,11 @@ export type DetachedProcessHandle = {
/** PID of the directly spawned process, if the OS assigned one. */
pid: number | undefined;
getOutput: () => string;
getExitError: () => Error | undefined;
stopAsync: () => Promise<void>;
};
function isProcessRunning(pid: number): boolean {
export function isProcessRunning(pid: number): boolean {
try {
process.kill(pid, 0);
return true;
@@ -559,7 +544,10 @@ function isProcessRunning(pid: number): boolean {
}
}
async function stopDetachedProcessAsync(pid: number | undefined): Promise<void> {
async function stopDetachedProcessAsync(
pid: number | undefined,
gracePeriodMs = 5_000
): Promise<void> {
if (pid === undefined || !isProcessRunning(pid)) {
return;
}
@@ -575,7 +563,7 @@ async function stopDetachedProcessAsync(pid: number | undefined): Promise<void>
}
}
const deadline = Date.now() + 5_000;
const deadline = Date.now() + gracePeriodMs;
while (Date.now() < deadline && isProcessRunning(pid)) {
await sleepAsync(100);
}
@@ -589,6 +577,43 @@ async function stopDetachedProcessAsync(pid: number | undefined): Promise<void>
process.kill(pid, 'SIGKILL');
} catch {}
}
// kill(pid, 0) succeeds on the zombie until libuv reaps it on a later loop turn.
const killDeadline = Date.now() + 5_000;
while (Date.now() < killDeadline && isProcessRunning(pid)) {
await sleepAsync(100);
}
}
/**
* Runs every named teardown to completion and logs each failure. The first failure is rethrown
* only when the session body succeeded, so a teardown error cannot replace the error that ended it.
*/
export async function finishRemoteSessionAsync({
teardown,
sessionFailed,
logger,
}: {
teardown: [name: string, task: Promise<unknown> | undefined][];
sessionFailed: boolean;
logger: bunyan;
}): Promise<void> {
const results = await Promise.allSettled(teardown.map(([, task]) => task));
const failures = results.flatMap((result, index) =>
result.status === 'rejected' ? [{ name: teardown[index][0], err: result.reason }] : []
);
for (const { name, err } of failures) {
logger.warn({ err }, `Could not stop the ${name} during remote session teardown.`);
if (sessionFailed) {
// The session error is what the step reports, so a swallowed teardown failure goes to Sentry.
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture(`Could not stop the ${name} after the remote session failed`, error, {
level: 'warning',
});
}
}
if (!sessionFailed && failures.length > 0) {
throw failures[0].err;
}
}
export function spawnDetached({
@@ -596,11 +621,13 @@ export function spawnDetached({
args,
cwd,
env,
stopGracePeriodMs,
}: {
command: string;
args: string[];
cwd?: string;
env: BuildStepEnv;
stopGracePeriodMs?: number;
}): DetachedProcessHandle {
const promise = spawn(command, args, {
cwd,
@@ -608,9 +635,24 @@ export function spawnDetached({
stdio: ['ignore', 'pipe', 'pipe'],
detached: true,
});
// We don't await the process — it should outlive this step. Failures show
// up in the captured output; suppress unhandled rejections here.
promise.catch(() => {});
// Observe completion without rejecting in the background. Startup callers can
// distinguish a dead process from one that is still preparing its state file.
let exitError: Error | undefined;
// The spawn promise waits for stdio to close. Descendants may keep those
// pipes open after the launcher exits, so observe the exit itself as well.
promise.child.once('exit', (code, signal) => {
exitError = new Error(
signal ? `Process exited with signal ${signal}.` : `Process exited with code ${code}.`
);
});
void promise.then(
() => {
exitError ??= new Error('Process exited with code 0.');
},
error => {
exitError ??= error instanceof Error ? error : new Error(String(error));
}
);
promise.child.unref();
let output = '';
@@ -624,50 +666,11 @@ export function spawnDetached({
return {
pid,
getOutput: () => output,
stopAsync: async () => await stopDetachedProcessAsync(pid),
getExitError: () => exitError,
stopAsync: async () => await stopDetachedProcessAsync(pid, stopGracePeriodMs),
};
}
export function websiteOrigin(env: BuildStepEnv): string {
return env.EXPO_LOCAL
? 'https://expo.test'
: env.EXPO_STAGING
? 'https://staging.expo.dev'
: 'https://expo.dev';
}
export function simulatorPreviewPageUrl(env: BuildStepEnv, subdomainId: string): string {
return new URL(`/simulator-preview/${subdomainId}`, websiteOrigin(env)).toString();
}
// Website dev servers on expo.test use staging sessions, and CORS matches ports exactly.
const WEBSITE_DEV_ORIGINS = [
'https://expo.test',
'https://expo.test:13001',
...Array.from({ length: 16 }, (_, index) => `https://expo.test:${13200 + index}`),
];
export function websiteOriginServeSimArgs(env: BuildStepEnv): string[] {
const origins = new Set([websiteOrigin(env)]);
if (!env.EXPO_LOCAL && env.EXPO_STAGING) {
origins.add('https://*.expo.dev');
}
if (env.EXPO_LOCAL || env.EXPO_STAGING) {
for (const origin of WEBSITE_DEV_ORIGINS) {
origins.add(origin);
}
}
return [...origins].flatMap(origin => ['--cors-origin', origin, '--frame-ancestor', origin]);
}
function createServeSimPackageSpec(packageVersion: string | undefined): string {
return `${SERVE_SIM_PACKAGE_NAME}@${packageVersion ?? 'latest'}`;
}
function createExpoDeviceHubPackageSpec(packageVersion: string | undefined): string {
return `${EXPO_DEVICE_HUB_PACKAGE_NAME}@${packageVersion ?? 'latest'}`;
}
export interface ServeSimLaunchOptions {
launchAppIdentifier?: string;
launchArgs?: string[];
@@ -742,91 +745,12 @@ export function describeServeSimLaunch({
return `serve-sim will launch ${launchAppIdentifier}${withArguments}${thenOpen}.`;
}
export function createServeSimArgs({
port,
turnArgs = [],
websiteArgs = [],
shareUrl,
packageVersion,
launchAppIdentifier,
launchArgs = [],
openUrl,
}: {
port: number;
turnArgs?: string[];
websiteArgs?: string[];
shareUrl?: string;
packageVersion?: string;
} & ServeSimLaunchOptions): string[] {
return [
createServeSimPackageSpec(packageVersion),
'--port',
String(port),
'--host',
WEB_PREVIEW_HOST,
'--require-token',
'--transport',
'webrtc',
'--webrtc-codec',
'h264',
'--max-dimension',
SERVE_SIM_MAX_DIMENSION,
'--mjpeg-quality',
SERVE_SIM_MJPEG_QUALITY,
'--video-bitrate',
SERVE_SIM_VIDEO_BITRATE,
'--video-fps',
SERVE_SIM_VIDEO_FPS,
...turnArgs,
...websiteArgs,
...(shareUrl ? ['--share-url', shareUrl] : []),
...(launchAppIdentifier ? ['--launch-app-identifier', launchAppIdentifier] : []),
...launchArgs.flatMap(argument => ['--launch-arg', argument]),
...(openUrl ? ['--open-url', openUrl] : []),
];
}
export function createExpoDeviceHubArgs({
port,
turnArgs = [],
packageVersion,
}: {
port: number;
turnArgs?: string[];
packageVersion?: string;
}): string[] {
return [
createExpoDeviceHubPackageSpec(packageVersion),
'--port',
String(port),
'--host',
WEB_PREVIEW_HOST,
'--platform',
'android',
'--transport',
'webrtc',
'--webrtc-codec',
'h264',
'--webrtc-ice-policy',
'all',
'--max-dimension',
EXPO_DEVICE_HUB_MAX_DIMENSION,
'--video-bitrate',
EXPO_DEVICE_HUB_VIDEO_BITRATE,
'--video-fps',
EXPO_DEVICE_HUB_VIDEO_FPS,
'--hide-sidebar',
'--hide-boot-device',
...turnArgs,
];
}
export async function findAvailablePortAsync(): Promise<number> {
const server = createServer();
server.unref();
await new Promise<void>((resolve, reject) => {
server.once('error', reject);
server.listen(0, WEB_PREVIEW_HOST, () => resolve());
server.listen(0, '127.0.0.1', () => resolve());
});
const address = server.address();
await new Promise<void>((resolve, reject) => {
@@ -838,270 +762,6 @@ export async function findAvailablePortAsync(): Promise<number> {
return address.port;
}
const WebPreviewReadyResponseSchema = z.object({
status: z.literal('ready'),
device: z.string(),
});
export async function waitForWebPreviewReadyAsync({
previewServer,
serverName,
port,
timeoutMs,
}: {
previewServer: Pick<DetachedProcessHandle, 'pid' | 'getOutput'>;
serverName: string;
port: number;
timeoutMs: number;
}): Promise<string> {
const readyUrl = `http://${WEB_PREVIEW_HOST}:${port}/readyz`;
const deadline = Date.now() + timeoutMs;
let lastError: unknown;
while (Date.now() < deadline) {
if (previewServer.pid !== undefined && !isProcessRunning(previewServer.pid)) {
throw new SystemError(
`${serverName} exited before becoming ready. Last output:\n${
previewServer.getOutput() || '<empty>'
}`
);
}
try {
const response = await turtleFetch(readyUrl, 'GET', {
retries: 0,
timeout: 2_000,
});
const ready = WebPreviewReadyResponseSchema.parse(await response.json());
return ready.device;
} catch (error) {
lastError = error;
}
await sleepAsync(1_000);
}
throw new SystemError(
`Timed out waiting for ${serverName} readiness at ${readyUrl}${
lastError instanceof Error ? `: ${lastError.message}` : ''
}. Last output:\n${previewServer.getOutput() || '<empty>'}`
);
}
export type DeviceWebPreviewHandle = {
previewPageUrl: string;
apiUrl: string;
/** Session token gating the preview. Only serve-sim mints one. */
previewToken?: string;
stopAsync: () => Promise<void>;
};
export type ServeSimPreviewHandle = DeviceWebPreviewHandle;
async function startWebPreviewWithTunnelAsync(
ctx: CustomBuildContext,
{
baseDomain,
env,
logger,
timeoutMs,
serverName,
packageSpec,
createArgs,
readPreviewTokenAsync,
}: {
baseDomain: string;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
serverName: string;
packageSpec: string;
createArgs: (port: number, turnArgs: string[], previewPageUrl: string) => string[];
readPreviewTokenAsync?: (device: string) => Promise<string>;
}
): Promise<DeviceWebPreviewHandle> {
const subdomainId = randomBytes(16).toString('hex');
const previewPageUrl = simulatorPreviewPageUrl(env, subdomainId);
const port = await findAvailablePortAsync();
const turnArgs = await fetchWebPreviewTurnArgsAsync(ctx, { env, logger });
const previewExec = resolvePackageExec(
resolveConfiguredPackageManager(env, PackageManager.NPM),
createArgs(port, turnArgs, previewPageUrl)
);
logger.info(
`Launching ${packageSpec} on ${WEB_PREVIEW_HOST}:${port} via ${previewExec.command}.`
);
const previewServer = spawnDetached({
command: previewExec.command,
args: previewExec.args,
env,
});
try {
logger.info(`Waiting for ${serverName} to become ready.`);
const device = await waitForWebPreviewReadyAsync({
previewServer,
serverName,
port,
timeoutMs,
});
const previewToken = await readPreviewTokenAsync?.(device);
const tunnel = await startNgrokTunnelAsync({
port,
subdomainPrefix: 'web-preview',
subdomainId,
baseDomain,
authtoken: getNgrokAuthtokenOrThrow(env),
logger,
});
return {
previewPageUrl,
apiUrl: tunnel.url,
previewToken,
stopAsync: async () => {
const results = await Promise.allSettled([tunnel.stopAsync(), previewServer.stopAsync()]);
for (const result of results) {
if (result.status === 'rejected') {
logger.warn({ err: result.reason }, `Could not stop a ${serverName} preview resource.`);
}
}
},
};
} catch (error) {
await previewServer.stopAsync();
throw error;
}
}
export async function readServeSimPreviewTokenAsync(
udid: string,
stateDir: string = SERVE_SIM_STATE_DIR
): Promise<string | undefined> {
const servers = await readServeSimServersAsync(stateDir);
return servers.find(server => server.udid === udid)?.token;
}
export async function startServeSimWithTunnelAsync(
ctx: CustomBuildContext,
{
baseDomain,
env,
logger,
timeoutMs,
packageVersion,
launchAppIdentifier,
launchArgs,
openUrl,
}: {
baseDomain: string;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
packageVersion?: string;
} & ServeSimLaunchOptions
): Promise<ServeSimPreviewHandle> {
const websiteArgs = websiteOriginServeSimArgs(env);
return await startWebPreviewWithTunnelAsync(ctx, {
baseDomain,
env,
logger,
timeoutMs,
serverName: 'serve-sim',
packageSpec: createServeSimPackageSpec(packageVersion),
createArgs: (port, turnArgs, previewPageUrl) =>
createServeSimArgs({
port,
turnArgs,
websiteArgs,
shareUrl: previewPageUrl,
packageVersion,
launchAppIdentifier,
launchArgs,
openUrl,
}),
readPreviewTokenAsync: async device => {
const previewToken = await readServeSimPreviewTokenAsync(device);
if (!previewToken) {
// A serve-sim that does not know --require-token fails earlier, in the readiness check, so
// reaching here means it started and left no token in its state file.
throw new SystemError(
`serve-sim became ready but wrote no session token for device ${device}. The preview is ` +
'on a public tunnel and would be reachable without one, so the session cannot continue. ' +
'This usually means the state file was not written as expected; retry the session, and ' +
'report it if it repeats.'
);
}
return previewToken;
},
});
}
export async function startExpoDeviceHubWithTunnelAsync(
ctx: CustomBuildContext,
{
runtimePlatform,
baseDomain,
env,
logger,
timeoutMs,
packageVersion,
}: {
runtimePlatform: BuildRuntimePlatform;
baseDomain: string;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
packageVersion?: string;
}
): Promise<DeviceWebPreviewHandle> {
if (runtimePlatform === BuildRuntimePlatform.LINUX) {
await ensureFfmpegInstalledOnceAsync({ runtimePlatform, env, logger });
}
return await startWebPreviewWithTunnelAsync(ctx, {
baseDomain,
env,
logger,
timeoutMs,
serverName: 'expo-device-hub',
packageSpec: createExpoDeviceHubPackageSpec(packageVersion),
createArgs: (port, turnArgs) => createExpoDeviceHubArgs({ port, turnArgs, packageVersion }),
});
}
export async function startDeviceWebPreviewWithTunnelAsync(
ctx: CustomBuildContext,
{
runtimePlatform,
launchAppIdentifier,
launchArgs,
openUrl,
...options
}: {
runtimePlatform: BuildRuntimePlatform;
baseDomain: string;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
packageVersion?: string;
} & ServeSimLaunchOptions
): Promise<DeviceWebPreviewHandle> {
switch (runtimePlatform) {
case BuildRuntimePlatform.DARWIN:
return await startServeSimWithTunnelAsync(ctx, {
...options,
launchAppIdentifier,
launchArgs,
openUrl,
});
case BuildRuntimePlatform.LINUX:
// Unreachable from the three step functions, which reject a non-Darwin launch while
// parsing. Kept because this function is exported and expo-device-hub cannot launch.
if (launchAppIdentifier) {
throw new UserError(
'EAS_LAUNCH_APPLICATION_INVALID_INPUT',
`Cannot launch ${launchAppIdentifier}: an application launch runs through serve-sim on an iOS simulator, and this session runs expo-device-hub on ${runtimePlatform}.`
);
}
return await startExpoDeviceHubWithTunnelAsync(ctx, { ...options, runtimePlatform });
}
}
export type NgrokTunnelHandle = {
url: string;
subdomainId: string;
@@ -0,0 +1,53 @@
import { type bunyan } from '@expo/logger';
import { createReadStream } from 'node:fs';
import { stat } from 'node:fs/promises';
import { type CustomBuildContext } from '../../customBuildContext';
import { uploadDeviceRunSessionArtifactAsync } from './deviceRunSessionArtifacts';
export async function uploadServeSimLogsFileAsync(
ctx: CustomBuildContext,
{
deviceRunSessionId,
udid,
filePath,
logger,
signal,
}: {
deviceRunSessionId: string;
udid: string;
filePath: string;
logger: bunyan;
signal?: AbortSignal;
}
): Promise<void> {
let stream: ReturnType<typeof createReadStream> | undefined;
try {
const { size } = await stat(filePath);
if (size === 0) {
return;
}
const fileController = new AbortController();
stream = createReadStream(filePath);
// A read error can occur while the upload session is still being allocated.
stream.on('error', err => fileController.abort(err));
await uploadDeviceRunSessionArtifactAsync(ctx, {
deviceRunSessionId,
artifactId: `simulator-log-${udid}`,
name: `App logs (${udid.slice(0, 8)})`,
filename: 'app-logs.ndjson',
kind: 'simulator-log',
metadata: { __eas_type: 'simulator-log', udid, scope: 'user-apps', source: 'serve-sim/logs' },
size,
stream,
signal: AbortSignal.any([fileController.signal, ...(signal ? [signal] : [])]),
});
} catch (err) {
logger.warn(
{ err },
`Could not upload simulator logs for ${udid}; other artifacts will continue.`
);
} finally {
stream?.destroy();
}
}
@@ -0,0 +1,316 @@
import { type bunyan } from '@expo/logger';
import fetch from 'node-fetch';
import { mkdtemp, open } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { type Readable } from 'node:stream';
import { setTimeout as delay } from 'node:timers/promises';
import { z } from 'zod';
import { Sentry } from '../../sentry';
import { SERVE_SIM_STATE_DIR, readServeSimServersAsync } from './serveSimMetricsRecorder';
// Require user-app scope acknowledgement; replay cursors avoid duplicate records
// on reconnect. Older servers that ignore the filter are not safe to collect from.
const MAX_BYTES_PER_DEVICE = 20 * 1024 * 1024;
const MAX_DURATION_MS = 30 * 60 * 1000;
const MAX_LINE_LENGTH = 1024 * 1024;
const MAX_CONSECUTIVE_FAILURES = 10;
const ServeSimLogEnvelopeSchema = z.object({
seq: z.number().int().nonnegative(),
raw: z.string(),
});
type CollectedLog = { udid: string; filePath: string };
type Session = {
controller: AbortController;
timer: NodeJS.Timeout;
polling: Promise<void>;
streams: Map<string, Promise<void>>;
files: Map<string, CollectedLog>;
finishing?: Promise<CollectedLog[]>;
};
let activeSession: Session | undefined;
export namespace ServeSimLogsRecorder {
export async function startAsync({
logger,
stateDir = SERVE_SIM_STATE_DIR,
pollIntervalMs = 2_000,
maxBytes = MAX_BYTES_PER_DEVICE,
maxDurationMs = MAX_DURATION_MS,
}: {
logger: bunyan;
stateDir?: string;
pollIntervalMs?: number;
maxBytes?: number;
maxDurationMs?: number;
}): Promise<void> {
if (activeSession) {
return;
}
const controller = new AbortController();
const session: Session = {
controller,
timer: setTimeout(() => controller.abort(), maxDurationMs),
polling: Promise.resolve(),
streams: new Map(),
files: new Map(),
};
session.timer.unref();
activeSession = session;
// Reserve the session before asynchronous setup so concurrent starts are harmless.
session.polling = (async () => {
const outputDirectory = await mkdtemp(path.join(os.tmpdir(), 'serve-sim-logs-'));
const failures = new Map<string, number>();
const bytes = new Map<string, number>();
const capped = new Set<string>();
const cursors = new Map<string, { identity: string; sequence?: number }>();
while (!controller.signal.aborted) {
const servers = await readServeSimServersAsync(stateDir);
const present = new Set(servers.map(server => server.udid));
for (const udid of failures.keys()) {
if (!present.has(udid)) {
failures.delete(udid);
cursors.delete(udid);
}
}
for (const server of servers) {
if (controller.signal.aborted) {
break;
}
if (session.streams.has(server.udid) || capped.has(server.udid)) {
continue;
}
const filePath = path.join(outputDirectory, `${encodeURIComponent(server.udid)}.ndjson`);
const identity = JSON.stringify([server.url, server.token]);
let cursor = cursors.get(server.udid);
if (cursor?.identity !== identity) {
cursor = { identity };
cursors.set(server.udid, cursor);
failures.delete(server.udid);
}
if ((failures.get(server.udid) ?? 0) >= MAX_CONSECUTIVE_FAILURES) {
continue;
}
failures.set(server.udid, (failures.get(server.udid) ?? 0) + 1);
const done = streamServeSimLogsToFileAsync({
serveSimUrl: server.url,
serveSimToken: server.token,
serveSimDevice: server.udid,
since: cursor?.sequence,
filePath,
signal: controller.signal,
logger,
maxBytes: maxBytes - (bytes.get(server.udid) ?? 0),
})
.then(result => {
bytes.set(server.udid, (bytes.get(server.udid) ?? 0) + result.bytesWritten);
if (result.lastSequence !== undefined && cursors.get(server.udid) === cursor) {
cursors.set(server.udid, { identity, sequence: result.lastSequence });
}
if (result.receivedData) {
session.files.set(server.udid, { udid: server.udid, filePath });
failures.set(server.udid, 0);
if (result.lastSequence === undefined) {
capped.add(server.udid);
if (!controller.signal.aborted && !result.limitReached) {
logger.warn(
`Simulator log stream for ${server.udid} ended without a replay cursor; retaining collected logs without reconnecting.`
);
}
}
}
if (result.limitReached) {
capped.add(server.udid);
logger.warn(`serve-sim simulator logs reached the size limit for ${server.udid}.`);
}
})
.finally(() => session.streams.delete(server.udid));
session.streams.set(server.udid, done);
}
await delay(pollIntervalMs, undefined, { signal: controller.signal }).catch(() => {});
}
})().catch(err => {
logger.warn({ err }, 'Could not collect serve-sim simulator logs.');
controller.abort();
});
logger.info('Started collecting user-app logs (up to 20 MiB per device and 30 minutes).');
}
export async function finishAsync({ logger }: { logger: bunyan }): Promise<CollectedLog[]> {
const session = activeSession;
if (!session || session.finishing) {
// Only the first finalizer receives files, preventing duplicate uploads.
await session?.finishing;
return [];
}
session.finishing = (async () => {
clearTimeout(session.timer);
session.controller.abort();
await session.polling;
await Promise.all(session.streams.values());
logger.info('Finished collecting serve-sim simulator logs.');
return [...session.files.values()];
})();
try {
return await session.finishing;
} finally {
activeSession = undefined;
}
}
}
export async function streamServeSimLogsToFileAsync({
serveSimUrl,
serveSimToken,
serveSimDevice,
since,
filePath,
signal,
logger,
maxBytes = MAX_BYTES_PER_DEVICE,
}: {
serveSimUrl: string;
serveSimToken?: string;
serveSimDevice?: string;
since?: number;
filePath: string;
signal: AbortSignal;
logger: bunyan;
maxBytes?: number;
}): Promise<{
receivedData: boolean;
limitReached: boolean;
bytesWritten: number;
lastSequence?: number;
}> {
let bytesWritten = 0;
let limitReached = maxBytes <= 0;
let lastSequence = since;
let malformedRecords = 0;
let file: Awaited<ReturnType<typeof open>> | undefined;
let body: NodeJS.ReadableStream | undefined;
const closeBody = (): void => {
(body as Readable | undefined)?.destroy?.();
};
try {
if (signal.aborted || limitReached) {
return { receivedData: false, limitReached, bytesWritten };
}
const url = new URL('/logs', serveSimUrl);
url.searchParams.set('envelope', 'true');
url.searchParams.set('scope', 'user-apps');
if (since !== undefined) {
url.searchParams.set('since', String(since));
}
if (serveSimDevice) {
url.searchParams.set('device', serveSimDevice);
}
const response = await fetch(url.toString(), {
signal,
...(serveSimToken ? { headers: { Authorization: `Bearer ${serveSimToken}` } } : {}),
});
body = response.body;
signal.addEventListener('abort', closeBody, { once: true });
if (signal.aborted) {
closeBody();
return { receivedData: false, limitReached, bytesWritten };
}
if (!response.ok || !response.body) {
logger.warn(`serve-sim /logs responded ${response.status}; simulator logs will be retried.`);
return { receivedData: false, limitReached, bytesWritten };
}
// Older servers ignore unknown query parameters and would return system logs.
if (response.headers.get('x-serve-sim-log-scope') !== 'user-apps') {
logger.warn(
'serve-sim did not confirm user-app log filtering. Skipping this stream; use a serve-sim version that supports /logs?scope=user-apps.'
);
return { receivedData: false, limitReached, bytesWritten };
}
file = await open(filePath, 'a');
const decoder = new TextDecoder();
let buffer = '';
let droppingLine = false;
stream: for await (const chunk of response.body) {
buffer +=
typeof chunk === 'string' ? chunk : decoder.decode(chunk as Buffer, { stream: true });
let newline: number;
while ((newline = buffer.indexOf('\n')) !== -1) {
const line = buffer.slice(0, newline).replace(/\r$/, '');
buffer = buffer.slice(newline + 1);
const skip = droppingLine || line.length > MAX_LINE_LENGTH;
droppingLine = false;
if (skip || !line.startsWith('data:')) {
continue;
}
let payload = line.slice(5).trim();
let sequence: number | undefined;
try {
const envelope = ServeSimLogEnvelopeSchema.safeParse(JSON.parse(payload));
if (envelope.success) {
sequence = envelope.data.seq;
if (lastSequence !== undefined && sequence <= lastSequence) {
continue;
}
// Keep the downloaded artifact homogeneous across old and new servers.
payload = JSON.stringify(JSON.parse(envelope.data.raw));
}
} catch {
// Skip the record so the rest of the stream is kept; the count is reported below.
malformedRecords += 1;
continue;
}
const record = payload + '\n';
const size = Buffer.byteLength(record);
if (bytesWritten + size > maxBytes) {
limitReached = true;
break stream;
}
// Await writes to bound buffering, including during teardown; close waits for these writes.
await file.writeFile(record);
bytesWritten += size;
if (sequence !== undefined) {
lastSequence = sequence;
}
if (bytesWritten >= maxBytes) {
limitReached = true;
break stream;
}
}
if (buffer.length > MAX_LINE_LENGTH) {
buffer = '';
droppingLine = true;
}
}
} catch (err) {
if (!signal.aborted) {
logger.warn(
{ err },
'serve-sim simulator log stream ended; collected logs will be retained.'
);
}
} finally {
if (malformedRecords > 0) {
Sentry.capture('serve-sim sent malformed simulator log records', {
extras: { malformedRecords, serveSimDevice },
});
logger.warn(
{ malformedRecords },
'serve-sim sent simulator log records that are not valid JSON; they were skipped.'
);
}
signal.removeEventListener('abort', closeBody);
closeBody();
await file?.close().catch(err => {
logger.warn({ err }, 'Could not close the simulator log file.');
});
}
return {
receivedData: bytesWritten > 0,
limitReached,
bytesWritten,
...(lastSequence !== undefined ? { lastSequence } : {}),
};
}
@@ -0,0 +1,30 @@
import { graphqlAbortContext } from '../graphqlAbort';
const realFetch = globalThis.fetch;
afterEach(() => {
globalThis.fetch = realFetch;
});
it('returns nothing without a signal', () => {
expect(graphqlAbortContext(undefined)).toBeUndefined();
});
it('aborts the request when either the caller or urql aborts', async () => {
const seen: AbortSignal[] = [];
globalThis.fetch = jest.fn(async (_input: RequestInfo | URL, init?: RequestInit) => {
seen.push(init!.signal!);
return new Response('{}');
}) as typeof fetch;
const caller = new AbortController();
const urql = new AbortController();
const context = graphqlAbortContext(caller.signal)!;
await context.fetch!('https://api.test/graphql', { method: 'POST', signal: urql.signal });
await context.fetch!('https://api.test/graphql', { method: 'POST' });
expect(seen).toHaveLength(2);
expect(seen.every(signal => !signal.aborted)).toBe(true);
urql.abort(new Error('urql gave up'));
expect(seen[0].aborted).toBe(true);
expect(seen[1].aborted).toBe(false);
caller.abort(new Error('caller gave up'));
expect(seen[1].aborted).toBe(true);
});
@@ -0,0 +1,21 @@
import type { OperationContext } from '@urql/core';
/**
* urql replaces `fetchOptions.signal` with its own, so a caller's signal has to be joined into
* fetch after urql has resolved auth and its signal. Returns nothing without a signal, so the
* client's default fetch stays in use.
*/
export function graphqlAbortContext(
signal: AbortSignal | undefined
): Pick<OperationContext, 'fetch'> | undefined {
if (!signal) {
return undefined;
}
return {
fetch: (input, init) =>
fetch(input, {
...init,
signal: init?.signal ? AbortSignal.any([init.signal, signal]) : signal,
}),
};
}
+60 -25
View File
@@ -1,36 +1,71 @@
import { EventEmitter } from 'events';
import type { ora as oraType } from '../ora';
jest.unmock('ora');
import { ora } from '../ora';
class FakeTTYStream extends EventEmitter {
isTTY = true;
columns = 200;
clearLineCalls = 0;
function createFakeTtyStream(columns: number): NodeJS.WriteStream & { clearedLines: number } {
return {
isTTY: true,
columns,
clearedLines: 0,
write: jest.fn(),
moveCursor: jest.fn(),
cursorTo: jest.fn(),
clearLine(this: { clearedLines: number }) {
this.clearedLines++;
},
} as unknown as NodeJS.WriteStream & { clearedLines: number };
write(): boolean {
return true;
}
clearLine(): boolean {
this.clearLineCalls++;
return true;
}
cursorTo(): boolean {
return true;
}
moveCursor(): boolean {
return true;
}
}
describe(ora, () => {
it('clears all wrapped lines after the terminal gets narrower', () => {
const stream = createFakeTtyStream(200);
const spinner = ora({ text: 'x'.repeat(100), stream, isEnabled: true });
describe('ora', () => {
const originalEnv = process.env;
const originalIsTTY = process.stdin.isTTY;
let ora: typeof oraType;
spinner.render();
spinner.render();
expect(stream.clearedLines).toBe(1);
beforeEach(() => {
jest.useFakeTimers();
process.env = { ...originalEnv };
delete process.env.CI;
delete process.env.EXPO_DEBUG;
Object.defineProperty(process.stdin, 'isTTY', { value: true, writable: true });
jest.isolateModules(() => {
ora = require('../ora').ora;
});
});
stream.columns = 40;
// The frame on screen was written at the old width, so it is cleared with the old line count.
afterEach(() => {
jest.useRealTimers();
process.env = originalEnv;
Object.defineProperty(process.stdin, 'isTTY', { value: originalIsTTY, writable: true });
});
it('clears every wrapped line after the terminal gets narrower', () => {
const stream = new FakeTTYStream();
const spinner = ora({
text: 'x'.repeat(150),
stream: stream as unknown as NodeJS.WritableStream,
discardStdin: false,
}).start();
stream.columns = 80;
stream.emit('resize');
spinner.render();
stream.clearedLines = 0;
stream.clearLineCalls = 0;
spinner.render();
// "- " + 100 characters wraps to 3 lines at 40 columns.
expect(stream.clearedLines).toBe(3);
expect(stream.clearLineCalls).toBe(2);
spinner.stop();
expect(stream.listenerCount('resize')).toBe(0);
});
});
@@ -1,4 +1,11 @@
import { validateWorkflowStructure } from '../validation';
import { vol } from 'memfs';
import { WorkflowRevisionMutation } from '../../../graphql/mutations/WorkflowRevisionMutation';
import { ExpoGraphqlClient } from '../../context/contextUtils/createGraphqlClient';
import { validateWorkflowFileAsync, validateWorkflowStructure } from '../validation';
jest.mock('fs');
jest.mock('../../../graphql/mutations/WorkflowRevisionMutation');
const workflowSchema = {
type: 'object',
@@ -60,3 +67,72 @@ describe(validateWorkflowStructure, () => {
}).toThrow('must be a valid URI string');
});
});
const schemaWithUsesJob = {
type: 'object',
properties: {
jobs: {
type: 'object',
additionalProperties: {
anyOf: [
{
type: 'object',
properties: {
type: { type: 'string', const: 'slack' },
params: { type: 'object' },
},
required: ['type', 'params'],
additionalProperties: false,
},
{
type: 'object',
properties: {
uses: { type: 'string' },
with: { type: 'object' },
},
required: ['uses'],
additionalProperties: false,
},
],
},
},
},
required: ['jobs'],
additionalProperties: false,
};
describe(validateWorkflowFileAsync, () => {
const schemaPath = '/schema.json';
const graphqlClient = {} as ExpoGraphqlClient;
beforeEach(() => {
vol.reset();
vol.fromJSON({ [schemaPath]: JSON.stringify({ data: schemaWithUsesJob }) });
process.env.EXPO_TESTING_WORKFLOW_SCHEMA_PATH = schemaPath;
jest.mocked(WorkflowRevisionMutation.validateWorkflowYamlConfigAsync).mockResolvedValue();
});
afterEach(() => {
delete process.env.EXPO_TESTING_WORKFLOW_SCHEMA_PATH;
});
const validateAsync = (yamlConfig: string): Promise<void> =>
validateWorkflowFileAsync(
{ yamlConfig, filePath: 'workflow.yml' },
'/project',
graphqlClient,
'project-id'
);
it('accepts a valid workflow when the schema has a job entry without a type', async () => {
await expect(
validateAsync('jobs:\n notify:\n type: slack\n params:\n message: hi\n')
).resolves.toBeUndefined();
});
it('rejects unknown job types when the schema has a job entry without a type', async () => {
await expect(validateAsync('jobs:\n bad:\n type: not-a-job\n')).rejects.toThrow(
'The following jobs have invalid types: bad. Valid types are: slack'
);
});
});
@@ -206,7 +206,9 @@ function jobsFromWorkflow(parsedYaml: any): any[] {
}
function jobTypesFromWorkflowSchema(workflowJsonSchema: any): string[] {
return workflowJsonSchema?.properties?.jobs?.additionalProperties?.anyOf.map(
(props: any) => props.properties.type.const
);
const jobSchemas: any[] = workflowJsonSchema?.properties?.jobs?.additionalProperties?.anyOf ?? [];
// Some job schemas (for example, `uses:` jobs) do not declare a `type`.
return jobSchemas
.map(jobSchema => jobSchema?.properties?.type?.const)
.filter((type): type is string => typeof type === 'string');
}
@@ -476,6 +476,20 @@ describe(Simulator, () => {
expect(mockResetSimulatorEnvAsync).toHaveBeenCalledWith(projectDir, 'session-123');
});
it('keeps the live spinner text short and prints the stop command once', async () => {
mockByIdAsync
.mockResolvedValueOnce(makeDeviceRunSession())
.mockResolvedValueOnce(makeDeviceRunSession({ status: DeviceRunSessionStatus.Stopped }));
const { command } = createCommand(['--platform', 'ios']);
await command.runAsync();
expect(Log.log).toHaveBeenCalledWith(
'To stop the session from another shell, run: eas simulator:stop --id session-123'
);
expect(mockOra).toHaveBeenLastCalledWith('Simulator session active — press Ctrl+C to stop');
});
it('forwards --name to the create mutation', async () => {
const { command } = createCommand([
'--platform',
@@ -187,7 +187,11 @@ describe(SimulatorList, () => {
after: 'page-cursor',
filter: {
statuses: [DeviceRunSessionStatus.InProgress, DeviceRunSessionStatus.New],
types: [DeviceRunSessionType.Appium, DeviceRunSessionType.WebPreviewOnly],
types: [
DeviceRunSessionType.Appium,
DeviceRunSessionType.ServeSim,
DeviceRunSessionType.WebPreviewOnly,
],
platforms: [AppPlatform.Ios],
name: 'checkout',
tags: ['variant:pro', 'nightly'],
@@ -35,6 +35,7 @@ import {
DEVICE_RUN_SESSION_RESOURCE_CLASS_BY_FLAG_VALUE,
DEVICE_RUN_SESSION_RESOURCE_CLASS_FLAG_VALUES,
DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE,
DEVICE_RUN_SESSION_TYPE_FLAG_OPTIONS,
DEVICE_RUN_SESSION_TYPE_FLAG_VALUES,
DeviceRunSessionRemoteConfig,
formatRemoteSessionInstructions,
@@ -120,7 +121,7 @@ export default class Simulator extends EasCommand {
type: Flags.option({
description:
'Type of simulator session to create. All session types include a web preview. agent-device, appium, and argent also include an automation interface; web-preview-only includes no automation interface.',
options: Object.values(DEVICE_RUN_SESSION_TYPE_FLAG_VALUES),
options: DEVICE_RUN_SESSION_TYPE_FLAG_OPTIONS,
default: DEVICE_RUN_SESSION_TYPE_FLAG_VALUES[DeviceRunSessionType.AgentDevice],
})(),
'package-version': Flags.string({
@@ -562,9 +563,12 @@ async function waitForSessionEndOrInterruptAsync({
projectDir: string;
sessionInterrupt: SessionInterrupt;
}): Promise<void> {
const spinner = ora(
`Simulator session active — press Ctrl+C to stop, or run \`eas simulator:stop --id ${deviceRunSessionId}\` from another shell`
).start();
Log.log(
`To stop the session from another shell, run: eas simulator:stop --id ${deviceRunSessionId}`
);
// Keep the spinner text short so it fits on one line. When the text wraps, resizing the
// terminal can leave copies of the spinner line behind.
const spinner = ora('Simulator session active — press Ctrl+C to stop').start();
const { signal } = sessionInterrupt;
try {
@@ -17,9 +17,9 @@ import { DeviceRunSessionQuery } from '../../graphql/queries/DeviceRunSessionQue
import Log, { link } from '../../log';
import { ora } from '../../ora';
import {
DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE,
DEVICE_RUN_SESSION_TYPE_FLAG_VALUES,
DEVICE_RUN_SESSION_TYPE_FLAG_OPTIONS,
deviceRunSessionTypeToFlagValue,
deviceRunSessionTypesForFlagValue,
} from '../../simulator/utils';
import { fromNow } from '../../utils/date';
import { enableJsonOutput, printJsonOnlyOutput } from '../../utils/json';
@@ -62,7 +62,7 @@ export default class SimulatorList extends EasCommand {
type: Flags.option({
description:
'Filter by session type (repeatable). All session types include a web preview. agent-device, appium, and argent also include an automation interface; web-preview-only includes no automation interface.',
options: Object.values(DEVICE_RUN_SESSION_TYPE_FLAG_VALUES),
options: DEVICE_RUN_SESSION_TYPE_FLAG_OPTIONS,
multiple: true,
})(),
platform: Flags.option({
@@ -110,7 +110,7 @@ export default class SimulatorList extends EasCommand {
filter.statuses = flags.status.map(value => STATUS_BY_FLAG_VALUE[value]);
}
if (flags.type && flags.type.length > 0) {
filter.types = flags.type.map(value => DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE[value]);
filter.types = flags.type.flatMap(deviceRunSessionTypesForFlagValue);
}
if (flags.platform && flags.platform.length > 0) {
filter.platforms = flags.platform.map(value => PLATFORM_BY_FLAG_VALUE[value]);
@@ -12,6 +12,7 @@ import {
SubmissionAndroidReleaseStatus,
} from '../../graphql/generated';
import { AppStoreConnectApiKeyQuery } from '../../graphql/queries/AppStoreConnectApiKeyQuery';
import { AppQuery } from '../../graphql/queries/AppQuery';
import { GoogleServiceAccountKeyQuery } from '../../graphql/queries/GoogleServiceAccountKeyQuery';
import AndroidSubmitCommand from '../../submit/android/AndroidSubmitCommand';
import { SubmissionContext, createSubmissionContextAsync } from '../../submit/context';
@@ -41,11 +42,19 @@ export default class SubmitInternal extends EasCommand {
description: 'ID of the build to submit',
required: true,
}),
'project-id': Flags.string({
description: 'EAS project ID. With --application-identifier, skips loading app config.',
dependsOn: ['application-identifier'],
}),
'application-identifier': Flags.string({
description: 'Bundle identifier or Android package name read from the build artifact.',
dependsOn: ['project-id'],
}),
};
static override contextDefinition = {
...this.ContextOptions.LoggedIn,
...this.ContextOptions.ProjectConfig,
...this.ContextOptions.DynamicProjectConfig,
...this.ContextOptions.ProjectDir,
...this.ContextOptions.Analytics,
...this.ContextOptions.Vcs,
@@ -58,7 +67,8 @@ export default class SubmitInternal extends EasCommand {
const {
loggedIn: { actor, graphqlClient },
privateProjectConfig: { exp, projectId, projectDir },
projectDir,
getDynamicPrivateProjectConfigAsync,
analytics,
vcsClient,
} = await this.getContextAsync(SubmitInternal, {
@@ -66,6 +76,15 @@ export default class SubmitInternal extends EasCommand {
withServerSideEnvironment: null,
});
const project = flags['project-id']
? await AppQuery.byIdAsync(graphqlClient, flags['project-id'])
: undefined;
// Worker submissions already know the project and artifact identifiers.
// The existing submission code only needs the project name and slug here.
const { exp, projectId } = project
? { exp: { name: project.name, slug: project.slug }, projectId: project.id }
: await getDynamicPrivateProjectConfigAsync();
if (vcsClient instanceof GitClient) {
// `build:internal` is run on EAS workers and the repo may have been changed
// by pre-install hooks or other scripts. We don't want to require committing changes
@@ -93,6 +112,7 @@ export default class SubmitInternal extends EasCommand {
analytics,
exp,
projectId,
applicationIdentifier: flags['application-identifier'],
vcsClient,
specifiedProfile: flags.profile,
groups: undefined, // use groups from submit profile
+14 -12
View File
@@ -30,27 +30,25 @@ export function isSpinnerEnabled(): boolean {
export function ora(options?: Options | string): Ora {
const inputOptions = typeof options === 'string' ? { text: options } : (options ?? {});
const disabled = !isSpinnerEnabled();
// In non-interactive mode, send the stream to stdout so it prevents looking like an error.
const stream: NodeJS.WritableStream =
inputOptions.stream ?? (disabled ? process.stdout : process.stderr);
const spinner = oraReal({
// Ensure our non-interactive mode emulates CI mode.
isEnabled: !disabled,
// In non-interactive mode, send the stream to stdout so it prevents looking like an error.
stream: disabled ? process.stdout : process.stderr,
...inputOptions,
stream,
});
// ora only recounts the wrapped lines of its text when the text changes. After the
// terminal gets narrower, it clears too few lines and every frame leaves a copy behind.
const onResize = (): void => {
spinner.text = spinner.text;
};
const oraStart = spinner.start.bind(spinner);
const oraStop = spinner.stop.bind(spinner);
const oraStopAndPersist = spinner.stopAndPersist.bind(spinner);
const oraRender = spinner.render.bind(spinner);
spinner.render = (): Ora => {
// ora only recalculates how many terminal lines the text wraps to when the text changes.
// After the terminal is resized, the stale count makes every frame clear too few lines,
// so the spinner text gets printed over and over. Setting the text recalculates the count.
const { text } = spinner;
spinner.text = text;
return oraRender();
};
const logWrap = (method: any, args: any[]): void => {
oraStop();
@@ -105,6 +103,8 @@ export function ora(options?: Options | string): Ora {
// Skipping wrapping native logs removes the repeated interleaved "Exporting..." messages.
if (!disabled) {
wrapNativeLogs();
stream.off('resize', onResize);
stream.on('resize', onResize);
}
return oraStart(text);
@@ -113,12 +113,14 @@ export function ora(options?: Options | string): Ora {
spinner.stopAndPersist = (options): Ora => {
const result = oraStopAndPersist(options);
resetNativeLogs();
stream.off('resize', onResize);
return result;
};
spinner.stop = (): Ora => {
const result = oraStop();
resetNativeLogs();
stream.off('resize', onResize);
return result;
};
@@ -2,10 +2,12 @@ import { DeviceRunSessionResourceClass, DeviceRunSessionType } from '../../graph
import {
DEVICE_RUN_SESSION_RESOURCE_CLASS_BY_FLAG_VALUE,
DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE,
DEVICE_RUN_SESSION_TYPE_FLAG_OPTIONS,
DEVICE_RUN_SESSION_TYPE_FLAG_VALUES,
DeviceRunSessionRemoteConfig,
EAS_SIMULATOR_WAITLIST_URL,
deviceRunSessionTypeToFlagValue,
deviceRunSessionTypesForFlagValue,
formatLoopbackForwardNotice,
formatPreviewUrl,
formatRemoteSessionInstructions,
@@ -226,6 +228,23 @@ describe('simulator session type flags', () => {
expect(DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE['serve-sim']).toBeUndefined();
expect(DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE['web-preview']).toBeUndefined();
});
it('lists each flag value once', () => {
expect(DEVICE_RUN_SESSION_TYPE_FLAG_OPTIONS).toEqual([
'agent-device',
'appium',
'argent',
'web-preview-only',
]);
});
it('filters web-preview-only by the new and the legacy session type', () => {
expect(deviceRunSessionTypesForFlagValue('web-preview-only')).toEqual([
DeviceRunSessionType.ServeSim,
DeviceRunSessionType.WebPreviewOnly,
]);
expect(deviceRunSessionTypesForFlagValue('appium')).toEqual([DeviceRunSessionType.Appium]);
});
});
describe('simulator resource class flags', () => {
+13
View File
@@ -45,6 +45,19 @@ export const DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE = Object.fromEntries(
)
) as Record<string, DeviceRunSessionType>;
// Legacy ServeSim and WebPreviewOnly share a flag value, so list each value once.
export const DEVICE_RUN_SESSION_TYPE_FLAG_OPTIONS = [
...new Set(Object.values(DEVICE_RUN_SESSION_TYPE_FLAG_VALUES)),
];
// Every enum value behind a flag value, so filtering by web-preview-only also
// matches sessions created with the legacy ServeSim type.
export function deviceRunSessionTypesForFlagValue(value: string): DeviceRunSessionType[] {
return (Object.keys(DEVICE_RUN_SESSION_TYPE_FLAG_VALUES) as DeviceRunSessionType[]).filter(
type => DEVICE_RUN_SESSION_TYPE_FLAG_VALUES[type] === value
);
}
export function deviceRunSessionTypeToFlagValue(type: DeviceRunSessionType): string {
return DEVICE_RUN_SESSION_TYPE_FLAG_VALUES[type];
}
@@ -79,7 +79,6 @@ export async function getSessionUsingBrowserAuthFlowAsync({ sso = false }): Prom
`code_challenge=${codeChallenge}`,
`code_challenge_method=S256`,
`state=${state}`,
`confirm_account=true`,
].join('&');
return `${expoWebsiteUrl}${sso ? '/sso-login' : '/login'}?${params}`;
};