Compare commits

..
99 Commits
Author SHA1 Message Date
Expo CI 86b85e7d8d v24.9.0 2026-10-02 17:32:47 +00:00
Gabe DebesandClaude 03a7d3a6ee [eas-cli] Only forward --non-interactive from update:rollback in non-interactive mode (#4504)
* [eas-cli] only forward --non-interactive from update:rollback when it was passed

* [eas-cli] fix the changelog entry

Co-Authored-By: Claude <[email protected]>
2026-10-02 10:15:42 -07:00
Gabe DebesandClaude Opus 5.5 a21db49184 [eas-cli] Allow new updates to end active rollouts (#4233)
* [eas-cli] allow publishing over an in-progress rollout

* [eas-cli] support superseding a rollout with a new rollout

* [eas-cli] prompt before starting the publish spinner

* [eas-cli] group the active rollout warning by runtime version

* [eas-cli] cover the active rollout paths

* [eas-cli] explain why the superseded rollout serves more users

* [eas-cli] cover publishing without the rollout check

* [eas-cli] refuse rolling out over a rollout in progress

* [eas-cli] rewrite the rollout publish errors as what, why, how

* [eas-cli] render the active rollout warning with renderTextTable

* [eas-cli] regenerate the GraphQL schema against the deployed API

* [eas-cli] add the new required event name fields to observe test fixtures

* [eas-cli] preserve interactive rollout confirmation during rollback

* [eas-cli] tweak changelog

* [eas-cli] require --force-end-active-rollout instead of prompting

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-02 10:04:41 -07:00
Stanisław Chmiela 6dd6da083d [build-tools] Pass cancellation signal to download_build (#4511)
## Why

Just a small improvement.

## How

Added `signal` from step to `downloadBuild` func.

## Test Plan

CI should pass.
2026-10-02 16:39:52 +00:00
Stanisław Chmiela 656921d776 [build-tools] Cache and refresh ASC tokens for long submissions (#4497)
* [build-tools] refresh ASC tokens for long submissions

* [build-tools] link token fix changelog

* [build-tools] own ASC token caching in the API client

* [build-tools] load ASC keys outside the API client

* [build-tools] extract metadata key and client variables

* [build-tools] extract upload API key variable

* [build-tools] name ASC token lifetime and refresh margin
2026-10-02 18:34:24 +02:00
Szymon Świerk fabde0c2bb [build-tools] ASC API client: in beta group response, allow hasAccessToAllBuilds to be nullish (#4528)
* [build-tools] ASC API client: in beta group response, allow hasAccessToAllBuilds to be nullish

* [build-tools] Fill in PR number in changelog
2026-10-02 15:42:24 +02:00
Stanisław Chmiela 7680f33879 [build-tools] Explain missing App Store Connect agreements (#4513)
* [build-tools] Explain missing App Store Connect agreements

* [build-tools] Preserve extra Apple error fields and add changelog

* [build-tools] Handle aggregate App Store Connect errors in callers

* [build-tools] Give each App Store Connect request error a single code
2026-10-02 12:39:13 +02:00
Stanisław Chmiela 199a996b38 [eas-cli] Fix simulator status compatibility with the live GraphQL schema (#4527)
GraphQL got outdated.
2026-10-02 12:28:27 +02:00
Brent Vatne 4728c64d3f [eas-cli] Correct simulator:start --max-duration-minutes help text (#4524) 2026-10-01 19:27:23 -07:00
Brent Vatne 0fdf96e795 Support --egress local for Android emulator sessions (#4519) 2026-10-01 18:14:11 -07:00
Gabe DebesandClaude 04f3bee0c4 [build-tools] install mitmproxy and pass network capture options to iOS simulator sessions (#4307)
Co-authored-by: Claude <[email protected]>
2026-10-01 16:56:04 -07:00
Gabe DebesandClaude 140cfe445a [build-tools] Poll the simulator sessions web preview readiness every 250ms (#4496)
The host answers /readyz locally within milliseconds, but the step
slept 1 s between polls, so a ready host waited up to a second more
before the session went live.

Co-authored-by: Claude <[email protected]>
2026-10-01 16:30:53 -07:00
Yuval GnessinandClaude Opus 5.5 da55f26cac [eas-cli] Fix eas workflow:validate crashes and exit code (#4507)
* [eas-cli] Fix eas workflow:validate crashes and exit code

- Skip job-schema variants without a `type` (reusable-workflow calls)
  when listing valid job types; the live schema now includes one, which
  crashed every validation with "reading 'const'".
- Check `repack` job profiles only when `params.profile` is set; it is
  optional, and the unset case crashed with "reading 'includes'".
- Set a non-zero exit code when validation fails.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_015v5pSST6sHZiUiPkU7sq47
2026-10-01 15:00:18 -07:00
Douglas LowderandClaude Opus 5 e24130c6e0 [eas-cli] Add --upload-source-maps to eas update (#4436)
* [eas-cli] Add --upload-source-maps to eas update

Upload the source map that expo export emits for each native platform,
so EAS Observe can symbolicate stack traces from an update.

The maps are not listed in metadata.json, so each one is located from the
bundle that metadata.json does name, falling back to a single sibling
.map file. sourcesContent is always stripped, which keeps application
source text out of EAS storage and gives the map a chance of fitting
under the 5 MiB upload limit. A map that is still too large is skipped
with a warning; publishing never fails because of a source map.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-10-01 14:05:49 -07:00
Brent Vatne 2a0fe12836 [build-tools] Upload embedded bundle by default on SDK 58+ and rename env var (#4433) 2026-10-01 20:16:44 +00:00
SJ Kim 4c153da14d [eas-cli] Keep following simulator sessions in any status that isn't final (#4506)
* [eas-cli] Keep following simulator sessions in any status that isn't final (ENG-26991)

* [eas-cli] Add changelog entry for #4506 (ENG-26991)
2026-10-01 15:56:24 -04:00
Vojtech NovakandClaude Opus 5.5 81166df987 [build-tools] Upload manual preview screenshots as session artifacts (#4481)
* [build-tools] Upload manual preview screenshots as session artifacts

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_013nLsuzD7ThA1uv3dVHUFQp

* [build-tools] Address review findings on preview screenshot uploads

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_013nLsuzD7ThA1uv3dVHUFQp

* [build-tools] Report retained preview screenshots to Sentry once and throttle retry warnings

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_013nLsuzD7ThA1uv3dVHUFQp

* [build-tools] Remove duplicated preview screenshot tests

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_013nLsuzD7ThA1uv3dVHUFQp

* [build-tools] Give the host test's spawn mock the getExitError member main added

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_013nLsuzD7ThA1uv3dVHUFQp

* [build-tools] Report preview-side screenshot save failures to Sentry

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_013nLsuzD7ThA1uv3dVHUFQp

* [build-tools] Report preview screenshot save failures from the host's failure records

The host now writes a structured record into the watched directory instead of
the worker scanning its stderr at shutdown.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_013nLsuzD7ThA1uv3dVHUFQp

* [build-tools] Name the patterns used to build screenshot artifact names

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_013nLsuzD7ThA1uv3dVHUFQp

* [build-tools] Split the preview screenshot collector into single-purpose functions

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_013nLsuzD7ThA1uv3dVHUFQp

* [build-tools] Keep the preview screenshot directory while the session host still runs

The collector finished and removed an empty directory even when the host could not be stopped,
so later captures went into a directory no one watched. It now takes hostStopped, warns and keeps
the directory when the host is still running, and uses rmdir so a capture that lands after the
final scan is retained instead of deleted.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_013nLsuzD7ThA1uv3dVHUFQp

* [build-tools] Upload fresh preview screenshots before retrying failed ones

Each pass uploaded files in readdir order, so on macOS the oldest failed capture went first every
time and a stalled one held back newer captures and could use the whole shutdown budget. Captures
that never failed now go first, oldest first. The symlink test uses fixed names so the link sorts
before its target and a symlink-following regression always fails it.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_013nLsuzD7ThA1uv3dVHUFQp

* [build-tools] Keep signed upload URLs out of device run session artifact errors

node-fetch puts the request URL in its error message, and the artifact PUT goes to a signed URL,
so a dropped connection wrote the signature to the job log and, for retained screenshots, to
Sentry. Network errors from the PUT are now rethrown with the artifact ID and error code only,
without the original as the cause. They stay retryable.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_013nLsuzD7ThA1uv3dVHUFQp

* [build-tools] Tag preview screenshot artifacts with __eas_type

Recordings and performance metrics set __eas_type to their kind; screenshots did not.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_013nLsuzD7ThA1uv3dVHUFQp

* [build-tools] Make two preview screenshot tests assert what their names claim

The session-lookup fallback test now checks the fallback name and metadata, and the host test
checks that the screenshot directory still exists when the host is stopped, which fails if the
collector finishes first.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_013nLsuzD7ThA1uv3dVHUFQp

* [build-tools] Format the preview screenshot changes

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_013nLsuzD7ThA1uv3dVHUFQp

* [build-tools] Call leftover preview screenshots "not uploaded"

"Retained" suggested the files were kept safe, but they are destroyed with the worker. The
shutdown report now says "N preview screenshots were not uploaded.".

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-10-01 14:23:59 +02:00
Stanisław Chmiela ef189b00be [build-tools] Nicer TestFlight metadata update output (#4503)
# Why

While expanding `eas/update_testflight_metadata` to more users I kept a close eye on the rollout and the results. I think there's a few improvements we can make to the output of metadata updates.

# How

- moved group verification into the p-limited group-managing function so it doesn't block changelog from being updated
- made lack of primary locale NOT a blocker -- we now fall back to en-US
- changed processing logic so one failure does not abort all updates
- if a user lists a group that is going to get the build automatically distributed to them, we mark it as "ok" instead of throwing

# Test Plan

Tested live.
2026-10-01 14:20:50 +02:00
Szymon Świerk 27e56afd12 [eas-build-job] Add schema for workflow Sentry interpolation context (#4453)
* [eas-build-job] Add schema for workflow Sentry interpolation context

* [eas-build-job] Add assignee, actor, priority, and count fields to Sentry context schema
2026-10-01 14:18:29 +02:00
Szymon DziedzicandClaude Opus 5.5 1c342621c0 [build-tools] Add a single step for agent-device sessions that starts independent parts in parallel (#4491)
* [build-tools] Add a single step for agent-device sessions that starts independent parts in parallel

Add `eas/start_agent_device_session`. It replaces the boot step,
`eas/download_build`, `eas/install_build`, `eas/launch_application` and
`eas/start_agent_device_remote_session` for agent-device sessions, so the
parts that do not depend on each other run at the same time: the device
boot, the build download, ffmpeg on Android, and the agent-device daemon
and its tunnel. The session host starts after the boot, and the install
after the boot and the download. Each part logs its duration, and a
summary line is logged at ready.

The old steps keep their behavior and share the code:
- `bootIosSimulatorAsync` and `startAndroidEmulatorAsync` are extracted
  from the boot steps.
- `runAgentDeviceRemoteSessionAsync` is extracted from
  `eas/start_agent_device_remote_session`. It also starts the daemon and
  the session host in parallel there, and it now stops the daemon when
  its credentials never appear.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* [build-tools] Add changelog entry

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* [build-tools] Abort agent-device session startup on the first failure

Review follow-ups for `eas/start_agent_device_session`:

- `StartupTasks` now has an abort signal. The first failed task aborts
  it. The download passes it to the request and the file stream, the
  install task checks it before the install and the launch, and the
  session host stops waiting for the boot when it fires. So a failed
  startup stops quickly, its teardown cannot wait for a stalled download,
  and nothing installs or launches after a failure.
- The install task waits for the download to settle, so a failed step no
  longer returns while the download still runs.
- Remove the ffmpeg prefetch: the Linux image now ships ffmpeg, and the
  session host still checks for it.
- Each startup task logs through a child logger with a `startupTask`
  field, so the lines of parallel tasks can be told apart.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* [build-tools] Cancel the build lookup, extraction and daemon setup on startup abort

More review follow-ups for `eas/start_agent_device_session`. The startup
abort signal now reaches the work that can stall, not only the checks
between stages:

- The build-ID GraphQL lookup gets it through `graphqlAbortContext`.
- `decompressTarAsync` takes an optional signal. With one, it feeds the
  archive to a `tar.Unpack` that the abort stops. Without one, callers
  keep `tar.extract`. The archive scan does not start after an abort.
- The agent-device package install, the git clone and the git install
  get it, so an abort kills them. An abort does not trigger the git
  fallback, and the daemon does not start after one.
- The session stops waiting for the daemon credentials on an abort. The
  daemon is already stored, so the teardown stops it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* [build-tools] Let a started build extraction finish before reporting the abort

`tar.Unpack.abort()` rejects at once but cannot stop the file writes that
the extraction already started, so the download could settle while tar
still wrote files. `decompressTarAsync` goes back to `tar.extract`
unchanged. `downloadBuildAsync` checks the startup signal right before
and right after the extraction instead: an abort during the extraction
lets this local, bounded work finish and is reported right after it, so
no file is written after the download settles.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-01 12:52:20 +02:00
Expo Bot 9b7098cb58 [license] Update BSL Change Date to 2029-10-01 2026-10-01 00:59:39 +00:00
SJ Kim 9b247a6197 [build-tools] Collect shared simulator logs (#4463)
* [build-tools] Collect shared simulator logs (ENG-26642)

* [build-tools] Link simulator log changelog entry (ENG-26642)

* [build-tools] Require user-app scope for session logs (ENG-26642)

* [build-tools] Name simulator log artifacts app-logs.ndjson (ENG-26642)

* [build-tools] Time out simulator log uploads per device (ENG-26642)

* [build-tools] Report simulator log failures to Sentry (ENG-26642)

* [build-tools] Use a type import for the log stream body (ENG-26642)

* [build-tools] Validate serve-sim log envelopes and report malformed records (ENG-26642)
2026-09-30 16:25:48 -04:00
Stanisław Chmiela 862d0a8f09 [build-tools] Clear old Maestro HTML reports before each attempt (#4409)
It seems when auto-retrying an old HTML report may be uploaded instead of a more recent one. This clears reports between attempts.

(I think behavior of auto-retry + relying on HTML reports is a bit wonky so I guess users should then use the "retry-all" retry behavior.)
2026-09-30 09:15:04 +02:00
Szymon Dziedzic 417996ade0 [eas-cli] Keep the simulator session spinner on one line (#4488) 2026-09-29 21:41:58 +02:00
Stanisław Chmiela 1c6161432a [build-tools] Add "native" TestFlight metadata step (#4475)
# Why

While working on moving off of Submission Service to job runs I found a feature discrepancy — Submission Service supported changelog and assigning to groups, but job runs did not.

# How

Added an EAS function that uses ASC API to add changelog to build and build to groups. It more or less follows what Fastlane does.

See https://github.com/expo/universe/pull/31473.

# Test Plan

- The deployed worker ran the built-in step against an existing Apple build: [staging workflow](https://staging.expo.dev/accounts/sjchmiela/projects/staging-app/workflows/01a0d929-8130-74b2-b014-0b95431907c2).
- Full end-to-end staging test passed with a [fresh signed IPA](https://staging.expo.dev/accounts/sjchmiela/projects/staging-app/builds/45c69b90-649f-4908-a50a-49044e92b18d): [workflow](https://staging.expo.dev/accounts/sjchmiela/projects/staging-app/workflows/01a0d934-b692-71ad-9aba-f9acec8d3888). `eas/upload_to_asc` returned upload ID `8ee25971-3463-4d6d-856f-947645988a7a`. The built-in step selected Apple build `3880e9fc-bf7b-430c-a865-e399a1398667`. Separate Apple API reads confirmed the exact changelog, test-group membership, and no beta review submission.
2026-09-29 13:51:40 +00:00
Stanisław Chmiela 3f667ae4a8 [build-tools] Support maestro-runner HTML and Allure reports (#4406)
We're seeing customers have good experience with `maestro-runner`. Let's add some support for exporting HTML and Allure reports from it.

Runner report generation: https://github.com/devicelab-dev/maestro-runner/blob/main/pkg/cli/test.go
2026-09-29 15:17:03 +02:00
Szymon DziedzicandClaude Opus 5.5 4fb6de564c [build-tools] Disable agent-device internal idle timeouts in remote sessions (#4485)
* [build-tools] Disable agent-device internal idle timeouts in remote sessions

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* [build-tools] Add changelog entry

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-09-29 12:29:30 +02:00
SJ Kim 975693e96e [build-tools] Detect early Argent startup exits (#4454)
* [build-tools] Detect early Argent startup exits (ENG-22026)

* [build-tools] Observe startup exit before pipe closure (ENG-22026)
2026-09-28 23:10:40 -04:00
Brent Vatne b84233bf07 [eas-cli] Recount spinner lines when the terminal resizes (#4470) 2026-09-28 18:14:11 -07:00
Stanisław Chmiela 6dec2f9a13 [eas-cli] Allow preparing submission credentials without loading app config (#4471)
<!-- You can skip the changelog check by labeling the PR with "no changelog". -->

# Why

Having to load app config requires installing node modules which takes time and requires thinking about Node.js version, images etc.

# How

Added two optional parameters to `submit:internal` — `project-id` and `application-identifier`. With them provided we don't need to load project config, so we don't need to install dependencies!

# Test Plan

I'll land this, then land https://github.com/expo/universe/pull/31467 in staging and see if it works!
2026-09-28 21:15:19 +02:00
Vojtech NovakandClaude Fable 5.1 c15e184bb8 [ci] Install ffmpeg so the recording recovery test runs (#4441)
* [ci] Install ffmpeg so the recording recovery test runs

The build-tools test that recovers unlisted recordings from a killed
host cuts a real MP4 and needs ffprobe. It skipped on every CI run.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Fail the recording recovery test loudly without ffmpeg

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [ci] Update the ffmpeg install comment

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-28 12:57:58 +02:00
Vojtech NovakandClaude Fable 5.1 89740dbbfd [build-tools] Share the urql abort-signal wrapper (#4435)
* [build-tools] Add one helper that joins a caller's signal into urql fetch

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Use the shared urql abort helper in both call sites

The artifact upload-session mutation and the sandbox-ready mutation each
wrapped fetch by hand to join a cancellation signal after urql set its own.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-28 12:57:58 +02:00
Vojtech NovakandClaude Fable 5.1 390a165c57 [build-tools] Share remote session teardown across the automation steps (#4431)
* [build-tools] Cover a teardown failure after the session body failed

The error that ended the session must reach the step, not the teardown error.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Share remote session teardown across the automation steps

The Agent Device, Appium and Argent steps each ran Promise.allSettled over
their teardown and rethrew the first rejection from a finally block, which
replaced the error that had ended the session. One helper now runs every
teardown, logs each failure, and rethrows only when the session body succeeded.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Attempt every remote session teardown step independently

A rejected event collection stop no longer leaves the automation process
running. The tunnel close, the collector-then-process pair and the host
finish are separate teardown entries; the process stop sits in a finally.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Name each remote session teardown step in its failure log

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Report teardown failures swallowed after a failed session to Sentry

Co-Authored-By: Claude Fable 5.1 <[email protected]>

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-28 12:57:57 +02:00
Vojtech NovakandClaude Fable 5.1 f7fb43b974 [build-tools] Only stop a session host that never became ready (#4430)
A readiness timeout ran the full finish pipeline: a stop request to a server
that never came up and a read of a recordings.json that did not exist, each
logging a warning unrelated to the real cause. finishAsync now skips
finalization and upload for a host that never answered /readyz. Finalization
and upload failures are reported to Sentry like the per-recording upload
failures one layer down, and the finishAsync contract states that it never
rejects.


Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-28 12:57:57 +02:00
Vojtech NovakandClaude Fable 5.1 a32080fadf [build-tools] Bound recording shutdown and artifact uploads (#4420)
* [build-tools] Bound session shutdown and recording uploads

* [build-tools] Close HTTP requests after each artifact PUT

* [build-tools] Separate artifact retries from PUT cleanup

* [build-tools] Cover a session host that ignores SIGTERM

stopAsync must not resolve while the killed process still answers kill(pid, 0).

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Wait for a killed session host to be reaped

After SIGKILL the child is a zombie until libuv reaps it on a later loop turn,
so the running check right after stopAsync always saw it alive and the
recording upload was skipped.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Cover uploads that progress past the PUT deadline

A body that keeps flowing for longer than the per-attempt cap must still upload.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Bound artifact upload stalls instead of total time

The 90 s PUT deadline now restarts on every chunk the request consumes, and the
300 s artifact and 305 s recording deadlines are gone. Upload time scales with
file size, so a whole-request cap failed every recording a slow link could not
send in time, including iOS and agent artifacts through the shared uploader.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Cover uploading the partial recording a killed Hub leaves

A killed Hub lists no recording; the host must find the fragmented partial,
skip one ffprobe cannot read, upload it flagged partial and remove the root.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Upload the partial recording a killed Hub leaves behind

When recordings.json is empty after the host stops, look for session
directories whose manifest is still at "recording" and whose fragmented
.partial ffprobe can read, then upload them through the shared recording
uploader with partial: true in the artifact metadata. The Hub writes that
manifest at the first keyframe, so the website can still place the video.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Log the Hub's reason when recording finalization is refused

The Hub answers 409 with why nothing was recorded, such as the booted-emulator
count. Read the body instead of throwing on the status so the reason reaches
the job log.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Cover recovering a failed partial recording with its reason

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Upload failed partial recordings with the Hub's reason

A Hub that failed or timed out while finalizing leaves the same fragmented
partial as a killed one, and its manifest now names the file and the error.
Recover "failed" manifests whose file decodes and pass the reason to the
website as partialReason.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Type the deadline error and simplify artifact PUT cleanup

The retry predicate matched the PUT deadline by message text and carried an
unreachable AbortError branch. A DeviceRunSessionTimeoutError replaces both.
node-fetch destroys the request body on abort and the retry loop destroys the
stream on every exit, so the extra abort listener and destroy call go. The
PUT error names the artifact and the status text again.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Do not warn about a preview tunnel that never opened

A preview whose open failed, or that its opener closed after finish began,
rejects previewTask. Retirement awaited that rejection and logged a deadline
warning for a tunnel that no longer existed.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Recover completed recordings a killed Hub never listed

recordings.json is written at finish. A recording that reached its duration
limit is complete on disk but unlisted until the session ends, so a Hub killed
in that window left a finished video the scan skipped. The scan now accepts
"complete" manifests as well and uploads them without the partial flag.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Log the session host's output when a recording is lost

The Hub reports capture failures on stderr, but the worker logged only the
stop route's one-line summary. When finalization fails or nothing is
uploaded, log the last 8 000 characters of the host's output.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Tell a Hub that never recorded apart from a failed finalization

A 409 from the stop route means nothing was captured, and the Hub says why.
Log that reason and skip the upload and the host output dump, which stay
for real capture failures.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Bound the finalization request once, through the deadline wrapper

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Ignore a deadline reset after the bounded operation settled

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Say when ffprobe is missing instead of calling a recording undecodable

Also log the recording by its session directory name rather than the full
temp path.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-28 12:57:57 +02:00
Vojtech NovakandClaude Fable 5.1 55017fbf70 [build-tools] Record Android sessions by default through a session host (#4418)
* [build-tools] Add opt-in Android session recording upload

* [build-tools] Document opt-in Android session recording

* [build-tools] Record Android device sessions by default

* [build-tools] Separate session recording from preview lifetime

* [build-tools] Stop automation alongside recording cleanup

* [build-tools] Fix session host formatting and add lifecycle release note

* [build-tools] Keep session host guidance in the README

* [build-tools] Extract session host shutdown operations

* [build-tools] Keep one changelog entry for the recording stack

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D29AEWN9ajZXbzobHUH4fF

* [build-tools] Type the session host variables with DeviceSessionHost

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Throw SystemError when recording finalization fails

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Keep the recording upload step Darwin-only

Android recordings are uploaded by the session host, and nothing on Linux
produces recordings_json.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Drop the session host section from the README

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [build-tools] Name the recording stop grace period after the Hub's SIGTERM deadline

Co-Authored-By: Claude Fable 5.1 <[email protected]>

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-28 12:57:56 +02:00
Byron Karlen 7fd8a3a75b [eas-cli] Remove unnecessary confirm_account query param (#4368) 2026-09-24 19:40:24 -07:00
Szymon DziedzicandClaude Opus 5.5 0f38c2ad82 [eas-cli] Include legacy sessions in the web-preview-only filter (#4466)
* [eas-cli] Include legacy sessions in the web-preview-only filter

`simulator:list --type web-preview-only` now also matches sessions with the
deprecated SERVE_SIM type. The --type flags list each value once.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* [eas-cli] Add changelog entry

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-25 00:21:04 +02:00
Expo CI 5198a2cc9c bump STAGING and PRODUCTION to 24.8.0 in cli-versions.json 2026-09-24 21:22:09 +00:00
Expo CI ca5f0d2306 update CHANGELOG.md for v24.8.0 2026-09-24 21:17:13 +00:00
Expo CI 9eefd5a5c4 v24.8.0 2026-09-24 21:13:58 +00:00
Tomek Czajecki a797caaa8a [eas-cli] Add --build-fingerprint to eas simulator (#4472)
* [eas-cli] Add --build-fingerprint to eas simulator

* [eas-cli] Add changelog entry
2026-09-25 00:05:35 +03:00
Gabe Debes 11323be04b [build-tools] boot the Android emulator headless from argent (#4465)
* [build-tools] start argent with ARGENT_EMULATOR_NO_WINDOW

* [build-tools] add the PR number to the changelog
2026-09-24 12:44:25 -07:00
David Mokos b1f886ef8d allows interpolated URIs in workflow validation (#4329) 2026-09-24 04:57:53 -07:00
David MokosandDavid Mokos 14e75df9e5 [eas-cli] Stop creating CLAUDE.md in new apps (#4446)
* [eas-cli] Stop creating CLAUDE.md in new apps

* [eas-cli] Link changelog entry to PR

---------

Co-authored-by: David Mokos <[email protected]>
2026-09-24 12:19:18 +02:00
Szymon Świerk dd88335654 [eas-cli] Gate the individual ASC API key prompt (#4464) 2026-09-24 01:01:57 +02:00
Gabe Debes 5183a6581a [build-tools] pass the website origin to serve-sim (#4437)
* [build-tools] pass the website origin to serve-sim

* [build-tools] allow deploy previews to reach staging sessions

* [build-tools] allow the website dev ports on staging sessions

* [build-tools] tidy the website origin helper and its tests

* [build-tools] name only https website dev origins
2026-09-23 13:43:06 -07:00
Gabe Debes fb7c812e50 [build-tools] raise the simulator preview stream to 1600px and 10 Mbps (#4439) 2026-09-23 13:01:19 -07:00
Gabe Debes e690ac0060 [build-tools] hand the iOS application launch to serve-sim (#4324) 2026-09-23 09:38:36 -07:00
Szymon Świerk c2190ca74f [eas-cli] Add individual key type prompt for submission keys (#4250) 2026-09-23 16:34:34 +02:00
Szymon Świerk 157b7cb79f [eas-cli] Accept individual ASC API keys for submissions, guard provisioning (#4249)
* [eas-cli] Allow resolving an ASC API key without an issuer ID

* [eas-cli] Accept individual ASC API keys during authentication

* [eas-cli] Guard credential provisioning operations against individual ASC API keys

* [eas-cli] Allow individual ASC API keys for metadata and TestFlight setup

* [eas-cli] Accept individual ASC API keys in EAS Submit

* [eas-cli] Reject individual ASC API keys in the ASC integration flow

* [eas-cli] Update CHANGELOG for individual ASC API key support
2026-09-23 16:21:16 +02:00
Tomasz Mazur 0d2b1c8d88 [eas-cli] add support for build id in workflow:ssh (#4452) 2026-09-23 10:48:18 +02:00
Szymon Dziedzic 788cb57ff1 [eas-cli] Exclude simulator queue time from startup timeout (#4411)
* [eas-cli] Exclude simulator queue time from startup timeout

* [eas-cli] Add PR attribution to simulator changelog

* [eas-cli] Clarify simulator queue wait message
2026-09-22 23:31:13 +02:00
Krystof WoldrichandClaude Opus 5 1634a50b6d [eas-cli] Make sim a hidden alias of simulator in the help print (#4457)
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
2026-09-22 20:04:39 +02:00
Szymon DziedzicandClaude Fable 5.1 805b403ae3 [eas-cli] Make eas simulator commands visible (#4456)
* [eas-cli] Make eas simulator commands visible

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [eas-cli] Add PR number to changelog entry

Co-Authored-By: Claude Fable 5.1 <[email protected]>

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-22 19:28:46 +02:00
Byron Karlen 1672bcdadd [eas-cli] Recognize new Release Manager role (#4300) 2026-09-21 11:32:53 -07:00
Juwan WheatleyandClaude Fable 5.1 30b1eb2dda [eas-cli] add integrations:supabase:advisors command (#4342)
* [eas-cli] add integrations:supabase:advisors command

* [eas-cli] prompt to re-authorize Supabase from integrations:supabase:advisors

* [eas-cli] link each Supabase advisor finding to its dashboard entry

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* [eas-cli] verify Supabase advisors against deployed API and preserve reauthorization state

* [eas-cli] make Supabase advisor titles clickable

* [eas-cli] remove advisor remediation links from terminal output

* [eas-cli] move Supabase advisor links below issue descriptions

* [eas-cli] print Supabase dashboard URL above advisor findings

* [eas-cli] tighten spacing within Supabase advisor findings

* [eas-cli] address Supabase advisors review feedback

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-21 13:20:28 -04:00
Stanisław Chmiela 99ce876d50 [build-tools] Preserve safe certificate import diagnostics (#4447)
* [build-tools] Preserve safe certificate import diagnostics

* [build-tools] Keep certificate import diagnostics in Keychain

* [build-tools] Simplify import error handling and log diagnostics as errors

* [build-tools] Narrow import failures to Error with optional process fields

* [build-tools] Name import diagnostics and simplify their tests

* [build-tools] Avoid duplicate certificate import diagnostics
2026-09-21 17:40:34 +02:00
Stanisław Chmiela 602502e502 [build-tools] Unify iOS keychain implementations (#4450)
# Why

We had two copies of the Keychain class (from the time period of migrating "legacy build process" to "step-based build process" and copying stuff "not to break anything").

# How

Unified the implementations. The difference between classes was whether the class received `BuildContext` as constructor param or not.

The unified interface requires `logger` being passed in to every command (since we're using the same `Keychain` instance in multiple build phases).

# Test Plan

CI should pass.
2026-09-21 17:14:23 +02:00
Gabe Debes 30a3924f43 [build-tools] pass --share-url so serve-sim copies the expo.dev preview page (#4434)
* [build-tools] pass --share-url so serve-sim copies the expo.dev preview page

* [build-tools] changelog for serve-sim --share-url

* [build-tools] reuse previewPageUrl for serve-sim --share-url
2026-09-17 15:48:10 -07:00
Douglas Lowder d2e6b4c45e [docs] Update RELEASING.md (#4417) 2026-09-17 12:28:30 -07:00
Douglas LowderandClaude Opus 5 419efa485f [ci] Fix and harden release workflow (#4425)
- Rework the workflow so that the changelog is read correctly, and the changelog commit is skipped if the new entry is already present
- Split the npm check into its own release job and raise the timeout
- Serialize release runs with a concurrency group, effectively forcing them to queue

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
2026-09-17 12:20:37 -07:00
Will Schurman 🦦 3bb4240cb4 Upgrade typescript to 6.x (#4400)
<!-- You can skip the changelog check by labeling the PR with "no changelog". -->

# Why

Follow-up to #4399 to upgrade typescript to latest version. 

# How

Tell claude to upgrade to 6.x.

# Test Plan

CI, tests
2026-09-17 11:51:16 -07:00
Will Schurman 🦦 1677efc00c Upgrade typescript to highest 5.x (#4399)
<!-- You can skip the changelog check by labeling the PR with "no changelog". -->

# Why

This is a bit behind in typescript. 6.x is where most of our repos live.

First step is to get to highest 5.x.

# How

Tell claude to upgrade to highest 5.x.

It required a new version of `https-proxy-agent` to be compatible with new node types.

# Test Plan

`yarn start`, tests.
2026-09-17 10:17:45 -07:00
Douglas Lowder c7b3105f9f [eas-cli] Skip managed-workflow checks for custom builds (#4424) 2026-09-16 19:08:46 -07:00
Expo CI 6fceead233 update CHANGELOG.md for v24.7.0 2026-09-16 22:16:08 +00:00
Expo CI 4a109648a6 v24.7.0 2026-09-16 22:13:30 +00:00
Gabe Debes ea139e207c [eas-cli] point preview links at the expo.dev page and expose the preview api url (#4352) 2026-09-16 15:07:49 -07:00
Gabe Debes 7b3c8cf245 [build-tools] report the expo.dev preview page url and the preview api url (#4353) 2026-09-16 14:31:37 -07:00
Gabe Debes efbe0bbdff [build-tools] Launch simulator packages with the resolved package manager (#4343)
* [build-tools] Launch simulator packages with the resolved package manager

Serve-sim, device-hub, and Argent were hardcoded to npx or bunx. Resolve the manager the same way other steps do so EAS_FALLBACK_PACKAGE_MANAGER can switch them off the npm audit path.

* [build-tools] Override the simulator package manager instead of using the lockfile fallback

EAS_FALLBACK_PACKAGE_MANAGER only applies when no lockfile is found, so a worker yarn.lock would keep npx. EAS_OVERRIDE_PACKAGE_MANAGER wins when set. Preview still defaults to npx and Argent to bunx.

* [build-tools] Honor fallback as well as override for simulator package exec

Override still wins when set. If it is unset, EAS_FALLBACK_PACKAGE_MANAGER selects the runner so serve-sim can use bunx without a lockfile-based resolve.

* [build-tools] Add changelog for simulator package manager env

* [build-tools] Format package manager changes

* [build-tools] Honor the simulator package manager for agent-device and Appium

Install these session packages through the same env override/fallback as serve-sim, instead of hardcoding bun add --global and npm install.

* [build-tools] Sort Appium session test imports for lint

* [build-tools] Keep simulator process comments concrete

* [build-tools] Cover simulator package install and agent-device fallback
2026-09-15 16:19:51 -07:00
Gabe Debes 8f645d8870 [build-tools] Default serve-sim iOS previews to H.264 (#4416)
* fix(build-tools): default serve-sim iOS previews to H.264

* changelog
2026-09-15 13:34:00 -07:00
Stanisław Chmiela 97007d17cc [build-tools] Use maestro-runner --parallel execution for shards (#4407)
`maestro-runner` supports sharding!

Runner parallel execution: https://github.com/devicelab-dev/maestro-runner/blob/main/pkg/executor/parallel.go
2026-09-15 19:09:52 +02:00
Expo CI 0dbaa34a8b bump STAGING and PRODUCTION to 24.6.0 in cli-versions.json 2026-09-15 16:21:45 +00:00
Expo CI 2fc537f9a7 update CHANGELOG.md for v24.6.0 2026-09-15 16:17:00 +00:00
Expo CI 71cf0aea74 v24.6.0 2026-09-15 16:14:36 +00:00
Will Schurman 🦦 f6a3028356 [eas-cli] Remove broad primaryAccount querying to support scoped token access (#4380) 2026-09-15 08:38:14 -07:00
Will Schurman 🦦 3182070f4c [eas-cli] Regenerate graphql and fix types (#4379) 2026-09-15 08:14:01 -07:00
Tomasz Mazur 097b58c7a9 [eas-cli] Display build logs when --wait flag is set (#4303)
* [eas-cli] Display build logs when --wait flag is set

* [eas-cli] use custom fetch

* [eas-cli] add timeouts to builds logs watch
2026-09-15 16:57:22 +02:00
Expo CI 9584d05708 bump STAGING and PRODUCTION to 24.5.0 in cli-versions.json 2026-09-15 09:48:19 +00:00
Expo CI b9ac59b336 update CHANGELOG.md for v24.5.0 2026-09-15 09:44:37 +00:00
Expo CI 43068db22a v24.5.0 2026-09-15 09:41:35 +00:00
Stanisław Chmiela 2b7f1954fc [build-tools] Execute sandbox commands over MCP connection (#4347)
## Why

Clients are telling sandboxes through MCP server what to do. Let's handle the requests!

## How

The endeavour starts from a (relatively, after all) complex task of starting commands and reading/writing to them.

Since commands may live longer than a tool call (request followed by a response), we need some kind of manager of running commands — Shell Session Manager. (Session is the running command.)

This is where most of the complexity lays, the rest is mostly a wrapper.

## Test plan

CI should pass. Tested manually.
2026-09-15 11:34:05 +02:00
Expo CI 3b64599aea bump STAGING and PRODUCTION to 24.4.2 in cli-versions.json 2026-09-15 05:07:36 +00:00
Expo CI 96fe9a239e update CHANGELOG.md for v24.4.2 2026-09-15 05:05:39 +00:00
Expo CI 195807643f v24.4.2 2026-09-15 05:03:15 +00:00
Brent Vatne 21256e6d55 [eas-cli] Support Xcode 27 Device Hub when running iOS simulator builds (#4405) 2026-09-14 22:00:43 -07:00
Expo CI 705fa875d3 bump STAGING and PRODUCTION to 24.4.1 in cli-versions.json 2026-09-15 04:14:14 +00:00
Expo CI 498384357e update CHANGELOG.md for v24.4.1 2026-09-15 04:10:19 +00:00
Expo CI 53938e4cf8 v24.4.1 2026-09-15 04:07:48 +00:00
Brent Vatne e18ffcd48e [eas-cli] Bump @expo/apple-utils to 2.2.1 (#4404) 2026-09-14 21:04:22 -07:00
Expo CI df12f86da9 bump STAGING and PRODUCTION to 24.4.0 in cli-versions.json 2026-09-14 21:17:19 +00:00
Expo CI 3dba7f2bf7 update CHANGELOG.md for v24.4.0 2026-09-14 21:13:37 +00:00
Expo CI e7efe14336 v24.4.0 2026-09-14 21:11:01 +00:00
Brent Vatne 78f2103bcb [build-tools] inject a guard into simulator processes that refuses connections bypassing the local egress proxy (#4393) 2026-09-14 14:08:02 -07:00
Brent Vatne 565b0ec425 [build-tools] set proxy environment variables in the Simulator for local egress (#4390) 2026-09-14 13:56:04 -07:00
Ramon Claudio 253d72bb4d [eas-cli] Bump Node to 20.18.3 (#4279)
* Update package.json to use new node version

* Update test workflow to use new node version

* Update important notes section in CLAUDE.md to use new node version

* Add changelog
2026-09-14 15:20:14 -04:00
Stanisław Chmiela 35522e9b28 [build-tools] Skip source refresh for jobs with no sources (#4389)
# Why

`workflow-orchestration` tests rightly use `projectArchive.type: NONE`. NONE sources don't need refresh so we can just handle them inline.

# How

If project archive type is NONE, just return, there's nothing to do, refresh, or limit based on job type.

# Test Plan

CI should fix. E2E tests should pass.
2026-09-14 11:34:15 +02:00
Expo CI 4aa9522825 bump STAGING and PRODUCTION to 24.3.0 in cli-versions.json 2026-09-11 12:37:23 +00:00
Expo CI a92b0b0ad5 update CHANGELOG.md for v24.3.0 2026-09-11 12:34:24 +00:00
303 changed files with 28168 additions and 3482 deletions

No files matched your search

+74 -9
View File
@@ -5,6 +5,14 @@ on:
name: Release workflow
# Releases push commits to main, so they must never overlap. The group is a
# constant rather than per-ref, so two different tags serialize. Queued runs
# are never cancelled, because a half-finished release leaves npm, the git
# history, and cli-versions.json out of sync.
concurrency:
group: release
cancel-in-progress: false
jobs:
release:
name: Create a release
@@ -47,6 +55,10 @@ jobs:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
token: ${{ secrets.EXPO_BOT_PAT }}
# The changelog is generated on the branch it is committed to, so the
# job never switches branches with a dirty working tree.
ref: main
fetch-depth: 0
- uses: ./.github/actions/setup-mise
- name: Install dependencies
working-directory: ./scripts
@@ -57,23 +69,67 @@ jobs:
run: |
git config user.email "[email protected]"
git config user.name "Expo CI"
- name: Check whether the changelog is already updated
run: |
if grep -qF "## [$EAS_CLI_VERSION](" CHANGELOG.md; then
echo "CHANGELOG.md already has an entry for v$EAS_CLI_VERSION."
echo "CHANGELOG_ALREADY_UPDATED=true" >> "$GITHUB_ENV"
else
echo "CHANGELOG_ALREADY_UPDATED=false" >> "$GITHUB_ENV"
fi
- name: Update changelog
if: env.CHANGELOG_ALREADY_UPDATED == 'false'
working-directory: ./scripts
run: yarn release-changelog "$EAS_CLI_VERSION" > /tmp/current-changelog
- name: Reuse the existing changelog entry
if: env.CHANGELOG_ALREADY_UPDATED == 'true'
run: |
{
echo "v$EAS_CLI_VERSION"
echo
awk -v heading="## [$EAS_CLI_VERSION](" '
index($0, heading) == 1 { found = 1; next }
found && /^## / { exit }
found { print }
' CHANGELOG.md | sed '/./,$!d'
} > /tmp/current-changelog
- name: Publish release
run: gh release edit "v$EAS_CLI_VERSION" --draft=false -F /tmp/current-changelog
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Commit and push changelog
run: |
git fetch
git checkout main
git add CHANGELOG.md
git commit --allow-empty -m "update CHANGELOG.md for v$EAS_CLI_VERSION"
git push
if git diff --cached --quiet; then
echo "No changelog changes to commit."
else
git commit -m "update CHANGELOG.md for v$EAS_CLI_VERSION"
git pull --rebase --autostash origin main
git push
fi
bump-cli-versions:
name: Gate on npm and bump cli-versions.json
runs-on: ubuntu-latest
needs: release-changelog
permissions:
contents: write
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
token: ${{ secrets.EXPO_BOT_PAT }}
ref: main
fetch-depth: 0
- uses: ./.github/actions/setup-mise
- name: Resolve version
run: echo "EAS_CLI_VERSION=${GITHUB_REF#refs/tags/v}" >> "$GITHUB_ENV"
- name: Configure git
run: |
git config user.email "[email protected]"
git config user.name "Expo CI"
- name: Verify eas-cli is available on npm
run: |
availability_deadline_epoch=$(( $(date +%s) + 600 ))
availability_timeout_seconds=1800
availability_deadline_epoch=$(( $(date +%s) + availability_timeout_seconds ))
while true; do
published="$(npm view "eas-cli@$EAS_CLI_VERSION" version --prefer-online 2>/dev/null || true)"
@@ -95,7 +151,7 @@ jobs:
sleep "$retry_delay"
done
echo "eas-cli@$EAS_CLI_VERSION was not available on npm after 10 minutes." >&2
echo "eas-cli@$EAS_CLI_VERSION was not available on npm after ${availability_timeout_seconds}s." >&2
exit 1
- name: Bump STAGING and PRODUCTION
run: |
@@ -109,11 +165,20 @@ jobs:
- name: Commit and push version update
run: |
git add cli-versions.json
git commit --allow-empty -m "bump STAGING and PRODUCTION to $EAS_CLI_VERSION in cli-versions.json"
git push
if git diff --cached --quiet; then
echo "cli-versions.json already points at $EAS_CLI_VERSION."
else
git commit -m "bump STAGING and PRODUCTION to $EAS_CLI_VERSION in cli-versions.json"
git pull --rebase --autostash origin main
git push
fi
- name: Prepare changelog for Slack
run: |
tail -n +3 /tmp/current-changelog > /tmp/changelog.md
awk -v heading="## [$EAS_CLI_VERSION](" '
index($0, heading) == 1 { found = 1; next }
found && /^## / { exit }
found { print }
' CHANGELOG.md | sed '/./,$!d' > /tmp/changelog.md
npx slackmd /tmp/changelog.md
{
echo 'SLACK_CHANGELOG<<EOF'
+31
View File
@@ -0,0 +1,31 @@
name: test-egress-guard
on:
workflow_dispatch: {}
pull_request:
paths:
- packages/build-tools/resources/egress-guard/**
- packages/build-tools/src/steps/utils/localEgressGuard.ts
- packages/build-tools/src/steps/utils/__tests__/localEgressGuard*.ts
- packages/build-tools/src/steps/functions/startIosSimulator.ts
- packages/worker/.eas/workflows/test-egress-guard.yml
- .github/workflows/test-egress-guard.yml
jobs:
test-egress-guard:
# The simulator tests need macOS with Xcode, which the EAS macOS workers have.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4
- uses: ./.github/actions/setup-mise
- name: Install eas-cli
run: npm install --global eas-cli@latest
- name: Run the test-egress-guard EAS workflow on macOS
working-directory: packages/worker
env:
EXPO_TOKEN: ${{ secrets.EXPO_DEV_EXPO_GITHUB_ROBOT_ACCESS_TOKEN }}
run: |
eas workflow:run test-egress-guard.yml \
--ref ${{ github.event.pull_request.head.sha || github.sha }} \
--non-interactive \
--wait
+5 -2
View File
@@ -15,9 +15,9 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
node: ['20', '22', '24']
node: ['20.18.3', '22', '24']
include:
- node: '20'
- node: '20.18.3'
coverage: true
name: Test with Node ${{ matrix.node }}
steps:
@@ -26,6 +26,9 @@ jobs:
uses: ./.github/actions/setup-mise
env:
MISE_NODE_VERSION: ${{ matrix.node }}
- name: Install ffmpeg
# build-tools recording-recovery tests cut and decode real MP4 files with ffmpeg and ffprobe.
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends ffmpeg
- name: Install dependencies
run: yarn install --immutable
env:
+124 -1
View File
@@ -8,12 +8,135 @@ This is the log of notable changes to EAS CLI and related packages.
### 🎉 New features
- [eas-build-job] Add a shared protocol for sandbox daemon commands. ([#4346](https://github.com/expo/eas-cli/pull/4346) by [@sjchmiela](https://github.com/sjchmiela))
- [eas-cli] Add `--force-end-active-rollout` to `eas update`, `eas update:republish`, `eas update:roll-back-to-embedded` and `eas update:rollback`. ([#4233](https://github.com/expo/eas-cli/pull/4233) by [@gwdp](https://github.com/gwdp))
- [eas-cli] Allow `eas simulator:start --platform android --egress local`. On Android, `--egress-allow localhost:<port>` also covers `10.0.2.2:<port>`, and refused destinations are shown as warnings. ([#4519](https://github.com/expo/eas-cli/pull/4519) by [@brentvatne](https://github.com/brentvatne))
- [build-tools] Support `eas/start_local_egress` on Linux for Android emulators: an always-listening relay for the emulator's proxy, a systemd scope and nftables fence that refuse traffic that cannot use the tunnel, and fence refusals in the session log. ([#4519](https://github.com/expo/eas-cli/pull/4519) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli] Add `--upload-source-maps` to `eas update`, which uploads the generated source maps to EAS so that stack traces from an update can be symbolicated. Source maps are stripped of `sourcesContent` before upload. ([#4436](https://github.com/expo/eas-cli/pull/4436) by [@douglowder](https://github.com/douglowder))
- [build-tools] Upload the embedded bundle after build by default for projects on SDK 58 and later. Set `EAS_UPDATE_UPLOAD_EMBEDDED_BUNDLE` to `0` to disable it. On SDK 57 and below the feature remains experimental and opt-in via `EAS_UPDATE_EXPERIMENTAL_UPLOAD_EMBEDDED_BUNDLE`. ([#4433](https://github.com/expo/eas-cli/pull/4433) by [@brentvatne](https://github.com/brentvatne))
- [build-tools] Save manual iOS and Android live preview screenshots as session artifacts when supported by the preview server. ([#4481](https://github.com/expo/eas-cli/pull/4481) by [@vonovak](https://github.com/vonovak))
- [build-tools] Add the `eas/install_mitmproxy` step. ([#4307](https://github.com/expo/eas-cli/pull/4307) by [@gwdp](https://github.com/gwdp))
- [build-tools] Start serve-sim with `--network-capture`, and with `--network-capture-field` for each requested field, when a simulator session requests network capture. ([#4307](https://github.com/expo/eas-cli/pull/4307) by [@gwdp](https://github.com/gwdp))
- [eas-cli] Allow workflow submissions to skip loading app config when project and application identifiers are supplied. ([#4471](https://github.com/expo/eas-cli/pull/4471) by [@sjchmiela](https://github.com/sjchmiela))
- [build-tools] Record Android device sessions and upload the screen recording, including the partial recording a killed Device Hub leaves behind. ([#4418](https://github.com/expo/eas-cli/pull/4418) by [@vonovak](https://github.com/vonovak))
- [build-tools] Add start and collect steps that attach bounded simulator logs from serve-sim to device run sessions. ([#4463](https://github.com/expo/eas-cli/pull/4463) by [@sjkim-expo](https://github.com/sjkim-expo))
- [build-tools] Add `eas/start_agent_device_session`, one step for agent-device sessions that boots the device, downloads, installs and launches the app, and starts the session, with the independent parts in parallel. ([#4491](https://github.com/expo/eas-cli/pull/4491) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-build-job] Add `sentry` workflow interpolation context schema for Sentry-triggered workflow runs. ([#4453](https://github.com/expo/eas-cli/pull/4453) by [@sswrk](https://github.com/sswrk))
### 🐛 Bug fixes
- [eas-cli] Stop `eas update:rollback` from always forcing non-interactive mode. ([#4504](https://github.com/expo/eas-cli/pull/4504) by [@gwdp](https://github.com/gwdp))
- [build-tools] Pass the step cancellation signal to `eas/download_build`. ([#4511](https://github.com/expo/eas-cli/pull/4511) by [@sjchmiela](https://github.com/sjchmiela))
- [build-tools] Accept `null` for the TestFlight group `hasAccessToAllBuilds` attribute from App Store Connect. ([#4528](https://github.com/expo/eas-cli/pull/4528) by [@sswrk](https://github.com/sswrk))
- [eas-cli] Support `queued` and `starting` status filters in `eas simulator:list` and keep simulator commands compatible with the current GraphQL schema. ([#4527](https://github.com/expo/eas-cli/pull/4527) by [@sjchmiela](https://github.com/sjchmiela))
- [build-tools] Explain how to resolve missing or expired Apple agreements when App Store Connect rejects a request. ([#4513](https://github.com/expo/eas-cli/pull/4513) by [@sjchmiela](https://github.com/sjchmiela))
- [eas-cli] Correct the `eas simulator:start --max-duration-minutes` help text: every plan can set a custom duration up to the plan's maximum, which is also the default. ([#4524](https://github.com/expo/eas-cli/pull/4524) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli] Fix `eas workflow:validate` crashing with certain workflow schemas, and exit with a non-zero code when validation fails. ([#4507](https://github.com/expo/eas-cli/pull/4507) by [@yuvalgnessin-qz](https://github.com/yuvalgnessin-qz))
- [build-tools] Reuse App Store Connect tokens and refresh them before expiry so long uploads and processing waits remain authenticated. ([#4497](https://github.com/expo/eas-cli/pull/4497) by [@sjchmiela](https://github.com/sjchmiela))
- [build-tools] Clear old Maestro CLI HTML reports before each test attempt so retries cannot upload an earlier report. ([#4409](https://github.com/expo/eas-cli/pull/4409) by [@sjchmiela](https://github.com/sjchmiela))
- [build-tools] Poll the web preview readiness every 250 ms instead of every second, so a session goes live sooner after its host is ready. ([#4496](https://github.com/expo/eas-cli/pull/4496) by [@gwdp](https://github.com/gwdp))
- [eas-cli] Keep the `eas simulator` session spinner on one line, so resizing the terminal no longer leaves copies of it behind. The `eas simulator:stop` command is now printed once before the spinner starts. ([#4488](https://github.com/expo/eas-cli/pull/4488) by [@szdziedzic](https://github.com/szdziedzic))
- [build-tools] Disable agent-device's internal idle timeouts in remote sessions, so only the session's own idle and duration limits end it. ([#4485](https://github.com/expo/eas-cli/pull/4485) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] Keep `eas simulator:events --follow` running while a session is queued or starting. ([#4506](https://github.com/expo/eas-cli/pull/4506) by [@sjkim-expo](https://github.com/sjkim-expo))
### 🧹 Chores
## [24.8.0](https://github.com/expo/eas-cli/releases/tag/v24.8.0) - 2026-09-24
### 🎉 New features
- [eas-cli] Add `--build-fingerprint` to `eas simulator` to install the newest build with a given fingerprint. ([#4472](https://github.com/expo/eas-cli/pull/4472) by [@tchayen](https://github.com/tchayen))
- [eas-cli] Accept build IDs in `workflow:ssh <id>`. ([#4452](https://github.com/expo/eas-cli/pull/4452) by [@AHGIJMKLKKZNPJKQR](https://github.com/AHGIJMKLKKZNPJKQR))
- [eas-cli] Make `eas simulator:*` commands visible in help output. ([#4456](https://github.com/expo/eas-cli/pull/4456) by [@szdziedzic](https://github.com/szdziedzic))
- [build-tools] Add `eas/update_testflight_metadata` to set changelogs and TestFlight groups for an uploaded build with an App Store Connect API key. The step fails if a requested group does not exist. ([#4475](https://github.com/expo/eas-cli/pull/4475) by [@sjchmiela](https://github.com/sjchmiela))
- [build-tools] Pass `--share-url` when launching serve-sim so Share copies the expo.dev preview page instead of the tunnel URL. ([#4434](https://github.com/expo/eas-cli/pull/4434) by [@gwdp](https://github.com/gwdp))
- [eas-cli] Add `eas integrations:supabase:advisors` to list unresolved Supabase Security and Performance Advisor findings for the linked Supabase project. ([#4342](https://github.com/expo/eas-cli/pull/4342) by [@fiberjw](https://github.com/fiberjw))
- [eas-cli] Accept individual App Store Connect API keys for submissions, TestFlight setup, and metadata. ([#4249](https://github.com/expo/eas-cli/pull/4249) by [@sswrk](https://github.com/sswrk))
- [build-tools] Add `launch_app_identifier`, `launch_args`, and `open_url` inputs to the serve-sim, agent-device, argent, and Appium remote sessions, so serve-sim launches an installed app before the stream starts. The inputs are iOS only, and Android sessions keep using `eas/launch_application`. ([#4324](https://github.com/expo/eas-cli/pull/4324) by [@gwdp](https://github.com/gwdp))
- [build-tools] Stream iOS Simulator previews at 1600px and 10 Mbps. ([#4439](https://github.com/expo/eas-cli/pull/4439) by [@gwdp](https://github.com/gwdp))
### 🐛 Bug fixes
- [build-tools] Report early Argent server exits during simulator startup instead of waiting for the full readiness timeout. ([#4454](https://github.com/expo/eas-cli/pull/4454) by [@sjkim-expo](https://github.com/sjkim-expo))
- [eas-cli] Stop spinners from printing a new line on every frame after the terminal gets narrower. ([#4470](https://github.com/expo/eas-cli/pull/4470) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli] Exclude time waiting for simulator concurrency from the session startup timeout and show queue progress. ([#4411](https://github.com/expo/eas-cli/pull/4411) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] List `eas simulator` only once in `eas --help`. The `sim` shorthand is now a hidden alias, so it still runs but no longer duplicates the entry. ([#4457](https://github.com/expo/eas-cli/pull/4457) by [@krystofwoldrich](https://github.com/krystofwoldrich))
- [build-tools] Start the Argent tool-server with `ARGENT_EMULATOR_NO_WINDOW=1`, so Argent can boot the Android emulator again after it stops. Without it, the windowed emulator fails on the Linux image because `libpulse.so.0` is missing. ([#4465](https://github.com/expo/eas-cli/pull/4465) by [@gwdp](https://github.com/gwdp))
- [eas-cli] Fix `eas account:view` printing `Role: undefined` for the new Release Manager role. ([#4300](https://github.com/expo/eas-cli/pull/4300) by [@byronkarlen](https://github.com/byronkarlen))
- [eas-cli] Include legacy serve-sim sessions in `eas simulator:list --type web-preview-only`, and list `web-preview-only` only once in the `--type` options. ([#4466](https://github.com/expo/eas-cli/pull/4466) by [@szdziedzic](https://github.com/szdziedzic))
### 🧹 Chores
- [eas-cli] Stop creating `CLAUDE.md` in new apps; use `AGENTS.md` for agent instructions. ([#4446](https://github.com/expo/eas-cli/pull/4446) by [@davidmokos](https://github.com/davidmokos))
- Upgrade `typescript` to 6.0.3, `ts-jest` to 29.4.12, `@types/node` to 20.19.43, `gql.tada` to 1.11.3, and `https-proxy-agent` to 7.0.6. ([#4400](https://github.com/expo/eas-cli/pull/4400) by [@wschurman](https://github.com/wschurman))
- [build-tools] Use the website origin for simulator preview CORS and framing, and allow deploy previews and local website dev servers on staging. ([#4437](https://github.com/expo/eas-cli/pull/4437) by [@gwdp](https://github.com/gwdp))
## [24.7.0](https://github.com/expo/eas-cli/releases/tag/v24.7.0) - 2026-09-16
### 🎉 New features
- [build-tools] Stream iOS Simulator previews with H.264. ([#4416](https://github.com/expo/eas-cli/pull/4416) by [@gwdp](https://github.com/gwdp))
- [build-tools] Report the expo.dev preview page as a simulator session's preview URL, and the preview server behind it as `previewApiUrl`. ([#4353](https://github.com/expo/eas-cli/pull/4353) by [@gwdp](https://github.com/gwdp))
- [eas-cli] `eas simulator:start` and `eas simulator:get` now report the expo.dev preview page instead of the raw tunnel URL, in both the printed instructions and `--json`. ([#4352](https://github.com/expo/eas-cli/pull/4352) by [@gwdp](https://github.com/gwdp))
### 🐛 Bug fixes
- [eas-cli] Skip the managed-workflow Expo SDK and Metro config checks for builds that use a custom build config, since those builds run their own steps instead of the standard managed pipeline. ([#4424](https://github.com/expo/eas-cli/pull/4424) by [@douglowder](https://github.com/douglowder))
- [build-tools] Honor `EAS_OVERRIDE_PACKAGE_MANAGER` and `EAS_FALLBACK_PACKAGE_MANAGER` when installing and launching simulator packages (serve-sim, device-hub, Argent, agent-device, Appium) so they can use bun instead of stalling on npm audit. ([#4343](https://github.com/expo/eas-cli/pull/4343) by [@gwdp](https://github.com/gwdp))
## [24.6.0](https://github.com/expo/eas-cli/releases/tag/v24.6.0) - 2026-09-15
### 🎉 New features
- [eas-cli] Display logs in `eas build --wait`. ([#4303](https://github.com/expo/eas-cli/pull/4303) by [@AHGIJMKLKKZNPJKQR](https://github.com/AHGIJMKLKKZNPJKQR))
### 🐛 Bug fixes
- [build-tools] Skip project source refresh for jobs with no sources. ([#4389](https://github.com/expo/eas-cli/pull/4389) by [@sjchmiela](https://github.com/sjchmiela))
- [eas-cli] Stop fetching the primary account in the login query. Commands that need it now fetch it on demand, so scoped access tokens without primary account access no longer fail at login. ([#4380](https://github.com/expo/eas-cli/pull/4380) by [@wschurman](https://github.com/wschurman))
### 🧹 Chores
- [eas-cli] Regenerate GraphQL types and update Observe tests for the new `firstSeenAt` and `lastSeenAt` event-name fields. ([#4379](https://github.com/expo/eas-cli/pull/4379) by [@wschurman](https://github.com/wschurman))
## [24.5.0](https://github.com/expo/eas-cli/releases/tag/v24.5.0) - 2026-09-15
### 🎉 New features
- [build-tools] Let connected sandboxes run shell commands through the Expo MCP server. ([#4347](https://github.com/expo/eas-cli/pull/4347) by [@sjchmiela](https://github.com/sjchmiela))
## [24.4.2](https://github.com/expo/eas-cli/releases/tag/v24.4.2) - 2026-09-15
### 🐛 Bug fixes
- [eas-cli] Support Xcode 27, which replaces Simulator.app with Device Hub, in `build:dev`, `build:run`, and `run` for iOS simulator builds. ([#4405](https://github.com/expo/eas-cli/pull/4405) by [@brentvatne](https://github.com/brentvatne))
## [24.4.1](https://github.com/expo/eas-cli/releases/tag/v24.4.1) - 2026-09-15
### 🧹 Chores
- [eas-cli] Bump `@expo/apple-utils` to `2.2.1`. ([#4404](https://github.com/expo/eas-cli/pull/4404) by [@brentvatne](https://github.com/brentvatne))
## [24.4.0](https://github.com/expo/eas-cli/releases/tag/v24.4.0) - 2026-09-14
### 🛠 Breaking changes
- [eas-cli] Require Node `^20.18.3 || >=22.0.0`. ([#4279](https://github.com/expo/eas-cli/pull/4279) by [@ramonclaudio](https://github.com/ramonclaudio))
### 🎉 New features
- [build-tools] Inject a guard into every iOS Simulator process during `--egress local` sessions that refuses connections which bypass the system proxy and reports them, with the calling frameworks, in the session log. ([#4393](https://github.com/expo/eas-cli/pull/4393) by [@brentvatne](https://github.com/brentvatne))
- [build-tools] Set proxy environment variables inside the iOS Simulator for `--egress local` sessions, so clients that read them (gRPC, libcurl) also exit from the egress client. ([#4390](https://github.com/expo/eas-cli/pull/4390) by [@brentvatne](https://github.com/brentvatne))
### 🐛 Bug fixes
- [build-tools] Skip project source refresh for jobs with no sources. ([#4389](https://github.com/expo/eas-cli/pull/4389) by [@sjchmiela](https://github.com/sjchmiela))
## [24.3.0](https://github.com/expo/eas-cli/releases/tag/v24.3.0) - 2026-09-11
### 🎉 New features
- [eas-build-job] Add a shared protocol for sandbox daemon commands. ([#4346](https://github.com/expo/eas-cli/pull/4346) by [@sjchmiela](https://github.com/sjchmiela))
## [24.2.0](https://github.com/expo/eas-cli/releases/tag/v24.2.0) - 2026-09-11
### 🎉 New features
+1 -1
View File
@@ -12,7 +12,7 @@ Additional Use Grant: You may make use of the Licensed Work, provided that you d
to access the functionality of and directly benefit from the
functionality of the Licensed Work.
Change Date: 2029-07-15
Change Date: 2029-10-01
Change License: MIT
+2 -2
View File
@@ -3,7 +3,7 @@
1. Invoke the GitHub workflow ["Manually trigger a release"](https://github.com/expo/eas-cli/actions/workflows/trigger-release.yml). The next version is chosen automatically based on the changelog entries. If you want to use different version, pass the version string as an input to the workflow, in semver format, e.g. "1.2.3"
2. That's it! GitHub Actions is going to take care of the rest. Watch the #eas-cli Slack channel for a successful release notification.
The release also sets the `STAGING` version in `cli-versions.json`. It does not change the `PRODUCTION` version. See [EAS Build CLI versions](#eas-build-cli-versions).
The release also sets the `STAGING` and `PRODUCTION` versions in `cli-versions.json`. See [EAS Build CLI versions](#eas-build-cli-versions).
## EAS Build CLI versions
@@ -14,7 +14,7 @@ The release also sets the `STAGING` version in `cli-versions.json`. It does not
This file replaces the old `latest-eas-build` and `latest-eas-build-staging` npm dist-tags. The build servers read the file from the `main` branch.
A release updates `STAGING` to the new version automatically. To promote a version to production, invoke the GitHub workflow ["Promote eas-cli to production"](https://github.com/expo/eas-cli/actions/workflows/promote-eas-cli-production.yml). Pass the version to promote as an input. If you leave the input empty, the workflow uses the current `STAGING` version. The workflow checks that the version is published on npm before it updates `PRODUCTION`.
A release updates `STAGING` and `PRODUCTION` to the new version automatically. To set a different version to production (to fix a regression, for example), invoke the GitHub workflow ["Promote eas-cli to production"](https://github.com/expo/eas-cli/actions/workflows/promote-eas-cli-production.yml). Pass the version to promote as an input. If you leave the input empty, the workflow uses the current `STAGING` version. The workflow checks that the version is published on npm before it updates `PRODUCTION`.
## Choosing the next version
+2 -2
View File
@@ -1,4 +1,4 @@
{
"STAGING": "24.2.0",
"PRODUCTION": "24.2.0"
"STAGING": "24.8.0",
"PRODUCTION": "24.8.0"
}
+1 -1
View File
@@ -1,5 +1,5 @@
{
"npmClient": "yarn",
"version": "24.3.0",
"version": "24.9.0",
"$schema": "node_modules/lerna/schemas/lerna-schema.json"
}
+3 -3
View File
@@ -22,7 +22,7 @@
},
"devDependencies": {
"@types/jest": "29.5.11",
"@types/node": "20.14.8",
"@types/node": "20.19.43",
"eslint-plugin-async-protect": "3.1.0",
"jest": "29.7.0",
"jest-watch-typeahead": "2.2.2",
@@ -32,8 +32,8 @@
"oxlint": "1.43.0",
"oxlint-tsgolint": "0.11.1",
"rimraf": "3.0.2",
"ts-jest": "29.1.1",
"typescript": "5.5.4"
"ts-jest": "29.4.12",
"typescript": "6.0.3"
},
"packageManager": "[email protected]"
}
+2
View File
@@ -8,3 +8,5 @@ bin/record-sim
src/graphql-env.d.ts
schema.graphql
bin/egress-guard.dylib
bin/egress-guard-check
+14 -11
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/build-tools",
"version": "24.3.0",
"version": "24.9.0",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "BUSL-1.1",
"author": "Expo <[email protected]>",
@@ -24,6 +24,8 @@
"prebuild": "yarn gql",
"build": "tsc",
"build:record-sim": "mkdir -p bin && record_sim_bin_path=$(swift build -c release --package-path resources/record-sim --build-path resources/record-sim/.build --show-bin-path) && swift build -c release --package-path resources/record-sim --build-path resources/record-sim/.build && cp \"$record_sim_bin_path/record-sim\" bin/record-sim && chmod +x bin/record-sim",
"build:egress-guard": "resources/egress-guard/build.sh",
"test:egress-guard": "resources/egress-guard/tests/run-policy-tests.sh && resources/egress-guard/tests/run-guard-tests.sh",
"typecheck": "tsc",
"generate-appium-commands": "mise exec [email protected] -- node scripts/generate-appium-commands.js",
"prepack": "rimraf dist \"*.tsbuildinfo\" && yarn gql && tsc -p tsconfig.build.json",
@@ -39,17 +41,17 @@
"dependencies": {
"@expo/config": "55.0.10",
"@expo/config-plugins": "55.0.7",
"@expo/downloader": "24.0.0",
"@expo/eas-build-job": "24.3.0",
"@expo/downloader": "24.8.0",
"@expo/eas-build-job": "24.9.0",
"@expo/env": "^0.4.0",
"@expo/logger": "24.0.0",
"@expo/logger": "24.8.0",
"@expo/package-manager": "1.9.10",
"@expo/plist": "^0.3.5",
"@expo/results": "^1.0.0",
"@expo/spawn-async": "1.7.2",
"@expo/steps": "24.3.0",
"@expo/template-file": "24.0.0",
"@expo/turtle-spawn": "24.0.0",
"@expo/steps": "24.9.0",
"@expo/template-file": "24.8.0",
"@expo/turtle-spawn": "24.8.0",
"@expo/xcpretty": "^4.3.1",
"@ngrok/ngrok": "1.7.0",
"@sentry/node": "7.77.0",
@@ -59,12 +61,13 @@
"fast-glob": "^3.3.2",
"fast-xml-parser": "^4.4.1",
"fs-extra": "^11.2.0",
"gql.tada": "^1.8.13",
"gql.tada": "^1.11.3",
"joi": "^17.13.1",
"jose": "5.10.0",
"lodash": "^4.18.1",
"node-fetch": "^2.7.0",
"node-forge": "^1.3.1",
"node-pty": "1.2.0-beta.15",
"node-stream-zip": "1.15.0",
"nullthrows": "^1.1.1",
"plist": "^3.1.0",
@@ -85,7 +88,7 @@
"@types/fs-extra": "^11.0.4",
"@types/jest": "^29.5.12",
"@types/lodash": "^4.17.4",
"@types/node": "20.14.2",
"@types/node": "20.19.43",
"@types/node-fetch": "^2.6.11",
"@types/node-forge": "^1.3.11",
"@types/plist": "^3.0.5",
@@ -98,10 +101,10 @@
"memfs": "^4.17.1",
"nock": "13.4.0",
"rimraf": "3.0.2",
"ts-jest": "^29.1.4",
"ts-jest": "^29.4.12",
"ts-mockito": "^2.6.1",
"tslib": "^2.6.3",
"typescript": "^5.5.4",
"typescript": "^6.0.3",
"uuid": "^9.0.1"
}
}
@@ -0,0 +1,83 @@
# egress-guard
A small dylib injected into every process the iOS Simulator launches during a
`--egress local` session. It interposes the socket calls that open outbound
traffic (`connect`, `connectx`, `sendto`, `sendmsg`, and the `$NOCANCEL`
variants of the last three that libsystem exports alongside them) and refuses
any destination that is not loopback. A sockaddr whose family was left
`AF_UNSPEC` is classified as the family its length implies, which is how the
kernel treats it for TCP `connect` and IPv4 sends; on a datagram socket
`connect(AF_UNSPEC)` is left to the kernel, which dissolves the association. The system proxy and the `--egress-allow`
forwards live on loopback, so everything that honors the proxy keeps working
and everything that bypasses it fails with `ECONNREFUSED` in the process that
tried, with the calling frameworks recorded.
Why interposing: simulator processes share the worker's uid, so a packet
filter cannot tell them apart from the worker's own traffic. dyld interposing
from an inserted library reaches calls made inside Apple's own frameworks
(CFNetwork, Network.framework), and `launchctl setenv DYLD_INSERT_LIBRARIES`
inside the simulator makes launchd hand the library to every process it
spawns. The simulator does not enforce library validation, which is what makes
this possible there and nowhere else.
Configuration comes from the environment the simulator's launchd provides:
- `EAS_EGRESS_GUARD_LOG`: file the guard appends events to (host path).
- `EAS_EGRESS_GUARD_MODE`: `block` (default, and what any unknown value means)
or `log` (observe only).
Each event is one tab-separated line:
```
eas-egress-guard\t<process>\t<pid>\t<function>\t<blocked|logged>\t<peer>\t<caller>,<caller>...
```
A process lists each `(function, peer)` once, up to 128 of them. When that
table fills it writes one more line with function `overflow` and the limit as
the peer; further distinct destinations are still refused but not listed.
Layout:
- `policy.c` / `policy.h`: classification, mode, formatting, per-process
dedupe. Pure C, host-testable.
- `guard.c`: the interposers and the constructor that reads the environment.
- `check.c`: `egress-guard-check`, run inside the simulator right after the
guard is installed. Exits non-zero unless the library is loaded in a fresh
process and behaves as the mode says on `connect`, `connect$NOCANCEL` and a
`connect` with an `AF_UNSPEC` sockaddr; the worker fails the session on that.
- `tests/policy_test.c`: host unit tests, `tests/run-policy-tests.sh`.
- `tests/guard_test.c`, `tests/guard_insert_test.c`: host tests of the
interposers themselves, `tests/run-guard-tests.sh`. The first compiles
`guard.c` in and forces the lock schedules (a signal handler making a socket
call inside the critical section, a fork while another thread holds the lock,
contention); the second inserts the built library like the simulator does and
checks descriptor reuse, the `$NOCANCEL` symbols, `AF_UNSPEC` shapes and the
UDP disconnect. Nothing in them opens a socket to a non-loopback address.
- `tests/nettest.swift`: probe binary the simulator end-to-end test runs inside
a device; exercises URLSession, Network.framework, BSD TCP and UDP, DNS.
- `build.sh`: universal simulator dylib into `packages/build-tools/bin/`.
Installation order matters. `simctl` forwards every `SIMCTL_CHILD_`-prefixed
variable of its own environment to the process it starts, and for `simctl
boot` that process is the simulator's launchd, so the worker boots with the
guard and proxy variables in that form and every process of the boot inherits
them (measured: 176 of 176). `launchctl setenv` after boot is kept for a
device that was already booted, but by then the boot's own processes have
started without it. It also cannot change a variable the boot already carried:
a device booted with one guard configuration keeps it until it is shut down,
which is why the worker boots devices itself and every configuration in the
end-to-end test gets its own boot. The self-check runs once boot completes and is followed
by a coverage report from `lsof`, listing any process without the library. Both the dylib and the check binary are built by
`packages/worker/package.sh` for the iOS worker tarball and by the
`test-egress-guard` EAS workflow, not committed.
Failure semantics: a missing library, a failed `launchctl setenv`, or a failed
self-check fails the session, since a `--egress local` session without the
guard would silently leak. Only an unwritable event log is a warning, because
refusals still happen and only the reporting is lost. Recording is best-effort
in the process too: the event log is opened per event rather than held open
(a process that closes descriptors it does not own, as `launchd_sim` does,
would otherwise make the guard write into whatever reused the number), and a
dedupe slot that cannot be taken within a bounded number of attempts (a signal
handler re-entering the guard, a fork child inheriting a held lock) skips the
event rather than blocking or aborting the process.
+30
View File
@@ -0,0 +1,30 @@
#!/usr/bin/env bash
# Builds the guard as a universal (arm64 + x86_64) iOS Simulator dylib into
# packages/build-tools/bin/egress-guard.dylib. Requires Xcode.
set -euo pipefail
cd "$(dirname "$0")"
# Output directory; defaults to packages/build-tools/bin.
bin_dir="${1:-../../bin}"
mkdir -p "$bin_dir"
out=$(mktemp -d)
sdk=$(xcrun --sdk iphonesimulator --show-sdk-path)
for arch in arm64 x86_64; do
xcrun --sdk iphonesimulator clang \
-std=c11 -Wall -Wextra -Werror -O2 \
-target "$arch-apple-ios15.0-simulator" -isysroot "$sdk" \
-dynamiclib -install_name @rpath/egress-guard.dylib \
-o "$out/egress-guard-$arch.dylib" policy.c guard.c
done
lipo -create -output "$bin_dir/egress-guard.dylib" "$out/egress-guard-arm64.dylib" "$out/egress-guard-x86_64.dylib"
lipo -info "$bin_dir/egress-guard.dylib"
# The self-check the worker runs inside the simulator after installing the guard.
for arch in arm64 x86_64; do
xcrun --sdk iphonesimulator clang \
-std=c11 -Wall -Wextra -Werror -O2 \
-target "$arch-apple-ios15.0-simulator" -isysroot "$sdk" \
-o "$out/egress-guard-check-$arch" check.c
done
lipo -create -output "$bin_dir/egress-guard-check" "$out/egress-guard-check-arm64" "$out/egress-guard-check-x86_64"
chmod +x "$bin_dir/egress-guard-check"
lipo -info "$bin_dir/egress-guard-check"
@@ -0,0 +1,112 @@
// Self-check run inside a simulator right after the guard is installed:
// proves the guard library is loaded in a freshly spawned process and that it
// behaves as the requested mode says, on every connect entry point the guard
// covers. Exit 0 on success; 2 when the library is not loaded; 3 when it is
// loaded but did not behave; 1 on usage errors.
//
// egress-guard-check --mode block|log
#include <arpa/inet.h>
#include <dlfcn.h>
#include <errno.h>
#include <fcntl.h>
#include <mach-o/dyld.h>
#include <netinet/in.h>
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <unistd.h>
// TEST-NET-1 (RFC 5737): never routed, so without the guard a non-blocking
// connect reports EINPROGRESS and nothing ever answers.
#define PROBE_ADDRESS "192.0.2.1"
#define PROBE_PORT 9
typedef int (*connect_fn)(int, const struct sockaddr *, socklen_t);
static int guard_loaded(void) {
for (uint32_t i = 0; i < _dyld_image_count(); i++) {
const char *name = _dyld_get_image_name(i);
if (name != NULL && strstr(name, "egress-guard.dylib") != NULL) {
return 1;
}
}
return 0;
}
// One non-blocking connect to the probe address through `entry`, with the
// sockaddr family the caller wants. Returns the connect result; errno is
// stored in *err.
static int probe(connect_fn entry, sa_family_t family, int *err) {
int fd = socket(AF_INET, SOCK_STREAM, 0);
if (fd < 0) {
*err = errno;
return -2;
}
fcntl(fd, F_SETFL, fcntl(fd, F_GETFL, 0) | O_NONBLOCK);
struct sockaddr_in to;
memset(&to, 0, sizeof to);
to.sin_family = family;
to.sin_len = sizeof to;
to.sin_port = htons(PROBE_PORT);
inet_pton(AF_INET, PROBE_ADDRESS, &to.sin_addr);
int rc = entry(fd, (struct sockaddr *)&to, sizeof to);
*err = errno;
close(fd);
return rc;
}
int main(int argc, char **argv) {
const char *mode = "block";
if (argc == 3 && strcmp(argv[1], "--mode") == 0) {
mode = argv[2];
} else if (argc != 1) {
fprintf(stderr, "usage: egress-guard-check [--mode block|log]\n");
return 1;
}
if (!guard_loaded()) {
printf("egress-guard-check: the guard library is not loaded in this process\n");
return 2;
}
connect_fn connect_nocancel = (connect_fn)dlsym(RTLD_DEFAULT, "connect$NOCANCEL");
if (connect_nocancel == NULL) {
printf("egress-guard-check: connect$NOCANCEL is not exported in this process\n");
return 3;
}
// The plain call, the non-cancelable variant, and a sockaddr whose family
// was left AF_UNSPEC, which the kernel connects as IPv4.
const struct {
const char *name;
connect_fn entry;
sa_family_t family;
} probes[] = {
{"connect", connect, AF_INET},
{"connect$NOCANCEL", connect_nocancel, AF_INET},
{"connect with AF_UNSPEC", connect, AF_UNSPEC},
};
int block = strcmp(mode, "block") == 0;
for (size_t i = 0; i < sizeof probes / sizeof probes[0]; i++) {
int err = 0;
int rc = probe(probes[i].entry, probes[i].family, &err);
if (rc == -2) {
printf("egress-guard-check: socket() failed: %s\n", strerror(err));
return 3;
}
if (block && !(rc == -1 && err == ECONNREFUSED)) {
printf("egress-guard-check: guard loaded but a non-loopback %s was not refused (rc=%d, errno=%d %s)\n",
probes[i].name, rc, err, strerror(err));
return 3;
}
if (!block && !(rc == 0 || (rc == -1 && err == EINPROGRESS))) {
printf("egress-guard-check: guard loaded in log mode but %s failed (errno=%d %s)\n",
probes[i].name, err, strerror(err));
return 3;
}
}
if (block) {
printf("egress-guard-check: guard loaded; non-loopback connections are refused\n");
} else {
printf("egress-guard-check: guard loaded in log mode; connections pass through\n");
}
return 0;
}
@@ -0,0 +1,274 @@
// The local egress guard: interposes the calls that open outbound traffic in
// every simulator process it is inserted into, refuses non-loopback
// destinations in block mode, and records one event per destination per
// process. See README.md and policy.h.
#include "policy.h"
#include <dlfcn.h>
#include <errno.h>
#include <execinfo.h>
#include <fcntl.h>
#include <os/lock.h>
#include <sched.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <unistd.h>
#define EG_LOG_ENV "EAS_EGRESS_GUARD_LOG"
#define EG_MODE_ENV "EAS_EGRESS_GUARD_MODE"
#define EG_MAX_CALLERS 6
#define EG_CALLER_LENGTH 64
#define EG_LOCK_ATTEMPTS 64
static int eg_initialized = 0;
static void *eg_own_image = NULL;
static char eg_log_path[1024];
static eg_mode_t eg_mode = EG_MODE_BLOCK;
static eg_seen_t eg_seen;
static int eg_overflow_reported = 0;
static os_unfair_lock eg_lock = OS_UNFAIR_LOCK_INIT;
static void eg_init(void) {
if (eg_initialized) {
return;
}
eg_mode = eg_parse_mode(getenv(EG_MODE_ENV));
Dl_info self;
if (dladdr((const void *)eg_init, &self)) {
eg_own_image = self.dli_fbase;
}
const char *log_path = getenv(EG_LOG_ENV);
// The path is kept rather than an open descriptor: a process that closes
// descriptors it does not know about (launchd_sim does) would otherwise
// leave the guard writing into whatever file reused the number.
if (log_path != NULL && log_path[0] != '\0' && strlen(log_path) < sizeof eg_log_path) {
strcpy(eg_log_path, log_path);
}
eg_initialized = 1;
}
__attribute__((constructor)) static void eg_constructor(void) { eg_init(); }
// Image names of the frames above the guard's own, innermost first, with
// consecutive repeats collapsed: "Network,CFNetwork,MyApp".
__attribute__((noinline)) static int eg_collect_callers(char names[EG_MAX_CALLERS][EG_CALLER_LENGTH]) {
void *frames[EG_MAX_CALLERS + 6];
int frame_count = backtrace(frames, EG_MAX_CALLERS + 6);
int count = 0;
for (int i = 1; i < frame_count && count < EG_MAX_CALLERS; i++) {
Dl_info info;
const char *image = "?";
if (dladdr(frames[i], &info)) {
if (info.dli_fbase == eg_own_image) {
continue;
}
if (info.dli_fname != NULL) {
const char *slash = strrchr(info.dli_fname, '/');
image = slash ? slash + 1 : info.dli_fname;
}
}
if (count > 0 && strcmp(names[count - 1], image) == 0) {
continue;
}
strncpy(names[count], image, EG_CALLER_LENGTH - 1);
names[count][EG_CALLER_LENGTH - 1] = '\0';
count++;
}
return count;
}
// A lock held by this same thread (a signal handler making a socket call
// while the interrupted call held it) or by a thread that did not survive
// fork() never becomes available and os_unfair_lock_lock aborts the process
// in both cases. Give up on recording the event instead; refusal does not
// depend on it. Ordinary contention over the microsecond-long critical
// section resolves within the first attempts.
static int eg_lock_bounded(void) {
for (int i = 0; i < EG_LOCK_ATTEMPTS; i++) {
if (os_unfair_lock_trylock(&eg_lock)) {
return 1;
}
sched_yield();
}
return 0;
}
// One line in the event log. O_APPEND keeps whole lines intact across
// processes writing concurrently; the log is opened per event, at most
// EG_SEEN_CAPACITY + 1 times per process.
__attribute__((noinline)) static void eg_write_event(const char *function, const char *action,
const char *peer, int with_callers) {
if (eg_log_path[0] == '\0') {
return;
}
char names[EG_MAX_CALLERS][EG_CALLER_LENGTH];
const char *callers[EG_MAX_CALLERS];
int caller_count = with_callers ? eg_collect_callers(names) : 0;
for (int i = 0; i < caller_count; i++) {
callers[i] = names[i];
}
char line[1024];
int n = eg_format_event(line, sizeof line, getprogname(), getpid(), function, action, peer,
callers, caller_count);
if (n <= 0) {
return;
}
int fd = open(eg_log_path, O_WRONLY | O_APPEND | O_CREAT | O_CLOEXEC, 0644);
if (fd >= 0) {
(void)write(fd, line, (size_t)n);
close(fd);
}
}
// Returns 1 when the call must be refused.
__attribute__((noinline)) static int eg_handle(const char *function, const struct sockaddr *sa,
socklen_t len) {
eg_class_t cls = eg_classify(sa, len);
if (cls != EG_REMOTE) {
return 0;
}
eg_init();
int deny = eg_should_deny(eg_mode, cls);
char peer[96];
if (eg_format_peer(sa, len, peer, sizeof peer) != 0) {
strncpy(peer, "?", sizeof peer);
}
char key[EG_SEEN_KEY_LENGTH];
snprintf(key, sizeof key, "%s %s", function, peer);
int fresh = 0;
int overflowed = 0;
if (eg_lock_bounded()) {
fresh = eg_seen_insert(&eg_seen, key);
if (!fresh && eg_seen.overflow > 0 && !eg_overflow_reported) {
eg_overflow_reported = 1;
overflowed = 1;
}
os_unfair_lock_unlock(&eg_lock);
}
const char *action = deny ? "blocked" : "logged";
if (fresh) {
eg_write_event(function, action, peer, 1);
} else if (overflowed) {
// Once per process: the table is full, so further distinct destinations
// are still refused but no longer listed.
char limit[64];
snprintf(limit, sizeof limit, "%d distinct destinations", EG_SEEN_CAPACITY);
eg_write_event(EG_OVERFLOW_FUNCTION, action, limit, 0);
}
return deny;
}
// connect() with an AF_UNSPEC address on a datagram socket dissolves the
// association and sends nothing; the kernel answers EAFNOSUPPORT. Leave that
// to the kernel. On a stream socket the kernel connects to the address the
// length implies, which eg_handle classifies.
static int eg_dissolves_association(int fd, const struct sockaddr *sa, socklen_t len) {
if (sa == NULL || len < 2 || sa->sa_family != AF_UNSPEC) {
return 0;
}
int type = 0;
socklen_t type_len = sizeof type;
return getsockopt(fd, SOL_SOCKET, SO_TYPE, &type, &type_len) == 0 && type == SOCK_DGRAM;
}
typedef int (*eg_connect_fn)(int, const struct sockaddr *, socklen_t);
typedef ssize_t (*eg_sendto_fn)(int, const void *, size_t, int, const struct sockaddr *,
socklen_t);
typedef ssize_t (*eg_sendmsg_fn)(int, const struct msghdr *, int);
// The non-cancelable variants libsystem_kernel exports next to the cancelable
// ones; dyld interposing is per symbol, so each needs its own entry.
extern int eg_connect_nocancel_original(int, const struct sockaddr *, socklen_t) __asm__(
"_connect$NOCANCEL");
extern ssize_t eg_sendto_nocancel_original(int, const void *, size_t, int,
const struct sockaddr *, socklen_t) __asm__(
"_sendto$NOCANCEL");
extern ssize_t eg_sendmsg_nocancel_original(int, const struct msghdr *, int) __asm__(
"_sendmsg$NOCANCEL");
static int eg_connect_through(eg_connect_fn original, int fd, const struct sockaddr *sa,
socklen_t len) {
if (!eg_dissolves_association(fd, sa, len) && eg_handle("connect", sa, len)) {
errno = ECONNREFUSED;
return -1;
}
return original(fd, sa, len);
}
static int eg_connect(int fd, const struct sockaddr *sa, socklen_t len) {
return eg_connect_through(connect, fd, sa, len);
}
static int eg_connect_nocancel(int fd, const struct sockaddr *sa, socklen_t len) {
return eg_connect_through(eg_connect_nocancel_original, fd, sa, len);
}
static int eg_connectx(int s, const sa_endpoints_t *endpoints, sae_associd_t associd,
unsigned int flags, const struct iovec *iov, unsigned int iovcnt,
size_t *len, sae_connid_t *connid) {
if (endpoints != NULL && endpoints->sae_dstaddr != NULL &&
!eg_dissolves_association(s, endpoints->sae_dstaddr, endpoints->sae_dstaddrlen) &&
eg_handle("connectx", endpoints->sae_dstaddr, endpoints->sae_dstaddrlen)) {
errno = ECONNREFUSED;
return -1;
}
return connectx(s, endpoints, associd, flags, iov, iovcnt, len, connid);
}
static ssize_t eg_sendto_through(eg_sendto_fn original, int fd, const void *buf, size_t n,
int flags, const struct sockaddr *sa, socklen_t len) {
if (sa != NULL && eg_handle("sendto", sa, len)) {
errno = ECONNREFUSED;
return -1;
}
return original(fd, buf, n, flags, sa, len);
}
static ssize_t eg_sendto(int fd, const void *buf, size_t n, int flags, const struct sockaddr *sa,
socklen_t len) {
return eg_sendto_through(sendto, fd, buf, n, flags, sa, len);
}
static ssize_t eg_sendto_nocancel(int fd, const void *buf, size_t n, int flags,
const struct sockaddr *sa, socklen_t len) {
return eg_sendto_through(eg_sendto_nocancel_original, fd, buf, n, flags, sa, len);
}
static ssize_t eg_sendmsg_through(eg_sendmsg_fn original, int fd, const struct msghdr *msg,
int flags) {
if (msg != NULL && msg->msg_name != NULL && msg->msg_namelen > 0 &&
eg_handle("sendmsg", (const struct sockaddr *)msg->msg_name, msg->msg_namelen)) {
errno = ECONNREFUSED;
return -1;
}
return original(fd, msg, flags);
}
static ssize_t eg_sendmsg(int fd, const struct msghdr *msg, int flags) {
return eg_sendmsg_through(sendmsg, fd, msg, flags);
}
static ssize_t eg_sendmsg_nocancel(int fd, const struct msghdr *msg, int flags) {
return eg_sendmsg_through(eg_sendmsg_nocancel_original, fd, msg, flags);
}
// dyld applies these to every image in the process, including the shared
// cache, which is how calls made inside CFNetwork and Network.framework are
// caught.
__attribute__((used)) static const struct {
const void *replacement;
const void *original;
} eg_interposers[] __attribute__((section("__DATA,__interpose"))) = {
{(const void *)eg_connect, (const void *)connect},
{(const void *)eg_connect_nocancel, (const void *)eg_connect_nocancel_original},
{(const void *)eg_connectx, (const void *)connectx},
{(const void *)eg_sendto, (const void *)sendto},
{(const void *)eg_sendto_nocancel, (const void *)eg_sendto_nocancel_original},
{(const void *)eg_sendmsg, (const void *)sendmsg},
{(const void *)eg_sendmsg_nocancel, (const void *)eg_sendmsg_nocancel_original},
};
@@ -0,0 +1,164 @@
#include "policy.h"
#include <arpa/inet.h>
#include <netinet/in.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
static int is_v4_local(uint32_t host_order) {
return (host_order >> 24) == 127 || host_order == 0;
}
// XNU accepts a sockaddr whose family is AF_UNSPEC as the family its length
// implies: TCP connect() translates it on both socket families and IPv4
// sendto()/sendmsg() deliver with it. Classify it the way the kernel uses it.
static sa_family_t eg_family(const struct sockaddr *sa, socklen_t len) {
if (sa->sa_family == AF_UNSPEC) {
if (len == sizeof(struct sockaddr_in)) {
return AF_INET;
}
if (len == sizeof(struct sockaddr_in6)) {
return AF_INET6;
}
}
return sa->sa_family;
}
eg_class_t eg_classify(const struct sockaddr *sa, socklen_t len) {
if (sa == NULL || len < 2) {
return EG_PASSTHROUGH;
}
sa_family_t family = eg_family(sa, len);
if (family == AF_INET) {
if (len < sizeof(struct sockaddr_in)) {
return EG_PASSTHROUGH;
}
const struct sockaddr_in *in = (const struct sockaddr_in *)sa;
return is_v4_local(ntohl(in->sin_addr.s_addr)) ? EG_LOOPBACK : EG_REMOTE;
}
if (family == AF_INET6) {
if (len < sizeof(struct sockaddr_in6)) {
return EG_PASSTHROUGH;
}
const struct in6_addr *a6 = &((const struct sockaddr_in6 *)sa)->sin6_addr;
if (IN6_IS_ADDR_LOOPBACK(a6) || IN6_IS_ADDR_UNSPECIFIED(a6)) {
return EG_LOOPBACK;
}
if (IN6_IS_ADDR_V4MAPPED(a6)) {
uint32_t v4;
memcpy(&v4, &a6->s6_addr[12], sizeof v4);
return is_v4_local(ntohl(v4)) ? EG_LOOPBACK : EG_REMOTE;
}
return EG_REMOTE;
}
return EG_PASSTHROUGH;
}
eg_mode_t eg_parse_mode(const char *value) {
if (value != NULL && strcmp(value, "log") == 0) {
return EG_MODE_LOG;
}
return EG_MODE_BLOCK;
}
int eg_should_deny(eg_mode_t mode, eg_class_t cls) {
return mode == EG_MODE_BLOCK && cls == EG_REMOTE;
}
int eg_format_peer(const struct sockaddr *sa, socklen_t len, char *out, size_t n) {
char ip[INET6_ADDRSTRLEN];
int written;
if (sa == NULL || len < 2) {
return -1;
}
sa_family_t family = eg_family(sa, len);
if (family == AF_INET && len >= sizeof(struct sockaddr_in)) {
const struct sockaddr_in *in = (const struct sockaddr_in *)sa;
if (inet_ntop(AF_INET, &in->sin_addr, ip, sizeof ip) == NULL) {
return -1;
}
written = snprintf(out, n, "%s:%d", ip, ntohs(in->sin_port));
} else if (family == AF_INET6 && len >= sizeof(struct sockaddr_in6)) {
const struct sockaddr_in6 *in6 = (const struct sockaddr_in6 *)sa;
if (inet_ntop(AF_INET6, &in6->sin6_addr, ip, sizeof ip) == NULL) {
return -1;
}
written = snprintf(out, n, "[%s]:%d", ip, ntohs(in6->sin6_port));
} else {
return -1;
}
return (written < 0 || (size_t)written >= n) ? -1 : 0;
}
int eg_seen_insert(eg_seen_t *seen, const char *key) {
for (int i = 0; i < seen->count; i++) {
if (strncmp(seen->keys[i], key, EG_SEEN_KEY_LENGTH - 1) == 0) {
return 0;
}
}
if (seen->count >= EG_SEEN_CAPACITY) {
seen->overflow++;
return 0;
}
strncpy(seen->keys[seen->count], key, EG_SEEN_KEY_LENGTH - 1);
seen->keys[seen->count][EG_SEEN_KEY_LENGTH - 1] = '\0';
seen->count++;
return 1;
}
// Append `text` to out[*pos], replacing field separators; returns 0 when it fit.
static int append_field(char *out, size_t n, size_t *pos, const char *text) {
for (const char *c = text; *c; c++) {
if (*pos + 1 >= n) {
return -1;
}
out[(*pos)++] = (*c == '\t' || *c == '\n' || *c == '\r') ? ' ' : *c;
}
return 0;
}
static int append_raw(char *out, size_t n, size_t *pos, const char *text) {
size_t len = strlen(text);
if (*pos + len + 1 > n) {
return -1;
}
memcpy(out + *pos, text, len);
*pos += len;
return 0;
}
int eg_format_event(char *out, size_t n, const char *progname, int pid, const char *function,
const char *action, const char *peer, const char *const *callers,
int caller_count) {
size_t pos = 0;
char pid_text[16];
snprintf(pid_text, sizeof pid_text, "%d", pid);
if (append_raw(out, n, &pos, EG_EVENT_PREFIX "\t") != 0 ||
append_field(out, n, &pos, progname ? progname : "?") != 0 ||
append_raw(out, n, &pos, "\t") != 0 || append_raw(out, n, &pos, pid_text) != 0 ||
append_raw(out, n, &pos, "\t") != 0 || append_field(out, n, &pos, function) != 0 ||
append_raw(out, n, &pos, "\t") != 0 || append_field(out, n, &pos, action) != 0 ||
append_raw(out, n, &pos, "\t") != 0 || append_field(out, n, &pos, peer) != 0 ||
append_raw(out, n, &pos, "\t") != 0) {
return -1;
}
for (int i = 0; i < caller_count && callers != NULL; i++) {
if (i > 0 && append_raw(out, n, &pos, ",") != 0) {
return -1;
}
// Commas separate callers, so they cannot appear inside one.
for (const char *c = callers[i]; *c; c++) {
char ch = (*c == ',' || *c == '\t' || *c == '\n' || *c == '\r') ? ' ' : *c;
if (pos + 1 >= n) {
return -1;
}
out[pos++] = ch;
}
}
if (append_raw(out, n, &pos, "\n") != 0) {
return -1;
}
out[pos] = '\0';
return (int)pos;
}
@@ -0,0 +1,60 @@
// Policy and formatting for the local egress guard. Pure functions with no
// interposing, so they compile and test on macOS as well as in the simulator.
#ifndef EAS_EGRESS_GUARD_POLICY_H
#define EAS_EGRESS_GUARD_POLICY_H
#include <stddef.h>
#include <sys/socket.h>
// What a destination address is, as far as the guard cares.
typedef enum {
EG_PASSTHROUGH = 0, // not an internet address (unix sockets, NULL, short); never touched
EG_LOOPBACK = 1, // 127.0.0.0/8, ::1, ::ffff:127.x, unspecified; the proxy and forwards live here
EG_REMOTE = 2, // anything else, including link-local and multicast
} eg_class_t;
typedef enum {
EG_MODE_BLOCK = 0, // refuse EG_REMOTE with ECONNREFUSED
EG_MODE_LOG = 1, // observe only
} eg_mode_t;
eg_class_t eg_classify(const struct sockaddr *sa, socklen_t len);
// "block" or unset selects block; "log" selects log. Anything else is block:
// an unknown mode must fail closed.
eg_mode_t eg_parse_mode(const char *value);
int eg_should_deny(eg_mode_t mode, eg_class_t cls);
// Writes "1.2.3.4:443" or "[2001:db8::1]:443". Returns 0 on success.
int eg_format_peer(const struct sockaddr *sa, socklen_t len, char *out, size_t n);
// Fixed-capacity set of strings, so a retrying client logs a destination once
// per process instead of once per attempt.
#define EG_SEEN_CAPACITY 128
#define EG_SEEN_KEY_LENGTH 192
typedef struct {
char keys[EG_SEEN_CAPACITY][EG_SEEN_KEY_LENGTH];
int count;
int overflow; // insertions refused because the table was full
} eg_seen_t;
// 1 when the key was not present and was inserted; 0 when already present or
// when the table is full (counted in overflow).
int eg_seen_insert(eg_seen_t *seen, const char *key);
#define EG_EVENT_PREFIX "eas-egress-guard"
// Function name of the one event a process writes when its seen table fills:
// from then on distinct destinations are still refused but no longer listed.
#define EG_OVERFLOW_FUNCTION "overflow"
// One tab-separated line, newline terminated:
// eas-egress-guard\t<progname>\t<pid>\t<function>\t<action>\t<peer>\t<caller>,<caller>...\n
// Tabs and newlines inside fields are replaced with spaces. Returns the length
// written, or -1 when it does not fit.
int eg_format_event(char *out, size_t n, const char *progname, int pid, const char *function,
const char *action, const char *peer, const char *const *callers,
int caller_count);
#endif
@@ -0,0 +1,291 @@
// Black-box host tests of the built guard, loaded through
// DYLD_INSERT_LIBRARIES like the simulator does. Remote calls use fd -1 or are
// refused before the kernel; the only real sockets are on loopback. Build and
// run with tests/run-guard-tests.sh.
//
// guard-insert-test <case> <log-path> [<extra-path>]
#include <arpa/inet.h>
#include <dlfcn.h>
#include <errno.h>
#include <fcntl.h>
#include <netinet/in.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/stat.h>
#include <sys/uio.h>
#include <unistd.h>
typedef int (*connect_fn)(int, const struct sockaddr *, socklen_t);
typedef ssize_t (*sendto_fn)(int, const void *, size_t, int, const struct sockaddr *, socklen_t);
typedef ssize_t (*sendmsg_fn)(int, const struct msghdr *, int);
static struct sockaddr_in remote(int port) {
struct sockaddr_in a;
memset(&a, 0, sizeof a);
a.sin_family = AF_INET;
a.sin_len = sizeof a;
a.sin_port = htons(port);
inet_pton(AF_INET, "192.0.2.1", &a.sin_addr);
return a;
}
static struct sockaddr_in6 remote6(int port) {
struct sockaddr_in6 a;
memset(&a, 0, sizeof a);
a.sin6_family = AF_INET6;
a.sin6_len = sizeof a;
a.sin6_port = htons(port);
inet_pton(AF_INET6, "2001:db8::1", &a.sin6_addr);
return a;
}
static int refused_sendto(int port) {
struct sockaddr_in to = remote(port);
errno = 0;
return sendto(-1, "x", 1, 0, (struct sockaddr *)&to, sizeof to) == -1 && errno == ECONNREFUSED;
}
static off_t file_size(const char *path) {
struct stat st;
return stat(path, &st) == 0 ? st.st_size : -1;
}
static int count_lines(const char *path, const char *needle) {
FILE *f = fopen(path, "r");
if (f == NULL) {
return -1;
}
int count = 0;
char line[1024];
while (fgets(line, sizeof line, f)) {
if (strstr(line, needle)) {
count++;
}
}
fclose(f);
return count;
}
// Daemon-style cleanup: close every descriptor the process does not know
// about, then open an application file. The guard's event must land in the
// event log, not in that file.
static int fd_reuse(const char *log_path, const char *app_path) {
for (int fd = 3; fd < getdtablesize(); fd++) {
close(fd);
}
int app_fd = open(app_path, O_CREAT | O_TRUNC | O_WRONLY, 0600);
if (app_fd < 0 || !refused_sendto(9)) {
return 2;
}
close(app_fd);
if (file_size(app_path) != 0) {
printf("FAIL fd-reuse: %lld bytes written into an unrelated file\n",
(long long)file_size(app_path));
return 1;
}
if (count_lines(log_path, "\tsendto\tblocked\t192.0.2.1:9\t") != 1) {
printf("FAIL fd-reuse: event missing from the event log\n");
return 1;
}
return 0;
}
// Started with stdout closed, as the runner arranges: the process's own stdout
// output must not end up in the event log, and the event must.
static int low_fd(const char *log_path) {
printf("STDOUT-LINE\n");
fflush(stdout);
if (!refused_sendto(9)) {
return 2;
}
if (count_lines(log_path, "STDOUT-LINE") != 0) {
return 1;
}
return count_lines(log_path, "\tsendto\tblocked\t192.0.2.1:9\t") == 1 ? 0 : 1;
}
static int nocancel(void) {
connect_fn nc_connect = (connect_fn)dlsym(RTLD_DEFAULT, "connect$NOCANCEL");
sendto_fn nc_sendto = (sendto_fn)dlsym(RTLD_DEFAULT, "sendto$NOCANCEL");
sendmsg_fn nc_sendmsg = (sendmsg_fn)dlsym(RTLD_DEFAULT, "sendmsg$NOCANCEL");
if (!nc_connect || !nc_sendto || !nc_sendmsg) {
return 2;
}
struct sockaddr_in to = remote(443);
int failures = 0;
errno = 0;
if (nc_connect(-1, (struct sockaddr *)&to, sizeof to) != -1 || errno != ECONNREFUSED) {
printf("FAIL nocancel: connect$NOCANCEL not refused (errno %d)\n", errno);
failures++;
}
errno = 0;
if (nc_sendto(-1, "x", 1, 0, (struct sockaddr *)&to, sizeof to) != -1 || errno != ECONNREFUSED) {
printf("FAIL nocancel: sendto$NOCANCEL not refused (errno %d)\n", errno);
failures++;
}
char payload[] = "x";
struct iovec iov = {.iov_base = payload, .iov_len = 1};
struct msghdr msg = {.msg_name = &to, .msg_namelen = sizeof to, .msg_iov = &iov, .msg_iovlen = 1};
errno = 0;
if (nc_sendmsg(-1, &msg, 0) != -1 || errno != ECONNREFUSED) {
printf("FAIL nocancel: sendmsg$NOCANCEL not refused (errno %d)\n", errno);
failures++;
}
return failures ? 1 : 0;
}
// The shapes the kernel accepts as IPv4/IPv6 despite sa_family == AF_UNSPEC.
static int unspec(void) {
int failures = 0;
struct sockaddr_in to = remote(443);
to.sin_family = AF_UNSPEC;
int tcp = socket(AF_INET, SOCK_STREAM, 0);
errno = 0;
if (connect(tcp, (struct sockaddr *)&to, sizeof to) != -1 || errno != ECONNREFUSED) {
printf("FAIL unspec: TCP connect with AF_UNSPEC not refused (errno %d)\n", errno);
failures++;
}
close(tcp);
struct sockaddr_in6 to6 = remote6(443);
to6.sin6_family = AF_UNSPEC;
int tcp6 = socket(AF_INET6, SOCK_STREAM, 0);
errno = 0;
if (connect(tcp6, (struct sockaddr *)&to6, sizeof to6) != -1 || errno != ECONNREFUSED) {
printf("FAIL unspec: TCP6 connect with AF_UNSPEC not refused (errno %d)\n", errno);
failures++;
}
close(tcp6);
errno = 0;
if (sendto(-1, "x", 1, 0, (struct sockaddr *)&to, sizeof to) != -1 || errno != ECONNREFUSED) {
printf("FAIL unspec: sendto with AF_UNSPEC not refused (errno %d)\n", errno);
failures++;
}
char payload[] = "x";
struct iovec iov = {.iov_base = payload, .iov_len = 1};
struct msghdr msg = {.msg_name = &to, .msg_namelen = sizeof to, .msg_iov = &iov, .msg_iovlen = 1};
errno = 0;
if (sendmsg(-1, &msg, 0) != -1 || errno != ECONNREFUSED) {
printf("FAIL unspec: sendmsg with AF_UNSPEC not refused (errno %d)\n", errno);
failures++;
}
return failures ? 1 : 0;
}
// A UDP socket associated with loopback dissolves the association through
// connect(AF_UNSPEC), whatever address bytes follow the family. The guard
// must let the kernel do that.
static int udp_disconnect(void) {
int fd = socket(AF_INET, SOCK_DGRAM, 0);
struct sockaddr_in loopback;
memset(&loopback, 0, sizeof loopback);
loopback.sin_family = AF_INET;
loopback.sin_len = sizeof loopback;
loopback.sin_port = htons(9);
loopback.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
if (connect(fd, (struct sockaddr *)&loopback, sizeof loopback) != 0) {
return 2;
}
struct sockaddr_in peer;
socklen_t peer_len = sizeof peer;
if (getpeername(fd, (struct sockaddr *)&peer, &peer_len) != 0) {
return 2;
}
int failures = 0;
for (int stale = 0; stale < 2; stale++) {
struct sockaddr_in dissolve = stale ? remote(443) : loopback;
if (!stale) {
memset(&dissolve, 0, sizeof dissolve);
dissolve.sin_len = sizeof dissolve;
}
dissolve.sin_family = AF_UNSPEC;
if (connect(fd, (struct sockaddr *)&loopback, sizeof loopback) != 0) {
return 2;
}
errno = 0;
int rc = connect(fd, (struct sockaddr *)&dissolve, sizeof dissolve);
int err = errno;
peer_len = sizeof peer;
int still_connected = getpeername(fd, (struct sockaddr *)&peer, &peer_len) == 0;
if (rc != -1 || err != EAFNOSUPPORT || still_connected) {
printf("FAIL udp-disconnect (%s address): rc=%d errno=%d still_connected=%d\n",
stale ? "stale remote" : "zeroed", rc, err, still_connected);
failures++;
}
}
close(fd);
return failures ? 1 : 0;
}
// Past 128 distinct destinations a process writes one overflow line and
// nothing more; refusal continues.
static int overflow(const char *log_path) {
for (int port = 1; port <= 140; port++) {
if (!refused_sendto(port)) {
return 2;
}
}
int listed = count_lines(log_path, "\tsendto\tblocked\t192.0.2.1:");
int overflow_lines = count_lines(log_path, "\toverflow\tblocked\t128 distinct destinations\t");
if (listed != 128 || overflow_lines != 1) {
printf("FAIL overflow: %d destinations listed, %d overflow line(s)\n", listed, overflow_lines);
return 1;
}
return 0;
}
// The event names the images above the guard, not the guard itself.
static int callers(const char *log_path) {
if (!refused_sendto(9)) {
return 2;
}
FILE *f = fopen(log_path, "r");
if (f == NULL) {
return 2;
}
char line[1024];
int ok = 0;
while (fgets(line, sizeof line, f)) {
if (strstr(line, "\tsendto\tblocked\t192.0.2.1:9\t")) {
const char *callers_field = strrchr(line, '\t') + 1;
ok = strstr(callers_field, "guard-insert-test") != NULL &&
strstr(callers_field, "egress-guard") == NULL;
if (!ok) {
printf("FAIL callers: %s", line);
}
}
}
fclose(f);
return ok ? 0 : 1;
}
int main(int argc, char **argv) {
if (argc < 3) {
return 2;
}
const char *name = argv[1];
const char *log_path = argv[2];
if (strcmp(name, "fd-reuse") == 0) {
return argc == 4 ? fd_reuse(log_path, argv[3]) : 2;
}
if (strcmp(name, "low-fd") == 0) {
return low_fd(log_path);
}
if (strcmp(name, "nocancel") == 0) {
return nocancel();
}
if (strcmp(name, "unspec") == 0) {
return unspec();
}
if (strcmp(name, "udp-disconnect") == 0) {
return udp_disconnect();
}
if (strcmp(name, "callers") == 0) {
return callers(log_path);
}
if (strcmp(name, "overflow") == 0) {
return overflow(log_path);
}
return 2;
}
@@ -0,0 +1,177 @@
// White-box host tests of guard.c's dedupe lock: the schedules that would
// abort the process are forced deterministically. Every case runs in its own
// process; sockets are never opened (fd -1), so nothing can leave the host.
// Build and run with tests/run-guard-tests.sh.
#include "../policy.h"
#include <pthread.h>
#include <signal.h>
#include <sys/wait.h>
static int scheduled_seen_insert(eg_seen_t *seen, const char *key);
#define eg_seen_insert scheduled_seen_insert
#include "../guard.c"
#undef eg_seen_insert
#include <arpa/inet.h>
#define CONTENTION_THREADS 8
#define CONTENTION_KEYS 12
#define CONTENTION_ROUNDS 25
static int interrupt_insert;
static int ready_pipe[2];
static int release_pipe[2];
static struct sockaddr_in remote(int port) {
struct sockaddr_in a;
memset(&a, 0, sizeof a);
a.sin_family = AF_INET;
a.sin_len = sizeof a;
a.sin_port = htons(port);
inet_pton(AF_INET, "192.0.2.1", &a.sin_addr);
return a;
}
// Block mode must refuse before touching the kernel, even on an invalid fd.
static void send_remote(int port) {
struct sockaddr_in to = remote(port);
if (eg_sendto(-1, "x", 1, 0, (struct sockaddr *)&to, sizeof to) != -1 || errno != ECONNREFUSED) {
_exit(2);
}
}
static void on_signal(int sig) {
(void)sig;
send_remote(9);
}
static int scheduled_seen_insert(eg_seen_t *seen, const char *key) {
if (interrupt_insert) {
interrupt_insert = 0;
raise(SIGUSR1);
}
return eg_seen_insert(seen, key);
}
static int signal_reentry(void) {
signal(SIGUSR1, on_signal);
interrupt_insert = 1;
send_remote(9);
send_remote(10);
return 0;
}
static void *hold_lock(void *unused) {
(void)unused;
char byte = 'x';
os_unfair_lock_lock(&eg_lock);
(void)write(ready_pipe[1], &byte, 1);
(void)read(release_pipe[0], &byte, 1);
os_unfair_lock_unlock(&eg_lock);
return NULL;
}
static int fork_while_locked(void) {
if (pipe(ready_pipe) || pipe(release_pipe)) {
return 2;
}
pthread_t thread;
if (pthread_create(&thread, NULL, hold_lock, NULL)) {
return 2;
}
char byte;
if (read(ready_pipe[0], &byte, 1) != 1) {
return 2;
}
pid_t child = fork();
if (child < 0) {
return 2;
}
if (child == 0) {
alarm(5);
send_remote(9);
send_remote(10);
_exit(0);
}
(void)write(release_pipe[1], "x", 1);
pthread_join(thread, NULL);
int status;
waitpid(child, &status, 0);
if (WIFSIGNALED(status)) {
printf("FAIL fork: child terminated by signal %d\n", WTERMSIG(status));
return 1;
}
return WIFEXITED(status) ? WEXITSTATUS(status) : 2;
}
static void *hammer(void *unused) {
(void)unused;
for (int round = 0; round < CONTENTION_ROUNDS; round++) {
for (int key = 0; key < CONTENTION_KEYS; key++) {
send_remote(1000 + key);
}
}
return NULL;
}
// Under contention a thread may give up its insert; the next attempt on the
// same key must then record it, so every key ends up logged exactly once.
static int contention(const char *log_path) {
pthread_t threads[CONTENTION_THREADS];
for (int i = 0; i < CONTENTION_THREADS; i++) {
if (pthread_create(&threads[i], NULL, hammer, NULL)) {
return 2;
}
}
for (int i = 0; i < CONTENTION_THREADS; i++) {
pthread_join(threads[i], NULL);
}
FILE *log = fopen(log_path, "r");
if (log == NULL) {
return 2;
}
int counts[CONTENTION_KEYS] = {0};
int total = 0;
char line[1024];
while (fgets(line, sizeof line, log)) {
total++;
for (int key = 0; key < CONTENTION_KEYS; key++) {
char needle[64];
snprintf(needle, sizeof needle, "\tsendto\tblocked\t192.0.2.1:%d\t", 1000 + key);
if (strstr(line, needle)) {
counts[key]++;
}
}
}
fclose(log);
int failures = 0;
for (int key = 0; key < CONTENTION_KEYS; key++) {
if (counts[key] != 1) {
printf("FAIL contention: 192.0.2.1:%d logged %d times\n", 1000 + key, counts[key]);
failures++;
}
}
if (total != CONTENTION_KEYS) {
printf("FAIL contention: %d lines for %d keys\n", total, CONTENTION_KEYS);
failures++;
}
return failures ? 1 : 0;
}
int main(int argc, char **argv) {
if (argc != 2) {
return 2;
}
eg_mode = EG_MODE_BLOCK;
if (strcmp(argv[1], "signal") == 0) {
return signal_reentry();
}
if (strcmp(argv[1], "fork") == 0) {
return fork_while_locked();
}
if (strcmp(argv[1], "contention") == 0) {
return contention(eg_log_path);
}
return 2;
}
@@ -0,0 +1,172 @@
// Probe run inside a simulator by the egress guard end-to-end test. Exercises
// every outbound path the guard must cover and prints one JSON object.
//
// nettest --proxy-port <port> --udp-port <port>
//
// proxy-port: a CONNECT proxy on the host's loopback, standing in for the
// egress proxy. udp-port: a UDP echo on loopback.
import Foundation
import Network
var results: [String: String] = [:]
let args = CommandLine.arguments
func arg(_ name: String) -> Int {
guard let i = args.firstIndex(of: name), i + 1 < args.count else { return 0 }
return Int(args[i + 1]) ?? 0
}
let proxyPort = arg("--proxy-port")
let udpPort = arg("--udp-port")
let remoteHost = "example.com"
let remotePort = 443
func classify(_ error: Error?) -> String {
guard let e = error as NSError? else { return "ok" }
if e.domain == NSURLErrorDomain && e.code == NSURLErrorCannotConnectToHost { return "refused" }
if e.domain == NSPOSIXErrorDomain && e.code == Int(ECONNREFUSED) { return "refused" }
return "error(\(e.domain):\(e.code))"
}
// 1. URLSession straight to the internet. Blocked by the guard.
func urlSessionDirect() {
let g = DispatchGroup(); g.enter()
let c = URLSessionConfiguration.ephemeral; c.timeoutIntervalForRequest = 10
c.connectionProxyDictionary = [:] // explicitly no proxy, even if the host has one
URLSession(configuration: c).dataTask(with: URL(string: "https://\(remoteHost)/")!) { _, r, e in
results["urlsessionDirect"] = e == nil ? "ok(\((r as? HTTPURLResponse)?.statusCode ?? 0))" : classify(e); g.leave()
}.resume(); g.wait()
}
// 2. URLSession through a CONNECT proxy on loopback. Allowed: the guard only
// sees a loopback connect.
func urlSessionProxied() {
let g = DispatchGroup(); g.enter()
let c = URLSessionConfiguration.ephemeral; c.timeoutIntervalForRequest = 15
// The kCFNetworkProxies* constants are not exposed to Swift on iOS; these are their values.
c.connectionProxyDictionary = ["HTTPSEnable": 1, "HTTPSProxy": "127.0.0.1", "HTTPSPort": proxyPort]
URLSession(configuration: c).dataTask(with: URL(string: "https://\(remoteHost)/")!) { _, r, e in
results["urlsessionProxied"] = e == nil ? "ok(\((r as? HTTPURLResponse)?.statusCode ?? 0))" : classify(e); g.leave()
}.resume(); g.wait()
}
// 3. Network.framework connection straight out. Blocked.
func nwConnectionDirect() {
let g = DispatchGroup(); g.enter()
let p = NWParameters.tls; p.preferNoProxies = true
let c = NWConnection(host: NWEndpoint.Host(remoteHost), port: NWEndpoint.Port(integerLiteral: UInt16(remotePort)), using: p)
var done = false
c.stateUpdateHandler = { s in
guard !done else { return }
switch s {
case .ready: done = true; results["nwconnectionDirect"] = "ok"; c.cancel(); g.leave()
case .failed(let e): done = true; results["nwconnectionDirect"] = classify(e); g.leave()
case .waiting(let e): done = true; results["nwconnectionDirect"] = "waiting(" + classify(e) + ")"; c.cancel(); g.leave()
default: break
}
}
c.start(queue: .global())
_ = g.wait(timeout: .now() + 15)
if results["nwconnectionDirect"] == nil { results["nwconnectionDirect"] = "timeout"; c.cancel() }
}
// 4. Plain BSD TCP connect to a resolved address. Blocked.
func bsdConnectDirect() {
var hints = addrinfo(); hints.ai_socktype = SOCK_STREAM; hints.ai_family = AF_INET
var info: UnsafeMutablePointer<addrinfo>? = nil
guard getaddrinfo(remoteHost, "443", &hints, &info) == 0, let ai = info else { results["bsdConnectDirect"] = "dns-failed"; return }
results["dns"] = "ok"
let fd = socket(ai.pointee.ai_family, ai.pointee.ai_socktype, ai.pointee.ai_protocol)
let rc = connect(fd, ai.pointee.ai_addr, ai.pointee.ai_addrlen)
results["bsdConnectDirect"] = rc == 0 ? "ok" : (errno == ECONNREFUSED ? "refused" : "errno(\(errno))")
close(fd); freeaddrinfo(info)
}
// 5. UDP datagram straight out. Blocked. 6. UDP datagram to loopback echo. Allowed.
func udp(_ key: String, _ host: String, _ port: Int, expectEcho: Bool) {
let fd = socket(AF_INET, SOCK_DGRAM, 0)
var to = sockaddr_in(); to.sin_family = sa_family_t(AF_INET); to.sin_port = in_port_t(UInt16(port)).bigEndian
inet_pton(AF_INET, host, &to.sin_addr)
var payload: [UInt8] = Array("ping".utf8)
let sent = withUnsafePointer(to: &to) { p in p.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
sendto(fd, &payload, payload.count, 0, sa, socklen_t(MemoryLayout<sockaddr_in>.size)) } }
if sent < 0 { results[key] = errno == ECONNREFUSED ? "refused" : "errno(\(errno))"; close(fd); return }
if expectEcho {
var tv = timeval(tv_sec: 3, tv_usec: 0); setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &tv, socklen_t(MemoryLayout<timeval>.size))
var buf = [UInt8](repeating: 0, count: 16)
let n = recv(fd, &buf, buf.count, 0)
results[key] = n > 0 ? "ok" : "no-echo"
} else { results[key] = "sent" }
close(fd)
}
urlSessionDirect()
urlSessionProxied()
nwConnectionDirect()
bsdConnectDirect()
udp("udpDirect", "8.8.8.8", 53, expectEcho: false)
udp("udpLoopback", "127.0.0.1", udpPort, expectEcho: true)
// 7. The same literal destination twice: the guard must log it once per process.
func bsdConnectLiteral(_ key: String) {
var to = sockaddr_in(); to.sin_family = sa_family_t(AF_INET); to.sin_port = in_port_t(UInt16(443)).bigEndian
inet_pton(AF_INET, "1.1.1.1", &to.sin_addr)
let fd = socket(AF_INET, SOCK_STREAM, 0)
let rc = withUnsafePointer(to: &to) { p in p.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
connect(fd, sa, socklen_t(MemoryLayout<sockaddr_in>.size)) } }
results[key] = rc == 0 ? "ok" : (errno == ECONNREFUSED ? "refused" : "errno(\(errno))")
close(fd)
}
bsdConnectLiteral("literalFirst")
bsdConnectLiteral("literalSecond")
// 8. A sockaddr_in whose family was left AF_UNSPEC: the kernel connects it as
// IPv4, so the guard must refuse it. 9. connect(AF_UNSPEC) on a UDP socket
// dissolves the association; the guard must leave that to the kernel.
func unspecConnect() {
var to = sockaddr_in(); to.sin_family = sa_family_t(AF_UNSPEC); to.sin_port = in_port_t(UInt16(443)).bigEndian
inet_pton(AF_INET, "1.0.0.1", &to.sin_addr)
let fd = socket(AF_INET, SOCK_STREAM, 0)
let rc = withUnsafePointer(to: &to) { p in p.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
connect(fd, sa, socklen_t(MemoryLayout<sockaddr_in>.size)) } }
results["unspecConnect"] = rc == 0 ? "ok" : (errno == ECONNREFUSED ? "refused" : "errno(\(errno))")
close(fd)
}
func udpDisconnect() {
let fd = socket(AF_INET, SOCK_DGRAM, 0)
var to = sockaddr_in(); to.sin_family = sa_family_t(AF_INET); to.sin_port = in_port_t(UInt16(udpPort)).bigEndian
inet_pton(AF_INET, "127.0.0.1", &to.sin_addr)
let associated = withUnsafePointer(to: &to) { p in p.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
connect(fd, sa, socklen_t(MemoryLayout<sockaddr_in>.size)) } }
var dissolve = sockaddr_in(); dissolve.sin_family = sa_family_t(AF_UNSPEC); dissolve.sin_port = in_port_t(UInt16(443)).bigEndian
inet_pton(AF_INET, "1.1.1.1", &dissolve.sin_addr)
let rc = withUnsafePointer(to: &dissolve) { p in p.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
connect(fd, sa, socklen_t(MemoryLayout<sockaddr_in>.size)) } }
let err = errno
var peer = sockaddr_in(); var peerLength = socklen_t(MemoryLayout<sockaddr_in>.size)
let stillConnected = withUnsafeMutablePointer(to: &peer) { p in p.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
getpeername(fd, sa, &peerLength) } } == 0
results["udpDisconnect"] = associated != 0 ? "associate-failed"
: (rc == -1 && err == EAFNOSUPPORT && !stillConnected) ? "ok" : "errno(\(err)),connected=\(stillConnected)"
close(fd)
}
// 10. The non-cancelable variant of sendto, as libsystem exports it. Blocked.
func nocancelSendto() {
typealias SendtoFn = @convention(c) (Int32, UnsafeRawPointer?, Int, Int32, UnsafePointer<sockaddr>?, socklen_t) -> Int
guard let symbol = dlsym(UnsafeMutableRawPointer(bitPattern: -2), "sendto$NOCANCEL") else {
results["nocancelSendto"] = "no-symbol"; return
}
let nocancelSendto = unsafeBitCast(symbol, to: SendtoFn.self)
let fd = socket(AF_INET, SOCK_DGRAM, 0)
var to = sockaddr_in(); to.sin_family = sa_family_t(AF_INET); to.sin_port = in_port_t(UInt16(53)).bigEndian
inet_pton(AF_INET, "8.8.8.8", &to.sin_addr)
var payload: [UInt8] = Array("ping".utf8)
let sent = withUnsafePointer(to: &to) { p in p.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
nocancelSendto(fd, &payload, payload.count, 0, sa, socklen_t(MemoryLayout<sockaddr_in>.size)) } }
results["nocancelSendto"] = sent < 0 ? (errno == ECONNREFUSED ? "refused" : "errno(\(errno))") : "sent"
close(fd)
}
unspecConnect()
udpDisconnect()
nocancelSendto()
let json = try! JSONSerialization.data(withJSONObject: results, options: [.sortedKeys])
print(String(data: json, encoding: .utf8)!)
@@ -0,0 +1,143 @@
// Host-runnable tests for the guard's policy and formatting. Build and run
// with tests/run-policy-tests.sh; every failure prints the failing line.
#include "../policy.h"
#include <arpa/inet.h>
#include <netinet/in.h>
#include <stdio.h>
#include <string.h>
#include <sys/un.h>
static int failures = 0;
#define CHECK(cond) \
do { \
if (!(cond)) { \
failures++; \
fprintf(stderr, "FAIL %s:%d: %s\n", __FILE__, __LINE__, #cond); \
} \
} while (0)
static struct sockaddr_in v4(const char *ip, int port) {
struct sockaddr_in a; memset(&a, 0, sizeof a);
a.sin_family = AF_INET; a.sin_len = sizeof a; a.sin_port = htons(port);
inet_pton(AF_INET, ip, &a.sin_addr);
return a;
}
static struct sockaddr_in6 v6(const char *ip, int port) {
struct sockaddr_in6 a; memset(&a, 0, sizeof a);
a.sin6_family = AF_INET6; a.sin6_len = sizeof a; a.sin6_port = htons(port);
inet_pton(AF_INET6, ip, &a.sin6_addr);
return a;
}
static void test_classify(void) {
struct sockaddr_in a4; struct sockaddr_in6 a6; struct sockaddr_un un;
a4 = v4("127.0.0.1", 8899); CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_LOOPBACK);
a4 = v4("127.42.0.9", 80); CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_LOOPBACK);
a4 = v4("0.0.0.0", 80); CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_LOOPBACK);
a4 = v4("93.184.216.34", 443); CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_REMOTE);
a4 = v4("192.168.1.10", 8081); CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_REMOTE);
a4 = v4("169.254.1.1", 80); CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_REMOTE);
a4 = v4("224.0.0.251", 5353); CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_REMOTE);
a6 = v6("::1", 8899); CHECK(eg_classify((struct sockaddr *)&a6, sizeof a6) == EG_LOOPBACK);
a6 = v6("::", 80); CHECK(eg_classify((struct sockaddr *)&a6, sizeof a6) == EG_LOOPBACK);
a6 = v6("::ffff:127.0.0.1", 8899); CHECK(eg_classify((struct sockaddr *)&a6, sizeof a6) == EG_LOOPBACK);
a6 = v6("::ffff:93.184.216.34", 443); CHECK(eg_classify((struct sockaddr *)&a6, sizeof a6) == EG_REMOTE);
a6 = v6("2606:4700::1", 443); CHECK(eg_classify((struct sockaddr *)&a6, sizeof a6) == EG_REMOTE);
a6 = v6("fe80::1", 443); CHECK(eg_classify((struct sockaddr *)&a6, sizeof a6) == EG_REMOTE);
memset(&un, 0, sizeof un); un.sun_family = AF_UNIX; strcpy(un.sun_path, "/var/run/mDNSResponder");
CHECK(eg_classify((struct sockaddr *)&un, sizeof un) == EG_PASSTHROUGH);
CHECK(eg_classify(NULL, 0) == EG_PASSTHROUGH);
a4 = v4("93.184.216.34", 443); CHECK(eg_classify((struct sockaddr *)&a4, 4) == EG_PASSTHROUGH); // too short to read
// AF_UNSPEC is what the kernel makes of it: the family the exact length implies.
a4 = v4("93.184.216.34", 443); a4.sin_family = AF_UNSPEC;
CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_REMOTE);
CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4 + 8) == EG_PASSTHROUGH); // kernel: EINVAL
a4 = v4("127.0.0.1", 8899); a4.sin_family = AF_UNSPEC;
CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_LOOPBACK);
a4 = v4("0.0.0.0", 0); a4.sin_family = AF_UNSPEC;
CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_LOOPBACK); // UDP disconnect shape
a6 = v6("2606:4700::1", 443); a6.sin6_family = AF_UNSPEC;
CHECK(eg_classify((struct sockaddr *)&a6, sizeof a6) == EG_REMOTE);
a6 = v6("::ffff:93.184.216.34", 443); a6.sin6_family = AF_UNSPEC;
CHECK(eg_classify((struct sockaddr *)&a6, sizeof a6) == EG_REMOTE);
a6 = v6("::1", 443); a6.sin6_family = AF_UNSPEC;
CHECK(eg_classify((struct sockaddr *)&a6, sizeof a6) == EG_LOOPBACK);
memset(&un, 0, sizeof un); un.sun_family = AF_UNSPEC;
CHECK(eg_classify((struct sockaddr *)&un, sizeof un) == EG_PASSTHROUGH);
}
static void test_mode(void) {
CHECK(eg_parse_mode(NULL) == EG_MODE_BLOCK);
CHECK(eg_parse_mode("") == EG_MODE_BLOCK);
CHECK(eg_parse_mode("block") == EG_MODE_BLOCK);
CHECK(eg_parse_mode("log") == EG_MODE_LOG);
CHECK(eg_parse_mode("LOG") == EG_MODE_BLOCK);
CHECK(eg_parse_mode("permissive") == EG_MODE_BLOCK);
CHECK(eg_should_deny(EG_MODE_BLOCK, EG_REMOTE) == 1);
CHECK(eg_should_deny(EG_MODE_BLOCK, EG_LOOPBACK) == 0);
CHECK(eg_should_deny(EG_MODE_BLOCK, EG_PASSTHROUGH) == 0);
CHECK(eg_should_deny(EG_MODE_LOG, EG_REMOTE) == 0);
}
static void test_format_peer(void) {
char out[64];
struct sockaddr_in a4 = v4("93.184.216.34", 443);
CHECK(eg_format_peer((struct sockaddr *)&a4, sizeof a4, out, sizeof out) == 0);
CHECK(strcmp(out, "93.184.216.34:443") == 0);
struct sockaddr_in6 a6 = v6("2606:4700::1", 443);
CHECK(eg_format_peer((struct sockaddr *)&a6, sizeof a6, out, sizeof out) == 0);
CHECK(strcmp(out, "[2606:4700::1]:443") == 0);
CHECK(eg_format_peer((struct sockaddr *)&a4, sizeof a4, out, 8) != 0);
struct sockaddr_un un; memset(&un, 0, sizeof un); un.sun_family = AF_UNIX;
CHECK(eg_format_peer((struct sockaddr *)&un, sizeof un, out, sizeof out) != 0);
a4.sin_family = AF_UNSPEC;
CHECK(eg_format_peer((struct sockaddr *)&a4, sizeof a4, out, sizeof out) == 0);
CHECK(strcmp(out, "93.184.216.34:443") == 0);
a6.sin6_family = AF_UNSPEC;
CHECK(eg_format_peer((struct sockaddr *)&a6, sizeof a6, out, sizeof out) == 0);
CHECK(strcmp(out, "[2606:4700::1]:443") == 0);
}
static void test_seen(void) {
static eg_seen_t seen; memset(&seen, 0, sizeof seen);
CHECK(eg_seen_insert(&seen, "connect 1.1.1.1:443") == 1);
CHECK(eg_seen_insert(&seen, "connect 1.1.1.1:443") == 0);
CHECK(eg_seen_insert(&seen, "sendto 1.1.1.1:443") == 1);
CHECK(seen.count == 2);
char key[64];
for (int i = 0; i < EG_SEEN_CAPACITY + 5; i++) { snprintf(key, sizeof key, "k%d", i); eg_seen_insert(&seen, key); }
CHECK(seen.count == EG_SEEN_CAPACITY);
CHECK(seen.overflow == 7); // 2 + 128 + 5 attempts, 128 fit
CHECK(eg_seen_insert(&seen, "k0") == 0); // still remembered
}
static void test_format_event(void) {
char out[512];
const char *callers[] = {"Network", "CFNetwork", "MyApp"};
int n = eg_format_event(out, sizeof out, "MyApp", 4242, "connect", "blocked", "93.184.216.34:443", callers, 3);
CHECK(n > 0);
CHECK(strcmp(out, "eas-egress-guard\tMyApp\t4242\tconnect\tblocked\t93.184.216.34:443\tNetwork,CFNetwork,MyApp\n") == 0);
const char *tabbed[] = {"a\tb"};
n = eg_format_event(out, sizeof out, "we\nird\tname", 1, "sendto", "logged", "1.2.3.4:53", tabbed, 1);
CHECK(n > 0);
CHECK(strcmp(out, "eas-egress-guard\twe ird name\t1\tsendto\tlogged\t1.2.3.4:53\ta b\n") == 0);
n = eg_format_event(out, sizeof out, "NoCallers", 7, "connectx", "blocked", "[::2]:80", NULL, 0);
CHECK(n > 0 && strcmp(out, "eas-egress-guard\tNoCallers\t7\tconnectx\tblocked\t[::2]:80\t\n") == 0);
CHECK(eg_format_event(out, 16, "MyApp", 4242, "connect", "blocked", "93.184.216.34:443", callers, 3) == -1);
}
int main(void) {
test_classify(); test_mode(); test_format_peer(); test_seen(); test_format_event();
if (failures) { fprintf(stderr, "%d failure(s)\n", failures); return 1; }
printf("egress-guard policy tests: all passed\n");
return 0;
}
@@ -0,0 +1,62 @@
#!/usr/bin/env bash
# Builds the guard for the host (macOS) and runs the guard tests against it:
# white-box lock schedules with guard.c compiled in, and black-box cases with
# the built library inserted the way the simulator inserts it. No socket is
# ever opened to a non-loopback address.
set -euo pipefail
cd "$(dirname "$0")/.."
out=$(mktemp -d)
trap 'rm -rf "$out"' EXIT
clang -std=c11 -Wall -Wextra -Werror -O2 -dynamiclib policy.c guard.c -o "$out/egress-guard.dylib"
clang -std=c11 -Wall -Wextra -Werror -O2 -pthread policy.c tests/guard_test.c -o "$out/guard-test"
clang -std=c11 -Wall -Wextra -Werror -O2 tests/guard_insert_test.c -o "$out/guard-insert-test"
failures=0
report() {
if [[ $2 -eq 0 ]]; then
echo "ok $1"
else
echo "FAIL $1 (exit $2)"
failures=$((failures + 1))
fi
}
# White-box: guard.c compiled into the test, lock schedules forced.
for name in signal fork contention; do
status=0
: > "$out/$name.log"
EAS_EGRESS_GUARD_MODE=block EAS_EGRESS_GUARD_LOG="$out/$name.log" \
"$out/guard-test" "$name" || status=$?
report "$name" "$status"
done
# Black-box: the built library inserted into a plain process.
export DYLD_INSERT_LIBRARIES="$out/egress-guard.dylib"
export EAS_EGRESS_GUARD_MODE=block
status=0
: > "$out/fd-reuse.log"
EAS_EGRESS_GUARD_LOG="$out/fd-reuse.log" \
"$out/guard-insert-test" fd-reuse "$out/fd-reuse.log" "$out/application-file" || status=$?
report fd-reuse "$status"
status=0
: > "$out/low-fd.log"
EAS_EGRESS_GUARD_LOG="$out/low-fd.log" \
"$out/guard-insert-test" low-fd "$out/low-fd.log" 1>&- || status=$?
report low-fd "$status"
for name in nocancel unspec udp-disconnect callers overflow; do
status=0
: > "$out/$name.log"
EAS_EGRESS_GUARD_LOG="$out/$name.log" \
"$out/guard-insert-test" "$name" "$out/$name.log" || status=$?
report "$name" "$status"
done
if [[ $failures -ne 0 ]]; then
echo "egress-guard guard tests: $failures failure(s)"
exit 1
fi
echo "egress-guard guard tests: all passed"
@@ -0,0 +1,7 @@
#!/usr/bin/env bash
# Builds the policy unit tests for the host (macOS) and runs them.
set -euo pipefail
cd "$(dirname "$0")/.."
out=$(mktemp -d)
clang -std=c11 -Wall -Wextra -Werror -o "$out/policy_test" policy.c tests/policy_test.c
"$out/policy_test"
@@ -60,6 +60,7 @@ jest.mock('../../utils/expoUpdates', () => ({
resolveRuntimeVersionForExpoUpdatesIfConfiguredAsync: jest.fn(async () => null),
}));
jest.mock('../../utils/expoUpdatesEmbedded', () => ({
...jest.requireActual('../../utils/expoUpdatesEmbedded'),
uploadEmbeddedBundleAsync: jest.fn(),
}));
jest.mock('../../utils/hooks', () => ({
@@ -317,36 +318,82 @@ describe(androidBuilder, () => {
expect(runBuilderWithHooksAsync).toHaveBeenCalledWith(ctx, expect.any(Function));
});
it('runs the embedded bundle upload phase when EAS_UPDATE_EXPERIMENTAL_UPLOAD_EMBEDDED_BUNDLE is set', async () => {
const ctx = new BuildContext(createTestAndroidJob(), {
workingdir: '/workingdir',
logBuffer: { getLogs: () => [], getPhaseLogs: () => [] },
logger: createMockLogger(),
env: {
__API_SERVER_URL: 'http://api.expo.test',
EAS_UPDATE_EXPERIMENTAL_UPLOAD_EMBEDDED_BUNDLE: '1',
},
uploadArtifact: jest.fn(),
describe('embedded bundle upload phase', () => {
function createContext({
env = {},
sdkVersion,
}: {
env?: Record<string, string>;
sdkVersion?: string;
}): BuildContext<Android.Job> {
return new BuildContext(createTestAndroidJob(), {
workingdir: '/workingdir',
logBuffer: { getLogs: () => [], getPhaseLogs: () => [] },
logger: createMockLogger(),
env: {
__API_SERVER_URL: 'http://api.expo.test',
...env,
},
uploadArtifact: jest.fn(),
metadata: sdkVersion ? { sdkVersion } : undefined,
});
}
it('runs by default on SDK 58 and later', async () => {
const ctx = createContext({ sdkVersion: '58.0.0' });
await androidBuilder(ctx);
expect(uploadEmbeddedBundleAsync).toHaveBeenCalledWith(ctx);
});
await androidBuilder(ctx);
it('skips by default on SDK 57 and earlier', async () => {
const ctx = createContext({ sdkVersion: '57.0.0' });
expect(uploadEmbeddedBundleAsync).toHaveBeenCalledWith(ctx);
});
await androidBuilder(ctx);
it('skips the embedded bundle upload phase when EAS_UPDATE_EXPERIMENTAL_UPLOAD_EMBEDDED_BUNDLE is not set', async () => {
const ctx = new BuildContext(createTestAndroidJob(), {
workingdir: '/workingdir',
logBuffer: { getLogs: () => [], getPhaseLogs: () => [] },
logger: createMockLogger(),
env: {
__API_SERVER_URL: 'http://api.expo.test',
},
uploadArtifact: jest.fn(),
expect(uploadEmbeddedBundleAsync).not.toHaveBeenCalled();
});
await androidBuilder(ctx);
it('skips by default when the SDK version is unknown', async () => {
const ctx = createContext({});
expect(uploadEmbeddedBundleAsync).not.toHaveBeenCalled();
await androidBuilder(ctx);
expect(uploadEmbeddedBundleAsync).not.toHaveBeenCalled();
});
it('runs on SDK 57 and earlier when EAS_UPDATE_UPLOAD_EMBEDDED_BUNDLE is 1', async () => {
const ctx = createContext({
sdkVersion: '57.0.0',
env: { EAS_UPDATE_UPLOAD_EMBEDDED_BUNDLE: '1' },
});
await androidBuilder(ctx);
expect(uploadEmbeddedBundleAsync).toHaveBeenCalledWith(ctx);
});
it('skips on SDK 58 and later when EAS_UPDATE_UPLOAD_EMBEDDED_BUNDLE is 0', async () => {
const ctx = createContext({
sdkVersion: '58.0.0',
env: { EAS_UPDATE_UPLOAD_EMBEDDED_BUNDLE: '0' },
});
await androidBuilder(ctx);
expect(uploadEmbeddedBundleAsync).not.toHaveBeenCalled();
});
it('runs on SDK 57 and earlier when the experimental EAS_UPDATE_EXPERIMENTAL_UPLOAD_EMBEDDED_BUNDLE is 1', async () => {
const ctx = createContext({
sdkVersion: '57.0.0',
env: { EAS_UPDATE_EXPERIMENTAL_UPLOAD_EMBEDDED_BUNDLE: '1' },
});
await androidBuilder(ctx);
expect(uploadEmbeddedBundleAsync).toHaveBeenCalledWith(ctx);
});
});
});
+5 -2
View File
@@ -32,7 +32,10 @@ import {
configureExpoUpdatesIfInstalledAsync,
resolveRuntimeVersionForExpoUpdatesIfConfiguredAsync,
} from '../utils/expoUpdates';
import { uploadEmbeddedBundleAsync } from '../utils/expoUpdatesEmbedded';
import {
shouldUploadEmbeddedBundle,
uploadEmbeddedBundleAsync,
} from '../utils/expoUpdatesEmbedded';
import { Hook, runHookIfPresent } from '../utils/hooks';
import { prepareExecutableAsync } from '../utils/prepareBuildExecutable';
@@ -223,7 +226,7 @@ async function buildInnerAsync(
});
});
if (ctx.env.EAS_UPDATE_EXPERIMENTAL_UPLOAD_EMBEDDED_BUNDLE) {
if (shouldUploadEmbeddedBundle(ctx)) {
await ctx.runBuildPhase(BuildPhase.UPLOAD_EMBEDDED_BUNDLE, async () => {
await uploadEmbeddedBundleAsync(ctx);
});
+5 -2
View File
@@ -29,7 +29,10 @@ import {
configureExpoUpdatesIfInstalledAsync,
resolveRuntimeVersionForExpoUpdatesIfConfiguredAsync,
} from '../utils/expoUpdates';
import { uploadEmbeddedBundleAsync } from '../utils/expoUpdatesEmbedded';
import {
shouldUploadEmbeddedBundle,
uploadEmbeddedBundleAsync,
} from '../utils/expoUpdatesEmbedded';
import { Hook, runHookIfPresent } from '../utils/hooks';
import { prepareExecutableAsync } from '../utils/prepareBuildExecutable';
import { getParentAndDescendantProcessPidsAsync } from '../utils/processes';
@@ -240,7 +243,7 @@ async function buildInnerAsync(
});
});
if (ctx.env.EAS_UPDATE_EXPERIMENTAL_UPLOAD_EMBEDDED_BUNDLE) {
if (shouldUploadEmbeddedBundle(ctx)) {
await ctx.runBuildPhase(BuildPhase.UPLOAD_EMBEDDED_BUNDLE, async () => {
await uploadEmbeddedBundleAsync(ctx);
});
@@ -629,20 +629,21 @@ describe('local project sources', () => {
expect(shallowCloneRepositoryAsync).not.toHaveBeenCalled();
});
it.each(Object.values(ArchiveSourceType).filter(type => type !== ArchiveSourceType.PATH))(
'rejects local %s sources before fetching or unpacking',
async type => {
const ctx = createContext(type);
const result = prepareProjectSourcesAsync(ctx, ctx.buildDirectory);
await expect(result).rejects.toBeInstanceOf(SystemError);
await expect(result).rejects.toMatchObject({
errorCode: 'SERVER_ERROR',
trackingCode: 'INVALID_LOCAL_PROJECT_SOURCE',
message: `Expected a PATH project source for a local build, received ${type}.`,
});
expect(fetch).not.toHaveBeenCalled();
expect(spawn).not.toHaveBeenCalled();
expect(shallowCloneRepositoryAsync).not.toHaveBeenCalled();
}
);
it.each(
Object.values(ArchiveSourceType).filter(
type => type !== ArchiveSourceType.PATH && type !== ArchiveSourceType.NONE
)
)('rejects local %s sources before fetching or unpacking', async type => {
const ctx = createContext(type);
const result = prepareProjectSourcesAsync(ctx, ctx.buildDirectory);
await expect(result).rejects.toBeInstanceOf(SystemError);
await expect(result).rejects.toMatchObject({
errorCode: 'SERVER_ERROR',
trackingCode: 'INVALID_LOCAL_PROJECT_SOURCE',
message: `Expected a PATH project source for a local build, received ${type}.`,
});
expect(fetch).not.toHaveBeenCalled();
expect(spawn).not.toHaveBeenCalled();
expect(shallowCloneRepositoryAsync).not.toHaveBeenCalled();
});
});
@@ -19,6 +19,10 @@ export async function prepareProjectSourcesAsync<TJob extends Job>(
destinationDirectory: string
): // Return type required to make switch exhaustive.
Promise<{ handled: boolean }> {
if (ctx.job.projectArchive.type === ArchiveSourceType.NONE) {
return { handled: true };
}
if (ctx.isLocal) {
if (ctx.job.projectArchive.type !== ArchiveSourceType.PATH) {
throw new SystemError(
@@ -1,11 +1,9 @@
import { Ios } from '@expo/eas-build-job';
import { createLogger } from '@expo/logger';
import fs from 'fs-extra';
import os from 'os';
import path from 'path';
import { v4 as uuid } from 'uuid';
import { BuildContext } from '../../../context';
import { distributionCertificate } from '../__tests__/fixtures';
import Keychain from '../keychain';
@@ -17,11 +15,9 @@ jest.setTimeout(60 * 1000);
// We need the JS code to modify the file system, so we need to mock it.
jest.unmock('fs');
let ctx: BuildContext<Ios.Job>;
describe('Keychain class', () => {
describe('ensureCertificateImported method', () => {
let keychain: Keychain<Ios.Job>;
let keychain: Keychain;
const certificatePath = path.join(os.tmpdir(), `cert-${uuid()}.p12`);
beforeAll(async () => {
@@ -36,37 +32,34 @@ describe('Keychain class', () => {
});
beforeEach(async () => {
ctx = new BuildContext({ projectRootDirectory: '.' } as Ios.Job, {
workingdir: '/workingdir',
logBuffer: { getLogs: () => [], getPhaseLogs: () => [] },
logger: mockLogger,
env: {},
uploadArtifact: jest.fn(),
});
keychain = new Keychain(ctx);
await keychain.create();
keychain = new Keychain();
await keychain.create({ logger: mockLogger });
});
afterEach(async () => {
await keychain.destroy();
await keychain.destroy({ logger: mockLogger });
});
it("should throw an error if the certificate hasn't been imported", async () => {
await expect(
keychain.ensureCertificateImported(
distributionCertificate.teamId,
distributionCertificate.fingerprint
)
keychain.ensureCertificateImported({
teamId: distributionCertificate.teamId,
fingerprint: distributionCertificate.fingerprint,
})
).rejects.toThrowError(/hasn't been imported successfully/);
});
it("shouldn't throw any error if the certificate has been imported successfully", async () => {
await keychain.importCertificate(certificatePath, distributionCertificate.password);
await keychain.importCertificate({
logger: mockLogger,
certPath: certificatePath,
certPassword: distributionCertificate.password,
});
await expect(
keychain.ensureCertificateImported(
distributionCertificate.teamId,
distributionCertificate.fingerprint
)
keychain.ensureCertificateImported({
teamId: distributionCertificate.teamId,
fingerprint: distributionCertificate.fingerprint,
})
).resolves.not.toThrow();
});
});
@@ -0,0 +1,133 @@
import { errors } from '@expo/eas-build-job';
import { createLogger } from '@expo/logger';
import { SpawnResult } from '@expo/turtle-spawn';
import { runFastlane } from '../../fastlane';
import Keychain from '../keychain';
jest.mock('../../fastlane');
const runFastlaneMock = jest.mocked(runFastlane);
const logger = createLogger({ name: 'test' });
const logError = jest.spyOn(logger, 'error').mockImplementation();
let keychain: Keychain;
const options = {
logger,
certPath: '/test.p12',
certPassword: 'certificate-password',
};
const result = (stdout: string, stderr = ''): SpawnResult => ({
stdout,
stderr,
output: [stdout, stderr],
pid: 1,
status: 0,
signal: null,
});
beforeEach(async () => {
keychain = new Keychain();
runFastlaneMock.mockResolvedValueOnce(result(''));
await keychain.create({ logger });
runFastlaneMock.mockClear();
});
it('runs the import with the certificate and keychain credentials, without streaming output', async () => {
runFastlaneMock.mockResolvedValueOnce(result('Result: true'));
await keychain.importCertificate(options);
expect(runFastlaneMock).toHaveBeenCalledWith([
'run',
'import_certificate',
'certificate_path:/test.p12',
'certificate_password:certificate-password',
`keychain_path:${keychain.data.path}`,
`keychain_password:${keychain.data.password}`,
]);
expect(logError).not.toHaveBeenCalled();
});
describe.each(['success', 'failure'])('Fastlane %s', outcome => {
it.each([
{
output: 'SecKeychainItemImport: MAC verification failed during PKCS12 import',
codes: ['PKCS12_MAC_VERIFICATION_FAILED'],
},
{
output: 'SecKeychainItemImport: Unknown format in import',
codes: ['PKCS12_UNKNOWN_FORMAT'],
},
{
output: 'SecKeychainItemImport: Some other error',
codes: ['CERTIFICATE_IMPORT_FAILED'],
},
{
output:
'SecKeychainItemImport: Unknown format in import\nSecKeychainItemSetAccessWithPassword: Access denied',
codes: ['PKCS12_UNKNOWN_FORMAT', 'PRIVATE_KEY_ACCESS_FAILED'],
},
{
output:
'seckeychainitemimport: unknown format in import\nSecKeychainItemImport: Some other error',
codes: ['PKCS12_UNKNOWN_FORMAT', 'CERTIFICATE_IMPORT_FAILED'],
},
])('reports only the applicable codes: $codes', async ({ output, codes }) => {
if (outcome === 'success') {
runFastlaneMock.mockResolvedValueOnce(result(output));
await keychain.importCertificate(options);
} else {
runFastlaneMock.mockRejectedValueOnce(
Object.assign(new Error('import failed'), {
stderr: output,
})
);
const failure = await keychain.importCertificate(options).catch(error => error);
expect(failure.metadata.diagnosticCodes).toEqual(codes);
}
expect(logError.mock.calls.map(([fields]) => fields.diagnosticCode)).toEqual(codes);
});
});
it('never forwards output, command arguments, private key data, or the original error', async () => {
const secret = 'very-secret-password';
const error = Object.assign(new Error(`fastlane certificate_password:${secret}`), {
stdout: `private key data ${secret}`,
stderr: `security: SecKeychainItemSetAccessWithPassword: ${secret}`,
});
runFastlaneMock.mockRejectedValueOnce(error);
const failure = await keychain.importCertificate(options).catch(error => error);
expect(failure).toBeInstanceOf(errors.UserError);
expect(failure.errorCode).toBe(errors.ErrorCode.UNKNOWN_ERROR);
expect(failure.metadata).toEqual({ diagnosticCodes: ['PRIVATE_KEY_ACCESS_FAILED'] });
expect(failure.toExternalExpoError().message).toContain(
'Fastlane could not complete certificate import'
);
expect(failure.cause).toBeUndefined();
expect(JSON.stringify([failure, logError.mock.calls])).not.toContain(secret);
expect(logError).toHaveBeenCalledWith(
{ diagnosticCode: 'PRIVATE_KEY_ACCESS_FAILED' },
expect.stringContaining('private key')
);
});
it.each(['ENOENT', 'EACCES'])(
'classifies a %s process launch failure as a system error',
async code => {
runFastlaneMock.mockRejectedValueOnce(Object.assign(new Error('secret command'), { code }));
const error = await keychain.importCertificate(options).catch(error => error);
expect(error).toBeInstanceOf(errors.SystemError);
expect(error.trackingCode).toBe('IOS_CERTIFICATE_IMPORT_PROCESS_START_FAILED');
expect(error.cause).toBeUndefined();
}
);
it.each([
null,
new Error('failure'),
{ stderr: 'SecKeychainItemImport: Unknown format in import' },
Object.assign(new Error('failure'), { stdout: 123, stderr: {} }),
])('handles rejected values without usable process output: %p', async value => {
runFastlaneMock.mockRejectedValueOnce(value);
await expect(keychain.importCertificate(options)).rejects.toThrow(
'Fastlane could not complete certificate import'
);
expect(logError).not.toHaveBeenCalled();
});
@@ -18,7 +18,7 @@ jest.unmock('fs');
describe('ProvisioningProfile class', () => {
describe('verifyCertificate method', () => {
let ctx: BuildContext<Ios.Job>;
let keychain: Keychain<Ios.Job>;
let keychain: Keychain;
beforeAll(async () => {
ctx = new BuildContext({ projectRootDirectory: '.' } as Ios.Job, {
@@ -30,12 +30,12 @@ describe('ProvisioningProfile class', () => {
},
uploadArtifact: jest.fn(),
});
keychain = new Keychain(ctx);
await keychain.create();
keychain = new Keychain();
await keychain.create({ logger: mockLogger });
});
afterAll(async () => {
await keychain.destroy();
await keychain.destroy({ logger: mockLogger });
});
it("shouldn't throw any error if the provisioning profile and distribution certificate match", async () => {
@@ -1,19 +1,53 @@
import { Ios } from '@expo/eas-build-job';
import spawn from '@expo/turtle-spawn';
import { errors } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import spawn, { SpawnResult } from '@expo/turtle-spawn';
import os from 'os';
import path from 'path';
import { v4 as uuid } from 'uuid';
import { BuildContext } from '../../context';
import { runFastlane } from '../fastlane';
export default class Keychain<TJob extends Ios.Job> {
// Do not log raw fastlane output or spawn errors: they can contain passwords,
// command arguments, and private key attributes from set-key-partition-list.
const IMPORT_CERTIFICATE_DIAGNOSTICS = [
{
code: 'PKCS12_MAC_VERIFICATION_FAILED',
pattern: /SecKeychainItemImport: MAC verification failed during PKCS12 import/i,
message:
'macOS could not verify the PKCS#12 MAC. Check the certificate password and PKCS#12 export format; this error does not prove that the password is wrong.',
},
{
code: 'PKCS12_UNKNOWN_FORMAT',
pattern: /SecKeychainItemImport: Unknown format in import/i,
message: 'macOS did not recognize the certificate import format.',
},
{
code: 'CERTIFICATE_IMPORT_FAILED',
pattern:
/SecKeychainItemImport:(?! MAC verification failed during PKCS12 import| Unknown format in import)/i,
message: 'macOS reported a certificate import error (SecKeychainItemImport).',
},
{
code: 'PRIVATE_KEY_ACCESS_FAILED',
pattern: /SecKeychainItemSetAccessWithPassword:/,
message:
'macOS could not set access to the imported private key (SecKeychainItemSetAccessWithPassword).',
},
{
code: 'KEYCHAIN_ITEM_LOOKUP_FAILED',
pattern: /SecItemCopyMatching:/,
message:
'macOS could not find an item while configuring private key access (SecItemCopyMatching).',
},
];
export default class Keychain {
private readonly keychainPath: string;
private readonly keychainPassword: string;
private created = false;
private destroyed = false;
constructor(private readonly ctx: BuildContext<TJob>) {
constructor() {
this.keychainPath = path.join(os.tmpdir(), `eas-build-${uuid()}.keychain`);
this.keychainPassword = uuid();
}
@@ -25,8 +59,8 @@ export default class Keychain<TJob extends Ios.Job> {
};
}
public async create(): Promise<void> {
this.ctx.logger.debug(`Creating keychain - ${this.keychainPath}`);
public async create({ logger }: { logger: bunyan }): Promise<void> {
logger.debug(`Creating keychain - ${this.keychainPath}`);
await runFastlane([
'run',
'create_keychain',
@@ -38,23 +72,77 @@ export default class Keychain<TJob extends Ios.Job> {
this.created = true;
}
public async importCertificate(certPath: string, certPassword: string): Promise<void> {
public async importCertificate({
logger,
certPath,
certPassword,
}: {
logger: bunyan;
certPath: string;
certPassword: string;
}): Promise<void> {
if (!this.created) {
throw new Error('You must create a keychain first.');
}
this.ctx.logger.debug(`Importing certificate ${certPath} into keychain ${this.keychainPath}`);
await runFastlane([
'run',
'import_certificate',
`certificate_path:${certPath}`,
`certificate_password:${certPassword}`,
`keychain_path:${this.keychainPath}`,
`keychain_password:${this.keychainPassword}`,
]);
logger.debug(`Importing certificate ${certPath} into keychain ${this.keychainPath}`);
try {
const result = await runFastlane([
'run',
'import_certificate',
`certificate_path:${certPath}`,
`certificate_password:${certPassword}`,
`keychain_path:${this.keychainPath}`,
`keychain_password:${this.keychainPassword}`,
]);
const output = [result.stdout, result.stderr].join('\n');
for (const diagnostic of IMPORT_CERTIFICATE_DIAGNOSTICS) {
if (diagnostic.pattern.test(output)) {
logger.error({ diagnosticCode: diagnostic.code }, diagnostic.message);
}
}
} catch (error) {
const processError =
error instanceof Error
? (error as Error &
Partial<Pick<SpawnResult, 'stdout' | 'stderr'> & NodeJS.ErrnoException>)
: undefined;
const output = [processError?.stdout, processError?.stderr]
.filter(value => typeof value === 'string')
.join('\n');
const diagnosticCodes: string[] = [];
for (const diagnostic of IMPORT_CERTIFICATE_DIAGNOSTICS) {
if (diagnostic.pattern.test(output)) {
diagnosticCodes.push(diagnostic.code);
logger.error({ diagnosticCode: diagnostic.code }, diagnostic.message);
}
}
// Never attach the original error: its message includes passwords.
if (processError?.code === 'ENOENT' || processError?.code === 'EACCES') {
throw new errors.SystemError('Fastlane could not be started to import the certificate.', {
trackingCode: 'IOS_CERTIFICATE_IMPORT_PROCESS_START_FAILED',
});
}
throw new errors.UserError(
errors.ErrorCode.UNKNOWN_ERROR,
'Fastlane could not complete certificate import. Check the certificate import diagnostics in the Prepare credentials logs.',
{
trackingCode: 'IOS_CERTIFICATE_IMPORT_FAILED',
metadata: { diagnosticCodes },
}
);
}
}
public async ensureCertificateImported(teamId: string, fingerprint: string): Promise<void> {
public async ensureCertificateImported({
teamId,
fingerprint,
}: {
teamId: string;
fingerprint: string;
}): Promise<void> {
const identities = await this.findIdentitiesByTeamId(teamId);
if (!identities.includes(fingerprint)) {
throw new Error(
@@ -63,27 +151,33 @@ export default class Keychain<TJob extends Ios.Job> {
}
}
public async destroy(keychainPath?: string): Promise<void> {
public async destroy({
logger,
keychainPath,
}: {
logger: bunyan;
keychainPath?: string;
}): Promise<void> {
if (!keychainPath && !this.created) {
this.ctx.logger.warn("There is nothing to destroy, a keychain hasn't been created yet.");
logger.warn("There is nothing to destroy, a keychain hasn't been created yet.");
return;
}
if (this.destroyed) {
this.ctx.logger.warn('The keychain has been already destroyed');
logger.warn('The keychain has been already destroyed');
return;
}
const keychainToDeletePath = keychainPath ?? this.keychainPath;
this.ctx.logger.info(`Destroying keychain - ${keychainToDeletePath}`);
logger.info(`Destroying keychain - ${keychainToDeletePath}`);
try {
await runFastlane(['run', 'delete_keychain', `keychain_path:${keychainToDeletePath}`]);
this.destroyed = true;
} catch (err) {
this.ctx.logger.error({ err }, 'Failed to delete the keychain\n');
logger.error({ err }, 'Failed to delete the keychain\n');
throw err;
}
}
public async cleanUpKeychains(): Promise<void> {
public async cleanUpKeychains({ logger }: { logger: bunyan }): Promise<void> {
const { stdout } = await spawn('security', ['list-keychains'], { stdio: 'pipe' });
const keychainList = (/"(.*)"/g.exec(stdout) ?? ([] as string[])).map(i =>
i.slice(1, i.length - 1)
@@ -92,7 +186,7 @@ export default class Keychain<TJob extends Ios.Job> {
/eas-build-[\w-]+\.keychain$/.exec(keychain)
);
for (const turtleKeychainPath of turtleKeychainList) {
await this.destroy(turtleKeychainPath);
await this.destroy({ logger, keychainPath: turtleKeychainPath });
}
}
@@ -26,7 +26,7 @@ export interface Credentials {
export type TargetProvisioningProfiles = Record<string, ProvisioningProfileData>;
export default class IosCredentialsManager<TJob extends Ios.Job> {
private keychain?: Keychain<TJob>;
private keychain?: Keychain;
private readonly provisioningProfiles: ProvisioningProfile<TJob>[] = [];
private cleanedUp = false;
@@ -48,8 +48,8 @@ export default class IosCredentialsManager<TJob extends Ios.Job> {
this.ctx.logger.info('Preparing credentials');
this.ctx.logger.info('Creating keychain');
this.keychain = new Keychain(this.ctx);
await this.keychain.create();
this.keychain = new Keychain();
await this.keychain.create({ logger: this.ctx.logger });
const targets = Object.keys(buildCredentials);
const targetProvisioningProfiles: TargetProvisioningProfiles = {};
@@ -82,7 +82,7 @@ export default class IosCredentialsManager<TJob extends Ios.Job> {
}
if (this.keychain) {
await this.keychain.destroy();
await this.keychain.destroy({ logger: this.ctx.logger });
}
if (this.provisioningProfiles) {
for (const provisioningProfile of this.provisioningProfiles) {
@@ -120,10 +120,11 @@ export default class IosCredentialsManager<TJob extends Ios.Job> {
);
this.ctx.logger.info('Importing distribution certificate into the keychain');
await this.keychain.importCertificate(
distCertPath,
targetCredentials.distributionCertificate.password
);
await this.keychain.importCertificate({
logger: this.ctx.logger,
certPath: distCertPath,
certPassword: targetCredentials.distributionCertificate.password,
});
this.ctx.logger.info('Initializing provisioning profile');
const provisioningProfile = new ProvisioningProfile(
@@ -138,10 +139,10 @@ export default class IosCredentialsManager<TJob extends Ios.Job> {
this.ctx.logger.info(
'Validating whether the distribution certificate has been imported successfully'
);
await this.keychain.ensureCertificateImported(
provisioningProfile.data.teamId,
certificateFingerprint
);
await this.keychain.ensureCertificateImported({
teamId: provisioningProfile.data.teamId,
fingerprint: certificateFingerprint,
});
this.ctx.logger.info(
'Verifying whether the distribution certificate and provisioning profile match'
@@ -19,6 +19,7 @@ import { generateGymfileFromTemplateFunction } from './functions/generateGymfile
import { createGetCredentialsForBuildTriggeredByGithubIntegration } from './functions/getCredentialsForBuildTriggeredByGitHubIntegration';
import { injectAndroidCredentialsFunction } from './functions/injectAndroidCredentials';
import { createInstallMaestroBuildFunction } from './functions/installMaestro';
import { createInstallMitmproxyBuildFunction } from './functions/installMitmproxy';
import { createInstallBuildFunction } from './functions/installBuild';
import { createInstallNodeModulesBuildFunction } from './functions/installNodeModules';
import { createInstallPodsBuildFunction } from './functions/installPods';
@@ -45,6 +46,7 @@ import { createRolloutPosthogFlagFunction } from './functions/rolloutPosthogFlag
import { createSaveCacheFunction } from './functions/saveCache';
import { createSendSlackMessageFunction } from './functions/sendSlackMessage';
import { createStartAgentDeviceRemoteSessionBuildFunction } from './functions/startAgentDeviceRemoteSession';
import { createStartAgentDeviceSessionBuildFunction } from './functions/startAgentDeviceSession';
import { createStartAndroidEmulatorBuildFunction } from './functions/startAndroidEmulator';
import { createStartArgentRemoteSessionBuildFunction } from './functions/startArgentRemoteSession';
import { createStartAppiumRemoteSessionBuildFunction } from './functions/startAppiumRemoteSession';
@@ -54,11 +56,14 @@ import { createStartIosSimulatorRecordingsBuildFunction } from './functions/star
import { createStartLocalEgressBuildFunction } from './functions/startLocalEgress';
import { createStartWebPreviewRemoteSessionBuildFunction } from './functions/startWebPreviewRemoteSession';
import { createStartServeSimMetricsBuildFunction } from './functions/startServeSimMetrics';
import { createStartServeSimLogsBuildFunction } from './functions/startServeSimLogs';
import { createCollectServeSimLogsBuildFunction } from './functions/collectServeSimLogs';
import { createStartSandboxBuildFunction } from './functions/startSandbox';
import { createCollectServeSimMetricsBuildFunction } from './functions/collectServeSimMetrics';
import { createUploadArtifactBuildFunction } from './functions/uploadArtifact';
import { createUploadDeviceRunSessionScreenRecordingsBuildFunction } from './functions/uploadDeviceRunSessionScreenRecordings';
import { createUploadToAscBuildFunction } from './functions/uploadToAsc';
import { createUpdateTestFlightMetadataBuildFunction } from './functions/updateTestFlightMetadata';
import { createSetUpNpmrcBuildFunction } from './functions/useNpmToken';
import { createWaitForPosthogMetricFunction } from './functions/waitForPosthogMetric';
import { createUploadPosthogSourcemapsFunction } from './functions/uploadPosthogSourcemaps';
@@ -100,6 +105,7 @@ export function getEasFunctions(ctx: CustomBuildContext): BuildFunction[] {
runFastlaneFunction(),
parseXcactivitylogFunction(),
createStartAgentDeviceRemoteSessionBuildFunction(ctx),
createStartAgentDeviceSessionBuildFunction(ctx),
createStartArgentRemoteSessionBuildFunction(ctx),
createStartAppiumRemoteSessionBuildFunction(ctx),
createStartAndroidEmulatorBuildFunction(),
@@ -111,9 +117,12 @@ export function getEasFunctions(ctx: CustomBuildContext): BuildFunction[] {
createUploadDeviceRunSessionScreenRecordingsBuildFunction(ctx),
createStartWebPreviewRemoteSessionBuildFunction(ctx),
createStartServeSimMetricsBuildFunction(),
createStartServeSimLogsBuildFunction(),
createCollectServeSimLogsBuildFunction(ctx),
createStartSandboxBuildFunction(ctx),
createCollectServeSimMetricsBuildFunction(ctx),
createInstallMaestroBuildFunction(),
createInstallMitmproxyBuildFunction(),
createInstallPodsBuildFunction(),
createSendSlackMessageFunction(),
@@ -128,6 +137,7 @@ export function getEasFunctions(ctx: CustomBuildContext): BuildFunction[] {
createSubmissionEntityFunction(),
createUploadToAscBuildFunction(),
createUpdateTestFlightMetadataBuildFunction(),
createReportMaestroTestResultsFunction(ctx),
createMaestroTestsBuildFunction(ctx),
@@ -0,0 +1,245 @@
import { BuildRuntimePlatform, type BuildStepContext } from '@expo/steps';
import spawn from '@expo/turtle-spawn';
import fs from 'node:fs';
import type { CustomBuildContext } from '../../../customBuildContext';
import { Sentry } from '../../../sentry';
import { AndroidEmulatorUtils } from '../../../utils/AndroidEmulatorUtils';
import { turtleFetch } from '../../../utils/turtleFetch';
import { startAgentDeviceEventCollectionAsync } from '../../utils/agentDeviceEvents';
import { startAppiumEventCollectionAsync } from '../../utils/appiumEvents';
import { startDeviceSessionHostAsync } from '../../utils/deviceSessionHost';
import {
spawnDetached,
startNgrokTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
waitForFileAsync,
} from '../../utils/remoteDeviceRunSession';
import { createStartAgentDeviceRemoteSessionBuildFunction } from '../startAgentDeviceRemoteSession';
import { createStartAppiumRemoteSessionBuildFunction } from '../startAppiumRemoteSession';
jest.mock('@expo/turtle-spawn');
jest.mock('../../../sentry');
jest.mock('../../../utils/turtleFetch');
jest.mock('../../../utils/AndroidEmulatorUtils');
jest.mock('../../utils/deviceSessionHost');
jest.mock('../../utils/agentDeviceEvents');
jest.mock('../../utils/appiumEvents');
jest.mock('../../utils/agentDeviceArtifacts');
jest.mock('../../utils/remoteDeviceRunSession', () => ({
...jest.requireActual('../../utils/remoteDeviceRunSession'),
spawnDetached: jest.fn(),
startNgrokTunnelAsync: jest.fn(),
uploadRemoteSessionConfigAsync: jest.fn(),
waitForDeviceRunSessionStoppedAsync: jest.fn(),
waitForFileAsync: jest.fn(),
}));
const finishHost = jest.fn();
const closePreview = jest.fn();
const stopTool = jest.fn();
const stopTunnel = jest.fn();
const stopEvents = jest.fn();
const openPreview = jest.fn();
const logger = { info: jest.fn(), warn: jest.fn(), child: jest.fn().mockReturnThis() };
beforeEach(() => {
jest.clearAllMocks();
// The daemon package is external. No installation or daemon process runs in these tests.
jest.spyOn(fs, 'existsSync').mockReturnValue(true);
jest.mocked(spawn).mockResolvedValue({ stdout: '{}' } as never);
jest
.mocked(AndroidEmulatorUtils.getAttachedDevicesAsync)
.mockResolvedValue([{ serialId: 'emulator-5554', state: 'device' } as never]);
jest.mocked(turtleFetch).mockResolvedValue({ ok: true } as never);
jest.mocked(waitForFileAsync).mockResolvedValue({ port: 4567, token: 'daemon-token' });
for (const stop of [finishHost, closePreview, stopTool, stopTunnel, stopEvents]) {
stop.mockResolvedValue(undefined);
}
openPreview.mockResolvedValue({
previewPageUrl: 'https://expo.dev/simulator-preview/preview-id',
apiUrl: 'https://preview.example.test',
closeAsync: closePreview,
});
jest
.mocked(startDeviceSessionHostAsync)
.mockResolvedValue({ openPreviewAsync: openPreview, finishAsync: finishHost });
jest.mocked(spawnDetached).mockReturnValue({
pid: undefined,
getOutput: () => '',
getExitError: () => undefined,
stopAsync: stopTool,
});
jest.mocked(startNgrokTunnelAsync).mockResolvedValue({
url: 'https://tool.example.test',
subdomainId: 'tool-id',
stopAsync: stopTunnel,
});
const events = { stopAsync: stopEvents, getLastEventObservedAt: () => undefined };
jest.mocked(startAgentDeviceEventCollectionAsync).mockResolvedValue(events);
jest.mocked(startAppiumEventCollectionAsync).mockResolvedValue(events);
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockResolvedValue(undefined);
});
afterEach(() => {
jest.restoreAllMocks();
});
describe.each([
['Appium', createStartAppiumRemoteSessionBuildFunction],
['Agent Device', createStartAgentDeviceRemoteSessionBuildFunction],
] as const)('%s host ownership', (name, createFunction) => {
async function runAsync() {
const fn = createFunction({} as CustomBuildContext);
await fn.fn!(
{
logger,
global: { runtimePlatform: BuildRuntimePlatform.LINUX },
} as unknown as BuildStepContext,
{
inputs: {
package_version: { value: undefined },
max_idle_time_minutes: { value: undefined },
},
outputs: {},
env: {
DEVICE_RUN_SESSION_ID: 'session-id',
EAS_SIMULATOR_NGROK_TUNNEL_DOMAIN: 'example.test',
NGROK_AUTHTOKEN: 'token',
},
} as never
);
}
it('holds the host until session completion', async () => {
await runAsync();
expect(openPreview).toHaveBeenCalledWith({ baseDomain: 'example.test' });
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledWith(
expect.objectContaining({
remoteConfig: expect.objectContaining({
webPreviewUrl: 'https://expo.dev/simulator-preview/preview-id',
previewApiUrl: 'https://preview.example.test',
}),
})
);
expect(finishHost).toHaveBeenCalledTimes(1);
expect(closePreview).not.toHaveBeenCalled();
expect(finishHost.mock.invocationCallOrder[0]).toBeGreaterThan(
jest.mocked(waitForDeviceRunSessionStoppedAsync).mock.invocationCallOrder[0]
);
expect(stopTool).toHaveBeenCalledTimes(1);
expect(stopEvents).toHaveBeenCalledTimes(1);
expect(stopTunnel).toHaveBeenCalledTimes(1);
});
it('stops automation while recording finalization or upload is pending', async () => {
let release!: () => void;
let stopped!: () => void;
const pendingFinish = new Promise<void>(resolve => {
release = resolve;
});
const toolStopped = new Promise<void>(resolve => {
stopped = resolve;
});
finishHost.mockReturnValueOnce(pendingFinish);
stopTool.mockImplementationOnce(async () => {
stopped();
});
let completed = false;
const running = runAsync().then(() => {
completed = true;
});
try {
await toolStopped;
expect(stopTunnel).toHaveBeenCalledTimes(1);
expect(finishHost).toHaveBeenCalledTimes(1);
expect(completed).toBe(false);
} finally {
release();
await running;
}
});
it('finalizes recording and stops automation while the tunnel close is pending', async () => {
let release!: () => void;
let finished!: () => void;
const pendingClose = new Promise<void>(resolve => {
release = resolve;
});
const hostFinished = new Promise<void>(resolve => {
finished = resolve;
});
const toolStopped = new Promise<void>(resolve => {
stopTool.mockImplementationOnce(async () => resolve());
});
stopTunnel.mockReturnValueOnce(pendingClose);
finishHost.mockImplementationOnce(async () => {
finished();
});
const running = runAsync();
try {
// Both settle while the tunnel close is still held open.
await Promise.all([hostFinished, toolStopped]);
expect(stopTunnel).toHaveBeenCalledTimes(1);
} finally {
release();
await running;
}
});
it('still stops the automation process when event collection stop rejects', async () => {
stopEvents.mockRejectedValueOnce(new Error('collector failed'));
// Agent Device wraps its collector stop in a catch-all helper; Appium does not.
const outcome = await runAsync().then(
() => 'resolved',
(err: Error) => err.message
);
expect(outcome).toBe(name === 'Appium' ? 'collector failed' : 'resolved');
expect(stopTool).toHaveBeenCalledTimes(1);
expect(stopTunnel).toHaveBeenCalledTimes(1);
expect(finishHost).toHaveBeenCalledTimes(1);
});
it('still stops automation when recording finalization rejects', async () => {
finishHost.mockRejectedValueOnce(new Error('recording cleanup failed'));
await expect(runAsync()).rejects.toThrow('recording cleanup failed');
expect(stopTool).toHaveBeenCalledTimes(1);
expect(stopTunnel).toHaveBeenCalledTimes(1);
expect(Sentry.capture).not.toHaveBeenCalled();
});
it.each(['preview', 'config', 'wait'])('finishes the host after %s fails', async phase => {
const error = new Error(`${phase} failed`);
// A teardown failure must not replace the error that ended the session.
finishHost.mockRejectedValueOnce(new Error('recording cleanup failed'));
if (phase === 'preview') {
openPreview.mockRejectedValueOnce(error);
} else if (phase === 'config') {
jest.mocked(uploadRemoteSessionConfigAsync).mockRejectedValueOnce(error);
} else {
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockRejectedValueOnce(error);
}
await expect(runAsync()).rejects.toBe(error);
expect(finishHost).toHaveBeenCalledTimes(1);
// An agent-device preview failure aborts the daemon task, which then may not start its
// daemon or tunnel. Everything that started must be stopped.
const abortsDaemonTask = name === 'Agent Device' && phase === 'preview';
expect(stopTool).toHaveBeenCalledTimes(
abortsDaemonTask ? jest.mocked(spawnDetached).mock.calls.length : 1
);
expect(stopTunnel).toHaveBeenCalledTimes(
abortsDaemonTask ? jest.mocked(startNgrokTunnelAsync).mock.calls.length : 1
);
expect(logger.warn).toHaveBeenCalledWith(
{ err: expect.objectContaining({ message: 'recording cleanup failed' }) },
'Could not stop the session host during remote session teardown.'
);
expect(Sentry.capture).toHaveBeenCalledWith(
'Could not stop the session host after the remote session failed',
expect.objectContaining({ message: 'recording cleanup failed' }),
{ level: 'warning' }
);
});
});
@@ -11,6 +11,7 @@ import * as tar from 'tar';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { createMockLogger } from '../../../__tests__/utils/logger';
import * as files from '../../../utils/files';
import { createDownloadBuildFunction, downloadBuildAsync } from '../downloadBuild';
// contains a 'TestApp.app/TestApp' file with 'i am executable' content
@@ -255,6 +256,144 @@ describe('downloadBuild', () => {
expect(await fs.promises.readFile(artifactPath, 'utf-8')).toBe('hello');
});
it('passes the abort signal to the build lookup', async () => {
const graphqlClient = createMockGraphqlClient({
applicationArchiveUrl: APPLICATION_ARCHIVE_URL,
});
jest
.mocked(fetch)
.mockResolvedValue(
createSuccessfulResponse({ body: APP_TAR_GZ_BUFFER, url: APPLICATION_ARCHIVE_URL })
);
const buildId = randomUUID();
await downloadBuildAsync({
logger: createLogger({ name: 'test' }),
buildId,
graphqlClient,
robotAccessToken: null,
extensions: ['app'],
signal: new AbortController().signal,
});
expect(graphqlClient.query).toHaveBeenCalledWith(
expect.anything(),
{ buildId },
{ fetch: expect.any(Function) }
);
expect(jest.mocked(fetch)).toHaveBeenCalledWith(
APPLICATION_ARCHIVE_URL,
expect.objectContaining({ signal: expect.any(Object) })
);
});
it('stops a pending build lookup when aborted', async () => {
// The lookup request runs through the abort-aware fetch from graphqlAbortContext and
// hangs until its signal fires, like a stalled API call.
const realFetch = globalThis.fetch;
globalThis.fetch = jest.fn(
(_input: unknown, init?: RequestInit) =>
new Promise<never>((_resolve, reject) => {
init!.signal!.addEventListener('abort', () => reject(init!.signal!.reason));
})
) as typeof globalThis.fetch;
const graphqlClient = {
query: jest.fn((_query, _variables, context: { fetch: typeof globalThis.fetch }) => ({
toPromise: () =>
context.fetch('https://api.expo.dev/graphql', {}).then(
() => ({ data: undefined }),
(networkError: Error) => ({ error: new CombinedError({ networkError }) })
),
})),
} as unknown as Client;
const controller = new AbortController();
const failure = new Error('daemon failed');
try {
const download = downloadBuildAsync({
logger: createLogger({ name: 'test' }),
buildId: randomUUID(),
graphqlClient,
robotAccessToken: null,
extensions: ['app'],
signal: controller.signal,
});
await new Promise(resolve => setImmediate(resolve));
expect(globalThis.fetch).toHaveBeenCalledTimes(1);
controller.abort(failure);
await expect(download).rejects.toBe(failure);
expect(jest.mocked(fetch)).not.toHaveBeenCalled();
} finally {
globalThis.fetch = realFetch;
}
});
it('lets a started extraction finish before it reports an abort', async () => {
jest
.mocked(fetch)
.mockResolvedValue(
createSuccessfulResponse({ body: APP_TAR_GZ_BUFFER, url: APPLICATION_ARCHIVE_URL })
);
const { decompressTarAsync } = jest.requireActual<typeof files>('../../../utils/files');
const controller = new AbortController();
const failure = new Error('daemon failed');
let extractionDirectory: string | undefined;
let extractionFinished = false;
const decompressSpy = jest.spyOn(files, 'decompressTarAsync').mockImplementation(async args => {
extractionDirectory = args.destinationDirectory;
// The startup fails while the archive is being extracted.
controller.abort(failure);
await decompressTarAsync(args);
extractionFinished = true;
});
try {
await expect(
downloadBuildAsync({
logger: createLogger({ name: 'test' }),
applicationArchiveUrl: APPLICATION_ARCHIVE_URL,
graphqlClient: createMockGraphqlClient({}),
robotAccessToken: null,
extensions: ['app'],
signal: controller.signal,
})
).rejects.toBe(failure);
// The download settled only after the extraction wrote its last file.
expect(extractionFinished).toBe(true);
expect(
await fs.promises.readFile(
path.join(extractionDirectory!, 'TestApp.app', 'TestApp'),
'utf8'
)
).toBe('i am executable\n');
} finally {
decompressSpy.mockRestore();
}
});
it('does not look up the build when already aborted', async () => {
const graphqlClient = createMockGraphqlClient({
applicationArchiveUrl: APPLICATION_ARCHIVE_URL,
});
const failure = new Error('boot failed');
await expect(
downloadBuildAsync({
logger: createLogger({ name: 'test' }),
buildId: randomUUID(),
graphqlClient,
robotAccessToken: null,
extensions: ['app'],
signal: AbortSignal.abort(failure),
})
).rejects.toBe(failure);
expect(graphqlClient.query).not.toHaveBeenCalled();
expect(jest.mocked(fetch)).not.toHaveBeenCalled();
});
it('rejects missing or ambiguous build sources', async () => {
const graphqlClient = createMockGraphqlClient({
applicationArchiveUrl: APPLICATION_ARCHIVE_URL,
@@ -424,6 +563,34 @@ describe('downloadBuild', () => {
});
describe('createDownloadBuildFunction', () => {
it('aborts a pending archive request when the step times out', async () => {
let requestAborted = false;
jest.mocked(fetch).mockImplementation(
async (_url, options) =>
await new Promise<Response>((_resolve, reject) => {
options?.signal?.addEventListener('abort', () => {
requestAborted = true;
reject(new Error('Archive request aborted'));
});
})
);
const graphqlClient = createMockGraphqlClient({});
const step = createDownloadBuildFunction({
graphqlClient,
} as any).createBuildStepFromFunctionCall(
createGlobalContextMock({ staticContextContent: { job: {} } }),
{
callInputs: { application_archive_url: APPLICATION_ARCHIVE_URL, extensions: ['ipa'] },
timeoutMs: 100,
}
);
await expect(step.executeAsync()).rejects.toThrow('timed out');
expect(fetch).toHaveBeenCalledTimes(1);
expect(requestAborted).toBe(true);
expect(graphqlClient.query).not.toHaveBeenCalled();
});
it('should download a build', async () => {
const buildId = randomUUID();
const graphqlClient = createMockGraphqlClient({
@@ -0,0 +1,69 @@
import { BuildRuntimePlatform, BuildStep } from '@expo/steps';
import spawn from '@expo/turtle-spawn';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { createInstallMitmproxyBuildFunction } from '../installMitmproxy';
jest.mock('@expo/turtle-spawn', () => ({
__esModule: true,
default: jest.fn(),
}));
const mockedSpawn = jest.mocked(spawn);
describe('createInstallMitmproxyBuildFunction', () => {
beforeEach(() => {
jest.clearAllMocks();
});
function createStep(env: Record<string, string>): BuildStep {
const globalCtx = createGlobalContextMock({ runtimePlatform: BuildRuntimePlatform.DARWIN });
globalCtx.updateEnv(env);
return createInstallMitmproxyBuildFunction().createBuildStepFromFunctionCall(globalCtx, {
callInputs: {},
});
}
it('does not install when mitmdump is already on PATH', async () => {
mockedSpawn.mockResolvedValueOnce({} as never);
await createStep({ EAS_BUILD_RUNNER: 'eas-build' }).executeAsync();
expect(mockedSpawn).toHaveBeenCalledWith('mitmdump', ['--version'], expect.anything());
expect(mockedSpawn).toHaveBeenCalledTimes(1);
});
it('warns instead of installing outside EAS Build VMs', async () => {
mockedSpawn.mockRejectedValueOnce(new Error('not found'));
await createStep({}).executeAsync();
expect(mockedSpawn).toHaveBeenCalledTimes(1);
});
it('installs the cask with Homebrew and leaves auto-update off', async () => {
mockedSpawn
.mockRejectedValueOnce(new Error('not found'))
.mockResolvedValueOnce({} as never)
.mockResolvedValueOnce({} as never);
await createStep({ EAS_BUILD_RUNNER: 'eas-build' }).executeAsync();
expect(mockedSpawn).toHaveBeenCalledWith(
'brew',
['install', '--cask', 'mitmproxy'],
expect.objectContaining({ env: expect.objectContaining({ HOMEBREW_NO_AUTO_UPDATE: '1' }) })
);
});
it('does not fail the job when mitmdump is still not runnable', async () => {
mockedSpawn
.mockRejectedValueOnce(new Error('not found'))
.mockResolvedValueOnce({} as never)
.mockRejectedValueOnce(new Error('not found'));
await expect(
createStep({ EAS_BUILD_RUNNER: 'eas-build' }).executeAsync()
).resolves.toBeUndefined();
});
});
@@ -256,7 +256,9 @@ describe('createMaestroTestsBuildFunction', () => {
expect(mockedSpawn).not.toHaveBeenCalled();
});
it('rejects sharding with maestro-runner', async () => {
it('passes shards to maestro-runner as parallel devices', async () => {
mockedSpawn.mockResolvedValue(SPAWN_SUCCESS);
jest.spyOn(fs, 'copyFile').mockResolvedValue();
const step = createStep({
flow_path: ['flows/a.yaml'],
platform: 'android',
@@ -264,26 +266,129 @@ describe('createMaestroTestsBuildFunction', () => {
shards: 2,
});
await expect(step.executeAsync()).rejects.toThrow(
'maestro-runner does not support EAS Maestro test sharding'
);
expect(mockedSpawn).not.toHaveBeenCalled();
await step.executeAsync();
expect(mockedSpawn.mock.calls[0][1]).toContain('--parallel=2');
});
it('rejects a non-junit output_format with maestro-runner', async () => {
it.each(['html', 'allure'])(
'accepts %s output_format with maestro-runner and keeps JUnit',
async format => {
mockedSpawn.mockResolvedValue(SPAWN_SUCCESS);
const copyFileSpy = jest.spyOn(fs, 'copyFile').mockResolvedValue();
const step = createStep({
flow_path: ['flows/a.yaml'],
platform: 'android',
backend: 'maestro-runner',
output_format: format,
});
await step.executeAsync();
expect(mockedSpawn.mock.calls[0][1]).toEqual(
expect.arrayContaining([
'--output=/home/expo/.maestro/tests/android-maestro-runner-attempt-0',
])
);
expect(copyFileSpy).toHaveBeenCalledWith(
'/home/expo/.maestro/tests/android-maestro-runner-attempt-0/junit-report.xml',
'/home/expo/.maestro/tests/junit-reports/android-maestro-junit-attempt-0.xml'
);
expect(parser.mergeJUnitReports).not.toHaveBeenCalled();
expect(step.getOutputValueByName('final_report_path')).toBe(
format === 'html'
? '/home/expo/.maestro/tests/android-maestro-runner-attempt-0/report.html'
: '/home/expo/.maestro/tests/android-maestro-runner-attempt-0/allure-results'
);
expect(mockedRunnerHarvest).not.toHaveBeenCalled();
expect(mockUploadArtifact).toHaveBeenCalledTimes(1);
expect(mockUploadArtifact).toHaveBeenCalledWith(
expect.objectContaining({
artifact: expect.objectContaining({
type: GenericArtifactType.OTHER,
name:
format === 'html' ? 'Maestro Runner HTML Report' : 'Maestro Runner Allure Results',
paths: [
format === 'html'
? '/home/expo/.maestro/tests/android-maestro-runner-attempt-0'
: '/home/expo/.maestro/tests/android-maestro-runner-attempt-0/allure-results',
],
}),
})
);
}
);
it('does not expose a selected runner report before an attempt starts', async () => {
const step = createStep({
flow_path: ['flows/a.yaml'],
flow_path: [],
platform: 'android',
backend: 'maestro-runner',
output_format: 'html',
});
await expect(step.executeAsync()).rejects.toThrow(UserError);
expect(step.getOutputValueByName('final_report_path')).toBeUndefined();
expect(step.getOutputValueByName('junit_report_directory')).toBe(
'/home/expo/.maestro/tests/junit-reports'
);
});
it('rejects an unknown output_format with maestro-runner', async () => {
const step = createStep({
flow_path: ['flows/a.yaml'],
platform: 'android',
backend: 'maestro-runner',
output_format: 'csv',
});
await expect(step.executeAsync()).rejects.toThrow(
'maestro-runner only supports the "junit" output_format'
'maestro-runner supports "junit", "html", and "allure" output_format values'
);
expect(mockedSpawn).not.toHaveBeenCalled();
});
it('uploads the last runner HTML report even when the test fails', async () => {
mockedSpawn.mockRejectedValue(rejectExit1());
jest.spyOn(fs, 'copyFile').mockResolvedValue();
const step = createStep({
flow_path: ['flows/a.yaml'],
platform: 'android',
backend: 'maestro-runner',
output_format: 'html',
retries: 1,
retry_failed_only: false,
});
await expect(step.executeAsync()).rejects.toThrow(UserError);
expect(mockUploadArtifact).toHaveBeenCalledWith(
expect.objectContaining({
artifact: expect.objectContaining({
name: 'Maestro Runner HTML Report',
paths: ['/home/expo/.maestro/tests/android-maestro-runner-attempt-1'],
}),
})
);
expect(step.getOutputValueByName('final_report_path')).toBe(
'/home/expo/.maestro/tests/android-maestro-runner-attempt-1/report.html'
);
});
it('does not fail a passing test when the runner report upload fails', async () => {
mockedSpawn.mockResolvedValue(SPAWN_SUCCESS);
jest.spyOn(fs, 'copyFile').mockResolvedValue();
mockUploadArtifact.mockRejectedValue(new Error('upload failed'));
const step = createStep({
flow_path: ['flows/a.yaml'],
platform: 'android',
backend: 'maestro-runner',
output_format: 'allure',
});
await expect(step.executeAsync()).resolves.toBeUndefined();
});
it('logs that maestro-runner does not support direct DADB', async () => {
mockedSpawn.mockResolvedValue(SPAWN_SUCCESS);
jest.spyOn(fs, 'copyFile').mockResolvedValue();
@@ -965,7 +1070,7 @@ describe('createMaestroTestsBuildFunction', () => {
await expect(step.executeAsync()).rejects.toThrow(UserError);
});
it('uses $HOME/.maestro/tests output path for non-junit formats (e.g. html)', async () => {
it('uses $HOME/.maestro/tests output path and uploads the Maestro CLI HTML report', async () => {
mockedSpawn.mockResolvedValue(SPAWN_SUCCESS);
const step = createStep({
@@ -976,11 +1081,119 @@ describe('createMaestroTestsBuildFunction', () => {
await step.executeAsync();
const args = mockedSpawn.mock.calls[0][1] as string[];
// Non-JUnit uses a fixed path inside $HOME/.maestro/tests so the
// whole-directory upload picks it up.
// Non-JUnit uses a fixed path inside $HOME/.maestro/tests.
const outputArg = args.find(a => a.startsWith('--output='));
expect(outputArg).toMatch(/\.maestro\/tests\/android-maestro-html\.html$/);
expect(outputArg).not.toMatch(/junit-reports/);
expect(step.getOutputValueByName('final_report_path')).toBe(
'/home/expo/.maestro/tests/android-maestro-html.html'
);
expect(mockUploadArtifact).toHaveBeenCalledWith(
expect.objectContaining({
artifact: expect.objectContaining({
type: GenericArtifactType.OTHER,
name: 'Maestro HTML Report',
paths: ['/home/expo/.maestro/tests/android-maestro-html.html'],
}),
})
);
});
it('uploads the Maestro CLI HTML report even when tests fail', async () => {
mockedSpawn.mockRejectedValue(rejectExit1());
const step = createStep({
flow_path: ['flows/a.yaml'],
output_format: 'html',
platform: 'android',
});
await expect(step.executeAsync()).rejects.toThrow(UserError);
expect(mockUploadArtifact).toHaveBeenCalledWith(
expect.objectContaining({
artifact: expect.objectContaining({
name: 'Maestro HTML Report',
paths: ['/home/expo/.maestro/tests/android-maestro-html.html'],
}),
})
);
});
it('warns that Maestro CLI does not support Allure reports', async () => {
mockedSpawn.mockRejectedValue(rejectExit1());
const logger = createMockLogger();
jest.mocked(logger.child).mockReturnValue(logger);
const step = createStep(
{
flow_path: ['flows/a.yaml'],
output_format: 'allure',
platform: 'android',
},
{ logger }
);
await expect(step.executeAsync()).rejects.toThrow(UserError);
expect(logger.warn).toHaveBeenCalledWith(
'Maestro CLI does not support Allure reports; no Allure artifact was uploaded.'
);
expect(mockUploadArtifact).not.toHaveBeenCalled();
});
it('clears the Maestro CLI HTML report before every retry attempt', async () => {
const removeSpy = jest.spyOn(fs, 'rm').mockResolvedValue();
mockedSpawn.mockRejectedValue(rejectExit1());
const step = createStep({
flow_path: ['flows/a.yaml'],
output_format: 'html',
platform: 'android',
retries: 1,
});
await expect(step.executeAsync()).rejects.toThrow(UserError);
expect(
removeSpy.mock.calls.filter(
([target]) => target === '/home/expo/.maestro/tests/android-maestro-html.html'
)
).toHaveLength(2);
expect(removeSpy).toHaveBeenNthCalledWith(
1,
'/home/expo/.maestro/tests/android-maestro-html.html',
{
force: true,
}
);
expect(removeSpy).toHaveBeenNthCalledWith(
2,
'/home/expo/.maestro/tests/android-maestro-html.html',
{
force: true,
}
);
expect(removeSpy.mock.invocationCallOrder[0]).toBeLessThan(
mockedSpawn.mock.invocationCallOrder[0]
);
expect(removeSpy.mock.invocationCallOrder[1]).toBeLessThan(
mockedSpawn.mock.invocationCallOrder[1]
);
});
it('does not run Maestro CLI or upload old HTML if the report cannot be cleared', async () => {
jest.spyOn(fs, 'rm').mockImplementation(async target => {
if (target === '/home/expo/.maestro/tests/android-maestro-html.html') {
throw new Error('permission denied');
}
});
const step = createStep({
flow_path: ['flows/a.yaml'],
output_format: 'html',
platform: 'android',
});
await expect(step.executeAsync()).rejects.toThrow(SystemError);
expect(mockedSpawn).not.toHaveBeenCalled();
expect(mockUploadArtifact).not.toHaveBeenCalled();
});
it('uses lowercase extension for non-junit formats regardless of input casing', async () => {
@@ -1215,28 +1428,31 @@ describe('createMaestroTestsBuildFunction', () => {
);
});
it('discovers maestro-runner flow results from the attempt report directory', async () => {
mockedSpawn.mockResolvedValue(SPAWN_SUCCESS);
const shot = makeShot(0);
jest.spyOn(fs, 'copyFile').mockResolvedValue();
mockedRunnerHarvest.mockResolvedValue([shot]);
const parseFlowResultsSpy = jest.spyOn(parser, 'parseMaestroRunnerReport').mockResolvedValue({
flows: [{ name: 'Login', sourceFile: 'a.yaml', status: 'failed' }],
});
it.each(['junit', undefined])(
'uploads runner screenshots with output_format=%s',
async format => {
mockedSpawn.mockResolvedValue(SPAWN_SUCCESS);
const shot = makeShot(0);
jest.spyOn(fs, 'copyFile').mockResolvedValue();
mockedRunnerHarvest.mockResolvedValue([shot]);
const parseFlowResultsSpy = jest.spyOn(parser, 'parseMaestroRunnerReport').mockResolvedValue({
flows: [{ name: 'Login', sourceFile: 'a.yaml', status: 'failed' }],
});
const step = createStep({
flow_path: ['a.yaml'],
platform: 'ios',
output_format: 'junit',
backend: 'maestro-runner',
});
await step.executeAsync();
const step = createStep({
flow_path: ['a.yaml'],
platform: 'ios',
...(format === undefined ? {} : { output_format: format }),
backend: 'maestro-runner',
});
await step.executeAsync();
expect(parseFlowResultsSpy).toHaveBeenCalledWith(
'/home/expo/.maestro/tests/ios-maestro-runner-attempt-0'
);
expect(mockUploadArtifact).toHaveBeenCalledTimes(1);
});
expect(parseFlowResultsSpy).toHaveBeenCalledWith(
'/home/expo/.maestro/tests/ios-maestro-runner-attempt-0'
);
expect(mockUploadArtifact).toHaveBeenCalledTimes(1);
}
);
it('uploads screenshots even when all attempts fail, before throwing ERR_MAESTRO_TESTS_FAILED', async () => {
mockedSpawn.mockRejectedValue(rejectExit1());
@@ -1300,14 +1516,15 @@ describe('createMaestroTestsBuildFunction', () => {
expect(mockUploadArtifact).toHaveBeenCalledTimes(30);
});
it('does not harvest or upload screenshots when output_format is not junit', async () => {
it('does not harvest or upload screenshots for Maestro CLI HTML', async () => {
mockedSpawn.mockResolvedValue(SPAWN_SUCCESS);
const step = createStep({ flow_path: ['a.yaml'], platform: 'android', output_format: 'html' });
await step.executeAsync();
expect(mockedHarvest).not.toHaveBeenCalled();
expect(mockUploadArtifact).not.toHaveBeenCalled();
expect(mockUploadArtifact).toHaveBeenCalledTimes(1);
expect(mockUploadArtifact.mock.calls[0][0].artifact.name).toBe('Maestro HTML Report');
});
it('uploads only the final attempt for a pure-failure flow, but every attempt for a flaky flow', async () => {
@@ -7,6 +7,7 @@ import { type CustomBuildContext } from '../../../customBuildContext';
import { IosSimulatorUtils } from '../../../utils/IosSimulatorUtils';
import { isProcessDescendantOfAsync } from '../../../utils/processes';
import { turtleFetch } from '../../../utils/turtleFetch';
import { startDeviceSessionHostAsync } from '../../utils/deviceSessionHost';
import {
monitorLocalEgressAsync,
readLocalEgressHandoffAsync,
@@ -14,7 +15,6 @@ import {
stopLocalEgressResourcesAsync,
} from '../../utils/localEgress';
import {
startDeviceWebPreviewWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../../utils/remoteDeviceRunSession';
@@ -27,6 +27,7 @@ jest.mock('@expo/turtle-spawn');
jest.mock('../../../utils/IosSimulatorUtils');
jest.mock('../../../utils/processes');
jest.mock('../../../utils/turtleFetch');
jest.mock('../../utils/deviceSessionHost');
jest.mock('../../utils/agentDeviceArtifacts');
jest.mock('../../utils/argentArtifacts');
jest.mock('../../utils/agentDeviceEvents', () => ({
@@ -45,6 +46,7 @@ jest.mock('../../utils/localEgress', () => ({
monitorLocalEgressAsync: jest.fn(),
}));
jest.mock('../../utils/remoteDeviceRunSession', () => ({
...jest.requireActual('../../utils/remoteDeviceRunSession'),
getDeviceRunSessionIdOrThrow: () => 'session-id',
getNgrokTunnelDomainOrThrow: () => 'example.test',
getNgrokAuthtokenOrThrow: () => 'ngrok-token',
@@ -52,13 +54,20 @@ jest.mock('../../utils/remoteDeviceRunSession', () => ({
ensureFfmpegInstalledOnceAsync: jest.fn(),
spawnDetached: () => ({ pid: 123, getOutput: () => '', stopAsync: jest.fn() }),
waitForFileAsync: async () => ({ port: 1234, token: 'controller-token' }),
startNgrokTunnelAsync: async () => ({ url: 'https://controller.test', stopAsync: jest.fn() }),
startDeviceWebPreviewWithTunnelAsync: jest.fn(),
startNgrokTunnelAsync: async () => ({
url: 'https://controller.test',
subdomainId: 'controller-id',
stopAsync: jest.fn(),
}),
uploadRemoteSessionConfigAsync: jest.fn(),
waitForDeviceRunSessionStoppedAsync: jest.fn(),
}));
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const logger = {
info: jest.fn(),
warn: jest.fn(),
child: jest.fn().mockReturnThis(),
} as unknown as bunyan;
const ctx = {} as CustomBuildContext;
const env = { DEVICE_RUN_SESSION_ID: 'session-id' };
const handoff = { url: 'https://egress.test', token: 'secret', fingerprint: 'key=', port: 8899 };
@@ -77,6 +86,7 @@ const controllers = [
describe.each(controllers)('%s local egress', (_name, createFunction, controllerField) => {
const stopPreview = jest.fn();
const openPreview = jest.fn();
const stopEgress = jest.fn();
let lifetime: AbortSignal;
async function run(signal?: AbortSignal): Promise<void> {
@@ -114,10 +124,15 @@ describe.each(controllers)('%s local egress', (_name, createFunction, controller
jest.mocked(turtleFetch).mockResolvedValue({ ok: true } as never);
jest.mocked(readLocalEgressHandoffAsync).mockResolvedValue(handoff);
jest.mocked(monitorLocalEgressAsync).mockResolvedValue(undefined);
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
previewUrl: 'https://preview.test',
openPreview.mockResolvedValue({
previewPageUrl: 'https://expo.dev/simulator-preview/preview-id',
apiUrl: 'https://preview.test',
previewToken: 'preview-secret',
stopAsync: stopPreview,
closeAsync: jest.fn(),
});
jest.mocked(startDeviceSessionHostAsync).mockResolvedValue({
openPreviewAsync: openPreview,
finishAsync: stopPreview,
});
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockImplementation(async () => {
@@ -163,12 +178,14 @@ describe.each(controllers)('%s local egress', (_name, createFunction, controller
expect(monitorLocalEgressAsync).not.toHaveBeenCalled();
expect(stopEgress).not.toHaveBeenCalled();
});
it.each(['startup', 'report', 'wait', 'teardown'])(
it.each(['startup', 'preview', 'report', 'wait', 'teardown'])(
'releases egress after %s fails',
async phase => {
const error = new Error(`${phase} failed`);
if (phase === 'startup') {
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockRejectedValue(error);
jest.mocked(startDeviceSessionHostAsync).mockRejectedValue(error);
} else if (phase === 'preview') {
openPreview.mockRejectedValue(error);
} else if (phase === 'report') {
jest.mocked(uploadRemoteSessionConfigAsync).mockRejectedValue(error);
} else if (phase === 'wait') {
@@ -180,7 +197,7 @@ describe.each(controllers)('%s local egress', (_name, createFunction, controller
expect(lifetime.aborted).toBe(true);
expect(stopEgress).toHaveBeenCalledTimes(1);
expect(monitorLocalEgressAsync).toHaveBeenCalledTimes(
phase === 'startup' || phase === 'report' ? 0 : 1
phase === 'startup' || phase === 'preview' || phase === 'report' ? 0 : 1
);
}
);
@@ -0,0 +1,70 @@
import { type bunyan } from '@expo/logger';
import { type BuildStepContext, type BuildStepEnv } from '@expo/steps';
import { type CustomBuildContext } from '../../../customBuildContext';
import { Sentry } from '../../../sentry';
import { getDeviceRunSessionIdOrThrow } from '../../utils/remoteDeviceRunSession';
import { uploadServeSimLogsFileAsync } from '../../utils/serveSimLogsArtifacts';
import { ServeSimLogsRecorder } from '../../utils/serveSimLogsRecorder';
import { createCollectServeSimLogsBuildFunction } from '../collectServeSimLogs';
import { createStartServeSimLogsBuildFunction } from '../startServeSimLogs';
jest.mock('../../../sentry');
jest.mock('../../utils/serveSimLogsRecorder');
jest.mock('../../utils/serveSimLogsArtifacts');
jest.mock('../../utils/remoteDeviceRunSession');
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const ctx = {} as CustomBuildContext;
const step = { logger } as BuildStepContext;
const args = { inputs: {}, outputs: {}, env: {} as BuildStepEnv };
beforeEach(() => {
jest.mocked(getDeviceRunSessionIdOrThrow).mockReturnValue('session-id');
jest.mocked(ServeSimLogsRecorder.finishAsync).mockResolvedValue([]);
});
it('starts collection and tolerates startup failures', async () => {
jest.mocked(ServeSimLogsRecorder.startAsync).mockRejectedValueOnce(new Error('disk failure'));
await expect(createStartServeSimLogsBuildFunction().fn?.(step, args)).resolves.toBeUndefined();
expect(logger.warn).toHaveBeenCalled();
expect(Sentry.capture).toHaveBeenCalledWith(
'Could not start serve-sim simulator logs',
expect.any(Error)
);
});
it('uploads each device with its own bounded signal', async () => {
jest.mocked(ServeSimLogsRecorder.finishAsync).mockResolvedValue([
{ udid: 'A', filePath: '/tmp/A.ndjson' },
{ udid: 'B', filePath: '/tmp/B.ndjson' },
]);
await createCollectServeSimLogsBuildFunction(ctx).fn?.(step, args);
expect(uploadServeSimLogsFileAsync).toHaveBeenCalledTimes(2);
expect(uploadServeSimLogsFileAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({
deviceRunSessionId: 'session-id',
udid: 'A',
filePath: '/tmp/A.ndjson',
signal: expect.any(AbortSignal),
})
);
expect(jest.mocked(uploadServeSimLogsFileAsync).mock.calls[0][1].signal).not.toBe(
jest.mocked(uploadServeSimLogsFileAsync).mock.calls[1][1].signal
);
});
it('skips empty collection and warns on finalization failure', async () => {
await createCollectServeSimLogsBuildFunction(ctx).fn?.(step, args);
expect(uploadServeSimLogsFileAsync).not.toHaveBeenCalled();
jest.mocked(ServeSimLogsRecorder.finishAsync).mockRejectedValueOnce(new Error('failure'));
await expect(
createCollectServeSimLogsBuildFunction(ctx).fn?.(step, args)
).resolves.toBeUndefined();
expect(logger.warn).toHaveBeenCalled();
expect(Sentry.capture).toHaveBeenCalledWith(
'Could not finalize serve-sim simulator logs',
expect.any(Error)
);
});
@@ -0,0 +1,427 @@
import { BuildRuntimePlatform, type BuildStepContext } from '@expo/steps';
import spawn from '@expo/turtle-spawn';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { type CustomBuildContext } from '../../../customBuildContext';
import { pollAgentDeviceArtifactsForUploadAsync } from '../../utils/agentDeviceArtifacts';
import { startAgentDeviceEventCollectionAsync } from '../../utils/agentDeviceEvents';
import { startDeviceSessionHostAsync } from '../../utils/deviceSessionHost';
import {
getDeviceRunSessionIdOrThrow,
getNgrokAuthtokenOrThrow,
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startNgrokTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
waitForFileAsync,
} from '../../utils/remoteDeviceRunSession';
import { type StartupTasks, createStartupTasks } from '../../utils/startupTasks';
import {
createStartAgentDeviceRemoteSessionBuildFunction,
runAgentDeviceRemoteSessionAsync,
} from '../startAgentDeviceRemoteSession';
// The daemon entry path and the state directory are resolved from the home directory when
// the module loads, so point it at a temp home we can populate.
jest.mock('node:os', () => {
const actual = jest.requireActual('node:os');
const actualPath = jest.requireActual('node:path');
return {
...actual,
homedir: () => actualPath.join(actual.tmpdir(), 'eas-agent-device-orchestration-home'),
};
});
jest.mock('@expo/turtle-spawn', () => ({ __esModule: true, default: jest.fn() }));
jest.mock('../../../sentry');
jest.mock('../../utils/agentDeviceArtifacts', () => ({
pollAgentDeviceArtifactsForUploadAsync: jest.fn(),
}));
jest.mock('../../utils/agentDeviceEvents', () => ({
startAgentDeviceEventCollectionAsync: jest.fn(),
}));
jest.mock('../../utils/deviceSessionHost');
jest.mock('../../utils/remoteDeviceRunSession', () => ({
...jest.requireActual('../../utils/remoteDeviceRunSession'),
getDeviceRunSessionIdOrThrow: jest.fn(),
getNgrokAuthtokenOrThrow: jest.fn(),
getNgrokTunnelDomainOrThrow: jest.fn(),
selectXcodeDeveloperDirectoryAsync: jest.fn(),
spawnDetached: jest.fn(),
startNgrokTunnelAsync: jest.fn(),
uploadRemoteSessionConfigAsync: jest.fn(),
waitForDeviceRunSessionStoppedAsync: jest.fn(),
waitForFileAsync: jest.fn(),
}));
const TEST_HOME = path.join(os.tmpdir(), 'eas-agent-device-orchestration-home');
const DAEMON_ENTRY_PATH = path.join(
TEST_HOME,
'.bun/install/global/node_modules/agent-device/dist/src/internal/daemon.js'
);
const ctx = {} as unknown as CustomBuildContext;
const mockPreviewStopAsync = jest.fn();
const mockTunnelStopAsync = jest.fn();
const mockDaemonStopAsync = jest.fn();
const mockEventCollectionStopAsync = jest.fn();
async function runAsync(
logger: { info: jest.Mock; warn: jest.Mock },
runtimePlatform: BuildRuntimePlatform,
launchInputs: Record<string, { value: unknown }> = {}
): Promise<void> {
const buildFunction = createStartAgentDeviceRemoteSessionBuildFunction(ctx);
await buildFunction.fn!(
{ logger, global: { runtimePlatform } } as unknown as BuildStepContext,
{
inputs: {
package_version: { value: undefined },
max_idle_time_minutes: { value: undefined },
max_duration_seconds: { value: undefined },
...launchInputs,
},
outputs: {},
env: {},
} as never
);
}
describe('createStartAgentDeviceRemoteSessionBuildFunction orchestration', () => {
beforeEach(async () => {
jest.clearAllMocks();
jest.mocked(spawn).mockResolvedValue(undefined as never);
jest.mocked(pollAgentDeviceArtifactsForUploadAsync).mockResolvedValue(undefined);
jest.mocked(startAgentDeviceEventCollectionAsync).mockResolvedValue({
stopAsync: mockEventCollectionStopAsync,
getLastEventObservedAt: () => undefined,
});
jest.mocked(getDeviceRunSessionIdOrThrow).mockReturnValue('device-run-session-id');
jest.mocked(getNgrokTunnelDomainOrThrow).mockReturnValue('tunnel.example.com');
jest.mocked(getNgrokAuthtokenOrThrow).mockReturnValue('ngrok-token');
jest.mocked(selectXcodeDeveloperDirectoryAsync).mockResolvedValue(undefined);
jest.mocked(spawnDetached).mockReturnValue({
pid: 4242,
getOutput: () => '',
getExitError: () => undefined,
stopAsync: mockDaemonStopAsync,
});
jest.mocked(waitForFileAsync).mockResolvedValue({ port: 5678, token: 'daemon-token' });
jest.mocked(startNgrokTunnelAsync).mockResolvedValue({
url: 'https://agent-device-abc.tunnel.example.com',
subdomainId: 'agent-device-abc',
stopAsync: mockTunnelStopAsync,
});
jest.mocked(startDeviceSessionHostAsync).mockResolvedValue({
openPreviewAsync: jest.fn().mockResolvedValue({
previewPageUrl: 'https://expo.dev/simulator-preview/preview-id',
apiUrl: 'https://web-preview.tunnel.example.com',
closeAsync: jest.fn(),
}),
finishAsync: mockPreviewStopAsync,
});
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockResolvedValue(undefined);
await fs.promises.mkdir(path.dirname(DAEMON_ENTRY_PATH), { recursive: true });
await fs.promises.writeFile(DAEMON_ENTRY_PATH, '');
});
afterEach(async () => {
await fs.promises.rm(TEST_HOME, { recursive: true, force: true });
});
it('reports the preview URL and tears every resource down', async () => {
const logger = { info: jest.fn(), warn: jest.fn(), child: jest.fn().mockReturnThis() };
await runAsync(logger, BuildRuntimePlatform.LINUX);
expect(selectXcodeDeveloperDirectoryAsync).not.toHaveBeenCalled();
expect(startDeviceSessionHostAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({ runtimePlatform: BuildRuntimePlatform.LINUX })
);
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledWith(
expect.objectContaining({
remoteConfig: expect.objectContaining({
agentDeviceRemoteSessionUrl: 'https://agent-device-abc.tunnel.example.com',
agentDeviceRemoteSessionToken: 'daemon-token',
webPreviewUrl: 'https://expo.dev/simulator-preview/preview-id',
previewApiUrl: 'https://web-preview.tunnel.example.com',
}),
})
);
expect(mockPreviewStopAsync).toHaveBeenCalledTimes(1);
expect(mockTunnelStopAsync).toHaveBeenCalledTimes(1);
expect(mockEventCollectionStopAsync).toHaveBeenCalledTimes(1);
expect(mockDaemonStopAsync).toHaveBeenCalledTimes(1);
});
it('hands the launch inputs to serve-sim and announces them on an iOS session', async () => {
const logger = { info: jest.fn(), warn: jest.fn(), child: jest.fn().mockReturnThis() };
await runAsync(logger, BuildRuntimePlatform.DARWIN, {
launch_app_identifier: { value: 'host.exp.Exponent' },
launch_args: { value: ['-EXDevMenuIsOnboardingFinished', '1'] },
open_url: { value: 'exp://127.0.0.1:8081' },
});
expect(startDeviceSessionHostAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({
runtimePlatform: BuildRuntimePlatform.DARWIN,
launchAppIdentifier: 'host.exp.Exponent',
launchArgs: ['-EXDevMenuIsOnboardingFinished', '1'],
openUrl: 'exp://127.0.0.1:8081',
})
);
expect(logger.info).toHaveBeenCalledWith(
'serve-sim will launch host.exp.Exponent with arguments ' +
'["-EXDevMenuIsOnboardingFinished","1"], then open exp://127.0.0.1:8081.'
);
});
it('hands network capture to serve-sim on an iOS session', async () => {
const logger = { info: jest.fn(), warn: jest.fn(), child: jest.fn().mockReturnThis() };
await runAsync(logger, BuildRuntimePlatform.DARWIN, {
network_capture: { value: true },
network_capture_fields: { value: ['header'] },
});
expect(startDeviceSessionHostAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({ networkCapture: true, networkCaptureFields: ['header'] })
);
});
it('fails before starting the daemon when network capture is asked for on Android', async () => {
const logger = { info: jest.fn(), warn: jest.fn(), child: jest.fn().mockReturnThis() };
await expect(
runAsync(logger, BuildRuntimePlatform.LINUX, { network_capture: { value: true } })
).rejects.toThrow('records traffic through serve-sim on an iOS simulator');
expect(spawnDetached).not.toHaveBeenCalled();
});
it('fails before starting the daemon when a launch is asked for on Android', async () => {
const logger = { info: jest.fn(), warn: jest.fn(), child: jest.fn().mockReturnThis() };
await expect(
runAsync(logger, BuildRuntimePlatform.LINUX, {
launch_app_identifier: { value: 'host.exp.Exponent' },
})
).rejects.toThrow('runs on linux');
expect(spawnDetached).not.toHaveBeenCalled();
});
it('starts the session host without waiting for the agent-device daemon', async () => {
const logger = { info: jest.fn(), warn: jest.fn(), child: jest.fn().mockReturnThis() };
const sessionHost = await jest.mocked(startDeviceSessionHostAsync).getMockImplementation()!(
ctx,
{} as never
);
let markHostStarted!: () => void;
const hostStarted = new Promise<void>(resolve => {
markHostStarted = resolve;
});
jest.mocked(startDeviceSessionHostAsync).mockImplementation(async () => {
markHostStarted();
return sessionHost;
});
// The daemon credentials only appear after the session host started. A sequential
// startup would wait here forever.
jest.mocked(waitForFileAsync).mockImplementation(async () => {
await hostStarted;
return { port: 5678, token: 'daemon-token' };
});
await runAsync(logger, BuildRuntimePlatform.DARWIN);
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledWith(
expect.objectContaining({
remoteConfig: expect.objectContaining({
agentDeviceRemoteSessionUrl: 'https://agent-device-abc.tunnel.example.com',
agentDeviceRemoteSessionToken: 'daemon-token',
webPreviewUrl: 'https://expo.dev/simulator-preview/preview-id',
}),
})
);
});
it('stops the daemon and its tunnel when the session host fails to start', async () => {
const logger = { info: jest.fn(), warn: jest.fn(), child: jest.fn().mockReturnThis() };
jest
.mocked(startDeviceSessionHostAsync)
.mockRejectedValue(new Error('serve-sim did not start'));
await expect(runAsync(logger, BuildRuntimePlatform.DARWIN)).rejects.toThrow(
'serve-sim did not start'
);
expect(uploadRemoteSessionConfigAsync).not.toHaveBeenCalled();
// The failure aborts the daemon task, which then may not start its daemon or tunnel.
// Whatever started is stopped.
expect(mockDaemonStopAsync).toHaveBeenCalledTimes(jest.mocked(spawnDetached).mock.calls.length);
expect(mockTunnelStopAsync).toHaveBeenCalledTimes(
jest.mocked(startNgrokTunnelAsync).mock.calls.length
);
});
it('stops the session host and the daemon when the daemon credentials never appear', async () => {
const logger = { info: jest.fn(), warn: jest.fn(), child: jest.fn().mockReturnThis() };
jest.mocked(waitForFileAsync).mockRejectedValue(new Error('no daemon credentials'));
await expect(runAsync(logger, BuildRuntimePlatform.DARWIN)).rejects.toThrow(
'no daemon credentials'
);
expect(uploadRemoteSessionConfigAsync).not.toHaveBeenCalled();
expect(startNgrokTunnelAsync).not.toHaveBeenCalled();
expect(mockPreviewStopAsync).toHaveBeenCalledTimes(1);
expect(mockDaemonStopAsync).toHaveBeenCalledTimes(1);
});
describe('runAgentDeviceRemoteSessionAsync with a device that is still starting', () => {
function deferred(): {
promise: Promise<void>;
resolve: () => void;
reject: (e: Error) => void;
} {
let resolve!: () => void;
let reject!: (e: Error) => void;
const promise = new Promise<void>((res, rej) => {
resolve = res;
reject = rej;
});
promise.catch(() => {});
return { promise, resolve, reject };
}
type Device = { booted: Promise<unknown>; ready: Promise<unknown> };
function startSession(device: Device | ((tasks: StartupTasks) => Device)) {
const logger = {
info: jest.fn(),
warn: jest.fn(),
child: jest.fn().mockReturnThis(),
} as never;
const tasks = createStartupTasks(logger);
return runAgentDeviceRemoteSessionAsync(ctx, {
env: {},
logger,
runtimePlatform: BuildRuntimePlatform.DARWIN,
sessionEnv: {
deviceRunSessionId: 'device-run-session-id',
ngrokTunnelDomain: 'tunnel.example.com',
ngrokAuthtoken: 'ngrok-token',
},
packageVersion: undefined,
maxIdleTimeMinutes: undefined,
maxDurationSeconds: undefined,
launch: {},
capture: { networkCapture: false, networkCaptureFields: [] },
tasks,
device: typeof device === 'function' ? device(tasks) : device,
});
}
async function flushAsync(): Promise<void> {
for (let i = 0; i < 20; i++) {
await new Promise(resolve => setImmediate(resolve));
}
}
it('starts the daemon during the boot and the session host after it', async () => {
const booted = deferred();
const session = startSession({ booted: booted.promise, ready: booted.promise });
await flushAsync();
expect(spawnDetached).toHaveBeenCalledTimes(1);
expect(startNgrokTunnelAsync).toHaveBeenCalledTimes(1);
expect(startDeviceSessionHostAsync).not.toHaveBeenCalled();
booted.resolve();
await session;
expect(startDeviceSessionHostAsync).toHaveBeenCalledTimes(1);
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledTimes(1);
});
it('reports the session as ready only after the app is launched', async () => {
const ready = deferred();
const session = startSession({ booted: Promise.resolve(), ready: ready.promise });
await flushAsync();
expect(startDeviceSessionHostAsync).toHaveBeenCalledTimes(1);
expect(uploadRemoteSessionConfigAsync).not.toHaveBeenCalled();
ready.resolve();
await session;
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledTimes(1);
});
it('stops waiting for the boot when the daemon fails', async () => {
jest.mocked(waitForFileAsync).mockRejectedValue(new Error('no daemon credentials'));
const neverBooted = new Promise<void>(() => {});
// In the combined step, `ready` stops waiting for the boot on an abort, like this.
await expect(
startSession(tasks => ({ booted: neverBooted, ready: tasks.untilAborted(neverBooted) }))
).rejects.toThrow('no daemon credentials');
expect(startDeviceSessionHostAsync).not.toHaveBeenCalled();
expect(mockDaemonStopAsync).toHaveBeenCalledTimes(1);
expect(uploadRemoteSessionConfigAsync).not.toHaveBeenCalled();
});
it('stops the session host when the app fails while the daemon install hangs', async () => {
jest.mocked(spawn).mockImplementation(((
command: string,
args: string[],
options?: { signal?: AbortSignal }
) =>
command === 'bun' && args[0] === 'add'
? new Promise((_resolve, reject) => {
options!.signal!.addEventListener('abort', () => reject(options!.signal!.reason));
})
: Promise.resolve(undefined)) as never);
const ready = deferred();
const session = startSession({ booted: Promise.resolve(), ready: ready.promise });
await flushAsync();
expect(startDeviceSessionHostAsync).toHaveBeenCalledTimes(1);
ready.reject(new Error('simctl install failed'));
await expect(session).rejects.toThrow('simctl install failed');
expect(mockPreviewStopAsync).toHaveBeenCalledTimes(1);
expect(spawnDetached).not.toHaveBeenCalled();
expect(uploadRemoteSessionConfigAsync).not.toHaveBeenCalled();
});
it('stops everything it started when the app install fails', async () => {
const ready = deferred();
const session = startSession({ booted: Promise.resolve(), ready: ready.promise });
await flushAsync();
ready.reject(new Error('simctl install failed'));
await expect(session).rejects.toThrow('simctl install failed');
expect(uploadRemoteSessionConfigAsync).not.toHaveBeenCalled();
expect(mockDaemonStopAsync).toHaveBeenCalledTimes(1);
expect(mockTunnelStopAsync).toHaveBeenCalledTimes(1);
expect(mockPreviewStopAsync).toHaveBeenCalledTimes(1);
});
});
it('declares the launch inputs', () => {
const buildFunction = createStartAgentDeviceRemoteSessionBuildFunction(ctx);
const globalCtx = createGlobalContextMock();
expect(
buildFunction.inputProviders?.map(provider => provider(globalCtx, 'Test step').id)
).toEqual(expect.arrayContaining(['launch_app_identifier', 'launch_args', 'open_url']));
});
});
@@ -1,9 +1,38 @@
import { type bunyan } from '@expo/logger';
import spawn from '@expo/turtle-spawn';
import fs from 'node:fs';
import path from 'node:path';
import { Sentry } from '../../../sentry';
import { stopAgentDeviceEventCollectionSafelyAsync } from '../startAgentDeviceRemoteSession';
import { spawnDetached } from '../../utils/remoteDeviceRunSession';
import {
startAgentDeviceDaemonAsync,
stopAgentDeviceEventCollectionSafelyAsync,
} from '../startAgentDeviceRemoteSession';
jest.mock('../../../sentry');
jest.mock('@expo/turtle-spawn', () => ({ __esModule: true, default: jest.fn() }));
jest.mock('../../utils/remoteDeviceRunSession', () => ({
...jest.requireActual('../../utils/remoteDeviceRunSession'),
spawnDetached: jest.fn(),
}));
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
async function writeDaemonEntry(cwd: string): Promise<string> {
const daemonPath = path.join(
cwd,
'node_modules',
'agent-device',
'dist',
'src',
'internal',
'daemon.js'
);
await fs.promises.mkdir(path.dirname(daemonPath), { recursive: true });
await fs.promises.writeFile(daemonPath, '');
return daemonPath;
}
describe(stopAgentDeviceEventCollectionSafelyAsync, () => {
beforeEach(() => {
@@ -12,7 +41,6 @@ describe(stopAgentDeviceEventCollectionSafelyAsync, () => {
it('reports an unexpected stop failure without rejecting', async () => {
const error = new Error('stop failed');
const logger = { warn: jest.fn() } as unknown as bunyan;
await expect(
stopAgentDeviceEventCollectionSafelyAsync({
@@ -37,3 +65,202 @@ describe(stopAgentDeviceEventCollectionSafelyAsync, () => {
);
});
});
describe(startAgentDeviceDaemonAsync, () => {
const stopAsync = jest.fn().mockResolvedValue(undefined);
beforeEach(() => {
jest.clearAllMocks();
stopAsync.mockResolvedValue(undefined);
jest.mocked(spawnDetached).mockReturnValue({
pid: 1234,
getOutput: () => '',
getExitError: () => undefined,
stopAsync,
});
jest.mocked(spawn).mockImplementation((async (
command: string,
args: string[],
options?: { cwd?: string }
) => {
if (command === 'bun' && args[0] === '--version') {
return { stdout: '1.3.11\n' };
}
if (typeof options?.cwd === 'string' && command !== 'git') {
await writeDaemonEntry(options.cwd);
}
return { stdout: '' };
}) as never);
});
it('installs with bun add by default and launches the published daemon', async () => {
const handle = await startAgentDeviceDaemonAsync({
packageVersion: '1.2.3',
env: {},
logger,
});
expect(spawn).toHaveBeenCalledWith(
'bun',
['add', '[email protected]'],
expect.objectContaining({ cwd: expect.stringContaining('eas-agent-device-') })
);
const addCwd = jest.mocked(spawn).mock.calls[0][2]?.cwd as string;
expect(spawnDetached).toHaveBeenCalledWith({
command: 'node',
args: [path.join(addCwd, 'node_modules/agent-device/dist/src/internal/daemon.js')],
env: expect.objectContaining({
AGENT_DEVICE_DAEMON_SERVER_MODE: 'http',
AGENT_DEVICE_DAEMON_IDLE_TIMEOUT_MS: '0',
AGENT_DEVICE_IOS_RUNNER_IDLE_STOP_MS: '0',
AGENT_DEVICE_SESSION_IDLE_TIMEOUT_MS: '0',
}),
});
await handle.stopAsync();
await expect(fs.promises.access(addCwd)).rejects.toThrow();
});
it('installs with npm when EAS_OVERRIDE_PACKAGE_MANAGER is npm', async () => {
await startAgentDeviceDaemonAsync({
packageVersion: undefined,
env: { EAS_OVERRIDE_PACKAGE_MANAGER: 'npm' },
logger,
});
expect(spawn).toHaveBeenCalledWith(
'npm',
['install', '--no-audit', 'agent-device@latest'],
expect.objectContaining({ cwd: expect.stringContaining('eas-agent-device-') })
);
});
it('kills the install and does not fall back to git when aborted', async () => {
jest.mocked(spawn).mockImplementation(
((_command: string, _args: string[], options?: { signal?: AbortSignal }) =>
new Promise((_resolve, reject) => {
options!.signal!.addEventListener('abort', () => reject(options!.signal!.reason));
})) as never
);
const controller = new AbortController();
const failure = new Error('app install failed');
const daemon = startAgentDeviceDaemonAsync({
packageVersion: '1.2.3',
env: {},
logger,
signal: controller.signal,
});
await new Promise(resolve => setImmediate(resolve));
controller.abort(failure);
await expect(daemon).rejects.toBe(failure);
expect(spawn).toHaveBeenCalledTimes(1);
expect(spawn).toHaveBeenCalledWith(
'bun',
['add', '[email protected]'],
expect.objectContaining({ signal: controller.signal })
);
expect(spawnDetached).not.toHaveBeenCalled();
expect(Sentry.capture).not.toHaveBeenCalled();
});
it('falls back to git clone when the published daemon is missing', async () => {
jest.mocked(spawn).mockImplementation((async (command: string, args: string[]) => {
if (command === 'bun' && args[0] === '--version') {
return { stdout: '1.3.11\n' };
}
return { stdout: '' };
}) as never);
await startAgentDeviceDaemonAsync({
packageVersion: '1.2.3',
env: {},
logger,
});
expect(spawn).toHaveBeenCalledWith(
'git',
[
'clone',
'--depth',
'1',
'--branch',
'v1.2.3',
'https://github.com/callstack/agent-device.git',
'/tmp/agent-device-src',
],
expect.objectContaining({ env: {}, logger })
);
expect(spawn).toHaveBeenCalledWith(
'bun',
['install', '--production'],
expect.objectContaining({ cwd: '/tmp/agent-device-src' })
);
expect(spawnDetached).toHaveBeenCalledWith({
command: 'bun',
args: ['run', 'src/daemon.ts'],
cwd: '/tmp/agent-device-src',
env: expect.objectContaining({
AGENT_DEVICE_DAEMON_SERVER_MODE: 'http',
AGENT_DEVICE_DAEMON_IDLE_TIMEOUT_MS: '0',
AGENT_DEVICE_IOS_RUNNER_IDLE_STOP_MS: '0',
AGENT_DEVICE_SESSION_IDLE_TIMEOUT_MS: '0',
}),
});
expect(Sentry.capture).toHaveBeenCalledWith(
'Failed to start agent-device daemon from the configured package manager; falling back to git clone',
expect.any(Error),
expect.objectContaining({
extras: expect.objectContaining({
packageSpec: '[email protected]',
packageManager: 'bun',
bunVersion: '1.3.11',
}),
})
);
});
it('clones latest and reports unknown bun when diagnostics fail', async () => {
jest.mocked(spawn).mockImplementation((async (command: string, args: string[]) => {
if (command === 'bun' && args[0] === '--version') {
throw new Error('bun missing');
}
return { stdout: '' };
}) as never);
await startAgentDeviceDaemonAsync({
packageVersion: undefined,
env: { EAS_OVERRIDE_PACKAGE_MANAGER: 'npm' },
logger,
});
expect(spawn).toHaveBeenCalledWith(
'git',
[
'clone',
'--depth',
'1',
'https://github.com/callstack/agent-device.git',
'/tmp/agent-device-src',
],
expect.objectContaining({ logger })
);
expect(spawn).toHaveBeenCalledWith(
'npm',
['install', '--no-audit', '--omit=dev'],
expect.objectContaining({ cwd: '/tmp/agent-device-src' })
);
expect(Sentry.capture).toHaveBeenCalledWith(
'Failed to start agent-device daemon from the configured package manager; falling back to git clone',
expect.any(Error),
expect.objectContaining({
extras: expect.objectContaining({
packageVersion: 'latest',
packageManager: 'npm',
bunVersion: 'unknown',
}),
})
);
});
});
@@ -0,0 +1,400 @@
import { BuildRuntimePlatform, type BuildStepContext } from '@expo/steps';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { type CustomBuildContext } from '../../../customBuildContext';
import { selectXcodeDeveloperDirectoryAsync } from '../../utils/remoteDeviceRunSession';
import { downloadBuildAsync } from '../downloadBuild';
import { installBuildAsync } from '../installBuild';
import { launchApplicationAsync } from '../launchApplication';
import {
getAgentDeviceRemoteSessionEnvOrThrow,
runAgentDeviceRemoteSessionAsync,
} from '../startAgentDeviceRemoteSession';
import { createStartAgentDeviceSessionBuildFunction } from '../startAgentDeviceSession';
import { startAndroidEmulatorAsync } from '../startAndroidEmulator';
import { bootIosSimulatorAsync } from '../startIosSimulator';
jest.mock('../../utils/localEgressSession', () => ({
withLocalEgressSession: (fn: unknown) => fn,
}));
jest.mock('../../utils/remoteDeviceRunSession', () => ({
...jest.requireActual('../../utils/remoteDeviceRunSession'),
selectXcodeDeveloperDirectoryAsync: jest.fn(),
}));
jest.mock('../startIosSimulator', () => ({ bootIosSimulatorAsync: jest.fn() }));
jest.mock('../startAndroidEmulator', () => ({ startAndroidEmulatorAsync: jest.fn() }));
jest.mock('../downloadBuild', () => ({ downloadBuildAsync: jest.fn() }));
jest.mock('../installBuild', () => ({ installBuildAsync: jest.fn() }));
jest.mock('../launchApplication', () => ({
...jest.requireActual('../launchApplication'),
launchApplicationAsync: jest.fn(),
}));
jest.mock('../startAgentDeviceRemoteSession', () => ({
getAgentDeviceRemoteSessionEnvOrThrow: jest.fn(),
runAgentDeviceRemoteSessionAsync: jest.fn(),
}));
const graphqlClient = { query: jest.fn() };
const ctx = { graphqlClient } as unknown as CustomBuildContext;
const sessionEnv = {
deviceRunSessionId: 'device-run-session-id',
ngrokTunnelDomain: 'tunnel.example.com',
ngrokAuthtoken: 'ngrok-token',
};
type Device = { booted: Promise<unknown>; ready: Promise<unknown> };
function deferred<T = void>(): {
promise: Promise<T>;
resolve: (value: T) => void;
reject: (err: Error) => void;
} {
let resolve!: (value: T) => void;
let reject!: (err: Error) => void;
const promise = new Promise<T>((res, rej) => {
resolve = res;
reject = rej;
});
promise.catch(() => {});
return { promise, resolve, reject };
}
async function flushAsync(): Promise<void> {
for (let i = 0; i < 20; i++) {
await new Promise(resolve => setImmediate(resolve));
}
}
function runStep(
runtimePlatform: BuildRuntimePlatform,
inputValues: Record<string, unknown> = {}
): Promise<void> {
const buildFunction = createStartAgentDeviceSessionBuildFunction(ctx);
const logger = { info: jest.fn(), warn: jest.fn(), child: jest.fn().mockReturnThis() };
const inputs = Object.fromEntries(
[
'device_identifier',
'system_image_package',
'lcd_width',
'lcd_height',
'lcd_density',
'build_id',
'application_archive_url',
'launch_args',
'open_url',
'network_capture',
'network_capture_fields',
'package_version',
'max_idle_time_minutes',
'max_duration_seconds',
].map(id => [id, { value: inputValues[id] }])
);
return buildFunction.fn!(
{
logger,
global: {
runtimePlatform,
staticContext: { job: { secrets: { robotAccessToken: 'robot-token' } } },
},
} as unknown as BuildStepContext,
{ inputs, outputs: {}, env: {}, signal: undefined } as never
) as Promise<void>;
}
/** The `device` promises handed to the shared agent-device session code. */
function sessionDevice(): Device {
return jest.mocked(runAgentDeviceRemoteSessionAsync).mock.calls[0][1].device;
}
describe(createStartAgentDeviceSessionBuildFunction, () => {
beforeEach(() => {
jest.clearAllMocks();
jest.mocked(getAgentDeviceRemoteSessionEnvOrThrow).mockReturnValue(sessionEnv);
jest.mocked(runAgentDeviceRemoteSessionAsync).mockImplementation(async (_ctx, { device }) => {
await device.ready;
});
jest.mocked(bootIosSimulatorAsync).mockResolvedValue({
deviceIdentifier: 'iPhone 17' as never,
udid: 'udid' as never,
displayName: 'iPhone 17',
});
jest.mocked(startAndroidEmulatorAsync).mockResolvedValue({
serialId: 'emulator-5554' as never,
emulatorPromise: Promise.resolve(),
shouldAdjustAnimationScale: true,
});
jest.mocked(downloadBuildAsync).mockResolvedValue({ artifactPath: '/tmp/App.app' });
jest
.mocked(installBuildAsync)
.mockResolvedValue({ applicationIdentifier: 'dev.example.app', activityName: '.Main' });
jest.mocked(launchApplicationAsync).mockResolvedValue(undefined);
});
it('downloads the build and starts the session while the iOS Simulator boots', async () => {
const boot = deferred<Awaited<ReturnType<typeof bootIosSimulatorAsync>>>();
jest.mocked(bootIosSimulatorAsync).mockReturnValue(boot.promise);
const step = runStep(BuildRuntimePlatform.DARWIN, {
build_id: 'build-id',
launch_args: ['-flag'],
open_url: 'exp://example.test',
});
await flushAsync();
expect(
jest.mocked(selectXcodeDeveloperDirectoryAsync).mock.invocationCallOrder[0]
).toBeLessThan(jest.mocked(bootIosSimulatorAsync).mock.invocationCallOrder[0]);
expect(downloadBuildAsync).toHaveBeenCalledWith(
expect.objectContaining({
buildId: 'build-id',
extensions: ['app'],
graphqlClient,
robotAccessToken: 'robot-token',
})
);
expect(runAgentDeviceRemoteSessionAsync).toHaveBeenCalledTimes(1);
// The install needs the booted Simulator.
expect(installBuildAsync).not.toHaveBeenCalled();
boot.resolve({
deviceIdentifier: 'iPhone 17' as never,
udid: 'udid' as never,
displayName: '',
});
await step;
expect(installBuildAsync).toHaveBeenCalledWith(
expect.objectContaining({ artifactPath: '/tmp/App.app' })
);
expect(launchApplicationAsync).toHaveBeenCalledWith(
expect.objectContaining({
applicationIdentifier: 'dev.example.app',
launchArgs: ['-flag'],
openUrl: 'exp://example.test',
})
);
expect(jest.mocked(installBuildAsync).mock.invocationCallOrder[0]).toBeLessThan(
jest.mocked(launchApplicationAsync).mock.invocationCallOrder[0]
);
});
it('waits for the download when the Simulator boots first', async () => {
const download = deferred<{ artifactPath: string }>();
jest.mocked(downloadBuildAsync).mockReturnValue(download.promise);
const step = runStep(BuildRuntimePlatform.DARWIN, {
application_archive_url: 'https://example.test/app.tar.gz',
});
await flushAsync();
expect(installBuildAsync).not.toHaveBeenCalled();
download.resolve({ artifactPath: '/tmp/App.app' });
await step;
expect(installBuildAsync).toHaveBeenCalledTimes(1);
});
it('makes the session ready when the device boots if there is no app', async () => {
await runStep(BuildRuntimePlatform.DARWIN);
expect(downloadBuildAsync).not.toHaveBeenCalled();
expect(installBuildAsync).not.toHaveBeenCalled();
expect(launchApplicationAsync).not.toHaveBeenCalled();
await expect(sessionDevice().ready).resolves.toBeUndefined();
});
it('boots the Android Emulator with the device inputs', async () => {
await runStep(BuildRuntimePlatform.LINUX, {
device_identifier: 'pixel_7',
system_image_package: 'system-images;android-35-ext15;google_apis_playstore;x86_64',
lcd_width: 720,
lcd_height: 1600,
lcd_density: 300,
build_id: 'build-id',
});
expect(selectXcodeDeveloperDirectoryAsync).not.toHaveBeenCalled();
expect(startAndroidEmulatorAsync).toHaveBeenCalledWith(
expect.objectContaining({
deviceIdentifier: 'pixel_7',
systemImagePackage: 'system-images;android-35-ext15;google_apis_playstore;x86_64',
lcdWidth: 720,
lcdHeight: 1600,
lcdDensity: 300,
})
);
expect(downloadBuildAsync).toHaveBeenCalledWith(
expect.objectContaining({ extensions: ['apk'] })
);
expect(launchApplicationAsync).toHaveBeenCalledWith(
expect.objectContaining({ applicationIdentifier: 'dev.example.app', activityName: '.Main' })
);
});
it('fails the session when the download fails, without an unhandled rejection', async () => {
const boot = deferred<Awaited<ReturnType<typeof bootIosSimulatorAsync>>>();
jest.mocked(bootIosSimulatorAsync).mockReturnValue(boot.promise);
jest.mocked(downloadBuildAsync).mockRejectedValue(new Error('download failed'));
const stepFailure = expect(
runStep(BuildRuntimePlatform.DARWIN, { build_id: 'build-id' })
).rejects.toThrow('download failed');
// The download fails while the boot still runs. Nothing inside the step may leave
// that rejection unhandled; Jest fails the test if it does.
await flushAsync();
boot.resolve({
deviceIdentifier: 'iPhone 17' as never,
udid: 'udid' as never,
displayName: '',
});
await stepFailure;
expect(installBuildAsync).not.toHaveBeenCalled();
});
/** A download that runs until its abort signal fires, like a stalled one. */
function mockStalledDownload(): { aborted: () => boolean; settled: () => boolean } {
let aborted = false;
let settled = false;
jest.mocked(downloadBuildAsync).mockImplementation(
({ signal }) =>
new Promise((_resolve, reject) => {
signal!.addEventListener('abort', () => {
aborted = true;
// Settles a bit later, like a request that winds down.
setImmediate(() => {
settled = true;
reject(new Error('The user aborted a request.'));
});
});
})
);
return { aborted: () => aborted, settled: () => settled };
}
/** Runs the session like the real one: fails with the daemon, then waits for every part. */
function mockSessionWithFailingDaemon(beforeFailure?: () => Promise<void>): void {
jest
.mocked(runAgentDeviceRemoteSessionAsync)
.mockImplementation(async (_ctx, { tasks, device }) => {
await beforeFailure?.();
const daemon = tasks.run('agent-device daemon', async () => {
throw new Error('daemon failed');
});
try {
await Promise.all([daemon, device.ready]);
} finally {
await Promise.allSettled([daemon, device.ready]);
}
});
}
it('stops the download and waits for it when the boot fails', async () => {
const download = mockStalledDownload();
jest.mocked(bootIosSimulatorAsync).mockRejectedValue(new Error('boot failed'));
await expect(runStep(BuildRuntimePlatform.DARWIN, { build_id: 'build-id' })).rejects.toThrow(
'boot failed'
);
expect(download.aborted()).toBe(true);
// The step returned only after the download stopped.
expect(download.settled()).toBe(true);
expect(installBuildAsync).not.toHaveBeenCalled();
});
it('stops a stalled download and does not install when the daemon fails', async () => {
const download = mockStalledDownload();
mockSessionWithFailingDaemon();
await expect(runStep(BuildRuntimePlatform.DARWIN, { build_id: 'build-id' })).rejects.toThrow(
'daemon failed'
);
expect(download.aborted()).toBe(true);
expect(download.settled()).toBe(true);
expect(installBuildAsync).not.toHaveBeenCalled();
expect(launchApplicationAsync).not.toHaveBeenCalled();
});
it('does not launch the app when startup fails during the install', async () => {
const install = deferred<Awaited<ReturnType<typeof installBuildAsync>>>();
const installStarted = deferred();
jest.mocked(installBuildAsync).mockImplementation(() => {
installStarted.resolve();
return install.promise;
});
mockSessionWithFailingDaemon(async () => {
await installStarted.promise;
// The install finishes only after the daemon failed.
setImmediate(() => install.resolve({ applicationIdentifier: 'dev.example.app' }));
});
await expect(runStep(BuildRuntimePlatform.DARWIN, { build_id: 'build-id' })).rejects.toThrow(
'daemon failed'
);
expect(installBuildAsync).toHaveBeenCalledTimes(1);
expect(launchApplicationAsync).not.toHaveBeenCalled();
});
it('rejects conflicting application inputs before it boots anything', async () => {
await expect(
runStep(BuildRuntimePlatform.DARWIN, {
build_id: 'build-id',
application_archive_url: 'https://example.test/app.tar.gz',
})
).rejects.toThrow('Pass only one of build_id or application_archive_url.');
await expect(
runStep(BuildRuntimePlatform.DARWIN, { open_url: 'exp://example.test' })
).rejects.toThrow('launch_args and open_url need an application');
expect(bootIosSimulatorAsync).not.toHaveBeenCalled();
expect(runAgentDeviceRemoteSessionAsync).not.toHaveBeenCalled();
});
it('declares the step inputs', () => {
const buildFunction = createStartAgentDeviceSessionBuildFunction(ctx);
const globalCtx = createGlobalContextMock();
expect(
buildFunction.inputProviders?.map(provider => provider(globalCtx, 'Test step').id)
).toEqual([
'device_identifier',
'system_image_package',
'lcd_width',
'lcd_height',
'lcd_density',
'build_id',
'application_archive_url',
'launch_args',
'open_url',
'network_capture',
'network_capture_fields',
'package_version',
'max_idle_time_minutes',
'max_duration_seconds',
]);
});
it('passes network capture to the session', async () => {
await runStep(BuildRuntimePlatform.DARWIN, {
network_capture: true,
network_capture_fields: ['header', 'response-body'],
});
expect(runAgentDeviceRemoteSessionAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({
capture: { networkCapture: true, networkCaptureFields: ['header', 'response-body'] },
})
);
});
it('rejects network capture on Android before it boots anything', async () => {
await expect(
runStep(BuildRuntimePlatform.LINUX, { network_capture: true, system_image_package: 'x' })
).rejects.toThrow('records traffic through serve-sim on an iOS simulator');
expect(startAndroidEmulatorAsync).not.toHaveBeenCalled();
expect(runAgentDeviceRemoteSessionAsync).not.toHaveBeenCalled();
});
});
@@ -8,6 +8,7 @@ import {
AndroidVirtualDeviceName,
} from '../../../utils/AndroidEmulatorUtils';
import { retryAsync } from '../../../utils/retry';
import { resolveAndroidEmulatorLocalEgressAsync } from '../../utils/androidLocalEgress';
import { createStartAndroidEmulatorBuildFunction } from '../startAndroidEmulator';
jest.mock('@expo/turtle-spawn', () => ({
@@ -19,6 +20,10 @@ jest.mock('../../../utils/retry', () => ({
retryAsync: jest.fn(),
}));
jest.mock('../../utils/androidLocalEgress', () => ({
resolveAndroidEmulatorLocalEgressAsync: jest.fn(),
}));
jest.mock('../../../utils/AndroidEmulatorUtils', () => ({
AndroidEmulatorUtils: {
defaultSystemImagePackage: 'system-images;android-30;default;x86_64',
@@ -35,6 +40,7 @@ jest.mock('../../../utils/AndroidEmulatorUtils', () => ({
const mockedSpawn = jest.mocked(spawn);
const mockedRetryAsync = jest.mocked(retryAsync);
const mockedAndroidUtils = jest.mocked(AndroidEmulatorUtils);
const mockedResolveLocalEgress = jest.mocked(resolveAndroidEmulatorLocalEgressAsync);
const mockedMkdtemp = jest.spyOn(fs.promises, 'mkdtemp');
function createStep(callInputs?: Record<string, unknown>, envOverrides?: NodeJS.ProcessEnv) {
const logger = createMockLogger();
@@ -65,6 +71,7 @@ describe(createStartAndroidEmulatorBuildFunction, () => {
mockedAndroidUtils.waitForReadyAsync.mockResolvedValue(undefined);
mockedAndroidUtils.disableWindowAndTransitionAnimationsAsync.mockResolvedValue(undefined);
mockedAndroidUtils.deleteAsync.mockResolvedValue(undefined);
mockedResolveLocalEgress.mockResolvedValue(null);
mockedRetryAsync.mockImplementation(async (fn, { retryOptions }) => {
let lastErr: unknown;
@@ -83,6 +90,82 @@ describe(createStartAndroidEmulatorBuildFunction, () => {
mockedMkdtemp.mockRestore();
});
describe('local egress', () => {
function createLocalEgress() {
return {
launchGate: {
emulatorArgs: ['-http-proxy', '127.0.0.1:8898', '-no-metrics'],
wrapperCommand: 'bash',
wrapperArgs: ['-c', 'gate', 'eas-egress-gate'],
admitAsync: jest.fn(),
},
networkReadyTarget: { host: '192.0.2.1', port: 443 },
configureBootedEmulatorAsync: jest.fn().mockResolvedValue(undefined),
};
}
it('starts the emulator exactly as before without a local egress session', async () => {
await createStep().executeAsync();
expect(mockedAndroidUtils.startAsync).toHaveBeenCalledWith({
deviceName: 'EasAndroidDevice01',
env: expect.any(Object),
logcatDirectory: '/tmp/logcat-directory',
});
expect(mockedAndroidUtils.waitForReadyAsync).toHaveBeenCalledWith({
env: expect.any(Object),
serialId: 'emulator-default',
timeoutMs: 60_000,
logger: expect.anything(),
});
expect(
mockedSpawn.mock.calls.filter(
([command, args]) =>
['sudo', 'nft', 'systemd-run'].includes(command) ||
(command === 'adb' && (args.includes('reverse') || args.includes('http_proxy')))
)
).toEqual([]);
});
it('fences, gates and configures the emulator', async () => {
const localEgress = createLocalEgress();
mockedResolveLocalEgress.mockResolvedValue(localEgress);
mockedAndroidUtils.startAsync.mockResolvedValueOnce(createStartResult('emulator-base'));
await createStep().executeAsync();
expect(mockedAndroidUtils.startAsync).toHaveBeenCalledWith(
expect.objectContaining({ launchGate: localEgress.launchGate })
);
expect(mockedAndroidUtils.waitForReadyAsync).toHaveBeenCalledWith(
expect.objectContaining({ networkReadyTarget: { host: '192.0.2.1', port: 443 } })
);
expect(localEgress.configureBootedEmulatorAsync).toHaveBeenCalledWith(
expect.objectContaining({ serialId: 'emulator-base' })
);
});
it('retries a boot whose local egress configuration fails', async () => {
const localEgress = createLocalEgress();
localEgress.configureBootedEmulatorAsync
.mockRejectedValueOnce(new Error('netsimd outside the fence'))
.mockResolvedValueOnce(undefined);
mockedResolveLocalEgress.mockResolvedValue(localEgress);
mockedAndroidUtils.startAsync
.mockResolvedValueOnce(createStartResult('emulator-1111'))
.mockResolvedValueOnce(createStartResult('emulator-2222'));
await createStep().executeAsync();
expect(mockedAndroidUtils.deleteAsync).toHaveBeenCalledWith(
expect.objectContaining({ serialId: 'emulator-1111' })
);
expect(localEgress.configureBootedEmulatorAsync).toHaveBeenLastCalledWith(
expect.objectContaining({ serialId: 'emulator-2222' })
);
});
});
it('passes profile and LCD inputs to emulator creation', async () => {
const systemImagePackage = 'system-images;android-35;default;x86_64';
await createStep({
@@ -1,10 +1,19 @@
import { BuildRuntimePlatform } from '@expo/steps';
import spawn from '@expo/turtle-spawn';
import fs from 'node:fs';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { type CustomBuildContext } from '../../../customBuildContext';
import { AndroidEmulatorUtils } from '../../../utils/AndroidEmulatorUtils';
import { IosSimulatorUtils } from '../../../utils/IosSimulatorUtils';
import { selectXcodeDeveloperDirectoryAsync } from '../../utils/remoteDeviceRunSession';
import { resolveAppium3VersionSpec, resolveAppiumDeviceAsync } from '../startAppiumRemoteSession';
import {
createStartAppiumRemoteSessionBuildFunction,
installAppiumAsync,
resolveAppium3VersionSpec,
resolveAppiumDeviceAsync,
} from '../startAppiumRemoteSession';
jest.mock('../../../utils/AndroidEmulatorUtils', () => ({
AndroidEmulatorUtils: { getAttachedDevicesAsync: jest.fn() },
@@ -13,8 +22,10 @@ jest.mock('../../../utils/IosSimulatorUtils', () => ({
IosSimulatorUtils: { getAvailableDevicesAsync: jest.fn() },
}));
jest.mock('../../utils/remoteDeviceRunSession', () => ({
...jest.requireActual('../../utils/remoteDeviceRunSession'),
selectXcodeDeveloperDirectoryAsync: jest.fn(),
}));
jest.mock('@expo/turtle-spawn', () => ({ __esModule: true, default: jest.fn() }));
const logger = { info: jest.fn(), warn: jest.fn() } as never;
@@ -84,3 +95,75 @@ describe(resolveAppiumDeviceAsync, () => {
expect(selectXcodeDeveloperDirectoryAsync).not.toHaveBeenCalled();
});
});
describe(installAppiumAsync, () => {
beforeEach(() => {
jest.clearAllMocks();
jest.mocked(spawn).mockImplementation((async (_command: string, args: string[]) => {
if (args.includes('--json')) {
return { stdout: JSON.stringify({}) };
}
return { stdout: '' };
}) as never);
});
it('installs Appium with npm by default', async () => {
const result = await installAppiumAsync({
versionSpec: '^3',
driverName: 'xcuitest',
env: { EXISTING: 'value' },
logger,
});
try {
expect(spawn).toHaveBeenNthCalledWith(
1,
'npm',
['install', '--no-audit', 'appium@^3'],
expect.objectContaining({
cwd: result.appiumHome,
env: expect.objectContaining({ APPIUM_HOME: result.appiumHome, EXISTING: 'value' }),
})
);
expect(spawn).toHaveBeenCalledWith(
result.appiumBinPath,
['driver', 'install', 'xcuitest'],
expect.objectContaining({ env: result.appiumEnv, logger })
);
} finally {
await fs.promises.rm(result.appiumHome, { recursive: true, force: true });
}
});
it('installs Appium with bun add when EAS_OVERRIDE_PACKAGE_MANAGER is bun', async () => {
const result = await installAppiumAsync({
versionSpec: '3.5.0',
driverName: 'uiautomator2',
env: { EAS_OVERRIDE_PACKAGE_MANAGER: 'bun' },
logger,
});
try {
expect(spawn).toHaveBeenNthCalledWith(
1,
'bun',
['add', '[email protected]'],
expect.objectContaining({ cwd: result.appiumHome })
);
} finally {
await fs.promises.rm(result.appiumHome, { recursive: true, force: true });
}
});
});
describe('createStartAppiumRemoteSessionBuildFunction', () => {
it('declares the launch inputs so serve-sim can launch the app', () => {
const ctx = {} as unknown as CustomBuildContext;
const buildFunction = createStartAppiumRemoteSessionBuildFunction(ctx);
const globalCtx = createGlobalContextMock();
expect(
buildFunction.inputProviders?.map(provider => provider(globalCtx, 'Test step').id)
).toEqual(expect.arrayContaining(['launch_app_identifier', 'launch_args', 'open_url']));
});
});
@@ -4,7 +4,9 @@ import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { type CustomBuildContext } from '../../../customBuildContext';
import { startDeviceSessionHostAsync } from '../../utils/deviceSessionHost';
import { isProcessDescendantOfAsync } from '../../../utils/processes';
import { pollArgentArtifactsForUploadAsync } from '../../utils/argentArtifacts';
import { startArgentEventCollectionAsync } from '../../utils/argentEvents';
@@ -15,7 +17,6 @@ import {
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
@@ -41,14 +42,17 @@ jest.mock('../../utils/argentEvents', () => ({
...jest.requireActual('../../utils/argentEvents'),
startArgentEventCollectionAsync: jest.fn(),
}));
jest.mock('../../utils/deviceSessionHost');
jest.mock('../../utils/remoteDeviceRunSession', () => ({
...jest.requireActual('../../utils/remoteDeviceRunSession'),
ensureFfmpegInstalledOnceAsync: jest.fn(),
finishRemoteSessionAsync: jest.requireActual('../../utils/remoteDeviceRunSession')
.finishRemoteSessionAsync,
getDeviceRunSessionIdOrThrow: jest.fn(),
getNgrokAuthtokenOrThrow: jest.fn(),
getNgrokTunnelDomainOrThrow: jest.fn(),
selectXcodeDeveloperDirectoryAsync: jest.fn(),
spawnDetached: jest.fn(),
startDeviceWebPreviewWithTunnelAsync: jest.fn(),
startNgrokTunnelAsync: jest.fn(),
uploadRemoteSessionConfigAsync: jest.fn(),
waitForDeviceRunSessionStoppedAsync: jest.fn(),
@@ -84,15 +88,21 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
jest.mocked(spawnDetached).mockReturnValue({
pid: 4242,
getOutput: () => '',
getExitError: () => undefined,
stopAsync: jest.fn(),
});
jest.mocked(startNgrokTunnelAsync).mockResolvedValue({
url: 'https://argent-abc.tunnel.example.com',
subdomainId: 'argent-abc',
stopAsync: mockTunnelStopAsync,
});
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
previewUrl: 'https://web-preview.tunnel.example.com',
stopAsync: mockPreviewStopAsync,
jest.mocked(startDeviceSessionHostAsync).mockResolvedValue({
openPreviewAsync: jest.fn().mockResolvedValue({
previewPageUrl: 'https://expo.dev/simulator-preview/preview-id',
apiUrl: 'https://web-preview.tunnel.example.com',
closeAsync: jest.fn(),
}),
finishAsync: mockPreviewStopAsync,
});
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockResolvedValue(undefined);
@@ -109,6 +119,75 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
await fs.promises.rm(TEST_HOME, { recursive: true, force: true });
});
it('reports an early exit with output before opening tunnels or publishing readiness', async () => {
jest.mocked(spawnDetached).mockReturnValue({
pid: 4242,
getOutput: () => 'could not bind server port',
getExitError: () => new Error('process exited with code 1'),
stopAsync: jest.fn(),
});
const buildFunction = createStartArgentRemoteSessionBuildFunction({} as CustomBuildContext);
await expect(
buildFunction.fn!(
{
logger: { info: jest.fn(), warn: jest.fn() },
global: { runtimePlatform: BuildRuntimePlatform.LINUX },
} as unknown as BuildStepContext,
{
inputs: {
package_version: { value: undefined },
max_idle_time_minutes: { value: undefined },
},
outputs: {},
env: {},
} as never
)
).rejects.toThrow(
'Argent exited before becoming ready: process exited with code 1\nArgent tool-server output:\ncould not bind server port'
);
expect(startNgrokTunnelAsync).not.toHaveBeenCalled();
expect(uploadRemoteSessionConfigAsync).not.toHaveBeenCalled();
expect(startArgentEventCollectionAsync).not.toHaveBeenCalled();
});
it.each(['preview', 'config', 'wait'])(
'finishes the host and tools after %s fails',
async phase => {
const error = new Error(`${phase} failed`);
if (phase === 'preview') {
jest.mocked(startDeviceSessionHostAsync).mockResolvedValueOnce({
openPreviewAsync: jest.fn().mockRejectedValue(error),
finishAsync: mockPreviewStopAsync,
});
} else if (phase === 'config') {
jest.mocked(uploadRemoteSessionConfigAsync).mockRejectedValueOnce(error);
} else {
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockRejectedValueOnce(error);
}
const buildFunction = createStartArgentRemoteSessionBuildFunction({} as CustomBuildContext);
await expect(
buildFunction.fn!(
{
logger: { info: jest.fn(), warn: jest.fn() },
global: { runtimePlatform: BuildRuntimePlatform.LINUX },
} as unknown as BuildStepContext,
{
inputs: {
package_version: { value: undefined },
max_idle_time_minutes: { value: undefined },
},
outputs: {},
env: {},
} as never
)
).rejects.toBe(error);
expect(mockPreviewStopAsync).toHaveBeenCalledTimes(1);
expect(mockTunnelStopAsync).toHaveBeenCalledTimes(1);
expect(mockStopAsync).toHaveBeenCalledTimes(1);
expect(jest.mocked(spawnDetached).mock.results[0].value.stopAsync).toHaveBeenCalledTimes(1);
}
);
it('enables the event log flag, shares one path, and starts/stops the collector', async () => {
const ctx = {} as unknown as CustomBuildContext;
const buildFunction = createStartArgentRemoteSessionBuildFunction(ctx);
@@ -155,6 +234,7 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
// (3) The tool-server and the collector are pinned to the exact same event log path.
const serverEnv = jest.mocked(spawnDetached).mock.calls[0][0].env;
expect(serverEnv.ARGENT_EVENT_LOG).toBe(EXPECTED_EVENT_LOG_PATH);
expect(serverEnv.ARGENT_EMULATOR_NO_WINDOW).toBe('1');
expect(serverEnv.EXISTING).toBe('value');
expect(jest.mocked(startArgentEventCollectionAsync).mock.calls[0][0]).toMatchObject({
deviceRunSessionId: 'device-run-session-id',
@@ -172,17 +252,138 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
jest.mocked(waitForDeviceRunSessionStoppedAsync).mock.invocationCallOrder[0]
);
expect(mockTunnelStopAsync).toHaveBeenCalledTimes(1);
expect(startDeviceWebPreviewWithTunnelAsync).toHaveBeenCalledWith(
expect(startDeviceSessionHostAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({ runtimePlatform: BuildRuntimePlatform.LINUX })
);
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledWith(
expect.objectContaining({
remoteConfig: expect.objectContaining({
webPreviewUrl: 'https://web-preview.tunnel.example.com',
webPreviewUrl: 'https://expo.dev/simulator-preview/preview-id',
previewApiUrl: 'https://web-preview.tunnel.example.com',
}),
})
);
expect(mockPreviewStopAsync).toHaveBeenCalledTimes(1);
});
it('hands the launch inputs to serve-sim and announces them on an iOS session', async () => {
const ctx = {} as unknown as CustomBuildContext;
const logger = { info: jest.fn(), warn: jest.fn() };
const buildFunction = createStartArgentRemoteSessionBuildFunction(ctx);
await buildFunction.fn!(
{
logger,
global: { runtimePlatform: BuildRuntimePlatform.DARWIN },
} as unknown as BuildStepContext,
{
inputs: {
package_version: { value: undefined },
max_idle_time_minutes: { value: undefined },
max_duration_seconds: { value: undefined },
launch_app_identifier: { value: 'host.exp.Exponent' },
launch_args: { value: ['-EXDevMenuIsOnboardingFinished', '1'] },
open_url: { value: 'exp://127.0.0.1:8081' },
},
outputs: {},
env: { EXISTING: 'value' },
} as never
);
expect(startDeviceSessionHostAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({
runtimePlatform: BuildRuntimePlatform.DARWIN,
launchAppIdentifier: 'host.exp.Exponent',
launchArgs: ['-EXDevMenuIsOnboardingFinished', '1'],
openUrl: 'exp://127.0.0.1:8081',
})
);
expect(logger.info).toHaveBeenCalledWith(
'serve-sim will launch host.exp.Exponent with arguments ' +
'["-EXDevMenuIsOnboardingFinished","1"], then open exp://127.0.0.1:8081.'
);
});
it('fails before starting anything when a launch is asked for on Android', async () => {
const ctx = {} as unknown as CustomBuildContext;
const logger = { info: jest.fn(), warn: jest.fn() };
const buildFunction = createStartArgentRemoteSessionBuildFunction(ctx);
await expect(
buildFunction.fn!(
{
logger,
global: { runtimePlatform: BuildRuntimePlatform.LINUX },
} as unknown as BuildStepContext,
{
inputs: {
package_version: { value: undefined },
max_idle_time_minutes: { value: undefined },
max_duration_seconds: { value: undefined },
launch_app_identifier: { value: 'host.exp.Exponent' },
},
outputs: {},
env: {},
} as never
)
).rejects.toThrow('runs on linux');
expect(spawn).not.toHaveBeenCalled();
});
it('declares the launch inputs', () => {
const ctx = {} as unknown as CustomBuildContext;
const buildFunction = createStartArgentRemoteSessionBuildFunction(ctx);
const globalCtx = createGlobalContextMock();
expect(
buildFunction.inputProviders?.map(provider => provider(globalCtx, 'Test step').id)
).toEqual(expect.arrayContaining(['launch_app_identifier', 'launch_args', 'open_url']));
});
it('stops automation without waiting for recording upload', async () => {
let release!: () => void;
let stopped!: () => void;
const pendingFinish = new Promise<void>(resolve => {
release = resolve;
});
const toolStopped = new Promise<void>(resolve => {
stopped = resolve;
});
mockPreviewStopAsync.mockReturnValueOnce(pendingFinish);
const stopServer = jest.fn(async () => {
stopped();
});
jest.mocked(spawnDetached).mockReturnValueOnce({
pid: 4242,
getOutput: () => '',
getExitError: () => undefined,
stopAsync: stopServer,
});
const buildFunction = createStartArgentRemoteSessionBuildFunction({} as CustomBuildContext);
const running = buildFunction.fn!(
{
logger: { info: jest.fn(), warn: jest.fn() },
global: { runtimePlatform: BuildRuntimePlatform.LINUX },
} as unknown as BuildStepContext,
{
inputs: {
package_version: { value: undefined },
max_idle_time_minutes: { value: undefined },
},
outputs: {},
env: {},
} as never
);
try {
await toolStopped;
expect(mockTunnelStopAsync).toHaveBeenCalledTimes(1);
expect(mockPreviewStopAsync).toHaveBeenCalledTimes(1);
expect(stopServer).toHaveBeenCalledTimes(1);
} finally {
release();
await running;
}
});
});
@@ -74,7 +74,7 @@ describe(warnIfArgentPackageVersionCannotBeVerified, () => {
).not.toThrow();
expect(warn).toHaveBeenCalledTimes(2);
expect(warn).toHaveBeenCalledWith(
expect.stringContaining('Continuing and letting bun x resolve it.')
expect.stringContaining('Continuing and letting the package manager resolve it.')
);
});
});
@@ -94,6 +94,59 @@ describe(waitForArgentToolServerStateAsync, () => {
return JSON.stringify({ port, pid, token });
}
it('reports process exit instead of waiting for the startup timeout', async () => {
const getExitError = jest.fn(() => new Error('server exited with code 1'));
await expect(
waitForArgentToolServerStateAsync({
stateDir,
ancestorPid: process.pid,
timeoutMs: 100,
pollIntervalMs: 1,
getExitError,
})
).rejects.toThrow('server exited with code 1');
expect(getExitError).toHaveBeenCalledTimes(1);
});
it('detects an exit after polling has begun', async () => {
const getExitError = jest
.fn<Error | undefined, []>()
.mockReturnValueOnce(undefined)
.mockReturnValueOnce(undefined)
.mockReturnValue(new Error('server stopped'));
await expect(
waitForArgentToolServerStateAsync({
stateDir,
ancestorPid: process.pid,
timeoutMs: 100,
pollIntervalMs: 1,
getExitError,
})
).rejects.toThrow('server stopped');
expect(getExitError).toHaveBeenCalledTimes(3);
});
it('rejects an exit observed while reading a matching state file', async () => {
await fs.promises.writeFile(
path.join(stateDir, 'tool-server.json'),
stateJson({ port: 4321, pid: process.pid })
);
const getExitError = jest
.fn<Error | undefined, []>()
.mockReturnValueOnce(undefined)
.mockReturnValue(new Error('server exited during startup'));
await expect(
waitForArgentToolServerStateAsync({
stateDir,
ancestorPid: process.ppid,
timeoutMs: 100,
pollIntervalMs: 1,
getExitError,
})
).rejects.toThrow('server exited during startup');
expect(getExitError).toHaveBeenCalledTimes(2);
});
it.each(['tool-server.json', 'tool-server-012345abcdef.json'])(
'reads a matching process from %s',
async fileName => {
@@ -3,6 +3,12 @@ import spawn from '@expo/turtle-spawn';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { createMockLogger } from '../../../__tests__/utils/logger';
import { IosSimulatorUtils } from '../../../utils/IosSimulatorUtils';
import { configureSimulatorProxyEnvironmentAsync } from '../../utils/localEgress';
import {
installLocalEgressGuardAsync,
resolveLocalEgressBootEnvironmentAsync,
verifyLocalEgressGuardAsync,
} from '../../utils/localEgressGuard';
import { createStartIosSimulatorBuildFunction } from '../startIosSimulator';
jest.mock('@expo/turtle-spawn', () => ({
@@ -16,14 +22,36 @@ jest.mock('../../../utils/IosSimulatorUtils', () => ({
getDeviceAsync: jest.fn(),
cloneAsync: jest.fn(),
enableAccessibilitySettingsAsync: jest.fn(),
resolveUdidAsync: jest.fn(),
bootAsync: jest.fn(),
startAsync: jest.fn(),
waitForReadyAsync: jest.fn(),
disableApsdAsync: jest.fn(),
},
}));
jest.mock('../../utils/localEgress', () => ({
configureSimulatorProxyEnvironmentAsync: jest.fn(),
}));
jest.mock('../../utils/localEgressGuard', () => ({
installLocalEgressGuardAsync: jest.fn(),
resolveLocalEgressBootEnvironmentAsync: jest.fn(),
verifyLocalEgressGuardAsync: jest.fn(),
}));
const mockedSpawn = jest.mocked(spawn);
const mockedUtils = jest.mocked(IosSimulatorUtils);
const mockedConfigureProxyEnvironment = jest.mocked(configureSimulatorProxyEnvironmentAsync);
const mockedInstallGuard = jest.mocked(installLocalEgressGuardAsync);
const mockedVerifyGuard = jest.mocked(verifyLocalEgressGuardAsync);
const mockedResolveBootEnvironment = jest.mocked(resolveLocalEgressBootEnvironmentAsync);
// Names resolve to udids the way the step expects; udids pass through.
const UDIDS: Record<string, string> = {
'iPhone 15': 'base',
'eas-simulator-1': 'clone-1',
'eas-simulator-2': 'clone-2',
};
function createStep(callInputs?: Record<string, unknown>) {
const logger = createMockLogger();
@@ -41,27 +69,41 @@ describe(createStartIosSimulatorBuildFunction, () => {
mockedUtils.getDeviceAsync.mockResolvedValue(null);
mockedUtils.cloneAsync.mockResolvedValue(undefined);
mockedUtils.enableAccessibilitySettingsAsync.mockResolvedValue(undefined);
mockedUtils.startAsync.mockResolvedValue({ udid: 'test-udid' as any });
mockedUtils.resolveUdidAsync.mockImplementation(
async ({ deviceIdentifier }) => (UDIDS[deviceIdentifier] ?? deviceIdentifier) as any
);
mockedUtils.bootAsync.mockResolvedValue(undefined);
mockedUtils.startAsync.mockImplementation(async ({ deviceIdentifier }) => ({
udid: deviceIdentifier as any,
}));
mockedUtils.waitForReadyAsync.mockResolvedValue(undefined);
mockedUtils.disableApsdAsync.mockResolvedValue(undefined);
mockedConfigureProxyEnvironment.mockResolvedValue(false);
mockedInstallGuard.mockResolvedValue(false);
mockedVerifyGuard.mockResolvedValue(undefined);
mockedResolveBootEnvironment.mockResolvedValue(null);
});
it('does not enable accessibility settings by default', async () => {
await createStep({ device_identifier: 'iPhone 15' }).executeAsync();
expect(mockedUtils.enableAccessibilitySettingsAsync).not.toHaveBeenCalled();
expect(mockedUtils.startAsync).toHaveBeenCalledWith({
expect(mockedUtils.resolveUdidAsync).toHaveBeenCalledWith({
deviceIdentifier: 'iPhone 15',
env: expect.any(Object),
});
expect(mockedUtils.bootAsync).toHaveBeenCalledWith({
deviceIdentifier: 'base',
env: expect.any(Object),
launchdEnvironment: {},
});
expect(mockedUtils.startAsync).toHaveBeenCalledWith({
deviceIdentifier: 'base',
env: expect.any(Object),
});
});
it('enables accessibility settings before starting the main device and every clone when requested', async () => {
mockedUtils.startAsync
.mockResolvedValueOnce({ udid: 'base' as any })
.mockResolvedValueOnce({ udid: 'clone-1' as any })
.mockResolvedValueOnce({ udid: 'clone-2' as any });
await createStep({
device_identifier: 'iPhone 15',
count: 2,
@@ -89,11 +131,6 @@ describe(createStartIosSimulatorBuildFunction, () => {
});
it('disables apsd on the main device and every clone', async () => {
mockedUtils.startAsync
.mockResolvedValueOnce({ udid: 'base' as any })
.mockResolvedValueOnce({ udid: 'clone-1' as any })
.mockResolvedValueOnce({ udid: 'clone-2' as any });
await createStep({ device_identifier: 'iPhone 15', count: 2 }).executeAsync();
expect(mockedUtils.disableApsdAsync).toHaveBeenCalledWith({
@@ -110,6 +147,84 @@ describe(createStartIosSimulatorBuildFunction, () => {
});
});
it('installs the proxy environment and guard between boot and boot completion, then verifies the guard', async () => {
mockedInstallGuard.mockResolvedValue(true);
await createStep({ device_identifier: 'iPhone 15', count: 2 }).executeAsync();
const udids = ['base', 'clone-1', 'clone-2'];
expect(
mockedUtils.bootAsync.mock.calls.map(([{ deviceIdentifier }]) => deviceIdentifier)
).toEqual(udids);
expect(mockedConfigureProxyEnvironment.mock.calls.map(([{ udid }]) => udid)).toEqual(udids);
expect(mockedInstallGuard.mock.calls.map(([{ udid }]) => udid)).toEqual(udids);
expect(mockedVerifyGuard.mock.calls.map(([{ udid }]) => udid)).toEqual(udids);
for (const [callIndex] of udids.entries()) {
const bootOrder = mockedUtils.bootAsync.mock.invocationCallOrder[callIndex];
const configureOrder = mockedConfigureProxyEnvironment.mock.invocationCallOrder[callIndex];
const guardOrder = mockedInstallGuard.mock.invocationCallOrder[callIndex];
const bootCompleteOrder = mockedUtils.startAsync.mock.invocationCallOrder[callIndex];
const verifyOrder = mockedVerifyGuard.mock.invocationCallOrder[callIndex];
const readyOrder = mockedUtils.waitForReadyAsync.mock.invocationCallOrder[callIndex];
// launchd is up when boot returns and has spawned nothing yet: that is the
// only moment at which its environment reaches every process of the boot.
expect(guardOrder).toBeGreaterThan(bootOrder);
expect(configureOrder).toBeGreaterThan(guardOrder);
expect(configureOrder).toBeLessThan(bootCompleteOrder);
// The self-check needs a completed boot and runs before anything else.
expect(verifyOrder).toBeGreaterThan(bootCompleteOrder);
expect(verifyOrder).toBeLessThan(readyOrder);
}
});
it('boots with the local egress environment so the first processes inherit it', async () => {
mockedResolveBootEnvironment.mockResolvedValue({
DYLD_INSERT_LIBRARIES: '/w/bin/egress-guard.dylib',
https_proxy: 'http://127.0.0.1:8899',
});
await createStep({ device_identifier: 'iPhone 15', count: 2 }).executeAsync();
for (const [{ deviceIdentifier, launchdEnvironment }] of mockedUtils.bootAsync.mock.calls) {
expect(['base', 'clone-1', 'clone-2']).toContain(deviceIdentifier);
expect(launchdEnvironment).toEqual({
DYLD_INSERT_LIBRARIES: '/w/bin/egress-guard.dylib',
https_proxy: 'http://127.0.0.1:8899',
});
}
});
it('boots with an empty launchd environment when no local egress session is active', async () => {
await createStep({ device_identifier: 'iPhone 15' }).executeAsync();
expect(mockedUtils.bootAsync).toHaveBeenCalledWith({
deviceIdentifier: 'base',
env: expect.any(Object),
launchdEnvironment: {},
});
});
it('skips the guard verification when no local egress session is active', async () => {
mockedInstallGuard.mockResolvedValue(false);
await createStep({ device_identifier: 'iPhone 15' }).executeAsync();
expect(mockedVerifyGuard).not.toHaveBeenCalled();
});
it('fails the step when the guard cannot be installed or verified', async () => {
mockedInstallGuard.mockRejectedValueOnce(new Error('guard library missing'));
await expect(createStep({ device_identifier: 'iPhone 15' }).executeAsync()).rejects.toThrow(
'guard library missing'
);
mockedInstallGuard.mockResolvedValue(true);
mockedVerifyGuard.mockRejectedValueOnce(new Error('self-check failed'));
await expect(createStep({ device_identifier: 'iPhone 15' }).executeAsync()).rejects.toThrow(
'self-check failed'
);
});
it('continues when disabling apsd fails', async () => {
mockedUtils.disableApsdAsync.mockRejectedValue(new Error('apsd disable failed'));
@@ -1,7 +1,8 @@
import { type bunyan } from '@expo/logger';
import { type BuildStepContext } from '@expo/steps';
import { BuildRuntimePlatform, type BuildStepContext } from '@expo/steps';
import fs from 'node:fs';
import { startAndroidLocalEgressAsync } from '../../utils/androidLocalEgress';
import {
LOCAL_EGRESS_HANDOFF_PATH,
configureSystemProxyAsync,
@@ -20,6 +21,12 @@ jest.mock('../../utils/localEgress', () => ({
startChiselServerAsync: jest.fn(),
writeLocalEgressHandoffAsync: jest.fn(),
}));
jest.mock('../../utils/androidLocalEgress', () => ({
startAndroidLocalEgressAsync: jest.fn(),
}));
jest.mock('../../utils/localEgressGuard', () => ({
stopLocalEgressGuardRelaysAsync: jest.fn().mockResolvedValue(undefined),
}));
jest.mock('../../utils/remoteDeviceRunSession', () => ({
findAvailablePortAsync: jest.fn(),
startNgrokTunnelAsync: jest.fn(),
@@ -28,8 +35,17 @@ jest.mock('../../utils/remoteDeviceRunSession', () => ({
}));
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const server = { pid: 12345, getOutput: () => '', stopAsync: jest.fn() };
const tunnel = { url: 'https://egress.example.com', stopAsync: jest.fn() };
const server = {
pid: 12345,
getOutput: () => '',
getExitError: () => undefined,
stopAsync: jest.fn(),
};
const tunnel = {
url: 'https://egress.example.com',
subdomainId: 'egress-id',
stopAsync: jest.fn(),
};
async function start(signal?: AbortSignal): Promise<void> {
await createStartLocalEgressBuildFunction().fn!({ logger } as BuildStepContext, {
@@ -40,6 +56,82 @@ async function start(signal?: AbortSignal): Promise<void> {
});
}
async function startOn(runtimePlatform: BuildRuntimePlatform): Promise<void> {
await createStartLocalEgressBuildFunction().fn!(
{ logger, global: { runtimePlatform } } as unknown as BuildStepContext,
{
env: {},
inputs: {},
outputs: {},
} as any
);
}
describe('local egress platforms', () => {
const androidStop = jest.fn();
beforeEach(() => {
jest.clearAllMocks();
jest.spyOn(fs.promises, 'mkdtemp').mockResolvedValue('/tmp/egress-acquisition-test');
jest.spyOn(fs.promises, 'writeFile').mockResolvedValue(undefined);
jest.spyOn(fs.promises, 'rm').mockResolvedValue(undefined);
server.stopAsync.mockResolvedValue(undefined);
tunnel.stopAsync.mockResolvedValue(undefined);
androidStop.mockResolvedValue(undefined);
jest.mocked(findAvailablePortAsync).mockResolvedValue(52001);
jest.mocked(downloadChiselAsync).mockResolvedValue('/tmp/egress-acquisition-test/chisel');
jest.mocked(startChiselServerAsync).mockResolvedValue({ process: server, fingerprint: 'key=' });
jest.mocked(startNgrokTunnelAsync).mockResolvedValue(tunnel);
jest.mocked(configureSystemProxyAsync).mockResolvedValue({ service: 'Ethernet' });
jest.mocked(writeLocalEgressHandoffAsync).mockResolvedValue(undefined);
jest.mocked(startAndroidLocalEgressAsync).mockResolvedValue({ stopAsync: androidStop });
});
afterEach(async () => {
await stopLocalEgressResourcesAsync(logger);
jest.restoreAllMocks();
});
it('sets the macOS system proxy and writes the same handoff as before', async () => {
await startOn(BuildRuntimePlatform.DARWIN);
expect(configureSystemProxyAsync).toHaveBeenCalled();
expect(startAndroidLocalEgressAsync).not.toHaveBeenCalled();
expect(writeLocalEgressHandoffAsync).toHaveBeenCalledWith({
url: tunnel.url,
token: expect.any(String),
fingerprint: 'key=',
port: 8899,
});
});
it('starts the Android relay and fence on Linux instead of the system proxy', async () => {
await startOn(BuildRuntimePlatform.LINUX);
expect(configureSystemProxyAsync).not.toHaveBeenCalled();
expect(startAndroidLocalEgressAsync).toHaveBeenCalledWith(
expect.objectContaining({
chiselPid: server.pid,
proxyPort: 8899,
controlPort: 52001,
workDir: '/tmp/egress-acquisition-test',
})
);
expect(writeLocalEgressHandoffAsync).toHaveBeenCalledWith(
expect.objectContaining({ platform: 'android' })
);
await stopLocalEgressResourcesAsync(logger);
expect(androidStop).toHaveBeenCalledTimes(1);
expect(server.stopAsync).toHaveBeenCalledTimes(1);
});
it('releases the tunnel when the Android fence cannot start', async () => {
jest.mocked(startAndroidLocalEgressAsync).mockRejectedValue(new Error('nft failed'));
await expect(startOn(BuildRuntimePlatform.LINUX)).rejects.toThrow('nft failed');
expect(server.stopAsync).toHaveBeenCalledTimes(1);
expect(tunnel.stopAsync).toHaveBeenCalledTimes(1);
expect(writeLocalEgressHandoffAsync).not.toHaveBeenCalled();
});
});
describe('local egress acquisition', () => {
beforeEach(() => {
jest.clearAllMocks();
@@ -59,19 +59,47 @@ describe('sandbox build functions', () => {
fn.fn!({ logger: {} } as any, {
inputs: { sandbox_id: { value: 'sandbox-id' } },
outputs: {},
env: {},
env: { __EAS_SANDBOX_MCP_TOKEN: 'token' },
signal: controller.signal,
})
).rejects.toThrow();
).rejects.toMatchObject({ name: 'AbortError' });
expect(mutation).not.toHaveBeenCalled();
expect(stopAsync).toHaveBeenCalledTimes(1);
} finally {
start.mockRestore();
}
});
it('provides one long-running step', () => {
const ctx = { job: {} } as any;
expect(createStartSandboxBuildFunction(ctx).getFullId()).toBe('eas/start_sandbox');
it('excludes the MCP token from the command environment', async () => {
const env = {
PREPARED_BY_EARLIER_STEP: 'value',
EXPO_TOKEN: 'expo-token',
__EAS_SANDBOX_MCP_TOKEN: 'token',
};
const startupError = new Error('stop after checking options');
const start = jest
.spyOn(sandboxDaemon, 'startSandboxDaemonAsync')
.mockRejectedValue(startupError);
const fn = createStartSandboxBuildFunction({
env: { __EAS_SANDBOX_MCP_TOKEN: 'token' },
mcpServerUrl: 'ws://localhost:8787',
} as any);
try {
await expect(
fn.fn!({ logger: {} } as any, {
inputs: { sandbox_id: { value: 'sandbox-id' } },
outputs: {},
env,
})
).rejects.toBe(startupError);
expect(start.mock.calls[0][0]).toMatchObject({ credential: 'token' });
expect(start.mock.calls[0][0].env).toEqual({
PREPARED_BY_EARLIER_STEP: 'value',
EXPO_TOKEN: 'expo-token',
});
expect(env.__EAS_SANDBOX_MCP_TOKEN).toBe('token');
} finally {
start.mockRestore();
}
});
it('throws a system error when the sandbox token is missing', async () => {
@@ -100,9 +128,9 @@ describe('sandbox build functions', () => {
fn.fn!({} as any, {
inputs: { sandbox_id: { value: 'sandbox-id' } },
outputs: {},
env: {},
env: { __EAS_SANDBOX_MCP_TOKEN: 'sandbox-token' },
})
).rejects.toBeInstanceOf(SystemError);
).rejects.toThrow('MCP server URL is required to start the sandbox daemon.');
});
it('marks the sandbox as ready', async () => {
@@ -1,25 +1,37 @@
import { type bunyan } from '@expo/logger';
import { BuildRuntimePlatform, type BuildStepContext, type BuildStepEnv } from '@expo/steps';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { type CustomBuildContext } from '../../../customBuildContext';
import { startDeviceSessionHostAsync } from '../../utils/deviceSessionHost';
import {
getDeviceRunSessionIdOrThrow,
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
startDeviceWebPreviewWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../../utils/remoteDeviceRunSession';
import { createStartWebPreviewRemoteSessionBuildFunction } from '../startWebPreviewRemoteSession';
jest.mock('../../utils/remoteDeviceRunSession');
jest.mock('../../utils/deviceSessionHost');
jest.mock('../../utils/remoteDeviceRunSession', () => ({
...jest.requireActual('../../utils/remoteDeviceRunSession'),
getDeviceRunSessionIdOrThrow: jest.fn(),
getNgrokTunnelDomainOrThrow: jest.fn(),
selectXcodeDeveloperDirectoryAsync: jest.fn(),
uploadRemoteSessionConfigAsync: jest.fn(),
waitForDeviceRunSessionStoppedAsync: jest.fn(),
}));
const ctx = {} as CustomBuildContext;
const env = {} as BuildStepEnv;
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const stopAsync = jest.fn();
async function runAsync(runtimePlatform: BuildRuntimePlatform): Promise<void> {
async function runAsync(
runtimePlatform: BuildRuntimePlatform,
launchInputs: Record<string, { value: unknown }> = {}
): Promise<void> {
const buildFunction = createStartWebPreviewRemoteSessionBuildFunction(ctx);
await buildFunction.fn!(
{
@@ -30,6 +42,7 @@ async function runAsync(runtimePlatform: BuildRuntimePlatform): Promise<void> {
inputs: {
package_version: { value: '1.2.3' },
max_duration_seconds: { value: 120 },
...launchInputs,
},
outputs: {},
env,
@@ -43,9 +56,13 @@ describe(createStartWebPreviewRemoteSessionBuildFunction, () => {
jest.mocked(getDeviceRunSessionIdOrThrow).mockReturnValue('device-run-session-id');
jest.mocked(getNgrokTunnelDomainOrThrow).mockReturnValue('tunnel.example.com');
jest.mocked(selectXcodeDeveloperDirectoryAsync).mockResolvedValue(undefined);
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
previewUrl: 'https://web-preview.example.test',
stopAsync,
jest.mocked(startDeviceSessionHostAsync).mockResolvedValue({
openPreviewAsync: jest.fn().mockResolvedValue({
previewPageUrl: 'https://expo.dev/simulator-preview/preview-id',
apiUrl: 'https://web-preview.example.test',
closeAsync: jest.fn(),
}),
finishAsync: stopAsync,
});
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockResolvedValue(undefined);
@@ -53,10 +70,14 @@ describe(createStartWebPreviewRemoteSessionBuildFunction, () => {
});
it('reports the session token when serve-sim minted one', async () => {
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
previewUrl: 'https://web-preview.example.test',
previewToken: 'tok-1',
stopAsync,
jest.mocked(startDeviceSessionHostAsync).mockResolvedValue({
openPreviewAsync: jest.fn().mockResolvedValue({
previewPageUrl: 'https://expo.dev/simulator-preview/preview-id',
apiUrl: 'https://web-preview.example.test',
previewToken: 'tok-1',
closeAsync: jest.fn(),
}),
finishAsync: stopAsync,
});
await runAsync(BuildRuntimePlatform.DARWIN);
@@ -64,13 +85,30 @@ describe(createStartWebPreviewRemoteSessionBuildFunction, () => {
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledWith(
expect.objectContaining({
remoteConfig: {
previewUrl: 'https://web-preview.example.test',
previewUrl: 'https://expo.dev/simulator-preview/preview-id',
previewApiUrl: 'https://web-preview.example.test',
previewToken: 'tok-1',
},
})
);
});
it.each(['preview', 'config', 'wait'])('finishes the session after %s fails', async phase => {
const error = new Error(`${phase} failed`);
if (phase === 'preview') {
jest.mocked(startDeviceSessionHostAsync).mockResolvedValueOnce({
openPreviewAsync: jest.fn().mockRejectedValue(error),
finishAsync: stopAsync,
});
} else if (phase === 'config') {
jest.mocked(uploadRemoteSessionConfigAsync).mockRejectedValueOnce(error);
} else {
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockRejectedValueOnce(error);
}
await expect(runAsync(BuildRuntimePlatform.LINUX)).rejects.toBe(error);
expect(stopAsync).toHaveBeenCalledTimes(1);
});
it.each([
[BuildRuntimePlatform.DARWIN, true],
[BuildRuntimePlatform.LINUX, false],
@@ -78,18 +116,25 @@ describe(createStartWebPreviewRemoteSessionBuildFunction, () => {
await runAsync(runtimePlatform);
expect(selectXcodeDeveloperDirectoryAsync).toHaveBeenCalledTimes(selectsXcode ? 1 : 0);
expect(startDeviceWebPreviewWithTunnelAsync).toHaveBeenCalledWith(ctx, {
expect(startDeviceSessionHostAsync).toHaveBeenCalledWith(ctx, {
runtimePlatform,
baseDomain: 'tunnel.example.com',
env,
logger,
timeoutMs: 60_000,
packageVersion: '1.2.3',
launchAppIdentifier: undefined,
launchArgs: [],
openUrl: undefined,
networkCapture: false,
networkCaptureFields: [],
});
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledWith({
ctx,
deviceRunSessionId: 'device-run-session-id',
remoteConfig: { previewUrl: 'https://web-preview.example.test' },
remoteConfig: {
previewUrl: 'https://expo.dev/simulator-preview/preview-id',
previewApiUrl: 'https://web-preview.example.test',
},
logger,
});
expect(waitForDeviceRunSessionStoppedAsync).toHaveBeenCalledWith({
@@ -101,4 +146,75 @@ describe(createStartWebPreviewRemoteSessionBuildFunction, () => {
});
expect(stopAsync).toHaveBeenCalledTimes(1);
});
it('declares the launch inputs', () => {
const buildFunction = createStartWebPreviewRemoteSessionBuildFunction(ctx);
const globalCtx = createGlobalContextMock();
expect(
buildFunction.inputProviders?.map(provider => provider(globalCtx, 'Test step').id)
).toEqual([
'launch_app_identifier',
'launch_args',
'open_url',
'network_capture',
'network_capture_fields',
'package_version',
'max_duration_seconds',
]);
});
it('hands the launch inputs to the session host and announces them', async () => {
await runAsync(BuildRuntimePlatform.DARWIN, {
launch_app_identifier: { value: 'host.exp.Exponent' },
launch_args: { value: ['-EXDevMenuIsOnboardingFinished', '1'] },
open_url: { value: 'exp://127.0.0.1:8081' },
});
expect(startDeviceSessionHostAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({
launchAppIdentifier: 'host.exp.Exponent',
launchArgs: ['-EXDevMenuIsOnboardingFinished', '1'],
openUrl: 'exp://127.0.0.1:8081',
})
);
expect(logger.info).toHaveBeenCalledWith(
'serve-sim will launch host.exp.Exponent with arguments ' +
'["-EXDevMenuIsOnboardingFinished","1"], then open exp://127.0.0.1:8081.'
);
});
it('hands network capture to the session host', async () => {
await runAsync(BuildRuntimePlatform.DARWIN, {
network_capture: { value: true },
network_capture_fields: { value: ['header', 'response-body'] },
});
expect(startDeviceSessionHostAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({
networkCapture: true,
networkCaptureFields: ['header', 'response-body'],
})
);
});
it('fails before starting anything when network capture is asked for on Android', async () => {
await expect(
runAsync(BuildRuntimePlatform.LINUX, {
network_capture: { value: true },
})
).rejects.toThrow('this session runs on linux');
expect(startDeviceSessionHostAsync).not.toHaveBeenCalled();
});
it('fails before starting anything when a launch is asked for on Android', async () => {
await expect(
runAsync(BuildRuntimePlatform.LINUX, {
launch_app_identifier: { value: 'host.exp.Exponent' },
})
).rejects.toThrow('runs on linux');
expect(startDeviceSessionHostAsync).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,533 @@
import nock from 'nock';
import { generateKeyPairSync } from 'node:crypto';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { createMockLogger } from '../../../__tests__/utils/logger';
import { AscApiClient } from '../../utils/ios/AscApiClient';
import {
createUpdateTestFlightMetadataBuildFunction,
updateTestFlightMetadataAsync,
} from '../updateTestFlightMetadata';
jest.unmock('node-fetch');
const { privateKey } = generateKeyPairSync('ec', { namedCurve: 'prime256v1' });
const key = { keyId: 'TESTKEY', privateKey };
const changelog = 'Test "quotes"\n$(not-a-command)';
const options = {
client: new AscApiClient({ key }),
buildUploadId: 'upload',
changelog,
groups: [] as string[],
logger: createMockLogger(),
};
const api = () => nock('https://api.appstoreconnect.apple.com');
function mockBuild(
state = 'COMPLETE',
buildId: string | null = 'build',
primaryLocale: string | null = 'en-US'
): void {
api()
.get('/v1/buildUploads/upload')
.query(true)
.reply(200, {
data: {
type: 'buildUploads',
id: 'upload',
attributes: { state: { state } },
relationships: { build: { data: buildId ? { type: 'builds', id: buildId } : null } },
},
});
if (state === 'COMPLETE' && buildId) {
api()
.get(`/v1/builds/${buildId}/app`)
.query(true)
.reply(200, {
data: { id: 'app', attributes: primaryLocale ? { primaryLocale } : {} },
});
}
}
function mockAssignedGroups(ids: string[] = []): void {
api()
.get('/v1/betaGroups')
.query({ 'filter[builds]': 'build', limit: '200' })
.reply(200, { data: ids.map(id => ({ id })) });
}
beforeAll(() => nock.disableNetConnect());
beforeEach(() => {
options.logger = createMockLogger();
options.client = new AscApiClient({ key });
});
afterAll(() => nock.enableNetConnect());
afterEach(() => {
try {
expect(nock.pendingMocks()).toEqual([]);
} finally {
nock.cleanAll();
}
});
it('accepts group arrays and preserves changelog input without shell parsing', async () => {
const fn = createUpdateTestFlightMetadataBuildFunction();
const spy = jest.spyOn(fn, 'fn').mockImplementation(async (_ctx, { inputs }) => {
expect(inputs.groups.value).toEqual(['A, "B"']);
expect(inputs.changelog.value).toBe(changelog);
});
const step = fn.createBuildStepFromFunctionCall(createGlobalContextMock(), {
callInputs: {
asc_api_key_path: '/tmp/key.json',
build_upload_id: 'upload',
groups: ['A, "B"'],
changelog,
},
});
await step.executeAsync();
expect(spy).toHaveBeenCalled();
});
it('uses empty metadata defaults', async () => {
const fn = createUpdateTestFlightMetadataBuildFunction();
const spy = jest.spyOn(fn, 'fn').mockImplementation(async (_ctx, { inputs }) => {
expect(inputs.groups.value).toEqual([]);
expect(inputs.changelog.value).toBe('');
});
const step = fn.createBuildStepFromFunctionCall(createGlobalContextMock(), {
callInputs: { asc_api_key_path: '/tmp/key.json', build_upload_id: 'upload' },
});
await step.executeAsync();
expect(spy).toHaveBeenCalled();
});
it('skips automatic internal groups and groups that already contain the build', async () => {
mockBuild();
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, {
data: [
{
id: 'automatic',
attributes: { name: 'Internal', isInternalGroup: true, hasAccessToAllBuilds: true },
},
{ id: 'assigned', attributes: { name: 'QA' } },
{
id: 'manual',
attributes: { name: 'External', isInternalGroup: false, hasAccessToAllBuilds: true },
},
],
});
api()
.get('/v1/betaGroups')
.query({ 'filter[builds]': 'build', limit: '200' })
.reply(200, { data: [{ id: 'assigned', attributes: { name: 'QA' } }] });
api()
.post('/v1/builds/build/relationships/betaGroups', {
data: [{ type: 'betaGroups', id: 'manual' }],
})
.reply(204);
await updateTestFlightMetadataAsync({
...options,
changelog: '',
groups: ['Internal', 'QA', 'External'],
});
expect(options.logger.info).toHaveBeenCalledWith(
expect.stringContaining('"Internal" (automatic): automatic access')
);
expect(options.logger.info).toHaveBeenCalledWith(
expect.stringContaining('"QA" (assigned): build already assigned')
);
expect(options.logger.info).toHaveBeenCalledWith(
expect.stringContaining('"External" (manual): assignment completed')
);
});
it.each(['en-US', null])(
'creates a localization when none exist (primary locale: %s)',
async primaryLocale => {
mockBuild('COMPLETE', 'build', primaryLocale);
api().get('/v1/builds/build/betaBuildLocalizations').query(true).reply(200, { data: [] });
api()
.post('/v1/betaBuildLocalizations', {
data: {
type: 'betaBuildLocalizations',
attributes: { locale: 'en-US', whatsNew: changelog },
relationships: { build: { data: { type: 'builds', id: 'build' } } },
},
})
.reply(201, { data: { id: 'locale' } });
await updateTestFlightMetadataAsync(options);
expect(options.logger.info).toHaveBeenCalledWith(
`Updating changelog: ${JSON.stringify(changelog)}`
);
expect(options.logger.info).toHaveBeenCalledWith(
expect.stringContaining('Locale "en-US": localization created')
);
}
);
it('reads all localization pages and updates existing localizations without replacing them', async () => {
mockBuild();
api()
.get('/v1/builds/build/betaBuildLocalizations')
.query(true)
.reply(200, {
data: [{ id: 'en-US', attributes: { locale: 'en-US' } }],
links: {
next: 'https://api.appstoreconnect.apple.com/v1/builds/build/betaBuildLocalizations?limit=200&cursor=next',
},
});
api()
.get('/v1/builds/build/betaBuildLocalizations')
.query({ limit: '200', cursor: 'next' })
.reply(200, { data: [{ id: 'pl', attributes: { locale: 'pl' } }] });
for (const id of ['en-US', 'pl']) {
api()
.patch(`/v1/betaBuildLocalizations/${id}`, {
data: { type: 'betaBuildLocalizations', id, attributes: { whatsNew: changelog } },
})
.reply(200, { data: { id } });
}
await updateTestFlightMetadataAsync(options);
for (const id of ['en-US', 'pl']) {
expect(options.logger.info).toHaveBeenCalledWith(
expect.stringContaining(`Locale "${id}" (${id})`)
);
}
});
it('updates by ID when Apple omits a localization locale and creates the primary locale', async () => {
mockBuild();
api()
.get('/v1/builds/build/betaBuildLocalizations')
.query(true)
.reply(200, { data: [{ id: 'localization' }] });
api()
.patch('/v1/betaBuildLocalizations/localization', {
data: {
type: 'betaBuildLocalizations',
id: 'localization',
attributes: { whatsNew: changelog },
},
})
.reply(200, { data: { id: 'localization' } });
api()
.post('/v1/betaBuildLocalizations', {
data: {
type: 'betaBuildLocalizations',
attributes: { locale: 'en-US', whatsNew: changelog },
relationships: { build: { data: { type: 'builds', id: 'build' } } },
},
})
.reply(201, { data: { id: 'primary' } });
await updateTestFlightMetadataAsync(options);
});
it('assigns requested groups when hasAccessToAllBuilds is null', async () => {
mockBuild();
mockAssignedGroups();
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, {
data: [
{
id: 'internal',
attributes: { name: 'Internal', isInternalGroup: true, hasAccessToAllBuilds: false },
},
{
id: 'external',
attributes: {
name: 'test external',
isInternalGroup: false,
hasAccessToAllBuilds: null,
publicLinkEnabled: false,
publicLink: 'https://testflight.apple.com/join/abc',
},
},
{
id: 'requested-external',
attributes: { name: 'Beta', isInternalGroup: false, hasAccessToAllBuilds: null },
},
],
});
for (const id of ['internal', 'requested-external']) {
api()
.post('/v1/builds/build/relationships/betaGroups', {
data: [{ type: 'betaGroups', id }],
})
.reply(204);
}
await expect(
updateTestFlightMetadataAsync({ ...options, changelog: '', groups: ['Internal', 'Beta'] })
).resolves.toBeUndefined();
});
it('adds groups without changing changelog or submitting beta review', async () => {
mockBuild();
mockAssignedGroups();
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, { data: [{ id: 'group', attributes: { name: 'A, "B"' } }] });
api()
.post('/v1/builds/build/relationships/betaGroups', {
data: [{ type: 'betaGroups', id: 'group' }],
})
.reply(204);
await updateTestFlightMetadataAsync({ ...options, changelog: '', groups: ['A, "B"', 'A, "B"'] });
});
it('reports missing groups independently of the changelog task', async () => {
mockBuild();
api().get('/v1/builds/build/betaBuildLocalizations').query(true).reply(200, { data: [] });
api()
.post('/v1/betaBuildLocalizations')
.reply(201, { data: { id: 'locale' } });
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, { data: [] });
await expect(updateTestFlightMetadataAsync({ ...options, groups: ['missing'] })).rejects.toThrow(
'The following TestFlight group was not found in App Store Connect: "missing".'
);
});
it('reads all pages and adds every group with a requested name', async () => {
mockBuild();
mockAssignedGroups();
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, {
data: [{ id: 'first', attributes: { name: 'A' } }],
links: {
next: 'https://api.appstoreconnect.apple.com/v1/betaGroups?filter%5Bapp%5D=app&limit=200&cursor=page-2',
},
});
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200', cursor: 'page-2' })
.reply(200, {
data: [
{ id: 'second', attributes: { name: 'A' } },
{ id: 'other', attributes: { name: 'B' } },
],
});
for (const id of ['first', 'second']) {
api()
.post('/v1/builds/build/relationships/betaGroups', {
data: [{ type: 'betaGroups', id }],
})
.reply(204);
}
await updateTestFlightMetadataAsync({ ...options, changelog: '', groups: ['A'] });
});
it('assigns found groups before reporting missing requested names', async () => {
mockBuild();
mockAssignedGroups();
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, { data: [{ id: 'group', attributes: { name: 'A' } }] });
api()
.post('/v1/builds/build/relationships/betaGroups', {
data: [{ type: 'betaGroups', id: 'group' }],
})
.reply(204);
await expect(
updateTestFlightMetadataAsync({ ...options, changelog: '', groups: ['A', 'B', 'C'] })
).rejects.toThrow(
'The following TestFlight groups were not found in App Store Connect: "B", "C".'
);
});
it('attempts all localization changes and reports every failed update', async () => {
mockBuild();
api()
.get('/v1/builds/build/betaBuildLocalizations')
.query(true)
.reply(200, {
data: ['pl', 'fr', 'de'].map(locale => ({ id: locale, attributes: { locale } })),
});
for (const id of ['pl', 'fr', 'de']) {
api()
.patch(`/v1/betaBuildLocalizations/${id}`)
.reply(
id === 'de' ? 200 : 403,
id === 'de' ? { data: { id } } : { errors: [{ code: `${id}_FAILED` }] }
);
}
api()
.post('/v1/betaBuildLocalizations', {
data: {
type: 'betaBuildLocalizations',
attributes: { locale: 'en-US', whatsNew: changelog },
relationships: { build: { data: { type: 'builds', id: 'build' } } },
},
})
.reply(201, { data: { id: 'locale' } });
await expect(updateTestFlightMetadataAsync(options)).rejects.toThrow(/pl_FAILED.*fr_FAILED/);
expect(options.logger.error).toHaveBeenCalledWith(expect.stringContaining('"pl" (pl)'));
expect(options.logger.error).toHaveBeenCalledWith(expect.stringContaining('"fr" (fr)'));
});
it.each([1, 2])('explains automatic internal group rejection (%s errors)', async errorCount => {
mockBuild();
mockAssignedGroups();
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, { data: [{ id: 'group', attributes: { name: 'A' } }] });
api()
.post('/v1/builds/build/relationships/betaGroups')
.reply(422, {
errors: Array.from({ length: errorCount }, () => ({
code: 'ENTITY_UNPROCESSABLE',
title: 'Builds cannot be assigned to this internal group.',
detail: 'Cannot add internal group to a build.',
})),
});
await expect(
updateTestFlightMetadataAsync({ ...options, changelog: '', groups: ['A'] })
).rejects.toThrow(
"App Store Connect can't add this build to a requested internal TestFlight group."
);
});
it('stops after 20 group pages', async () => {
mockBuild();
const next =
'https://api.appstoreconnect.apple.com/v1/betaGroups?filter%5Bapp%5D=app&limit=200&cursor=next';
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, { data: [], links: { next } });
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200', cursor: 'next' })
.times(19)
.reply(200, { data: [], links: { next } });
await expect(
updateTestFlightMetadataAsync({ ...options, changelog: '', groups: ['A'] })
).rejects.toThrow('We only support TestFlight group lists with up to 20 pages');
});
it('tries group assignment if the changelog update fails', async () => {
mockBuild();
mockAssignedGroups();
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, { data: [{ id: 'group', attributes: { name: 'A' } }] });
api().get('/v1/builds/build/betaBuildLocalizations').query(true).reply(200, { data: [] });
api()
.post('/v1/betaBuildLocalizations')
.reply(403, { errors: [{ code: 'FORBIDDEN', detail: 'Not allowed' }] });
api()
.post('/v1/builds/build/relationships/betaGroups', {
data: [{ type: 'betaGroups', id: 'group' }],
})
.reply(204);
await expect(updateTestFlightMetadataAsync({ ...options, groups: ['A'] })).rejects.toThrow('403');
});
it('reports both write failures', async () => {
mockBuild();
mockAssignedGroups();
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, { data: [{ id: 'group', attributes: { name: 'A' } }] });
api().get('/v1/builds/build/betaBuildLocalizations').query(true).reply(200, { data: [] });
api()
.post('/v1/betaBuildLocalizations')
.reply(403, { errors: [{ code: 'CHANGELOG_FAILED' }] });
api()
.post('/v1/builds/build/relationships/betaGroups')
.reply(403, { errors: [{ code: 'GROUPS_FAILED' }] });
await expect(updateTestFlightMetadataAsync({ ...options, groups: ['A'] })).rejects.toThrow(
/CHANGELOG_FAILED.*GROUPS_FAILED/
);
});
it.each([false, true])(
'attempts all groups and reports assignment errors (multiple: %s)',
async multiple => {
mockBuild();
mockAssignedGroups();
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, { data: ['A', 'B', 'C'].map(id => ({ id, attributes: { name: id } })) });
api()
.post('/v1/builds/build/relationships/betaGroups', {
data: [{ type: 'betaGroups', id: 'A' }],
})
.reply(
multiple ? 403 : 204,
multiple ? { errors: [{ code: 'FIRST_GROUP_FAILED' }] } : undefined
);
api()
.post('/v1/builds/build/relationships/betaGroups', {
data: [{ type: 'betaGroups', id: 'B' }],
})
.reply(422, { errors: [{ code: 'ENTITY_UNPROCESSABLE', title: 'Partial assignment' }] });
api()
.post('/v1/builds/build/relationships/betaGroups', {
data: [{ type: 'betaGroups', id: 'C' }],
})
.reply(204);
await expect(
updateTestFlightMetadataAsync({ ...options, changelog: '', groups: ['A', 'B', 'C'] })
).rejects.toThrow(multiple ? /FIRST_GROUP_FAILED.*Partial assignment/ : 'Partial assignment');
expect(options.logger.error).toHaveBeenCalledWith(
expect.stringContaining('Group "B" (B): assignment failed.')
);
}
);
it('reads all membership pages before assigning groups', async () => {
mockBuild();
api()
.get('/v1/betaGroups')
.query({ 'filter[app]': 'app', limit: '200' })
.reply(200, { data: [{ id: 'group', attributes: { name: 'A' } }] });
api()
.get('/v1/betaGroups')
.query({ 'filter[builds]': 'build', limit: '200' })
.reply(200, {
data: [],
links: {
next: 'https://api.appstoreconnect.apple.com/v1/betaGroups?filter%5Bbuilds%5D=build&limit=200&cursor=next',
},
});
api()
.get('/v1/betaGroups')
.query({ 'filter[builds]': 'build', limit: '200', cursor: 'next' })
.reply(200, { data: [{ id: 'group' }] });
await updateTestFlightMetadataAsync({ ...options, changelog: '', groups: ['A'] });
});
it.each(['PROCESSING', 'FAILED'])('rejects a %s upload', async state => {
mockBuild(state);
await expect(updateTestFlightMetadataAsync(options)).rejects.toThrow('not ready');
});
it('rejects a completed upload without its build relationship', async () => {
mockBuild('COMPLETE', null);
await expect(updateTestFlightMetadataAsync(options)).rejects.toThrow('not ready');
});
it('propagates Apple write failures', async () => {
mockBuild();
api().get('/v1/builds/build/betaBuildLocalizations').query(true).reply(200, { data: [] });
api()
.post('/v1/betaBuildLocalizations')
.reply(403, { errors: [{ code: 'FORBIDDEN', detail: 'Not allowed' }] });
await expect(updateTestFlightMetadataAsync(options)).rejects.toThrow('403');
});
@@ -29,7 +29,20 @@ describe(createUploadDeviceRunSessionScreenRecordingsBuildFunction, () => {
});
});
it('uploads simulator device metadata', async () => {
it.each([
{
platform: BuildRuntimePlatform.DARWIN,
udid: SIMULATOR_UDID,
name: 'iPhone 16',
runtime: 'iOS 18.6',
},
{
platform: BuildRuntimePlatform.LINUX,
udid: 'emulator-5554',
name: 'Pixel',
runtime: 'Android 16',
},
])('uploads device metadata on $platform', async ({ platform, udid, name, runtime }) => {
const recordingDirectory = await fs.mkdtemp(path.join(os.tmpdir(), 'ios-recording-test-'));
const recordingPath = path.join(recordingDirectory, 'recording.mp4');
await fs.writeFile(recordingPath, 'recording');
@@ -45,7 +58,7 @@ describe(createUploadDeviceRunSessionScreenRecordingsBuildFunction, () => {
try {
const globalContext = createGlobalContextMock({
runtimePlatform: BuildRuntimePlatform.DARWIN,
runtimePlatform: platform,
});
const buildStep = createUploadDeviceRunSessionScreenRecordingsBuildFunction(
{} as CustomBuildContext
@@ -54,9 +67,9 @@ describe(createUploadDeviceRunSessionScreenRecordingsBuildFunction, () => {
callInputs: {
recordings_json: [
{
udid: SIMULATOR_UDID,
deviceName: 'iPhone 16',
runtimeDisplayName: 'iOS 18.6',
udid,
deviceName: name,
runtimeDisplayName: runtime,
directory: recordingDirectory,
},
],
@@ -68,13 +81,13 @@ describe(createUploadDeviceRunSessionScreenRecordingsBuildFunction, () => {
expect(uploadDeviceRunSessionArtifactAsync).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({
name: 'iPhone 16 screen recording (01234567-…, started at Jul 10, 2026, 10:00:00.000 UTC)',
name: `${name} screen recording (${udid.slice(0, 8)}-…, started at Jul 10, 2026, 10:00:00.000 UTC)`,
metadata: {
__eas_type: 'screen-recording',
__eas_screen_recording: '1',
udid: SIMULATOR_UDID,
deviceName: 'iPhone 16',
runtimeDisplayName: 'iOS 18.6',
udid,
deviceName: name,
runtimeDisplayName: runtime,
firstFrameAt: '2026-07-10T10:00:00.000Z',
width: 1179,
height: 2556,
@@ -0,0 +1,46 @@
import { BuildFunction, BuildRuntimePlatform } from '@expo/steps';
import { type CustomBuildContext } from '../../customBuildContext';
import { Sentry } from '../../sentry';
import { getDeviceRunSessionIdOrThrow } from '../utils/remoteDeviceRunSession';
import { uploadServeSimLogsFileAsync } from '../utils/serveSimLogsArtifacts';
import { ServeSimLogsRecorder } from '../utils/serveSimLogsRecorder';
const UPLOAD_TIMEOUT_MS = 30_000;
export function createCollectServeSimLogsBuildFunction(ctx: CustomBuildContext): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'collect_serve_sim_logs',
name: 'Collect serve-sim simulator logs',
__metricsId: 'eas/collect_serve_sim_logs',
supportedRuntimePlatforms: [BuildRuntimePlatform.DARWIN],
fn: async ({ logger }, { env }) => {
try {
const collected = await ServeSimLogsRecorder.finishAsync({ logger });
if (collected.length === 0) {
logger.info('No simulator logs collected; skipping upload.');
return;
}
const deviceRunSessionId = getDeviceRunSessionIdOrThrow(env);
for (const { udid, filePath } of collected) {
// Each device gets its own budget so one stuck upload does not cancel the rest.
await uploadServeSimLogsFileAsync(ctx, {
deviceRunSessionId,
udid,
filePath,
logger,
signal: AbortSignal.timeout(UPLOAD_TIMEOUT_MS),
});
}
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not finalize serve-sim simulator logs', error);
logger.warn(
{ err: error },
'Could not finalize simulator logs; the session result is unchanged.'
);
}
},
});
}
@@ -25,6 +25,7 @@ import plist from 'plist';
import { CustomBuildContext } from '../../customBuildContext';
import { formatBytes } from '../../utils/artifacts';
import { decompressTarAsync, isFileTarGzAsync } from '../../utils/files';
import { graphqlAbortContext } from '../../utils/graphqlAbort';
import { retryOnDNSFailure } from '../../utils/retryOnDNSFailure';
import { pluralize } from '../../utils/strings';
@@ -78,7 +79,7 @@ export function createDownloadBuildFunction(ctx: CustomBuildContext): BuildFunct
required: true,
}),
],
fn: async (stepsCtx, { inputs, outputs }) => {
fn: async (stepsCtx, { inputs, outputs, signal }) => {
const { logger } = stepsCtx;
const extensions = z.array(z.string()).parse(inputs.extensions.value);
@@ -119,6 +120,7 @@ export function createDownloadBuildFunction(ctx: CustomBuildContext): BuildFunct
graphqlClient: ctx.graphqlClient,
robotAccessToken: stepsCtx.global.staticContext.job.secrets?.robotAccessToken ?? null,
extensions,
signal,
});
outputs.artifact_path.set(artifactPath);
@@ -129,11 +131,17 @@ export function createDownloadBuildFunction(ctx: CustomBuildContext): BuildFunct
async function fetchApplicationArchiveUrlAsync({
buildId,
graphqlClient,
signal,
}: {
buildId: string;
graphqlClient: Client;
signal?: AbortSignal;
}): Promise<string> {
const result = await graphqlClient.query(BUILD_BY_ID_QUERY, { buildId }).toPromise();
signal?.throwIfAborted();
const result = await graphqlClient
.query(BUILD_BY_ID_QUERY, { buildId }, graphqlAbortContext(signal))
.toPromise();
signal?.throwIfAborted();
if (result.error) {
const { error } = result;
@@ -161,9 +169,11 @@ export async function downloadBuildAsync(
graphqlClient: Client;
robotAccessToken: string | null;
extensions: string[];
/** Stops the build lookup, the request and the file write when aborted, and skips the rest. */
signal?: AbortSignal;
}
): Promise<{ artifactPath: string }> {
const { logger, graphqlClient, robotAccessToken, extensions } = params;
const { logger, graphqlClient, robotAccessToken, extensions, signal } = params;
let downloadUrl: string;
let headers: { Authorization: string } | undefined;
@@ -178,7 +188,7 @@ export async function downloadBuildAsync(
headers = undefined;
} else if (params.buildId) {
const buildId = z.string().uuid().parse(params.buildId);
downloadUrl = await fetchApplicationArchiveUrlAsync({ buildId, graphqlClient });
downloadUrl = await fetchApplicationArchiveUrlAsync({ buildId, graphqlClient, signal });
headers = robotAccessToken ? { Authorization: `Bearer ${robotAccessToken}` } : undefined;
} else {
throw new UserError(
@@ -191,7 +201,8 @@ export async function downloadBuildAsync(
path.join(os.tmpdir(), 'download_build-downloaded-')
);
const response = await retryOnDNSFailure(fetch)(downloadUrl, { headers });
signal?.throwIfAborted();
const response = await retryOnDNSFailure(fetch)(downloadUrl, { headers, signal });
if (!response.ok) {
const textResult = await asyncResult(response.text());
@@ -201,7 +212,9 @@ export async function downloadBuildAsync(
const archiveFilename = resolveArchiveFilename({ response, extensions });
const archivePath = path.join(downloadDestinationDirectory, archiveFilename);
// On abort, node-fetch emits an error on the body stream, which fails the pipeline.
await streamPipeline(response.body, fs.createWriteStream(archivePath));
signal?.throwIfAborted();
const { size } = await fs.promises.stat(archivePath);
@@ -217,10 +230,15 @@ export async function downloadBuildAsync(
const extractionDirectory = await fs.promises.mkdtemp(
path.join(os.tmpdir(), 'download_build-extracted-')
);
// Not interrupted on abort: tar cannot stop its in-flight file writes, so an abort during
// the extraction lets it finish (local, bounded work) and is reported right after it.
// This way no file is written after this function settles.
signal?.throwIfAborted();
await decompressTarAsync({
archivePath,
destinationDirectory: extractionDirectory,
});
signal?.throwIfAborted();
const matchingFiles = await glob(`**/*.(${extensions.join('|')})`, {
absolute: true,
@@ -0,0 +1,65 @@
import { bunyan } from '@expo/logger';
import { asyncResult } from '@expo/results';
import { BuildFunction, BuildRuntimePlatform, BuildStepEnv } from '@expo/steps';
import spawn from '@expo/turtle-spawn';
import { Sentry } from '../../sentry';
export function createInstallMitmproxyBuildFunction(): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'install_mitmproxy',
name: 'Install mitmproxy',
__metricsId: 'eas/install_mitmproxy',
supportedRuntimePlatforms: [BuildRuntimePlatform.DARWIN],
fn: async ({ logger }, { env }) => {
if (await isMitmproxyAvailableAsync(env)) {
logger.info('mitmproxy is already installed.');
return;
}
if (env.EAS_BUILD_RUNNER !== 'eas-build') {
logger.warn(
'mitmproxy is not installed and network capture needs it. Install it with `brew install --cask mitmproxy` and rerun the job.'
);
return;
}
try {
logger.info('Installing mitmproxy with Homebrew.');
await installMitmproxyWithHomebrewAsync({ env, logger });
if (!(await isMitmproxyAvailableAsync(env))) {
throw new Error('`brew install --cask mitmproxy` succeeded but mitmdump is not on PATH.');
}
logger.info('Installed mitmproxy.');
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not install mitmproxy for the device session', error, {
level: 'warning',
});
logger.warn(
{ err: error },
'Could not install mitmproxy, so network capture will record nothing in this session. Re-run the session to retry the install, or run it without network capture.'
);
}
},
});
}
async function isMitmproxyAvailableAsync(env: BuildStepEnv): Promise<boolean> {
return (await asyncResult(spawn('mitmdump', ['--version'], { env }))).ok;
}
async function installMitmproxyWithHomebrewAsync({
env,
logger,
}: {
env: BuildStepEnv;
logger: bunyan;
}): Promise<void> {
await spawn('brew', ['install', '--cask', 'mitmproxy'], {
env: { ...env, HOMEBREW_NO_AUTO_UPDATE: '1' },
logger,
});
}
@@ -184,7 +184,7 @@ function logApplicationLaunch(
logger.info(`Launching ${applicationIdentifier}${argumentsDescription}.`);
}
function parseNonEmptyStringInput(value: unknown, inputName: string): string {
export function parseNonEmptyStringInput(value: unknown, inputName: string): string {
if (typeof value !== 'string' || value.length === 0) {
throw new UserError(
'EAS_LAUNCH_APPLICATION_INVALID_INPUT',
@@ -194,7 +194,7 @@ function parseNonEmptyStringInput(value: unknown, inputName: string): string {
return value;
}
function parseLaunchArgsInput(value: unknown): string[] {
export function parseLaunchArgsInput(value: unknown): string[] {
if (value === undefined) {
return [];
}
@@ -207,7 +207,7 @@ function parseLaunchArgsInput(value: unknown): string[] {
return value;
}
function parseOpenUrlInput(value: unknown): string {
export function parseOpenUrlInput(value: unknown): string {
const openUrl = parseNonEmptyStringInput(value, 'open_url');
if (!URL.canParse(openUrl)) {
throw new UserError(
@@ -107,7 +107,12 @@ function buildMaestroArgs({
return { executable: 'maestro', args };
}
case 'maestro-runner': {
// The runner writes every report format to one directory. output_format only
// selects which extra report artifact EAS uploads after the test.
const args = [`--platform=${platform}`, 'test', `--output=${output}`, '--flatten'];
if (shards !== undefined && shards > 1) {
args.push(`--parallel=${shards}`);
}
if (includeTags) {
args.push(`--include-tags=${includeTags}`);
}
@@ -210,23 +215,20 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
}
const testsDirectory = path.join(home, '.maestro', 'tests');
const junitReportDirectory = path.join(testsDirectory, 'junit-reports');
const finalReportPath =
outputFormat === 'junit'
? path.join(testsDirectory, `${platform}-maestro-junit.xml`)
: undefined;
const junitFinalReportPath = path.join(testsDirectory, `${platform}-maestro-junit.xml`);
// Public docs (EAS workflows pre-packaged-jobs) document
// `${MAESTRO_TESTS_DIR}` for users to save screenshots/recordings into
// the uploaded dir.
const spawnEnv: NodeJS.ProcessEnv = { ...env, MAESTRO_TESTS_DIR: testsDirectory };
// Outputs are published BEFORE any throw below so downstream
// `if: always()` upload steps still see populated values when this
// step fails early.
// Publish known paths before backend validation so `if: always()` upload
// steps can use them after an early failure. final_report_path names the
// report selected by output_format, not every report the backend produces.
outputs.tests_directory.set(testsDirectory);
outputs.junit_report_directory.set(junitReportDirectory);
if (finalReportPath !== undefined) {
outputs.final_report_path.set(finalReportPath);
if (outputFormat === 'junit') {
outputs.final_report_path.set(junitFinalReportPath);
}
// Resolved after the output assignments above: an invalid backend input or
@@ -236,6 +238,16 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
input: inputs.backend.value,
env,
});
// The runner can write JUnit regardless of output_format; Maestro CLI only
// writes it when JUnit is selected. Either command may fail before it writes a file.
const mayHaveJUnitReports = backend === 'maestro-runner' || outputFormat === 'junit';
const maestroCliOutputPath =
backend === 'maestro' && outputFormat && outputFormat !== 'junit'
? path.join(testsDirectory, `${platform}-maestro-${outputFormat}.${outputFormat}`)
: null;
if (backend === 'maestro' && outputFormat === 'html' && maestroCliOutputPath) {
outputs.final_report_path.set(maestroCliOutputPath);
}
const flowPaths = parseInput(
FlowPathSchema,
@@ -259,16 +271,14 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
undefined,
'android_connection_mode and EAS_MAESTRO_ANDROID_CONNECTION_MODE must be either "adb" or "dadb".'
);
if (backend === 'maestro-runner' && shards !== undefined && shards > 1) {
if (
backend === 'maestro-runner' &&
outputFormat !== undefined &&
!['junit', 'html', 'allure'].includes(outputFormat)
) {
throw new UserError(
'ERR_MAESTRO_INVALID_INPUT',
'maestro-runner does not support EAS Maestro test sharding. Remove shards or set it to 1.'
);
}
if (backend === 'maestro-runner' && outputFormat !== undefined && outputFormat !== 'junit') {
throw new UserError(
'ERR_MAESTRO_INVALID_INPUT',
`maestro-runner only supports the "junit" output_format, but received "${outputFormat}".`
`maestro-runner supports "junit", "html", and "allure" output_format values, but received "${outputFormat}".`
);
}
const retryFailedOnly = inputs.retry_failed_only.value as boolean;
@@ -340,17 +350,14 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
}
for (let attempt = 0; attempt <= retries; attempt++) {
// maestro-runner writes its JUnit report and screenshot metadata to this directory.
// maestro-runner writes all report formats and screenshot metadata here.
const runnerOutputDirectory = path.join(
testsDirectory,
`${platform}-maestro-runner-attempt-${attempt}`
);
const outputPath =
outputFormat === 'junit'
? path.join(junitReportDirectory, `${platform}-maestro-junit-attempt-${attempt}.xml`)
: backend === 'maestro' && outputFormat
? path.join(testsDirectory, `${platform}-maestro-${outputFormat}.${outputFormat}`)
: null;
const outputPath = mayHaveJUnitReports
? path.join(junitReportDirectory, `${platform}-maestro-junit-attempt-${attempt}.xml`)
: maestroCliOutputPath;
const { executable, args: maestroArgs } = buildMaestroArgs({
backend,
platform,
@@ -374,6 +381,17 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
logger.warn({ err }, `Failed to clear ${runnerOutputDirectory} before the attempt.`);
}
}
// Maestro CLI uses the same HTML path for every retry. Remove the previous report
// before spawning so an attempt that fails before report generation cannot upload it.
if (backend === 'maestro' && outputFormat === 'html' && maestroCliOutputPath) {
try {
await fs.rm(maestroCliOutputPath, { force: true });
} catch (err) {
throw new SystemError('Failed to clear the previous Maestro HTML report', {
cause: err,
});
}
}
const attemptStartedAtMs = Date.now();
@@ -415,9 +433,8 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
reportDirectories.push(runnerOutputDirectory);
}
// Harvest this attempt's failure screenshots before any retry subsetting. Gated on
// junit: test-case-result rows (and therefore the summary icons) only exist for junit
// runs, so harvesting other formats would just create orphan artifacts the website hides.
// The pre-packaged job reports test cases only for JUnit (the default format).
// Individual screenshots need those test cases to be visible on the website.
if (outputFormat === 'junit') {
let screenshots: HarvestedScreenshot[];
switch (backend) {
@@ -507,14 +524,15 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
await sleepAsync(2000);
}
// Smart merge first; on data errors (bad XML, missing input) fall back
// to copy-latest so the caller still gets a single JUnit file.
// Only JUnit is merged into final_report_path. Runner HTML/Allure keeps its
// per-attempt JUnit files, but final_report_path names the selected report.
// On data errors (bad XML, missing input), fall back to copy-latest.
// Filesystem errors short-circuit straight to SystemError.
if (finalReportPath !== undefined) {
if (outputFormat === 'junit') {
try {
await mergeJUnitReports({
sourceDir: junitReportDirectory,
outputPath: finalReportPath,
outputPath: junitFinalReportPath,
});
} catch (mergeErr: any) {
if (isFilesystemError(mergeErr)) {
@@ -524,7 +542,7 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
try {
await copyLatestAttemptXml({
sourceDir: junitReportDirectory,
outputPath: finalReportPath,
outputPath: junitFinalReportPath,
});
} catch (copyErr: any) {
// Swallow: a copy failure here usually means maestro itself failed
@@ -532,14 +550,13 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
// the real reason and cancel billing for a user-side failure — let
// the lastAttemptExitCode check below surface ERR_MAESTRO_TESTS_FAILED.
logger.warn(
`Failed to produce final_report_path at ${finalReportPath}: ${copyErr?.message ?? copyErr}`
`Failed to produce final_report_path at ${junitFinalReportPath}: ${copyErr?.message ?? copyErr}`
);
}
}
}
// Upload before the ERR_MAESTRO_TESTS_FAILED throw below so fully-failed runs (which need
// screenshots most) still upload. Harvest only ran for junit, so guard the same way.
// Upload before the failure verdict so fully-failed runs still get screenshots.
if (outputFormat === 'junit') {
await uploadFailureScreenshotsAsync({
harvested,
@@ -550,6 +567,81 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
});
}
// Upload the selected non-JUnit report before the failure verdict. The pre-packaged
// job also uploads testsDirectory, but a named artifact makes the report easy to find.
let selectedReport:
| { name: string; artifactPath: string; finalReportPath: string }
| undefined;
switch (outputFormat) {
case 'html': {
switch (backend) {
case 'maestro': {
if (!maestroCliOutputPath) {
break;
}
selectedReport = {
name: 'Maestro HTML Report',
artifactPath: maestroCliOutputPath,
finalReportPath: maestroCliOutputPath,
};
break;
}
case 'maestro-runner': {
const reportDirectory = reportDirectories.at(-1);
if (!reportDirectory) {
break;
}
// HTML references nearby screenshots, so keep its whole attempt directory.
selectedReport = {
name: 'Maestro Runner HTML Report',
artifactPath: reportDirectory,
finalReportPath: path.join(reportDirectory, 'report.html'),
};
break;
}
}
break;
}
case 'allure': {
switch (backend) {
case 'maestro':
logger.warn(
'Maestro CLI does not support Allure reports; no Allure artifact was uploaded.'
);
break;
case 'maestro-runner': {
const reportDirectory = reportDirectories.at(-1);
if (!reportDirectory) {
break;
}
const allureResultsDirectory = path.join(reportDirectory, 'allure-results');
selectedReport = {
name: 'Maestro Runner Allure Results',
artifactPath: allureResultsDirectory,
finalReportPath: allureResultsDirectory,
};
break;
}
}
break;
}
}
if (selectedReport) {
outputs.final_report_path.set(selectedReport.finalReportPath);
try {
await ctx.runtimeApi.uploadArtifact({
artifact: {
type: GenericArtifactType.OTHER,
name: selectedReport.name,
paths: [selectedReport.artifactPath],
},
logger,
});
} catch (err: any) {
logger.warn({ err }, `Failed to upload ${selectedReport.name}.`);
}
}
// The retry loop exits via success (0), numeric status (retryable),
// or throw (infra). A non-null non-zero status means the user's tests
// failed every attempt.
@@ -17,17 +17,32 @@ import {
uploadRemoteSessionConfigWithLocalEgressAsync,
withLocalEgressSession,
} from '../utils/localEgressSession';
import { type DeviceSessionHost, startDeviceSessionHostAsync } from '../utils/deviceSessionHost';
import { Sentry } from '../../sentry';
import {
PackageManager,
resolveConfiguredPackageManager,
resolvePackageAdd,
resolvePackageInstall,
} from '../../utils/packageManager';
import { pollAgentDeviceArtifactsForUploadAsync } from '../utils/agentDeviceArtifacts';
import {
createNetworkCaptureInputProviders,
parseNetworkCaptureInputs,
} from '../utils/networkCaptureFields';
import { startAgentDeviceEventCollectionAsync } from '../utils/agentDeviceEvents';
import { type StartupTasks, createStartupTasks } from '../utils/startupTasks';
import {
type DetachedProcessHandle,
createServeSimLaunchInputProviders,
describeServeSimLaunch,
finishRemoteSessionAsync,
getDeviceRunSessionIdOrThrow,
getNgrokAuthtokenOrThrow,
getNgrokTunnelDomainOrThrow,
parseServeSimLaunchInputs,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
waitForDeviceRunSessionStoppedAsync,
waitForFileAsync,
@@ -42,6 +57,13 @@ const STARTUP_TIMEOUT_MS = 60_000;
const AGENT_DEVICE_DAEMON_ENV = {
AGENT_DEVICE_DAEMON_SERVER_MODE: 'http',
AGENT_DEVICE_RETAIN_ARTIFACTS: '1',
// The session lifetime is owned by max_idle_time_minutes / max_duration_seconds
// and the device run session stop, so disable agent-device's own idle timers.
// Without this the daemon exits after 5 minutes with no open session, and the
// tunnel is left without a daemon behind it.
AGENT_DEVICE_DAEMON_IDLE_TIMEOUT_MS: '0',
AGENT_DEVICE_IOS_RUNNER_IDLE_STOP_MS: '0',
AGENT_DEVICE_SESSION_IDLE_TIMEOUT_MS: '0',
};
export function createStartAgentDeviceRemoteSessionBuildFunction(
@@ -53,6 +75,8 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
name: 'Start agent device remote session',
__metricsId: 'eas/start_agent_device_remote_session',
inputProviders: [
...createServeSimLaunchInputProviders(),
...createNetworkCaptureInputProviders(),
BuildStepInput.createProvider({
id: 'package_version',
required: false,
@@ -70,119 +94,350 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
}),
],
fn: withLocalEgressSession(async ({ logger, global }, { inputs, env, signal }) => {
// Fail fast before any expensive setup if the injected env
// vars are missing: DEVICE_RUN_SESSION_ID (to report the remote config
// back to the API server), EAS_SIMULATOR_NGROK_TUNNEL_DOMAIN (base domain
// for our ngrok tunnels), and NGROK_AUTHTOKEN (to authenticate them).
const deviceRunSessionId = getDeviceRunSessionIdOrThrow(env);
const ngrokTunnelDomain = getNgrokTunnelDomainOrThrow(env);
const ngrokAuthtoken = getNgrokAuthtokenOrThrow(env);
// Fail fast before any expensive setup if the injected env vars are missing.
const sessionEnv = getAgentDeviceRemoteSessionEnvOrThrow(env);
const packageVersion = inputs.package_version.value as string | undefined;
// A missing or non-positive value disables the idle timeout (opt-in feature).
const maxIdleTimeMinutes = inputs.max_idle_time_minutes.value as number | undefined;
const maxDurationSeconds = inputs.max_duration_seconds?.value as number | undefined;
const { runtimePlatform } = global;
logger.info(
`Starting agent-device remote session (version: ${packageVersion ?? 'latest'}, runtime: ${runtimePlatform}).`
const launch = parseServeSimLaunchInputs(
{
launchAppIdentifier: inputs.launch_app_identifier?.value as string | undefined,
launchArgs: inputs.launch_args?.value,
openUrl: inputs.open_url?.value as string | undefined,
},
{ runtimePlatform }
);
const capture = parseNetworkCaptureInputs(
{
networkCapture: inputs.network_capture?.value,
networkCaptureFields: inputs.network_capture_fields?.value,
},
{ runtimePlatform }
);
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
await selectXcodeDeveloperDirectoryAsync({ env, logger });
}
logger.info('Launching agent-device daemon.');
const daemonProcess = await startAgentDeviceDaemonAsync({ packageVersion, env, logger });
logger.info(`Waiting for daemon credentials at ${DAEMON_JSON_PATH}.`);
const { port: daemonPort, token: daemonToken } = await waitForDaemonInfoAsync({
daemonProcess,
});
logger.info(`Daemon is listening on port ${daemonPort}; loaded auth token.`);
const agentDeviceTunnel = await startNgrokTunnelAsync({
port: daemonPort,
subdomainPrefix: 'agent-device',
baseDomain: ngrokTunnelDomain,
authtoken: ngrokAuthtoken,
// Earlier steps booted the device and installed and launched the app.
await runAgentDeviceRemoteSessionAsync(ctx, {
env,
logger,
signal,
runtimePlatform,
sessionEnv,
packageVersion,
maxIdleTimeMinutes,
maxDurationSeconds,
launch,
capture,
tasks: createStartupTasks(logger),
device: { booted: Promise.resolve(), ready: Promise.resolve() },
});
const agentDeviceRemoteSessionUrl = agentDeviceTunnel.url;
logger.info(`Tunnel is ready at ${agentDeviceRemoteSessionUrl}.`);
let webPreview: Awaited<ReturnType<typeof startDeviceWebPreviewWithTunnelAsync>> | undefined;
let eventCollection:
| Awaited<ReturnType<typeof startAgentDeviceEventCollectionAsync>>
| undefined;
try {
webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
logger.info(`Web preview URL: ${webPreview.previewUrl}`);
await uploadRemoteSessionConfigWithLocalEgressAsync({
env,
signal,
ctx,
deviceRunSessionId,
remoteConfig: {
agentDeviceRemoteSessionUrl,
agentDeviceRemoteSessionToken: daemonToken,
webPreviewUrl: webPreview.previewUrl,
...(webPreview.previewToken ? { webPreviewToken: webPreview.previewToken } : {}),
},
logger,
});
void pollAgentDeviceArtifactsForUploadAsync(ctx, {
deviceRunSessionId,
daemonUrl: `http://127.0.0.1:${daemonPort}`,
daemonToken,
logger,
});
eventCollection = await startAgentDeviceEventCollectionAsync({
ctx,
deviceRunSessionId,
stateDir: AGENT_DEVICE_STATE_DIR,
logger,
});
await waitForDeviceRunSessionStoppedAsync({
ctx,
deviceRunSessionId,
logger,
maxDurationSeconds,
signal,
idleTimeout:
maxIdleTimeMinutes !== undefined && maxIdleTimeMinutes > 0
? {
maxIdleTimeMinutes,
getLastEventObservedAt: eventCollection.getLastEventObservedAt,
}
: undefined,
});
} finally {
if (webPreview) {
await webPreview.stopAsync();
}
await agentDeviceTunnel.stopAsync();
if (eventCollection) {
await stopAgentDeviceEventCollectionSafelyAsync({
eventCollection,
deviceRunSessionId,
logger,
});
}
await daemonProcess.stopAsync();
}
}),
});
}
export type AgentDeviceRemoteSessionEnv = {
deviceRunSessionId: string;
ngrokTunnelDomain: string;
ngrokAuthtoken: string;
};
/**
* Reads the env vars that the API server injects into a device run session job:
* DEVICE_RUN_SESSION_ID (to report the remote config back to the API server),
* EAS_SIMULATOR_NGROK_TUNNEL_DOMAIN (base domain for our ngrok tunnels), and
* NGROK_AUTHTOKEN (to authenticate them).
*/
export function getAgentDeviceRemoteSessionEnvOrThrow(
env: BuildStepEnv
): AgentDeviceRemoteSessionEnv {
return {
deviceRunSessionId: getDeviceRunSessionIdOrThrow(env),
ngrokTunnelDomain: getNgrokTunnelDomainOrThrow(env),
ngrokAuthtoken: getNgrokAuthtokenOrThrow(env),
};
}
/**
* Starts the agent-device daemon, its tunnel, the session host and the web preview,
* reports the session as ready, and keeps it alive until it stops.
*
* The daemon does not need the device, so it starts at once. The session host starts
* when `device.booted` resolves. The session is reported as ready only when both are
* up and `device.ready` (the app is installed and launched) resolved too.
*
* The first failure aborts `tasks.signal`: each part stops before its next stage, and
* the teardown then stops whatever was started. `device.ready` must settle soon after
* an abort, because the teardown waits for it.
*/
export async function runAgentDeviceRemoteSessionAsync(
ctx: CustomBuildContext,
{
env,
logger,
signal,
runtimePlatform,
sessionEnv: { deviceRunSessionId, ngrokTunnelDomain, ngrokAuthtoken },
packageVersion,
maxIdleTimeMinutes,
maxDurationSeconds,
launch,
capture,
tasks,
device,
}: {
env: BuildStepEnv;
logger: bunyan;
signal?: AbortSignal;
runtimePlatform: BuildRuntimePlatform;
sessionEnv: AgentDeviceRemoteSessionEnv;
packageVersion: string | undefined;
maxIdleTimeMinutes: number | undefined;
maxDurationSeconds: number | undefined;
launch: ReturnType<typeof parseServeSimLaunchInputs>;
capture: ReturnType<typeof parseNetworkCaptureInputs>;
tasks: StartupTasks;
device: { booted: Promise<unknown>; ready: Promise<unknown> };
}
): Promise<void> {
logger.info(
`Starting agent-device remote session (version: ${packageVersion ?? 'latest'}, runtime: ${runtimePlatform}).`
);
let daemonProcess: DetachedProcessHandle | undefined;
let agentDeviceTunnel: Awaited<ReturnType<typeof startNgrokTunnelAsync>> | undefined;
let sessionHost: DeviceSessionHost | undefined;
let eventCollection: Awaited<ReturnType<typeof startAgentDeviceEventCollectionAsync>> | undefined;
let sessionFailed = false;
// Each task stores what it started, so the teardown below can stop it even when
// another task failed first.
const agentDeviceStartup = tasks.run('agent-device daemon', async taskLogger => {
taskLogger.info('Launching agent-device daemon.');
daemonProcess = await startAgentDeviceDaemonAsync({
packageVersion,
env,
logger: taskLogger,
signal: tasks.signal,
});
taskLogger.info(`Waiting for daemon credentials at ${DAEMON_JSON_PATH}.`);
// The daemon is stored above, so the teardown stops it. The wait itself is bounded
// (STARTUP_TIMEOUT_MS) and only reads a file, so it may finish in the background.
const daemonInfo = await tasks.untilAborted(waitForDaemonInfoAsync({ daemonProcess }));
taskLogger.info(`Daemon is listening on port ${daemonInfo.port}; loaded auth token.`);
tasks.signal.throwIfAborted();
agentDeviceTunnel = await startNgrokTunnelAsync({
port: daemonInfo.port,
subdomainPrefix: 'agent-device',
baseDomain: ngrokTunnelDomain,
authtoken: ngrokAuthtoken,
logger: taskLogger,
});
taskLogger.info(`Tunnel is ready at ${agentDeviceTunnel.url}.`);
return { ...daemonInfo, remoteSessionUrl: agentDeviceTunnel.url };
});
const sessionHostStartup = tasks.run('session host', async taskLogger => {
// A boot cannot be cancelled, so stop waiting for it when startup is aborted.
await tasks.untilAborted(device.booted);
tasks.signal.throwIfAborted();
const launchDescription = describeServeSimLaunch(launch);
if (launchDescription) {
taskLogger.info(launchDescription);
}
sessionHost = await startDeviceSessionHostAsync(ctx, {
runtimePlatform,
env,
logger: taskLogger,
timeoutMs: STARTUP_TIMEOUT_MS,
launchAppIdentifier: launch.launchAppIdentifier,
launchArgs: launch.launchArgs,
openUrl: launch.openUrl,
networkCapture: capture.networkCapture,
networkCaptureFields: capture.networkCaptureFields,
});
tasks.signal.throwIfAborted();
const webPreview = await sessionHost.openPreviewAsync({ baseDomain: ngrokTunnelDomain });
taskLogger.info(
`Web preview URL: ${webPreview.previewPageUrl} (server: ${webPreview.apiUrl}).`
);
return webPreview;
});
try {
const [
{ port: daemonPort, token: daemonToken, remoteSessionUrl: agentDeviceRemoteSessionUrl },
webPreview,
] = await Promise.all([agentDeviceStartup, sessionHostStartup, device.ready]).catch(
(err: unknown) => {
// Also aborts for a `device.ready` that does not come from `tasks.run`.
tasks.abort(err);
throw err;
}
);
logger.info(tasks.summary());
await uploadRemoteSessionConfigWithLocalEgressAsync({
env,
signal,
ctx,
deviceRunSessionId,
remoteConfig: {
agentDeviceRemoteSessionUrl,
agentDeviceRemoteSessionToken: daemonToken,
webPreviewUrl: webPreview.previewPageUrl,
previewApiUrl: webPreview.apiUrl,
...(webPreview.previewToken ? { webPreviewToken: webPreview.previewToken } : {}),
},
logger,
});
void pollAgentDeviceArtifactsForUploadAsync(ctx, {
deviceRunSessionId,
daemonUrl: `http://127.0.0.1:${daemonPort}`,
daemonToken,
logger,
});
eventCollection = await startAgentDeviceEventCollectionAsync({
ctx,
deviceRunSessionId,
stateDir: AGENT_DEVICE_STATE_DIR,
logger,
});
await waitForDeviceRunSessionStoppedAsync({
ctx,
deviceRunSessionId,
logger,
maxDurationSeconds,
signal,
idleTimeout:
maxIdleTimeMinutes !== undefined && maxIdleTimeMinutes > 0
? {
maxIdleTimeMinutes,
getLastEventObservedAt: eventCollection.getLastEventObservedAt,
}
: undefined,
});
} catch (error) {
sessionFailed = true;
throw error;
} finally {
// Promise.all rejects on the first failure while other tasks can still be starting.
// They stop at their next abort check. Wait for all of them, so the teardown sees
// everything that was started.
await Promise.allSettled([agentDeviceStartup, sessionHostStartup, device.ready]);
const startedDaemon = daemonProcess;
await finishRemoteSessionAsync({
logger,
sessionFailed,
teardown: [
['agent-device tunnel', agentDeviceTunnel?.stopAsync()],
[
'agent-device daemon',
startedDaemon &&
(async () => {
try {
if (eventCollection) {
await stopAgentDeviceEventCollectionSafelyAsync({
eventCollection,
deviceRunSessionId,
logger,
});
}
} finally {
await startedDaemon.stopAsync();
}
})(),
],
['session host', sessionHost?.finishAsync()],
],
});
}
}
export async function startAgentDeviceDaemonAsync({
packageVersion,
env,
logger,
signal,
}: {
packageVersion: string | undefined;
env: BuildStepEnv;
logger: bunyan;
/** Kills the install and stops before the daemon starts, when aborted. No git fallback then. */
signal?: AbortSignal;
}): Promise<DetachedProcessHandle> {
const packageSpec = createAgentDevicePackageSpec(packageVersion);
const packageManager = resolveConfiguredPackageManager(env, PackageManager.BUN);
const installDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'eas-agent-device-'));
await fs.promises.writeFile(
path.join(installDir, 'package.json'),
`${JSON.stringify({ name: 'eas-agent-device', private: true })}\n`
);
try {
const add = resolvePackageAdd(packageManager, packageSpec);
logger.info(`Installing ${packageSpec} with ${add.command}.`);
await spawn(add.command, add.args, { cwd: installDir, env, logger, signal });
const daemonPath = getInstalledAgentDeviceDaemonPath(installDir);
if (!fs.existsSync(daemonPath)) {
throw new SystemError(`Expected agent-device daemon entry at ${daemonPath}.`);
}
signal?.throwIfAborted();
logger.info(`Launching daemon from ${daemonPath} after ${add.command} install.`);
const daemonProcess = spawnDetached({
command: 'node',
args: [daemonPath],
env: { ...env, ...AGENT_DEVICE_DAEMON_ENV },
});
return {
...daemonProcess,
stopAsync: async () => {
await daemonProcess.stopAsync();
await fs.promises.rm(installDir, { recursive: true, force: true });
},
};
} catch (err) {
await fs.promises.rm(installDir, { recursive: true, force: true });
// An abort is not an install problem: stop instead of falling back to git.
if (signal?.aborted) {
throw err;
}
const error = err instanceof Error ? err : new Error(String(err));
const bunVersion = await getBunVersionForDiagnosticsAsync(env);
Sentry.capture(
'Failed to start agent-device daemon from the configured package manager; falling back to git clone',
error,
{
level: 'warning',
tags: {
phase: 'agent-device-daemon-start',
fallback: 'git-clone',
},
extras: {
packageSpec,
packageVersion: packageVersion ?? 'latest',
packageManager,
bunVersion,
},
}
);
logger.warn(
`Failed to start daemon from ${packageSpec} via ${packageManager}; falling back to git clone: ${error.message}`
);
return await startAgentDeviceDaemonFromGitAsync({ packageVersion, env, logger, signal });
}
}
export async function stopAgentDeviceEventCollectionSafelyAsync({
eventCollection,
deviceRunSessionId,
@@ -205,85 +460,37 @@ export async function stopAgentDeviceEventCollectionSafelyAsync({
}
}
async function startAgentDeviceDaemonAsync({
packageVersion,
env,
logger,
}: {
packageVersion: string | undefined;
env: BuildStepEnv;
logger: bunyan;
}): Promise<DetachedProcessHandle> {
const packageSpec = createAgentDevicePackageSpec(packageVersion);
try {
logger.info(`Installing ${packageSpec} globally with Bun.`);
await spawn('bun', ['add', '--global', packageSpec], {
env,
logger,
});
const daemonPath = getGlobalAgentDeviceDaemonPath(env);
if (!fs.existsSync(daemonPath)) {
throw new SystemError(`Expected agent-device daemon entry at ${daemonPath}.`);
}
logger.info(`Launching daemon from ${daemonPath}.`);
return spawnDetached({
command: 'node',
args: [daemonPath],
env: { ...env, ...AGENT_DEVICE_DAEMON_ENV },
});
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
const bunVersion = await getBunVersionForDiagnosticsAsync(env);
Sentry.capture(
'Failed to start agent-device daemon from global Bun package; falling back to git clone',
error,
{
level: 'warning',
tags: {
phase: 'agent-device-daemon-start',
fallback: 'git-clone',
},
extras: {
packageSpec,
packageVersion: packageVersion ?? 'latest',
bunVersion,
bunInstallConfigured: Boolean(env.BUN_INSTALL?.trim()),
},
}
);
logger.warn(
`Failed to start daemon from global ${packageSpec}; falling back to git clone: ${error.message}`
);
return await startAgentDeviceDaemonFromGitAsync({ packageVersion, env, logger });
}
}
async function startAgentDeviceDaemonFromGitAsync({
packageVersion,
env,
logger,
signal,
}: {
packageVersion: string | undefined;
env: BuildStepEnv;
logger: bunyan;
signal?: AbortSignal;
}): Promise<DetachedProcessHandle> {
logger.info(
packageVersion
? `Cloning agent-device @ v${packageVersion} into ${SRC_DIR}.`
: `Cloning agent-device (latest) into ${SRC_DIR}.`
);
await cloneAgentDeviceAsync({ packageVersion, env, logger });
await cloneAgentDeviceAsync({ packageVersion, env, logger, signal });
logger.info('Installing agent-device dependencies.');
await spawn('bun', ['install', '--production'], {
const packageManager = resolveConfiguredPackageManager(env, PackageManager.BUN);
const install = resolvePackageInstall(packageManager, { production: true });
logger.info(`Installing agent-device dependencies with ${install.command}.`);
await spawn(install.command, install.args, {
cwd: SRC_DIR,
env,
logger,
signal,
});
signal?.throwIfAborted();
logger.info('Launching daemon from cloned agent-device source.');
// Git fallback is TypeScript source. The published path runs node on dist JS.
return spawnDetached({
command: 'bun',
args: ['run', 'src/daemon.ts'],
@@ -296,15 +503,18 @@ async function cloneAgentDeviceAsync({
packageVersion,
env,
logger,
signal,
}: {
packageVersion: string | undefined;
env: BuildStepEnv;
logger: bunyan;
signal?: AbortSignal;
}): Promise<void> {
const branchArgs = packageVersion ? ['--branch', `v${packageVersion}`] : [];
await spawn('git', ['clone', '--depth', '1', ...branchArgs, AGENT_DEVICE_REPO_URL, SRC_DIR], {
env,
logger,
signal,
});
}
@@ -346,11 +556,9 @@ function createAgentDevicePackageSpec(packageVersion: string | undefined): strin
return `${AGENT_DEVICE_PACKAGE_NAME}@${versionSpec}`;
}
function getGlobalAgentDeviceDaemonPath(env: BuildStepEnv): string {
function getInstalledAgentDeviceDaemonPath(installDir: string): string {
return path.join(
getBunInstallDirectory(env),
'install',
'global',
installDir,
'node_modules',
AGENT_DEVICE_PACKAGE_NAME,
'dist',
@@ -360,11 +568,6 @@ function getGlobalAgentDeviceDaemonPath(env: BuildStepEnv): string {
);
}
function getBunInstallDirectory(env: BuildStepEnv): string {
const bunInstall = env.BUN_INSTALL?.trim();
return bunInstall ? bunInstall : path.join(os.homedir(), '.bun');
}
function parseDaemonInfo(raw: string): { port: number; token: string } {
const parsed = JSON.parse(raw) as unknown;
if (
@@ -0,0 +1,272 @@
import { UserError } from '@expo/eas-build-job';
import {
BuildFunction,
BuildRuntimePlatform,
BuildStepInput,
BuildStepInputValueTypeName,
} from '@expo/steps';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { type CustomBuildContext } from '../../customBuildContext';
import {
AndroidDeviceName,
AndroidEmulatorUtils,
AndroidVirtualDeviceName,
} from '../../utils/AndroidEmulatorUtils';
import { IosSimulatorName, IosSimulatorUuid } from '../../utils/IosSimulatorUtils';
import { withLocalEgressSession } from '../utils/localEgressSession';
import {
parseServeSimLaunchInputs,
selectXcodeDeveloperDirectoryAsync,
} from '../utils/remoteDeviceRunSession';
import {
createNetworkCaptureInputProviders,
parseNetworkCaptureInputs,
} from '../utils/networkCaptureFields';
import { createStartupTasks } from '../utils/startupTasks';
import { downloadBuildAsync } from './downloadBuild';
import { installBuildAsync } from './installBuild';
import {
launchApplicationAsync,
parseLaunchArgsInput,
parseOpenUrlInput,
} from './launchApplication';
import {
getAgentDeviceRemoteSessionEnvOrThrow,
runAgentDeviceRemoteSessionAsync,
} from './startAgentDeviceRemoteSession';
import { startAndroidEmulatorAsync } from './startAndroidEmulator';
import { bootIosSimulatorAsync } from './startIosSimulator';
const ANDROID_DEVICE_NAME = 'EasAndroidDevice01' as AndroidVirtualDeviceName;
/**
* One step for a whole agent-device session: boot the device, download, install and
* launch the app, and start the agent-device daemon and the web preview.
*
* It replaces eas/start_ios_simulator or eas/start_android_emulator, eas/download_build,
* eas/install_build, eas/launch_application and eas/start_agent_device_remote_session,
* so the parts that do not depend on each other can run at the same time:
*
* boot ─────────────┬─► install ─► launch ─┐
* download ─────────┘ │
* boot ─► session host ─► web preview ─────┼─► ready
* agent-device daemon ─► tunnel ───────────┘
*
* The first failure aborts the rest: the download stops, and nothing installs, launches
* or starts after it. A boot cannot be cancelled, so it can still run when a failed
* step returns.
*/
export function createStartAgentDeviceSessionBuildFunction(ctx: CustomBuildContext): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'start_agent_device_session',
name: 'Start agent-device session',
__metricsId: 'eas/start_agent_device_session',
inputProviders: [
// Device. On iOS a Simulator name or UDID, on Android an AVD hardware profile.
BuildStepInput.createProvider({
id: 'device_identifier',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
id: 'system_image_package',
required: false,
defaultValue: AndroidEmulatorUtils.defaultSystemImagePackage,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
id: 'lcd_width',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
}),
BuildStepInput.createProvider({
id: 'lcd_height',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
}),
BuildStepInput.createProvider({
id: 'lcd_density',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
}),
// Application. Pass at most one of build_id and application_archive_url.
BuildStepInput.createProvider({
id: 'build_id',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
id: 'application_archive_url',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
id: 'launch_args',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.JSON,
}),
BuildStepInput.createProvider({
id: 'open_url',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
// Session.
...createNetworkCaptureInputProviders(),
BuildStepInput.createProvider({
id: 'package_version',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
id: 'max_idle_time_minutes',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
}),
BuildStepInput.createProvider({
id: 'max_duration_seconds',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
}),
],
fn: withLocalEgressSession(async (stepsCtx, { inputs, env, signal }) => {
const { logger, global } = stepsCtx;
const { runtimePlatform } = global;
const isIos = runtimePlatform === BuildRuntimePlatform.DARWIN;
// Validate everything before any expensive work starts.
const sessionEnv = getAgentDeviceRemoteSessionEnvOrThrow(env);
const buildId = inputs.build_id.value as string | undefined;
const applicationArchiveUrl = inputs.application_archive_url.value as string | undefined;
if (buildId && applicationArchiveUrl) {
throw new UserError(
'EAS_DOWNLOAD_BUILD_INVALID_SOURCE',
'Pass only one of build_id or application_archive_url.'
);
}
const hasApplication = Boolean(buildId || applicationArchiveUrl);
const launchArgs = parseLaunchArgsInput(inputs.launch_args.value);
const openUrl =
inputs.open_url.value === undefined ? undefined : parseOpenUrlInput(inputs.open_url.value);
if (!hasApplication && (launchArgs.length > 0 || openUrl)) {
throw new UserError(
'EAS_LAUNCH_APPLICATION_INVALID_INPUT',
'launch_args and open_url need an application: pass build_id or application_archive_url.'
);
}
const deviceIdentifier = inputs.device_identifier.value as string | undefined;
const capture = parseNetworkCaptureInputs(
{
networkCapture: inputs.network_capture.value,
networkCaptureFields: inputs.network_capture_fields.value,
},
{ runtimePlatform }
);
if (isIos) {
// Before the boot, so every Xcode tool below uses the same developer directory.
await selectXcodeDeveloperDirectoryAsync({ env, logger });
}
const tasks = createStartupTasks(logger);
const booted = tasks.run(
isIos ? 'iOS Simulator boot' : 'Android Emulator boot',
async taskLogger => {
if (isIos) {
await bootIosSimulatorAsync({
deviceIdentifier: deviceIdentifier as IosSimulatorUuid | IosSimulatorName | undefined,
env,
logger: taskLogger,
});
return;
}
await startAndroidEmulatorAsync({
deviceName: ANDROID_DEVICE_NAME,
systemImagePackage: `${inputs.system_image_package.value}`,
deviceIdentifier: deviceIdentifier as AndroidDeviceName | undefined,
lcdWidth: inputs.lcd_width.value as number | undefined,
lcdHeight: inputs.lcd_height.value as number | undefined,
lcdDensity: inputs.lcd_density.value as number | undefined,
logcatDirectory: await fs.promises.mkdtemp(
path.join(os.tmpdir(), 'eas-android-emulator-logcat-')
),
env,
logger: taskLogger,
});
}
);
const downloaded = hasApplication
? tasks.run('build download', async taskLogger => {
const { artifactPath } = await downloadBuildAsync({
logger: taskLogger,
...(buildId ? { buildId } : { applicationArchiveUrl: applicationArchiveUrl! }),
graphqlClient: ctx.graphqlClient,
robotAccessToken: global.staticContext.job.secrets?.robotAccessToken ?? null,
extensions: isIos ? ['app'] : ['apk'],
signal: tasks.signal,
});
return artifactPath;
})
: undefined;
// `ready` settles only after everything it started has settled, so the session
// teardown, which waits for it, never leaves a download running. It must also settle
// soon after an abort: it stops waiting for the boot then, and the download stops.
const ready = downloaded
? tasks.run('app install and launch', async taskLogger => {
const [download, boot] = await Promise.allSettled([
downloaded,
tasks.untilAborted(booted),
]);
// After a failure elsewhere, report that failure, not the abort it caused here.
tasks.signal.throwIfAborted();
if (download.status === 'rejected') {
throw download.reason;
}
if (boot.status === 'rejected') {
throw boot.reason;
}
const { applicationIdentifier, activityName } = await installBuildAsync({
artifactPath: download.value,
runtimePlatform,
env,
logger: taskLogger,
});
tasks.signal.throwIfAborted();
await launchApplicationAsync({
applicationIdentifier,
activityName,
launchArgs,
openUrl,
runtimePlatform,
env,
logger: taskLogger,
});
})
: tasks.untilAborted(booted);
await runAgentDeviceRemoteSessionAsync(ctx, {
env,
logger,
signal,
runtimePlatform,
sessionEnv,
packageVersion: inputs.package_version.value as string | undefined,
// A missing or non-positive value disables the idle timeout (opt-in feature).
maxIdleTimeMinutes: inputs.max_idle_time_minutes.value as number | undefined,
maxDurationSeconds: inputs.max_duration_seconds.value as number | undefined,
// The app is launched with simctl / adb above, not by serve-sim.
launch: parseServeSimLaunchInputs({}, { runtimePlatform }),
capture,
tasks,
device: { booted, ready },
});
}),
});
}
@@ -1,8 +1,10 @@
import { UserError } from '@expo/eas-build-job';
import { type bunyan } from '@expo/logger';
import { asyncResult } from '@expo/results';
import {
BuildFunction,
BuildRuntimePlatform,
type BuildStepEnv,
BuildStepInput,
BuildStepInputValueTypeName,
BuildStepOutput,
@@ -14,10 +16,12 @@ import path from 'node:path';
import {
AndroidDeviceName,
AndroidDeviceSerialId,
AndroidEmulatorUtils,
AndroidVirtualDeviceName,
} from '../../utils/AndroidEmulatorUtils';
import { retryAsync } from '../../utils/retry';
import { resolveAndroidEmulatorLocalEgressAsync } from '../utils/androidLocalEgress';
const ANDROID_STARTUP_ATTEMPT_TIMEOUT_MS = [60_000, 120_000, 180_000];
const ANDROID_STARTUP_RETRIES_COUNT = ANDROID_STARTUP_ATTEMPT_TIMEOUT_MS.length - 1;
@@ -80,140 +84,20 @@ export function createStartAndroidEmulatorBuildFunction(): BuildFunction {
);
outputs.logcat_directory.set(logcatDirectory);
if (env.EAS_NO_EMULATOR_HOST_SUPPORT_CHECK !== '1') {
await assertAndroidEmulatorHostSupportAsync({ env });
} else {
logger.info(
'Skipping Android emulator host support check because $EAS_NO_EMULATOR_HOST_SUPPORT_CHECK is enabled.'
);
}
try {
const availableDevices = await AndroidEmulatorUtils.getAvailableDevicesAsync({ env });
logger.info(`Available Android devices:\n- ${availableDevices.join(`\n- `)}`);
} catch (error) {
logger.info('Failed to list available Android devices.', error);
} finally {
logger.info('');
}
const deviceName = `${inputs.device_name.value}` as AndroidVirtualDeviceName;
const systemImagePackage = `${inputs.system_image_package.value}`;
// We can cast because allowedValueTypeName validated this is a string.
const deviceIdentifier = inputs.device_identifier.value as AndroidDeviceName | undefined;
const lcdWidth = inputs.lcd_width.value as number | undefined;
const lcdHeight = inputs.lcd_height.value as number | undefined;
const lcdDensity = inputs.lcd_density.value as number | undefined;
const shouldAdjustAnimationScale =
env.ANDROID_EMULATOR_ADJUST_ANIMATION_SCALE !== 'false' &&
env.ANDROID_EMULATOR_ADJUST_ANIMATION_SCALE !== '0';
if (!shouldAdjustAnimationScale) {
logger.info(
'Skipping Android emulator animation scale adjustments because $ANDROID_EMULATOR_ADJUST_ANIMATION_SCALE is disabled.'
);
}
logger.info('Making sure system image is installed');
await retryAsync(
async () => {
await spawn('sdkmanager', [systemImagePackage], {
env,
logger,
});
},
{
const { serialId, emulatorPromise, shouldAdjustAnimationScale } =
await startAndroidEmulatorAsync({
deviceName,
systemImagePackage: `${inputs.system_image_package.value}`,
// We can cast because allowedValueTypeName validated this is a string.
deviceIdentifier: inputs.device_identifier.value as AndroidDeviceName | undefined,
lcdWidth: inputs.lcd_width.value as number | undefined,
lcdHeight: inputs.lcd_height.value as number | undefined,
lcdDensity: inputs.lcd_density.value as number | undefined,
logcatDirectory,
env,
logger,
retryOptions: {
retries: 3, // Retry 3 times
retryIntervalMs: 1_000,
},
}
);
let emulatorPromise = null;
let serialId = null;
await retryAsync(
async attemptCount => {
const timeoutMs = ANDROID_STARTUP_ATTEMPT_TIMEOUT_MS[attemptCount];
const attempt = attemptCount + 1;
const maxAttempts = ANDROID_STARTUP_ATTEMPT_TIMEOUT_MS.length;
const attemptSuffix = attempt > 1 ? ` (attempt ${attempt}/${maxAttempts})` : '';
let attemptSerialId = null;
try {
logger.info(`Creating emulator device${attemptSuffix}.`);
await AndroidEmulatorUtils.createAsync({
deviceName,
systemImagePackage,
deviceIdentifier: deviceIdentifier ?? null,
lcdWidth: lcdWidth ?? null,
lcdHeight: lcdHeight ?? null,
lcdDensity: lcdDensity ?? null,
env,
logger,
});
logger.info(`Starting emulator device${attemptSuffix}.`);
const startResult = await AndroidEmulatorUtils.startAsync({
deviceName,
env,
logcatDirectory,
});
attemptSerialId = startResult.serialId;
await AndroidEmulatorUtils.waitForReadyAsync({
env,
serialId: attemptSerialId,
timeoutMs,
logger,
});
if (shouldAdjustAnimationScale) {
await AndroidEmulatorUtils.disableWindowAndTransitionAnimationsAsync({
env,
logger,
serialId: attemptSerialId,
});
}
logger.info(`${deviceName} is ready.`);
serialId = attemptSerialId;
emulatorPromise = startResult.emulatorPromise;
} catch (err) {
logger.warn(
{ err },
`${deviceName} failed to start on attempt ${attempt}/${maxAttempts}.`
);
try {
if (attemptSerialId) {
await AndroidEmulatorUtils.deleteAsync({
serialId: attemptSerialId,
deviceName,
env,
});
} else {
await AndroidEmulatorUtils.deleteAsync({
deviceName,
env,
});
}
} catch (cleanupErr) {
logger.warn({ err: cleanupErr }, `Failed to clean up ${deviceName}.`);
}
throw err;
}
},
{
logger,
retryOptions: {
retries: ANDROID_STARTUP_RETRIES_COUNT,
retryIntervalMs: 1_000,
},
}
);
if (!serialId || !emulatorPromise) {
throw new Error(`Failed to start emulator ${deviceName}.`);
}
});
const count = Number(inputs.count.value ?? 1);
if (count > 1) {
@@ -306,6 +190,167 @@ export function createStartAndroidEmulatorBuildFunction(): BuildFunction {
});
}
/**
* Checks the host, installs the system image, then creates and starts one emulator and
* waits until it is ready, with retries.
*/
export async function startAndroidEmulatorAsync({
deviceName,
systemImagePackage,
deviceIdentifier,
lcdWidth,
lcdHeight,
lcdDensity,
logcatDirectory,
env,
logger,
}: {
deviceName: AndroidVirtualDeviceName;
systemImagePackage: string;
deviceIdentifier?: AndroidDeviceName;
lcdWidth?: number;
lcdHeight?: number;
lcdDensity?: number;
logcatDirectory: string;
env: BuildStepEnv;
logger: bunyan;
}): Promise<{
serialId: AndroidDeviceSerialId;
emulatorPromise: Promise<unknown>;
shouldAdjustAnimationScale: boolean;
}> {
if (env.EAS_NO_EMULATOR_HOST_SUPPORT_CHECK !== '1') {
await assertAndroidEmulatorHostSupportAsync({ env });
} else {
logger.info(
'Skipping Android emulator host support check because $EAS_NO_EMULATOR_HOST_SUPPORT_CHECK is enabled.'
);
}
try {
const availableDevices = await AndroidEmulatorUtils.getAvailableDevicesAsync({ env });
logger.info(`Available Android devices:\n- ${availableDevices.join(`\n- `)}`);
} catch (error) {
logger.info('Failed to list available Android devices.', error);
} finally {
logger.info('');
}
const shouldAdjustAnimationScale =
env.ANDROID_EMULATOR_ADJUST_ANIMATION_SCALE !== 'false' &&
env.ANDROID_EMULATOR_ADJUST_ANIMATION_SCALE !== '0';
if (!shouldAdjustAnimationScale) {
logger.info(
'Skipping Android emulator animation scale adjustments because $ANDROID_EMULATOR_ADJUST_ANIMATION_SCALE is disabled.'
);
}
const localEgress = await resolveAndroidEmulatorLocalEgressAsync();
logger.info('Making sure system image is installed');
await retryAsync(
async () => {
await spawn('sdkmanager', [systemImagePackage], {
env,
logger,
});
},
{
logger,
retryOptions: {
retries: 3, // Retry 3 times
retryIntervalMs: 1_000,
},
}
);
let emulatorPromise = null;
let serialId = null;
await retryAsync(
async attemptCount => {
const timeoutMs = ANDROID_STARTUP_ATTEMPT_TIMEOUT_MS[attemptCount];
const attempt = attemptCount + 1;
const maxAttempts = ANDROID_STARTUP_ATTEMPT_TIMEOUT_MS.length;
const attemptSuffix = attempt > 1 ? ` (attempt ${attempt}/${maxAttempts})` : '';
let attemptSerialId = null;
try {
logger.info(`Creating emulator device${attemptSuffix}.`);
await AndroidEmulatorUtils.createAsync({
deviceName,
systemImagePackage,
deviceIdentifier: deviceIdentifier ?? null,
lcdWidth: lcdWidth ?? null,
lcdHeight: lcdHeight ?? null,
lcdDensity: lcdDensity ?? null,
env,
logger,
});
logger.info(`Starting emulator device${attemptSuffix}.`);
const startResult = await AndroidEmulatorUtils.startAsync({
deviceName,
env,
logcatDirectory,
...(localEgress ? { launchGate: localEgress.launchGate } : {}),
});
attemptSerialId = startResult.serialId;
await AndroidEmulatorUtils.waitForReadyAsync({
env,
serialId: attemptSerialId,
timeoutMs,
logger,
...(localEgress ? { networkReadyTarget: localEgress.networkReadyTarget } : {}),
});
if (shouldAdjustAnimationScale) {
await AndroidEmulatorUtils.disableWindowAndTransitionAnimationsAsync({
env,
logger,
serialId: attemptSerialId,
});
}
await localEgress?.configureBootedEmulatorAsync({ serialId: attemptSerialId, logger });
logger.info(`${deviceName} is ready.`);
serialId = attemptSerialId;
emulatorPromise = startResult.emulatorPromise;
} catch (err) {
logger.warn({ err }, `${deviceName} failed to start on attempt ${attempt}/${maxAttempts}.`);
try {
if (attemptSerialId) {
await AndroidEmulatorUtils.deleteAsync({
serialId: attemptSerialId,
deviceName,
env,
});
} else {
await AndroidEmulatorUtils.deleteAsync({
deviceName,
env,
});
}
} catch (cleanupErr) {
logger.warn({ err: cleanupErr }, `Failed to clean up ${deviceName}.`);
}
throw err;
}
},
{
logger,
retryOptions: {
retries: ANDROID_STARTUP_RETRIES_COUNT,
retryIntervalMs: 1_000,
},
}
);
if (!serialId || !emulatorPromise) {
throw new Error(`Failed to start emulator ${deviceName}.`);
}
return { serialId, emulatorPromise, shouldAdjustAnimationScale };
}
async function assertAndroidEmulatorHostSupportAsync({
env,
}: {
@@ -19,18 +19,31 @@ import {
uploadRemoteSessionConfigWithLocalEgressAsync,
withLocalEgressSession,
} from '../utils/localEgressSession';
import { type DeviceSessionHost, startDeviceSessionHostAsync } from '../utils/deviceSessionHost';
import {
createNetworkCaptureInputProviders,
parseNetworkCaptureInputs,
} from '../utils/networkCaptureFields';
import { AndroidEmulatorUtils } from '../../utils/AndroidEmulatorUtils';
import { IosSimulatorUtils } from '../../utils/IosSimulatorUtils';
import {
PackageManager,
resolveConfiguredPackageManager,
resolvePackageAdd,
} from '../../utils/packageManager';
import { sleepAsync } from '../../utils/retry';
import { turtleFetch } from '../../utils/turtleFetch';
import { startAppiumEventCollectionAsync } from '../utils/appiumEvents';
import {
createServeSimLaunchInputProviders,
describeServeSimLaunch,
finishRemoteSessionAsync,
getDeviceRunSessionIdOrThrow,
getNgrokAuthtokenOrThrow,
getNgrokTunnelDomainOrThrow,
parseServeSimLaunchInputs,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
@@ -54,6 +67,8 @@ export function createStartAppiumRemoteSessionBuildFunction(
name: 'Start Appium remote session',
__metricsId: 'eas/start_appium_remote_session',
inputProviders: [
...createServeSimLaunchInputProviders(),
...createNetworkCaptureInputProviders(),
BuildStepInput.createProvider({
id: 'package_version',
required: false,
@@ -72,6 +87,21 @@ export function createStartAppiumRemoteSessionBuildFunction(
const packageVersion = inputs.package_version.value as string | undefined;
const maxIdleTimeMinutes = inputs.max_idle_time_minutes.value as number | undefined;
const { runtimePlatform } = global;
const launch = parseServeSimLaunchInputs(
{
launchAppIdentifier: inputs.launch_app_identifier?.value,
launchArgs: inputs.launch_args?.value,
openUrl: inputs.open_url?.value,
},
{ runtimePlatform }
);
const { networkCapture, networkCaptureFields } = parseNetworkCaptureInputs(
{
networkCapture: inputs.network_capture?.value,
networkCaptureFields: inputs.network_capture_fields?.value,
},
{ runtimePlatform }
);
const versionSpec = resolveAppium3VersionSpec(packageVersion);
logger.info(
@@ -121,7 +151,8 @@ export function createStartAppiumRemoteSessionBuildFunction(
logger,
});
let appiumTunnel: Awaited<ReturnType<typeof startNgrokTunnelAsync>> | undefined;
let webPreview: Awaited<ReturnType<typeof startDeviceWebPreviewWithTunnelAsync>> | undefined;
let sessionHost: DeviceSessionHost | undefined;
let sessionFailed = false;
try {
appiumTunnel = await startNgrokTunnelAsync({
port: APPIUM_PORT,
@@ -133,13 +164,22 @@ export function createStartAppiumRemoteSessionBuildFunction(
// expo-device-hub has no serial-selection flag. Device run session workflows must expose
// a single booted Android emulator so the Hub and Appium resolve the same device.
webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
const launchDescription = describeServeSimLaunch(launch);
if (launchDescription) {
logger.info(launchDescription);
}
sessionHost = await startDeviceSessionHostAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: APPIUM_STARTUP_TIMEOUT_MS,
launchAppIdentifier: launch.launchAppIdentifier,
launchArgs: launch.launchArgs,
openUrl: launch.openUrl,
networkCapture,
networkCaptureFields,
});
const webPreview = await sessionHost.openPreviewAsync({ baseDomain: ngrokTunnelDomain });
await uploadRemoteSessionConfigWithLocalEgressAsync({
env,
@@ -153,7 +193,8 @@ export function createStartAppiumRemoteSessionBuildFunction(
'appium:automationName': device.automationName,
'appium:udid': device.udid,
},
webPreviewUrl: webPreview.previewUrl,
webPreviewUrl: webPreview.previewPageUrl,
previewApiUrl: webPreview.apiUrl,
...(webPreview.previewToken ? { webPreviewToken: webPreview.previewToken } : {}),
},
logger,
@@ -172,16 +213,29 @@ export function createStartAppiumRemoteSessionBuildFunction(
}
: undefined,
});
} catch (error) {
sessionFailed = true;
throw error;
} finally {
if (webPreview) {
await webPreview.stopAsync();
}
if (appiumTunnel) {
await appiumTunnel.stopAsync();
}
await eventCollection.stopAsync();
await appiumProcess.stopAsync();
await fs.promises.rm(appiumHome, { recursive: true, force: true });
await finishRemoteSessionAsync({
logger,
sessionFailed,
teardown: [
['Appium tunnel', appiumTunnel?.stopAsync()],
[
'Appium server',
(async () => {
try {
await eventCollection.stopAsync();
} finally {
await appiumProcess.stopAsync();
await fs.promises.rm(appiumHome, { recursive: true, force: true });
}
})(),
],
['session host', sessionHost?.finishAsync()],
],
});
}
}),
});
@@ -198,7 +252,7 @@ export function resolveAppium3VersionSpec(packageVersion: string | undefined): s
return versionSpec;
}
type AppiumDevice = {
export type AppiumDevice = {
platformName: 'iOS' | 'Android';
automationName: 'XCUITest' | 'UiAutomator2';
driverName: 'xcuitest' | 'uiautomator2';
@@ -247,7 +301,7 @@ export async function resolveAppiumDeviceAsync({
}
}
async function installAppiumAsync({
export async function installAppiumAsync({
versionSpec,
driverName,
env,
@@ -265,9 +319,14 @@ async function installAppiumAsync({
);
const appiumEnv: BuildStepEnv = { ...env, APPIUM_HOME: appiumHome };
const appiumBinPath = path.join(appiumHome, 'node_modules', '.bin', 'appium');
const add = resolvePackageAdd(
resolveConfiguredPackageManager(env, PackageManager.NPM),
`appium@${versionSpec}`
);
logger.info(`Installing appium@${versionSpec}.`);
await spawn('npm', ['install', '--prefix', appiumHome, `appium@${versionSpec}`], {
logger.info(`Installing appium@${versionSpec} with ${add.command}.`);
await spawn(add.command, add.args, {
cwd: appiumHome,
env: appiumEnv,
logger,
});
@@ -18,19 +18,32 @@ import {
uploadRemoteSessionConfigWithLocalEgressAsync,
withLocalEgressSession,
} from '../utils/localEgressSession';
import { type DeviceSessionHost, startDeviceSessionHostAsync } from '../utils/deviceSessionHost';
import {
createNetworkCaptureInputProviders,
parseNetworkCaptureInputs,
} from '../utils/networkCaptureFields';
import { Sentry } from '../../sentry';
import {
PackageManager,
resolveConfiguredPackageManager,
resolvePackageExec,
} from '../../utils/packageManager';
import { isProcessDescendantOfAsync } from '../../utils/processes';
import { sleepAsync } from '../../utils/retry';
import { pollArgentArtifactsForUploadAsync } from '../utils/argentArtifacts';
import { ARGENT_EVENT_LOG_FILENAME, startArgentEventCollectionAsync } from '../utils/argentEvents';
import {
createServeSimLaunchInputProviders,
describeServeSimLaunch,
ensureFfmpegInstalledOnceAsync,
finishRemoteSessionAsync,
getDeviceRunSessionIdOrThrow,
getNgrokAuthtokenOrThrow,
getNgrokTunnelDomainOrThrow,
parseServeSimLaunchInputs,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
@@ -66,6 +79,8 @@ export function createStartArgentRemoteSessionBuildFunction(
name: 'Start argent remote session',
__metricsId: 'eas/start_argent_remote_session',
inputProviders: [
...createServeSimLaunchInputProviders(),
...createNetworkCaptureInputProviders(),
BuildStepInput.createProvider({
id: 'package_version',
required: false,
@@ -98,6 +113,21 @@ export function createStartArgentRemoteSessionBuildFunction(
warnIfArgentPackageVersionCannotBeVerified({ packageVersion, logger });
const versionSpec = packageVersion ?? 'latest';
const { runtimePlatform } = global;
const launch = parseServeSimLaunchInputs(
{
launchAppIdentifier: inputs.launch_app_identifier?.value as string | undefined,
launchArgs: inputs.launch_args?.value,
openUrl: inputs.open_url?.value as string | undefined,
},
{ runtimePlatform }
);
const { networkCapture, networkCaptureFields } = parseNetworkCaptureInputs(
{
networkCapture: inputs.network_capture?.value,
networkCaptureFields: inputs.network_capture_fields?.value,
},
{ runtimePlatform }
);
logger.info(
`Starting argent remote session (version: ${versionSpec}, runtime: ${runtimePlatform}).`
);
@@ -113,42 +143,45 @@ export function createStartArgentRemoteSessionBuildFunction(
// Never rejects, so `void` is safe.
void ensureFfmpegInstalledOnceAsync({ runtimePlatform, env, logger });
const packageManager = resolveConfiguredPackageManager(env, PackageManager.BUN);
const argentExec = (args: string[]): { command: string; args: string[] } =>
resolvePackageExec(packageManager, args);
logger.info('Enabling the Argent artifacts list endpoint flag.');
await spawn(
'bun',
[
'x',
`${ARGENT_PACKAGE_NAME}@${versionSpec}`,
'enable',
ARGENT_ARTIFACTS_LIST_ENDPOINT_FLAG,
],
{ env, logger }
);
const enableArtifacts = argentExec([
`${ARGENT_PACKAGE_NAME}@${versionSpec}`,
'enable',
ARGENT_ARTIFACTS_LIST_ENDPOINT_FLAG,
]);
await spawn(enableArtifacts.command, enableArtifacts.args, { env, logger });
logger.info('Enabling the Argent tool-server event log flag.');
await spawn(
'bun',
['x', `${ARGENT_PACKAGE_NAME}@${versionSpec}`, 'enable', ARGENT_EVENT_LOG_FLAG],
{ env, logger }
);
const enableEventLog = argentExec([
`${ARGENT_PACKAGE_NAME}@${versionSpec}`,
'enable',
ARGENT_EVENT_LOG_FLAG,
]);
await spawn(enableEventLog.command, enableEventLog.args, { env, logger });
logger.info(`Launching ${ARGENT_PACKAGE_NAME}@${versionSpec} tool-server via bun x.`);
// Keep Argent itself in foreground mode under the detached bun process. This preserves
// the bun -> Argent CLI -> tool-server ancestry used to identify the matching state file.
const startServer = argentExec([
`${ARGENT_PACKAGE_NAME}@${versionSpec}`,
'server',
'start',
'--port',
'0',
'--idle-timeout',
'0',
'--force',
]);
logger.info(
`Launching ${ARGENT_PACKAGE_NAME}@${versionSpec} tool-server via ${startServer.command}.`
);
// Keep Argent itself in foreground mode under the detached process. This preserves
// the npx/bun -> Argent CLI -> tool-server ancestry used to identify the matching state file.
const argentServer = spawnDetached({
command: 'bun',
args: [
'x',
`${ARGENT_PACKAGE_NAME}@${versionSpec}`,
'server',
'start',
'--port',
'0',
'--idle-timeout',
'0',
'--force',
],
env: { ...env, ARGENT_EVENT_LOG: ARGENT_EVENT_LOG_PATH },
command: startServer.command,
args: startServer.args,
env: { ...env, ARGENT_EVENT_LOG: ARGENT_EVENT_LOG_PATH, ARGENT_EMULATOR_NO_WINDOW: '1' },
});
if (argentServer.pid === undefined) {
throw new SystemError(
@@ -164,6 +197,7 @@ export function createStartArgentRemoteSessionBuildFunction(
stateDir: ARGENT_STATE_DIR,
ancestorPid: argentServer.pid,
timeoutMs: STARTUP_TIMEOUT_MS,
getExitError: argentServer.getExitError,
});
toolServerPort = toolServerState.port;
toolServerToken = toolServerState.token;
@@ -196,7 +230,8 @@ export function createStartArgentRemoteSessionBuildFunction(
});
let toolsTunnel: Awaited<ReturnType<typeof startNgrokTunnelAsync>> | undefined;
let webPreview: Awaited<ReturnType<typeof startDeviceWebPreviewWithTunnelAsync>> | undefined;
let sessionHost: DeviceSessionHost | undefined;
let sessionFailed = false;
try {
toolsTunnel = await startNgrokTunnelAsync({
port: toolServerPort,
@@ -209,14 +244,25 @@ export function createStartArgentRemoteSessionBuildFunction(
const publicToolsUrl = toolsTunnel.url;
logger.info(`Tunnel is ready at ${publicToolsUrl}.`);
webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
const launchDescription = describeServeSimLaunch(launch);
if (launchDescription) {
logger.info(launchDescription);
}
sessionHost = await startDeviceSessionHostAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
launchAppIdentifier: launch.launchAppIdentifier,
launchArgs: launch.launchArgs,
openUrl: launch.openUrl,
networkCapture,
networkCaptureFields,
});
logger.info(`Web preview URL: ${webPreview.previewUrl}`);
const webPreview = await sessionHost.openPreviewAsync({ baseDomain: ngrokTunnelDomain });
logger.info(
`Web preview URL: ${webPreview.previewPageUrl} (server: ${webPreview.apiUrl}).`
);
await uploadRemoteSessionConfigWithLocalEgressAsync({
env,
@@ -226,7 +272,8 @@ export function createStartArgentRemoteSessionBuildFunction(
remoteConfig: {
toolsUrl: publicToolsUrl,
...(toolServerToken ? { toolsAuthToken: toolServerToken } : {}),
webPreviewUrl: webPreview.previewUrl,
webPreviewUrl: webPreview.previewPageUrl,
previewApiUrl: webPreview.apiUrl,
...(webPreview.previewToken ? { webPreviewToken: webPreview.previewToken } : {}),
},
logger,
@@ -246,23 +293,46 @@ export function createStartArgentRemoteSessionBuildFunction(
}
: undefined,
});
} catch (error) {
sessionFailed = true;
throw error;
} finally {
if (webPreview) {
await webPreview.stopAsync();
}
if (toolsTunnel) {
await toolsTunnel.stopAsync();
}
await stopArgentEventCollectionSafelyAsync({ eventCollection, deviceRunSessionId, logger });
artifactPollAbortController.abort();
try {
await artifactPollingPromise;
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not finish Argent remote session artifact polling', error);
logger.warn({ err: error }, 'Could not finish Argent remote session artifact polling.');
}
await argentServer.stopAsync();
await finishRemoteSessionAsync({
logger,
sessionFailed,
teardown: [
['Argent tunnel', toolsTunnel?.stopAsync()],
[
'Argent tool-server',
(async () => {
try {
await stopArgentEventCollectionSafelyAsync({
eventCollection,
deviceRunSessionId,
logger,
});
artifactPollAbortController.abort();
try {
await artifactPollingPromise;
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture(
'Could not finish Argent remote session artifact polling',
error
);
logger.warn(
{ err: error },
'Could not finish Argent remote session artifact polling.'
);
}
} finally {
await argentServer.stopAsync();
}
})(),
],
['session host', sessionHost?.finishAsync()],
],
});
}
}),
});
@@ -306,7 +376,7 @@ export function warnIfArgentPackageVersionCannotBeVerified({
logger.warn(
`Argent remote simulator sessions require ${ARGENT_PACKAGE_NAME}@${MIN_ARGENT_REMOTE_SESSION_VERSION} or newer, ` +
`but package_version "${packageVersion}" is not an exact semver version that EAS can verify. ` +
`Continuing and letting bun x resolve it.`
`Continuing and letting the package manager resolve it.`
);
return;
}
@@ -341,16 +411,26 @@ export async function waitForArgentToolServerStateAsync({
ancestorPid,
timeoutMs,
pollIntervalMs = 1_000,
getExitError,
}: {
stateDir: string;
ancestorPid: number;
timeoutMs: number;
pollIntervalMs?: number;
getExitError?: () => Error | undefined;
}): Promise<ArgentToolServerState> {
const deadline = Date.now() + timeoutMs;
let lastError: unknown;
const throwIfExited = (): void => {
const error = getExitError?.();
if (error) {
throw new SystemError(`Argent exited before becoming ready: ${error.message}`);
}
};
while (Date.now() < deadline) {
throwIfExited();
let matchingState: ArgentToolServerState | undefined;
try {
const stateFileNames = (await fs.promises.readdir(stateDir)).filter(
name => name.startsWith('tool-server') && name.endsWith('.json')
@@ -361,7 +441,8 @@ export async function waitForArgentToolServerStateAsync({
await fs.promises.readFile(path.join(stateDir, stateFileName), 'utf8')
);
if (await isProcessDescendantOfAsync(state.pid, ancestorPid)) {
return state;
matchingState = state;
break;
}
} catch (err) {
if ((err as NodeJS.ErrnoException).code !== 'ENOENT' && !(err instanceof SystemError)) {
@@ -372,6 +453,11 @@ export async function waitForArgentToolServerStateAsync({
} catch (err) {
lastError = err;
}
// Keep process failures outside the retryable state-file read errors.
throwIfExited();
if (matchingState) {
return matchingState;
}
await sleepAsync(pollIntervalMs);
}
@@ -4,9 +4,17 @@ import {
BuildStepInput,
BuildStepInputValueTypeName,
} from '@expo/steps';
import { type bunyan } from '@expo/logger';
import spawn from '@expo/turtle-spawn';
import { minBy } from 'lodash';
import { configureSimulatorProxyEnvironmentAsync } from '../utils/localEgress';
import {
installLocalEgressGuardAsync,
resolveLocalEgressBootEnvironmentAsync,
verifyLocalEgressGuardAsync,
} from '../utils/localEgressGuard';
import {
IosSimulatorName,
IosSimulatorUtils,
@@ -39,58 +47,18 @@ export function createStartIosSimulatorBuildFunction(): BuildFunction {
}),
],
fn: async ({ logger }, { inputs, env }) => {
try {
const availableDevices = await IosSimulatorUtils.getAvailableDevicesAsync({
env,
filter: 'available',
});
logger.info(
`Available Simulator devices:\n- ${availableDevices
.map(device => device.displayName)
.join(`\n- `)}`
);
} catch (error) {
logger.info('Failed to list available Simulator devices.', error);
} finally {
logger.info('');
}
const deviceIdentifierInput = inputs.device_identifier.value?.toString() as
| IosSimulatorUuid
| IosSimulatorName
| undefined;
const originalDeviceIdentifier =
deviceIdentifierInput ?? (await findMostGenericIphoneUuidAsync({ env }));
const enableAccessibilitySettings = Boolean(inputs.enable_accessibility_settings.value);
if (!originalDeviceIdentifier) {
throw new Error('Could not find an iPhone among available simulator devices.');
}
if (enableAccessibilitySettings) {
await IosSimulatorUtils.enableAccessibilitySettingsAsync({
deviceIdentifier: originalDeviceIdentifier,
const { deviceIdentifier: originalDeviceIdentifier, displayName: formattedDevice } =
await bootIosSimulatorAsync({
deviceIdentifier: deviceIdentifierInput,
enableAccessibilitySettings,
env,
logger,
});
}
const { udid } = await IosSimulatorUtils.startAsync({
deviceIdentifier: originalDeviceIdentifier,
env,
});
try {
await IosSimulatorUtils.disableApsdAsync({ udid, env });
} catch (err) {
logger.warn({ err }, 'Failed to disable apsd in the Simulator.');
}
await IosSimulatorUtils.waitForReadyAsync({ udid, env });
logger.info('');
const device = await IosSimulatorUtils.getDeviceAsync({ udid, env });
const formattedDevice = device?.displayName ?? originalDeviceIdentifier;
logger.info(`${formattedDevice} is ready.`);
const count = Number(inputs.count.value ?? 1);
if (count > 1) {
@@ -116,9 +84,10 @@ export function createStartIosSimulatorBuildFunction(): BuildFunction {
env,
});
}
const { udid: cloneUdid } = await IosSimulatorUtils.startAsync({
const cloneUdid = await bootWithLocalEgressAsync({
deviceIdentifier: cloneDeviceName,
env,
logger,
});
try {
@@ -140,6 +109,104 @@ export function createStartIosSimulatorBuildFunction(): BuildFunction {
});
}
/**
* Boots one iOS Simulator and waits until it is ready: the requested device, or the
* most generic iPhone when none is given.
*/
export async function bootIosSimulatorAsync({
deviceIdentifier: deviceIdentifierInput,
enableAccessibilitySettings = false,
env,
logger,
}: {
deviceIdentifier?: IosSimulatorUuid | IosSimulatorName;
enableAccessibilitySettings?: boolean;
env: BuildStepEnv;
logger: bunyan;
}): Promise<{
deviceIdentifier: IosSimulatorUuid | IosSimulatorName;
udid: IosSimulatorUuid;
displayName: string;
}> {
try {
const availableDevices = await IosSimulatorUtils.getAvailableDevicesAsync({
env,
filter: 'available',
});
logger.info(
`Available Simulator devices:\n- ${availableDevices
.map(device => device.displayName)
.join(`\n- `)}`
);
} catch (error) {
logger.info('Failed to list available Simulator devices.', error);
} finally {
logger.info('');
}
const deviceIdentifier = deviceIdentifierInput ?? (await findMostGenericIphoneUuidAsync({ env }));
if (!deviceIdentifier) {
throw new Error('Could not find an iPhone among available simulator devices.');
}
if (enableAccessibilitySettings) {
await IosSimulatorUtils.enableAccessibilitySettingsAsync({ deviceIdentifier, env });
}
const udid = await bootWithLocalEgressAsync({ deviceIdentifier, env, logger });
try {
await IosSimulatorUtils.disableApsdAsync({ udid, env });
} catch (err) {
logger.warn({ err }, 'Failed to disable apsd in the Simulator.');
}
await IosSimulatorUtils.waitForReadyAsync({ udid, env });
logger.info('');
const device = await IosSimulatorUtils.getDeviceAsync({ udid, env });
const displayName = device?.displayName ?? deviceIdentifier;
logger.info(`${displayName} is ready.`);
return { deviceIdentifier, udid, displayName };
}
/**
* Boot a device with the local egress environment in place before anything
* inside it starts. `simctl boot` returns once the simulator's launchd is up
* and before it has spawned anything else, which is the only moment at which
* launchd environment reaches every process of the boot. The proxy variables
* and the guard are set in that gap; then the boot is waited for and the
* guard is verified in a fresh process. Nothing here applies when no local
* egress session is active.
*/
async function bootWithLocalEgressAsync({
deviceIdentifier,
env,
logger,
}: {
deviceIdentifier: IosSimulatorUuid | IosSimulatorName;
env: BuildStepEnv;
logger: bunyan;
}): Promise<IosSimulatorUuid> {
const udid = await IosSimulatorUtils.resolveUdidAsync({ deviceIdentifier, env });
// The guard and proxy variables ride the boot itself, so launchd has them
// before it spawns its first process.
const launchdEnvironment = await resolveLocalEgressBootEnvironmentAsync();
await IosSimulatorUtils.bootAsync({
deviceIdentifier: udid,
env,
launchdEnvironment: launchdEnvironment ?? {},
});
// Also set them through launchctl, for a device that was already booted.
const guardInstalled = await installLocalEgressGuardAsync({ udid, env, logger });
await configureSimulatorProxyEnvironmentAsync({ udid, env, logger });
await IosSimulatorUtils.startAsync({ deviceIdentifier: udid, env });
if (guardInstalled) {
await verifyLocalEgressGuardAsync({ udid, env, logger });
}
return udid;
}
async function findMostGenericIphoneUuidAsync({
env,
}: {
@@ -4,6 +4,7 @@ import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { startAndroidLocalEgressAsync } from '../utils/androidLocalEgress';
import {
CHISEL_VERSION,
LOCAL_EGRESS_HANDOFF_PATH,
@@ -17,6 +18,7 @@ import {
stopLocalEgressResourcesAsync,
writeLocalEgressHandoffAsync,
} from '../utils/localEgress';
import { stopLocalEgressGuardRelaysAsync } from '../utils/localEgressGuard';
import {
type DetachedProcessHandle,
type NgrokTunnelHandle,
@@ -75,10 +77,12 @@ function awaitLocalEgressAcquisitionAsync<T>(
/**
* Points the device host's system HTTP(S) proxy at the EAS CLI's egress client.
* Must run before `eas/start_ios_simulator`: the simulator reads the system proxy
* at boot. Nothing else on the host changes, so requests from libraries that
* bypass the system proxy are not covered; the shared session monitor reports
* them. The shared session cleanup releases the resources started here when
* the session ends, with a job finalizer as a fallback.
* at boot. Once a simulator boots, `eas/start_ios_simulator` also sets proxy
* environment variables inside it for clients that read them (gRPC, libcurl)
* and installs the local egress guard, which refuses connections that ignore
* both. On Linux it starts the Android emulator relay and fence instead; see
* androidLocalEgress.ts. The shared session cleanup releases the resources
* started here when the session ends, with a job finalizer as a fallback.
*/
export function createStartLocalEgressBuildFunction(): BuildFunction {
return new BuildFunction({
@@ -86,13 +90,15 @@ export function createStartLocalEgressBuildFunction(): BuildFunction {
id: 'start_local_egress',
name: 'Start local egress',
__metricsId: 'eas/start_local_egress',
supportedRuntimePlatforms: [BuildRuntimePlatform.DARWIN],
fn: async ({ logger }, { env, signal }) => {
supportedRuntimePlatforms: [BuildRuntimePlatform.DARWIN, BuildRuntimePlatform.LINUX],
fn: async ({ logger, global }, { env, signal }) => {
const ngrokTunnelDomain = getNgrokTunnelDomainOrThrow(env);
const ngrokAuthtoken = getNgrokAuthtokenOrThrow(env);
const isAndroid = global?.runtimePlatform === BuildRuntimePlatform.LINUX;
let workDir: string | undefined;
let server: DetachedProcessHandle | undefined;
let tunnel: NgrokTunnelHandle | undefined;
let android: { stopAsync: () => Promise<void> } | undefined;
let finishSetup!: () => void;
const setupFinished = new Promise<void>(resolve => {
finishSetup = resolve;
@@ -102,6 +108,8 @@ export function createStartLocalEgressBuildFunction(): BuildFunction {
const lifetimeSignal = registerLocalEgressResources(async () => {
await setupFinished;
const results = await Promise.allSettled([
Promise.resolve().then(() => stopLocalEgressGuardRelaysAsync(logger)),
Promise.resolve().then(() => android?.stopAsync()),
Promise.resolve().then(() => tunnel?.stopAsync()),
Promise.resolve().then(() => server?.stopAsync()),
]);
@@ -166,6 +174,35 @@ export function createStartLocalEgressBuildFunction(): BuildFunction {
);
startupSignal.throwIfAborted();
if (isAndroid) {
if (server.pid === undefined) {
throw new Error('The reverse tunnel server has no process id.');
}
android = await startAndroidLocalEgressAsync({
env,
logger,
workDir,
chiselPid: server.pid,
proxyPort: LOCAL_EGRESS_PROXY_PORT,
controlPort,
});
startupSignal.throwIfAborted();
await writeLocalEgressHandoffAsync({
url: tunnel.url,
token: credentials.password,
fingerprint: started.fingerprint,
port: LOCAL_EGRESS_PROXY_PORT,
platform: 'android',
});
startupSignal.throwIfAborted();
logger.info(
"Local egress is configured for the Android emulator. Its TCP traffic goes through the emulator's " +
'proxy and fails until the EAS CLI egress client connects, then exits from that machine. ' +
'Traffic that cannot use the tunnel, such as UDP, is refused and reported here.'
);
return;
}
const { service } = await configureSystemProxyAsync({
env,
logger,
@@ -183,9 +220,10 @@ export function createStartLocalEgressBuildFunction(): BuildFunction {
logger.info(
`Local egress is configured on network service "${service}". HTTP(S) and WebSocket ` +
'requests that honor the system proxy (WebKit, URLSession and other CFNetwork clients) ' +
'fail until the EAS CLI egress client connects, then exit from that machine. Requests ' +
'from libraries that bypass the system proxy are not covered and exit from this worker. ' +
'Connection sampling may report these bypasses, but can miss short connections and unconnected UDP traffic.'
'fail until the EAS CLI egress client connects, then exit from that machine. Once the ' +
'Simulator boots, proxy environment variables are set inside it for clients that read ' +
'them (gRPC, libcurl), and the local egress guard is installed so that connections which ' +
'ignore both are refused in the process that makes them and reported here.'
);
} catch (error) {
finishSetup();
@@ -3,6 +3,7 @@ import { BuildFunction, BuildStepInput, BuildStepInputValueTypeName } from '@exp
import { graphql } from 'gql.tada';
import { CustomBuildContext } from '../../customBuildContext';
import { graphqlAbortContext } from '../../utils/graphqlAbort';
import { withLogPhaseAsync } from '../../utils/logPhase';
import { startSandboxDaemonAsync } from '../utils/sandboxDaemon';
@@ -32,8 +33,9 @@ export function createStartSandboxBuildFunction(ctx: CustomBuildContext): BuildF
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
],
fn: async (stepCtx, { inputs, signal }) => {
const sandboxToken = ctx.env.__EAS_SANDBOX_MCP_TOKEN;
fn: async (stepCtx, { inputs, signal, env }) => {
// The daemon needs this credential, but shell commands do not.
const { __EAS_SANDBOX_MCP_TOKEN: sandboxToken, ...commandEnv } = env;
if (!sandboxToken) {
throw new SystemError('__EAS_SANDBOX_MCP_TOKEN is required to start the sandbox daemon.');
}
@@ -48,6 +50,8 @@ export function createStartSandboxBuildFunction(ctx: CustomBuildContext): BuildF
reconnectDelayMs: RECONNECT_DELAY_MS,
logger: stepCtx.logger,
signal,
workingDirectory: stepCtx.workingDirectory,
env: commandEnv,
});
try {
await daemon.ready;
@@ -73,19 +77,7 @@ export async function markSandboxReadyAsync(
): Promise<void> {
signal?.throwIfAborted();
const result = await ctx.graphqlClient
.mutation(
MARK_SANDBOX_READY_MUTATION,
{ sandboxId },
signal
? {
fetch: (input, init) =>
fetch(input, {
...init,
signal: init?.signal ? AbortSignal.any([signal, init.signal]) : signal,
}),
}
: undefined
)
.mutation(MARK_SANDBOX_READY_MUTATION, { sandboxId }, graphqlAbortContext(signal))
.toPromise();
signal?.throwIfAborted();
if (result.error) {
@@ -0,0 +1,26 @@
import { BuildFunction, BuildRuntimePlatform } from '@expo/steps';
import { Sentry } from '../../sentry';
import { ServeSimLogsRecorder } from '../utils/serveSimLogsRecorder';
export function createStartServeSimLogsBuildFunction(): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'start_serve_sim_logs',
name: 'Start serve-sim simulator logs',
__metricsId: 'eas/start_serve_sim_logs',
supportedRuntimePlatforms: [BuildRuntimePlatform.DARWIN],
fn: async ({ logger }) => {
try {
await ServeSimLogsRecorder.startAsync({ logger });
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not start serve-sim simulator logs', error);
logger.warn(
{ err: error },
'Could not start simulator log collection; the session will continue.'
);
}
},
});
}
@@ -6,15 +6,22 @@ import {
} from '@expo/steps';
import { CustomBuildContext } from '../../customBuildContext';
import { startDeviceSessionHostAsync } from '../utils/deviceSessionHost';
import {
createNetworkCaptureInputProviders,
parseNetworkCaptureInputs,
} from '../utils/networkCaptureFields';
import {
uploadRemoteSessionConfigWithLocalEgressAsync,
withLocalEgressSession,
} from '../utils/localEgressSession';
import {
createServeSimLaunchInputProviders,
describeServeSimLaunch,
getDeviceRunSessionIdOrThrow,
getNgrokTunnelDomainOrThrow,
parseServeSimLaunchInputs,
selectXcodeDeveloperDirectoryAsync,
startDeviceWebPreviewWithTunnelAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
@@ -29,6 +36,8 @@ export function createStartWebPreviewRemoteSessionBuildFunction(
name: 'Start web preview remote session',
__metricsId: 'eas/start_serve_sim_remote_session',
inputProviders: [
...createServeSimLaunchInputProviders(),
...createNetworkCaptureInputProviders(),
BuildStepInput.createProvider({
id: 'package_version',
required: false,
@@ -46,31 +55,56 @@ export function createStartWebPreviewRemoteSessionBuildFunction(
const maxDurationSeconds = inputs.max_duration_seconds?.value as number | undefined;
const packageVersion = inputs.package_version?.value as string | undefined;
const { runtimePlatform } = global;
const launch = parseServeSimLaunchInputs(
{
launchAppIdentifier: inputs.launch_app_identifier?.value as string | undefined,
launchArgs: inputs.launch_args?.value,
openUrl: inputs.open_url?.value as string | undefined,
},
{ runtimePlatform }
);
const { networkCapture, networkCaptureFields } = parseNetworkCaptureInputs(
{
networkCapture: inputs.network_capture?.value,
networkCaptureFields: inputs.network_capture_fields?.value,
},
{ runtimePlatform }
);
logger.info(`Starting web preview remote session (runtime: ${runtimePlatform}).`);
const launchDescription = describeServeSimLaunch(launch);
if (launchDescription) {
logger.info(launchDescription);
}
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
await selectXcodeDeveloperDirectoryAsync({ env, logger });
}
const webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
const sessionHost = await startDeviceSessionHostAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
packageVersion,
launchAppIdentifier: launch.launchAppIdentifier,
launchArgs: launch.launchArgs,
openUrl: launch.openUrl,
networkCapture,
networkCaptureFields,
});
logger.info(`Preview URL: ${webPreview.previewUrl}`);
try {
const webPreview = await sessionHost.openPreviewAsync({ baseDomain: ngrokTunnelDomain });
logger.info(`Preview URL: ${webPreview.previewPageUrl} (server: ${webPreview.apiUrl}).`);
await uploadRemoteSessionConfigWithLocalEgressAsync({
env,
signal,
ctx,
deviceRunSessionId,
remoteConfig: {
previewUrl: webPreview.previewUrl,
previewUrl: webPreview.previewPageUrl,
previewApiUrl: webPreview.apiUrl,
...(webPreview.previewToken ? { previewToken: webPreview.previewToken } : {}),
},
logger,
@@ -84,7 +118,7 @@ export function createStartWebPreviewRemoteSessionBuildFunction(
signal,
});
} finally {
await webPreview.stopAsync();
await sessionHost.finishAsync();
}
}),
});
@@ -0,0 +1,284 @@
import { UserError } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import { BuildFunction, BuildStepInput, BuildStepInputValueTypeName } from '@expo/steps';
import path from 'node:path';
import limitFactory from 'promise-limit';
import { z } from 'zod';
import { AscApiClient, AscApiRequestError } from '../utils/ios/AscApiClient';
import { AscApiUtils } from '../utils/ios/AscApiUtils';
export function createUpdateTestFlightMetadataBuildFunction(): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'update_testflight_metadata',
name: 'Update TestFlight metadata',
__metricsId: 'eas/update_testflight_metadata',
inputProviders: [
BuildStepInput.createProvider({
id: 'asc_api_key_path',
required: true,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
id: 'build_upload_id',
required: true,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
id: 'changelog',
required: false,
defaultValue: '',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
id: 'groups',
required: false,
defaultValue: [],
allowedValueTypeName: BuildStepInputValueTypeName.JSON,
}),
],
fn: async (ctx, { inputs }) => {
const parsedInputs = z
.object({
asc_api_key_path: z.string(),
build_upload_id: z.string(),
changelog: z.string(),
groups: z.array(z.string()),
})
.parse({
asc_api_key_path: inputs.asc_api_key_path.value,
build_upload_id: inputs.build_upload_id.value,
changelog: inputs.changelog.value,
groups: inputs.groups.value,
});
const keyPath = path.resolve(ctx.workingDirectory, parsedInputs.asc_api_key_path);
const key = await AscApiUtils.loadApiKeyAsync({ keyPath });
const client = new AscApiClient({ key, logger: ctx.logger });
await updateTestFlightMetadataAsync({
client,
buildUploadId: parsedInputs.build_upload_id,
changelog: parsedInputs.changelog,
groups: parsedInputs.groups,
logger: ctx.logger,
});
ctx.logger.info('TestFlight metadata updated.');
},
});
}
export async function updateTestFlightMetadataAsync({
client,
buildUploadId,
changelog,
groups,
logger,
}: {
client: AscApiClient;
buildUploadId: string;
changelog: string;
groups: string[];
logger: bunyan;
}): Promise<void> {
const { data: upload } = await client.getAsync(
'/v1/buildUploads/:id',
{
'fields[buildUploads]': ['state', 'build'],
include: ['build'],
},
{ id: buildUploadId }
);
const buildId = upload.relationships?.build?.data?.id;
if (upload.attributes?.state?.state !== 'COMPLETE' || !buildId) {
throw new UserError(
'EAS_TESTFLIGHT_BUILD_NOT_READY',
'The uploaded build is not ready for TestFlight metadata. Run eas/upload_to_asc with wait_for_processing: true first.'
);
}
logger.info(`Updating TestFlight metadata for Apple build ${buildId}...`);
const { data: app } = await client.getAsync('/v1/builds/:id/app', {}, { id: buildId });
const limit = limitFactory<void>(1);
const results = await Promise.allSettled([
limit(async () => {
if (!changelog) {
return;
}
logger.info(`Updating changelog: ${JSON.stringify(changelog)}`);
const primaryLocale = app.attributes?.primaryLocale || 'en-US';
if (!app.attributes?.primaryLocale) {
logger.warn('App Store Connect did not return the primary locale; using "en-US".');
}
logger.info(`Primary locale: ${JSON.stringify(primaryLocale)}.`);
const localizations = await AscApiUtils.getAllBetaBuildLocalizationsAsync({
client,
buildId,
});
const localizationLimit = limitFactory<void>(1);
const updates = localizations.map(localization =>
localizationLimit(async () => {
const label = `${JSON.stringify(localization.attributes?.locale ?? 'unknown')} (${localization.id})`;
try {
await client.patchAsync(
'/v1/betaBuildLocalizations/:id',
{
data: {
type: 'betaBuildLocalizations',
id: localization.id,
attributes: { whatsNew: changelog },
},
},
{ id: localization.id }
);
} catch (error) {
logger.error(`❌ Locale ${label}: changelog update failed. ${String(error)}`);
throw error;
}
logger.info(`✅ Locale ${label}: changelog updated.`);
})
);
if (!localizations.some(localization => localization.attributes?.locale === primaryLocale)) {
updates.push(
localizationLimit(async () => {
try {
await client.postAsync('/v1/betaBuildLocalizations', {
data: {
type: 'betaBuildLocalizations',
attributes: { locale: primaryLocale, whatsNew: changelog },
relationships: { build: { data: { type: 'builds', id: buildId } } },
},
});
} catch (error) {
logger.error(
`❌ Locale ${JSON.stringify(primaryLocale)}: localization creation failed. ${String(error)}`
);
throw error;
}
logger.info(`✅ Locale ${JSON.stringify(primaryLocale)}: localization created.`);
})
);
}
const localizationResults = await Promise.allSettled(updates);
const localizationFailures = localizationResults.filter(
result => result.status === 'rejected'
);
if (localizationFailures.length === 1) {
throw localizationFailures[0].reason;
}
if (localizationFailures.length > 1) {
throw new AggregateError(
localizationFailures.map(failure => failure.reason),
`Failed to update TestFlight localizations: ${localizationFailures.map(failure => String(failure.reason)).join('; ')}`
);
}
}),
limit(async () => {
if (!groups.length) {
return;
}
logger.info(`Apple app: ${app.id}. Requested groups: ${JSON.stringify(groups)}.`);
const requestedNames = new Set(groups);
const allGroups = await AscApiUtils.getAllBetaGroupsAsync({ client, appId: app.id });
const requestedGroups = allGroups.filter(group =>
requestedNames.has(group.attributes?.name ?? '')
);
const foundNames = new Set(requestedGroups.map(group => group.attributes?.name));
const missingNames = [...requestedNames].filter(name => !foundNames.has(name));
logger.info(`Found ${requestedGroups.length} TestFlight group(s).`);
const assignedGroups = requestedGroups.length
? await AscApiUtils.getAllBetaGroupsAsync({
client,
buildId,
})
: [];
const assignedIds = new Set(assignedGroups.map(group => group.id));
const groupLimit = limitFactory<void>(1);
const groupResults = await Promise.allSettled(
requestedGroups.map(group =>
groupLimit(async () => {
const label = `${JSON.stringify(group.attributes?.name)} (${group.id})`;
if (assignedIds.has(group.id)) {
logger.info(`✅ Group ${label}: build already assigned; no assignment needed.`);
return;
}
if (
group.attributes?.isInternalGroup === true &&
group.attributes.hasAccessToAllBuilds === true
) {
logger.info(
`✅ Group ${label}: automatic access to all builds; no assignment needed.`
);
return;
}
try {
await client.postAsync(
'/v1/builds/:id/relationships/betaGroups',
{
data: [{ type: 'betaGroups', id: group.id }],
},
{ id: buildId }
);
} catch (error) {
logger.error(`❌ Group ${label}: assignment failed. ${String(error)}`);
if (isInternalGroupAssignmentError(error)) {
throw new UserError(
'EAS_TESTFLIGHT_INTERNAL_GROUP_ASSIGNMENT_FAILED',
"App Store Connect can't add this build to a requested internal TestFlight group. " +
"Internal groups that automatically receive new builds can't be assigned to manually. " +
'Remove the group from the list, or turn off automatic distribution in App Store Connect. ' +
`Manage groups at https://appstoreconnect.apple.com/apps/${app.id}/testflight`,
{ cause: error }
);
}
throw error;
}
logger.info(`✅ Group ${label}: assignment completed.`);
})
)
);
const groupFailures = groupResults.filter(result => result.status === 'rejected');
if (missingNames.length) {
const error = new UserError(
'EAS_TESTFLIGHT_GROUPS_NOT_FOUND',
`The following TestFlight group${missingNames.length > 1 ? 's were' : ' was'} not found in App Store Connect: ${missingNames.map(name => `"${name}"`).join(', ')}. Check the group names and try again.`
);
logger.error(`❌ ${error.message}`);
groupFailures.push({ status: 'rejected', reason: error });
}
if (groupFailures.length === 1) {
throw groupFailures[0].reason;
}
if (groupFailures.length > 1) {
throw new AggregateError(
groupFailures.map(failure => failure.reason),
`Failed to assign TestFlight groups: ${groupFailures.map(failure => String(failure.reason)).join('; ')}`
);
}
}),
]);
const failures = results.filter(result => result.status === 'rejected');
if (failures.length === 1) {
throw failures[0].reason;
}
if (failures.length > 1) {
throw new AggregateError(
failures.map(failure => failure.reason),
`Failed to update the TestFlight changelog and groups: ${failures
.map(failure => String(failure.reason))
.join('; ')}`
);
}
}
// Apple returns a generic 422 code, so match the title or detail too.
function isInternalGroupAssignmentError(error: unknown): boolean {
if (error instanceof AggregateError) {
return error.errors.some(isInternalGroupAssignmentError);
}
return (
error instanceof AscApiRequestError &&
error.status === 422 &&
error.code === 'ENTITY_UNPROCESSABLE' &&
(error.responseJson.title === 'Builds cannot be assigned to this internal group.' ||
error.responseJson.detail === 'Cannot add internal group to a build.')
);
}
@@ -1,53 +1,17 @@
import { SystemError } from '@expo/eas-build-job';
import {
BuildFunction,
BuildRuntimePlatform,
BuildStepInput,
BuildStepInputValueTypeName,
} from '@expo/steps';
import { createReadStream } from 'node:fs';
import { readFile, stat } from 'node:fs/promises';
import path from 'node:path';
import limitFactory from 'promise-limit';
import { z } from 'zod';
import { type CustomBuildContext } from '../../customBuildContext';
import { Sentry } from '../../sentry';
import { formatBytes } from '../../utils/artifacts';
import { uploadDeviceRunSessionArtifactAsync } from '../utils/deviceRunSessionArtifacts';
import {
parseDeviceScreenRecordings,
uploadDeviceRunSessionScreenRecordingsAsync,
} from '../utils/deviceRunSessionScreenRecordings';
import { getDeviceRunSessionIdOrThrow } from '../utils/remoteDeviceRunSession';
const RecordingsSchema = z.array(
z.object({
udid: z.string(),
deviceName: z.string(),
runtimeDisplayName: z.string(),
directory: z.string(),
})
);
const RecordingManifestSchema = z.object({
firstFrameWallClock: z.object({
iso8601: z.string(),
}),
width: z.number().int().positive(),
height: z.number().int().positive(),
recording: z.string(),
});
const recordingStartTimeFormatter = new Intl.DateTimeFormat('en-US', {
year: 'numeric',
month: 'short',
day: 'numeric',
hour: '2-digit',
minute: '2-digit',
second: '2-digit',
fractionalSecondDigits: 3,
hourCycle: 'h23',
timeZone: 'UTC',
timeZoneName: 'short',
});
export function createUploadDeviceRunSessionScreenRecordingsBuildFunction(
ctx: CustomBuildContext
): BuildFunction {
@@ -65,69 +29,16 @@ export function createUploadDeviceRunSessionScreenRecordingsBuildFunction(
}),
],
fn: async ({ logger }, { env, inputs }) => {
const result = RecordingsSchema.safeParse(inputs.recordings_json.value ?? []);
if (!result.success) {
throw new SystemError('Invalid iOS Simulator recordings input.', {
cause: result.error,
});
}
const recordings = result.data;
const recordings = parseDeviceScreenRecordings(inputs.recordings_json.value ?? []);
if (recordings.length === 0) {
logger.info('No iOS Simulator recordings found; skipping uploads.');
logger.info('No device screen recordings found; skipping uploads.');
return;
}
const deviceRunSessionId = getDeviceRunSessionIdOrThrow(env);
const limit = limitFactory(5);
await Promise.all(
recordings.map(recording =>
limit(async () => {
try {
const metadata = RecordingManifestSchema.parse(
JSON.parse(await readFile(path.join(recording.directory, 'session.json'), 'utf-8'))
);
const startedAt = recordingStartTimeFormatter.format(
new Date(metadata.firstFrameWallClock.iso8601)
);
const shortUdid = `${recording.udid.slice(0, 8)}-…`;
const displayName = `${recording.deviceName} screen recording (${shortUdid}, started at ${startedAt})`;
const recordingPath = path.join(recording.directory, metadata.recording);
const { size } = await stat(recordingPath);
const recordingId = path.basename(recording.directory);
logger.info(
`Uploading screen recording for ${recording.deviceName} (${formatBytes(size)}).`
);
await uploadDeviceRunSessionArtifactAsync(ctx, {
deviceRunSessionId,
artifactId: recordingId,
name: displayName,
filename: `${recordingId}.mp4`,
kind: 'screen-recording',
metadata: {
__eas_type: 'screen-recording',
__eas_screen_recording: '1',
udid: recording.udid,
deviceName: recording.deviceName,
runtimeDisplayName: recording.runtimeDisplayName,
firstFrameAt: metadata.firstFrameWallClock.iso8601,
width: metadata.width,
height: metadata.height,
},
size,
stream: createReadStream(recordingPath),
});
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not upload iOS Simulator screen recording', error);
logger.warn(
{ err: error },
`Could not upload screen recording for ${recording.deviceName}.`
);
}
})
)
);
await uploadDeviceRunSessionScreenRecordingsAsync(ctx, {
logger,
deviceRunSessionId: getDeviceRunSessionIdOrThrow(env),
recordings,
});
},
});
}
@@ -7,7 +7,6 @@ import {
BuildStepOutput,
} from '@expo/steps';
import fs from 'fs-extra';
import * as jose from 'jose';
import fetch from 'node-fetch';
import path from 'node:path';
import { setTimeout } from 'node:timers/promises';
@@ -87,29 +86,8 @@ export function createUploadToAscBuildFunction(): BuildFunction {
throw new Error(`ASC API Key file not found: ${ascApiKeyPath}`);
}
const ascApiKeyJson = await fs.readJson(ascApiKeyPath);
const ascApiKey = z
.object({
issuer_id: z.string().nullish(),
key_id: z.string(),
key: z.string(),
})
.parse(ascApiKeyJson);
const privateKey = await jose.importPKCS8(ascApiKey.key, 'ES256');
const jwt = new jose.SignJWT({})
.setProtectedHeader({ alg: 'ES256', kid: ascApiKey.key_id })
.setAudience('appstoreconnect-v1')
.setExpirationTime('20m');
if (ascApiKey.issuer_id) {
jwt.setIssuer(ascApiKey.issuer_id);
} else {
// Nullish issuer_id means an individual API key
jwt.setSubject('user');
}
const token = await jwt.sign(privateKey);
const client = new AscApiClient({ token, logger: stepsCtx.logger });
const key = await AscApiUtils.loadApiKeyAsync({ keyPath: ascApiKeyPath });
const client = new AscApiClient({ key, logger: stepsCtx.logger });
stepsCtx.logger.info(
`Reading App information for Apple app identifier: ${appleAppIdentifier}...`
@@ -0,0 +1,319 @@
import fs from 'node:fs';
import net from 'node:net';
import os from 'node:os';
import path from 'node:path';
import { createMockLogger } from '../../../__tests__/utils/logger';
import {
buildEgressFenceRuleset,
findProcessesOutsideScope,
getCgroupLevel,
listEmulatorProcessesAsync,
parseAdbReversePorts,
parseConnectAuthority,
parseFenceCounters,
parseFenceLogLine,
parseLoopbackListenerPorts,
resolveAndroidEmulatorLocalEgressAsync,
startLocalEgressRelayAsync,
} from '../androidLocalEgress';
import { writeLocalEgressHandoffAsync } from '../localEgress';
describe(parseConnectAuthority, () => {
it('parses IPv4, hostname and bracketed IPv6 authorities', () => {
expect(parseConnectAuthority('CONNECT 192.0.2.1:443 HTTP/1.1\r\nHost: x\r\n')).toEqual({
host: '192.0.2.1',
port: 443,
});
expect(parseConnectAuthority('CONNECT Example.com:8443 HTTP/1.0\r\n')).toEqual({
host: 'example.com',
port: 8443,
});
expect(parseConnectAuthority('CONNECT [2001:db8::1]:443 HTTP/1.1\r\n')).toEqual({
host: '2001:db8::1',
port: 443,
});
});
it('ignores requests that are not CONNECT', () => {
expect(parseConnectAuthority('GET http://example.com/ HTTP/1.1\r\n')).toBeNull();
});
});
describe(buildEgressFenceRuleset, () => {
it('refuses traffic from the session scope that cannot use the tunnel', () => {
const ruleset = buildEgressFenceRuleset({ cgroupLevel: 2 });
expect(ruleset).toContain('socket cgroupv2 level 2 @fenced jump fence');
expect(ruleset).toContain('oif "lo" meta l4proto tcp accept');
expect(ruleset).toContain('oif "lo" ip daddr 127.0.0.53 udp dport 53 accept');
expect(ruleset).toContain('log prefix "eas-egress " level info');
expect(ruleset).toContain('meta l4proto tcp counter name fence_tcp reject with tcp reset');
expect(ruleset).toContain(
'meta l4proto udp counter name fence_udp reject with icmpx admin-prohibited'
);
expect(ruleset).not.toMatch(/fence_\w+ accept/);
});
});
describe(parseFenceLogLine, () => {
it('parses a refused IPv4 TCP connection from the emulator', () => {
expect(
parseFenceLogLine(
'eas-egress IN= OUT=ens3 SRC=10.128.0.5 DST=172.253.155.190 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=1 DF PROTO=TCP SPT=42408 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0'
)
).toEqual({
process: 'Android emulator',
pid: null,
function: 'tcp',
action: 'blocked',
peer: '172.253.155.190:443',
callers: [],
note: expect.stringContaining('the emulator itself opened'),
});
});
it('labels QUIC and compresses IPv6 destinations', () => {
expect(
parseFenceLogLine(
'eas-egress IN= OUT=ens3 SRC=10.128.0.5 DST=142.250.152.94 LEN=1278 PROTO=UDP SPT=51000 DPT=443 LEN=1258'
)
).toMatchObject({
function: 'udp',
peer: '142.250.152.94:443',
note: expect.stringContaining('QUIC'),
});
expect(
parseFenceLogLine(
'eas-egress IN= OUT=ens3 SRC=fe80:0000:0000:0000:0000:0000:0000:0001 DST=ff02:0000:0000:0000:0000:0000:0000:00fb LEN=100 TC=0 HOPLIMIT=255 FLOWLBL=0 PROTO=UDP SPT=5353 DPT=5353 LEN=60'
)
).toMatchObject({ function: 'udp', peer: '[ff02::fb]:5353', note: undefined });
});
it('reports ICMP without a port', () => {
expect(
parseFenceLogLine(
'eas-egress IN= OUT=ens3 SRC=10.128.0.5 DST=8.8.8.8 LEN=84 PROTO=ICMP TYPE=8 CODE=0 ID=7 SEQ=1'
)
).toMatchObject({ function: 'icmp', action: 'blocked', peer: '8.8.8.8' });
});
it('ignores unrelated kernel log lines', () => {
expect(parseFenceLogLine('kvm: some unrelated message')).toBeNull();
expect(parseFenceLogLine('eas-egress IN= OUT=ens3')).toBeNull();
});
});
describe(parseFenceCounters, () => {
it('reads packet counts from nft JSON output', () => {
expect(
parseFenceCounters(
JSON.stringify({
nftables: [
{ metainfo: { version: '1.1.6' } },
{ counter: { family: 'inet', name: 'fence_tcp', table: 'eas_fence', packets: 4 } },
{ counter: { family: 'inet', name: 'fence_udp', table: 'eas_fence', packets: 57 } },
{ counter: { family: 'inet', name: 'fence_other', table: 'eas_fence', packets: 2 } },
],
})
)
).toEqual({ tcp: 4, udp: 57, other: 2 });
});
});
describe('process coverage', () => {
let procRoot: string;
beforeEach(async () => {
procRoot = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'fake-proc-'));
});
afterEach(async () => {
await fs.promises.rm(procRoot, { recursive: true, force: true });
});
async function addProcessAsync(pid: number, comm: string, cgroup: string): Promise<void> {
await fs.promises.mkdir(path.join(procRoot, String(pid)));
await fs.promises.writeFile(path.join(procRoot, String(pid), 'comm'), `${comm}\n`);
await fs.promises.writeFile(path.join(procRoot, String(pid), 'cgroup'), `0::${cgroup}\n`);
}
it('lists emulator processes and finds those outside the session scope', async () => {
const scope = 'system.slice/eas-egress-session.scope';
await addProcessAsync(10, 'qemu-system-x86', `/${scope}`);
await addProcessAsync(11, 'netsimd', `/${scope}`);
await addProcessAsync(12, 'netsimd', '/system.slice/eas-build-worker.service');
await addProcessAsync(13, 'node', '/system.slice/eas-build-worker.service');
await fs.promises.mkdir(path.join(procRoot, 'self'));
const processes = await listEmulatorProcessesAsync(procRoot);
expect(processes.map(({ pid }) => pid).sort()).toEqual([10, 11, 12]);
expect(findProcessesOutsideScope(processes, scope)).toEqual([
{ pid: 12, comm: 'netsimd', cgroup: '/system.slice/eas-build-worker.service' },
]);
expect(getCgroupLevel(scope)).toBe(2);
});
});
describe('adb reverse parsing', () => {
it('finds loopback ports a process listens on', () => {
const ss = [
'LISTEN 0 4096 127.0.0.1:8899 0.0.0.0:* users:(("chisel",pid=4321,fd=9))',
'LISTEN 0 4096 127.0.0.1:8082 0.0.0.0:* users:(("chisel",pid=4321,fd=10))',
'LISTEN 0 4096 127.0.0.1:5037 0.0.0.0:* users:(("adb",pid=999,fd=3))',
'LISTEN 0 4096 127.0.0.1:52001 0.0.0.0:* users:(("chisel",pid=43210,fd=3))',
].join('\n');
expect(parseLoopbackListenerPorts(ss, 4321)).toEqual([8082, 8899]);
});
it('parses adb reverse --list output', () => {
expect(
parseAdbReversePorts('emulator-5554 tcp:8082 tcp:8082\nemulator-5554 tcp:8083 tcp:8083\n')
).toEqual(new Set([8082, 8083]));
});
});
describe(startLocalEgressRelayAsync, () => {
async function getFreePortAsync(): Promise<number> {
const server = net.createServer();
await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve));
const { port } = server.address() as net.AddressInfo;
await new Promise<void>(resolve => server.close(() => resolve()));
return port;
}
async function exchangeAsync(port: number, request: string): Promise<string> {
return await new Promise((resolve, reject) => {
const socket = net.connect({ host: '127.0.0.1', port });
let received = '';
socket.on('data', chunk => {
received += chunk.toString();
});
socket.on('error', reject);
socket.on('close', () => resolve(received));
socket.write(request);
});
}
async function startUpstreamAsync(
port: number
): Promise<{ requests: string[]; close: () => Promise<void> }> {
const requests: string[] = [];
const server = net.createServer(socket => {
socket.once('data', chunk => {
requests.push(chunk.toString());
socket.end('upstream reply');
});
});
await new Promise<void>(resolve => server.listen(port, '127.0.0.1', resolve));
return {
requests,
close: async () => await new Promise<void>(resolve => server.close(() => resolve())),
};
}
it('answers the readiness CONNECT itself, even without an egress client', async () => {
const logger = createMockLogger();
const relay = await startLocalEgressRelayAsync({
port: await getFreePortAsync(),
upstreamPort: await getFreePortAsync(),
logger,
});
try {
await expect(
exchangeAsync(relay.port, 'CONNECT 192.0.2.1:443 HTTP/1.1\r\nHost: 192.0.2.1:443\r\n\r\n')
).resolves.toBe('HTTP/1.1 200 Connection Established\r\n\r\n');
expect(logger.warn).not.toHaveBeenCalled();
} finally {
await relay.stopAsync();
}
});
it('refuses while the client is detached and forwards once it attaches', async () => {
const logger = createMockLogger();
const upstreamPort = await getFreePortAsync();
const relay = await startLocalEgressRelayAsync({
port: await getFreePortAsync(),
upstreamPort,
logger,
});
const request = 'CONNECT 93.184.216.34:443 HTTP/1.1\r\nHost: 93.184.216.34:443\r\n\r\n';
try {
await expect(exchangeAsync(relay.port, request)).resolves.toMatch(
/^HTTP\/1\.1 503 Service Unavailable\r\n[\s\S]*not connected/
);
await exchangeAsync(relay.port, request);
expect(logger.warn).toHaveBeenCalledTimes(1);
const upstream = await startUpstreamAsync(upstreamPort);
try {
await expect(exchangeAsync(relay.port, request)).resolves.toBe('upstream reply');
expect(upstream.requests).toEqual([request]);
expect(logger.info).toHaveBeenCalledWith(
expect.stringContaining('2 request(s) from the Android emulator were refused')
);
} finally {
await upstream.close();
}
} finally {
await relay.stopAsync();
}
});
it('unwraps a CONNECT to itself and forwards the inner request', async () => {
const upstreamPort = await getFreePortAsync();
const relay = await startLocalEgressRelayAsync({
port: await getFreePortAsync(),
upstreamPort,
logger: createMockLogger(),
});
const upstream = await startUpstreamAsync(upstreamPort);
const inner = 'CONNECT example.com:443 HTTP/1.1\r\nHost: example.com:443\r\n\r\n';
try {
await expect(
exchangeAsync(
relay.port,
`CONNECT 10.0.2.2:${relay.port} HTTP/1.1\r\nHost: 10.0.2.2:${relay.port}\r\n\r\n${inner}`
)
).resolves.toBe('HTTP/1.1 200 Connection Established\r\n\r\nupstream reply');
expect(upstream.requests).toEqual([inner]);
} finally {
await upstream.close();
await relay.stopAsync();
}
});
});
describe(resolveAndroidEmulatorLocalEgressAsync, () => {
let handoffPath: string;
beforeEach(async () => {
handoffPath = path.join(
await fs.promises.mkdtemp(path.join(os.tmpdir(), 'android-egress-handoff-')),
'handoff.json'
);
});
afterEach(async () => {
await fs.promises.rm(path.dirname(handoffPath), { recursive: true, force: true });
});
it('returns null without a local egress session', async () => {
await expect(resolveAndroidEmulatorLocalEgressAsync({ handoffPath })).resolves.toBeNull();
});
it('fails when a session exists but the fence was not started', async () => {
await writeLocalEgressHandoffAsync(
{
url: 'https://egress.example.com',
token: 'token',
fingerprint: 'key=',
port: 8899,
platform: 'android',
},
handoffPath
);
await expect(resolveAndroidEmulatorLocalEgressAsync({ handoffPath })).rejects.toThrow(
'the Android emulator fence was not started'
);
});
});
@@ -63,7 +63,8 @@ describe(uploadDeviceRunSessionArtifactAsync, () => {
metadata: { firstFrameRecordAt: 'test-time' },
size: reportedSize,
},
})
}),
{ fetch: expect.any(Function) }
);
expect(jest.mocked(fetch)).toHaveBeenCalledWith(
'https://uploads.expo.test/artifact',
@@ -73,4 +74,52 @@ describe(uploadDeviceRunSessionArtifactAsync, () => {
})
);
});
it('forwards cancellation through upload allocation and skips PUT when allocation aborts', async () => {
const controller = new AbortController();
const mutation = jest.fn().mockReturnValue({
toPromise: async () => {
controller.abort();
return {
data: {
deviceRunSession: {
createArtifactUploadSession: {
uploadSession: {
url: 'https://uploads.expo.test/artifact',
headers: {},
},
},
},
},
};
},
});
const ctx = { graphqlClient: { mutation } } as unknown as CustomBuildContext;
await expect(
uploadDeviceRunSessionArtifactAsync(ctx, {
deviceRunSessionId: 'session',
artifactId: 'id',
name: 'Logs',
filename: 'logs.ndjson',
kind: 'simulator-log',
size: 0,
stream: Readable.from([]),
signal: controller.signal,
})
).rejects.toThrow();
expect(fetch).not.toHaveBeenCalled();
const allocationFetch = mutation.mock.calls[0][2].fetch;
const mockFetch = jest.spyOn(globalThis, 'fetch').mockRejectedValueOnce(new Error('aborted'));
try {
await expect(allocationFetch('https://api.expo.test/graphql', {})).rejects.toThrow('aborted');
expect(mockFetch).toHaveBeenCalledWith(
'https://api.expo.test/graphql',
expect.objectContaining({
signal: expect.objectContaining({ aborted: true }),
})
);
} finally {
mockFetch.mockRestore();
}
});
});
@@ -0,0 +1,47 @@
import { screenshotArtifactDetails } from '../deviceRunSessionScreenshotNames';
const timestamp = '2026-09-24T08-45-59-123Z';
const name = 'Screenshot 2026-09-24 08:45:59 UTC';
const gitCommitHash = 'a1b2c3d' + '0'.repeat(33);
const session = {
platform: 'IOS' as const,
app: { slug: 'workflow-project' },
build: { id: 'installed-build', gitCommitHash, app: { slug: 'my-app' } },
};
it('omits the commit for archive and empty sessions', () => {
expect(screenshotArtifactDetails(timestamp, { ...session, build: null })).toEqual({
name,
filename: `workflow-project-ios-${timestamp}.png`,
metadata: { __eas_type: 'screenshot', appSlug: 'workflow-project', platform: 'ios' },
});
});
it.each([null, 'not-a-commit'])('omits a missing or invalid build commit: %s', gitCommitHash => {
const result = screenshotArtifactDetails(timestamp, {
...session,
build: { ...session.build, gitCommitHash },
});
expect(result.filename).toBe(`my-app-ios-${timestamp}.png`);
expect(result.metadata.gitCommitHash).toBeUndefined();
});
it('sanitizes and bounds the app slug for filesystem-safe names', () => {
const result = screenshotArtifactDetails(timestamp, {
...session,
build: { ...session.build, app: { slug: '../../My App/' + 'x'.repeat(100) } },
});
expect(result.filename).toMatch(/^My-App-x+-ios-a1b2c3d-/);
expect(result.filename).not.toMatch(/[/\\]/);
expect(result.filename.length).toBeLessThan(150);
});
it('accepts a full SHA-256 commit hash', () => {
const gitCommitHash = 'b'.repeat(64);
const result = screenshotArtifactDetails(timestamp, {
...session,
build: { ...session.build, gitCommitHash },
});
expect(result.filename).toBe(`my-app-ios-bbbbbbb-${timestamp}.png`);
expect(result.metadata.gitCommitHash).toBe(gitCommitHash);
});
@@ -0,0 +1,398 @@
import type { bunyan } from '@expo/logger';
import fs from 'node:fs';
import { randomBytes } from 'node:crypto';
import { lstat, mkdtemp, readdir, rename, rm, symlink, writeFile } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import type { CustomBuildContext } from '../../../customBuildContext';
import { Sentry } from '../../../sentry';
import { uploadDeviceRunSessionArtifactAsync } from '../deviceRunSessionArtifacts';
import {
startDeviceRunSessionScreenshotsAsync,
uploadDeviceRunSessionScreenshotsAsync,
} from '../deviceRunSessionScreenshots';
jest.mock('../deviceRunSessionArtifacts');
jest.mock('../../../sentry');
const ctx = {} as CustomBuildContext;
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const options = { deviceRunSessionId: 'session-id', logger };
let directory: string;
let uploaded: Buffer[];
let failedUploads: Map<string, { attempts: number; lastError: Error }>;
beforeEach(async () => {
jest.clearAllMocks();
directory = await mkdtemp(path.join(os.tmpdir(), 'screenshots-test-'));
uploaded = [];
failedUploads = new Map();
jest.mocked(uploadDeviceRunSessionArtifactAsync).mockImplementation(async (_ctx, { stream }) => {
const chunks: Buffer[] = [];
for await (const chunk of stream) {
chunks.push(Buffer.from(chunk));
}
uploaded.push(Buffer.concat(chunks));
});
});
afterEach(async () => {
for (const [, { stream }] of jest.mocked(uploadDeviceRunSessionArtifactAsync).mock.calls) {
stream.destroy();
}
await rm(directory, { recursive: true, force: true });
});
const flush = () =>
uploadDeviceRunSessionScreenshotsAsync(ctx, {
...options,
directory,
failedUploads,
signal: new AbortController().signal,
});
it('uploads completed captures with their bytes and stable IDs, ignoring partial files and symlinks', async () => {
const filename = 'screenshot-2026-09-24T08-45-59-123Z-ffffffffffff.png';
// Sorts before the real file, so following the link would upload it before the target is gone.
const link = 'screenshot-2026-09-24T08-45-59-123Z-000000000000.png';
await writeFile(path.join(directory, filename), 'png-content');
await writeFile(path.join(directory, `${filename}.tmp`), 'partial');
await symlink(path.join(directory, filename), path.join(directory, link));
await flush();
await flush();
expect(uploaded).toEqual([Buffer.from('png-content')]);
expect(uploadDeviceRunSessionArtifactAsync).toHaveBeenCalledTimes(1);
expect(uploadDeviceRunSessionArtifactAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({
deviceRunSessionId: 'session-id',
artifactId: filename.slice(0, -4),
name: 'Screenshot 2026-09-24 08:45:59 UTC',
filename: 'screenshot-2026-09-24T08-45-59-123Z.png',
kind: 'screenshot',
size: 11,
metadata: { __eas_type: 'screenshot' },
})
);
expect(await readdir(directory)).not.toContain(filename);
expect((await lstat(path.join(directory, link))).isSymbolicLink()).toBe(true);
});
it('uploads captures that never failed first, oldest first, before retrying failed ones', async () => {
const failed = 'screenshot-2026-09-24T08-45-00-000Z-aaaaaaaaaaaa.png';
const newer = 'screenshot-2026-09-24T08-45-59-123Z-bbbbbbbbbbbb.png';
const older = 'screenshot-2026-09-24T08-45-30-000Z-cccccccccccc.png';
for (const name of [newer, failed, older]) {
await writeFile(path.join(directory, name), name);
}
failedUploads.set(failed, { attempts: 1, lastError: new Error('stalled') });
await flush();
expect(
jest
.mocked(uploadDeviceRunSessionArtifactAsync)
.mock.calls.map(([, { artifactId }]) => artifactId)
).toEqual([older, newer, failed].map(name => name.slice(0, -4)));
});
it('keeps failed uploads and retries the same artifact ID, logging the failed attempt count', async () => {
const filename = `screenshot-2026-09-24T08-45-59-123Z-${randomBytes(6).toString('hex')}.png`;
await writeFile(path.join(directory, filename), 'png');
jest.mocked(uploadDeviceRunSessionArtifactAsync).mockImplementationOnce(async () => {
throw new Error('offline');
});
await flush();
expect(await readdir(directory)).toContain(filename);
await flush();
expect(uploaded).toEqual([Buffer.from('png')]);
expect(await readdir(directory)).toEqual([]);
expect(
jest.mocked(uploadDeviceRunSessionArtifactAsync).mock.calls.map(([, args]) => args.artifactId)
).toEqual([filename.slice(0, -4), filename.slice(0, -4)]);
expect(logger.info).toHaveBeenCalledWith(
expect.anything(),
expect.stringMatching(
/^Uploaded preview screenshot screenshot-2026-09-24T08-45-59-123Z\.png \(3 B\) in \d+ ms after 1 failed attempt\.$/
)
);
expect(failedUploads.size).toBe(0);
expect(Sentry.capture).not.toHaveBeenCalled();
});
it('uploads atomic captures immediately, including a capture arriving during another upload', async () => {
jest.useFakeTimers();
const collector = await startDeviceRunSessionScreenshotsAsync(ctx, options);
let releaseFirstUpload!: () => void;
const firstUploadGate = new Promise<void>(resolve => {
releaseFirstUpload = resolve;
});
let signalFirstUpload!: () => void;
const firstUploadStarted = new Promise<void>(resolve => {
signalFirstUpload = resolve;
});
let signalSecondUpload!: () => void;
const secondUploadStarted = new Promise<void>(resolve => {
signalSecondUpload = resolve;
});
const uploadNormally = jest.mocked(uploadDeviceRunSessionArtifactAsync).getMockImplementation();
if (!uploadNormally) {
throw new Error('Expected the default upload implementation.');
}
jest
.mocked(uploadDeviceRunSessionArtifactAsync)
.mockImplementationOnce(async (...args) => {
signalFirstUpload();
await firstUploadGate;
await uploadNormally(...args);
})
.mockImplementationOnce(async (...args) => {
signalSecondUpload();
await uploadNormally(...args);
});
const firstFile = path.join(
collector.directory,
'screenshot-2026-09-24T08-45-59-123Z-aaaaaaaaaaaa.png'
);
const secondFile = path.join(
collector.directory,
'screenshot-2026-09-24T08-45-59-124Z-bbbbbbbbbbbb.png'
);
try {
await writeFile(`${firstFile}.tmp`, 'first-capture');
await rename(`${firstFile}.tmp`, firstFile);
await firstUploadStarted;
await writeFile(`${secondFile}.tmp`, 'second-capture');
await rename(`${secondFile}.tmp`, secondFile);
expect(uploadDeviceRunSessionArtifactAsync).toHaveBeenCalledTimes(1);
releaseFirstUpload();
await secondUploadStarted;
await collector.finishAsync(true);
expect(uploaded).toEqual([Buffer.from('first-capture'), Buffer.from('second-capture')]);
expect(uploadDeviceRunSessionArtifactAsync).toHaveBeenCalledTimes(2);
} finally {
releaseFirstUpload();
jest.useRealTimers();
await collector.finishAsync(true);
await rm(collector.directory, { recursive: true, force: true });
}
});
it('uses periodic scans when watching is unavailable and does not overlap uploads', async () => {
jest.useFakeTimers();
const watchSpy = jest.spyOn(fs, 'watch').mockImplementation(() => {
throw new Error('Watching unavailable');
});
const collector = await startDeviceRunSessionScreenshotsAsync(ctx, options);
let releaseUpload!: () => void;
const uploadGate = new Promise<void>(resolve => {
releaseUpload = resolve;
});
let signalUploadStarted!: () => void;
const uploadStarted = new Promise<void>(resolve => {
signalUploadStarted = resolve;
});
try {
await writeFile(
path.join(collector.directory, 'screenshot-2026-09-24T08-45-59-123Z-aaaaaaaaaaaa.png'),
'live-capture'
);
jest.mocked(uploadDeviceRunSessionArtifactAsync).mockImplementationOnce(async () => {
signalUploadStarted();
await uploadGate;
});
jest.advanceTimersByTime(30_000);
await uploadStarted;
jest.advanceTimersByTime(60_000);
expect(uploadDeviceRunSessionArtifactAsync).toHaveBeenCalledTimes(1);
releaseUpload();
await collector.finishAsync(true);
expect(uploadDeviceRunSessionArtifactAsync).toHaveBeenCalledTimes(1);
} finally {
releaseUpload();
watchSpy.mockRestore();
jest.useRealTimers();
await collector.finishAsync(true);
await rm(collector.directory, { recursive: true, force: true });
}
});
it('warns for the first three attempts and every tenth after, without reporting to Sentry', async () => {
const filename = `screenshot-2026-09-24T08-45-59-123Z-${randomBytes(6).toString('hex')}.png`;
await writeFile(path.join(directory, filename), 'png');
jest.mocked(uploadDeviceRunSessionArtifactAsync).mockRejectedValue(new Error('offline'));
for (let i = 0; i < 12; i++) {
await flush();
}
expect(jest.mocked(logger.warn).mock.calls.map(([fields]) => fields.attempt)).toEqual([
1, 2, 3, 10,
]);
expect(logger.warn).toHaveBeenNthCalledWith(
1,
expect.objectContaining({ file: path.join(directory, filename), attempt: 1, size: 3 }),
'Could not upload preview screenshot (attempt 1). Keeping it for retry.'
);
expect(logger.warn).toHaveBeenNthCalledWith(
2,
expect.objectContaining({ attempt: 2 }),
'Could not upload preview screenshot (attempt 2). Keeping it for retry.'
);
expect(failedUploads.get(filename)).toEqual({
attempts: 12,
lastError: expect.objectContaining({ message: 'offline' }),
});
expect(Sentry.capture).not.toHaveBeenCalled();
});
it('removes an empty directory on finish and returns the same task when finish repeats', async () => {
const collector = await startDeviceRunSessionScreenshotsAsync(ctx, options);
const finish = collector.finishAsync(true);
expect(collector.finishAsync(true)).toBe(finish);
await finish;
await expect(readdir(collector.directory)).rejects.toMatchObject({ code: 'ENOENT' });
expect(logger.info).toHaveBeenCalledWith('Uploaded 0 preview screenshots during the session.');
expect(Sentry.capture).not.toHaveBeenCalled();
});
it('keeps the directory and warns when the session host is still running', async () => {
const collector = await startDeviceRunSessionScreenshotsAsync(ctx, options);
try {
await collector.finishAsync(false);
expect(await readdir(collector.directory)).toEqual([]);
expect(logger.warn).toHaveBeenCalledWith(
{ directory: collector.directory },
'The session host is still running, so preview screenshots it saves from now on are not uploaded.'
);
expect(Sentry.capture).not.toHaveBeenCalled();
} finally {
await rm(collector.directory, { recursive: true, force: true });
}
});
it('warns with the files not uploaded and their last error, and reports them to Sentry once', async () => {
const collector = await startDeviceRunSessionScreenshotsAsync(ctx, options);
const filename = 'screenshot-2026-09-24T08-45-59-123Z-aaaaaaaaaaaa.png';
jest.mocked(uploadDeviceRunSessionArtifactAsync).mockRejectedValue(new Error('offline'));
try {
await writeFile(path.join(collector.directory, filename), 'png');
await collector.finishAsync(true);
const notUploaded = [{ name: filename, attempts: expect.any(Number), lastError: 'offline' }];
expect(logger.warn).toHaveBeenCalledWith(
{ directory: collector.directory, files: notUploaded },
'1 preview screenshot was not uploaded.'
);
expect(Sentry.capture).toHaveBeenCalledTimes(1);
expect(Sentry.capture).toHaveBeenCalledWith(
'Preview screenshots were not uploaded',
expect.objectContaining({
message: '1 preview screenshot was not uploaded.',
}),
{ extras: { files: notUploaded } }
);
} finally {
await rm(collector.directory, { recursive: true, force: true });
}
});
it('reports a screenshot as not uploaded when the shutdown deadline cuts off its upload', async () => {
jest.useFakeTimers();
const collector = await startDeviceRunSessionScreenshotsAsync(ctx, options);
const filename = 'screenshot-2026-09-24T08-45-59-123Z-aaaaaaaaaaaa.png';
const file = path.join(collector.directory, filename);
let signalUploadStarted!: () => void;
const uploadStarted = new Promise<void>(resolve => {
signalUploadStarted = resolve;
});
jest.mocked(uploadDeviceRunSessionArtifactAsync).mockImplementation(async (_ctx, { signal }) => {
signalUploadStarted();
await new Promise((_resolve, reject) => {
signal?.addEventListener('abort', () => reject(signal.reason));
});
});
try {
await writeFile(file, 'png');
const finish = collector.finishAsync(true);
await uploadStarted;
jest.advanceTimersByTime(30_000);
await finish;
expect(logger.warn).toHaveBeenCalledWith(
{ directory: collector.directory },
'Preview screenshot flush timed out after 30 s.'
);
expect(logger.warn).toHaveBeenCalledWith(
{ file, size: 3 },
'Shutdown deadline reached before this preview screenshot uploaded. Keeping the file.'
);
expect(logger.warn).toHaveBeenCalledWith(
{ directory: collector.directory, files: [{ name: filename }] },
'1 preview screenshot was not uploaded.'
);
expect(logger.info).toHaveBeenCalledWith('Uploaded 0 preview screenshots during the session.');
expect(Sentry.capture).toHaveBeenCalledTimes(1);
expect(await readdir(collector.directory)).toEqual([filename]);
} finally {
jest.useRealTimers();
await rm(collector.directory, { recursive: true, force: true });
}
});
it('reports a failure record once and deletes it while uploading captures from the same scan', async () => {
const png = 'screenshot-2026-09-24T08-45-59-123Z-aaaaaaaaaaaa.png';
const failed = 'screenshot-2026-09-24T08-45-59-124Z-bbbbbbbbbbbb.png';
const record = { file: failed, error: 'ENOSPC: no space left', at: '2026-09-24T08:45:59.124Z' };
await writeFile(path.join(directory, png), 'png');
await writeFile(
path.join(directory, 'screenshot-2026-09-24T08-45-59-124Z-bbbbbbbbbbbb.failed.json'),
JSON.stringify(record)
);
expect(await flush()).toEqual({ uploaded: 1, saveFailures: 1 });
expect(await flush()).toEqual({ uploaded: 0, saveFailures: 0 });
expect(uploaded).toEqual([Buffer.from('png')]);
expect(await readdir(directory)).toEqual([]);
expect(logger.warn).toHaveBeenCalledTimes(1);
expect(logger.warn).toHaveBeenCalledWith(
record,
'The session host could not save a preview screenshot.'
);
expect(Sentry.capture).toHaveBeenCalledTimes(1);
expect(Sentry.capture).toHaveBeenCalledWith(
'The session host could not save a preview screenshot',
expect.objectContaining({ message: 'ENOSPC: no space left' }),
{ extras: { file: failed, at: record.at } }
);
});
it('warns about a malformed failure record, deletes it, and does not report it to Sentry', async () => {
const file = path.join(directory, 'screenshot-2026-09-24T08-45-59-124Z-bbbbbbbbbbbb.failed.json');
await writeFile(file, JSON.stringify({ file: 'screenshot.png' }));
await flush();
expect(logger.warn).toHaveBeenCalledWith(
{ err: expect.anything(), file },
'Could not read a preview screenshot failure record.'
);
expect(await readdir(directory)).toEqual([]);
expect(Sentry.capture).not.toHaveBeenCalled();
});
it('handles a failure record as soon as it appears and counts it on finish', async () => {
jest.useFakeTimers();
const collector = await startDeviceRunSessionScreenshotsAsync(ctx, options);
let signalReported!: () => void;
const reported = new Promise<void>(resolve => {
signalReported = resolve;
});
jest.mocked(Sentry.capture).mockImplementationOnce(() => signalReported());
const record = path.join(
collector.directory,
'screenshot-2026-09-24T08-45-59-124Z-bbbbbbbbbbbb.failed.json'
);
try {
await writeFile(`${record}.tmp`, JSON.stringify({ file: 'x.png', error: 'EIO', at: 'now' }));
await rename(`${record}.tmp`, record);
await reported;
await collector.finishAsync(true);
expect(logger.info).toHaveBeenCalledWith(
'Uploaded 0 preview screenshots during the session; the session host could not save 1.'
);
} finally {
jest.useRealTimers();
await collector.finishAsync(true);
await rm(collector.directory, { recursive: true, force: true });
}
});
@@ -0,0 +1,265 @@
import type { bunyan } from '@expo/logger';
import { Client, fetchExchange } from '@urql/core';
import { mkdtemp, rm, stat, writeFile } from 'node:fs/promises';
import { type IncomingMessage, type ServerResponse, createServer } from 'node:http';
import { tmpdir } from 'node:os';
import path from 'node:path';
import type { CustomBuildContext } from '../../../customBuildContext';
import {
loadScreenshotSessionAsync,
screenshotArtifactDetails,
} from '../deviceRunSessionScreenshotNames';
import { uploadDeviceRunSessionScreenshotsAsync } from '../deviceRunSessionScreenshots';
jest.unmock('node-fetch');
jest.mock('../../../sentry');
function deferred<T>() {
let resolve!: (value: T) => void;
const promise = new Promise<T>(r => {
resolve = r;
});
return { promise, resolve };
}
let handle: (request: IncomingMessage, response: ServerResponse) => void;
let server: ReturnType<typeof createServer>;
let ctx: CustomBuildContext;
let url: string;
let directory: string;
let creates: number;
let authorization: string | undefined;
let stallCreation: boolean;
let creationStarted: ReturnType<typeof deferred<IncomingMessage>>;
let creationBodies: string[];
const installedCommit = 'a1b2c3d' + '0'.repeat(33);
let sessionLookupFails: boolean;
let sessionLookups: number;
let uploadQuery: string;
beforeEach(async () => {
creates = 0;
sessionLookups = 0;
sessionLookupFails = false;
uploadQuery = '';
authorization = undefined;
stallCreation = false;
creationBodies = [];
creationStarted = deferred();
directory = await mkdtemp(path.join(tmpdir(), 'recording-upload-http-'));
server = createServer((request, response) => {
if (request.url?.startsWith('/graphql')) {
authorization = request.headers.authorization;
request.setEncoding('utf8');
let body = '';
if (request.method === 'GET') {
const requestUrl = new URL(request.url, 'http://localhost');
body = JSON.stringify({ query: requestUrl.searchParams.get('query') });
}
request.on('data', chunk => (body += chunk));
request.on('end', () => {
const operation = JSON.parse(body);
if (operation.query.includes('query ScreenshotSession')) {
sessionLookups++;
response.setHeader('Content-Type', 'application/json');
if (sessionLookupFails) {
response.end(JSON.stringify({ errors: [{ message: 'Lookup unavailable' }] }));
} else {
response.end(
JSON.stringify({
data: {
deviceRunSessions: {
byId: {
platform: 'IOS',
app: { slug: 'session-project' },
build: {
id: 'installed-build',
gitCommitHash: installedCommit,
app: { slug: 'my-app' },
},
},
},
},
})
);
}
return;
}
creates++;
creationBodies.push(body);
creationStarted.resolve(request);
if (stallCreation) {
return;
}
response.setHeader('Content-Type', 'application/json');
response.end(
JSON.stringify({
data: {
deviceRunSession: {
createArtifactUploadSession: {
uploadSession: { url: `${url}/artifact${uploadQuery}`, headers: {} },
},
},
},
})
);
});
} else {
handle(request, response);
}
});
await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve));
const address = server.address();
if (!address || typeof address === 'string') {
throw new Error('Missing test server address');
}
url = `http://127.0.0.1:${address.port}`;
ctx = {
graphqlClient: new Client({
url: `${url}/graphql`,
exchanges: [fetchExchange],
fetchOptions: { headers: { Authorization: 'Bearer test-auth' } },
}),
} as CustomBuildContext;
});
afterEach(async () => {
server.closeAllConnections();
await new Promise<void>(resolve => server.close(() => resolve()));
await rm(directory, { recursive: true, force: true });
});
it('publishes a manual preview PNG through the artifact GraphQL and HTTP upload path', async () => {
const payload = Buffer.from(
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+aF9sAAAAASUVORK5CYII=',
'base64'
);
const filename = 'screenshot-2026-09-24T08-45-59-123Z-a1b2c3d4e5f6.png';
await writeFile(path.join(directory, filename), payload);
const bodies: Buffer[] = [];
handle = (request, response) => {
const chunks: Buffer[] = [];
request.on('data', chunk => chunks.push(Buffer.from(chunk)));
request.on('end', () => {
bodies.push(Buffer.concat(chunks));
response.end();
});
};
const session = loadScreenshotSessionAsync(ctx, 'drs-id', {
warn: jest.fn(),
} as unknown as bunyan);
await uploadDeviceRunSessionScreenshotsAsync(ctx, {
session,
directory,
deviceRunSessionId: 'drs-id',
failedUploads: new Map(),
logger: { info: jest.fn(), warn: jest.fn() } as unknown as bunyan,
signal: new AbortController().signal,
});
expect(bodies).toEqual([payload]);
expect(creates).toBe(1);
expect(sessionLookups).toBe(1);
expect(authorization).toBe('Bearer test-auth');
expect(JSON.parse(creationBodies[0]).variables).toEqual(
expect.objectContaining({
deviceRunSessionId: 'drs-id',
input: expect.objectContaining({
name: 'Screenshot 2026-09-24 08:45:59 UTC',
filename: 'my-app-ios-a1b2c3d-2026-09-24T08-45-59-123Z.png',
kind: 'screenshot',
size: payload.length,
metadata: {
__eas_type: 'screenshot',
appSlug: 'my-app',
platform: 'ios',
buildId: 'installed-build',
gitCommitHash: installedCommit,
},
}),
})
);
await expect(stat(path.join(directory, filename))).rejects.toMatchObject({ code: 'ENOENT' });
});
it('rereads the PNG when the artifact PUT is retried', async () => {
const filename = 'screenshot-2026-09-24T08-45-59-123Z-a1b2c3d4e5f6.png';
await writeFile(path.join(directory, filename), 'retried-capture');
const bodies: string[] = [];
handle = (request, response) => {
let body = '';
request.setEncoding('utf8');
request.on('data', chunk => (body += chunk));
request.on('end', () => {
bodies.push(body);
response.statusCode = bodies.length === 1 ? 503 : 200;
response.end();
});
};
const uploaded = await uploadDeviceRunSessionScreenshotsAsync(ctx, {
directory,
deviceRunSessionId: 'drs-id',
failedUploads: new Map(),
logger: { info: jest.fn(), warn: jest.fn() } as unknown as bunyan,
signal: new AbortController().signal,
});
expect(uploaded).toEqual({ uploaded: 1, saveFailures: 0 });
expect(creates).toBe(1);
expect(bodies).toEqual(['retried-capture', 'retried-capture']);
await expect(stat(path.join(directory, filename))).rejects.toMatchObject({ code: 'ENOENT' });
});
it('aborts a stalled GraphQL upload request and keeps the PNG', async () => {
stallCreation = true;
const filename = 'screenshot-2026-09-24T08-45-59-123Z-a1b2c3d4e5f6.png';
await writeFile(path.join(directory, filename), 'png');
const controller = new AbortController();
const upload = uploadDeviceRunSessionScreenshotsAsync(ctx, {
directory,
deviceRunSessionId: 'drs-id',
failedUploads: new Map(),
logger: { info: jest.fn(), warn: jest.fn() } as unknown as bunyan,
signal: controller.signal,
});
await creationStarted.promise;
controller.abort();
await upload;
expect((await stat(path.join(directory, filename))).size).toBe(3);
});
it('keeps the signed upload URL out of the error when the upload connection drops', async () => {
uploadQuery = '?X-Goog-Signature=secret-signature';
handle = request => request.socket.destroy();
const filename = 'screenshot-2026-09-24T08-45-59-123Z-a1b2c3d4e5f6.png';
await writeFile(path.join(directory, filename), 'png');
const failedUploads = new Map<string, { attempts: number; lastError: Error }>();
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
await uploadDeviceRunSessionScreenshotsAsync(ctx, {
directory,
deviceRunSessionId: 'drs-id',
failedUploads,
logger,
signal: new AbortController().signal,
});
const error = failedUploads.get(filename)?.lastError;
expect(error?.message).toBe(
`Failed to upload device run session artifact ${filename.slice(0, -4)}: ECONNRESET.`
);
expect(error?.cause).toBeUndefined();
expect(JSON.stringify(jest.mocked(logger.warn).mock.calls)).not.toContain('secret-signature');
expect(JSON.stringify(jest.mocked(logger.warn).mock.calls)).not.toContain('/artifact');
});
it('falls back without using the workflow commit when the session lookup fails', async () => {
sessionLookupFails = true;
ctx.metadata = { appName: 'wrong-app', gitCommitHash: 'f'.repeat(40) };
const logger = { warn: jest.fn() } as unknown as bunyan;
const session = await loadScreenshotSessionAsync(ctx, 'drs-id', logger);
expect(session).toBeNull();
expect(logger.warn).toHaveBeenCalledWith(
expect.objectContaining({ deviceRunSessionId: 'drs-id' }),
'Could not load screenshot session details; using capture timestamps.'
);
expect(screenshotArtifactDetails('2026-09-24T08-45-59-123Z', session)).toEqual({
name: 'Screenshot 2026-09-24 08:45:59 UTC',
filename: 'screenshot-2026-09-24T08-45-59-123Z.png',
metadata: { __eas_type: 'screenshot' },
});
});
@@ -0,0 +1,80 @@
import {
DeviceRunSessionTimeoutError,
withDeviceRunSessionTimeoutAsync,
} from '../deviceRunSessionTimeout';
afterEach(() => jest.useRealTimers());
it('aborts a stalled operation when its deadline expires', async () => {
jest.useFakeTimers();
let signal: AbortSignal | undefined;
const operation = withDeviceRunSessionTimeoutAsync(
{ name: 'test operation', timeoutMs: 100 },
async value => {
signal = value;
return await new Promise<void>(() => {});
}
);
const rejected = expect(operation).rejects.toThrow('test operation timed out after 100ms');
const typed = expect(operation).rejects.toBeInstanceOf(DeviceRunSessionTimeoutError);
await jest.advanceTimersByTimeAsync(100);
await rejected;
await typed;
expect(signal?.aborted).toBe(true);
expect(jest.getTimerCount()).toBe(0);
});
it('does not start an operation with an already aborted parent', async () => {
const operation = jest.fn();
await expect(
withDeviceRunSessionTimeoutAsync(
{
name: 'test',
timeoutMs: 100,
signal: AbortSignal.abort(new Error('parent ended')),
},
operation
)
).rejects.toThrow('parent ended');
expect(operation).not.toHaveBeenCalled();
});
it('clears the deadline when an operation succeeds', async () => {
jest.useFakeTimers();
await expect(
withDeviceRunSessionTimeoutAsync({ name: 'test', timeoutMs: 100 }, async () => 42)
).resolves.toBe(42);
expect(jest.getTimerCount()).toBe(0);
});
it('ignores a deadline reset that arrives after the operation settled', async () => {
jest.useFakeTimers();
let resetDeadline!: () => void;
await expect(
withDeviceRunSessionTimeoutAsync({ name: 'test', timeoutMs: 100 }, async (_signal, reset) => {
resetDeadline = reset;
return 42;
})
).resolves.toBe(42);
resetDeadline();
expect(jest.getTimerCount()).toBe(0);
});
it('keeps an operation alive while it reports progress', async () => {
jest.useFakeTimers();
const operation = withDeviceRunSessionTimeoutAsync(
{ name: 'test operation', timeoutMs: 100 },
async (signal, resetDeadline) => {
await new Promise<void>(resolve => setTimeout(resolve, 80));
resetDeadline();
await new Promise<void>(resolve => setTimeout(resolve, 80));
resetDeadline();
await new Promise<void>(resolve => setTimeout(resolve, 80));
return signal.aborted;
}
);
const settled = expect(operation).resolves.toBe(false);
await jest.advanceTimersByTimeAsync(240);
await settled;
expect(jest.getTimerCount()).toBe(0);
});
@@ -0,0 +1,463 @@
import type { bunyan } from '@expo/logger';
import type { BuildStepEnv } from '@expo/steps';
import { Client, fetchExchange } from '@urql/core';
import { execFileSync } from 'node:child_process';
import { createReadStream } from 'node:fs';
import { copyFile, mkdir, mkdtemp, rm, stat, writeFile } from 'node:fs/promises';
import { type IncomingMessage, type ServerResponse, createServer } from 'node:http';
import type { Socket } from 'node:net';
import { tmpdir } from 'node:os';
import path from 'node:path';
import { Readable } from 'node:stream';
import { setTimeout as delay } from 'node:timers/promises';
import type { CustomBuildContext } from '../../../customBuildContext';
import { uploadDeviceRunSessionArtifactAsync } from '../deviceRunSessionArtifacts';
import {
findUnlistedDeviceScreenRecordingsAsync,
uploadDeviceRunSessionScreenRecordingsAsync,
} from '../deviceRunSessionScreenRecordings';
jest.unmock('node-fetch');
jest.unmock('node:fs');
jest.unmock('node:fs/promises');
function deferred<T>() {
let resolve!: (value: T) => void;
const promise = new Promise<T>(r => {
resolve = r;
});
return { promise, resolve };
}
const metadata = {
deviceRunSessionId: 'drs-id',
artifactId: 'recording-id',
name: 'Recording',
filename: 'recording.mp4',
kind: 'screen-recording',
size: 16,
};
let handle: (request: IncomingMessage, response: ServerResponse) => void;
let server: ReturnType<typeof createServer>;
let ctx: CustomBuildContext;
let url: string;
let directory: string;
let creates: number;
let authorization: string | undefined;
let stallCreation: boolean;
let creationStarted: ReturnType<typeof deferred<IncomingMessage>>;
let creationBodies: string[];
beforeEach(async () => {
creates = 0;
authorization = undefined;
stallCreation = false;
creationBodies = [];
creationStarted = deferred();
directory = await mkdtemp(path.join(tmpdir(), 'recording-upload-http-'));
server = createServer((request, response) => {
if (request.url === '/graphql') {
creates++;
authorization = request.headers.authorization;
creationStarted.resolve(request);
request.setEncoding('utf8');
let body = '';
request.on('data', chunk => (body += chunk));
request.on('end', () => creationBodies.push(body));
request.on('end', () => {
if (stallCreation) {
return;
}
response.setHeader('Content-Type', 'application/json');
response.end(
JSON.stringify({
data: {
deviceRunSession: {
createArtifactUploadSession: {
uploadSession: { url: `${url}/artifact`, headers: {} },
},
},
},
})
);
});
} else {
handle(request, response);
}
});
await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve));
const address = server.address();
if (!address || typeof address === 'string') {
throw new Error('Missing test server address');
}
url = `http://127.0.0.1:${address.port}`;
ctx = {
graphqlClient: new Client({
url: `${url}/graphql`,
exchanges: [fetchExchange],
fetchOptions: { headers: { Authorization: 'Bearer test-auth' } },
}),
} as CustomBuildContext;
});
afterEach(async () => {
server.closeAllConnections();
await new Promise<void>(resolve => server.close(() => resolve()));
await rm(directory, { recursive: true, force: true });
});
it('replays the full file after a transient PUT failure without recreating the upload session', async () => {
const bodies: string[] = [];
handle = (request, response) => {
const chunks: string[] = [];
request.setEncoding('utf8');
request.on('data', chunk => chunks.push(chunk));
request.on('end', () => {
bodies.push(chunks.join(''));
response.statusCode = bodies.length === 1 ? 503 : 200;
response.end();
});
};
const file = path.join(directory, 'recording.mp4');
const payload = '*'.repeat(128 * 1024);
await writeFile(file, payload);
const streams: Readable[] = [];
const open = () => {
const stream = createReadStream(file);
streams.push(stream);
return stream;
};
await uploadDeviceRunSessionArtifactAsync(ctx, {
...metadata,
size: payload.length,
stream: open(),
reopenStream: open,
});
expect(creates).toBe(1);
expect(authorization).toBe('Bearer test-auth');
expect(bodies).toEqual([payload, payload]);
expect(streams).toHaveLength(2);
expect(streams.every(stream => stream.destroyed)).toBe(true);
});
it('keeps a slow PUT alive while its body is still flowing', async () => {
jest.useFakeTimers({ doNotFake: ['nextTick', 'setImmediate', 'queueMicrotask'] });
let received = 0;
let onChunk = () => {};
handle = (request, response) => {
request.on('data', chunk => {
received += chunk.length;
onChunk();
});
request.on('end', () => {
response.statusCode = 200;
response.end();
});
};
const waitForBytes = (total: number) =>
new Promise<void>(resolve => {
onChunk = () => {
if (received >= total) {
resolve();
}
};
onChunk();
});
const stream = new Readable({ read() {} });
const upload = uploadDeviceRunSessionArtifactAsync(ctx, { ...metadata, size: 3, stream });
const settled = expect(upload).resolves.toBeUndefined();
stream.push('a');
await waitForBytes(1);
await jest.advanceTimersByTimeAsync(60_000);
stream.push('b');
await waitForBytes(2);
await jest.advanceTimersByTimeAsync(60_000);
stream.push('c');
stream.push(null);
await settled;
expect(received).toBe(3);
jest.useRealTimers();
});
it('retries a PUT that stalled past the idle deadline with a fresh stream', async () => {
jest.useFakeTimers({ doNotFake: ['nextTick', 'setImmediate', 'queueMicrotask'] });
let puts = 0;
const firstBodyRead = deferred<void>();
handle = (request, response) => {
puts++;
request.resume();
request.on('end', () => {
if (puts === 1) {
firstBodyRead.resolve();
return;
}
response.statusCode = 200;
response.end();
});
};
const streams: Readable[] = [];
const open = () => {
const stream = Readable.from('recording');
streams.push(stream);
return stream;
};
const upload = uploadDeviceRunSessionArtifactAsync(ctx, {
...metadata,
size: 9,
stream: open(),
reopenStream: open,
});
const settled = expect(upload).resolves.toBeUndefined();
await firstBodyRead.promise;
await jest.advanceTimersByTimeAsync(90_000);
await jest.advanceTimersByTimeAsync(500);
await settled;
expect(puts).toBe(2);
expect(streams).toHaveLength(2);
expect(streams.every(stream => stream.destroyed)).toBe(true);
jest.useRealTimers();
});
it('cancels an active PUT socket and destroys its source without retrying', async () => {
const started = deferred<void>();
const closed = deferred<void>();
let puts = 0;
handle = request => {
puts++;
request.socket.once('close', () => closed.resolve());
request.on('data', () => started.resolve());
};
const controller = new AbortController();
const stream = new Readable({ read() {} });
stream.push(Buffer.alloc(1024));
const reopenStream = jest.fn(() => Readable.from('unexpected retry'));
const upload = uploadDeviceRunSessionArtifactAsync(ctx, {
...metadata,
stream,
reopenStream,
signal: controller.signal,
});
const rejected = expect(upload).rejects.toThrow('test cancellation');
await started.promise;
controller.abort(new Error('test cancellation'));
await rejected;
await closed.promise;
expect(stream.destroyed).toBe(true);
expect(reopenStream).not.toHaveBeenCalled();
expect(puts).toBe(1);
});
it.each([200, 400, 503])(
'closes a stalled HTTP %s response before leaving or retrying the PUT',
async status => {
const sockets = new Set<Socket>();
let puts = 0;
let maxOpenSockets = 0;
handle = (request, response) => {
puts++;
sockets.add(request.socket);
maxOpenSockets = Math.max(maxOpenSockets, sockets.size);
request.socket.once('close', () => sockets.delete(request.socket));
request.resume();
request.on('end', () => {
response.writeHead(status, { 'Content-Length': '1000' });
response.flushHeaders();
response.write('partial response');
});
};
const upload = uploadDeviceRunSessionArtifactAsync(ctx, {
...metadata,
stream: Readable.from('recording'),
reopenStream: () => Readable.from('recording'),
});
if (status === 200) {
await expect(upload).resolves.toBeUndefined();
} else {
await expect(upload).rejects.toThrow(`HTTP ${status}`);
}
for (let i = 0; i < 100 && sockets.size > 0; i++) {
await delay(10);
}
expect(creates).toBe(1);
expect(puts).toBe(status === 503 ? 3 : 1);
expect(maxOpenSockets).toBe(1);
expect(sockets.size).toBe(0);
}
);
it('cancels urql session creation with auth intact and never starts a PUT', async () => {
stallCreation = true;
const puts = jest.fn();
handle = puts;
const controller = new AbortController();
const stream = Readable.from('recording');
const upload = uploadDeviceRunSessionArtifactAsync(ctx, {
...metadata,
stream,
signal: controller.signal,
});
const rejected = expect(upload).rejects.toThrow('test cancellation');
const request = await creationStarted.promise;
const closed = new Promise<void>(resolve => request.socket.once('close', resolve));
controller.abort(new Error('test cancellation'));
await rejected;
await closed;
expect(authorization).toBe('Bearer test-auth');
expect(creates).toBe(1);
expect(puts).not.toHaveBeenCalled();
expect(stream.destroyed).toBe(true);
});
it('says ffprobe is missing instead of calling an unlisted recording undecodable', async () => {
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const session = path.join(directory, 'session');
await mkdir(session);
await writeFile(path.join(session, 'recording.mp4.partial'), new Uint8Array(28));
await writeFile(
path.join(session, 'session.json'),
JSON.stringify({
udid: 'emulator-5554',
deviceName: 'Pixel',
runtimeDisplayName: 'Android 16',
status: 'recording',
recording: 'recording.mp4.partial',
})
);
const recordings = await findUnlistedDeviceScreenRecordingsAsync({
root: directory,
env: { ...process.env, PATH: path.join(directory, 'no-tools') } as BuildStepEnv,
logger,
});
expect(recordings).toEqual([]);
expect(logger.warn).toHaveBeenCalledWith(
'ffprobe is not available; cannot recover the unlisted recording session/recording.mp4.partial.'
);
});
it('recovers unlisted recordings from a killed host and flags the unfinished ones as partial', async () => {
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const env = process.env as BuildStepEnv;
const cut = path.join(directory, 'cut');
const empty = path.join(directory, 'empty');
const failed = path.join(directory, 'failed');
const done = path.join(directory, 'done');
await mkdir(cut);
await mkdir(empty);
await mkdir(failed);
await mkdir(done);
const manifest = {
udid: 'emulator-5554',
deviceName: 'Pixel',
runtimeDisplayName: 'Android 16',
status: 'recording',
recording: 'recording.mp4.partial',
firstFrameWallClock: { iso8601: '2026-07-10T10:00:00.000Z' },
width: 128,
height: 96,
};
execFileSync('ffmpeg', [
'-v',
'error',
'-f',
'lavfi',
'-i',
'color=c=red:s=128x96:r=1',
'-t',
'3',
'-c:v',
'libx264',
'-g',
'1',
'-pix_fmt',
'yuv420p',
'-movflags',
'frag_keyframe+empty_moov',
'-f',
'mp4',
path.join(cut, 'recording.mp4.partial'),
]);
await writeFile(path.join(cut, 'session.json'), JSON.stringify(manifest));
await writeFile(path.join(empty, 'recording.mp4.partial'), new Uint8Array(28));
await writeFile(path.join(empty, 'session.json'), JSON.stringify(manifest));
await copyFile(
path.join(cut, 'recording.mp4.partial'),
path.join(failed, 'recording.mp4.partial')
);
await writeFile(
path.join(failed, 'session.json'),
JSON.stringify({
...manifest,
status: 'failed',
error: 'Recording finalization exceeded 30000 ms.',
})
);
await copyFile(path.join(cut, 'recording.mp4.partial'), path.join(done, 'recording.mp4'));
await writeFile(
path.join(done, 'session.json'),
JSON.stringify({ ...manifest, status: 'complete', recording: 'recording.mp4' })
);
const recordings = await findUnlistedDeviceScreenRecordingsAsync({
root: directory,
env,
logger,
});
expect(recordings).toEqual([
{
udid: 'emulator-5554',
deviceName: 'Pixel',
runtimeDisplayName: 'Android 16',
directory: cut,
},
{
udid: 'emulator-5554',
deviceName: 'Pixel',
runtimeDisplayName: 'Android 16',
directory: done,
},
{
udid: 'emulator-5554',
deviceName: 'Pixel',
runtimeDisplayName: 'Android 16',
directory: failed,
},
]);
expect(logger.warn).toHaveBeenCalledWith(expect.stringContaining('empty/recording.mp4.partial'));
let putBytes = 0;
handle = (request, response) => {
request.on('data', chunk => (putBytes += chunk.length));
request.on('end', () => {
response.statusCode = 200;
response.end();
});
};
await expect(
uploadDeviceRunSessionScreenRecordingsAsync(ctx, {
logger,
deviceRunSessionId: 'drs-id',
recordings,
})
).resolves.toBe(true);
expect(putBytes).toBe(3 * (await stat(path.join(cut, 'recording.mp4.partial'))).size);
const inputs = creationBodies
.map(body => JSON.parse(body).variables.input)
.sort((a, b) => a.filename.localeCompare(b.filename));
expect(inputs.map(input => input.filename)).toEqual(['cut.mp4', 'done.mp4', 'failed.mp4']);
expect(inputs.map(input => input.name.includes(', partial)'))).toEqual([true, false, true]);
expect(inputs[1].metadata).not.toHaveProperty('partial');
expect(inputs[0].metadata).toMatchObject({
partial: true,
partialReason: 'The Device Hub stopped before it could finalize the recording.',
width: 128,
height: 96,
});
expect(inputs[2].metadata).toMatchObject({
partial: true,
partialReason: 'Recording finalization exceeded 30000 ms.',
});
});
@@ -0,0 +1,504 @@
import type { bunyan } from '@expo/logger';
import { BuildRuntimePlatform, type BuildStepEnv } from '@expo/steps';
import * as ngrok from '@ngrok/ngrok';
import { access, mkdir, readdir, rm, writeFile } from 'node:fs/promises';
import path from 'node:path';
import type { CustomBuildContext } from '../../../customBuildContext';
import { Sentry } from '../../../sentry';
import { turtleFetch } from '../../../utils/turtleFetch';
import { uploadDeviceRunSessionArtifactAsync } from '../deviceRunSessionArtifacts';
import {
findUnlistedDeviceScreenRecordingsAsync,
uploadDeviceRunSessionScreenRecordingsAsync,
} from '../deviceRunSessionScreenRecordings';
import { startDeviceSessionHostAsync } from '../deviceSessionHost';
import { spawnDetached } from '../remoteDeviceRunSession';
jest.mock('@ngrok/ngrok');
jest.mock('../deviceRunSessionArtifacts');
jest.mock('../../../sentry');
jest.mock('../serveSimMetricsRecorder', () => ({
readServeSimServersAsync: jest
.fn()
.mockResolvedValue([{ udid: 'emulator-5554', token: 'preview-token' }]),
}));
jest.mock('../../../utils/turtleFetch');
jest.mock('../remoteDeviceRunSession', () => ({
...jest.requireActual('../remoteDeviceRunSession'),
ensureFfmpegInstalledOnceAsync: jest.fn(),
fetchWebPreviewTurnArgsAsync: jest.fn().mockResolvedValue([]),
spawnDetached: jest.fn(),
}));
jest.mock('../deviceRunSessionScreenRecordings', () => ({
...jest.requireActual('../deviceRunSessionScreenRecordings'),
findUnlistedDeviceScreenRecordingsAsync: jest.fn(),
uploadDeviceRunSessionScreenRecordingsAsync: jest.fn(),
}));
function deferred<T>() {
let resolve!: (value: T) => void;
const promise = new Promise<T>(r => {
resolve = r;
});
return { promise, resolve };
}
const env = {
DEVICE_RUN_SESSION_ID: 'drs-id',
NGROK_AUTHTOKEN: 'token',
} as BuildStepEnv;
const ctx = {} as CustomBuildContext;
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const stopServer = jest.fn();
const closeTunnel = jest.fn();
const directories: string[] = [];
const baseDomain = 'preview.example.test';
async function startHostAsync() {
return await startDeviceSessionHostAsync(ctx, {
runtimePlatform: BuildRuntimePlatform.LINUX,
env,
logger,
timeoutMs: 10_000,
});
}
beforeEach(() => {
jest.clearAllMocks();
stopServer.mockResolvedValue(undefined);
closeTunnel.mockResolvedValue(undefined);
jest.mocked(uploadDeviceRunSessionScreenRecordingsAsync).mockReset().mockResolvedValue(false);
jest.mocked(findUnlistedDeviceScreenRecordingsAsync).mockReset().mockResolvedValue([]);
jest.mocked(spawnDetached).mockImplementation(options => {
const flag = options.args.indexOf('--android-recording-directory');
if (flag >= 0) {
directories.push(options.args[flag + 1]);
}
return {
pid: undefined,
getOutput: () => '',
getExitError: () => undefined,
stopAsync: stopServer,
};
});
jest.mocked(ngrok.forward).mockResolvedValue({
url: () => 'https://public.example.test',
close: closeTunnel,
} as never);
jest.mocked(turtleFetch).mockImplementation(
async () =>
({
ok: true,
json: async () => ({ status: 'ready', device: 'emulator-5554' }),
}) as Awaited<ReturnType<typeof turtleFetch>>
);
});
afterEach(async () => {
jest.useRealTimers();
await Promise.all(
directories.splice(0).map(directory => rm(directory, { recursive: true, force: true }))
);
});
it('records by default with no preview and finalizes before process stop, uploading exactly once', async () => {
const host = await startHostAsync();
expect(ngrok.forward).not.toHaveBeenCalled();
const directory = directories[0];
const recordings = [
{
udid: 'emulator-5554',
deviceName: 'Pixel',
runtimeDisplayName: 'Android 16',
directory: path.join(directory, 'session'),
},
];
await writeFile(path.join(directory, 'recordings.json'), JSON.stringify(recordings));
await mkdir(recordings[0].directory);
const started = deferred<void>();
const response = deferred<Awaited<ReturnType<typeof turtleFetch>>>();
jest.mocked(turtleFetch).mockImplementationOnce(async () => {
started.resolve();
return await response.promise;
});
const finishing = host.finishAsync();
expect(host.finishAsync()).toBe(finishing);
await started.promise;
expect(stopServer).not.toHaveBeenCalled();
expect(uploadDeviceRunSessionScreenRecordingsAsync).not.toHaveBeenCalled();
const token =
jest.mocked(spawnDetached).mock.calls[0][0].env.EXPO_DEVICE_HUB_RECORDING_CONTROL_TOKEN;
expect(token).toMatch(/^[a-f0-9]{64}$/);
expect(turtleFetch).toHaveBeenLastCalledWith(
expect.stringMatching(/\/_eas\/android-recording\/stop$/),
'POST',
{
headers: { Authorization: `Bearer ${token}` },
retries: 0,
shouldThrowOnNotOk: false,
signal: expect.any(AbortSignal),
}
);
response.resolve({ ok: true } as Awaited<ReturnType<typeof turtleFetch>>);
await finishing;
await host.finishAsync();
expect(stopServer).toHaveBeenCalledTimes(1);
expect(uploadDeviceRunSessionScreenRecordingsAsync).toHaveBeenCalledTimes(1);
expect(uploadDeviceRunSessionScreenRecordingsAsync).toHaveBeenCalledWith(ctx, {
logger,
deviceRunSessionId: 'drs-id',
recordings,
});
expect(stopServer.mock.invocationCallOrder[0]).toBeLessThan(
jest.mocked(uploadDeviceRunSessionScreenRecordingsAsync).mock.invocationCallOrder[0]
);
await expect(host.openPreviewAsync({ baseDomain })).rejects.toThrow(
'after session host finalization'
);
});
it('coalesces preview opens and allows close/reopen without ending capture', async () => {
const host = await startHostAsync();
const opening = host.openPreviewAsync({ baseDomain });
expect(host.openPreviewAsync({ baseDomain })).toBe(opening);
const first = await opening;
const closing = first.closeAsync();
expect(first.closeAsync()).toBe(closing);
await closing;
const secondClose = jest.fn().mockResolvedValue(undefined);
jest.mocked(ngrok.forward).mockResolvedValueOnce({
url: () => 'https://replacement.example.test',
close: secondClose,
} as never);
const second = await host.openPreviewAsync({ baseDomain });
await first.closeAsync();
expect(await host.openPreviewAsync({ baseDomain })).toBe(second);
expect(secondClose).not.toHaveBeenCalled();
expect(spawnDetached).toHaveBeenCalledTimes(1);
expect(stopServer).not.toHaveBeenCalled();
expect(uploadDeviceRunSessionScreenRecordingsAsync).not.toHaveBeenCalled();
expect(jest.mocked(turtleFetch).mock.calls.some(([, method]) => method === 'POST')).toBe(false);
await host.finishAsync();
expect(closeTunnel).toHaveBeenCalledTimes(1);
expect(secondClose).toHaveBeenCalledTimes(1);
expect(stopServer).toHaveBeenCalledTimes(1);
});
it('retries a failed tunnel without restarting the host', async () => {
const host = await startHostAsync();
jest.mocked(ngrok.forward).mockRejectedValueOnce(new Error('tunnel unavailable'));
await expect(host.openPreviewAsync({ baseDomain })).rejects.toThrow('tunnel unavailable');
expect(stopServer).not.toHaveBeenCalled();
await host.openPreviewAsync({ baseDomain });
expect(spawnDetached).toHaveBeenCalledTimes(1);
await host.finishAsync();
expect(closeTunnel).toHaveBeenCalledTimes(1);
});
it('drains an in-flight tunnel when finishing and rejects new opens', async () => {
const host = await startHostAsync();
const tunnel = deferred<Awaited<ReturnType<typeof ngrok.forward>>>();
jest.mocked(ngrok.forward).mockReturnValueOnce(tunnel.promise);
const opening = host.openPreviewAsync({ baseDomain });
const finishing = host.finishAsync();
await expect(host.openPreviewAsync({ baseDomain })).rejects.toThrow(
'after session host finalization'
);
tunnel.resolve({ url: () => 'https://late.example.test', close: closeTunnel } as never);
await expect(opening).rejects.toThrow('finalized while the preview was opening');
await finishing;
expect(closeTunnel).toHaveBeenCalledTimes(1);
expect(stopServer).toHaveBeenCalledTimes(1);
});
it('keeps cleanup and upload best-effort when tunnel close and finalization fail', async () => {
jest.mocked(spawnDetached).mockImplementationOnce(options => {
directories.push(options.args[options.args.indexOf('--android-recording-directory') + 1]);
return {
pid: undefined,
getOutput: () => '[serve-emu] emulator-5554 capture error: scrcpy exited with code 255',
getExitError: () => undefined,
stopAsync: stopServer,
};
});
const host = await startHostAsync();
await writeFile(path.join(directories[0], 'recordings.json'), '[]');
await host.openPreviewAsync({ baseDomain });
closeTunnel.mockRejectedValueOnce(new Error('tunnel close failure'));
jest.mocked(turtleFetch).mockResolvedValueOnce({
ok: false,
status: 500,
text: async () => 'Error: scrcpy exited with code 255',
} as Awaited<ReturnType<typeof turtleFetch>>);
await host.finishAsync();
expect(stopServer).toHaveBeenCalledTimes(1);
expect(logger.warn).toHaveBeenCalledWith(
{
err: expect.objectContaining({
message: expect.stringContaining('HTTP 500: Error: scrcpy exited with code 255'),
}),
},
'Could not finalize Android recording before shutdown.'
);
expect(Sentry.capture).toHaveBeenCalledWith(
'Could not finalize Android recording before shutdown',
expect.any(Error),
{ level: 'warning' }
);
expect(uploadDeviceRunSessionScreenRecordingsAsync).toHaveBeenCalledTimes(1);
expect(logger.warn).toHaveBeenCalledWith(
{ hostOutput: '[serve-emu] emulator-5554 capture error: scrcpy exited with code 255' },
'Session host output around the recording failure.'
);
});
it('logs the reason and skips upload and output dump when the Hub never recorded', async () => {
jest.mocked(spawnDetached).mockImplementationOnce(options => {
directories.push(options.args[options.args.indexOf('--android-recording-directory') + 1]);
return {
pid: undefined,
getOutput: () => '[serve-emu] Android recording skipped',
getExitError: () => undefined,
stopAsync: stopServer,
};
});
const host = await startHostAsync();
jest.mocked(turtleFetch).mockResolvedValueOnce({
ok: false,
status: 409,
text: async () => 'Android recording requires exactly one booted emulator; found 0.',
} as Awaited<ReturnType<typeof turtleFetch>>);
await host.finishAsync();
expect(stopServer).toHaveBeenCalledTimes(1);
expect(logger.warn).toHaveBeenCalledWith(
'Android recording was not captured: Android recording requires exactly one booted emulator; found 0.'
);
expect(uploadDeviceRunSessionScreenRecordingsAsync).not.toHaveBeenCalled();
expect(logger.warn).not.toHaveBeenCalledWith(
expect.anything(),
'Session host output around the recording failure.'
);
});
it('rolls back failed host startup without replacing the original error', async () => {
stopServer.mockRejectedValueOnce(new Error('stop failure'));
await expect(
startDeviceSessionHostAsync(ctx, {
runtimePlatform: BuildRuntimePlatform.LINUX,
env,
logger,
timeoutMs: 0,
})
).rejects.toThrow('Timed out waiting');
expect(stopServer).toHaveBeenCalledTimes(1);
const directory =
jest.mocked(spawnDetached).mock.calls[0][0].env.EXPO_DEVICE_HUB_SCREENSHOT_DIRECTORY;
if (!directory) {
throw new Error('Missing screenshot artifact directory');
}
// The host could not be stopped, so it may still write captures there.
await expect(access(directory)).resolves.toBeUndefined();
expect(ngrok.forward).not.toHaveBeenCalled();
expect(jest.mocked(turtleFetch).mock.calls.some(([, method]) => method === 'POST')).toBe(false);
expect(uploadDeviceRunSessionScreenRecordingsAsync).not.toHaveBeenCalled();
const hostStillRunning =
'The session host is still running, so preview screenshots it saves from now on are not uploaded.';
expect(logger.warn).toHaveBeenCalledWith({ directory }, hostStillRunning);
const recordingWarnings = jest
.mocked(logger.warn)
.mock.calls.map(([, message]) => message)
.filter(message => message !== hostStillRunning);
expect(recordingWarnings).not.toContainEqual(expect.stringMatching(/finalize|upload/));
});
async function writeRecordingDescriptorAsync(directory: string) {
const child = path.join(directory, 'session');
await mkdir(child);
await writeFile(
path.join(directory, 'recordings.json'),
JSON.stringify([
{
udid: 'emulator-5554',
deviceName: 'Pixel',
runtimeDisplayName: 'Android',
directory: child,
},
])
);
}
it.each([true, false])(
'removes its recording root only when every upload succeeds (%s)',
async uploaded => {
const host = await startHostAsync();
const directory = directories[0];
await writeRecordingDescriptorAsync(directory);
jest.mocked(uploadDeviceRunSessionScreenRecordingsAsync).mockResolvedValueOnce(uploaded);
await host.finishAsync();
if (uploaded) {
await expect(access(directory)).rejects.toMatchObject({ code: 'ENOENT' });
} else {
await expect(access(directory)).resolves.toBeUndefined();
}
}
);
it('finishes despite a stalled tunnel close', async () => {
const host = await startHostAsync();
await host.openPreviewAsync({ baseDomain });
const closing = deferred<void>();
closeTunnel.mockReturnValueOnce(closing.promise);
jest.useFakeTimers();
const finishing = host.finishAsync();
await jest.advanceTimersByTimeAsync(5_000);
await finishing;
expect(stopServer).toHaveBeenCalledTimes(1);
expect(logger.warn).toHaveBeenCalledWith(expect.anything(), expect.stringContaining('deadline'));
closing.resolve();
});
it('aborts a stalled finalization request before stopping the host', async () => {
const host = await startHostAsync();
const response = deferred<Awaited<ReturnType<typeof turtleFetch>>>();
jest.mocked(turtleFetch).mockReturnValueOnce(response.promise);
jest.useFakeTimers();
const finishing = host.finishAsync();
await jest.advanceTimersByTimeAsync(60_000);
await finishing;
const options = jest.mocked(turtleFetch).mock.calls.at(-1)?.[2];
expect(options?.signal?.aborted).toBe(true);
expect(stopServer).toHaveBeenCalledTimes(1);
response.resolve({ ok: true } as Awaited<ReturnType<typeof turtleFetch>>);
});
it('uploads the partial recording a killed host left behind and then removes the root', async () => {
const host = await startHostAsync();
const directory = directories[0];
const child = path.join(directory, 'session');
await mkdir(child);
await writeFile(path.join(directory, 'recordings.json'), '[]');
const partial = [
{ udid: 'emulator-5554', deviceName: 'Pixel', runtimeDisplayName: 'Android', directory: child },
];
jest.mocked(findUnlistedDeviceScreenRecordingsAsync).mockResolvedValueOnce(partial);
jest.mocked(uploadDeviceRunSessionScreenRecordingsAsync).mockResolvedValueOnce(true);
await host.finishAsync();
expect(findUnlistedDeviceScreenRecordingsAsync).toHaveBeenCalledWith({
root: directory,
env,
logger,
});
expect(uploadDeviceRunSessionScreenRecordingsAsync).toHaveBeenCalledWith(ctx, {
logger,
deviceRunSessionId: 'drs-id',
recordings: partial,
});
await expect(access(directory)).rejects.toMatchObject({ code: 'ENOENT' });
});
it('rejects descriptors outside its recording root', async () => {
const host = await startHostAsync();
const directory = directories[0];
await writeFile(
path.join(directory, 'recordings.json'),
JSON.stringify([
{
udid: 'emulator-5554',
deviceName: 'Pixel',
runtimeDisplayName: 'Android',
directory: path.dirname(directory),
},
])
);
await host.finishAsync();
expect(uploadDeviceRunSessionScreenRecordingsAsync).not.toHaveBeenCalled();
await expect(access(directory)).resolves.toBeUndefined();
});
it('retains recordings without uploading if the host cannot be stopped', async () => {
const host = await startHostAsync();
const directory = directories[0];
await writeRecordingDescriptorAsync(directory);
stopServer.mockRejectedValueOnce(new Error('process stop failed'));
await host.finishAsync();
expect(uploadDeviceRunSessionScreenRecordingsAsync).not.toHaveBeenCalled();
await expect(access(directory)).resolves.toBeUndefined();
});
it('closes a listener that arrives after finish has already timed out waiting for it', async () => {
const host = await startHostAsync();
const tunnel = deferred<Awaited<ReturnType<typeof ngrok.forward>>>();
jest.mocked(ngrok.forward).mockReturnValueOnce(tunnel.promise);
const opening = host.openPreviewAsync({ baseDomain });
const rejected = expect(opening).rejects.toThrow('finalized while the preview was opening');
jest.useFakeTimers();
const finishing = host.finishAsync();
await jest.advanceTimersByTimeAsync(5_000);
await finishing;
expect(stopServer).toHaveBeenCalledTimes(1);
tunnel.resolve({ url: () => 'https://late.example.test', close: closeTunnel } as never);
await rejected;
expect(closeTunnel).toHaveBeenCalledTimes(1);
});
it('leaves iOS recording to its existing build steps', async () => {
const host = await startDeviceSessionHostAsync(ctx, {
runtimePlatform: BuildRuntimePlatform.DARWIN,
env,
logger,
timeoutMs: 10_000,
});
const options = jest.mocked(spawnDetached).mock.calls[0][0];
expect(options.args).not.toContain('--android-recording-directory');
expect(options.env.EXPO_DEVICE_HUB_RECORDING_CONTROL_TOKEN).toBeUndefined();
await host.finishAsync();
expect(jest.mocked(turtleFetch).mock.calls.some(([, method]) => method === 'POST')).toBe(false);
expect(uploadDeviceRunSessionScreenRecordingsAsync).not.toHaveBeenCalled();
expect(stopServer).toHaveBeenCalledTimes(1);
});
it.each([BuildRuntimePlatform.LINUX, BuildRuntimePlatform.DARWIN])(
'uploads screenshots and flushes them exactly once when the %s host finishes',
async runtimePlatform => {
const uploads: Buffer[] = [];
jest
.mocked(uploadDeviceRunSessionArtifactAsync)
.mockImplementation(async (_ctx, { stream }) => {
const chunks: Buffer[] = [];
for await (const chunk of stream) {
chunks.push(Buffer.from(chunk));
}
uploads.push(Buffer.concat(chunks));
});
const host = await startDeviceSessionHostAsync(ctx, {
runtimePlatform,
env,
logger,
timeoutMs: 10_000,
});
await host.openPreviewAsync({ baseDomain });
const directory =
jest.mocked(spawnDetached).mock.calls[0][0].env.EXPO_DEVICE_HUB_SCREENSHOT_DIRECTORY;
if (!directory) {
throw new Error('Missing screenshot artifact directory');
}
await writeFile(
path.join(directory, 'screenshot-2026-09-24T08-45-59-123Z-a1b2c3d4e5f6.png'),
'manual-capture'
);
let directoryWhenStopping: string[] | undefined;
stopServer.mockImplementationOnce(async () => {
directoryWhenStopping = await readdir(directory);
});
const finishing = host.finishAsync();
expect(host.finishAsync()).toBe(finishing);
await finishing;
expect(uploads).toEqual([Buffer.from('manual-capture')]);
expect(stopServer).toHaveBeenCalledTimes(1);
expect(closeTunnel).toHaveBeenCalledTimes(1);
// The collector finishes, and removes its directory, only after the host has stopped.
expect(directoryWhenStopping).toBeDefined();
await expect(access(directory)).rejects.toMatchObject({ code: 'ENOENT' });
}
);
@@ -1,4 +1,5 @@
import { type bunyan } from '@expo/logger';
import spawn from '@expo/turtle-spawn';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
@@ -7,10 +8,13 @@ import { spawnDetached } from '../remoteDeviceRunSession';
import {
CHISEL_VERSION,
LOCAL_EGRESS_NO_PROXY,
LOCAL_EGRESS_PROXY_PORT,
buildEgressRemoteConfigFields,
buildLocalEgressSimulatorEnvironment,
buildNetworksetupProxyArgs,
collectSimulatorProcessIds,
configureSimulatorProxyEnvironmentAsync,
createChiselAuthfileContents,
getChiselAssetName,
getChiselDownloadUrl,
@@ -313,6 +317,125 @@ describe('local egress handoff', () => {
});
});
describe(buildLocalEgressSimulatorEnvironment, () => {
it('points every proxy variable at the loopback port and excludes loopback', () => {
expect(buildLocalEgressSimulatorEnvironment(8899)).toEqual({
http_proxy: 'http://127.0.0.1:8899',
https_proxy: 'http://127.0.0.1:8899',
HTTP_PROXY: 'http://127.0.0.1:8899',
HTTPS_PROXY: 'http://127.0.0.1:8899',
grpc_proxy: 'http://127.0.0.1:8899',
no_proxy: LOCAL_EGRESS_NO_PROXY,
NO_PROXY: LOCAL_EGRESS_NO_PROXY,
});
});
});
describe(configureSimulatorProxyEnvironmentAsync, () => {
const mockedSpawn = jest.mocked(spawn);
let tempDir: string;
let logger: bunyan;
beforeEach(async () => {
mockedSpawn.mockReset();
mockedSpawn.mockResolvedValue({ stdout: '', stderr: '' } as any);
tempDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'local-egress-env-test-'));
logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
});
afterEach(async () => {
await fs.promises.rm(tempDir, { recursive: true, force: true });
});
it('does nothing when no local egress session is active', async () => {
await expect(
configureSimulatorProxyEnvironmentAsync({
udid: 'test-udid' as any,
env: process.env,
logger,
handoffPath: path.join(tempDir, 'missing.json'),
})
).resolves.toBe(false);
expect(mockedSpawn).not.toHaveBeenCalled();
expect(logger.warn).not.toHaveBeenCalled();
});
it('sets the proxy environment in the simulator when a handoff exists', async () => {
const handoffPath = path.join(tempDir, 'handoff.json');
await writeLocalEgressHandoffAsync(
{ url: 'https://egress.example', token: 'pw', fingerprint: 'fp=', port: 8899 },
handoffPath
);
await expect(
configureSimulatorProxyEnvironmentAsync({
udid: 'test-udid' as any,
env: process.env,
logger,
handoffPath,
})
).resolves.toBe(true);
expect(mockedSpawn.mock.calls.map(([, args]) => args)).toEqual([
[
'simctl',
'spawn',
'test-udid',
'launchctl',
'setenv',
...Object.entries(buildLocalEgressSimulatorEnvironment(8899)).flat(),
],
]);
expect(logger.info).toHaveBeenCalledWith(
expect.stringContaining('proxy environment variables set')
);
});
it('warns and continues when launchctl fails', async () => {
const handoffPath = path.join(tempDir, 'handoff.json');
await writeLocalEgressHandoffAsync(
{ url: 'https://egress.example', token: 'pw', fingerprint: 'fp=', port: 8899 },
handoffPath
);
mockedSpawn.mockRejectedValue(new Error('launchctl failed'));
await expect(
configureSimulatorProxyEnvironmentAsync({
udid: 'test-udid' as any,
env: process.env,
logger,
handoffPath,
})
).resolves.toBe(false);
expect(logger.warn).toHaveBeenCalledWith(
{ err: expect.any(Error) },
expect.stringContaining('will bypass local egress and exit from this worker')
);
});
it('warns and continues when the handoff is malformed', async () => {
const handoffPath = path.join(tempDir, 'handoff.json');
await fs.promises.writeFile(handoffPath, JSON.stringify({ url: 'x' }), 'utf8');
await expect(
configureSimulatorProxyEnvironmentAsync({
udid: 'test-udid' as any,
env: process.env,
logger,
handoffPath,
})
).resolves.toBe(false);
expect(mockedSpawn).not.toHaveBeenCalled();
expect(logger.warn).toHaveBeenCalledWith(
{ err: expect.any(Error) },
expect.stringContaining('could not read the local egress handoff')
);
});
});
describe(startChiselServerAsync, () => {
const options = {
chiselPath: '/tmp/chisel',
@@ -331,6 +454,7 @@ describe(startChiselServerAsync, () => {
jest.mocked(spawnDetached).mockReturnValue({
pid: 12345,
getOutput: () => output,
getExitError: () => undefined,
stopAsync,
});
});
@@ -0,0 +1,426 @@
/**
* End-to-end test of the local egress guard against a real iOS Simulator.
* Needs Xcode with a simulator runtime, a shut-down iPhone simulator, and
* network access. Opt in with EAS_LOCAL_EGRESS_E2E=1; the test-egress-guard
* EAS workflow runs it on macOS.
*
* Variables handed to launchd at boot cannot be changed afterwards with
* `launchctl setenv`, so each configuration gets its own boot.
*/
import { type bunyan } from '@expo/logger';
import spawn from '@expo/turtle-spawn';
import dgram from 'node:dgram';
import fs from 'node:fs';
import http from 'node:http';
import net from 'node:net';
import os from 'node:os';
import path from 'node:path';
import { setTimeout as sleep } from 'node:timers/promises';
import { IosSimulatorUtils, type IosSimulatorUuid } from '../../../utils/IosSimulatorUtils';
import { writeLocalEgressHandoffAsync } from '../localEgress';
import {
type GuardEvent,
installLocalEgressGuardAsync,
parseGuardLogLine,
reportLocalEgressGuardCoverageAsync,
resolveEgressGuardCheckAsync,
resolveEgressGuardLibraryAsync,
resolveLocalEgressBootEnvironmentAsync,
stopLocalEgressGuardRelaysAsync,
verifyLocalEgressGuardAsync,
} from '../localEgressGuard';
jest.unmock('fs');
jest.unmock('node:fs');
jest.unmock('fs/promises');
jest.unmock('node:fs/promises');
jest.setTimeout(15 * 60 * 1000);
const enabled = process.platform === 'darwin' && process.env.EAS_LOCAL_EGRESS_E2E === '1';
const describeE2E = enabled ? describe : describe.skip;
const GUARD_DIR = path.join(__dirname, '..', '..', '..', '..', 'resources', 'egress-guard');
const env = process.env;
function createLogger(): bunyan & { lines: string[] } {
const lines: string[] = [];
const record = (a: unknown, b?: unknown) => {
lines.push(typeof a === 'string' ? a : String(b));
};
return { info: record, warn: record, debug: record, lines } as any;
}
async function readEventsAsync(logPath: string): Promise<GuardEvent[]> {
let raw = '';
try {
raw = await fs.promises.readFile(logPath, 'utf8');
} catch {
return [];
}
return raw
.split('\n')
.map(parseGuardLogLine)
.filter((e): e is GuardEvent => e !== null);
}
async function waitForAsync<T>(
probe: () => Promise<T | undefined>,
{ timeoutMs, intervalMs = 1000 }: { timeoutMs: number; intervalMs?: number }
): Promise<T | undefined> {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
const value = await probe();
if (value !== undefined) {
return value;
}
await sleep(intervalMs);
}
return undefined;
}
describeE2E('local egress guard in a simulator', () => {
let workDir: string;
let udid: IosSimulatorUuid;
let libraryPath: string;
let checkPath: string;
let nettestPath: string;
let handoffPath: string;
let proxyServer: http.Server;
let proxyPort: number;
let udpServer: dgram.Socket;
let udpPort: number;
const logger = createLogger();
/** Shut the device down and boot it again with the given launchd environment. */
async function rebootAsync(launchdEnvironment: Record<string, string>): Promise<void> {
await spawn('xcrun', ['simctl', 'shutdown', udid], { env, stdio: 'pipe' }).catch(() => {});
await IosSimulatorUtils.bootAsync({ deviceIdentifier: udid, env, launchdEnvironment });
await IosSimulatorUtils.startAsync({ deviceIdentifier: udid, env });
}
async function bootEnvironmentAsync(
mode: 'block' | 'log',
logPath: string
): Promise<Record<string, string>> {
const variables = await resolveLocalEgressBootEnvironmentAsync({
handoffPath,
libraryPath,
logPath,
mode,
});
if (!variables) {
throw new Error('The handoff was not written.');
}
return variables;
}
async function runNettestAsync(): Promise<Record<string, string>> {
const result = await spawn(
'xcrun',
[
'simctl',
'spawn',
udid,
nettestPath,
'--proxy-port',
String(proxyPort),
'--udp-port',
String(udpPort),
],
{ env, stdio: 'pipe' }
);
const last = result.stdout.trim().split('\n').at(-1) ?? '{}';
return JSON.parse(last);
}
beforeAll(async () => {
workDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'egress-guard-e2e-'));
// The guard library and self-check, built on demand.
let resolved = await resolveEgressGuardLibraryAsync();
if (!resolved) {
await spawn('bash', [path.join(GUARD_DIR, 'build.sh')], { env, stdio: 'pipe' });
resolved = await resolveEgressGuardLibraryAsync();
}
if (!resolved) {
throw new Error('egress-guard.dylib is not available and could not be built.');
}
libraryPath = resolved;
const resolvedCheck = await resolveEgressGuardCheckAsync();
if (!resolvedCheck) {
throw new Error('egress-guard-check is not available and could not be built.');
}
checkPath = resolvedCheck;
// The probe binary, compiled for the simulator running on this host.
nettestPath = path.join(workDir, 'nettest');
await spawn(
'xcrun',
[
'-sdk',
'iphonesimulator',
'swiftc',
'-target',
`${process.arch === 'arm64' ? 'arm64' : 'x86_64'}-apple-ios15.0-simulator`,
path.join(GUARD_DIR, 'tests', 'nettest.swift'),
'-o',
nettestPath,
],
{ env, stdio: 'pipe' }
);
handoffPath = path.join(workDir, 'handoff.json');
await writeLocalEgressHandoffAsync(
{ url: 'https://egress.example', token: 'pw', fingerprint: 'fp=', port: 8899 },
handoffPath
);
// A shut-down iPhone the test can boot as many times as it needs. A device
// someone else booted is left alone. Nothing is ever deleted.
const booted = new Set(
(await IosSimulatorUtils.getAvailableDevicesAsync({ env, filter: 'booted' })).map(d => d.udid)
);
const available = await IosSimulatorUtils.getAvailableDevicesAsync({
env,
filter: 'available',
});
const iphone =
available.find(d => /^iPhone \d/.test(d.name) && !booted.has(d.udid)) ??
available.find(d => /^iPhone/.test(d.name) && !booted.has(d.udid));
if (!iphone) {
throw new Error('No shut-down iPhone simulator is available for the test to boot.');
}
udid = iphone.udid;
// Loopback stand-ins for the egress proxy and for a local UDP service.
proxyServer = http.createServer((_req, res) => {
res.writeHead(502);
res.end();
});
proxyServer.on('connect', (req, socket, head) => {
const [host, port] = String(req.url).split(':');
const upstream = net.connect(Number(port) || 443, host, () => {
socket.write('HTTP/1.1 200 Connection Established\r\n\r\n');
if (head.length) {
upstream.write(new Uint8Array(head));
}
upstream.pipe(socket);
socket.pipe(upstream);
});
upstream.on('error', () => socket.destroy());
socket.on('error', () => upstream.destroy());
});
await new Promise<void>(resolve => proxyServer.listen(0, '127.0.0.1', () => resolve()));
proxyPort = (proxyServer.address() as net.AddressInfo).port;
udpServer = dgram.createSocket('udp4');
udpServer.on('message', (msg, rinfo) =>
udpServer.send(new Uint8Array(msg), rinfo.port, rinfo.address)
);
await new Promise<void>(resolve => udpServer.bind(0, '127.0.0.1', () => resolve()));
udpPort = udpServer.address().port;
});
afterAll(async () => {
await stopLocalEgressGuardRelaysAsync(logger);
proxyServer?.close();
udpServer?.close();
if (udid) {
await spawn('xcrun', ['simctl', 'shutdown', udid], { env, stdio: 'pipe' }).catch(() => {});
}
if (workDir) {
await fs.promises.rm(workDir, { recursive: true, force: true });
}
});
describe('booted with the guard in launchd, as the worker boots', () => {
let logPath: string;
beforeAll(async () => {
logPath = path.join(workDir, 'boot.log');
await rebootAsync(await bootEnvironmentAsync('block', logPath));
await verifyLocalEgressGuardAsync({ udid, env, logger, mode: 'block', checkPath });
});
it('covers every process of the boot', async () => {
const coverage = await reportLocalEgressGuardCoverageAsync({ env, logger });
expect(coverage).not.toBeNull();
expect(coverage!.covered.length).toBeGreaterThan(50);
expect(coverage!.uncovered).toEqual([]);
// Refusals were recorded by processes that started during the boot, before
// any launchctl call could have reached them.
expect((await readEventsAsync(logPath)).length).toBeGreaterThan(0);
});
it('refuses every direct path and leaves loopback and DNS alone', async () => {
const results = await runNettestAsync();
expect(results).toMatchObject({
urlsessionDirect: 'refused',
urlsessionProxied: 'ok(200)',
bsdConnectDirect: 'refused',
dns: 'ok',
udpDirect: 'refused',
udpLoopback: 'ok',
literalFirst: 'refused',
literalSecond: 'refused',
unspecConnect: 'refused',
udpDisconnect: 'ok',
nocancelSendto: 'refused',
});
expect(results.nwconnectionDirect).not.toMatch(/^ok|^timeout/);
const events = (await readEventsAsync(logPath)).filter(e => e.process === 'nettest');
expect(events.length).toBeGreaterThanOrEqual(4);
expect(events.every(e => e.action === 'blocked')).toBe(true);
// URLSession and Network.framework connects are caught inside Apple's frameworks.
expect(
events.some(e => e.function === 'connect' && e.callers.some(c => /CFNetwork/.test(c)))
).toBe(true);
expect(
events.some(
e =>
e.function === 'connect' &&
e.callers.some(c => /Network/.test(c) && !/CFNetwork/.test(c))
)
).toBe(true);
// UDP is covered, through the cancelable and non-cancelable entry points.
expect(events.some(e => e.function === 'sendto' && e.peer === '8.8.8.8:53')).toBe(true);
// An AF_UNSPEC sockaddr is recorded as what the kernel would have used.
expect(events.some(e => e.function === 'connect' && e.peer === '1.0.0.1:443')).toBe(true);
// The callers name the process and frameworks, never the guard itself.
expect(events.every(e => !e.callers.some(c => /egress-guard/.test(c)))).toBe(true);
// One line per destination per process, even when the process retries.
expect(events.filter(e => e.function === 'connect' && e.peer === '1.1.1.1:443')).toHaveLength(
1
);
// Nothing on loopback is ever reported.
expect(events.some(e => /^127\./.test(e.peer))).toBe(false);
});
it('guards processes the simulator launches on its own', async () => {
await spawn('xcrun', ['simctl', 'openurl', udid, 'https://example.net/'], {
env,
stdio: 'pipe',
});
const daemonEvent = await waitForAsync(
async () =>
(await readEventsAsync(logPath)).find(
e => e.process !== 'nettest' && e.action === 'blocked'
),
{ timeoutMs: 60_000 }
);
expect(daemonEvent).toBeDefined();
});
it('does not interfere with the simulator tooling the worker relies on', async () => {
const screenshot = path.join(workDir, 'screenshot.png');
for (const args of [
['simctl', 'spawn', udid, 'launchctl', 'list'],
[
'simctl',
'spawn',
udid,
'defaults',
'write',
'dev.expo.egress-guard-test',
'key',
'-string',
'value',
],
['simctl', 'spawn', udid, '/usr/bin/env'],
['simctl', 'launch', udid, 'com.apple.Preferences'],
['simctl', 'io', udid, 'screenshot', screenshot],
]) {
await expect(spawn('xcrun', args, { env, stdio: 'pipe' })).resolves.toBeDefined();
}
expect(fs.existsSync(screenshot)).toBe(true);
});
});
describe('booted in log mode', () => {
let logPath: string;
beforeAll(async () => {
logPath = path.join(workDir, 'log-mode.log');
await rebootAsync(await bootEnvironmentAsync('log', logPath));
await verifyLocalEgressGuardAsync({ udid, env, logger, mode: 'log', checkPath });
});
it('observes without refusing', async () => {
const results = await runNettestAsync();
expect(results.urlsessionDirect).toBe('ok(200)');
expect(results.bsdConnectDirect).toBe('ok');
expect(results.udpDirect).toBe('sent');
const events = (await readEventsAsync(logPath)).filter(e => e.process === 'nettest');
expect(events.length).toBeGreaterThan(0);
expect(events.every(e => e.action === 'logged')).toBe(true);
});
});
describe('booted without the guard, as a device that was already running', () => {
beforeAll(async () => {
await rebootAsync({});
});
it('is reported as not in effect by the self-check', async () => {
await expect(
verifyLocalEgressGuardAsync({ udid, env, logger, mode: 'block', checkPath })
).rejects.toThrow(/not in effect/);
});
it('can be installed after boot through launchctl for processes started from then on', async () => {
const logPath = path.join(workDir, 'install.log');
expect(
await installLocalEgressGuardAsync({
udid,
env,
logger,
handoffPath,
libraryPath,
logPath,
mode: 'block',
tailIntervalMs: 100,
})
).toBe(true);
await verifyLocalEgressGuardAsync({ udid, env, logger, mode: 'block', checkPath });
const results = await runNettestAsync();
expect(results.urlsessionDirect).toBe('refused');
expect(results.bsdConnectDirect).toBe('refused');
// The relay turns the guard's events into session log lines.
const relayed = await waitForAsync(
async () =>
logger.lines.find(l =>
/refused connect from nettest \(pid \d+\) to 1\.1\.1\.1:443/.test(l)
),
{ timeoutMs: 5_000, intervalMs: 200 }
);
expect(relayed).toBeDefined();
// Processes from before the install are the uncovered set the coverage
// report names; nettest itself was covered.
const coverage = await reportLocalEgressGuardCoverageAsync({ env, logger });
expect(coverage!.uncovered.length).toBeGreaterThan(0);
});
});
describe('booted with an unwritable event log', () => {
beforeAll(async () => {
await rebootAsync(
await bootEnvironmentAsync('block', path.join(workDir, 'no-such-dir', 'guard.log'))
);
});
it('still refuses', async () => {
await verifyLocalEgressGuardAsync({ udid, env, logger, mode: 'block', checkPath });
const results = await runNettestAsync();
expect(results.urlsessionDirect).toBe('refused');
expect(results.bsdConnectDirect).toBe('refused');
expect(results.urlsessionProxied).toBe('ok(200)');
});
});
});
@@ -0,0 +1,582 @@
import { SystemError } from '@expo/eas-build-job';
import { type bunyan } from '@expo/logger';
import spawn from '@expo/turtle-spawn';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { setTimeout as sleep } from 'node:timers/promises';
import { IosSimulatorUtils } from '../../../utils/IosSimulatorUtils';
import { writeLocalEgressHandoffAsync } from '../localEgress';
import {
EGRESS_GUARD_LOG_ENV,
EGRESS_GUARD_MODE_ENV,
GuardEventRelay,
GuardLogTailer,
buildGuardLaunchdEnvironment,
installLocalEgressGuardAsync,
mergeGuardCoverageSamples,
parseGuardCoverage,
parseGuardLogLine,
rebindLocalEgressGuardRelays,
resolveEgressGuardLibraryAsync,
resolveLocalEgressBootEnvironmentAsync,
stopLocalEgressGuardRelaysAsync,
verifyLocalEgressGuardAsync,
} from '../localEgressGuard';
jest.mock('@expo/turtle-spawn');
jest.mock('../../../utils/IosSimulatorUtils', () => ({
IosSimulatorUtils: { setLaunchdEnvironmentAsync: jest.fn() },
}));
const mockedSetEnv = jest.mocked(IosSimulatorUtils.setLaunchdEnvironmentAsync);
const mockedSpawn = jest.mocked(spawn);
function createLogger(): bunyan & { lines: { level: string; msg: string }[] } {
const lines: { level: string; msg: string }[] = [];
const record = (level: string) => (a: unknown, b?: unknown) =>
lines.push({ level, msg: typeof a === 'string' ? a : String(b) });
return { info: record('info'), warn: record('warn'), debug: record('debug'), lines } as any;
}
const handoff = { url: 'https://egress.example', token: 'pw', fingerprint: 'fp=', port: 8899 };
describe(buildGuardLaunchdEnvironment, () => {
it('inserts the library and passes the log path and mode', () => {
expect(
buildGuardLaunchdEnvironment({
libraryPath: '/w/bin/egress-guard.dylib',
logPath: '/tmp/guard.log',
mode: 'block',
})
).toEqual({
DYLD_INSERT_LIBRARIES: '/w/bin/egress-guard.dylib',
[EGRESS_GUARD_LOG_ENV]: '/tmp/guard.log',
[EGRESS_GUARD_MODE_ENV]: 'block',
});
});
});
describe(parseGuardLogLine, () => {
it('parses a blocked event with callers', () => {
expect(
parseGuardLogLine(
'eas-egress-guard\tMyApp\t4242\tconnect\tblocked\t93.184.216.34:443\tNetwork,CFNetwork,MyApp'
)
).toEqual({
process: 'MyApp',
pid: 4242,
function: 'connect',
action: 'blocked',
peer: '93.184.216.34:443',
callers: ['Network', 'CFNetwork', 'MyApp'],
});
});
it('parses a logged event without callers and IPv6 peers', () => {
expect(
parseGuardLogLine('eas-egress-guard\tdaemon\t7\tsendto\tlogged\t[2606:4700::1]:53\t')
).toEqual({
process: 'daemon',
pid: 7,
function: 'sendto',
action: 'logged',
peer: '[2606:4700::1]:53',
callers: [],
});
});
it('ignores lines that are not guard events', () => {
expect(parseGuardLogLine('')).toBeNull();
expect(parseGuardLogLine('something else\ta\tb')).toBeNull();
expect(
parseGuardLogLine('eas-egress-guard\tMyApp\tnot-a-pid\tconnect\tblocked\t1.1.1.1:443\t')
).toBeNull();
expect(
parseGuardLogLine('eas-egress-guard\tMyApp\t1\tconnect\tmaybe\t1.1.1.1:443\t')
).toBeNull();
});
});
describe(GuardEventRelay, () => {
const blocked = (process: string, peer: string, pid = 1) => ({
process,
pid,
function: 'connect',
action: 'blocked' as const,
peer,
callers: ['Network', 'CFNetwork'],
});
it('logs the first sighting of a process and destination, then only counts', () => {
const logger = createLogger();
const relay = new GuardEventRelay(logger);
relay.handle(blocked('MyApp', '1.1.1.1:443'));
relay.handle(blocked('MyApp', '1.1.1.1:443', 2));
relay.handle(blocked('MyApp', '2.2.2.2:443'));
expect(logger.lines.map(l => l.msg)).toEqual([
expect.stringContaining(
'refused connect from MyApp (pid 1) to 1.1.1.1:443; callers: Network, CFNetwork'
),
expect.stringContaining('refused connect from MyApp (pid 1) to 2.2.2.2:443'),
]);
expect(relay.summary()).toEqual({ blocked: 3, logged: 0, distinct: 2, suppressed: 0 });
});
it('omits the pid when the source has none and appends a note', () => {
const logger = createLogger();
const relay = new GuardEventRelay(logger);
relay.handle({
process: 'Android emulator',
pid: null,
function: 'udp',
action: 'blocked',
peer: '142.250.152.94:443',
callers: [],
note: 'QUIC',
});
expect(logger.lines.map(l => l.msg)).toEqual([
'Local egress guard: refused udp from Android emulator to 142.250.152.94:443 (QUIC)',
]);
});
it('words observed events differently and stops logging past the limit', () => {
const logger = createLogger();
const relay = new GuardEventRelay(logger, /* limit */ 1);
relay.handle({ ...blocked('MyApp', '1.1.1.1:443'), action: 'logged' });
relay.handle({ ...blocked('MyApp', '2.2.2.2:443'), action: 'logged' });
expect(logger.lines).toHaveLength(1);
expect(logger.lines[0].msg).toContain('observed connect from MyApp');
expect(relay.summary()).toEqual({ blocked: 0, logged: 2, distinct: 2, suppressed: 1 });
});
it('explains a process that filled its table instead of counting it as a destination', () => {
const logger = createLogger();
const relay = new GuardEventRelay(logger);
relay.handle(blocked('MyApp', '1.1.1.1:443'));
relay.handle({
...blocked('MyApp', '128 distinct destinations'),
function: 'overflow',
callers: [],
});
expect(logger.lines.at(-1)?.msg).toContain(
'MyApp (pid 1) reached the limit of 128 distinct destinations listed per process; further distinct destinations from it are refused but not listed'
);
expect(relay.summary()).toEqual({ blocked: 1, logged: 0, distinct: 1, suppressed: 0 });
});
it('ignores the self-check probe, which trips the guard on purpose', () => {
const logger = createLogger();
const relay = new GuardEventRelay(logger);
relay.handle(blocked('egress-guard-check', '192.0.2.1:9'));
expect(logger.lines).toHaveLength(0);
expect(relay.summary()).toEqual({ blocked: 0, logged: 0, distinct: 0, suppressed: 0 });
});
it('writes a summary line', () => {
const logger = createLogger();
const relay = new GuardEventRelay(logger);
relay.handle(blocked('MyApp', '1.1.1.1:443'));
relay.handle(blocked('MyApp', '1.1.1.1:443'));
relay.handle(blocked('daemon', '3.3.3.3:443', 9));
relay.logSummary();
expect(logger.lines.at(-1)?.msg).toContain(
'refused 3 connection attempt(s) to 2 distinct destination(s) from 2 process(es)'
);
});
});
describe(GuardLogTailer, () => {
let dir: string;
beforeEach(async () => {
dir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'guard-tailer-'));
});
afterEach(async () => {
await fs.promises.rm(dir, { recursive: true, force: true });
});
it('delivers whole lines as they are appended, keeps partial lines, and survives truncation', async () => {
const file = path.join(dir, 'guard.log');
const lines: string[] = [];
const tailer = new GuardLogTailer({ path: file, onLine: l => lines.push(l), intervalMs: 20 });
tailer.start();
await sleep(50); // file does not exist yet; must not throw
await fs.promises.writeFile(file, 'first\nsecond\npart', 'utf8');
await sleep(80);
expect(lines).toEqual(['first', 'second']);
await fs.promises.appendFile(file, 'ial\n', 'utf8');
await sleep(80);
expect(lines).toEqual(['first', 'second', 'partial']);
await fs.promises.writeFile(file, 'after-truncate\n', 'utf8');
await sleep(80);
expect(lines).toEqual(['first', 'second', 'partial', 'after-truncate']);
await tailer.stopAsync();
await fs.promises.appendFile(file, 'late\n', 'utf8');
await sleep(60);
expect(lines).not.toContain('late');
});
it('flushes what is already in the file when stopped', async () => {
const file = path.join(dir, 'guard.log');
const lines: string[] = [];
const tailer = new GuardLogTailer({
path: file,
onLine: l => lines.push(l),
intervalMs: 10_000,
});
tailer.start();
await fs.promises.writeFile(file, 'only\n', 'utf8');
await tailer.stopAsync();
expect(lines).toEqual(['only']);
});
});
describe(resolveEgressGuardLibraryAsync, () => {
it('returns null when the library is not built', async () => {
await expect(
resolveEgressGuardLibraryAsync(path.join(os.tmpdir(), 'no-such-dir'))
).resolves.toBeNull();
});
it('returns the packaged path when present', async () => {
const dir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'guard-bin-'));
const lib = path.join(dir, 'egress-guard.dylib');
await fs.promises.writeFile(lib, '');
await expect(resolveEgressGuardLibraryAsync(dir)).resolves.toBe(lib);
await fs.promises.rm(dir, { recursive: true, force: true });
});
});
describe(installLocalEgressGuardAsync, () => {
let dir: string;
let logger: ReturnType<typeof createLogger>;
beforeEach(async () => {
mockedSetEnv.mockReset();
mockedSetEnv.mockResolvedValue(undefined);
mockedSpawn.mockReset();
mockedSpawn.mockResolvedValue({ stdout: '', stderr: '' } as any);
dir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'guard-install-'));
logger = createLogger();
});
afterEach(async () => {
await stopLocalEgressGuardRelaysAsync(logger);
await fs.promises.rm(dir, { recursive: true, force: true });
});
it('does nothing without a local egress session', async () => {
await expect(
installLocalEgressGuardAsync({
udid: 'u' as any,
env: process.env,
logger,
handoffPath: path.join(dir, 'missing.json'),
libraryPath: path.join(dir, 'egress-guard.dylib'),
logPath: path.join(dir, 'guard.log'),
})
).resolves.toBe(false);
expect(mockedSetEnv).not.toHaveBeenCalled();
});
it('fails the session when the library is not available', async () => {
const handoffPath = path.join(dir, 'handoff.json');
await writeLocalEgressHandoffAsync(handoff, handoffPath);
await expect(
installLocalEgressGuardAsync({
udid: 'u' as any,
env: process.env,
logger,
handoffPath,
libraryPath: null,
logPath: path.join(dir, 'guard.log'),
})
).rejects.toThrow(SystemError);
expect(mockedSetEnv).not.toHaveBeenCalled();
});
it('logs the simulator processes that were already running and are not covered', async () => {
const handoffPath = path.join(dir, 'handoff.json');
await writeLocalEgressHandoffAsync(handoff, handoffPath);
const libraryPath = path.join(dir, 'egress-guard.dylib');
await fs.promises.writeFile(libraryPath, '');
mockedSpawn.mockResolvedValue({
stdout: [
' 1 0 /sbin/launchd',
' 4000 1 /Library/Developer/CoreSimulator/Volumes/iOS/Runtimes/x/sbin/launchd_sim',
' 4001 4000 /Library/Developer/CoreSimulator/Volumes/iOS/Runtimes/x/System/Library/CoreServices/SpringBoard.app/SpringBoard',
' 4002 4000 /Library/Developer/CoreSimulator/Volumes/iOS/Runtimes/x/usr/libexec/backboardd',
' 3758 1 node',
].join('\n'),
stderr: '',
} as any);
await installLocalEgressGuardAsync({
udid: 'u' as any,
env: process.env,
logger,
handoffPath,
libraryPath,
logPath: path.join(dir, 'guard.log'),
});
expect(
logger.lines.some(l =>
/2 simulator process\(es\) were already running .* not covered by it: SpringBoard, backboardd\./.test(
l.msg
)
)
).toBe(true);
});
it('sets the launchd environment, creates the log file, and relays events into the session log', async () => {
const handoffPath = path.join(dir, 'handoff.json');
await writeLocalEgressHandoffAsync(handoff, handoffPath);
const libraryPath = path.join(dir, 'egress-guard.dylib');
await fs.promises.writeFile(libraryPath, '');
const logPath = path.join(dir, 'guard.log');
await expect(
installLocalEgressGuardAsync({
udid: 'u' as any,
env: process.env,
logger,
handoffPath,
libraryPath,
logPath,
mode: 'block',
tailIntervalMs: 20,
})
).resolves.toBe(true);
expect(mockedSetEnv).toHaveBeenCalledWith({
udid: 'u',
env: process.env,
variables: buildGuardLaunchdEnvironment({ libraryPath, logPath, mode: 'block' }),
});
expect(fs.existsSync(logPath)).toBe(true);
expect(logger.lines.some(l => /guard installed/.test(l.msg) && /block/.test(l.msg))).toBe(true);
await fs.promises.appendFile(
logPath,
'eas-egress-guard\tMyApp\t4242\tconnect\tblocked\t1.1.1.1:443\tNetwork\n'
);
await sleep(80);
expect(
logger.lines.some(l =>
/refused connect from MyApp \(pid 4242\) to 1\.1\.1\.1:443/.test(l.msg)
)
).toBe(true);
await stopLocalEgressGuardRelaysAsync(logger);
expect(logger.lines.at(-1)?.msg).toContain('refused 1 connection attempt(s)');
});
it('attributes later events and the summary to a rebound logger', async () => {
const handoffPath = path.join(dir, 'handoff.json');
await writeLocalEgressHandoffAsync(handoff, handoffPath);
const libraryPath = path.join(dir, 'egress-guard.dylib');
await fs.promises.writeFile(libraryPath, '');
const logPath = path.join(dir, 'guard.log');
await installLocalEgressGuardAsync({
udid: 'u' as any,
env: process.env,
logger,
handoffPath,
libraryPath,
logPath,
tailIntervalMs: 20,
});
const sessionLogger = createLogger();
rebindLocalEgressGuardRelays(sessionLogger);
await fs.promises.appendFile(
logPath,
'eas-egress-guard\tMyApp\t4242\tconnect\tblocked\t1.1.1.1:443\tNetwork\n'
);
await sleep(80);
expect(sessionLogger.lines.some(l => /refused connect from MyApp/.test(l.msg))).toBe(true);
expect(logger.lines.some(l => /refused connect from MyApp/.test(l.msg))).toBe(false);
await stopLocalEgressGuardRelaysAsync(logger);
expect(sessionLogger.lines.at(-1)?.msg).toContain('refused 1 connection attempt(s)');
});
it('fails the session when launchctl fails', async () => {
const handoffPath = path.join(dir, 'handoff.json');
await writeLocalEgressHandoffAsync(handoff, handoffPath);
const libraryPath = path.join(dir, 'egress-guard.dylib');
await fs.promises.writeFile(libraryPath, '');
mockedSetEnv.mockRejectedValue(new Error('launchctl failed'));
await expect(
installLocalEgressGuardAsync({
udid: 'u' as any,
env: process.env,
logger,
handoffPath,
libraryPath,
logPath: path.join(dir, 'guard.log'),
})
).rejects.toThrow(/Could not install the local egress guard/);
});
});
describe(resolveLocalEgressBootEnvironmentAsync, () => {
let dir: string;
beforeEach(async () => {
dir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'guard-bootenv-'));
});
afterEach(async () => {
await fs.promises.rm(dir, { recursive: true, force: true });
});
it('returns null without a local egress session', async () => {
await expect(
resolveLocalEgressBootEnvironmentAsync({ handoffPath: path.join(dir, 'missing.json') })
).resolves.toBeNull();
});
it('fails when the guard library is not available', async () => {
const handoffPath = path.join(dir, 'handoff.json');
await writeLocalEgressHandoffAsync(handoff, handoffPath);
await expect(
resolveLocalEgressBootEnvironmentAsync({ handoffPath, libraryPath: null })
).rejects.toThrow(SystemError);
});
it('combines the guard variables with the proxy variables for the handoff port', async () => {
const handoffPath = path.join(dir, 'handoff.json');
await writeLocalEgressHandoffAsync({ ...handoff, port: 8899 }, handoffPath);
await expect(
resolveLocalEgressBootEnvironmentAsync({
handoffPath,
libraryPath: '/w/bin/egress-guard.dylib',
logPath: '/tmp/guard.log',
mode: 'block',
})
).resolves.toEqual({
DYLD_INSERT_LIBRARIES: '/w/bin/egress-guard.dylib',
[EGRESS_GUARD_LOG_ENV]: '/tmp/guard.log',
[EGRESS_GUARD_MODE_ENV]: 'block',
http_proxy: 'http://127.0.0.1:8899',
https_proxy: 'http://127.0.0.1:8899',
HTTP_PROXY: 'http://127.0.0.1:8899',
HTTPS_PROXY: 'http://127.0.0.1:8899',
grpc_proxy: 'http://127.0.0.1:8899',
no_proxy: 'localhost,127.0.0.1,::1',
NO_PROXY: 'localhost,127.0.0.1,::1',
});
});
});
describe(parseGuardCoverage, () => {
const ps = [
' 1 0 /sbin/launchd',
' 4000 1 /Runtimes/x/sbin/launchd_sim',
' 4001 4000 /Runtimes/x/System/Library/CoreServices/SpringBoard.app/SpringBoard',
' 4002 4000 /Runtimes/x/usr/libexec/backboardd',
' 4003 4001 /Containers/Bundle/Application/1/Expo Go.app/Expo Go',
' 3758 1 node',
].join('\n');
const lsof = [
'p4001',
'n/Runtimes/x/System/Library/CoreServices/SpringBoard.app/SpringBoard',
'n/usr/lib/dyld',
'p4003',
'n/Containers/Bundle/Application/1/Expo Go.app/Expo Go',
'n/Users/expo/build-tools/bin/egress-guard.dylib',
'p3758',
'n/usr/local/bin/node',
].join('\n');
it('splits simulator processes by whether the guard library is mapped', () => {
expect(parseGuardCoverage(ps, lsof)).toEqual({
covered: ['Expo Go'],
uncovered: ['SpringBoard', 'backboardd'],
uncoveredPids: [4001, 4002],
});
});
it('only keeps processes uncovered in both samples, so exec-in-progress ones drop out', () => {
const first = parseGuardCoverage(ps, lsof);
const secondPs = ps + '\n 4009 4000 /Runtimes/x/usr/libexec/newlystarted';
const second = parseGuardCoverage(
secondPs,
lsof + '\np4002\nn/Users/expo/bin/egress-guard.dylib'
);
expect(mergeGuardCoverageSamples(first, second)).toEqual({
covered: ['Expo Go', 'backboardd'],
uncovered: ['SpringBoard'],
uncoveredPids: [4001],
});
});
it("ignores launchd's exec trampoline, which has nothing mapped yet", () => {
const withTrampoline = ps + '\n 4004 4000 /Runtimes/x/usr/libexec/xpcproxy_sim';
expect(parseGuardCoverage(withTrampoline, lsof).uncovered).toEqual([
'SpringBoard',
'backboardd',
]);
});
});
describe(verifyLocalEgressGuardAsync, () => {
let logger: ReturnType<typeof createLogger>;
beforeEach(() => {
mockedSpawn.mockReset();
logger = createLogger();
});
it('runs the packaged self-check inside the simulator and logs its verdict', async () => {
mockedSpawn
.mockResolvedValueOnce({
stdout: 'egress-guard-check: guard loaded; non-loopback connections are refused\n',
stderr: '',
} as any)
.mockResolvedValue({ stdout: '', stderr: '' } as any);
await verifyLocalEgressGuardAsync({
udid: 'u' as any,
env: process.env,
logger,
checkPath: '/w/bin/egress-guard-check',
});
expect(mockedSpawn).toHaveBeenCalledWith(
'xcrun',
['simctl', 'spawn', 'u', '/w/bin/egress-guard-check', '--mode', 'block'],
expect.objectContaining({ stdio: 'pipe' })
);
expect(
logger.lines.some(l =>
/verified in the Simulator: egress-guard-check: guard loaded/.test(l.msg)
)
).toBe(true);
});
it('fails the session with the self-check output when the check fails', async () => {
mockedSpawn.mockRejectedValue(
Object.assign(new Error('exited with non-zero code: 2'), {
status: 2,
stdout: 'egress-guard-check: the guard library is not loaded in this process\n',
stderr: '',
})
);
await expect(
verifyLocalEgressGuardAsync({
udid: 'u' as any,
env: process.env,
logger,
checkPath: '/w/bin/egress-guard-check',
})
).rejects.toThrow(/not in effect .*\(exit 2\).*guard library is not loaded/);
});
it('fails the session when the self-check binary is missing', async () => {
await expect(
verifyLocalEgressGuardAsync({ udid: 'u' as any, env: process.env, logger, checkPath: null })
).rejects.toThrow(/self-check is not available/);
expect(mockedSpawn).not.toHaveBeenCalled();
});
});
@@ -1,32 +1,47 @@
import { UserError } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import { BuildRuntimePlatform, BuildStepEnv } from '@expo/steps';
import { BuildRuntimePlatform, BuildStepEnv, BuildStepInputValueTypeName } from '@expo/steps';
import spawn from '@expo/turtle-spawn';
import * as ngrok from '@ngrok/ngrok';
import { EventEmitter } from 'node:events';
import fs from 'node:fs/promises';
import path from 'node:path';
import {
clearTimeout as clearTimeoutCallback,
setTimeout as setTimeoutCallback,
} from 'node:timers';
import { setTimeout as setTimeoutAsync } from 'node:timers/promises';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { CustomBuildContext } from '../../../customBuildContext';
import { Sentry } from '../../../sentry';
import { turtleFetch } from '../../../utils/turtleFetch';
import { readServeSimServersAsync } from '../serveSimMetricsRecorder';
import { sleepAsync } from '../../../utils/retry';
import { uploadDeviceRunSessionScreenRecordingsAsync } from '../deviceRunSessionScreenRecordings';
import {
createExpoDeviceHubArgs,
createServeSimArgs,
createServeSimLaunchInputProviders,
describeServeSimLaunch,
ensureFfmpegInstalledOnceAsync,
fetchWebPreviewTurnArgsAsync,
metricsCorsOriginToServeSimArgs,
startDeviceWebPreviewWithTunnelAsync,
startExpoDeviceHubWithTunnelAsync,
parseServeSimLaunchInputs,
spawnDetached,
startNgrokTunnelAsync,
turnIceServersToWebPreviewArgs,
waitForDeviceRunSessionStoppedAsync,
waitForWebPreviewReadyAsync,
} from '../remoteDeviceRunSession';
import {
createExpoDeviceHubArgs,
createServeSimArgs,
simulatorPreviewPageUrl,
startDeviceSessionHostAsync,
waitForWebPreviewReadyAsync,
websiteOrigin,
websiteOriginServeSimArgs,
} from '../deviceSessionHost';
import { parseNetworkCaptureFieldsInput, parseNetworkCaptureInputs } from '../networkCaptureFields';
jest.mock('@ngrok/ngrok');
jest.mock('node:timers');
jest.mock('node:timers/promises');
@@ -34,6 +49,10 @@ jest.mock('../../../utils/turtleFetch');
jest.mock('../../../utils/retry', () => ({ sleepAsync: jest.fn() }));
jest.mock('../../../sentry');
jest.mock('@expo/turtle-spawn');
jest.mock('../deviceRunSessionScreenRecordings', () => ({
...jest.requireActual('../deviceRunSessionScreenRecordings'),
uploadDeviceRunSessionScreenRecordingsAsync: jest.fn(),
}));
// Spyable so a test can stand in for the serve-sim state directory, which a local serve-sim owns.
jest.mock('../serveSimMetricsRecorder', () => {
const actual = jest.requireActual('../serveSimMetricsRecorder');
@@ -49,6 +68,69 @@ function createLoggerMock(): bunyan {
} as unknown as bunyan;
}
describe(spawnDetached, () => {
function mockProcess(promise: Promise<unknown>): void {
jest
.mocked(spawn)
.mockReturnValue(
Object.assign(promise, { child: { pid: 1234, unref: jest.fn(), once: jest.fn() } }) as never
);
}
it('observes exit without waiting for inherited output pipes to close', () => {
const child = Object.assign(new EventEmitter(), { pid: 1234, unref: jest.fn() });
jest.mocked(spawn).mockReturnValue(Object.assign(new Promise(() => {}), { child }) as never);
const handle = spawnDetached({ command: 'server', args: [], env: {} });
child.emit('exit', 1, null);
expect(handle.getExitError()?.message).toContain('code 1');
expect(child.listenerCount('exit')).toBe(0);
});
it('does not report an exit while the process is running', () => {
mockProcess(new Promise(() => {}));
const handle = spawnDetached({ command: 'server', args: [], env: {} });
expect(handle.getExitError()).toBeUndefined();
});
it('reports a clean exit so startup does not keep waiting', async () => {
const completion = Promise.resolve();
mockProcess(completion);
const handle = spawnDetached({ command: 'server', args: [], env: {} });
await completion;
expect(handle.getExitError()?.message).toContain('code 0');
});
it.each([0, 1])('observes a real subprocess exiting with code %s', async exitCode => {
const actualSpawn =
jest.requireActual<typeof import('@expo/turtle-spawn')>('@expo/turtle-spawn').default;
let completion: ReturnType<typeof actualSpawn>;
jest.mocked(spawn).mockImplementationOnce((...args) => {
completion = actualSpawn(...args);
return completion;
});
const handle = spawnDetached({
command: process.execPath,
args: ['-e', `console.error('startup-marker'); process.exit(${exitCode});`],
env: {},
});
await completion!.catch(() => {});
expect(handle.getExitError()).toBeInstanceOf(Error);
expect(handle.getOutput()).toContain('startup-marker');
});
it.each(['spawn ENOENT', 'server exited with code 1', 'server terminated by SIGTERM'])(
'preserves the process failure: %s',
async message => {
const error = new Error(message);
const completion = Promise.reject(error);
mockProcess(completion);
const handle = spawnDetached({ command: 'server', args: [], env: {} });
await completion.catch(() => {});
expect(handle.getExitError()).toBe(error);
}
);
});
function createCtxMock(): CustomBuildContext {
return {
env: {
@@ -122,6 +204,161 @@ function createEnvMock(): BuildStepEnv {
return { DEVICE_RUN_SESSION_ID: 'drs-id' } as unknown as BuildStepEnv;
}
describe(createServeSimLaunchInputProviders, () => {
it('declares the launch inputs as optional', () => {
const globalCtx = createGlobalContextMock();
const inputs = createServeSimLaunchInputProviders().map(provider =>
provider(globalCtx, 'Test step')
);
expect(
inputs.map(({ id, required, allowedValueTypeName }) => ({
id,
required,
allowedValueTypeName,
}))
).toEqual([
{
id: 'launch_app_identifier',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
},
{
id: 'launch_args',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.JSON,
},
{
id: 'open_url',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
},
]);
});
});
describe(parseServeSimLaunchInputs, () => {
const darwin = { runtimePlatform: BuildRuntimePlatform.DARWIN };
it('reads the launch identifier, arguments and URL', () => {
expect(
parseServeSimLaunchInputs(
{
launchAppIdentifier: 'host.exp.Exponent',
launchArgs: ['-EXDevMenuIsOnboardingFinished', '1'],
openUrl: 'exp://127.0.0.1:8081',
},
darwin
)
).toEqual({
launchAppIdentifier: 'host.exp.Exponent',
launchArgs: ['-EXDevMenuIsOnboardingFinished', '1'],
openUrl: 'exp://127.0.0.1:8081',
});
});
it('defaults to no launch when the step declares nothing', () => {
expect(parseServeSimLaunchInputs({}, { runtimePlatform: BuildRuntimePlatform.LINUX })).toEqual({
launchAppIdentifier: undefined,
launchArgs: [],
openUrl: undefined,
});
});
it('rejects launch arguments that are not a list', () => {
expect(() =>
parseServeSimLaunchInputs(
{
launchAppIdentifier: 'host.exp.Exponent',
launchArgs: 'oops',
},
darwin
)
).toThrow('must be an array of strings');
});
it('rejects launch arguments that are not all strings', () => {
expect(() =>
parseServeSimLaunchInputs(
{
launchAppIdentifier: 'host.exp.Exponent',
launchArgs: ['-flag', 1],
},
darwin
)
).toThrow('must be an array of strings');
});
it('rejects launch arguments with no application to launch', () => {
expect(() => parseServeSimLaunchInputs({ launchArgs: ['-flag'] }, darwin)).toThrow(
'Pass "launch_app_identifier"'
);
});
it('rejects a URL with no application to open it in', () => {
expect(() => parseServeSimLaunchInputs({ openUrl: 'exp://127.0.0.1:8081' }, darwin)).toThrow(
'Pass "launch_app_identifier"'
);
});
it('rejects an identifier that resolved to an empty string', () => {
expect(() => parseServeSimLaunchInputs({ launchAppIdentifier: '' }, darwin)).toThrow(
'must be a non-empty string'
);
});
it('rejects a URL that is not a URL', () => {
expect(() =>
parseServeSimLaunchInputs(
{ launchAppIdentifier: 'host.exp.Exponent', openUrl: 'not a url' },
darwin
)
).toThrow('must be a valid URL');
});
it('raises a user error, not a platform error', () => {
expect(() => parseServeSimLaunchInputs({ launchAppIdentifier: '' }, darwin)).toThrow(UserError);
try {
parseServeSimLaunchInputs({ launchAppIdentifier: '' }, darwin);
} catch (error) {
expect((error as UserError).errorCode).toBe('EAS_LAUNCH_APPLICATION_INVALID_INPUT');
}
});
it('rejects a launch on a session that does not run an iOS simulator', () => {
expect(() =>
parseServeSimLaunchInputs(
{ launchAppIdentifier: 'host.exp.Exponent' },
{ runtimePlatform: BuildRuntimePlatform.LINUX }
)
).toThrow('runs on linux');
});
});
describe(describeServeSimLaunch, () => {
it('says nothing when there is no application to launch', () => {
expect(describeServeSimLaunch({ launchArgs: [] })).toBeNull();
});
it('names only the application when there are no arguments or URL', () => {
expect(describeServeSimLaunch({ launchAppIdentifier: 'host.exp.Exponent' })).toBe(
'serve-sim will launch host.exp.Exponent.'
);
});
it('names the application, its arguments and the URL', () => {
expect(
describeServeSimLaunch({
launchAppIdentifier: 'host.exp.Exponent',
launchArgs: ['-flag', '1'],
openUrl: 'exp://127.0.0.1:8081',
})
).toBe(
'serve-sim will launch host.exp.Exponent with arguments ["-flag","1"], then open exp://127.0.0.1:8081.'
);
});
});
describe(createServeSimArgs, () => {
it('uses the latest Expo package and applies the EAS streaming policy', () => {
expect(
@@ -130,7 +367,6 @@ describe(createServeSimArgs, () => {
turnArgs: ['--turn-url', 'turns:turn.example.test:443'],
})
).toEqual([
'--yes',
'@expo/serve-sim@latest',
'--port',
'4321',
@@ -140,13 +376,13 @@ describe(createServeSimArgs, () => {
'--transport',
'webrtc',
'--webrtc-codec',
'vp8',
'h264',
'--max-dimension',
'960',
'1600',
'--mjpeg-quality',
'0.55',
'--video-bitrate',
'6000000',
'10000000',
'--video-fps',
'60',
'--turn-url',
@@ -154,36 +390,158 @@ describe(createServeSimArgs, () => {
]);
});
it('appends metrics CORS args after the TURN args when provided', () => {
it('appends the website args after the TURN args when provided', () => {
const args = createServeSimArgs({
port: 4321,
turnArgs: ['--turn-url', 'turns:turn.example.test:443'],
metricsCorsArgs: ['--metrics-cors-origin', 'https://expo.dev'],
websiteArgs: ['--cors-origin', 'https://expo.dev'],
});
expect(args.slice(-4)).toEqual([
'--turn-url',
'turns:turn.example.test:443',
'--metrics-cors-origin',
'--cors-origin',
'https://expo.dev',
]);
});
it('pins the requested package version', () => {
expect(createServeSimArgs({ port: 4321, packageVersion: '0.1.38' }).slice(0, 2)).toEqual([
'--yes',
'@expo/[email protected]',
]);
expect(createServeSimArgs({ port: 4321, packageVersion: '0.1.38' })[0]).toBe(
'@expo/[email protected]'
);
});
it('pins a dist-tag', () => {
expect(createServeSimArgs({ port: 4321, packageVersion: 'next' }).slice(0, 2)).toEqual([
'--yes',
'@expo/serve-sim@next',
expect(createServeSimArgs({ port: 4321, packageVersion: 'next' })[0]).toBe(
'@expo/serve-sim@next'
);
});
it('appends --share-url after the website args', () => {
const args = createServeSimArgs({
port: 4321,
websiteArgs: ['--cors-origin', 'https://expo.dev', '--frame-ancestor', 'https://expo.dev'],
shareUrl: 'https://expo.dev/simulator-preview/abc',
});
expect(args.slice(-4)).toEqual([
'--frame-ancestor',
'https://expo.dev',
'--share-url',
'https://expo.dev/simulator-preview/abc',
]);
});
it('appends the launch flags after the streaming policy', () => {
const args = createServeSimArgs({
port: 4321,
launchAppIdentifier: 'host.exp.Exponent',
launchArgs: ['-EXDevMenuIsOnboardingFinished', '1'],
openUrl: 'exp://127.0.0.1:8081',
});
expect(args.slice(-8)).toEqual([
'--launch-app-identifier',
'host.exp.Exponent',
'--launch-arg',
'-EXDevMenuIsOnboardingFinished',
'--launch-arg',
'1',
'--open-url',
'exp://127.0.0.1:8081',
]);
});
it('omits the launch flags when there is no application to launch', () => {
const args = createServeSimArgs({ port: 4321 });
expect(args.some(argument => argument.startsWith('--launch'))).toBe(false);
expect(args).not.toContain('--open-url');
});
it('omits --network-capture by default', () => {
expect(createServeSimArgs({ port: 4321 })).not.toContain('--network-capture');
expect(createServeSimArgs({ port: 4321, networkCapture: false })).not.toContain(
'--network-capture'
);
});
it('appends --network-capture when enabled, which also covers an already booted simulator', () => {
const args = createServeSimArgs({ port: 4321, networkCapture: true });
expect(args).toContain('--network-capture');
expect(args).not.toContain('--enable');
});
it('repeats --network-capture-field once per requested field', () => {
expect(
createServeSimArgs({
port: 4321,
networkCapture: true,
networkCaptureFields: ['header', 'query'],
})
).toEqual(
expect.arrayContaining([
'--network-capture',
'--network-capture-field',
'header',
'--network-capture-field',
'query',
])
);
});
it('keeps capture metadata-only when no field is requested', () => {
expect(createServeSimArgs({ port: 4321, networkCapture: true })).not.toContain(
'--network-capture-field'
);
});
it('does not pass fields when capture itself is off', () => {
expect(
createServeSimArgs({ port: 4321, networkCapture: false, networkCaptureFields: ['header'] })
).not.toContain('--network-capture-field');
});
it('rejects network capture on a runtime that has no serve-sim', () => {
expect(() =>
parseNetworkCaptureInputs(
{ networkCapture: true },
{ runtimePlatform: BuildRuntimePlatform.LINUX }
)
).toThrow('this session runs on linux');
expect(() =>
parseNetworkCaptureInputs(
{ networkCaptureFields: ['header'] },
{ runtimePlatform: BuildRuntimePlatform.DARWIN }
)
).toThrow('needs "network_capture: true"');
expect(parseNetworkCaptureInputs({}, { runtimePlatform: BuildRuntimePlatform.LINUX })).toEqual({
networkCapture: false,
networkCaptureFields: [],
});
expect(
parseNetworkCaptureInputs(
{ networkCapture: true, networkCaptureFields: ['header'] },
{ runtimePlatform: BuildRuntimePlatform.DARWIN }
)
).toEqual({ networkCapture: true, networkCaptureFields: ['header'] });
});
it('rejects a step input that is not an array of strings', () => {
// A JSON step input is whatever the workflow author wrote, so the shape has to be checked.
expect(() => parseNetworkCaptureFieldsInput('header,query')).toThrow(UserError);
expect(() => parseNetworkCaptureFieldsInput('header,query')).toThrow(
/must be an array of strings/
);
expect(() => parseNetworkCaptureFieldsInput([1, 2])).toThrow(UserError);
expect(parseNetworkCaptureFieldsInput(undefined)).toEqual([]);
expect(parseNetworkCaptureFieldsInput(['header'])).toEqual(['header']);
});
});
describe(createExpoDeviceHubArgs, () => {
it('opts in to recording only when a directory is provided', () => {
expect(createExpoDeviceHubArgs({ port: 4321 })).not.toContain('--android-recording-directory');
expect(
createExpoDeviceHubArgs({ port: 4321, recordingDirectory: '/tmp/recordings' }).slice(-2)
).toEqual(['--android-recording-directory', '/tmp/recordings']);
});
it('uses the latest Expo package and applies the EAS Android streaming policy', () => {
expect(
createExpoDeviceHubArgs({
@@ -191,7 +549,6 @@ describe(createExpoDeviceHubArgs, () => {
turnArgs: ['--turn-url', 'turns:turn.example.test:443'],
})
).toEqual([
'--yes',
'expo-device-hub@latest',
'--port',
'4321',
@@ -219,37 +576,51 @@ describe(createExpoDeviceHubArgs, () => {
});
it('pins the requested package version', () => {
expect(createExpoDeviceHubArgs({ port: 4321, packageVersion: '0.7.0' }).slice(0, 2)).toEqual([
'--yes',
'[email protected]',
]);
expect(createExpoDeviceHubArgs({ port: 4321, packageVersion: '0.7.0' })[0]).toBe(
'[email protected]'
);
});
});
describe(metricsCorsOriginToServeSimArgs, () => {
it('returns no args when the origin is unset or empty', () => {
expect(metricsCorsOriginToServeSimArgs({} as BuildStepEnv)).toEqual([]);
expect(
metricsCorsOriginToServeSimArgs({ EAS_SIMULATOR_METRICS_CORS_ORIGIN: '' } as BuildStepEnv)
).toEqual([]);
describe(websiteOriginServeSimArgs, () => {
it('names only the production website by default', () => {
expect(websiteOriginServeSimArgs({} as BuildStepEnv)).toEqual([
'--cors-origin',
'https://expo.dev',
'--frame-ancestor',
'https://expo.dev',
]);
});
it('builds one flag per comma-separated origin', () => {
expect(
metricsCorsOriginToServeSimArgs({
EAS_SIMULATOR_METRICS_CORS_ORIGIN: 'https://expo.dev',
} as BuildStepEnv)
).toEqual(['--metrics-cors-origin', 'https://expo.dev']);
expect(
metricsCorsOriginToServeSimArgs({
EAS_SIMULATOR_METRICS_CORS_ORIGIN: 'https://expo.dev, https://staging.expo.dev',
} as BuildStepEnv)
).toEqual([
'--metrics-cors-origin',
'https://expo.dev',
'--metrics-cors-origin',
it('names staging, its deploy previews and each website dev port on staging', () => {
const args = websiteOriginServeSimArgs({ EXPO_STAGING: '1' } as BuildStepEnv);
expect(args.slice(0, 4)).toEqual([
'--cors-origin',
'https://staging.expo.dev',
'--frame-ancestor',
'https://staging.expo.dev',
]);
expect(args).toContain('https://*.expo.dev');
expect(args).toContain('https://expo.test:13001');
expect(args).toContain('https://expo.test:13215');
expect(args).not.toContain('https://expo.test:13216');
expect(args).not.toContain('https://expo.dev');
});
it('names only https origins', () => {
for (const env of [{}, { EXPO_STAGING: '1' }, { EXPO_LOCAL: '1' }]) {
const args = websiteOriginServeSimArgs(env as BuildStepEnv);
expect(args.filter(value => value.startsWith('http://'))).toEqual([]);
}
});
it('names the website dev ports on local, without the deploy-preview wildcard', () => {
for (const env of [{ EXPO_LOCAL: '1' }, { EXPO_LOCAL: '1', EXPO_STAGING: '1' }]) {
const args = websiteOriginServeSimArgs(env as BuildStepEnv);
expect(args).toContain('https://expo.test:13001');
expect(args).not.toContain('https://*.expo.dev');
expect(args.filter(value => value === 'https://expo.test')).toHaveLength(2);
}
});
});
@@ -282,6 +653,7 @@ describe(waitForWebPreviewReadyAsync, () => {
expect.objectContaining({ retries: 0 })
);
expect(sleepAsync).toHaveBeenCalledTimes(1);
expect(sleepAsync).toHaveBeenCalledWith(250);
});
});
@@ -309,13 +681,18 @@ describe(startNgrokTunnelAsync, () => {
})
);
expect(tunnel.url).toBe('https://web-preview.example.test');
expect(ngrok.forward).toHaveBeenCalledWith(
expect.objectContaining({
domain: `web-preview-${tunnel.subdomainId}.eas-simulator.ngrok.dev`,
})
);
await tunnel.stopAsync();
await tunnel.stopAsync();
expect(close).toHaveBeenCalledTimes(1);
});
});
describe(startDeviceWebPreviewWithTunnelAsync, () => {
describe(startDeviceSessionHostAsync, () => {
const baseDomain = 'eas-simulator.ngrok.dev';
const turnArgs = [
'--turn-url',
@@ -325,14 +702,14 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
'--turn-credential',
'turn-credential',
];
const metricsCorsArgs = ['--metrics-cors-origin', 'https://metrics.expo.test'];
const websiteArgs = ['--cors-origin', 'https://expo.dev', '--frame-ancestor', 'https://expo.dev'];
const env = {
DEVICE_RUN_SESSION_ID: 'drs-id',
EAS_SIMULATOR_METRICS_CORS_ORIGIN: 'https://metrics.expo.test',
NGROK_AUTHTOKEN: 'ngrok-token',
} as unknown as BuildStepEnv;
beforeEach(() => {
jest.mocked(uploadDeviceRunSessionScreenRecordingsAsync).mockReset();
jest.mocked(spawn).mockReset();
jest.mocked(ngrok.forward).mockReset();
jest.mocked(turtleFetch).mockReset();
@@ -344,6 +721,7 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
child: {
pid: undefined,
unref: jest.fn(),
once: jest.fn(),
},
});
jest.mocked(spawn).mockReturnValue(spawnPromise as never);
@@ -385,14 +763,14 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
.mockReturnValueOnce(Promise.resolve({}) as unknown as ReturnType<typeof spawn>)
.mockReturnValueOnce(Promise.resolve({}) as unknown as ReturnType<typeof spawn>);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
const host = await startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.LINUX,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
packageVersion,
});
const preview = await host.openPreviewAsync({ baseDomain });
const spawnCalls = jest.mocked(spawn).mock.calls;
expect(spawnCalls[0]).toEqual(['ffmpeg', ['-version'], { env }]);
@@ -419,11 +797,17 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
const port = Number(args[args.indexOf('--port') + 1]);
expect(port).toBeGreaterThan(0);
expect(command).toBe('npx');
expect(args).toEqual(createExpoDeviceHubArgs({ port, turnArgs, packageVersion }));
expect(args).toContain('--android-recording-directory');
const recordingDirectory = args[args.indexOf('--android-recording-directory') + 1];
expect(args).toEqual([
'--yes',
...createExpoDeviceHubArgs({ port, turnArgs, packageVersion, recordingDirectory }),
]);
expect(ngrok.forward).toHaveBeenCalledWith(expect.objectContaining({ addr: port }));
expect(preview.previewUrl).toBe('https://android-preview.example.test');
expect(preview.apiUrl).toBe('https://android-preview.example.test');
await preview.stopAsync();
await host.finishAsync();
await fs.rm(recordingDirectory, { recursive: true, force: true });
expect(close).toHaveBeenCalledTimes(1);
});
@@ -433,16 +817,50 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
close: jest.fn().mockResolvedValue(undefined),
} as never);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
const host = await startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
});
const preview = await host.openPreviewAsync({ baseDomain });
expect(preview.previewToken).toBe('tok-1');
expect(preview.previewUrl).toBe('https://preview.example.test');
expect(preview.apiUrl).toBe('https://preview.example.test');
});
it.each([
[{}, 'https://expo.dev'],
[{ EXPO_STAGING: '1' }, 'https://staging.expo.dev'],
[{ EXPO_LOCAL: '1' }, 'https://expo.test'],
])('points the preview page at the website for the stage the worker runs in', (stage, origin) => {
expect(websiteOrigin({ ...env, ...stage })).toBe(origin);
expect(simulatorPreviewPageUrl({ ...env, ...stage }, 'abc')).toBe(
`${origin}/simulator-preview/abc`
);
});
it('points the preview URL at the website page for the tunnel', async () => {
jest.mocked(ngrok.forward).mockResolvedValue({
url: () => 'https://preview.example.test',
close: jest.fn().mockResolvedValue(undefined),
} as never);
const host = await startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
env: { ...env, EXPO_STAGING: '1' },
logger: createLoggerMock(),
timeoutMs: 10_000,
});
const preview = await host.openPreviewAsync({ baseDomain });
expect(preview.previewPageUrl).toMatch(
/^https:\/\/staging\.expo\.dev\/simulator-preview\/[a-f0-9]{32}$/
);
const previewId = preview.previewPageUrl.split('/').at(-1);
expect(ngrok.forward).toHaveBeenCalledWith(
expect.objectContaining({ domain: `web-preview-${previewId}.${baseDomain}` })
);
});
// serve-sim is always launched with --require-token, so a missing token means it is running
@@ -453,9 +871,8 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
.mockResolvedValue([{ udid: 'device-id', url: 'http://127.0.0.1:1' }]);
await expect(
startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
@@ -471,13 +888,13 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
close: jest.fn().mockResolvedValue(undefined),
} as never);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
const host = await startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.LINUX,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
});
const preview = await host.openPreviewAsync({ baseDomain });
expect(preview.previewToken).toBeUndefined();
const [, args] = jest.mocked(spawn).mock.calls[0];
@@ -492,52 +909,145 @@ describe(startDeviceWebPreviewWithTunnelAsync, () => {
close,
} as never);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
const host = await startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
packageVersion,
});
const preview = await host.openPreviewAsync({ baseDomain });
const [command, args] = jest.mocked(spawn).mock.calls[0];
const port = Number(args[args.indexOf('--port') + 1]);
expect(port).toBeGreaterThan(0);
expect(command).toBe('npx');
expect(args).toEqual(createServeSimArgs({ port, turnArgs, metricsCorsArgs, packageVersion }));
expect(args).toEqual([
'--yes',
...createServeSimArgs({
port,
turnArgs,
websiteArgs,
shareUrl: preview.previewPageUrl,
packageVersion,
}),
]);
expect(ngrok.forward).toHaveBeenCalledWith(expect.objectContaining({ addr: port }));
expect(preview.previewUrl).toBe('https://ios-preview.example.test');
expect(preview.apiUrl).toBe('https://ios-preview.example.test');
await preview.stopAsync();
await host.finishAsync();
expect(close).toHaveBeenCalledTimes(1);
});
it('does not install ffmpeg before starting expo-device-hub outside Linux', async () => {
it('launches serve-sim with bun x when EAS_OVERRIDE_PACKAGE_MANAGER is bun', async () => {
const close = jest.fn().mockResolvedValue(undefined);
jest.mocked(ngrok.forward).mockResolvedValue({
url: () => 'https://android-preview.example.test',
url: () => 'https://ios-preview.example.test',
close,
} as never);
const preview = await startExpoDeviceHubWithTunnelAsync(createCtxMock(), {
const bunEnv = { ...env, EAS_OVERRIDE_PACKAGE_MANAGER: 'bun' };
const host = await startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env,
env: bunEnv,
logger: createLoggerMock(),
timeoutMs: 10_000,
packageVersion: '4.5.6',
});
const preview = await host.openPreviewAsync({ baseDomain });
const [command, args] = jest.mocked(spawn).mock.calls[0];
const port = Number(args[args.indexOf('--port') + 1]);
expect(command).toBe('bun');
expect(args).toEqual([
'x',
...createServeSimArgs({
port,
turnArgs,
websiteArgs,
shareUrl: preview.previewPageUrl,
packageVersion: '4.5.6',
}),
]);
await host.finishAsync();
expect(close).toHaveBeenCalledTimes(1);
});
it('launches serve-sim with bun x when EAS_FALLBACK_PACKAGE_MANAGER is bun', async () => {
const close = jest.fn().mockResolvedValue(undefined);
jest.mocked(ngrok.forward).mockResolvedValue({
url: () => 'https://ios-preview.example.test',
close,
} as never);
const host = await startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
env: { ...env, EAS_FALLBACK_PACKAGE_MANAGER: 'bun' },
logger: createLoggerMock(),
timeoutMs: 10_000,
});
const preview = await host.openPreviewAsync({ baseDomain });
expect(jest.mocked(spawn).mock.calls[0][0]).toBe('npx');
expect(jest.mocked(spawn)).not.toHaveBeenCalledWith(
'ffmpeg',
expect.anything(),
expect.anything()
);
const [command, args] = jest.mocked(spawn).mock.calls[0];
const port = Number(args[args.indexOf('--port') + 1]);
expect(command).toBe('bun');
expect(args).toEqual([
'x',
...createServeSimArgs({
port,
turnArgs,
websiteArgs,
shareUrl: preview.previewPageUrl,
}),
]);
await preview.stopAsync();
await host.finishAsync();
expect(close).toHaveBeenCalledTimes(1);
});
it('hands the launch options to serve-sim on Darwin', async () => {
jest.mocked(ngrok.forward).mockResolvedValue({
url: () => 'https://ios-preview.example.test',
close: jest.fn().mockResolvedValue(undefined),
} as never);
const host = await startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
launchAppIdentifier: 'host.exp.Exponent',
launchArgs: ['-EXDevMenuIsOnboardingFinished', '1'],
openUrl: 'exp://127.0.0.1:8081',
});
await host.finishAsync();
const [, args] = jest.mocked(spawn).mock.calls[0];
expect(args.slice(-8)).toEqual([
'--launch-app-identifier',
'host.exp.Exponent',
'--launch-arg',
'-EXDevMenuIsOnboardingFinished',
'--launch-arg',
'1',
'--open-url',
'exp://127.0.0.1:8081',
]);
});
it('refuses to launch an application on Linux, where expo-device-hub cannot', async () => {
await expect(
startDeviceSessionHostAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.LINUX,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
launchAppIdentifier: 'host.exp.Exponent',
})
).rejects.toThrow('Cannot launch host.exp.Exponent');
expect(spawn).not.toHaveBeenCalled();
});
});
describe(turnIceServersToWebPreviewArgs, () => {
@@ -0,0 +1,25 @@
import type { BuildStepEnv } from '@expo/steps';
import { isProcessRunning, spawnDetached } from '../remoteDeviceRunSession';
jest.mock('@ngrok/ngrok');
it('resolves stopAsync only after a process that ignores SIGTERM is gone', async () => {
const handle = spawnDetached({
command: 'sh',
args: ['-c', 'trap "" TERM; echo ready; sleep 30 & wait'],
env: process.env as BuildStepEnv,
stopGracePeriodMs: 300,
});
const pid = handle.pid;
if (pid === undefined) {
throw new Error('spawn failed');
}
while (!handle.getOutput().includes('ready')) {
await new Promise(resolve => setTimeout(resolve, 20));
}
const startedAt = Date.now();
await handle.stopAsync();
expect(Date.now() - startedAt).toBeGreaterThanOrEqual(300);
expect(isProcessRunning(pid)).toBe(false);
});
@@ -1,4 +1,4 @@
import { SystemError } from '@expo/eas-build-job';
import { SandboxDaemonResponseZ, SystemError } from '@expo/eas-build-job';
import Log from '@expo/logger';
import http from 'node:http';
import net, { AddressInfo } from 'node:net';
@@ -7,6 +7,10 @@ import WebSocket, { WebSocketServer } from 'ws';
import { startSandboxDaemonAsync } from '../sandboxDaemon';
jest.unmock('@expo/logger');
jest.unmock('fs');
jest.unmock('fs/promises');
jest.unmock('node:fs');
jest.unmock('node:fs/promises');
describe(startSandboxDaemonAsync.name, () => {
const logger = Log.child({ buildStepId: 'sandbox-test' });
@@ -17,6 +21,8 @@ describe(startSandboxDaemonAsync.name, () => {
serverUrl,
reconnectDelayMs: 10,
logger,
workingDirectory: process.cwd(),
env: process.env,
});
try {
await expect(daemon.ready).rejects.toBeInstanceOf(SystemError);
@@ -40,6 +46,8 @@ describe(startSandboxDaemonAsync.name, () => {
serverUrl: `ws://127.0.0.1:${(server.address() as AddressInfo).port}`,
reconnectDelayMs: 60_000,
logger,
workingDirectory: process.cwd(),
env: process.env,
});
try {
const socket = await connection;
@@ -72,6 +80,8 @@ describe(startSandboxDaemonAsync.name, () => {
serverUrl: `ws://127.0.0.1:${address.port}`,
reconnectDelayMs: 10,
logger,
workingDirectory: process.cwd(),
env: process.env,
});
try {
await expect(daemon.ready).rejects.toBeInstanceOf(SystemError);
@@ -100,6 +110,8 @@ describe(startSandboxDaemonAsync.name, () => {
reconnectDelayMs: 60_000,
logger,
signal: controller.signal,
workingDirectory: process.cwd(),
env: process.env,
});
const connectionError = daemon.ready.catch(error => error);
@@ -126,6 +138,8 @@ describe(startSandboxDaemonAsync.name, () => {
serverUrl: `ws://127.0.0.1:${address.port}`,
reconnectDelayMs: 60_000,
logger,
workingDirectory: process.cwd(),
env: process.env,
});
await expect(daemon.ready).rejects.toThrow('Sandbox MCP server connection failed');
@@ -157,6 +171,8 @@ describe(startSandboxDaemonAsync.name, () => {
serverUrl: `ws://127.0.0.1:${address.port}`,
reconnectDelayMs: 10,
logger,
workingDirectory: process.cwd(),
env: process.env,
});
const socket = await connection;
const daemon = await daemonPromise;
@@ -170,6 +186,117 @@ describe(startSandboxDaemonAsync.name, () => {
await new Promise<void>(resolve => httpServer.close(() => resolve()));
});
it('executes command messages', async () => {
const httpServer = http.createServer();
const mcpServer = new WebSocketServer({ noServer: true });
httpServer.on('upgrade', (request, socket, head) => {
mcpServer.handleUpgrade(request, socket, head, client =>
mcpServer.emit('connection', client)
);
});
await new Promise<void>(resolve => httpServer.listen(0, '127.0.0.1', resolve));
const address = httpServer.address() as AddressInfo;
const connection = new Promise<WebSocket>(resolve => mcpServer.once('connection', resolve));
const daemon = await startSandboxDaemonAsync({
credential: 'secret-token',
serverUrl: `ws://127.0.0.1:${address.port}`,
reconnectDelayMs: 10,
logger,
workingDirectory: process.cwd(),
env: process.env,
});
const socket = await connection;
await daemon.ready;
const commandResponse = new Promise<string>(resolve =>
socket.once('message', data => resolve(`${data}`))
);
socket.send(
JSON.stringify({
jsonrpc: '2.0',
id: 'command-1',
method: 'execCommand',
params: { cmd: 'printf hello' },
})
);
expect(JSON.parse(await commandResponse)).toMatchObject({
jsonrpc: '2.0',
id: 'command-1',
result: { output: 'hello', exitCode: 0 },
});
await daemon.stopAsync();
mcpServer.close();
await new Promise<void>(resolve => httpServer.close(() => resolve()));
});
it('returns JSON-RPC errors for invalid messages', async () => {
const httpServer = http.createServer();
const mcpServer = new WebSocketServer({ noServer: true });
httpServer.on('upgrade', (request, socket, head) => {
mcpServer.handleUpgrade(request, socket, head, client =>
mcpServer.emit('connection', client)
);
});
await new Promise<void>(resolve => httpServer.listen(0, '127.0.0.1', resolve));
const address = httpServer.address() as AddressInfo;
const connection = new Promise<WebSocket>(resolve => mcpServer.once('connection', resolve));
const daemon = await startSandboxDaemonAsync({
credential: 'secret-token',
serverUrl: `ws://127.0.0.1:${address.port}`,
reconnectDelayMs: 10,
logger,
workingDirectory: process.cwd(),
env: process.env,
});
const socket = await connection;
await daemon.ready;
const sendAsync = async (message: string): Promise<unknown> => {
const response = new Promise<string>(resolve =>
socket.once('message', data => resolve(`${data}`))
);
socket.send(message);
return SandboxDaemonResponseZ.parse(JSON.parse(await response));
};
await expect(sendAsync('{')).resolves.toMatchObject({
id: null,
error: { code: -32700, message: 'Parse error' },
});
await expect(sendAsync('{}')).resolves.toMatchObject({
id: null,
error: { code: -32600, message: 'Invalid request' },
});
await expect(
sendAsync(JSON.stringify({ jsonrpc: '2.0', id: '1', method: 'unknown' }))
).resolves.toMatchObject({
id: '1',
error: { code: -32601, message: 'Method not found' },
});
await expect(
sendAsync(JSON.stringify({ jsonrpc: '2.0', id: '2', method: 'execCommand', params: {} }))
).resolves.toMatchObject({
id: '2',
error: { code: -32602, message: 'Invalid params' },
});
await expect(
sendAsync(
JSON.stringify({
jsonrpc: '2.0',
id: '3',
method: 'writeStdin',
params: { sessionId: 999 },
})
)
).resolves.toMatchObject({
id: '3',
error: { code: -32603, message: 'Command session 999 does not exist.' },
});
await daemon.stopAsync();
mcpServer.close();
await new Promise<void>(resolve => httpServer.close(() => resolve()));
});
it('cancels the reconnect delay when stopped', async () => {
const httpServer = http.createServer();
const mcpServer = new WebSocketServer({ noServer: true });
@@ -186,6 +313,8 @@ describe(startSandboxDaemonAsync.name, () => {
serverUrl: `ws://127.0.0.1:${address.port}`,
reconnectDelayMs: 60_000,
logger,
workingDirectory: process.cwd(),
env: process.env,
});
const socket = await connection;
const daemon = await daemonPromise;
@@ -0,0 +1,286 @@
jest.unmock('fs');
jest.unmock('fs/promises');
jest.unmock('node:fs');
jest.unmock('node:fs/promises');
import { type SandboxDaemonCommandResult, type SandboxDaemonMethod } from '@expo/eas-build-job';
import fs from 'node:fs/promises';
import { spawnSync } from 'node:child_process';
import os from 'node:os';
import path from 'node:path';
import { performance } from 'node:perf_hooks';
import { setTimeout as setTimeoutAsync } from 'node:timers/promises';
import {
type SandboxDaemonCommandImplementations,
createSandboxCommandImplementations,
} from '../sandboxCommandImplementations';
describe('sandbox daemon commands', () => {
let commandImplementations: SandboxDaemonCommandImplementations;
let stopAsync: () => Promise<void>;
let workingDirectory: string;
beforeEach(async () => {
workingDirectory = await fs.mkdtemp(path.join(os.tmpdir(), 'sandbox-command-executor-'));
const controller = new AbortController();
const commands = createSandboxCommandImplementations({
workingDirectory,
env: process.env,
signal: controller.signal,
});
commandImplementations = commands.commandImplementations;
stopAsync = async () => {
controller.abort();
await commands.stoppedPromise;
};
});
afterEach(async () => {
await stopAsync();
await fs.chmod(workingDirectory, 0o700);
await fs.rm(workingDirectory, { recursive: true, force: true });
});
it('returns output and the exit code for a completed command', async () => {
const result = await commandImplementations.execCommand({ cmd: 'printf hello' });
expect(result).toMatchObject({ output: 'hello', exitCode: 0 });
expect('sessionId' in result).toBe(false);
});
it('rejects a command canceled during directory validation', async () => {
const controller = new AbortController();
const commands = createSandboxCommandImplementations({
workingDirectory,
env: process.env,
signal: controller.signal,
});
const command = commands.commandImplementations.execCommand({ cmd: 'printf should-not-run' });
controller.abort();
await commands.stoppedPromise;
await expect(command).rejects.toMatchObject({ name: 'AbortError' });
await expect(
commands.commandImplementations.writeStdin({ sessionId: 1 })
).rejects.toMatchObject({ name: 'AbortError' });
});
it('returns a session id and accepts stdin for a running command', async () => {
const started = await commandImplementations.execCommand({
cmd: `node -e "process.stdin.once('data', data => process.stdout.write(data.toString().toUpperCase(), () => process.exit(0)))"`,
yieldTimeMs: 10,
});
const sessionId = getSessionId(started);
const completed = await commandImplementations.writeStdin({
sessionId,
chars: 'hello',
yieldTimeMs: 1_000,
});
expect(completed).toMatchObject({ output: 'HELLO', exitCode: 0 });
expect('sessionId' in completed).toBe(false);
});
it('returns only output produced since the previous call', async () => {
const started = await commandImplementations.execCommand({
cmd: `node -e "process.stdin.once('data', () => process.stdout.write('second\\n', () => process.exit(0))); console.log('first')"`,
yieldTimeMs: 50,
});
const sessionId = getSessionId(started);
expect(await readUntilAsync(started, output => output.includes('first\n'))).toMatchObject({
output: 'first\n',
});
const completed = await readUntilAsync(
await commandImplementations.writeStdin({ sessionId, chars: 'continue', yieldTimeMs: 100 }),
(_, result) => !('sessionId' in result)
);
expect(completed).toMatchObject({ output: 'second\n', exitCode: 0 });
});
it('provides a TTY when requested', async () => {
const result = await commandImplementations.execCommand({
cmd: 'test -t 0 && test -t 1 && test -t 2',
tty: true,
});
expect(result).toMatchObject({ exitCode: 0 });
});
it('uses pipes by default', async () => {
const result = await commandImplementations.execCommand({ cmd: 'test -t 0' });
expect(result).toMatchObject({ exitCode: 1 });
});
it('sends Ctrl+C to the foreground PTY process', async () => {
const started = await commandImplementations.execCommand({
cmd: `exec node -e "process.on('SIGINT', () => process.stdout.write('interrupted\\n', () => process.exit(0))); console.log('ready'); setInterval(() => {}, 1000)"`,
tty: true,
yieldTimeMs: 50,
});
await readUntilAsync(started, output => output.includes('ready'));
const completed = await readUntilAsync(
await commandImplementations.writeStdin({
sessionId: getSessionId(started),
chars: '\u0003',
yieldTimeMs: 1_000,
}),
(_, result) => !('sessionId' in result)
);
expect(completed.output).toContain('interrupted');
expect(completed).toMatchObject({ exitCode: 0 });
});
it('writes U+0003 as literal input in pipe mode', async () => {
const started = await commandImplementations.execCommand({
cmd: `node -e "process.stdin.once('data', data => process.stdout.write(data[0] + '\\n', () => process.exit(0)))"`,
yieldTimeMs: 10,
});
const completed = await readUntilAsync(
await commandImplementations.writeStdin({
sessionId: getSessionId(started),
chars: '\u0003',
yieldTimeMs: 1_000,
}),
(_, result) => !('sessionId' in result)
);
expect(completed).toMatchObject({ output: '3\n', exitCode: 0 });
});
it('reports a termination signal without an exit code', async () => {
const result = await commandImplementations.execCommand({ cmd: 'kill -TERM $$' });
expect(result).toMatchObject({ terminationSignal: 'SIGTERM' });
expect('exitCode' in result).toBe(false);
});
it('measures each call separately', async () => {
const now = jest.spyOn(performance, 'now');
try {
now.mockReturnValueOnce(1_000).mockReturnValueOnce(1_250);
const started = await commandImplementations.execCommand({
cmd: 'read value',
yieldTimeMs: 0,
});
expect(started.wallTimeSeconds).toBe(0.25);
now.mockReturnValueOnce(5_000).mockReturnValueOnce(5_100);
const polled = await commandImplementations.writeStdin({
sessionId: getSessionId(started),
yieldTimeMs: 0,
});
expect(polled.wallTimeSeconds).toBe(0.1);
} finally {
now.mockRestore();
}
});
it('uses the requested working directory', async () => {
const childDirectory = path.join(workingDirectory, 'child');
await fs.mkdir(childDirectory);
const result = await commandImplementations.execCommand({ cmd: 'pwd', workdir: 'child' });
expect(result).toMatchObject({ output: `${await fs.realpath(childDirectory)}\n`, exitCode: 0 });
});
it('reports a missing working directory', async () => {
await expect(
commandImplementations.execCommand({ cmd: 'true', workdir: 'missing' })
).rejects.toThrow(
`Working directory does not exist: ${path.join(workingDirectory, 'missing')}`
);
});
it('reports a working directory that is not a directory', async () => {
const file = path.join(workingDirectory, 'file');
await fs.writeFile(file, 'content');
await expect(
commandImplementations.execCommand({ cmd: 'true', workdir: 'file' })
).rejects.toThrow(`Working directory is not a directory: ${file}`);
});
it('reports an inaccessible working directory', async () => {
if (process.platform === 'win32') {
return;
}
const directory = path.join(workingDirectory, 'inaccessible');
await fs.mkdir(directory, 0o000);
await expect(
commandImplementations.execCommand({ cmd: 'true', workdir: 'inaccessible' })
).rejects.toThrow(`Working directory is not accessible: ${directory}`);
await fs.chmod(directory, 0o700);
});
it.each([false, true])('stops command descendants when tty is %s', async tty => {
const started = await commandImplementations.execCommand({
cmd: `node -e "const { spawn } = require('node:child_process'); const child = spawn(process.execPath, ['-e', 'setInterval(() => {}, 1000)'], { stdio: 'ignore' }); process.stdout.write(String(child.pid) + '\\n'); setInterval(() => {}, 1000)"`,
tty,
yieldTimeMs: 100,
});
const ready = await readUntilAsync(started, output => /\d+\r?\n/.test(output));
const childPid = Number(/\d+/.exec(ready.output)?.[0]);
expect(childPid).toBeGreaterThan(0);
await stopAsync();
await setTimeoutAsync(50);
expect(isProcessRunning(childPid)).toBe(false);
});
async function readUntilAsync(
initial: SandboxDaemonCommandResult<SandboxDaemonMethod>,
isReady: (output: string, result: SandboxDaemonCommandResult<SandboxDaemonMethod>) => boolean
): Promise<SandboxDaemonCommandResult<SandboxDaemonMethod>> {
let result = initial;
let output = result.output;
const deadline = Date.now() + 10_000;
while (!isReady(output, result)) {
if (!('sessionId' in result) || Date.now() >= deadline) {
throw new Error(
`Command did not reach the expected state: ${JSON.stringify({ ...result, output })}`
);
}
result = await commandImplementations.writeStdin({
sessionId: result.sessionId,
yieldTimeMs: 100,
});
output += result.output;
}
return { ...result, output };
}
});
function getSessionId(result: unknown): number {
if (
typeof result !== 'object' ||
result === null ||
!('sessionId' in result) ||
typeof result.sessionId !== 'number'
) {
throw new Error('Expected the command to still be running.');
}
return result.sessionId;
}
function isProcessRunning(pid: number): boolean {
const result = spawnSync('ps', ['-o', 'stat=', '-p', String(pid)], { encoding: 'utf8' });
if (result.error) {
throw result.error;
}
if (result.status === 1 && !result.stdout.trim()) {
return false;
}
expect(result.status).toBe(0);
// A zombie has exited, but its parent has not yet collected its exit status.
return !result.stdout.trim().startsWith('Z');
}
@@ -0,0 +1,57 @@
import { type bunyan } from '@expo/logger';
import { Readable } from 'node:stream';
import { writeFile } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { type CustomBuildContext } from '../../../customBuildContext';
import { uploadDeviceRunSessionArtifactAsync } from '../deviceRunSessionArtifacts';
import { uploadServeSimLogsFileAsync } from '../serveSimLogsArtifacts';
jest.mock('../deviceRunSessionArtifacts');
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const ctx = {} as CustomBuildContext;
const filePath = path.join(os.tmpdir(), 'logs.ndjson');
const args = { deviceRunSessionId: 'session', udid: 'A', filePath, logger };
it('uploads NDJSON with explicit user-app scope and destroys the stream afterward', async () => {
await writeFile(filePath, '{"pid":42}\n');
await uploadServeSimLogsFileAsync(ctx, args);
expect(uploadDeviceRunSessionArtifactAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({
kind: 'simulator-log',
filename: 'app-logs.ndjson',
size: 11,
metadata: expect.objectContaining({ scope: 'user-apps', udid: 'A' }),
})
);
expect(
(jest.mocked(uploadDeviceRunSessionArtifactAsync).mock.calls[0][1].stream as Readable).destroyed
).toBe(true);
});
it('skips empty and missing files', async () => {
await uploadServeSimLogsFileAsync(ctx, args);
await writeFile(filePath, '');
await uploadServeSimLogsFileAsync(ctx, args);
expect(uploadDeviceRunSessionArtifactAsync).not.toHaveBeenCalled();
});
it('handles a file read error while upload allocation is pending without an unhandled error', async () => {
await writeFile(filePath, '{}\n');
jest
.mocked(uploadDeviceRunSessionArtifactAsync)
.mockImplementationOnce(async (_ctx, { stream, signal }) => {
const aborted = new Promise<void>((_resolve, reject) => {
signal!.addEventListener('abort', () => reject(signal!.reason), { once: true });
});
(stream as Readable).destroy(new Error('read failure'));
await aborted;
});
await expect(uploadServeSimLogsFileAsync(ctx, args)).resolves.toBeUndefined();
expect(logger.warn).toHaveBeenCalledWith(
expect.objectContaining({ err: expect.any(Error) }),
expect.stringContaining('Could not upload')
);
});
@@ -0,0 +1,204 @@
import { type bunyan } from '@expo/logger';
import { mkdtemp, readFile, rm } from 'node:fs/promises';
import { type Server, createServer } from 'node:http';
import { type AddressInfo } from 'node:net';
import os from 'node:os';
import path from 'node:path';
import { type CustomBuildContext } from '../../../customBuildContext';
import { uploadServeSimLogsFileAsync } from '../serveSimLogsArtifacts';
import { streamServeSimLogsToFileAsync } from '../serveSimLogsRecorder';
// This test exercises actual sockets, UTF-8 decoding, files, and PUT uploads.
// Only WWW's allocation response is substituted; it does not verify cloud storage.
jest.mock('../../../sentry');
jest.unmock('fs');
jest.unmock('node:fs');
jest.unmock('fs/promises');
jest.unmock('node:fs/promises');
jest.unmock('node-fetch');
const logger = {
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
debug: jest.fn(),
} as unknown as bunyan;
let directory: string;
let server: Server;
let baseUrl: string;
beforeEach(async () => {
directory = await mkdtemp(path.join(os.tmpdir(), 'simulator-logs-http-test-'));
server = createServer();
await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve));
baseUrl = `http://127.0.0.1:${(server.address() as AddressInfo).port}`;
});
afterEach(async () => {
server.closeAllConnections();
await new Promise<void>((resolve, reject) =>
server.close(error => (error ? reject(error) : resolve()))
);
await rm(directory, { recursive: true, force: true });
});
it('streams selected-device logs into NDJSON and uploads the exact bytes as a session artifact', async () => {
const record = JSON.stringify({ eventMessage: 'Hello 서울 🌲', process: 'MyApp' });
const expected = Buffer.from(record + '\n');
const envelope = JSON.stringify({ seq: 7, at: 1000, raw: record });
const wire = Buffer.from(
`: heartbeat\n\ndata: ${envelope}\n\ndata: invalid json\n\ndata: {"partial":`
);
const splitAt = wire.indexOf(Buffer.from('서울')) + 1;
let streamRequest: { url?: string; authorization?: string } | undefined;
let uploaded: Buffer | undefined;
let uploadedLength: string | undefined;
server.on('request', (request, response) => {
if (request.method === 'GET') {
streamRequest = { url: request.url, authorization: request.headers.authorization };
response.writeHead(200, {
'Content-Type': 'text/event-stream',
'X-Serve-Sim-Log-Scope': 'user-apps',
});
response.write(wire.subarray(0, splitAt));
setImmediate(() => response.end(wire.subarray(splitAt)));
return;
}
const chunks: Buffer[] = [];
request.on('data', chunk => chunks.push(Buffer.from(chunk)));
request.on('end', () => {
uploaded = Buffer.concat(chunks);
uploadedLength = request.headers['content-length'];
response.writeHead(200).end();
});
});
const filePath = path.join(directory, 'simulator.ndjson');
const result = await streamServeSimLogsToFileAsync({
serveSimUrl: baseUrl,
serveSimToken: 'local-test-token',
serveSimDevice: 'device-B',
filePath,
signal: new AbortController().signal,
logger,
});
expect(streamRequest).toEqual({
url: '/logs?envelope=true&scope=user-apps&device=device-B',
authorization: 'Bearer local-test-token',
});
expect(result).toEqual({
receivedData: true,
limitReached: false,
bytesWritten: expected.length,
lastSequence: 7,
});
expect(await readFile(filePath)).toEqual(expected);
const mutation = jest.fn().mockReturnValue({
toPromise: async () => ({
data: {
deviceRunSession: {
createArtifactUploadSession: {
uploadSession: {
url: `${baseUrl}/upload`,
headers: { 'Content-Length': String(expected.length) },
},
},
},
},
}),
});
await uploadServeSimLogsFileAsync(
{ graphqlClient: { mutation } } as unknown as CustomBuildContext,
{ deviceRunSessionId: 'session-test', udid: 'device-B', filePath, logger }
);
expect(mutation.mock.calls[0][1]).toEqual({
deviceRunSessionId: 'session-test',
input: {
name: 'App logs (device-B)',
filename: 'app-logs.ndjson',
kind: 'simulator-log',
metadata: {
__eas_type: 'simulator-log',
udid: 'device-B',
scope: 'user-apps',
source: 'serve-sim/logs',
},
size: expected.length,
},
});
expect(uploaded).toEqual(expected);
expect(uploadedLength).toBe(String(expected.length));
// Only the invalid JSON record is reported; the partial trailing record is not a record yet.
expect(logger.warn).toHaveBeenCalledTimes(1);
expect(logger.warn).toHaveBeenCalledWith(
{ malformedRecords: 1 },
expect.stringContaining('not valid JSON')
);
});
it('resumes a buffered stream without appending already persisted sequence numbers', async () => {
const urls: string[] = [];
const first = JSON.stringify({ eventMessage: 'first' });
const second = JSON.stringify({ eventMessage: 'second' });
const frame = (seq: number, raw: string): string =>
`data: ${JSON.stringify({ seq, at: seq * 1000, raw })}\n\n`;
server.on('request', (request, response) => {
urls.push(request.url!);
response.writeHead(200, {
'Content-Type': 'text/event-stream',
'X-Serve-Sim-Log-Scope': 'user-apps',
});
// Deliberately resend seq=1 on reconnect to exercise the client's guard too.
response.end(frame(1, first) + (urls.length > 1 ? frame(2, second) : ''));
});
const filePath = path.join(directory, 'resumed.ndjson');
const options = {
serveSimUrl: baseUrl,
serveSimDevice: 'device-A',
filePath,
signal: new AbortController().signal,
logger,
};
const initial = await streamServeSimLogsToFileAsync(options);
const resumed = await streamServeSimLogsToFileAsync({
...options,
since: initial.lastSequence,
});
expect(initial.lastSequence).toBe(1);
expect(resumed.lastSequence).toBe(2);
expect(resumed.bytesWritten).toBe(Buffer.byteLength(second + '\n'));
expect(urls).toEqual([
'/logs?envelope=true&scope=user-apps&device=device-A',
'/logs?envelope=true&scope=user-apps&since=1&device=device-A',
]);
expect(await readFile(filePath, 'utf8')).toBe(first + '\n' + second + '\n');
});
it('aborts a real open HTTP stream without waiting for the server to finish', async () => {
const controller = new AbortController();
let sawRequest: () => void = () => {};
const requestStarted = new Promise<void>(resolve => (sawRequest = resolve));
server.on('request', (_request, response) => {
response.writeHead(200, {
'Content-Type': 'text/event-stream',
'X-Serve-Sim-Log-Scope': 'user-apps',
});
response.write(': heartbeat\n\n');
sawRequest();
});
const streaming = streamServeSimLogsToFileAsync({
serveSimUrl: baseUrl,
filePath: path.join(directory, 'aborted.ndjson'),
signal: controller.signal,
logger,
});
await requestStarted;
controller.abort();
await expect(streaming).resolves.toEqual({
receivedData: false,
limitReached: false,
bytesWritten: 0,
});
});
@@ -0,0 +1,280 @@
import { type bunyan } from '@expo/logger';
import fetch from 'node-fetch';
import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { Readable } from 'node:stream';
import { setTimeout as delay } from 'node:timers/promises';
import { Sentry } from '../../../sentry';
import { ServeSimLogsRecorder, streamServeSimLogsToFileAsync } from '../serveSimLogsRecorder';
jest.mock('node-fetch');
jest.mock('../../../sentry');
jest.unmock('node:fs');
jest.unmock('node:fs/promises');
const { Response: FetchResponse } = jest.requireActual('node-fetch') as typeof import('node-fetch');
// A compatible serve-sim explicitly acknowledges the requested log scope.
class Response extends FetchResponse {
constructor(...args: ConstructorParameters<typeof FetchResponse>) {
super(...args);
this.headers.set('x-serve-sim-log-scope', 'user-apps');
}
}
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
let directory: string;
beforeEach(async () => {
jest.mocked(fetch).mockReset();
directory = await mkdtemp(path.join(os.tmpdir(), 'logs-test-'));
});
afterEach(async () => {
const files = await ServeSimLogsRecorder.finishAsync({ logger });
for (const file of files) {
await rm(path.dirname(file.filePath), { recursive: true, force: true });
}
await rm(directory, { recursive: true, force: true });
});
function record(options: Partial<Parameters<typeof streamServeSimLogsToFileAsync>[0]> = {}) {
return streamServeSimLogsToFileAsync({
serveSimUrl: 'http://localhost:1234',
filePath: path.join(directory, 'logs.ndjson'),
signal: new AbortController().signal,
logger,
...options,
});
}
it('preserves split UTF-8 and CRLF records, skips heartbeat and reports invalid JSON, and selects a device', async () => {
const input = Buffer.from(
':\r\n\r\ndata: {"message":"안녕"}\r\n\r\ndata: bad\n\ndata: {"pid":42}\n\n'
);
jest
.mocked(fetch)
.mockResolvedValue(new Response(Readable.from([...input].map(byte => Buffer.from([byte])))));
const result = await record({ serveSimToken: 'secret', serveSimDevice: 'device two' });
expect(result.receivedData).toBe(true);
expect(await readFile(path.join(directory, 'logs.ndjson'), 'utf8')).toBe(
'{"message":"안녕"}\n{"pid":42}\n'
);
expect(Sentry.capture).toHaveBeenCalledWith('serve-sim sent malformed simulator log records', {
extras: { malformedRecords: 1, serveSimDevice: 'device two' },
});
expect(fetch).toHaveBeenCalledWith(
'http://localhost:1234/logs?envelope=true&scope=user-apps&device=device+two',
expect.objectContaining({ headers: { Authorization: 'Bearer secret' } })
);
});
it('retains completed records when a connection errors', async () => {
const stream = Readable.from(
(async function* () {
yield Buffer.from('data: {"pid":42}\n\n');
await delay(10);
throw new Error('disconnect');
})()
);
jest.mocked(fetch).mockResolvedValue(new Response(stream));
expect(await record()).toEqual({ receivedData: true, bytesWritten: 11, limitReached: false });
expect(await readFile(path.join(directory, 'logs.ndjson'), 'utf8')).toBe('{"pid":42}\n');
});
it.each([undefined, 'all'])('refuses an unconfirmed user-app scope (%s)', async scope => {
const body = Readable.from(['data: {"eventMessage":"system noise"}\n\n']);
const response = new FetchResponse(body);
if (scope) {
response.headers.set('x-serve-sim-log-scope', scope);
}
jest.mocked(fetch).mockResolvedValue(response);
expect(await record()).toEqual({ receivedData: false, bytesWritten: 0, limitReached: false });
expect(body.destroyed).toBe(true);
await expect(readFile(path.join(directory, 'logs.ndjson'))).rejects.toMatchObject({
code: 'ENOENT',
});
expect(logger.warn).toHaveBeenCalledWith(expect.stringContaining('did not confirm user-app'));
});
it('aborts a quiet live stream and drains preceding writes before returning', async () => {
const controller = new AbortController();
const stream = new Readable({ read() {} });
stream.push('data: {"pid":42}\n\n');
jest.mocked(fetch).mockResolvedValue(new Response(stream));
const done = record({ signal: controller.signal });
await delay(20);
controller.abort();
expect((await done).receivedData).toBe(true);
expect(stream.destroyed).toBe(true);
expect(await readFile(path.join(directory, 'logs.ndjson'), 'utf8')).toBe('{"pid":42}\n');
});
it('enforces the byte limit without writing a truncated record', async () => {
jest
.mocked(fetch)
.mockResolvedValue(new Response(Readable.from(['data: {"pid":42}\n\ndata: {"pid":43}\n\n'])));
expect(await record({ maxBytes: 15 })).toEqual({
receivedData: true,
bytesWritten: 11,
limitReached: true,
});
expect(await readFile(path.join(directory, 'logs.ndjson'), 'utf8')).toBe('{"pid":42}\n');
});
it('drops an oversized partial SSE line and resumes at the next complete record', async () => {
jest
.mocked(fetch)
.mockResolvedValue(
new Response(
Readable.from(['data: ' + 'x'.repeat(1024 * 1024 + 1), 'tail\n\ndata: {"pid":42}\n\n'])
)
);
expect((await record()).bytesWritten).toBe(11);
});
it('warns and resolves on HTTP and file failures', async () => {
jest.mocked(fetch).mockResolvedValueOnce(new Response('missing', { status: 404 }));
expect((await record()).receivedData).toBe(false);
jest.mocked(fetch).mockResolvedValueOnce(new Response(Readable.from(['data: {}\n\n'])));
expect((await record({ filePath: '/missing-dir/log.ndjson' })).receivedData).toBe(false);
expect(logger.warn).toHaveBeenCalledTimes(2);
});
async function register(): Promise<string> {
const stateDir = path.join(directory, 'state');
await mkdir(stateDir);
await writeFile(
path.join(stateDir, 'server-A.json'),
JSON.stringify({ device: 'A', url: 'http://localhost:1234' })
);
return stateDir;
}
it('caps bytes across reconnects and finalizes only once even concurrently', async () => {
const stateDir = await register();
let seq = 0;
jest
.mocked(fetch)
.mockImplementation(
async () =>
new Response(
Readable.from([`data: ${JSON.stringify({ seq: ++seq, raw: '{"pid":42}' })}\n\n`])
)
);
await ServeSimLogsRecorder.startAsync({ logger, stateDir, pollIntervalMs: 5, maxBytes: 22 });
await delay(100);
expect(fetch).toHaveBeenCalledTimes(2);
const [first, second] = await Promise.all([
ServeSimLogsRecorder.finishAsync({ logger }),
ServeSimLogsRecorder.finishAsync({ logger }),
]);
expect(first).toHaveLength(1);
expect(second).toEqual([]);
expect(await readFile(first[0].filePath, 'utf8')).toBe('{"pid":42}\n{"pid":42}\n');
expect(await ServeSimLogsRecorder.finishAsync({ logger })).toEqual([]);
});
it('returns immediately before a server exists and discovers a later registration', async () => {
const stateDir = path.join(directory, 'state');
jest.mocked(fetch).mockImplementation(async () => new Response(Readable.from(['data: {}\n\n'])));
await ServeSimLogsRecorder.startAsync({ logger, stateDir, pollIntervalMs: 5 });
expect(fetch).not.toHaveBeenCalled();
await register();
await delay(30);
expect(await ServeSimLogsRecorder.finishAsync({ logger })).toHaveLength(1);
});
it('stops discovery and active streams at the session time limit', async () => {
const stateDir = await register();
const stream = new Readable({ read() {} });
stream.push('data: {}\n\n');
jest.mocked(fetch).mockResolvedValue(new Response(stream));
await ServeSimLogsRecorder.startAsync({ logger, stateDir, pollIntervalMs: 5, maxDurationMs: 30 });
await delay(80);
expect(stream.destroyed).toBe(true);
expect(fetch).toHaveBeenCalledTimes(1);
expect(await ServeSimLogsRecorder.finishAsync({ logger })).toHaveLength(1);
});
it('stops retrying after ten consecutive failures', async () => {
const stateDir = await register();
jest.mocked(fetch).mockImplementation(async () => new Response('bad', { status: 503 }));
await ServeSimLogsRecorder.startAsync({ logger, stateDir, pollIntervalMs: 5 });
await delay(150);
expect(fetch).toHaveBeenCalledTimes(10);
expect(await ServeSimLogsRecorder.finishAsync({ logger })).toEqual([]);
});
it('selects each registered device when two simulators share one server URL', async () => {
const stateDir = await register();
await writeFile(
path.join(stateDir, 'server-B.json'),
JSON.stringify({ device: 'B', url: 'http://localhost:1234' })
);
jest.mocked(fetch).mockImplementation(async () => new Response(Readable.from(['data: {}\n\n'])));
await ServeSimLogsRecorder.startAsync({ logger, stateDir, pollIntervalMs: 5, maxBytes: 3 });
await delay(50);
expect(fetch).toHaveBeenCalledWith(
'http://localhost:1234/logs?envelope=true&scope=user-apps&device=A',
expect.anything()
);
expect(fetch).toHaveBeenCalledWith(
'http://localhost:1234/logs?envelope=true&scope=user-apps&device=B',
expect.anything()
);
expect(fetch).toHaveBeenCalledTimes(2);
});
it('persists the envelope cursor only for written records and drops replayed sequences', async () => {
const frames = [1, 2, 3].map(
seq => `data: ${JSON.stringify({ seq, raw: JSON.stringify({ pid: seq }) })}\n\n`
);
jest.mocked(fetch).mockResolvedValue(new Response(Readable.from(frames)));
expect(await record({ since: 1, maxBytes: 10 })).toEqual({
receivedData: true,
bytesWritten: 10,
limitReached: true,
lastSequence: 2,
});
expect(await readFile(path.join(directory, 'logs.ndjson'), 'utf8')).toBe('{"pid":2}\n');
expect(fetch).toHaveBeenCalledWith(
'http://localhost:1234/logs?envelope=true&scope=user-apps&since=1',
expect.anything()
);
});
it('does not reconnect a legacy data stream without a replay cursor', async () => {
const stateDir = await register();
jest.mocked(fetch).mockImplementation(async () => new Response(Readable.from(['data: {}\n\n'])));
await ServeSimLogsRecorder.startAsync({ logger, stateDir, pollIntervalMs: 5 });
await delay(50);
expect(fetch).toHaveBeenCalledTimes(1);
expect(logger.warn).toHaveBeenCalledWith(expect.stringContaining('without a replay cursor'));
});
it('resets the replay cursor when a replacement server changes its token', async () => {
const stateDir = await register();
let calls = 0;
jest.mocked(fetch).mockImplementation(async (_url, options) => {
calls++;
if (calls === 1) {
await writeFile(
path.join(stateDir, 'server-A.json'),
JSON.stringify({
device: 'A',
url: 'http://localhost:1234',
token: 'new-token',
})
);
}
const seq = options?.headers ? 1 : 50;
return new Response(Readable.from([`data: ${JSON.stringify({ seq, raw: '{}' })}\n\n`]));
});
await ServeSimLogsRecorder.startAsync({ logger, stateDir, pollIntervalMs: 5, maxBytes: 6 });
await delay(50);
expect(fetch).toHaveBeenCalledTimes(2);
expect(String(jest.mocked(fetch).mock.calls[1][0])).not.toContain('since=');
expect(jest.mocked(fetch).mock.calls[1][1]?.headers).toEqual({
Authorization: 'Bearer new-token',
});
});
@@ -0,0 +1,105 @@
jest.unmock('node:fs/promises');
import { ShellSessionManager } from '../shellSessionManager';
import * as processes from '../../../utils/processes';
describe('ShellSessionManager', () => {
it.each([false, true])('uses only the supplied environment (tty: %s)', async tty => {
const controller = new AbortController();
const previous = process.env.SANDBOX_WORKER_ONLY;
process.env.SANDBOX_WORKER_ONLY = 'worker-secret';
const manager = new ShellSessionManager({
workingDirectory: process.cwd(),
env: { SHELL: '/bin/sh', SANDBOX_PREPARED: 'prepared-value' },
signal: controller.signal,
});
try {
const sessionId = await manager.startAsync({
cmd: 'printf "%s:%s" "$SANDBOX_PREPARED" "${SANDBOX_WORKER_ONLY-unset}"',
tty,
});
expect(await manager.readAsync(sessionId, 10_000)).toEqual({
output: 'prepared-value:unset',
exitCode: 0,
});
} finally {
controller.abort();
await manager.stoppedPromise;
if (previous === undefined) {
delete process.env.SANDBOX_WORKER_ONLY;
} else {
process.env.SANDBOX_WORKER_ONLY = previous;
}
}
});
it.each([false, true])('does not signal an exited leader (tty: %s)', async tty => {
const controller = new AbortController();
const manager = new ShellSessionManager({
workingDirectory: process.cwd(),
env: process.env,
signal: controller.signal,
});
const kill = jest.spyOn(processes, 'killProcessGroup');
try {
const sessionId = await manager.startAsync({ cmd: 'exit 0', tty });
expect(await manager.readAsync(sessionId, 10_000)).toMatchObject({ exitCode: 0 });
controller.abort();
await manager.stoppedPromise;
expect(kill).not.toHaveBeenCalled();
} finally {
controller.abort();
await manager.stoppedPromise;
kill.mockRestore();
}
});
it('keeps completed sessions without repeating their output', async () => {
const controller = new AbortController();
const manager = new ShellSessionManager({
workingDirectory: process.cwd(),
env: process.env,
signal: controller.signal,
});
try {
const sessionId = await manager.startAsync({ cmd: 'printf hello' });
expect(await manager.readAsync(sessionId, 1_000)).toEqual({ output: 'hello', exitCode: 0 });
expect(await manager.readAsync(sessionId, 1_000)).toEqual({ output: '', exitCode: 0 });
} finally {
controller.abort();
await manager.stoppedPromise;
}
});
it('keeps sessions separate and stops only its own manager', async () => {
const firstController = new AbortController();
const secondController = new AbortController();
const first = new ShellSessionManager({
workingDirectory: process.cwd(),
env: process.env,
signal: firstController.signal,
});
const second = new ShellSessionManager({
workingDirectory: process.cwd(),
env: process.env,
signal: secondController.signal,
});
try {
const firstId = await first.startAsync({ cmd: 'printf first' });
const secondId = await second.startAsync({ cmd: 'read value; printf "%s" "$value"' });
expect(await first.readAsync(firstId, 1_000)).toEqual({ output: 'first', exitCode: 0 });
firstController.abort();
await first.stoppedPromise;
await expect(first.startAsync({ cmd: 'printf stopped' })).rejects.toMatchObject({
name: 'AbortError',
});
expect(await second.readAsync(secondId, 0)).toEqual({ output: '', sessionId: secondId });
second.write(secondId, 'second\n');
expect(await second.readAsync(secondId, 1_000)).toEqual({ output: 'second', exitCode: 0 });
} finally {
firstController.abort();
secondController.abort();
await Promise.all([first.stoppedPromise, second.stoppedPromise]);
}
});
});
@@ -0,0 +1,91 @@
import { bunyan } from '@expo/logger';
import { createStartupTasks } from '../startupTasks';
function createLoggerMock(): bunyan {
return {
info: jest.fn(),
warn: jest.fn(),
child: jest.fn().mockReturnThis(),
} as unknown as bunyan;
}
describe(createStartupTasks, () => {
it('logs each task and sums them up in the order they finished', async () => {
const logger = createLoggerMock();
const tasks = createStartupTasks(logger);
let finishSlow!: () => void;
const slow = tasks.run(
'slow task',
() =>
new Promise<string>(resolve => {
finishSlow = () => resolve('slow');
})
);
const fast = tasks.run('fast task', async () => 'fast');
await expect(fast).resolves.toBe('fast');
finishSlow();
await expect(slow).resolves.toBe('slow');
expect(logger.info).toHaveBeenCalledWith('Starting slow task.');
expect(logger.info).toHaveBeenCalledWith(
expect.stringMatching(/^Finished fast task in \d+\.\d s\.$/)
);
expect(tasks.summary()).toMatch(
/^Startup took \d+\.\d s \(fast task \d+\.\d s, slow task \d+\.\d s\)\.$/
);
});
it('keeps the error for awaiters without an unhandled rejection', async () => {
const logger = createLoggerMock();
const tasks = createStartupTasks(logger);
const failing = tasks.run('failing task', async () => {
throw new Error('boom');
});
// Nothing awaits the task yet. Jest fails the test on an unhandled rejection.
await new Promise(resolve => setImmediate(resolve));
await expect(failing).rejects.toThrow('boom');
expect(logger.warn).toHaveBeenCalledWith(expect.stringMatching(/^failing task failed after/));
expect(tasks.summary()).toMatch(/\(\)\.$/);
});
it('aborts the signal with the first failure and gives each task a child logger', async () => {
const logger = createLoggerMock();
const tasks = createStartupTasks(logger);
let taskLogger: unknown;
await tasks.run('named task', async l => {
taskLogger = l;
});
expect(logger.child).toHaveBeenCalledWith({ startupTask: 'named task' });
expect(taskLogger).toBe(logger);
expect(tasks.signal.aborted).toBe(false);
const first = new Error('first failure');
await expect(tasks.run('first', async () => Promise.reject(first))).rejects.toBe(first);
await expect(
tasks.run('second', async () => Promise.reject(new Error('second failure')))
).rejects.toThrow('second failure');
expect(tasks.signal.aborted).toBe(true);
expect(tasks.signal.reason).toBe(first);
});
it('stops waiting on untilAborted when startup is aborted', async () => {
const tasks = createStartupTasks(createLoggerMock());
const never = new Promise<void>(() => {});
const waiting = tasks.untilAborted(never);
const failure = new Error('boot failed');
tasks.abort(failure);
await expect(waiting).rejects.toBe(failure);
await expect(tasks.untilAborted(Promise.resolve('late'))).rejects.toBe(failure);
await expect(
createStartupTasks(createLoggerMock()).untilAborted(Promise.resolve('ok'))
).resolves.toBe('ok');
});
});
@@ -0,0 +1,860 @@
import { SystemError } from '@expo/eas-build-job';
import { type bunyan } from '@expo/logger';
import { asyncResult } from '@expo/results';
import { type BuildStepEnv } from '@expo/steps';
import spawn from '@expo/turtle-spawn';
import fs from 'node:fs';
import net from 'node:net';
import path from 'node:path';
import readline from 'node:readline';
import {
type AndroidDeviceSerialId,
type AndroidEmulatorLaunchGate,
} from '../../utils/AndroidEmulatorUtils';
import { sleepAsync } from '../../utils/retry';
import {
LOCAL_EGRESS_HANDOFF_PATH,
LOCAL_EGRESS_PROXY_HOST,
readLocalEgressHandoffAsync,
} from './localEgress';
import {
type GuardEvent,
GuardEventRelay,
registerLocalEgressGuardRelay,
} from './localEgressGuard';
/**
* Android emulator side of local egress, on Linux device hosts.
*
* - The emulator's `-http-proxy` points at a relay on LOCAL_EGRESS_RELAY_PORT.
* The relay always listens, because qemu tests the proxy once at startup and
* ignores it for the emulator's whole life if that test fails. It forwards
* each connection to the chisel listener, which exists only while the EAS
* CLI egress client is connected, and refuses the request otherwise.
* - The guest's system proxy is the relay as well (10.0.2.2 is host loopback),
* so proxy-aware clients send hostnames instead of IP addresses.
* - Every emulator process runs in one systemd scope. An nftables table refuses
* traffic from that scope that is not loopback TCP or DNS to
* systemd-resolved: UDP, QUIC, ICMP and the emulator's
* own direct connections. Kernel log lines from the fence become session log
* lines through the guard relay.
*
* This keeps routing faithful; it is not a security boundary.
*/
export const LOCAL_EGRESS_RELAY_PORT = 8898;
export const ANDROID_EMULATOR_HOST_LOOPBACK_ALIAS = '10.0.2.2';
/** TEST-NET-1; the relay answers CONNECTs to it itself, so it proves the proxy path is up. */
export const LOCAL_EGRESS_READINESS_TARGET = { host: '192.0.2.1', port: 443 } as const;
export const LOCAL_EGRESS_SESSION_UNIT = 'eas-egress-session';
export const EGRESS_FENCE_TABLE = 'eas_fence';
const FENCE_LOG_PREFIX = 'eas-egress';
const FENCE_RELAY_KEY = 'android-emulator-fence';
const FENCED_PROCESS_LABEL = 'Android emulator';
const FENCED_PROCESS_PATTERN = /^(qemu-system|netsimd|crashpad_handle)/;
const RELAY_MAX_HEAD_BYTES = 16 * 1024;
const SCOPE_STARTUP_TIMEOUT_MS = 10_000;
const ADB_REVERSE_INTERVAL_MS = 3_000;
const GATE_SCRIPT = 'read -r _ || exit 1; exec "$@" < /dev/null';
const CONNECTION_ESTABLISHED = 'HTTP/1.1 200 Connection Established\r\n\r\n';
const CLIENT_NOT_CONNECTED_BODY = 'The EAS CLI local egress client is not connected.\r\n';
const CLIENT_NOT_CONNECTED =
'HTTP/1.1 503 Service Unavailable\r\nConnection: close\r\nContent-Type: text/plain\r\n' +
`Content-Length: ${Buffer.byteLength(CLIENT_NOT_CONNECTED_BODY)}\r\n\r\n${CLIENT_NOT_CONNECTED_BODY}`;
// ---------------------------------------------------------------------------
// Relay
// ---------------------------------------------------------------------------
export function parseConnectAuthority(head: string): { host: string; port: number } | null {
const match = /^CONNECT\s+(\[[^\]]+\]|[^\s:]+):(\d{1,5})\s+HTTP\/1\.[01]\r?\n/i.exec(head);
if (!match) {
return null;
}
return { host: match[1].replace(/^\[|\]$/g, '').toLowerCase(), port: Number(match[2]) };
}
function isReadinessTarget({ host, port }: { host: string; port: number }): boolean {
return host === LOCAL_EGRESS_READINESS_TARGET.host && port === LOCAL_EGRESS_READINESS_TARGET.port;
}
/**
* On the emulator's Wi-Fi path, a guest connection to the guest proxy
* (10.0.2.2:<relay port>) is itself wrapped in `-http-proxy`'s CONNECT.
*/
function isRelayTarget({ host, port }: { host: string; port: number }, relayPort: number): boolean {
return (
port === relayPort &&
(host === LOCAL_EGRESS_PROXY_HOST ||
host === ANDROID_EMULATOR_HOST_LOOPBACK_ALIAS ||
host === 'localhost')
);
}
export type LocalEgressRelay = {
port: number;
setLogger: (logger: bunyan) => void;
stopAsync: () => Promise<void>;
};
export async function startLocalEgressRelayAsync({
port = LOCAL_EGRESS_RELAY_PORT,
upstreamPort,
logger,
}: {
port?: number;
upstreamPort: number;
logger: bunyan;
}): Promise<LocalEgressRelay> {
let currentLogger = logger;
let refusedWhileDetached = 0;
const sockets = new Set<net.Socket>();
const track = (socket: net.Socket): void => {
sockets.add(socket);
socket.on('error', () => socket.destroy());
socket.once('close', () => sockets.delete(socket));
};
const endWith = (client: net.Socket, response: string): void => {
client.end(response, () => client.destroy());
};
const refuse = (client: net.Socket): void => {
if (refusedWhileDetached === 0) {
currentLogger.warn(
'Local egress: the Android emulator made a request while the EAS CLI egress client is not ' +
'connected, so it was refused. Requests from the emulator fail until the client connects.'
);
}
refusedWhileDetached++;
endWith(client, CLIENT_NOT_CONNECTED);
};
const attached = (): void => {
if (refusedWhileDetached > 0) {
currentLogger.info(
`Local egress: the EAS CLI egress client is connected. ${refusedWhileDetached} request(s) ` +
'from the Android emulator were refused while it was not.'
);
refusedWhileDetached = 0;
}
};
const forward = (client: net.Socket, buffered: Buffer): void => {
const upstream = net.connect({ host: LOCAL_EGRESS_PROXY_HOST, port: upstreamPort });
track(upstream);
let connected = false;
upstream.once('connect', () => {
connected = true;
attached();
if (buffered.length > 0) {
upstream.write(buffered);
}
client.pipe(upstream);
upstream.pipe(client);
client.resume();
});
upstream.once('error', () => {
if (!connected && !client.destroyed) {
refuse(client);
}
});
upstream.once('close', () => {
if (connected) {
client.destroy();
}
});
client.once('close', () => upstream.destroy());
};
const route = (client: net.Socket, initial: Buffer): void => {
let buffered = initial;
const decide = (): void => {
const headEnd = buffered.indexOf('\r\n\r\n');
if (headEnd === -1 && buffered.length < RELAY_MAX_HEAD_BYTES) {
return;
}
client.off('data', onData);
client.pause();
const target =
headEnd === -1
? null
: parseConnectAuthority(buffered.subarray(0, headEnd + 2).toString('latin1'));
if (target && isReadinessTarget(target)) {
endWith(client, CONNECTION_ESTABLISHED);
} else if (target && isRelayTarget(target, port)) {
client.write(CONNECTION_ESTABLISHED);
route(client, buffered.subarray(headEnd + 4));
} else {
forward(client, buffered);
}
};
const onData = (chunk: Buffer): void => {
buffered = Buffer.concat([buffered, chunk]);
decide();
};
client.on('data', onData);
client.resume();
if (buffered.length > 0) {
decide();
}
};
const server = net.createServer(client => {
track(client);
route(client, Buffer.alloc(0));
});
await new Promise<void>((resolve, reject) => {
server.once('error', reject);
server.listen(port, LOCAL_EGRESS_PROXY_HOST, () => {
server.removeListener('error', reject);
resolve();
});
});
server.unref();
return {
port,
setLogger: nextLogger => {
currentLogger = nextLogger;
},
stopAsync: async () => {
await new Promise<void>(resolve => {
server.close(() => resolve());
for (const socket of sockets) {
socket.destroy();
}
});
},
};
}
// ---------------------------------------------------------------------------
// Fence
// ---------------------------------------------------------------------------
export function buildEgressFenceRuleset({ cgroupLevel }: { cgroupLevel: number }): string {
return [
`table inet ${EGRESS_FENCE_TABLE} {`,
' set fenced { type cgroupsv2; }',
' counter fence_tcp {}',
' counter fence_udp {}',
' counter fence_other {}',
' chain out {',
' type filter hook output priority filter; policy accept;',
' ct state established,related accept',
` socket cgroupv2 level ${cgroupLevel} @fenced jump fence`,
' }',
' chain fence {',
// The emulator and netsimd talk to each other and to adb over loopback TCP;
// guest TCP reaches loopback only through the proxy.
' oif "lo" meta l4proto tcp accept',
' oif "lo" ip daddr 127.0.0.53 udp dport 53 accept',
` limit rate 50/second burst 200 packets log prefix "${FENCE_LOG_PREFIX} " level info`,
' meta l4proto tcp counter name fence_tcp reject with tcp reset',
' meta l4proto udp counter name fence_udp reject with icmpx admin-prohibited',
' counter name fence_other reject with icmpx admin-prohibited',
' }',
'}',
'',
].join('\n');
}
function compressIpv6(address: string): string {
try {
return new URL(`http://[${address}]`).hostname.slice(1, -1);
} catch {
return address;
}
}
/**
* One kernel log line from the fence's `log` rule, for example
* `eas-egress IN= OUT=ens3 SRC=10.0.0.2 DST=142.250.1.1 LEN=60 ... PROTO=TCP SPT=40000 DPT=443 ...`.
* nft sees the emulator's sockets, not the guest app behind them, so events
* carry no pid or callers.
*/
export function parseFenceLogLine(line: string): GuardEvent | null {
const match = new RegExp(`${FENCE_LOG_PREFIX} (IN=.*)$`).exec(line);
if (!match) {
return null;
}
const fields = new Map<string, string>();
for (const token of match[1].split(/\s+/)) {
const separator = token.indexOf('=');
if (separator > 0) {
fields.set(token.slice(0, separator), token.slice(separator + 1));
}
}
const protocol = fields.get('PROTO')?.toLowerCase();
const destination = fields.get('DST');
if (!protocol || !destination) {
return null;
}
const fn = protocol === 'icmpv6' ? 'icmp' : protocol;
const host = net.isIPv6(destination) ? compressIpv6(destination) : destination;
const destinationPort = fields.get('DPT');
const peer = destinationPort
? `${net.isIPv6(host) ? `[${host}]` : host}:${destinationPort}`
: host;
return {
process: FENCED_PROCESS_LABEL,
pid: null,
function: fn,
action: 'blocked',
peer,
callers: [],
note: describeFenceEvent(fn, destinationPort),
};
}
function describeFenceEvent(fn: string, destinationPort: string | undefined): string | undefined {
if (fn === 'tcp') {
return 'a connection the emulator itself opened, such as its update check; app TCP goes through the proxy';
}
if (fn === 'udp' && destinationPort === '443') {
return 'QUIC; clients usually retry over TCP through the proxy';
}
return undefined;
}
export type FenceCounters = { tcp: number; udp: number; other: number };
/** `nft -j list counters table inet eas_fence` output. */
export function parseFenceCounters(json: string): FenceCounters {
const counters: FenceCounters = { tcp: 0, udp: 0, other: 0 };
const parsed = JSON.parse(json) as {
nftables?: { counter?: { name?: string; packets?: number } }[];
};
for (const entry of parsed.nftables ?? []) {
const name = entry.counter?.name;
const packets = entry.counter?.packets ?? 0;
if (name === 'fence_tcp') {
counters.tcp = packets;
} else if (name === 'fence_udp') {
counters.udp = packets;
} else if (name === 'fence_other') {
counters.other = packets;
}
}
return counters;
}
export function getCgroupLevel(cgroupPath: string): number {
return cgroupPath.split('/').filter(Boolean).length;
}
async function startSessionScopeAsync({ env }: { env: BuildStepEnv }): Promise<string> {
const scope = spawn(
'sudo',
[
'systemd-run',
'--scope',
'--quiet',
`--unit=${LOCAL_EGRESS_SESSION_UNIT}`,
'--',
'sleep',
'infinity',
],
{ env, detached: true, stdio: 'ignore' }
);
scope.child.unref();
let exited = false;
void asyncResult(scope).then(() => {
exited = true;
});
const deadline = Date.now() + SCOPE_STARTUP_TIMEOUT_MS;
while (Date.now() < deadline) {
const result = await asyncResult(
spawn(
'systemctl',
['show', '-p', 'ControlGroup', '--value', `${LOCAL_EGRESS_SESSION_UNIT}.scope`],
{ env, stdio: 'pipe' }
)
);
const cgroupPath = result.ok ? result.value.stdout.trim().replace(/^\//, '') : '';
if (cgroupPath) {
return cgroupPath;
}
if (exited) {
break;
}
await sleepAsync(200);
}
throw new SystemError(
`Could not start the ${LOCAL_EGRESS_SESSION_UNIT} systemd scope for the Android emulator, so ` +
'local egress cannot keep its traffic on the tunnel. Retry the session; if it keeps failing, ' +
'please contact support.'
);
}
async function loadFenceAsync({
cgroupPath,
workDir,
env,
}: {
cgroupPath: string;
workDir: string;
env: BuildStepEnv;
}): Promise<void> {
const rulesetPath = path.join(workDir, 'fence.nft');
await fs.promises.writeFile(
rulesetPath,
buildEgressFenceRuleset({ cgroupLevel: getCgroupLevel(cgroupPath) })
);
await asyncResult(
spawn('sudo', ['nft', 'delete', 'table', 'inet', EGRESS_FENCE_TABLE], { env, stdio: 'pipe' })
);
try {
await spawn('sudo', ['nft', '-f', rulesetPath], { env, stdio: 'pipe' });
await spawn(
'sudo',
['nft', 'add', 'element', 'inet', EGRESS_FENCE_TABLE, 'fenced', `{ "${cgroupPath}" }`],
{ env, stdio: 'pipe' }
);
} catch (err) {
const failed = err as { stderr?: string };
throw new SystemError(
'Could not install the network fence for the Android emulator, so local egress cannot keep its ' +
'traffic on the tunnel. Retry the session; if it keeps failing, please contact support.' +
(failed.stderr ? ` nft output: ${failed.stderr.trim()}` : ''),
{ cause: err }
);
}
}
async function readFenceCountersAsync({ env }: { env: BuildStepEnv }): Promise<FenceCounters> {
const result = await spawn(
'sudo',
['nft', '-j', 'list', 'counters', 'table', 'inet', EGRESS_FENCE_TABLE],
{ env, stdio: 'pipe' }
);
return parseFenceCounters(result.stdout);
}
function startFenceLogReader({
env,
sinceEpochSeconds,
relay,
getLogger,
}: {
env: BuildStepEnv;
sinceEpochSeconds: number;
relay: GuardEventRelay;
getLogger: () => bunyan;
}): { stopAsync: () => Promise<void> } {
const reader = spawn(
'sudo',
['journalctl', '-k', '-f', '-o', 'cat', '--since', `@${sinceEpochSeconds}`],
{ env, stdio: ['ignore', 'pipe', 'pipe'], ignoreStdio: true }
);
let stopping = false;
const lines = reader.child.stdout
? readline.createInterface({ input: reader.child.stdout }).on('line', line => {
const event = parseFenceLogLine(line);
if (event) {
relay.handle(event);
}
})
: undefined;
const finished = asyncResult(reader).then(() => {
if (!stopping) {
getLogger().warn(
'Local egress: stopped reading the kernel log, so further connections the Android emulator ' +
'fence refuses are not listed here. They are still refused.'
);
}
});
reader.child.unref();
for (const stream of [reader.child.stdout, reader.child.stderr]) {
if (stream instanceof net.Socket) {
stream.unref();
}
}
return {
stopAsync: async () => {
stopping = true;
// Kernel log lines arrive slightly after the packet; give the last ones a moment.
await sleepAsync(500);
reader.child.kill('SIGTERM');
await finished;
lines?.close();
},
};
}
// ---------------------------------------------------------------------------
// Process coverage
// ---------------------------------------------------------------------------
export type EmulatorProcess = { pid: number; comm: string; cgroup: string };
export async function listEmulatorProcessesAsync(procRoot = '/proc'): Promise<EmulatorProcess[]> {
const entries = await fs.promises.readdir(procRoot);
const processes: EmulatorProcess[] = [];
for (const entry of entries) {
if (!/^\d+$/.test(entry)) {
continue;
}
try {
const comm = (await fs.promises.readFile(path.join(procRoot, entry, 'comm'), 'utf8')).trim();
if (!FENCED_PROCESS_PATTERN.test(comm)) {
continue;
}
const cgroupFile = await fs.promises.readFile(path.join(procRoot, entry, 'cgroup'), 'utf8');
const cgroup = /^0::(\S*)$/m.exec(cgroupFile)?.[1] ?? '';
processes.push({ pid: Number(entry), comm, cgroup });
} catch {
// The process exited while it was being read.
}
}
return processes;
}
export function findProcessesOutsideScope(
processes: readonly EmulatorProcess[],
cgroupPath: string
): EmulatorProcess[] {
const scope = `/${cgroupPath}`;
return processes.filter(({ cgroup }) => cgroup !== scope && !cgroup.startsWith(`${scope}/`));
}
async function assertEmulatorProcessesFencedAsync({
cgroupPath,
when,
}: {
cgroupPath: string;
when: string;
}): Promise<void> {
const outside = findProcessesOutsideScope(await listEmulatorProcessesAsync(), cgroupPath);
if (outside.length === 0) {
return;
}
throw new SystemError(
`Android emulator processes are running outside the local egress fence ${when}: ${outside
.map(({ comm, pid, cgroup }) => `${comm} (pid ${pid}, ${cgroup || 'unknown cgroup'})`)
.join(
', '
)}. Their traffic would not use the tunnel, so the session was stopped. Retry the ` +
'session; if it keeps failing, please contact support.'
);
}
// ---------------------------------------------------------------------------
// adb reverse for --egress-allow localhost:<port>
// ---------------------------------------------------------------------------
/** Loopback ports `pid` listens on, from `ss -Hltnp`. */
export function parseLoopbackListenerPorts(ssOutput: string, pid: number): number[] {
const ports = new Set<number>();
for (const line of ssOutput.split('\n')) {
if (!line.includes(`pid=${pid},`)) {
continue;
}
const match = /\s127\.0\.0\.1:(\d+)\s/.exec(line);
if (match) {
ports.add(Number(match[1]));
}
}
return [...ports].sort((a, b) => a - b);
}
export function parseAdbReversePorts(output: string): Set<number> {
const ports = new Set<number>();
for (const match of output.matchAll(/tcp:(\d+)\s+tcp:\d+/g)) {
ports.add(Number(match[1]));
}
return ports;
}
/**
* The EAS CLI forwards each `--egress-allow localhost:<port>` to the same
* port on this host's loopback, where the guest cannot see it. `adb reverse`
* makes guest `localhost:<port>` reach it. Reverse mappings are lost when an
* emulator or the adb server restarts, so this runs on a timer.
*/
async function reconcileAdbReverseAsync({
env,
logger,
chiselPid,
tunnelPorts,
reported,
serialIds,
}: {
env: BuildStepEnv;
logger: bunyan;
chiselPid: number;
/** chisel's own listeners: the proxy port and the control port ngrok forwards to. */
tunnelPorts: readonly number[];
reported: Set<string>;
serialIds?: AndroidDeviceSerialId[];
}): Promise<void> {
const listeners = await spawn('ss', ['-Hltnp'], { env, stdio: 'pipe' });
const ports = parseLoopbackListenerPorts(listeners.stdout, chiselPid).filter(
port => !tunnelPorts.includes(port)
);
if (ports.length === 0) {
return;
}
const serials =
serialIds ??
(await spawn('adb', ['devices'], { env, stdio: 'pipe' })).stdout
.split('\n')
.map(line => /^(emulator-\d+)\s+device\b/.exec(line)?.[1])
.filter((serial): serial is AndroidDeviceSerialId => !!serial);
for (const serial of serials) {
const list = await asyncResult(
spawn('adb', ['-s', serial, 'reverse', '--list'], { env, stdio: 'pipe' })
);
const existing = list.ok ? parseAdbReversePorts(list.value.stdout) : new Set<number>();
for (const port of ports) {
if (existing.has(port)) {
continue;
}
const result = await asyncResult(
spawn('adb', ['-s', serial, 'reverse', `tcp:${port}`, `tcp:${port}`], {
env,
stdio: 'pipe',
})
);
const key = `${serial}:${port}`;
if (result.ok && !reported.has(key)) {
reported.add(key);
logger.info(
`Local egress: localhost:${port} in ${serial} now reaches port ${port} on the EAS CLI machine.`
);
}
}
}
}
// ---------------------------------------------------------------------------
// Session
// ---------------------------------------------------------------------------
type AndroidLocalEgressSession = {
cgroupPath: string;
env: BuildStepEnv;
chiselPid: number;
tunnelPorts: number[];
relay: LocalEgressRelay;
reportedReverse: Set<string>;
reverseTimer?: NodeJS.Timeout;
logger: bunyan;
};
let activeSession: AndroidLocalEgressSession | undefined;
/**
* Start the relay, the session scope and the fence, and begin reporting fence
* events. Runs in `eas/start_local_egress`, before any emulator starts. The
* fence and scope stay in place until the job ends: removing them while an
* emulator still runs would let its traffic leave from this host.
*/
export async function startAndroidLocalEgressAsync({
env,
logger,
workDir,
chiselPid,
proxyPort,
controlPort,
}: {
env: BuildStepEnv;
logger: bunyan;
workDir: string;
chiselPid: number;
proxyPort: number;
controlPort: number;
}): Promise<{ stopAsync: () => Promise<void> }> {
if (activeSession) {
throw new SystemError('Android local egress is already started for this job.');
}
const relay = await startLocalEgressRelayAsync({ upstreamPort: proxyPort, logger });
try {
const cgroupPath = await startSessionScopeAsync({ env });
const sinceEpochSeconds = Math.floor(Date.now() / 1000) - 1;
await loadFenceAsync({ cgroupPath, workDir, env });
const guardRelay = new GuardEventRelay(logger);
let guardLogger = logger;
const reader = startFenceLogReader({
env,
sinceEpochSeconds,
relay: guardRelay,
getLogger: () => guardLogger,
});
registerLocalEgressGuardRelay(FENCE_RELAY_KEY, {
source: reader,
relay: guardRelay,
onRebind: nextLogger => {
guardLogger = nextLogger;
relay.setLogger(nextLogger);
},
afterSummaryAsync: async () => {
const { tcp, udp, other } = await readFenceCountersAsync({ env });
guardLogger.info(
`Local egress fence: refused ${tcp} TCP, ${udp} UDP and ${other} other packet(s) from the ` +
'Android emulator in total, including retries and any not listed above.'
);
},
});
const session: AndroidLocalEgressSession = {
cgroupPath,
env,
chiselPid,
tunnelPorts: [proxyPort, controlPort],
relay,
reportedReverse: new Set(),
logger,
};
session.reverseTimer = setInterval(() => {
void reconcileAdbReverseAsync({
env,
logger: guardLogger,
chiselPid,
tunnelPorts: session.tunnelPorts,
reported: session.reportedReverse,
}).catch(() => {});
}, ADB_REVERSE_INTERVAL_MS);
session.reverseTimer.unref();
activeSession = session;
logger.info(
`Local egress: Android emulator processes run in ${cgroupPath}, and the network fence ` +
'refuses traffic that cannot use the tunnel and reports it here.'
);
} catch (err) {
await relay.stopAsync();
throw err;
}
return {
stopAsync: async () => {
const session = activeSession;
activeSession = undefined;
if (session?.reverseTimer) {
clearInterval(session.reverseTimer);
}
await relay.stopAsync();
},
};
}
export type AndroidEmulatorLocalEgress = {
launchGate: AndroidEmulatorLaunchGate;
networkReadyTarget: { host: string; port: number };
configureBootedEmulatorAsync: (args: {
serialId: AndroidDeviceSerialId;
logger: bunyan;
}) => Promise<void>;
};
/**
* What `eas/start_android_emulator` changes for a local egress session, or
* null for every other session, which then starts the emulator exactly as
* before.
*/
export async function resolveAndroidEmulatorLocalEgressAsync({
handoffPath = LOCAL_EGRESS_HANDOFF_PATH,
}: { handoffPath?: string } = {}): Promise<AndroidEmulatorLocalEgress | null> {
const handoff = await readLocalEgressHandoffAsync(handoffPath);
if (!handoff) {
return null;
}
const session = activeSession;
if (handoff.platform !== 'android' || !session) {
throw new SystemError(
'This session uses local egress, but the Android emulator fence was not started, so the ' +
"emulator's traffic would not use the tunnel. Please contact support."
);
}
const relayAddress = `${LOCAL_EGRESS_PROXY_HOST}:${session.relay.port}`;
return {
launchGate: {
emulatorArgs: ['-http-proxy', relayAddress, '-no-metrics'],
wrapperCommand: 'bash',
wrapperArgs: ['-c', GATE_SCRIPT, 'eas-egress-gate'],
admitAsync: async pid => {
await assertEmulatorProcessesFencedAsync({
cgroupPath: session.cgroupPath,
when: 'before the emulator started',
});
await admitToScopeAsync({ pid, cgroupPath: session.cgroupPath, env: session.env });
},
},
networkReadyTarget: LOCAL_EGRESS_READINESS_TARGET,
configureBootedEmulatorAsync: async ({ serialId, logger }) => {
await assertEmulatorProcessesFencedAsync({
cgroupPath: session.cgroupPath,
when: 'after boot',
});
const guestProxy = `${ANDROID_EMULATOR_HOST_LOOPBACK_ALIAS}:${session.relay.port}`;
const proxyResult = await asyncResult(
spawn(
'adb',
['-s', serialId, 'shell', 'settings', 'put', 'global', 'http_proxy', guestProxy],
{
env: session.env,
stdio: 'pipe',
}
)
);
if (proxyResult.ok) {
logger.info(
`Local egress: the system proxy in ${serialId} is ${guestProxy}, so apps that honor it ` +
"resolve hostnames on the EAS CLI machine. Other TCP still exits from that machine through the emulator's proxy."
);
} else {
logger.warn(
{ err: proxyResult.reason },
`Local egress: could not set the system proxy in ${serialId}. Traffic still exits from the ` +
"EAS CLI machine through the emulator's proxy, but hostnames resolve on this worker."
);
}
await reconcileAdbReverseAsync({
env: session.env,
logger,
chiselPid: session.chiselPid,
tunnelPorts: session.tunnelPorts,
reported: session.reportedReverse,
serialIds: [serialId],
}).catch(err =>
logger.warn(
{ err },
`Local egress: could not forward allowed localhost ports to ${serialId}.`
)
);
},
};
}
async function admitToScopeAsync({
pid,
cgroupPath,
env,
}: {
pid: number;
cgroupPath: string;
env: BuildStepEnv;
}): Promise<void> {
const procsPath = `/sys/fs/cgroup/${cgroupPath}/cgroup.procs`;
try {
await spawn('sudo', ['sh', '-c', 'echo "$1" > "$2"', 'sh', String(pid), procsPath], {
env,
stdio: 'pipe',
});
const cgroupFile = await fs.promises.readFile(`/proc/${pid}/cgroup`, 'utf8');
if (!cgroupFile.includes(`0::/${cgroupPath}\n`)) {
throw new Error(`/proc/${pid}/cgroup is ${cgroupFile.trim()}`);
}
} catch (err) {
throw new SystemError(
'Could not move the Android emulator into the local egress fence, so its traffic would not use ' +
'the tunnel. Retry the session; if it keeps failing, please contact support.',
{ cause: err }
);
}
}
@@ -1,8 +1,15 @@
import { SystemError } from '@expo/eas-build-job';
import { graphql } from 'gql.tada';
import fetch, { Headers } from 'node-fetch';
import type { Readable } from 'node:stream';
import { setTimeout as delay } from 'node:timers/promises';
import { CustomBuildContext } from '../../customBuildContext';
import { graphqlAbortContext } from '../../utils/graphqlAbort';
import {
DeviceRunSessionTimeoutError,
withDeviceRunSessionTimeoutAsync,
} from './deviceRunSessionTimeout';
const CREATE_DEVICE_RUN_SESSION_ARTIFACT_UPLOAD_SESSION_MUTATION = graphql(`
mutation CreateDeviceRunSessionArtifactUploadSession(
@@ -20,82 +27,178 @@ const CREATE_DEVICE_RUN_SESSION_ARTIFACT_UPLOAD_SESSION_MUTATION = graphql(`
}
`);
type ArtifactMetadata = {
deviceRunSessionId: string;
artifactId: string;
name: string;
filename: string;
kind: string | undefined;
metadata?: Record<string, unknown>;
size: number;
};
type ArtifactSource = { stream: Readable; reopenStream?: () => Readable };
type ArtifactUploadSession = Awaited<
ReturnType<typeof createDeviceRunSessionArtifactUploadSessionAsync>
>;
export async function uploadDeviceRunSessionArtifactAsync(
ctx: CustomBuildContext,
{
deviceRunSessionId,
artifactId,
name,
filename,
kind,
metadata,
size,
stream,
}: {
deviceRunSessionId: string;
artifactId: string;
name: string;
filename: string;
kind: string | undefined;
metadata?: Record<string, unknown>;
size: number;
stream: NodeJS.ReadableStream;
}
options: ArtifactMetadata & ArtifactSource & { signal?: AbortSignal }
): Promise<void> {
const uploadSession = await createDeviceRunSessionArtifactUploadSessionAsync(ctx, {
deviceRunSessionId,
artifactId,
name,
filename,
kind,
metadata,
size,
});
const response = await fetch(uploadSession.url, {
method: 'PUT',
headers: new Headers(uploadSession.headers as Record<string, string>),
body: stream,
});
if (!response.ok) {
throw new SystemError(
`Failed to upload device run session artifact ${artifactId}: ${response.status} ${response.statusText}`,
{ cause: response }
try {
const uploadSession = await withDeviceRunSessionTimeoutAsync(
{ name: 'Artifact upload session creation', timeoutMs: 15_000, signal: options.signal },
async signal => await createDeviceRunSessionArtifactUploadSessionAsync(ctx, options, signal)
);
options.signal?.throwIfAborted();
await putArtifactWithRetriesAsync({
uploadSession,
source: options,
artifactId: options.artifactId,
signal: options.signal,
});
} finally {
// One-shot callers may have opened their stream before upload-session creation failed.
options.stream.destroy();
}
}
async function putArtifactWithRetriesAsync({
uploadSession,
source,
artifactId,
signal,
}: {
uploadSession: ArtifactUploadSession;
source: ArtifactSource;
artifactId: string;
signal?: AbortSignal;
}): Promise<void> {
const attempts = source.reopenStream ? 3 : 1;
for (let attempt = 0; attempt < attempts; attempt++) {
signal?.throwIfAborted();
const stream = attempt > 0 && source.reopenStream ? source.reopenStream() : source.stream;
try {
// Upload time scales with file size, so the deadline bounds stalls, not the whole request.
await withDeviceRunSessionTimeoutAsync(
{ name: 'Artifact PUT', timeoutMs: 90_000, signal },
async (putSignal, resetDeadline) =>
await putArtifactAsync({
uploadSession,
stream,
artifactId,
signal: putSignal,
onProgress: resetDeadline,
})
);
return;
} catch (error) {
signal?.throwIfAborted();
const retryable =
error instanceof ArtifactPutError
? [408, 429, 500, 502, 503, 504].includes(error.status)
: error instanceof DeviceRunSessionTimeoutError ||
error instanceof ArtifactPutNetworkError;
if (!retryable || attempt + 1 === attempts) {
throw error;
}
await delay(500 * (attempt + 1), undefined, { signal });
} finally {
stream.destroy();
}
}
}
async function putArtifactAsync({
uploadSession,
stream,
artifactId,
signal,
onProgress,
}: {
uploadSession: ArtifactUploadSession;
stream: Readable;
artifactId: string;
signal: AbortSignal;
onProgress: () => void;
}): Promise<void> {
const requestController = new AbortController();
try {
const responding = fetch(uploadSession.url, {
method: 'PUT',
headers: new Headers(uploadSession.headers as Record<string, string>),
body: stream,
signal: AbortSignal.any([signal, requestController.signal]),
});
// fetch has piped the body already, so this listener observes flow without starting it.
stream.on('data', onProgress);
let response;
try {
response = await responding;
} catch (error) {
throw error instanceof Error && error.name === 'FetchError'
? new ArtifactPutNetworkError(artifactId, error)
: error;
}
signal.throwIfAborted();
if (!response.ok) {
throw new ArtifactPutError(artifactId, response);
}
} finally {
// node-fetch destroys the request body on abort, and its response stream is a PassThrough
// whose destruction leaves the socket open, so the request itself is aborted here.
requestController.abort();
}
}
class ArtifactPutError extends SystemError {
readonly status: number;
constructor(artifactId: string, response: { status: number; statusText: string }) {
super(
`Failed to upload device run session artifact ${artifactId}: HTTP ${response.status} ${response.statusText}.`
);
this.status = response.status;
}
}
// node-fetch names the request URL in its message, and here that URL is signed. The message keeps
// only the error code, and the original is not kept as the cause because loggers and Sentry print causes.
class ArtifactPutNetworkError extends SystemError {
readonly code: string | undefined;
constructor(artifactId: string, error: Error & { code?: unknown }) {
const code = typeof error.code === 'string' ? error.code : undefined;
super(
`Failed to upload device run session artifact ${artifactId}: ${code ?? 'network error'}.`
);
this.code = code;
}
}
async function createDeviceRunSessionArtifactUploadSessionAsync(
ctx: CustomBuildContext,
{
deviceRunSessionId,
artifactId,
name,
filename,
kind,
metadata,
size,
}: {
deviceRunSessionId: string;
artifactId: string;
name: string;
filename: string;
kind: string | undefined;
metadata?: Record<string, unknown>;
size: number;
}
{ deviceRunSessionId, artifactId, name, filename, kind, metadata, size }: ArtifactMetadata,
signal: AbortSignal
) {
const result = await ctx.graphqlClient
.mutation(CREATE_DEVICE_RUN_SESSION_ARTIFACT_UPLOAD_SESSION_MUTATION, {
deviceRunSessionId,
input: {
name,
filename,
...(kind !== undefined ? { kind } : {}),
...(metadata !== undefined ? { metadata } : {}),
size,
.mutation(
CREATE_DEVICE_RUN_SESSION_ARTIFACT_UPLOAD_SESSION_MUTATION,
{
deviceRunSessionId,
input: {
name,
filename,
...(kind !== undefined ? { kind } : {}),
...(metadata !== undefined ? { metadata } : {}),
size,
},
},
})
graphqlAbortContext(signal)
)
.toPromise();
signal.throwIfAborted();
if (result.error) {
throw new SystemError(
`Failed to create upload session for device run session artifact ${artifactId}: ${result.error.message}`,
@@ -0,0 +1,203 @@
import { SystemError } from '@expo/eas-build-job';
import type { bunyan } from '@expo/logger';
import type { BuildStepEnv } from '@expo/steps';
import spawn from '@expo/turtle-spawn';
import { createReadStream } from 'node:fs';
import { readFile, readdir, stat } from 'node:fs/promises';
import path from 'node:path';
import limitFactory from 'promise-limit';
import { z } from 'zod';
import { type CustomBuildContext } from '../../customBuildContext';
import { Sentry } from '../../sentry';
import { formatBytes } from '../../utils/artifacts';
import { uploadDeviceRunSessionArtifactAsync } from './deviceRunSessionArtifacts';
const RecordingsSchema = z.array(
z.object({
udid: z.string(),
deviceName: z.string(),
runtimeDisplayName: z.string(),
directory: z.string(),
})
);
const RecordingManifestSchema = z.object({
firstFrameWallClock: z.object({
iso8601: z.string(),
}),
width: z.number().int().positive(),
height: z.number().int().positive(),
recording: z.string(),
status: z.string().optional(),
error: z.string().optional(),
});
const UnlistedRecordingManifestSchema = z.object({
udid: z.string(),
deviceName: z.string(),
runtimeDisplayName: z.string(),
status: z.enum(['recording', 'failed', 'complete']),
recording: z.string(),
});
const recordingStartTimeFormatter = new Intl.DateTimeFormat('en-US', {
year: 'numeric',
month: 'short',
day: 'numeric',
hour: '2-digit',
minute: '2-digit',
second: '2-digit',
fractionalSecondDigits: 3,
hourCycle: 'h23',
timeZone: 'UTC',
timeZoneName: 'short',
});
export function parseDeviceScreenRecordings(input: unknown): z.infer<typeof RecordingsSchema> {
const result = RecordingsSchema.safeParse(input);
if (!result.success) {
throw new SystemError('Invalid device screen recordings input.', {
cause: result.error,
});
}
return result.data;
}
/**
* A Hub that was killed never lists its recording, even one it had already completed at the
* duration limit. session.json is at "recording" or "failed" with a fragmented .partial that plays
* up to its last keyframe, or at "complete" with the finished file. Return the ones ffprobe can read.
*/
export async function findUnlistedDeviceScreenRecordingsAsync({
root,
env,
logger,
}: {
root: string;
env: BuildStepEnv;
logger: bunyan;
}): Promise<z.infer<typeof RecordingsSchema>> {
const found: z.infer<typeof RecordingsSchema> = [];
for (const entry of await readdir(root, { withFileTypes: true })) {
if (!entry.isDirectory()) {
continue;
}
const directory = path.join(root, entry.name);
const manifest = UnlistedRecordingManifestSchema.safeParse(
await readFile(path.join(directory, 'session.json'), 'utf-8')
.then(text => JSON.parse(text))
.catch(() => null)
);
if (!manifest.success || path.basename(manifest.data.recording) !== manifest.data.recording) {
continue;
}
const file = path.join(directory, manifest.data.recording);
const name = path.join(entry.name, manifest.data.recording);
let duration = 0;
try {
const probe = await spawn(
'ffprobe',
['-v', 'error', '-show_entries', 'format=duration', '-of', 'csv=p=0', file],
{ env, stdio: 'pipe' }
);
duration = Number(probe.stdout.trim());
} catch (err) {
if ((err as NodeJS.ErrnoException).code === 'ENOENT') {
logger.warn(`ffprobe is not available; cannot recover the unlisted recording ${name}.`);
continue;
}
}
if (!(duration > 0)) {
logger.warn(`Unlisted recording ${name} does not decode; skipping it.`);
continue;
}
const { udid, deviceName, runtimeDisplayName } = manifest.data;
found.push({ udid, deviceName, runtimeDisplayName, directory });
}
return found;
}
export async function uploadDeviceRunSessionScreenRecordingsAsync(
ctx: CustomBuildContext,
{
logger,
deviceRunSessionId,
recordings,
}: {
logger: bunyan;
deviceRunSessionId: string;
recordings: z.infer<typeof RecordingsSchema>;
}
): Promise<boolean> {
if (recordings.length === 0) {
logger.info('No device screen recordings found; skipping uploads.');
return true;
}
const limit = limitFactory(5);
const uploaded = await Promise.all(
recordings.map(recording =>
limit(async () => {
try {
const metadata = RecordingManifestSchema.parse(
JSON.parse(await readFile(path.join(recording.directory, 'session.json'), 'utf-8'))
);
if (path.basename(metadata.recording) !== metadata.recording) {
throw new Error('Recording filename must not contain a directory.');
}
const startedAt = recordingStartTimeFormatter.format(
new Date(metadata.firstFrameWallClock.iso8601)
);
const partial = metadata.status !== undefined && metadata.status !== 'complete';
const shortUdid = `${recording.udid.slice(0, 8)}-…`;
const displayName = `${recording.deviceName} screen recording (${shortUdid}, started at ${startedAt}${partial ? ', partial' : ''})`;
const recordingPath = path.join(recording.directory, metadata.recording);
const { size } = await stat(recordingPath);
const recordingId = path.basename(recording.directory);
logger.info(
`Uploading screen recording for ${recording.deviceName} (${formatBytes(size)}).`
);
await uploadDeviceRunSessionArtifactAsync(ctx, {
deviceRunSessionId,
artifactId: recordingId,
name: displayName,
filename: `${recordingId}.mp4`,
kind: 'screen-recording',
metadata: {
__eas_type: 'screen-recording',
__eas_screen_recording: '1',
udid: recording.udid,
deviceName: recording.deviceName,
runtimeDisplayName: recording.runtimeDisplayName,
firstFrameAt: metadata.firstFrameWallClock.iso8601,
width: metadata.width,
height: metadata.height,
...(partial
? {
partial: true,
partialReason:
metadata.error ??
'The Device Hub stopped before it could finalize the recording.',
}
: {}),
},
size,
stream: createReadStream(recordingPath),
reopenStream: () => createReadStream(recordingPath),
});
return true;
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not upload device screen recording', error);
logger.warn(
{ err: error },
`Could not upload screen recording for ${recording.deviceName}.`
);
return false;
}
})
)
);
return uploaded.every(Boolean);
}
@@ -0,0 +1,120 @@
import type { bunyan } from '@expo/logger';
import { graphql } from 'gql.tada';
import { z } from 'zod';
import type { CustomBuildContext } from '../../customBuildContext';
import { graphqlAbortContext } from '../../utils/graphqlAbort';
const SHORT_COMMIT_HASH_LENGTH = 7;
const MAX_APP_SLUG_LENGTH = 80;
const SESSION_LOOKUP_TIMEOUT_MS = 5_000;
// Runs of characters that are not letters, digits, dots, underscores, or hyphens; each run becomes one hyphen.
const FILENAME_UNSAFE_RUN = /[^a-zA-Z0-9._-]+/g;
// Dots and hyphens at either end of the slug, dropped so the name cannot start with a hidden-file dot.
const LEADING_OR_TRAILING_DOTS_AND_HYPHENS = /^[.-]+|[.-]+$/g;
// A full git commit hash: 40 hex characters for SHA-1 or 64 for SHA-256.
const FULL_GIT_COMMIT_HASH = /^(?:[a-f0-9]{40}|[a-f0-9]{64})$/i;
const SCREENSHOT_SESSION_QUERY = graphql(`
query ScreenshotSession($deviceRunSessionId: ID!) {
deviceRunSessions {
byId(deviceRunSessionId: $deviceRunSessionId) {
platform
app { slug }
build {
id
gitCommitHash
app { slug }
}
}
}
}
`);
const AppSchema = z.object({ slug: z.string() });
const ScreenshotSessionSchema = z.object({
platform: z.enum(['IOS', 'ANDROID']),
app: AppSchema,
build: z
.object({
id: z.string(),
gitCommitHash: z.string().nullable(),
app: AppSchema,
})
.nullable(),
});
export type ScreenshotSession = z.infer<typeof ScreenshotSessionSchema>;
export async function loadScreenshotSessionAsync(
ctx: CustomBuildContext,
deviceRunSessionId: string,
logger: bunyan
): Promise<ScreenshotSession | null> {
try {
const result = await ctx.graphqlClient
.query(
SCREENSHOT_SESSION_QUERY,
{ deviceRunSessionId },
graphqlAbortContext(AbortSignal.timeout(SESSION_LOOKUP_TIMEOUT_MS))
)
.toPromise();
if (result.error) {
throw result.error;
}
return ScreenshotSessionSchema.parse(result.data?.deviceRunSessions.byId);
} catch (err) {
logger.warn(
{ err, deviceRunSessionId },
'Could not load screenshot session details; using capture timestamps.'
);
return null;
}
}
export function screenshotArtifactDetails(
timestamp: string,
session: ScreenshotSession | null
): { name: string; filename: string; metadata: Record<string, string> } {
const name = screenshotArtifactName(timestamp);
if (!session) {
return {
name,
filename: `screenshot-${timestamp}.png`,
metadata: { __eas_type: 'screenshot' },
};
}
const app = session.build?.app ?? session.app;
const appSlug =
app.slug
.replace(FILENAME_UNSAFE_RUN, '-')
.replace(LEADING_OR_TRAILING_DOTS_AND_HYPHENS, '')
.slice(0, MAX_APP_SLUG_LENGTH) || 'app';
const platform = session.platform.toLowerCase();
const filenameParts = [appSlug, platform];
const metadata: Record<string, string> = {
__eas_type: 'screenshot',
appSlug: app.slug,
platform,
};
if (session.build) {
metadata.buildId = session.build.id;
const commitHash = session.build.gitCommitHash;
const hasValidCommitHash = commitHash !== null && FULL_GIT_COMMIT_HASH.test(commitHash);
if (hasValidCommitHash) {
filenameParts.push(commitHash.slice(0, SHORT_COMMIT_HASH_LENGTH).toLowerCase());
metadata.gitCommitHash = commitHash;
}
}
filenameParts.push(timestamp);
return { name, filename: `${filenameParts.join('-')}.png`, metadata };
}
function screenshotArtifactName(timestamp: string): string {
// The caller already matched the timestamp against the filename pattern, so this split cannot fail.
const [date, time] = timestamp.split('T');
const [hours, minutes, seconds] = time.split('-');
return `Screenshot ${date} ${hours}:${minutes}:${seconds} UTC`;
}
@@ -0,0 +1,426 @@
import type { bunyan } from '@expo/logger';
import { type FSWatcher, createReadStream, watch } from 'node:fs';
import { mkdtemp, readFile, readdir, rm, rmdir, stat } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { z } from 'zod';
import type { CustomBuildContext } from '../../customBuildContext';
import { Sentry } from '../../sentry';
import { formatBytes } from '../../utils/artifacts';
import { uploadDeviceRunSessionArtifactAsync } from './deviceRunSessionArtifacts';
import {
type ScreenshotSession,
loadScreenshotSessionAsync,
screenshotArtifactDetails,
} from './deviceRunSessionScreenshotNames';
// The device-hub repository writes these files from
// packages/serve-sim/packages/serve-sim/src/screenshot-artifacts.ts and
// packages/serve-emu/packages/serve-emu/src/screenshot-artifacts.ts.
// When a capture cannot be saved, the same files write a failure record in its place.
// These patterns, the record schema and the producer code must change together.
const SCREENSHOT_FILENAME_PATTERN =
/^screenshot-(?<timestamp>\d{4}-\d{2}-\d{2}T\d{2}-\d{2}-\d{2}-\d{3}Z)-[a-f0-9]{12}\.png$/;
const SCREENSHOT_FAILURE_RECORD_PATTERN =
/^screenshot-\d{4}-\d{2}-\d{2}T\d{2}-\d{2}-\d{2}-\d{3}Z-[a-f0-9]{12}\.failed\.json$/;
const ScreenshotFailureRecordSchema = z.object({
file: z.string(),
error: z.string(),
at: z.string(),
});
async function reportScreenshotFailureRecordAsync(file: string, logger: bunyan): Promise<void> {
try {
const record = ScreenshotFailureRecordSchema.parse(JSON.parse(await readFile(file, 'utf8')));
logger.warn(
{ file: record.file, error: record.error, at: record.at },
'The session host could not save a preview screenshot.'
);
Sentry.capture(
'The session host could not save a preview screenshot',
new Error(record.error),
{
extras: { file: record.file, at: record.at },
}
);
} catch (err) {
logger.warn({ err, file }, 'Could not read a preview screenshot failure record.');
}
await rm(file, { force: true });
}
type FailedUploads = Map<string, { attempts: number; lastError: Error }>;
type ScreenshotEntry = { kind: 'failureRecord' } | { kind: 'screenshot'; timestamp: string };
function classifyScreenshotEntry(name: string): ScreenshotEntry | null {
if (SCREENSHOT_FAILURE_RECORD_PATTERN.test(name)) {
return { kind: 'failureRecord' };
}
const timestamp = SCREENSHOT_FILENAME_PATTERN.exec(name)?.groups?.timestamp;
return timestamp ? { kind: 'screenshot', timestamp } : null;
}
async function uploadScreenshotAsync(
ctx: CustomBuildContext,
{
file,
name,
timestamp,
size,
deviceRunSessionId,
session,
signal,
failedUploads,
logger,
}: {
file: string;
name: string;
timestamp: string;
size: number;
deviceRunSessionId: string;
session: Promise<ScreenshotSession | null>;
signal: AbortSignal;
failedUploads: FailedUploads;
logger: bunyan;
}
): Promise<void> {
const artifactDetails = screenshotArtifactDetails(timestamp, await session);
const startedAt = Date.now();
await uploadDeviceRunSessionArtifactAsync(ctx, {
deviceRunSessionId,
artifactId: name.slice(0, -4),
name: artifactDetails.name,
filename: artifactDetails.filename,
metadata: artifactDetails.metadata,
kind: 'screenshot',
size,
stream: createReadStream(file),
reopenStream: () => createReadStream(file),
signal,
});
await rm(file);
const previousFailures = failedUploads.get(name)?.attempts ?? 0;
failedUploads.delete(name);
// Each earlier failed attempt may have left an artifact row without a file on the server.
// The API deletes such rows when the job run finishes (expo/universe#31539).
const retryNote =
previousFailures > 0
? ` after ${previousFailures} failed attempt${previousFailures === 1 ? '' : 's'}`
: '';
logger.info(
{ file },
`Uploaded preview screenshot ${artifactDetails.filename} (${formatBytes(size)}) in ${Date.now() - startedAt} ms${retryNote}.`
);
}
function recordUploadFailure(
err: unknown,
{
file,
name,
size,
signal,
failedUploads,
logger,
}: {
file: string;
name: string;
size: number | undefined;
signal: AbortSignal;
failedUploads: FailedUploads;
logger: bunyan;
}
): 'aborted' | 'recorded' {
if (signal.aborted) {
logger.warn(
{ file, size },
'Shutdown deadline reached before this preview screenshot uploaded. Keeping the file.'
);
return 'aborted';
}
const error = err instanceof Error ? err : new Error(String(err));
const attempt = (failedUploads.get(name)?.attempts ?? 0) + 1;
failedUploads.set(name, { attempts: attempt, lastError: error });
// A sustained outage would otherwise warn for every waiting file on each 30 s scan.
if (attempt <= 3 || attempt % 10 === 0) {
logger.warn(
{ err: error, file, attempt, size },
`Could not upload preview screenshot (attempt ${attempt}). Keeping it for retry.`
);
}
return 'recorded';
}
/**
* Only atomically completed PNGs are eligible; failed uploads stay on disk for retry.
* Failure records are reported once and deleted.
*/
export async function uploadDeviceRunSessionScreenshotsAsync(
ctx: CustomBuildContext,
{
directory,
deviceRunSessionId,
logger,
signal,
failedUploads,
session = Promise.resolve(null),
}: {
directory: string;
deviceRunSessionId: string;
logger: bunyan;
signal: AbortSignal;
failedUploads: FailedUploads;
session?: Promise<ScreenshotSession | null>;
}
): Promise<{ uploaded: number; saveFailures: number }> {
let uploadedCount = 0;
let saveFailures = 0;
// Names start with the capture time. Captures that never failed go first, oldest first, so a
// capture that keeps stalling cannot hold back newer ones or use up the shutdown budget.
const entries = (await readdir(directory, { withFileTypes: true })).sort(
(a, b) =>
Number(failedUploads.has(a.name)) - Number(failedUploads.has(b.name)) ||
(a.name < b.name ? -1 : a.name > b.name ? 1 : 0)
);
for (const entry of entries) {
if (signal.aborted) {
break;
}
const classified = entry.isFile() ? classifyScreenshotEntry(entry.name) : null;
if (!classified) {
continue;
}
const file = path.join(directory, entry.name);
if (classified.kind === 'failureRecord') {
await reportScreenshotFailureRecordAsync(file, logger);
saveFailures++;
continue;
}
let size: number | undefined;
try {
size = (await stat(file)).size;
await uploadScreenshotAsync(ctx, {
file,
name: entry.name,
timestamp: classified.timestamp,
size,
deviceRunSessionId,
session,
signal,
failedUploads,
logger,
});
uploadedCount++;
} catch (err) {
if (size === undefined && (err as NodeJS.ErrnoException).code === 'ENOENT') {
continue;
}
const outcome = recordUploadFailure(err, {
file,
name: entry.name,
size,
signal,
failedUploads,
logger,
});
if (outcome === 'aborted') {
break;
}
}
}
return { uploaded: uploadedCount, saveFailures };
}
function watchScreenshotDirectory(
directory: string,
onChange: () => void,
logger: bunyan
): () => void {
let watcher: FSWatcher | undefined;
try {
watcher = watch(directory, { persistent: false }, (_event, filename) => {
if (
filename === null ||
SCREENSHOT_FILENAME_PATTERN.test(filename) ||
SCREENSHOT_FAILURE_RECORD_PATTERN.test(filename)
) {
onChange();
}
});
watcher.on('error', err => {
logger.warn({ err, directory }, 'Screenshot watcher failed; using periodic scans.');
watcher?.close();
});
} catch (err) {
logger.warn({ err, directory }, 'Could not watch screenshots; using periodic scans.');
}
return () => watcher?.close();
}
// rmdir refuses a non-empty directory, so a file still there at the end is reported as not uploaded,
// never deleted.
async function removeEmptyDirectoryAsync(directory: string): Promise<boolean> {
try {
await rmdir(directory);
return true;
} catch (err) {
if ((err as NodeJS.ErrnoException).code === 'ENOTEMPTY') {
return false;
}
throw err;
}
}
async function reportNotUploadedScreenshotsAsync({
directory,
hostStopped,
failedUploads,
logger,
}: {
directory: string;
hostStopped: boolean;
failedUploads: FailedUploads;
logger: bunyan;
}): Promise<void> {
if (hostStopped && (await removeEmptyDirectoryAsync(directory))) {
return;
}
const files = await readdir(directory);
if (files.length === 0) {
return;
}
const notUploaded = files.map(name => {
const failure = failedUploads.get(name);
return failure
? { name, attempts: failure.attempts, lastError: failure.lastError.message }
: { name };
});
const message = `${files.length} preview screenshot${files.length === 1 ? ' was' : 's were'} not uploaded.`;
logger.warn({ directory, files: notUploaded }, message);
Sentry.capture('Preview screenshots were not uploaded', new Error(message), {
extras: { files: notUploaded },
});
}
async function finishCollectionAsync({
directory,
hostStopped,
logger,
stopWatching,
timer,
controller,
drainAsync,
totals,
failedUploads,
}: {
directory: string;
hostStopped: boolean;
logger: bunyan;
stopWatching: () => void;
timer: NodeJS.Timeout;
controller: AbortController;
drainAsync: () => Promise<void>;
totals: { uploaded: number; saveFailures: number };
failedUploads: FailedUploads;
}): Promise<void> {
stopWatching();
clearInterval(timer);
// This shutdown budget wins over the uploader's own 90 s stall deadline, so a slow upload
// at shutdown is reported as not uploaded rather than delaying teardown.
const deadline = setTimeout(() => controller.abort(), 30_000);
try {
await drainAsync();
if (controller.signal.aborted) {
logger.warn({ directory }, 'Preview screenshot flush timed out after 30 s.');
}
const saveFailureNote =
totals.saveFailures > 0 ? `; the session host could not save ${totals.saveFailures}` : '';
logger.info(
`Uploaded ${totals.uploaded} preview screenshots during the session${saveFailureNote}.`
);
if (!hostStopped) {
logger.warn(
{ directory },
'The session host is still running, so preview screenshots it saves from now on are not uploaded.'
);
}
await reportNotUploadedScreenshotsAsync({ directory, hostStopped, failedUploads, logger });
} catch (err) {
logger.warn({ err, directory }, 'Could not finish preview screenshot uploads.');
} finally {
clearTimeout(deadline);
}
}
export async function startDeviceRunSessionScreenshotsAsync(
ctx: CustomBuildContext,
options: { deviceRunSessionId: string; logger: bunyan }
): Promise<{ directory: string; finishAsync: (hostStopped: boolean) => Promise<void> }> {
const session = loadScreenshotSessionAsync(ctx, options.deviceRunSessionId, options.logger);
const directory = await mkdtemp(path.join(os.tmpdir(), 'device-session-screenshots-'));
const failedUploads: FailedUploads = new Map();
const totals = { uploaded: 0, saveFailures: 0 };
const controller = new AbortController();
let pending: Promise<void> | null = null;
let scanRequested = false;
const collectRequestedScreenshotsAsync = async (): Promise<void> => {
while (scanRequested && !controller.signal.aborted) {
scanRequested = false;
try {
const { uploaded, saveFailures } = await uploadDeviceRunSessionScreenshotsAsync(ctx, {
...options,
session,
directory,
failedUploads,
signal: controller.signal,
});
totals.uploaded += uploaded;
totals.saveFailures += saveFailures;
} catch (err) {
options.logger.warn({ err, directory }, 'Could not collect preview screenshots.');
}
}
};
const flush = (): Promise<void> => {
scanRequested = true;
if (pending) {
return pending;
}
pending = collectRequestedScreenshotsAsync().finally(() => {
pending = null;
});
return pending;
};
const stopWatching = watchScreenshotDirectory(directory, () => void flush(), options.logger);
const timer = setInterval(() => {
void flush();
}, 30_000);
timer.unref();
let finishTask: Promise<void> | null = null;
return {
directory,
finishAsync(hostStopped) {
finishTask ??= finishCollectionAsync({
directory,
hostStopped,
logger: options.logger,
stopWatching,
timer,
controller,
drainAsync: async () => {
await pending;
await flush();
},
totals,
failedUploads,
});
return finishTask;
},
};
}
@@ -0,0 +1,43 @@
export class DeviceRunSessionTimeoutError extends Error {
constructor(name: string, timeoutMs: number) {
super(`${name} timed out after ${timeoutMs}ms.`);
this.name = 'DeviceRunSessionTimeoutError';
}
}
/**
* Bounds waiting and signals cancellation. Operations must check the signal before later effects.
* Calling resetDeadline restarts the timer, so the bound applies to stalls instead of total time.
*/
export async function withDeviceRunSessionTimeoutAsync<T>(
{ name, timeoutMs, signal: parent }: { name: string; timeoutMs: number; signal?: AbortSignal },
operation: (signal: AbortSignal, resetDeadline: () => void) => Promise<T>
): Promise<T> {
const controller = new AbortController();
const signal = parent ? AbortSignal.any([parent, controller.signal]) : controller.signal;
signal.throwIfAborted();
let rejectAbort: (reason: unknown) => void = () => {};
const aborted = new Promise<never>((_resolve, reject) => {
rejectAbort = reject;
});
const onAbort = () => rejectAbort(signal.reason);
signal.addEventListener('abort', onAbort, { once: true });
const expire = () => controller.abort(new DeviceRunSessionTimeoutError(name, timeoutMs));
let timer = setTimeout(expire, timeoutMs);
let settled = false;
const resetDeadline = () => {
clearTimeout(timer);
if (!settled && !signal.aborted) {
timer = setTimeout(expire, timeoutMs);
}
};
try {
const result = await Promise.race([operation(signal, resetDeadline), aborted]);
signal.throwIfAborted();
return result;
} finally {
settled = true;
clearTimeout(timer);
signal.removeEventListener('abort', onAbort);
}
}
@@ -0,0 +1,633 @@
import { SystemError, UserError } from '@expo/eas-build-job';
import type { bunyan } from '@expo/logger';
import { BuildRuntimePlatform, type BuildStepEnv } from '@expo/steps';
import { randomBytes } from 'node:crypto';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { z } from 'zod';
import type { CustomBuildContext } from '../../customBuildContext';
import { Sentry } from '../../sentry';
import {
PackageManager,
resolveConfiguredPackageManager,
resolvePackageExec,
} from '../../utils/packageManager';
import { sleepAsync } from '../../utils/retry';
import { turtleFetch } from '../../utils/turtleFetch';
import {
findUnlistedDeviceScreenRecordingsAsync,
parseDeviceScreenRecordings,
uploadDeviceRunSessionScreenRecordingsAsync,
} from './deviceRunSessionScreenRecordings';
import { startDeviceRunSessionScreenshotsAsync } from './deviceRunSessionScreenshots';
import {
type DetachedProcessHandle,
type ServeSimLaunchOptions,
ensureFfmpegInstalledOnceAsync,
fetchWebPreviewTurnArgsAsync,
findAvailablePortAsync,
getDeviceRunSessionIdOrThrow,
getNgrokAuthtokenOrThrow,
isProcessRunning,
spawnDetached,
startNgrokTunnelAsync,
} from './remoteDeviceRunSession';
import { withDeviceRunSessionTimeoutAsync } from './deviceRunSessionTimeout';
import { SERVE_SIM_STATE_DIR, readServeSimServersAsync } from './serveSimMetricsRecorder';
const WEB_PREVIEW_HOST = '127.0.0.1';
const SERVE_SIM_PACKAGE_NAME = '@expo/serve-sim';
const SERVE_SIM_MAX_DIMENSION = '1600';
const SERVE_SIM_MJPEG_QUALITY = '0.55';
const SERVE_SIM_VIDEO_BITRATE = '10000000';
const SERVE_SIM_VIDEO_FPS = '60';
const EXPO_DEVICE_HUB_PACKAGE_NAME = 'expo-device-hub';
const EXPO_DEVICE_HUB_MAX_DIMENSION = '960';
const EXPO_DEVICE_HUB_VIDEO_BITRATE = '6000000';
const EXPO_DEVICE_HUB_VIDEO_FPS = '60';
// On SIGTERM the Hub finalizes the recording itself, with this deadline before it force-exits,
// so a stop request that failed here still gets one more chance to write the MP4.
const EXPO_DEVICE_HUB_SIGTERM_FINALIZE_DEADLINE_MS = 60_000;
const EXPO_DEVICE_HUB_EXIT_LEEWAY_MS = 10_000;
const RECORDING_STOP_GRACE_PERIOD_MS =
EXPO_DEVICE_HUB_SIGTERM_FINALIZE_DEADLINE_MS + EXPO_DEVICE_HUB_EXIT_LEEWAY_MS;
const HOST_OUTPUT_TAIL_CHARS = 8_000;
const WEB_PREVIEW_READY_POLL_INTERVAL_MS = 250;
export function websiteOrigin(env: BuildStepEnv): string {
return env.EXPO_LOCAL
? 'https://expo.test'
: env.EXPO_STAGING
? 'https://staging.expo.dev'
: 'https://expo.dev';
}
export function simulatorPreviewPageUrl(env: BuildStepEnv, subdomainId: string): string {
return new URL(`/simulator-preview/${subdomainId}`, websiteOrigin(env)).toString();
}
// Website dev servers on expo.test use staging sessions, and CORS matches ports exactly.
const WEBSITE_DEV_ORIGINS = [
'https://expo.test',
'https://expo.test:13001',
...Array.from({ length: 16 }, (_, index) => `https://expo.test:${13200 + index}`),
];
export function websiteOriginServeSimArgs(env: BuildStepEnv): string[] {
const origins = new Set([websiteOrigin(env)]);
if (!env.EXPO_LOCAL && env.EXPO_STAGING) {
origins.add('https://*.expo.dev');
}
if (env.EXPO_LOCAL || env.EXPO_STAGING) {
for (const origin of WEBSITE_DEV_ORIGINS) {
origins.add(origin);
}
}
return [...origins].flatMap(origin => ['--cors-origin', origin, '--frame-ancestor', origin]);
}
function createServeSimPackageSpec(packageVersion: string | undefined): string {
return `${SERVE_SIM_PACKAGE_NAME}@${packageVersion ?? 'latest'}`;
}
function createExpoDeviceHubPackageSpec(packageVersion: string | undefined): string {
return `${EXPO_DEVICE_HUB_PACKAGE_NAME}@${packageVersion ?? 'latest'}`;
}
export function createServeSimArgs({
port,
turnArgs = [],
websiteArgs = [],
shareUrl,
packageVersion,
launchAppIdentifier,
launchArgs = [],
openUrl,
networkCapture = false,
networkCaptureFields = [],
}: {
port: number;
turnArgs?: string[];
websiteArgs?: string[];
shareUrl?: string;
packageVersion?: string;
networkCapture?: boolean;
networkCaptureFields?: string[];
} & ServeSimLaunchOptions): string[] {
return [
createServeSimPackageSpec(packageVersion),
'--port',
String(port),
'--host',
WEB_PREVIEW_HOST,
'--require-token',
'--transport',
'webrtc',
'--webrtc-codec',
'h264',
'--max-dimension',
SERVE_SIM_MAX_DIMENSION,
'--mjpeg-quality',
SERVE_SIM_MJPEG_QUALITY,
'--video-bitrate',
SERVE_SIM_VIDEO_BITRATE,
'--video-fps',
SERVE_SIM_VIDEO_FPS,
...turnArgs,
...websiteArgs,
...(shareUrl ? ['--share-url', shareUrl] : []),
...(launchAppIdentifier ? ['--launch-app-identifier', launchAppIdentifier] : []),
...launchArgs.flatMap(argument => ['--launch-arg', argument]),
...(openUrl ? ['--open-url', openUrl] : []),
// `--network-capture` also covers an already booted simulator. Fields are repeated, not
// comma-joined, so serve-sim's error names the bad value.
...(networkCapture
? [
'--network-capture',
...networkCaptureFields.flatMap(field => ['--network-capture-field', field]),
]
: []),
];
}
export function createExpoDeviceHubArgs({
port,
turnArgs = [],
packageVersion,
recordingDirectory,
}: {
port: number;
turnArgs?: string[];
packageVersion?: string;
recordingDirectory?: string;
}): string[] {
return [
createExpoDeviceHubPackageSpec(packageVersion),
'--port',
String(port),
'--host',
WEB_PREVIEW_HOST,
'--platform',
'android',
'--transport',
'webrtc',
'--webrtc-codec',
'h264',
'--webrtc-ice-policy',
'all',
'--max-dimension',
EXPO_DEVICE_HUB_MAX_DIMENSION,
'--video-bitrate',
EXPO_DEVICE_HUB_VIDEO_BITRATE,
'--video-fps',
EXPO_DEVICE_HUB_VIDEO_FPS,
'--hide-sidebar',
'--hide-boot-device',
...(recordingDirectory ? ['--android-recording-directory', recordingDirectory] : []),
...turnArgs,
];
}
const WebPreviewReadyResponseSchema = z.object({
status: z.literal('ready'),
device: z.string(),
});
export async function waitForWebPreviewReadyAsync({
previewServer,
serverName,
port,
timeoutMs,
}: {
previewServer: Pick<DetachedProcessHandle, 'pid' | 'getOutput'>;
serverName: string;
port: number;
timeoutMs: number;
}): Promise<string> {
const readyUrl = `http://${WEB_PREVIEW_HOST}:${port}/readyz`;
const deadline = Date.now() + timeoutMs;
let lastError: unknown;
while (Date.now() < deadline) {
if (previewServer.pid !== undefined && !isProcessRunning(previewServer.pid)) {
throw new SystemError(
`${serverName} exited before becoming ready. Last output:\n${
previewServer.getOutput() || '<empty>'
}`
);
}
try {
const response = await turtleFetch(readyUrl, 'GET', {
retries: 0,
timeout: 2_000,
});
const ready = WebPreviewReadyResponseSchema.parse(await response.json());
return ready.device;
} catch (error) {
lastError = error;
}
await sleepAsync(WEB_PREVIEW_READY_POLL_INTERVAL_MS);
}
throw new SystemError(
`Timed out waiting for ${serverName} readiness at ${readyUrl}${
lastError instanceof Error ? `: ${lastError.message}` : ''
}. Last output:\n${previewServer.getOutput() || '<empty>'}`
);
}
export async function readServeSimPreviewTokenAsync(
udid: string,
stateDir: string = SERVE_SIM_STATE_DIR
): Promise<string | undefined> {
const servers = await readServeSimServersAsync(stateDir);
return servers.find(server => server.udid === udid)?.token;
}
export type DeviceWebPreview = {
previewPageUrl: string;
apiUrl: string;
previewToken?: string;
/** Closes this tunnel only. The session host and recording remain running. */
closeAsync(): Promise<void>;
};
export type DeviceSessionHost = {
openPreviewAsync(options: { baseDomain: string }): Promise<DeviceWebPreview>;
/**
* Terminal and idempotent. Never rejects: a failed finalization, host stop or upload is logged
* and reported, so callers need no error handling around it.
*/
finishAsync(): Promise<void>;
};
type AndroidSessionRecording = {
deviceRunSessionId: string;
directory: string;
controlToken: string;
env: BuildStepEnv;
};
export async function startDeviceSessionHostAsync(
ctx: CustomBuildContext,
{
runtimePlatform,
env,
logger,
timeoutMs,
packageVersion,
launchAppIdentifier,
launchArgs,
openUrl,
networkCapture = false,
networkCaptureFields = [],
}: {
runtimePlatform: BuildRuntimePlatform;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
packageVersion?: string;
networkCapture?: boolean;
networkCaptureFields?: string[];
} & ServeSimLaunchOptions
): Promise<DeviceSessionHost> {
const isAndroid = runtimePlatform === BuildRuntimePlatform.LINUX;
// Unreachable from the step functions, which reject a non-Darwin launch while parsing.
// Kept because this function is exported and expo-device-hub cannot launch.
if (isAndroid && networkCapture) {
throw new UserError(
'EAS_NETWORK_CAPTURE_INVALID_INPUT',
`Cannot record network traffic: capture runs through serve-sim on an iOS simulator, and this session runs expo-device-hub on ${runtimePlatform}.`
);
}
if (isAndroid && launchAppIdentifier) {
throw new UserError(
'EAS_LAUNCH_APPLICATION_INVALID_INPUT',
`Cannot launch ${launchAppIdentifier}: an application launch runs through serve-sim on an iOS simulator, and this session runs expo-device-hub on ${runtimePlatform}.`
);
}
if (isAndroid) {
await ensureFfmpegInstalledOnceAsync({ runtimePlatform, env, logger });
}
const recording: AndroidSessionRecording | null = isAndroid
? {
deviceRunSessionId: getDeviceRunSessionIdOrThrow(env),
directory: await fs.promises.mkdtemp(path.join(os.tmpdir(), 'android-session-recordings-')),
controlToken: randomBytes(32).toString('hex'),
env,
}
: null;
const subdomainId = randomBytes(16).toString('hex');
const previewPageUrl = simulatorPreviewPageUrl(env, subdomainId);
const port = await findAvailablePortAsync();
const serverName = isAndroid ? 'expo-device-hub' : 'serve-sim';
const packageSpec = isAndroid
? createExpoDeviceHubPackageSpec(packageVersion)
: createServeSimPackageSpec(packageVersion);
const turnArgs = await fetchWebPreviewTurnArgsAsync(ctx, { env, logger });
const previewExec = resolvePackageExec(
resolveConfiguredPackageManager(env, PackageManager.NPM),
isAndroid
? createExpoDeviceHubArgs({
port,
turnArgs,
packageVersion,
recordingDirectory: recording?.directory,
})
: createServeSimArgs({
port,
turnArgs,
packageVersion,
websiteArgs: websiteOriginServeSimArgs(env),
shareUrl: previewPageUrl,
launchAppIdentifier,
launchArgs,
openUrl,
networkCapture,
networkCaptureFields,
})
);
logger.info(
`Launching ${packageSpec} on ${WEB_PREVIEW_HOST}:${port} via ${previewExec.command}.`
);
const screenshots = await startDeviceRunSessionScreenshotsAsync(ctx, {
deviceRunSessionId: getDeviceRunSessionIdOrThrow(env),
logger,
});
let previewServer: DetachedProcessHandle;
try {
previewServer = spawnDetached({
command: previewExec.command,
args: previewExec.args,
env: {
...env,
EXPO_DEVICE_HUB_SCREENSHOT_DIRECTORY: screenshots.directory,
...(recording ? { EXPO_DEVICE_HUB_RECORDING_CONTROL_TOKEN: recording.controlToken } : {}),
},
stopGracePeriodMs: recording ? RECORDING_STOP_GRACE_PERIOD_MS : undefined,
});
} catch (error) {
// Nothing was spawned, so nothing can still write into the directory.
await screenshots.finishAsync(true);
throw error;
}
let previewToken: string | undefined;
let previewTask: Promise<DeviceWebPreview> | null = null;
let finishTask: Promise<void> | null = null;
let hostReady = false;
const host: DeviceSessionHost = {
openPreviewAsync({ baseDomain }) {
if (finishTask) {
return Promise.reject(
new SystemError('Cannot open a preview after session host finalization.')
);
}
if (previewTask) {
return previewTask;
}
const opening = (async (): Promise<DeviceWebPreview> => {
const tunnel = await startNgrokTunnelAsync({
port,
subdomainPrefix: 'web-preview',
subdomainId,
baseDomain,
authtoken: getNgrokAuthtokenOrThrow(env),
logger,
});
if (finishTask) {
await withDeviceRunSessionTimeoutAsync(
{ name: 'Late preview tunnel close', timeoutMs: 5_000 },
async () => await tunnel.stopAsync()
).catch(err => logger.warn({ err }, 'Could not close a late preview tunnel.'));
throw new SystemError('Session host finalized while the preview was opening.');
}
let closeTask: Promise<void> | null = null;
return {
previewPageUrl,
apiUrl: tunnel.url,
previewToken,
closeAsync() {
return (closeTask ??= (async () => {
try {
await tunnel.stopAsync();
} finally {
if (previewTask === opening) {
previewTask = null;
}
}
})());
},
};
})();
previewTask = opening;
// A failed tunnel may be retried without restarting capture.
void opening.catch(() => {
if (previewTask === opening) {
previewTask = null;
}
});
return opening;
},
finishAsync() {
return (finishTask ??= finishDeviceSessionHostAsync(ctx, {
previewTask,
previewServer,
screenshots,
serverName,
port,
// A host that never answered /readyz has nothing to finalize or upload.
recording: hostReady ? recording : null,
logger,
}));
},
};
try {
logger.info(`Waiting for ${serverName} to become ready.`);
const device = await waitForWebPreviewReadyAsync({
previewServer,
serverName,
port,
timeoutMs,
});
hostReady = true;
if (!isAndroid) {
previewToken = await readServeSimPreviewTokenAsync(device);
if (!previewToken) {
throw new SystemError(
`serve-sim became ready but wrote no session token for device ${device}. The preview is ` +
'on a public tunnel and would be reachable without one, so the session cannot continue. ' +
'This usually means the state file was not written as expected; retry the session, and ' +
'report it if it repeats.'
);
}
}
return host;
} catch (error) {
await host.finishAsync();
throw error;
}
}
async function finishDeviceSessionHostAsync(
ctx: CustomBuildContext,
{
previewTask,
previewServer,
screenshots,
serverName,
port,
recording,
logger,
}: {
previewTask: Promise<DeviceWebPreview> | null;
previewServer: DetachedProcessHandle;
screenshots: { finishAsync(hostStopped: boolean): Promise<void> };
serverName: string;
port: number;
recording: AndroidSessionRecording | null;
logger: bunyan;
}
): Promise<void> {
// Native ngrok operations have no scoped cancellation. Retire a late listener too.
const retirePreview = withDeviceRunSessionTimeoutAsync(
{ name: 'Preview tunnel retirement', timeoutMs: 5_000 },
async () => {
// A preview that failed to open, or was closed by its opener after finish began, has no tunnel left.
const preview = await previewTask?.catch(() => null);
await preview?.closeAsync();
}
).catch(err => {
logger.warn({ err }, `Could not close the ${serverName} preview tunnel within its deadline.`);
});
let finalization: AndroidRecordingFinalization | null = null;
if (recording) {
// stopAsync signals the whole process group, including capture's encoder.
// Finalize the MP4 first; the token protects this route on the preview server.
finalization = await finalizeAndroidRecordingAsync({
port,
controlToken: recording.controlToken,
logger,
});
}
let hostStopped = false;
try {
await previewServer.stopAsync();
if (previewServer.pid !== undefined && isProcessRunning(previewServer.pid)) {
throw new Error('Session host is still running after shutdown.');
}
hostStopped = true;
} catch (err) {
logger.warn({ err }, `Could not stop the ${serverName} session host.`);
}
await retirePreview;
await screenshots.finishAsync(hostStopped);
// A Hub that never recorded has logged its reason and left nothing to upload.
const captured = finalization !== 'not-recording';
let uploaded = false;
if (recording && captured && hostStopped) {
uploaded = await uploadFinishedAndroidRecordingAsync(ctx, { recording, logger });
}
if (recording && captured && (finalization === 'failed' || !uploaded)) {
// The Hub reports capture failures on stderr; the stop route answers with only a summary.
logger.warn(
{ hostOutput: previewServer.getOutput().slice(-HOST_OUTPUT_TAIL_CHARS) || '<empty>' },
'Session host output around the recording failure.'
);
}
}
type AndroidRecordingFinalization = 'finalized' | 'not-recording' | 'failed';
async function finalizeAndroidRecordingAsync({
port,
controlToken,
logger,
}: {
port: number;
controlToken: string;
logger: bunyan;
}): Promise<AndroidRecordingFinalization> {
try {
const response = await withDeviceRunSessionTimeoutAsync(
{ name: 'Android recording finalization', timeoutMs: 60_000 },
async signal =>
await turtleFetch(
`http://${WEB_PREVIEW_HOST}:${port}/_eas/android-recording/stop`,
'POST',
{
headers: { Authorization: `Bearer ${controlToken}` },
retries: 0,
shouldThrowOnNotOk: false,
signal,
}
)
);
if (response.status === 409) {
// The Hub answers 409 with the reason nothing was recorded, such as the emulator count.
logger.warn(`Android recording was not captured: ${await response.text()}`);
return 'not-recording';
}
if (!response.ok) {
throw new SystemError(
`Android recording finalization returned HTTP ${response.status}: ${await response.text()}`
);
}
return 'finalized';
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not finalize Android recording before shutdown', error, {
level: 'warning',
});
logger.warn({ err: error }, 'Could not finalize Android recording before shutdown.');
return 'failed';
}
}
async function uploadFinishedAndroidRecordingAsync(
ctx: CustomBuildContext,
{ recording, logger }: { recording: AndroidSessionRecording; logger: bunyan }
): Promise<boolean> {
try {
let recordings = parseDeviceScreenRecordings(
JSON.parse(
await fs.promises.readFile(path.join(recording.directory, 'recordings.json'), 'utf8')
)
);
if (recordings.length === 0) {
// recordings.json is written at finish; a killed Hub leaves what it had on disk unlisted.
recordings = await findUnlistedDeviceScreenRecordingsAsync({
root: recording.directory,
env: recording.env,
logger,
});
}
for (const item of recordings) {
if (
path.dirname(path.resolve(item.directory)) !== recording.directory ||
(await fs.promises.lstat(item.directory)).isSymbolicLink()
) {
throw new Error('Recording directory is not an owned session child.');
}
}
const uploaded = await uploadDeviceRunSessionScreenRecordingsAsync(ctx, {
logger,
deviceRunSessionId: recording.deviceRunSessionId,
recordings,
});
if (recordings.length > 0 && uploaded) {
await fs.promises.rm(recording.directory, { recursive: true });
return true;
}
return false;
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not upload the Android session recording', error, { level: 'warning' });
logger.warn(
{ err: error, recordingDirectory: recording.directory },
'Could not upload the Android session recording.'
);
return false;
}
}
@@ -1,8 +1,13 @@
import { UserError } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import { asyncResult } from '@expo/results';
import * as jose from 'jose';
import fetch from 'node-fetch';
import { ZodError, z } from 'zod';
const TOKEN_LIFETIME_SECONDS = 20 /* minutes */ * 60 /* seconds */;
const TOKEN_REFRESH_MARGIN_SECONDS = 60 /* seconds */;
type ApiSchema = {
[Path in string]: {
path?: z.ZodType<Record<string, string>>;
@@ -14,7 +19,7 @@ type ApiSchema = {
const AscErrorResponseSchema = z.object({
errors: z
.array(
z.object({
z.looseObject({
id: z.string().optional(),
status: z.string().optional(),
code: z.string().optional(),
@@ -27,6 +32,62 @@ const AscErrorResponseSchema = z.object({
});
const GetApi = {
// https://developer.apple.com/documentation/appstoreconnectapi/get-v1-builds-_id_-app
'/v1/builds/:id/app': {
path: z.object({ id: z.string() }),
request: z.object({}),
response: z.object({
data: z.object({
id: z.string(),
attributes: z.object({ primaryLocale: z.string().optional() }).optional(),
}),
}),
},
// https://developer.apple.com/documentation/appstoreconnectapi/get-v1-builds-_id_-betabuildlocalizations
'/v1/builds/:id/betaBuildLocalizations': {
path: z.object({ id: z.string() }),
request: z.object({ limit: z.number().int().max(200) }),
response: z.object({
data: z.array(
z.object({
id: z.string(),
attributes: z.object({ locale: z.string().optional() }).optional(),
})
),
links: z.object({ next: z.string().nullish() }).optional(),
}),
},
// https://developer.apple.com/documentation/appstoreconnectapi/get-v1-betagroups
'/v1/betaGroups': {
path: z.object({}),
request: z.union([
z.object({
'filter[app]': z.string(),
'filter[builds]': z.never().optional(),
limit: z.number().int().max(200),
}),
z.object({
'filter[app]': z.never().optional(),
'filter[builds]': z.string(),
limit: z.number().int().max(200),
}),
]),
response: z.object({
data: z.array(
z.object({
id: z.string(),
attributes: z
.object({
name: z.string().optional(),
isInternalGroup: z.boolean().optional(),
hasAccessToAllBuilds: z.boolean().nullish(),
})
.optional(),
})
),
links: z.object({ next: z.string().nullish() }).optional(),
}),
},
'/v1/apps': {
path: z.object({}),
request: z.object({
@@ -119,12 +180,40 @@ const GetApi = {
warnings: z.array(z.object({ code: z.string(), description: z.string() })).optional(),
}),
}),
relationships: z
.object({
build: z.object({
data: z.object({ type: z.literal('builds'), id: z.string() }).nullable(),
}),
})
.optional(),
}),
}),
},
} satisfies ApiSchema;
const PostApi = {
// https://developer.apple.com/documentation/appstoreconnectapi/post-v1-builds-_id_-relationships-betagroups
'/v1/builds/:id/relationships/betaGroups': {
path: z.object({ id: z.string() }),
request: z.object({
data: z.array(z.object({ type: z.literal('betaGroups'), id: z.string() })),
}),
response: z.undefined(),
},
// https://developer.apple.com/documentation/appstoreconnectapi/post-v1-betabuildlocalizations
'/v1/betaBuildLocalizations': {
request: z.object({
data: z.object({
type: z.literal('betaBuildLocalizations'),
attributes: z.object({ locale: z.string(), whatsNew: z.string().nullish() }),
relationships: z.object({
build: z.object({ data: z.object({ type: z.literal('builds'), id: z.string() }) }),
}),
}),
}),
response: z.object({ data: z.object({ id: z.string() }) }),
},
'/v1/buildUploads': {
request: z.object({
// https://developer.apple.com/documentation/appstoreconnectapi/builduploadcreaterequest/data-data.dictionary
@@ -223,6 +312,18 @@ const PostApi = {
} satisfies ApiSchema;
const PatchApi = {
// https://developer.apple.com/documentation/appstoreconnectapi/patch-v1-betabuildlocalizations-_id_
'/v1/betaBuildLocalizations/:id': {
path: z.object({ id: z.string() }),
request: z.object({
data: z.object({
type: z.literal('betaBuildLocalizations'),
id: z.string(),
attributes: z.object({ whatsNew: z.string().nullish() }).optional(),
}),
}),
response: z.object({ data: z.object({ id: z.string() }) }),
},
// https://developer.apple.com/documentation/appstoreconnectapi/patch-v1-builduploadfiles-_id_
'/v1/buildUploadFiles/:id': {
path: z.object({
@@ -287,23 +388,65 @@ export class AscApiRequestError extends Error {
constructor(
message: string,
public readonly status: number,
public readonly responseJson: z.output<typeof AscErrorResponseSchema>,
public readonly responseJson: z.output<typeof AscErrorResponseSchema>['errors'][number],
options?: { cause?: unknown }
) {
super(message, { cause: options?.cause });
}
get code(): string | undefined {
return this.responseJson.code;
}
}
export type AscApiKey = {
keyId: string;
issuerId?: string | null;
privateKey: jose.KeyLike;
};
export class AscApiClient {
private readonly baseUrl = 'https://api.appstoreconnect.apple.com';
private readonly token: string;
private readonly key: AscApiKey;
private cachedToken?: { value: Promise<string>; expiresAt: number };
private readonly logger?: bunyan;
constructor({ token, logger }: { token: string; logger?: bunyan }) {
this.token = token;
constructor({ key, logger }: { key: AscApiKey; logger?: bunyan }) {
this.key = key;
this.logger = logger;
}
private async getTokenAsync(): Promise<string> {
const now = Math.floor(Date.now() / 1000);
// Reuse valid tokens, but leave time for a request to reach Apple before expiry.
if (!this.cachedToken || this.cachedToken.expiresAt <= now + TOKEN_REFRESH_MARGIN_SECONDS) {
const expiresAt = now + TOKEN_LIFETIME_SECONDS;
this.cachedToken = { value: this.signTokenAsync(expiresAt), expiresAt };
}
const cachedToken = this.cachedToken;
try {
return await cachedToken.value;
} catch (error) {
if (this.cachedToken === cachedToken) {
this.cachedToken = undefined;
}
throw error;
}
}
private async signTokenAsync(expiresAt: number): Promise<string> {
const jwt = new jose.SignJWT({})
.setProtectedHeader({ alg: 'ES256', kid: this.key.keyId })
.setAudience('appstoreconnect-v1')
.setExpirationTime(expiresAt);
if (this.key.issuerId) {
jwt.setIssuer(this.key.issuerId);
} else {
jwt.setSubject('user');
}
return await jwt.sign(this.key.privateKey);
}
public async getAsync<TPath extends keyof typeof GetApi>(
path: TPath,
body: z.input<(typeof GetApi)[TPath]['request']>,
@@ -334,14 +477,40 @@ export class AscApiClient {
});
}
public async getNextPageAsync<TPath extends keyof typeof GetApi>(
path: TPath,
next: string
): Promise<z.output<(typeof GetApi)[TPath]['response']>> {
const nextUrl = new URL(next, this.baseUrl);
if (nextUrl.origin !== this.baseUrl) {
throw new Error('App Store Connect returned a pagination link on another host.');
}
return await this.sendRequestAsync({
method: 'GET',
path: nextUrl.toString(),
body: {},
requestSchema: z.object({}),
responseSchema: GetApi[path].response,
});
}
public async postAsync<TPath extends keyof typeof PostApi>(
path: TPath,
body: z.input<(typeof PostApi)[TPath]['request']>
body: z.input<(typeof PostApi)[TPath]['request']>,
// Require path params if the API schema defines them.
...params: (typeof PostApi)[TPath] extends { path: infer TSchema extends z.ZodType }
? [params: z.input<TSchema>]
: []
): Promise<z.output<(typeof PostApi)[TPath]['response']>> {
const schema = PostApi[path];
let effectivePath: string = path;
const pathParams = 'path' in schema ? schema.path.parse(params[0]) : {};
for (const [key, value] of Object.entries(pathParams)) {
effectivePath = effectivePath.replaceAll(`:${key}`, encodeURIComponent(String(value)));
}
return await this.sendRequestAsync({
method: 'POST',
path,
path: effectivePath,
body,
requestSchema: schema.request,
responseSchema: schema.response,
@@ -397,7 +566,7 @@ export class AscApiClient {
method,
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${this.token}`,
Authorization: `Bearer ${await this.getTokenAsync()}`,
},
body: method === 'GET' ? undefined : JSON.stringify(parsedBody.value),
});
@@ -408,10 +577,31 @@ export class AscApiClient {
(async () => AscErrorResponseSchema.parse(JSON.parse(text)))()
);
if (parsedAscErrorResponse.ok) {
throw new AscApiRequestError(
`Unexpected response (${response.status}) from App Store Connect: ${text}`,
response.status,
parsedAscErrorResponse.value,
const errors = parsedAscErrorResponse.value.errors.map(ascError => {
if (ascError.code === 'FORBIDDEN.REQUIRED_AGREEMENTS_MISSING_OR_EXPIRED') {
return new UserError(
'EAS_ASC_REQUIRED_AGREEMENTS_MISSING_OR_EXPIRED',
'A required Apple agreement is missing or has expired. ' +
"Ask your Apple Developer team's Account Holder to review and accept the required agreements " +
'in App Store Connect: https://appstoreconnect.apple.com/business. ' +
'If prompted, also accept the Apple Developer Program License Agreement at https://developer.apple.com/account. ' +
'Then try again.',
{ cause: ascError }
);
}
return new AscApiRequestError(
`Unexpected response (${response.status}) from App Store Connect: ${JSON.stringify(ascError)}`,
response.status,
ascError,
{ cause: response }
);
});
if (errors.length === 1) {
throw errors[0];
}
throw new AggregateError(
errors,
`App Store Connect returned multiple errors:\n${errors.map(error => error.message).join('\n')}`,
{ cause: response }
);
}
@@ -421,6 +611,9 @@ export class AscApiClient {
});
}
if (response.status === 204) {
return responseSchema.parse(undefined);
}
const text = await response.text();
const parsedJson = await asyncResult((async () => JSON.parse(text))());
if (!parsedJson.ok) {
@@ -1,14 +1,81 @@
import { UserError } from '@expo/eas-build-job';
import { SystemError, UserError } from '@expo/eas-build-job';
import fs from 'fs-extra';
import * as jose from 'jose';
import { z } from 'zod';
import {
AscApiClient,
AscApiClientGetApi,
AscApiClientPostApi,
AscApiKey,
AscApiRequestError,
AscPlatform,
} from './AscApiClient';
export namespace AscApiUtils {
export async function getAllBetaBuildLocalizationsAsync({
client,
buildId,
}: {
client: AscApiClient;
buildId: string;
}) {
let response = await client.getAsync(
'/v1/builds/:id/betaBuildLocalizations',
{ limit: 200 },
{ id: buildId }
);
const localizations = [...response.data];
for (let page = 1; response.links?.next; page++) {
if (page === 20) {
throw new SystemError(
'We only support TestFlight localization lists with up to 20 pages (4,000 localizations). Contact Expo support if you need a larger localization list.'
);
}
response = await client.getNextPageAsync(
'/v1/builds/:id/betaBuildLocalizations',
response.links.next
);
localizations.push(...response.data);
}
return localizations;
}
export async function getAllBetaGroupsAsync({
client,
appId,
buildId,
}: {
client: AscApiClient;
} & ({ appId: string; buildId?: never } | { appId?: never; buildId: string })) {
let response = await client.getAsync('/v1/betaGroups', {
...(buildId !== undefined ? { 'filter[builds]': buildId } : { 'filter[app]': appId }),
limit: 200,
});
const groups = [...response.data];
for (let page = 1; response.links?.next; page++) {
if (page === 20) {
throw new SystemError(
'We only support TestFlight group lists with up to 20 pages (4,000 groups). Contact Expo support if you need a larger group list.'
);
}
response = await client.getNextPageAsync('/v1/betaGroups', response.links.next);
groups.push(...response.data);
}
return groups;
}
export async function loadApiKeyAsync({ keyPath }: { keyPath: string }): Promise<AscApiKey> {
const keyJson = z
.object({ issuer_id: z.string().nullish(), key_id: z.string(), key: z.string() })
.parse(await fs.readJson(keyPath));
return {
keyId: keyJson.key_id,
issuerId: keyJson.issuer_id,
privateKey: await jose.importPKCS8(keyJson.key, 'ES256'),
};
}
/**
* Maps a bundle's `DTPlatformName` (from its Info.plist) to the App Store
* Connect platform used for a build upload. Unknown or missing values fall
@@ -57,12 +124,13 @@ export namespace AscApiUtils {
{ id: appleAppIdentifier }
);
} catch (error) {
const notFoundErrors =
error instanceof AscApiRequestError && error.status === 404
? error.responseJson.errors
: [];
const errors = error instanceof AggregateError ? error.errors : [error];
const isAppNotFoundError =
notFoundErrors.length > 0 && notFoundErrors.every(item => item.code === 'NOT_FOUND');
errors.length > 0 &&
errors.every(
item =>
item instanceof AscApiRequestError && item.status === 404 && item.code === 'NOT_FOUND'
);
if (!isAppNotFoundError) {
throw error;
}
@@ -124,13 +192,15 @@ export namespace AscApiUtils {
},
});
} catch (error) {
const errors =
error instanceof AscApiRequestError && error.status === 409
? error.responseJson.errors
: [];
const errors = error instanceof AggregateError ? error.errors : [error];
const isDuplicateVersionError =
errors.length > 0 &&
errors.every(item => item.code === 'ENTITY_ERROR.ATTRIBUTE.INVALID.DUPLICATE');
errors.every(
item =>
item instanceof AscApiRequestError &&
item.status === 409 &&
item.code === 'ENTITY_ERROR.ATTRIBUTE.INVALID.DUPLICATE'
);
if (isDuplicateVersionError) {
throw new UserError(
Loaded 100 of 303 files, more files were not shown because too many files have changed in this diff. Show more