Compare commits

...
127 Commits
Author SHA1 Message Date
Expo CI 195807643f v24.4.2 2026-09-15 05:03:15 +00:00
Brent Vatne 21256e6d55 [eas-cli] Support Xcode 27 Device Hub when running iOS simulator builds (#4405) 2026-09-14 22:00:43 -07:00
Expo CI 705fa875d3 bump STAGING and PRODUCTION to 24.4.1 in cli-versions.json 2026-09-15 04:14:14 +00:00
Expo CI 498384357e update CHANGELOG.md for v24.4.1 2026-09-15 04:10:19 +00:00
Expo CI 53938e4cf8 v24.4.1 2026-09-15 04:07:48 +00:00
Brent Vatne e18ffcd48e [eas-cli] Bump @expo/apple-utils to 2.2.1 (#4404) 2026-09-14 21:04:22 -07:00
Expo CI df12f86da9 bump STAGING and PRODUCTION to 24.4.0 in cli-versions.json 2026-09-14 21:17:19 +00:00
Expo CI 3dba7f2bf7 update CHANGELOG.md for v24.4.0 2026-09-14 21:13:37 +00:00
Expo CI e7efe14336 v24.4.0 2026-09-14 21:11:01 +00:00
Brent Vatne 78f2103bcb [build-tools] inject a guard into simulator processes that refuses connections bypassing the local egress proxy (#4393) 2026-09-14 14:08:02 -07:00
Brent Vatne 565b0ec425 [build-tools] set proxy environment variables in the Simulator for local egress (#4390) 2026-09-14 13:56:04 -07:00
Ramon Claudio 253d72bb4d [eas-cli] Bump Node to 20.18.3 (#4279)
* Update package.json to use new node version

* Update test workflow to use new node version

* Update important notes section in CLAUDE.md to use new node version

* Add changelog
2026-09-14 15:20:14 -04:00
Stanisław Chmiela 35522e9b28 [build-tools] Skip source refresh for jobs with no sources (#4389)
# Why

`workflow-orchestration` tests rightly use `projectArchive.type: NONE`. NONE sources don't need refresh so we can just handle them inline.

# How

If project archive type is NONE, just return, there's nothing to do, refresh, or limit based on job type.

# Test Plan

CI should fix. E2E tests should pass.
2026-09-14 11:34:15 +02:00
Expo CI 4aa9522825 bump STAGING and PRODUCTION to 24.3.0 in cli-versions.json 2026-09-11 12:37:23 +00:00
Expo CI a92b0b0ad5 update CHANGELOG.md for v24.3.0 2026-09-11 12:34:24 +00:00
Expo CI 66cdccb0e7 v24.3.0 2026-09-11 12:31:21 +00:00
Stanisław Chmiela 2eefa44f8f [eas-build-job] Add sandbox daemon protocol (#4346)
## Why

"sandboxd" from `build-tools` needs to connect to the MCP server in `universe`. They need to be able to talk to each other. I'm thinking it might be nice if they shared an "API protocol"?

## How

Added Zod schema for input and output of to-be-supported messages to `eas-build-job`.

## Test plan

CI should pass. I think I tested this locally too.
2026-09-11 14:23:53 +02:00
Stanisław Chmiela 55302dd830 [build-tools] Add sandbox MCP connection step (#4333)
# Why

See https://github.com/expo/universe/pull/30766.

# How

Added a simple "sandbox daemon" that connects to MCP and an `eas/start_sandbox` EAS function.

This sets up the basic structure, planning to take care of protocol, handling messages and executing commands will come in future PRs.

# Test Plan

Tested with https://github.com/expo/universe/pull/30766 and https://github.com/expo/universe/pull/30763.

```
<-- GET /sandbox/connect
New sandbox daemon - sandboxId[01a0692f-b464-77c8-8ab0-e2143eec9872] ipAddress[::1]
--> GET /sandbox/connect 101 23ms
[wrangler:info] GET /sandbox/connect 101 Switching Protocols (50ms)
```
2026-09-11 14:02:24 +02:00
Expo CI 924eb87157 bump STAGING and PRODUCTION to 24.2.0 in cli-versions.json 2026-09-11 11:01:44 +00:00
Expo CI f546bf3352 update CHANGELOG.md for v24.2.0 2026-09-11 10:55:55 +00:00
Expo CI 97553334d4 v24.2.0 2026-09-11 10:52:46 +00:00
Stanisław Chmiela 31d12b9888 [eas-build-job] Remove repository URLs from Git job inputs (#4360)
* [eas-build-job] Allow jobs to omit the Git repository URL

* [build-tools] Separate resolved archive sources from job inputs

* [eas-build-job] Remove repository URLs from Git job inputs

* [eas-build-job] Remove the legacy repository URL schema field

* [eas-build-job] Move changelog entry to unreleased section

* [eas-build-job] Correct unreleased changelog placement
2026-09-11 12:19:58 +02:00
Ash Wu dc2658f63b [eas-cli] add eas workflow:insights:maestro command (#4370)
Signed-off-by: Ash Wu <[email protected]>
2026-09-11 17:57:30 +08:00
Ash Wu b8b9de78fc [eas-cli] add eas workflow:insights command (#4367)
* [eas-cli] add eas workflow:insights command

Signed-off-by: Ash Wu <[email protected]>

* [eas-cli] address review feedback on workflow:insights

- Show workflow file names in the workflows table, which is the value
  --workflow accepts, and add fileName to the JSON output
- Omit buckets with no runs from the runs-over-time table; the JSON output
  keeps every bucket
- Replace the per-command --project-id context resolvers with a protected
  EasCommand helper shared with the observe commands

Signed-off-by: Ash Wu <[email protected]>

* [eas-cli] send a bucket-aligned time range from workflow:insights

The server widens only the runs-over-time query to whole buckets and leaves the
overview and per-workflow queries on the exact bounds, so an unaligned window
made those sections count different runs and the bucket totals could exceed the
overview's total. Widen the window client-side instead, the way the dashboard
does, so the server's own widening is a no-op and every section covers the same
period.

Widening lengthens the window, which can push it past the threshold of the
bucket size it was widened by, so the bucket size is chosen by checking the
widened window rather than the requested one. Without that, --days 4 would ask
for hourly buckets over a 97-hour window and the server would reject it.

- Reject a range that does not run forwards in resolveTimeRange, next to the
  existing flag checks; widening would otherwise turn a backwards range into a
  valid one-bucket query
- Enable JSON output before resolving the time range and the project context,
  both of which can write prose to stdout
- Show a metric's change in parentheses, matching how the observe formatters
  pair a statistic with its secondary figure

Signed-off-by: Ash Wu <[email protected]>

---------

Signed-off-by: Ash Wu <[email protected]>
2026-09-11 17:57:30 +08:00
Stanisław Chmiela 8ff599b1f2 [build-tools] Require PATH sources for local builds (#4362)
* [build-tools] Require PATH sources for local builds

* [build-tools] Document local source validation

* [build-tools] Report invalid local sources as user errors

* [build-tools] Track invalid local sources as system errors
2026-09-11 11:03:45 +02:00
Expo CI e6f02af973 bump STAGING and PRODUCTION to 24.1.2 in cli-versions.json 2026-09-11 00:59:22 +00:00
Expo CI a163b5a76d update CHANGELOG.md for v24.1.2 2026-09-11 00:57:25 +00:00
Expo CI cc37c1c360 v24.1.2 2026-09-11 00:55:13 +00:00
Brent Vatne e78771e0f4 [eas-cli] drop the duplicate non-interactive egress hint and the disconnect warning on stop (#4384) 2026-09-10 17:52:34 -07:00
Brent Vatne 288bc44dfc [eas-cli] shorten the local egress banner and drop the run-it-yourself instruction in interactive mode (#4383) 2026-09-10 17:42:22 -07:00
Expo CI d3db3926e3 bump STAGING and PRODUCTION to 24.1.1 in cli-versions.json 2026-09-11 00:15:23 +00:00
Expo CI 5b87ba0adb update CHANGELOG.md for v24.1.1 2026-09-11 00:13:05 +00:00
Expo CI 0b2cea7bb7 v24.1.1 2026-09-11 00:10:40 +00:00
Brent Vatne eab4fc3cd5 [eas-cli] describe the inline egress client instead of asking to start one (#4382) 2026-09-10 17:07:16 -07:00
Brent Vatne e2e8cd245c [eas-cli] improve EasCommand test reliability (#4381) 2026-09-10 17:05:16 -07:00
Expo CI 80d46f0783 bump STAGING and PRODUCTION to 24.1.0 in cli-versions.json 2026-09-10 23:27:17 +00:00
Expo CI 3b6b63a273 update CHANGELOG.md for v24.1.0 2026-09-10 23:24:59 +00:00
Expo CI 8373ae353c v24.1.0 2026-09-10 23:22:26 +00:00
Brent Vatne 994794dbad [eas-cli] add --egress-allow to reach local dev servers through local egress (#4369)
* [eas-cli] add --egress-allow so local egress sessions can reach dev servers on this machine

Adds repeatable --egress-allow host:port, requiring --egress local. Entries
are normalized and deduplicated; wildcards, ranges, and malformed values are
rejected before session creation. Exceptions match the exact host and port;
unlisted destinations keep the default policy. Exceptions persist in the
session configuration and are shown when starting or reconnecting, across all
iOS controllers.

Squashes the earlier --egress-allow commits and base-branch sync merges so the
branch sits directly on the rebased base.

* [eas-cli] forward loopback --egress-allow ports from the simulator host like adb reverse

iOS never sends 127.0.0.1 requests to the system proxy and Expo CLI rewrites
localhost to 127.0.0.1 in manifests, so --egress-allow localhost:<port> alone
could not serve a dev server to Expo Go. For each localhost or 127.0.0.1 entry
the egress client now opens a chisel reverse remote so 127.0.0.1:<port> on the
device host reaches the same port on this machine. Privileged ports and the
proxy port stay name-only. The start output explains the forwarding, and
rejected remotes are surfaced instead of looping silently.
2026-09-10 14:12:01 -07:00
Brent Vatne d103da3bc5 [eas-cli][build-tools] local egress for iOS simulator sessions (--egress local) (#4364)
* [build-tools] add start_local_egress step for local egress simulator sessions

* [eas-cli] add --egress local to simulator:start and add simulator:egress

* [eas-cli] update CHANGELOG

* [build-tools] drop the pf UDP block from local egress and report direct simulator connections

* [eas-cli] harden the local egress proxy and narrow the contract wording

* [build-tools][eas-cli] report the egress token separately from the fixed username

* [build-tools][eas-cli] harden local egress startup and proxy resource limits

* [build-tools][eas-cli] support local egress across iOS simulator controllers

* [build-tools] allow local egress reverse remotes on unprivileged loopback ports

iOS never sends loopback-literal requests to the system proxy, and Expo CLI
rewrites localhost to 127.0.0.1 in manifests, so a dev server allowed as
localhost:<port> could not serve bundles through the proxy. Let the EAS CLI
open one chisel reverse remote per loopback allow entry (like adb reverse)
by permitting reverse remotes on any unprivileged loopback port, not only
the proxy port. Other interfaces and privileged ports stay denied.

* [eas-cli] stop logging completed proxied responses as failed in the local egress client

Releasing a proxy operation aborts its shared signal after the response has
been sent, which errors the already-completed upstream request and logged
"failed: The operation was aborted" for every successful plain-HTTP request in
debug output. Track completion (response end or upgrade handoff) and treat the
late abort as cleanup. Truncated upstream responses still log as failed.
2026-09-10 14:12:01 -07:00
Stanisław Chmiela ee3c62683b [build-tools] Require project source refresh for remote builds (#4359)
# Why

I want to start skipping authenticated repository URL in the job payload. We never expect to use it — we'd rather use the just-loaded one from www.

# How

Change fallback to a `SystemError`.

# Test Plan

CI should pass.
2026-09-10 22:40:41 +02:00
Gabe Debes 8ac9d5f101 [build-tools] gate serve-sim with the session token (#4306)
* [eas-cli] gate device-run-session serve-sim with the session token

* [eas-cli] describe the state the missing-token check actually catches

* [eas-cli] match the missing-token test to the message it asserts
2026-09-09 17:40:01 -07:00
Expo CI 1eaa799b9f bump STAGING and PRODUCTION to 24.0.0 in cli-versions.json 2026-09-09 20:13:34 +00:00
Expo CI bb0f2dfb21 update CHANGELOG.md for v24.0.0 2026-09-09 20:11:16 +00:00
Expo CI 5409c03d58 v24.0.0 2026-09-09 20:07:38 +00:00
Gabe Debes a941930ea7 [eas-cli] adds serve-sim session token in simulator preview links (#4312)
* [eas-cli] carry the serve-sim session token in simulator preview links

* [eas-cli] sanitize the preview config in json output
2026-09-09 12:52:08 -07:00
Szymon Świerk 74572101ac [eas-cli] Make ASC API key issuer identifier optional in types (#4248)
* [eas-cli] Make ASC API key issuer identifier optional in types

* [eas-cli] Bump @expo/apple-utils to 2.2.0

* [eas-cli] Pass optional issuer identifier through to TestFlight app lookup
2026-09-09 21:22:32 +02:00
Stanisław Chmiela 7b050cae3f [worker] Add MCP server URL to build context (#4335)
# Why

Worker has config for API server URL. I now need MCP server URL too.

# How

Added `mcpServerUrl` in `config` and in contexts.

# Test Plan

CI should pass.
2026-09-09 16:56:49 +02:00
Douglas LowderandClaude Opus 4.8 d193860668 [eas-cli] Add observe:event command to fetch a single event by ID (#4252)
* [eas-cli] Add observe:event command to fetch a single event by ID

Add `eas observe:event <id>` to display one Observe event (either metric or userEvent) by its ID. Also, ensure event IDs are propagated to the results of `eas observe:session <sessionId>` so that they appear in JSON output.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-09 06:07:57 -07:00
Krystof WoldrichandCodex 1da0be8529 [build-tools] Use bun x to start Argent remote sessions (#4366)
Co-authored-by: Codex (GPT-6) <[email protected]>
2026-09-08 15:22:47 +02:00
Douglas LowderandClaude Opus 4.8 08b4346620 [eas-cli] Add observe:errors command (#4339)
* [eas-cli] Add observe:errors command

Add `eas observe:errors` to display error and exception issue groups grouped by
fingerprint, using the new observe.errors.groups GraphQL namespace. Exceptions
are excluded from observe:events (userEvents.list) and surfaced here instead.
Supports --platform, --severity (fatal|error), time-range, --app-version,
--build-number, --update-id, and --environment filters, with table and --json
output.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

* [eas-cli] Add --fingerprint drill-down to observe:errors

Add `eas observe:errors --fingerprint <fingerprint>` to list an error group's
individual occurrences with stack traces, instead of only the grouped summary.
Occurrences are fetched via observe.errors.occurrences (newest first) with
--limit and --after paging, and rendered as per-occurrence detail (fields,
stack trace, body, properties) or as --json. The grouped table now shows a
Fingerprint column and a hint pointing to the flag.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

* Respond to review comments

* Regenerate GraphQL from server

* Fix errors from nullability change in issuerIdentifier

---------

Co-authored-by: Claude Opus 4.8 <[email protected]>
2026-09-07 20:29:56 -07:00
4d6d9b1fc4 [build-tools] Allow configuring Android emulator display (#4349)
Co-authored-by: Krystof Woldrich <[email protected]>
Co-authored-by: Codex (GPT-6) <[email protected]>
2026-09-07 15:39:56 +02:00
Stanisław Chmiela 59f9add53c [build-tools] Add custom log phase helper (#4334)
# Why

I want a single EAS function to appear as running as two separate phases.

# How

Added `withLogPhaseAsync` which is going to create a logger child with necessary fields overwritten so a new log phase appears.

# Test Plan

CI should pass. Verified locally.

<img width="1343" height="712" alt="Zrzut ekranu 2026-09-4 o 12 17 25" src="https://github.com/user-attachments/assets/f4733c83-3119-42d6-b7dc-fc46aedaf434" />
2026-09-07 15:11:57 +02:00
1946562247 [build-tools] Update Appium command list (#4356)
Co-authored-by: Expo Bot <[email protected]>
Co-authored-by: Stanisław Chmiela <[email protected]>
2026-09-07 10:41:29 +00:00
SJ Kim 24a65ad80f [eas-cli] Support protected update channels (#4319)
* [eas-cli] Support protected update channels

* [eas-cli] Add protected channels changelog entry

* [eas-cli] Handle missing protected channels

* [eas-cli] Address channel protection review feedback (ENG-25975)
2026-09-04 14:03:53 -04:00
Hirbod 37ea60c20c [eas-cli] remove hidden AI chat command (#4348) 2026-09-04 17:03:29 +02:00
Douglas LowderandClaude Opus 4.8 99cbcae249 [eas-cli] Align Observe code with new server schema (#4338)
* [eas-cli] Align observe commands with the new Observe GraphQL schema

The Observe GraphQL schema was reorganized into scoped namespaces (universe
#30606). Migrate the observe queries onto the new fields — metrics.list,
userEvents.list/names, navigation.routes, session(id){metrics logs} — and adopt
the renamed fields in --json output (metricName->name, metricValue->value,
eventName->name). observe:events now lists user-defined events only.
observe:session timelines now include errors via session.logs. appVersions
(observe:versions, observe:metrics-summary) is unchanged. Regenerate GraphQL and
update tests.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-04 06:29:01 -07:00
Szymon Świerk a426042784 [build-tools] Support individual ASC API keys in upload_to_asc (#4247)
* [build-tools] Support individual ASC API keys in upload_to_asc

* [build-tools] In upoload_to_asc function, set jwt subject with a dedicated function instead of setting the claim manually
2026-09-03 16:34:50 +02:00
1cb06a6cb2 [build-tools] Install FFmpeg before Android device hub (#4332)
Co-authored-by: Krystof Woldrich <[email protected]>
Co-authored-by: Codex <[email protected]>
2026-09-03 15:08:04 +02:00
Krystof WoldrichandCodex 5236871b0b fix(ci): changelog for PRs from forks (#4331)
Co-authored-by: Codex <[email protected]>
2026-09-03 12:49:30 +02:00
ceb20c9dd6 [build-tools] Use lower 980px Android preview resolution (#4326)
Co-authored-by: Krystof Woldrich <[email protected]>
Co-authored-by: Codex <[email protected]>
2026-09-03 10:43:09 +02:00
Krystof WoldrichandCodex 83fa42c179 [eas-cli] Avoid GraphQL argument type collisions (#4328)
Co-authored-by: Codex <[email protected]>
2026-09-02 16:50:52 +02:00
Stanisław Chmiela 7727b9c347 [ci] Gate worker CLI versions on npm availability (#4313)
We currently:
- publish to NPM without awaiting the release being ready and public
- immediately change `cli-versions.json`
- this makes builds immediately try to use a new `eas-cli` version despite it NOT being public yet, which causes problems, see [Datadog](https://app.datadoghq.com/logs?query=%22No%20matching%20version%20found%20for%20eas-cli%22&agg_m=count&agg_m_source=base&agg_t=count&cols=host%2Cservice&fromUser=true&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=desc&viz=stream&from_ts=1788174847023&to_ts=1788178586558&live=false).

This pull request makes it so we:
- publish to NPM, make a GitHub release
- update changelog and push to main
- await NPM readiness for 10 minutes (last release took 6)
- only once it's ready, update `cli-versions.json`.
2026-09-02 09:02:12 +00:00
Stanisław Chmiela b015c0b226 [worker] Keep legacy cache failures non-fatal (#4295)
Add Sentry logging to legacy build cache.
2026-09-02 09:29:59 +02:00
Kadi Kraman 882687818f [eas-cli] add --build-number to observe:metrics and observe:events (#4278)
* Add --build-number to observe:metrics and observe:events

* Add changelog
2026-09-01 20:46:41 +01:00
Kadi Kraman 37f5163872 [eas-cli] add apple platform value to observe commands (#4276)
* Regenerate GraphQL schema

* Add apple platform target model

* Query observe data by platform target

* Wire --platform apple through the observe commands

* Update observe tests for platform targets

* Add changelog

* Format

* Regenerate gql

* Sync gql
2026-09-01 20:08:39 +01:00
Stanisław Chmiela df2c81d131 [eas-cli] Update simulator device GraphQL inputs (#4327) 2026-09-01 19:37:08 +01:00
Kadi Kraman 236398a823 [eas-cli] add environment filter to observe commands (#4275)
* Add shared --environment flag and thread it through the observe query layer

* Add --environment to observe:metrics-summary

* Add --environment to observe:metrics

* Add --environment to observe:routes

* Add --environment to observe:events

* Add --environment to observe:versions

* Add --environment to the observe:session picker

* Add changelog

* Add links to changelog
2026-09-01 13:49:33 +01:00
Jon Samp eebe02eef2 [eas-cli] Add project:status command (#4006)
* [eas-cli] Add project:status command

Adds `eas project:status` (alias `eas status`), a snapshot of the project
overview: recent production builds, development builds, workflow runs,
submissions, and updates. Mirrors the website project dashboard and
supports --json and --non-interactive for agent consumption.

* [eas-cli] Fix changelog PR link

* [eas-cli] Address project status review feedback

* [eas-cli] Improve project status for agents

* [eas-cli] Clarify project status pagination lookahead

* [eas-cli] Adapt project status to current GraphQL fields

* [eas-cli] Remove dead README link to simulator/start.ts

* [eas-cli] Normalize unknown workflow run statuses in project status
2026-08-31 22:43:59 -05:00
Expo CI 4b1e5a76e2 update CHANGELOG.md and bump STAGING and PRODUCTION in cli-versions.json 2026-08-31 20:28:03 +00:00
Expo CI 74292d2ace v23.2.0 2026-08-31 20:25:25 +00:00
Szymon Dziedzic b0b7cad316 [eas-cli] Warn about Android simulator parity (#4320)
* [eas-cli] warn about Android simulator parity

* [eas-cli] add Android simulator warning changelog
2026-08-31 22:22:53 +02:00
expo-tuft[bot]andtchayen 926887d14e [eas-cli] Add --tag to simulator session commands (#4318)
Lets projects with several app variants group and filter their sessions.

Co-authored-by: expo-tuft[bot] <288127324+expo-tuft[bot]@users.noreply.github.com>
Co-authored-by: tchayen <[email protected]>
2026-08-31 22:40:52 +03:00
Gabe DebesandTomasz Mazur 2e6bb2d276 [eas-cli] Add --ssh to workflow:run (#4033)
* [eas-cli] add --ssh to workflow:run

* [eas-cli] Restore WorkflowRunSshInput types dropped by staging regen

Staging schema does not include the --ssh input yet (universe #29128),
but the CLI stack already depends on it. Keep the types until that ships.

* Regenerate GraphQL

* Dedupe changelog

---------

Co-authored-by: Tomasz Mazur <[email protected]>
2026-08-31 17:30:17 +02:00
Gabe DebesandTomasz Mazur 30656509ac [eas-cli] Add workflow:ssh command (#4032)
* [eas-cli] add workflow:ssh command

* Poll for an open SSH session before sleeping.

Avoids a needless wait when the first connect-info fetch is already ready.

* Update SSH wait test for poll-before-sleep.

The wait loop now polls first, so keep one pending poll before ready.

* Cover WorkflowJobSshQuery error and turtleBuild session paths.

* [eas-cli] Regen GraphQL types from staging (TurtleSshSession)

* [eas-cli] address workflow:ssh review feedback

* [eas-cli] drop the GET_BUILD check from sshRequested

* [eas-cli] accept a job run id in workflow:ssh

* Use URL to parse the host

* Regenerate GraphQL

* Move changelog entry

---------

Co-authored-by: Tomasz Mazur <[email protected]>
2026-08-31 17:30:17 +02:00
66e3f1ad7b [build-tools] Use expo-device-hub for Android web previews (#4304)
Co-authored-by: szdziedzic <[email protected]>
Co-authored-by: Codex <[email protected]>
2026-08-31 15:49:49 +02:00
Tomasz Mazur 9aadf2d467 [eas-cli] Stream workflow run logs in real time (#4228)
* [eas-cli] Add centrifuge and ws to deps

* [eas-cli] Stream workflow job logs while waiting for a run

* [eas-cli] Add changelog entry for realtime workflow logs

* [eas-cli] Make msg optional and fix flickering

* [eas-cli] Fetch job logs without casing on job type

* [eas-cli] Ingest realtime incrementally instead of rebuilding full logs on every message (#4293)
2026-08-31 13:38:38 +00:00
Stanisław Chmiela ba80e57753 [eas-cli] Update GraphQL schema for web preview sessions (#4315)
Updated GraphQL and adjusted code.
2026-08-31 15:20:33 +02:00
Expo CI 50c76f41e3 update CHANGELOG.md and bump STAGING and PRODUCTION in cli-versions.json 2026-08-31 11:27:55 +00:00
Expo CI 0658d58efd v23.1.0 2026-08-31 11:24:34 +00:00
Phil Pluckthun 76a700c45c Remove verify step (#4310) 2026-08-29 18:07:45 +01:00
Szymon Dziedzic 5ded193ccf [eas-cli] Fix v23.0.0 changelog (#4309) 2026-08-29 11:24:10 +02:00
Brent Vatne 93420d1fa0 [eas-cli] Add prebuild fallback regression tests (#4285)
* [eas-cli] Add prebuild fallback regression tests

* Update changelog for prebuild fallback tests
2026-08-28 15:45:29 -07:00
Brent Vatne add787f625 [local-build-plugin] Replace bunyan with @expo/bunyan (#4284)
* [local-build-plugin] Replace bunyan with @expo/bunyan

* Update changelog for @expo/bunyan
2026-08-28 15:45:28 -07:00
Brent Vatne ec2c2aca5e [steps] Replace lodash.get with a local helper (#4283)
* [steps] Replace lodash.get with a local helper

* Update changelog for lodash.get removal
2026-08-28 15:45:28 -07:00
Brent Vatne 1e86c48f48 [eas-cli][steps] Bump uuid to v11 (#4282)
* [eas-cli][steps] Bump uuid to v11

* Update changelog for uuid v11
2026-08-28 15:45:27 -07:00
Brent Vatne 545c2dbf9b [eas-cli] Bump minizlib to 3.1.0 (#4281) 2026-08-28 15:45:27 -07:00
Brent Vatne 3cb029147e [eas-cli] Prompt for Expo SDK version in eas new (#4301) 2026-08-28 12:05:38 -07:00
expo-tuft[bot]andabbanmustafa d48cfbba69 [build-tools] Normalize extracted project permissions (#4287)
* [build-tools] Normalize extracted project permissions

Co-authored-by: abbanmustafa <[email protected]>

* [build-tools] Add project permission fix changelog

Co-authored-by: abbanmustafa <[email protected]>

* [build-tools] Log project permission normalization

Co-authored-by: abbanmustafa <[email protected]>

---------

Co-authored-by: expo-tuft[bot] <288127324+expo-tuft[bot]@users.noreply.github.com>
Co-authored-by: abbanmustafa <[email protected]>
2026-08-28 12:51:33 -04:00
Robin Gullo 07b5c0daa3 [build-tools] Support pnpm install --filter to install scoped dependencies (#4302)
# Why

`EAS_YARN_FOCUS_WORKSPACE` ([expo/eas-build#598](https://github.com/expo/eas-build/pull/598)) and `EAS_BUN_FILTER_WORKSPACE` ([expo/eas-cli#4292](https://github.com/expo/eas-cli/pull/4292)) let monorepos install only the dependencies of the app being built, so `node_modules` stays slim, the install is faster, and the fingerprint matches what you get locally. pnpm can do the same thing with [`pnpm install --filter <workspace>`](https://pnpm.io/filtering), but there is no way to reach it from EAS today — the pnpm branch always installs every workspace.

# How

Set it in the build profile's `env` in `eas.json`:

```json
{
  "build": {
    "production": {
      "env": {
        "EAS_PNPM_FILTER_WORKSPACE": "my-app"
      }
    }
  }
}
```

The install step then runs `pnpm install --filter my-app`. This is the pnpm counterpart of `EAS_YARN_FOCUS_WORKSPACE`, introduced in expo/eas-build#598, and `EAS_BUN_FILTER_WORKSPACE`, introduced in expo/eas-cli#4292.

# Test Plan

Added two unit tests for the flag set/unset — `yarn jest-unit src/common/__tests__/installDependencies.test.ts --runInBand` in `packages/build-tools`:

```
Test Suites: 1 passed, 1 total
Tests:       9 passed, 9 total
```

Also ran the resulting commands by hand with pnpm 10.32.1, in a workspace with `pkg-a` (depends on `is-odd`), `pkg-b` (depends on `lodash`) and an in-sync `pnpm-lock.yaml`:

| command | installed |
| --- | --- |
| `pnpm install --frozen-lockfile` | both `packages/a/node_modules/is-odd` and `packages/b/node_modules/lodash` |
| `pnpm install --filter pkg-a --frozen-lockfile` | only `packages/a/node_modules/is-odd`, `pnpm-lock.yaml` unchanged |
2026-08-28 18:32:10 +02:00
David Mokos b78c84af2c updates agent cli detector to 0.1.7 (#4272) 2026-08-28 17:31:20 +02:00
Phil Pluckthun dfaa2df90d [eas-cli] Add dynamic retries for deploy command (#4299) 2026-08-28 16:12:40 +01:00
Expo CI 2e2aaa4ac9 v23.0.0 2026-08-28 11:15:00 +00:00
Szymon Dziedzic 3ef0f74110 [eas-cli] Rename serve-sim session type to web-preview-only (#4298)
* [eas-cli] Rename simulator web preview session type

* [eas-cli] Add simulator flag rename changelog

* [eas-cli] Clarify preview-only simulator type
2026-08-28 13:12:33 +02:00
Stanisław Chmiela 51ba7198d7 [build-tools] Ignore minimum release age for internal npx (#4296)
# Why

When a user defines a min package age policy a fresh release of `eas-cli` may fail.

Reported in [the internal support thread](https://exponent-internal.slack.com/archives/C09H6RFAX27/p1787835815444139).

# How

Added `NPM_CONFIG_MIN_RELEASE_AGE=0` to NPM environment to override min release age to 0.

# Test Plan

Codex verified with different npx versions.
2026-08-27 18:19:03 +00:00
Szymon Dziedzic e10e6a2235 [build-tools] Lower serve-sim preview resolution (#4297)
* [build-tools] Lower serve-sim preview resolution

* [build-tools] Add serve-sim resolution changelog
2026-08-27 19:10:56 +02:00
Konrad Armatys 17f84cf7d6 [build-tools] Support bun install --filter to install scoped dependencies (#4292)
# Why

`EAS_YARN_FOCUS_WORKSPACE` ([expo/eas-build#598](https://github.com/expo/eas-build/pull/598)) lets a monorepo install only the dependencies of the app being built, so `node_modules` stays slim, the install is faster, and the fingerprint matches what you get locally. Bun can do the same thing with `bun install --filter <workspace>` ([oven-sh/bun#16093](https://github.com/oven-sh/bun/pull/16093)), but there is no way to reach it from EAS today — the bun branch always installs every workspace.

# How

Set it in the build profile's `env` in `eas.json`:

```json
{
  "build": {
    "production": {
      "env": {
        "EAS_BUN_FILTER_WORKSPACE": "my-app"
      }
    }
  }
}
```

The install step then runs `bun install --filter my-app`. This is the bun counterpart of `EAS_YARN_FOCUS_WORKSPACE`, introduced in expo/eas-build#598.

# Test Plan

Added two unit tests for the flag set/unset — `yarn jest-unit src/common/__tests__/installDependencies.test.ts` in `packages/build-tools`:

```
Test Suites: 1 passed, 1 total
Tests:       7 passed, 7 total
```

Also ran the resulting commands by hand, in a workspace with `pkg-a` (depends on `is-odd`), `pkg-b` (depends on `lodash`) and an in-sync `bun.lock`:

| command | installed |
| --- | --- |
| `bun install --frozen-lockfile` | 4 packages, `packages/b/node_modules/lodash` present |
| `bun install --filter pkg-a --frozen-lockfile` | 2 packages, only `packages/a/node_modules/is-odd`, `bun.lock` unchanged |
2026-08-27 18:41:24 +02:00
Mustafa Abban f7d9b7b297 [build-tools] support remote CocoaPods caching (#4266)
* [build-tools] Cache CocoaPods dependencies between iOS builds

* restore/save cache

* remove CocoaPods download cache

* remove no-lockfile cache key
2026-08-27 10:20:11 -04:00
Stanisław Chmiela b287d7a4e0 [worker] Use cache API for the legacy cache (#3860)
The legacy cache does not work in `workflow-orchestration`.

This rewrites the `CacheManager` to the new/not-so-new Turtle caches API.

## Summary
- replace worker build cache restore/save GCS config URLs with turtle cache download/upload helpers
- derive stable build cache keys without mutating job.cache and preserve clear for force uploads
- keep existing tar archive restore/save semantics and add CacheManager unit coverage

## Test plan
- cd packages/worker && yarn test CacheManager.test.ts
- cd packages/worker && yarn typecheck
- cd packages/build-tools && yarn typecheck
- git diff --check
2026-08-27 15:44:28 +02:00
Douglas Lowder d9d1b91fa5 [ci] Bump both STAGING and PRODUCTION cli-versions in release workflow (#4290) 2026-08-27 04:16:20 -07:00
Expo CI a7436edb50 update CHANGELOG.md and bump STAGING in cli-versions.json 2026-08-26 23:54:31 +00:00
Expo CI 89fe2cb3e1 v22.6.0 2026-08-26 23:49:36 +00:00
Szymon Dziedzic 0e8e8cc0aa [build-tools] Increase serve-sim bitrate to 6 Mbps (#4291)
* [build-tools] Increase serve-sim bitrate to 16 Mbps

* [build-tools] Add serve-sim bitrate changelog entry

* [build-tools] Set serve-sim bitrate to 6 Mbps
2026-08-27 01:45:23 +02:00
Gabe Debes 6d7cef80a3 [eas-cli] add experimental support to resource class on simulator sessions (#4268)
* [eas-cli] let simulator sessions request a large resource class

* [eas-cli] add changelog for hidden --resource-class

* [eas-cli] call --resource-class experimental in the changelog

* [eas-cli] oxfmt simulator resource-class test

* [eas-cli] regenerate graphql for resource class
2026-08-26 23:34:56 +00:00
Gabe Debes 5a744b73f5 [worker] Wire SSH session into build (#4031)
* [worker] wire ssh session into build

* Rename SSH phase sessionPromise to done.

Same anti-flatten object wrapper; clearer that callers await tunnel teardown.

* Cover SSH phase supervise callbacks and teardown failure paths.

* [worker] Stop baking upterm into the worker package

build-tools resolves upterm from PATH or downloads darwin-arm64 /
linux-amd64 from gs://turtle-v2/upterm at runtime.

* [worker] align ssh phase with build-tools api

* [worker] build the ssh target from the job type
2026-08-26 16:25:12 -07:00
Gabe Debes b8b53e262a [build-tools] Add worker-side SSH session helper (#4030)
* [build-tools] add worker-side ssh session helper

* Keep host.connectionConfig current and simplify SSH reconnect.

Update connectionConfig on every redial, linearize ensureConnectedAsync, and
parse upterm session objects without a JSON stringify round-trip.

* Reuse SshSettings instead of inlining idleTimeout/relay fields.

* Cover remaining SSH helper branches in unit tests.

Invalid upterm host URLs, missing process pid, stdout chunks, unknown
client counts during the job, and turtleBuild targets.

* Format unknownGraceMs ternary for oxfmt.

* [build-tools] Drop dup changelog entry; redact upterm exit logs

* [build-tools] gitignore baked upterm binaries

* [build-tools] Cover SSH helpers; drop upterm gitignore; restore review fixes

Zod-parse upterm session JSON and move isChildProcessAlive into processes.ts
(claimed in review but never landed). Expand unit coverage and stop gitignoring
baked upterm binaries now that the worker fetches from GCS.

* [build-tools] Resolve upterm from PATH or GCS instead of baking

Restore the orphaned review-fix tip: prefer an image/PATH upterm, else
download darwin-arm64 / linux-amd64 from gs://turtle-v2/upterm. Keeps the
stricter zod session parse, process-group/admin-socket comments, and GCS
cache/path unit coverage.

* [build-tools] Redact upterm tokens from ws:// relay URLs too

* [build-tools] Align upterm PATH probe with job env; scrub controls in logs

* [build-tools] address SSH review feedback

* [build-tools] address second round of SSH review feedback

* [build-tools] send a discriminated target on the ssh session mutation

* [build-tools] sort ssh test imports
2026-08-26 08:09:27 -07:00
Expo CI 3f74ea369d v22.5.0 2026-08-26 14:49:23 +00:00
Szymon DziedzicandClaude Fable 5 28f80da648 [eas-cli] Add --max-idle-time-minutes flag to eas simulator (#4156)
Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-26 16:39:07 +02:00
Szymon Dziedzic a7260dfed3 [eas-cli] Bump CLI environment versions to 22.4.0 (#4286) 2026-08-26 14:30:58 +02:00
Stanisław Chmiela 5cd1b5eee3 [build-tools] Preserve Argent artifact kinds (#4280)
# Why

Argent now exposes artifact kind!!

# How

Extended the schema. Kept mime-types though I'd be more than happy to remove it (mime types can lie).

# Test Plan

CI should pass.
2026-08-26 13:46:20 +02:00
Stanisław Chmiela 9d627c6917 [build-tools] Humanize Appium command summaries (#4227)
# Why

We currently present Appium session events as HTTP methods which is not too human-readable (`getWindowRect`).

# How

Added a GHA workflow which runs weekly and submits a pull request adding to an `AppiumCommand` type so TSC fails and we need to add descriptions to commands.

# Test Plan

CI should pass.
2026-08-26 13:46:07 +02:00
Szymon Dziedzic 66dbcaf432 [build-tools] Bypass simulator URL confirmation (#4274)
* [build-tools] bypass simulator URL confirmation

* [build-tools] add URL confirmation fix changelog

* [build-tools] make URL preapproval best effort
2026-08-25 00:49:48 +02:00
Expo CI 39c39ad4bf v22.4.0 2026-08-24 21:35:36 +00:00
Szymon Dziedzic 1ad3822421 [eas-cli] Add simulator launch options (#4264)
* [eas-cli] add simulator launch options

* [eas-cli] add simulator launch options changelog

* [eas-cli] resolve simulator Expo Go archives server-side

* [eas-cli] rename simulator open URL flag

* [eas-cli] rename simulator open URL GraphQL field
2026-08-24 23:33:13 +02:00
Szymon Dziedzic 0929787398 [build-tools] Support simulator application launch options (#4263)
* [build-tools] support simulator application launch options

* [build-tools] add simulator launch options changelog

* [build-tools] rename launch application open URL input

* [build-tools] clarify simulator launch arguments
2026-08-24 23:19:30 +02:00
Szymon Dziedzic 756016a433 [build-tools] Support flat app bundle archives (#4262)
* [build-tools] Support flat app bundle archives

* [build-tools] Add flat app archive changelog
2026-08-24 23:07:07 +02:00
Jon Samp c6e3c332ac [eas-cli] Add eas billing:subscribe and billing:manage commands (#4007) 2026-08-24 14:44:44 -05:00
Stanisław Chmiela 7c95a6b4b2 [build-tools] Restore maestro-runner WDA cache (#4195)
## Why

`maestro-runner` needs WDA driver to run tests. By default it compiles it on first run which takes a minute.

## How

Compiled WDA on all images, uploaded to `turtle-v2` GCS and added optimistic download to `eas/install_maestro`. See https://github.com/expo/turtle-v2/pull/2603.

## Test plan

https://staging.expo.dev/accounts/expo-services/projects/workflows-testing/workflows/01a0009e-b171-7497-9335-af1fc470a8d5
2026-08-24 21:32:24 +02:00
Szymon Świerk 16d234b11d [steps] Report a broken composite if: on the call, not an inner step (#4216)
* [steps] Report a broken composite if: on the call, not an inner step

* [steps] Simplify condition evaluation error handling by building the erro message in the error constructor
2026-08-24 18:52:39 +02:00
Expo CI 70066fc8be v22.3.0 2026-08-24 14:45:57 +00:00
Szymon Dziedzic d27691adef [eas-cli] Add simulator build source flags (#4223)
* [eas-cli] Add simulator build source flags

* [eas-cli] align simulator application archive input

* [eas-cli] add simulator build flags changelog

* [eas-cli] add Expo Go simulator source

* [eas-cli] rename Expo Go simulator flag

* [eas-cli] rename application archive URL flag

* [eas-cli] parse Expo Go download output
2026-08-24 16:42:15 +02:00
Stanisław Chmiela 97d64f4054 [build-tools] Separate iOS ccache by destination (#4238)
# Why

Looks like ccache cache built for iOS simulator is not helpful for iOS device builds and vice versa.

Context: https://exponent-internal.slack.com/archives/C08PBDW7ZC3/p1787242902697509?thread_ts=1787171142.251299&cid=C08PBDW7ZC3

# How

Separated `ios-device-ccache-` and `ios-simulator-ccache-` ccaches.

# Test Plan

- Before: the [simulator build](https://staging.expo.dev/accounts/expo-services/projects/worker-system-tests/builds/5b9eccae-58a7-4a20-919e-61fbbd8ab68d) restored the [device build’s](https://staging.expo.dev/accounts/expo-services/projects/worker-system-tests/builds/026d0b35-0a6d-418d-bf34-488ed68f7d19) cache, got 0 hits, and could not save its own cache because the key already existed.
- After: the [device](https://staging.expo.dev/accounts/expo-services/projects/worker-system-tests/builds/02279218-8450-4679-b377-aec998dff0dd) and [simulator](https://staging.expo.dev/accounts/expo-services/projects/worker-system-tests/builds/0e6892fd-a9ee-4a90-ba23-97c7a17ad1e9) builds used separate keys. The simulator build got 48 direct hits.
2026-08-24 15:59:02 +02:00
Szymon Dziedzic 1fe8f24313 [build-tools] Install builds in simulator sessions (#4222)
* [build-tools] Install builds in simulator sessions

* [build-tools] split simulator app setup functions

* [build-tools] add simulator app setup changelog

* [build-tools] decouple application launch from artifact

* [build-tools] simplify build source validation

* [build-tools] enforce build source input type

* [build-tools] address simulator install review feedback
2026-08-24 15:08:14 +02:00
Gabe Debes 790231ea18 [build-tools] Support pinning the serve-sim version in a simulator session (#4253) 2026-08-21 17:18:48 -07:00
Gabriel Donadel Dall'Agnol 1f24d00ee9 [build-tools] Add ios_signing_backend option to the repack step (#4239)
* [build-tools] Add ios_signing_backend option to the repack step

* Update CHANGELOG.md

* split out a dedicated resolver for resolveIosSigningOptionsAsync
2026-08-21 19:58:25 +02:00
Douglas LowderandClaude Opus 4.8 3f8ffac11c [build-tools] Submit IPAs to the App Store Connect platform from the bundle (#4234)
eas submit uploaded every IPA to the iOS train because the build upload
hardcoded platform 'IOS', so tvOS binaries were validated against iOS rules
and rejected (ITMS-90508/90545/90713/90039), and their build numbers collided
with the iOS train.

Read DTPlatformName from the IPA in upload_to_asc, map it to the App Store
Connect platform (appletvos -> TV_OS, macosx -> MAC_OS, xros -> VISION_OS,
otherwise IOS), and pass it to createBuildUploadAsync and the TestFlight URL.
This also makes the duplicate-build-number precheck run against the correct
platform train. Falls back to IOS when the platform cannot be read.

Co-authored-by: Claude Opus 4.8 <[email protected]>
2026-08-20 15:04:47 -07:00
Expo CI 137bee4db0 update CHANGELOG.md and bump STAGING in cli-versions.json 2026-08-20 16:22:16 +00:00
379 changed files with 46943 additions and 6940 deletions

No files matched your search

+3 -1
View File
@@ -31,7 +31,9 @@ jobs:
script: |
core.setFailed('Please add a changelog entry!')
- uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2.8.2
if: always()
if: >-
always() &&
(github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository)
with:
message-id: changelog-entry-check
refresh-message-position: true
+44 -19
View File
@@ -37,20 +37,6 @@ jobs:
run: yarn build
- name: Publish packages to npm
run: yarn run -T lerna publish from-package --yes --no-verify-access
- name: Verify eas-cli was published as latest on npm
run: |
expected="$(jq -r .version lerna.json)"
for attempt in $(seq 1 10); do
published="$(npm view eas-cli@latest version 2>/dev/null || true)"
if [ "$published" = "$expected" ]; then
echo "Verified eas-cli@$expected is latest on npm."
exit 0
fi
echo "Attempt $attempt: npm latest is '$published', expected '$expected'. Retrying in 15s..."
sleep 15
done
echo "eas-cli@$expected was not published as latest on npm; the publish step failed." >&2
exit 1
release-changelog:
name: Update changelog and publish release
runs-on: ubuntu-latest
@@ -67,6 +53,10 @@ jobs:
run: yarn install --immutable
- name: Resolve version
run: echo "EAS_CLI_VERSION=${GITHUB_REF#refs/tags/v}" >> "$GITHUB_ENV"
- name: Configure git
run: |
git config user.email "[email protected]"
git config user.name "Expo CI"
- name: Update changelog
working-directory: ./scripts
run: yarn release-changelog "$EAS_CLI_VERSION" > /tmp/current-changelog
@@ -74,17 +64,52 @@ jobs:
run: gh release edit "v$EAS_CLI_VERSION" --draft=false -F /tmp/current-changelog
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Commit and push
- name: Commit and push changelog
run: |
git fetch
git checkout main
git add CHANGELOG.md
git commit --allow-empty -m "update CHANGELOG.md for v$EAS_CLI_VERSION"
git push
- name: Verify eas-cli is available on npm
run: |
availability_deadline_epoch=$(( $(date +%s) + 600 ))
while true; do
published="$(npm view "eas-cli@$EAS_CLI_VERSION" version --prefer-online 2>/dev/null || true)"
if [ "$published" = "$EAS_CLI_VERSION" ]; then
echo "Verified eas-cli@$EAS_CLI_VERSION is available on npm."
exit 0
fi
current_epoch=$(date +%s)
if [ "$current_epoch" -ge "$availability_deadline_epoch" ]; then
break
fi
retry_delay=$((availability_deadline_epoch - current_epoch))
if [ "$retry_delay" -gt 10 ]; then
retry_delay=10
fi
echo "eas-cli@$EAS_CLI_VERSION is not available on npm yet. Retrying in ${retry_delay}s."
sleep "$retry_delay"
done
echo "eas-cli@$EAS_CLI_VERSION was not available on npm after 10 minutes." >&2
exit 1
- name: Bump STAGING and PRODUCTION
run: |
git pull --rebase --autostash origin main
jq --arg version "$EAS_CLI_VERSION" '.STAGING = $version' \
cli-versions.json > cli-versions.json.tmp
mv cli-versions.json.tmp cli-versions.json
git config --global user.email "[email protected]"
git config --global user.name "Expo CI"
git add cli-versions.json CHANGELOG.md
git commit --allow-empty -m "update CHANGELOG.md and bump STAGING in cli-versions.json"
jq --arg version "$EAS_CLI_VERSION" '.PRODUCTION = $version' \
cli-versions.json > cli-versions.json.tmp
mv cli-versions.json.tmp cli-versions.json
- name: Commit and push version update
run: |
git add cli-versions.json
git commit --allow-empty -m "bump STAGING and PRODUCTION to $EAS_CLI_VERSION in cli-versions.json"
git push
- name: Prepare changelog for Slack
run: |
+31
View File
@@ -0,0 +1,31 @@
name: test-egress-guard
on:
workflow_dispatch: {}
pull_request:
paths:
- packages/build-tools/resources/egress-guard/**
- packages/build-tools/src/steps/utils/localEgressGuard.ts
- packages/build-tools/src/steps/utils/__tests__/localEgressGuard*.ts
- packages/build-tools/src/steps/functions/startIosSimulator.ts
- packages/worker/.eas/workflows/test-egress-guard.yml
- .github/workflows/test-egress-guard.yml
jobs:
test-egress-guard:
# The simulator tests need macOS with Xcode, which the EAS macOS workers have.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4
- uses: ./.github/actions/setup-mise
- name: Install eas-cli
run: npm install --global eas-cli@latest
- name: Run the test-egress-guard EAS workflow on macOS
working-directory: packages/worker
env:
EXPO_TOKEN: ${{ secrets.EXPO_DEV_EXPO_GITHUB_ROBOT_ACCESS_TOKEN }}
run: |
eas workflow:run test-egress-guard.yml \
--ref ${{ github.event.pull_request.head.sha || github.sha }} \
--non-interactive \
--wait
+2 -2
View File
@@ -15,9 +15,9 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
node: ['20', '22', '24']
node: ['20.18.3', '22', '24']
include:
- node: '20'
- node: '20.18.3'
coverage: true
name: Test with Node ${{ matrix.node }}
steps:
@@ -0,0 +1,90 @@
name: Update Appium command list
on:
schedule:
- cron: '0 6 * * 1' # Every Monday at 06:00 UTC
push:
branches:
- main
paths:
- .github/workflows/update-appium-commands.yml
workflow_dispatch:
permissions:
contents: read
jobs:
generate:
runs-on: ubuntu-latest
name: Regenerate Appium command list
outputs:
changed: ${{ steps.diff.outputs.changed }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- name: Setup tools
uses: ./.github/actions/setup-mise
- name: Install dependencies
run: yarn install --immutable
- name: Regenerate the Appium command list
# The generator installs Appium, XCUITest, and UiAutomator2 on demand,
# so it always picks up the latest supported command sets.
run: yarn workspace @expo/build-tools generate-appium-commands
- name: Check whether the command list changed
id: diff
run: |
if git diff --quiet -- packages/build-tools/src/steps/utils/appiumCommands.generated.ts; then
echo 'changed=false' >> "$GITHUB_OUTPUT"
else
echo 'changed=true' >> "$GITHUB_OUTPUT"
fi
- name: Upload the generated command list
if: steps.diff.outputs.changed == 'true'
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: appium-command-list
path: packages/build-tools/src/steps/utils/appiumCommands.generated.ts
if-no-files-found: error
retention-days: 1
publish:
needs: generate
if: needs.generate.outputs.changed == 'true'
runs-on: ubuntu-latest
name: Publish Appium command list update
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- name: Download the generated command list
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: appium-command-list
path: packages/build-tools/src/steps/utils
- name: Open a pull request
env:
GH_TOKEN: ${{ secrets.EXPO_BOT_PAT }}
run: |
branch=bot/update-appium-commands
title='[build-tools] Update Appium command list'
body=$'Automated update of the Appium, XCUITest, and UiAutomator2 command list.\n\nIf this PR adds new commands, typechecking will fail until a curated summary is added for each of them in `packages/build-tools/src/steps/utils/appiumCommandSummary.ts`.'
gh auth setup-git
if git ls-remote --exit-code --heads origin "$branch" >/dev/null; then
git fetch origin "refs/heads/$branch:refs/remotes/origin/$branch"
fi
git config user.name 'Expo Bot'
git config user.email [email protected]
git checkout -B "$branch"
git add packages/build-tools/src/steps/utils/appiumCommands.generated.ts
git commit -m "$title"
git push --force-with-lease origin "$branch"
if gh pr view "$branch" --json state --jq '.state' 2>/dev/null | grep -q OPEN; then
gh pr edit "$branch" --title "$title" --body "$body" --add-reviewer sjchmiela
else
gh pr create --base main --head "$branch" --title "$title" --body "$body" \
--label 'no changelog' --reviewer sjchmiela
fi
+204
View File
@@ -8,6 +8,210 @@ This is the log of notable changes to EAS CLI and related packages.
### 🎉 New features
### 🐛 Bug fixes
- [eas-cli] Support Xcode 27, which replaces Simulator.app with Device Hub, in `build:dev`, `build:run`, and `run` for iOS simulator builds. ([#4405](https://github.com/expo/eas-cli/pull/4405) by [@brentvatne](https://github.com/brentvatne))
### 🧹 Chores
## [24.4.1](https://github.com/expo/eas-cli/releases/tag/v24.4.1) - 2026-09-15
### 🧹 Chores
- [eas-cli] Bump `@expo/apple-utils` to `2.2.1`. ([#4404](https://github.com/expo/eas-cli/pull/4404) by [@brentvatne](https://github.com/brentvatne))
## [24.4.0](https://github.com/expo/eas-cli/releases/tag/v24.4.0) - 2026-09-14
### 🛠 Breaking changes
- [eas-cli] Require Node `^20.18.3 || >=22.0.0`. ([#4279](https://github.com/expo/eas-cli/pull/4279) by [@ramonclaudio](https://github.com/ramonclaudio))
### 🎉 New features
- [build-tools] Inject a guard into every iOS Simulator process during `--egress local` sessions that refuses connections which bypass the system proxy and reports them, with the calling frameworks, in the session log. ([#4393](https://github.com/expo/eas-cli/pull/4393) by [@brentvatne](https://github.com/brentvatne))
- [build-tools] Set proxy environment variables inside the iOS Simulator for `--egress local` sessions, so clients that read them (gRPC, libcurl) also exit from the egress client. ([#4390](https://github.com/expo/eas-cli/pull/4390) by [@brentvatne](https://github.com/brentvatne))
### 🐛 Bug fixes
- [build-tools] Skip project source refresh for jobs with no sources. ([#4389](https://github.com/expo/eas-cli/pull/4389) by [@sjchmiela](https://github.com/sjchmiela))
## [24.3.0](https://github.com/expo/eas-cli/releases/tag/v24.3.0) - 2026-09-11
### 🎉 New features
- [eas-build-job] Add a shared protocol for sandbox daemon commands. ([#4346](https://github.com/expo/eas-cli/pull/4346) by [@sjchmiela](https://github.com/sjchmiela))
## [24.2.0](https://github.com/expo/eas-cli/releases/tag/v24.2.0) - 2026-09-11
### 🎉 New features
- [eas-build-job] Remove repository URLs from Git job inputs and fetch them when resolving project sources. ([#4360](https://github.com/expo/eas-cli/pull/4360) by [@sjchmiela](https://github.com/sjchmiela))
- [eas-cli] Add `eas workflow:insights` to display Workflows Insights for a time range: run counts and success rate compared with the previous period, runs over time, and a per-workflow breakdown, with `--json` output for scripts. ([#4367](https://github.com/expo/eas-cli/pull/4367) by [@hSATAC](https://github.com/hSATAC))
- [eas-cli] Add `eas workflow:insights:maestro` to display Maestro test insights for a time range: pass and flake rates, a sortable per-flow table, and a single flow's history with error patterns and recent runs, with `--json` output for scripts. ([#4370](https://github.com/expo/eas-cli/pull/4370) by [@hSATAC](https://github.com/hSATAC))
### 🐛 Bug fixes
- [eas-cli] Reject `--end` without `--start` in the `observe:*`, `update:insights`, `channel:insights`, and `update:view --insights` commands instead of silently falling back to the default time range. ([#4367](https://github.com/expo/eas-cli/pull/4367) by [@hSATAC](https://github.com/hSATAC))
- [eas-cli] Reject a time range that does not run forwards in the `observe:*`, `update:insights`, `channel:insights`, `update:view --insights`, and `workflow:insights` commands instead of sending it to the server. ([#4367](https://github.com/expo/eas-cli/pull/4367) by [@hSATAC](https://github.com/hSATAC))
- [eas-cli] Reject non-integer values for `--limit` and `--offset` instead of sending them to the server. ([#4367](https://github.com/expo/eas-cli/pull/4367) by [@hSATAC](https://github.com/hSATAC))
### 🧹 Chores
- [build-tools] Require PATH project sources for local builds. ([#4362](https://github.com/expo/eas-cli/pull/4362) by [@sjchmiela](https://github.com/sjchmiela))
## [24.1.2](https://github.com/expo/eas-cli/releases/tag/v24.1.2) - 2026-09-11
### 🐛 Bug fixes
- [eas-cli] Shorten the local egress banner printed by `eas simulator:start` and stop mentioning `eas simulator:egress` in interactive mode, where the client runs in the same terminal. ([#4383](https://github.com/expo/eas-cli/pull/4383) by [@brentvatne](https://github.com/brentvatne))
## [24.1.1](https://github.com/expo/eas-cli/releases/tag/v24.1.1) - 2026-09-11
### 🐛 Bug fixes
- [eas-cli] Stop telling interactive `eas simulator:start --egress local` users to run `eas simulator:egress` when the egress client already runs in the same terminal. ([#4382](https://github.com/expo/eas-cli/pull/4382) by [@brentvatne](https://github.com/brentvatne))
## [24.1.0](https://github.com/expo/eas-cli/releases/tag/v24.1.0) - 2026-09-10
### 🎉 New features
- [eas-cli] Add `--egress-allow <host:port>` to `eas simulator:start` so local egress sessions can reach dev servers on this machine or its network. ([#4369](https://github.com/expo/eas-cli/pull/4369) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli][build-tools] Forward `--egress-allow localhost:<port>` and `127.0.0.1:<port>` entries from the simulator host's loopback interface to this machine (like `adb reverse`), so dev server URLs that use `127.0.0.1` work through local egress. ([#4369](https://github.com/expo/eas-cli/pull/4369) by [@brentvatne](https://github.com/brentvatne))
- [build-tools] Add the `eas/start_local_egress` step for iOS simulator sessions with local egress: the device host's system proxy points at a reverse tunnel to the EAS CLI machine, so HTTP(S) requests that honor it exit from that machine and third parties see its IP. ([#4364](https://github.com/expo/eas-cli/pull/4364) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli] Add `--egress local` to `eas simulator:start` and the `eas simulator:egress` command, which run the local egress client for the session. ([#4364](https://github.com/expo/eas-cli/pull/4364) by [@brentvatne](https://github.com/brentvatne))
- [build-tools] Require the serve-sim session token for device run session previews, so the tunnel no longer exposes the shell-exec route. ([#4306](https://github.com/expo/eas-cli/pull/4306) by [@gwdp](https://github.com/gwdp))
### 🐛 Bug fixes
- [build-tools] Report a system error when project source refresh fails for a remote build. ([#4359](https://github.com/expo/eas-cli/pull/4359) by [@sjchmiela](https://github.com/sjchmiela))
## [24.0.0](https://github.com/expo/eas-cli/releases/tag/v24.0.0) - 2026-09-09
### 🛠 Breaking changes
- [eas-cli] Align the `eas observe:*` commands with the renamed Observe GraphQL schema. `--json` output now uses `name`/`value` instead of `metricName`/`metricValue` (metrics) and `name` instead of `eventName` (log events and event-name summaries), and `eas observe:events` now lists user-defined events only — exceptions moved to the new `eas observe:errors`. ([#4338](https://github.com/expo/eas-cli/pull/4338) by [@douglowder](https://github.com/douglowder))
### 🎉 New features
- [build-tools] Add `lcd_width`, `lcd_height`, and `lcd_density` inputs to configure the Android emulator display. ([#4349](https://github.com/expo/eas-cli/pull/4349) by [@krystofwoldrich-agent](https://github.com/krystofwoldrich-agent))
- [eas-cli] Add `eas channel:protect` and `eas channel:unprotect` to manage EAS Update channel protection, and show protection in channel list and view output. ([#4319](https://github.com/expo/eas-cli/pull/4319) by [@sjkim-expo](https://github.com/sjkim-expo))
- [eas-cli] Add `eas observe:errors` to display error and exception issue groups (grouped by fingerprint), with `--fingerprint <fingerprint>` to drill into a group's individual occurrences and stack traces. ([#4339](https://github.com/expo/eas-cli/pull/4339) by [@douglowder](https://github.com/douglowder))
- [eas-cli] Add `eas observe:event` to display a single Observe event (metric, log, or error) by its ID. ([#4252](https://github.com/expo/eas-cli/pull/4252) by [@douglowder](https://github.com/douglowder))
### 🐛 Bug fixes
- [build-tools] Start Argent remote sessions with `bun x` so they work when the `bunx` symlink is missing. ([491b372a](https://github.com/expo/eas-cli/commit/491b372a) by [@krystofwoldrich](https://github.com/krystofwoldrich))
- [build-tools] Reduce `expo-device-hub` preview resolution from 1280 px to 960 px to match `serve-sim` and lower streaming bandwidth. ([#4326](https://github.com/expo/eas-cli/pull/4326) by [@krystofwoldrich-agent](https://github.com/krystofwoldrich-agent))
- [build-tools] Install FFmpeg before launching `expo-device-hub` in Android device sessions. ([#4332](https://github.com/expo/eas-cli/pull/4332) by [@krystofwoldrich-agent](https://github.com/krystofwoldrich-agent))
- [eas-cli] Print the session token in simulator preview links, so they still open once the preview is gated. ([#4312](https://github.com/expo/eas-cli/pull/4312) by [@gwdp](https://github.com/gwdp))
### 🧹 Chores
- [eas-cli] Make ASC API key issuer identifier optional in GraphQL and credentials types. ([#4248](https://github.com/expo/eas-cli/pull/4248) by [@sswrk](https://github.com/sswrk))
## [23.2.0](https://github.com/expo/eas-cli/releases/tag/v23.2.0) - 2026-08-31
### 🎉 New features
- [eas-cli] Add `--tag` to `eas simulator` and `eas simulator:list` to group and filter simulator sessions by app variant. ([#4318](https://github.com/expo/eas-cli/pull/4318) by [@tchayen](https://github.com/tchayen))
- [eas-cli] Update workflow run logs in real time, instead of every 10 seconds. ([#4228](https://github.com/expo/eas-cli/pull/4228) by [@AHGIJMKLKKZNPJKQR](https://github.com/AHGIJMKLKKZNPJKQR))
- [build-tools] Add `expo-device-hub` web previews to Android web-preview-only, Agent Device, Argent, and Appium remote sessions. ([#4304](https://github.com/expo/eas-cli/pull/4304) by [@krystofwoldrich](https://github.com/krystofwoldrich))
- [eas-cli] Add `eas workflow:ssh <workflow-job-id> [command...]` to open an ssh session on the worker running a workflow job. ([#4032](https://github.com/expo/eas-cli/pull/4032) by [@gwdp](https://github.com/gwdp))
- [eas-cli] Add `--ssh` (and `--ssh-idle-timeout`) to `eas workflow:run` to enable ssh on the run's VM jobs. ([#4033](https://github.com/expo/eas-cli/pull/4033) by [@gwdp](https://github.com/gwdp))
### 🧹 Chores
- [eas-cli] Warn when starting Android EAS Simulator sessions that support is still in development and iOS parity is coming soon. ([#4320](https://github.com/expo/eas-cli/pull/4320) by [@szdziedzic](https://github.com/szdziedzic))
## [23.1.0](https://github.com/expo/eas-cli/releases/tag/v23.1.0) - 2026-08-31
### 🎉 New features
- [eas-cli] Prompt for the Expo SDK version in `eas new` and add a `--sdk-version` flag. The template now downloads from npm instead of GitHub to support SDK-specific versions. ([#4301](https://github.com/expo/eas-cli/pull/4301) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli] Clean up template metadata in projects created with `eas new`: remove the template LICENSE file and reset the package.json name, version, and license fields. ([#4301](https://github.com/expo/eas-cli/pull/4301) by [@brentvatne](https://github.com/brentvatne))
- [build-tools] Support `EAS_PNPM_FILTER_WORKSPACE` to install only the selected workspace's dependencies with `pnpm install --filter`. ([#4302](https://github.com/expo/eas-cli/pull/4302) by [@robingullo](https://github.com/robingullo))
### 🐛 Bug fixes
- [build-tools] Normalize extracted project permissions so source archives cannot leave files unreadable by EAS workers. ([#4287](https://github.com/expo/eas-cli/pull/4287) by [@AbbanMustafa](https://github.com/AbbanMustafa))
- [eas-cli] Add dynamic retries to deploy command that scale up more generously, depending on deployment size ([#4299](https://github.com/expo/eas-cli/pull/4299) by [@kitten](https://github.com/kitten))
### 🧹 Chores
- [eas-cli] Add regression coverage for the bundled iOS entitlement introspection fallback. ([#4285](https://github.com/expo/eas-cli/pull/4285) by [@brentvatne](https://github.com/brentvatne))
- [local-build-plugin] Replace `bunyan` with `@expo/bunyan` while preserving the Steps formatter binary. ([#4284](https://github.com/expo/eas-cli/pull/4284) by [@brentvatne](https://github.com/brentvatne))
- [steps] Replace deprecated `lodash.get` with a local helper. ([#4283](https://github.com/expo/eas-cli/pull/4283) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli][steps] Bump `uuid` to v11 and remove the redundant `@types/uuid` dependency. ([#4282](https://github.com/expo/eas-cli/pull/4282) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli] Bump `minizlib` to 3.1.0 to drop its deprecated `rimraf@5` transitive dependency. ([#4281](https://github.com/expo/eas-cli/pull/4281) by [@brentvatne](https://github.com/brentvatne))
## [23.0.0](https://github.com/expo/eas-cli/releases/tag/v23.0.0) - 2026-08-28
### 🛠 Breaking changes
- [eas-cli] Rename the EAS Simulator preview-only session type from `--type serve-sim` to `--type web-preview-only`. ([#4298](https://github.com/expo/eas-cli/pull/4298) by [@szdziedzic](https://github.com/szdziedzic))
### 🎉 New features
- [build-tools] Cache CocoaPods dependencies between iOS builds. ([#4266](https://github.com/expo/eas-cli/pull/4266) by [@AbbanMustafa](https://github.com/AbbanMustafa))
- [build-tools] Support `EAS_BUN_FILTER_WORKSPACE` to install only the selected workspace's dependencies with `bun install --filter`. ([#4292](https://github.com/expo/eas-cli/pull/4292) by [@konrad-armatys](https://github.com/konrad-armatys))
- [eas-cli] Add `eas project:status` (alias `eas status`) command that prints a snapshot of the project's recent builds, development builds, workflow runs, submissions, and updates, with agent-friendly JSON metadata, activity links, structured errors, submission-to-build correlation, and `--non-interactive` support. ([#4006](https://github.com/expo/eas-cli/pull/4006) by [@jonsamp](https://github.com/jonsamp))
### 🐛 Bug fixes
- [build-tools] Prevent user npm minimum release age settings from applying to internal EAS CLI commands. ([#4296](https://github.com/expo/eas-cli/pull/4296) by [@sjchmiela](https://github.com/sjchmiela))
- [build-tools] Reduce serve-sim preview resolution from 1280 px to 960 px to lower streaming bandwidth. ([#4297](https://github.com/expo/eas-cli/pull/4297) by [@szdziedzic](https://github.com/szdziedzic))
### 🧹 Chores
- [worker] Use turtle cache upload and download URLs for eas.json build cache restore and save. ([#3860](https://github.com/expo/eas-cli/pull/3860) by [@sjchmiela](https://github.com/sjchmiela))
- [ci] Bump both STAGING and PRODUCTION cli-versions in release workflow. ([#4290](https://github.com/expo/eas-cli/pull/4290) by [@douglowder](https://github.com/douglowder))
## [22.6.0](https://github.com/expo/eas-cli/releases/tag/v22.6.0) - 2026-08-26
### 🎉 New features
- [eas-cli] Add an opt-in `--max-idle-time-minutes` flag to `eas simulator`; the session stops automatically after that many minutes without activity. ([#4156](https://github.com/expo/eas-cli/pull/4156) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] Add experimental `--resource-class` flag to `eas simulator`. ([#4268](https://github.com/expo/eas-cli/pull/4268) by [@gwdp](https://github.com/gwdp))
- [build-tools] Add worker-side SSH session helpers (upterm relay + session create/report/close). ([#4030](https://github.com/expo/eas-cli/pull/4030) by [@gwdp](https://github.com/gwdp))
- [worker] Wire an SSH session into the build lifecycle behind the workflow ssh flag. ([#4031](https://github.com/expo/eas-cli/pull/4031) by [@gwdp](https://github.com/gwdp))
### 🐛 Bug fixes
- [build-tools] Increase the serve-sim video bitrate to 6 Mbps for higher-quality simulator previews. ([#4291](https://github.com/expo/eas-cli/pull/4291) by [@szdziedzic](https://github.com/szdziedzic))
- [build-tools] Preapprove custom URL schemes before opening them in iOS Simulator sessions to avoid the first-use confirmation prompt. ([#4274](https://github.com/expo/eas-cli/pull/4274) by [@szdziedzic](https://github.com/szdziedzic))
## [22.4.0](https://github.com/expo/eas-cli/releases/tag/v22.4.0) - 2026-08-24
### 🎉 New features
- [eas-cli] Add `--launch-arg`, `--open-url`, and Expo Go `--sdk-version` options to `eas simulator`. ([#4264](https://github.com/expo/eas-cli/pull/4264) by [@szdziedzic](https://github.com/szdziedzic))
- [build-tools] Pass launch arguments and a URL to open when launching applications in simulator sessions. ([#4263](https://github.com/expo/eas-cli/pull/4263) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] Add `eas billing:subscribe <plan>` to start a Stripe checkout for an EAS plan, and `eas billing:manage` to open the Stripe customer portal. ([#4007](https://github.com/expo/eas-cli/pull/4007) by [@jonsamp](https://github.com/jonsamp))
### 🐛 Bug fixes
- [build-tools] Support downloading iOS simulator archives whose app bundle contents are stored at the archive root. ([#4262](https://github.com/expo/eas-cli/pull/4262) by [@szdziedzic](https://github.com/szdziedzic))
## [22.3.0](https://github.com/expo/eas-cli/releases/tag/v22.3.0) - 2026-08-24
### 🎉 New features
- [build-tools] Add `ios_signing_backend` option to the repack step. ([#4239](https://github.com/expo/eas-cli/pull/4239) by [@gabrieldonadel](https://github.com/gabrieldonadel))
- [build-tools] Support an optional `package_version` input on `eas/start_serve_sim_remote_session`, so a simulator session can pin the `@expo/serve-sim` version instead of always running `latest`. ([#4253](https://github.com/expo/eas-cli/pull/4253) by [@gwdp](https://github.com/gwdp))
- [build-tools] Add composable custom build functions for downloading, installing, and launching application archives in simulator sessions. ([#4222](https://github.com/expo/eas-cli/pull/4222) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] Add `--build-id`, `--application-archive-url`, and `--expo-go` to `eas simulator` to install and launch an application before the session is ready. ([#4223](https://github.com/expo/eas-cli/pull/4223) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] Add `--environment` flag to the `eas observe:*` commands. ([#4275](https://github.com/expo/eas-cli/pull/4275) by [@kadikraman](https://github.com/kadikraman))
- [eas-cli] Add `apple` value to the `--platform` flag of the `eas observe:*` commands. ([#4276](https://github.com/expo/eas-cli/pull/4276) by [@kadikraman](https://github.com/kadikraman))
- [eas-cli] Add `--build-number` flag to `eas observe:metrics` and `eas observe:events`. ([#4278](https://github.com/expo/eas-cli/pull/4278) by [@kadikraman](https://github.com/kadikraman))
### 🐛 Bug fixes
- [build-tools] Upload tvOS (and other non-iOS) IPAs to the App Store Connect platform declared in the bundle's `DTPlatformName`, instead of always uploading to the iOS train. ([#4234](https://github.com/expo/eas-cli/pull/4234) by [@douglowder](https://github.com/douglowder))
## [22.2.0](https://github.com/expo/eas-cli/releases/tag/v22.2.0) - 2026-08-20
### 🎉 New features
- [eas-cli] Add `--device` flag to `eas simulator` for selecting the virtual device to start. ([#4172](https://github.com/expo/eas-cli/pull/4172) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] Add support for Appium EAS Simulator sessions. ([#4190](https://github.com/expo/eas-cli/pull/4190) by [@sjchmiela](https://github.com/sjchmiela))
+2 -2
View File
@@ -1,4 +1,4 @@
{
"STAGING": "22.0.0",
"PRODUCTION": "21.5.0"
"STAGING": "24.4.1",
"PRODUCTION": "24.4.1"
}
+1 -1
View File
@@ -1,5 +1,5 @@
{
"npmClient": "yarn",
"version": "22.2.0",
"version": "24.4.2",
"$schema": "node_modules/lerna/schemas/lerna-schema.json"
}
+2
View File
@@ -8,3 +8,5 @@ bin/record-sim
src/graphql-env.d.ts
schema.graphql
bin/egress-guard.dylib
bin/egress-guard-check
+15 -9
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/build-tools",
"version": "22.2.0",
"version": "24.4.0",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "BUSL-1.1",
"author": "Expo <[email protected]>",
@@ -24,7 +24,10 @@
"prebuild": "yarn gql",
"build": "tsc",
"build:record-sim": "mkdir -p bin && record_sim_bin_path=$(swift build -c release --package-path resources/record-sim --build-path resources/record-sim/.build --show-bin-path) && swift build -c release --package-path resources/record-sim --build-path resources/record-sim/.build && cp \"$record_sim_bin_path/record-sim\" bin/record-sim && chmod +x bin/record-sim",
"build:egress-guard": "resources/egress-guard/build.sh",
"test:egress-guard": "resources/egress-guard/tests/run-policy-tests.sh && resources/egress-guard/tests/run-guard-tests.sh",
"typecheck": "tsc",
"generate-appium-commands": "mise exec [email protected] -- node scripts/generate-appium-commands.js",
"prepack": "rimraf dist \"*.tsbuildinfo\" && yarn gql && tsc -p tsconfig.build.json",
"jest-unit": "jest --config jest/unit-config.ts",
"jest-integration": "jest --config jest/integration-config.ts",
@@ -38,22 +41,23 @@
"dependencies": {
"@expo/config": "55.0.10",
"@expo/config-plugins": "55.0.7",
"@expo/downloader": "22.0.0",
"@expo/eas-build-job": "22.0.0",
"@expo/downloader": "24.0.0",
"@expo/eas-build-job": "24.3.0",
"@expo/env": "^0.4.0",
"@expo/logger": "22.0.0",
"@expo/logger": "24.0.0",
"@expo/package-manager": "1.9.10",
"@expo/plist": "^0.3.5",
"@expo/results": "^1.0.0",
"@expo/spawn-async": "1.7.2",
"@expo/steps": "22.1.0",
"@expo/template-file": "22.0.0",
"@expo/turtle-spawn": "22.0.0",
"@expo/steps": "24.3.0",
"@expo/template-file": "24.0.0",
"@expo/turtle-spawn": "24.0.0",
"@expo/xcpretty": "^4.3.1",
"@ngrok/ngrok": "1.7.0",
"@sentry/node": "7.77.0",
"@urql/core": "^6.0.1",
"bplist-parser": "0.3.2",
"content-disposition": "1.0.1",
"fast-glob": "^3.3.2",
"fast-xml-parser": "^4.4.1",
"fs-extra": "^11.2.0",
@@ -73,11 +77,13 @@
"semver": "^7.6.2",
"tar": "7.5.19",
"uuid": "9.0.1",
"ws": "8.21.1",
"yaml": "^2.8.1",
"zod": "^4.3.5"
},
"devDependencies": {
"@expo/repack-app": "~0.6.1",
"@expo/repack-app": "~0.9.0",
"@types/content-disposition": "0.5.9",
"@types/fs-extra": "^11.0.4",
"@types/jest": "^29.5.12",
"@types/lodash": "^4.17.4",
@@ -89,7 +95,7 @@
"@types/retry": "^0.12.5",
"@types/semver": "^7.5.8",
"@types/uuid": "^9.0.8",
"bunyan": "^1.8.15",
"@types/ws": "8.5.10",
"jest": "^29.7.0",
"memfs": "^4.17.1",
"nock": "13.4.0",
@@ -0,0 +1,83 @@
# egress-guard
A small dylib injected into every process the iOS Simulator launches during a
`--egress local` session. It interposes the socket calls that open outbound
traffic (`connect`, `connectx`, `sendto`, `sendmsg`, and the `$NOCANCEL`
variants of the last three that libsystem exports alongside them) and refuses
any destination that is not loopback. A sockaddr whose family was left
`AF_UNSPEC` is classified as the family its length implies, which is how the
kernel treats it for TCP `connect` and IPv4 sends; on a datagram socket
`connect(AF_UNSPEC)` is left to the kernel, which dissolves the association. The system proxy and the `--egress-allow`
forwards live on loopback, so everything that honors the proxy keeps working
and everything that bypasses it fails with `ECONNREFUSED` in the process that
tried, with the calling frameworks recorded.
Why interposing: simulator processes share the worker's uid, so a packet
filter cannot tell them apart from the worker's own traffic. dyld interposing
from an inserted library reaches calls made inside Apple's own frameworks
(CFNetwork, Network.framework), and `launchctl setenv DYLD_INSERT_LIBRARIES`
inside the simulator makes launchd hand the library to every process it
spawns. The simulator does not enforce library validation, which is what makes
this possible there and nowhere else.
Configuration comes from the environment the simulator's launchd provides:
- `EAS_EGRESS_GUARD_LOG`: file the guard appends events to (host path).
- `EAS_EGRESS_GUARD_MODE`: `block` (default, and what any unknown value means)
or `log` (observe only).
Each event is one tab-separated line:
```
eas-egress-guard\t<process>\t<pid>\t<function>\t<blocked|logged>\t<peer>\t<caller>,<caller>...
```
A process lists each `(function, peer)` once, up to 128 of them. When that
table fills it writes one more line with function `overflow` and the limit as
the peer; further distinct destinations are still refused but not listed.
Layout:
- `policy.c` / `policy.h`: classification, mode, formatting, per-process
dedupe. Pure C, host-testable.
- `guard.c`: the interposers and the constructor that reads the environment.
- `check.c`: `egress-guard-check`, run inside the simulator right after the
guard is installed. Exits non-zero unless the library is loaded in a fresh
process and behaves as the mode says on `connect`, `connect$NOCANCEL` and a
`connect` with an `AF_UNSPEC` sockaddr; the worker fails the session on that.
- `tests/policy_test.c`: host unit tests, `tests/run-policy-tests.sh`.
- `tests/guard_test.c`, `tests/guard_insert_test.c`: host tests of the
interposers themselves, `tests/run-guard-tests.sh`. The first compiles
`guard.c` in and forces the lock schedules (a signal handler making a socket
call inside the critical section, a fork while another thread holds the lock,
contention); the second inserts the built library like the simulator does and
checks descriptor reuse, the `$NOCANCEL` symbols, `AF_UNSPEC` shapes and the
UDP disconnect. Nothing in them opens a socket to a non-loopback address.
- `tests/nettest.swift`: probe binary the simulator end-to-end test runs inside
a device; exercises URLSession, Network.framework, BSD TCP and UDP, DNS.
- `build.sh`: universal simulator dylib into `packages/build-tools/bin/`.
Installation order matters. `simctl` forwards every `SIMCTL_CHILD_`-prefixed
variable of its own environment to the process it starts, and for `simctl
boot` that process is the simulator's launchd, so the worker boots with the
guard and proxy variables in that form and every process of the boot inherits
them (measured: 176 of 176). `launchctl setenv` after boot is kept for a
device that was already booted, but by then the boot's own processes have
started without it. It also cannot change a variable the boot already carried:
a device booted with one guard configuration keeps it until it is shut down,
which is why the worker boots devices itself and every configuration in the
end-to-end test gets its own boot. The self-check runs once boot completes and is followed
by a coverage report from `lsof`, listing any process without the library. Both the dylib and the check binary are built by
`packages/worker/package.sh` for the iOS worker tarball and by the
`test-egress-guard` EAS workflow, not committed.
Failure semantics: a missing library, a failed `launchctl setenv`, or a failed
self-check fails the session, since a `--egress local` session without the
guard would silently leak. Only an unwritable event log is a warning, because
refusals still happen and only the reporting is lost. Recording is best-effort
in the process too: the event log is opened per event rather than held open
(a process that closes descriptors it does not own, as `launchd_sim` does,
would otherwise make the guard write into whatever reused the number), and a
dedupe slot that cannot be taken within a bounded number of attempts (a signal
handler re-entering the guard, a fork child inheriting a held lock) skips the
event rather than blocking or aborting the process.
+30
View File
@@ -0,0 +1,30 @@
#!/usr/bin/env bash
# Builds the guard as a universal (arm64 + x86_64) iOS Simulator dylib into
# packages/build-tools/bin/egress-guard.dylib. Requires Xcode.
set -euo pipefail
cd "$(dirname "$0")"
# Output directory; defaults to packages/build-tools/bin.
bin_dir="${1:-../../bin}"
mkdir -p "$bin_dir"
out=$(mktemp -d)
sdk=$(xcrun --sdk iphonesimulator --show-sdk-path)
for arch in arm64 x86_64; do
xcrun --sdk iphonesimulator clang \
-std=c11 -Wall -Wextra -Werror -O2 \
-target "$arch-apple-ios15.0-simulator" -isysroot "$sdk" \
-dynamiclib -install_name @rpath/egress-guard.dylib \
-o "$out/egress-guard-$arch.dylib" policy.c guard.c
done
lipo -create -output "$bin_dir/egress-guard.dylib" "$out/egress-guard-arm64.dylib" "$out/egress-guard-x86_64.dylib"
lipo -info "$bin_dir/egress-guard.dylib"
# The self-check the worker runs inside the simulator after installing the guard.
for arch in arm64 x86_64; do
xcrun --sdk iphonesimulator clang \
-std=c11 -Wall -Wextra -Werror -O2 \
-target "$arch-apple-ios15.0-simulator" -isysroot "$sdk" \
-o "$out/egress-guard-check-$arch" check.c
done
lipo -create -output "$bin_dir/egress-guard-check" "$out/egress-guard-check-arm64" "$out/egress-guard-check-x86_64"
chmod +x "$bin_dir/egress-guard-check"
lipo -info "$bin_dir/egress-guard-check"
@@ -0,0 +1,112 @@
// Self-check run inside a simulator right after the guard is installed:
// proves the guard library is loaded in a freshly spawned process and that it
// behaves as the requested mode says, on every connect entry point the guard
// covers. Exit 0 on success; 2 when the library is not loaded; 3 when it is
// loaded but did not behave; 1 on usage errors.
//
// egress-guard-check --mode block|log
#include <arpa/inet.h>
#include <dlfcn.h>
#include <errno.h>
#include <fcntl.h>
#include <mach-o/dyld.h>
#include <netinet/in.h>
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <unistd.h>
// TEST-NET-1 (RFC 5737): never routed, so without the guard a non-blocking
// connect reports EINPROGRESS and nothing ever answers.
#define PROBE_ADDRESS "192.0.2.1"
#define PROBE_PORT 9
typedef int (*connect_fn)(int, const struct sockaddr *, socklen_t);
static int guard_loaded(void) {
for (uint32_t i = 0; i < _dyld_image_count(); i++) {
const char *name = _dyld_get_image_name(i);
if (name != NULL && strstr(name, "egress-guard.dylib") != NULL) {
return 1;
}
}
return 0;
}
// One non-blocking connect to the probe address through `entry`, with the
// sockaddr family the caller wants. Returns the connect result; errno is
// stored in *err.
static int probe(connect_fn entry, sa_family_t family, int *err) {
int fd = socket(AF_INET, SOCK_STREAM, 0);
if (fd < 0) {
*err = errno;
return -2;
}
fcntl(fd, F_SETFL, fcntl(fd, F_GETFL, 0) | O_NONBLOCK);
struct sockaddr_in to;
memset(&to, 0, sizeof to);
to.sin_family = family;
to.sin_len = sizeof to;
to.sin_port = htons(PROBE_PORT);
inet_pton(AF_INET, PROBE_ADDRESS, &to.sin_addr);
int rc = entry(fd, (struct sockaddr *)&to, sizeof to);
*err = errno;
close(fd);
return rc;
}
int main(int argc, char **argv) {
const char *mode = "block";
if (argc == 3 && strcmp(argv[1], "--mode") == 0) {
mode = argv[2];
} else if (argc != 1) {
fprintf(stderr, "usage: egress-guard-check [--mode block|log]\n");
return 1;
}
if (!guard_loaded()) {
printf("egress-guard-check: the guard library is not loaded in this process\n");
return 2;
}
connect_fn connect_nocancel = (connect_fn)dlsym(RTLD_DEFAULT, "connect$NOCANCEL");
if (connect_nocancel == NULL) {
printf("egress-guard-check: connect$NOCANCEL is not exported in this process\n");
return 3;
}
// The plain call, the non-cancelable variant, and a sockaddr whose family
// was left AF_UNSPEC, which the kernel connects as IPv4.
const struct {
const char *name;
connect_fn entry;
sa_family_t family;
} probes[] = {
{"connect", connect, AF_INET},
{"connect$NOCANCEL", connect_nocancel, AF_INET},
{"connect with AF_UNSPEC", connect, AF_UNSPEC},
};
int block = strcmp(mode, "block") == 0;
for (size_t i = 0; i < sizeof probes / sizeof probes[0]; i++) {
int err = 0;
int rc = probe(probes[i].entry, probes[i].family, &err);
if (rc == -2) {
printf("egress-guard-check: socket() failed: %s\n", strerror(err));
return 3;
}
if (block && !(rc == -1 && err == ECONNREFUSED)) {
printf("egress-guard-check: guard loaded but a non-loopback %s was not refused (rc=%d, errno=%d %s)\n",
probes[i].name, rc, err, strerror(err));
return 3;
}
if (!block && !(rc == 0 || (rc == -1 && err == EINPROGRESS))) {
printf("egress-guard-check: guard loaded in log mode but %s failed (errno=%d %s)\n",
probes[i].name, err, strerror(err));
return 3;
}
}
if (block) {
printf("egress-guard-check: guard loaded; non-loopback connections are refused\n");
} else {
printf("egress-guard-check: guard loaded in log mode; connections pass through\n");
}
return 0;
}
@@ -0,0 +1,274 @@
// The local egress guard: interposes the calls that open outbound traffic in
// every simulator process it is inserted into, refuses non-loopback
// destinations in block mode, and records one event per destination per
// process. See README.md and policy.h.
#include "policy.h"
#include <dlfcn.h>
#include <errno.h>
#include <execinfo.h>
#include <fcntl.h>
#include <os/lock.h>
#include <sched.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <unistd.h>
#define EG_LOG_ENV "EAS_EGRESS_GUARD_LOG"
#define EG_MODE_ENV "EAS_EGRESS_GUARD_MODE"
#define EG_MAX_CALLERS 6
#define EG_CALLER_LENGTH 64
#define EG_LOCK_ATTEMPTS 64
static int eg_initialized = 0;
static void *eg_own_image = NULL;
static char eg_log_path[1024];
static eg_mode_t eg_mode = EG_MODE_BLOCK;
static eg_seen_t eg_seen;
static int eg_overflow_reported = 0;
static os_unfair_lock eg_lock = OS_UNFAIR_LOCK_INIT;
static void eg_init(void) {
if (eg_initialized) {
return;
}
eg_mode = eg_parse_mode(getenv(EG_MODE_ENV));
Dl_info self;
if (dladdr((const void *)eg_init, &self)) {
eg_own_image = self.dli_fbase;
}
const char *log_path = getenv(EG_LOG_ENV);
// The path is kept rather than an open descriptor: a process that closes
// descriptors it does not know about (launchd_sim does) would otherwise
// leave the guard writing into whatever file reused the number.
if (log_path != NULL && log_path[0] != '\0' && strlen(log_path) < sizeof eg_log_path) {
strcpy(eg_log_path, log_path);
}
eg_initialized = 1;
}
__attribute__((constructor)) static void eg_constructor(void) { eg_init(); }
// Image names of the frames above the guard's own, innermost first, with
// consecutive repeats collapsed: "Network,CFNetwork,MyApp".
__attribute__((noinline)) static int eg_collect_callers(char names[EG_MAX_CALLERS][EG_CALLER_LENGTH]) {
void *frames[EG_MAX_CALLERS + 6];
int frame_count = backtrace(frames, EG_MAX_CALLERS + 6);
int count = 0;
for (int i = 1; i < frame_count && count < EG_MAX_CALLERS; i++) {
Dl_info info;
const char *image = "?";
if (dladdr(frames[i], &info)) {
if (info.dli_fbase == eg_own_image) {
continue;
}
if (info.dli_fname != NULL) {
const char *slash = strrchr(info.dli_fname, '/');
image = slash ? slash + 1 : info.dli_fname;
}
}
if (count > 0 && strcmp(names[count - 1], image) == 0) {
continue;
}
strncpy(names[count], image, EG_CALLER_LENGTH - 1);
names[count][EG_CALLER_LENGTH - 1] = '\0';
count++;
}
return count;
}
// A lock held by this same thread (a signal handler making a socket call
// while the interrupted call held it) or by a thread that did not survive
// fork() never becomes available and os_unfair_lock_lock aborts the process
// in both cases. Give up on recording the event instead; refusal does not
// depend on it. Ordinary contention over the microsecond-long critical
// section resolves within the first attempts.
static int eg_lock_bounded(void) {
for (int i = 0; i < EG_LOCK_ATTEMPTS; i++) {
if (os_unfair_lock_trylock(&eg_lock)) {
return 1;
}
sched_yield();
}
return 0;
}
// One line in the event log. O_APPEND keeps whole lines intact across
// processes writing concurrently; the log is opened per event, at most
// EG_SEEN_CAPACITY + 1 times per process.
__attribute__((noinline)) static void eg_write_event(const char *function, const char *action,
const char *peer, int with_callers) {
if (eg_log_path[0] == '\0') {
return;
}
char names[EG_MAX_CALLERS][EG_CALLER_LENGTH];
const char *callers[EG_MAX_CALLERS];
int caller_count = with_callers ? eg_collect_callers(names) : 0;
for (int i = 0; i < caller_count; i++) {
callers[i] = names[i];
}
char line[1024];
int n = eg_format_event(line, sizeof line, getprogname(), getpid(), function, action, peer,
callers, caller_count);
if (n <= 0) {
return;
}
int fd = open(eg_log_path, O_WRONLY | O_APPEND | O_CREAT | O_CLOEXEC, 0644);
if (fd >= 0) {
(void)write(fd, line, (size_t)n);
close(fd);
}
}
// Returns 1 when the call must be refused.
__attribute__((noinline)) static int eg_handle(const char *function, const struct sockaddr *sa,
socklen_t len) {
eg_class_t cls = eg_classify(sa, len);
if (cls != EG_REMOTE) {
return 0;
}
eg_init();
int deny = eg_should_deny(eg_mode, cls);
char peer[96];
if (eg_format_peer(sa, len, peer, sizeof peer) != 0) {
strncpy(peer, "?", sizeof peer);
}
char key[EG_SEEN_KEY_LENGTH];
snprintf(key, sizeof key, "%s %s", function, peer);
int fresh = 0;
int overflowed = 0;
if (eg_lock_bounded()) {
fresh = eg_seen_insert(&eg_seen, key);
if (!fresh && eg_seen.overflow > 0 && !eg_overflow_reported) {
eg_overflow_reported = 1;
overflowed = 1;
}
os_unfair_lock_unlock(&eg_lock);
}
const char *action = deny ? "blocked" : "logged";
if (fresh) {
eg_write_event(function, action, peer, 1);
} else if (overflowed) {
// Once per process: the table is full, so further distinct destinations
// are still refused but no longer listed.
char limit[64];
snprintf(limit, sizeof limit, "%d distinct destinations", EG_SEEN_CAPACITY);
eg_write_event(EG_OVERFLOW_FUNCTION, action, limit, 0);
}
return deny;
}
// connect() with an AF_UNSPEC address on a datagram socket dissolves the
// association and sends nothing; the kernel answers EAFNOSUPPORT. Leave that
// to the kernel. On a stream socket the kernel connects to the address the
// length implies, which eg_handle classifies.
static int eg_dissolves_association(int fd, const struct sockaddr *sa, socklen_t len) {
if (sa == NULL || len < 2 || sa->sa_family != AF_UNSPEC) {
return 0;
}
int type = 0;
socklen_t type_len = sizeof type;
return getsockopt(fd, SOL_SOCKET, SO_TYPE, &type, &type_len) == 0 && type == SOCK_DGRAM;
}
typedef int (*eg_connect_fn)(int, const struct sockaddr *, socklen_t);
typedef ssize_t (*eg_sendto_fn)(int, const void *, size_t, int, const struct sockaddr *,
socklen_t);
typedef ssize_t (*eg_sendmsg_fn)(int, const struct msghdr *, int);
// The non-cancelable variants libsystem_kernel exports next to the cancelable
// ones; dyld interposing is per symbol, so each needs its own entry.
extern int eg_connect_nocancel_original(int, const struct sockaddr *, socklen_t) __asm__(
"_connect$NOCANCEL");
extern ssize_t eg_sendto_nocancel_original(int, const void *, size_t, int,
const struct sockaddr *, socklen_t) __asm__(
"_sendto$NOCANCEL");
extern ssize_t eg_sendmsg_nocancel_original(int, const struct msghdr *, int) __asm__(
"_sendmsg$NOCANCEL");
static int eg_connect_through(eg_connect_fn original, int fd, const struct sockaddr *sa,
socklen_t len) {
if (!eg_dissolves_association(fd, sa, len) && eg_handle("connect", sa, len)) {
errno = ECONNREFUSED;
return -1;
}
return original(fd, sa, len);
}
static int eg_connect(int fd, const struct sockaddr *sa, socklen_t len) {
return eg_connect_through(connect, fd, sa, len);
}
static int eg_connect_nocancel(int fd, const struct sockaddr *sa, socklen_t len) {
return eg_connect_through(eg_connect_nocancel_original, fd, sa, len);
}
static int eg_connectx(int s, const sa_endpoints_t *endpoints, sae_associd_t associd,
unsigned int flags, const struct iovec *iov, unsigned int iovcnt,
size_t *len, sae_connid_t *connid) {
if (endpoints != NULL && endpoints->sae_dstaddr != NULL &&
!eg_dissolves_association(s, endpoints->sae_dstaddr, endpoints->sae_dstaddrlen) &&
eg_handle("connectx", endpoints->sae_dstaddr, endpoints->sae_dstaddrlen)) {
errno = ECONNREFUSED;
return -1;
}
return connectx(s, endpoints, associd, flags, iov, iovcnt, len, connid);
}
static ssize_t eg_sendto_through(eg_sendto_fn original, int fd, const void *buf, size_t n,
int flags, const struct sockaddr *sa, socklen_t len) {
if (sa != NULL && eg_handle("sendto", sa, len)) {
errno = ECONNREFUSED;
return -1;
}
return original(fd, buf, n, flags, sa, len);
}
static ssize_t eg_sendto(int fd, const void *buf, size_t n, int flags, const struct sockaddr *sa,
socklen_t len) {
return eg_sendto_through(sendto, fd, buf, n, flags, sa, len);
}
static ssize_t eg_sendto_nocancel(int fd, const void *buf, size_t n, int flags,
const struct sockaddr *sa, socklen_t len) {
return eg_sendto_through(eg_sendto_nocancel_original, fd, buf, n, flags, sa, len);
}
static ssize_t eg_sendmsg_through(eg_sendmsg_fn original, int fd, const struct msghdr *msg,
int flags) {
if (msg != NULL && msg->msg_name != NULL && msg->msg_namelen > 0 &&
eg_handle("sendmsg", (const struct sockaddr *)msg->msg_name, msg->msg_namelen)) {
errno = ECONNREFUSED;
return -1;
}
return original(fd, msg, flags);
}
static ssize_t eg_sendmsg(int fd, const struct msghdr *msg, int flags) {
return eg_sendmsg_through(sendmsg, fd, msg, flags);
}
static ssize_t eg_sendmsg_nocancel(int fd, const struct msghdr *msg, int flags) {
return eg_sendmsg_through(eg_sendmsg_nocancel_original, fd, msg, flags);
}
// dyld applies these to every image in the process, including the shared
// cache, which is how calls made inside CFNetwork and Network.framework are
// caught.
__attribute__((used)) static const struct {
const void *replacement;
const void *original;
} eg_interposers[] __attribute__((section("__DATA,__interpose"))) = {
{(const void *)eg_connect, (const void *)connect},
{(const void *)eg_connect_nocancel, (const void *)eg_connect_nocancel_original},
{(const void *)eg_connectx, (const void *)connectx},
{(const void *)eg_sendto, (const void *)sendto},
{(const void *)eg_sendto_nocancel, (const void *)eg_sendto_nocancel_original},
{(const void *)eg_sendmsg, (const void *)sendmsg},
{(const void *)eg_sendmsg_nocancel, (const void *)eg_sendmsg_nocancel_original},
};
@@ -0,0 +1,164 @@
#include "policy.h"
#include <arpa/inet.h>
#include <netinet/in.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
static int is_v4_local(uint32_t host_order) {
return (host_order >> 24) == 127 || host_order == 0;
}
// XNU accepts a sockaddr whose family is AF_UNSPEC as the family its length
// implies: TCP connect() translates it on both socket families and IPv4
// sendto()/sendmsg() deliver with it. Classify it the way the kernel uses it.
static sa_family_t eg_family(const struct sockaddr *sa, socklen_t len) {
if (sa->sa_family == AF_UNSPEC) {
if (len == sizeof(struct sockaddr_in)) {
return AF_INET;
}
if (len == sizeof(struct sockaddr_in6)) {
return AF_INET6;
}
}
return sa->sa_family;
}
eg_class_t eg_classify(const struct sockaddr *sa, socklen_t len) {
if (sa == NULL || len < 2) {
return EG_PASSTHROUGH;
}
sa_family_t family = eg_family(sa, len);
if (family == AF_INET) {
if (len < sizeof(struct sockaddr_in)) {
return EG_PASSTHROUGH;
}
const struct sockaddr_in *in = (const struct sockaddr_in *)sa;
return is_v4_local(ntohl(in->sin_addr.s_addr)) ? EG_LOOPBACK : EG_REMOTE;
}
if (family == AF_INET6) {
if (len < sizeof(struct sockaddr_in6)) {
return EG_PASSTHROUGH;
}
const struct in6_addr *a6 = &((const struct sockaddr_in6 *)sa)->sin6_addr;
if (IN6_IS_ADDR_LOOPBACK(a6) || IN6_IS_ADDR_UNSPECIFIED(a6)) {
return EG_LOOPBACK;
}
if (IN6_IS_ADDR_V4MAPPED(a6)) {
uint32_t v4;
memcpy(&v4, &a6->s6_addr[12], sizeof v4);
return is_v4_local(ntohl(v4)) ? EG_LOOPBACK : EG_REMOTE;
}
return EG_REMOTE;
}
return EG_PASSTHROUGH;
}
eg_mode_t eg_parse_mode(const char *value) {
if (value != NULL && strcmp(value, "log") == 0) {
return EG_MODE_LOG;
}
return EG_MODE_BLOCK;
}
int eg_should_deny(eg_mode_t mode, eg_class_t cls) {
return mode == EG_MODE_BLOCK && cls == EG_REMOTE;
}
int eg_format_peer(const struct sockaddr *sa, socklen_t len, char *out, size_t n) {
char ip[INET6_ADDRSTRLEN];
int written;
if (sa == NULL || len < 2) {
return -1;
}
sa_family_t family = eg_family(sa, len);
if (family == AF_INET && len >= sizeof(struct sockaddr_in)) {
const struct sockaddr_in *in = (const struct sockaddr_in *)sa;
if (inet_ntop(AF_INET, &in->sin_addr, ip, sizeof ip) == NULL) {
return -1;
}
written = snprintf(out, n, "%s:%d", ip, ntohs(in->sin_port));
} else if (family == AF_INET6 && len >= sizeof(struct sockaddr_in6)) {
const struct sockaddr_in6 *in6 = (const struct sockaddr_in6 *)sa;
if (inet_ntop(AF_INET6, &in6->sin6_addr, ip, sizeof ip) == NULL) {
return -1;
}
written = snprintf(out, n, "[%s]:%d", ip, ntohs(in6->sin6_port));
} else {
return -1;
}
return (written < 0 || (size_t)written >= n) ? -1 : 0;
}
int eg_seen_insert(eg_seen_t *seen, const char *key) {
for (int i = 0; i < seen->count; i++) {
if (strncmp(seen->keys[i], key, EG_SEEN_KEY_LENGTH - 1) == 0) {
return 0;
}
}
if (seen->count >= EG_SEEN_CAPACITY) {
seen->overflow++;
return 0;
}
strncpy(seen->keys[seen->count], key, EG_SEEN_KEY_LENGTH - 1);
seen->keys[seen->count][EG_SEEN_KEY_LENGTH - 1] = '\0';
seen->count++;
return 1;
}
// Append `text` to out[*pos], replacing field separators; returns 0 when it fit.
static int append_field(char *out, size_t n, size_t *pos, const char *text) {
for (const char *c = text; *c; c++) {
if (*pos + 1 >= n) {
return -1;
}
out[(*pos)++] = (*c == '\t' || *c == '\n' || *c == '\r') ? ' ' : *c;
}
return 0;
}
static int append_raw(char *out, size_t n, size_t *pos, const char *text) {
size_t len = strlen(text);
if (*pos + len + 1 > n) {
return -1;
}
memcpy(out + *pos, text, len);
*pos += len;
return 0;
}
int eg_format_event(char *out, size_t n, const char *progname, int pid, const char *function,
const char *action, const char *peer, const char *const *callers,
int caller_count) {
size_t pos = 0;
char pid_text[16];
snprintf(pid_text, sizeof pid_text, "%d", pid);
if (append_raw(out, n, &pos, EG_EVENT_PREFIX "\t") != 0 ||
append_field(out, n, &pos, progname ? progname : "?") != 0 ||
append_raw(out, n, &pos, "\t") != 0 || append_raw(out, n, &pos, pid_text) != 0 ||
append_raw(out, n, &pos, "\t") != 0 || append_field(out, n, &pos, function) != 0 ||
append_raw(out, n, &pos, "\t") != 0 || append_field(out, n, &pos, action) != 0 ||
append_raw(out, n, &pos, "\t") != 0 || append_field(out, n, &pos, peer) != 0 ||
append_raw(out, n, &pos, "\t") != 0) {
return -1;
}
for (int i = 0; i < caller_count && callers != NULL; i++) {
if (i > 0 && append_raw(out, n, &pos, ",") != 0) {
return -1;
}
// Commas separate callers, so they cannot appear inside one.
for (const char *c = callers[i]; *c; c++) {
char ch = (*c == ',' || *c == '\t' || *c == '\n' || *c == '\r') ? ' ' : *c;
if (pos + 1 >= n) {
return -1;
}
out[pos++] = ch;
}
}
if (append_raw(out, n, &pos, "\n") != 0) {
return -1;
}
out[pos] = '\0';
return (int)pos;
}
@@ -0,0 +1,60 @@
// Policy and formatting for the local egress guard. Pure functions with no
// interposing, so they compile and test on macOS as well as in the simulator.
#ifndef EAS_EGRESS_GUARD_POLICY_H
#define EAS_EGRESS_GUARD_POLICY_H
#include <stddef.h>
#include <sys/socket.h>
// What a destination address is, as far as the guard cares.
typedef enum {
EG_PASSTHROUGH = 0, // not an internet address (unix sockets, NULL, short); never touched
EG_LOOPBACK = 1, // 127.0.0.0/8, ::1, ::ffff:127.x, unspecified; the proxy and forwards live here
EG_REMOTE = 2, // anything else, including link-local and multicast
} eg_class_t;
typedef enum {
EG_MODE_BLOCK = 0, // refuse EG_REMOTE with ECONNREFUSED
EG_MODE_LOG = 1, // observe only
} eg_mode_t;
eg_class_t eg_classify(const struct sockaddr *sa, socklen_t len);
// "block" or unset selects block; "log" selects log. Anything else is block:
// an unknown mode must fail closed.
eg_mode_t eg_parse_mode(const char *value);
int eg_should_deny(eg_mode_t mode, eg_class_t cls);
// Writes "1.2.3.4:443" or "[2001:db8::1]:443". Returns 0 on success.
int eg_format_peer(const struct sockaddr *sa, socklen_t len, char *out, size_t n);
// Fixed-capacity set of strings, so a retrying client logs a destination once
// per process instead of once per attempt.
#define EG_SEEN_CAPACITY 128
#define EG_SEEN_KEY_LENGTH 192
typedef struct {
char keys[EG_SEEN_CAPACITY][EG_SEEN_KEY_LENGTH];
int count;
int overflow; // insertions refused because the table was full
} eg_seen_t;
// 1 when the key was not present and was inserted; 0 when already present or
// when the table is full (counted in overflow).
int eg_seen_insert(eg_seen_t *seen, const char *key);
#define EG_EVENT_PREFIX "eas-egress-guard"
// Function name of the one event a process writes when its seen table fills:
// from then on distinct destinations are still refused but no longer listed.
#define EG_OVERFLOW_FUNCTION "overflow"
// One tab-separated line, newline terminated:
// eas-egress-guard\t<progname>\t<pid>\t<function>\t<action>\t<peer>\t<caller>,<caller>...\n
// Tabs and newlines inside fields are replaced with spaces. Returns the length
// written, or -1 when it does not fit.
int eg_format_event(char *out, size_t n, const char *progname, int pid, const char *function,
const char *action, const char *peer, const char *const *callers,
int caller_count);
#endif
@@ -0,0 +1,291 @@
// Black-box host tests of the built guard, loaded through
// DYLD_INSERT_LIBRARIES like the simulator does. Remote calls use fd -1 or are
// refused before the kernel; the only real sockets are on loopback. Build and
// run with tests/run-guard-tests.sh.
//
// guard-insert-test <case> <log-path> [<extra-path>]
#include <arpa/inet.h>
#include <dlfcn.h>
#include <errno.h>
#include <fcntl.h>
#include <netinet/in.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/stat.h>
#include <sys/uio.h>
#include <unistd.h>
typedef int (*connect_fn)(int, const struct sockaddr *, socklen_t);
typedef ssize_t (*sendto_fn)(int, const void *, size_t, int, const struct sockaddr *, socklen_t);
typedef ssize_t (*sendmsg_fn)(int, const struct msghdr *, int);
static struct sockaddr_in remote(int port) {
struct sockaddr_in a;
memset(&a, 0, sizeof a);
a.sin_family = AF_INET;
a.sin_len = sizeof a;
a.sin_port = htons(port);
inet_pton(AF_INET, "192.0.2.1", &a.sin_addr);
return a;
}
static struct sockaddr_in6 remote6(int port) {
struct sockaddr_in6 a;
memset(&a, 0, sizeof a);
a.sin6_family = AF_INET6;
a.sin6_len = sizeof a;
a.sin6_port = htons(port);
inet_pton(AF_INET6, "2001:db8::1", &a.sin6_addr);
return a;
}
static int refused_sendto(int port) {
struct sockaddr_in to = remote(port);
errno = 0;
return sendto(-1, "x", 1, 0, (struct sockaddr *)&to, sizeof to) == -1 && errno == ECONNREFUSED;
}
static off_t file_size(const char *path) {
struct stat st;
return stat(path, &st) == 0 ? st.st_size : -1;
}
static int count_lines(const char *path, const char *needle) {
FILE *f = fopen(path, "r");
if (f == NULL) {
return -1;
}
int count = 0;
char line[1024];
while (fgets(line, sizeof line, f)) {
if (strstr(line, needle)) {
count++;
}
}
fclose(f);
return count;
}
// Daemon-style cleanup: close every descriptor the process does not know
// about, then open an application file. The guard's event must land in the
// event log, not in that file.
static int fd_reuse(const char *log_path, const char *app_path) {
for (int fd = 3; fd < getdtablesize(); fd++) {
close(fd);
}
int app_fd = open(app_path, O_CREAT | O_TRUNC | O_WRONLY, 0600);
if (app_fd < 0 || !refused_sendto(9)) {
return 2;
}
close(app_fd);
if (file_size(app_path) != 0) {
printf("FAIL fd-reuse: %lld bytes written into an unrelated file\n",
(long long)file_size(app_path));
return 1;
}
if (count_lines(log_path, "\tsendto\tblocked\t192.0.2.1:9\t") != 1) {
printf("FAIL fd-reuse: event missing from the event log\n");
return 1;
}
return 0;
}
// Started with stdout closed, as the runner arranges: the process's own stdout
// output must not end up in the event log, and the event must.
static int low_fd(const char *log_path) {
printf("STDOUT-LINE\n");
fflush(stdout);
if (!refused_sendto(9)) {
return 2;
}
if (count_lines(log_path, "STDOUT-LINE") != 0) {
return 1;
}
return count_lines(log_path, "\tsendto\tblocked\t192.0.2.1:9\t") == 1 ? 0 : 1;
}
static int nocancel(void) {
connect_fn nc_connect = (connect_fn)dlsym(RTLD_DEFAULT, "connect$NOCANCEL");
sendto_fn nc_sendto = (sendto_fn)dlsym(RTLD_DEFAULT, "sendto$NOCANCEL");
sendmsg_fn nc_sendmsg = (sendmsg_fn)dlsym(RTLD_DEFAULT, "sendmsg$NOCANCEL");
if (!nc_connect || !nc_sendto || !nc_sendmsg) {
return 2;
}
struct sockaddr_in to = remote(443);
int failures = 0;
errno = 0;
if (nc_connect(-1, (struct sockaddr *)&to, sizeof to) != -1 || errno != ECONNREFUSED) {
printf("FAIL nocancel: connect$NOCANCEL not refused (errno %d)\n", errno);
failures++;
}
errno = 0;
if (nc_sendto(-1, "x", 1, 0, (struct sockaddr *)&to, sizeof to) != -1 || errno != ECONNREFUSED) {
printf("FAIL nocancel: sendto$NOCANCEL not refused (errno %d)\n", errno);
failures++;
}
char payload[] = "x";
struct iovec iov = {.iov_base = payload, .iov_len = 1};
struct msghdr msg = {.msg_name = &to, .msg_namelen = sizeof to, .msg_iov = &iov, .msg_iovlen = 1};
errno = 0;
if (nc_sendmsg(-1, &msg, 0) != -1 || errno != ECONNREFUSED) {
printf("FAIL nocancel: sendmsg$NOCANCEL not refused (errno %d)\n", errno);
failures++;
}
return failures ? 1 : 0;
}
// The shapes the kernel accepts as IPv4/IPv6 despite sa_family == AF_UNSPEC.
static int unspec(void) {
int failures = 0;
struct sockaddr_in to = remote(443);
to.sin_family = AF_UNSPEC;
int tcp = socket(AF_INET, SOCK_STREAM, 0);
errno = 0;
if (connect(tcp, (struct sockaddr *)&to, sizeof to) != -1 || errno != ECONNREFUSED) {
printf("FAIL unspec: TCP connect with AF_UNSPEC not refused (errno %d)\n", errno);
failures++;
}
close(tcp);
struct sockaddr_in6 to6 = remote6(443);
to6.sin6_family = AF_UNSPEC;
int tcp6 = socket(AF_INET6, SOCK_STREAM, 0);
errno = 0;
if (connect(tcp6, (struct sockaddr *)&to6, sizeof to6) != -1 || errno != ECONNREFUSED) {
printf("FAIL unspec: TCP6 connect with AF_UNSPEC not refused (errno %d)\n", errno);
failures++;
}
close(tcp6);
errno = 0;
if (sendto(-1, "x", 1, 0, (struct sockaddr *)&to, sizeof to) != -1 || errno != ECONNREFUSED) {
printf("FAIL unspec: sendto with AF_UNSPEC not refused (errno %d)\n", errno);
failures++;
}
char payload[] = "x";
struct iovec iov = {.iov_base = payload, .iov_len = 1};
struct msghdr msg = {.msg_name = &to, .msg_namelen = sizeof to, .msg_iov = &iov, .msg_iovlen = 1};
errno = 0;
if (sendmsg(-1, &msg, 0) != -1 || errno != ECONNREFUSED) {
printf("FAIL unspec: sendmsg with AF_UNSPEC not refused (errno %d)\n", errno);
failures++;
}
return failures ? 1 : 0;
}
// A UDP socket associated with loopback dissolves the association through
// connect(AF_UNSPEC), whatever address bytes follow the family. The guard
// must let the kernel do that.
static int udp_disconnect(void) {
int fd = socket(AF_INET, SOCK_DGRAM, 0);
struct sockaddr_in loopback;
memset(&loopback, 0, sizeof loopback);
loopback.sin_family = AF_INET;
loopback.sin_len = sizeof loopback;
loopback.sin_port = htons(9);
loopback.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
if (connect(fd, (struct sockaddr *)&loopback, sizeof loopback) != 0) {
return 2;
}
struct sockaddr_in peer;
socklen_t peer_len = sizeof peer;
if (getpeername(fd, (struct sockaddr *)&peer, &peer_len) != 0) {
return 2;
}
int failures = 0;
for (int stale = 0; stale < 2; stale++) {
struct sockaddr_in dissolve = stale ? remote(443) : loopback;
if (!stale) {
memset(&dissolve, 0, sizeof dissolve);
dissolve.sin_len = sizeof dissolve;
}
dissolve.sin_family = AF_UNSPEC;
if (connect(fd, (struct sockaddr *)&loopback, sizeof loopback) != 0) {
return 2;
}
errno = 0;
int rc = connect(fd, (struct sockaddr *)&dissolve, sizeof dissolve);
int err = errno;
peer_len = sizeof peer;
int still_connected = getpeername(fd, (struct sockaddr *)&peer, &peer_len) == 0;
if (rc != -1 || err != EAFNOSUPPORT || still_connected) {
printf("FAIL udp-disconnect (%s address): rc=%d errno=%d still_connected=%d\n",
stale ? "stale remote" : "zeroed", rc, err, still_connected);
failures++;
}
}
close(fd);
return failures ? 1 : 0;
}
// Past 128 distinct destinations a process writes one overflow line and
// nothing more; refusal continues.
static int overflow(const char *log_path) {
for (int port = 1; port <= 140; port++) {
if (!refused_sendto(port)) {
return 2;
}
}
int listed = count_lines(log_path, "\tsendto\tblocked\t192.0.2.1:");
int overflow_lines = count_lines(log_path, "\toverflow\tblocked\t128 distinct destinations\t");
if (listed != 128 || overflow_lines != 1) {
printf("FAIL overflow: %d destinations listed, %d overflow line(s)\n", listed, overflow_lines);
return 1;
}
return 0;
}
// The event names the images above the guard, not the guard itself.
static int callers(const char *log_path) {
if (!refused_sendto(9)) {
return 2;
}
FILE *f = fopen(log_path, "r");
if (f == NULL) {
return 2;
}
char line[1024];
int ok = 0;
while (fgets(line, sizeof line, f)) {
if (strstr(line, "\tsendto\tblocked\t192.0.2.1:9\t")) {
const char *callers_field = strrchr(line, '\t') + 1;
ok = strstr(callers_field, "guard-insert-test") != NULL &&
strstr(callers_field, "egress-guard") == NULL;
if (!ok) {
printf("FAIL callers: %s", line);
}
}
}
fclose(f);
return ok ? 0 : 1;
}
int main(int argc, char **argv) {
if (argc < 3) {
return 2;
}
const char *name = argv[1];
const char *log_path = argv[2];
if (strcmp(name, "fd-reuse") == 0) {
return argc == 4 ? fd_reuse(log_path, argv[3]) : 2;
}
if (strcmp(name, "low-fd") == 0) {
return low_fd(log_path);
}
if (strcmp(name, "nocancel") == 0) {
return nocancel();
}
if (strcmp(name, "unspec") == 0) {
return unspec();
}
if (strcmp(name, "udp-disconnect") == 0) {
return udp_disconnect();
}
if (strcmp(name, "callers") == 0) {
return callers(log_path);
}
if (strcmp(name, "overflow") == 0) {
return overflow(log_path);
}
return 2;
}
@@ -0,0 +1,177 @@
// White-box host tests of guard.c's dedupe lock: the schedules that would
// abort the process are forced deterministically. Every case runs in its own
// process; sockets are never opened (fd -1), so nothing can leave the host.
// Build and run with tests/run-guard-tests.sh.
#include "../policy.h"
#include <pthread.h>
#include <signal.h>
#include <sys/wait.h>
static int scheduled_seen_insert(eg_seen_t *seen, const char *key);
#define eg_seen_insert scheduled_seen_insert
#include "../guard.c"
#undef eg_seen_insert
#include <arpa/inet.h>
#define CONTENTION_THREADS 8
#define CONTENTION_KEYS 12
#define CONTENTION_ROUNDS 25
static int interrupt_insert;
static int ready_pipe[2];
static int release_pipe[2];
static struct sockaddr_in remote(int port) {
struct sockaddr_in a;
memset(&a, 0, sizeof a);
a.sin_family = AF_INET;
a.sin_len = sizeof a;
a.sin_port = htons(port);
inet_pton(AF_INET, "192.0.2.1", &a.sin_addr);
return a;
}
// Block mode must refuse before touching the kernel, even on an invalid fd.
static void send_remote(int port) {
struct sockaddr_in to = remote(port);
if (eg_sendto(-1, "x", 1, 0, (struct sockaddr *)&to, sizeof to) != -1 || errno != ECONNREFUSED) {
_exit(2);
}
}
static void on_signal(int sig) {
(void)sig;
send_remote(9);
}
static int scheduled_seen_insert(eg_seen_t *seen, const char *key) {
if (interrupt_insert) {
interrupt_insert = 0;
raise(SIGUSR1);
}
return eg_seen_insert(seen, key);
}
static int signal_reentry(void) {
signal(SIGUSR1, on_signal);
interrupt_insert = 1;
send_remote(9);
send_remote(10);
return 0;
}
static void *hold_lock(void *unused) {
(void)unused;
char byte = 'x';
os_unfair_lock_lock(&eg_lock);
(void)write(ready_pipe[1], &byte, 1);
(void)read(release_pipe[0], &byte, 1);
os_unfair_lock_unlock(&eg_lock);
return NULL;
}
static int fork_while_locked(void) {
if (pipe(ready_pipe) || pipe(release_pipe)) {
return 2;
}
pthread_t thread;
if (pthread_create(&thread, NULL, hold_lock, NULL)) {
return 2;
}
char byte;
if (read(ready_pipe[0], &byte, 1) != 1) {
return 2;
}
pid_t child = fork();
if (child < 0) {
return 2;
}
if (child == 0) {
alarm(5);
send_remote(9);
send_remote(10);
_exit(0);
}
(void)write(release_pipe[1], "x", 1);
pthread_join(thread, NULL);
int status;
waitpid(child, &status, 0);
if (WIFSIGNALED(status)) {
printf("FAIL fork: child terminated by signal %d\n", WTERMSIG(status));
return 1;
}
return WIFEXITED(status) ? WEXITSTATUS(status) : 2;
}
static void *hammer(void *unused) {
(void)unused;
for (int round = 0; round < CONTENTION_ROUNDS; round++) {
for (int key = 0; key < CONTENTION_KEYS; key++) {
send_remote(1000 + key);
}
}
return NULL;
}
// Under contention a thread may give up its insert; the next attempt on the
// same key must then record it, so every key ends up logged exactly once.
static int contention(const char *log_path) {
pthread_t threads[CONTENTION_THREADS];
for (int i = 0; i < CONTENTION_THREADS; i++) {
if (pthread_create(&threads[i], NULL, hammer, NULL)) {
return 2;
}
}
for (int i = 0; i < CONTENTION_THREADS; i++) {
pthread_join(threads[i], NULL);
}
FILE *log = fopen(log_path, "r");
if (log == NULL) {
return 2;
}
int counts[CONTENTION_KEYS] = {0};
int total = 0;
char line[1024];
while (fgets(line, sizeof line, log)) {
total++;
for (int key = 0; key < CONTENTION_KEYS; key++) {
char needle[64];
snprintf(needle, sizeof needle, "\tsendto\tblocked\t192.0.2.1:%d\t", 1000 + key);
if (strstr(line, needle)) {
counts[key]++;
}
}
}
fclose(log);
int failures = 0;
for (int key = 0; key < CONTENTION_KEYS; key++) {
if (counts[key] != 1) {
printf("FAIL contention: 192.0.2.1:%d logged %d times\n", 1000 + key, counts[key]);
failures++;
}
}
if (total != CONTENTION_KEYS) {
printf("FAIL contention: %d lines for %d keys\n", total, CONTENTION_KEYS);
failures++;
}
return failures ? 1 : 0;
}
int main(int argc, char **argv) {
if (argc != 2) {
return 2;
}
eg_mode = EG_MODE_BLOCK;
if (strcmp(argv[1], "signal") == 0) {
return signal_reentry();
}
if (strcmp(argv[1], "fork") == 0) {
return fork_while_locked();
}
if (strcmp(argv[1], "contention") == 0) {
return contention(eg_log_path);
}
return 2;
}
@@ -0,0 +1,172 @@
// Probe run inside a simulator by the egress guard end-to-end test. Exercises
// every outbound path the guard must cover and prints one JSON object.
//
// nettest --proxy-port <port> --udp-port <port>
//
// proxy-port: a CONNECT proxy on the host's loopback, standing in for the
// egress proxy. udp-port: a UDP echo on loopback.
import Foundation
import Network
var results: [String: String] = [:]
let args = CommandLine.arguments
func arg(_ name: String) -> Int {
guard let i = args.firstIndex(of: name), i + 1 < args.count else { return 0 }
return Int(args[i + 1]) ?? 0
}
let proxyPort = arg("--proxy-port")
let udpPort = arg("--udp-port")
let remoteHost = "example.com"
let remotePort = 443
func classify(_ error: Error?) -> String {
guard let e = error as NSError? else { return "ok" }
if e.domain == NSURLErrorDomain && e.code == NSURLErrorCannotConnectToHost { return "refused" }
if e.domain == NSPOSIXErrorDomain && e.code == Int(ECONNREFUSED) { return "refused" }
return "error(\(e.domain):\(e.code))"
}
// 1. URLSession straight to the internet. Blocked by the guard.
func urlSessionDirect() {
let g = DispatchGroup(); g.enter()
let c = URLSessionConfiguration.ephemeral; c.timeoutIntervalForRequest = 10
c.connectionProxyDictionary = [:] // explicitly no proxy, even if the host has one
URLSession(configuration: c).dataTask(with: URL(string: "https://\(remoteHost)/")!) { _, r, e in
results["urlsessionDirect"] = e == nil ? "ok(\((r as? HTTPURLResponse)?.statusCode ?? 0))" : classify(e); g.leave()
}.resume(); g.wait()
}
// 2. URLSession through a CONNECT proxy on loopback. Allowed: the guard only
// sees a loopback connect.
func urlSessionProxied() {
let g = DispatchGroup(); g.enter()
let c = URLSessionConfiguration.ephemeral; c.timeoutIntervalForRequest = 15
// The kCFNetworkProxies* constants are not exposed to Swift on iOS; these are their values.
c.connectionProxyDictionary = ["HTTPSEnable": 1, "HTTPSProxy": "127.0.0.1", "HTTPSPort": proxyPort]
URLSession(configuration: c).dataTask(with: URL(string: "https://\(remoteHost)/")!) { _, r, e in
results["urlsessionProxied"] = e == nil ? "ok(\((r as? HTTPURLResponse)?.statusCode ?? 0))" : classify(e); g.leave()
}.resume(); g.wait()
}
// 3. Network.framework connection straight out. Blocked.
func nwConnectionDirect() {
let g = DispatchGroup(); g.enter()
let p = NWParameters.tls; p.preferNoProxies = true
let c = NWConnection(host: NWEndpoint.Host(remoteHost), port: NWEndpoint.Port(integerLiteral: UInt16(remotePort)), using: p)
var done = false
c.stateUpdateHandler = { s in
guard !done else { return }
switch s {
case .ready: done = true; results["nwconnectionDirect"] = "ok"; c.cancel(); g.leave()
case .failed(let e): done = true; results["nwconnectionDirect"] = classify(e); g.leave()
case .waiting(let e): done = true; results["nwconnectionDirect"] = "waiting(" + classify(e) + ")"; c.cancel(); g.leave()
default: break
}
}
c.start(queue: .global())
_ = g.wait(timeout: .now() + 15)
if results["nwconnectionDirect"] == nil { results["nwconnectionDirect"] = "timeout"; c.cancel() }
}
// 4. Plain BSD TCP connect to a resolved address. Blocked.
func bsdConnectDirect() {
var hints = addrinfo(); hints.ai_socktype = SOCK_STREAM; hints.ai_family = AF_INET
var info: UnsafeMutablePointer<addrinfo>? = nil
guard getaddrinfo(remoteHost, "443", &hints, &info) == 0, let ai = info else { results["bsdConnectDirect"] = "dns-failed"; return }
results["dns"] = "ok"
let fd = socket(ai.pointee.ai_family, ai.pointee.ai_socktype, ai.pointee.ai_protocol)
let rc = connect(fd, ai.pointee.ai_addr, ai.pointee.ai_addrlen)
results["bsdConnectDirect"] = rc == 0 ? "ok" : (errno == ECONNREFUSED ? "refused" : "errno(\(errno))")
close(fd); freeaddrinfo(info)
}
// 5. UDP datagram straight out. Blocked. 6. UDP datagram to loopback echo. Allowed.
func udp(_ key: String, _ host: String, _ port: Int, expectEcho: Bool) {
let fd = socket(AF_INET, SOCK_DGRAM, 0)
var to = sockaddr_in(); to.sin_family = sa_family_t(AF_INET); to.sin_port = in_port_t(UInt16(port)).bigEndian
inet_pton(AF_INET, host, &to.sin_addr)
var payload: [UInt8] = Array("ping".utf8)
let sent = withUnsafePointer(to: &to) { p in p.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
sendto(fd, &payload, payload.count, 0, sa, socklen_t(MemoryLayout<sockaddr_in>.size)) } }
if sent < 0 { results[key] = errno == ECONNREFUSED ? "refused" : "errno(\(errno))"; close(fd); return }
if expectEcho {
var tv = timeval(tv_sec: 3, tv_usec: 0); setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &tv, socklen_t(MemoryLayout<timeval>.size))
var buf = [UInt8](repeating: 0, count: 16)
let n = recv(fd, &buf, buf.count, 0)
results[key] = n > 0 ? "ok" : "no-echo"
} else { results[key] = "sent" }
close(fd)
}
urlSessionDirect()
urlSessionProxied()
nwConnectionDirect()
bsdConnectDirect()
udp("udpDirect", "8.8.8.8", 53, expectEcho: false)
udp("udpLoopback", "127.0.0.1", udpPort, expectEcho: true)
// 7. The same literal destination twice: the guard must log it once per process.
func bsdConnectLiteral(_ key: String) {
var to = sockaddr_in(); to.sin_family = sa_family_t(AF_INET); to.sin_port = in_port_t(UInt16(443)).bigEndian
inet_pton(AF_INET, "1.1.1.1", &to.sin_addr)
let fd = socket(AF_INET, SOCK_STREAM, 0)
let rc = withUnsafePointer(to: &to) { p in p.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
connect(fd, sa, socklen_t(MemoryLayout<sockaddr_in>.size)) } }
results[key] = rc == 0 ? "ok" : (errno == ECONNREFUSED ? "refused" : "errno(\(errno))")
close(fd)
}
bsdConnectLiteral("literalFirst")
bsdConnectLiteral("literalSecond")
// 8. A sockaddr_in whose family was left AF_UNSPEC: the kernel connects it as
// IPv4, so the guard must refuse it. 9. connect(AF_UNSPEC) on a UDP socket
// dissolves the association; the guard must leave that to the kernel.
func unspecConnect() {
var to = sockaddr_in(); to.sin_family = sa_family_t(AF_UNSPEC); to.sin_port = in_port_t(UInt16(443)).bigEndian
inet_pton(AF_INET, "1.0.0.1", &to.sin_addr)
let fd = socket(AF_INET, SOCK_STREAM, 0)
let rc = withUnsafePointer(to: &to) { p in p.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
connect(fd, sa, socklen_t(MemoryLayout<sockaddr_in>.size)) } }
results["unspecConnect"] = rc == 0 ? "ok" : (errno == ECONNREFUSED ? "refused" : "errno(\(errno))")
close(fd)
}
func udpDisconnect() {
let fd = socket(AF_INET, SOCK_DGRAM, 0)
var to = sockaddr_in(); to.sin_family = sa_family_t(AF_INET); to.sin_port = in_port_t(UInt16(udpPort)).bigEndian
inet_pton(AF_INET, "127.0.0.1", &to.sin_addr)
let associated = withUnsafePointer(to: &to) { p in p.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
connect(fd, sa, socklen_t(MemoryLayout<sockaddr_in>.size)) } }
var dissolve = sockaddr_in(); dissolve.sin_family = sa_family_t(AF_UNSPEC); dissolve.sin_port = in_port_t(UInt16(443)).bigEndian
inet_pton(AF_INET, "1.1.1.1", &dissolve.sin_addr)
let rc = withUnsafePointer(to: &dissolve) { p in p.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
connect(fd, sa, socklen_t(MemoryLayout<sockaddr_in>.size)) } }
let err = errno
var peer = sockaddr_in(); var peerLength = socklen_t(MemoryLayout<sockaddr_in>.size)
let stillConnected = withUnsafeMutablePointer(to: &peer) { p in p.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
getpeername(fd, sa, &peerLength) } } == 0
results["udpDisconnect"] = associated != 0 ? "associate-failed"
: (rc == -1 && err == EAFNOSUPPORT && !stillConnected) ? "ok" : "errno(\(err)),connected=\(stillConnected)"
close(fd)
}
// 10. The non-cancelable variant of sendto, as libsystem exports it. Blocked.
func nocancelSendto() {
typealias SendtoFn = @convention(c) (Int32, UnsafeRawPointer?, Int, Int32, UnsafePointer<sockaddr>?, socklen_t) -> Int
guard let symbol = dlsym(UnsafeMutableRawPointer(bitPattern: -2), "sendto$NOCANCEL") else {
results["nocancelSendto"] = "no-symbol"; return
}
let nocancelSendto = unsafeBitCast(symbol, to: SendtoFn.self)
let fd = socket(AF_INET, SOCK_DGRAM, 0)
var to = sockaddr_in(); to.sin_family = sa_family_t(AF_INET); to.sin_port = in_port_t(UInt16(53)).bigEndian
inet_pton(AF_INET, "8.8.8.8", &to.sin_addr)
var payload: [UInt8] = Array("ping".utf8)
let sent = withUnsafePointer(to: &to) { p in p.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
nocancelSendto(fd, &payload, payload.count, 0, sa, socklen_t(MemoryLayout<sockaddr_in>.size)) } }
results["nocancelSendto"] = sent < 0 ? (errno == ECONNREFUSED ? "refused" : "errno(\(errno))") : "sent"
close(fd)
}
unspecConnect()
udpDisconnect()
nocancelSendto()
let json = try! JSONSerialization.data(withJSONObject: results, options: [.sortedKeys])
print(String(data: json, encoding: .utf8)!)
@@ -0,0 +1,143 @@
// Host-runnable tests for the guard's policy and formatting. Build and run
// with tests/run-policy-tests.sh; every failure prints the failing line.
#include "../policy.h"
#include <arpa/inet.h>
#include <netinet/in.h>
#include <stdio.h>
#include <string.h>
#include <sys/un.h>
static int failures = 0;
#define CHECK(cond) \
do { \
if (!(cond)) { \
failures++; \
fprintf(stderr, "FAIL %s:%d: %s\n", __FILE__, __LINE__, #cond); \
} \
} while (0)
static struct sockaddr_in v4(const char *ip, int port) {
struct sockaddr_in a; memset(&a, 0, sizeof a);
a.sin_family = AF_INET; a.sin_len = sizeof a; a.sin_port = htons(port);
inet_pton(AF_INET, ip, &a.sin_addr);
return a;
}
static struct sockaddr_in6 v6(const char *ip, int port) {
struct sockaddr_in6 a; memset(&a, 0, sizeof a);
a.sin6_family = AF_INET6; a.sin6_len = sizeof a; a.sin6_port = htons(port);
inet_pton(AF_INET6, ip, &a.sin6_addr);
return a;
}
static void test_classify(void) {
struct sockaddr_in a4; struct sockaddr_in6 a6; struct sockaddr_un un;
a4 = v4("127.0.0.1", 8899); CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_LOOPBACK);
a4 = v4("127.42.0.9", 80); CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_LOOPBACK);
a4 = v4("0.0.0.0", 80); CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_LOOPBACK);
a4 = v4("93.184.216.34", 443); CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_REMOTE);
a4 = v4("192.168.1.10", 8081); CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_REMOTE);
a4 = v4("169.254.1.1", 80); CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_REMOTE);
a4 = v4("224.0.0.251", 5353); CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_REMOTE);
a6 = v6("::1", 8899); CHECK(eg_classify((struct sockaddr *)&a6, sizeof a6) == EG_LOOPBACK);
a6 = v6("::", 80); CHECK(eg_classify((struct sockaddr *)&a6, sizeof a6) == EG_LOOPBACK);
a6 = v6("::ffff:127.0.0.1", 8899); CHECK(eg_classify((struct sockaddr *)&a6, sizeof a6) == EG_LOOPBACK);
a6 = v6("::ffff:93.184.216.34", 443); CHECK(eg_classify((struct sockaddr *)&a6, sizeof a6) == EG_REMOTE);
a6 = v6("2606:4700::1", 443); CHECK(eg_classify((struct sockaddr *)&a6, sizeof a6) == EG_REMOTE);
a6 = v6("fe80::1", 443); CHECK(eg_classify((struct sockaddr *)&a6, sizeof a6) == EG_REMOTE);
memset(&un, 0, sizeof un); un.sun_family = AF_UNIX; strcpy(un.sun_path, "/var/run/mDNSResponder");
CHECK(eg_classify((struct sockaddr *)&un, sizeof un) == EG_PASSTHROUGH);
CHECK(eg_classify(NULL, 0) == EG_PASSTHROUGH);
a4 = v4("93.184.216.34", 443); CHECK(eg_classify((struct sockaddr *)&a4, 4) == EG_PASSTHROUGH); // too short to read
// AF_UNSPEC is what the kernel makes of it: the family the exact length implies.
a4 = v4("93.184.216.34", 443); a4.sin_family = AF_UNSPEC;
CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_REMOTE);
CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4 + 8) == EG_PASSTHROUGH); // kernel: EINVAL
a4 = v4("127.0.0.1", 8899); a4.sin_family = AF_UNSPEC;
CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_LOOPBACK);
a4 = v4("0.0.0.0", 0); a4.sin_family = AF_UNSPEC;
CHECK(eg_classify((struct sockaddr *)&a4, sizeof a4) == EG_LOOPBACK); // UDP disconnect shape
a6 = v6("2606:4700::1", 443); a6.sin6_family = AF_UNSPEC;
CHECK(eg_classify((struct sockaddr *)&a6, sizeof a6) == EG_REMOTE);
a6 = v6("::ffff:93.184.216.34", 443); a6.sin6_family = AF_UNSPEC;
CHECK(eg_classify((struct sockaddr *)&a6, sizeof a6) == EG_REMOTE);
a6 = v6("::1", 443); a6.sin6_family = AF_UNSPEC;
CHECK(eg_classify((struct sockaddr *)&a6, sizeof a6) == EG_LOOPBACK);
memset(&un, 0, sizeof un); un.sun_family = AF_UNSPEC;
CHECK(eg_classify((struct sockaddr *)&un, sizeof un) == EG_PASSTHROUGH);
}
static void test_mode(void) {
CHECK(eg_parse_mode(NULL) == EG_MODE_BLOCK);
CHECK(eg_parse_mode("") == EG_MODE_BLOCK);
CHECK(eg_parse_mode("block") == EG_MODE_BLOCK);
CHECK(eg_parse_mode("log") == EG_MODE_LOG);
CHECK(eg_parse_mode("LOG") == EG_MODE_BLOCK);
CHECK(eg_parse_mode("permissive") == EG_MODE_BLOCK);
CHECK(eg_should_deny(EG_MODE_BLOCK, EG_REMOTE) == 1);
CHECK(eg_should_deny(EG_MODE_BLOCK, EG_LOOPBACK) == 0);
CHECK(eg_should_deny(EG_MODE_BLOCK, EG_PASSTHROUGH) == 0);
CHECK(eg_should_deny(EG_MODE_LOG, EG_REMOTE) == 0);
}
static void test_format_peer(void) {
char out[64];
struct sockaddr_in a4 = v4("93.184.216.34", 443);
CHECK(eg_format_peer((struct sockaddr *)&a4, sizeof a4, out, sizeof out) == 0);
CHECK(strcmp(out, "93.184.216.34:443") == 0);
struct sockaddr_in6 a6 = v6("2606:4700::1", 443);
CHECK(eg_format_peer((struct sockaddr *)&a6, sizeof a6, out, sizeof out) == 0);
CHECK(strcmp(out, "[2606:4700::1]:443") == 0);
CHECK(eg_format_peer((struct sockaddr *)&a4, sizeof a4, out, 8) != 0);
struct sockaddr_un un; memset(&un, 0, sizeof un); un.sun_family = AF_UNIX;
CHECK(eg_format_peer((struct sockaddr *)&un, sizeof un, out, sizeof out) != 0);
a4.sin_family = AF_UNSPEC;
CHECK(eg_format_peer((struct sockaddr *)&a4, sizeof a4, out, sizeof out) == 0);
CHECK(strcmp(out, "93.184.216.34:443") == 0);
a6.sin6_family = AF_UNSPEC;
CHECK(eg_format_peer((struct sockaddr *)&a6, sizeof a6, out, sizeof out) == 0);
CHECK(strcmp(out, "[2606:4700::1]:443") == 0);
}
static void test_seen(void) {
static eg_seen_t seen; memset(&seen, 0, sizeof seen);
CHECK(eg_seen_insert(&seen, "connect 1.1.1.1:443") == 1);
CHECK(eg_seen_insert(&seen, "connect 1.1.1.1:443") == 0);
CHECK(eg_seen_insert(&seen, "sendto 1.1.1.1:443") == 1);
CHECK(seen.count == 2);
char key[64];
for (int i = 0; i < EG_SEEN_CAPACITY + 5; i++) { snprintf(key, sizeof key, "k%d", i); eg_seen_insert(&seen, key); }
CHECK(seen.count == EG_SEEN_CAPACITY);
CHECK(seen.overflow == 7); // 2 + 128 + 5 attempts, 128 fit
CHECK(eg_seen_insert(&seen, "k0") == 0); // still remembered
}
static void test_format_event(void) {
char out[512];
const char *callers[] = {"Network", "CFNetwork", "MyApp"};
int n = eg_format_event(out, sizeof out, "MyApp", 4242, "connect", "blocked", "93.184.216.34:443", callers, 3);
CHECK(n > 0);
CHECK(strcmp(out, "eas-egress-guard\tMyApp\t4242\tconnect\tblocked\t93.184.216.34:443\tNetwork,CFNetwork,MyApp\n") == 0);
const char *tabbed[] = {"a\tb"};
n = eg_format_event(out, sizeof out, "we\nird\tname", 1, "sendto", "logged", "1.2.3.4:53", tabbed, 1);
CHECK(n > 0);
CHECK(strcmp(out, "eas-egress-guard\twe ird name\t1\tsendto\tlogged\t1.2.3.4:53\ta b\n") == 0);
n = eg_format_event(out, sizeof out, "NoCallers", 7, "connectx", "blocked", "[::2]:80", NULL, 0);
CHECK(n > 0 && strcmp(out, "eas-egress-guard\tNoCallers\t7\tconnectx\tblocked\t[::2]:80\t\n") == 0);
CHECK(eg_format_event(out, 16, "MyApp", 4242, "connect", "blocked", "93.184.216.34:443", callers, 3) == -1);
}
int main(void) {
test_classify(); test_mode(); test_format_peer(); test_seen(); test_format_event();
if (failures) { fprintf(stderr, "%d failure(s)\n", failures); return 1; }
printf("egress-guard policy tests: all passed\n");
return 0;
}
@@ -0,0 +1,62 @@
#!/usr/bin/env bash
# Builds the guard for the host (macOS) and runs the guard tests against it:
# white-box lock schedules with guard.c compiled in, and black-box cases with
# the built library inserted the way the simulator inserts it. No socket is
# ever opened to a non-loopback address.
set -euo pipefail
cd "$(dirname "$0")/.."
out=$(mktemp -d)
trap 'rm -rf "$out"' EXIT
clang -std=c11 -Wall -Wextra -Werror -O2 -dynamiclib policy.c guard.c -o "$out/egress-guard.dylib"
clang -std=c11 -Wall -Wextra -Werror -O2 -pthread policy.c tests/guard_test.c -o "$out/guard-test"
clang -std=c11 -Wall -Wextra -Werror -O2 tests/guard_insert_test.c -o "$out/guard-insert-test"
failures=0
report() {
if [[ $2 -eq 0 ]]; then
echo "ok $1"
else
echo "FAIL $1 (exit $2)"
failures=$((failures + 1))
fi
}
# White-box: guard.c compiled into the test, lock schedules forced.
for name in signal fork contention; do
status=0
: > "$out/$name.log"
EAS_EGRESS_GUARD_MODE=block EAS_EGRESS_GUARD_LOG="$out/$name.log" \
"$out/guard-test" "$name" || status=$?
report "$name" "$status"
done
# Black-box: the built library inserted into a plain process.
export DYLD_INSERT_LIBRARIES="$out/egress-guard.dylib"
export EAS_EGRESS_GUARD_MODE=block
status=0
: > "$out/fd-reuse.log"
EAS_EGRESS_GUARD_LOG="$out/fd-reuse.log" \
"$out/guard-insert-test" fd-reuse "$out/fd-reuse.log" "$out/application-file" || status=$?
report fd-reuse "$status"
status=0
: > "$out/low-fd.log"
EAS_EGRESS_GUARD_LOG="$out/low-fd.log" \
"$out/guard-insert-test" low-fd "$out/low-fd.log" 1>&- || status=$?
report low-fd "$status"
for name in nocancel unspec udp-disconnect callers overflow; do
status=0
: > "$out/$name.log"
EAS_EGRESS_GUARD_LOG="$out/$name.log" \
"$out/guard-insert-test" "$name" "$out/$name.log" || status=$?
report "$name" "$status"
done
if [[ $failures -ne 0 ]]; then
echo "egress-guard guard tests: $failures failure(s)"
exit 1
fi
echo "egress-guard guard tests: all passed"
@@ -0,0 +1,7 @@
#!/usr/bin/env bash
# Builds the policy unit tests for the host (macOS) and runs them.
set -euo pipefail
cd "$(dirname "$0")/.."
out=$(mktemp -d)
clang -std=c11 -Wall -Wextra -Werror -o "$out/policy_test" policy.c tests/policy_test.c
"$out/policy_test"
@@ -0,0 +1,130 @@
// Regenerates src/steps/utils/appiumCommands.generated.ts from Appium and the
// XCUITest and UiAutomator2 drivers, so TypeScript can verify that we handle
// every command that can appear in Appium event timings.
//
// Appium and its drivers are NOT dependencies of this package. Like repack, we
// install them on demand into a temp directory just for this script, so nothing
// heavy ships with @expo/build-tools.
//
// Run with: yarn workspace @expo/build-tools generate-appium-commands
const { execFileSync } = require('node:child_process');
const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');
const { pathToFileURL } = require('node:url');
// Track the Appium 3 line, matching the appium@^3 the worker installs for
// simulator sessions (see startAppiumRemoteSession.ts).
const APPIUM_VERSION = '^3';
const DRIVERS = [
{
packageName: 'appium-xcuitest-driver',
exportName: 'XCUITestDriver',
typeName: 'XCUITestAppiumCommand',
},
{
packageName: 'appium-uiautomator2-driver',
exportName: 'AndroidUiautomator2Driver',
typeName: 'UiAutomator2AppiumCommand',
},
];
function renderCommandType(typeName, commands) {
return `export type ${typeName} =\n${commands.map(command => ` | ${serializeStringLiteral(command)}`).join('\n')};`;
}
function serializeStringLiteral(value) {
const jsonContents = JSON.stringify(value).slice(1, -1).replaceAll('\\"', '"');
return `'${jsonContents.replaceAll("'", "\\'")}'`;
}
function readRouteCommands(methodMap) {
return Object.values(methodMap).flatMap(route =>
Object.values(route).map(method => method.command)
);
}
async function readDriverCommands({ packageName, exportName }, sandbox) {
const packageJsonPath = require.resolve(`${packageName}/package.json`, { paths: [sandbox] });
const packageJson = JSON.parse(fs.readFileSync(packageJsonPath, 'utf8'));
const modulePath = path.join(path.dirname(packageJsonPath), packageJson.exports['.'].import);
const driverModule = await import(pathToFileURL(modulePath).href);
const Driver = driverModule[exportName];
return [
...readRouteCommands(Driver.newMethodMap),
...Object.values(Driver.executeMethodMap).map(method => method.command),
];
}
async function main() {
const sandbox = fs.mkdtempSync(path.join(os.tmpdir(), 'appium-command-generator-'));
try {
fs.writeFileSync(
path.join(sandbox, 'package.json'),
`${JSON.stringify({ name: 'appium-command-generator-sandbox', private: true })}\n`
);
execFileSync(
'npm',
[
'install',
'--prefix',
sandbox,
`appium@${APPIUM_VERSION}`,
...DRIVERS.map(({ packageName }) => packageName),
],
{ stdio: 'inherit' }
);
// eslint-disable-next-line @typescript-eslint/no-var-requires
const { ALL_COMMANDS } = require(require.resolve('@appium/base-driver', { paths: [sandbox] }));
const baseDriverCommands = [...new Set(ALL_COMMANDS)].sort();
const baseDriverCommandSet = new Set(baseDriverCommands);
const driverCommands = await Promise.all(
DRIVERS.map(async driver => ({
typeName: driver.typeName,
commands: [...new Set(await readDriverCommands(driver, sandbox))]
.filter(command => !baseDriverCommandSet.has(command))
.sort(),
}))
);
const commands = [
...new Set([...baseDriverCommands, ...driverCommands.flatMap(({ commands }) => commands)]),
];
const outputPath = path.join(
__dirname,
'..',
'src',
'steps',
'utils',
'appiumCommands.generated.ts'
);
const contents = `// AUTO-GENERATED. Do not edit by hand.
// Run \`yarn workspace @expo/build-tools generate-appium-commands\` to refresh this
// list from Appium, XCUITest, and UiAutomator2 (installed on demand by the generator).
${renderCommandType('BaseDriverAppiumCommand', baseDriverCommands)}
${driverCommands
.map(({ typeName, commands: commandsForDriver }) =>
renderCommandType(typeName, commandsForDriver)
)
.join('\n\n')}
export type AppiumCommand =
| BaseDriverAppiumCommand
| XCUITestAppiumCommand
| UiAutomator2AppiumCommand;
`;
fs.writeFileSync(outputPath, contents);
// eslint-disable-next-line no-console
console.log(`Wrote ${commands.length} Appium commands to ${outputPath}`);
} finally {
fs.rmSync(sandbox, { recursive: true, force: true });
}
}
main().catch(error => {
// eslint-disable-next-line no-console
console.error(error);
process.exitCode = 1;
});
@@ -1,4 +1,4 @@
import bunyan from 'bunyan';
import type { bunyan } from '@expo/logger';
export function createLogger(): bunyan {
const logger = {
@@ -11,6 +11,19 @@ jest.mock('../utils/stepMetrics');
const mockUploadStepMetricsToWwwAsync = jest.mocked(uploadStepMetricsToWwwAsync);
describe(CustomBuildContext, () => {
it('copies the MCP server URL from the build context', () => {
const ctx = new BuildContext(createTestIosJob(), {
env: { __API_SERVER_URL: 'http://api.expo.test' },
logBuffer: { getLogs: () => [], getPhaseLogs: () => [] },
logger: createMockLogger(),
uploadArtifact: jest.fn(),
workingdir: '',
mcpServerUrl: 'ws://localhost:8787',
});
expect(new CustomBuildContext(ctx).mcpServerUrl).toBe('ws://localhost:8787');
});
it('should not lose workflowInterpolationContext', () => {
const contextUploadArtifact = jest.fn();
const ctx = new BuildContext(
@@ -4,9 +4,11 @@ import { BuildStepGlobalContext, StepsConfigParser } from '@expo/steps';
import { runGenericJobAsync } from '../generic';
import { CustomBuildContext } from '../customBuildContext';
import { uploadJobOutputsToWwwAsync } from '../utils/outputs';
import { stopLocalEgressResourcesAsync } from '../steps/utils/localEgress';
jest.mock('../customBuildContext');
jest.mock('../utils/outputs');
jest.mock('../steps/utils/localEgress', () => ({ stopLocalEgressResourcesAsync: jest.fn() }));
jest.mock('../common/projectSources');
jest.mock('../steps/easFunctions', () => ({ getEasFunctions: jest.fn().mockReturnValue([]) }));
jest.mock('../steps/easFunctionGroups', () => ({
@@ -58,6 +60,25 @@ describe(runGenericJobAsync, () => {
mockUploadJobOutputsToWwwAsync.mockResolvedValue(undefined);
});
it.each(['success', 'failure'])(
'releases local egress after workflow %s, before uploading outputs',
async outcome => {
const executeAsync =
outcome === 'success'
? jest.fn().mockResolvedValue(undefined)
: jest.fn().mockRejectedValue(new Error(outcome));
(StepsConfigParser as unknown as jest.Mock).mockImplementation(() => ({
parseAsync: jest.fn().mockResolvedValue({ executeAsync }),
}));
const { runResult } = await runGenericJobAsync(mockCtx);
expect(runResult.ok).toBe(outcome === 'success');
expect(stopLocalEgressResourcesAsync).toHaveBeenCalledWith(mockCtx.logger);
expect(jest.mocked(stopLocalEgressResourcesAsync).mock.invocationCallOrder[0]).toBeLessThan(
mockUploadJobOutputsToWwwAsync.mock.invocationCallOrder[0]
);
}
);
it('awaits drainPendingMetricUploads in COMPLETE_JOB phase', async () => {
const mockWorkflow = { executeAsync: jest.fn().mockResolvedValue(undefined) };
(StepsConfigParser as unknown as jest.Mock).mockImplementation(() => ({
+2 -2
View File
@@ -98,7 +98,7 @@ async function buildInnerAsync(
await restoreCcacheAsync({
logger: ctx.logger,
workingDirectory,
platform: ctx.job.platform,
target: { platform: ctx.job.platform },
env: ctx.env,
secrets: ctx.job.secrets,
});
@@ -238,7 +238,7 @@ async function buildInnerAsync(
await saveCcacheAsync({
logger: ctx.logger,
workingDirectory,
platform: ctx.job.platform,
target: { platform: ctx.job.platform },
evictUsedBefore,
env: ctx.env,
secrets: ctx.job.secrets,
+20 -4
View File
@@ -18,8 +18,12 @@ import { downloadApplicationArchiveAsync } from '../ios/resign';
import { resolveArtifactPath, resolveBuildConfiguration, resolveScheme } from '../ios/resolve';
import { Sentry } from '../sentry';
import { parseAndReportXcactivitylog } from '../steps/utils/ios/xcactivitylog';
import { cacheStatsAsync, restoreCcacheAsync } from '../steps/functions/restoreBuildCache';
import { saveCcacheAsync } from '../steps/functions/saveBuildCache';
import {
cacheStatsAsync,
restoreCcacheAsync,
restoreCocoapodsCacheAsync,
} from '../steps/functions/restoreBuildCache';
import { saveCcacheAsync, saveCocoapodsCacheAsync } from '../steps/functions/saveBuildCache';
import { uploadApplicationArchive } from '../utils/artifacts';
import {
configureExpoUpdatesIfInstalledAsync,
@@ -103,7 +107,13 @@ async function buildInnerAsync(
await restoreCcacheAsync({
logger: ctx.logger,
workingDirectory,
platform: ctx.job.platform,
target: { platform: ctx.job.platform, simulator: ctx.job.simulator === true },
env: ctx.env,
secrets: ctx.job.secrets,
});
await restoreCocoapodsCacheAsync({
logger: ctx.logger,
workingDirectory,
env: ctx.env,
secrets: ctx.job.secrets,
});
@@ -245,11 +255,17 @@ async function buildInnerAsync(
await saveCcacheAsync({
logger: ctx.logger,
workingDirectory,
platform: ctx.job.platform,
target: { platform: ctx.job.platform, simulator: ctx.job.simulator === true },
evictUsedBefore,
env: ctx.env,
secrets: ctx.job.secrets,
});
await saveCocoapodsCacheAsync({
logger: ctx.logger,
workingDirectory,
env: ctx.env,
secrets: ctx.job.secrets,
});
});
await ctx.runBuildPhase(BuildPhase.CACHE_STATS, async () => {
@@ -70,6 +70,48 @@ describe('easBuildInternal', () => {
expect(spawn).not.toHaveBeenCalled();
});
it('overrides the minimum release age only for the internal config command', async () => {
const logger = {
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
child: jest.fn(),
} as any;
const ctx = {
logger,
env: {
FOO: 'bar',
NPM_CONFIG_MIN_RELEASE_AGE: '365',
},
job: { platform: 'ios', buildProfile: 'production' },
} as any;
jest.mocked(resolveEasCommandPrefixAndEnvAsync).mockResolvedValueOnce({
cmd: 'npx',
args: ['-y', 'eas-cli@latest'],
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
jest.mocked(spawn).mockResolvedValueOnce({
stdout: Buffer.from(JSON.stringify({ buildProfile: { env: {} } })),
} as any);
await resolveEnvFromBuildProfileAsync(ctx, { cwd: '/tmp/project' });
expect(spawn).toHaveBeenCalledWith(
'npx',
expect.any(Array),
expect.objectContaining({
env: {
FOO: 'bar',
NPM_CONFIG_MIN_RELEASE_AGE: '0',
},
})
);
expect(ctx.env).toEqual({
FOO: 'bar',
NPM_CONFIG_MIN_RELEASE_AGE: '365',
});
});
it('passes --refresh-ad-hoc-provisioning-profile to build:internal for iOS jobs with refreshAdHocProvisioningProfile', async () => {
const logger = {
info: jest.fn(),
@@ -3,7 +3,10 @@ import spawn, { SpawnPromise, SpawnResult } from '@expo/turtle-spawn';
import { createMockLogger } from '../../__tests__/utils/logger';
import { Sentry } from '../../sentry';
import { PackageManager } from '../../utils/packageManager';
import { installDependenciesWithNpmCacheFallbackAsync } from '../installDependencies';
import {
installDependenciesAsync,
installDependenciesWithNpmCacheFallbackAsync,
} from '../installDependencies';
jest.mock('@expo/turtle-spawn', () => jest.fn());
jest.mock('../../sentry', () => ({
@@ -236,6 +239,85 @@ describe(installDependenciesWithNpmCacheFallbackAsync, () => {
});
});
describe(installDependenciesAsync, () => {
beforeEach(() => {
jest.clearAllMocks();
jest.mocked(spawn).mockReturnValue(createSpawnPromise(Promise.resolve(createSpawnResult())));
});
it('installs only the selected workspace when EAS_PNPM_FILTER_WORKSPACE is set', async () => {
const logger = createMockLogger();
await installDependenciesAsync({
packageManager: PackageManager.PNPM,
env: { EAS_PNPM_FILTER_WORKSPACE: 'my-app' },
logger,
cwd: '/tmp/build',
useFrozenLockfile: true,
});
expect(spawn).toHaveBeenCalledWith(
'pnpm',
['install', '--filter', 'my-app', '--frozen-lockfile'],
expect.objectContaining({ cwd: '/tmp/build' })
);
});
it('installs all workspaces when EAS_PNPM_FILTER_WORKSPACE is not set', async () => {
const logger = createMockLogger();
await installDependenciesAsync({
packageManager: PackageManager.PNPM,
env: {},
logger,
cwd: '/tmp/build',
useFrozenLockfile: true,
});
expect(spawn).toHaveBeenCalledWith(
'pnpm',
['install', '--frozen-lockfile'],
expect.objectContaining({ cwd: '/tmp/build' })
);
});
it('installs only the selected workspace when EAS_BUN_FILTER_WORKSPACE is set', async () => {
const logger = createMockLogger();
await installDependenciesAsync({
packageManager: PackageManager.BUN,
env: { EAS_BUN_FILTER_WORKSPACE: 'my-app' },
logger,
cwd: '/tmp/build',
useFrozenLockfile: true,
});
expect(spawn).toHaveBeenCalledWith(
'bun',
['install', '--filter', 'my-app', '--frozen-lockfile'],
expect.objectContaining({ cwd: '/tmp/build' })
);
});
it('installs all workspaces when EAS_BUN_FILTER_WORKSPACE is not set', async () => {
const logger = createMockLogger();
await installDependenciesAsync({
packageManager: PackageManager.BUN,
env: {},
logger,
cwd: '/tmp/build',
useFrozenLockfile: true,
});
expect(spawn).toHaveBeenCalledWith(
'bun',
['install', '--frozen-lockfile'],
expect.objectContaining({ cwd: '/tmp/build' })
);
});
});
function createSpawnPromise(result: Promise<SpawnResult>): SpawnPromise<SpawnResult> {
return Object.assign(result, {
child: {
@@ -2,10 +2,13 @@ import {
ArchiveSourceType,
BuildMode,
BuildTrigger,
ErrorCode,
Job,
Platform,
SystemError,
Workflow,
} from '@expo/eas-build-job';
import spawn from '@expo/turtle-spawn';
import { randomBytes, randomUUID } from 'crypto';
import { vol } from 'memfs';
import fetch, { Response } from 'node-fetch';
@@ -14,7 +17,7 @@ import { setTimeout } from 'timers/promises';
import { createMockLogger } from '../../__tests__/utils/logger';
import { BuildContext } from '../../context';
import { shallowCloneRepositoryAsync } from '../git';
import { prepareProjectSourcesAsync } from '../projectSources';
import { prepareProjectSourcesAsync, unpackTarGzAsync } from '../projectSources';
jest.mock('@expo/turtle-spawn');
jest.mock('node-fetch');
@@ -23,7 +26,28 @@ jest.mock('@expo/downloader');
jest.mock('@urql/core');
describe('projectSources', () => {
it('should use the refreshed repository URL', async () => {
it('makes extracted project sources readable and writable by the worker', async () => {
const logger = createMockLogger();
await unpackTarGzAsync({
source: '/workingdir/project.tar.gz',
destination: '/workingdir/build',
logger,
});
expect(spawn).toHaveBeenNthCalledWith(
1,
'tar',
['-C', '/workingdir/build', '--strip-components', '1', '-zxf', '/workingdir/project.tar.gz'],
{ logger }
);
expect(spawn).toHaveBeenNthCalledWith(2, 'chmod', ['-R', 'u+rwX', '/workingdir/build'], {
logger,
});
expect(logger.info).toHaveBeenCalledWith('Normalizing project source permissions');
});
it.each([true, false])('uses the refreshed URL (legacy URL: %s)', async includeRepositoryUrl => {
const robotAccessToken = randomUUID();
const buildId = randomUUID();
await vol.promises.mkdir('/workingdir/environment-secrets/', { recursive: true });
@@ -39,7 +63,9 @@ describe('projectSources', () => {
appId: randomUUID(),
projectArchive: {
type: ArchiveSourceType.GIT,
repositoryUrl: 'https://x-access-token:[email protected]/expo/eas-cli.git',
...(includeRepositoryUrl
? { repositoryUrl: 'https://x-access-token:[email protected]/expo/eas-cli.git' }
: {}),
gitRef: 'refs/heads/main',
gitCommitHash,
},
@@ -89,102 +115,88 @@ describe('projectSources', () => {
);
});
it('should fallback to the original repository URL if the refresh fails', async () => {
const robotAccessToken = randomUUID();
const buildId = randomUUID();
await vol.promises.mkdir('/workingdir/environment-secrets/', { recursive: true });
it.each(['http', 'network', 'json', 'schema', 'missing-url', 'PATH', 'GCS', 'R2'])(
'throws a system error if refresh fails (%s)',
async failure => {
const robotAccessToken = randomUUID();
const buildId = randomUUID();
await vol.promises.mkdir('/workingdir/environment-secrets/', { recursive: true });
const ctx = new BuildContext(
{
triggeredBy: BuildTrigger.GIT_BASED_INTEGRATION,
type: Workflow.MANAGED,
mode: BuildMode.BUILD,
initiatingUserId: randomUUID(),
appId: randomUUID(),
projectArchive: {
type: ArchiveSourceType.GIT,
repositoryUrl: 'https://x-access-token:[email protected]/expo/eas-cli.git',
gitRef: 'refs/heads/main',
gitCommitHash: randomBytes(20).toString('hex'),
},
platform: Platform.IOS,
secrets: {
robotAccessToken,
environmentSecrets: [],
},
} as Job,
{
env: {
__API_SERVER_URL: 'https://api.expo.dev',
EXPO_TOKEN: robotAccessToken,
EAS_BUILD_ID: buildId,
EAS_BUILD_RUNNER: 'eas-build',
},
workingdir: '/workingdir',
logger: createMockLogger(),
logBuffer: { getLogs: () => [], getPhaseLogs: () => [] },
uploadArtifact: jest.fn(),
const ctx = new BuildContext(
{
triggeredBy: BuildTrigger.GIT_BASED_INTEGRATION,
type: Workflow.MANAGED,
mode: BuildMode.BUILD,
initiatingUserId: randomUUID(),
appId: randomUUID(),
projectArchive: {
type: ArchiveSourceType.GIT,
gitRef: 'refs/heads/main',
gitCommitHash: randomBytes(20).toString('hex'),
},
platform: Platform.IOS,
secrets: {
robotAccessToken,
environmentSecrets: [],
},
} as Job,
{
env: {
__API_SERVER_URL: 'https://api.expo.dev',
EXPO_TOKEN: robotAccessToken,
EAS_BUILD_ID: buildId,
EAS_BUILD_RUNNER: 'eas-build',
},
workingdir: '/workingdir',
logger: createMockLogger(),
logBuffer: { getLogs: () => [], getPhaseLogs: () => [] },
uploadArtifact: jest.fn(),
}
);
const fetchMock = jest.mocked(fetch);
const cause = new Error('Source request failed');
fetchMock.mockImplementation(async () => {
if (failure === 'network') {
throw cause;
}
return {
ok: failure !== 'http',
status: failure === 'http' ? 500 : 200,
json: async () => {
if (failure === 'json') {
throw cause;
}
if (['PATH', 'GCS', 'R2'].includes(failure)) {
return { data: { type: failure, path: '/project.tar.gz', bucketKey: 'project' } };
}
if (failure === 'missing-url') {
return {
data: {
type: ArchiveSourceType.GIT,
gitRef: 'refs/heads/main',
gitCommitHash: randomBytes(20).toString('hex'),
},
};
}
return { data: { repository_url: 'https://github.com/expo/eas-cli.git' } };
},
} as Response;
});
const result = prepareProjectSourcesAsync(ctx, ctx.buildDirectory);
await expect(result).rejects.toBeInstanceOf(SystemError);
await expect(result).rejects.toMatchObject({
errorCode: ErrorCode.SERVER_ERROR,
cause: expect.any(Error),
});
if (failure === 'network') {
await expect(result).rejects.toMatchObject({ cause });
}
);
const fetchMock = jest.mocked(fetch);
fetchMock.mockImplementation(
async () =>
({
ok: false,
text: async () => 'Failed to generate repository URL',
}) as Response
);
await prepareProjectSourcesAsync(ctx, ctx.buildDirectory);
expect(shallowCloneRepositoryAsync).toHaveBeenLastCalledWith(
expect.objectContaining({
archiveSource: {
...ctx.job.projectArchive,
repositoryUrl: 'https://x-access-token:[email protected]/expo/eas-cli.git',
},
})
);
fetchMock.mockImplementation(
async () =>
({
ok: false,
json: async () => ({
// repositoryUrl is the right key
repository_url: 'https://x-access-token:[email protected]/expo/eas-cli.git',
}),
}) as Response
);
await prepareProjectSourcesAsync(ctx, ctx.buildDirectory);
expect(shallowCloneRepositoryAsync).toHaveBeenLastCalledWith(
expect.objectContaining({
archiveSource: {
...ctx.job.projectArchive,
repositoryUrl: 'https://x-access-token:[email protected]/expo/eas-cli.git',
},
})
);
fetchMock.mockImplementation(
async () =>
({
ok: false,
json: () => Promise.reject(new Error('Failed to generate repository URL')),
}) as Response
);
await prepareProjectSourcesAsync(ctx, ctx.buildDirectory);
expect(shallowCloneRepositoryAsync).toHaveBeenLastCalledWith(
expect.objectContaining({
archiveSource: {
...ctx.job.projectArchive,
repositoryUrl: 'https://x-access-token:[email protected]/expo/eas-cli.git',
},
})
);
expect(shallowCloneRepositoryAsync).toHaveBeenCalledTimes(3);
}, 15_000);
expect(shallowCloneRepositoryAsync).not.toHaveBeenCalled();
expect(spawn).not.toHaveBeenCalled();
},
15_000
);
it('should retry fetching the repository URL', async () => {
const robotAccessToken = randomUUID();
@@ -202,7 +214,6 @@ describe('projectSources', () => {
appId: randomUUID(),
projectArchive: {
type: ArchiveSourceType.GIT,
repositoryUrl: 'https://x-access-token:[email protected]/expo/eas-cli.git',
gitRef: 'refs/heads/main',
gitCommitHash,
},
@@ -259,7 +270,7 @@ describe('projectSources', () => {
);
}, 15_000);
it(`should fallback to the original repository URL if we're missing some config`, async () => {
it(`throws a system error if refresh configuration is missing`, async () => {
const robotAccessToken = randomUUID();
await vol.promises.mkdir('/workingdir/environment-secrets/', { recursive: true });
const logger = createMockLogger();
@@ -273,7 +284,6 @@ describe('projectSources', () => {
appId: randomUUID(),
projectArchive: {
type: ArchiveSourceType.GIT,
repositoryUrl: 'https://x-access-token:[email protected]/expo/eas-cli.git',
gitRef: 'refs/heads/main',
gitCommitHash: randomBytes(20).toString('hex'),
},
@@ -297,12 +307,11 @@ describe('projectSources', () => {
}
);
await prepareProjectSourcesAsync(ctx, ctx.buildDirectory);
expect(logger.error).toHaveBeenCalledWith(
{ err: expect.any(Error) },
'Failed to refresh project archive, falling back to the original one'
);
await expect(prepareProjectSourcesAsync(ctx, ctx.buildDirectory)).rejects.toMatchObject({
errorCode: ErrorCode.SERVER_ERROR,
cause: new Error('EAS_BUILD_ID is not set'),
});
expect(shallowCloneRepositoryAsync).not.toHaveBeenCalled();
});
describe('uploadProjectMetadataAsFireAndForget', () => {
@@ -382,6 +391,10 @@ describe('projectSources', () => {
const fetchMock = jest.mocked(fetch);
fetchMock.mockImplementation(async () => ({ ok: true }) as Response);
fetchMock.mockResolvedValueOnce({
ok: true,
json: async () => ({ data: ctx.job.projectArchive }),
} as Response);
// Call prepareProjectSourcesAsync and don't await metadata upload
await prepareProjectSourcesAsync(ctx, ctx.buildDirectory);
@@ -495,6 +508,10 @@ describe('projectSources', () => {
const fetchMock = jest.mocked(fetch);
fetchMock.mockResolvedValue({ ok: true } as Response);
fetchMock.mockResolvedValueOnce({
ok: true,
json: async () => ({ data: ctx.job.projectArchive }),
} as Response);
const startTime = Date.now();
await prepareProjectSourcesAsync(ctx, ctx.buildDirectory);
@@ -559,6 +576,10 @@ describe('projectSources', () => {
const fetchMock = jest.mocked(fetch);
fetchMock.mockResolvedValue({ ok: true } as Response);
fetchMock.mockResolvedValueOnce({
ok: true,
json: async () => ({ data: ctx.job.projectArchive }),
} as Response);
// Should not throw even though upload will fail
await expect(prepareProjectSourcesAsync(ctx, ctx.buildDirectory)).resolves.not.toThrow();
@@ -573,3 +594,56 @@ describe('projectSources', () => {
});
});
});
describe('local project sources', () => {
function createContext(type: ArchiveSourceType): BuildContext<Job> {
return new BuildContext(
{
platform: Platform.IOS,
projectArchive: { type, path: '/source.tar.gz' },
} as Job,
{
env: { EAS_BUILD_RUNNER: 'local-build-plugin', __API_SERVER_URL: 'https://api.expo.dev' },
workingdir: '/workingdir',
logger: createMockLogger(),
logBuffer: { getLogs: () => [], getPhaseLogs: () => [] },
uploadArtifact: jest.fn(),
}
);
}
it('copies and unpacks a PATH source without fetching from www', async () => {
await vol.promises.mkdir('/workingdir', { recursive: true });
await vol.promises.writeFile('/source.tar.gz', 'local archive');
const ctx = createContext(ArchiveSourceType.PATH);
await expect(prepareProjectSourcesAsync(ctx, ctx.buildDirectory)).resolves.toEqual({
handled: true,
});
expect(await vol.promises.readFile('/workingdir/project.tar.gz', 'utf8')).toBe('local archive');
expect(spawn).toHaveBeenCalledWith(
'tar',
['-C', ctx.buildDirectory, '--strip-components', '1', '-zxf', '/workingdir/project.tar.gz'],
{ logger: ctx.logger }
);
expect(fetch).not.toHaveBeenCalled();
expect(shallowCloneRepositoryAsync).not.toHaveBeenCalled();
});
it.each(
Object.values(ArchiveSourceType).filter(
type => type !== ArchiveSourceType.PATH && type !== ArchiveSourceType.NONE
)
)('rejects local %s sources before fetching or unpacking', async type => {
const ctx = createContext(type);
const result = prepareProjectSourcesAsync(ctx, ctx.buildDirectory);
await expect(result).rejects.toBeInstanceOf(SystemError);
await expect(result).rejects.toMatchObject({
errorCode: 'SERVER_ERROR',
trackingCode: 'INVALID_LOCAL_PROJECT_SOURCE',
message: `Expected a PATH project source for a local build, received ${type}.`,
});
expect(fetch).not.toHaveBeenCalled();
expect(spawn).not.toHaveBeenCalled();
expect(shallowCloneRepositoryAsync).not.toHaveBeenCalled();
});
});
+1 -1
View File
@@ -10,7 +10,7 @@ export async function shallowCloneRepositoryAsync({
destinationDirectory,
}: {
logger: bunyan;
archiveSource: ArchiveSource & { type: ArchiveSourceType.GIT };
archiveSource: ArchiveSource & { type: ArchiveSourceType.GIT; repositoryUrl: string };
destinationDirectory: string;
}): Promise<void> {
const { repositoryUrl } = archiveSource;
@@ -32,7 +32,11 @@ export async function installDependenciesAsync({
break;
}
case PackageManager.PNPM: {
args = ['install', useFrozenLockfile ? '--frozen-lockfile' : '--no-frozen-lockfile'];
args = [
'install',
...(env['EAS_PNPM_FILTER_WORKSPACE'] ? ['--filter', env['EAS_PNPM_FILTER_WORKSPACE']] : []),
useFrozenLockfile ? '--frozen-lockfile' : '--no-frozen-lockfile',
];
break;
}
case PackageManager.YARN: {
@@ -58,7 +62,11 @@ export async function installDependenciesAsync({
break;
}
case PackageManager.BUN:
args = ['install', ...(useFrozenLockfile ? ['--frozen-lockfile'] : [])];
args = [
'install',
...(env['EAS_BUN_FILTER_WORKSPACE'] ? ['--filter', env['EAS_BUN_FILTER_WORKSPACE']] : []),
...(useFrozenLockfile ? ['--frozen-lockfile'] : []),
];
break;
default:
throw new Error(`Unsupported package manager: ${packageManager}`);
@@ -1,5 +1,5 @@
import downloadFile from '@expo/downloader';
import { ArchiveSource, ArchiveSourceSchemaZ, ArchiveSourceType, Job } from '@expo/eas-build-job';
import { ArchiveSourceType, Job, SystemError } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import { asyncResult } from '@expo/results';
import spawn from '@expo/turtle-spawn';
@@ -19,33 +19,30 @@ export async function prepareProjectSourcesAsync<TJob extends Job>(
destinationDirectory: string
): // Return type required to make switch exhaustive.
Promise<{ handled: boolean }> {
let projectArchive: ArchiveSource = ctx.job.projectArchive;
if (ctx.isLocal) {
console.warn('Local build, skipping project archive refresh');
} else {
const projectArchiveResult = await asyncResult(fetchProjectArchiveSourceAsync(ctx));
if (!projectArchiveResult.ok) {
ctx.logger.error(
{ err: projectArchiveResult.reason },
'Failed to refresh project archive, falling back to the original one'
);
}
projectArchive = projectArchiveResult.value ?? ctx.job.projectArchive;
if (ctx.job.projectArchive.type === ArchiveSourceType.NONE) {
return { handled: true };
}
if (ctx.isLocal) {
if (ctx.job.projectArchive.type !== ArchiveSourceType.PATH) {
throw new SystemError(
`Expected a PATH project source for a local build, received ${ctx.job.projectArchive.type}.`,
{ trackingCode: 'INVALID_LOCAL_PROJECT_SOURCE' }
);
}
await prepareProjectSourcesLocallyAsync(ctx, ctx.job.projectArchive.path, destinationDirectory);
return { handled: true };
}
const projectArchiveResult = await asyncResult(fetchProjectArchiveSourceAsync(ctx));
if (!projectArchiveResult.ok) {
throw new SystemError('Failed to fetch project sources. Re-run the job.', {
cause: projectArchiveResult.reason,
});
}
const projectArchive = projectArchiveResult.value;
switch (projectArchive.type) {
case ArchiveSourceType.R2:
case ArchiveSourceType.GCS: {
throw new Error('Remote project sources should be resolved earlier to URL');
}
case ArchiveSourceType.PATH: {
await prepareProjectSourcesLocallyAsync(ctx, projectArchive.path, destinationDirectory); // used in eas build --local
return { handled: true };
}
case ArchiveSourceType.NONE: {
// May be used in no-sources jobs like submission jobs.
return { handled: true };
@@ -110,7 +107,7 @@ async function prepareProjectSourcesLocallyAsync<TJob extends Job>(
});
}
async function unpackTarGzAsync({
export async function unpackTarGzAsync({
logger,
source,
destination,
@@ -122,6 +119,8 @@ async function unpackTarGzAsync({
await spawn('tar', ['-C', destination, '--strip-components', '1', '-zxf', source], {
logger,
});
logger.info('Normalizing project source permissions');
await spawn('chmod', ['-R', 'u+rwX', destination], { logger });
}
function uploadProjectMetadataAsFireAndForget(
@@ -247,7 +246,7 @@ async function uploadProjectMetadataAsync(
}
}
async function fetchProjectArchiveSourceAsync(ctx: BuildContext<Job>): Promise<ArchiveSource> {
async function fetchProjectArchiveSourceAsync(ctx: BuildContext<Job>) {
const taskId = nullthrows(ctx.env.EAS_BUILD_ID, 'EAS_BUILD_ID is not set');
const expoApiServerURL = nullthrows(ctx.env.__API_SERVER_URL, '__API_SERVER_URL is not set');
const robotAccessToken = nullthrows(
@@ -285,7 +284,20 @@ async function fetchProjectArchiveSourceAsync(ctx: BuildContext<Job>): Promise<A
);
}
const dataResult = z.object({ data: ArchiveSourceSchemaZ }).safeParse(jsonResult.value);
const dataResult = z
.object({
data: z.discriminatedUnion('type', [
z.object({
type: z.literal(ArchiveSourceType.GIT),
repositoryUrl: z.string().url(),
gitRef: z.string().nullable(),
gitCommitHash: z.string(),
}),
z.object({ type: z.literal(ArchiveSourceType.URL), url: z.string().url() }),
z.object({ type: z.literal(ArchiveSourceType.NONE) }),
]),
})
.safeParse(jsonResult.value);
if (!dataResult.success) {
throw new Error(
`Unexpected data from server (${response.status}): ${z.prettifyError(dataResult.error)}`
+3
View File
@@ -68,6 +68,7 @@ export interface BuildContextOptions {
skipNativeBuild?: boolean;
metadata?: Metadata;
expoApiV2BaseUrl?: string;
mcpServerUrl?: string;
}
export class SkipNativeBuildError extends Error {}
@@ -84,6 +85,7 @@ export class BuildContext<TJob extends Job = Job> {
) => void;
public readonly skipNativeBuild?: boolean;
public readonly expoApiV2BaseUrl?: string;
public readonly mcpServerUrl?: string;
public artifacts: Artifacts = {};
private readonly _isLocal: boolean;
@@ -112,6 +114,7 @@ export class BuildContext<TJob extends Job = Job> {
this._metadata = options.metadata;
this.skipNativeBuild = options.skipNativeBuild;
this.expoApiV2BaseUrl = options.expoApiV2BaseUrl;
this.mcpServerUrl = options.mcpServerUrl;
const environmentSecrets = this.getEnvironmentSecrets(job);
this._env = {
@@ -63,6 +63,7 @@ export class CustomBuildContext<TJob extends Job = Job> implements ExternalBuild
public readonly logger: bunyan;
public readonly graphqlClient: Client;
public readonly runtimeApi: BuilderRuntimeApi;
public readonly mcpServerUrl?: string;
public job: TJob;
public metadata?: Metadata;
@@ -84,6 +85,7 @@ export class CustomBuildContext<TJob extends Job = Job> implements ExternalBuild
this.runtimeApi = {
uploadArtifact: (...args) => buildCtx['uploadArtifact'](...args),
};
this.mcpServerUrl = buildCtx.mcpServerUrl;
this.expoApiV2BaseUrl = buildCtx.expoApiV2BaseUrl;
this.startTime = new Date();
}
+12 -1
View File
@@ -16,6 +16,7 @@ import { BuildContext } from './context';
import { CustomBuildContext } from './customBuildContext';
import { getEasFunctionGroups } from './steps/easFunctionGroups';
import { getEasFunctions } from './steps/easFunctions';
import { stopLocalEgressResourcesAsync } from './steps/utils/localEgress';
import { uploadJobOutputsToWwwAsync } from './utils/outputs';
import { retryAsync } from './utils/retry';
@@ -82,7 +83,17 @@ export async function runGenericJobAsync(
}
});
const runResult = await asyncResult(workflow.executeAsync());
const runResult = await asyncResult(
(async () => {
try {
await workflow.executeAsync();
} finally {
// The session-owning step may never run if simulator boot/setup fails,
// or may time out while resource acquisition is still in flight.
await stopLocalEgressResourcesAsync(ctx.logger);
}
})()
);
await ctx.runBuildPhase(BuildPhase.COMPLETE_JOB, async () => {
const results = await Promise.allSettled([
+5
View File
@@ -5,6 +5,8 @@ import RemoteLoggerStream from './logging/RemoteLoggerStream';
export { Builders, HttpLogStream, RemoteLoggerStream };
export { uploadWithSignedUrl } from './storage/uploadWithSignedUrl';
export type { SignedUrl, UploadWithSignedUrlParams } from './storage/uploadWithSignedUrl';
export { downloadCacheAsync } from './steps/functions/restoreCache';
export { uploadCacheAsync } from './steps/functions/saveCache';
export {
ArtifactToUpload,
@@ -18,6 +20,9 @@ export {
export { PackageManager } from './utils/packageManager';
export * as TurtleSshSession from './utils/turtleSshSession';
export { formatSecondsForLog } from './utils/formatDuration';
export { findAndUploadXcodeBuildLogsAsync } from './ios/xcodeBuildLogs';
export { Hook, runHookIfPresent } from './utils/hooks';
@@ -0,0 +1,46 @@
import { SystemError } from '@expo/eas-build-job';
import spawn from '@expo/turtle-spawn';
import { getXcodeVersionAsync } from '../xcode';
jest.mock('@expo/turtle-spawn', () => ({
__esModule: true,
default: jest.fn(),
}));
const mockedSpawn = jest.mocked(spawn);
describe(getXcodeVersionAsync, () => {
it('returns the Xcode version', async () => {
mockedSpawn.mockResolvedValue({
stdout: 'Xcode 26.0\nBuild version 17A324\n',
} as any);
await expect(getXcodeVersionAsync({ env: process.env })).resolves.toBe('26.0');
expect(mockedSpawn).toHaveBeenCalledWith('xcodebuild', ['-version'], {
stdio: 'pipe',
env: process.env,
});
});
it('throws when the Xcode version cannot be parsed', async () => {
mockedSpawn.mockResolvedValue({ stdout: 'unexpected output' } as any);
await expect(getXcodeVersionAsync({ env: process.env })).rejects.toThrow(
'Failed to determine Xcode version'
);
});
it('wraps errors from xcodebuild in a SystemError', async () => {
const cause = new Error('xcode-select: error: invalid developer directory');
mockedSpawn.mockRejectedValue(cause);
const promise = getXcodeVersionAsync({ env: process.env });
await expect(promise).rejects.toBeInstanceOf(SystemError);
await expect(promise).rejects.toMatchObject({
message: 'Failed to get Xcode version',
cause,
});
});
});
+16
View File
@@ -0,0 +1,16 @@
import { SystemError } from '@expo/eas-build-job';
import spawn from '@expo/turtle-spawn';
export async function getXcodeVersionAsync({ env }: { env: NodeJS.ProcessEnv }): Promise<string> {
let stdout: string;
try {
({ stdout } = await spawn('xcodebuild', ['-version'], { stdio: 'pipe', env }));
} catch (error) {
throw new SystemError('Failed to get Xcode version', { cause: error });
}
const version = /^Xcode\s+(\d+(?:\.\d+)*)$/m.exec(stdout)?.[1];
if (!version) {
throw new SystemError(`Failed to determine Xcode version from: ${stdout.trim()}`);
}
return version;
}
+10 -2
View File
@@ -19,8 +19,10 @@ import { generateGymfileFromTemplateFunction } from './functions/generateGymfile
import { createGetCredentialsForBuildTriggeredByGithubIntegration } from './functions/getCredentialsForBuildTriggeredByGitHubIntegration';
import { injectAndroidCredentialsFunction } from './functions/injectAndroidCredentials';
import { createInstallMaestroBuildFunction } from './functions/installMaestro';
import { createInstallBuildFunction } from './functions/installBuild';
import { createInstallNodeModulesBuildFunction } from './functions/installNodeModules';
import { createInstallPodsBuildFunction } from './functions/installPods';
import { createLaunchApplicationFunction } from './functions/launchApplication';
import { createPrebuildBuildFunction } from './functions/prebuild';
import { createReadAppConfigBuildFunction } from './functions/readAppConfig';
import { createReadIpaInfoBuildFunction } from './functions/readIpaInfo';
@@ -49,8 +51,10 @@ import { createStartAppiumRemoteSessionBuildFunction } from './functions/startAp
import { createStartCuttlefishDeviceBuildFunction } from './functions/startCuttlefishDevice';
import { createStartIosSimulatorBuildFunction } from './functions/startIosSimulator';
import { createStartIosSimulatorRecordingsBuildFunction } from './functions/startIosSimulatorRecordings';
import { createStartServeSimRemoteSessionBuildFunction } from './functions/startServeSimRemoteSession';
import { createStartLocalEgressBuildFunction } from './functions/startLocalEgress';
import { createStartWebPreviewRemoteSessionBuildFunction } from './functions/startWebPreviewRemoteSession';
import { createStartServeSimMetricsBuildFunction } from './functions/startServeSimMetrics';
import { createStartSandboxBuildFunction } from './functions/startSandbox';
import { createCollectServeSimMetricsBuildFunction } from './functions/collectServeSimMetrics';
import { createUploadArtifactBuildFunction } from './functions/uploadArtifact';
import { createUploadDeviceRunSessionScreenRecordingsBuildFunction } from './functions/uploadDeviceRunSessionScreenRecordings';
@@ -74,6 +78,8 @@ export function getEasFunctions(ctx: CustomBuildContext): BuildFunction[] {
createPrebuildBuildFunction(),
createReadIpaInfoBuildFunction(),
createDownloadBuildFunction(ctx),
createInstallBuildFunction(),
createLaunchApplicationFunction(),
createEasExportBuildFunction(),
createEasDeployBuildFunction(),
createRepackBuildFunction(),
@@ -99,11 +105,13 @@ export function getEasFunctions(ctx: CustomBuildContext): BuildFunction[] {
createStartAndroidEmulatorBuildFunction(),
createStartCuttlefishDeviceBuildFunction(),
createStartIosSimulatorBuildFunction(),
createStartLocalEgressBuildFunction(),
createStartIosSimulatorRecordingsBuildFunction(),
createFinishIosSimulatorRecordingsBuildFunction(),
createUploadDeviceRunSessionScreenRecordingsBuildFunction(ctx),
createStartServeSimRemoteSessionBuildFunction(ctx),
createStartWebPreviewRemoteSessionBuildFunction(ctx),
createStartServeSimMetricsBuildFunction(),
createStartSandboxBuildFunction(ctx),
createCollectServeSimMetricsBuildFunction(ctx),
createInstallMaestroBuildFunction(),
@@ -162,6 +162,28 @@ describe(createEasBuildBuildFunctionGroup, () => {
expect(installPodsStep!.ctx.relativeWorkingDirectory).toBe('./ios');
});
it('restores and saves caches around an iOS simulator build', () => {
const buildToolsContext = createMockBuildToolsContext({
platform: Platform.IOS,
simulator: true,
});
const functionGroup = createEasBuildBuildFunctionGroup(buildToolsContext);
const globalCtx = createGlobalContextMock({ logger: createMockLogger() });
const steps = functionGroup.createBuildStepsFromFunctionGroupCall(globalCtx);
const stepNames = steps.map(step => step.displayName);
const restoreCacheIndex = stepNames.indexOf('Restore Cache');
const installPodsIndex = stepNames.indexOf('Install Pods');
const saveCacheIndex = stepNames.indexOf('Save Cache');
expect(restoreCacheIndex).toBeGreaterThan(-1);
expect(restoreCacheIndex).toBeLessThan(installPodsIndex);
expect(saveCacheIndex).toBeGreaterThan(installPodsIndex);
const restoreCacheStep = steps[restoreCacheIndex];
expect(restoreCacheStep.inputs?.find(input => input.id === 'simulator')?.rawValue).toBe(true);
});
it('sets working directory on all steps except checkout (Android with credentials)', () => {
const buildToolsContext = createMockBuildToolsContext({
platform: Platform.ANDROID,
@@ -98,6 +98,8 @@ function createStepsForIosSimulatorBuild({
buildToolsContext,
workingDirectory,
}: HelperFunctionsInput): BuildStep[] {
const evictUsedBefore = new Date();
const calculateEASUpdateRuntimeVersion =
calculateEASUpdateRuntimeVersionFunction().createBuildStepFromFunctionCall(globalCtx, {
id: 'calculate_eas_update_runtime_version',
@@ -106,6 +108,16 @@ function createStepsForIosSimulatorBuild({
const installPods = createInstallPodsBuildFunction().createBuildStepFromFunctionCall(globalCtx, {
workingDirectory: workingDirectory ? path.join(workingDirectory, './ios') : './ios',
});
const restoreCache = createRestoreBuildCacheFunction().createBuildStepFromFunctionCall(
globalCtx,
{
workingDirectory,
callInputs: {
platform: Platform.IOS,
simulator: true,
},
}
);
const configureEASUpdate =
configureEASUpdateIfInstalledFunction().createBuildStepFromFunctionCall(globalCtx, {
workingDirectory,
@@ -123,6 +135,16 @@ function createStepsForIosSimulatorBuild({
'${ steps.calculate_eas_update_runtime_version.resolved_eas_update_runtime_version }',
},
});
const saveCache = createSaveBuildCacheFunction(evictUsedBefore).createBuildStepFromFunctionCall(
globalCtx,
{
workingDirectory,
callInputs: {
platform: Platform.IOS,
simulator: true,
},
}
);
return [
createCheckoutBuildFunction().createBuildStepFromFunctionCall(globalCtx),
createSetUpNpmrcBuildFunction().createBuildStepFromFunctionCall(globalCtx, {
@@ -138,6 +160,7 @@ function createStepsForIosSimulatorBuild({
createPrebuildBuildFunction().createBuildStepFromFunctionCall(globalCtx, {
workingDirectory,
}),
restoreCache,
calculateEASUpdateRuntimeVersion,
installPods,
configureEASUpdate,
@@ -159,6 +182,10 @@ function createStepsForIosSimulatorBuild({
createFindAndUploadBuildArtifactsBuildFunction(
buildToolsContext
).createBuildStepFromFunctionCall(globalCtx, { workingDirectory }),
saveCache,
createCacheStatsBuildFunction().createBuildStepFromFunctionCall(globalCtx, {
workingDirectory,
}),
];
}
@@ -14,7 +14,6 @@ const projectTargetDirectory = '/workingdir/project';
const GIT_PROJECT_ARCHIVE = {
type: ArchiveSourceType.GIT,
repositoryUrl: 'https://x-access-token:[email protected]/expo/app.git',
gitRef: 'refs/heads/main',
gitCommitHash: '0123456789abcdef0123456789abcdef01234567',
};
@@ -0,0 +1,136 @@
import { spawnAsync } from '@expo/steps';
import { vol } from 'memfs';
import { createMockLogger } from '../../../__tests__/utils/logger';
import { Datadog } from '../../../datadog';
import {
compressCocoapodsCacheAsync,
getCocoapodsCachePaths,
resolveCocoapodsCacheKeyAsync,
restoreCocoapodsCacheArchiveAsync,
} from '../../../utils/cocoapodsCache';
import { downloadCacheAsync } from '../restoreCache';
import { restoreCocoapodsCacheAsync } from '../restoreBuildCache';
import { uploadCacheAsync } from '../saveCache';
import { saveCocoapodsCacheAsync } from '../saveBuildCache';
jest.mock('@expo/steps', () => ({
...jest.requireActual('@expo/steps'),
spawnAsync: jest.fn(),
}));
jest.mock('../restoreCache', () => ({
decompressCacheAsync: jest.fn(),
downloadCacheAsync: jest.fn(),
downloadPublicCacheAsync: jest.fn(),
}));
jest.mock('../saveCache', () => ({
compressCacheAsync: jest.fn(),
uploadCacheAsync: jest.fn(),
}));
jest.mock('../../../utils/cocoapodsCache', () => ({
compressCocoapodsCacheAsync: jest.fn(),
getCocoapodsCachePaths: jest.fn(),
resolveCocoapodsCacheKeyAsync: jest.fn(),
restoreCocoapodsCacheArchiveAsync: jest.fn(),
}));
const logger = createMockLogger();
const env = {
EAS_PODS_CACHE: '1',
EAS_BUILD_ID: 'build-id',
__API_SERVER_URL: 'https://api.expo.test',
};
const secrets = { robotAccessToken: 'robot-token' };
describe('CocoaPods build cache', () => {
beforeEach(() => {
jest.clearAllMocks();
vol.fromJSON(
{
'/workingdir/ios/Pods/Manifest.lock': 'manifest',
'/workingdir/ios/Podfile.lock': 'lockfile',
'/tmp/cocoapods-cache.tar.gz': 'archive',
},
'/'
);
jest.mocked(getCocoapodsCachePaths).mockReturnValue({
iosDirectory: '/workingdir/ios',
podsDirectory: '/workingdir/ios/Pods',
podfileLockPath: '/workingdir/ios/Podfile.lock',
});
jest.mocked(resolveCocoapodsCacheKeyAsync).mockResolvedValue({
key: 'ios-pods-1.16.2-lock-hash',
keyPrefix: 'ios-pods-1.16.2-',
});
jest.mocked(spawnAsync).mockResolvedValue({ stdout: '1.16.2\n' } as any);
});
it('does nothing when the cache is disabled', async () => {
await restoreCocoapodsCacheAsync({
logger,
workingDirectory: '/workingdir',
env: {},
secrets,
});
await saveCocoapodsCacheAsync({
logger,
workingDirectory: '/workingdir',
env: {},
secrets,
});
expect(spawnAsync).not.toHaveBeenCalled();
});
it('restores the newest matching CocoaPods cache', async () => {
jest.mocked(downloadCacheAsync).mockResolvedValue({
archivePath: '/tmp/cocoapods-cache.tar.gz',
matchedKey: 'ios-pods-1.16.2-older-lock-hash',
});
const datadogLogSpy = jest.spyOn(Datadog, 'log');
await restoreCocoapodsCacheAsync({
logger,
workingDirectory: '/workingdir',
env,
secrets,
});
expect(downloadCacheAsync).toHaveBeenCalledWith(
expect.objectContaining({
key: 'ios-pods-1.16.2-lock-hash',
keyPrefixes: ['ios-pods-1.16.2-'],
paths: ['/workingdir/ios/Pods'],
})
);
expect(restoreCocoapodsCacheArchiveAsync).toHaveBeenCalledWith({
archivePath: '/tmp/cocoapods-cache.tar.gz',
workingDirectory: '/workingdir',
});
expect(datadogLogSpy).toHaveBeenCalledWith('CocoaPods cache restored (prefix_match)', {
event: 'cocoapods_cache_restored',
cache_hit_type: 'prefix_match',
});
});
it('compresses and saves the installed Pods directory', async () => {
jest.mocked(compressCocoapodsCacheAsync).mockResolvedValue({
archivePath: '/tmp/cocoapods-cache.tar.gz',
});
await saveCocoapodsCacheAsync({
logger,
workingDirectory: '/workingdir',
env,
secrets,
});
expect(uploadCacheAsync).toHaveBeenCalledWith(
expect.objectContaining({
archivePath: '/tmp/cocoapods-cache.tar.gz',
key: 'ios-pods-1.16.2-lock-hash',
paths: ['/workingdir/ios/Pods'],
})
);
});
});
@@ -4,8 +4,10 @@ import { Client, CombinedError } from '@urql/core';
import fetch, { Response } from 'node-fetch';
import { randomUUID } from 'node:crypto';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { Readable } from 'node:stream';
import * as tar from 'tar';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { createMockLogger } from '../../../__tests__/utils/logger';
@@ -19,6 +21,9 @@ const APP_TAR_GZ_BUFFER = Buffer.from(
'base64'
);
const FLAT_APP_INFO_PLIST_BASE64 =
'YnBsaXN0MDDSAQIDBF8QEkNGQnVuZGxlRXhlY3V0YWJsZV8QE0NGQnVuZGxlUGFja2FnZVR5cGVXVGVzdEFwcFRBUFBMCA0iOEAAAAAAAAABAQAAAAAAAAAFAAAAAAAAAAAAAAAAAAAARQ==';
const APPLICATION_ARCHIVE_URL = `https://expo.dev/artifacts/eas/${randomUUID()}.tar.gz`;
function createMockGraphqlClient({
@@ -51,6 +56,57 @@ function createMockGraphqlClient({
} as unknown as Client;
}
function createSuccessfulResponse({
body,
url,
headers,
}: {
body: Buffer;
url: string;
headers?: Record<string, string>;
}): Response {
const normalizedHeaders = Object.fromEntries(
Object.entries(headers ?? {}).map(([name, value]) => [name.toLowerCase(), value])
);
return {
ok: true,
body: Readable.from(body),
url,
headers: {
get: (name: string): string | null => normalizedHeaders[name.toLowerCase()] ?? null,
},
} as unknown as Response;
}
async function createFlatAppTarGzBufferAsync(): Promise<Buffer> {
const sourceDirectory = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'flat-app-source-'));
const archivePath = path.join(os.tmpdir(), `${randomUUID()}.tar.gz`);
try {
await Promise.all([
fs.promises.writeFile(path.join(sourceDirectory, 'Info.plist'), FLAT_APP_INFO_PLIST_BASE64, {
encoding: 'base64',
}),
fs.promises.writeFile(path.join(sourceDirectory, 'TestApp'), 'i am executable\n'),
]);
await tar.create(
{
cwd: sourceDirectory,
file: archivePath,
gzip: true,
},
['Info.plist', 'TestApp']
);
return await fs.promises.readFile(archivePath);
} finally {
await Promise.all([
fs.promises.rm(sourceDirectory, { recursive: true, force: true }),
fs.promises.rm(archivePath, { force: true }),
]);
}
}
describe('downloadBuild', () => {
it('downloads from applicationArchiveUrl returned by GraphQL', async () => {
const buildId = randomUUID();
@@ -58,11 +114,12 @@ describe('downloadBuild', () => {
applicationArchiveUrl: APPLICATION_ARCHIVE_URL,
});
jest.mocked(fetch).mockResolvedValue({
ok: true,
body: Readable.from(APP_TAR_GZ_BUFFER),
url: APPLICATION_ARCHIVE_URL,
} as unknown as Response);
jest.mocked(fetch).mockResolvedValue(
createSuccessfulResponse({
body: APP_TAR_GZ_BUFFER,
url: APPLICATION_ARCHIVE_URL,
})
);
const { artifactPath } = await downloadBuildAsync({
logger: createLogger({ name: 'test' }),
@@ -82,17 +139,39 @@ describe('downloadBuild', () => {
);
});
it('downloads an archive containing app bundle contents at its root', async () => {
jest.mocked(fetch).mockResolvedValue(
createSuccessfulResponse({
body: await createFlatAppTarGzBufferAsync(),
url: APPLICATION_ARCHIVE_URL,
})
);
const { artifactPath } = await downloadBuildAsync({
logger: createLogger({ name: 'test' }),
applicationArchiveUrl: APPLICATION_ARCHIVE_URL,
graphqlClient: createMockGraphqlClient({}),
robotAccessToken: null,
extensions: ['app'],
});
expect(path.extname(artifactPath)).toBe('.app');
expect(await fs.promises.readFile(path.join(artifactPath, 'TestApp'), 'utf8')).toBe(
'i am executable\n'
);
});
it('should handle a straight-up file', async () => {
const applicationArchiveUrl = `https://expo.dev/artifacts/eas/${randomUUID()}.apk`;
const graphqlClient = createMockGraphqlClient({
applicationArchiveUrl,
});
jest.mocked(fetch).mockResolvedValue({
ok: true,
body: Readable.from(Buffer.from('hello')),
url: applicationArchiveUrl,
} as unknown as Response);
jest
.mocked(fetch)
.mockResolvedValue(
createSuccessfulResponse({ body: Buffer.from('hello'), url: applicationArchiveUrl })
);
const { artifactPath } = await downloadBuildAsync({
logger: createLogger({ name: 'test' }),
@@ -106,6 +185,128 @@ describe('downloadBuild', () => {
expect(await fs.promises.readFile(artifactPath, 'utf-8')).toBe('hello');
});
it('downloads a direct application archive URL without querying GraphQL or forwarding the EAS token', async () => {
const applicationArchiveUrl = `https://artifacts.example.test/${randomUUID()}.apk`;
const graphqlClient = createMockGraphqlClient({
applicationArchiveUrl: APPLICATION_ARCHIVE_URL,
});
jest
.mocked(fetch)
.mockResolvedValue(
createSuccessfulResponse({ body: Buffer.from('hello'), url: applicationArchiveUrl })
);
const { artifactPath } = await downloadBuildAsync({
logger: createLogger({ name: 'test' }),
applicationArchiveUrl,
graphqlClient,
robotAccessToken: 'scoped-eas-token',
extensions: ['apk'],
});
expect(graphqlClient.query).not.toHaveBeenCalled();
expect(jest.mocked(fetch)).toHaveBeenCalledWith(
applicationArchiveUrl,
expect.objectContaining({ headers: undefined })
);
expect(await fs.promises.readFile(artifactPath, 'utf-8')).toBe('hello');
});
it('uses the Content-Disposition filename for an extensionless URL', async () => {
const applicationArchiveUrl = 'https://artifacts.example.test/download';
jest.mocked(fetch).mockResolvedValue(
createSuccessfulResponse({
body: Buffer.from('hello'),
url: applicationArchiveUrl,
headers: { 'content-disposition': 'attachment; filename="app.apk"' },
})
);
const { artifactPath } = await downloadBuildAsync({
logger: createLogger({ name: 'test' }),
applicationArchiveUrl,
graphqlClient: createMockGraphqlClient({}),
robotAccessToken: null,
extensions: ['apk'],
});
expect(path.basename(artifactPath)).toBe('app.apk');
expect(await fs.promises.readFile(artifactPath, 'utf-8')).toBe('hello');
});
it('uses the expected extension when the URL has no filename', async () => {
const applicationArchiveUrl = 'https://artifacts.example.test/';
jest
.mocked(fetch)
.mockResolvedValue(
createSuccessfulResponse({ body: Buffer.from('hello'), url: applicationArchiveUrl })
);
const { artifactPath } = await downloadBuildAsync({
logger: createLogger({ name: 'test' }),
applicationArchiveUrl,
graphqlClient: createMockGraphqlClient({}),
robotAccessToken: null,
extensions: ['apk'],
});
expect(path.basename(artifactPath)).toBe('application.apk');
expect(await fs.promises.readFile(artifactPath, 'utf-8')).toBe('hello');
});
it('rejects missing or ambiguous build sources', async () => {
const graphqlClient = createMockGraphqlClient({
applicationArchiveUrl: APPLICATION_ARCHIVE_URL,
});
await expect(
// @ts-expect-error Verify the runtime guard for callers without TypeScript.
downloadBuildAsync({
logger: createLogger({ name: 'test' }),
graphqlClient,
robotAccessToken: null,
extensions: ['app'],
})
).rejects.toMatchObject({
errorCode: 'EAS_DOWNLOAD_BUILD_INVALID_SOURCE',
message: 'Pass buildId or applicationArchiveUrl.',
});
await expect(
// @ts-expect-error Verify the runtime guard for callers without TypeScript.
downloadBuildAsync({
logger: createLogger({ name: 'test' }),
buildId: randomUUID(),
applicationArchiveUrl: APPLICATION_ARCHIVE_URL,
graphqlClient,
robotAccessToken: null,
extensions: ['app'],
})
).rejects.toMatchObject({
errorCode: 'EAS_DOWNLOAD_BUILD_INVALID_SOURCE',
message: 'Pass only one of buildId or applicationArchiveUrl.',
});
expect(graphqlClient.query).not.toHaveBeenCalled();
expect(jest.mocked(fetch)).not.toHaveBeenCalled();
});
it('rejects a non-HTTP application archive URL', async () => {
await expect(
downloadBuildAsync({
logger: createLogger({ name: 'test' }),
applicationArchiveUrl: 'file:///tmp/app.apk',
graphqlClient: createMockGraphqlClient({}),
robotAccessToken: null,
extensions: ['apk'],
})
).rejects.toMatchObject({
errorCode: 'EAS_DOWNLOAD_BUILD_INVALID_APPLICATION_ARCHIVE_URL',
message: 'application_archive_url must be a valid HTTP or HTTPS URL.',
});
});
it('throws UserError when the build has no application archive url', async () => {
const graphqlClient = createMockGraphqlClient({ applicationArchiveUrl: null });
@@ -203,11 +404,12 @@ describe('downloadBuild', () => {
applicationArchiveUrl: APPLICATION_ARCHIVE_URL,
});
jest.mocked(fetch).mockResolvedValue({
ok: true,
body: Readable.from(APP_TAR_GZ_BUFFER),
url: APPLICATION_ARCHIVE_URL,
} as unknown as Response);
jest.mocked(fetch).mockResolvedValue(
createSuccessfulResponse({
body: APP_TAR_GZ_BUFFER,
url: APPLICATION_ARCHIVE_URL,
})
);
await expect(
downloadBuildAsync({
@@ -0,0 +1,158 @@
import { BuildRuntimePlatform } from '@expo/steps';
import spawn from '@expo/turtle-spawn';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { createMockLogger } from '../../../__tests__/utils/logger';
import { createInstallBuildFunction, installBuildAsync } from '../installBuild';
jest.mock('@expo/turtle-spawn', () => ({
__esModule: true,
default: jest.fn(),
}));
const mockedSpawn = jest.mocked(spawn);
const temporaryDirectories: string[] = [];
async function makeTemporaryDirectoryAsync(): Promise<string> {
const temporaryDirectory = await fs.promises.mkdtemp(
path.join(os.tmpdir(), 'install-build-test-')
);
temporaryDirectories.push(temporaryDirectory);
return temporaryDirectory;
}
describe(installBuildAsync, () => {
beforeEach(() => {
jest.clearAllMocks();
mockedSpawn.mockResolvedValue({ stdout: '', stderr: '' } as any);
});
afterAll(async () => {
await Promise.all(
temporaryDirectories.map(temporaryDirectory =>
fs.promises.rm(temporaryDirectory, { recursive: true, force: true })
)
);
});
it('installs an iOS Simulator .app', async () => {
const temporaryDirectory = await makeTemporaryDirectoryAsync();
const artifactPath = path.join(temporaryDirectory, 'Example.app');
await fs.promises.mkdir(artifactPath);
const logger = createMockLogger();
mockedSpawn.mockResolvedValueOnce({ stdout: 'com.example.app\n', stderr: '' } as any);
const result = await installBuildAsync({
artifactPath,
runtimePlatform: BuildRuntimePlatform.DARWIN,
env: {},
logger,
});
expect(result).toEqual({ applicationIdentifier: 'com.example.app' });
expect(mockedSpawn.mock.calls).toEqual([
[
'plutil',
['-extract', 'CFBundleIdentifier', 'raw', '-o', '-', path.join(artifactPath, 'Info.plist')],
{ stdio: 'pipe', env: {} },
],
['xcrun', ['simctl', 'install', 'booted', artifactPath], { env: {}, logger }],
]);
});
it('installs an Android Emulator .apk', async () => {
const temporaryDirectory = await makeTemporaryDirectoryAsync();
const artifactPath = path.join(temporaryDirectory, 'example.apk');
await fs.promises.writeFile(artifactPath, 'apk');
const logger = createMockLogger();
mockedSpawn.mockResolvedValueOnce({
stdout:
"package: name='com.example.app'\nlaunchable-activity: name='com.example.app.MainActivity'\n",
stderr: '',
} as any);
const result = await installBuildAsync({
artifactPath,
runtimePlatform: BuildRuntimePlatform.LINUX,
env: {},
logger,
});
expect(result).toEqual({
applicationIdentifier: 'com.example.app',
activityName: 'com.example.app.MainActivity',
});
expect(mockedSpawn.mock.calls).toEqual([
['aapt2', ['dump', 'badging', artifactPath], { stdio: 'pipe', env: {} }],
['adb', ['install', '-r', artifactPath], { env: {}, logger }],
]);
});
it.each([
[BuildRuntimePlatform.DARWIN, 'example.ipa'],
[BuildRuntimePlatform.LINUX, 'example.aab'],
])('rejects an incompatible %s artifact', async (runtimePlatform, filename) => {
const temporaryDirectory = await makeTemporaryDirectoryAsync();
const artifactPath = path.join(temporaryDirectory, filename);
await fs.promises.writeFile(artifactPath, 'artifact');
await expect(
installBuildAsync({
artifactPath,
runtimePlatform,
env: {},
logger: createMockLogger(),
})
).rejects.toMatchObject({ errorCode: 'EAS_INSTALL_BUILD_INVALID_ARTIFACT' });
expect(mockedSpawn).not.toHaveBeenCalled();
});
it('installs an Android artifact without a launchable activity', async () => {
const temporaryDirectory = await makeTemporaryDirectoryAsync();
const artifactPath = path.join(temporaryDirectory, 'example.apk');
await fs.promises.writeFile(artifactPath, 'apk');
const logger = createMockLogger();
mockedSpawn.mockResolvedValue({
stdout: "package: name='com.example.app'\n",
stderr: '',
} as any);
const result = await installBuildAsync({
artifactPath,
runtimePlatform: BuildRuntimePlatform.LINUX,
env: {},
logger,
});
expect(result).toEqual({ applicationIdentifier: 'com.example.app' });
expect(mockedSpawn.mock.calls).toEqual([
['aapt2', ['dump', 'badging', artifactPath], { stdio: 'pipe', env: {} }],
['adb', ['install', '-r', artifactPath], { env: {}, logger }],
]);
});
it('exposes the installed application identifier and activity as step outputs', async () => {
const temporaryDirectory = await makeTemporaryDirectoryAsync();
const artifactPath = path.join(temporaryDirectory, 'example.apk');
await fs.promises.writeFile(artifactPath, 'apk');
mockedSpawn.mockResolvedValueOnce({
stdout:
"package: name='com.example.app'\nlaunchable-activity: name='com.example.app.MainActivity'\n",
stderr: '',
} as any);
const installBuild = createInstallBuildFunction();
const buildStep = installBuild.createBuildStepFromFunctionCall(
createGlobalContextMock({ runtimePlatform: BuildRuntimePlatform.LINUX }),
{ callInputs: { artifact_path: artifactPath } }
);
await buildStep.executeAsync();
expect(buildStep.outputById.application_identifier.value).toBe('com.example.app');
expect(buildStep.outputById.activity_name.value).toBe('com.example.app.MainActivity');
});
});
@@ -1,6 +1,10 @@
import downloadFile from '@expo/downloader';
import { SystemError, UserError } from '@expo/eas-build-job';
import spawn from '@expo/turtle-spawn';
import { BuildRuntimePlatform } from '@expo/steps';
import fs from 'fs';
import os from 'os';
import path from 'path';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { Datadog } from '../../../datadog';
@@ -11,6 +15,11 @@ jest.mock('@expo/turtle-spawn', () => ({
default: jest.fn(),
}));
jest.mock('@expo/downloader', () => ({
__esModule: true,
default: jest.fn(),
}));
jest.mock('../../../datadog', () => ({
Datadog: {
distribution: jest.fn(),
@@ -18,10 +27,24 @@ jest.mock('../../../datadog', () => ({
}));
const mockedSpawn = jest.mocked(spawn);
const mockedDownloadFile = jest.mocked(downloadFile);
async function writeInstalledWdaVersion(homeDirectory: string, version: string): Promise<void> {
const wdaDirectory = path.join(
homeDirectory,
'.maestro-runner',
'drivers',
'ios',
'WebDriverAgent'
);
await fs.promises.mkdir(wdaDirectory, { recursive: true });
await fs.promises.writeFile(path.join(wdaDirectory, 'package.json'), JSON.stringify({ version }));
}
describe('createInstallMaestroBuildFunction', () => {
beforeEach(() => {
jest.clearAllMocks();
mockedDownloadFile.mockResolvedValue(undefined);
// `maestro --version` reports an installed version; `java -version` succeeds.
// With no requested version and Maestro present, the step skips installation.
mockedSpawn.mockImplementation((async (command: string) => ({
@@ -439,4 +462,184 @@ describe('createInstallMaestroBuildFunction', () => {
process.env.PATH = originalPath;
}
});
it('installs the prebuilt WDA cache for each available iOS runtime', async () => {
const homeDirectory = await fs.promises.mkdtemp(
path.join(os.tmpdir(), 'install-maestro-test-')
);
mockedSpawn.mockImplementation((async (command: string, args: string[]) => {
if (command === 'maestro-runner') {
return { stdout: 'maestro-runner 1.2.3\n' };
}
if (command === 'xcrun') {
return {
stdout: JSON.stringify({
runtimes: [
{
identifier: 'com.apple.CoreSimulator.SimRuntime.iOS-18-3',
isAvailable: true,
version: '18.3.1',
},
{
identifier: 'com.apple.CoreSimulator.SimRuntime.iOS-26-0',
isAvailable: true,
version: '26.0',
},
{
identifier: 'com.apple.CoreSimulator.SimRuntime.iOS-17-5',
isAvailable: false,
version: '17.5',
},
],
}),
};
}
if (command === 'xcodebuild') {
return { stdout: 'Xcode 26.0\nBuild version 17A324\n' };
}
if (command === 'tar') {
const maestroRunnerHome = args[args.indexOf('-C') + 1];
const productsDirectory = path.join(
maestroRunnerHome,
'cache',
'wda-builds',
'generic',
'DerivedData',
'Build',
'Products'
);
await fs.promises.mkdir(productsDirectory, { recursive: true });
await fs.promises.writeFile(
path.join(productsDirectory, 'WebDriverAgentRunner.xctestrun'),
'cached'
);
}
return { stdout: '' };
}) as any);
try {
await writeInstalledWdaVersion(homeDirectory, '11.1.3');
const installMaestro = createInstallMaestroBuildFunction();
const globalCtx = createGlobalContextMock({
runtimePlatform: BuildRuntimePlatform.DARWIN,
});
globalCtx.updateEnv({
EAS_BUILD_RUNNER: 'eas-build',
EAS_BUILD_COCOAPODS_CACHE_URL: 'https://cache.example.com',
HOME: homeDirectory,
});
const step = installMaestro.createBuildStepFromFunctionCall(globalCtx, {
callInputs: { backend: 'maestro-runner' },
});
await step.executeAsync();
expect(mockedDownloadFile).toHaveBeenCalledWith(
'https://cache.example.com/storage.googleapis.com/turtle-v2/maestro-runner-wda-cache/xcode-26.0-wda-11.1.3.tar.gz',
expect.stringMatching(/install_maestro_runner_wda_cache.*\/wda-cache\.tar\.gz$/),
{ retry: 3, timeout: 20_000 }
);
for (const runtimeVersion of ['18.3', '26.0']) {
await expect(
fs.promises.readFile(
path.join(
homeDirectory,
'.maestro-runner',
'cache',
'wda-builds',
`sim-ios${runtimeVersion}-iphone`,
'DerivedData',
'Build',
'Products',
'WebDriverAgentRunner.xctestrun'
),
'utf8'
)
).resolves.toBe('cached');
}
await expect(
fs.promises.access(
path.join(homeDirectory, '.maestro-runner', 'cache', 'wda-builds', 'sim-ios17.5-iphone')
)
).rejects.toThrow();
} finally {
await fs.promises.rm(homeDirectory, { force: true, recursive: true });
}
});
it('does not download the WDA cache when the installed WDA version is unknown', async () => {
mockedSpawn.mockImplementation((async (command: string) => ({
stdout:
command === 'xcodebuild' ? 'Xcode 26.5\nBuild version 17F90\n' : 'maestro-runner 1.2.3\n',
})) as any);
const installMaestro = createInstallMaestroBuildFunction();
const globalCtx = createGlobalContextMock({
runtimePlatform: BuildRuntimePlatform.DARWIN,
});
globalCtx.updateEnv({ EAS_BUILD_RUNNER: 'eas-build', HOME: '/home/expo' });
const step = installMaestro.createBuildStepFromFunctionCall(globalCtx, {
callInputs: { backend: 'maestro-runner' },
});
await step.executeAsync();
// The installed WDA version is unknown (no package.json under $HOME), so the
// cache download is skipped before resolving the Xcode/runtime versions.
expect(mockedDownloadFile).not.toHaveBeenCalled();
expect(mockedSpawn.mock.calls.map(([command]) => command)).not.toContain('xcrun');
});
it('continues when the prebuilt WDA cache is not available', async () => {
mockedSpawn.mockImplementation((async (command: string) => {
switch (command) {
case 'maestro-runner':
return { stdout: 'maestro-runner 1.2.3\n' };
case 'xcodebuild':
return { stdout: 'Xcode 26.5\nBuild version 17F90\n' };
case 'xcrun':
return {
stdout: JSON.stringify({
runtimes: [
{
identifier: 'com.apple.CoreSimulator.SimRuntime.iOS-26-5',
isAvailable: true,
version: '26.5',
},
],
}),
};
default:
return { stdout: '' };
}
}) as any);
mockedDownloadFile.mockRejectedValue(new Error('HTTP 404'));
const homeDirectory = await fs.promises.mkdtemp(
path.join(os.tmpdir(), 'install-maestro-test-')
);
try {
await writeInstalledWdaVersion(homeDirectory, '11.1.3');
const installMaestro = createInstallMaestroBuildFunction();
const globalCtx = createGlobalContextMock({
runtimePlatform: BuildRuntimePlatform.DARWIN,
});
globalCtx.updateEnv({
EAS_BUILD_RUNNER: 'eas-build',
EAS_BUILD_COCOAPODS_CACHE_URL: 'https://cache.example.com',
HOME: homeDirectory,
});
const step = installMaestro.createBuildStepFromFunctionCall(globalCtx, {
callInputs: { backend: 'maestro-runner' },
});
await expect(step.executeAsync()).resolves.toBeUndefined();
expect(step.getOutputValueByName('maestro_version')).toBe('1.2.3');
expect(mockedDownloadFile.mock.calls.map(([url]) => url)).toEqual([
'https://cache.example.com/storage.googleapis.com/turtle-v2/maestro-runner-wda-cache/xcode-26.5-wda-11.1.3.tar.gz',
'https://storage.googleapis.com/turtle-v2/maestro-runner-wda-cache/xcode-26.5-wda-11.1.3.tar.gz',
]);
} finally {
await fs.promises.rm(homeDirectory, { force: true, recursive: true });
}
});
});
@@ -0,0 +1,320 @@
import { BuildRuntimePlatform } from '@expo/steps';
import spawn from '@expo/turtle-spawn';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { createMockLogger } from '../../../__tests__/utils/logger';
import { createLaunchApplicationFunction, launchApplicationAsync } from '../launchApplication';
jest.mock('@expo/turtle-spawn', () => ({
__esModule: true,
default: jest.fn(),
}));
const mockedSpawn = jest.mocked(spawn);
describe(launchApplicationAsync, () => {
beforeEach(() => {
jest.clearAllMocks();
mockedSpawn.mockResolvedValue({ stdout: '', stderr: '' } as any);
});
it('launches an iOS Simulator application by bundle identifier', async () => {
const logger = createMockLogger();
await launchApplicationAsync({
applicationIdentifier: 'com.example.app',
runtimePlatform: BuildRuntimePlatform.DARWIN,
env: {},
logger,
});
expect(mockedSpawn).toHaveBeenCalledWith(
'xcrun',
['simctl', 'launch', 'booted', 'com.example.app'],
{ env: {}, logger }
);
});
it('passes launch arguments and opens a URL on iOS', async () => {
const logger = createMockLogger();
await launchApplicationAsync({
applicationIdentifier: 'host.exp.Exponent',
launchArgs: ['--uitesting', 'true'],
openUrl: 'exp://example.test',
runtimePlatform: BuildRuntimePlatform.DARWIN,
env: {},
logger,
});
expect(mockedSpawn).toHaveBeenNthCalledWith(
1,
'xcrun',
['simctl', 'launch', 'booted', 'host.exp.Exponent', '--uitesting', 'true'],
{ env: {}, logger }
);
expect(mockedSpawn).toHaveBeenNthCalledWith(
2,
'xcrun',
[
'simctl',
'spawn',
'booted',
'defaults',
'write',
'com.apple.launchservices.schemeapproval',
'com.apple.CoreSimulator.CoreSimulatorBridge-->exp',
'-string',
'host.exp.Exponent',
],
{ env: {}, logger }
);
expect(mockedSpawn).toHaveBeenNthCalledWith(
3,
'xcrun',
['simctl', 'openurl', 'booted', 'exp://example.test'],
{ env: {}, logger }
);
});
it('opens a web URL on iOS without preapproving its scheme', async () => {
const logger = createMockLogger();
await launchApplicationAsync({
applicationIdentifier: 'com.example.app',
openUrl: 'https://example.test',
runtimePlatform: BuildRuntimePlatform.DARWIN,
env: {},
logger,
});
expect(mockedSpawn).toHaveBeenNthCalledWith(
1,
'xcrun',
['simctl', 'launch', 'booted', 'com.example.app'],
{ env: {}, logger }
);
expect(mockedSpawn).toHaveBeenNthCalledWith(
2,
'xcrun',
['simctl', 'openurl', 'booted', 'https://example.test'],
{ env: {}, logger }
);
expect(mockedSpawn).toHaveBeenCalledTimes(2);
});
it('opens a custom URL on iOS when scheme preapproval fails', async () => {
const logger = createMockLogger();
const approvalError = new Error('Scheme approval is unavailable.');
mockedSpawn
.mockResolvedValueOnce({ stdout: '', stderr: '' } as any)
.mockRejectedValueOnce(approvalError);
await launchApplicationAsync({
applicationIdentifier: 'host.exp.Exponent',
openUrl: 'exp://example.test',
runtimePlatform: BuildRuntimePlatform.DARWIN,
env: {},
logger,
});
expect(mockedSpawn).toHaveBeenNthCalledWith(
3,
'xcrun',
['simctl', 'openurl', 'booted', 'exp://example.test'],
{ env: {}, logger }
);
expect(logger.warn).toHaveBeenCalledWith(
{ err: approvalError },
'Could not preapprove the exp URL scheme for host.exp.Exponent. Opening the URL anyway; the Simulator might require manual confirmation.'
);
});
it('launches an Android Emulator application by package and activity', async () => {
const logger = createMockLogger();
await launchApplicationAsync({
applicationIdentifier: 'com.example.app',
activityName: 'com.example.app.MainActivity',
runtimePlatform: BuildRuntimePlatform.LINUX,
env: {},
logger,
});
expect(mockedSpawn).toHaveBeenCalledWith(
'adb',
['shell', 'am', 'start', '-n', 'com.example.app/com.example.app.MainActivity'],
{ env: {}, logger }
);
});
it('passes launch arguments and opens a URL on Android', async () => {
const logger = createMockLogger();
await launchApplicationAsync({
applicationIdentifier: 'host.exp.exponent',
activityName: 'host.exp.exponent.MainActivity',
launchArgs: ['--ez', 'isTest', 'true'],
openUrl: 'exp://example.test',
runtimePlatform: BuildRuntimePlatform.LINUX,
env: {},
logger,
});
expect(mockedSpawn).toHaveBeenNthCalledWith(
1,
'adb',
[
'shell',
'am',
'start',
'--ez',
'isTest',
'true',
'-n',
'host.exp.exponent/host.exp.exponent.MainActivity',
],
{ env: {}, logger }
);
expect(mockedSpawn).toHaveBeenNthCalledWith(
2,
'adb',
[
'shell',
'am',
'start',
'-a',
'android.intent.action.VIEW',
'-d',
'exp://example.test',
'-n',
'host.exp.exponent/host.exp.exponent.MainActivity',
],
{ env: {}, logger }
);
expect(logger.info).toHaveBeenNthCalledWith(
1,
'Launching host.exp.exponent with arguments ["--ez","isTest","true"].'
);
});
it('requires an activity when launching an Android application', async () => {
await expect(
launchApplicationAsync({
applicationIdentifier: 'com.example.app',
runtimePlatform: BuildRuntimePlatform.LINUX,
env: {},
logger: createMockLogger(),
})
).rejects.toMatchObject({ errorCode: 'EAS_LAUNCH_APPLICATION_MISSING_ACTIVITY' });
expect(mockedSpawn).not.toHaveBeenCalled();
});
it('launches using application metadata passed to the build step', async () => {
const launchApplication = createLaunchApplicationFunction();
const buildStep = launchApplication.createBuildStepFromFunctionCall(
createGlobalContextMock({ runtimePlatform: BuildRuntimePlatform.LINUX }),
{
callInputs: {
application_identifier: 'com.example.app',
activity_name: 'com.example.app.MainActivity',
launch_args: ['--ez', 'isTest', 'true'],
open_url: 'exp://example.test',
},
}
);
await buildStep.executeAsync();
expect(mockedSpawn).toHaveBeenCalledWith(
'adb',
[
'shell',
'am',
'start',
'--ez',
'isTest',
'true',
'-n',
'com.example.app/com.example.app.MainActivity',
],
expect.any(Object)
);
expect(mockedSpawn).toHaveBeenCalledWith(
'adb',
[
'shell',
'am',
'start',
'-a',
'android.intent.action.VIEW',
'-d',
'exp://example.test',
'-n',
'com.example.app/com.example.app.MainActivity',
],
expect.any(Object)
);
});
it.each([{}, ['valid', 42]])('rejects invalid launch arguments (%j)', async launchArgs => {
const launchApplication = createLaunchApplicationFunction();
const buildStep = launchApplication.createBuildStepFromFunctionCall(
createGlobalContextMock({ runtimePlatform: BuildRuntimePlatform.DARWIN }),
{
callInputs: {
application_identifier: 'com.example.app',
launch_args: launchArgs,
},
}
);
await expect(buildStep.executeAsync()).rejects.toMatchObject({
errorCode: 'EAS_LAUNCH_APPLICATION_INVALID_INPUT',
message: 'Input "launch_args" must be an array of strings.',
});
expect(mockedSpawn).not.toHaveBeenCalled();
});
it('rejects an invalid URL to open', async () => {
const launchApplication = createLaunchApplicationFunction();
const buildStep = launchApplication.createBuildStepFromFunctionCall(
createGlobalContextMock({ runtimePlatform: BuildRuntimePlatform.DARWIN }),
{
callInputs: {
application_identifier: 'com.example.app',
open_url: 'not a URL',
},
}
);
await expect(buildStep.executeAsync()).rejects.toMatchObject({
errorCode: 'EAS_LAUNCH_APPLICATION_INVALID_INPUT',
message: 'Input "open_url" must be a valid URL.',
});
expect(mockedSpawn).not.toHaveBeenCalled();
});
it.each(['application_identifier', 'activity_name'])(
'reports an empty %s input as a user error',
async inputName => {
const launchApplication = createLaunchApplicationFunction();
const buildStep = launchApplication.createBuildStepFromFunctionCall(
createGlobalContextMock({ runtimePlatform: BuildRuntimePlatform.LINUX }),
{
callInputs: {
application_identifier: 'com.example.app',
activity_name: 'com.example.app.MainActivity',
[inputName]: '',
},
}
);
await expect(buildStep.executeAsync()).rejects.toMatchObject({
errorCode: 'EAS_LAUNCH_APPLICATION_INVALID_INPUT',
message: `Input "${inputName}" must be a non-empty string. Pass the "${inputName}" output from eas/install_build.`,
});
expect(mockedSpawn).not.toHaveBeenCalled();
}
);
});
@@ -26,6 +26,7 @@ describe(readIpaInfoAsync, () => {
bundleIdentifier: 'dev.expo.SmallestAppExample',
bundleShortVersion: '1.0',
bundleVersion: '1',
dtPlatformName: 'iphoneos',
});
});
@@ -0,0 +1,205 @@
import { type bunyan } from '@expo/logger';
import { BuildRuntimePlatform, type BuildStepContext } from '@expo/steps';
import spawn from '@expo/turtle-spawn';
import fs from 'node:fs';
import { type CustomBuildContext } from '../../../customBuildContext';
import { IosSimulatorUtils } from '../../../utils/IosSimulatorUtils';
import { isProcessDescendantOfAsync } from '../../../utils/processes';
import { turtleFetch } from '../../../utils/turtleFetch';
import {
monitorLocalEgressAsync,
readLocalEgressHandoffAsync,
registerLocalEgressResources,
stopLocalEgressResourcesAsync,
} from '../../utils/localEgress';
import {
startDeviceWebPreviewWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../../utils/remoteDeviceRunSession';
import { createStartAgentDeviceRemoteSessionBuildFunction } from '../startAgentDeviceRemoteSession';
import { createStartAppiumRemoteSessionBuildFunction } from '../startAppiumRemoteSession';
import { createStartArgentRemoteSessionBuildFunction } from '../startArgentRemoteSession';
import { createStartWebPreviewRemoteSessionBuildFunction } from '../startWebPreviewRemoteSession';
jest.mock('@expo/turtle-spawn');
jest.mock('../../../utils/IosSimulatorUtils');
jest.mock('../../../utils/processes');
jest.mock('../../../utils/turtleFetch');
jest.mock('../../utils/agentDeviceArtifacts');
jest.mock('../../utils/argentArtifacts');
jest.mock('../../utils/agentDeviceEvents', () => ({
startAgentDeviceEventCollectionAsync: async () => ({ stopAsync: jest.fn() }),
}));
jest.mock('../../utils/argentEvents', () => ({
ARGENT_EVENT_LOG_FILENAME: 'events.jsonl',
startArgentEventCollectionAsync: async () => ({ stopAsync: jest.fn() }),
}));
jest.mock('../../utils/appiumEvents', () => ({
startAppiumEventCollectionAsync: async () => ({ stopAsync: jest.fn() }),
}));
jest.mock('../../utils/localEgress', () => ({
...jest.requireActual('../../utils/localEgress'),
readLocalEgressHandoffAsync: jest.fn(),
monitorLocalEgressAsync: jest.fn(),
}));
jest.mock('../../utils/remoteDeviceRunSession', () => ({
getDeviceRunSessionIdOrThrow: () => 'session-id',
getNgrokTunnelDomainOrThrow: () => 'example.test',
getNgrokAuthtokenOrThrow: () => 'ngrok-token',
selectXcodeDeveloperDirectoryAsync: jest.fn(),
ensureFfmpegInstalledOnceAsync: jest.fn(),
spawnDetached: () => ({ pid: 123, getOutput: () => '', stopAsync: jest.fn() }),
waitForFileAsync: async () => ({ port: 1234, token: 'controller-token' }),
startNgrokTunnelAsync: async () => ({ url: 'https://controller.test', stopAsync: jest.fn() }),
startDeviceWebPreviewWithTunnelAsync: jest.fn(),
uploadRemoteSessionConfigAsync: jest.fn(),
waitForDeviceRunSessionStoppedAsync: jest.fn(),
}));
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const ctx = {} as CustomBuildContext;
const env = { DEVICE_RUN_SESSION_ID: 'session-id' };
const handoff = { url: 'https://egress.test', token: 'secret', fingerprint: 'key=', port: 8899 };
const fields = {
egressUrl: handoff.url,
egressToken: handoff.token,
egressFingerprint: handoff.fingerprint,
egressPort: handoff.port,
};
const controllers = [
['agent-device', createStartAgentDeviceRemoteSessionBuildFunction, 'agentDeviceRemoteSessionUrl'],
['Appium', createStartAppiumRemoteSessionBuildFunction, 'appiumUrl'],
['Argent', createStartArgentRemoteSessionBuildFunction, 'toolsUrl'],
['web preview', createStartWebPreviewRemoteSessionBuildFunction, 'previewUrl'],
] as const;
describe.each(controllers)('%s local egress', (_name, createFunction, controllerField) => {
const stopPreview = jest.fn();
const stopEgress = jest.fn();
let lifetime: AbortSignal;
async function run(signal?: AbortSignal): Promise<void> {
await createFunction(ctx).fn!(
{ logger, global: { runtimePlatform: BuildRuntimePlatform.DARWIN } } as BuildStepContext,
{
inputs: {
package_version: { value: undefined },
max_idle_time_minutes: { value: undefined },
},
outputs: {},
env,
signal,
} as never
);
}
beforeEach(() => {
jest.clearAllMocks();
lifetime = registerLocalEgressResources(stopEgress);
stopEgress.mockResolvedValue(undefined);
stopPreview.mockResolvedValue(undefined);
jest.spyOn(fs, 'existsSync').mockReturnValue(true);
jest.spyOn(fs.promises, 'mkdtemp').mockResolvedValue('/tmp/appium-egress-test');
jest.spyOn(fs.promises, 'writeFile').mockResolvedValue(undefined);
jest.spyOn(fs.promises, 'rm').mockResolvedValue(undefined);
jest.spyOn(fs.promises, 'readdir').mockResolvedValue(['tool-server.json'] as never);
jest
.spyOn(fs.promises, 'readFile')
.mockResolvedValue(JSON.stringify({ port: 1234, pid: 124, token: 'controller-token' }));
jest.mocked(spawn).mockResolvedValue({ stdout: '{}' } as never);
jest.mocked(isProcessDescendantOfAsync).mockResolvedValue(true);
jest
.mocked(IosSimulatorUtils.getAvailableDevicesAsync)
.mockResolvedValue([{ udid: 'sim' }] as never);
jest.mocked(turtleFetch).mockResolvedValue({ ok: true } as never);
jest.mocked(readLocalEgressHandoffAsync).mockResolvedValue(handoff);
jest.mocked(monitorLocalEgressAsync).mockResolvedValue(undefined);
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
previewUrl: 'https://preview.test',
previewToken: 'preview-secret',
stopAsync: stopPreview,
});
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockImplementation(async () => {
expect(lifetime.aborted).toBe(false);
expect(monitorLocalEgressAsync).toHaveBeenCalledTimes(1);
});
});
afterEach(async () => {
await stopLocalEgressResourcesAsync(logger);
jest.restoreAllMocks();
});
it('reports credentials and starts monitoring before waiting, then releases resources', async () => {
await run();
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledWith({
ctx,
deviceRunSessionId: 'session-id',
logger,
remoteConfig: expect.objectContaining({
...fields,
[controllerField]: expect.any(String),
[controllerField === 'previewUrl' ? 'previewToken' : 'webPreviewToken']: 'preview-secret',
}),
});
expect(monitorLocalEgressAsync).toHaveBeenCalledWith({
port: handoff.port,
env,
logger,
signal: expect.any(AbortSignal),
});
expect(lifetime.aborted).toBe(true);
expect(stopEgress).toHaveBeenCalledTimes(1);
});
it('leaves ordinary remote config unchanged and does not monitor', async () => {
await stopLocalEgressResourcesAsync(logger);
stopEgress.mockClear();
jest.mocked(readLocalEgressHandoffAsync).mockResolvedValue(null);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockResolvedValue(undefined);
await run();
const { remoteConfig } = jest.mocked(uploadRemoteSessionConfigAsync).mock.calls[0][0];
expect(remoteConfig).toHaveProperty(controllerField);
expect(Object.keys(remoteConfig).some(key => key.startsWith('egress'))).toBe(false);
expect(monitorLocalEgressAsync).not.toHaveBeenCalled();
expect(stopEgress).not.toHaveBeenCalled();
});
it.each(['startup', 'report', 'wait', 'teardown'])(
'releases egress after %s fails',
async phase => {
const error = new Error(`${phase} failed`);
if (phase === 'startup') {
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockRejectedValue(error);
} else if (phase === 'report') {
jest.mocked(uploadRemoteSessionConfigAsync).mockRejectedValue(error);
} else if (phase === 'wait') {
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockRejectedValue(error);
} else {
stopPreview.mockRejectedValue(error);
}
await expect(run()).rejects.toThrow(error);
expect(lifetime.aborted).toBe(true);
expect(stopEgress).toHaveBeenCalledTimes(1);
expect(monitorLocalEgressAsync).toHaveBeenCalledTimes(
phase === 'startup' || phase === 'report' ? 0 : 1
);
}
);
it('passes cancellation to the monitor and releases the lifetime', async () => {
const abort = new AbortController();
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockImplementation(async () => abort.abort());
await run(abort.signal);
expect(monitorLocalEgressAsync).toHaveBeenCalledWith(
expect.objectContaining({ signal: abort.signal })
);
expect(lifetime.aborted).toBe(true);
expect(stopEgress).toHaveBeenCalledTimes(1);
});
it('does not report or start a monitor after cancellation', async () => {
const abort = new AbortController();
abort.abort();
await expect(run(abort.signal)).rejects.toThrow();
expect(uploadRemoteSessionConfigAsync).not.toHaveBeenCalled();
expect(monitorLocalEgressAsync).not.toHaveBeenCalled();
expect(stopEgress).toHaveBeenCalledTimes(1);
});
});
@@ -1,4 +1,5 @@
import { type bunyan } from '@expo/logger';
import assert from 'node:assert';
import fg from 'fast-glob';
import { vol } from 'memfs';
import path from 'node:path';
@@ -193,22 +194,53 @@ describe(resolveIosSigningOptionsAsync, () => {
const signingOptions = await resolveIosSigningOptionsAsync({
job,
logger: mockLogger,
tmpDir: '/tmp',
});
expect(signingOptions).not.toBeNull();
expect(signingOptions?.keychainPath).toEqual('/tmp/ios_keychain');
expect(signingOptions?.signingIdentity).toEqual('Test App Certificate');
expect(signingOptions?.provisioningProfile).toEqual({
'com.example.testapp': '/tmp/ios_provisioning_profile',
expect(signingOptions).toMatchObject({
keychainPath: '/tmp/ios_keychain',
signingIdentity: 'Test App Certificate',
provisioningProfile: {
'com.example.testapp': '/tmp/ios_provisioning_profile',
},
});
});
it('should resolve zsign signing options without touching the keychain', async () => {
const job = createTestIosJob();
const tmpDir = '/tmp';
vol.mkdirSync(tmpDir, { recursive: true });
const credentialsManagerSpy = jest.spyOn(IosCredentialsManager.prototype, 'prepare');
const signingOptions = await resolveIosSigningOptionsAsync({
job,
logger: mockLogger,
backend: 'zsign',
tmpDir,
});
expect(credentialsManagerSpy).not.toHaveBeenCalled();
expect(signingOptions).toMatchObject({
backend: 'zsign',
keyPassword: job.secrets?.buildCredentials?.['testapp'].distributionCertificate.password,
});
assert(signingOptions?.backend === 'zsign');
expect(signingOptions.certificatePath).toMatch(/\/tmp\/dist-cert-.*\.p12/);
expect(vol.existsSync(signingOptions.certificatePath)).toBe(true);
assert(typeof signingOptions.provisioningProfile === 'object');
const profilePath = signingOptions.provisioningProfile['testapp'];
expect(profilePath).toMatch(/\/tmp\/profile-testapp-.*\.mobileprovision/);
expect(vol.existsSync(profilePath)).toBe(true);
});
it('should return undefined if no build credentials are provided', async () => {
// @ts-expect-error: createTestIosJob does not support buildCredentials as null
const job = createTestIosJob({ buildCredentials: null });
const signingOptions = await resolveIosSigningOptionsAsync({
job,
logger: mockLogger,
tmpDir: '/tmp',
});
expect(signingOptions).toBeUndefined();
});
@@ -219,6 +251,7 @@ describe(resolveIosSigningOptionsAsync, () => {
const signingOptions = await resolveIosSigningOptionsAsync({
job,
logger: mockLogger,
tmpDir: '/tmp',
});
expect(signingOptions).toBeUndefined();
});
@@ -1,11 +1,17 @@
import { type bunyan } from '@expo/logger';
import { Platform } from '@expo/eas-build-job';
import fs from 'fs';
import fetch from 'node-fetch';
import os from 'os';
import path from 'path';
import { compressCacheAsync } from '../saveCache';
import { compressCacheAsync, uploadCacheAsync } from '../saveCache';
import { decompressCacheAsync } from '../restoreCache';
jest.mock('node-fetch');
const { Response } = jest.requireActual('node-fetch') as typeof import('node-fetch');
function createLoggerMock(): bunyan {
return { info: jest.fn(), warn: jest.fn(), error: jest.fn() } as unknown as bunyan;
}
@@ -81,3 +87,71 @@ describe('cache compress/decompress round trip', () => {
expect(Math.floor(restoredStat.mtimeMs / 1000)).toBe(Math.floor(mtime.getTime() / 1000));
});
});
describe(uploadCacheAsync, () => {
afterEach(() => {
jest.mocked(fetch).mockReset();
});
it.each([
['normal upload', undefined, false],
['forced upload', true, true],
] as const)('sends force for %s', async (_name, force, expectedForce) => {
const tempDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'upload-cache-test-'));
const archivePath = path.join(tempDir, 'cache.tar.gz');
await fs.promises.writeFile(archivePath, 'cache archive');
jest
.mocked(fetch)
.mockResolvedValueOnce(
new Response(
JSON.stringify({
data: {
headers: {},
url: 'https://storage.expo.test/cache',
},
}),
{ status: 200 }
)
)
.mockImplementationOnce(async (_url, request) => {
const body = request!.body as NodeJS.ReadableStream & AsyncIterable<Buffer>;
for await (const _chunk of body) {
// Consume the archive stream as a real upload would.
}
return new Response(undefined, { status: 200 });
});
try {
await uploadCacheAsync({
logger: createLoggerMock(),
jobId: 'build-id',
expoApiServerURL: 'https://api.expo.test',
robotAccessToken: 'robot-token',
paths: ['/cache/path'],
key: 'cache-key',
archivePath,
size: 13,
platform: Platform.ANDROID,
force,
});
expect(fetch).toHaveBeenCalledTimes(2);
const [uploadSessionUrl, uploadSessionRequest] = jest.mocked(fetch).mock.calls[0];
expect(uploadSessionUrl.toString()).toBe(
'https://api.expo.test/v2/turtle-builds/caches/upload-sessions'
);
expect(JSON.parse(uploadSessionRequest!.body as string)).toMatchObject({
buildId: 'build-id',
force: expectedForce,
key: 'cache-key',
size: 13,
});
expect(jest.mocked(fetch).mock.calls[1][0].toString()).toBe(
'https://storage.expo.test/cache'
);
} finally {
await fs.promises.rm(tempDir, { recursive: true, force: true });
}
});
});
@@ -83,6 +83,31 @@ describe(createStartAndroidEmulatorBuildFunction, () => {
mockedMkdtemp.mockRestore();
});
it('passes profile and LCD inputs to emulator creation', async () => {
const systemImagePackage = 'system-images;android-35;default;x86_64';
await createStep({
device_identifier: 'medium_phone',
system_image_package: systemImagePackage,
lcd_width: 720,
lcd_height: 1600,
lcd_density: 262,
}).executeAsync();
expect(mockedSpawn).toHaveBeenCalledWith('sdkmanager', [systemImagePackage], expect.anything());
expect(mockedAndroidUtils.createAsync).toHaveBeenCalledWith(
expect.objectContaining({
deviceIdentifier: 'medium_phone',
systemImagePackage,
lcdWidth: 720,
lcdHeight: 1600,
lcdDensity: 262,
})
);
expect(mockedAndroidUtils.createAsync.mock.invocationCallOrder[0]).toBeLessThan(
mockedAndroidUtils.startAsync.mock.invocationCallOrder[0]
);
});
it('retries base emulator startup with increasing readiness timeouts', async () => {
mockedAndroidUtils.startAsync
.mockResolvedValueOnce(createStartResult('emulator-1111'))
@@ -9,11 +9,13 @@ import { isProcessDescendantOfAsync } from '../../../utils/processes';
import { pollArgentArtifactsForUploadAsync } from '../../utils/argentArtifacts';
import { startArgentEventCollectionAsync } from '../../utils/argentEvents';
import {
ensureFfmpegInstalledOnceAsync,
getDeviceRunSessionIdOrThrow,
getNgrokAuthtokenOrThrow,
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
@@ -40,14 +42,14 @@ jest.mock('../../utils/argentEvents', () => ({
startArgentEventCollectionAsync: jest.fn(),
}));
jest.mock('../../utils/remoteDeviceRunSession', () => ({
ensureFfmpegInstalledAsync: jest.fn(),
ensureFfmpegInstalledOnceAsync: jest.fn(),
getDeviceRunSessionIdOrThrow: jest.fn(),
getNgrokAuthtokenOrThrow: jest.fn(),
getNgrokTunnelDomainOrThrow: jest.fn(),
selectXcodeDeveloperDirectoryAsync: jest.fn(),
spawnDetached: jest.fn(),
startDeviceWebPreviewWithTunnelAsync: jest.fn(),
startNgrokTunnelAsync: jest.fn(),
startServeSimWithTunnelAsync: jest.fn(),
uploadRemoteSessionConfigAsync: jest.fn(),
waitForDeviceRunSessionStoppedAsync: jest.fn(),
}));
@@ -58,6 +60,7 @@ const EXPECTED_EVENT_LOG_PATH = path.join(ARGENT_STATE_DIR, 'tool-server-events.
const mockStopAsync = jest.fn();
const mockTunnelStopAsync = jest.fn();
const mockPreviewStopAsync = jest.fn();
describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
beforeEach(async () => {
@@ -68,6 +71,7 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
jest.mocked(pollArgentArtifactsForUploadAsync).mockResolvedValue(undefined);
mockStopAsync.mockResolvedValue(undefined);
mockTunnelStopAsync.mockResolvedValue(undefined);
mockPreviewStopAsync.mockResolvedValue(undefined);
jest.mocked(startArgentEventCollectionAsync).mockResolvedValue({
stopAsync: mockStopAsync,
getLastEventObservedAt: () => undefined,
@@ -86,6 +90,10 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
url: 'https://argent-abc.tunnel.example.com',
stopAsync: mockTunnelStopAsync,
});
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
previewUrl: 'https://web-preview.tunnel.example.com',
stopAsync: mockPreviewStopAsync,
});
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockResolvedValue(undefined);
@@ -120,18 +128,31 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
} as never
);
// (1) The event log flag is enabled, before the tool-server is launched.
// (1) FFmpeg setup starts in the background before Argent setup.
expect(ensureFfmpegInstalledOnceAsync).toHaveBeenCalledWith({
runtimePlatform: BuildRuntimePlatform.LINUX,
env: { EXISTING: 'value' },
logger: expect.anything(),
});
expect(jest.mocked(ensureFfmpegInstalledOnceAsync).mock.invocationCallOrder[0]).toBeLessThan(
jest.mocked(spawn).mock.invocationCallOrder[0]
);
// (2) The event log flag is enabled, before the tool-server is launched.
const spawnCalls = jest.mocked(spawn).mock.calls;
const enableEventLogIndex = spawnCalls.findIndex(
([command, args]) =>
command === 'bunx' && Array.isArray(args) && args.includes('tool-server-event-log')
command === 'bun' &&
Array.isArray(args) &&
args[0] === 'x' &&
args.includes('tool-server-event-log')
);
expect(enableEventLogIndex).toBeGreaterThanOrEqual(0);
expect(jest.mocked(spawn).mock.invocationCallOrder[enableEventLogIndex]).toBeLessThan(
jest.mocked(spawnDetached).mock.invocationCallOrder[0]
);
// (2) The tool-server and the collector are pinned to the exact same event log path.
// (3) The tool-server and the collector are pinned to the exact same event log path.
const serverEnv = jest.mocked(spawnDetached).mock.calls[0][0].env;
expect(serverEnv.ARGENT_EVENT_LOG).toBe(EXPECTED_EVENT_LOG_PATH);
expect(serverEnv.EXISTING).toBe('value');
@@ -140,7 +161,7 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
eventLogPath: EXPECTED_EVENT_LOG_PATH,
});
// (3) Collection starts (before we wait for the session to stop) and (4) is torn down
// (4) Collection starts (before we wait for the session to stop) and (5) is torn down
// afterwards, exactly once.
expect(startArgentEventCollectionAsync).toHaveBeenCalledTimes(1);
expect(mockStopAsync).toHaveBeenCalledTimes(1);
@@ -151,5 +172,17 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
jest.mocked(waitForDeviceRunSessionStoppedAsync).mock.invocationCallOrder[0]
);
expect(mockTunnelStopAsync).toHaveBeenCalledTimes(1);
expect(startDeviceWebPreviewWithTunnelAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({ runtimePlatform: BuildRuntimePlatform.LINUX })
);
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledWith(
expect.objectContaining({
remoteConfig: expect.objectContaining({
webPreviewUrl: 'https://web-preview.tunnel.example.com',
}),
})
);
expect(mockPreviewStopAsync).toHaveBeenCalledTimes(1);
});
});
@@ -74,7 +74,7 @@ describe(warnIfArgentPackageVersionCannotBeVerified, () => {
).not.toThrow();
expect(warn).toHaveBeenCalledTimes(2);
expect(warn).toHaveBeenCalledWith(
expect.stringContaining('Continuing and letting bunx resolve it.')
expect.stringContaining('Continuing and letting bun x resolve it.')
);
});
});
@@ -3,6 +3,12 @@ import spawn from '@expo/turtle-spawn';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { createMockLogger } from '../../../__tests__/utils/logger';
import { IosSimulatorUtils } from '../../../utils/IosSimulatorUtils';
import { configureSimulatorProxyEnvironmentAsync } from '../../utils/localEgress';
import {
installLocalEgressGuardAsync,
resolveLocalEgressBootEnvironmentAsync,
verifyLocalEgressGuardAsync,
} from '../../utils/localEgressGuard';
import { createStartIosSimulatorBuildFunction } from '../startIosSimulator';
jest.mock('@expo/turtle-spawn', () => ({
@@ -16,14 +22,36 @@ jest.mock('../../../utils/IosSimulatorUtils', () => ({
getDeviceAsync: jest.fn(),
cloneAsync: jest.fn(),
enableAccessibilitySettingsAsync: jest.fn(),
resolveUdidAsync: jest.fn(),
bootAsync: jest.fn(),
startAsync: jest.fn(),
waitForReadyAsync: jest.fn(),
disableApsdAsync: jest.fn(),
},
}));
jest.mock('../../utils/localEgress', () => ({
configureSimulatorProxyEnvironmentAsync: jest.fn(),
}));
jest.mock('../../utils/localEgressGuard', () => ({
installLocalEgressGuardAsync: jest.fn(),
resolveLocalEgressBootEnvironmentAsync: jest.fn(),
verifyLocalEgressGuardAsync: jest.fn(),
}));
const mockedSpawn = jest.mocked(spawn);
const mockedUtils = jest.mocked(IosSimulatorUtils);
const mockedConfigureProxyEnvironment = jest.mocked(configureSimulatorProxyEnvironmentAsync);
const mockedInstallGuard = jest.mocked(installLocalEgressGuardAsync);
const mockedVerifyGuard = jest.mocked(verifyLocalEgressGuardAsync);
const mockedResolveBootEnvironment = jest.mocked(resolveLocalEgressBootEnvironmentAsync);
// Names resolve to udids the way the step expects; udids pass through.
const UDIDS: Record<string, string> = {
'iPhone 15': 'base',
'eas-simulator-1': 'clone-1',
'eas-simulator-2': 'clone-2',
};
function createStep(callInputs?: Record<string, unknown>) {
const logger = createMockLogger();
@@ -41,27 +69,41 @@ describe(createStartIosSimulatorBuildFunction, () => {
mockedUtils.getDeviceAsync.mockResolvedValue(null);
mockedUtils.cloneAsync.mockResolvedValue(undefined);
mockedUtils.enableAccessibilitySettingsAsync.mockResolvedValue(undefined);
mockedUtils.startAsync.mockResolvedValue({ udid: 'test-udid' as any });
mockedUtils.resolveUdidAsync.mockImplementation(
async ({ deviceIdentifier }) => (UDIDS[deviceIdentifier] ?? deviceIdentifier) as any
);
mockedUtils.bootAsync.mockResolvedValue(undefined);
mockedUtils.startAsync.mockImplementation(async ({ deviceIdentifier }) => ({
udid: deviceIdentifier as any,
}));
mockedUtils.waitForReadyAsync.mockResolvedValue(undefined);
mockedUtils.disableApsdAsync.mockResolvedValue(undefined);
mockedConfigureProxyEnvironment.mockResolvedValue(false);
mockedInstallGuard.mockResolvedValue(false);
mockedVerifyGuard.mockResolvedValue(undefined);
mockedResolveBootEnvironment.mockResolvedValue(null);
});
it('does not enable accessibility settings by default', async () => {
await createStep({ device_identifier: 'iPhone 15' }).executeAsync();
expect(mockedUtils.enableAccessibilitySettingsAsync).not.toHaveBeenCalled();
expect(mockedUtils.startAsync).toHaveBeenCalledWith({
expect(mockedUtils.resolveUdidAsync).toHaveBeenCalledWith({
deviceIdentifier: 'iPhone 15',
env: expect.any(Object),
});
expect(mockedUtils.bootAsync).toHaveBeenCalledWith({
deviceIdentifier: 'base',
env: expect.any(Object),
launchdEnvironment: {},
});
expect(mockedUtils.startAsync).toHaveBeenCalledWith({
deviceIdentifier: 'base',
env: expect.any(Object),
});
});
it('enables accessibility settings before starting the main device and every clone when requested', async () => {
mockedUtils.startAsync
.mockResolvedValueOnce({ udid: 'base' as any })
.mockResolvedValueOnce({ udid: 'clone-1' as any })
.mockResolvedValueOnce({ udid: 'clone-2' as any });
await createStep({
device_identifier: 'iPhone 15',
count: 2,
@@ -89,11 +131,6 @@ describe(createStartIosSimulatorBuildFunction, () => {
});
it('disables apsd on the main device and every clone', async () => {
mockedUtils.startAsync
.mockResolvedValueOnce({ udid: 'base' as any })
.mockResolvedValueOnce({ udid: 'clone-1' as any })
.mockResolvedValueOnce({ udid: 'clone-2' as any });
await createStep({ device_identifier: 'iPhone 15', count: 2 }).executeAsync();
expect(mockedUtils.disableApsdAsync).toHaveBeenCalledWith({
@@ -110,6 +147,84 @@ describe(createStartIosSimulatorBuildFunction, () => {
});
});
it('installs the proxy environment and guard between boot and boot completion, then verifies the guard', async () => {
mockedInstallGuard.mockResolvedValue(true);
await createStep({ device_identifier: 'iPhone 15', count: 2 }).executeAsync();
const udids = ['base', 'clone-1', 'clone-2'];
expect(
mockedUtils.bootAsync.mock.calls.map(([{ deviceIdentifier }]) => deviceIdentifier)
).toEqual(udids);
expect(mockedConfigureProxyEnvironment.mock.calls.map(([{ udid }]) => udid)).toEqual(udids);
expect(mockedInstallGuard.mock.calls.map(([{ udid }]) => udid)).toEqual(udids);
expect(mockedVerifyGuard.mock.calls.map(([{ udid }]) => udid)).toEqual(udids);
for (const [callIndex] of udids.entries()) {
const bootOrder = mockedUtils.bootAsync.mock.invocationCallOrder[callIndex];
const configureOrder = mockedConfigureProxyEnvironment.mock.invocationCallOrder[callIndex];
const guardOrder = mockedInstallGuard.mock.invocationCallOrder[callIndex];
const bootCompleteOrder = mockedUtils.startAsync.mock.invocationCallOrder[callIndex];
const verifyOrder = mockedVerifyGuard.mock.invocationCallOrder[callIndex];
const readyOrder = mockedUtils.waitForReadyAsync.mock.invocationCallOrder[callIndex];
// launchd is up when boot returns and has spawned nothing yet: that is the
// only moment at which its environment reaches every process of the boot.
expect(guardOrder).toBeGreaterThan(bootOrder);
expect(configureOrder).toBeGreaterThan(guardOrder);
expect(configureOrder).toBeLessThan(bootCompleteOrder);
// The self-check needs a completed boot and runs before anything else.
expect(verifyOrder).toBeGreaterThan(bootCompleteOrder);
expect(verifyOrder).toBeLessThan(readyOrder);
}
});
it('boots with the local egress environment so the first processes inherit it', async () => {
mockedResolveBootEnvironment.mockResolvedValue({
DYLD_INSERT_LIBRARIES: '/w/bin/egress-guard.dylib',
https_proxy: 'http://127.0.0.1:8899',
});
await createStep({ device_identifier: 'iPhone 15', count: 2 }).executeAsync();
for (const [{ deviceIdentifier, launchdEnvironment }] of mockedUtils.bootAsync.mock.calls) {
expect(['base', 'clone-1', 'clone-2']).toContain(deviceIdentifier);
expect(launchdEnvironment).toEqual({
DYLD_INSERT_LIBRARIES: '/w/bin/egress-guard.dylib',
https_proxy: 'http://127.0.0.1:8899',
});
}
});
it('boots with an empty launchd environment when no local egress session is active', async () => {
await createStep({ device_identifier: 'iPhone 15' }).executeAsync();
expect(mockedUtils.bootAsync).toHaveBeenCalledWith({
deviceIdentifier: 'base',
env: expect.any(Object),
launchdEnvironment: {},
});
});
it('skips the guard verification when no local egress session is active', async () => {
mockedInstallGuard.mockResolvedValue(false);
await createStep({ device_identifier: 'iPhone 15' }).executeAsync();
expect(mockedVerifyGuard).not.toHaveBeenCalled();
});
it('fails the step when the guard cannot be installed or verified', async () => {
mockedInstallGuard.mockRejectedValueOnce(new Error('guard library missing'));
await expect(createStep({ device_identifier: 'iPhone 15' }).executeAsync()).rejects.toThrow(
'guard library missing'
);
mockedInstallGuard.mockResolvedValue(true);
mockedVerifyGuard.mockRejectedValueOnce(new Error('self-check failed'));
await expect(createStep({ device_identifier: 'iPhone 15' }).executeAsync()).rejects.toThrow(
'self-check failed'
);
});
it('continues when disabling apsd fails', async () => {
mockedUtils.disableApsdAsync.mockRejectedValue(new Error('apsd disable failed'));
@@ -0,0 +1,190 @@
import { type bunyan } from '@expo/logger';
import { type BuildStepContext } from '@expo/steps';
import fs from 'node:fs';
import {
LOCAL_EGRESS_HANDOFF_PATH,
configureSystemProxyAsync,
downloadChiselAsync,
startChiselServerAsync,
stopLocalEgressResourcesAsync,
writeLocalEgressHandoffAsync,
} from '../../utils/localEgress';
import { findAvailablePortAsync, startNgrokTunnelAsync } from '../../utils/remoteDeviceRunSession';
import { createStartLocalEgressBuildFunction } from '../startLocalEgress';
jest.mock('../../utils/localEgress', () => ({
...jest.requireActual('../../utils/localEgress'),
configureSystemProxyAsync: jest.fn(),
downloadChiselAsync: jest.fn(),
startChiselServerAsync: jest.fn(),
writeLocalEgressHandoffAsync: jest.fn(),
}));
jest.mock('../../utils/localEgressGuard', () => ({
stopLocalEgressGuardRelaysAsync: jest.fn().mockResolvedValue(undefined),
}));
jest.mock('../../utils/remoteDeviceRunSession', () => ({
findAvailablePortAsync: jest.fn(),
startNgrokTunnelAsync: jest.fn(),
getNgrokAuthtokenOrThrow: () => 'ngrok-token',
getNgrokTunnelDomainOrThrow: () => 'example.com',
}));
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const server = { pid: 12345, getOutput: () => '', stopAsync: jest.fn() };
const tunnel = { url: 'https://egress.example.com', stopAsync: jest.fn() };
async function start(signal?: AbortSignal): Promise<void> {
await createStartLocalEgressBuildFunction().fn!({ logger } as BuildStepContext, {
env: {},
inputs: {},
outputs: {},
signal,
});
}
describe('local egress acquisition', () => {
beforeEach(() => {
jest.clearAllMocks();
jest.spyOn(fs.promises, 'mkdtemp').mockResolvedValue('/tmp/egress-acquisition-test');
jest.spyOn(fs.promises, 'writeFile').mockResolvedValue(undefined);
jest.spyOn(fs.promises, 'rm').mockResolvedValue(undefined);
server.stopAsync.mockResolvedValue(undefined);
tunnel.stopAsync.mockResolvedValue(undefined);
jest.mocked(findAvailablePortAsync).mockResolvedValue(52001);
jest.mocked(downloadChiselAsync).mockResolvedValue('/tmp/egress-acquisition-test/chisel');
jest.mocked(startChiselServerAsync).mockResolvedValue({ process: server, fingerprint: 'key=' });
jest.mocked(startNgrokTunnelAsync).mockResolvedValue(tunnel);
jest.mocked(configureSystemProxyAsync).mockResolvedValue({ service: 'Ethernet' });
jest.mocked(writeLocalEgressHandoffAsync).mockResolvedValue(undefined);
});
afterEach(async () => {
await stopLocalEgressResourcesAsync(logger);
jest.restoreAllMocks();
});
it('stops the detached server and removes credentials when ngrok acquisition fails', async () => {
jest.mocked(startNgrokTunnelAsync).mockRejectedValue(new Error('listener quota'));
await expect(start()).rejects.toThrow('listener quota');
expect(server.stopAsync).toHaveBeenCalledTimes(1);
expect(configureSystemProxyAsync).not.toHaveBeenCalled();
expect(fs.promises.rm).toHaveBeenCalledWith('/tmp/egress-acquisition-test', {
recursive: true,
force: true,
});
});
it('attempts every release even when one resource cannot stop', async () => {
await start();
tunnel.stopAsync.mockRejectedValue(new Error('ngrok close failed'));
await stopLocalEgressResourcesAsync(logger);
expect(server.stopAsync).toHaveBeenCalledTimes(1);
expect(fs.promises.rm).toHaveBeenCalledWith(LOCAL_EGRESS_HANDOFF_PATH, { force: true });
expect(logger.warn).toHaveBeenCalledWith(
expect.objectContaining({ err: expect.any(Error) }),
'Could not stop a local egress resource.'
);
});
it.each([false, true])(
'disposes late listeners without delaying cleanup (close fails: %s)',
async closeFails => {
let finishTunnel!: (value: typeof tunnel) => void;
jest.mocked(startNgrokTunnelAsync).mockImplementation(
() =>
new Promise(resolve => {
finishTunnel = resolve;
})
);
const started = start();
const rejected = expect(started).rejects.toThrow();
await new Promise(resolve => setImmediate(resolve));
expect(startNgrokTunnelAsync).toHaveBeenCalledTimes(1);
const stopped = stopLocalEgressResourcesAsync(logger);
await Promise.all([stopped, rejected]);
expect(server.stopAsync).toHaveBeenCalledTimes(1);
expect(tunnel.stopAsync).not.toHaveBeenCalled();
if (closeFails) {
tunnel.stopAsync.mockRejectedValue(new Error('late close failed'));
}
finishTunnel(tunnel);
await new Promise(resolve => setImmediate(resolve));
if (closeFails) {
expect(logger.warn).toHaveBeenCalledWith(
expect.objectContaining({ err: expect.any(Error) }),
'Could not release a cancelled local egress acquisition.'
);
}
expect(tunnel.stopAsync).toHaveBeenCalledTimes(1);
expect(configureSystemProxyAsync).not.toHaveBeenCalled();
expect(writeLocalEgressHandoffAsync).not.toHaveBeenCalled();
}
);
it.each(['job', 'step'])('finishes %s cancellation when ngrok never settles', async source => {
jest.mocked(startNgrokTunnelAsync).mockReturnValue(new Promise(() => {}));
const controller = new AbortController();
const started = start(controller.signal);
const rejected = expect(started).rejects.toThrow();
await new Promise(resolve => setImmediate(resolve));
expect(startNgrokTunnelAsync).toHaveBeenCalledTimes(1);
if (source === 'step') {
controller.abort(new Error('cancelled'));
}
const stopped = source === 'job' ? stopLocalEgressResourcesAsync(logger) : Promise.resolve();
const outcome = await Promise.race([
Promise.all([stopped, rejected]).then(() => 'stopped'),
new Promise(resolve => setImmediate(() => resolve('still waiting'))),
]);
expect(outcome).toBe('stopped');
expect(server.stopAsync).toHaveBeenCalledTimes(1);
expect(configureSystemProxyAsync).not.toHaveBeenCalled();
expect(writeLocalEgressHandoffAsync).not.toHaveBeenCalled();
});
it('handles ngrok rejection after finalization without an unhandled rejection', async () => {
let rejectTunnel!: (error: Error) => void;
jest.mocked(startNgrokTunnelAsync).mockReturnValue(
new Promise((_resolve, reject) => {
rejectTunnel = reject;
})
);
const started = start();
const rejected = expect(started).rejects.toThrow();
await new Promise(resolve => setImmediate(resolve));
await Promise.all([stopLocalEgressResourcesAsync(logger), rejected]);
rejectTunnel(new Error('late connection failure'));
await new Promise(resolve => setImmediate(resolve));
expect(server.stopAsync).toHaveBeenCalledTimes(1);
expect(configureSystemProxyAsync).not.toHaveBeenCalled();
});
it('does not launch a server after a download finishes following cancellation', async () => {
let finishDownload!: (value: string) => void;
jest.mocked(downloadChiselAsync).mockImplementation(
() =>
new Promise(resolve => {
finishDownload = resolve;
})
);
const controller = new AbortController();
const started = start(controller.signal);
const rejected = expect(started).rejects.toThrow('cancelled');
await new Promise(resolve => setImmediate(resolve));
controller.abort(new Error('cancelled'));
await rejected;
finishDownload('/tmp/chisel');
await new Promise(resolve => setImmediate(resolve));
expect(startChiselServerAsync).not.toHaveBeenCalled();
expect(
jest
.mocked(fs.promises.rm)
.mock.calls.filter(([target]) => target === '/tmp/egress-acquisition-test')
).toHaveLength(2);
expect(fs.promises.rm).toHaveBeenCalledWith('/tmp/egress-acquisition-test', {
recursive: true,
force: true,
});
});
});
@@ -0,0 +1,131 @@
import { SystemError } from '@expo/eas-build-job';
import { Client, fetchExchange } from '@urql/core';
import * as sandboxDaemon from '../../utils/sandboxDaemon';
import { createStartSandboxBuildFunction, markSandboxReadyAsync } from '../startSandbox';
describe('sandbox build functions', () => {
it('aborts an active ready request and keeps its authentication headers', async () => {
const controller = new AbortController();
let requestStarted!: () => void;
const started = new Promise<void>(resolve => {
requestStarted = resolve;
});
const fetchMock = jest.spyOn(globalThis, 'fetch').mockImplementation(async (_input, init) => {
expect(new Headers(init?.headers).get('Authorization')).toBe('Bearer test-token');
return await new Promise<Response>((_resolve, reject) => {
init!.signal!.addEventListener('abort', () => reject(init!.signal!.reason), { once: true });
requestStarted();
});
});
const graphqlClient = new Client({
url: 'https://api.expo.test/graphql',
exchanges: [fetchExchange],
fetchOptions: { headers: { Authorization: 'Bearer test-token' } },
});
try {
const pending = markSandboxReadyAsync(
{ graphqlClient } as any,
'sandbox-id',
controller.signal
);
const rejected = expect(pending).rejects.toMatchObject({ name: 'AbortError' });
await started;
controller.abort();
await rejected;
} finally {
fetchMock.mockRestore();
}
});
it('does not mark the sandbox ready if startup was canceled', async () => {
const controller = new AbortController();
const stopAsync = jest.fn(async () => {});
const start = jest
.spyOn(sandboxDaemon, 'startSandboxDaemonAsync')
.mockImplementation(async options => {
expect(options.signal).toBe(controller.signal);
controller.abort();
return { ready: Promise.resolve(), stopAsync };
});
const mutation = jest.fn();
const fn = createStartSandboxBuildFunction({
env: { __EAS_SANDBOX_MCP_TOKEN: 'token' },
mcpServerUrl: 'ws://localhost:8787',
graphqlClient: { mutation },
} as any);
try {
await expect(
fn.fn!({ logger: {} } as any, {
inputs: { sandbox_id: { value: 'sandbox-id' } },
outputs: {},
env: {},
signal: controller.signal,
})
).rejects.toThrow();
expect(mutation).not.toHaveBeenCalled();
expect(stopAsync).toHaveBeenCalledTimes(1);
} finally {
start.mockRestore();
}
});
it('provides one long-running step', () => {
const ctx = { job: {} } as any;
expect(createStartSandboxBuildFunction(ctx).getFullId()).toBe('eas/start_sandbox');
});
it('throws a system error when the sandbox token is missing', async () => {
const fn = createStartSandboxBuildFunction({
job: {},
env: {},
mcpServerUrl: 'ws://localhost:8787',
} as any);
await expect(
fn.fn!({} as any, {
inputs: { sandbox_id: { value: 'sandbox-id' } },
outputs: {},
env: {},
})
).rejects.toBeInstanceOf(SystemError);
});
it('throws a system error when the MCP server URL is missing', async () => {
const fn = createStartSandboxBuildFunction({
job: {},
env: { __EAS_SANDBOX_MCP_TOKEN: 'sandbox-token' },
} as any);
await expect(
fn.fn!({} as any, {
inputs: { sandbox_id: { value: 'sandbox-id' } },
outputs: {},
env: {},
})
).rejects.toBeInstanceOf(SystemError);
});
it('marks the sandbox as ready', async () => {
const mutation = jest.fn(() => ({
toPromise: async () => ({ data: { sandbox: { markSandboxReady: { id: 'sandbox-id' } } } }),
}));
await markSandboxReadyAsync({ graphqlClient: { mutation } } as any, 'sandbox-id');
expect(mutation).toHaveBeenCalledWith(
expect.anything(),
{ sandboxId: 'sandbox-id' },
undefined
);
});
it('throws a system error when marking the sandbox as ready fails', async () => {
const mutation = jest.fn(() => ({
toPromise: async () => ({ error: new Error('request failed') }),
}));
await expect(
markSandboxReadyAsync({ graphqlClient: { mutation } } as any, 'sandbox-id')
).rejects.toThrow('Failed to mark sandbox sandbox-id as ready.');
});
});
@@ -0,0 +1,104 @@
import { type bunyan } from '@expo/logger';
import { BuildRuntimePlatform, type BuildStepContext, type BuildStepEnv } from '@expo/steps';
import { type CustomBuildContext } from '../../../customBuildContext';
import {
getDeviceRunSessionIdOrThrow,
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
startDeviceWebPreviewWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../../utils/remoteDeviceRunSession';
import { createStartWebPreviewRemoteSessionBuildFunction } from '../startWebPreviewRemoteSession';
jest.mock('../../utils/remoteDeviceRunSession');
const ctx = {} as CustomBuildContext;
const env = {} as BuildStepEnv;
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const stopAsync = jest.fn();
async function runAsync(runtimePlatform: BuildRuntimePlatform): Promise<void> {
const buildFunction = createStartWebPreviewRemoteSessionBuildFunction(ctx);
await buildFunction.fn!(
{
logger,
global: { runtimePlatform },
} as unknown as BuildStepContext,
{
inputs: {
package_version: { value: '1.2.3' },
max_duration_seconds: { value: 120 },
},
outputs: {},
env,
} as never
);
}
describe(createStartWebPreviewRemoteSessionBuildFunction, () => {
beforeEach(() => {
jest.clearAllMocks();
jest.mocked(getDeviceRunSessionIdOrThrow).mockReturnValue('device-run-session-id');
jest.mocked(getNgrokTunnelDomainOrThrow).mockReturnValue('tunnel.example.com');
jest.mocked(selectXcodeDeveloperDirectoryAsync).mockResolvedValue(undefined);
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
previewUrl: 'https://web-preview.example.test',
stopAsync,
});
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockResolvedValue(undefined);
stopAsync.mockResolvedValue(undefined);
});
it('reports the session token when serve-sim minted one', async () => {
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
previewUrl: 'https://web-preview.example.test',
previewToken: 'tok-1',
stopAsync,
});
await runAsync(BuildRuntimePlatform.DARWIN);
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledWith(
expect.objectContaining({
remoteConfig: {
previewUrl: 'https://web-preview.example.test',
previewToken: 'tok-1',
},
})
);
});
it.each([
[BuildRuntimePlatform.DARWIN, true],
[BuildRuntimePlatform.LINUX, false],
])('starts the web preview for %s', async (runtimePlatform, selectsXcode) => {
await runAsync(runtimePlatform);
expect(selectXcodeDeveloperDirectoryAsync).toHaveBeenCalledTimes(selectsXcode ? 1 : 0);
expect(startDeviceWebPreviewWithTunnelAsync).toHaveBeenCalledWith(ctx, {
runtimePlatform,
baseDomain: 'tunnel.example.com',
env,
logger,
timeoutMs: 60_000,
packageVersion: '1.2.3',
});
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledWith({
ctx,
deviceRunSessionId: 'device-run-session-id',
remoteConfig: { previewUrl: 'https://web-preview.example.test' },
logger,
});
expect(waitForDeviceRunSessionStoppedAsync).toHaveBeenCalledWith({
ctx,
deviceRunSessionId: 'device-run-session-id',
logger,
maxDurationSeconds: 120,
signal: undefined,
});
expect(stopAsync).toHaveBeenCalledTimes(1);
});
});
@@ -8,15 +8,19 @@ import {
BuildStepOutput,
} from '@expo/steps';
import { Client } from '@urql/core';
import contentDisposition from 'content-disposition';
import { glob } from 'fast-glob';
import { graphql } from 'gql.tada';
import fetch from 'node-fetch';
import assert from 'node:assert';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import stream from 'stream';
import { promisify } from 'util';
import { z } from 'zod';
import bplistParser from 'bplist-parser';
import plist from 'plist';
import { CustomBuildContext } from '../../customBuildContext';
import { formatBytes } from '../../utils/artifacts';
@@ -26,6 +30,10 @@ import { pluralize } from '../../utils/strings';
const streamPipeline = promisify(stream.pipeline);
type DownloadBuildSource =
| { buildId: string; applicationArchiveUrl?: never }
| { buildId?: never; applicationArchiveUrl: string };
const BUILD_BY_ID_QUERY = graphql(`
query DownloadBuildByIdQuery($buildId: ID!) {
builds {
@@ -49,7 +57,12 @@ export function createDownloadBuildFunction(ctx: CustomBuildContext): BuildFunct
inputProviders: [
BuildStepInput.createProvider({
id: 'build_id',
required: true,
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
id: 'application_archive_url',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
@@ -70,12 +83,39 @@ export function createDownloadBuildFunction(ctx: CustomBuildContext): BuildFunct
const extensions = z.array(z.string()).parse(inputs.extensions.value);
logger.info(`Expected extensions: [${extensions.join(', ')}]`);
const buildId = z.string().uuid().parse(inputs.build_id.value);
logger.info(`Downloading build ${buildId}...`);
const buildId = inputs.build_id.value
? z.string().uuid().parse(inputs.build_id.value)
: undefined;
const applicationArchiveUrl = inputs.application_archive_url.value
? parseHttpApplicationArchiveUrl(inputs.application_archive_url.value)
: undefined;
let source: DownloadBuildSource;
if (buildId) {
if (applicationArchiveUrl) {
throw new UserError(
'EAS_DOWNLOAD_BUILD_INVALID_SOURCE',
'Pass only one of build_id or application_archive_url.'
);
}
source = { buildId };
} else {
if (!applicationArchiveUrl) {
throw new UserError(
'EAS_DOWNLOAD_BUILD_INVALID_SOURCE',
'Pass build_id or application_archive_url.'
);
}
source = { applicationArchiveUrl };
}
logger.info(
buildId ? `Downloading build ${buildId}...` : `Downloading application archive...`
);
const { artifactPath } = await downloadBuildAsync({
logger,
buildId,
...source,
graphqlClient: ctx.graphqlClient,
robotAccessToken: stepsCtx.global.staticContext.job.secrets?.robotAccessToken ?? null,
extensions,
@@ -115,39 +155,50 @@ async function fetchApplicationArchiveUrlAsync({
return applicationArchiveUrl;
}
export async function downloadBuildAsync({
logger,
buildId,
graphqlClient,
robotAccessToken,
extensions,
}: {
logger: bunyan;
buildId: string;
graphqlClient: Client;
robotAccessToken: string | null;
extensions: string[];
}): Promise<{ artifactPath: string }> {
export async function downloadBuildAsync(
params: DownloadBuildSource & {
logger: bunyan;
graphqlClient: Client;
robotAccessToken: string | null;
extensions: string[];
}
): Promise<{ artifactPath: string }> {
const { logger, graphqlClient, robotAccessToken, extensions } = params;
let downloadUrl: string;
let headers: { Authorization: string } | undefined;
if (params.applicationArchiveUrl) {
if (params.buildId) {
throw new UserError(
'EAS_DOWNLOAD_BUILD_INVALID_SOURCE',
'Pass only one of buildId or applicationArchiveUrl.'
);
}
downloadUrl = parseHttpApplicationArchiveUrl(params.applicationArchiveUrl);
headers = undefined;
} else if (params.buildId) {
const buildId = z.string().uuid().parse(params.buildId);
downloadUrl = await fetchApplicationArchiveUrlAsync({ buildId, graphqlClient });
headers = robotAccessToken ? { Authorization: `Bearer ${robotAccessToken}` } : undefined;
} else {
throw new UserError(
'EAS_DOWNLOAD_BUILD_INVALID_SOURCE',
'Pass buildId or applicationArchiveUrl.'
);
}
const downloadDestinationDirectory = await fs.promises.mkdtemp(
path.join(os.tmpdir(), 'download_build-downloaded-')
);
const downloadUrl = await fetchApplicationArchiveUrlAsync({ buildId, graphqlClient });
const response = await retryOnDNSFailure(fetch)(downloadUrl, {
headers: robotAccessToken ? { Authorization: `Bearer ${robotAccessToken}` } : undefined,
});
const response = await retryOnDNSFailure(fetch)(downloadUrl, { headers });
if (!response.ok) {
const textResult = await asyncResult(response.text());
throw new Error(`Unexpected response from server (${response.status}): ${textResult.value}`);
}
// URL may contain percent-encoded characters, e.g. my%20file.apk
// this replaces all non-alphanumeric characters (excluding dot) with underscore
const archiveFilename = path
.basename(new URL(response.url).pathname)
.replace(/([^a-z0-9.-]+)/gi, '_');
const archiveFilename = resolveArchiveFilename({ response, extensions });
const archivePath = path.join(downloadDestinationDirectory, archiveFilename);
await streamPipeline(response.body, fs.createWriteStream(archivePath));
@@ -177,6 +228,18 @@ export async function downloadBuildAsync({
onlyFiles: false,
onlyDirectories: false,
});
let matchingFilesRoot = extractionDirectory;
if (
matchingFiles.length === 0 &&
extensions.includes('app') &&
(await isIosAppBundleAsync(extractionDirectory))
) {
const appBundlePath = `${extractionDirectory}.app`;
await fs.promises.rename(extractionDirectory, appBundlePath);
matchingFiles.push(appBundlePath);
matchingFilesRoot = path.dirname(appBundlePath);
}
if (matchingFiles.length === 0) {
throw new UserError(
@@ -189,8 +252,68 @@ export async function downloadBuildAsync({
`Found ${matchingFiles.length} matching ${pluralize(
matchingFiles.length,
'entry'
)}:\n${matchingFiles.map(f => `- ${path.relative(extractionDirectory, f)}`).join('\n')}`
)}:\n${matchingFiles.map(f => `- ${path.relative(matchingFilesRoot, f)}`).join('\n')}`
);
return { artifactPath: matchingFiles[0] };
}
async function isIosAppBundleAsync(directory: string): Promise<boolean> {
try {
const infoPlist = await fs.promises.readFile(path.join(directory, 'Info.plist'));
const isBinaryPlist = infoPlist.subarray(0, 8).toString('ascii') === 'bplist00';
const parsedInfoPlist = (
isBinaryPlist
? bplistParser.parseBuffer(infoPlist)[0]
: plist.parse(infoPlist.toString('utf8'))
) as Record<string, unknown> | undefined;
return parsedInfoPlist?.CFBundlePackageType === 'APPL';
} catch {
return false;
}
}
function parseHttpApplicationArchiveUrl(value: unknown): string {
try {
const applicationArchiveUrl = z.string().parse(value);
const parsedUrl = new URL(applicationArchiveUrl);
assert(parsedUrl.protocol === 'http:' || parsedUrl.protocol === 'https:');
return applicationArchiveUrl;
} catch {
throw new UserError(
'EAS_DOWNLOAD_BUILD_INVALID_APPLICATION_ARCHIVE_URL',
'application_archive_url must be a valid HTTP or HTTPS URL.'
);
}
}
function resolveArchiveFilename({
response,
extensions,
}: {
response: Awaited<ReturnType<typeof fetch>>;
extensions: string[];
}): string {
const contentDispositionHeader = response.headers.get('content-disposition');
let headerFilename: string | undefined;
if (contentDispositionHeader) {
try {
headerFilename = contentDisposition.parse(contentDispositionHeader).parameters.filename;
} catch {
// Ignore malformed Content-Disposition headers and fall back to the response URL.
}
}
const urlFilename = path.basename(new URL(response.url).pathname);
let archiveFilename = path.basename(headerFilename ?? urlFilename ?? '');
if (!archiveFilename || archiveFilename === '.' || archiveFilename === '..') {
archiveFilename = 'application';
}
if (!path.extname(archiveFilename) && extensions.length === 1) {
archiveFilename = `${archiveFilename}.${extensions[0]}`;
}
// URL and header filenames may contain percent-encoded or unsafe filesystem characters.
return archiveFilename.replace(/([^a-z0-9.-]+)/gi, '_');
}
@@ -0,0 +1,124 @@
import { UserError } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import {
BuildFunction,
BuildRuntimePlatform,
BuildStepEnv,
BuildStepInput,
BuildStepInputValueTypeName,
BuildStepOutput,
} from '@expo/steps';
import spawn from '@expo/turtle-spawn';
import fs from 'node:fs';
import path from 'node:path';
import { z } from 'zod';
export function createInstallBuildFunction(): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'install_build',
name: 'Install build',
__metricsId: 'eas/install_build',
inputProviders: [
BuildStepInput.createProvider({
id: 'artifact_path',
required: true,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
],
outputProviders: [
BuildStepOutput.createProvider({
id: 'application_identifier',
required: true,
}),
BuildStepOutput.createProvider({
id: 'activity_name',
required: false,
}),
],
fn: async ({ global, logger }, { inputs, outputs, env }) => {
const artifactPath = z.string().min(1).parse(inputs.artifact_path.value);
const { applicationIdentifier, activityName } = await installBuildAsync({
artifactPath,
runtimePlatform: global.runtimePlatform,
env,
logger,
});
outputs.application_identifier.set(applicationIdentifier);
if (activityName) {
outputs.activity_name.set(activityName);
}
},
});
}
export async function installBuildAsync({
artifactPath,
runtimePlatform,
env,
logger,
}: {
artifactPath: string;
runtimePlatform: BuildRuntimePlatform;
env: BuildStepEnv;
logger: bunyan;
}): Promise<{ applicationIdentifier: string; activityName?: string }> {
const artifactStat = await fs.promises.stat(artifactPath).catch(err => {
throw new UserError(
'EAS_INSTALL_BUILD_INVALID_ARTIFACT',
`Build artifact does not exist at ${artifactPath}.`,
{ cause: err }
);
});
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
if (path.extname(artifactPath) !== '.app' || !artifactStat.isDirectory()) {
throw new UserError(
'EAS_INSTALL_BUILD_INVALID_ARTIFACT',
'iOS Simulator sessions require a .app build artifact.'
);
}
const infoPlistPath = path.join(artifactPath, 'Info.plist');
const { stdout } = await spawn(
'plutil',
['-extract', 'CFBundleIdentifier', 'raw', '-o', '-', infoPlistPath],
{ stdio: 'pipe', env }
);
const applicationIdentifier = stdout.trim();
if (!applicationIdentifier) {
throw new UserError(
'EAS_INSTALL_BUILD_MISSING_IDENTIFIER',
`Could not read CFBundleIdentifier from ${infoPlistPath}.`
);
}
logger.info(`Installing ${artifactPath} on the iOS Simulator.`);
await spawn('xcrun', ['simctl', 'install', 'booted', artifactPath], { env, logger });
return { applicationIdentifier };
}
if (path.extname(artifactPath) !== '.apk' || !artifactStat.isFile()) {
throw new UserError(
'EAS_INSTALL_BUILD_INVALID_ARTIFACT',
'Android Emulator sessions require an .apk build artifact.'
);
}
const { stdout } = await spawn('aapt2', ['dump', 'badging', artifactPath], {
stdio: 'pipe',
env,
});
const applicationIdentifier = stdout.match(/package: name='([^']+)'/)?.[1];
const activityName = stdout.match(/launchable-activity: name='([^']+)'/)?.[1];
if (!applicationIdentifier) {
throw new UserError(
'EAS_INSTALL_BUILD_MISSING_IDENTIFIER',
`Could not read an Android application identifier from ${artifactPath}.`
);
}
logger.info(`Installing ${artifactPath} on the Android Emulator.`);
await spawn('adb', ['install', '-r', artifactPath], { env, logger });
return { applicationIdentifier, activityName };
}
@@ -1,3 +1,4 @@
import downloadFile from '@expo/downloader';
import { SystemError, UserError } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import { asyncResult } from '@expo/results';
@@ -19,6 +20,11 @@ import semver from 'semver';
import { MaestroBackend, resolveMaestroBackend } from './maestroBackend';
import { Datadog } from '../../datadog';
import { getXcodeVersionAsync } from '../../ios/xcode';
import { IosSimulatorUtils } from '../../utils/IosSimulatorUtils';
import { getProxiedDownloadUrl } from '../../utils/download';
const MAESTRO_RUNNER_WDA_CACHE_DOWNLOAD_TIMEOUT_MS = 20_000;
export function createInstallMaestroBuildFunction(): BuildFunction {
return new BuildFunction({
@@ -148,11 +154,13 @@ export function createInstallMaestroBuildFunction(): BuildFunction {
(requestedVersion === undefined &&
(!currentMaestroRunnerVersion || semver.gt(currentMaestroRunnerVersion, '1.1.15'))))
) {
const xcodeVersion = await getXcodeVersion({ env });
// `getXcodeVersionAsync` returns the raw version (e.g. "16.4"), which
// `semver.coerce` normalizes (e.g. "16.4.0") for comparison and display.
const xcodeVersion = semver.coerce(await getXcodeVersionAsync({ env }));
// maestro-runner 1.1.16 added `arch` to its xcodebuild destination. Xcode versions
// below 26 reject that option, so use the last compatible maestro-runner version.
if (semver.lt(xcodeVersion, '26.0.0')) {
if (xcodeVersion && semver.lt(xcodeVersion, '26.0.0')) {
if (requestedMaestroRunnerVersion) {
throw new UserError(
'ERR_MAESTRO_INVALID_INPUT',
@@ -190,6 +198,13 @@ export function createInstallMaestroBuildFunction(): BuildFunction {
throw new Error(`Failed to ensure ${backend} is installed.`);
}
if (backend === 'maestro-runner' && global.runtimePlatform === BuildRuntimePlatform.DARWIN) {
await installMaestroRunnerWdaCache({
logger,
env,
});
}
logger.info(`${backend} ${maestroVersionResult.value} is ready.`);
outputs.maestro_version.set(maestroVersionResult.value);
@@ -201,19 +216,140 @@ export function createInstallMaestroBuildFunction(): BuildFunction {
});
}
async function getXcodeVersion({ env }: { env: BuildStepEnv }): Promise<string> {
let stdout: string;
try {
({ stdout } = await spawn('xcodebuild', ['-version'], { stdio: 'pipe', env }));
} catch (error) {
throw new SystemError('Failed to get Xcode version', { cause: error });
async function installMaestroRunnerWdaCache({
logger,
env,
}: {
logger: bunyan;
env: BuildStepEnv;
}): Promise<void> {
if (!env.HOME) {
logger.warn(
'Skipping the prebuilt WebDriverAgent cache because the $HOME environment variable is empty.'
);
return;
}
const maestroRunnerHome = path.join(env.HOME, '.maestro-runner');
const xcodeVersion = semver.coerce(/^Xcode\s+(\S+)/m.exec(stdout)?.[1])?.version;
if (!xcodeVersion) {
throw new SystemError(`Failed to parse Xcode version from xcodebuild output: ${stdout.trim()}`);
try {
const wdaVersion = await getMaestroRunnerWdaVersion({ maestroRunnerHome });
if (!wdaVersion) {
logger.info(
'Skipping the prebuilt WebDriverAgent cache because the installed WDA version is unknown.'
);
return;
}
const xcodeVersion = await getXcodeVersionAsync({ env });
const iosRuntimes = await IosSimulatorUtils.getAvailableRuntimesAsync({ env });
const iosRuntimeIdentifierPrefix = 'com.apple.CoreSimulator.SimRuntime.iOS-';
// maestro-runner derives its WDA cache key from the runtime identifier, not from the runtime
// version. These values can differ for patched runtimes, for example the iOS-26-3 identifier
// can report version 26.3.1.
const iosRuntimeVersions = [
...new Set(
iosRuntimes.map(runtime =>
runtime.identifier.slice(iosRuntimeIdentifierPrefix.length).replaceAll('-', '.')
)
),
];
if (iosRuntimeVersions.length === 0) {
logger.info(
'Skipping the prebuilt WebDriverAgent cache because no iOS runtime is available.'
);
return;
}
const tempDirectory = await fs.promises.mkdtemp(
path.join(os.tmpdir(), 'install_maestro_runner_wda_cache')
);
try {
const archivePath = path.join(tempDirectory, 'wda-cache.tar.gz');
const directArchiveUrl =
`https://storage.googleapis.com/turtle-v2/maestro-runner-wda-cache/` +
`xcode-${encodeURIComponent(xcodeVersion)}-wda-${encodeURIComponent(wdaVersion)}.tar.gz`;
const proxiedArchiveUrl = getProxiedDownloadUrl({
directUrl: directArchiveUrl,
proxyBaseUrl: env.EAS_BUILD_COCOAPODS_CACHE_URL,
});
logger.info(`Downloading the prebuilt WebDriverAgent cache for Xcode ${xcodeVersion}`);
if (proxiedArchiveUrl) {
try {
await downloadFile(proxiedArchiveUrl, archivePath, {
retry: 3,
timeout: MAESTRO_RUNNER_WDA_CACHE_DOWNLOAD_TIMEOUT_MS,
});
} catch (err) {
logger.debug(
{ err },
'Failed to download the prebuilt WebDriverAgent cache via the proxy; falling back to the direct URL.'
);
await downloadFile(directArchiveUrl, archivePath, {
retry: 3,
timeout: MAESTRO_RUNNER_WDA_CACHE_DOWNLOAD_TIMEOUT_MS,
});
}
} else {
await downloadFile(directArchiveUrl, archivePath, {
retry: 3,
timeout: MAESTRO_RUNNER_WDA_CACHE_DOWNLOAD_TIMEOUT_MS,
});
}
await fs.promises.mkdir(maestroRunnerHome, { recursive: true });
await spawn('tar', ['-xzf', archivePath, '-C', maestroRunnerHome], { logger, env });
const genericProductsDirectory = path.join(
maestroRunnerHome,
'cache',
'wda-builds',
'generic',
'DerivedData',
'Build',
'Products'
);
for (const runtimeVersion of iosRuntimeVersions) {
const productsDirectory = path.join(
maestroRunnerHome,
'cache',
'wda-builds',
`sim-ios${runtimeVersion}-iphone`,
'DerivedData',
'Build',
'Products'
);
await fs.promises.cp(genericProductsDirectory, productsDirectory, { recursive: true });
}
logger.info(
`Installed the prebuilt WebDriverAgent cache for iOS ${iosRuntimeVersions.join(', ')}.`
);
} finally {
await fs.promises.rm(tempDirectory, { force: true, recursive: true });
}
} catch (err: any) {
logger.warn(
{ err },
'Failed to install the prebuilt WebDriverAgent cache. maestro-runner will build WebDriverAgent when it is needed.'
);
}
}
async function getMaestroRunnerWdaVersion({
maestroRunnerHome,
}: {
maestroRunnerHome: string;
}): Promise<string | null> {
try {
const packageJson = JSON.parse(
await fs.promises.readFile(
path.join(maestroRunnerHome, 'drivers', 'ios', 'WebDriverAgent', 'package.json'),
'utf8'
)
);
return typeof packageJson.version === 'string' ? packageJson.version : null;
} catch {
return null;
}
return xcodeVersion;
}
async function getMaestroVersion({
@@ -0,0 +1,219 @@
import { UserError } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import {
BuildFunction,
BuildRuntimePlatform,
BuildStepEnv,
BuildStepInput,
BuildStepInputValueTypeName,
} from '@expo/steps';
import spawn from '@expo/turtle-spawn';
const IOS_URL_SCHEME_APPROVAL_DOMAIN = 'com.apple.launchservices.schemeapproval';
const IOS_URL_SCHEME_APPROVAL_KEY_PREFIX = 'com.apple.CoreSimulator.CoreSimulatorBridge-->';
export function createLaunchApplicationFunction(): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'launch_application',
name: 'Launch application',
__metricsId: 'eas/launch_application',
inputProviders: [
BuildStepInput.createProvider({
id: 'application_identifier',
required: true,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
id: 'activity_name',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
id: 'launch_args',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.JSON,
}),
BuildStepInput.createProvider({
id: 'open_url',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
],
fn: async ({ global, logger }, { inputs, env }) => {
const applicationIdentifier = parseNonEmptyStringInput(
inputs.application_identifier.value,
'application_identifier'
);
const activityName =
inputs.activity_name.value === undefined
? undefined
: parseNonEmptyStringInput(inputs.activity_name.value, 'activity_name');
const launchArgs = parseLaunchArgsInput(inputs.launch_args.value);
const openUrl =
inputs.open_url.value === undefined ? undefined : parseOpenUrlInput(inputs.open_url.value);
await launchApplicationAsync({
applicationIdentifier,
activityName,
launchArgs,
openUrl,
runtimePlatform: global.runtimePlatform,
env,
logger,
});
},
});
}
export async function launchApplicationAsync({
applicationIdentifier,
activityName,
launchArgs = [],
openUrl,
runtimePlatform,
env,
logger,
}: {
applicationIdentifier: string;
activityName?: string;
launchArgs?: string[];
openUrl?: string;
runtimePlatform: BuildRuntimePlatform;
env: BuildStepEnv;
logger: bunyan;
}): Promise<void> {
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
logApplicationLaunch(logger, applicationIdentifier, launchArgs);
await spawn('xcrun', ['simctl', 'launch', 'booted', applicationIdentifier, ...launchArgs], {
env,
logger,
});
if (openUrl) {
await preapproveIosUrlSchemeAsync({ applicationIdentifier, openUrl, env, logger });
logger.info(`Opening ${openUrl} in ${applicationIdentifier}.`);
await spawn('xcrun', ['simctl', 'openurl', 'booted', openUrl], { env, logger });
}
return;
}
if (!activityName) {
throw new UserError(
'EAS_LAUNCH_APPLICATION_MISSING_ACTIVITY',
'Launching an Android application requires activity_name.'
);
}
logApplicationLaunch(logger, applicationIdentifier, launchArgs);
// Android does not support process arguments like iOS. Pass raw `am start` Intent
// arguments instead, such as `--es key value` or `--ez key true`.
await spawn(
'adb',
['shell', 'am', 'start', ...launchArgs, '-n', `${applicationIdentifier}/${activityName}`],
{
env,
logger,
}
);
if (openUrl) {
logger.info(`Opening ${openUrl} in ${applicationIdentifier}.`);
await spawn(
'adb',
[
'shell',
'am',
'start',
'-a',
'android.intent.action.VIEW',
'-d',
openUrl,
'-n',
`${applicationIdentifier}/${activityName}`,
],
{ env, logger }
);
}
}
async function preapproveIosUrlSchemeAsync({
applicationIdentifier,
openUrl,
env,
logger,
}: {
applicationIdentifier: string;
openUrl: string;
env: BuildStepEnv;
logger: bunyan;
}): Promise<void> {
const urlScheme = new URL(openUrl).protocol.slice(0, -1);
if (urlScheme === 'http' || urlScheme === 'https') {
return;
}
try {
await spawn(
'xcrun',
[
'simctl',
'spawn',
'booted',
'defaults',
'write',
IOS_URL_SCHEME_APPROVAL_DOMAIN,
`${IOS_URL_SCHEME_APPROVAL_KEY_PREFIX}${urlScheme}`,
'-string',
applicationIdentifier,
],
{ env, logger }
);
} catch (error) {
logger.warn(
{ err: error },
`Could not preapprove the ${urlScheme} URL scheme for ${applicationIdentifier}. Opening the URL anyway; the Simulator might require manual confirmation.`
);
}
}
function logApplicationLaunch(
logger: bunyan,
applicationIdentifier: string,
launchArgs: string[]
): void {
const argumentsDescription =
launchArgs.length > 0 ? ` with arguments ${JSON.stringify(launchArgs)}` : '';
logger.info(`Launching ${applicationIdentifier}${argumentsDescription}.`);
}
function parseNonEmptyStringInput(value: unknown, inputName: string): string {
if (typeof value !== 'string' || value.length === 0) {
throw new UserError(
'EAS_LAUNCH_APPLICATION_INVALID_INPUT',
`Input "${inputName}" must be a non-empty string. Pass the "${inputName}" output from eas/install_build.`
);
}
return value;
}
function parseLaunchArgsInput(value: unknown): string[] {
if (value === undefined) {
return [];
}
if (!Array.isArray(value) || !value.every(argument => typeof argument === 'string')) {
throw new UserError(
'EAS_LAUNCH_APPLICATION_INVALID_INPUT',
'Input "launch_args" must be an array of strings.'
);
}
return value;
}
function parseOpenUrlInput(value: unknown): string {
const openUrl = parseNonEmptyStringInput(value, 'open_url');
if (!URL.canParse(openUrl)) {
throw new UserError(
'EAS_LAUNCH_APPLICATION_INVALID_INPUT',
'Input "open_url" must be a valid URL.'
);
}
return openUrl;
}
@@ -16,6 +16,10 @@ export type IpaInfo = {
bundleIdentifier: string;
bundleShortVersion: string;
bundleVersion: string;
// `DTPlatformName` from the bundle's Info.plist (e.g. `iphoneos`, `appletvos`,
// `macosx`, `xros`), or null when absent. Used to pick the App Store Connect
// platform for the upload. See `AscApiUtils.ascPlatformFromDtPlatformName`.
dtPlatformName: string | null;
};
const INFO_PLIST_PATH_REGEXP = /^Payload\/[^/]+\.app\/Info\.plist$/;
@@ -91,10 +95,14 @@ export async function readIpaInfoAsync(ipaPath: string): Promise<IpaInfo> {
);
}
const dtPlatformName =
typeof infoPlist.DTPlatformName === 'string' ? infoPlist.DTPlatformName : null;
return {
bundleIdentifier,
bundleShortVersion,
bundleVersion,
dtPlatformName,
};
} catch (error) {
if (error instanceof UserError) {
@@ -62,6 +62,12 @@ export function createRepackBuildFunction(): BuildFunction {
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: false,
}),
BuildStepInput.createProvider({
id: 'ios_signing_backend',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: false,
allowedValues: ['fastlane', 'zsign'],
}),
BuildStepInput.createProvider({
id: 'repack_version',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
@@ -144,12 +150,14 @@ export function createRepackBuildFunction(): BuildFunction {
iosSigningOptions: await resolveIosSigningOptionsAsync({
job: stepsCtx.global.staticContext.job,
logger: stepsCtx.logger,
backend: inputs.ios_signing_backend.value as 'fastlane' | 'zsign' | undefined,
useAppEntitlements: inputs.ios_signing_use_source_app_entitlements.value as
| boolean
| undefined,
entitlementsPath: inputs.ios_signing_app_entitlements_path.value as
| string
| undefined,
tmpDir,
}),
logger: stepsCtx.logger,
spawnAsync: repackSpawnAsync,
@@ -297,24 +305,51 @@ export async function resolveAndroidSigningOptionsAsync({
}
/**
* Resolves iOS signing options from the job secrets.
* Resolves iOS signing options from the job secrets, dispatching on the
* requested signing backend.
*/
export async function resolveIosSigningOptionsAsync({
job,
logger,
backend,
useAppEntitlements,
entitlementsPath,
tmpDir,
}: {
job: Job;
logger: bunyan;
backend?: 'fastlane' | 'zsign';
useAppEntitlements?: boolean;
entitlementsPath?: string;
tmpDir: string;
}): Promise<IosSigningOptions | undefined> {
const iosJob = job as Ios.Job;
const buildCredentials = iosJob.secrets?.buildCredentials;
if (iosJob.simulator || buildCredentials == null) {
return undefined;
}
const commonOptions = { buildCredentials, logger, useAppEntitlements, entitlementsPath };
return backend === 'zsign'
? await createIosZsignOptionsAsync({ ...commonOptions, tmpDir })
: await createIosFastlaneOptionsAsync(commonOptions);
}
/**
* Creates signing options for the fastlane backend: certificates are imported
* into a temporary keychain and provisioning profiles are parsed with the
* macOS `security` tool.
*/
async function createIosFastlaneOptionsAsync({
buildCredentials,
logger,
useAppEntitlements,
entitlementsPath,
}: {
buildCredentials: Ios.BuildCredentials;
logger: bunyan;
useAppEntitlements?: boolean;
entitlementsPath?: string;
}): Promise<IosSigningOptions> {
const credentialsManager = new IosCredentialsManager(buildCredentials);
const credentials = await credentialsManager.prepare(logger);
@@ -330,3 +365,53 @@ export async function resolveIosSigningOptionsAsync({
entitlementsPath,
};
}
/**
* Creates signing options for the zsign backend. The distribution certificate
* secret is already a PKCS#12 file, so it goes to disk as-is together with the
* provisioning profiles.
*/
async function createIosZsignOptionsAsync({
buildCredentials,
logger,
useAppEntitlements,
entitlementsPath,
tmpDir,
}: {
buildCredentials: Ios.BuildCredentials;
logger: bunyan;
useAppEntitlements?: boolean;
entitlementsPath?: string;
tmpDir: string;
}): Promise<IosSigningOptions> {
const targets = Object.entries(buildCredentials);
const [targetName, targetCredentials] = targets[0];
logger.info(`Using the distribution certificate from target '${targetName}' for zsign`);
const certificatePath = path.join(tmpDir, `dist-cert-${randomUUID()}.p12`);
await fs.promises.writeFile(
certificatePath,
new Uint8Array(Buffer.from(targetCredentials.distributionCertificate.dataBase64, 'base64'))
);
// zsign matches profiles to bundles by the app-id suffix itself, so the
// record keys are informational only.
const provisioningProfile: Record<string, string> = {};
for (const [target, credentials] of targets) {
const profilePath = path.join(tmpDir, `profile-${target}-${randomUUID()}.mobileprovision`);
await fs.promises.writeFile(
profilePath,
new Uint8Array(Buffer.from(credentials.provisioningProfileBase64, 'base64'))
);
provisioningProfile[target] = profilePath;
}
return {
backend: 'zsign',
certificatePath,
keyPassword: targetCredentials.distributionCertificate.password,
provisioningProfile,
useAppEntitlements,
entitlementsPath,
};
}
@@ -15,10 +15,16 @@ import path from 'path';
import { sendCcacheStatsAsync } from './ccacheStats';
import { decompressCacheAsync, downloadCacheAsync, downloadPublicCacheAsync } from './restoreCache';
import {
CACHE_KEY_PREFIX_BY_PLATFORM,
CcacheBuildTarget,
generateDefaultBuildCacheKeyAsync,
getCcacheKeyPrefix,
getCcachePath,
} from '../../utils/cacheKey';
import {
getCocoapodsCachePaths,
resolveCocoapodsCacheKeyAsync,
restoreCocoapodsCacheArchiveAsync,
} from '../../utils/cocoapodsCache';
import { Datadog } from '../../datadog';
import { GRADLE_CACHE_KEY_PREFIX, generateGradleCacheKeyAsync } from '../../utils/gradleCacheKey';
import { TurtleFetchError, turtleFetch } from '../../utils/turtleFetch';
@@ -35,6 +41,11 @@ export function createRestoreBuildCacheFunction(): BuildFunction {
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
id: 'simulator',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.BOOLEAN,
}),
],
fn: async (stepCtx, { env, inputs }) => {
const { logger } = stepCtx;
@@ -48,10 +59,20 @@ export function createRestoreBuildCacheFunction(): BuildFunction {
);
}
const target: CcacheBuildTarget =
platform === Platform.IOS
? {
platform,
simulator:
(inputs.simulator.value as boolean | undefined) ??
(stepCtx.global.staticContext.job.platform === Platform.IOS &&
stepCtx.global.staticContext.job.simulator === true),
}
: { platform };
await restoreCcacheAsync({
logger,
workingDirectory,
platform,
target,
env,
secrets: stepCtx.global.staticContext.job.secrets,
});
@@ -63,6 +84,13 @@ export function createRestoreBuildCacheFunction(): BuildFunction {
env,
secrets: stepCtx.global.staticContext.job.secrets,
});
} else {
await restoreCocoapodsCacheAsync({
logger,
workingDirectory,
env,
secrets: stepCtx.global.staticContext.job.secrets,
});
}
},
});
@@ -92,13 +120,13 @@ export function createCacheStatsBuildFunction(): BuildFunction {
export async function restoreCcacheAsync({
logger,
workingDirectory,
platform,
target,
env,
secrets,
}: {
logger: bunyan;
workingDirectory: string;
platform: Platform;
target: CcacheBuildTarget;
env: Record<string, string | undefined>;
secrets?: { robotAccessToken?: string };
}): Promise<void> {
@@ -136,7 +164,7 @@ export async function restoreCcacheAsync({
})
);
const cacheKey = await generateDefaultBuildCacheKeyAsync(workingDirectory, platform);
const cacheKey = await generateDefaultBuildCacheKeyAsync(workingDirectory, target);
logger.info(`Restoring cache key: ${cacheKey}`);
const jobId = nullthrows(env.EAS_BUILD_ID, 'EAS_BUILD_ID is not set');
@@ -147,8 +175,8 @@ export async function restoreCcacheAsync({
robotAccessToken,
paths: [cachePath],
key: cacheKey,
keyPrefixes: [CACHE_KEY_PREFIX_BY_PLATFORM[platform]],
platform,
keyPrefixes: [getCcacheKeyPrefix(target)],
platform: target.platform,
});
await decompressCacheAsync({
@@ -175,7 +203,7 @@ export async function restoreCcacheAsync({
expoApiServerURL,
robotAccessToken,
paths: [cachePath],
platform,
platform: target.platform,
});
await decompressCacheAsync({
archivePath,
@@ -190,6 +218,67 @@ export async function restoreCcacheAsync({
}
}
export async function restoreCocoapodsCacheAsync({
logger,
workingDirectory,
env,
secrets,
}: {
logger: bunyan;
workingDirectory: string;
env: Record<string, string | undefined>;
secrets?: { robotAccessToken?: string };
}): Promise<void> {
if (env.EAS_PODS_CACHE !== '1') {
return;
}
try {
const { stdout } = await spawnAsync('pod', ['--version'], {
env,
stdio: 'pipe',
});
const { key, keyPrefix } = await resolveCocoapodsCacheKeyAsync(workingDirectory, stdout);
logger.info(`Restoring CocoaPods cache key: ${key}`);
const jobId = nullthrows(env.EAS_BUILD_ID, 'EAS_BUILD_ID is not set');
const robotAccessToken = nullthrows(
secrets?.robotAccessToken,
'Robot access token is required for cache operations'
);
const expoApiServerURL = nullthrows(env.__API_SERVER_URL, '__API_SERVER_URL is not set');
const { podsDirectory } = getCocoapodsCachePaths(workingDirectory);
const { archivePath, matchedKey } = await downloadCacheAsync({
logger,
jobId,
expoApiServerURL,
robotAccessToken,
paths: [podsDirectory],
key,
keyPrefixes: [keyPrefix],
platform: Platform.IOS,
});
await restoreCocoapodsCacheArchiveAsync({ archivePath, workingDirectory });
const hitType = matchedKey === key ? 'direct_hit' : 'prefix_match';
logger.info(
`CocoaPods cache restored to ${podsDirectory} (${hitType === 'direct_hit' ? 'direct hit' : 'prefix match'})`
);
Datadog.log(`CocoaPods cache restored (${hitType})`, {
event: 'cocoapods_cache_restored',
cache_hit_type: hitType,
});
} catch (err: unknown) {
if (err instanceof TurtleFetchError && err.response?.status === 404) {
logger.info('No CocoaPods cache found for this key');
} else {
logger.warn('Failed to restore CocoaPods cache: ', err);
}
}
}
export async function restoreGradleCacheAsync({
logger,
workingDirectory,
@@ -15,7 +15,16 @@ import path from 'path';
import { compressCacheAsync, uploadCacheAsync } from './saveCache';
import { formatBytes } from '../../utils/artifacts';
import { generateDefaultBuildCacheKeyAsync, getCcachePath } from '../../utils/cacheKey';
import {
CcacheBuildTarget,
generateDefaultBuildCacheKeyAsync,
getCcachePath,
} from '../../utils/cacheKey';
import {
compressCocoapodsCacheAsync,
getCocoapodsCachePaths,
resolveCocoapodsCacheKeyAsync,
} from '../../utils/cocoapodsCache';
import { generateGradleCacheKeyAsync } from '../../utils/gradleCacheKey';
export function createSaveBuildCacheFunction(evictUsedBefore: Date): BuildFunction {
@@ -30,6 +39,11 @@ export function createSaveBuildCacheFunction(evictUsedBefore: Date): BuildFuncti
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
id: 'simulator',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.BOOLEAN,
}),
],
fn: async (stepCtx, { env, inputs }) => {
const { logger } = stepCtx;
@@ -43,10 +57,20 @@ export function createSaveBuildCacheFunction(evictUsedBefore: Date): BuildFuncti
);
}
const target: CcacheBuildTarget =
platform === Platform.IOS
? {
platform,
simulator:
(inputs.simulator.value as boolean | undefined) ??
(stepCtx.global.staticContext.job.platform === Platform.IOS &&
stepCtx.global.staticContext.job.simulator === true),
}
: { platform };
await saveCcacheAsync({
logger,
workingDirectory,
platform,
target,
evictUsedBefore,
env,
secrets: stepCtx.global.staticContext.job.secrets,
@@ -59,22 +83,88 @@ export function createSaveBuildCacheFunction(evictUsedBefore: Date): BuildFuncti
env,
secrets: stepCtx.global.staticContext.job.secrets,
});
} else {
await saveCocoapodsCacheAsync({
logger,
workingDirectory,
env,
secrets: stepCtx.global.staticContext.job.secrets,
});
}
},
});
}
export async function saveCocoapodsCacheAsync({
logger,
workingDirectory,
env,
secrets,
}: {
logger: bunyan;
workingDirectory: string;
env: Record<string, string | undefined>;
secrets?: { robotAccessToken?: string };
}): Promise<void> {
if (env.EAS_PODS_CACHE !== '1') {
return;
}
const { podsDirectory, podfileLockPath } = getCocoapodsCachePaths(workingDirectory);
try {
await Promise.all([fs.promises.access(podsDirectory), fs.promises.access(podfileLockPath)]);
} catch {
logger.warn('No CocoaPods installation found, skipping cache save');
return;
}
try {
const { stdout } = await spawnAsync('pod', ['--version'], {
env,
stdio: 'pipe',
});
const { key } = await resolveCocoapodsCacheKeyAsync(workingDirectory, stdout);
logger.info(`Saving CocoaPods cache key: ${key}`);
const jobId = nullthrows(env.EAS_BUILD_ID, 'EAS_BUILD_ID is not set');
const robotAccessToken = nullthrows(
secrets?.robotAccessToken,
'Robot access token is required for cache operations'
);
const expoApiServerURL = nullthrows(env.__API_SERVER_URL, '__API_SERVER_URL is not set');
logger.info('Compressing CocoaPods cache...');
const { archivePath } = await compressCocoapodsCacheAsync({ workingDirectory });
const { size } = await fs.promises.stat(archivePath);
logger.info(`CocoaPods cache archive size: ${formatBytes(size)}`);
await uploadCacheAsync({
logger,
jobId,
expoApiServerURL,
robotAccessToken,
archivePath,
key,
paths: [podsDirectory],
size,
platform: Platform.IOS,
});
} catch (err) {
logger.error({ err }, 'Failed to save CocoaPods cache');
}
}
export async function saveCcacheAsync({
logger,
workingDirectory,
platform,
target,
evictUsedBefore,
env,
secrets,
}: {
logger: bunyan;
workingDirectory: string;
platform: Platform;
target: CcacheBuildTarget;
evictUsedBefore: Date;
env: Record<string, string | undefined>;
secrets?: { robotAccessToken?: string };
@@ -99,7 +189,7 @@ export async function saveCcacheAsync({
}
try {
const cacheKey = await generateDefaultBuildCacheKeyAsync(workingDirectory, platform);
const cacheKey = await generateDefaultBuildCacheKeyAsync(workingDirectory, target);
logger.info(`Saving cache key: ${cacheKey}`);
const jobId = nullthrows(env.EAS_BUILD_ID, 'EAS_BUILD_ID is not set');
@@ -143,7 +233,7 @@ export async function saveCcacheAsync({
key: cacheKey,
paths: [cachePath],
size,
platform,
platform: target.platform,
});
} catch (err) {
logger.error({ err }, 'Failed to save cache');
@@ -103,6 +103,7 @@ export async function uploadCacheAsync({
archivePath,
size,
platform,
force = false,
}: {
logger: bunyan;
jobId: string;
@@ -113,6 +114,7 @@ export async function uploadCacheAsync({
archivePath: string;
size: number;
platform: Platform | undefined;
force?: boolean;
}): Promise<void> {
const routerURL = platform
? 'v2/turtle-builds/caches/upload-sessions'
@@ -127,12 +129,14 @@ export async function uploadCacheAsync({
key,
version: getCacheVersion(paths),
size,
force,
})
: JSON.stringify({
jobRunId: jobId,
key,
version: getCacheVersion(paths),
size,
force,
}),
headers: {
Authorization: `Bearer ${robotAccessToken}`,
@@ -13,6 +13,10 @@ import os from 'node:os';
import path from 'node:path';
import { type CustomBuildContext } from '../../customBuildContext';
import {
uploadRemoteSessionConfigWithLocalEgressAsync,
withLocalEgressSession,
} from '../utils/localEgressSession';
import { Sentry } from '../../sentry';
import { pollAgentDeviceArtifactsForUploadAsync } from '../utils/agentDeviceArtifacts';
import { startAgentDeviceEventCollectionAsync } from '../utils/agentDeviceEvents';
@@ -23,9 +27,8 @@ import {
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
startServeSimWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
waitForFileAsync,
} from '../utils/remoteDeviceRunSession';
@@ -66,7 +69,7 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
}),
],
fn: async ({ logger, global }, { inputs, env, signal }) => {
fn: withLocalEgressSession(async ({ logger, global }, { inputs, env, signal }) => {
// Fail fast before any expensive setup if the injected env
// vars are missing: DEVICE_RUN_SESSION_ID (to report the remote config
// back to the API server), EAS_SIMULATOR_NGROK_TUNNEL_DOMAIN (base domain
@@ -107,30 +110,30 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
const agentDeviceRemoteSessionUrl = agentDeviceTunnel.url;
logger.info(`Tunnel is ready at ${agentDeviceRemoteSessionUrl}.`);
let serveSim: Awaited<ReturnType<typeof startServeSimWithTunnelAsync>> | undefined;
let webPreview: Awaited<ReturnType<typeof startDeviceWebPreviewWithTunnelAsync>> | undefined;
let eventCollection:
| Awaited<ReturnType<typeof startAgentDeviceEventCollectionAsync>>
| undefined;
try {
// serve-sim is iOS-only — only launch it (and report a webPreviewUrl)
// on Darwin. Android sessions go without a preview URL.
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
serveSim = await startServeSimWithTunnelAsync(ctx, {
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
logger.info(`Web preview URL: ${serveSim.previewUrl}`);
}
webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
logger.info(`Web preview URL: ${webPreview.previewUrl}`);
await uploadRemoteSessionConfigAsync({
await uploadRemoteSessionConfigWithLocalEgressAsync({
env,
signal,
ctx,
deviceRunSessionId,
remoteConfig: {
agentDeviceRemoteSessionUrl,
agentDeviceRemoteSessionToken: daemonToken,
...(serveSim ? { webPreviewUrl: serveSim.previewUrl } : {}),
webPreviewUrl: webPreview.previewUrl,
...(webPreview.previewToken ? { webPreviewToken: webPreview.previewToken } : {}),
},
logger,
});
@@ -163,8 +166,8 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
: undefined,
});
} finally {
if (serveSim) {
await serveSim.stopAsync();
if (webPreview) {
await webPreview.stopAsync();
}
await agentDeviceTunnel.stopAsync();
if (eventCollection) {
@@ -176,7 +179,7 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
}
await daemonProcess.stopAsync();
}
},
}),
});
}
@@ -46,6 +46,21 @@ export function createStartAndroidEmulatorBuildFunction(): BuildFunction {
defaultValue: AndroidEmulatorUtils.defaultSystemImagePackage,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
id: 'lcd_width',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
}),
BuildStepInput.createProvider({
id: 'lcd_height',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
}),
BuildStepInput.createProvider({
id: 'lcd_density',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
}),
BuildStepInput.createProvider({
id: 'count',
required: false,
@@ -86,6 +101,9 @@ export function createStartAndroidEmulatorBuildFunction(): BuildFunction {
const systemImagePackage = `${inputs.system_image_package.value}`;
// We can cast because allowedValueTypeName validated this is a string.
const deviceIdentifier = inputs.device_identifier.value as AndroidDeviceName | undefined;
const lcdWidth = inputs.lcd_width.value as number | undefined;
const lcdHeight = inputs.lcd_height.value as number | undefined;
const lcdDensity = inputs.lcd_density.value as number | undefined;
const shouldAdjustAnimationScale =
env.ANDROID_EMULATOR_ADJUST_ANIMATION_SCALE !== 'false' &&
env.ANDROID_EMULATOR_ADJUST_ANIMATION_SCALE !== '0';
@@ -129,6 +147,9 @@ export function createStartAndroidEmulatorBuildFunction(): BuildFunction {
deviceName,
systemImagePackage,
deviceIdentifier: deviceIdentifier ?? null,
lcdWidth: lcdWidth ?? null,
lcdHeight: lcdHeight ?? null,
lcdDensity: lcdDensity ?? null,
env,
logger,
});
@@ -15,6 +15,10 @@ import semver from 'semver';
import { z } from 'zod';
import { type CustomBuildContext } from '../../customBuildContext';
import {
uploadRemoteSessionConfigWithLocalEgressAsync,
withLocalEgressSession,
} from '../utils/localEgressSession';
import { AndroidEmulatorUtils } from '../../utils/AndroidEmulatorUtils';
import { IosSimulatorUtils } from '../../utils/IosSimulatorUtils';
import { sleepAsync } from '../../utils/retry';
@@ -26,9 +30,8 @@ import {
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
startServeSimWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
@@ -62,7 +65,7 @@ export function createStartAppiumRemoteSessionBuildFunction(
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
}),
],
fn: async ({ logger, global }, { inputs, env, signal }) => {
fn: withLocalEgressSession(async ({ logger, global }, { inputs, env, signal }) => {
const deviceRunSessionId = getDeviceRunSessionIdOrThrow(env);
const ngrokTunnelDomain = getNgrokTunnelDomainOrThrow(env);
const ngrokAuthtoken = getNgrokAuthtokenOrThrow(env);
@@ -118,7 +121,7 @@ export function createStartAppiumRemoteSessionBuildFunction(
logger,
});
let appiumTunnel: Awaited<ReturnType<typeof startNgrokTunnelAsync>> | undefined;
let serveSim: Awaited<ReturnType<typeof startServeSimWithTunnelAsync>> | undefined;
let webPreview: Awaited<ReturnType<typeof startDeviceWebPreviewWithTunnelAsync>> | undefined;
try {
appiumTunnel = await startNgrokTunnelAsync({
port: APPIUM_PORT,
@@ -128,20 +131,19 @@ export function createStartAppiumRemoteSessionBuildFunction(
logger,
});
switch (runtimePlatform) {
case BuildRuntimePlatform.DARWIN:
serveSim = await startServeSimWithTunnelAsync(ctx, {
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: APPIUM_STARTUP_TIMEOUT_MS,
});
break;
case BuildRuntimePlatform.LINUX:
break;
}
// expo-device-hub has no serial-selection flag. Device run session workflows must expose
// a single booted Android emulator so the Hub and Appium resolve the same device.
webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: APPIUM_STARTUP_TIMEOUT_MS,
});
await uploadRemoteSessionConfigAsync({
await uploadRemoteSessionConfigWithLocalEgressAsync({
env,
signal,
ctx,
deviceRunSessionId,
remoteConfig: {
@@ -151,7 +153,8 @@ export function createStartAppiumRemoteSessionBuildFunction(
'appium:automationName': device.automationName,
'appium:udid': device.udid,
},
...(serveSim ? { webPreviewUrl: serveSim.previewUrl } : {}),
webPreviewUrl: webPreview.previewUrl,
...(webPreview.previewToken ? { webPreviewToken: webPreview.previewToken } : {}),
},
logger,
});
@@ -170,8 +173,8 @@ export function createStartAppiumRemoteSessionBuildFunction(
: undefined,
});
} finally {
if (serveSim) {
await serveSim.stopAsync();
if (webPreview) {
await webPreview.stopAsync();
}
if (appiumTunnel) {
await appiumTunnel.stopAsync();
@@ -180,7 +183,7 @@ export function createStartAppiumRemoteSessionBuildFunction(
await appiumProcess.stopAsync();
await fs.promises.rm(appiumHome, { recursive: true, force: true });
}
},
}),
});
}
@@ -14,21 +14,24 @@ import semver from 'semver';
import { z } from 'zod';
import { CustomBuildContext } from '../../customBuildContext';
import {
uploadRemoteSessionConfigWithLocalEgressAsync,
withLocalEgressSession,
} from '../utils/localEgressSession';
import { Sentry } from '../../sentry';
import { isProcessDescendantOfAsync } from '../../utils/processes';
import { sleepAsync } from '../../utils/retry';
import { pollArgentArtifactsForUploadAsync } from '../utils/argentArtifacts';
import { ARGENT_EVENT_LOG_FILENAME, startArgentEventCollectionAsync } from '../utils/argentEvents';
import {
ensureFfmpegInstalledAsync,
ensureFfmpegInstalledOnceAsync,
getDeviceRunSessionIdOrThrow,
getNgrokAuthtokenOrThrow,
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
startServeSimWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
@@ -79,7 +82,7 @@ export function createStartArgentRemoteSessionBuildFunction(
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
}),
],
fn: async ({ logger, global }, { inputs, env, signal }) => {
fn: withLocalEgressSession(async ({ logger, global }, { inputs, env, signal }) => {
// Fail fast before any expensive setup if the injected env
// vars are missing: DEVICE_RUN_SESSION_ID (to report the remote config
// back to the API server), EAS_SIMULATOR_NGROK_TUNNEL_DOMAIN (base domain
@@ -103,32 +106,39 @@ export function createStartArgentRemoteSessionBuildFunction(
await selectXcodeDeveloperDirectoryAsync({ env, logger });
}
// Argent shells out to ffmpeg to record the screen. Installing it pulls a
// large Homebrew dependency tree, so do not block session readiness on it:
// the session comes up at its usual speed and only a recording started in
// the first moments misses ffmpeg. Never rejects, so `void` is safe.
void ensureFfmpegInstalledAsync({ runtimePlatform, env, logger });
// Start the potentially slow installation while Argent is being prepared.
// On Linux expo-device-hub calls this again and awaits the same in-flight
// setup before launching. On macOS this remains non-blocking, so only a
// recording started in the first moments may miss ffmpeg.
// Never rejects, so `void` is safe.
void ensureFfmpegInstalledOnceAsync({ runtimePlatform, env, logger });
logger.info('Enabling the Argent artifacts list endpoint flag.');
await spawn(
'bunx',
[`${ARGENT_PACKAGE_NAME}@${versionSpec}`, 'enable', ARGENT_ARTIFACTS_LIST_ENDPOINT_FLAG],
'bun',
[
'x',
`${ARGENT_PACKAGE_NAME}@${versionSpec}`,
'enable',
ARGENT_ARTIFACTS_LIST_ENDPOINT_FLAG,
],
{ env, logger }
);
logger.info('Enabling the Argent tool-server event log flag.');
await spawn(
'bunx',
[`${ARGENT_PACKAGE_NAME}@${versionSpec}`, 'enable', ARGENT_EVENT_LOG_FLAG],
'bun',
['x', `${ARGENT_PACKAGE_NAME}@${versionSpec}`, 'enable', ARGENT_EVENT_LOG_FLAG],
{ env, logger }
);
logger.info(`Launching ${ARGENT_PACKAGE_NAME}@${versionSpec} tool-server via bunx.`);
// Keep Argent itself in foreground mode under the detached bunx process. This preserves
// the bunx -> Argent CLI -> tool-server ancestry used to identify the matching state file.
logger.info(`Launching ${ARGENT_PACKAGE_NAME}@${versionSpec} tool-server via bun x.`);
// Keep Argent itself in foreground mode under the detached bun process. This preserves
// the bun -> Argent CLI -> tool-server ancestry used to identify the matching state file.
const argentServer = spawnDetached({
command: 'bunx',
command: 'bun',
args: [
'x',
`${ARGENT_PACKAGE_NAME}@${versionSpec}`,
'server',
'start',
@@ -186,7 +196,7 @@ export function createStartArgentRemoteSessionBuildFunction(
});
let toolsTunnel: Awaited<ReturnType<typeof startNgrokTunnelAsync>> | undefined;
let serveSim: Awaited<ReturnType<typeof startServeSimWithTunnelAsync>> | undefined;
let webPreview: Awaited<ReturnType<typeof startDeviceWebPreviewWithTunnelAsync>> | undefined;
try {
toolsTunnel = await startNgrokTunnelAsync({
port: toolServerPort,
@@ -199,26 +209,25 @@ export function createStartArgentRemoteSessionBuildFunction(
const publicToolsUrl = toolsTunnel.url;
logger.info(`Tunnel is ready at ${publicToolsUrl}.`);
// serve-sim is iOS-only — Android sessions go without a preview URL.
let webPreviewUrl: string | undefined;
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
serveSim = await startServeSimWithTunnelAsync(ctx, {
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
webPreviewUrl = serveSim.previewUrl;
logger.info(`Web preview URL: ${webPreviewUrl}`);
}
webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
logger.info(`Web preview URL: ${webPreview.previewUrl}`);
await uploadRemoteSessionConfigAsync({
await uploadRemoteSessionConfigWithLocalEgressAsync({
env,
signal,
ctx,
deviceRunSessionId,
remoteConfig: {
toolsUrl: publicToolsUrl,
...(toolServerToken ? { toolsAuthToken: toolServerToken } : {}),
...(webPreviewUrl ? { webPreviewUrl } : {}),
webPreviewUrl: webPreview.previewUrl,
...(webPreview.previewToken ? { webPreviewToken: webPreview.previewToken } : {}),
},
logger,
});
@@ -238,8 +247,8 @@ export function createStartArgentRemoteSessionBuildFunction(
: undefined,
});
} finally {
if (serveSim) {
await serveSim.stopAsync();
if (webPreview) {
await webPreview.stopAsync();
}
if (toolsTunnel) {
await toolsTunnel.stopAsync();
@@ -255,7 +264,7 @@ export function createStartArgentRemoteSessionBuildFunction(
}
await argentServer.stopAsync();
}
},
}),
});
}
@@ -297,7 +306,7 @@ export function warnIfArgentPackageVersionCannotBeVerified({
logger.warn(
`Argent remote simulator sessions require ${ARGENT_PACKAGE_NAME}@${MIN_ARGENT_REMOTE_SESSION_VERSION} or newer, ` +
`but package_version "${packageVersion}" is not an exact semver version that EAS can verify. ` +
`Continuing and letting bunx resolve it.`
`Continuing and letting bun x resolve it.`
);
return;
}
@@ -4,9 +4,17 @@ import {
BuildStepInput,
BuildStepInputValueTypeName,
} from '@expo/steps';
import { type bunyan } from '@expo/logger';
import spawn from '@expo/turtle-spawn';
import { minBy } from 'lodash';
import { configureSimulatorProxyEnvironmentAsync } from '../utils/localEgress';
import {
installLocalEgressGuardAsync,
resolveLocalEgressBootEnvironmentAsync,
verifyLocalEgressGuardAsync,
} from '../utils/localEgressGuard';
import {
IosSimulatorName,
IosSimulatorUtils,
@@ -73,9 +81,10 @@ export function createStartIosSimulatorBuildFunction(): BuildFunction {
env,
});
}
const { udid } = await IosSimulatorUtils.startAsync({
const udid = await bootWithLocalEgressAsync({
deviceIdentifier: originalDeviceIdentifier,
env,
logger,
});
try {
@@ -116,9 +125,10 @@ export function createStartIosSimulatorBuildFunction(): BuildFunction {
env,
});
}
const { udid: cloneUdid } = await IosSimulatorUtils.startAsync({
const cloneUdid = await bootWithLocalEgressAsync({
deviceIdentifier: cloneDeviceName,
env,
logger,
});
try {
@@ -140,6 +150,43 @@ export function createStartIosSimulatorBuildFunction(): BuildFunction {
});
}
/**
* Boot a device with the local egress environment in place before anything
* inside it starts. `simctl boot` returns once the simulator's launchd is up
* and before it has spawned anything else, which is the only moment at which
* launchd environment reaches every process of the boot. The proxy variables
* and the guard are set in that gap; then the boot is waited for and the
* guard is verified in a fresh process. Nothing here applies when no local
* egress session is active.
*/
async function bootWithLocalEgressAsync({
deviceIdentifier,
env,
logger,
}: {
deviceIdentifier: IosSimulatorUuid | IosSimulatorName;
env: BuildStepEnv;
logger: bunyan;
}): Promise<IosSimulatorUuid> {
const udid = await IosSimulatorUtils.resolveUdidAsync({ deviceIdentifier, env });
// The guard and proxy variables ride the boot itself, so launchd has them
// before it spawns its first process.
const launchdEnvironment = await resolveLocalEgressBootEnvironmentAsync();
await IosSimulatorUtils.bootAsync({
deviceIdentifier: udid,
env,
launchdEnvironment: launchdEnvironment ?? {},
});
// Also set them through launchctl, for a device that was already booted.
const guardInstalled = await installLocalEgressGuardAsync({ udid, env, logger });
await configureSimulatorProxyEnvironmentAsync({ udid, env, logger });
await IosSimulatorUtils.startAsync({ deviceIdentifier: udid, env });
if (guardInstalled) {
await verifyLocalEgressGuardAsync({ udid, env, logger });
}
return udid;
}
async function findMostGenericIphoneUuidAsync({
env,
}: {
@@ -0,0 +1,203 @@
import { type bunyan } from '@expo/logger';
import { BuildFunction, BuildRuntimePlatform } from '@expo/steps';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import {
CHISEL_VERSION,
LOCAL_EGRESS_HANDOFF_PATH,
LOCAL_EGRESS_PROXY_PORT,
configureSystemProxyAsync,
createChiselAuthfileContents,
downloadChiselAsync,
generateEgressCredentials,
registerLocalEgressResources,
startChiselServerAsync,
stopLocalEgressResourcesAsync,
writeLocalEgressHandoffAsync,
} from '../utils/localEgress';
import { stopLocalEgressGuardRelaysAsync } from '../utils/localEgressGuard';
import {
type DetachedProcessHandle,
type NgrokTunnelHandle,
findAvailablePortAsync,
getNgrokAuthtokenOrThrow,
getNgrokTunnelDomainOrThrow,
startNgrokTunnelAsync,
} from '../utils/remoteDeviceRunSession';
/**
* The downloader and ngrok SDK do not accept an AbortSignal. Stop awaiting them
* on cancellation, while retaining handlers that dispose any eventual result.
*/
function awaitLocalEgressAcquisitionAsync<T>(
pending: Promise<T>,
signal: AbortSignal,
disposeAbandonedAsync: (result?: T) => Promise<void>,
logger: bunyan
): Promise<T> {
return new Promise<T>((resolve, reject) => {
let abandoned = false;
const abort = (): void => {
signal.removeEventListener('abort', abort);
abandoned = true;
reject(signal.reason);
};
signal.addEventListener('abort', abort, { once: true });
if (signal.aborted) {
abort();
}
void pending
.then(
async result => {
signal.removeEventListener('abort', abort);
if (abandoned) {
await disposeAbandonedAsync(result);
} else {
resolve(result);
}
},
async error => {
signal.removeEventListener('abort', abort);
if (abandoned) {
await disposeAbandonedAsync();
} else {
reject(error);
}
}
)
.catch(err =>
logger.warn({ err }, 'Could not release a cancelled local egress acquisition.')
);
});
}
/**
* Points the device host's system HTTP(S) proxy at the EAS CLI's egress client.
* Must run before `eas/start_ios_simulator`: the simulator reads the system proxy
* at boot. Once a simulator boots, `eas/start_ios_simulator` also sets proxy
* environment variables inside it for clients that read them (gRPC, libcurl)
* and installs the local egress guard, which refuses connections that ignore
* both. The shared session cleanup releases the resources started here when
* the session ends, with a job finalizer as a fallback.
*/
export function createStartLocalEgressBuildFunction(): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'start_local_egress',
name: 'Start local egress',
__metricsId: 'eas/start_local_egress',
supportedRuntimePlatforms: [BuildRuntimePlatform.DARWIN],
fn: async ({ logger }, { env, signal }) => {
const ngrokTunnelDomain = getNgrokTunnelDomainOrThrow(env);
const ngrokAuthtoken = getNgrokAuthtokenOrThrow(env);
let workDir: string | undefined;
let server: DetachedProcessHandle | undefined;
let tunnel: NgrokTunnelHandle | undefined;
let finishSetup!: () => void;
const setupFinished = new Promise<void>(resolve => {
finishSetup = resolve;
});
// Register before acquisition. Cancellation ends setup promptly; pending
// SDK calls retain their own late-result disposal handlers.
const lifetimeSignal = registerLocalEgressResources(async () => {
await setupFinished;
const results = await Promise.allSettled([
Promise.resolve().then(() => stopLocalEgressGuardRelaysAsync(logger)),
Promise.resolve().then(() => tunnel?.stopAsync()),
Promise.resolve().then(() => server?.stopAsync()),
]);
for (const result of results) {
if (result.status === 'rejected') {
logger.warn({ err: result.reason }, 'Could not stop a local egress resource.');
}
}
await Promise.all([
fs.promises.rm(LOCAL_EGRESS_HANDOFF_PATH, { force: true }),
workDir ? fs.promises.rm(workDir, { recursive: true, force: true }) : undefined,
]);
});
const startupSignal = signal ? AbortSignal.any([signal, lifetimeSignal]) : lifetimeSignal;
try {
startupSignal.throwIfAborted();
workDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'eas-local-egress-'));
startupSignal.throwIfAborted();
logger.info(`Downloading the reverse tunnel server (chisel ${CHISEL_VERSION}).`);
const downloadDir = workDir;
const chiselPath = await awaitLocalEgressAcquisitionAsync(
downloadChiselAsync({ destinationDir: downloadDir, logger }),
startupSignal,
async () => await fs.promises.rm(downloadDir, { recursive: true, force: true }),
logger
);
startupSignal.throwIfAborted();
const credentials = generateEgressCredentials();
const authfilePath = path.join(workDir, 'authfile.json');
await fs.promises.writeFile(
authfilePath,
createChiselAuthfileContents({ ...credentials, port: LOCAL_EGRESS_PROXY_PORT }),
{ encoding: 'utf8', mode: 0o600 }
);
startupSignal.throwIfAborted();
const controlPort = await findAvailablePortAsync();
startupSignal.throwIfAborted();
logger.info(`Starting the reverse tunnel server on 127.0.0.1:${controlPort}.`);
const started = await startChiselServerAsync({
chiselPath,
controlPort,
authfilePath,
env,
signal: startupSignal,
});
server = started.process;
startupSignal.throwIfAborted();
tunnel = await awaitLocalEgressAcquisitionAsync(
startNgrokTunnelAsync({
port: controlPort,
subdomainPrefix: 'egress',
baseDomain: ngrokTunnelDomain,
authtoken: ngrokAuthtoken,
logger,
}),
startupSignal,
async listener => await listener?.stopAsync(),
logger
);
startupSignal.throwIfAborted();
const { service } = await configureSystemProxyAsync({
env,
logger,
port: LOCAL_EGRESS_PROXY_PORT,
signal: startupSignal,
});
startupSignal.throwIfAborted();
await writeLocalEgressHandoffAsync({
url: tunnel.url,
token: credentials.password,
fingerprint: started.fingerprint,
port: LOCAL_EGRESS_PROXY_PORT,
});
startupSignal.throwIfAborted();
logger.info(
`Local egress is configured on network service "${service}". HTTP(S) and WebSocket ` +
'requests that honor the system proxy (WebKit, URLSession and other CFNetwork clients) ' +
'fail until the EAS CLI egress client connects, then exit from that machine. Once the ' +
'Simulator boots, proxy environment variables are set inside it for clients that read ' +
'them (gRPC, libcurl), and the local egress guard is installed so that connections which ' +
'ignore both are refused in the process that makes them and reported here.'
);
} catch (error) {
finishSetup();
await stopLocalEgressResourcesAsync(logger);
throw error;
} finally {
finishSetup();
}
},
});
}
@@ -0,0 +1,105 @@
import { SystemError } from '@expo/eas-build-job';
import { BuildFunction, BuildStepInput, BuildStepInputValueTypeName } from '@expo/steps';
import { graphql } from 'gql.tada';
import { CustomBuildContext } from '../../customBuildContext';
import { withLogPhaseAsync } from '../../utils/logPhase';
import { startSandboxDaemonAsync } from '../utils/sandboxDaemon';
const MARK_SANDBOX_READY_MUTATION = graphql(`
mutation MarkSandboxReady($sandboxId: ID!) {
sandbox {
markSandboxReady(sandboxId: $sandboxId) {
id
}
}
}
`);
const RECONNECT_DELAY_MS = 1_000;
const AWAIT_SANDBOX_STEP_NAME = 'Await sandbox completion';
export function createStartSandboxBuildFunction(ctx: CustomBuildContext): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'start_sandbox',
name: 'Start sandbox daemon',
__metricsId: 'eas/start_sandbox',
inputProviders: [
BuildStepInput.createProvider({
id: 'sandbox_id',
required: true,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
],
fn: async (stepCtx, { inputs, signal }) => {
const sandboxToken = ctx.env.__EAS_SANDBOX_MCP_TOKEN;
if (!sandboxToken) {
throw new SystemError('__EAS_SANDBOX_MCP_TOKEN is required to start the sandbox daemon.');
}
const mcpServerUrl = ctx.mcpServerUrl;
if (!mcpServerUrl) {
throw new SystemError('MCP server URL is required to start the sandbox daemon.');
}
const sandboxId = String(inputs.sandbox_id.value);
const daemon = await startSandboxDaemonAsync({
credential: sandboxToken,
serverUrl: mcpServerUrl,
reconnectDelayMs: RECONNECT_DELAY_MS,
logger: stepCtx.logger,
signal,
});
try {
await daemon.ready;
signal?.throwIfAborted();
await markSandboxReadyAsync(ctx, sandboxId, signal);
signal?.throwIfAborted();
stepCtx.logger.info('Sandbox daemon started.');
await withLogPhaseAsync(stepCtx.logger, AWAIT_SANDBOX_STEP_NAME, async () => {
await waitForCancellationAsync(signal);
});
} finally {
await daemon.stopAsync();
}
},
});
}
export async function markSandboxReadyAsync(
ctx: CustomBuildContext,
sandboxId: string,
signal?: AbortSignal
): Promise<void> {
signal?.throwIfAborted();
const result = await ctx.graphqlClient
.mutation(
MARK_SANDBOX_READY_MUTATION,
{ sandboxId },
signal
? {
fetch: (input, init) =>
fetch(input, {
...init,
signal: init?.signal ? AbortSignal.any([signal, init.signal]) : signal,
}),
}
: undefined
)
.toPromise();
signal?.throwIfAborted();
if (result.error) {
throw new SystemError(`Failed to mark sandbox ${sandboxId} as ready.`, {
cause: result.error,
});
}
}
async function waitForCancellationAsync(signal?: AbortSignal): Promise<void> {
if (signal?.aborted) {
return;
}
await new Promise<void>(resolve => {
signal?.addEventListener('abort', () => resolve(), { once: true });
});
}
@@ -6,55 +6,73 @@ import {
} from '@expo/steps';
import { CustomBuildContext } from '../../customBuildContext';
import {
uploadRemoteSessionConfigWithLocalEgressAsync,
withLocalEgressSession,
} from '../utils/localEgressSession';
import {
getDeviceRunSessionIdOrThrow,
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
startServeSimWithTunnelAsync,
uploadRemoteSessionConfigAsync,
startDeviceWebPreviewWithTunnelAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
const STARTUP_TIMEOUT_MS = 60_000;
export function createStartServeSimRemoteSessionBuildFunction(
export function createStartWebPreviewRemoteSessionBuildFunction(
ctx: CustomBuildContext
): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'start_serve_sim_remote_session',
name: 'Start serve-sim remote session',
name: 'Start web preview remote session',
__metricsId: 'eas/start_serve_sim_remote_session',
supportedRuntimePlatforms: [BuildRuntimePlatform.DARWIN],
inputProviders: [
BuildStepInput.createProvider({
id: 'package_version',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
id: 'max_duration_seconds',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
}),
],
fn: async ({ logger }, { inputs, env, signal }) => {
fn: withLocalEgressSession(async ({ logger, global }, { inputs, env, signal }) => {
const deviceRunSessionId = getDeviceRunSessionIdOrThrow(env);
const ngrokTunnelDomain = getNgrokTunnelDomainOrThrow(env);
const maxDurationSeconds = inputs.max_duration_seconds?.value as number | undefined;
const packageVersion = inputs.package_version?.value as string | undefined;
const { runtimePlatform } = global;
logger.info('Starting serve-sim remote session.');
logger.info(`Starting web preview remote session (runtime: ${runtimePlatform}).`);
await selectXcodeDeveloperDirectoryAsync({ env, logger });
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
await selectXcodeDeveloperDirectoryAsync({ env, logger });
}
const serveSim = await startServeSimWithTunnelAsync(ctx, {
const webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
packageVersion,
});
logger.info(`Preview URL: ${serveSim.previewUrl}`);
logger.info(`Preview URL: ${webPreview.previewUrl}`);
try {
await uploadRemoteSessionConfigAsync({
await uploadRemoteSessionConfigWithLocalEgressAsync({
env,
signal,
ctx,
deviceRunSessionId,
remoteConfig: { previewUrl: serveSim.previewUrl },
remoteConfig: {
previewUrl: webPreview.previewUrl,
...(webPreview.previewToken ? { previewToken: webPreview.previewToken } : {}),
},
logger,
});
@@ -66,8 +84,8 @@ export function createStartServeSimRemoteSessionBuildFunction(
signal,
});
} finally {
await serveSim.stopAsync();
await webPreview.stopAsync();
}
},
}),
});
}
@@ -90,19 +90,24 @@ export function createUploadToAscBuildFunction(): BuildFunction {
const ascApiKeyJson = await fs.readJson(ascApiKeyPath);
const ascApiKey = z
.object({
issuer_id: z.string(),
issuer_id: z.string().nullish(),
key_id: z.string(),
key: z.string(),
})
.parse(ascApiKeyJson);
const privateKey = await jose.importPKCS8(ascApiKey.key, 'ES256');
const token = await new jose.SignJWT({})
const jwt = new jose.SignJWT({})
.setProtectedHeader({ alg: 'ES256', kid: ascApiKey.key_id })
.setIssuer(ascApiKey.issuer_id)
.setAudience('appstoreconnect-v1')
.setExpirationTime('20m')
.sign(privateKey);
.setExpirationTime('20m');
if (ascApiKey.issuer_id) {
jwt.setIssuer(ascApiKey.issuer_id);
} else {
// Nullish issuer_id means an individual API key
jwt.setSubject('user');
}
const token = await jwt.sign(privateKey);
const client = new AscApiClient({ token, logger: stepsCtx.logger });
@@ -115,16 +120,27 @@ export function createUploadToAscBuildFunction(): BuildFunction {
`Uploading Build to "${appResponse.data.attributes.name}" (${ascAppBundleIdentifier})...`
);
// Derive the App Store Connect platform from the IPA itself so tvOS (and
// other) binaries are uploaded to the correct version train instead of the
// iOS one. Falls back to `IOS` if the platform cannot be read.
const ipaInfoResult = await asyncResult(readIpaInfoAsync(ipaPath));
const platform = ipaInfoResult.ok
? AscApiUtils.ascPlatformFromDtPlatformName(ipaInfoResult.value.dtPlatformName)
: 'IOS';
stepsCtx.logger.info(`Detected App Store Connect platform: ${platform}`);
stepsCtx.logger.info('Creating Build Upload...');
const buildUploadResponse = await AscApiUtils.createBuildUploadAsync({
client,
appleAppIdentifier,
bundleShortVersion,
bundleVersion,
platform,
});
const buildUploadId = buildUploadResponse.data.id;
const buildUploadUrl = `https://appstoreconnect.apple.com/apps/${appleAppIdentifier}/testflight/ios/${buildUploadId}`;
const platformPathSegment = AscApiUtils.testFlightPlatformPathSegment(platform);
const buildUploadUrl = `https://appstoreconnect.apple.com/apps/${appleAppIdentifier}/testflight/${platformPathSegment}/${buildUploadId}`;
outputs.build_upload_id.set(buildUploadId);
outputs.build_upload_url.set(buildUploadUrl);
@@ -0,0 +1,38 @@
import { APPIUM_COMMAND_SUMMARIES, humanizeAppiumCommand } from '../appiumCommandSummary';
// Coverage of the generated Appium and driver command sets is enforced at compile time by the
// `Record<AppiumCommand, string>` type in appiumCommandSummary.ts, so it is not
// re-checked here.
describe('APPIUM_COMMAND_SUMMARIES', () => {
it('has a non-empty summary that differs from the command name for every entry', () => {
for (const [command, summary] of Object.entries(APPIUM_COMMAND_SUMMARIES)) {
expect(summary).toBeTruthy();
expect(summary).not.toBe(command);
}
});
});
describe(humanizeAppiumCommand, () => {
it.each([
['getWindowRect', 'Read the screen size'],
['getScreenshot', 'Took a screenshot'],
['getPageSource', 'Read the screen contents'],
['click', 'Tapped an element'],
['setValue', 'Typed into an element'],
['createSession', 'Started the session'],
['deleteSession', 'Ended the session'],
['postAcceptAlert', 'Accepted an alert'],
['mobileScroll', 'Scrolled the screen'],
['openNotifications', 'Opened notifications'],
])('maps the known command %s to a curated summary', (command, expected) => {
expect(humanizeAppiumCommand(command)).toBe(expected);
});
it.each(['proxyReqRes', 'mobile: scroll', 'getDeviceInfo', 'toString', 'constructor', ''])(
'returns the raw command %p when it has no curated summary',
command => {
expect(humanizeAppiumCommand(command)).toBe(command);
}
);
});
@@ -58,7 +58,10 @@ describe(startAppiumEventCollectionAsync, () => {
producer: 'appium',
type: 'operation.completed',
durationMs: 25,
summary: 'getOrientation',
summary: 'Read the screen orientation',
data: {
command: 'getOrientation',
},
},
});
@@ -67,10 +67,17 @@ describe(listArgentArtifactsAsync, () => {
artifacts: [
{
id: 'artifact-id',
kind: 'native-profile-report',
filename: 'report.json',
mimeType: 'application/json',
isDirectory: false,
},
{
id: 'legacy-artifact-id',
filename: 'screen.png',
mimeType: 'image/png',
isDirectory: false,
},
],
})
)
@@ -84,10 +91,17 @@ describe(listArgentArtifactsAsync, () => {
expect(artifacts).toEqual([
{
id: 'artifact-id',
kind: 'native-profile-report',
filename: 'report.json',
mimeType: 'application/json',
isDirectory: false,
},
{
id: 'legacy-artifact-id',
filename: 'screen.png',
mimeType: 'image/png',
isDirectory: false,
},
]);
expect(jest.mocked(fetch)).toHaveBeenCalledWith('http://127.0.0.1:1234/artifacts', {
headers: { Authorization: 'Bearer tools-token' },
@@ -143,6 +157,7 @@ describe(uploadArgentArtifactAsync, () => {
async function uploadAsync(artifact: {
id: string;
kind?: string;
filename: string;
mimeType: string;
isDirectory?: boolean;
@@ -185,7 +200,42 @@ describe(uploadArgentArtifactAsync, () => {
);
});
it('leaves directory artifacts unclassified because they are uploaded as a tarball', async () => {
it('uses the semantic kind reported by Argent', async () => {
const ctx = await uploadAsync({
id: 'artifact-id',
kind: 'native-profile-report',
filename: 'native-profile-report.md',
mimeType: 'text/markdown',
});
expect(jest.mocked(uploadDeviceRunSessionArtifactAsync)).toHaveBeenCalledWith(
ctx,
expect.objectContaining({
filename: 'native-profile-report.md',
kind: 'native-profile-report',
})
);
});
it('keeps the semantic kind when a directory is uploaded as a tarball', async () => {
const ctx = await uploadAsync({
id: 'artifact-id',
kind: 'native-profile-trace',
filename: 'native-profile.trace',
mimeType: 'application/octet-stream',
isDirectory: true,
});
expect(jest.mocked(uploadDeviceRunSessionArtifactAsync)).toHaveBeenCalledWith(
ctx,
expect.objectContaining({
filename: 'native-profile.trace.tar.gz',
kind: 'native-profile-trace',
})
);
});
it('leaves an older unclassified directory artifact without a kind', async () => {
const ctx = await uploadAsync({
id: 'artifact-id',
filename: 'screenshots',
@@ -0,0 +1,548 @@
import { type bunyan } from '@expo/logger';
import spawn from '@expo/turtle-spawn';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { spawnDetached } from '../remoteDeviceRunSession';
import {
CHISEL_VERSION,
LOCAL_EGRESS_NO_PROXY,
LOCAL_EGRESS_PROXY_PORT,
buildEgressRemoteConfigFields,
buildLocalEgressSimulatorEnvironment,
buildNetworksetupProxyArgs,
collectSimulatorProcessIds,
configureSimulatorProxyEnvironmentAsync,
createChiselAuthfileContents,
getChiselAssetName,
getChiselDownloadUrl,
parseChiselFingerprint,
parseDefaultRouteInterface,
parseDirectSimulatorConnections,
parseExitIpResponse,
parseNetworkServiceNameForDevice,
readLocalEgressHandoffAsync,
registerLocalEgressResources,
startChiselServerAsync,
stopLocalEgressResourcesAsync,
writeLocalEgressHandoffAsync,
} from '../localEgress';
jest.mock('@ngrok/ngrok');
jest.mock('@expo/turtle-spawn');
jest.mock('../remoteDeviceRunSession', () => ({ spawnDetached: jest.fn() }));
describe(getChiselAssetName, () => {
it.each([
['darwin', 'arm64', `chisel_${CHISEL_VERSION}_darwin_arm64.gz`],
['darwin', 'x64', `chisel_${CHISEL_VERSION}_darwin_amd64.gz`],
['linux', 'x64', `chisel_${CHISEL_VERSION}_linux_amd64.gz`],
['linux', 'arm64', `chisel_${CHISEL_VERSION}_linux_arm64.gz`],
] as const)('maps %s/%s to %s', (platform, arch, expected) => {
expect(getChiselAssetName({ platform, arch })).toBe(expected);
});
it('rejects platforms without a pinned binary', () => {
expect(() => getChiselAssetName({ platform: 'win32', arch: 'x64' })).toThrow(
'Local egress is not supported on win32/x64'
);
expect(() => getChiselAssetName({ platform: 'linux', arch: 'ia32' })).toThrow(
'Local egress is not supported on linux/ia32'
);
});
it('downloads from the pinned GitHub release', () => {
expect(getChiselDownloadUrl(`chisel_${CHISEL_VERSION}_darwin_arm64.gz`)).toBe(
`https://github.com/jpillora/chisel/releases/download/v${CHISEL_VERSION}/chisel_${CHISEL_VERSION}_darwin_arm64.gz`
);
});
});
describe(createChiselAuthfileContents, () => {
it('allows reverse remotes on the loopback proxy port and unprivileged loopback ports only', () => {
const contents = createChiselAuthfileContents({ user: 'eas', password: 'pw', port: 8899 });
const parsed = JSON.parse(contents) as Record<string, string[]>;
expect(parsed['eas:pw']).toHaveLength(2);
expect(parsed['eas:pw'][0]).toBe('^R:127\\.0\\.0\\.1:8899$');
const patterns = parsed['eas:pw'].map(source => new RegExp(source));
const allowed = (remote: string): boolean => patterns.some(pattern => pattern.test(remote));
expect(allowed('R:127.0.0.1:8899')).toBe(true);
expect(allowed('R:127.0.0.1:1024')).toBe(true);
expect(allowed('R:127.0.0.1:3000')).toBe(true);
expect(allowed('R:127.0.0.1:8081')).toBe(true);
expect(allowed('R:127.0.0.1:65535')).toBe(true);
// Privileged and out-of-range ports, other interfaces, and forward remotes stay denied.
expect(allowed('R:127.0.0.1:1023')).toBe(false);
expect(allowed('R:127.0.0.1:80')).toBe(false);
expect(allowed('R:127.0.0.1:0')).toBe(false);
expect(allowed('R:127.0.0.1:65536')).toBe(false);
expect(allowed('R:127.0.0.1:88990')).toBe(false);
expect(allowed('R:0.0.0.0:8899')).toBe(false);
expect(allowed('R:0.0.0.0:3000')).toBe(false);
expect(allowed('R:[::1]:3000')).toBe(false);
expect(allowed('R:127x0x0x1:8899')).toBe(false);
expect(allowed('R:socks')).toBe(false);
expect(allowed('example.com:443')).toBe(false);
expect(allowed('127.0.0.1:3000')).toBe(false);
});
});
describe(parseChiselFingerprint, () => {
it('extracts the fingerprint chisel prints at startup', () => {
const output =
'2026/09/07 22:00:00 server: Fingerprint eZm5d2ZQ+Zb2aS0lJm7A5GkWm9G9ZqnPz2W0i2Y6Q1c=\n' +
'2026/09/07 22:00:00 server: Reverse tunnelling enabled\n' +
'2026/09/07 22:00:00 server: Listening on http://127.0.0.1:52001\n';
expect(parseChiselFingerprint(output)).toBe('eZm5d2ZQ+Zb2aS0lJm7A5GkWm9G9ZqnPz2W0i2Y6Q1c=');
});
it('returns null before the fingerprint line appears', () => {
expect(parseChiselFingerprint('')).toBeNull();
expect(parseChiselFingerprint('2026/09/07 22:00:00 server: starting')).toBeNull();
});
});
describe(parseDefaultRouteInterface, () => {
it('reads the interface from `route -n get default`', () => {
const output =
' route to: default\n' +
'destination: default\n' +
' mask: default\n' +
' gateway: 172.20.10.1\n' +
' interface: en0\n' +
' flags: <UP,GATEWAY,DONE,STATIC,PRCLONING,GLOBAL>\n';
expect(parseDefaultRouteInterface(output)).toBe('en0');
});
it('returns null when there is no default route', () => {
expect(
parseDefaultRouteInterface('route: writing to routing socket: not in table\n')
).toBeNull();
});
});
describe(parseNetworkServiceNameForDevice, () => {
const listOutput =
'An asterisk (*) denotes that a network service is disabled.\n' +
'(1) Thunderbolt Ethernet Slot 0\n' +
'(Hardware Port: Thunderbolt Ethernet Slot 0, Device: en8)\n' +
'\n' +
'(2) Thunderbolt Bridge\n' +
'(Hardware Port: Thunderbolt Bridge, Device: bridge0)\n' +
'\n' +
'(3) Wi-Fi\n' +
'(Hardware Port: Wi-Fi, Device: en0)\n' +
'\n' +
'(4) iPhone USB\n' +
'(Hardware Port: iPhone USB, Device: en9)\n';
it('returns the service whose device matches', () => {
expect(parseNetworkServiceNameForDevice(listOutput, 'en0')).toBe('Wi-Fi');
expect(parseNetworkServiceNameForDevice(listOutput, 'en8')).toBe('Thunderbolt Ethernet Slot 0');
});
it('returns null for an unknown device', () => {
expect(parseNetworkServiceNameForDevice(listOutput, 'utun3')).toBeNull();
});
});
describe(buildNetworksetupProxyArgs, () => {
it('sets both the HTTP and HTTPS proxy to the loopback egress port', () => {
expect(
buildNetworksetupProxyArgs({ service: 'Ethernet', host: '127.0.0.1', port: 8899 })
).toEqual([
['-setwebproxy', 'Ethernet', '127.0.0.1', '8899'],
['-setsecurewebproxy', 'Ethernet', '127.0.0.1', '8899'],
]);
});
});
describe(collectSimulatorProcessIds, () => {
const runtimeRoot =
'/Library/Developer/CoreSimulator/Volumes/iOS_23F77/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 26.5.simruntime/Contents/Resources/RuntimeRoot';
const psOutput =
' 1 0 /sbin/launchd\n' +
' 501 1 /usr/local/bin/node\n' +
` 600 1 ${runtimeRoot}/sbin/launchd_sim\n` +
' 610 600 /Users/expo/Library/Developer/CoreSimulator/Devices/ABC/data/Containers/Bundle/Application/DEF/App.app/App\n' +
` 611 600 ${runtimeRoot}/System/Library/ExtensionKit/Extensions/NetworkingExtension.appex/com.apple.WebKit.Networking\n` +
' 620 610 /usr/bin/helper\n' +
' 700 1 /usr/sbin/lsof\n';
it('returns every descendant of launchd_sim and nothing else', () => {
expect(collectSimulatorProcessIds(psOutput).sort((a, b) => a - b)).toEqual([610, 611, 620]);
});
it('returns nothing when no simulator is running', () => {
expect(
collectSimulatorProcessIds(' 1 0 /sbin/launchd\n 501 1 /usr/local/bin/node\n')
).toEqual([]);
});
});
describe(parseDirectSimulatorConnections, () => {
const lsofOutput = [
'p610',
'cApp',
'f10',
'PTCP',
'n127.0.0.1:52344->127.0.0.1:8899',
'TST=ESTABLISHED',
'f11',
'PTCP',
'n192.168.64.2:52345->93.184.216.34:443',
'TST=ESTABLISHED',
'f12',
'PTCP',
'n192.168.64.2:52346->104.16.0.1:443',
'TST=SYN_SENT',
'f13',
'PTCP',
'n*:8080',
'TST=LISTEN',
'f14',
'PUDP',
'n*:5353',
'f15',
'PUDP',
'n192.168.64.2:60000->1.1.1.1:443',
'f16',
'PTCP',
'n[::1]:52347->[::1]:8899',
'TST=ESTABLISHED',
'f17',
'PTCP',
'n[fd00::2]:52348->[2606:4700:4700::1111]:443',
'TST=ESTABLISHED',
'p700',
'cnode',
'f20',
'PTCP',
'n192.168.64.2:52349->93.184.216.34:443',
'TST=ESTABLISHED',
'',
].join('\n');
it('reports simulator connections to non-loopback peers only', () => {
expect(parseDirectSimulatorConnections(lsofOutput, new Set([610]))).toEqual([
{ pid: 610, command: 'App', protocol: 'TCP', remote: '93.184.216.34:443' },
{ pid: 610, command: 'App', protocol: 'TCP', remote: '104.16.0.1:443' },
{ pid: 610, command: 'App', protocol: 'UDP', remote: '1.1.1.1:443' },
{ pid: 610, command: 'App', protocol: 'TCP', remote: '[2606:4700:4700::1111]:443' },
]);
});
it('ignores processes outside the simulator', () => {
expect(parseDirectSimulatorConnections(lsofOutput, new Set([700]))).toEqual([
{ pid: 700, command: 'node', protocol: 'TCP', remote: '93.184.216.34:443' },
]);
expect(parseDirectSimulatorConnections(lsofOutput, new Set([999]))).toEqual([]);
});
it('returns nothing for an empty listing', () => {
expect(parseDirectSimulatorConnections('', new Set([610]))).toEqual([]);
});
});
describe(buildEgressRemoteConfigFields, () => {
it('returns nothing without a handoff', () => {
expect(buildEgressRemoteConfigFields(null)).toEqual({});
});
it('maps the handoff to the remoteConfig field names', () => {
expect(
buildEgressRemoteConfigFields({
url: 'https://egress-abc.eas-simulator.ngrok.dev',
token: 'pw',
fingerprint: 'fp=',
port: LOCAL_EGRESS_PROXY_PORT,
})
).toEqual({
egressUrl: 'https://egress-abc.eas-simulator.ngrok.dev',
egressToken: 'pw',
egressFingerprint: 'fp=',
egressPort: LOCAL_EGRESS_PROXY_PORT,
});
});
});
describe(parseExitIpResponse, () => {
it('reads the ip field', () => {
expect(parseExitIpResponse('{"ip":"64.114.211.44"}')).toBe('64.114.211.44');
});
it('rejects responses without an ip', () => {
expect(() => parseExitIpResponse('{"error":"rate limited"}')).toThrow(
'Unexpected exit IP response'
);
});
});
describe('local egress handoff', () => {
let tempDir: string;
beforeEach(async () => {
tempDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'local-egress-test-'));
});
afterEach(async () => {
await fs.promises.rm(tempDir, { recursive: true, force: true });
});
it('round-trips through the handoff file', async () => {
const handoffPath = path.join(tempDir, 'handoff.json');
const handoff = {
url: 'https://egress-abc.eas-simulator.ngrok.dev',
token: 'pw',
fingerprint: 'fp=',
port: 8899,
};
await writeLocalEgressHandoffAsync(handoff, handoffPath);
await expect(readLocalEgressHandoffAsync(handoffPath)).resolves.toEqual(handoff);
});
it('returns null when no egress step ran', async () => {
await expect(
readLocalEgressHandoffAsync(path.join(tempDir, 'missing.json'))
).resolves.toBeNull();
});
it('rejects a malformed handoff', async () => {
const handoffPath = path.join(tempDir, 'handoff.json');
await fs.promises.writeFile(handoffPath, JSON.stringify({ url: 'x' }), 'utf8');
await expect(readLocalEgressHandoffAsync(handoffPath)).rejects.toThrow('malformed');
});
});
describe(buildLocalEgressSimulatorEnvironment, () => {
it('points every proxy variable at the loopback port and excludes loopback', () => {
expect(buildLocalEgressSimulatorEnvironment(8899)).toEqual({
http_proxy: 'http://127.0.0.1:8899',
https_proxy: 'http://127.0.0.1:8899',
HTTP_PROXY: 'http://127.0.0.1:8899',
HTTPS_PROXY: 'http://127.0.0.1:8899',
grpc_proxy: 'http://127.0.0.1:8899',
no_proxy: LOCAL_EGRESS_NO_PROXY,
NO_PROXY: LOCAL_EGRESS_NO_PROXY,
});
});
});
describe(configureSimulatorProxyEnvironmentAsync, () => {
const mockedSpawn = jest.mocked(spawn);
let tempDir: string;
let logger: bunyan;
beforeEach(async () => {
mockedSpawn.mockReset();
mockedSpawn.mockResolvedValue({ stdout: '', stderr: '' } as any);
tempDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'local-egress-env-test-'));
logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
});
afterEach(async () => {
await fs.promises.rm(tempDir, { recursive: true, force: true });
});
it('does nothing when no local egress session is active', async () => {
await expect(
configureSimulatorProxyEnvironmentAsync({
udid: 'test-udid' as any,
env: process.env,
logger,
handoffPath: path.join(tempDir, 'missing.json'),
})
).resolves.toBe(false);
expect(mockedSpawn).not.toHaveBeenCalled();
expect(logger.warn).not.toHaveBeenCalled();
});
it('sets the proxy environment in the simulator when a handoff exists', async () => {
const handoffPath = path.join(tempDir, 'handoff.json');
await writeLocalEgressHandoffAsync(
{ url: 'https://egress.example', token: 'pw', fingerprint: 'fp=', port: 8899 },
handoffPath
);
await expect(
configureSimulatorProxyEnvironmentAsync({
udid: 'test-udid' as any,
env: process.env,
logger,
handoffPath,
})
).resolves.toBe(true);
expect(mockedSpawn.mock.calls.map(([, args]) => args)).toEqual([
[
'simctl',
'spawn',
'test-udid',
'launchctl',
'setenv',
...Object.entries(buildLocalEgressSimulatorEnvironment(8899)).flat(),
],
]);
expect(logger.info).toHaveBeenCalledWith(
expect.stringContaining('proxy environment variables set')
);
});
it('warns and continues when launchctl fails', async () => {
const handoffPath = path.join(tempDir, 'handoff.json');
await writeLocalEgressHandoffAsync(
{ url: 'https://egress.example', token: 'pw', fingerprint: 'fp=', port: 8899 },
handoffPath
);
mockedSpawn.mockRejectedValue(new Error('launchctl failed'));
await expect(
configureSimulatorProxyEnvironmentAsync({
udid: 'test-udid' as any,
env: process.env,
logger,
handoffPath,
})
).resolves.toBe(false);
expect(logger.warn).toHaveBeenCalledWith(
{ err: expect.any(Error) },
expect.stringContaining('will bypass local egress and exit from this worker')
);
});
it('warns and continues when the handoff is malformed', async () => {
const handoffPath = path.join(tempDir, 'handoff.json');
await fs.promises.writeFile(handoffPath, JSON.stringify({ url: 'x' }), 'utf8');
await expect(
configureSimulatorProxyEnvironmentAsync({
udid: 'test-udid' as any,
env: process.env,
logger,
handoffPath,
})
).resolves.toBe(false);
expect(mockedSpawn).not.toHaveBeenCalled();
expect(logger.warn).toHaveBeenCalledWith(
{ err: expect.any(Error) },
expect.stringContaining('could not read the local egress handoff')
);
});
});
describe(startChiselServerAsync, () => {
const options = {
chiselPath: '/tmp/chisel',
controlPort: 52001,
authfilePath: '/tmp/authfile.json',
env: { PATH: '/bin', AUTH: 'unexpected:password' },
};
let output: string;
let stopAsync: jest.Mock;
beforeEach(() => {
jest.useFakeTimers();
output = 'server: Fingerprint pinned-key=\n';
stopAsync = jest.fn().mockResolvedValue(undefined);
jest.spyOn(process, 'kill').mockReturnValue(true);
jest.mocked(spawnDetached).mockReturnValue({
pid: 12345,
getOutput: () => output,
stopAsync,
});
});
afterEach(() => {
jest.restoreAllMocks();
jest.useRealTimers();
});
it('waits for the child to bind its exact port and clears inherited AUTH', async () => {
let resolved = false;
const started = startChiselServerAsync(options).then(result => {
resolved = true;
return result;
});
output += 'server: Listening on http://127.0.0.1:520010\n';
await jest.advanceTimersByTimeAsync(250);
expect(resolved).toBe(false);
output += 'server: Listening on http://127.0.0.1:52001\n';
await jest.advanceTimersByTimeAsync(250);
await expect(started).resolves.toMatchObject({ fingerprint: 'pinned-key=' });
expect(spawnDetached).toHaveBeenCalledWith(
expect.objectContaining({ env: { PATH: '/bin', AUTH: '' } })
);
expect(options.env.AUTH).toBe('unexpected:password');
expect(stopAsync).not.toHaveBeenCalled();
});
it('rejects and stops a child that prints its fingerprint then fails to bind', async () => {
const started = startChiselServerAsync(options);
const rejected = expect(started).rejects.toThrow('address already in use');
output += 'listen tcp 127.0.0.1:52001: bind: address already in use\n';
jest.mocked(process.kill).mockImplementation(() => {
throw new Error('ESRCH');
});
await jest.advanceTimersByTimeAsync(250);
await rejected;
expect(stopAsync).toHaveBeenCalledTimes(1);
});
it('stops the child when startup is cancelled', async () => {
const controller = new AbortController();
const started = startChiselServerAsync({ ...options, signal: controller.signal });
const rejected = expect(started).rejects.toThrow('cancelled');
controller.abort(new Error('cancelled'));
await jest.advanceTimersByTimeAsync(250);
await rejected;
expect(stopAsync).toHaveBeenCalledTimes(1);
});
});
describe('local egress resource lifetime', () => {
const logger = { warn: jest.fn() } as unknown as bunyan;
it('aborts the lifetime and awaits the same cleanup for concurrent callers', async () => {
let finish!: () => void;
const cleanup = jest.fn(
() =>
new Promise<void>(resolve => {
finish = resolve;
})
);
const signal = registerLocalEgressResources(cleanup);
const first = stopLocalEgressResourcesAsync(logger);
const second = stopLocalEgressResourcesAsync(logger);
expect(signal.aborted).toBe(true);
await Promise.resolve();
expect(cleanup).toHaveBeenCalledTimes(1);
let complete = false;
void second.then(() => {
complete = true;
});
await Promise.resolve();
expect(complete).toBe(false);
finish();
await Promise.all([first, second]);
await stopLocalEgressResourcesAsync(logger);
expect(cleanup).toHaveBeenCalledTimes(1);
});
it('reports cleanup failure without masking the job result', async () => {
const error = new Error('cleanup failed');
registerLocalEgressResources(async () => {
throw error;
});
await expect(stopLocalEgressResourcesAsync(logger)).resolves.toBeUndefined();
expect(logger.warn).toHaveBeenCalledWith(
{ err: error },
'Could not stop a local egress resource.'
);
});
});
@@ -0,0 +1,426 @@
/**
* End-to-end test of the local egress guard against a real iOS Simulator.
* Needs Xcode with a simulator runtime, a shut-down iPhone simulator, and
* network access. Opt in with EAS_LOCAL_EGRESS_E2E=1; the test-egress-guard
* EAS workflow runs it on macOS.
*
* Variables handed to launchd at boot cannot be changed afterwards with
* `launchctl setenv`, so each configuration gets its own boot.
*/
import { type bunyan } from '@expo/logger';
import spawn from '@expo/turtle-spawn';
import dgram from 'node:dgram';
import fs from 'node:fs';
import http from 'node:http';
import net from 'node:net';
import os from 'node:os';
import path from 'node:path';
import { setTimeout as sleep } from 'node:timers/promises';
import { IosSimulatorUtils, type IosSimulatorUuid } from '../../../utils/IosSimulatorUtils';
import { writeLocalEgressHandoffAsync } from '../localEgress';
import {
type GuardEvent,
installLocalEgressGuardAsync,
parseGuardLogLine,
reportLocalEgressGuardCoverageAsync,
resolveEgressGuardCheckAsync,
resolveEgressGuardLibraryAsync,
resolveLocalEgressBootEnvironmentAsync,
stopLocalEgressGuardRelaysAsync,
verifyLocalEgressGuardAsync,
} from '../localEgressGuard';
jest.unmock('fs');
jest.unmock('node:fs');
jest.unmock('fs/promises');
jest.unmock('node:fs/promises');
jest.setTimeout(15 * 60 * 1000);
const enabled = process.platform === 'darwin' && process.env.EAS_LOCAL_EGRESS_E2E === '1';
const describeE2E = enabled ? describe : describe.skip;
const GUARD_DIR = path.join(__dirname, '..', '..', '..', '..', 'resources', 'egress-guard');
const env = process.env;
function createLogger(): bunyan & { lines: string[] } {
const lines: string[] = [];
const record = (a: unknown, b?: unknown) => {
lines.push(typeof a === 'string' ? a : String(b));
};
return { info: record, warn: record, debug: record, lines } as any;
}
async function readEventsAsync(logPath: string): Promise<GuardEvent[]> {
let raw = '';
try {
raw = await fs.promises.readFile(logPath, 'utf8');
} catch {
return [];
}
return raw
.split('\n')
.map(parseGuardLogLine)
.filter((e): e is GuardEvent => e !== null);
}
async function waitForAsync<T>(
probe: () => Promise<T | undefined>,
{ timeoutMs, intervalMs = 1000 }: { timeoutMs: number; intervalMs?: number }
): Promise<T | undefined> {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
const value = await probe();
if (value !== undefined) {
return value;
}
await sleep(intervalMs);
}
return undefined;
}
describeE2E('local egress guard in a simulator', () => {
let workDir: string;
let udid: IosSimulatorUuid;
let libraryPath: string;
let checkPath: string;
let nettestPath: string;
let handoffPath: string;
let proxyServer: http.Server;
let proxyPort: number;
let udpServer: dgram.Socket;
let udpPort: number;
const logger = createLogger();
/** Shut the device down and boot it again with the given launchd environment. */
async function rebootAsync(launchdEnvironment: Record<string, string>): Promise<void> {
await spawn('xcrun', ['simctl', 'shutdown', udid], { env, stdio: 'pipe' }).catch(() => {});
await IosSimulatorUtils.bootAsync({ deviceIdentifier: udid, env, launchdEnvironment });
await IosSimulatorUtils.startAsync({ deviceIdentifier: udid, env });
}
async function bootEnvironmentAsync(
mode: 'block' | 'log',
logPath: string
): Promise<Record<string, string>> {
const variables = await resolveLocalEgressBootEnvironmentAsync({
handoffPath,
libraryPath,
logPath,
mode,
});
if (!variables) {
throw new Error('The handoff was not written.');
}
return variables;
}
async function runNettestAsync(): Promise<Record<string, string>> {
const result = await spawn(
'xcrun',
[
'simctl',
'spawn',
udid,
nettestPath,
'--proxy-port',
String(proxyPort),
'--udp-port',
String(udpPort),
],
{ env, stdio: 'pipe' }
);
const last = result.stdout.trim().split('\n').at(-1) ?? '{}';
return JSON.parse(last);
}
beforeAll(async () => {
workDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'egress-guard-e2e-'));
// The guard library and self-check, built on demand.
let resolved = await resolveEgressGuardLibraryAsync();
if (!resolved) {
await spawn('bash', [path.join(GUARD_DIR, 'build.sh')], { env, stdio: 'pipe' });
resolved = await resolveEgressGuardLibraryAsync();
}
if (!resolved) {
throw new Error('egress-guard.dylib is not available and could not be built.');
}
libraryPath = resolved;
const resolvedCheck = await resolveEgressGuardCheckAsync();
if (!resolvedCheck) {
throw new Error('egress-guard-check is not available and could not be built.');
}
checkPath = resolvedCheck;
// The probe binary, compiled for the simulator running on this host.
nettestPath = path.join(workDir, 'nettest');
await spawn(
'xcrun',
[
'-sdk',
'iphonesimulator',
'swiftc',
'-target',
`${process.arch === 'arm64' ? 'arm64' : 'x86_64'}-apple-ios15.0-simulator`,
path.join(GUARD_DIR, 'tests', 'nettest.swift'),
'-o',
nettestPath,
],
{ env, stdio: 'pipe' }
);
handoffPath = path.join(workDir, 'handoff.json');
await writeLocalEgressHandoffAsync(
{ url: 'https://egress.example', token: 'pw', fingerprint: 'fp=', port: 8899 },
handoffPath
);
// A shut-down iPhone the test can boot as many times as it needs. A device
// someone else booted is left alone. Nothing is ever deleted.
const booted = new Set(
(await IosSimulatorUtils.getAvailableDevicesAsync({ env, filter: 'booted' })).map(d => d.udid)
);
const available = await IosSimulatorUtils.getAvailableDevicesAsync({
env,
filter: 'available',
});
const iphone =
available.find(d => /^iPhone \d/.test(d.name) && !booted.has(d.udid)) ??
available.find(d => /^iPhone/.test(d.name) && !booted.has(d.udid));
if (!iphone) {
throw new Error('No shut-down iPhone simulator is available for the test to boot.');
}
udid = iphone.udid;
// Loopback stand-ins for the egress proxy and for a local UDP service.
proxyServer = http.createServer((_req, res) => {
res.writeHead(502);
res.end();
});
proxyServer.on('connect', (req, socket, head) => {
const [host, port] = String(req.url).split(':');
const upstream = net.connect(Number(port) || 443, host, () => {
socket.write('HTTP/1.1 200 Connection Established\r\n\r\n');
if (head.length) {
upstream.write(new Uint8Array(head));
}
upstream.pipe(socket);
socket.pipe(upstream);
});
upstream.on('error', () => socket.destroy());
socket.on('error', () => upstream.destroy());
});
await new Promise<void>(resolve => proxyServer.listen(0, '127.0.0.1', () => resolve()));
proxyPort = (proxyServer.address() as net.AddressInfo).port;
udpServer = dgram.createSocket('udp4');
udpServer.on('message', (msg, rinfo) =>
udpServer.send(new Uint8Array(msg), rinfo.port, rinfo.address)
);
await new Promise<void>(resolve => udpServer.bind(0, '127.0.0.1', () => resolve()));
udpPort = udpServer.address().port;
});
afterAll(async () => {
await stopLocalEgressGuardRelaysAsync(logger);
proxyServer?.close();
udpServer?.close();
if (udid) {
await spawn('xcrun', ['simctl', 'shutdown', udid], { env, stdio: 'pipe' }).catch(() => {});
}
if (workDir) {
await fs.promises.rm(workDir, { recursive: true, force: true });
}
});
describe('booted with the guard in launchd, as the worker boots', () => {
let logPath: string;
beforeAll(async () => {
logPath = path.join(workDir, 'boot.log');
await rebootAsync(await bootEnvironmentAsync('block', logPath));
await verifyLocalEgressGuardAsync({ udid, env, logger, mode: 'block', checkPath });
});
it('covers every process of the boot', async () => {
const coverage = await reportLocalEgressGuardCoverageAsync({ env, logger });
expect(coverage).not.toBeNull();
expect(coverage!.covered.length).toBeGreaterThan(50);
expect(coverage!.uncovered).toEqual([]);
// Refusals were recorded by processes that started during the boot, before
// any launchctl call could have reached them.
expect((await readEventsAsync(logPath)).length).toBeGreaterThan(0);
});
it('refuses every direct path and leaves loopback and DNS alone', async () => {
const results = await runNettestAsync();
expect(results).toMatchObject({
urlsessionDirect: 'refused',
urlsessionProxied: 'ok(200)',
bsdConnectDirect: 'refused',
dns: 'ok',
udpDirect: 'refused',
udpLoopback: 'ok',
literalFirst: 'refused',
literalSecond: 'refused',
unspecConnect: 'refused',
udpDisconnect: 'ok',
nocancelSendto: 'refused',
});
expect(results.nwconnectionDirect).not.toMatch(/^ok|^timeout/);
const events = (await readEventsAsync(logPath)).filter(e => e.process === 'nettest');
expect(events.length).toBeGreaterThanOrEqual(4);
expect(events.every(e => e.action === 'blocked')).toBe(true);
// URLSession and Network.framework connects are caught inside Apple's frameworks.
expect(
events.some(e => e.function === 'connect' && e.callers.some(c => /CFNetwork/.test(c)))
).toBe(true);
expect(
events.some(
e =>
e.function === 'connect' &&
e.callers.some(c => /Network/.test(c) && !/CFNetwork/.test(c))
)
).toBe(true);
// UDP is covered, through the cancelable and non-cancelable entry points.
expect(events.some(e => e.function === 'sendto' && e.peer === '8.8.8.8:53')).toBe(true);
// An AF_UNSPEC sockaddr is recorded as what the kernel would have used.
expect(events.some(e => e.function === 'connect' && e.peer === '1.0.0.1:443')).toBe(true);
// The callers name the process and frameworks, never the guard itself.
expect(events.every(e => !e.callers.some(c => /egress-guard/.test(c)))).toBe(true);
// One line per destination per process, even when the process retries.
expect(events.filter(e => e.function === 'connect' && e.peer === '1.1.1.1:443')).toHaveLength(
1
);
// Nothing on loopback is ever reported.
expect(events.some(e => /^127\./.test(e.peer))).toBe(false);
});
it('guards processes the simulator launches on its own', async () => {
await spawn('xcrun', ['simctl', 'openurl', udid, 'https://example.net/'], {
env,
stdio: 'pipe',
});
const daemonEvent = await waitForAsync(
async () =>
(await readEventsAsync(logPath)).find(
e => e.process !== 'nettest' && e.action === 'blocked'
),
{ timeoutMs: 60_000 }
);
expect(daemonEvent).toBeDefined();
});
it('does not interfere with the simulator tooling the worker relies on', async () => {
const screenshot = path.join(workDir, 'screenshot.png');
for (const args of [
['simctl', 'spawn', udid, 'launchctl', 'list'],
[
'simctl',
'spawn',
udid,
'defaults',
'write',
'dev.expo.egress-guard-test',
'key',
'-string',
'value',
],
['simctl', 'spawn', udid, '/usr/bin/env'],
['simctl', 'launch', udid, 'com.apple.Preferences'],
['simctl', 'io', udid, 'screenshot', screenshot],
]) {
await expect(spawn('xcrun', args, { env, stdio: 'pipe' })).resolves.toBeDefined();
}
expect(fs.existsSync(screenshot)).toBe(true);
});
});
describe('booted in log mode', () => {
let logPath: string;
beforeAll(async () => {
logPath = path.join(workDir, 'log-mode.log');
await rebootAsync(await bootEnvironmentAsync('log', logPath));
await verifyLocalEgressGuardAsync({ udid, env, logger, mode: 'log', checkPath });
});
it('observes without refusing', async () => {
const results = await runNettestAsync();
expect(results.urlsessionDirect).toBe('ok(200)');
expect(results.bsdConnectDirect).toBe('ok');
expect(results.udpDirect).toBe('sent');
const events = (await readEventsAsync(logPath)).filter(e => e.process === 'nettest');
expect(events.length).toBeGreaterThan(0);
expect(events.every(e => e.action === 'logged')).toBe(true);
});
});
describe('booted without the guard, as a device that was already running', () => {
beforeAll(async () => {
await rebootAsync({});
});
it('is reported as not in effect by the self-check', async () => {
await expect(
verifyLocalEgressGuardAsync({ udid, env, logger, mode: 'block', checkPath })
).rejects.toThrow(/not in effect/);
});
it('can be installed after boot through launchctl for processes started from then on', async () => {
const logPath = path.join(workDir, 'install.log');
expect(
await installLocalEgressGuardAsync({
udid,
env,
logger,
handoffPath,
libraryPath,
logPath,
mode: 'block',
tailIntervalMs: 100,
})
).toBe(true);
await verifyLocalEgressGuardAsync({ udid, env, logger, mode: 'block', checkPath });
const results = await runNettestAsync();
expect(results.urlsessionDirect).toBe('refused');
expect(results.bsdConnectDirect).toBe('refused');
// The relay turns the guard's events into session log lines.
const relayed = await waitForAsync(
async () =>
logger.lines.find(l =>
/refused connect from nettest \(pid \d+\) to 1\.1\.1\.1:443/.test(l)
),
{ timeoutMs: 5_000, intervalMs: 200 }
);
expect(relayed).toBeDefined();
// Processes from before the install are the uncovered set the coverage
// report names; nettest itself was covered.
const coverage = await reportLocalEgressGuardCoverageAsync({ env, logger });
expect(coverage!.uncovered.length).toBeGreaterThan(0);
});
});
describe('booted with an unwritable event log', () => {
beforeAll(async () => {
await rebootAsync(
await bootEnvironmentAsync('block', path.join(workDir, 'no-such-dir', 'guard.log'))
);
});
it('still refuses', async () => {
await verifyLocalEgressGuardAsync({ udid, env, logger, mode: 'block', checkPath });
const results = await runNettestAsync();
expect(results.urlsessionDirect).toBe('refused');
expect(results.bsdConnectDirect).toBe('refused');
expect(results.urlsessionProxied).toBe('ok(200)');
});
});
});
@@ -0,0 +1,565 @@
import { SystemError } from '@expo/eas-build-job';
import { type bunyan } from '@expo/logger';
import spawn from '@expo/turtle-spawn';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { setTimeout as sleep } from 'node:timers/promises';
import { IosSimulatorUtils } from '../../../utils/IosSimulatorUtils';
import { writeLocalEgressHandoffAsync } from '../localEgress';
import {
EGRESS_GUARD_LOG_ENV,
EGRESS_GUARD_MODE_ENV,
GuardEventRelay,
GuardLogTailer,
buildGuardLaunchdEnvironment,
installLocalEgressGuardAsync,
mergeGuardCoverageSamples,
parseGuardCoverage,
parseGuardLogLine,
rebindLocalEgressGuardRelays,
resolveEgressGuardLibraryAsync,
resolveLocalEgressBootEnvironmentAsync,
stopLocalEgressGuardRelaysAsync,
verifyLocalEgressGuardAsync,
} from '../localEgressGuard';
jest.mock('@expo/turtle-spawn');
jest.mock('../../../utils/IosSimulatorUtils', () => ({
IosSimulatorUtils: { setLaunchdEnvironmentAsync: jest.fn() },
}));
const mockedSetEnv = jest.mocked(IosSimulatorUtils.setLaunchdEnvironmentAsync);
const mockedSpawn = jest.mocked(spawn);
function createLogger(): bunyan & { lines: { level: string; msg: string }[] } {
const lines: { level: string; msg: string }[] = [];
const record = (level: string) => (a: unknown, b?: unknown) =>
lines.push({ level, msg: typeof a === 'string' ? a : String(b) });
return { info: record('info'), warn: record('warn'), debug: record('debug'), lines } as any;
}
const handoff = { url: 'https://egress.example', token: 'pw', fingerprint: 'fp=', port: 8899 };
describe(buildGuardLaunchdEnvironment, () => {
it('inserts the library and passes the log path and mode', () => {
expect(
buildGuardLaunchdEnvironment({
libraryPath: '/w/bin/egress-guard.dylib',
logPath: '/tmp/guard.log',
mode: 'block',
})
).toEqual({
DYLD_INSERT_LIBRARIES: '/w/bin/egress-guard.dylib',
[EGRESS_GUARD_LOG_ENV]: '/tmp/guard.log',
[EGRESS_GUARD_MODE_ENV]: 'block',
});
});
});
describe(parseGuardLogLine, () => {
it('parses a blocked event with callers', () => {
expect(
parseGuardLogLine(
'eas-egress-guard\tMyApp\t4242\tconnect\tblocked\t93.184.216.34:443\tNetwork,CFNetwork,MyApp'
)
).toEqual({
process: 'MyApp',
pid: 4242,
function: 'connect',
action: 'blocked',
peer: '93.184.216.34:443',
callers: ['Network', 'CFNetwork', 'MyApp'],
});
});
it('parses a logged event without callers and IPv6 peers', () => {
expect(
parseGuardLogLine('eas-egress-guard\tdaemon\t7\tsendto\tlogged\t[2606:4700::1]:53\t')
).toEqual({
process: 'daemon',
pid: 7,
function: 'sendto',
action: 'logged',
peer: '[2606:4700::1]:53',
callers: [],
});
});
it('ignores lines that are not guard events', () => {
expect(parseGuardLogLine('')).toBeNull();
expect(parseGuardLogLine('something else\ta\tb')).toBeNull();
expect(
parseGuardLogLine('eas-egress-guard\tMyApp\tnot-a-pid\tconnect\tblocked\t1.1.1.1:443\t')
).toBeNull();
expect(
parseGuardLogLine('eas-egress-guard\tMyApp\t1\tconnect\tmaybe\t1.1.1.1:443\t')
).toBeNull();
});
});
describe(GuardEventRelay, () => {
const blocked = (process: string, peer: string, pid = 1) => ({
process,
pid,
function: 'connect',
action: 'blocked' as const,
peer,
callers: ['Network', 'CFNetwork'],
});
it('logs the first sighting of a process and destination, then only counts', () => {
const logger = createLogger();
const relay = new GuardEventRelay(logger);
relay.handle(blocked('MyApp', '1.1.1.1:443'));
relay.handle(blocked('MyApp', '1.1.1.1:443', 2));
relay.handle(blocked('MyApp', '2.2.2.2:443'));
expect(logger.lines.map(l => l.msg)).toEqual([
expect.stringContaining(
'refused connect from MyApp (pid 1) to 1.1.1.1:443; callers: Network, CFNetwork'
),
expect.stringContaining('refused connect from MyApp (pid 1) to 2.2.2.2:443'),
]);
expect(relay.summary()).toEqual({ blocked: 3, logged: 0, distinct: 2, suppressed: 0 });
});
it('words observed events differently and stops logging past the limit', () => {
const logger = createLogger();
const relay = new GuardEventRelay(logger, /* limit */ 1);
relay.handle({ ...blocked('MyApp', '1.1.1.1:443'), action: 'logged' });
relay.handle({ ...blocked('MyApp', '2.2.2.2:443'), action: 'logged' });
expect(logger.lines).toHaveLength(1);
expect(logger.lines[0].msg).toContain('observed connect from MyApp');
expect(relay.summary()).toEqual({ blocked: 0, logged: 2, distinct: 2, suppressed: 1 });
});
it('explains a process that filled its table instead of counting it as a destination', () => {
const logger = createLogger();
const relay = new GuardEventRelay(logger);
relay.handle(blocked('MyApp', '1.1.1.1:443'));
relay.handle({
...blocked('MyApp', '128 distinct destinations'),
function: 'overflow',
callers: [],
});
expect(logger.lines.at(-1)?.msg).toContain(
'MyApp (pid 1) reached the limit of 128 distinct destinations listed per process; further distinct destinations from it are refused but not listed'
);
expect(relay.summary()).toEqual({ blocked: 1, logged: 0, distinct: 1, suppressed: 0 });
});
it('ignores the self-check probe, which trips the guard on purpose', () => {
const logger = createLogger();
const relay = new GuardEventRelay(logger);
relay.handle(blocked('egress-guard-check', '192.0.2.1:9'));
expect(logger.lines).toHaveLength(0);
expect(relay.summary()).toEqual({ blocked: 0, logged: 0, distinct: 0, suppressed: 0 });
});
it('writes a summary line', () => {
const logger = createLogger();
const relay = new GuardEventRelay(logger);
relay.handle(blocked('MyApp', '1.1.1.1:443'));
relay.handle(blocked('MyApp', '1.1.1.1:443'));
relay.handle(blocked('daemon', '3.3.3.3:443', 9));
relay.logSummary();
expect(logger.lines.at(-1)?.msg).toContain(
'refused 3 connection attempt(s) to 2 distinct destination(s) from 2 process(es)'
);
});
});
describe(GuardLogTailer, () => {
let dir: string;
beforeEach(async () => {
dir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'guard-tailer-'));
});
afterEach(async () => {
await fs.promises.rm(dir, { recursive: true, force: true });
});
it('delivers whole lines as they are appended, keeps partial lines, and survives truncation', async () => {
const file = path.join(dir, 'guard.log');
const lines: string[] = [];
const tailer = new GuardLogTailer({ path: file, onLine: l => lines.push(l), intervalMs: 20 });
tailer.start();
await sleep(50); // file does not exist yet; must not throw
await fs.promises.writeFile(file, 'first\nsecond\npart', 'utf8');
await sleep(80);
expect(lines).toEqual(['first', 'second']);
await fs.promises.appendFile(file, 'ial\n', 'utf8');
await sleep(80);
expect(lines).toEqual(['first', 'second', 'partial']);
await fs.promises.writeFile(file, 'after-truncate\n', 'utf8');
await sleep(80);
expect(lines).toEqual(['first', 'second', 'partial', 'after-truncate']);
await tailer.stopAsync();
await fs.promises.appendFile(file, 'late\n', 'utf8');
await sleep(60);
expect(lines).not.toContain('late');
});
it('flushes what is already in the file when stopped', async () => {
const file = path.join(dir, 'guard.log');
const lines: string[] = [];
const tailer = new GuardLogTailer({
path: file,
onLine: l => lines.push(l),
intervalMs: 10_000,
});
tailer.start();
await fs.promises.writeFile(file, 'only\n', 'utf8');
await tailer.stopAsync();
expect(lines).toEqual(['only']);
});
});
describe(resolveEgressGuardLibraryAsync, () => {
it('returns null when the library is not built', async () => {
await expect(
resolveEgressGuardLibraryAsync(path.join(os.tmpdir(), 'no-such-dir'))
).resolves.toBeNull();
});
it('returns the packaged path when present', async () => {
const dir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'guard-bin-'));
const lib = path.join(dir, 'egress-guard.dylib');
await fs.promises.writeFile(lib, '');
await expect(resolveEgressGuardLibraryAsync(dir)).resolves.toBe(lib);
await fs.promises.rm(dir, { recursive: true, force: true });
});
});
describe(installLocalEgressGuardAsync, () => {
let dir: string;
let logger: ReturnType<typeof createLogger>;
beforeEach(async () => {
mockedSetEnv.mockReset();
mockedSetEnv.mockResolvedValue(undefined);
mockedSpawn.mockReset();
mockedSpawn.mockResolvedValue({ stdout: '', stderr: '' } as any);
dir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'guard-install-'));
logger = createLogger();
});
afterEach(async () => {
await stopLocalEgressGuardRelaysAsync(logger);
await fs.promises.rm(dir, { recursive: true, force: true });
});
it('does nothing without a local egress session', async () => {
await expect(
installLocalEgressGuardAsync({
udid: 'u' as any,
env: process.env,
logger,
handoffPath: path.join(dir, 'missing.json'),
libraryPath: path.join(dir, 'egress-guard.dylib'),
logPath: path.join(dir, 'guard.log'),
})
).resolves.toBe(false);
expect(mockedSetEnv).not.toHaveBeenCalled();
});
it('fails the session when the library is not available', async () => {
const handoffPath = path.join(dir, 'handoff.json');
await writeLocalEgressHandoffAsync(handoff, handoffPath);
await expect(
installLocalEgressGuardAsync({
udid: 'u' as any,
env: process.env,
logger,
handoffPath,
libraryPath: null,
logPath: path.join(dir, 'guard.log'),
})
).rejects.toThrow(SystemError);
expect(mockedSetEnv).not.toHaveBeenCalled();
});
it('logs the simulator processes that were already running and are not covered', async () => {
const handoffPath = path.join(dir, 'handoff.json');
await writeLocalEgressHandoffAsync(handoff, handoffPath);
const libraryPath = path.join(dir, 'egress-guard.dylib');
await fs.promises.writeFile(libraryPath, '');
mockedSpawn.mockResolvedValue({
stdout: [
' 1 0 /sbin/launchd',
' 4000 1 /Library/Developer/CoreSimulator/Volumes/iOS/Runtimes/x/sbin/launchd_sim',
' 4001 4000 /Library/Developer/CoreSimulator/Volumes/iOS/Runtimes/x/System/Library/CoreServices/SpringBoard.app/SpringBoard',
' 4002 4000 /Library/Developer/CoreSimulator/Volumes/iOS/Runtimes/x/usr/libexec/backboardd',
' 3758 1 node',
].join('\n'),
stderr: '',
} as any);
await installLocalEgressGuardAsync({
udid: 'u' as any,
env: process.env,
logger,
handoffPath,
libraryPath,
logPath: path.join(dir, 'guard.log'),
});
expect(
logger.lines.some(l =>
/2 simulator process\(es\) were already running .* not covered by it: SpringBoard, backboardd\./.test(
l.msg
)
)
).toBe(true);
});
it('sets the launchd environment, creates the log file, and relays events into the session log', async () => {
const handoffPath = path.join(dir, 'handoff.json');
await writeLocalEgressHandoffAsync(handoff, handoffPath);
const libraryPath = path.join(dir, 'egress-guard.dylib');
await fs.promises.writeFile(libraryPath, '');
const logPath = path.join(dir, 'guard.log');
await expect(
installLocalEgressGuardAsync({
udid: 'u' as any,
env: process.env,
logger,
handoffPath,
libraryPath,
logPath,
mode: 'block',
tailIntervalMs: 20,
})
).resolves.toBe(true);
expect(mockedSetEnv).toHaveBeenCalledWith({
udid: 'u',
env: process.env,
variables: buildGuardLaunchdEnvironment({ libraryPath, logPath, mode: 'block' }),
});
expect(fs.existsSync(logPath)).toBe(true);
expect(logger.lines.some(l => /guard installed/.test(l.msg) && /block/.test(l.msg))).toBe(true);
await fs.promises.appendFile(
logPath,
'eas-egress-guard\tMyApp\t4242\tconnect\tblocked\t1.1.1.1:443\tNetwork\n'
);
await sleep(80);
expect(
logger.lines.some(l =>
/refused connect from MyApp \(pid 4242\) to 1\.1\.1\.1:443/.test(l.msg)
)
).toBe(true);
await stopLocalEgressGuardRelaysAsync(logger);
expect(logger.lines.at(-1)?.msg).toContain('refused 1 connection attempt(s)');
});
it('attributes later events and the summary to a rebound logger', async () => {
const handoffPath = path.join(dir, 'handoff.json');
await writeLocalEgressHandoffAsync(handoff, handoffPath);
const libraryPath = path.join(dir, 'egress-guard.dylib');
await fs.promises.writeFile(libraryPath, '');
const logPath = path.join(dir, 'guard.log');
await installLocalEgressGuardAsync({
udid: 'u' as any,
env: process.env,
logger,
handoffPath,
libraryPath,
logPath,
tailIntervalMs: 20,
});
const sessionLogger = createLogger();
rebindLocalEgressGuardRelays(sessionLogger);
await fs.promises.appendFile(
logPath,
'eas-egress-guard\tMyApp\t4242\tconnect\tblocked\t1.1.1.1:443\tNetwork\n'
);
await sleep(80);
expect(sessionLogger.lines.some(l => /refused connect from MyApp/.test(l.msg))).toBe(true);
expect(logger.lines.some(l => /refused connect from MyApp/.test(l.msg))).toBe(false);
await stopLocalEgressGuardRelaysAsync(logger);
expect(sessionLogger.lines.at(-1)?.msg).toContain('refused 1 connection attempt(s)');
});
it('fails the session when launchctl fails', async () => {
const handoffPath = path.join(dir, 'handoff.json');
await writeLocalEgressHandoffAsync(handoff, handoffPath);
const libraryPath = path.join(dir, 'egress-guard.dylib');
await fs.promises.writeFile(libraryPath, '');
mockedSetEnv.mockRejectedValue(new Error('launchctl failed'));
await expect(
installLocalEgressGuardAsync({
udid: 'u' as any,
env: process.env,
logger,
handoffPath,
libraryPath,
logPath: path.join(dir, 'guard.log'),
})
).rejects.toThrow(/Could not install the local egress guard/);
});
});
describe(resolveLocalEgressBootEnvironmentAsync, () => {
let dir: string;
beforeEach(async () => {
dir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'guard-bootenv-'));
});
afterEach(async () => {
await fs.promises.rm(dir, { recursive: true, force: true });
});
it('returns null without a local egress session', async () => {
await expect(
resolveLocalEgressBootEnvironmentAsync({ handoffPath: path.join(dir, 'missing.json') })
).resolves.toBeNull();
});
it('fails when the guard library is not available', async () => {
const handoffPath = path.join(dir, 'handoff.json');
await writeLocalEgressHandoffAsync(handoff, handoffPath);
await expect(
resolveLocalEgressBootEnvironmentAsync({ handoffPath, libraryPath: null })
).rejects.toThrow(SystemError);
});
it('combines the guard variables with the proxy variables for the handoff port', async () => {
const handoffPath = path.join(dir, 'handoff.json');
await writeLocalEgressHandoffAsync({ ...handoff, port: 8899 }, handoffPath);
await expect(
resolveLocalEgressBootEnvironmentAsync({
handoffPath,
libraryPath: '/w/bin/egress-guard.dylib',
logPath: '/tmp/guard.log',
mode: 'block',
})
).resolves.toEqual({
DYLD_INSERT_LIBRARIES: '/w/bin/egress-guard.dylib',
[EGRESS_GUARD_LOG_ENV]: '/tmp/guard.log',
[EGRESS_GUARD_MODE_ENV]: 'block',
http_proxy: 'http://127.0.0.1:8899',
https_proxy: 'http://127.0.0.1:8899',
HTTP_PROXY: 'http://127.0.0.1:8899',
HTTPS_PROXY: 'http://127.0.0.1:8899',
grpc_proxy: 'http://127.0.0.1:8899',
no_proxy: 'localhost,127.0.0.1,::1',
NO_PROXY: 'localhost,127.0.0.1,::1',
});
});
});
describe(parseGuardCoverage, () => {
const ps = [
' 1 0 /sbin/launchd',
' 4000 1 /Runtimes/x/sbin/launchd_sim',
' 4001 4000 /Runtimes/x/System/Library/CoreServices/SpringBoard.app/SpringBoard',
' 4002 4000 /Runtimes/x/usr/libexec/backboardd',
' 4003 4001 /Containers/Bundle/Application/1/Expo Go.app/Expo Go',
' 3758 1 node',
].join('\n');
const lsof = [
'p4001',
'n/Runtimes/x/System/Library/CoreServices/SpringBoard.app/SpringBoard',
'n/usr/lib/dyld',
'p4003',
'n/Containers/Bundle/Application/1/Expo Go.app/Expo Go',
'n/Users/expo/build-tools/bin/egress-guard.dylib',
'p3758',
'n/usr/local/bin/node',
].join('\n');
it('splits simulator processes by whether the guard library is mapped', () => {
expect(parseGuardCoverage(ps, lsof)).toEqual({
covered: ['Expo Go'],
uncovered: ['SpringBoard', 'backboardd'],
uncoveredPids: [4001, 4002],
});
});
it('only keeps processes uncovered in both samples, so exec-in-progress ones drop out', () => {
const first = parseGuardCoverage(ps, lsof);
const secondPs = ps + '\n 4009 4000 /Runtimes/x/usr/libexec/newlystarted';
const second = parseGuardCoverage(
secondPs,
lsof + '\np4002\nn/Users/expo/bin/egress-guard.dylib'
);
expect(mergeGuardCoverageSamples(first, second)).toEqual({
covered: ['Expo Go', 'backboardd'],
uncovered: ['SpringBoard'],
uncoveredPids: [4001],
});
});
it("ignores launchd's exec trampoline, which has nothing mapped yet", () => {
const withTrampoline = ps + '\n 4004 4000 /Runtimes/x/usr/libexec/xpcproxy_sim';
expect(parseGuardCoverage(withTrampoline, lsof).uncovered).toEqual([
'SpringBoard',
'backboardd',
]);
});
});
describe(verifyLocalEgressGuardAsync, () => {
let logger: ReturnType<typeof createLogger>;
beforeEach(() => {
mockedSpawn.mockReset();
logger = createLogger();
});
it('runs the packaged self-check inside the simulator and logs its verdict', async () => {
mockedSpawn
.mockResolvedValueOnce({
stdout: 'egress-guard-check: guard loaded; non-loopback connections are refused\n',
stderr: '',
} as any)
.mockResolvedValue({ stdout: '', stderr: '' } as any);
await verifyLocalEgressGuardAsync({
udid: 'u' as any,
env: process.env,
logger,
checkPath: '/w/bin/egress-guard-check',
});
expect(mockedSpawn).toHaveBeenCalledWith(
'xcrun',
['simctl', 'spawn', 'u', '/w/bin/egress-guard-check', '--mode', 'block'],
expect.objectContaining({ stdio: 'pipe' })
);
expect(
logger.lines.some(l =>
/verified in the Simulator: egress-guard-check: guard loaded/.test(l.msg)
)
).toBe(true);
});
it('fails the session with the self-check output when the check fails', async () => {
mockedSpawn.mockRejectedValue(
Object.assign(new Error('exited with non-zero code: 2'), {
status: 2,
stdout: 'egress-guard-check: the guard library is not loaded in this process\n',
stderr: '',
})
);
await expect(
verifyLocalEgressGuardAsync({
udid: 'u' as any,
env: process.env,
logger,
checkPath: '/w/bin/egress-guard-check',
})
).rejects.toThrow(/not in effect .*\(exit 2\).*guard library is not loaded/);
});
it('fails the session when the self-check binary is missing', async () => {
await expect(
verifyLocalEgressGuardAsync({ udid: 'u' as any, env: process.env, logger, checkPath: null })
).rejects.toThrow(/self-check is not available/);
expect(mockedSpawn).not.toHaveBeenCalled();
});
});
@@ -11,16 +11,20 @@ import { setTimeout as setTimeoutAsync } from 'node:timers/promises';
import { CustomBuildContext } from '../../../customBuildContext';
import { Sentry } from '../../../sentry';
import { turtleFetch } from '../../../utils/turtleFetch';
import { readServeSimServersAsync } from '../serveSimMetricsRecorder';
import { sleepAsync } from '../../../utils/retry';
import {
createExpoDeviceHubArgs,
createServeSimArgs,
ensureFfmpegInstalledAsync,
fetchServeSimTurnArgsAsync,
ensureFfmpegInstalledOnceAsync,
fetchWebPreviewTurnArgsAsync,
metricsCorsOriginToServeSimArgs,
startDeviceWebPreviewWithTunnelAsync,
startExpoDeviceHubWithTunnelAsync,
startNgrokTunnelAsync,
turnIceServersToServeSimArgs,
turnIceServersToWebPreviewArgs,
waitForDeviceRunSessionStoppedAsync,
waitForServeSimReadyAsync,
waitForWebPreviewReadyAsync,
} from '../remoteDeviceRunSession';
jest.mock('@ngrok/ngrok');
@@ -30,6 +34,11 @@ jest.mock('../../../utils/turtleFetch');
jest.mock('../../../utils/retry', () => ({ sleepAsync: jest.fn() }));
jest.mock('../../../sentry');
jest.mock('@expo/turtle-spawn');
// Spyable so a test can stand in for the serve-sim state directory, which a local serve-sim owns.
jest.mock('../serveSimMetricsRecorder', () => {
const actual = jest.requireActual('../serveSimMetricsRecorder');
return { ...actual, readServeSimServersAsync: jest.fn(actual.readServeSimServersAsync) };
});
function createLoggerMock(): bunyan {
return {
@@ -127,16 +136,17 @@ describe(createServeSimArgs, () => {
'4321',
'--host',
'127.0.0.1',
'--require-token',
'--transport',
'webrtc',
'--webrtc-codec',
'vp8',
'--max-dimension',
'1280',
'960',
'--mjpeg-quality',
'0.55',
'--video-bitrate',
'3000000',
'6000000',
'--video-fps',
'60',
'--turn-url',
@@ -157,6 +167,63 @@ describe(createServeSimArgs, () => {
'https://expo.dev',
]);
});
it('pins the requested package version', () => {
expect(createServeSimArgs({ port: 4321, packageVersion: '0.1.38' }).slice(0, 2)).toEqual([
'--yes',
'@expo/[email protected]',
]);
});
it('pins a dist-tag', () => {
expect(createServeSimArgs({ port: 4321, packageVersion: 'next' }).slice(0, 2)).toEqual([
'--yes',
'@expo/serve-sim@next',
]);
});
});
describe(createExpoDeviceHubArgs, () => {
it('uses the latest Expo package and applies the EAS Android streaming policy', () => {
expect(
createExpoDeviceHubArgs({
port: 4321,
turnArgs: ['--turn-url', 'turns:turn.example.test:443'],
})
).toEqual([
'--yes',
'expo-device-hub@latest',
'--port',
'4321',
'--host',
'127.0.0.1',
'--platform',
'android',
'--transport',
'webrtc',
'--webrtc-codec',
'h264',
'--webrtc-ice-policy',
'all',
'--max-dimension',
'960',
'--video-bitrate',
'6000000',
'--video-fps',
'60',
'--hide-sidebar',
'--hide-boot-device',
'--turn-url',
'turns:turn.example.test:443',
]);
});
it('pins the requested package version', () => {
expect(createExpoDeviceHubArgs({ port: 4321, packageVersion: '0.7.0' }).slice(0, 2)).toEqual([
'--yes',
'[email protected]',
]);
});
});
describe(metricsCorsOriginToServeSimArgs, () => {
@@ -186,7 +253,7 @@ describe(metricsCorsOriginToServeSimArgs, () => {
});
});
describe(waitForServeSimReadyAsync, () => {
describe(waitForWebPreviewReadyAsync, () => {
beforeEach(() => {
jest.mocked(turtleFetch).mockReset();
jest.mocked(sleepAsync).mockReset();
@@ -201,8 +268,9 @@ describe(waitForServeSimReadyAsync, () => {
json: async () => ({ status: 'ready', device: 'DEVICE-A' }),
} as unknown as Awaited<ReturnType<typeof turtleFetch>>);
await waitForServeSimReadyAsync({
serveSim: { pid: undefined, getOutput: () => '' },
await waitForWebPreviewReadyAsync({
previewServer: { pid: undefined, getOutput: () => '' },
serverName: 'expo-device-hub',
port: 4321,
timeoutMs: 10_000,
});
@@ -247,14 +315,239 @@ describe(startNgrokTunnelAsync, () => {
});
});
describe(turnIceServersToServeSimArgs, () => {
describe(startDeviceWebPreviewWithTunnelAsync, () => {
const baseDomain = 'eas-simulator.ngrok.dev';
const turnArgs = [
'--turn-url',
'turns:turn.example.test:443',
'--turn-username',
'turn-user',
'--turn-credential',
'turn-credential',
];
const metricsCorsArgs = ['--metrics-cors-origin', 'https://metrics.expo.test'];
const env = {
DEVICE_RUN_SESSION_ID: 'drs-id',
EAS_SIMULATOR_METRICS_CORS_ORIGIN: 'https://metrics.expo.test',
NGROK_AUTHTOKEN: 'ngrok-token',
} as unknown as BuildStepEnv;
beforeEach(() => {
jest.mocked(spawn).mockReset();
jest.mocked(ngrok.forward).mockReset();
jest.mocked(turtleFetch).mockReset();
jest
.mocked(readServeSimServersAsync)
.mockResolvedValue([{ udid: 'device-id', url: 'http://127.0.0.1:1', token: 'tok-1' }]);
const spawnPromise = Object.assign(Promise.resolve(undefined), {
child: {
pid: undefined,
unref: jest.fn(),
},
});
jest.mocked(spawn).mockReturnValue(spawnPromise as never);
jest.mocked(turtleFetch).mockImplementation(async url => {
if (url.endsWith('/turn-ice-servers')) {
return {
json: async () => ({
data: {
iceServers: [
{
urls: ['turns:turn.example.test:443'],
username: 'turn-user',
credential: 'turn-credential',
},
],
},
}),
} as unknown as Awaited<ReturnType<typeof turtleFetch>>;
}
return {
json: async () => ({ status: 'ready', device: 'device-id' }),
} as unknown as Awaited<ReturnType<typeof turtleFetch>>;
});
});
it('installs ffmpeg before starting expo-device-hub for Linux', async () => {
const packageVersion = '1.2.3';
const close = jest.fn().mockResolvedValue(undefined);
jest.mocked(ngrok.forward).mockResolvedValue({
url: () => 'https://android-preview.example.test',
close,
} as never);
jest
.mocked(spawn)
.mockReturnValueOnce(
Promise.reject(new Error('ffmpeg is missing')) as ReturnType<typeof spawn>
)
.mockReturnValueOnce(Promise.resolve({}) as unknown as ReturnType<typeof spawn>)
.mockReturnValueOnce(Promise.resolve({}) as unknown as ReturnType<typeof spawn>);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.LINUX,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
packageVersion,
});
const spawnCalls = jest.mocked(spawn).mock.calls;
expect(spawnCalls[0]).toEqual(['ffmpeg', ['-version'], { env }]);
expect(spawnCalls[1]).toEqual([
'sudo',
['apt-get', 'update'],
expect.objectContaining({
env: expect.objectContaining({ DEBIAN_FRONTEND: 'noninteractive' }),
}),
]);
expect(spawnCalls[2]).toEqual([
'sudo',
['apt-get', 'install', '-y', 'ffmpeg'],
expect.objectContaining({
env: expect.objectContaining({ DEBIAN_FRONTEND: 'noninteractive' }),
}),
]);
const expoDeviceHubCallIndex = spawnCalls.findIndex(([command]) => command === 'npx');
expect(expoDeviceHubCallIndex).toBeGreaterThan(2);
expect(jest.mocked(spawn).mock.invocationCallOrder[2]).toBeLessThan(
jest.mocked(spawn).mock.invocationCallOrder[expoDeviceHubCallIndex]
);
const [command, args] = spawnCalls[expoDeviceHubCallIndex];
const port = Number(args[args.indexOf('--port') + 1]);
expect(port).toBeGreaterThan(0);
expect(command).toBe('npx');
expect(args).toEqual(createExpoDeviceHubArgs({ port, turnArgs, packageVersion }));
expect(ngrok.forward).toHaveBeenCalledWith(expect.objectContaining({ addr: port }));
expect(preview.previewUrl).toBe('https://android-preview.example.test');
await preview.stopAsync();
expect(close).toHaveBeenCalledTimes(1);
});
it('carries the serve-sim session token for Darwin', async () => {
jest.mocked(ngrok.forward).mockResolvedValue({
url: () => 'https://preview.example.test',
close: jest.fn().mockResolvedValue(undefined),
} as never);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
});
expect(preview.previewToken).toBe('tok-1');
expect(preview.previewUrl).toBe('https://preview.example.test');
});
// serve-sim is always launched with --require-token, so a missing token means it is running
// ungated on a public tunnel. Failing beats handing out a preview that is dead or unprotected.
it('fails for Darwin when serve-sim reports no token', async () => {
jest
.mocked(readServeSimServersAsync)
.mockResolvedValue([{ udid: 'device-id', url: 'http://127.0.0.1:1' }]);
await expect(
startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
})
).rejects.toThrow(/wrote no session token/);
});
// expo-device-hub mints no token, so the Android preview must not require one.
it('starts for Linux without a token, and does not gate expo-device-hub', async () => {
jest.mocked(readServeSimServersAsync).mockResolvedValue([]);
jest.mocked(ngrok.forward).mockResolvedValue({
url: () => 'https://android-preview.example.test',
close: jest.fn().mockResolvedValue(undefined),
} as never);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.LINUX,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
});
expect(preview.previewToken).toBeUndefined();
const [, args] = jest.mocked(spawn).mock.calls[0];
expect(args).not.toContain('--require-token');
});
it('starts serve-sim for Darwin with its metrics policy and cleans up the preview resources', async () => {
const packageVersion = '4.5.6';
const close = jest.fn().mockResolvedValue(undefined);
jest.mocked(ngrok.forward).mockResolvedValue({
url: () => 'https://ios-preview.example.test',
close,
} as never);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
packageVersion,
});
const [command, args] = jest.mocked(spawn).mock.calls[0];
const port = Number(args[args.indexOf('--port') + 1]);
expect(port).toBeGreaterThan(0);
expect(command).toBe('npx');
expect(args).toEqual(createServeSimArgs({ port, turnArgs, metricsCorsArgs, packageVersion }));
expect(ngrok.forward).toHaveBeenCalledWith(expect.objectContaining({ addr: port }));
expect(preview.previewUrl).toBe('https://ios-preview.example.test');
await preview.stopAsync();
expect(close).toHaveBeenCalledTimes(1);
});
it('does not install ffmpeg before starting expo-device-hub outside Linux', async () => {
const close = jest.fn().mockResolvedValue(undefined);
jest.mocked(ngrok.forward).mockResolvedValue({
url: () => 'https://android-preview.example.test',
close,
} as never);
const preview = await startExpoDeviceHubWithTunnelAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
});
expect(jest.mocked(spawn).mock.calls[0][0]).toBe('npx');
expect(jest.mocked(spawn)).not.toHaveBeenCalledWith(
'ffmpeg',
expect.anything(),
expect.anything()
);
await preview.stopAsync();
expect(close).toHaveBeenCalledTimes(1);
});
});
describe(turnIceServersToWebPreviewArgs, () => {
it('returns no args for an empty ICE server list', () => {
expect(turnIceServersToServeSimArgs([])).toEqual([]);
expect(turnIceServersToWebPreviewArgs([])).toEqual([]);
});
it('builds --stun-url and --turn-url flags from Cloudflare ICE servers', () => {
expect(
turnIceServersToServeSimArgs([
turnIceServersToWebPreviewArgs([
{ urls: ['stun:stun.cloudflare.com:3478', 'stun:stun.cloudflare.com:53'] },
{
urls: [
@@ -279,7 +572,7 @@ describe(turnIceServersToServeSimArgs, () => {
it('emits only --turn-url flags when no credential-less (STUN) entry is present', () => {
expect(
turnIceServersToServeSimArgs([
turnIceServersToWebPreviewArgs([
{
urls: ['turns:turn.cloudflare.com:443?transport=tcp'],
username: 'u',
@@ -297,19 +590,19 @@ describe(turnIceServersToServeSimArgs, () => {
});
it('emits only --stun-url when there is no credentialed TURN entry', () => {
expect(turnIceServersToServeSimArgs([{ urls: ['stun:stun.cloudflare.com:3478'] }])).toEqual([
expect(turnIceServersToWebPreviewArgs([{ urls: ['stun:stun.cloudflare.com:3478'] }])).toEqual([
'--stun-url',
'stun:stun.cloudflare.com:3478',
]);
});
});
describe(fetchServeSimTurnArgsAsync, () => {
describe(fetchWebPreviewTurnArgsAsync, () => {
beforeEach(() => {
jest.mocked(turtleFetch).mockReset();
});
it('requests TURN ICE servers from the device run session endpoint and returns serve-sim args', async () => {
it('requests TURN ICE servers from the device run session endpoint and returns web preview args', async () => {
jest.mocked(turtleFetch).mockResolvedValue({
json: async () => ({
data: {
@@ -325,7 +618,7 @@ describe(fetchServeSimTurnArgsAsync, () => {
}),
} as unknown as Awaited<ReturnType<typeof turtleFetch>>);
const args = await fetchServeSimTurnArgsAsync(createCtxMock(), {
const args = await fetchWebPreviewTurnArgsAsync(createCtxMock(), {
env: createEnvMock(),
logger: createLoggerMock(),
});
@@ -349,11 +642,11 @@ describe(fetchServeSimTurnArgsAsync, () => {
);
});
it('returns [] and warns when the request fails so serve-sim falls back to P2P/STUN', async () => {
it('returns [] and warns when the request fails so the web preview falls back to P2P/STUN', async () => {
jest.mocked(turtleFetch).mockRejectedValue(new Error('boom'));
const logger = createLoggerMock();
const args = await fetchServeSimTurnArgsAsync(createCtxMock(), {
const args = await fetchWebPreviewTurnArgsAsync(createCtxMock(), {
env: createEnvMock(),
logger,
});
@@ -614,7 +907,7 @@ describe(waitForDeviceRunSessionStoppedAsync, () => {
});
});
describe(ensureFfmpegInstalledAsync, () => {
describe(ensureFfmpegInstalledOnceAsync, () => {
const spawnMock = jest.mocked(spawn);
function spawnResolved(): ReturnType<typeof spawn> {
@@ -633,7 +926,7 @@ describe(ensureFfmpegInstalledAsync, () => {
it('does not install when ffmpeg is on PATH', async () => {
spawnMock.mockReturnValueOnce(spawnResolved());
await ensureFfmpegInstalledAsync({
await ensureFfmpegInstalledOnceAsync({
runtimePlatform: BuildRuntimePlatform.DARWIN,
env: createEnvMock(),
logger: createLoggerMock(),
@@ -646,7 +939,7 @@ describe(ensureFfmpegInstalledAsync, () => {
it('installs ffmpeg with Homebrew on darwin when it is missing', async () => {
spawnMock.mockReturnValueOnce(spawnRejected()).mockReturnValueOnce(spawnResolved());
await ensureFfmpegInstalledAsync({
await ensureFfmpegInstalledOnceAsync({
runtimePlatform: BuildRuntimePlatform.DARWIN,
env: createEnvMock(),
logger: createLoggerMock(),
@@ -667,7 +960,7 @@ describe(ensureFfmpegInstalledAsync, () => {
.mockReturnValueOnce(spawnResolved()) // apt-get update
.mockReturnValueOnce(spawnResolved()); // apt-get install
await ensureFfmpegInstalledAsync({
await ensureFfmpegInstalledOnceAsync({
runtimePlatform: BuildRuntimePlatform.LINUX,
env: createEnvMock(),
logger: createLoggerMock(),
@@ -689,6 +982,33 @@ describe(ensureFfmpegInstalledAsync, () => {
);
});
it('shares an in-flight ffmpeg setup between callers', async () => {
let finishInstall: (() => void) | undefined;
const pendingInstall = new Promise<void>(resolve => {
finishInstall = resolve;
});
spawnMock
.mockReturnValueOnce(spawnRejected()) // ffmpeg -version
.mockReturnValueOnce(spawnResolved()) // apt-get update
.mockReturnValueOnce(pendingInstall as unknown as ReturnType<typeof spawn>); // apt-get install
const options = {
runtimePlatform: BuildRuntimePlatform.LINUX,
env: createEnvMock(),
logger: createLoggerMock(),
};
const firstSetup = ensureFfmpegInstalledOnceAsync(options);
await new Promise<void>(resolve => setImmediate(resolve));
expect(spawnMock).toHaveBeenCalledTimes(3);
const secondSetup = ensureFfmpegInstalledOnceAsync(options);
expect(spawnMock).toHaveBeenCalledTimes(3);
finishInstall?.();
await Promise.all([firstSetup, secondSetup]);
expect(spawnMock).toHaveBeenCalledTimes(3);
});
it('still installs on linux when the apt index refresh fails', async () => {
spawnMock
.mockReturnValueOnce(spawnRejected()) // ffmpeg -version
@@ -696,7 +1016,7 @@ describe(ensureFfmpegInstalledAsync, () => {
.mockReturnValueOnce(spawnResolved()); // apt-get install
const logger = createLoggerMock();
await ensureFfmpegInstalledAsync({
await ensureFfmpegInstalledOnceAsync({
runtimePlatform: BuildRuntimePlatform.LINUX,
env: createEnvMock(),
logger,
@@ -715,7 +1035,7 @@ describe(ensureFfmpegInstalledAsync, () => {
const logger = createLoggerMock();
await expect(
ensureFfmpegInstalledAsync({
ensureFfmpegInstalledOnceAsync({
runtimePlatform: BuildRuntimePlatform.DARWIN,
env: createEnvMock(),
logger,
@@ -736,7 +1056,7 @@ describe(ensureFfmpegInstalledAsync, () => {
const logger = createLoggerMock();
await expect(
ensureFfmpegInstalledAsync({
ensureFfmpegInstalledOnceAsync({
runtimePlatform: BuildRuntimePlatform.DARWIN,
env: createEnvMock(),
logger,
@@ -0,0 +1,203 @@
import { SystemError } from '@expo/eas-build-job';
import Log from '@expo/logger';
import http from 'node:http';
import net, { AddressInfo } from 'node:net';
import WebSocket, { WebSocketServer } from 'ws';
import { startSandboxDaemonAsync } from '../sandboxDaemon';
jest.unmock('@expo/logger');
describe(startSandboxDaemonAsync.name, () => {
const logger = Log.child({ buildStepId: 'sandbox-test' });
it.each(['ftp://localhost', 'not a URL'])('reports invalid server URL %s', async serverUrl => {
const errorLog = jest.spyOn(logger, 'error').mockImplementation(() => {});
const daemon = await startSandboxDaemonAsync({
credential: 'secret-token',
serverUrl,
reconnectDelayMs: 10,
logger,
});
try {
await expect(daemon.ready).rejects.toBeInstanceOf(SystemError);
expect(errorLog).toHaveBeenCalledWith(
{ err: expect.objectContaining({ message: expect.any(String) }) },
expect.any(String)
);
} finally {
await daemon.stopAsync();
errorLog.mockRestore();
}
});
it('logs errors after the connection opens', async () => {
const server = new WebSocketServer({ port: 0, host: '127.0.0.1' });
await new Promise<void>(resolve => server.once('listening', resolve));
const connection = new Promise<WebSocket>(resolve => server.once('connection', resolve));
const warn = jest.spyOn(logger, 'warn').mockImplementation(() => {});
const daemon = await startSandboxDaemonAsync({
credential: 'secret-token',
serverUrl: `ws://127.0.0.1:${(server.address() as AddressInfo).port}`,
reconnectDelayMs: 60_000,
logger,
});
try {
const socket = await connection;
await daemon.ready;
// An invalid opcode makes the client emit a protocol error.
const closed = new Promise<void>(resolve => socket.once('close', () => resolve()));
(socket as any)._socket.write(Buffer.from([0x83, 0x00]));
await closed;
expect(warn).toHaveBeenCalledWith(
{ err: expect.objectContaining({ message: expect.stringContaining('invalid opcode') }) },
expect.stringContaining('invalid opcode')
);
} finally {
await daemon.stopAsync();
await new Promise<void>(resolve => server.close(() => resolve()));
warn.mockRestore();
}
});
it('fails if the server does not complete the handshake', async () => {
const sockets = new Set<net.Socket>();
const tcpServer = net.createServer(socket => {
sockets.add(socket);
socket.on('close', () => sockets.delete(socket));
});
await new Promise<void>(resolve => tcpServer.listen(0, '127.0.0.1', resolve));
const address = tcpServer.address() as AddressInfo;
const errorLog = jest.spyOn(logger, 'error').mockImplementation(() => {});
const daemon = await startSandboxDaemonAsync({
credential: 'secret-token',
serverUrl: `ws://127.0.0.1:${address.port}`,
reconnectDelayMs: 10,
logger,
});
try {
await expect(daemon.ready).rejects.toBeInstanceOf(SystemError);
expect(errorLog).toHaveBeenCalledWith(
{ err: expect.objectContaining({ message: 'Opening handshake has timed out' }) },
expect.stringContaining('Opening handshake has timed out')
);
} finally {
await daemon.stopAsync();
for (const socket of sockets) {
socket.destroy();
}
await new Promise<void>(resolve => tcpServer.close(() => resolve()));
errorLog.mockRestore();
}
}, 15_000);
it.each(['stop', 'abort'])('can %s while the initial connection is pending', async action => {
const controller = new AbortController();
const tcpServer = net.createServer();
await new Promise<void>(resolve => tcpServer.listen(0, '127.0.0.1', resolve));
const address = tcpServer.address() as AddressInfo;
const daemon = await startSandboxDaemonAsync({
credential: 'secret-token',
serverUrl: `ws://127.0.0.1:${address.port}`,
reconnectDelayMs: 60_000,
logger,
signal: controller.signal,
});
const connectionError = daemon.ready.catch(error => error);
if (action === 'abort') {
controller.abort();
} else {
await daemon.stopAsync();
}
await expect(connectionError).resolves.toBeInstanceOf(Error);
await daemon.stopAsync();
await new Promise<void>(resolve => tcpServer.close(() => resolve()));
});
it('fails when the initial connection cannot be opened', async () => {
const errorLog = jest.spyOn(logger, 'error').mockImplementation(() => {});
const httpServer = http.createServer();
await new Promise<void>(resolve => httpServer.listen(0, '127.0.0.1', resolve));
const address = httpServer.address() as AddressInfo;
await new Promise<void>(resolve => httpServer.close(() => resolve()));
const daemon = await startSandboxDaemonAsync({
credential: 'secret-token',
serverUrl: `ws://127.0.0.1:${address.port}`,
reconnectDelayMs: 60_000,
logger,
});
await expect(daemon.ready).rejects.toThrow('Sandbox MCP server connection failed');
await expect(daemon.ready).rejects.toBeInstanceOf(SystemError);
expect(errorLog).toHaveBeenCalledWith(
{ err: expect.objectContaining({ message: expect.stringContaining('ECONNREFUSED') }) },
expect.stringContaining('ECONNREFUSED')
);
await daemon.stopAsync();
errorLog.mockRestore();
});
it('connects with the credential and closes on stop', async () => {
const httpServer = http.createServer();
const mcpServer = new WebSocketServer({ noServer: true });
httpServer.on('upgrade', (request, socket, head) => {
expect(request.url).toBe('/sandbox/connect');
expect(request.headers.authorization).toBe('Bearer secret-token');
mcpServer.handleUpgrade(request, socket, head, client =>
mcpServer.emit('connection', client)
);
});
await new Promise<void>(resolve => httpServer.listen(0, '127.0.0.1', resolve));
const address = httpServer.address() as AddressInfo;
const connection = new Promise<WebSocket>(resolve => mcpServer.once('connection', resolve));
const daemonPromise = startSandboxDaemonAsync({
credential: 'secret-token',
serverUrl: `ws://127.0.0.1:${address.port}`,
reconnectDelayMs: 10,
logger,
});
const socket = await connection;
const daemon = await daemonPromise;
await daemon.ready;
const socketClosed = new Promise<void>(resolve => socket.once('close', () => resolve()));
await daemon.stopAsync();
await socketClosed;
mcpServer.close();
await new Promise<void>(resolve => httpServer.close(() => resolve()));
});
it('cancels the reconnect delay when stopped', async () => {
const httpServer = http.createServer();
const mcpServer = new WebSocketServer({ noServer: true });
httpServer.on('upgrade', (request, socket, head) => {
mcpServer.handleUpgrade(request, socket, head, client =>
mcpServer.emit('connection', client)
);
});
await new Promise<void>(resolve => httpServer.listen(0, '127.0.0.1', resolve));
const address = httpServer.address() as AddressInfo;
const connection = new Promise<WebSocket>(resolve => mcpServer.once('connection', resolve));
const daemonPromise = startSandboxDaemonAsync({
credential: 'secret-token',
serverUrl: `ws://127.0.0.1:${address.port}`,
reconnectDelayMs: 60_000,
logger,
});
const socket = await connection;
const daemon = await daemonPromise;
await daemon.ready;
const socketClosed = new Promise<void>(resolve => socket.once('close', () => resolve()));
socket.close();
await socketClosed;
await new Promise<void>(resolve => setImmediate(resolve));
await daemon.stopAsync();
mcpServer.close();
await new Promise<void>(resolve => httpServer.close(() => resolve()));
});
});
@@ -101,6 +101,14 @@ describe(readServeSimServersAsync, () => {
]);
});
it('includes the session token when the state file carries one', async () => {
await writeServerStateAsync('E', { device: 'E', url: 'http://127.0.0.1:5', token: 'tok-abc' });
expect(await readServeSimServersAsync(stateDir)).toEqual([
{ udid: 'E', url: 'http://127.0.0.1:5', token: 'tok-abc' },
]);
});
it('returns [] when the state dir does not exist', async () => {
expect(await readServeSimServersAsync(path.join(workDir, 'nope'))).toEqual([]);
});
@@ -120,6 +128,20 @@ describe(streamServeSimMetricsToFileAsync, () => {
expect(String(jest.mocked(fetch).mock.calls[0][0])).toBe('https://sim.example/metrics');
});
it('sends the session token as a bearer header when given one', async () => {
jest.mocked(fetch).mockResolvedValue(sseResponse());
await streamServeSimMetricsToFileAsync({
serveSimUrl: 'https://sim.example',
serveSimToken: 'tok-abc',
filePath: path.join(workDir, 'auth.ndjson'),
signal: new AbortController().signal,
logger: createLoggerMock(),
});
expect(jest.mocked(fetch).mock.calls[0][1]?.headers).toMatchObject({
Authorization: 'Bearer tok-abc',
});
});
it('writes nothing on a non-ok response', async () => {
jest.mocked(fetch).mockResolvedValue(new Response('nope', { status: 502 }));
const filePath = path.join(workDir, 'empty.ndjson');
@@ -0,0 +1,421 @@
import { type AppiumCommand } from './appiumCommands.generated';
export const APPIUM_COMMAND_SUMMARIES: Record<AppiumCommand, string> = {
// Session lifecycle
createSession: 'Started the session',
deleteSession: 'Ended the session',
getSession: 'Read the session details',
getStatus: 'Checked the server status',
getAppiumSessions: 'Listed Appium sessions',
getAppiumSessionCapabilities: 'Read the session capabilities',
getTimeouts: 'Read timeouts',
timeouts: 'Set timeouts',
getSettings: 'Read Appium settings',
updateSettings: 'Updated Appium settings',
listCommands: 'Listed available commands',
listExtensions: 'Listed available extensions',
receiveAsyncResponse: 'Received an async response',
// Screen and media
getScreenshot: 'Took a screenshot',
getElementScreenshot: 'Took a screenshot of an element',
getPageSource: 'Read the screen contents',
printPage: 'Printed the page',
// Window and orientation
getWindowRect: 'Read the screen size',
setWindowRect: 'Resized the window',
maximizeWindow: 'Maximized the window',
minimizeWindow: 'Minimized the window',
fullScreenWindow: 'Made the window full screen',
getWindowHandle: 'Read the current window',
getWindowHandles: 'Listed open windows',
setWindow: 'Switched window',
closeWindow: 'Closed the window',
createNewWindow: 'Opened a new window',
setFrame: 'Switched to a frame',
switchToParentFrame: 'Switched to the parent frame',
getOrientation: 'Read the screen orientation',
setOrientation: 'Changed the screen orientation',
getRotation: 'Read the device rotation',
setRotation: 'Rotated the device',
// Device
getDeviceTime: 'Read the device time',
getGeoLocation: 'Read the device location',
setGeoLocation: 'Set the device location',
getNetworkConnection: 'Read the network connection',
setNetworkConnection: 'Changed the network connection',
setDevicePosture: 'Set the device posture',
clearDevicePosture: 'Cleared the device posture',
setPermissions: 'Set app permissions',
// Element discovery
findElement: 'Found an element',
findElements: 'Found elements',
findElementFromElement: 'Found a nested element',
findElementsFromElement: 'Found nested elements',
findElementFromShadowRoot: 'Found an element in a shadow root',
findElementsFromShadowRoot: 'Found elements in a shadow root',
elementShadowRoot: 'Read an element shadow root',
active: 'Read the focused element',
// Element interaction
click: 'Tapped an element',
clear: 'Cleared an element',
setValue: 'Typed into an element',
getText: 'Read element text',
getName: 'Read an element tag',
getAttribute: 'Read an element attribute',
getProperty: 'Read an element property',
getCssProperty: 'Read an element style',
getComputedLabel: 'Read an element label',
getComputedRole: 'Read an element role',
getElementRect: 'Read an element position and size',
elementDisplayed: 'Checked if an element was visible',
elementEnabled: 'Checked if an element was enabled',
elementSelected: 'Checked if an element was selected',
// Gestures
performActions: 'Performed a gesture',
releaseActions: 'Finished a gesture',
// App management
installApp: 'Installed an app',
removeApp: 'Removed an app',
isAppInstalled: 'Checked if an app was installed',
activateApp: 'Activated an app',
terminateApp: 'Terminated an app',
queryAppState: 'Read the app state',
background: 'Sent the app to the background',
// Contexts
getContexts: 'Listed available contexts',
getCurrentContext: 'Read the current context',
setContext: 'Switched context',
// Web navigation
setUrl: 'Opened a URL',
getUrl: 'Read the current URL',
back: 'Navigated back',
forward: 'Navigated forward',
refresh: 'Refreshed the page',
title: 'Read the page title',
// Cookies
getCookie: 'Read a cookie',
getCookies: 'Read all cookies',
setCookie: 'Set a cookie',
deleteCookie: 'Deleted a cookie',
deleteCookies: 'Deleted all cookies',
// Alerts
getAlertText: 'Read an alert',
setAlertText: 'Typed into an alert',
postAcceptAlert: 'Accepted an alert',
postDismissAlert: 'Dismissed an alert',
// Keyboard and input methods
hideKeyboard: 'Hid the keyboard',
isKeyboardShown: 'Checked if the keyboard was shown',
activateIMEEngine: 'Activated an input method',
deactivateIMEEngine: 'Deactivated the input method',
getActiveIMEEngine: 'Read the active input method',
availableIMEEngines: 'Listed input methods',
isIMEActivated: 'Checked if an input method was active',
// Clipboard (driver-level, not declared by base-driver)
getClipboard: 'Read the clipboard',
setClipboard: 'Wrote to the clipboard',
// Files
pushFile: 'Pushed a file to the device',
pullFile: 'Pulled a file from the device',
pullFolder: 'Pulled a folder from the device',
// Logs
getLog: 'Read device logs',
getLogTypes: 'Listed log types',
getLogEvents: 'Read logged events',
logCustomEvent: 'Logged a custom event',
generateTestReport: 'Generated a test report',
// Scripts
execute: 'Ran a script command',
executeAsync: 'Ran an async script command',
executeCdp: 'Ran a Chrome DevTools command',
// Web authentication (virtual authenticators)
addVirtualAuthenticator: 'Added a virtual authenticator',
removeVirtualAuthenticator: 'Removed a virtual authenticator',
addAuthCredential: 'Added an auth credential',
getAuthCredential: 'Read auth credentials',
removeAuthCredential: 'Removed an auth credential',
removeAllAuthCredentials: 'Removed all auth credentials',
setUserAuthVerified: 'Set the user verification state',
// Federated sign-in (FedCM)
fedCMGetAccounts: 'Listed federated sign-in accounts',
fedCMSelectAccount: 'Selected a federated sign-in account',
fedCMGetDialogType: 'Read the federated sign-in dialog type',
fedCMGetTitle: 'Read the federated sign-in dialog title',
fedCMClickDialogButton: 'Clicked a federated sign-in dialog button',
fedCMCancelDialog: 'Canceled the federated sign-in dialog',
fedCMResetCooldown: 'Reset the federated sign-in cooldown',
fedCMSetDelayEnabled: 'Toggled the federated sign-in delay',
// Virtual sensors and pressure sources
createVirtualSensor: 'Created a virtual sensor',
updateVirtualSensorReading: 'Updated a virtual sensor reading',
getVirtualSensorInfo: 'Read virtual sensor info',
deleteVirtualSensor: 'Removed a virtual sensor',
createVirtualPressureSource: 'Created a virtual pressure source',
updateVirtualPressureSource: 'Updated a virtual pressure source',
deleteVirtualPressureSource: 'Removed a virtual pressure source',
// Privacy / experimental web platform
getGlobalPrivacyControl: 'Read the Global Privacy Control setting',
setGlobalPrivacyControl: 'Set the Global Privacy Control setting',
setStorageAccess: 'Set storage access',
setRPHRegistrationMode: 'Set the protocol handler registration mode',
setSPCTransactionMode: 'Set the payment transaction mode',
// Driver routes and compatibility commands
asyncScriptTimeout: 'Set the async script timeout',
closeApp: 'Closed the app',
disableConditionInducer: 'Disabled the condition inducer',
enableConditionInducer: 'Enabled the condition inducer',
fingerprint: 'Simulated a fingerprint',
getCurrentActivity: 'Read the current activity',
getCurrentPackage: 'Read the current package',
getDisplayDensity: 'Read the display density',
getLocation: 'Read an element location',
getLocationInView: 'Read an element location in view',
getPerformanceData: 'Read performance data',
getPerformanceDataTypes: 'Listed performance data types',
getScreenInfo: 'Read screen information',
getSize: 'Read an element size',
getStrings: 'Read app strings',
getSystemBars: 'Read system bar information',
getViewportRect: 'Read the viewport size',
getViewportScreenshot: 'Took a viewport screenshot',
getWindowSize: 'Read the window size',
gsmCall: 'Simulated a phone call',
gsmSignal: 'Changed the cellular signal',
gsmVoice: 'Changed the cellular voice state',
implicitWait: 'Set the element lookup timeout',
isLocationServicesEnabled: 'Checked if location services were enabled',
isLocked: 'Checked if the device was locked',
keyevent: 'Sent a key event',
keys: 'Sent keyboard input',
launchApp: 'Launched the app',
listConditionInducers: 'Listed condition inducers',
lock: 'Locked the device',
longPressKeyCode: 'Long-pressed a key',
networkSpeed: 'Changed the network speed',
openNotifications: 'Opened notifications',
powerAC: 'Changed the charging state',
powerCapacity: 'Changed the battery level',
pressKeyCode: 'Pressed a key',
replaceValue: 'Replaced an element value',
reset: 'Reset the app',
sendSMS: 'Simulated a text message',
sensorSet: 'Changed a sensor value',
setStylusHandwriting: 'Changed stylus handwriting mode',
setValueImmediate: 'Typed into an element immediately',
startActivity: 'Started an activity',
startAudioRecording: 'Started audio recording',
startRecordingScreen: 'Started screen recording',
stopAudioRecording: 'Stopped audio recording',
stopRecordingScreen: 'Stopped screen recording',
submit: 'Submitted an element',
toggleData: 'Toggled mobile data',
toggleEnrollTouchId: 'Changed Touch ID enrollment',
toggleFlightMode: 'Toggled airplane mode',
toggleLocationServices: 'Toggled location services',
toggleWiFi: 'Toggled Wi-Fi',
touchId: 'Simulated Touch ID',
unlock: 'Unlocked the device',
// Driver mobile commands
mobileAcceptAlert: 'Accepted an alert',
mobileActivateApp: 'Activated an app',
mobileBackgroundApp: 'Sent the app to the background',
mobileBluetooth: 'Changed Bluetooth state',
mobileBroadcast: 'Sent an Android broadcast',
mobileCalibrateWebToRealCoordinatesTranslation: 'Calibrated web coordinates',
mobileChangePermissions: 'Changed app permissions',
mobileClearApp: 'Cleared app data',
mobileClearKeychains: 'Cleared keychains',
mobileClickGesture: 'Performed a click gesture',
mobileConfigureLocalization: 'Changed device localization',
mobileDeepLink: 'Opened a deep link',
mobileDeleteFile: 'Deleted a device file',
mobileDeleteFolder: 'Deleted a device folder',
mobileDeviceidle: 'Changed device idle state',
mobileDisableVoiceOver: 'Disabled VoiceOver',
mobileDismissAlert: 'Dismissed an alert',
mobileDoubleClickGesture: 'Performed a double-click gesture',
mobileDoubleTap: 'Double-tapped the screen',
mobileDragFromToForDuration: 'Dragged between screen points',
mobileDragFromToWithVelocity: 'Dragged with a set velocity',
mobileDragGesture: 'Performed a drag gesture',
mobileEnableVoiceOver: 'Enabled VoiceOver',
mobileEnrollBiometric: 'Changed biometric enrollment',
mobileExecEmuConsoleCommand: 'Ran an emulator console command',
mobileExpectNotification: 'Waited for a notification',
mobileFingerprint: 'Simulated a fingerprint',
mobileFlingGesture: 'Performed a fling gesture',
mobileForcePress: 'Force-pressed the screen',
mobileGetActionHistory: 'Read scheduled action history',
mobileGetActiveAppInfo: 'Read active app information',
mobileGetAppearance: 'Read the device appearance',
mobileGetBatteryInfo: 'Read battery information',
mobileGetChromeCapabilities: 'Read Chrome capabilities',
mobileGetConnectivity: 'Read device connectivity',
mobileGetContentSize: 'Read scrollable content size',
mobileGetContexts: 'Listed available contexts',
mobileGetDeviceInfo: 'Read device information',
mobileGetDeviceTime: 'Read the device time',
mobileGetDeclaredOrientation: 'Read the declared device orientation',
mobileGetGeolocation: 'Read the device location',
mobileGetIncreaseContrast: 'Read the Increase Contrast setting',
mobileGetNotifications: 'Read notifications',
mobileGetPasteboard: 'Read the pasteboard',
mobileGetPerformanceData: 'Read app performance data',
mobileGetPermission: 'Read an app permission',
mobileGetPermissions: 'Read app permissions',
mobileGetSimulatedLocation: 'Read the simulated location',
mobileGetSource: 'Read the screen contents',
mobileGetUiMode: 'Read the Android UI mode',
mobileGetXctestScreenRecordingInfo: 'Read XCTest screen recording information',
mobileGsmCall: 'Simulated a phone call',
mobileGsmSignal: 'Changed the cellular signal',
mobileGsmVoice: 'Changed the cellular voice state',
mobileHandleAlert: 'Handled an alert',
mobileHideKeyboard: 'Hid the keyboard',
mobileInjectEmulatorCameraImage: 'Injected an emulator camera image',
mobileInstallApp: 'Installed an app',
mobileInstallCertificate: 'Installed a certificate',
mobileInstallMultipleApks: 'Installed multiple APKs',
mobileInstallXCTestBundle: 'Installed an XCTest bundle',
mobileIsAppInstalled: 'Checked if an app was installed',
mobileIsBiometricEnrolled: 'Checked biometric enrollment',
mobileIsMediaProjectionRecordingRunning: 'Checked media projection recording',
mobileIsVoiceOverEnabled: 'Checked if VoiceOver was enabled',
mobileKeys: 'Sent keyboard input',
mobileKillApp: 'Stopped an app',
mobileLaunchApp: 'Launched an app',
mobileListApps: 'Listed installed apps',
mobileListCertificates: 'Listed certificates',
mobileListDisplays: 'Listed displays',
mobileListSms: 'Listed text messages',
mobileListWindows: 'Listed app windows',
mobileListXCTestBundles: 'Listed XCTest bundles',
mobileLongClickGesture: 'Performed a long-click gesture',
mobileNetworkSpeed: 'Changed the network speed',
mobileNfc: 'Changed NFC state',
mobilePerformAccessibilityAudit: 'Ran an accessibility audit',
mobilePerformEditorAction: 'Performed a keyboard editor action',
mobilePerformHandGesture: 'Performed a hand gesture',
mobilePerformIndigoHidEvent: 'Performed an Indigo HID event',
mobilePerformIoHidEvent: 'Performed an IOHID event',
mobilePerformStatusBarCommand: 'Performed a status bar command',
mobilePinch: 'Pinched the screen',
mobilePinchCloseGesture: 'Performed a pinch-close gesture',
mobilePinchOpenGesture: 'Performed a pinch-open gesture',
mobilePowerAc: 'Changed the charging state',
mobilePowerCapacity: 'Changed the battery level',
mobilePressButton: 'Pressed a device button',
mobilePressKey: 'Pressed a key',
mobilePullFile: 'Pulled a file from the device',
mobilePullFolder: 'Pulled a folder from the device',
mobilePushFile: 'Pushed a file to the device',
mobilePushNotification: 'Sent a push notification',
mobileQueryAppState: 'Read the app state',
mobileRefreshGpsCache: 'Refreshed the GPS cache',
mobileRemoveApp: 'Removed an app',
mobileRemoveCertificate: 'Removed a certificate',
mobileReplaceElementValue: 'Replaced an element value',
mobileResetAccessibilityCache: 'Reset the accessibility cache',
mobileResetGeolocation: 'Reset the device location',
mobileResetLocationService: 'Reset location services',
mobileResetPermission: 'Reset an app permission',
mobileResetSimulatedLocation: 'Reset the simulated location',
mobileRotateDigitalCrown: 'Rotated the Digital Crown',
mobileRotateElement: 'Rotated an element',
mobileRunXCTest: 'Ran an XCTest bundle',
mobileScheduleAction: 'Scheduled a device action',
mobileScreenshots: 'Took display screenshots',
mobileScroll: 'Scrolled the screen',
mobileScrollBackTo: 'Scrolled back to an element',
mobileScrollGesture: 'Performed a scroll gesture',
mobileScrollToElement: 'Scrolled to an element',
mobileSelectPickerWheelValue: 'Changed a picker wheel value',
mobileSendBiometricMatch: 'Simulated biometric authentication',
mobileSendMemoryWarning: 'Sent a memory warning',
mobileSendSms: 'Simulated a text message',
mobileSendTrimMemory: 'Sent an Android memory trim event',
mobileSetAppearance: 'Changed the device appearance',
mobileSetConnectivity: 'Changed device connectivity',
mobileSetContentSize: 'Changed the content size',
mobileSetGeolocation: 'Set the device location',
mobileSetIncreaseContrast: 'Changed the Increase Contrast setting',
mobileSetPasteboard: 'Wrote to the pasteboard',
mobileSetPermissions: 'Set app permissions',
mobileSetSimulatedLocation: 'Set a simulated location',
mobileSetUiMode: 'Changed the Android UI mode',
mobileShake: 'Shook the device',
mobileShell: 'Ran an Android shell command',
mobileSimctl: 'Ran a simctl command',
mobileSiriCommand: 'Sent a Siri command',
mobileStartActivity: 'Started an activity',
mobileStartAutomationSession: 'Started an automation session',
mobileStartLogsBroadcast: 'Started log streaming',
mobileStartMediaProjectionRecording: 'Started media projection recording',
mobileStartNetworkMonitor: 'Started network monitoring',
mobileStartPerfRecord: 'Started performance recording',
mobileStartScreenRecording: 'Started screen recording',
mobileStartScreenStreaming: 'Started screen streaming',
mobileStartService: 'Started a service',
mobileStartSystemMonitor: 'Started system monitoring',
mobileStartXctestScreenRecording: 'Started XCTest screen recording',
mobileStopAutomationSession: 'Stopped an automation session',
mobileStopLogsBroadcast: 'Stopped log streaming',
mobileStopMediaProjectionRecording: 'Stopped media projection recording',
mobileStopNetworkMonitor: 'Stopped network monitoring',
mobileStopPerfRecord: 'Stopped performance recording',
mobileStopScreenRecording: 'Stopped screen recording',
mobileStopScreenStreaming: 'Stopped screen streaming',
mobileStopService: 'Stopped a service',
mobileStopSystemMonitor: 'Stopped system monitoring',
mobileStopXctestScreenRecording: 'Stopped XCTest screen recording',
mobileSwipe: 'Swiped the screen',
mobileSwipeGesture: 'Performed a swipe gesture',
mobileTap: 'Tapped the screen',
mobileTapWithNumberOfTaps: 'Tapped the screen multiple times',
mobileTerminateApp: 'Terminated an app',
mobileTouchAndHold: 'Touched and held the screen',
mobileTwoFingerTap: 'Tapped with two fingers',
mobileType: 'Typed text',
mobileUnlock: 'Unlocked the device',
mobileUnscheduleAction: 'Canceled a scheduled action',
mobileUpdateSafariPreferences: 'Updated Safari preferences',
mobileViewPortRect: 'Read the viewport size',
mobileViewportElementRect: 'Read an element position and size in the viewport',
mobileViewportScreenshot: 'Took a viewport screenshot',
mobileVoiceOverCurrentSpeech: 'Read current VoiceOver speech',
mobileVoiceOverMove: 'Moved the VoiceOver cursor',
};
/**
* Translate a raw Appium command name into a short, human-readable summary.
*
* If a command has no curated summary, the raw command name is returned
* unchanged — we intentionally do not guess a phrasing.
*/
export function humanizeAppiumCommand(command: string): string {
return Object.hasOwn(APPIUM_COMMAND_SUMMARIES, command)
? (APPIUM_COMMAND_SUMMARIES as Record<string, string>)[command]
: command;
}
@@ -0,0 +1,404 @@
// AUTO-GENERATED. Do not edit by hand.
// Run `yarn workspace @expo/build-tools generate-appium-commands` to refresh this
// list from Appium, XCUITest, and UiAutomator2 (installed on demand by the generator).
export type BaseDriverAppiumCommand =
| 'activateApp'
| 'activateIMEEngine'
| 'active'
| 'addAuthCredential'
| 'addVirtualAuthenticator'
| 'availableIMEEngines'
| 'back'
| 'clear'
| 'clearDevicePosture'
| 'click'
| 'closeWindow'
| 'createNewWindow'
| 'createSession'
| 'createVirtualPressureSource'
| 'createVirtualSensor'
| 'deactivateIMEEngine'
| 'deleteCookie'
| 'deleteCookies'
| 'deleteSession'
| 'deleteVirtualPressureSource'
| 'deleteVirtualSensor'
| 'elementDisplayed'
| 'elementEnabled'
| 'elementSelected'
| 'elementShadowRoot'
| 'execute'
| 'executeAsync'
| 'executeCdp'
| 'fedCMCancelDialog'
| 'fedCMClickDialogButton'
| 'fedCMGetAccounts'
| 'fedCMGetDialogType'
| 'fedCMGetTitle'
| 'fedCMResetCooldown'
| 'fedCMSelectAccount'
| 'fedCMSetDelayEnabled'
| 'findElement'
| 'findElementFromElement'
| 'findElementFromShadowRoot'
| 'findElements'
| 'findElementsFromElement'
| 'findElementsFromShadowRoot'
| 'forward'
| 'fullScreenWindow'
| 'generateTestReport'
| 'getActiveIMEEngine'
| 'getAlertText'
| 'getAppiumSessionCapabilities'
| 'getAppiumSessions'
| 'getAttribute'
| 'getAuthCredential'
| 'getComputedLabel'
| 'getComputedRole'
| 'getContexts'
| 'getCookie'
| 'getCookies'
| 'getCssProperty'
| 'getCurrentContext'
| 'getDeviceTime'
| 'getElementRect'
| 'getElementScreenshot'
| 'getGeoLocation'
| 'getGlobalPrivacyControl'
| 'getLog'
| 'getLogEvents'
| 'getLogTypes'
| 'getName'
| 'getNetworkConnection'
| 'getOrientation'
| 'getPageSource'
| 'getProperty'
| 'getRotation'
| 'getScreenshot'
| 'getSession'
| 'getSettings'
| 'getStatus'
| 'getText'
| 'getTimeouts'
| 'getUrl'
| 'getVirtualSensorInfo'
| 'getWindowHandle'
| 'getWindowHandles'
| 'getWindowRect'
| 'hideKeyboard'
| 'installApp'
| 'isAppInstalled'
| 'isIMEActivated'
| 'isKeyboardShown'
| 'listCommands'
| 'listExtensions'
| 'logCustomEvent'
| 'maximizeWindow'
| 'minimizeWindow'
| 'performActions'
| 'postAcceptAlert'
| 'postDismissAlert'
| 'printPage'
| 'pullFile'
| 'pullFolder'
| 'pushFile'
| 'queryAppState'
| 'receiveAsyncResponse'
| 'refresh'
| 'releaseActions'
| 'removeAllAuthCredentials'
| 'removeApp'
| 'removeAuthCredential'
| 'removeVirtualAuthenticator'
| 'setAlertText'
| 'setContext'
| 'setCookie'
| 'setDevicePosture'
| 'setFrame'
| 'setGeoLocation'
| 'setGlobalPrivacyControl'
| 'setNetworkConnection'
| 'setOrientation'
| 'setPermissions'
| 'setRPHRegistrationMode'
| 'setRotation'
| 'setSPCTransactionMode'
| 'setStorageAccess'
| 'setUrl'
| 'setUserAuthVerified'
| 'setValue'
| 'setWindow'
| 'setWindowRect'
| 'switchToParentFrame'
| 'terminateApp'
| 'timeouts'
| 'title'
| 'updateSettings'
| 'updateVirtualPressureSource'
| 'updateVirtualSensorReading';
export type XCUITestAppiumCommand =
| 'asyncScriptTimeout'
| 'background'
| 'closeApp'
| 'disableConditionInducer'
| 'enableConditionInducer'
| 'getClipboard'
| 'getLocation'
| 'getLocationInView'
| 'getScreenInfo'
| 'getSize'
| 'getStrings'
| 'getViewportRect'
| 'getViewportScreenshot'
| 'getWindowSize'
| 'implicitWait'
| 'isLocked'
| 'keys'
| 'launchApp'
| 'listConditionInducers'
| 'lock'
| 'mobileActivateApp'
| 'mobileCalibrateWebToRealCoordinatesTranslation'
| 'mobileClearApp'
| 'mobileClearKeychains'
| 'mobileConfigureLocalization'
| 'mobileDeepLink'
| 'mobileDeleteFile'
| 'mobileDeleteFolder'
| 'mobileDisableVoiceOver'
| 'mobileDoubleTap'
| 'mobileDragFromToForDuration'
| 'mobileDragFromToWithVelocity'
| 'mobileEnableVoiceOver'
| 'mobileEnrollBiometric'
| 'mobileExpectNotification'
| 'mobileForcePress'
| 'mobileGetActiveAppInfo'
| 'mobileGetAppearance'
| 'mobileGetBatteryInfo'
| 'mobileGetContentSize'
| 'mobileGetContexts'
| 'mobileGetDeviceInfo'
| 'mobileGetDeviceTime'
| 'mobileGetIncreaseContrast'
| 'mobileGetPasteboard'
| 'mobileGetPermission'
| 'mobileGetSimulatedLocation'
| 'mobileGetSource'
| 'mobileGetXctestScreenRecordingInfo'
| 'mobileHandleAlert'
| 'mobileHideKeyboard'
| 'mobileInstallApp'
| 'mobileInstallCertificate'
| 'mobileInstallXCTestBundle'
| 'mobileIsAppInstalled'
| 'mobileIsBiometricEnrolled'
| 'mobileIsVoiceOverEnabled'
| 'mobileKeys'
| 'mobileKillApp'
| 'mobileLaunchApp'
| 'mobileListApps'
| 'mobileListCertificates'
| 'mobileListXCTestBundles'
| 'mobilePerformAccessibilityAudit'
| 'mobilePerformHandGesture'
| 'mobilePerformIndigoHidEvent'
| 'mobilePerformIoHidEvent'
| 'mobilePinch'
| 'mobilePressButton'
| 'mobilePullFile'
| 'mobilePullFolder'
| 'mobilePushFile'
| 'mobilePushNotification'
| 'mobileQueryAppState'
| 'mobileRemoveApp'
| 'mobileRemoveCertificate'
| 'mobileResetLocationService'
| 'mobileResetPermission'
| 'mobileResetSimulatedLocation'
| 'mobileRotateDigitalCrown'
| 'mobileRotateElement'
| 'mobileRunXCTest'
| 'mobileScroll'
| 'mobileScrollToElement'
| 'mobileSelectPickerWheelValue'
| 'mobileSendBiometricMatch'
| 'mobileSendMemoryWarning'
| 'mobileSetAppearance'
| 'mobileSetContentSize'
| 'mobileSetIncreaseContrast'
| 'mobileSetPasteboard'
| 'mobileSetPermissions'
| 'mobileSetSimulatedLocation'
| 'mobileShake'
| 'mobileSimctl'
| 'mobileSiriCommand'
| 'mobileStartAutomationSession'
| 'mobileStartLogsBroadcast'
| 'mobileStartNetworkMonitor'
| 'mobileStartPerfRecord'
| 'mobileStartScreenRecording'
| 'mobileStartSystemMonitor'
| 'mobileStartXctestScreenRecording'
| 'mobileStopAutomationSession'
| 'mobileStopLogsBroadcast'
| 'mobileStopNetworkMonitor'
| 'mobileStopPerfRecord'
| 'mobileStopScreenRecording'
| 'mobileStopSystemMonitor'
| 'mobileStopXctestScreenRecording'
| 'mobileSwipe'
| 'mobileTap'
| 'mobileTapWithNumberOfTaps'
| 'mobileTerminateApp'
| 'mobileTouchAndHold'
| 'mobileTwoFingerTap'
| 'mobileUpdateSafariPreferences'
| 'mobileVoiceOverCurrentSpeech'
| 'mobileVoiceOverMove'
| 'reset'
| 'setClipboard'
| 'setValueImmediate'
| 'startAudioRecording'
| 'startRecordingScreen'
| 'stopAudioRecording'
| 'stopRecordingScreen'
| 'submit'
| 'toggleEnrollTouchId'
| 'touchId'
| 'unlock';
export type UiAutomator2AppiumCommand =
| 'background'
| 'fingerprint'
| 'getClipboard'
| 'getCurrentActivity'
| 'getCurrentPackage'
| 'getDisplayDensity'
| 'getLocation'
| 'getLocationInView'
| 'getPerformanceData'
| 'getPerformanceDataTypes'
| 'getSize'
| 'getStrings'
| 'getSystemBars'
| 'getWindowSize'
| 'gsmCall'
| 'gsmSignal'
| 'gsmVoice'
| 'implicitWait'
| 'isLocationServicesEnabled'
| 'isLocked'
| 'keyevent'
| 'keys'
| 'lock'
| 'longPressKeyCode'
| 'mobileAcceptAlert'
| 'mobileBackgroundApp'
| 'mobileBluetooth'
| 'mobileBroadcast'
| 'mobileChangePermissions'
| 'mobileClearApp'
| 'mobileClickGesture'
| 'mobileDeepLink'
| 'mobileDeleteFile'
| 'mobileDeviceidle'
| 'mobileDismissAlert'
| 'mobileDoubleClickGesture'
| 'mobileDragGesture'
| 'mobileExecEmuConsoleCommand'
| 'mobileFingerprint'
| 'mobileFlingGesture'
| 'mobileGetActionHistory'
| 'mobileGetBatteryInfo'
| 'mobileGetChromeCapabilities'
| 'mobileGetConnectivity'
| 'mobileGetContexts'
| 'mobileGetDeclaredOrientation'
| 'mobileGetDeviceInfo'
| 'mobileGetDeviceTime'
| 'mobileGetGeolocation'
| 'mobileGetNotifications'
| 'mobileGetPerformanceData'
| 'mobileGetPermissions'
| 'mobileGetUiMode'
| 'mobileGsmCall'
| 'mobileGsmSignal'
| 'mobileGsmVoice'
| 'mobileInjectEmulatorCameraImage'
| 'mobileInstallApp'
| 'mobileInstallMultipleApks'
| 'mobileIsAppInstalled'
| 'mobileIsMediaProjectionRecordingRunning'
| 'mobileListApps'
| 'mobileListDisplays'
| 'mobileListSms'
| 'mobileListWindows'
| 'mobileLongClickGesture'
| 'mobileNetworkSpeed'
| 'mobileNfc'
| 'mobilePerformEditorAction'
| 'mobilePerformStatusBarCommand'
| 'mobilePinchCloseGesture'
| 'mobilePinchOpenGesture'
| 'mobilePowerAc'
| 'mobilePowerCapacity'
| 'mobilePressKey'
| 'mobileRefreshGpsCache'
| 'mobileRemoveApp'
| 'mobileReplaceElementValue'
| 'mobileResetAccessibilityCache'
| 'mobileResetGeolocation'
| 'mobileScheduleAction'
| 'mobileScreenshots'
| 'mobileScroll'
| 'mobileScrollBackTo'
| 'mobileScrollGesture'
| 'mobileSendSms'
| 'mobileSendTrimMemory'
| 'mobileSetConnectivity'
| 'mobileSetGeolocation'
| 'mobileSetUiMode'
| 'mobileShell'
| 'mobileStartActivity'
| 'mobileStartLogsBroadcast'
| 'mobileStartMediaProjectionRecording'
| 'mobileStartScreenStreaming'
| 'mobileStartService'
| 'mobileStopLogsBroadcast'
| 'mobileStopMediaProjectionRecording'
| 'mobileStopScreenStreaming'
| 'mobileStopService'
| 'mobileSwipeGesture'
| 'mobileTerminateApp'
| 'mobileType'
| 'mobileUnlock'
| 'mobileUnscheduleAction'
| 'mobileViewPortRect'
| 'mobileViewportElementRect'
| 'mobileViewportScreenshot'
| 'networkSpeed'
| 'openNotifications'
| 'powerAC'
| 'powerCapacity'
| 'pressKeyCode'
| 'replaceValue'
| 'sendSMS'
| 'sensorSet'
| 'setClipboard'
| 'setStylusHandwriting'
| 'setValueImmediate'
| 'startActivity'
| 'startRecordingScreen'
| 'stopRecordingScreen'
| 'toggleData'
| 'toggleFlightMode'
| 'toggleLocationServices'
| 'toggleWiFi'
| 'unlock';
export type AppiumCommand =
| BaseDriverAppiumCommand
| XCUITestAppiumCommand
| UiAutomator2AppiumCommand;
@@ -7,6 +7,7 @@ import { z } from 'zod';
import { type CustomBuildContext } from '../../customBuildContext';
import { turtleFetch } from '../../utils/turtleFetch';
import { humanizeAppiumCommand } from './appiumCommandSummary';
import {
type DeviceRunSessionEventCollection,
type DeviceRunSessionEventSource,
@@ -146,7 +147,7 @@ function createAppiumEventSource(eventFile: string): DeviceRunSessionEventSource
type: 'operation.completed',
operationId,
durationMs: Math.max(0, command.endTime - command.startTime),
summary: command.cmd,
summary: humanizeAppiumCommand(command.cmd),
data: {
command: command.cmd,
appiumSessionId: command.appiumSessionId,
@@ -18,11 +18,11 @@ const ARGENT_ARTIFACT_UPLOAD_POLL_INTERVAL_MS = 5_000;
const ARGENT_ARTIFACT_UPLOAD_CLEANUP_TIMEOUT_MS = 30_000;
const ARGENT_ARTIFACT_FETCH_TIMEOUT_MS = 10_000;
// Kinds the EAS dashboard groups device run session artifacts by. Only media types the dashboard
// renders specially are mapped; anything else stays unclassified and lands in its "Other" group.
// Note that these kinds only affect grouping and labelling. The dashboard gates its inline video
// player on the `__eas_screen_recording` metadata flag, which Argent artifacts deliberately do not
// set, so a `screen-recording` kind here does not add a player to the session page.
// Fallback kinds for Argent versions that do not include a semantic artifact kind. Only media types
// the EAS dashboard renders specially can be inferred safely. Note that these kinds only affect
// grouping and labelling. The dashboard gates its inline video player on the
// `__eas_screen_recording` metadata flag, which Argent artifacts deliberately do not set, so a
// `screen-recording` kind here does not add a player to the session page.
const ARGENT_ARTIFACT_KIND_BY_MIME_TYPE = new Map<string, string>([
['image/png', 'screenshot'],
['image/jpeg', 'screenshot'],
@@ -32,6 +32,9 @@ const ARGENT_ARTIFACT_KIND_BY_MIME_TYPE = new Map<string, string>([
const ArgentArtifactSchema = z.object({
id: z.string(),
// Optional for compatibility with older Argent tool servers. Keep this open to new kinds so a
// newer Argent version can add a category without requiring an EAS worker release first.
kind: z.string().optional(),
filename: z.string(),
mimeType: z.string(),
isDirectory: z.boolean().optional(),
@@ -43,8 +46,12 @@ const ArgentArtifactsListResponseSchema = z.object({
type ArgentArtifact = z.infer<typeof ArgentArtifactSchema>;
function getArgentArtifactKind(artifact: ArgentArtifact): string | undefined {
if (artifact.kind) {
return artifact.kind;
}
// Directories are repackaged as a tarball before upload, so the reported media type describes the
// contents rather than the file we actually store.
// contents rather than the file we actually store. A semantic kind above remains valid because it
// describes what the artifact represents, not its transport format.
if (artifact.isDirectory) {
return undefined;
}
@@ -272,6 +272,10 @@ export type AscApiClientPostApi = {
};
};
// https://developer.apple.com/documentation/appstoreconnectapi/platform
export type AscPlatform =
AscApiClientPostApi['/v1/buildUploads']['request']['data']['attributes']['platform'];
export type AscApiClientPatchApi = {
[Path in keyof typeof PatchApi]: {
request: z.input<(typeof PatchApi)[Path]['request']>;
@@ -5,9 +5,44 @@ import {
AscApiClientGetApi,
AscApiClientPostApi,
AscApiRequestError,
AscPlatform,
} from './AscApiClient';
export namespace AscApiUtils {
/**
* Maps a bundle's `DTPlatformName` (from its Info.plist) to the App Store
* Connect platform used for a build upload. Unknown or missing values fall
* back to `IOS` to preserve the previous default.
*/
export function ascPlatformFromDtPlatformName(dtPlatformName: string | null): AscPlatform {
switch (dtPlatformName) {
case 'appletvos':
return 'TV_OS';
case 'macosx':
return 'MAC_OS';
case 'xros':
return 'VISION_OS';
case 'iphoneos':
default:
return 'IOS';
}
}
/** The App Store Connect TestFlight URL path segment for a platform. */
export function testFlightPlatformPathSegment(platform: AscPlatform): string {
switch (platform) {
case 'TV_OS':
return 'tvos';
case 'MAC_OS':
return 'macos';
case 'VISION_OS':
return 'visionos';
case 'IOS':
default:
return 'ios';
}
}
export async function getAppInfoAsync({
client,
appleAppIdentifier,
@@ -61,18 +96,20 @@ export namespace AscApiUtils {
appleAppIdentifier,
bundleShortVersion,
bundleVersion,
platform,
}: {
client: Pick<AscApiClient, 'postAsync'>;
appleAppIdentifier: string;
bundleShortVersion: string;
bundleVersion: string;
platform: AscPlatform;
}): Promise<AscApiClientPostApi['/v1/buildUploads']['response']> {
try {
return await client.postAsync('/v1/buildUploads', {
data: {
type: 'buildUploads',
attributes: {
platform: 'IOS',
platform,
cfBundleShortVersionString: bundleShortVersion,
cfBundleVersion: bundleVersion,
},
@@ -148,6 +148,7 @@ describe('AscApiUtils', () => {
appleAppIdentifier: '1491144534',
bundleShortVersion: '1.2.3',
bundleVersion: '42',
platform: 'IOS',
})
).rejects.toEqual(
expect.objectContaining({
@@ -176,10 +177,40 @@ describe('AscApiUtils', () => {
appleAppIdentifier: '1491144534',
bundleShortVersion: '1.2.3',
bundleVersion: '42',
platform: 'IOS',
})
).resolves.toEqual(response);
});
it('sends the requested platform in the build upload attributes', async () => {
const response = {
data: {
type: 'buildUploads',
id: 'fdf9c476-aaa4-4ead-b91c-6e3cc3a47805',
},
} as const;
const client = {
postAsync: jest.fn().mockResolvedValue(response),
};
await AscApiUtils.createBuildUploadAsync({
client,
appleAppIdentifier: '1491144534',
bundleShortVersion: '1.2.3',
bundleVersion: '42',
platform: 'TV_OS',
});
expect(client.postAsync).toHaveBeenCalledWith(
'/v1/buildUploads',
expect.objectContaining({
data: expect.objectContaining({
attributes: expect.objectContaining({ platform: 'TV_OS' }),
}),
})
);
});
it('rethrows when error payload includes mixed error codes', async () => {
const payload = {
errors: [
@@ -208,8 +239,32 @@ describe('AscApiUtils', () => {
appleAppIdentifier: '1491144534',
bundleShortVersion: '1.2.3',
bundleVersion: '42',
platform: 'IOS',
})
).rejects.toBe(mixedError);
});
});
describe('ascPlatformFromDtPlatformName', () => {
it('maps known DTPlatformName values to App Store Connect platforms', () => {
expect(AscApiUtils.ascPlatformFromDtPlatformName('iphoneos')).toBe('IOS');
expect(AscApiUtils.ascPlatformFromDtPlatformName('appletvos')).toBe('TV_OS');
expect(AscApiUtils.ascPlatformFromDtPlatformName('macosx')).toBe('MAC_OS');
expect(AscApiUtils.ascPlatformFromDtPlatformName('xros')).toBe('VISION_OS');
});
it('falls back to IOS for unknown or missing values', () => {
expect(AscApiUtils.ascPlatformFromDtPlatformName(null)).toBe('IOS');
expect(AscApiUtils.ascPlatformFromDtPlatformName('watchos')).toBe('IOS');
});
});
describe('testFlightPlatformPathSegment', () => {
it('returns the TestFlight URL segment for each platform', () => {
expect(AscApiUtils.testFlightPlatformPathSegment('IOS')).toBe('ios');
expect(AscApiUtils.testFlightPlatformPathSegment('TV_OS')).toBe('tvos');
expect(AscApiUtils.testFlightPlatformPathSegment('MAC_OS')).toBe('macos');
expect(AscApiUtils.testFlightPlatformPathSegment('VISION_OS')).toBe('visionos');
});
});
});
@@ -9,6 +9,7 @@ import path from 'path';
import { z } from 'zod';
import { Sentry } from '../../../sentry';
import { getProxiedDownloadUrl } from '../../../utils/download';
const DEFAULT_XCLOGPARSER_VERSION = 'v0.2.47';
const XCLOGPARSER_DOWNLOAD_URL = 'https://storage.googleapis.com/turtle-v2/xclogparser';
@@ -294,24 +295,6 @@ function getXclogparserZipName(version: string): string {
return `XCLogParser-macOS-x86-64-arm64-${version}.zip`;
}
function getProxiedDownloadUrl({
directUrl,
proxyBaseUrl,
}: {
directUrl: string;
proxyBaseUrl?: string;
}): string | null {
if (!proxyBaseUrl) {
return null;
}
const parsedUrl = new URL(directUrl);
return directUrl.replace(
`${parsedUrl.protocol}//${parsedUrl.host}`,
`${proxyBaseUrl}/${parsedUrl.host}`
);
}
export function isCompileStep(step: Partial<XcactivitylogStep>): boolean {
const detailStepType = step.detailStepType ?? '';
const signature = step.signature ?? '';
@@ -0,0 +1,801 @@
import downloadFile from '@expo/downloader';
import { SystemError } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import { BuildStepEnv } from '@expo/steps';
import spawn from '@expo/turtle-spawn';
import { createHash, randomBytes } from 'node:crypto';
import fs from 'node:fs';
import net from 'node:net';
import os from 'node:os';
import path from 'node:path';
import { pipeline } from 'node:stream/promises';
import zlib from 'node:zlib';
import { IosSimulatorUtils, type IosSimulatorUuid } from '../../utils/IosSimulatorUtils';
import { sleepAsync } from '../../utils/retry';
import { type DetachedProcessHandle, spawnDetached } from './remoteDeviceRunSession';
/**
* Local egress provides an HTTP(S) proxy through the machine running the EAS CLI.
* Requests that use this proxy exit through that machine's network.
*
* Worker side: a chisel server in reverse mode listens on loopback and is exposed
* through the session's ngrok domain. When the CLI's egress client connects, chisel
* opens LOCAL_EGRESS_PROXY_PORT on this host and forwards every connection to the
* HTTP proxy the CLI runs. The macOS system proxy points at that port. Until the
* client connects, nothing listens on the port and proxied requests fail.
*
* Contract: HTTP(S) and WebSocket requests that honor the system proxy (WebKit,
* URLSession and other CFNetwork clients) exit from the egress client's network
* and fail while the client is disconnected. Requests from libraries that bypass
* the system proxy are not covered and exit from this host. Nothing here enforces
* routing; the session monitor reports such direct connections instead.
*/
export const LOCAL_EGRESS_PROXY_HOST = '127.0.0.1';
export const LOCAL_EGRESS_PROXY_PORT = 8899;
export const LOCAL_EGRESS_USERNAME = 'eas';
export const LOCAL_EGRESS_HANDOFF_PATH = path.join(os.tmpdir(), 'eas-simulator-local-egress.json');
export const CHISEL_VERSION = '1.12.0';
// sha256 of the release .gz assets, cross-checked against chisel_1.12.0_checksums.txt.
const CHISEL_SHA256_BY_ASSET: Record<string, string> = {
'chisel_1.12.0_darwin_arm64.gz':
'707a4b932eea214765146504a0df246cefc415b4297af65a80dd67cf69ba85a9',
'chisel_1.12.0_darwin_amd64.gz':
'4aeae36c867f11c8e8c3f2b913a0e063ea3c6d29e1c14a52ed2e6eef8cfc4395',
'chisel_1.12.0_linux_amd64.gz':
'f3f180f1d93aa72cce4e6386f98cc06569a0146fbd65eb4423cf83e6434bcfe6',
'chisel_1.12.0_linux_arm64.gz':
'2ec6152cd2c74fe0146d4d79e4e7aa174521368c56e433d55e023a92ea404ec3',
};
const CHISEL_STARTUP_TIMEOUT_MS = 15_000;
const EGRESS_MONITOR_INTERVAL_MS = 2_000;
const EGRESS_ESCAPE_SCAN_INTERVAL_MS = 5_000;
const EGRESS_ESCAPE_LOG_LIMIT = 50;
export type LocalEgressHandoff = {
/** Public URL of the reverse tunnel server, reachable through ngrok. */
url: string;
/** Secret the client presents to the tunnel server, paired with LOCAL_EGRESS_USERNAME. */
token: string;
/** Fingerprint of the tunnel server key, for the client to pin. */
fingerprint: string;
/** Loopback port on this host that the client must serve. */
port: number;
};
export function getChiselAssetName({
platform,
arch,
}: {
platform: NodeJS.Platform;
arch: string;
}): string {
const os = platform === 'darwin' ? 'darwin' : platform === 'linux' ? 'linux' : null;
const cpu = arch === 'arm64' ? 'arm64' : arch === 'x64' ? 'amd64' : null;
if (!os || !cpu) {
throw new SystemError(
`Local egress is not supported on ${platform}/${arch}. The reverse tunnel binary is available for macOS and Linux on arm64 and x64.`
);
}
return `chisel_${CHISEL_VERSION}_${os}_${cpu}.gz`;
}
export function getChiselDownloadUrl(assetName: string): string {
return `https://github.com/jpillora/chisel/releases/download/v${CHISEL_VERSION}/${assetName}`;
}
async function sha256FileAsync(filePath: string): Promise<string> {
const hash = createHash('sha256');
await pipeline(fs.createReadStream(filePath), hash);
return hash.digest('hex');
}
export async function downloadChiselAsync({
destinationDir,
logger,
platform = process.platform,
arch = process.arch,
}: {
destinationDir: string;
logger: bunyan;
platform?: NodeJS.Platform;
arch?: string;
}): Promise<string> {
const assetName = getChiselAssetName({ platform, arch });
const expectedSha256 = CHISEL_SHA256_BY_ASSET[assetName];
if (!expectedSha256) {
throw new SystemError(`No pinned checksum for ${assetName}.`);
}
const url = getChiselDownloadUrl(assetName);
const archivePath = path.join(destinationDir, assetName);
logger.info(`Downloading ${url}.`);
await downloadFile(url, archivePath, { retry: 3, timeout: 60_000 });
const actualSha256 = await sha256FileAsync(archivePath);
if (actualSha256 !== expectedSha256) {
throw new SystemError(
`Checksum mismatch for ${assetName}: expected ${expectedSha256}, got ${actualSha256}. ` +
'The download may be corrupted or tampered with; the session was not started.'
);
}
const binaryPath = path.join(destinationDir, 'chisel');
await pipeline(
fs.createReadStream(archivePath),
zlib.createGunzip(),
fs.createWriteStream(binaryPath)
);
await fs.promises.chmod(binaryPath, 0o755);
return binaryPath;
}
export function generateEgressCredentials(): { user: string; password: string } {
return { user: LOCAL_EGRESS_USERNAME, password: randomBytes(24).toString('base64url') };
}
// Matches 1024-65535: every unprivileged TCP port.
const UNPRIVILEGED_PORT_PATTERN =
'(102[4-9]|10[3-9][0-9]|1[1-9][0-9]{2}|[2-9][0-9]{3}|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-5])';
/**
* chisel authfile: one user, allowed to open reverse remotes on loopback only.
* The proxy port carries proxied requests. Any unprivileged loopback port may be
* forwarded too: the EAS CLI opens one per `--egress-allow localhost:<port>`
* entry so `127.0.0.1:<port>` inside the simulator reaches that port on the
* developer's machine, like `adb reverse`. iOS never sends loopback-literal
* requests to the system proxy, so the proxy alone cannot serve them.
* Reverse remotes are matched as `R:<interface>:<port>`.
*/
export function createChiselAuthfileContents({
user,
password,
port,
}: {
user: string;
password: string;
port: number;
}): string {
const escapedHost = LOCAL_EGRESS_PROXY_HOST.replace(/\./g, '\\.');
return JSON.stringify({
[`${user}:${password}`]: [
`^R:${escapedHost}:${port}$`,
`^R:${escapedHost}:${UNPRIVILEGED_PORT_PATTERN}$`,
],
});
}
export function parseChiselFingerprint(output: string): string | null {
const match = /Fingerprint\s+(\S+)/.exec(output);
return match?.[1] ?? null;
}
function isProcessRunning(pid: number): boolean {
try {
process.kill(pid, 0);
return true;
} catch {
return false;
}
}
export async function startChiselServerAsync({
chiselPath,
controlPort,
authfilePath,
env,
signal,
}: {
chiselPath: string;
controlPort: number;
authfilePath: string;
env: BuildStepEnv;
signal?: AbortSignal;
}): Promise<{ process: DetachedProcessHandle; fingerprint: string }> {
const server = spawnDetached({
command: chiselPath,
args: [
'server',
'--host',
LOCAL_EGRESS_PROXY_HOST,
'--port',
String(controlPort),
'--reverse',
'--authfile',
authfilePath,
],
// AUTH adds an unrestricted user even when --authfile is supplied.
env: { ...env, AUTH: '' },
});
try {
const deadline = Date.now() + CHISEL_STARTUP_TIMEOUT_MS;
while (Date.now() < deadline) {
signal?.throwIfAborted();
if (server.pid === undefined || !isProcessRunning(server.pid)) {
break;
}
const output = server.getOutput();
const fingerprint = parseChiselFingerprint(output);
// Chisel prints the fingerprint before binding. This line comes from the
// child only after its listener succeeds, unlike a probe of a reused port.
const listening = output
.split('\n')
.some(line =>
line
.trimEnd()
.endsWith(`server: Listening on http://${LOCAL_EGRESS_PROXY_HOST}:${controlPort}`)
);
if (fingerprint && listening) {
return { process: server, fingerprint };
}
await sleepAsync(250);
}
throw new SystemError(
`The reverse tunnel server did not start within ${CHISEL_STARTUP_TIMEOUT_MS / 1000}s. Output:\n${
server.getOutput() || '<empty>'
}`
);
} catch (error) {
await server.stopAsync();
throw error;
}
}
export function parseDefaultRouteInterface(routeOutput: string): string | null {
const match = /^\s*interface:\s*(\S+)/m.exec(routeOutput);
return match?.[1] ?? null;
}
/**
* `networksetup -listnetworkserviceorder` prints each service as two lines:
* "(1) Wi-Fi" followed by "(Hardware Port: Wi-Fi, Device: en0)". Return the
* service name whose device matches.
*/
export function parseNetworkServiceNameForDevice(
listOutput: string,
device: string
): string | null {
const lines = listOutput.split('\n').map(line => line.trim());
for (let i = 0; i < lines.length; i++) {
const hardwarePortMatch = /^\(Hardware Port: (.+), Device: (\S+)\)$/.exec(lines[i]);
if (!hardwarePortMatch || hardwarePortMatch[2] !== device) {
continue;
}
const serviceLineMatch = /^\(\d+\) (.+)$/.exec(lines[i - 1] ?? '');
return serviceLineMatch?.[1] ?? hardwarePortMatch[1];
}
return null;
}
export async function resolveActiveNetworkServiceNameAsync({
env,
}: {
env: BuildStepEnv;
}): Promise<string> {
const routeResult = await spawn('route', ['-n', 'get', 'default'], { env, stdio: 'pipe' });
const device = parseDefaultRouteInterface(routeResult.stdout);
if (!device) {
throw new SystemError(
'Could not determine the default network interface of the device host, so the system proxy ' +
'cannot be configured. Output of `route -n get default`:\n' +
(routeResult.stdout || '<empty>')
);
}
const listResult = await spawn('networksetup', ['-listnetworkserviceorder'], {
env,
stdio: 'pipe',
});
const service = parseNetworkServiceNameForDevice(listResult.stdout, device);
if (!service) {
throw new SystemError(
`Could not find the network service for interface ${device}, so the system proxy cannot be ` +
'configured. Output of `networksetup -listnetworkserviceorder`:\n' +
(listResult.stdout || '<empty>')
);
}
return service;
}
export function buildNetworksetupProxyArgs({
service,
host,
port,
}: {
service: string;
host: string;
port: number;
}): string[][] {
return [
['-setwebproxy', service, host, String(port)],
['-setsecurewebproxy', service, host, String(port)],
];
}
/**
* Point the macOS system HTTP and HTTPS proxy at the loopback egress port. The
* simulator reads these settings when it boots, so this must run before
* `start_ios_simulator`. Existing bypass and automatic proxy settings are left
* unchanged. Requests that bypass this proxy can leave from this host's address.
*/
export async function configureSystemProxyAsync({
env,
logger,
port,
signal,
}: {
env: BuildStepEnv;
logger: bunyan;
port: number;
signal?: AbortSignal;
}): Promise<{ service: string }> {
if (process.env.ENVIRONMENT === 'development') {
logger.info('Job running outside of EAS, not changing the system proxy.');
return { service: '<development>' };
}
const service = await resolveActiveNetworkServiceNameAsync({ env });
for (const args of buildNetworksetupProxyArgs({ service, host: LOCAL_EGRESS_PROXY_HOST, port })) {
signal?.throwIfAborted();
await spawn('networksetup', args, { env, logger, signal });
}
logger.info(`System proxy for "${service}" set to ${LOCAL_EGRESS_PROXY_HOST}:${port}.`);
return { service };
}
export const LOCAL_EGRESS_NO_PROXY = 'localhost,127.0.0.1,::1';
/**
* Proxy environment for processes inside the simulator. CFNetwork ignores these
* variables and follows the system proxy, but clients with their own network
* stack that read them (gRPC and therefore Firestore, libcurl, Go) then send
* HTTP CONNECT requests to the same loopback port. Loopback is excluded so the
* ports forwarded for `--egress-allow` behave the same on both paths. Lowercase
* and uppercase forms are both set because clients differ in which they read.
*/
export function buildLocalEgressSimulatorEnvironment(port: number): Record<string, string> {
const proxyUrl = `http://${LOCAL_EGRESS_PROXY_HOST}:${port}`;
return {
http_proxy: proxyUrl,
https_proxy: proxyUrl,
HTTP_PROXY: proxyUrl,
HTTPS_PROXY: proxyUrl,
grpc_proxy: proxyUrl,
no_proxy: LOCAL_EGRESS_NO_PROXY,
NO_PROXY: LOCAL_EGRESS_NO_PROXY,
};
}
/**
* Set the local egress proxy environment in a booted simulator when a local
* egress session is active. Returns false when local egress is not configured
* or the environment could not be set. Failure is a warning, not an error: the
* session still works, but clients that read these variables then exit from
* this worker instead of the egress client, which the warning spells out.
*/
export async function configureSimulatorProxyEnvironmentAsync({
udid,
env,
logger,
handoffPath = LOCAL_EGRESS_HANDOFF_PATH,
}: {
udid: IosSimulatorUuid;
env: NodeJS.ProcessEnv;
logger: bunyan;
handoffPath?: string;
}): Promise<boolean> {
let handoff: LocalEgressHandoff | null;
try {
handoff = await readLocalEgressHandoffAsync(handoffPath);
} catch (err) {
logger.warn(
{ err },
'Local egress: could not read the local egress handoff, so proxy environment variables were ' +
'not set in the Simulator. Clients that read http_proxy/https_proxy (for example gRPC and ' +
'libcurl) will bypass local egress and exit from this worker.'
);
return false;
}
if (!handoff) {
return false;
}
const variables = buildLocalEgressSimulatorEnvironment(handoff.port);
try {
await IosSimulatorUtils.setLaunchdEnvironmentAsync({ udid, env, variables });
} catch (err) {
logger.warn(
{ err },
'Local egress: could not set proxy environment variables in the Simulator. Clients that read ' +
'http_proxy/https_proxy (for example gRPC and libcurl) will bypass local egress and exit from ' +
'this worker. CFNetwork clients (WebKit, URLSession) still follow the system proxy.'
);
return false;
}
logger.info(
`Local egress: proxy environment variables set in the Simulator (${Object.keys(variables).join(
', '
)}), so clients that read them, such as gRPC and libcurl, also exit from the egress client.`
);
return true;
}
export async function writeLocalEgressHandoffAsync(
handoff: LocalEgressHandoff,
handoffPath: string = LOCAL_EGRESS_HANDOFF_PATH
): Promise<void> {
await fs.promises.writeFile(handoffPath, JSON.stringify(handoff), {
encoding: 'utf8',
mode: 0o600,
});
}
export async function readLocalEgressHandoffAsync(
handoffPath: string = LOCAL_EGRESS_HANDOFF_PATH
): Promise<LocalEgressHandoff | null> {
let raw: string;
try {
raw = await fs.promises.readFile(handoffPath, 'utf8');
} catch (err) {
if ((err as NodeJS.ErrnoException).code === 'ENOENT') {
return null;
}
throw err;
}
const parsed = JSON.parse(raw) as Partial<LocalEgressHandoff>;
if (
typeof parsed.url !== 'string' ||
typeof parsed.token !== 'string' ||
typeof parsed.fingerprint !== 'string' ||
typeof parsed.port !== 'number'
) {
throw new SystemError(`Local egress handoff at ${handoffPath} is malformed.`);
}
return {
url: parsed.url,
token: parsed.token,
fingerprint: parsed.fingerprint,
port: parsed.port,
};
}
/** remoteConfig fields the CLI needs to start the egress client. */
export function buildEgressRemoteConfigFields(
handoff: LocalEgressHandoff | null
): Record<string, string | number> {
if (!handoff) {
return {};
}
return {
egressUrl: handoff.url,
egressToken: handoff.token,
egressFingerprint: handoff.fingerprint,
egressPort: handoff.port,
};
}
// The egress step returns before the session ends, so the resources it created
// are held here and released by the session step or the job's finally block.
let activeLocalEgressResources:
| { stopAsync: () => Promise<void>; controller: AbortController; stopping?: Promise<void> }
| undefined;
export function registerLocalEgressResources(stopAsync: () => Promise<void>): AbortSignal {
if (activeLocalEgressResources) {
throw new SystemError('Local egress resources are already registered for this job.');
}
const controller = new AbortController();
activeLocalEgressResources = { stopAsync, controller };
return controller.signal;
}
export async function stopLocalEgressResourcesAsync(logger: bunyan): Promise<void> {
const resources = activeLocalEgressResources;
if (!resources) {
return;
}
resources.controller.abort();
resources.stopping ??= Promise.resolve()
.then(resources.stopAsync)
.catch(err => logger.warn({ err }, 'Could not stop a local egress resource.'))
.finally(() => {
activeLocalEgressResources = undefined;
});
await resources.stopping;
}
export async function isPortListeningAsync({
host,
port,
timeoutMs = 1_000,
}: {
host: string;
port: number;
timeoutMs?: number;
}): Promise<boolean> {
return await new Promise<boolean>(resolve => {
const socket = net.connect({ host, port });
const finish = (listening: boolean): void => {
socket.destroy();
resolve(listening);
};
socket.setTimeout(timeoutMs, () => finish(false));
socket.once('connect', () => finish(true));
socket.once('error', () => finish(false));
});
}
export function parseExitIpResponse(body: string): string {
const parsed = JSON.parse(body) as { ip?: unknown };
if (typeof parsed.ip !== 'string' || parsed.ip.length === 0) {
throw new SystemError(`Unexpected exit IP response: ${body}`);
}
return parsed.ip;
}
export async function fetchExitIpThroughProxyAsync({
port,
env,
}: {
port: number;
env: BuildStepEnv;
}): Promise<string> {
const result = await spawn(
'curl',
[
'-sS',
'--max-time',
'10',
'-x',
`http://${LOCAL_EGRESS_PROXY_HOST}:${port}`,
'https://api.ipify.org?format=json',
],
{ env, stdio: 'pipe' }
);
return parseExitIpResponse(result.stdout);
}
/**
* Processes inside the simulator run on the host as descendants of launchd_sim,
* so `ps -axo pid=,ppid=,comm=` ancestry identifies them. Returns every
* descendant pid; launchd_sim itself opens no network connections.
*/
export function collectSimulatorProcessIds(psOutput: string): number[] {
const childrenByParent = new Map<number, number[]>();
const roots: number[] = [];
for (const line of psOutput.split('\n')) {
const match = /^\s*(\d+)\s+(\d+)\s+(\S.*)$/.exec(line);
if (!match) {
continue;
}
const pid = Number(match[1]);
const parentPid = Number(match[2]);
const command = match[3].trim();
const siblings = childrenByParent.get(parentPid) ?? [];
siblings.push(pid);
childrenByParent.set(parentPid, siblings);
if (path.basename(command) === 'launchd_sim') {
roots.push(pid);
}
}
const descendants: number[] = [];
const seen = new Set<number>(roots);
const queue = [...roots];
for (let i = 0; i < queue.length; i++) {
for (const child of childrenByParent.get(queue[i]) ?? []) {
if (seen.has(child)) {
continue;
}
seen.add(child);
descendants.push(child);
queue.push(child);
}
}
return descendants;
}
export type DirectSimulatorConnection = {
pid: number;
command: string;
protocol: string;
/** Remote peer as lsof prints it, e.g. `93.184.216.34:443` or `[2606::1]:443`. */
remote: string;
};
function remoteHostOf(remote: string): string {
if (remote.startsWith('[')) {
const end = remote.indexOf(']');
return end === -1 ? remote : remote.slice(1, end);
}
const colon = remote.lastIndexOf(':');
return colon === -1 ? remote : remote.slice(0, colon);
}
function isLoopbackHost(host: string): boolean {
return (
host === 'localhost' || host.startsWith('127.') || host === '::1' || host === '::ffff:127.0.0.1'
);
}
/**
* Parse `lsof -nP -i -F pcPnT` output. Returns the TCP and UDP connections that
* simulator processes hold to peers outside loopback. Those did not go through
* the proxy on 127.0.0.1, so they exit from this host instead of the egress
* client. Listening sockets, unconnected UDP sockets and connections to
* loopback (the proxy itself) are ignored.
*/
export function parseDirectSimulatorConnections(
lsofOutput: string,
simulatorPids: ReadonlySet<number>
): DirectSimulatorConnection[] {
const connections: DirectSimulatorConnection[] = [];
let pid: number | null = null;
let command = '';
let protocol: string | null = null;
let name: string | null = null;
let tcpState: string | null = null;
const flush = (): void => {
if (pid !== null && simulatorPids.has(pid) && protocol && name) {
const arrow = name.indexOf('->');
const remote = arrow === -1 ? null : name.slice(arrow + 2);
const active =
protocol !== 'TCP' ||
tcpState === null ||
tcpState === 'ESTABLISHED' ||
tcpState === 'SYN_SENT';
if (remote && active && !isLoopbackHost(remoteHostOf(remote))) {
connections.push({ pid, command, protocol, remote });
}
}
protocol = null;
name = null;
tcpState = null;
};
for (const line of lsofOutput.split('\n')) {
const field = line[0];
const value = line.slice(1);
switch (field) {
case 'p':
flush();
pid = Number(value);
command = '';
break;
case 'c':
command = value;
break;
case 'f':
flush();
break;
case 'P':
protocol = value;
break;
case 'n':
name = value;
break;
case 'T':
if (value.startsWith('ST=')) {
tcpState = value.slice('ST='.length);
}
break;
default:
break;
}
}
flush();
return connections;
}
async function findDirectSimulatorConnectionsAsync({
env,
}: {
env: BuildStepEnv;
}): Promise<DirectSimulatorConnection[]> {
const ps = await spawn('ps', ['-axo', 'pid=,ppid=,comm='], { env, stdio: 'pipe' });
const pids = collectSimulatorProcessIds(ps.stdout);
if (pids.length === 0) {
return [];
}
let lsofOutput: string;
try {
const lsof = await spawn('lsof', ['-nP', '-i', '-F', 'pcPnT', '-a', '-p', pids.join(',')], {
env,
stdio: 'pipe',
});
lsofOutput = lsof.stdout;
} catch (err) {
// lsof exits with 1 when none of the processes holds a matching socket, and
// when one of them exited between `ps` and `lsof`. Its stdout is still valid.
const result = err as { status?: number | null; stdout?: string };
if (result.status !== 1) {
throw err;
}
lsofOutput = result.stdout ?? '';
}
return parseDirectSimulatorConnections(lsofOutput, new Set(pids));
}
/**
* Log proxy listener availability, the exit IP observed by a worker request
* through it, and simulator connections that bypassed the proxy. Neither check
* verifies that proxied simulator requests reach the egress client. Never
* rejects: it runs in the background for the whole session.
*/
export async function monitorLocalEgressAsync({
port,
env,
logger,
signal,
}: {
port: number;
env: BuildStepEnv;
logger: bunyan;
signal: AbortSignal;
}): Promise<void> {
let connected = false;
let lastEscapeScanAt = 0;
let escapeScanBroken = false;
const reportedEscapes = new Set<string>();
const lifetimeSignal = activeLocalEgressResources?.controller.signal;
try {
while (!signal.aborted && !lifetimeSignal?.aborted) {
const listening = await isPortListeningAsync({ host: LOCAL_EGRESS_PROXY_HOST, port });
if (listening && !connected) {
connected = true;
logger.info('Local egress proxy listener is available.');
try {
const exitIp = await fetchExitIpThroughProxyAsync({ port, env });
logger.info(
`Worker proxy exit-IP check observed ${exitIp}. This does not verify simulator routing.`
);
} catch (err) {
logger.warn(
{ err },
'The local egress proxy listener is available, but the worker exit-IP check through it failed.'
);
}
} else if (!listening && connected) {
connected = false;
logger.warn(
'Local egress proxy listener is unavailable. Proxied HTTP(S) requests fail until it returns.'
);
}
if (!escapeScanBroken && Date.now() - lastEscapeScanAt >= EGRESS_ESCAPE_SCAN_INTERVAL_MS) {
lastEscapeScanAt = Date.now();
try {
for (const connection of await findDirectSimulatorConnectionsAsync({ env })) {
const key = `${connection.pid}|${connection.protocol}|${connection.remote}`;
if (reportedEscapes.has(key)) {
continue;
}
reportedEscapes.add(key);
if (reportedEscapes.size > EGRESS_ESCAPE_LOG_LIMIT) {
if (reportedEscapes.size === EGRESS_ESCAPE_LOG_LIMIT + 1) {
logger.warn(
`Local egress: more than ${EGRESS_ESCAPE_LOG_LIMIT} direct connections were reported; further ones are not logged.`
);
}
continue;
}
logger.warn(
`Local egress: ${connection.command} (pid ${connection.pid}) opened a direct ${connection.protocol} connection to ${connection.remote}, bypassing the system proxy. That traffic exits from this worker, not from the egress client.`
);
}
} catch (err) {
escapeScanBroken = true;
logger.warn(
{ err },
'Local egress: could not inspect simulator connections. Direct connections that bypass the proxy will not be reported.'
);
}
}
await sleepAsync(EGRESS_MONITOR_INTERVAL_MS);
}
} catch (err) {
logger.warn({ err }, 'Local egress monitoring stopped unexpectedly.');
}
}
@@ -0,0 +1,699 @@
import { SystemError } from '@expo/eas-build-job';
import { type bunyan } from '@expo/logger';
import spawn from '@expo/turtle-spawn';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { IosSimulatorUtils, type IosSimulatorUuid } from '../../utils/IosSimulatorUtils';
import {
LOCAL_EGRESS_HANDOFF_PATH,
buildLocalEgressSimulatorEnvironment,
collectSimulatorProcessIds,
readLocalEgressHandoffAsync,
} from './localEgress';
/**
* Worker side of the local egress guard: a dylib injected into every process
* the simulator launches, which refuses connections that do not go to
* loopback (where the proxy and the `--egress-allow` forwards live) and
* records one event per destination per process. This module installs it
* through the simulator's launchd environment and relays its events into the
* session log. See resources/egress-guard/README.md.
*/
export const EGRESS_GUARD_LIBRARY_FILE = 'egress-guard.dylib';
export const EGRESS_GUARD_CHECK_FILE = 'egress-guard-check';
export const EGRESS_GUARD_LOG_ENV = 'EAS_EGRESS_GUARD_LOG';
export const EGRESS_GUARD_MODE_ENV = 'EAS_EGRESS_GUARD_MODE';
export const LOCAL_EGRESS_GUARD_LOG_PATH = path.join(os.tmpdir(), 'eas-local-egress-guard.log');
const GUARD_EVENT_PREFIX = 'eas-egress-guard';
/** Written once by a process whose per-process table of destinations is full; see policy.h. */
const GUARD_OVERFLOW_FUNCTION = 'overflow';
const GUARD_RELAY_LOG_LIMIT = 200;
const GUARD_TAIL_INTERVAL_MS = 1_000;
export type EgressGuardMode = 'block' | 'log';
export function buildGuardLaunchdEnvironment({
libraryPath,
logPath,
mode,
}: {
libraryPath: string;
logPath: string;
mode: EgressGuardMode;
}): Record<string, string> {
return {
DYLD_INSERT_LIBRARIES: libraryPath,
[EGRESS_GUARD_LOG_ENV]: logPath,
[EGRESS_GUARD_MODE_ENV]: mode,
};
}
export type GuardEvent = {
process: string;
pid: number;
function: string;
action: 'blocked' | 'logged';
peer: string;
/** Image names above the interposer, innermost first. */
callers: string[];
};
/** One tab-separated line written by the guard; see policy.h for the format. */
export function parseGuardLogLine(line: string): GuardEvent | null {
const fields = line.split('\t');
if (fields.length < 7 || fields[0] !== GUARD_EVENT_PREFIX) {
return null;
}
const [, process, pidText, fn, action, peer, callerText] = fields;
const pid = Number(pidText);
if (!Number.isInteger(pid) || (action !== 'blocked' && action !== 'logged') || !fn || !peer) {
return null;
}
return {
process,
pid,
function: fn,
action,
peer,
callers: callerText ? callerText.split(',').filter(Boolean) : [],
};
}
const PACKAGED_BIN_DIR = path.join(__dirname, '..', '..', '..', 'bin');
async function resolvePackagedFileAsync(binDir: string, file: string): Promise<string | null> {
const filePath = path.join(binDir, file);
try {
await fs.promises.access(filePath);
return filePath;
} catch {
return null;
}
}
/** The packaged library, next to the compiled package like record-sim. */
export async function resolveEgressGuardLibraryAsync(
binDir: string = PACKAGED_BIN_DIR
): Promise<string | null> {
return await resolvePackagedFileAsync(binDir, EGRESS_GUARD_LIBRARY_FILE);
}
/** The packaged self-check binary, built alongside the library. */
export async function resolveEgressGuardCheckAsync(
binDir: string = PACKAGED_BIN_DIR
): Promise<string | null> {
return await resolvePackagedFileAsync(binDir, EGRESS_GUARD_CHECK_FILE);
}
/**
* Turns guard events into session log lines: one line the first time a
* process reaches a destination through a given call, counts after that, and
* a summary at the end.
*/
export class GuardEventRelay {
private readonly seen = new Set<string>();
private readonly peers = new Set<string>();
private readonly processes = new Set<string>();
private blocked = 0;
private logged = 0;
private suppressed = 0;
private logger: bunyan;
constructor(
logger: bunyan,
private readonly limit: number = GUARD_RELAY_LOG_LIMIT
) {
this.logger = logger;
}
/**
* Log through a different step's logger from now on. Refusals happen for
* the life of the session, so they belong to the step that runs the
* session, not the one that booted the simulator minutes earlier.
*/
setLogger(logger: bunyan): void {
this.logger = logger;
}
handle(event: GuardEvent): void {
if (event.process === EGRESS_GUARD_CHECK_FILE) {
// The self-check deliberately trips the guard once; not a bypass.
return;
}
if (event.function === GUARD_OVERFLOW_FUNCTION) {
const verb = event.action === 'blocked' ? 'refused' : 'observed';
this.logger.info(
`Local egress guard: ${event.process} (pid ${event.pid}) reached the limit of ${event.peer} listed per process; further distinct destinations from it are ${verb} but not listed.`
);
return;
}
if (event.action === 'blocked') {
this.blocked++;
} else {
this.logged++;
}
this.peers.add(event.peer);
this.processes.add(event.process);
const key = `${event.process}|${event.function}|${event.peer}`;
if (this.seen.has(key)) {
return;
}
this.seen.add(key);
if (this.seen.size > this.limit) {
this.suppressed++;
return;
}
const verb = event.action === 'blocked' ? 'refused' : 'observed';
const callers = event.callers.length ? `; callers: ${event.callers.join(', ')}` : '';
this.logger.info(
`Local egress guard: ${verb} ${event.function} from ${event.process} (pid ${event.pid}) to ${event.peer}${callers}`
);
}
summary(): { blocked: number; logged: number; distinct: number; suppressed: number } {
return {
blocked: this.blocked,
logged: this.logged,
distinct: this.peers.size,
suppressed: this.suppressed,
};
}
logSummary(): void {
const { blocked, logged, distinct, suppressed } = this.summary();
const observed = logged ? ` and observed ${logged} more without refusing` : '';
const dropped = suppressed
? ` ${suppressed} further distinct destination(s) were not logged individually.`
: '';
this.logger.info(
`Local egress guard: refused ${blocked} connection attempt(s) to ${distinct} distinct destination(s) from ${this.processes.size} process(es)${observed}.${dropped}`
);
}
}
/**
* Polls a file the simulator processes append to and delivers whole lines.
* Tolerates the file not existing yet, partial trailing lines, and truncation.
*/
export class GuardLogTailer {
private readonly path: string;
private readonly onLine: (line: string) => void;
private readonly intervalMs: number;
private offset = 0;
private partial = '';
private timer: NodeJS.Timeout | undefined;
private reading: Promise<void> = Promise.resolve();
constructor({
path: filePath,
onLine,
intervalMs = GUARD_TAIL_INTERVAL_MS,
}: {
path: string;
onLine: (line: string) => void;
intervalMs?: number;
}) {
this.path = filePath;
this.onLine = onLine;
this.intervalMs = intervalMs;
}
start(): void {
this.timer = setInterval(() => {
this.reading = this.reading.then(() => this.readAsync()).catch(() => {});
}, this.intervalMs);
this.timer.unref();
}
async stopAsync(): Promise<void> {
if (this.timer) {
clearInterval(this.timer);
this.timer = undefined;
}
await this.reading.catch(() => {});
await this.readAsync().catch(() => {});
}
private async readAsync(): Promise<void> {
let handle: fs.promises.FileHandle;
try {
handle = await fs.promises.open(this.path, 'r');
} catch (err) {
if ((err as NodeJS.ErrnoException).code === 'ENOENT') {
return;
}
throw err;
}
try {
const { size } = await handle.stat();
if (size < this.offset) {
// Truncated or replaced; start over.
this.offset = 0;
this.partial = '';
}
if (size === this.offset) {
return;
}
const buffer = new Uint8Array(size - this.offset);
const { bytesRead } = await handle.read(buffer, 0, buffer.length, this.offset);
this.offset += bytesRead;
const text = this.partial + Buffer.from(buffer.buffer, 0, bytesRead).toString('utf8');
const lines = text.split('\n');
this.partial = lines.pop() ?? '';
for (const line of lines) {
if (line.length > 0) {
this.onLine(line);
}
}
} finally {
await handle.close();
}
}
}
/**
* The environment the simulator's launchd must have from its first process:
* the guard and the proxy variables. Pass it to `IosSimulatorUtils.bootAsync`,
* which hands it to launchd before anything is spawned; `launchctl setenv`
* after boot only reaches later processes. Returns null when no local egress
* session is active. Throws when the guard library is not packaged, since a
* local egress session without it would silently leak.
*/
export async function resolveLocalEgressBootEnvironmentAsync({
handoffPath = LOCAL_EGRESS_HANDOFF_PATH,
libraryPath,
logPath = LOCAL_EGRESS_GUARD_LOG_PATH,
mode = 'block',
}: {
handoffPath?: string;
/** Explicit library path, `null` for "not available"; resolved from the package when omitted. */
libraryPath?: string | null;
logPath?: string;
mode?: EgressGuardMode;
} = {}): Promise<Record<string, string> | null> {
const handoff = await readLocalEgressHandoffAsync(handoffPath);
if (!handoff) {
return null;
}
const resolvedLibrary =
libraryPath === undefined ? await resolveEgressGuardLibraryAsync() : libraryPath;
if (!resolvedLibrary) {
throw new SystemError(
'The local egress guard library is not available on this device host, so this local egress session ' +
'cannot guarantee that connections bypassing the system proxy are refused. The device host image is ' +
'missing bin/egress-guard.dylib; this is a service problem, please contact support.'
);
}
return {
...buildGuardLaunchdEnvironment({ libraryPath: resolvedLibrary, logPath, mode }),
...buildLocalEgressSimulatorEnvironment(handoff.port),
};
}
type ActiveRelay = { tailer: GuardLogTailer; relay: GuardEventRelay };
const activeRelays = new Map<string, ActiveRelay>();
/**
* Install the guard into a simulator when a local egress session is active,
* and start relaying its events into the session log. Returns false when
* there is no local egress session. Throws when the library is not packaged
* or launchd could not be configured: a local egress session without the
* guard would silently leak, so it must not start. Only an unwritable event
* log is a warning, since refusals still happen and only reporting is lost.
*
* Call this as soon as `simctl boot` returns: launchd is up and nothing else
* has started, so every process the boot spawns inherits the guard. Verify
* with `verifyLocalEgressGuardAsync` once boot completes.
*/
export async function installLocalEgressGuardAsync({
udid,
env,
logger,
handoffPath = LOCAL_EGRESS_HANDOFF_PATH,
libraryPath,
logPath = LOCAL_EGRESS_GUARD_LOG_PATH,
mode = 'block',
tailIntervalMs,
}: {
udid: IosSimulatorUuid;
env: NodeJS.ProcessEnv;
logger: bunyan;
handoffPath?: string;
/** Explicit library path, `null` for "not available"; resolved from the package when omitted. */
libraryPath?: string | null;
logPath?: string;
mode?: EgressGuardMode;
tailIntervalMs?: number;
}): Promise<boolean> {
let handoff;
try {
handoff = await readLocalEgressHandoffAsync(handoffPath);
} catch (err) {
logger.warn(
{ err },
'Local egress guard: could not read the local egress handoff, so the guard was not installed. ' +
'Connections that bypass the proxy will exit from this worker.'
);
return false;
}
if (!handoff) {
return false;
}
const resolvedLibrary =
libraryPath === undefined ? await resolveEgressGuardLibraryAsync() : libraryPath;
if (!resolvedLibrary) {
throw new SystemError(
'The local egress guard library is not available on this device host, so this local egress session ' +
'cannot guarantee that connections bypassing the system proxy are refused. The device host image is ' +
'missing bin/egress-guard.dylib; this is a service problem, please contact support.'
);
}
await logAlreadyRunningProcessesAsync({ env, logger });
let logWritable = true;
try {
await fs.promises.mkdir(path.dirname(logPath), { recursive: true });
await fs.promises.appendFile(logPath, '');
} catch (err) {
logWritable = false;
logger.warn(
{ err },
`Local egress guard: could not create ${logPath}, so refused connections will not be reported in this log. They are still refused.`
);
}
try {
await IosSimulatorUtils.setLaunchdEnvironmentAsync({
udid,
env,
variables: buildGuardLaunchdEnvironment({ libraryPath: resolvedLibrary, logPath, mode }),
});
} catch (err) {
throw new SystemError(
'Could not install the local egress guard in the Simulator (launchctl setenv failed), so this local ' +
'egress session cannot guarantee that connections bypassing the system proxy are refused. Retry the ' +
'session; if it keeps failing, please contact support.',
{ cause: err }
);
}
if (logWritable && !activeRelays.has(logPath)) {
const relay = new GuardEventRelay(logger);
const tailer = new GuardLogTailer({
path: logPath,
intervalMs: tailIntervalMs,
onLine: line => {
const event = parseGuardLogLine(line);
if (event) {
relay.handle(event);
}
},
});
tailer.start();
activeRelays.set(logPath, { tailer, relay });
}
logger.info(
`Local egress guard installed in the Simulator (mode ${mode}): connections that bypass the system proxy are ${
mode === 'block' ? 'refused' : 'observed'
} in the process that makes them and reported here as they happen.`
);
return true;
}
/**
* Processes the simulator already runs when the guard is installed never get
* it. Right after `simctl boot` that is nothing; later it is SpringBoard and
* the early daemons, which follow the system proxy anyway. Log them so the
* uncovered set is visible rather than assumed.
*/
async function logAlreadyRunningProcessesAsync({
env,
logger,
}: {
env: NodeJS.ProcessEnv;
logger: bunyan;
}): Promise<void> {
let psOutput = '';
try {
psOutput = (await spawn('ps', ['-axo', 'pid=,ppid=,comm='], { env, stdio: 'pipe' })).stdout;
} catch {
return;
}
const pids = new Set(collectSimulatorProcessIds(psOutput));
const names = new Set<string>();
for (const line of psOutput.split('\n')) {
const match = /^\s*(\d+)\s+\d+\s+(\S.*)$/.exec(line);
if (match && pids.has(Number(match[1]))) {
names.add(path.basename(match[2].trim()));
}
}
if (names.size === 0) {
logger.info(
'Local egress guard: no simulator process was running before the guard was installed.'
);
return;
}
const listed = [...names].sort();
const shown =
listed.slice(0, 20).join(', ') + (listed.length > 20 ? `, and ${listed.length - 20} more` : '');
logger.info(
`Local egress guard: ${names.size} simulator process(es) were already running before the guard was installed and are not covered by it: ${shown}.`
);
}
/**
* launchd's trampoline exists for milliseconds between fork and exec of the
* real service, with nothing mapped yet; it never makes a connection itself.
*/
const COVERAGE_IGNORED_PROCESSES = new Set(['xpcproxy_sim']);
export type GuardCoverage = {
/** Simulator processes with the guard library mapped. */
covered: string[];
/** Simulator processes without it: started before the guard was installed. */
uncovered: string[];
/** Pids behind `uncovered`, for comparing samples. */
uncoveredPids: number[];
};
/**
* Which simulator processes have the guard library mapped, from
* `ps -axo pid=,ppid=,comm=` and `lsof -nP -a -p <pids> -d txt -F pn` output.
* A process is covered when any of its mapped images is the guard library.
*/
export function parseGuardCoverage(psOutput: string, lsofOutput: string): GuardCoverage {
const simulatorPids = new Set(collectSimulatorProcessIds(psOutput));
const commandsByPid = new Map<number, string>();
for (const line of psOutput.split('\n')) {
const match = /^\s*(\d+)\s+\d+\s+(\S.*)$/.exec(line);
if (match) {
commandsByPid.set(Number(match[1]), path.basename(match[2].trim()));
}
}
const loaded = new Set<number>();
let pid: number | null = null;
for (const line of lsofOutput.split('\n')) {
if (line[0] === 'p') {
pid = Number(line.slice(1));
} else if (line[0] === 'n' && pid !== null && line.endsWith(EGRESS_GUARD_LIBRARY_FILE)) {
loaded.add(pid);
}
}
const covered: string[] = [];
const uncovered: string[] = [];
const uncoveredPids: number[] = [];
for (const simulatorPid of simulatorPids) {
const name = commandsByPid.get(simulatorPid) ?? String(simulatorPid);
if (COVERAGE_IGNORED_PROCESSES.has(name)) {
continue;
}
if (loaded.has(simulatorPid)) {
covered.push(name);
} else {
uncovered.push(name);
uncoveredPids.push(simulatorPid);
}
}
covered.sort();
uncovered.sort();
return { covered, uncovered, uncoveredPids };
}
/**
* A process caught between fork and exec has nothing mapped yet and looks
* uncovered for an instant. Two samples a moment apart separate those from
* processes that really run without the guard: only pids uncovered in both
* count, reported with the later sample's names.
*/
export function mergeGuardCoverageSamples(
first: GuardCoverage,
second: GuardCoverage
): GuardCoverage {
const persistent = new Set(first.uncoveredPids.filter(pid => second.uncoveredPids.includes(pid)));
const uncovered: string[] = [];
const uncoveredPids: number[] = [];
second.uncoveredPids.forEach((pid, index) => {
if (persistent.has(pid)) {
uncovered.push(second.uncovered[index]);
uncoveredPids.push(pid);
}
});
return { covered: second.covered, uncovered, uncoveredPids };
}
/**
* Measure and log guard coverage across the simulator's processes. Observation
* only: the uncovered set is whatever started before the guard was installed,
* which is nothing when installation runs right after `simctl boot`.
*/
export async function reportLocalEgressGuardCoverageAsync({
env,
logger,
sampleIntervalMs = 1_000,
}: {
env: NodeJS.ProcessEnv;
logger: bunyan;
sampleIntervalMs?: number;
}): Promise<GuardCoverage | null> {
const first = await sampleGuardCoverageAsync({ env });
if (!first) {
return null;
}
await new Promise(resolve => setTimeout(resolve, sampleIntervalMs));
const second = await sampleGuardCoverageAsync({ env });
const coverage = second ? mergeGuardCoverageSamples(first, second) : first;
const total = coverage.covered.length + coverage.uncovered.length;
if (coverage.uncovered.length === 0) {
logger.info(
`Local egress guard coverage: all ${total} simulator process(es) have the guard loaded.`
);
} else {
const shown =
coverage.uncovered.slice(0, 20).join(', ') +
(coverage.uncovered.length > 20 ? `, and ${coverage.uncovered.length - 20} more` : '');
logger.info(
`Local egress guard coverage: ${coverage.covered.length} of ${total} simulator process(es) have the guard loaded. Not covered, started before the guard was installed: ${shown}.`
);
}
return coverage;
}
async function sampleGuardCoverageAsync({
env,
}: {
env: NodeJS.ProcessEnv;
}): Promise<GuardCoverage | null> {
let psOutput: string;
try {
psOutput = (await spawn('ps', ['-axo', 'pid=,ppid=,comm='], { env, stdio: 'pipe' })).stdout;
} catch {
return null;
}
const pids = collectSimulatorProcessIds(psOutput);
if (pids.length === 0) {
return null;
}
let lsofOutput: string;
try {
lsofOutput = (
await spawn('lsof', ['-nP', '-a', '-p', pids.join(','), '-d', 'txt', '-F', 'pn'], {
env,
stdio: 'pipe',
})
).stdout;
} catch (err) {
// lsof exits 1 when some listed pid has already exited; stdout is still valid.
const result = err as { status?: number | null; stdout?: string };
if (result.status !== 1) {
return null;
}
lsofOutput = result.stdout ?? '';
}
return parseGuardCoverage(psOutput, lsofOutput);
}
/**
* Run the packaged self-check inside the simulator: it must find the guard
* loaded in a fresh process and see it behave as `mode` says. Throws when the
* check binary is missing or the check fails, which fails the session.
*/
export async function verifyLocalEgressGuardAsync({
udid,
env,
logger,
mode = 'block',
checkPath,
}: {
udid: IosSimulatorUuid;
env: NodeJS.ProcessEnv;
logger: bunyan;
mode?: EgressGuardMode;
/** Explicit check binary path, `null` for "not available"; resolved from the package when omitted. */
checkPath?: string | null;
}): Promise<void> {
const resolvedCheck = checkPath === undefined ? await resolveEgressGuardCheckAsync() : checkPath;
if (!resolvedCheck) {
throw new SystemError(
'The local egress guard self-check is not available on this device host, so this session cannot ' +
'verify that the guard is in effect. The device host image is missing bin/egress-guard-check; ' +
'this is a service problem, please contact support.'
);
}
let output: string;
try {
const result = await spawn('xcrun', ['simctl', 'spawn', udid, resolvedCheck, '--mode', mode], {
env,
stdio: 'pipe',
});
output = result.stdout.trim();
} catch (err) {
const failed = err as { status?: number | null; stdout?: string; stderr?: string };
const detail = [failed.stdout, failed.stderr].filter(Boolean).join('\n').trim();
throw new SystemError(
'The local egress guard is not in effect in the Simulator: the self-check run right after boot ' +
`failed${failed.status != null ? ` (exit ${failed.status})` : ''}. Connections that bypass the ` +
'system proxy would leave from the device host, so the session was stopped. Retry the session; if it ' +
`keeps failing, please contact support.${detail ? ` Self-check output: ${detail}` : ''}`,
{ cause: err }
);
}
logger.info(`Local egress guard verified in the Simulator: ${output || 'self-check passed'}.`);
await reportLocalEgressGuardCoverageAsync({ env, logger });
}
/**
* Attribute guard events to the step that runs the session from now on. The
* relay starts under the simulator boot step's logger; once the session step
* takes over, its lines should appear under that step in the job log, next
* to the rest of the session's output, rather than under a step that already
* finished.
*/
export function rebindLocalEgressGuardRelays(logger: bunyan): void {
for (const { relay } of activeRelays.values()) {
relay.setLogger(logger);
}
}
/** Stop relaying and write each relay's summary; called from the session cleanup. */
export async function stopLocalEgressGuardRelaysAsync(logger: bunyan): Promise<void> {
const relays = [...activeRelays.values()];
activeRelays.clear();
for (const { tailer, relay } of relays) {
try {
await tailer.stopAsync();
} catch (err) {
logger.warn(
{ err },
'Local egress guard: could not read the last events from the guard log.'
);
}
relay.logSummary();
}
}
@@ -0,0 +1,59 @@
import { type BuildFunction, type BuildStepEnv } from '@expo/steps';
import {
buildEgressRemoteConfigFields,
monitorLocalEgressAsync,
readLocalEgressHandoffAsync,
stopLocalEgressResourcesAsync,
} from './localEgress';
import { rebindLocalEgressGuardRelays } from './localEgressGuard';
import { uploadRemoteSessionConfigAsync } from './remoteDeviceRunSession';
/** Release the pre-boot egress resources even if controller startup or teardown fails. */
export function withLocalEgressSession(
fn: NonNullable<BuildFunction['fn']>
): NonNullable<BuildFunction['fn']> {
return async (context, args) => {
try {
args.signal?.throwIfAborted();
await fn(context, args);
} finally {
await stopLocalEgressResourcesAsync(context.logger);
}
};
}
/** All simulator controllers publish the same egress handoff and run the same monitor. */
export async function uploadRemoteSessionConfigWithLocalEgressAsync({
env,
signal,
...options
}: Parameters<typeof uploadRemoteSessionConfigAsync>[0] & {
env: BuildStepEnv;
signal?: AbortSignal;
}): Promise<void> {
// Written by start_local_egress before simulator boot; absent for ordinary sessions.
const localEgress = await readLocalEgressHandoffAsync();
signal?.throwIfAborted();
await uploadRemoteSessionConfigAsync({
...options,
remoteConfig: { ...options.remoteConfig, ...buildEgressRemoteConfigFields(localEgress) },
});
if (localEgress && !signal?.aborted) {
// Guard refusals from here on show up under this step in the job log,
// alongside the monitor's reports, instead of under the boot step.
rebindLocalEgressGuardRelays(options.logger);
options.logger.info(
'Local egress: waiting for the EAS CLI egress client to connect. Proxied HTTP(S) ' +
'requests are unavailable until it does.'
);
// The monitor also observes the registered resources' lifetime signal, which
// withLocalEgressSession aborts on success, failure, or cancellation.
void monitorLocalEgressAsync({
port: localEgress.port,
env,
logger: options.logger,
signal: signal ?? new AbortController().signal,
});
}
}
@@ -17,14 +17,19 @@ import { CustomBuildContext } from '../../customBuildContext';
import { Sentry } from '../../sentry';
import { sleepAsync } from '../../utils/retry';
import { turtleFetch } from '../../utils/turtleFetch';
import { SERVE_SIM_STATE_DIR, readServeSimServersAsync } from './serveSimMetricsRecorder';
const XCODE_DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer';
const SERVE_SIM_PACKAGE_SPEC = '@expo/serve-sim@latest';
const SERVE_SIM_HOST = '127.0.0.1';
const SERVE_SIM_MAX_DIMENSION = '1280';
const WEB_PREVIEW_HOST = '127.0.0.1';
const SERVE_SIM_PACKAGE_NAME = '@expo/serve-sim';
const SERVE_SIM_MAX_DIMENSION = '960';
const SERVE_SIM_MJPEG_QUALITY = '0.55';
const SERVE_SIM_VIDEO_BITRATE = '3000000';
const SERVE_SIM_VIDEO_BITRATE = '6000000';
const SERVE_SIM_VIDEO_FPS = '60';
const EXPO_DEVICE_HUB_PACKAGE_NAME = 'expo-device-hub';
const EXPO_DEVICE_HUB_MAX_DIMENSION = '960';
const EXPO_DEVICE_HUB_VIDEO_BITRATE = '6000000';
const EXPO_DEVICE_HUB_VIDEO_FPS = '60';
const START_DEVICE_RUN_SESSION_MUTATION = graphql(`
mutation StartDeviceRunSession($deviceRunSessionId: ID!, $remoteConfig: JSONObject!) {
@@ -293,10 +298,9 @@ async function sleepUntilAbortedAsync(
}
}
// Argent encodes screen recordings by piping simulator frames into `ffmpeg`,
// which it resolves from PATH. The tool-server inherits this step's env, so
// spawning ffmpeg resolves against the same PATH argent will search: it rejects
// with ENOENT when the binary is absent, and running it also proves it works.
// Device-session tools resolve `ffmpeg` from PATH. Spawning it with the step's
// environment rejects with ENOENT when the binary is absent, and running it also
// proves that the installed binary works.
async function isFfmpegAvailableAsync(env: BuildStepEnv): Promise<boolean> {
return (await asyncResult(spawn('ffmpeg', ['-version'], { env }))).ok;
}
@@ -329,14 +333,15 @@ async function installFfmpegWithAptAsync({
await spawn('sudo', ['apt-get', 'install', '-y', 'ffmpeg'], { env: aptEnv, logger });
}
let ffmpegSetupPromise: Promise<void> | undefined;
/**
* Install ffmpeg when the runtime does not already provide it, so argent's
* `screen-recording-start` tool can encode a video. The worker images do not
* ship ffmpeg yet, so without this the tool fails with "`ffmpeg` was not found
* on PATH" — on macOS (iOS simulators) and Linux (Android emulators) alike.
* Install ffmpeg when the runtime does not already provide it. Device-session
* tools use it for video encoding on macOS (iOS simulators) and Linux (Android
* emulators) alike, but the worker images do not ship it yet.
*
* Best-effort by design: screen recording is one optional argent tool, so a
* failure here is logged and the session continues without it.
* Best-effort by design: a failure here is logged and the session continues
* without FFmpeg-dependent features.
*
* The whole body is wrapped because the caller runs this in the background with
* `void`. There is no unhandledRejection handler in the worker, so a rejection
@@ -344,7 +349,7 @@ async function installFfmpegWithAptAsync({
* `spawn` is not an async function and can throw synchronously, which
* `asyncResult` cannot catch — it only wraps an already-created promise.
*/
export async function ensureFfmpegInstalledAsync({
async function ensureFfmpegInstalledAsync({
runtimePlatform,
env,
logger,
@@ -363,7 +368,7 @@ export async function ensureFfmpegInstalledAsync({
logger.info(
`ffmpeg is not installed, installing it with ${
isDarwin ? 'Homebrew' : 'apt'
} for argent screen recording.`
} for the device session.`
);
if (isDarwin) {
await installFfmpegWithHomebrewAsync({ env, logger });
@@ -373,16 +378,41 @@ export async function ensureFfmpegInstalledAsync({
logger.info('Installed ffmpeg.');
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not install ffmpeg for argent screen recording', error, {
Sentry.capture('Could not install ffmpeg for the device session', error, {
level: 'warning',
});
logger.warn(
{ err: error },
'Could not install ffmpeg. Argent screen recording will not work in this session.'
'Could not install ffmpeg. FFmpeg-dependent features may not work in this session.'
);
}
}
export async function ensureFfmpegInstalledOnceAsync({
runtimePlatform,
env,
logger,
}: {
runtimePlatform: BuildRuntimePlatform;
env: BuildStepEnv;
logger: bunyan;
}): Promise<void> {
if (ffmpegSetupPromise) {
await ffmpegSetupPromise;
return;
}
const setupPromise = ensureFfmpegInstalledAsync({ runtimePlatform, env, logger });
ffmpegSetupPromise = setupPromise;
try {
await setupPromise;
} finally {
if (ffmpegSetupPromise === setupPromise) {
ffmpegSetupPromise = undefined;
}
}
}
const TurnIceServersResponseSchema = z.object({
data: z.object({
iceServers: TurnIceServersSchema,
@@ -390,11 +420,12 @@ const TurnIceServersResponseSchema = z.object({
});
/**
* Translate Cloudflare ICE servers into serve-sim CLI flags: `--stun-url` (the
* Translate Cloudflare ICE servers into web preview CLI flags: `--stun-url` (the
* credential-less entries) and `--turn-url`/`--turn-username`/`--turn-credential`
* (the entry carrying the short-lived credentials).
* (the entry carrying the short-lived credentials). serve-sim and expo-device-hub
* intentionally expose the same ICE flag contract.
*/
export function turnIceServersToServeSimArgs(iceServers: TurnIceServers): string[] {
export function turnIceServersToWebPreviewArgs(iceServers: TurnIceServers): string[] {
const stunUrls = iceServers
.filter(server => !server.username && !server.credential)
.flatMap(server => server.urls);
@@ -420,14 +451,14 @@ export function turnIceServersToServeSimArgs(iceServers: TurnIceServers): string
/**
* Fetch short-lived Cloudflare TURN ICE servers for this job run from www
* (minted on demand, mirroring how the worker fetches project clone URLs) and
* translate them into serve-sim CLI flags.
* translate them into web preview CLI flags.
*
* Best-effort: on any failure we log and return [] so serve-sim falls back to
* its built-in P2P/STUN behavior. The credential is passed to serve-sim as a
* process arg and deliberately not logged (turtle-spawn never logs argv and the
* worker is single-tenant).
* Best-effort: on any failure we log and return [] so the preview server falls
* back to its built-in P2P/STUN behavior. The credential is passed as a process
* arg and deliberately not logged (turtle-spawn never logs argv and the worker
* is single-tenant).
*/
export async function fetchServeSimTurnArgsAsync(
export async function fetchWebPreviewTurnArgsAsync(
ctx: CustomBuildContext,
{ env, logger }: { env: BuildStepEnv; logger: bunyan }
): Promise<string[]> {
@@ -456,9 +487,9 @@ export async function fetchServeSimTurnArgsAsync(
);
const { data } = TurnIceServersResponseSchema.parse(await response.json());
const args = turnIceServersToServeSimArgs(data.iceServers);
const args = turnIceServersToWebPreviewArgs(data.iceServers);
if (args.length > 0) {
logger.info('Configured serve-sim with Cloudflare TURN ICE servers.');
logger.info('Configured the web preview with Cloudflare TURN ICE servers.');
}
return args;
} catch (err) {
@@ -466,7 +497,7 @@ export async function fetchServeSimTurnArgsAsync(
Sentry.capture('Could not fetch Cloudflare TURN ICE servers', error, { level: 'warning' });
logger.warn(
{ err: error },
'Could not fetch Cloudflare TURN ICE servers; serve-sim will fall back to P2P/STUN.'
'Could not fetch Cloudflare TURN ICE servers; the web preview will fall back to P2P/STUN.'
);
return [];
}
@@ -596,22 +627,33 @@ export function metricsCorsOriginToServeSimArgs(env: BuildStepEnv): string[] {
return args;
}
function createServeSimPackageSpec(packageVersion: string | undefined): string {
return `${SERVE_SIM_PACKAGE_NAME}@${packageVersion ?? 'latest'}`;
}
function createExpoDeviceHubPackageSpec(packageVersion: string | undefined): string {
return `${EXPO_DEVICE_HUB_PACKAGE_NAME}@${packageVersion ?? 'latest'}`;
}
export function createServeSimArgs({
port,
turnArgs = [],
metricsCorsArgs = [],
packageVersion,
}: {
port: number;
turnArgs?: string[];
metricsCorsArgs?: string[];
packageVersion?: string;
}): string[] {
return [
'--yes',
SERVE_SIM_PACKAGE_SPEC,
createServeSimPackageSpec(packageVersion),
'--port',
String(port),
'--host',
SERVE_SIM_HOST,
WEB_PREVIEW_HOST,
'--require-token',
'--transport',
'webrtc',
'--webrtc-codec',
@@ -629,44 +671,84 @@ export function createServeSimArgs({
];
}
async function findAvailablePortAsync(): Promise<number> {
export function createExpoDeviceHubArgs({
port,
turnArgs = [],
packageVersion,
}: {
port: number;
turnArgs?: string[];
packageVersion?: string;
}): string[] {
return [
'--yes',
createExpoDeviceHubPackageSpec(packageVersion),
'--port',
String(port),
'--host',
WEB_PREVIEW_HOST,
'--platform',
'android',
'--transport',
'webrtc',
'--webrtc-codec',
'h264',
'--webrtc-ice-policy',
'all',
'--max-dimension',
EXPO_DEVICE_HUB_MAX_DIMENSION,
'--video-bitrate',
EXPO_DEVICE_HUB_VIDEO_BITRATE,
'--video-fps',
EXPO_DEVICE_HUB_VIDEO_FPS,
'--hide-sidebar',
'--hide-boot-device',
...turnArgs,
];
}
export async function findAvailablePortAsync(): Promise<number> {
const server = createServer();
server.unref();
await new Promise<void>((resolve, reject) => {
server.once('error', reject);
server.listen(0, SERVE_SIM_HOST, () => resolve());
server.listen(0, WEB_PREVIEW_HOST, () => resolve());
});
const address = server.address();
await new Promise<void>((resolve, reject) => {
server.close(err => (err ? reject(err) : resolve()));
});
if (!address || typeof address === 'string') {
throw new SystemError('Could not allocate a local port for serve-sim.');
throw new SystemError('Could not allocate a local port for the web preview.');
}
return address.port;
}
const ServeSimReadyResponseSchema = z.object({
const WebPreviewReadyResponseSchema = z.object({
status: z.literal('ready'),
device: z.string(),
});
export async function waitForServeSimReadyAsync({
serveSim,
export async function waitForWebPreviewReadyAsync({
previewServer,
serverName,
port,
timeoutMs,
}: {
serveSim: Pick<DetachedProcessHandle, 'pid' | 'getOutput'>;
previewServer: Pick<DetachedProcessHandle, 'pid' | 'getOutput'>;
serverName: string;
port: number;
timeoutMs: number;
}): Promise<void> {
const readyUrl = `http://${SERVE_SIM_HOST}:${port}/readyz`;
}): Promise<string> {
const readyUrl = `http://${WEB_PREVIEW_HOST}:${port}/readyz`;
const deadline = Date.now() + timeoutMs;
let lastError: unknown;
while (Date.now() < deadline) {
if (serveSim.pid !== undefined && !isProcessRunning(serveSim.pid)) {
if (previewServer.pid !== undefined && !isProcessRunning(previewServer.pid)) {
throw new SystemError(
`serve-sim exited before becoming ready. Last output:\n${serveSim.getOutput() || '<empty>'}`
`${serverName} exited before becoming ready. Last output:\n${
previewServer.getOutput() || '<empty>'
}`
);
}
try {
@@ -674,52 +756,69 @@ export async function waitForServeSimReadyAsync({
retries: 0,
timeout: 2_000,
});
ServeSimReadyResponseSchema.parse(await response.json());
return;
const ready = WebPreviewReadyResponseSchema.parse(await response.json());
return ready.device;
} catch (error) {
lastError = error;
}
await sleepAsync(1_000);
}
throw new SystemError(
`Timed out waiting for serve-sim readiness at ${readyUrl}${
`Timed out waiting for ${serverName} readiness at ${readyUrl}${
lastError instanceof Error ? `: ${lastError.message}` : ''
}. Last output:\n${serveSim.getOutput() || '<empty>'}`
}. Last output:\n${previewServer.getOutput() || '<empty>'}`
);
}
export type ServeSimPreviewHandle = {
export type DeviceWebPreviewHandle = {
previewUrl: string;
/** Session token gating the preview. Only serve-sim mints one. */
previewToken?: string;
stopAsync: () => Promise<void>;
};
export async function startServeSimWithTunnelAsync(
export type ServeSimPreviewHandle = DeviceWebPreviewHandle;
async function startWebPreviewWithTunnelAsync(
ctx: CustomBuildContext,
{
baseDomain,
env,
logger,
timeoutMs,
serverName,
packageSpec,
createArgs,
readPreviewTokenAsync,
}: {
baseDomain: string;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
serverName: string;
packageSpec: string;
createArgs: (port: number, turnArgs: string[]) => string[];
readPreviewTokenAsync?: (device: string) => Promise<string>;
}
): Promise<ServeSimPreviewHandle> {
): Promise<DeviceWebPreviewHandle> {
const port = await findAvailablePortAsync();
logger.info(`Launching ${SERVE_SIM_PACKAGE_SPEC} on ${SERVE_SIM_HOST}:${port}.`);
const turnArgs = await fetchServeSimTurnArgsAsync(ctx, { env, logger });
const metricsCorsArgs = metricsCorsOriginToServeSimArgs(env);
const serveSim = spawnDetached({
logger.info(`Launching ${packageSpec} on ${WEB_PREVIEW_HOST}:${port}.`);
const turnArgs = await fetchWebPreviewTurnArgsAsync(ctx, { env, logger });
const previewServer = spawnDetached({
command: 'npx',
args: createServeSimArgs({ port, turnArgs, metricsCorsArgs }),
args: createArgs(port, turnArgs),
env,
});
try {
logger.info('Waiting for serve-sim to become ready.');
await waitForServeSimReadyAsync({ serveSim, port, timeoutMs });
logger.info(`Waiting for ${serverName} to become ready.`);
const device = await waitForWebPreviewReadyAsync({
previewServer,
serverName,
port,
timeoutMs,
});
const previewToken = await readPreviewTokenAsync?.(device);
const tunnel = await startNgrokTunnelAsync({
port,
subdomainPrefix: 'web-preview',
@@ -729,21 +828,127 @@ export async function startServeSimWithTunnelAsync(
});
return {
previewUrl: tunnel.url,
previewToken,
stopAsync: async () => {
const results = await Promise.allSettled([tunnel.stopAsync(), serveSim.stopAsync()]);
const results = await Promise.allSettled([tunnel.stopAsync(), previewServer.stopAsync()]);
for (const result of results) {
if (result.status === 'rejected') {
logger.warn({ err: result.reason }, 'Could not stop a serve-sim preview resource.');
logger.warn({ err: result.reason }, `Could not stop a ${serverName} preview resource.`);
}
}
},
};
} catch (error) {
await serveSim.stopAsync();
await previewServer.stopAsync();
throw error;
}
}
export async function readServeSimPreviewTokenAsync(
udid: string,
stateDir: string = SERVE_SIM_STATE_DIR
): Promise<string | undefined> {
const servers = await readServeSimServersAsync(stateDir);
return servers.find(server => server.udid === udid)?.token;
}
export async function startServeSimWithTunnelAsync(
ctx: CustomBuildContext,
{
baseDomain,
env,
logger,
timeoutMs,
packageVersion,
}: {
baseDomain: string;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
packageVersion?: string;
}
): Promise<ServeSimPreviewHandle> {
const metricsCorsArgs = metricsCorsOriginToServeSimArgs(env);
return await startWebPreviewWithTunnelAsync(ctx, {
baseDomain,
env,
logger,
timeoutMs,
serverName: 'serve-sim',
packageSpec: createServeSimPackageSpec(packageVersion),
createArgs: (port, turnArgs) =>
createServeSimArgs({ port, turnArgs, metricsCorsArgs, packageVersion }),
readPreviewTokenAsync: async device => {
const previewToken = await readServeSimPreviewTokenAsync(device);
if (!previewToken) {
// A serve-sim that does not know --require-token fails earlier, in the readiness check, so
// reaching here means it started and left no token in its state file.
throw new SystemError(
`serve-sim became ready but wrote no session token for device ${device}. The preview is ` +
'on a public tunnel and would be reachable without one, so the session cannot continue. ' +
'This usually means the state file was not written as expected; retry the session, and ' +
'report it if it repeats.'
);
}
return previewToken;
},
});
}
export async function startExpoDeviceHubWithTunnelAsync(
ctx: CustomBuildContext,
{
runtimePlatform,
baseDomain,
env,
logger,
timeoutMs,
packageVersion,
}: {
runtimePlatform: BuildRuntimePlatform;
baseDomain: string;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
packageVersion?: string;
}
): Promise<DeviceWebPreviewHandle> {
if (runtimePlatform === BuildRuntimePlatform.LINUX) {
await ensureFfmpegInstalledOnceAsync({ runtimePlatform, env, logger });
}
return await startWebPreviewWithTunnelAsync(ctx, {
baseDomain,
env,
logger,
timeoutMs,
serverName: 'expo-device-hub',
packageSpec: createExpoDeviceHubPackageSpec(packageVersion),
createArgs: (port, turnArgs) => createExpoDeviceHubArgs({ port, turnArgs, packageVersion }),
});
}
export async function startDeviceWebPreviewWithTunnelAsync(
ctx: CustomBuildContext,
{
runtimePlatform,
...options
}: {
runtimePlatform: BuildRuntimePlatform;
baseDomain: string;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
packageVersion?: string;
}
): Promise<DeviceWebPreviewHandle> {
switch (runtimePlatform) {
case BuildRuntimePlatform.DARWIN:
return await startServeSimWithTunnelAsync(ctx, options);
case BuildRuntimePlatform.LINUX:
return await startExpoDeviceHubWithTunnelAsync(ctx, { ...options, runtimePlatform });
}
}
export type NgrokTunnelHandle = {
url: string;
stopAsync: () => Promise<void>;
@@ -0,0 +1,106 @@
import { SystemError } from '@expo/eas-build-job';
import { type bunyan } from '@expo/logger';
import { setTimeout as setTimeoutAsync } from 'node:timers/promises';
import WebSocket from 'ws';
export interface SandboxDaemonOptions {
credential: string;
serverUrl: string;
reconnectDelayMs: number;
logger: bunyan;
signal?: AbortSignal;
}
export interface SandboxDaemon {
ready: Promise<void>;
stopAsync(): Promise<void>;
}
export async function startSandboxDaemonAsync(
options: SandboxDaemonOptions
): Promise<SandboxDaemon> {
options.signal?.throwIfAborted();
let socket: WebSocket | undefined;
const abortController = new AbortController();
let hasConnected = false;
let resolveConnected!: () => void;
let rejectConnected!: (error: Error) => void;
const connected = new Promise<void>((resolve, reject) => {
resolveConnected = resolve;
rejectConnected = reject;
});
const stop = (): void => {
abortController.abort();
socket?.close(1000, 'sandbox stopped');
};
options.signal?.addEventListener('abort', stop, { once: true });
const connectionLoop = (async () => {
while (!abortController.signal.aborted) {
try {
socket = new WebSocket(new URL('/sandbox/connect', options.serverUrl), {
handshakeTimeout: 10_000,
headers: { Authorization: `Bearer ${options.credential}` },
});
await waitForOpen(socket);
options.logger.info('Sandbox MCP server connected.');
hasConnected = true;
resolveConnected();
await waitForClose(socket);
} catch (error: any) {
if (!hasConnected) {
const message = `Sandbox MCP server connection failed: ${error?.message ?? 'unknown error'}`;
if (!abortController.signal.aborted) {
options.logger.error({ err: error }, message);
}
rejectConnected(new SystemError(message, { cause: error }));
return;
}
if (!abortController.signal.aborted) {
options.logger.warn(
{ err: error },
`Sandbox MCP server connection failed: ${error?.message ?? 'unknown error'}`
);
}
}
if (!abortController.signal.aborted) {
try {
await setTimeoutAsync(options.reconnectDelayMs, undefined, {
signal: abortController.signal,
});
} catch (error) {
if (!abortController.signal.aborted) {
throw error;
}
}
}
}
})();
return {
ready: connected,
async stopAsync(): Promise<void> {
options.signal?.removeEventListener('abort', stop);
stop();
await connectionLoop;
},
};
}
function waitForOpen(socket: WebSocket): Promise<void> {
return new Promise((resolve, reject) => {
socket.once('open', resolve);
socket.once('error', reject);
socket.once('close', () => reject(new Error('Sandbox MCP server closed before it connected.')));
});
}
function waitForClose(socket: WebSocket): Promise<void> {
if (socket.readyState === WebSocket.CLOSING || socket.readyState === WebSocket.CLOSED) {
return Promise.resolve();
}
return new Promise((resolve, reject) => {
socket.once('close', resolve);
socket.once('error', reject);
});
}
Loaded 100 of 379 files, more files were not shown because too many files have changed in this diff. Show more