Compare commits

..
36 Commits
Author SHA1 Message Date
Expo CI 74292d2ace v23.2.0 2026-08-31 20:25:25 +00:00
Szymon Dziedzic b0b7cad316 [eas-cli] Warn about Android simulator parity (#4320)
* [eas-cli] warn about Android simulator parity

* [eas-cli] add Android simulator warning changelog
2026-08-31 22:22:53 +02:00
expo-tuft[bot]andtchayen 926887d14e [eas-cli] Add --tag to simulator session commands (#4318)
Lets projects with several app variants group and filter their sessions.

Co-authored-by: expo-tuft[bot] <288127324+expo-tuft[bot]@users.noreply.github.com>
Co-authored-by: tchayen <[email protected]>
2026-08-31 22:40:52 +03:00
Gabe DebesandTomasz Mazur 2e6bb2d276 [eas-cli] Add --ssh to workflow:run (#4033)
* [eas-cli] add --ssh to workflow:run

* [eas-cli] Restore WorkflowRunSshInput types dropped by staging regen

Staging schema does not include the --ssh input yet (universe #29128),
but the CLI stack already depends on it. Keep the types until that ships.

* Regenerate GraphQL

* Dedupe changelog

---------

Co-authored-by: Tomasz Mazur <[email protected]>
2026-08-31 17:30:17 +02:00
Gabe DebesandTomasz Mazur 30656509ac [eas-cli] Add workflow:ssh command (#4032)
* [eas-cli] add workflow:ssh command

* Poll for an open SSH session before sleeping.

Avoids a needless wait when the first connect-info fetch is already ready.

* Update SSH wait test for poll-before-sleep.

The wait loop now polls first, so keep one pending poll before ready.

* Cover WorkflowJobSshQuery error and turtleBuild session paths.

* [eas-cli] Regen GraphQL types from staging (TurtleSshSession)

* [eas-cli] address workflow:ssh review feedback

* [eas-cli] drop the GET_BUILD check from sshRequested

* [eas-cli] accept a job run id in workflow:ssh

* Use URL to parse the host

* Regenerate GraphQL

* Move changelog entry

---------

Co-authored-by: Tomasz Mazur <[email protected]>
2026-08-31 17:30:17 +02:00
66e3f1ad7b [build-tools] Use expo-device-hub for Android web previews (#4304)
Co-authored-by: szdziedzic <[email protected]>
Co-authored-by: Codex <[email protected]>
2026-08-31 15:49:49 +02:00
Tomasz Mazur 9aadf2d467 [eas-cli] Stream workflow run logs in real time (#4228)
* [eas-cli] Add centrifuge and ws to deps

* [eas-cli] Stream workflow job logs while waiting for a run

* [eas-cli] Add changelog entry for realtime workflow logs

* [eas-cli] Make msg optional and fix flickering

* [eas-cli] Fetch job logs without casing on job type

* [eas-cli] Ingest realtime incrementally instead of rebuilding full logs on every message (#4293)
2026-08-31 13:38:38 +00:00
Stanisław Chmiela ba80e57753 [eas-cli] Update GraphQL schema for web preview sessions (#4315)
Updated GraphQL and adjusted code.
2026-08-31 15:20:33 +02:00
Expo CI 50c76f41e3 update CHANGELOG.md and bump STAGING and PRODUCTION in cli-versions.json 2026-08-31 11:27:55 +00:00
Expo CI 0658d58efd v23.1.0 2026-08-31 11:24:34 +00:00
Phil Pluckthun 76a700c45c Remove verify step (#4310) 2026-08-29 18:07:45 +01:00
Szymon Dziedzic 5ded193ccf [eas-cli] Fix v23.0.0 changelog (#4309) 2026-08-29 11:24:10 +02:00
Brent Vatne 93420d1fa0 [eas-cli] Add prebuild fallback regression tests (#4285)
* [eas-cli] Add prebuild fallback regression tests

* Update changelog for prebuild fallback tests
2026-08-28 15:45:29 -07:00
Brent Vatne add787f625 [local-build-plugin] Replace bunyan with @expo/bunyan (#4284)
* [local-build-plugin] Replace bunyan with @expo/bunyan

* Update changelog for @expo/bunyan
2026-08-28 15:45:28 -07:00
Brent Vatne ec2c2aca5e [steps] Replace lodash.get with a local helper (#4283)
* [steps] Replace lodash.get with a local helper

* Update changelog for lodash.get removal
2026-08-28 15:45:28 -07:00
Brent Vatne 1e86c48f48 [eas-cli][steps] Bump uuid to v11 (#4282)
* [eas-cli][steps] Bump uuid to v11

* Update changelog for uuid v11
2026-08-28 15:45:27 -07:00
Brent Vatne 545c2dbf9b [eas-cli] Bump minizlib to 3.1.0 (#4281) 2026-08-28 15:45:27 -07:00
Brent Vatne 3cb029147e [eas-cli] Prompt for Expo SDK version in eas new (#4301) 2026-08-28 12:05:38 -07:00
expo-tuft[bot]andabbanmustafa d48cfbba69 [build-tools] Normalize extracted project permissions (#4287)
* [build-tools] Normalize extracted project permissions

Co-authored-by: abbanmustafa <[email protected]>

* [build-tools] Add project permission fix changelog

Co-authored-by: abbanmustafa <[email protected]>

* [build-tools] Log project permission normalization

Co-authored-by: abbanmustafa <[email protected]>

---------

Co-authored-by: expo-tuft[bot] <288127324+expo-tuft[bot]@users.noreply.github.com>
Co-authored-by: abbanmustafa <[email protected]>
2026-08-28 12:51:33 -04:00
Robin Gullo 07b5c0daa3 [build-tools] Support pnpm install --filter to install scoped dependencies (#4302)
# Why

`EAS_YARN_FOCUS_WORKSPACE` ([expo/eas-build#598](https://github.com/expo/eas-build/pull/598)) and `EAS_BUN_FILTER_WORKSPACE` ([expo/eas-cli#4292](https://github.com/expo/eas-cli/pull/4292)) let monorepos install only the dependencies of the app being built, so `node_modules` stays slim, the install is faster, and the fingerprint matches what you get locally. pnpm can do the same thing with [`pnpm install --filter <workspace>`](https://pnpm.io/filtering), but there is no way to reach it from EAS today — the pnpm branch always installs every workspace.

# How

Set it in the build profile's `env` in `eas.json`:

```json
{
  "build": {
    "production": {
      "env": {
        "EAS_PNPM_FILTER_WORKSPACE": "my-app"
      }
    }
  }
}
```

The install step then runs `pnpm install --filter my-app`. This is the pnpm counterpart of `EAS_YARN_FOCUS_WORKSPACE`, introduced in expo/eas-build#598, and `EAS_BUN_FILTER_WORKSPACE`, introduced in expo/eas-cli#4292.

# Test Plan

Added two unit tests for the flag set/unset — `yarn jest-unit src/common/__tests__/installDependencies.test.ts --runInBand` in `packages/build-tools`:

```
Test Suites: 1 passed, 1 total
Tests:       9 passed, 9 total
```

Also ran the resulting commands by hand with pnpm 10.32.1, in a workspace with `pkg-a` (depends on `is-odd`), `pkg-b` (depends on `lodash`) and an in-sync `pnpm-lock.yaml`:

| command | installed |
| --- | --- |
| `pnpm install --frozen-lockfile` | both `packages/a/node_modules/is-odd` and `packages/b/node_modules/lodash` |
| `pnpm install --filter pkg-a --frozen-lockfile` | only `packages/a/node_modules/is-odd`, `pnpm-lock.yaml` unchanged |
2026-08-28 18:32:10 +02:00
David Mokos b78c84af2c updates agent cli detector to 0.1.7 (#4272) 2026-08-28 17:31:20 +02:00
Phil Pluckthun dfaa2df90d [eas-cli] Add dynamic retries for deploy command (#4299) 2026-08-28 16:12:40 +01:00
Expo CI 2e2aaa4ac9 v23.0.0 2026-08-28 11:15:00 +00:00
Szymon Dziedzic 3ef0f74110 [eas-cli] Rename serve-sim session type to web-preview-only (#4298)
* [eas-cli] Rename simulator web preview session type

* [eas-cli] Add simulator flag rename changelog

* [eas-cli] Clarify preview-only simulator type
2026-08-28 13:12:33 +02:00
Stanisław Chmiela 51ba7198d7 [build-tools] Ignore minimum release age for internal npx (#4296)
# Why

When a user defines a min package age policy a fresh release of `eas-cli` may fail.

Reported in [the internal support thread](https://exponent-internal.slack.com/archives/C09H6RFAX27/p1787835815444139).

# How

Added `NPM_CONFIG_MIN_RELEASE_AGE=0` to NPM environment to override min release age to 0.

# Test Plan

Codex verified with different npx versions.
2026-08-27 18:19:03 +00:00
Szymon Dziedzic e10e6a2235 [build-tools] Lower serve-sim preview resolution (#4297)
* [build-tools] Lower serve-sim preview resolution

* [build-tools] Add serve-sim resolution changelog
2026-08-27 19:10:56 +02:00
Konrad Armatys 17f84cf7d6 [build-tools] Support bun install --filter to install scoped dependencies (#4292)
# Why

`EAS_YARN_FOCUS_WORKSPACE` ([expo/eas-build#598](https://github.com/expo/eas-build/pull/598)) lets a monorepo install only the dependencies of the app being built, so `node_modules` stays slim, the install is faster, and the fingerprint matches what you get locally. Bun can do the same thing with `bun install --filter <workspace>` ([oven-sh/bun#16093](https://github.com/oven-sh/bun/pull/16093)), but there is no way to reach it from EAS today — the bun branch always installs every workspace.

# How

Set it in the build profile's `env` in `eas.json`:

```json
{
  "build": {
    "production": {
      "env": {
        "EAS_BUN_FILTER_WORKSPACE": "my-app"
      }
    }
  }
}
```

The install step then runs `bun install --filter my-app`. This is the bun counterpart of `EAS_YARN_FOCUS_WORKSPACE`, introduced in expo/eas-build#598.

# Test Plan

Added two unit tests for the flag set/unset — `yarn jest-unit src/common/__tests__/installDependencies.test.ts` in `packages/build-tools`:

```
Test Suites: 1 passed, 1 total
Tests:       7 passed, 7 total
```

Also ran the resulting commands by hand, in a workspace with `pkg-a` (depends on `is-odd`), `pkg-b` (depends on `lodash`) and an in-sync `bun.lock`:

| command | installed |
| --- | --- |
| `bun install --frozen-lockfile` | 4 packages, `packages/b/node_modules/lodash` present |
| `bun install --filter pkg-a --frozen-lockfile` | 2 packages, only `packages/a/node_modules/is-odd`, `bun.lock` unchanged |
2026-08-27 18:41:24 +02:00
Mustafa Abban f7d9b7b297 [build-tools] support remote CocoaPods caching (#4266)
* [build-tools] Cache CocoaPods dependencies between iOS builds

* restore/save cache

* remove CocoaPods download cache

* remove no-lockfile cache key
2026-08-27 10:20:11 -04:00
Stanisław Chmiela b287d7a4e0 [worker] Use cache API for the legacy cache (#3860)
The legacy cache does not work in `workflow-orchestration`.

This rewrites the `CacheManager` to the new/not-so-new Turtle caches API.

## Summary
- replace worker build cache restore/save GCS config URLs with turtle cache download/upload helpers
- derive stable build cache keys without mutating job.cache and preserve clear for force uploads
- keep existing tar archive restore/save semantics and add CacheManager unit coverage

## Test plan
- cd packages/worker && yarn test CacheManager.test.ts
- cd packages/worker && yarn typecheck
- cd packages/build-tools && yarn typecheck
- git diff --check
2026-08-27 15:44:28 +02:00
Douglas Lowder d9d1b91fa5 [ci] Bump both STAGING and PRODUCTION cli-versions in release workflow (#4290) 2026-08-27 04:16:20 -07:00
Expo CI a7436edb50 update CHANGELOG.md and bump STAGING in cli-versions.json 2026-08-26 23:54:31 +00:00
Expo CI 89fe2cb3e1 v22.6.0 2026-08-26 23:49:36 +00:00
Szymon Dziedzic 0e8e8cc0aa [build-tools] Increase serve-sim bitrate to 6 Mbps (#4291)
* [build-tools] Increase serve-sim bitrate to 16 Mbps

* [build-tools] Add serve-sim bitrate changelog entry

* [build-tools] Set serve-sim bitrate to 6 Mbps
2026-08-27 01:45:23 +02:00
Gabe Debes 6d7cef80a3 [eas-cli] add experimental support to resource class on simulator sessions (#4268)
* [eas-cli] let simulator sessions request a large resource class

* [eas-cli] add changelog for hidden --resource-class

* [eas-cli] call --resource-class experimental in the changelog

* [eas-cli] oxfmt simulator resource-class test

* [eas-cli] regenerate graphql for resource class
2026-08-26 23:34:56 +00:00
Gabe Debes 5a744b73f5 [worker] Wire SSH session into build (#4031)
* [worker] wire ssh session into build

* Rename SSH phase sessionPromise to done.

Same anti-flatten object wrapper; clearer that callers await tunnel teardown.

* Cover SSH phase supervise callbacks and teardown failure paths.

* [worker] Stop baking upterm into the worker package

build-tools resolves upterm from PATH or downloads darwin-arm64 /
linux-amd64 from gs://turtle-v2/upterm at runtime.

* [worker] align ssh phase with build-tools api

* [worker] build the ssh target from the job type
2026-08-26 16:25:12 -07:00
Gabe Debes b8b53e262a [build-tools] Add worker-side SSH session helper (#4030)
* [build-tools] add worker-side ssh session helper

* Keep host.connectionConfig current and simplify SSH reconnect.

Update connectionConfig on every redial, linearize ensureConnectedAsync, and
parse upterm session objects without a JSON stringify round-trip.

* Reuse SshSettings instead of inlining idleTimeout/relay fields.

* Cover remaining SSH helper branches in unit tests.

Invalid upterm host URLs, missing process pid, stdout chunks, unknown
client counts during the job, and turtleBuild targets.

* Format unknownGraceMs ternary for oxfmt.

* [build-tools] Drop dup changelog entry; redact upterm exit logs

* [build-tools] gitignore baked upterm binaries

* [build-tools] Cover SSH helpers; drop upterm gitignore; restore review fixes

Zod-parse upterm session JSON and move isChildProcessAlive into processes.ts
(claimed in review but never landed). Expand unit coverage and stop gitignoring
baked upterm binaries now that the worker fetches from GCS.

* [build-tools] Resolve upterm from PATH or GCS instead of baking

Restore the orphaned review-fix tip: prefer an image/PATH upterm, else
download darwin-arm64 / linux-amd64 from gs://turtle-v2/upterm. Keeps the
stricter zod session parse, process-group/admin-socket comments, and GCS
cache/path unit coverage.

* [build-tools] Redact upterm tokens from ws:// relay URLs too

* [build-tools] Align upterm PATH probe with job env; scrub controls in logs

* [build-tools] address SSH review feedback

* [build-tools] address second round of SSH review feedback

* [build-tools] send a discriminated target on the ssh session mutation

* [build-tools] sort ssh test imports
2026-08-26 08:09:27 -07:00
119 changed files with 10934 additions and 1202 deletions

No files matched your search

+4 -15
View File
@@ -37,20 +37,6 @@ jobs:
run: yarn build
- name: Publish packages to npm
run: yarn run -T lerna publish from-package --yes --no-verify-access
- name: Verify eas-cli was published as latest on npm
run: |
expected="$(jq -r .version lerna.json)"
for attempt in $(seq 1 10); do
published="$(npm view eas-cli@latest version 2>/dev/null || true)"
if [ "$published" = "$expected" ]; then
echo "Verified eas-cli@$expected is latest on npm."
exit 0
fi
echo "Attempt $attempt: npm latest is '$published', expected '$expected'. Retrying in 15s..."
sleep 15
done
echo "eas-cli@$expected was not published as latest on npm; the publish step failed." >&2
exit 1
release-changelog:
name: Update changelog and publish release
runs-on: ubuntu-latest
@@ -81,10 +67,13 @@ jobs:
jq --arg version "$EAS_CLI_VERSION" '.STAGING = $version' \
cli-versions.json > cli-versions.json.tmp
mv cli-versions.json.tmp cli-versions.json
jq --arg version "$EAS_CLI_VERSION" '.PRODUCTION = $version' \
cli-versions.json > cli-versions.json.tmp
mv cli-versions.json.tmp cli-versions.json
git config --global user.email "[email protected]"
git config --global user.name "Expo CI"
git add cli-versions.json CHANGELOG.md
git commit --allow-empty -m "update CHANGELOG.md and bump STAGING in cli-versions.json"
git commit --allow-empty -m "update CHANGELOG.md and bump STAGING and PRODUCTION in cli-versions.json"
git push
- name: Prepare changelog for Slack
run: |
+62 -2
View File
@@ -8,14 +8,74 @@ This is the log of notable changes to EAS CLI and related packages.
### 🎉 New features
- [eas-cli] Add an opt-in `--max-idle-time-minutes` flag to `eas simulator`; the session stops automatically after that many minutes without activity. ([#4156](https://github.com/expo/eas-cli/pull/4156) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] Add `--tag` to `eas simulator` and `eas simulator:list` to group and filter simulator sessions by app variant. ([#4318](https://github.com/expo/eas-cli/pull/4318) by [@tchayen](https://github.com/tchayen))
- [eas-cli] Update workflow run logs in real time, instead of every 10 seconds. ([#4228](https://github.com/expo/eas-cli/pull/4228) by [@AHGIJMKLKKZNPJKQR](https://github.com/AHGIJMKLKKZNPJKQR))
- [build-tools] Add `expo-device-hub` web previews to Android web-preview-only, Agent Device, Argent, and Appium remote sessions. ([#4304](https://github.com/expo/eas-cli/pull/4304) by [@krystofwoldrich](https://github.com/krystofwoldrich))
- [eas-cli] Add `eas workflow:ssh <workflow-job-id> [command...]` to open an ssh session on the worker running a workflow job. ([#4032](https://github.com/expo/eas-cli/pull/4032) by [@gwdp](https://github.com/gwdp))
- [eas-cli] Add `--ssh` (and `--ssh-idle-timeout`) to `eas workflow:run` to enable ssh on the run's VM jobs. ([#4033](https://github.com/expo/eas-cli/pull/4033) by [@gwdp](https://github.com/gwdp))
### 🐛 Bug fixes
- [build-tools] Preapprove custom URL schemes before opening them in iOS Simulator sessions to avoid the first-use confirmation prompt. ([#4274](https://github.com/expo/eas-cli/pull/4274) by [@szdziedzic](https://github.com/szdziedzic))
### 🧹 Chores
- [eas-cli] Warn when starting Android EAS Simulator sessions that support is still in development and iOS parity is coming soon. ([#4320](https://github.com/expo/eas-cli/pull/4320) by [@szdziedzic](https://github.com/szdziedzic))
## [23.1.0](https://github.com/expo/eas-cli/releases/tag/v23.1.0) - 2026-08-31
### 🎉 New features
- [eas-cli] Prompt for the Expo SDK version in `eas new` and add a `--sdk-version` flag. The template now downloads from npm instead of GitHub to support SDK-specific versions. ([#4301](https://github.com/expo/eas-cli/pull/4301) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli] Clean up template metadata in projects created with `eas new`: remove the template LICENSE file and reset the package.json name, version, and license fields. ([#4301](https://github.com/expo/eas-cli/pull/4301) by [@brentvatne](https://github.com/brentvatne))
- [build-tools] Support `EAS_PNPM_FILTER_WORKSPACE` to install only the selected workspace's dependencies with `pnpm install --filter`. ([#4302](https://github.com/expo/eas-cli/pull/4302) by [@robingullo](https://github.com/robingullo))
### 🐛 Bug fixes
- [build-tools] Normalize extracted project permissions so source archives cannot leave files unreadable by EAS workers. ([#4287](https://github.com/expo/eas-cli/pull/4287) by [@AbbanMustafa](https://github.com/AbbanMustafa))
- [eas-cli] Add dynamic retries to deploy command that scale up more generously, depending on deployment size ([#4299](https://github.com/expo/eas-cli/pull/4299) by [@kitten](https://github.com/kitten))
### 🧹 Chores
- [eas-cli] Add regression coverage for the bundled iOS entitlement introspection fallback. ([#4285](https://github.com/expo/eas-cli/pull/4285) by [@brentvatne](https://github.com/brentvatne))
- [local-build-plugin] Replace `bunyan` with `@expo/bunyan` while preserving the Steps formatter binary. ([#4284](https://github.com/expo/eas-cli/pull/4284) by [@brentvatne](https://github.com/brentvatne))
- [steps] Replace deprecated `lodash.get` with a local helper. ([#4283](https://github.com/expo/eas-cli/pull/4283) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli][steps] Bump `uuid` to v11 and remove the redundant `@types/uuid` dependency. ([#4282](https://github.com/expo/eas-cli/pull/4282) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli] Bump `minizlib` to 3.1.0 to drop its deprecated `rimraf@5` transitive dependency. ([#4281](https://github.com/expo/eas-cli/pull/4281) by [@brentvatne](https://github.com/brentvatne))
## [23.0.0](https://github.com/expo/eas-cli/releases/tag/v23.0.0) - 2026-08-28
### 🛠 Breaking changes
- [eas-cli] Rename the EAS Simulator preview-only session type from `--type serve-sim` to `--type web-preview-only`. ([#4298](https://github.com/expo/eas-cli/pull/4298) by [@szdziedzic](https://github.com/szdziedzic))
### 🎉 New features
- [build-tools] Cache CocoaPods dependencies between iOS builds. ([#4266](https://github.com/expo/eas-cli/pull/4266) by [@AbbanMustafa](https://github.com/AbbanMustafa))
- [build-tools] Support `EAS_BUN_FILTER_WORKSPACE` to install only the selected workspace's dependencies with `bun install --filter`. ([#4292](https://github.com/expo/eas-cli/pull/4292) by [@konrad-armatys](https://github.com/konrad-armatys))
### 🐛 Bug fixes
- [build-tools] Prevent user npm minimum release age settings from applying to internal EAS CLI commands. ([#4296](https://github.com/expo/eas-cli/pull/4296) by [@sjchmiela](https://github.com/sjchmiela))
- [build-tools] Reduce serve-sim preview resolution from 1280 px to 960 px to lower streaming bandwidth. ([#4297](https://github.com/expo/eas-cli/pull/4297) by [@szdziedzic](https://github.com/szdziedzic))
### 🧹 Chores
- [worker] Use turtle cache upload and download URLs for eas.json build cache restore and save. ([#3860](https://github.com/expo/eas-cli/pull/3860) by [@sjchmiela](https://github.com/sjchmiela))
- [ci] Bump both STAGING and PRODUCTION cli-versions in release workflow. ([#4290](https://github.com/expo/eas-cli/pull/4290) by [@douglowder](https://github.com/douglowder))
## [22.6.0](https://github.com/expo/eas-cli/releases/tag/v22.6.0) - 2026-08-26
### 🎉 New features
- [eas-cli] Add an opt-in `--max-idle-time-minutes` flag to `eas simulator`; the session stops automatically after that many minutes without activity. ([#4156](https://github.com/expo/eas-cli/pull/4156) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] Add experimental `--resource-class` flag to `eas simulator`. ([#4268](https://github.com/expo/eas-cli/pull/4268) by [@gwdp](https://github.com/gwdp))
- [build-tools] Add worker-side SSH session helpers (upterm relay + session create/report/close). ([#4030](https://github.com/expo/eas-cli/pull/4030) by [@gwdp](https://github.com/gwdp))
- [worker] Wire an SSH session into the build lifecycle behind the workflow ssh flag. ([#4031](https://github.com/expo/eas-cli/pull/4031) by [@gwdp](https://github.com/gwdp))
### 🐛 Bug fixes
- [build-tools] Increase the serve-sim video bitrate to 6 Mbps for higher-quality simulator previews. ([#4291](https://github.com/expo/eas-cli/pull/4291) by [@szdziedzic](https://github.com/szdziedzic))
- [build-tools] Preapprove custom URL schemes before opening them in iOS Simulator sessions to avoid the first-use confirmation prompt. ([#4274](https://github.com/expo/eas-cli/pull/4274) by [@szdziedzic](https://github.com/szdziedzic))
## [22.4.0](https://github.com/expo/eas-cli/releases/tag/v22.4.0) - 2026-08-24
### 🎉 New features
+2 -2
View File
@@ -1,4 +1,4 @@
{
"STAGING": "22.4.0",
"PRODUCTION": "22.4.0"
"STAGING": "23.1.0",
"PRODUCTION": "23.1.0"
}
+1 -1
View File
@@ -1,5 +1,5 @@
{
"npmClient": "yarn",
"version": "22.5.0",
"version": "23.2.0",
"$schema": "node_modules/lerna/schemas/lerna-schema.json"
}
+7 -8
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/build-tools",
"version": "22.5.0",
"version": "23.2.0",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "BUSL-1.1",
"author": "Expo <[email protected]>",
@@ -39,17 +39,17 @@
"dependencies": {
"@expo/config": "55.0.10",
"@expo/config-plugins": "55.0.7",
"@expo/downloader": "22.0.0",
"@expo/eas-build-job": "22.0.0",
"@expo/downloader": "23.0.0",
"@expo/eas-build-job": "23.1.0",
"@expo/env": "^0.4.0",
"@expo/logger": "22.0.0",
"@expo/logger": "23.1.0",
"@expo/package-manager": "1.9.10",
"@expo/plist": "^0.3.5",
"@expo/results": "^1.0.0",
"@expo/spawn-async": "1.7.2",
"@expo/steps": "22.4.0",
"@expo/template-file": "22.0.0",
"@expo/turtle-spawn": "22.0.0",
"@expo/steps": "23.1.0",
"@expo/template-file": "23.0.0",
"@expo/turtle-spawn": "23.1.0",
"@expo/xcpretty": "^4.3.1",
"@ngrok/ngrok": "1.7.0",
"@sentry/node": "7.77.0",
@@ -92,7 +92,6 @@
"@types/retry": "^0.12.5",
"@types/semver": "^7.5.8",
"@types/uuid": "^9.0.8",
"bunyan": "^1.8.15",
"jest": "^29.7.0",
"memfs": "^4.17.1",
"nock": "13.4.0",
@@ -1,4 +1,4 @@
import bunyan from 'bunyan';
import type { bunyan } from '@expo/logger';
export function createLogger(): bunyan {
const logger = {
+18 -2
View File
@@ -18,8 +18,12 @@ import { downloadApplicationArchiveAsync } from '../ios/resign';
import { resolveArtifactPath, resolveBuildConfiguration, resolveScheme } from '../ios/resolve';
import { Sentry } from '../sentry';
import { parseAndReportXcactivitylog } from '../steps/utils/ios/xcactivitylog';
import { cacheStatsAsync, restoreCcacheAsync } from '../steps/functions/restoreBuildCache';
import { saveCcacheAsync } from '../steps/functions/saveBuildCache';
import {
cacheStatsAsync,
restoreCcacheAsync,
restoreCocoapodsCacheAsync,
} from '../steps/functions/restoreBuildCache';
import { saveCcacheAsync, saveCocoapodsCacheAsync } from '../steps/functions/saveBuildCache';
import { uploadApplicationArchive } from '../utils/artifacts';
import {
configureExpoUpdatesIfInstalledAsync,
@@ -107,6 +111,12 @@ async function buildInnerAsync(
env: ctx.env,
secrets: ctx.job.secrets,
});
await restoreCocoapodsCacheAsync({
logger: ctx.logger,
workingDirectory,
env: ctx.env,
secrets: ctx.job.secrets,
});
});
await ctx.runBuildPhase(BuildPhase.INSTALL_PODS, async () => {
@@ -250,6 +260,12 @@ async function buildInnerAsync(
env: ctx.env,
secrets: ctx.job.secrets,
});
await saveCocoapodsCacheAsync({
logger: ctx.logger,
workingDirectory,
env: ctx.env,
secrets: ctx.job.secrets,
});
});
await ctx.runBuildPhase(BuildPhase.CACHE_STATS, async () => {
@@ -70,6 +70,48 @@ describe('easBuildInternal', () => {
expect(spawn).not.toHaveBeenCalled();
});
it('overrides the minimum release age only for the internal config command', async () => {
const logger = {
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
child: jest.fn(),
} as any;
const ctx = {
logger,
env: {
FOO: 'bar',
NPM_CONFIG_MIN_RELEASE_AGE: '365',
},
job: { platform: 'ios', buildProfile: 'production' },
} as any;
jest.mocked(resolveEasCommandPrefixAndEnvAsync).mockResolvedValueOnce({
cmd: 'npx',
args: ['-y', 'eas-cli@latest'],
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
jest.mocked(spawn).mockResolvedValueOnce({
stdout: Buffer.from(JSON.stringify({ buildProfile: { env: {} } })),
} as any);
await resolveEnvFromBuildProfileAsync(ctx, { cwd: '/tmp/project' });
expect(spawn).toHaveBeenCalledWith(
'npx',
expect.any(Array),
expect.objectContaining({
env: {
FOO: 'bar',
NPM_CONFIG_MIN_RELEASE_AGE: '0',
},
})
);
expect(ctx.env).toEqual({
FOO: 'bar',
NPM_CONFIG_MIN_RELEASE_AGE: '365',
});
});
it('passes --refresh-ad-hoc-provisioning-profile to build:internal for iOS jobs with refreshAdHocProvisioningProfile', async () => {
const logger = {
info: jest.fn(),
@@ -3,7 +3,10 @@ import spawn, { SpawnPromise, SpawnResult } from '@expo/turtle-spawn';
import { createMockLogger } from '../../__tests__/utils/logger';
import { Sentry } from '../../sentry';
import { PackageManager } from '../../utils/packageManager';
import { installDependenciesWithNpmCacheFallbackAsync } from '../installDependencies';
import {
installDependenciesAsync,
installDependenciesWithNpmCacheFallbackAsync,
} from '../installDependencies';
jest.mock('@expo/turtle-spawn', () => jest.fn());
jest.mock('../../sentry', () => ({
@@ -236,6 +239,85 @@ describe(installDependenciesWithNpmCacheFallbackAsync, () => {
});
});
describe(installDependenciesAsync, () => {
beforeEach(() => {
jest.clearAllMocks();
jest.mocked(spawn).mockReturnValue(createSpawnPromise(Promise.resolve(createSpawnResult())));
});
it('installs only the selected workspace when EAS_PNPM_FILTER_WORKSPACE is set', async () => {
const logger = createMockLogger();
await installDependenciesAsync({
packageManager: PackageManager.PNPM,
env: { EAS_PNPM_FILTER_WORKSPACE: 'my-app' },
logger,
cwd: '/tmp/build',
useFrozenLockfile: true,
});
expect(spawn).toHaveBeenCalledWith(
'pnpm',
['install', '--filter', 'my-app', '--frozen-lockfile'],
expect.objectContaining({ cwd: '/tmp/build' })
);
});
it('installs all workspaces when EAS_PNPM_FILTER_WORKSPACE is not set', async () => {
const logger = createMockLogger();
await installDependenciesAsync({
packageManager: PackageManager.PNPM,
env: {},
logger,
cwd: '/tmp/build',
useFrozenLockfile: true,
});
expect(spawn).toHaveBeenCalledWith(
'pnpm',
['install', '--frozen-lockfile'],
expect.objectContaining({ cwd: '/tmp/build' })
);
});
it('installs only the selected workspace when EAS_BUN_FILTER_WORKSPACE is set', async () => {
const logger = createMockLogger();
await installDependenciesAsync({
packageManager: PackageManager.BUN,
env: { EAS_BUN_FILTER_WORKSPACE: 'my-app' },
logger,
cwd: '/tmp/build',
useFrozenLockfile: true,
});
expect(spawn).toHaveBeenCalledWith(
'bun',
['install', '--filter', 'my-app', '--frozen-lockfile'],
expect.objectContaining({ cwd: '/tmp/build' })
);
});
it('installs all workspaces when EAS_BUN_FILTER_WORKSPACE is not set', async () => {
const logger = createMockLogger();
await installDependenciesAsync({
packageManager: PackageManager.BUN,
env: {},
logger,
cwd: '/tmp/build',
useFrozenLockfile: true,
});
expect(spawn).toHaveBeenCalledWith(
'bun',
['install', '--frozen-lockfile'],
expect.objectContaining({ cwd: '/tmp/build' })
);
});
});
function createSpawnPromise(result: Promise<SpawnResult>): SpawnPromise<SpawnResult> {
return Object.assign(result, {
child: {
@@ -6,6 +6,7 @@ import {
Platform,
Workflow,
} from '@expo/eas-build-job';
import spawn from '@expo/turtle-spawn';
import { randomBytes, randomUUID } from 'crypto';
import { vol } from 'memfs';
import fetch, { Response } from 'node-fetch';
@@ -14,7 +15,7 @@ import { setTimeout } from 'timers/promises';
import { createMockLogger } from '../../__tests__/utils/logger';
import { BuildContext } from '../../context';
import { shallowCloneRepositoryAsync } from '../git';
import { prepareProjectSourcesAsync } from '../projectSources';
import { prepareProjectSourcesAsync, unpackTarGzAsync } from '../projectSources';
jest.mock('@expo/turtle-spawn');
jest.mock('node-fetch');
@@ -23,6 +24,27 @@ jest.mock('@expo/downloader');
jest.mock('@urql/core');
describe('projectSources', () => {
it('makes extracted project sources readable and writable by the worker', async () => {
const logger = createMockLogger();
await unpackTarGzAsync({
source: '/workingdir/project.tar.gz',
destination: '/workingdir/build',
logger,
});
expect(spawn).toHaveBeenNthCalledWith(
1,
'tar',
['-C', '/workingdir/build', '--strip-components', '1', '-zxf', '/workingdir/project.tar.gz'],
{ logger }
);
expect(spawn).toHaveBeenNthCalledWith(2, 'chmod', ['-R', 'u+rwX', '/workingdir/build'], {
logger,
});
expect(logger.info).toHaveBeenCalledWith('Normalizing project source permissions');
});
it('should use the refreshed repository URL', async () => {
const robotAccessToken = randomUUID();
const buildId = randomUUID();
@@ -32,7 +32,11 @@ export async function installDependenciesAsync({
break;
}
case PackageManager.PNPM: {
args = ['install', useFrozenLockfile ? '--frozen-lockfile' : '--no-frozen-lockfile'];
args = [
'install',
...(env['EAS_PNPM_FILTER_WORKSPACE'] ? ['--filter', env['EAS_PNPM_FILTER_WORKSPACE']] : []),
useFrozenLockfile ? '--frozen-lockfile' : '--no-frozen-lockfile',
];
break;
}
case PackageManager.YARN: {
@@ -58,7 +62,11 @@ export async function installDependenciesAsync({
break;
}
case PackageManager.BUN:
args = ['install', ...(useFrozenLockfile ? ['--frozen-lockfile'] : [])];
args = [
'install',
...(env['EAS_BUN_FILTER_WORKSPACE'] ? ['--filter', env['EAS_BUN_FILTER_WORKSPACE']] : []),
...(useFrozenLockfile ? ['--frozen-lockfile'] : []),
];
break;
default:
throw new Error(`Unsupported package manager: ${packageManager}`);
@@ -110,7 +110,7 @@ async function prepareProjectSourcesLocallyAsync<TJob extends Job>(
});
}
async function unpackTarGzAsync({
export async function unpackTarGzAsync({
logger,
source,
destination,
@@ -122,6 +122,8 @@ async function unpackTarGzAsync({
await spawn('tar', ['-C', destination, '--strip-components', '1', '-zxf', source], {
logger,
});
logger.info('Normalizing project source permissions');
await spawn('chmod', ['-R', 'u+rwX', destination], { logger });
}
function uploadProjectMetadataAsFireAndForget(
+5
View File
@@ -5,6 +5,8 @@ import RemoteLoggerStream from './logging/RemoteLoggerStream';
export { Builders, HttpLogStream, RemoteLoggerStream };
export { uploadWithSignedUrl } from './storage/uploadWithSignedUrl';
export type { SignedUrl, UploadWithSignedUrlParams } from './storage/uploadWithSignedUrl';
export { downloadCacheAsync } from './steps/functions/restoreCache';
export { uploadCacheAsync } from './steps/functions/saveCache';
export {
ArtifactToUpload,
@@ -18,6 +20,9 @@ export {
export { PackageManager } from './utils/packageManager';
export * as TurtleSshSession from './utils/turtleSshSession';
export { formatSecondsForLog } from './utils/formatDuration';
export { findAndUploadXcodeBuildLogsAsync } from './ios/xcodeBuildLogs';
export { Hook, runHookIfPresent } from './utils/hooks';
@@ -51,7 +51,7 @@ import { createStartAppiumRemoteSessionBuildFunction } from './functions/startAp
import { createStartCuttlefishDeviceBuildFunction } from './functions/startCuttlefishDevice';
import { createStartIosSimulatorBuildFunction } from './functions/startIosSimulator';
import { createStartIosSimulatorRecordingsBuildFunction } from './functions/startIosSimulatorRecordings';
import { createStartServeSimRemoteSessionBuildFunction } from './functions/startServeSimRemoteSession';
import { createStartWebPreviewRemoteSessionBuildFunction } from './functions/startWebPreviewRemoteSession';
import { createStartServeSimMetricsBuildFunction } from './functions/startServeSimMetrics';
import { createCollectServeSimMetricsBuildFunction } from './functions/collectServeSimMetrics';
import { createUploadArtifactBuildFunction } from './functions/uploadArtifact';
@@ -106,7 +106,7 @@ export function getEasFunctions(ctx: CustomBuildContext): BuildFunction[] {
createStartIosSimulatorRecordingsBuildFunction(),
createFinishIosSimulatorRecordingsBuildFunction(),
createUploadDeviceRunSessionScreenRecordingsBuildFunction(ctx),
createStartServeSimRemoteSessionBuildFunction(ctx),
createStartWebPreviewRemoteSessionBuildFunction(ctx),
createStartServeSimMetricsBuildFunction(),
createCollectServeSimMetricsBuildFunction(ctx),
createInstallMaestroBuildFunction(),
@@ -162,6 +162,28 @@ describe(createEasBuildBuildFunctionGroup, () => {
expect(installPodsStep!.ctx.relativeWorkingDirectory).toBe('./ios');
});
it('restores and saves caches around an iOS simulator build', () => {
const buildToolsContext = createMockBuildToolsContext({
platform: Platform.IOS,
simulator: true,
});
const functionGroup = createEasBuildBuildFunctionGroup(buildToolsContext);
const globalCtx = createGlobalContextMock({ logger: createMockLogger() });
const steps = functionGroup.createBuildStepsFromFunctionGroupCall(globalCtx);
const stepNames = steps.map(step => step.displayName);
const restoreCacheIndex = stepNames.indexOf('Restore Cache');
const installPodsIndex = stepNames.indexOf('Install Pods');
const saveCacheIndex = stepNames.indexOf('Save Cache');
expect(restoreCacheIndex).toBeGreaterThan(-1);
expect(restoreCacheIndex).toBeLessThan(installPodsIndex);
expect(saveCacheIndex).toBeGreaterThan(installPodsIndex);
const restoreCacheStep = steps[restoreCacheIndex];
expect(restoreCacheStep.inputs?.find(input => input.id === 'simulator')?.rawValue).toBe(true);
});
it('sets working directory on all steps except checkout (Android with credentials)', () => {
const buildToolsContext = createMockBuildToolsContext({
platform: Platform.ANDROID,
@@ -98,6 +98,8 @@ function createStepsForIosSimulatorBuild({
buildToolsContext,
workingDirectory,
}: HelperFunctionsInput): BuildStep[] {
const evictUsedBefore = new Date();
const calculateEASUpdateRuntimeVersion =
calculateEASUpdateRuntimeVersionFunction().createBuildStepFromFunctionCall(globalCtx, {
id: 'calculate_eas_update_runtime_version',
@@ -106,6 +108,16 @@ function createStepsForIosSimulatorBuild({
const installPods = createInstallPodsBuildFunction().createBuildStepFromFunctionCall(globalCtx, {
workingDirectory: workingDirectory ? path.join(workingDirectory, './ios') : './ios',
});
const restoreCache = createRestoreBuildCacheFunction().createBuildStepFromFunctionCall(
globalCtx,
{
workingDirectory,
callInputs: {
platform: Platform.IOS,
simulator: true,
},
}
);
const configureEASUpdate =
configureEASUpdateIfInstalledFunction().createBuildStepFromFunctionCall(globalCtx, {
workingDirectory,
@@ -123,6 +135,16 @@ function createStepsForIosSimulatorBuild({
'${ steps.calculate_eas_update_runtime_version.resolved_eas_update_runtime_version }',
},
});
const saveCache = createSaveBuildCacheFunction(evictUsedBefore).createBuildStepFromFunctionCall(
globalCtx,
{
workingDirectory,
callInputs: {
platform: Platform.IOS,
simulator: true,
},
}
);
return [
createCheckoutBuildFunction().createBuildStepFromFunctionCall(globalCtx),
createSetUpNpmrcBuildFunction().createBuildStepFromFunctionCall(globalCtx, {
@@ -138,6 +160,7 @@ function createStepsForIosSimulatorBuild({
createPrebuildBuildFunction().createBuildStepFromFunctionCall(globalCtx, {
workingDirectory,
}),
restoreCache,
calculateEASUpdateRuntimeVersion,
installPods,
configureEASUpdate,
@@ -159,6 +182,10 @@ function createStepsForIosSimulatorBuild({
createFindAndUploadBuildArtifactsBuildFunction(
buildToolsContext
).createBuildStepFromFunctionCall(globalCtx, { workingDirectory }),
saveCache,
createCacheStatsBuildFunction().createBuildStepFromFunctionCall(globalCtx, {
workingDirectory,
}),
];
}
@@ -0,0 +1,136 @@
import { spawnAsync } from '@expo/steps';
import { vol } from 'memfs';
import { createMockLogger } from '../../../__tests__/utils/logger';
import { Datadog } from '../../../datadog';
import {
compressCocoapodsCacheAsync,
getCocoapodsCachePaths,
resolveCocoapodsCacheKeyAsync,
restoreCocoapodsCacheArchiveAsync,
} from '../../../utils/cocoapodsCache';
import { downloadCacheAsync } from '../restoreCache';
import { restoreCocoapodsCacheAsync } from '../restoreBuildCache';
import { uploadCacheAsync } from '../saveCache';
import { saveCocoapodsCacheAsync } from '../saveBuildCache';
jest.mock('@expo/steps', () => ({
...jest.requireActual('@expo/steps'),
spawnAsync: jest.fn(),
}));
jest.mock('../restoreCache', () => ({
decompressCacheAsync: jest.fn(),
downloadCacheAsync: jest.fn(),
downloadPublicCacheAsync: jest.fn(),
}));
jest.mock('../saveCache', () => ({
compressCacheAsync: jest.fn(),
uploadCacheAsync: jest.fn(),
}));
jest.mock('../../../utils/cocoapodsCache', () => ({
compressCocoapodsCacheAsync: jest.fn(),
getCocoapodsCachePaths: jest.fn(),
resolveCocoapodsCacheKeyAsync: jest.fn(),
restoreCocoapodsCacheArchiveAsync: jest.fn(),
}));
const logger = createMockLogger();
const env = {
EAS_PODS_CACHE: '1',
EAS_BUILD_ID: 'build-id',
__API_SERVER_URL: 'https://api.expo.test',
};
const secrets = { robotAccessToken: 'robot-token' };
describe('CocoaPods build cache', () => {
beforeEach(() => {
jest.clearAllMocks();
vol.fromJSON(
{
'/workingdir/ios/Pods/Manifest.lock': 'manifest',
'/workingdir/ios/Podfile.lock': 'lockfile',
'/tmp/cocoapods-cache.tar.gz': 'archive',
},
'/'
);
jest.mocked(getCocoapodsCachePaths).mockReturnValue({
iosDirectory: '/workingdir/ios',
podsDirectory: '/workingdir/ios/Pods',
podfileLockPath: '/workingdir/ios/Podfile.lock',
});
jest.mocked(resolveCocoapodsCacheKeyAsync).mockResolvedValue({
key: 'ios-pods-1.16.2-lock-hash',
keyPrefix: 'ios-pods-1.16.2-',
});
jest.mocked(spawnAsync).mockResolvedValue({ stdout: '1.16.2\n' } as any);
});
it('does nothing when the cache is disabled', async () => {
await restoreCocoapodsCacheAsync({
logger,
workingDirectory: '/workingdir',
env: {},
secrets,
});
await saveCocoapodsCacheAsync({
logger,
workingDirectory: '/workingdir',
env: {},
secrets,
});
expect(spawnAsync).not.toHaveBeenCalled();
});
it('restores the newest matching CocoaPods cache', async () => {
jest.mocked(downloadCacheAsync).mockResolvedValue({
archivePath: '/tmp/cocoapods-cache.tar.gz',
matchedKey: 'ios-pods-1.16.2-older-lock-hash',
});
const datadogLogSpy = jest.spyOn(Datadog, 'log');
await restoreCocoapodsCacheAsync({
logger,
workingDirectory: '/workingdir',
env,
secrets,
});
expect(downloadCacheAsync).toHaveBeenCalledWith(
expect.objectContaining({
key: 'ios-pods-1.16.2-lock-hash',
keyPrefixes: ['ios-pods-1.16.2-'],
paths: ['/workingdir/ios/Pods'],
})
);
expect(restoreCocoapodsCacheArchiveAsync).toHaveBeenCalledWith({
archivePath: '/tmp/cocoapods-cache.tar.gz',
workingDirectory: '/workingdir',
});
expect(datadogLogSpy).toHaveBeenCalledWith('CocoaPods cache restored (prefix_match)', {
event: 'cocoapods_cache_restored',
cache_hit_type: 'prefix_match',
});
});
it('compresses and saves the installed Pods directory', async () => {
jest.mocked(compressCocoapodsCacheAsync).mockResolvedValue({
archivePath: '/tmp/cocoapods-cache.tar.gz',
});
await saveCocoapodsCacheAsync({
logger,
workingDirectory: '/workingdir',
env,
secrets,
});
expect(uploadCacheAsync).toHaveBeenCalledWith(
expect.objectContaining({
archivePath: '/tmp/cocoapods-cache.tar.gz',
key: 'ios-pods-1.16.2-lock-hash',
paths: ['/workingdir/ios/Pods'],
})
);
});
});
@@ -1,11 +1,17 @@
import { type bunyan } from '@expo/logger';
import { Platform } from '@expo/eas-build-job';
import fs from 'fs';
import fetch from 'node-fetch';
import os from 'os';
import path from 'path';
import { compressCacheAsync } from '../saveCache';
import { compressCacheAsync, uploadCacheAsync } from '../saveCache';
import { decompressCacheAsync } from '../restoreCache';
jest.mock('node-fetch');
const { Response } = jest.requireActual('node-fetch') as typeof import('node-fetch');
function createLoggerMock(): bunyan {
return { info: jest.fn(), warn: jest.fn(), error: jest.fn() } as unknown as bunyan;
}
@@ -81,3 +87,71 @@ describe('cache compress/decompress round trip', () => {
expect(Math.floor(restoredStat.mtimeMs / 1000)).toBe(Math.floor(mtime.getTime() / 1000));
});
});
describe(uploadCacheAsync, () => {
afterEach(() => {
jest.mocked(fetch).mockReset();
});
it.each([
['normal upload', undefined, false],
['forced upload', true, true],
] as const)('sends force for %s', async (_name, force, expectedForce) => {
const tempDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'upload-cache-test-'));
const archivePath = path.join(tempDir, 'cache.tar.gz');
await fs.promises.writeFile(archivePath, 'cache archive');
jest
.mocked(fetch)
.mockResolvedValueOnce(
new Response(
JSON.stringify({
data: {
headers: {},
url: 'https://storage.expo.test/cache',
},
}),
{ status: 200 }
)
)
.mockImplementationOnce(async (_url, request) => {
const body = request!.body as NodeJS.ReadableStream & AsyncIterable<Buffer>;
for await (const _chunk of body) {
// Consume the archive stream as a real upload would.
}
return new Response(undefined, { status: 200 });
});
try {
await uploadCacheAsync({
logger: createLoggerMock(),
jobId: 'build-id',
expoApiServerURL: 'https://api.expo.test',
robotAccessToken: 'robot-token',
paths: ['/cache/path'],
key: 'cache-key',
archivePath,
size: 13,
platform: Platform.ANDROID,
force,
});
expect(fetch).toHaveBeenCalledTimes(2);
const [uploadSessionUrl, uploadSessionRequest] = jest.mocked(fetch).mock.calls[0];
expect(uploadSessionUrl.toString()).toBe(
'https://api.expo.test/v2/turtle-builds/caches/upload-sessions'
);
expect(JSON.parse(uploadSessionRequest!.body as string)).toMatchObject({
buildId: 'build-id',
force: expectedForce,
key: 'cache-key',
size: 13,
});
expect(jest.mocked(fetch).mock.calls[1][0].toString()).toBe(
'https://storage.expo.test/cache'
);
} finally {
await fs.promises.rm(tempDir, { recursive: true, force: true });
}
});
});
@@ -14,6 +14,7 @@ import {
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
@@ -46,8 +47,8 @@ jest.mock('../../utils/remoteDeviceRunSession', () => ({
getNgrokTunnelDomainOrThrow: jest.fn(),
selectXcodeDeveloperDirectoryAsync: jest.fn(),
spawnDetached: jest.fn(),
startDeviceWebPreviewWithTunnelAsync: jest.fn(),
startNgrokTunnelAsync: jest.fn(),
startServeSimWithTunnelAsync: jest.fn(),
uploadRemoteSessionConfigAsync: jest.fn(),
waitForDeviceRunSessionStoppedAsync: jest.fn(),
}));
@@ -58,6 +59,7 @@ const EXPECTED_EVENT_LOG_PATH = path.join(ARGENT_STATE_DIR, 'tool-server-events.
const mockStopAsync = jest.fn();
const mockTunnelStopAsync = jest.fn();
const mockPreviewStopAsync = jest.fn();
describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
beforeEach(async () => {
@@ -68,6 +70,7 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
jest.mocked(pollArgentArtifactsForUploadAsync).mockResolvedValue(undefined);
mockStopAsync.mockResolvedValue(undefined);
mockTunnelStopAsync.mockResolvedValue(undefined);
mockPreviewStopAsync.mockResolvedValue(undefined);
jest.mocked(startArgentEventCollectionAsync).mockResolvedValue({
stopAsync: mockStopAsync,
getLastEventObservedAt: () => undefined,
@@ -86,6 +89,10 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
url: 'https://argent-abc.tunnel.example.com',
stopAsync: mockTunnelStopAsync,
});
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
previewUrl: 'https://web-preview.tunnel.example.com',
stopAsync: mockPreviewStopAsync,
});
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockResolvedValue(undefined);
@@ -151,5 +158,17 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
jest.mocked(waitForDeviceRunSessionStoppedAsync).mock.invocationCallOrder[0]
);
expect(mockTunnelStopAsync).toHaveBeenCalledTimes(1);
expect(startDeviceWebPreviewWithTunnelAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({ runtimePlatform: BuildRuntimePlatform.LINUX })
);
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledWith(
expect.objectContaining({
remoteConfig: expect.objectContaining({
webPreviewUrl: 'https://web-preview.tunnel.example.com',
}),
})
);
expect(mockPreviewStopAsync).toHaveBeenCalledTimes(1);
});
});
@@ -0,0 +1,85 @@
import { type bunyan } from '@expo/logger';
import { BuildRuntimePlatform, type BuildStepContext, type BuildStepEnv } from '@expo/steps';
import { type CustomBuildContext } from '../../../customBuildContext';
import {
getDeviceRunSessionIdOrThrow,
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
startDeviceWebPreviewWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../../utils/remoteDeviceRunSession';
import { createStartWebPreviewRemoteSessionBuildFunction } from '../startWebPreviewRemoteSession';
jest.mock('../../utils/remoteDeviceRunSession');
const ctx = {} as CustomBuildContext;
const env = {} as BuildStepEnv;
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const stopAsync = jest.fn();
async function runAsync(runtimePlatform: BuildRuntimePlatform): Promise<void> {
const buildFunction = createStartWebPreviewRemoteSessionBuildFunction(ctx);
await buildFunction.fn!(
{
logger,
global: { runtimePlatform },
} as unknown as BuildStepContext,
{
inputs: {
package_version: { value: '1.2.3' },
max_duration_seconds: { value: 120 },
},
outputs: {},
env,
} as never
);
}
describe(createStartWebPreviewRemoteSessionBuildFunction, () => {
beforeEach(() => {
jest.clearAllMocks();
jest.mocked(getDeviceRunSessionIdOrThrow).mockReturnValue('device-run-session-id');
jest.mocked(getNgrokTunnelDomainOrThrow).mockReturnValue('tunnel.example.com');
jest.mocked(selectXcodeDeveloperDirectoryAsync).mockResolvedValue(undefined);
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
previewUrl: 'https://web-preview.example.test',
stopAsync,
});
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockResolvedValue(undefined);
stopAsync.mockResolvedValue(undefined);
});
it.each([
[BuildRuntimePlatform.DARWIN, true],
[BuildRuntimePlatform.LINUX, false],
])('starts the web preview for %s', async (runtimePlatform, selectsXcode) => {
await runAsync(runtimePlatform);
expect(selectXcodeDeveloperDirectoryAsync).toHaveBeenCalledTimes(selectsXcode ? 1 : 0);
expect(startDeviceWebPreviewWithTunnelAsync).toHaveBeenCalledWith(ctx, {
runtimePlatform,
baseDomain: 'tunnel.example.com',
env,
logger,
timeoutMs: 60_000,
packageVersion: '1.2.3',
});
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledWith({
ctx,
deviceRunSessionId: 'device-run-session-id',
remoteConfig: { previewUrl: 'https://web-preview.example.test' },
logger,
});
expect(waitForDeviceRunSessionStoppedAsync).toHaveBeenCalledWith({
ctx,
deviceRunSessionId: 'device-run-session-id',
logger,
maxDurationSeconds: 120,
signal: undefined,
});
expect(stopAsync).toHaveBeenCalledTimes(1);
});
});
@@ -20,6 +20,11 @@ import {
getCcacheKeyPrefix,
getCcachePath,
} from '../../utils/cacheKey';
import {
getCocoapodsCachePaths,
resolveCocoapodsCacheKeyAsync,
restoreCocoapodsCacheArchiveAsync,
} from '../../utils/cocoapodsCache';
import { Datadog } from '../../datadog';
import { GRADLE_CACHE_KEY_PREFIX, generateGradleCacheKeyAsync } from '../../utils/gradleCacheKey';
import { TurtleFetchError, turtleFetch } from '../../utils/turtleFetch';
@@ -79,6 +84,13 @@ export function createRestoreBuildCacheFunction(): BuildFunction {
env,
secrets: stepCtx.global.staticContext.job.secrets,
});
} else {
await restoreCocoapodsCacheAsync({
logger,
workingDirectory,
env,
secrets: stepCtx.global.staticContext.job.secrets,
});
}
},
});
@@ -206,6 +218,67 @@ export async function restoreCcacheAsync({
}
}
export async function restoreCocoapodsCacheAsync({
logger,
workingDirectory,
env,
secrets,
}: {
logger: bunyan;
workingDirectory: string;
env: Record<string, string | undefined>;
secrets?: { robotAccessToken?: string };
}): Promise<void> {
if (env.EAS_PODS_CACHE !== '1') {
return;
}
try {
const { stdout } = await spawnAsync('pod', ['--version'], {
env,
stdio: 'pipe',
});
const { key, keyPrefix } = await resolveCocoapodsCacheKeyAsync(workingDirectory, stdout);
logger.info(`Restoring CocoaPods cache key: ${key}`);
const jobId = nullthrows(env.EAS_BUILD_ID, 'EAS_BUILD_ID is not set');
const robotAccessToken = nullthrows(
secrets?.robotAccessToken,
'Robot access token is required for cache operations'
);
const expoApiServerURL = nullthrows(env.__API_SERVER_URL, '__API_SERVER_URL is not set');
const { podsDirectory } = getCocoapodsCachePaths(workingDirectory);
const { archivePath, matchedKey } = await downloadCacheAsync({
logger,
jobId,
expoApiServerURL,
robotAccessToken,
paths: [podsDirectory],
key,
keyPrefixes: [keyPrefix],
platform: Platform.IOS,
});
await restoreCocoapodsCacheArchiveAsync({ archivePath, workingDirectory });
const hitType = matchedKey === key ? 'direct_hit' : 'prefix_match';
logger.info(
`CocoaPods cache restored to ${podsDirectory} (${hitType === 'direct_hit' ? 'direct hit' : 'prefix match'})`
);
Datadog.log(`CocoaPods cache restored (${hitType})`, {
event: 'cocoapods_cache_restored',
cache_hit_type: hitType,
});
} catch (err: unknown) {
if (err instanceof TurtleFetchError && err.response?.status === 404) {
logger.info('No CocoaPods cache found for this key');
} else {
logger.warn('Failed to restore CocoaPods cache: ', err);
}
}
}
export async function restoreGradleCacheAsync({
logger,
workingDirectory,
@@ -20,6 +20,11 @@ import {
generateDefaultBuildCacheKeyAsync,
getCcachePath,
} from '../../utils/cacheKey';
import {
compressCocoapodsCacheAsync,
getCocoapodsCachePaths,
resolveCocoapodsCacheKeyAsync,
} from '../../utils/cocoapodsCache';
import { generateGradleCacheKeyAsync } from '../../utils/gradleCacheKey';
export function createSaveBuildCacheFunction(evictUsedBefore: Date): BuildFunction {
@@ -78,11 +83,77 @@ export function createSaveBuildCacheFunction(evictUsedBefore: Date): BuildFuncti
env,
secrets: stepCtx.global.staticContext.job.secrets,
});
} else {
await saveCocoapodsCacheAsync({
logger,
workingDirectory,
env,
secrets: stepCtx.global.staticContext.job.secrets,
});
}
},
});
}
export async function saveCocoapodsCacheAsync({
logger,
workingDirectory,
env,
secrets,
}: {
logger: bunyan;
workingDirectory: string;
env: Record<string, string | undefined>;
secrets?: { robotAccessToken?: string };
}): Promise<void> {
if (env.EAS_PODS_CACHE !== '1') {
return;
}
const { podsDirectory, podfileLockPath } = getCocoapodsCachePaths(workingDirectory);
try {
await Promise.all([fs.promises.access(podsDirectory), fs.promises.access(podfileLockPath)]);
} catch {
logger.warn('No CocoaPods installation found, skipping cache save');
return;
}
try {
const { stdout } = await spawnAsync('pod', ['--version'], {
env,
stdio: 'pipe',
});
const { key } = await resolveCocoapodsCacheKeyAsync(workingDirectory, stdout);
logger.info(`Saving CocoaPods cache key: ${key}`);
const jobId = nullthrows(env.EAS_BUILD_ID, 'EAS_BUILD_ID is not set');
const robotAccessToken = nullthrows(
secrets?.robotAccessToken,
'Robot access token is required for cache operations'
);
const expoApiServerURL = nullthrows(env.__API_SERVER_URL, '__API_SERVER_URL is not set');
logger.info('Compressing CocoaPods cache...');
const { archivePath } = await compressCocoapodsCacheAsync({ workingDirectory });
const { size } = await fs.promises.stat(archivePath);
logger.info(`CocoaPods cache archive size: ${formatBytes(size)}`);
await uploadCacheAsync({
logger,
jobId,
expoApiServerURL,
robotAccessToken,
archivePath,
key,
paths: [podsDirectory],
size,
platform: Platform.IOS,
});
} catch (err) {
logger.error({ err }, 'Failed to save CocoaPods cache');
}
}
export async function saveCcacheAsync({
logger,
workingDirectory,
@@ -103,6 +103,7 @@ export async function uploadCacheAsync({
archivePath,
size,
platform,
force = false,
}: {
logger: bunyan;
jobId: string;
@@ -113,6 +114,7 @@ export async function uploadCacheAsync({
archivePath: string;
size: number;
platform: Platform | undefined;
force?: boolean;
}): Promise<void> {
const routerURL = platform
? 'v2/turtle-builds/caches/upload-sessions'
@@ -127,12 +129,14 @@ export async function uploadCacheAsync({
key,
version: getCacheVersion(paths),
size,
force,
})
: JSON.stringify({
jobRunId: jobId,
key,
version: getCacheVersion(paths),
size,
force,
}),
headers: {
Authorization: `Bearer ${robotAccessToken}`,
@@ -23,8 +23,8 @@ import {
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
startServeSimWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
waitForFileAsync,
@@ -107,22 +107,19 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
const agentDeviceRemoteSessionUrl = agentDeviceTunnel.url;
logger.info(`Tunnel is ready at ${agentDeviceRemoteSessionUrl}.`);
let serveSim: Awaited<ReturnType<typeof startServeSimWithTunnelAsync>> | undefined;
let webPreview: Awaited<ReturnType<typeof startDeviceWebPreviewWithTunnelAsync>> | undefined;
let eventCollection:
| Awaited<ReturnType<typeof startAgentDeviceEventCollectionAsync>>
| undefined;
try {
// serve-sim is iOS-only — only launch it (and report a webPreviewUrl)
// on Darwin. Android sessions go without a preview URL.
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
serveSim = await startServeSimWithTunnelAsync(ctx, {
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
logger.info(`Web preview URL: ${serveSim.previewUrl}`);
}
webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
logger.info(`Web preview URL: ${webPreview.previewUrl}`);
await uploadRemoteSessionConfigAsync({
ctx,
@@ -130,7 +127,7 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
remoteConfig: {
agentDeviceRemoteSessionUrl,
agentDeviceRemoteSessionToken: daemonToken,
...(serveSim ? { webPreviewUrl: serveSim.previewUrl } : {}),
webPreviewUrl: webPreview.previewUrl,
},
logger,
});
@@ -163,8 +160,8 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
: undefined,
});
} finally {
if (serveSim) {
await serveSim.stopAsync();
if (webPreview) {
await webPreview.stopAsync();
}
await agentDeviceTunnel.stopAsync();
if (eventCollection) {
@@ -26,8 +26,8 @@ import {
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
startServeSimWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
@@ -118,7 +118,7 @@ export function createStartAppiumRemoteSessionBuildFunction(
logger,
});
let appiumTunnel: Awaited<ReturnType<typeof startNgrokTunnelAsync>> | undefined;
let serveSim: Awaited<ReturnType<typeof startServeSimWithTunnelAsync>> | undefined;
let webPreview: Awaited<ReturnType<typeof startDeviceWebPreviewWithTunnelAsync>> | undefined;
try {
appiumTunnel = await startNgrokTunnelAsync({
port: APPIUM_PORT,
@@ -128,18 +128,15 @@ export function createStartAppiumRemoteSessionBuildFunction(
logger,
});
switch (runtimePlatform) {
case BuildRuntimePlatform.DARWIN:
serveSim = await startServeSimWithTunnelAsync(ctx, {
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: APPIUM_STARTUP_TIMEOUT_MS,
});
break;
case BuildRuntimePlatform.LINUX:
break;
}
// expo-device-hub has no serial-selection flag. Device run session workflows must expose
// a single booted Android emulator so the Hub and Appium resolve the same device.
webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: APPIUM_STARTUP_TIMEOUT_MS,
});
await uploadRemoteSessionConfigAsync({
ctx,
@@ -151,7 +148,7 @@ export function createStartAppiumRemoteSessionBuildFunction(
'appium:automationName': device.automationName,
'appium:udid': device.udid,
},
...(serveSim ? { webPreviewUrl: serveSim.previewUrl } : {}),
webPreviewUrl: webPreview.previewUrl,
},
logger,
});
@@ -170,8 +167,8 @@ export function createStartAppiumRemoteSessionBuildFunction(
: undefined,
});
} finally {
if (serveSim) {
await serveSim.stopAsync();
if (webPreview) {
await webPreview.stopAsync();
}
if (appiumTunnel) {
await appiumTunnel.stopAsync();
@@ -26,8 +26,8 @@ import {
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
startServeSimWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
@@ -186,7 +186,7 @@ export function createStartArgentRemoteSessionBuildFunction(
});
let toolsTunnel: Awaited<ReturnType<typeof startNgrokTunnelAsync>> | undefined;
let serveSim: Awaited<ReturnType<typeof startServeSimWithTunnelAsync>> | undefined;
let webPreview: Awaited<ReturnType<typeof startDeviceWebPreviewWithTunnelAsync>> | undefined;
try {
toolsTunnel = await startNgrokTunnelAsync({
port: toolServerPort,
@@ -199,18 +199,14 @@ export function createStartArgentRemoteSessionBuildFunction(
const publicToolsUrl = toolsTunnel.url;
logger.info(`Tunnel is ready at ${publicToolsUrl}.`);
// serve-sim is iOS-only — Android sessions go without a preview URL.
let webPreviewUrl: string | undefined;
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
serveSim = await startServeSimWithTunnelAsync(ctx, {
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
webPreviewUrl = serveSim.previewUrl;
logger.info(`Web preview URL: ${webPreviewUrl}`);
}
webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
logger.info(`Web preview URL: ${webPreview.previewUrl}`);
await uploadRemoteSessionConfigAsync({
ctx,
@@ -218,7 +214,7 @@ export function createStartArgentRemoteSessionBuildFunction(
remoteConfig: {
toolsUrl: publicToolsUrl,
...(toolServerToken ? { toolsAuthToken: toolServerToken } : {}),
...(webPreviewUrl ? { webPreviewUrl } : {}),
webPreviewUrl: webPreview.previewUrl,
},
logger,
});
@@ -238,8 +234,8 @@ export function createStartArgentRemoteSessionBuildFunction(
: undefined,
});
} finally {
if (serveSim) {
await serveSim.stopAsync();
if (webPreview) {
await webPreview.stopAsync();
}
if (toolsTunnel) {
await toolsTunnel.stopAsync();
@@ -10,22 +10,21 @@ import {
getDeviceRunSessionIdOrThrow,
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
startServeSimWithTunnelAsync,
startDeviceWebPreviewWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
const STARTUP_TIMEOUT_MS = 60_000;
export function createStartServeSimRemoteSessionBuildFunction(
export function createStartWebPreviewRemoteSessionBuildFunction(
ctx: CustomBuildContext
): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'start_serve_sim_remote_session',
name: 'Start serve-sim remote session',
name: 'Start web preview remote session',
__metricsId: 'eas/start_serve_sim_remote_session',
supportedRuntimePlatforms: [BuildRuntimePlatform.DARWIN],
inputProviders: [
BuildStepInput.createProvider({
id: 'package_version',
@@ -38,30 +37,34 @@ export function createStartServeSimRemoteSessionBuildFunction(
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
}),
],
fn: async ({ logger }, { inputs, env, signal }) => {
fn: async ({ logger, global }, { inputs, env, signal }) => {
const deviceRunSessionId = getDeviceRunSessionIdOrThrow(env);
const ngrokTunnelDomain = getNgrokTunnelDomainOrThrow(env);
const maxDurationSeconds = inputs.max_duration_seconds?.value as number | undefined;
const packageVersion = inputs.package_version?.value as string | undefined;
const { runtimePlatform } = global;
logger.info('Starting serve-sim remote session.');
logger.info(`Starting web preview remote session (runtime: ${runtimePlatform}).`);
await selectXcodeDeveloperDirectoryAsync({ env, logger });
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
await selectXcodeDeveloperDirectoryAsync({ env, logger });
}
const serveSim = await startServeSimWithTunnelAsync(ctx, {
const webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
packageVersion,
});
logger.info(`Preview URL: ${serveSim.previewUrl}`);
logger.info(`Preview URL: ${webPreview.previewUrl}`);
try {
await uploadRemoteSessionConfigAsync({
ctx,
deviceRunSessionId,
remoteConfig: { previewUrl: serveSim.previewUrl },
remoteConfig: { previewUrl: webPreview.previewUrl },
logger,
});
@@ -73,7 +76,7 @@ export function createStartServeSimRemoteSessionBuildFunction(
signal,
});
} finally {
await serveSim.stopAsync();
await webPreview.stopAsync();
}
},
});
@@ -13,14 +13,16 @@ import { Sentry } from '../../../sentry';
import { turtleFetch } from '../../../utils/turtleFetch';
import { sleepAsync } from '../../../utils/retry';
import {
createExpoDeviceHubArgs,
createServeSimArgs,
ensureFfmpegInstalledAsync,
fetchServeSimTurnArgsAsync,
fetchWebPreviewTurnArgsAsync,
metricsCorsOriginToServeSimArgs,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
turnIceServersToServeSimArgs,
turnIceServersToWebPreviewArgs,
waitForDeviceRunSessionStoppedAsync,
waitForServeSimReadyAsync,
waitForWebPreviewReadyAsync,
} from '../remoteDeviceRunSession';
jest.mock('@ngrok/ngrok');
@@ -132,11 +134,11 @@ describe(createServeSimArgs, () => {
'--webrtc-codec',
'vp8',
'--max-dimension',
'1280',
'960',
'--mjpeg-quality',
'0.55',
'--video-bitrate',
'3000000',
'6000000',
'--video-fps',
'60',
'--turn-url',
@@ -173,6 +175,49 @@ describe(createServeSimArgs, () => {
});
});
describe(createExpoDeviceHubArgs, () => {
it('uses the latest Expo package and applies the EAS Android streaming policy', () => {
expect(
createExpoDeviceHubArgs({
port: 4321,
turnArgs: ['--turn-url', 'turns:turn.example.test:443'],
})
).toEqual([
'--yes',
'expo-device-hub@latest',
'--port',
'4321',
'--host',
'127.0.0.1',
'--platform',
'android',
'--transport',
'webrtc',
'--webrtc-codec',
'h264',
'--webrtc-ice-policy',
'all',
'--max-dimension',
'1280',
'--video-bitrate',
'6000000',
'--video-fps',
'60',
'--hide-sidebar',
'--hide-boot-device',
'--turn-url',
'turns:turn.example.test:443',
]);
});
it('pins the requested package version', () => {
expect(createExpoDeviceHubArgs({ port: 4321, packageVersion: '0.7.0' }).slice(0, 2)).toEqual([
'--yes',
'[email protected]',
]);
});
});
describe(metricsCorsOriginToServeSimArgs, () => {
it('returns no args when the origin is unset or empty', () => {
expect(metricsCorsOriginToServeSimArgs({} as BuildStepEnv)).toEqual([]);
@@ -200,7 +245,7 @@ describe(metricsCorsOriginToServeSimArgs, () => {
});
});
describe(waitForServeSimReadyAsync, () => {
describe(waitForWebPreviewReadyAsync, () => {
beforeEach(() => {
jest.mocked(turtleFetch).mockReset();
jest.mocked(sleepAsync).mockReset();
@@ -215,8 +260,9 @@ describe(waitForServeSimReadyAsync, () => {
json: async () => ({ status: 'ready', device: 'DEVICE-A' }),
} as unknown as Awaited<ReturnType<typeof turtleFetch>>);
await waitForServeSimReadyAsync({
serveSim: { pid: undefined, getOutput: () => '' },
await waitForWebPreviewReadyAsync({
previewServer: { pid: undefined, getOutput: () => '' },
serverName: 'expo-device-hub',
port: 4321,
timeoutMs: 10_000,
});
@@ -261,14 +307,125 @@ describe(startNgrokTunnelAsync, () => {
});
});
describe(turnIceServersToServeSimArgs, () => {
describe(startDeviceWebPreviewWithTunnelAsync, () => {
const baseDomain = 'eas-simulator.ngrok.dev';
const turnArgs = [
'--turn-url',
'turns:turn.example.test:443',
'--turn-username',
'turn-user',
'--turn-credential',
'turn-credential',
];
const metricsCorsArgs = ['--metrics-cors-origin', 'https://metrics.expo.test'];
const env = {
DEVICE_RUN_SESSION_ID: 'drs-id',
EAS_SIMULATOR_METRICS_CORS_ORIGIN: 'https://metrics.expo.test',
NGROK_AUTHTOKEN: 'ngrok-token',
} as unknown as BuildStepEnv;
beforeEach(() => {
jest.mocked(spawn).mockReset();
jest.mocked(ngrok.forward).mockReset();
jest.mocked(turtleFetch).mockReset();
const spawnPromise = Object.assign(Promise.resolve(undefined), {
child: {
pid: undefined,
unref: jest.fn(),
},
});
jest.mocked(spawn).mockReturnValue(spawnPromise as never);
jest.mocked(turtleFetch).mockImplementation(async url => {
if (url.endsWith('/turn-ice-servers')) {
return {
json: async () => ({
data: {
iceServers: [
{
urls: ['turns:turn.example.test:443'],
username: 'turn-user',
credential: 'turn-credential',
},
],
},
}),
} as unknown as Awaited<ReturnType<typeof turtleFetch>>;
}
return {
json: async () => ({ status: 'ready', device: 'device-id' }),
} as unknown as Awaited<ReturnType<typeof turtleFetch>>;
});
});
it('starts expo-device-hub for Linux and cleans up the preview resources', async () => {
const packageVersion = '1.2.3';
const close = jest.fn().mockResolvedValue(undefined);
jest.mocked(ngrok.forward).mockResolvedValue({
url: () => 'https://android-preview.example.test',
close,
} as never);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.LINUX,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
packageVersion,
});
const [command, args] = jest.mocked(spawn).mock.calls[0];
const port = Number(args[args.indexOf('--port') + 1]);
expect(port).toBeGreaterThan(0);
expect(command).toBe('npx');
expect(args).toEqual(createExpoDeviceHubArgs({ port, turnArgs, packageVersion }));
expect(ngrok.forward).toHaveBeenCalledWith(expect.objectContaining({ addr: port }));
expect(preview.previewUrl).toBe('https://android-preview.example.test');
await preview.stopAsync();
expect(close).toHaveBeenCalledTimes(1);
});
it('starts serve-sim for Darwin with its metrics policy and cleans up the preview resources', async () => {
const packageVersion = '4.5.6';
const close = jest.fn().mockResolvedValue(undefined);
jest.mocked(ngrok.forward).mockResolvedValue({
url: () => 'https://ios-preview.example.test',
close,
} as never);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
packageVersion,
});
const [command, args] = jest.mocked(spawn).mock.calls[0];
const port = Number(args[args.indexOf('--port') + 1]);
expect(port).toBeGreaterThan(0);
expect(command).toBe('npx');
expect(args).toEqual(createServeSimArgs({ port, turnArgs, metricsCorsArgs, packageVersion }));
expect(ngrok.forward).toHaveBeenCalledWith(expect.objectContaining({ addr: port }));
expect(preview.previewUrl).toBe('https://ios-preview.example.test');
await preview.stopAsync();
expect(close).toHaveBeenCalledTimes(1);
});
});
describe(turnIceServersToWebPreviewArgs, () => {
it('returns no args for an empty ICE server list', () => {
expect(turnIceServersToServeSimArgs([])).toEqual([]);
expect(turnIceServersToWebPreviewArgs([])).toEqual([]);
});
it('builds --stun-url and --turn-url flags from Cloudflare ICE servers', () => {
expect(
turnIceServersToServeSimArgs([
turnIceServersToWebPreviewArgs([
{ urls: ['stun:stun.cloudflare.com:3478', 'stun:stun.cloudflare.com:53'] },
{
urls: [
@@ -293,7 +450,7 @@ describe(turnIceServersToServeSimArgs, () => {
it('emits only --turn-url flags when no credential-less (STUN) entry is present', () => {
expect(
turnIceServersToServeSimArgs([
turnIceServersToWebPreviewArgs([
{
urls: ['turns:turn.cloudflare.com:443?transport=tcp'],
username: 'u',
@@ -311,19 +468,19 @@ describe(turnIceServersToServeSimArgs, () => {
});
it('emits only --stun-url when there is no credentialed TURN entry', () => {
expect(turnIceServersToServeSimArgs([{ urls: ['stun:stun.cloudflare.com:3478'] }])).toEqual([
expect(turnIceServersToWebPreviewArgs([{ urls: ['stun:stun.cloudflare.com:3478'] }])).toEqual([
'--stun-url',
'stun:stun.cloudflare.com:3478',
]);
});
});
describe(fetchServeSimTurnArgsAsync, () => {
describe(fetchWebPreviewTurnArgsAsync, () => {
beforeEach(() => {
jest.mocked(turtleFetch).mockReset();
});
it('requests TURN ICE servers from the device run session endpoint and returns serve-sim args', async () => {
it('requests TURN ICE servers from the device run session endpoint and returns web preview args', async () => {
jest.mocked(turtleFetch).mockResolvedValue({
json: async () => ({
data: {
@@ -339,7 +496,7 @@ describe(fetchServeSimTurnArgsAsync, () => {
}),
} as unknown as Awaited<ReturnType<typeof turtleFetch>>);
const args = await fetchServeSimTurnArgsAsync(createCtxMock(), {
const args = await fetchWebPreviewTurnArgsAsync(createCtxMock(), {
env: createEnvMock(),
logger: createLoggerMock(),
});
@@ -363,11 +520,11 @@ describe(fetchServeSimTurnArgsAsync, () => {
);
});
it('returns [] and warns when the request fails so serve-sim falls back to P2P/STUN', async () => {
it('returns [] and warns when the request fails so the web preview falls back to P2P/STUN', async () => {
jest.mocked(turtleFetch).mockRejectedValue(new Error('boom'));
const logger = createLoggerMock();
const args = await fetchServeSimTurnArgsAsync(createCtxMock(), {
const args = await fetchWebPreviewTurnArgsAsync(createCtxMock(), {
env: createEnvMock(),
logger,
});
@@ -19,12 +19,16 @@ import { sleepAsync } from '../../utils/retry';
import { turtleFetch } from '../../utils/turtleFetch';
const XCODE_DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer';
const WEB_PREVIEW_HOST = '127.0.0.1';
const SERVE_SIM_PACKAGE_NAME = '@expo/serve-sim';
const SERVE_SIM_HOST = '127.0.0.1';
const SERVE_SIM_MAX_DIMENSION = '1280';
const SERVE_SIM_MAX_DIMENSION = '960';
const SERVE_SIM_MJPEG_QUALITY = '0.55';
const SERVE_SIM_VIDEO_BITRATE = '3000000';
const SERVE_SIM_VIDEO_BITRATE = '6000000';
const SERVE_SIM_VIDEO_FPS = '60';
const EXPO_DEVICE_HUB_PACKAGE_NAME = 'expo-device-hub';
const EXPO_DEVICE_HUB_MAX_DIMENSION = '1280';
const EXPO_DEVICE_HUB_VIDEO_BITRATE = '6000000';
const EXPO_DEVICE_HUB_VIDEO_FPS = '60';
const START_DEVICE_RUN_SESSION_MUTATION = graphql(`
mutation StartDeviceRunSession($deviceRunSessionId: ID!, $remoteConfig: JSONObject!) {
@@ -390,11 +394,12 @@ const TurnIceServersResponseSchema = z.object({
});
/**
* Translate Cloudflare ICE servers into serve-sim CLI flags: `--stun-url` (the
* Translate Cloudflare ICE servers into web preview CLI flags: `--stun-url` (the
* credential-less entries) and `--turn-url`/`--turn-username`/`--turn-credential`
* (the entry carrying the short-lived credentials).
* (the entry carrying the short-lived credentials). serve-sim and expo-device-hub
* intentionally expose the same ICE flag contract.
*/
export function turnIceServersToServeSimArgs(iceServers: TurnIceServers): string[] {
export function turnIceServersToWebPreviewArgs(iceServers: TurnIceServers): string[] {
const stunUrls = iceServers
.filter(server => !server.username && !server.credential)
.flatMap(server => server.urls);
@@ -420,14 +425,14 @@ export function turnIceServersToServeSimArgs(iceServers: TurnIceServers): string
/**
* Fetch short-lived Cloudflare TURN ICE servers for this job run from www
* (minted on demand, mirroring how the worker fetches project clone URLs) and
* translate them into serve-sim CLI flags.
* translate them into web preview CLI flags.
*
* Best-effort: on any failure we log and return [] so serve-sim falls back to
* its built-in P2P/STUN behavior. The credential is passed to serve-sim as a
* process arg and deliberately not logged (turtle-spawn never logs argv and the
* worker is single-tenant).
* Best-effort: on any failure we log and return [] so the preview server falls
* back to its built-in P2P/STUN behavior. The credential is passed as a process
* arg and deliberately not logged (turtle-spawn never logs argv and the worker
* is single-tenant).
*/
export async function fetchServeSimTurnArgsAsync(
export async function fetchWebPreviewTurnArgsAsync(
ctx: CustomBuildContext,
{ env, logger }: { env: BuildStepEnv; logger: bunyan }
): Promise<string[]> {
@@ -456,9 +461,9 @@ export async function fetchServeSimTurnArgsAsync(
);
const { data } = TurnIceServersResponseSchema.parse(await response.json());
const args = turnIceServersToServeSimArgs(data.iceServers);
const args = turnIceServersToWebPreviewArgs(data.iceServers);
if (args.length > 0) {
logger.info('Configured serve-sim with Cloudflare TURN ICE servers.');
logger.info('Configured the web preview with Cloudflare TURN ICE servers.');
}
return args;
} catch (err) {
@@ -466,7 +471,7 @@ export async function fetchServeSimTurnArgsAsync(
Sentry.capture('Could not fetch Cloudflare TURN ICE servers', error, { level: 'warning' });
logger.warn(
{ err: error },
'Could not fetch Cloudflare TURN ICE servers; serve-sim will fall back to P2P/STUN.'
'Could not fetch Cloudflare TURN ICE servers; the web preview will fall back to P2P/STUN.'
);
return [];
}
@@ -600,6 +605,10 @@ function createServeSimPackageSpec(packageVersion: string | undefined): string {
return `${SERVE_SIM_PACKAGE_NAME}@${packageVersion ?? 'latest'}`;
}
function createExpoDeviceHubPackageSpec(packageVersion: string | undefined): string {
return `${EXPO_DEVICE_HUB_PACKAGE_NAME}@${packageVersion ?? 'latest'}`;
}
export function createServeSimArgs({
port,
turnArgs = [],
@@ -617,7 +626,7 @@ export function createServeSimArgs({
'--port',
String(port),
'--host',
SERVE_SIM_HOST,
WEB_PREVIEW_HOST,
'--transport',
'webrtc',
'--webrtc-codec',
@@ -635,44 +644,84 @@ export function createServeSimArgs({
];
}
export function createExpoDeviceHubArgs({
port,
turnArgs = [],
packageVersion,
}: {
port: number;
turnArgs?: string[];
packageVersion?: string;
}): string[] {
return [
'--yes',
createExpoDeviceHubPackageSpec(packageVersion),
'--port',
String(port),
'--host',
WEB_PREVIEW_HOST,
'--platform',
'android',
'--transport',
'webrtc',
'--webrtc-codec',
'h264',
'--webrtc-ice-policy',
'all',
'--max-dimension',
EXPO_DEVICE_HUB_MAX_DIMENSION,
'--video-bitrate',
EXPO_DEVICE_HUB_VIDEO_BITRATE,
'--video-fps',
EXPO_DEVICE_HUB_VIDEO_FPS,
'--hide-sidebar',
'--hide-boot-device',
...turnArgs,
];
}
async function findAvailablePortAsync(): Promise<number> {
const server = createServer();
server.unref();
await new Promise<void>((resolve, reject) => {
server.once('error', reject);
server.listen(0, SERVE_SIM_HOST, () => resolve());
server.listen(0, WEB_PREVIEW_HOST, () => resolve());
});
const address = server.address();
await new Promise<void>((resolve, reject) => {
server.close(err => (err ? reject(err) : resolve()));
});
if (!address || typeof address === 'string') {
throw new SystemError('Could not allocate a local port for serve-sim.');
throw new SystemError('Could not allocate a local port for the web preview.');
}
return address.port;
}
const ServeSimReadyResponseSchema = z.object({
const WebPreviewReadyResponseSchema = z.object({
status: z.literal('ready'),
device: z.string(),
});
export async function waitForServeSimReadyAsync({
serveSim,
export async function waitForWebPreviewReadyAsync({
previewServer,
serverName,
port,
timeoutMs,
}: {
serveSim: Pick<DetachedProcessHandle, 'pid' | 'getOutput'>;
previewServer: Pick<DetachedProcessHandle, 'pid' | 'getOutput'>;
serverName: string;
port: number;
timeoutMs: number;
}): Promise<void> {
const readyUrl = `http://${SERVE_SIM_HOST}:${port}/readyz`;
const readyUrl = `http://${WEB_PREVIEW_HOST}:${port}/readyz`;
const deadline = Date.now() + timeoutMs;
let lastError: unknown;
while (Date.now() < deadline) {
if (serveSim.pid !== undefined && !isProcessRunning(serveSim.pid)) {
if (previewServer.pid !== undefined && !isProcessRunning(previewServer.pid)) {
throw new SystemError(
`serve-sim exited before becoming ready. Last output:\n${serveSim.getOutput() || '<empty>'}`
`${serverName} exited before becoming ready. Last output:\n${
previewServer.getOutput() || '<empty>'
}`
);
}
try {
@@ -680,7 +729,7 @@ export async function waitForServeSimReadyAsync({
retries: 0,
timeout: 2_000,
});
ServeSimReadyResponseSchema.parse(await response.json());
WebPreviewReadyResponseSchema.parse(await response.json());
return;
} catch (error) {
lastError = error;
@@ -688,17 +737,75 @@ export async function waitForServeSimReadyAsync({
await sleepAsync(1_000);
}
throw new SystemError(
`Timed out waiting for serve-sim readiness at ${readyUrl}${
`Timed out waiting for ${serverName} readiness at ${readyUrl}${
lastError instanceof Error ? `: ${lastError.message}` : ''
}. Last output:\n${serveSim.getOutput() || '<empty>'}`
}. Last output:\n${previewServer.getOutput() || '<empty>'}`
);
}
export type ServeSimPreviewHandle = {
export type DeviceWebPreviewHandle = {
previewUrl: string;
stopAsync: () => Promise<void>;
};
export type ServeSimPreviewHandle = DeviceWebPreviewHandle;
async function startWebPreviewWithTunnelAsync(
ctx: CustomBuildContext,
{
baseDomain,
env,
logger,
timeoutMs,
serverName,
packageSpec,
createArgs,
}: {
baseDomain: string;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
serverName: string;
packageSpec: string;
createArgs: (port: number, turnArgs: string[]) => string[];
}
): Promise<DeviceWebPreviewHandle> {
const port = await findAvailablePortAsync();
logger.info(`Launching ${packageSpec} on ${WEB_PREVIEW_HOST}:${port}.`);
const turnArgs = await fetchWebPreviewTurnArgsAsync(ctx, { env, logger });
const previewServer = spawnDetached({
command: 'npx',
args: createArgs(port, turnArgs),
env,
});
try {
logger.info(`Waiting for ${serverName} to become ready.`);
await waitForWebPreviewReadyAsync({ previewServer, serverName, port, timeoutMs });
const tunnel = await startNgrokTunnelAsync({
port,
subdomainPrefix: 'web-preview',
baseDomain,
authtoken: getNgrokAuthtokenOrThrow(env),
logger,
});
return {
previewUrl: tunnel.url,
stopAsync: async () => {
const results = await Promise.allSettled([tunnel.stopAsync(), previewServer.stopAsync()]);
for (const result of results) {
if (result.status === 'rejected') {
logger.warn({ err: result.reason }, `Could not stop a ${serverName} preview resource.`);
}
}
},
};
} catch (error) {
await previewServer.stopAsync();
throw error;
}
}
export async function startServeSimWithTunnelAsync(
ctx: CustomBuildContext,
{
@@ -715,42 +822,65 @@ export async function startServeSimWithTunnelAsync(
packageVersion?: string;
}
): Promise<ServeSimPreviewHandle> {
const port = await findAvailablePortAsync();
logger.info(
`Launching ${createServeSimPackageSpec(packageVersion)} on ${SERVE_SIM_HOST}:${port}.`
);
const turnArgs = await fetchServeSimTurnArgsAsync(ctx, { env, logger });
const metricsCorsArgs = metricsCorsOriginToServeSimArgs(env);
const serveSim = spawnDetached({
command: 'npx',
args: createServeSimArgs({ port, turnArgs, metricsCorsArgs, packageVersion }),
return await startWebPreviewWithTunnelAsync(ctx, {
baseDomain,
env,
logger,
timeoutMs,
serverName: 'serve-sim',
packageSpec: createServeSimPackageSpec(packageVersion),
createArgs: (port, turnArgs) =>
createServeSimArgs({ port, turnArgs, metricsCorsArgs, packageVersion }),
});
}
try {
logger.info('Waiting for serve-sim to become ready.');
await waitForServeSimReadyAsync({ serveSim, port, timeoutMs });
const tunnel = await startNgrokTunnelAsync({
port,
subdomainPrefix: 'web-preview',
baseDomain,
authtoken: getNgrokAuthtokenOrThrow(env),
logger,
});
return {
previewUrl: tunnel.url,
stopAsync: async () => {
const results = await Promise.allSettled([tunnel.stopAsync(), serveSim.stopAsync()]);
for (const result of results) {
if (result.status === 'rejected') {
logger.warn({ err: result.reason }, 'Could not stop a serve-sim preview resource.');
}
}
},
};
} catch (error) {
await serveSim.stopAsync();
throw error;
export async function startExpoDeviceHubWithTunnelAsync(
ctx: CustomBuildContext,
{
baseDomain,
env,
logger,
timeoutMs,
packageVersion,
}: {
baseDomain: string;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
packageVersion?: string;
}
): Promise<DeviceWebPreviewHandle> {
return await startWebPreviewWithTunnelAsync(ctx, {
baseDomain,
env,
logger,
timeoutMs,
serverName: 'expo-device-hub',
packageSpec: createExpoDeviceHubPackageSpec(packageVersion),
createArgs: (port, turnArgs) => createExpoDeviceHubArgs({ port, turnArgs, packageVersion }),
});
}
export async function startDeviceWebPreviewWithTunnelAsync(
ctx: CustomBuildContext,
{
runtimePlatform,
...options
}: {
runtimePlatform: BuildRuntimePlatform;
baseDomain: string;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
packageVersion?: string;
}
): Promise<DeviceWebPreviewHandle> {
switch (runtimePlatform) {
case BuildRuntimePlatform.DARWIN:
return await startServeSimWithTunnelAsync(ctx, options);
case BuildRuntimePlatform.LINUX:
return await startExpoDeviceHubWithTunnelAsync(ctx, options);
}
}
@@ -0,0 +1,78 @@
import fs from 'fs';
import os from 'os';
import path from 'path';
import {
compressCocoapodsCacheAsync,
getCocoapodsCachePaths,
resolveCocoapodsCacheKeyAsync,
restoreCocoapodsCacheArchiveAsync,
} from '../cocoapodsCache';
jest.unmock('fs');
jest.unmock('node:fs');
jest.unmock('fs/promises');
jest.unmock('node:fs/promises');
describe('CocoaPods cache utilities', () => {
let workingDirectory: string;
const cocoapodsVersion = '1.16.2';
beforeEach(async () => {
workingDirectory = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'cocoapods-cache-test-'));
await fs.promises.mkdir(path.join(workingDirectory, 'ios'), { recursive: true });
});
afterEach(async () => {
await fs.promises.rm(workingDirectory, { recursive: true, force: true });
});
it('uses a CocoaPods version prefix when no Podfile.lock exists', async () => {
await expect(
resolveCocoapodsCacheKeyAsync(workingDirectory, cocoapodsVersion)
).resolves.toEqual({
key: 'ios-pods-1.16.2-',
keyPrefix: 'ios-pods-1.16.2-',
});
});
it('includes the Podfile.lock hash when it exists', async () => {
const { podfileLockPath } = getCocoapodsCachePaths(workingDirectory);
await fs.promises.writeFile(podfileLockPath, 'PODS:\n - Expo (55.0.0)\n');
const firstResult = await resolveCocoapodsCacheKeyAsync(workingDirectory, cocoapodsVersion);
expect(firstResult.key).toMatch(/^ios-pods-1\.16\.2-[a-f0-9]+$/);
expect(firstResult.keyPrefix).toBe('ios-pods-1.16.2-');
await fs.promises.writeFile(podfileLockPath, 'PODS:\n - Expo (56.0.0)\n');
const secondResult = await resolveCocoapodsCacheKeyAsync(workingDirectory, cocoapodsVersion);
expect(secondResult.key).not.toBe(firstResult.key);
});
it('preserves symlinks and executable permissions through an archive round trip', async () => {
const { podsDirectory } = getCocoapodsCachePaths(workingDirectory);
const frameworkVersionsDirectory = path.join(podsDirectory, 'Example.framework', 'Versions');
const frameworkVersionDirectory = path.join(frameworkVersionsDirectory, 'A');
await fs.promises.mkdir(frameworkVersionDirectory, { recursive: true });
await fs.promises.writeFile(path.join(frameworkVersionDirectory, 'Example'), 'binary');
await fs.promises.symlink('A', path.join(frameworkVersionsDirectory, 'Current'));
const scriptPath = path.join(podsDirectory, 'Target Support Files', 'script.sh');
await fs.promises.mkdir(path.dirname(scriptPath), { recursive: true });
await fs.promises.writeFile(scriptPath, '#!/bin/sh\n');
await fs.promises.chmod(scriptPath, 0o755);
const { archivePath } = await compressCocoapodsCacheAsync({ workingDirectory });
await fs.promises.rm(podsDirectory, { recursive: true, force: true });
await restoreCocoapodsCacheArchiveAsync({ archivePath, workingDirectory });
const restoredSymlinkPath = path.join(frameworkVersionsDirectory, 'Current');
expect((await fs.promises.lstat(restoredSymlinkPath)).isSymbolicLink()).toBe(true);
await expect(fs.promises.readlink(restoredSymlinkPath)).resolves.toBe('A');
const restoredScriptStat = await fs.promises.stat(scriptPath);
expect(restoredScriptStat.mode & 0o111).toBe(0o111);
await fs.promises.rm(path.dirname(archivePath), { recursive: true, force: true });
});
});
@@ -78,7 +78,7 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(result).toEqual({
cmd: 'npx',
args: ['-y', `eas-cli@${STAGING_VERSION}`],
extraEnv: {},
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
});
@@ -91,7 +91,7 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(result).toEqual({
cmd: 'npx',
args: ['-y', `eas-cli@${STAGING_VERSION}`],
extraEnv: {},
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
});
@@ -101,7 +101,7 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(result).toEqual({
cmd: 'npx',
args: ['-y', `eas-cli@${STAGING_VERSION}`],
extraEnv: { EXPO_STAGING: '1' },
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0', EXPO_STAGING: '1' },
});
expect(spawn).not.toHaveBeenCalled();
});
@@ -112,7 +112,7 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(result).toEqual({
cmd: 'npx',
args: ['-y', `eas-cli@${PRODUCTION_VERSION}`],
extraEnv: {},
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
expect(spawn).not.toHaveBeenCalled();
expect(Sentry.capture).not.toHaveBeenCalled();
@@ -133,7 +133,7 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(stagingResult).toEqual({
cmd: 'npx',
args: ['-y', `eas-cli@${EasCliNpmTags.STAGING}`],
extraEnv: { EXPO_STAGING: '1' },
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0', EXPO_STAGING: '1' },
});
process.env.ENVIRONMENT = 'production';
@@ -141,7 +141,7 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(productionResult).toEqual({
cmd: 'npx',
args: ['-y', `eas-cli@${EasCliNpmTags.PRODUCTION}`],
extraEnv: {},
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
expect(Sentry.capture).toHaveBeenCalledWith(
@@ -173,9 +173,17 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(result).toEqual({
cmd: 'npx',
args: [`eas-cli@${PRODUCTION_VERSION}`],
extraEnv: {},
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
});
it('disables the minimum release age for every npx invocation', async () => {
process.env.ENVIRONMENT = 'production';
const result = await resolveEasCommandPrefixAndEnvAsync();
expect(result.extraEnv).toEqual({ NPM_CONFIG_MIN_RELEASE_AGE: '0' });
});
});
describe(runEasCliCommand, () => {
@@ -218,4 +226,25 @@ describe(runEasCliCommand, () => {
})
);
});
it('overrides a user-provided minimum release age', async () => {
process.env.ENVIRONMENT = 'production';
await runEasCliCommand({
args: ['deploy', '--json'],
options: {
cwd: '/tmp/project',
env: {
FOO: 'bar',
NPM_CONFIG_MIN_RELEASE_AGE: '365',
},
},
});
const spawnEnv = jest.mocked(spawn).mock.calls[0][2]?.env;
expect(spawnEnv).toEqual({
FOO: 'bar',
NPM_CONFIG_MIN_RELEASE_AGE: '0',
});
});
});
@@ -0,0 +1,17 @@
import { formatSecondsForLog } from '../formatDuration';
describe(formatSecondsForLog, () => {
it.each([
[0, '0 seconds'],
[1, '1 second'],
[45, '45 seconds'],
[60, '1 minute'],
[90, '1 minute 30 seconds'],
[120, '2 minutes'],
[3600, '1 hour'],
[3661, '1 hour 1 minute 1 second'],
[7200, '2 hours'],
] as const)('formats %i as %s', (seconds, expected) => {
expect(formatSecondsForLog(seconds)).toBe(expected);
});
});
@@ -0,0 +1,43 @@
import { ChildProcess } from 'node:child_process';
import { isChildProcessAlive } from '../processes';
function child(partial: {
exitCode: number | null;
signalCode: NodeJS.Signals | null;
killed: boolean;
}): ChildProcess {
return partial as ChildProcess;
}
describe(isChildProcessAlive, () => {
it('is alive while the process is running', () => {
expect(isChildProcessAlive(child({ exitCode: null, signalCode: null, killed: false }))).toBe(
true
);
});
it('is dead after a normal exit', () => {
expect(isChildProcessAlive(child({ exitCode: 0, signalCode: null, killed: false }))).toBe(
false
);
expect(isChildProcessAlive(child({ exitCode: 1, signalCode: null, killed: false }))).toBe(
false
);
});
it('is dead after an external signal termination (the regression case)', () => {
expect(
isChildProcessAlive(child({ exitCode: null, signalCode: 'SIGTERM', killed: false }))
).toBe(false);
expect(
isChildProcessAlive(child({ exitCode: null, signalCode: 'SIGKILL', killed: false }))
).toBe(false);
});
it('is dead once we have killed it', () => {
expect(isChildProcessAlive(child({ exitCode: null, signalCode: null, killed: true }))).toBe(
false
);
});
});
@@ -0,0 +1,503 @@
import { SshSettings, SystemError } from '@expo/eas-build-job';
import { createMockLogger } from '../../__tests__/utils/logger';
import { BuildContext } from '../../context';
import { sleepAsync } from '../retry';
import {
SshSessionHandle,
getSshIdleTimeoutSeconds,
getSshRelayServerUrl,
isSshEnabled,
startSshSessionAsync,
superviseSshSessionAsync,
} from '../turtleSshSession';
import { SshConnectionConfig, UptermHost, startUptermHostAsync } from '../upterm';
jest.mock('../retry', () => ({
sleepAsync: jest.fn().mockResolvedValue(undefined),
}));
jest.mock('../upterm', () => ({
...jest.requireActual('../upterm'),
startUptermHostAsync: jest.fn(),
}));
const mockedStartUptermHost = jest.mocked(startUptermHostAsync);
function sshJob(idleTimeoutSeconds: number): { ssh: SshSettings } {
return { ssh: { idleTimeoutSeconds, relayServerUrl: 'wss://relay.expo.dev' } };
}
function superviseHandle(
overrides: Partial<SshSessionHandle> &
Pick<SshSessionHandle, 'getConnectedClientCountAsync' | 'ensureConnectedAsync'>
): SshSessionHandle {
return {
stopAsync: async () => {},
...overrides,
};
}
describe(isSshEnabled, () => {
it('is true when job.ssh is present', () => {
expect(isSshEnabled(sshJob(0))).toBe(true);
});
it('is false otherwise', () => {
expect(isSshEnabled({})).toBe(false);
});
});
describe(getSshIdleTimeoutSeconds, () => {
it('returns idleTimeoutSeconds from job.ssh', () => {
expect(getSshIdleTimeoutSeconds(sshJob(900))).toBe(900);
expect(getSshIdleTimeoutSeconds(sshJob(0))).toBe(0);
expect(getSshIdleTimeoutSeconds(sshJob(3600))).toBe(3600);
});
it('defaults to 0 when job.ssh is missing', () => {
expect(getSshIdleTimeoutSeconds({})).toBe(0);
});
it('throws when the value is not an integer in range', () => {
for (const idleTimeoutSeconds of [300.5, -1, 3601]) {
expect(() => getSshIdleTimeoutSeconds(sshJob(idleTimeoutSeconds))).toThrow(SystemError);
}
});
});
describe(getSshRelayServerUrl, () => {
it('returns the relay url from the job ssh settings', () => {
expect(getSshRelayServerUrl(sshJob(0))).toBe('wss://relay.expo.dev');
});
it('throws when the job carries no relay url', () => {
expect(() => getSshRelayServerUrl({})).toThrow(/no relay server URL/);
});
});
describe(startSshSessionAsync, () => {
const config1: SshConnectionConfig = {
type: 'upterm-v1',
host: 'relay.expo.dev',
secret: 'secret-1',
};
const config2: SshConnectionConfig = { ...config1, secret: 'secret-2' };
const target = { type: 'JOB_RUN', id: 'jr-1' } as const;
let createOrUpdateResult: { data?: unknown; error?: { message: string } };
let mutation: jest.Mock;
let ctx: BuildContext;
function makeHost(overrides: Partial<UptermHost> = {}): UptermHost {
const host: UptermHost = {
connectionConfig: config1,
getConnectedClientCountAsync: jest.fn().mockResolvedValue(3),
isAlive: jest.fn().mockReturnValue(true),
redialAsync: jest.fn(async () => {
host.connectionConfig = config2;
return config2;
}),
stopAsync: jest.fn().mockResolvedValue(undefined),
...overrides,
};
return host;
}
beforeEach(() => {
createOrUpdateResult = {
data: {
turtleSshSession: {
createOrUpdateTurtleSshSession: {
id: 'ts-1',
sessionSettings: { idleTimeoutSeconds: 900 },
},
},
},
};
mutation = jest.fn().mockReturnValue({ toPromise: async () => createOrUpdateResult });
ctx = {
logger: createMockLogger(),
graphqlClient: { mutation },
} as unknown as BuildContext;
});
it('creates or updates the session and returns a handle plus idle timeout that delegates to the host', async () => {
const host = makeHost();
mockedStartUptermHost.mockResolvedValue(host);
const { handle, idleTimeoutSeconds } = await startSshSessionAsync(ctx, {
target,
relayServerUrl: 'wss://relay.expo.dev',
idleTimeoutSeconds: 900,
});
expect(mutation).toHaveBeenCalledWith(expect.anything(), {
target: { type: 'JOB_RUN', id: 'jr-1' },
connectionConfig: { ...config1, reconnecting: false, type: 'UPTERM_V1' },
sessionSettings: { idleTimeoutSeconds: 900 },
});
expect(idleTimeoutSeconds).toBe(900);
await handle.getConnectedClientCountAsync();
expect(host.getConnectedClientCountAsync).toHaveBeenCalled();
await handle.stopAsync();
expect(host.stopAsync).toHaveBeenCalled();
});
it('reports a build target', async () => {
const host = makeHost();
mockedStartUptermHost.mockResolvedValue(host);
await startSshSessionAsync(ctx, {
target: { type: 'BUILD', id: 'b-1' },
relayServerUrl: 'wss://relay.expo.dev',
idleTimeoutSeconds: 900,
});
expect(mutation).toHaveBeenCalledWith(expect.anything(), {
target: { type: 'BUILD', id: 'b-1' },
connectionConfig: { ...config1, reconnecting: false, type: 'UPTERM_V1' },
sessionSettings: { idleTimeoutSeconds: 900 },
});
});
it('throws and tears down the dialed host when creating or updating the session fails', async () => {
const host = makeHost();
mockedStartUptermHost.mockResolvedValue(host);
createOrUpdateResult = { error: { message: 'boom' } };
await expect(
startSshSessionAsync(ctx, {
target,
relayServerUrl: 'wss://r',
idleTimeoutSeconds: 300,
})
).rejects.toThrow('boom');
expect(host.stopAsync).toHaveBeenCalled();
});
it('still rethrows when teardown after a create/update failure also fails', async () => {
const host = makeHost({
stopAsync: jest.fn().mockRejectedValue(new Error('already gone')),
});
mockedStartUptermHost.mockResolvedValue(host);
createOrUpdateResult = { error: { message: 'boom' } };
await expect(
startSshSessionAsync(ctx, {
target,
relayServerUrl: 'wss://r',
idleTimeoutSeconds: 300,
})
).rejects.toThrow('boom');
expect(host.stopAsync).toHaveBeenCalled();
});
it('throws "no data returned" when create or update succeeds without a payload', async () => {
const host = makeHost();
mockedStartUptermHost.mockResolvedValue(host);
createOrUpdateResult = {};
await expect(
startSshSessionAsync(ctx, {
target,
relayServerUrl: 'wss://r',
idleTimeoutSeconds: 300,
})
).rejects.toThrow(/no data returned/);
expect(host.stopAsync).toHaveBeenCalled();
});
it('ensureConnected is a no-op while the host is alive', async () => {
const host = makeHost({ isAlive: jest.fn().mockReturnValue(true) });
mockedStartUptermHost.mockResolvedValue(host);
const { handle } = await startSshSessionAsync(ctx, {
target,
relayServerUrl: 'wss://r',
idleTimeoutSeconds: 300,
});
await handle.ensureConnectedAsync();
expect(host.redialAsync).not.toHaveBeenCalled();
expect(mutation).toHaveBeenCalledTimes(1);
});
it('marks reconnecting, redials, then reports the fresh config', async () => {
const host = makeHost({ isAlive: jest.fn().mockReturnValue(false) });
mockedStartUptermHost.mockResolvedValue(host);
const { handle } = await startSshSessionAsync(ctx, {
target,
relayServerUrl: 'wss://r',
idleTimeoutSeconds: 300,
});
await handle.ensureConnectedAsync();
expect(host.redialAsync).toHaveBeenCalledTimes(1);
expect(mutation).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({
connectionConfig: expect.objectContaining({ reconnecting: true }),
})
);
expect(mutation).toHaveBeenLastCalledWith(expect.anything(), {
target: { type: 'JOB_RUN', id: 'jr-1' },
connectionConfig: { ...config2, reconnecting: false, type: 'UPTERM_V1' },
sessionSettings: { idleTimeoutSeconds: 300 },
});
});
it('continues redial when the reconnecting status update fails', async () => {
const host = makeHost({ isAlive: jest.fn().mockReturnValue(false) });
mockedStartUptermHost.mockResolvedValue(host);
mutation
.mockReturnValueOnce({ toPromise: async () => createOrUpdateResult }) // initial create
.mockReturnValueOnce({
toPromise: async () => {
throw new Error('www unreachable');
},
}) // reconnecting=true best-effort
.mockReturnValue({ toPromise: async () => createOrUpdateResult }); // post-redial
const { handle } = await startSshSessionAsync(ctx, {
target,
relayServerUrl: 'wss://r',
idleTimeoutSeconds: 300,
});
await handle.ensureConnectedAsync();
expect(host.redialAsync).toHaveBeenCalledTimes(1);
});
it('retries only the report (no second redial) when reporting the fresh config fails', async () => {
let alive = false;
const host = makeHost({
isAlive: jest.fn(() => alive),
});
host.redialAsync = jest.fn(async () => {
alive = true;
host.connectionConfig = config2;
return config2;
});
mockedStartUptermHost.mockResolvedValue(host);
mutation = jest
.fn()
.mockReturnValueOnce({ toPromise: async () => createOrUpdateResult })
.mockReturnValueOnce({ toPromise: async () => createOrUpdateResult })
.mockReturnValueOnce({ toPromise: async () => ({ error: { message: 'flaky' } }) })
.mockReturnValue({ toPromise: async () => createOrUpdateResult });
ctx = { logger: createMockLogger(), graphqlClient: { mutation } } as unknown as BuildContext;
const { handle } = await startSshSessionAsync(ctx, {
target,
relayServerUrl: 'wss://r',
idleTimeoutSeconds: 300,
});
await handle.ensureConnectedAsync();
expect(host.redialAsync).toHaveBeenCalledTimes(1);
expect(mutation).toHaveBeenLastCalledWith(expect.anything(), {
target: { type: 'JOB_RUN', id: 'jr-1' },
connectionConfig: { ...config2, reconnecting: false, type: 'UPTERM_V1' },
sessionSettings: { idleTimeoutSeconds: 300 },
});
});
it('throws once the redial budget is exhausted', async () => {
const host = makeHost({
isAlive: jest.fn().mockReturnValue(false),
redialAsync: jest.fn().mockRejectedValue(new Error('relay down')),
});
mockedStartUptermHost.mockResolvedValue(host);
const { handle } = await startSshSessionAsync(ctx, {
target,
relayServerUrl: 'wss://r',
idleTimeoutSeconds: 300,
});
await expect(handle.ensureConnectedAsync()).rejects.toThrow(/after 10 attempts/);
expect(host.redialAsync).toHaveBeenCalledTimes(10);
});
});
describe(superviseSshSessionAsync, () => {
const logger = createMockLogger();
const mockedSleep = jest.mocked(sleepAsync);
beforeEach(() => {
jest.clearAllMocks();
mockedSleep.mockResolvedValue(undefined);
});
it('keeps the session open while the job is running, even with a 0 idle timeout', async () => {
let polls = 0;
await superviseSshSessionAsync({
handle: superviseHandle({
ensureConnectedAsync: async () => {
if (polls >= 5) {
throw new SystemError('stop the loop');
}
},
getConnectedClientCountAsync: async () => {
polls += 1;
return 0;
},
}),
idleTimeoutSeconds: 0,
hasJobFinished: () => false,
logger,
});
expect(polls).toBe(5);
});
it('closes as soon as the job finishes when the idle timeout is 0', async () => {
let polls = 0;
await superviseSshSessionAsync({
handle: superviseHandle({
ensureConnectedAsync: async () => {},
getConnectedClientCountAsync: async () => {
polls += 1;
return 0;
},
}),
idleTimeoutSeconds: 0,
hasJobFinished: () => true,
logger,
});
expect(polls).toBe(1);
expect(logger.info).toHaveBeenCalledWith(
'The job finished and no SSH client is connected. Closing the session.'
);
});
it('waits for a connected client to disconnect before closing with a 0 idle timeout', async () => {
const clientCounts = [1, 1, 0];
let polls = 0;
await superviseSshSessionAsync({
handle: superviseHandle({
ensureConnectedAsync: async () => {},
getConnectedClientCountAsync: async () => {
polls += 1;
return clientCounts.shift() ?? 0;
},
}),
idleTimeoutSeconds: 0,
hasJobFinished: () => true,
logger,
});
expect(polls).toBe(3);
});
it('starts the idle countdown when the job finishes, not when the session opens', async () => {
jest.useFakeTimers();
jest.setSystemTime(0);
try {
let jobHasFinished = false;
let polls = 0;
mockedSleep.mockImplementation(async () => {
jest.setSystemTime(Date.now() + 60_000);
});
await superviseSshSessionAsync({
handle: superviseHandle({
ensureConnectedAsync: async () => {},
getConnectedClientCountAsync: async () => {
polls += 1;
if (polls >= 5) {
jobHasFinished = true;
}
return 0;
},
}),
idleTimeoutSeconds: 300,
hasJobFinished: () => jobHasFinished,
logger,
});
// Nobody connects, so the 4 idle minutes spent during the job must not count towards the
// 5 minute timeout: it closes 5 minutes after the job finished, on the 10th poll.
expect(polls).toBe(10);
} finally {
jest.useRealTimers();
}
});
it('logs the connected client count whenever it changes', async () => {
const clientCounts = [1, 2, 1, 0];
await superviseSshSessionAsync({
handle: superviseHandle({
ensureConnectedAsync: async () => {},
getConnectedClientCountAsync: async () => clientCounts.shift() ?? 0,
}),
idleTimeoutSeconds: 0,
hasJobFinished: () => true,
logger,
});
expect(logger.info).toHaveBeenCalledWith('SSH clients connected: 1');
expect(logger.info).toHaveBeenCalledWith('SSH clients connected: 2');
expect(logger.info).toHaveBeenCalledWith('SSH clients connected: 1');
expect(logger.info).toHaveBeenCalledWith('SSH clients connected: 0');
});
it('closes the session when the relay connection cannot be restored', async () => {
let clientCountCalls = 0;
await superviseSshSessionAsync({
handle: superviseHandle({
ensureConnectedAsync: async () => {
throw new SystemError('relay down');
},
getConnectedClientCountAsync: async () => {
clientCountCalls += 1;
return 1;
},
}),
idleTimeoutSeconds: 300,
hasJobFinished: () => false,
logger,
});
expect(clientCountCalls).toBe(0);
});
it('does not close while a client stays connected, even past the idle timeout', async () => {
let polls = 0;
await superviseSshSessionAsync({
handle: superviseHandle({
ensureConnectedAsync: async () => {
if (polls > 3) {
throw new SystemError('stop the loop');
}
},
getConnectedClientCountAsync: async () => {
polls += 1;
return 1;
},
}),
idleTimeoutSeconds: 0,
hasJobFinished: () => true,
logger,
});
expect(polls).toBeGreaterThan(3);
});
it('closes when the client count cannot be read', async () => {
let polls = 0;
await superviseSshSessionAsync({
handle: superviseHandle({
ensureConnectedAsync: async () => {},
getConnectedClientCountAsync: async () => {
polls += 1;
throw new SystemError(
'Could not read the SSH client count from the upterm admin socket.'
);
},
}),
idleTimeoutSeconds: 300,
hasJobFinished: () => false,
logger,
});
expect(polls).toBe(1);
expect(logger.warn).toHaveBeenCalledWith(
expect.anything(),
'Could not read the SSH client count. Closing the session.'
);
});
});
@@ -0,0 +1,466 @@
import { SystemError } from '@expo/eas-build-job';
import downloadFile from '@expo/downloader';
import spawn from '@expo/turtle-spawn';
import fs from 'node:fs/promises';
import { createMockLogger } from '../../__tests__/utils/logger';
import { BuildContext } from '../../context';
import { sleepAsync } from '../retry';
import {
connectionConfigFromUptermSession,
redactConnectionSecrets,
redactSpawnErrorForLog,
resolveUptermGcsObjectName,
startUptermHostAsync,
} from '../upterm';
jest.mock('@expo/turtle-spawn', () => ({ __esModule: true, default: jest.fn() }));
jest.mock('@expo/downloader', () => ({ __esModule: true, default: jest.fn() }));
jest.mock('../retry', () => ({ sleepAsync: jest.fn() }));
jest.mock('node:fs/promises', () => ({
__esModule: true,
default: {
access: jest.fn(),
mkdir: jest.fn(),
chmod: jest.fn(),
mkdtemp: jest.fn(),
writeFile: jest.fn(),
rm: jest.fn(),
readdir: jest.fn(),
},
}));
describe(resolveUptermGcsObjectName, () => {
it('maps worker platforms to GCS object names', () => {
expect(resolveUptermGcsObjectName('darwin', 'arm64')).toBe('upterm-darwin-arm64');
expect(resolveUptermGcsObjectName('linux', 'x64')).toBe('upterm-linux-amd64');
});
it('rejects unsupported platforms', () => {
expect(() => resolveUptermGcsObjectName('darwin', 'x64')).toThrow(/only available/);
expect(() => resolveUptermGcsObjectName('linux', 'arm64')).toThrow(/only available/);
});
});
describe(connectionConfigFromUptermSession, () => {
it('parses sessionId and host from session current fields', () => {
expect(
connectionConfigFromUptermSession({
sessionId: 'TOKENabc123',
host: 'ssh://relay.expo.dev:22',
})
).toEqual({
type: 'upterm-v1',
host: 'relay.expo.dev',
secret: 'TOKENabc123',
});
});
it('accepts a bare hostname', () => {
expect(connectionConfigFromUptermSession({ sessionId: 'tok', host: 'relay.expo.dev' })).toEqual(
{
type: 'upterm-v1',
host: 'relay.expo.dev',
secret: 'tok',
}
);
});
it('keeps a non-default port so the CLI can pass it to ssh -p', () => {
expect(
connectionConfigFromUptermSession({
sessionId: 'tok',
host: 'ssh://relay.expo.dev:2222',
})
).toEqual({
type: 'upterm-v1',
host: 'relay.expo.dev:2222',
secret: 'tok',
});
});
it('returns null when the host is not a parseable url', () => {
expect(connectionConfigFromUptermSession({ sessionId: 'tok', host: 'ssh://' })).toBeNull();
expect(connectionConfigFromUptermSession({ sessionId: 'tok', host: 'ssh://[' })).toBeNull();
});
});
describe(redactConnectionSecrets, () => {
it('strips the userinfo secret from a connection string', () => {
expect(
redactConnectionSecrets('Connect: upterm proxy wss://[email protected] now')
).toBe('Connect: upterm proxy wss://<redacted>@relay.expo.dev now');
});
it('strips the bare token that trails the ssh ProxyCommand line', () => {
const line =
"ssh -o ProxyCommand='upterm proxy wss://[email protected]' [email protected]";
const redacted = redactConnectionSecrets(line);
expect(redacted).not.toContain('TOKENabc123');
expect(redacted).toBe(
"ssh -o ProxyCommand='upterm proxy wss://<redacted>@uptermd.upterm.dev' <redacted>@uptermd.upterm.dev"
);
});
it('strips the bare token for ws:// relay URLs too', () => {
const line =
"ssh -o ProxyCommand='upterm proxy ws://[email protected]' [email protected]";
const redacted = redactConnectionSecrets(line);
expect(redacted).not.toContain('TOKENabc123');
expect(redacted).toBe(
"ssh -o ProxyCommand='upterm proxy ws://<redacted>@relay.local' <redacted>@relay.local"
);
});
it('leaves output without userinfo untouched', () => {
expect(redactConnectionSecrets('dialing wss://relay.expo.dev')).toBe(
'dialing wss://relay.expo.dev'
);
});
it('strips control characters (incl. ESC) while keeping newlines', () => {
// ESC alone is enough to neutralize CSI; leftover printable "[31m" is harmless in logs.
expect(redactConnectionSecrets('ok\n\u001b[31mfail\u001b[0m\r\nbad\u0007')).toBe(
'ok\n[31mfail[0m\nbad'
);
});
});
describe(redactSpawnErrorForLog, () => {
it('redacts secrets in message, stdout, and stderr', () => {
expect(
redactSpawnErrorForLog({
message: 'failed wss://[email protected]',
stdout: 'upterm proxy wss://[email protected]',
stderr: 'ssh dial error: wss://[email protected]',
code: 1,
})
).toEqual({
message: 'failed wss://<redacted>@relay.expo.dev',
stdout: 'upterm proxy wss://<redacted>@relay.expo.dev',
stderr: 'ssh dial error: wss://<redacted>@relay.expo.dev',
code: 1,
});
});
it('passes through non-objects', () => {
expect(redactSpawnErrorForLog('boom')).toBe('boom');
expect(redactSpawnErrorForLog(null)).toBeNull();
});
it('leaves non-string message/stdout/stderr fields alone', () => {
expect(
redactSpawnErrorForLog({
message: 42,
stdout: Buffer.from('x'),
code: 1,
})
).toEqual({
message: 42,
stdout: Buffer.from('x'),
code: 1,
});
});
});
describe(startUptermHostAsync, () => {
const mockedSpawn = jest.mocked(spawn);
const mockedSleep = jest.mocked(sleepAsync);
const mockedFs = jest.mocked(fs);
const mockedDownloadFile = jest.mocked(downloadFile);
let hostProcess: {
child: {
stdout: { on: (event: string, cb: (chunk: Buffer) => void) => void };
stderr: { on: jest.Mock };
pid: number;
kill: jest.Mock;
unref: jest.Mock;
exitCode: number | null;
signalCode: NodeJS.Signals | null;
killed: boolean;
};
catch: (cb: (err: unknown) => void) => Promise<void>;
};
let sessionStdout: string;
let uptermOnPath = true;
function makeHostProcess(): typeof hostProcess {
return {
child: {
stdout: {
on: jest.fn((_event: string, cb: (chunk: Buffer) => void) => {
cb(Buffer.from('host output'));
}),
},
stderr: { on: jest.fn() },
pid: 4242,
kill: jest.fn(),
unref: jest.fn(),
exitCode: null,
signalCode: null,
killed: false,
},
catch: cb => {
cb(new Error('exited'));
return Promise.resolve();
},
};
}
function makeCtx(env: Record<string, string> = {}): BuildContext {
return {
logger: createMockLogger(),
env,
} as unknown as BuildContext;
}
beforeEach(() => {
jest.clearAllMocks();
jest.spyOn(process, 'kill').mockReturnValue(true);
hostProcess = makeHostProcess();
uptermOnPath = true;
sessionStdout = JSON.stringify({
sessionId: 'TOKENx',
host: 'ssh://relay.expo.dev:22',
clientCount: 0,
});
mockedSleep.mockResolvedValue(undefined);
mockedFs.mkdir.mockResolvedValue(undefined as never);
mockedFs.chmod.mockResolvedValue(undefined as never);
mockedFs.mkdtemp.mockImplementation((async (prefix: string) =>
String(prefix).includes('eas-upterm') ? '/tmp/eas-upterm-1' : '/tmp/eas-ssh-1') as never);
mockedFs.writeFile.mockResolvedValue(undefined as never);
mockedFs.rm.mockResolvedValue(undefined as never);
mockedFs.readdir.mockResolvedValue(['default.sock'] as never);
mockedDownloadFile.mockResolvedValue(undefined as never);
mockedSpawn.mockImplementation(((cmd: string, args: string[]) => {
if (cmd === 'upterm' && Array.isArray(args) && args[0] === 'version') {
if (!uptermOnPath) {
return Promise.reject(new Error('not found')) as never;
}
return Promise.resolve({ stdout: 'upterm version 0.24.0\n', stderr: '' }) as never;
}
if (Array.isArray(args) && args[0] === 'host') {
return hostProcess as never;
}
if (Array.isArray(args) && args[0] === 'session') {
return Promise.resolve({ stdout: sessionStdout, stderr: '' }) as never;
}
return Promise.resolve({ stdout: '', stderr: '' }) as never;
}) as never);
});
it('uses upterm on PATH, dials, and returns a handle with the parsed config', async () => {
const env = { PATH: '/job/bin', CUSTOM: '1' };
const host = await startUptermHostAsync(makeCtx(env), {
relayServerUrl: 'wss://relay.expo.dev',
});
expect(host.connectionConfig).toEqual({
type: 'upterm-v1',
host: 'relay.expo.dev',
secret: 'TOKENx',
});
expect(host.isAlive()).toBe(true);
expect(mockedDownloadFile).not.toHaveBeenCalled();
expect(mockedSpawn).toHaveBeenCalledWith('upterm', ['version'], {
stdio: 'pipe',
env,
});
expect(mockedSpawn).toHaveBeenCalledWith(
'upterm',
expect.arrayContaining(['host']),
expect.objectContaining({ env: expect.objectContaining({ CUSTOM: '1', PATH: '/job/bin' }) })
);
});
it('downloads from GCS into a fresh temp dir when upterm is not on PATH', async () => {
uptermOnPath = false;
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://relay.expo.dev' });
expect(host.connectionConfig.secret).toBe('TOKENx');
expect(mockedDownloadFile).toHaveBeenCalledWith(
expect.stringMatching(/storage\.googleapis\.com\/turtle-v2\/upterm\/upterm-/),
expect.stringContaining('/tmp/eas-upterm-1'),
expect.objectContaining({ retry: 3 })
);
expect(mockedSpawn).toHaveBeenCalledWith(
expect.stringContaining('/tmp/eas-upterm-1'),
expect.arrayContaining(['host']),
expect.anything()
);
});
it('throws when upterm is missing from PATH and the GCS download fails', async () => {
uptermOnPath = false;
mockedDownloadFile.mockRejectedValue(new Error('403 Forbidden') as never);
await expect(startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' })).rejects.toThrow(
/could not be downloaded/
);
});
it('still throws when download fails with a non-Error and temp cleanup also fails', async () => {
uptermOnPath = false;
mockedDownloadFile.mockRejectedValue('network down' as never);
mockedFs.rm.mockRejectedValue(new Error('EPERM') as never);
await expect(startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' })).rejects.toThrow(
/network down/
);
});
it('tears down and rethrows when the host never registers with the relay', async () => {
jest.useFakeTimers();
jest.setSystemTime(0);
try {
mockedFs.readdir.mockResolvedValue([] as never);
mockedSleep.mockImplementation(async () => {
jest.setSystemTime(Date.now() + 61_000);
});
await expect(startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' })).rejects.toThrow(
/did not register/
);
expect(process.kill).toHaveBeenCalledWith(-4242, 'SIGTERM');
expect(mockedFs.rm).toHaveBeenCalledWith('/tmp/eas-ssh-1', expect.anything());
} finally {
jest.useRealTimers();
}
});
it('stopAsync kills the process and removes the state dir', async () => {
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
await host.stopAsync();
expect(process.kill).toHaveBeenCalledWith(-4242, 'SIGTERM');
expect(mockedFs.rm).toHaveBeenCalledWith('/tmp/eas-ssh-1', expect.anything());
});
it('falls back to child.kill when process.kill on the process group fails', async () => {
jest.spyOn(process, 'kill').mockImplementation(() => {
throw new Error('ESRCH');
});
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
await host.stopAsync();
expect(hostProcess.child.kill).toHaveBeenCalled();
});
it('skips process.kill when the host process has no pid', async () => {
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
hostProcess.child.pid = undefined as unknown as number;
await host.stopAsync();
expect(process.kill).not.toHaveBeenCalled();
expect(hostProcess.child.kill).not.toHaveBeenCalled();
});
it('redialAsync re-establishes the connection', async () => {
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
const config = await host.redialAsync();
expect(config).toEqual({
type: 'upterm-v1',
host: 'relay.expo.dev',
secret: 'TOKENx',
});
expect(host.connectionConfig).toEqual(config);
});
it('redialAsync waits for the old process to exit before clearing its socket dir', async () => {
const order: string[] = [];
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
// Keep the give-up timer pending so only the process exit can unblock the redial.
mockedSleep.mockImplementation(() => new Promise<void>(() => {}));
let resolveExit: (() => void) | undefined;
hostProcess.catch = () =>
new Promise<void>(resolve => {
resolveExit = () => {
order.push('exited');
resolve();
};
});
mockedFs.rm.mockImplementation((() => {
order.push('rm');
return Promise.resolve(undefined);
}) as never);
jest.spyOn(process, 'kill').mockImplementation(() => {
order.push('kill');
// A real SIGTERM is asynchronous, so the exit lands after the kill returns.
setImmediate(() => resolveExit?.());
return true;
});
await host.redialAsync();
expect(order).toEqual(['kill', 'exited', 'rm']);
});
it('redialAsync gives up waiting when the old process does not exit', async () => {
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
hostProcess.catch = () => new Promise<void>(() => {});
await expect(host.redialAsync()).resolves.toEqual({
type: 'upterm-v1',
host: 'relay.expo.dev',
secret: 'TOKENx',
});
expect(mockedSleep).toHaveBeenCalledWith(5_000);
});
it('continues when the previous socket dir cannot be cleared', async () => {
mockedFs.rm.mockImplementation(((dir: string) =>
dir === '/tmp/eas-ssh-1/upterm'
? Promise.reject(new Error('busy'))
: Promise.resolve(undefined)) as never);
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
expect(host.connectionConfig.secret).toBe('TOKENx');
});
describe('getConnectedClientCountAsync', () => {
it('throws when no admin socket exists yet (after retries)', async () => {
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
mockedFs.readdir.mockResolvedValue([] as never);
await expect(host.getConnectedClientCountAsync()).rejects.toThrow(/client count/i);
});
it('returns the client count reported by the admin socket', async () => {
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
sessionStdout = JSON.stringify({
sessionId: 'TOKENx',
host: 'relay.expo.dev',
clientCount: 4,
});
expect(await host.getConnectedClientCountAsync()).toBe(4);
});
it('throws when the count query fails (after retries)', async () => {
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
mockedSpawn.mockImplementation((() => Promise.reject(new Error('socket gone'))) as never);
await expect(host.getConnectedClientCountAsync()).rejects.toThrow(/client count/i);
});
it('throws when the reported count is not a number (after retries)', async () => {
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
sessionStdout = JSON.stringify({
sessionId: 'TOKENx',
host: 'relay.expo.dev',
clientCount: 'nope',
});
await expect(host.getConnectedClientCountAsync()).rejects.toThrow(/client count/i);
});
it('throws when the socket dir cannot be read (after retries)', async () => {
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
mockedFs.readdir.mockRejectedValue(new Error('no dir') as never);
await expect(host.getConnectedClientCountAsync()).rejects.toThrow(/client count/i);
});
});
});
@@ -0,0 +1,106 @@
import { hashFiles } from '@expo/steps';
import fs from 'fs';
import os from 'os';
import path from 'path';
import * as tar from 'tar';
export const COCOAPODS_CACHE_KEY_PREFIX = 'ios-pods-';
const PODS_DIRECTORY_NAME = 'Pods';
const PODFILE_LOCK_NAME = 'Podfile.lock';
export function getCocoapodsCachePaths(workingDirectory: string): {
iosDirectory: string;
podsDirectory: string;
podfileLockPath: string;
} {
const iosDirectory = path.join(workingDirectory, 'ios');
return {
iosDirectory,
podsDirectory: path.join(iosDirectory, PODS_DIRECTORY_NAME),
podfileLockPath: path.join(iosDirectory, PODFILE_LOCK_NAME),
};
}
export async function resolveCocoapodsCacheKeyAsync(
workingDirectory: string,
cocoapodsVersion: string
): Promise<{ key: string; keyPrefix: string }> {
const normalizedVersion = cocoapodsVersion.trim();
if (!normalizedVersion) {
throw new Error('Failed to determine CocoaPods version');
}
const keyPrefix = `${COCOAPODS_CACHE_KEY_PREFIX}${normalizedVersion}-`;
const { podfileLockPath } = getCocoapodsCachePaths(workingDirectory);
try {
await fs.promises.access(podfileLockPath);
} catch {
return { key: keyPrefix, keyPrefix };
}
return {
key: `${keyPrefix}${hashFiles([podfileLockPath])}`,
keyPrefix,
};
}
export async function compressCocoapodsCacheAsync({
workingDirectory,
}: {
workingDirectory: string;
}): Promise<{ archivePath: string }> {
const { iosDirectory, podsDirectory } = getCocoapodsCachePaths(workingDirectory);
await fs.promises.access(podsDirectory);
const archiveDestinationDirectory = await fs.promises.mkdtemp(
path.join(os.tmpdir(), 'save-cocoapods-cache-')
);
const archivePath = path.join(archiveDestinationDirectory, 'cache.tar.gz');
await tar.create(
{
file: archivePath,
cwd: iosDirectory,
gzip: true,
},
[PODS_DIRECTORY_NAME]
);
return { archivePath };
}
export async function restoreCocoapodsCacheArchiveAsync({
archivePath,
workingDirectory,
}: {
archivePath: string;
workingDirectory: string;
}): Promise<void> {
const { iosDirectory, podsDirectory } = getCocoapodsCachePaths(workingDirectory);
await fs.promises.mkdir(iosDirectory, { recursive: true });
const temporaryRestoreDirectory = await fs.promises.mkdtemp(
path.join(iosDirectory, '.eas-pods-cache-')
);
try {
await tar.extract({
file: archivePath,
cwd: temporaryRestoreDirectory,
filter: entryPath =>
entryPath === PODS_DIRECTORY_NAME || entryPath.startsWith(`${PODS_DIRECTORY_NAME}/`),
});
const restoredPodsDirectory = path.join(temporaryRestoreDirectory, PODS_DIRECTORY_NAME);
const restoredPodsStat = await fs.promises.stat(restoredPodsDirectory);
if (!restoredPodsStat.isDirectory()) {
throw new Error('CocoaPods cache archive does not contain a Pods directory');
}
await fs.promises.rm(podsDirectory, { recursive: true, force: true });
await fs.promises.rename(restoredPodsDirectory, podsDirectory);
} finally {
await fs.promises.rm(temporaryRestoreDirectory, { recursive: true, force: true });
}
}
+5 -3
View File
@@ -12,6 +12,8 @@ import { Sentry } from '../sentry';
let cachedEasCliVersionsPromise: Promise<EasCliVersions> | undefined;
const NPX_ENV = { NPM_CONFIG_MIN_RELEASE_AGE: '0' };
/**
* Resolves the `eas-cli` versions to install for staging/production. Prefers the
* versions committed to `cli-versions.json` (fetched from expo/eas-cli); on any
@@ -74,21 +76,21 @@ export async function resolveEasCommandPrefixAndEnvAsync(): Promise<{
return {
cmd: 'npx',
args: [...npxArgsPrefix, `eas-cli@${versions.STAGING}`],
extraEnv: {},
extraEnv: NPX_ENV,
};
} else if (process.env.ENVIRONMENT === 'staging') {
const versions = await resolveEasCliVersionsAsync();
return {
cmd: 'npx',
args: [...npxArgsPrefix, `eas-cli@${versions.STAGING}`],
extraEnv: { EXPO_STAGING: '1' },
extraEnv: { ...NPX_ENV, EXPO_STAGING: '1' },
};
} else {
const versions = await resolveEasCliVersionsAsync();
return {
cmd: 'npx',
args: [...npxArgsPrefix, `eas-cli@${versions.PRODUCTION}`],
extraEnv: {},
extraEnv: NPX_ENV,
};
}
}
@@ -0,0 +1,16 @@
export function formatSecondsForLog(totalSeconds: number): string {
const hours = Math.floor(totalSeconds / 3600);
const minutes = Math.floor((totalSeconds % 3600) / 60);
const seconds = totalSeconds % 60;
const parts: string[] = [];
if (hours > 0) {
parts.push(hours === 1 ? '1 hour' : `${hours} hours`);
}
if (minutes > 0) {
parts.push(minutes === 1 ? '1 minute' : `${minutes} minutes`);
}
if (seconds > 0 || parts.length === 0) {
parts.push(seconds === 1 ? '1 second' : `${seconds} seconds`);
}
return parts.join(' ');
}
@@ -1,5 +1,26 @@
import { ChildProcess } from 'node:child_process';
import spawn from '@expo/turtle-spawn';
export function isChildProcessAlive(child: ChildProcess): boolean {
return child.exitCode === null && child.signalCode === null && !child.killed;
}
/**
* Kill a detached spawn's process group. Negated pid targets the group so bash/sleep
* children cannot survive after the parent is gone (e.g. across upterm redial).
*/
export function killProcessGroup(child: ChildProcess): void {
if (child.pid == null) {
return;
}
try {
process.kill(-child.pid, 'SIGTERM');
} catch {
child.kill();
}
}
async function getChildrenPidsAsync(parentPids: number[]): Promise<number[]> {
try {
const result = await spawn('pgrep', ['-P', parentPids.join(',')], {
@@ -0,0 +1,250 @@
import { Job, SshSettings, SystemError } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import { graphql } from 'gql.tada';
import { formatSecondsForLog } from './formatDuration';
import { sleepAsync } from './retry';
import { SshConnectionConfig, startUptermHostAsync } from './upterm';
import { BuildContext } from '../context';
import { Sentry } from '../sentry';
const MAX_SSH_REDIALS = 10;
const REDIAL_BACKOFF_MS = 6_000;
const CLIENT_COUNT_POLL_INTERVAL_MS = 5_000;
const MAX_SSH_IDLE_TIMEOUT_SECONDS = 3600;
const DEFAULT_SSH_IDLE_TIMEOUT_SECONDS = 0;
const CREATE_OR_UPDATE_TURTLE_SSH_SESSION_MUTATION = graphql(`
mutation CreateOrUpdateTurtleSshSession(
$target: TurtleSshTargetInput!
$connectionConfig: TurtleSshConnectionConfigInput!
$sessionSettings: TurtleSshSessionSettingsInput!
) {
turtleSshSession {
createOrUpdateTurtleSshSession(
target: $target
connectionConfig: $connectionConfig
sessionSettings: $sessionSettings
) {
id
sessionSettings {
idleTimeoutSeconds
}
}
}
}
`);
export type TurtleSshTarget = { type: 'BUILD' | 'JOB_RUN'; id: string };
export function isSshEnabled(job: Pick<Job, 'ssh'>): boolean {
return job.ssh != null;
}
export function getSshIdleTimeoutSeconds(job: Pick<Job, 'ssh'>): number {
const idleTimeoutSeconds = job.ssh?.idleTimeoutSeconds ?? DEFAULT_SSH_IDLE_TIMEOUT_SECONDS;
if (
!Number.isInteger(idleTimeoutSeconds) ||
idleTimeoutSeconds < 0 ||
idleTimeoutSeconds > MAX_SSH_IDLE_TIMEOUT_SECONDS
) {
throw new SystemError(
`SSH idle timeout must be an integer between 0 and ${MAX_SSH_IDLE_TIMEOUT_SECONDS} seconds, got ${idleTimeoutSeconds}.`,
{ trackingCode: 'SSH_IDLE_TIMEOUT_INVALID' }
);
}
return idleTimeoutSeconds;
}
export function getSshRelayServerUrl(job: Pick<Job, 'ssh'>): string {
const relayServerUrl = job.ssh?.relayServerUrl;
if (!relayServerUrl) {
throw new SystemError('SSH is enabled but no relay server URL was configured on the job.', {
trackingCode: 'SSH_RELAY_SERVER_URL_MISSING',
});
}
return relayServerUrl;
}
export type SshSessionHandle = {
getConnectedClientCountAsync: () => Promise<number>;
ensureConnectedAsync: () => Promise<void>;
stopAsync: () => Promise<void>;
};
export type StartedSshSession = {
handle: SshSessionHandle;
idleTimeoutSeconds: number;
};
async function createOrUpdateSessionAsync(
ctx: BuildContext,
{
target,
connectionConfig,
idleTimeoutSeconds,
}: {
target: TurtleSshTarget;
connectionConfig: SshConnectionConfig & { reconnecting?: boolean };
idleTimeoutSeconds: number;
}
): Promise<{ idleTimeoutSeconds: number }> {
const result = await ctx.graphqlClient
.mutation(CREATE_OR_UPDATE_TURTLE_SSH_SESSION_MUTATION, {
target,
connectionConfig: {
...connectionConfig,
type: 'UPTERM_V1',
},
sessionSettings: { idleTimeoutSeconds },
})
.toPromise();
if (result.error || !result.data) {
throw new SystemError(
`Failed to create or update the SSH session: ${result.error?.message ?? 'no data returned'}`,
{ cause: result.error }
);
}
const session = result.data.turtleSshSession.createOrUpdateTurtleSshSession;
return { idleTimeoutSeconds: session.sessionSettings.idleTimeoutSeconds };
}
export async function startSshSessionAsync(
ctx: BuildContext,
{
target,
relayServerUrl,
idleTimeoutSeconds: requestedIdleTimeoutSeconds,
}: {
target: TurtleSshTarget;
} & SshSettings
): Promise<StartedSshSession> {
const logger = ctx.logger;
const host = await startUptermHostAsync(ctx, { relayServerUrl });
let idleTimeoutSeconds: number;
try {
({ idleTimeoutSeconds } = await createOrUpdateSessionAsync(ctx, {
target,
connectionConfig: { ...host.connectionConfig, reconnecting: false },
idleTimeoutSeconds: requestedIdleTimeoutSeconds,
}));
} catch (err) {
await host.stopAsync().catch(() => {});
throw err;
}
const ensureConnectedAsync = async (): Promise<void> => {
if (host.isAlive()) {
return;
}
for (let attempt = 1; attempt <= MAX_SSH_REDIALS; attempt++) {
try {
if (!host.isAlive()) {
logger.warn('The SSH relay connection dropped. Reconnecting...');
await createOrUpdateSessionAsync(ctx, {
target,
connectionConfig: { ...host.connectionConfig, reconnecting: true },
idleTimeoutSeconds: requestedIdleTimeoutSeconds,
}).catch(() => {});
await host.redialAsync();
}
await createOrUpdateSessionAsync(ctx, {
target,
connectionConfig: { ...host.connectionConfig, reconnecting: false },
idleTimeoutSeconds: requestedIdleTimeoutSeconds,
});
logger.info('The SSH relay connection was restored.');
return;
} catch (err) {
logger.warn({ err }, `SSH reconnect attempt ${attempt} of ${MAX_SSH_REDIALS} failed.`);
if (attempt < MAX_SSH_REDIALS) {
await sleepAsync(REDIAL_BACKOFF_MS);
}
}
}
throw new SystemError(
`The SSH relay connection dropped and could not be restored after ${MAX_SSH_REDIALS} attempts.`
);
};
return {
handle: {
getConnectedClientCountAsync: () => host.getConnectedClientCountAsync(),
ensureConnectedAsync,
stopAsync: () => host.stopAsync(),
},
idleTimeoutSeconds,
};
}
export async function superviseSshSessionAsync({
handle,
idleTimeoutSeconds,
hasJobFinished,
logger,
}: {
handle: SshSessionHandle;
idleTimeoutSeconds: number;
hasJobFinished: () => boolean;
logger: bunyan;
}): Promise<void> {
const idleTimeoutMs = idleTimeoutSeconds * 1_000;
let idleSince: number | null = null;
let previousClientCount = 0;
for (;;) {
try {
await handle.ensureConnectedAsync();
} catch (err) {
logger.warn({ err }, 'Could not restore the SSH relay connection. Closing the session.');
Sentry.capture(
'Could not restore the SSH relay connection',
err instanceof Error ? err : undefined,
{
tags: { trackingCode: 'SSH_RELAY_RECONNECT_FAILED' },
}
);
return;
}
let connectedClientCount: number;
try {
connectedClientCount = await handle.getConnectedClientCountAsync();
} catch (err) {
logger.warn({ err }, 'Could not read the SSH client count. Closing the session.');
Sentry.capture(
'Could not read the SSH client count',
err instanceof Error ? err : undefined,
{
tags: { trackingCode: 'SSH_CLIENT_COUNT_UNREADABLE' },
}
);
return;
}
if (connectedClientCount !== previousClientCount) {
logger.info(`SSH clients connected: ${connectedClientCount}`);
previousClientCount = connectedClientCount;
}
const jobHasFinished = hasJobFinished();
if (connectedClientCount > 0 || !jobHasFinished) {
idleSince = null;
} else if (idleSince === null) {
idleSince = Date.now();
}
if (jobHasFinished && connectedClientCount === 0) {
if (idleTimeoutSeconds === 0) {
logger.info('The job finished and no SSH client is connected. Closing the session.');
return;
}
if (idleSince !== null && Date.now() - idleSince >= idleTimeoutMs) {
logger.info(
`No SSH client connected for ${formatSecondsForLog(idleTimeoutSeconds)} after the job finished. Closing the session.`
);
return;
}
}
await sleepAsync(CLIENT_COUNT_POLL_INTERVAL_MS);
}
}
+318
View File
@@ -0,0 +1,318 @@
import { Env, SystemError } from '@expo/eas-build-job';
import downloadFile from '@expo/downloader';
import spawn, { SpawnPromise, SpawnResult } from '@expo/turtle-spawn';
import fs from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { z } from 'zod';
import { isChildProcessAlive, killProcessGroup } from './processes';
import { sleepAsync } from './retry';
import { BuildContext } from '../context';
const CONTROL_CHARACTERS = /[\u0000-\u0008\u000B-\u001F\u007F-\u009F]/g;
export function resolveUptermGcsObjectName(
platform: NodeJS.Platform = process.platform,
arch: string = process.arch
): string {
if (platform === 'darwin' && arch === 'arm64') {
return 'upterm-darwin-arm64';
}
if (platform === 'linux' && arch === 'x64') {
return 'upterm-linux-amd64';
}
throw new SystemError(
`SSH upterm is only available on darwin/arm64 and linux/x64 (got ${platform}/${arch}).`
);
}
const UPTERM_GCS_BASE_URL = 'https://storage.googleapis.com/turtle-v2/upterm';
const UPTERM_DOWNLOAD_TIMEOUT_MS = 60_000;
const UPTERM_KEEPALIVE_SLEEP_SECONDS = 6 * 60 * 60;
const CONNECTION_POLL_INTERVAL_MS = 500;
const CONNECTION_STARTUP_TIMEOUT_MS = 60_000;
const PROCESS_EXIT_TIMEOUT_MS = 5_000;
const CLIENT_COUNT_READ_ATTEMPTS = 4;
const CLIENT_COUNT_READ_RETRY_MS = 500;
const DEFAULT_SSH_PORT = '22';
export type SshConnectionConfig = {
type: 'upterm-v1';
host: string;
secret: string;
};
export type UptermHost = {
connectionConfig: SshConnectionConfig;
getConnectedClientCountAsync: () => Promise<number>;
isAlive: () => boolean;
redialAsync: () => Promise<SshConnectionConfig>;
stopAsync: () => Promise<void>;
};
const UptermSessionJsonZ = z.object({
sessionId: z.string().min(1),
host: z.string().min(1),
clientCount: z.number().optional(),
});
type UptermSessionJson = z.infer<typeof UptermSessionJsonZ>;
export function connectionConfigFromUptermSession(
parsed: Pick<UptermSessionJson, 'sessionId' | 'host'>
): SshConnectionConfig | null {
let host = parsed.host;
if (host.includes('://')) {
let url: URL;
try {
url = new URL(host);
} catch {
return null;
}
host = url.port && url.port !== DEFAULT_SSH_PORT ? `${url.hostname}:${url.port}` : url.hostname;
}
if (!host) {
return null;
}
return { type: 'upterm-v1', host, secret: parsed.sessionId };
}
/**
* Strip upterm session secrets out of text before it reaches a log or an error message. upterm
* prints the session id as the userinfo of the connect line it advertises
* (`upterm proxy ws(s)://<sessionId>@host`) and repeats it bare as the ssh destination on the
* same line. Control characters are dropped first so they cannot split a token mid-match. The
* loop lifts the id out of the proxy URL and removes every occurrence of it; the final replace
* blanks userinfo in any other URL as a catch-all.
*/
export function redactConnectionSecrets(text: string): string {
let redacted = text.replace(CONTROL_CHARACTERS, '');
for (const [, token] of redacted.matchAll(/upterm proxy wss?:\/\/([^@\s]+)@/g)) {
redacted = redacted.split(token).join('<redacted>');
}
return redacted.replace(/([a-z][a-z0-9+.-]*:\/\/)[^@\s/]+@/gi, '$1<redacted>@');
}
export function redactSpawnErrorForLog(err: unknown): unknown {
if (!err || typeof err !== 'object') {
return err;
}
const spawnErr = err as { message?: unknown; stdout?: unknown; stderr?: unknown };
return {
...spawnErr,
...(typeof spawnErr.message === 'string'
? { message: redactConnectionSecrets(spawnErr.message) }
: {}),
...(typeof spawnErr.stdout === 'string'
? { stdout: redactConnectionSecrets(spawnErr.stdout) }
: {}),
...(typeof spawnErr.stderr === 'string'
? { stderr: redactConnectionSecrets(spawnErr.stderr) }
: {}),
};
}
export async function resolveUptermPathAsync(env: Env): Promise<string> {
try {
await spawn('upterm', ['version'], { stdio: 'pipe', env });
return 'upterm';
} catch {}
const objectName = resolveUptermGcsObjectName();
const downloadDir = await fs.mkdtemp(path.join(os.tmpdir(), 'eas-upterm-'));
const uptermPath = path.join(downloadDir, objectName);
const url = `${UPTERM_GCS_BASE_URL}/${objectName}`;
try {
await downloadFile(url, uptermPath, { retry: 3, timeout: UPTERM_DOWNLOAD_TIMEOUT_MS });
await fs.chmod(uptermPath, 0o755);
} catch (err) {
await fs.rm(downloadDir, { recursive: true, force: true }).catch(() => {});
throw new SystemError(
`The upterm SSH client was not on PATH and could not be downloaded from ${url}. ${
err instanceof Error ? err.message : String(err)
}`
);
}
return uptermPath;
}
async function findAdminSocketPathAsync(uptermSocketDir: string): Promise<string | null> {
// Use this dial's own admin socket, not upterm's default, which can still point at a previous
// dial's session after a redial and break client-count reads.
const entries = await fs.readdir(uptermSocketDir).catch(() => [] as string[]);
const socketName = entries.find(entry => entry.endsWith('.sock'));
return socketName ? path.join(uptermSocketDir, socketName) : null;
}
async function readCurrentSessionJsonAsync(
uptermPath: string,
adminSocketPath: string
): Promise<UptermSessionJson | null> {
try {
const result = await spawn(
uptermPath,
['session', 'current', '--admin-socket', adminSocketPath, '--output', 'json'],
{ stdio: 'pipe' }
);
const parsed = UptermSessionJsonZ.safeParse(JSON.parse(result.stdout));
return parsed.success ? parsed.data : null;
} catch {
return null;
}
}
async function waitForConnectionConfigAsync(
uptermPath: string,
uptermSocketDir: string,
getHostOutput: () => string
): Promise<SshConnectionConfig> {
const deadline = Date.now() + CONNECTION_STARTUP_TIMEOUT_MS;
for (;;) {
const adminSocketPath = await findAdminSocketPathAsync(uptermSocketDir);
if (adminSocketPath) {
const session = await readCurrentSessionJsonAsync(uptermPath, adminSocketPath);
if (session) {
const connectionConfig = connectionConfigFromUptermSession(session);
if (connectionConfig) {
return connectionConfig;
}
}
}
if (Date.now() >= deadline) {
throw new SystemError(
`The upterm client did not register with the relay within ${
CONNECTION_STARTUP_TIMEOUT_MS / 1_000
}s. Output:\n${redactConnectionSecrets(getHostOutput())}`
);
}
await sleepAsync(CONNECTION_POLL_INTERVAL_MS);
}
}
export async function startUptermHostAsync(
ctx: BuildContext,
{ relayServerUrl }: { relayServerUrl: string }
): Promise<UptermHost> {
const uptermPath = await resolveUptermPathAsync(ctx.env);
const stateDir = await fs.mkdtemp(path.join(os.tmpdir(), 'eas-ssh-'));
const hostKeyPath = path.join(stateDir, 'id_host');
const forceCommandPath = path.join(stateDir, 'join.sh');
const uptermSocketDir = path.join(stateDir, 'upterm');
await spawn('ssh-keygen', ['-t', 'ed25519', '-N', '', '-f', hostKeyPath, '-q'], {
logger: ctx.logger,
});
await fs.writeFile(forceCommandPath, '#!/usr/bin/env bash\nexec bash -l\n', { mode: 0o755 });
let currentProcess: SpawnPromise<SpawnResult> | null = null;
const stopCurrentProcessAsync = async (): Promise<void> => {
const previousProcess = currentProcess;
currentProcess = null;
if (!previousProcess) {
return;
}
killProcessGroup(previousProcess.child);
await Promise.race([
previousProcess.catch(() => {}),
sleepAsync(PROCESS_EXIT_TIMEOUT_MS).then(() => {
ctx.logger.debug('The previous upterm host process did not exit in time.');
}),
]);
};
const getConnectedClientCountAsync = async (): Promise<number> => {
for (let attempt = 1; attempt <= CLIENT_COUNT_READ_ATTEMPTS; attempt++) {
const adminSocketPath = await findAdminSocketPathAsync(uptermSocketDir);
const session = adminSocketPath
? await readCurrentSessionJsonAsync(uptermPath, adminSocketPath)
: null;
if (session && typeof session.clientCount === 'number') {
return session.clientCount;
}
if (attempt < CLIENT_COUNT_READ_ATTEMPTS) {
await sleepAsync(CLIENT_COUNT_READ_RETRY_MS);
}
}
throw new SystemError('Could not read the SSH client count from the upterm admin socket.', {
trackingCode: 'SSH_CLIENT_COUNT_UNREADABLE',
});
};
const dialAsync = async (): Promise<SshConnectionConfig> => {
await stopCurrentProcessAsync();
await fs.rm(uptermSocketDir, { recursive: true, force: true }).catch(err => {
ctx.logger.debug({ err }, 'Failed to clear the previous SSH socket directory.');
});
ctx.logger.debug('Connecting to the SSH relay.');
// --force-command is what each connecting SSH client runs (a login shell). The `sleep` after
// `--` is the host-side process that keeps `upterm host` up while nobody is connected.
const uptermProcess = spawn(
uptermPath,
[
'host',
'--server',
relayServerUrl,
'--accept',
'--skip-host-key-check',
'-i',
hostKeyPath,
'--force-command',
forceCommandPath,
'--',
'bash',
'-lc',
`sleep ${UPTERM_KEEPALIVE_SLEEP_SECONDS}`,
],
{
// upterm puts its admin socket under XDG_RUNTIME_DIR; point it at our state dir so we can
// find it for `session current` and clean it up on stop/redial.
env: { ...ctx.env, XDG_RUNTIME_DIR: stateDir },
stdio: ['ignore', 'pipe', 'pipe'],
detached: true,
}
);
uptermProcess.catch(err =>
ctx.logger.debug({ err: redactSpawnErrorForLog(err) }, 'The upterm host process exited.')
);
uptermProcess.child.unref();
currentProcess = uptermProcess;
let output = '';
const appendChunk = (chunk: Buffer | string): void => {
output += chunk.toString();
};
uptermProcess.child.stdout?.on('data', appendChunk);
uptermProcess.child.stderr?.on('data', appendChunk);
return await waitForConnectionConfigAsync(uptermPath, uptermSocketDir, () => output);
};
const stopAsync = async (): Promise<void> => {
await stopCurrentProcessAsync();
await fs.rm(stateDir, { recursive: true, force: true });
};
let connectionConfig: SshConnectionConfig;
try {
connectionConfig = await dialAsync();
} catch (err) {
await stopAsync();
throw err;
}
return {
get connectionConfig() {
return connectionConfig;
},
getConnectedClientCountAsync,
isAlive: () => currentProcess != null && isChildProcessAlive(currentProcess.child),
redialAsync: async () => {
connectionConfig = await dialAsync();
return connectionConfig;
},
stopAsync,
};
}
@@ -1,6 +1,6 @@
{
"name": "create-eas-build-function",
"version": "22.4.0",
"version": "23.1.0",
"description": "Create functions for use in EAS Build custom builds.",
"keywords": [
"expo",
@@ -35,7 +35,7 @@
"clean": "rimraf ./build/ \"*.tsbuildinfo\""
},
"dependencies": {
"@expo/steps": "22.4.0"
"@expo/steps": "23.1.0"
},
"devDependencies": {
"@expo/package-manager": "1.9.10",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/downloader",
"version": "22.0.0",
"version": "23.0.0",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "BUSL-1.1",
"author": "Expo <[email protected]>",
@@ -1,6 +1,6 @@
{
"name": "eas-build-cache-provider",
"version": "22.0.0",
"version": "23.0.0",
"description": "A build cache provider plugin for the Expo CLI",
"homepage": "https://github.com/expo/eas-cli",
"bugs": "https://github.com/expo/eas-cli/issues",
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/eas-build-job",
"version": "22.0.0",
"version": "23.1.0",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "MIT",
"author": "Expo <[email protected]>",
@@ -25,9 +25,9 @@
"clean": "rimraf node_modules dist coverage \"*.tsbuildinfo\""
},
"dependencies": {
"@expo/logger": "22.0.0",
"@expo/logger": "23.1.0",
"@expo/results": "1.0.0",
"@expo/turtle-spawn": "22.0.0",
"@expo/turtle-spawn": "23.1.0",
"joi": "^17.13.1",
"semver": "^7.6.2",
"zod": "^4.3.5"
+189 -156
View File
File diff suppressed because it is too large. Load diff
+1633 -25
View File
File diff suppressed because it is too large. Load diff
+11 -9
View File
@@ -1,6 +1,6 @@
{
"name": "eas-cli",
"version": "22.5.0",
"version": "23.2.0",
"description": "EAS command line tool",
"keywords": [
"cli",
@@ -48,11 +48,11 @@
"@expo/code-signing-certificates": "0.0.5",
"@expo/config": "55.0.10",
"@expo/config-plugins": "55.0.7",
"@expo/eas-build-job": "22.0.0",
"@expo/eas-json": "22.0.0",
"@expo/eas-build-job": "23.1.0",
"@expo/eas-json": "23.1.0",
"@expo/env": "^1.0.0",
"@expo/json-file": "8.3.3",
"@expo/logger": "22.0.0",
"@expo/logger": "23.1.0",
"@expo/multipart-body-parser": "2.0.0",
"@expo/osascript": "2.1.4",
"@expo/package-manager": "1.9.10",
@@ -64,7 +64,7 @@
"@expo/results": "1.0.0",
"@expo/rudder-sdk-node": "1.1.1",
"@expo/spawn-async": "1.7.2",
"@expo/steps": "22.4.0",
"@expo/steps": "23.1.0",
"@expo/timeago.js": "1.0.0",
"@oclif/core": "^4.8.3",
"@oclif/plugin-autocomplete": "^3.2.40",
@@ -72,11 +72,12 @@
"@sentry/node": "7.77.0",
"@urql/core": "4.0.11",
"@urql/exchange-retry": "1.2.0",
"agent-cli-detector": "0.1.6",
"agent-cli-detector": "0.1.7",
"ajv": "8.11.0",
"ajv-formats": "2.1.1",
"better-opn": "3.0.2",
"bplist-parser": "^0.3.0",
"centrifuge": "5.7.1",
"chalk": "4.1.2",
"cli-progress": "3.12.0",
"dateformat": "4.6.3",
@@ -104,7 +105,7 @@
"log-symbols": "4.1.0",
"mime": "3.0.0",
"minimatch": "5.1.2",
"minizlib": "3.0.1",
"minizlib": "3.1.0",
"nanoid": "3.3.8",
"node-fetch": "2.6.7",
"node-forge": "1.4.0",
@@ -130,8 +131,9 @@
"tslib": "2.6.2",
"turndown": "7.1.2",
"untildify": "4.0.0",
"uuid": "9.0.1",
"uuid": "11.1.1",
"wrap-ansi": "7.0.0",
"ws": "8.21.1",
"yaml": "2.6.0",
"zod": "^4.1.3"
},
@@ -159,8 +161,8 @@
"@types/tar": "6.1.10",
"@types/tar-stream": "3.1.3",
"@types/tough-cookie": "4.0.2",
"@types/uuid": "9.0.7",
"@types/wrap-ansi": "3.0.0",
"@types/ws": "8.5.10",
"axios": "1.18.1",
"eslint-plugin-graphql": "4.0.0",
"jest": "29.7.0",
@@ -8,10 +8,7 @@ import {
mockProjectId,
mockTestCommand,
} from './utils';
import {
fetchRawLogsForBuildJobAsync,
fetchRawLogsForCustomJobAsync,
} from '../../commandUtils/workflow/fetchLogs';
import { fetchRawLogsForJobAsync } from '../../commandUtils/workflow/logs/fetchLogs';
import WorkflowLogView from '../../commands/workflow/logs';
import { AppPlatform, BuildPriority, BuildStatus } from '../../graphql/generated';
import { AppQuery } from '../../graphql/queries/AppQuery';
@@ -38,7 +35,7 @@ jest.mock('fs');
jest.mock('../../log');
jest.mock('../../prompts');
jest.mock('../../utils/json');
jest.mock('../../commandUtils/workflow/fetchLogs');
jest.mock('../../commandUtils/workflow/logs/fetchLogs');
describe(WorkflowLogView, () => {
beforeEach(() => {
@@ -49,6 +46,7 @@ describe(WorkflowLogView, () => {
});
afterEach(() => {
jest.clearAllMocks();
jest.mocked(fetchRawLogsForJobAsync).mockReset();
});
test('view logs, passing in no parameters, no runs found', async () => {
const ctx = mockCommandContext(WorkflowLogView, {
@@ -105,7 +103,7 @@ describe(WorkflowLogView, () => {
return { selectedStep: 'step1' };
}
});
jest.mocked(fetchRawLogsForCustomJobAsync).mockResolvedValue(null);
jest.mocked(fetchRawLogsForJobAsync).mockResolvedValue(null);
await cmd.run();
expect(AppQuery.byIdWorkflowRunsFilteredByStatusAsync).toHaveBeenCalledWith(
ctx.loggedIn.graphqlClient,
@@ -139,7 +137,7 @@ describe(WorkflowLogView, () => {
}
});
jest
.mocked(fetchRawLogsForCustomJobAsync)
.mocked(fetchRawLogsForJobAsync)
.mockResolvedValue(
'{"result":"success","marker":"end-step","buildStepDisplayName":"Install dependencies","buildStepId":"step-id-1","time":"2022-01-01T00:00:00.000Z","msg":"test"}'
);
@@ -209,7 +207,7 @@ describe(WorkflowLogView, () => {
}
});
jest
.mocked(fetchRawLogsForBuildJobAsync)
.mocked(fetchRawLogsForJobAsync)
.mockResolvedValue(
'{"result":"success","marker":"end-step","buildStepDisplayName":"step1","buildStepId":"step-id-1","time":"2022-01-01T00:00:00.000Z","msg":"test"}'
);
@@ -255,7 +253,7 @@ describe(WorkflowLogView, () => {
}
});
jest
.mocked(fetchRawLogsForCustomJobAsync)
.mocked(fetchRawLogsForJobAsync)
.mockResolvedValue(
[
'{"result":"success","marker":"end-step","buildStepDisplayName":"Install","buildStepId":"step-id-1","time":"2022-01-01T00:00:00.000Z","msg":"first"}',
+10
View File
@@ -112,3 +112,13 @@ export function getEASUpdateURL(projectId: string, manifestHostOverride: string
export function getExpoApiWorkflowSchemaURL(): string {
return getExpoApiBaseUrl() + '/v2/workflows/schema';
}
export function getEASLogsWebsocketUrl(): string {
if (process.env.EXPO_STAGING) {
return `wss://staging-logs.expo.dev/connection/websocket`;
} else if (process.env.EXPO_LOCAL) {
return `ws://localhost:4997/connection/websocket`;
} else {
return `wss://logs.expo.dev/connection/websocket`;
}
}
@@ -1,68 +1,109 @@
import { canAccessRepositoryUsingSshAsync, runGitCloneAsync } from '../../../onboarding/git';
import glob from 'fast-glob';
import fs from 'fs-extra';
import { pipeline } from 'stream/promises';
import { extract } from 'tar';
import fetch from '../../../fetch';
import { installDependenciesAsync } from '../../../onboarding/installDependencies';
import { runCommandAsync } from '../../../onboarding/runCommand';
import { expoCommandAsync } from '../../../utils/expoCli';
import {
cloneTemplateAsync,
downloadTemplateAsync,
fetchTemplatePackumentAsync,
initializeGitRepositoryAsync,
installProjectDependenciesAsync,
} from '../commands';
jest.mock('../../../utils/expoCli');
jest.mock('../../../onboarding/git');
jest.mock('../../../fetch');
jest.mock('../../../onboarding/runCommand');
jest.mock('../../../onboarding/installDependencies');
jest.mock('../../../ora');
jest.mock('fs-extra');
jest.mock('tar');
jest.mock('stream/promises');
jest.mock('fast-glob');
describe('commands', () => {
beforeEach(() => {
jest.clearAllMocks();
});
describe('cloneTemplateAsync', () => {
it('should clone the template project with ssh', async () => {
describe('fetchTemplatePackumentAsync', () => {
it('should fetch the packument from the npm registry', async () => {
const packument = { 'dist-tags': { latest: '57.0.19' } };
jest.mocked(fetch).mockResolvedValueOnce({ json: async () => packument } as any);
expect(await fetchTemplatePackumentAsync()).toEqual(packument);
expect(fetch).toHaveBeenCalledWith(
'https://registry.npmjs.org/expo-template-default',
expect.objectContaining({
headers: { accept: 'application/vnd.npm.install-v1+json' },
})
);
});
});
describe('downloadTemplateAsync', () => {
const template = {
npmTag: 'sdk-56',
version: '56.0.34',
tarballUrl: 'https://registry.npmjs.org/expo-template-default/-/56.0.34.tgz',
};
it('should download and extract the template', async () => {
const targetProjectDir = '/test/target-project';
const finalTargetProjectDir = '/test/final-target-project';
const tarballBody = 'tarball-stream';
jest.mocked(canAccessRepositoryUsingSshAsync).mockResolvedValue(true);
jest.mocked(runGitCloneAsync).mockResolvedValue({
targetProjectDir: finalTargetProjectDir,
});
jest.mocked(fs.pathExists).mockResolvedValueOnce(false as never);
jest.mocked(fetch).mockResolvedValueOnce({ body: tarballBody } as any);
jest.mocked(glob).mockResolvedValueOnce(['gitignore', '_vscode'] as any);
const result = await cloneTemplateAsync(targetProjectDir);
const result = await downloadTemplateAsync(targetProjectDir, template);
expect(runGitCloneAsync).toHaveBeenCalledWith({
githubUsername: 'expo',
githubRepositoryName: 'expo-template-default',
targetProjectDir,
cloneMethod: 'ssh',
showOutput: false,
});
expect(result).toBe(finalTargetProjectDir);
expect(fetch).toHaveBeenCalledWith(
'https://registry.npmjs.org/expo-template-default/-/56.0.34.tgz'
);
expect(extract).toHaveBeenCalledWith({ cwd: targetProjectDir, strip: 1 });
expect(pipeline).toHaveBeenCalledWith(tarballBody, extract({} as any));
expect(fs.move).toHaveBeenCalledWith(
'/test/target-project/gitignore',
'/test/target-project/.gitignore',
{ overwrite: true }
);
expect(fs.move).toHaveBeenCalledWith(
'/test/target-project/_vscode',
'/test/target-project/.vscode',
{ overwrite: true }
);
expect(result).toBe(targetProjectDir);
});
it('should clone the template project with https', async () => {
const targetProjectDir = '/test/target-project';
const finalTargetProjectDir = '/test/final-target-project';
it('should throw when the target directory exists and is not empty', async () => {
jest.mocked(fs.pathExists).mockResolvedValueOnce(true as never);
jest.mocked(fs.readdir).mockResolvedValueOnce(['existing-file'] as never);
jest.mocked(canAccessRepositoryUsingSshAsync).mockResolvedValue(false);
jest.mocked(runGitCloneAsync).mockResolvedValue({
targetProjectDir: finalTargetProjectDir,
});
await expect(downloadTemplateAsync('/test/target-project', template)).rejects.toThrow(
'already exists and is not empty'
);
});
const result = await cloneTemplateAsync(targetProjectDir);
it('should not remove a pre-existing directory on failure', async () => {
jest.mocked(fs.pathExists).mockResolvedValueOnce(true as never);
jest.mocked(fs.readdir).mockResolvedValueOnce(['existing-file'] as never);
expect(runGitCloneAsync).toHaveBeenCalledWith({
githubUsername: 'expo',
githubRepositoryName: 'expo-template-default',
targetProjectDir,
cloneMethod: 'https',
showOutput: false,
});
await expect(downloadTemplateAsync('/test/target-project', template)).rejects.toThrow();
expect(fs.remove).not.toHaveBeenCalled();
});
expect(result).toBe(finalTargetProjectDir);
it('should clean up a partially extracted directory on failure', async () => {
jest.mocked(fs.pathExists).mockResolvedValueOnce(false as never);
jest.mocked(fetch).mockRejectedValueOnce(new Error('network error'));
await expect(downloadTemplateAsync('/test/target-project', template)).rejects.toThrow(
'network error'
);
expect(fs.remove).toHaveBeenCalledWith('/test/target-project');
});
});
@@ -9,6 +9,7 @@ import {
copyProjectTemplatesAsync,
generateAppConfigAsync,
generateEasConfigAsync,
removeTemplateFilesAsync,
updatePackageJsonAsync,
updateReadmeAsync,
} from '../projectFiles';
@@ -239,17 +240,24 @@ describe('projectFiles', () => {
});
describe('updatePackageJsonAsync', () => {
it('should update the package.json', async () => {
it('should replace template metadata and add workflow scripts', async () => {
jest.mocked(fs.readJson).mockResolvedValue({
name: 'test-app',
version: '1.0.0',
name: 'expo-template-default',
version: '57.0.19',
license: '0BSD',
description: 'A starter template',
repository: 'https://github.com/expo/expo-template-default',
tags: ['expo'],
main: 'expo-router/entry',
});
const projectDir = '/test/project-dir';
await updatePackageJsonAsync(projectDir);
await updatePackageJsonAsync(projectDir, 'My App');
const expectedPackageJson = {
name: 'test-app',
name: 'myapp',
version: '1.0.0',
private: true,
main: 'expo-router/entry',
scripts: {
draft: 'npx eas-cli@latest workflow:run create-draft.yml',
'development-builds': 'npx eas-cli@latest workflow:run create-development-builds.yml',
@@ -261,6 +269,28 @@ describe('projectFiles', () => {
spaces: 2,
});
});
it('should keep a non-template license', async () => {
jest.mocked(fs.readJson).mockResolvedValue({
name: 'some-template',
license: 'MIT',
});
await updatePackageJsonAsync('/test/project-dir', 'my-app');
expect(fs.writeJson).toHaveBeenCalledWith(
'/test/project-dir/package.json',
expect.objectContaining({ license: 'MIT' }),
{ spaces: 2 }
);
});
});
describe('removeTemplateFilesAsync', () => {
it('should remove the template LICENSE file', async () => {
await removeTemplateFilesAsync('/test/project-dir');
expect(fs.remove).toHaveBeenCalledWith('/test/project-dir/LICENSE');
});
});
describe('copyProjectTemplatesAsync', () => {
@@ -0,0 +1,219 @@
import fetch from '../../../fetch';
import { promptAsync } from '../../../prompts';
import { fetchTemplatePackumentAsync } from '../commands';
import {
fetchSdkVersionsAsync,
promptForSdkVersionAsync,
resolveTemplateAsync,
} from '../sdkVersion';
jest.mock('../../../fetch');
jest.mock('../../../prompts');
jest.mock('../../../api', () => ({
getExpoApiBaseUrl: () => 'https://api.expo.dev',
}));
jest.mock('../commands');
const packument = {
'dist-tags': { latest: '57.0.19', 'sdk-57': '57.0.19', 'sdk-56': '56.0.34' },
versions: {
'57.0.19': {
dist: { tarball: 'https://registry.npmjs.org/expo-template-default/-/57.0.19.tgz' },
},
'56.0.34': {
dist: { tarball: 'https://registry.npmjs.org/expo-template-default/-/56.0.34.tgz' },
},
},
};
function mockVersionsEndpoint(json: object): void {
jest.mocked(fetch).mockResolvedValueOnce({ json: async () => json } as any);
}
beforeEach(() => {
jest.clearAllMocks();
jest.mocked(fetchTemplatePackumentAsync).mockResolvedValue(packument);
});
describe('fetchSdkVersionsAsync', () => {
it('returns released SDK versions sorted descending', async () => {
mockVersionsEndpoint({
sdkVersions: {
'55.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-55' },
'57.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-57' },
'56.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-56' },
'58.0.0': {}, // canary, no release notes
'54.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-54', isDeprecated: true },
},
expoGoSdkVersion: '56.0.0',
});
const versions = await fetchSdkVersionsAsync();
expect(versions).toEqual({
latest: 57,
expoGoCompatible: 56,
available: [57, 56, 55],
});
});
it('returns null when the endpoint is unreachable', async () => {
jest.mocked(fetch).mockRejectedValueOnce(new Error('network error'));
expect(await fetchSdkVersionsAsync()).toBeNull();
});
it('returns null when no released versions are available', async () => {
mockVersionsEndpoint({ sdkVersions: {} });
expect(await fetchSdkVersionsAsync()).toBeNull();
});
});
describe('promptForSdkVersionAsync', () => {
it('offers latest and Expo Go compatible choices', async () => {
jest.mocked(promptAsync).mockResolvedValueOnce({ answer: 56 });
const result = await promptForSdkVersionAsync({
latest: 57,
expoGoCompatible: 56,
available: [57, 56, 55],
});
expect(result).toBe(56);
expect(promptAsync).toHaveBeenCalledWith(
expect.objectContaining({
message: 'Select an Expo SDK version:',
choices: [
expect.objectContaining({ title: 'Latest (SDK 57)', value: 57 }),
expect.objectContaining({
title: 'For learning with Expo Go (SDK 56)',
value: 56,
}),
expect.objectContaining({ title: 'Other SDK version…', value: 'other' }),
],
})
);
});
it('hides the Expo Go choice when it matches the latest SDK or is unavailable', async () => {
jest.mocked(promptAsync).mockResolvedValue({ answer: 57 });
await promptForSdkVersionAsync({ latest: 57, expoGoCompatible: 57, available: [57, 56] });
await promptForSdkVersionAsync({ latest: 57, expoGoCompatible: 54, available: [57, 56] });
for (const call of jest.mocked(promptAsync).mock.calls) {
expect(call[0]).toEqual(
expect.objectContaining({
choices: [
expect.objectContaining({ title: 'Latest (SDK 57)' }),
expect.objectContaining({ title: 'Other SDK version…' }),
],
})
);
}
});
it('hides the "Other" choice when no other versions are available', async () => {
jest.mocked(promptAsync).mockResolvedValueOnce({ answer: 57 });
await promptForSdkVersionAsync({ latest: 57, expoGoCompatible: null, available: [57] });
expect(promptAsync).toHaveBeenCalledWith(
expect.objectContaining({
choices: [expect.objectContaining({ title: 'Latest (SDK 57)' })],
})
);
});
it('offers the remaining versions when "other" is selected', async () => {
jest
.mocked(promptAsync)
.mockResolvedValueOnce({ answer: 'other' })
.mockResolvedValueOnce({ sdkVersion: 53 });
const result = await promptForSdkVersionAsync({
latest: 57,
expoGoCompatible: 56,
available: [57, 56, 55, 54, 53],
});
expect(result).toBe(53);
expect(promptAsync).toHaveBeenLastCalledWith(
expect.objectContaining({
message: 'Select an SDK version:',
choices: [
{ title: 'SDK 55', value: 55 },
{ title: 'SDK 54', value: 54 },
{ title: 'SDK 53', value: 53 },
],
})
);
});
});
describe('resolveTemplateAsync', () => {
it('uses the provided SDK version without prompting', async () => {
expect(await resolveTemplateAsync({ sdkVersion: '56' })).toEqual({
npmTag: 'sdk-56',
version: '56.0.34',
tarballUrl: 'https://registry.npmjs.org/expo-template-default/-/56.0.34.tgz',
});
expect((await resolveTemplateAsync({ sdkVersion: 'sdk-57' })).npmTag).toBe('sdk-57');
expect((await resolveTemplateAsync({ sdkVersion: 'latest' })).npmTag).toBe('latest');
expect(promptAsync).not.toHaveBeenCalled();
expect(fetch).not.toHaveBeenCalled();
});
it('throws for an invalid SDK version', async () => {
await expect(resolveTemplateAsync({ sdkVersion: 'abc' })).rejects.toThrow(
'Invalid SDK version'
);
});
it('throws for an SDK version without a published template', async () => {
await expect(resolveTemplateAsync({ sdkVersion: '50' })).rejects.toThrow(
'A project template for "sdk-50" is not available. Supported SDK versions: 57, 56.'
);
});
it('prompts with only the SDK versions that have a published template', async () => {
mockVersionsEndpoint({
sdkVersions: {
'57.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-57' },
'56.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-56' },
'55.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-55' },
},
});
jest.mocked(promptAsync).mockResolvedValueOnce({ answer: 56 });
const template = await resolveTemplateAsync({});
expect(template.npmTag).toBe('sdk-56');
// SDK 55 has no dist-tag in the packument, so it must not be offered.
expect(promptAsync).toHaveBeenCalledWith(
expect.objectContaining({
choices: [
expect.objectContaining({ title: 'Latest (SDK 57)' }),
expect.objectContaining({ title: 'Other SDK version…' }),
],
})
);
});
it('falls back to the latest tag when the versions endpoint is unavailable', async () => {
jest.mocked(fetch).mockRejectedValueOnce(new Error('network error'));
expect((await resolveTemplateAsync({})).npmTag).toBe('latest');
expect(promptAsync).not.toHaveBeenCalled();
});
it('falls back to the latest tag when no listed version has a template', async () => {
mockVersionsEndpoint({
sdkVersions: { '99.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-99' } },
});
expect((await resolveTemplateAsync({})).npmTag).toBe('latest');
expect(promptAsync).not.toHaveBeenCalled();
});
});
@@ -1,40 +1,106 @@
import chalk from 'chalk';
import glob from 'fast-glob';
import fs from 'fs-extra';
import path from 'path';
import { pipeline } from 'stream/promises';
import { extract } from 'tar';
import { printDirectory } from './utils';
import { canAccessRepositoryUsingSshAsync, runGitCloneAsync } from '../../onboarding/git';
import fetch from '../../fetch';
import { PackageManager, installDependenciesAsync } from '../../onboarding/installDependencies';
import { runCommandAsync } from '../../onboarding/runCommand';
import { ora } from '../../ora';
import { expoCommandAsync } from '../../utils/expoCli';
export async function cloneTemplateAsync(targetProjectDir: string): Promise<string> {
const githubUsername = 'expo';
const githubRepositoryName = 'expo-template-default';
export const TEMPLATE_PACKAGE_NAME = 'expo-template-default';
const NPM_REGISTRY_URL =
process.env.npm_config_registry?.replace(/\/$/, '') ?? 'https://registry.npmjs.org';
export interface TemplateInfo {
npmTag: string;
version: string;
tarballUrl: string;
}
export interface TemplatePackument {
'dist-tags'?: Record<string, string>;
versions?: Record<string, { dist: { tarball: string } }>;
}
export async function fetchTemplatePackumentAsync(): Promise<TemplatePackument> {
const response = await fetch(`${NPM_REGISTRY_URL}/${TEMPLATE_PACKAGE_NAME}`, {
headers: { accept: 'application/vnd.npm.install-v1+json' },
signal: AbortSignal.timeout(10000),
});
return (await response.json()) as TemplatePackument;
}
export async function downloadTemplateAsync(
targetProjectDir: string,
template: TemplateInfo
): Promise<string> {
const spinner = ora(
`${chalk.bold(`Cloning the project to ${printDirectory(targetProjectDir)}`)}`
`${chalk.bold(`Downloading the project template to ${printDirectory(targetProjectDir)}`)}`
).start();
const cloneMethod = (await canAccessRepositoryUsingSshAsync({
githubUsername,
githubRepositoryName,
}))
? 'ssh'
: 'https';
const targetExisted = await fs.pathExists(targetProjectDir);
try {
// The directory was verified to not exist earlier, but it may have been
// created while the user was answering prompts.
if (targetExisted && (await fs.readdir(targetProjectDir)).length > 0) {
throw new Error(
`Directory ${printDirectory(targetProjectDir)} already exists and is not empty.`
);
}
const { targetProjectDir: finalTargetProjectDirectory } = await runGitCloneAsync({
githubUsername,
githubRepositoryName,
targetProjectDir,
cloneMethod,
showOutput: false,
await fs.mkdirp(targetProjectDir);
const tarballResponse = await fetch(template.tarballUrl);
await pipeline(tarballResponse.body, extract({ cwd: targetProjectDir, strip: 1 }));
await restoreTemplateDotfilesAsync(targetProjectDir);
spinner.succeed(
`Downloaded ${chalk.bold(`${TEMPLATE_PACKAGE_NAME}@${template.version}`)} to ${printDirectory(
targetProjectDir
)}`
);
} catch (error) {
spinner.fail();
if (!targetExisted) {
await fs.remove(targetProjectDir);
}
throw error;
}
return targetProjectDir;
}
/**
* npm strips `.gitignore` files from published packages, and the template ships
* dot-directories with an underscore prefix (e.g. `_vscode`), so restore the
* real names after extraction. This mirrors the renames in create-expo-app;
* keep the directory list in sync with SUPPORTED_DIRECTORIES in
* https://github.com/expo/expo/blob/main/packages/create-expo-app/src/createFileTransform.ts
*/
async function restoreTemplateDotfilesAsync(projectDir: string): Promise<void> {
const entries = await glob('**/{gitignore,_eas,_vscode,_github,_cursor}', {
cwd: projectDir,
onlyFiles: false,
dot: true,
});
spinner.succeed(`Cloned the project to ${printDirectory(finalTargetProjectDirectory)}`);
// Rename deeper entries first so parent renames do not invalidate child paths.
entries.sort((a, b) => b.split('/').length - a.split('/').length);
return finalTargetProjectDirectory;
for (const entry of entries) {
const basename = path.basename(entry);
const dotName = basename === 'gitignore' ? '.gitignore' : `.${basename.slice(1)}`;
await fs.move(
path.join(projectDir, entry),
path.join(projectDir, path.dirname(entry), dotName),
{ overwrite: true }
);
}
}
export async function installProjectDependenciesAsync(
@@ -110,10 +110,37 @@ export async function generateEasConfigAsync(projectDir: string): Promise<void>
await fs.writeJson(easJsonPath, easJson, { spaces: 2 });
}
export async function updatePackageJsonAsync(projectDir: string): Promise<void> {
const TEMPLATE_LICENSE = '0BSD';
// https://github.com/npm/validate-npm-package-name#naming-rules
function sanitizeNpmPackageName(name: string): string {
return (
name
.normalize('NFD')
.replace(/[\u0300-\u036f]/g, '')
.toLowerCase()
.replace(/^[._]+/, '')
.replace(/[^a-z0-9._\-/@]/g, '') || 'app'
);
}
export async function updatePackageJsonAsync(
projectDir: string,
projectName: string
): Promise<void> {
const packageJsonPath = path.join(projectDir, 'package.json');
const packageJson = await fs.readJson(packageJsonPath);
packageJson.name = sanitizeNpmPackageName(projectName);
packageJson.version = '1.0.0';
packageJson.private = true;
delete packageJson.description;
delete packageJson.tags;
delete packageJson.repository;
if (packageJson.license === TEMPLATE_LICENSE) {
delete packageJson.license;
}
if (!packageJson.scripts) {
packageJson.scripts = {};
}
@@ -125,6 +152,10 @@ export async function updatePackageJsonAsync(projectDir: string): Promise<void>
await fs.writeJson(packageJsonPath, packageJson, { spaces: 2 });
}
export async function removeTemplateFilesAsync(projectDir: string): Promise<void> {
await fs.remove(path.join(projectDir, 'LICENSE'));
}
export async function copyProjectTemplatesAsync(projectDir: string): Promise<void> {
const templatesSourceDir = path.join(__dirname, 'templates');
@@ -0,0 +1,181 @@
import { TemplateInfo, fetchTemplatePackumentAsync } from './commands';
import { getExpoApiBaseUrl } from '../../api';
import fetch from '../../fetch';
import Log from '../../log';
import { promptAsync } from '../../prompts';
export interface SdkVersions {
latest: number;
expoGoCompatible: number | null;
available: number[];
}
interface VersionsEndpointResponse {
sdkVersions?: Record<string, { releaseNoteUrl?: string; isDeprecated?: boolean }>;
expoGoSdkVersion?: string;
}
/**
* SDKs without a `releaseNoteUrl` are canary/in-development and are excluded.
*/
export async function fetchSdkVersionsAsync(): Promise<SdkVersions | null> {
let json: VersionsEndpointResponse;
try {
const response = await fetch(`${getExpoApiBaseUrl()}/v2/versions`, {
signal: AbortSignal.timeout(5000),
});
json = (await response.json()) as VersionsEndpointResponse;
} catch (error) {
Log.debug(`Failed to fetch SDK versions: ${error}`);
return null;
}
const available = Object.entries(json.sdkVersions ?? {})
.filter(([, info]) => !!info.releaseNoteUrl && !info.isDeprecated)
.map(([version]) => parseSdkMajor(version))
.filter((major): major is number => major !== null)
.sort((a, b) => b - a);
const latest = available[0];
if (latest == null) {
return null;
}
return {
latest,
expoGoCompatible: parseSdkMajor(json.expoGoSdkVersion),
available,
};
}
function parseSdkMajor(version: string | undefined): number | null {
const major = parseInt(version?.split('.')[0] ?? '', 10);
return Number.isFinite(major) ? major : null;
}
export async function promptForSdkVersionAsync({
latest,
expoGoCompatible,
available,
}: SdkVersions): Promise<number> {
const choices: { title: string; value: number | 'other'; description?: string }[] = [
{
title: `Latest (SDK ${latest})`,
value: latest,
description: 'Recommended for most projects',
},
];
if (
expoGoCompatible !== null &&
expoGoCompatible !== latest &&
available.includes(expoGoCompatible)
) {
choices.push({
title: `For learning with Expo Go (SDK ${expoGoCompatible})`,
value: expoGoCompatible,
description: 'Compatible with Expo Go on App Store and Play Store',
});
}
const otherVersions = available.filter(sdk => sdk !== latest && sdk !== expoGoCompatible);
if (otherVersions.length > 0) {
choices.push({ title: 'Other SDK version…', value: 'other' });
}
const { answer } = await promptAsync({
type: 'select',
name: 'answer',
message: 'Select an Expo SDK version:',
choices,
});
if (answer !== 'other') {
return answer;
}
const { sdkVersion } = await promptAsync({
type: 'select',
name: 'sdkVersion',
message: 'Select an SDK version:',
choices: otherVersions.map(sdk => ({
title: `SDK ${sdk}`,
value: sdk,
})),
});
return sdkVersion;
}
function resolveTemplateForTag(
distTags: Record<string, string>,
versions: Record<string, { dist: { tarball: string } }>,
npmTag: string
): TemplateInfo {
const version = distTags[npmTag];
const tarballUrl = version ? versions[version]?.dist.tarball : undefined;
if (!version || !tarballUrl) {
const supported = Object.keys(distTags)
.filter(tag => /^sdk-\d+$/.test(tag))
.sort((a, b) => parseInt(b.slice(4), 10) - parseInt(a.slice(4), 10));
throw new Error(
`A project template for "${npmTag}" is not available. Supported SDK versions: ${supported
.map(tag => tag.slice(4))
.join(', ')}.`
);
}
return { npmTag, version, tarballUrl };
}
/**
* `sdkVersion` accepts "57", "sdk-57", or "latest".
*/
export async function resolveTemplateAsync({
sdkVersion,
}: {
sdkVersion?: string;
}): Promise<TemplateInfo> {
const packument = await fetchTemplatePackumentAsync();
const templateDistTags = packument['dist-tags'] ?? {};
const packumentVersions = packument.versions ?? {};
if (sdkVersion) {
if (sdkVersion === 'latest') {
return resolveTemplateForTag(templateDistTags, packumentVersions, 'latest');
}
const major = parseSdkMajor(sdkVersion.replace(/^sdk-/, ''));
if (major === null) {
throw new Error(
`Invalid SDK version: "${sdkVersion}". Specify a version number (e.g. 57) or "latest".`
);
}
return resolveTemplateForTag(templateDistTags, packumentVersions, `sdk-${major}`);
}
const versions = await fetchSdkVersionsAsync();
const availableWithTemplates =
versions?.available.filter(sdk => `sdk-${sdk}` in templateDistTags) ?? [];
if (!versions || availableWithTemplates.length === 0) {
if (!versions) {
Log.warn('Could not fetch the list of Expo SDK versions. Using the latest SDK.');
}
return resolveTemplateForTag(templateDistTags, packumentVersions, 'latest');
}
const latest = availableWithTemplates[0];
// Match the prompt guard in prompts.ts: pin to the latest released SDK
// instead of prompting when stdin is not interactive.
if (!process.stdin.isTTY && !global.test) {
Log.withInfo(`Using Expo SDK ${latest}`);
return resolveTemplateForTag(templateDistTags, packumentVersions, `sdk-${latest}`);
}
const selectedSdk = await promptForSdkVersionAsync({
latest,
expoGoCompatible: versions.expoGoCompatible,
available: availableWithTemplates,
});
return resolveTemplateForTag(templateDistTags, packumentVersions, `sdk-${selectedSdk}`);
}
@@ -1,35 +1,137 @@
import { getMockWorkflowRunWithJobsFragment } from '../../../__tests__/commands/utils';
import { fetchRawLogsForCustomJobAsync } from '../fetchLogs';
import { infoForActiveWorkflowRunAsync } from '../utils';
import { WorkflowJobStatus } from '../../../graphql/generated';
import { groupLogLinesIntoSteps, parseLogLines } from '../logs/parseLogs';
import { WorkflowLogs, WorkflowRawLogLine } from '../types';
import { formatActiveWorkflowRun, formatFailedWorkflowRun } from '../utils';
jest.mock('../fetchLogs');
function jobWithLogs(
logLines: WorkflowRawLogLine[],
status: WorkflowJobStatus = WorkflowJobStatus.InProgress
): {
job: ReturnType<typeof getMockWorkflowRunWithJobsFragment>['jobs'][number];
logs: WorkflowLogs;
} {
return {
job: { ...getMockWorkflowRunWithJobsFragment().jobs[0], status },
logs: groupLogLinesIntoSteps(logLines),
};
}
describe('workflow utils', () => {
afterEach(() => {
jest.clearAllMocks();
});
function stepLines(buildStepId: string, count: number): WorkflowRawLogLine[] {
return Array.from({ length: count }, (_, index) => ({ buildStepId, msg: `line${index}` }));
}
test('shows the display name for the current step while keying logs by step id', async () => {
const workflowRun = getMockWorkflowRunWithJobsFragment();
workflowRun.jobs = workflowRun.jobs.map(job => ({
...job,
status: WorkflowJobStatus.InProgress,
}));
jest
.mocked(fetchRawLogsForCustomJobAsync)
.mockResolvedValue(
[
'{"buildStepId":"step-id-1","buildStepDisplayName":"Install dependencies","time":"2022-01-01T00:00:00.000Z","msg":"npm ci"}',
'{"buildStepId":"step-id-1","buildStepDisplayName":"Install dependencies","marker":"end-step","result":"success","time":"2022-01-01T00:00:01.000Z","msg":"done"}',
].join('\n')
);
const output = await infoForActiveWorkflowRunAsync({} as any, workflowRun);
describe(formatActiveWorkflowRun, () => {
test('shows the display name for the current step while keying logs by step id', () => {
const output = formatActiveWorkflowRun([
jobWithLogs(
parseLogLines(
[
'{"buildStepId":"step-id-1","buildStepDisplayName":"Install dependencies","time":"2022-01-01T00:00:00.000Z","msg":"npm ci"}',
'{"buildStepId":"step-id-1","buildStepDisplayName":"Install dependencies","marker":"end-step","result":"success","time":"2022-01-01T00:00:01.000Z","msg":"done"}',
].join('\n')
).logLines
),
]);
expect(output).toContain('Current step');
expect(output).toContain('Install dependencies');
expect(output).not.toContain('step-id-1');
});
test('shows exactly maxLogLines trailing lines of the current step', () => {
const output = formatActiveWorkflowRun([jobWithLogs(stepLines('step-id-1', 10))], 5);
expect(output).toContain('line5');
expect(output).toContain('line9');
expect(output).not.toContain('line4');
});
test('keeps five trailing lines by default', () => {
const output = formatActiveWorkflowRun([jobWithLogs(stepLines('step-id-1', 10))]);
expect(output).toContain('line5');
expect(output).toContain('line9');
expect(output).not.toContain('line4');
});
test('reports the last step as the current one', () => {
const output = formatActiveWorkflowRun([
jobWithLogs([
{ buildStepId: 'install', buildStepDisplayName: 'Install', msg: 'installing' },
{ buildStepId: 'build', buildStepDisplayName: 'Build', msg: 'compiling' },
]),
]);
expect(output).toContain('Build');
expect(output).not.toContain('Install');
});
test('passes over a trailing skipped step', () => {
const output = formatActiveWorkflowRun([
jobWithLogs([
{ buildStepId: 'install', buildStepDisplayName: 'Install', msg: 'installing' },
{
buildStepId: 'build',
buildStepDisplayName: 'Build',
msg: 'skipping',
marker: 'end-step',
result: 'skipped',
},
]),
]);
expect(output).toContain('Install');
expect(output).not.toContain('Build');
});
test('reports a step that has not ended yet', () => {
const output = formatActiveWorkflowRun([
jobWithLogs([
{
buildStepId: 'install',
buildStepDisplayName: 'Install',
msg: 'installed',
marker: 'end-step',
result: 'success',
},
{ buildStepId: 'build', buildStepDisplayName: 'Build', msg: 'compiling' },
]),
]);
expect(output).toContain('Build');
});
test('omits the current step when the job has no steps yet', () => {
const output = formatActiveWorkflowRun([jobWithLogs([])]);
expect(output).not.toContain('Current step');
expect(output).not.toContain('Current logs');
});
test('omits the current step for a job that is not in progress', () => {
const output = formatActiveWorkflowRun([
jobWithLogs(stepLines('step-id-1', 3), WorkflowJobStatus.Success),
]);
expect(output).not.toContain('Current step');
expect(output).not.toContain('line0');
});
});
describe(formatFailedWorkflowRun, () => {
test('prints every line of the failed step when there is no limit', () => {
const output = formatFailedWorkflowRun([
jobWithLogs(
[
...stepLines('build', 8),
{ buildStepId: 'build', msg: 'it failed', marker: 'end-step', result: 'fail' },
],
WorkflowJobStatus.Failure
),
]);
expect(output).toContain('line0');
expect(output).toContain('line7');
});
});
@@ -1,51 +0,0 @@
import { WorkflowJobResult } from './types';
import { BuildQuery } from '../../graphql/queries/BuildQuery';
import { ExpoGraphqlClient } from '../context/contextUtils/createGraphqlClient';
// This function is in a separate module for testing purposes
export async function fetchRawLogsForCustomJobAsync(
job: WorkflowJobResult
): Promise<string | null> {
const firstLogFileUrl = job.turtleJobRun?.logFileUrls?.[0];
if (!firstLogFileUrl) {
return null;
}
const response = await fetch(firstLogFileUrl, {
method: 'GET',
});
const rawLogs = await response.text();
return rawLogs;
}
export async function fetchRawLogsForBuildJobAsync(
state: { graphqlClient: ExpoGraphqlClient },
job: WorkflowJobResult
): Promise<string | null> {
// Prefer turtleJobRun logs, which contain JSONL-formatted step logs
// for both built-in and custom build steps
const turtleLogFileUrl = job.turtleJobRun?.logFileUrls?.[0];
if (turtleLogFileUrl) {
const response = await fetch(turtleLogFileUrl, {
method: 'GET',
});
return await response.text();
}
// Fall back to build logFiles
const buildId = job.outputs?.build_id;
if (!buildId) {
return null;
}
const buildFragment = await BuildQuery.byIdAsync(state.graphqlClient, buildId, {
useCache: false,
});
const firstLogFileUrl = buildFragment.logFiles?.[0];
if (!firstLogFileUrl) {
return null;
}
const response = await fetch(firstLogFileUrl, {
method: 'GET',
});
return await response.text();
}
@@ -0,0 +1,27 @@
import { getMockWorkflowCustomJobFragment } from '../../../../__tests__/commands/utils';
import { BuildQuery } from '../../../../graphql/queries/BuildQuery';
import { ExpoGraphqlClient } from '../../../context/contextUtils/createGraphqlClient';
import { fetchRawLogsForJobAsync } from '../fetchLogs';
jest.mock('../../../../graphql/queries/BuildQuery');
describe(fetchRawLogsForJobAsync, () => {
afterEach(() => {
jest.restoreAllMocks();
});
it('reads a custom job from its turtle job run log file', async () => {
const rawLogs = '{"logId":"1","buildStepId":"install","msg":"from the turtle job run"}';
const fetchSpy = jest.spyOn(globalThis, 'fetch').mockResolvedValue(new Response(rawLogs));
await expect(
fetchRawLogsForJobAsync(
{ graphqlClient: {} as ExpoGraphqlClient },
getMockWorkflowCustomJobFragment()
)
).resolves.toBe(rawLogs);
expect(fetchSpy).toHaveBeenCalledWith('https://example.com/log1');
expect(BuildQuery.byIdAsync).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,186 @@
import { groupLogLinesIntoSteps, mergeLogLines, parseLogLines } from '../parseLogs';
import { WorkflowRawLogLine } from '../../types';
function logLine(overrides: Partial<WorkflowRawLogLine> = {}): WorkflowRawLogLine {
return { msg: 'a message', ...overrides };
}
describe(parseLogLines, () => {
it('parses a JSONL log file', () => {
const { logLines, errors } = parseLogLines(
[
'{"logId":"1","buildStepId":"install","msg":"npm ci"}',
'{"logId":"2","buildStepId":"install","msg":"done"}',
].join('\n')
);
expect(errors).toEqual([]);
expect(logLines).toEqual([
{ logId: '1', buildStepId: 'install', msg: 'npm ci' },
{ logId: '2', buildStepId: 'install', msg: 'done' },
]);
});
it('skips blank lines, including a trailing newline', () => {
const { logLines, errors } = parseLogLines('{"logId":"1","msg":"one"}\n\n');
expect(errors).toEqual([]);
expect(logLines).toHaveLength(1);
});
it('reports malformed lines as errors while keeping the parsable ones', () => {
const { logLines, errors } = parseLogLines(
['{"logId":"1","msg":"one"}', 'this is not json', '{"logId":"2","msg":"two"}'].join('\n')
);
expect(logLines.map(line => line.msg)).toEqual(['one', 'two']);
expect(errors).toHaveLength(1);
expect(errors[0]).toBeInstanceOf(Error);
});
it('parses a line that carries no message', () => {
const { logLines, errors } = parseLogLines('{"logId":"1"}');
expect(errors).toEqual([]);
expect(logLines).toEqual([{ logId: '1' }]);
});
});
describe(mergeLogLines, () => {
it('keeps the rightmost line for a repeated logId', () => {
const fileLogLines = [
logLine({ logId: '1', msg: 'from the file' }),
logLine({ logId: '2', msg: 'from the file' }),
];
const realtimeLogLines = [
logLine({ logId: '2', msg: 'from realtime' }),
logLine({ logId: '3', msg: 'from realtime' }),
];
expect(mergeLogLines(fileLogLines, realtimeLogLines)).toEqual([
{ logId: '1', msg: 'from the file' },
{ logId: '2', msg: 'from realtime' },
{ logId: '3', msg: 'from realtime' },
]);
});
it('keeps identical messages that have different logIds', () => {
const merged = mergeLogLines(
[logLine({ logId: '1', msg: 'Repeated text' })],
[logLine({ logId: '2', msg: 'Repeated text' })]
);
expect(merged).toEqual([
{ logId: '1', msg: 'Repeated text' },
{ logId: '2', msg: 'Repeated text' },
]);
});
it('keeps identical lines without logId', () => {
const merged = mergeLogLines(
[logLine({ msg: 'same' }), logLine({ msg: 'same' })],
[logLine({ msg: 'same' })]
);
expect(merged).toHaveLength(3);
});
it('deduplicates within a single group', () => {
const merged = mergeLogLines([logLine({ logId: '1' }), logLine({ logId: '1' })]);
expect(merged).toHaveLength(1);
});
});
describe(groupLogLinesIntoSteps, () => {
it('groups lines by step id and labels the step with its display name', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepId: 'step-id-1', msg: 'npm ci' }),
logLine({
buildStepId: 'step-id-1',
buildStepDisplayName: 'Install dependencies',
msg: 'ok',
}),
]);
expect(Array.from(logs.keys())).toEqual(['step-id-1']);
expect(logs.get('step-id-1')).toEqual({
key: 'step-id-1',
label: 'Install dependencies',
logLines: [
{ time: undefined, msg: 'npm ci', result: undefined, marker: undefined, err: undefined },
{ time: undefined, msg: 'ok', result: undefined, marker: undefined, err: undefined },
],
});
});
it('keeps steps in the order they first appear', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepId: 'first' }),
logLine({ buildStepId: 'second' }),
logLine({ buildStepId: 'first' }),
logLine({ buildStepId: 'third' }),
]);
expect(Array.from(logs.keys())).toEqual(['first', 'second', 'third']);
});
it('falls back to the display name, then the phase, when there is no step id', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepDisplayName: 'Run fastlane' }),
logLine({ phase: 'PREPARE_CREDENTIALS' }),
]);
expect(Array.from(logs.keys())).toEqual(['Run fastlane', 'PREPARE_CREDENTIALS']);
expect(logs.get('PREPARE_CREDENTIALS')?.label).toBe('PREPARE_CREDENTIALS');
});
it('drops lines that belong to no step', () => {
const logs = groupLogLinesIntoSteps([logLine({ msg: 'a line with no step' })]);
expect(logs.size).toBe(0);
});
it('drops lines that carry no message, keeping the rest of the step', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepId: 'install', msg: undefined, marker: 'start-step' }),
logLine({ buildStepId: 'install', msg: 'npm ci' }),
]);
expect(logs.get('install')?.logLines).toEqual([
{ time: undefined, msg: 'npm ci', result: undefined, marker: undefined, err: undefined },
]);
});
it('creates no step for a line that carries no message', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepId: 'install', msg: undefined, marker: 'end-step', result: 'success' }),
]);
expect(logs.size).toBe(0);
});
it('records the result of the first end marker and ignores a later one', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepId: 'install', msg: 'skipping', marker: 'end-step', result: 'skipped' }),
logLine({ buildStepId: 'install', msg: 'retried', marker: 'end-step', result: 'success' }),
]);
expect(logs.get('install')?.result).toBe('skipped');
});
it('leaves the result undefined for a step that has not ended', () => {
const logs = groupLogLinesIntoSteps([logLine({ buildStepId: 'install', msg: 'npm ci' })]);
expect(logs.get('install')?.result).toBeUndefined();
});
it('records an end marker that carries no result and does not let a later one replace it', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepId: 'install', msg: 'ended', marker: 'END_PHASE' }),
logLine({ buildStepId: 'install', msg: 'retried', marker: 'END_PHASE', result: 'success' }),
]);
expect(logs.get('install')?.result).toBe('');
});
});
@@ -0,0 +1,308 @@
import { getMockWorkflowRunWithJobsFragment } from '../../../../__tests__/commands/utils';
import { RealtimeLogsTargetType, WorkflowJobStatus } from '../../../../graphql/generated';
import { RealtimeLogsClient } from '../../../../utils/centrifuge';
import { fetchAndParseLogsFromJobAsync } from '../parseLogs';
import { WorkflowJobLogsState, WorkflowRunLogsWatcher, realtimeLogsTargetForJob } from '../watcher';
import { WorkflowJobResult, WorkflowRawLogLine } from '../../types';
jest.mock('../parseLogs', () => ({
...jest.requireActual('../parseLogs'),
fetchAndParseLogsFromJobAsync: jest.fn(),
}));
function createFakeClient(): {
client: RealtimeLogsClient;
publish: (data: unknown) => void;
subscribeCalls: unknown[];
closeCount: () => number;
} {
const listeners: ((data: unknown) => void)[] = [];
const subscribeCalls: unknown[] = [];
let closed = 0;
return {
subscribeCalls,
closeCount: () => closed,
publish: data => {
listeners.forEach(listener => {
listener(data);
});
},
client: {
subscribeAsync: async (args, onPublication) => {
subscribeCalls.push(args);
listeners.push(onPublication);
return { close: () => closed++ };
},
close: () => closed++,
},
};
}
function inProgressJob(): WorkflowJobResult {
const job = getMockWorkflowRunWithJobsFragment().jobs[0];
return { ...job, status: WorkflowJobStatus.InProgress };
}
describe(realtimeLogsTargetForJob, () => {
it('targets the turtle job run when there is one', () => {
const job = { ...inProgressJob(), turtleJobRun: { id: 'job-run-id' } } as WorkflowJobResult;
expect(realtimeLogsTargetForJob(job)).toEqual({
type: RealtimeLogsTargetType.JobRun,
id: 'job-run-id',
});
});
it('falls back to the build when there is no turtle job run', () => {
const job = {
...inProgressJob(),
turtleJobRun: null,
turtleBuild: { id: 'build-id' },
} as WorkflowJobResult;
expect(realtimeLogsTargetForJob(job)).toEqual({
type: RealtimeLogsTargetType.Build,
id: 'build-id',
});
});
it('has no target when neither is present', () => {
const job = {
...inProgressJob(),
turtleJobRun: null,
turtleBuild: null,
outputs: null,
} as WorkflowJobResult;
expect(realtimeLogsTargetForJob(job)).toBeNull();
});
});
describe(WorkflowRunLogsWatcher, () => {
beforeEach(() => {
jest
.mocked(fetchAndParseLogsFromJobAsync)
.mockResolvedValue([{ logId: '1', buildStepId: 'install', msg: 'from the file' }]);
});
afterEach(() => {
jest.clearAllMocks();
});
it('subscribes once per in-progress job', async () => {
const fake = createFakeClient();
const watcher = new WorkflowRunLogsWatcher(
{} as any,
() => fake.client,
() => {}
);
const job = inProgressJob();
await watcher.syncJobsAsync([job]);
await watcher.syncJobsAsync([job]);
expect(fake.subscribeCalls).toHaveLength(1);
});
it('reports realtime logs as they arrive', async () => {
const fake = createFakeClient();
const onRealtimeLogs = jest.fn();
const watcher = new WorkflowRunLogsWatcher({} as any, () => fake.client, onRealtimeLogs);
await watcher.syncJobsAsync([inProgressJob()]);
fake.publish([{ logId: '2', buildStepId: 'install', msg: 'pushed' }]);
expect(onRealtimeLogs).toHaveBeenCalledTimes(1);
});
it('does not report a publication that carries no usable log lines', async () => {
const fake = createFakeClient();
const onRealtimeLogs = jest.fn();
const watcher = new WorkflowRunLogsWatcher({} as any, () => fake.client, onRealtimeLogs);
await watcher.syncJobsAsync([inProgressJob()]);
fake.publish([{ msg: 'no logId' }]);
expect(onRealtimeLogs).not.toHaveBeenCalled();
});
it('still fetches logs when the realtime logs client is unavailable', async () => {
const watcher = new WorkflowRunLogsWatcher(
{} as any,
() => null,
() => {}
);
await watcher.syncJobsAsync([inProgressJob()]);
expect(fetchAndParseLogsFromJobAsync).toHaveBeenCalledTimes(1);
});
it('fetches logs on every sync while in progress, and stops once the job completes', async () => {
const fake = createFakeClient();
const watcher = new WorkflowRunLogsWatcher(
{} as any,
() => fake.client,
() => {}
);
const job = inProgressJob();
await watcher.syncJobsAsync([job]);
await watcher.syncJobsAsync([job]);
expect(fetchAndParseLogsFromJobAsync).toHaveBeenCalledTimes(2);
await watcher.syncJobsAsync([{ ...job, status: WorkflowJobStatus.Success }]);
expect(fetchAndParseLogsFromJobAsync).toHaveBeenCalledTimes(2);
});
it('does not fetch logs for a job that has not started', async () => {
const fake = createFakeClient();
const watcher = new WorkflowRunLogsWatcher(
{} as any,
() => fake.client,
() => {}
);
await watcher.syncJobsAsync([{ ...inProgressJob(), status: WorkflowJobStatus.New }]);
expect(fetchAndParseLogsFromJobAsync).not.toHaveBeenCalled();
});
it('closes the subscription when a job leaves in-progress', async () => {
const fake = createFakeClient();
const watcher = new WorkflowRunLogsWatcher(
{} as any,
() => fake.client,
() => {}
);
const job = inProgressJob();
await watcher.syncJobsAsync([job]);
expect(fake.closeCount()).toBe(0);
await watcher.syncJobsAsync([{ ...job, status: WorkflowJobStatus.Failure }]);
expect(fake.closeCount()).toBe(1);
});
it('closes the client and its subscriptions', async () => {
const fake = createFakeClient();
const watcher = new WorkflowRunLogsWatcher(
{} as any,
() => fake.client,
() => {}
);
await watcher.syncJobsAsync([inProgressJob()]);
watcher.close();
expect(fake.closeCount()).toBe(2);
});
});
function fileLine(logId: string, msg: string, buildStepId = 'install'): WorkflowRawLogLine {
return { logId, buildStepId, msg };
}
function stepMessages(logsState: WorkflowJobLogsState): string[] {
return Array.from(logsState.getLogs().values()).flatMap(group =>
group.logLines.map(logLine => logLine.msg)
);
}
describe(WorkflowJobLogsState, () => {
it('hides realtime lines until the file logs reach them', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('5', 'pushed')]);
expect(stepMessages(logsState)).toEqual(['first']);
});
it('shows realtime lines once a published logId also appears in the file', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('2', 'pushed')]);
logsState.ingestFileLogLines([fileLine('1', 'first'), fileLine('2', 'pushed')]);
logsState.ingestRealtimeLogLines([fileLine('3', 'newer')]);
expect(stepMessages(logsState)).toEqual(['first', 'pushed', 'newer']);
});
it('shows realtime logs when a publication repeats a logId already in the file', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('1', 'first'), fileLine('2', 'pushed')]);
expect(stepMessages(logsState)).toEqual(['first', 'pushed']);
});
it('shows buffered realtime lines once the job is completed', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('9', 'pushed')]);
logsState.markCompleted();
logsState.markCompleted();
expect(stepMessages(logsState)).toEqual(['first', 'pushed']);
});
it('folds later publications once completion showed them', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('9', 'buffered')]);
logsState.markCompleted();
logsState.ingestRealtimeLogLines([fileLine('10', 'pushed')]);
expect(stepMessages(logsState)).toEqual(['first', 'buffered', 'pushed']);
});
it('replaces the file snapshot instead of accumulating it', () => {
const logsState = new WorkflowJobLogsState();
const keyless: WorkflowRawLogLine = { buildStepId: 'install', msg: 'submission log' };
logsState.ingestFileLogLines([keyless]);
logsState.ingestFileLogLines([keyless]);
expect(stepMessages(logsState)).toEqual(['submission log']);
});
it('drops a buffered realtime line once it lands in the file', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('2', 'pushed')]);
logsState.ingestFileLogLines([fileLine('1', 'first'), fileLine('2', 'pushed')]);
expect(stepMessages(logsState)).toEqual(['first', 'pushed']);
});
it('ignores publications that are not arrays of log lines', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
expect(logsState.ingestRealtimeLogLines({ logId: '2', msg: 'not an array' })).toBe(false);
expect(logsState.ingestRealtimeLogLines(['a string', 42, null])).toBe(false);
expect(logsState.ingestRealtimeLogLines([{ msg: 'no logId' }])).toBe(false);
expect(stepMessages(logsState)).toEqual(['first']);
});
it('deduplicates a line republished on the same channel', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('1', 'first'), fileLine('2', 'pushed')]);
logsState.ingestRealtimeLogLines([fileLine('2', 'pushed')]);
expect(stepMessages(logsState)).toEqual(['first', 'pushed']);
});
it('deduplicates a line republished later', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('1', 'first'), fileLine('2', 'pushed')]);
logsState.ingestRealtimeLogLines([fileLine('3', 'newer')]);
logsState.ingestRealtimeLogLines([fileLine('2', 'pushed')]);
expect(stepMessages(logsState)).toEqual(['first', 'pushed', 'newer']);
});
});
@@ -0,0 +1,28 @@
import { WorkflowJobResult } from '../types';
import { BuildQuery } from '../../../graphql/queries/BuildQuery';
import { ExpoGraphqlClient } from '../../context/contextUtils/createGraphqlClient';
export async function fetchRawLogsForJobAsync(
state: { graphqlClient: ExpoGraphqlClient },
job: WorkflowJobResult
): Promise<string | null> {
const turtleLogFileUrl = job.turtleJobRun?.logFileUrls?.[0];
if (turtleLogFileUrl) {
const response = await fetch(turtleLogFileUrl);
return await response.text();
}
const buildId = job.outputs?.build_id;
if (!buildId) {
return null;
}
const buildFragment = await BuildQuery.byIdAsync(state.graphqlClient, buildId, {
useCache: false,
});
const firstLogFileUrl = buildFragment.logFiles?.[0];
if (!firstLogFileUrl) {
return null;
}
const response = await fetch(firstLogFileUrl);
return await response.text();
}
@@ -0,0 +1,83 @@
import { fetchRawLogsForJobAsync } from './fetchLogs';
import { WorkflowJobResult, WorkflowLogs, WorkflowRawLogLine } from '../types';
import Log from '../../../log';
import uniqBy from '../../../utils/expodash/uniqBy';
import { ExpoGraphqlClient } from '../../context/contextUtils/createGraphqlClient';
export function parseLogLines(rawLogs: string): {
logLines: WorkflowRawLogLine[];
errors: Error[];
} {
const logLines: WorkflowRawLogLine[] = [];
const errors: Error[] = [];
for (const rawLogLine of rawLogs.split('\n')) {
if (!rawLogLine) {
continue;
}
try {
logLines.push(JSON.parse(rawLogLine));
} catch (err) {
errors.push(err as Error);
}
}
return { logLines, errors };
}
export function mergeLogLines<T extends WorkflowRawLogLine>(...logLineGroups: T[][]): T[] {
return uniqBy(logLineGroups.flat().reverse(), logLine => logLine.logId ?? Symbol()).reverse();
}
export function groupLogLinesIntoSteps(
logLines: WorkflowRawLogLine[],
accumulator: WorkflowLogs = new Map()
): WorkflowLogs {
for (const logLine of logLines) {
const { buildStepDisplayName, buildStepId, phase, time, msg, result, marker, err } = logLine;
const stepKey = buildStepId ?? buildStepDisplayName ?? phase;
const stepLabel = buildStepDisplayName ?? buildStepId ?? phase;
if (!stepKey || !stepLabel || !msg) {
continue;
}
let logGroup = accumulator.get(stepKey);
if (!logGroup) {
logGroup = { key: stepKey, label: stepLabel, logLines: [] };
accumulator.set(stepKey, logGroup);
}
if (buildStepDisplayName) {
logGroup.label = buildStepDisplayName;
}
if (logGroup.result === undefined && (marker === 'end-step' || marker === 'END_PHASE')) {
logGroup.result = result ?? '';
}
logGroup.logLines.push({ time, msg, result, marker, err });
}
return accumulator;
}
export async function fetchAndParseLogsFromJobAsync(
state: { graphqlClient: ExpoGraphqlClient },
job: WorkflowJobResult
): Promise<WorkflowRawLogLine[] | null> {
const rawLogs = await fetchRawLogsForJobAsync(state, job);
if (!rawLogs) {
return null;
}
Log.debug(`rawLogs = ${JSON.stringify(rawLogs, null, 2)}`);
const { logLines, errors } = parseLogLines(rawLogs);
for (const error of errors) {
Log.debug(`Failed to parse a log line: ${error.message}`);
}
return logLines;
}
export async function fetchAndProcessLogsFromJobAsync(
state: { graphqlClient: ExpoGraphqlClient },
job: WorkflowJobResult
): Promise<WorkflowLogs | null> {
const logLines = await fetchAndParseLogsFromJobAsync(state, job);
return logLines && groupLogLinesIntoSteps(logLines);
}
@@ -0,0 +1,210 @@
import { fetchAndParseLogsFromJobAsync, groupLogLinesIntoSteps, mergeLogLines } from './parseLogs';
import { WorkflowJobResult, WorkflowLogs, WorkflowRawLogLine } from '../types';
import {
RealtimeLogsTargetInput,
RealtimeLogsTargetType,
WorkflowJobStatus,
} from '../../../graphql/generated';
import Log from '../../../log';
import { RealtimeLogsClient, RealtimeLogsSubscription } from '../../../utils/centrifuge';
import { ExpoGraphqlClient } from '../../context/contextUtils/createGraphqlClient';
import nullthrows from 'nullthrows';
type RealtimeLogLine = WorkflowRawLogLine & { logId: string };
function isRealtimeLogLine(entry: unknown): entry is RealtimeLogLine {
return (
typeof entry === 'object' &&
entry !== null &&
typeof (entry as Partial<WorkflowRawLogLine>).logId === 'string'
);
}
export class WorkflowJobLogsState {
private fileLogIds = new Set<string>();
private realtimeLogLines: RealtimeLogLine[] = [];
private realtimeLogIds = new Set<string>();
private haveFileLogsCaughtUp = false;
private realtimeLogsRevealed = false;
private groupedLogs: WorkflowLogs = new Map();
public ingestFileLogLines(logLines: WorkflowRawLogLine[]): void {
this.fileLogIds = new Set(logLines.flatMap(logLine => (logLine.logId ? [logLine.logId] : [])));
if (
!this.haveFileLogsCaughtUp &&
this.realtimeLogLines.some(logLine => this.fileLogIds.has(logLine.logId))
) {
this.haveFileLogsCaughtUp = true;
}
this.realtimeLogLines = this.realtimeLogLines.filter(
logLine => !this.fileLogIds.has(logLine.logId)
);
this.realtimeLogIds = new Set(this.realtimeLogLines.map(logLine => logLine.logId));
if (this.haveFileLogsCaughtUp) {
this.realtimeLogsRevealed = true;
}
this.groupedLogs = groupLogLinesIntoSteps(
mergeLogLines(logLines, this.realtimeLogsRevealed ? this.realtimeLogLines : [])
);
}
public ingestRealtimeLogLines(data: unknown): boolean {
if (!Array.isArray(data)) {
return false;
}
const publishedLogLines = data.filter(isRealtimeLogLine);
if (publishedLogLines.length === 0) {
return false;
}
if (
!this.haveFileLogsCaughtUp &&
publishedLogLines.some(logLine => this.fileLogIds.has(logLine.logId))
) {
this.haveFileLogsCaughtUp = true;
}
const newLogLines = publishedLogLines.filter(
logLine => !this.fileLogIds.has(logLine.logId) && !this.realtimeLogIds.has(logLine.logId)
);
for (const logLine of newLogLines) {
this.realtimeLogLines.push(logLine);
this.realtimeLogIds.add(logLine.logId);
}
if (!this.realtimeLogsRevealed) {
if (this.haveFileLogsCaughtUp) {
this.revealRealtimeLogs();
}
} else if (newLogLines.length > 0) {
groupLogLinesIntoSteps(newLogLines, this.groupedLogs);
}
return true;
}
public markCompleted(): void {
this.revealRealtimeLogs();
}
public getLogs(): WorkflowLogs {
return this.groupedLogs;
}
private revealRealtimeLogs(): void {
if (this.realtimeLogsRevealed) {
return;
}
this.realtimeLogsRevealed = true;
groupLogLinesIntoSteps(this.realtimeLogLines, this.groupedLogs);
}
}
type TrackedJob = {
logsState: WorkflowJobLogsState;
subscription: RealtimeLogsSubscription | null;
};
export function realtimeLogsTargetForJob(job: WorkflowJobResult): RealtimeLogsTargetInput | null {
if (job.turtleJobRun?.id) {
return { type: RealtimeLogsTargetType.JobRun, id: job.turtleJobRun.id };
}
const buildId = job.turtleBuild?.id ?? job.outputs?.build_id;
if (buildId) {
return { type: RealtimeLogsTargetType.Build, id: buildId };
}
return null;
}
export class WorkflowRunLogsWatcher {
private readonly trackedJobs = new Map<string, TrackedJob>();
private realtimeLogsClient?: RealtimeLogsClient | null;
constructor(
private readonly graphqlClient: ExpoGraphqlClient,
private readonly createRealtimeLogsClient: () => RealtimeLogsClient | null,
private readonly onRealtimeLogs: () => void
) {}
public async syncJobsAsync(
jobs: WorkflowJobResult[]
): Promise<Map<string, WorkflowJobLogsState>> {
await Promise.all(jobs.map(job => this.syncJobAsync(job)));
return new Map(
jobs.map(job => [
job.id,
nullthrows(
this.trackedJobs.get(job.id),
'job has to be in tracked jobs after it was synced'
).logsState,
])
);
}
public close(): void {
for (const trackedJob of this.trackedJobs.values()) {
trackedJob.subscription?.close();
trackedJob.subscription = null;
}
this.realtimeLogsClient?.close();
}
private async syncJobAsync(job: WorkflowJobResult): Promise<void> {
let trackedJob = this.trackedJobs.get(job.id);
if (!trackedJob) {
trackedJob = { logsState: new WorkflowJobLogsState(), subscription: null };
this.trackedJobs.set(job.id, trackedJob);
}
const workflowInProgress = job.status === WorkflowJobStatus.InProgress;
if (!workflowInProgress) {
trackedJob.subscription?.close();
trackedJob.subscription = null;
return;
}
await Promise.all([
trackedJob.subscription ? Promise.resolve() : this.subscribeAsync(job, trackedJob),
this.fetchLogsAsync(job, trackedJob),
]);
}
private async fetchLogsAsync(job: WorkflowJobResult, trackedJob: TrackedJob): Promise<void> {
try {
const logLines = await fetchAndParseLogsFromJobAsync(
{ graphqlClient: this.graphqlClient },
job
);
if (logLines) {
trackedJob.logsState.ingestFileLogLines(logLines);
}
} catch (err: any) {
Log.debug(`Failed to fetch logs for job ${job.id}: ${err.message}`);
}
}
private getRealtimeLogsClient(): RealtimeLogsClient | null {
if (this.realtimeLogsClient === undefined) {
this.realtimeLogsClient = this.createRealtimeLogsClient();
}
return this.realtimeLogsClient;
}
private async subscribeAsync(job: WorkflowJobResult, trackedJob: TrackedJob): Promise<void> {
const target = realtimeLogsTargetForJob(job);
if (!target) {
return;
}
const realtimeLogsClient = this.getRealtimeLogsClient();
if (!realtimeLogsClient) {
return;
}
trackedJob.subscription = await realtimeLogsClient.subscribeAsync({ target }, data => {
if (trackedJob.logsState.ingestRealtimeLogLines(data)) {
this.onRealtimeLogs();
}
});
}
}
@@ -1,11 +1,11 @@
import { Choice } from 'prompts';
import { WorkflowJobResult, WorkflowLogs } from './types';
import { fetchAndProcessLogsFromJobAsync } from './logs/parseLogs';
import {
choiceFromWorkflowJob,
choiceFromWorkflowRun,
choicesFromWorkflowLogs,
fetchAndProcessLogsFromJobAsync,
processWorkflowRuns,
} from './utils';
import { AppQuery } from '../../graphql/queries/AppQuery';
@@ -42,19 +42,26 @@ export type WorkflowRunWithJobsResult = WorkflowRunResult & {
logs?: string;
};
export type WorkflowLogLine = {
time: string;
msg: string;
export type WorkflowRawLogLine = {
logId?: string;
buildStepId?: string;
buildStepDisplayName?: string;
phase?: string;
time?: string;
msg?: string;
result?: string;
marker?: string;
err?: any;
};
export type WorkflowLogs = Map<
string,
{
key: string;
label: string;
logLines: WorkflowLogLine[];
}
>;
export type WorkflowLogLine = Pick<WorkflowRawLogLine, 'time' | 'result' | 'marker' | 'err'> &
Required<Pick<WorkflowRawLogLine, 'msg'>>;
export type StepLogs = {
key: string;
label: string;
result?: string;
logLines: WorkflowLogLine[];
};
export type WorkflowLogs = Map<string, StepLogs>;
@@ -1,7 +1,7 @@
import chalk from 'chalk';
import * as fs from 'node:fs';
import { fetchRawLogsForBuildJobAsync, fetchRawLogsForCustomJobAsync } from './fetchLogs';
import { fetchAndProcessLogsFromJobAsync } from './logs/parseLogs';
import {
WorkflowJobResult,
WorkflowLogLine,
@@ -11,20 +11,22 @@ import {
} from './types';
import {
WorkflowJobStatus,
WorkflowJobType,
WorkflowRunByIdQuery,
WorkflowRunByIdWithJobsQuery,
WorkflowRunFragment,
WorkflowRunStatus,
WorkflowRunTriggerEventType,
} from '../../graphql/generated';
import { WorkflowRunLogsWatcher } from './logs/watcher';
import { WorkflowRunQuery } from '../../graphql/queries/WorkflowRunQuery';
import Log from '../../log';
import { ora } from '../../ora';
import { ora, updateSpinnerText } from '../../ora';
import { Choice } from '../../prompts';
import formatFields from '../../utils/formatFields';
import { createRealtimeLogsClient } from '../../utils/centrifuge';
import { sleepAsync } from '../../utils/promise';
import { ExpoGraphqlClient } from '../context/contextUtils/createGraphqlClient';
import nullthrows from 'nullthrows';
export function computeTriggerInfoForWorkflowRun(run: WorkflowRunFragment): {
triggerType: WorkflowTriggerType;
@@ -85,19 +87,13 @@ export function choicesFromWorkflowLogs(
logs: WorkflowLogs
): (Choice & { name: string; status: string; logLines: WorkflowLogLine[] | undefined })[] {
return Array.from(logs.values())
.map(({ key, label, logLines }) => {
const stepStatus =
logLines?.filter(
(line: WorkflowLogLine) => line.marker === 'end-step' || line.marker === 'END_PHASE'
)[0]?.result ?? '';
return {
title: `${label} - ${stepStatus}`,
name: label,
status: stepStatus,
value: key,
logLines,
};
})
.map(({ key, label, result, logLines }) => ({
title: `${label} - ${result ?? ''}`,
name: label,
status: result ?? '',
value: key,
logLines,
}))
.filter(step => step.status !== 'skipped');
}
@@ -121,52 +117,6 @@ export function processWorkflowRuns(runs: WorkflowRunFragment[]): WorkflowRunRes
});
}
export async function fetchAndProcessLogsFromJobAsync(
state: { graphqlClient: ExpoGraphqlClient },
job: WorkflowJobResult
): Promise<WorkflowLogs | null> {
let rawLogs: string | null;
switch (job.type) {
case WorkflowJobType.Build:
case WorkflowJobType.Repack:
rawLogs = await fetchRawLogsForBuildJobAsync(state, job);
break;
default:
rawLogs = await fetchRawLogsForCustomJobAsync(job);
break;
}
if (!rawLogs) {
return null;
}
Log.debug(`rawLogs = ${JSON.stringify(rawLogs, null, 2)}`);
const logs: WorkflowLogs = new Map();
rawLogs.split('\n').forEach((line, index) => {
Log.debug(`line ${index} = ${JSON.stringify(line, null, 2)}`);
try {
const parsedLine = JSON.parse(line);
const { buildStepDisplayName, buildStepId, phase, time, msg, result, marker, err } =
parsedLine;
const stepKey = buildStepId ?? buildStepDisplayName ?? phase;
const stepLabel = buildStepDisplayName ?? buildStepId ?? phase;
if (stepKey && stepLabel) {
if (!logs.has(stepKey)) {
logs.set(stepKey, {
key: stepKey,
label: stepLabel,
logLines: [],
});
}
const logGroup = logs.get(stepKey)!;
if (buildStepDisplayName) {
logGroup.label = buildStepDisplayName;
}
logGroup.logLines.push({ time, msg, result, marker, err });
}
} catch {}
});
return logs;
}
function descriptionForJobStatus(status: WorkflowJobStatus): string {
switch (status) {
case WorkflowJobStatus.New:
@@ -188,82 +138,106 @@ function descriptionForJobStatus(status: WorkflowJobStatus): string {
}
}
export async function infoForActiveWorkflowRunAsync(
function isJobCompleted(status: WorkflowJobStatus): boolean {
switch (status) {
case WorkflowJobStatus.Success:
case WorkflowJobStatus.Failure:
case WorkflowJobStatus.Canceled:
case WorkflowJobStatus.Skipped:
return true;
case WorkflowJobStatus.New:
case WorkflowJobStatus.InProgress:
case WorkflowJobStatus.ActionRequired:
case WorkflowJobStatus.PendingCancel:
return false;
}
}
type WorkflowRunWithJobs = WorkflowRunByIdWithJobsQuery['workflowRuns']['byId'];
type JobWithLogs = {
job: WorkflowRunWithJobs['jobs'][number];
logs: WorkflowLogs;
};
function stepLogTail(
step: { logLines?: WorkflowLogLine[] },
maxLogLines: number // -1 means no limit
): string[] {
const logLines = step.logLines ?? [];
const tail = maxLogLines === -1 ? logLines : logLines.slice(-maxLogLines);
return tail.map(line => line.msg);
}
export async function logsForFailedWorkflowRunAsync(
graphqlClient: ExpoGraphqlClient,
workflowRun: WorkflowRunByIdWithJobsQuery['workflowRuns']['byId'],
workflowRun: WorkflowRunWithJobs
): Promise<JobWithLogs[]> {
const jobs = workflowRun.jobs.filter(job => job.status === WorkflowJobStatus.Failure);
return await Promise.all(
jobs.map(async job => {
const maybeLogs = await fetchAndProcessLogsFromJobAsync({ graphqlClient }, job);
return { job, logs: maybeLogs ?? new Map() };
})
);
}
export function formatActiveWorkflowRun(
jobLogs: JobWithLogs[],
maxLogLines: number = 5 // -1 means no limit
): Promise<string> {
const statusLines = [];
): string {
const statusValues = [];
for (const job of workflowRun.jobs) {
for (const { job, logs } of jobLogs) {
statusValues.push({ label: '', value: '' });
statusValues.push({ label: ' Job', value: job.name });
statusValues.push({ label: ' Status', value: descriptionForJobStatus(job.status) });
if (job.status !== WorkflowJobStatus.InProgress) {
continue;
}
const logs = await fetchAndProcessLogsFromJobAsync({ graphqlClient }, job);
const steps = logs ? choicesFromWorkflowLogs(logs) : [];
if (steps.length > 0) {
const currentStep = steps[steps.length - 1];
statusValues.push({ label: ' Current step', value: currentStep.name });
if (currentStep?.logLines?.length) {
statusValues.push({ label: ' Current logs', value: '' });
const currentLogs =
currentStep.logLines
?.map(line => line.msg)
.filter((_, index) => {
if (maxLogLines === -1) {
return true;
}
return index > (currentStep.logLines?.length ?? 0) - maxLogLines;
}) ?? [];
for (const log of currentLogs) {
statusValues.push({ label: '', value: log });
}
}
}
}
statusValues.push({ label: '', value: '' });
statusLines.push(formatFields(statusValues));
return statusLines.join('\n');
}
export async function infoForFailedWorkflowRunAsync(
graphqlClient: ExpoGraphqlClient,
workflowRun: WorkflowRunByIdWithJobsQuery['workflowRuns']['byId'],
maxLogLines: number = -1 // -1 means no limit
): Promise<string> {
const statusLines = [];
const statusValues = [];
const logLinesToKeep = maxLogLines === -1 ? Infinity : maxLogLines;
for (const job of workflowRun.jobs) {
if (job.status !== WorkflowJobStatus.Failure) {
const steps = choicesFromWorkflowLogs(logs);
if (steps.length === 0) {
continue;
}
const logs = await fetchAndProcessLogsFromJobAsync({ graphqlClient }, job);
const steps = logs ? choicesFromWorkflowLogs(logs) : [];
statusValues.push({ label: '', value: '' });
statusValues.push({ label: ' Failed job', value: job.name });
if (steps.length > 0) {
const failedStep = steps.find(step => step.status === 'fail' || step.status === 'failed');
if (failedStep) {
const logs = failedStep.logLines?.map(line => line.msg).slice(-logLinesToKeep) ?? [];
statusValues.push({ label: ' Failed step', value: failedStep.name });
statusValues.push({
label: ' Logs for failed step',
value: '',
});
for (const log of logs) {
statusValues.push({ label: '', value: log });
}
const currentStep = steps[steps.length - 1];
statusValues.push({ label: ' Current step', value: currentStep.name });
if (currentStep.logLines?.length) {
statusValues.push({ label: ' Current logs', value: '' });
for (const log of stepLogTail(currentStep, maxLogLines)) {
statusValues.push({ label: '', value: log });
}
}
}
statusValues.push({ label: '', value: '' });
statusLines.push(formatFields(statusValues));
return statusLines.join('\n');
return formatFields(statusValues);
}
export function formatFailedWorkflowRun(
jobLogs: JobWithLogs[],
maxLogLines: number = -1 // -1 means no limit
): string {
const statusValues = [];
for (const { job, logs } of jobLogs) {
statusValues.push({ label: '', value: '' });
statusValues.push({ label: ' Failed job', value: job.name });
const steps = choicesFromWorkflowLogs(logs);
const failedStep = steps.find(step => step.status === 'fail' || step.status === 'failed');
if (!failedStep) {
continue;
}
statusValues.push({ label: ' Failed step', value: failedStep.name });
statusValues.push({ label: ' Logs for failed step', value: '' });
for (const log of stepLogTail(failedStep, maxLogLines)) {
statusValues.push({ label: '', value: log });
}
}
statusValues.push({ label: '', value: '' });
return formatFields(statusValues);
}
export async function fileExistsAsync(filePath: string): Promise<boolean> {
return await fs.promises
.access(filePath, fs.constants.F_OK)
@@ -312,65 +286,111 @@ export async function showWorkflowStatusAsync(
stream: spinnerUsesStdErr ? process.stderr : process.stdout,
text: '',
}).start();
spinner.prefixText = chalk`{bold.yellow Workflow run is waiting to start:}`;
updateSpinnerText(spinner, {
prefixText: chalk`{bold.yellow Workflow run is waiting to start:}`,
});
let failedFetchesCount = 0;
while (true) {
try {
const workflowRun = await WorkflowRunQuery.withJobsByIdAsync(graphqlClient, workflowRunId, {
useCache: false,
});
failedFetchesCount = 0;
switch (workflowRun.status) {
case WorkflowRunStatus.New:
break;
case WorkflowRunStatus.InProgress: {
spinner.prefixText = chalk`{bold.green Workflow run is in progress:}`;
spinner.text = await infoForActiveWorkflowRunAsync(graphqlClient, workflowRun, 5);
break;
}
case WorkflowRunStatus.ActionRequired:
spinner.prefixText = chalk`{bold.yellow Workflow run is waiting for action:}`;
break;
case WorkflowRunStatus.Canceled:
spinner.prefixText = chalk`{bold.yellow Workflow has been canceled.}`;
spinner.stopAndPersist();
return workflowRun;
case WorkflowRunStatus.Failure: {
spinner.prefixText = chalk`{bold.red Workflow has failed.}`;
const failedInfo = await infoForFailedWorkflowRunAsync(graphqlClient, workflowRun, 30);
spinner.fail(failedInfo);
return workflowRun;
}
case WorkflowRunStatus.Success:
spinner.prefixText = chalk`{bold.green Workflow has completed successfully.}`;
spinner.text = '';
spinner.succeed('');
return workflowRun;
}
if (!waitForCompletion) {
if (spinner.isSpinning) {
spinner.stopAndPersist();
}
return workflowRun;
}
} catch {
spinner.text = '⚠ Failed to fetch the workflow run status. Check your network connection.';
failedFetchesCount += 1;
if (failedFetchesCount > 6) {
spinner.fail('Failed to fetch the workflow run status 6 times in a row. Aborting wait.');
process.exit(workflowRunExitCodes.WAIT_ABORTED);
}
const watcher = new WorkflowRunLogsWatcher(
graphqlClient,
() => createRealtimeLogsClient(graphqlClient),
() => {
renderActiveWorkflowRun();
}
);
let failedFetchesCount = 0;
let renderActiveWorkflowRun = (): void => {};
await sleepAsync(10 /* seconds */ * 1000 /* milliseconds */);
try {
while (true) {
try {
const workflowRun = await WorkflowRunQuery.withJobsByIdAsync(graphqlClient, workflowRunId, {
useCache: false,
});
failedFetchesCount = 0;
switch (workflowRun.status) {
case WorkflowRunStatus.New:
break;
case WorkflowRunStatus.InProgress: {
updateSpinnerText(spinner, {
prefixText: chalk`{bold.green Workflow run is in progress:}`,
});
const logsStates = await watcher.syncJobsAsync(workflowRun.jobs);
for (const job of workflowRun.jobs) {
if (isJobCompleted(job.status)) {
nullthrows(
logsStates.get(job.id),
'syncJobsAsync must have been called before markCompleted'
).markCompleted();
}
}
renderActiveWorkflowRun = () => {
const text = formatActiveWorkflowRun(
workflowRun.jobs.map(job => ({
job,
logs: nullthrows(
logsStates.get(job.id),
'syncJobsAsync must have been called before getLogs'
).getLogs(),
}))
);
updateSpinnerText(spinner, { text });
};
renderActiveWorkflowRun();
break;
}
case WorkflowRunStatus.ActionRequired:
updateSpinnerText(spinner, {
prefixText: chalk`{bold.yellow Workflow run is waiting for action:}`,
});
break;
case WorkflowRunStatus.Canceled:
updateSpinnerText(spinner, {
prefixText: chalk`{bold.yellow Workflow has been canceled.}`,
});
spinner.stopAndPersist();
return workflowRun;
case WorkflowRunStatus.Failure: {
updateSpinnerText(spinner, {
prefixText: chalk`{bold.red Workflow has failed.}`,
});
const jobLogs = await logsForFailedWorkflowRunAsync(graphqlClient, workflowRun);
spinner.fail(formatFailedWorkflowRun(jobLogs, 30));
return workflowRun;
}
case WorkflowRunStatus.Success:
updateSpinnerText(spinner, {
prefixText: chalk`{bold.green Workflow has completed successfully.}`,
});
spinner.succeed('');
return workflowRun;
}
if (!waitForCompletion) {
if (spinner.isSpinning) {
spinner.stopAndPersist();
}
return workflowRun;
}
} catch {
updateSpinnerText(spinner, {
text: '⚠ Failed to fetch the workflow run status. Check your network connection.',
});
failedFetchesCount += 1;
if (failedFetchesCount > 6) {
spinner.fail('Failed to fetch the workflow run status 6 times in a row. Aborting wait.');
process.exit(workflowRunExitCodes.WAIT_ABORTED);
}
}
await sleepAsync(10 /* seconds */ * 1000 /* milliseconds */);
}
} finally {
watcher.close();
}
}
@@ -17,7 +17,6 @@ jest.mock('fs-extra', () => ({
jest.mock('../../../prompts', () => ({
promptAsync: jest.fn(),
}));
jest.mock('../../../onboarding/git');
jest.mock('../../../onboarding/runCommand');
jest.mock('../../../graphql/mutations/AppMutation');
jest.mock('../../../project/expoConfig');
+17 -3
View File
@@ -6,7 +6,7 @@ import { getProjectDashboardUrl } from '../../build/utils/url';
import EasCommand from '../../commandUtils/EasCommand';
import { ExpoGraphqlClient } from '../../commandUtils/context/contextUtils/createGraphqlClient';
import {
cloneTemplateAsync,
downloadTemplateAsync,
initializeGitRepositoryAsync,
installProjectDependenciesAsync,
} from '../../commandUtils/new/commands';
@@ -14,10 +14,12 @@ import {
generateProjectConfigAsync,
promptForProjectAccountAsync,
} from '../../commandUtils/new/configs';
import { resolveTemplateAsync } from '../../commandUtils/new/sdkVersion';
import {
copyProjectTemplatesAsync,
generateAppConfigAsync,
generateEasConfigAsync,
removeTemplateFilesAsync,
updatePackageJsonAsync,
updateReadmeAsync,
} from '../../commandUtils/new/projectFiles';
@@ -108,7 +110,9 @@ export async function generateProjectFilesAsync(
await generateEasConfigAsync(projectDir);
await updatePackageJsonAsync(projectDir);
await updatePackageJsonAsync(projectDir, app.slug);
await removeTemplateFilesAsync(projectDir);
await copyProjectTemplatesAsync(projectDir);
@@ -146,6 +150,9 @@ export default class New extends EasCommand {
options: [...PACKAGE_MANAGERS] as const,
default: 'npm',
})(),
'sdk-version': Flags.string({
description: 'Expo SDK version to use for the new project (e.g. 57, or "latest")',
}),
};
static override contextDefinition = {
@@ -167,13 +174,20 @@ export default class New extends EasCommand {
Log.log(`👋 Welcome to Expo, ${actor.username}!`);
// Resolve the template before any prompts when --sdk-version is provided,
// so an invalid version fails fast.
let template = flags['sdk-version']
? await resolveTemplateAsync({ sdkVersion: flags['sdk-version'] })
: null;
const {
projectName,
projectDirectory: targetProjectDirectory,
projectAccount,
} = await generateConfigsAsync(args, actor, graphqlClient);
const projectDirectory = await cloneTemplateAsync(targetProjectDirectory);
template ??= await resolveTemplateAsync({});
const projectDirectory = await downloadTemplateAsync(targetProjectDirectory, template);
const packageManager = flags['package-manager'];
await installProjectDependenciesAsync(projectDirectory, packageManager);
@@ -48,6 +48,7 @@ function makeDeviceRunSession(overrides: Partial<DeviceRunSessionById> = {}): De
return {
id: 'session-123',
name: null,
tags: [],
status: DeviceRunSessionStatus.InProgress,
type: DeviceRunSessionType.AgentDevice,
platform: AppPlatform.Ios,
@@ -122,7 +123,13 @@ describe(SimulatorGet, () => {
}
it('emits JSON when --json is passed', async () => {
const session = makeDeviceRunSession();
const session = makeDeviceRunSession({
type: DeviceRunSessionType.ServeSim,
remoteConfig: {
__typename: 'ServeSimRunSessionRemoteConfig',
previewUrl: 'https://preview.example.com',
},
});
mockByIdAsync.mockResolvedValue(session);
const { command, getContextAsync } = createCommand(['--id', 'session-123', '--json']);
@@ -137,7 +144,8 @@ describe(SimulatorGet, () => {
expect(mockPrintJsonOnlyOutput).toHaveBeenCalledWith({
id: 'session-123',
name: undefined,
type: 'agent-device',
tags: [],
type: 'web-preview-only',
status: DeviceRunSessionStatus.InProgress,
platform: AppPlatform.Ios,
createdAt: '2025-01-01T00:00:00.000Z',
@@ -6,6 +6,7 @@ import {
AppPlatform,
CreateDeviceRunSessionMutation,
DeviceRunSessionByIdQuery,
DeviceRunSessionResourceClass,
DeviceRunSessionStatus,
DeviceRunSessionType,
JobRunStatus,
@@ -109,6 +110,7 @@ function makeDeviceRunSession(overrides: Partial<DeviceRunSessionById> = {}): De
return {
id: 'session-123',
name: null,
tags: [],
status: DeviceRunSessionStatus.InProgress,
type: DeviceRunSessionType.AgentDevice,
platform: AppPlatform.Ios,
@@ -331,6 +333,33 @@ describe(Simulator, () => {
);
});
it('creates a WebPreviewOnly session for --type web-preview-only', async () => {
mockByIdAsync.mockResolvedValue(
makeDeviceRunSession({
type: DeviceRunSessionType.WebPreviewOnly,
remoteConfig: {
__typename: 'WebPreviewOnlyRunSessionRemoteConfig',
previewUrl: 'https://preview.example.test',
},
})
);
const { command } = createCommand([
'--platform',
'ios',
'--type',
'web-preview-only',
'--non-interactive',
]);
await command.runAsync();
expect(mockCreateDeviceRunSessionAsync).toHaveBeenCalledWith(
graphqlClient,
expect.objectContaining({ type: DeviceRunSessionType.WebPreviewOnly })
);
expect(Log.log).toHaveBeenCalledWith(expect.stringContaining('https://preview.example.test'));
});
it('overwrites .env.eas-simulator when outputting dotenv and the file exists', async () => {
const { command } = createCommand([
'--platform',
@@ -522,6 +551,35 @@ describe(Simulator, () => {
);
});
it('omits resourceClass when --resource-class is not set', async () => {
const { command } = createCommand(['--platform', 'ios', '--non-interactive']);
await command.runAsync();
expect(mockCreateDeviceRunSessionAsync.mock.calls[0][1]).not.toHaveProperty('resourceClass');
});
it.each([
['large', DeviceRunSessionResourceClass.Large],
['medium', DeviceRunSessionResourceClass.Medium],
] as const)(
'forwards --resource-class %s to the create mutation',
async (flag, resourceClass) => {
const { command } = createCommand([
'--platform',
'ios',
'--non-interactive',
'--resource-class',
flag,
]);
await command.runAsync();
expect(mockCreateDeviceRunSessionAsync).toHaveBeenCalledWith(
graphqlClient,
expect.objectContaining({ resourceClass })
);
}
);
it('forwards --build-id to the create mutation', async () => {
const { command } = createCommand([
'--platform',
@@ -556,6 +614,19 @@ describe(Simulator, () => {
);
});
it('warns that Android emulator support is still in development before creating a session', async () => {
const { command } = createCommand(['--platform', 'android', '--non-interactive']);
await command.runAsync();
expect(Log.warn).toHaveBeenCalledWith(
'Android emulator support in EAS Simulator is still in development. Some features available on iOS may not work on Android yet. Full parity with iOS is coming soon.'
);
expect(jest.mocked(Log.warn).mock.invocationCallOrder[0]).toBeLessThan(
mockCreateDeviceRunSessionAsync.mock.invocationCallOrder[0]
);
});
it.each([
['ios', AppPlatform.Ios],
['android', AppPlatform.Android],
@@ -639,6 +710,36 @@ describe(Simulator, () => {
);
});
it('forwards repeated tags', async () => {
const { command } = createCommand([
'--platform',
'ios',
'--non-interactive',
'--tag',
'variant:pro',
'--tag',
' nightly ',
]);
await command.runAsync();
expect(mockCreateDeviceRunSessionAsync).toHaveBeenCalledWith(
graphqlClient,
expect.objectContaining({ tags: ['variant:pro', 'nightly'] })
);
});
it('omits tags when every tag is blank', async () => {
const { command } = createCommand(['--platform', 'ios', '--non-interactive', '--tag', ' ']);
await command.runAsync();
expect(mockCreateDeviceRunSessionAsync).toHaveBeenCalledWith(
graphqlClient,
expect.not.objectContaining({ tags: expect.anything() })
);
});
it.each([
['--launch-arg', '--uitesting'],
['--open-url', 'exp://example.test'],
@@ -42,6 +42,7 @@ function makeSession(overrides: Partial<DeviceRunSessionNode> = {}): DeviceRunSe
return {
id: 'session-123',
name: null,
tags: [],
status: DeviceRunSessionStatus.InProgress,
type: DeviceRunSessionType.AgentDevice,
platform: AppPlatform.Ios,
@@ -111,7 +112,7 @@ describe(SimulatorList, () => {
}
it('emits JSON when --json is passed', async () => {
const session = makeSession();
const session = makeSession({ type: DeviceRunSessionType.ServeSim });
mockListByAppIdAsync.mockResolvedValue(makeConnection([session]));
const { command, getContextAsync } = createCommand(['--json']);
@@ -132,7 +133,8 @@ describe(SimulatorList, () => {
{
id: 'session-123',
name: undefined,
type: 'agent-device',
tags: [],
type: 'web-preview-only',
status: DeviceRunSessionStatus.InProgress,
platform: AppPlatform.Ios,
createdAt: '2025-01-01T00:00:00.000Z',
@@ -162,10 +164,16 @@ describe(SimulatorList, () => {
'new',
'--type',
'appium',
'--type',
'web-preview-only',
'--platform',
'ios',
'--name',
'checkout',
'--tag',
'variant:pro',
'--tag',
'nightly',
'--limit',
'25',
'--after',
@@ -179,13 +187,33 @@ describe(SimulatorList, () => {
after: 'page-cursor',
filter: {
statuses: [DeviceRunSessionStatus.InProgress, DeviceRunSessionStatus.New],
types: [DeviceRunSessionType.Appium],
types: [DeviceRunSessionType.Appium, DeviceRunSessionType.WebPreviewOnly],
platforms: [AppPlatform.Ios],
name: 'checkout',
tags: ['variant:pro', 'nightly'],
},
});
});
it('prints a tags row for every session, tagged or not', async () => {
mockListByAppIdAsync.mockResolvedValue(
makeConnection([
makeSession({ id: 'session-1', tags: ['variant:pro', 'nightly'] }),
makeSession({ id: 'session-2' }),
])
);
const { command } = createCommand([]);
await command.runAsync();
const printed = jest
.mocked(Log.log)
.mock.calls.map(([entry]) => String(entry))
.join('\n');
expect(printed).toContain('Tags: variant:pro, nightly');
expect(printed).toContain('Tags: none');
});
it('prints a name row for every session, named or not', async () => {
mockListByAppIdAsync.mockResolvedValue(
makeConnection([
@@ -87,6 +87,7 @@ export default class SimulatorGet extends EasCommand {
printJsonOnlyOutput({
id: session.id,
name: session.name ?? undefined,
tags: session.tags,
type: deviceRunSessionTypeToFlagValue(session.type),
status: session.status,
platform: session.platform,
@@ -123,6 +124,7 @@ function formatSessionFields(session: DeviceRunSessionById, deviceRunSessionUrl:
return formatFields([
{ label: 'ID', value: session.id },
{ label: 'Name', value: formatNullable(session.name) },
{ label: 'Tags', value: session.tags.length > 0 ? session.tags.join(', ') : 'none' },
{ label: 'Type', value: session.type },
{ label: 'Status', value: session.status },
{ label: 'Platform', value: session.platform },
@@ -30,6 +30,8 @@ import {
} from '../../simulator/env';
import { resolveExpoGoSdkVersionAsync } from '../../simulator/expoGo';
import {
DEVICE_RUN_SESSION_RESOURCE_CLASS_BY_FLAG_VALUE,
DEVICE_RUN_SESSION_RESOURCE_CLASS_FLAG_VALUES,
DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE,
DEVICE_RUN_SESSION_TYPE_FLAG_VALUES,
DeviceRunSessionRemoteConfig,
@@ -69,6 +71,11 @@ export default class Simulator extends EasCommand {
description:
'Human-readable name for the simulator session, shown in eas simulator:list and on expo.dev. Defaults to unnamed.',
}),
tag: Flags.string({
description:
'Label used to group simulator sessions, for example one per app variant. Repeat for multiple tags. Stored lowercased.',
multiple: true,
}),
device: Flags.string({
description:
'Virtual device to start for the session. On iOS, a Simulator device name or UDID (e.g. "iPhone 16 Pro"). On Android, an AVD hardware profile id (e.g. "pixel_7"). Defaults to a device chosen by the runner.',
@@ -101,7 +108,8 @@ export default class Simulator extends EasCommand {
'Expo or development-client URL to open in the installed application after it launches.',
}),
type: Flags.option({
description: 'Type of simulator session to create',
description:
'Type of simulator session to create. All session types include a web preview. agent-device, appium, and argent also include an automation interface; web-preview-only includes no automation interface.',
options: Object.values(DEVICE_RUN_SESSION_TYPE_FLAG_VALUES),
default: DEVICE_RUN_SESSION_TYPE_FLAG_VALUES[DeviceRunSessionType.AgentDevice],
})(),
@@ -119,6 +127,11 @@ export default class Simulator extends EasCommand {
'Stop the simulator session automatically after this many minutes without session activity. When omitted, the session has no idle timeout and runs until its maximum duration.',
min: 0,
}),
'resource-class': Flags.option({
description: 'The instance type that will be used to run this simulator session',
hidden: true,
options: Object.values(DEVICE_RUN_SESSION_RESOURCE_CLASS_FLAG_VALUES),
})(),
force: Flags.boolean({
description:
'[default: true] Create a new simulator session even when an existing simulator session is present in the environment.',
@@ -171,12 +184,16 @@ export default class Simulator extends EasCommand {
// The server rejects blank names, so trim here and treat a whitespace-only
// --name as if it had been omitted rather than surfacing a validation error.
const name = flags.name?.trim() || undefined;
const tags = flags.tag?.map(tag => tag.trim()).filter(tag => tag.length > 0);
const deviceIdentifier = flags.device?.trim() || undefined;
const buildId = flags['build-id']?.trim() || undefined;
const applicationArchiveUrlFromFlag = flags['application-archive-url']?.trim() || undefined;
const sdkVersionFromFlag = flags['sdk-version']?.trim() || undefined;
const launchArgs = flags['launch-arg'];
const openUrl = flags['open-url']?.trim() || undefined;
const resourceClass = flags['resource-class']
? DEVICE_RUN_SESSION_RESOURCE_CLASS_BY_FLAG_VALUE[flags['resource-class']]
: undefined;
if (sdkVersionFromFlag && !flags['expo-go']) {
throw new EasCommandError('The --sdk-version flag can only be used with --expo-go.');
@@ -201,6 +218,12 @@ export default class Simulator extends EasCommand {
}
const platform = await resolvePlatformAsync(flags.platform, nonInteractive);
if (platform === AppPlatform.Android) {
Log.warn(
'Android emulator support in EAS Simulator is still in development. Some features available on iOS may not work on Android yet. Full parity with iOS is coming soon.'
);
Log.newLine();
}
const expoGoSdkVersion = flags['expo-go']
? await resolveExpoGoSdkVersionAsync({ projectDir, sdkVersion: sdkVersionFromFlag })
: undefined;
@@ -222,6 +245,7 @@ export default class Simulator extends EasCommand {
const session = await DeviceRunSessionMutation.createDeviceRunSessionAsync(graphqlClient, {
appId: projectId,
name,
...(tags?.length ? { tags } : {}),
platform,
type: DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE[flags.type],
packageVersion: flags['package-version'],
@@ -233,6 +257,7 @@ export default class Simulator extends EasCommand {
...(expoGoSdkVersion ? { expoGo: true, sdkVersion: expoGoSdkVersion } : {}),
...(launchArgs?.length ? { launchArgs } : {}),
...(openUrl ? { openUrl } : {}),
...(resourceClass ? { resourceClass } : {}),
maxRunTimeMinutes: flags['max-duration-minutes'],
maxIdleTimeMinutes: flags['max-idle-time-minutes'],
});
@@ -61,7 +61,8 @@ export default class SimulatorList extends EasCommand {
multiple: true,
})(),
type: Flags.option({
description: 'Filter by session type (repeatable)',
description:
'Filter by session type (repeatable). All session types include a web preview. agent-device, appium, and argent also include an automation interface; web-preview-only includes no automation interface.',
options: Object.values(DEVICE_RUN_SESSION_TYPE_FLAG_VALUES),
multiple: true,
})(),
@@ -73,6 +74,10 @@ export default class SimulatorList extends EasCommand {
name: Flags.string({
description: 'Filter by session name (case-insensitive prefix match)',
}),
tag: Flags.string({
description: 'Filter by tag (repeatable). A session must carry every tag listed.',
multiple: true,
}),
limit: getLimitFlagWithCustomValues({ defaultTo: DEFAULT_LIMIT, limit: MAX_LIMIT }),
after: Flags.string({
description:
@@ -114,6 +119,9 @@ export default class SimulatorList extends EasCommand {
if (flags.name) {
filter.name = flags.name;
}
if (flags.tag && flags.tag.length > 0) {
filter.tags = flags.tag;
}
const limit = flags.limit ?? DEFAULT_LIMIT;
@@ -139,6 +147,7 @@ export default class SimulatorList extends EasCommand {
sessions: sessions.map(session => ({
id: session.id,
name: session.name ?? undefined,
tags: session.tags,
type: deviceRunSessionTypeToFlagValue(session.type),
status: session.status,
platform: session.platform,
@@ -172,6 +181,7 @@ export default class SimulatorList extends EasCommand {
const lines = [
`ID: ${session.id}`,
`Name: ${session.name ?? 'null'}`,
`Tags: ${session.tags.length > 0 ? session.tags.join(', ') : 'none'}`,
`Type: ${session.type}`,
`Status: ${session.status}`,
`Platform: ${session.platform}`,
@@ -0,0 +1,171 @@
import spawnAsync from '@expo/spawn-async';
import { Config } from '@oclif/core';
import { fileExistsAsync, maybeReadStdinAsync } from '../../../commandUtils/workflow/utils';
import { WorkflowRevisionMutation } from '../../../graphql/mutations/WorkflowRevisionMutation';
import { WorkflowRunMutation } from '../../../graphql/mutations/WorkflowRunMutation';
import Log from '../../../log';
import { getOwnerAccountForProjectIdAsync } from '../../../project/projectUtils';
import { uploadAccountScopedFileAsync } from '../../../project/uploadAccountScopedFileAsync';
import { uploadAccountScopedProjectSourceAsync } from '../../../project/uploadAccountScopedProjectSourceAsync';
import { WorkflowFile } from '../../../utils/workflowFile';
import WorkflowRun from '../run';
jest.mock('@expo/spawn-async', () => ({ __esModule: true, default: jest.fn() }));
jest.mock('../../../graphql/mutations/WorkflowRevisionMutation');
jest.mock('../../../graphql/mutations/WorkflowRunMutation');
jest.mock('../../../project/projectUtils');
jest.mock('../../../project/uploadAccountScopedFileAsync');
jest.mock('../../../project/uploadAccountScopedProjectSourceAsync');
jest.mock('../../../utils/workflowFile');
jest.mock('../../../commandUtils/workflow/utils', () => ({
...jest.requireActual('../../../commandUtils/workflow/utils'),
fileExistsAsync: jest.fn(),
maybeReadStdinAsync: jest.fn(),
}));
jest.mock('../../../log', () => ({
__esModule: true,
default: {
log: jest.fn(),
newLine: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
debug: jest.fn(),
},
link: jest.fn((url: string) => url),
}));
const mockSpawn = jest.mocked(spawnAsync);
const mockValidate = jest.mocked(WorkflowRevisionMutation.validateWorkflowYamlConfigAsync);
const mockGetOrCreateRevision = jest.mocked(
WorkflowRevisionMutation.getOrCreateWorkflowRevisionFromGitRefAsync
);
const mockCreateFromGitRef = jest.mocked(WorkflowRunMutation.createWorkflowRunFromGitRefAsync);
const mockCreateRun = jest.mocked(WorkflowRunMutation.createWorkflowRunAsync);
const mockGetOwner = jest.mocked(getOwnerAccountForProjectIdAsync);
const mockUploadProject = jest.mocked(uploadAccountScopedProjectSourceAsync);
const mockUploadFile = jest.mocked(uploadAccountScopedFileAsync);
const mockReadWorkflow = jest.mocked(WorkflowFile.readWorkflowFileContentsAsync);
const mockFileExists = jest.mocked(fileExistsAsync);
const mockReadStdin = jest.mocked(maybeReadStdinAsync);
describe('WorkflowRun --ssh', () => {
let mockConfig: Config;
let exitSpy: jest.SpyInstance;
beforeAll(async () => {
mockConfig = new Config({ root: __dirname });
mockConfig.runHook = async () => ({ failures: [], successes: [] });
});
beforeEach(() => {
jest.clearAllMocks();
exitSpy = jest.spyOn(process, 'exit').mockImplementation(((code?: number) => {
throw new Error(`process.exit:${code ?? 0}`);
}) as never);
mockReadStdin.mockResolvedValue(null);
mockGetOwner.mockResolvedValue({ id: 'acct-1', name: 'testuser' } as never);
mockValidate.mockResolvedValue(undefined as never);
mockReadWorkflow.mockResolvedValue({
filePath: '.eas/workflows/test.yml',
yamlConfig: 'jobs:\n a:\n steps: []\n',
} as never);
mockFileExists.mockResolvedValue(false);
mockUploadProject.mockResolvedValue({ projectArchiveBucketKey: 'archive-key' } as never);
mockUploadFile.mockResolvedValue({ fileBucketKey: 'file-key' } as never);
mockCreateRun.mockResolvedValue({ id: 'run-1' } as never);
mockCreateFromGitRef.mockResolvedValue({ id: 'run-git-1' } as never);
mockSpawn.mockResolvedValue({ output: ['abc123\n'] } as never);
mockGetOrCreateRevision.mockResolvedValue({
id: 'rev-1',
yamlConfig: 'jobs:\n a:\n steps: []\n',
} as never);
});
afterEach(() => {
exitSpy.mockRestore();
});
function createCommand(flags: {
ssh?: boolean;
'ssh-idle-timeout'?: number;
ref?: string;
}): WorkflowRun {
const command = new WorkflowRun(['.eas/workflows/test.yml'], mockConfig);
// @ts-expect-error getContextAsync/parse are protected
jest.spyOn(command, 'getContextAsync').mockResolvedValue({
getDynamicPrivateProjectConfigAsync: async () => ({
projectId: 'app-1',
exp: { slug: 'testapp' },
}),
loggedIn: { graphqlClient: {} },
vcsClient: { getRootPathAsync: async () => '/test/project' },
projectDir: '/test/project',
});
// @ts-expect-error parse is protected on the oclif base
jest.spyOn(command, 'parse').mockResolvedValue({
flags: {
json: false,
wait: false,
'non-interactive': true,
input: undefined,
ref: flags.ref,
ssh: flags.ssh ?? false,
'ssh-idle-timeout': flags['ssh-idle-timeout'],
},
args: { file: '.eas/workflows/test.yml' },
});
return command;
}
it('passes ssh input into createWorkflowRunAsync and logs that SSH is enabled', async () => {
await expect(createCommand({ ssh: true, 'ssh-idle-timeout': 120 }).runAsync()).rejects.toThrow(
'process.exit:0'
);
expect(mockCreateRun).toHaveBeenCalledWith(
{},
expect.objectContaining({
workflowRunInput: expect.objectContaining({
ssh: { idleTimeoutSeconds: 120 },
}),
})
);
expect(Log.log).toHaveBeenCalledWith(
'SSH enabled. Each VM job logs the `eas workflow:ssh` command to connect.'
);
expect(exitSpy).toHaveBeenCalledWith(0);
});
it('does not log the SSH banner when --ssh is unset', async () => {
await expect(createCommand({ ssh: false }).runAsync()).rejects.toThrow('process.exit:0');
expect(mockCreateRun).toHaveBeenCalledWith(
{},
expect.objectContaining({
workflowRunInput: expect.objectContaining({
ssh: null,
}),
})
);
expect(Log.log).not.toHaveBeenCalledWith(
'SSH enabled. Each VM job logs the `eas workflow:ssh` command to connect.'
);
});
it('passes ssh input into createWorkflowRunFromGitRefAsync when --ref is set', async () => {
await expect(createCommand({ ssh: true, ref: 'main' }).runAsync()).rejects.toThrow(
'process.exit:0'
);
expect(mockCreateFromGitRef).toHaveBeenCalledWith(
{},
expect.objectContaining({
ssh: { idleTimeoutSeconds: undefined },
})
);
expect(Log.log).toHaveBeenCalledWith(
'SSH enabled. Each VM job logs the `eas workflow:ssh` command to connect.'
);
});
});
@@ -3,6 +3,31 @@ import {
parseJsonInputs,
parseWorkflowInputsFromYaml,
} from '../../../commandUtils/workflow/inputs';
import { resolveWorkflowRunSshInput } from '../run';
describe(resolveWorkflowRunSshInput, () => {
it('is null when --ssh is not set', () => {
expect(resolveWorkflowRunSshInput({ ssh: false, idleTimeoutSeconds: undefined })).toBeNull();
});
it('forwards ssh with an undefined timeout when --ssh is set alone', () => {
expect(resolveWorkflowRunSshInput({ ssh: true, idleTimeoutSeconds: undefined })).toEqual({
idleTimeoutSeconds: undefined,
});
});
it('forwards the requested idle timeout when both flags are set', () => {
expect(resolveWorkflowRunSshInput({ ssh: true, idleTimeoutSeconds: 60 })).toEqual({
idleTimeoutSeconds: 60,
});
});
it('rejects --ssh-idle-timeout without --ssh', () => {
expect(() => resolveWorkflowRunSshInput({ ssh: false, idleTimeoutSeconds: 60 })).toThrow(
'--ssh-idle-timeout requires --ssh.'
);
});
});
describe('parseInputs', () => {
it('should parse single key=value pair', () => {
@@ -0,0 +1,387 @@
import spawnAsync from '@expo/spawn-async';
import { Config } from '@oclif/core';
import fs from 'node:fs';
import { WorkflowJobSshQuery } from '../../../graphql/queries/WorkflowJobSshQuery';
import Log from '../../../log';
import { sleepAsync } from '../../../utils/promise';
import WorkflowSsh, {
parseSshArgv,
resolveSshConnectStatus,
splitConnectionHost,
sshHostAliasForResource,
terminalSshStatusMessage,
} from '../ssh';
jest.mock('@expo/spawn-async', () => ({ __esModule: true, default: jest.fn() }));
jest.mock('../../../graphql/queries/WorkflowJobSshQuery');
jest.mock('../../../log', () => ({
__esModule: true,
default: { log: jest.fn(), error: jest.fn(), newLine: jest.fn() },
}));
jest.mock('../../../ora', () => ({
ora: jest.fn(() => {
const spinner = { start: jest.fn(), fail: jest.fn(), succeed: jest.fn(), stop: jest.fn() };
spinner.start.mockReturnValue(spinner);
return spinner;
}),
}));
jest.mock('../../../utils/promise', () => ({ sleepAsync: jest.fn() }));
jest.mock('node:fs', () => {
const actual = jest.requireActual('node:fs');
return {
...actual,
promises: { ...actual.promises, mkdtemp: jest.fn(), writeFile: jest.fn(), rm: jest.fn() },
};
});
describe(parseSshArgv, () => {
it('takes the first token as the resource id and the rest as the passthrough command', () => {
expect(parseSshArgv(['job-id', 'ls', '-la'])).toEqual({
showConnect: false,
resourceId: 'job-id',
command: ['ls', '-la'],
});
});
it('detects --show-connect before the id and strips it', () => {
expect(parseSshArgv(['--show-connect', 'job-id'])).toEqual({
showConnect: true,
resourceId: 'job-id',
command: [],
});
});
it('detects --show-connect=true before the id', () => {
expect(parseSshArgv(['--show-connect=true', 'job-id'])).toEqual({
showConnect: true,
resourceId: 'job-id',
command: [],
});
});
it('leaves --show-connect after the id in the passthrough command', () => {
expect(parseSshArgv(['job-id', '--show-connect'])).toEqual({
showConnect: false,
resourceId: 'job-id',
command: ['--show-connect'],
});
});
it('throws on an unknown flag before the id', () => {
expect(() => parseSshArgv(['--show-conect', 'job-id'])).toThrow('Unknown flag');
});
it('reports an undefined resource id when none is given', () => {
expect(parseSshArgv([])).toEqual({ showConnect: false, resourceId: undefined, command: [] });
expect(parseSshArgv(['--show-connect'])).toEqual({
showConnect: true,
resourceId: undefined,
command: [],
});
});
});
describe(resolveSshConnectStatus, () => {
it('is unknown when the resource did not resolve', () => {
expect(resolveSshConnectStatus(null)).toBe('unknown');
});
it('is not-enabled when the job did not request ssh', () => {
expect(
resolveSshConnectStatus({ sshRequested: false, jobCompleted: false, session: null })
).toBe('not-enabled');
});
it('is ended when ssh was requested, the job finished, and no session remains', () => {
expect(resolveSshConnectStatus({ sshRequested: true, jobCompleted: true, session: null })).toBe(
'ended'
);
});
it('is pending when ssh was requested and the job is still running without a session', () => {
expect(
resolveSshConnectStatus({ sshRequested: true, jobCompleted: false, session: null })
).toBe('pending');
});
it('is pending while the worker is reconnecting', () => {
expect(
resolveSshConnectStatus({
sshRequested: true,
jobCompleted: false,
session: {
connectionConfig: { host: 'uptermd.upterm.dev', secret: 'tok', reconnecting: true },
},
})
).toBe('pending');
});
it('is ready when the session has a connection config', () => {
expect(
resolveSshConnectStatus({
sshRequested: true,
jobCompleted: false,
session: {
connectionConfig: { host: 'uptermd.upterm.dev', secret: 'tok', reconnecting: false },
},
})
).toBe('ready');
});
});
describe(splitConnectionHost, () => {
it('returns the host with no port for a plain hostname', () => {
expect(splitConnectionHost('relay.expo.dev')).toEqual({ hostname: 'relay.expo.dev' });
});
it('splits a host:port into host and numeric port', () => {
expect(splitConnectionHost('relay.expo.dev:8022')).toEqual({
hostname: 'relay.expo.dev',
port: 8022,
});
});
});
describe(terminalSshStatusMessage, () => {
it('has no message while the session can still open', () => {
expect(terminalSshStatusMessage('pending', 'job-1')).toBeNull();
expect(terminalSshStatusMessage('ready', 'job-1')).toBeNull();
});
it('distinguishes not-enabled from ended', () => {
expect(terminalSshStatusMessage('not-enabled', 'job-1')).toMatch(/SSH was not enabled/);
expect(terminalSshStatusMessage('ended', 'job-1')).toMatch(/has ended/);
expect(terminalSshStatusMessage('unknown', 'job-1')).toMatch(/No workflow job found/);
});
});
describe(sshHostAliasForResource, () => {
it('suffixes the alias with the start of the resource id', () => {
expect(sshHostAliasForResource('job-1')).toBe('eas-workflow-ssh-job1');
});
it('drops characters that are not safe in an ssh config host alias', () => {
expect(sshHostAliasForResource('a b\nHost evil')).toBe('eas-workflow-ssh-abHostev');
});
it('falls back to the bare alias when nothing usable remains', () => {
expect(sshHostAliasForResource('---')).toBe('eas-workflow-ssh');
});
});
describe(WorkflowSsh, () => {
const graphqlClient = {} as never;
const mockConnectInfo = jest.mocked(WorkflowJobSshQuery.connectInfoForResourceIdAsync);
const mockSpawn = jest.mocked(spawnAsync);
const mockSleep = jest.mocked(sleepAsync);
const mockMkdtemp = jest.mocked(fs.promises.mkdtemp);
const mockWriteFile = jest.mocked(fs.promises.writeFile);
const mockRm = jest.mocked(fs.promises.rm);
let mockConfig: Config;
let previousExitCode: typeof process.exitCode;
const readyInfo = {
sshRequested: true,
jobCompleted: false,
session: {
id: 'ts-1',
connectionConfig: { host: 'relay.expo.dev', secret: 'TOKENx', reconnecting: false },
},
};
const pendingInfo = { sshRequested: true, jobCompleted: false, session: null };
const endedInfo = { sshRequested: true, jobCompleted: true, session: null };
const notEnabledInfo = { sshRequested: false, jobCompleted: false, session: null };
beforeAll(async () => {
mockConfig = new Config({ root: __dirname });
mockConfig.runHook = async () => ({ failures: [], successes: [] });
});
beforeEach(() => {
jest.clearAllMocks();
previousExitCode = process.exitCode;
process.exitCode = undefined;
mockMkdtemp.mockResolvedValue('/tmp/eas-ssh-1' as never);
mockWriteFile.mockResolvedValue(undefined as never);
mockRm.mockResolvedValue(undefined as never);
mockSpawn.mockResolvedValue({ stdout: '', stderr: '' } as never);
mockSleep.mockResolvedValue(undefined);
});
afterEach(() => {
process.exitCode = previousExitCode;
});
function createCommand(argv: string[]): WorkflowSsh {
const command = new WorkflowSsh(argv, mockConfig);
// @ts-expect-error getContextAsync/parse are protected
jest.spyOn(command, 'getContextAsync').mockResolvedValue({ loggedIn: { graphqlClient } });
// @ts-expect-error parse is protected on the oclif base
jest.spyOn(command, 'parse').mockResolvedValue({});
return command;
}
it('throws when no workflow job id is given', async () => {
await expect(createCommand([]).runAsync()).rejects.toThrow('Provide a workflow job id');
});
it('reports unknown and exits 1 when the id does not resolve', async () => {
mockConnectInfo.mockResolvedValue(null);
await createCommand(['job-1']).runAsync();
expect(Log.error).toHaveBeenCalledWith(expect.stringContaining('No workflow job found'));
expect(process.exitCode).toBe(1);
});
it('reports not-enabled and exits 1 when the job did not request ssh', async () => {
mockConnectInfo.mockResolvedValue(notEnabledInfo as never);
await createCommand(['job-1']).runAsync();
expect(Log.error).toHaveBeenCalledWith(expect.stringContaining('was not enabled'));
expect(process.exitCode).toBe(1);
});
it('reports ended and exits 1 when the job finished without a live session', async () => {
mockConnectInfo.mockResolvedValue(endedInfo as never);
await createCommand(['job-1']).runAsync();
expect(Log.error).toHaveBeenCalledWith(expect.stringContaining('has ended'));
expect(process.exitCode).toBe(1);
});
it('prints the connect command for --show-connect without spawning ssh', async () => {
mockConnectInfo.mockResolvedValue(readyInfo as never);
await createCommand(['--show-connect', 'job-1']).runAsync();
expect(Log.log).toHaveBeenCalledWith(
'ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null [email protected]'
);
expect(mockSpawn).not.toHaveBeenCalled();
});
it('prints --show-connect with -p and a hostname-only wss proxy for non-default SSH ports', async () => {
mockConnectInfo.mockResolvedValue({
...readyInfo,
session: {
id: 'ts-1',
connectionConfig: {
host: 'relay.expo.dev:8022',
secret: 'TOKENx',
reconnecting: false,
},
},
} as never);
await createCommand(['--show-connect', 'job-1']).runAsync();
expect(Log.log).toHaveBeenCalledWith(
'ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -p 8022 [email protected]'
);
expect(Log.log).toHaveBeenCalledWith(
' ssh -o ProxyCommand="upterm proxy wss://[email protected]" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -p 8022 [email protected]'
);
});
it('writes a 0600 config and opens ssh, then removes the config dir', async () => {
mockConnectInfo.mockResolvedValue(readyInfo as never);
await createCommand(['job-1', 'ls', '-la']).runAsync();
expect(mockWriteFile).toHaveBeenCalledWith(
'/tmp/eas-ssh-1/config',
expect.stringContaining('HostName relay.expo.dev'),
{ mode: 0o600 }
);
expect(mockWriteFile).toHaveBeenCalledWith(
'/tmp/eas-ssh-1/config',
expect.stringContaining('Host eas-workflow-ssh-job1'),
{ mode: 0o600 }
);
expect(mockSpawn).toHaveBeenCalledWith(
'ssh',
['-F', '/tmp/eas-ssh-1/config', 'eas-workflow-ssh-job1', 'ls', '-la'],
{ stdio: 'inherit' }
);
expect(mockRm).toHaveBeenCalledWith('/tmp/eas-ssh-1', { recursive: true, force: true });
});
it('waits for a pending session to open, then connects', async () => {
mockConnectInfo
.mockResolvedValueOnce(pendingInfo as never) // initial status check
.mockResolvedValueOnce(pendingInfo as never) // first wait poll
.mockResolvedValue(readyInfo as never);
await createCommand(['job-1']).runAsync();
expect(mockSleep).toHaveBeenCalled();
expect(mockSpawn).toHaveBeenCalled();
});
it('exits 1 when a pending session ends before it opens', async () => {
mockConnectInfo
.mockResolvedValueOnce(pendingInfo as never)
.mockResolvedValue(endedInfo as never);
await createCommand(['job-1']).runAsync();
expect(mockSpawn).not.toHaveBeenCalled();
expect(process.exitCode).toBe(1);
});
it('exits 1 when waiting for the session to open times out', async () => {
jest.useFakeTimers();
jest.setSystemTime(0);
try {
mockConnectInfo.mockResolvedValue(pendingInfo as never);
mockSleep.mockImplementation(async () => {
jest.setSystemTime(Date.now() + 6 * 60 * 1000);
});
await createCommand(['job-1']).runAsync();
expect(mockSpawn).not.toHaveBeenCalled();
expect(process.exitCode).toBe(1);
} finally {
jest.useRealTimers();
}
});
it('throws on an unexpected connection host', async () => {
mockConnectInfo.mockResolvedValue({
sshRequested: true,
jobCompleted: false,
session: {
connectionConfig: { host: 'bad host', secret: 'TOKENx', reconnecting: false },
},
} as never);
await expect(createCommand(['job-1']).runAsync()).rejects.toThrow('connection host');
});
it('propagates the ssh exit code from the catch handler', async () => {
const command = createCommand(['job-1']);
// @ts-expect-error isRunningSubprocess is private
command.isRunningSubprocess = true;
const err = Object.assign(new Error('ssh exited'), { status: 42 });
// @ts-expect-error catch is protected
await command.catch(err);
expect(process.exitCode).toBe(42);
});
it('reports a missing ssh client (ENOENT) from the catch handler', async () => {
const command = createCommand(['job-1']);
// @ts-expect-error isRunningSubprocess is private
command.isRunningSubprocess = true;
const err = Object.assign(new Error('spawn ssh ENOENT'), { code: 'ENOENT' });
// @ts-expect-error catch is protected
await command.catch(err);
expect(process.exitCode).toBe(1);
expect(Log.error).toHaveBeenCalledWith(expect.stringContaining('Install an OpenSSH client'));
});
it('delegates non-subprocess errors to the base handler', async () => {
const command = createCommand(['job-1']);
let threw = false;
try {
// @ts-expect-error catch is protected
await command.catch(new Error('boom'));
} catch {
threw = true;
}
expect(threw).toBe(true);
});
it('stops the spinner and rethrows when polling fails while waiting', async () => {
mockConnectInfo
.mockResolvedValueOnce(pendingInfo as never)
.mockRejectedValue(new Error('network blip'));
await expect(createCommand(['job-1']).runAsync()).rejects.toThrow('network blip');
});
});
@@ -78,6 +78,19 @@ import { uploadAccountScopedProjectSourceAsync } from '../../project/uploadAccou
import { enableJsonOutput, printJsonOnlyOutput } from '../../utils/json';
import { WorkflowFile } from '../../utils/workflowFile';
export function resolveWorkflowRunSshInput({
ssh,
idleTimeoutSeconds,
}: {
ssh: boolean;
idleTimeoutSeconds: number | undefined;
}): { idleTimeoutSeconds: number | undefined } | null {
if (idleTimeoutSeconds !== undefined && !ssh) {
throw new Error('--ssh-idle-timeout requires --ssh.');
}
return ssh ? { idleTimeoutSeconds } : null;
}
export default class WorkflowRun extends EasCommand {
static override description =
'run an EAS workflow. The entire local project directory will be packaged and uploaded to EAS servers for the workflow run, unless the --ref flag is used.';
@@ -110,6 +123,18 @@ export default class WorkflowRun extends EasCommand {
"The git reference must exist in the project's git repository, and the workflow file must exist at that reference. When this flag is used, the local project is not uploaded; instead, the workflow is run from the exact state of the project at the chosen reference.",
summary: 'Git reference to run the workflow on',
}),
ssh: Flags.boolean({
default: false,
description: 'Open an SSH session on each VM job for live debugging with `eas workflow:ssh`.',
summary: '[EXPERIMENTAL] Enable SSH on the run',
}),
'ssh-idle-timeout': Flags.integer({
description:
'Seconds to keep an SSH session open after its job finishes, while no client is connected. The session is always available for the whole job. Requires --ssh. Must be between 0 and 3600; defaults to 0 (close as soon as the job finishes and nobody is connected).',
summary: '[EXPERIMENTAL] SSH idle timeout in seconds',
min: 0,
max: 3600,
}),
...EasJsonOnlyFlag,
};
@@ -127,6 +152,11 @@ export default class WorkflowRun extends EasCommand {
enableJsonOutput();
}
const sshInput = resolveWorkflowRunSshInput({
ssh: flags.ssh,
idleTimeoutSeconds: flags['ssh-idle-timeout'],
});
const {
getDynamicPrivateProjectConfigAsync,
loggedIn: { graphqlClient },
@@ -272,6 +302,7 @@ export default class WorkflowRun extends EasCommand {
workflowRevisionId: workflowRevisionId ?? '',
gitRef,
inputs,
ssh: sshInput,
});
} catch (err) {
throw new Error(`Failed to create workflow run: ${err}`);
@@ -341,6 +372,7 @@ export default class WorkflowRun extends EasCommand {
packageJsonBucketKey,
projectRootDirectory,
},
ssh: sshInput,
},
}));
} catch (err) {
@@ -353,6 +385,10 @@ export default class WorkflowRun extends EasCommand {
Log.newLine();
Log.log(`See logs: ${link(getWorkflowRunUrl(account.name, projectName, workflowRunId))}`);
if (sshInput) {
Log.log('SSH enabled. Each VM job logs the `eas workflow:ssh` command to connect.');
}
if (!flags.wait) {
if (flags.json) {
printJsonOnlyOutput({
@@ -0,0 +1,252 @@
import spawnAsync from '@expo/spawn-async';
import { Flags } from '@oclif/core';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import EasCommand from '../../commandUtils/EasCommand';
import { ExpoGraphqlClient } from '../../commandUtils/context/contextUtils/createGraphqlClient';
import {
WorkflowJobSshConnectInfo,
WorkflowJobSshQuery,
WorkflowJobSshSession,
} from '../../graphql/queries/WorkflowJobSshQuery';
import Log from '../../log';
import { ora } from '../../ora';
import { sleepAsync } from '../../utils/promise';
export function splitConnectionHost(connectionHost: string): { hostname: string; port?: number } {
try {
const { hostname, port: stringPort } = new URL(`ssh://${connectionHost}`);
const port = stringPort === '' ? undefined : Number(stringPort);
return { hostname, port };
} catch (err) {
throw new Error(
'Unexpected connection host reported for this ssh session. Update eas-cli and try again, or contact support if it persists.',
{ cause: err }
);
}
}
const SSH_INSECURE_OPTS = '-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null';
const SESSION_OPEN_TIMEOUT_MS = 5 * 60 * 1000;
const SESSION_OPEN_POLL_INTERVAL_MS = 3000;
export type SshConnectStatus = 'unknown' | 'not-enabled' | 'ended' | 'pending' | 'ready';
export function resolveSshConnectStatus(
connectInfo:
| (Pick<WorkflowJobSshConnectInfo, 'sshRequested' | 'jobCompleted'> & {
session?: Pick<WorkflowJobSshSession, 'connectionConfig'> | null;
})
| null
): SshConnectStatus {
if (!connectInfo) {
return 'unknown';
}
const { sshRequested, jobCompleted, session } = connectInfo;
if (session?.connectionConfig && !session.connectionConfig.reconnecting) {
return 'ready';
}
if (session?.connectionConfig?.reconnecting) {
return 'pending';
}
if (!sshRequested) {
return 'not-enabled';
}
return jobCompleted ? 'ended' : 'pending';
}
export function terminalSshStatusMessage(
status: SshConnectStatus,
resourceId: string
): string | null {
switch (status) {
case 'unknown':
return `No workflow job found for "${resourceId}". Pass a workflow job id from a run started with \`eas workflow:run --ssh\`.`;
case 'not-enabled':
return `SSH was not enabled for "${resourceId}". Start the run with \`eas workflow:run --ssh\` to enable it.`;
case 'ended':
return 'This ssh session has ended.';
case 'pending':
case 'ready':
return null;
}
}
export function sshHostAliasForResource(resourceId: string): string {
const suffix = resourceId.replace(/[^A-Za-z0-9]/g, '').slice(0, 8);
return suffix ? `eas-workflow-ssh-${suffix}` : 'eas-workflow-ssh';
}
export function parseSshArgv(rawArgv: readonly string[]): {
showConnect: boolean;
resourceId: string | undefined;
command: string[];
} {
let showConnect = false;
let index = 0;
while (index < rawArgv.length && rawArgv[index].startsWith('-')) {
if (rawArgv[index] === '--show-connect' || rawArgv[index] === '--show-connect=true') {
showConnect = true;
} else {
throw new Error(
`Unknown flag "${rawArgv[index]}" before the id. The only supported flag is --show-connect; everything after the id is passed to the remote shell.`
);
}
index += 1;
}
const [resourceId, ...command] = rawArgv.slice(index);
return { showConnect, resourceId, command };
}
export default class WorkflowSsh extends EasCommand {
static override hidden = true;
static override description =
'[EXPERIMENTAL] open an ssh session on the worker running a workflow job';
static override strict = false;
static override flags = {
'show-connect': Flags.boolean({
description: 'Print the ssh connection command (host and token) instead of opening a session',
default: false,
}),
};
static override contextDefinition = {
...this.ContextOptions.LoggedIn,
};
private isRunningSubprocess = false;
async runAsync(): Promise<void> {
const rawArgv = [...this.argv];
await this.parse(WorkflowSsh, []);
const { showConnect, resourceId, command } = parseSshArgv(rawArgv);
if (typeof resourceId !== 'string' || resourceId.length === 0) {
throw new Error('Provide a workflow job id: eas workflow:ssh <workflow-job-id> [command...]');
}
const {
loggedIn: { graphqlClient },
} = await this.getContextAsync(WorkflowSsh, { nonInteractive: true });
const connectInfo = await WorkflowJobSshQuery.connectInfoForResourceIdAsync(
graphqlClient,
resourceId
);
const status = resolveSshConnectStatus(connectInfo);
const statusMessage = terminalSshStatusMessage(status, resourceId);
if (statusMessage) {
Log.error(statusMessage);
process.exitCode = 1;
return;
}
const connectionConfig =
(status === 'ready' ? connectInfo?.session?.connectionConfig : null) ??
(await waitForSessionToOpenAsync(graphqlClient, resourceId));
if (!connectionConfig) {
process.exitCode = 1;
return;
}
const { host: connectionHost, secret } = connectionConfig;
const { hostname, port } = splitConnectionHost(connectionHost);
const portOption = port !== undefined ? ` -p ${port}` : '';
if (showConnect) {
Log.log(`ssh ${SSH_INSECURE_OPTS}${portOption} ${secret}@${hostname}`);
Log.newLine();
Log.log(
'If your network blocks the direct SSH connection, reach the session through the WebSocket relay with upterm (https://upterm.dev):'
);
// WSS terminates on the relay hostname (default 443). Do not paste an SSH
// :port into the wss:// URL — that port is only for the ssh destination.
Log.log(
` ssh -o ProxyCommand="upterm proxy wss://${secret}@${hostname}" ${SSH_INSECURE_OPTS}${portOption} ${secret}@${hostname}`
);
return;
}
const configDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'eas-workflow-ssh-'));
try {
const configPath = path.join(configDir, 'config');
const hostAlias = sshHostAliasForResource(resourceId);
await fs.promises.writeFile(
configPath,
[
`Host ${hostAlias}`,
` HostName ${hostname}`,
...(port !== undefined ? [` Port ${port}`] : []),
` User ${secret}`,
' StrictHostKeyChecking no',
' UserKnownHostsFile /dev/null',
'',
].join('\n'),
{ mode: 0o600 }
);
this.isRunningSubprocess = true;
await spawnAsync('ssh', ['-F', configPath, hostAlias, ...command], {
stdio: 'inherit',
});
} finally {
await fs.promises.rm(configDir, { recursive: true, force: true });
}
}
protected override catch(err: Error): Promise<void> {
if (this.isRunningSubprocess) {
if ((err as Error & { code?: string }).code === 'ENOENT') {
Log.error(
'Could not run `ssh`. Install an OpenSSH client and make sure `ssh` is on your PATH, then try again.'
);
process.exitCode = 1;
return Promise.resolve();
}
const status = (err as Error & { status?: number | null }).status;
process.exitCode = process.exitCode ?? status ?? 1;
return Promise.resolve();
}
return super.catch(err);
}
}
async function waitForSessionToOpenAsync(
graphqlClient: ExpoGraphqlClient,
workflowJobId: string
): Promise<NonNullable<WorkflowJobSshSession['connectionConfig']> | null> {
const spinner = ora('Waiting for the worker to open the ssh session').start();
const deadline = Date.now() + SESSION_OPEN_TIMEOUT_MS;
try {
while (Date.now() < deadline) {
const connectInfo = await WorkflowJobSshQuery.connectInfoForResourceIdAsync(
graphqlClient,
workflowJobId
);
const status = resolveSshConnectStatus(connectInfo);
const statusMessage = terminalSshStatusMessage(status, workflowJobId);
if (statusMessage) {
spinner.fail(statusMessage);
return null;
}
if (status === 'ready') {
spinner.succeed('The ssh session is ready.');
return connectInfo?.session?.connectionConfig ?? null;
}
await sleepAsync(SESSION_OPEN_POLL_INTERVAL_MS);
}
spinner.fail(
'Timed out waiting for the ssh session to open. The worker may still be starting up; try again in a moment.'
);
return null;
} catch (err) {
spinner.stop();
throw err;
}
}
+295 -11
View File
@@ -83,6 +83,8 @@ export type Account = {
accountFeatureGates: Scalars['JSONObject']['output'];
/** Coalesced project activity for all apps belonging to this account. */
activityTimelineProjectActivities: Array<ActivityTimelineProjectActivity>;
/** AI provider connections available to agent sandbox sessions. */
agentProviderConnections: Array<AgentProviderConnection>;
aiChatEnabled: Scalars['Boolean']['output'];
appCount: Scalars['Int']['output'];
/** @deprecated Use appStoreConnectApiKeysPaginated */
@@ -1071,6 +1073,8 @@ export type AgentDeviceRunSessionRemoteConfig = {
__typename?: 'AgentDeviceRunSessionRemoteConfig';
agentDeviceRemoteSessionToken: Scalars['String']['output'];
agentDeviceRemoteSessionUrl: Scalars['String']['output'];
/** Session token gating the web preview. Null when the preview runs ungated. */
webPreviewToken?: Maybe<Scalars['String']['output']>;
/**
* URL of the web preview surface for the session. Null when web previews are
* not available for the platform (e.g. Android).
@@ -1078,6 +1082,80 @@ export type AgentDeviceRunSessionRemoteConfig = {
webPreviewUrl?: Maybe<Scalars['String']['output']>;
};
export enum AgentProvider {
Anthropic = 'ANTHROPIC',
Openai = 'OPENAI'
}
export type AgentProviderConnection = {
__typename?: 'AgentProviderConnection';
account: Account;
createdAt: Scalars['DateTime']['output'];
createdByActor?: Maybe<Actor>;
expiresAt?: Maybe<Scalars['DateTime']['output']>;
id: Scalars['ID']['output'];
name?: Maybe<Scalars['String']['output']>;
provider: AgentProvider;
providerAccountEmail?: Maybe<Scalars['String']['output']>;
providerAccountId?: Maybe<Scalars['String']['output']>;
providerPlan?: Maybe<Scalars['String']['output']>;
status: AgentProviderConnectionStatus;
updatedAt: Scalars['DateTime']['output'];
};
export enum AgentProviderConnectionAuthorizationStatus {
Expired = 'EXPIRED',
Pending = 'PENDING',
Succeeded = 'SUCCEEDED'
}
export type AgentProviderConnectionMutation = {
__typename?: 'AgentProviderConnectionMutation';
completeClaudeConnection: AgentProviderConnection;
deleteAgentProviderConnection: AgentProviderConnection;
pollCodexConnection: CodexConnectionPollResult;
startClaudeConnection: ClaudeAuthorization;
startCodexConnection: CodexDeviceAuthorization;
updateAgentProviderConnection: AgentProviderConnection;
};
export type AgentProviderConnectionMutationCompleteClaudeConnectionArgs = {
input: CompleteClaudeConnectionInput;
};
export type AgentProviderConnectionMutationDeleteAgentProviderConnectionArgs = {
id: Scalars['ID']['input'];
};
export type AgentProviderConnectionMutationPollCodexConnectionArgs = {
requestId: Scalars['ID']['input'];
};
export type AgentProviderConnectionMutationStartClaudeConnectionArgs = {
input: StartClaudeConnectionInput;
};
export type AgentProviderConnectionMutationStartCodexConnectionArgs = {
accountId: Scalars['ID']['input'];
};
export type AgentProviderConnectionMutationUpdateAgentProviderConnectionArgs = {
id: Scalars['ID']['input'];
input: UpdateAgentProviderConnectionInput;
};
export enum AgentProviderConnectionStatus {
Invalid = 'INVALID',
UnableToVerify = 'UNABLE_TO_VERIFY',
Valid = 'VALID'
}
export type AndroidAppBuildCredentials = {
__typename?: 'AndroidAppBuildCredentials';
androidKeystore?: Maybe<AndroidKeystore>;
@@ -1444,6 +1522,8 @@ export type App = Project & {
/** @deprecated Classic updates have been deprecated. */
description: Scalars['String']['output'];
devDomainName?: Maybe<AppDevDomainName>;
/** Every tag used by this app's device run sessions, sorted alphabetically. */
deviceRunSessionTags: Array<Scalars['String']['output']>;
deviceRunSessionsPaginated: AppDeviceRunSessionsConnection;
embeddedUpdatesPaginated: AppEmbeddedUpdatesConnection;
/** Environment secrets for an app */
@@ -1472,6 +1552,8 @@ export type App = Project & {
internalDistributionBuildPrivacy: AppInternalDistributionBuildPrivacy;
/** iOS app credentials for the project */
iosAppCredentials: Array<IosAppCredentials>;
/** Whether this app qualifies for the observe-notification-lapsed-paid experiment */
isEligibleForObserveLapsedPaidNotification: Scalars['Boolean']['output'];
/**
* Tells you if the project can show the Observe promotional notification. This field is part of
* the observe-notification experiment.
@@ -2173,6 +2255,11 @@ export type AppNotificationPreferenceInput = {
export type AppObserve = {
__typename?: 'AppObserve';
appVersions: Array<AppObserveAppVersion>;
/**
* A single custom event by the `id` of an `AppObserveCustomEvent`. Null when the app has no
* such event, including when it has aged out of retention.
*/
customEvent?: Maybe<AppObserveCustomEvent>;
customEventCounts: AppObserveCustomEventCounts;
customEventList: AppObserveCustomEventListConnection;
customEventNames: AppObserveCustomEventNames;
@@ -2185,6 +2272,14 @@ export type AppObserve = {
errorStats: AppObserveErrorStats;
/** Time-bucketed exception counts split into fatal vs non-fatal, for the stacked-bar chart. */
errorTimeSeries: AppObserveErrorTimeSeries;
/**
* A single metric event by the `id` of an `AppObserveEvent`. Null when the app has no such
* event, including when it has aged out of retention or the id is not one this API issued.
*
* `sessionEventCount` and `userEventCount` are always null here: they are aggregates over a
* time range, and a single event does not supply one.
*/
event?: Maybe<AppObserveEvent>;
events: AppObserveEventsConnection;
/** Highest Expo SDK version seen in telemetry over the trailing 30 days; null when none was reported. */
latestExpoSdkVersion?: Maybe<Scalars['String']['output']>;
@@ -2208,6 +2303,7 @@ export type AppObserve = {
* universe used by `appVersions` headline counts.
*/
uniqueActiveUserCount: Scalars['Int']['output'];
/** Update download activity (the Recent updates list). Not available on the free tier or legacy plans. */
updates: AppObserveUpdatesConnection;
};
@@ -2217,6 +2313,11 @@ export type AppObserveAppVersionsArgs = {
};
export type AppObserveCustomEventArgs = {
id: Scalars['ID']['input'];
};
export type AppObserveCustomEventCountsArgs = {
input: AppObserveCustomEventCountsInput;
};
@@ -2242,6 +2343,7 @@ export type AppObserveCustomEventNamesArgs = {
isEmbeddedUpdate?: InputMaybe<Scalars['Boolean']['input']>;
orderBy?: InputMaybe<AppObserveCustomEventNamesOrderBy>;
platform?: InputMaybe<AppObservePlatform>;
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime: Scalars['DateTime']['input'];
};
@@ -2249,6 +2351,7 @@ export type AppObserveCustomEventNamesArgs = {
export type AppObserveEnvironmentsArgs = {
endTime?: InputMaybe<Scalars['DateTime']['input']>;
platform?: InputMaybe<AppObservePlatform>;
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime?: InputMaybe<Scalars['DateTime']['input']>;
};
@@ -2273,6 +2376,11 @@ export type AppObserveErrorTimeSeriesArgs = {
};
export type AppObserveEventArgs = {
id: Scalars['ID']['input'];
};
export type AppObserveEventsArgs = {
after?: InputMaybe<Scalars['String']['input']>;
before?: InputMaybe<Scalars['String']['input']>;
@@ -2479,7 +2587,9 @@ export type AppObserveCustomEventCountsInput = {
environment?: InputMaybe<Scalars['String']['input']>;
eventName: Scalars['String']['input'];
isEmbeddedUpdate?: InputMaybe<Scalars['Boolean']['input']>;
platform: AppObservePlatform;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. One of platform or platforms is required. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime: Scalars['DateTime']['input'];
};
@@ -2508,6 +2618,8 @@ export type AppObserveCustomEventListFilter = {
eventName?: InputMaybe<Scalars['String']['input']>;
isEmbeddedUpdate?: InputMaybe<Scalars['Boolean']['input']>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
propertyFilters?: InputMaybe<Array<AppObserveCustomEventPropertyFilter>>;
sessionId?: InputMaybe<Scalars['String']['input']>;
startTime?: InputMaybe<Scalars['DateTime']['input']>;
@@ -2615,6 +2727,8 @@ export type AppObserveErrorGroupBreakdownInput = {
fingerprint: Scalars['String']['input'];
isEmbeddedUpdate?: InputMaybe<Scalars['Boolean']['input']>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime: Scalars['DateTime']['input'];
};
@@ -2647,6 +2761,8 @@ export type AppObserveErrorGroupsInput = {
isEmbeddedUpdate?: InputMaybe<Scalars['Boolean']['input']>;
orderBy?: InputMaybe<AppObserveErrorGroupsOrderBy>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
/** Restrict to fatal-only or non-fatal-only errors. */
severity?: InputMaybe<AppObserveErrorSeverity>;
/** Restrict to a capture source (e.g. global). */
@@ -2710,6 +2826,8 @@ export type AppObserveErrorStatsInput = {
environment?: InputMaybe<Scalars['String']['input']>;
isEmbeddedUpdate?: InputMaybe<Scalars['Boolean']['input']>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime: Scalars['DateTime']['input'];
};
@@ -2746,6 +2864,8 @@ export type AppObserveErrorTimeSeriesInput = {
fingerprint?: InputMaybe<Scalars['String']['input']>;
isEmbeddedUpdate?: InputMaybe<Scalars['Boolean']['input']>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime: Scalars['DateTime']['input'];
};
@@ -2817,6 +2937,8 @@ export type AppObserveEventsFilter = {
isEmbeddedUpdate?: InputMaybe<Scalars['Boolean']['input']>;
metricName?: InputMaybe<Scalars['String']['input']>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
routeName?: InputMaybe<Scalars['String']['input']>;
sessionId?: InputMaybe<Scalars['String']['input']>;
startTime?: InputMaybe<Scalars['DateTime']['input']>;
@@ -2871,6 +2993,8 @@ export type AppObserveNavigationRoutesFilter = {
endTime: Scalars['DateTime']['input'];
environment?: InputMaybe<Scalars['String']['input']>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
routeNames?: InputMaybe<Array<Scalars['String']['input']>>;
startTime: Scalars['DateTime']['input'];
};
@@ -2924,6 +3048,8 @@ export type AppObserveOverviewEngagementInput = {
endTime: Scalars['DateTime']['input'];
environment?: InputMaybe<Scalars['String']['input']>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime: Scalars['DateTime']['input'];
};
@@ -2971,6 +3097,8 @@ export type AppObserveOverviewStabilityInput = {
endTime: Scalars['DateTime']['input'];
environment?: InputMaybe<Scalars['String']['input']>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime: Scalars['DateTime']['input'];
};
@@ -2978,9 +3106,13 @@ export type AppObserveOverviewUpdate = {
__typename?: 'AppObserveOverviewUpdate';
/** Null for the embedded bundle. */
appUpdateId?: Maybe<Scalars['ID']['output']>;
/** Downloads recorded in range across platforms; null when none were recorded (and always for the embedded bundle). */
/**
* Downloads recorded in range across platforms; null when none were recorded
* (and always for the embedded bundle). Download data is paid-only: on the
* free tier and legacy plans this is always null.
*/
downloadCount?: Maybe<Scalars['Int']['output']>;
/** One entry per platform with downloads in range; empty for the embedded bundle. */
/** One entry per platform with downloads in range; empty for the embedded bundle and on the free tier and legacy plans. */
downloads: Array<AppObserveOverviewUpdateDownload>;
eventCount: Scalars['Int']['output'];
firstSeenAt: Scalars['DateTime']['output'];
@@ -3092,6 +3224,8 @@ export type AppObserveReleasesInput = {
environment?: InputMaybe<Scalars['String']['input']>;
metricNames?: InputMaybe<Array<Scalars['String']['input']>>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime: Scalars['DateTime']['input'];
};
@@ -3129,6 +3263,8 @@ export type AppObserveTimeSeriesInput = {
isEmbeddedUpdate?: InputMaybe<Scalars['Boolean']['input']>;
metricName: Scalars['String']['input'];
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
routeName?: InputMaybe<Scalars['String']['input']>;
startTime: Scalars['DateTime']['input'];
};
@@ -3180,6 +3316,8 @@ export type AppObserveUpdatesInput = {
first?: InputMaybe<Scalars['Int']['input']>;
orderBy?: InputMaybe<AppObserveUpdatesOrderBy>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime: Scalars['DateTime']['input'];
};
@@ -3674,6 +3812,8 @@ export type AppiumRunSessionRemoteConfig = {
appiumUrl: Scalars['String']['output'];
/** W3C capabilities for the device that backs this session. */
capabilities: Scalars['JSONObject']['output'];
/** Session token gating the web preview. Null when the preview runs ungated. */
webPreviewToken?: Maybe<Scalars['String']['output']>;
/**
* URL of the web preview surface for the session. Null when web previews are
* not available for the platform (e.g. Android).
@@ -4082,6 +4222,8 @@ export type ArgentRunSessionRemoteConfig = {
__typename?: 'ArgentRunSessionRemoteConfig';
toolsAuthToken?: Maybe<Scalars['String']['output']>;
toolsUrl: Scalars['String']['output'];
/** Session token gating the web preview. Null when the preview runs ungated. */
webPreviewToken?: Maybe<Scalars['String']['output']>;
/**
* URL of the web preview surface for the session. Null when web previews are
* not available for the platform (e.g. Android).
@@ -5031,6 +5173,13 @@ export type ChatTokenUsage = {
usedTokens: Scalars['Int']['output'];
};
export type ClaudeAuthorization = {
__typename?: 'ClaudeAuthorization';
authorizationUrl: Scalars['String']['output'];
expiresAt: Scalars['DateTime']['output'];
requestId: Scalars['ID']['output'];
};
export type CodeSigningInfo = {
__typename?: 'CodeSigningInfo';
alg: Scalars['String']['output'];
@@ -5044,6 +5193,26 @@ export type CodeSigningInfoInput = {
sig: Scalars['String']['input'];
};
export type CodexConnectionPollResult = {
__typename?: 'CodexConnectionPollResult';
connection?: Maybe<AgentProviderConnection>;
status: AgentProviderConnectionAuthorizationStatus;
};
export type CodexDeviceAuthorization = {
__typename?: 'CodexDeviceAuthorization';
expiresAt: Scalars['DateTime']['output'];
pollingIntervalSeconds: Scalars['Int']['output'];
requestId: Scalars['ID']['output'];
userCode: Scalars['String']['output'];
verificationUrl: Scalars['String']['output'];
};
export type CompleteClaudeConnectionInput = {
authorizationCode: Scalars['String']['input'];
requestId: Scalars['ID']['input'];
};
export type CompletePostHogConnectionInput = {
code: Scalars['String']['input'];
state: Scalars['ID']['input'];
@@ -5265,7 +5434,7 @@ export type CreateDeviceRunSessionInput = {
/**
* Stop the session automatically after this many minutes without observed
* session activity. Must be positive and smaller than the session's maximum
* duration (2 hours, or maxRunTimeMinutes when set). Only supported for
* duration (maxRunTimeMinutes when set). Only supported for
* agent-device, argent, and Appium sessions. If omitted, the session has no idle
* timeout.
*/
@@ -5293,11 +5462,14 @@ export type CreateDeviceRunSessionInput = {
*/
packageVersion?: InputMaybe<Scalars['String']['input']>;
platform: AppPlatform;
resourceClass?: InputMaybe<DeviceRunSessionResourceClass>;
/**
* Expo SDK version used to select an Expo Go application archive. Only supported when expoGo is
* true. If omitted, the current Expo Go archive for the platform is used.
*/
sdkVersion?: InputMaybe<Scalars['String']['input']>;
/** Free-form labels for grouping sessions, for example one per app variant. */
tags?: InputMaybe<Array<Scalars['String']['input']>>;
type: DeviceRunSessionType;
};
@@ -6043,7 +6215,17 @@ export type DeploymentsMutationDeleteWorkerDeploymentByIdentifierArgs = {
export type DeviceRunSession = {
__typename?: 'DeviceRunSession';
app: App;
/**
* Direct application archive URL installed and launched for this session. Null when
* the session uses an EAS Build or starts without an app.
*/
applicationArchiveUrl?: Maybe<Scalars['String']['output']>;
artifacts: Array<DeviceRunSessionArtifact>;
/**
* EAS Build installed and launched for this session. Null when the session uses
* a direct application archive URL or starts without an app.
*/
build?: Maybe<Build>;
createdAt: Scalars['DateTime']['output'];
finishedAt?: Maybe<Scalars['DateTime']['output']>;
id: Scalars['ID']['output'];
@@ -6067,6 +6249,8 @@ export type DeviceRunSession = {
remoteConfig?: Maybe<DeviceRunSessionRemoteConfig>;
startedAt?: Maybe<Scalars['DateTime']['output']>;
status: DeviceRunSessionStatus;
/** Free-form labels for grouping sessions, for example one per app variant. */
tags: Array<Scalars['String']['output']>;
turtleJobRun?: Maybe<JobRun>;
type: DeviceRunSessionType;
updatedAt: Scalars['DateTime']['output'];
@@ -6113,6 +6297,8 @@ export type DeviceRunSessionFilterInput = {
name?: InputMaybe<Scalars['String']['input']>;
platforms?: InputMaybe<Array<AppPlatform>>;
statuses?: InputMaybe<Array<DeviceRunSessionStatus>>;
/** Matches sessions carrying every listed tag. Case-insensitive. */
tags?: InputMaybe<Array<Scalars['String']['input']>>;
types?: InputMaybe<Array<DeviceRunSessionType>>;
};
@@ -6174,7 +6360,12 @@ export type DeviceRunSessionQueryByIdArgs = {
deviceRunSessionId: Scalars['ID']['input'];
};
export type DeviceRunSessionRemoteConfig = AgentDeviceRunSessionRemoteConfig | AppiumRunSessionRemoteConfig | ArgentRunSessionRemoteConfig | ServeSimRunSessionRemoteConfig;
export type DeviceRunSessionRemoteConfig = AgentDeviceRunSessionRemoteConfig | AppiumRunSessionRemoteConfig | ArgentRunSessionRemoteConfig | ServeSimRunSessionRemoteConfig | WebPreviewOnlyRunSessionRemoteConfig;
export enum DeviceRunSessionResourceClass {
Large = 'LARGE',
Medium = 'MEDIUM'
}
export enum DeviceRunSessionStatus {
Errored = 'ERRORED',
@@ -6187,7 +6378,10 @@ export enum DeviceRunSessionType {
AgentDevice = 'AGENT_DEVICE',
Appium = 'APPIUM',
Argent = 'ARGENT',
ServeSim = 'SERVE_SIM'
/** @deprecated Use WEB_PREVIEW_ONLY instead. */
ServeSim = 'SERVE_SIM',
/** A session accessed only through its web preview. Currently supported on iOS. */
WebPreviewOnly = 'WEB_PREVIEW_ONLY'
}
export type DiscordUser = {
@@ -7054,6 +7248,7 @@ export type EmbeddedUpdateQueryByIdArgs = {
export enum EntityTypeName {
AccountEntity = 'AccountEntity',
AccountSsoConfigurationEntity = 'AccountSSOConfigurationEntity',
AgentProviderConnectionEntity = 'AgentProviderConnectionEntity',
AndroidAppCredentialsEntity = 'AndroidAppCredentialsEntity',
AndroidKeystoreEntity = 'AndroidKeystoreEntity',
AppEntity = 'AppEntity',
@@ -9437,6 +9632,7 @@ export type RootMutation = {
accountSSOConfiguration: AccountSsoConfigurationMutation;
/** Mutations for Actor experiments */
actorExperiment: ActorExperimentMutation;
agentProviderConnection: AgentProviderConnectionMutation;
/** Mutations that modify the build credentials for an Android app */
androidAppBuildCredentials: AndroidAppBuildCredentialsMutation;
/** Mutations that modify the credentials for an Android app */
@@ -10039,8 +10235,14 @@ export type SentryProjectMutationDeleteSentryProjectArgs = {
sentryProjectId: Scalars['ID']['input'];
};
/**
* Legacy remote config for SERVE_SIM sessions. Use WEB_PREVIEW_ONLY sessions and
* WebPreviewOnlyRunSessionRemoteConfig instead.
*/
export type ServeSimRunSessionRemoteConfig = {
__typename?: 'ServeSimRunSessionRemoteConfig';
/** Session token gating the preview. Null when the preview runs ungated. */
previewToken?: Maybe<Scalars['String']['output']>;
previewUrl: Scalars['String']['output'];
/** @deprecated Use previewUrl instead. */
streamUrl?: Maybe<Scalars['String']['output']>;
@@ -10129,6 +10331,10 @@ export enum StandardOffer {
YearlySub = 'YEARLY_SUB'
}
export type StartClaudeConnectionInput = {
accountId: Scalars['ID']['input'];
};
export type StartPostHogConnectionResult = PostHogOrganizationConnection | PostHogPendingConnection;
/** Incident for a given component from Expo status page API. */
@@ -10644,8 +10850,7 @@ export type TurtleSshSessionMutation = {
export type TurtleSshSessionMutationCreateOrUpdateTurtleSshSessionArgs = {
connectionConfig: TurtleSshConnectionConfigInput;
sessionSettings: TurtleSshSessionSettingsInput;
turtleBuildId?: InputMaybe<Scalars['ID']['input']>;
turtleJobRunId?: InputMaybe<Scalars['ID']['input']>;
target: TurtleSshTargetInput;
};
export type TurtleSshSessionSettings = {
@@ -10657,6 +10862,16 @@ export type TurtleSshSessionSettingsInput = {
idleTimeoutSeconds: Scalars['Int']['input'];
};
export type TurtleSshTargetInput = {
id: Scalars['ID']['input'];
type: TurtleSshTargetType;
};
export enum TurtleSshTargetType {
Build = 'BUILD',
JobRun = 'JOB_RUN'
}
export enum TurtleSshTransportType {
UptermV1 = 'UPTERM_V1'
}
@@ -10728,6 +10943,10 @@ export type UpdateDiffReceiptsArgs = {
orderBy?: InputMaybe<Array<UpdateDiffReceiptOrderByClause>>;
};
export type UpdateAgentProviderConnectionInput = {
name?: InputMaybe<Scalars['String']['input']>;
};
export type UpdateBranch = {
__typename?: 'UpdateBranch';
app: App;
@@ -12053,6 +12272,11 @@ export type WebNotificationUpdateReadStateInput = {
isRead: Scalars['Boolean']['input'];
};
export type WebPreviewOnlyRunSessionRemoteConfig = {
__typename?: 'WebPreviewOnlyRunSessionRemoteConfig';
webPreviewUrl: Scalars['String']['output'];
};
export type Webhook = {
__typename?: 'Webhook';
appId: Scalars['ID']['output'];
@@ -13277,6 +13501,11 @@ export type WorkflowRun = ActivityTimelineProjectActivity & {
__typename?: 'WorkflowRun';
activityTimestamp: Scalars['DateTime']['output'];
actor?: Maybe<Actor>;
/**
* Why the server canceled this run. Null for manually canceled runs and for
* runs canceled before the reason was recorded.
*/
cancelReason?: Maybe<WorkflowRunCancelReason>;
createdAt: Scalars['DateTime']['output'];
durationSeconds?: Maybe<Scalars['Int']['output']>;
errors: Array<WorkflowRunError>;
@@ -13303,6 +13532,17 @@ export type WorkflowRun = ActivityTimelineProjectActivity & {
workflowRevision?: Maybe<WorkflowRevision>;
};
export type WorkflowRunCancelReason = WorkflowRunCancelReasonConcurrencyGroup;
export type WorkflowRunCancelReasonConcurrencyGroup = {
__typename?: 'WorkflowRunCancelReasonConcurrencyGroup';
/**
* The newer run in the same concurrency group whose creation canceled this run.
* Null if that run no longer exists.
*/
cancelingWorkflowRun?: Maybe<WorkflowRun>;
};
export type WorkflowRunEdge = {
__typename?: 'WorkflowRunEdge';
cursor: Scalars['String']['output'];
@@ -13328,6 +13568,7 @@ export type WorkflowRunGitBranchFilterInput = {
export type WorkflowRunInput = {
inputs?: InputMaybe<Scalars['JSONObject']['input']>;
projectSource: WorkflowProjectSourceInput;
ssh?: InputMaybe<WorkflowRunSshInput>;
};
export type WorkflowRunMutation = {
@@ -13362,12 +13603,14 @@ export type WorkflowRunMutationCreateWorkflowRunArgs = {
export type WorkflowRunMutationCreateWorkflowRunFromGitRefArgs = {
gitRef: Scalars['String']['input'];
inputs?: InputMaybe<Scalars['JSONObject']['input']>;
ssh?: InputMaybe<WorkflowRunSshInput>;
workflowRevisionId: Scalars['ID']['input'];
};
export type WorkflowRunMutationRetryWorkflowRunArgs = {
fromFailedJobs?: InputMaybe<Scalars['Boolean']['input']>;
ssh?: InputMaybe<WorkflowRunSshInput>;
workflowRunId: Scalars['ID']['input'];
};
@@ -13381,6 +13624,14 @@ export type WorkflowRunQueryByIdArgs = {
workflowRunId: Scalars['ID']['input'];
};
/**
* Enables ssh on the run's VM jobs. Presence turns ssh on; idleTimeoutSeconds is optional,
* defaults server-side, and is validated against a supported range.
*/
export type WorkflowRunSshInput = {
idleTimeoutSeconds?: InputMaybe<Scalars['Int']['input']>;
};
export type WorkflowRunSshSettings = {
__typename?: 'WorkflowRunSshSettings';
idleTimeoutSeconds: Scalars['Int']['output'];
@@ -13392,7 +13643,8 @@ export enum WorkflowRunStatus {
Failure = 'FAILURE',
InProgress = 'IN_PROGRESS',
New = 'NEW',
Success = 'SUCCESS'
Success = 'SUCCESS',
Waiting = 'WAITING'
}
export type WorkflowRunTimeRangeInput = {
@@ -13408,6 +13660,9 @@ export enum WorkflowRunTriggerEventType {
EasSubmit = 'EAS_SUBMIT',
ExpoLaunch = 'EXPO_LAUNCH',
GithubPullRequestBaseRefChanged = 'GITHUB_PULL_REQUEST_BASE_REF_CHANGED',
GithubPullRequestCommentCreated = 'GITHUB_PULL_REQUEST_COMMENT_CREATED',
GithubPullRequestCommentDeleted = 'GITHUB_PULL_REQUEST_COMMENT_DELETED',
GithubPullRequestCommentEdited = 'GITHUB_PULL_REQUEST_COMMENT_EDITED',
GithubPullRequestEdited = 'GITHUB_PULL_REQUEST_EDITED',
GithubPullRequestLabeled = 'GITHUB_PULL_REQUEST_LABELED',
GithubPullRequestOpened = 'GITHUB_PULL_REQUEST_OPENED',
@@ -14478,6 +14733,19 @@ export type SetRolloutPercentageMutation = { __typename?: 'RootMutation', update
| { __typename: 'User', username: string, id: string }
| null, branch: { __typename?: 'UpdateBranch', id: string, name: string }, codeSigningInfo?: { __typename?: 'CodeSigningInfo', keyid: string, sig: string, alg: string } | null, rolloutControlUpdate?: { __typename?: 'Update', id: string, group: string } | null, fingerprint?: { __typename?: 'Fingerprint', id: string, hash: string, debugInfoUrl?: string | null, source?: { __typename?: 'FingerprintSource', type: FingerprintSourceType, bucketKey: string, isDebugFingerprint?: boolean | null } | null } | null } } };
export type GenerateRealtimeLogsCentrifugoConnectionTokenMutationVariables = Exact<{ [key: string]: never; }>;
export type GenerateRealtimeLogsCentrifugoConnectionTokenMutation = { __typename?: 'RootMutation', realtimeLogs: { __typename?: 'RealtimeLogsMutation', generateCentrifugoConnectionToken: { __typename?: 'RealtimeLogsCentrifugoConnectionToken', token: string } } };
export type GenerateRealtimeLogsCentrifugoSubscriptionTokenMutationVariables = Exact<{
target: RealtimeLogsTargetInput;
thread?: InputMaybe<Scalars['String']['input']>;
}>;
export type GenerateRealtimeLogsCentrifugoSubscriptionTokenMutation = { __typename?: 'RootMutation', realtimeLogs: { __typename?: 'RealtimeLogsMutation', generateCentrifugoSubscriptionToken: { __typename?: 'RealtimeLogsCentrifugoSubscriptionToken', channel: string, token: string } } };
export type CreateAndroidSubmissionMutationVariables = Exact<{
appId: Scalars['ID']['input'];
config: AndroidSubmissionConfigInput;
@@ -14652,6 +14920,7 @@ export type CreateWorkflowRunFromGitRefMutationVariables = Exact<{
workflowRevisionId: Scalars['ID']['input'];
gitRef: Scalars['String']['input'];
inputs?: InputMaybe<Scalars['JSONObject']['input']>;
ssh?: InputMaybe<WorkflowRunSshInput>;
}>;
@@ -15028,11 +15297,12 @@ export type DeviceRunSessionByIdQueryVariables = Exact<{
}>;
export type DeviceRunSessionByIdQuery = { __typename?: 'RootQuery', deviceRunSessions: { __typename?: 'DeviceRunSessionQuery', byId: { __typename?: 'DeviceRunSession', id: string, name?: string | null, status: DeviceRunSessionStatus, type: DeviceRunSessionType, platform: AppPlatform, createdAt: any, startedAt?: any | null, finishedAt?: any | null, updatedAt: any, app: { __typename?: 'App', id: string, slug: string, ownerAccount: { __typename?: 'Account', id: string, name: string } }, artifacts: Array<{ __typename?: 'DeviceRunSessionArtifact', id: string, name: string, filename: string, downloadUrl: string, fileSizeBytes?: number | null, metadata?: any | null, createdAt: any, updatedAt: any }>, remoteConfig?:
export type DeviceRunSessionByIdQuery = { __typename?: 'RootQuery', deviceRunSessions: { __typename?: 'DeviceRunSessionQuery', byId: { __typename?: 'DeviceRunSession', id: string, name?: string | null, tags: Array<string>, status: DeviceRunSessionStatus, type: DeviceRunSessionType, platform: AppPlatform, createdAt: any, startedAt?: any | null, finishedAt?: any | null, updatedAt: any, app: { __typename?: 'App', id: string, slug: string, ownerAccount: { __typename?: 'Account', id: string, name: string } }, artifacts: Array<{ __typename?: 'DeviceRunSessionArtifact', id: string, name: string, filename: string, downloadUrl: string, fileSizeBytes?: number | null, metadata?: any | null, createdAt: any, updatedAt: any }>, remoteConfig?:
| { __typename: 'AgentDeviceRunSessionRemoteConfig', agentDeviceRemoteSessionUrl: string, agentDeviceRemoteSessionToken: string, webPreviewUrl?: string | null }
| { __typename: 'AppiumRunSessionRemoteConfig', appiumUrl: string, capabilities: any, webPreviewUrl?: string | null }
| { __typename: 'ArgentRunSessionRemoteConfig', toolsUrl: string, toolsAuthToken?: string | null, webPreviewUrl?: string | null }
| { __typename: 'ServeSimRunSessionRemoteConfig', previewUrl: string }
| { __typename: 'WebPreviewOnlyRunSessionRemoteConfig', previewUrl: string }
| null, turtleJobRun?: { __typename?: 'JobRun', id: string, status: JobRunStatus } | null } } };
export type DeviceRunSessionsByAppIdQueryVariables = Exact<{
@@ -15043,7 +15313,7 @@ export type DeviceRunSessionsByAppIdQueryVariables = Exact<{
}>;
export type DeviceRunSessionsByAppIdQuery = { __typename?: 'RootQuery', app: { __typename?: 'AppQuery', byId: { __typename?: 'App', id: string, deviceRunSessionsPaginated: { __typename?: 'AppDeviceRunSessionsConnection', edges: Array<{ __typename?: 'AppDeviceRunSessionEdge', cursor: string, node: { __typename?: 'DeviceRunSession', id: string, name?: string | null, status: DeviceRunSessionStatus, type: DeviceRunSessionType, platform: AppPlatform, createdAt: any, startedAt?: any | null, finishedAt?: any | null, app: { __typename?: 'App', id: string, slug: string, ownerAccount: { __typename?: 'Account', id: string, name: string } }, turtleJobRun?: { __typename?: 'JobRun', id: string, status: JobRunStatus } | null } }>, pageInfo: { __typename?: 'PageInfo', hasNextPage: boolean, hasPreviousPage: boolean, startCursor?: string | null, endCursor?: string | null } } } } };
export type DeviceRunSessionsByAppIdQuery = { __typename?: 'RootQuery', app: { __typename?: 'AppQuery', byId: { __typename?: 'App', id: string, deviceRunSessionsPaginated: { __typename?: 'AppDeviceRunSessionsConnection', edges: Array<{ __typename?: 'AppDeviceRunSessionEdge', cursor: string, node: { __typename?: 'DeviceRunSession', id: string, name?: string | null, tags: Array<string>, status: DeviceRunSessionStatus, type: DeviceRunSessionType, platform: AppPlatform, createdAt: any, startedAt?: any | null, finishedAt?: any | null, app: { __typename?: 'App', id: string, slug: string, ownerAccount: { __typename?: 'Account', id: string, name: string } }, turtleJobRun?: { __typename?: 'JobRun', id: string, status: JobRunStatus } | null } }>, pageInfo: { __typename?: 'PageInfo', hasNextPage: boolean, hasPreviousPage: boolean, startCursor?: string | null, endCursor?: string | null } } } } };
export type ViewEmbeddedUpdateByIdQueryVariables = Exact<{
embeddedUpdateId: Scalars['ID']['input'];
@@ -15429,6 +15699,20 @@ export type WorkflowJobByIdQuery = { __typename?: 'RootQuery', workflowJobs: { _
| { __typename: 'User', id: string, displayName: string }
| null, app: { __typename: 'App', id: string, name: string, slug: string, ownerAccount: { __typename?: 'Account', id: string, name: string } }, updateChannel?: { __typename?: 'UpdateChannel', id: string, name: string } | null, runtime?: { __typename?: 'Runtime', id: string, version: string } | null, metrics?: { __typename?: 'BuildMetrics', buildWaitTime?: number | null, buildQueueTime?: number | null, buildDuration?: number | null } | null } | null, errors: Array<{ __typename?: 'WorkflowJobError', title: string, message: string }> } } };
export type WorkflowJobSshPollQueryVariables = Exact<{
workflowJobId: Scalars['ID']['input'];
}>;
export type WorkflowJobSshPollQuery = { __typename?: 'RootQuery', workflowJobs: { __typename?: 'WorkflowJobQuery', byId: { __typename?: 'WorkflowJob', id: string, status: WorkflowJobStatus, workflowRun: { __typename?: 'WorkflowRun', id: string, sshSettings?: { __typename?: 'WorkflowRunSshSettings', idleTimeoutSeconds: number } | null }, turtleJobRun?: { __typename?: 'JobRun', id: string, sshSession?: { __typename?: 'TurtleSshSession', id: string, connectionConfig: { __typename?: 'TurtleSshConnectionConfig', host: string, secret: string, reconnecting: boolean } } | null } | null, turtleBuild?: { __typename?: 'Build', id: string, sshSession?: { __typename?: 'TurtleSshSession', id: string, connectionConfig: { __typename?: 'TurtleSshConnectionConfig', host: string, secret: string, reconnecting: boolean } } | null } | null } } };
export type JobRunSshPollQueryVariables = Exact<{
jobRunId: Scalars['ID']['input'];
}>;
export type JobRunSshPollQuery = { __typename?: 'RootQuery', jobRun: { __typename?: 'JobRunQuery', byId: { __typename?: 'JobRun', id: string, status: JobRunStatus, sshSession?: { __typename?: 'TurtleSshSession', id: string, connectionConfig: { __typename?: 'TurtleSshConnectionConfig', host: string, secret: string, reconnecting: boolean } } | null } } };
export type ExpoGoSupportedSdkVersionsQueryVariables = Exact<{ [key: string]: never; }>;
@@ -0,0 +1,69 @@
import gql from 'graphql-tag';
import { ExpoGraphqlClient } from '../../commandUtils/context/contextUtils/createGraphqlClient';
import { withErrorHandlingAsync } from '../client';
import {
GenerateRealtimeLogsCentrifugoConnectionTokenMutation,
GenerateRealtimeLogsCentrifugoConnectionTokenMutationVariables,
GenerateRealtimeLogsCentrifugoSubscriptionTokenMutation,
GenerateRealtimeLogsCentrifugoSubscriptionTokenMutationVariables,
RealtimeLogsCentrifugoConnectionToken,
RealtimeLogsCentrifugoSubscriptionToken,
RealtimeLogsMutationGenerateCentrifugoSubscriptionTokenArgs,
} from '../generated';
export const RealtimeLogsMutation = {
async generateCentrifugoConnectionTokenAsync(
graphqlClient: ExpoGraphqlClient
): Promise<RealtimeLogsCentrifugoConnectionToken> {
const data = await withErrorHandlingAsync(
graphqlClient
.mutation<
GenerateRealtimeLogsCentrifugoConnectionTokenMutation,
GenerateRealtimeLogsCentrifugoConnectionTokenMutationVariables
>(
gql`
mutation GenerateRealtimeLogsCentrifugoConnectionToken {
realtimeLogs {
generateCentrifugoConnectionToken {
token
}
}
}
`,
{}
)
.toPromise()
);
return data.realtimeLogs.generateCentrifugoConnectionToken;
},
async generateCentrifugoSubscriptionTokenAsync(
graphqlClient: ExpoGraphqlClient,
{ target, thread }: RealtimeLogsMutationGenerateCentrifugoSubscriptionTokenArgs
): Promise<RealtimeLogsCentrifugoSubscriptionToken> {
const data = await withErrorHandlingAsync(
graphqlClient
.mutation<
GenerateRealtimeLogsCentrifugoSubscriptionTokenMutation,
GenerateRealtimeLogsCentrifugoSubscriptionTokenMutationVariables
>(
gql`
mutation GenerateRealtimeLogsCentrifugoSubscriptionToken(
$target: RealtimeLogsTargetInput!
$thread: String
) {
realtimeLogs {
generateCentrifugoSubscriptionToken(target: $target, thread: $thread) {
channel
token
}
}
}
`,
{ target, thread }
)
.toPromise()
);
return data.realtimeLogs.generateCentrifugoSubscriptionToken;
},
};
@@ -12,6 +12,7 @@ import {
WorkflowProjectSourceInput,
WorkflowRevisionInput,
WorkflowRunInput,
WorkflowRunSshInput,
} from '../generated';
export namespace WorkflowRunMutation {
@@ -64,10 +65,12 @@ export namespace WorkflowRunMutation {
workflowRevisionId,
gitRef,
inputs,
ssh,
}: {
workflowRevisionId: string;
gitRef: string;
inputs?: Record<string, any>;
ssh?: WorkflowRunSshInput | null;
}
): Promise<{ id: string }> {
const data = await withErrorHandlingAsync(
@@ -81,12 +84,14 @@ export namespace WorkflowRunMutation {
$workflowRevisionId: ID!
$gitRef: String!
$inputs: JSONObject
$ssh: WorkflowRunSshInput
) {
workflowRun {
createWorkflowRunFromGitRef(
workflowRevisionId: $workflowRevisionId
gitRef: $gitRef
inputs: $inputs
ssh: $ssh
) {
id
}
@@ -97,6 +102,7 @@ export namespace WorkflowRunMutation {
workflowRevisionId,
gitRef,
inputs,
ssh,
}
)
.toPromise()
@@ -55,6 +55,7 @@ export const DeviceRunSessionQuery = {
byId(deviceRunSessionId: $deviceRunSessionId) {
id
name
tags
status
type
platform
@@ -100,6 +101,9 @@ export const DeviceRunSessionQuery = {
... on ServeSimRunSessionRemoteConfig {
previewUrl
}
... on WebPreviewOnlyRunSessionRemoteConfig {
previewUrl: webPreviewUrl
}
}
turtleJobRun {
id
@@ -149,6 +153,7 @@ export const DeviceRunSessionQuery = {
node {
id
name
tags
status
type
platform
@@ -0,0 +1,213 @@
import gql from 'graphql-tag';
import { ExpoGraphqlClient } from '../../commandUtils/context/contextUtils/createGraphqlClient';
import { GraphqlError, withErrorHandlingAsync } from '../client';
import { JobRunStatus, WorkflowJobStatus } from '../generated';
const FINAL_WORKFLOW_JOB_STATUSES = new Set<WorkflowJobStatus>([
WorkflowJobStatus.Success,
WorkflowJobStatus.Failure,
WorkflowJobStatus.Canceled,
WorkflowJobStatus.Skipped,
]);
const FINAL_JOB_RUN_STATUSES = new Set<JobRunStatus>([
JobRunStatus.Errored,
JobRunStatus.Finished,
JobRunStatus.Canceled,
]);
export type WorkflowJobSshConnectionConfig = {
host: string;
secret: string;
reconnecting: boolean;
};
export type WorkflowJobSshSession = {
id: string;
connectionConfig: WorkflowJobSshConnectionConfig;
};
export type WorkflowJobSshConnectInfo = {
sshRequested: boolean;
jobCompleted: boolean;
session: WorkflowJobSshSession | null;
};
type WorkflowJobSshPollQuery = {
workflowJobs: {
byId: {
id: string;
status: WorkflowJobStatus;
workflowRun: {
id: string;
sshSettings: { idleTimeoutSeconds: number } | null;
};
turtleJobRun: { id: string; sshSession: WorkflowJobSshSession | null } | null;
turtleBuild: { id: string; sshSession: WorkflowJobSshSession | null } | null;
};
};
};
type WorkflowJobSshPollQueryVariables = {
workflowJobId: string;
};
type JobRunSshPollQuery = {
jobRun: {
byId: {
id: string;
status: JobRunStatus;
sshSession: WorkflowJobSshSession | null;
};
};
};
type JobRunSshPollQueryVariables = {
jobRunId: string;
};
function toConnectInfo(
job: WorkflowJobSshPollQuery['workflowJobs']['byId']
): WorkflowJobSshConnectInfo {
const hasTurtleTarget = job.turtleJobRun != null || job.turtleBuild != null;
return {
sshRequested: hasTurtleTarget && job.workflowRun.sshSettings != null,
jobCompleted: FINAL_WORKFLOW_JOB_STATUSES.has(job.status),
session: job.turtleJobRun?.sshSession ?? job.turtleBuild?.sshSession ?? null,
};
}
function isNotFoundError(error: unknown): boolean {
if (!(error instanceof GraphqlError)) {
return false;
}
return error.graphQLErrors.some(e => {
const code = e?.extensions?.errorCode ?? e?.extensions?.code;
return (
code === 'ENTITY_NOT_FOUND' ||
/Entity not found/i.test(e?.message ?? '') ||
/not found/i.test(e?.message ?? '')
);
});
}
export const WorkflowJobSshQuery = {
async connectInfoForWorkflowJobAsync(
graphqlClient: ExpoGraphqlClient,
workflowJobId: string
): Promise<WorkflowJobSshConnectInfo | null> {
let data: WorkflowJobSshPollQuery;
try {
data = await withErrorHandlingAsync(
graphqlClient
.query<WorkflowJobSshPollQuery, WorkflowJobSshPollQueryVariables>(
gql`
query WorkflowJobSshPoll($workflowJobId: ID!) {
workflowJobs {
byId(workflowJobId: $workflowJobId) {
id
status
workflowRun {
id
sshSettings {
idleTimeoutSeconds
}
}
turtleJobRun {
id
sshSession {
id
connectionConfig {
host
secret
reconnecting
}
}
}
turtleBuild {
id
sshSession {
id
connectionConfig {
host
secret
reconnecting
}
}
}
}
}
}
`,
{ workflowJobId },
{ requestPolicy: 'network-only' }
)
.toPromise()
);
} catch (error) {
if (isNotFoundError(error)) {
return null;
}
throw error;
}
return toConnectInfo(data.workflowJobs.byId);
},
async connectInfoForJobRunAsync(
graphqlClient: ExpoGraphqlClient,
jobRunId: string
): Promise<WorkflowJobSshConnectInfo | null> {
let data: JobRunSshPollQuery;
try {
data = await withErrorHandlingAsync(
graphqlClient
.query<JobRunSshPollQuery, JobRunSshPollQueryVariables>(
gql`
query JobRunSshPoll($jobRunId: ID!) {
jobRun {
byId(jobRunId: $jobRunId) {
id
status
sshSession {
id
connectionConfig {
host
secret
reconnecting
}
}
}
}
}
`,
{ jobRunId },
{ requestPolicy: 'network-only' }
)
.toPromise()
);
} catch (error) {
if (isNotFoundError(error)) {
return null;
}
throw error;
}
const jobRun = data.jobRun.byId;
const jobCompleted = FINAL_JOB_RUN_STATUSES.has(jobRun.status);
return {
sshRequested: jobRun.sshSession != null || jobCompleted,
jobCompleted,
session: jobRun.sshSession,
};
},
async connectInfoForResourceIdAsync(
graphqlClient: ExpoGraphqlClient,
resourceId: string
): Promise<WorkflowJobSshConnectInfo | null> {
return (
(await WorkflowJobSshQuery.connectInfoForWorkflowJobAsync(graphqlClient, resourceId)) ??
(await WorkflowJobSshQuery.connectInfoForJobRunAsync(graphqlClient, resourceId))
);
},
};
@@ -0,0 +1,155 @@
import { ExpoGraphqlClient } from '../../../commandUtils/context/contextUtils/createGraphqlClient';
import { GraphqlError } from '../../client';
import { WorkflowJobStatus } from '../../generated';
import { WorkflowJobSshQuery } from '../WorkflowJobSshQuery';
describe(WorkflowJobSshQuery.connectInfoForWorkflowJobAsync.name, () => {
function makeClient(byId: unknown): {
graphqlClient: ExpoGraphqlClient;
query: jest.Mock;
} {
const query = jest.fn().mockReturnValue({
toPromise: async () => ({ data: { workflowJobs: { byId } } }),
});
return { graphqlClient: { query } as unknown as ExpoGraphqlClient, query };
}
it('maps workflowJobs.byId into connect info', async () => {
const { graphqlClient, query } = makeClient({
id: 'job-1',
status: WorkflowJobStatus.InProgress,
workflowRun: { sshSettings: { idleTimeoutSeconds: 0 } },
turtleJobRun: {
sshSession: {
id: 'ts-1',
connectionConfig: { host: 'relay.expo.dev', secret: 'TOKENx', reconnecting: false },
},
},
turtleBuild: null,
});
expect(
await WorkflowJobSshQuery.connectInfoForWorkflowJobAsync(graphqlClient, 'job-1')
).toEqual({
sshRequested: true,
jobCompleted: false,
session: {
id: 'ts-1',
connectionConfig: { host: 'relay.expo.dev', secret: 'TOKENx', reconnecting: false },
},
});
expect(query).toHaveBeenCalledWith(
expect.anything(),
{ workflowJobId: 'job-1' },
{ requestPolicy: 'network-only' }
);
});
it('is not ssh-requested when the run has no sshSettings', async () => {
const { graphqlClient } = makeClient({
id: 'job-1',
status: WorkflowJobStatus.InProgress,
workflowRun: { sshSettings: null },
turtleJobRun: null,
turtleBuild: { sshSession: null },
});
expect(
await WorkflowJobSshQuery.connectInfoForWorkflowJobAsync(graphqlClient, 'job-1')
).toMatchObject({ sshRequested: false });
});
it('returns null when the job is not found', async () => {
const query = jest.fn().mockReturnValue({
toPromise: async () => ({
data: undefined,
error: new GraphqlError({
graphQLErrors: [
{ message: 'Entity not found', extensions: { errorCode: 'ENTITY_NOT_FOUND' } },
],
networkError: undefined,
response: undefined,
}),
}),
});
const graphqlClient = { query } as unknown as ExpoGraphqlClient;
expect(
await WorkflowJobSshQuery.connectInfoForWorkflowJobAsync(graphqlClient, 'missing')
).toBeNull();
});
it('returns null for not-found errors matched by message or code', async () => {
for (const graphQLErrors of [
[{ message: 'Workflow job not found', extensions: {} }],
[{ message: 'missing', extensions: { code: 'ENTITY_NOT_FOUND' } }],
]) {
const query = jest.fn().mockReturnValue({
toPromise: async () => ({
data: undefined,
error: new GraphqlError({
graphQLErrors,
networkError: undefined,
response: undefined,
}),
}),
});
const graphqlClient = { query } as unknown as ExpoGraphqlClient;
expect(
await WorkflowJobSshQuery.connectInfoForWorkflowJobAsync(graphqlClient, 'missing')
).toBeNull();
}
});
it('uses the turtleBuild session when there is no turtleJobRun', async () => {
const { graphqlClient } = makeClient({
id: 'job-1',
status: WorkflowJobStatus.Success,
workflowRun: { sshSettings: { idleTimeoutSeconds: 60 } },
turtleJobRun: null,
turtleBuild: {
sshSession: {
id: 'ts-build',
connectionConfig: { host: 'relay.expo.dev', secret: 'TOK', reconnecting: true },
},
},
});
expect(
await WorkflowJobSshQuery.connectInfoForWorkflowJobAsync(graphqlClient, 'job-1')
).toEqual({
sshRequested: true,
jobCompleted: true,
session: {
id: 'ts-build',
connectionConfig: { host: 'relay.expo.dev', secret: 'TOK', reconnecting: true },
},
});
});
it('rethrows GraphQL errors that are not not-found', async () => {
const error = new GraphqlError({
graphQLErrors: [{ message: 'boom', extensions: { errorCode: 'INTERNAL' } }],
networkError: undefined,
response: undefined,
});
const query = jest.fn().mockReturnValue({
toPromise: async () => ({ data: undefined, error }),
});
const graphqlClient = { query } as unknown as ExpoGraphqlClient;
await expect(
WorkflowJobSshQuery.connectInfoForWorkflowJobAsync(graphqlClient, 'job-1')
).rejects.toBe(error);
});
it('rethrows unexpected non-GraphQL errors', async () => {
const query = jest.fn().mockReturnValue({
toPromise: async () => {
throw new Error('network down');
},
});
const graphqlClient = { query } as unknown as ExpoGraphqlClient;
await expect(
WorkflowJobSshQuery.connectInfoForWorkflowJobAsync(graphqlClient, 'job-1')
).rejects.toThrow('network down');
});
});
-109
View File
@@ -1,109 +0,0 @@
import spawnAsync from '@expo/spawn-async';
import { runCommandAsync } from './runCommand';
import Log from '../log';
import { confirmAsync, promptAsync } from '../prompts';
export async function runGitCloneAsync({
targetProjectDir,
githubRepositoryName,
githubUsername,
cloneMethod,
showOutput = true,
}: {
githubUsername: string;
githubRepositoryName: string;
targetProjectDir: string;
cloneMethod: 'ssh' | 'https';
showOutput?: boolean;
}): Promise<{
targetProjectDir: string;
}> {
const url =
cloneMethod === 'ssh'
? `[email protected]:${githubUsername}/${githubRepositoryName}.git`
: `https://github.com/${githubUsername}/${githubRepositoryName}.git`;
try {
await runCommandAsync({
command: 'git',
args: ['clone', url, targetProjectDir],
shouldPrintStderrLineAsStdout: line => {
return line.includes('Cloning into');
},
showSpinner: false,
showOutput,
});
return { targetProjectDir };
} catch (error: any) {
if (error.stderr.includes('already exists')) {
Log.warn(`Directory ${targetProjectDir} already exists.`);
const shouldContinue = await confirmAsync({
message: 'Do you want to clone your project to some other destination?',
});
if (!shouldContinue) {
throw new Error('Directory already exists. Aborting...');
}
const { newTargetProjectDir } = await promptAsync({
type: 'text',
name: 'newTargetProjectDir',
message: 'New target directory path:',
validate: (input: string) => input !== '',
});
return await runGitCloneAsync({
githubRepositoryName,
githubUsername,
targetProjectDir: newTargetProjectDir,
cloneMethod,
});
} else if (error.stderr.includes('Permission denied')) {
Log.warn(
`It seems like you do not have permission to clone the repository using ${cloneMethod}.`
);
const newMethod = cloneMethod === 'ssh' ? 'https' : 'ssh';
const shouldContinue = await confirmAsync({
message: `Do you want to clone the repository using ${newMethod} instead?`,
});
if (!shouldContinue) {
throw new Error('Permission denied. Aborting...');
}
return await runGitCloneAsync({
githubRepositoryName,
githubUsername,
targetProjectDir,
cloneMethod: newMethod,
});
} else {
throw error;
}
}
}
export async function runGitPushAsync({
targetProjectDir,
}: {
targetProjectDir: string;
}): Promise<void> {
await runCommandAsync({
command: 'git',
args: ['push'],
cwd: targetProjectDir,
});
}
export async function canAccessRepositoryUsingSshAsync({
githubUsername,
githubRepositoryName,
}: {
githubUsername: string;
githubRepositoryName: string;
}): Promise<boolean> {
try {
await spawnAsync('git', [
'ls-remote',
`[email protected]:${githubUsername}/${githubRepositoryName}.git`,
]);
return true;
} catch {
return false;
}
}
+15
View File
@@ -110,3 +110,18 @@ export function ora(options?: Options | string): Ora {
return spinner;
}
/**
* Updates the text of a spinner. Does nothing if new text is the same as old text to prevent flickering.
*/
export function updateSpinnerText(
spinner: Ora,
{ prefixText, text }: { prefixText?: string; text?: string }
): void {
if (prefixText !== undefined && spinner.prefixText !== prefixText) {
spinner.prefixText = prefixText;
}
if (text !== undefined && spinner.text !== text) {
spinner.text = text;
}
}
@@ -0,0 +1,112 @@
import { compileModsAsync } from '@expo/config-plugins';
import { getPrebuildConfigAsync } from '@expo/prebuild-config';
import { spawnExpoCommand } from '../../../utils/expoCli';
import { Client } from '../../../vcs/vcs';
import { isExpoInstalled } from '../../projectUtils';
import { hasIgnoredIosProjectAsync } from '../../workflow';
import { getManagedApplicationTargetEntitlementsAsync } from '../entitlements';
jest.mock('@expo/config-plugins', () => ({
...jest.requireActual('@expo/config-plugins'),
compileModsAsync: jest.fn(),
}));
jest.mock('@expo/prebuild-config');
jest.mock('../../../log', () => ({
__esModule: true,
default: { warn: jest.fn() },
}));
jest.mock('../../../sentry', () => ({
__esModule: true,
default: {
withScope: jest.fn(),
captureMessage: jest.fn(),
},
}));
jest.mock('../../../utils/expoCli');
jest.mock('../../projectUtils');
jest.mock('../../workflow');
const projectDir = '/project';
const vcsClient = {} as Client;
const prebuildConfig = {
name: 'test-app',
slug: 'test-app',
ios: { bundleIdentifier: 'com.example.test' },
};
const entitlements = {
'aps-environment': 'production',
'com.apple.developer.associated-domains': ['applinks:example.com'],
};
beforeEach(() => {
jest.resetAllMocks();
jest.mocked(hasIgnoredIosProjectAsync).mockResolvedValue(true);
jest.mocked(getPrebuildConfigAsync).mockResolvedValue({ exp: prebuildConfig } as any);
jest.mocked(compileModsAsync).mockResolvedValue({
...prebuildConfig,
ios: { ...prebuildConfig.ios, entitlements },
} as any);
});
it('uses the bundled config fallback when Expo is not installed', async () => {
jest.mocked(isExpoInstalled).mockReturnValue(false);
const originalProcessEnv = process.env;
let envDuringPrebuild: string | undefined;
jest.mocked(getPrebuildConfigAsync).mockImplementation(async () => {
envDuringPrebuild = process.env.TEST_ENTITLEMENTS_ENV;
return { exp: prebuildConfig } as any;
});
await expect(
getManagedApplicationTargetEntitlementsAsync(
projectDir,
{ TEST_ENTITLEMENTS_ENV: 'from-build-profile' },
vcsClient
)
).resolves.toEqual(entitlements);
expect(spawnExpoCommand).not.toHaveBeenCalled();
expect(getPrebuildConfigAsync).toHaveBeenCalledWith(projectDir, { platforms: ['ios'] });
expect(compileModsAsync).toHaveBeenCalledWith(prebuildConfig, {
projectRoot: projectDir,
platforms: ['ios'],
introspect: true,
ignoreExistingNativeFiles: true,
});
expect(envDuringPrebuild).toBe('from-build-profile');
expect(process.env).toBe(originalProcessEnv);
});
it('uses the bundled config fallback when the local Expo CLI fails', async () => {
jest.mocked(isExpoInstalled).mockReturnValue(true);
jest
.mocked(spawnExpoCommand)
.mockRejectedValue(
Object.assign(new Error('local Expo CLI failed'), { stderr: 'config failed' })
);
await expect(
getManagedApplicationTargetEntitlementsAsync(projectDir, {}, vcsClient)
).resolves.toEqual(entitlements);
expect(spawnExpoCommand).toHaveBeenCalledWith(
projectDir,
['config', '--json', '--type', 'introspect'],
{ env: expect.objectContaining({ EXPO_NO_DOTENV: '1' }) }
);
expect(getPrebuildConfigAsync).toHaveBeenCalledWith(projectDir, { platforms: ['ios'] });
});
it('does not make Expo a required peer of the bundled config fallback', () => {
const packageJson = require('@expo/prebuild-config/package.json') as {
peerDependencies?: Record<string, string>;
peerDependenciesMeta?: Record<string, { optional?: boolean }>;
};
const hasRequiredExpoPeer =
packageJson.peerDependencies?.expo !== undefined &&
packageJson.peerDependenciesMeta?.expo?.optional !== true;
expect(hasRequiredExpoPeer).toBe(false);
});
@@ -1,6 +1,9 @@
import { DeviceRunSessionType } from '../../graphql/generated';
import { DeviceRunSessionResourceClass, DeviceRunSessionType } from '../../graphql/generated';
import {
DEVICE_RUN_SESSION_RESOURCE_CLASS_BY_FLAG_VALUE,
DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE,
DEVICE_RUN_SESSION_TYPE_FLAG_VALUES,
deviceRunSessionTypeToFlagValue,
formatRemoteSessionInstructions,
getRemoteSessionEnvironmentVariables,
} from '../utils';
@@ -34,6 +37,39 @@ describe('Appium simulator configuration', () => {
expect(instructions).toContain('eas simulator:exec <appium-client> [args...]');
expect(instructions).toContain('https://preview.example.test');
expect(instructions).toContain('Open the simulator preview:');
expect(instructions).not.toContain('iOS simulator preview');
expect(instructions).not.toContain('https://appium.example.test');
});
});
describe('simulator session type flags', () => {
it('maps web-preview-only to the WebPreviewOnly GraphQL enum', () => {
expect(DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE['web-preview-only']).toBe(
DeviceRunSessionType.WebPreviewOnly
);
expect(DEVICE_RUN_SESSION_TYPE_FLAG_VALUES[DeviceRunSessionType.ServeSim]).toBe(
'web-preview-only'
);
expect(DEVICE_RUN_SESSION_TYPE_FLAG_VALUES[DeviceRunSessionType.WebPreviewOnly]).toBe(
'web-preview-only'
);
expect(deviceRunSessionTypeToFlagValue(DeviceRunSessionType.ServeSim)).toBe('web-preview-only');
expect(deviceRunSessionTypeToFlagValue(DeviceRunSessionType.WebPreviewOnly)).toBe(
'web-preview-only'
);
expect(DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE['serve-sim']).toBeUndefined();
expect(DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE['web-preview']).toBeUndefined();
});
});
describe('simulator resource class flags', () => {
it('maps CLI values to the GraphQL enum', () => {
expect(DEVICE_RUN_SESSION_RESOURCE_CLASS_BY_FLAG_VALUE.large).toBe(
DeviceRunSessionResourceClass.Large
);
expect(DEVICE_RUN_SESSION_RESOURCE_CLASS_BY_FLAG_VALUE.medium).toBe(
DeviceRunSessionResourceClass.Medium
);
});
});
+32 -3
View File
@@ -1,4 +1,8 @@
import { DeviceRunSessionByIdQuery, DeviceRunSessionType } from '../graphql/generated';
import {
DeviceRunSessionByIdQuery,
DeviceRunSessionResourceClass,
DeviceRunSessionType,
} from '../graphql/generated';
import { link } from '../log';
type DeviceRunSessionByIdResult = DeviceRunSessionByIdQuery['deviceRunSessions']['byId'];
@@ -24,7 +28,8 @@ export const DEVICE_RUN_SESSION_TYPE_FLAG_VALUES: Record<DeviceRunSessionType, s
[DeviceRunSessionType.AgentDevice]: 'agent-device',
[DeviceRunSessionType.Appium]: 'appium',
[DeviceRunSessionType.Argent]: 'argent',
[DeviceRunSessionType.ServeSim]: 'serve-sim',
[DeviceRunSessionType.ServeSim]: 'web-preview-only',
[DeviceRunSessionType.WebPreviewOnly]: 'web-preview-only',
};
export const DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE = Object.fromEntries(
@@ -37,6 +42,23 @@ export function deviceRunSessionTypeToFlagValue(type: DeviceRunSessionType): str
return DEVICE_RUN_SESSION_TYPE_FLAG_VALUES[type];
}
export const DEVICE_RUN_SESSION_RESOURCE_CLASS_FLAG_VALUES: Record<
DeviceRunSessionResourceClass,
string
> = {
[DeviceRunSessionResourceClass.Large]: 'large',
[DeviceRunSessionResourceClass.Medium]: 'medium',
};
export const DEVICE_RUN_SESSION_RESOURCE_CLASS_BY_FLAG_VALUE = Object.fromEntries(
(
Object.entries(DEVICE_RUN_SESSION_RESOURCE_CLASS_FLAG_VALUES) as [
DeviceRunSessionResourceClass,
string,
][]
).map(([resourceClass, value]) => [value, resourceClass])
) as Record<string, DeviceRunSessionResourceClass>;
export function getRemoteSessionEnvironmentVariables(
remoteConfig: DeviceRunSessionRemoteConfig
): Record<string, string> {
@@ -57,6 +79,7 @@ export function getRemoteSessionEnvironmentVariables(
APPIUM_CAPS: JSON.stringify(remoteConfig.capabilities),
};
case 'ServeSimRunSessionRemoteConfig':
case 'WebPreviewOnlyRunSessionRemoteConfig':
return {};
}
}
@@ -151,7 +174,7 @@ export function formatRemoteSessionInstructions(
'<appium-client> [args...]',
];
if (remoteConfig.webPreviewUrl) {
lines.push('', 'Open the iOS simulator preview:', '', remoteConfig.webPreviewUrl);
lines.push('', 'Open the simulator preview:', '', remoteConfig.webPreviewUrl);
}
return lines.join('\n');
}
@@ -161,5 +184,11 @@ export function formatRemoteSessionInstructions(
'',
remoteConfig.previewUrl,
].join('\n');
case 'WebPreviewOnlyRunSessionRemoteConfig':
return [
'🌐 Open the following URL in your browser to access the simulator:',
'',
remoteConfig.previewUrl,
].join('\n');
}
}
@@ -0,0 +1,26 @@
import { Centrifuge } from 'centrifuge';
import { createRealtimeLogsClient } from '../centrifuge';
jest.mock('centrifuge');
describe(createRealtimeLogsClient, () => {
afterEach(() => {
jest.clearAllMocks();
});
it('returns null instead of throwing when the connection cannot be created', () => {
jest.mocked(Centrifuge).mockImplementationOnce(() => {
throw new Error('unsupported url scheme');
});
expect(createRealtimeLogsClient({} as any)).toBeNull();
});
it('connects and returns a client when the connection can be created', () => {
expect(createRealtimeLogsClient({} as any)).not.toBeNull();
const centrifuge = jest.mocked(Centrifuge).mock.instances[0];
expect(jest.mocked(centrifuge.connect)).toHaveBeenCalled();
});
});
+117
View File
@@ -0,0 +1,117 @@
import { Centrifuge, Subscription } from 'centrifuge';
import { Agent } from 'https';
import WebSocket from 'ws';
import { getEASLogsWebsocketUrl } from '../api';
import { ExpoGraphqlClient } from '../commandUtils/context/contextUtils/createGraphqlClient';
import { RealtimeLogsMutationGenerateCentrifugoSubscriptionTokenArgs } from '../graphql/generated';
import { RealtimeLogsMutation } from '../graphql/mutations/RealtimeLogsMutation';
import { httpsProxyAgent } from '../fetch';
import Log from '../log';
export type RealtimeLogsSubscription = {
close: () => void;
};
export type RealtimeLogsClient = {
subscribeAsync: (
args: RealtimeLogsMutationGenerateCentrifugoSubscriptionTokenArgs,
onPublication: (data: unknown) => void
) => Promise<RealtimeLogsSubscription | null>;
close: () => void;
};
type WebSocketConstructor = new (address: string, protocols?: string | string[]) => WebSocket;
function createProxiedWebSocketConstructor(agent: Agent): WebSocketConstructor {
return class ProxiedWebSocket extends WebSocket {
constructor(address: string | URL, protocols?: string | string[]) {
super(address, protocols, { agent });
}
};
}
function createWebSocketConstructor(): WebSocketConstructor {
return httpsProxyAgent ? createProxiedWebSocketConstructor(httpsProxyAgent) : WebSocket;
}
export function createRealtimeLogsClient(
graphqlClient: ExpoGraphqlClient
): RealtimeLogsClient | null {
const getTokenAsync = async (): Promise<string> => {
const { token } =
await RealtimeLogsMutation.generateCentrifugoConnectionTokenAsync(graphqlClient);
return token;
};
let client: Centrifuge;
try {
client = new Centrifuge(getEASLogsWebsocketUrl(), {
websocket: createWebSocketConstructor(),
getToken: getTokenAsync,
});
client.on('error', ({ error }) => {
Log.debug(`Realtime logs connection error: ${error.message}`);
});
client.connect();
} catch (err: any) {
Log.debug(`Failed to connect to realtime logs: ${err.message}`);
return null;
}
return {
subscribeAsync: async (args, onPublication) => {
const getSubscriptionTokenAsync = async (): Promise<string> => {
const { token } = await RealtimeLogsMutation.generateCentrifugoSubscriptionTokenAsync(
graphqlClient,
args
);
return token;
};
try {
const { channel, token } =
await RealtimeLogsMutation.generateCentrifugoSubscriptionTokenAsync(graphqlClient, args);
const subscription = client.newSubscription(channel, {
token,
getToken: getSubscriptionTokenAsync,
});
subscription.on('error', ({ error }) => {
Log.debug(`Realtime logs subscription error on ${channel}: ${error.message}`);
});
subscription.on('publication', ({ data }) => {
onPublication(data);
});
subscription.subscribe();
return {
close: () => {
closeSubscription(client, subscription);
},
};
} catch (err: any) {
Log.debug(`Failed to subscribe to realtime logs: ${err.message}`);
return null;
}
},
close: () => {
for (const subscription of Object.values(client.subscriptions())) {
closeSubscription(client, subscription);
}
client.removeAllListeners();
client.disconnect();
},
};
}
function closeSubscription(client: Centrifuge, subscription: Subscription): void {
try {
subscription.unsubscribe();
subscription.removeAllListeners();
client.removeSubscription(subscription);
} catch (err: any) {
Log.debug(`Failed to close realtime logs subscription: ${err.message}`);
}
}
@@ -18,4 +18,12 @@ describe(uniqBy, () => {
{ a: 4, b: 12 },
]);
});
it('calls getKey once per item', () => {
const getKey = jest.fn(({ a }: { a: number }) => a);
uniqBy([{ a: 1 }, { a: 2 }, { a: 2 }], getKey);
expect(getKey).toHaveBeenCalledTimes(3);
});
});
@@ -1,10 +1,11 @@
export default function uniqBy<T, K = any>(list: T[], getKey: (item: T) => K): T[] {
const uniqueValues = new Set();
const uniqueValues = new Set<K>();
const result: T[] = [];
for (const i of list) {
if (!uniqueValues.has(getKey(i))) {
const key = getKey(i);
if (!uniqueValues.has(key)) {
result.push(i);
uniqueValues.add(getKey(i));
uniqueValues.add(key);
}
}
return result;
@@ -0,0 +1,70 @@
import fs from 'node:fs';
import { Readable } from 'node:stream';
import { extract } from 'tar-stream';
import zlib from 'node:zlib';
import { FileEntry, packFilesIterableAsync } from '../assets';
async function extractTarEntriesAsync(tarData: Buffer): Promise<Map<string, string>> {
const entries = new Map<string, string>();
const extractor = extract();
extractor.on('entry', (header, stream, next) => {
const chunks: Buffer[] = [];
stream.on('data', chunk => chunks.push(chunk));
stream.on('end', () => {
entries.set(header.name, Buffer.concat(chunks).toString('utf8'));
next();
});
stream.resume();
});
await new Promise<void>((resolve, reject) => {
extractor.on('finish', resolve);
extractor.on('error', reject);
Readable.from(tarData).pipe(extractor);
});
return entries;
}
describe(packFilesIterableAsync, () => {
let tarPath: string | null = null;
afterEach(async () => {
if (tarPath) {
await fs.promises.rm(tarPath, { force: true });
tarPath = null;
}
});
it('packs file entries into a gzip-compressed tarball', async () => {
const files: FileEntry[] = [
['index.html', '<h1>hello</h1>'],
['assets/data.json', '{"key":"value"}'],
];
tarPath = await packFilesIterableAsync(files);
const compressed = await fs.promises.readFile(tarPath);
// gzip magic bytes
expect(compressed[0]).toBe(0x1f);
expect(compressed[1]).toBe(0x8b);
// portable mode zeroes the gzip MTIME field (bytes 4-7)
expect(compressed.subarray(4, 8)).toEqual(Buffer.from([0, 0, 0, 0]));
const entries = await extractTarEntriesAsync(zlib.gunzipSync(compressed));
expect(entries.get('index.html')).toBe('<h1>hello</h1>');
expect(entries.get('assets/data.json')).toBe('{"key":"value"}');
expect(entries.size).toBe(2);
});
it('accepts async iterables and forwards gzip options', async () => {
async function* files(): AsyncIterable<FileEntry> {
yield ['a.txt', 'aaaa'];
yield ['b.txt', Buffer.from('bbbb')];
}
tarPath = await packFilesIterableAsync(files(), { level: 9 });
const compressed = await fs.promises.readFile(tarPath);
const entries = await extractTarEntriesAsync(zlib.gunzipSync(compressed));
expect(entries.get('a.txt')).toBe('aaaa');
expect(entries.get('b.txt')).toBe('bbbb');
});
});
@@ -0,0 +1,443 @@
import cliProgress from 'cli-progress';
import fetch, { Headers, Response } from 'node-fetch';
import Log from '../../log';
import { AssetFileEntry } from '../assets';
import {
UploadPayload,
batchUploadAsync,
callUploadApiAsync,
createProgressBar,
uploadAsync,
} from '../upload';
jest.mock('node-fetch', () => ({
...jest.requireActual('node-fetch'),
__esModule: true,
default: jest.fn(),
}));
const mockedFetch = jest.mocked(fetch);
const asset: AssetFileEntry = {
normalizedPath: 'index.html',
path: __filename,
size: 123,
sha512: 'a'.repeat(128),
type: 'text/html',
};
const response = (status = 200, body: unknown = { success: true }): Response =>
new Response(JSON.stringify(body), {
status,
statusText: status === 200 ? 'OK' : 'Request failed',
headers: { 'content-type': 'application/json' },
});
async function runTimersAndExpectRejection(
promise: Promise<unknown>,
expected: string | RegExp
): Promise<void> {
const expectation = expect(promise).rejects.toThrow(expected);
await jest.runAllTimersAsync();
await expectation;
}
describe(uploadAsync, () => {
beforeEach(() => {
jest.useFakeTimers({ now: 0 });
jest.spyOn(Math, 'random').mockReturnValue(0);
jest.spyOn(Log, 'warn').mockImplementation(() => {});
mockedFetch.mockReset();
});
afterEach(() => {
jest.useRealTimers();
jest.restoreAllMocks();
});
it('uploads an individual asset with its hash and metadata', async () => {
mockedFetch.mockResolvedValue(response());
const result = await uploadAsync({ baseURL: 'https://eas.expo.app/?token=secret' }, { asset });
expect(result.payload).toEqual({ asset });
expect(mockedFetch).toHaveBeenCalledTimes(1);
const [url, init] = mockedFetch.mock.calls[0];
expect(String(url)).toBe(`https://eas.expo.app/asset/${asset.sha512}?token=secret`);
expect(init?.method).toBe('POST');
expect((init?.headers as Headers).get('content-type')).toBe('text/html');
expect((init?.headers as Headers).get('content-length')).toBe('123');
expect(init?.body).toBeDefined();
});
it('uploads a worker deployment file without changing the URL', async () => {
mockedFetch.mockResolvedValue(response());
await uploadAsync({ baseURL: 'https://eas.expo.app/deploy' }, { filePath: __filename });
const [url, init] = mockedFetch.mock.calls[0];
expect(String(url)).toBe('https://eas.expo.app/deploy');
expect(init?.method).toBe('POST');
expect(init?.body).toBeDefined();
});
it('uploads multipart assets to the batch endpoint and reports progress', async () => {
mockedFetch.mockResolvedValue(response());
const onProgress = jest.fn();
await uploadAsync(
{ baseURL: 'https://eas.expo.app/asset/?token=secret' },
{ multipart: [asset, { ...asset, sha512: 'b'.repeat(128) }] },
onProgress
);
const [url, init] = mockedFetch.mock.calls[0];
expect(String(url)).toBe('https://eas.expo.app/asset/batch?token=secret');
expect(init?.method).toBe('PATCH');
expect((init?.headers as Headers).get('content-type')).toMatch(/^multipart\/form-data;/);
expect(onProgress).toHaveBeenNthCalledWith(1, 0);
expect(onProgress).toHaveBeenLastCalledWith(1);
});
it.each([408, 409, 429, 500, 503])('retries HTTP %s responses', async status => {
mockedFetch.mockImplementation(async () => response(status, { error: `status ${status}` }));
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset });
await runTimersAndExpectRejection(promise, `status ${status}`);
expect(mockedFetch).toHaveBeenCalledTimes(5);
});
it('does not retry a terminal HTTP response', async () => {
mockedFetch.mockResolvedValue(response(400, { error: 'invalid upload' }));
await expect(uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset })).rejects.toThrow(
'invalid upload'
);
expect(mockedFetch).toHaveBeenCalledTimes(1);
});
it('reports payload-too-large responses without retrying', async () => {
mockedFetch.mockResolvedValue(response(413));
await expect(uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset })).rejects.toThrow(
'File size exceeded the upload limit'
);
expect(mockedFetch).toHaveBeenCalledTimes(1);
});
it('reports CDN HTML errors with the request ID', async () => {
mockedFetch.mockResolvedValue(
new Response('<html></html>', {
status: 403,
statusText: 'Forbidden',
headers: { 'content-type': 'text/html', 'cf-ray': 'ray-id' },
})
);
await expect(uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset })).rejects.toThrow(
'Request ID ray-id'
);
});
it('switches to network mode and preserves the original network error', async () => {
const networkError = Object.assign(new Error('getaddrinfo ENOTFOUND eas.expo.app'), {
code: 'ENOTFOUND',
});
mockedFetch.mockRejectedValue(networkError);
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset });
await runTimersAndExpectRejection(promise, networkError.message);
expect(mockedFetch).toHaveBeenCalledTimes(8);
expect(Log.warn).toHaveBeenCalledTimes(1);
expect(Log.warn).toHaveBeenCalledWith(
`The upload encountered an error but is still retrying: ${networkError.message}`
);
});
it('smoothly increases retries based on the total request count', async () => {
mockedFetch.mockImplementation(async () => response(503, { error: 'overloaded' }));
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
totalRequests: 32,
});
await runTimersAndExpectRejection(promise, 'overloaded');
// log10(32) / 3 results in six retries, plus the initial request.
expect(mockedFetch).toHaveBeenCalledTimes(7);
});
it('caps request-count scaling at 1,000 requests', async () => {
mockedFetch.mockImplementation(async () => response(503, { error: 'overloaded' }));
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
totalRequests: 1_000_000,
});
await runTimersAndExpectRejection(promise, 'overloaded');
// The scaled 40s ceiling cuts the last attempt the backoff schedule would have allowed.
expect(mockedFetch).toHaveBeenCalledTimes(8);
});
it('keeps the retry state when it switches to network mode', async () => {
let calls = 0;
mockedFetch.mockImplementation(async () => {
if (++calls <= 7) {
return response(503, { error: 'overloaded' });
}
throw new Error('socket disconnected');
});
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
totalRequests: 1_000,
});
await runTimersAndExpectRejection(promise, 'socket disconnected');
// The warning must not repeat once the network-mode attempts take over.
expect(Log.warn).toHaveBeenCalledTimes(1);
expect(Log.warn).toHaveBeenCalledWith(
'The upload encountered an error but is still retrying: overloaded'
);
});
it('warns before a short retry budget is exhausted', async () => {
mockedFetch.mockImplementation(async () => response(503, { error: 'overloaded' }));
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset });
await runTimersAndExpectRejection(promise, 'overloaded');
expect(Log.warn).toHaveBeenCalledTimes(1);
});
it('stops retrying at the scaled upload ceiling', async () => {
mockedFetch.mockImplementation(async () => response(503, { error: 'overloaded' }));
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
totalRequests: 1_000,
});
await runTimersAndExpectRejection(promise, 'overloaded');
// The ceiling stops the next retry from being scheduled, so the last attempt already in
// flight can overshoot it by up to one maxTimeout (10s at this scale).
expect(Date.now()).toBeLessThanOrEqual(40_000 + 10_000);
});
it('clears the network flag after a success', async () => {
const state = { shared: { hasSeenNetworkError: false, hasWarned: false } };
mockedFetch.mockResolvedValueOnce(response(503)).mockResolvedValueOnce(response());
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
state,
});
await jest.runAllTimersAsync();
await promise;
expect(state.shared.hasSeenNetworkError).toBe(false);
});
it('keeps retry warning clocks independent when retry state is shared', async () => {
const shared = { hasSeenNetworkError: false, hasWarned: false };
mockedFetch
.mockResolvedValueOnce(response(503, { error: 'overloaded' }))
.mockResolvedValueOnce(response())
.mockImplementation(async () => response(503, { error: 'overloaded' }));
const retryingUpload = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
state: { shared },
});
await uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
state: { shared },
});
await runTimersAndExpectRejection(retryingUpload, 'overloaded');
expect(Log.warn).toHaveBeenCalledTimes(1);
});
it('does not retry or enter network mode after cancellation', async () => {
const controller = new AbortController();
controller.abort();
const abortError = Object.assign(new Error('The operation was aborted'), {
name: 'AbortError',
});
mockedFetch.mockRejectedValue(abortError);
await expect(
uploadAsync({ baseURL: 'https://eas.expo.app', signal: controller.signal }, { asset })
).rejects.toBe(abortError);
expect(mockedFetch).toHaveBeenCalledTimes(1);
expect(Log.warn).not.toHaveBeenCalled();
});
});
describe(callUploadApiAsync, () => {
beforeEach(() => {
jest.useFakeTimers({ now: 0 });
jest.spyOn(Math, 'random').mockReturnValue(0);
jest.spyOn(Log, 'warn').mockImplementation(() => {});
mockedFetch.mockReset();
});
afterEach(() => {
jest.useRealTimers();
jest.restoreAllMocks();
});
it('returns a successful JSON response', async () => {
mockedFetch.mockResolvedValue(response(200, { result: 'complete' }));
await expect(callUploadApiAsync('https://eas.expo.app/finalize')).resolves.toEqual({
result: 'complete',
});
});
it('retries server errors and passes through the final error', async () => {
mockedFetch.mockResolvedValue(response(503));
const promise = callUploadApiAsync('https://eas.expo.app/finalize');
await runTimersAndExpectRejection(promise, 'Deployment failed: Request failed');
// Deploy API calls get a somewhat larger retry budget than a single asset upload.
expect(mockedFetch).toHaveBeenCalledTimes(7);
});
it('retries invalid JSON responses', async () => {
mockedFetch
.mockResolvedValueOnce(new Response('invalid JSON'))
.mockResolvedValueOnce(response(200, { success: true }));
const promise = callUploadApiAsync('https://eas.expo.app/finalize');
await jest.runAllTimersAsync();
await expect(promise).resolves.toEqual({ success: true });
expect(mockedFetch).toHaveBeenCalledTimes(2);
});
it('uses network retries for fetch errors', async () => {
const networkError = new Error('socket disconnected');
mockedFetch.mockRejectedValue(networkError);
const promise = callUploadApiAsync('https://eas.expo.app/finalize');
await runTimersAndExpectRejection(promise, networkError.message);
// Network mode adds attempts but shares the 30s deadline, so it cannot double the wait.
expect(mockedFetch).toHaveBeenCalledTimes(10);
});
it('stops retrying at the 30s ceiling', async () => {
mockedFetch.mockResolvedValue(response(503));
const promise = callUploadApiAsync('https://eas.expo.app/finalize');
await runTimersAndExpectRejection(promise, 'Deployment failed: Request failed');
expect(Date.now()).toBeLessThanOrEqual(30_000);
});
it('shares the ceiling with the network-mode attempts', async () => {
mockedFetch.mockRejectedValue(new Error('socket disconnected'));
const promise = callUploadApiAsync('https://eas.expo.app/finalize');
await runTimersAndExpectRejection(promise, 'socket disconnected');
expect(Date.now()).toBeLessThanOrEqual(30_000 + 5_000);
});
it('does not retry an aborted request', async () => {
const controller = new AbortController();
controller.abort();
const abortError = Object.assign(new Error('The operation was aborted'), {
name: 'AbortError',
});
mockedFetch.mockRejectedValue(abortError);
await expect(
callUploadApiAsync('https://eas.expo.app/finalize', { signal: controller.signal as any })
).rejects.toBe(abortError);
expect(mockedFetch).toHaveBeenCalledTimes(1);
});
});
describe(batchUploadAsync, () => {
beforeEach(() => {
mockedFetch.mockReset();
});
afterEach(() => {
jest.restoreAllMocks();
});
it('uploads every payload and reports aggregate progress', async () => {
mockedFetch.mockResolvedValue(response());
const payloads: UploadPayload[] = [
{ asset },
{ asset: { ...asset, sha512: 'b'.repeat(128) } },
{ asset: { ...asset, sha512: 'c'.repeat(128) } },
];
const progress = jest.fn();
const results = [];
for await (const result of batchUploadAsync(
{ baseURL: 'https://eas.expo.app' },
payloads,
progress
)) {
results.push(result);
}
expect(mockedFetch).toHaveBeenCalledTimes(3);
expect(results.at(-1)?.progress).toBe(1);
expect(progress).toHaveBeenLastCalledWith(1);
});
it('accepts an empty upload queue', async () => {
const results = [];
for await (const result of batchUploadAsync({ baseURL: 'https://eas.expo.app' }, [])) {
results.push(result);
}
expect(results).toEqual([]);
expect(mockedFetch).not.toHaveBeenCalled();
});
it('aborts sibling requests after the first upload failure', async () => {
mockedFetch.mockImplementation(async (url, init) => {
if (String(url).includes(asset.sha512)) {
return response(400, { error: 'terminal failure' });
}
return await new Promise((_resolve, reject) => {
init?.signal?.addEventListener('abort', () => {
reject(new Error('aborted'));
});
});
});
const iterator = batchUploadAsync({ baseURL: 'https://eas.expo.app' }, [
{ asset },
{ asset: { ...asset, sha512: 'b'.repeat(128) } },
]);
await iterator.next();
await iterator.next();
await expect(iterator.next()).rejects.toThrow('terminal failure');
});
});
describe(createProgressBar, () => {
afterEach(() => {
jest.restoreAllMocks();
});
it('starts, updates, and stops the upload progress bar', () => {
const start = jest.spyOn(cliProgress.SingleBar.prototype, 'start').mockImplementation(() => {});
const update = jest
.spyOn(cliProgress.SingleBar.prototype, 'update')
.mockImplementation(() => {});
const stop = jest.spyOn(cliProgress.SingleBar.prototype, 'stop').mockImplementation(() => {});
const progressBar = createProgressBar('Uploading 3 assets');
progressBar.update(0.5);
progressBar.stop();
expect(start).toHaveBeenCalledWith(1, 0);
expect(update).toHaveBeenCalledWith(0.5);
expect(stop).toHaveBeenCalledTimes(1);
});
});
+252 -106
View File
@@ -7,6 +7,7 @@ import os from 'node:os';
import { Readable } from 'node:stream';
import promiseRetry from 'promise-retry';
import Log from '../log';
import { AssetFileEntry } from './assets';
import {
createMultipartBodyFromFilesAsync,
@@ -15,7 +16,86 @@ import {
} from './utils/multipart';
const MAX_CONCURRENCY = Math.min(10, Math.max(os.availableParallelism() * 2, 20));
const MAX_RETRIES = 4;
const MAX_RETRY_WARNING_DELAY_MS = 30_000;
const UPLOAD_RETRY_LIMITS: RetryLimits = { retries: 4, maxTimeout: 5_000, maxRetryTime: 20_000 };
const API_RETRY_LIMITS: RetryLimits = { retries: 6, maxTimeout: 5_000, maxRetryTime: 30_000 };
interface SharedRetryState {
hasSeenNetworkError: boolean;
hasWarned: boolean;
}
interface RetryState {
shared: SharedRetryState;
firstRetryAt?: number;
}
interface UploadRetryOptions {
totalRequests?: number;
state?: RetryState;
deadlineAt?: number;
}
interface RetryLimits {
retries: number;
maxTimeout: number;
maxRetryTime: number;
}
interface RetryOptions {
retries: number;
factor: number;
minTimeout: number;
maxTimeout: number;
maxRetryTime: number;
randomize: boolean;
}
interface RetryWarningContext {
state: RetryState;
warningDelayMs: number;
subject: string;
}
const getRetryScale = (totalRequests: number): number =>
Math.min(1, Math.log10(Math.max(1, totalRequests)) / 3);
const getErrorMessage = (error: unknown): string =>
error instanceof Error ? error.message : String(error);
const getRetryOptions = (
limits: RetryLimits,
totalRequests: number,
networkMode: boolean
): RetryOptions => {
const scale = getRetryScale(totalRequests);
return {
retries: Math.round(limits.retries * (1 + scale)) + (networkMode ? 2 : 0),
factor: 2,
minTimeout: 1_000,
maxTimeout: Math.round(limits.maxTimeout * (1 + scale)),
maxRetryTime: Math.round(limits.maxRetryTime * (1 + scale)),
randomize: true,
};
};
const getRetryWarningDelay = (maxRetryTime: number): number =>
Math.min(MAX_RETRY_WARNING_DELAY_MS, Math.round(maxRetryTime / 2));
const retryWithWarning = (
retry: (error: unknown) => never,
error: unknown,
attempt: number,
{ state, warningDelayMs, subject }: RetryWarningContext
): never => {
state.firstRetryAt ??= Date.now();
if (!state.shared.hasWarned && attempt > 1 && Date.now() - state.firstRetryAt >= warningDelayMs) {
state.shared.hasWarned = true;
Log.warn(`${subject} encountered an error but is still retrying: ${getErrorMessage(error)}`);
}
return retry(error);
};
export type UploadPayload =
| { filePath: string }
@@ -56,110 +136,156 @@ type OnProgressUpdateCallback = (progress: number) => void;
export async function uploadAsync(
init: UploadRequestInit,
payload: UploadPayload,
onProgressUpdate?: OnProgressUpdateCallback
onProgressUpdate?: OnProgressUpdateCallback,
retryOptions: UploadRetryOptions = {},
networkMode = retryOptions.state?.shared.hasSeenNetworkError ?? false
): Promise<UploadResult> {
return await promiseRetry(
async retry => {
if (onProgressUpdate) {
onProgressUpdate(0);
}
const headers = new Headers(init.headers);
const url = new URL(`${init.baseURL}`);
let errorPrefix: string;
let body: BodyInit | undefined;
let method = init.method || 'POST';
if ('asset' in payload) {
const { asset } = payload;
errorPrefix = `Upload of "${asset.normalizedPath}" failed`;
if (asset.type) {
headers.set('content-type', asset.type);
}
if (asset.size) {
headers.set('content-length', `${asset.size}`);
}
method = 'POST';
url.pathname = `/asset/${asset.sha512}`;
body = Readable.from(createReadStreamAsync(asset), { objectMode: false });
} else if ('filePath' in payload) {
const { filePath } = payload;
errorPrefix = 'Worker deployment failed';
body = fs.createReadStream(filePath);
} else if ('multipart' in payload) {
const { multipart } = payload;
errorPrefix = `Upload of ${multipart.length} assets failed`;
headers.set('content-type', multipartContentType);
method = 'PATCH';
url.pathname = '/asset/batch';
body = Readable.from(createMultipartBodyFromFilesAsync(multipart, onProgressUpdate), {
objectMode: false,
});
}
let response: Response;
try {
response = await fetch(url, {
method,
body,
headers,
agent: getAgent(),
signal: init.signal as any,
});
} catch (error) {
return retry(error);
}
const getErrorMessageAsync = async (): Promise<string> => {
const rayId = response.headers.get('cf-ray');
const contentType = response.headers.get('Content-Type');
if (contentType?.startsWith('text/html')) {
// NOTE(@kitten): We've received a CDN error most likely. There's not much we can do
// except for quoting the Request ID, so a user can send it to us. We can check
// why a request was blocked by looking up a WAF event via the "Ray ID" here:
// https://dash.cloudflare.com/e6f39f67f543faa6038768e8f37e4234/expo.app/security/events
let message = `CDN firewall has aborted the upload with ${response.statusText}.`;
if (rayId) {
message += `\nReport this error quoting Request ID ${rayId}`;
}
return `${errorPrefix}: ${message}`;
} else {
const json = await response.json().catch(() => null);
return json?.error ?? `${errorPrefix}: ${response.statusText}`;
}
};
if (
response.status === 408 ||
response.status === 409 ||
response.status === 429 ||
(response.status >= 500 && response.status <= 599)
) {
return retry(new Error(await getErrorMessageAsync()));
} else if (response.status === 413) {
const message = `${errorPrefix!}: File size exceeded the upload limit`;
throw new Error(message);
} else if (!response.ok) {
throw new Error(await getErrorMessageAsync());
} else if (onProgressUpdate) {
onProgressUpdate(1);
}
return {
payload,
response,
};
},
{
retries: MAX_RETRIES,
minTimeout: 50,
randomize: false,
}
const state = retryOptions.state ?? {
shared: { hasSeenNetworkError: false, hasWarned: false },
};
const retryOptionsForAttempts = getRetryOptions(
UPLOAD_RETRY_LIMITS,
retryOptions.totalRequests ?? 1,
networkMode
);
const warningDelayMs = getRetryWarningDelay(retryOptionsForAttempts.maxRetryTime);
const deadlineAt = retryOptions.deadlineAt ?? Date.now() + retryOptionsForAttempts.maxRetryTime;
retryOptionsForAttempts.maxRetryTime = Math.max(0, deadlineAt - Date.now());
const warningContext: RetryWarningContext = {
state,
warningDelayMs,
subject: 'The upload',
};
return await promiseRetry(async (retry, attempt) => {
if (onProgressUpdate) {
onProgressUpdate(0);
}
const headers = new Headers(init.headers);
const url = new URL(`${init.baseURL}`);
let errorPrefix: string;
let body: BodyInit | undefined;
let method = init.method || 'POST';
if ('asset' in payload) {
const { asset } = payload;
errorPrefix = `Upload of "${asset.normalizedPath}" failed`;
if (asset.type) {
headers.set('content-type', asset.type);
}
if (asset.size) {
headers.set('content-length', `${asset.size}`);
}
method = 'POST';
url.pathname = `/asset/${asset.sha512}`;
body = Readable.from(createReadStreamAsync(asset), { objectMode: false });
} else if ('filePath' in payload) {
const { filePath } = payload;
errorPrefix = 'Worker deployment failed';
body = fs.createReadStream(filePath);
} else if ('multipart' in payload) {
const { multipart } = payload;
errorPrefix = `Upload of ${multipart.length} assets failed`;
headers.set('content-type', multipartContentType);
method = 'PATCH';
url.pathname = '/asset/batch';
body = Readable.from(createMultipartBodyFromFilesAsync(multipart, onProgressUpdate), {
objectMode: false,
});
}
let response: Response;
try {
response = await fetch(url, {
method,
body,
headers,
agent: getAgent(),
signal: init.signal as any,
});
} catch (error) {
if (init.signal?.aborted) {
throw error;
}
if (!networkMode) {
state.shared.hasSeenNetworkError = true;
return await uploadAsync(
init,
payload,
onProgressUpdate,
{ ...retryOptions, state, deadlineAt },
true
);
}
return retryWithWarning(retry, error, attempt, warningContext);
}
const getErrorMessageAsync = async (): Promise<string> => {
const rayId = response.headers.get('cf-ray');
const contentType = response.headers.get('Content-Type');
if (contentType?.startsWith('text/html')) {
// NOTE(@kitten): We've received a CDN error most likely. There's not much we can do
// except for quoting the Request ID, so a user can send it to us. We can check
// why a request was blocked by looking up a WAF event via the "Ray ID" here:
// https://dash.cloudflare.com/e6f39f67f543faa6038768e8f37e4234/expo.app/security/events
let message = `CDN firewall has aborted the upload with ${response.statusText}.`;
if (rayId) {
message += `\nReport this error quoting Request ID ${rayId}`;
}
return `${errorPrefix}: ${message}`;
} else {
const json = await response.json().catch(() => null);
return json?.error ?? `${errorPrefix}: ${response.statusText}`;
}
};
if (
response.status === 408 ||
response.status === 409 ||
response.status === 429 ||
(response.status >= 500 && response.status <= 599)
) {
return retryWithWarning(
retry,
new Error(await getErrorMessageAsync()),
attempt,
warningContext
);
} else if (response.status === 413) {
const message = `${errorPrefix!}: File size exceeded the upload limit`;
throw new Error(message);
} else if (!response.ok) {
throw new Error(await getErrorMessageAsync());
} else if (onProgressUpdate) {
onProgressUpdate(1);
}
state.shared.hasSeenNetworkError = false;
return {
payload,
response,
};
}, retryOptionsForAttempts);
}
export async function callUploadApiAsync(url: string | URL, init?: RequestInit): Promise<unknown> {
return await promiseRetry(async retry => {
async function callUploadApiWithRetryAsync(
url: string | URL,
init: RequestInit | undefined,
networkMode = false,
state: RetryState = { shared: { hasSeenNetworkError: false, hasWarned: false } },
deadlineAt?: number
): Promise<unknown> {
const retryOptions = getRetryOptions(API_RETRY_LIMITS, 1, networkMode);
const warningDelayMs = getRetryWarningDelay(retryOptions.maxRetryTime);
const retryDeadlineAt = deadlineAt ?? Date.now() + retryOptions.maxRetryTime;
retryOptions.maxRetryTime = Math.max(0, retryDeadlineAt - Date.now());
const warningContext: RetryWarningContext = {
state,
warningDelayMs,
subject: 'The deployment',
};
return await promiseRetry(async (retry, attempt) => {
let response: Response;
try {
response = await fetch(url, {
@@ -167,17 +293,33 @@ export async function callUploadApiAsync(url: string | URL, init?: RequestInit):
agent: getAgent(),
});
} catch (error) {
return retry(error);
if (init?.signal?.aborted) {
throw error;
}
if (!networkMode) {
state.shared.hasSeenNetworkError = true;
return await callUploadApiWithRetryAsync(url, init, true, state, retryDeadlineAt);
}
return retryWithWarning(retry, error, attempt, warningContext);
}
if (response.status >= 500 && response.status <= 599) {
retry(new Error(`Deployment failed: ${response.statusText}`));
retryWithWarning(
retry,
new Error(`Deployment failed: ${response.statusText}`),
attempt,
warningContext
);
}
try {
return await response.json();
} catch (error) {
retry(error);
return retryWithWarning(retry, error, attempt, warningContext);
}
});
}, retryOptions);
}
export async function callUploadApiAsync(url: string | URL, init?: RequestInit): Promise<unknown> {
return await callUploadApiWithRetryAsync(url, init);
}
export interface UploadPending {
@@ -192,6 +334,7 @@ export async function* batchUploadAsync(
): AsyncGenerator<UploadPending> {
const progressTracker = new Array(payloads.length).fill(0);
const controller = new AbortController();
const sharedRetryState: SharedRetryState = { hasSeenNetworkError: false, hasWarned: false };
const queue = new Set<Promise<UploadResult>>();
const initWithSignal = { ...init, signal: controller.signal };
const getProgressValue = (): number => {
@@ -216,7 +359,10 @@ export async function* batchUploadAsync(
progressTracker[currentIndex] = progress;
sendProgressUpdate();
});
const uploadPromise = uploadAsync(initWithSignal, payload, onChildProgressUpdate).then(
const uploadPromise = uploadAsync(initWithSignal, payload, onChildProgressUpdate, {
totalRequests: payloads.length,
state: { shared: sharedRetryState },
}).then(
result => {
queue.delete(uploadPromise);
progressTracker[currentIndex] = 1;
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/eas-json",
"version": "22.0.0",
"version": "23.1.0",
"description": "A library for interacting with eas.json",
"homepage": "https://github.com/expo/eas-cli",
"bugs": "https://github.com/expo/eas-cli/issues",
@@ -32,7 +32,7 @@
},
"dependencies": {
"@babel/code-frame": "7.23.5",
"@expo/eas-build-job": "22.0.0",
"@expo/eas-build-job": "23.1.0",
"chalk": "4.1.2",
"env-string": "1.0.1",
"fs-extra": "11.2.0",
+6 -7
View File
@@ -1,6 +1,6 @@
{
"name": "eas-cli-local-build-plugin",
"version": "22.5.0",
"version": "23.2.0",
"description": "Tool for running EAS compatible builds on a local machine.",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "BUSL-1.1",
@@ -26,12 +26,12 @@
"clean": "rimraf node_modules dist coverage \"*.tsbuildinfo\""
},
"dependencies": {
"@expo/build-tools": "22.5.0",
"@expo/eas-build-job": "22.0.0",
"@expo/logger": "22.0.0",
"@expo/build-tools": "23.2.0",
"@expo/bunyan": "^4.0.1",
"@expo/eas-build-job": "23.1.0",
"@expo/logger": "23.1.0",
"@expo/spawn-async": "^1.7.2",
"@expo/turtle-spawn": "22.0.0",
"bunyan": "^1.8.15",
"@expo/turtle-spawn": "23.1.0",
"chalk": "4.1.2",
"env-paths": "^2.2.1",
"fs-extra": "^11.2.0",
@@ -43,7 +43,6 @@
"uuid": "^9.0.1"
},
"devDependencies": {
"@types/bunyan": "^1.8.11",
"@types/fs-extra": "^11.0.4",
"@types/hapi__joi": "^17.1.14",
"@types/lodash": "^4.17.4",
Loaded 100 of 119 files, more files were not shown because too many files have changed in this diff. Show more