Compare commits

...
Author SHA1 Message Date
Krystof WoldrichandCodex 415b5bd643 [build-tools] Link Android emulator changelog entry
Co-authored-by: Codex <[email protected]>
2026-08-31 17:59:10 +02:00
Krystof WoldrichandCodex 236024b073 [build-tools] Default Android emulator to Pixel 9 on API 36
Co-authored-by: Codex <[email protected]>
2026-08-31 17:57:50 +02:00
66e3f1ad7b [build-tools] Use expo-device-hub for Android web previews (#4304)
Co-authored-by: szdziedzic <[email protected]>
Co-authored-by: Codex <[email protected]>
2026-08-31 15:49:49 +02:00
Tomasz Mazur 9aadf2d467 [eas-cli] Stream workflow run logs in real time (#4228)
* [eas-cli] Add centrifuge and ws to deps

* [eas-cli] Stream workflow job logs while waiting for a run

* [eas-cli] Add changelog entry for realtime workflow logs

* [eas-cli] Make msg optional and fix flickering

* [eas-cli] Fetch job logs without casing on job type

* [eas-cli] Ingest realtime incrementally instead of rebuilding full logs on every message (#4293)
2026-08-31 13:38:38 +00:00
Stanisław Chmiela ba80e57753 [eas-cli] Update GraphQL schema for web preview sessions (#4315)
Updated GraphQL and adjusted code.
2026-08-31 15:20:33 +02:00
Expo CI 50c76f41e3 update CHANGELOG.md and bump STAGING and PRODUCTION in cli-versions.json 2026-08-31 11:27:55 +00:00
Expo CI 0658d58efd v23.1.0 2026-08-31 11:24:34 +00:00
Phil Pluckthun 76a700c45c Remove verify step (#4310) 2026-08-29 18:07:45 +01:00
Szymon Dziedzic 5ded193ccf [eas-cli] Fix v23.0.0 changelog (#4309) 2026-08-29 11:24:10 +02:00
Brent Vatne 93420d1fa0 [eas-cli] Add prebuild fallback regression tests (#4285)
* [eas-cli] Add prebuild fallback regression tests

* Update changelog for prebuild fallback tests
2026-08-28 15:45:29 -07:00
Brent Vatne add787f625 [local-build-plugin] Replace bunyan with @expo/bunyan (#4284)
* [local-build-plugin] Replace bunyan with @expo/bunyan

* Update changelog for @expo/bunyan
2026-08-28 15:45:28 -07:00
Brent Vatne ec2c2aca5e [steps] Replace lodash.get with a local helper (#4283)
* [steps] Replace lodash.get with a local helper

* Update changelog for lodash.get removal
2026-08-28 15:45:28 -07:00
Brent Vatne 1e86c48f48 [eas-cli][steps] Bump uuid to v11 (#4282)
* [eas-cli][steps] Bump uuid to v11

* Update changelog for uuid v11
2026-08-28 15:45:27 -07:00
Brent Vatne 545c2dbf9b [eas-cli] Bump minizlib to 3.1.0 (#4281) 2026-08-28 15:45:27 -07:00
Brent Vatne 3cb029147e [eas-cli] Prompt for Expo SDK version in eas new (#4301) 2026-08-28 12:05:38 -07:00
expo-tuft[bot]andabbanmustafa d48cfbba69 [build-tools] Normalize extracted project permissions (#4287)
* [build-tools] Normalize extracted project permissions

Co-authored-by: abbanmustafa <[email protected]>

* [build-tools] Add project permission fix changelog

Co-authored-by: abbanmustafa <[email protected]>

* [build-tools] Log project permission normalization

Co-authored-by: abbanmustafa <[email protected]>

---------

Co-authored-by: expo-tuft[bot] <288127324+expo-tuft[bot]@users.noreply.github.com>
Co-authored-by: abbanmustafa <[email protected]>
2026-08-28 12:51:33 -04:00
Robin Gullo 07b5c0daa3 [build-tools] Support pnpm install --filter to install scoped dependencies (#4302)
# Why

`EAS_YARN_FOCUS_WORKSPACE` ([expo/eas-build#598](https://github.com/expo/eas-build/pull/598)) and `EAS_BUN_FILTER_WORKSPACE` ([expo/eas-cli#4292](https://github.com/expo/eas-cli/pull/4292)) let monorepos install only the dependencies of the app being built, so `node_modules` stays slim, the install is faster, and the fingerprint matches what you get locally. pnpm can do the same thing with [`pnpm install --filter <workspace>`](https://pnpm.io/filtering), but there is no way to reach it from EAS today — the pnpm branch always installs every workspace.

# How

Set it in the build profile's `env` in `eas.json`:

```json
{
  "build": {
    "production": {
      "env": {
        "EAS_PNPM_FILTER_WORKSPACE": "my-app"
      }
    }
  }
}
```

The install step then runs `pnpm install --filter my-app`. This is the pnpm counterpart of `EAS_YARN_FOCUS_WORKSPACE`, introduced in expo/eas-build#598, and `EAS_BUN_FILTER_WORKSPACE`, introduced in expo/eas-cli#4292.

# Test Plan

Added two unit tests for the flag set/unset — `yarn jest-unit src/common/__tests__/installDependencies.test.ts --runInBand` in `packages/build-tools`:

```
Test Suites: 1 passed, 1 total
Tests:       9 passed, 9 total
```

Also ran the resulting commands by hand with pnpm 10.32.1, in a workspace with `pkg-a` (depends on `is-odd`), `pkg-b` (depends on `lodash`) and an in-sync `pnpm-lock.yaml`:

| command | installed |
| --- | --- |
| `pnpm install --frozen-lockfile` | both `packages/a/node_modules/is-odd` and `packages/b/node_modules/lodash` |
| `pnpm install --filter pkg-a --frozen-lockfile` | only `packages/a/node_modules/is-odd`, `pnpm-lock.yaml` unchanged |
2026-08-28 18:32:10 +02:00
David Mokos b78c84af2c updates agent cli detector to 0.1.7 (#4272) 2026-08-28 17:31:20 +02:00
Phil Pluckthun dfaa2df90d [eas-cli] Add dynamic retries for deploy command (#4299) 2026-08-28 16:12:40 +01:00
Expo CI 2e2aaa4ac9 v23.0.0 2026-08-28 11:15:00 +00:00
Szymon Dziedzic 3ef0f74110 [eas-cli] Rename serve-sim session type to web-preview-only (#4298)
* [eas-cli] Rename simulator web preview session type

* [eas-cli] Add simulator flag rename changelog

* [eas-cli] Clarify preview-only simulator type
2026-08-28 13:12:33 +02:00
Stanisław Chmiela 51ba7198d7 [build-tools] Ignore minimum release age for internal npx (#4296)
# Why

When a user defines a min package age policy a fresh release of `eas-cli` may fail.

Reported in [the internal support thread](https://exponent-internal.slack.com/archives/C09H6RFAX27/p1787835815444139).

# How

Added `NPM_CONFIG_MIN_RELEASE_AGE=0` to NPM environment to override min release age to 0.

# Test Plan

Codex verified with different npx versions.
2026-08-27 18:19:03 +00:00
Szymon Dziedzic e10e6a2235 [build-tools] Lower serve-sim preview resolution (#4297)
* [build-tools] Lower serve-sim preview resolution

* [build-tools] Add serve-sim resolution changelog
2026-08-27 19:10:56 +02:00
Konrad Armatys 17f84cf7d6 [build-tools] Support bun install --filter to install scoped dependencies (#4292)
# Why

`EAS_YARN_FOCUS_WORKSPACE` ([expo/eas-build#598](https://github.com/expo/eas-build/pull/598)) lets a monorepo install only the dependencies of the app being built, so `node_modules` stays slim, the install is faster, and the fingerprint matches what you get locally. Bun can do the same thing with `bun install --filter <workspace>` ([oven-sh/bun#16093](https://github.com/oven-sh/bun/pull/16093)), but there is no way to reach it from EAS today — the bun branch always installs every workspace.

# How

Set it in the build profile's `env` in `eas.json`:

```json
{
  "build": {
    "production": {
      "env": {
        "EAS_BUN_FILTER_WORKSPACE": "my-app"
      }
    }
  }
}
```

The install step then runs `bun install --filter my-app`. This is the bun counterpart of `EAS_YARN_FOCUS_WORKSPACE`, introduced in expo/eas-build#598.

# Test Plan

Added two unit tests for the flag set/unset — `yarn jest-unit src/common/__tests__/installDependencies.test.ts` in `packages/build-tools`:

```
Test Suites: 1 passed, 1 total
Tests:       7 passed, 7 total
```

Also ran the resulting commands by hand, in a workspace with `pkg-a` (depends on `is-odd`), `pkg-b` (depends on `lodash`) and an in-sync `bun.lock`:

| command | installed |
| --- | --- |
| `bun install --frozen-lockfile` | 4 packages, `packages/b/node_modules/lodash` present |
| `bun install --filter pkg-a --frozen-lockfile` | 2 packages, only `packages/a/node_modules/is-odd`, `bun.lock` unchanged |
2026-08-27 18:41:24 +02:00
Mustafa Abban f7d9b7b297 [build-tools] support remote CocoaPods caching (#4266)
* [build-tools] Cache CocoaPods dependencies between iOS builds

* restore/save cache

* remove CocoaPods download cache

* remove no-lockfile cache key
2026-08-27 10:20:11 -04:00
Stanisław Chmiela b287d7a4e0 [worker] Use cache API for the legacy cache (#3860)
The legacy cache does not work in `workflow-orchestration`.

This rewrites the `CacheManager` to the new/not-so-new Turtle caches API.

## Summary
- replace worker build cache restore/save GCS config URLs with turtle cache download/upload helpers
- derive stable build cache keys without mutating job.cache and preserve clear for force uploads
- keep existing tar archive restore/save semantics and add CacheManager unit coverage

## Test plan
- cd packages/worker && yarn test CacheManager.test.ts
- cd packages/worker && yarn typecheck
- cd packages/build-tools && yarn typecheck
- git diff --check
2026-08-27 15:44:28 +02:00
Douglas Lowder d9d1b91fa5 [ci] Bump both STAGING and PRODUCTION cli-versions in release workflow (#4290) 2026-08-27 04:16:20 -07:00
Expo CI a7436edb50 update CHANGELOG.md and bump STAGING in cli-versions.json 2026-08-26 23:54:31 +00:00
Expo CI 89fe2cb3e1 v22.6.0 2026-08-26 23:49:36 +00:00
Szymon Dziedzic 0e8e8cc0aa [build-tools] Increase serve-sim bitrate to 6 Mbps (#4291)
* [build-tools] Increase serve-sim bitrate to 16 Mbps

* [build-tools] Add serve-sim bitrate changelog entry

* [build-tools] Set serve-sim bitrate to 6 Mbps
2026-08-27 01:45:23 +02:00
Gabe Debes 6d7cef80a3 [eas-cli] add experimental support to resource class on simulator sessions (#4268)
* [eas-cli] let simulator sessions request a large resource class

* [eas-cli] add changelog for hidden --resource-class

* [eas-cli] call --resource-class experimental in the changelog

* [eas-cli] oxfmt simulator resource-class test

* [eas-cli] regenerate graphql for resource class
2026-08-26 23:34:56 +00:00
Gabe Debes 5a744b73f5 [worker] Wire SSH session into build (#4031)
* [worker] wire ssh session into build

* Rename SSH phase sessionPromise to done.

Same anti-flatten object wrapper; clearer that callers await tunnel teardown.

* Cover SSH phase supervise callbacks and teardown failure paths.

* [worker] Stop baking upterm into the worker package

build-tools resolves upterm from PATH or downloads darwin-arm64 /
linux-amd64 from gs://turtle-v2/upterm at runtime.

* [worker] align ssh phase with build-tools api

* [worker] build the ssh target from the job type
2026-08-26 16:25:12 -07:00
Gabe Debes b8b53e262a [build-tools] Add worker-side SSH session helper (#4030)
* [build-tools] add worker-side ssh session helper

* Keep host.connectionConfig current and simplify SSH reconnect.

Update connectionConfig on every redial, linearize ensureConnectedAsync, and
parse upterm session objects without a JSON stringify round-trip.

* Reuse SshSettings instead of inlining idleTimeout/relay fields.

* Cover remaining SSH helper branches in unit tests.

Invalid upterm host URLs, missing process pid, stdout chunks, unknown
client counts during the job, and turtleBuild targets.

* Format unknownGraceMs ternary for oxfmt.

* [build-tools] Drop dup changelog entry; redact upterm exit logs

* [build-tools] gitignore baked upterm binaries

* [build-tools] Cover SSH helpers; drop upterm gitignore; restore review fixes

Zod-parse upterm session JSON and move isChildProcessAlive into processes.ts
(claimed in review but never landed). Expand unit coverage and stop gitignoring
baked upterm binaries now that the worker fetches from GCS.

* [build-tools] Resolve upterm from PATH or GCS instead of baking

Restore the orphaned review-fix tip: prefer an image/PATH upterm, else
download darwin-arm64 / linux-amd64 from gs://turtle-v2/upterm. Keeps the
stricter zod session parse, process-group/admin-socket comments, and GCS
cache/path unit coverage.

* [build-tools] Redact upterm tokens from ws:// relay URLs too

* [build-tools] Align upterm PATH probe with job env; scrub controls in logs

* [build-tools] address SSH review feedback

* [build-tools] address second round of SSH review feedback

* [build-tools] send a discriminated target on the ssh session mutation

* [build-tools] sort ssh test imports
2026-08-26 08:09:27 -07:00
113 changed files with 9341 additions and 1193 deletions

No files matched your search

+4 -15
View File
@@ -37,20 +37,6 @@ jobs:
run: yarn build
- name: Publish packages to npm
run: yarn run -T lerna publish from-package --yes --no-verify-access
- name: Verify eas-cli was published as latest on npm
run: |
expected="$(jq -r .version lerna.json)"
for attempt in $(seq 1 10); do
published="$(npm view eas-cli@latest version 2>/dev/null || true)"
if [ "$published" = "$expected" ]; then
echo "Verified eas-cli@$expected is latest on npm."
exit 0
fi
echo "Attempt $attempt: npm latest is '$published', expected '$expected'. Retrying in 15s..."
sleep 15
done
echo "eas-cli@$expected was not published as latest on npm; the publish step failed." >&2
exit 1
release-changelog:
name: Update changelog and publish release
runs-on: ubuntu-latest
@@ -81,10 +67,13 @@ jobs:
jq --arg version "$EAS_CLI_VERSION" '.STAGING = $version' \
cli-versions.json > cli-versions.json.tmp
mv cli-versions.json.tmp cli-versions.json
jq --arg version "$EAS_CLI_VERSION" '.PRODUCTION = $version' \
cli-versions.json > cli-versions.json.tmp
mv cli-versions.json.tmp cli-versions.json
git config --global user.email "[email protected]"
git config --global user.name "Expo CI"
git add cli-versions.json CHANGELOG.md
git commit --allow-empty -m "update CHANGELOG.md and bump STAGING in cli-versions.json"
git commit --allow-empty -m "update CHANGELOG.md and bump STAGING and PRODUCTION in cli-versions.json"
git push
- name: Prepare changelog for Slack
run: |
+59 -2
View File
@@ -8,14 +8,71 @@ This is the log of notable changes to EAS CLI and related packages.
### 🎉 New features
- [eas-cli] Add an opt-in `--max-idle-time-minutes` flag to `eas simulator`; the session stops automatically after that many minutes without activity. ([#4156](https://github.com/expo/eas-cli/pull/4156) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] Update workflow run logs in real time, instead of every 10 seconds. ([#4228](https://github.com/expo/eas-cli/pull/4228) by [@AHGIJMKLKKZNPJKQR](https://github.com/AHGIJMKLKKZNPJKQR))
- [build-tools] Add `expo-device-hub` web previews to Android web-preview-only, Agent Device, Argent, and Appium remote sessions. ([#4304](https://github.com/expo/eas-cli/pull/4304) by [@krystofwoldrich](https://github.com/krystofwoldrich))
### 🐛 Bug fixes
- [build-tools] Preapprove custom URL schemes before opening them in iOS Simulator sessions to avoid the first-use confirmation prompt. ([#4274](https://github.com/expo/eas-cli/pull/4274) by [@szdziedzic](https://github.com/szdziedzic))
### 🧹 Chores
- [build-tools] Update the default Android emulator to a Pixel 9 running Android 36. ([#4316](https://github.com/expo/eas-cli/pull/4316) by [@krystofwoldrich](https://github.com/krystofwoldrich))
## [23.1.0](https://github.com/expo/eas-cli/releases/tag/v23.1.0) - 2026-08-31
### 🎉 New features
- [eas-cli] Prompt for the Expo SDK version in `eas new` and add a `--sdk-version` flag. The template now downloads from npm instead of GitHub to support SDK-specific versions. ([#4301](https://github.com/expo/eas-cli/pull/4301) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli] Clean up template metadata in projects created with `eas new`: remove the template LICENSE file and reset the package.json name, version, and license fields. ([#4301](https://github.com/expo/eas-cli/pull/4301) by [@brentvatne](https://github.com/brentvatne))
- [build-tools] Support `EAS_PNPM_FILTER_WORKSPACE` to install only the selected workspace's dependencies with `pnpm install --filter`. ([#4302](https://github.com/expo/eas-cli/pull/4302) by [@robingullo](https://github.com/robingullo))
### 🐛 Bug fixes
- [build-tools] Normalize extracted project permissions so source archives cannot leave files unreadable by EAS workers. ([#4287](https://github.com/expo/eas-cli/pull/4287) by [@AbbanMustafa](https://github.com/AbbanMustafa))
- [eas-cli] Add dynamic retries to deploy command that scale up more generously, depending on deployment size ([#4299](https://github.com/expo/eas-cli/pull/4299) by [@kitten](https://github.com/kitten))
### 🧹 Chores
- [eas-cli] Add regression coverage for the bundled iOS entitlement introspection fallback. ([#4285](https://github.com/expo/eas-cli/pull/4285) by [@brentvatne](https://github.com/brentvatne))
- [local-build-plugin] Replace `bunyan` with `@expo/bunyan` while preserving the Steps formatter binary. ([#4284](https://github.com/expo/eas-cli/pull/4284) by [@brentvatne](https://github.com/brentvatne))
- [steps] Replace deprecated `lodash.get` with a local helper. ([#4283](https://github.com/expo/eas-cli/pull/4283) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli][steps] Bump `uuid` to v11 and remove the redundant `@types/uuid` dependency. ([#4282](https://github.com/expo/eas-cli/pull/4282) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli] Bump `minizlib` to 3.1.0 to drop its deprecated `rimraf@5` transitive dependency. ([#4281](https://github.com/expo/eas-cli/pull/4281) by [@brentvatne](https://github.com/brentvatne))
## [23.0.0](https://github.com/expo/eas-cli/releases/tag/v23.0.0) - 2026-08-28
### 🛠 Breaking changes
- [eas-cli] Rename the EAS Simulator preview-only session type from `--type serve-sim` to `--type web-preview-only`. ([#4298](https://github.com/expo/eas-cli/pull/4298) by [@szdziedzic](https://github.com/szdziedzic))
### 🎉 New features
- [build-tools] Cache CocoaPods dependencies between iOS builds. ([#4266](https://github.com/expo/eas-cli/pull/4266) by [@AbbanMustafa](https://github.com/AbbanMustafa))
- [build-tools] Support `EAS_BUN_FILTER_WORKSPACE` to install only the selected workspace's dependencies with `bun install --filter`. ([#4292](https://github.com/expo/eas-cli/pull/4292) by [@konrad-armatys](https://github.com/konrad-armatys))
### 🐛 Bug fixes
- [build-tools] Prevent user npm minimum release age settings from applying to internal EAS CLI commands. ([#4296](https://github.com/expo/eas-cli/pull/4296) by [@sjchmiela](https://github.com/sjchmiela))
- [build-tools] Reduce serve-sim preview resolution from 1280 px to 960 px to lower streaming bandwidth. ([#4297](https://github.com/expo/eas-cli/pull/4297) by [@szdziedzic](https://github.com/szdziedzic))
### 🧹 Chores
- [worker] Use turtle cache upload and download URLs for eas.json build cache restore and save. ([#3860](https://github.com/expo/eas-cli/pull/3860) by [@sjchmiela](https://github.com/sjchmiela))
- [ci] Bump both STAGING and PRODUCTION cli-versions in release workflow. ([#4290](https://github.com/expo/eas-cli/pull/4290) by [@douglowder](https://github.com/douglowder))
## [22.6.0](https://github.com/expo/eas-cli/releases/tag/v22.6.0) - 2026-08-26
### 🎉 New features
- [eas-cli] Add an opt-in `--max-idle-time-minutes` flag to `eas simulator`; the session stops automatically after that many minutes without activity. ([#4156](https://github.com/expo/eas-cli/pull/4156) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] Add experimental `--resource-class` flag to `eas simulator`. ([#4268](https://github.com/expo/eas-cli/pull/4268) by [@gwdp](https://github.com/gwdp))
- [build-tools] Add worker-side SSH session helpers (upterm relay + session create/report/close). ([#4030](https://github.com/expo/eas-cli/pull/4030) by [@gwdp](https://github.com/gwdp))
- [worker] Wire an SSH session into the build lifecycle behind the workflow ssh flag. ([#4031](https://github.com/expo/eas-cli/pull/4031) by [@gwdp](https://github.com/gwdp))
### 🐛 Bug fixes
- [build-tools] Increase the serve-sim video bitrate to 6 Mbps for higher-quality simulator previews. ([#4291](https://github.com/expo/eas-cli/pull/4291) by [@szdziedzic](https://github.com/szdziedzic))
- [build-tools] Preapprove custom URL schemes before opening them in iOS Simulator sessions to avoid the first-use confirmation prompt. ([#4274](https://github.com/expo/eas-cli/pull/4274) by [@szdziedzic](https://github.com/szdziedzic))
## [22.4.0](https://github.com/expo/eas-cli/releases/tag/v22.4.0) - 2026-08-24
### 🎉 New features
+2 -2
View File
@@ -1,4 +1,4 @@
{
"STAGING": "22.4.0",
"PRODUCTION": "22.4.0"
"STAGING": "23.1.0",
"PRODUCTION": "23.1.0"
}
+1 -1
View File
@@ -1,5 +1,5 @@
{
"npmClient": "yarn",
"version": "22.5.0",
"version": "23.1.0",
"$schema": "node_modules/lerna/schemas/lerna-schema.json"
}
+7 -8
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/build-tools",
"version": "22.5.0",
"version": "23.1.0",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "BUSL-1.1",
"author": "Expo <[email protected]>",
@@ -39,17 +39,17 @@
"dependencies": {
"@expo/config": "55.0.10",
"@expo/config-plugins": "55.0.7",
"@expo/downloader": "22.0.0",
"@expo/eas-build-job": "22.0.0",
"@expo/downloader": "23.0.0",
"@expo/eas-build-job": "23.1.0",
"@expo/env": "^0.4.0",
"@expo/logger": "22.0.0",
"@expo/logger": "23.1.0",
"@expo/package-manager": "1.9.10",
"@expo/plist": "^0.3.5",
"@expo/results": "^1.0.0",
"@expo/spawn-async": "1.7.2",
"@expo/steps": "22.4.0",
"@expo/template-file": "22.0.0",
"@expo/turtle-spawn": "22.0.0",
"@expo/steps": "23.1.0",
"@expo/template-file": "23.0.0",
"@expo/turtle-spawn": "23.1.0",
"@expo/xcpretty": "^4.3.1",
"@ngrok/ngrok": "1.7.0",
"@sentry/node": "7.77.0",
@@ -92,7 +92,6 @@
"@types/retry": "^0.12.5",
"@types/semver": "^7.5.8",
"@types/uuid": "^9.0.8",
"bunyan": "^1.8.15",
"jest": "^29.7.0",
"memfs": "^4.17.1",
"nock": "13.4.0",
@@ -1,4 +1,4 @@
import bunyan from 'bunyan';
import type { bunyan } from '@expo/logger';
export function createLogger(): bunyan {
const logger = {
+18 -2
View File
@@ -18,8 +18,12 @@ import { downloadApplicationArchiveAsync } from '../ios/resign';
import { resolveArtifactPath, resolveBuildConfiguration, resolveScheme } from '../ios/resolve';
import { Sentry } from '../sentry';
import { parseAndReportXcactivitylog } from '../steps/utils/ios/xcactivitylog';
import { cacheStatsAsync, restoreCcacheAsync } from '../steps/functions/restoreBuildCache';
import { saveCcacheAsync } from '../steps/functions/saveBuildCache';
import {
cacheStatsAsync,
restoreCcacheAsync,
restoreCocoapodsCacheAsync,
} from '../steps/functions/restoreBuildCache';
import { saveCcacheAsync, saveCocoapodsCacheAsync } from '../steps/functions/saveBuildCache';
import { uploadApplicationArchive } from '../utils/artifacts';
import {
configureExpoUpdatesIfInstalledAsync,
@@ -107,6 +111,12 @@ async function buildInnerAsync(
env: ctx.env,
secrets: ctx.job.secrets,
});
await restoreCocoapodsCacheAsync({
logger: ctx.logger,
workingDirectory,
env: ctx.env,
secrets: ctx.job.secrets,
});
});
await ctx.runBuildPhase(BuildPhase.INSTALL_PODS, async () => {
@@ -250,6 +260,12 @@ async function buildInnerAsync(
env: ctx.env,
secrets: ctx.job.secrets,
});
await saveCocoapodsCacheAsync({
logger: ctx.logger,
workingDirectory,
env: ctx.env,
secrets: ctx.job.secrets,
});
});
await ctx.runBuildPhase(BuildPhase.CACHE_STATS, async () => {
@@ -70,6 +70,48 @@ describe('easBuildInternal', () => {
expect(spawn).not.toHaveBeenCalled();
});
it('overrides the minimum release age only for the internal config command', async () => {
const logger = {
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
child: jest.fn(),
} as any;
const ctx = {
logger,
env: {
FOO: 'bar',
NPM_CONFIG_MIN_RELEASE_AGE: '365',
},
job: { platform: 'ios', buildProfile: 'production' },
} as any;
jest.mocked(resolveEasCommandPrefixAndEnvAsync).mockResolvedValueOnce({
cmd: 'npx',
args: ['-y', 'eas-cli@latest'],
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
jest.mocked(spawn).mockResolvedValueOnce({
stdout: Buffer.from(JSON.stringify({ buildProfile: { env: {} } })),
} as any);
await resolveEnvFromBuildProfileAsync(ctx, { cwd: '/tmp/project' });
expect(spawn).toHaveBeenCalledWith(
'npx',
expect.any(Array),
expect.objectContaining({
env: {
FOO: 'bar',
NPM_CONFIG_MIN_RELEASE_AGE: '0',
},
})
);
expect(ctx.env).toEqual({
FOO: 'bar',
NPM_CONFIG_MIN_RELEASE_AGE: '365',
});
});
it('passes --refresh-ad-hoc-provisioning-profile to build:internal for iOS jobs with refreshAdHocProvisioningProfile', async () => {
const logger = {
info: jest.fn(),
@@ -3,7 +3,10 @@ import spawn, { SpawnPromise, SpawnResult } from '@expo/turtle-spawn';
import { createMockLogger } from '../../__tests__/utils/logger';
import { Sentry } from '../../sentry';
import { PackageManager } from '../../utils/packageManager';
import { installDependenciesWithNpmCacheFallbackAsync } from '../installDependencies';
import {
installDependenciesAsync,
installDependenciesWithNpmCacheFallbackAsync,
} from '../installDependencies';
jest.mock('@expo/turtle-spawn', () => jest.fn());
jest.mock('../../sentry', () => ({
@@ -236,6 +239,85 @@ describe(installDependenciesWithNpmCacheFallbackAsync, () => {
});
});
describe(installDependenciesAsync, () => {
beforeEach(() => {
jest.clearAllMocks();
jest.mocked(spawn).mockReturnValue(createSpawnPromise(Promise.resolve(createSpawnResult())));
});
it('installs only the selected workspace when EAS_PNPM_FILTER_WORKSPACE is set', async () => {
const logger = createMockLogger();
await installDependenciesAsync({
packageManager: PackageManager.PNPM,
env: { EAS_PNPM_FILTER_WORKSPACE: 'my-app' },
logger,
cwd: '/tmp/build',
useFrozenLockfile: true,
});
expect(spawn).toHaveBeenCalledWith(
'pnpm',
['install', '--filter', 'my-app', '--frozen-lockfile'],
expect.objectContaining({ cwd: '/tmp/build' })
);
});
it('installs all workspaces when EAS_PNPM_FILTER_WORKSPACE is not set', async () => {
const logger = createMockLogger();
await installDependenciesAsync({
packageManager: PackageManager.PNPM,
env: {},
logger,
cwd: '/tmp/build',
useFrozenLockfile: true,
});
expect(spawn).toHaveBeenCalledWith(
'pnpm',
['install', '--frozen-lockfile'],
expect.objectContaining({ cwd: '/tmp/build' })
);
});
it('installs only the selected workspace when EAS_BUN_FILTER_WORKSPACE is set', async () => {
const logger = createMockLogger();
await installDependenciesAsync({
packageManager: PackageManager.BUN,
env: { EAS_BUN_FILTER_WORKSPACE: 'my-app' },
logger,
cwd: '/tmp/build',
useFrozenLockfile: true,
});
expect(spawn).toHaveBeenCalledWith(
'bun',
['install', '--filter', 'my-app', '--frozen-lockfile'],
expect.objectContaining({ cwd: '/tmp/build' })
);
});
it('installs all workspaces when EAS_BUN_FILTER_WORKSPACE is not set', async () => {
const logger = createMockLogger();
await installDependenciesAsync({
packageManager: PackageManager.BUN,
env: {},
logger,
cwd: '/tmp/build',
useFrozenLockfile: true,
});
expect(spawn).toHaveBeenCalledWith(
'bun',
['install', '--frozen-lockfile'],
expect.objectContaining({ cwd: '/tmp/build' })
);
});
});
function createSpawnPromise(result: Promise<SpawnResult>): SpawnPromise<SpawnResult> {
return Object.assign(result, {
child: {
@@ -6,6 +6,7 @@ import {
Platform,
Workflow,
} from '@expo/eas-build-job';
import spawn from '@expo/turtle-spawn';
import { randomBytes, randomUUID } from 'crypto';
import { vol } from 'memfs';
import fetch, { Response } from 'node-fetch';
@@ -14,7 +15,7 @@ import { setTimeout } from 'timers/promises';
import { createMockLogger } from '../../__tests__/utils/logger';
import { BuildContext } from '../../context';
import { shallowCloneRepositoryAsync } from '../git';
import { prepareProjectSourcesAsync } from '../projectSources';
import { prepareProjectSourcesAsync, unpackTarGzAsync } from '../projectSources';
jest.mock('@expo/turtle-spawn');
jest.mock('node-fetch');
@@ -23,6 +24,27 @@ jest.mock('@expo/downloader');
jest.mock('@urql/core');
describe('projectSources', () => {
it('makes extracted project sources readable and writable by the worker', async () => {
const logger = createMockLogger();
await unpackTarGzAsync({
source: '/workingdir/project.tar.gz',
destination: '/workingdir/build',
logger,
});
expect(spawn).toHaveBeenNthCalledWith(
1,
'tar',
['-C', '/workingdir/build', '--strip-components', '1', '-zxf', '/workingdir/project.tar.gz'],
{ logger }
);
expect(spawn).toHaveBeenNthCalledWith(2, 'chmod', ['-R', 'u+rwX', '/workingdir/build'], {
logger,
});
expect(logger.info).toHaveBeenCalledWith('Normalizing project source permissions');
});
it('should use the refreshed repository URL', async () => {
const robotAccessToken = randomUUID();
const buildId = randomUUID();
@@ -32,7 +32,11 @@ export async function installDependenciesAsync({
break;
}
case PackageManager.PNPM: {
args = ['install', useFrozenLockfile ? '--frozen-lockfile' : '--no-frozen-lockfile'];
args = [
'install',
...(env['EAS_PNPM_FILTER_WORKSPACE'] ? ['--filter', env['EAS_PNPM_FILTER_WORKSPACE']] : []),
useFrozenLockfile ? '--frozen-lockfile' : '--no-frozen-lockfile',
];
break;
}
case PackageManager.YARN: {
@@ -58,7 +62,11 @@ export async function installDependenciesAsync({
break;
}
case PackageManager.BUN:
args = ['install', ...(useFrozenLockfile ? ['--frozen-lockfile'] : [])];
args = [
'install',
...(env['EAS_BUN_FILTER_WORKSPACE'] ? ['--filter', env['EAS_BUN_FILTER_WORKSPACE']] : []),
...(useFrozenLockfile ? ['--frozen-lockfile'] : []),
];
break;
default:
throw new Error(`Unsupported package manager: ${packageManager}`);
@@ -110,7 +110,7 @@ async function prepareProjectSourcesLocallyAsync<TJob extends Job>(
});
}
async function unpackTarGzAsync({
export async function unpackTarGzAsync({
logger,
source,
destination,
@@ -122,6 +122,8 @@ async function unpackTarGzAsync({
await spawn('tar', ['-C', destination, '--strip-components', '1', '-zxf', source], {
logger,
});
logger.info('Normalizing project source permissions');
await spawn('chmod', ['-R', 'u+rwX', destination], { logger });
}
function uploadProjectMetadataAsFireAndForget(
+5
View File
@@ -5,6 +5,8 @@ import RemoteLoggerStream from './logging/RemoteLoggerStream';
export { Builders, HttpLogStream, RemoteLoggerStream };
export { uploadWithSignedUrl } from './storage/uploadWithSignedUrl';
export type { SignedUrl, UploadWithSignedUrlParams } from './storage/uploadWithSignedUrl';
export { downloadCacheAsync } from './steps/functions/restoreCache';
export { uploadCacheAsync } from './steps/functions/saveCache';
export {
ArtifactToUpload,
@@ -18,6 +20,9 @@ export {
export { PackageManager } from './utils/packageManager';
export * as TurtleSshSession from './utils/turtleSshSession';
export { formatSecondsForLog } from './utils/formatDuration';
export { findAndUploadXcodeBuildLogsAsync } from './ios/xcodeBuildLogs';
export { Hook, runHookIfPresent } from './utils/hooks';
@@ -51,7 +51,7 @@ import { createStartAppiumRemoteSessionBuildFunction } from './functions/startAp
import { createStartCuttlefishDeviceBuildFunction } from './functions/startCuttlefishDevice';
import { createStartIosSimulatorBuildFunction } from './functions/startIosSimulator';
import { createStartIosSimulatorRecordingsBuildFunction } from './functions/startIosSimulatorRecordings';
import { createStartServeSimRemoteSessionBuildFunction } from './functions/startServeSimRemoteSession';
import { createStartWebPreviewRemoteSessionBuildFunction } from './functions/startWebPreviewRemoteSession';
import { createStartServeSimMetricsBuildFunction } from './functions/startServeSimMetrics';
import { createCollectServeSimMetricsBuildFunction } from './functions/collectServeSimMetrics';
import { createUploadArtifactBuildFunction } from './functions/uploadArtifact';
@@ -106,7 +106,7 @@ export function getEasFunctions(ctx: CustomBuildContext): BuildFunction[] {
createStartIosSimulatorRecordingsBuildFunction(),
createFinishIosSimulatorRecordingsBuildFunction(),
createUploadDeviceRunSessionScreenRecordingsBuildFunction(ctx),
createStartServeSimRemoteSessionBuildFunction(ctx),
createStartWebPreviewRemoteSessionBuildFunction(ctx),
createStartServeSimMetricsBuildFunction(),
createCollectServeSimMetricsBuildFunction(ctx),
createInstallMaestroBuildFunction(),
@@ -162,6 +162,28 @@ describe(createEasBuildBuildFunctionGroup, () => {
expect(installPodsStep!.ctx.relativeWorkingDirectory).toBe('./ios');
});
it('restores and saves caches around an iOS simulator build', () => {
const buildToolsContext = createMockBuildToolsContext({
platform: Platform.IOS,
simulator: true,
});
const functionGroup = createEasBuildBuildFunctionGroup(buildToolsContext);
const globalCtx = createGlobalContextMock({ logger: createMockLogger() });
const steps = functionGroup.createBuildStepsFromFunctionGroupCall(globalCtx);
const stepNames = steps.map(step => step.displayName);
const restoreCacheIndex = stepNames.indexOf('Restore Cache');
const installPodsIndex = stepNames.indexOf('Install Pods');
const saveCacheIndex = stepNames.indexOf('Save Cache');
expect(restoreCacheIndex).toBeGreaterThan(-1);
expect(restoreCacheIndex).toBeLessThan(installPodsIndex);
expect(saveCacheIndex).toBeGreaterThan(installPodsIndex);
const restoreCacheStep = steps[restoreCacheIndex];
expect(restoreCacheStep.inputs?.find(input => input.id === 'simulator')?.rawValue).toBe(true);
});
it('sets working directory on all steps except checkout (Android with credentials)', () => {
const buildToolsContext = createMockBuildToolsContext({
platform: Platform.ANDROID,
@@ -98,6 +98,8 @@ function createStepsForIosSimulatorBuild({
buildToolsContext,
workingDirectory,
}: HelperFunctionsInput): BuildStep[] {
const evictUsedBefore = new Date();
const calculateEASUpdateRuntimeVersion =
calculateEASUpdateRuntimeVersionFunction().createBuildStepFromFunctionCall(globalCtx, {
id: 'calculate_eas_update_runtime_version',
@@ -106,6 +108,16 @@ function createStepsForIosSimulatorBuild({
const installPods = createInstallPodsBuildFunction().createBuildStepFromFunctionCall(globalCtx, {
workingDirectory: workingDirectory ? path.join(workingDirectory, './ios') : './ios',
});
const restoreCache = createRestoreBuildCacheFunction().createBuildStepFromFunctionCall(
globalCtx,
{
workingDirectory,
callInputs: {
platform: Platform.IOS,
simulator: true,
},
}
);
const configureEASUpdate =
configureEASUpdateIfInstalledFunction().createBuildStepFromFunctionCall(globalCtx, {
workingDirectory,
@@ -123,6 +135,16 @@ function createStepsForIosSimulatorBuild({
'${ steps.calculate_eas_update_runtime_version.resolved_eas_update_runtime_version }',
},
});
const saveCache = createSaveBuildCacheFunction(evictUsedBefore).createBuildStepFromFunctionCall(
globalCtx,
{
workingDirectory,
callInputs: {
platform: Platform.IOS,
simulator: true,
},
}
);
return [
createCheckoutBuildFunction().createBuildStepFromFunctionCall(globalCtx),
createSetUpNpmrcBuildFunction().createBuildStepFromFunctionCall(globalCtx, {
@@ -138,6 +160,7 @@ function createStepsForIosSimulatorBuild({
createPrebuildBuildFunction().createBuildStepFromFunctionCall(globalCtx, {
workingDirectory,
}),
restoreCache,
calculateEASUpdateRuntimeVersion,
installPods,
configureEASUpdate,
@@ -159,6 +182,10 @@ function createStepsForIosSimulatorBuild({
createFindAndUploadBuildArtifactsBuildFunction(
buildToolsContext
).createBuildStepFromFunctionCall(globalCtx, { workingDirectory }),
saveCache,
createCacheStatsBuildFunction().createBuildStepFromFunctionCall(globalCtx, {
workingDirectory,
}),
];
}
@@ -0,0 +1,136 @@
import { spawnAsync } from '@expo/steps';
import { vol } from 'memfs';
import { createMockLogger } from '../../../__tests__/utils/logger';
import { Datadog } from '../../../datadog';
import {
compressCocoapodsCacheAsync,
getCocoapodsCachePaths,
resolveCocoapodsCacheKeyAsync,
restoreCocoapodsCacheArchiveAsync,
} from '../../../utils/cocoapodsCache';
import { downloadCacheAsync } from '../restoreCache';
import { restoreCocoapodsCacheAsync } from '../restoreBuildCache';
import { uploadCacheAsync } from '../saveCache';
import { saveCocoapodsCacheAsync } from '../saveBuildCache';
jest.mock('@expo/steps', () => ({
...jest.requireActual('@expo/steps'),
spawnAsync: jest.fn(),
}));
jest.mock('../restoreCache', () => ({
decompressCacheAsync: jest.fn(),
downloadCacheAsync: jest.fn(),
downloadPublicCacheAsync: jest.fn(),
}));
jest.mock('../saveCache', () => ({
compressCacheAsync: jest.fn(),
uploadCacheAsync: jest.fn(),
}));
jest.mock('../../../utils/cocoapodsCache', () => ({
compressCocoapodsCacheAsync: jest.fn(),
getCocoapodsCachePaths: jest.fn(),
resolveCocoapodsCacheKeyAsync: jest.fn(),
restoreCocoapodsCacheArchiveAsync: jest.fn(),
}));
const logger = createMockLogger();
const env = {
EAS_PODS_CACHE: '1',
EAS_BUILD_ID: 'build-id',
__API_SERVER_URL: 'https://api.expo.test',
};
const secrets = { robotAccessToken: 'robot-token' };
describe('CocoaPods build cache', () => {
beforeEach(() => {
jest.clearAllMocks();
vol.fromJSON(
{
'/workingdir/ios/Pods/Manifest.lock': 'manifest',
'/workingdir/ios/Podfile.lock': 'lockfile',
'/tmp/cocoapods-cache.tar.gz': 'archive',
},
'/'
);
jest.mocked(getCocoapodsCachePaths).mockReturnValue({
iosDirectory: '/workingdir/ios',
podsDirectory: '/workingdir/ios/Pods',
podfileLockPath: '/workingdir/ios/Podfile.lock',
});
jest.mocked(resolveCocoapodsCacheKeyAsync).mockResolvedValue({
key: 'ios-pods-1.16.2-lock-hash',
keyPrefix: 'ios-pods-1.16.2-',
});
jest.mocked(spawnAsync).mockResolvedValue({ stdout: '1.16.2\n' } as any);
});
it('does nothing when the cache is disabled', async () => {
await restoreCocoapodsCacheAsync({
logger,
workingDirectory: '/workingdir',
env: {},
secrets,
});
await saveCocoapodsCacheAsync({
logger,
workingDirectory: '/workingdir',
env: {},
secrets,
});
expect(spawnAsync).not.toHaveBeenCalled();
});
it('restores the newest matching CocoaPods cache', async () => {
jest.mocked(downloadCacheAsync).mockResolvedValue({
archivePath: '/tmp/cocoapods-cache.tar.gz',
matchedKey: 'ios-pods-1.16.2-older-lock-hash',
});
const datadogLogSpy = jest.spyOn(Datadog, 'log');
await restoreCocoapodsCacheAsync({
logger,
workingDirectory: '/workingdir',
env,
secrets,
});
expect(downloadCacheAsync).toHaveBeenCalledWith(
expect.objectContaining({
key: 'ios-pods-1.16.2-lock-hash',
keyPrefixes: ['ios-pods-1.16.2-'],
paths: ['/workingdir/ios/Pods'],
})
);
expect(restoreCocoapodsCacheArchiveAsync).toHaveBeenCalledWith({
archivePath: '/tmp/cocoapods-cache.tar.gz',
workingDirectory: '/workingdir',
});
expect(datadogLogSpy).toHaveBeenCalledWith('CocoaPods cache restored (prefix_match)', {
event: 'cocoapods_cache_restored',
cache_hit_type: 'prefix_match',
});
});
it('compresses and saves the installed Pods directory', async () => {
jest.mocked(compressCocoapodsCacheAsync).mockResolvedValue({
archivePath: '/tmp/cocoapods-cache.tar.gz',
});
await saveCocoapodsCacheAsync({
logger,
workingDirectory: '/workingdir',
env,
secrets,
});
expect(uploadCacheAsync).toHaveBeenCalledWith(
expect.objectContaining({
archivePath: '/tmp/cocoapods-cache.tar.gz',
key: 'ios-pods-1.16.2-lock-hash',
paths: ['/workingdir/ios/Pods'],
})
);
});
});
@@ -1,11 +1,17 @@
import { type bunyan } from '@expo/logger';
import { Platform } from '@expo/eas-build-job';
import fs from 'fs';
import fetch from 'node-fetch';
import os from 'os';
import path from 'path';
import { compressCacheAsync } from '../saveCache';
import { compressCacheAsync, uploadCacheAsync } from '../saveCache';
import { decompressCacheAsync } from '../restoreCache';
jest.mock('node-fetch');
const { Response } = jest.requireActual('node-fetch') as typeof import('node-fetch');
function createLoggerMock(): bunyan {
return { info: jest.fn(), warn: jest.fn(), error: jest.fn() } as unknown as bunyan;
}
@@ -81,3 +87,71 @@ describe('cache compress/decompress round trip', () => {
expect(Math.floor(restoredStat.mtimeMs / 1000)).toBe(Math.floor(mtime.getTime() / 1000));
});
});
describe(uploadCacheAsync, () => {
afterEach(() => {
jest.mocked(fetch).mockReset();
});
it.each([
['normal upload', undefined, false],
['forced upload', true, true],
] as const)('sends force for %s', async (_name, force, expectedForce) => {
const tempDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'upload-cache-test-'));
const archivePath = path.join(tempDir, 'cache.tar.gz');
await fs.promises.writeFile(archivePath, 'cache archive');
jest
.mocked(fetch)
.mockResolvedValueOnce(
new Response(
JSON.stringify({
data: {
headers: {},
url: 'https://storage.expo.test/cache',
},
}),
{ status: 200 }
)
)
.mockImplementationOnce(async (_url, request) => {
const body = request!.body as NodeJS.ReadableStream & AsyncIterable<Buffer>;
for await (const _chunk of body) {
// Consume the archive stream as a real upload would.
}
return new Response(undefined, { status: 200 });
});
try {
await uploadCacheAsync({
logger: createLoggerMock(),
jobId: 'build-id',
expoApiServerURL: 'https://api.expo.test',
robotAccessToken: 'robot-token',
paths: ['/cache/path'],
key: 'cache-key',
archivePath,
size: 13,
platform: Platform.ANDROID,
force,
});
expect(fetch).toHaveBeenCalledTimes(2);
const [uploadSessionUrl, uploadSessionRequest] = jest.mocked(fetch).mock.calls[0];
expect(uploadSessionUrl.toString()).toBe(
'https://api.expo.test/v2/turtle-builds/caches/upload-sessions'
);
expect(JSON.parse(uploadSessionRequest!.body as string)).toMatchObject({
buildId: 'build-id',
force: expectedForce,
key: 'cache-key',
size: 13,
});
expect(jest.mocked(fetch).mock.calls[1][0].toString()).toBe(
'https://storage.expo.test/cache'
);
} finally {
await fs.promises.rm(tempDir, { recursive: true, force: true });
}
});
});
@@ -21,7 +21,8 @@ jest.mock('../../../utils/retry', () => ({
jest.mock('../../../utils/AndroidEmulatorUtils', () => ({
AndroidEmulatorUtils: {
defaultSystemImagePackage: 'system-images;android-30;default;x86_64',
defaultDeviceIdentifier: 'pixel_9',
defaultSystemImagePackage: 'system-images;android-36;default;x86_64',
getAvailableDevicesAsync: jest.fn(),
createAsync: jest.fn(),
cloneAsync: jest.fn(),
@@ -259,6 +260,25 @@ describe(createStartAndroidEmulatorBuildFunction, () => {
);
});
it('creates a Pixel 9 emulator with the Android 36 system image by default', async () => {
await createStep().executeAsync();
expect(mockedSpawn).toHaveBeenCalledWith(
'sdkmanager',
['system-images;android-36;default;x86_64'],
expect.objectContaining({
env: expect.any(Object),
})
);
expect(mockedAndroidUtils.createAsync).toHaveBeenCalledWith(
expect.objectContaining({
deviceIdentifier: 'pixel_9',
deviceName: 'EasAndroidDevice01',
systemImagePackage: 'system-images;android-36;default;x86_64',
})
);
});
it('skips animation scale adjustments when opt out env var is disabled', async () => {
const step = createStep(undefined, {
ANDROID_EMULATOR_ADJUST_ANIMATION_SCALE: 'false',
@@ -14,6 +14,7 @@ import {
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
@@ -46,8 +47,8 @@ jest.mock('../../utils/remoteDeviceRunSession', () => ({
getNgrokTunnelDomainOrThrow: jest.fn(),
selectXcodeDeveloperDirectoryAsync: jest.fn(),
spawnDetached: jest.fn(),
startDeviceWebPreviewWithTunnelAsync: jest.fn(),
startNgrokTunnelAsync: jest.fn(),
startServeSimWithTunnelAsync: jest.fn(),
uploadRemoteSessionConfigAsync: jest.fn(),
waitForDeviceRunSessionStoppedAsync: jest.fn(),
}));
@@ -58,6 +59,7 @@ const EXPECTED_EVENT_LOG_PATH = path.join(ARGENT_STATE_DIR, 'tool-server-events.
const mockStopAsync = jest.fn();
const mockTunnelStopAsync = jest.fn();
const mockPreviewStopAsync = jest.fn();
describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
beforeEach(async () => {
@@ -68,6 +70,7 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
jest.mocked(pollArgentArtifactsForUploadAsync).mockResolvedValue(undefined);
mockStopAsync.mockResolvedValue(undefined);
mockTunnelStopAsync.mockResolvedValue(undefined);
mockPreviewStopAsync.mockResolvedValue(undefined);
jest.mocked(startArgentEventCollectionAsync).mockResolvedValue({
stopAsync: mockStopAsync,
getLastEventObservedAt: () => undefined,
@@ -86,6 +89,10 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
url: 'https://argent-abc.tunnel.example.com',
stopAsync: mockTunnelStopAsync,
});
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
previewUrl: 'https://web-preview.tunnel.example.com',
stopAsync: mockPreviewStopAsync,
});
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockResolvedValue(undefined);
@@ -151,5 +158,17 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
jest.mocked(waitForDeviceRunSessionStoppedAsync).mock.invocationCallOrder[0]
);
expect(mockTunnelStopAsync).toHaveBeenCalledTimes(1);
expect(startDeviceWebPreviewWithTunnelAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({ runtimePlatform: BuildRuntimePlatform.LINUX })
);
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledWith(
expect.objectContaining({
remoteConfig: expect.objectContaining({
webPreviewUrl: 'https://web-preview.tunnel.example.com',
}),
})
);
expect(mockPreviewStopAsync).toHaveBeenCalledTimes(1);
});
});
@@ -0,0 +1,85 @@
import { type bunyan } from '@expo/logger';
import { BuildRuntimePlatform, type BuildStepContext, type BuildStepEnv } from '@expo/steps';
import { type CustomBuildContext } from '../../../customBuildContext';
import {
getDeviceRunSessionIdOrThrow,
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
startDeviceWebPreviewWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../../utils/remoteDeviceRunSession';
import { createStartWebPreviewRemoteSessionBuildFunction } from '../startWebPreviewRemoteSession';
jest.mock('../../utils/remoteDeviceRunSession');
const ctx = {} as CustomBuildContext;
const env = {} as BuildStepEnv;
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const stopAsync = jest.fn();
async function runAsync(runtimePlatform: BuildRuntimePlatform): Promise<void> {
const buildFunction = createStartWebPreviewRemoteSessionBuildFunction(ctx);
await buildFunction.fn!(
{
logger,
global: { runtimePlatform },
} as unknown as BuildStepContext,
{
inputs: {
package_version: { value: '1.2.3' },
max_duration_seconds: { value: 120 },
},
outputs: {},
env,
} as never
);
}
describe(createStartWebPreviewRemoteSessionBuildFunction, () => {
beforeEach(() => {
jest.clearAllMocks();
jest.mocked(getDeviceRunSessionIdOrThrow).mockReturnValue('device-run-session-id');
jest.mocked(getNgrokTunnelDomainOrThrow).mockReturnValue('tunnel.example.com');
jest.mocked(selectXcodeDeveloperDirectoryAsync).mockResolvedValue(undefined);
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
previewUrl: 'https://web-preview.example.test',
stopAsync,
});
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockResolvedValue(undefined);
stopAsync.mockResolvedValue(undefined);
});
it.each([
[BuildRuntimePlatform.DARWIN, true],
[BuildRuntimePlatform.LINUX, false],
])('starts the web preview for %s', async (runtimePlatform, selectsXcode) => {
await runAsync(runtimePlatform);
expect(selectXcodeDeveloperDirectoryAsync).toHaveBeenCalledTimes(selectsXcode ? 1 : 0);
expect(startDeviceWebPreviewWithTunnelAsync).toHaveBeenCalledWith(ctx, {
runtimePlatform,
baseDomain: 'tunnel.example.com',
env,
logger,
timeoutMs: 60_000,
packageVersion: '1.2.3',
});
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledWith({
ctx,
deviceRunSessionId: 'device-run-session-id',
remoteConfig: { previewUrl: 'https://web-preview.example.test' },
logger,
});
expect(waitForDeviceRunSessionStoppedAsync).toHaveBeenCalledWith({
ctx,
deviceRunSessionId: 'device-run-session-id',
logger,
maxDurationSeconds: 120,
signal: undefined,
});
expect(stopAsync).toHaveBeenCalledTimes(1);
});
});
@@ -20,6 +20,11 @@ import {
getCcacheKeyPrefix,
getCcachePath,
} from '../../utils/cacheKey';
import {
getCocoapodsCachePaths,
resolveCocoapodsCacheKeyAsync,
restoreCocoapodsCacheArchiveAsync,
} from '../../utils/cocoapodsCache';
import { Datadog } from '../../datadog';
import { GRADLE_CACHE_KEY_PREFIX, generateGradleCacheKeyAsync } from '../../utils/gradleCacheKey';
import { TurtleFetchError, turtleFetch } from '../../utils/turtleFetch';
@@ -79,6 +84,13 @@ export function createRestoreBuildCacheFunction(): BuildFunction {
env,
secrets: stepCtx.global.staticContext.job.secrets,
});
} else {
await restoreCocoapodsCacheAsync({
logger,
workingDirectory,
env,
secrets: stepCtx.global.staticContext.job.secrets,
});
}
},
});
@@ -206,6 +218,67 @@ export async function restoreCcacheAsync({
}
}
export async function restoreCocoapodsCacheAsync({
logger,
workingDirectory,
env,
secrets,
}: {
logger: bunyan;
workingDirectory: string;
env: Record<string, string | undefined>;
secrets?: { robotAccessToken?: string };
}): Promise<void> {
if (env.EAS_PODS_CACHE !== '1') {
return;
}
try {
const { stdout } = await spawnAsync('pod', ['--version'], {
env,
stdio: 'pipe',
});
const { key, keyPrefix } = await resolveCocoapodsCacheKeyAsync(workingDirectory, stdout);
logger.info(`Restoring CocoaPods cache key: ${key}`);
const jobId = nullthrows(env.EAS_BUILD_ID, 'EAS_BUILD_ID is not set');
const robotAccessToken = nullthrows(
secrets?.robotAccessToken,
'Robot access token is required for cache operations'
);
const expoApiServerURL = nullthrows(env.__API_SERVER_URL, '__API_SERVER_URL is not set');
const { podsDirectory } = getCocoapodsCachePaths(workingDirectory);
const { archivePath, matchedKey } = await downloadCacheAsync({
logger,
jobId,
expoApiServerURL,
robotAccessToken,
paths: [podsDirectory],
key,
keyPrefixes: [keyPrefix],
platform: Platform.IOS,
});
await restoreCocoapodsCacheArchiveAsync({ archivePath, workingDirectory });
const hitType = matchedKey === key ? 'direct_hit' : 'prefix_match';
logger.info(
`CocoaPods cache restored to ${podsDirectory} (${hitType === 'direct_hit' ? 'direct hit' : 'prefix match'})`
);
Datadog.log(`CocoaPods cache restored (${hitType})`, {
event: 'cocoapods_cache_restored',
cache_hit_type: hitType,
});
} catch (err: unknown) {
if (err instanceof TurtleFetchError && err.response?.status === 404) {
logger.info('No CocoaPods cache found for this key');
} else {
logger.warn('Failed to restore CocoaPods cache: ', err);
}
}
}
export async function restoreGradleCacheAsync({
logger,
workingDirectory,
@@ -20,6 +20,11 @@ import {
generateDefaultBuildCacheKeyAsync,
getCcachePath,
} from '../../utils/cacheKey';
import {
compressCocoapodsCacheAsync,
getCocoapodsCachePaths,
resolveCocoapodsCacheKeyAsync,
} from '../../utils/cocoapodsCache';
import { generateGradleCacheKeyAsync } from '../../utils/gradleCacheKey';
export function createSaveBuildCacheFunction(evictUsedBefore: Date): BuildFunction {
@@ -78,11 +83,77 @@ export function createSaveBuildCacheFunction(evictUsedBefore: Date): BuildFuncti
env,
secrets: stepCtx.global.staticContext.job.secrets,
});
} else {
await saveCocoapodsCacheAsync({
logger,
workingDirectory,
env,
secrets: stepCtx.global.staticContext.job.secrets,
});
}
},
});
}
export async function saveCocoapodsCacheAsync({
logger,
workingDirectory,
env,
secrets,
}: {
logger: bunyan;
workingDirectory: string;
env: Record<string, string | undefined>;
secrets?: { robotAccessToken?: string };
}): Promise<void> {
if (env.EAS_PODS_CACHE !== '1') {
return;
}
const { podsDirectory, podfileLockPath } = getCocoapodsCachePaths(workingDirectory);
try {
await Promise.all([fs.promises.access(podsDirectory), fs.promises.access(podfileLockPath)]);
} catch {
logger.warn('No CocoaPods installation found, skipping cache save');
return;
}
try {
const { stdout } = await spawnAsync('pod', ['--version'], {
env,
stdio: 'pipe',
});
const { key } = await resolveCocoapodsCacheKeyAsync(workingDirectory, stdout);
logger.info(`Saving CocoaPods cache key: ${key}`);
const jobId = nullthrows(env.EAS_BUILD_ID, 'EAS_BUILD_ID is not set');
const robotAccessToken = nullthrows(
secrets?.robotAccessToken,
'Robot access token is required for cache operations'
);
const expoApiServerURL = nullthrows(env.__API_SERVER_URL, '__API_SERVER_URL is not set');
logger.info('Compressing CocoaPods cache...');
const { archivePath } = await compressCocoapodsCacheAsync({ workingDirectory });
const { size } = await fs.promises.stat(archivePath);
logger.info(`CocoaPods cache archive size: ${formatBytes(size)}`);
await uploadCacheAsync({
logger,
jobId,
expoApiServerURL,
robotAccessToken,
archivePath,
key,
paths: [podsDirectory],
size,
platform: Platform.IOS,
});
} catch (err) {
logger.error({ err }, 'Failed to save CocoaPods cache');
}
}
export async function saveCcacheAsync({
logger,
workingDirectory,
@@ -103,6 +103,7 @@ export async function uploadCacheAsync({
archivePath,
size,
platform,
force = false,
}: {
logger: bunyan;
jobId: string;
@@ -113,6 +114,7 @@ export async function uploadCacheAsync({
archivePath: string;
size: number;
platform: Platform | undefined;
force?: boolean;
}): Promise<void> {
const routerURL = platform
? 'v2/turtle-builds/caches/upload-sessions'
@@ -127,12 +129,14 @@ export async function uploadCacheAsync({
key,
version: getCacheVersion(paths),
size,
force,
})
: JSON.stringify({
jobRunId: jobId,
key,
version: getCacheVersion(paths),
size,
force,
}),
headers: {
Authorization: `Bearer ${robotAccessToken}`,
@@ -23,8 +23,8 @@ import {
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
startServeSimWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
waitForFileAsync,
@@ -107,22 +107,19 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
const agentDeviceRemoteSessionUrl = agentDeviceTunnel.url;
logger.info(`Tunnel is ready at ${agentDeviceRemoteSessionUrl}.`);
let serveSim: Awaited<ReturnType<typeof startServeSimWithTunnelAsync>> | undefined;
let webPreview: Awaited<ReturnType<typeof startDeviceWebPreviewWithTunnelAsync>> | undefined;
let eventCollection:
| Awaited<ReturnType<typeof startAgentDeviceEventCollectionAsync>>
| undefined;
try {
// serve-sim is iOS-only — only launch it (and report a webPreviewUrl)
// on Darwin. Android sessions go without a preview URL.
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
serveSim = await startServeSimWithTunnelAsync(ctx, {
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
logger.info(`Web preview URL: ${serveSim.previewUrl}`);
}
webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
logger.info(`Web preview URL: ${webPreview.previewUrl}`);
await uploadRemoteSessionConfigAsync({
ctx,
@@ -130,7 +127,7 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
remoteConfig: {
agentDeviceRemoteSessionUrl,
agentDeviceRemoteSessionToken: daemonToken,
...(serveSim ? { webPreviewUrl: serveSim.previewUrl } : {}),
webPreviewUrl: webPreview.previewUrl,
},
logger,
});
@@ -163,8 +160,8 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
: undefined,
});
} finally {
if (serveSim) {
await serveSim.stopAsync();
if (webPreview) {
await webPreview.stopAsync();
}
await agentDeviceTunnel.stopAsync();
if (eventCollection) {
@@ -38,6 +38,7 @@ export function createStartAndroidEmulatorBuildFunction(): BuildFunction {
BuildStepInput.createProvider({
id: 'device_identifier',
required: false,
defaultValue: AndroidEmulatorUtils.defaultDeviceIdentifier,
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
BuildStepInput.createProvider({
@@ -26,8 +26,8 @@ import {
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
startServeSimWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
@@ -118,7 +118,7 @@ export function createStartAppiumRemoteSessionBuildFunction(
logger,
});
let appiumTunnel: Awaited<ReturnType<typeof startNgrokTunnelAsync>> | undefined;
let serveSim: Awaited<ReturnType<typeof startServeSimWithTunnelAsync>> | undefined;
let webPreview: Awaited<ReturnType<typeof startDeviceWebPreviewWithTunnelAsync>> | undefined;
try {
appiumTunnel = await startNgrokTunnelAsync({
port: APPIUM_PORT,
@@ -128,18 +128,15 @@ export function createStartAppiumRemoteSessionBuildFunction(
logger,
});
switch (runtimePlatform) {
case BuildRuntimePlatform.DARWIN:
serveSim = await startServeSimWithTunnelAsync(ctx, {
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: APPIUM_STARTUP_TIMEOUT_MS,
});
break;
case BuildRuntimePlatform.LINUX:
break;
}
// expo-device-hub has no serial-selection flag. Device run session workflows must expose
// a single booted Android emulator so the Hub and Appium resolve the same device.
webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: APPIUM_STARTUP_TIMEOUT_MS,
});
await uploadRemoteSessionConfigAsync({
ctx,
@@ -151,7 +148,7 @@ export function createStartAppiumRemoteSessionBuildFunction(
'appium:automationName': device.automationName,
'appium:udid': device.udid,
},
...(serveSim ? { webPreviewUrl: serveSim.previewUrl } : {}),
webPreviewUrl: webPreview.previewUrl,
},
logger,
});
@@ -170,8 +167,8 @@ export function createStartAppiumRemoteSessionBuildFunction(
: undefined,
});
} finally {
if (serveSim) {
await serveSim.stopAsync();
if (webPreview) {
await webPreview.stopAsync();
}
if (appiumTunnel) {
await appiumTunnel.stopAsync();
@@ -26,8 +26,8 @@ import {
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
startServeSimWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
@@ -186,7 +186,7 @@ export function createStartArgentRemoteSessionBuildFunction(
});
let toolsTunnel: Awaited<ReturnType<typeof startNgrokTunnelAsync>> | undefined;
let serveSim: Awaited<ReturnType<typeof startServeSimWithTunnelAsync>> | undefined;
let webPreview: Awaited<ReturnType<typeof startDeviceWebPreviewWithTunnelAsync>> | undefined;
try {
toolsTunnel = await startNgrokTunnelAsync({
port: toolServerPort,
@@ -199,18 +199,14 @@ export function createStartArgentRemoteSessionBuildFunction(
const publicToolsUrl = toolsTunnel.url;
logger.info(`Tunnel is ready at ${publicToolsUrl}.`);
// serve-sim is iOS-only — Android sessions go without a preview URL.
let webPreviewUrl: string | undefined;
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
serveSim = await startServeSimWithTunnelAsync(ctx, {
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
webPreviewUrl = serveSim.previewUrl;
logger.info(`Web preview URL: ${webPreviewUrl}`);
}
webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
logger.info(`Web preview URL: ${webPreview.previewUrl}`);
await uploadRemoteSessionConfigAsync({
ctx,
@@ -218,7 +214,7 @@ export function createStartArgentRemoteSessionBuildFunction(
remoteConfig: {
toolsUrl: publicToolsUrl,
...(toolServerToken ? { toolsAuthToken: toolServerToken } : {}),
...(webPreviewUrl ? { webPreviewUrl } : {}),
webPreviewUrl: webPreview.previewUrl,
},
logger,
});
@@ -238,8 +234,8 @@ export function createStartArgentRemoteSessionBuildFunction(
: undefined,
});
} finally {
if (serveSim) {
await serveSim.stopAsync();
if (webPreview) {
await webPreview.stopAsync();
}
if (toolsTunnel) {
await toolsTunnel.stopAsync();
@@ -10,22 +10,21 @@ import {
getDeviceRunSessionIdOrThrow,
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
startServeSimWithTunnelAsync,
startDeviceWebPreviewWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
const STARTUP_TIMEOUT_MS = 60_000;
export function createStartServeSimRemoteSessionBuildFunction(
export function createStartWebPreviewRemoteSessionBuildFunction(
ctx: CustomBuildContext
): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'start_serve_sim_remote_session',
name: 'Start serve-sim remote session',
name: 'Start web preview remote session',
__metricsId: 'eas/start_serve_sim_remote_session',
supportedRuntimePlatforms: [BuildRuntimePlatform.DARWIN],
inputProviders: [
BuildStepInput.createProvider({
id: 'package_version',
@@ -38,30 +37,34 @@ export function createStartServeSimRemoteSessionBuildFunction(
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
}),
],
fn: async ({ logger }, { inputs, env, signal }) => {
fn: async ({ logger, global }, { inputs, env, signal }) => {
const deviceRunSessionId = getDeviceRunSessionIdOrThrow(env);
const ngrokTunnelDomain = getNgrokTunnelDomainOrThrow(env);
const maxDurationSeconds = inputs.max_duration_seconds?.value as number | undefined;
const packageVersion = inputs.package_version?.value as string | undefined;
const { runtimePlatform } = global;
logger.info('Starting serve-sim remote session.');
logger.info(`Starting web preview remote session (runtime: ${runtimePlatform}).`);
await selectXcodeDeveloperDirectoryAsync({ env, logger });
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
await selectXcodeDeveloperDirectoryAsync({ env, logger });
}
const serveSim = await startServeSimWithTunnelAsync(ctx, {
const webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
packageVersion,
});
logger.info(`Preview URL: ${serveSim.previewUrl}`);
logger.info(`Preview URL: ${webPreview.previewUrl}`);
try {
await uploadRemoteSessionConfigAsync({
ctx,
deviceRunSessionId,
remoteConfig: { previewUrl: serveSim.previewUrl },
remoteConfig: { previewUrl: webPreview.previewUrl },
logger,
});
@@ -73,7 +76,7 @@ export function createStartServeSimRemoteSessionBuildFunction(
signal,
});
} finally {
await serveSim.stopAsync();
await webPreview.stopAsync();
}
},
});
@@ -13,14 +13,16 @@ import { Sentry } from '../../../sentry';
import { turtleFetch } from '../../../utils/turtleFetch';
import { sleepAsync } from '../../../utils/retry';
import {
createExpoDeviceHubArgs,
createServeSimArgs,
ensureFfmpegInstalledAsync,
fetchServeSimTurnArgsAsync,
fetchWebPreviewTurnArgsAsync,
metricsCorsOriginToServeSimArgs,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
turnIceServersToServeSimArgs,
turnIceServersToWebPreviewArgs,
waitForDeviceRunSessionStoppedAsync,
waitForServeSimReadyAsync,
waitForWebPreviewReadyAsync,
} from '../remoteDeviceRunSession';
jest.mock('@ngrok/ngrok');
@@ -132,11 +134,11 @@ describe(createServeSimArgs, () => {
'--webrtc-codec',
'vp8',
'--max-dimension',
'1280',
'960',
'--mjpeg-quality',
'0.55',
'--video-bitrate',
'3000000',
'6000000',
'--video-fps',
'60',
'--turn-url',
@@ -173,6 +175,49 @@ describe(createServeSimArgs, () => {
});
});
describe(createExpoDeviceHubArgs, () => {
it('uses the latest Expo package and applies the EAS Android streaming policy', () => {
expect(
createExpoDeviceHubArgs({
port: 4321,
turnArgs: ['--turn-url', 'turns:turn.example.test:443'],
})
).toEqual([
'--yes',
'expo-device-hub@latest',
'--port',
'4321',
'--host',
'127.0.0.1',
'--platform',
'android',
'--transport',
'webrtc',
'--webrtc-codec',
'h264',
'--webrtc-ice-policy',
'all',
'--max-dimension',
'1280',
'--video-bitrate',
'6000000',
'--video-fps',
'60',
'--hide-sidebar',
'--hide-boot-device',
'--turn-url',
'turns:turn.example.test:443',
]);
});
it('pins the requested package version', () => {
expect(createExpoDeviceHubArgs({ port: 4321, packageVersion: '0.7.0' }).slice(0, 2)).toEqual([
'--yes',
'[email protected]',
]);
});
});
describe(metricsCorsOriginToServeSimArgs, () => {
it('returns no args when the origin is unset or empty', () => {
expect(metricsCorsOriginToServeSimArgs({} as BuildStepEnv)).toEqual([]);
@@ -200,7 +245,7 @@ describe(metricsCorsOriginToServeSimArgs, () => {
});
});
describe(waitForServeSimReadyAsync, () => {
describe(waitForWebPreviewReadyAsync, () => {
beforeEach(() => {
jest.mocked(turtleFetch).mockReset();
jest.mocked(sleepAsync).mockReset();
@@ -215,8 +260,9 @@ describe(waitForServeSimReadyAsync, () => {
json: async () => ({ status: 'ready', device: 'DEVICE-A' }),
} as unknown as Awaited<ReturnType<typeof turtleFetch>>);
await waitForServeSimReadyAsync({
serveSim: { pid: undefined, getOutput: () => '' },
await waitForWebPreviewReadyAsync({
previewServer: { pid: undefined, getOutput: () => '' },
serverName: 'expo-device-hub',
port: 4321,
timeoutMs: 10_000,
});
@@ -261,14 +307,125 @@ describe(startNgrokTunnelAsync, () => {
});
});
describe(turnIceServersToServeSimArgs, () => {
describe(startDeviceWebPreviewWithTunnelAsync, () => {
const baseDomain = 'eas-simulator.ngrok.dev';
const turnArgs = [
'--turn-url',
'turns:turn.example.test:443',
'--turn-username',
'turn-user',
'--turn-credential',
'turn-credential',
];
const metricsCorsArgs = ['--metrics-cors-origin', 'https://metrics.expo.test'];
const env = {
DEVICE_RUN_SESSION_ID: 'drs-id',
EAS_SIMULATOR_METRICS_CORS_ORIGIN: 'https://metrics.expo.test',
NGROK_AUTHTOKEN: 'ngrok-token',
} as unknown as BuildStepEnv;
beforeEach(() => {
jest.mocked(spawn).mockReset();
jest.mocked(ngrok.forward).mockReset();
jest.mocked(turtleFetch).mockReset();
const spawnPromise = Object.assign(Promise.resolve(undefined), {
child: {
pid: undefined,
unref: jest.fn(),
},
});
jest.mocked(spawn).mockReturnValue(spawnPromise as never);
jest.mocked(turtleFetch).mockImplementation(async url => {
if (url.endsWith('/turn-ice-servers')) {
return {
json: async () => ({
data: {
iceServers: [
{
urls: ['turns:turn.example.test:443'],
username: 'turn-user',
credential: 'turn-credential',
},
],
},
}),
} as unknown as Awaited<ReturnType<typeof turtleFetch>>;
}
return {
json: async () => ({ status: 'ready', device: 'device-id' }),
} as unknown as Awaited<ReturnType<typeof turtleFetch>>;
});
});
it('starts expo-device-hub for Linux and cleans up the preview resources', async () => {
const packageVersion = '1.2.3';
const close = jest.fn().mockResolvedValue(undefined);
jest.mocked(ngrok.forward).mockResolvedValue({
url: () => 'https://android-preview.example.test',
close,
} as never);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.LINUX,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
packageVersion,
});
const [command, args] = jest.mocked(spawn).mock.calls[0];
const port = Number(args[args.indexOf('--port') + 1]);
expect(port).toBeGreaterThan(0);
expect(command).toBe('npx');
expect(args).toEqual(createExpoDeviceHubArgs({ port, turnArgs, packageVersion }));
expect(ngrok.forward).toHaveBeenCalledWith(expect.objectContaining({ addr: port }));
expect(preview.previewUrl).toBe('https://android-preview.example.test');
await preview.stopAsync();
expect(close).toHaveBeenCalledTimes(1);
});
it('starts serve-sim for Darwin with its metrics policy and cleans up the preview resources', async () => {
const packageVersion = '4.5.6';
const close = jest.fn().mockResolvedValue(undefined);
jest.mocked(ngrok.forward).mockResolvedValue({
url: () => 'https://ios-preview.example.test',
close,
} as never);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
packageVersion,
});
const [command, args] = jest.mocked(spawn).mock.calls[0];
const port = Number(args[args.indexOf('--port') + 1]);
expect(port).toBeGreaterThan(0);
expect(command).toBe('npx');
expect(args).toEqual(createServeSimArgs({ port, turnArgs, metricsCorsArgs, packageVersion }));
expect(ngrok.forward).toHaveBeenCalledWith(expect.objectContaining({ addr: port }));
expect(preview.previewUrl).toBe('https://ios-preview.example.test');
await preview.stopAsync();
expect(close).toHaveBeenCalledTimes(1);
});
});
describe(turnIceServersToWebPreviewArgs, () => {
it('returns no args for an empty ICE server list', () => {
expect(turnIceServersToServeSimArgs([])).toEqual([]);
expect(turnIceServersToWebPreviewArgs([])).toEqual([]);
});
it('builds --stun-url and --turn-url flags from Cloudflare ICE servers', () => {
expect(
turnIceServersToServeSimArgs([
turnIceServersToWebPreviewArgs([
{ urls: ['stun:stun.cloudflare.com:3478', 'stun:stun.cloudflare.com:53'] },
{
urls: [
@@ -293,7 +450,7 @@ describe(turnIceServersToServeSimArgs, () => {
it('emits only --turn-url flags when no credential-less (STUN) entry is present', () => {
expect(
turnIceServersToServeSimArgs([
turnIceServersToWebPreviewArgs([
{
urls: ['turns:turn.cloudflare.com:443?transport=tcp'],
username: 'u',
@@ -311,19 +468,19 @@ describe(turnIceServersToServeSimArgs, () => {
});
it('emits only --stun-url when there is no credentialed TURN entry', () => {
expect(turnIceServersToServeSimArgs([{ urls: ['stun:stun.cloudflare.com:3478'] }])).toEqual([
expect(turnIceServersToWebPreviewArgs([{ urls: ['stun:stun.cloudflare.com:3478'] }])).toEqual([
'--stun-url',
'stun:stun.cloudflare.com:3478',
]);
});
});
describe(fetchServeSimTurnArgsAsync, () => {
describe(fetchWebPreviewTurnArgsAsync, () => {
beforeEach(() => {
jest.mocked(turtleFetch).mockReset();
});
it('requests TURN ICE servers from the device run session endpoint and returns serve-sim args', async () => {
it('requests TURN ICE servers from the device run session endpoint and returns web preview args', async () => {
jest.mocked(turtleFetch).mockResolvedValue({
json: async () => ({
data: {
@@ -339,7 +496,7 @@ describe(fetchServeSimTurnArgsAsync, () => {
}),
} as unknown as Awaited<ReturnType<typeof turtleFetch>>);
const args = await fetchServeSimTurnArgsAsync(createCtxMock(), {
const args = await fetchWebPreviewTurnArgsAsync(createCtxMock(), {
env: createEnvMock(),
logger: createLoggerMock(),
});
@@ -363,11 +520,11 @@ describe(fetchServeSimTurnArgsAsync, () => {
);
});
it('returns [] and warns when the request fails so serve-sim falls back to P2P/STUN', async () => {
it('returns [] and warns when the request fails so the web preview falls back to P2P/STUN', async () => {
jest.mocked(turtleFetch).mockRejectedValue(new Error('boom'));
const logger = createLoggerMock();
const args = await fetchServeSimTurnArgsAsync(createCtxMock(), {
const args = await fetchWebPreviewTurnArgsAsync(createCtxMock(), {
env: createEnvMock(),
logger,
});
@@ -19,12 +19,16 @@ import { sleepAsync } from '../../utils/retry';
import { turtleFetch } from '../../utils/turtleFetch';
const XCODE_DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer';
const WEB_PREVIEW_HOST = '127.0.0.1';
const SERVE_SIM_PACKAGE_NAME = '@expo/serve-sim';
const SERVE_SIM_HOST = '127.0.0.1';
const SERVE_SIM_MAX_DIMENSION = '1280';
const SERVE_SIM_MAX_DIMENSION = '960';
const SERVE_SIM_MJPEG_QUALITY = '0.55';
const SERVE_SIM_VIDEO_BITRATE = '3000000';
const SERVE_SIM_VIDEO_BITRATE = '6000000';
const SERVE_SIM_VIDEO_FPS = '60';
const EXPO_DEVICE_HUB_PACKAGE_NAME = 'expo-device-hub';
const EXPO_DEVICE_HUB_MAX_DIMENSION = '1280';
const EXPO_DEVICE_HUB_VIDEO_BITRATE = '6000000';
const EXPO_DEVICE_HUB_VIDEO_FPS = '60';
const START_DEVICE_RUN_SESSION_MUTATION = graphql(`
mutation StartDeviceRunSession($deviceRunSessionId: ID!, $remoteConfig: JSONObject!) {
@@ -390,11 +394,12 @@ const TurnIceServersResponseSchema = z.object({
});
/**
* Translate Cloudflare ICE servers into serve-sim CLI flags: `--stun-url` (the
* Translate Cloudflare ICE servers into web preview CLI flags: `--stun-url` (the
* credential-less entries) and `--turn-url`/`--turn-username`/`--turn-credential`
* (the entry carrying the short-lived credentials).
* (the entry carrying the short-lived credentials). serve-sim and expo-device-hub
* intentionally expose the same ICE flag contract.
*/
export function turnIceServersToServeSimArgs(iceServers: TurnIceServers): string[] {
export function turnIceServersToWebPreviewArgs(iceServers: TurnIceServers): string[] {
const stunUrls = iceServers
.filter(server => !server.username && !server.credential)
.flatMap(server => server.urls);
@@ -420,14 +425,14 @@ export function turnIceServersToServeSimArgs(iceServers: TurnIceServers): string
/**
* Fetch short-lived Cloudflare TURN ICE servers for this job run from www
* (minted on demand, mirroring how the worker fetches project clone URLs) and
* translate them into serve-sim CLI flags.
* translate them into web preview CLI flags.
*
* Best-effort: on any failure we log and return [] so serve-sim falls back to
* its built-in P2P/STUN behavior. The credential is passed to serve-sim as a
* process arg and deliberately not logged (turtle-spawn never logs argv and the
* worker is single-tenant).
* Best-effort: on any failure we log and return [] so the preview server falls
* back to its built-in P2P/STUN behavior. The credential is passed as a process
* arg and deliberately not logged (turtle-spawn never logs argv and the worker
* is single-tenant).
*/
export async function fetchServeSimTurnArgsAsync(
export async function fetchWebPreviewTurnArgsAsync(
ctx: CustomBuildContext,
{ env, logger }: { env: BuildStepEnv; logger: bunyan }
): Promise<string[]> {
@@ -456,9 +461,9 @@ export async function fetchServeSimTurnArgsAsync(
);
const { data } = TurnIceServersResponseSchema.parse(await response.json());
const args = turnIceServersToServeSimArgs(data.iceServers);
const args = turnIceServersToWebPreviewArgs(data.iceServers);
if (args.length > 0) {
logger.info('Configured serve-sim with Cloudflare TURN ICE servers.');
logger.info('Configured the web preview with Cloudflare TURN ICE servers.');
}
return args;
} catch (err) {
@@ -466,7 +471,7 @@ export async function fetchServeSimTurnArgsAsync(
Sentry.capture('Could not fetch Cloudflare TURN ICE servers', error, { level: 'warning' });
logger.warn(
{ err: error },
'Could not fetch Cloudflare TURN ICE servers; serve-sim will fall back to P2P/STUN.'
'Could not fetch Cloudflare TURN ICE servers; the web preview will fall back to P2P/STUN.'
);
return [];
}
@@ -600,6 +605,10 @@ function createServeSimPackageSpec(packageVersion: string | undefined): string {
return `${SERVE_SIM_PACKAGE_NAME}@${packageVersion ?? 'latest'}`;
}
function createExpoDeviceHubPackageSpec(packageVersion: string | undefined): string {
return `${EXPO_DEVICE_HUB_PACKAGE_NAME}@${packageVersion ?? 'latest'}`;
}
export function createServeSimArgs({
port,
turnArgs = [],
@@ -617,7 +626,7 @@ export function createServeSimArgs({
'--port',
String(port),
'--host',
SERVE_SIM_HOST,
WEB_PREVIEW_HOST,
'--transport',
'webrtc',
'--webrtc-codec',
@@ -635,44 +644,84 @@ export function createServeSimArgs({
];
}
export function createExpoDeviceHubArgs({
port,
turnArgs = [],
packageVersion,
}: {
port: number;
turnArgs?: string[];
packageVersion?: string;
}): string[] {
return [
'--yes',
createExpoDeviceHubPackageSpec(packageVersion),
'--port',
String(port),
'--host',
WEB_PREVIEW_HOST,
'--platform',
'android',
'--transport',
'webrtc',
'--webrtc-codec',
'h264',
'--webrtc-ice-policy',
'all',
'--max-dimension',
EXPO_DEVICE_HUB_MAX_DIMENSION,
'--video-bitrate',
EXPO_DEVICE_HUB_VIDEO_BITRATE,
'--video-fps',
EXPO_DEVICE_HUB_VIDEO_FPS,
'--hide-sidebar',
'--hide-boot-device',
...turnArgs,
];
}
async function findAvailablePortAsync(): Promise<number> {
const server = createServer();
server.unref();
await new Promise<void>((resolve, reject) => {
server.once('error', reject);
server.listen(0, SERVE_SIM_HOST, () => resolve());
server.listen(0, WEB_PREVIEW_HOST, () => resolve());
});
const address = server.address();
await new Promise<void>((resolve, reject) => {
server.close(err => (err ? reject(err) : resolve()));
});
if (!address || typeof address === 'string') {
throw new SystemError('Could not allocate a local port for serve-sim.');
throw new SystemError('Could not allocate a local port for the web preview.');
}
return address.port;
}
const ServeSimReadyResponseSchema = z.object({
const WebPreviewReadyResponseSchema = z.object({
status: z.literal('ready'),
device: z.string(),
});
export async function waitForServeSimReadyAsync({
serveSim,
export async function waitForWebPreviewReadyAsync({
previewServer,
serverName,
port,
timeoutMs,
}: {
serveSim: Pick<DetachedProcessHandle, 'pid' | 'getOutput'>;
previewServer: Pick<DetachedProcessHandle, 'pid' | 'getOutput'>;
serverName: string;
port: number;
timeoutMs: number;
}): Promise<void> {
const readyUrl = `http://${SERVE_SIM_HOST}:${port}/readyz`;
const readyUrl = `http://${WEB_PREVIEW_HOST}:${port}/readyz`;
const deadline = Date.now() + timeoutMs;
let lastError: unknown;
while (Date.now() < deadline) {
if (serveSim.pid !== undefined && !isProcessRunning(serveSim.pid)) {
if (previewServer.pid !== undefined && !isProcessRunning(previewServer.pid)) {
throw new SystemError(
`serve-sim exited before becoming ready. Last output:\n${serveSim.getOutput() || '<empty>'}`
`${serverName} exited before becoming ready. Last output:\n${
previewServer.getOutput() || '<empty>'
}`
);
}
try {
@@ -680,7 +729,7 @@ export async function waitForServeSimReadyAsync({
retries: 0,
timeout: 2_000,
});
ServeSimReadyResponseSchema.parse(await response.json());
WebPreviewReadyResponseSchema.parse(await response.json());
return;
} catch (error) {
lastError = error;
@@ -688,17 +737,75 @@ export async function waitForServeSimReadyAsync({
await sleepAsync(1_000);
}
throw new SystemError(
`Timed out waiting for serve-sim readiness at ${readyUrl}${
`Timed out waiting for ${serverName} readiness at ${readyUrl}${
lastError instanceof Error ? `: ${lastError.message}` : ''
}. Last output:\n${serveSim.getOutput() || '<empty>'}`
}. Last output:\n${previewServer.getOutput() || '<empty>'}`
);
}
export type ServeSimPreviewHandle = {
export type DeviceWebPreviewHandle = {
previewUrl: string;
stopAsync: () => Promise<void>;
};
export type ServeSimPreviewHandle = DeviceWebPreviewHandle;
async function startWebPreviewWithTunnelAsync(
ctx: CustomBuildContext,
{
baseDomain,
env,
logger,
timeoutMs,
serverName,
packageSpec,
createArgs,
}: {
baseDomain: string;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
serverName: string;
packageSpec: string;
createArgs: (port: number, turnArgs: string[]) => string[];
}
): Promise<DeviceWebPreviewHandle> {
const port = await findAvailablePortAsync();
logger.info(`Launching ${packageSpec} on ${WEB_PREVIEW_HOST}:${port}.`);
const turnArgs = await fetchWebPreviewTurnArgsAsync(ctx, { env, logger });
const previewServer = spawnDetached({
command: 'npx',
args: createArgs(port, turnArgs),
env,
});
try {
logger.info(`Waiting for ${serverName} to become ready.`);
await waitForWebPreviewReadyAsync({ previewServer, serverName, port, timeoutMs });
const tunnel = await startNgrokTunnelAsync({
port,
subdomainPrefix: 'web-preview',
baseDomain,
authtoken: getNgrokAuthtokenOrThrow(env),
logger,
});
return {
previewUrl: tunnel.url,
stopAsync: async () => {
const results = await Promise.allSettled([tunnel.stopAsync(), previewServer.stopAsync()]);
for (const result of results) {
if (result.status === 'rejected') {
logger.warn({ err: result.reason }, `Could not stop a ${serverName} preview resource.`);
}
}
},
};
} catch (error) {
await previewServer.stopAsync();
throw error;
}
}
export async function startServeSimWithTunnelAsync(
ctx: CustomBuildContext,
{
@@ -715,42 +822,65 @@ export async function startServeSimWithTunnelAsync(
packageVersion?: string;
}
): Promise<ServeSimPreviewHandle> {
const port = await findAvailablePortAsync();
logger.info(
`Launching ${createServeSimPackageSpec(packageVersion)} on ${SERVE_SIM_HOST}:${port}.`
);
const turnArgs = await fetchServeSimTurnArgsAsync(ctx, { env, logger });
const metricsCorsArgs = metricsCorsOriginToServeSimArgs(env);
const serveSim = spawnDetached({
command: 'npx',
args: createServeSimArgs({ port, turnArgs, metricsCorsArgs, packageVersion }),
return await startWebPreviewWithTunnelAsync(ctx, {
baseDomain,
env,
logger,
timeoutMs,
serverName: 'serve-sim',
packageSpec: createServeSimPackageSpec(packageVersion),
createArgs: (port, turnArgs) =>
createServeSimArgs({ port, turnArgs, metricsCorsArgs, packageVersion }),
});
}
try {
logger.info('Waiting for serve-sim to become ready.');
await waitForServeSimReadyAsync({ serveSim, port, timeoutMs });
const tunnel = await startNgrokTunnelAsync({
port,
subdomainPrefix: 'web-preview',
baseDomain,
authtoken: getNgrokAuthtokenOrThrow(env),
logger,
});
return {
previewUrl: tunnel.url,
stopAsync: async () => {
const results = await Promise.allSettled([tunnel.stopAsync(), serveSim.stopAsync()]);
for (const result of results) {
if (result.status === 'rejected') {
logger.warn({ err: result.reason }, 'Could not stop a serve-sim preview resource.');
}
}
},
};
} catch (error) {
await serveSim.stopAsync();
throw error;
export async function startExpoDeviceHubWithTunnelAsync(
ctx: CustomBuildContext,
{
baseDomain,
env,
logger,
timeoutMs,
packageVersion,
}: {
baseDomain: string;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
packageVersion?: string;
}
): Promise<DeviceWebPreviewHandle> {
return await startWebPreviewWithTunnelAsync(ctx, {
baseDomain,
env,
logger,
timeoutMs,
serverName: 'expo-device-hub',
packageSpec: createExpoDeviceHubPackageSpec(packageVersion),
createArgs: (port, turnArgs) => createExpoDeviceHubArgs({ port, turnArgs, packageVersion }),
});
}
export async function startDeviceWebPreviewWithTunnelAsync(
ctx: CustomBuildContext,
{
runtimePlatform,
...options
}: {
runtimePlatform: BuildRuntimePlatform;
baseDomain: string;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
packageVersion?: string;
}
): Promise<DeviceWebPreviewHandle> {
switch (runtimePlatform) {
case BuildRuntimePlatform.DARWIN:
return await startServeSimWithTunnelAsync(ctx, options);
case BuildRuntimePlatform.LINUX:
return await startExpoDeviceHubWithTunnelAsync(ctx, options);
}
}
@@ -24,7 +24,8 @@ export type AndroidDeviceSerialId = string & z.BRAND<'AndroidDeviceSerialId'>;
export namespace AndroidEmulatorUtils {
const RETRY_INTERVAL_MS = 1_000;
export const defaultSystemImagePackage = `system-images;android-30;default;${
export const defaultDeviceIdentifier = 'pixel_9' as AndroidDeviceName;
export const defaultSystemImagePackage = `system-images;android-36;default;${
process.arch === 'arm64' ? 'arm64-v8a' : 'x86_64'
}`;
@@ -0,0 +1,78 @@
import fs from 'fs';
import os from 'os';
import path from 'path';
import {
compressCocoapodsCacheAsync,
getCocoapodsCachePaths,
resolveCocoapodsCacheKeyAsync,
restoreCocoapodsCacheArchiveAsync,
} from '../cocoapodsCache';
jest.unmock('fs');
jest.unmock('node:fs');
jest.unmock('fs/promises');
jest.unmock('node:fs/promises');
describe('CocoaPods cache utilities', () => {
let workingDirectory: string;
const cocoapodsVersion = '1.16.2';
beforeEach(async () => {
workingDirectory = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'cocoapods-cache-test-'));
await fs.promises.mkdir(path.join(workingDirectory, 'ios'), { recursive: true });
});
afterEach(async () => {
await fs.promises.rm(workingDirectory, { recursive: true, force: true });
});
it('uses a CocoaPods version prefix when no Podfile.lock exists', async () => {
await expect(
resolveCocoapodsCacheKeyAsync(workingDirectory, cocoapodsVersion)
).resolves.toEqual({
key: 'ios-pods-1.16.2-',
keyPrefix: 'ios-pods-1.16.2-',
});
});
it('includes the Podfile.lock hash when it exists', async () => {
const { podfileLockPath } = getCocoapodsCachePaths(workingDirectory);
await fs.promises.writeFile(podfileLockPath, 'PODS:\n - Expo (55.0.0)\n');
const firstResult = await resolveCocoapodsCacheKeyAsync(workingDirectory, cocoapodsVersion);
expect(firstResult.key).toMatch(/^ios-pods-1\.16\.2-[a-f0-9]+$/);
expect(firstResult.keyPrefix).toBe('ios-pods-1.16.2-');
await fs.promises.writeFile(podfileLockPath, 'PODS:\n - Expo (56.0.0)\n');
const secondResult = await resolveCocoapodsCacheKeyAsync(workingDirectory, cocoapodsVersion);
expect(secondResult.key).not.toBe(firstResult.key);
});
it('preserves symlinks and executable permissions through an archive round trip', async () => {
const { podsDirectory } = getCocoapodsCachePaths(workingDirectory);
const frameworkVersionsDirectory = path.join(podsDirectory, 'Example.framework', 'Versions');
const frameworkVersionDirectory = path.join(frameworkVersionsDirectory, 'A');
await fs.promises.mkdir(frameworkVersionDirectory, { recursive: true });
await fs.promises.writeFile(path.join(frameworkVersionDirectory, 'Example'), 'binary');
await fs.promises.symlink('A', path.join(frameworkVersionsDirectory, 'Current'));
const scriptPath = path.join(podsDirectory, 'Target Support Files', 'script.sh');
await fs.promises.mkdir(path.dirname(scriptPath), { recursive: true });
await fs.promises.writeFile(scriptPath, '#!/bin/sh\n');
await fs.promises.chmod(scriptPath, 0o755);
const { archivePath } = await compressCocoapodsCacheAsync({ workingDirectory });
await fs.promises.rm(podsDirectory, { recursive: true, force: true });
await restoreCocoapodsCacheArchiveAsync({ archivePath, workingDirectory });
const restoredSymlinkPath = path.join(frameworkVersionsDirectory, 'Current');
expect((await fs.promises.lstat(restoredSymlinkPath)).isSymbolicLink()).toBe(true);
await expect(fs.promises.readlink(restoredSymlinkPath)).resolves.toBe('A');
const restoredScriptStat = await fs.promises.stat(scriptPath);
expect(restoredScriptStat.mode & 0o111).toBe(0o111);
await fs.promises.rm(path.dirname(archivePath), { recursive: true, force: true });
});
});
@@ -78,7 +78,7 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(result).toEqual({
cmd: 'npx',
args: ['-y', `eas-cli@${STAGING_VERSION}`],
extraEnv: {},
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
});
@@ -91,7 +91,7 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(result).toEqual({
cmd: 'npx',
args: ['-y', `eas-cli@${STAGING_VERSION}`],
extraEnv: {},
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
});
@@ -101,7 +101,7 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(result).toEqual({
cmd: 'npx',
args: ['-y', `eas-cli@${STAGING_VERSION}`],
extraEnv: { EXPO_STAGING: '1' },
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0', EXPO_STAGING: '1' },
});
expect(spawn).not.toHaveBeenCalled();
});
@@ -112,7 +112,7 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(result).toEqual({
cmd: 'npx',
args: ['-y', `eas-cli@${PRODUCTION_VERSION}`],
extraEnv: {},
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
expect(spawn).not.toHaveBeenCalled();
expect(Sentry.capture).not.toHaveBeenCalled();
@@ -133,7 +133,7 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(stagingResult).toEqual({
cmd: 'npx',
args: ['-y', `eas-cli@${EasCliNpmTags.STAGING}`],
extraEnv: { EXPO_STAGING: '1' },
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0', EXPO_STAGING: '1' },
});
process.env.ENVIRONMENT = 'production';
@@ -141,7 +141,7 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(productionResult).toEqual({
cmd: 'npx',
args: ['-y', `eas-cli@${EasCliNpmTags.PRODUCTION}`],
extraEnv: {},
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
expect(Sentry.capture).toHaveBeenCalledWith(
@@ -173,9 +173,17 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(result).toEqual({
cmd: 'npx',
args: [`eas-cli@${PRODUCTION_VERSION}`],
extraEnv: {},
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
});
it('disables the minimum release age for every npx invocation', async () => {
process.env.ENVIRONMENT = 'production';
const result = await resolveEasCommandPrefixAndEnvAsync();
expect(result.extraEnv).toEqual({ NPM_CONFIG_MIN_RELEASE_AGE: '0' });
});
});
describe(runEasCliCommand, () => {
@@ -218,4 +226,25 @@ describe(runEasCliCommand, () => {
})
);
});
it('overrides a user-provided minimum release age', async () => {
process.env.ENVIRONMENT = 'production';
await runEasCliCommand({
args: ['deploy', '--json'],
options: {
cwd: '/tmp/project',
env: {
FOO: 'bar',
NPM_CONFIG_MIN_RELEASE_AGE: '365',
},
},
});
const spawnEnv = jest.mocked(spawn).mock.calls[0][2]?.env;
expect(spawnEnv).toEqual({
FOO: 'bar',
NPM_CONFIG_MIN_RELEASE_AGE: '0',
});
});
});
@@ -0,0 +1,17 @@
import { formatSecondsForLog } from '../formatDuration';
describe(formatSecondsForLog, () => {
it.each([
[0, '0 seconds'],
[1, '1 second'],
[45, '45 seconds'],
[60, '1 minute'],
[90, '1 minute 30 seconds'],
[120, '2 minutes'],
[3600, '1 hour'],
[3661, '1 hour 1 minute 1 second'],
[7200, '2 hours'],
] as const)('formats %i as %s', (seconds, expected) => {
expect(formatSecondsForLog(seconds)).toBe(expected);
});
});
@@ -0,0 +1,43 @@
import { ChildProcess } from 'node:child_process';
import { isChildProcessAlive } from '../processes';
function child(partial: {
exitCode: number | null;
signalCode: NodeJS.Signals | null;
killed: boolean;
}): ChildProcess {
return partial as ChildProcess;
}
describe(isChildProcessAlive, () => {
it('is alive while the process is running', () => {
expect(isChildProcessAlive(child({ exitCode: null, signalCode: null, killed: false }))).toBe(
true
);
});
it('is dead after a normal exit', () => {
expect(isChildProcessAlive(child({ exitCode: 0, signalCode: null, killed: false }))).toBe(
false
);
expect(isChildProcessAlive(child({ exitCode: 1, signalCode: null, killed: false }))).toBe(
false
);
});
it('is dead after an external signal termination (the regression case)', () => {
expect(
isChildProcessAlive(child({ exitCode: null, signalCode: 'SIGTERM', killed: false }))
).toBe(false);
expect(
isChildProcessAlive(child({ exitCode: null, signalCode: 'SIGKILL', killed: false }))
).toBe(false);
});
it('is dead once we have killed it', () => {
expect(isChildProcessAlive(child({ exitCode: null, signalCode: null, killed: true }))).toBe(
false
);
});
});
@@ -0,0 +1,503 @@
import { SshSettings, SystemError } from '@expo/eas-build-job';
import { createMockLogger } from '../../__tests__/utils/logger';
import { BuildContext } from '../../context';
import { sleepAsync } from '../retry';
import {
SshSessionHandle,
getSshIdleTimeoutSeconds,
getSshRelayServerUrl,
isSshEnabled,
startSshSessionAsync,
superviseSshSessionAsync,
} from '../turtleSshSession';
import { SshConnectionConfig, UptermHost, startUptermHostAsync } from '../upterm';
jest.mock('../retry', () => ({
sleepAsync: jest.fn().mockResolvedValue(undefined),
}));
jest.mock('../upterm', () => ({
...jest.requireActual('../upterm'),
startUptermHostAsync: jest.fn(),
}));
const mockedStartUptermHost = jest.mocked(startUptermHostAsync);
function sshJob(idleTimeoutSeconds: number): { ssh: SshSettings } {
return { ssh: { idleTimeoutSeconds, relayServerUrl: 'wss://relay.expo.dev' } };
}
function superviseHandle(
overrides: Partial<SshSessionHandle> &
Pick<SshSessionHandle, 'getConnectedClientCountAsync' | 'ensureConnectedAsync'>
): SshSessionHandle {
return {
stopAsync: async () => {},
...overrides,
};
}
describe(isSshEnabled, () => {
it('is true when job.ssh is present', () => {
expect(isSshEnabled(sshJob(0))).toBe(true);
});
it('is false otherwise', () => {
expect(isSshEnabled({})).toBe(false);
});
});
describe(getSshIdleTimeoutSeconds, () => {
it('returns idleTimeoutSeconds from job.ssh', () => {
expect(getSshIdleTimeoutSeconds(sshJob(900))).toBe(900);
expect(getSshIdleTimeoutSeconds(sshJob(0))).toBe(0);
expect(getSshIdleTimeoutSeconds(sshJob(3600))).toBe(3600);
});
it('defaults to 0 when job.ssh is missing', () => {
expect(getSshIdleTimeoutSeconds({})).toBe(0);
});
it('throws when the value is not an integer in range', () => {
for (const idleTimeoutSeconds of [300.5, -1, 3601]) {
expect(() => getSshIdleTimeoutSeconds(sshJob(idleTimeoutSeconds))).toThrow(SystemError);
}
});
});
describe(getSshRelayServerUrl, () => {
it('returns the relay url from the job ssh settings', () => {
expect(getSshRelayServerUrl(sshJob(0))).toBe('wss://relay.expo.dev');
});
it('throws when the job carries no relay url', () => {
expect(() => getSshRelayServerUrl({})).toThrow(/no relay server URL/);
});
});
describe(startSshSessionAsync, () => {
const config1: SshConnectionConfig = {
type: 'upterm-v1',
host: 'relay.expo.dev',
secret: 'secret-1',
};
const config2: SshConnectionConfig = { ...config1, secret: 'secret-2' };
const target = { type: 'JOB_RUN', id: 'jr-1' } as const;
let createOrUpdateResult: { data?: unknown; error?: { message: string } };
let mutation: jest.Mock;
let ctx: BuildContext;
function makeHost(overrides: Partial<UptermHost> = {}): UptermHost {
const host: UptermHost = {
connectionConfig: config1,
getConnectedClientCountAsync: jest.fn().mockResolvedValue(3),
isAlive: jest.fn().mockReturnValue(true),
redialAsync: jest.fn(async () => {
host.connectionConfig = config2;
return config2;
}),
stopAsync: jest.fn().mockResolvedValue(undefined),
...overrides,
};
return host;
}
beforeEach(() => {
createOrUpdateResult = {
data: {
turtleSshSession: {
createOrUpdateTurtleSshSession: {
id: 'ts-1',
sessionSettings: { idleTimeoutSeconds: 900 },
},
},
},
};
mutation = jest.fn().mockReturnValue({ toPromise: async () => createOrUpdateResult });
ctx = {
logger: createMockLogger(),
graphqlClient: { mutation },
} as unknown as BuildContext;
});
it('creates or updates the session and returns a handle plus idle timeout that delegates to the host', async () => {
const host = makeHost();
mockedStartUptermHost.mockResolvedValue(host);
const { handle, idleTimeoutSeconds } = await startSshSessionAsync(ctx, {
target,
relayServerUrl: 'wss://relay.expo.dev',
idleTimeoutSeconds: 900,
});
expect(mutation).toHaveBeenCalledWith(expect.anything(), {
target: { type: 'JOB_RUN', id: 'jr-1' },
connectionConfig: { ...config1, reconnecting: false, type: 'UPTERM_V1' },
sessionSettings: { idleTimeoutSeconds: 900 },
});
expect(idleTimeoutSeconds).toBe(900);
await handle.getConnectedClientCountAsync();
expect(host.getConnectedClientCountAsync).toHaveBeenCalled();
await handle.stopAsync();
expect(host.stopAsync).toHaveBeenCalled();
});
it('reports a build target', async () => {
const host = makeHost();
mockedStartUptermHost.mockResolvedValue(host);
await startSshSessionAsync(ctx, {
target: { type: 'BUILD', id: 'b-1' },
relayServerUrl: 'wss://relay.expo.dev',
idleTimeoutSeconds: 900,
});
expect(mutation).toHaveBeenCalledWith(expect.anything(), {
target: { type: 'BUILD', id: 'b-1' },
connectionConfig: { ...config1, reconnecting: false, type: 'UPTERM_V1' },
sessionSettings: { idleTimeoutSeconds: 900 },
});
});
it('throws and tears down the dialed host when creating or updating the session fails', async () => {
const host = makeHost();
mockedStartUptermHost.mockResolvedValue(host);
createOrUpdateResult = { error: { message: 'boom' } };
await expect(
startSshSessionAsync(ctx, {
target,
relayServerUrl: 'wss://r',
idleTimeoutSeconds: 300,
})
).rejects.toThrow('boom');
expect(host.stopAsync).toHaveBeenCalled();
});
it('still rethrows when teardown after a create/update failure also fails', async () => {
const host = makeHost({
stopAsync: jest.fn().mockRejectedValue(new Error('already gone')),
});
mockedStartUptermHost.mockResolvedValue(host);
createOrUpdateResult = { error: { message: 'boom' } };
await expect(
startSshSessionAsync(ctx, {
target,
relayServerUrl: 'wss://r',
idleTimeoutSeconds: 300,
})
).rejects.toThrow('boom');
expect(host.stopAsync).toHaveBeenCalled();
});
it('throws "no data returned" when create or update succeeds without a payload', async () => {
const host = makeHost();
mockedStartUptermHost.mockResolvedValue(host);
createOrUpdateResult = {};
await expect(
startSshSessionAsync(ctx, {
target,
relayServerUrl: 'wss://r',
idleTimeoutSeconds: 300,
})
).rejects.toThrow(/no data returned/);
expect(host.stopAsync).toHaveBeenCalled();
});
it('ensureConnected is a no-op while the host is alive', async () => {
const host = makeHost({ isAlive: jest.fn().mockReturnValue(true) });
mockedStartUptermHost.mockResolvedValue(host);
const { handle } = await startSshSessionAsync(ctx, {
target,
relayServerUrl: 'wss://r',
idleTimeoutSeconds: 300,
});
await handle.ensureConnectedAsync();
expect(host.redialAsync).not.toHaveBeenCalled();
expect(mutation).toHaveBeenCalledTimes(1);
});
it('marks reconnecting, redials, then reports the fresh config', async () => {
const host = makeHost({ isAlive: jest.fn().mockReturnValue(false) });
mockedStartUptermHost.mockResolvedValue(host);
const { handle } = await startSshSessionAsync(ctx, {
target,
relayServerUrl: 'wss://r',
idleTimeoutSeconds: 300,
});
await handle.ensureConnectedAsync();
expect(host.redialAsync).toHaveBeenCalledTimes(1);
expect(mutation).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({
connectionConfig: expect.objectContaining({ reconnecting: true }),
})
);
expect(mutation).toHaveBeenLastCalledWith(expect.anything(), {
target: { type: 'JOB_RUN', id: 'jr-1' },
connectionConfig: { ...config2, reconnecting: false, type: 'UPTERM_V1' },
sessionSettings: { idleTimeoutSeconds: 300 },
});
});
it('continues redial when the reconnecting status update fails', async () => {
const host = makeHost({ isAlive: jest.fn().mockReturnValue(false) });
mockedStartUptermHost.mockResolvedValue(host);
mutation
.mockReturnValueOnce({ toPromise: async () => createOrUpdateResult }) // initial create
.mockReturnValueOnce({
toPromise: async () => {
throw new Error('www unreachable');
},
}) // reconnecting=true best-effort
.mockReturnValue({ toPromise: async () => createOrUpdateResult }); // post-redial
const { handle } = await startSshSessionAsync(ctx, {
target,
relayServerUrl: 'wss://r',
idleTimeoutSeconds: 300,
});
await handle.ensureConnectedAsync();
expect(host.redialAsync).toHaveBeenCalledTimes(1);
});
it('retries only the report (no second redial) when reporting the fresh config fails', async () => {
let alive = false;
const host = makeHost({
isAlive: jest.fn(() => alive),
});
host.redialAsync = jest.fn(async () => {
alive = true;
host.connectionConfig = config2;
return config2;
});
mockedStartUptermHost.mockResolvedValue(host);
mutation = jest
.fn()
.mockReturnValueOnce({ toPromise: async () => createOrUpdateResult })
.mockReturnValueOnce({ toPromise: async () => createOrUpdateResult })
.mockReturnValueOnce({ toPromise: async () => ({ error: { message: 'flaky' } }) })
.mockReturnValue({ toPromise: async () => createOrUpdateResult });
ctx = { logger: createMockLogger(), graphqlClient: { mutation } } as unknown as BuildContext;
const { handle } = await startSshSessionAsync(ctx, {
target,
relayServerUrl: 'wss://r',
idleTimeoutSeconds: 300,
});
await handle.ensureConnectedAsync();
expect(host.redialAsync).toHaveBeenCalledTimes(1);
expect(mutation).toHaveBeenLastCalledWith(expect.anything(), {
target: { type: 'JOB_RUN', id: 'jr-1' },
connectionConfig: { ...config2, reconnecting: false, type: 'UPTERM_V1' },
sessionSettings: { idleTimeoutSeconds: 300 },
});
});
it('throws once the redial budget is exhausted', async () => {
const host = makeHost({
isAlive: jest.fn().mockReturnValue(false),
redialAsync: jest.fn().mockRejectedValue(new Error('relay down')),
});
mockedStartUptermHost.mockResolvedValue(host);
const { handle } = await startSshSessionAsync(ctx, {
target,
relayServerUrl: 'wss://r',
idleTimeoutSeconds: 300,
});
await expect(handle.ensureConnectedAsync()).rejects.toThrow(/after 10 attempts/);
expect(host.redialAsync).toHaveBeenCalledTimes(10);
});
});
describe(superviseSshSessionAsync, () => {
const logger = createMockLogger();
const mockedSleep = jest.mocked(sleepAsync);
beforeEach(() => {
jest.clearAllMocks();
mockedSleep.mockResolvedValue(undefined);
});
it('keeps the session open while the job is running, even with a 0 idle timeout', async () => {
let polls = 0;
await superviseSshSessionAsync({
handle: superviseHandle({
ensureConnectedAsync: async () => {
if (polls >= 5) {
throw new SystemError('stop the loop');
}
},
getConnectedClientCountAsync: async () => {
polls += 1;
return 0;
},
}),
idleTimeoutSeconds: 0,
hasJobFinished: () => false,
logger,
});
expect(polls).toBe(5);
});
it('closes as soon as the job finishes when the idle timeout is 0', async () => {
let polls = 0;
await superviseSshSessionAsync({
handle: superviseHandle({
ensureConnectedAsync: async () => {},
getConnectedClientCountAsync: async () => {
polls += 1;
return 0;
},
}),
idleTimeoutSeconds: 0,
hasJobFinished: () => true,
logger,
});
expect(polls).toBe(1);
expect(logger.info).toHaveBeenCalledWith(
'The job finished and no SSH client is connected. Closing the session.'
);
});
it('waits for a connected client to disconnect before closing with a 0 idle timeout', async () => {
const clientCounts = [1, 1, 0];
let polls = 0;
await superviseSshSessionAsync({
handle: superviseHandle({
ensureConnectedAsync: async () => {},
getConnectedClientCountAsync: async () => {
polls += 1;
return clientCounts.shift() ?? 0;
},
}),
idleTimeoutSeconds: 0,
hasJobFinished: () => true,
logger,
});
expect(polls).toBe(3);
});
it('starts the idle countdown when the job finishes, not when the session opens', async () => {
jest.useFakeTimers();
jest.setSystemTime(0);
try {
let jobHasFinished = false;
let polls = 0;
mockedSleep.mockImplementation(async () => {
jest.setSystemTime(Date.now() + 60_000);
});
await superviseSshSessionAsync({
handle: superviseHandle({
ensureConnectedAsync: async () => {},
getConnectedClientCountAsync: async () => {
polls += 1;
if (polls >= 5) {
jobHasFinished = true;
}
return 0;
},
}),
idleTimeoutSeconds: 300,
hasJobFinished: () => jobHasFinished,
logger,
});
// Nobody connects, so the 4 idle minutes spent during the job must not count towards the
// 5 minute timeout: it closes 5 minutes after the job finished, on the 10th poll.
expect(polls).toBe(10);
} finally {
jest.useRealTimers();
}
});
it('logs the connected client count whenever it changes', async () => {
const clientCounts = [1, 2, 1, 0];
await superviseSshSessionAsync({
handle: superviseHandle({
ensureConnectedAsync: async () => {},
getConnectedClientCountAsync: async () => clientCounts.shift() ?? 0,
}),
idleTimeoutSeconds: 0,
hasJobFinished: () => true,
logger,
});
expect(logger.info).toHaveBeenCalledWith('SSH clients connected: 1');
expect(logger.info).toHaveBeenCalledWith('SSH clients connected: 2');
expect(logger.info).toHaveBeenCalledWith('SSH clients connected: 1');
expect(logger.info).toHaveBeenCalledWith('SSH clients connected: 0');
});
it('closes the session when the relay connection cannot be restored', async () => {
let clientCountCalls = 0;
await superviseSshSessionAsync({
handle: superviseHandle({
ensureConnectedAsync: async () => {
throw new SystemError('relay down');
},
getConnectedClientCountAsync: async () => {
clientCountCalls += 1;
return 1;
},
}),
idleTimeoutSeconds: 300,
hasJobFinished: () => false,
logger,
});
expect(clientCountCalls).toBe(0);
});
it('does not close while a client stays connected, even past the idle timeout', async () => {
let polls = 0;
await superviseSshSessionAsync({
handle: superviseHandle({
ensureConnectedAsync: async () => {
if (polls > 3) {
throw new SystemError('stop the loop');
}
},
getConnectedClientCountAsync: async () => {
polls += 1;
return 1;
},
}),
idleTimeoutSeconds: 0,
hasJobFinished: () => true,
logger,
});
expect(polls).toBeGreaterThan(3);
});
it('closes when the client count cannot be read', async () => {
let polls = 0;
await superviseSshSessionAsync({
handle: superviseHandle({
ensureConnectedAsync: async () => {},
getConnectedClientCountAsync: async () => {
polls += 1;
throw new SystemError(
'Could not read the SSH client count from the upterm admin socket.'
);
},
}),
idleTimeoutSeconds: 300,
hasJobFinished: () => false,
logger,
});
expect(polls).toBe(1);
expect(logger.warn).toHaveBeenCalledWith(
expect.anything(),
'Could not read the SSH client count. Closing the session.'
);
});
});
@@ -0,0 +1,466 @@
import { SystemError } from '@expo/eas-build-job';
import downloadFile from '@expo/downloader';
import spawn from '@expo/turtle-spawn';
import fs from 'node:fs/promises';
import { createMockLogger } from '../../__tests__/utils/logger';
import { BuildContext } from '../../context';
import { sleepAsync } from '../retry';
import {
connectionConfigFromUptermSession,
redactConnectionSecrets,
redactSpawnErrorForLog,
resolveUptermGcsObjectName,
startUptermHostAsync,
} from '../upterm';
jest.mock('@expo/turtle-spawn', () => ({ __esModule: true, default: jest.fn() }));
jest.mock('@expo/downloader', () => ({ __esModule: true, default: jest.fn() }));
jest.mock('../retry', () => ({ sleepAsync: jest.fn() }));
jest.mock('node:fs/promises', () => ({
__esModule: true,
default: {
access: jest.fn(),
mkdir: jest.fn(),
chmod: jest.fn(),
mkdtemp: jest.fn(),
writeFile: jest.fn(),
rm: jest.fn(),
readdir: jest.fn(),
},
}));
describe(resolveUptermGcsObjectName, () => {
it('maps worker platforms to GCS object names', () => {
expect(resolveUptermGcsObjectName('darwin', 'arm64')).toBe('upterm-darwin-arm64');
expect(resolveUptermGcsObjectName('linux', 'x64')).toBe('upterm-linux-amd64');
});
it('rejects unsupported platforms', () => {
expect(() => resolveUptermGcsObjectName('darwin', 'x64')).toThrow(/only available/);
expect(() => resolveUptermGcsObjectName('linux', 'arm64')).toThrow(/only available/);
});
});
describe(connectionConfigFromUptermSession, () => {
it('parses sessionId and host from session current fields', () => {
expect(
connectionConfigFromUptermSession({
sessionId: 'TOKENabc123',
host: 'ssh://relay.expo.dev:22',
})
).toEqual({
type: 'upterm-v1',
host: 'relay.expo.dev',
secret: 'TOKENabc123',
});
});
it('accepts a bare hostname', () => {
expect(connectionConfigFromUptermSession({ sessionId: 'tok', host: 'relay.expo.dev' })).toEqual(
{
type: 'upterm-v1',
host: 'relay.expo.dev',
secret: 'tok',
}
);
});
it('keeps a non-default port so the CLI can pass it to ssh -p', () => {
expect(
connectionConfigFromUptermSession({
sessionId: 'tok',
host: 'ssh://relay.expo.dev:2222',
})
).toEqual({
type: 'upterm-v1',
host: 'relay.expo.dev:2222',
secret: 'tok',
});
});
it('returns null when the host is not a parseable url', () => {
expect(connectionConfigFromUptermSession({ sessionId: 'tok', host: 'ssh://' })).toBeNull();
expect(connectionConfigFromUptermSession({ sessionId: 'tok', host: 'ssh://[' })).toBeNull();
});
});
describe(redactConnectionSecrets, () => {
it('strips the userinfo secret from a connection string', () => {
expect(
redactConnectionSecrets('Connect: upterm proxy wss://[email protected] now')
).toBe('Connect: upterm proxy wss://<redacted>@relay.expo.dev now');
});
it('strips the bare token that trails the ssh ProxyCommand line', () => {
const line =
"ssh -o ProxyCommand='upterm proxy wss://[email protected]' [email protected]";
const redacted = redactConnectionSecrets(line);
expect(redacted).not.toContain('TOKENabc123');
expect(redacted).toBe(
"ssh -o ProxyCommand='upterm proxy wss://<redacted>@uptermd.upterm.dev' <redacted>@uptermd.upterm.dev"
);
});
it('strips the bare token for ws:// relay URLs too', () => {
const line =
"ssh -o ProxyCommand='upterm proxy ws://[email protected]' [email protected]";
const redacted = redactConnectionSecrets(line);
expect(redacted).not.toContain('TOKENabc123');
expect(redacted).toBe(
"ssh -o ProxyCommand='upterm proxy ws://<redacted>@relay.local' <redacted>@relay.local"
);
});
it('leaves output without userinfo untouched', () => {
expect(redactConnectionSecrets('dialing wss://relay.expo.dev')).toBe(
'dialing wss://relay.expo.dev'
);
});
it('strips control characters (incl. ESC) while keeping newlines', () => {
// ESC alone is enough to neutralize CSI; leftover printable "[31m" is harmless in logs.
expect(redactConnectionSecrets('ok\n\u001b[31mfail\u001b[0m\r\nbad\u0007')).toBe(
'ok\n[31mfail[0m\nbad'
);
});
});
describe(redactSpawnErrorForLog, () => {
it('redacts secrets in message, stdout, and stderr', () => {
expect(
redactSpawnErrorForLog({
message: 'failed wss://[email protected]',
stdout: 'upterm proxy wss://[email protected]',
stderr: 'ssh dial error: wss://[email protected]',
code: 1,
})
).toEqual({
message: 'failed wss://<redacted>@relay.expo.dev',
stdout: 'upterm proxy wss://<redacted>@relay.expo.dev',
stderr: 'ssh dial error: wss://<redacted>@relay.expo.dev',
code: 1,
});
});
it('passes through non-objects', () => {
expect(redactSpawnErrorForLog('boom')).toBe('boom');
expect(redactSpawnErrorForLog(null)).toBeNull();
});
it('leaves non-string message/stdout/stderr fields alone', () => {
expect(
redactSpawnErrorForLog({
message: 42,
stdout: Buffer.from('x'),
code: 1,
})
).toEqual({
message: 42,
stdout: Buffer.from('x'),
code: 1,
});
});
});
describe(startUptermHostAsync, () => {
const mockedSpawn = jest.mocked(spawn);
const mockedSleep = jest.mocked(sleepAsync);
const mockedFs = jest.mocked(fs);
const mockedDownloadFile = jest.mocked(downloadFile);
let hostProcess: {
child: {
stdout: { on: (event: string, cb: (chunk: Buffer) => void) => void };
stderr: { on: jest.Mock };
pid: number;
kill: jest.Mock;
unref: jest.Mock;
exitCode: number | null;
signalCode: NodeJS.Signals | null;
killed: boolean;
};
catch: (cb: (err: unknown) => void) => Promise<void>;
};
let sessionStdout: string;
let uptermOnPath = true;
function makeHostProcess(): typeof hostProcess {
return {
child: {
stdout: {
on: jest.fn((_event: string, cb: (chunk: Buffer) => void) => {
cb(Buffer.from('host output'));
}),
},
stderr: { on: jest.fn() },
pid: 4242,
kill: jest.fn(),
unref: jest.fn(),
exitCode: null,
signalCode: null,
killed: false,
},
catch: cb => {
cb(new Error('exited'));
return Promise.resolve();
},
};
}
function makeCtx(env: Record<string, string> = {}): BuildContext {
return {
logger: createMockLogger(),
env,
} as unknown as BuildContext;
}
beforeEach(() => {
jest.clearAllMocks();
jest.spyOn(process, 'kill').mockReturnValue(true);
hostProcess = makeHostProcess();
uptermOnPath = true;
sessionStdout = JSON.stringify({
sessionId: 'TOKENx',
host: 'ssh://relay.expo.dev:22',
clientCount: 0,
});
mockedSleep.mockResolvedValue(undefined);
mockedFs.mkdir.mockResolvedValue(undefined as never);
mockedFs.chmod.mockResolvedValue(undefined as never);
mockedFs.mkdtemp.mockImplementation((async (prefix: string) =>
String(prefix).includes('eas-upterm') ? '/tmp/eas-upterm-1' : '/tmp/eas-ssh-1') as never);
mockedFs.writeFile.mockResolvedValue(undefined as never);
mockedFs.rm.mockResolvedValue(undefined as never);
mockedFs.readdir.mockResolvedValue(['default.sock'] as never);
mockedDownloadFile.mockResolvedValue(undefined as never);
mockedSpawn.mockImplementation(((cmd: string, args: string[]) => {
if (cmd === 'upterm' && Array.isArray(args) && args[0] === 'version') {
if (!uptermOnPath) {
return Promise.reject(new Error('not found')) as never;
}
return Promise.resolve({ stdout: 'upterm version 0.24.0\n', stderr: '' }) as never;
}
if (Array.isArray(args) && args[0] === 'host') {
return hostProcess as never;
}
if (Array.isArray(args) && args[0] === 'session') {
return Promise.resolve({ stdout: sessionStdout, stderr: '' }) as never;
}
return Promise.resolve({ stdout: '', stderr: '' }) as never;
}) as never);
});
it('uses upterm on PATH, dials, and returns a handle with the parsed config', async () => {
const env = { PATH: '/job/bin', CUSTOM: '1' };
const host = await startUptermHostAsync(makeCtx(env), {
relayServerUrl: 'wss://relay.expo.dev',
});
expect(host.connectionConfig).toEqual({
type: 'upterm-v1',
host: 'relay.expo.dev',
secret: 'TOKENx',
});
expect(host.isAlive()).toBe(true);
expect(mockedDownloadFile).not.toHaveBeenCalled();
expect(mockedSpawn).toHaveBeenCalledWith('upterm', ['version'], {
stdio: 'pipe',
env,
});
expect(mockedSpawn).toHaveBeenCalledWith(
'upterm',
expect.arrayContaining(['host']),
expect.objectContaining({ env: expect.objectContaining({ CUSTOM: '1', PATH: '/job/bin' }) })
);
});
it('downloads from GCS into a fresh temp dir when upterm is not on PATH', async () => {
uptermOnPath = false;
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://relay.expo.dev' });
expect(host.connectionConfig.secret).toBe('TOKENx');
expect(mockedDownloadFile).toHaveBeenCalledWith(
expect.stringMatching(/storage\.googleapis\.com\/turtle-v2\/upterm\/upterm-/),
expect.stringContaining('/tmp/eas-upterm-1'),
expect.objectContaining({ retry: 3 })
);
expect(mockedSpawn).toHaveBeenCalledWith(
expect.stringContaining('/tmp/eas-upterm-1'),
expect.arrayContaining(['host']),
expect.anything()
);
});
it('throws when upterm is missing from PATH and the GCS download fails', async () => {
uptermOnPath = false;
mockedDownloadFile.mockRejectedValue(new Error('403 Forbidden') as never);
await expect(startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' })).rejects.toThrow(
/could not be downloaded/
);
});
it('still throws when download fails with a non-Error and temp cleanup also fails', async () => {
uptermOnPath = false;
mockedDownloadFile.mockRejectedValue('network down' as never);
mockedFs.rm.mockRejectedValue(new Error('EPERM') as never);
await expect(startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' })).rejects.toThrow(
/network down/
);
});
it('tears down and rethrows when the host never registers with the relay', async () => {
jest.useFakeTimers();
jest.setSystemTime(0);
try {
mockedFs.readdir.mockResolvedValue([] as never);
mockedSleep.mockImplementation(async () => {
jest.setSystemTime(Date.now() + 61_000);
});
await expect(startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' })).rejects.toThrow(
/did not register/
);
expect(process.kill).toHaveBeenCalledWith(-4242, 'SIGTERM');
expect(mockedFs.rm).toHaveBeenCalledWith('/tmp/eas-ssh-1', expect.anything());
} finally {
jest.useRealTimers();
}
});
it('stopAsync kills the process and removes the state dir', async () => {
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
await host.stopAsync();
expect(process.kill).toHaveBeenCalledWith(-4242, 'SIGTERM');
expect(mockedFs.rm).toHaveBeenCalledWith('/tmp/eas-ssh-1', expect.anything());
});
it('falls back to child.kill when process.kill on the process group fails', async () => {
jest.spyOn(process, 'kill').mockImplementation(() => {
throw new Error('ESRCH');
});
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
await host.stopAsync();
expect(hostProcess.child.kill).toHaveBeenCalled();
});
it('skips process.kill when the host process has no pid', async () => {
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
hostProcess.child.pid = undefined as unknown as number;
await host.stopAsync();
expect(process.kill).not.toHaveBeenCalled();
expect(hostProcess.child.kill).not.toHaveBeenCalled();
});
it('redialAsync re-establishes the connection', async () => {
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
const config = await host.redialAsync();
expect(config).toEqual({
type: 'upterm-v1',
host: 'relay.expo.dev',
secret: 'TOKENx',
});
expect(host.connectionConfig).toEqual(config);
});
it('redialAsync waits for the old process to exit before clearing its socket dir', async () => {
const order: string[] = [];
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
// Keep the give-up timer pending so only the process exit can unblock the redial.
mockedSleep.mockImplementation(() => new Promise<void>(() => {}));
let resolveExit: (() => void) | undefined;
hostProcess.catch = () =>
new Promise<void>(resolve => {
resolveExit = () => {
order.push('exited');
resolve();
};
});
mockedFs.rm.mockImplementation((() => {
order.push('rm');
return Promise.resolve(undefined);
}) as never);
jest.spyOn(process, 'kill').mockImplementation(() => {
order.push('kill');
// A real SIGTERM is asynchronous, so the exit lands after the kill returns.
setImmediate(() => resolveExit?.());
return true;
});
await host.redialAsync();
expect(order).toEqual(['kill', 'exited', 'rm']);
});
it('redialAsync gives up waiting when the old process does not exit', async () => {
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
hostProcess.catch = () => new Promise<void>(() => {});
await expect(host.redialAsync()).resolves.toEqual({
type: 'upterm-v1',
host: 'relay.expo.dev',
secret: 'TOKENx',
});
expect(mockedSleep).toHaveBeenCalledWith(5_000);
});
it('continues when the previous socket dir cannot be cleared', async () => {
mockedFs.rm.mockImplementation(((dir: string) =>
dir === '/tmp/eas-ssh-1/upterm'
? Promise.reject(new Error('busy'))
: Promise.resolve(undefined)) as never);
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
expect(host.connectionConfig.secret).toBe('TOKENx');
});
describe('getConnectedClientCountAsync', () => {
it('throws when no admin socket exists yet (after retries)', async () => {
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
mockedFs.readdir.mockResolvedValue([] as never);
await expect(host.getConnectedClientCountAsync()).rejects.toThrow(/client count/i);
});
it('returns the client count reported by the admin socket', async () => {
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
sessionStdout = JSON.stringify({
sessionId: 'TOKENx',
host: 'relay.expo.dev',
clientCount: 4,
});
expect(await host.getConnectedClientCountAsync()).toBe(4);
});
it('throws when the count query fails (after retries)', async () => {
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
mockedSpawn.mockImplementation((() => Promise.reject(new Error('socket gone'))) as never);
await expect(host.getConnectedClientCountAsync()).rejects.toThrow(/client count/i);
});
it('throws when the reported count is not a number (after retries)', async () => {
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
sessionStdout = JSON.stringify({
sessionId: 'TOKENx',
host: 'relay.expo.dev',
clientCount: 'nope',
});
await expect(host.getConnectedClientCountAsync()).rejects.toThrow(/client count/i);
});
it('throws when the socket dir cannot be read (after retries)', async () => {
const host = await startUptermHostAsync(makeCtx(), { relayServerUrl: 'wss://r' });
mockedFs.readdir.mockRejectedValue(new Error('no dir') as never);
await expect(host.getConnectedClientCountAsync()).rejects.toThrow(/client count/i);
});
});
});
@@ -0,0 +1,106 @@
import { hashFiles } from '@expo/steps';
import fs from 'fs';
import os from 'os';
import path from 'path';
import * as tar from 'tar';
export const COCOAPODS_CACHE_KEY_PREFIX = 'ios-pods-';
const PODS_DIRECTORY_NAME = 'Pods';
const PODFILE_LOCK_NAME = 'Podfile.lock';
export function getCocoapodsCachePaths(workingDirectory: string): {
iosDirectory: string;
podsDirectory: string;
podfileLockPath: string;
} {
const iosDirectory = path.join(workingDirectory, 'ios');
return {
iosDirectory,
podsDirectory: path.join(iosDirectory, PODS_DIRECTORY_NAME),
podfileLockPath: path.join(iosDirectory, PODFILE_LOCK_NAME),
};
}
export async function resolveCocoapodsCacheKeyAsync(
workingDirectory: string,
cocoapodsVersion: string
): Promise<{ key: string; keyPrefix: string }> {
const normalizedVersion = cocoapodsVersion.trim();
if (!normalizedVersion) {
throw new Error('Failed to determine CocoaPods version');
}
const keyPrefix = `${COCOAPODS_CACHE_KEY_PREFIX}${normalizedVersion}-`;
const { podfileLockPath } = getCocoapodsCachePaths(workingDirectory);
try {
await fs.promises.access(podfileLockPath);
} catch {
return { key: keyPrefix, keyPrefix };
}
return {
key: `${keyPrefix}${hashFiles([podfileLockPath])}`,
keyPrefix,
};
}
export async function compressCocoapodsCacheAsync({
workingDirectory,
}: {
workingDirectory: string;
}): Promise<{ archivePath: string }> {
const { iosDirectory, podsDirectory } = getCocoapodsCachePaths(workingDirectory);
await fs.promises.access(podsDirectory);
const archiveDestinationDirectory = await fs.promises.mkdtemp(
path.join(os.tmpdir(), 'save-cocoapods-cache-')
);
const archivePath = path.join(archiveDestinationDirectory, 'cache.tar.gz');
await tar.create(
{
file: archivePath,
cwd: iosDirectory,
gzip: true,
},
[PODS_DIRECTORY_NAME]
);
return { archivePath };
}
export async function restoreCocoapodsCacheArchiveAsync({
archivePath,
workingDirectory,
}: {
archivePath: string;
workingDirectory: string;
}): Promise<void> {
const { iosDirectory, podsDirectory } = getCocoapodsCachePaths(workingDirectory);
await fs.promises.mkdir(iosDirectory, { recursive: true });
const temporaryRestoreDirectory = await fs.promises.mkdtemp(
path.join(iosDirectory, '.eas-pods-cache-')
);
try {
await tar.extract({
file: archivePath,
cwd: temporaryRestoreDirectory,
filter: entryPath =>
entryPath === PODS_DIRECTORY_NAME || entryPath.startsWith(`${PODS_DIRECTORY_NAME}/`),
});
const restoredPodsDirectory = path.join(temporaryRestoreDirectory, PODS_DIRECTORY_NAME);
const restoredPodsStat = await fs.promises.stat(restoredPodsDirectory);
if (!restoredPodsStat.isDirectory()) {
throw new Error('CocoaPods cache archive does not contain a Pods directory');
}
await fs.promises.rm(podsDirectory, { recursive: true, force: true });
await fs.promises.rename(restoredPodsDirectory, podsDirectory);
} finally {
await fs.promises.rm(temporaryRestoreDirectory, { recursive: true, force: true });
}
}
+5 -3
View File
@@ -12,6 +12,8 @@ import { Sentry } from '../sentry';
let cachedEasCliVersionsPromise: Promise<EasCliVersions> | undefined;
const NPX_ENV = { NPM_CONFIG_MIN_RELEASE_AGE: '0' };
/**
* Resolves the `eas-cli` versions to install for staging/production. Prefers the
* versions committed to `cli-versions.json` (fetched from expo/eas-cli); on any
@@ -74,21 +76,21 @@ export async function resolveEasCommandPrefixAndEnvAsync(): Promise<{
return {
cmd: 'npx',
args: [...npxArgsPrefix, `eas-cli@${versions.STAGING}`],
extraEnv: {},
extraEnv: NPX_ENV,
};
} else if (process.env.ENVIRONMENT === 'staging') {
const versions = await resolveEasCliVersionsAsync();
return {
cmd: 'npx',
args: [...npxArgsPrefix, `eas-cli@${versions.STAGING}`],
extraEnv: { EXPO_STAGING: '1' },
extraEnv: { ...NPX_ENV, EXPO_STAGING: '1' },
};
} else {
const versions = await resolveEasCliVersionsAsync();
return {
cmd: 'npx',
args: [...npxArgsPrefix, `eas-cli@${versions.PRODUCTION}`],
extraEnv: {},
extraEnv: NPX_ENV,
};
}
}
@@ -0,0 +1,16 @@
export function formatSecondsForLog(totalSeconds: number): string {
const hours = Math.floor(totalSeconds / 3600);
const minutes = Math.floor((totalSeconds % 3600) / 60);
const seconds = totalSeconds % 60;
const parts: string[] = [];
if (hours > 0) {
parts.push(hours === 1 ? '1 hour' : `${hours} hours`);
}
if (minutes > 0) {
parts.push(minutes === 1 ? '1 minute' : `${minutes} minutes`);
}
if (seconds > 0 || parts.length === 0) {
parts.push(seconds === 1 ? '1 second' : `${seconds} seconds`);
}
return parts.join(' ');
}
@@ -1,5 +1,26 @@
import { ChildProcess } from 'node:child_process';
import spawn from '@expo/turtle-spawn';
export function isChildProcessAlive(child: ChildProcess): boolean {
return child.exitCode === null && child.signalCode === null && !child.killed;
}
/**
* Kill a detached spawn's process group. Negated pid targets the group so bash/sleep
* children cannot survive after the parent is gone (e.g. across upterm redial).
*/
export function killProcessGroup(child: ChildProcess): void {
if (child.pid == null) {
return;
}
try {
process.kill(-child.pid, 'SIGTERM');
} catch {
child.kill();
}
}
async function getChildrenPidsAsync(parentPids: number[]): Promise<number[]> {
try {
const result = await spawn('pgrep', ['-P', parentPids.join(',')], {
@@ -0,0 +1,250 @@
import { Job, SshSettings, SystemError } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import { graphql } from 'gql.tada';
import { formatSecondsForLog } from './formatDuration';
import { sleepAsync } from './retry';
import { SshConnectionConfig, startUptermHostAsync } from './upterm';
import { BuildContext } from '../context';
import { Sentry } from '../sentry';
const MAX_SSH_REDIALS = 10;
const REDIAL_BACKOFF_MS = 6_000;
const CLIENT_COUNT_POLL_INTERVAL_MS = 5_000;
const MAX_SSH_IDLE_TIMEOUT_SECONDS = 3600;
const DEFAULT_SSH_IDLE_TIMEOUT_SECONDS = 0;
const CREATE_OR_UPDATE_TURTLE_SSH_SESSION_MUTATION = graphql(`
mutation CreateOrUpdateTurtleSshSession(
$target: TurtleSshTargetInput!
$connectionConfig: TurtleSshConnectionConfigInput!
$sessionSettings: TurtleSshSessionSettingsInput!
) {
turtleSshSession {
createOrUpdateTurtleSshSession(
target: $target
connectionConfig: $connectionConfig
sessionSettings: $sessionSettings
) {
id
sessionSettings {
idleTimeoutSeconds
}
}
}
}
`);
export type TurtleSshTarget = { type: 'BUILD' | 'JOB_RUN'; id: string };
export function isSshEnabled(job: Pick<Job, 'ssh'>): boolean {
return job.ssh != null;
}
export function getSshIdleTimeoutSeconds(job: Pick<Job, 'ssh'>): number {
const idleTimeoutSeconds = job.ssh?.idleTimeoutSeconds ?? DEFAULT_SSH_IDLE_TIMEOUT_SECONDS;
if (
!Number.isInteger(idleTimeoutSeconds) ||
idleTimeoutSeconds < 0 ||
idleTimeoutSeconds > MAX_SSH_IDLE_TIMEOUT_SECONDS
) {
throw new SystemError(
`SSH idle timeout must be an integer between 0 and ${MAX_SSH_IDLE_TIMEOUT_SECONDS} seconds, got ${idleTimeoutSeconds}.`,
{ trackingCode: 'SSH_IDLE_TIMEOUT_INVALID' }
);
}
return idleTimeoutSeconds;
}
export function getSshRelayServerUrl(job: Pick<Job, 'ssh'>): string {
const relayServerUrl = job.ssh?.relayServerUrl;
if (!relayServerUrl) {
throw new SystemError('SSH is enabled but no relay server URL was configured on the job.', {
trackingCode: 'SSH_RELAY_SERVER_URL_MISSING',
});
}
return relayServerUrl;
}
export type SshSessionHandle = {
getConnectedClientCountAsync: () => Promise<number>;
ensureConnectedAsync: () => Promise<void>;
stopAsync: () => Promise<void>;
};
export type StartedSshSession = {
handle: SshSessionHandle;
idleTimeoutSeconds: number;
};
async function createOrUpdateSessionAsync(
ctx: BuildContext,
{
target,
connectionConfig,
idleTimeoutSeconds,
}: {
target: TurtleSshTarget;
connectionConfig: SshConnectionConfig & { reconnecting?: boolean };
idleTimeoutSeconds: number;
}
): Promise<{ idleTimeoutSeconds: number }> {
const result = await ctx.graphqlClient
.mutation(CREATE_OR_UPDATE_TURTLE_SSH_SESSION_MUTATION, {
target,
connectionConfig: {
...connectionConfig,
type: 'UPTERM_V1',
},
sessionSettings: { idleTimeoutSeconds },
})
.toPromise();
if (result.error || !result.data) {
throw new SystemError(
`Failed to create or update the SSH session: ${result.error?.message ?? 'no data returned'}`,
{ cause: result.error }
);
}
const session = result.data.turtleSshSession.createOrUpdateTurtleSshSession;
return { idleTimeoutSeconds: session.sessionSettings.idleTimeoutSeconds };
}
export async function startSshSessionAsync(
ctx: BuildContext,
{
target,
relayServerUrl,
idleTimeoutSeconds: requestedIdleTimeoutSeconds,
}: {
target: TurtleSshTarget;
} & SshSettings
): Promise<StartedSshSession> {
const logger = ctx.logger;
const host = await startUptermHostAsync(ctx, { relayServerUrl });
let idleTimeoutSeconds: number;
try {
({ idleTimeoutSeconds } = await createOrUpdateSessionAsync(ctx, {
target,
connectionConfig: { ...host.connectionConfig, reconnecting: false },
idleTimeoutSeconds: requestedIdleTimeoutSeconds,
}));
} catch (err) {
await host.stopAsync().catch(() => {});
throw err;
}
const ensureConnectedAsync = async (): Promise<void> => {
if (host.isAlive()) {
return;
}
for (let attempt = 1; attempt <= MAX_SSH_REDIALS; attempt++) {
try {
if (!host.isAlive()) {
logger.warn('The SSH relay connection dropped. Reconnecting...');
await createOrUpdateSessionAsync(ctx, {
target,
connectionConfig: { ...host.connectionConfig, reconnecting: true },
idleTimeoutSeconds: requestedIdleTimeoutSeconds,
}).catch(() => {});
await host.redialAsync();
}
await createOrUpdateSessionAsync(ctx, {
target,
connectionConfig: { ...host.connectionConfig, reconnecting: false },
idleTimeoutSeconds: requestedIdleTimeoutSeconds,
});
logger.info('The SSH relay connection was restored.');
return;
} catch (err) {
logger.warn({ err }, `SSH reconnect attempt ${attempt} of ${MAX_SSH_REDIALS} failed.`);
if (attempt < MAX_SSH_REDIALS) {
await sleepAsync(REDIAL_BACKOFF_MS);
}
}
}
throw new SystemError(
`The SSH relay connection dropped and could not be restored after ${MAX_SSH_REDIALS} attempts.`
);
};
return {
handle: {
getConnectedClientCountAsync: () => host.getConnectedClientCountAsync(),
ensureConnectedAsync,
stopAsync: () => host.stopAsync(),
},
idleTimeoutSeconds,
};
}
export async function superviseSshSessionAsync({
handle,
idleTimeoutSeconds,
hasJobFinished,
logger,
}: {
handle: SshSessionHandle;
idleTimeoutSeconds: number;
hasJobFinished: () => boolean;
logger: bunyan;
}): Promise<void> {
const idleTimeoutMs = idleTimeoutSeconds * 1_000;
let idleSince: number | null = null;
let previousClientCount = 0;
for (;;) {
try {
await handle.ensureConnectedAsync();
} catch (err) {
logger.warn({ err }, 'Could not restore the SSH relay connection. Closing the session.');
Sentry.capture(
'Could not restore the SSH relay connection',
err instanceof Error ? err : undefined,
{
tags: { trackingCode: 'SSH_RELAY_RECONNECT_FAILED' },
}
);
return;
}
let connectedClientCount: number;
try {
connectedClientCount = await handle.getConnectedClientCountAsync();
} catch (err) {
logger.warn({ err }, 'Could not read the SSH client count. Closing the session.');
Sentry.capture(
'Could not read the SSH client count',
err instanceof Error ? err : undefined,
{
tags: { trackingCode: 'SSH_CLIENT_COUNT_UNREADABLE' },
}
);
return;
}
if (connectedClientCount !== previousClientCount) {
logger.info(`SSH clients connected: ${connectedClientCount}`);
previousClientCount = connectedClientCount;
}
const jobHasFinished = hasJobFinished();
if (connectedClientCount > 0 || !jobHasFinished) {
idleSince = null;
} else if (idleSince === null) {
idleSince = Date.now();
}
if (jobHasFinished && connectedClientCount === 0) {
if (idleTimeoutSeconds === 0) {
logger.info('The job finished and no SSH client is connected. Closing the session.');
return;
}
if (idleSince !== null && Date.now() - idleSince >= idleTimeoutMs) {
logger.info(
`No SSH client connected for ${formatSecondsForLog(idleTimeoutSeconds)} after the job finished. Closing the session.`
);
return;
}
}
await sleepAsync(CLIENT_COUNT_POLL_INTERVAL_MS);
}
}
+318
View File
@@ -0,0 +1,318 @@
import { Env, SystemError } from '@expo/eas-build-job';
import downloadFile from '@expo/downloader';
import spawn, { SpawnPromise, SpawnResult } from '@expo/turtle-spawn';
import fs from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { z } from 'zod';
import { isChildProcessAlive, killProcessGroup } from './processes';
import { sleepAsync } from './retry';
import { BuildContext } from '../context';
const CONTROL_CHARACTERS = /[\u0000-\u0008\u000B-\u001F\u007F-\u009F]/g;
export function resolveUptermGcsObjectName(
platform: NodeJS.Platform = process.platform,
arch: string = process.arch
): string {
if (platform === 'darwin' && arch === 'arm64') {
return 'upterm-darwin-arm64';
}
if (platform === 'linux' && arch === 'x64') {
return 'upterm-linux-amd64';
}
throw new SystemError(
`SSH upterm is only available on darwin/arm64 and linux/x64 (got ${platform}/${arch}).`
);
}
const UPTERM_GCS_BASE_URL = 'https://storage.googleapis.com/turtle-v2/upterm';
const UPTERM_DOWNLOAD_TIMEOUT_MS = 60_000;
const UPTERM_KEEPALIVE_SLEEP_SECONDS = 6 * 60 * 60;
const CONNECTION_POLL_INTERVAL_MS = 500;
const CONNECTION_STARTUP_TIMEOUT_MS = 60_000;
const PROCESS_EXIT_TIMEOUT_MS = 5_000;
const CLIENT_COUNT_READ_ATTEMPTS = 4;
const CLIENT_COUNT_READ_RETRY_MS = 500;
const DEFAULT_SSH_PORT = '22';
export type SshConnectionConfig = {
type: 'upterm-v1';
host: string;
secret: string;
};
export type UptermHost = {
connectionConfig: SshConnectionConfig;
getConnectedClientCountAsync: () => Promise<number>;
isAlive: () => boolean;
redialAsync: () => Promise<SshConnectionConfig>;
stopAsync: () => Promise<void>;
};
const UptermSessionJsonZ = z.object({
sessionId: z.string().min(1),
host: z.string().min(1),
clientCount: z.number().optional(),
});
type UptermSessionJson = z.infer<typeof UptermSessionJsonZ>;
export function connectionConfigFromUptermSession(
parsed: Pick<UptermSessionJson, 'sessionId' | 'host'>
): SshConnectionConfig | null {
let host = parsed.host;
if (host.includes('://')) {
let url: URL;
try {
url = new URL(host);
} catch {
return null;
}
host = url.port && url.port !== DEFAULT_SSH_PORT ? `${url.hostname}:${url.port}` : url.hostname;
}
if (!host) {
return null;
}
return { type: 'upterm-v1', host, secret: parsed.sessionId };
}
/**
* Strip upterm session secrets out of text before it reaches a log or an error message. upterm
* prints the session id as the userinfo of the connect line it advertises
* (`upterm proxy ws(s)://<sessionId>@host`) and repeats it bare as the ssh destination on the
* same line. Control characters are dropped first so they cannot split a token mid-match. The
* loop lifts the id out of the proxy URL and removes every occurrence of it; the final replace
* blanks userinfo in any other URL as a catch-all.
*/
export function redactConnectionSecrets(text: string): string {
let redacted = text.replace(CONTROL_CHARACTERS, '');
for (const [, token] of redacted.matchAll(/upterm proxy wss?:\/\/([^@\s]+)@/g)) {
redacted = redacted.split(token).join('<redacted>');
}
return redacted.replace(/([a-z][a-z0-9+.-]*:\/\/)[^@\s/]+@/gi, '$1<redacted>@');
}
export function redactSpawnErrorForLog(err: unknown): unknown {
if (!err || typeof err !== 'object') {
return err;
}
const spawnErr = err as { message?: unknown; stdout?: unknown; stderr?: unknown };
return {
...spawnErr,
...(typeof spawnErr.message === 'string'
? { message: redactConnectionSecrets(spawnErr.message) }
: {}),
...(typeof spawnErr.stdout === 'string'
? { stdout: redactConnectionSecrets(spawnErr.stdout) }
: {}),
...(typeof spawnErr.stderr === 'string'
? { stderr: redactConnectionSecrets(spawnErr.stderr) }
: {}),
};
}
export async function resolveUptermPathAsync(env: Env): Promise<string> {
try {
await spawn('upterm', ['version'], { stdio: 'pipe', env });
return 'upterm';
} catch {}
const objectName = resolveUptermGcsObjectName();
const downloadDir = await fs.mkdtemp(path.join(os.tmpdir(), 'eas-upterm-'));
const uptermPath = path.join(downloadDir, objectName);
const url = `${UPTERM_GCS_BASE_URL}/${objectName}`;
try {
await downloadFile(url, uptermPath, { retry: 3, timeout: UPTERM_DOWNLOAD_TIMEOUT_MS });
await fs.chmod(uptermPath, 0o755);
} catch (err) {
await fs.rm(downloadDir, { recursive: true, force: true }).catch(() => {});
throw new SystemError(
`The upterm SSH client was not on PATH and could not be downloaded from ${url}. ${
err instanceof Error ? err.message : String(err)
}`
);
}
return uptermPath;
}
async function findAdminSocketPathAsync(uptermSocketDir: string): Promise<string | null> {
// Use this dial's own admin socket, not upterm's default, which can still point at a previous
// dial's session after a redial and break client-count reads.
const entries = await fs.readdir(uptermSocketDir).catch(() => [] as string[]);
const socketName = entries.find(entry => entry.endsWith('.sock'));
return socketName ? path.join(uptermSocketDir, socketName) : null;
}
async function readCurrentSessionJsonAsync(
uptermPath: string,
adminSocketPath: string
): Promise<UptermSessionJson | null> {
try {
const result = await spawn(
uptermPath,
['session', 'current', '--admin-socket', adminSocketPath, '--output', 'json'],
{ stdio: 'pipe' }
);
const parsed = UptermSessionJsonZ.safeParse(JSON.parse(result.stdout));
return parsed.success ? parsed.data : null;
} catch {
return null;
}
}
async function waitForConnectionConfigAsync(
uptermPath: string,
uptermSocketDir: string,
getHostOutput: () => string
): Promise<SshConnectionConfig> {
const deadline = Date.now() + CONNECTION_STARTUP_TIMEOUT_MS;
for (;;) {
const adminSocketPath = await findAdminSocketPathAsync(uptermSocketDir);
if (adminSocketPath) {
const session = await readCurrentSessionJsonAsync(uptermPath, adminSocketPath);
if (session) {
const connectionConfig = connectionConfigFromUptermSession(session);
if (connectionConfig) {
return connectionConfig;
}
}
}
if (Date.now() >= deadline) {
throw new SystemError(
`The upterm client did not register with the relay within ${
CONNECTION_STARTUP_TIMEOUT_MS / 1_000
}s. Output:\n${redactConnectionSecrets(getHostOutput())}`
);
}
await sleepAsync(CONNECTION_POLL_INTERVAL_MS);
}
}
export async function startUptermHostAsync(
ctx: BuildContext,
{ relayServerUrl }: { relayServerUrl: string }
): Promise<UptermHost> {
const uptermPath = await resolveUptermPathAsync(ctx.env);
const stateDir = await fs.mkdtemp(path.join(os.tmpdir(), 'eas-ssh-'));
const hostKeyPath = path.join(stateDir, 'id_host');
const forceCommandPath = path.join(stateDir, 'join.sh');
const uptermSocketDir = path.join(stateDir, 'upterm');
await spawn('ssh-keygen', ['-t', 'ed25519', '-N', '', '-f', hostKeyPath, '-q'], {
logger: ctx.logger,
});
await fs.writeFile(forceCommandPath, '#!/usr/bin/env bash\nexec bash -l\n', { mode: 0o755 });
let currentProcess: SpawnPromise<SpawnResult> | null = null;
const stopCurrentProcessAsync = async (): Promise<void> => {
const previousProcess = currentProcess;
currentProcess = null;
if (!previousProcess) {
return;
}
killProcessGroup(previousProcess.child);
await Promise.race([
previousProcess.catch(() => {}),
sleepAsync(PROCESS_EXIT_TIMEOUT_MS).then(() => {
ctx.logger.debug('The previous upterm host process did not exit in time.');
}),
]);
};
const getConnectedClientCountAsync = async (): Promise<number> => {
for (let attempt = 1; attempt <= CLIENT_COUNT_READ_ATTEMPTS; attempt++) {
const adminSocketPath = await findAdminSocketPathAsync(uptermSocketDir);
const session = adminSocketPath
? await readCurrentSessionJsonAsync(uptermPath, adminSocketPath)
: null;
if (session && typeof session.clientCount === 'number') {
return session.clientCount;
}
if (attempt < CLIENT_COUNT_READ_ATTEMPTS) {
await sleepAsync(CLIENT_COUNT_READ_RETRY_MS);
}
}
throw new SystemError('Could not read the SSH client count from the upterm admin socket.', {
trackingCode: 'SSH_CLIENT_COUNT_UNREADABLE',
});
};
const dialAsync = async (): Promise<SshConnectionConfig> => {
await stopCurrentProcessAsync();
await fs.rm(uptermSocketDir, { recursive: true, force: true }).catch(err => {
ctx.logger.debug({ err }, 'Failed to clear the previous SSH socket directory.');
});
ctx.logger.debug('Connecting to the SSH relay.');
// --force-command is what each connecting SSH client runs (a login shell). The `sleep` after
// `--` is the host-side process that keeps `upterm host` up while nobody is connected.
const uptermProcess = spawn(
uptermPath,
[
'host',
'--server',
relayServerUrl,
'--accept',
'--skip-host-key-check',
'-i',
hostKeyPath,
'--force-command',
forceCommandPath,
'--',
'bash',
'-lc',
`sleep ${UPTERM_KEEPALIVE_SLEEP_SECONDS}`,
],
{
// upterm puts its admin socket under XDG_RUNTIME_DIR; point it at our state dir so we can
// find it for `session current` and clean it up on stop/redial.
env: { ...ctx.env, XDG_RUNTIME_DIR: stateDir },
stdio: ['ignore', 'pipe', 'pipe'],
detached: true,
}
);
uptermProcess.catch(err =>
ctx.logger.debug({ err: redactSpawnErrorForLog(err) }, 'The upterm host process exited.')
);
uptermProcess.child.unref();
currentProcess = uptermProcess;
let output = '';
const appendChunk = (chunk: Buffer | string): void => {
output += chunk.toString();
};
uptermProcess.child.stdout?.on('data', appendChunk);
uptermProcess.child.stderr?.on('data', appendChunk);
return await waitForConnectionConfigAsync(uptermPath, uptermSocketDir, () => output);
};
const stopAsync = async (): Promise<void> => {
await stopCurrentProcessAsync();
await fs.rm(stateDir, { recursive: true, force: true });
};
let connectionConfig: SshConnectionConfig;
try {
connectionConfig = await dialAsync();
} catch (err) {
await stopAsync();
throw err;
}
return {
get connectionConfig() {
return connectionConfig;
},
getConnectedClientCountAsync,
isAlive: () => currentProcess != null && isChildProcessAlive(currentProcess.child),
redialAsync: async () => {
connectionConfig = await dialAsync();
return connectionConfig;
},
stopAsync,
};
}
@@ -1,6 +1,6 @@
{
"name": "create-eas-build-function",
"version": "22.4.0",
"version": "23.1.0",
"description": "Create functions for use in EAS Build custom builds.",
"keywords": [
"expo",
@@ -35,7 +35,7 @@
"clean": "rimraf ./build/ \"*.tsbuildinfo\""
},
"dependencies": {
"@expo/steps": "22.4.0"
"@expo/steps": "23.1.0"
},
"devDependencies": {
"@expo/package-manager": "1.9.10",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/downloader",
"version": "22.0.0",
"version": "23.0.0",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "BUSL-1.1",
"author": "Expo <[email protected]>",
@@ -1,6 +1,6 @@
{
"name": "eas-build-cache-provider",
"version": "22.0.0",
"version": "23.0.0",
"description": "A build cache provider plugin for the Expo CLI",
"homepage": "https://github.com/expo/eas-cli",
"bugs": "https://github.com/expo/eas-cli/issues",
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/eas-build-job",
"version": "22.0.0",
"version": "23.1.0",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "MIT",
"author": "Expo <[email protected]>",
@@ -25,9 +25,9 @@
"clean": "rimraf node_modules dist coverage \"*.tsbuildinfo\""
},
"dependencies": {
"@expo/logger": "22.0.0",
"@expo/logger": "23.1.0",
"@expo/results": "1.0.0",
"@expo/turtle-spawn": "22.0.0",
"@expo/turtle-spawn": "23.1.0",
"joi": "^17.13.1",
"semver": "^7.6.2",
"zod": "^4.3.5"
+157 -147
View File
File diff suppressed because it is too large. Load diff
+1438 -25
View File
File diff suppressed because it is too large. Load diff
+11 -9
View File
@@ -1,6 +1,6 @@
{
"name": "eas-cli",
"version": "22.5.0",
"version": "23.1.0",
"description": "EAS command line tool",
"keywords": [
"cli",
@@ -48,11 +48,11 @@
"@expo/code-signing-certificates": "0.0.5",
"@expo/config": "55.0.10",
"@expo/config-plugins": "55.0.7",
"@expo/eas-build-job": "22.0.0",
"@expo/eas-json": "22.0.0",
"@expo/eas-build-job": "23.1.0",
"@expo/eas-json": "23.1.0",
"@expo/env": "^1.0.0",
"@expo/json-file": "8.3.3",
"@expo/logger": "22.0.0",
"@expo/logger": "23.1.0",
"@expo/multipart-body-parser": "2.0.0",
"@expo/osascript": "2.1.4",
"@expo/package-manager": "1.9.10",
@@ -64,7 +64,7 @@
"@expo/results": "1.0.0",
"@expo/rudder-sdk-node": "1.1.1",
"@expo/spawn-async": "1.7.2",
"@expo/steps": "22.4.0",
"@expo/steps": "23.1.0",
"@expo/timeago.js": "1.0.0",
"@oclif/core": "^4.8.3",
"@oclif/plugin-autocomplete": "^3.2.40",
@@ -72,11 +72,12 @@
"@sentry/node": "7.77.0",
"@urql/core": "4.0.11",
"@urql/exchange-retry": "1.2.0",
"agent-cli-detector": "0.1.6",
"agent-cli-detector": "0.1.7",
"ajv": "8.11.0",
"ajv-formats": "2.1.1",
"better-opn": "3.0.2",
"bplist-parser": "^0.3.0",
"centrifuge": "5.7.1",
"chalk": "4.1.2",
"cli-progress": "3.12.0",
"dateformat": "4.6.3",
@@ -104,7 +105,7 @@
"log-symbols": "4.1.0",
"mime": "3.0.0",
"minimatch": "5.1.2",
"minizlib": "3.0.1",
"minizlib": "3.1.0",
"nanoid": "3.3.8",
"node-fetch": "2.6.7",
"node-forge": "1.4.0",
@@ -130,8 +131,9 @@
"tslib": "2.6.2",
"turndown": "7.1.2",
"untildify": "4.0.0",
"uuid": "9.0.1",
"uuid": "11.1.1",
"wrap-ansi": "7.0.0",
"ws": "8.21.1",
"yaml": "2.6.0",
"zod": "^4.1.3"
},
@@ -159,8 +161,8 @@
"@types/tar": "6.1.10",
"@types/tar-stream": "3.1.3",
"@types/tough-cookie": "4.0.2",
"@types/uuid": "9.0.7",
"@types/wrap-ansi": "3.0.0",
"@types/ws": "8.5.10",
"axios": "1.18.1",
"eslint-plugin-graphql": "4.0.0",
"jest": "29.7.0",
@@ -8,10 +8,7 @@ import {
mockProjectId,
mockTestCommand,
} from './utils';
import {
fetchRawLogsForBuildJobAsync,
fetchRawLogsForCustomJobAsync,
} from '../../commandUtils/workflow/fetchLogs';
import { fetchRawLogsForJobAsync } from '../../commandUtils/workflow/logs/fetchLogs';
import WorkflowLogView from '../../commands/workflow/logs';
import { AppPlatform, BuildPriority, BuildStatus } from '../../graphql/generated';
import { AppQuery } from '../../graphql/queries/AppQuery';
@@ -38,7 +35,7 @@ jest.mock('fs');
jest.mock('../../log');
jest.mock('../../prompts');
jest.mock('../../utils/json');
jest.mock('../../commandUtils/workflow/fetchLogs');
jest.mock('../../commandUtils/workflow/logs/fetchLogs');
describe(WorkflowLogView, () => {
beforeEach(() => {
@@ -49,6 +46,7 @@ describe(WorkflowLogView, () => {
});
afterEach(() => {
jest.clearAllMocks();
jest.mocked(fetchRawLogsForJobAsync).mockReset();
});
test('view logs, passing in no parameters, no runs found', async () => {
const ctx = mockCommandContext(WorkflowLogView, {
@@ -105,7 +103,7 @@ describe(WorkflowLogView, () => {
return { selectedStep: 'step1' };
}
});
jest.mocked(fetchRawLogsForCustomJobAsync).mockResolvedValue(null);
jest.mocked(fetchRawLogsForJobAsync).mockResolvedValue(null);
await cmd.run();
expect(AppQuery.byIdWorkflowRunsFilteredByStatusAsync).toHaveBeenCalledWith(
ctx.loggedIn.graphqlClient,
@@ -139,7 +137,7 @@ describe(WorkflowLogView, () => {
}
});
jest
.mocked(fetchRawLogsForCustomJobAsync)
.mocked(fetchRawLogsForJobAsync)
.mockResolvedValue(
'{"result":"success","marker":"end-step","buildStepDisplayName":"Install dependencies","buildStepId":"step-id-1","time":"2022-01-01T00:00:00.000Z","msg":"test"}'
);
@@ -209,7 +207,7 @@ describe(WorkflowLogView, () => {
}
});
jest
.mocked(fetchRawLogsForBuildJobAsync)
.mocked(fetchRawLogsForJobAsync)
.mockResolvedValue(
'{"result":"success","marker":"end-step","buildStepDisplayName":"step1","buildStepId":"step-id-1","time":"2022-01-01T00:00:00.000Z","msg":"test"}'
);
@@ -255,7 +253,7 @@ describe(WorkflowLogView, () => {
}
});
jest
.mocked(fetchRawLogsForCustomJobAsync)
.mocked(fetchRawLogsForJobAsync)
.mockResolvedValue(
[
'{"result":"success","marker":"end-step","buildStepDisplayName":"Install","buildStepId":"step-id-1","time":"2022-01-01T00:00:00.000Z","msg":"first"}',
+10
View File
@@ -112,3 +112,13 @@ export function getEASUpdateURL(projectId: string, manifestHostOverride: string
export function getExpoApiWorkflowSchemaURL(): string {
return getExpoApiBaseUrl() + '/v2/workflows/schema';
}
export function getEASLogsWebsocketUrl(): string {
if (process.env.EXPO_STAGING) {
return `wss://staging-logs.expo.dev/connection/websocket`;
} else if (process.env.EXPO_LOCAL) {
return `ws://localhost:4997/connection/websocket`;
} else {
return `wss://logs.expo.dev/connection/websocket`;
}
}
@@ -1,68 +1,109 @@
import { canAccessRepositoryUsingSshAsync, runGitCloneAsync } from '../../../onboarding/git';
import glob from 'fast-glob';
import fs from 'fs-extra';
import { pipeline } from 'stream/promises';
import { extract } from 'tar';
import fetch from '../../../fetch';
import { installDependenciesAsync } from '../../../onboarding/installDependencies';
import { runCommandAsync } from '../../../onboarding/runCommand';
import { expoCommandAsync } from '../../../utils/expoCli';
import {
cloneTemplateAsync,
downloadTemplateAsync,
fetchTemplatePackumentAsync,
initializeGitRepositoryAsync,
installProjectDependenciesAsync,
} from '../commands';
jest.mock('../../../utils/expoCli');
jest.mock('../../../onboarding/git');
jest.mock('../../../fetch');
jest.mock('../../../onboarding/runCommand');
jest.mock('../../../onboarding/installDependencies');
jest.mock('../../../ora');
jest.mock('fs-extra');
jest.mock('tar');
jest.mock('stream/promises');
jest.mock('fast-glob');
describe('commands', () => {
beforeEach(() => {
jest.clearAllMocks();
});
describe('cloneTemplateAsync', () => {
it('should clone the template project with ssh', async () => {
describe('fetchTemplatePackumentAsync', () => {
it('should fetch the packument from the npm registry', async () => {
const packument = { 'dist-tags': { latest: '57.0.19' } };
jest.mocked(fetch).mockResolvedValueOnce({ json: async () => packument } as any);
expect(await fetchTemplatePackumentAsync()).toEqual(packument);
expect(fetch).toHaveBeenCalledWith(
'https://registry.npmjs.org/expo-template-default',
expect.objectContaining({
headers: { accept: 'application/vnd.npm.install-v1+json' },
})
);
});
});
describe('downloadTemplateAsync', () => {
const template = {
npmTag: 'sdk-56',
version: '56.0.34',
tarballUrl: 'https://registry.npmjs.org/expo-template-default/-/56.0.34.tgz',
};
it('should download and extract the template', async () => {
const targetProjectDir = '/test/target-project';
const finalTargetProjectDir = '/test/final-target-project';
const tarballBody = 'tarball-stream';
jest.mocked(canAccessRepositoryUsingSshAsync).mockResolvedValue(true);
jest.mocked(runGitCloneAsync).mockResolvedValue({
targetProjectDir: finalTargetProjectDir,
});
jest.mocked(fs.pathExists).mockResolvedValueOnce(false as never);
jest.mocked(fetch).mockResolvedValueOnce({ body: tarballBody } as any);
jest.mocked(glob).mockResolvedValueOnce(['gitignore', '_vscode'] as any);
const result = await cloneTemplateAsync(targetProjectDir);
const result = await downloadTemplateAsync(targetProjectDir, template);
expect(runGitCloneAsync).toHaveBeenCalledWith({
githubUsername: 'expo',
githubRepositoryName: 'expo-template-default',
targetProjectDir,
cloneMethod: 'ssh',
showOutput: false,
});
expect(result).toBe(finalTargetProjectDir);
expect(fetch).toHaveBeenCalledWith(
'https://registry.npmjs.org/expo-template-default/-/56.0.34.tgz'
);
expect(extract).toHaveBeenCalledWith({ cwd: targetProjectDir, strip: 1 });
expect(pipeline).toHaveBeenCalledWith(tarballBody, extract({} as any));
expect(fs.move).toHaveBeenCalledWith(
'/test/target-project/gitignore',
'/test/target-project/.gitignore',
{ overwrite: true }
);
expect(fs.move).toHaveBeenCalledWith(
'/test/target-project/_vscode',
'/test/target-project/.vscode',
{ overwrite: true }
);
expect(result).toBe(targetProjectDir);
});
it('should clone the template project with https', async () => {
const targetProjectDir = '/test/target-project';
const finalTargetProjectDir = '/test/final-target-project';
it('should throw when the target directory exists and is not empty', async () => {
jest.mocked(fs.pathExists).mockResolvedValueOnce(true as never);
jest.mocked(fs.readdir).mockResolvedValueOnce(['existing-file'] as never);
jest.mocked(canAccessRepositoryUsingSshAsync).mockResolvedValue(false);
jest.mocked(runGitCloneAsync).mockResolvedValue({
targetProjectDir: finalTargetProjectDir,
});
await expect(downloadTemplateAsync('/test/target-project', template)).rejects.toThrow(
'already exists and is not empty'
);
});
const result = await cloneTemplateAsync(targetProjectDir);
it('should not remove a pre-existing directory on failure', async () => {
jest.mocked(fs.pathExists).mockResolvedValueOnce(true as never);
jest.mocked(fs.readdir).mockResolvedValueOnce(['existing-file'] as never);
expect(runGitCloneAsync).toHaveBeenCalledWith({
githubUsername: 'expo',
githubRepositoryName: 'expo-template-default',
targetProjectDir,
cloneMethod: 'https',
showOutput: false,
});
await expect(downloadTemplateAsync('/test/target-project', template)).rejects.toThrow();
expect(fs.remove).not.toHaveBeenCalled();
});
expect(result).toBe(finalTargetProjectDir);
it('should clean up a partially extracted directory on failure', async () => {
jest.mocked(fs.pathExists).mockResolvedValueOnce(false as never);
jest.mocked(fetch).mockRejectedValueOnce(new Error('network error'));
await expect(downloadTemplateAsync('/test/target-project', template)).rejects.toThrow(
'network error'
);
expect(fs.remove).toHaveBeenCalledWith('/test/target-project');
});
});
@@ -9,6 +9,7 @@ import {
copyProjectTemplatesAsync,
generateAppConfigAsync,
generateEasConfigAsync,
removeTemplateFilesAsync,
updatePackageJsonAsync,
updateReadmeAsync,
} from '../projectFiles';
@@ -239,17 +240,24 @@ describe('projectFiles', () => {
});
describe('updatePackageJsonAsync', () => {
it('should update the package.json', async () => {
it('should replace template metadata and add workflow scripts', async () => {
jest.mocked(fs.readJson).mockResolvedValue({
name: 'test-app',
version: '1.0.0',
name: 'expo-template-default',
version: '57.0.19',
license: '0BSD',
description: 'A starter template',
repository: 'https://github.com/expo/expo-template-default',
tags: ['expo'],
main: 'expo-router/entry',
});
const projectDir = '/test/project-dir';
await updatePackageJsonAsync(projectDir);
await updatePackageJsonAsync(projectDir, 'My App');
const expectedPackageJson = {
name: 'test-app',
name: 'myapp',
version: '1.0.0',
private: true,
main: 'expo-router/entry',
scripts: {
draft: 'npx eas-cli@latest workflow:run create-draft.yml',
'development-builds': 'npx eas-cli@latest workflow:run create-development-builds.yml',
@@ -261,6 +269,28 @@ describe('projectFiles', () => {
spaces: 2,
});
});
it('should keep a non-template license', async () => {
jest.mocked(fs.readJson).mockResolvedValue({
name: 'some-template',
license: 'MIT',
});
await updatePackageJsonAsync('/test/project-dir', 'my-app');
expect(fs.writeJson).toHaveBeenCalledWith(
'/test/project-dir/package.json',
expect.objectContaining({ license: 'MIT' }),
{ spaces: 2 }
);
});
});
describe('removeTemplateFilesAsync', () => {
it('should remove the template LICENSE file', async () => {
await removeTemplateFilesAsync('/test/project-dir');
expect(fs.remove).toHaveBeenCalledWith('/test/project-dir/LICENSE');
});
});
describe('copyProjectTemplatesAsync', () => {
@@ -0,0 +1,219 @@
import fetch from '../../../fetch';
import { promptAsync } from '../../../prompts';
import { fetchTemplatePackumentAsync } from '../commands';
import {
fetchSdkVersionsAsync,
promptForSdkVersionAsync,
resolveTemplateAsync,
} from '../sdkVersion';
jest.mock('../../../fetch');
jest.mock('../../../prompts');
jest.mock('../../../api', () => ({
getExpoApiBaseUrl: () => 'https://api.expo.dev',
}));
jest.mock('../commands');
const packument = {
'dist-tags': { latest: '57.0.19', 'sdk-57': '57.0.19', 'sdk-56': '56.0.34' },
versions: {
'57.0.19': {
dist: { tarball: 'https://registry.npmjs.org/expo-template-default/-/57.0.19.tgz' },
},
'56.0.34': {
dist: { tarball: 'https://registry.npmjs.org/expo-template-default/-/56.0.34.tgz' },
},
},
};
function mockVersionsEndpoint(json: object): void {
jest.mocked(fetch).mockResolvedValueOnce({ json: async () => json } as any);
}
beforeEach(() => {
jest.clearAllMocks();
jest.mocked(fetchTemplatePackumentAsync).mockResolvedValue(packument);
});
describe('fetchSdkVersionsAsync', () => {
it('returns released SDK versions sorted descending', async () => {
mockVersionsEndpoint({
sdkVersions: {
'55.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-55' },
'57.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-57' },
'56.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-56' },
'58.0.0': {}, // canary, no release notes
'54.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-54', isDeprecated: true },
},
expoGoSdkVersion: '56.0.0',
});
const versions = await fetchSdkVersionsAsync();
expect(versions).toEqual({
latest: 57,
expoGoCompatible: 56,
available: [57, 56, 55],
});
});
it('returns null when the endpoint is unreachable', async () => {
jest.mocked(fetch).mockRejectedValueOnce(new Error('network error'));
expect(await fetchSdkVersionsAsync()).toBeNull();
});
it('returns null when no released versions are available', async () => {
mockVersionsEndpoint({ sdkVersions: {} });
expect(await fetchSdkVersionsAsync()).toBeNull();
});
});
describe('promptForSdkVersionAsync', () => {
it('offers latest and Expo Go compatible choices', async () => {
jest.mocked(promptAsync).mockResolvedValueOnce({ answer: 56 });
const result = await promptForSdkVersionAsync({
latest: 57,
expoGoCompatible: 56,
available: [57, 56, 55],
});
expect(result).toBe(56);
expect(promptAsync).toHaveBeenCalledWith(
expect.objectContaining({
message: 'Select an Expo SDK version:',
choices: [
expect.objectContaining({ title: 'Latest (SDK 57)', value: 57 }),
expect.objectContaining({
title: 'For learning with Expo Go (SDK 56)',
value: 56,
}),
expect.objectContaining({ title: 'Other SDK version…', value: 'other' }),
],
})
);
});
it('hides the Expo Go choice when it matches the latest SDK or is unavailable', async () => {
jest.mocked(promptAsync).mockResolvedValue({ answer: 57 });
await promptForSdkVersionAsync({ latest: 57, expoGoCompatible: 57, available: [57, 56] });
await promptForSdkVersionAsync({ latest: 57, expoGoCompatible: 54, available: [57, 56] });
for (const call of jest.mocked(promptAsync).mock.calls) {
expect(call[0]).toEqual(
expect.objectContaining({
choices: [
expect.objectContaining({ title: 'Latest (SDK 57)' }),
expect.objectContaining({ title: 'Other SDK version…' }),
],
})
);
}
});
it('hides the "Other" choice when no other versions are available', async () => {
jest.mocked(promptAsync).mockResolvedValueOnce({ answer: 57 });
await promptForSdkVersionAsync({ latest: 57, expoGoCompatible: null, available: [57] });
expect(promptAsync).toHaveBeenCalledWith(
expect.objectContaining({
choices: [expect.objectContaining({ title: 'Latest (SDK 57)' })],
})
);
});
it('offers the remaining versions when "other" is selected', async () => {
jest
.mocked(promptAsync)
.mockResolvedValueOnce({ answer: 'other' })
.mockResolvedValueOnce({ sdkVersion: 53 });
const result = await promptForSdkVersionAsync({
latest: 57,
expoGoCompatible: 56,
available: [57, 56, 55, 54, 53],
});
expect(result).toBe(53);
expect(promptAsync).toHaveBeenLastCalledWith(
expect.objectContaining({
message: 'Select an SDK version:',
choices: [
{ title: 'SDK 55', value: 55 },
{ title: 'SDK 54', value: 54 },
{ title: 'SDK 53', value: 53 },
],
})
);
});
});
describe('resolveTemplateAsync', () => {
it('uses the provided SDK version without prompting', async () => {
expect(await resolveTemplateAsync({ sdkVersion: '56' })).toEqual({
npmTag: 'sdk-56',
version: '56.0.34',
tarballUrl: 'https://registry.npmjs.org/expo-template-default/-/56.0.34.tgz',
});
expect((await resolveTemplateAsync({ sdkVersion: 'sdk-57' })).npmTag).toBe('sdk-57');
expect((await resolveTemplateAsync({ sdkVersion: 'latest' })).npmTag).toBe('latest');
expect(promptAsync).not.toHaveBeenCalled();
expect(fetch).not.toHaveBeenCalled();
});
it('throws for an invalid SDK version', async () => {
await expect(resolveTemplateAsync({ sdkVersion: 'abc' })).rejects.toThrow(
'Invalid SDK version'
);
});
it('throws for an SDK version without a published template', async () => {
await expect(resolveTemplateAsync({ sdkVersion: '50' })).rejects.toThrow(
'A project template for "sdk-50" is not available. Supported SDK versions: 57, 56.'
);
});
it('prompts with only the SDK versions that have a published template', async () => {
mockVersionsEndpoint({
sdkVersions: {
'57.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-57' },
'56.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-56' },
'55.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-55' },
},
});
jest.mocked(promptAsync).mockResolvedValueOnce({ answer: 56 });
const template = await resolveTemplateAsync({});
expect(template.npmTag).toBe('sdk-56');
// SDK 55 has no dist-tag in the packument, so it must not be offered.
expect(promptAsync).toHaveBeenCalledWith(
expect.objectContaining({
choices: [
expect.objectContaining({ title: 'Latest (SDK 57)' }),
expect.objectContaining({ title: 'Other SDK version…' }),
],
})
);
});
it('falls back to the latest tag when the versions endpoint is unavailable', async () => {
jest.mocked(fetch).mockRejectedValueOnce(new Error('network error'));
expect((await resolveTemplateAsync({})).npmTag).toBe('latest');
expect(promptAsync).not.toHaveBeenCalled();
});
it('falls back to the latest tag when no listed version has a template', async () => {
mockVersionsEndpoint({
sdkVersions: { '99.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-99' } },
});
expect((await resolveTemplateAsync({})).npmTag).toBe('latest');
expect(promptAsync).not.toHaveBeenCalled();
});
});
@@ -1,40 +1,106 @@
import chalk from 'chalk';
import glob from 'fast-glob';
import fs from 'fs-extra';
import path from 'path';
import { pipeline } from 'stream/promises';
import { extract } from 'tar';
import { printDirectory } from './utils';
import { canAccessRepositoryUsingSshAsync, runGitCloneAsync } from '../../onboarding/git';
import fetch from '../../fetch';
import { PackageManager, installDependenciesAsync } from '../../onboarding/installDependencies';
import { runCommandAsync } from '../../onboarding/runCommand';
import { ora } from '../../ora';
import { expoCommandAsync } from '../../utils/expoCli';
export async function cloneTemplateAsync(targetProjectDir: string): Promise<string> {
const githubUsername = 'expo';
const githubRepositoryName = 'expo-template-default';
export const TEMPLATE_PACKAGE_NAME = 'expo-template-default';
const NPM_REGISTRY_URL =
process.env.npm_config_registry?.replace(/\/$/, '') ?? 'https://registry.npmjs.org';
export interface TemplateInfo {
npmTag: string;
version: string;
tarballUrl: string;
}
export interface TemplatePackument {
'dist-tags'?: Record<string, string>;
versions?: Record<string, { dist: { tarball: string } }>;
}
export async function fetchTemplatePackumentAsync(): Promise<TemplatePackument> {
const response = await fetch(`${NPM_REGISTRY_URL}/${TEMPLATE_PACKAGE_NAME}`, {
headers: { accept: 'application/vnd.npm.install-v1+json' },
signal: AbortSignal.timeout(10000),
});
return (await response.json()) as TemplatePackument;
}
export async function downloadTemplateAsync(
targetProjectDir: string,
template: TemplateInfo
): Promise<string> {
const spinner = ora(
`${chalk.bold(`Cloning the project to ${printDirectory(targetProjectDir)}`)}`
`${chalk.bold(`Downloading the project template to ${printDirectory(targetProjectDir)}`)}`
).start();
const cloneMethod = (await canAccessRepositoryUsingSshAsync({
githubUsername,
githubRepositoryName,
}))
? 'ssh'
: 'https';
const targetExisted = await fs.pathExists(targetProjectDir);
try {
// The directory was verified to not exist earlier, but it may have been
// created while the user was answering prompts.
if (targetExisted && (await fs.readdir(targetProjectDir)).length > 0) {
throw new Error(
`Directory ${printDirectory(targetProjectDir)} already exists and is not empty.`
);
}
const { targetProjectDir: finalTargetProjectDirectory } = await runGitCloneAsync({
githubUsername,
githubRepositoryName,
targetProjectDir,
cloneMethod,
showOutput: false,
await fs.mkdirp(targetProjectDir);
const tarballResponse = await fetch(template.tarballUrl);
await pipeline(tarballResponse.body, extract({ cwd: targetProjectDir, strip: 1 }));
await restoreTemplateDotfilesAsync(targetProjectDir);
spinner.succeed(
`Downloaded ${chalk.bold(`${TEMPLATE_PACKAGE_NAME}@${template.version}`)} to ${printDirectory(
targetProjectDir
)}`
);
} catch (error) {
spinner.fail();
if (!targetExisted) {
await fs.remove(targetProjectDir);
}
throw error;
}
return targetProjectDir;
}
/**
* npm strips `.gitignore` files from published packages, and the template ships
* dot-directories with an underscore prefix (e.g. `_vscode`), so restore the
* real names after extraction. This mirrors the renames in create-expo-app;
* keep the directory list in sync with SUPPORTED_DIRECTORIES in
* https://github.com/expo/expo/blob/main/packages/create-expo-app/src/createFileTransform.ts
*/
async function restoreTemplateDotfilesAsync(projectDir: string): Promise<void> {
const entries = await glob('**/{gitignore,_eas,_vscode,_github,_cursor}', {
cwd: projectDir,
onlyFiles: false,
dot: true,
});
spinner.succeed(`Cloned the project to ${printDirectory(finalTargetProjectDirectory)}`);
// Rename deeper entries first so parent renames do not invalidate child paths.
entries.sort((a, b) => b.split('/').length - a.split('/').length);
return finalTargetProjectDirectory;
for (const entry of entries) {
const basename = path.basename(entry);
const dotName = basename === 'gitignore' ? '.gitignore' : `.${basename.slice(1)}`;
await fs.move(
path.join(projectDir, entry),
path.join(projectDir, path.dirname(entry), dotName),
{ overwrite: true }
);
}
}
export async function installProjectDependenciesAsync(
@@ -110,10 +110,37 @@ export async function generateEasConfigAsync(projectDir: string): Promise<void>
await fs.writeJson(easJsonPath, easJson, { spaces: 2 });
}
export async function updatePackageJsonAsync(projectDir: string): Promise<void> {
const TEMPLATE_LICENSE = '0BSD';
// https://github.com/npm/validate-npm-package-name#naming-rules
function sanitizeNpmPackageName(name: string): string {
return (
name
.normalize('NFD')
.replace(/[\u0300-\u036f]/g, '')
.toLowerCase()
.replace(/^[._]+/, '')
.replace(/[^a-z0-9._\-/@]/g, '') || 'app'
);
}
export async function updatePackageJsonAsync(
projectDir: string,
projectName: string
): Promise<void> {
const packageJsonPath = path.join(projectDir, 'package.json');
const packageJson = await fs.readJson(packageJsonPath);
packageJson.name = sanitizeNpmPackageName(projectName);
packageJson.version = '1.0.0';
packageJson.private = true;
delete packageJson.description;
delete packageJson.tags;
delete packageJson.repository;
if (packageJson.license === TEMPLATE_LICENSE) {
delete packageJson.license;
}
if (!packageJson.scripts) {
packageJson.scripts = {};
}
@@ -125,6 +152,10 @@ export async function updatePackageJsonAsync(projectDir: string): Promise<void>
await fs.writeJson(packageJsonPath, packageJson, { spaces: 2 });
}
export async function removeTemplateFilesAsync(projectDir: string): Promise<void> {
await fs.remove(path.join(projectDir, 'LICENSE'));
}
export async function copyProjectTemplatesAsync(projectDir: string): Promise<void> {
const templatesSourceDir = path.join(__dirname, 'templates');
@@ -0,0 +1,181 @@
import { TemplateInfo, fetchTemplatePackumentAsync } from './commands';
import { getExpoApiBaseUrl } from '../../api';
import fetch from '../../fetch';
import Log from '../../log';
import { promptAsync } from '../../prompts';
export interface SdkVersions {
latest: number;
expoGoCompatible: number | null;
available: number[];
}
interface VersionsEndpointResponse {
sdkVersions?: Record<string, { releaseNoteUrl?: string; isDeprecated?: boolean }>;
expoGoSdkVersion?: string;
}
/**
* SDKs without a `releaseNoteUrl` are canary/in-development and are excluded.
*/
export async function fetchSdkVersionsAsync(): Promise<SdkVersions | null> {
let json: VersionsEndpointResponse;
try {
const response = await fetch(`${getExpoApiBaseUrl()}/v2/versions`, {
signal: AbortSignal.timeout(5000),
});
json = (await response.json()) as VersionsEndpointResponse;
} catch (error) {
Log.debug(`Failed to fetch SDK versions: ${error}`);
return null;
}
const available = Object.entries(json.sdkVersions ?? {})
.filter(([, info]) => !!info.releaseNoteUrl && !info.isDeprecated)
.map(([version]) => parseSdkMajor(version))
.filter((major): major is number => major !== null)
.sort((a, b) => b - a);
const latest = available[0];
if (latest == null) {
return null;
}
return {
latest,
expoGoCompatible: parseSdkMajor(json.expoGoSdkVersion),
available,
};
}
function parseSdkMajor(version: string | undefined): number | null {
const major = parseInt(version?.split('.')[0] ?? '', 10);
return Number.isFinite(major) ? major : null;
}
export async function promptForSdkVersionAsync({
latest,
expoGoCompatible,
available,
}: SdkVersions): Promise<number> {
const choices: { title: string; value: number | 'other'; description?: string }[] = [
{
title: `Latest (SDK ${latest})`,
value: latest,
description: 'Recommended for most projects',
},
];
if (
expoGoCompatible !== null &&
expoGoCompatible !== latest &&
available.includes(expoGoCompatible)
) {
choices.push({
title: `For learning with Expo Go (SDK ${expoGoCompatible})`,
value: expoGoCompatible,
description: 'Compatible with Expo Go on App Store and Play Store',
});
}
const otherVersions = available.filter(sdk => sdk !== latest && sdk !== expoGoCompatible);
if (otherVersions.length > 0) {
choices.push({ title: 'Other SDK version…', value: 'other' });
}
const { answer } = await promptAsync({
type: 'select',
name: 'answer',
message: 'Select an Expo SDK version:',
choices,
});
if (answer !== 'other') {
return answer;
}
const { sdkVersion } = await promptAsync({
type: 'select',
name: 'sdkVersion',
message: 'Select an SDK version:',
choices: otherVersions.map(sdk => ({
title: `SDK ${sdk}`,
value: sdk,
})),
});
return sdkVersion;
}
function resolveTemplateForTag(
distTags: Record<string, string>,
versions: Record<string, { dist: { tarball: string } }>,
npmTag: string
): TemplateInfo {
const version = distTags[npmTag];
const tarballUrl = version ? versions[version]?.dist.tarball : undefined;
if (!version || !tarballUrl) {
const supported = Object.keys(distTags)
.filter(tag => /^sdk-\d+$/.test(tag))
.sort((a, b) => parseInt(b.slice(4), 10) - parseInt(a.slice(4), 10));
throw new Error(
`A project template for "${npmTag}" is not available. Supported SDK versions: ${supported
.map(tag => tag.slice(4))
.join(', ')}.`
);
}
return { npmTag, version, tarballUrl };
}
/**
* `sdkVersion` accepts "57", "sdk-57", or "latest".
*/
export async function resolveTemplateAsync({
sdkVersion,
}: {
sdkVersion?: string;
}): Promise<TemplateInfo> {
const packument = await fetchTemplatePackumentAsync();
const templateDistTags = packument['dist-tags'] ?? {};
const packumentVersions = packument.versions ?? {};
if (sdkVersion) {
if (sdkVersion === 'latest') {
return resolveTemplateForTag(templateDistTags, packumentVersions, 'latest');
}
const major = parseSdkMajor(sdkVersion.replace(/^sdk-/, ''));
if (major === null) {
throw new Error(
`Invalid SDK version: "${sdkVersion}". Specify a version number (e.g. 57) or "latest".`
);
}
return resolveTemplateForTag(templateDistTags, packumentVersions, `sdk-${major}`);
}
const versions = await fetchSdkVersionsAsync();
const availableWithTemplates =
versions?.available.filter(sdk => `sdk-${sdk}` in templateDistTags) ?? [];
if (!versions || availableWithTemplates.length === 0) {
if (!versions) {
Log.warn('Could not fetch the list of Expo SDK versions. Using the latest SDK.');
}
return resolveTemplateForTag(templateDistTags, packumentVersions, 'latest');
}
const latest = availableWithTemplates[0];
// Match the prompt guard in prompts.ts: pin to the latest released SDK
// instead of prompting when stdin is not interactive.
if (!process.stdin.isTTY && !global.test) {
Log.withInfo(`Using Expo SDK ${latest}`);
return resolveTemplateForTag(templateDistTags, packumentVersions, `sdk-${latest}`);
}
const selectedSdk = await promptForSdkVersionAsync({
latest,
expoGoCompatible: versions.expoGoCompatible,
available: availableWithTemplates,
});
return resolveTemplateForTag(templateDistTags, packumentVersions, `sdk-${selectedSdk}`);
}
@@ -1,35 +1,137 @@
import { getMockWorkflowRunWithJobsFragment } from '../../../__tests__/commands/utils';
import { fetchRawLogsForCustomJobAsync } from '../fetchLogs';
import { infoForActiveWorkflowRunAsync } from '../utils';
import { WorkflowJobStatus } from '../../../graphql/generated';
import { groupLogLinesIntoSteps, parseLogLines } from '../logs/parseLogs';
import { WorkflowLogs, WorkflowRawLogLine } from '../types';
import { formatActiveWorkflowRun, formatFailedWorkflowRun } from '../utils';
jest.mock('../fetchLogs');
function jobWithLogs(
logLines: WorkflowRawLogLine[],
status: WorkflowJobStatus = WorkflowJobStatus.InProgress
): {
job: ReturnType<typeof getMockWorkflowRunWithJobsFragment>['jobs'][number];
logs: WorkflowLogs;
} {
return {
job: { ...getMockWorkflowRunWithJobsFragment().jobs[0], status },
logs: groupLogLinesIntoSteps(logLines),
};
}
describe('workflow utils', () => {
afterEach(() => {
jest.clearAllMocks();
});
function stepLines(buildStepId: string, count: number): WorkflowRawLogLine[] {
return Array.from({ length: count }, (_, index) => ({ buildStepId, msg: `line${index}` }));
}
test('shows the display name for the current step while keying logs by step id', async () => {
const workflowRun = getMockWorkflowRunWithJobsFragment();
workflowRun.jobs = workflowRun.jobs.map(job => ({
...job,
status: WorkflowJobStatus.InProgress,
}));
jest
.mocked(fetchRawLogsForCustomJobAsync)
.mockResolvedValue(
[
'{"buildStepId":"step-id-1","buildStepDisplayName":"Install dependencies","time":"2022-01-01T00:00:00.000Z","msg":"npm ci"}',
'{"buildStepId":"step-id-1","buildStepDisplayName":"Install dependencies","marker":"end-step","result":"success","time":"2022-01-01T00:00:01.000Z","msg":"done"}',
].join('\n')
);
const output = await infoForActiveWorkflowRunAsync({} as any, workflowRun);
describe(formatActiveWorkflowRun, () => {
test('shows the display name for the current step while keying logs by step id', () => {
const output = formatActiveWorkflowRun([
jobWithLogs(
parseLogLines(
[
'{"buildStepId":"step-id-1","buildStepDisplayName":"Install dependencies","time":"2022-01-01T00:00:00.000Z","msg":"npm ci"}',
'{"buildStepId":"step-id-1","buildStepDisplayName":"Install dependencies","marker":"end-step","result":"success","time":"2022-01-01T00:00:01.000Z","msg":"done"}',
].join('\n')
).logLines
),
]);
expect(output).toContain('Current step');
expect(output).toContain('Install dependencies');
expect(output).not.toContain('step-id-1');
});
test('shows exactly maxLogLines trailing lines of the current step', () => {
const output = formatActiveWorkflowRun([jobWithLogs(stepLines('step-id-1', 10))], 5);
expect(output).toContain('line5');
expect(output).toContain('line9');
expect(output).not.toContain('line4');
});
test('keeps five trailing lines by default', () => {
const output = formatActiveWorkflowRun([jobWithLogs(stepLines('step-id-1', 10))]);
expect(output).toContain('line5');
expect(output).toContain('line9');
expect(output).not.toContain('line4');
});
test('reports the last step as the current one', () => {
const output = formatActiveWorkflowRun([
jobWithLogs([
{ buildStepId: 'install', buildStepDisplayName: 'Install', msg: 'installing' },
{ buildStepId: 'build', buildStepDisplayName: 'Build', msg: 'compiling' },
]),
]);
expect(output).toContain('Build');
expect(output).not.toContain('Install');
});
test('passes over a trailing skipped step', () => {
const output = formatActiveWorkflowRun([
jobWithLogs([
{ buildStepId: 'install', buildStepDisplayName: 'Install', msg: 'installing' },
{
buildStepId: 'build',
buildStepDisplayName: 'Build',
msg: 'skipping',
marker: 'end-step',
result: 'skipped',
},
]),
]);
expect(output).toContain('Install');
expect(output).not.toContain('Build');
});
test('reports a step that has not ended yet', () => {
const output = formatActiveWorkflowRun([
jobWithLogs([
{
buildStepId: 'install',
buildStepDisplayName: 'Install',
msg: 'installed',
marker: 'end-step',
result: 'success',
},
{ buildStepId: 'build', buildStepDisplayName: 'Build', msg: 'compiling' },
]),
]);
expect(output).toContain('Build');
});
test('omits the current step when the job has no steps yet', () => {
const output = formatActiveWorkflowRun([jobWithLogs([])]);
expect(output).not.toContain('Current step');
expect(output).not.toContain('Current logs');
});
test('omits the current step for a job that is not in progress', () => {
const output = formatActiveWorkflowRun([
jobWithLogs(stepLines('step-id-1', 3), WorkflowJobStatus.Success),
]);
expect(output).not.toContain('Current step');
expect(output).not.toContain('line0');
});
});
describe(formatFailedWorkflowRun, () => {
test('prints every line of the failed step when there is no limit', () => {
const output = formatFailedWorkflowRun([
jobWithLogs(
[
...stepLines('build', 8),
{ buildStepId: 'build', msg: 'it failed', marker: 'end-step', result: 'fail' },
],
WorkflowJobStatus.Failure
),
]);
expect(output).toContain('line0');
expect(output).toContain('line7');
});
});
@@ -1,51 +0,0 @@
import { WorkflowJobResult } from './types';
import { BuildQuery } from '../../graphql/queries/BuildQuery';
import { ExpoGraphqlClient } from '../context/contextUtils/createGraphqlClient';
// This function is in a separate module for testing purposes
export async function fetchRawLogsForCustomJobAsync(
job: WorkflowJobResult
): Promise<string | null> {
const firstLogFileUrl = job.turtleJobRun?.logFileUrls?.[0];
if (!firstLogFileUrl) {
return null;
}
const response = await fetch(firstLogFileUrl, {
method: 'GET',
});
const rawLogs = await response.text();
return rawLogs;
}
export async function fetchRawLogsForBuildJobAsync(
state: { graphqlClient: ExpoGraphqlClient },
job: WorkflowJobResult
): Promise<string | null> {
// Prefer turtleJobRun logs, which contain JSONL-formatted step logs
// for both built-in and custom build steps
const turtleLogFileUrl = job.turtleJobRun?.logFileUrls?.[0];
if (turtleLogFileUrl) {
const response = await fetch(turtleLogFileUrl, {
method: 'GET',
});
return await response.text();
}
// Fall back to build logFiles
const buildId = job.outputs?.build_id;
if (!buildId) {
return null;
}
const buildFragment = await BuildQuery.byIdAsync(state.graphqlClient, buildId, {
useCache: false,
});
const firstLogFileUrl = buildFragment.logFiles?.[0];
if (!firstLogFileUrl) {
return null;
}
const response = await fetch(firstLogFileUrl, {
method: 'GET',
});
return await response.text();
}
@@ -0,0 +1,27 @@
import { getMockWorkflowCustomJobFragment } from '../../../../__tests__/commands/utils';
import { BuildQuery } from '../../../../graphql/queries/BuildQuery';
import { ExpoGraphqlClient } from '../../../context/contextUtils/createGraphqlClient';
import { fetchRawLogsForJobAsync } from '../fetchLogs';
jest.mock('../../../../graphql/queries/BuildQuery');
describe(fetchRawLogsForJobAsync, () => {
afterEach(() => {
jest.restoreAllMocks();
});
it('reads a custom job from its turtle job run log file', async () => {
const rawLogs = '{"logId":"1","buildStepId":"install","msg":"from the turtle job run"}';
const fetchSpy = jest.spyOn(globalThis, 'fetch').mockResolvedValue(new Response(rawLogs));
await expect(
fetchRawLogsForJobAsync(
{ graphqlClient: {} as ExpoGraphqlClient },
getMockWorkflowCustomJobFragment()
)
).resolves.toBe(rawLogs);
expect(fetchSpy).toHaveBeenCalledWith('https://example.com/log1');
expect(BuildQuery.byIdAsync).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,186 @@
import { groupLogLinesIntoSteps, mergeLogLines, parseLogLines } from '../parseLogs';
import { WorkflowRawLogLine } from '../../types';
function logLine(overrides: Partial<WorkflowRawLogLine> = {}): WorkflowRawLogLine {
return { msg: 'a message', ...overrides };
}
describe(parseLogLines, () => {
it('parses a JSONL log file', () => {
const { logLines, errors } = parseLogLines(
[
'{"logId":"1","buildStepId":"install","msg":"npm ci"}',
'{"logId":"2","buildStepId":"install","msg":"done"}',
].join('\n')
);
expect(errors).toEqual([]);
expect(logLines).toEqual([
{ logId: '1', buildStepId: 'install', msg: 'npm ci' },
{ logId: '2', buildStepId: 'install', msg: 'done' },
]);
});
it('skips blank lines, including a trailing newline', () => {
const { logLines, errors } = parseLogLines('{"logId":"1","msg":"one"}\n\n');
expect(errors).toEqual([]);
expect(logLines).toHaveLength(1);
});
it('reports malformed lines as errors while keeping the parsable ones', () => {
const { logLines, errors } = parseLogLines(
['{"logId":"1","msg":"one"}', 'this is not json', '{"logId":"2","msg":"two"}'].join('\n')
);
expect(logLines.map(line => line.msg)).toEqual(['one', 'two']);
expect(errors).toHaveLength(1);
expect(errors[0]).toBeInstanceOf(Error);
});
it('parses a line that carries no message', () => {
const { logLines, errors } = parseLogLines('{"logId":"1"}');
expect(errors).toEqual([]);
expect(logLines).toEqual([{ logId: '1' }]);
});
});
describe(mergeLogLines, () => {
it('keeps the rightmost line for a repeated logId', () => {
const fileLogLines = [
logLine({ logId: '1', msg: 'from the file' }),
logLine({ logId: '2', msg: 'from the file' }),
];
const realtimeLogLines = [
logLine({ logId: '2', msg: 'from realtime' }),
logLine({ logId: '3', msg: 'from realtime' }),
];
expect(mergeLogLines(fileLogLines, realtimeLogLines)).toEqual([
{ logId: '1', msg: 'from the file' },
{ logId: '2', msg: 'from realtime' },
{ logId: '3', msg: 'from realtime' },
]);
});
it('keeps identical messages that have different logIds', () => {
const merged = mergeLogLines(
[logLine({ logId: '1', msg: 'Repeated text' })],
[logLine({ logId: '2', msg: 'Repeated text' })]
);
expect(merged).toEqual([
{ logId: '1', msg: 'Repeated text' },
{ logId: '2', msg: 'Repeated text' },
]);
});
it('keeps identical lines without logId', () => {
const merged = mergeLogLines(
[logLine({ msg: 'same' }), logLine({ msg: 'same' })],
[logLine({ msg: 'same' })]
);
expect(merged).toHaveLength(3);
});
it('deduplicates within a single group', () => {
const merged = mergeLogLines([logLine({ logId: '1' }), logLine({ logId: '1' })]);
expect(merged).toHaveLength(1);
});
});
describe(groupLogLinesIntoSteps, () => {
it('groups lines by step id and labels the step with its display name', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepId: 'step-id-1', msg: 'npm ci' }),
logLine({
buildStepId: 'step-id-1',
buildStepDisplayName: 'Install dependencies',
msg: 'ok',
}),
]);
expect(Array.from(logs.keys())).toEqual(['step-id-1']);
expect(logs.get('step-id-1')).toEqual({
key: 'step-id-1',
label: 'Install dependencies',
logLines: [
{ time: undefined, msg: 'npm ci', result: undefined, marker: undefined, err: undefined },
{ time: undefined, msg: 'ok', result: undefined, marker: undefined, err: undefined },
],
});
});
it('keeps steps in the order they first appear', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepId: 'first' }),
logLine({ buildStepId: 'second' }),
logLine({ buildStepId: 'first' }),
logLine({ buildStepId: 'third' }),
]);
expect(Array.from(logs.keys())).toEqual(['first', 'second', 'third']);
});
it('falls back to the display name, then the phase, when there is no step id', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepDisplayName: 'Run fastlane' }),
logLine({ phase: 'PREPARE_CREDENTIALS' }),
]);
expect(Array.from(logs.keys())).toEqual(['Run fastlane', 'PREPARE_CREDENTIALS']);
expect(logs.get('PREPARE_CREDENTIALS')?.label).toBe('PREPARE_CREDENTIALS');
});
it('drops lines that belong to no step', () => {
const logs = groupLogLinesIntoSteps([logLine({ msg: 'a line with no step' })]);
expect(logs.size).toBe(0);
});
it('drops lines that carry no message, keeping the rest of the step', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepId: 'install', msg: undefined, marker: 'start-step' }),
logLine({ buildStepId: 'install', msg: 'npm ci' }),
]);
expect(logs.get('install')?.logLines).toEqual([
{ time: undefined, msg: 'npm ci', result: undefined, marker: undefined, err: undefined },
]);
});
it('creates no step for a line that carries no message', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepId: 'install', msg: undefined, marker: 'end-step', result: 'success' }),
]);
expect(logs.size).toBe(0);
});
it('records the result of the first end marker and ignores a later one', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepId: 'install', msg: 'skipping', marker: 'end-step', result: 'skipped' }),
logLine({ buildStepId: 'install', msg: 'retried', marker: 'end-step', result: 'success' }),
]);
expect(logs.get('install')?.result).toBe('skipped');
});
it('leaves the result undefined for a step that has not ended', () => {
const logs = groupLogLinesIntoSteps([logLine({ buildStepId: 'install', msg: 'npm ci' })]);
expect(logs.get('install')?.result).toBeUndefined();
});
it('records an end marker that carries no result and does not let a later one replace it', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepId: 'install', msg: 'ended', marker: 'END_PHASE' }),
logLine({ buildStepId: 'install', msg: 'retried', marker: 'END_PHASE', result: 'success' }),
]);
expect(logs.get('install')?.result).toBe('');
});
});
@@ -0,0 +1,308 @@
import { getMockWorkflowRunWithJobsFragment } from '../../../../__tests__/commands/utils';
import { RealtimeLogsTargetType, WorkflowJobStatus } from '../../../../graphql/generated';
import { RealtimeLogsClient } from '../../../../utils/centrifuge';
import { fetchAndParseLogsFromJobAsync } from '../parseLogs';
import { WorkflowJobLogsState, WorkflowRunLogsWatcher, realtimeLogsTargetForJob } from '../watcher';
import { WorkflowJobResult, WorkflowRawLogLine } from '../../types';
jest.mock('../parseLogs', () => ({
...jest.requireActual('../parseLogs'),
fetchAndParseLogsFromJobAsync: jest.fn(),
}));
function createFakeClient(): {
client: RealtimeLogsClient;
publish: (data: unknown) => void;
subscribeCalls: unknown[];
closeCount: () => number;
} {
const listeners: ((data: unknown) => void)[] = [];
const subscribeCalls: unknown[] = [];
let closed = 0;
return {
subscribeCalls,
closeCount: () => closed,
publish: data => {
listeners.forEach(listener => {
listener(data);
});
},
client: {
subscribeAsync: async (args, onPublication) => {
subscribeCalls.push(args);
listeners.push(onPublication);
return { close: () => closed++ };
},
close: () => closed++,
},
};
}
function inProgressJob(): WorkflowJobResult {
const job = getMockWorkflowRunWithJobsFragment().jobs[0];
return { ...job, status: WorkflowJobStatus.InProgress };
}
describe(realtimeLogsTargetForJob, () => {
it('targets the turtle job run when there is one', () => {
const job = { ...inProgressJob(), turtleJobRun: { id: 'job-run-id' } } as WorkflowJobResult;
expect(realtimeLogsTargetForJob(job)).toEqual({
type: RealtimeLogsTargetType.JobRun,
id: 'job-run-id',
});
});
it('falls back to the build when there is no turtle job run', () => {
const job = {
...inProgressJob(),
turtleJobRun: null,
turtleBuild: { id: 'build-id' },
} as WorkflowJobResult;
expect(realtimeLogsTargetForJob(job)).toEqual({
type: RealtimeLogsTargetType.Build,
id: 'build-id',
});
});
it('has no target when neither is present', () => {
const job = {
...inProgressJob(),
turtleJobRun: null,
turtleBuild: null,
outputs: null,
} as WorkflowJobResult;
expect(realtimeLogsTargetForJob(job)).toBeNull();
});
});
describe(WorkflowRunLogsWatcher, () => {
beforeEach(() => {
jest
.mocked(fetchAndParseLogsFromJobAsync)
.mockResolvedValue([{ logId: '1', buildStepId: 'install', msg: 'from the file' }]);
});
afterEach(() => {
jest.clearAllMocks();
});
it('subscribes once per in-progress job', async () => {
const fake = createFakeClient();
const watcher = new WorkflowRunLogsWatcher(
{} as any,
() => fake.client,
() => {}
);
const job = inProgressJob();
await watcher.syncJobsAsync([job]);
await watcher.syncJobsAsync([job]);
expect(fake.subscribeCalls).toHaveLength(1);
});
it('reports realtime logs as they arrive', async () => {
const fake = createFakeClient();
const onRealtimeLogs = jest.fn();
const watcher = new WorkflowRunLogsWatcher({} as any, () => fake.client, onRealtimeLogs);
await watcher.syncJobsAsync([inProgressJob()]);
fake.publish([{ logId: '2', buildStepId: 'install', msg: 'pushed' }]);
expect(onRealtimeLogs).toHaveBeenCalledTimes(1);
});
it('does not report a publication that carries no usable log lines', async () => {
const fake = createFakeClient();
const onRealtimeLogs = jest.fn();
const watcher = new WorkflowRunLogsWatcher({} as any, () => fake.client, onRealtimeLogs);
await watcher.syncJobsAsync([inProgressJob()]);
fake.publish([{ msg: 'no logId' }]);
expect(onRealtimeLogs).not.toHaveBeenCalled();
});
it('still fetches logs when the realtime logs client is unavailable', async () => {
const watcher = new WorkflowRunLogsWatcher(
{} as any,
() => null,
() => {}
);
await watcher.syncJobsAsync([inProgressJob()]);
expect(fetchAndParseLogsFromJobAsync).toHaveBeenCalledTimes(1);
});
it('fetches logs on every sync while in progress, and stops once the job completes', async () => {
const fake = createFakeClient();
const watcher = new WorkflowRunLogsWatcher(
{} as any,
() => fake.client,
() => {}
);
const job = inProgressJob();
await watcher.syncJobsAsync([job]);
await watcher.syncJobsAsync([job]);
expect(fetchAndParseLogsFromJobAsync).toHaveBeenCalledTimes(2);
await watcher.syncJobsAsync([{ ...job, status: WorkflowJobStatus.Success }]);
expect(fetchAndParseLogsFromJobAsync).toHaveBeenCalledTimes(2);
});
it('does not fetch logs for a job that has not started', async () => {
const fake = createFakeClient();
const watcher = new WorkflowRunLogsWatcher(
{} as any,
() => fake.client,
() => {}
);
await watcher.syncJobsAsync([{ ...inProgressJob(), status: WorkflowJobStatus.New }]);
expect(fetchAndParseLogsFromJobAsync).not.toHaveBeenCalled();
});
it('closes the subscription when a job leaves in-progress', async () => {
const fake = createFakeClient();
const watcher = new WorkflowRunLogsWatcher(
{} as any,
() => fake.client,
() => {}
);
const job = inProgressJob();
await watcher.syncJobsAsync([job]);
expect(fake.closeCount()).toBe(0);
await watcher.syncJobsAsync([{ ...job, status: WorkflowJobStatus.Failure }]);
expect(fake.closeCount()).toBe(1);
});
it('closes the client and its subscriptions', async () => {
const fake = createFakeClient();
const watcher = new WorkflowRunLogsWatcher(
{} as any,
() => fake.client,
() => {}
);
await watcher.syncJobsAsync([inProgressJob()]);
watcher.close();
expect(fake.closeCount()).toBe(2);
});
});
function fileLine(logId: string, msg: string, buildStepId = 'install'): WorkflowRawLogLine {
return { logId, buildStepId, msg };
}
function stepMessages(logsState: WorkflowJobLogsState): string[] {
return Array.from(logsState.getLogs().values()).flatMap(group =>
group.logLines.map(logLine => logLine.msg)
);
}
describe(WorkflowJobLogsState, () => {
it('hides realtime lines until the file logs reach them', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('5', 'pushed')]);
expect(stepMessages(logsState)).toEqual(['first']);
});
it('shows realtime lines once a published logId also appears in the file', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('2', 'pushed')]);
logsState.ingestFileLogLines([fileLine('1', 'first'), fileLine('2', 'pushed')]);
logsState.ingestRealtimeLogLines([fileLine('3', 'newer')]);
expect(stepMessages(logsState)).toEqual(['first', 'pushed', 'newer']);
});
it('shows realtime logs when a publication repeats a logId already in the file', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('1', 'first'), fileLine('2', 'pushed')]);
expect(stepMessages(logsState)).toEqual(['first', 'pushed']);
});
it('shows buffered realtime lines once the job is completed', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('9', 'pushed')]);
logsState.markCompleted();
logsState.markCompleted();
expect(stepMessages(logsState)).toEqual(['first', 'pushed']);
});
it('folds later publications once completion showed them', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('9', 'buffered')]);
logsState.markCompleted();
logsState.ingestRealtimeLogLines([fileLine('10', 'pushed')]);
expect(stepMessages(logsState)).toEqual(['first', 'buffered', 'pushed']);
});
it('replaces the file snapshot instead of accumulating it', () => {
const logsState = new WorkflowJobLogsState();
const keyless: WorkflowRawLogLine = { buildStepId: 'install', msg: 'submission log' };
logsState.ingestFileLogLines([keyless]);
logsState.ingestFileLogLines([keyless]);
expect(stepMessages(logsState)).toEqual(['submission log']);
});
it('drops a buffered realtime line once it lands in the file', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('2', 'pushed')]);
logsState.ingestFileLogLines([fileLine('1', 'first'), fileLine('2', 'pushed')]);
expect(stepMessages(logsState)).toEqual(['first', 'pushed']);
});
it('ignores publications that are not arrays of log lines', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
expect(logsState.ingestRealtimeLogLines({ logId: '2', msg: 'not an array' })).toBe(false);
expect(logsState.ingestRealtimeLogLines(['a string', 42, null])).toBe(false);
expect(logsState.ingestRealtimeLogLines([{ msg: 'no logId' }])).toBe(false);
expect(stepMessages(logsState)).toEqual(['first']);
});
it('deduplicates a line republished on the same channel', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('1', 'first'), fileLine('2', 'pushed')]);
logsState.ingestRealtimeLogLines([fileLine('2', 'pushed')]);
expect(stepMessages(logsState)).toEqual(['first', 'pushed']);
});
it('deduplicates a line republished later', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('1', 'first'), fileLine('2', 'pushed')]);
logsState.ingestRealtimeLogLines([fileLine('3', 'newer')]);
logsState.ingestRealtimeLogLines([fileLine('2', 'pushed')]);
expect(stepMessages(logsState)).toEqual(['first', 'pushed', 'newer']);
});
});
@@ -0,0 +1,28 @@
import { WorkflowJobResult } from '../types';
import { BuildQuery } from '../../../graphql/queries/BuildQuery';
import { ExpoGraphqlClient } from '../../context/contextUtils/createGraphqlClient';
export async function fetchRawLogsForJobAsync(
state: { graphqlClient: ExpoGraphqlClient },
job: WorkflowJobResult
): Promise<string | null> {
const turtleLogFileUrl = job.turtleJobRun?.logFileUrls?.[0];
if (turtleLogFileUrl) {
const response = await fetch(turtleLogFileUrl);
return await response.text();
}
const buildId = job.outputs?.build_id;
if (!buildId) {
return null;
}
const buildFragment = await BuildQuery.byIdAsync(state.graphqlClient, buildId, {
useCache: false,
});
const firstLogFileUrl = buildFragment.logFiles?.[0];
if (!firstLogFileUrl) {
return null;
}
const response = await fetch(firstLogFileUrl);
return await response.text();
}
@@ -0,0 +1,83 @@
import { fetchRawLogsForJobAsync } from './fetchLogs';
import { WorkflowJobResult, WorkflowLogs, WorkflowRawLogLine } from '../types';
import Log from '../../../log';
import uniqBy from '../../../utils/expodash/uniqBy';
import { ExpoGraphqlClient } from '../../context/contextUtils/createGraphqlClient';
export function parseLogLines(rawLogs: string): {
logLines: WorkflowRawLogLine[];
errors: Error[];
} {
const logLines: WorkflowRawLogLine[] = [];
const errors: Error[] = [];
for (const rawLogLine of rawLogs.split('\n')) {
if (!rawLogLine) {
continue;
}
try {
logLines.push(JSON.parse(rawLogLine));
} catch (err) {
errors.push(err as Error);
}
}
return { logLines, errors };
}
export function mergeLogLines<T extends WorkflowRawLogLine>(...logLineGroups: T[][]): T[] {
return uniqBy(logLineGroups.flat().reverse(), logLine => logLine.logId ?? Symbol()).reverse();
}
export function groupLogLinesIntoSteps(
logLines: WorkflowRawLogLine[],
accumulator: WorkflowLogs = new Map()
): WorkflowLogs {
for (const logLine of logLines) {
const { buildStepDisplayName, buildStepId, phase, time, msg, result, marker, err } = logLine;
const stepKey = buildStepId ?? buildStepDisplayName ?? phase;
const stepLabel = buildStepDisplayName ?? buildStepId ?? phase;
if (!stepKey || !stepLabel || !msg) {
continue;
}
let logGroup = accumulator.get(stepKey);
if (!logGroup) {
logGroup = { key: stepKey, label: stepLabel, logLines: [] };
accumulator.set(stepKey, logGroup);
}
if (buildStepDisplayName) {
logGroup.label = buildStepDisplayName;
}
if (logGroup.result === undefined && (marker === 'end-step' || marker === 'END_PHASE')) {
logGroup.result = result ?? '';
}
logGroup.logLines.push({ time, msg, result, marker, err });
}
return accumulator;
}
export async function fetchAndParseLogsFromJobAsync(
state: { graphqlClient: ExpoGraphqlClient },
job: WorkflowJobResult
): Promise<WorkflowRawLogLine[] | null> {
const rawLogs = await fetchRawLogsForJobAsync(state, job);
if (!rawLogs) {
return null;
}
Log.debug(`rawLogs = ${JSON.stringify(rawLogs, null, 2)}`);
const { logLines, errors } = parseLogLines(rawLogs);
for (const error of errors) {
Log.debug(`Failed to parse a log line: ${error.message}`);
}
return logLines;
}
export async function fetchAndProcessLogsFromJobAsync(
state: { graphqlClient: ExpoGraphqlClient },
job: WorkflowJobResult
): Promise<WorkflowLogs | null> {
const logLines = await fetchAndParseLogsFromJobAsync(state, job);
return logLines && groupLogLinesIntoSteps(logLines);
}
@@ -0,0 +1,210 @@
import { fetchAndParseLogsFromJobAsync, groupLogLinesIntoSteps, mergeLogLines } from './parseLogs';
import { WorkflowJobResult, WorkflowLogs, WorkflowRawLogLine } from '../types';
import {
RealtimeLogsTargetInput,
RealtimeLogsTargetType,
WorkflowJobStatus,
} from '../../../graphql/generated';
import Log from '../../../log';
import { RealtimeLogsClient, RealtimeLogsSubscription } from '../../../utils/centrifuge';
import { ExpoGraphqlClient } from '../../context/contextUtils/createGraphqlClient';
import nullthrows from 'nullthrows';
type RealtimeLogLine = WorkflowRawLogLine & { logId: string };
function isRealtimeLogLine(entry: unknown): entry is RealtimeLogLine {
return (
typeof entry === 'object' &&
entry !== null &&
typeof (entry as Partial<WorkflowRawLogLine>).logId === 'string'
);
}
export class WorkflowJobLogsState {
private fileLogIds = new Set<string>();
private realtimeLogLines: RealtimeLogLine[] = [];
private realtimeLogIds = new Set<string>();
private haveFileLogsCaughtUp = false;
private realtimeLogsRevealed = false;
private groupedLogs: WorkflowLogs = new Map();
public ingestFileLogLines(logLines: WorkflowRawLogLine[]): void {
this.fileLogIds = new Set(logLines.flatMap(logLine => (logLine.logId ? [logLine.logId] : [])));
if (
!this.haveFileLogsCaughtUp &&
this.realtimeLogLines.some(logLine => this.fileLogIds.has(logLine.logId))
) {
this.haveFileLogsCaughtUp = true;
}
this.realtimeLogLines = this.realtimeLogLines.filter(
logLine => !this.fileLogIds.has(logLine.logId)
);
this.realtimeLogIds = new Set(this.realtimeLogLines.map(logLine => logLine.logId));
if (this.haveFileLogsCaughtUp) {
this.realtimeLogsRevealed = true;
}
this.groupedLogs = groupLogLinesIntoSteps(
mergeLogLines(logLines, this.realtimeLogsRevealed ? this.realtimeLogLines : [])
);
}
public ingestRealtimeLogLines(data: unknown): boolean {
if (!Array.isArray(data)) {
return false;
}
const publishedLogLines = data.filter(isRealtimeLogLine);
if (publishedLogLines.length === 0) {
return false;
}
if (
!this.haveFileLogsCaughtUp &&
publishedLogLines.some(logLine => this.fileLogIds.has(logLine.logId))
) {
this.haveFileLogsCaughtUp = true;
}
const newLogLines = publishedLogLines.filter(
logLine => !this.fileLogIds.has(logLine.logId) && !this.realtimeLogIds.has(logLine.logId)
);
for (const logLine of newLogLines) {
this.realtimeLogLines.push(logLine);
this.realtimeLogIds.add(logLine.logId);
}
if (!this.realtimeLogsRevealed) {
if (this.haveFileLogsCaughtUp) {
this.revealRealtimeLogs();
}
} else if (newLogLines.length > 0) {
groupLogLinesIntoSteps(newLogLines, this.groupedLogs);
}
return true;
}
public markCompleted(): void {
this.revealRealtimeLogs();
}
public getLogs(): WorkflowLogs {
return this.groupedLogs;
}
private revealRealtimeLogs(): void {
if (this.realtimeLogsRevealed) {
return;
}
this.realtimeLogsRevealed = true;
groupLogLinesIntoSteps(this.realtimeLogLines, this.groupedLogs);
}
}
type TrackedJob = {
logsState: WorkflowJobLogsState;
subscription: RealtimeLogsSubscription | null;
};
export function realtimeLogsTargetForJob(job: WorkflowJobResult): RealtimeLogsTargetInput | null {
if (job.turtleJobRun?.id) {
return { type: RealtimeLogsTargetType.JobRun, id: job.turtleJobRun.id };
}
const buildId = job.turtleBuild?.id ?? job.outputs?.build_id;
if (buildId) {
return { type: RealtimeLogsTargetType.Build, id: buildId };
}
return null;
}
export class WorkflowRunLogsWatcher {
private readonly trackedJobs = new Map<string, TrackedJob>();
private realtimeLogsClient?: RealtimeLogsClient | null;
constructor(
private readonly graphqlClient: ExpoGraphqlClient,
private readonly createRealtimeLogsClient: () => RealtimeLogsClient | null,
private readonly onRealtimeLogs: () => void
) {}
public async syncJobsAsync(
jobs: WorkflowJobResult[]
): Promise<Map<string, WorkflowJobLogsState>> {
await Promise.all(jobs.map(job => this.syncJobAsync(job)));
return new Map(
jobs.map(job => [
job.id,
nullthrows(
this.trackedJobs.get(job.id),
'job has to be in tracked jobs after it was synced'
).logsState,
])
);
}
public close(): void {
for (const trackedJob of this.trackedJobs.values()) {
trackedJob.subscription?.close();
trackedJob.subscription = null;
}
this.realtimeLogsClient?.close();
}
private async syncJobAsync(job: WorkflowJobResult): Promise<void> {
let trackedJob = this.trackedJobs.get(job.id);
if (!trackedJob) {
trackedJob = { logsState: new WorkflowJobLogsState(), subscription: null };
this.trackedJobs.set(job.id, trackedJob);
}
const workflowInProgress = job.status === WorkflowJobStatus.InProgress;
if (!workflowInProgress) {
trackedJob.subscription?.close();
trackedJob.subscription = null;
return;
}
await Promise.all([
trackedJob.subscription ? Promise.resolve() : this.subscribeAsync(job, trackedJob),
this.fetchLogsAsync(job, trackedJob),
]);
}
private async fetchLogsAsync(job: WorkflowJobResult, trackedJob: TrackedJob): Promise<void> {
try {
const logLines = await fetchAndParseLogsFromJobAsync(
{ graphqlClient: this.graphqlClient },
job
);
if (logLines) {
trackedJob.logsState.ingestFileLogLines(logLines);
}
} catch (err: any) {
Log.debug(`Failed to fetch logs for job ${job.id}: ${err.message}`);
}
}
private getRealtimeLogsClient(): RealtimeLogsClient | null {
if (this.realtimeLogsClient === undefined) {
this.realtimeLogsClient = this.createRealtimeLogsClient();
}
return this.realtimeLogsClient;
}
private async subscribeAsync(job: WorkflowJobResult, trackedJob: TrackedJob): Promise<void> {
const target = realtimeLogsTargetForJob(job);
if (!target) {
return;
}
const realtimeLogsClient = this.getRealtimeLogsClient();
if (!realtimeLogsClient) {
return;
}
trackedJob.subscription = await realtimeLogsClient.subscribeAsync({ target }, data => {
if (trackedJob.logsState.ingestRealtimeLogLines(data)) {
this.onRealtimeLogs();
}
});
}
}
@@ -1,11 +1,11 @@
import { Choice } from 'prompts';
import { WorkflowJobResult, WorkflowLogs } from './types';
import { fetchAndProcessLogsFromJobAsync } from './logs/parseLogs';
import {
choiceFromWorkflowJob,
choiceFromWorkflowRun,
choicesFromWorkflowLogs,
fetchAndProcessLogsFromJobAsync,
processWorkflowRuns,
} from './utils';
import { AppQuery } from '../../graphql/queries/AppQuery';
@@ -42,19 +42,26 @@ export type WorkflowRunWithJobsResult = WorkflowRunResult & {
logs?: string;
};
export type WorkflowLogLine = {
time: string;
msg: string;
export type WorkflowRawLogLine = {
logId?: string;
buildStepId?: string;
buildStepDisplayName?: string;
phase?: string;
time?: string;
msg?: string;
result?: string;
marker?: string;
err?: any;
};
export type WorkflowLogs = Map<
string,
{
key: string;
label: string;
logLines: WorkflowLogLine[];
}
>;
export type WorkflowLogLine = Pick<WorkflowRawLogLine, 'time' | 'result' | 'marker' | 'err'> &
Required<Pick<WorkflowRawLogLine, 'msg'>>;
export type StepLogs = {
key: string;
label: string;
result?: string;
logLines: WorkflowLogLine[];
};
export type WorkflowLogs = Map<string, StepLogs>;
@@ -1,7 +1,7 @@
import chalk from 'chalk';
import * as fs from 'node:fs';
import { fetchRawLogsForBuildJobAsync, fetchRawLogsForCustomJobAsync } from './fetchLogs';
import { fetchAndProcessLogsFromJobAsync } from './logs/parseLogs';
import {
WorkflowJobResult,
WorkflowLogLine,
@@ -11,20 +11,22 @@ import {
} from './types';
import {
WorkflowJobStatus,
WorkflowJobType,
WorkflowRunByIdQuery,
WorkflowRunByIdWithJobsQuery,
WorkflowRunFragment,
WorkflowRunStatus,
WorkflowRunTriggerEventType,
} from '../../graphql/generated';
import { WorkflowRunLogsWatcher } from './logs/watcher';
import { WorkflowRunQuery } from '../../graphql/queries/WorkflowRunQuery';
import Log from '../../log';
import { ora } from '../../ora';
import { ora, updateSpinnerText } from '../../ora';
import { Choice } from '../../prompts';
import formatFields from '../../utils/formatFields';
import { createRealtimeLogsClient } from '../../utils/centrifuge';
import { sleepAsync } from '../../utils/promise';
import { ExpoGraphqlClient } from '../context/contextUtils/createGraphqlClient';
import nullthrows from 'nullthrows';
export function computeTriggerInfoForWorkflowRun(run: WorkflowRunFragment): {
triggerType: WorkflowTriggerType;
@@ -85,19 +87,13 @@ export function choicesFromWorkflowLogs(
logs: WorkflowLogs
): (Choice & { name: string; status: string; logLines: WorkflowLogLine[] | undefined })[] {
return Array.from(logs.values())
.map(({ key, label, logLines }) => {
const stepStatus =
logLines?.filter(
(line: WorkflowLogLine) => line.marker === 'end-step' || line.marker === 'END_PHASE'
)[0]?.result ?? '';
return {
title: `${label} - ${stepStatus}`,
name: label,
status: stepStatus,
value: key,
logLines,
};
})
.map(({ key, label, result, logLines }) => ({
title: `${label} - ${result ?? ''}`,
name: label,
status: result ?? '',
value: key,
logLines,
}))
.filter(step => step.status !== 'skipped');
}
@@ -121,52 +117,6 @@ export function processWorkflowRuns(runs: WorkflowRunFragment[]): WorkflowRunRes
});
}
export async function fetchAndProcessLogsFromJobAsync(
state: { graphqlClient: ExpoGraphqlClient },
job: WorkflowJobResult
): Promise<WorkflowLogs | null> {
let rawLogs: string | null;
switch (job.type) {
case WorkflowJobType.Build:
case WorkflowJobType.Repack:
rawLogs = await fetchRawLogsForBuildJobAsync(state, job);
break;
default:
rawLogs = await fetchRawLogsForCustomJobAsync(job);
break;
}
if (!rawLogs) {
return null;
}
Log.debug(`rawLogs = ${JSON.stringify(rawLogs, null, 2)}`);
const logs: WorkflowLogs = new Map();
rawLogs.split('\n').forEach((line, index) => {
Log.debug(`line ${index} = ${JSON.stringify(line, null, 2)}`);
try {
const parsedLine = JSON.parse(line);
const { buildStepDisplayName, buildStepId, phase, time, msg, result, marker, err } =
parsedLine;
const stepKey = buildStepId ?? buildStepDisplayName ?? phase;
const stepLabel = buildStepDisplayName ?? buildStepId ?? phase;
if (stepKey && stepLabel) {
if (!logs.has(stepKey)) {
logs.set(stepKey, {
key: stepKey,
label: stepLabel,
logLines: [],
});
}
const logGroup = logs.get(stepKey)!;
if (buildStepDisplayName) {
logGroup.label = buildStepDisplayName;
}
logGroup.logLines.push({ time, msg, result, marker, err });
}
} catch {}
});
return logs;
}
function descriptionForJobStatus(status: WorkflowJobStatus): string {
switch (status) {
case WorkflowJobStatus.New:
@@ -188,82 +138,106 @@ function descriptionForJobStatus(status: WorkflowJobStatus): string {
}
}
export async function infoForActiveWorkflowRunAsync(
function isJobCompleted(status: WorkflowJobStatus): boolean {
switch (status) {
case WorkflowJobStatus.Success:
case WorkflowJobStatus.Failure:
case WorkflowJobStatus.Canceled:
case WorkflowJobStatus.Skipped:
return true;
case WorkflowJobStatus.New:
case WorkflowJobStatus.InProgress:
case WorkflowJobStatus.ActionRequired:
case WorkflowJobStatus.PendingCancel:
return false;
}
}
type WorkflowRunWithJobs = WorkflowRunByIdWithJobsQuery['workflowRuns']['byId'];
type JobWithLogs = {
job: WorkflowRunWithJobs['jobs'][number];
logs: WorkflowLogs;
};
function stepLogTail(
step: { logLines?: WorkflowLogLine[] },
maxLogLines: number // -1 means no limit
): string[] {
const logLines = step.logLines ?? [];
const tail = maxLogLines === -1 ? logLines : logLines.slice(-maxLogLines);
return tail.map(line => line.msg);
}
export async function logsForFailedWorkflowRunAsync(
graphqlClient: ExpoGraphqlClient,
workflowRun: WorkflowRunByIdWithJobsQuery['workflowRuns']['byId'],
workflowRun: WorkflowRunWithJobs
): Promise<JobWithLogs[]> {
const jobs = workflowRun.jobs.filter(job => job.status === WorkflowJobStatus.Failure);
return await Promise.all(
jobs.map(async job => {
const maybeLogs = await fetchAndProcessLogsFromJobAsync({ graphqlClient }, job);
return { job, logs: maybeLogs ?? new Map() };
})
);
}
export function formatActiveWorkflowRun(
jobLogs: JobWithLogs[],
maxLogLines: number = 5 // -1 means no limit
): Promise<string> {
const statusLines = [];
): string {
const statusValues = [];
for (const job of workflowRun.jobs) {
for (const { job, logs } of jobLogs) {
statusValues.push({ label: '', value: '' });
statusValues.push({ label: ' Job', value: job.name });
statusValues.push({ label: ' Status', value: descriptionForJobStatus(job.status) });
if (job.status !== WorkflowJobStatus.InProgress) {
continue;
}
const logs = await fetchAndProcessLogsFromJobAsync({ graphqlClient }, job);
const steps = logs ? choicesFromWorkflowLogs(logs) : [];
if (steps.length > 0) {
const currentStep = steps[steps.length - 1];
statusValues.push({ label: ' Current step', value: currentStep.name });
if (currentStep?.logLines?.length) {
statusValues.push({ label: ' Current logs', value: '' });
const currentLogs =
currentStep.logLines
?.map(line => line.msg)
.filter((_, index) => {
if (maxLogLines === -1) {
return true;
}
return index > (currentStep.logLines?.length ?? 0) - maxLogLines;
}) ?? [];
for (const log of currentLogs) {
statusValues.push({ label: '', value: log });
}
}
}
}
statusValues.push({ label: '', value: '' });
statusLines.push(formatFields(statusValues));
return statusLines.join('\n');
}
export async function infoForFailedWorkflowRunAsync(
graphqlClient: ExpoGraphqlClient,
workflowRun: WorkflowRunByIdWithJobsQuery['workflowRuns']['byId'],
maxLogLines: number = -1 // -1 means no limit
): Promise<string> {
const statusLines = [];
const statusValues = [];
const logLinesToKeep = maxLogLines === -1 ? Infinity : maxLogLines;
for (const job of workflowRun.jobs) {
if (job.status !== WorkflowJobStatus.Failure) {
const steps = choicesFromWorkflowLogs(logs);
if (steps.length === 0) {
continue;
}
const logs = await fetchAndProcessLogsFromJobAsync({ graphqlClient }, job);
const steps = logs ? choicesFromWorkflowLogs(logs) : [];
statusValues.push({ label: '', value: '' });
statusValues.push({ label: ' Failed job', value: job.name });
if (steps.length > 0) {
const failedStep = steps.find(step => step.status === 'fail' || step.status === 'failed');
if (failedStep) {
const logs = failedStep.logLines?.map(line => line.msg).slice(-logLinesToKeep) ?? [];
statusValues.push({ label: ' Failed step', value: failedStep.name });
statusValues.push({
label: ' Logs for failed step',
value: '',
});
for (const log of logs) {
statusValues.push({ label: '', value: log });
}
const currentStep = steps[steps.length - 1];
statusValues.push({ label: ' Current step', value: currentStep.name });
if (currentStep.logLines?.length) {
statusValues.push({ label: ' Current logs', value: '' });
for (const log of stepLogTail(currentStep, maxLogLines)) {
statusValues.push({ label: '', value: log });
}
}
}
statusValues.push({ label: '', value: '' });
statusLines.push(formatFields(statusValues));
return statusLines.join('\n');
return formatFields(statusValues);
}
export function formatFailedWorkflowRun(
jobLogs: JobWithLogs[],
maxLogLines: number = -1 // -1 means no limit
): string {
const statusValues = [];
for (const { job, logs } of jobLogs) {
statusValues.push({ label: '', value: '' });
statusValues.push({ label: ' Failed job', value: job.name });
const steps = choicesFromWorkflowLogs(logs);
const failedStep = steps.find(step => step.status === 'fail' || step.status === 'failed');
if (!failedStep) {
continue;
}
statusValues.push({ label: ' Failed step', value: failedStep.name });
statusValues.push({ label: ' Logs for failed step', value: '' });
for (const log of stepLogTail(failedStep, maxLogLines)) {
statusValues.push({ label: '', value: log });
}
}
statusValues.push({ label: '', value: '' });
return formatFields(statusValues);
}
export async function fileExistsAsync(filePath: string): Promise<boolean> {
return await fs.promises
.access(filePath, fs.constants.F_OK)
@@ -312,65 +286,111 @@ export async function showWorkflowStatusAsync(
stream: spinnerUsesStdErr ? process.stderr : process.stdout,
text: '',
}).start();
spinner.prefixText = chalk`{bold.yellow Workflow run is waiting to start:}`;
updateSpinnerText(spinner, {
prefixText: chalk`{bold.yellow Workflow run is waiting to start:}`,
});
let failedFetchesCount = 0;
while (true) {
try {
const workflowRun = await WorkflowRunQuery.withJobsByIdAsync(graphqlClient, workflowRunId, {
useCache: false,
});
failedFetchesCount = 0;
switch (workflowRun.status) {
case WorkflowRunStatus.New:
break;
case WorkflowRunStatus.InProgress: {
spinner.prefixText = chalk`{bold.green Workflow run is in progress:}`;
spinner.text = await infoForActiveWorkflowRunAsync(graphqlClient, workflowRun, 5);
break;
}
case WorkflowRunStatus.ActionRequired:
spinner.prefixText = chalk`{bold.yellow Workflow run is waiting for action:}`;
break;
case WorkflowRunStatus.Canceled:
spinner.prefixText = chalk`{bold.yellow Workflow has been canceled.}`;
spinner.stopAndPersist();
return workflowRun;
case WorkflowRunStatus.Failure: {
spinner.prefixText = chalk`{bold.red Workflow has failed.}`;
const failedInfo = await infoForFailedWorkflowRunAsync(graphqlClient, workflowRun, 30);
spinner.fail(failedInfo);
return workflowRun;
}
case WorkflowRunStatus.Success:
spinner.prefixText = chalk`{bold.green Workflow has completed successfully.}`;
spinner.text = '';
spinner.succeed('');
return workflowRun;
}
if (!waitForCompletion) {
if (spinner.isSpinning) {
spinner.stopAndPersist();
}
return workflowRun;
}
} catch {
spinner.text = '⚠ Failed to fetch the workflow run status. Check your network connection.';
failedFetchesCount += 1;
if (failedFetchesCount > 6) {
spinner.fail('Failed to fetch the workflow run status 6 times in a row. Aborting wait.');
process.exit(workflowRunExitCodes.WAIT_ABORTED);
}
const watcher = new WorkflowRunLogsWatcher(
graphqlClient,
() => createRealtimeLogsClient(graphqlClient),
() => {
renderActiveWorkflowRun();
}
);
let failedFetchesCount = 0;
let renderActiveWorkflowRun = (): void => {};
await sleepAsync(10 /* seconds */ * 1000 /* milliseconds */);
try {
while (true) {
try {
const workflowRun = await WorkflowRunQuery.withJobsByIdAsync(graphqlClient, workflowRunId, {
useCache: false,
});
failedFetchesCount = 0;
switch (workflowRun.status) {
case WorkflowRunStatus.New:
break;
case WorkflowRunStatus.InProgress: {
updateSpinnerText(spinner, {
prefixText: chalk`{bold.green Workflow run is in progress:}`,
});
const logsStates = await watcher.syncJobsAsync(workflowRun.jobs);
for (const job of workflowRun.jobs) {
if (isJobCompleted(job.status)) {
nullthrows(
logsStates.get(job.id),
'syncJobsAsync must have been called before markCompleted'
).markCompleted();
}
}
renderActiveWorkflowRun = () => {
const text = formatActiveWorkflowRun(
workflowRun.jobs.map(job => ({
job,
logs: nullthrows(
logsStates.get(job.id),
'syncJobsAsync must have been called before getLogs'
).getLogs(),
}))
);
updateSpinnerText(spinner, { text });
};
renderActiveWorkflowRun();
break;
}
case WorkflowRunStatus.ActionRequired:
updateSpinnerText(spinner, {
prefixText: chalk`{bold.yellow Workflow run is waiting for action:}`,
});
break;
case WorkflowRunStatus.Canceled:
updateSpinnerText(spinner, {
prefixText: chalk`{bold.yellow Workflow has been canceled.}`,
});
spinner.stopAndPersist();
return workflowRun;
case WorkflowRunStatus.Failure: {
updateSpinnerText(spinner, {
prefixText: chalk`{bold.red Workflow has failed.}`,
});
const jobLogs = await logsForFailedWorkflowRunAsync(graphqlClient, workflowRun);
spinner.fail(formatFailedWorkflowRun(jobLogs, 30));
return workflowRun;
}
case WorkflowRunStatus.Success:
updateSpinnerText(spinner, {
prefixText: chalk`{bold.green Workflow has completed successfully.}`,
});
spinner.succeed('');
return workflowRun;
}
if (!waitForCompletion) {
if (spinner.isSpinning) {
spinner.stopAndPersist();
}
return workflowRun;
}
} catch {
updateSpinnerText(spinner, {
text: '⚠ Failed to fetch the workflow run status. Check your network connection.',
});
failedFetchesCount += 1;
if (failedFetchesCount > 6) {
spinner.fail('Failed to fetch the workflow run status 6 times in a row. Aborting wait.');
process.exit(workflowRunExitCodes.WAIT_ABORTED);
}
}
await sleepAsync(10 /* seconds */ * 1000 /* milliseconds */);
}
} finally {
watcher.close();
}
}
@@ -17,7 +17,6 @@ jest.mock('fs-extra', () => ({
jest.mock('../../../prompts', () => ({
promptAsync: jest.fn(),
}));
jest.mock('../../../onboarding/git');
jest.mock('../../../onboarding/runCommand');
jest.mock('../../../graphql/mutations/AppMutation');
jest.mock('../../../project/expoConfig');
+17 -3
View File
@@ -6,7 +6,7 @@ import { getProjectDashboardUrl } from '../../build/utils/url';
import EasCommand from '../../commandUtils/EasCommand';
import { ExpoGraphqlClient } from '../../commandUtils/context/contextUtils/createGraphqlClient';
import {
cloneTemplateAsync,
downloadTemplateAsync,
initializeGitRepositoryAsync,
installProjectDependenciesAsync,
} from '../../commandUtils/new/commands';
@@ -14,10 +14,12 @@ import {
generateProjectConfigAsync,
promptForProjectAccountAsync,
} from '../../commandUtils/new/configs';
import { resolveTemplateAsync } from '../../commandUtils/new/sdkVersion';
import {
copyProjectTemplatesAsync,
generateAppConfigAsync,
generateEasConfigAsync,
removeTemplateFilesAsync,
updatePackageJsonAsync,
updateReadmeAsync,
} from '../../commandUtils/new/projectFiles';
@@ -108,7 +110,9 @@ export async function generateProjectFilesAsync(
await generateEasConfigAsync(projectDir);
await updatePackageJsonAsync(projectDir);
await updatePackageJsonAsync(projectDir, app.slug);
await removeTemplateFilesAsync(projectDir);
await copyProjectTemplatesAsync(projectDir);
@@ -146,6 +150,9 @@ export default class New extends EasCommand {
options: [...PACKAGE_MANAGERS] as const,
default: 'npm',
})(),
'sdk-version': Flags.string({
description: 'Expo SDK version to use for the new project (e.g. 57, or "latest")',
}),
};
static override contextDefinition = {
@@ -167,13 +174,20 @@ export default class New extends EasCommand {
Log.log(`👋 Welcome to Expo, ${actor.username}!`);
// Resolve the template before any prompts when --sdk-version is provided,
// so an invalid version fails fast.
let template = flags['sdk-version']
? await resolveTemplateAsync({ sdkVersion: flags['sdk-version'] })
: null;
const {
projectName,
projectDirectory: targetProjectDirectory,
projectAccount,
} = await generateConfigsAsync(args, actor, graphqlClient);
const projectDirectory = await cloneTemplateAsync(targetProjectDirectory);
template ??= await resolveTemplateAsync({});
const projectDirectory = await downloadTemplateAsync(targetProjectDirectory, template);
const packageManager = flags['package-manager'];
await installProjectDependenciesAsync(projectDirectory, packageManager);
@@ -122,7 +122,13 @@ describe(SimulatorGet, () => {
}
it('emits JSON when --json is passed', async () => {
const session = makeDeviceRunSession();
const session = makeDeviceRunSession({
type: DeviceRunSessionType.ServeSim,
remoteConfig: {
__typename: 'ServeSimRunSessionRemoteConfig',
previewUrl: 'https://preview.example.com',
},
});
mockByIdAsync.mockResolvedValue(session);
const { command, getContextAsync } = createCommand(['--id', 'session-123', '--json']);
@@ -137,7 +143,7 @@ describe(SimulatorGet, () => {
expect(mockPrintJsonOnlyOutput).toHaveBeenCalledWith({
id: 'session-123',
name: undefined,
type: 'agent-device',
type: 'web-preview-only',
status: DeviceRunSessionStatus.InProgress,
platform: AppPlatform.Ios,
createdAt: '2025-01-01T00:00:00.000Z',
@@ -6,6 +6,7 @@ import {
AppPlatform,
CreateDeviceRunSessionMutation,
DeviceRunSessionByIdQuery,
DeviceRunSessionResourceClass,
DeviceRunSessionStatus,
DeviceRunSessionType,
JobRunStatus,
@@ -331,6 +332,33 @@ describe(Simulator, () => {
);
});
it('creates a WebPreviewOnly session for --type web-preview-only', async () => {
mockByIdAsync.mockResolvedValue(
makeDeviceRunSession({
type: DeviceRunSessionType.WebPreviewOnly,
remoteConfig: {
__typename: 'WebPreviewOnlyRunSessionRemoteConfig',
previewUrl: 'https://preview.example.test',
},
})
);
const { command } = createCommand([
'--platform',
'ios',
'--type',
'web-preview-only',
'--non-interactive',
]);
await command.runAsync();
expect(mockCreateDeviceRunSessionAsync).toHaveBeenCalledWith(
graphqlClient,
expect.objectContaining({ type: DeviceRunSessionType.WebPreviewOnly })
);
expect(Log.log).toHaveBeenCalledWith(expect.stringContaining('https://preview.example.test'));
});
it('overwrites .env.eas-simulator when outputting dotenv and the file exists', async () => {
const { command } = createCommand([
'--platform',
@@ -522,6 +550,35 @@ describe(Simulator, () => {
);
});
it('omits resourceClass when --resource-class is not set', async () => {
const { command } = createCommand(['--platform', 'ios', '--non-interactive']);
await command.runAsync();
expect(mockCreateDeviceRunSessionAsync.mock.calls[0][1]).not.toHaveProperty('resourceClass');
});
it.each([
['large', DeviceRunSessionResourceClass.Large],
['medium', DeviceRunSessionResourceClass.Medium],
] as const)(
'forwards --resource-class %s to the create mutation',
async (flag, resourceClass) => {
const { command } = createCommand([
'--platform',
'ios',
'--non-interactive',
'--resource-class',
flag,
]);
await command.runAsync();
expect(mockCreateDeviceRunSessionAsync).toHaveBeenCalledWith(
graphqlClient,
expect.objectContaining({ resourceClass })
);
}
);
it('forwards --build-id to the create mutation', async () => {
const { command } = createCommand([
'--platform',
@@ -111,7 +111,7 @@ describe(SimulatorList, () => {
}
it('emits JSON when --json is passed', async () => {
const session = makeSession();
const session = makeSession({ type: DeviceRunSessionType.ServeSim });
mockListByAppIdAsync.mockResolvedValue(makeConnection([session]));
const { command, getContextAsync } = createCommand(['--json']);
@@ -132,7 +132,7 @@ describe(SimulatorList, () => {
{
id: 'session-123',
name: undefined,
type: 'agent-device',
type: 'web-preview-only',
status: DeviceRunSessionStatus.InProgress,
platform: AppPlatform.Ios,
createdAt: '2025-01-01T00:00:00.000Z',
@@ -162,6 +162,8 @@ describe(SimulatorList, () => {
'new',
'--type',
'appium',
'--type',
'web-preview-only',
'--platform',
'ios',
'--name',
@@ -179,7 +181,7 @@ describe(SimulatorList, () => {
after: 'page-cursor',
filter: {
statuses: [DeviceRunSessionStatus.InProgress, DeviceRunSessionStatus.New],
types: [DeviceRunSessionType.Appium],
types: [DeviceRunSessionType.Appium, DeviceRunSessionType.WebPreviewOnly],
platforms: [AppPlatform.Ios],
name: 'checkout',
},
@@ -30,6 +30,8 @@ import {
} from '../../simulator/env';
import { resolveExpoGoSdkVersionAsync } from '../../simulator/expoGo';
import {
DEVICE_RUN_SESSION_RESOURCE_CLASS_BY_FLAG_VALUE,
DEVICE_RUN_SESSION_RESOURCE_CLASS_FLAG_VALUES,
DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE,
DEVICE_RUN_SESSION_TYPE_FLAG_VALUES,
DeviceRunSessionRemoteConfig,
@@ -101,7 +103,8 @@ export default class Simulator extends EasCommand {
'Expo or development-client URL to open in the installed application after it launches.',
}),
type: Flags.option({
description: 'Type of simulator session to create',
description:
'Type of simulator session to create. All session types include a web preview. agent-device, appium, and argent also include an automation interface; web-preview-only includes no automation interface.',
options: Object.values(DEVICE_RUN_SESSION_TYPE_FLAG_VALUES),
default: DEVICE_RUN_SESSION_TYPE_FLAG_VALUES[DeviceRunSessionType.AgentDevice],
})(),
@@ -119,6 +122,11 @@ export default class Simulator extends EasCommand {
'Stop the simulator session automatically after this many minutes without session activity. When omitted, the session has no idle timeout and runs until its maximum duration.',
min: 0,
}),
'resource-class': Flags.option({
description: 'The instance type that will be used to run this simulator session',
hidden: true,
options: Object.values(DEVICE_RUN_SESSION_RESOURCE_CLASS_FLAG_VALUES),
})(),
force: Flags.boolean({
description:
'[default: true] Create a new simulator session even when an existing simulator session is present in the environment.',
@@ -177,6 +185,9 @@ export default class Simulator extends EasCommand {
const sdkVersionFromFlag = flags['sdk-version']?.trim() || undefined;
const launchArgs = flags['launch-arg'];
const openUrl = flags['open-url']?.trim() || undefined;
const resourceClass = flags['resource-class']
? DEVICE_RUN_SESSION_RESOURCE_CLASS_BY_FLAG_VALUE[flags['resource-class']]
: undefined;
if (sdkVersionFromFlag && !flags['expo-go']) {
throw new EasCommandError('The --sdk-version flag can only be used with --expo-go.');
@@ -233,6 +244,7 @@ export default class Simulator extends EasCommand {
...(expoGoSdkVersion ? { expoGo: true, sdkVersion: expoGoSdkVersion } : {}),
...(launchArgs?.length ? { launchArgs } : {}),
...(openUrl ? { openUrl } : {}),
...(resourceClass ? { resourceClass } : {}),
maxRunTimeMinutes: flags['max-duration-minutes'],
maxIdleTimeMinutes: flags['max-idle-time-minutes'],
});
@@ -61,7 +61,8 @@ export default class SimulatorList extends EasCommand {
multiple: true,
})(),
type: Flags.option({
description: 'Filter by session type (repeatable)',
description:
'Filter by session type (repeatable). All session types include a web preview. agent-device, appium, and argent also include an automation interface; web-preview-only includes no automation interface.',
options: Object.values(DEVICE_RUN_SESSION_TYPE_FLAG_VALUES),
multiple: true,
})(),
+251 -9
View File
@@ -83,6 +83,8 @@ export type Account = {
accountFeatureGates: Scalars['JSONObject']['output'];
/** Coalesced project activity for all apps belonging to this account. */
activityTimelineProjectActivities: Array<ActivityTimelineProjectActivity>;
/** AI provider connections available to agent sandbox sessions. */
agentProviderConnections: Array<AgentProviderConnection>;
aiChatEnabled: Scalars['Boolean']['output'];
appCount: Scalars['Int']['output'];
/** @deprecated Use appStoreConnectApiKeysPaginated */
@@ -1078,6 +1080,80 @@ export type AgentDeviceRunSessionRemoteConfig = {
webPreviewUrl?: Maybe<Scalars['String']['output']>;
};
export enum AgentProvider {
Anthropic = 'ANTHROPIC',
Openai = 'OPENAI'
}
export type AgentProviderConnection = {
__typename?: 'AgentProviderConnection';
account: Account;
createdAt: Scalars['DateTime']['output'];
createdByActor?: Maybe<Actor>;
expiresAt?: Maybe<Scalars['DateTime']['output']>;
id: Scalars['ID']['output'];
name?: Maybe<Scalars['String']['output']>;
provider: AgentProvider;
providerAccountEmail?: Maybe<Scalars['String']['output']>;
providerAccountId?: Maybe<Scalars['String']['output']>;
providerPlan?: Maybe<Scalars['String']['output']>;
status: AgentProviderConnectionStatus;
updatedAt: Scalars['DateTime']['output'];
};
export enum AgentProviderConnectionAuthorizationStatus {
Expired = 'EXPIRED',
Pending = 'PENDING',
Succeeded = 'SUCCEEDED'
}
export type AgentProviderConnectionMutation = {
__typename?: 'AgentProviderConnectionMutation';
completeClaudeConnection: AgentProviderConnection;
deleteAgentProviderConnection: AgentProviderConnection;
pollCodexConnection: CodexConnectionPollResult;
startClaudeConnection: ClaudeAuthorization;
startCodexConnection: CodexDeviceAuthorization;
updateAgentProviderConnection: AgentProviderConnection;
};
export type AgentProviderConnectionMutationCompleteClaudeConnectionArgs = {
input: CompleteClaudeConnectionInput;
};
export type AgentProviderConnectionMutationDeleteAgentProviderConnectionArgs = {
id: Scalars['ID']['input'];
};
export type AgentProviderConnectionMutationPollCodexConnectionArgs = {
requestId: Scalars['ID']['input'];
};
export type AgentProviderConnectionMutationStartClaudeConnectionArgs = {
input: StartClaudeConnectionInput;
};
export type AgentProviderConnectionMutationStartCodexConnectionArgs = {
accountId: Scalars['ID']['input'];
};
export type AgentProviderConnectionMutationUpdateAgentProviderConnectionArgs = {
id: Scalars['ID']['input'];
input: UpdateAgentProviderConnectionInput;
};
export enum AgentProviderConnectionStatus {
Invalid = 'INVALID',
UnableToVerify = 'UNABLE_TO_VERIFY',
Valid = 'VALID'
}
export type AndroidAppBuildCredentials = {
__typename?: 'AndroidAppBuildCredentials';
androidKeystore?: Maybe<AndroidKeystore>;
@@ -1472,6 +1548,8 @@ export type App = Project & {
internalDistributionBuildPrivacy: AppInternalDistributionBuildPrivacy;
/** iOS app credentials for the project */
iosAppCredentials: Array<IosAppCredentials>;
/** Whether this app qualifies for the observe-notification-lapsed-paid experiment */
isEligibleForObserveLapsedPaidNotification: Scalars['Boolean']['output'];
/**
* Tells you if the project can show the Observe promotional notification. This field is part of
* the observe-notification experiment.
@@ -2173,6 +2251,11 @@ export type AppNotificationPreferenceInput = {
export type AppObserve = {
__typename?: 'AppObserve';
appVersions: Array<AppObserveAppVersion>;
/**
* A single custom event by the `id` of an `AppObserveCustomEvent`. Null when the app has no
* such event, including when it has aged out of retention.
*/
customEvent?: Maybe<AppObserveCustomEvent>;
customEventCounts: AppObserveCustomEventCounts;
customEventList: AppObserveCustomEventListConnection;
customEventNames: AppObserveCustomEventNames;
@@ -2185,6 +2268,14 @@ export type AppObserve = {
errorStats: AppObserveErrorStats;
/** Time-bucketed exception counts split into fatal vs non-fatal, for the stacked-bar chart. */
errorTimeSeries: AppObserveErrorTimeSeries;
/**
* A single metric event by the `id` of an `AppObserveEvent`. Null when the app has no such
* event, including when it has aged out of retention or the id is not one this API issued.
*
* `sessionEventCount` and `userEventCount` are always null here: they are aggregates over a
* time range, and a single event does not supply one.
*/
event?: Maybe<AppObserveEvent>;
events: AppObserveEventsConnection;
/** Highest Expo SDK version seen in telemetry over the trailing 30 days; null when none was reported. */
latestExpoSdkVersion?: Maybe<Scalars['String']['output']>;
@@ -2208,6 +2299,7 @@ export type AppObserve = {
* universe used by `appVersions` headline counts.
*/
uniqueActiveUserCount: Scalars['Int']['output'];
/** Update download activity (the Recent updates list). Not available on the free tier or legacy plans. */
updates: AppObserveUpdatesConnection;
};
@@ -2217,6 +2309,11 @@ export type AppObserveAppVersionsArgs = {
};
export type AppObserveCustomEventArgs = {
id: Scalars['ID']['input'];
};
export type AppObserveCustomEventCountsArgs = {
input: AppObserveCustomEventCountsInput;
};
@@ -2242,6 +2339,7 @@ export type AppObserveCustomEventNamesArgs = {
isEmbeddedUpdate?: InputMaybe<Scalars['Boolean']['input']>;
orderBy?: InputMaybe<AppObserveCustomEventNamesOrderBy>;
platform?: InputMaybe<AppObservePlatform>;
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime: Scalars['DateTime']['input'];
};
@@ -2249,6 +2347,7 @@ export type AppObserveCustomEventNamesArgs = {
export type AppObserveEnvironmentsArgs = {
endTime?: InputMaybe<Scalars['DateTime']['input']>;
platform?: InputMaybe<AppObservePlatform>;
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime?: InputMaybe<Scalars['DateTime']['input']>;
};
@@ -2273,6 +2372,11 @@ export type AppObserveErrorTimeSeriesArgs = {
};
export type AppObserveEventArgs = {
id: Scalars['ID']['input'];
};
export type AppObserveEventsArgs = {
after?: InputMaybe<Scalars['String']['input']>;
before?: InputMaybe<Scalars['String']['input']>;
@@ -2479,7 +2583,9 @@ export type AppObserveCustomEventCountsInput = {
environment?: InputMaybe<Scalars['String']['input']>;
eventName: Scalars['String']['input'];
isEmbeddedUpdate?: InputMaybe<Scalars['Boolean']['input']>;
platform: AppObservePlatform;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. One of platform or platforms is required. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime: Scalars['DateTime']['input'];
};
@@ -2508,6 +2614,8 @@ export type AppObserveCustomEventListFilter = {
eventName?: InputMaybe<Scalars['String']['input']>;
isEmbeddedUpdate?: InputMaybe<Scalars['Boolean']['input']>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
propertyFilters?: InputMaybe<Array<AppObserveCustomEventPropertyFilter>>;
sessionId?: InputMaybe<Scalars['String']['input']>;
startTime?: InputMaybe<Scalars['DateTime']['input']>;
@@ -2615,6 +2723,8 @@ export type AppObserveErrorGroupBreakdownInput = {
fingerprint: Scalars['String']['input'];
isEmbeddedUpdate?: InputMaybe<Scalars['Boolean']['input']>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime: Scalars['DateTime']['input'];
};
@@ -2647,6 +2757,8 @@ export type AppObserveErrorGroupsInput = {
isEmbeddedUpdate?: InputMaybe<Scalars['Boolean']['input']>;
orderBy?: InputMaybe<AppObserveErrorGroupsOrderBy>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
/** Restrict to fatal-only or non-fatal-only errors. */
severity?: InputMaybe<AppObserveErrorSeverity>;
/** Restrict to a capture source (e.g. global). */
@@ -2710,6 +2822,8 @@ export type AppObserveErrorStatsInput = {
environment?: InputMaybe<Scalars['String']['input']>;
isEmbeddedUpdate?: InputMaybe<Scalars['Boolean']['input']>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime: Scalars['DateTime']['input'];
};
@@ -2746,6 +2860,8 @@ export type AppObserveErrorTimeSeriesInput = {
fingerprint?: InputMaybe<Scalars['String']['input']>;
isEmbeddedUpdate?: InputMaybe<Scalars['Boolean']['input']>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime: Scalars['DateTime']['input'];
};
@@ -2817,6 +2933,8 @@ export type AppObserveEventsFilter = {
isEmbeddedUpdate?: InputMaybe<Scalars['Boolean']['input']>;
metricName?: InputMaybe<Scalars['String']['input']>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
routeName?: InputMaybe<Scalars['String']['input']>;
sessionId?: InputMaybe<Scalars['String']['input']>;
startTime?: InputMaybe<Scalars['DateTime']['input']>;
@@ -2871,6 +2989,8 @@ export type AppObserveNavigationRoutesFilter = {
endTime: Scalars['DateTime']['input'];
environment?: InputMaybe<Scalars['String']['input']>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
routeNames?: InputMaybe<Array<Scalars['String']['input']>>;
startTime: Scalars['DateTime']['input'];
};
@@ -2924,6 +3044,8 @@ export type AppObserveOverviewEngagementInput = {
endTime: Scalars['DateTime']['input'];
environment?: InputMaybe<Scalars['String']['input']>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime: Scalars['DateTime']['input'];
};
@@ -2971,6 +3093,8 @@ export type AppObserveOverviewStabilityInput = {
endTime: Scalars['DateTime']['input'];
environment?: InputMaybe<Scalars['String']['input']>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime: Scalars['DateTime']['input'];
};
@@ -2978,9 +3102,13 @@ export type AppObserveOverviewUpdate = {
__typename?: 'AppObserveOverviewUpdate';
/** Null for the embedded bundle. */
appUpdateId?: Maybe<Scalars['ID']['output']>;
/** Downloads recorded in range across platforms; null when none were recorded (and always for the embedded bundle). */
/**
* Downloads recorded in range across platforms; null when none were recorded
* (and always for the embedded bundle). Download data is paid-only: on the
* free tier and legacy plans this is always null.
*/
downloadCount?: Maybe<Scalars['Int']['output']>;
/** One entry per platform with downloads in range; empty for the embedded bundle. */
/** One entry per platform with downloads in range; empty for the embedded bundle and on the free tier and legacy plans. */
downloads: Array<AppObserveOverviewUpdateDownload>;
eventCount: Scalars['Int']['output'];
firstSeenAt: Scalars['DateTime']['output'];
@@ -3092,6 +3220,8 @@ export type AppObserveReleasesInput = {
environment?: InputMaybe<Scalars['String']['input']>;
metricNames?: InputMaybe<Array<Scalars['String']['input']>>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime: Scalars['DateTime']['input'];
};
@@ -3129,6 +3259,8 @@ export type AppObserveTimeSeriesInput = {
isEmbeddedUpdate?: InputMaybe<Scalars['Boolean']['input']>;
metricName: Scalars['String']['input'];
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
routeName?: InputMaybe<Scalars['String']['input']>;
startTime: Scalars['DateTime']['input'];
};
@@ -3180,6 +3312,8 @@ export type AppObserveUpdatesInput = {
first?: InputMaybe<Scalars['Int']['input']>;
orderBy?: InputMaybe<AppObserveUpdatesOrderBy>;
platform?: InputMaybe<AppObservePlatform>;
/** Filter to these platforms. Cannot be set together with platform. */
platforms?: InputMaybe<Array<AppObservePlatform>>;
startTime: Scalars['DateTime']['input'];
};
@@ -5031,6 +5165,13 @@ export type ChatTokenUsage = {
usedTokens: Scalars['Int']['output'];
};
export type ClaudeAuthorization = {
__typename?: 'ClaudeAuthorization';
authorizationUrl: Scalars['String']['output'];
expiresAt: Scalars['DateTime']['output'];
requestId: Scalars['ID']['output'];
};
export type CodeSigningInfo = {
__typename?: 'CodeSigningInfo';
alg: Scalars['String']['output'];
@@ -5044,6 +5185,26 @@ export type CodeSigningInfoInput = {
sig: Scalars['String']['input'];
};
export type CodexConnectionPollResult = {
__typename?: 'CodexConnectionPollResult';
connection?: Maybe<AgentProviderConnection>;
status: AgentProviderConnectionAuthorizationStatus;
};
export type CodexDeviceAuthorization = {
__typename?: 'CodexDeviceAuthorization';
expiresAt: Scalars['DateTime']['output'];
pollingIntervalSeconds: Scalars['Int']['output'];
requestId: Scalars['ID']['output'];
userCode: Scalars['String']['output'];
verificationUrl: Scalars['String']['output'];
};
export type CompleteClaudeConnectionInput = {
authorizationCode: Scalars['String']['input'];
requestId: Scalars['ID']['input'];
};
export type CompletePostHogConnectionInput = {
code: Scalars['String']['input'];
state: Scalars['ID']['input'];
@@ -5265,7 +5426,7 @@ export type CreateDeviceRunSessionInput = {
/**
* Stop the session automatically after this many minutes without observed
* session activity. Must be positive and smaller than the session's maximum
* duration (2 hours, or maxRunTimeMinutes when set). Only supported for
* duration (maxRunTimeMinutes when set). Only supported for
* agent-device, argent, and Appium sessions. If omitted, the session has no idle
* timeout.
*/
@@ -5293,6 +5454,7 @@ export type CreateDeviceRunSessionInput = {
*/
packageVersion?: InputMaybe<Scalars['String']['input']>;
platform: AppPlatform;
resourceClass?: InputMaybe<DeviceRunSessionResourceClass>;
/**
* Expo SDK version used to select an Expo Go application archive. Only supported when expoGo is
* true. If omitted, the current Expo Go archive for the platform is used.
@@ -6043,7 +6205,17 @@ export type DeploymentsMutationDeleteWorkerDeploymentByIdentifierArgs = {
export type DeviceRunSession = {
__typename?: 'DeviceRunSession';
app: App;
/**
* Direct application archive URL installed and launched for this session. Null when
* the session uses an EAS Build or starts without an app.
*/
applicationArchiveUrl?: Maybe<Scalars['String']['output']>;
artifacts: Array<DeviceRunSessionArtifact>;
/**
* EAS Build installed and launched for this session. Null when the session uses
* a direct application archive URL or starts without an app.
*/
build?: Maybe<Build>;
createdAt: Scalars['DateTime']['output'];
finishedAt?: Maybe<Scalars['DateTime']['output']>;
id: Scalars['ID']['output'];
@@ -6174,7 +6346,12 @@ export type DeviceRunSessionQueryByIdArgs = {
deviceRunSessionId: Scalars['ID']['input'];
};
export type DeviceRunSessionRemoteConfig = AgentDeviceRunSessionRemoteConfig | AppiumRunSessionRemoteConfig | ArgentRunSessionRemoteConfig | ServeSimRunSessionRemoteConfig;
export type DeviceRunSessionRemoteConfig = AgentDeviceRunSessionRemoteConfig | AppiumRunSessionRemoteConfig | ArgentRunSessionRemoteConfig | ServeSimRunSessionRemoteConfig | WebPreviewOnlyRunSessionRemoteConfig;
export enum DeviceRunSessionResourceClass {
Large = 'LARGE',
Medium = 'MEDIUM'
}
export enum DeviceRunSessionStatus {
Errored = 'ERRORED',
@@ -6187,7 +6364,10 @@ export enum DeviceRunSessionType {
AgentDevice = 'AGENT_DEVICE',
Appium = 'APPIUM',
Argent = 'ARGENT',
ServeSim = 'SERVE_SIM'
/** @deprecated Use WEB_PREVIEW_ONLY instead. */
ServeSim = 'SERVE_SIM',
/** A session accessed only through its web preview. Currently supported on iOS. */
WebPreviewOnly = 'WEB_PREVIEW_ONLY'
}
export type DiscordUser = {
@@ -7054,6 +7234,7 @@ export type EmbeddedUpdateQueryByIdArgs = {
export enum EntityTypeName {
AccountEntity = 'AccountEntity',
AccountSsoConfigurationEntity = 'AccountSSOConfigurationEntity',
AgentProviderConnectionEntity = 'AgentProviderConnectionEntity',
AndroidAppCredentialsEntity = 'AndroidAppCredentialsEntity',
AndroidKeystoreEntity = 'AndroidKeystoreEntity',
AppEntity = 'AppEntity',
@@ -9437,6 +9618,7 @@ export type RootMutation = {
accountSSOConfiguration: AccountSsoConfigurationMutation;
/** Mutations for Actor experiments */
actorExperiment: ActorExperimentMutation;
agentProviderConnection: AgentProviderConnectionMutation;
/** Mutations that modify the build credentials for an Android app */
androidAppBuildCredentials: AndroidAppBuildCredentialsMutation;
/** Mutations that modify the credentials for an Android app */
@@ -10039,6 +10221,10 @@ export type SentryProjectMutationDeleteSentryProjectArgs = {
sentryProjectId: Scalars['ID']['input'];
};
/**
* Legacy remote config for SERVE_SIM sessions. Use WEB_PREVIEW_ONLY sessions and
* WebPreviewOnlyRunSessionRemoteConfig instead.
*/
export type ServeSimRunSessionRemoteConfig = {
__typename?: 'ServeSimRunSessionRemoteConfig';
previewUrl: Scalars['String']['output'];
@@ -10129,6 +10315,10 @@ export enum StandardOffer {
YearlySub = 'YEARLY_SUB'
}
export type StartClaudeConnectionInput = {
accountId: Scalars['ID']['input'];
};
export type StartPostHogConnectionResult = PostHogOrganizationConnection | PostHogPendingConnection;
/** Incident for a given component from Expo status page API. */
@@ -10644,8 +10834,7 @@ export type TurtleSshSessionMutation = {
export type TurtleSshSessionMutationCreateOrUpdateTurtleSshSessionArgs = {
connectionConfig: TurtleSshConnectionConfigInput;
sessionSettings: TurtleSshSessionSettingsInput;
turtleBuildId?: InputMaybe<Scalars['ID']['input']>;
turtleJobRunId?: InputMaybe<Scalars['ID']['input']>;
target: TurtleSshTargetInput;
};
export type TurtleSshSessionSettings = {
@@ -10657,6 +10846,16 @@ export type TurtleSshSessionSettingsInput = {
idleTimeoutSeconds: Scalars['Int']['input'];
};
export type TurtleSshTargetInput = {
id: Scalars['ID']['input'];
type: TurtleSshTargetType;
};
export enum TurtleSshTargetType {
Build = 'BUILD',
JobRun = 'JOB_RUN'
}
export enum TurtleSshTransportType {
UptermV1 = 'UPTERM_V1'
}
@@ -10728,6 +10927,10 @@ export type UpdateDiffReceiptsArgs = {
orderBy?: InputMaybe<Array<UpdateDiffReceiptOrderByClause>>;
};
export type UpdateAgentProviderConnectionInput = {
name?: InputMaybe<Scalars['String']['input']>;
};
export type UpdateBranch = {
__typename?: 'UpdateBranch';
app: App;
@@ -12053,6 +12256,11 @@ export type WebNotificationUpdateReadStateInput = {
isRead: Scalars['Boolean']['input'];
};
export type WebPreviewOnlyRunSessionRemoteConfig = {
__typename?: 'WebPreviewOnlyRunSessionRemoteConfig';
webPreviewUrl: Scalars['String']['output'];
};
export type Webhook = {
__typename?: 'Webhook';
appId: Scalars['ID']['output'];
@@ -13277,6 +13485,11 @@ export type WorkflowRun = ActivityTimelineProjectActivity & {
__typename?: 'WorkflowRun';
activityTimestamp: Scalars['DateTime']['output'];
actor?: Maybe<Actor>;
/**
* Why the server canceled this run. Null for manually canceled runs and for
* runs canceled before the reason was recorded.
*/
cancelReason?: Maybe<WorkflowRunCancelReason>;
createdAt: Scalars['DateTime']['output'];
durationSeconds?: Maybe<Scalars['Int']['output']>;
errors: Array<WorkflowRunError>;
@@ -13303,6 +13516,17 @@ export type WorkflowRun = ActivityTimelineProjectActivity & {
workflowRevision?: Maybe<WorkflowRevision>;
};
export type WorkflowRunCancelReason = WorkflowRunCancelReasonConcurrencyGroup;
export type WorkflowRunCancelReasonConcurrencyGroup = {
__typename?: 'WorkflowRunCancelReasonConcurrencyGroup';
/**
* The newer run in the same concurrency group whose creation canceled this run.
* Null if that run no longer exists.
*/
cancelingWorkflowRun?: Maybe<WorkflowRun>;
};
export type WorkflowRunEdge = {
__typename?: 'WorkflowRunEdge';
cursor: Scalars['String']['output'];
@@ -13392,7 +13616,8 @@ export enum WorkflowRunStatus {
Failure = 'FAILURE',
InProgress = 'IN_PROGRESS',
New = 'NEW',
Success = 'SUCCESS'
Success = 'SUCCESS',
Waiting = 'WAITING'
}
export type WorkflowRunTimeRangeInput = {
@@ -13408,6 +13633,9 @@ export enum WorkflowRunTriggerEventType {
EasSubmit = 'EAS_SUBMIT',
ExpoLaunch = 'EXPO_LAUNCH',
GithubPullRequestBaseRefChanged = 'GITHUB_PULL_REQUEST_BASE_REF_CHANGED',
GithubPullRequestCommentCreated = 'GITHUB_PULL_REQUEST_COMMENT_CREATED',
GithubPullRequestCommentDeleted = 'GITHUB_PULL_REQUEST_COMMENT_DELETED',
GithubPullRequestCommentEdited = 'GITHUB_PULL_REQUEST_COMMENT_EDITED',
GithubPullRequestEdited = 'GITHUB_PULL_REQUEST_EDITED',
GithubPullRequestLabeled = 'GITHUB_PULL_REQUEST_LABELED',
GithubPullRequestOpened = 'GITHUB_PULL_REQUEST_OPENED',
@@ -14478,6 +14706,19 @@ export type SetRolloutPercentageMutation = { __typename?: 'RootMutation', update
| { __typename: 'User', username: string, id: string }
| null, branch: { __typename?: 'UpdateBranch', id: string, name: string }, codeSigningInfo?: { __typename?: 'CodeSigningInfo', keyid: string, sig: string, alg: string } | null, rolloutControlUpdate?: { __typename?: 'Update', id: string, group: string } | null, fingerprint?: { __typename?: 'Fingerprint', id: string, hash: string, debugInfoUrl?: string | null, source?: { __typename?: 'FingerprintSource', type: FingerprintSourceType, bucketKey: string, isDebugFingerprint?: boolean | null } | null } | null } } };
export type GenerateRealtimeLogsCentrifugoConnectionTokenMutationVariables = Exact<{ [key: string]: never; }>;
export type GenerateRealtimeLogsCentrifugoConnectionTokenMutation = { __typename?: 'RootMutation', realtimeLogs: { __typename?: 'RealtimeLogsMutation', generateCentrifugoConnectionToken: { __typename?: 'RealtimeLogsCentrifugoConnectionToken', token: string } } };
export type GenerateRealtimeLogsCentrifugoSubscriptionTokenMutationVariables = Exact<{
target: RealtimeLogsTargetInput;
thread?: InputMaybe<Scalars['String']['input']>;
}>;
export type GenerateRealtimeLogsCentrifugoSubscriptionTokenMutation = { __typename?: 'RootMutation', realtimeLogs: { __typename?: 'RealtimeLogsMutation', generateCentrifugoSubscriptionToken: { __typename?: 'RealtimeLogsCentrifugoSubscriptionToken', channel: string, token: string } } };
export type CreateAndroidSubmissionMutationVariables = Exact<{
appId: Scalars['ID']['input'];
config: AndroidSubmissionConfigInput;
@@ -15033,6 +15274,7 @@ export type DeviceRunSessionByIdQuery = { __typename?: 'RootQuery', deviceRunSes
| { __typename: 'AppiumRunSessionRemoteConfig', appiumUrl: string, capabilities: any, webPreviewUrl?: string | null }
| { __typename: 'ArgentRunSessionRemoteConfig', toolsUrl: string, toolsAuthToken?: string | null, webPreviewUrl?: string | null }
| { __typename: 'ServeSimRunSessionRemoteConfig', previewUrl: string }
| { __typename: 'WebPreviewOnlyRunSessionRemoteConfig', previewUrl: string }
| null, turtleJobRun?: { __typename?: 'JobRun', id: string, status: JobRunStatus } | null } } };
export type DeviceRunSessionsByAppIdQueryVariables = Exact<{
@@ -0,0 +1,69 @@
import gql from 'graphql-tag';
import { ExpoGraphqlClient } from '../../commandUtils/context/contextUtils/createGraphqlClient';
import { withErrorHandlingAsync } from '../client';
import {
GenerateRealtimeLogsCentrifugoConnectionTokenMutation,
GenerateRealtimeLogsCentrifugoConnectionTokenMutationVariables,
GenerateRealtimeLogsCentrifugoSubscriptionTokenMutation,
GenerateRealtimeLogsCentrifugoSubscriptionTokenMutationVariables,
RealtimeLogsCentrifugoConnectionToken,
RealtimeLogsCentrifugoSubscriptionToken,
RealtimeLogsMutationGenerateCentrifugoSubscriptionTokenArgs,
} from '../generated';
export const RealtimeLogsMutation = {
async generateCentrifugoConnectionTokenAsync(
graphqlClient: ExpoGraphqlClient
): Promise<RealtimeLogsCentrifugoConnectionToken> {
const data = await withErrorHandlingAsync(
graphqlClient
.mutation<
GenerateRealtimeLogsCentrifugoConnectionTokenMutation,
GenerateRealtimeLogsCentrifugoConnectionTokenMutationVariables
>(
gql`
mutation GenerateRealtimeLogsCentrifugoConnectionToken {
realtimeLogs {
generateCentrifugoConnectionToken {
token
}
}
}
`,
{}
)
.toPromise()
);
return data.realtimeLogs.generateCentrifugoConnectionToken;
},
async generateCentrifugoSubscriptionTokenAsync(
graphqlClient: ExpoGraphqlClient,
{ target, thread }: RealtimeLogsMutationGenerateCentrifugoSubscriptionTokenArgs
): Promise<RealtimeLogsCentrifugoSubscriptionToken> {
const data = await withErrorHandlingAsync(
graphqlClient
.mutation<
GenerateRealtimeLogsCentrifugoSubscriptionTokenMutation,
GenerateRealtimeLogsCentrifugoSubscriptionTokenMutationVariables
>(
gql`
mutation GenerateRealtimeLogsCentrifugoSubscriptionToken(
$target: RealtimeLogsTargetInput!
$thread: String
) {
realtimeLogs {
generateCentrifugoSubscriptionToken(target: $target, thread: $thread) {
channel
token
}
}
}
`,
{ target, thread }
)
.toPromise()
);
return data.realtimeLogs.generateCentrifugoSubscriptionToken;
},
};
@@ -100,6 +100,9 @@ export const DeviceRunSessionQuery = {
... on ServeSimRunSessionRemoteConfig {
previewUrl
}
... on WebPreviewOnlyRunSessionRemoteConfig {
previewUrl: webPreviewUrl
}
}
turtleJobRun {
id
-109
View File
@@ -1,109 +0,0 @@
import spawnAsync from '@expo/spawn-async';
import { runCommandAsync } from './runCommand';
import Log from '../log';
import { confirmAsync, promptAsync } from '../prompts';
export async function runGitCloneAsync({
targetProjectDir,
githubRepositoryName,
githubUsername,
cloneMethod,
showOutput = true,
}: {
githubUsername: string;
githubRepositoryName: string;
targetProjectDir: string;
cloneMethod: 'ssh' | 'https';
showOutput?: boolean;
}): Promise<{
targetProjectDir: string;
}> {
const url =
cloneMethod === 'ssh'
? `[email protected]:${githubUsername}/${githubRepositoryName}.git`
: `https://github.com/${githubUsername}/${githubRepositoryName}.git`;
try {
await runCommandAsync({
command: 'git',
args: ['clone', url, targetProjectDir],
shouldPrintStderrLineAsStdout: line => {
return line.includes('Cloning into');
},
showSpinner: false,
showOutput,
});
return { targetProjectDir };
} catch (error: any) {
if (error.stderr.includes('already exists')) {
Log.warn(`Directory ${targetProjectDir} already exists.`);
const shouldContinue = await confirmAsync({
message: 'Do you want to clone your project to some other destination?',
});
if (!shouldContinue) {
throw new Error('Directory already exists. Aborting...');
}
const { newTargetProjectDir } = await promptAsync({
type: 'text',
name: 'newTargetProjectDir',
message: 'New target directory path:',
validate: (input: string) => input !== '',
});
return await runGitCloneAsync({
githubRepositoryName,
githubUsername,
targetProjectDir: newTargetProjectDir,
cloneMethod,
});
} else if (error.stderr.includes('Permission denied')) {
Log.warn(
`It seems like you do not have permission to clone the repository using ${cloneMethod}.`
);
const newMethod = cloneMethod === 'ssh' ? 'https' : 'ssh';
const shouldContinue = await confirmAsync({
message: `Do you want to clone the repository using ${newMethod} instead?`,
});
if (!shouldContinue) {
throw new Error('Permission denied. Aborting...');
}
return await runGitCloneAsync({
githubRepositoryName,
githubUsername,
targetProjectDir,
cloneMethod: newMethod,
});
} else {
throw error;
}
}
}
export async function runGitPushAsync({
targetProjectDir,
}: {
targetProjectDir: string;
}): Promise<void> {
await runCommandAsync({
command: 'git',
args: ['push'],
cwd: targetProjectDir,
});
}
export async function canAccessRepositoryUsingSshAsync({
githubUsername,
githubRepositoryName,
}: {
githubUsername: string;
githubRepositoryName: string;
}): Promise<boolean> {
try {
await spawnAsync('git', [
'ls-remote',
`[email protected]:${githubUsername}/${githubRepositoryName}.git`,
]);
return true;
} catch {
return false;
}
}
+15
View File
@@ -110,3 +110,18 @@ export function ora(options?: Options | string): Ora {
return spinner;
}
/**
* Updates the text of a spinner. Does nothing if new text is the same as old text to prevent flickering.
*/
export function updateSpinnerText(
spinner: Ora,
{ prefixText, text }: { prefixText?: string; text?: string }
): void {
if (prefixText !== undefined && spinner.prefixText !== prefixText) {
spinner.prefixText = prefixText;
}
if (text !== undefined && spinner.text !== text) {
spinner.text = text;
}
}
@@ -0,0 +1,112 @@
import { compileModsAsync } from '@expo/config-plugins';
import { getPrebuildConfigAsync } from '@expo/prebuild-config';
import { spawnExpoCommand } from '../../../utils/expoCli';
import { Client } from '../../../vcs/vcs';
import { isExpoInstalled } from '../../projectUtils';
import { hasIgnoredIosProjectAsync } from '../../workflow';
import { getManagedApplicationTargetEntitlementsAsync } from '../entitlements';
jest.mock('@expo/config-plugins', () => ({
...jest.requireActual('@expo/config-plugins'),
compileModsAsync: jest.fn(),
}));
jest.mock('@expo/prebuild-config');
jest.mock('../../../log', () => ({
__esModule: true,
default: { warn: jest.fn() },
}));
jest.mock('../../../sentry', () => ({
__esModule: true,
default: {
withScope: jest.fn(),
captureMessage: jest.fn(),
},
}));
jest.mock('../../../utils/expoCli');
jest.mock('../../projectUtils');
jest.mock('../../workflow');
const projectDir = '/project';
const vcsClient = {} as Client;
const prebuildConfig = {
name: 'test-app',
slug: 'test-app',
ios: { bundleIdentifier: 'com.example.test' },
};
const entitlements = {
'aps-environment': 'production',
'com.apple.developer.associated-domains': ['applinks:example.com'],
};
beforeEach(() => {
jest.resetAllMocks();
jest.mocked(hasIgnoredIosProjectAsync).mockResolvedValue(true);
jest.mocked(getPrebuildConfigAsync).mockResolvedValue({ exp: prebuildConfig } as any);
jest.mocked(compileModsAsync).mockResolvedValue({
...prebuildConfig,
ios: { ...prebuildConfig.ios, entitlements },
} as any);
});
it('uses the bundled config fallback when Expo is not installed', async () => {
jest.mocked(isExpoInstalled).mockReturnValue(false);
const originalProcessEnv = process.env;
let envDuringPrebuild: string | undefined;
jest.mocked(getPrebuildConfigAsync).mockImplementation(async () => {
envDuringPrebuild = process.env.TEST_ENTITLEMENTS_ENV;
return { exp: prebuildConfig } as any;
});
await expect(
getManagedApplicationTargetEntitlementsAsync(
projectDir,
{ TEST_ENTITLEMENTS_ENV: 'from-build-profile' },
vcsClient
)
).resolves.toEqual(entitlements);
expect(spawnExpoCommand).not.toHaveBeenCalled();
expect(getPrebuildConfigAsync).toHaveBeenCalledWith(projectDir, { platforms: ['ios'] });
expect(compileModsAsync).toHaveBeenCalledWith(prebuildConfig, {
projectRoot: projectDir,
platforms: ['ios'],
introspect: true,
ignoreExistingNativeFiles: true,
});
expect(envDuringPrebuild).toBe('from-build-profile');
expect(process.env).toBe(originalProcessEnv);
});
it('uses the bundled config fallback when the local Expo CLI fails', async () => {
jest.mocked(isExpoInstalled).mockReturnValue(true);
jest
.mocked(spawnExpoCommand)
.mockRejectedValue(
Object.assign(new Error('local Expo CLI failed'), { stderr: 'config failed' })
);
await expect(
getManagedApplicationTargetEntitlementsAsync(projectDir, {}, vcsClient)
).resolves.toEqual(entitlements);
expect(spawnExpoCommand).toHaveBeenCalledWith(
projectDir,
['config', '--json', '--type', 'introspect'],
{ env: expect.objectContaining({ EXPO_NO_DOTENV: '1' }) }
);
expect(getPrebuildConfigAsync).toHaveBeenCalledWith(projectDir, { platforms: ['ios'] });
});
it('does not make Expo a required peer of the bundled config fallback', () => {
const packageJson = require('@expo/prebuild-config/package.json') as {
peerDependencies?: Record<string, string>;
peerDependenciesMeta?: Record<string, { optional?: boolean }>;
};
const hasRequiredExpoPeer =
packageJson.peerDependencies?.expo !== undefined &&
packageJson.peerDependenciesMeta?.expo?.optional !== true;
expect(hasRequiredExpoPeer).toBe(false);
});
@@ -1,6 +1,9 @@
import { DeviceRunSessionType } from '../../graphql/generated';
import { DeviceRunSessionResourceClass, DeviceRunSessionType } from '../../graphql/generated';
import {
DEVICE_RUN_SESSION_RESOURCE_CLASS_BY_FLAG_VALUE,
DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE,
DEVICE_RUN_SESSION_TYPE_FLAG_VALUES,
deviceRunSessionTypeToFlagValue,
formatRemoteSessionInstructions,
getRemoteSessionEnvironmentVariables,
} from '../utils';
@@ -34,6 +37,39 @@ describe('Appium simulator configuration', () => {
expect(instructions).toContain('eas simulator:exec <appium-client> [args...]');
expect(instructions).toContain('https://preview.example.test');
expect(instructions).toContain('Open the simulator preview:');
expect(instructions).not.toContain('iOS simulator preview');
expect(instructions).not.toContain('https://appium.example.test');
});
});
describe('simulator session type flags', () => {
it('maps web-preview-only to the WebPreviewOnly GraphQL enum', () => {
expect(DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE['web-preview-only']).toBe(
DeviceRunSessionType.WebPreviewOnly
);
expect(DEVICE_RUN_SESSION_TYPE_FLAG_VALUES[DeviceRunSessionType.ServeSim]).toBe(
'web-preview-only'
);
expect(DEVICE_RUN_SESSION_TYPE_FLAG_VALUES[DeviceRunSessionType.WebPreviewOnly]).toBe(
'web-preview-only'
);
expect(deviceRunSessionTypeToFlagValue(DeviceRunSessionType.ServeSim)).toBe('web-preview-only');
expect(deviceRunSessionTypeToFlagValue(DeviceRunSessionType.WebPreviewOnly)).toBe(
'web-preview-only'
);
expect(DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE['serve-sim']).toBeUndefined();
expect(DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE['web-preview']).toBeUndefined();
});
});
describe('simulator resource class flags', () => {
it('maps CLI values to the GraphQL enum', () => {
expect(DEVICE_RUN_SESSION_RESOURCE_CLASS_BY_FLAG_VALUE.large).toBe(
DeviceRunSessionResourceClass.Large
);
expect(DEVICE_RUN_SESSION_RESOURCE_CLASS_BY_FLAG_VALUE.medium).toBe(
DeviceRunSessionResourceClass.Medium
);
});
});
+32 -3
View File
@@ -1,4 +1,8 @@
import { DeviceRunSessionByIdQuery, DeviceRunSessionType } from '../graphql/generated';
import {
DeviceRunSessionByIdQuery,
DeviceRunSessionResourceClass,
DeviceRunSessionType,
} from '../graphql/generated';
import { link } from '../log';
type DeviceRunSessionByIdResult = DeviceRunSessionByIdQuery['deviceRunSessions']['byId'];
@@ -24,7 +28,8 @@ export const DEVICE_RUN_SESSION_TYPE_FLAG_VALUES: Record<DeviceRunSessionType, s
[DeviceRunSessionType.AgentDevice]: 'agent-device',
[DeviceRunSessionType.Appium]: 'appium',
[DeviceRunSessionType.Argent]: 'argent',
[DeviceRunSessionType.ServeSim]: 'serve-sim',
[DeviceRunSessionType.ServeSim]: 'web-preview-only',
[DeviceRunSessionType.WebPreviewOnly]: 'web-preview-only',
};
export const DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE = Object.fromEntries(
@@ -37,6 +42,23 @@ export function deviceRunSessionTypeToFlagValue(type: DeviceRunSessionType): str
return DEVICE_RUN_SESSION_TYPE_FLAG_VALUES[type];
}
export const DEVICE_RUN_SESSION_RESOURCE_CLASS_FLAG_VALUES: Record<
DeviceRunSessionResourceClass,
string
> = {
[DeviceRunSessionResourceClass.Large]: 'large',
[DeviceRunSessionResourceClass.Medium]: 'medium',
};
export const DEVICE_RUN_SESSION_RESOURCE_CLASS_BY_FLAG_VALUE = Object.fromEntries(
(
Object.entries(DEVICE_RUN_SESSION_RESOURCE_CLASS_FLAG_VALUES) as [
DeviceRunSessionResourceClass,
string,
][]
).map(([resourceClass, value]) => [value, resourceClass])
) as Record<string, DeviceRunSessionResourceClass>;
export function getRemoteSessionEnvironmentVariables(
remoteConfig: DeviceRunSessionRemoteConfig
): Record<string, string> {
@@ -57,6 +79,7 @@ export function getRemoteSessionEnvironmentVariables(
APPIUM_CAPS: JSON.stringify(remoteConfig.capabilities),
};
case 'ServeSimRunSessionRemoteConfig':
case 'WebPreviewOnlyRunSessionRemoteConfig':
return {};
}
}
@@ -151,7 +174,7 @@ export function formatRemoteSessionInstructions(
'<appium-client> [args...]',
];
if (remoteConfig.webPreviewUrl) {
lines.push('', 'Open the iOS simulator preview:', '', remoteConfig.webPreviewUrl);
lines.push('', 'Open the simulator preview:', '', remoteConfig.webPreviewUrl);
}
return lines.join('\n');
}
@@ -161,5 +184,11 @@ export function formatRemoteSessionInstructions(
'',
remoteConfig.previewUrl,
].join('\n');
case 'WebPreviewOnlyRunSessionRemoteConfig':
return [
'🌐 Open the following URL in your browser to access the simulator:',
'',
remoteConfig.previewUrl,
].join('\n');
}
}
@@ -0,0 +1,26 @@
import { Centrifuge } from 'centrifuge';
import { createRealtimeLogsClient } from '../centrifuge';
jest.mock('centrifuge');
describe(createRealtimeLogsClient, () => {
afterEach(() => {
jest.clearAllMocks();
});
it('returns null instead of throwing when the connection cannot be created', () => {
jest.mocked(Centrifuge).mockImplementationOnce(() => {
throw new Error('unsupported url scheme');
});
expect(createRealtimeLogsClient({} as any)).toBeNull();
});
it('connects and returns a client when the connection can be created', () => {
expect(createRealtimeLogsClient({} as any)).not.toBeNull();
const centrifuge = jest.mocked(Centrifuge).mock.instances[0];
expect(jest.mocked(centrifuge.connect)).toHaveBeenCalled();
});
});
+117
View File
@@ -0,0 +1,117 @@
import { Centrifuge, Subscription } from 'centrifuge';
import { Agent } from 'https';
import WebSocket from 'ws';
import { getEASLogsWebsocketUrl } from '../api';
import { ExpoGraphqlClient } from '../commandUtils/context/contextUtils/createGraphqlClient';
import { RealtimeLogsMutationGenerateCentrifugoSubscriptionTokenArgs } from '../graphql/generated';
import { RealtimeLogsMutation } from '../graphql/mutations/RealtimeLogsMutation';
import { httpsProxyAgent } from '../fetch';
import Log from '../log';
export type RealtimeLogsSubscription = {
close: () => void;
};
export type RealtimeLogsClient = {
subscribeAsync: (
args: RealtimeLogsMutationGenerateCentrifugoSubscriptionTokenArgs,
onPublication: (data: unknown) => void
) => Promise<RealtimeLogsSubscription | null>;
close: () => void;
};
type WebSocketConstructor = new (address: string, protocols?: string | string[]) => WebSocket;
function createProxiedWebSocketConstructor(agent: Agent): WebSocketConstructor {
return class ProxiedWebSocket extends WebSocket {
constructor(address: string | URL, protocols?: string | string[]) {
super(address, protocols, { agent });
}
};
}
function createWebSocketConstructor(): WebSocketConstructor {
return httpsProxyAgent ? createProxiedWebSocketConstructor(httpsProxyAgent) : WebSocket;
}
export function createRealtimeLogsClient(
graphqlClient: ExpoGraphqlClient
): RealtimeLogsClient | null {
const getTokenAsync = async (): Promise<string> => {
const { token } =
await RealtimeLogsMutation.generateCentrifugoConnectionTokenAsync(graphqlClient);
return token;
};
let client: Centrifuge;
try {
client = new Centrifuge(getEASLogsWebsocketUrl(), {
websocket: createWebSocketConstructor(),
getToken: getTokenAsync,
});
client.on('error', ({ error }) => {
Log.debug(`Realtime logs connection error: ${error.message}`);
});
client.connect();
} catch (err: any) {
Log.debug(`Failed to connect to realtime logs: ${err.message}`);
return null;
}
return {
subscribeAsync: async (args, onPublication) => {
const getSubscriptionTokenAsync = async (): Promise<string> => {
const { token } = await RealtimeLogsMutation.generateCentrifugoSubscriptionTokenAsync(
graphqlClient,
args
);
return token;
};
try {
const { channel, token } =
await RealtimeLogsMutation.generateCentrifugoSubscriptionTokenAsync(graphqlClient, args);
const subscription = client.newSubscription(channel, {
token,
getToken: getSubscriptionTokenAsync,
});
subscription.on('error', ({ error }) => {
Log.debug(`Realtime logs subscription error on ${channel}: ${error.message}`);
});
subscription.on('publication', ({ data }) => {
onPublication(data);
});
subscription.subscribe();
return {
close: () => {
closeSubscription(client, subscription);
},
};
} catch (err: any) {
Log.debug(`Failed to subscribe to realtime logs: ${err.message}`);
return null;
}
},
close: () => {
for (const subscription of Object.values(client.subscriptions())) {
closeSubscription(client, subscription);
}
client.removeAllListeners();
client.disconnect();
},
};
}
function closeSubscription(client: Centrifuge, subscription: Subscription): void {
try {
subscription.unsubscribe();
subscription.removeAllListeners();
client.removeSubscription(subscription);
} catch (err: any) {
Log.debug(`Failed to close realtime logs subscription: ${err.message}`);
}
}
@@ -18,4 +18,12 @@ describe(uniqBy, () => {
{ a: 4, b: 12 },
]);
});
it('calls getKey once per item', () => {
const getKey = jest.fn(({ a }: { a: number }) => a);
uniqBy([{ a: 1 }, { a: 2 }, { a: 2 }], getKey);
expect(getKey).toHaveBeenCalledTimes(3);
});
});
@@ -1,10 +1,11 @@
export default function uniqBy<T, K = any>(list: T[], getKey: (item: T) => K): T[] {
const uniqueValues = new Set();
const uniqueValues = new Set<K>();
const result: T[] = [];
for (const i of list) {
if (!uniqueValues.has(getKey(i))) {
const key = getKey(i);
if (!uniqueValues.has(key)) {
result.push(i);
uniqueValues.add(getKey(i));
uniqueValues.add(key);
}
}
return result;
@@ -0,0 +1,70 @@
import fs from 'node:fs';
import { Readable } from 'node:stream';
import { extract } from 'tar-stream';
import zlib from 'node:zlib';
import { FileEntry, packFilesIterableAsync } from '../assets';
async function extractTarEntriesAsync(tarData: Buffer): Promise<Map<string, string>> {
const entries = new Map<string, string>();
const extractor = extract();
extractor.on('entry', (header, stream, next) => {
const chunks: Buffer[] = [];
stream.on('data', chunk => chunks.push(chunk));
stream.on('end', () => {
entries.set(header.name, Buffer.concat(chunks).toString('utf8'));
next();
});
stream.resume();
});
await new Promise<void>((resolve, reject) => {
extractor.on('finish', resolve);
extractor.on('error', reject);
Readable.from(tarData).pipe(extractor);
});
return entries;
}
describe(packFilesIterableAsync, () => {
let tarPath: string | null = null;
afterEach(async () => {
if (tarPath) {
await fs.promises.rm(tarPath, { force: true });
tarPath = null;
}
});
it('packs file entries into a gzip-compressed tarball', async () => {
const files: FileEntry[] = [
['index.html', '<h1>hello</h1>'],
['assets/data.json', '{"key":"value"}'],
];
tarPath = await packFilesIterableAsync(files);
const compressed = await fs.promises.readFile(tarPath);
// gzip magic bytes
expect(compressed[0]).toBe(0x1f);
expect(compressed[1]).toBe(0x8b);
// portable mode zeroes the gzip MTIME field (bytes 4-7)
expect(compressed.subarray(4, 8)).toEqual(Buffer.from([0, 0, 0, 0]));
const entries = await extractTarEntriesAsync(zlib.gunzipSync(compressed));
expect(entries.get('index.html')).toBe('<h1>hello</h1>');
expect(entries.get('assets/data.json')).toBe('{"key":"value"}');
expect(entries.size).toBe(2);
});
it('accepts async iterables and forwards gzip options', async () => {
async function* files(): AsyncIterable<FileEntry> {
yield ['a.txt', 'aaaa'];
yield ['b.txt', Buffer.from('bbbb')];
}
tarPath = await packFilesIterableAsync(files(), { level: 9 });
const compressed = await fs.promises.readFile(tarPath);
const entries = await extractTarEntriesAsync(zlib.gunzipSync(compressed));
expect(entries.get('a.txt')).toBe('aaaa');
expect(entries.get('b.txt')).toBe('bbbb');
});
});
@@ -0,0 +1,443 @@
import cliProgress from 'cli-progress';
import fetch, { Headers, Response } from 'node-fetch';
import Log from '../../log';
import { AssetFileEntry } from '../assets';
import {
UploadPayload,
batchUploadAsync,
callUploadApiAsync,
createProgressBar,
uploadAsync,
} from '../upload';
jest.mock('node-fetch', () => ({
...jest.requireActual('node-fetch'),
__esModule: true,
default: jest.fn(),
}));
const mockedFetch = jest.mocked(fetch);
const asset: AssetFileEntry = {
normalizedPath: 'index.html',
path: __filename,
size: 123,
sha512: 'a'.repeat(128),
type: 'text/html',
};
const response = (status = 200, body: unknown = { success: true }): Response =>
new Response(JSON.stringify(body), {
status,
statusText: status === 200 ? 'OK' : 'Request failed',
headers: { 'content-type': 'application/json' },
});
async function runTimersAndExpectRejection(
promise: Promise<unknown>,
expected: string | RegExp
): Promise<void> {
const expectation = expect(promise).rejects.toThrow(expected);
await jest.runAllTimersAsync();
await expectation;
}
describe(uploadAsync, () => {
beforeEach(() => {
jest.useFakeTimers({ now: 0 });
jest.spyOn(Math, 'random').mockReturnValue(0);
jest.spyOn(Log, 'warn').mockImplementation(() => {});
mockedFetch.mockReset();
});
afterEach(() => {
jest.useRealTimers();
jest.restoreAllMocks();
});
it('uploads an individual asset with its hash and metadata', async () => {
mockedFetch.mockResolvedValue(response());
const result = await uploadAsync({ baseURL: 'https://eas.expo.app/?token=secret' }, { asset });
expect(result.payload).toEqual({ asset });
expect(mockedFetch).toHaveBeenCalledTimes(1);
const [url, init] = mockedFetch.mock.calls[0];
expect(String(url)).toBe(`https://eas.expo.app/asset/${asset.sha512}?token=secret`);
expect(init?.method).toBe('POST');
expect((init?.headers as Headers).get('content-type')).toBe('text/html');
expect((init?.headers as Headers).get('content-length')).toBe('123');
expect(init?.body).toBeDefined();
});
it('uploads a worker deployment file without changing the URL', async () => {
mockedFetch.mockResolvedValue(response());
await uploadAsync({ baseURL: 'https://eas.expo.app/deploy' }, { filePath: __filename });
const [url, init] = mockedFetch.mock.calls[0];
expect(String(url)).toBe('https://eas.expo.app/deploy');
expect(init?.method).toBe('POST');
expect(init?.body).toBeDefined();
});
it('uploads multipart assets to the batch endpoint and reports progress', async () => {
mockedFetch.mockResolvedValue(response());
const onProgress = jest.fn();
await uploadAsync(
{ baseURL: 'https://eas.expo.app/asset/?token=secret' },
{ multipart: [asset, { ...asset, sha512: 'b'.repeat(128) }] },
onProgress
);
const [url, init] = mockedFetch.mock.calls[0];
expect(String(url)).toBe('https://eas.expo.app/asset/batch?token=secret');
expect(init?.method).toBe('PATCH');
expect((init?.headers as Headers).get('content-type')).toMatch(/^multipart\/form-data;/);
expect(onProgress).toHaveBeenNthCalledWith(1, 0);
expect(onProgress).toHaveBeenLastCalledWith(1);
});
it.each([408, 409, 429, 500, 503])('retries HTTP %s responses', async status => {
mockedFetch.mockImplementation(async () => response(status, { error: `status ${status}` }));
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset });
await runTimersAndExpectRejection(promise, `status ${status}`);
expect(mockedFetch).toHaveBeenCalledTimes(5);
});
it('does not retry a terminal HTTP response', async () => {
mockedFetch.mockResolvedValue(response(400, { error: 'invalid upload' }));
await expect(uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset })).rejects.toThrow(
'invalid upload'
);
expect(mockedFetch).toHaveBeenCalledTimes(1);
});
it('reports payload-too-large responses without retrying', async () => {
mockedFetch.mockResolvedValue(response(413));
await expect(uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset })).rejects.toThrow(
'File size exceeded the upload limit'
);
expect(mockedFetch).toHaveBeenCalledTimes(1);
});
it('reports CDN HTML errors with the request ID', async () => {
mockedFetch.mockResolvedValue(
new Response('<html></html>', {
status: 403,
statusText: 'Forbidden',
headers: { 'content-type': 'text/html', 'cf-ray': 'ray-id' },
})
);
await expect(uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset })).rejects.toThrow(
'Request ID ray-id'
);
});
it('switches to network mode and preserves the original network error', async () => {
const networkError = Object.assign(new Error('getaddrinfo ENOTFOUND eas.expo.app'), {
code: 'ENOTFOUND',
});
mockedFetch.mockRejectedValue(networkError);
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset });
await runTimersAndExpectRejection(promise, networkError.message);
expect(mockedFetch).toHaveBeenCalledTimes(8);
expect(Log.warn).toHaveBeenCalledTimes(1);
expect(Log.warn).toHaveBeenCalledWith(
`The upload encountered an error but is still retrying: ${networkError.message}`
);
});
it('smoothly increases retries based on the total request count', async () => {
mockedFetch.mockImplementation(async () => response(503, { error: 'overloaded' }));
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
totalRequests: 32,
});
await runTimersAndExpectRejection(promise, 'overloaded');
// log10(32) / 3 results in six retries, plus the initial request.
expect(mockedFetch).toHaveBeenCalledTimes(7);
});
it('caps request-count scaling at 1,000 requests', async () => {
mockedFetch.mockImplementation(async () => response(503, { error: 'overloaded' }));
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
totalRequests: 1_000_000,
});
await runTimersAndExpectRejection(promise, 'overloaded');
// The scaled 40s ceiling cuts the last attempt the backoff schedule would have allowed.
expect(mockedFetch).toHaveBeenCalledTimes(8);
});
it('keeps the retry state when it switches to network mode', async () => {
let calls = 0;
mockedFetch.mockImplementation(async () => {
if (++calls <= 7) {
return response(503, { error: 'overloaded' });
}
throw new Error('socket disconnected');
});
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
totalRequests: 1_000,
});
await runTimersAndExpectRejection(promise, 'socket disconnected');
// The warning must not repeat once the network-mode attempts take over.
expect(Log.warn).toHaveBeenCalledTimes(1);
expect(Log.warn).toHaveBeenCalledWith(
'The upload encountered an error but is still retrying: overloaded'
);
});
it('warns before a short retry budget is exhausted', async () => {
mockedFetch.mockImplementation(async () => response(503, { error: 'overloaded' }));
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset });
await runTimersAndExpectRejection(promise, 'overloaded');
expect(Log.warn).toHaveBeenCalledTimes(1);
});
it('stops retrying at the scaled upload ceiling', async () => {
mockedFetch.mockImplementation(async () => response(503, { error: 'overloaded' }));
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
totalRequests: 1_000,
});
await runTimersAndExpectRejection(promise, 'overloaded');
// The ceiling stops the next retry from being scheduled, so the last attempt already in
// flight can overshoot it by up to one maxTimeout (10s at this scale).
expect(Date.now()).toBeLessThanOrEqual(40_000 + 10_000);
});
it('clears the network flag after a success', async () => {
const state = { shared: { hasSeenNetworkError: false, hasWarned: false } };
mockedFetch.mockResolvedValueOnce(response(503)).mockResolvedValueOnce(response());
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
state,
});
await jest.runAllTimersAsync();
await promise;
expect(state.shared.hasSeenNetworkError).toBe(false);
});
it('keeps retry warning clocks independent when retry state is shared', async () => {
const shared = { hasSeenNetworkError: false, hasWarned: false };
mockedFetch
.mockResolvedValueOnce(response(503, { error: 'overloaded' }))
.mockResolvedValueOnce(response())
.mockImplementation(async () => response(503, { error: 'overloaded' }));
const retryingUpload = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
state: { shared },
});
await uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
state: { shared },
});
await runTimersAndExpectRejection(retryingUpload, 'overloaded');
expect(Log.warn).toHaveBeenCalledTimes(1);
});
it('does not retry or enter network mode after cancellation', async () => {
const controller = new AbortController();
controller.abort();
const abortError = Object.assign(new Error('The operation was aborted'), {
name: 'AbortError',
});
mockedFetch.mockRejectedValue(abortError);
await expect(
uploadAsync({ baseURL: 'https://eas.expo.app', signal: controller.signal }, { asset })
).rejects.toBe(abortError);
expect(mockedFetch).toHaveBeenCalledTimes(1);
expect(Log.warn).not.toHaveBeenCalled();
});
});
describe(callUploadApiAsync, () => {
beforeEach(() => {
jest.useFakeTimers({ now: 0 });
jest.spyOn(Math, 'random').mockReturnValue(0);
jest.spyOn(Log, 'warn').mockImplementation(() => {});
mockedFetch.mockReset();
});
afterEach(() => {
jest.useRealTimers();
jest.restoreAllMocks();
});
it('returns a successful JSON response', async () => {
mockedFetch.mockResolvedValue(response(200, { result: 'complete' }));
await expect(callUploadApiAsync('https://eas.expo.app/finalize')).resolves.toEqual({
result: 'complete',
});
});
it('retries server errors and passes through the final error', async () => {
mockedFetch.mockResolvedValue(response(503));
const promise = callUploadApiAsync('https://eas.expo.app/finalize');
await runTimersAndExpectRejection(promise, 'Deployment failed: Request failed');
// Deploy API calls get a somewhat larger retry budget than a single asset upload.
expect(mockedFetch).toHaveBeenCalledTimes(7);
});
it('retries invalid JSON responses', async () => {
mockedFetch
.mockResolvedValueOnce(new Response('invalid JSON'))
.mockResolvedValueOnce(response(200, { success: true }));
const promise = callUploadApiAsync('https://eas.expo.app/finalize');
await jest.runAllTimersAsync();
await expect(promise).resolves.toEqual({ success: true });
expect(mockedFetch).toHaveBeenCalledTimes(2);
});
it('uses network retries for fetch errors', async () => {
const networkError = new Error('socket disconnected');
mockedFetch.mockRejectedValue(networkError);
const promise = callUploadApiAsync('https://eas.expo.app/finalize');
await runTimersAndExpectRejection(promise, networkError.message);
// Network mode adds attempts but shares the 30s deadline, so it cannot double the wait.
expect(mockedFetch).toHaveBeenCalledTimes(10);
});
it('stops retrying at the 30s ceiling', async () => {
mockedFetch.mockResolvedValue(response(503));
const promise = callUploadApiAsync('https://eas.expo.app/finalize');
await runTimersAndExpectRejection(promise, 'Deployment failed: Request failed');
expect(Date.now()).toBeLessThanOrEqual(30_000);
});
it('shares the ceiling with the network-mode attempts', async () => {
mockedFetch.mockRejectedValue(new Error('socket disconnected'));
const promise = callUploadApiAsync('https://eas.expo.app/finalize');
await runTimersAndExpectRejection(promise, 'socket disconnected');
expect(Date.now()).toBeLessThanOrEqual(30_000 + 5_000);
});
it('does not retry an aborted request', async () => {
const controller = new AbortController();
controller.abort();
const abortError = Object.assign(new Error('The operation was aborted'), {
name: 'AbortError',
});
mockedFetch.mockRejectedValue(abortError);
await expect(
callUploadApiAsync('https://eas.expo.app/finalize', { signal: controller.signal as any })
).rejects.toBe(abortError);
expect(mockedFetch).toHaveBeenCalledTimes(1);
});
});
describe(batchUploadAsync, () => {
beforeEach(() => {
mockedFetch.mockReset();
});
afterEach(() => {
jest.restoreAllMocks();
});
it('uploads every payload and reports aggregate progress', async () => {
mockedFetch.mockResolvedValue(response());
const payloads: UploadPayload[] = [
{ asset },
{ asset: { ...asset, sha512: 'b'.repeat(128) } },
{ asset: { ...asset, sha512: 'c'.repeat(128) } },
];
const progress = jest.fn();
const results = [];
for await (const result of batchUploadAsync(
{ baseURL: 'https://eas.expo.app' },
payloads,
progress
)) {
results.push(result);
}
expect(mockedFetch).toHaveBeenCalledTimes(3);
expect(results.at(-1)?.progress).toBe(1);
expect(progress).toHaveBeenLastCalledWith(1);
});
it('accepts an empty upload queue', async () => {
const results = [];
for await (const result of batchUploadAsync({ baseURL: 'https://eas.expo.app' }, [])) {
results.push(result);
}
expect(results).toEqual([]);
expect(mockedFetch).not.toHaveBeenCalled();
});
it('aborts sibling requests after the first upload failure', async () => {
mockedFetch.mockImplementation(async (url, init) => {
if (String(url).includes(asset.sha512)) {
return response(400, { error: 'terminal failure' });
}
return await new Promise((_resolve, reject) => {
init?.signal?.addEventListener('abort', () => {
reject(new Error('aborted'));
});
});
});
const iterator = batchUploadAsync({ baseURL: 'https://eas.expo.app' }, [
{ asset },
{ asset: { ...asset, sha512: 'b'.repeat(128) } },
]);
await iterator.next();
await iterator.next();
await expect(iterator.next()).rejects.toThrow('terminal failure');
});
});
describe(createProgressBar, () => {
afterEach(() => {
jest.restoreAllMocks();
});
it('starts, updates, and stops the upload progress bar', () => {
const start = jest.spyOn(cliProgress.SingleBar.prototype, 'start').mockImplementation(() => {});
const update = jest
.spyOn(cliProgress.SingleBar.prototype, 'update')
.mockImplementation(() => {});
const stop = jest.spyOn(cliProgress.SingleBar.prototype, 'stop').mockImplementation(() => {});
const progressBar = createProgressBar('Uploading 3 assets');
progressBar.update(0.5);
progressBar.stop();
expect(start).toHaveBeenCalledWith(1, 0);
expect(update).toHaveBeenCalledWith(0.5);
expect(stop).toHaveBeenCalledTimes(1);
});
});
+252 -106
View File
@@ -7,6 +7,7 @@ import os from 'node:os';
import { Readable } from 'node:stream';
import promiseRetry from 'promise-retry';
import Log from '../log';
import { AssetFileEntry } from './assets';
import {
createMultipartBodyFromFilesAsync,
@@ -15,7 +16,86 @@ import {
} from './utils/multipart';
const MAX_CONCURRENCY = Math.min(10, Math.max(os.availableParallelism() * 2, 20));
const MAX_RETRIES = 4;
const MAX_RETRY_WARNING_DELAY_MS = 30_000;
const UPLOAD_RETRY_LIMITS: RetryLimits = { retries: 4, maxTimeout: 5_000, maxRetryTime: 20_000 };
const API_RETRY_LIMITS: RetryLimits = { retries: 6, maxTimeout: 5_000, maxRetryTime: 30_000 };
interface SharedRetryState {
hasSeenNetworkError: boolean;
hasWarned: boolean;
}
interface RetryState {
shared: SharedRetryState;
firstRetryAt?: number;
}
interface UploadRetryOptions {
totalRequests?: number;
state?: RetryState;
deadlineAt?: number;
}
interface RetryLimits {
retries: number;
maxTimeout: number;
maxRetryTime: number;
}
interface RetryOptions {
retries: number;
factor: number;
minTimeout: number;
maxTimeout: number;
maxRetryTime: number;
randomize: boolean;
}
interface RetryWarningContext {
state: RetryState;
warningDelayMs: number;
subject: string;
}
const getRetryScale = (totalRequests: number): number =>
Math.min(1, Math.log10(Math.max(1, totalRequests)) / 3);
const getErrorMessage = (error: unknown): string =>
error instanceof Error ? error.message : String(error);
const getRetryOptions = (
limits: RetryLimits,
totalRequests: number,
networkMode: boolean
): RetryOptions => {
const scale = getRetryScale(totalRequests);
return {
retries: Math.round(limits.retries * (1 + scale)) + (networkMode ? 2 : 0),
factor: 2,
minTimeout: 1_000,
maxTimeout: Math.round(limits.maxTimeout * (1 + scale)),
maxRetryTime: Math.round(limits.maxRetryTime * (1 + scale)),
randomize: true,
};
};
const getRetryWarningDelay = (maxRetryTime: number): number =>
Math.min(MAX_RETRY_WARNING_DELAY_MS, Math.round(maxRetryTime / 2));
const retryWithWarning = (
retry: (error: unknown) => never,
error: unknown,
attempt: number,
{ state, warningDelayMs, subject }: RetryWarningContext
): never => {
state.firstRetryAt ??= Date.now();
if (!state.shared.hasWarned && attempt > 1 && Date.now() - state.firstRetryAt >= warningDelayMs) {
state.shared.hasWarned = true;
Log.warn(`${subject} encountered an error but is still retrying: ${getErrorMessage(error)}`);
}
return retry(error);
};
export type UploadPayload =
| { filePath: string }
@@ -56,110 +136,156 @@ type OnProgressUpdateCallback = (progress: number) => void;
export async function uploadAsync(
init: UploadRequestInit,
payload: UploadPayload,
onProgressUpdate?: OnProgressUpdateCallback
onProgressUpdate?: OnProgressUpdateCallback,
retryOptions: UploadRetryOptions = {},
networkMode = retryOptions.state?.shared.hasSeenNetworkError ?? false
): Promise<UploadResult> {
return await promiseRetry(
async retry => {
if (onProgressUpdate) {
onProgressUpdate(0);
}
const headers = new Headers(init.headers);
const url = new URL(`${init.baseURL}`);
let errorPrefix: string;
let body: BodyInit | undefined;
let method = init.method || 'POST';
if ('asset' in payload) {
const { asset } = payload;
errorPrefix = `Upload of "${asset.normalizedPath}" failed`;
if (asset.type) {
headers.set('content-type', asset.type);
}
if (asset.size) {
headers.set('content-length', `${asset.size}`);
}
method = 'POST';
url.pathname = `/asset/${asset.sha512}`;
body = Readable.from(createReadStreamAsync(asset), { objectMode: false });
} else if ('filePath' in payload) {
const { filePath } = payload;
errorPrefix = 'Worker deployment failed';
body = fs.createReadStream(filePath);
} else if ('multipart' in payload) {
const { multipart } = payload;
errorPrefix = `Upload of ${multipart.length} assets failed`;
headers.set('content-type', multipartContentType);
method = 'PATCH';
url.pathname = '/asset/batch';
body = Readable.from(createMultipartBodyFromFilesAsync(multipart, onProgressUpdate), {
objectMode: false,
});
}
let response: Response;
try {
response = await fetch(url, {
method,
body,
headers,
agent: getAgent(),
signal: init.signal as any,
});
} catch (error) {
return retry(error);
}
const getErrorMessageAsync = async (): Promise<string> => {
const rayId = response.headers.get('cf-ray');
const contentType = response.headers.get('Content-Type');
if (contentType?.startsWith('text/html')) {
// NOTE(@kitten): We've received a CDN error most likely. There's not much we can do
// except for quoting the Request ID, so a user can send it to us. We can check
// why a request was blocked by looking up a WAF event via the "Ray ID" here:
// https://dash.cloudflare.com/e6f39f67f543faa6038768e8f37e4234/expo.app/security/events
let message = `CDN firewall has aborted the upload with ${response.statusText}.`;
if (rayId) {
message += `\nReport this error quoting Request ID ${rayId}`;
}
return `${errorPrefix}: ${message}`;
} else {
const json = await response.json().catch(() => null);
return json?.error ?? `${errorPrefix}: ${response.statusText}`;
}
};
if (
response.status === 408 ||
response.status === 409 ||
response.status === 429 ||
(response.status >= 500 && response.status <= 599)
) {
return retry(new Error(await getErrorMessageAsync()));
} else if (response.status === 413) {
const message = `${errorPrefix!}: File size exceeded the upload limit`;
throw new Error(message);
} else if (!response.ok) {
throw new Error(await getErrorMessageAsync());
} else if (onProgressUpdate) {
onProgressUpdate(1);
}
return {
payload,
response,
};
},
{
retries: MAX_RETRIES,
minTimeout: 50,
randomize: false,
}
const state = retryOptions.state ?? {
shared: { hasSeenNetworkError: false, hasWarned: false },
};
const retryOptionsForAttempts = getRetryOptions(
UPLOAD_RETRY_LIMITS,
retryOptions.totalRequests ?? 1,
networkMode
);
const warningDelayMs = getRetryWarningDelay(retryOptionsForAttempts.maxRetryTime);
const deadlineAt = retryOptions.deadlineAt ?? Date.now() + retryOptionsForAttempts.maxRetryTime;
retryOptionsForAttempts.maxRetryTime = Math.max(0, deadlineAt - Date.now());
const warningContext: RetryWarningContext = {
state,
warningDelayMs,
subject: 'The upload',
};
return await promiseRetry(async (retry, attempt) => {
if (onProgressUpdate) {
onProgressUpdate(0);
}
const headers = new Headers(init.headers);
const url = new URL(`${init.baseURL}`);
let errorPrefix: string;
let body: BodyInit | undefined;
let method = init.method || 'POST';
if ('asset' in payload) {
const { asset } = payload;
errorPrefix = `Upload of "${asset.normalizedPath}" failed`;
if (asset.type) {
headers.set('content-type', asset.type);
}
if (asset.size) {
headers.set('content-length', `${asset.size}`);
}
method = 'POST';
url.pathname = `/asset/${asset.sha512}`;
body = Readable.from(createReadStreamAsync(asset), { objectMode: false });
} else if ('filePath' in payload) {
const { filePath } = payload;
errorPrefix = 'Worker deployment failed';
body = fs.createReadStream(filePath);
} else if ('multipart' in payload) {
const { multipart } = payload;
errorPrefix = `Upload of ${multipart.length} assets failed`;
headers.set('content-type', multipartContentType);
method = 'PATCH';
url.pathname = '/asset/batch';
body = Readable.from(createMultipartBodyFromFilesAsync(multipart, onProgressUpdate), {
objectMode: false,
});
}
let response: Response;
try {
response = await fetch(url, {
method,
body,
headers,
agent: getAgent(),
signal: init.signal as any,
});
} catch (error) {
if (init.signal?.aborted) {
throw error;
}
if (!networkMode) {
state.shared.hasSeenNetworkError = true;
return await uploadAsync(
init,
payload,
onProgressUpdate,
{ ...retryOptions, state, deadlineAt },
true
);
}
return retryWithWarning(retry, error, attempt, warningContext);
}
const getErrorMessageAsync = async (): Promise<string> => {
const rayId = response.headers.get('cf-ray');
const contentType = response.headers.get('Content-Type');
if (contentType?.startsWith('text/html')) {
// NOTE(@kitten): We've received a CDN error most likely. There's not much we can do
// except for quoting the Request ID, so a user can send it to us. We can check
// why a request was blocked by looking up a WAF event via the "Ray ID" here:
// https://dash.cloudflare.com/e6f39f67f543faa6038768e8f37e4234/expo.app/security/events
let message = `CDN firewall has aborted the upload with ${response.statusText}.`;
if (rayId) {
message += `\nReport this error quoting Request ID ${rayId}`;
}
return `${errorPrefix}: ${message}`;
} else {
const json = await response.json().catch(() => null);
return json?.error ?? `${errorPrefix}: ${response.statusText}`;
}
};
if (
response.status === 408 ||
response.status === 409 ||
response.status === 429 ||
(response.status >= 500 && response.status <= 599)
) {
return retryWithWarning(
retry,
new Error(await getErrorMessageAsync()),
attempt,
warningContext
);
} else if (response.status === 413) {
const message = `${errorPrefix!}: File size exceeded the upload limit`;
throw new Error(message);
} else if (!response.ok) {
throw new Error(await getErrorMessageAsync());
} else if (onProgressUpdate) {
onProgressUpdate(1);
}
state.shared.hasSeenNetworkError = false;
return {
payload,
response,
};
}, retryOptionsForAttempts);
}
export async function callUploadApiAsync(url: string | URL, init?: RequestInit): Promise<unknown> {
return await promiseRetry(async retry => {
async function callUploadApiWithRetryAsync(
url: string | URL,
init: RequestInit | undefined,
networkMode = false,
state: RetryState = { shared: { hasSeenNetworkError: false, hasWarned: false } },
deadlineAt?: number
): Promise<unknown> {
const retryOptions = getRetryOptions(API_RETRY_LIMITS, 1, networkMode);
const warningDelayMs = getRetryWarningDelay(retryOptions.maxRetryTime);
const retryDeadlineAt = deadlineAt ?? Date.now() + retryOptions.maxRetryTime;
retryOptions.maxRetryTime = Math.max(0, retryDeadlineAt - Date.now());
const warningContext: RetryWarningContext = {
state,
warningDelayMs,
subject: 'The deployment',
};
return await promiseRetry(async (retry, attempt) => {
let response: Response;
try {
response = await fetch(url, {
@@ -167,17 +293,33 @@ export async function callUploadApiAsync(url: string | URL, init?: RequestInit):
agent: getAgent(),
});
} catch (error) {
return retry(error);
if (init?.signal?.aborted) {
throw error;
}
if (!networkMode) {
state.shared.hasSeenNetworkError = true;
return await callUploadApiWithRetryAsync(url, init, true, state, retryDeadlineAt);
}
return retryWithWarning(retry, error, attempt, warningContext);
}
if (response.status >= 500 && response.status <= 599) {
retry(new Error(`Deployment failed: ${response.statusText}`));
retryWithWarning(
retry,
new Error(`Deployment failed: ${response.statusText}`),
attempt,
warningContext
);
}
try {
return await response.json();
} catch (error) {
retry(error);
return retryWithWarning(retry, error, attempt, warningContext);
}
});
}, retryOptions);
}
export async function callUploadApiAsync(url: string | URL, init?: RequestInit): Promise<unknown> {
return await callUploadApiWithRetryAsync(url, init);
}
export interface UploadPending {
@@ -192,6 +334,7 @@ export async function* batchUploadAsync(
): AsyncGenerator<UploadPending> {
const progressTracker = new Array(payloads.length).fill(0);
const controller = new AbortController();
const sharedRetryState: SharedRetryState = { hasSeenNetworkError: false, hasWarned: false };
const queue = new Set<Promise<UploadResult>>();
const initWithSignal = { ...init, signal: controller.signal };
const getProgressValue = (): number => {
@@ -216,7 +359,10 @@ export async function* batchUploadAsync(
progressTracker[currentIndex] = progress;
sendProgressUpdate();
});
const uploadPromise = uploadAsync(initWithSignal, payload, onChildProgressUpdate).then(
const uploadPromise = uploadAsync(initWithSignal, payload, onChildProgressUpdate, {
totalRequests: payloads.length,
state: { shared: sharedRetryState },
}).then(
result => {
queue.delete(uploadPromise);
progressTracker[currentIndex] = 1;
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/eas-json",
"version": "22.0.0",
"version": "23.1.0",
"description": "A library for interacting with eas.json",
"homepage": "https://github.com/expo/eas-cli",
"bugs": "https://github.com/expo/eas-cli/issues",
@@ -32,7 +32,7 @@
},
"dependencies": {
"@babel/code-frame": "7.23.5",
"@expo/eas-build-job": "22.0.0",
"@expo/eas-build-job": "23.1.0",
"chalk": "4.1.2",
"env-string": "1.0.1",
"fs-extra": "11.2.0",
+6 -7
View File
@@ -1,6 +1,6 @@
{
"name": "eas-cli-local-build-plugin",
"version": "22.5.0",
"version": "23.1.0",
"description": "Tool for running EAS compatible builds on a local machine.",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "BUSL-1.1",
@@ -26,12 +26,12 @@
"clean": "rimraf node_modules dist coverage \"*.tsbuildinfo\""
},
"dependencies": {
"@expo/build-tools": "22.5.0",
"@expo/eas-build-job": "22.0.0",
"@expo/logger": "22.0.0",
"@expo/build-tools": "23.1.0",
"@expo/bunyan": "^4.0.1",
"@expo/eas-build-job": "23.1.0",
"@expo/logger": "23.1.0",
"@expo/spawn-async": "^1.7.2",
"@expo/turtle-spawn": "22.0.0",
"bunyan": "^1.8.15",
"@expo/turtle-spawn": "23.1.0",
"chalk": "4.1.2",
"env-paths": "^2.2.1",
"fs-extra": "^11.2.0",
@@ -43,7 +43,6 @@
"uuid": "^9.0.1"
},
"devDependencies": {
"@types/bunyan": "^1.8.11",
"@types/fs-extra": "^11.0.4",
"@types/hapi__joi": "^17.1.14",
"@types/lodash": "^4.17.4",
+1 -1
View File
@@ -1,6 +1,6 @@
import { Artifacts, SkipNativeBuildError } from '@expo/build-tools';
import { ArchiveSourceType, BuildJob, Metadata, Platform, Workflow } from '@expo/eas-build-job';
import { DEBUG } from 'bunyan';
import { DEBUG } from '@expo/bunyan';
import chalk from 'chalk';
import fs from 'fs-extra';
import pickBy from 'lodash/pickBy';
+6 -3
View File
@@ -1,6 +1,7 @@
import { LogBuffer } from '@expo/build-tools';
import { LoggerLevel } from '@expo/logger';
import bunyan from 'bunyan';
import type { bunyan as ExpoLogger } from '@expo/logger';
import bunyan from '@expo/bunyan';
import chalk from 'chalk';
import omit from 'lodash/omit';
import { Writable } from 'stream';
@@ -129,7 +130,9 @@ class PrettyStream extends Writable {
export const logBuffer = new BuildCliLogBuffer(MAX_LINES_IN_BUFFER);
export function createLogger(level?: LoggerLevel): bunyan {
export function createLogger(level?: LoggerLevel): ExpoLogger {
// @expo/bunyan provides the Bunyan API used here, but its stricter stream types are not
// structurally compatible with the legacy @types/bunyan types exposed by @expo/logger.
return bunyan.createLogger({
name: 'eas-build-cli',
serializers: bunyan.stdSerializers,
@@ -143,5 +146,5 @@ export function createLogger(level?: LoggerLevel): bunyan {
stream: logBuffer,
},
],
});
}) as unknown as ExpoLogger;
}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/logger",
"version": "22.0.0",
"version": "23.1.0",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "BUSL-1.1",
"author": "Expo <[email protected]>",
+192
View File
@@ -0,0 +1,192 @@
import fs from 'fs';
import os from 'os';
import path from 'path';
import { Readable } from 'stream';
import bunyan from 'bunyan';
import defaultLogger, { LoggerLevel, PipeMode, createLogger, pipeSpawnOutput } from '../index';
// Collects raw bunyan records written to the logger.
function createRecordCollector(logger: ReturnType<typeof createLogger>): any[] {
const records: any[] = [];
logger.addStream({
type: 'raw',
stream: {
write(record: any) {
records.push(record);
},
} as any,
level: bunyan.TRACE,
});
return records;
}
function createSilentLogger(): ReturnType<typeof createLogger> {
const logger = createLogger({ name: 'test-logger', streams: [] });
return logger;
}
describe(createLogger, () => {
it('creates a logger that writes records with name, level, and msg', () => {
const logger = createSilentLogger();
const records = createRecordCollector(logger);
logger.info('hello world');
expect(records).toHaveLength(1);
expect(records[0].name).toBe('test-logger');
expect(records[0].level).toBe(bunyan.INFO);
expect(records[0].msg).toBe('hello world');
});
it('supports structured fields and util.format-style messages', () => {
const logger = createSilentLogger();
const records = createRecordCollector(logger);
logger.info({ phase: 'INSTALL_DEPENDENCIES' }, 'step %d done', 2);
expect(records[0].phase).toBe('INSTALL_DEPENDENCIES');
expect(records[0].msg).toBe('step 2 done');
});
it('serializes errors with the standard err serializer', () => {
const logger = createSilentLogger();
const records = createRecordCollector(logger);
logger.error(new Error('boom'));
expect(records[0].err.message).toBe('boom');
expect(records[0].err.stack).toEqual(expect.stringContaining('boom'));
expect(records[0].msg).toBe('boom');
});
it('creates child loggers that inherit fields and add their own', () => {
const logger = createSilentLogger();
const records = createRecordCollector(logger);
const child = logger.child({ source: 'stdout' });
child.warn('from child');
expect(records[0].name).toBe('test-logger');
expect(records[0].source).toBe('stdout');
expect(records[0].level).toBe(bunyan.WARN);
});
it('respects the configured level', () => {
const logger = createLogger({ name: 'test-logger', streams: [] });
const records: any[] = [];
logger.addStream({
type: 'raw',
stream: {
write(record: any) {
records.push(record);
},
} as any,
level: bunyan.INFO,
});
logger.debug('too quiet');
logger.info('loud enough');
expect(records).toHaveLength(1);
expect(records[0].msg).toBe('loud enough');
});
it('reports and sets the level programmatically', () => {
const logger = createLogger({
name: 'test-logger',
streams: [
{
type: 'raw',
stream: { write() {} } as any,
level: LoggerLevel.INFO,
},
],
});
expect(logger.level()).toBe(bunyan.INFO);
logger.level(LoggerLevel.DEBUG);
expect(logger.level()).toBe(bunyan.DEBUG);
});
it('supports rotating-file streams', async () => {
const directory = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'expo-logger-test-'));
const logPath = path.join(directory, 'rotating.log');
let rotatingStream: any;
try {
const logger = createLogger({
name: 'test-logger',
streams: [
{
type: 'rotating-file',
path: logPath,
period: '1d',
count: 1,
},
],
});
rotatingStream = (logger as any).streams[0].stream;
logger.info('written to a rotating file');
await new Promise<void>((resolve, reject) => {
rotatingStream.stream.once('error', reject);
rotatingStream.stream.end(resolve);
});
expect(await fs.promises.readFile(logPath, 'utf8')).toContain('written to a rotating file');
} finally {
clearTimeout(rotatingStream?.timeout);
rotatingStream?.stream.destroy();
await fs.promises.rm(directory, { recursive: true, force: true });
}
});
});
describe('defaultLogger', () => {
it('is configured with the expo-logger name at INFO level', () => {
expect(defaultLogger.fields.name).toBe('expo-logger');
expect(defaultLogger.level()).toBe(bunyan.INFO);
});
});
describe(pipeSpawnOutput, () => {
async function pipeAndCollect(
stdoutData: string | null,
stderrData: string | null,
options?: Parameters<typeof pipeSpawnOutput>[2]
): Promise<any[]> {
const logger = createSilentLogger();
const records = createRecordCollector(logger);
const stdout = stdoutData === null ? null : Readable.from([stdoutData]);
const stderr = stderrData === null ? null : Readable.from([stderrData]);
pipeSpawnOutput(logger, { stdout, stderr }, options);
await new Promise(resolve => setImmediate(resolve));
return records;
}
it('logs stdout and stderr lines tagged with their source', async () => {
const records = await pipeAndCollect('out line 1\nout line 2\n', 'err line\n');
const stdoutRecords = records.filter(r => r.source === 'stdout');
const stderrRecords = records.filter(r => r.source === 'stderr');
expect(stdoutRecords.map(r => r.msg)).toEqual(['out line 1', 'out line 2']);
expect(stderrRecords.map(r => r.msg)).toEqual(['err line']);
});
it('tags stderr as stdout in COMBINED_AS_STDOUT mode', async () => {
const records = await pipeAndCollect('out\n', 'err\n', { mode: PipeMode.COMBINED_AS_STDOUT });
expect(records).toHaveLength(2);
expect(records.every(r => r.source === 'stdout')).toBe(true);
});
it('applies the line transformer and drops lines it nulls out', async () => {
const records = await pipeAndCollect('keep\ndrop\n', null, {
lineTransformer: line => (line === 'drop' ? null : `[prefix] ${line}`),
});
expect(records.map(r => r.msg)).toEqual(['[prefix] keep']);
});
});
+5 -6
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/steps",
"version": "22.4.0",
"version": "23.1.0",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "BUSL-1.1",
"author": "Expo <[email protected]>",
@@ -27,8 +27,8 @@
"clean": "rimraf node_modules dist coverage \"*.tsbuildinfo\""
},
"dependencies": {
"@expo/eas-build-job": "22.0.0",
"@expo/logger": "22.0.0",
"@expo/eas-build-job": "23.1.0",
"@expo/logger": "23.1.0",
"@expo/spawn-async": "^1.7.2",
"arg": "^5.0.2",
"fast-glob": "^3.3.2",
@@ -36,8 +36,7 @@
"joi": "^17.13.1",
"jsep": "^1.3.8",
"lodash.clonedeep": "^4.5.0",
"lodash.get": "^4.4.2",
"uuid": "^9.0.1",
"uuid": "^11.1.1",
"yaml": "^2.4.3",
"zod": "^4.3.5"
},
@@ -45,8 +44,8 @@
"@jest/globals": "^29.7.0",
"@types/jest": "^29.5.12",
"@types/lodash.clonedeep": "^4.5.9",
"@types/lodash.get": "^4.4.9",
"@types/node": "20.14.2",
"bunyan": "^1.8.15",
"eslint-plugin-async-protect": "^3.1.0",
"jest": "^29.7.0",
"rimraf": "3.0.2",
@@ -0,0 +1,92 @@
import { get, parsePath } from '../get';
describe(parsePath, () => {
it('splits dot-separated paths', () => {
expect(parsePath('steps.step1.outputs.out')).toEqual(['steps', 'step1', 'outputs', 'out']);
});
it('parses bracketed array indices', () => {
expect(parsePath('list[0].name')).toEqual(['list', '0', 'name']);
expect(parsePath('a[0][1]')).toEqual(['a', '0', '1']);
});
it('parses bracketed keys without quotes as emitted by jsepEval', () => {
expect(parsePath('a[b]')).toEqual(['a', 'b']);
});
it('parses quoted bracketed keys, keeping dots inside them intact', () => {
expect(parsePath('a["b.c"]')).toEqual(['a', 'b.c']);
expect(parsePath("a['b']")).toEqual(['a', 'b']);
});
it('throws on unterminated brackets', () => {
expect(() => parsePath('a[b')).toThrow('unterminated bracket');
expect(() => parsePath('a["b]')).toThrow('unterminated quoted bracket');
});
});
describe(get, () => {
const context = {
steps: {
step1: {
outputs: {
my_output: 'value1',
},
},
},
eas: {
job: {
version: { buildNumber: '42' },
secrets: null,
list: ['first', { name: 'second' }],
},
runtimeVersion: undefined,
},
};
it('resolves dot-separated paths, as used by template interpolation', () => {
expect(get(context, 'steps.step1.outputs.my_output')).toBe('value1');
expect(get(context, 'eas.job.version.buildNumber')).toBe('42');
});
it('resolves single identifiers, as used by jsepEval', () => {
expect(get(context, 'steps')).toBe(context.steps);
});
it('resolves array indices in both bracket and dot form', () => {
expect(get(context, 'eas.job.list[0]')).toBe('first');
expect(get(context, 'eas.job.list[1].name')).toBe('second');
expect(get(context, 'eas.job.list.1.name')).toBe('second');
});
it('resolves computed member paths emitted by jsepEval, e.g. a[b]', () => {
expect(get(context, 'eas[job]')).toBe(context.eas.job);
expect(get({ a: { b: 1 } }, 'a[b]')).toBe(1);
});
it('returns undefined for missing paths instead of throwing', () => {
expect(get(context, 'eas.job.doesNotExist')).toBe(undefined);
expect(get(context, 'eas.job.doesNotExist.nested')).toBe(undefined);
expect(get(context, 'doesNotExist.at.all')).toBe(undefined);
});
it('returns undefined when traversing through null or undefined', () => {
expect(get(context, 'eas.job.secrets.robotAccessToken')).toBe(undefined);
expect(get(context, 'eas.runtimeVersion.anything')).toBe(undefined);
expect(get(null, 'a.b')).toBe(undefined);
expect(get(undefined, 'a')).toBe(undefined);
});
it('returns null values as-is rather than undefined', () => {
expect(get(context, 'eas.job.secrets')).toBe(null);
});
it('prefers a verbatim own property over a nested path, like lodash.get', () => {
expect(get({ 'a.b': 'flat', a: { b: 'nested' } }, 'a.b')).toBe('flat');
expect(get({ a: { b: 'nested' } }, 'a.b')).toBe('nested');
});
it('reads properties of primitives encountered mid-path', () => {
expect(get({ a: 'text' }, 'a.length')).toBe(4);
});
});
Loaded 100 of 113 files, more files were not shown because too many files have changed in this diff. Show more