Compare commits

...
Author SHA1 Message Date
Expo CI 74292d2ace v23.2.0 2026-08-31 20:25:25 +00:00
Szymon Dziedzic b0b7cad316 [eas-cli] Warn about Android simulator parity (#4320)
* [eas-cli] warn about Android simulator parity

* [eas-cli] add Android simulator warning changelog
2026-08-31 22:22:53 +02:00
expo-tuft[bot]andtchayen 926887d14e [eas-cli] Add --tag to simulator session commands (#4318)
Lets projects with several app variants group and filter their sessions.

Co-authored-by: expo-tuft[bot] <288127324+expo-tuft[bot]@users.noreply.github.com>
Co-authored-by: tchayen <[email protected]>
2026-08-31 22:40:52 +03:00
Gabe DebesandTomasz Mazur 2e6bb2d276 [eas-cli] Add --ssh to workflow:run (#4033)
* [eas-cli] add --ssh to workflow:run

* [eas-cli] Restore WorkflowRunSshInput types dropped by staging regen

Staging schema does not include the --ssh input yet (universe #29128),
but the CLI stack already depends on it. Keep the types until that ships.

* Regenerate GraphQL

* Dedupe changelog

---------

Co-authored-by: Tomasz Mazur <[email protected]>
2026-08-31 17:30:17 +02:00
Gabe DebesandTomasz Mazur 30656509ac [eas-cli] Add workflow:ssh command (#4032)
* [eas-cli] add workflow:ssh command

* Poll for an open SSH session before sleeping.

Avoids a needless wait when the first connect-info fetch is already ready.

* Update SSH wait test for poll-before-sleep.

The wait loop now polls first, so keep one pending poll before ready.

* Cover WorkflowJobSshQuery error and turtleBuild session paths.

* [eas-cli] Regen GraphQL types from staging (TurtleSshSession)

* [eas-cli] address workflow:ssh review feedback

* [eas-cli] drop the GET_BUILD check from sshRequested

* [eas-cli] accept a job run id in workflow:ssh

* Use URL to parse the host

* Regenerate GraphQL

* Move changelog entry

---------

Co-authored-by: Tomasz Mazur <[email protected]>
2026-08-31 17:30:17 +02:00
66e3f1ad7b [build-tools] Use expo-device-hub for Android web previews (#4304)
Co-authored-by: szdziedzic <[email protected]>
Co-authored-by: Codex <[email protected]>
2026-08-31 15:49:49 +02:00
Tomasz Mazur 9aadf2d467 [eas-cli] Stream workflow run logs in real time (#4228)
* [eas-cli] Add centrifuge and ws to deps

* [eas-cli] Stream workflow job logs while waiting for a run

* [eas-cli] Add changelog entry for realtime workflow logs

* [eas-cli] Make msg optional and fix flickering

* [eas-cli] Fetch job logs without casing on job type

* [eas-cli] Ingest realtime incrementally instead of rebuilding full logs on every message (#4293)
2026-08-31 13:38:38 +00:00
Stanisław Chmiela ba80e57753 [eas-cli] Update GraphQL schema for web preview sessions (#4315)
Updated GraphQL and adjusted code.
2026-08-31 15:20:33 +02:00
Expo CI 50c76f41e3 update CHANGELOG.md and bump STAGING and PRODUCTION in cli-versions.json 2026-08-31 11:27:55 +00:00
Expo CI 0658d58efd v23.1.0 2026-08-31 11:24:34 +00:00
Phil Pluckthun 76a700c45c Remove verify step (#4310) 2026-08-29 18:07:45 +01:00
Szymon Dziedzic 5ded193ccf [eas-cli] Fix v23.0.0 changelog (#4309) 2026-08-29 11:24:10 +02:00
Brent Vatne 93420d1fa0 [eas-cli] Add prebuild fallback regression tests (#4285)
* [eas-cli] Add prebuild fallback regression tests

* Update changelog for prebuild fallback tests
2026-08-28 15:45:29 -07:00
Brent Vatne add787f625 [local-build-plugin] Replace bunyan with @expo/bunyan (#4284)
* [local-build-plugin] Replace bunyan with @expo/bunyan

* Update changelog for @expo/bunyan
2026-08-28 15:45:28 -07:00
Brent Vatne ec2c2aca5e [steps] Replace lodash.get with a local helper (#4283)
* [steps] Replace lodash.get with a local helper

* Update changelog for lodash.get removal
2026-08-28 15:45:28 -07:00
Brent Vatne 1e86c48f48 [eas-cli][steps] Bump uuid to v11 (#4282)
* [eas-cli][steps] Bump uuid to v11

* Update changelog for uuid v11
2026-08-28 15:45:27 -07:00
Brent Vatne 545c2dbf9b [eas-cli] Bump minizlib to 3.1.0 (#4281) 2026-08-28 15:45:27 -07:00
Brent Vatne 3cb029147e [eas-cli] Prompt for Expo SDK version in eas new (#4301) 2026-08-28 12:05:38 -07:00
expo-tuft[bot]andabbanmustafa d48cfbba69 [build-tools] Normalize extracted project permissions (#4287)
* [build-tools] Normalize extracted project permissions

Co-authored-by: abbanmustafa <[email protected]>

* [build-tools] Add project permission fix changelog

Co-authored-by: abbanmustafa <[email protected]>

* [build-tools] Log project permission normalization

Co-authored-by: abbanmustafa <[email protected]>

---------

Co-authored-by: expo-tuft[bot] <288127324+expo-tuft[bot]@users.noreply.github.com>
Co-authored-by: abbanmustafa <[email protected]>
2026-08-28 12:51:33 -04:00
Robin Gullo 07b5c0daa3 [build-tools] Support pnpm install --filter to install scoped dependencies (#4302)
# Why

`EAS_YARN_FOCUS_WORKSPACE` ([expo/eas-build#598](https://github.com/expo/eas-build/pull/598)) and `EAS_BUN_FILTER_WORKSPACE` ([expo/eas-cli#4292](https://github.com/expo/eas-cli/pull/4292)) let monorepos install only the dependencies of the app being built, so `node_modules` stays slim, the install is faster, and the fingerprint matches what you get locally. pnpm can do the same thing with [`pnpm install --filter <workspace>`](https://pnpm.io/filtering), but there is no way to reach it from EAS today — the pnpm branch always installs every workspace.

# How

Set it in the build profile's `env` in `eas.json`:

```json
{
  "build": {
    "production": {
      "env": {
        "EAS_PNPM_FILTER_WORKSPACE": "my-app"
      }
    }
  }
}
```

The install step then runs `pnpm install --filter my-app`. This is the pnpm counterpart of `EAS_YARN_FOCUS_WORKSPACE`, introduced in expo/eas-build#598, and `EAS_BUN_FILTER_WORKSPACE`, introduced in expo/eas-cli#4292.

# Test Plan

Added two unit tests for the flag set/unset — `yarn jest-unit src/common/__tests__/installDependencies.test.ts --runInBand` in `packages/build-tools`:

```
Test Suites: 1 passed, 1 total
Tests:       9 passed, 9 total
```

Also ran the resulting commands by hand with pnpm 10.32.1, in a workspace with `pkg-a` (depends on `is-odd`), `pkg-b` (depends on `lodash`) and an in-sync `pnpm-lock.yaml`:

| command | installed |
| --- | --- |
| `pnpm install --frozen-lockfile` | both `packages/a/node_modules/is-odd` and `packages/b/node_modules/lodash` |
| `pnpm install --filter pkg-a --frozen-lockfile` | only `packages/a/node_modules/is-odd`, `pnpm-lock.yaml` unchanged |
2026-08-28 18:32:10 +02:00
David Mokos b78c84af2c updates agent cli detector to 0.1.7 (#4272) 2026-08-28 17:31:20 +02:00
Phil Pluckthun dfaa2df90d [eas-cli] Add dynamic retries for deploy command (#4299) 2026-08-28 16:12:40 +01:00
Expo CI 2e2aaa4ac9 v23.0.0 2026-08-28 11:15:00 +00:00
Szymon Dziedzic 3ef0f74110 [eas-cli] Rename serve-sim session type to web-preview-only (#4298)
* [eas-cli] Rename simulator web preview session type

* [eas-cli] Add simulator flag rename changelog

* [eas-cli] Clarify preview-only simulator type
2026-08-28 13:12:33 +02:00
Stanisław Chmiela 51ba7198d7 [build-tools] Ignore minimum release age for internal npx (#4296)
# Why

When a user defines a min package age policy a fresh release of `eas-cli` may fail.

Reported in [the internal support thread](https://exponent-internal.slack.com/archives/C09H6RFAX27/p1787835815444139).

# How

Added `NPM_CONFIG_MIN_RELEASE_AGE=0` to NPM environment to override min release age to 0.

# Test Plan

Codex verified with different npx versions.
2026-08-27 18:19:03 +00:00
Szymon Dziedzic e10e6a2235 [build-tools] Lower serve-sim preview resolution (#4297)
* [build-tools] Lower serve-sim preview resolution

* [build-tools] Add serve-sim resolution changelog
2026-08-27 19:10:56 +02:00
Konrad Armatys 17f84cf7d6 [build-tools] Support bun install --filter to install scoped dependencies (#4292)
# Why

`EAS_YARN_FOCUS_WORKSPACE` ([expo/eas-build#598](https://github.com/expo/eas-build/pull/598)) lets a monorepo install only the dependencies of the app being built, so `node_modules` stays slim, the install is faster, and the fingerprint matches what you get locally. Bun can do the same thing with `bun install --filter <workspace>` ([oven-sh/bun#16093](https://github.com/oven-sh/bun/pull/16093)), but there is no way to reach it from EAS today — the bun branch always installs every workspace.

# How

Set it in the build profile's `env` in `eas.json`:

```json
{
  "build": {
    "production": {
      "env": {
        "EAS_BUN_FILTER_WORKSPACE": "my-app"
      }
    }
  }
}
```

The install step then runs `bun install --filter my-app`. This is the bun counterpart of `EAS_YARN_FOCUS_WORKSPACE`, introduced in expo/eas-build#598.

# Test Plan

Added two unit tests for the flag set/unset — `yarn jest-unit src/common/__tests__/installDependencies.test.ts` in `packages/build-tools`:

```
Test Suites: 1 passed, 1 total
Tests:       7 passed, 7 total
```

Also ran the resulting commands by hand, in a workspace with `pkg-a` (depends on `is-odd`), `pkg-b` (depends on `lodash`) and an in-sync `bun.lock`:

| command | installed |
| --- | --- |
| `bun install --frozen-lockfile` | 4 packages, `packages/b/node_modules/lodash` present |
| `bun install --filter pkg-a --frozen-lockfile` | 2 packages, only `packages/a/node_modules/is-odd`, `bun.lock` unchanged |
2026-08-27 18:41:24 +02:00
Mustafa Abban f7d9b7b297 [build-tools] support remote CocoaPods caching (#4266)
* [build-tools] Cache CocoaPods dependencies between iOS builds

* restore/save cache

* remove CocoaPods download cache

* remove no-lockfile cache key
2026-08-27 10:20:11 -04:00
Stanisław Chmiela b287d7a4e0 [worker] Use cache API for the legacy cache (#3860)
The legacy cache does not work in `workflow-orchestration`.

This rewrites the `CacheManager` to the new/not-so-new Turtle caches API.

## Summary
- replace worker build cache restore/save GCS config URLs with turtle cache download/upload helpers
- derive stable build cache keys without mutating job.cache and preserve clear for force uploads
- keep existing tar archive restore/save semantics and add CacheManager unit coverage

## Test plan
- cd packages/worker && yarn test CacheManager.test.ts
- cd packages/worker && yarn typecheck
- cd packages/build-tools && yarn typecheck
- git diff --check
2026-08-27 15:44:28 +02:00
Douglas Lowder d9d1b91fa5 [ci] Bump both STAGING and PRODUCTION cli-versions in release workflow (#4290) 2026-08-27 04:16:20 -07:00
108 changed files with 8204 additions and 1163 deletions

No files matched your search

+4 -15
View File
@@ -37,20 +37,6 @@ jobs:
run: yarn build
- name: Publish packages to npm
run: yarn run -T lerna publish from-package --yes --no-verify-access
- name: Verify eas-cli was published as latest on npm
run: |
expected="$(jq -r .version lerna.json)"
for attempt in $(seq 1 10); do
published="$(npm view eas-cli@latest version 2>/dev/null || true)"
if [ "$published" = "$expected" ]; then
echo "Verified eas-cli@$expected is latest on npm."
exit 0
fi
echo "Attempt $attempt: npm latest is '$published', expected '$expected'. Retrying in 15s..."
sleep 15
done
echo "eas-cli@$expected was not published as latest on npm; the publish step failed." >&2
exit 1
release-changelog:
name: Update changelog and publish release
runs-on: ubuntu-latest
@@ -81,10 +67,13 @@ jobs:
jq --arg version "$EAS_CLI_VERSION" '.STAGING = $version' \
cli-versions.json > cli-versions.json.tmp
mv cli-versions.json.tmp cli-versions.json
jq --arg version "$EAS_CLI_VERSION" '.PRODUCTION = $version' \
cli-versions.json > cli-versions.json.tmp
mv cli-versions.json.tmp cli-versions.json
git config --global user.email "[email protected]"
git config --global user.name "Expo CI"
git add cli-versions.json CHANGELOG.md
git commit --allow-empty -m "update CHANGELOG.md and bump STAGING in cli-versions.json"
git commit --allow-empty -m "update CHANGELOG.md and bump STAGING and PRODUCTION in cli-versions.json"
git push
- name: Prepare changelog for Slack
run: |
+50
View File
@@ -8,10 +8,60 @@ This is the log of notable changes to EAS CLI and related packages.
### 🎉 New features
- [eas-cli] Add `--tag` to `eas simulator` and `eas simulator:list` to group and filter simulator sessions by app variant. ([#4318](https://github.com/expo/eas-cli/pull/4318) by [@tchayen](https://github.com/tchayen))
- [eas-cli] Update workflow run logs in real time, instead of every 10 seconds. ([#4228](https://github.com/expo/eas-cli/pull/4228) by [@AHGIJMKLKKZNPJKQR](https://github.com/AHGIJMKLKKZNPJKQR))
- [build-tools] Add `expo-device-hub` web previews to Android web-preview-only, Agent Device, Argent, and Appium remote sessions. ([#4304](https://github.com/expo/eas-cli/pull/4304) by [@krystofwoldrich](https://github.com/krystofwoldrich))
- [eas-cli] Add `eas workflow:ssh <workflow-job-id> [command...]` to open an ssh session on the worker running a workflow job. ([#4032](https://github.com/expo/eas-cli/pull/4032) by [@gwdp](https://github.com/gwdp))
- [eas-cli] Add `--ssh` (and `--ssh-idle-timeout`) to `eas workflow:run` to enable ssh on the run's VM jobs. ([#4033](https://github.com/expo/eas-cli/pull/4033) by [@gwdp](https://github.com/gwdp))
### 🐛 Bug fixes
### 🧹 Chores
- [eas-cli] Warn when starting Android EAS Simulator sessions that support is still in development and iOS parity is coming soon. ([#4320](https://github.com/expo/eas-cli/pull/4320) by [@szdziedzic](https://github.com/szdziedzic))
## [23.1.0](https://github.com/expo/eas-cli/releases/tag/v23.1.0) - 2026-08-31
### 🎉 New features
- [eas-cli] Prompt for the Expo SDK version in `eas new` and add a `--sdk-version` flag. The template now downloads from npm instead of GitHub to support SDK-specific versions. ([#4301](https://github.com/expo/eas-cli/pull/4301) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli] Clean up template metadata in projects created with `eas new`: remove the template LICENSE file and reset the package.json name, version, and license fields. ([#4301](https://github.com/expo/eas-cli/pull/4301) by [@brentvatne](https://github.com/brentvatne))
- [build-tools] Support `EAS_PNPM_FILTER_WORKSPACE` to install only the selected workspace's dependencies with `pnpm install --filter`. ([#4302](https://github.com/expo/eas-cli/pull/4302) by [@robingullo](https://github.com/robingullo))
### 🐛 Bug fixes
- [build-tools] Normalize extracted project permissions so source archives cannot leave files unreadable by EAS workers. ([#4287](https://github.com/expo/eas-cli/pull/4287) by [@AbbanMustafa](https://github.com/AbbanMustafa))
- [eas-cli] Add dynamic retries to deploy command that scale up more generously, depending on deployment size ([#4299](https://github.com/expo/eas-cli/pull/4299) by [@kitten](https://github.com/kitten))
### 🧹 Chores
- [eas-cli] Add regression coverage for the bundled iOS entitlement introspection fallback. ([#4285](https://github.com/expo/eas-cli/pull/4285) by [@brentvatne](https://github.com/brentvatne))
- [local-build-plugin] Replace `bunyan` with `@expo/bunyan` while preserving the Steps formatter binary. ([#4284](https://github.com/expo/eas-cli/pull/4284) by [@brentvatne](https://github.com/brentvatne))
- [steps] Replace deprecated `lodash.get` with a local helper. ([#4283](https://github.com/expo/eas-cli/pull/4283) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli][steps] Bump `uuid` to v11 and remove the redundant `@types/uuid` dependency. ([#4282](https://github.com/expo/eas-cli/pull/4282) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli] Bump `minizlib` to 3.1.0 to drop its deprecated `rimraf@5` transitive dependency. ([#4281](https://github.com/expo/eas-cli/pull/4281) by [@brentvatne](https://github.com/brentvatne))
## [23.0.0](https://github.com/expo/eas-cli/releases/tag/v23.0.0) - 2026-08-28
### 🛠 Breaking changes
- [eas-cli] Rename the EAS Simulator preview-only session type from `--type serve-sim` to `--type web-preview-only`. ([#4298](https://github.com/expo/eas-cli/pull/4298) by [@szdziedzic](https://github.com/szdziedzic))
### 🎉 New features
- [build-tools] Cache CocoaPods dependencies between iOS builds. ([#4266](https://github.com/expo/eas-cli/pull/4266) by [@AbbanMustafa](https://github.com/AbbanMustafa))
- [build-tools] Support `EAS_BUN_FILTER_WORKSPACE` to install only the selected workspace's dependencies with `bun install --filter`. ([#4292](https://github.com/expo/eas-cli/pull/4292) by [@konrad-armatys](https://github.com/konrad-armatys))
### 🐛 Bug fixes
- [build-tools] Prevent user npm minimum release age settings from applying to internal EAS CLI commands. ([#4296](https://github.com/expo/eas-cli/pull/4296) by [@sjchmiela](https://github.com/sjchmiela))
- [build-tools] Reduce serve-sim preview resolution from 1280 px to 960 px to lower streaming bandwidth. ([#4297](https://github.com/expo/eas-cli/pull/4297) by [@szdziedzic](https://github.com/szdziedzic))
### 🧹 Chores
- [worker] Use turtle cache upload and download URLs for eas.json build cache restore and save. ([#3860](https://github.com/expo/eas-cli/pull/3860) by [@sjchmiela](https://github.com/sjchmiela))
- [ci] Bump both STAGING and PRODUCTION cli-versions in release workflow. ([#4290](https://github.com/expo/eas-cli/pull/4290) by [@douglowder](https://github.com/douglowder))
## [22.6.0](https://github.com/expo/eas-cli/releases/tag/v22.6.0) - 2026-08-26
### 🎉 New features
+2 -2
View File
@@ -1,4 +1,4 @@
{
"STAGING": "22.6.0",
"PRODUCTION": "22.4.0"
"STAGING": "23.1.0",
"PRODUCTION": "23.1.0"
}
+1 -1
View File
@@ -1,5 +1,5 @@
{
"npmClient": "yarn",
"version": "22.6.0",
"version": "23.2.0",
"$schema": "node_modules/lerna/schemas/lerna-schema.json"
}
+7 -8
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/build-tools",
"version": "22.6.0",
"version": "23.2.0",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "BUSL-1.1",
"author": "Expo <[email protected]>",
@@ -39,17 +39,17 @@
"dependencies": {
"@expo/config": "55.0.10",
"@expo/config-plugins": "55.0.7",
"@expo/downloader": "22.0.0",
"@expo/eas-build-job": "22.0.0",
"@expo/downloader": "23.0.0",
"@expo/eas-build-job": "23.1.0",
"@expo/env": "^0.4.0",
"@expo/logger": "22.0.0",
"@expo/logger": "23.1.0",
"@expo/package-manager": "1.9.10",
"@expo/plist": "^0.3.5",
"@expo/results": "^1.0.0",
"@expo/spawn-async": "1.7.2",
"@expo/steps": "22.4.0",
"@expo/template-file": "22.0.0",
"@expo/turtle-spawn": "22.0.0",
"@expo/steps": "23.1.0",
"@expo/template-file": "23.0.0",
"@expo/turtle-spawn": "23.1.0",
"@expo/xcpretty": "^4.3.1",
"@ngrok/ngrok": "1.7.0",
"@sentry/node": "7.77.0",
@@ -92,7 +92,6 @@
"@types/retry": "^0.12.5",
"@types/semver": "^7.5.8",
"@types/uuid": "^9.0.8",
"bunyan": "^1.8.15",
"jest": "^29.7.0",
"memfs": "^4.17.1",
"nock": "13.4.0",
@@ -1,4 +1,4 @@
import bunyan from 'bunyan';
import type { bunyan } from '@expo/logger';
export function createLogger(): bunyan {
const logger = {
+18 -2
View File
@@ -18,8 +18,12 @@ import { downloadApplicationArchiveAsync } from '../ios/resign';
import { resolveArtifactPath, resolveBuildConfiguration, resolveScheme } from '../ios/resolve';
import { Sentry } from '../sentry';
import { parseAndReportXcactivitylog } from '../steps/utils/ios/xcactivitylog';
import { cacheStatsAsync, restoreCcacheAsync } from '../steps/functions/restoreBuildCache';
import { saveCcacheAsync } from '../steps/functions/saveBuildCache';
import {
cacheStatsAsync,
restoreCcacheAsync,
restoreCocoapodsCacheAsync,
} from '../steps/functions/restoreBuildCache';
import { saveCcacheAsync, saveCocoapodsCacheAsync } from '../steps/functions/saveBuildCache';
import { uploadApplicationArchive } from '../utils/artifacts';
import {
configureExpoUpdatesIfInstalledAsync,
@@ -107,6 +111,12 @@ async function buildInnerAsync(
env: ctx.env,
secrets: ctx.job.secrets,
});
await restoreCocoapodsCacheAsync({
logger: ctx.logger,
workingDirectory,
env: ctx.env,
secrets: ctx.job.secrets,
});
});
await ctx.runBuildPhase(BuildPhase.INSTALL_PODS, async () => {
@@ -250,6 +260,12 @@ async function buildInnerAsync(
env: ctx.env,
secrets: ctx.job.secrets,
});
await saveCocoapodsCacheAsync({
logger: ctx.logger,
workingDirectory,
env: ctx.env,
secrets: ctx.job.secrets,
});
});
await ctx.runBuildPhase(BuildPhase.CACHE_STATS, async () => {
@@ -70,6 +70,48 @@ describe('easBuildInternal', () => {
expect(spawn).not.toHaveBeenCalled();
});
it('overrides the minimum release age only for the internal config command', async () => {
const logger = {
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
child: jest.fn(),
} as any;
const ctx = {
logger,
env: {
FOO: 'bar',
NPM_CONFIG_MIN_RELEASE_AGE: '365',
},
job: { platform: 'ios', buildProfile: 'production' },
} as any;
jest.mocked(resolveEasCommandPrefixAndEnvAsync).mockResolvedValueOnce({
cmd: 'npx',
args: ['-y', 'eas-cli@latest'],
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
jest.mocked(spawn).mockResolvedValueOnce({
stdout: Buffer.from(JSON.stringify({ buildProfile: { env: {} } })),
} as any);
await resolveEnvFromBuildProfileAsync(ctx, { cwd: '/tmp/project' });
expect(spawn).toHaveBeenCalledWith(
'npx',
expect.any(Array),
expect.objectContaining({
env: {
FOO: 'bar',
NPM_CONFIG_MIN_RELEASE_AGE: '0',
},
})
);
expect(ctx.env).toEqual({
FOO: 'bar',
NPM_CONFIG_MIN_RELEASE_AGE: '365',
});
});
it('passes --refresh-ad-hoc-provisioning-profile to build:internal for iOS jobs with refreshAdHocProvisioningProfile', async () => {
const logger = {
info: jest.fn(),
@@ -3,7 +3,10 @@ import spawn, { SpawnPromise, SpawnResult } from '@expo/turtle-spawn';
import { createMockLogger } from '../../__tests__/utils/logger';
import { Sentry } from '../../sentry';
import { PackageManager } from '../../utils/packageManager';
import { installDependenciesWithNpmCacheFallbackAsync } from '../installDependencies';
import {
installDependenciesAsync,
installDependenciesWithNpmCacheFallbackAsync,
} from '../installDependencies';
jest.mock('@expo/turtle-spawn', () => jest.fn());
jest.mock('../../sentry', () => ({
@@ -236,6 +239,85 @@ describe(installDependenciesWithNpmCacheFallbackAsync, () => {
});
});
describe(installDependenciesAsync, () => {
beforeEach(() => {
jest.clearAllMocks();
jest.mocked(spawn).mockReturnValue(createSpawnPromise(Promise.resolve(createSpawnResult())));
});
it('installs only the selected workspace when EAS_PNPM_FILTER_WORKSPACE is set', async () => {
const logger = createMockLogger();
await installDependenciesAsync({
packageManager: PackageManager.PNPM,
env: { EAS_PNPM_FILTER_WORKSPACE: 'my-app' },
logger,
cwd: '/tmp/build',
useFrozenLockfile: true,
});
expect(spawn).toHaveBeenCalledWith(
'pnpm',
['install', '--filter', 'my-app', '--frozen-lockfile'],
expect.objectContaining({ cwd: '/tmp/build' })
);
});
it('installs all workspaces when EAS_PNPM_FILTER_WORKSPACE is not set', async () => {
const logger = createMockLogger();
await installDependenciesAsync({
packageManager: PackageManager.PNPM,
env: {},
logger,
cwd: '/tmp/build',
useFrozenLockfile: true,
});
expect(spawn).toHaveBeenCalledWith(
'pnpm',
['install', '--frozen-lockfile'],
expect.objectContaining({ cwd: '/tmp/build' })
);
});
it('installs only the selected workspace when EAS_BUN_FILTER_WORKSPACE is set', async () => {
const logger = createMockLogger();
await installDependenciesAsync({
packageManager: PackageManager.BUN,
env: { EAS_BUN_FILTER_WORKSPACE: 'my-app' },
logger,
cwd: '/tmp/build',
useFrozenLockfile: true,
});
expect(spawn).toHaveBeenCalledWith(
'bun',
['install', '--filter', 'my-app', '--frozen-lockfile'],
expect.objectContaining({ cwd: '/tmp/build' })
);
});
it('installs all workspaces when EAS_BUN_FILTER_WORKSPACE is not set', async () => {
const logger = createMockLogger();
await installDependenciesAsync({
packageManager: PackageManager.BUN,
env: {},
logger,
cwd: '/tmp/build',
useFrozenLockfile: true,
});
expect(spawn).toHaveBeenCalledWith(
'bun',
['install', '--frozen-lockfile'],
expect.objectContaining({ cwd: '/tmp/build' })
);
});
});
function createSpawnPromise(result: Promise<SpawnResult>): SpawnPromise<SpawnResult> {
return Object.assign(result, {
child: {
@@ -6,6 +6,7 @@ import {
Platform,
Workflow,
} from '@expo/eas-build-job';
import spawn from '@expo/turtle-spawn';
import { randomBytes, randomUUID } from 'crypto';
import { vol } from 'memfs';
import fetch, { Response } from 'node-fetch';
@@ -14,7 +15,7 @@ import { setTimeout } from 'timers/promises';
import { createMockLogger } from '../../__tests__/utils/logger';
import { BuildContext } from '../../context';
import { shallowCloneRepositoryAsync } from '../git';
import { prepareProjectSourcesAsync } from '../projectSources';
import { prepareProjectSourcesAsync, unpackTarGzAsync } from '../projectSources';
jest.mock('@expo/turtle-spawn');
jest.mock('node-fetch');
@@ -23,6 +24,27 @@ jest.mock('@expo/downloader');
jest.mock('@urql/core');
describe('projectSources', () => {
it('makes extracted project sources readable and writable by the worker', async () => {
const logger = createMockLogger();
await unpackTarGzAsync({
source: '/workingdir/project.tar.gz',
destination: '/workingdir/build',
logger,
});
expect(spawn).toHaveBeenNthCalledWith(
1,
'tar',
['-C', '/workingdir/build', '--strip-components', '1', '-zxf', '/workingdir/project.tar.gz'],
{ logger }
);
expect(spawn).toHaveBeenNthCalledWith(2, 'chmod', ['-R', 'u+rwX', '/workingdir/build'], {
logger,
});
expect(logger.info).toHaveBeenCalledWith('Normalizing project source permissions');
});
it('should use the refreshed repository URL', async () => {
const robotAccessToken = randomUUID();
const buildId = randomUUID();
@@ -32,7 +32,11 @@ export async function installDependenciesAsync({
break;
}
case PackageManager.PNPM: {
args = ['install', useFrozenLockfile ? '--frozen-lockfile' : '--no-frozen-lockfile'];
args = [
'install',
...(env['EAS_PNPM_FILTER_WORKSPACE'] ? ['--filter', env['EAS_PNPM_FILTER_WORKSPACE']] : []),
useFrozenLockfile ? '--frozen-lockfile' : '--no-frozen-lockfile',
];
break;
}
case PackageManager.YARN: {
@@ -58,7 +62,11 @@ export async function installDependenciesAsync({
break;
}
case PackageManager.BUN:
args = ['install', ...(useFrozenLockfile ? ['--frozen-lockfile'] : [])];
args = [
'install',
...(env['EAS_BUN_FILTER_WORKSPACE'] ? ['--filter', env['EAS_BUN_FILTER_WORKSPACE']] : []),
...(useFrozenLockfile ? ['--frozen-lockfile'] : []),
];
break;
default:
throw new Error(`Unsupported package manager: ${packageManager}`);
@@ -110,7 +110,7 @@ async function prepareProjectSourcesLocallyAsync<TJob extends Job>(
});
}
async function unpackTarGzAsync({
export async function unpackTarGzAsync({
logger,
source,
destination,
@@ -122,6 +122,8 @@ async function unpackTarGzAsync({
await spawn('tar', ['-C', destination, '--strip-components', '1', '-zxf', source], {
logger,
});
logger.info('Normalizing project source permissions');
await spawn('chmod', ['-R', 'u+rwX', destination], { logger });
}
function uploadProjectMetadataAsFireAndForget(
+2
View File
@@ -5,6 +5,8 @@ import RemoteLoggerStream from './logging/RemoteLoggerStream';
export { Builders, HttpLogStream, RemoteLoggerStream };
export { uploadWithSignedUrl } from './storage/uploadWithSignedUrl';
export type { SignedUrl, UploadWithSignedUrlParams } from './storage/uploadWithSignedUrl';
export { downloadCacheAsync } from './steps/functions/restoreCache';
export { uploadCacheAsync } from './steps/functions/saveCache';
export {
ArtifactToUpload,
@@ -51,7 +51,7 @@ import { createStartAppiumRemoteSessionBuildFunction } from './functions/startAp
import { createStartCuttlefishDeviceBuildFunction } from './functions/startCuttlefishDevice';
import { createStartIosSimulatorBuildFunction } from './functions/startIosSimulator';
import { createStartIosSimulatorRecordingsBuildFunction } from './functions/startIosSimulatorRecordings';
import { createStartServeSimRemoteSessionBuildFunction } from './functions/startServeSimRemoteSession';
import { createStartWebPreviewRemoteSessionBuildFunction } from './functions/startWebPreviewRemoteSession';
import { createStartServeSimMetricsBuildFunction } from './functions/startServeSimMetrics';
import { createCollectServeSimMetricsBuildFunction } from './functions/collectServeSimMetrics';
import { createUploadArtifactBuildFunction } from './functions/uploadArtifact';
@@ -106,7 +106,7 @@ export function getEasFunctions(ctx: CustomBuildContext): BuildFunction[] {
createStartIosSimulatorRecordingsBuildFunction(),
createFinishIosSimulatorRecordingsBuildFunction(),
createUploadDeviceRunSessionScreenRecordingsBuildFunction(ctx),
createStartServeSimRemoteSessionBuildFunction(ctx),
createStartWebPreviewRemoteSessionBuildFunction(ctx),
createStartServeSimMetricsBuildFunction(),
createCollectServeSimMetricsBuildFunction(ctx),
createInstallMaestroBuildFunction(),
@@ -162,6 +162,28 @@ describe(createEasBuildBuildFunctionGroup, () => {
expect(installPodsStep!.ctx.relativeWorkingDirectory).toBe('./ios');
});
it('restores and saves caches around an iOS simulator build', () => {
const buildToolsContext = createMockBuildToolsContext({
platform: Platform.IOS,
simulator: true,
});
const functionGroup = createEasBuildBuildFunctionGroup(buildToolsContext);
const globalCtx = createGlobalContextMock({ logger: createMockLogger() });
const steps = functionGroup.createBuildStepsFromFunctionGroupCall(globalCtx);
const stepNames = steps.map(step => step.displayName);
const restoreCacheIndex = stepNames.indexOf('Restore Cache');
const installPodsIndex = stepNames.indexOf('Install Pods');
const saveCacheIndex = stepNames.indexOf('Save Cache');
expect(restoreCacheIndex).toBeGreaterThan(-1);
expect(restoreCacheIndex).toBeLessThan(installPodsIndex);
expect(saveCacheIndex).toBeGreaterThan(installPodsIndex);
const restoreCacheStep = steps[restoreCacheIndex];
expect(restoreCacheStep.inputs?.find(input => input.id === 'simulator')?.rawValue).toBe(true);
});
it('sets working directory on all steps except checkout (Android with credentials)', () => {
const buildToolsContext = createMockBuildToolsContext({
platform: Platform.ANDROID,
@@ -98,6 +98,8 @@ function createStepsForIosSimulatorBuild({
buildToolsContext,
workingDirectory,
}: HelperFunctionsInput): BuildStep[] {
const evictUsedBefore = new Date();
const calculateEASUpdateRuntimeVersion =
calculateEASUpdateRuntimeVersionFunction().createBuildStepFromFunctionCall(globalCtx, {
id: 'calculate_eas_update_runtime_version',
@@ -106,6 +108,16 @@ function createStepsForIosSimulatorBuild({
const installPods = createInstallPodsBuildFunction().createBuildStepFromFunctionCall(globalCtx, {
workingDirectory: workingDirectory ? path.join(workingDirectory, './ios') : './ios',
});
const restoreCache = createRestoreBuildCacheFunction().createBuildStepFromFunctionCall(
globalCtx,
{
workingDirectory,
callInputs: {
platform: Platform.IOS,
simulator: true,
},
}
);
const configureEASUpdate =
configureEASUpdateIfInstalledFunction().createBuildStepFromFunctionCall(globalCtx, {
workingDirectory,
@@ -123,6 +135,16 @@ function createStepsForIosSimulatorBuild({
'${ steps.calculate_eas_update_runtime_version.resolved_eas_update_runtime_version }',
},
});
const saveCache = createSaveBuildCacheFunction(evictUsedBefore).createBuildStepFromFunctionCall(
globalCtx,
{
workingDirectory,
callInputs: {
platform: Platform.IOS,
simulator: true,
},
}
);
return [
createCheckoutBuildFunction().createBuildStepFromFunctionCall(globalCtx),
createSetUpNpmrcBuildFunction().createBuildStepFromFunctionCall(globalCtx, {
@@ -138,6 +160,7 @@ function createStepsForIosSimulatorBuild({
createPrebuildBuildFunction().createBuildStepFromFunctionCall(globalCtx, {
workingDirectory,
}),
restoreCache,
calculateEASUpdateRuntimeVersion,
installPods,
configureEASUpdate,
@@ -159,6 +182,10 @@ function createStepsForIosSimulatorBuild({
createFindAndUploadBuildArtifactsBuildFunction(
buildToolsContext
).createBuildStepFromFunctionCall(globalCtx, { workingDirectory }),
saveCache,
createCacheStatsBuildFunction().createBuildStepFromFunctionCall(globalCtx, {
workingDirectory,
}),
];
}
@@ -0,0 +1,136 @@
import { spawnAsync } from '@expo/steps';
import { vol } from 'memfs';
import { createMockLogger } from '../../../__tests__/utils/logger';
import { Datadog } from '../../../datadog';
import {
compressCocoapodsCacheAsync,
getCocoapodsCachePaths,
resolveCocoapodsCacheKeyAsync,
restoreCocoapodsCacheArchiveAsync,
} from '../../../utils/cocoapodsCache';
import { downloadCacheAsync } from '../restoreCache';
import { restoreCocoapodsCacheAsync } from '../restoreBuildCache';
import { uploadCacheAsync } from '../saveCache';
import { saveCocoapodsCacheAsync } from '../saveBuildCache';
jest.mock('@expo/steps', () => ({
...jest.requireActual('@expo/steps'),
spawnAsync: jest.fn(),
}));
jest.mock('../restoreCache', () => ({
decompressCacheAsync: jest.fn(),
downloadCacheAsync: jest.fn(),
downloadPublicCacheAsync: jest.fn(),
}));
jest.mock('../saveCache', () => ({
compressCacheAsync: jest.fn(),
uploadCacheAsync: jest.fn(),
}));
jest.mock('../../../utils/cocoapodsCache', () => ({
compressCocoapodsCacheAsync: jest.fn(),
getCocoapodsCachePaths: jest.fn(),
resolveCocoapodsCacheKeyAsync: jest.fn(),
restoreCocoapodsCacheArchiveAsync: jest.fn(),
}));
const logger = createMockLogger();
const env = {
EAS_PODS_CACHE: '1',
EAS_BUILD_ID: 'build-id',
__API_SERVER_URL: 'https://api.expo.test',
};
const secrets = { robotAccessToken: 'robot-token' };
describe('CocoaPods build cache', () => {
beforeEach(() => {
jest.clearAllMocks();
vol.fromJSON(
{
'/workingdir/ios/Pods/Manifest.lock': 'manifest',
'/workingdir/ios/Podfile.lock': 'lockfile',
'/tmp/cocoapods-cache.tar.gz': 'archive',
},
'/'
);
jest.mocked(getCocoapodsCachePaths).mockReturnValue({
iosDirectory: '/workingdir/ios',
podsDirectory: '/workingdir/ios/Pods',
podfileLockPath: '/workingdir/ios/Podfile.lock',
});
jest.mocked(resolveCocoapodsCacheKeyAsync).mockResolvedValue({
key: 'ios-pods-1.16.2-lock-hash',
keyPrefix: 'ios-pods-1.16.2-',
});
jest.mocked(spawnAsync).mockResolvedValue({ stdout: '1.16.2\n' } as any);
});
it('does nothing when the cache is disabled', async () => {
await restoreCocoapodsCacheAsync({
logger,
workingDirectory: '/workingdir',
env: {},
secrets,
});
await saveCocoapodsCacheAsync({
logger,
workingDirectory: '/workingdir',
env: {},
secrets,
});
expect(spawnAsync).not.toHaveBeenCalled();
});
it('restores the newest matching CocoaPods cache', async () => {
jest.mocked(downloadCacheAsync).mockResolvedValue({
archivePath: '/tmp/cocoapods-cache.tar.gz',
matchedKey: 'ios-pods-1.16.2-older-lock-hash',
});
const datadogLogSpy = jest.spyOn(Datadog, 'log');
await restoreCocoapodsCacheAsync({
logger,
workingDirectory: '/workingdir',
env,
secrets,
});
expect(downloadCacheAsync).toHaveBeenCalledWith(
expect.objectContaining({
key: 'ios-pods-1.16.2-lock-hash',
keyPrefixes: ['ios-pods-1.16.2-'],
paths: ['/workingdir/ios/Pods'],
})
);
expect(restoreCocoapodsCacheArchiveAsync).toHaveBeenCalledWith({
archivePath: '/tmp/cocoapods-cache.tar.gz',
workingDirectory: '/workingdir',
});
expect(datadogLogSpy).toHaveBeenCalledWith('CocoaPods cache restored (prefix_match)', {
event: 'cocoapods_cache_restored',
cache_hit_type: 'prefix_match',
});
});
it('compresses and saves the installed Pods directory', async () => {
jest.mocked(compressCocoapodsCacheAsync).mockResolvedValue({
archivePath: '/tmp/cocoapods-cache.tar.gz',
});
await saveCocoapodsCacheAsync({
logger,
workingDirectory: '/workingdir',
env,
secrets,
});
expect(uploadCacheAsync).toHaveBeenCalledWith(
expect.objectContaining({
archivePath: '/tmp/cocoapods-cache.tar.gz',
key: 'ios-pods-1.16.2-lock-hash',
paths: ['/workingdir/ios/Pods'],
})
);
});
});
@@ -1,11 +1,17 @@
import { type bunyan } from '@expo/logger';
import { Platform } from '@expo/eas-build-job';
import fs from 'fs';
import fetch from 'node-fetch';
import os from 'os';
import path from 'path';
import { compressCacheAsync } from '../saveCache';
import { compressCacheAsync, uploadCacheAsync } from '../saveCache';
import { decompressCacheAsync } from '../restoreCache';
jest.mock('node-fetch');
const { Response } = jest.requireActual('node-fetch') as typeof import('node-fetch');
function createLoggerMock(): bunyan {
return { info: jest.fn(), warn: jest.fn(), error: jest.fn() } as unknown as bunyan;
}
@@ -81,3 +87,71 @@ describe('cache compress/decompress round trip', () => {
expect(Math.floor(restoredStat.mtimeMs / 1000)).toBe(Math.floor(mtime.getTime() / 1000));
});
});
describe(uploadCacheAsync, () => {
afterEach(() => {
jest.mocked(fetch).mockReset();
});
it.each([
['normal upload', undefined, false],
['forced upload', true, true],
] as const)('sends force for %s', async (_name, force, expectedForce) => {
const tempDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'upload-cache-test-'));
const archivePath = path.join(tempDir, 'cache.tar.gz');
await fs.promises.writeFile(archivePath, 'cache archive');
jest
.mocked(fetch)
.mockResolvedValueOnce(
new Response(
JSON.stringify({
data: {
headers: {},
url: 'https://storage.expo.test/cache',
},
}),
{ status: 200 }
)
)
.mockImplementationOnce(async (_url, request) => {
const body = request!.body as NodeJS.ReadableStream & AsyncIterable<Buffer>;
for await (const _chunk of body) {
// Consume the archive stream as a real upload would.
}
return new Response(undefined, { status: 200 });
});
try {
await uploadCacheAsync({
logger: createLoggerMock(),
jobId: 'build-id',
expoApiServerURL: 'https://api.expo.test',
robotAccessToken: 'robot-token',
paths: ['/cache/path'],
key: 'cache-key',
archivePath,
size: 13,
platform: Platform.ANDROID,
force,
});
expect(fetch).toHaveBeenCalledTimes(2);
const [uploadSessionUrl, uploadSessionRequest] = jest.mocked(fetch).mock.calls[0];
expect(uploadSessionUrl.toString()).toBe(
'https://api.expo.test/v2/turtle-builds/caches/upload-sessions'
);
expect(JSON.parse(uploadSessionRequest!.body as string)).toMatchObject({
buildId: 'build-id',
force: expectedForce,
key: 'cache-key',
size: 13,
});
expect(jest.mocked(fetch).mock.calls[1][0].toString()).toBe(
'https://storage.expo.test/cache'
);
} finally {
await fs.promises.rm(tempDir, { recursive: true, force: true });
}
});
});
@@ -14,6 +14,7 @@ import {
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
@@ -46,8 +47,8 @@ jest.mock('../../utils/remoteDeviceRunSession', () => ({
getNgrokTunnelDomainOrThrow: jest.fn(),
selectXcodeDeveloperDirectoryAsync: jest.fn(),
spawnDetached: jest.fn(),
startDeviceWebPreviewWithTunnelAsync: jest.fn(),
startNgrokTunnelAsync: jest.fn(),
startServeSimWithTunnelAsync: jest.fn(),
uploadRemoteSessionConfigAsync: jest.fn(),
waitForDeviceRunSessionStoppedAsync: jest.fn(),
}));
@@ -58,6 +59,7 @@ const EXPECTED_EVENT_LOG_PATH = path.join(ARGENT_STATE_DIR, 'tool-server-events.
const mockStopAsync = jest.fn();
const mockTunnelStopAsync = jest.fn();
const mockPreviewStopAsync = jest.fn();
describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
beforeEach(async () => {
@@ -68,6 +70,7 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
jest.mocked(pollArgentArtifactsForUploadAsync).mockResolvedValue(undefined);
mockStopAsync.mockResolvedValue(undefined);
mockTunnelStopAsync.mockResolvedValue(undefined);
mockPreviewStopAsync.mockResolvedValue(undefined);
jest.mocked(startArgentEventCollectionAsync).mockResolvedValue({
stopAsync: mockStopAsync,
getLastEventObservedAt: () => undefined,
@@ -86,6 +89,10 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
url: 'https://argent-abc.tunnel.example.com',
stopAsync: mockTunnelStopAsync,
});
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
previewUrl: 'https://web-preview.tunnel.example.com',
stopAsync: mockPreviewStopAsync,
});
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockResolvedValue(undefined);
@@ -151,5 +158,17 @@ describe('createStartArgentRemoteSessionBuildFunction orchestration', () => {
jest.mocked(waitForDeviceRunSessionStoppedAsync).mock.invocationCallOrder[0]
);
expect(mockTunnelStopAsync).toHaveBeenCalledTimes(1);
expect(startDeviceWebPreviewWithTunnelAsync).toHaveBeenCalledWith(
ctx,
expect.objectContaining({ runtimePlatform: BuildRuntimePlatform.LINUX })
);
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledWith(
expect.objectContaining({
remoteConfig: expect.objectContaining({
webPreviewUrl: 'https://web-preview.tunnel.example.com',
}),
})
);
expect(mockPreviewStopAsync).toHaveBeenCalledTimes(1);
});
});
@@ -0,0 +1,85 @@
import { type bunyan } from '@expo/logger';
import { BuildRuntimePlatform, type BuildStepContext, type BuildStepEnv } from '@expo/steps';
import { type CustomBuildContext } from '../../../customBuildContext';
import {
getDeviceRunSessionIdOrThrow,
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
startDeviceWebPreviewWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../../utils/remoteDeviceRunSession';
import { createStartWebPreviewRemoteSessionBuildFunction } from '../startWebPreviewRemoteSession';
jest.mock('../../utils/remoteDeviceRunSession');
const ctx = {} as CustomBuildContext;
const env = {} as BuildStepEnv;
const logger = { info: jest.fn(), warn: jest.fn() } as unknown as bunyan;
const stopAsync = jest.fn();
async function runAsync(runtimePlatform: BuildRuntimePlatform): Promise<void> {
const buildFunction = createStartWebPreviewRemoteSessionBuildFunction(ctx);
await buildFunction.fn!(
{
logger,
global: { runtimePlatform },
} as unknown as BuildStepContext,
{
inputs: {
package_version: { value: '1.2.3' },
max_duration_seconds: { value: 120 },
},
outputs: {},
env,
} as never
);
}
describe(createStartWebPreviewRemoteSessionBuildFunction, () => {
beforeEach(() => {
jest.clearAllMocks();
jest.mocked(getDeviceRunSessionIdOrThrow).mockReturnValue('device-run-session-id');
jest.mocked(getNgrokTunnelDomainOrThrow).mockReturnValue('tunnel.example.com');
jest.mocked(selectXcodeDeveloperDirectoryAsync).mockResolvedValue(undefined);
jest.mocked(startDeviceWebPreviewWithTunnelAsync).mockResolvedValue({
previewUrl: 'https://web-preview.example.test',
stopAsync,
});
jest.mocked(uploadRemoteSessionConfigAsync).mockResolvedValue(undefined);
jest.mocked(waitForDeviceRunSessionStoppedAsync).mockResolvedValue(undefined);
stopAsync.mockResolvedValue(undefined);
});
it.each([
[BuildRuntimePlatform.DARWIN, true],
[BuildRuntimePlatform.LINUX, false],
])('starts the web preview for %s', async (runtimePlatform, selectsXcode) => {
await runAsync(runtimePlatform);
expect(selectXcodeDeveloperDirectoryAsync).toHaveBeenCalledTimes(selectsXcode ? 1 : 0);
expect(startDeviceWebPreviewWithTunnelAsync).toHaveBeenCalledWith(ctx, {
runtimePlatform,
baseDomain: 'tunnel.example.com',
env,
logger,
timeoutMs: 60_000,
packageVersion: '1.2.3',
});
expect(uploadRemoteSessionConfigAsync).toHaveBeenCalledWith({
ctx,
deviceRunSessionId: 'device-run-session-id',
remoteConfig: { previewUrl: 'https://web-preview.example.test' },
logger,
});
expect(waitForDeviceRunSessionStoppedAsync).toHaveBeenCalledWith({
ctx,
deviceRunSessionId: 'device-run-session-id',
logger,
maxDurationSeconds: 120,
signal: undefined,
});
expect(stopAsync).toHaveBeenCalledTimes(1);
});
});
@@ -20,6 +20,11 @@ import {
getCcacheKeyPrefix,
getCcachePath,
} from '../../utils/cacheKey';
import {
getCocoapodsCachePaths,
resolveCocoapodsCacheKeyAsync,
restoreCocoapodsCacheArchiveAsync,
} from '../../utils/cocoapodsCache';
import { Datadog } from '../../datadog';
import { GRADLE_CACHE_KEY_PREFIX, generateGradleCacheKeyAsync } from '../../utils/gradleCacheKey';
import { TurtleFetchError, turtleFetch } from '../../utils/turtleFetch';
@@ -79,6 +84,13 @@ export function createRestoreBuildCacheFunction(): BuildFunction {
env,
secrets: stepCtx.global.staticContext.job.secrets,
});
} else {
await restoreCocoapodsCacheAsync({
logger,
workingDirectory,
env,
secrets: stepCtx.global.staticContext.job.secrets,
});
}
},
});
@@ -206,6 +218,67 @@ export async function restoreCcacheAsync({
}
}
export async function restoreCocoapodsCacheAsync({
logger,
workingDirectory,
env,
secrets,
}: {
logger: bunyan;
workingDirectory: string;
env: Record<string, string | undefined>;
secrets?: { robotAccessToken?: string };
}): Promise<void> {
if (env.EAS_PODS_CACHE !== '1') {
return;
}
try {
const { stdout } = await spawnAsync('pod', ['--version'], {
env,
stdio: 'pipe',
});
const { key, keyPrefix } = await resolveCocoapodsCacheKeyAsync(workingDirectory, stdout);
logger.info(`Restoring CocoaPods cache key: ${key}`);
const jobId = nullthrows(env.EAS_BUILD_ID, 'EAS_BUILD_ID is not set');
const robotAccessToken = nullthrows(
secrets?.robotAccessToken,
'Robot access token is required for cache operations'
);
const expoApiServerURL = nullthrows(env.__API_SERVER_URL, '__API_SERVER_URL is not set');
const { podsDirectory } = getCocoapodsCachePaths(workingDirectory);
const { archivePath, matchedKey } = await downloadCacheAsync({
logger,
jobId,
expoApiServerURL,
robotAccessToken,
paths: [podsDirectory],
key,
keyPrefixes: [keyPrefix],
platform: Platform.IOS,
});
await restoreCocoapodsCacheArchiveAsync({ archivePath, workingDirectory });
const hitType = matchedKey === key ? 'direct_hit' : 'prefix_match';
logger.info(
`CocoaPods cache restored to ${podsDirectory} (${hitType === 'direct_hit' ? 'direct hit' : 'prefix match'})`
);
Datadog.log(`CocoaPods cache restored (${hitType})`, {
event: 'cocoapods_cache_restored',
cache_hit_type: hitType,
});
} catch (err: unknown) {
if (err instanceof TurtleFetchError && err.response?.status === 404) {
logger.info('No CocoaPods cache found for this key');
} else {
logger.warn('Failed to restore CocoaPods cache: ', err);
}
}
}
export async function restoreGradleCacheAsync({
logger,
workingDirectory,
@@ -20,6 +20,11 @@ import {
generateDefaultBuildCacheKeyAsync,
getCcachePath,
} from '../../utils/cacheKey';
import {
compressCocoapodsCacheAsync,
getCocoapodsCachePaths,
resolveCocoapodsCacheKeyAsync,
} from '../../utils/cocoapodsCache';
import { generateGradleCacheKeyAsync } from '../../utils/gradleCacheKey';
export function createSaveBuildCacheFunction(evictUsedBefore: Date): BuildFunction {
@@ -78,11 +83,77 @@ export function createSaveBuildCacheFunction(evictUsedBefore: Date): BuildFuncti
env,
secrets: stepCtx.global.staticContext.job.secrets,
});
} else {
await saveCocoapodsCacheAsync({
logger,
workingDirectory,
env,
secrets: stepCtx.global.staticContext.job.secrets,
});
}
},
});
}
export async function saveCocoapodsCacheAsync({
logger,
workingDirectory,
env,
secrets,
}: {
logger: bunyan;
workingDirectory: string;
env: Record<string, string | undefined>;
secrets?: { robotAccessToken?: string };
}): Promise<void> {
if (env.EAS_PODS_CACHE !== '1') {
return;
}
const { podsDirectory, podfileLockPath } = getCocoapodsCachePaths(workingDirectory);
try {
await Promise.all([fs.promises.access(podsDirectory), fs.promises.access(podfileLockPath)]);
} catch {
logger.warn('No CocoaPods installation found, skipping cache save');
return;
}
try {
const { stdout } = await spawnAsync('pod', ['--version'], {
env,
stdio: 'pipe',
});
const { key } = await resolveCocoapodsCacheKeyAsync(workingDirectory, stdout);
logger.info(`Saving CocoaPods cache key: ${key}`);
const jobId = nullthrows(env.EAS_BUILD_ID, 'EAS_BUILD_ID is not set');
const robotAccessToken = nullthrows(
secrets?.robotAccessToken,
'Robot access token is required for cache operations'
);
const expoApiServerURL = nullthrows(env.__API_SERVER_URL, '__API_SERVER_URL is not set');
logger.info('Compressing CocoaPods cache...');
const { archivePath } = await compressCocoapodsCacheAsync({ workingDirectory });
const { size } = await fs.promises.stat(archivePath);
logger.info(`CocoaPods cache archive size: ${formatBytes(size)}`);
await uploadCacheAsync({
logger,
jobId,
expoApiServerURL,
robotAccessToken,
archivePath,
key,
paths: [podsDirectory],
size,
platform: Platform.IOS,
});
} catch (err) {
logger.error({ err }, 'Failed to save CocoaPods cache');
}
}
export async function saveCcacheAsync({
logger,
workingDirectory,
@@ -103,6 +103,7 @@ export async function uploadCacheAsync({
archivePath,
size,
platform,
force = false,
}: {
logger: bunyan;
jobId: string;
@@ -113,6 +114,7 @@ export async function uploadCacheAsync({
archivePath: string;
size: number;
platform: Platform | undefined;
force?: boolean;
}): Promise<void> {
const routerURL = platform
? 'v2/turtle-builds/caches/upload-sessions'
@@ -127,12 +129,14 @@ export async function uploadCacheAsync({
key,
version: getCacheVersion(paths),
size,
force,
})
: JSON.stringify({
jobRunId: jobId,
key,
version: getCacheVersion(paths),
size,
force,
}),
headers: {
Authorization: `Bearer ${robotAccessToken}`,
@@ -23,8 +23,8 @@ import {
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
startServeSimWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
waitForFileAsync,
@@ -107,22 +107,19 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
const agentDeviceRemoteSessionUrl = agentDeviceTunnel.url;
logger.info(`Tunnel is ready at ${agentDeviceRemoteSessionUrl}.`);
let serveSim: Awaited<ReturnType<typeof startServeSimWithTunnelAsync>> | undefined;
let webPreview: Awaited<ReturnType<typeof startDeviceWebPreviewWithTunnelAsync>> | undefined;
let eventCollection:
| Awaited<ReturnType<typeof startAgentDeviceEventCollectionAsync>>
| undefined;
try {
// serve-sim is iOS-only — only launch it (and report a webPreviewUrl)
// on Darwin. Android sessions go without a preview URL.
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
serveSim = await startServeSimWithTunnelAsync(ctx, {
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
logger.info(`Web preview URL: ${serveSim.previewUrl}`);
}
webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
logger.info(`Web preview URL: ${webPreview.previewUrl}`);
await uploadRemoteSessionConfigAsync({
ctx,
@@ -130,7 +127,7 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
remoteConfig: {
agentDeviceRemoteSessionUrl,
agentDeviceRemoteSessionToken: daemonToken,
...(serveSim ? { webPreviewUrl: serveSim.previewUrl } : {}),
webPreviewUrl: webPreview.previewUrl,
},
logger,
});
@@ -163,8 +160,8 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
: undefined,
});
} finally {
if (serveSim) {
await serveSim.stopAsync();
if (webPreview) {
await webPreview.stopAsync();
}
await agentDeviceTunnel.stopAsync();
if (eventCollection) {
@@ -26,8 +26,8 @@ import {
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
startServeSimWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
@@ -118,7 +118,7 @@ export function createStartAppiumRemoteSessionBuildFunction(
logger,
});
let appiumTunnel: Awaited<ReturnType<typeof startNgrokTunnelAsync>> | undefined;
let serveSim: Awaited<ReturnType<typeof startServeSimWithTunnelAsync>> | undefined;
let webPreview: Awaited<ReturnType<typeof startDeviceWebPreviewWithTunnelAsync>> | undefined;
try {
appiumTunnel = await startNgrokTunnelAsync({
port: APPIUM_PORT,
@@ -128,18 +128,15 @@ export function createStartAppiumRemoteSessionBuildFunction(
logger,
});
switch (runtimePlatform) {
case BuildRuntimePlatform.DARWIN:
serveSim = await startServeSimWithTunnelAsync(ctx, {
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: APPIUM_STARTUP_TIMEOUT_MS,
});
break;
case BuildRuntimePlatform.LINUX:
break;
}
// expo-device-hub has no serial-selection flag. Device run session workflows must expose
// a single booted Android emulator so the Hub and Appium resolve the same device.
webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: APPIUM_STARTUP_TIMEOUT_MS,
});
await uploadRemoteSessionConfigAsync({
ctx,
@@ -151,7 +148,7 @@ export function createStartAppiumRemoteSessionBuildFunction(
'appium:automationName': device.automationName,
'appium:udid': device.udid,
},
...(serveSim ? { webPreviewUrl: serveSim.previewUrl } : {}),
webPreviewUrl: webPreview.previewUrl,
},
logger,
});
@@ -170,8 +167,8 @@ export function createStartAppiumRemoteSessionBuildFunction(
: undefined,
});
} finally {
if (serveSim) {
await serveSim.stopAsync();
if (webPreview) {
await webPreview.stopAsync();
}
if (appiumTunnel) {
await appiumTunnel.stopAsync();
@@ -26,8 +26,8 @@ import {
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
spawnDetached,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
startServeSimWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
@@ -186,7 +186,7 @@ export function createStartArgentRemoteSessionBuildFunction(
});
let toolsTunnel: Awaited<ReturnType<typeof startNgrokTunnelAsync>> | undefined;
let serveSim: Awaited<ReturnType<typeof startServeSimWithTunnelAsync>> | undefined;
let webPreview: Awaited<ReturnType<typeof startDeviceWebPreviewWithTunnelAsync>> | undefined;
try {
toolsTunnel = await startNgrokTunnelAsync({
port: toolServerPort,
@@ -199,18 +199,14 @@ export function createStartArgentRemoteSessionBuildFunction(
const publicToolsUrl = toolsTunnel.url;
logger.info(`Tunnel is ready at ${publicToolsUrl}.`);
// serve-sim is iOS-only — Android sessions go without a preview URL.
let webPreviewUrl: string | undefined;
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
serveSim = await startServeSimWithTunnelAsync(ctx, {
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
webPreviewUrl = serveSim.previewUrl;
logger.info(`Web preview URL: ${webPreviewUrl}`);
}
webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
logger.info(`Web preview URL: ${webPreview.previewUrl}`);
await uploadRemoteSessionConfigAsync({
ctx,
@@ -218,7 +214,7 @@ export function createStartArgentRemoteSessionBuildFunction(
remoteConfig: {
toolsUrl: publicToolsUrl,
...(toolServerToken ? { toolsAuthToken: toolServerToken } : {}),
...(webPreviewUrl ? { webPreviewUrl } : {}),
webPreviewUrl: webPreview.previewUrl,
},
logger,
});
@@ -238,8 +234,8 @@ export function createStartArgentRemoteSessionBuildFunction(
: undefined,
});
} finally {
if (serveSim) {
await serveSim.stopAsync();
if (webPreview) {
await webPreview.stopAsync();
}
if (toolsTunnel) {
await toolsTunnel.stopAsync();
@@ -10,22 +10,21 @@ import {
getDeviceRunSessionIdOrThrow,
getNgrokTunnelDomainOrThrow,
selectXcodeDeveloperDirectoryAsync,
startServeSimWithTunnelAsync,
startDeviceWebPreviewWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
const STARTUP_TIMEOUT_MS = 60_000;
export function createStartServeSimRemoteSessionBuildFunction(
export function createStartWebPreviewRemoteSessionBuildFunction(
ctx: CustomBuildContext
): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'start_serve_sim_remote_session',
name: 'Start serve-sim remote session',
name: 'Start web preview remote session',
__metricsId: 'eas/start_serve_sim_remote_session',
supportedRuntimePlatforms: [BuildRuntimePlatform.DARWIN],
inputProviders: [
BuildStepInput.createProvider({
id: 'package_version',
@@ -38,30 +37,34 @@ export function createStartServeSimRemoteSessionBuildFunction(
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
}),
],
fn: async ({ logger }, { inputs, env, signal }) => {
fn: async ({ logger, global }, { inputs, env, signal }) => {
const deviceRunSessionId = getDeviceRunSessionIdOrThrow(env);
const ngrokTunnelDomain = getNgrokTunnelDomainOrThrow(env);
const maxDurationSeconds = inputs.max_duration_seconds?.value as number | undefined;
const packageVersion = inputs.package_version?.value as string | undefined;
const { runtimePlatform } = global;
logger.info('Starting serve-sim remote session.');
logger.info(`Starting web preview remote session (runtime: ${runtimePlatform}).`);
await selectXcodeDeveloperDirectoryAsync({ env, logger });
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
await selectXcodeDeveloperDirectoryAsync({ env, logger });
}
const serveSim = await startServeSimWithTunnelAsync(ctx, {
const webPreview = await startDeviceWebPreviewWithTunnelAsync(ctx, {
runtimePlatform,
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
packageVersion,
});
logger.info(`Preview URL: ${serveSim.previewUrl}`);
logger.info(`Preview URL: ${webPreview.previewUrl}`);
try {
await uploadRemoteSessionConfigAsync({
ctx,
deviceRunSessionId,
remoteConfig: { previewUrl: serveSim.previewUrl },
remoteConfig: { previewUrl: webPreview.previewUrl },
logger,
});
@@ -73,7 +76,7 @@ export function createStartServeSimRemoteSessionBuildFunction(
signal,
});
} finally {
await serveSim.stopAsync();
await webPreview.stopAsync();
}
},
});
@@ -13,14 +13,16 @@ import { Sentry } from '../../../sentry';
import { turtleFetch } from '../../../utils/turtleFetch';
import { sleepAsync } from '../../../utils/retry';
import {
createExpoDeviceHubArgs,
createServeSimArgs,
ensureFfmpegInstalledAsync,
fetchServeSimTurnArgsAsync,
fetchWebPreviewTurnArgsAsync,
metricsCorsOriginToServeSimArgs,
startDeviceWebPreviewWithTunnelAsync,
startNgrokTunnelAsync,
turnIceServersToServeSimArgs,
turnIceServersToWebPreviewArgs,
waitForDeviceRunSessionStoppedAsync,
waitForServeSimReadyAsync,
waitForWebPreviewReadyAsync,
} from '../remoteDeviceRunSession';
jest.mock('@ngrok/ngrok');
@@ -132,7 +134,7 @@ describe(createServeSimArgs, () => {
'--webrtc-codec',
'vp8',
'--max-dimension',
'1280',
'960',
'--mjpeg-quality',
'0.55',
'--video-bitrate',
@@ -173,6 +175,49 @@ describe(createServeSimArgs, () => {
});
});
describe(createExpoDeviceHubArgs, () => {
it('uses the latest Expo package and applies the EAS Android streaming policy', () => {
expect(
createExpoDeviceHubArgs({
port: 4321,
turnArgs: ['--turn-url', 'turns:turn.example.test:443'],
})
).toEqual([
'--yes',
'expo-device-hub@latest',
'--port',
'4321',
'--host',
'127.0.0.1',
'--platform',
'android',
'--transport',
'webrtc',
'--webrtc-codec',
'h264',
'--webrtc-ice-policy',
'all',
'--max-dimension',
'1280',
'--video-bitrate',
'6000000',
'--video-fps',
'60',
'--hide-sidebar',
'--hide-boot-device',
'--turn-url',
'turns:turn.example.test:443',
]);
});
it('pins the requested package version', () => {
expect(createExpoDeviceHubArgs({ port: 4321, packageVersion: '0.7.0' }).slice(0, 2)).toEqual([
'--yes',
'[email protected]',
]);
});
});
describe(metricsCorsOriginToServeSimArgs, () => {
it('returns no args when the origin is unset or empty', () => {
expect(metricsCorsOriginToServeSimArgs({} as BuildStepEnv)).toEqual([]);
@@ -200,7 +245,7 @@ describe(metricsCorsOriginToServeSimArgs, () => {
});
});
describe(waitForServeSimReadyAsync, () => {
describe(waitForWebPreviewReadyAsync, () => {
beforeEach(() => {
jest.mocked(turtleFetch).mockReset();
jest.mocked(sleepAsync).mockReset();
@@ -215,8 +260,9 @@ describe(waitForServeSimReadyAsync, () => {
json: async () => ({ status: 'ready', device: 'DEVICE-A' }),
} as unknown as Awaited<ReturnType<typeof turtleFetch>>);
await waitForServeSimReadyAsync({
serveSim: { pid: undefined, getOutput: () => '' },
await waitForWebPreviewReadyAsync({
previewServer: { pid: undefined, getOutput: () => '' },
serverName: 'expo-device-hub',
port: 4321,
timeoutMs: 10_000,
});
@@ -261,14 +307,125 @@ describe(startNgrokTunnelAsync, () => {
});
});
describe(turnIceServersToServeSimArgs, () => {
describe(startDeviceWebPreviewWithTunnelAsync, () => {
const baseDomain = 'eas-simulator.ngrok.dev';
const turnArgs = [
'--turn-url',
'turns:turn.example.test:443',
'--turn-username',
'turn-user',
'--turn-credential',
'turn-credential',
];
const metricsCorsArgs = ['--metrics-cors-origin', 'https://metrics.expo.test'];
const env = {
DEVICE_RUN_SESSION_ID: 'drs-id',
EAS_SIMULATOR_METRICS_CORS_ORIGIN: 'https://metrics.expo.test',
NGROK_AUTHTOKEN: 'ngrok-token',
} as unknown as BuildStepEnv;
beforeEach(() => {
jest.mocked(spawn).mockReset();
jest.mocked(ngrok.forward).mockReset();
jest.mocked(turtleFetch).mockReset();
const spawnPromise = Object.assign(Promise.resolve(undefined), {
child: {
pid: undefined,
unref: jest.fn(),
},
});
jest.mocked(spawn).mockReturnValue(spawnPromise as never);
jest.mocked(turtleFetch).mockImplementation(async url => {
if (url.endsWith('/turn-ice-servers')) {
return {
json: async () => ({
data: {
iceServers: [
{
urls: ['turns:turn.example.test:443'],
username: 'turn-user',
credential: 'turn-credential',
},
],
},
}),
} as unknown as Awaited<ReturnType<typeof turtleFetch>>;
}
return {
json: async () => ({ status: 'ready', device: 'device-id' }),
} as unknown as Awaited<ReturnType<typeof turtleFetch>>;
});
});
it('starts expo-device-hub for Linux and cleans up the preview resources', async () => {
const packageVersion = '1.2.3';
const close = jest.fn().mockResolvedValue(undefined);
jest.mocked(ngrok.forward).mockResolvedValue({
url: () => 'https://android-preview.example.test',
close,
} as never);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.LINUX,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
packageVersion,
});
const [command, args] = jest.mocked(spawn).mock.calls[0];
const port = Number(args[args.indexOf('--port') + 1]);
expect(port).toBeGreaterThan(0);
expect(command).toBe('npx');
expect(args).toEqual(createExpoDeviceHubArgs({ port, turnArgs, packageVersion }));
expect(ngrok.forward).toHaveBeenCalledWith(expect.objectContaining({ addr: port }));
expect(preview.previewUrl).toBe('https://android-preview.example.test');
await preview.stopAsync();
expect(close).toHaveBeenCalledTimes(1);
});
it('starts serve-sim for Darwin with its metrics policy and cleans up the preview resources', async () => {
const packageVersion = '4.5.6';
const close = jest.fn().mockResolvedValue(undefined);
jest.mocked(ngrok.forward).mockResolvedValue({
url: () => 'https://ios-preview.example.test',
close,
} as never);
const preview = await startDeviceWebPreviewWithTunnelAsync(createCtxMock(), {
runtimePlatform: BuildRuntimePlatform.DARWIN,
baseDomain,
env,
logger: createLoggerMock(),
timeoutMs: 10_000,
packageVersion,
});
const [command, args] = jest.mocked(spawn).mock.calls[0];
const port = Number(args[args.indexOf('--port') + 1]);
expect(port).toBeGreaterThan(0);
expect(command).toBe('npx');
expect(args).toEqual(createServeSimArgs({ port, turnArgs, metricsCorsArgs, packageVersion }));
expect(ngrok.forward).toHaveBeenCalledWith(expect.objectContaining({ addr: port }));
expect(preview.previewUrl).toBe('https://ios-preview.example.test');
await preview.stopAsync();
expect(close).toHaveBeenCalledTimes(1);
});
});
describe(turnIceServersToWebPreviewArgs, () => {
it('returns no args for an empty ICE server list', () => {
expect(turnIceServersToServeSimArgs([])).toEqual([]);
expect(turnIceServersToWebPreviewArgs([])).toEqual([]);
});
it('builds --stun-url and --turn-url flags from Cloudflare ICE servers', () => {
expect(
turnIceServersToServeSimArgs([
turnIceServersToWebPreviewArgs([
{ urls: ['stun:stun.cloudflare.com:3478', 'stun:stun.cloudflare.com:53'] },
{
urls: [
@@ -293,7 +450,7 @@ describe(turnIceServersToServeSimArgs, () => {
it('emits only --turn-url flags when no credential-less (STUN) entry is present', () => {
expect(
turnIceServersToServeSimArgs([
turnIceServersToWebPreviewArgs([
{
urls: ['turns:turn.cloudflare.com:443?transport=tcp'],
username: 'u',
@@ -311,19 +468,19 @@ describe(turnIceServersToServeSimArgs, () => {
});
it('emits only --stun-url when there is no credentialed TURN entry', () => {
expect(turnIceServersToServeSimArgs([{ urls: ['stun:stun.cloudflare.com:3478'] }])).toEqual([
expect(turnIceServersToWebPreviewArgs([{ urls: ['stun:stun.cloudflare.com:3478'] }])).toEqual([
'--stun-url',
'stun:stun.cloudflare.com:3478',
]);
});
});
describe(fetchServeSimTurnArgsAsync, () => {
describe(fetchWebPreviewTurnArgsAsync, () => {
beforeEach(() => {
jest.mocked(turtleFetch).mockReset();
});
it('requests TURN ICE servers from the device run session endpoint and returns serve-sim args', async () => {
it('requests TURN ICE servers from the device run session endpoint and returns web preview args', async () => {
jest.mocked(turtleFetch).mockResolvedValue({
json: async () => ({
data: {
@@ -339,7 +496,7 @@ describe(fetchServeSimTurnArgsAsync, () => {
}),
} as unknown as Awaited<ReturnType<typeof turtleFetch>>);
const args = await fetchServeSimTurnArgsAsync(createCtxMock(), {
const args = await fetchWebPreviewTurnArgsAsync(createCtxMock(), {
env: createEnvMock(),
logger: createLoggerMock(),
});
@@ -363,11 +520,11 @@ describe(fetchServeSimTurnArgsAsync, () => {
);
});
it('returns [] and warns when the request fails so serve-sim falls back to P2P/STUN', async () => {
it('returns [] and warns when the request fails so the web preview falls back to P2P/STUN', async () => {
jest.mocked(turtleFetch).mockRejectedValue(new Error('boom'));
const logger = createLoggerMock();
const args = await fetchServeSimTurnArgsAsync(createCtxMock(), {
const args = await fetchWebPreviewTurnArgsAsync(createCtxMock(), {
env: createEnvMock(),
logger,
});
@@ -19,12 +19,16 @@ import { sleepAsync } from '../../utils/retry';
import { turtleFetch } from '../../utils/turtleFetch';
const XCODE_DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer';
const WEB_PREVIEW_HOST = '127.0.0.1';
const SERVE_SIM_PACKAGE_NAME = '@expo/serve-sim';
const SERVE_SIM_HOST = '127.0.0.1';
const SERVE_SIM_MAX_DIMENSION = '1280';
const SERVE_SIM_MAX_DIMENSION = '960';
const SERVE_SIM_MJPEG_QUALITY = '0.55';
const SERVE_SIM_VIDEO_BITRATE = '6000000';
const SERVE_SIM_VIDEO_FPS = '60';
const EXPO_DEVICE_HUB_PACKAGE_NAME = 'expo-device-hub';
const EXPO_DEVICE_HUB_MAX_DIMENSION = '1280';
const EXPO_DEVICE_HUB_VIDEO_BITRATE = '6000000';
const EXPO_DEVICE_HUB_VIDEO_FPS = '60';
const START_DEVICE_RUN_SESSION_MUTATION = graphql(`
mutation StartDeviceRunSession($deviceRunSessionId: ID!, $remoteConfig: JSONObject!) {
@@ -390,11 +394,12 @@ const TurnIceServersResponseSchema = z.object({
});
/**
* Translate Cloudflare ICE servers into serve-sim CLI flags: `--stun-url` (the
* Translate Cloudflare ICE servers into web preview CLI flags: `--stun-url` (the
* credential-less entries) and `--turn-url`/`--turn-username`/`--turn-credential`
* (the entry carrying the short-lived credentials).
* (the entry carrying the short-lived credentials). serve-sim and expo-device-hub
* intentionally expose the same ICE flag contract.
*/
export function turnIceServersToServeSimArgs(iceServers: TurnIceServers): string[] {
export function turnIceServersToWebPreviewArgs(iceServers: TurnIceServers): string[] {
const stunUrls = iceServers
.filter(server => !server.username && !server.credential)
.flatMap(server => server.urls);
@@ -420,14 +425,14 @@ export function turnIceServersToServeSimArgs(iceServers: TurnIceServers): string
/**
* Fetch short-lived Cloudflare TURN ICE servers for this job run from www
* (minted on demand, mirroring how the worker fetches project clone URLs) and
* translate them into serve-sim CLI flags.
* translate them into web preview CLI flags.
*
* Best-effort: on any failure we log and return [] so serve-sim falls back to
* its built-in P2P/STUN behavior. The credential is passed to serve-sim as a
* process arg and deliberately not logged (turtle-spawn never logs argv and the
* worker is single-tenant).
* Best-effort: on any failure we log and return [] so the preview server falls
* back to its built-in P2P/STUN behavior. The credential is passed as a process
* arg and deliberately not logged (turtle-spawn never logs argv and the worker
* is single-tenant).
*/
export async function fetchServeSimTurnArgsAsync(
export async function fetchWebPreviewTurnArgsAsync(
ctx: CustomBuildContext,
{ env, logger }: { env: BuildStepEnv; logger: bunyan }
): Promise<string[]> {
@@ -456,9 +461,9 @@ export async function fetchServeSimTurnArgsAsync(
);
const { data } = TurnIceServersResponseSchema.parse(await response.json());
const args = turnIceServersToServeSimArgs(data.iceServers);
const args = turnIceServersToWebPreviewArgs(data.iceServers);
if (args.length > 0) {
logger.info('Configured serve-sim with Cloudflare TURN ICE servers.');
logger.info('Configured the web preview with Cloudflare TURN ICE servers.');
}
return args;
} catch (err) {
@@ -466,7 +471,7 @@ export async function fetchServeSimTurnArgsAsync(
Sentry.capture('Could not fetch Cloudflare TURN ICE servers', error, { level: 'warning' });
logger.warn(
{ err: error },
'Could not fetch Cloudflare TURN ICE servers; serve-sim will fall back to P2P/STUN.'
'Could not fetch Cloudflare TURN ICE servers; the web preview will fall back to P2P/STUN.'
);
return [];
}
@@ -600,6 +605,10 @@ function createServeSimPackageSpec(packageVersion: string | undefined): string {
return `${SERVE_SIM_PACKAGE_NAME}@${packageVersion ?? 'latest'}`;
}
function createExpoDeviceHubPackageSpec(packageVersion: string | undefined): string {
return `${EXPO_DEVICE_HUB_PACKAGE_NAME}@${packageVersion ?? 'latest'}`;
}
export function createServeSimArgs({
port,
turnArgs = [],
@@ -617,7 +626,7 @@ export function createServeSimArgs({
'--port',
String(port),
'--host',
SERVE_SIM_HOST,
WEB_PREVIEW_HOST,
'--transport',
'webrtc',
'--webrtc-codec',
@@ -635,44 +644,84 @@ export function createServeSimArgs({
];
}
export function createExpoDeviceHubArgs({
port,
turnArgs = [],
packageVersion,
}: {
port: number;
turnArgs?: string[];
packageVersion?: string;
}): string[] {
return [
'--yes',
createExpoDeviceHubPackageSpec(packageVersion),
'--port',
String(port),
'--host',
WEB_PREVIEW_HOST,
'--platform',
'android',
'--transport',
'webrtc',
'--webrtc-codec',
'h264',
'--webrtc-ice-policy',
'all',
'--max-dimension',
EXPO_DEVICE_HUB_MAX_DIMENSION,
'--video-bitrate',
EXPO_DEVICE_HUB_VIDEO_BITRATE,
'--video-fps',
EXPO_DEVICE_HUB_VIDEO_FPS,
'--hide-sidebar',
'--hide-boot-device',
...turnArgs,
];
}
async function findAvailablePortAsync(): Promise<number> {
const server = createServer();
server.unref();
await new Promise<void>((resolve, reject) => {
server.once('error', reject);
server.listen(0, SERVE_SIM_HOST, () => resolve());
server.listen(0, WEB_PREVIEW_HOST, () => resolve());
});
const address = server.address();
await new Promise<void>((resolve, reject) => {
server.close(err => (err ? reject(err) : resolve()));
});
if (!address || typeof address === 'string') {
throw new SystemError('Could not allocate a local port for serve-sim.');
throw new SystemError('Could not allocate a local port for the web preview.');
}
return address.port;
}
const ServeSimReadyResponseSchema = z.object({
const WebPreviewReadyResponseSchema = z.object({
status: z.literal('ready'),
device: z.string(),
});
export async function waitForServeSimReadyAsync({
serveSim,
export async function waitForWebPreviewReadyAsync({
previewServer,
serverName,
port,
timeoutMs,
}: {
serveSim: Pick<DetachedProcessHandle, 'pid' | 'getOutput'>;
previewServer: Pick<DetachedProcessHandle, 'pid' | 'getOutput'>;
serverName: string;
port: number;
timeoutMs: number;
}): Promise<void> {
const readyUrl = `http://${SERVE_SIM_HOST}:${port}/readyz`;
const readyUrl = `http://${WEB_PREVIEW_HOST}:${port}/readyz`;
const deadline = Date.now() + timeoutMs;
let lastError: unknown;
while (Date.now() < deadline) {
if (serveSim.pid !== undefined && !isProcessRunning(serveSim.pid)) {
if (previewServer.pid !== undefined && !isProcessRunning(previewServer.pid)) {
throw new SystemError(
`serve-sim exited before becoming ready. Last output:\n${serveSim.getOutput() || '<empty>'}`
`${serverName} exited before becoming ready. Last output:\n${
previewServer.getOutput() || '<empty>'
}`
);
}
try {
@@ -680,7 +729,7 @@ export async function waitForServeSimReadyAsync({
retries: 0,
timeout: 2_000,
});
ServeSimReadyResponseSchema.parse(await response.json());
WebPreviewReadyResponseSchema.parse(await response.json());
return;
} catch (error) {
lastError = error;
@@ -688,17 +737,75 @@ export async function waitForServeSimReadyAsync({
await sleepAsync(1_000);
}
throw new SystemError(
`Timed out waiting for serve-sim readiness at ${readyUrl}${
`Timed out waiting for ${serverName} readiness at ${readyUrl}${
lastError instanceof Error ? `: ${lastError.message}` : ''
}. Last output:\n${serveSim.getOutput() || '<empty>'}`
}. Last output:\n${previewServer.getOutput() || '<empty>'}`
);
}
export type ServeSimPreviewHandle = {
export type DeviceWebPreviewHandle = {
previewUrl: string;
stopAsync: () => Promise<void>;
};
export type ServeSimPreviewHandle = DeviceWebPreviewHandle;
async function startWebPreviewWithTunnelAsync(
ctx: CustomBuildContext,
{
baseDomain,
env,
logger,
timeoutMs,
serverName,
packageSpec,
createArgs,
}: {
baseDomain: string;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
serverName: string;
packageSpec: string;
createArgs: (port: number, turnArgs: string[]) => string[];
}
): Promise<DeviceWebPreviewHandle> {
const port = await findAvailablePortAsync();
logger.info(`Launching ${packageSpec} on ${WEB_PREVIEW_HOST}:${port}.`);
const turnArgs = await fetchWebPreviewTurnArgsAsync(ctx, { env, logger });
const previewServer = spawnDetached({
command: 'npx',
args: createArgs(port, turnArgs),
env,
});
try {
logger.info(`Waiting for ${serverName} to become ready.`);
await waitForWebPreviewReadyAsync({ previewServer, serverName, port, timeoutMs });
const tunnel = await startNgrokTunnelAsync({
port,
subdomainPrefix: 'web-preview',
baseDomain,
authtoken: getNgrokAuthtokenOrThrow(env),
logger,
});
return {
previewUrl: tunnel.url,
stopAsync: async () => {
const results = await Promise.allSettled([tunnel.stopAsync(), previewServer.stopAsync()]);
for (const result of results) {
if (result.status === 'rejected') {
logger.warn({ err: result.reason }, `Could not stop a ${serverName} preview resource.`);
}
}
},
};
} catch (error) {
await previewServer.stopAsync();
throw error;
}
}
export async function startServeSimWithTunnelAsync(
ctx: CustomBuildContext,
{
@@ -715,42 +822,65 @@ export async function startServeSimWithTunnelAsync(
packageVersion?: string;
}
): Promise<ServeSimPreviewHandle> {
const port = await findAvailablePortAsync();
logger.info(
`Launching ${createServeSimPackageSpec(packageVersion)} on ${SERVE_SIM_HOST}:${port}.`
);
const turnArgs = await fetchServeSimTurnArgsAsync(ctx, { env, logger });
const metricsCorsArgs = metricsCorsOriginToServeSimArgs(env);
const serveSim = spawnDetached({
command: 'npx',
args: createServeSimArgs({ port, turnArgs, metricsCorsArgs, packageVersion }),
return await startWebPreviewWithTunnelAsync(ctx, {
baseDomain,
env,
logger,
timeoutMs,
serverName: 'serve-sim',
packageSpec: createServeSimPackageSpec(packageVersion),
createArgs: (port, turnArgs) =>
createServeSimArgs({ port, turnArgs, metricsCorsArgs, packageVersion }),
});
}
try {
logger.info('Waiting for serve-sim to become ready.');
await waitForServeSimReadyAsync({ serveSim, port, timeoutMs });
const tunnel = await startNgrokTunnelAsync({
port,
subdomainPrefix: 'web-preview',
baseDomain,
authtoken: getNgrokAuthtokenOrThrow(env),
logger,
});
return {
previewUrl: tunnel.url,
stopAsync: async () => {
const results = await Promise.allSettled([tunnel.stopAsync(), serveSim.stopAsync()]);
for (const result of results) {
if (result.status === 'rejected') {
logger.warn({ err: result.reason }, 'Could not stop a serve-sim preview resource.');
}
}
},
};
} catch (error) {
await serveSim.stopAsync();
throw error;
export async function startExpoDeviceHubWithTunnelAsync(
ctx: CustomBuildContext,
{
baseDomain,
env,
logger,
timeoutMs,
packageVersion,
}: {
baseDomain: string;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
packageVersion?: string;
}
): Promise<DeviceWebPreviewHandle> {
return await startWebPreviewWithTunnelAsync(ctx, {
baseDomain,
env,
logger,
timeoutMs,
serverName: 'expo-device-hub',
packageSpec: createExpoDeviceHubPackageSpec(packageVersion),
createArgs: (port, turnArgs) => createExpoDeviceHubArgs({ port, turnArgs, packageVersion }),
});
}
export async function startDeviceWebPreviewWithTunnelAsync(
ctx: CustomBuildContext,
{
runtimePlatform,
...options
}: {
runtimePlatform: BuildRuntimePlatform;
baseDomain: string;
env: BuildStepEnv;
logger: bunyan;
timeoutMs: number;
packageVersion?: string;
}
): Promise<DeviceWebPreviewHandle> {
switch (runtimePlatform) {
case BuildRuntimePlatform.DARWIN:
return await startServeSimWithTunnelAsync(ctx, options);
case BuildRuntimePlatform.LINUX:
return await startExpoDeviceHubWithTunnelAsync(ctx, options);
}
}
@@ -0,0 +1,78 @@
import fs from 'fs';
import os from 'os';
import path from 'path';
import {
compressCocoapodsCacheAsync,
getCocoapodsCachePaths,
resolveCocoapodsCacheKeyAsync,
restoreCocoapodsCacheArchiveAsync,
} from '../cocoapodsCache';
jest.unmock('fs');
jest.unmock('node:fs');
jest.unmock('fs/promises');
jest.unmock('node:fs/promises');
describe('CocoaPods cache utilities', () => {
let workingDirectory: string;
const cocoapodsVersion = '1.16.2';
beforeEach(async () => {
workingDirectory = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'cocoapods-cache-test-'));
await fs.promises.mkdir(path.join(workingDirectory, 'ios'), { recursive: true });
});
afterEach(async () => {
await fs.promises.rm(workingDirectory, { recursive: true, force: true });
});
it('uses a CocoaPods version prefix when no Podfile.lock exists', async () => {
await expect(
resolveCocoapodsCacheKeyAsync(workingDirectory, cocoapodsVersion)
).resolves.toEqual({
key: 'ios-pods-1.16.2-',
keyPrefix: 'ios-pods-1.16.2-',
});
});
it('includes the Podfile.lock hash when it exists', async () => {
const { podfileLockPath } = getCocoapodsCachePaths(workingDirectory);
await fs.promises.writeFile(podfileLockPath, 'PODS:\n - Expo (55.0.0)\n');
const firstResult = await resolveCocoapodsCacheKeyAsync(workingDirectory, cocoapodsVersion);
expect(firstResult.key).toMatch(/^ios-pods-1\.16\.2-[a-f0-9]+$/);
expect(firstResult.keyPrefix).toBe('ios-pods-1.16.2-');
await fs.promises.writeFile(podfileLockPath, 'PODS:\n - Expo (56.0.0)\n');
const secondResult = await resolveCocoapodsCacheKeyAsync(workingDirectory, cocoapodsVersion);
expect(secondResult.key).not.toBe(firstResult.key);
});
it('preserves symlinks and executable permissions through an archive round trip', async () => {
const { podsDirectory } = getCocoapodsCachePaths(workingDirectory);
const frameworkVersionsDirectory = path.join(podsDirectory, 'Example.framework', 'Versions');
const frameworkVersionDirectory = path.join(frameworkVersionsDirectory, 'A');
await fs.promises.mkdir(frameworkVersionDirectory, { recursive: true });
await fs.promises.writeFile(path.join(frameworkVersionDirectory, 'Example'), 'binary');
await fs.promises.symlink('A', path.join(frameworkVersionsDirectory, 'Current'));
const scriptPath = path.join(podsDirectory, 'Target Support Files', 'script.sh');
await fs.promises.mkdir(path.dirname(scriptPath), { recursive: true });
await fs.promises.writeFile(scriptPath, '#!/bin/sh\n');
await fs.promises.chmod(scriptPath, 0o755);
const { archivePath } = await compressCocoapodsCacheAsync({ workingDirectory });
await fs.promises.rm(podsDirectory, { recursive: true, force: true });
await restoreCocoapodsCacheArchiveAsync({ archivePath, workingDirectory });
const restoredSymlinkPath = path.join(frameworkVersionsDirectory, 'Current');
expect((await fs.promises.lstat(restoredSymlinkPath)).isSymbolicLink()).toBe(true);
await expect(fs.promises.readlink(restoredSymlinkPath)).resolves.toBe('A');
const restoredScriptStat = await fs.promises.stat(scriptPath);
expect(restoredScriptStat.mode & 0o111).toBe(0o111);
await fs.promises.rm(path.dirname(archivePath), { recursive: true, force: true });
});
});
@@ -78,7 +78,7 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(result).toEqual({
cmd: 'npx',
args: ['-y', `eas-cli@${STAGING_VERSION}`],
extraEnv: {},
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
});
@@ -91,7 +91,7 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(result).toEqual({
cmd: 'npx',
args: ['-y', `eas-cli@${STAGING_VERSION}`],
extraEnv: {},
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
});
@@ -101,7 +101,7 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(result).toEqual({
cmd: 'npx',
args: ['-y', `eas-cli@${STAGING_VERSION}`],
extraEnv: { EXPO_STAGING: '1' },
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0', EXPO_STAGING: '1' },
});
expect(spawn).not.toHaveBeenCalled();
});
@@ -112,7 +112,7 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(result).toEqual({
cmd: 'npx',
args: ['-y', `eas-cli@${PRODUCTION_VERSION}`],
extraEnv: {},
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
expect(spawn).not.toHaveBeenCalled();
expect(Sentry.capture).not.toHaveBeenCalled();
@@ -133,7 +133,7 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(stagingResult).toEqual({
cmd: 'npx',
args: ['-y', `eas-cli@${EasCliNpmTags.STAGING}`],
extraEnv: { EXPO_STAGING: '1' },
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0', EXPO_STAGING: '1' },
});
process.env.ENVIRONMENT = 'production';
@@ -141,7 +141,7 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(productionResult).toEqual({
cmd: 'npx',
args: ['-y', `eas-cli@${EasCliNpmTags.PRODUCTION}`],
extraEnv: {},
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
expect(Sentry.capture).toHaveBeenCalledWith(
@@ -173,9 +173,17 @@ describe(resolveEasCommandPrefixAndEnvAsync, () => {
expect(result).toEqual({
cmd: 'npx',
args: [`eas-cli@${PRODUCTION_VERSION}`],
extraEnv: {},
extraEnv: { NPM_CONFIG_MIN_RELEASE_AGE: '0' },
});
});
it('disables the minimum release age for every npx invocation', async () => {
process.env.ENVIRONMENT = 'production';
const result = await resolveEasCommandPrefixAndEnvAsync();
expect(result.extraEnv).toEqual({ NPM_CONFIG_MIN_RELEASE_AGE: '0' });
});
});
describe(runEasCliCommand, () => {
@@ -218,4 +226,25 @@ describe(runEasCliCommand, () => {
})
);
});
it('overrides a user-provided minimum release age', async () => {
process.env.ENVIRONMENT = 'production';
await runEasCliCommand({
args: ['deploy', '--json'],
options: {
cwd: '/tmp/project',
env: {
FOO: 'bar',
NPM_CONFIG_MIN_RELEASE_AGE: '365',
},
},
});
const spawnEnv = jest.mocked(spawn).mock.calls[0][2]?.env;
expect(spawnEnv).toEqual({
FOO: 'bar',
NPM_CONFIG_MIN_RELEASE_AGE: '0',
});
});
});
@@ -0,0 +1,106 @@
import { hashFiles } from '@expo/steps';
import fs from 'fs';
import os from 'os';
import path from 'path';
import * as tar from 'tar';
export const COCOAPODS_CACHE_KEY_PREFIX = 'ios-pods-';
const PODS_DIRECTORY_NAME = 'Pods';
const PODFILE_LOCK_NAME = 'Podfile.lock';
export function getCocoapodsCachePaths(workingDirectory: string): {
iosDirectory: string;
podsDirectory: string;
podfileLockPath: string;
} {
const iosDirectory = path.join(workingDirectory, 'ios');
return {
iosDirectory,
podsDirectory: path.join(iosDirectory, PODS_DIRECTORY_NAME),
podfileLockPath: path.join(iosDirectory, PODFILE_LOCK_NAME),
};
}
export async function resolveCocoapodsCacheKeyAsync(
workingDirectory: string,
cocoapodsVersion: string
): Promise<{ key: string; keyPrefix: string }> {
const normalizedVersion = cocoapodsVersion.trim();
if (!normalizedVersion) {
throw new Error('Failed to determine CocoaPods version');
}
const keyPrefix = `${COCOAPODS_CACHE_KEY_PREFIX}${normalizedVersion}-`;
const { podfileLockPath } = getCocoapodsCachePaths(workingDirectory);
try {
await fs.promises.access(podfileLockPath);
} catch {
return { key: keyPrefix, keyPrefix };
}
return {
key: `${keyPrefix}${hashFiles([podfileLockPath])}`,
keyPrefix,
};
}
export async function compressCocoapodsCacheAsync({
workingDirectory,
}: {
workingDirectory: string;
}): Promise<{ archivePath: string }> {
const { iosDirectory, podsDirectory } = getCocoapodsCachePaths(workingDirectory);
await fs.promises.access(podsDirectory);
const archiveDestinationDirectory = await fs.promises.mkdtemp(
path.join(os.tmpdir(), 'save-cocoapods-cache-')
);
const archivePath = path.join(archiveDestinationDirectory, 'cache.tar.gz');
await tar.create(
{
file: archivePath,
cwd: iosDirectory,
gzip: true,
},
[PODS_DIRECTORY_NAME]
);
return { archivePath };
}
export async function restoreCocoapodsCacheArchiveAsync({
archivePath,
workingDirectory,
}: {
archivePath: string;
workingDirectory: string;
}): Promise<void> {
const { iosDirectory, podsDirectory } = getCocoapodsCachePaths(workingDirectory);
await fs.promises.mkdir(iosDirectory, { recursive: true });
const temporaryRestoreDirectory = await fs.promises.mkdtemp(
path.join(iosDirectory, '.eas-pods-cache-')
);
try {
await tar.extract({
file: archivePath,
cwd: temporaryRestoreDirectory,
filter: entryPath =>
entryPath === PODS_DIRECTORY_NAME || entryPath.startsWith(`${PODS_DIRECTORY_NAME}/`),
});
const restoredPodsDirectory = path.join(temporaryRestoreDirectory, PODS_DIRECTORY_NAME);
const restoredPodsStat = await fs.promises.stat(restoredPodsDirectory);
if (!restoredPodsStat.isDirectory()) {
throw new Error('CocoaPods cache archive does not contain a Pods directory');
}
await fs.promises.rm(podsDirectory, { recursive: true, force: true });
await fs.promises.rename(restoredPodsDirectory, podsDirectory);
} finally {
await fs.promises.rm(temporaryRestoreDirectory, { recursive: true, force: true });
}
}
+5 -3
View File
@@ -12,6 +12,8 @@ import { Sentry } from '../sentry';
let cachedEasCliVersionsPromise: Promise<EasCliVersions> | undefined;
const NPX_ENV = { NPM_CONFIG_MIN_RELEASE_AGE: '0' };
/**
* Resolves the `eas-cli` versions to install for staging/production. Prefers the
* versions committed to `cli-versions.json` (fetched from expo/eas-cli); on any
@@ -74,21 +76,21 @@ export async function resolveEasCommandPrefixAndEnvAsync(): Promise<{
return {
cmd: 'npx',
args: [...npxArgsPrefix, `eas-cli@${versions.STAGING}`],
extraEnv: {},
extraEnv: NPX_ENV,
};
} else if (process.env.ENVIRONMENT === 'staging') {
const versions = await resolveEasCliVersionsAsync();
return {
cmd: 'npx',
args: [...npxArgsPrefix, `eas-cli@${versions.STAGING}`],
extraEnv: { EXPO_STAGING: '1' },
extraEnv: { ...NPX_ENV, EXPO_STAGING: '1' },
};
} else {
const versions = await resolveEasCliVersionsAsync();
return {
cmd: 'npx',
args: [...npxArgsPrefix, `eas-cli@${versions.PRODUCTION}`],
extraEnv: {},
extraEnv: NPX_ENV,
};
}
}
@@ -1,6 +1,6 @@
{
"name": "create-eas-build-function",
"version": "22.4.0",
"version": "23.1.0",
"description": "Create functions for use in EAS Build custom builds.",
"keywords": [
"expo",
@@ -35,7 +35,7 @@
"clean": "rimraf ./build/ \"*.tsbuildinfo\""
},
"dependencies": {
"@expo/steps": "22.4.0"
"@expo/steps": "23.1.0"
},
"devDependencies": {
"@expo/package-manager": "1.9.10",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/downloader",
"version": "22.0.0",
"version": "23.0.0",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "BUSL-1.1",
"author": "Expo <[email protected]>",
@@ -1,6 +1,6 @@
{
"name": "eas-build-cache-provider",
"version": "22.0.0",
"version": "23.0.0",
"description": "A build cache provider plugin for the Expo CLI",
"homepage": "https://github.com/expo/eas-cli",
"bugs": "https://github.com/expo/eas-cli/issues",
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/eas-build-job",
"version": "22.0.0",
"version": "23.1.0",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "MIT",
"author": "Expo <[email protected]>",
@@ -25,9 +25,9 @@
"clean": "rimraf node_modules dist coverage \"*.tsbuildinfo\""
},
"dependencies": {
"@expo/logger": "22.0.0",
"@expo/logger": "23.1.0",
"@expo/results": "1.0.0",
"@expo/turtle-spawn": "22.0.0",
"@expo/turtle-spawn": "23.1.0",
"joi": "^17.13.1",
"semver": "^7.6.2",
"zod": "^4.3.5"
+186 -153
View File
File diff suppressed because it is too large. Load diff
+1082 -2
View File
File diff suppressed because it is too large. Load diff
+11 -9
View File
@@ -1,6 +1,6 @@
{
"name": "eas-cli",
"version": "22.6.0",
"version": "23.2.0",
"description": "EAS command line tool",
"keywords": [
"cli",
@@ -48,11 +48,11 @@
"@expo/code-signing-certificates": "0.0.5",
"@expo/config": "55.0.10",
"@expo/config-plugins": "55.0.7",
"@expo/eas-build-job": "22.0.0",
"@expo/eas-json": "22.0.0",
"@expo/eas-build-job": "23.1.0",
"@expo/eas-json": "23.1.0",
"@expo/env": "^1.0.0",
"@expo/json-file": "8.3.3",
"@expo/logger": "22.0.0",
"@expo/logger": "23.1.0",
"@expo/multipart-body-parser": "2.0.0",
"@expo/osascript": "2.1.4",
"@expo/package-manager": "1.9.10",
@@ -64,7 +64,7 @@
"@expo/results": "1.0.0",
"@expo/rudder-sdk-node": "1.1.1",
"@expo/spawn-async": "1.7.2",
"@expo/steps": "22.4.0",
"@expo/steps": "23.1.0",
"@expo/timeago.js": "1.0.0",
"@oclif/core": "^4.8.3",
"@oclif/plugin-autocomplete": "^3.2.40",
@@ -72,11 +72,12 @@
"@sentry/node": "7.77.0",
"@urql/core": "4.0.11",
"@urql/exchange-retry": "1.2.0",
"agent-cli-detector": "0.1.6",
"agent-cli-detector": "0.1.7",
"ajv": "8.11.0",
"ajv-formats": "2.1.1",
"better-opn": "3.0.2",
"bplist-parser": "^0.3.0",
"centrifuge": "5.7.1",
"chalk": "4.1.2",
"cli-progress": "3.12.0",
"dateformat": "4.6.3",
@@ -104,7 +105,7 @@
"log-symbols": "4.1.0",
"mime": "3.0.0",
"minimatch": "5.1.2",
"minizlib": "3.0.1",
"minizlib": "3.1.0",
"nanoid": "3.3.8",
"node-fetch": "2.6.7",
"node-forge": "1.4.0",
@@ -130,8 +131,9 @@
"tslib": "2.6.2",
"turndown": "7.1.2",
"untildify": "4.0.0",
"uuid": "9.0.1",
"uuid": "11.1.1",
"wrap-ansi": "7.0.0",
"ws": "8.21.1",
"yaml": "2.6.0",
"zod": "^4.1.3"
},
@@ -159,8 +161,8 @@
"@types/tar": "6.1.10",
"@types/tar-stream": "3.1.3",
"@types/tough-cookie": "4.0.2",
"@types/uuid": "9.0.7",
"@types/wrap-ansi": "3.0.0",
"@types/ws": "8.5.10",
"axios": "1.18.1",
"eslint-plugin-graphql": "4.0.0",
"jest": "29.7.0",
@@ -8,10 +8,7 @@ import {
mockProjectId,
mockTestCommand,
} from './utils';
import {
fetchRawLogsForBuildJobAsync,
fetchRawLogsForCustomJobAsync,
} from '../../commandUtils/workflow/fetchLogs';
import { fetchRawLogsForJobAsync } from '../../commandUtils/workflow/logs/fetchLogs';
import WorkflowLogView from '../../commands/workflow/logs';
import { AppPlatform, BuildPriority, BuildStatus } from '../../graphql/generated';
import { AppQuery } from '../../graphql/queries/AppQuery';
@@ -38,7 +35,7 @@ jest.mock('fs');
jest.mock('../../log');
jest.mock('../../prompts');
jest.mock('../../utils/json');
jest.mock('../../commandUtils/workflow/fetchLogs');
jest.mock('../../commandUtils/workflow/logs/fetchLogs');
describe(WorkflowLogView, () => {
beforeEach(() => {
@@ -49,6 +46,7 @@ describe(WorkflowLogView, () => {
});
afterEach(() => {
jest.clearAllMocks();
jest.mocked(fetchRawLogsForJobAsync).mockReset();
});
test('view logs, passing in no parameters, no runs found', async () => {
const ctx = mockCommandContext(WorkflowLogView, {
@@ -105,7 +103,7 @@ describe(WorkflowLogView, () => {
return { selectedStep: 'step1' };
}
});
jest.mocked(fetchRawLogsForCustomJobAsync).mockResolvedValue(null);
jest.mocked(fetchRawLogsForJobAsync).mockResolvedValue(null);
await cmd.run();
expect(AppQuery.byIdWorkflowRunsFilteredByStatusAsync).toHaveBeenCalledWith(
ctx.loggedIn.graphqlClient,
@@ -139,7 +137,7 @@ describe(WorkflowLogView, () => {
}
});
jest
.mocked(fetchRawLogsForCustomJobAsync)
.mocked(fetchRawLogsForJobAsync)
.mockResolvedValue(
'{"result":"success","marker":"end-step","buildStepDisplayName":"Install dependencies","buildStepId":"step-id-1","time":"2022-01-01T00:00:00.000Z","msg":"test"}'
);
@@ -209,7 +207,7 @@ describe(WorkflowLogView, () => {
}
});
jest
.mocked(fetchRawLogsForBuildJobAsync)
.mocked(fetchRawLogsForJobAsync)
.mockResolvedValue(
'{"result":"success","marker":"end-step","buildStepDisplayName":"step1","buildStepId":"step-id-1","time":"2022-01-01T00:00:00.000Z","msg":"test"}'
);
@@ -255,7 +253,7 @@ describe(WorkflowLogView, () => {
}
});
jest
.mocked(fetchRawLogsForCustomJobAsync)
.mocked(fetchRawLogsForJobAsync)
.mockResolvedValue(
[
'{"result":"success","marker":"end-step","buildStepDisplayName":"Install","buildStepId":"step-id-1","time":"2022-01-01T00:00:00.000Z","msg":"first"}',
+10
View File
@@ -112,3 +112,13 @@ export function getEASUpdateURL(projectId: string, manifestHostOverride: string
export function getExpoApiWorkflowSchemaURL(): string {
return getExpoApiBaseUrl() + '/v2/workflows/schema';
}
export function getEASLogsWebsocketUrl(): string {
if (process.env.EXPO_STAGING) {
return `wss://staging-logs.expo.dev/connection/websocket`;
} else if (process.env.EXPO_LOCAL) {
return `ws://localhost:4997/connection/websocket`;
} else {
return `wss://logs.expo.dev/connection/websocket`;
}
}
@@ -1,68 +1,109 @@
import { canAccessRepositoryUsingSshAsync, runGitCloneAsync } from '../../../onboarding/git';
import glob from 'fast-glob';
import fs from 'fs-extra';
import { pipeline } from 'stream/promises';
import { extract } from 'tar';
import fetch from '../../../fetch';
import { installDependenciesAsync } from '../../../onboarding/installDependencies';
import { runCommandAsync } from '../../../onboarding/runCommand';
import { expoCommandAsync } from '../../../utils/expoCli';
import {
cloneTemplateAsync,
downloadTemplateAsync,
fetchTemplatePackumentAsync,
initializeGitRepositoryAsync,
installProjectDependenciesAsync,
} from '../commands';
jest.mock('../../../utils/expoCli');
jest.mock('../../../onboarding/git');
jest.mock('../../../fetch');
jest.mock('../../../onboarding/runCommand');
jest.mock('../../../onboarding/installDependencies');
jest.mock('../../../ora');
jest.mock('fs-extra');
jest.mock('tar');
jest.mock('stream/promises');
jest.mock('fast-glob');
describe('commands', () => {
beforeEach(() => {
jest.clearAllMocks();
});
describe('cloneTemplateAsync', () => {
it('should clone the template project with ssh', async () => {
describe('fetchTemplatePackumentAsync', () => {
it('should fetch the packument from the npm registry', async () => {
const packument = { 'dist-tags': { latest: '57.0.19' } };
jest.mocked(fetch).mockResolvedValueOnce({ json: async () => packument } as any);
expect(await fetchTemplatePackumentAsync()).toEqual(packument);
expect(fetch).toHaveBeenCalledWith(
'https://registry.npmjs.org/expo-template-default',
expect.objectContaining({
headers: { accept: 'application/vnd.npm.install-v1+json' },
})
);
});
});
describe('downloadTemplateAsync', () => {
const template = {
npmTag: 'sdk-56',
version: '56.0.34',
tarballUrl: 'https://registry.npmjs.org/expo-template-default/-/56.0.34.tgz',
};
it('should download and extract the template', async () => {
const targetProjectDir = '/test/target-project';
const finalTargetProjectDir = '/test/final-target-project';
const tarballBody = 'tarball-stream';
jest.mocked(canAccessRepositoryUsingSshAsync).mockResolvedValue(true);
jest.mocked(runGitCloneAsync).mockResolvedValue({
targetProjectDir: finalTargetProjectDir,
});
jest.mocked(fs.pathExists).mockResolvedValueOnce(false as never);
jest.mocked(fetch).mockResolvedValueOnce({ body: tarballBody } as any);
jest.mocked(glob).mockResolvedValueOnce(['gitignore', '_vscode'] as any);
const result = await cloneTemplateAsync(targetProjectDir);
const result = await downloadTemplateAsync(targetProjectDir, template);
expect(runGitCloneAsync).toHaveBeenCalledWith({
githubUsername: 'expo',
githubRepositoryName: 'expo-template-default',
targetProjectDir,
cloneMethod: 'ssh',
showOutput: false,
});
expect(result).toBe(finalTargetProjectDir);
expect(fetch).toHaveBeenCalledWith(
'https://registry.npmjs.org/expo-template-default/-/56.0.34.tgz'
);
expect(extract).toHaveBeenCalledWith({ cwd: targetProjectDir, strip: 1 });
expect(pipeline).toHaveBeenCalledWith(tarballBody, extract({} as any));
expect(fs.move).toHaveBeenCalledWith(
'/test/target-project/gitignore',
'/test/target-project/.gitignore',
{ overwrite: true }
);
expect(fs.move).toHaveBeenCalledWith(
'/test/target-project/_vscode',
'/test/target-project/.vscode',
{ overwrite: true }
);
expect(result).toBe(targetProjectDir);
});
it('should clone the template project with https', async () => {
const targetProjectDir = '/test/target-project';
const finalTargetProjectDir = '/test/final-target-project';
it('should throw when the target directory exists and is not empty', async () => {
jest.mocked(fs.pathExists).mockResolvedValueOnce(true as never);
jest.mocked(fs.readdir).mockResolvedValueOnce(['existing-file'] as never);
jest.mocked(canAccessRepositoryUsingSshAsync).mockResolvedValue(false);
jest.mocked(runGitCloneAsync).mockResolvedValue({
targetProjectDir: finalTargetProjectDir,
});
await expect(downloadTemplateAsync('/test/target-project', template)).rejects.toThrow(
'already exists and is not empty'
);
});
const result = await cloneTemplateAsync(targetProjectDir);
it('should not remove a pre-existing directory on failure', async () => {
jest.mocked(fs.pathExists).mockResolvedValueOnce(true as never);
jest.mocked(fs.readdir).mockResolvedValueOnce(['existing-file'] as never);
expect(runGitCloneAsync).toHaveBeenCalledWith({
githubUsername: 'expo',
githubRepositoryName: 'expo-template-default',
targetProjectDir,
cloneMethod: 'https',
showOutput: false,
});
await expect(downloadTemplateAsync('/test/target-project', template)).rejects.toThrow();
expect(fs.remove).not.toHaveBeenCalled();
});
expect(result).toBe(finalTargetProjectDir);
it('should clean up a partially extracted directory on failure', async () => {
jest.mocked(fs.pathExists).mockResolvedValueOnce(false as never);
jest.mocked(fetch).mockRejectedValueOnce(new Error('network error'));
await expect(downloadTemplateAsync('/test/target-project', template)).rejects.toThrow(
'network error'
);
expect(fs.remove).toHaveBeenCalledWith('/test/target-project');
});
});
@@ -9,6 +9,7 @@ import {
copyProjectTemplatesAsync,
generateAppConfigAsync,
generateEasConfigAsync,
removeTemplateFilesAsync,
updatePackageJsonAsync,
updateReadmeAsync,
} from '../projectFiles';
@@ -239,17 +240,24 @@ describe('projectFiles', () => {
});
describe('updatePackageJsonAsync', () => {
it('should update the package.json', async () => {
it('should replace template metadata and add workflow scripts', async () => {
jest.mocked(fs.readJson).mockResolvedValue({
name: 'test-app',
version: '1.0.0',
name: 'expo-template-default',
version: '57.0.19',
license: '0BSD',
description: 'A starter template',
repository: 'https://github.com/expo/expo-template-default',
tags: ['expo'],
main: 'expo-router/entry',
});
const projectDir = '/test/project-dir';
await updatePackageJsonAsync(projectDir);
await updatePackageJsonAsync(projectDir, 'My App');
const expectedPackageJson = {
name: 'test-app',
name: 'myapp',
version: '1.0.0',
private: true,
main: 'expo-router/entry',
scripts: {
draft: 'npx eas-cli@latest workflow:run create-draft.yml',
'development-builds': 'npx eas-cli@latest workflow:run create-development-builds.yml',
@@ -261,6 +269,28 @@ describe('projectFiles', () => {
spaces: 2,
});
});
it('should keep a non-template license', async () => {
jest.mocked(fs.readJson).mockResolvedValue({
name: 'some-template',
license: 'MIT',
});
await updatePackageJsonAsync('/test/project-dir', 'my-app');
expect(fs.writeJson).toHaveBeenCalledWith(
'/test/project-dir/package.json',
expect.objectContaining({ license: 'MIT' }),
{ spaces: 2 }
);
});
});
describe('removeTemplateFilesAsync', () => {
it('should remove the template LICENSE file', async () => {
await removeTemplateFilesAsync('/test/project-dir');
expect(fs.remove).toHaveBeenCalledWith('/test/project-dir/LICENSE');
});
});
describe('copyProjectTemplatesAsync', () => {
@@ -0,0 +1,219 @@
import fetch from '../../../fetch';
import { promptAsync } from '../../../prompts';
import { fetchTemplatePackumentAsync } from '../commands';
import {
fetchSdkVersionsAsync,
promptForSdkVersionAsync,
resolveTemplateAsync,
} from '../sdkVersion';
jest.mock('../../../fetch');
jest.mock('../../../prompts');
jest.mock('../../../api', () => ({
getExpoApiBaseUrl: () => 'https://api.expo.dev',
}));
jest.mock('../commands');
const packument = {
'dist-tags': { latest: '57.0.19', 'sdk-57': '57.0.19', 'sdk-56': '56.0.34' },
versions: {
'57.0.19': {
dist: { tarball: 'https://registry.npmjs.org/expo-template-default/-/57.0.19.tgz' },
},
'56.0.34': {
dist: { tarball: 'https://registry.npmjs.org/expo-template-default/-/56.0.34.tgz' },
},
},
};
function mockVersionsEndpoint(json: object): void {
jest.mocked(fetch).mockResolvedValueOnce({ json: async () => json } as any);
}
beforeEach(() => {
jest.clearAllMocks();
jest.mocked(fetchTemplatePackumentAsync).mockResolvedValue(packument);
});
describe('fetchSdkVersionsAsync', () => {
it('returns released SDK versions sorted descending', async () => {
mockVersionsEndpoint({
sdkVersions: {
'55.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-55' },
'57.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-57' },
'56.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-56' },
'58.0.0': {}, // canary, no release notes
'54.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-54', isDeprecated: true },
},
expoGoSdkVersion: '56.0.0',
});
const versions = await fetchSdkVersionsAsync();
expect(versions).toEqual({
latest: 57,
expoGoCompatible: 56,
available: [57, 56, 55],
});
});
it('returns null when the endpoint is unreachable', async () => {
jest.mocked(fetch).mockRejectedValueOnce(new Error('network error'));
expect(await fetchSdkVersionsAsync()).toBeNull();
});
it('returns null when no released versions are available', async () => {
mockVersionsEndpoint({ sdkVersions: {} });
expect(await fetchSdkVersionsAsync()).toBeNull();
});
});
describe('promptForSdkVersionAsync', () => {
it('offers latest and Expo Go compatible choices', async () => {
jest.mocked(promptAsync).mockResolvedValueOnce({ answer: 56 });
const result = await promptForSdkVersionAsync({
latest: 57,
expoGoCompatible: 56,
available: [57, 56, 55],
});
expect(result).toBe(56);
expect(promptAsync).toHaveBeenCalledWith(
expect.objectContaining({
message: 'Select an Expo SDK version:',
choices: [
expect.objectContaining({ title: 'Latest (SDK 57)', value: 57 }),
expect.objectContaining({
title: 'For learning with Expo Go (SDK 56)',
value: 56,
}),
expect.objectContaining({ title: 'Other SDK version…', value: 'other' }),
],
})
);
});
it('hides the Expo Go choice when it matches the latest SDK or is unavailable', async () => {
jest.mocked(promptAsync).mockResolvedValue({ answer: 57 });
await promptForSdkVersionAsync({ latest: 57, expoGoCompatible: 57, available: [57, 56] });
await promptForSdkVersionAsync({ latest: 57, expoGoCompatible: 54, available: [57, 56] });
for (const call of jest.mocked(promptAsync).mock.calls) {
expect(call[0]).toEqual(
expect.objectContaining({
choices: [
expect.objectContaining({ title: 'Latest (SDK 57)' }),
expect.objectContaining({ title: 'Other SDK version…' }),
],
})
);
}
});
it('hides the "Other" choice when no other versions are available', async () => {
jest.mocked(promptAsync).mockResolvedValueOnce({ answer: 57 });
await promptForSdkVersionAsync({ latest: 57, expoGoCompatible: null, available: [57] });
expect(promptAsync).toHaveBeenCalledWith(
expect.objectContaining({
choices: [expect.objectContaining({ title: 'Latest (SDK 57)' })],
})
);
});
it('offers the remaining versions when "other" is selected', async () => {
jest
.mocked(promptAsync)
.mockResolvedValueOnce({ answer: 'other' })
.mockResolvedValueOnce({ sdkVersion: 53 });
const result = await promptForSdkVersionAsync({
latest: 57,
expoGoCompatible: 56,
available: [57, 56, 55, 54, 53],
});
expect(result).toBe(53);
expect(promptAsync).toHaveBeenLastCalledWith(
expect.objectContaining({
message: 'Select an SDK version:',
choices: [
{ title: 'SDK 55', value: 55 },
{ title: 'SDK 54', value: 54 },
{ title: 'SDK 53', value: 53 },
],
})
);
});
});
describe('resolveTemplateAsync', () => {
it('uses the provided SDK version without prompting', async () => {
expect(await resolveTemplateAsync({ sdkVersion: '56' })).toEqual({
npmTag: 'sdk-56',
version: '56.0.34',
tarballUrl: 'https://registry.npmjs.org/expo-template-default/-/56.0.34.tgz',
});
expect((await resolveTemplateAsync({ sdkVersion: 'sdk-57' })).npmTag).toBe('sdk-57');
expect((await resolveTemplateAsync({ sdkVersion: 'latest' })).npmTag).toBe('latest');
expect(promptAsync).not.toHaveBeenCalled();
expect(fetch).not.toHaveBeenCalled();
});
it('throws for an invalid SDK version', async () => {
await expect(resolveTemplateAsync({ sdkVersion: 'abc' })).rejects.toThrow(
'Invalid SDK version'
);
});
it('throws for an SDK version without a published template', async () => {
await expect(resolveTemplateAsync({ sdkVersion: '50' })).rejects.toThrow(
'A project template for "sdk-50" is not available. Supported SDK versions: 57, 56.'
);
});
it('prompts with only the SDK versions that have a published template', async () => {
mockVersionsEndpoint({
sdkVersions: {
'57.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-57' },
'56.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-56' },
'55.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-55' },
},
});
jest.mocked(promptAsync).mockResolvedValueOnce({ answer: 56 });
const template = await resolveTemplateAsync({});
expect(template.npmTag).toBe('sdk-56');
// SDK 55 has no dist-tag in the packument, so it must not be offered.
expect(promptAsync).toHaveBeenCalledWith(
expect.objectContaining({
choices: [
expect.objectContaining({ title: 'Latest (SDK 57)' }),
expect.objectContaining({ title: 'Other SDK version…' }),
],
})
);
});
it('falls back to the latest tag when the versions endpoint is unavailable', async () => {
jest.mocked(fetch).mockRejectedValueOnce(new Error('network error'));
expect((await resolveTemplateAsync({})).npmTag).toBe('latest');
expect(promptAsync).not.toHaveBeenCalled();
});
it('falls back to the latest tag when no listed version has a template', async () => {
mockVersionsEndpoint({
sdkVersions: { '99.0.0': { releaseNoteUrl: 'https://expo.dev/changelog/sdk-99' } },
});
expect((await resolveTemplateAsync({})).npmTag).toBe('latest');
expect(promptAsync).not.toHaveBeenCalled();
});
});
@@ -1,40 +1,106 @@
import chalk from 'chalk';
import glob from 'fast-glob';
import fs from 'fs-extra';
import path from 'path';
import { pipeline } from 'stream/promises';
import { extract } from 'tar';
import { printDirectory } from './utils';
import { canAccessRepositoryUsingSshAsync, runGitCloneAsync } from '../../onboarding/git';
import fetch from '../../fetch';
import { PackageManager, installDependenciesAsync } from '../../onboarding/installDependencies';
import { runCommandAsync } from '../../onboarding/runCommand';
import { ora } from '../../ora';
import { expoCommandAsync } from '../../utils/expoCli';
export async function cloneTemplateAsync(targetProjectDir: string): Promise<string> {
const githubUsername = 'expo';
const githubRepositoryName = 'expo-template-default';
export const TEMPLATE_PACKAGE_NAME = 'expo-template-default';
const NPM_REGISTRY_URL =
process.env.npm_config_registry?.replace(/\/$/, '') ?? 'https://registry.npmjs.org';
export interface TemplateInfo {
npmTag: string;
version: string;
tarballUrl: string;
}
export interface TemplatePackument {
'dist-tags'?: Record<string, string>;
versions?: Record<string, { dist: { tarball: string } }>;
}
export async function fetchTemplatePackumentAsync(): Promise<TemplatePackument> {
const response = await fetch(`${NPM_REGISTRY_URL}/${TEMPLATE_PACKAGE_NAME}`, {
headers: { accept: 'application/vnd.npm.install-v1+json' },
signal: AbortSignal.timeout(10000),
});
return (await response.json()) as TemplatePackument;
}
export async function downloadTemplateAsync(
targetProjectDir: string,
template: TemplateInfo
): Promise<string> {
const spinner = ora(
`${chalk.bold(`Cloning the project to ${printDirectory(targetProjectDir)}`)}`
`${chalk.bold(`Downloading the project template to ${printDirectory(targetProjectDir)}`)}`
).start();
const cloneMethod = (await canAccessRepositoryUsingSshAsync({
githubUsername,
githubRepositoryName,
}))
? 'ssh'
: 'https';
const targetExisted = await fs.pathExists(targetProjectDir);
try {
// The directory was verified to not exist earlier, but it may have been
// created while the user was answering prompts.
if (targetExisted && (await fs.readdir(targetProjectDir)).length > 0) {
throw new Error(
`Directory ${printDirectory(targetProjectDir)} already exists and is not empty.`
);
}
const { targetProjectDir: finalTargetProjectDirectory } = await runGitCloneAsync({
githubUsername,
githubRepositoryName,
targetProjectDir,
cloneMethod,
showOutput: false,
await fs.mkdirp(targetProjectDir);
const tarballResponse = await fetch(template.tarballUrl);
await pipeline(tarballResponse.body, extract({ cwd: targetProjectDir, strip: 1 }));
await restoreTemplateDotfilesAsync(targetProjectDir);
spinner.succeed(
`Downloaded ${chalk.bold(`${TEMPLATE_PACKAGE_NAME}@${template.version}`)} to ${printDirectory(
targetProjectDir
)}`
);
} catch (error) {
spinner.fail();
if (!targetExisted) {
await fs.remove(targetProjectDir);
}
throw error;
}
return targetProjectDir;
}
/**
* npm strips `.gitignore` files from published packages, and the template ships
* dot-directories with an underscore prefix (e.g. `_vscode`), so restore the
* real names after extraction. This mirrors the renames in create-expo-app;
* keep the directory list in sync with SUPPORTED_DIRECTORIES in
* https://github.com/expo/expo/blob/main/packages/create-expo-app/src/createFileTransform.ts
*/
async function restoreTemplateDotfilesAsync(projectDir: string): Promise<void> {
const entries = await glob('**/{gitignore,_eas,_vscode,_github,_cursor}', {
cwd: projectDir,
onlyFiles: false,
dot: true,
});
spinner.succeed(`Cloned the project to ${printDirectory(finalTargetProjectDirectory)}`);
// Rename deeper entries first so parent renames do not invalidate child paths.
entries.sort((a, b) => b.split('/').length - a.split('/').length);
return finalTargetProjectDirectory;
for (const entry of entries) {
const basename = path.basename(entry);
const dotName = basename === 'gitignore' ? '.gitignore' : `.${basename.slice(1)}`;
await fs.move(
path.join(projectDir, entry),
path.join(projectDir, path.dirname(entry), dotName),
{ overwrite: true }
);
}
}
export async function installProjectDependenciesAsync(
@@ -110,10 +110,37 @@ export async function generateEasConfigAsync(projectDir: string): Promise<void>
await fs.writeJson(easJsonPath, easJson, { spaces: 2 });
}
export async function updatePackageJsonAsync(projectDir: string): Promise<void> {
const TEMPLATE_LICENSE = '0BSD';
// https://github.com/npm/validate-npm-package-name#naming-rules
function sanitizeNpmPackageName(name: string): string {
return (
name
.normalize('NFD')
.replace(/[\u0300-\u036f]/g, '')
.toLowerCase()
.replace(/^[._]+/, '')
.replace(/[^a-z0-9._\-/@]/g, '') || 'app'
);
}
export async function updatePackageJsonAsync(
projectDir: string,
projectName: string
): Promise<void> {
const packageJsonPath = path.join(projectDir, 'package.json');
const packageJson = await fs.readJson(packageJsonPath);
packageJson.name = sanitizeNpmPackageName(projectName);
packageJson.version = '1.0.0';
packageJson.private = true;
delete packageJson.description;
delete packageJson.tags;
delete packageJson.repository;
if (packageJson.license === TEMPLATE_LICENSE) {
delete packageJson.license;
}
if (!packageJson.scripts) {
packageJson.scripts = {};
}
@@ -125,6 +152,10 @@ export async function updatePackageJsonAsync(projectDir: string): Promise<void>
await fs.writeJson(packageJsonPath, packageJson, { spaces: 2 });
}
export async function removeTemplateFilesAsync(projectDir: string): Promise<void> {
await fs.remove(path.join(projectDir, 'LICENSE'));
}
export async function copyProjectTemplatesAsync(projectDir: string): Promise<void> {
const templatesSourceDir = path.join(__dirname, 'templates');
@@ -0,0 +1,181 @@
import { TemplateInfo, fetchTemplatePackumentAsync } from './commands';
import { getExpoApiBaseUrl } from '../../api';
import fetch from '../../fetch';
import Log from '../../log';
import { promptAsync } from '../../prompts';
export interface SdkVersions {
latest: number;
expoGoCompatible: number | null;
available: number[];
}
interface VersionsEndpointResponse {
sdkVersions?: Record<string, { releaseNoteUrl?: string; isDeprecated?: boolean }>;
expoGoSdkVersion?: string;
}
/**
* SDKs without a `releaseNoteUrl` are canary/in-development and are excluded.
*/
export async function fetchSdkVersionsAsync(): Promise<SdkVersions | null> {
let json: VersionsEndpointResponse;
try {
const response = await fetch(`${getExpoApiBaseUrl()}/v2/versions`, {
signal: AbortSignal.timeout(5000),
});
json = (await response.json()) as VersionsEndpointResponse;
} catch (error) {
Log.debug(`Failed to fetch SDK versions: ${error}`);
return null;
}
const available = Object.entries(json.sdkVersions ?? {})
.filter(([, info]) => !!info.releaseNoteUrl && !info.isDeprecated)
.map(([version]) => parseSdkMajor(version))
.filter((major): major is number => major !== null)
.sort((a, b) => b - a);
const latest = available[0];
if (latest == null) {
return null;
}
return {
latest,
expoGoCompatible: parseSdkMajor(json.expoGoSdkVersion),
available,
};
}
function parseSdkMajor(version: string | undefined): number | null {
const major = parseInt(version?.split('.')[0] ?? '', 10);
return Number.isFinite(major) ? major : null;
}
export async function promptForSdkVersionAsync({
latest,
expoGoCompatible,
available,
}: SdkVersions): Promise<number> {
const choices: { title: string; value: number | 'other'; description?: string }[] = [
{
title: `Latest (SDK ${latest})`,
value: latest,
description: 'Recommended for most projects',
},
];
if (
expoGoCompatible !== null &&
expoGoCompatible !== latest &&
available.includes(expoGoCompatible)
) {
choices.push({
title: `For learning with Expo Go (SDK ${expoGoCompatible})`,
value: expoGoCompatible,
description: 'Compatible with Expo Go on App Store and Play Store',
});
}
const otherVersions = available.filter(sdk => sdk !== latest && sdk !== expoGoCompatible);
if (otherVersions.length > 0) {
choices.push({ title: 'Other SDK version…', value: 'other' });
}
const { answer } = await promptAsync({
type: 'select',
name: 'answer',
message: 'Select an Expo SDK version:',
choices,
});
if (answer !== 'other') {
return answer;
}
const { sdkVersion } = await promptAsync({
type: 'select',
name: 'sdkVersion',
message: 'Select an SDK version:',
choices: otherVersions.map(sdk => ({
title: `SDK ${sdk}`,
value: sdk,
})),
});
return sdkVersion;
}
function resolveTemplateForTag(
distTags: Record<string, string>,
versions: Record<string, { dist: { tarball: string } }>,
npmTag: string
): TemplateInfo {
const version = distTags[npmTag];
const tarballUrl = version ? versions[version]?.dist.tarball : undefined;
if (!version || !tarballUrl) {
const supported = Object.keys(distTags)
.filter(tag => /^sdk-\d+$/.test(tag))
.sort((a, b) => parseInt(b.slice(4), 10) - parseInt(a.slice(4), 10));
throw new Error(
`A project template for "${npmTag}" is not available. Supported SDK versions: ${supported
.map(tag => tag.slice(4))
.join(', ')}.`
);
}
return { npmTag, version, tarballUrl };
}
/**
* `sdkVersion` accepts "57", "sdk-57", or "latest".
*/
export async function resolveTemplateAsync({
sdkVersion,
}: {
sdkVersion?: string;
}): Promise<TemplateInfo> {
const packument = await fetchTemplatePackumentAsync();
const templateDistTags = packument['dist-tags'] ?? {};
const packumentVersions = packument.versions ?? {};
if (sdkVersion) {
if (sdkVersion === 'latest') {
return resolveTemplateForTag(templateDistTags, packumentVersions, 'latest');
}
const major = parseSdkMajor(sdkVersion.replace(/^sdk-/, ''));
if (major === null) {
throw new Error(
`Invalid SDK version: "${sdkVersion}". Specify a version number (e.g. 57) or "latest".`
);
}
return resolveTemplateForTag(templateDistTags, packumentVersions, `sdk-${major}`);
}
const versions = await fetchSdkVersionsAsync();
const availableWithTemplates =
versions?.available.filter(sdk => `sdk-${sdk}` in templateDistTags) ?? [];
if (!versions || availableWithTemplates.length === 0) {
if (!versions) {
Log.warn('Could not fetch the list of Expo SDK versions. Using the latest SDK.');
}
return resolveTemplateForTag(templateDistTags, packumentVersions, 'latest');
}
const latest = availableWithTemplates[0];
// Match the prompt guard in prompts.ts: pin to the latest released SDK
// instead of prompting when stdin is not interactive.
if (!process.stdin.isTTY && !global.test) {
Log.withInfo(`Using Expo SDK ${latest}`);
return resolveTemplateForTag(templateDistTags, packumentVersions, `sdk-${latest}`);
}
const selectedSdk = await promptForSdkVersionAsync({
latest,
expoGoCompatible: versions.expoGoCompatible,
available: availableWithTemplates,
});
return resolveTemplateForTag(templateDistTags, packumentVersions, `sdk-${selectedSdk}`);
}
@@ -1,35 +1,137 @@
import { getMockWorkflowRunWithJobsFragment } from '../../../__tests__/commands/utils';
import { fetchRawLogsForCustomJobAsync } from '../fetchLogs';
import { infoForActiveWorkflowRunAsync } from '../utils';
import { WorkflowJobStatus } from '../../../graphql/generated';
import { groupLogLinesIntoSteps, parseLogLines } from '../logs/parseLogs';
import { WorkflowLogs, WorkflowRawLogLine } from '../types';
import { formatActiveWorkflowRun, formatFailedWorkflowRun } from '../utils';
jest.mock('../fetchLogs');
function jobWithLogs(
logLines: WorkflowRawLogLine[],
status: WorkflowJobStatus = WorkflowJobStatus.InProgress
): {
job: ReturnType<typeof getMockWorkflowRunWithJobsFragment>['jobs'][number];
logs: WorkflowLogs;
} {
return {
job: { ...getMockWorkflowRunWithJobsFragment().jobs[0], status },
logs: groupLogLinesIntoSteps(logLines),
};
}
describe('workflow utils', () => {
afterEach(() => {
jest.clearAllMocks();
});
function stepLines(buildStepId: string, count: number): WorkflowRawLogLine[] {
return Array.from({ length: count }, (_, index) => ({ buildStepId, msg: `line${index}` }));
}
test('shows the display name for the current step while keying logs by step id', async () => {
const workflowRun = getMockWorkflowRunWithJobsFragment();
workflowRun.jobs = workflowRun.jobs.map(job => ({
...job,
status: WorkflowJobStatus.InProgress,
}));
jest
.mocked(fetchRawLogsForCustomJobAsync)
.mockResolvedValue(
[
'{"buildStepId":"step-id-1","buildStepDisplayName":"Install dependencies","time":"2022-01-01T00:00:00.000Z","msg":"npm ci"}',
'{"buildStepId":"step-id-1","buildStepDisplayName":"Install dependencies","marker":"end-step","result":"success","time":"2022-01-01T00:00:01.000Z","msg":"done"}',
].join('\n')
);
const output = await infoForActiveWorkflowRunAsync({} as any, workflowRun);
describe(formatActiveWorkflowRun, () => {
test('shows the display name for the current step while keying logs by step id', () => {
const output = formatActiveWorkflowRun([
jobWithLogs(
parseLogLines(
[
'{"buildStepId":"step-id-1","buildStepDisplayName":"Install dependencies","time":"2022-01-01T00:00:00.000Z","msg":"npm ci"}',
'{"buildStepId":"step-id-1","buildStepDisplayName":"Install dependencies","marker":"end-step","result":"success","time":"2022-01-01T00:00:01.000Z","msg":"done"}',
].join('\n')
).logLines
),
]);
expect(output).toContain('Current step');
expect(output).toContain('Install dependencies');
expect(output).not.toContain('step-id-1');
});
test('shows exactly maxLogLines trailing lines of the current step', () => {
const output = formatActiveWorkflowRun([jobWithLogs(stepLines('step-id-1', 10))], 5);
expect(output).toContain('line5');
expect(output).toContain('line9');
expect(output).not.toContain('line4');
});
test('keeps five trailing lines by default', () => {
const output = formatActiveWorkflowRun([jobWithLogs(stepLines('step-id-1', 10))]);
expect(output).toContain('line5');
expect(output).toContain('line9');
expect(output).not.toContain('line4');
});
test('reports the last step as the current one', () => {
const output = formatActiveWorkflowRun([
jobWithLogs([
{ buildStepId: 'install', buildStepDisplayName: 'Install', msg: 'installing' },
{ buildStepId: 'build', buildStepDisplayName: 'Build', msg: 'compiling' },
]),
]);
expect(output).toContain('Build');
expect(output).not.toContain('Install');
});
test('passes over a trailing skipped step', () => {
const output = formatActiveWorkflowRun([
jobWithLogs([
{ buildStepId: 'install', buildStepDisplayName: 'Install', msg: 'installing' },
{
buildStepId: 'build',
buildStepDisplayName: 'Build',
msg: 'skipping',
marker: 'end-step',
result: 'skipped',
},
]),
]);
expect(output).toContain('Install');
expect(output).not.toContain('Build');
});
test('reports a step that has not ended yet', () => {
const output = formatActiveWorkflowRun([
jobWithLogs([
{
buildStepId: 'install',
buildStepDisplayName: 'Install',
msg: 'installed',
marker: 'end-step',
result: 'success',
},
{ buildStepId: 'build', buildStepDisplayName: 'Build', msg: 'compiling' },
]),
]);
expect(output).toContain('Build');
});
test('omits the current step when the job has no steps yet', () => {
const output = formatActiveWorkflowRun([jobWithLogs([])]);
expect(output).not.toContain('Current step');
expect(output).not.toContain('Current logs');
});
test('omits the current step for a job that is not in progress', () => {
const output = formatActiveWorkflowRun([
jobWithLogs(stepLines('step-id-1', 3), WorkflowJobStatus.Success),
]);
expect(output).not.toContain('Current step');
expect(output).not.toContain('line0');
});
});
describe(formatFailedWorkflowRun, () => {
test('prints every line of the failed step when there is no limit', () => {
const output = formatFailedWorkflowRun([
jobWithLogs(
[
...stepLines('build', 8),
{ buildStepId: 'build', msg: 'it failed', marker: 'end-step', result: 'fail' },
],
WorkflowJobStatus.Failure
),
]);
expect(output).toContain('line0');
expect(output).toContain('line7');
});
});
@@ -1,51 +0,0 @@
import { WorkflowJobResult } from './types';
import { BuildQuery } from '../../graphql/queries/BuildQuery';
import { ExpoGraphqlClient } from '../context/contextUtils/createGraphqlClient';
// This function is in a separate module for testing purposes
export async function fetchRawLogsForCustomJobAsync(
job: WorkflowJobResult
): Promise<string | null> {
const firstLogFileUrl = job.turtleJobRun?.logFileUrls?.[0];
if (!firstLogFileUrl) {
return null;
}
const response = await fetch(firstLogFileUrl, {
method: 'GET',
});
const rawLogs = await response.text();
return rawLogs;
}
export async function fetchRawLogsForBuildJobAsync(
state: { graphqlClient: ExpoGraphqlClient },
job: WorkflowJobResult
): Promise<string | null> {
// Prefer turtleJobRun logs, which contain JSONL-formatted step logs
// for both built-in and custom build steps
const turtleLogFileUrl = job.turtleJobRun?.logFileUrls?.[0];
if (turtleLogFileUrl) {
const response = await fetch(turtleLogFileUrl, {
method: 'GET',
});
return await response.text();
}
// Fall back to build logFiles
const buildId = job.outputs?.build_id;
if (!buildId) {
return null;
}
const buildFragment = await BuildQuery.byIdAsync(state.graphqlClient, buildId, {
useCache: false,
});
const firstLogFileUrl = buildFragment.logFiles?.[0];
if (!firstLogFileUrl) {
return null;
}
const response = await fetch(firstLogFileUrl, {
method: 'GET',
});
return await response.text();
}
@@ -0,0 +1,27 @@
import { getMockWorkflowCustomJobFragment } from '../../../../__tests__/commands/utils';
import { BuildQuery } from '../../../../graphql/queries/BuildQuery';
import { ExpoGraphqlClient } from '../../../context/contextUtils/createGraphqlClient';
import { fetchRawLogsForJobAsync } from '../fetchLogs';
jest.mock('../../../../graphql/queries/BuildQuery');
describe(fetchRawLogsForJobAsync, () => {
afterEach(() => {
jest.restoreAllMocks();
});
it('reads a custom job from its turtle job run log file', async () => {
const rawLogs = '{"logId":"1","buildStepId":"install","msg":"from the turtle job run"}';
const fetchSpy = jest.spyOn(globalThis, 'fetch').mockResolvedValue(new Response(rawLogs));
await expect(
fetchRawLogsForJobAsync(
{ graphqlClient: {} as ExpoGraphqlClient },
getMockWorkflowCustomJobFragment()
)
).resolves.toBe(rawLogs);
expect(fetchSpy).toHaveBeenCalledWith('https://example.com/log1');
expect(BuildQuery.byIdAsync).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,186 @@
import { groupLogLinesIntoSteps, mergeLogLines, parseLogLines } from '../parseLogs';
import { WorkflowRawLogLine } from '../../types';
function logLine(overrides: Partial<WorkflowRawLogLine> = {}): WorkflowRawLogLine {
return { msg: 'a message', ...overrides };
}
describe(parseLogLines, () => {
it('parses a JSONL log file', () => {
const { logLines, errors } = parseLogLines(
[
'{"logId":"1","buildStepId":"install","msg":"npm ci"}',
'{"logId":"2","buildStepId":"install","msg":"done"}',
].join('\n')
);
expect(errors).toEqual([]);
expect(logLines).toEqual([
{ logId: '1', buildStepId: 'install', msg: 'npm ci' },
{ logId: '2', buildStepId: 'install', msg: 'done' },
]);
});
it('skips blank lines, including a trailing newline', () => {
const { logLines, errors } = parseLogLines('{"logId":"1","msg":"one"}\n\n');
expect(errors).toEqual([]);
expect(logLines).toHaveLength(1);
});
it('reports malformed lines as errors while keeping the parsable ones', () => {
const { logLines, errors } = parseLogLines(
['{"logId":"1","msg":"one"}', 'this is not json', '{"logId":"2","msg":"two"}'].join('\n')
);
expect(logLines.map(line => line.msg)).toEqual(['one', 'two']);
expect(errors).toHaveLength(1);
expect(errors[0]).toBeInstanceOf(Error);
});
it('parses a line that carries no message', () => {
const { logLines, errors } = parseLogLines('{"logId":"1"}');
expect(errors).toEqual([]);
expect(logLines).toEqual([{ logId: '1' }]);
});
});
describe(mergeLogLines, () => {
it('keeps the rightmost line for a repeated logId', () => {
const fileLogLines = [
logLine({ logId: '1', msg: 'from the file' }),
logLine({ logId: '2', msg: 'from the file' }),
];
const realtimeLogLines = [
logLine({ logId: '2', msg: 'from realtime' }),
logLine({ logId: '3', msg: 'from realtime' }),
];
expect(mergeLogLines(fileLogLines, realtimeLogLines)).toEqual([
{ logId: '1', msg: 'from the file' },
{ logId: '2', msg: 'from realtime' },
{ logId: '3', msg: 'from realtime' },
]);
});
it('keeps identical messages that have different logIds', () => {
const merged = mergeLogLines(
[logLine({ logId: '1', msg: 'Repeated text' })],
[logLine({ logId: '2', msg: 'Repeated text' })]
);
expect(merged).toEqual([
{ logId: '1', msg: 'Repeated text' },
{ logId: '2', msg: 'Repeated text' },
]);
});
it('keeps identical lines without logId', () => {
const merged = mergeLogLines(
[logLine({ msg: 'same' }), logLine({ msg: 'same' })],
[logLine({ msg: 'same' })]
);
expect(merged).toHaveLength(3);
});
it('deduplicates within a single group', () => {
const merged = mergeLogLines([logLine({ logId: '1' }), logLine({ logId: '1' })]);
expect(merged).toHaveLength(1);
});
});
describe(groupLogLinesIntoSteps, () => {
it('groups lines by step id and labels the step with its display name', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepId: 'step-id-1', msg: 'npm ci' }),
logLine({
buildStepId: 'step-id-1',
buildStepDisplayName: 'Install dependencies',
msg: 'ok',
}),
]);
expect(Array.from(logs.keys())).toEqual(['step-id-1']);
expect(logs.get('step-id-1')).toEqual({
key: 'step-id-1',
label: 'Install dependencies',
logLines: [
{ time: undefined, msg: 'npm ci', result: undefined, marker: undefined, err: undefined },
{ time: undefined, msg: 'ok', result: undefined, marker: undefined, err: undefined },
],
});
});
it('keeps steps in the order they first appear', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepId: 'first' }),
logLine({ buildStepId: 'second' }),
logLine({ buildStepId: 'first' }),
logLine({ buildStepId: 'third' }),
]);
expect(Array.from(logs.keys())).toEqual(['first', 'second', 'third']);
});
it('falls back to the display name, then the phase, when there is no step id', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepDisplayName: 'Run fastlane' }),
logLine({ phase: 'PREPARE_CREDENTIALS' }),
]);
expect(Array.from(logs.keys())).toEqual(['Run fastlane', 'PREPARE_CREDENTIALS']);
expect(logs.get('PREPARE_CREDENTIALS')?.label).toBe('PREPARE_CREDENTIALS');
});
it('drops lines that belong to no step', () => {
const logs = groupLogLinesIntoSteps([logLine({ msg: 'a line with no step' })]);
expect(logs.size).toBe(0);
});
it('drops lines that carry no message, keeping the rest of the step', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepId: 'install', msg: undefined, marker: 'start-step' }),
logLine({ buildStepId: 'install', msg: 'npm ci' }),
]);
expect(logs.get('install')?.logLines).toEqual([
{ time: undefined, msg: 'npm ci', result: undefined, marker: undefined, err: undefined },
]);
});
it('creates no step for a line that carries no message', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepId: 'install', msg: undefined, marker: 'end-step', result: 'success' }),
]);
expect(logs.size).toBe(0);
});
it('records the result of the first end marker and ignores a later one', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepId: 'install', msg: 'skipping', marker: 'end-step', result: 'skipped' }),
logLine({ buildStepId: 'install', msg: 'retried', marker: 'end-step', result: 'success' }),
]);
expect(logs.get('install')?.result).toBe('skipped');
});
it('leaves the result undefined for a step that has not ended', () => {
const logs = groupLogLinesIntoSteps([logLine({ buildStepId: 'install', msg: 'npm ci' })]);
expect(logs.get('install')?.result).toBeUndefined();
});
it('records an end marker that carries no result and does not let a later one replace it', () => {
const logs = groupLogLinesIntoSteps([
logLine({ buildStepId: 'install', msg: 'ended', marker: 'END_PHASE' }),
logLine({ buildStepId: 'install', msg: 'retried', marker: 'END_PHASE', result: 'success' }),
]);
expect(logs.get('install')?.result).toBe('');
});
});
@@ -0,0 +1,308 @@
import { getMockWorkflowRunWithJobsFragment } from '../../../../__tests__/commands/utils';
import { RealtimeLogsTargetType, WorkflowJobStatus } from '../../../../graphql/generated';
import { RealtimeLogsClient } from '../../../../utils/centrifuge';
import { fetchAndParseLogsFromJobAsync } from '../parseLogs';
import { WorkflowJobLogsState, WorkflowRunLogsWatcher, realtimeLogsTargetForJob } from '../watcher';
import { WorkflowJobResult, WorkflowRawLogLine } from '../../types';
jest.mock('../parseLogs', () => ({
...jest.requireActual('../parseLogs'),
fetchAndParseLogsFromJobAsync: jest.fn(),
}));
function createFakeClient(): {
client: RealtimeLogsClient;
publish: (data: unknown) => void;
subscribeCalls: unknown[];
closeCount: () => number;
} {
const listeners: ((data: unknown) => void)[] = [];
const subscribeCalls: unknown[] = [];
let closed = 0;
return {
subscribeCalls,
closeCount: () => closed,
publish: data => {
listeners.forEach(listener => {
listener(data);
});
},
client: {
subscribeAsync: async (args, onPublication) => {
subscribeCalls.push(args);
listeners.push(onPublication);
return { close: () => closed++ };
},
close: () => closed++,
},
};
}
function inProgressJob(): WorkflowJobResult {
const job = getMockWorkflowRunWithJobsFragment().jobs[0];
return { ...job, status: WorkflowJobStatus.InProgress };
}
describe(realtimeLogsTargetForJob, () => {
it('targets the turtle job run when there is one', () => {
const job = { ...inProgressJob(), turtleJobRun: { id: 'job-run-id' } } as WorkflowJobResult;
expect(realtimeLogsTargetForJob(job)).toEqual({
type: RealtimeLogsTargetType.JobRun,
id: 'job-run-id',
});
});
it('falls back to the build when there is no turtle job run', () => {
const job = {
...inProgressJob(),
turtleJobRun: null,
turtleBuild: { id: 'build-id' },
} as WorkflowJobResult;
expect(realtimeLogsTargetForJob(job)).toEqual({
type: RealtimeLogsTargetType.Build,
id: 'build-id',
});
});
it('has no target when neither is present', () => {
const job = {
...inProgressJob(),
turtleJobRun: null,
turtleBuild: null,
outputs: null,
} as WorkflowJobResult;
expect(realtimeLogsTargetForJob(job)).toBeNull();
});
});
describe(WorkflowRunLogsWatcher, () => {
beforeEach(() => {
jest
.mocked(fetchAndParseLogsFromJobAsync)
.mockResolvedValue([{ logId: '1', buildStepId: 'install', msg: 'from the file' }]);
});
afterEach(() => {
jest.clearAllMocks();
});
it('subscribes once per in-progress job', async () => {
const fake = createFakeClient();
const watcher = new WorkflowRunLogsWatcher(
{} as any,
() => fake.client,
() => {}
);
const job = inProgressJob();
await watcher.syncJobsAsync([job]);
await watcher.syncJobsAsync([job]);
expect(fake.subscribeCalls).toHaveLength(1);
});
it('reports realtime logs as they arrive', async () => {
const fake = createFakeClient();
const onRealtimeLogs = jest.fn();
const watcher = new WorkflowRunLogsWatcher({} as any, () => fake.client, onRealtimeLogs);
await watcher.syncJobsAsync([inProgressJob()]);
fake.publish([{ logId: '2', buildStepId: 'install', msg: 'pushed' }]);
expect(onRealtimeLogs).toHaveBeenCalledTimes(1);
});
it('does not report a publication that carries no usable log lines', async () => {
const fake = createFakeClient();
const onRealtimeLogs = jest.fn();
const watcher = new WorkflowRunLogsWatcher({} as any, () => fake.client, onRealtimeLogs);
await watcher.syncJobsAsync([inProgressJob()]);
fake.publish([{ msg: 'no logId' }]);
expect(onRealtimeLogs).not.toHaveBeenCalled();
});
it('still fetches logs when the realtime logs client is unavailable', async () => {
const watcher = new WorkflowRunLogsWatcher(
{} as any,
() => null,
() => {}
);
await watcher.syncJobsAsync([inProgressJob()]);
expect(fetchAndParseLogsFromJobAsync).toHaveBeenCalledTimes(1);
});
it('fetches logs on every sync while in progress, and stops once the job completes', async () => {
const fake = createFakeClient();
const watcher = new WorkflowRunLogsWatcher(
{} as any,
() => fake.client,
() => {}
);
const job = inProgressJob();
await watcher.syncJobsAsync([job]);
await watcher.syncJobsAsync([job]);
expect(fetchAndParseLogsFromJobAsync).toHaveBeenCalledTimes(2);
await watcher.syncJobsAsync([{ ...job, status: WorkflowJobStatus.Success }]);
expect(fetchAndParseLogsFromJobAsync).toHaveBeenCalledTimes(2);
});
it('does not fetch logs for a job that has not started', async () => {
const fake = createFakeClient();
const watcher = new WorkflowRunLogsWatcher(
{} as any,
() => fake.client,
() => {}
);
await watcher.syncJobsAsync([{ ...inProgressJob(), status: WorkflowJobStatus.New }]);
expect(fetchAndParseLogsFromJobAsync).not.toHaveBeenCalled();
});
it('closes the subscription when a job leaves in-progress', async () => {
const fake = createFakeClient();
const watcher = new WorkflowRunLogsWatcher(
{} as any,
() => fake.client,
() => {}
);
const job = inProgressJob();
await watcher.syncJobsAsync([job]);
expect(fake.closeCount()).toBe(0);
await watcher.syncJobsAsync([{ ...job, status: WorkflowJobStatus.Failure }]);
expect(fake.closeCount()).toBe(1);
});
it('closes the client and its subscriptions', async () => {
const fake = createFakeClient();
const watcher = new WorkflowRunLogsWatcher(
{} as any,
() => fake.client,
() => {}
);
await watcher.syncJobsAsync([inProgressJob()]);
watcher.close();
expect(fake.closeCount()).toBe(2);
});
});
function fileLine(logId: string, msg: string, buildStepId = 'install'): WorkflowRawLogLine {
return { logId, buildStepId, msg };
}
function stepMessages(logsState: WorkflowJobLogsState): string[] {
return Array.from(logsState.getLogs().values()).flatMap(group =>
group.logLines.map(logLine => logLine.msg)
);
}
describe(WorkflowJobLogsState, () => {
it('hides realtime lines until the file logs reach them', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('5', 'pushed')]);
expect(stepMessages(logsState)).toEqual(['first']);
});
it('shows realtime lines once a published logId also appears in the file', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('2', 'pushed')]);
logsState.ingestFileLogLines([fileLine('1', 'first'), fileLine('2', 'pushed')]);
logsState.ingestRealtimeLogLines([fileLine('3', 'newer')]);
expect(stepMessages(logsState)).toEqual(['first', 'pushed', 'newer']);
});
it('shows realtime logs when a publication repeats a logId already in the file', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('1', 'first'), fileLine('2', 'pushed')]);
expect(stepMessages(logsState)).toEqual(['first', 'pushed']);
});
it('shows buffered realtime lines once the job is completed', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('9', 'pushed')]);
logsState.markCompleted();
logsState.markCompleted();
expect(stepMessages(logsState)).toEqual(['first', 'pushed']);
});
it('folds later publications once completion showed them', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('9', 'buffered')]);
logsState.markCompleted();
logsState.ingestRealtimeLogLines([fileLine('10', 'pushed')]);
expect(stepMessages(logsState)).toEqual(['first', 'buffered', 'pushed']);
});
it('replaces the file snapshot instead of accumulating it', () => {
const logsState = new WorkflowJobLogsState();
const keyless: WorkflowRawLogLine = { buildStepId: 'install', msg: 'submission log' };
logsState.ingestFileLogLines([keyless]);
logsState.ingestFileLogLines([keyless]);
expect(stepMessages(logsState)).toEqual(['submission log']);
});
it('drops a buffered realtime line once it lands in the file', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('2', 'pushed')]);
logsState.ingestFileLogLines([fileLine('1', 'first'), fileLine('2', 'pushed')]);
expect(stepMessages(logsState)).toEqual(['first', 'pushed']);
});
it('ignores publications that are not arrays of log lines', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
expect(logsState.ingestRealtimeLogLines({ logId: '2', msg: 'not an array' })).toBe(false);
expect(logsState.ingestRealtimeLogLines(['a string', 42, null])).toBe(false);
expect(logsState.ingestRealtimeLogLines([{ msg: 'no logId' }])).toBe(false);
expect(stepMessages(logsState)).toEqual(['first']);
});
it('deduplicates a line republished on the same channel', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('1', 'first'), fileLine('2', 'pushed')]);
logsState.ingestRealtimeLogLines([fileLine('2', 'pushed')]);
expect(stepMessages(logsState)).toEqual(['first', 'pushed']);
});
it('deduplicates a line republished later', () => {
const logsState = new WorkflowJobLogsState();
logsState.ingestFileLogLines([fileLine('1', 'first')]);
logsState.ingestRealtimeLogLines([fileLine('1', 'first'), fileLine('2', 'pushed')]);
logsState.ingestRealtimeLogLines([fileLine('3', 'newer')]);
logsState.ingestRealtimeLogLines([fileLine('2', 'pushed')]);
expect(stepMessages(logsState)).toEqual(['first', 'pushed', 'newer']);
});
});
@@ -0,0 +1,28 @@
import { WorkflowJobResult } from '../types';
import { BuildQuery } from '../../../graphql/queries/BuildQuery';
import { ExpoGraphqlClient } from '../../context/contextUtils/createGraphqlClient';
export async function fetchRawLogsForJobAsync(
state: { graphqlClient: ExpoGraphqlClient },
job: WorkflowJobResult
): Promise<string | null> {
const turtleLogFileUrl = job.turtleJobRun?.logFileUrls?.[0];
if (turtleLogFileUrl) {
const response = await fetch(turtleLogFileUrl);
return await response.text();
}
const buildId = job.outputs?.build_id;
if (!buildId) {
return null;
}
const buildFragment = await BuildQuery.byIdAsync(state.graphqlClient, buildId, {
useCache: false,
});
const firstLogFileUrl = buildFragment.logFiles?.[0];
if (!firstLogFileUrl) {
return null;
}
const response = await fetch(firstLogFileUrl);
return await response.text();
}
@@ -0,0 +1,83 @@
import { fetchRawLogsForJobAsync } from './fetchLogs';
import { WorkflowJobResult, WorkflowLogs, WorkflowRawLogLine } from '../types';
import Log from '../../../log';
import uniqBy from '../../../utils/expodash/uniqBy';
import { ExpoGraphqlClient } from '../../context/contextUtils/createGraphqlClient';
export function parseLogLines(rawLogs: string): {
logLines: WorkflowRawLogLine[];
errors: Error[];
} {
const logLines: WorkflowRawLogLine[] = [];
const errors: Error[] = [];
for (const rawLogLine of rawLogs.split('\n')) {
if (!rawLogLine) {
continue;
}
try {
logLines.push(JSON.parse(rawLogLine));
} catch (err) {
errors.push(err as Error);
}
}
return { logLines, errors };
}
export function mergeLogLines<T extends WorkflowRawLogLine>(...logLineGroups: T[][]): T[] {
return uniqBy(logLineGroups.flat().reverse(), logLine => logLine.logId ?? Symbol()).reverse();
}
export function groupLogLinesIntoSteps(
logLines: WorkflowRawLogLine[],
accumulator: WorkflowLogs = new Map()
): WorkflowLogs {
for (const logLine of logLines) {
const { buildStepDisplayName, buildStepId, phase, time, msg, result, marker, err } = logLine;
const stepKey = buildStepId ?? buildStepDisplayName ?? phase;
const stepLabel = buildStepDisplayName ?? buildStepId ?? phase;
if (!stepKey || !stepLabel || !msg) {
continue;
}
let logGroup = accumulator.get(stepKey);
if (!logGroup) {
logGroup = { key: stepKey, label: stepLabel, logLines: [] };
accumulator.set(stepKey, logGroup);
}
if (buildStepDisplayName) {
logGroup.label = buildStepDisplayName;
}
if (logGroup.result === undefined && (marker === 'end-step' || marker === 'END_PHASE')) {
logGroup.result = result ?? '';
}
logGroup.logLines.push({ time, msg, result, marker, err });
}
return accumulator;
}
export async function fetchAndParseLogsFromJobAsync(
state: { graphqlClient: ExpoGraphqlClient },
job: WorkflowJobResult
): Promise<WorkflowRawLogLine[] | null> {
const rawLogs = await fetchRawLogsForJobAsync(state, job);
if (!rawLogs) {
return null;
}
Log.debug(`rawLogs = ${JSON.stringify(rawLogs, null, 2)}`);
const { logLines, errors } = parseLogLines(rawLogs);
for (const error of errors) {
Log.debug(`Failed to parse a log line: ${error.message}`);
}
return logLines;
}
export async function fetchAndProcessLogsFromJobAsync(
state: { graphqlClient: ExpoGraphqlClient },
job: WorkflowJobResult
): Promise<WorkflowLogs | null> {
const logLines = await fetchAndParseLogsFromJobAsync(state, job);
return logLines && groupLogLinesIntoSteps(logLines);
}
@@ -0,0 +1,210 @@
import { fetchAndParseLogsFromJobAsync, groupLogLinesIntoSteps, mergeLogLines } from './parseLogs';
import { WorkflowJobResult, WorkflowLogs, WorkflowRawLogLine } from '../types';
import {
RealtimeLogsTargetInput,
RealtimeLogsTargetType,
WorkflowJobStatus,
} from '../../../graphql/generated';
import Log from '../../../log';
import { RealtimeLogsClient, RealtimeLogsSubscription } from '../../../utils/centrifuge';
import { ExpoGraphqlClient } from '../../context/contextUtils/createGraphqlClient';
import nullthrows from 'nullthrows';
type RealtimeLogLine = WorkflowRawLogLine & { logId: string };
function isRealtimeLogLine(entry: unknown): entry is RealtimeLogLine {
return (
typeof entry === 'object' &&
entry !== null &&
typeof (entry as Partial<WorkflowRawLogLine>).logId === 'string'
);
}
export class WorkflowJobLogsState {
private fileLogIds = new Set<string>();
private realtimeLogLines: RealtimeLogLine[] = [];
private realtimeLogIds = new Set<string>();
private haveFileLogsCaughtUp = false;
private realtimeLogsRevealed = false;
private groupedLogs: WorkflowLogs = new Map();
public ingestFileLogLines(logLines: WorkflowRawLogLine[]): void {
this.fileLogIds = new Set(logLines.flatMap(logLine => (logLine.logId ? [logLine.logId] : [])));
if (
!this.haveFileLogsCaughtUp &&
this.realtimeLogLines.some(logLine => this.fileLogIds.has(logLine.logId))
) {
this.haveFileLogsCaughtUp = true;
}
this.realtimeLogLines = this.realtimeLogLines.filter(
logLine => !this.fileLogIds.has(logLine.logId)
);
this.realtimeLogIds = new Set(this.realtimeLogLines.map(logLine => logLine.logId));
if (this.haveFileLogsCaughtUp) {
this.realtimeLogsRevealed = true;
}
this.groupedLogs = groupLogLinesIntoSteps(
mergeLogLines(logLines, this.realtimeLogsRevealed ? this.realtimeLogLines : [])
);
}
public ingestRealtimeLogLines(data: unknown): boolean {
if (!Array.isArray(data)) {
return false;
}
const publishedLogLines = data.filter(isRealtimeLogLine);
if (publishedLogLines.length === 0) {
return false;
}
if (
!this.haveFileLogsCaughtUp &&
publishedLogLines.some(logLine => this.fileLogIds.has(logLine.logId))
) {
this.haveFileLogsCaughtUp = true;
}
const newLogLines = publishedLogLines.filter(
logLine => !this.fileLogIds.has(logLine.logId) && !this.realtimeLogIds.has(logLine.logId)
);
for (const logLine of newLogLines) {
this.realtimeLogLines.push(logLine);
this.realtimeLogIds.add(logLine.logId);
}
if (!this.realtimeLogsRevealed) {
if (this.haveFileLogsCaughtUp) {
this.revealRealtimeLogs();
}
} else if (newLogLines.length > 0) {
groupLogLinesIntoSteps(newLogLines, this.groupedLogs);
}
return true;
}
public markCompleted(): void {
this.revealRealtimeLogs();
}
public getLogs(): WorkflowLogs {
return this.groupedLogs;
}
private revealRealtimeLogs(): void {
if (this.realtimeLogsRevealed) {
return;
}
this.realtimeLogsRevealed = true;
groupLogLinesIntoSteps(this.realtimeLogLines, this.groupedLogs);
}
}
type TrackedJob = {
logsState: WorkflowJobLogsState;
subscription: RealtimeLogsSubscription | null;
};
export function realtimeLogsTargetForJob(job: WorkflowJobResult): RealtimeLogsTargetInput | null {
if (job.turtleJobRun?.id) {
return { type: RealtimeLogsTargetType.JobRun, id: job.turtleJobRun.id };
}
const buildId = job.turtleBuild?.id ?? job.outputs?.build_id;
if (buildId) {
return { type: RealtimeLogsTargetType.Build, id: buildId };
}
return null;
}
export class WorkflowRunLogsWatcher {
private readonly trackedJobs = new Map<string, TrackedJob>();
private realtimeLogsClient?: RealtimeLogsClient | null;
constructor(
private readonly graphqlClient: ExpoGraphqlClient,
private readonly createRealtimeLogsClient: () => RealtimeLogsClient | null,
private readonly onRealtimeLogs: () => void
) {}
public async syncJobsAsync(
jobs: WorkflowJobResult[]
): Promise<Map<string, WorkflowJobLogsState>> {
await Promise.all(jobs.map(job => this.syncJobAsync(job)));
return new Map(
jobs.map(job => [
job.id,
nullthrows(
this.trackedJobs.get(job.id),
'job has to be in tracked jobs after it was synced'
).logsState,
])
);
}
public close(): void {
for (const trackedJob of this.trackedJobs.values()) {
trackedJob.subscription?.close();
trackedJob.subscription = null;
}
this.realtimeLogsClient?.close();
}
private async syncJobAsync(job: WorkflowJobResult): Promise<void> {
let trackedJob = this.trackedJobs.get(job.id);
if (!trackedJob) {
trackedJob = { logsState: new WorkflowJobLogsState(), subscription: null };
this.trackedJobs.set(job.id, trackedJob);
}
const workflowInProgress = job.status === WorkflowJobStatus.InProgress;
if (!workflowInProgress) {
trackedJob.subscription?.close();
trackedJob.subscription = null;
return;
}
await Promise.all([
trackedJob.subscription ? Promise.resolve() : this.subscribeAsync(job, trackedJob),
this.fetchLogsAsync(job, trackedJob),
]);
}
private async fetchLogsAsync(job: WorkflowJobResult, trackedJob: TrackedJob): Promise<void> {
try {
const logLines = await fetchAndParseLogsFromJobAsync(
{ graphqlClient: this.graphqlClient },
job
);
if (logLines) {
trackedJob.logsState.ingestFileLogLines(logLines);
}
} catch (err: any) {
Log.debug(`Failed to fetch logs for job ${job.id}: ${err.message}`);
}
}
private getRealtimeLogsClient(): RealtimeLogsClient | null {
if (this.realtimeLogsClient === undefined) {
this.realtimeLogsClient = this.createRealtimeLogsClient();
}
return this.realtimeLogsClient;
}
private async subscribeAsync(job: WorkflowJobResult, trackedJob: TrackedJob): Promise<void> {
const target = realtimeLogsTargetForJob(job);
if (!target) {
return;
}
const realtimeLogsClient = this.getRealtimeLogsClient();
if (!realtimeLogsClient) {
return;
}
trackedJob.subscription = await realtimeLogsClient.subscribeAsync({ target }, data => {
if (trackedJob.logsState.ingestRealtimeLogLines(data)) {
this.onRealtimeLogs();
}
});
}
}
@@ -1,11 +1,11 @@
import { Choice } from 'prompts';
import { WorkflowJobResult, WorkflowLogs } from './types';
import { fetchAndProcessLogsFromJobAsync } from './logs/parseLogs';
import {
choiceFromWorkflowJob,
choiceFromWorkflowRun,
choicesFromWorkflowLogs,
fetchAndProcessLogsFromJobAsync,
processWorkflowRuns,
} from './utils';
import { AppQuery } from '../../graphql/queries/AppQuery';
@@ -42,19 +42,26 @@ export type WorkflowRunWithJobsResult = WorkflowRunResult & {
logs?: string;
};
export type WorkflowLogLine = {
time: string;
msg: string;
export type WorkflowRawLogLine = {
logId?: string;
buildStepId?: string;
buildStepDisplayName?: string;
phase?: string;
time?: string;
msg?: string;
result?: string;
marker?: string;
err?: any;
};
export type WorkflowLogs = Map<
string,
{
key: string;
label: string;
logLines: WorkflowLogLine[];
}
>;
export type WorkflowLogLine = Pick<WorkflowRawLogLine, 'time' | 'result' | 'marker' | 'err'> &
Required<Pick<WorkflowRawLogLine, 'msg'>>;
export type StepLogs = {
key: string;
label: string;
result?: string;
logLines: WorkflowLogLine[];
};
export type WorkflowLogs = Map<string, StepLogs>;
@@ -1,7 +1,7 @@
import chalk from 'chalk';
import * as fs from 'node:fs';
import { fetchRawLogsForBuildJobAsync, fetchRawLogsForCustomJobAsync } from './fetchLogs';
import { fetchAndProcessLogsFromJobAsync } from './logs/parseLogs';
import {
WorkflowJobResult,
WorkflowLogLine,
@@ -11,20 +11,22 @@ import {
} from './types';
import {
WorkflowJobStatus,
WorkflowJobType,
WorkflowRunByIdQuery,
WorkflowRunByIdWithJobsQuery,
WorkflowRunFragment,
WorkflowRunStatus,
WorkflowRunTriggerEventType,
} from '../../graphql/generated';
import { WorkflowRunLogsWatcher } from './logs/watcher';
import { WorkflowRunQuery } from '../../graphql/queries/WorkflowRunQuery';
import Log from '../../log';
import { ora } from '../../ora';
import { ora, updateSpinnerText } from '../../ora';
import { Choice } from '../../prompts';
import formatFields from '../../utils/formatFields';
import { createRealtimeLogsClient } from '../../utils/centrifuge';
import { sleepAsync } from '../../utils/promise';
import { ExpoGraphqlClient } from '../context/contextUtils/createGraphqlClient';
import nullthrows from 'nullthrows';
export function computeTriggerInfoForWorkflowRun(run: WorkflowRunFragment): {
triggerType: WorkflowTriggerType;
@@ -85,19 +87,13 @@ export function choicesFromWorkflowLogs(
logs: WorkflowLogs
): (Choice & { name: string; status: string; logLines: WorkflowLogLine[] | undefined })[] {
return Array.from(logs.values())
.map(({ key, label, logLines }) => {
const stepStatus =
logLines?.filter(
(line: WorkflowLogLine) => line.marker === 'end-step' || line.marker === 'END_PHASE'
)[0]?.result ?? '';
return {
title: `${label} - ${stepStatus}`,
name: label,
status: stepStatus,
value: key,
logLines,
};
})
.map(({ key, label, result, logLines }) => ({
title: `${label} - ${result ?? ''}`,
name: label,
status: result ?? '',
value: key,
logLines,
}))
.filter(step => step.status !== 'skipped');
}
@@ -121,52 +117,6 @@ export function processWorkflowRuns(runs: WorkflowRunFragment[]): WorkflowRunRes
});
}
export async function fetchAndProcessLogsFromJobAsync(
state: { graphqlClient: ExpoGraphqlClient },
job: WorkflowJobResult
): Promise<WorkflowLogs | null> {
let rawLogs: string | null;
switch (job.type) {
case WorkflowJobType.Build:
case WorkflowJobType.Repack:
rawLogs = await fetchRawLogsForBuildJobAsync(state, job);
break;
default:
rawLogs = await fetchRawLogsForCustomJobAsync(job);
break;
}
if (!rawLogs) {
return null;
}
Log.debug(`rawLogs = ${JSON.stringify(rawLogs, null, 2)}`);
const logs: WorkflowLogs = new Map();
rawLogs.split('\n').forEach((line, index) => {
Log.debug(`line ${index} = ${JSON.stringify(line, null, 2)}`);
try {
const parsedLine = JSON.parse(line);
const { buildStepDisplayName, buildStepId, phase, time, msg, result, marker, err } =
parsedLine;
const stepKey = buildStepId ?? buildStepDisplayName ?? phase;
const stepLabel = buildStepDisplayName ?? buildStepId ?? phase;
if (stepKey && stepLabel) {
if (!logs.has(stepKey)) {
logs.set(stepKey, {
key: stepKey,
label: stepLabel,
logLines: [],
});
}
const logGroup = logs.get(stepKey)!;
if (buildStepDisplayName) {
logGroup.label = buildStepDisplayName;
}
logGroup.logLines.push({ time, msg, result, marker, err });
}
} catch {}
});
return logs;
}
function descriptionForJobStatus(status: WorkflowJobStatus): string {
switch (status) {
case WorkflowJobStatus.New:
@@ -188,82 +138,106 @@ function descriptionForJobStatus(status: WorkflowJobStatus): string {
}
}
export async function infoForActiveWorkflowRunAsync(
function isJobCompleted(status: WorkflowJobStatus): boolean {
switch (status) {
case WorkflowJobStatus.Success:
case WorkflowJobStatus.Failure:
case WorkflowJobStatus.Canceled:
case WorkflowJobStatus.Skipped:
return true;
case WorkflowJobStatus.New:
case WorkflowJobStatus.InProgress:
case WorkflowJobStatus.ActionRequired:
case WorkflowJobStatus.PendingCancel:
return false;
}
}
type WorkflowRunWithJobs = WorkflowRunByIdWithJobsQuery['workflowRuns']['byId'];
type JobWithLogs = {
job: WorkflowRunWithJobs['jobs'][number];
logs: WorkflowLogs;
};
function stepLogTail(
step: { logLines?: WorkflowLogLine[] },
maxLogLines: number // -1 means no limit
): string[] {
const logLines = step.logLines ?? [];
const tail = maxLogLines === -1 ? logLines : logLines.slice(-maxLogLines);
return tail.map(line => line.msg);
}
export async function logsForFailedWorkflowRunAsync(
graphqlClient: ExpoGraphqlClient,
workflowRun: WorkflowRunByIdWithJobsQuery['workflowRuns']['byId'],
workflowRun: WorkflowRunWithJobs
): Promise<JobWithLogs[]> {
const jobs = workflowRun.jobs.filter(job => job.status === WorkflowJobStatus.Failure);
return await Promise.all(
jobs.map(async job => {
const maybeLogs = await fetchAndProcessLogsFromJobAsync({ graphqlClient }, job);
return { job, logs: maybeLogs ?? new Map() };
})
);
}
export function formatActiveWorkflowRun(
jobLogs: JobWithLogs[],
maxLogLines: number = 5 // -1 means no limit
): Promise<string> {
const statusLines = [];
): string {
const statusValues = [];
for (const job of workflowRun.jobs) {
for (const { job, logs } of jobLogs) {
statusValues.push({ label: '', value: '' });
statusValues.push({ label: ' Job', value: job.name });
statusValues.push({ label: ' Status', value: descriptionForJobStatus(job.status) });
if (job.status !== WorkflowJobStatus.InProgress) {
continue;
}
const logs = await fetchAndProcessLogsFromJobAsync({ graphqlClient }, job);
const steps = logs ? choicesFromWorkflowLogs(logs) : [];
if (steps.length > 0) {
const currentStep = steps[steps.length - 1];
statusValues.push({ label: ' Current step', value: currentStep.name });
if (currentStep?.logLines?.length) {
statusValues.push({ label: ' Current logs', value: '' });
const currentLogs =
currentStep.logLines
?.map(line => line.msg)
.filter((_, index) => {
if (maxLogLines === -1) {
return true;
}
return index > (currentStep.logLines?.length ?? 0) - maxLogLines;
}) ?? [];
for (const log of currentLogs) {
statusValues.push({ label: '', value: log });
}
}
}
}
statusValues.push({ label: '', value: '' });
statusLines.push(formatFields(statusValues));
return statusLines.join('\n');
}
export async function infoForFailedWorkflowRunAsync(
graphqlClient: ExpoGraphqlClient,
workflowRun: WorkflowRunByIdWithJobsQuery['workflowRuns']['byId'],
maxLogLines: number = -1 // -1 means no limit
): Promise<string> {
const statusLines = [];
const statusValues = [];
const logLinesToKeep = maxLogLines === -1 ? Infinity : maxLogLines;
for (const job of workflowRun.jobs) {
if (job.status !== WorkflowJobStatus.Failure) {
const steps = choicesFromWorkflowLogs(logs);
if (steps.length === 0) {
continue;
}
const logs = await fetchAndProcessLogsFromJobAsync({ graphqlClient }, job);
const steps = logs ? choicesFromWorkflowLogs(logs) : [];
statusValues.push({ label: '', value: '' });
statusValues.push({ label: ' Failed job', value: job.name });
if (steps.length > 0) {
const failedStep = steps.find(step => step.status === 'fail' || step.status === 'failed');
if (failedStep) {
const logs = failedStep.logLines?.map(line => line.msg).slice(-logLinesToKeep) ?? [];
statusValues.push({ label: ' Failed step', value: failedStep.name });
statusValues.push({
label: ' Logs for failed step',
value: '',
});
for (const log of logs) {
statusValues.push({ label: '', value: log });
}
const currentStep = steps[steps.length - 1];
statusValues.push({ label: ' Current step', value: currentStep.name });
if (currentStep.logLines?.length) {
statusValues.push({ label: ' Current logs', value: '' });
for (const log of stepLogTail(currentStep, maxLogLines)) {
statusValues.push({ label: '', value: log });
}
}
}
statusValues.push({ label: '', value: '' });
statusLines.push(formatFields(statusValues));
return statusLines.join('\n');
return formatFields(statusValues);
}
export function formatFailedWorkflowRun(
jobLogs: JobWithLogs[],
maxLogLines: number = -1 // -1 means no limit
): string {
const statusValues = [];
for (const { job, logs } of jobLogs) {
statusValues.push({ label: '', value: '' });
statusValues.push({ label: ' Failed job', value: job.name });
const steps = choicesFromWorkflowLogs(logs);
const failedStep = steps.find(step => step.status === 'fail' || step.status === 'failed');
if (!failedStep) {
continue;
}
statusValues.push({ label: ' Failed step', value: failedStep.name });
statusValues.push({ label: ' Logs for failed step', value: '' });
for (const log of stepLogTail(failedStep, maxLogLines)) {
statusValues.push({ label: '', value: log });
}
}
statusValues.push({ label: '', value: '' });
return formatFields(statusValues);
}
export async function fileExistsAsync(filePath: string): Promise<boolean> {
return await fs.promises
.access(filePath, fs.constants.F_OK)
@@ -312,65 +286,111 @@ export async function showWorkflowStatusAsync(
stream: spinnerUsesStdErr ? process.stderr : process.stdout,
text: '',
}).start();
spinner.prefixText = chalk`{bold.yellow Workflow run is waiting to start:}`;
updateSpinnerText(spinner, {
prefixText: chalk`{bold.yellow Workflow run is waiting to start:}`,
});
let failedFetchesCount = 0;
while (true) {
try {
const workflowRun = await WorkflowRunQuery.withJobsByIdAsync(graphqlClient, workflowRunId, {
useCache: false,
});
failedFetchesCount = 0;
switch (workflowRun.status) {
case WorkflowRunStatus.New:
break;
case WorkflowRunStatus.InProgress: {
spinner.prefixText = chalk`{bold.green Workflow run is in progress:}`;
spinner.text = await infoForActiveWorkflowRunAsync(graphqlClient, workflowRun, 5);
break;
}
case WorkflowRunStatus.ActionRequired:
spinner.prefixText = chalk`{bold.yellow Workflow run is waiting for action:}`;
break;
case WorkflowRunStatus.Canceled:
spinner.prefixText = chalk`{bold.yellow Workflow has been canceled.}`;
spinner.stopAndPersist();
return workflowRun;
case WorkflowRunStatus.Failure: {
spinner.prefixText = chalk`{bold.red Workflow has failed.}`;
const failedInfo = await infoForFailedWorkflowRunAsync(graphqlClient, workflowRun, 30);
spinner.fail(failedInfo);
return workflowRun;
}
case WorkflowRunStatus.Success:
spinner.prefixText = chalk`{bold.green Workflow has completed successfully.}`;
spinner.text = '';
spinner.succeed('');
return workflowRun;
}
if (!waitForCompletion) {
if (spinner.isSpinning) {
spinner.stopAndPersist();
}
return workflowRun;
}
} catch {
spinner.text = '⚠ Failed to fetch the workflow run status. Check your network connection.';
failedFetchesCount += 1;
if (failedFetchesCount > 6) {
spinner.fail('Failed to fetch the workflow run status 6 times in a row. Aborting wait.');
process.exit(workflowRunExitCodes.WAIT_ABORTED);
}
const watcher = new WorkflowRunLogsWatcher(
graphqlClient,
() => createRealtimeLogsClient(graphqlClient),
() => {
renderActiveWorkflowRun();
}
);
let failedFetchesCount = 0;
let renderActiveWorkflowRun = (): void => {};
await sleepAsync(10 /* seconds */ * 1000 /* milliseconds */);
try {
while (true) {
try {
const workflowRun = await WorkflowRunQuery.withJobsByIdAsync(graphqlClient, workflowRunId, {
useCache: false,
});
failedFetchesCount = 0;
switch (workflowRun.status) {
case WorkflowRunStatus.New:
break;
case WorkflowRunStatus.InProgress: {
updateSpinnerText(spinner, {
prefixText: chalk`{bold.green Workflow run is in progress:}`,
});
const logsStates = await watcher.syncJobsAsync(workflowRun.jobs);
for (const job of workflowRun.jobs) {
if (isJobCompleted(job.status)) {
nullthrows(
logsStates.get(job.id),
'syncJobsAsync must have been called before markCompleted'
).markCompleted();
}
}
renderActiveWorkflowRun = () => {
const text = formatActiveWorkflowRun(
workflowRun.jobs.map(job => ({
job,
logs: nullthrows(
logsStates.get(job.id),
'syncJobsAsync must have been called before getLogs'
).getLogs(),
}))
);
updateSpinnerText(spinner, { text });
};
renderActiveWorkflowRun();
break;
}
case WorkflowRunStatus.ActionRequired:
updateSpinnerText(spinner, {
prefixText: chalk`{bold.yellow Workflow run is waiting for action:}`,
});
break;
case WorkflowRunStatus.Canceled:
updateSpinnerText(spinner, {
prefixText: chalk`{bold.yellow Workflow has been canceled.}`,
});
spinner.stopAndPersist();
return workflowRun;
case WorkflowRunStatus.Failure: {
updateSpinnerText(spinner, {
prefixText: chalk`{bold.red Workflow has failed.}`,
});
const jobLogs = await logsForFailedWorkflowRunAsync(graphqlClient, workflowRun);
spinner.fail(formatFailedWorkflowRun(jobLogs, 30));
return workflowRun;
}
case WorkflowRunStatus.Success:
updateSpinnerText(spinner, {
prefixText: chalk`{bold.green Workflow has completed successfully.}`,
});
spinner.succeed('');
return workflowRun;
}
if (!waitForCompletion) {
if (spinner.isSpinning) {
spinner.stopAndPersist();
}
return workflowRun;
}
} catch {
updateSpinnerText(spinner, {
text: '⚠ Failed to fetch the workflow run status. Check your network connection.',
});
failedFetchesCount += 1;
if (failedFetchesCount > 6) {
spinner.fail('Failed to fetch the workflow run status 6 times in a row. Aborting wait.');
process.exit(workflowRunExitCodes.WAIT_ABORTED);
}
}
await sleepAsync(10 /* seconds */ * 1000 /* milliseconds */);
}
} finally {
watcher.close();
}
}
@@ -17,7 +17,6 @@ jest.mock('fs-extra', () => ({
jest.mock('../../../prompts', () => ({
promptAsync: jest.fn(),
}));
jest.mock('../../../onboarding/git');
jest.mock('../../../onboarding/runCommand');
jest.mock('../../../graphql/mutations/AppMutation');
jest.mock('../../../project/expoConfig');
+17 -3
View File
@@ -6,7 +6,7 @@ import { getProjectDashboardUrl } from '../../build/utils/url';
import EasCommand from '../../commandUtils/EasCommand';
import { ExpoGraphqlClient } from '../../commandUtils/context/contextUtils/createGraphqlClient';
import {
cloneTemplateAsync,
downloadTemplateAsync,
initializeGitRepositoryAsync,
installProjectDependenciesAsync,
} from '../../commandUtils/new/commands';
@@ -14,10 +14,12 @@ import {
generateProjectConfigAsync,
promptForProjectAccountAsync,
} from '../../commandUtils/new/configs';
import { resolveTemplateAsync } from '../../commandUtils/new/sdkVersion';
import {
copyProjectTemplatesAsync,
generateAppConfigAsync,
generateEasConfigAsync,
removeTemplateFilesAsync,
updatePackageJsonAsync,
updateReadmeAsync,
} from '../../commandUtils/new/projectFiles';
@@ -108,7 +110,9 @@ export async function generateProjectFilesAsync(
await generateEasConfigAsync(projectDir);
await updatePackageJsonAsync(projectDir);
await updatePackageJsonAsync(projectDir, app.slug);
await removeTemplateFilesAsync(projectDir);
await copyProjectTemplatesAsync(projectDir);
@@ -146,6 +150,9 @@ export default class New extends EasCommand {
options: [...PACKAGE_MANAGERS] as const,
default: 'npm',
})(),
'sdk-version': Flags.string({
description: 'Expo SDK version to use for the new project (e.g. 57, or "latest")',
}),
};
static override contextDefinition = {
@@ -167,13 +174,20 @@ export default class New extends EasCommand {
Log.log(`👋 Welcome to Expo, ${actor.username}!`);
// Resolve the template before any prompts when --sdk-version is provided,
// so an invalid version fails fast.
let template = flags['sdk-version']
? await resolveTemplateAsync({ sdkVersion: flags['sdk-version'] })
: null;
const {
projectName,
projectDirectory: targetProjectDirectory,
projectAccount,
} = await generateConfigsAsync(args, actor, graphqlClient);
const projectDirectory = await cloneTemplateAsync(targetProjectDirectory);
template ??= await resolveTemplateAsync({});
const projectDirectory = await downloadTemplateAsync(targetProjectDirectory, template);
const packageManager = flags['package-manager'];
await installProjectDependenciesAsync(projectDirectory, packageManager);
@@ -48,6 +48,7 @@ function makeDeviceRunSession(overrides: Partial<DeviceRunSessionById> = {}): De
return {
id: 'session-123',
name: null,
tags: [],
status: DeviceRunSessionStatus.InProgress,
type: DeviceRunSessionType.AgentDevice,
platform: AppPlatform.Ios,
@@ -122,7 +123,13 @@ describe(SimulatorGet, () => {
}
it('emits JSON when --json is passed', async () => {
const session = makeDeviceRunSession();
const session = makeDeviceRunSession({
type: DeviceRunSessionType.ServeSim,
remoteConfig: {
__typename: 'ServeSimRunSessionRemoteConfig',
previewUrl: 'https://preview.example.com',
},
});
mockByIdAsync.mockResolvedValue(session);
const { command, getContextAsync } = createCommand(['--id', 'session-123', '--json']);
@@ -137,7 +144,8 @@ describe(SimulatorGet, () => {
expect(mockPrintJsonOnlyOutput).toHaveBeenCalledWith({
id: 'session-123',
name: undefined,
type: 'agent-device',
tags: [],
type: 'web-preview-only',
status: DeviceRunSessionStatus.InProgress,
platform: AppPlatform.Ios,
createdAt: '2025-01-01T00:00:00.000Z',
@@ -110,6 +110,7 @@ function makeDeviceRunSession(overrides: Partial<DeviceRunSessionById> = {}): De
return {
id: 'session-123',
name: null,
tags: [],
status: DeviceRunSessionStatus.InProgress,
type: DeviceRunSessionType.AgentDevice,
platform: AppPlatform.Ios,
@@ -332,6 +333,33 @@ describe(Simulator, () => {
);
});
it('creates a WebPreviewOnly session for --type web-preview-only', async () => {
mockByIdAsync.mockResolvedValue(
makeDeviceRunSession({
type: DeviceRunSessionType.WebPreviewOnly,
remoteConfig: {
__typename: 'WebPreviewOnlyRunSessionRemoteConfig',
previewUrl: 'https://preview.example.test',
},
})
);
const { command } = createCommand([
'--platform',
'ios',
'--type',
'web-preview-only',
'--non-interactive',
]);
await command.runAsync();
expect(mockCreateDeviceRunSessionAsync).toHaveBeenCalledWith(
graphqlClient,
expect.objectContaining({ type: DeviceRunSessionType.WebPreviewOnly })
);
expect(Log.log).toHaveBeenCalledWith(expect.stringContaining('https://preview.example.test'));
});
it('overwrites .env.eas-simulator when outputting dotenv and the file exists', async () => {
const { command } = createCommand([
'--platform',
@@ -586,6 +614,19 @@ describe(Simulator, () => {
);
});
it('warns that Android emulator support is still in development before creating a session', async () => {
const { command } = createCommand(['--platform', 'android', '--non-interactive']);
await command.runAsync();
expect(Log.warn).toHaveBeenCalledWith(
'Android emulator support in EAS Simulator is still in development. Some features available on iOS may not work on Android yet. Full parity with iOS is coming soon.'
);
expect(jest.mocked(Log.warn).mock.invocationCallOrder[0]).toBeLessThan(
mockCreateDeviceRunSessionAsync.mock.invocationCallOrder[0]
);
});
it.each([
['ios', AppPlatform.Ios],
['android', AppPlatform.Android],
@@ -669,6 +710,36 @@ describe(Simulator, () => {
);
});
it('forwards repeated tags', async () => {
const { command } = createCommand([
'--platform',
'ios',
'--non-interactive',
'--tag',
'variant:pro',
'--tag',
' nightly ',
]);
await command.runAsync();
expect(mockCreateDeviceRunSessionAsync).toHaveBeenCalledWith(
graphqlClient,
expect.objectContaining({ tags: ['variant:pro', 'nightly'] })
);
});
it('omits tags when every tag is blank', async () => {
const { command } = createCommand(['--platform', 'ios', '--non-interactive', '--tag', ' ']);
await command.runAsync();
expect(mockCreateDeviceRunSessionAsync).toHaveBeenCalledWith(
graphqlClient,
expect.not.objectContaining({ tags: expect.anything() })
);
});
it.each([
['--launch-arg', '--uitesting'],
['--open-url', 'exp://example.test'],
@@ -42,6 +42,7 @@ function makeSession(overrides: Partial<DeviceRunSessionNode> = {}): DeviceRunSe
return {
id: 'session-123',
name: null,
tags: [],
status: DeviceRunSessionStatus.InProgress,
type: DeviceRunSessionType.AgentDevice,
platform: AppPlatform.Ios,
@@ -111,7 +112,7 @@ describe(SimulatorList, () => {
}
it('emits JSON when --json is passed', async () => {
const session = makeSession();
const session = makeSession({ type: DeviceRunSessionType.ServeSim });
mockListByAppIdAsync.mockResolvedValue(makeConnection([session]));
const { command, getContextAsync } = createCommand(['--json']);
@@ -132,7 +133,8 @@ describe(SimulatorList, () => {
{
id: 'session-123',
name: undefined,
type: 'agent-device',
tags: [],
type: 'web-preview-only',
status: DeviceRunSessionStatus.InProgress,
platform: AppPlatform.Ios,
createdAt: '2025-01-01T00:00:00.000Z',
@@ -162,10 +164,16 @@ describe(SimulatorList, () => {
'new',
'--type',
'appium',
'--type',
'web-preview-only',
'--platform',
'ios',
'--name',
'checkout',
'--tag',
'variant:pro',
'--tag',
'nightly',
'--limit',
'25',
'--after',
@@ -179,13 +187,33 @@ describe(SimulatorList, () => {
after: 'page-cursor',
filter: {
statuses: [DeviceRunSessionStatus.InProgress, DeviceRunSessionStatus.New],
types: [DeviceRunSessionType.Appium],
types: [DeviceRunSessionType.Appium, DeviceRunSessionType.WebPreviewOnly],
platforms: [AppPlatform.Ios],
name: 'checkout',
tags: ['variant:pro', 'nightly'],
},
});
});
it('prints a tags row for every session, tagged or not', async () => {
mockListByAppIdAsync.mockResolvedValue(
makeConnection([
makeSession({ id: 'session-1', tags: ['variant:pro', 'nightly'] }),
makeSession({ id: 'session-2' }),
])
);
const { command } = createCommand([]);
await command.runAsync();
const printed = jest
.mocked(Log.log)
.mock.calls.map(([entry]) => String(entry))
.join('\n');
expect(printed).toContain('Tags: variant:pro, nightly');
expect(printed).toContain('Tags: none');
});
it('prints a name row for every session, named or not', async () => {
mockListByAppIdAsync.mockResolvedValue(
makeConnection([
@@ -87,6 +87,7 @@ export default class SimulatorGet extends EasCommand {
printJsonOnlyOutput({
id: session.id,
name: session.name ?? undefined,
tags: session.tags,
type: deviceRunSessionTypeToFlagValue(session.type),
status: session.status,
platform: session.platform,
@@ -123,6 +124,7 @@ function formatSessionFields(session: DeviceRunSessionById, deviceRunSessionUrl:
return formatFields([
{ label: 'ID', value: session.id },
{ label: 'Name', value: formatNullable(session.name) },
{ label: 'Tags', value: session.tags.length > 0 ? session.tags.join(', ') : 'none' },
{ label: 'Type', value: session.type },
{ label: 'Status', value: session.status },
{ label: 'Platform', value: session.platform },
@@ -71,6 +71,11 @@ export default class Simulator extends EasCommand {
description:
'Human-readable name for the simulator session, shown in eas simulator:list and on expo.dev. Defaults to unnamed.',
}),
tag: Flags.string({
description:
'Label used to group simulator sessions, for example one per app variant. Repeat for multiple tags. Stored lowercased.',
multiple: true,
}),
device: Flags.string({
description:
'Virtual device to start for the session. On iOS, a Simulator device name or UDID (e.g. "iPhone 16 Pro"). On Android, an AVD hardware profile id (e.g. "pixel_7"). Defaults to a device chosen by the runner.',
@@ -103,7 +108,8 @@ export default class Simulator extends EasCommand {
'Expo or development-client URL to open in the installed application after it launches.',
}),
type: Flags.option({
description: 'Type of simulator session to create',
description:
'Type of simulator session to create. All session types include a web preview. agent-device, appium, and argent also include an automation interface; web-preview-only includes no automation interface.',
options: Object.values(DEVICE_RUN_SESSION_TYPE_FLAG_VALUES),
default: DEVICE_RUN_SESSION_TYPE_FLAG_VALUES[DeviceRunSessionType.AgentDevice],
})(),
@@ -178,6 +184,7 @@ export default class Simulator extends EasCommand {
// The server rejects blank names, so trim here and treat a whitespace-only
// --name as if it had been omitted rather than surfacing a validation error.
const name = flags.name?.trim() || undefined;
const tags = flags.tag?.map(tag => tag.trim()).filter(tag => tag.length > 0);
const deviceIdentifier = flags.device?.trim() || undefined;
const buildId = flags['build-id']?.trim() || undefined;
const applicationArchiveUrlFromFlag = flags['application-archive-url']?.trim() || undefined;
@@ -211,6 +218,12 @@ export default class Simulator extends EasCommand {
}
const platform = await resolvePlatformAsync(flags.platform, nonInteractive);
if (platform === AppPlatform.Android) {
Log.warn(
'Android emulator support in EAS Simulator is still in development. Some features available on iOS may not work on Android yet. Full parity with iOS is coming soon.'
);
Log.newLine();
}
const expoGoSdkVersion = flags['expo-go']
? await resolveExpoGoSdkVersionAsync({ projectDir, sdkVersion: sdkVersionFromFlag })
: undefined;
@@ -232,6 +245,7 @@ export default class Simulator extends EasCommand {
const session = await DeviceRunSessionMutation.createDeviceRunSessionAsync(graphqlClient, {
appId: projectId,
name,
...(tags?.length ? { tags } : {}),
platform,
type: DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE[flags.type],
packageVersion: flags['package-version'],
@@ -61,7 +61,8 @@ export default class SimulatorList extends EasCommand {
multiple: true,
})(),
type: Flags.option({
description: 'Filter by session type (repeatable)',
description:
'Filter by session type (repeatable). All session types include a web preview. agent-device, appium, and argent also include an automation interface; web-preview-only includes no automation interface.',
options: Object.values(DEVICE_RUN_SESSION_TYPE_FLAG_VALUES),
multiple: true,
})(),
@@ -73,6 +74,10 @@ export default class SimulatorList extends EasCommand {
name: Flags.string({
description: 'Filter by session name (case-insensitive prefix match)',
}),
tag: Flags.string({
description: 'Filter by tag (repeatable). A session must carry every tag listed.',
multiple: true,
}),
limit: getLimitFlagWithCustomValues({ defaultTo: DEFAULT_LIMIT, limit: MAX_LIMIT }),
after: Flags.string({
description:
@@ -114,6 +119,9 @@ export default class SimulatorList extends EasCommand {
if (flags.name) {
filter.name = flags.name;
}
if (flags.tag && flags.tag.length > 0) {
filter.tags = flags.tag;
}
const limit = flags.limit ?? DEFAULT_LIMIT;
@@ -139,6 +147,7 @@ export default class SimulatorList extends EasCommand {
sessions: sessions.map(session => ({
id: session.id,
name: session.name ?? undefined,
tags: session.tags,
type: deviceRunSessionTypeToFlagValue(session.type),
status: session.status,
platform: session.platform,
@@ -172,6 +181,7 @@ export default class SimulatorList extends EasCommand {
const lines = [
`ID: ${session.id}`,
`Name: ${session.name ?? 'null'}`,
`Tags: ${session.tags.length > 0 ? session.tags.join(', ') : 'none'}`,
`Type: ${session.type}`,
`Status: ${session.status}`,
`Platform: ${session.platform}`,
@@ -0,0 +1,171 @@
import spawnAsync from '@expo/spawn-async';
import { Config } from '@oclif/core';
import { fileExistsAsync, maybeReadStdinAsync } from '../../../commandUtils/workflow/utils';
import { WorkflowRevisionMutation } from '../../../graphql/mutations/WorkflowRevisionMutation';
import { WorkflowRunMutation } from '../../../graphql/mutations/WorkflowRunMutation';
import Log from '../../../log';
import { getOwnerAccountForProjectIdAsync } from '../../../project/projectUtils';
import { uploadAccountScopedFileAsync } from '../../../project/uploadAccountScopedFileAsync';
import { uploadAccountScopedProjectSourceAsync } from '../../../project/uploadAccountScopedProjectSourceAsync';
import { WorkflowFile } from '../../../utils/workflowFile';
import WorkflowRun from '../run';
jest.mock('@expo/spawn-async', () => ({ __esModule: true, default: jest.fn() }));
jest.mock('../../../graphql/mutations/WorkflowRevisionMutation');
jest.mock('../../../graphql/mutations/WorkflowRunMutation');
jest.mock('../../../project/projectUtils');
jest.mock('../../../project/uploadAccountScopedFileAsync');
jest.mock('../../../project/uploadAccountScopedProjectSourceAsync');
jest.mock('../../../utils/workflowFile');
jest.mock('../../../commandUtils/workflow/utils', () => ({
...jest.requireActual('../../../commandUtils/workflow/utils'),
fileExistsAsync: jest.fn(),
maybeReadStdinAsync: jest.fn(),
}));
jest.mock('../../../log', () => ({
__esModule: true,
default: {
log: jest.fn(),
newLine: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
debug: jest.fn(),
},
link: jest.fn((url: string) => url),
}));
const mockSpawn = jest.mocked(spawnAsync);
const mockValidate = jest.mocked(WorkflowRevisionMutation.validateWorkflowYamlConfigAsync);
const mockGetOrCreateRevision = jest.mocked(
WorkflowRevisionMutation.getOrCreateWorkflowRevisionFromGitRefAsync
);
const mockCreateFromGitRef = jest.mocked(WorkflowRunMutation.createWorkflowRunFromGitRefAsync);
const mockCreateRun = jest.mocked(WorkflowRunMutation.createWorkflowRunAsync);
const mockGetOwner = jest.mocked(getOwnerAccountForProjectIdAsync);
const mockUploadProject = jest.mocked(uploadAccountScopedProjectSourceAsync);
const mockUploadFile = jest.mocked(uploadAccountScopedFileAsync);
const mockReadWorkflow = jest.mocked(WorkflowFile.readWorkflowFileContentsAsync);
const mockFileExists = jest.mocked(fileExistsAsync);
const mockReadStdin = jest.mocked(maybeReadStdinAsync);
describe('WorkflowRun --ssh', () => {
let mockConfig: Config;
let exitSpy: jest.SpyInstance;
beforeAll(async () => {
mockConfig = new Config({ root: __dirname });
mockConfig.runHook = async () => ({ failures: [], successes: [] });
});
beforeEach(() => {
jest.clearAllMocks();
exitSpy = jest.spyOn(process, 'exit').mockImplementation(((code?: number) => {
throw new Error(`process.exit:${code ?? 0}`);
}) as never);
mockReadStdin.mockResolvedValue(null);
mockGetOwner.mockResolvedValue({ id: 'acct-1', name: 'testuser' } as never);
mockValidate.mockResolvedValue(undefined as never);
mockReadWorkflow.mockResolvedValue({
filePath: '.eas/workflows/test.yml',
yamlConfig: 'jobs:\n a:\n steps: []\n',
} as never);
mockFileExists.mockResolvedValue(false);
mockUploadProject.mockResolvedValue({ projectArchiveBucketKey: 'archive-key' } as never);
mockUploadFile.mockResolvedValue({ fileBucketKey: 'file-key' } as never);
mockCreateRun.mockResolvedValue({ id: 'run-1' } as never);
mockCreateFromGitRef.mockResolvedValue({ id: 'run-git-1' } as never);
mockSpawn.mockResolvedValue({ output: ['abc123\n'] } as never);
mockGetOrCreateRevision.mockResolvedValue({
id: 'rev-1',
yamlConfig: 'jobs:\n a:\n steps: []\n',
} as never);
});
afterEach(() => {
exitSpy.mockRestore();
});
function createCommand(flags: {
ssh?: boolean;
'ssh-idle-timeout'?: number;
ref?: string;
}): WorkflowRun {
const command = new WorkflowRun(['.eas/workflows/test.yml'], mockConfig);
// @ts-expect-error getContextAsync/parse are protected
jest.spyOn(command, 'getContextAsync').mockResolvedValue({
getDynamicPrivateProjectConfigAsync: async () => ({
projectId: 'app-1',
exp: { slug: 'testapp' },
}),
loggedIn: { graphqlClient: {} },
vcsClient: { getRootPathAsync: async () => '/test/project' },
projectDir: '/test/project',
});
// @ts-expect-error parse is protected on the oclif base
jest.spyOn(command, 'parse').mockResolvedValue({
flags: {
json: false,
wait: false,
'non-interactive': true,
input: undefined,
ref: flags.ref,
ssh: flags.ssh ?? false,
'ssh-idle-timeout': flags['ssh-idle-timeout'],
},
args: { file: '.eas/workflows/test.yml' },
});
return command;
}
it('passes ssh input into createWorkflowRunAsync and logs that SSH is enabled', async () => {
await expect(createCommand({ ssh: true, 'ssh-idle-timeout': 120 }).runAsync()).rejects.toThrow(
'process.exit:0'
);
expect(mockCreateRun).toHaveBeenCalledWith(
{},
expect.objectContaining({
workflowRunInput: expect.objectContaining({
ssh: { idleTimeoutSeconds: 120 },
}),
})
);
expect(Log.log).toHaveBeenCalledWith(
'SSH enabled. Each VM job logs the `eas workflow:ssh` command to connect.'
);
expect(exitSpy).toHaveBeenCalledWith(0);
});
it('does not log the SSH banner when --ssh is unset', async () => {
await expect(createCommand({ ssh: false }).runAsync()).rejects.toThrow('process.exit:0');
expect(mockCreateRun).toHaveBeenCalledWith(
{},
expect.objectContaining({
workflowRunInput: expect.objectContaining({
ssh: null,
}),
})
);
expect(Log.log).not.toHaveBeenCalledWith(
'SSH enabled. Each VM job logs the `eas workflow:ssh` command to connect.'
);
});
it('passes ssh input into createWorkflowRunFromGitRefAsync when --ref is set', async () => {
await expect(createCommand({ ssh: true, ref: 'main' }).runAsync()).rejects.toThrow(
'process.exit:0'
);
expect(mockCreateFromGitRef).toHaveBeenCalledWith(
{},
expect.objectContaining({
ssh: { idleTimeoutSeconds: undefined },
})
);
expect(Log.log).toHaveBeenCalledWith(
'SSH enabled. Each VM job logs the `eas workflow:ssh` command to connect.'
);
});
});
@@ -3,6 +3,31 @@ import {
parseJsonInputs,
parseWorkflowInputsFromYaml,
} from '../../../commandUtils/workflow/inputs';
import { resolveWorkflowRunSshInput } from '../run';
describe(resolveWorkflowRunSshInput, () => {
it('is null when --ssh is not set', () => {
expect(resolveWorkflowRunSshInput({ ssh: false, idleTimeoutSeconds: undefined })).toBeNull();
});
it('forwards ssh with an undefined timeout when --ssh is set alone', () => {
expect(resolveWorkflowRunSshInput({ ssh: true, idleTimeoutSeconds: undefined })).toEqual({
idleTimeoutSeconds: undefined,
});
});
it('forwards the requested idle timeout when both flags are set', () => {
expect(resolveWorkflowRunSshInput({ ssh: true, idleTimeoutSeconds: 60 })).toEqual({
idleTimeoutSeconds: 60,
});
});
it('rejects --ssh-idle-timeout without --ssh', () => {
expect(() => resolveWorkflowRunSshInput({ ssh: false, idleTimeoutSeconds: 60 })).toThrow(
'--ssh-idle-timeout requires --ssh.'
);
});
});
describe('parseInputs', () => {
it('should parse single key=value pair', () => {
@@ -0,0 +1,387 @@
import spawnAsync from '@expo/spawn-async';
import { Config } from '@oclif/core';
import fs from 'node:fs';
import { WorkflowJobSshQuery } from '../../../graphql/queries/WorkflowJobSshQuery';
import Log from '../../../log';
import { sleepAsync } from '../../../utils/promise';
import WorkflowSsh, {
parseSshArgv,
resolveSshConnectStatus,
splitConnectionHost,
sshHostAliasForResource,
terminalSshStatusMessage,
} from '../ssh';
jest.mock('@expo/spawn-async', () => ({ __esModule: true, default: jest.fn() }));
jest.mock('../../../graphql/queries/WorkflowJobSshQuery');
jest.mock('../../../log', () => ({
__esModule: true,
default: { log: jest.fn(), error: jest.fn(), newLine: jest.fn() },
}));
jest.mock('../../../ora', () => ({
ora: jest.fn(() => {
const spinner = { start: jest.fn(), fail: jest.fn(), succeed: jest.fn(), stop: jest.fn() };
spinner.start.mockReturnValue(spinner);
return spinner;
}),
}));
jest.mock('../../../utils/promise', () => ({ sleepAsync: jest.fn() }));
jest.mock('node:fs', () => {
const actual = jest.requireActual('node:fs');
return {
...actual,
promises: { ...actual.promises, mkdtemp: jest.fn(), writeFile: jest.fn(), rm: jest.fn() },
};
});
describe(parseSshArgv, () => {
it('takes the first token as the resource id and the rest as the passthrough command', () => {
expect(parseSshArgv(['job-id', 'ls', '-la'])).toEqual({
showConnect: false,
resourceId: 'job-id',
command: ['ls', '-la'],
});
});
it('detects --show-connect before the id and strips it', () => {
expect(parseSshArgv(['--show-connect', 'job-id'])).toEqual({
showConnect: true,
resourceId: 'job-id',
command: [],
});
});
it('detects --show-connect=true before the id', () => {
expect(parseSshArgv(['--show-connect=true', 'job-id'])).toEqual({
showConnect: true,
resourceId: 'job-id',
command: [],
});
});
it('leaves --show-connect after the id in the passthrough command', () => {
expect(parseSshArgv(['job-id', '--show-connect'])).toEqual({
showConnect: false,
resourceId: 'job-id',
command: ['--show-connect'],
});
});
it('throws on an unknown flag before the id', () => {
expect(() => parseSshArgv(['--show-conect', 'job-id'])).toThrow('Unknown flag');
});
it('reports an undefined resource id when none is given', () => {
expect(parseSshArgv([])).toEqual({ showConnect: false, resourceId: undefined, command: [] });
expect(parseSshArgv(['--show-connect'])).toEqual({
showConnect: true,
resourceId: undefined,
command: [],
});
});
});
describe(resolveSshConnectStatus, () => {
it('is unknown when the resource did not resolve', () => {
expect(resolveSshConnectStatus(null)).toBe('unknown');
});
it('is not-enabled when the job did not request ssh', () => {
expect(
resolveSshConnectStatus({ sshRequested: false, jobCompleted: false, session: null })
).toBe('not-enabled');
});
it('is ended when ssh was requested, the job finished, and no session remains', () => {
expect(resolveSshConnectStatus({ sshRequested: true, jobCompleted: true, session: null })).toBe(
'ended'
);
});
it('is pending when ssh was requested and the job is still running without a session', () => {
expect(
resolveSshConnectStatus({ sshRequested: true, jobCompleted: false, session: null })
).toBe('pending');
});
it('is pending while the worker is reconnecting', () => {
expect(
resolveSshConnectStatus({
sshRequested: true,
jobCompleted: false,
session: {
connectionConfig: { host: 'uptermd.upterm.dev', secret: 'tok', reconnecting: true },
},
})
).toBe('pending');
});
it('is ready when the session has a connection config', () => {
expect(
resolveSshConnectStatus({
sshRequested: true,
jobCompleted: false,
session: {
connectionConfig: { host: 'uptermd.upterm.dev', secret: 'tok', reconnecting: false },
},
})
).toBe('ready');
});
});
describe(splitConnectionHost, () => {
it('returns the host with no port for a plain hostname', () => {
expect(splitConnectionHost('relay.expo.dev')).toEqual({ hostname: 'relay.expo.dev' });
});
it('splits a host:port into host and numeric port', () => {
expect(splitConnectionHost('relay.expo.dev:8022')).toEqual({
hostname: 'relay.expo.dev',
port: 8022,
});
});
});
describe(terminalSshStatusMessage, () => {
it('has no message while the session can still open', () => {
expect(terminalSshStatusMessage('pending', 'job-1')).toBeNull();
expect(terminalSshStatusMessage('ready', 'job-1')).toBeNull();
});
it('distinguishes not-enabled from ended', () => {
expect(terminalSshStatusMessage('not-enabled', 'job-1')).toMatch(/SSH was not enabled/);
expect(terminalSshStatusMessage('ended', 'job-1')).toMatch(/has ended/);
expect(terminalSshStatusMessage('unknown', 'job-1')).toMatch(/No workflow job found/);
});
});
describe(sshHostAliasForResource, () => {
it('suffixes the alias with the start of the resource id', () => {
expect(sshHostAliasForResource('job-1')).toBe('eas-workflow-ssh-job1');
});
it('drops characters that are not safe in an ssh config host alias', () => {
expect(sshHostAliasForResource('a b\nHost evil')).toBe('eas-workflow-ssh-abHostev');
});
it('falls back to the bare alias when nothing usable remains', () => {
expect(sshHostAliasForResource('---')).toBe('eas-workflow-ssh');
});
});
describe(WorkflowSsh, () => {
const graphqlClient = {} as never;
const mockConnectInfo = jest.mocked(WorkflowJobSshQuery.connectInfoForResourceIdAsync);
const mockSpawn = jest.mocked(spawnAsync);
const mockSleep = jest.mocked(sleepAsync);
const mockMkdtemp = jest.mocked(fs.promises.mkdtemp);
const mockWriteFile = jest.mocked(fs.promises.writeFile);
const mockRm = jest.mocked(fs.promises.rm);
let mockConfig: Config;
let previousExitCode: typeof process.exitCode;
const readyInfo = {
sshRequested: true,
jobCompleted: false,
session: {
id: 'ts-1',
connectionConfig: { host: 'relay.expo.dev', secret: 'TOKENx', reconnecting: false },
},
};
const pendingInfo = { sshRequested: true, jobCompleted: false, session: null };
const endedInfo = { sshRequested: true, jobCompleted: true, session: null };
const notEnabledInfo = { sshRequested: false, jobCompleted: false, session: null };
beforeAll(async () => {
mockConfig = new Config({ root: __dirname });
mockConfig.runHook = async () => ({ failures: [], successes: [] });
});
beforeEach(() => {
jest.clearAllMocks();
previousExitCode = process.exitCode;
process.exitCode = undefined;
mockMkdtemp.mockResolvedValue('/tmp/eas-ssh-1' as never);
mockWriteFile.mockResolvedValue(undefined as never);
mockRm.mockResolvedValue(undefined as never);
mockSpawn.mockResolvedValue({ stdout: '', stderr: '' } as never);
mockSleep.mockResolvedValue(undefined);
});
afterEach(() => {
process.exitCode = previousExitCode;
});
function createCommand(argv: string[]): WorkflowSsh {
const command = new WorkflowSsh(argv, mockConfig);
// @ts-expect-error getContextAsync/parse are protected
jest.spyOn(command, 'getContextAsync').mockResolvedValue({ loggedIn: { graphqlClient } });
// @ts-expect-error parse is protected on the oclif base
jest.spyOn(command, 'parse').mockResolvedValue({});
return command;
}
it('throws when no workflow job id is given', async () => {
await expect(createCommand([]).runAsync()).rejects.toThrow('Provide a workflow job id');
});
it('reports unknown and exits 1 when the id does not resolve', async () => {
mockConnectInfo.mockResolvedValue(null);
await createCommand(['job-1']).runAsync();
expect(Log.error).toHaveBeenCalledWith(expect.stringContaining('No workflow job found'));
expect(process.exitCode).toBe(1);
});
it('reports not-enabled and exits 1 when the job did not request ssh', async () => {
mockConnectInfo.mockResolvedValue(notEnabledInfo as never);
await createCommand(['job-1']).runAsync();
expect(Log.error).toHaveBeenCalledWith(expect.stringContaining('was not enabled'));
expect(process.exitCode).toBe(1);
});
it('reports ended and exits 1 when the job finished without a live session', async () => {
mockConnectInfo.mockResolvedValue(endedInfo as never);
await createCommand(['job-1']).runAsync();
expect(Log.error).toHaveBeenCalledWith(expect.stringContaining('has ended'));
expect(process.exitCode).toBe(1);
});
it('prints the connect command for --show-connect without spawning ssh', async () => {
mockConnectInfo.mockResolvedValue(readyInfo as never);
await createCommand(['--show-connect', 'job-1']).runAsync();
expect(Log.log).toHaveBeenCalledWith(
'ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null [email protected]'
);
expect(mockSpawn).not.toHaveBeenCalled();
});
it('prints --show-connect with -p and a hostname-only wss proxy for non-default SSH ports', async () => {
mockConnectInfo.mockResolvedValue({
...readyInfo,
session: {
id: 'ts-1',
connectionConfig: {
host: 'relay.expo.dev:8022',
secret: 'TOKENx',
reconnecting: false,
},
},
} as never);
await createCommand(['--show-connect', 'job-1']).runAsync();
expect(Log.log).toHaveBeenCalledWith(
'ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -p 8022 [email protected]'
);
expect(Log.log).toHaveBeenCalledWith(
' ssh -o ProxyCommand="upterm proxy wss://[email protected]" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -p 8022 [email protected]'
);
});
it('writes a 0600 config and opens ssh, then removes the config dir', async () => {
mockConnectInfo.mockResolvedValue(readyInfo as never);
await createCommand(['job-1', 'ls', '-la']).runAsync();
expect(mockWriteFile).toHaveBeenCalledWith(
'/tmp/eas-ssh-1/config',
expect.stringContaining('HostName relay.expo.dev'),
{ mode: 0o600 }
);
expect(mockWriteFile).toHaveBeenCalledWith(
'/tmp/eas-ssh-1/config',
expect.stringContaining('Host eas-workflow-ssh-job1'),
{ mode: 0o600 }
);
expect(mockSpawn).toHaveBeenCalledWith(
'ssh',
['-F', '/tmp/eas-ssh-1/config', 'eas-workflow-ssh-job1', 'ls', '-la'],
{ stdio: 'inherit' }
);
expect(mockRm).toHaveBeenCalledWith('/tmp/eas-ssh-1', { recursive: true, force: true });
});
it('waits for a pending session to open, then connects', async () => {
mockConnectInfo
.mockResolvedValueOnce(pendingInfo as never) // initial status check
.mockResolvedValueOnce(pendingInfo as never) // first wait poll
.mockResolvedValue(readyInfo as never);
await createCommand(['job-1']).runAsync();
expect(mockSleep).toHaveBeenCalled();
expect(mockSpawn).toHaveBeenCalled();
});
it('exits 1 when a pending session ends before it opens', async () => {
mockConnectInfo
.mockResolvedValueOnce(pendingInfo as never)
.mockResolvedValue(endedInfo as never);
await createCommand(['job-1']).runAsync();
expect(mockSpawn).not.toHaveBeenCalled();
expect(process.exitCode).toBe(1);
});
it('exits 1 when waiting for the session to open times out', async () => {
jest.useFakeTimers();
jest.setSystemTime(0);
try {
mockConnectInfo.mockResolvedValue(pendingInfo as never);
mockSleep.mockImplementation(async () => {
jest.setSystemTime(Date.now() + 6 * 60 * 1000);
});
await createCommand(['job-1']).runAsync();
expect(mockSpawn).not.toHaveBeenCalled();
expect(process.exitCode).toBe(1);
} finally {
jest.useRealTimers();
}
});
it('throws on an unexpected connection host', async () => {
mockConnectInfo.mockResolvedValue({
sshRequested: true,
jobCompleted: false,
session: {
connectionConfig: { host: 'bad host', secret: 'TOKENx', reconnecting: false },
},
} as never);
await expect(createCommand(['job-1']).runAsync()).rejects.toThrow('connection host');
});
it('propagates the ssh exit code from the catch handler', async () => {
const command = createCommand(['job-1']);
// @ts-expect-error isRunningSubprocess is private
command.isRunningSubprocess = true;
const err = Object.assign(new Error('ssh exited'), { status: 42 });
// @ts-expect-error catch is protected
await command.catch(err);
expect(process.exitCode).toBe(42);
});
it('reports a missing ssh client (ENOENT) from the catch handler', async () => {
const command = createCommand(['job-1']);
// @ts-expect-error isRunningSubprocess is private
command.isRunningSubprocess = true;
const err = Object.assign(new Error('spawn ssh ENOENT'), { code: 'ENOENT' });
// @ts-expect-error catch is protected
await command.catch(err);
expect(process.exitCode).toBe(1);
expect(Log.error).toHaveBeenCalledWith(expect.stringContaining('Install an OpenSSH client'));
});
it('delegates non-subprocess errors to the base handler', async () => {
const command = createCommand(['job-1']);
let threw = false;
try {
// @ts-expect-error catch is protected
await command.catch(new Error('boom'));
} catch {
threw = true;
}
expect(threw).toBe(true);
});
it('stops the spinner and rethrows when polling fails while waiting', async () => {
mockConnectInfo
.mockResolvedValueOnce(pendingInfo as never)
.mockRejectedValue(new Error('network blip'));
await expect(createCommand(['job-1']).runAsync()).rejects.toThrow('network blip');
});
});
@@ -78,6 +78,19 @@ import { uploadAccountScopedProjectSourceAsync } from '../../project/uploadAccou
import { enableJsonOutput, printJsonOnlyOutput } from '../../utils/json';
import { WorkflowFile } from '../../utils/workflowFile';
export function resolveWorkflowRunSshInput({
ssh,
idleTimeoutSeconds,
}: {
ssh: boolean;
idleTimeoutSeconds: number | undefined;
}): { idleTimeoutSeconds: number | undefined } | null {
if (idleTimeoutSeconds !== undefined && !ssh) {
throw new Error('--ssh-idle-timeout requires --ssh.');
}
return ssh ? { idleTimeoutSeconds } : null;
}
export default class WorkflowRun extends EasCommand {
static override description =
'run an EAS workflow. The entire local project directory will be packaged and uploaded to EAS servers for the workflow run, unless the --ref flag is used.';
@@ -110,6 +123,18 @@ export default class WorkflowRun extends EasCommand {
"The git reference must exist in the project's git repository, and the workflow file must exist at that reference. When this flag is used, the local project is not uploaded; instead, the workflow is run from the exact state of the project at the chosen reference.",
summary: 'Git reference to run the workflow on',
}),
ssh: Flags.boolean({
default: false,
description: 'Open an SSH session on each VM job for live debugging with `eas workflow:ssh`.',
summary: '[EXPERIMENTAL] Enable SSH on the run',
}),
'ssh-idle-timeout': Flags.integer({
description:
'Seconds to keep an SSH session open after its job finishes, while no client is connected. The session is always available for the whole job. Requires --ssh. Must be between 0 and 3600; defaults to 0 (close as soon as the job finishes and nobody is connected).',
summary: '[EXPERIMENTAL] SSH idle timeout in seconds',
min: 0,
max: 3600,
}),
...EasJsonOnlyFlag,
};
@@ -127,6 +152,11 @@ export default class WorkflowRun extends EasCommand {
enableJsonOutput();
}
const sshInput = resolveWorkflowRunSshInput({
ssh: flags.ssh,
idleTimeoutSeconds: flags['ssh-idle-timeout'],
});
const {
getDynamicPrivateProjectConfigAsync,
loggedIn: { graphqlClient },
@@ -272,6 +302,7 @@ export default class WorkflowRun extends EasCommand {
workflowRevisionId: workflowRevisionId ?? '',
gitRef,
inputs,
ssh: sshInput,
});
} catch (err) {
throw new Error(`Failed to create workflow run: ${err}`);
@@ -341,6 +372,7 @@ export default class WorkflowRun extends EasCommand {
packageJsonBucketKey,
projectRootDirectory,
},
ssh: sshInput,
},
}));
} catch (err) {
@@ -353,6 +385,10 @@ export default class WorkflowRun extends EasCommand {
Log.newLine();
Log.log(`See logs: ${link(getWorkflowRunUrl(account.name, projectName, workflowRunId))}`);
if (sshInput) {
Log.log('SSH enabled. Each VM job logs the `eas workflow:ssh` command to connect.');
}
if (!flags.wait) {
if (flags.json) {
printJsonOnlyOutput({
@@ -0,0 +1,252 @@
import spawnAsync from '@expo/spawn-async';
import { Flags } from '@oclif/core';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import EasCommand from '../../commandUtils/EasCommand';
import { ExpoGraphqlClient } from '../../commandUtils/context/contextUtils/createGraphqlClient';
import {
WorkflowJobSshConnectInfo,
WorkflowJobSshQuery,
WorkflowJobSshSession,
} from '../../graphql/queries/WorkflowJobSshQuery';
import Log from '../../log';
import { ora } from '../../ora';
import { sleepAsync } from '../../utils/promise';
export function splitConnectionHost(connectionHost: string): { hostname: string; port?: number } {
try {
const { hostname, port: stringPort } = new URL(`ssh://${connectionHost}`);
const port = stringPort === '' ? undefined : Number(stringPort);
return { hostname, port };
} catch (err) {
throw new Error(
'Unexpected connection host reported for this ssh session. Update eas-cli and try again, or contact support if it persists.',
{ cause: err }
);
}
}
const SSH_INSECURE_OPTS = '-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null';
const SESSION_OPEN_TIMEOUT_MS = 5 * 60 * 1000;
const SESSION_OPEN_POLL_INTERVAL_MS = 3000;
export type SshConnectStatus = 'unknown' | 'not-enabled' | 'ended' | 'pending' | 'ready';
export function resolveSshConnectStatus(
connectInfo:
| (Pick<WorkflowJobSshConnectInfo, 'sshRequested' | 'jobCompleted'> & {
session?: Pick<WorkflowJobSshSession, 'connectionConfig'> | null;
})
| null
): SshConnectStatus {
if (!connectInfo) {
return 'unknown';
}
const { sshRequested, jobCompleted, session } = connectInfo;
if (session?.connectionConfig && !session.connectionConfig.reconnecting) {
return 'ready';
}
if (session?.connectionConfig?.reconnecting) {
return 'pending';
}
if (!sshRequested) {
return 'not-enabled';
}
return jobCompleted ? 'ended' : 'pending';
}
export function terminalSshStatusMessage(
status: SshConnectStatus,
resourceId: string
): string | null {
switch (status) {
case 'unknown':
return `No workflow job found for "${resourceId}". Pass a workflow job id from a run started with \`eas workflow:run --ssh\`.`;
case 'not-enabled':
return `SSH was not enabled for "${resourceId}". Start the run with \`eas workflow:run --ssh\` to enable it.`;
case 'ended':
return 'This ssh session has ended.';
case 'pending':
case 'ready':
return null;
}
}
export function sshHostAliasForResource(resourceId: string): string {
const suffix = resourceId.replace(/[^A-Za-z0-9]/g, '').slice(0, 8);
return suffix ? `eas-workflow-ssh-${suffix}` : 'eas-workflow-ssh';
}
export function parseSshArgv(rawArgv: readonly string[]): {
showConnect: boolean;
resourceId: string | undefined;
command: string[];
} {
let showConnect = false;
let index = 0;
while (index < rawArgv.length && rawArgv[index].startsWith('-')) {
if (rawArgv[index] === '--show-connect' || rawArgv[index] === '--show-connect=true') {
showConnect = true;
} else {
throw new Error(
`Unknown flag "${rawArgv[index]}" before the id. The only supported flag is --show-connect; everything after the id is passed to the remote shell.`
);
}
index += 1;
}
const [resourceId, ...command] = rawArgv.slice(index);
return { showConnect, resourceId, command };
}
export default class WorkflowSsh extends EasCommand {
static override hidden = true;
static override description =
'[EXPERIMENTAL] open an ssh session on the worker running a workflow job';
static override strict = false;
static override flags = {
'show-connect': Flags.boolean({
description: 'Print the ssh connection command (host and token) instead of opening a session',
default: false,
}),
};
static override contextDefinition = {
...this.ContextOptions.LoggedIn,
};
private isRunningSubprocess = false;
async runAsync(): Promise<void> {
const rawArgv = [...this.argv];
await this.parse(WorkflowSsh, []);
const { showConnect, resourceId, command } = parseSshArgv(rawArgv);
if (typeof resourceId !== 'string' || resourceId.length === 0) {
throw new Error('Provide a workflow job id: eas workflow:ssh <workflow-job-id> [command...]');
}
const {
loggedIn: { graphqlClient },
} = await this.getContextAsync(WorkflowSsh, { nonInteractive: true });
const connectInfo = await WorkflowJobSshQuery.connectInfoForResourceIdAsync(
graphqlClient,
resourceId
);
const status = resolveSshConnectStatus(connectInfo);
const statusMessage = terminalSshStatusMessage(status, resourceId);
if (statusMessage) {
Log.error(statusMessage);
process.exitCode = 1;
return;
}
const connectionConfig =
(status === 'ready' ? connectInfo?.session?.connectionConfig : null) ??
(await waitForSessionToOpenAsync(graphqlClient, resourceId));
if (!connectionConfig) {
process.exitCode = 1;
return;
}
const { host: connectionHost, secret } = connectionConfig;
const { hostname, port } = splitConnectionHost(connectionHost);
const portOption = port !== undefined ? ` -p ${port}` : '';
if (showConnect) {
Log.log(`ssh ${SSH_INSECURE_OPTS}${portOption} ${secret}@${hostname}`);
Log.newLine();
Log.log(
'If your network blocks the direct SSH connection, reach the session through the WebSocket relay with upterm (https://upterm.dev):'
);
// WSS terminates on the relay hostname (default 443). Do not paste an SSH
// :port into the wss:// URL — that port is only for the ssh destination.
Log.log(
` ssh -o ProxyCommand="upterm proxy wss://${secret}@${hostname}" ${SSH_INSECURE_OPTS}${portOption} ${secret}@${hostname}`
);
return;
}
const configDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'eas-workflow-ssh-'));
try {
const configPath = path.join(configDir, 'config');
const hostAlias = sshHostAliasForResource(resourceId);
await fs.promises.writeFile(
configPath,
[
`Host ${hostAlias}`,
` HostName ${hostname}`,
...(port !== undefined ? [` Port ${port}`] : []),
` User ${secret}`,
' StrictHostKeyChecking no',
' UserKnownHostsFile /dev/null',
'',
].join('\n'),
{ mode: 0o600 }
);
this.isRunningSubprocess = true;
await spawnAsync('ssh', ['-F', configPath, hostAlias, ...command], {
stdio: 'inherit',
});
} finally {
await fs.promises.rm(configDir, { recursive: true, force: true });
}
}
protected override catch(err: Error): Promise<void> {
if (this.isRunningSubprocess) {
if ((err as Error & { code?: string }).code === 'ENOENT') {
Log.error(
'Could not run `ssh`. Install an OpenSSH client and make sure `ssh` is on your PATH, then try again.'
);
process.exitCode = 1;
return Promise.resolve();
}
const status = (err as Error & { status?: number | null }).status;
process.exitCode = process.exitCode ?? status ?? 1;
return Promise.resolve();
}
return super.catch(err);
}
}
async function waitForSessionToOpenAsync(
graphqlClient: ExpoGraphqlClient,
workflowJobId: string
): Promise<NonNullable<WorkflowJobSshSession['connectionConfig']> | null> {
const spinner = ora('Waiting for the worker to open the ssh session').start();
const deadline = Date.now() + SESSION_OPEN_TIMEOUT_MS;
try {
while (Date.now() < deadline) {
const connectInfo = await WorkflowJobSshQuery.connectInfoForResourceIdAsync(
graphqlClient,
workflowJobId
);
const status = resolveSshConnectStatus(connectInfo);
const statusMessage = terminalSshStatusMessage(status, workflowJobId);
if (statusMessage) {
spinner.fail(statusMessage);
return null;
}
if (status === 'ready') {
spinner.succeed('The ssh session is ready.');
return connectInfo?.session?.connectionConfig ?? null;
}
await sleepAsync(SESSION_OPEN_POLL_INTERVAL_MS);
}
spinner.fail(
'Timed out waiting for the ssh session to open. The worker may still be starting up; try again in a moment.'
);
return null;
} catch (err) {
spinner.stop();
throw err;
}
}
+191 -5
View File
@@ -83,6 +83,8 @@ export type Account = {
accountFeatureGates: Scalars['JSONObject']['output'];
/** Coalesced project activity for all apps belonging to this account. */
activityTimelineProjectActivities: Array<ActivityTimelineProjectActivity>;
/** AI provider connections available to agent sandbox sessions. */
agentProviderConnections: Array<AgentProviderConnection>;
aiChatEnabled: Scalars['Boolean']['output'];
appCount: Scalars['Int']['output'];
/** @deprecated Use appStoreConnectApiKeysPaginated */
@@ -1071,6 +1073,8 @@ export type AgentDeviceRunSessionRemoteConfig = {
__typename?: 'AgentDeviceRunSessionRemoteConfig';
agentDeviceRemoteSessionToken: Scalars['String']['output'];
agentDeviceRemoteSessionUrl: Scalars['String']['output'];
/** Session token gating the web preview. Null when the preview runs ungated. */
webPreviewToken?: Maybe<Scalars['String']['output']>;
/**
* URL of the web preview surface for the session. Null when web previews are
* not available for the platform (e.g. Android).
@@ -1078,6 +1082,80 @@ export type AgentDeviceRunSessionRemoteConfig = {
webPreviewUrl?: Maybe<Scalars['String']['output']>;
};
export enum AgentProvider {
Anthropic = 'ANTHROPIC',
Openai = 'OPENAI'
}
export type AgentProviderConnection = {
__typename?: 'AgentProviderConnection';
account: Account;
createdAt: Scalars['DateTime']['output'];
createdByActor?: Maybe<Actor>;
expiresAt?: Maybe<Scalars['DateTime']['output']>;
id: Scalars['ID']['output'];
name?: Maybe<Scalars['String']['output']>;
provider: AgentProvider;
providerAccountEmail?: Maybe<Scalars['String']['output']>;
providerAccountId?: Maybe<Scalars['String']['output']>;
providerPlan?: Maybe<Scalars['String']['output']>;
status: AgentProviderConnectionStatus;
updatedAt: Scalars['DateTime']['output'];
};
export enum AgentProviderConnectionAuthorizationStatus {
Expired = 'EXPIRED',
Pending = 'PENDING',
Succeeded = 'SUCCEEDED'
}
export type AgentProviderConnectionMutation = {
__typename?: 'AgentProviderConnectionMutation';
completeClaudeConnection: AgentProviderConnection;
deleteAgentProviderConnection: AgentProviderConnection;
pollCodexConnection: CodexConnectionPollResult;
startClaudeConnection: ClaudeAuthorization;
startCodexConnection: CodexDeviceAuthorization;
updateAgentProviderConnection: AgentProviderConnection;
};
export type AgentProviderConnectionMutationCompleteClaudeConnectionArgs = {
input: CompleteClaudeConnectionInput;
};
export type AgentProviderConnectionMutationDeleteAgentProviderConnectionArgs = {
id: Scalars['ID']['input'];
};
export type AgentProviderConnectionMutationPollCodexConnectionArgs = {
requestId: Scalars['ID']['input'];
};
export type AgentProviderConnectionMutationStartClaudeConnectionArgs = {
input: StartClaudeConnectionInput;
};
export type AgentProviderConnectionMutationStartCodexConnectionArgs = {
accountId: Scalars['ID']['input'];
};
export type AgentProviderConnectionMutationUpdateAgentProviderConnectionArgs = {
id: Scalars['ID']['input'];
input: UpdateAgentProviderConnectionInput;
};
export enum AgentProviderConnectionStatus {
Invalid = 'INVALID',
UnableToVerify = 'UNABLE_TO_VERIFY',
Valid = 'VALID'
}
export type AndroidAppBuildCredentials = {
__typename?: 'AndroidAppBuildCredentials';
androidKeystore?: Maybe<AndroidKeystore>;
@@ -1444,6 +1522,8 @@ export type App = Project & {
/** @deprecated Classic updates have been deprecated. */
description: Scalars['String']['output'];
devDomainName?: Maybe<AppDevDomainName>;
/** Every tag used by this app's device run sessions, sorted alphabetically. */
deviceRunSessionTags: Array<Scalars['String']['output']>;
deviceRunSessionsPaginated: AppDeviceRunSessionsConnection;
embeddedUpdatesPaginated: AppEmbeddedUpdatesConnection;
/** Environment secrets for an app */
@@ -1472,6 +1552,8 @@ export type App = Project & {
internalDistributionBuildPrivacy: AppInternalDistributionBuildPrivacy;
/** iOS app credentials for the project */
iosAppCredentials: Array<IosAppCredentials>;
/** Whether this app qualifies for the observe-notification-lapsed-paid experiment */
isEligibleForObserveLapsedPaidNotification: Scalars['Boolean']['output'];
/**
* Tells you if the project can show the Observe promotional notification. This field is part of
* the observe-notification experiment.
@@ -3730,6 +3812,8 @@ export type AppiumRunSessionRemoteConfig = {
appiumUrl: Scalars['String']['output'];
/** W3C capabilities for the device that backs this session. */
capabilities: Scalars['JSONObject']['output'];
/** Session token gating the web preview. Null when the preview runs ungated. */
webPreviewToken?: Maybe<Scalars['String']['output']>;
/**
* URL of the web preview surface for the session. Null when web previews are
* not available for the platform (e.g. Android).
@@ -4138,6 +4222,8 @@ export type ArgentRunSessionRemoteConfig = {
__typename?: 'ArgentRunSessionRemoteConfig';
toolsAuthToken?: Maybe<Scalars['String']['output']>;
toolsUrl: Scalars['String']['output'];
/** Session token gating the web preview. Null when the preview runs ungated. */
webPreviewToken?: Maybe<Scalars['String']['output']>;
/**
* URL of the web preview surface for the session. Null when web previews are
* not available for the platform (e.g. Android).
@@ -5087,6 +5173,13 @@ export type ChatTokenUsage = {
usedTokens: Scalars['Int']['output'];
};
export type ClaudeAuthorization = {
__typename?: 'ClaudeAuthorization';
authorizationUrl: Scalars['String']['output'];
expiresAt: Scalars['DateTime']['output'];
requestId: Scalars['ID']['output'];
};
export type CodeSigningInfo = {
__typename?: 'CodeSigningInfo';
alg: Scalars['String']['output'];
@@ -5100,6 +5193,26 @@ export type CodeSigningInfoInput = {
sig: Scalars['String']['input'];
};
export type CodexConnectionPollResult = {
__typename?: 'CodexConnectionPollResult';
connection?: Maybe<AgentProviderConnection>;
status: AgentProviderConnectionAuthorizationStatus;
};
export type CodexDeviceAuthorization = {
__typename?: 'CodexDeviceAuthorization';
expiresAt: Scalars['DateTime']['output'];
pollingIntervalSeconds: Scalars['Int']['output'];
requestId: Scalars['ID']['output'];
userCode: Scalars['String']['output'];
verificationUrl: Scalars['String']['output'];
};
export type CompleteClaudeConnectionInput = {
authorizationCode: Scalars['String']['input'];
requestId: Scalars['ID']['input'];
};
export type CompletePostHogConnectionInput = {
code: Scalars['String']['input'];
state: Scalars['ID']['input'];
@@ -5321,7 +5434,7 @@ export type CreateDeviceRunSessionInput = {
/**
* Stop the session automatically after this many minutes without observed
* session activity. Must be positive and smaller than the session's maximum
* duration (2 hours, or maxRunTimeMinutes when set). Only supported for
* duration (maxRunTimeMinutes when set). Only supported for
* agent-device, argent, and Appium sessions. If omitted, the session has no idle
* timeout.
*/
@@ -5355,6 +5468,8 @@ export type CreateDeviceRunSessionInput = {
* true. If omitted, the current Expo Go archive for the platform is used.
*/
sdkVersion?: InputMaybe<Scalars['String']['input']>;
/** Free-form labels for grouping sessions, for example one per app variant. */
tags?: InputMaybe<Array<Scalars['String']['input']>>;
type: DeviceRunSessionType;
};
@@ -6134,6 +6249,8 @@ export type DeviceRunSession = {
remoteConfig?: Maybe<DeviceRunSessionRemoteConfig>;
startedAt?: Maybe<Scalars['DateTime']['output']>;
status: DeviceRunSessionStatus;
/** Free-form labels for grouping sessions, for example one per app variant. */
tags: Array<Scalars['String']['output']>;
turtleJobRun?: Maybe<JobRun>;
type: DeviceRunSessionType;
updatedAt: Scalars['DateTime']['output'];
@@ -6180,6 +6297,8 @@ export type DeviceRunSessionFilterInput = {
name?: InputMaybe<Scalars['String']['input']>;
platforms?: InputMaybe<Array<AppPlatform>>;
statuses?: InputMaybe<Array<DeviceRunSessionStatus>>;
/** Matches sessions carrying every listed tag. Case-insensitive. */
tags?: InputMaybe<Array<Scalars['String']['input']>>;
types?: InputMaybe<Array<DeviceRunSessionType>>;
};
@@ -6241,7 +6360,7 @@ export type DeviceRunSessionQueryByIdArgs = {
deviceRunSessionId: Scalars['ID']['input'];
};
export type DeviceRunSessionRemoteConfig = AgentDeviceRunSessionRemoteConfig | AppiumRunSessionRemoteConfig | ArgentRunSessionRemoteConfig | ServeSimRunSessionRemoteConfig;
export type DeviceRunSessionRemoteConfig = AgentDeviceRunSessionRemoteConfig | AppiumRunSessionRemoteConfig | ArgentRunSessionRemoteConfig | ServeSimRunSessionRemoteConfig | WebPreviewOnlyRunSessionRemoteConfig;
export enum DeviceRunSessionResourceClass {
Large = 'LARGE',
@@ -6259,7 +6378,10 @@ export enum DeviceRunSessionType {
AgentDevice = 'AGENT_DEVICE',
Appium = 'APPIUM',
Argent = 'ARGENT',
ServeSim = 'SERVE_SIM'
/** @deprecated Use WEB_PREVIEW_ONLY instead. */
ServeSim = 'SERVE_SIM',
/** A session accessed only through its web preview. Currently supported on iOS. */
WebPreviewOnly = 'WEB_PREVIEW_ONLY'
}
export type DiscordUser = {
@@ -7126,6 +7248,7 @@ export type EmbeddedUpdateQueryByIdArgs = {
export enum EntityTypeName {
AccountEntity = 'AccountEntity',
AccountSsoConfigurationEntity = 'AccountSSOConfigurationEntity',
AgentProviderConnectionEntity = 'AgentProviderConnectionEntity',
AndroidAppCredentialsEntity = 'AndroidAppCredentialsEntity',
AndroidKeystoreEntity = 'AndroidKeystoreEntity',
AppEntity = 'AppEntity',
@@ -9509,6 +9632,7 @@ export type RootMutation = {
accountSSOConfiguration: AccountSsoConfigurationMutation;
/** Mutations for Actor experiments */
actorExperiment: ActorExperimentMutation;
agentProviderConnection: AgentProviderConnectionMutation;
/** Mutations that modify the build credentials for an Android app */
androidAppBuildCredentials: AndroidAppBuildCredentialsMutation;
/** Mutations that modify the credentials for an Android app */
@@ -10111,8 +10235,14 @@ export type SentryProjectMutationDeleteSentryProjectArgs = {
sentryProjectId: Scalars['ID']['input'];
};
/**
* Legacy remote config for SERVE_SIM sessions. Use WEB_PREVIEW_ONLY sessions and
* WebPreviewOnlyRunSessionRemoteConfig instead.
*/
export type ServeSimRunSessionRemoteConfig = {
__typename?: 'ServeSimRunSessionRemoteConfig';
/** Session token gating the preview. Null when the preview runs ungated. */
previewToken?: Maybe<Scalars['String']['output']>;
previewUrl: Scalars['String']['output'];
/** @deprecated Use previewUrl instead. */
streamUrl?: Maybe<Scalars['String']['output']>;
@@ -10201,6 +10331,10 @@ export enum StandardOffer {
YearlySub = 'YEARLY_SUB'
}
export type StartClaudeConnectionInput = {
accountId: Scalars['ID']['input'];
};
export type StartPostHogConnectionResult = PostHogOrganizationConnection | PostHogPendingConnection;
/** Incident for a given component from Expo status page API. */
@@ -10809,6 +10943,10 @@ export type UpdateDiffReceiptsArgs = {
orderBy?: InputMaybe<Array<UpdateDiffReceiptOrderByClause>>;
};
export type UpdateAgentProviderConnectionInput = {
name?: InputMaybe<Scalars['String']['input']>;
};
export type UpdateBranch = {
__typename?: 'UpdateBranch';
app: App;
@@ -12134,6 +12272,11 @@ export type WebNotificationUpdateReadStateInput = {
isRead: Scalars['Boolean']['input'];
};
export type WebPreviewOnlyRunSessionRemoteConfig = {
__typename?: 'WebPreviewOnlyRunSessionRemoteConfig';
webPreviewUrl: Scalars['String']['output'];
};
export type Webhook = {
__typename?: 'Webhook';
appId: Scalars['ID']['output'];
@@ -13425,6 +13568,7 @@ export type WorkflowRunGitBranchFilterInput = {
export type WorkflowRunInput = {
inputs?: InputMaybe<Scalars['JSONObject']['input']>;
projectSource: WorkflowProjectSourceInput;
ssh?: InputMaybe<WorkflowRunSshInput>;
};
export type WorkflowRunMutation = {
@@ -13459,12 +13603,14 @@ export type WorkflowRunMutationCreateWorkflowRunArgs = {
export type WorkflowRunMutationCreateWorkflowRunFromGitRefArgs = {
gitRef: Scalars['String']['input'];
inputs?: InputMaybe<Scalars['JSONObject']['input']>;
ssh?: InputMaybe<WorkflowRunSshInput>;
workflowRevisionId: Scalars['ID']['input'];
};
export type WorkflowRunMutationRetryWorkflowRunArgs = {
fromFailedJobs?: InputMaybe<Scalars['Boolean']['input']>;
ssh?: InputMaybe<WorkflowRunSshInput>;
workflowRunId: Scalars['ID']['input'];
};
@@ -13478,6 +13624,14 @@ export type WorkflowRunQueryByIdArgs = {
workflowRunId: Scalars['ID']['input'];
};
/**
* Enables ssh on the run's VM jobs. Presence turns ssh on; idleTimeoutSeconds is optional,
* defaults server-side, and is validated against a supported range.
*/
export type WorkflowRunSshInput = {
idleTimeoutSeconds?: InputMaybe<Scalars['Int']['input']>;
};
export type WorkflowRunSshSettings = {
__typename?: 'WorkflowRunSshSettings';
idleTimeoutSeconds: Scalars['Int']['output'];
@@ -13506,6 +13660,9 @@ export enum WorkflowRunTriggerEventType {
EasSubmit = 'EAS_SUBMIT',
ExpoLaunch = 'EXPO_LAUNCH',
GithubPullRequestBaseRefChanged = 'GITHUB_PULL_REQUEST_BASE_REF_CHANGED',
GithubPullRequestCommentCreated = 'GITHUB_PULL_REQUEST_COMMENT_CREATED',
GithubPullRequestCommentDeleted = 'GITHUB_PULL_REQUEST_COMMENT_DELETED',
GithubPullRequestCommentEdited = 'GITHUB_PULL_REQUEST_COMMENT_EDITED',
GithubPullRequestEdited = 'GITHUB_PULL_REQUEST_EDITED',
GithubPullRequestLabeled = 'GITHUB_PULL_REQUEST_LABELED',
GithubPullRequestOpened = 'GITHUB_PULL_REQUEST_OPENED',
@@ -14576,6 +14733,19 @@ export type SetRolloutPercentageMutation = { __typename?: 'RootMutation', update
| { __typename: 'User', username: string, id: string }
| null, branch: { __typename?: 'UpdateBranch', id: string, name: string }, codeSigningInfo?: { __typename?: 'CodeSigningInfo', keyid: string, sig: string, alg: string } | null, rolloutControlUpdate?: { __typename?: 'Update', id: string, group: string } | null, fingerprint?: { __typename?: 'Fingerprint', id: string, hash: string, debugInfoUrl?: string | null, source?: { __typename?: 'FingerprintSource', type: FingerprintSourceType, bucketKey: string, isDebugFingerprint?: boolean | null } | null } | null } } };
export type GenerateRealtimeLogsCentrifugoConnectionTokenMutationVariables = Exact<{ [key: string]: never; }>;
export type GenerateRealtimeLogsCentrifugoConnectionTokenMutation = { __typename?: 'RootMutation', realtimeLogs: { __typename?: 'RealtimeLogsMutation', generateCentrifugoConnectionToken: { __typename?: 'RealtimeLogsCentrifugoConnectionToken', token: string } } };
export type GenerateRealtimeLogsCentrifugoSubscriptionTokenMutationVariables = Exact<{
target: RealtimeLogsTargetInput;
thread?: InputMaybe<Scalars['String']['input']>;
}>;
export type GenerateRealtimeLogsCentrifugoSubscriptionTokenMutation = { __typename?: 'RootMutation', realtimeLogs: { __typename?: 'RealtimeLogsMutation', generateCentrifugoSubscriptionToken: { __typename?: 'RealtimeLogsCentrifugoSubscriptionToken', channel: string, token: string } } };
export type CreateAndroidSubmissionMutationVariables = Exact<{
appId: Scalars['ID']['input'];
config: AndroidSubmissionConfigInput;
@@ -14750,6 +14920,7 @@ export type CreateWorkflowRunFromGitRefMutationVariables = Exact<{
workflowRevisionId: Scalars['ID']['input'];
gitRef: Scalars['String']['input'];
inputs?: InputMaybe<Scalars['JSONObject']['input']>;
ssh?: InputMaybe<WorkflowRunSshInput>;
}>;
@@ -15126,11 +15297,12 @@ export type DeviceRunSessionByIdQueryVariables = Exact<{
}>;
export type DeviceRunSessionByIdQuery = { __typename?: 'RootQuery', deviceRunSessions: { __typename?: 'DeviceRunSessionQuery', byId: { __typename?: 'DeviceRunSession', id: string, name?: string | null, status: DeviceRunSessionStatus, type: DeviceRunSessionType, platform: AppPlatform, createdAt: any, startedAt?: any | null, finishedAt?: any | null, updatedAt: any, app: { __typename?: 'App', id: string, slug: string, ownerAccount: { __typename?: 'Account', id: string, name: string } }, artifacts: Array<{ __typename?: 'DeviceRunSessionArtifact', id: string, name: string, filename: string, downloadUrl: string, fileSizeBytes?: number | null, metadata?: any | null, createdAt: any, updatedAt: any }>, remoteConfig?:
export type DeviceRunSessionByIdQuery = { __typename?: 'RootQuery', deviceRunSessions: { __typename?: 'DeviceRunSessionQuery', byId: { __typename?: 'DeviceRunSession', id: string, name?: string | null, tags: Array<string>, status: DeviceRunSessionStatus, type: DeviceRunSessionType, platform: AppPlatform, createdAt: any, startedAt?: any | null, finishedAt?: any | null, updatedAt: any, app: { __typename?: 'App', id: string, slug: string, ownerAccount: { __typename?: 'Account', id: string, name: string } }, artifacts: Array<{ __typename?: 'DeviceRunSessionArtifact', id: string, name: string, filename: string, downloadUrl: string, fileSizeBytes?: number | null, metadata?: any | null, createdAt: any, updatedAt: any }>, remoteConfig?:
| { __typename: 'AgentDeviceRunSessionRemoteConfig', agentDeviceRemoteSessionUrl: string, agentDeviceRemoteSessionToken: string, webPreviewUrl?: string | null }
| { __typename: 'AppiumRunSessionRemoteConfig', appiumUrl: string, capabilities: any, webPreviewUrl?: string | null }
| { __typename: 'ArgentRunSessionRemoteConfig', toolsUrl: string, toolsAuthToken?: string | null, webPreviewUrl?: string | null }
| { __typename: 'ServeSimRunSessionRemoteConfig', previewUrl: string }
| { __typename: 'WebPreviewOnlyRunSessionRemoteConfig', previewUrl: string }
| null, turtleJobRun?: { __typename?: 'JobRun', id: string, status: JobRunStatus } | null } } };
export type DeviceRunSessionsByAppIdQueryVariables = Exact<{
@@ -15141,7 +15313,7 @@ export type DeviceRunSessionsByAppIdQueryVariables = Exact<{
}>;
export type DeviceRunSessionsByAppIdQuery = { __typename?: 'RootQuery', app: { __typename?: 'AppQuery', byId: { __typename?: 'App', id: string, deviceRunSessionsPaginated: { __typename?: 'AppDeviceRunSessionsConnection', edges: Array<{ __typename?: 'AppDeviceRunSessionEdge', cursor: string, node: { __typename?: 'DeviceRunSession', id: string, name?: string | null, status: DeviceRunSessionStatus, type: DeviceRunSessionType, platform: AppPlatform, createdAt: any, startedAt?: any | null, finishedAt?: any | null, app: { __typename?: 'App', id: string, slug: string, ownerAccount: { __typename?: 'Account', id: string, name: string } }, turtleJobRun?: { __typename?: 'JobRun', id: string, status: JobRunStatus } | null } }>, pageInfo: { __typename?: 'PageInfo', hasNextPage: boolean, hasPreviousPage: boolean, startCursor?: string | null, endCursor?: string | null } } } } };
export type DeviceRunSessionsByAppIdQuery = { __typename?: 'RootQuery', app: { __typename?: 'AppQuery', byId: { __typename?: 'App', id: string, deviceRunSessionsPaginated: { __typename?: 'AppDeviceRunSessionsConnection', edges: Array<{ __typename?: 'AppDeviceRunSessionEdge', cursor: string, node: { __typename?: 'DeviceRunSession', id: string, name?: string | null, tags: Array<string>, status: DeviceRunSessionStatus, type: DeviceRunSessionType, platform: AppPlatform, createdAt: any, startedAt?: any | null, finishedAt?: any | null, app: { __typename?: 'App', id: string, slug: string, ownerAccount: { __typename?: 'Account', id: string, name: string } }, turtleJobRun?: { __typename?: 'JobRun', id: string, status: JobRunStatus } | null } }>, pageInfo: { __typename?: 'PageInfo', hasNextPage: boolean, hasPreviousPage: boolean, startCursor?: string | null, endCursor?: string | null } } } } };
export type ViewEmbeddedUpdateByIdQueryVariables = Exact<{
embeddedUpdateId: Scalars['ID']['input'];
@@ -15527,6 +15699,20 @@ export type WorkflowJobByIdQuery = { __typename?: 'RootQuery', workflowJobs: { _
| { __typename: 'User', id: string, displayName: string }
| null, app: { __typename: 'App', id: string, name: string, slug: string, ownerAccount: { __typename?: 'Account', id: string, name: string } }, updateChannel?: { __typename?: 'UpdateChannel', id: string, name: string } | null, runtime?: { __typename?: 'Runtime', id: string, version: string } | null, metrics?: { __typename?: 'BuildMetrics', buildWaitTime?: number | null, buildQueueTime?: number | null, buildDuration?: number | null } | null } | null, errors: Array<{ __typename?: 'WorkflowJobError', title: string, message: string }> } } };
export type WorkflowJobSshPollQueryVariables = Exact<{
workflowJobId: Scalars['ID']['input'];
}>;
export type WorkflowJobSshPollQuery = { __typename?: 'RootQuery', workflowJobs: { __typename?: 'WorkflowJobQuery', byId: { __typename?: 'WorkflowJob', id: string, status: WorkflowJobStatus, workflowRun: { __typename?: 'WorkflowRun', id: string, sshSettings?: { __typename?: 'WorkflowRunSshSettings', idleTimeoutSeconds: number } | null }, turtleJobRun?: { __typename?: 'JobRun', id: string, sshSession?: { __typename?: 'TurtleSshSession', id: string, connectionConfig: { __typename?: 'TurtleSshConnectionConfig', host: string, secret: string, reconnecting: boolean } } | null } | null, turtleBuild?: { __typename?: 'Build', id: string, sshSession?: { __typename?: 'TurtleSshSession', id: string, connectionConfig: { __typename?: 'TurtleSshConnectionConfig', host: string, secret: string, reconnecting: boolean } } | null } | null } } };
export type JobRunSshPollQueryVariables = Exact<{
jobRunId: Scalars['ID']['input'];
}>;
export type JobRunSshPollQuery = { __typename?: 'RootQuery', jobRun: { __typename?: 'JobRunQuery', byId: { __typename?: 'JobRun', id: string, status: JobRunStatus, sshSession?: { __typename?: 'TurtleSshSession', id: string, connectionConfig: { __typename?: 'TurtleSshConnectionConfig', host: string, secret: string, reconnecting: boolean } } | null } } };
export type ExpoGoSupportedSdkVersionsQueryVariables = Exact<{ [key: string]: never; }>;
@@ -0,0 +1,69 @@
import gql from 'graphql-tag';
import { ExpoGraphqlClient } from '../../commandUtils/context/contextUtils/createGraphqlClient';
import { withErrorHandlingAsync } from '../client';
import {
GenerateRealtimeLogsCentrifugoConnectionTokenMutation,
GenerateRealtimeLogsCentrifugoConnectionTokenMutationVariables,
GenerateRealtimeLogsCentrifugoSubscriptionTokenMutation,
GenerateRealtimeLogsCentrifugoSubscriptionTokenMutationVariables,
RealtimeLogsCentrifugoConnectionToken,
RealtimeLogsCentrifugoSubscriptionToken,
RealtimeLogsMutationGenerateCentrifugoSubscriptionTokenArgs,
} from '../generated';
export const RealtimeLogsMutation = {
async generateCentrifugoConnectionTokenAsync(
graphqlClient: ExpoGraphqlClient
): Promise<RealtimeLogsCentrifugoConnectionToken> {
const data = await withErrorHandlingAsync(
graphqlClient
.mutation<
GenerateRealtimeLogsCentrifugoConnectionTokenMutation,
GenerateRealtimeLogsCentrifugoConnectionTokenMutationVariables
>(
gql`
mutation GenerateRealtimeLogsCentrifugoConnectionToken {
realtimeLogs {
generateCentrifugoConnectionToken {
token
}
}
}
`,
{}
)
.toPromise()
);
return data.realtimeLogs.generateCentrifugoConnectionToken;
},
async generateCentrifugoSubscriptionTokenAsync(
graphqlClient: ExpoGraphqlClient,
{ target, thread }: RealtimeLogsMutationGenerateCentrifugoSubscriptionTokenArgs
): Promise<RealtimeLogsCentrifugoSubscriptionToken> {
const data = await withErrorHandlingAsync(
graphqlClient
.mutation<
GenerateRealtimeLogsCentrifugoSubscriptionTokenMutation,
GenerateRealtimeLogsCentrifugoSubscriptionTokenMutationVariables
>(
gql`
mutation GenerateRealtimeLogsCentrifugoSubscriptionToken(
$target: RealtimeLogsTargetInput!
$thread: String
) {
realtimeLogs {
generateCentrifugoSubscriptionToken(target: $target, thread: $thread) {
channel
token
}
}
}
`,
{ target, thread }
)
.toPromise()
);
return data.realtimeLogs.generateCentrifugoSubscriptionToken;
},
};
@@ -12,6 +12,7 @@ import {
WorkflowProjectSourceInput,
WorkflowRevisionInput,
WorkflowRunInput,
WorkflowRunSshInput,
} from '../generated';
export namespace WorkflowRunMutation {
@@ -64,10 +65,12 @@ export namespace WorkflowRunMutation {
workflowRevisionId,
gitRef,
inputs,
ssh,
}: {
workflowRevisionId: string;
gitRef: string;
inputs?: Record<string, any>;
ssh?: WorkflowRunSshInput | null;
}
): Promise<{ id: string }> {
const data = await withErrorHandlingAsync(
@@ -81,12 +84,14 @@ export namespace WorkflowRunMutation {
$workflowRevisionId: ID!
$gitRef: String!
$inputs: JSONObject
$ssh: WorkflowRunSshInput
) {
workflowRun {
createWorkflowRunFromGitRef(
workflowRevisionId: $workflowRevisionId
gitRef: $gitRef
inputs: $inputs
ssh: $ssh
) {
id
}
@@ -97,6 +102,7 @@ export namespace WorkflowRunMutation {
workflowRevisionId,
gitRef,
inputs,
ssh,
}
)
.toPromise()
@@ -55,6 +55,7 @@ export const DeviceRunSessionQuery = {
byId(deviceRunSessionId: $deviceRunSessionId) {
id
name
tags
status
type
platform
@@ -100,6 +101,9 @@ export const DeviceRunSessionQuery = {
... on ServeSimRunSessionRemoteConfig {
previewUrl
}
... on WebPreviewOnlyRunSessionRemoteConfig {
previewUrl: webPreviewUrl
}
}
turtleJobRun {
id
@@ -149,6 +153,7 @@ export const DeviceRunSessionQuery = {
node {
id
name
tags
status
type
platform
@@ -0,0 +1,213 @@
import gql from 'graphql-tag';
import { ExpoGraphqlClient } from '../../commandUtils/context/contextUtils/createGraphqlClient';
import { GraphqlError, withErrorHandlingAsync } from '../client';
import { JobRunStatus, WorkflowJobStatus } from '../generated';
const FINAL_WORKFLOW_JOB_STATUSES = new Set<WorkflowJobStatus>([
WorkflowJobStatus.Success,
WorkflowJobStatus.Failure,
WorkflowJobStatus.Canceled,
WorkflowJobStatus.Skipped,
]);
const FINAL_JOB_RUN_STATUSES = new Set<JobRunStatus>([
JobRunStatus.Errored,
JobRunStatus.Finished,
JobRunStatus.Canceled,
]);
export type WorkflowJobSshConnectionConfig = {
host: string;
secret: string;
reconnecting: boolean;
};
export type WorkflowJobSshSession = {
id: string;
connectionConfig: WorkflowJobSshConnectionConfig;
};
export type WorkflowJobSshConnectInfo = {
sshRequested: boolean;
jobCompleted: boolean;
session: WorkflowJobSshSession | null;
};
type WorkflowJobSshPollQuery = {
workflowJobs: {
byId: {
id: string;
status: WorkflowJobStatus;
workflowRun: {
id: string;
sshSettings: { idleTimeoutSeconds: number } | null;
};
turtleJobRun: { id: string; sshSession: WorkflowJobSshSession | null } | null;
turtleBuild: { id: string; sshSession: WorkflowJobSshSession | null } | null;
};
};
};
type WorkflowJobSshPollQueryVariables = {
workflowJobId: string;
};
type JobRunSshPollQuery = {
jobRun: {
byId: {
id: string;
status: JobRunStatus;
sshSession: WorkflowJobSshSession | null;
};
};
};
type JobRunSshPollQueryVariables = {
jobRunId: string;
};
function toConnectInfo(
job: WorkflowJobSshPollQuery['workflowJobs']['byId']
): WorkflowJobSshConnectInfo {
const hasTurtleTarget = job.turtleJobRun != null || job.turtleBuild != null;
return {
sshRequested: hasTurtleTarget && job.workflowRun.sshSettings != null,
jobCompleted: FINAL_WORKFLOW_JOB_STATUSES.has(job.status),
session: job.turtleJobRun?.sshSession ?? job.turtleBuild?.sshSession ?? null,
};
}
function isNotFoundError(error: unknown): boolean {
if (!(error instanceof GraphqlError)) {
return false;
}
return error.graphQLErrors.some(e => {
const code = e?.extensions?.errorCode ?? e?.extensions?.code;
return (
code === 'ENTITY_NOT_FOUND' ||
/Entity not found/i.test(e?.message ?? '') ||
/not found/i.test(e?.message ?? '')
);
});
}
export const WorkflowJobSshQuery = {
async connectInfoForWorkflowJobAsync(
graphqlClient: ExpoGraphqlClient,
workflowJobId: string
): Promise<WorkflowJobSshConnectInfo | null> {
let data: WorkflowJobSshPollQuery;
try {
data = await withErrorHandlingAsync(
graphqlClient
.query<WorkflowJobSshPollQuery, WorkflowJobSshPollQueryVariables>(
gql`
query WorkflowJobSshPoll($workflowJobId: ID!) {
workflowJobs {
byId(workflowJobId: $workflowJobId) {
id
status
workflowRun {
id
sshSettings {
idleTimeoutSeconds
}
}
turtleJobRun {
id
sshSession {
id
connectionConfig {
host
secret
reconnecting
}
}
}
turtleBuild {
id
sshSession {
id
connectionConfig {
host
secret
reconnecting
}
}
}
}
}
}
`,
{ workflowJobId },
{ requestPolicy: 'network-only' }
)
.toPromise()
);
} catch (error) {
if (isNotFoundError(error)) {
return null;
}
throw error;
}
return toConnectInfo(data.workflowJobs.byId);
},
async connectInfoForJobRunAsync(
graphqlClient: ExpoGraphqlClient,
jobRunId: string
): Promise<WorkflowJobSshConnectInfo | null> {
let data: JobRunSshPollQuery;
try {
data = await withErrorHandlingAsync(
graphqlClient
.query<JobRunSshPollQuery, JobRunSshPollQueryVariables>(
gql`
query JobRunSshPoll($jobRunId: ID!) {
jobRun {
byId(jobRunId: $jobRunId) {
id
status
sshSession {
id
connectionConfig {
host
secret
reconnecting
}
}
}
}
}
`,
{ jobRunId },
{ requestPolicy: 'network-only' }
)
.toPromise()
);
} catch (error) {
if (isNotFoundError(error)) {
return null;
}
throw error;
}
const jobRun = data.jobRun.byId;
const jobCompleted = FINAL_JOB_RUN_STATUSES.has(jobRun.status);
return {
sshRequested: jobRun.sshSession != null || jobCompleted,
jobCompleted,
session: jobRun.sshSession,
};
},
async connectInfoForResourceIdAsync(
graphqlClient: ExpoGraphqlClient,
resourceId: string
): Promise<WorkflowJobSshConnectInfo | null> {
return (
(await WorkflowJobSshQuery.connectInfoForWorkflowJobAsync(graphqlClient, resourceId)) ??
(await WorkflowJobSshQuery.connectInfoForJobRunAsync(graphqlClient, resourceId))
);
},
};
@@ -0,0 +1,155 @@
import { ExpoGraphqlClient } from '../../../commandUtils/context/contextUtils/createGraphqlClient';
import { GraphqlError } from '../../client';
import { WorkflowJobStatus } from '../../generated';
import { WorkflowJobSshQuery } from '../WorkflowJobSshQuery';
describe(WorkflowJobSshQuery.connectInfoForWorkflowJobAsync.name, () => {
function makeClient(byId: unknown): {
graphqlClient: ExpoGraphqlClient;
query: jest.Mock;
} {
const query = jest.fn().mockReturnValue({
toPromise: async () => ({ data: { workflowJobs: { byId } } }),
});
return { graphqlClient: { query } as unknown as ExpoGraphqlClient, query };
}
it('maps workflowJobs.byId into connect info', async () => {
const { graphqlClient, query } = makeClient({
id: 'job-1',
status: WorkflowJobStatus.InProgress,
workflowRun: { sshSettings: { idleTimeoutSeconds: 0 } },
turtleJobRun: {
sshSession: {
id: 'ts-1',
connectionConfig: { host: 'relay.expo.dev', secret: 'TOKENx', reconnecting: false },
},
},
turtleBuild: null,
});
expect(
await WorkflowJobSshQuery.connectInfoForWorkflowJobAsync(graphqlClient, 'job-1')
).toEqual({
sshRequested: true,
jobCompleted: false,
session: {
id: 'ts-1',
connectionConfig: { host: 'relay.expo.dev', secret: 'TOKENx', reconnecting: false },
},
});
expect(query).toHaveBeenCalledWith(
expect.anything(),
{ workflowJobId: 'job-1' },
{ requestPolicy: 'network-only' }
);
});
it('is not ssh-requested when the run has no sshSettings', async () => {
const { graphqlClient } = makeClient({
id: 'job-1',
status: WorkflowJobStatus.InProgress,
workflowRun: { sshSettings: null },
turtleJobRun: null,
turtleBuild: { sshSession: null },
});
expect(
await WorkflowJobSshQuery.connectInfoForWorkflowJobAsync(graphqlClient, 'job-1')
).toMatchObject({ sshRequested: false });
});
it('returns null when the job is not found', async () => {
const query = jest.fn().mockReturnValue({
toPromise: async () => ({
data: undefined,
error: new GraphqlError({
graphQLErrors: [
{ message: 'Entity not found', extensions: { errorCode: 'ENTITY_NOT_FOUND' } },
],
networkError: undefined,
response: undefined,
}),
}),
});
const graphqlClient = { query } as unknown as ExpoGraphqlClient;
expect(
await WorkflowJobSshQuery.connectInfoForWorkflowJobAsync(graphqlClient, 'missing')
).toBeNull();
});
it('returns null for not-found errors matched by message or code', async () => {
for (const graphQLErrors of [
[{ message: 'Workflow job not found', extensions: {} }],
[{ message: 'missing', extensions: { code: 'ENTITY_NOT_FOUND' } }],
]) {
const query = jest.fn().mockReturnValue({
toPromise: async () => ({
data: undefined,
error: new GraphqlError({
graphQLErrors,
networkError: undefined,
response: undefined,
}),
}),
});
const graphqlClient = { query } as unknown as ExpoGraphqlClient;
expect(
await WorkflowJobSshQuery.connectInfoForWorkflowJobAsync(graphqlClient, 'missing')
).toBeNull();
}
});
it('uses the turtleBuild session when there is no turtleJobRun', async () => {
const { graphqlClient } = makeClient({
id: 'job-1',
status: WorkflowJobStatus.Success,
workflowRun: { sshSettings: { idleTimeoutSeconds: 60 } },
turtleJobRun: null,
turtleBuild: {
sshSession: {
id: 'ts-build',
connectionConfig: { host: 'relay.expo.dev', secret: 'TOK', reconnecting: true },
},
},
});
expect(
await WorkflowJobSshQuery.connectInfoForWorkflowJobAsync(graphqlClient, 'job-1')
).toEqual({
sshRequested: true,
jobCompleted: true,
session: {
id: 'ts-build',
connectionConfig: { host: 'relay.expo.dev', secret: 'TOK', reconnecting: true },
},
});
});
it('rethrows GraphQL errors that are not not-found', async () => {
const error = new GraphqlError({
graphQLErrors: [{ message: 'boom', extensions: { errorCode: 'INTERNAL' } }],
networkError: undefined,
response: undefined,
});
const query = jest.fn().mockReturnValue({
toPromise: async () => ({ data: undefined, error }),
});
const graphqlClient = { query } as unknown as ExpoGraphqlClient;
await expect(
WorkflowJobSshQuery.connectInfoForWorkflowJobAsync(graphqlClient, 'job-1')
).rejects.toBe(error);
});
it('rethrows unexpected non-GraphQL errors', async () => {
const query = jest.fn().mockReturnValue({
toPromise: async () => {
throw new Error('network down');
},
});
const graphqlClient = { query } as unknown as ExpoGraphqlClient;
await expect(
WorkflowJobSshQuery.connectInfoForWorkflowJobAsync(graphqlClient, 'job-1')
).rejects.toThrow('network down');
});
});
-109
View File
@@ -1,109 +0,0 @@
import spawnAsync from '@expo/spawn-async';
import { runCommandAsync } from './runCommand';
import Log from '../log';
import { confirmAsync, promptAsync } from '../prompts';
export async function runGitCloneAsync({
targetProjectDir,
githubRepositoryName,
githubUsername,
cloneMethod,
showOutput = true,
}: {
githubUsername: string;
githubRepositoryName: string;
targetProjectDir: string;
cloneMethod: 'ssh' | 'https';
showOutput?: boolean;
}): Promise<{
targetProjectDir: string;
}> {
const url =
cloneMethod === 'ssh'
? `[email protected]:${githubUsername}/${githubRepositoryName}.git`
: `https://github.com/${githubUsername}/${githubRepositoryName}.git`;
try {
await runCommandAsync({
command: 'git',
args: ['clone', url, targetProjectDir],
shouldPrintStderrLineAsStdout: line => {
return line.includes('Cloning into');
},
showSpinner: false,
showOutput,
});
return { targetProjectDir };
} catch (error: any) {
if (error.stderr.includes('already exists')) {
Log.warn(`Directory ${targetProjectDir} already exists.`);
const shouldContinue = await confirmAsync({
message: 'Do you want to clone your project to some other destination?',
});
if (!shouldContinue) {
throw new Error('Directory already exists. Aborting...');
}
const { newTargetProjectDir } = await promptAsync({
type: 'text',
name: 'newTargetProjectDir',
message: 'New target directory path:',
validate: (input: string) => input !== '',
});
return await runGitCloneAsync({
githubRepositoryName,
githubUsername,
targetProjectDir: newTargetProjectDir,
cloneMethod,
});
} else if (error.stderr.includes('Permission denied')) {
Log.warn(
`It seems like you do not have permission to clone the repository using ${cloneMethod}.`
);
const newMethod = cloneMethod === 'ssh' ? 'https' : 'ssh';
const shouldContinue = await confirmAsync({
message: `Do you want to clone the repository using ${newMethod} instead?`,
});
if (!shouldContinue) {
throw new Error('Permission denied. Aborting...');
}
return await runGitCloneAsync({
githubRepositoryName,
githubUsername,
targetProjectDir,
cloneMethod: newMethod,
});
} else {
throw error;
}
}
}
export async function runGitPushAsync({
targetProjectDir,
}: {
targetProjectDir: string;
}): Promise<void> {
await runCommandAsync({
command: 'git',
args: ['push'],
cwd: targetProjectDir,
});
}
export async function canAccessRepositoryUsingSshAsync({
githubUsername,
githubRepositoryName,
}: {
githubUsername: string;
githubRepositoryName: string;
}): Promise<boolean> {
try {
await spawnAsync('git', [
'ls-remote',
`[email protected]:${githubUsername}/${githubRepositoryName}.git`,
]);
return true;
} catch {
return false;
}
}
+15
View File
@@ -110,3 +110,18 @@ export function ora(options?: Options | string): Ora {
return spinner;
}
/**
* Updates the text of a spinner. Does nothing if new text is the same as old text to prevent flickering.
*/
export function updateSpinnerText(
spinner: Ora,
{ prefixText, text }: { prefixText?: string; text?: string }
): void {
if (prefixText !== undefined && spinner.prefixText !== prefixText) {
spinner.prefixText = prefixText;
}
if (text !== undefined && spinner.text !== text) {
spinner.text = text;
}
}
@@ -0,0 +1,112 @@
import { compileModsAsync } from '@expo/config-plugins';
import { getPrebuildConfigAsync } from '@expo/prebuild-config';
import { spawnExpoCommand } from '../../../utils/expoCli';
import { Client } from '../../../vcs/vcs';
import { isExpoInstalled } from '../../projectUtils';
import { hasIgnoredIosProjectAsync } from '../../workflow';
import { getManagedApplicationTargetEntitlementsAsync } from '../entitlements';
jest.mock('@expo/config-plugins', () => ({
...jest.requireActual('@expo/config-plugins'),
compileModsAsync: jest.fn(),
}));
jest.mock('@expo/prebuild-config');
jest.mock('../../../log', () => ({
__esModule: true,
default: { warn: jest.fn() },
}));
jest.mock('../../../sentry', () => ({
__esModule: true,
default: {
withScope: jest.fn(),
captureMessage: jest.fn(),
},
}));
jest.mock('../../../utils/expoCli');
jest.mock('../../projectUtils');
jest.mock('../../workflow');
const projectDir = '/project';
const vcsClient = {} as Client;
const prebuildConfig = {
name: 'test-app',
slug: 'test-app',
ios: { bundleIdentifier: 'com.example.test' },
};
const entitlements = {
'aps-environment': 'production',
'com.apple.developer.associated-domains': ['applinks:example.com'],
};
beforeEach(() => {
jest.resetAllMocks();
jest.mocked(hasIgnoredIosProjectAsync).mockResolvedValue(true);
jest.mocked(getPrebuildConfigAsync).mockResolvedValue({ exp: prebuildConfig } as any);
jest.mocked(compileModsAsync).mockResolvedValue({
...prebuildConfig,
ios: { ...prebuildConfig.ios, entitlements },
} as any);
});
it('uses the bundled config fallback when Expo is not installed', async () => {
jest.mocked(isExpoInstalled).mockReturnValue(false);
const originalProcessEnv = process.env;
let envDuringPrebuild: string | undefined;
jest.mocked(getPrebuildConfigAsync).mockImplementation(async () => {
envDuringPrebuild = process.env.TEST_ENTITLEMENTS_ENV;
return { exp: prebuildConfig } as any;
});
await expect(
getManagedApplicationTargetEntitlementsAsync(
projectDir,
{ TEST_ENTITLEMENTS_ENV: 'from-build-profile' },
vcsClient
)
).resolves.toEqual(entitlements);
expect(spawnExpoCommand).not.toHaveBeenCalled();
expect(getPrebuildConfigAsync).toHaveBeenCalledWith(projectDir, { platforms: ['ios'] });
expect(compileModsAsync).toHaveBeenCalledWith(prebuildConfig, {
projectRoot: projectDir,
platforms: ['ios'],
introspect: true,
ignoreExistingNativeFiles: true,
});
expect(envDuringPrebuild).toBe('from-build-profile');
expect(process.env).toBe(originalProcessEnv);
});
it('uses the bundled config fallback when the local Expo CLI fails', async () => {
jest.mocked(isExpoInstalled).mockReturnValue(true);
jest
.mocked(spawnExpoCommand)
.mockRejectedValue(
Object.assign(new Error('local Expo CLI failed'), { stderr: 'config failed' })
);
await expect(
getManagedApplicationTargetEntitlementsAsync(projectDir, {}, vcsClient)
).resolves.toEqual(entitlements);
expect(spawnExpoCommand).toHaveBeenCalledWith(
projectDir,
['config', '--json', '--type', 'introspect'],
{ env: expect.objectContaining({ EXPO_NO_DOTENV: '1' }) }
);
expect(getPrebuildConfigAsync).toHaveBeenCalledWith(projectDir, { platforms: ['ios'] });
});
it('does not make Expo a required peer of the bundled config fallback', () => {
const packageJson = require('@expo/prebuild-config/package.json') as {
peerDependencies?: Record<string, string>;
peerDependenciesMeta?: Record<string, { optional?: boolean }>;
};
const hasRequiredExpoPeer =
packageJson.peerDependencies?.expo !== undefined &&
packageJson.peerDependenciesMeta?.expo?.optional !== true;
expect(hasRequiredExpoPeer).toBe(false);
});
@@ -2,6 +2,8 @@ import { DeviceRunSessionResourceClass, DeviceRunSessionType } from '../../graph
import {
DEVICE_RUN_SESSION_RESOURCE_CLASS_BY_FLAG_VALUE,
DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE,
DEVICE_RUN_SESSION_TYPE_FLAG_VALUES,
deviceRunSessionTypeToFlagValue,
formatRemoteSessionInstructions,
getRemoteSessionEnvironmentVariables,
} from '../utils';
@@ -35,10 +37,32 @@ describe('Appium simulator configuration', () => {
expect(instructions).toContain('eas simulator:exec <appium-client> [args...]');
expect(instructions).toContain('https://preview.example.test');
expect(instructions).toContain('Open the simulator preview:');
expect(instructions).not.toContain('iOS simulator preview');
expect(instructions).not.toContain('https://appium.example.test');
});
});
describe('simulator session type flags', () => {
it('maps web-preview-only to the WebPreviewOnly GraphQL enum', () => {
expect(DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE['web-preview-only']).toBe(
DeviceRunSessionType.WebPreviewOnly
);
expect(DEVICE_RUN_SESSION_TYPE_FLAG_VALUES[DeviceRunSessionType.ServeSim]).toBe(
'web-preview-only'
);
expect(DEVICE_RUN_SESSION_TYPE_FLAG_VALUES[DeviceRunSessionType.WebPreviewOnly]).toBe(
'web-preview-only'
);
expect(deviceRunSessionTypeToFlagValue(DeviceRunSessionType.ServeSim)).toBe('web-preview-only');
expect(deviceRunSessionTypeToFlagValue(DeviceRunSessionType.WebPreviewOnly)).toBe(
'web-preview-only'
);
expect(DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE['serve-sim']).toBeUndefined();
expect(DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE['web-preview']).toBeUndefined();
});
});
describe('simulator resource class flags', () => {
it('maps CLI values to the GraphQL enum', () => {
expect(DEVICE_RUN_SESSION_RESOURCE_CLASS_BY_FLAG_VALUE.large).toBe(
+10 -2
View File
@@ -28,7 +28,8 @@ export const DEVICE_RUN_SESSION_TYPE_FLAG_VALUES: Record<DeviceRunSessionType, s
[DeviceRunSessionType.AgentDevice]: 'agent-device',
[DeviceRunSessionType.Appium]: 'appium',
[DeviceRunSessionType.Argent]: 'argent',
[DeviceRunSessionType.ServeSim]: 'serve-sim',
[DeviceRunSessionType.ServeSim]: 'web-preview-only',
[DeviceRunSessionType.WebPreviewOnly]: 'web-preview-only',
};
export const DEVICE_RUN_SESSION_TYPE_BY_FLAG_VALUE = Object.fromEntries(
@@ -78,6 +79,7 @@ export function getRemoteSessionEnvironmentVariables(
APPIUM_CAPS: JSON.stringify(remoteConfig.capabilities),
};
case 'ServeSimRunSessionRemoteConfig':
case 'WebPreviewOnlyRunSessionRemoteConfig':
return {};
}
}
@@ -172,7 +174,7 @@ export function formatRemoteSessionInstructions(
'<appium-client> [args...]',
];
if (remoteConfig.webPreviewUrl) {
lines.push('', 'Open the iOS simulator preview:', '', remoteConfig.webPreviewUrl);
lines.push('', 'Open the simulator preview:', '', remoteConfig.webPreviewUrl);
}
return lines.join('\n');
}
@@ -182,5 +184,11 @@ export function formatRemoteSessionInstructions(
'',
remoteConfig.previewUrl,
].join('\n');
case 'WebPreviewOnlyRunSessionRemoteConfig':
return [
'🌐 Open the following URL in your browser to access the simulator:',
'',
remoteConfig.previewUrl,
].join('\n');
}
}
@@ -0,0 +1,26 @@
import { Centrifuge } from 'centrifuge';
import { createRealtimeLogsClient } from '../centrifuge';
jest.mock('centrifuge');
describe(createRealtimeLogsClient, () => {
afterEach(() => {
jest.clearAllMocks();
});
it('returns null instead of throwing when the connection cannot be created', () => {
jest.mocked(Centrifuge).mockImplementationOnce(() => {
throw new Error('unsupported url scheme');
});
expect(createRealtimeLogsClient({} as any)).toBeNull();
});
it('connects and returns a client when the connection can be created', () => {
expect(createRealtimeLogsClient({} as any)).not.toBeNull();
const centrifuge = jest.mocked(Centrifuge).mock.instances[0];
expect(jest.mocked(centrifuge.connect)).toHaveBeenCalled();
});
});
+117
View File
@@ -0,0 +1,117 @@
import { Centrifuge, Subscription } from 'centrifuge';
import { Agent } from 'https';
import WebSocket from 'ws';
import { getEASLogsWebsocketUrl } from '../api';
import { ExpoGraphqlClient } from '../commandUtils/context/contextUtils/createGraphqlClient';
import { RealtimeLogsMutationGenerateCentrifugoSubscriptionTokenArgs } from '../graphql/generated';
import { RealtimeLogsMutation } from '../graphql/mutations/RealtimeLogsMutation';
import { httpsProxyAgent } from '../fetch';
import Log from '../log';
export type RealtimeLogsSubscription = {
close: () => void;
};
export type RealtimeLogsClient = {
subscribeAsync: (
args: RealtimeLogsMutationGenerateCentrifugoSubscriptionTokenArgs,
onPublication: (data: unknown) => void
) => Promise<RealtimeLogsSubscription | null>;
close: () => void;
};
type WebSocketConstructor = new (address: string, protocols?: string | string[]) => WebSocket;
function createProxiedWebSocketConstructor(agent: Agent): WebSocketConstructor {
return class ProxiedWebSocket extends WebSocket {
constructor(address: string | URL, protocols?: string | string[]) {
super(address, protocols, { agent });
}
};
}
function createWebSocketConstructor(): WebSocketConstructor {
return httpsProxyAgent ? createProxiedWebSocketConstructor(httpsProxyAgent) : WebSocket;
}
export function createRealtimeLogsClient(
graphqlClient: ExpoGraphqlClient
): RealtimeLogsClient | null {
const getTokenAsync = async (): Promise<string> => {
const { token } =
await RealtimeLogsMutation.generateCentrifugoConnectionTokenAsync(graphqlClient);
return token;
};
let client: Centrifuge;
try {
client = new Centrifuge(getEASLogsWebsocketUrl(), {
websocket: createWebSocketConstructor(),
getToken: getTokenAsync,
});
client.on('error', ({ error }) => {
Log.debug(`Realtime logs connection error: ${error.message}`);
});
client.connect();
} catch (err: any) {
Log.debug(`Failed to connect to realtime logs: ${err.message}`);
return null;
}
return {
subscribeAsync: async (args, onPublication) => {
const getSubscriptionTokenAsync = async (): Promise<string> => {
const { token } = await RealtimeLogsMutation.generateCentrifugoSubscriptionTokenAsync(
graphqlClient,
args
);
return token;
};
try {
const { channel, token } =
await RealtimeLogsMutation.generateCentrifugoSubscriptionTokenAsync(graphqlClient, args);
const subscription = client.newSubscription(channel, {
token,
getToken: getSubscriptionTokenAsync,
});
subscription.on('error', ({ error }) => {
Log.debug(`Realtime logs subscription error on ${channel}: ${error.message}`);
});
subscription.on('publication', ({ data }) => {
onPublication(data);
});
subscription.subscribe();
return {
close: () => {
closeSubscription(client, subscription);
},
};
} catch (err: any) {
Log.debug(`Failed to subscribe to realtime logs: ${err.message}`);
return null;
}
},
close: () => {
for (const subscription of Object.values(client.subscriptions())) {
closeSubscription(client, subscription);
}
client.removeAllListeners();
client.disconnect();
},
};
}
function closeSubscription(client: Centrifuge, subscription: Subscription): void {
try {
subscription.unsubscribe();
subscription.removeAllListeners();
client.removeSubscription(subscription);
} catch (err: any) {
Log.debug(`Failed to close realtime logs subscription: ${err.message}`);
}
}
@@ -18,4 +18,12 @@ describe(uniqBy, () => {
{ a: 4, b: 12 },
]);
});
it('calls getKey once per item', () => {
const getKey = jest.fn(({ a }: { a: number }) => a);
uniqBy([{ a: 1 }, { a: 2 }, { a: 2 }], getKey);
expect(getKey).toHaveBeenCalledTimes(3);
});
});
@@ -1,10 +1,11 @@
export default function uniqBy<T, K = any>(list: T[], getKey: (item: T) => K): T[] {
const uniqueValues = new Set();
const uniqueValues = new Set<K>();
const result: T[] = [];
for (const i of list) {
if (!uniqueValues.has(getKey(i))) {
const key = getKey(i);
if (!uniqueValues.has(key)) {
result.push(i);
uniqueValues.add(getKey(i));
uniqueValues.add(key);
}
}
return result;
@@ -0,0 +1,70 @@
import fs from 'node:fs';
import { Readable } from 'node:stream';
import { extract } from 'tar-stream';
import zlib from 'node:zlib';
import { FileEntry, packFilesIterableAsync } from '../assets';
async function extractTarEntriesAsync(tarData: Buffer): Promise<Map<string, string>> {
const entries = new Map<string, string>();
const extractor = extract();
extractor.on('entry', (header, stream, next) => {
const chunks: Buffer[] = [];
stream.on('data', chunk => chunks.push(chunk));
stream.on('end', () => {
entries.set(header.name, Buffer.concat(chunks).toString('utf8'));
next();
});
stream.resume();
});
await new Promise<void>((resolve, reject) => {
extractor.on('finish', resolve);
extractor.on('error', reject);
Readable.from(tarData).pipe(extractor);
});
return entries;
}
describe(packFilesIterableAsync, () => {
let tarPath: string | null = null;
afterEach(async () => {
if (tarPath) {
await fs.promises.rm(tarPath, { force: true });
tarPath = null;
}
});
it('packs file entries into a gzip-compressed tarball', async () => {
const files: FileEntry[] = [
['index.html', '<h1>hello</h1>'],
['assets/data.json', '{"key":"value"}'],
];
tarPath = await packFilesIterableAsync(files);
const compressed = await fs.promises.readFile(tarPath);
// gzip magic bytes
expect(compressed[0]).toBe(0x1f);
expect(compressed[1]).toBe(0x8b);
// portable mode zeroes the gzip MTIME field (bytes 4-7)
expect(compressed.subarray(4, 8)).toEqual(Buffer.from([0, 0, 0, 0]));
const entries = await extractTarEntriesAsync(zlib.gunzipSync(compressed));
expect(entries.get('index.html')).toBe('<h1>hello</h1>');
expect(entries.get('assets/data.json')).toBe('{"key":"value"}');
expect(entries.size).toBe(2);
});
it('accepts async iterables and forwards gzip options', async () => {
async function* files(): AsyncIterable<FileEntry> {
yield ['a.txt', 'aaaa'];
yield ['b.txt', Buffer.from('bbbb')];
}
tarPath = await packFilesIterableAsync(files(), { level: 9 });
const compressed = await fs.promises.readFile(tarPath);
const entries = await extractTarEntriesAsync(zlib.gunzipSync(compressed));
expect(entries.get('a.txt')).toBe('aaaa');
expect(entries.get('b.txt')).toBe('bbbb');
});
});
@@ -0,0 +1,443 @@
import cliProgress from 'cli-progress';
import fetch, { Headers, Response } from 'node-fetch';
import Log from '../../log';
import { AssetFileEntry } from '../assets';
import {
UploadPayload,
batchUploadAsync,
callUploadApiAsync,
createProgressBar,
uploadAsync,
} from '../upload';
jest.mock('node-fetch', () => ({
...jest.requireActual('node-fetch'),
__esModule: true,
default: jest.fn(),
}));
const mockedFetch = jest.mocked(fetch);
const asset: AssetFileEntry = {
normalizedPath: 'index.html',
path: __filename,
size: 123,
sha512: 'a'.repeat(128),
type: 'text/html',
};
const response = (status = 200, body: unknown = { success: true }): Response =>
new Response(JSON.stringify(body), {
status,
statusText: status === 200 ? 'OK' : 'Request failed',
headers: { 'content-type': 'application/json' },
});
async function runTimersAndExpectRejection(
promise: Promise<unknown>,
expected: string | RegExp
): Promise<void> {
const expectation = expect(promise).rejects.toThrow(expected);
await jest.runAllTimersAsync();
await expectation;
}
describe(uploadAsync, () => {
beforeEach(() => {
jest.useFakeTimers({ now: 0 });
jest.spyOn(Math, 'random').mockReturnValue(0);
jest.spyOn(Log, 'warn').mockImplementation(() => {});
mockedFetch.mockReset();
});
afterEach(() => {
jest.useRealTimers();
jest.restoreAllMocks();
});
it('uploads an individual asset with its hash and metadata', async () => {
mockedFetch.mockResolvedValue(response());
const result = await uploadAsync({ baseURL: 'https://eas.expo.app/?token=secret' }, { asset });
expect(result.payload).toEqual({ asset });
expect(mockedFetch).toHaveBeenCalledTimes(1);
const [url, init] = mockedFetch.mock.calls[0];
expect(String(url)).toBe(`https://eas.expo.app/asset/${asset.sha512}?token=secret`);
expect(init?.method).toBe('POST');
expect((init?.headers as Headers).get('content-type')).toBe('text/html');
expect((init?.headers as Headers).get('content-length')).toBe('123');
expect(init?.body).toBeDefined();
});
it('uploads a worker deployment file without changing the URL', async () => {
mockedFetch.mockResolvedValue(response());
await uploadAsync({ baseURL: 'https://eas.expo.app/deploy' }, { filePath: __filename });
const [url, init] = mockedFetch.mock.calls[0];
expect(String(url)).toBe('https://eas.expo.app/deploy');
expect(init?.method).toBe('POST');
expect(init?.body).toBeDefined();
});
it('uploads multipart assets to the batch endpoint and reports progress', async () => {
mockedFetch.mockResolvedValue(response());
const onProgress = jest.fn();
await uploadAsync(
{ baseURL: 'https://eas.expo.app/asset/?token=secret' },
{ multipart: [asset, { ...asset, sha512: 'b'.repeat(128) }] },
onProgress
);
const [url, init] = mockedFetch.mock.calls[0];
expect(String(url)).toBe('https://eas.expo.app/asset/batch?token=secret');
expect(init?.method).toBe('PATCH');
expect((init?.headers as Headers).get('content-type')).toMatch(/^multipart\/form-data;/);
expect(onProgress).toHaveBeenNthCalledWith(1, 0);
expect(onProgress).toHaveBeenLastCalledWith(1);
});
it.each([408, 409, 429, 500, 503])('retries HTTP %s responses', async status => {
mockedFetch.mockImplementation(async () => response(status, { error: `status ${status}` }));
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset });
await runTimersAndExpectRejection(promise, `status ${status}`);
expect(mockedFetch).toHaveBeenCalledTimes(5);
});
it('does not retry a terminal HTTP response', async () => {
mockedFetch.mockResolvedValue(response(400, { error: 'invalid upload' }));
await expect(uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset })).rejects.toThrow(
'invalid upload'
);
expect(mockedFetch).toHaveBeenCalledTimes(1);
});
it('reports payload-too-large responses without retrying', async () => {
mockedFetch.mockResolvedValue(response(413));
await expect(uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset })).rejects.toThrow(
'File size exceeded the upload limit'
);
expect(mockedFetch).toHaveBeenCalledTimes(1);
});
it('reports CDN HTML errors with the request ID', async () => {
mockedFetch.mockResolvedValue(
new Response('<html></html>', {
status: 403,
statusText: 'Forbidden',
headers: { 'content-type': 'text/html', 'cf-ray': 'ray-id' },
})
);
await expect(uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset })).rejects.toThrow(
'Request ID ray-id'
);
});
it('switches to network mode and preserves the original network error', async () => {
const networkError = Object.assign(new Error('getaddrinfo ENOTFOUND eas.expo.app'), {
code: 'ENOTFOUND',
});
mockedFetch.mockRejectedValue(networkError);
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset });
await runTimersAndExpectRejection(promise, networkError.message);
expect(mockedFetch).toHaveBeenCalledTimes(8);
expect(Log.warn).toHaveBeenCalledTimes(1);
expect(Log.warn).toHaveBeenCalledWith(
`The upload encountered an error but is still retrying: ${networkError.message}`
);
});
it('smoothly increases retries based on the total request count', async () => {
mockedFetch.mockImplementation(async () => response(503, { error: 'overloaded' }));
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
totalRequests: 32,
});
await runTimersAndExpectRejection(promise, 'overloaded');
// log10(32) / 3 results in six retries, plus the initial request.
expect(mockedFetch).toHaveBeenCalledTimes(7);
});
it('caps request-count scaling at 1,000 requests', async () => {
mockedFetch.mockImplementation(async () => response(503, { error: 'overloaded' }));
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
totalRequests: 1_000_000,
});
await runTimersAndExpectRejection(promise, 'overloaded');
// The scaled 40s ceiling cuts the last attempt the backoff schedule would have allowed.
expect(mockedFetch).toHaveBeenCalledTimes(8);
});
it('keeps the retry state when it switches to network mode', async () => {
let calls = 0;
mockedFetch.mockImplementation(async () => {
if (++calls <= 7) {
return response(503, { error: 'overloaded' });
}
throw new Error('socket disconnected');
});
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
totalRequests: 1_000,
});
await runTimersAndExpectRejection(promise, 'socket disconnected');
// The warning must not repeat once the network-mode attempts take over.
expect(Log.warn).toHaveBeenCalledTimes(1);
expect(Log.warn).toHaveBeenCalledWith(
'The upload encountered an error but is still retrying: overloaded'
);
});
it('warns before a short retry budget is exhausted', async () => {
mockedFetch.mockImplementation(async () => response(503, { error: 'overloaded' }));
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset });
await runTimersAndExpectRejection(promise, 'overloaded');
expect(Log.warn).toHaveBeenCalledTimes(1);
});
it('stops retrying at the scaled upload ceiling', async () => {
mockedFetch.mockImplementation(async () => response(503, { error: 'overloaded' }));
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
totalRequests: 1_000,
});
await runTimersAndExpectRejection(promise, 'overloaded');
// The ceiling stops the next retry from being scheduled, so the last attempt already in
// flight can overshoot it by up to one maxTimeout (10s at this scale).
expect(Date.now()).toBeLessThanOrEqual(40_000 + 10_000);
});
it('clears the network flag after a success', async () => {
const state = { shared: { hasSeenNetworkError: false, hasWarned: false } };
mockedFetch.mockResolvedValueOnce(response(503)).mockResolvedValueOnce(response());
const promise = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
state,
});
await jest.runAllTimersAsync();
await promise;
expect(state.shared.hasSeenNetworkError).toBe(false);
});
it('keeps retry warning clocks independent when retry state is shared', async () => {
const shared = { hasSeenNetworkError: false, hasWarned: false };
mockedFetch
.mockResolvedValueOnce(response(503, { error: 'overloaded' }))
.mockResolvedValueOnce(response())
.mockImplementation(async () => response(503, { error: 'overloaded' }));
const retryingUpload = uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
state: { shared },
});
await uploadAsync({ baseURL: 'https://eas.expo.app' }, { asset }, undefined, {
state: { shared },
});
await runTimersAndExpectRejection(retryingUpload, 'overloaded');
expect(Log.warn).toHaveBeenCalledTimes(1);
});
it('does not retry or enter network mode after cancellation', async () => {
const controller = new AbortController();
controller.abort();
const abortError = Object.assign(new Error('The operation was aborted'), {
name: 'AbortError',
});
mockedFetch.mockRejectedValue(abortError);
await expect(
uploadAsync({ baseURL: 'https://eas.expo.app', signal: controller.signal }, { asset })
).rejects.toBe(abortError);
expect(mockedFetch).toHaveBeenCalledTimes(1);
expect(Log.warn).not.toHaveBeenCalled();
});
});
describe(callUploadApiAsync, () => {
beforeEach(() => {
jest.useFakeTimers({ now: 0 });
jest.spyOn(Math, 'random').mockReturnValue(0);
jest.spyOn(Log, 'warn').mockImplementation(() => {});
mockedFetch.mockReset();
});
afterEach(() => {
jest.useRealTimers();
jest.restoreAllMocks();
});
it('returns a successful JSON response', async () => {
mockedFetch.mockResolvedValue(response(200, { result: 'complete' }));
await expect(callUploadApiAsync('https://eas.expo.app/finalize')).resolves.toEqual({
result: 'complete',
});
});
it('retries server errors and passes through the final error', async () => {
mockedFetch.mockResolvedValue(response(503));
const promise = callUploadApiAsync('https://eas.expo.app/finalize');
await runTimersAndExpectRejection(promise, 'Deployment failed: Request failed');
// Deploy API calls get a somewhat larger retry budget than a single asset upload.
expect(mockedFetch).toHaveBeenCalledTimes(7);
});
it('retries invalid JSON responses', async () => {
mockedFetch
.mockResolvedValueOnce(new Response('invalid JSON'))
.mockResolvedValueOnce(response(200, { success: true }));
const promise = callUploadApiAsync('https://eas.expo.app/finalize');
await jest.runAllTimersAsync();
await expect(promise).resolves.toEqual({ success: true });
expect(mockedFetch).toHaveBeenCalledTimes(2);
});
it('uses network retries for fetch errors', async () => {
const networkError = new Error('socket disconnected');
mockedFetch.mockRejectedValue(networkError);
const promise = callUploadApiAsync('https://eas.expo.app/finalize');
await runTimersAndExpectRejection(promise, networkError.message);
// Network mode adds attempts but shares the 30s deadline, so it cannot double the wait.
expect(mockedFetch).toHaveBeenCalledTimes(10);
});
it('stops retrying at the 30s ceiling', async () => {
mockedFetch.mockResolvedValue(response(503));
const promise = callUploadApiAsync('https://eas.expo.app/finalize');
await runTimersAndExpectRejection(promise, 'Deployment failed: Request failed');
expect(Date.now()).toBeLessThanOrEqual(30_000);
});
it('shares the ceiling with the network-mode attempts', async () => {
mockedFetch.mockRejectedValue(new Error('socket disconnected'));
const promise = callUploadApiAsync('https://eas.expo.app/finalize');
await runTimersAndExpectRejection(promise, 'socket disconnected');
expect(Date.now()).toBeLessThanOrEqual(30_000 + 5_000);
});
it('does not retry an aborted request', async () => {
const controller = new AbortController();
controller.abort();
const abortError = Object.assign(new Error('The operation was aborted'), {
name: 'AbortError',
});
mockedFetch.mockRejectedValue(abortError);
await expect(
callUploadApiAsync('https://eas.expo.app/finalize', { signal: controller.signal as any })
).rejects.toBe(abortError);
expect(mockedFetch).toHaveBeenCalledTimes(1);
});
});
describe(batchUploadAsync, () => {
beforeEach(() => {
mockedFetch.mockReset();
});
afterEach(() => {
jest.restoreAllMocks();
});
it('uploads every payload and reports aggregate progress', async () => {
mockedFetch.mockResolvedValue(response());
const payloads: UploadPayload[] = [
{ asset },
{ asset: { ...asset, sha512: 'b'.repeat(128) } },
{ asset: { ...asset, sha512: 'c'.repeat(128) } },
];
const progress = jest.fn();
const results = [];
for await (const result of batchUploadAsync(
{ baseURL: 'https://eas.expo.app' },
payloads,
progress
)) {
results.push(result);
}
expect(mockedFetch).toHaveBeenCalledTimes(3);
expect(results.at(-1)?.progress).toBe(1);
expect(progress).toHaveBeenLastCalledWith(1);
});
it('accepts an empty upload queue', async () => {
const results = [];
for await (const result of batchUploadAsync({ baseURL: 'https://eas.expo.app' }, [])) {
results.push(result);
}
expect(results).toEqual([]);
expect(mockedFetch).not.toHaveBeenCalled();
});
it('aborts sibling requests after the first upload failure', async () => {
mockedFetch.mockImplementation(async (url, init) => {
if (String(url).includes(asset.sha512)) {
return response(400, { error: 'terminal failure' });
}
return await new Promise((_resolve, reject) => {
init?.signal?.addEventListener('abort', () => {
reject(new Error('aborted'));
});
});
});
const iterator = batchUploadAsync({ baseURL: 'https://eas.expo.app' }, [
{ asset },
{ asset: { ...asset, sha512: 'b'.repeat(128) } },
]);
await iterator.next();
await iterator.next();
await expect(iterator.next()).rejects.toThrow('terminal failure');
});
});
describe(createProgressBar, () => {
afterEach(() => {
jest.restoreAllMocks();
});
it('starts, updates, and stops the upload progress bar', () => {
const start = jest.spyOn(cliProgress.SingleBar.prototype, 'start').mockImplementation(() => {});
const update = jest
.spyOn(cliProgress.SingleBar.prototype, 'update')
.mockImplementation(() => {});
const stop = jest.spyOn(cliProgress.SingleBar.prototype, 'stop').mockImplementation(() => {});
const progressBar = createProgressBar('Uploading 3 assets');
progressBar.update(0.5);
progressBar.stop();
expect(start).toHaveBeenCalledWith(1, 0);
expect(update).toHaveBeenCalledWith(0.5);
expect(stop).toHaveBeenCalledTimes(1);
});
});
+252 -106
View File
@@ -7,6 +7,7 @@ import os from 'node:os';
import { Readable } from 'node:stream';
import promiseRetry from 'promise-retry';
import Log from '../log';
import { AssetFileEntry } from './assets';
import {
createMultipartBodyFromFilesAsync,
@@ -15,7 +16,86 @@ import {
} from './utils/multipart';
const MAX_CONCURRENCY = Math.min(10, Math.max(os.availableParallelism() * 2, 20));
const MAX_RETRIES = 4;
const MAX_RETRY_WARNING_DELAY_MS = 30_000;
const UPLOAD_RETRY_LIMITS: RetryLimits = { retries: 4, maxTimeout: 5_000, maxRetryTime: 20_000 };
const API_RETRY_LIMITS: RetryLimits = { retries: 6, maxTimeout: 5_000, maxRetryTime: 30_000 };
interface SharedRetryState {
hasSeenNetworkError: boolean;
hasWarned: boolean;
}
interface RetryState {
shared: SharedRetryState;
firstRetryAt?: number;
}
interface UploadRetryOptions {
totalRequests?: number;
state?: RetryState;
deadlineAt?: number;
}
interface RetryLimits {
retries: number;
maxTimeout: number;
maxRetryTime: number;
}
interface RetryOptions {
retries: number;
factor: number;
minTimeout: number;
maxTimeout: number;
maxRetryTime: number;
randomize: boolean;
}
interface RetryWarningContext {
state: RetryState;
warningDelayMs: number;
subject: string;
}
const getRetryScale = (totalRequests: number): number =>
Math.min(1, Math.log10(Math.max(1, totalRequests)) / 3);
const getErrorMessage = (error: unknown): string =>
error instanceof Error ? error.message : String(error);
const getRetryOptions = (
limits: RetryLimits,
totalRequests: number,
networkMode: boolean
): RetryOptions => {
const scale = getRetryScale(totalRequests);
return {
retries: Math.round(limits.retries * (1 + scale)) + (networkMode ? 2 : 0),
factor: 2,
minTimeout: 1_000,
maxTimeout: Math.round(limits.maxTimeout * (1 + scale)),
maxRetryTime: Math.round(limits.maxRetryTime * (1 + scale)),
randomize: true,
};
};
const getRetryWarningDelay = (maxRetryTime: number): number =>
Math.min(MAX_RETRY_WARNING_DELAY_MS, Math.round(maxRetryTime / 2));
const retryWithWarning = (
retry: (error: unknown) => never,
error: unknown,
attempt: number,
{ state, warningDelayMs, subject }: RetryWarningContext
): never => {
state.firstRetryAt ??= Date.now();
if (!state.shared.hasWarned && attempt > 1 && Date.now() - state.firstRetryAt >= warningDelayMs) {
state.shared.hasWarned = true;
Log.warn(`${subject} encountered an error but is still retrying: ${getErrorMessage(error)}`);
}
return retry(error);
};
export type UploadPayload =
| { filePath: string }
@@ -56,110 +136,156 @@ type OnProgressUpdateCallback = (progress: number) => void;
export async function uploadAsync(
init: UploadRequestInit,
payload: UploadPayload,
onProgressUpdate?: OnProgressUpdateCallback
onProgressUpdate?: OnProgressUpdateCallback,
retryOptions: UploadRetryOptions = {},
networkMode = retryOptions.state?.shared.hasSeenNetworkError ?? false
): Promise<UploadResult> {
return await promiseRetry(
async retry => {
if (onProgressUpdate) {
onProgressUpdate(0);
}
const headers = new Headers(init.headers);
const url = new URL(`${init.baseURL}`);
let errorPrefix: string;
let body: BodyInit | undefined;
let method = init.method || 'POST';
if ('asset' in payload) {
const { asset } = payload;
errorPrefix = `Upload of "${asset.normalizedPath}" failed`;
if (asset.type) {
headers.set('content-type', asset.type);
}
if (asset.size) {
headers.set('content-length', `${asset.size}`);
}
method = 'POST';
url.pathname = `/asset/${asset.sha512}`;
body = Readable.from(createReadStreamAsync(asset), { objectMode: false });
} else if ('filePath' in payload) {
const { filePath } = payload;
errorPrefix = 'Worker deployment failed';
body = fs.createReadStream(filePath);
} else if ('multipart' in payload) {
const { multipart } = payload;
errorPrefix = `Upload of ${multipart.length} assets failed`;
headers.set('content-type', multipartContentType);
method = 'PATCH';
url.pathname = '/asset/batch';
body = Readable.from(createMultipartBodyFromFilesAsync(multipart, onProgressUpdate), {
objectMode: false,
});
}
let response: Response;
try {
response = await fetch(url, {
method,
body,
headers,
agent: getAgent(),
signal: init.signal as any,
});
} catch (error) {
return retry(error);
}
const getErrorMessageAsync = async (): Promise<string> => {
const rayId = response.headers.get('cf-ray');
const contentType = response.headers.get('Content-Type');
if (contentType?.startsWith('text/html')) {
// NOTE(@kitten): We've received a CDN error most likely. There's not much we can do
// except for quoting the Request ID, so a user can send it to us. We can check
// why a request was blocked by looking up a WAF event via the "Ray ID" here:
// https://dash.cloudflare.com/e6f39f67f543faa6038768e8f37e4234/expo.app/security/events
let message = `CDN firewall has aborted the upload with ${response.statusText}.`;
if (rayId) {
message += `\nReport this error quoting Request ID ${rayId}`;
}
return `${errorPrefix}: ${message}`;
} else {
const json = await response.json().catch(() => null);
return json?.error ?? `${errorPrefix}: ${response.statusText}`;
}
};
if (
response.status === 408 ||
response.status === 409 ||
response.status === 429 ||
(response.status >= 500 && response.status <= 599)
) {
return retry(new Error(await getErrorMessageAsync()));
} else if (response.status === 413) {
const message = `${errorPrefix!}: File size exceeded the upload limit`;
throw new Error(message);
} else if (!response.ok) {
throw new Error(await getErrorMessageAsync());
} else if (onProgressUpdate) {
onProgressUpdate(1);
}
return {
payload,
response,
};
},
{
retries: MAX_RETRIES,
minTimeout: 50,
randomize: false,
}
const state = retryOptions.state ?? {
shared: { hasSeenNetworkError: false, hasWarned: false },
};
const retryOptionsForAttempts = getRetryOptions(
UPLOAD_RETRY_LIMITS,
retryOptions.totalRequests ?? 1,
networkMode
);
const warningDelayMs = getRetryWarningDelay(retryOptionsForAttempts.maxRetryTime);
const deadlineAt = retryOptions.deadlineAt ?? Date.now() + retryOptionsForAttempts.maxRetryTime;
retryOptionsForAttempts.maxRetryTime = Math.max(0, deadlineAt - Date.now());
const warningContext: RetryWarningContext = {
state,
warningDelayMs,
subject: 'The upload',
};
return await promiseRetry(async (retry, attempt) => {
if (onProgressUpdate) {
onProgressUpdate(0);
}
const headers = new Headers(init.headers);
const url = new URL(`${init.baseURL}`);
let errorPrefix: string;
let body: BodyInit | undefined;
let method = init.method || 'POST';
if ('asset' in payload) {
const { asset } = payload;
errorPrefix = `Upload of "${asset.normalizedPath}" failed`;
if (asset.type) {
headers.set('content-type', asset.type);
}
if (asset.size) {
headers.set('content-length', `${asset.size}`);
}
method = 'POST';
url.pathname = `/asset/${asset.sha512}`;
body = Readable.from(createReadStreamAsync(asset), { objectMode: false });
} else if ('filePath' in payload) {
const { filePath } = payload;
errorPrefix = 'Worker deployment failed';
body = fs.createReadStream(filePath);
} else if ('multipart' in payload) {
const { multipart } = payload;
errorPrefix = `Upload of ${multipart.length} assets failed`;
headers.set('content-type', multipartContentType);
method = 'PATCH';
url.pathname = '/asset/batch';
body = Readable.from(createMultipartBodyFromFilesAsync(multipart, onProgressUpdate), {
objectMode: false,
});
}
let response: Response;
try {
response = await fetch(url, {
method,
body,
headers,
agent: getAgent(),
signal: init.signal as any,
});
} catch (error) {
if (init.signal?.aborted) {
throw error;
}
if (!networkMode) {
state.shared.hasSeenNetworkError = true;
return await uploadAsync(
init,
payload,
onProgressUpdate,
{ ...retryOptions, state, deadlineAt },
true
);
}
return retryWithWarning(retry, error, attempt, warningContext);
}
const getErrorMessageAsync = async (): Promise<string> => {
const rayId = response.headers.get('cf-ray');
const contentType = response.headers.get('Content-Type');
if (contentType?.startsWith('text/html')) {
// NOTE(@kitten): We've received a CDN error most likely. There's not much we can do
// except for quoting the Request ID, so a user can send it to us. We can check
// why a request was blocked by looking up a WAF event via the "Ray ID" here:
// https://dash.cloudflare.com/e6f39f67f543faa6038768e8f37e4234/expo.app/security/events
let message = `CDN firewall has aborted the upload with ${response.statusText}.`;
if (rayId) {
message += `\nReport this error quoting Request ID ${rayId}`;
}
return `${errorPrefix}: ${message}`;
} else {
const json = await response.json().catch(() => null);
return json?.error ?? `${errorPrefix}: ${response.statusText}`;
}
};
if (
response.status === 408 ||
response.status === 409 ||
response.status === 429 ||
(response.status >= 500 && response.status <= 599)
) {
return retryWithWarning(
retry,
new Error(await getErrorMessageAsync()),
attempt,
warningContext
);
} else if (response.status === 413) {
const message = `${errorPrefix!}: File size exceeded the upload limit`;
throw new Error(message);
} else if (!response.ok) {
throw new Error(await getErrorMessageAsync());
} else if (onProgressUpdate) {
onProgressUpdate(1);
}
state.shared.hasSeenNetworkError = false;
return {
payload,
response,
};
}, retryOptionsForAttempts);
}
export async function callUploadApiAsync(url: string | URL, init?: RequestInit): Promise<unknown> {
return await promiseRetry(async retry => {
async function callUploadApiWithRetryAsync(
url: string | URL,
init: RequestInit | undefined,
networkMode = false,
state: RetryState = { shared: { hasSeenNetworkError: false, hasWarned: false } },
deadlineAt?: number
): Promise<unknown> {
const retryOptions = getRetryOptions(API_RETRY_LIMITS, 1, networkMode);
const warningDelayMs = getRetryWarningDelay(retryOptions.maxRetryTime);
const retryDeadlineAt = deadlineAt ?? Date.now() + retryOptions.maxRetryTime;
retryOptions.maxRetryTime = Math.max(0, retryDeadlineAt - Date.now());
const warningContext: RetryWarningContext = {
state,
warningDelayMs,
subject: 'The deployment',
};
return await promiseRetry(async (retry, attempt) => {
let response: Response;
try {
response = await fetch(url, {
@@ -167,17 +293,33 @@ export async function callUploadApiAsync(url: string | URL, init?: RequestInit):
agent: getAgent(),
});
} catch (error) {
return retry(error);
if (init?.signal?.aborted) {
throw error;
}
if (!networkMode) {
state.shared.hasSeenNetworkError = true;
return await callUploadApiWithRetryAsync(url, init, true, state, retryDeadlineAt);
}
return retryWithWarning(retry, error, attempt, warningContext);
}
if (response.status >= 500 && response.status <= 599) {
retry(new Error(`Deployment failed: ${response.statusText}`));
retryWithWarning(
retry,
new Error(`Deployment failed: ${response.statusText}`),
attempt,
warningContext
);
}
try {
return await response.json();
} catch (error) {
retry(error);
return retryWithWarning(retry, error, attempt, warningContext);
}
});
}, retryOptions);
}
export async function callUploadApiAsync(url: string | URL, init?: RequestInit): Promise<unknown> {
return await callUploadApiWithRetryAsync(url, init);
}
export interface UploadPending {
@@ -192,6 +334,7 @@ export async function* batchUploadAsync(
): AsyncGenerator<UploadPending> {
const progressTracker = new Array(payloads.length).fill(0);
const controller = new AbortController();
const sharedRetryState: SharedRetryState = { hasSeenNetworkError: false, hasWarned: false };
const queue = new Set<Promise<UploadResult>>();
const initWithSignal = { ...init, signal: controller.signal };
const getProgressValue = (): number => {
@@ -216,7 +359,10 @@ export async function* batchUploadAsync(
progressTracker[currentIndex] = progress;
sendProgressUpdate();
});
const uploadPromise = uploadAsync(initWithSignal, payload, onChildProgressUpdate).then(
const uploadPromise = uploadAsync(initWithSignal, payload, onChildProgressUpdate, {
totalRequests: payloads.length,
state: { shared: sharedRetryState },
}).then(
result => {
queue.delete(uploadPromise);
progressTracker[currentIndex] = 1;
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/eas-json",
"version": "22.0.0",
"version": "23.1.0",
"description": "A library for interacting with eas.json",
"homepage": "https://github.com/expo/eas-cli",
"bugs": "https://github.com/expo/eas-cli/issues",
@@ -32,7 +32,7 @@
},
"dependencies": {
"@babel/code-frame": "7.23.5",
"@expo/eas-build-job": "22.0.0",
"@expo/eas-build-job": "23.1.0",
"chalk": "4.1.2",
"env-string": "1.0.1",
"fs-extra": "11.2.0",
+6 -7
View File
@@ -1,6 +1,6 @@
{
"name": "eas-cli-local-build-plugin",
"version": "22.6.0",
"version": "23.2.0",
"description": "Tool for running EAS compatible builds on a local machine.",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "BUSL-1.1",
@@ -26,12 +26,12 @@
"clean": "rimraf node_modules dist coverage \"*.tsbuildinfo\""
},
"dependencies": {
"@expo/build-tools": "22.6.0",
"@expo/eas-build-job": "22.0.0",
"@expo/logger": "22.0.0",
"@expo/build-tools": "23.2.0",
"@expo/bunyan": "^4.0.1",
"@expo/eas-build-job": "23.1.0",
"@expo/logger": "23.1.0",
"@expo/spawn-async": "^1.7.2",
"@expo/turtle-spawn": "22.0.0",
"bunyan": "^1.8.15",
"@expo/turtle-spawn": "23.1.0",
"chalk": "4.1.2",
"env-paths": "^2.2.1",
"fs-extra": "^11.2.0",
@@ -43,7 +43,6 @@
"uuid": "^9.0.1"
},
"devDependencies": {
"@types/bunyan": "^1.8.11",
"@types/fs-extra": "^11.0.4",
"@types/hapi__joi": "^17.1.14",
"@types/lodash": "^4.17.4",
+1 -1
View File
@@ -1,6 +1,6 @@
import { Artifacts, SkipNativeBuildError } from '@expo/build-tools';
import { ArchiveSourceType, BuildJob, Metadata, Platform, Workflow } from '@expo/eas-build-job';
import { DEBUG } from 'bunyan';
import { DEBUG } from '@expo/bunyan';
import chalk from 'chalk';
import fs from 'fs-extra';
import pickBy from 'lodash/pickBy';
+6 -3
View File
@@ -1,6 +1,7 @@
import { LogBuffer } from '@expo/build-tools';
import { LoggerLevel } from '@expo/logger';
import bunyan from 'bunyan';
import type { bunyan as ExpoLogger } from '@expo/logger';
import bunyan from '@expo/bunyan';
import chalk from 'chalk';
import omit from 'lodash/omit';
import { Writable } from 'stream';
@@ -129,7 +130,9 @@ class PrettyStream extends Writable {
export const logBuffer = new BuildCliLogBuffer(MAX_LINES_IN_BUFFER);
export function createLogger(level?: LoggerLevel): bunyan {
export function createLogger(level?: LoggerLevel): ExpoLogger {
// @expo/bunyan provides the Bunyan API used here, but its stricter stream types are not
// structurally compatible with the legacy @types/bunyan types exposed by @expo/logger.
return bunyan.createLogger({
name: 'eas-build-cli',
serializers: bunyan.stdSerializers,
@@ -143,5 +146,5 @@ export function createLogger(level?: LoggerLevel): bunyan {
stream: logBuffer,
},
],
});
}) as unknown as ExpoLogger;
}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/logger",
"version": "22.0.0",
"version": "23.1.0",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "BUSL-1.1",
"author": "Expo <[email protected]>",
+192
View File
@@ -0,0 +1,192 @@
import fs from 'fs';
import os from 'os';
import path from 'path';
import { Readable } from 'stream';
import bunyan from 'bunyan';
import defaultLogger, { LoggerLevel, PipeMode, createLogger, pipeSpawnOutput } from '../index';
// Collects raw bunyan records written to the logger.
function createRecordCollector(logger: ReturnType<typeof createLogger>): any[] {
const records: any[] = [];
logger.addStream({
type: 'raw',
stream: {
write(record: any) {
records.push(record);
},
} as any,
level: bunyan.TRACE,
});
return records;
}
function createSilentLogger(): ReturnType<typeof createLogger> {
const logger = createLogger({ name: 'test-logger', streams: [] });
return logger;
}
describe(createLogger, () => {
it('creates a logger that writes records with name, level, and msg', () => {
const logger = createSilentLogger();
const records = createRecordCollector(logger);
logger.info('hello world');
expect(records).toHaveLength(1);
expect(records[0].name).toBe('test-logger');
expect(records[0].level).toBe(bunyan.INFO);
expect(records[0].msg).toBe('hello world');
});
it('supports structured fields and util.format-style messages', () => {
const logger = createSilentLogger();
const records = createRecordCollector(logger);
logger.info({ phase: 'INSTALL_DEPENDENCIES' }, 'step %d done', 2);
expect(records[0].phase).toBe('INSTALL_DEPENDENCIES');
expect(records[0].msg).toBe('step 2 done');
});
it('serializes errors with the standard err serializer', () => {
const logger = createSilentLogger();
const records = createRecordCollector(logger);
logger.error(new Error('boom'));
expect(records[0].err.message).toBe('boom');
expect(records[0].err.stack).toEqual(expect.stringContaining('boom'));
expect(records[0].msg).toBe('boom');
});
it('creates child loggers that inherit fields and add their own', () => {
const logger = createSilentLogger();
const records = createRecordCollector(logger);
const child = logger.child({ source: 'stdout' });
child.warn('from child');
expect(records[0].name).toBe('test-logger');
expect(records[0].source).toBe('stdout');
expect(records[0].level).toBe(bunyan.WARN);
});
it('respects the configured level', () => {
const logger = createLogger({ name: 'test-logger', streams: [] });
const records: any[] = [];
logger.addStream({
type: 'raw',
stream: {
write(record: any) {
records.push(record);
},
} as any,
level: bunyan.INFO,
});
logger.debug('too quiet');
logger.info('loud enough');
expect(records).toHaveLength(1);
expect(records[0].msg).toBe('loud enough');
});
it('reports and sets the level programmatically', () => {
const logger = createLogger({
name: 'test-logger',
streams: [
{
type: 'raw',
stream: { write() {} } as any,
level: LoggerLevel.INFO,
},
],
});
expect(logger.level()).toBe(bunyan.INFO);
logger.level(LoggerLevel.DEBUG);
expect(logger.level()).toBe(bunyan.DEBUG);
});
it('supports rotating-file streams', async () => {
const directory = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'expo-logger-test-'));
const logPath = path.join(directory, 'rotating.log');
let rotatingStream: any;
try {
const logger = createLogger({
name: 'test-logger',
streams: [
{
type: 'rotating-file',
path: logPath,
period: '1d',
count: 1,
},
],
});
rotatingStream = (logger as any).streams[0].stream;
logger.info('written to a rotating file');
await new Promise<void>((resolve, reject) => {
rotatingStream.stream.once('error', reject);
rotatingStream.stream.end(resolve);
});
expect(await fs.promises.readFile(logPath, 'utf8')).toContain('written to a rotating file');
} finally {
clearTimeout(rotatingStream?.timeout);
rotatingStream?.stream.destroy();
await fs.promises.rm(directory, { recursive: true, force: true });
}
});
});
describe('defaultLogger', () => {
it('is configured with the expo-logger name at INFO level', () => {
expect(defaultLogger.fields.name).toBe('expo-logger');
expect(defaultLogger.level()).toBe(bunyan.INFO);
});
});
describe(pipeSpawnOutput, () => {
async function pipeAndCollect(
stdoutData: string | null,
stderrData: string | null,
options?: Parameters<typeof pipeSpawnOutput>[2]
): Promise<any[]> {
const logger = createSilentLogger();
const records = createRecordCollector(logger);
const stdout = stdoutData === null ? null : Readable.from([stdoutData]);
const stderr = stderrData === null ? null : Readable.from([stderrData]);
pipeSpawnOutput(logger, { stdout, stderr }, options);
await new Promise(resolve => setImmediate(resolve));
return records;
}
it('logs stdout and stderr lines tagged with their source', async () => {
const records = await pipeAndCollect('out line 1\nout line 2\n', 'err line\n');
const stdoutRecords = records.filter(r => r.source === 'stdout');
const stderrRecords = records.filter(r => r.source === 'stderr');
expect(stdoutRecords.map(r => r.msg)).toEqual(['out line 1', 'out line 2']);
expect(stderrRecords.map(r => r.msg)).toEqual(['err line']);
});
it('tags stderr as stdout in COMBINED_AS_STDOUT mode', async () => {
const records = await pipeAndCollect('out\n', 'err\n', { mode: PipeMode.COMBINED_AS_STDOUT });
expect(records).toHaveLength(2);
expect(records.every(r => r.source === 'stdout')).toBe(true);
});
it('applies the line transformer and drops lines it nulls out', async () => {
const records = await pipeAndCollect('keep\ndrop\n', null, {
lineTransformer: line => (line === 'drop' ? null : `[prefix] ${line}`),
});
expect(records.map(r => r.msg)).toEqual(['[prefix] keep']);
});
});
+5 -6
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/steps",
"version": "22.4.0",
"version": "23.1.0",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "BUSL-1.1",
"author": "Expo <[email protected]>",
@@ -27,8 +27,8 @@
"clean": "rimraf node_modules dist coverage \"*.tsbuildinfo\""
},
"dependencies": {
"@expo/eas-build-job": "22.0.0",
"@expo/logger": "22.0.0",
"@expo/eas-build-job": "23.1.0",
"@expo/logger": "23.1.0",
"@expo/spawn-async": "^1.7.2",
"arg": "^5.0.2",
"fast-glob": "^3.3.2",
@@ -36,8 +36,7 @@
"joi": "^17.13.1",
"jsep": "^1.3.8",
"lodash.clonedeep": "^4.5.0",
"lodash.get": "^4.4.2",
"uuid": "^9.0.1",
"uuid": "^11.1.1",
"yaml": "^2.4.3",
"zod": "^4.3.5"
},
@@ -45,8 +44,8 @@
"@jest/globals": "^29.7.0",
"@types/jest": "^29.5.12",
"@types/lodash.clonedeep": "^4.5.9",
"@types/lodash.get": "^4.4.9",
"@types/node": "20.14.2",
"bunyan": "^1.8.15",
"eslint-plugin-async-protect": "^3.1.0",
"jest": "^29.7.0",
"rimraf": "3.0.2",
@@ -0,0 +1,92 @@
import { get, parsePath } from '../get';
describe(parsePath, () => {
it('splits dot-separated paths', () => {
expect(parsePath('steps.step1.outputs.out')).toEqual(['steps', 'step1', 'outputs', 'out']);
});
it('parses bracketed array indices', () => {
expect(parsePath('list[0].name')).toEqual(['list', '0', 'name']);
expect(parsePath('a[0][1]')).toEqual(['a', '0', '1']);
});
it('parses bracketed keys without quotes as emitted by jsepEval', () => {
expect(parsePath('a[b]')).toEqual(['a', 'b']);
});
it('parses quoted bracketed keys, keeping dots inside them intact', () => {
expect(parsePath('a["b.c"]')).toEqual(['a', 'b.c']);
expect(parsePath("a['b']")).toEqual(['a', 'b']);
});
it('throws on unterminated brackets', () => {
expect(() => parsePath('a[b')).toThrow('unterminated bracket');
expect(() => parsePath('a["b]')).toThrow('unterminated quoted bracket');
});
});
describe(get, () => {
const context = {
steps: {
step1: {
outputs: {
my_output: 'value1',
},
},
},
eas: {
job: {
version: { buildNumber: '42' },
secrets: null,
list: ['first', { name: 'second' }],
},
runtimeVersion: undefined,
},
};
it('resolves dot-separated paths, as used by template interpolation', () => {
expect(get(context, 'steps.step1.outputs.my_output')).toBe('value1');
expect(get(context, 'eas.job.version.buildNumber')).toBe('42');
});
it('resolves single identifiers, as used by jsepEval', () => {
expect(get(context, 'steps')).toBe(context.steps);
});
it('resolves array indices in both bracket and dot form', () => {
expect(get(context, 'eas.job.list[0]')).toBe('first');
expect(get(context, 'eas.job.list[1].name')).toBe('second');
expect(get(context, 'eas.job.list.1.name')).toBe('second');
});
it('resolves computed member paths emitted by jsepEval, e.g. a[b]', () => {
expect(get(context, 'eas[job]')).toBe(context.eas.job);
expect(get({ a: { b: 1 } }, 'a[b]')).toBe(1);
});
it('returns undefined for missing paths instead of throwing', () => {
expect(get(context, 'eas.job.doesNotExist')).toBe(undefined);
expect(get(context, 'eas.job.doesNotExist.nested')).toBe(undefined);
expect(get(context, 'doesNotExist.at.all')).toBe(undefined);
});
it('returns undefined when traversing through null or undefined', () => {
expect(get(context, 'eas.job.secrets.robotAccessToken')).toBe(undefined);
expect(get(context, 'eas.runtimeVersion.anything')).toBe(undefined);
expect(get(null, 'a.b')).toBe(undefined);
expect(get(undefined, 'a')).toBe(undefined);
});
it('returns null values as-is rather than undefined', () => {
expect(get(context, 'eas.job.secrets')).toBe(null);
});
it('prefers a verbatim own property over a nested path, like lodash.get', () => {
expect(get({ 'a.b': 'flat', a: { b: 'nested' } }, 'a.b')).toBe('flat');
expect(get({ a: { b: 'nested' } }, 'a.b')).toBe('nested');
});
it('reads properties of primitives encountered mid-path', () => {
expect(get({ a: 'text' }, 'a.length')).toBe(4);
});
});
+60
View File
@@ -0,0 +1,60 @@
// Minimal replacement for lodash.get, covering the path shapes this package
// produces: dot-separated keys (`steps.step1.outputs.out`), bracketed array
// indices (`list[0].name`), and bracketed keys with or without quotes
// (`a[b]`, `a["b.c"]`, `a['b']`) as emitted by jsepEval's getParameterPath.
const QUOTE_CHARACTERS = ["'", '"'];
export function parsePath(path: string): string[] {
const keys: string[] = [];
let index = 0;
while (index < path.length) {
const character = path[index];
if (character === '.') {
index += 1;
} else if (character === '[') {
const maybeQuote = path[index + 1];
if (QUOTE_CHARACTERS.includes(maybeQuote)) {
const closingQuoteIndex = path.indexOf(`${maybeQuote}]`, index + 2);
if (closingQuoteIndex === -1) {
throw new Error(`Invalid path: unterminated quoted bracket in "${path}"`);
}
keys.push(path.slice(index + 2, closingQuoteIndex));
index = closingQuoteIndex + 2;
} else {
const closingBracketIndex = path.indexOf(']', index + 1);
if (closingBracketIndex === -1) {
throw new Error(`Invalid path: unterminated bracket in "${path}"`);
}
keys.push(path.slice(index + 1, closingBracketIndex));
index = closingBracketIndex + 1;
}
} else {
let end = index;
while (end < path.length && path[end] !== '.' && path[end] !== '[') {
end += 1;
}
keys.push(path.slice(index, end));
index = end;
}
}
return keys;
}
export function get(object: unknown, path: string): any {
// Like lodash.get, a path that exists verbatim as an own property wins
// over interpreting it as a nested path.
if (object != null && typeof object === 'object' && Object.hasOwn(object, path)) {
return (object as Record<string, unknown>)[path];
}
let current: any = object;
for (const key of parsePath(path)) {
if (current == null) {
return undefined;
}
current = current[key];
}
return current;
}
export default get;
+2 -1
View File
@@ -6,7 +6,8 @@
import assert from 'assert';
import jsep from 'jsep';
import get from 'lodash.get';
import { get } from './get';
const binaryOperatorFunctions = {
'===': (a: any, b: any) => a === b,
Loaded 100 of 108 files, more files were not shown because too many files have changed in this diff. Show more