Compare commits

..
Author SHA1 Message Date
Expo CI 62d46fece5 v21.0.3 2026-07-22 12:31:49 +00:00
Ash Wu d66dc2920f [eas-build-job] add update hook key (#4056)
# Why

EAS Workflows' `update` job is getting `before_update` / `after_update` hooks (ENG-25364). First concrete ask: uploading PostHog sourcemaps right after `eas update` publishes — sourcemaps only exist inside the job, and we don't want a vendor-specific step for every provider.

# How

`update` joins `HOOK_ANCHORS`. It is a stamp-only anchor: the server stamps `__hook_id: 'update'` on the update job's generated publish step (expo/universe#29311); no eas-cli function declares it.

# Test Plan

CI passes.
2026-07-22 19:39:55 +08:00
Ash Wu d725a6c7f4 [build-tools] fix maestro screenshot harvest for v2.7.0 (#4051)
Signed-off-by: Ash Wu <[email protected]>
2026-07-22 16:54:58 +08:00
Ash Wu f93b75b9a6 [build-tools] run workflow hooks around managed native build phases (#4044)
Signed-off-by: Ash Wu <[email protected]>
2026-07-21 20:22:00 +08:00
Jon Samp a323655f7c [eas-cli] Add env:set and deprecate env:create and env:update (#4041)
* [eas-cli] Add `eas env:set`, deprecate `env:create` and `env:update`

Introduce `eas env:set`, which creates or updates an environment
variable in a single command (upsert). `env:create` and `env:update`
now warn that they are deprecated in favor of `env:set` while keeping
their existing behavior for backwards compatibility.

* [eas-cli] Add `--json` to `eas env:set`, preserve file type on update

Look up the existing variable before resolving its value so updating a file
variable no longer requires re-passing `--type file`, mirroring the behavior
that `eas env:update` had. Also fix the `--value` flag description typo.

* [eas-cli] Preserve env memberships and hide deprecated commands

* [eas-cli] Remove redundant env:set comment

* [eas-cli] Align env:set flag names

* [eas-cli] Keep short env:set flags canonical

* [eas-cli] Preserve visibility in non-interactive env:set
2026-07-20 16:39:06 -05:00
Jon Samp 4bb500ae01 [eas-cli] Add eas chat command (#4005)
* [eas-cli] Add hidden `eas chat` command

Ask the Expo dashboard AI assistant about your account and EAS projects from the terminal. Streams the reply, supports interactive multi-turn follow-ups, `--project`/`--account` scoping, `--non-interactive`, and `--json` for agent-friendly structured output.

Calls the website's `/api/chat` route using the CLI session secret; command is hidden while the backend feature gate rolls out.

* [eas-cli] Update changelog PR link for `eas chat`

* [eas-cli] Render markdown and steer output for `eas chat`

Render the streamed assistant reply as ANSI (bold, inline code, headings, bullets, links) one line at a time. Prepend a client-side system message steering the assistant to write terminal-friendly answers and include expo.dev links for referenced artifacts (builds, updates, etc.).

* [eas-cli] Add readline REPL with history and slash commands to `eas chat`

Replace the per-turn prompt with a reusable Node readline interface: arrow-key history across turns and /help, /clear, /exit slash commands, all in the normal scrolling buffer (no new dependency).

* [eas-cli] Fix `eas chat` interactive prompt exiting immediately

The ora spinner leaves stdin paused when it stops, so the readline prompt had nothing keeping the event loop alive and the process exited the moment the prompt appeared. Resume stdin before each prompt and stop the spinner from grabbing stdin (discardStdin: false).

* [eas-cli] Auto-scope `eas chat` to the current directory's project

When run inside a linked EAS project and neither --project nor --account is given, chat now focuses on that project automatically. Detection is silent and read-only: it never prompts, links, or creates a config, and falls back to account scope when the directory is not a linked project.

* [eas-cli] Seed `eas chat` history with the initial command-line message

The first message comes from argv, not the readline prompt, so Up-arrow had nothing to recall at the first follow-up. Pass it as readline's initial history.

* [eas-cli] Label and align `eas chat` speakers (Chat > / Expo >)

Prefix the assistant reply with a magenta "Expo > " label and indent its continuation lines to match, and label the user's turn "Chat > " (both 4-char names so the two align).

* [eas-cli] Wrap `eas chat` reply to terminal width and indent the spinner

Wrap each rendered line to the terminal width minus the indent (ANSI-aware, hard-breaking long URLs) and re-apply the indent to every wrapped piece, so terminal soft-wrapping no longer drops wrapped text to column 0. Also indent the ora spinner to align with the reply body.

* [eas-cli] Fix `eas chat` reply indent after spinner and reorder intro

The indented ora spinner leaves the cursor at its indent column when it stops (ora's clear() ends with cursorTo(indent)), which pushed the whole reply right by the indent. Reset the cursor to column 0 after stopping. Also move the commands hint above the echoed message and shorten it.

* [eas-cli] Render `eas chat` spinner with the Expo label prefix

Use ora's prefixText to show "Expo > <spinner> Thinking…" from the first frame instead of the indent option, which rendered unindented for a tick then snapped right. The spinner now reads as the reply and is cleared/replaced by the streamed text.

* [eas-cli] Decode `eas chat` stream with StringDecoder to avoid splitting multi-byte characters

* [eas-cli] Remove CHANGELOG entry for hidden `eas chat` command

* [eas-cli] Preserve chat context across follow-ups

* [eas-cli] Let `eas chat` open an interactive prompt with no message argument

* [eas-cli] Remove comments from `eas chat`

* [eas-cli] Keep `eas chat` interactive session alive when a request fails

In interactive mode, catch errors from the chat stream, print the message, and re-prompt instead of exiting the whole session. Single-shot and --json invocations still throw.

* [eas-cli] Keep `eas chat` spinner visible until the reply is ready to render

The spinner was cleared on the first text delta, but the line-buffered renderer only draws once a full line is buffered, leaving a blank gap. Stop the spinner lazily when the first visible line is written so the labeled loading indicator stays put until output appears.

* [eas-cli] Spell out abbreviated variable names in `eas chat`

rl -> readlineInterface, exp -> expoConfig, the *_RE markdown constants -> *_REGEX, and the repl test var -> replInput.

* [eas-cli] Simplify `eas chat` tool tracking and markdown renderer

Derive the toolCalls result from the assistant message parts instead of maintaining a second parallel map, and drop markdown constructs (headings, blockquotes, horizontal rules, italics) the system prompt tells the model not to emit.

* [eas-cli] Sanitize streamed chat output
2026-07-20 14:38:22 -05:00
Szymon Dziedzic a207e5d62e [build-tools] Generalize remote log uploads (#4027)
* [build-tools] Generalize remote log uploads

* [build-tools] Remove remote logger size policy

* [build-tools] Preserve remote logger behavior
2026-07-20 13:02:33 +02:00
Douglas Lowder f81efbacde [eas-cli] Support drill-down from event lists in observe:session (#3987)
# Why

When session ID is omitted and the `observe:session` command is running interactively, walk the user through picking an event whose session they want to inspect. Non-interactive without a session ID still errors as before.

The goal is to support a flow similar to that already available on our website, where a user can select an event from a displayed list, and navigate to the session timeline that contains that event.

# How

Add new flags `--event-name` and `--sort-order`. If these are not provided, user will be prompted first to select from a list of event names (including custom events, if applicable), then select a sort order. The command also now takes the same time period flags as the other commands (`--days`, `--start`, `--end`).

Once the user has made the above selections, the user can choose from the list of events shown (up to 25), and the session for that event will be shown.

# Test Plan

- Tested manually with an SDK 57 project
- New unit tests added
2026-07-17 05:39:37 -07:00
Expo CI 06c6a8416d update CHANGELOG.md 2026-07-17 06:03:55 +00:00
Expo CI e9ebc41413 v21.0.2 2026-07-17 06:00:55 +00:00
9313ef11ca [eas-cli] Show accurate build overage count in pre-build usage warning (#3932)
* [eas-cli] Show accurate build overage count in pre-build usage warning

The overage count shown to paid users ("N builds beyond your included
credits") summed the value of every overageMetrics row, including rows
without build metadata (e.g. rollup/summary rows), which could inflate the
number. Filter to genuine per-worker build overage rows (those carrying
billingResourceClass + platform metadata) before summing.

Fetch that metadata on overageMetrics in the warning query so the rows can
be distinguished.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* [eas-cli] Fix changelog PR reference for build overage warning entry

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

---------

Co-authored-by: Sarah Lane <[email protected]>
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
2026-07-16 16:43:31 -06:00
Stanisław Chmiela fd80532a7e [build-tools] Remove serve-sim stream URL output (#4018)
I found it sometimes isn't printed.

---

## Summary

- Stop waiting for `serve-sim` to emit a `Stream:` URL before reporting startup success.
- Upload only `previewUrl` in the serve-sim remote config payload.
- Remove `streamUrl` from the simulator session query selection and generated query result type.
- Simplify the serve-sim URL parser to match only the tunnel URL.

## Why

The stream URL is no longer part of the expected serve-sim output path. Keeping it in the startup contract made the build step wait for and upload a value that is not needed by the CLI display flow.

## Test plan

- `yarn --cwd packages/build-tools typecheck`
- `yarn --cwd packages/eas-cli typecheck`
- `yarn --cwd packages/build-tools jest-unit src/steps/utils/__tests__/remoteDeviceRunSession.test.ts`
- `yarn --cwd packages/eas-cli test src/commands/simulator/__tests__/get.test.ts src/commands/simulator/__tests__/start.test.ts --runInBand`
- `git diff --check`
2026-07-16 15:03:37 +02:00
Stanisław Chmiela f20ccebcad [build-tools] Clean up GCS logging and retry handling (#4026)
Remove unused functions from `GCS.Client`. Removes big `@google-cloud/storage` dependency!
2026-07-16 14:55:20 +02:00
Stanisław Chmiela e8602f7062 [build-tools] Use unique simulator recording artifact names (#4020)
## Summary

- include a shortened simulator UDID and first-frame timestamp in each screen recording artifact name
- format the timestamp as a readable, stable UTC value
- add `__eas_type: "screen-recording"` metadata for typed artifact detection
- cover recordings from a rebooted simulator and simultaneous same-model simulators

Artifact names now use this format:

```
iPhone 16 screen recording (01234567-…, started at Jul 10, 2026, 10:00:00.000 UTC)
```

## Why

A simulator can reboot during one device run session, producing multiple recordings with the same device name and UDID. A session can also contain two same-model simulators whose recordings start at precisely the same time. Device name alone, or device name plus timestamp alone, therefore does not reliably distinguish every recording.

The first eight UDID characters distinguish same-model simulators without putting the full UUID in the user-facing name. The millisecond first-frame timestamp distinguishes subsequent recordings after a reboot. The full UDID and canonical ISO timestamp remain in artifact metadata for exact identity, parsing, and ordering.

The new `__eas_type` marker provides a descriptive discriminator for screen recordings. The existing `__eas_screen_recording: "1"` marker remains temporarily so currently deployed consumers continue recognizing these artifacts during migration.

The artifact metadata also provides `udid`, `deviceName`, `runtimeDisplayName`, `firstFrameAt`, and encoded `width` and `height`.

## Validation

- focused screen recording upload tests covering rebooted and simultaneous same-model simulators
- build-tools TypeScript typecheck
2026-07-16 14:09:51 +02:00
Ash Wu d14c0b196b [eas-build-job] internal refactor general hook (#4023)
Signed-off-by: Ash Wu <[email protected]>
2026-07-16 19:59:08 +08:00
Stanisław Chmiela c24b9a037a [codemention] Limit default notifications to eas-cli (#4024) 2026-07-16 03:43:53 -07:00
Stanisław Chmiela c3c3f087ac [worker] Increase Node heap on capable workers (#4011)
## Summary

- set `NODE_OPTIONS=--max-old-space-size=3072` for build child processes when the worker has at least 6 GiB of constrained or physical memory and its effective V8 heap limit is below 3 GiB
- preserve existing worker `NODE_OPTIONS`; project-provided environment values retain their later override position
- leave workers with less memory or an existing heap limit of at least 3 GiB unchanged

## Why

V8 defaults to a roughly 2 GiB heap on our smaller build workers. Dependency installation and Expo export can exhaust that heap even though the VM still has enough memory. Raising the ceiling to 3 GiB gives those child processes additional headroom without preallocating memory or changing larger workers that already receive a higher V8 default.

## Validation

- `yarn build`
- `yarn workspace @expo/worker test:unit --runInBand src/__unit__/env.test.ts`
- `yarn workspace @expo/worker typecheck`
- focused Oxlint and Oxfmt checks
2026-07-16 12:25:44 +02:00
Stanisław Chmiela ac761edb5c Upgrade Lodash and GraphQL Code Generator (#4009)
# Why

Supersedes #3968.

Dependabot alert 354 requires `lodash` and `lodash-es` 4.18.1. The original PR upgraded the worker's direct dependency and enforced the patched transitive version with a root resolution. That resolution is no longer needed: this PR upgrades the EAS-owned declarations and the GraphQL Code Generator packages that brought in the remaining restrictive `lodash ~4.17.0` dependency.

# How

- Upgrade direct Lodash declarations in build-tools, local-build-plugin, template-file, and worker to 4.18.1.
- Upgrade the scripts project's `lodash-es` declaration and its independent lockfile to 4.18.1.
- Upgrade the complete GraphQL Code Generator family used by EAS CLI:
  - `@graphql-codegen/cli` 5.0.0 → 6.3.1
  - `@graphql-codegen/introspection` 4.0.0 → 5.0.2
  - `@graphql-codegen/typescript` 4.0.1 → 5.0.10
  - `@graphql-codegen/typescript-operations` 4.0.1 → 5.1.0
- Regenerate the checked-in GraphQL TypeScript output.
- Keep the root free of `resolutions` and regenerate both affected lockfiles.

These are the newest aligned Codegen releases before `typescript-operations` v6. That v6 upgrade changes generated operation type semantics (including nullable response-field presence) and causes broad fixture/type failures across EAS CLI, so it should be handled as a separate GraphQL contract migration rather than bundled into this security dependency update. The selected packages use `@graphql-codegen/plugin-helpers` 6.3.x, which no longer depends on Lodash.

The root dependency graph now reports only `[email protected].1`, selected through ordinary declared dependency ranges. The separate scripts project reports only `[email protected].1`. There is no resolution or override.

# Test Plan

- `yarn install`
- `yarn why lodash` — only `[email protected].1`; no Codegen consumer
- `cd scripts && yarn install && yarn why lodash-es` — only `[email protected].1`
- `yarn build` — all 13 projects passed
- `yarn workspace eas-cli verify-graphql-code` — generated output compiles against the current live schema
- `cd scripts && yarn typecheck`
- `yarn lint-changelog`
- template-file tests — 7 passed
- worker tests — 51 passed
- `yarn fmt`
2026-07-15 22:32:56 +02:00
Stanisław Chmiela 78c30acbc9 [build-tools] Improve simulator recording finalization diagnostics (#4019)
## Summary

- increase the graceful `record-sim` shutdown timeout from 30 to 70 seconds
- record readable finalization milestones after a termination signal
- include the recorder's recent stdout/stderr when build-tools must force-stop it

## Why

A production device run session finalized two simulator recordings concurrently. One recording completed normally, while the other was force-killed after exactly 30 seconds and was therefore unable to write its manifest or upload its MP4.

The parent timeout was shorter than the recorder's own valid finalization window: `record-sim` can spend up to 5 seconds appending its final frame and up to 60 seconds waiting for `AVAssetWriter.finishWriting()`. Killing it after 30 seconds could therefore interrupt legitimate finalization.

The existing logs also could not distinguish between a process that failed to observe `SIGINT` and one that observed it but stalled later. The new milestones identify whether the recorder:

1. observed the termination signal,
2. stopped screen capture and began saving the video,
3. saved the video and began writing recording details.

These messages stay buffered during successful recordings. They are added to the user-visible forced-stop warning only when finalization exceeds 70 seconds, keeping normal job logs concise while making failures diagnosable.

## Validation

- built the release `record-sim` executable
- ran the build-tools TypeScript typecheck
- ran all 88 build-tools unit test suites (831 tests)
2026-07-15 22:02:17 +02:00
Szymon Dziedzic a716925ed7 [codex] fix tmp dependabot alert (#3965)
* fix tmp dependabot alert

* add changelog for tmp security fix
2026-07-15 16:25:19 +02:00
Szymon Dziedzic e579441d5b [eas-cli] clean up build inspect temp directory (#3981)
* Clean up build inspect temp directory

* [eas-cli] Add build inspect cleanup changelog

* [eas-cli] Fix build inspect test types
2026-07-15 16:09:27 +02:00
Szymon Dziedzic 4f5831305b [codex] fix ws dependabot alert (#3963)
* fix ws dependabot alert

* add changelog for ws security fix

* [worker] Update ws to latest hardened patch releases
2026-07-15 15:53:01 +02:00
Szymon Dziedzic 98db228cc8 [codex] fix sigstore dependabot alert (#3961)
* fix sigstore dependabot alert

* add changelog for sigstore security fix
2026-07-15 14:12:47 +02:00
Szymon Dziedzic 5c762d300e [codex] fix handlebars dependabot alert (#3958)
* fix handlebars dependabot alert

* add changelog for handlebars security fix
2026-07-15 13:36:18 +02:00
Stanisław Chmiela 2a6bee7f5a [build-tools] Remove updateEnv merge-order Datadog log (#3935)
## Summary
- remove the temporary Datadog log that compared old and new updateEnv merge-order results
- keep the new updateEnv merge order behavior unchanged
- remove the log-specific test assertions

## Test plan
- yarn fmt packages/build-tools/src/context.ts packages/build-tools/src/__tests__/context.test.ts
- yarn --cwd packages/build-tools jest-unit src/__tests__/context.test.ts
- yarn --cwd packages/build-tools typecheck

This is intended to be merged after we confirm the migration looks good in Datadog.
2026-07-15 13:30:54 +02:00
Stanisław Chmiela 4d365e0355 [build-tools][eas-cli] Upgrade xmldom through parent dependencies (#4010)
# Why

Supersedes #3967.

Dependabot alert 372 requires `@xmldom/xmldom` 0.8.13. The original PR used a root resolution to force 0.8.13 through parents that declared the incompatible `~0.7.7` range. This replacement upgrades those parent dependencies instead, so xmldom can be updated without overriding any package's declared contract.

# How

- Upgrade EAS CLI's direct `@expo/plist` dependency from 0.2.0 to 0.3.5.
- Upgrade build-tools' `@expo/plist` range from `^0.2.0` to `^0.3.5`.
- Upgrade EAS CLI's `@expo/prebuild-config` from 8.0.17 to 9.0.12, moving its config-plugin chain to a version that uses the newer plist package.
- Upgrade the transitive lockfile entry to `@xmldom/xmldom` 0.8.13.

After these changes, every installed plist consumer declares `@xmldom/xmldom ^0.8.8`, and `yarn why @xmldom/xmldom` reports a single natural 0.8.13 installation. No root resolution is added.

The relevant `getPrebuildConfigAsync` iOS path used by EAS CLI for entitlement inspection is unchanged across the prebuild-config upgrade; the direct implementation change is confined to passing `projectRoot` to the Android plugin path.

# Test Plan

- `yarn install`
- `yarn why @xmldom/xmldom`
- `yarn build` — all 13 projects passed with xmldom 0.8.13 installed
- Four build-tools plist/configuration suites — 19 tests and 14 snapshots passed
- EAS CLI `UpdatesModule` iOS suite — 2 tests passed
- `yarn fmt`
- `yarn fmt:check`
- `yarn lint` — 0 errors; 13 pre-existing warnings also present on `main`
2026-07-15 13:30:32 +02:00
Stanisław Chmiela 48aa2ccde2 [eas-cli] Remove Express from image URL test (#4008)
## Why

The image URL test only needs to serve a single PNG over HTTP. Using Express for that adds an otherwise-unused framework and its dependency tree to EAS CLI test dependencies.

This supersedes #3969. That PR upgrades Express to address the vulnerable transitive `path-to-regexp` version, but after this change EAS CLI no longer depends on Express, so the affected dependency path is removed instead of upgraded.

## What changed

- replace the Express static server with Node's built-in `http` server
- bind to port `0` so the operating system selects an available port
- remove `express` and `@types/express` from `packages/eas-cli`
- regenerate the lockfile, removing the associated Express 4 dependency tree

## Impact

This only changes test infrastructure. The test continues to exercise the real remote-URL fetch and PNG transparency validation path. There is no EAS CLI runtime behavior change.

## Validation

- `yarn workspace eas-cli test src/utils/__tests__/image-test.ts --runInBand`
- `yarn workspace eas-cli typecheck`
- formatting check
- `git diff --check`
2026-07-15 13:28:51 +02:00
Stanisław Chmiela bdc04e3e6b [build-tools] Add width and height to screen recording metadata (#4017) 2026-07-15 13:09:00 +02:00
Expo Bot 935491f967 [license] Update BSL Change Date to 2029-07-15 2026-07-15 09:31:33 +00:00
Stanisław Chmiela 031198d1aa Restore automated BSL change date updates (#4016)
## Summary

- restore the quarterly Business Source License change-date workflow from `expo/eas-build`
- update `LICENSE-BUSL` instead of the former repository's root `LICENSE`
- run once when the workflow lands on `main`, then every three months

## Why

The EAS Build packages moved into this repository, but their scheduled BSL change-date automation was not moved with them. As a result, `LICENSE-BUSL` still has the last date written by the archived repository.

After this workflow lands on `main`, its path-based push trigger will advance the stale change date and future scheduled runs will keep it three years ahead.

The workflow deliberately does not run on pull requests because it uses the write-capable `EXPO_BOT_PAT`. Existing production evidence confirms that token can update the protected default branch: [`Release workflow` run 29394979387](https://github.com/expo/eas-cli/actions/runs/29394979387) successfully pushed `main -> main` on July 15, 2026 using the same repository secret.

## Validation

- `actionlint .github/workflows/license.yml`
- YAML parse check
- embedded Bash syntax check
- `git diff --check`
2026-07-15 09:31:24 +00:00
Szymon Dziedzic ac60d159f2 [codex] fix axios dependabot alert (#3964)
* fix axios dependabot alert

* add changelog for axios security fix

* align axios version with nx
2026-07-15 11:20:28 +02:00
Expo CI fc09fe4a34 update CHANGELOG.md 2026-07-15 06:45:02 +00:00
Expo CI a62289b988 v21.0.1 2026-07-15 06:42:00 +00:00
Brent Vatne 78c0584331 [eas-cli] Add -p alias for --platform in simulator:start (#3902) 2026-07-14 15:46:37 -07:00
Brent Vatne c4a7b7c9f1 [eas-cli] Add eas project:delete command (#4012) 2026-07-14 09:52:30 -07:00
Ash Wu f448bc75b5 [build-tools] unify workflows hook (#3957)
* [eas-build-job] Add hook anchors registry and step stamp fields

Signed-off-by: Ash Wu <[email protected]>

* [steps] Add hook insertion, after-hook gate, and source function retention

Signed-off-by: Ash Wu <[email protected]>

* [build-tools] Pass job hooks into the steps parser

Signed-off-by: Ash Wu <[email protected]>

* address comments

Signed-off-by: Ash Wu <[email protected]>

---------

Signed-off-by: Ash Wu <[email protected]>
2026-07-14 15:31:11 +08:00
Brent VatneandDouglas Lowder 0dda7ba880 [eas-cli] Add --dev-domain flag to eas deploy (#4004)
Co-authored-by: Douglas Lowder <[email protected]>
2026-07-13 20:26:02 -07:00
Jon SampandDouglas Lowder 9679af9670 [eas-cli] Add eas browse command (#4003)
* [eas-cli] Add `eas browse` command

Add `eas browse [PAGE]` to open the project page on expo.dev, with an
optional whitelisted page argument (e.g. `build`, `hosting`, `cicd`) for
project subpages and a `--no-browser` flag to print the URL instead.
`eas open` becomes a thin alias that opens the project dashboard.

* [eas-cli] Update changelog PR link

* [eas-cli] Add `observe` page to `eas browse`

* [eas-cli] Support --json and --non-interactive in eas browse, hide eas open

* Fix CHANGELOG

---------

Co-authored-by: Douglas Lowder <[email protected]>
2026-07-13 17:44:19 -05:00
Szymon Dziedzic 3e0e823e56 Bump tar (#3971)
* Bump tar

* Add changelog for tar bump
2026-07-13 21:28:20 +02:00
Expo CI fbb8af6fca update CHANGELOG.md 2026-07-13 17:25:08 +00:00
Expo CI c72cee372f v21.0.0 2026-07-13 17:21:41 +00:00
Szymon Dziedzic 73b2a716da [eas-cli] Bump node-forge (#3970)
* [eas-cli] Bump node-forge

* [eas-cli] Add changelog for node-forge bump
2026-07-13 18:54:28 +02:00
Gabe Debes 55191f9697 [build-tools] Add eas/posthog_wait_for_query workflow function (#3949)
* [build-tools] Add eas/posthog_capture_event workflow function

* [build-tools] address PostHog client review nits

* [build-tools] report which PostHog credentials are missing

* [build-tools] add poll helpers, credentials-error factory, query abort signal

* [build-tools] move credential catalog and error into PosthogUtils

* [build-tools] accept an abort signal on all PostHog client requests

* [build-tools] Add eas/posthog_flag_rollout workflow function

* [build-tools] use PosthogUtils namespace in flag rollout

* [build-tools] name missing credentials in flag rollout

* [build-tools] use credentials-error factory in flag rollout

* [build-tools] use PosthogUtils.missingCredentialsError in flag rollout

* [build-tools] thread abort signal through flag rollout requests

* [build-tools] Add eas/posthog_wait_for_metric workflow function

* [build-tools] respect abort signal while waiting for metric

* [build-tools] name missing credentials when waiting for metric

* [build-tools] share poll helpers, harden operator guard, thread signal (metric)

* [build-tools] use PosthogUtils.missingCredentialsError when waiting for metric

* [build-tools] Add eas/posthog_upload_sourcemaps workflow function

* [build-tools] use PosthogUtils namespace in sourcemap upload

* [build-tools] name missing credentials in sourcemap upload

* [build-tools] use credentials-error factory in sourcemap upload

* [build-tools] use PosthogUtils.missingCredentialsError in sourcemap upload

* [build-tools] pass abort signal to the sourcemap upload subprocess

* [build-tools] Add eas/posthog_annotation workflow function

* [build-tools] use PosthogUtils namespace in annotation

* [build-tools] name missing credentials in annotation

* [build-tools] use credentials-error factory in annotation

* [build-tools] use PosthogUtils.missingCredentialsError in annotation

* [build-tools] thread abort signal through annotation request

* [build-tools] Add eas/posthog_wait_for_query workflow function

* [build-tools] respect abort signal while waiting for query

* [build-tools] name missing credentials when waiting for query

* [build-tools] share poll helpers and thread signal (query)

* [build-tools] use PosthogUtils.missingCredentialsError when waiting for query
2026-07-13 09:32:03 -07:00
Szymon Dziedzic a08beb6392 [codex] fix form-data dependabot alert (#3962)
* fix form-data dependabot alert

* add changelog for form-data security fix
2026-07-13 18:29:07 +02:00
Szymon Dziedzic 643cc9e387 [codex] fix shell-quote dependabot alert (#3959)
* fix shell-quote dependabot alert

* add changelog for shell-quote security fix
2026-07-13 18:06:12 +02:00
Gabe Debes 7c44d5515a [build-tools] Add eas/posthog_annotation workflow function (#3948)
* [build-tools] Add eas/posthog_capture_event workflow function

* [build-tools] address PostHog client review nits

* [build-tools] report which PostHog credentials are missing

* [build-tools] add poll helpers, credentials-error factory, query abort signal

* [build-tools] move credential catalog and error into PosthogUtils

* [build-tools] accept an abort signal on all PostHog client requests

* [build-tools] Add eas/posthog_flag_rollout workflow function

* [build-tools] use PosthogUtils namespace in flag rollout

* [build-tools] name missing credentials in flag rollout

* [build-tools] use credentials-error factory in flag rollout

* [build-tools] use PosthogUtils.missingCredentialsError in flag rollout

* [build-tools] thread abort signal through flag rollout requests

* [build-tools] Add eas/posthog_wait_for_metric workflow function

* [build-tools] respect abort signal while waiting for metric

* [build-tools] name missing credentials when waiting for metric

* [build-tools] share poll helpers, harden operator guard, thread signal (metric)

* [build-tools] use PosthogUtils.missingCredentialsError when waiting for metric

* [build-tools] Add eas/posthog_upload_sourcemaps workflow function

* [build-tools] use PosthogUtils namespace in sourcemap upload

* [build-tools] name missing credentials in sourcemap upload

* [build-tools] use credentials-error factory in sourcemap upload

* [build-tools] use PosthogUtils.missingCredentialsError in sourcemap upload

* [build-tools] pass abort signal to the sourcemap upload subprocess

* [build-tools] Add eas/posthog_annotation workflow function

* [build-tools] use PosthogUtils namespace in annotation

* [build-tools] name missing credentials in annotation

* [build-tools] use credentials-error factory in annotation

* [build-tools] use PosthogUtils.missingCredentialsError in annotation

* [build-tools] thread abort signal through annotation request
2026-07-13 09:00:07 -07:00
Gabe Debes 6fd172ee30 [build-tools] Add eas/posthog_upload_sourcemaps workflow function (#3947)
* [build-tools] Add eas/posthog_capture_event workflow function

* [build-tools] address PostHog client review nits

* [build-tools] report which PostHog credentials are missing

* [build-tools] add poll helpers, credentials-error factory, query abort signal

* [build-tools] move credential catalog and error into PosthogUtils

* [build-tools] accept an abort signal on all PostHog client requests

* [build-tools] Add eas/posthog_flag_rollout workflow function

* [build-tools] use PosthogUtils namespace in flag rollout

* [build-tools] name missing credentials in flag rollout

* [build-tools] use credentials-error factory in flag rollout

* [build-tools] use PosthogUtils.missingCredentialsError in flag rollout

* [build-tools] thread abort signal through flag rollout requests

* [build-tools] Add eas/posthog_wait_for_metric workflow function

* [build-tools] respect abort signal while waiting for metric

* [build-tools] name missing credentials when waiting for metric

* [build-tools] share poll helpers, harden operator guard, thread signal (metric)

* [build-tools] use PosthogUtils.missingCredentialsError when waiting for metric

* [build-tools] Add eas/posthog_upload_sourcemaps workflow function

* [build-tools] use PosthogUtils namespace in sourcemap upload

* [build-tools] name missing credentials in sourcemap upload

* [build-tools] use credentials-error factory in sourcemap upload

* [build-tools] use PosthogUtils.missingCredentialsError in sourcemap upload

* [build-tools] pass abort signal to the sourcemap upload subprocess
2026-07-13 08:53:19 -07:00
Gabe Debes 4edf0dc547 [build-tools] Add eas/posthog_wait_for_metric workflow function (#3945)
* [build-tools] Add eas/posthog_capture_event workflow function

* [build-tools] address PostHog client review nits

* [build-tools] report which PostHog credentials are missing

* [build-tools] add poll helpers, credentials-error factory, query abort signal

* [build-tools] move credential catalog and error into PosthogUtils

* [build-tools] accept an abort signal on all PostHog client requests

* [build-tools] Add eas/posthog_flag_rollout workflow function

* [build-tools] use PosthogUtils namespace in flag rollout

* [build-tools] name missing credentials in flag rollout

* [build-tools] use credentials-error factory in flag rollout

* [build-tools] use PosthogUtils.missingCredentialsError in flag rollout

* [build-tools] thread abort signal through flag rollout requests

* [build-tools] Add eas/posthog_wait_for_metric workflow function

* [build-tools] respect abort signal while waiting for metric

* [build-tools] name missing credentials when waiting for metric

* [build-tools] share poll helpers, harden operator guard, thread signal (metric)

* [build-tools] use PosthogUtils.missingCredentialsError when waiting for metric
2026-07-13 08:46:03 -07:00
Gabe Debes f82bdb57e6 [build-tools] Add eas/posthog_flag_rollout workflow function (#3944)
* [build-tools] Add eas/posthog_capture_event workflow function

* [build-tools] address PostHog client review nits

* [build-tools] report which PostHog credentials are missing

* [build-tools] add poll helpers, credentials-error factory, query abort signal

* [build-tools] move credential catalog and error into PosthogUtils

* [build-tools] accept an abort signal on all PostHog client requests

* [build-tools] Add eas/posthog_flag_rollout workflow function

* [build-tools] use PosthogUtils namespace in flag rollout

* [build-tools] name missing credentials in flag rollout

* [build-tools] use credentials-error factory in flag rollout

* [build-tools] use PosthogUtils.missingCredentialsError in flag rollout

* [build-tools] thread abort signal through flag rollout requests
2026-07-13 08:39:43 -07:00
Szymon Dziedzic 9641735781 [codex] fix fast-xml-parser dependabot alert (#3960)
* fix fast-xml-parser dependabot alert

* add changelog for fast-xml-parser security fix
2026-07-13 17:33:25 +02:00
Gabe Debes 1ce05e3b28 [build-tools] Add eas/posthog_capture_event workflow function (#3934)
* [build-tools] Add eas/posthog_capture_event workflow function

* [build-tools] address PostHog client review nits

* [build-tools] report which PostHog credentials are missing

* [build-tools] add poll helpers, credentials-error factory, query abort signal

* [build-tools] move credential catalog and error into PosthogUtils

* [build-tools] accept an abort signal on all PostHog client requests
2026-07-13 08:33:06 -07:00
Szymon Dziedzic b2f34e5c03 [codex] fix fast-xml-builder dependabot alert (#3966)
* fix fast-xml-builder dependabot alert

* add changelog for fast-xml-builder security fix
2026-07-13 17:04:09 +02:00
Stanisław Chmiela 8204d18951 [build-tools] Add simulator details to screen recording metadata (#4000)
## Summary

- include the simulator device name and human-readable runtime name in Device Run Session screen recording metadata
- carry those fields through the finish-step output and construct the artifact display name only during upload
- normalize CoreSimulator runtime identifiers such as `iOS-18-6` to `iOS 18.6`
- add focused coverage for runtime normalization and the uploaded metadata payload

## Why

The website needs device and runtime details to identify and present EAS-generated simulator screen recordings without parsing artifact names or CoreSimulator identifiers.

## Validation

- `yarn workspace @expo/build-tools typecheck`
- `yarn workspace @expo/build-tools jest-unit --runInBand src/utils/__tests__/IosSimulatorUtils.test.ts src/steps/functions/__tests__/uploadDeviceRunSessionScreenRecordings.test.ts`
2026-07-13 15:22:40 +02:00
Szymon Dziedzic 69024620f9 Bump cross-spawn in steps fixture (#3977)
* Bump cross-spawn in steps fixture

* Add changelog entry for cross-spawn bump
2026-07-13 14:51:04 +02:00
Szymon Dziedzic c9442988c7 Bump Koa dependency (#3974)
* Bump Koa dependency

* Add changelog entry for Koa bump
2026-07-13 13:31:49 +02:00
Szymon Dziedzic 2a258e80a9 Bump jws in root lockfile (#3978)
* Bump jws in root lockfile

* Add changelog entry for jws bump
2026-07-13 13:11:51 +02:00
Stanisław Chmiela 4b1f785abe [build-tools] Handle detached Android emulator exits (#3998)
## Summary

Observe the detached Android emulator process promise immediately after startup so expected exits during retry cleanup do not surface as unhandled promise rejections.

The emulator is managed through adb after launch. When a startup attempt times out and cleanup kills the emulator, the underlying SpawnPromise can reject while the retry loop is handling the readiness failure. Observing the promise keeps Sentry from capturing that cleanup path as an unhandled SIGKILL while preserving the returned promise for callers that still await it later.

## Test plan

- yarn run -T oxfmt packages/build-tools/src/utils/AndroidEmulatorUtils.ts
- yarn workspace @expo/build-tools typecheck
2026-07-10 17:39:39 +02:00
Stanisław Chmiela 2e2e8ffacc [build-tools] Fix Argent tool server state discovery (#3997)
# Why

Argent now writes per-install tool-server state files such as `~/.argent/tool-server-<hash>.json`. The remote session step only watched the legacy `tool-server.json`, so it could time out even when the newly launched tool server was running.

# How

- expose the PID of detached processes
- add a process ancestry helper that works on macOS and Linux
- run Argent in the foreground beneath the detached `bunx` process and pass `--force`
- scan legacy and hashed Argent state files, selecting only the state whose PID descends from the launched `bunx` process
- validate the state schema includes the tool-server PID
- cover legacy, hashed, matching, and unrelated state files

# Test Plan

- `yarn workspace @expo/build-tools jest-unit packages/build-tools/src/steps/functions/__tests__/startArgentRemoteSession.test.ts --runInBand`
- `yarn workspace @expo/build-tools build`
- `yarn workspace @expo/worker build`
- `yarn workspace eas-cli build`
- `yarn workspace @expo/build-tools typecheck`
- `yarn lint`
- `yarn fmt:check`
2026-07-10 15:57:48 +02:00
Stanisław Chmiela 708bddd2ad [build-tools] Add screen recording metadata marker (#3996)
Didn't realize haven't pushed this commit before merging.

https://github.com/expo/eas-cli/pull/3994
2026-07-10 15:29:55 +02:00
Stanisław Chmiela 6d02728a25 [build-tools] Wire record-sim into EAS functions (#3994)
## Summary

- add separate `eas/start_ios_simulator_recordings` and `eas/finish_ios_simulator_recordings` build functions
- poll booted iOS Simulators and record each session with `record-sim`
- retry failed recorder processes up to three times per simulator boot
- upload completed MP4 files as screen-recording device run session artifacts with recording metadata
- extend device run session artifact uploads to accept metadata

## Why

Device run sessions need simulator screen recordings that can start independently from remote sessions and be finalized and uploaded when the build workflow finishes.

## Impact

Workflows can explicitly start simulator recording polling and later finish and upload recordings. Missing `record-sim` binaries or device run session IDs disable recording or upload without failing the workflow.

## Validation

- `yarn workspace @expo/build-tools typecheck`
- `yarn workspace @expo/build-tools jest-unit steps/utils/__tests__/deviceRunSessionArtifacts.test.ts --runInBand`
- `yarn oxfmt --check` on all changed files
- `git diff --check origin/main...HEAD`
2026-07-10 15:22:59 +02:00
Douglas Lowder 9a94201ce5 [eas-cli] Rethrow Apple errors during capability sync instead of silently succeeding (#3989)
# Why

`syncCapabilitiesForEntitlementsAsync` in bundleIdCapabilities.ts wrapped `bundleId.updateBundleIdCapabilityAsync` in a try/catch whose block only rethrew for the specific error "bundle '…' cannot be deleted. Delete all the Apps". Every other Apple API error was silently swallowed while the function still returned the intended enabled/disabled lists — leading to undebuggable signing failures downstream. (Reported in issue #3986 against 18.12.3 and 20.5.1.)

# How

Rethrow every non-matching error with the same actionable context we already gave the "cannot be deleted" case — the Apple developer console URL and the EXPO_NO_CAPABILITY_SYNC=1 escape hatch — and preserve the underlying Apple error via `.cause` so EXPO_DEBUG=1 still surfaces the raw response. Log the failing request payload for all failures, not just the matched one, so future variants are debuggable from CLI output.
    
Also fix the "developer-mdn.apple.com" typo in the existing error message (the correct host is developer.apple.com).

Note that capability sync will not work correctly until the fix in https://github.com/expo/third-party/pull/161 is merged and released.

# Test Plan

- Expand the existing "throw useful error message" test to cover generic Apple errors
- Correct expected URL in test assertion
2026-07-10 05:57:41 -07:00
Szymon Dziedzic 6189ae9ed3 [eas-cli] Expose simulator session artifacts in get (#3991) 2026-07-10 13:27:52 +02:00
Stanisław Chmiela 9d373d9778 [build-tools] Fix iOS Simulator apsd disable for iOS 26.5 (#3988)
## Summary

Fixes the iOS Simulator `apsd` shutdown helper to use the launchd domain requested by iOS 26.5:

- disables and boots out `user/foreground/com.apple.apsd` first
- falls back to `system/com.apple.apsd` for compatibility
- verifies whether the service is still `state = running`
- treats `launchctl print` returning service-not-found as a successful stopped state

## Root cause

The previous implementation used `system/com.apple.apsd`. On iOS 26.5, `launchctl` warns to switch to `user/foreground/com.apple.apsd`, and workflow logs showed `bootout system/com.apple.apsd` exiting non-zero. That made a successful or partially successful disable look like a failure and could leave `apsd` running.

## Validation

- `yarn test packages/build-tools/src/utils/__tests__/IosSimulatorUtils.test.ts`
- `yarn typecheck` in the original checkout
- `git diff --check`

I also verified locally on a disposable iOS 26.5 simulator clone that `disable` + `bootout` with `user/foreground/com.apple.apsd` stops the service, and subsequent `launchctl print` returns service-not-found.
2026-07-09 16:41:34 +00:00
Byron Karlen 43662ca9f5 [eas-cli] Remove unused eas onboarding command (#3993) 2026-07-09 10:41:04 -06:00
Jon Samp cc42fdc28d Improves workflow:create command (#3943)
* improves create command

* save

* fable review

* removes question

* dont prompt for application bundle id when not needed

* [eas-cli] Exit non-zero when workflow:create fails

* [eas-cli] Simplify project configuration in workflow:create

* [eas-cli] Document why workflow:create skips owner/slug consistency check
2026-07-08 14:51:20 -05:00
Douglas Lowder 0727ae467a [eas-cli] Accept navigation metric aliases as positional args to observe:metrics (#3973)
# Why

When running `observe:metrics` in non-interactive mode, navigation metric names could not be passed into the command; only app startup metrics were accepted.

# How

- Included the navigation metrics in the validation check
- Renamed the short names (keys) for the navigation metrics to avoid confusion with app startup metric short names

# Test Plan

- Tested locally
- Unit tests added and updated
2026-07-08 05:17:35 -07:00
Gabe Debes fc0ebff305 [eas-cli] Open the PostHog dashboard via a signed-in deep link (#3975)
# Why

`eas integrations:posthog:dashboard` should open the linked PostHog project already signed in, instead of a static URL that hits a login wall.

# How

Mints a one-time, signed-in link via the new `createPostHogDeepLink` mutation and opens it. The link carries a single-use token, so it isn't printed by default; pass `--show-link` to print it. `--json` / `--non-interactive` return the stable static URL, which minting failures also fall back to.

# Test Plan

Unit tests + verified end-to-end with `easd`

<img width="1548" height="484" alt="carbon (5)" src="https://github.com/user-attachments/assets/c00961e5-da49-456c-af1e-c4a2700eece4" />
2026-07-07 15:39:51 -07:00
David Mokos f89e9ddef0 [cli] add agent detection to telemetry (#3983)
Ports the Expo CLI agent telemetry integration from https://github.com/expo/expo/pull/47248 to EAS CLI.

Adds `agent-cli-detector` to EAS CLI telemetry so events recorded from known coding agents include lightweight agent context. The CLI uses `[email protected]`, which exposes a CommonJS entry point, so detection stays synchronous and does not participate in telemetry initialization.

When a known agent is detected, telemetry event context includes:

```ts
context.agent = {
  id: string,
  sessionId: string | undefined,
}
```

When no known agent is detected, `context.agent` is omitted to keep RudderStack and BigQuery payloads sparse. CLI version is already available in telemetry, so analysis can distinguish newer CLI versions with no detected agent from older CLI versions without this integration.

Tests cover detected and undetected telemetry payloads. Locally verified with `yarn workspace eas-cli test src/analytics/__tests__/AnalyticsManager-test.ts --runInBand`, `yarn workspace eas-cli typecheck`, `yarn lint-changelog`, and targeted `oxlint`.
2026-07-07 16:53:26 +02:00
Szymon Dziedzic b9d59e2628 Bump glob in root lockfile (#3979)
* Bump glob in root lockfile

* Add changelog entry for glob bump
2026-07-07 15:20:18 +02:00
Szymon Dziedzic 502f713467 Remove stale changelog PR template instruction (#3984) 2026-07-07 15:09:12 +02:00
Stanisław Chmiela 48bdd42bc3 [build-tools] Wait for remote sessions to stop cleanly (#3946)
## Summary

- replace the remote device session infinite waits with polling for the device run session to become `STOPPED`
- let agent-device, Argent, and serve-sim session steps return normally after the stop signal so the worker can report success and the launcher can close the VM
- make Argent artifact polling stoppable, drain one final artifact list on shutdown, and wait briefly for pending uploads

## Why

Stopping a device run session currently cancels the backing turtle job run, which tears down the VM immediately. That skips graceful shutdown work such as finishing recordings and uploading session artifacts. This change makes the worker observe the session stop state and finish cleanup before exiting.

## Validation

- `yarn oxfmt <modified files>`
- `yarn --cwd packages/build-tools test -- remoteDeviceRunSession`
- `yarn --cwd packages/build-tools test -- argentArtifacts`
- `yarn --cwd packages/build-tools typecheck`
- `git diff --check`

## Note

This relies on the backend/API behavior changing so `ensureDeviceRunSessionStopped` marks the device run session `STOPPED` without canceling the turtle job run, and allows artifact uploads while the stopped session is still cleaning up.
2026-07-07 14:52:58 +02:00
Stanisław Chmiela 81d09514fc [build-tools] Wire agent-device artifact kinds (#3980)
## Summary

- Preserve `artifactType` from agent-device artifact inventory responses.
- Send that value to the device run session artifact upload-session mutation as `kind`.
- Make the local device run session artifact upload helper require an explicit `kind: string | undefined` so untyped uploads are intentional.
- Regenerate EAS CLI GraphQL schema/types with the repository codegen script and update the one fixture affected by regenerated schema drift.

## Why

agent-device now emits semantic artifact types, and universe now stores them on device run session artifacts. This wires the build-tools upload path between those two changes so users can see producer-owned artifact kinds instead of relying on filenames.

## Validation

CI should pass.
2026-07-07 14:17:19 +02:00
Stanisław Chmiela 081e1a9cf7 [build-tools] Add iOS Simulator recorder (#3950)
<!-- If this PR requires a changelog entry, add it by commenting the PR with the command `/changelog-entry [breaking-change|new-feature|bug-fix|chore] [message]`. -->
<!-- You can skip the changelog check by labeling the PR with "no changelog". -->

# Why

Add a small Swift helper for recording iOS Simulator screen output from EAS build workers. The helper produces either segmented fMP4 output with manifest metadata for playlist construction or one continuous MP4 when segmentation is disabled.

# How

- Added a `record-sim` Swift package under `@expo/build-tools` resources.
- Added `yarn workspace @expo/build-tools build:record-sim` for local builds into `packages/build-tools/bin/record-sim`.
- Updated iOS worker packaging to build `record-sim` from source and copy only the executable into the worker archive.
- Ignored the generated local `bin/record-sim` binary.

# Test Plan

- `yarn workspace @expo/build-tools build:record-sim`
- `packages/build-tools/bin/record-sim --help`
- `bash -n packages/worker/package.sh`
- `yarn workspace @expo/build-tools typecheck`
- `./node_modules/.bin/oxfmt --check packages/build-tools/.gitignore packages/build-tools/package.json packages/worker/package.sh packages/build-tools/src/index.ts`
2026-07-07 12:29:14 +02:00
Gabe Debes bc6369204d [build-tools] Skip embedded bundle upload for dev client builds (#3940)
## Why

Dev client builds don't embed a bundle. We only skipped the upload for iOS simulators, so other dev builds warned about a missing bundle. This skips all dev client builds.

## Test plan

Added a unit test. CI passes.
2026-07-06 16:36:12 -07:00
Szymon Dziedzic 4855745d67 [eas-cli] Upload agent-device session artifacts (#3939)
* [eas-cli] Retain agent-device session artifacts

* [eas-cli] Upload agent-device session artifacts
2026-07-06 18:22:27 +02:00
Gabe Debes a848411278 [eas-cli] Re-upload assets stuck in polling loop when GCF finalize retry window is exhausted (#3918)
* [eas-cli] Retry uploading update assets that do not finish processing

* [eas-cli] Prefix changelog entry with [eas-cli]

* [eas-cli] Restore poll-window comment and parallelize asset limit fetch

* [eas-cli] Move changelog entry back under main
2026-07-02 15:18:53 -07:00
Gabe Debes 58968ddcd9 [eas-cli] Regenerate GraphQL schema and fix affected types (#3938) 2026-07-02 12:44:05 -07:00
Mustafa Abban 72ca5c2527 [build-tools] Apply npm cache registry only to the dependency install (#3925)
* [build-tools] Apply npm cache registry only to the dependency install

* cache install test

* inline env, remove deprecated env

* use cache url as signal
2026-07-02 10:17:30 -04:00
Stanisław Chmiela 8aa4cae097 [build-tools] Reset Gradle oom score before launch (#3898)
## Summary

Replace the delayed Gradle descendant `oom_score_adj` scan with an immediate shell-level reset before launching Gradle.

The worker service is protected with `OOMScoreAdjust=-999`, so Gradle inherits that value unless we reset it. This changes the Gradle launch wrapper to write `0` to `/proc/$$/oom_score_adj` before `exec ./gradlew ...`, which makes Gradle and its child processes inherit the default OOM score instead of the worker's protected score.

## Validation

- `yarn --cwd packages/build-tools typecheck`
- `yarn oxfmt --check packages/build-tools/src/android/gradle.ts packages/build-tools/src/steps/utils/android/gradle.ts`
- `yarn oxlint --type-aware --import-plugin --tsconfig ./tsconfig.oxlint.json packages/build-tools/src/android/gradle.ts packages/build-tools/src/steps/utils/android/gradle.ts`
2026-07-02 12:25:28 +02:00
Stanisław Chmiela 8a66d3808e [build-tools] Let updateEnv override existing values (#3901)
## Summary
- Change `BuildContext.updateEnv` so incoming values override existing env values.
- Add a regression test covering overrides and clearing via undefined.

## Why
This makes `updateEnv` behave like an update operation and lets callers intentionally refresh or clear existing environment entries.

No idea how it existed so long like this. I guess the values we update with don't overlap with the values we already have that often.

## Test Plan

CI should pass.
2026-07-02 11:06:20 +02:00
Expo CI 6697add2c0 update CHANGELOG.md 2026-07-01 16:37:38 +00:00
Expo CI cd47185f29 v20.5.1 2026-07-01 16:34:31 +00:00
88c51c3ccc [eas-cli] Fix Free plan users being falsely warned they hit their build limit (#3927)
The "over" usage tier fired whenever the summed value of any overageMetrics
row was greater than zero, independent of the plan limit or whether a charge
was actually incurred. For Free plans that message collapsed into the
"you've reached your limit, upgrade to keep building" warning, so free users
well below their limit (e.g. 6 builds) were told they were blocked.

Free plans can never be in an "over" state (they are blocked at the limit
and never charged pay-as-you-go), and even paid plans now require a real
billable overage cost (totalCost > 0) before showing the overage warning.

Co-authored-by: Sarah Lane <[email protected]>
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
2026-07-01 09:38:27 -06:00
243 changed files with 23183 additions and 7962 deletions

No files matched your search

-1
View File
@@ -1,4 +1,3 @@
<!-- If this PR requires a changelog entry, add it by commenting the PR with the command `/changelog-entry [breaking-change|new-feature|bug-fix|chore] [message]`. -->
<!-- You can skip the changelog check by labeling the PR with "no changelog". -->
# Why
+1 -1
View File
@@ -2,7 +2,7 @@ commentConfiguration:
preamble: 'Subscribed to pull request'
epilogue: 'Generated by [CodeMention](https://github.com/expo/expo/actions/workflows/codemention.yaml)'
rules:
- patterns: ['**/*']
- patterns: ['packages/eas-cli/**']
mentions: ['douglowder']
- patterns: ['packages/eas-cli/src/update/**']
mentions: ['wschurman', 'quinlanj']
+37
View File
@@ -0,0 +1,37 @@
name: License
on:
schedule:
- cron: '0 0 1 1-12/3 *' # Run every three months
push:
branches:
- main
paths:
- .github/workflows/license.yml
workflow_dispatch:
jobs:
update:
runs-on: ubuntu-latest
name: Update BSL change date
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
token: ${{ secrets.EXPO_BOT_PAT }}
- name: Check last commit date
run: |
# The date in the license
change_date=$(sed --quiet --regexp-extended 's/^Change Date:\s+([0-9]{4}-[0-9]{2}-[0-9]{2})/\1/p' < LICENSE-BUSL)
[ -z "$change_date" ] && exit 1
new_change_date=$(date --date "+3 years" --iso-8601)
if [ "$(date --date "$new_change_date" +%s)" -gt "$(date --date "$change_date" +%s)" ]; then
sed --in-place "s/$change_date/$new_change_date/" LICENSE-BUSL
git config user.name 'Expo Bot'
git config user.email [email protected]
git add LICENSE-BUSL
git commit -m "[license] Update BSL Change Date to $new_change_date"
if [ "$(git branch --show-current)" = main ]; then
git push
fi
fi
+77
View File
@@ -8,8 +8,85 @@ This is the log of notable changes to EAS CLI and related packages.
### 🎉 New features
- [eas-cli] Support drill-down from event lists in observe:session. ([#3987](https://github.com/expo/eas-cli/pull/3987) by [@douglowder](https://github.com/douglowder))
### 🐛 Bug fixes
### 🧹 Chores
## [21.0.2](https://github.com/expo/eas-cli/releases/tag/v21.0.2) - 2026-07-17
### 🐛 Bug fixes
- [eas-cli] Remove temporary working directories after `eas build:inspect` copies inspect output, avoiding leftover disk usage after failed builds. ([#3981](https://github.com/expo/eas-cli/pull/3981) by [@szdziedzic](https://github.com/szdziedzic))
### 🧹 Chores
- [eas-cli] Bump `tmp` to `0.2.7` to resolve [Dependabot alert 423](https://github.com/expo/eas-cli/security/dependabot/423). ([#3965](https://github.com/expo/eas-cli/pull/3965) by [@szdziedzic](https://github.com/szdziedzic))
- [worker] Bump `ws` to patched releases to resolve [Dependabot alert 462](https://github.com/expo/eas-cli/security/dependabot/462). ([#3963](https://github.com/expo/eas-cli/pull/3963) by [@szdziedzic](https://github.com/szdziedzic))
- Bump `lodash` and `lodash-es` to `4.18.1` across workspaces and upgrade GraphQL Code Generator to remove the remaining vulnerable transitive copy for Dependabot alert 354. ([#4009](https://github.com/expo/eas-cli/pull/4009) by [@sjchmiela](https://github.com/sjchmiela))
- [eas-cli] Bump `sigstore` to `4.1.1` to resolve [Dependabot alert 475](https://github.com/expo/eas-cli/security/dependabot/475). ([#3961](https://github.com/expo/eas-cli/pull/3961) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] Bump `axios` to patched releases to resolve [Dependabot alert 439](https://github.com/expo/eas-cli/security/dependabot/439). ([#3964](https://github.com/expo/eas-cli/pull/3964) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] Bump `handlebars` to `4.7.9` to resolve [Dependabot alert 334](https://github.com/expo/eas-cli/security/dependabot/334). ([#3958](https://github.com/expo/eas-cli/pull/3958) by [@szdziedzic](https://github.com/szdziedzic))
## [21.0.1](https://github.com/expo/eas-cli/releases/tag/v21.0.1) - 2026-07-15
### 🎉 New features
- [eas-cli] Add `eas project:delete` command to delete a project — the current directory's project by default, or any project passed as an argument (`@account/slug` or ID). Confirmation requires the project's full name, typed at the interactive prompt or passed via `--dangerously-confirm-deletion` (required in non-interactive mode). Deletion requires sudo mode: the command prompts for your password (and OTP) to upgrade the session when the server requires it. Deletion was previously only possible from the website. ([#4012](https://github.com/expo/eas-cli/pull/4012) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli] Add `eas browse` to open the project page on expo.dev in a web browser, with an optional page argument (e.g. `eas browse build`, `eas browse hosting`) for project subpages and a `--no-browser` flag to print the URL. `eas open` is now an alias of `eas browse`. ([#4003](https://github.com/expo/eas-cli/pull/4003) by [@jonsamp](https://github.com/jonsamp))
- [eas-cli] Add `--dev-domain` flag to `eas deploy` to choose the project's preview URL (`<name>.expo.app`) without prompting, allowing first-time deploys with `--non-interactive`. ([#4004](https://github.com/expo/eas-cli/pull/4004) by [@brentvatne](https://github.com/brentvatne))
- [eas-cli] Add `-p` as an alias for `--platform` in `eas simulator:start`. ([#3902](https://github.com/expo/eas-cli/pull/3902) by [@brentvatne](https://github.com/brentvatne))
### 🧹 Chores
- [eas-cli][build-tools][local-build-plugin][worker] Bump `tar` to `7.5.19`. ([#3971](https://github.com/expo/eas-cli/pull/3971) by [@szdziedzic](https://github.com/szdziedzic))
## [21.0.0](https://github.com/expo/eas-cli/releases/tag/v21.0.0) - 2026-07-13
### 🛠 Breaking changes
- [eas-cli] Remove `eas onboarding` command (aliases `init:onboarding`, `project:onboarding`). ([#3993](https://github.com/expo/eas-cli/pull/3993) by [@byronkarlen](https://github.com/byronkarlen))
### 🎉 New features
- [eas-cli] Add `eas env:set`, which creates or updates an environment variable in one command. `eas env:create` and `eas env:update` are now deprecated. ([#4002](https://github.com/expo/eas-cli/pull/4002) by [@jonsamp](https://github.com/jonsamp))
- [eas-cli] Improve `eas workflow:create`: add a `--template` flag, generate a placeholder workflow when a file name is passed, use shorter default file names (`build.yml`, `update.yml`, `deploy.yml`), configure EAS Build and EAS Update automatically when the chosen template requires them, set default app identifiers without prompting for the development build and deploy templates, install `expo-dev-client` during development build setup, and tighten the generated comments and next steps. ([#3943](https://github.com/expo/eas-cli/pull/3943) by [@jonsamp](https://github.com/jonsamp))
- [eas-cli] `eas integrations:posthog:dashboard` now opens PostHog via a signed-in link, skipping the login prompt. ([#3975](https://github.com/expo/eas-cli/pull/3975) by [@gwdp](https://github.com/gwdp))
- [build-tools] Add `eas/posthog_capture_event` workflow function to send PostHog events from workflow runs. ([#3934](https://github.com/expo/eas-cli/pull/3934) by [@gwdp](https://github.com/gwdp))
- [build-tools] Add `eas/posthog_flag_rollout` workflow function to update PostHog feature flags from workflow runs. ([#3944](https://github.com/expo/eas-cli/pull/3944) by [@gwdp](https://github.com/gwdp))
- [build-tools] Add `eas/posthog_wait_for_metric` workflow function to gate workflow runs on a PostHog metric. ([#3945](https://github.com/expo/eas-cli/pull/3945) by [@gwdp](https://github.com/gwdp))
- [build-tools] Add `eas/posthog_upload_sourcemaps` workflow function to upload source maps to PostHog. ([#3947](https://github.com/expo/eas-cli/pull/3947) by [@gwdp](https://github.com/gwdp))
- [build-tools] Add `eas/posthog_annotation` workflow function to create PostHog annotations from workflow runs. ([#3948](https://github.com/expo/eas-cli/pull/3948) by [@gwdp](https://github.com/gwdp))
- [build-tools] Add `eas/posthog_wait_for_query` workflow function to gate workflow runs on a PostHog query. ([#3949](https://github.com/expo/eas-cli/pull/3949) by [@gwdp](https://github.com/gwdp))
### 🐛 Bug fixes
- [eas-cli] Rethrow Apple errors during capability sync instead of silently succeeding. ([#3989](https://github.com/expo/eas-cli/pull/3989) by [@douglowder](https://github.com/douglowder))
- [build-tools] Skip embedded bundle upload for development client builds instead of warning that the bundle is missing. ([#3940](https://github.com/expo/eas-cli/pull/3940) by [@gwdp](https://github.com/gwdp))
- [eas-cli] Accept a navigation metric name in observe:metrics when passed in on the command line. ([#3973](https://github.com/expo/eas-cli/pull/3973) by [@douglowder](https://github.com/douglowder))
- [eas-cli] Retry uploading assets that don't finish processing during `eas update`, instead of failing the update. ([#3918](https://github.com/expo/eas-cli/pull/3918) by [@gwdp](https://github.com/gwdp))
### 🧹 Chores
- Upgrade Expo plist dependencies to use `@xmldom/xmldom` `0.8.13` without a root resolution, addressing Dependabot alert 372. ([#4010](https://github.com/expo/eas-cli/pull/4010) by [@sjchmiela](https://github.com/sjchmiela))
- [eas-cli] add agent detection to analytics events. ([#3983](https://github.com/expo/eas-cli/pull/3983) by [@davidmokos](https://github.com/davidmokos))
- Bump `glob` 10.x to a patched version in the root lockfile. ([#3979](https://github.com/expo/eas-cli/pull/3979) by [@szdziedzic](https://github.com/szdziedzic))
- [build-tools] Bump `jws` to a patched version in the root lockfile. ([#3978](https://github.com/expo/eas-cli/pull/3978) by [@szdziedzic](https://github.com/szdziedzic))
- [worker] Bump `koa` to `3.1.2`. ([#3974](https://github.com/expo/eas-cli/pull/3974) by [@szdziedzic](https://github.com/szdziedzic))
- [steps] Bump `cross-spawn` to a patched version in the TypeScript custom function fixture. ([#3977](https://github.com/expo/eas-cli/pull/3977) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] Bump `node-forge` to `1.4.0`. ([#3970](https://github.com/expo/eas-cli/pull/3970) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] Bump `fast-xml-builder` to `1.2.1` to resolve [Dependabot alert 400](https://github.com/expo/eas-cli/security/dependabot/400). ([#3966](https://github.com/expo/eas-cli/pull/3966) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] Bump `form-data` to patched releases to resolve [Dependabot alert 465](https://github.com/expo/eas-cli/security/dependabot/465). ([#3962](https://github.com/expo/eas-cli/pull/3962) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] Bump `fast-xml-parser` to `4.5.7` to resolve [Dependabot alert 286](https://github.com/expo/eas-cli/security/dependabot/286). ([#3960](https://github.com/expo/eas-cli/pull/3960) by [@szdziedzic](https://github.com/szdziedzic))
- [eas-cli] Bump `shell-quote` to `1.9.0` to resolve [Dependabot alert 440](https://github.com/expo/eas-cli/security/dependabot/440). ([#3959](https://github.com/expo/eas-cli/pull/3959) by [@szdziedzic](https://github.com/szdziedzic))
## [20.5.1](https://github.com/expo/eas-cli/releases/tag/v20.5.1) - 2026-07-01
### 🐛 Bug fixes
- [eas-cli] Fix Free plan users being warned they had reached their build limit while still well below it. Overage warnings are now shown only for paid plans with an actual billable overage cost. ([#3882](https://github.com/expo/eas-cli/pull/3882) by [@sarahlane8](https://github.com/sarahlane8))
- [eas-cli] Count only genuine per-worker build overages in the pre-build usage warning, so the reported number of builds beyond included credits is accurate. ([#3932](https://github.com/expo/eas-cli/pull/3932) by [@sarahlane8](https://github.com/sarahlane8))
- [eas-cli] Select correct tvOS build target for non-interactive Apple builds when EXPO_TV env variable is set. ([#3907](https://github.com/expo/eas-cli/pull/3907) by [@douglowder](https://github.com/douglowder))
### 🧹 Chores
+1 -1
View File
@@ -12,7 +12,7 @@ Additional Use Grant: You may make use of the Licensed Work, provided that you d
to access the functionality of and directly benefit from the
functionality of the Licensed Work.
Change Date: 2028-04-01
Change Date: 2029-07-15
Change License: MIT
+1 -1
View File
@@ -1,5 +1,5 @@
{
"npmClient": "yarn",
"version": "20.5.0",
"version": "21.0.3",
"$schema": "node_modules/lerna/schemas/lerna-schema.json"
}
+1
View File
@@ -1,5 +1,6 @@
node_modules/
dist/
bin/record-sim
.secrets*
!.secrets.example
+11 -11
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/build-tools",
"version": "20.4.0",
"version": "21.0.3",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "BUSL-1.1",
"author": "Expo <[email protected]>",
@@ -23,6 +23,7 @@
"watch": "tsc --watch --preserveWatchOutput",
"prebuild": "yarn gql",
"build": "tsc",
"build:record-sim": "mkdir -p bin && record_sim_bin_path=$(swift build -c release --package-path resources/record-sim --build-path resources/record-sim/.build --show-bin-path) && swift build -c release --package-path resources/record-sim --build-path resources/record-sim/.build && cp \"$record_sim_bin_path/record-sim\" bin/record-sim && chmod +x bin/record-sim",
"typecheck": "tsc",
"prepack": "rimraf dist \"*.tsbuildinfo\" && yarn gql && tsc -p tsconfig.build.json",
"jest-unit": "jest --config jest/unit-config.ts",
@@ -37,19 +38,18 @@
"dependencies": {
"@expo/config": "55.0.10",
"@expo/config-plugins": "55.0.7",
"@expo/downloader": "20.0.0",
"@expo/eas-build-job": "20.1.0",
"@expo/downloader": "21.0.0",
"@expo/eas-build-job": "21.0.3",
"@expo/env": "^0.4.0",
"@expo/logger": "20.0.0",
"@expo/logger": "21.0.0",
"@expo/package-manager": "1.9.10",
"@expo/plist": "^0.2.0",
"@expo/plist": "^0.3.5",
"@expo/results": "^1.0.0",
"@expo/spawn-async": "1.7.2",
"@expo/steps": "20.1.0",
"@expo/template-file": "20.0.0",
"@expo/turtle-spawn": "20.0.0",
"@expo/steps": "21.0.3",
"@expo/template-file": "21.0.2",
"@expo/turtle-spawn": "21.0.0",
"@expo/xcpretty": "^4.3.1",
"@google-cloud/storage": "^7.11.2",
"@ngrok/ngrok": "1.7.0",
"@sentry/node": "7.77.0",
"@urql/core": "^6.0.1",
@@ -60,7 +60,7 @@
"gql.tada": "^1.8.13",
"joi": "^17.13.1",
"jose": "5.10.0",
"lodash": "^4.17.21",
"lodash": "^4.18.1",
"node-fetch": "^2.7.0",
"node-forge": "^1.3.1",
"node-stream-zip": "1.15.0",
@@ -71,7 +71,7 @@
"resolve-from": "^5.0.0",
"retry": "^0.13.1",
"semver": "^7.6.2",
"tar": "7.5.7",
"tar": "7.5.19",
"uuid": "9.0.1",
"yaml": "^2.8.1",
"zod": "^4.3.5"
@@ -0,0 +1,4 @@
.build/
.swiftpm/
DerivedData/
.DS_Store
@@ -0,0 +1,28 @@
// swift-tools-version: 5.9
import PackageDescription
let package = Package(
name: "record-sim",
platforms: [
.macOS(.v14),
],
products: [
.library(name: "RecordSim", targets: ["RecordSim"]),
.executable(name: "record-sim", targets: ["record-sim"]),
],
targets: [
.target(
name: "RecordSim",
linkerSettings: [
.linkedFramework("AVFoundation"),
.linkedFramework("CoreGraphics"),
.linkedFramework("CoreMedia"),
.linkedFramework("CoreVideo"),
.linkedFramework("IOSurface"),
.linkedFramework("UniformTypeIdentifiers"),
]
),
.executableTarget(name: "record-sim", dependencies: ["RecordSim"]),
]
)
@@ -0,0 +1,79 @@
# RecordSim
Small Swift library for non-exclusive iOS Simulator screen recording.
It implements one recording path:
1. Listen to SimRenderServer framebuffer callbacks.
2. Copy the live `IOSurface` immediately into an owned `CVPixelBuffer`.
3. Feed one continuous `AVAssetWriter`.
4. Emit Apple HLS/CMAF-style fragmented MP4 segments.
The output is suitable for uploading while recording. `session.json` contains
the ordered segment metadata needed to build a playlist later.
## Output
```text
session/
init.mp4
session.json
segments/
segment-000000.m4s
segment-000001.m4s
```
`init.mp4` is the fMP4 initialization segment. Individual `.m4s` files are not
standalone MP4 files. `session.json` stores the wall-clock timestamp for the
first video frame plus the metadata needed to build a playlist: HLS version,
target duration, media sequence, `initSegment`, and the ordered `segments` array
with file path and duration for each media segment.
## CLI
```bash
swift run record-sim \
--udid <BOOTED_SIMULATOR_UDID> \
--output /tmp/sim-recording \
--segment-duration 120 \
--fps 30 \
--bitrate 30000000 \
--codec h264
```
For sparse 2h sessions, `--segment-duration 120` is a reasonable default: about
60 media objects plus the init segment. The recorder appends a single duplicate
hold frame near each segment boundary when the simulator is idle, so long idle
periods still produce uploadable segments without encoding idle frames at 60fps.
Use `--segment-duration 0` to write one continuous `recording.mp4` instead of
`init.mp4` plus media segments.
Frame timestamps use monotonic host time, not wall-clock time. The recorder also
probes the framebuffer seed once per second; if the surface changes without
callbacks for 5 seconds, it captures that frame and rewires the private callback
registration.
## Library
```swift
let recorder = SimulatorRecorder(
configuration: SimulatorRecordingConfiguration(
deviceUDID: udid,
outputDirectory: sessionDirectory,
segmentDuration: 120
)
)
recorder.onSegment = { segment in
// Enqueue upload of segment.url.
}
try recorder.start()
try recorder.waitUntilFirstFrame()
// ...
let manifest = try recorder.stop()
```
`session.json` contains the precise wall-clock time for video PTS zero. In
segmented mode it also contains the init segment and ordered media segment
file/duration entries; in single-file mode it points at `recording.mp4`.
@@ -0,0 +1,192 @@
import Foundation
import IOSurface
import ObjectiveC
final class FramebufferDisplaySource {
struct SurfaceSnapshot {
let surface: IOSurface
let width: Int
let height: Int
let seed: UInt32
}
private let deviceUDID: String
private let callbackQueue: DispatchQueue
private let onFrame: () -> Void
private let onSurfaceChange: () -> Void
private var descriptors: [NSObject] = []
private var callbackUUIDs: [ObjectIdentifier: NSUUID] = [:]
private var retainedBlocks: [AnyObject] = []
private var ioClient: NSObject?
init(
deviceUDID: String,
callbackQueue: DispatchQueue,
onFrame: @escaping () -> Void,
onSurfaceChange: @escaping () -> Void
) {
self.deviceUDID = deviceUDID
self.callbackQueue = callbackQueue
self.onFrame = onFrame
self.onSurfaceChange = onSurfaceChange
}
func start() throws {
try PrivateSimulatorFrameworks.load()
guard let device = SimulatorDeviceLookup.find(udid: deviceUDID) else {
throw RecorderError.make(1, "Simulator \(deviceUDID) not found")
}
let state = device.value(forKey: "stateString") as? String ?? "unknown"
guard state == "Booted" else {
throw RecorderError.make(2, "Simulator \(deviceUDID) is not booted (state: \(state))")
}
guard
let io = device.perform(NSSelectorFromString("io"))?.takeUnretainedValue() as? NSObject
else {
throw RecorderError.make(3, "Failed to get simulator IO client")
}
ioClient = io
try wireUpFramebuffer()
}
func stop() {
unregisterCallbacks()
descriptors.removeAll()
retainedBlocks.removeAll()
ioClient = nil
}
func surfaceSnapshot() -> SurfaceSnapshot? {
let surfaceSelector = NSSelectorFromString("framebufferSurface")
var bestSnapshot: SurfaceSnapshot?
var bestArea = 0
for descriptor in descriptors {
guard let surfaceObject = descriptor.perform(surfaceSelector)?.takeUnretainedValue()
else {
continue
}
let surface = unsafeBitCast(surfaceObject, to: IOSurface.self)
let width = IOSurfaceGetWidth(surface)
let height = IOSurfaceGetHeight(surface)
let area = width * height
if area > bestArea {
bestSnapshot = SurfaceSnapshot(
surface: surface,
width: width,
height: height,
seed: IOSurfaceGetSeed(surface)
)
bestArea = area
}
}
return bestSnapshot
}
func rewireFramebuffer() throws {
try wireUpFramebuffer()
}
private func wireUpFramebuffer() throws {
guard let io = ioClient else {
throw RecorderError.make(3, "No simulator IO client")
}
io.perform(NSSelectorFromString("updateIOPorts"))
let nextDescriptors = try findFramebufferDescriptors(io: io)
unregisterCallbacks()
descriptors = nextDescriptors
retainedBlocks.removeAll()
do {
for descriptor in descriptors {
try registerCallbacks(descriptor: descriptor)
}
} catch {
unregisterCallbacks()
descriptors.removeAll()
retainedBlocks.removeAll()
throw error
}
}
private func findFramebufferDescriptors(io: NSObject) throws -> [NSObject] {
guard let ports = io.value(forKey: "deviceIOPorts") as? [NSObject] else {
throw RecorderError.make(4, "Failed to read simulator IO ports")
}
let portIdentifierSelector = NSSelectorFromString("portIdentifier")
let descriptorSelector = NSSelectorFromString("descriptor")
let surfaceSelector = NSSelectorFromString("framebufferSurface")
var candidates: [NSObject] = []
for port in ports {
guard port.responds(to: portIdentifierSelector),
let portIdentifier = port.perform(portIdentifierSelector)?.takeUnretainedValue(),
"\(portIdentifier)" == "com.apple.framebuffer.display",
port.responds(to: descriptorSelector),
let descriptor = port.perform(descriptorSelector)?.takeUnretainedValue()
as? NSObject,
descriptor.responds(to: surfaceSelector)
else {
continue
}
candidates.append(descriptor)
}
if candidates.isEmpty {
throw RecorderError.make(5, "No framebuffer display descriptor found")
}
return candidates
}
private func registerCallbacks(descriptor: NSObject) throws {
let selector = NSSelectorFromString(
"registerScreenCallbacksWithUUID:callbackQueue:frameCallback:surfacesChangedCallback:propertiesChangedCallback:"
)
guard descriptor.responds(to: selector) else {
throw RecorderError.make(6, "Framebuffer descriptor does not support screen callbacks")
}
guard let msgSendPointer = dlsym(UnsafeMutableRawPointer(bitPattern: -2), "objc_msgSend")
else {
throw RecorderError.make(7, "objc_msgSend not found")
}
typealias MsgSend =
@convention(c) (
AnyObject, Selector, AnyObject, AnyObject, AnyObject, AnyObject, AnyObject
) -> Void
let msgSend = unsafeBitCast(msgSendPointer, to: MsgSend.self)
let uuid = NSUUID()
callbackUUIDs[ObjectIdentifier(descriptor)] = uuid
let frameCallback: @convention(block) () -> Void = { [weak self] in
self?.onFrame()
}
let surfacesCallback: @convention(block) (AnyObject?, AnyObject?) -> Void = {
[weak self] _, _ in
self?.onSurfaceChange()
}
let propertiesCallback: @convention(block) () -> Void = {}
retainedBlocks.append(frameCallback as AnyObject)
retainedBlocks.append(surfacesCallback as AnyObject)
retainedBlocks.append(propertiesCallback as AnyObject)
msgSend(
descriptor,
selector,
uuid,
callbackQueue as AnyObject,
frameCallback as AnyObject,
surfacesCallback as AnyObject,
propertiesCallback as AnyObject
)
}
private func unregisterCallbacks() {
let selector = NSSelectorFromString("unregisterScreenCallbacksWithUUID:")
for descriptor in descriptors {
if let uuid = callbackUUIDs[ObjectIdentifier(descriptor)],
descriptor.responds(to: selector)
{
descriptor.perform(selector, with: uuid)
}
}
callbackUUIDs.removeAll()
}
}
@@ -0,0 +1,96 @@
import Foundation
import ObjectiveC
enum XcodeDeveloperDirectory {
static let current: String = {
if let developerDir = ProcessInfo.processInfo.environment["DEVELOPER_DIR"],
!developerDir.isEmpty
{
return developerDir
}
let fallback = "/Applications/Xcode.app/Contents/Developer"
let pipe = Pipe()
let process = Process()
process.executableURL = URL(fileURLWithPath: "/usr/bin/xcode-select")
process.arguments = ["-p"]
process.standardOutput = pipe
do {
try process.run()
} catch {
return fallback
}
process.waitUntilExit()
return String(data: pipe.fileHandleForReading.readDataToEndOfFile(), encoding: .utf8)?
.trimmingCharacters(in: .whitespacesAndNewlines) ?? fallback
}()
}
enum PrivateSimulatorFrameworks {
static func load() throws {
try loadResult.get()
}
private static let loadResult: Result<Void, Error> = {
let developerDir = XcodeDeveloperDirectory.current
let candidates = [
"/Library/Developer/PrivateFrameworks/CoreSimulator.framework/CoreSimulator",
"\(developerDir)/Library/PrivateFrameworks/CoreSimulator.framework/CoreSimulator",
"\(developerDir)/../SharedFrameworks/SimulatorKit.framework/SimulatorKit",
"\(developerDir)/Library/PrivateFrameworks/SimulatorKit.framework/SimulatorKit",
]
var failures: [String] = []
for path in candidates {
if dlopen(path, RTLD_NOW) == nil {
let message = dlerror().map { String(cString: $0) } ?? "unknown error"
failures.append("\(path): \(message)")
}
}
guard NSClassFromString("SimServiceContext") != nil else {
let tried = candidates.joined(separator: ", ")
let errors = failures.isEmpty ? "no dlopen errors" : failures.joined(separator: "; ")
return .failure(
RecorderError.make(
8,
"Failed to load CoreSimulator/SimulatorKit (tried: \(tried); errors: \(errors))"
)
)
}
return .success(())
}()
}
enum SimulatorDeviceLookup {
private static let context: NSObject? = {
guard let contextClass = NSClassFromString("SimServiceContext") as? NSObject.Type else {
return nil
}
let sharedSelector = NSSelectorFromString("sharedServiceContextForDeveloperDir:error:")
return contextClass.perform(
sharedSelector, with: XcodeDeveloperDirectory.current, with: nil)?
.takeUnretainedValue() as? NSObject
}()
static func find(udid: String) -> NSObject? {
guard let context else {
return nil
}
let deviceSetSelector = NSSelectorFromString("defaultDeviceSetWithError:")
guard
let deviceSet = context.perform(deviceSetSelector, with: nil)?
.takeUnretainedValue() as? NSObject
else {
return nil
}
guard let devices = deviceSet.value(forKey: "devices") as? [NSObject] else {
return nil
}
return devices.first {
($0.value(forKey: "UDID") as? NSUUID)?.uuidString == udid
}
}
static func stateString(udid: String) -> String? {
find(udid: udid)?.value(forKey: "stateString") as? String
}
}
@@ -0,0 +1,77 @@
import Foundation
public enum RecorderCodec: String, Codable, Sendable {
case h264
case hevc
}
public struct SimulatorRecordingConfiguration: Sendable {
public var deviceUDID: String
public var outputDirectory: URL
public var fps: Int
public var bitrate: Int
public var codec: RecorderCodec
public var segmentDuration: TimeInterval
public var maxPendingFrames: Int
public var overwrite: Bool
public init(
deviceUDID: String,
outputDirectory: URL,
fps: Int = 30,
bitrate: Int = 30_000_000,
codec: RecorderCodec = .h264,
segmentDuration: TimeInterval = 120,
maxPendingFrames: Int = 8,
overwrite: Bool = true
) {
self.deviceUDID = deviceUDID
self.outputDirectory = outputDirectory
self.fps = fps
self.bitrate = bitrate
self.codec = codec
self.segmentDuration = segmentDuration
self.maxPendingFrames = maxPendingFrames
self.overwrite = overwrite
}
}
public enum SegmentKind: String, Codable, Sendable {
case initialization
case media
}
public struct SegmentOutput: Sendable {
public let kind: SegmentKind
public let relativePath: String
public let url: URL
public let byteCount: Int
public let durationSeconds: Double?
}
public struct MediaSegmentRecord: Codable, Sendable {
public let file: String
public let durationSeconds: Double
}
public struct FirstFrameWallClock: Codable, Sendable {
public let unixMs: Int64
public let iso8601: String
}
public enum RecordingFinalizationStage: Sendable {
case captureStopped
case videoSaved
}
public struct RecordingManifest: Codable, Sendable {
public let firstFrameWallClock: FirstFrameWallClock
public let width: Int
public let height: Int
public let hlsVersion: Int?
public let hlsTargetDurationSeconds: Int?
public let hlsMediaSequence: Int?
public let recording: String?
public let initSegment: String?
public let segments: [MediaSegmentRecord]
}
@@ -0,0 +1,140 @@
import AVFoundation
import Foundation
final class RecordingOutputWriter: NSObject, AVAssetWriterDelegate {
private let rootDirectory: URL
private let segmentsDirectory: URL
let singleRecordingURL: URL
private let stateQueue = DispatchQueue(label: "record-sim.segment-writer")
private let notificationQueue = DispatchQueue(label: "record-sim.segment-notifications")
private let onSegment: ((SegmentOutput) -> Void)?
private var initSegmentPath: String?
private var segments: [MediaSegmentRecord] = []
private var nextMediaSegmentIndex = 0
private var firstError: Error?
init(rootDirectory: URL, onSegment: ((SegmentOutput) -> Void)?) {
self.rootDirectory = rootDirectory
self.segmentsDirectory = rootDirectory.appendingPathComponent("segments", isDirectory: true)
self.singleRecordingURL = rootDirectory.appendingPathComponent("recording.mp4")
self.onSegment = onSegment
super.init()
}
func prepare(overwrite: Bool, segmented: Bool) throws {
if overwrite {
try? FileManager.default.removeItem(at: rootDirectory)
}
try FileManager.default.createDirectory(at: rootDirectory, withIntermediateDirectories: true)
if segmented {
try FileManager.default.createDirectory(at: segmentsDirectory, withIntermediateDirectories: true)
}
}
var error: Error? {
stateQueue.sync { firstError }
}
func assetWriter(
_ writer: AVAssetWriter,
didOutputSegmentData segmentData: Data,
segmentType: AVAssetSegmentType,
segmentReport: AVAssetSegmentReport?
) {
var notification: SegmentOutput?
stateQueue.sync {
if firstError != nil {
return
}
do {
switch segmentType {
case .initialization:
let relativePath = "init.mp4"
let url = rootDirectory.appendingPathComponent(relativePath)
try segmentData.write(to: url, options: .atomic)
initSegmentPath = relativePath
notification = SegmentOutput(
kind: .initialization,
relativePath: relativePath,
url: url,
byteCount: segmentData.count,
durationSeconds: nil
)
case .separable:
let index = nextMediaSegmentIndex
let relativePath = String(format: "segments/segment-%06d.m4s", index)
let url = rootDirectory.appendingPathComponent(relativePath)
try segmentData.write(to: url, options: .atomic)
let videoReport = segmentReport?.trackReports.first(where: { $0.mediaType == .video })
let duration = videoReport.flatMap { timeSeconds($0.duration) } ?? 0
let record = MediaSegmentRecord(
file: relativePath,
durationSeconds: duration
)
nextMediaSegmentIndex += 1
segments.append(record)
notification = SegmentOutput(
kind: .media,
relativePath: relativePath,
url: url,
byteCount: segmentData.count,
durationSeconds: duration
)
default:
break
}
} catch {
firstError = error
}
}
if let notification, let onSegment {
notificationQueue.async {
onSegment(notification)
}
}
}
@discardableResult
func writeManifest(
configuration: SimulatorRecordingConfiguration,
firstFrameWallClock: Date,
width: Int,
height: Int
) throws -> RecordingManifest {
try stateQueue.sync {
if let firstError {
throw firstError
}
let segmented = configuration.segmentDuration > 0
if segmented, initSegmentPath == nil {
throw RecorderError.make(50, "Missing initialization segment")
}
if segmented, segments.isEmpty {
throw RecorderError.make(51, "No media segments were written")
}
let targetDuration = segmented
? max(1, Int(ceil(segments.map(\.durationSeconds).max() ?? configuration.segmentDuration)))
: nil
let manifest = RecordingManifest(
firstFrameWallClock: FirstFrameWallClock(
unixMs: unixMs(firstFrameWallClock),
iso8601: iso8601(firstFrameWallClock)
),
width: width,
height: height,
hlsVersion: segmented ? 7 : nil,
hlsTargetDurationSeconds: targetDuration,
hlsMediaSequence: segmented ? 0 : nil,
recording: segmented ? nil : "recording.mp4",
initSegment: segmented ? initSegmentPath : nil,
segments: segments
)
let data = try JSONEncoder.prettySorted.encode(manifest)
try data.write(to: rootDirectory.appendingPathComponent("session.json"), options: .atomic)
return manifest
}
}
}
@@ -0,0 +1,644 @@
import AVFoundation
import CoreVideo
import Foundation
import IOSurface
import UniformTypeIdentifiers
public final class SimulatorRecorder {
public var onSegment: ((SegmentOutput) -> Void)?
public var onSimulatorStopped: ((String) -> Void)?
public var onFinalizationStage: ((RecordingFinalizationStage) -> Void)?
private static let callbackStalenessTimeout: TimeInterval = 5
private static let firstFrameRewireInterval: TimeInterval = 1
private let configuration: SimulatorRecordingConfiguration
private let callbackQueue = DispatchQueue(
label: "record-sim.frame-callbacks", qos: .userInteractive)
private let writerQueue = DispatchQueue(label: "record-sim.asset-writer", qos: .userInitiated)
private let eventQueue = DispatchQueue(label: "record-sim.events", qos: .utility)
private let pendingLock = NSLock()
private var pendingFrames = 0
private var displaySource: FramebufferDisplaySource?
private var outputWriter: RecordingOutputWriter?
private var writer: AVAssetWriter?
private var input: AVAssetWriterInput?
private var adaptor: AVAssetWriterInputPixelBufferAdaptor?
private var boundaryTimer: DispatchSourceTimer?
private var monotonicClock = MonotonicClock()
private var firstAcceptedCaptureTime: CMTime?
private var firstAcceptedWallClock: Date?
private var recordingWidth: Int?
private var recordingHeight: Int?
private var lastPTS: CMTime?
private var lastSeed: UInt32?
private var lastFrameCallbackElapsed: TimeInterval?
private var lastFirstFrameRewireElapsed: TimeInterval = 0
private var lastAppendedPixelBuffer: CVPixelBuffer?
private var nextBoundaryElapsed: TimeInterval = 0
private var simulatorStoppedReason: String?
private var stopped = false
private var firstFrameReady = false
private let firstFrameSemaphore = DispatchSemaphore(value: 0)
private var firstError: Error?
public init(configuration: SimulatorRecordingConfiguration) {
self.configuration = configuration
}
public func start() throws {
try validateConfiguration()
let outputWriter = RecordingOutputWriter(
rootDirectory: configuration.outputDirectory,
onSegment: { [weak self] segment in
self?.onSegment?(segment)
})
try outputWriter.prepare(
overwrite: configuration.overwrite, segmented: configuration.segmentDuration > 0)
self.outputWriter = outputWriter
monotonicClock = MonotonicClock()
nextBoundaryElapsed = configuration.segmentDuration
lastFirstFrameRewireElapsed = 0
stopped = false
let source = FramebufferDisplaySource(
deviceUDID: configuration.deviceUDID,
callbackQueue: callbackQueue,
onFrame: { [weak self] in
self?.captureFrame(force: false, reason: .callback)
},
onSurfaceChange: { [weak self] in
self?.captureFrame(force: true, reason: .surfaceChange)
}
)
try source.start()
displaySource = source
startBoundaryTimer()
}
public func waitUntilFirstFrame(timeoutSeconds: TimeInterval = 15) throws {
let alreadyReady = try writerQueue.sync {
if let firstError {
throw firstError
}
return firstFrameReady
}
if alreadyReady {
return
}
if firstFrameSemaphore.wait(timeout: .now() + timeoutSeconds) == .timedOut {
throw RecorderError.make(20, "Timed out waiting for first frame")
}
try writerQueue.sync {
if let firstError {
throw firstError
}
if !firstFrameReady {
throw RecorderError.make(21, "Recorder stopped before first frame")
}
}
}
public func simulatorStopReason() -> String? {
callbackQueue.sync {
simulatorStoppedReason
}
}
@discardableResult
public func stop() throws -> RecordingManifest {
callbackQueue.sync {
stopped = true
boundaryTimer?.cancel()
boundaryTimer = nil
displaySource?.stop()
displaySource = nil
}
onFinalizationStage?(.captureStopped)
return try writerQueue.sync {
if let firstError {
throw firstError
}
guard let writer, let input, let outputWriter else {
let error = RecorderError.make(22, "No frames were captured")
firstError = error
signalFirstFrameReadyIfNeeded()
throw error
}
try appendTailFrameIfNeeded(finalPTS: monotonicClock.elapsedTime())
input.markAsFinished()
let semaphore = DispatchSemaphore(value: 0)
writer.finishWriting {
semaphore.signal()
}
if semaphore.wait(timeout: .now() + 60) == .timedOut {
throw RecorderError.make(23, "Timed out finishing AVAssetWriter")
}
if writer.status == .failed {
throw writer.error ?? RecorderError.make(24, "AVAssetWriter failed")
}
if let error = outputWriter.error {
throw error
}
onFinalizationStage?(.videoSaved)
guard let firstAcceptedWallClock else {
throw RecorderError.make(25, "Missing first frame wall-clock timestamp")
}
guard let recordingWidth, let recordingHeight else {
throw RecorderError.make(27, "Missing recording dimensions")
}
return try outputWriter.writeManifest(
configuration: configuration,
firstFrameWallClock: firstAcceptedWallClock,
width: recordingWidth,
height: recordingHeight
)
}
}
private enum CaptureReason {
case callback
case surfaceChange
case boundary
case healthProbe
}
private func validateConfiguration() throws {
guard !configuration.deviceUDID.isEmpty else {
throw RecorderError.make(30, "deviceUDID must not be empty")
}
guard configuration.fps > 0, configuration.fps <= 120 else {
throw RecorderError.make(31, "fps must be between 1 and 120")
}
guard configuration.bitrate > 0 else {
throw RecorderError.make(32, "bitrate must be positive")
}
guard configuration.segmentDuration >= 0 else {
throw RecorderError.make(33, "segmentDuration must be non-negative")
}
guard configuration.maxPendingFrames > 0 else {
throw RecorderError.make(34, "maxPendingFrames must be positive")
}
}
private func startBoundaryTimer() {
let timer = DispatchSource.makeTimerSource(queue: callbackQueue)
timer.schedule(deadline: .now() + 1, repeating: 1, leeway: .milliseconds(100))
timer.setEventHandler { [weak self] in
self?.probeSimulatorState()
self?.probeFramebufferHealth()
self?.appendBoundaryFrameIfNeeded()
}
timer.resume()
boundaryTimer = timer
}
private func appendBoundaryFrameIfNeeded() {
if stopped {
return
}
guard configuration.segmentDuration > 0 else {
return
}
writerQueue.sync {
guard firstAcceptedCaptureTime != nil else {
return
}
let currentPTSSeconds = CMTimeGetSeconds(
normalizedPresentationTime(for: monotonicClock.elapsedTime()))
guard currentPTSSeconds >= nextBoundaryElapsed else {
return
}
let boundary = nextBoundaryElapsed
guard let lastPTS else {
return
}
do {
if CMTimeGetSeconds(lastPTS) < boundary {
let boundaryPTS = CMTime(seconds: boundary, preferredTimescale: 1_000_000_000)
guard try appendHeldFrame(at: boundaryPTS) else {
return
}
}
nextBoundaryElapsed = boundary + configuration.segmentDuration
} catch {
firstError = error
signalFirstFrameReadyIfNeeded()
}
}
}
private func captureFrame(force: Bool, reason: CaptureReason) {
if stopped {
return
}
if reason == .callback {
lastFrameCallbackElapsed = monotonicClock.elapsedSeconds()
}
guard let snapshot = displaySource?.surfaceSnapshot() else {
return
}
let surface = snapshot.surface
let surfaceWidth = snapshot.width
let surfaceHeight = snapshot.height
guard surfaceWidth > 0, surfaceHeight > 0 else {
return
}
let seed = snapshot.seed
let matchesLastAppendedSeed = writerQueue.sync { lastSeed == seed }
if !force, matchesLastAppendedSeed {
return
}
guard reservePendingFrame() else {
return
}
let capturedAt = monotonicClock.elapsedTime()
let capturedAtWallClock = Date()
do {
let pixelBuffer = try copySurfaceToOwnedPixelBuffer(
surface,
width: surfaceWidth,
height: surfaceHeight
)
appendOwned(
pixelBuffer: pixelBuffer,
seed: seed,
capturedAt: capturedAt,
capturedAtWallClock: capturedAtWallClock
)
} catch {
releasePendingFrame()
writerQueue.async {
self.firstError = error
self.signalFirstFrameReadyIfNeeded()
}
}
}
private func probeFramebufferHealth() {
guard !stopped, let displaySource else {
return
}
let elapsed = monotonicClock.elapsedSeconds()
if !isFirstFrameReady(),
elapsed - lastFirstFrameRewireElapsed >= Self.firstFrameRewireInterval
{
lastFirstFrameRewireElapsed = elapsed
rewireFramebuffer()
captureFrame(force: true, reason: .healthProbe)
return
}
guard let snapshot = displaySource.surfaceSnapshot(),
snapshot.width > 0,
snapshot.height > 0
else {
rewireFramebuffer()
return
}
let observedSeed = snapshot.seed
let lastAppendedSeed = writerQueue.sync { lastSeed }
guard let lastAppendedSeed, observedSeed != lastAppendedSeed else {
return
}
let lastCallback = lastFrameCallbackElapsed ?? 0
if elapsed - lastCallback >= Self.callbackStalenessTimeout {
captureFrame(force: false, reason: .healthProbe)
rewireFramebuffer()
}
}
private func probeSimulatorState() {
guard !stopped, simulatorStoppedReason == nil else {
return
}
let state = SimulatorDeviceLookup.stateString(udid: configuration.deviceUDID) ?? "missing"
guard state != "Booted" else {
return
}
simulatorStoppedReason = state
let error = RecorderError.make(26, "Simulator stopped before first frame (state: \(state))")
writerQueue.async {
if self.firstError == nil, !self.firstFrameReady {
self.firstError = error
self.signalFirstFrameReadyIfNeeded()
}
}
let onSimulatorStopped = onSimulatorStopped
eventQueue.async {
onSimulatorStopped?(state)
}
}
private func rewireFramebuffer() {
guard let displaySource else {
return
}
do {
try displaySource.rewireFramebuffer()
} catch {
writerQueue.async {
self.firstError = error
self.signalFirstFrameReadyIfNeeded()
}
}
}
private func reservePendingFrame() -> Bool {
pendingLock.lock()
defer { pendingLock.unlock() }
if pendingFrames >= configuration.maxPendingFrames {
return false
}
pendingFrames += 1
return true
}
private func releasePendingFrame() {
pendingLock.lock()
pendingFrames = max(0, pendingFrames - 1)
pendingLock.unlock()
}
private func copySurfaceToOwnedPixelBuffer(
_ surface: IOSurface,
width: Int,
height: Int
) throws -> CVPixelBuffer {
let pool = try writerQueue.sync {
if let firstError {
throw firstError
}
if writer == nil {
try startWriter(width: width, height: height)
}
guard let pool = adaptor?.pixelBufferPool else {
throw RecorderError.make(40, "AVAssetWriter pixel buffer pool is unavailable")
}
return pool
}
var output: CVPixelBuffer?
let status = CVPixelBufferPoolCreatePixelBuffer(kCFAllocatorDefault, pool, &output)
guard status == kCVReturnSuccess, let destination = output else {
throw RecorderError.make(41, "Failed to allocate pixel buffer from pool: \(status)")
}
let destinationLockStatus = CVPixelBufferLockBaseAddress(destination, [])
guard destinationLockStatus == kCVReturnSuccess else {
throw RecorderError.make(50, "Failed to lock pixel buffer: \(destinationLockStatus)")
}
let surfaceLockStatus = IOSurfaceLock(surface, .readOnly, nil)
guard surfaceLockStatus == KERN_SUCCESS else {
CVPixelBufferUnlockBaseAddress(destination, [])
throw RecorderError.make(49, "Failed to lock IOSurface: \(surfaceLockStatus)")
}
defer {
IOSurfaceUnlock(surface, .readOnly, nil)
CVPixelBufferUnlockBaseAddress(destination, [])
}
guard let destinationAddress = CVPixelBufferGetBaseAddress(destination) else {
throw RecorderError.make(42, "Pixel buffer has no base address")
}
let sourceAddress = IOSurfaceGetBaseAddress(surface)
let sourceStride = IOSurfaceGetBytesPerRow(surface)
let destinationStride = CVPixelBufferGetBytesPerRow(destination)
let copyWidth = min(width, IOSurfaceGetWidth(surface), CVPixelBufferGetWidth(destination))
let copyHeight = min(
height, IOSurfaceGetHeight(surface), CVPixelBufferGetHeight(destination))
let copyBytes = min(copyWidth * 4, sourceStride, destinationStride)
for row in 0..<copyHeight {
memcpy(
destinationAddress.advanced(by: row * destinationStride),
sourceAddress.advanced(by: row * sourceStride),
copyBytes
)
}
return destination
}
private func appendOwned(
pixelBuffer: CVPixelBuffer,
seed: UInt32,
capturedAt: CMTime,
capturedAtWallClock: Date
) {
writerQueue.async {
defer { self.releasePendingFrame() }
if self.firstError != nil {
return
}
do {
try self.appendOnWriterQueue(
pixelBuffer: pixelBuffer,
seed: seed,
capturedAt: capturedAt,
capturedAtWallClock: capturedAtWallClock
)
} catch {
self.firstError = error
self.signalFirstFrameReadyIfNeeded()
}
}
}
private func appendOnWriterQueue(
pixelBuffer: CVPixelBuffer,
seed: UInt32,
capturedAt: CMTime,
capturedAtWallClock: Date
) throws {
guard let writer, let input, let adaptor else {
throw RecorderError.make(43, "AVAssetWriter is not initialized")
}
let isFirstFrame = firstAcceptedCaptureTime == nil
let pts = isFirstFrame ? .zero : normalizedPresentationTime(for: capturedAt)
if let lastPTS, CMTimeCompare(pts, lastPTS) <= 0 {
return
}
guard try appendPendingBoundaryFrames(before: pts) else {
return
}
guard input.isReadyForMoreMediaData else {
return
}
if !adaptor.append(pixelBuffer, withPresentationTime: pts) {
throw writer.error ?? RecorderError.make(44, "Failed to append pixel buffer")
}
if isFirstFrame {
firstAcceptedCaptureTime = capturedAt
firstAcceptedWallClock = capturedAtWallClock
}
lastPTS = pts
lastSeed = seed
lastAppendedPixelBuffer = pixelBuffer
signalFirstFrameReadyIfNeeded()
}
private func appendTailFrameIfNeeded(finalPTS: CMTime) throws {
let deadline = Date().addingTimeInterval(5)
while true {
if try appendTailFrame(finalPTS: finalPTS) {
return
}
if Date() >= deadline {
throw RecorderError.make(45, "Timed out waiting to append tail frame")
}
Thread.sleep(forTimeInterval: 0.05)
}
}
private func appendTailFrame(finalPTS: CMTime) throws -> Bool {
let pts = normalizedPresentationTime(for: finalPTS)
return try appendHeldFrame(at: pts)
}
private func appendPendingBoundaryFrames(before pts: CMTime) throws -> Bool {
guard configuration.segmentDuration > 0,
firstAcceptedCaptureTime != nil
else {
return true
}
while CMTimeGetSeconds(pts) > nextBoundaryElapsed {
if let lastPTS, CMTimeGetSeconds(lastPTS) >= nextBoundaryElapsed {
nextBoundaryElapsed += configuration.segmentDuration
continue
}
let boundaryPTS = CMTime(
seconds: nextBoundaryElapsed, preferredTimescale: 1_000_000_000)
guard try appendHeldFrame(at: boundaryPTS) else {
return false
}
nextBoundaryElapsed += configuration.segmentDuration
}
return true
}
private func appendHeldFrame(at pts: CMTime) throws -> Bool {
guard let input,
let adaptor,
let lastPTS,
let lastAppendedPixelBuffer
else {
return true
}
if CMTimeCompare(pts, lastPTS) <= 0 {
return true
}
guard input.isReadyForMoreMediaData else {
return false
}
if !adaptor.append(lastAppendedPixelBuffer, withPresentationTime: pts) {
throw writer?.error ?? RecorderError.make(45, "Failed to append held frame")
}
self.lastPTS = pts
return true
}
private func normalizedPresentationTime(for capturedAt: CMTime) -> CMTime {
let start = firstAcceptedCaptureTime ?? capturedAt
return CMTimeSubtract(capturedAt, start)
}
private func startWriter(width: Int, height: Int) throws {
guard let outputWriter else {
throw RecorderError.make(46, "Missing output writer")
}
let writer: AVAssetWriter
if configuration.segmentDuration > 0 {
writer = AVAssetWriter(contentType: .mpeg4Movie)
writer.delegate = outputWriter
writer.outputFileTypeProfile = .mpeg4AppleHLS
writer.preferredOutputSegmentInterval = CMTime(
seconds: configuration.segmentDuration,
preferredTimescale: 600
)
} else {
writer = try AVAssetWriter(outputURL: outputWriter.singleRecordingURL, fileType: .mp4)
}
writer.initialSegmentStartTime = .zero
let input = AVAssetWriterInput(
mediaType: .video,
outputSettings: assetWriterSettings(width: width, height: height)
)
input.expectsMediaDataInRealTime = true
let attributes: [String: Any] = [
kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA,
kCVPixelBufferWidthKey as String: width,
kCVPixelBufferHeightKey as String: height,
kCVPixelBufferIOSurfacePropertiesKey as String: [:],
]
let adaptor = AVAssetWriterInputPixelBufferAdaptor(
assetWriterInput: input,
sourcePixelBufferAttributes: attributes
)
guard writer.canAdd(input) else {
throw RecorderError.make(47, "AVAssetWriter cannot add video input")
}
writer.add(input)
guard writer.startWriting() else {
throw writer.error ?? RecorderError.make(48, "AVAssetWriter failed to start")
}
writer.startSession(atSourceTime: .zero)
self.writer = writer
self.input = input
self.adaptor = adaptor
recordingWidth = width
recordingHeight = height
}
private func assetWriterSettings(width: Int, height: Int) -> [String: Any] {
let codecType: AVVideoCodecType = configuration.codec == .hevc ? .hevc : .h264
var compression: [String: Any] = [
AVVideoAverageBitRateKey: configuration.bitrate,
AVVideoExpectedSourceFrameRateKey: configuration.fps,
AVVideoAllowFrameReorderingKey: true,
]
if configuration.segmentDuration > 0 {
compression[AVVideoMaxKeyFrameIntervalKey] = max(
1,
Int((configuration.segmentDuration * Double(configuration.fps)).rounded())
)
}
if configuration.codec == .h264 {
compression[AVVideoProfileLevelKey] = AVVideoProfileLevelH264HighAutoLevel
}
return [
AVVideoCodecKey: codecType,
AVVideoWidthKey: width,
AVVideoHeightKey: height,
AVVideoCompressionPropertiesKey: compression,
]
}
private func signalFirstFrameReadyIfNeeded() {
if firstFrameReady {
return
}
firstFrameReady = true
firstFrameSemaphore.signal()
}
private func isFirstFrameReady() -> Bool {
writerQueue.sync {
firstFrameReady
}
}
}
@@ -0,0 +1,60 @@
import CoreMedia
import Foundation
enum RecorderError {
static func make(_ code: Int, _ message: String) -> NSError {
NSError(
domain: "RecordSim",
code: code,
userInfo: [NSLocalizedDescriptionKey: message]
)
}
}
extension JSONEncoder {
static var prettySorted: JSONEncoder {
let encoder = JSONEncoder()
encoder.outputFormatting = [.prettyPrinted, .sortedKeys]
return encoder
}
}
func timeSeconds(_ time: CMTime) -> Double? {
guard time.isValid, !time.isIndefinite, !time.isNegativeInfinity, !time.isPositiveInfinity else {
return nil
}
return CMTimeGetSeconds(time)
}
func iso8601(_ date: Date) -> String {
let formatter = ISO8601DateFormatter()
formatter.formatOptions = [.withInternetDateTime, .withFractionalSeconds]
formatter.timeZone = TimeZone(secondsFromGMT: 0)
return formatter.string(from: date)
}
func unixMs(_ date: Date) -> Int64 {
Int64((date.timeIntervalSince1970 * 1000).rounded())
}
struct MonotonicClock {
private static let timescale: CMTimeScale = 1_000_000_000
let startedAtUptimeNanoseconds: UInt64
init() {
startedAtUptimeNanoseconds = DispatchTime.now().uptimeNanoseconds
}
func elapsedSeconds() -> TimeInterval {
TimeInterval(elapsedNanoseconds()) / TimeInterval(Self.timescale)
}
func elapsedTime() -> CMTime {
CMTime(value: CMTimeValue(elapsedNanoseconds()), timescale: Self.timescale)
}
private func elapsedNanoseconds() -> UInt64 {
DispatchTime.now().uptimeNanoseconds - startedAtUptimeNanoseconds
}
}
@@ -0,0 +1,159 @@
import Foundation
import RecordSim
struct CLIOptions {
var udid: String?
var output: String?
var segmentDuration: TimeInterval = 120
var fps = 30
var bitrate = 30_000_000
var codec: RecorderCodec = .h264
}
private var requestedStopSignal: CInt = 0
private var simulatorStopped: CInt = 0
private func handleStopSignal(_ signal: CInt) {
requestedStopSignal = signal
}
private func isStopRequested() -> Bool {
requestedStopSignal != 0 || simulatorStopped != 0
}
func usage() -> String {
"""
Usage:
record-sim --udid <UDID> --output <session-dir> [options]
Options:
--segment-duration <seconds> Target fMP4 media segment duration. Use 0 for one MP4. Default: 120
--fps <fps> Expected source frame rate. Default: 30
--bitrate <bits/sec> AVAssetWriter average bitrate. Default: 30000000
--codec <h264|hevc> Video codec. Default: h264
-h, --help Show this help
"""
}
func parseOptions(_ args: [String]) throws -> CLIOptions {
var options = CLIOptions()
var i = 0
func next(_ flag: String) throws -> String {
i += 1
guard i < args.count else {
throw NSError(domain: "record-sim", code: 2, userInfo: [NSLocalizedDescriptionKey: "Missing value for \(flag)"])
}
return args[i]
}
while i < args.count {
let arg = args[i]
switch arg {
case "--udid", "-u":
options.udid = try next(arg)
case "--output", "--output-dir", "--out", "-o":
options.output = try next(arg)
case "--segment-duration":
options.segmentDuration = Double(try next(arg)) ?? -1
case "--fps":
options.fps = Int(try next(arg)) ?? 0
case "--bitrate":
options.bitrate = Int(try next(arg)) ?? 0
case "--codec":
let raw = try next(arg)
guard let codec = RecorderCodec(rawValue: raw) else {
throw NSError(domain: "record-sim", code: 2, userInfo: [NSLocalizedDescriptionKey: "--codec must be h264 or hevc"])
}
options.codec = codec
case "--help", "-h":
print(usage())
exit(0)
default:
throw NSError(domain: "record-sim", code: 2, userInfo: [NSLocalizedDescriptionKey: "Unknown argument: \(arg)"])
}
i += 1
}
guard options.udid != nil else {
throw NSError(domain: "record-sim", code: 2, userInfo: [NSLocalizedDescriptionKey: "Missing required --udid"])
}
guard options.output != nil else {
throw NSError(domain: "record-sim", code: 2, userInfo: [NSLocalizedDescriptionKey: "Missing required --output"])
}
guard options.segmentDuration >= 0 else {
throw NSError(domain: "record-sim", code: 2, userInfo: [NSLocalizedDescriptionKey: "--segment-duration must be non-negative"])
}
guard options.fps > 0, options.fps <= 120 else {
throw NSError(domain: "record-sim", code: 2, userInfo: [NSLocalizedDescriptionKey: "--fps must be between 1 and 120"])
}
guard options.bitrate > 0 else {
throw NSError(domain: "record-sim", code: 2, userInfo: [NSLocalizedDescriptionKey: "--bitrate must be positive"])
}
return options
}
do {
signal(SIGINT, handleStopSignal)
signal(SIGTERM, handleStopSignal)
let options = try parseOptions(Array(CommandLine.arguments.dropFirst()))
let configuration = SimulatorRecordingConfiguration(
deviceUDID: options.udid!,
outputDirectory: URL(fileURLWithPath: options.output!),
fps: options.fps,
bitrate: options.bitrate,
codec: options.codec,
segmentDuration: options.segmentDuration
)
let recorder = SimulatorRecorder(configuration: configuration)
recorder.onSimulatorStopped = { state in
simulatorStopped = 1
fputs("[record-sim] simulator stopped state=\(state); finalizing recording\n", stderr)
}
recorder.onFinalizationStage = { stage in
switch stage {
case .captureStopped:
fputs("[record-sim] screen capture stopped; saving final video\n", stderr)
case .videoSaved:
fputs("[record-sim] final video saved; writing recording details\n", stderr)
}
}
recorder.onSegment = { segment in
switch segment.kind {
case .initialization:
fputs("[record-sim] init \(segment.relativePath) bytes=\(segment.byteCount)\n", stderr)
case .media:
let duration = segment.durationSeconds ?? 0
fputs(String(format: "[record-sim] segment %@ duration=%.3fs bytes=%d\n", segment.relativePath, duration, segment.byteCount), stderr)
}
}
try recorder.start()
try recorder.waitUntilFirstFrame()
fputs("[record-sim] recording -> \(options.output!)\n", stderr)
while !isStopRequested() {
Thread.sleep(forTimeInterval: 0.1)
}
if requestedStopSignal != 0 {
let signalName = switch requestedStopSignal {
case SIGINT: "SIGINT"
case SIGTERM: "SIGTERM"
default: "signal \(requestedStopSignal)"
}
fputs("[record-sim] received \(signalName); finishing recording\n", stderr)
}
let manifest = try recorder.stop()
if let recording = manifest.recording {
fputs("[record-sim] done recording=\(recording)\n", stderr)
} else {
fputs("[record-sim] done segments=\(manifest.segments.count)\n", stderr)
}
fputs("[record-sim] manifest \(URL(fileURLWithPath: options.output!).appendingPathComponent("session.json").path)\n", stderr)
} catch {
fputs("error: \(error.localizedDescription)\n", stderr)
exit(1)
}
@@ -139,6 +139,38 @@ describe('BuildContext', () => {
});
});
it('overwrites existing environment variables when updating env', async () => {
await vol.promises.mkdir('/workingdir/eas-environment-secrets/', { recursive: true });
const ctx = new BuildContext(
{
triggeredBy: BuildTrigger.GIT_BASED_INTEGRATION,
secrets: {},
} as Job,
{
env: {
__API_SERVER_URL: 'http://api.expo.test',
EXISTING_ENV: 'old-value',
REMOVED_ENV: 'old-value',
},
workingdir: '/workingdir',
logger: createMockLogger(),
logBuffer: { getLogs: () => [], getPhaseLogs: () => [] },
uploadArtifact: jest.fn(),
}
);
ctx.updateEnv({
EXISTING_ENV: 'new-value',
REMOVED_ENV: undefined,
NEW_ENV: 'new-env-value',
});
expect(ctx.env.EXISTING_ENV).toBe('new-value');
expect(ctx.env.REMOVED_ENV).toBeUndefined();
expect(ctx.env.NEW_ENV).toBe('new-env-value');
});
it('emits build phase duration metrics for successful build phases', async () => {
const ctx = createTestBuildContext({ platform: Platform.IOS });
@@ -111,6 +111,37 @@ describe(runGenericJobAsync, () => {
expect(rejected).toBe(true);
});
it('passes the job hooks into the steps parser', async () => {
// @expo/steps is fully mocked here — assert the plumbing only (real
// insertion behavior is covered by the parser's own tests).
const mockWorkflow = { executeAsync: jest.fn().mockResolvedValue(undefined) };
(StepsConfigParser as unknown as jest.Mock).mockImplementation(() => ({
parseAsync: jest.fn().mockResolvedValue(mockWorkflow),
}));
mockCtx.job.hooks = { before_install_node_modules: [{ run: 'echo hi' }] };
await runGenericJobAsync(mockCtx);
expect(StepsConfigParser).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({ hooks: mockCtx.job.hooks })
);
});
it('passes hooks: undefined into the steps parser for a hook-less job (dormancy)', async () => {
const mockWorkflow = { executeAsync: jest.fn().mockResolvedValue(undefined) };
(StepsConfigParser as unknown as jest.Mock).mockImplementation(() => ({
parseAsync: jest.fn().mockResolvedValue(mockWorkflow),
}));
await runGenericJobAsync(mockCtx);
expect(StepsConfigParser).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({ hooks: undefined })
);
});
it('throws before workflow execution when expoApiV2BaseUrl is missing', async () => {
mockCtx.expoApiV2BaseUrl = undefined;
@@ -10,7 +10,10 @@ import os from 'os';
import path from 'path';
import { v4 as uuidv4 } from 'uuid';
import { createTestIosJob } from './job';
import { createMockLogger } from './logger';
import { BuildContext } from '../../context';
import { CustomBuildContext } from '../../customBuildContext';
export class MockContextProvider implements ExternalBuildContextProvider {
private _env: BuildStepEnv = {};
@@ -72,6 +75,19 @@ export function createStepContextMock({
});
}
export function createCustomBuildContextMock(): CustomBuildContext {
const ctx = new BuildContext(createTestIosJob({}), {
env: {
__API_SERVER_URL: 'http://api.expo.test',
},
logBuffer: { getLogs: () => [], getPhaseLogs: () => [] },
logger: createMockLogger(),
uploadArtifact: jest.fn(),
workingdir: '',
});
return new CustomBuildContext(ctx);
}
export function createGlobalContextMock({
logger,
skipCleanup,
+24 -39
View File
@@ -1,12 +1,10 @@
import { Android, Env, Job, Platform } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import spawn, { SpawnPromise, SpawnResult } from '@expo/turtle-spawn';
import assert from 'assert';
import spawn from '@expo/turtle-spawn';
import fs from 'fs-extra';
import path from 'path';
import { BuildContext } from '../context';
import { getParentAndDescendantProcessPidsAsync } from '../utils/processes';
export async function ensureLFLineEndingsInGradlewScript<TJob extends Job>(
ctx: BuildContext<TJob>
@@ -31,10 +29,19 @@ export async function runGradleCommand(
logger.info(`Running 'gradlew ${gradleCommand}' in ${androidDir}`);
await fs.chmod(path.join(androidDir, 'gradlew'), 0o755);
const verboseFlag = ctx.env['EAS_VERBOSE'] === '1' ? '--info' : '';
const shouldResetOOMScore =
ctx.env.EAS_BUILD_RUNNER === 'eas-build' && process.platform === 'linux';
const spawnPromise = spawn(
await spawn(
'bash',
['-c', `./gradlew ${gradleCommand} --profile ${verboseFlag}`],
[
'-c',
getGradleShellCommand({
gradleCommand,
oomScoreAdj: shouldResetOOMScore ? 0 : undefined,
verboseFlag,
}),
],
{
cwd: androidDir,
logger,
@@ -53,42 +60,20 @@ export async function runGradleCommand(
},
}
);
if (ctx.env.EAS_BUILD_RUNNER === 'eas-build' && process.platform === 'linux') {
adjustOOMScore(spawnPromise, logger);
}
await spawnPromise;
}
/**
* OOM Killer sometimes kills worker server while build is exceeding memory limits.
* `oom_score_adj` is a value between -1000 and 1000 and it defaults to 0.
* It defines which process is more likely to get killed (higher value more likely).
*
* This function sets oom_score_adj for Gradle process and all its child processes.
*/
function adjustOOMScore(spawnPromise: SpawnPromise<SpawnResult>, logger: bunyan): void {
setTimeout(
async () => {
try {
assert(spawnPromise.child.pid);
const pids = await getParentAndDescendantProcessPidsAsync(spawnPromise.child.pid);
await Promise.all(
pids.map(async (pid: number) => {
// Value 800 is just a guess here. It's probably higher than most other
// process. I didn't want to set it any higher, because I'm not sure if OOM Killer
// can start killing processes when there is still enough memory left.
const oomScoreOverride = 800;
await fs.writeFile(`/proc/${pid}/oom_score_adj`, `${oomScoreOverride}\n`);
})
);
} catch (err: any) {
logger.debug({ err, stderr: err?.stderr }, 'Failed to override oom_score_adj');
}
},
// Wait 20 seconds to make sure all child processes are started
20000
);
export function getGradleShellCommand({
gradleCommand,
oomScoreAdj,
verboseFlag,
}: {
gradleCommand: string;
oomScoreAdj?: number;
verboseFlag: string;
}): string {
const oomScoreAdjPrefix =
oomScoreAdj === undefined ? '' : `echo ${oomScoreAdj} > /proc/$$/oom_score_adj || true; `;
return `${oomScoreAdjPrefix}exec ./gradlew ${gradleCommand} --profile ${verboseFlag}`;
}
// Version envs should be set at the beginning of the build, but when building
@@ -9,6 +9,7 @@ import { restoreCredentials } from '../../android/credentials';
import { uploadEmbeddedBundleAsync } from '../../utils/expoUpdatesEmbedded';
import androidBuilder from '../android';
import { runBuilderWithHooksAsync } from '../common';
import { setupAsync } from '../../common/setup';
import {
injectConfigureVersionGradleConfig,
injectCredentialsGradleConfig,
@@ -89,6 +90,48 @@ describe(androidBuilder, () => {
);
});
it('passes the install_node_modules anchor and a hooks ref to setupAsync', async () => {
const ctx = new BuildContext(createTestAndroidJob(), {
workingdir: '/workingdir',
logBuffer: { getLogs: () => [], getPhaseLogs: () => [] },
logger: createMockLogger(),
env: { __API_SERVER_URL: 'http://api.expo.test' },
uploadArtifact: jest.fn(),
});
await androidBuilder(ctx);
expect(setupAsync).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({
wrappedAnchors: ['install_node_modules'],
jobHooksRef: expect.objectContaining({ current: null }),
})
);
});
it('drains queued hook metric uploads even when setup fails after assigning the ref', async () => {
const drainPendingMetricUploads = jest.fn();
jest.mocked(setupAsync).mockImplementationOnce(async (_ctx, { jobHooksRef }) => {
// A before-hook can queue a metric upload and then fail setup; the drain
// must still run.
jobHooksRef.current = {
customBuildContext: { drainPendingMetricUploads },
} as never;
throw new Error('setup boom');
});
const ctx = new BuildContext(createTestAndroidJob(), {
workingdir: '/workingdir',
logBuffer: { getLogs: () => [], getPhaseLogs: () => [] },
logger: createMockLogger(),
env: { __API_SERVER_URL: 'http://api.expo.test' },
uploadArtifact: jest.fn(),
});
await expect(androidBuilder(ctx)).rejects.toThrow('setup boom');
expect(drainPendingMetricUploads).toHaveBeenCalled();
});
it('injects Android version config without build credentials', async () => {
const job: Android.Job = {
...createTestAndroidJob(),
+17 -3
View File
@@ -10,11 +10,11 @@ import {
resolveGradleCommand,
runGradleCommand,
} from '../android/gradle';
import { parseGradleProfile, formatGradleProfileReport } from '../android/gradleProfile';
import { formatGradleProfileReport, parseGradleProfile } from '../android/gradleProfile';
import { Sentry } from '../sentry';
import { eagerBundleAsync, shouldUseEagerBundle } from '../common/eagerBundle';
import { prebuildAsync } from '../common/prebuild';
import { setupAsync } from '../common/setup';
import { JobHooksRef, setupAsync } from '../common/setup';
import { Artifacts, BuildContext, SkipNativeBuildError } from '../context';
import {
cacheStatsAsync,
@@ -50,7 +50,21 @@ export default async function androidBuilder(ctx: BuildContext<Android.Job>): Pr
}
async function buildAsync(ctx: BuildContext<Android.Job>): Promise<void> {
await setupAsync(ctx);
const jobHooksRef: JobHooksRef = { current: null };
try {
await buildInnerAsync(ctx, jobHooksRef);
} finally {
// Native builds have no other drain path for the hook context's queued
// step-metric uploads.
await jobHooksRef.current?.customBuildContext.drainPendingMetricUploads();
}
}
async function buildInnerAsync(
ctx: BuildContext<Android.Job>,
jobHooksRef: JobHooksRef
): Promise<void> {
await setupAsync(ctx, { wrappedAnchors: ['install_node_modules'], jobHooksRef });
const evictUsedBefore = new Date();
const workingDirectory = ctx.getReactNativeProjectDirectory();
const hasNativeCode = ctx.job.type === Workflow.GENERIC;
@@ -63,6 +63,7 @@ export async function runCustomBuildAsync(ctx: BuildContext<BuildJob>): Promise<
externalFunctions: easFunctions,
externalFunctionGroups: easFunctionGroups,
steps: ctx.job.steps,
hooks: ctx.job.hooks,
})
: new BuildConfigParser(globalContext, {
externalFunctions: easFunctions,
+17 -2
View File
@@ -8,7 +8,7 @@ import { runBuilderWithHooksAsync } from './common';
import { runCustomBuildAsync } from './custom';
import { eagerBundleAsync, shouldUseEagerBundle } from '../common/eagerBundle';
import { prebuildAsync } from '../common/prebuild';
import { setupAsync } from '../common/setup';
import { JobHooksRef, setupAsync } from '../common/setup';
import { Artifacts, BuildContext } from '../context';
import { configureXcodeProject } from '../ios/configure';
import CredentialsManager from '../ios/credentials/manager';
@@ -49,7 +49,22 @@ export default async function iosBuilder(ctx: BuildContext<Ios.Job>): Promise<Ar
}
async function buildAsync(ctx: BuildContext<Ios.Job>): Promise<void> {
await setupAsync(ctx);
const jobHooksRef: JobHooksRef = { current: null };
try {
await buildInnerAsync(ctx, jobHooksRef);
} finally {
// Native builds have no other drain path for the hook context's queued
// step-metric uploads. Must wrap the WHOLE body — iOS's existing inner
// try/finally only starts after setupAsync returns.
await jobHooksRef.current?.customBuildContext.drainPendingMetricUploads();
}
}
async function buildInnerAsync(
ctx: BuildContext<Ios.Job>,
jobHooksRef: JobHooksRef
): Promise<void> {
await setupAsync(ctx, { wrappedAnchors: ['install_node_modules'], jobHooksRef });
const hasNativeCode = ctx.job.type === Workflow.GENERIC;
const evictUsedBefore = new Date();
const credentialsManager = new CredentialsManager(ctx);
@@ -196,4 +196,67 @@ describe('easBuildInternal', () => {
expect((newJob as any).refreshAdHocProvisioningProfile).toBe(true);
});
describe('hooks retention', () => {
const logger = { info: jest.fn(), warn: jest.fn(), error: jest.fn(), child: jest.fn() } as any;
const originalHooks = { before_install_node_modules: [{ run: 'echo original' }] };
function mockRegeneratedJob(hooks?: object): void {
const internalJob = {
platform: Platform.IOS,
type: Workflow.GENERIC,
triggeredBy: 'EAS_CLI',
projectArchive: { type: ArchiveSourceType.URL, url: 'https://example.com' },
projectRootDirectory: '.',
secrets: {
buildCredentials: {
testapp: {
distributionCertificate: {
dataBase64: 'YmluYXJ5Y29udGVudDE=',
password: 'distCertPassword',
},
provisioningProfileBase64: 'MnRuZXRub2N5cmFuaWI=',
},
},
},
initiatingUserId: 'user-id',
appId: 'app-id',
...(hooks ? { hooks } : null),
};
jest.mocked(spawn).mockResolvedValue({
stdout: Buffer.from(JSON.stringify({ job: internalJob, metadata: {} })),
stderr: Buffer.from(''),
} as any);
}
function runWith(jobHooks?: object): Promise<{ newJob: BuildJob }> {
const job = {
platform: Platform.IOS,
buildProfile: 'preview',
appId: 'app-id',
initiatingUserId: 'user-id',
secrets: { robotAccessToken: 'token' },
...(jobHooks ? { hooks: jobHooks } : null),
} as unknown as BuildJob;
return runEasBuildInternalAsync({ job, logger, env: {}, cwd: '/tmp/project' });
}
it('keeps the original hooks when the regenerated job has none', async () => {
mockRegeneratedJob();
const { newJob } = await runWith(originalHooks);
expect((newJob as any).hooks).toEqual(originalHooks);
});
it('lets the original hooks win over hooks in the regenerated job', async () => {
mockRegeneratedJob({ after_install_node_modules: [{ run: 'echo from-eas-json' }] });
const { newJob } = await runWith(originalHooks);
expect((newJob as any).hooks).toEqual(originalHooks);
});
it('carries no hooks when the original had none, even if the regenerated job adds some', async () => {
mockRegeneratedJob({ before_install_node_modules: [{ run: 'echo from-eas-json' }] });
const { newJob } = await runWith(undefined);
expect((newJob as any).hooks).toBeUndefined();
});
});
});
@@ -0,0 +1,164 @@
import { BuildPhase, BuildTrigger, Hooks, Ios } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import { executeHookStepsAsync } from '@expo/steps';
import { createTestIosJob } from '../../__tests__/utils/job';
import { BuildContext } from '../../context';
import { runHookableBuildPhaseAsync } from '../hookableBuildPhase';
import { ParsedJobHooks, parseJobHooksAsync } from '../jobHooks';
// The primitive's step-execution rules (failure()/success(), no-if continuation,
// metric emission) are covered in @expo/steps. Here we mock it to unit-test the
// wrapper's own orchestration: ordering, error precedence, and the env bridge.
jest.mock('@expo/steps', () => ({
...jest.requireActual('@expo/steps'),
executeHookStepsAsync: jest.fn(),
}));
const executeHookStepsMock = jest.mocked(executeHookStepsAsync);
async function setup(hooks: Hooks | undefined): Promise<{
ctx: BuildContext<Ios.Job>;
hooks: ParsedJobHooks | null;
}> {
const logger = {
info: jest.fn(),
debug: jest.fn(),
error: jest.fn(),
warn: jest.fn(),
child: jest.fn(() => logger),
} as unknown as bunyan;
const ctx = new BuildContext<Ios.Job>(
{ ...createTestIosJob({ triggeredBy: BuildTrigger.GIT_BASED_INTEGRATION }), hooks },
{
env: { __API_SERVER_URL: 'http://api.expo.test' },
logBuffer: { getLogs: () => [], getPhaseLogs: () => [] },
logger,
uploadArtifact: jest.fn(),
workingdir: '/tmp/wd',
}
);
jest
.spyOn(ctx, 'runBuildPhase')
.mockImplementation((_phase: BuildPhase, fn: () => Promise<unknown>) => fn() as never);
return { ctx, hooks: await parseJobHooksAsync(ctx, ['install_node_modules']) };
}
const runArgs = <TJob extends Ios.Job>(
ctx: BuildContext<TJob>,
hooks: ParsedJobHooks | null,
fn: () => Promise<unknown>
): Parameters<typeof runHookableBuildPhaseAsync>[0] => ({
ctx,
hooks,
buildPhase: BuildPhase.INSTALL_DEPENDENCIES,
anchor: 'install_node_modules',
fn,
});
describe(runHookableBuildPhaseAsync, () => {
beforeEach(() => {
executeHookStepsMock.mockReset();
executeHookStepsMock.mockResolvedValue({ failedLocally: false, firstError: undefined });
});
it('runs the phase once with no hook execution when there are no hooks', async () => {
const { ctx, hooks } = await setup(undefined);
const fn = jest.fn().mockResolvedValue('ok');
expect(await runHookableBuildPhaseAsync(runArgs(ctx, hooks, fn))).toBe('ok');
expect(ctx.runBuildPhase).toHaveBeenCalledTimes(1);
expect(executeHookStepsMock).not.toHaveBeenCalled();
});
it('runs before -> phase -> after in order and returns the phase result', async () => {
const { ctx, hooks } = await setup({
before_install_node_modules: [{ run: 'echo before' }],
after_install_node_modules: [{ run: 'echo after' }],
});
const order: string[] = [];
executeHookStepsMock.mockImplementation(async (_ctx, _entries, opts) => {
order.push(opts.timing);
return { failedLocally: false, firstError: undefined };
});
const fn = jest.fn().mockImplementation(async () => {
order.push('phase');
return 'result';
});
expect(await runHookableBuildPhaseAsync(runArgs(ctx, hooks, fn))).toBe('result');
expect(order).toEqual(['before', 'phase', 'after']);
expect(executeHookStepsMock).toHaveBeenLastCalledWith(hooks!.globalContext, expect.anything(), {
anchor: 'install_node_modules',
timing: 'after',
anchorResult: 'success',
});
});
it('skips the phase and after-hooks when a before-hook fails', async () => {
const { ctx, hooks } = await setup({
before_install_node_modules: [{ run: 'echo before' }],
after_install_node_modules: [{ run: 'echo after' }],
});
executeHookStepsMock.mockResolvedValue({
failedLocally: true,
firstError: new Error('before boom'),
});
const fn = jest.fn().mockResolvedValue('ok');
await expect(runHookableBuildPhaseAsync(runArgs(ctx, hooks, fn))).rejects.toThrow(
'Hook "before_install_node_modules" failed'
);
expect(fn).not.toHaveBeenCalled();
expect(executeHookStepsMock).toHaveBeenCalledTimes(1);
});
it('throws the after-hook error when the phase succeeded but an after-hook fails', async () => {
const { ctx, hooks } = await setup({
after_install_node_modules: [{ run: 'echo after' }],
});
executeHookStepsMock.mockResolvedValue({
failedLocally: true,
firstError: new Error('after boom'),
});
await expect(
runHookableBuildPhaseAsync(runArgs(ctx, hooks, jest.fn().mockResolvedValue('ok')))
).rejects.toThrow('Hook "after_install_node_modules" failed');
});
it('lets the phase error win over a failing after-hook, marking the context failed', async () => {
const { ctx, hooks } = await setup({
after_install_node_modules: [{ run: 'echo after' }],
});
const markAsFailed = jest.spyOn(hooks!.globalContext, 'markAsFailed');
executeHookStepsMock.mockResolvedValue({
failedLocally: true,
firstError: new Error('after boom'),
});
await expect(
runHookableBuildPhaseAsync(
runArgs(ctx, hooks, jest.fn().mockRejectedValue(new Error('phase boom')))
)
).rejects.toThrow('phase boom');
// after ran against the failed anchor
expect(markAsFailed).toHaveBeenCalled();
expect(executeHookStepsMock).toHaveBeenLastCalledWith(hooks!.globalContext, expect.anything(), {
anchor: 'install_node_modules',
timing: 'after',
anchorResult: 'failed',
});
});
it('refreshes the hook context before and writes env back after each hook', async () => {
const { ctx, hooks } = await setup({
before_install_node_modules: [{ run: 'echo before' }],
});
const forwardRefresh = jest.spyOn(hooks!.customBuildContext, 'updateEnv');
const writeBack = jest.spyOn(ctx, 'updateEnv');
// Captured before the run: the reverse write-back replaces ctx.env, so
// comparing the forward-refresh arg against the post-run ctx.env would drift.
const envBeforeRun = ctx.env;
await runHookableBuildPhaseAsync(runArgs(ctx, hooks, jest.fn().mockResolvedValue('ok')));
expect(forwardRefresh).toHaveBeenCalledWith(envBeforeRun);
// The job/metadata snapshot is synced onto the hook context before the hook.
expect(hooks!.customBuildContext.job).toBe(ctx.job);
expect(hooks!.customBuildContext.metadata).toBe(ctx.metadata);
expect(writeBack).toHaveBeenCalled();
});
});
@@ -0,0 +1,139 @@
import { BuildTrigger, ErrorCode, Hooks, Ios, UserError } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import { createTestIosJob } from '../../__tests__/utils/job';
import { BuildContext } from '../../context';
import { parseJobHooksAsync } from '../jobHooks';
const INSTALL: ['install_node_modules'] = ['install_node_modules'];
function createCtx(hooks: Hooks | undefined): {
ctx: BuildContext<Ios.Job>;
warn: jest.Mock;
} {
const warn = jest.fn();
const logger = {
info: jest.fn(),
debug: jest.fn(),
error: jest.fn(),
warn,
child: jest.fn(() => logger),
} as unknown as bunyan;
const ctx = new BuildContext<Ios.Job>(
{ ...createTestIosJob({ triggeredBy: BuildTrigger.GIT_BASED_INTEGRATION }), hooks },
{
env: { __API_SERVER_URL: 'http://api.expo.test' },
logBuffer: { getLogs: () => [], getPhaseLogs: () => [] },
logger,
uploadArtifact: jest.fn(),
workingdir: '/tmp/wd',
}
);
return { ctx, warn };
}
describe(parseJobHooksAsync, () => {
it('returns null when the job has no hooks', async () => {
const { ctx } = createCtx(undefined);
expect(await parseJobHooksAsync(ctx, INSTALL)).toBeNull();
});
it('parses a wrapped-anchor hook into one entry in a checked-out context', async () => {
const { ctx } = createCtx({
before_install_node_modules: [{ run: 'echo hi' }],
});
const parsed = await parseJobHooksAsync(ctx, INSTALL);
expect(parsed).not.toBeNull();
expect(parsed!.hookEntriesByKey.before_install_node_modules).toHaveLength(1);
expect(parsed!.globalContext.wasCheckedOut()).toBe(true);
});
it('throws a shape error for a malformed step, tagged with the hooks error code', async () => {
const { ctx } = createCtx({
before_install_node_modules: [{} as never],
});
await expect(parseJobHooksAsync(ctx, INSTALL)).rejects.toThrow(
'Invalid steps in "hooks.before_install_node_modules"'
);
await expect(parseJobHooksAsync(ctx, INSTALL)).rejects.toMatchObject({
errorCode: ErrorCode.HOOKS_ERROR,
});
});
it('treats an unknown hook key as inert and warns', async () => {
const { ctx, warn } = createCtx({
before_bogus: [{ run: 'echo hi' }],
});
const parsed = await parseJobHooksAsync(ctx, INSTALL);
expect(parsed!.hookEntriesByKey).toEqual({});
expect(warn).toHaveBeenCalledWith(expect.stringContaining('unknown hook key "before_bogus"'));
});
it('shape-checks a registered-but-unwrapped key, then warns and skips it', async () => {
const { ctx, warn } = createCtx({
before_submit: [{ run: 'echo hi' }],
});
const parsed = await parseJobHooksAsync(ctx, INSTALL);
expect(parsed!.hookEntriesByKey.before_submit).toBeUndefined();
expect(warn).toHaveBeenCalledWith(expect.stringContaining('hooks.before_submit'));
});
it('shape-validates a registered-but-unwrapped key even though it is skipped', async () => {
const { ctx } = createCtx({
before_submit: [{} as never],
});
await expect(parseJobHooksAsync(ctx, INSTALL)).rejects.toThrow(
'Invalid steps in "hooks.before_submit"'
);
});
it('accepts an empty-array opt-out without validating or constructing it', async () => {
const { ctx } = createCtx({
before_install_node_modules: [],
});
const parsed = await parseJobHooksAsync(ctx, INSTALL);
expect(parsed!.hookEntriesByKey.before_install_node_modules).toBeUndefined();
});
it('errors on an unknown function under a wrapped key (construction, not shape)', async () => {
const { ctx } = createCtx({
before_install_node_modules: [{ uses: 'eas/nonexistent' }],
});
await expect(parseJobHooksAsync(ctx, INSTALL)).rejects.toThrow(
'Failed to parse hooks.before_install_node_modules'
);
});
it('parses a function group in an after-hook (no after-hook fence)', async () => {
const { ctx } = createCtx({
after_install_node_modules: [{ uses: 'eas/checkout' }],
});
const parsed = await parseJobHooksAsync(ctx, INSTALL);
expect(parsed!.hookEntriesByKey.after_install_node_modules).toHaveLength(1);
});
it('shares one global context across the before and after entries of an anchor', async () => {
const { ctx } = createCtx({
before_install_node_modules: [{ run: 'echo before' }],
after_install_node_modules: [{ run: 'echo after' }],
});
const parsed = await parseJobHooksAsync(ctx, INSTALL);
const before = parsed!.hookEntriesByKey.before_install_node_modules![0];
const after = parsed!.hookEntriesByKey.after_install_node_modules![0];
expect(before.steps[0].ctx.global).toBe(parsed!.globalContext);
expect(after.steps[0].ctx.global).toBe(parsed!.globalContext);
});
it('wraps a cross-key duplicate step id in the aggregate error', async () => {
const { ctx } = createCtx({
before_install_node_modules: [{ id: 'dup', run: 'echo a' }],
after_install_node_modules: [{ id: 'dup', run: 'echo b' }],
});
const result = parseJobHooksAsync(ctx, INSTALL);
await expect(result).rejects.toThrow(UserError);
await expect(result).rejects.toMatchObject({
errorCode: ErrorCode.HOOKS_ERROR,
message: expect.stringContaining("The job's hooks are invalid"),
});
});
});
@@ -0,0 +1,131 @@
import { Android, BuildTrigger } from '@expo/eas-build-job';
import { createTestAndroidJob } from '../../__tests__/utils/job';
import { createMockLogger } from '../../__tests__/utils/logger';
import { BuildContext } from '../../context';
import { resolveEnvFromBuildProfileAsync, runEasBuildInternalAsync } from '../easBuildInternal';
import { runHookableBuildPhaseAsync } from '../hookableBuildPhase';
import { JobHooksRef, setupAsync } from '../setup';
import { ParsedJobHooks, parseJobHooksAsync } from '../jobHooks';
import { prepareProjectSourcesAsync } from '../projectSources';
import { runHookIfPresent } from '../../utils/hooks';
jest.mock('../projectSources', () => ({ prepareProjectSourcesAsync: jest.fn() }));
jest.mock('../easBuildInternal', () => ({
resolveEnvFromBuildProfileAsync: jest.fn(),
runEasBuildInternalAsync: jest.fn(),
}));
jest.mock('../hookableBuildPhase', () => ({ runHookableBuildPhaseAsync: jest.fn() }));
jest.mock('../jobHooks', () => ({ parseJobHooksAsync: jest.fn() }));
jest.mock('../../utils/npmrc', () => ({ setUpNpmrcAsync: jest.fn() }));
jest.mock('../../utils/hooks', () => ({
Hook: { PRE_INSTALL: 'PRE_INSTALL' },
runHookIfPresent: jest.fn(),
}));
jest.mock('../../utils/project', () => ({
readAndLogPackageJson: jest.fn(() => ({})),
readEasJsonContents: jest.fn(() => '{}'),
readPackageJson: jest.fn(() => ({})),
}));
jest.mock('../../ios/xcodeEnv', () => ({ deleteXcodeEnvLocalIfExistsAsync: jest.fn() }));
// A sentinel the parse mock returns; the install wrapper must receive exactly it.
const PARSED = { sentinel: true } as unknown as ParsedJobHooks;
describe(setupAsync, () => {
let order: string[];
function createCtx(triggeredBy: BuildTrigger): BuildContext<Android.Job> {
const ctx = new BuildContext<Android.Job>(
{ ...createTestAndroidJob(), triggeredBy },
{
env: { __API_SERVER_URL: 'http://api.expo.test' },
logBuffer: { getLogs: () => [], getPhaseLogs: () => [] },
logger: createMockLogger(),
uploadArtifact: jest.fn(),
workingdir: '/workingdir',
}
);
// Run each phase closure inline; skip the phase markers.
jest
.spyOn(ctx, 'runBuildPhase')
.mockImplementation((_phase, fn: () => Promise<unknown>) => fn() as never);
// READ_APP_CONFIG reads this; a minimal config passes validateAppConfigAsync
// and the EAS_BUILD_INTERNAL bundle-id/package checks.
jest.spyOn(ctx, 'appConfig', 'get').mockReturnValue(
Promise.resolve({
android: { package: 'com.test' },
ios: { bundleIdentifier: 'com.test' },
} as never)
);
return ctx;
}
beforeEach(() => {
jest.clearAllMocks();
order = [];
jest.mocked(prepareProjectSourcesAsync).mockImplementation(async () => {
order.push('prepare-project');
return { handled: false };
});
jest.mocked(resolveEnvFromBuildProfileAsync).mockImplementation(async () => {
order.push('profile-env');
return {};
});
jest.mocked(runHookIfPresent).mockImplementation(async () => {
order.push('pre-install-hook');
});
jest.mocked(parseJobHooksAsync).mockImplementation(async () => {
order.push('parse-hooks');
return PARSED;
});
jest.mocked(runHookableBuildPhaseAsync).mockImplementation(async () => {
// Record the wrapper call; deliberately do NOT run its `fn` so the install
// internals stay out of the ordering test.
order.push('install-wrapper');
return undefined as never;
});
jest.mocked(runEasBuildInternalAsync).mockImplementation(async ({ job }) => ({
newJob: job,
newMetadata: {} as never,
}));
});
it('parses hooks after project prep + profile-env + pre-install hook, and before the install wrapper', async () => {
const jobHooksRef: JobHooksRef = { current: null };
await setupAsync(createCtx(BuildTrigger.GIT_BASED_INTEGRATION), {
wrappedAnchors: ['install_node_modules'],
jobHooksRef,
});
const parseAt = order.indexOf('parse-hooks');
expect(order.indexOf('prepare-project')).toBeLessThan(parseAt);
expect(order.indexOf('profile-env')).toBeLessThan(parseAt);
expect(order.indexOf('pre-install-hook')).toBeLessThan(parseAt);
expect(parseAt).toBeLessThan(order.indexOf('install-wrapper'));
});
it('parses hooks unconditionally on a non-git-based build (never parked in the git-only branch)', async () => {
const jobHooksRef: JobHooksRef = { current: null };
await setupAsync(createCtx(BuildTrigger.EAS_CLI), {
wrappedAnchors: ['install_node_modules'],
jobHooksRef,
});
expect(parseJobHooksAsync).toHaveBeenCalledTimes(1);
// The profile-env block is git-only; the parse must not be coupled to it.
expect(resolveEnvFromBuildProfileAsync).not.toHaveBeenCalled();
expect(order.indexOf('parse-hooks')).toBeLessThan(order.indexOf('install-wrapper'));
});
it('publishes the parsed hooks to the ref before the install wrapper runs', async () => {
const jobHooksRef: JobHooksRef = { current: null };
await setupAsync(createCtx(BuildTrigger.GIT_BASED_INTEGRATION), {
wrappedAnchors: ['install_node_modules'],
jobHooksRef,
});
expect(jobHooksRef.current).toBe(PARSED);
// The wrapper received the same parsed hooks the ref holds.
expect(runHookableBuildPhaseAsync).toHaveBeenCalledWith(
expect.objectContaining({ hooks: PARSED, anchor: 'install_node_modules' })
);
});
});
@@ -152,6 +152,9 @@ function validateEasBuildInternalResult<TJob extends BuildJob>({
// We want to retain values that we have set on the job.
appId: oldJob.appId,
initiatingUserId: oldJob.initiatingUserId,
// Hooks are resolved server-side at submit; the mid-build eas.json
// regeneration must not add, drop, or change them (the original wins).
hooks: oldJob.hooks,
...(oldJob.platform === Platform.IOS && oldJob.refreshAdHocProvisioningProfile === true
? { refreshAdHocProvisioningProfile: true }
: null),
@@ -0,0 +1,116 @@
import {
BuildJob,
BuildPhase,
ErrorCode,
HookAnchorId,
HookKey,
UserError,
} from '@expo/eas-build-job';
import { executeHookStepsAsync } from '@expo/steps';
import { BuildContext } from '../context';
import { ParsedJobHooks } from './jobHooks';
/**
* Runs a native build phase with its `before_<anchor>` / `after_<anchor>` hooks
* wrapped around it: before-entries -> phase (in a try) -> after-entries, with
* after-on-failure guaranteed. A failing before sequence skips the phase AND the
* after-hooks (steps-engine parity). The phase error always wins over an
* after-hook error. Passthrough (no hooks or no keys for this anchor) runs the
* phase exactly once with zero overhead.
*/
export async function runHookableBuildPhaseAsync<TJob extends BuildJob, T>({
ctx,
hooks,
buildPhase,
anchor,
fn,
}: {
ctx: BuildContext<TJob>;
hooks: ParsedJobHooks | null;
buildPhase: BuildPhase;
anchor: HookAnchorId;
fn: () => Promise<T>;
}): Promise<T> {
// runHookEntriesAsync no-ops when the key has no entries.
if (hooks) {
await runHookEntriesAsync({ ctx, hooks, timing: 'before', anchor });
}
let result: T;
try {
result = await ctx.runBuildPhase(buildPhase, fn);
} catch (phaseError) {
if (hooks) {
try {
await runHookEntriesAsync({ ctx, hooks, timing: 'after', anchor, anchorFailed: true });
} catch (hookError) {
// The phase error wins; still surface the after-hook failure in the log.
ctx.logger.error({ err: hookError }, `after_${anchor} hook failed`);
}
}
throw phaseError;
}
if (hooks) {
await runHookEntriesAsync({ ctx, hooks, timing: 'after', anchor, anchorFailed: false });
}
return result;
}
async function runHookEntriesAsync<TJob extends BuildJob>({
ctx,
hooks,
timing,
anchor,
anchorFailed = false,
}: {
ctx: BuildContext<TJob>;
hooks: ParsedJobHooks;
timing: 'before' | 'after';
anchor: HookAnchorId;
anchorFailed?: boolean;
}): Promise<void> {
const hookKey: HookKey = `${timing}_${anchor}`;
const entries = hooks.hookEntriesByKey[hookKey];
if (!entries?.length) {
return;
}
// Refresh the hook context from the live BuildContext before running: env
// moves as the build runs (set-env written back below; profile env), and
// job/metadata are replaced by eas build:internal.
hooks.customBuildContext.updateEnv(ctx.env);
// Overwrite the job/metadata snapshot; NOT updateJobInformation, whose
// delta-merge would duplicate environmentSecrets on each refresh.
hooks.customBuildContext.job = ctx.job;
hooks.customBuildContext.metadata = ctx.metadata;
if (timing === 'after' && anchorFailed) {
// Mirror the engine: the anchor's failure is marked on the shared context so
// `failure()` / `success()` on after-hook steps read the phase outcome.
hooks.globalContext.markAsFailed();
}
const { failedLocally, firstError } = await executeHookStepsAsync(hooks.globalContext, entries, {
anchor,
timing,
...(timing === 'after' ? { anchorResult: anchorFailed ? 'failed' : 'success' } : null),
});
// Reverse write-back BEFORE propagating a failure: env set by the entries that
// did run must reach the build context even when a later entry failed, so the
// native phases (and the outer lifecycle hooks) see it. Hook-carrying builds
// are git-based, so updateEnv's gate does not fire here.
ctx.updateEnv(hooks.globalContext.env);
if (failedLocally) {
throw new UserError(
ErrorCode.HOOKS_ERROR,
`Hook "${hookKey}" failed: ${
firstError instanceof Error ? firstError.message : String(firstError)
}. Check the failing hook step's logs above.`,
{ cause: firstError }
);
}
}
+140
View File
@@ -0,0 +1,140 @@
import {
BuildJob,
ErrorCode,
HookAnchorId,
HookKey,
UserError,
parseHookKey,
validateSteps,
} from '@expo/eas-build-job';
import {
BuildStep,
BuildStepGlobalContext,
HookEntry,
constructHookEntriesAsync,
validateHookStepsAsync,
} from '@expo/steps';
import { BuildContext } from '../context';
import { CustomBuildContext } from '../customBuildContext';
import { getEasFunctionGroups } from '../steps/easFunctionGroups';
import { getEasFunctions } from '../steps/easFunctions';
export interface ParsedJobHooks {
globalContext: BuildStepGlobalContext;
// The hook runner re-syncs this from the live BuildContext and runs the
// hooks against it.
customBuildContext: CustomBuildContext;
// One HookEntry per authored hook step (a `uses:` group's whole expansion
// stays inside one entry). The engine evaluates each entry's `if:` once and
// isolates within-entry failures, so the boundary must survive to execution.
hookEntriesByKey: Partial<Record<HookKey, HookEntry[]>>;
}
/**
* Parses the native job's `hooks` into engine-ready entries, once, at setup
* time. Shape-validates every registered key (reachable or not); constructs
* entries only for the anchors this build actually wraps. Returns null when the
* job declares no hooks.
*/
export async function parseJobHooksAsync<TJob extends BuildJob>(
ctx: BuildContext<TJob>,
wrappedAnchors: readonly HookAnchorId[]
): Promise<ParsedJobHooks | null> {
const hooks = ctx.job.hooks;
if (!hooks || Object.keys(hooks).length === 0) {
return null;
}
const customBuildContext = new CustomBuildContext(ctx);
const globalContext = new BuildStepGlobalContext(customBuildContext, false);
// Hook steps run in the checked-out project directory, not the pre-checkout
// target directory the global context defaults to.
globalContext.markAsCheckedOut(ctx.logger);
const externalFunctions = getEasFunctions(customBuildContext);
const externalFunctionGroups = getEasFunctionGroups(customBuildContext);
// Shape-validate every REGISTERED key, reachable or not (steps-world parity:
// a malformed `before_submit` fails a steps build, so it fails here too), and
// warn on keys this build won't run. An unknown key is inert — a key newer
// than this worker must be ignored, never fail the job.
for (const [key, steps] of Object.entries(hooks)) {
const parsed = parseHookKey(key);
if (parsed === null) {
ctx.logger.warn(`Ignoring unknown hook key "${key}".`);
continue;
}
if (Array.isArray(steps) && steps.length === 0) {
// `[]` is an explicit opt-out; validateSteps requires >= 1 step.
continue;
}
try {
validateSteps(steps);
} catch (err) {
throw new UserError(
ErrorCode.HOOKS_ERROR,
`Invalid steps in "hooks.${key}": ${err instanceof Error ? err.message : String(err)}`,
{ cause: err }
);
}
// Registered but not wrapped by this build: shape-checked above, then
// skipped. Function existence is deliberately NOT checked (that happens only
// during construction) — an unknown `uses:` under an unwrapped key warns
// rather than failing the build.
if (!wrappedAnchors.includes(parsed.anchorId)) {
ctx.logger.warn(
`Ignoring "hooks.${key}": this build does not run the "${parsed.anchorId}" step.`
);
}
}
// Construct entries for the wrapped anchors in EXECUTION order (per anchor,
// before_ then after_) so generated step ids and output references follow the
// order steps actually run. All entries share one globalContext, so env and
// outputs accumulate across keys.
const hookEntriesByKey: Partial<Record<HookKey, HookEntry[]>> = {};
const orderedSteps: BuildStep[] = [];
for (const anchor of wrappedAnchors) {
for (const side of ['before', 'after'] as const) {
const key: HookKey = `${side}_${anchor}`;
const steps = hooks[key];
if (!Array.isArray(steps) || steps.length === 0) {
continue;
}
let entries: HookEntry[];
try {
entries = await constructHookEntriesAsync(globalContext, steps, {
externalFunctions,
externalFunctionGroups,
});
} catch (err) {
throw new UserError(
ErrorCode.HOOKS_ERROR,
`Failed to parse hooks.${key}: ${err instanceof Error ? err.message : String(err)}. Hooks use the same syntax as workflow steps.`,
{ cause: err }
);
}
hookEntriesByKey[key] = entries;
for (const entry of entries) {
orderedSteps.push(...entry.steps);
}
}
}
// One aggregate pass over every constructed step in execution order: unique
// ids, output references, and platform allowance. Per-key validation misses
// cross-key id collisions and would reject a legal after -> before output
// reference as a future reference.
try {
await validateHookStepsAsync(globalContext, orderedSteps);
} catch (err) {
throw new UserError(
ErrorCode.HOOKS_ERROR,
`The job's hooks are invalid: ${err instanceof Error ? err.message : String(err)}. Fix the reported step ids or output references.`,
{ cause: err }
);
}
return { globalContext, customBuildContext, hookEntriesByKey };
}
+61 -25
View File
@@ -3,6 +3,7 @@ import {
BuildJob,
BuildPhase,
BuildTrigger,
HookAnchorId,
Ios,
Job,
Platform,
@@ -14,7 +15,9 @@ import nullthrows from 'nullthrows';
import path from 'path';
import { resolveEnvFromBuildProfileAsync, runEasBuildInternalAsync } from './easBuildInternal';
import { runHookableBuildPhaseAsync } from './hookableBuildPhase';
import { installDependenciesAsync, resolvePackagerDir } from './installDependencies';
import { ParsedJobHooks, parseJobHooksAsync } from './jobHooks';
import { prepareProjectSourcesAsync } from './projectSources';
import { BuildContext } from '../context';
import { deleteXcodeEnvLocalIfExistsAsync } from '../ios/xcodeEnv';
@@ -26,7 +29,7 @@ import {
shouldUseFrozenLockfile,
} from '../utils/packageManager';
import { getParentAndDescendantProcessPidsAsync } from '../utils/processes';
import { readAndLogPackageJson, readEasJsonContents } from '../utils/project';
import { readAndLogPackageJson, readEasJsonContents, readPackageJson } from '../utils/project';
import { retryAsync } from '../utils/retry';
const MAX_EXPO_DOCTOR_TIMEOUT_MS = 30 * 1000;
@@ -36,7 +39,22 @@ const INSTALL_DEPENDENCIES_KILL_TIMEOUT_MS = 30 * 60 * 1000;
class DoctorTimeoutError extends Error {}
class InstallDependenciesTimeoutError extends Error {}
export async function setupAsync<TJob extends BuildJob>(ctx: BuildContext<TJob>): Promise<void> {
/**
* Mutable holder the builder passes into setupAsync. setupAsync assigns
* `current` the moment hooks are parsed (before any hook runs), so the builder's
* finally can drain queued metric uploads even if setup fails partway through.
*/
export interface JobHooksRef {
current: ParsedJobHooks | null;
}
export async function setupAsync<TJob extends BuildJob>(
ctx: BuildContext<TJob>,
{
wrappedAnchors,
jobHooksRef,
}: { wrappedAnchors: readonly HookAnchorId[]; jobHooksRef: JobHooksRef }
): Promise<void> {
await ctx.runBuildPhase(BuildPhase.PREPARE_PROJECT, async () => {
await retryAsync(
async () => {
@@ -81,32 +99,48 @@ export async function setupAsync<TJob extends BuildJob>(ctx: BuildContext<TJob>)
}
});
const packageJson = await ctx.runBuildPhase(BuildPhase.READ_PACKAGE_JSON, async () => {
return readAndLogPackageJson(ctx.logger, ctx.getReactNativeProjectDirectory());
await ctx.runBuildPhase(BuildPhase.READ_PACKAGE_JSON, async () => {
readAndLogPackageJson(ctx.logger, ctx.getReactNativeProjectDirectory());
});
await ctx.runBuildPhase(BuildPhase.INSTALL_DEPENDENCIES, async () => {
const expoVersion =
ctx.metadata?.sdkVersion ??
getPackageVersionFromPackageJson({
packageJson,
packageName: 'expo',
});
// Parse hooks unconditionally, after every pre-install env mutation
// (PREPARE_PROJECT, the git-based profile-env block, PRE_INSTALL_HOOK) and
// before the first wrapped phase.
jobHooksRef.current = await parseJobHooksAsync(ctx, wrappedAnchors);
const reactNativeVersion =
ctx.metadata?.reactNativeVersion ??
getPackageVersionFromPackageJson({
packageJson,
packageName: 'react-native',
});
await runHookableBuildPhaseAsync({
ctx,
hooks: jobHooksRef.current,
buildPhase: BuildPhase.INSTALL_DEPENDENCIES,
anchor: 'install_node_modules',
fn: async () => {
// Read package.json fresh here rather than reusing the READ_PACKAGE_JSON
// capture: a before_install_node_modules hook may have patched it, and the
// steps-world install already reads it fresh. Silent read — READ_PACKAGE_JSON
// stays the only logging site.
const freshPackageJson = readPackageJson(ctx.getReactNativeProjectDirectory());
const expoVersion =
ctx.metadata?.sdkVersion ??
getPackageVersionFromPackageJson({
packageJson: freshPackageJson,
packageName: 'expo',
});
await runInstallDependenciesAsync(ctx, {
useFrozenLockfile: shouldUseFrozenLockfile({
env: ctx.env,
sdkVersion: expoVersion,
reactNativeVersion,
}),
});
const reactNativeVersion =
ctx.metadata?.reactNativeVersion ??
getPackageVersionFromPackageJson({
packageJson: freshPackageJson,
packageName: 'react-native',
});
await runInstallDependenciesAsync(ctx, {
useFrozenLockfile: shouldUseFrozenLockfile({
env: ctx.env,
sdkVersion: expoVersion,
reactNativeVersion,
}),
});
},
});
await ctx.runBuildPhase(BuildPhase.READ_APP_CONFIG, async () => {
@@ -140,7 +174,9 @@ export async function setupAsync<TJob extends BuildJob>(ctx: BuildContext<TJob>)
});
}
const hasExpoPackage = !!packageJson.dependencies?.expo;
// Read fresh: a before_install_node_modules hook may have added or removed
// the expo dependency.
const hasExpoPackage = !!readPackageJson(ctx.getReactNativeProjectDirectory()).dependencies?.expo;
if (!ctx.env.EAS_BUILD_DISABLE_EXPO_DOCTOR_STEP && hasExpoPackage) {
await ctx.runBuildPhase(BuildPhase.RUN_EXPO_DOCTOR, async () => {
try {
+2 -1
View File
@@ -247,9 +247,10 @@ export class BuildContext<TJob extends Job = Job> {
'Updating environment variables is only allowed when build was triggered by a git-based integration.'
);
}
this._env = {
...env,
...this._env,
...env,
__EAS_BUILD_ENVS_DIR: this.buildEnvsDirectory,
};
this._env.PATH = this._env.PATH
+11 -1
View File
@@ -9,12 +9,18 @@ import {
StaticJobInterpolationContext,
} from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import { BuildRuntimePlatform, ExternalBuildContextProvider, StepMetric } from '@expo/steps';
import {
BuildRuntimePlatform,
ExternalBuildContextProvider,
StepMetric,
WorkflowHookMetric,
} from '@expo/steps';
import { Client } from '@urql/core';
import assert from 'assert';
import path from 'path';
import { ArtifactToUpload, BuildContext } from './context';
import { reportWorkflowHookMetricToDatadog } from './utils/hookMetrics';
import { uploadStepMetricsToWwwAsync } from './utils/stepMetrics';
const platformToBuildRuntimePlatform: Record<Platform, BuildRuntimePlatform> = {
@@ -165,6 +171,10 @@ export class CustomBuildContext<TJob extends Job = Job> implements ExternalBuild
this.pendingMetricUploads.push(p);
}
public reportWorkflowHookMetric(metric: WorkflowHookMetric): void {
reportWorkflowHookMetricToDatadog(metric);
}
public async drainPendingMetricUploads(): Promise<void> {
await Promise.allSettled(this.pendingMetricUploads);
}
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.9 KiB

@@ -1,13 +0,0 @@
{
"type": "service_account",
"project_id": "mock-project",
"private_key_id": "mock-private-key-id",
"private_key": "-----BEGIN RSA PRIVATE KEY-----\nMIICXQIBAAKBgQCMk5Qbd5E8RQeP+2vObJVJVnXWXfW/OStxkAYe7r+KVXzHyLYU\nD6Ejt94OiC3X2+tqrMIZgNAUR3VF5tSiP57Il0iWHSspmNIHz7cW6/FHiQsvN2oJ\n8Fzz03vK/FfaU50QWO+BwRnixJ0xG1kSX3MxNyRZ5omGJgcwx0l6Jt878QIDAQAB\nAoGAcGZbqO6ceNU068H6/A1D/GSeSa3NHX2np+ChlEAPdJtP7yojs7yfekC801+9\nT/gurpe9hsjBF0XflemwIJ6/5KucWHrcp2cCkdBHtkK6ScHeIVl3DcOYYhxqJrAA\nYiyGNh1+pf3FMPFBcPJ04pQT4PqOif61/W3p8eJFUke5x7UCQQDoyJ427wrUCZb+\nj2GD5uVyxQa4OJxDbRFriK5LTdwLeFeOmQhoo6kaGywILn6UtlPTUJ38ROA4Jv58\nEJf/jssDAkEAmpi/JkWlOzSM9mnb9blt/fH/DRKu9z0JPvl47V5YAzBhUtUMK15g\nSbEl+AJZyaklpJcgv6ShiGY0PLdoY7oQ+wJBAMF6VX4lKqPYIKcN9ygRlm3Q8ufV\nLZQhKBRvsyYl3Zmu+V8tNL78IEXxhaR7OHxUGtINNHKDsLUbO/NUO3GOdo8CQQCV\nV0sIIK0+JUSq4ZYvqKI9d6Fnso28noSpBfuwabvh0MGjb9Viq7eeWHeSPksYSMLp\nXWiwWMwGZJy/rnk0JVEzAkADiiARp70BZgRB8ogBx+DGTravfhPP64eFMHJnUgAE\nmDKlj3v0x3OZfdUz6Us5hfy3WqFyYbcCGLf7joVVZAEm\n-----END RSA PRIVATE KEY-----",
"client_email": "mock-client-email",
"client_id": "mock-client-id",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/mock-client-email",
"universe_domain": "googleapis.com"
}
-210
View File
@@ -1,210 +0,0 @@
import { CreateWriteStreamOptions, File, Storage } from '@google-cloud/storage';
import fs from 'fs-extra';
import fetch, { FetchError } from 'node-fetch';
import stream from 'node:stream';
import { promisify } from 'node:util';
import { Readable } from 'stream';
import { URL } from 'url';
import { RetryOptions, retryOnGCSUploadFailure } from './retry';
const pipeline = promisify(stream.pipeline);
interface SignedUrlParams {
key: string;
expirationTime: number;
contentType?: string;
extensionHeaders?: { [key: string]: number | string | string[] };
}
interface UploadWithSignedUrlParams {
signedUrl: GCS.SignedUrl;
srcGeneratorAsync: () => Promise<Readable>;
retryIntervalMs?: RetryOptions['retryIntervalMs'];
retries?: RetryOptions['retries'];
}
class GCS {
private readonly client = new Storage();
constructor(private readonly bucket: string) {
this.bucket = bucket;
}
public static async uploadWithSignedUrl({
signedUrl,
srcGeneratorAsync,
retries = 2,
retryIntervalMs = 30_000,
}: UploadWithSignedUrlParams): Promise<string> {
let resp;
try {
resp = await retryOnGCSUploadFailure(
async () => {
const src = await srcGeneratorAsync();
return await fetch(signedUrl.url, {
method: 'PUT',
headers: signedUrl.headers,
body: src,
});
},
{
retries,
retryIntervalMs,
}
);
} catch (err: any) {
if (err instanceof FetchError) {
throw new Error(`Failed to upload the file, reason: ${err.code}`);
} else {
throw err;
}
}
if (!resp.ok) {
let body: string | undefined;
try {
body = await resp.text();
} catch {}
throw new Error(
`Failed to upload file: status: ${resp.status} status text: ${resp.statusText}, body: ${body}`
);
}
const url = new URL(signedUrl.url);
return `${url.protocol}//${url.host}${url.pathname}`; // strip query string
}
public formatHttpUrl(key: string): string {
return this.client.bucket(this.bucket).file(key).publicUrl();
}
public async uploadFile({
key,
src,
streamOptions,
}: {
key: string;
src: Readable;
streamOptions?: CreateWriteStreamOptions;
}): Promise<{ Location: string }> {
const file = this.client.bucket(this.bucket).file(key);
await new Promise<void>((res, rej) => {
src.pipe(
file
.createWriteStream(streamOptions)
.on('error', err => {
rej(err);
})
.on('finish', () => {
res();
})
);
});
return { Location: file.publicUrl() };
}
public async deleteFile(key: string): Promise<void> {
try {
await this.client.bucket(this.bucket).file(key).delete();
} catch (err: any) {
if (err.response?.statusCode === 404) {
return;
}
throw err;
}
}
public async createSignedUploadUrl({
key,
expirationTime,
contentType,
extensionHeaders = {},
}: SignedUrlParams): Promise<GCS.SignedUrl> {
const config = {
version: 'v4' as const,
action: 'write' as const,
expires: Date.now() + expirationTime,
contentType,
extensionHeaders,
};
const [url] = await this.client.bucket(this.bucket).file(key).getSignedUrl(config);
return {
url,
headers: {
...(contentType ? { 'content-type': contentType } : {}),
...extensionHeaders,
},
};
}
public async createSignedDownloadUrl({
key,
expirationTime,
}: {
key: string;
expirationTime: number;
}): Promise<string> {
const options = {
version: 'v4' as const,
action: 'read' as const,
expires: Date.now() + expirationTime,
};
const [url] = await this.client.bucket(this.bucket).file(key).getSignedUrl(options);
return url;
}
public async checkIfFileExists(key: string, fileHash?: string): Promise<boolean> {
let metadata;
try {
[metadata] = await this.client.bucket(this.bucket).file(key).getMetadata();
} catch (error: any) {
if (error.code === 404) {
return false;
}
throw error;
}
return fileHash ? metadata.etag === fileHash : true;
}
public async listDirectory(prefix: string): Promise<string[]> {
const [files] = await this.client.bucket(this.bucket).getFiles({ prefix });
return files.map(x => this.formatHttpUrl(x.name));
}
public async moveFile(src: string, dest: string): Promise<void> {
await this.client.bucket(this.bucket).file(src).move(dest);
}
public async deleteFiles(keys: string[]): Promise<void> {
await Promise.all(keys.map(key => this.deleteFile(key)));
}
public async downloadFile(key: string, destinationPath: string): Promise<void> {
const stream = this.client.bucket(this.bucket).file(key).createReadStream();
await pipeline(stream, fs.createWriteStream(destinationPath));
}
public getFile(key: string): File {
return this.client.bucket(this.bucket).file(key);
}
}
namespace GCS {
export interface SignedUrl {
url: string;
headers: { [key: string]: string };
}
export interface Config {
accessKeyId: string;
secretAccessKey: string;
region: string;
bucket: string;
}
}
export default GCS;
+1
View File
@@ -47,6 +47,7 @@ export async function runGenericJobAsync(
externalFunctions: getEasFunctions(customBuildCtx),
externalFunctionGroups: getEasFunctionGroups(customBuildCtx),
steps: ctx.job.steps,
hooks: ctx.job.hooks,
});
const workflow = await ctx.runBuildPhase(BuildPhase.PARSE_CUSTOM_WORKFLOW_CONFIG, async () => {
+4 -3
View File
@@ -1,8 +1,9 @@
import * as Builders from './builders';
import GCSLoggerStream from './gcs/LoggerStream';
import GCS from './gcs/client';
import RemoteLoggerStream from './logging/RemoteLoggerStream';
export { Builders, GCS, GCSLoggerStream };
export { Builders, RemoteLoggerStream };
export { uploadWithSignedUrl } from './storage/uploadWithSignedUrl';
export type { SignedUrl, UploadWithSignedUrlParams } from './storage/uploadWithSignedUrl';
export {
ArtifactToUpload,
@@ -8,17 +8,17 @@ import { Readable, Writable, pipeline } from 'stream';
import { promisify } from 'util';
import zlib from 'zlib';
import GCS from './client';
import { type SignedUrl, uploadWithSignedUrl } from '../storage/uploadWithSignedUrl';
type PromiseResolveFn = (value?: void | PromiseLike<void> | undefined) => void;
const pipe = promisify(pipeline);
class GCSLoggerStream extends Writable {
class RemoteLoggerStream extends Writable {
public writable = true;
private readonly logger: bunyan;
private readonly uploadMethod?: GCSLoggerStream.UploadMethod;
private readonly options: GCSLoggerStream.Options;
private readonly uploadMethod?: RemoteLoggerStream.UploadMethod;
private readonly options: RemoteLoggerStream.Options;
private readonly temporaryLogsPath: string;
private readonly temporaryCompressedLogsPath: string;
private readonly compress: string | null;
@@ -30,7 +30,7 @@ class GCSLoggerStream extends Writable {
private writePromise?: Promise<any>;
private cleanUpCalled: boolean = false;
constructor({ logger, uploadMethod, options }: GCSLoggerStream.Config) {
constructor({ logger, uploadMethod, options }: RemoteLoggerStream.Config) {
super();
this.logger = logger;
this.uploadMethod = uploadMethod;
@@ -41,7 +41,7 @@ class GCSLoggerStream extends Writable {
}
private findNormalizedHeader(name: string): string | null {
if (!this.uploadMethod || 'client' in this.uploadMethod) {
if (!this.uploadMethod) {
return null;
}
@@ -84,7 +84,7 @@ class GCSLoggerStream extends Writable {
await fs.remove(this.temporaryLogsPath);
await fs.remove(this.temporaryCompressedLogsPath);
this.logger.info('Cleaning up GCS log stream');
this.logger.info('Cleaning up log stream');
}
public write(rec: any): boolean {
@@ -115,7 +115,7 @@ class GCSLoggerStream extends Writable {
this.buffer = this.buffer.slice(buffer.length);
this.hasChangesToUpload = true;
} catch (err) {
this.logger.error({ err, origin: 'gcs-logger' }, 'Failed to write logs to file');
this.logger.error({ err, origin: 'remote-logger' }, 'Failed to write logs to file');
} finally {
this.writePromise = undefined;
}
@@ -144,7 +144,7 @@ class GCSLoggerStream extends Writable {
return result;
})
.catch(err => {
this.logger.error({ err }, 'Failed to upload logs file to GCS');
this.logger.error({ err }, 'Failed to upload logs file');
})
.then(result => {
this.uploadingPromise = undefined;
@@ -165,25 +165,10 @@ class GCSLoggerStream extends Writable {
);
};
if ('signedUrl' in this.uploadMethod) {
return await GCS.uploadWithSignedUrl({
signedUrl: this.uploadMethod.signedUrl,
srcGeneratorAsync,
});
}
const { Location } = await this.uploadMethod.client.uploadFile({
key: this.uploadMethod.key,
src: await srcGeneratorAsync(),
streamOptions: {
metadata: {
contentType: 'text/plain;charset=utf-8',
...(this.compress !== null ? { contentEncoding: this.compress } : {}),
customTime: this.uploadMethod.customTime,
},
},
return await uploadWithSignedUrl({
signedUrl: this.uploadMethod.signedUrl,
srcGeneratorAsync,
});
return Location;
}
private async createCompressedStream(src: Readable): Promise<Readable> {
@@ -212,7 +197,7 @@ class GCSLoggerStream extends Writable {
}
}
namespace GCSLoggerStream {
namespace RemoteLoggerStream {
export enum CompressionMethod {
GZIP = 'gzip',
BR = 'br',
@@ -223,9 +208,9 @@ namespace GCSLoggerStream {
compress?: CompressionMethod;
}
export type UploadMethod =
| { client: GCS; key: string; customTime: Date | null }
| { signedUrl: GCS.SignedUrl };
export type UploadMethod = {
signedUrl: SignedUrl;
};
export interface Config {
logger: bunyan;
@@ -234,4 +219,4 @@ namespace GCSLoggerStream {
}
}
export default GCSLoggerStream;
export default RemoteLoggerStream;
@@ -0,0 +1,117 @@
import { HookAnchorId } from '@expo/eas-build-job';
import { StepsConfigParser } from '@expo/steps';
import {
createCustomBuildContextMock,
createGlobalContextMock,
} from '../../__tests__/utils/context';
import { getEasFunctionGroups } from '../easFunctionGroups';
import { getEasFunctions } from '../easFunctions';
jest.mock('fs');
describe('hook anchor self-declarations', () => {
it('exactly the four v0 functions declare anchors, with these bindings', () => {
const declarations: Record<string, HookAnchorId> = {};
for (const buildFunction of getEasFunctions(createCustomBuildContextMock())) {
if (buildFunction.__hookId !== undefined) {
declarations[buildFunction.getFullId()] = buildFunction.__hookId;
}
}
// An empty declaration set must fail: without these, every user-authored
// unstamped `uses:` step and every `eas/build` expansion loses hooks.
expect(declarations).toEqual({
'eas/checkout': 'checkout',
'eas/install_node_modules': 'install_node_modules',
'eas/upload_to_asc': 'submit',
'eas/maestro_tests': 'maestro_tests',
});
});
it('no function group id collides with a declaring function id', () => {
// Group lookup shadows function lookup in StepsConfigParser, so a
// collision would silently unbind the anchor.
const declaringFunctionIds = getEasFunctions(createCustomBuildContextMock())
.filter(buildFunction => buildFunction.__hookId !== undefined)
.map(buildFunction => buildFunction.getFullId());
const functionGroupIds = getEasFunctionGroups(createCustomBuildContextMock()).map(
functionGroup => functionGroup.getFullId()
);
for (const functionId of declaringFunctionIds) {
expect(functionGroupIds).not.toContain(functionId);
}
});
it('attaches hooks inside a uses: eas/build expansion (declaration-driven anchor discovery)', async () => {
// This is the guard against a semantically stale declaration — the
// declaration existence test alone cannot catch a binding that no longer
// corresponds to what the group actually expands to.
const customBuildCtx = createCustomBuildContextMock();
const globalContext = createGlobalContextMock({});
const parser = new StepsConfigParser(globalContext, {
steps: [{ uses: 'eas/build' }],
hooks: {
before_install_node_modules: [{ run: 'echo before', id: 'before-hook' }],
after_install_node_modules: [{ run: 'echo after', id: 'after-hook' }],
},
externalFunctions: getEasFunctions(customBuildCtx),
externalFunctionGroups: getEasFunctionGroups(customBuildCtx),
});
const workflow = await parser.parseAsync();
const installStep = workflow.buildSteps.find(step => step.__hookId === 'install_node_modules');
expect(installStep).toBeDefined();
const anchorHooks = workflow.hooksByAnchorStep.get(installStep!);
expect(anchorHooks?.anchor).toBe('install_node_modules');
expect(anchorHooks?.before[0].steps[0].id).toBe('before-hook');
expect(anchorHooks?.after[0].steps[0].id).toBe('after-hook');
});
it.each([
['eas/checkout', 'before_checkout', 'after_checkout', undefined],
[
'eas/install_node_modules',
'before_install_node_modules',
'after_install_node_modules',
undefined,
],
[
'eas/upload_to_asc',
'before_submit',
'after_submit',
{
ipa_path: 'app.ipa',
asc_api_key_path: 'key.p8',
apple_app_identifier: '1234567890',
bundle_version: '1',
bundle_short_version: '1.0.0',
},
],
[
'eas/maestro_tests',
'before_maestro_tests',
'after_maestro_tests',
{ flow_path: ['flow.yaml'] },
],
] as const)(
'an unstamped user-authored %s step fires its hook keys end-to-end',
async (functionId, beforeKey, afterKey, callInputs) => {
const customBuildCtx = createCustomBuildContextMock();
const globalContext = createGlobalContextMock({});
const parser = new StepsConfigParser(globalContext, {
steps: [{ uses: functionId, ...(callInputs !== undefined ? { with: callInputs } : null) }],
hooks: {
[beforeKey]: [{ run: 'echo before', id: 'before-hook' }],
[afterKey]: [{ run: 'echo after', id: 'after-hook' }],
},
externalFunctions: getEasFunctions(customBuildCtx),
externalFunctionGroups: getEasFunctionGroups(customBuildCtx),
});
const workflow = await parser.parseAsync();
expect(workflow.hooksByAnchorStep.size).toBe(1);
const anchorHooks = [...workflow.hooksByAnchorStep.values()][0];
expect(anchorHooks.before[0].steps[0].id).toBe('before-hook');
expect(anchorHooks.after[0].steps[0].id).toBe('after-hook');
}
);
});
@@ -1,6 +1,7 @@
import { BuildFunction } from '@expo/steps';
import { calculateEASUpdateRuntimeVersionFunction } from './functions/calculateEASUpdateRuntimeVersion';
import { createCapturePosthogEventFunction } from './functions/capturePosthogEvent';
import { createCheckoutBuildFunction } from './functions/checkout';
import { configureAndroidVersionFunction } from './functions/configureAndroidVersion';
import { configureEASUpdateIfInstalledFunction } from './functions/configureEASUpdateIfInstalled';
@@ -13,6 +14,7 @@ import { createEasDeployBuildFunction } from './functions/deploy';
import { createEasExportBuildFunction } from './functions/export';
import { eagerBundleBuildFunction } from './functions/eagerBundle';
import { createFindAndUploadBuildArtifactsBuildFunction } from './functions/findAndUploadBuildArtifacts';
import { createFinishIosSimulatorRecordingsBuildFunction } from './functions/finishIosSimulatorRecordings';
import { generateGymfileFromTemplateFunction } from './functions/generateGymfileFromTemplate';
import { createGetCredentialsForBuildTriggeredByGithubIntegration } from './functions/getCredentialsForBuildTriggeredByGitHubIntegration';
import { injectAndroidCredentialsFunction } from './functions/injectAndroidCredentials';
@@ -37,6 +39,7 @@ import { runFastlaneFunction } from './functions/runFastlane';
import { runGradleFunction } from './functions/runGradle';
import { createMaestroTestsBuildFunction } from './functions/maestroTests';
import { createSaveBuildCacheFunction } from './functions/saveBuildCache';
import { createRolloutPosthogFlagFunction } from './functions/rolloutPosthogFlag';
import { createSaveCacheFunction } from './functions/saveCache';
import { createSendSlackMessageFunction } from './functions/sendSlackMessage';
import { createStartAgentDeviceRemoteSessionBuildFunction } from './functions/startAgentDeviceRemoteSession';
@@ -44,10 +47,16 @@ import { createStartAndroidEmulatorBuildFunction } from './functions/startAndroi
import { createStartArgentRemoteSessionBuildFunction } from './functions/startArgentRemoteSession';
import { createStartCuttlefishDeviceBuildFunction } from './functions/startCuttlefishDevice';
import { createStartIosSimulatorBuildFunction } from './functions/startIosSimulator';
import { createStartIosSimulatorRecordingsBuildFunction } from './functions/startIosSimulatorRecordings';
import { createStartServeSimRemoteSessionBuildFunction } from './functions/startServeSimRemoteSession';
import { createUploadArtifactBuildFunction } from './functions/uploadArtifact';
import { createUploadDeviceRunSessionScreenRecordingsBuildFunction } from './functions/uploadDeviceRunSessionScreenRecordings';
import { createUploadToAscBuildFunction } from './functions/uploadToAsc';
import { createSetUpNpmrcBuildFunction } from './functions/useNpmToken';
import { createWaitForPosthogMetricFunction } from './functions/waitForPosthogMetric';
import { createUploadPosthogSourcemapsFunction } from './functions/uploadPosthogSourcemaps';
import { createPosthogAnnotationFunction } from './functions/createPosthogAnnotation';
import { createWaitForPosthogQueryFunction } from './functions/waitForPosthogQuery';
import { CustomBuildContext } from '../customBuildContext';
export function getEasFunctions(ctx: CustomBuildContext): BuildFunction[] {
@@ -86,11 +95,20 @@ export function getEasFunctions(ctx: CustomBuildContext): BuildFunction[] {
createStartAndroidEmulatorBuildFunction(),
createStartCuttlefishDeviceBuildFunction(),
createStartIosSimulatorBuildFunction(),
createStartIosSimulatorRecordingsBuildFunction(),
createFinishIosSimulatorRecordingsBuildFunction(),
createUploadDeviceRunSessionScreenRecordingsBuildFunction(ctx),
createStartServeSimRemoteSessionBuildFunction(ctx),
createInstallMaestroBuildFunction(),
createInstallPodsBuildFunction(),
createSendSlackMessageFunction(),
createCapturePosthogEventFunction(),
createRolloutPosthogFlagFunction(),
createWaitForPosthogMetricFunction(),
createUploadPosthogSourcemapsFunction(),
createPosthogAnnotationFunction(),
createWaitForPosthogQueryFunction(),
calculateEASUpdateRuntimeVersionFunction(),
@@ -0,0 +1,176 @@
import fetch, { Response } from 'node-fetch';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { createCapturePosthogEventFunction } from '../capturePosthogEvent';
jest.mock('@expo/logger');
jest.mock('node-fetch');
describe(createCapturePosthogEventFunction, () => {
const fetchMock = jest.mocked(fetch);
const captureEvent = createCapturePosthogEventFunction();
afterEach(() => {
jest.resetAllMocks();
});
function createStep(
callInputs: Record<string, unknown>,
env: Record<string, string>
): ReturnType<typeof captureEvent.createBuildStepFromFunctionCall> {
return captureEvent.createBuildStepFromFunctionCall(createGlobalContextMock({}), {
callInputs,
env,
id: captureEvent.id,
});
}
it('sends the event to the ingestion endpoint using the env key', async () => {
fetchMock.mockResolvedValue({ status: 200, ok: true } as Response);
const step = createStep(
{ event: 'workflow_finished', distinct_id: 'account-1' },
{
EXPO_PUBLIC_POSTHOG_API_KEY: 'phc_test',
EXPO_PUBLIC_POSTHOG_HOST: 'https://us.posthog.com',
}
);
await step.executeAsync();
expect(fetchMock).toHaveBeenCalledTimes(1);
const [url, init] = fetchMock.mock.calls[0];
expect(url).toBe('https://us.posthog.com/i/v0/e/');
const body = JSON.parse(init?.body as string);
expect(body).toMatchObject({
api_key: 'phc_test',
event: 'workflow_finished',
distinct_id: 'account-1',
});
expect(body.properties?.$process_person_profile).toBeUndefined();
});
it('sends an anonymous system event when no distinct_id is given', async () => {
fetchMock.mockResolvedValue({ status: 200, ok: true } as Response);
const step = createStep(
{ event: 'workflow_finished' },
{ EXPO_PUBLIC_POSTHOG_API_KEY: 'phc_test' }
);
await step.executeAsync();
const body = JSON.parse(fetchMock.mock.calls[0][1]?.body as string);
expect(body.distinct_id).toBe('eas-workflow');
expect(body.properties.$process_person_profile).toBe(false);
});
it('skips when no api key is available and ignore_error is set', async () => {
const step = createStep(
{ event: 'workflow_finished', distinct_id: 'account-1', ignore_error: true },
{}
);
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await step.executeAsync();
expect(fetchMock).not.toHaveBeenCalled();
expect(warnMock).toHaveBeenCalledWith(
{ err: expect.any(Error) },
expect.stringMatching(/Ignoring error\.$/)
);
});
it('logs and does not throw on a non-2xx response when ignore_error is set', async () => {
fetchMock.mockResolvedValue({
status: 400,
ok: false,
text: async () => '',
} as unknown as Response);
const step = createStep(
{ event: 'workflow_finished', distinct_id: 'account-1', ignore_error: true },
{ EXPO_PUBLIC_POSTHOG_API_KEY: 'phc_test' }
);
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await expect(step.executeAsync()).resolves.not.toThrow();
expect(warnMock).toHaveBeenCalledWith(
{ err: expect.any(Error) },
'Sending PostHog event "workflow_finished" failed with status 400. Ignoring error.'
);
});
it('does not let properties override the anonymous marker', async () => {
fetchMock.mockResolvedValue({ status: 200, ok: true } as Response);
const step = createStep(
{ event: 'workflow_finished', properties: { $process_person_profile: true, foo: 'bar' } },
{ EXPO_PUBLIC_POSTHOG_API_KEY: 'phc_test' }
);
await step.executeAsync();
const body = JSON.parse(fetchMock.mock.calls[0][1]?.body as string);
expect(body.properties.$process_person_profile).toBe(false);
expect(body.properties.foo).toBe('bar');
});
it('throws on a network error by default', async () => {
fetchMock.mockRejectedValue(new Error('ECONNRESET'));
const step = createStep(
{ event: 'workflow_finished' },
{ EXPO_PUBLIC_POSTHOG_API_KEY: 'phc_test' }
);
await expect(step.executeAsync()).rejects.toThrow(
'Sending PostHog event "workflow_finished" failed.'
);
});
it('logs and does not throw on a network error when ignore_error is set', async () => {
fetchMock.mockRejectedValue(new Error('ECONNRESET'));
const step = createStep(
{ event: 'workflow_finished', ignore_error: true },
{ EXPO_PUBLIC_POSTHOG_API_KEY: 'phc_test' }
);
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await expect(step.executeAsync()).resolves.not.toThrow();
expect(warnMock).toHaveBeenCalledWith(
{ err: expect.any(Error) },
'Sending PostHog event "workflow_finished" failed. Ignoring error.'
);
});
it('throws with the PostHog error detail on failure by default', async () => {
fetchMock.mockResolvedValue({
status: 400,
ok: false,
text: async () => JSON.stringify({ detail: 'Invalid event name' }),
} as unknown as Response);
const step = createStep(
{ event: 'workflow_finished' },
{ EXPO_PUBLIC_POSTHOG_API_KEY: 'phc_test' }
);
await expect(step.executeAsync()).rejects.toThrow(/failed with status 400: Invalid event name/);
});
it('falls back to the raw error body when PostHog omits a detail', async () => {
fetchMock.mockResolvedValue({
status: 400,
ok: false,
text: async () => JSON.stringify({ error: 'nope' }),
} as unknown as Response);
const step = createStep(
{ event: 'workflow_finished' },
{ EXPO_PUBLIC_POSTHOG_API_KEY: 'phc_test' }
);
await expect(step.executeAsync()).rejects.toThrow(/failed with status 400: {"error":"nope"}/);
});
it('throws on missing key by default', async () => {
const step = createStep({ event: 'workflow_finished' }, {});
await expect(step.executeAsync()).rejects.toThrow(/PostHog API key not provided/);
expect(fetchMock).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,138 @@
import fetch, { Response } from 'node-fetch';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { createPosthogAnnotationFunction } from '../createPosthogAnnotation';
jest.mock('@expo/logger');
jest.mock('node-fetch');
function jsonResponse(data: unknown, { ok = true, status = 200 } = {}): Response {
return {
ok,
status,
json: async () => data,
text: async () => JSON.stringify(data),
} as unknown as Response;
}
const ENV = {
POSTHOG_CLI_API_KEY: 'phx_test',
POSTHOG_CLI_PROJECT_ID: '123',
POSTHOG_CLI_HOST: 'https://us.posthog.com',
};
describe(createPosthogAnnotationFunction, () => {
const fetchMock = jest.mocked(fetch);
const annotation = createPosthogAnnotationFunction();
afterEach(() => {
jest.resetAllMocks();
});
function createStep(
callInputs: Record<string, unknown>,
env: Record<string, string> = ENV
): ReturnType<typeof annotation.createBuildStepFromFunctionCall> {
return annotation.createBuildStepFromFunctionCall(createGlobalContextMock({}), {
callInputs,
env,
id: annotation.id,
});
}
it('posts the annotation with an explicit date_marker', async () => {
fetchMock.mockResolvedValue(jsonResponse({ id: 1 }, { status: 201 }));
const step = createStep({ content: 'Released abc123', date_marker: '2026-07-03T00:00:00Z' });
await step.executeAsync();
const [url, init] = fetchMock.mock.calls[0];
expect(url).toBe('https://us.posthog.com/api/projects/123/annotations/');
expect(JSON.parse(init?.body as string)).toEqual({
content: 'Released abc123',
date_marker: '2026-07-03T00:00:00Z',
});
});
it('defaults date_marker to now when not given', async () => {
fetchMock.mockResolvedValue(jsonResponse({ id: 1 }, { status: 201 }));
const step = createStep({ content: 'Deploy' });
await step.executeAsync();
const body = JSON.parse(fetchMock.mock.calls[0][1]?.body as string);
expect(body.content).toBe('Deploy');
expect(typeof body.date_marker).toBe('string');
});
it('throws when credentials are missing by default', async () => {
const step = createStep({ content: 'Deploy' }, {});
await expect(step.executeAsync()).rejects.toThrow(
/Missing PostHog credentials: personal API key, project id/
);
expect(fetchMock).not.toHaveBeenCalled();
});
it('warns and skips when credentials are missing and ignore_error is set', async () => {
const step = createStep({ content: 'Deploy', ignore_error: true }, {});
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await step.executeAsync();
expect(fetchMock).not.toHaveBeenCalled();
expect(warnMock).toHaveBeenCalledWith(
{ err: expect.any(Error) },
expect.stringMatching(/Ignoring error/)
);
});
it('throws on a network error by default', async () => {
fetchMock.mockRejectedValue(new Error('ECONNRESET'));
const step = createStep({ content: 'Deploy' });
await expect(step.executeAsync()).rejects.toThrow(/Creating the PostHog annotation failed/);
});
it('warns and continues on a network error when ignore_error is set', async () => {
fetchMock.mockRejectedValue(new Error('ECONNRESET'));
const step = createStep({ content: 'Deploy', ignore_error: true });
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await expect(step.executeAsync()).resolves.not.toThrow();
expect(warnMock).toHaveBeenCalledWith(
{ err: expect.any(Error) },
expect.stringMatching(/Ignoring error/)
);
});
it('throws on a 403 without swallowing it', async () => {
fetchMock.mockResolvedValue(jsonResponse({}, { ok: false, status: 403 }));
const step = createStep({ content: 'Deploy' });
await expect(step.executeAsync()).rejects.toThrow(/annotation:write/);
});
it('throws with the PostHog error detail on a non-2xx response', async () => {
fetchMock.mockResolvedValue(
jsonResponse({ detail: 'Invalid date_marker' }, { ok: false, status: 400 })
);
const step = createStep({ content: 'Deploy' });
await expect(step.executeAsync()).rejects.toThrow(
/failed with status 400: Invalid date_marker/
);
});
it('warns and continues on a non-2xx response when ignore_error is set', async () => {
fetchMock.mockResolvedValue(jsonResponse({}, { ok: false, status: 500 }));
const step = createStep({ content: 'Deploy', ignore_error: true });
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await expect(step.executeAsync()).resolves.not.toThrow();
expect(warnMock).toHaveBeenCalledWith(
{ err: expect.any(Error) },
expect.stringMatching(/Ignoring error/)
);
});
});
@@ -1,132 +0,0 @@
import { BuildStepContext, BuildStepEnv } from '@expo/steps';
import spawn, { SpawnPromise, SpawnResult } from '@expo/turtle-spawn';
import { createMockLogger } from '../../../__tests__/utils/logger';
import {
PackageManager,
findPackagerRootDir,
resolvePackageManager,
shouldUseFrozenLockfile,
} from '../../../utils/packageManager';
import { readPackageJson } from '../../../utils/project';
import { installNodeModules } from '../installNodeModules';
jest.mock('@expo/turtle-spawn', () => jest.fn());
jest.mock('../../../sentry', () => ({
Sentry: {
capture: jest.fn(),
},
}));
jest.mock('../../../utils/packageManager', () => ({
...jest.requireActual('../../../utils/packageManager'),
resolvePackageManager: jest.fn(),
findPackagerRootDir: jest.fn(),
shouldUseFrozenLockfile: jest.fn(),
}));
jest.mock('../../../utils/project', () => ({
...jest.requireActual('../../../utils/project'),
readPackageJson: jest.fn(),
}));
describe(installNodeModules, () => {
beforeEach(() => {
jest.clearAllMocks();
jest.mocked(resolvePackageManager).mockReturnValue(PackageManager.NPM);
jest.mocked(findPackagerRootDir).mockReturnValue('/tmp/build');
jest.mocked(shouldUseFrozenLockfile).mockReturnValue(false);
jest.mocked(readPackageJson).mockReturnValue({});
});
function createStepCtx(logger: ReturnType<typeof createMockLogger>): BuildStepContext {
return {
logger,
workingDirectory: '/tmp/build',
global: {
projectTargetDirectory: '/tmp/build',
staticContext: { metadata: {} },
},
} as unknown as BuildStepContext;
}
it('retries the install without the npm cache registry when the cache install fails', async () => {
const logger = createMockLogger();
const npmCacheUrl = 'http://npm.caches.eas-build.internal';
const env: BuildStepEnv = {
EAS_USE_NPM_CACHE: '1',
EAS_BUILD_NPM_CACHE_URL: npmCacheUrl,
};
const error = Object.assign(new Error('npm install exited with non-zero code: 1'), {
status: 1,
signal: null,
stdout: '',
stderr: `npm error request to ${npmCacheUrl}/left-pad failed, reason: getaddrinfo ENOTFOUND`,
});
jest
.mocked(spawn)
.mockReturnValueOnce(createSpawnPromise(Promise.reject(error)))
.mockReturnValueOnce(createSpawnPromise(Promise.resolve(createSpawnResult())));
await installNodeModules(createStepCtx(logger), env);
expect(spawn).toHaveBeenCalledTimes(2);
expect(spawn).toHaveBeenNthCalledWith(
1,
'npm',
['install', '--include=dev'],
expect.objectContaining({
env: {
EAS_USE_NPM_CACHE: '1',
EAS_BUILD_NPM_CACHE_URL: npmCacheUrl,
NPM_CONFIG_REGISTRY: npmCacheUrl,
},
})
);
expect(spawn).toHaveBeenNthCalledWith(
2,
'npm',
['install', '--include=dev'],
expect.objectContaining({
env: { EAS_USE_NPM_CACHE: '1', EAS_BUILD_NPM_CACHE_URL: npmCacheUrl },
})
);
expect(logger.warn).toHaveBeenCalledWith(
`Failed to install dependencies using the npm cache registry (${npmCacheUrl}). Retrying without the npm cache registry.`
);
});
it('does not retry when the npm cache is not enabled', async () => {
const logger = createMockLogger();
const error = Object.assign(new Error('npm install exited with non-zero code: 1'), {
status: 1,
signal: null,
stdout: '',
stderr: 'npm error some unrelated failure',
});
jest.mocked(spawn).mockReturnValueOnce(createSpawnPromise(Promise.reject(error)));
await expect(installNodeModules(createStepCtx(logger), {})).rejects.toThrow(error);
expect(spawn).toHaveBeenCalledTimes(1);
});
});
function createSpawnPromise(result: Promise<SpawnResult>): SpawnPromise<SpawnResult> {
return Object.assign(result, {
child: {
pid: 123,
},
}) as SpawnPromise<SpawnResult>;
}
function createSpawnResult(): SpawnResult {
return {
pid: 123,
output: ['', ''],
stdout: '',
stderr: '',
status: 0,
signal: null,
};
}
@@ -7,6 +7,7 @@ import {
isFileAttrRun,
junitFileHasFileAttrs,
mergeJUnitReports,
parseFailedFlowNamesFromJUnitFile,
parseFailedFlowsFromFileAttrs,
parseFailedFlowsFromJUnit,
parseJUnitTestCases,
@@ -14,6 +15,33 @@ import {
parseMaestroResultsFromFileAttrs,
} from '../maestroResultParser';
describe(parseFailedFlowNamesFromJUnitFile, () => {
it('returns the names of failed testcases, keeping slashes', async () => {
vol.fromJSON({
'/attempt/android-maestro-junit-attempt-0.xml': [
'<?xml version="1.0"?>',
'<testsuites>',
' <testsuite name="Test Suite" tests="3" failures="2">',
' <testcase name="home" status="SUCCESS"/>',
' <testcase name="sub/login" status="ERROR"><failure>boom</failure></testcase>',
' <testcase name="checkout" status="ERROR"><failure>boom</failure></testcase>',
' </testsuite>',
'</testsuites>',
].join('\n'),
});
const failed = await parseFailedFlowNamesFromJUnitFile(
'/attempt/android-maestro-junit-attempt-0.xml'
);
expect([...failed].sort()).toEqual(['checkout', 'sub/login']);
});
it('returns an empty set when the file cannot be read', async () => {
expect((await parseFailedFlowNamesFromJUnitFile('/missing.xml')).size).toBe(0);
});
});
describe(parseMaestroResults, () => {
it('parses JUnit results and enriches with name→path map', async () => {
vol.fromJSON({
@@ -80,6 +80,7 @@ describe(harvestFailureScreenshotsAsync, () => {
testsDirectory,
capturedSinceMs: sinceMtimeMs,
attemptIndex: 1,
failedFlowNames: new Set(),
logger,
});
@@ -113,6 +114,7 @@ describe(harvestFailureScreenshotsAsync, () => {
testsDirectory,
capturedSinceMs: sinceMtimeMs,
attemptIndex: 0,
failedFlowNames: new Set(),
logger,
});
@@ -136,6 +138,7 @@ describe(harvestFailureScreenshotsAsync, () => {
testsDirectory,
capturedSinceMs: sinceMtimeMs,
attemptIndex: 0,
failedFlowNames: new Set(),
logger,
});
@@ -154,6 +157,7 @@ describe(harvestFailureScreenshotsAsync, () => {
testsDirectory,
capturedSinceMs: sinceMtimeMs,
attemptIndex: 0,
failedFlowNames: new Set(),
logger,
});
@@ -175,6 +179,7 @@ describe(harvestFailureScreenshotsAsync, () => {
testsDirectory,
capturedSinceMs: sinceMtimeMs,
attemptIndex: 1,
failedFlowNames: new Set(),
logger,
});
@@ -186,11 +191,304 @@ describe(harvestFailureScreenshotsAsync, () => {
testsDirectory: path.join(testsDirectory, 'does-not-exist'),
capturedSinceMs: 0,
attemptIndex: 0,
failedFlowNames: new Set(),
logger,
});
expect(shots).toEqual([]);
});
// --- Maestro >= 2.7.0 bundle layout: <session>/<flow>[-shard-N]/screenshots/step-<NNN>-<slug>.png ---
async function makeBundle(args: {
sessionDir: string;
flowDir: string;
stepFiles: { name: string; mtimeMs: number }[];
sessionDirMtimeMs: number;
}): Promise<void> {
const sessionDir = path.join(testsDirectory, args.sessionDir);
const screenshotsDir = path.join(sessionDir, args.flowDir, 'screenshots');
await fs.mkdir(screenshotsDir, { recursive: true });
for (const { name, mtimeMs } of args.stepFiles) {
const filePath = path.join(screenshotsDir, name);
await fs.writeFile(filePath, '');
const when = new Date(mtimeMs);
await fs.utimes(filePath, when, when);
}
const dirWhen = new Date(args.sessionDirMtimeMs);
await fs.utimes(sessionDir, dirWhen, dirWhen);
}
it('picks the highest step-NNN screenshot in a failed flow bundle (mtime as capturedAtMs)', async () => {
const since = 1_000_000;
await makeBundle({
sessionDir: '2026-01-01_100000',
flowDir: 'Login Flow',
stepFiles: [
{ name: 'step-001-launchApp.png', mtimeMs: since + 1_000 },
{ name: 'step-003-assertVisible-Welcome.png', mtimeMs: since + 3_000 },
],
sessionDirMtimeMs: since + 5_000,
});
const shots = await harvestFailureScreenshotsAsync({
testsDirectory,
capturedSinceMs: since,
attemptIndex: 0,
failedFlowNames: new Set(['Login Flow']),
logger,
});
expect(shots).toEqual([
{
fileAbsPath: path.join(
testsDirectory,
'2026-01-01_100000',
'Login Flow',
'screenshots',
'step-003-assertVisible-Welcome.png'
),
displayName: 'Failure Screenshot: Login Flow (attempt 1)',
metadata: {
kind: 'maestro-test-screenshot',
flowName: 'Login Flow',
attemptIndex: 0,
capturedAtMs: since + 3_000,
},
},
]);
});
it('ignores final.png and non-step pngs in the bundle', async () => {
const since = 1_000_000;
await makeBundle({
sessionDir: 'session',
flowDir: 'Flow',
stepFiles: [
{ name: 'step-002-tapOn.png', mtimeMs: since + 2_000 },
{ name: 'final.png', mtimeMs: since + 9_000 },
],
sessionDirMtimeMs: since + 9_000,
});
const shots = await harvestFailureScreenshotsAsync({
testsDirectory,
capturedSinceMs: since,
attemptIndex: 0,
failedFlowNames: new Set(['Flow']),
logger,
});
expect(shots).toHaveLength(1);
expect(shots[0].fileAbsPath.endsWith('step-002-tapOn.png')).toBe(true);
});
it('harvests only flows that JUnit reports as failed', async () => {
const since = 1_000_000;
await makeBundle({
sessionDir: 'session',
flowDir: 'Failed Flow',
stepFiles: [{ name: 'step-001-tapOn.png', mtimeMs: since + 1_000 }],
sessionDirMtimeMs: since + 5_000,
});
// A passing flow can still leave a warned-step screenshot; it must not be harvested.
await makeBundle({
sessionDir: 'session',
flowDir: 'Passed Flow',
stepFiles: [{ name: 'step-001-assertVisible.png', mtimeMs: since + 1_000 }],
sessionDirMtimeMs: since + 5_000,
});
const shots = await harvestFailureScreenshotsAsync({
testsDirectory,
capturedSinceMs: since,
attemptIndex: 0,
failedFlowNames: new Set(['Failed Flow']),
logger,
});
expect(shots.map(shot => shot.metadata.flowName)).toEqual(['Failed Flow']);
});
it('strips the -shard-N suffix to match the JUnit flow name', async () => {
const since = 1_000_000;
await makeBundle({
sessionDir: 'session',
flowDir: 'Checkout-shard-2',
stepFiles: [{ name: 'step-004-tapOn-Pay.png', mtimeMs: since + 1_000 }],
sessionDirMtimeMs: since + 5_000,
});
const shots = await harvestFailureScreenshotsAsync({
testsDirectory,
capturedSinceMs: since,
attemptIndex: 0,
failedFlowNames: new Set(['Checkout']),
logger,
});
expect(shots).toHaveLength(1);
expect(shots[0].metadata.flowName).toBe('Checkout');
});
it('resolves a combined shard + collision dir name to the flow', async () => {
const since = 1_000_000;
await makeBundle({
sessionDir: 'session',
flowDir: 'Checkout-shard-2-3',
stepFiles: [{ name: 'step-004-tapOn-Pay.png', mtimeMs: since + 1_000 }],
sessionDirMtimeMs: since + 5_000,
});
const shots = await harvestFailureScreenshotsAsync({
testsDirectory,
capturedSinceMs: since,
attemptIndex: 0,
failedFlowNames: new Set(['Checkout']),
logger,
});
expect(shots).toHaveLength(1);
expect(shots[0].metadata.flowName).toBe('Checkout');
});
it('normalizes / to _ when matching the bundle dir to the JUnit flow name', async () => {
const since = 1_000_000;
await makeBundle({
sessionDir: 'session',
flowDir: 'sub_Login',
stepFiles: [{ name: 'step-002-tapOn.png', mtimeMs: since + 1_000 }],
sessionDirMtimeMs: since + 5_000,
});
const shots = await harvestFailureScreenshotsAsync({
testsDirectory,
capturedSinceMs: since,
attemptIndex: 0,
failedFlowNames: new Set(['sub/Login']),
logger,
});
expect(shots).toHaveLength(1);
expect(shots[0].metadata.flowName).toBe('sub_Login');
});
it('skips a bundle whose dir name is ambiguous against the failed set', async () => {
const since = 1_000_000;
// 'foo-2' could be the literal flow 'foo-2' or a collision-suffixed 'foo' — unresolvable.
await makeBundle({
sessionDir: 'session',
flowDir: 'foo-2',
stepFiles: [{ name: 'step-001-tapOn.png', mtimeMs: since + 1_000 }],
sessionDirMtimeMs: since + 5_000,
});
const shots = await harvestFailureScreenshotsAsync({
testsDirectory,
capturedSinceMs: since,
attemptIndex: 0,
failedFlowNames: new Set(['foo', 'foo-2']),
logger,
});
expect(shots).toEqual([]);
});
it('skips a step screenshot captured before capturedSinceMs (dir touched later)', async () => {
const since = 1_000_000;
await makeBundle({
sessionDir: 'session',
flowDir: 'Flow',
stepFiles: [{ name: 'step-001-tapOn.png', mtimeMs: since - 5_000 }],
sessionDirMtimeMs: since + 5_000,
});
const shots = await harvestFailureScreenshotsAsync({
testsDirectory,
capturedSinceMs: since,
attemptIndex: 0,
failedFlowNames: new Set(['Flow']),
logger,
});
expect(shots).toEqual([]);
});
it('yields nothing for a failed flow whose screenshots dir is empty', async () => {
const since = 1_000_000;
await makeBundle({
sessionDir: 'session',
flowDir: 'Flow',
stepFiles: [],
sessionDirMtimeMs: since + 5_000,
});
const shots = await harvestFailureScreenshotsAsync({
testsDirectory,
capturedSinceMs: since,
attemptIndex: 0,
failedFlowNames: new Set(['Flow']),
logger,
});
expect(shots).toEqual([]);
});
it('reduces two bundle dirs that resolve to the same flow to a single shot', async () => {
const since = 1_000_000;
await makeBundle({
sessionDir: 'session',
flowDir: 'Dup',
stepFiles: [{ name: 'step-001-tapOn.png', mtimeMs: since + 1_000 }],
sessionDirMtimeMs: since + 5_000,
});
await makeBundle({
sessionDir: 'session',
flowDir: 'Dup-2',
stepFiles: [{ name: 'step-001-tapOn.png', mtimeMs: since + 2_000 }],
sessionDirMtimeMs: since + 5_000,
});
const shots = await harvestFailureScreenshotsAsync({
testsDirectory,
capturedSinceMs: since,
attemptIndex: 0,
failedFlowNames: new Set(['Dup']),
logger,
});
expect(shots).toHaveLength(1);
expect(shots[0].metadata.flowName).toBe('Dup');
});
it('collects both a legacy flat screenshot and a new-layout bundle in the same session dir', async () => {
const since = 1_000_000;
const sessionDir = path.join(testsDirectory, 'session');
await fs.mkdir(sessionDir, { recursive: true });
await fs.writeFile(
path.join(sessionDir, `screenshot-❌-${since + 1_000}-(Legacy Flow).png`),
''
);
await makeBundle({
sessionDir: 'session',
flowDir: 'Bundle Flow',
stepFiles: [{ name: 'step-002-tapOn.png', mtimeMs: since + 2_000 }],
sessionDirMtimeMs: since + 5_000,
});
const shots = await harvestFailureScreenshotsAsync({
testsDirectory,
capturedSinceMs: since,
attemptIndex: 0,
failedFlowNames: new Set(['Bundle Flow']),
logger,
});
expect(shots.map(shot => shot.metadata.flowName).sort()).toEqual([
'Bundle Flow',
'Legacy Flow',
]);
});
});
describe(computePureFailureFlowNames, () => {
@@ -0,0 +1,443 @@
import fetch, { Response } from 'node-fetch';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { createRolloutPosthogFlagFunction } from '../rolloutPosthogFlag';
jest.mock('@expo/logger');
jest.mock('node-fetch');
function jsonResponse(data: unknown, { ok = true, status = 200 } = {}): Response {
return {
ok,
status,
json: async () => data,
text: async () => JSON.stringify(data),
} as unknown as Response;
}
const ENV = {
POSTHOG_CLI_API_KEY: 'phx_test',
POSTHOG_CLI_PROJECT_ID: '123',
POSTHOG_CLI_HOST: 'https://us.posthog.com',
};
describe(createRolloutPosthogFlagFunction, () => {
const fetchMock = jest.mocked(fetch);
const rolloutFlag = createRolloutPosthogFlagFunction();
afterEach(() => {
jest.resetAllMocks();
});
function createStep(
callInputs: Record<string, unknown>,
env: Record<string, string> = ENV
): ReturnType<typeof rolloutFlag.createBuildStepFromFunctionCall> {
return rolloutFlag.createBuildStepFromFunctionCall(createGlobalContextMock({}), {
callInputs,
env,
id: rolloutFlag.id,
});
}
it('enables a flag by key', async () => {
fetchMock
.mockResolvedValueOnce(jsonResponse({ results: [{ id: 42, key: 'my-flag', filters: {} }] }))
.mockResolvedValueOnce(jsonResponse({ id: 42 }));
const step = createStep({ flag: 'my-flag', active: 'true' });
await step.executeAsync();
const [patchUrl, patchInit] = fetchMock.mock.calls[1];
expect(patchUrl).toBe('https://us.posthog.com/api/projects/123/feature_flags/42/');
expect(patchInit?.method).toBe('PATCH');
expect(JSON.parse(patchInit?.body as string)).toEqual({ active: true });
});
it('sets rollout_percentage only on the catch-all group, preserving others', async () => {
fetchMock
.mockResolvedValueOnce(
jsonResponse({
results: [
{
id: 7,
key: 'gradual',
filters: {
groups: [
{ properties: [{ key: 'is_internal' }], rollout_percentage: 100 },
{ properties: [], rollout_percentage: 5 },
],
},
},
],
})
)
.mockResolvedValueOnce(jsonResponse({ id: 7 }));
const step = createStep({ flag: 'gradual', rollout_percentage: '50' });
await step.executeAsync();
const patchBody = JSON.parse(fetchMock.mock.calls[1][1]?.body as string);
expect(patchBody.filters.groups).toEqual([
{ properties: [{ key: 'is_internal' }], rollout_percentage: 100 },
{ properties: [], rollout_percentage: 50 },
]);
});
it('creates a catch-all group when the flag has none', async () => {
fetchMock
.mockResolvedValueOnce(jsonResponse({ results: [{ id: 8, key: 'bare', filters: {} }] }))
.mockResolvedValueOnce(jsonResponse({ id: 8 }));
const step = createStep({ flag: 'bare', rollout_percentage: '25' });
await step.executeAsync();
const patchBody = JSON.parse(fetchMock.mock.calls[1][1]?.body as string);
expect(patchBody.filters.groups).toEqual([{ properties: [], rollout_percentage: 25 }]);
});
it('falls back to the first group with a warning when no catch-all exists', async () => {
fetchMock
.mockResolvedValueOnce(
jsonResponse({
results: [
{
id: 9,
key: 'scoped',
filters: { groups: [{ properties: [{ key: 'is_internal' }] }] },
},
],
})
)
.mockResolvedValueOnce(jsonResponse({ id: 9 }));
const step = createStep({ flag: 'scoped', rollout_percentage: '10' });
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await step.executeAsync();
const patchBody = JSON.parse(fetchMock.mock.calls[1][1]?.body as string);
expect(patchBody.filters.groups).toEqual([
{ properties: [{ key: 'is_internal' }], rollout_percentage: 10 },
]);
expect(warnMock).toHaveBeenCalledWith(expect.stringMatching(/no catch-all/));
});
it('throws when the flag key does not exist', async () => {
fetchMock.mockResolvedValueOnce(jsonResponse({ results: [{ id: 1, key: 'other-flag' }] }));
const step = createStep({ flag: 'missing-flag', active: 'true' });
await expect(step.executeAsync()).rejects.toThrow(
/No PostHog feature flag with key "missing-flag"/
);
});
it('throws on a 403 even when ignore_error is set', async () => {
fetchMock.mockResolvedValueOnce(jsonResponse({}, { ok: false, status: 403 }));
const step = createStep({ flag: 'my-flag', active: 'true', ignore_error: true });
await expect(step.executeAsync()).rejects.toThrow(/forbidden \(403\)/);
});
it('throws when none of active, rollout_percentage, or payload is provided', async () => {
const step = createStep({ flag: 'my-flag' });
await expect(step.executeAsync()).rejects.toThrow(
/"active", "rollout_percentage", or "payload"/
);
expect(fetchMock).not.toHaveBeenCalled();
});
it('sets a boolean-flag payload under the "true" key as a JSON string', async () => {
fetchMock
.mockResolvedValueOnce(jsonResponse({ results: [{ id: 42, key: 'my-flag', filters: {} }] }))
.mockResolvedValueOnce(jsonResponse({ id: 42 }));
const step = createStep({ flag: 'my-flag', payload: { color: 'blue', count: 3 } });
await step.executeAsync();
const patchBody = JSON.parse(fetchMock.mock.calls[1][1]?.body as string);
expect(patchBody.filters.payloads).toEqual({ true: '{"color":"blue","count":3}' });
});
it('keys the payload under a given variant', async () => {
fetchMock
.mockResolvedValueOnce(jsonResponse({ results: [{ id: 42, key: 'my-flag', filters: {} }] }))
.mockResolvedValueOnce(jsonResponse({ id: 42 }));
const step = createStep({ flag: 'my-flag', payload: { v: 1 }, variant: 'control' });
await step.executeAsync();
const patchBody = JSON.parse(fetchMock.mock.calls[1][1]?.body as string);
expect(patchBody.filters.payloads).toEqual({ control: '{"v":1}' });
});
it('sets rollout_percentage and payload together in one filters object', async () => {
fetchMock
.mockResolvedValueOnce(
jsonResponse({
results: [{ id: 7, key: 'my-flag', filters: { groups: [{ properties: [] }] } }],
})
)
.mockResolvedValueOnce(jsonResponse({ id: 7 }));
const step = createStep({ flag: 'my-flag', rollout_percentage: 25, payload: { a: 1 } });
await step.executeAsync();
const patchBody = JSON.parse(fetchMock.mock.calls[1][1]?.body as string);
expect(patchBody.filters.groups[0].rollout_percentage).toBe(25);
expect(patchBody.filters.payloads).toEqual({ true: '{"a":1}' });
});
it('surfaces the PostHog error detail on a failed update', async () => {
fetchMock
.mockResolvedValueOnce(jsonResponse({ results: [{ id: 42, key: 'my-flag', filters: {} }] }))
.mockResolvedValueOnce(
jsonResponse(
{ type: 'validation_error', detail: 'Variant control does not exist' },
{ ok: false, status: 400 }
)
);
const step = createStep({ flag: 'my-flag', active: 'true' });
await expect(step.executeAsync()).rejects.toThrow(
/failed with status 400: Variant control does not exist/
);
});
it('falls back to the raw body when the error body is not JSON', async () => {
fetchMock
.mockResolvedValueOnce(jsonResponse({ results: [{ id: 42, key: 'my-flag', filters: {} }] }))
.mockResolvedValueOnce({
ok: false,
status: 502,
text: async () => 'Bad Gateway',
} as unknown as Response);
const step = createStep({ flag: 'my-flag', active: 'true' });
await expect(step.executeAsync()).rejects.toThrow(/failed with status 502: Bad Gateway/);
});
it('warns and skips when the flag is not found and ignore_error is set', async () => {
fetchMock.mockResolvedValueOnce(jsonResponse({ results: [] }));
const step = createStep({ flag: 'missing-flag', active: 'true', ignore_error: true });
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await expect(step.executeAsync()).resolves.not.toThrow();
expect(warnMock).toHaveBeenCalledWith(
{ err: expect.any(Error) },
expect.stringMatching(/No PostHog feature flag with key "missing-flag"/)
);
});
it('throws on an out-of-range rollout_percentage', async () => {
const step = createStep({ flag: 'my-flag', rollout_percentage: '150' });
await expect(step.executeAsync()).rejects.toThrow(/between 0 and 100/);
expect(fetchMock).not.toHaveBeenCalled();
});
it('throws when credentials are missing by default', async () => {
const step = createStep({ flag: 'my-flag', active: 'true' }, {});
await expect(step.executeAsync()).rejects.toThrow(
/Missing PostHog credentials: personal API key, project id/
);
expect(fetchMock).not.toHaveBeenCalled();
});
it('warns and skips on missing credentials when ignore_error is set', async () => {
const step = createStep({ flag: 'my-flag', active: 'true', ignore_error: true }, {});
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await step.executeAsync();
expect(fetchMock).not.toHaveBeenCalled();
expect(warnMock).toHaveBeenCalledWith(
{ err: expect.any(Error) },
expect.stringMatching(/Ignoring error\.$/)
);
});
it('throws on a failed update by default', async () => {
fetchMock
.mockResolvedValueOnce(jsonResponse({ results: [{ id: 42, key: 'my-flag', filters: {} }] }))
.mockResolvedValueOnce(jsonResponse({}, { ok: false, status: 500 }));
const step = createStep({ flag: 'my-flag', active: 'true' });
await expect(step.executeAsync()).rejects.toThrow(/failed with status 500/);
});
it('warns and skips on a failed update when ignore_error is set', async () => {
fetchMock
.mockResolvedValueOnce(jsonResponse({ results: [{ id: 42, key: 'my-flag', filters: {} }] }))
.mockResolvedValueOnce(jsonResponse({}, { ok: false, status: 500 }));
const step = createStep({ flag: 'my-flag', active: 'true', ignore_error: true });
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await expect(step.executeAsync()).resolves.not.toThrow();
expect(warnMock).toHaveBeenCalledWith(
{ err: expect.any(Error) },
'Updating PostHog flag "my-flag" failed with status 500: {}. Ignoring error.'
);
});
it('warns and skips on a lookup network error when ignore_error is set', async () => {
fetchMock.mockRejectedValueOnce(new Error('ECONNRESET'));
const step = createStep({ flag: 'my-flag', active: 'true', ignore_error: true });
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await expect(step.executeAsync()).resolves.not.toThrow();
expect(warnMock).toHaveBeenCalledWith(
{ err: expect.any(Error) },
'Looking up PostHog flag "my-flag" failed. Ignoring error.'
);
});
it('warns and skips on a non-object lookup response when ignore_error is set', async () => {
fetchMock.mockResolvedValueOnce(jsonResponse('not-json-object'));
const step = createStep({ flag: 'my-flag', active: 'true', ignore_error: true });
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await step.executeAsync();
expect(warnMock).toHaveBeenCalledWith(
{ err: expect.anything() },
expect.stringMatching(/unexpected response/)
);
});
it('warns and skips on an unexpected lookup response when ignore_error is set', async () => {
fetchMock.mockResolvedValueOnce(jsonResponse({ nope: true }));
const step = createStep({ flag: 'my-flag', active: 'true', ignore_error: true });
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await step.executeAsync();
expect(warnMock).toHaveBeenCalledWith(
{ err: expect.anything() },
expect.stringMatching(/unexpected response/)
);
});
it('ignores malformed entries in the lookup results', async () => {
fetchMock
.mockResolvedValueOnce(
jsonResponse({ results: ['junk', null, { id: 42, key: 'my-flag', filters: {} }] })
)
.mockResolvedValueOnce(jsonResponse({ id: 42 }));
const step = createStep({ flag: 'my-flag', active: 'true' });
await step.executeAsync();
expect(fetchMock).toHaveBeenCalledTimes(2);
});
it('throws on a failed lookup by default', async () => {
fetchMock.mockResolvedValueOnce(jsonResponse({}, { ok: false, status: 500 }));
const step = createStep({ flag: 'my-flag', active: 'true' });
await expect(step.executeAsync()).rejects.toThrow(
/Looking up PostHog flag "my-flag" failed with status 500/
);
});
it('warns and skips on a failed lookup when ignore_error is set', async () => {
fetchMock.mockResolvedValueOnce(jsonResponse({}, { ok: false, status: 500 }));
const step = createStep({ flag: 'my-flag', active: 'true', ignore_error: true });
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await expect(step.executeAsync()).resolves.not.toThrow();
expect(warnMock).toHaveBeenCalledWith(
{ err: expect.any(Error) },
'Looking up PostHog flag "my-flag" failed with status 500: {}. Ignoring error.'
);
expect(fetchMock).toHaveBeenCalledTimes(1);
});
it('warns and skips on an update network error when ignore_error is set', async () => {
fetchMock
.mockResolvedValueOnce(jsonResponse({ results: [{ id: 42, key: 'my-flag', filters: {} }] }))
.mockRejectedValueOnce(new Error('ECONNRESET'));
const step = createStep({ flag: 'my-flag', active: 'true', ignore_error: true });
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await expect(step.executeAsync()).resolves.not.toThrow();
expect(warnMock).toHaveBeenCalledWith(
{ err: expect.any(Error) },
'Updating PostHog flag "my-flag" failed. Ignoring error.'
);
});
it('throws on a 403 update even when ignore_error is set', async () => {
fetchMock
.mockResolvedValueOnce(jsonResponse({ results: [{ id: 42, key: 'my-flag', filters: {} }] }))
.mockResolvedValueOnce(jsonResponse({}, { ok: false, status: 403 }));
const step = createStep({ flag: 'my-flag', active: 'true', ignore_error: true });
await expect(step.executeAsync()).rejects.toThrow(/feature_flag:write/);
});
it('debugs the underlying error before throwing on a network failure by default', async () => {
fetchMock.mockRejectedValueOnce(new Error('ECONNRESET'));
const step = createStep({ flag: 'my-flag', active: 'true' });
const debugMock = jest.spyOn(step.ctx.logger, 'debug');
await expect(step.executeAsync()).rejects.toThrow(/Looking up PostHog flag "my-flag" failed/);
expect(debugMock).toHaveBeenCalled();
});
it('throws on an invalid JSON lookup response by default', async () => {
fetchMock.mockResolvedValueOnce({
ok: true,
status: 200,
json: async () => {
throw new Error('unexpected token');
},
} as unknown as Response);
const step = createStep({ flag: 'my-flag', active: 'true' });
await expect(step.executeAsync()).rejects.toThrow(/returned an unexpected response/);
});
it('warns and skips on an invalid JSON lookup response when ignore_error is set', async () => {
fetchMock.mockResolvedValueOnce({
ok: true,
status: 200,
json: async () => {
throw new Error('unexpected token');
},
} as unknown as Response);
const step = createStep({ flag: 'my-flag', active: 'true', ignore_error: true });
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await expect(step.executeAsync()).resolves.not.toThrow();
expect(warnMock).toHaveBeenCalledWith(
{ err: expect.any(Error) },
'PostHog flag lookup for "my-flag" returned an unexpected response. Ignoring error.'
);
});
it('throws on a non-integer rollout_percentage', async () => {
const step = createStep({ flag: 'my-flag', rollout_percentage: 25.5 });
await expect(step.executeAsync()).rejects.toThrow(/must be an integer between 0 and 100/);
expect(fetchMock).not.toHaveBeenCalled();
});
it('falls back to the default host when POSTHOG_CLI_HOST is unset', async () => {
fetchMock
.mockResolvedValueOnce(jsonResponse({ results: [{ id: 42, key: 'my-flag', filters: {} }] }))
.mockResolvedValueOnce(jsonResponse({ id: 42 }));
const step = createStep(
{ flag: 'my-flag', active: 'true' },
{ POSTHOG_CLI_API_KEY: 'phx_test', POSTHOG_CLI_PROJECT_ID: '123' }
);
await step.executeAsync();
const [searchUrl] = fetchMock.mock.calls[0];
expect(searchUrl).toBe(
'https://us.posthog.com/api/projects/123/feature_flags/?limit=200&search=my-flag'
);
});
});
@@ -1,8 +1,12 @@
import { SystemError } from '@expo/eas-build-job';
import { type bunyan } from '@expo/logger';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import {
MIN_ARGENT_REMOTE_SESSION_VERSION,
waitForArgentToolServerStateAsync,
warnIfArgentPackageVersionCannotBeVerified,
} from '../startArgentRemoteSession';
@@ -70,3 +74,72 @@ describe(warnIfArgentPackageVersionCannotBeVerified, () => {
);
});
});
describe(waitForArgentToolServerStateAsync, () => {
let stateDir: string;
beforeEach(async () => {
stateDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'argent-state-test-'));
});
afterEach(async () => {
await fs.promises.rm(stateDir, { recursive: true, force: true });
});
function stateJson({ port, pid, token }: { port: number; pid: number; token?: string }): string {
return JSON.stringify({ port, pid, token });
}
it.each(['tool-server.json', 'tool-server-012345abcdef.json'])(
'reads a matching process from %s',
async fileName => {
await fs.promises.writeFile(
path.join(stateDir, fileName),
stateJson({ port: 4321, pid: process.pid, token: 'secret' })
);
await expect(
waitForArgentToolServerStateAsync({
stateDir,
ancestorPid: process.ppid,
timeoutMs: 100,
pollIntervalMs: 1,
})
).resolves.toEqual({ port: 4321, pid: process.pid, token: 'secret' });
}
);
it('selects the state whose pid belongs to the launched process tree', async () => {
await fs.promises.writeFile(
path.join(stateDir, 'tool-server-111111111111.json'),
stateJson({ port: 1111, pid: Number.MAX_SAFE_INTEGER })
);
await fs.promises.writeFile(
path.join(stateDir, 'tool-server-222222222222.json'),
stateJson({ port: 2222, pid: process.pid })
);
await expect(
waitForArgentToolServerStateAsync({
stateDir,
ancestorPid: process.ppid,
timeoutMs: 100,
pollIntervalMs: 1,
})
).resolves.toEqual({ port: 2222, pid: process.pid });
});
it('does not return a state file outside the launched process tree', async () => {
const filePath = path.join(stateDir, 'tool-server-012345abcdef.json');
await fs.promises.writeFile(filePath, stateJson({ port: 1111, pid: Number.MAX_SAFE_INTEGER }));
await expect(
waitForArgentToolServerStateAsync({
stateDir,
ancestorPid: process.pid,
timeoutMs: 10,
pollIntervalMs: 1,
})
).rejects.toThrow(`state file belonging to process ${process.pid}`);
});
});
@@ -0,0 +1,146 @@
import { BuildRuntimePlatform } from '@expo/steps';
import fs from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { type CustomBuildContext } from '../../../customBuildContext';
import { uploadDeviceRunSessionArtifactAsync } from '../../utils/deviceRunSessionArtifacts';
import { createUploadDeviceRunSessionScreenRecordingsBuildFunction } from '../uploadDeviceRunSessionScreenRecordings';
jest.mock('../../utils/deviceRunSessionArtifacts', () => ({
uploadDeviceRunSessionArtifactAsync: jest.fn(),
}));
const SIMULATOR_UDID = '01234567-89AB-CDEF-0123-456789ABCDEF';
const SECOND_SIMULATOR_UDID = 'FEDCBA98-7654-3210-FEDC-BA9876543210';
describe(createUploadDeviceRunSessionScreenRecordingsBuildFunction, () => {
beforeEach(() => {
jest
.mocked(uploadDeviceRunSessionArtifactAsync)
.mockReset()
.mockImplementation(async (_ctx, { stream }) => {
await new Promise<void>((resolve, reject) => {
stream.once('end', resolve);
stream.once('error', reject);
stream.resume();
});
});
});
it('uploads simulator device metadata', async () => {
const recordingDirectory = await fs.mkdtemp(path.join(os.tmpdir(), 'ios-recording-test-'));
const recordingPath = path.join(recordingDirectory, 'recording.mp4');
await fs.writeFile(recordingPath, 'recording');
await fs.writeFile(
path.join(recordingDirectory, 'session.json'),
JSON.stringify({
recording: path.basename(recordingPath),
firstFrameWallClock: { iso8601: '2026-07-10T10:00:00.000Z' },
width: 1179,
height: 2556,
})
);
try {
const globalContext = createGlobalContextMock({
runtimePlatform: BuildRuntimePlatform.DARWIN,
});
const buildStep = createUploadDeviceRunSessionScreenRecordingsBuildFunction(
{} as CustomBuildContext
).createBuildStepFromFunctionCall(globalContext, {
env: { DEVICE_RUN_SESSION_ID: 'device-run-session-id' },
callInputs: {
recordings_json: [
{
udid: SIMULATOR_UDID,
deviceName: 'iPhone 16',
runtimeDisplayName: 'iOS 18.6',
directory: recordingDirectory,
},
],
},
});
await buildStep.executeAsync();
expect(uploadDeviceRunSessionArtifactAsync).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({
name: 'iPhone 16 screen recording (01234567-…, started at Jul 10, 2026, 10:00:00.000 UTC)',
metadata: {
__eas_type: 'screen-recording',
__eas_screen_recording: '1',
udid: SIMULATOR_UDID,
deviceName: 'iPhone 16',
runtimeDisplayName: 'iOS 18.6',
firstFrameAt: '2026-07-10T10:00:00.000Z',
width: 1179,
height: 2556,
},
})
);
} finally {
await fs.rm(recordingDirectory, { recursive: true, force: true });
}
});
it('uses unique names for recordings from rebooted and same-model simulators', async () => {
const recordingDirectories = await Promise.all(
['2026-07-10T10:00:00.000Z', '2026-07-10T10:05:00.000Z', '2026-07-10T10:00:00.000Z'].map(
async firstFrameAt => {
const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'ios-recording-test-'));
await fs.writeFile(path.join(directory, 'recording.mp4'), 'recording');
await fs.writeFile(
path.join(directory, 'session.json'),
JSON.stringify({
recording: 'recording.mp4',
firstFrameWallClock: { iso8601: firstFrameAt },
width: 1179,
height: 2556,
})
);
return directory;
}
)
);
try {
const globalContext = createGlobalContextMock({
runtimePlatform: BuildRuntimePlatform.DARWIN,
});
const buildStep = createUploadDeviceRunSessionScreenRecordingsBuildFunction(
{} as CustomBuildContext
).createBuildStepFromFunctionCall(globalContext, {
env: { DEVICE_RUN_SESSION_ID: 'device-run-session-id' },
callInputs: {
recordings_json: recordingDirectories.map((directory, index) => ({
udid: index === 2 ? SECOND_SIMULATOR_UDID : SIMULATOR_UDID,
deviceName: 'iPhone 16',
runtimeDisplayName: 'iOS 18.6',
directory,
})),
},
});
await buildStep.executeAsync();
const names = jest
.mocked(uploadDeviceRunSessionArtifactAsync)
.mock.calls.map(([, options]) => options.name);
expect(names).toEqual(
expect.arrayContaining([
'iPhone 16 screen recording (01234567-…, started at Jul 10, 2026, 10:00:00.000 UTC)',
'iPhone 16 screen recording (01234567-…, started at Jul 10, 2026, 10:05:00.000 UTC)',
'iPhone 16 screen recording (FEDCBA98-…, started at Jul 10, 2026, 10:00:00.000 UTC)',
])
);
expect(new Set(names).size).toBe(3);
} finally {
await Promise.all(
recordingDirectories.map(directory => fs.rm(directory, { recursive: true, force: true }))
);
}
});
});
@@ -0,0 +1,144 @@
import { PipeMode } from '@expo/logger';
import spawn from '@expo/turtle-spawn';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { createUploadPosthogSourcemapsFunction } from '../uploadPosthogSourcemaps';
jest.mock('@expo/logger');
jest.mock('@expo/turtle-spawn');
const spawnMock = jest.mocked(spawn);
const ENV = {
POSTHOG_CLI_API_KEY: 'phx_test',
POSTHOG_CLI_PROJECT_ID: '123',
POSTHOG_CLI_HOST: 'https://us.posthog.com',
};
describe(createUploadPosthogSourcemapsFunction, () => {
const uploadSourcemaps = createUploadPosthogSourcemapsFunction();
afterEach(() => {
jest.resetAllMocks();
});
function createStep(
callInputs: Record<string, unknown> = {},
env: Record<string, string> = ENV
): ReturnType<typeof uploadSourcemaps.createBuildStepFromFunctionCall> {
return uploadSourcemaps.createBuildStepFromFunctionCall(createGlobalContextMock({}), {
callInputs,
env,
id: uploadSourcemaps.id,
});
}
it('runs @posthog/cli hermes upload with the resolved credentials and host', async () => {
spawnMock.mockResolvedValue({ stdout: '', stderr: '' } as unknown as Awaited<
ReturnType<typeof spawn>
>);
const step = createStep();
await step.executeAsync();
expect(spawnMock).toHaveBeenCalledTimes(1);
const [command, args, options] = spawnMock.mock.calls[0];
expect(command).toBe('npx');
expect(args).toEqual([
'-y',
'@posthog/cli',
'--host',
'https://us.posthog.com',
'hermes',
'upload',
'--directory',
'dist',
]);
expect(options?.env).toMatchObject({
POSTHOG_CLI_API_KEY: 'phx_test',
POSTHOG_CLI_PROJECT_ID: '123',
});
expect(options?.mode).toBe(PipeMode.COMBINED_AS_STDOUT);
});
it('passes a custom host through to --host', async () => {
spawnMock.mockResolvedValue({ stdout: '', stderr: '' } as unknown as Awaited<
ReturnType<typeof spawn>
>);
const step = createStep({}, { ...ENV, POSTHOG_CLI_HOST: 'https://eu.i.posthog.com' });
await step.executeAsync();
expect(spawnMock.mock.calls[0][1]).toEqual([
'-y',
'@posthog/cli',
'--host',
'https://eu.i.posthog.com',
'hermes',
'upload',
'--directory',
'dist',
]);
});
it('uploads the given directory', async () => {
spawnMock.mockResolvedValue({ stdout: '', stderr: '' } as unknown as Awaited<
ReturnType<typeof spawn>
>);
const step = createStep({ directory: 'build-output' });
await step.executeAsync();
expect(spawnMock.mock.calls[0][1]).toEqual([
'-y',
'@posthog/cli',
'--host',
'https://us.posthog.com',
'hermes',
'upload',
'--directory',
'build-output',
]);
});
it('throws when credentials are missing by default', async () => {
const step = createStep({}, {});
await expect(step.executeAsync()).rejects.toThrow(
/Missing PostHog credentials: personal API key, project id/
);
expect(spawnMock).not.toHaveBeenCalled();
});
it('warns and skips when credentials are missing and ignore_error is set', async () => {
const step = createStep({ ignore_error: true }, {});
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await step.executeAsync();
expect(spawnMock).not.toHaveBeenCalled();
expect(warnMock).toHaveBeenCalledWith(
{ err: expect.any(Error) },
expect.stringMatching(/Ignoring error/)
);
});
it('throws when the upload fails by default', async () => {
spawnMock.mockRejectedValue(new Error('upload failed'));
const step = createStep();
await expect(step.executeAsync()).rejects.toThrow(/Uploading source maps to PostHog failed/);
});
it('warns and continues when the upload fails and ignore_error is set', async () => {
spawnMock.mockRejectedValue(new Error('upload failed'));
const step = createStep({ ignore_error: true });
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await expect(step.executeAsync()).resolves.not.toThrow();
expect(warnMock).toHaveBeenCalledWith(
{ err: expect.any(Error) },
expect.stringMatching(/Ignoring error/)
);
});
});
@@ -0,0 +1,263 @@
import fetch, { Response } from 'node-fetch';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { createWaitForPosthogMetricFunction } from '../waitForPosthogMetric';
jest.mock('@expo/logger');
jest.mock('node-fetch');
function jsonResponse(data: unknown, { ok = true, status = 200 } = {}): Response {
return {
ok,
status,
json: async () => data,
text: async () => JSON.stringify(data),
} as unknown as Response;
}
const ENV = {
POSTHOG_CLI_API_KEY: 'phx_test',
POSTHOG_CLI_PROJECT_ID: '123',
POSTHOG_CLI_HOST: 'https://us.posthog.com',
};
describe(createWaitForPosthogMetricFunction, () => {
const fetchMock = jest.mocked(fetch);
const waitForMetric = createWaitForPosthogMetricFunction();
afterEach(() => {
jest.resetAllMocks();
});
function createStep(
callInputs: Record<string, unknown>,
env: Record<string, string> = ENV
): ReturnType<typeof waitForMetric.createBuildStepFromFunctionCall> {
return waitForMetric.createBuildStepFromFunctionCall(createGlobalContextMock({}), {
callInputs,
env,
id: waitForMetric.id,
});
}
it('clears when the metric already meets the condition (blocking query)', async () => {
fetchMock.mockResolvedValueOnce(jsonResponse({ results: [[3]] }));
const step = createStep({
query: 'select count() from events',
operator: 'lt',
threshold: '5',
});
await step.executeAsync();
expect(fetchMock).toHaveBeenCalledTimes(1);
const [url, init] = fetchMock.mock.calls[0];
expect(url).toBe('https://us.posthog.com/api/projects/123/query/');
expect(JSON.parse(init?.body as string)).toMatchObject({
query: { kind: 'HogQLQuery' },
refresh: 'blocking',
});
expect(step.outputById.value.value).toBe('3');
});
it('throws when credentials are missing', async () => {
const step = createStep({ query: 'q', operator: 'lt', threshold: '5' }, {});
await expect(step.executeAsync()).rejects.toThrow(
/Missing PostHog credentials: personal API key, project id/
);
expect(fetchMock).not.toHaveBeenCalled();
});
it('throws on an invalid operator', async () => {
const step = createStep({ query: 'q', operator: 'between', threshold: '5' });
await expect(step.executeAsync()).rejects.toThrow(/Invalid "operator"/);
});
it('rejects an Object.prototype key as an operator', async () => {
const step = createStep({ query: 'q', operator: 'toString', threshold: '5' });
await expect(step.executeAsync()).rejects.toThrow(/Invalid "operator"/);
});
it('throws with the PostHog error detail when the query fails to run', async () => {
fetchMock.mockResolvedValueOnce(
jsonResponse({ detail: 'Malformed HogQL query' }, { ok: false, status: 400 })
);
const step = createStep({ query: 'bad', operator: 'lt', threshold: '5' });
await expect(step.executeAsync()).rejects.toThrow(
/failed with status 400: Malformed HogQL query/
);
});
it('retries after a transient 500 and clears when the metric recovers', async () => {
fetchMock
.mockResolvedValueOnce(jsonResponse({}, { ok: false, status: 500 }))
.mockResolvedValueOnce(jsonResponse({ results: [[1]] }));
const step = createStep({
query: 'q',
operator: 'lt',
threshold: '5',
interval_seconds: 0.05,
timeout_seconds: 5,
});
await step.executeAsync();
expect(fetchMock).toHaveBeenCalledTimes(2);
});
it('keeps polling while the query returns no numeric value', async () => {
fetchMock
.mockResolvedValueOnce(jsonResponse({ results: [['not-a-number']] }))
.mockResolvedValueOnce(jsonResponse({ results: [[2]] }));
const step = createStep({
query: 'q',
operator: 'lt',
threshold: '5',
interval_seconds: 0.05,
timeout_seconds: 5,
});
await step.executeAsync();
expect(fetchMock).toHaveBeenCalledTimes(2);
});
it('reports a query-shape problem when no numeric value ever arrives', async () => {
fetchMock.mockResolvedValue(jsonResponse({ results: [['not-a-number']] }));
const step = createStep({
query: 'q',
operator: 'lt',
threshold: '5',
interval_seconds: 0.05,
timeout_seconds: 0.15,
});
await expect(step.executeAsync()).rejects.toThrow(/never returned a numeric value/);
});
it('includes the last observed value in the timeout error', async () => {
fetchMock.mockResolvedValue(jsonResponse({ results: [[10]] }));
const step = createStep({
query: 'q',
operator: 'lt',
threshold: '5',
interval_seconds: 0.05,
timeout_seconds: 0.15,
});
await expect(step.executeAsync()).rejects.toThrow(/last value: 10/);
});
it('throws on a non-positive interval', async () => {
const step = createStep({ query: 'q', operator: 'lt', threshold: '5', interval_seconds: 0 });
await expect(step.executeAsync()).rejects.toThrow(/greater than 0/);
expect(fetchMock).not.toHaveBeenCalled();
});
it.each([
['lte', 3, 3],
['gt', 10, 5],
['gte', 5, 5],
['eq', 3, 3],
] as const)('clears with the %s operator', async (operator, value, threshold) => {
fetchMock.mockResolvedValueOnce(jsonResponse({ results: [[value]] }));
const step = createStep({ query: 'q', operator, threshold });
await step.executeAsync();
expect(fetchMock).toHaveBeenCalledTimes(1);
});
it('treats a non-numeric non-string cell as no value and keeps polling', async () => {
fetchMock
.mockResolvedValueOnce(jsonResponse({ results: [[true]] }))
.mockResolvedValueOnce(jsonResponse({ results: [[1]] }));
const step = createStep({
query: 'q',
operator: 'lt',
threshold: '5',
interval_seconds: 0.05,
timeout_seconds: 5,
});
await step.executeAsync();
expect(fetchMock).toHaveBeenCalledTimes(2);
});
it('retries after a network error and clears when the query recovers', async () => {
fetchMock
.mockRejectedValueOnce(new Error('ECONNRESET'))
.mockResolvedValueOnce(jsonResponse({ results: [[1]] }));
const step = createStep({
query: 'q',
operator: 'lt',
threshold: '5',
interval_seconds: 0.05,
timeout_seconds: 5,
});
const warnMock = jest.spyOn(step.ctx.logger, 'warn');
await step.executeAsync();
expect(fetchMock).toHaveBeenCalledTimes(2);
expect(warnMock).toHaveBeenCalledWith(
'Running the PostHog query failed with a network error; will retry on the next poll.'
);
});
it('retries after an invalid JSON response and clears when the query recovers', async () => {
fetchMock
.mockResolvedValueOnce({
ok: true,
status: 200,
json: async () => {
throw new Error('unexpected token');
},
} as unknown as Response)
.mockResolvedValueOnce(jsonResponse({ results: [[1]] }));
const step = createStep({
query: 'q',
operator: 'lt',
threshold: '5',
interval_seconds: 0.05,
timeout_seconds: 5,
});
await step.executeAsync();
expect(fetchMock).toHaveBeenCalledTimes(2);
});
it('throws on a 403 without retrying', async () => {
fetchMock.mockResolvedValue(jsonResponse({}, { ok: false, status: 403 }));
const step = createStep({ query: 'q', operator: 'lt', threshold: '5' });
await expect(step.executeAsync()).rejects.toThrow(/forbidden \(403\)/);
expect(fetchMock).toHaveBeenCalledTimes(1);
});
it('keeps polling through malformed response shapes until a numeric value arrives', async () => {
fetchMock
.mockResolvedValueOnce(jsonResponse({ results: [[Infinity]] }))
.mockResolvedValueOnce(jsonResponse('not an object'))
.mockResolvedValueOnce(jsonResponse({ results: [5] }))
.mockResolvedValueOnce(jsonResponse({ results: [[1]] }));
const step = createStep({
query: 'q',
operator: 'lt',
threshold: '5',
interval_seconds: 0.05,
timeout_seconds: 5,
});
await step.executeAsync();
expect(fetchMock).toHaveBeenCalledTimes(4);
});
});
@@ -0,0 +1,109 @@
import fetch, { Response } from 'node-fetch';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { createWaitForPosthogQueryFunction } from '../waitForPosthogQuery';
jest.mock('@expo/logger');
jest.mock('node-fetch');
function jsonResponse(data: unknown, { ok = true, status = 200 } = {}): Response {
return { ok, status, json: async () => data } as unknown as Response;
}
const ENV = {
POSTHOG_CLI_API_KEY: 'phx_test',
POSTHOG_CLI_PROJECT_ID: '123',
POSTHOG_CLI_HOST: 'https://us.posthog.com',
};
describe(createWaitForPosthogQueryFunction, () => {
const fetchMock = jest.mocked(fetch);
const waitForQuery = createWaitForPosthogQueryFunction();
afterEach(() => {
jest.resetAllMocks();
});
function createStep(
callInputs: Record<string, unknown>,
env: Record<string, string> = ENV
): ReturnType<typeof waitForQuery.createBuildStepFromFunctionCall> {
return waitForQuery.createBuildStepFromFunctionCall(createGlobalContextMock({}), {
callInputs,
env,
id: waitForQuery.id,
});
}
it('resolves when the query returns true', async () => {
fetchMock.mockResolvedValueOnce(jsonResponse({ results: [[true]] }));
const step = createStep({ query: 'SELECT count() > 0 FROM events' });
await expect(step.executeAsync()).resolves.not.toThrow();
expect(fetchMock).toHaveBeenCalledTimes(1);
const [, init] = fetchMock.mock.calls[0];
expect(JSON.parse(init?.body as string).query).toEqual({
kind: 'HogQLQuery',
query: 'SELECT count() > 0 FROM events',
});
});
it('resolves when the query returns a nonzero number', async () => {
fetchMock.mockResolvedValueOnce(jsonResponse({ results: [[1]] }));
const step = createStep({ query: 'q' });
await expect(step.executeAsync()).resolves.not.toThrow();
});
it('keeps polling until the query turns true', async () => {
fetchMock
.mockResolvedValueOnce(jsonResponse({ results: [[0]] }))
.mockResolvedValueOnce(jsonResponse({ results: [[true]] }));
const step = createStep({ query: 'q', interval_seconds: 0.05, timeout_seconds: 5 });
await step.executeAsync();
expect(fetchMock).toHaveBeenCalledTimes(2);
});
it('keeps polling after a transient failure and clears when the query recovers', async () => {
fetchMock
.mockRejectedValueOnce(new Error('ECONNRESET'))
.mockResolvedValueOnce(jsonResponse({ results: [[true]] }));
const step = createStep({ query: 'q', interval_seconds: 0.05, timeout_seconds: 5 });
await expect(step.executeAsync()).resolves.not.toThrow();
expect(fetchMock).toHaveBeenCalledTimes(2);
});
it('throws when the query never returns true within the timeout', async () => {
fetchMock.mockResolvedValue(jsonResponse({ results: [[false]] }));
const step = createStep({ query: 'q', interval_seconds: 0.05, timeout_seconds: 0.1 });
await expect(step.executeAsync()).rejects.toThrow(/did not return true within/);
});
it('throws when credentials are missing', async () => {
const step = createStep({ query: 'q' }, {});
await expect(step.executeAsync()).rejects.toThrow(
/Missing PostHog credentials: personal API key, project id/
);
expect(fetchMock).not.toHaveBeenCalled();
});
it('throws on a non-positive interval', async () => {
const step = createStep({ query: 'q', interval_seconds: 0 });
await expect(step.executeAsync()).rejects.toThrow(/greater than 0/);
expect(fetchMock).not.toHaveBeenCalled();
});
it('throws on a 403 without retrying', async () => {
fetchMock.mockResolvedValue(jsonResponse({}, { ok: false, status: 403 }));
const step = createStep({ query: 'q' });
await expect(step.executeAsync()).rejects.toThrow(/forbidden \(403\)/);
expect(fetchMock).toHaveBeenCalledTimes(1);
});
});
@@ -0,0 +1,82 @@
import { UserError } from '@expo/eas-build-job';
import { BuildFunction, BuildStepInput, BuildStepInputValueTypeName } from '@expo/steps';
import { PosthogClient } from '../utils/PosthogClient';
import { PosthogUtils } from '../utils/PosthogUtils';
export function createCapturePosthogEventFunction(): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'posthog_capture_event',
name: 'Capture PostHog event',
__metricsId: 'eas/posthog_capture_event',
inputProviders: [
BuildStepInput.createProvider({
id: 'event',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: true,
}),
BuildStepInput.createProvider({
id: 'distinct_id',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: false,
}),
BuildStepInput.createProvider({
id: 'properties',
allowedValueTypeName: BuildStepInputValueTypeName.JSON,
required: false,
}),
BuildStepInput.createProvider({
id: 'api_key',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: false,
}),
BuildStepInput.createProvider({
id: 'host',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: false,
}),
BuildStepInput.createProvider({
id: 'ignore_error',
allowedValueTypeName: BuildStepInputValueTypeName.BOOLEAN,
required: false,
}),
],
fn: async (stepCtx, { inputs, env, signal }) => {
const { logger } = stepCtx;
const ignoreError = Boolean(inputs.ignore_error.value);
const event = inputs.event.value as string;
const client = PosthogClient.forIngestion({
apiKeyOverride: inputs.api_key.value as string | undefined,
hostOverride: inputs.host.value as string | undefined,
env,
});
if (!client) {
PosthogUtils.failOrLogError({
logger,
ignoreError,
error: new UserError(
'EAS_POSTHOG_MISSING_API_KEY',
'PostHog API key not provided. Set the "api_key" input or the EXPO_PUBLIC_POSTHOG_API_KEY environment variable.'
),
});
return;
}
logger.info(`Sending PostHog event "${event}"`);
try {
await client.captureEventAsync({
event,
distinctId: inputs.distinct_id.value as string | undefined,
properties: inputs.properties.value as Record<string, unknown> | undefined,
signal,
});
} catch (error) {
PosthogUtils.failOrLogError({ logger, ignoreError, error });
return;
}
logger.info(`Sent PostHog event "${event}"`);
},
});
}
@@ -7,6 +7,7 @@ export function createCheckoutBuildFunction(): BuildFunction {
id: 'checkout',
name: 'Checkout',
__metricsId: 'eas/checkout',
__hookId: 'checkout',
fn: async stepsCtx => {
if (stepsCtx.global.wasCheckedOut()) {
stepsCtx.logger.info('Project directory is already checked out');
@@ -0,0 +1,78 @@
import { BuildFunction, BuildStepInput, BuildStepInputValueTypeName } from '@expo/steps';
import { PosthogClient } from '../utils/PosthogClient';
import { PosthogUtils } from '../utils/PosthogUtils';
export function createPosthogAnnotationFunction(): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'posthog_annotation',
name: 'Create a PostHog annotation',
__metricsId: 'eas/posthog_annotation',
inputProviders: [
BuildStepInput.createProvider({
id: 'content',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: true,
}),
BuildStepInput.createProvider({
id: 'date_marker',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: false,
}),
BuildStepInput.createProvider({
id: 'api_key',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: false,
}),
BuildStepInput.createProvider({
id: 'project_id',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: false,
}),
BuildStepInput.createProvider({
id: 'ignore_error',
allowedValueTypeName: BuildStepInputValueTypeName.BOOLEAN,
required: false,
}),
],
fn: async (stepCtx, { inputs, env, signal }) => {
const { logger } = stepCtx;
const ignoreError = Boolean(inputs.ignore_error.value);
const result = PosthogClient.fromEnv({
apiKeyOverride: inputs.api_key.value as string | undefined,
projectIdOverride: inputs.project_id.value as string | undefined,
env,
});
if (!result.client) {
PosthogUtils.failOrLogError({
logger,
ignoreError,
error: PosthogUtils.missingCredentialsError(result.missing),
});
return;
}
const client = result.client;
const content = inputs.content.value as string;
const dateMarker =
(inputs.date_marker.value as string | undefined) ?? new Date().toISOString();
logger.info(`Creating PostHog annotation "${content}"`);
try {
await client.requestAsync('POST', '/annotations/', {
action: 'Creating the PostHog annotation',
forbiddenScope: 'annotation:write',
body: { content, date_marker: dateMarker },
signal,
});
} catch (error) {
PosthogUtils.failOrLogError({ logger, ignoreError, error });
return;
}
logger.info('Created PostHog annotation');
},
});
}
@@ -0,0 +1,23 @@
import { BuildFunction, BuildRuntimePlatform, BuildStepOutput } from '@expo/steps';
import { IosSimulatorRecordingUtils } from '../utils/IosSimulatorRecordingUtils';
export function createFinishIosSimulatorRecordingsBuildFunction(): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'finish_ios_simulator_recordings',
name: 'Finish iOS Simulator recordings',
__metricsId: 'eas/finish_ios_simulator_recordings',
supportedRuntimePlatforms: [BuildRuntimePlatform.DARWIN],
outputProviders: [
BuildStepOutput.createProvider({
id: 'recordings_json',
required: true,
}),
],
fn: async ({ logger }, { outputs }) => {
const recordings = await IosSimulatorRecordingUtils.finishAsync({ logger });
outputs.recordings_json.set(JSON.stringify(recordings));
},
});
}
@@ -1,7 +1,7 @@
import { BuildFunction, BuildStepContext, BuildStepEnv } from '@expo/steps';
import path from 'path';
import { installDependenciesWithNpmCacheFallbackAsync } from '../../common/installDependencies';
import { installDependenciesAsync } from '../../common/installDependencies';
import {
findPackagerRootDir,
getPackageVersionFromPackageJson,
@@ -16,6 +16,7 @@ export function createInstallNodeModulesBuildFunction(): BuildFunction {
id: 'install_node_modules',
name: 'Install node modules',
__metricsId: 'eas/install_node_modules',
__hookId: 'install_node_modules',
fn: async (stepCtx, { env }) => {
await installNodeModules(stepCtx, env);
},
@@ -63,7 +64,7 @@ export async function installNodeModules(
packageName: 'react-native',
});
await installDependenciesWithNpmCacheFallbackAsync({
const { spawnPromise } = await installDependenciesAsync({
packageManager,
env,
logger: stepCtx.logger,
@@ -74,4 +75,5 @@ export async function installNodeModules(
reactNativeVersion,
}),
});
await spawnPromise;
}
@@ -122,6 +122,18 @@ async function parseJUnitFile(filePath: string): Promise<JUnitTestCaseResult[]>
}
}
// Raw names (slashes kept) of the testcases a single JUnit file marks failed. Best-effort: an
// unreadable or malformed file yields an empty set rather than throwing.
export async function parseFailedFlowNamesFromJUnitFile(junitFile: string): Promise<Set<string>> {
const failed = new Set<string>();
for (const testcase of await parseJUnitFile(junitFile)) {
if (testcase.status === 'failed') {
failed.add(testcase.name);
}
}
return failed;
}
export async function parseJUnitTestCases(junitDirectory: string): Promise<JUnitTestCaseResult[]> {
let entries: string[];
try {
@@ -8,11 +8,20 @@ export interface ParsedFailureScreenshot {
capturedAtMs: number;
}
// Maestro failure screenshot: `screenshot-[shard-N-]❌-<epochMillis>-(<flowName>).png`.
// pre-v2.7.0 (Maestro <= 2.6.x): failure screenshots are flat files named
// `screenshot-[shard-N-]❌-<epochMillis>-(<flowName>).png` directly in the session dir.
// The flow name may contain parentheses, so anchor on the `-(` after the epoch and the
// `).png` suffix rather than scanning for parens.
// `).png` suffix rather than scanning for parens. Remove this path once the build fleet is
// entirely on Maestro >= 2.7.0.
const FAILURE_SCREENSHOT_PATTERN = /^screenshot-(?:shard-\d+-)?❌-(\d+)-\((.+)\)\.png$/u;
// Maestro >= 2.7.0 bundle layout. Each flow gets its own dir under the session dir, with step
// screenshots under `screenshots/` named `step-<NNN>-<slug>.png` (NNN is a 1-based, zero-padded
// monotonic step sequence). Under the CLI (no `--analyze`) only failed/warned steps produce one.
const STEP_SCREENSHOTS_DIRNAME = 'screenshots';
const STEP_SCREENSHOT_PATTERN = /^step-(\d+)(?:-.*)?\.png$/;
// pre-v2.7.0: parses the legacy flat failure-screenshot filename.
export function parseFailureScreenshotFilename(filename: string): ParsedFailureScreenshot | null {
const match = filename.match(FAILURE_SCREENSHOT_PATTERN);
if (!match) {
@@ -41,65 +50,212 @@ export interface HarvestedScreenshot {
};
}
// Scans every debug dir under testsDirectory whose mtime is recent enough (mtime-based to cover
// the maestro <2.5.0 bug that splits one invocation across two dirs), then keeps only screenshots
// whose own capturedAtMs is at/after capturedSinceMs. Never throws: an unreadable dir is logged
// and skipped so a screenshot harvest can't affect the maestro step outcome.
// Harvests one attempt's failure screenshots from the maestro debug output. Walks each session
// dir under testsDirectory whose mtime is recent enough (mtime-based to also cover the maestro
// <2.5.0 bug that splits one invocation across two dirs). Within a session dir, a child is either
// a flat pre-v2.7.0 screenshot file or a Maestro >= 2.7.0 per-flow bundle dir — the single
// structural seam below. Never throws: an unreadable dir/file is logged and skipped so a
// screenshot harvest can't affect the maestro step outcome.
export async function harvestFailureScreenshotsAsync(args: {
testsDirectory: string;
capturedSinceMs: number;
attemptIndex: number;
// Flow names the JUnit report marks failed for THIS attempt (raw; normalized here). A v2.7.0
// bundle dir carries no failure marker, so this is what tells a failed flow's bundle apart from
// a passing flow that merely left a warned-step screenshot. Unused by the legacy path.
failedFlowNames: ReadonlySet<string>;
logger: bunyan;
}): Promise<HarvestedScreenshot[]> {
// Gate dirs by mtime floored to the second, so a dir created within the same second as the
// attempt start isn't stat'd below the baseline. The exact capturedAtMs gate below is what
// attempt start isn't stat'd below the baseline. The exact capturedAtMs gate per file is what
// prevents mis-attributing an earlier attempt's screenshot.
const dirSinceMtimeMs = Math.floor(args.capturedSinceMs / 1000) * 1000;
let entries: Dirent[];
// Bundle dir names already have '/' -> '_'; normalize the JUnit names the same way to match.
const normalizedFailedFlowNames = new Set([...args.failedFlowNames].map(normalizeFlowName));
let sessionEntries: Dirent[];
try {
entries = await fs.readdir(args.testsDirectory, { withFileTypes: true });
sessionEntries = await fs.readdir(args.testsDirectory, { withFileTypes: true });
} catch (err: any) {
args.logger.info({ err }, `Skipping screenshot harvest: cannot read ${args.testsDirectory}.`);
return [];
}
const results: HarvestedScreenshot[] = [];
for (const entry of entries) {
if (!entry.isDirectory()) {
const legacyShots: HarvestedScreenshot[] = [];
const bundleShots: HarvestedScreenshot[] = [];
for (const sessionEntry of sessionEntries) {
if (!sessionEntry.isDirectory()) {
continue;
}
const dirPath = path.join(args.testsDirectory, entry.name);
let filenames: string[];
const sessionDir = path.join(args.testsDirectory, sessionEntry.name);
let children: Dirent[];
try {
if ((await fs.stat(dirPath)).mtimeMs < dirSinceMtimeMs) {
if ((await fs.stat(sessionDir)).mtimeMs < dirSinceMtimeMs) {
continue;
}
filenames = await fs.readdir(dirPath);
children = await fs.readdir(sessionDir, { withFileTypes: true });
} catch (err: any) {
args.logger.info({ err }, `Skipping unreadable debug dir ${dirPath}.`);
args.logger.info({ err }, `Skipping unreadable debug dir ${sessionDir}.`);
continue;
}
for (const filename of filenames) {
const parsed = parseFailureScreenshotFilename(filename);
// Re-check capture time per file: a dir's mtime can advance after this attempt began (or
// fall within the floored-second dir baseline), so the dir gate alone would re-attribute an
// earlier attempt's screenshot to this one.
if (parsed === null || parsed.capturedAtMs < args.capturedSinceMs) {
continue;
}
results.push({
fileAbsPath: path.join(dirPath, filename),
displayName: `Failure Screenshot: ${parsed.flowName} (attempt ${args.attemptIndex + 1})`,
metadata: {
kind: 'maestro-test-screenshot',
flowName: parsed.flowName,
for (const child of children) {
if (child.isDirectory()) {
// Maestro >= 2.7.0: a per-flow bundle dir.
const shot = await harvestFromBundleAsync({
bundleDir: path.join(sessionDir, child.name),
bundleName: child.name,
normalizedFailedFlowNames,
capturedSinceMs: args.capturedSinceMs,
attemptIndex: args.attemptIndex,
capturedAtMs: parsed.capturedAtMs,
},
});
logger: args.logger,
});
if (shot !== null) {
bundleShots.push(shot);
}
} else {
// pre-v2.7.0: a flat failure screenshot file in the session dir.
const shot = harvestLegacyFlatShot({
fileName: child.name,
sessionDir,
capturedSinceMs: args.capturedSinceMs,
attemptIndex: args.attemptIndex,
});
if (shot !== null) {
legacyShots.push(shot);
}
}
}
}
return results;
// The website shows one screenshot per (flow, attempt). Two v2.7.0 bundle dirs can resolve to
// the same flow — a collision dir (`<flow>-2`) or a duplicate flow name — so keep the latest.
// Legacy shots need no dedupe: a flat filename is emitted once per failed flow, so there is no
// `<flow>-2` collision to collapse (and deduping them would change historical behavior).
return [...legacyShots, ...dedupeBundleShotsByFlowName(bundleShots)];
}
// Maestro >= 2.7.0: resolve a bundle dir to its owning failed flow and return that flow's failure
// screenshot — the highest-numbered step in `screenshots/`. A required failure halts the flow, so
// the failed step is normally the last (highest-numbered) captured step; earlier warned steps have
// lower numbers. Best-effort limitation: without reading Maestro's per-command JSON we can't tell a
// failed-step frame from a warned-step frame, so if the failing command captured nothing (e.g. a
// non-visible command like runScript) but an earlier optional step warned, we return that warned
// frame instead. Returns null when the bundle isn't a failed flow, its name is ambiguous, or it has
// no usable step screenshot. Never throws.
async function harvestFromBundleAsync(args: {
bundleDir: string;
bundleName: string;
normalizedFailedFlowNames: ReadonlySet<string>;
capturedSinceMs: number;
attemptIndex: number;
logger: bunyan;
}): Promise<HarvestedScreenshot | null> {
const flowName = resolveBundleFlowName(args.bundleName, args.normalizedFailedFlowNames);
if (flowName === null) {
return null;
}
const screenshotsDir = path.join(args.bundleDir, STEP_SCREENSHOTS_DIRNAME);
let filenames: string[];
try {
filenames = await fs.readdir(screenshotsDir);
} catch {
// No screenshots/ dir — e.g. the flow failed on a non-visible command, which captures none.
return null;
}
let best: { fileAbsPath: string; stepIndex: number; capturedAtMs: number } | null = null;
for (const filename of filenames) {
const match = filename.match(STEP_SCREENSHOT_PATTERN);
if (match === null) {
continue; // final.png and any non-step file
}
const stepIndex = Number(match[1]);
const fileAbsPath = path.join(screenshotsDir, filename);
let capturedAtMs: number;
try {
capturedAtMs = Math.round((await fs.stat(fileAbsPath)).mtimeMs);
} catch (err: any) {
args.logger.info({ err }, `Skipping unreadable screenshot ${fileAbsPath}.`);
continue;
}
if (capturedAtMs < args.capturedSinceMs) {
continue; // a prior attempt's frame (bundle dir touched later than this attempt started)
}
if (best === null || stepIndex > best.stepIndex) {
best = { fileAbsPath, stepIndex, capturedAtMs };
}
}
if (best === null) {
return null;
}
return {
fileAbsPath: best.fileAbsPath,
displayName: `Failure Screenshot: ${flowName} (attempt ${args.attemptIndex + 1})`,
metadata: {
kind: 'maestro-test-screenshot',
flowName,
attemptIndex: args.attemptIndex,
capturedAtMs: best.capturedAtMs,
},
};
}
// Maestro names a bundle dir `<cleanFlow>` optionally + `-shard-<n>` (sharded) + `-<n>` (name
// collision), where cleanFlow is the flow name with '/' -> '_'. Those suffixes share a namespace
// with legitimate flow names, so we can't strip blindly (a flow may be named `x-shard-1` or `x-2`).
// Enumerate the ways the dir name could peel back to a flow name, keep those that match a failed
// flow, and use the result only if exactly one fits — otherwise skip rather than guess.
function resolveBundleFlowName(
bundleName: string,
normalizedFailedFlowNames: ReadonlySet<string>
): string | null {
const withoutCollision = bundleName.replace(/-\d+$/, '');
const preimages = [
bundleName,
withoutCollision,
withoutCollision.replace(/-shard-\d+$/, ''),
bundleName.replace(/-shard-\d+$/, ''),
];
const matches = new Set(preimages.filter(name => normalizedFailedFlowNames.has(name)));
return matches.size === 1 ? [...matches][0] : null;
}
// Keep one screenshot per flow name (attemptIndex is constant within a harvest call), preferring
// the latest capture when two bundles resolve to the same flow.
function dedupeBundleShotsByFlowName(shots: HarvestedScreenshot[]): HarvestedScreenshot[] {
const byFlowName = new Map<string, HarvestedScreenshot>();
for (const shot of shots) {
const existing = byFlowName.get(shot.metadata.flowName);
if (existing === undefined || shot.metadata.capturedAtMs > existing.metadata.capturedAtMs) {
byFlowName.set(shot.metadata.flowName, shot);
}
}
return [...byFlowName.values()];
}
// pre-v2.7.0 (Maestro <= 2.6.x) reader: a flat `screenshot-...❌...png` file whose name carries the
// flow name and capture epoch.
function harvestLegacyFlatShot(args: {
fileName: string;
sessionDir: string;
capturedSinceMs: number;
attemptIndex: number;
}): HarvestedScreenshot | null {
const parsed = parseFailureScreenshotFilename(args.fileName);
// Re-check capture time per file — a dir's mtime can advance after this attempt began, so the dir
// gate alone would re-attribute an earlier attempt's screenshot to this one.
if (parsed === null || parsed.capturedAtMs < args.capturedSinceMs) {
return null;
}
return {
fileAbsPath: path.join(args.sessionDir, args.fileName),
displayName: `Failure Screenshot: ${parsed.flowName} (attempt ${args.attemptIndex + 1})`,
metadata: {
kind: 'maestro-test-screenshot',
flowName: parsed.flowName,
attemptIndex: args.attemptIndex,
capturedAtMs: parsed.capturedAtMs,
},
};
}
// Maestro substitutes '/' -> '_' in screenshot filenames (so HarvestedScreenshot.flowName
@@ -18,6 +18,7 @@ import {
copyLatestAttemptXml,
junitFileHasFileAttrs,
mergeJUnitReports,
parseFailedFlowNamesFromJUnitFile,
parseFailedFlowsFromFileAttrs,
parseFailedFlowsFromJUnit,
parseJUnitTestCases,
@@ -97,6 +98,7 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
id: 'maestro_tests',
name: 'Run Maestro Tests',
__metricsId: 'eas/maestro_tests',
__hookId: 'maestro_tests',
inputProviders: [
BuildStepInput.createProvider({
id: 'flow_path',
@@ -275,11 +277,15 @@ export function createMaestroTestsBuildFunction(ctx: CustomBuildContext): BuildF
// junit: test-case-result rows (and therefore the summary icons) only exist for junit
// runs, so harvesting other formats would just create orphan artifacts the website hides.
if (outputFormat === 'junit') {
const failedFlowNames = outputPath
? await parseFailedFlowNamesFromJUnitFile(outputPath)
: new Set<string>();
harvested.push(
...(await harvestFailureScreenshotsAsync({
testsDirectory,
capturedSinceMs: attemptStartedAtMs,
attemptIndex: attempt,
failedFlowNames,
logger,
}))
);
@@ -0,0 +1,279 @@
import { UserError } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import { BuildFunction, BuildStepInput, BuildStepInputValueTypeName } from '@expo/steps';
import { z } from 'zod';
import { PosthogClient } from '../utils/PosthogClient';
import { PosthogUtils } from '../utils/PosthogUtils';
const SEARCH_LIMIT = 200;
const FeatureFlagGroupSchema = z.object({ properties: z.unknown().optional() }).passthrough();
const FeatureFlagSchema = z
.object({
id: z.number(),
key: z.string(),
filters: z
.object({
groups: z.array(FeatureFlagGroupSchema).optional(),
payloads: z.record(z.string(), z.unknown()).optional(),
})
.passthrough()
.optional(),
})
.passthrough();
const FlagSearchResponseSchema = z.object({ results: z.array(z.unknown()) });
type FeatureFlagGroup = z.infer<typeof FeatureFlagGroupSchema>;
type FeatureFlagFilters = NonNullable<z.infer<typeof FeatureFlagSchema>['filters']>;
function isCatchAllGroup(group: FeatureFlagGroup): boolean {
const { properties } = group;
return (
properties === undefined ||
properties === null ||
(Array.isArray(properties) && properties.length === 0)
);
}
export function createRolloutPosthogFlagFunction(): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'posthog_flag_rollout',
name: 'Roll out a PostHog feature flag',
__metricsId: 'eas/posthog_flag_rollout',
inputProviders: [
BuildStepInput.createProvider({
id: 'flag',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: true,
}),
BuildStepInput.createProvider({
id: 'active',
allowedValueTypeName: BuildStepInputValueTypeName.BOOLEAN,
required: false,
}),
BuildStepInput.createProvider({
id: 'rollout_percentage',
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
required: false,
}),
BuildStepInput.createProvider({
id: 'payload',
allowedValueTypeName: BuildStepInputValueTypeName.JSON,
required: false,
}),
BuildStepInput.createProvider({
id: 'variant',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: false,
}),
BuildStepInput.createProvider({
id: 'api_key',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: false,
}),
BuildStepInput.createProvider({
id: 'project_id',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: false,
}),
BuildStepInput.createProvider({
id: 'ignore_error',
allowedValueTypeName: BuildStepInputValueTypeName.BOOLEAN,
required: false,
}),
],
fn: async (stepCtx, { inputs, env, signal }) => {
const { logger } = stepCtx;
const ignoreError = Boolean(inputs.ignore_error.value);
const flagKey = inputs.flag.value as string;
const active = inputs.active.value as boolean | undefined;
const rolloutPercentage = inputs.rollout_percentage.value as number | undefined;
const payload = inputs.payload.value as Record<string, unknown> | undefined;
const variant = inputs.variant.value as string | undefined;
if (active === undefined && rolloutPercentage === undefined && payload === undefined) {
throw new UserError(
'EAS_POSTHOG_FLAG_NO_CHANGE',
'You need to provide "active", "rollout_percentage", or "payload" to change the feature flag.'
);
}
if (
rolloutPercentage !== undefined &&
(!Number.isInteger(rolloutPercentage) || rolloutPercentage < 0 || rolloutPercentage > 100)
) {
throw new UserError(
'EAS_POSTHOG_FLAG_INVALID_ROLLOUT',
`"rollout_percentage" must be an integer between 0 and 100, got ${rolloutPercentage}.`
);
}
const result = PosthogClient.fromEnv({
apiKeyOverride: inputs.api_key.value as string | undefined,
projectIdOverride: inputs.project_id.value as string | undefined,
env,
});
if (!result.client) {
PosthogUtils.failOrLogError({
logger,
ignoreError,
error: PosthogUtils.missingCredentialsError(result.missing),
});
return;
}
const client = result.client;
await rolloutPosthogFlagAsync({
logger,
client,
flagKey,
active,
rolloutPercentage,
payload,
variant,
ignoreError,
signal,
});
},
});
}
async function rolloutPosthogFlagAsync({
logger,
client,
flagKey,
active,
rolloutPercentage,
payload,
variant,
ignoreError,
signal,
}: {
logger: bunyan;
client: PosthogClient;
flagKey: string;
active: boolean | undefined;
rolloutPercentage: number | undefined;
payload: Record<string, unknown> | undefined;
variant: string | undefined;
ignoreError: boolean;
signal: AbortSignal | undefined;
}): Promise<void> {
let searchResponse;
try {
searchResponse = await client.requestAsync(
'GET',
`/feature_flags/?limit=${SEARCH_LIMIT}&search=${encodeURIComponent(flagKey)}`,
{
action: `Looking up PostHog flag "${flagKey}"`,
forbiddenScope: 'feature_flag:read',
signal,
}
);
} catch (error) {
PosthogUtils.failOrLogError({ logger, ignoreError, error });
return;
}
let results: unknown[];
try {
results = FlagSearchResponseSchema.parse(await searchResponse.json()).results;
} catch (error) {
PosthogUtils.failOrLogError({
logger,
ignoreError,
error: new UserError(
'EAS_POSTHOG_FLAG_UNEXPECTED_RESPONSE',
`PostHog flag lookup for "${flagKey}" returned an unexpected response.`,
{ cause: error }
),
});
return;
}
const flag = results
.flatMap(result => {
const parsed = FeatureFlagSchema.safeParse(result);
return parsed.success ? [parsed.data] : [];
})
.find(candidate => candidate.key === flagKey);
if (!flag) {
PosthogUtils.failOrLogError({
logger,
ignoreError,
error: new UserError(
'EAS_POSTHOG_FLAG_NOT_FOUND',
`No PostHog feature flag with key "${flagKey}" was found. Check the "flag" input.`
),
});
return;
}
const patch: Record<string, unknown> = {};
if (active !== undefined) {
patch.active = active;
}
let filters = flag.filters;
if (rolloutPercentage !== undefined) {
filters = buildRolloutFilters({ logger, flagKey, filters, rolloutPercentage });
}
if (payload !== undefined) {
filters = {
...filters,
payloads: { ...filters?.payloads, [variant ?? 'true']: JSON.stringify(payload) },
};
}
if (rolloutPercentage !== undefined || payload !== undefined) {
patch.filters = filters;
}
try {
await client.requestAsync('PATCH', `/feature_flags/${flag.id}/`, {
action: `Updating PostHog flag "${flagKey}"`,
forbiddenScope: 'feature_flag:write',
body: patch,
signal,
});
} catch (error) {
PosthogUtils.failOrLogError({ logger, ignoreError, error });
return;
}
logger.info(`Updated PostHog feature flag "${flagKey}".`);
}
function buildRolloutFilters({
logger,
flagKey,
filters,
rolloutPercentage,
}: {
logger: bunyan;
flagKey: string;
filters: FeatureFlagFilters | undefined;
rolloutPercentage: number;
}): FeatureFlagFilters {
const groups = filters?.groups ?? [];
if (groups.length === 0) {
return { ...filters, groups: [{ properties: [], rollout_percentage: rolloutPercentage }] };
}
const catchAllIndex = groups.findIndex(isCatchAllGroup);
const hasCatchAll = catchAllIndex >= 0;
const targetIndex = hasCatchAll ? catchAllIndex : 0;
if (!hasCatchAll) {
logger.warn(
`Flag "${flagKey}" has no catch-all release-condition group; setting rollout_percentage on the first group, which is condition-scoped.`
);
} else if (groups.length > 1) {
logger.warn(
`Flag "${flagKey}" has ${groups.length} release-condition groups; setting rollout_percentage only on the catch-all group to preserve the others.`
);
}
const groupsWithRollout = groups.map((group, index) =>
index === targetIndex ? { ...group, rollout_percentage: rolloutPercentage } : group
);
return { ...filters, groups: groupsWithRollout };
}
@@ -14,6 +14,7 @@ import path from 'node:path';
import { type CustomBuildContext } from '../../customBuildContext';
import { Sentry } from '../../sentry';
import { pollAgentDeviceArtifactsForUploadAsync } from '../utils/agentDeviceArtifacts';
import {
type DetachedProcessHandle,
getDeviceRunSessionIdOrThrow,
@@ -24,14 +25,19 @@ import {
startNgrokTunnelAsync,
startServeSimWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
waitForFileAsync,
} from '../utils/remoteDeviceRunSession';
const AGENT_DEVICE_PACKAGE_NAME = 'agent-device';
const AGENT_DEVICE_REPO_URL = 'https://github.com/callstackincubator/agent-device.git';
const AGENT_DEVICE_REPO_URL = 'https://github.com/callstack/agent-device.git';
const SRC_DIR = '/tmp/agent-device-src';
const DAEMON_JSON_PATH = path.join(os.homedir(), '.agent-device', 'daemon.json');
const STARTUP_TIMEOUT_MS = 60_000;
const AGENT_DEVICE_DAEMON_ENV = {
AGENT_DEVICE_DAEMON_SERVER_MODE: 'http',
AGENT_DEVICE_RETAIN_ARTIFACTS: '1',
};
export function createStartAgentDeviceRemoteSessionBuildFunction(
ctx: CustomBuildContext
@@ -48,7 +54,7 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
],
fn: async ({ logger, global }, { inputs, env }) => {
fn: async ({ logger, global }, { inputs, env, signal }) => {
// Fail fast before any expensive setup if the injected env
// vars are missing: DEVICE_RUN_SESSION_ID (to report the remote config
// back to the API server), EAS_SIMULATOR_NGROK_TUNNEL_DOMAIN (base domain
@@ -109,11 +115,19 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(
},
logger,
});
void pollAgentDeviceArtifactsForUploadAsync(ctx, {
deviceRunSessionId,
daemonUrl: `http://127.0.0.1:${daemonPort}`,
daemonToken,
logger,
});
logger.info('Remote session is live. Keeping the job alive until the session is stopped.');
// Keep the turtle job alive so the daemon and tunnel stay reachable
// until stopDeviceRunSession cancels the run.
await new Promise<never>(() => {});
await waitForDeviceRunSessionStoppedAsync({
ctx,
deviceRunSessionId,
logger,
signal,
});
},
});
}
@@ -144,7 +158,7 @@ async function startAgentDeviceDaemonAsync({
return spawnDetached({
command: 'node',
args: [daemonPath],
env: { ...env, AGENT_DEVICE_DAEMON_SERVER_MODE: 'http' },
env: { ...env, ...AGENT_DEVICE_DAEMON_ENV },
});
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
@@ -201,7 +215,7 @@ async function startAgentDeviceDaemonFromGitAsync({
command: 'bun',
args: ['run', 'src/daemon.ts'],
cwd: SRC_DIR,
env: { ...env, AGENT_DEVICE_DAEMON_SERVER_MODE: 'http' },
env: { ...env, ...AGENT_DEVICE_DAEMON_ENV },
});
}
@@ -14,6 +14,9 @@ import semver from 'semver';
import { z } from 'zod';
import { CustomBuildContext } from '../../customBuildContext';
import { Sentry } from '../../sentry';
import { isProcessDescendantOfAsync } from '../../utils/processes';
import { sleepAsync } from '../../utils/retry';
import { pollArgentArtifactsForUploadAsync } from '../utils/argentArtifacts';
import {
getDeviceRunSessionIdOrThrow,
@@ -24,20 +27,23 @@ import {
startNgrokTunnelAsync,
startServeSimWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForFileAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
const ARGENT_PACKAGE_NAME = '@swmansion/argent';
export const MIN_ARGENT_REMOTE_SESSION_VERSION = '0.12.0';
const ARGENT_ARTIFACTS_LIST_ENDPOINT_FLAG = 'artifacts-list-endpoint';
const ARGENT_STATE_FILE = path.join(os.homedir(), '.argent', 'tool-server.json');
const ARGENT_STATE_DIR = path.join(os.homedir(), '.argent');
const STARTUP_TIMEOUT_MS = 60_000;
const ArgentToolServerStateSchema = z.object({
port: z.number(),
pid: z.number(),
token: z.string().optional(),
});
type ArgentToolServerState = z.infer<typeof ArgentToolServerStateSchema>;
export function createStartArgentRemoteSessionBuildFunction(
ctx: CustomBuildContext
): BuildFunction {
@@ -53,7 +59,7 @@ export function createStartArgentRemoteSessionBuildFunction(
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
],
fn: async ({ logger, global }, { inputs, env }) => {
fn: async ({ logger, global }, { inputs, env, signal }) => {
// Fail fast before any expensive setup if the injected env
// vars are missing: DEVICE_RUN_SESSION_ID (to report the remote config
// back to the API server), EAS_SIMULATOR_NGROK_TUNNEL_DOMAIN (base domain
@@ -74,8 +80,6 @@ export function createStartArgentRemoteSessionBuildFunction(
await selectXcodeDeveloperDirectoryAsync({ env, logger });
}
// Stale state from a previous run would mask the new server's port.
await fs.promises.rm(ARGENT_STATE_FILE, { force: true });
logger.info('Enabling the Argent artifacts list endpoint flag.');
await spawn(
'bunx',
@@ -84,6 +88,8 @@ export function createStartArgentRemoteSessionBuildFunction(
);
logger.info(`Launching ${ARGENT_PACKAGE_NAME}@${versionSpec} tool-server via bunx.`);
// Keep Argent itself in foreground mode under the detached bunx process. This preserves
// the bunx -> Argent CLI -> tool-server ancestry used to identify the matching state file.
const argentServer = spawnDetached({
command: 'bunx',
args: [
@@ -94,20 +100,24 @@ export function createStartArgentRemoteSessionBuildFunction(
'0',
'--idle-timeout',
'0',
'--detach',
'--force',
],
env,
});
if (argentServer.pid === undefined) {
throw new SystemError(
'Failed to start Argent: could not determine the PID of the launched process.'
);
}
logger.info(`Waiting for argent tool-server state at ${ARGENT_STATE_FILE}.`);
logger.info(`Waiting for argent tool-server state in ${ARGENT_STATE_DIR}.`);
let toolServerPort: number;
let toolServerToken: string | undefined;
try {
const toolServerState = await waitForFileAsync({
filePath: ARGENT_STATE_FILE,
const toolServerState = await waitForArgentToolServerStateAsync({
stateDir: ARGENT_STATE_DIR,
ancestorPid: argentServer.pid,
timeoutMs: STARTUP_TIMEOUT_MS,
description: 'argent tool-server state',
parse: parseArgentToolServerState,
});
toolServerPort = toolServerState.port;
toolServerToken = toolServerState.token;
@@ -120,51 +130,69 @@ export function createStartArgentRemoteSessionBuildFunction(
);
}
logger.info(`Argent tool-server is listening on port ${toolServerPort}.`);
void pollArgentArtifactsForUploadAsync(ctx, {
const artifactPollAbortController = new AbortController();
const artifactPollSignal = signal
? AbortSignal.any([signal, artifactPollAbortController.signal])
: artifactPollAbortController.signal;
const artifactPollingPromise = pollArgentArtifactsForUploadAsync(ctx, {
deviceRunSessionId,
toolsUrl: `http://127.0.0.1:${toolServerPort}`,
toolsAuthToken: toolServerToken,
logger,
signal: artifactPollSignal,
});
const publicToolsUrl = await startNgrokTunnelAsync({
port: toolServerPort,
subdomainPrefix: 'argent',
baseDomain: ngrokTunnelDomain,
authtoken: ngrokAuthtoken,
rewriteHostHeader: true,
logger,
});
logger.info(`Tunnel is ready at ${publicToolsUrl}.`);
// serve-sim is iOS-only — Android sessions go without a preview URL.
let webPreviewUrl: string | undefined;
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
const serveSim = await startServeSimWithTunnelAsync(ctx, {
try {
const publicToolsUrl = await startNgrokTunnelAsync({
port: toolServerPort,
subdomainPrefix: 'argent',
baseDomain: ngrokTunnelDomain,
env,
authtoken: ngrokAuthtoken,
rewriteHostHeader: true,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
webPreviewUrl = serveSim.previewUrl;
logger.info(`Web preview URL: ${webPreviewUrl}`);
logger.info(`Tunnel is ready at ${publicToolsUrl}.`);
// serve-sim is iOS-only — Android sessions go without a preview URL.
let webPreviewUrl: string | undefined;
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
const serveSim = await startServeSimWithTunnelAsync(ctx, {
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
webPreviewUrl = serveSim.previewUrl;
logger.info(`Web preview URL: ${webPreviewUrl}`);
}
await uploadRemoteSessionConfigAsync({
ctx,
deviceRunSessionId,
remoteConfig: {
toolsUrl: publicToolsUrl,
...(toolServerToken ? { toolsAuthToken: toolServerToken } : {}),
...(webPreviewUrl ? { webPreviewUrl } : {}),
},
logger,
});
await waitForDeviceRunSessionStoppedAsync({
ctx,
deviceRunSessionId,
logger,
signal,
});
} finally {
artifactPollAbortController.abort();
try {
await artifactPollingPromise;
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not finish Argent remote session artifact polling', error);
logger.warn({ err: error }, 'Could not finish Argent remote session artifact polling.');
}
}
await uploadRemoteSessionConfigAsync({
ctx,
deviceRunSessionId,
remoteConfig: {
toolsUrl: publicToolsUrl,
...(toolServerToken ? { toolsAuthToken: toolServerToken } : {}),
...(webPreviewUrl ? { webPreviewUrl } : {}),
},
logger,
});
logger.info('Remote session is live. Keeping the job alive until the session is stopped.');
// Keep the turtle job alive so the tool-server and tunnel stay reachable
// until stopDeviceRunSession cancels the run.
await new Promise<never>(() => {});
},
});
}
@@ -199,12 +227,64 @@ export function warnIfArgentPackageVersionCannotBeVerified({
}
}
function parseArgentToolServerState(raw: string): { port: number; token?: string } {
const result = ArgentToolServerStateSchema.safeParse(JSON.parse(raw));
function parseArgentToolServerState(raw: string): ArgentToolServerState {
let json: unknown;
try {
json = JSON.parse(raw);
} catch (err) {
throw new SystemError('Expected tool-server state to contain valid JSON.', { cause: err });
}
const result = ArgentToolServerStateSchema.safeParse(json);
if (!result.success) {
throw new SystemError(
`Expected tool-server state to contain { "port": <number>, ... }: ${result.error.message}`
`Expected tool-server state to contain { "port": <number>, "pid": <number>, ... }: ${result.error.message}`
);
}
return result.data;
}
export async function waitForArgentToolServerStateAsync({
stateDir,
ancestorPid,
timeoutMs,
pollIntervalMs = 1_000,
}: {
stateDir: string;
ancestorPid: number;
timeoutMs: number;
pollIntervalMs?: number;
}): Promise<ArgentToolServerState> {
const deadline = Date.now() + timeoutMs;
let lastError: unknown;
while (Date.now() < deadline) {
try {
const stateFileNames = (await fs.promises.readdir(stateDir)).filter(
name => name.startsWith('tool-server') && name.endsWith('.json')
);
for (const stateFileName of stateFileNames) {
try {
const state = parseArgentToolServerState(
await fs.promises.readFile(path.join(stateDir, stateFileName), 'utf8')
);
if (await isProcessDescendantOfAsync(state.pid, ancestorPid)) {
return state;
}
} catch (err) {
if ((err as NodeJS.ErrnoException).code !== 'ENOENT' && !(err instanceof SystemError)) {
throw err;
}
}
}
} catch (err) {
lastError = err;
}
await sleepAsync(pollIntervalMs);
}
throw new SystemError(
`Timed out waiting for an argent tool-server state file belonging to process ${ancestorPid} in ${stateDir}${
lastError instanceof Error ? `: ${lastError.message}` : ''
}.`
);
}
@@ -0,0 +1,19 @@
import { BuildFunction, BuildRuntimePlatform } from '@expo/steps';
import { IosSimulatorRecordingUtils } from '../utils/IosSimulatorRecordingUtils';
export function createStartIosSimulatorRecordingsBuildFunction(): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'start_ios_simulator_recordings',
name: 'Start iOS Simulator recordings',
__metricsId: 'eas/start_ios_simulator_recordings',
supportedRuntimePlatforms: [BuildRuntimePlatform.DARWIN],
fn: async ({ logger }, { env }) => {
await IosSimulatorRecordingUtils.startAsync({
env,
logger,
});
},
});
}
@@ -7,6 +7,7 @@ import {
selectXcodeDeveloperDirectoryAsync,
startServeSimWithTunnelAsync,
uploadRemoteSessionConfigAsync,
waitForDeviceRunSessionStoppedAsync,
} from '../utils/remoteDeviceRunSession';
const STARTUP_TIMEOUT_MS = 60_000;
@@ -20,7 +21,7 @@ export function createStartServeSimRemoteSessionBuildFunction(
name: 'Start serve-sim remote session',
__metricsId: 'eas/start_serve_sim_remote_session',
supportedRuntimePlatforms: [BuildRuntimePlatform.DARWIN],
fn: async ({ logger }, { env }) => {
fn: async ({ logger }, { env, signal }) => {
const deviceRunSessionId = getDeviceRunSessionIdOrThrow(env);
const ngrokTunnelDomain = getNgrokTunnelDomainOrThrow(env);
@@ -28,26 +29,27 @@ export function createStartServeSimRemoteSessionBuildFunction(
await selectXcodeDeveloperDirectoryAsync({ env, logger });
const { previewUrl, streamUrl } = await startServeSimWithTunnelAsync(ctx, {
const { previewUrl } = await startServeSimWithTunnelAsync(ctx, {
baseDomain: ngrokTunnelDomain,
env,
logger,
timeoutMs: STARTUP_TIMEOUT_MS,
});
logger.info(`Preview URL: ${previewUrl}`);
logger.info(`Stream URL: ${streamUrl}`);
await uploadRemoteSessionConfigAsync({
ctx,
deviceRunSessionId,
remoteConfig: { previewUrl, streamUrl },
remoteConfig: { previewUrl },
logger,
});
logger.info('Remote session is live. Keeping the job alive until the session is stopped.');
// Keep the turtle job alive so the serve-sim tunnel stays reachable
// until stopDeviceRunSession cancels the run.
await new Promise<never>(() => {});
await waitForDeviceRunSessionStoppedAsync({
ctx,
deviceRunSessionId,
logger,
signal,
});
},
});
}
@@ -0,0 +1,133 @@
import { SystemError } from '@expo/eas-build-job';
import {
BuildFunction,
BuildRuntimePlatform,
BuildStepInput,
BuildStepInputValueTypeName,
} from '@expo/steps';
import { createReadStream } from 'node:fs';
import { readFile, stat } from 'node:fs/promises';
import path from 'node:path';
import limitFactory from 'promise-limit';
import { z } from 'zod';
import { type CustomBuildContext } from '../../customBuildContext';
import { Sentry } from '../../sentry';
import { formatBytes } from '../../utils/artifacts';
import { uploadDeviceRunSessionArtifactAsync } from '../utils/deviceRunSessionArtifacts';
import { getDeviceRunSessionIdOrThrow } from '../utils/remoteDeviceRunSession';
const RecordingsSchema = z.array(
z.object({
udid: z.string(),
deviceName: z.string(),
runtimeDisplayName: z.string(),
directory: z.string(),
})
);
const RecordingManifestSchema = z.object({
firstFrameWallClock: z.object({
iso8601: z.string(),
}),
width: z.number().int().positive(),
height: z.number().int().positive(),
recording: z.string(),
});
const recordingStartTimeFormatter = new Intl.DateTimeFormat('en-US', {
year: 'numeric',
month: 'short',
day: 'numeric',
hour: '2-digit',
minute: '2-digit',
second: '2-digit',
fractionalSecondDigits: 3,
hourCycle: 'h23',
timeZone: 'UTC',
timeZoneName: 'short',
});
export function createUploadDeviceRunSessionScreenRecordingsBuildFunction(
ctx: CustomBuildContext
): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'upload_device_run_session_screen_recordings',
name: 'Upload device run session screen recordings',
__metricsId: 'eas/upload_device_run_session_screen_recordings',
supportedRuntimePlatforms: [BuildRuntimePlatform.DARWIN],
inputProviders: [
BuildStepInput.createProvider({
id: 'recordings_json',
required: false,
allowedValueTypeName: BuildStepInputValueTypeName.JSON,
}),
],
fn: async ({ logger }, { env, inputs }) => {
const result = RecordingsSchema.safeParse(inputs.recordings_json.value ?? []);
if (!result.success) {
throw new SystemError('Invalid iOS Simulator recordings input.', {
cause: result.error,
});
}
const recordings = result.data;
if (recordings.length === 0) {
logger.info('No iOS Simulator recordings found; skipping uploads.');
return;
}
const deviceRunSessionId = getDeviceRunSessionIdOrThrow(env);
const limit = limitFactory(5);
await Promise.all(
recordings.map(recording =>
limit(async () => {
try {
const metadata = RecordingManifestSchema.parse(
JSON.parse(await readFile(path.join(recording.directory, 'session.json'), 'utf-8'))
);
const startedAt = recordingStartTimeFormatter.format(
new Date(metadata.firstFrameWallClock.iso8601)
);
const shortUdid = `${recording.udid.slice(0, 8)}-…`;
const displayName = `${recording.deviceName} screen recording (${shortUdid}, started at ${startedAt})`;
const recordingPath = path.join(recording.directory, metadata.recording);
const { size } = await stat(recordingPath);
const recordingId = path.basename(recording.directory);
logger.info(
`Uploading screen recording for ${recording.deviceName} (${formatBytes(size)}).`
);
await uploadDeviceRunSessionArtifactAsync(ctx, {
deviceRunSessionId,
artifactId: recordingId,
name: displayName,
filename: `${recordingId}.mp4`,
kind: 'screen-recording',
metadata: {
__eas_type: 'screen-recording',
__eas_screen_recording: '1',
udid: recording.udid,
deviceName: recording.deviceName,
runtimeDisplayName: recording.runtimeDisplayName,
firstFrameAt: metadata.firstFrameWallClock.iso8601,
width: metadata.width,
height: metadata.height,
},
size,
stream: createReadStream(recordingPath),
});
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not upload iOS Simulator screen recording', error);
logger.warn(
{ err: error },
`Could not upload screen recording for ${recording.deviceName}.`
);
}
})
)
);
},
});
}
@@ -0,0 +1,89 @@
import { UserError } from '@expo/eas-build-job';
import { PipeMode } from '@expo/logger';
import { BuildFunction, BuildStepInput, BuildStepInputValueTypeName } from '@expo/steps';
import spawn from '@expo/turtle-spawn';
import { PosthogClient } from '../utils/PosthogClient';
import { PosthogUtils } from '../utils/PosthogUtils';
export function createUploadPosthogSourcemapsFunction(): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'posthog_upload_sourcemaps',
name: 'Upload source maps to PostHog',
__metricsId: 'eas/posthog_upload_sourcemaps',
inputProviders: [
BuildStepInput.createProvider({
id: 'directory',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: false,
defaultValue: 'dist',
}),
BuildStepInput.createProvider({
id: 'api_key',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: false,
}),
BuildStepInput.createProvider({
id: 'project_id',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: false,
}),
BuildStepInput.createProvider({
id: 'ignore_error',
allowedValueTypeName: BuildStepInputValueTypeName.BOOLEAN,
required: false,
}),
],
fn: async (stepCtx, { inputs, env, signal }) => {
const { logger } = stepCtx;
const ignoreError = Boolean(inputs.ignore_error.value);
const result = PosthogClient.fromEnv({
apiKeyOverride: inputs.api_key.value as string | undefined,
projectIdOverride: inputs.project_id.value as string | undefined,
env,
});
if (!result.client) {
PosthogUtils.failOrLogError({
logger,
ignoreError,
error: PosthogUtils.missingCredentialsError(result.missing),
});
return;
}
const client = result.client;
const directory = inputs.directory.value as string;
const { host, env: cliEnv } = client.cliConfig();
logger.info(`Uploading source maps from "${directory}" to PostHog`);
try {
await spawn(
'npx',
['-y', '@posthog/cli', '--host', host, 'hermes', 'upload', '--directory', directory],
{
logger,
// @posthog/cli writes all its logs, progress and success included, to stderr; tag them as stdout so they don't surface as build errors.
mode: PipeMode.COMBINED_AS_STDOUT,
cwd: stepCtx.workingDirectory,
env: { ...env, ...cliEnv },
signal,
}
);
} catch (error) {
PosthogUtils.failOrLogError({
logger,
ignoreError,
error: new UserError(
'EAS_POSTHOG_SOURCEMAPS_FAILED',
'Uploading source maps to PostHog failed.',
{ cause: error }
),
});
return;
}
logger.info('Uploaded source maps to PostHog');
},
});
}
@@ -23,6 +23,7 @@ export function createUploadToAscBuildFunction(): BuildFunction {
id: 'upload_to_asc',
name: 'Upload to App Store Connect',
__metricsId: 'eas/upload_to_asc',
__hookId: 'submit',
inputProviders: [
BuildStepInput.createProvider({
id: 'ipa_path',
@@ -0,0 +1,185 @@
import { UserError } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import {
BuildFunction,
BuildStepInput,
BuildStepInputValueTypeName,
BuildStepOutput,
} from '@expo/steps';
import { PosthogClient, PosthogRetryableError } from '../utils/PosthogClient';
import { PosthogUtils } from '../utils/PosthogUtils';
const OPERATORS = {
lt: { symbol: '<', test: (value: number, threshold: number) => value < threshold },
lte: { symbol: '<=', test: (value: number, threshold: number) => value <= threshold },
gt: { symbol: '>', test: (value: number, threshold: number) => value > threshold },
gte: { symbol: '>=', test: (value: number, threshold: number) => value >= threshold },
eq: { symbol: '=', test: (value: number, threshold: number) => value === threshold },
};
type Operator = keyof typeof OPERATORS;
export function createWaitForPosthogMetricFunction(): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'posthog_wait_for_metric',
name: 'Wait for a PostHog metric',
__metricsId: 'eas/posthog_wait_for_metric',
inputProviders: [
BuildStepInput.createProvider({
id: 'query',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: true,
}),
BuildStepInput.createProvider({
id: 'operator',
allowedValues: Object.keys(OPERATORS),
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: true,
}),
BuildStepInput.createProvider({
id: 'threshold',
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
required: true,
}),
BuildStepInput.createProvider({
id: 'timeout_seconds',
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
required: false,
defaultValue: PosthogUtils.DEFAULT_POLL_TIMEOUT_SECONDS,
}),
BuildStepInput.createProvider({
id: 'interval_seconds',
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
required: false,
defaultValue: PosthogUtils.DEFAULT_POLL_INTERVAL_SECONDS,
}),
BuildStepInput.createProvider({
id: 'api_key',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: false,
}),
BuildStepInput.createProvider({
id: 'project_id',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: false,
}),
],
outputProviders: [BuildStepOutput.createProvider({ id: 'value', required: false })],
fn: async (stepCtx, { inputs, outputs, env, signal }) => {
const { logger } = stepCtx;
const operator = inputs.operator.value as string;
if (!Object.keys(OPERATORS).includes(operator)) {
throw new UserError(
'EAS_POSTHOG_METRIC_INVALID_OPERATOR',
`Invalid "operator" "${operator}". Must be one of: ${Object.keys(OPERATORS).join(', ')}.`
);
}
const timeoutSeconds = inputs.timeout_seconds.value as number;
const intervalSeconds = inputs.interval_seconds.value as number;
PosthogUtils.assertPollBoundsPositive({
timeoutSeconds,
intervalSeconds,
errorCode: 'EAS_POSTHOG_METRIC_INVALID_INTERVAL',
});
const result = PosthogClient.fromEnv({
apiKeyOverride: inputs.api_key.value as string | undefined,
projectIdOverride: inputs.project_id.value as string | undefined,
env,
});
if (!result.client) {
throw PosthogUtils.missingCredentialsError(result.missing);
}
const client = result.client;
const value = await waitForPosthogMetricAsync({
logger,
client,
query: inputs.query.value as string,
operator: operator as Operator,
threshold: inputs.threshold.value as number,
timeoutSeconds,
intervalSeconds,
signal,
});
outputs.value.set(String(value));
},
});
}
async function waitForPosthogMetricAsync({
logger,
client,
query,
operator,
threshold,
timeoutSeconds,
intervalSeconds,
signal,
}: {
logger: bunyan;
client: PosthogClient;
query: string;
operator: Operator;
threshold: number;
timeoutSeconds: number;
intervalSeconds: number;
signal: AbortSignal | undefined;
}): Promise<number> {
const deadline = Date.now() + timeoutSeconds * 1000;
const { symbol, test } = OPERATORS[operator];
const target = `${symbol} ${threshold}`;
let lastValue: number | undefined;
logger.info(
`Waiting for the PostHog metric to reach ${target}. Checking every ${intervalSeconds}s for up to ${timeoutSeconds}s.`
);
for (;;) {
let value: number | undefined = undefined;
try {
value = await queryMetricAsync({ logger, client, query, signal });
} catch (error) {
if (!(error instanceof PosthogRetryableError)) {
throw error;
}
}
if (value !== undefined) {
lastValue = value;
if (test(value, threshold)) {
logger.info(`Metric is ${value}. Target ${target} met.`);
return value;
}
logger.info(`Metric is ${value}. Still waiting for ${target}.`);
} else {
logger.info('The query has not returned a numeric value yet. Still waiting.');
}
if (!(await PosthogUtils.waitForNextPollAsync({ intervalSeconds, deadline, signal }))) {
throw new UserError(
'EAS_POSTHOG_METRIC_TIMEOUT',
lastValue !== undefined
? `The PostHog metric did not reach ${target} within ${timeoutSeconds}s (last value: ${lastValue}).`
: `The PostHog query never returned a numeric value within ${timeoutSeconds}s. Check that the query returns a single number and that PostHog is reachable.`
);
}
}
}
async function queryMetricAsync({
logger,
client,
query,
signal,
}: {
logger: bunyan;
client: PosthogClient;
query: string;
signal: AbortSignal | undefined;
}): Promise<number | undefined> {
const cell = await client.runQueryAsync(query, logger, signal);
return typeof cell === 'number' && Number.isFinite(cell) ? cell : undefined;
}
@@ -0,0 +1,120 @@
import { UserError } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import { BuildFunction, BuildStepInput, BuildStepInputValueTypeName } from '@expo/steps';
import { PosthogClient, PosthogRetryableError } from '../utils/PosthogClient';
import { PosthogUtils } from '../utils/PosthogUtils';
export function createWaitForPosthogQueryFunction(): BuildFunction {
return new BuildFunction({
namespace: 'eas',
id: 'posthog_wait_for_query',
name: 'Wait for a PostHog query',
__metricsId: 'eas/posthog_wait_for_query',
inputProviders: [
BuildStepInput.createProvider({
id: 'query',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: true,
}),
BuildStepInput.createProvider({
id: 'timeout_seconds',
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
required: false,
defaultValue: PosthogUtils.DEFAULT_POLL_TIMEOUT_SECONDS,
}),
BuildStepInput.createProvider({
id: 'interval_seconds',
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
required: false,
defaultValue: PosthogUtils.DEFAULT_POLL_INTERVAL_SECONDS,
}),
BuildStepInput.createProvider({
id: 'api_key',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: false,
}),
BuildStepInput.createProvider({
id: 'project_id',
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
required: false,
}),
],
fn: async (stepCtx, { inputs, env, signal }) => {
const { logger } = stepCtx;
const timeoutSeconds = inputs.timeout_seconds.value as number;
const intervalSeconds = inputs.interval_seconds.value as number;
PosthogUtils.assertPollBoundsPositive({
timeoutSeconds,
intervalSeconds,
errorCode: 'EAS_POSTHOG_QUERY_INVALID_INTERVAL',
});
const result = PosthogClient.fromEnv({
apiKeyOverride: inputs.api_key.value as string | undefined,
projectIdOverride: inputs.project_id.value as string | undefined,
env,
});
if (!result.client) {
throw PosthogUtils.missingCredentialsError(result.missing);
}
const client = result.client;
await waitForPosthogQueryAsync({
logger,
client,
query: inputs.query.value as string,
timeoutSeconds,
intervalSeconds,
signal,
});
},
});
}
async function waitForPosthogQueryAsync({
logger,
client,
query,
timeoutSeconds,
intervalSeconds,
signal,
}: {
logger: bunyan;
client: PosthogClient;
query: string;
timeoutSeconds: number;
intervalSeconds: number;
signal: AbortSignal | undefined;
}): Promise<void> {
const deadline = Date.now() + timeoutSeconds * 1000;
logger.info(
`Waiting for the PostHog query to return true. Checking every ${intervalSeconds}s for up to ${timeoutSeconds}s.`
);
for (;;) {
let cell: unknown;
try {
cell = await client.runQueryAsync(query, logger, signal);
} catch (error) {
if (!(error instanceof PosthogRetryableError)) {
throw error;
}
cell = undefined;
}
if (cell === true || (typeof cell === 'number' && cell !== 0)) {
logger.info('Query returned true.');
return;
}
logger.info('Query is not true yet. Still waiting.');
if (!(await PosthogUtils.waitForNextPollAsync({ intervalSeconds, deadline, signal }))) {
throw new UserError(
'EAS_POSTHOG_QUERY_TIMEOUT',
`The PostHog query did not return true within ${timeoutSeconds}s. It must select a single boolean, e.g. "SELECT count() > 100 FROM events".`
);
}
}
}
@@ -0,0 +1,312 @@
import { type Env } from '@expo/eas-build-job';
import { type bunyan } from '@expo/logger';
import spawn from '@expo/turtle-spawn';
import { type ChildProcess } from 'node:child_process';
import { randomUUID } from 'node:crypto';
import { access, mkdir, mkdtemp } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { setTimeout } from 'node:timers/promises';
import { Sentry } from '../../sentry';
import { IosSimulatorUtils, type IosSimulatorUuid } from '../../utils/IosSimulatorUtils';
const IOS_SIMULATOR_RECORDING_POLL_INTERVAL_MS = 2_000;
const RECORD_SIM_FINISH_TIMEOUT_MS = 70_000;
const RECORD_SIM_FORCE_STOP_TIMEOUT_MS = 5_000;
const RECORD_SIM_MAX_ATTEMPTS_PER_BOOT = 3;
const RECORD_SIM_COMMAND = 'record-sim';
type IosSimulatorRecording = {
id: string;
udid: IosSimulatorUuid;
deviceName: string;
runtimeDisplayName: string;
outputDirectory: string;
startedAt: Date;
getOutput: () => string;
};
type ActiveIosSimulatorRecording = IosSimulatorRecording & {
recordingProcess: ChildProcess;
completionPromise: Promise<void>;
};
type IosSimulatorRecordingSession = {
env: Env;
logger: bunyan;
recordSimCommand: string;
recordingsRootDirectory: string;
activeRecordings: Map<IosSimulatorUuid, ActiveIosSimulatorRecording>;
completedRecordings: IosSimulatorRecording[];
recordingFailureCounts: Map<IosSimulatorUuid, number>;
pollingPromise: Promise<void>;
abortController: AbortController;
};
let activeIosSimulatorRecordingSession: IosSimulatorRecordingSession | null = null;
export namespace IosSimulatorRecordingUtils {
export async function startAsync({ env, logger }: { env: Env; logger: bunyan }): Promise<void> {
if (activeIosSimulatorRecordingSession) {
logger.info('iOS Simulator screen recording polling is already running.');
return;
}
const recordSimCommand = await resolveRecordSimCommandAsync({ env });
if (!recordSimCommand) {
logger.warn(
'record-sim binary is not available; iOS Simulator screen recordings are disabled.'
);
return;
}
const recordingsRootDirectory = await mkdtemp(
path.join(os.tmpdir(), 'ios-simulator-recordings-')
);
const session: IosSimulatorRecordingSession = {
env,
logger,
recordSimCommand,
recordingsRootDirectory,
activeRecordings: new Map(),
completedRecordings: [],
recordingFailureCounts: new Map(),
pollingPromise: Promise.resolve(),
abortController: new AbortController(),
};
activeIosSimulatorRecordingSession = session;
logger.info('Started polling iOS Simulators for screen recordings.');
session.pollingPromise = pollIosSimulatorRecordingsAsync(session).catch(err => {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('iOS Simulator screen recording poller failed', error);
logger.warn({ err: error }, 'iOS Simulator screen recording poller failed.');
});
}
export async function finishAsync({ logger }: { logger: bunyan }): Promise<
{
udid: IosSimulatorUuid;
deviceName: string;
runtimeDisplayName: string;
directory: string;
}[]
> {
const session = activeIosSimulatorRecordingSession;
if (!session) {
logger.info('No iOS Simulator screen recordings are running.');
return [];
}
activeIosSimulatorRecordingSession = null;
session.abortController.abort();
await session.pollingPromise;
await Promise.all(
[...session.activeRecordings.values()].map(async recording => {
logger.info(`Stopping screen recording for ${recording.deviceName}.`);
recording.recordingProcess.kill('SIGINT');
const finished = await Promise.race([
recording.completionPromise.then(() => true),
setTimeout(RECORD_SIM_FINISH_TIMEOUT_MS, false, { ref: false }),
]);
if (finished) {
return;
}
const recordSimOutput = recording.getOutput().trim();
const finishTimeoutSeconds = Math.round(RECORD_SIM_FINISH_TIMEOUT_MS / 1_000);
logger.warn(
{ recordSimOutput },
`Screen recording for ${recording.deviceName} did not finish within ${finishTimeoutSeconds} seconds and will be stopped.${
recordSimOutput
? `\nRecent recorder messages:\n${recordSimOutput}`
: '\nNo recorder messages were captured.'
}`
);
recording.recordingProcess.kill('SIGKILL');
const killed = await Promise.race([
recording.completionPromise.then(() => true),
setTimeout(RECORD_SIM_FORCE_STOP_TIMEOUT_MS, false, { ref: false }),
]);
if (!killed) {
logger.warn(
`iOS Simulator recording process for ${recording.deviceName} did not exit after SIGKILL.`
);
Sentry.capture(
`iOS Simulator recording process for ${recording.deviceName} did not exit after SIGKILL.`,
{
extras: {
output: recording.getOutput(),
},
}
);
}
})
);
const completedRecordings = [...session.completedRecordings].sort(
(a, b) => a.startedAt.getTime() - b.startedAt.getTime()
);
return completedRecordings.map(recording => ({
udid: recording.udid,
deviceName: recording.deviceName,
runtimeDisplayName: recording.runtimeDisplayName,
directory: recording.outputDirectory,
}));
}
}
async function pollIosSimulatorRecordingsAsync(
session: IosSimulatorRecordingSession
): Promise<void> {
let listDevicesErrorCount = 0;
const signal = session.abortController.signal;
while (!signal.aborted) {
try {
const bootedDevices = await IosSimulatorUtils.getAvailableDevicesAsync({
env: session.env,
filter: 'booted',
});
if (signal.aborted) {
break;
}
listDevicesErrorCount = 0;
const bootedUdids = new Set(bootedDevices.map(device => device.udid));
for (const udid of session.recordingFailureCounts.keys()) {
if (!bootedUdids.has(udid)) {
session.recordingFailureCounts.delete(udid);
}
}
for (const device of bootedDevices) {
if (
session.activeRecordings.has(device.udid) ||
(session.recordingFailureCounts.get(device.udid) ?? 0) >= RECORD_SIM_MAX_ATTEMPTS_PER_BOOT
) {
continue;
}
await startIosSimulatorRecordingAsync(session, {
udid: device.udid,
deviceName: device.name,
runtimeDisplayName: device.runtimeDisplayName,
});
}
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
listDevicesErrorCount += 1;
if (listDevicesErrorCount === 1 || listDevicesErrorCount % 5 === 0) {
Sentry.capture('Could not poll iOS Simulators for screen recordings', error);
session.logger.warn(
{ err: error, failedSimulatorListCount: listDevicesErrorCount },
'Could not poll iOS Simulators for screen recordings.'
);
}
}
if (!signal.aborted) {
try {
await setTimeout(IOS_SIMULATOR_RECORDING_POLL_INTERVAL_MS, undefined, {
signal,
});
} catch (err) {
if (!signal.aborted) {
throw err;
}
}
}
}
}
async function startIosSimulatorRecordingAsync(
session: IosSimulatorRecordingSession,
{
udid,
deviceName,
runtimeDisplayName,
}: {
udid: IosSimulatorUuid;
deviceName: string;
runtimeDisplayName: string;
}
): Promise<void> {
const startedAt = new Date();
const recordingId = randomUUID();
const outputDirectory = path.join(session.recordingsRootDirectory, recordingId);
await mkdir(outputDirectory, { recursive: true });
session.logger.info(`Starting screen recording for ${deviceName}.`);
const recordingSpawn = spawn(
session.recordSimCommand,
['--udid', udid, '--output', outputDirectory, '--segment-duration', '0'],
{
env: session.env,
stdio: ['ignore', 'pipe', 'pipe'],
}
);
const getOutput = captureProcessOutput(recordingSpawn.child);
const completionPromise = recordingSpawn
.then(() => undefined)
.catch((err: unknown) => {
session.recordingFailureCounts.set(udid, (session.recordingFailureCounts.get(udid) ?? 0) + 1);
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('iOS Simulator screen recording process failed', error);
session.logger.warn(
{ err: error, recordSimOutput: getOutput() },
`Screen recording process failed for ${deviceName}.`
);
})
.finally(() => {
session.activeRecordings.delete(udid);
session.completedRecordings.push({
id: recordingId,
udid,
deviceName,
runtimeDisplayName,
outputDirectory,
startedAt,
getOutput,
});
});
session.activeRecordings.set(udid, {
id: recordingId,
udid,
deviceName,
runtimeDisplayName,
outputDirectory,
recordingProcess: recordingSpawn.child,
completionPromise,
startedAt,
getOutput,
});
}
async function resolveRecordSimCommandAsync({ env }: { env: Env }): Promise<string | null> {
try {
await spawn('which', [RECORD_SIM_COMMAND], { env });
return RECORD_SIM_COMMAND;
} catch {}
const packagedRecordSimPath = path.join(__dirname, '..', '..', '..', 'bin', RECORD_SIM_COMMAND);
try {
await access(packagedRecordSimPath);
return packagedRecordSimPath;
} catch {
return null;
}
}
function captureProcessOutput(recordingProcess: ChildProcess): () => string {
let output = '';
const appendChunk = (chunk: Buffer | string): void => {
// Keep enough recorder stderr/stdout for diagnostics without retaining unbounded output.
output = `${output}${chunk.toString()}`.slice(-16_384);
};
recordingProcess.stdout?.on('data', appendChunk);
recordingProcess.stderr?.on('data', appendChunk);
return () => output;
}
@@ -0,0 +1,226 @@
import { UserError } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import { type BuildStepEnv } from '@expo/steps';
import fetch, { Response } from 'node-fetch';
import { z } from 'zod';
import { PosthogUtils } from './PosthogUtils';
const POSTHOG_DEFAULT_HOST = 'https://us.posthog.com';
const SYSTEM_DISTINCT_ID = 'eas-workflow';
const QueryResponseSchema = z.object({ results: z.array(z.array(z.unknown())) });
export class PosthogRetryableError extends Error {}
export class PosthogClient {
private constructor(
private readonly host: string,
private readonly apiKey: string,
private readonly projectId: string
) {}
static fromEnv({
apiKeyOverride,
projectIdOverride,
env,
}: {
apiKeyOverride: string | undefined;
projectIdOverride: string | undefined;
env: BuildStepEnv;
}): { client: PosthogClient } | { client: undefined; missing: PosthogUtils.Credential[] } {
const apiKey = apiKeyOverride || env.POSTHOG_CLI_API_KEY;
const projectId = projectIdOverride || env.POSTHOG_CLI_PROJECT_ID;
if (!apiKey || !projectId) {
return {
client: undefined,
missing: [
...(apiKey ? [] : [PosthogUtils.API_CREDENTIALS.apiKey]),
...(projectId ? [] : [PosthogUtils.API_CREDENTIALS.projectId]),
],
};
}
return {
client: new PosthogClient(
(env.POSTHOG_CLI_HOST || POSTHOG_DEFAULT_HOST).replace(/\/+$/, ''),
apiKey,
projectId
),
};
}
static forIngestion({
apiKeyOverride,
hostOverride,
env,
}: {
apiKeyOverride: string | undefined;
hostOverride: string | undefined;
env: BuildStepEnv;
}): PosthogClient | undefined {
const apiKey = apiKeyOverride || env.EXPO_PUBLIC_POSTHOG_API_KEY || env.POSTHOG_API_KEY;
if (!apiKey) {
return undefined;
}
return new PosthogClient(
(hostOverride || env.EXPO_PUBLIC_POSTHOG_HOST || POSTHOG_DEFAULT_HOST).replace(/\/+$/, ''),
apiKey,
''
);
}
async captureEventAsync({
event,
distinctId,
properties,
signal,
}: {
event: string;
distinctId: string | undefined;
properties: Record<string, unknown> | undefined;
signal: AbortSignal | undefined;
}): Promise<void> {
let response: Response;
try {
response = await fetch(`${this.host}/i/v0/e/`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
api_key: this.apiKey,
event,
distinct_id: distinctId ?? SYSTEM_DISTINCT_ID,
properties:
distinctId !== undefined
? properties
: { ...properties, $process_person_profile: false },
timestamp: new Date().toISOString(),
}),
signal,
});
} catch (error) {
if (signal?.aborted) {
throw error;
}
throw new UserError(
'EAS_POSTHOG_CAPTURE_FAILED',
`Sending PostHog event "${event}" failed.`,
{
cause: error,
}
);
}
if (!response.ok) {
throw new UserError(
'EAS_POSTHOG_CAPTURE_FAILED',
`Sending PostHog event "${event}" failed with ${await PosthogUtils.readErrorAsync(response)}.`
);
}
}
async requestAsync(
method: 'GET' | 'POST' | 'PATCH',
path: string,
{
action,
forbiddenScope,
body,
signal,
}: { action: string; forbiddenScope: string; body?: unknown; signal: AbortSignal | undefined }
): Promise<Response> {
let response: Response;
try {
response = await fetch(this.apiUrl(path), {
method,
headers: this.authHeaders(),
body: body !== undefined ? JSON.stringify(body) : undefined,
signal,
});
} catch (error) {
if (signal?.aborted) {
throw error;
}
throw new UserError('EAS_POSTHOG_REQUEST_FAILED', `${action} failed.`, { cause: error });
}
if (response.status === 403) {
throw new UserError(
'EAS_POSTHOG_FORBIDDEN',
`${action} was forbidden (403). The personal API key likely lacks the "${forbiddenScope}" scope.`
);
}
if (!response.ok) {
throw new UserError(
'EAS_POSTHOG_REQUEST_FAILED',
`${action} failed with ${await PosthogUtils.readErrorAsync(response)}.`
);
}
return response;
}
async runQueryAsync(
query: string,
logger: bunyan,
signal: AbortSignal | undefined
): Promise<unknown> {
let response: Response;
try {
response = await fetch(this.apiUrl('/query/'), {
method: 'POST',
headers: this.authHeaders(),
body: JSON.stringify({ query: { kind: 'HogQLQuery', query }, refresh: 'blocking' }),
signal,
});
} catch (error) {
if (signal?.aborted) {
throw error;
}
logger.debug(error);
logger.warn(
'Running the PostHog query failed with a network error; will retry on the next poll.'
);
throw new PosthogRetryableError();
}
if (response.status === 403) {
throw new UserError(
'EAS_POSTHOG_FORBIDDEN',
'Running the PostHog query was forbidden (403). The personal API key likely lacks the "query:read" scope.'
);
}
if (response.status >= 500 || response.status === 429) {
logger.warn(
`Running the PostHog query failed with status ${response.status}; will retry on the next poll.`
);
throw new PosthogRetryableError();
}
if (!response.ok) {
throw new UserError(
'EAS_POSTHOG_REQUEST_FAILED',
`Running the PostHog query failed with ${await PosthogUtils.readErrorAsync(response)}.`
);
}
let body: unknown;
try {
body = await response.json();
} catch (error) {
logger.debug(error);
logger.warn('The PostHog query response was not valid JSON; will retry on the next poll.');
throw new PosthogRetryableError();
}
const parsed = QueryResponseSchema.safeParse(body);
return parsed.success ? parsed.data.results[0]?.[0] : undefined;
}
cliConfig(): { host: string; env: Record<string, string> } {
return {
host: this.host,
env: { POSTHOG_CLI_API_KEY: this.apiKey, POSTHOG_CLI_PROJECT_ID: this.projectId },
};
}
private apiUrl(path: string): string {
return `${this.host}/api/projects/${encodeURIComponent(this.projectId)}${path}`;
}
private authHeaders(): Record<string, string> {
return { Authorization: `Bearer ${this.apiKey}`, 'Content-Type': 'application/json' };
}
}
@@ -0,0 +1,103 @@
import { setTimeout as setTimeoutAsync } from 'node:timers/promises';
import { UserError } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import { Response } from 'node-fetch';
export namespace PosthogUtils {
export interface Credential {
label: string;
envVar: string;
input: string;
}
export const API_CREDENTIALS = {
apiKey: { label: 'personal API key', envVar: 'POSTHOG_CLI_API_KEY', input: 'api_key' },
projectId: { label: 'project id', envVar: 'POSTHOG_CLI_PROJECT_ID', input: 'project_id' },
} satisfies Record<string, Credential>;
export function missingCredentialsError(missing: Credential[]): UserError {
const labels = missing.map(credential => credential.label).join(', ');
const envVars = missing.map(credential => credential.envVar).join(', ');
const inputs = missing.map(credential => credential.input).join(', ');
return new UserError(
'EAS_POSTHOG_MISSING_CREDENTIALS',
`Missing PostHog credentials: ${labels}. Set the environment variables (${envVars}) or step inputs (${inputs}) on EAS, or re-run "eas integrations:posthog:connect" with error tracking enabled.`
);
}
export const DEFAULT_POLL_TIMEOUT_SECONDS = 600;
export const DEFAULT_POLL_INTERVAL_SECONDS = 30;
export function assertPollBoundsPositive({
timeoutSeconds,
intervalSeconds,
errorCode,
}: {
timeoutSeconds: number;
intervalSeconds: number;
errorCode: string;
}): void {
if (timeoutSeconds <= 0 || intervalSeconds <= 0) {
throw new UserError(
errorCode,
'"timeout_seconds" and "interval_seconds" must be greater than 0.'
);
}
}
export async function waitForNextPollAsync({
intervalSeconds,
deadline,
signal,
}: {
intervalSeconds: number;
deadline: number;
signal: AbortSignal | undefined;
}): Promise<boolean> {
const remainingMs = deadline - Date.now();
if (remainingMs <= 0) {
return false;
}
await setTimeoutAsync(Math.min(intervalSeconds * 1000, remainingMs), undefined, { signal });
return true;
}
export function failOrLogError({
logger,
ignoreError,
error,
}: {
logger: bunyan;
ignoreError: boolean;
error: unknown;
}): void {
if (
!ignoreError ||
(error instanceof UserError && error.errorCode === 'EAS_POSTHOG_FORBIDDEN')
) {
throw error;
}
const message = error instanceof Error ? error.message : String(error);
logger.warn({ err: error }, `${message} Ignoring error.`);
}
export async function readErrorAsync(response: Response): Promise<string> {
const text = await response.text().catch(() => '');
if (text) {
try {
const body = JSON.parse(text) as unknown;
if (typeof body === 'object' && body !== null) {
const { detail } = body as Record<string, unknown>;
if (typeof detail === 'string' && detail !== '') {
return `status ${response.status}: ${detail} (${text})`;
}
}
} catch {
return `status ${response.status}: ${text}`;
}
return `status ${response.status}: ${text}`;
}
return `status ${response.status}`;
}
}
@@ -0,0 +1,369 @@
import { UserError } from '@expo/eas-build-job';
import { createLogger } from '@expo/logger';
import fetch, { Response } from 'node-fetch';
import { PosthogClient, PosthogRetryableError } from '../PosthogClient';
jest.mock('@expo/logger');
jest.mock('node-fetch');
const fetchMock = jest.mocked(fetch);
const logger = createLogger({ name: 'test' });
function res({
ok = true,
status = 200,
json,
text,
}: {
ok?: boolean;
status?: number;
json?: unknown;
text?: string;
}): Response {
return {
ok,
status,
json: async () => json,
text: async () => text ?? (json !== undefined ? JSON.stringify(json) : ''),
} as unknown as Response;
}
const API_ENV = {
POSTHOG_CLI_API_KEY: 'phx_test',
POSTHOG_CLI_PROJECT_ID: '123',
POSTHOG_CLI_HOST: 'https://us.posthog.com',
};
afterEach(() => jest.resetAllMocks());
describe('PosthogClient.fromEnv', () => {
it('builds a client from env vars', () => {
expect(
PosthogClient.fromEnv({
apiKeyOverride: undefined,
projectIdOverride: undefined,
env: API_ENV,
}).client
).toBeInstanceOf(PosthogClient);
});
it('prefers overrides over env', async () => {
fetchMock.mockResolvedValue(res({ json: { results: [[1]] } }));
const { client } = PosthogClient.fromEnv({
apiKeyOverride: 'phx_override',
projectIdOverride: '999',
env: {},
});
await client?.runQueryAsync('select 1', logger, undefined);
const [url, init] = fetchMock.mock.calls[0];
expect(url).toBe('https://us.posthog.com/api/projects/999/query/');
expect((init?.headers as Record<string, string>).Authorization).toBe('Bearer phx_override');
});
it('exposes host and CLI env for subprocess uploads', () => {
const { client } = PosthogClient.fromEnv({
apiKeyOverride: 'phx_k',
projectIdOverride: '55',
env: { POSTHOG_CLI_HOST: 'https://eu.posthog.com' },
});
expect(client?.cliConfig()).toEqual({
host: 'https://eu.posthog.com',
env: { POSTHOG_CLI_API_KEY: 'phx_k', POSTHOG_CLI_PROJECT_ID: '55' },
});
});
it('reports both missing credentials when neither is provided', () => {
const result = PosthogClient.fromEnv({
apiKeyOverride: undefined,
projectIdOverride: undefined,
env: {},
});
expect(result.client).toBeUndefined();
expect(result).toMatchObject({
missing: [{ label: 'personal API key' }, { label: 'project id' }],
});
});
it('reports only the missing project id when the key is present', () => {
const result = PosthogClient.fromEnv({
apiKeyOverride: 'phx',
projectIdOverride: undefined,
env: {},
});
expect(result.client).toBeUndefined();
expect(result).toMatchObject({ missing: [{ label: 'project id' }] });
});
it('reports only the missing api key when the project id is present', () => {
const result = PosthogClient.fromEnv({
apiKeyOverride: undefined,
projectIdOverride: '1',
env: {},
});
expect(result.client).toBeUndefined();
expect(result).toMatchObject({ missing: [{ label: 'personal API key' }] });
});
it('falls back to the default host', async () => {
fetchMock.mockResolvedValue(res({ json: { results: [[1]] } }));
const { client } = PosthogClient.fromEnv({
apiKeyOverride: 'phx',
projectIdOverride: '1',
env: {},
});
await client?.runQueryAsync('select 1', logger, undefined);
expect(fetchMock.mock.calls[0][0]).toBe('https://us.posthog.com/api/projects/1/query/');
});
});
describe('PosthogClient.forIngestion', () => {
it('builds a client and normalizes a trailing-slash host', async () => {
fetchMock.mockResolvedValue(res({ ok: true, status: 200 }));
const client = PosthogClient.forIngestion({
apiKeyOverride: 'phc_x',
hostOverride: 'https://eu.posthog.com//',
env: {},
});
await client?.captureEventAsync({
event: 'e',
distinctId: 'u',
properties: undefined,
signal: undefined,
});
expect(fetchMock.mock.calls[0][0]).toBe('https://eu.posthog.com/i/v0/e/');
});
it('falls back to a non-public POSTHOG_API_KEY', () => {
expect(
PosthogClient.forIngestion({
apiKeyOverride: undefined,
hostOverride: undefined,
env: { POSTHOG_API_KEY: 'phc_secret' },
})
).toBeInstanceOf(PosthogClient);
});
it('returns undefined when no key is available', () => {
expect(
PosthogClient.forIngestion({ apiKeyOverride: undefined, hostOverride: undefined, env: {} })
).toBeUndefined();
});
});
describe('captureEventAsync', () => {
const client = PosthogClient.forIngestion({
apiKeyOverride: 'phc_x',
hostOverride: undefined,
env: {},
});
it('sends an anonymous system event when no distinct_id is given', async () => {
fetchMock.mockResolvedValue(res({ ok: true, status: 200 }));
await client?.captureEventAsync({
event: 'e',
distinctId: undefined,
properties: { a: 1 },
signal: undefined,
});
const body = JSON.parse(fetchMock.mock.calls[0][1]?.body as string);
expect(body.distinct_id).toBe('eas-workflow');
expect(body.properties).toEqual({ a: 1, $process_person_profile: false });
});
it('throws a UserError on a non-2xx response', async () => {
fetchMock.mockResolvedValue(res({ ok: false, status: 400, text: 'bad' }));
await expect(
client?.captureEventAsync({
event: 'e',
distinctId: 'u',
properties: undefined,
signal: undefined,
})
).rejects.toThrow(/failed with status 400: bad/);
});
it('throws a UserError on a network error', async () => {
fetchMock.mockRejectedValue(new Error('ECONNRESET'));
await expect(
client?.captureEventAsync({
event: 'e',
distinctId: 'u',
properties: undefined,
signal: undefined,
})
).rejects.toThrow(/Sending PostHog event "e" failed\.$/);
});
it('passes the signal and propagates without wrapping when aborted', async () => {
const controller = new AbortController();
controller.abort();
fetchMock.mockRejectedValue(new Error('The operation was aborted'));
await expect(
client?.captureEventAsync({
event: 'e',
distinctId: 'u',
properties: undefined,
signal: controller.signal,
})
).rejects.toThrow('The operation was aborted');
expect(fetchMock.mock.calls[0][1]?.signal).toBe(controller.signal);
});
});
describe('requestAsync', () => {
const { client } = PosthogClient.fromEnv({
apiKeyOverride: undefined,
projectIdOverride: undefined,
env: API_ENV,
});
it('sends an authenticated request and returns the response', async () => {
fetchMock.mockResolvedValue(res({ ok: true, status: 200, json: { id: 1 } }));
const response = await client?.requestAsync('PATCH', '/feature_flags/1/', {
action: 'Updating flag',
forbiddenScope: 'feature_flag:write',
body: { active: true },
signal: undefined,
});
expect(response?.ok).toBe(true);
const [url, init] = fetchMock.mock.calls[0];
expect(url).toBe('https://us.posthog.com/api/projects/123/feature_flags/1/');
expect(init?.method).toBe('PATCH');
expect(JSON.parse(init?.body as string)).toEqual({ active: true });
});
it('throws a forbidden UserError with the named scope on 403', async () => {
fetchMock.mockResolvedValue(res({ ok: false, status: 403 }));
await expect(
client?.requestAsync('GET', '/feature_flags/', {
action: 'Looking up flag',
forbiddenScope: 'feature_flag:read',
signal: undefined,
})
).rejects.toMatchObject({
errorCode: 'EAS_POSTHOG_FORBIDDEN',
message: expect.stringContaining('"feature_flag:read"'),
});
});
it('throws with the error detail on a non-2xx response', async () => {
fetchMock.mockResolvedValue(res({ ok: false, status: 400, json: { detail: 'nope' } }));
await expect(
client?.requestAsync('GET', '/feature_flags/', {
action: 'Looking up flag',
forbiddenScope: 'feature_flag:read',
signal: undefined,
})
).rejects.toThrow(/Looking up flag failed with status 400: nope/);
});
it('throws on a network error', async () => {
fetchMock.mockRejectedValue(new Error('ECONNRESET'));
await expect(
client?.requestAsync('GET', '/feature_flags/', {
action: 'Looking up flag',
forbiddenScope: 'feature_flag:read',
signal: undefined,
})
).rejects.toThrow(/Looking up flag failed\.$/);
});
it('passes the signal and propagates without wrapping when aborted', async () => {
const controller = new AbortController();
controller.abort();
fetchMock.mockRejectedValue(new Error('The operation was aborted'));
await expect(
client?.requestAsync('GET', '/feature_flags/', {
action: 'Looking up flag',
forbiddenScope: 'feature_flag:read',
signal: controller.signal,
})
).rejects.toThrow('The operation was aborted');
expect(fetchMock.mock.calls[0][1]?.signal).toBe(controller.signal);
});
});
describe('runQueryAsync', () => {
const { client } = PosthogClient.fromEnv({
apiKeyOverride: undefined,
projectIdOverride: undefined,
env: API_ENV,
});
it('returns the top-left cell', async () => {
fetchMock.mockResolvedValue(res({ json: { results: [[42]] } }));
await expect(client?.runQueryAsync('select 1', logger, undefined)).resolves.toBe(42);
});
it('throws a retryable error on a network error', async () => {
fetchMock.mockRejectedValue(new Error('ECONNRESET'));
await expect(client?.runQueryAsync('q', logger, undefined)).rejects.toBeInstanceOf(
PosthogRetryableError
);
});
it('passes the signal to fetch and propagates without retrying when aborted', async () => {
const controller = new AbortController();
controller.abort();
fetchMock.mockRejectedValue(new Error('The operation was aborted'));
await expect(client?.runQueryAsync('q', logger, controller.signal)).rejects.toThrow(
'The operation was aborted'
);
expect(fetchMock.mock.calls[0][1]?.signal).toBe(controller.signal);
});
it('throws a retryable error on 5xx and 429', async () => {
fetchMock.mockResolvedValueOnce(res({ ok: false, status: 500 }));
await expect(client?.runQueryAsync('q', logger, undefined)).rejects.toBeInstanceOf(
PosthogRetryableError
);
fetchMock.mockResolvedValueOnce(res({ ok: false, status: 429 }));
await expect(client?.runQueryAsync('q', logger, undefined)).rejects.toBeInstanceOf(
PosthogRetryableError
);
});
it('throws a retryable error on an invalid JSON body', async () => {
fetchMock.mockResolvedValue({
ok: true,
status: 200,
json: async () => {
throw new Error('bad json');
},
} as unknown as Response);
await expect(client?.runQueryAsync('q', logger, undefined)).rejects.toBeInstanceOf(
PosthogRetryableError
);
});
it('returns undefined for a malformed result shape', async () => {
fetchMock.mockResolvedValue(res({ json: { results: 'nope' } }));
await expect(client?.runQueryAsync('q', logger, undefined)).resolves.toBeUndefined();
});
it('throws a forbidden UserError on 403', async () => {
fetchMock.mockResolvedValue(res({ ok: false, status: 403 }));
await expect(client?.runQueryAsync('q', logger, undefined)).rejects.toMatchObject({
errorCode: 'EAS_POSTHOG_FORBIDDEN',
message: expect.stringContaining('"query:read"'),
});
});
it('throws with the error detail on a non-2xx response', async () => {
fetchMock.mockResolvedValue(res({ ok: false, status: 400, text: 'Malformed HogQL' }));
await expect(client?.runQueryAsync('q', logger, undefined)).rejects.toThrow(
/failed with status 400: Malformed HogQL/
);
});
});
it('is a UserError subclass so the step surfaces it to the user', async () => {
const { client } = PosthogClient.fromEnv({
apiKeyOverride: undefined,
projectIdOverride: undefined,
env: API_ENV,
});
fetchMock.mockResolvedValue(res({ ok: false, status: 403 }));
await expect(client?.runQueryAsync('q', logger, undefined)).rejects.toBeInstanceOf(UserError);
});
@@ -0,0 +1,173 @@
import { UserError } from '@expo/eas-build-job';
import { bunyan, createLogger } from '@expo/logger';
import { Response } from 'node-fetch';
import { PosthogUtils } from '../PosthogUtils';
jest.mock('@expo/logger');
const logger = createLogger({ name: 'test' }) as bunyan;
function errorResponse({ status, text }: { status: number; text?: string }): Response {
return { status, text: async () => text ?? '' } as unknown as Response;
}
describe(PosthogUtils.readErrorAsync, () => {
it('surfaces a JSON detail followed by the raw body', async () => {
await expect(
PosthogUtils.readErrorAsync(errorResponse({ status: 400, text: '{"detail":"boom"}' }))
).resolves.toBe('status 400: boom ({"detail":"boom"})');
});
it('falls back to the raw body for detail-less JSON', async () => {
await expect(
PosthogUtils.readErrorAsync(errorResponse({ status: 400, text: '{"other":1}' }))
).resolves.toBe('status 400: {"other":1}');
});
it('ignores an empty-string detail', async () => {
await expect(
PosthogUtils.readErrorAsync(errorResponse({ status: 400, text: '{"detail":""}' }))
).resolves.toBe('status 400: {"detail":""}');
});
it('falls back to the raw text for a non-JSON body', async () => {
await expect(
PosthogUtils.readErrorAsync(errorResponse({ status: 502, text: 'Bad Gateway' }))
).resolves.toBe('status 502: Bad Gateway');
});
it('uses just the status when the body is empty', async () => {
await expect(
PosthogUtils.readErrorAsync(errorResponse({ status: 500, text: '' }))
).resolves.toBe('status 500');
});
it('uses just the status when reading the body throws', async () => {
const response = {
status: 500,
text: async () => {
throw new Error('read fail');
},
} as unknown as Response;
await expect(PosthogUtils.readErrorAsync(response)).resolves.toBe('status 500');
});
});
describe(PosthogUtils.missingCredentialsError, () => {
it('names both missing credentials with their env vars and inputs', () => {
expect(
PosthogUtils.missingCredentialsError([
PosthogUtils.API_CREDENTIALS.apiKey,
PosthogUtils.API_CREDENTIALS.projectId,
])
).toMatchObject({
errorCode: 'EAS_POSTHOG_MISSING_CREDENTIALS',
message:
'Missing PostHog credentials: personal API key, project id. Set the environment variables (POSTHOG_CLI_API_KEY, POSTHOG_CLI_PROJECT_ID) or step inputs (api_key, project_id) on EAS, or re-run "eas integrations:posthog:connect" with error tracking enabled.',
});
});
it('names only the missing credential', () => {
expect(
PosthogUtils.missingCredentialsError([PosthogUtils.API_CREDENTIALS.projectId]).message
).toBe(
'Missing PostHog credentials: project id. Set the environment variables (POSTHOG_CLI_PROJECT_ID) or step inputs (project_id) on EAS, or re-run "eas integrations:posthog:connect" with error tracking enabled.'
);
});
});
describe(PosthogUtils.assertPollBoundsPositive, () => {
it('throws with the given error code when a bound is not positive', () => {
expect(() =>
PosthogUtils.assertPollBoundsPositive({
timeoutSeconds: 0,
intervalSeconds: 30,
errorCode: 'EAS_POSTHOG_TEST_INTERVAL',
})
).toThrow(/greater than 0/);
expect(() =>
PosthogUtils.assertPollBoundsPositive({
timeoutSeconds: 600,
intervalSeconds: -1,
errorCode: 'EAS_POSTHOG_TEST_INTERVAL',
})
).toThrow(/greater than 0/);
});
it('does nothing when both bounds are positive', () => {
expect(() =>
PosthogUtils.assertPollBoundsPositive({
timeoutSeconds: 600,
intervalSeconds: 30,
errorCode: 'EAS_POSTHOG_TEST_INTERVAL',
})
).not.toThrow();
});
});
describe(PosthogUtils.waitForNextPollAsync, () => {
it('returns false without sleeping when the deadline has passed', async () => {
await expect(
PosthogUtils.waitForNextPollAsync({
intervalSeconds: 30,
deadline: Date.now() - 1,
signal: undefined,
})
).resolves.toBe(false);
});
it('sleeps for the clamped interval and returns true when time remains', async () => {
await expect(
PosthogUtils.waitForNextPollAsync({
intervalSeconds: 0.01,
deadline: Date.now() + 1000,
signal: undefined,
})
).resolves.toBe(true);
});
it('rejects when the signal aborts during the sleep', async () => {
const controller = new AbortController();
const promise = PosthogUtils.waitForNextPollAsync({
intervalSeconds: 30,
deadline: Date.now() + 60_000,
signal: controller.signal,
});
controller.abort();
await expect(promise).rejects.toThrow();
});
});
describe(PosthogUtils.failOrLogError, () => {
it('throws the error when ignoreError is false', () => {
const error = new Error('boom');
expect(() => PosthogUtils.failOrLogError({ logger, ignoreError: false, error })).toThrow(
'boom'
);
});
it('warns and swallows the error when ignoreError is true', () => {
const warnMock = jest.spyOn(logger, 'warn');
const error = new Error('boom');
PosthogUtils.failOrLogError({ logger, ignoreError: true, error });
expect(warnMock).toHaveBeenCalledWith({ err: error }, 'boom Ignoring error.');
});
it('rethrows a forbidden error even when ignoreError is true', () => {
const error = new UserError('EAS_POSTHOG_FORBIDDEN', 'no scope');
expect(() => PosthogUtils.failOrLogError({ logger, ignoreError: true, error })).toThrow(
'no scope'
);
});
it('stringifies a non-Error value when swallowing', () => {
const warnMock = jest.spyOn(logger, 'warn');
PosthogUtils.failOrLogError({ logger, ignoreError: true, error: 'plain string' });
expect(warnMock).toHaveBeenCalledWith({ err: 'plain string' }, 'plain string Ignoring error.');
});
});
@@ -0,0 +1,255 @@
import { bunyan } from '@expo/logger';
import fetch from 'node-fetch';
import { Readable } from 'node:stream';
import { CustomBuildContext } from '../../../customBuildContext';
import { uploadDeviceRunSessionArtifactAsync } from '../deviceRunSessionArtifacts';
import {
listAgentDeviceArtifactsAsync,
pollAgentDeviceArtifactsForUploadAsync,
uploadAgentDeviceArtifactAsync,
} from '../agentDeviceArtifacts';
jest.mock('../deviceRunSessionArtifacts');
jest.mock('node-fetch');
const { Response } = jest.requireActual('node-fetch') as typeof import('node-fetch');
async function readStreamAsync(stream: NodeJS.ReadableStream): Promise<void> {
for await (const chunk of stream as Readable) {
void chunk;
}
}
async function flushPromisesAsync(): Promise<void> {
for (let i = 0; i < 10; i++) {
await jest.advanceTimersByTimeAsync(0);
await Promise.resolve();
}
}
function createLoggerMock(): bunyan {
return {
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
debug: jest.fn(),
} as unknown as bunyan;
}
describe(listAgentDeviceArtifactsAsync, () => {
beforeEach(() => {
jest.mocked(fetch).mockReset();
});
it('lists agent-device artifacts with bearer auth', async () => {
jest.mocked(fetch).mockResolvedValue(
new Response(
JSON.stringify({
artifacts: [
{
id: 'artifact-id',
artifactType: 'agent-device-test-report',
filename: 'report.json',
mimeType: 'application/json',
sizeBytes: 123,
createdAt: '2026-07-02T12:00:00.000Z',
expiresAt: '2026-07-02T12:15:00.000Z',
},
{
id: 'untyped-artifact-id',
artifactType: null,
filename: 'untyped-report.json',
mimeType: 'application/json',
sizeBytes: 456,
createdAt: '2026-07-02T12:00:00.000Z',
expiresAt: '2026-07-02T12:15:00.000Z',
},
],
})
)
);
const artifacts = await listAgentDeviceArtifactsAsync({
daemonUrl: 'http://127.0.0.1:1234',
daemonToken: 'daemon-token',
});
expect(artifacts).toEqual([
{
id: 'artifact-id',
artifactType: 'agent-device-test-report',
filename: 'report.json',
},
{
id: 'untyped-artifact-id',
artifactType: null,
filename: 'untyped-report.json',
},
]);
expect(jest.mocked(fetch)).toHaveBeenCalledWith('http://127.0.0.1:1234/artifacts', {
headers: { Authorization: 'Bearer daemon-token' },
});
});
it('reports artifact inventory as unsupported when the daemon does not expose the endpoint', async () => {
jest.mocked(fetch).mockResolvedValue(new Response('Not found', { status: 404 }));
await expect(
listAgentDeviceArtifactsAsync({
daemonUrl: 'http://127.0.0.1:1234',
daemonToken: 'daemon-token',
})
).rejects.toThrow('agent-device daemon does not expose artifact inventory.');
});
});
describe(uploadAgentDeviceArtifactAsync, () => {
beforeEach(() => {
jest.mocked(fetch).mockReset();
jest.mocked(uploadDeviceRunSessionArtifactAsync).mockReset();
});
it('downloads an agent-device artifact and uploads it as a device run session artifact', async () => {
const data = Buffer.from('artifact-data');
const logger = createLoggerMock();
const ctx = {} as unknown as CustomBuildContext;
jest.mocked(fetch).mockResolvedValueOnce(new Response(Readable.from([data])));
jest
.mocked(uploadDeviceRunSessionArtifactAsync)
.mockImplementationOnce(async (_ctx, { stream }) => {
await readStreamAsync(stream);
});
await uploadAgentDeviceArtifactAsync(ctx, {
deviceRunSessionId: 'drs-id',
daemonUrl: 'http://127.0.0.1:1234',
daemonToken: 'daemon-token',
logger,
artifact: {
id: 'artifact-id',
artifactType: 'agent-device-test-report',
filename: 'report.json',
},
});
expect(jest.mocked(fetch)).toHaveBeenCalledWith('http://127.0.0.1:1234/artifacts/artifact-id', {
headers: { Authorization: 'Bearer daemon-token' },
});
expect(jest.mocked(uploadDeviceRunSessionArtifactAsync)).toHaveBeenCalledWith(ctx, {
deviceRunSessionId: 'drs-id',
artifactId: 'artifact-id',
name: 'report.json (artifact-id)',
filename: 'report.json',
kind: 'agent-device-test-report',
size: data.length,
stream: expect.anything(),
});
});
it('uploads null artifact types as an explicit undefined kind', async () => {
const data = Buffer.from('artifact-data');
const logger = createLoggerMock();
const ctx = {} as unknown as CustomBuildContext;
jest.mocked(fetch).mockResolvedValueOnce(new Response(Readable.from([data])));
jest
.mocked(uploadDeviceRunSessionArtifactAsync)
.mockImplementationOnce(async (_ctx, { stream }) => {
await readStreamAsync(stream);
});
await uploadAgentDeviceArtifactAsync(ctx, {
deviceRunSessionId: 'drs-id',
daemonUrl: 'http://127.0.0.1:1234',
daemonToken: 'daemon-token',
logger,
artifact: {
id: 'artifact-id',
artifactType: null,
filename: 'report.json',
},
});
expect(jest.mocked(uploadDeviceRunSessionArtifactAsync)).toHaveBeenCalledWith(
ctx,
expect.objectContaining({
kind: undefined,
})
);
});
});
describe(pollAgentDeviceArtifactsForUploadAsync, () => {
beforeEach(() => {
jest.useFakeTimers();
jest.mocked(fetch).mockReset();
jest.mocked(uploadDeviceRunSessionArtifactAsync).mockReset();
});
afterEach(() => {
jest.clearAllTimers();
jest.useRealTimers();
});
it('retries an artifact after a failed upload', async () => {
const data = Buffer.from('artifact-data');
const logger = createLoggerMock();
const ctx = {} as unknown as CustomBuildContext;
const artifact = {
id: 'artifact-id',
filename: 'report.json',
};
const listResponse = () => new Response(JSON.stringify({ artifacts: [artifact] }));
jest
.mocked(fetch)
.mockResolvedValueOnce(listResponse())
.mockResolvedValueOnce(new Response(Readable.from([data])))
.mockResolvedValueOnce(listResponse())
.mockResolvedValueOnce(new Response(Readable.from([data])));
jest
.mocked(uploadDeviceRunSessionArtifactAsync)
.mockRejectedValueOnce(new Error('upload failed'))
.mockImplementationOnce(async (_ctx, { stream }) => {
await readStreamAsync(stream);
});
void pollAgentDeviceArtifactsForUploadAsync(ctx, {
deviceRunSessionId: 'drs-id',
daemonUrl: 'http://127.0.0.1:1234',
daemonToken: 'daemon-token',
logger,
});
await flushPromisesAsync();
expect(jest.mocked(uploadDeviceRunSessionArtifactAsync)).toHaveBeenCalledTimes(1);
await jest.advanceTimersByTimeAsync(5_000);
await flushPromisesAsync();
expect(jest.mocked(uploadDeviceRunSessionArtifactAsync)).toHaveBeenCalledTimes(2);
});
it('stops polling when the daemon does not expose artifact inventory', async () => {
const logger = createLoggerMock();
const ctx = {} as unknown as CustomBuildContext;
jest.mocked(fetch).mockResolvedValueOnce(new Response('Not found', { status: 404 }));
await expect(
pollAgentDeviceArtifactsForUploadAsync(ctx, {
deviceRunSessionId: 'drs-id',
daemonUrl: 'http://127.0.0.1:1234',
daemonToken: 'daemon-token',
logger,
})
).resolves.toBeUndefined();
expect(jest.mocked(fetch)).toHaveBeenCalledTimes(1);
expect(logger.warn).toHaveBeenCalledWith(
'agent-device daemon does not expose artifact inventory; remote session artifact uploads are disabled.'
);
});
});
@@ -4,9 +4,14 @@ import { Readable } from 'node:stream';
import { CustomBuildContext } from '../../../customBuildContext';
import { uploadDeviceRunSessionArtifactAsync } from '../deviceRunSessionArtifacts';
import { listArgentArtifactsAsync, uploadArgentArtifactAsync } from '../argentArtifacts';
import {
listArgentArtifactsAsync,
pollArgentArtifactsForUploadAsync,
uploadArgentArtifactAsync,
} from '../argentArtifacts';
jest.mock('../deviceRunSessionArtifacts');
jest.mock('../../../sentry');
jest.mock('node-fetch');
const { Response } = jest.requireActual('node-fetch') as typeof import('node-fetch');
@@ -17,6 +22,21 @@ async function readStreamAsync(stream: NodeJS.ReadableStream): Promise<void> {
}
}
async function waitForAssertionAsync(assertion: () => void): Promise<void> {
const deadline = Date.now() + 1_000;
let lastError: unknown;
while (Date.now() < deadline) {
try {
assertion();
return;
} catch (err) {
lastError = err;
await new Promise(resolve => setTimeout(resolve, 10));
}
}
throw lastError;
}
function createLoggerMock(): bunyan {
return {
info: jest.fn(),
@@ -62,6 +82,7 @@ describe(listArgentArtifactsAsync, () => {
]);
expect(jest.mocked(fetch)).toHaveBeenCalledWith('http://127.0.0.1:1234/artifacts', {
headers: { Authorization: 'Bearer tools-token' },
signal: expect.any(AbortSignal),
});
});
});
@@ -98,6 +119,7 @@ describe(uploadArgentArtifactAsync, () => {
expect(jest.mocked(fetch)).toHaveBeenCalledWith('http://127.0.0.1:1234/artifacts/artifact-id', {
headers: { Authorization: 'Bearer tools-token' },
signal: expect.any(AbortSignal),
});
expect(jest.mocked(uploadDeviceRunSessionArtifactAsync)).toHaveBeenCalledWith(ctx, {
deviceRunSessionId: 'drs-id',
@@ -109,3 +131,150 @@ describe(uploadArgentArtifactAsync, () => {
});
});
});
describe(pollArgentArtifactsForUploadAsync, () => {
beforeEach(() => {
jest.mocked(fetch).mockReset();
jest.mocked(uploadDeviceRunSessionArtifactAsync).mockReset();
});
it('stops polling when aborted, performs a final drain, and waits for uploads', async () => {
const logger = createLoggerMock();
const ctx = {} as unknown as CustomBuildContext;
const abortController = new AbortController();
let listCallCount = 0;
jest.mocked(fetch).mockImplementation(async url => {
const urlString = String(url);
if (urlString.endsWith('/artifacts')) {
listCallCount += 1;
return new Response(
JSON.stringify({
artifacts:
listCallCount === 1
? [
{
id: 'artifact-a',
filename: 'a.json',
mimeType: 'application/json',
},
]
: [
{
id: 'artifact-a',
filename: 'a.json',
mimeType: 'application/json',
},
{
id: 'artifact-b',
filename: 'b.json',
mimeType: 'application/json',
},
],
})
);
}
if (urlString.endsWith('/artifacts/artifact-a')) {
return new Response(Readable.from([Buffer.from('artifact-a-data')]));
}
if (urlString.endsWith('/artifacts/artifact-b')) {
return new Response(Readable.from([Buffer.from('artifact-b-data')]));
}
throw new Error(`Unexpected URL ${urlString}`);
});
jest
.mocked(uploadDeviceRunSessionArtifactAsync)
.mockImplementation(async (_ctx, { stream }) => {
await readStreamAsync(stream);
});
const pollingPromise = pollArgentArtifactsForUploadAsync(ctx, {
deviceRunSessionId: 'drs-id',
toolsUrl: 'http://127.0.0.1:1234',
toolsAuthToken: 'tools-token',
logger,
signal: abortController.signal,
});
await waitForAssertionAsync(() => {
expect(jest.mocked(uploadDeviceRunSessionArtifactAsync)).toHaveBeenCalledTimes(1);
});
abortController.abort();
await pollingPromise;
expect(listCallCount).toBe(2);
expect(jest.mocked(uploadDeviceRunSessionArtifactAsync)).toHaveBeenCalledTimes(2);
expect(jest.mocked(uploadDeviceRunSessionArtifactAsync)).toHaveBeenLastCalledWith(ctx, {
deviceRunSessionId: 'drs-id',
artifactId: 'artifact-b',
name: 'b.json (artifact-b)',
filename: 'b.json',
size: 'artifact-b-data'.length,
stream: expect.anything(),
});
});
it('throws when pending uploads do not finish before the cleanup timeout', async () => {
const logger = createLoggerMock();
const ctx = {} as unknown as CustomBuildContext;
const abortController = new AbortController();
const originalSetTimeout = global.setTimeout;
const setTimeoutSpy = jest
.spyOn(global, 'setTimeout')
.mockImplementation(((
callback: Parameters<typeof setTimeout>[0],
timeout?: number,
...args: unknown[]
) =>
originalSetTimeout(
callback,
timeout === 30_000 ? 0 : timeout,
...args
)) as typeof setTimeout);
try {
jest.mocked(fetch).mockImplementation(async url => {
const urlString = String(url);
if (urlString.endsWith('/artifacts')) {
return new Response(
JSON.stringify({
artifacts: [
{
id: 'artifact-a',
filename: 'a.json',
mimeType: 'application/json',
},
],
})
);
}
if (urlString.endsWith('/artifacts/artifact-a')) {
return new Response(Readable.from([Buffer.from('artifact-a-data')]));
}
throw new Error(`Unexpected URL ${urlString}`);
});
jest
.mocked(uploadDeviceRunSessionArtifactAsync)
.mockImplementation(() => new Promise<void>(() => {}));
const pollingPromise = pollArgentArtifactsForUploadAsync(ctx, {
deviceRunSessionId: 'drs-id',
toolsUrl: 'http://127.0.0.1:1234',
toolsAuthToken: 'tools-token',
logger,
signal: abortController.signal,
});
await waitForAssertionAsync(() => {
expect(jest.mocked(uploadDeviceRunSessionArtifactAsync)).toHaveBeenCalledTimes(1);
});
abortController.abort();
await expect(pollingPromise).rejects.toThrow(
'Timed out after 30s waiting for Argent artifact uploads.'
);
} finally {
setTimeoutSpy.mockRestore();
}
});
});
@@ -46,6 +46,8 @@ describe(uploadDeviceRunSessionArtifactAsync, () => {
artifactId: 'artifact-id',
name: 'Artifact report.json (artifact-id)',
filename: 'report.json',
kind: 'agent-device-test-report',
metadata: { firstFrameRecordAt: 'test-time' },
size: reportedSize,
stream,
});
@@ -57,6 +59,8 @@ describe(uploadDeviceRunSessionArtifactAsync, () => {
input: {
name: 'Artifact report.json (artifact-id)',
filename: 'report.json',
kind: 'agent-device-test-report',
metadata: { firstFrameRecordAt: 'test-time' },
size: reportedSize,
},
})
@@ -1,5 +1,6 @@
import { bunyan } from '@expo/logger';
import { BuildStepEnv } from '@expo/steps';
import { setTimeout as setTimeoutAsync } from 'node:timers/promises';
import { CustomBuildContext } from '../../../customBuildContext';
import { Sentry } from '../../../sentry';
@@ -7,8 +8,10 @@ import { turtleFetch } from '../../../utils/turtleFetch';
import {
fetchServeSimTurnArgsAsync,
turnIceServersToServeSimArgs,
waitForDeviceRunSessionStoppedAsync,
} from '../remoteDeviceRunSession';
jest.mock('node:timers/promises');
jest.mock('../../../utils/turtleFetch');
jest.mock('../../../sentry');
@@ -32,6 +35,47 @@ function createCtxMock(): CustomBuildContext {
} as unknown as CustomBuildContext;
}
function createStatusCtxMock(
results: (
| { status: 'NEW' | 'IN_PROGRESS' | 'STOPPED' | 'ERRORED' }
| { error: Error }
| { data: unknown }
)[]
): CustomBuildContext {
const query = jest.fn(() => {
const result = results.shift();
if (!result) {
throw new Error('No mocked status result available');
}
return {
toPromise: async () => {
if ('error' in result) {
throw result.error;
}
if ('data' in result) {
return { data: result.data };
}
return {
data: {
deviceRunSessions: {
byId: {
id: 'drs-id',
status: result.status,
},
},
},
};
},
};
});
return {
graphqlClient: {
query,
},
} as unknown as CustomBuildContext;
}
function createEnvMock(): BuildStepEnv {
return { DEVICE_RUN_SESSION_ID: 'drs-id' } as unknown as BuildStepEnv;
}
@@ -152,3 +196,86 @@ describe(fetchServeSimTurnArgsAsync, () => {
expect(jest.mocked(Sentry).capture).toHaveBeenCalled();
});
});
describe(waitForDeviceRunSessionStoppedAsync, () => {
beforeEach(() => {
jest.mocked(Sentry).capture.mockReset();
jest.mocked(setTimeoutAsync).mockReset();
jest.mocked(setTimeoutAsync).mockResolvedValue(undefined);
});
it('continues polling until the device run session is stopped', async () => {
const ctx = createStatusCtxMock([{ status: 'IN_PROGRESS' }, { status: 'STOPPED' }]);
const logger = createLoggerMock();
await waitForDeviceRunSessionStoppedAsync({
ctx,
deviceRunSessionId: 'drs-id',
logger,
});
expect(ctx.graphqlClient.query).toHaveBeenCalledTimes(2);
expect(logger.info).toHaveBeenCalledWith('Device run session drs-id was stopped.');
});
it('throws when the device run session errors', async () => {
const ctx = createStatusCtxMock([{ status: 'ERRORED' }]);
await expect(
waitForDeviceRunSessionStoppedAsync({
ctx,
deviceRunSessionId: 'drs-id',
logger: createLoggerMock(),
})
).rejects.toThrow('Device run session drs-id errored.');
});
it('logs and retries transient polling errors', async () => {
const ctx = createStatusCtxMock([{ error: new Error('network down') }, { status: 'STOPPED' }]);
const logger = createLoggerMock();
await waitForDeviceRunSessionStoppedAsync({
ctx,
deviceRunSessionId: 'drs-id',
logger,
});
expect(ctx.graphqlClient.query).toHaveBeenCalledTimes(2);
expect(logger.warn).toHaveBeenCalledWith(
expect.objectContaining({ failedStatusPollCount: 1 }),
'Could not poll device run session status; will retry.'
);
expect(jest.mocked(Sentry).capture).toHaveBeenCalledWith(
'Could not poll device run session status',
expect.any(Error),
{ level: 'warning' }
);
});
it('logs and retries when the status response is missing', async () => {
const ctx = createStatusCtxMock([
{ data: { deviceRunSessions: { byId: null } } },
{ status: 'STOPPED' },
]);
const logger = createLoggerMock();
await waitForDeviceRunSessionStoppedAsync({
ctx,
deviceRunSessionId: 'drs-id',
logger,
});
expect(ctx.graphqlClient.query).toHaveBeenCalledTimes(2);
expect(logger.warn).toHaveBeenCalledWith(
expect.objectContaining({ failedStatusPollCount: 1 }),
'Could not poll device run session status; will retry.'
);
expect(jest.mocked(Sentry).capture).toHaveBeenCalledWith(
'Could not poll device run session status',
expect.objectContaining({
message: 'Device run session drs-id status response was missing.',
}),
{ level: 'warning' }
);
});
});
@@ -0,0 +1,193 @@
import { SystemError } from '@expo/eas-build-job';
import { type bunyan } from '@expo/logger';
import { createReadStream, createWriteStream } from 'node:fs';
import { mkdtemp, rm, stat } from 'node:fs/promises';
import fetch from 'node-fetch';
import os from 'node:os';
import path from 'node:path';
import { pipeline } from 'node:stream/promises';
import { z } from 'zod';
import { CustomBuildContext } from '../../customBuildContext';
import { Sentry } from '../../sentry';
import { formatBytes } from '../../utils/artifacts';
import { sleepAsync } from '../../utils/retry';
import { uploadDeviceRunSessionArtifactAsync } from './deviceRunSessionArtifacts';
const AGENT_DEVICE_ARTIFACT_UPLOAD_POLL_INTERVAL_MS = 5_000;
const AgentDeviceArtifactSchema = z.object({
id: z.string(),
artifactType: z.string().nullish(),
filename: z.string(),
});
const AgentDeviceArtifactsListResponseSchema = z.object({
artifacts: z.array(AgentDeviceArtifactSchema),
});
export type AgentDeviceArtifact = z.infer<typeof AgentDeviceArtifactSchema>;
class AgentDeviceArtifactsUnsupportedError extends SystemError {
constructor() {
super('agent-device daemon does not expose artifact inventory.');
}
}
export async function pollAgentDeviceArtifactsForUploadAsync(
ctx: CustomBuildContext,
{
deviceRunSessionId,
daemonUrl,
daemonToken,
logger,
}: {
deviceRunSessionId: string;
daemonUrl: string;
daemonToken: string;
logger: bunyan;
}
): Promise<void> {
logger.info('Started polling agent-device daemon for artifacts.');
const uploadedArtifactIds = new Set<string>();
let listArtifactsErrorCount = 0;
for (;;) {
try {
const artifacts = await listAgentDeviceArtifactsAsync({ daemonUrl, daemonToken });
listArtifactsErrorCount = 0;
for (const artifact of artifacts) {
if (uploadedArtifactIds.has(artifact.id)) {
continue;
}
try {
await uploadAgentDeviceArtifactAsync(ctx, {
deviceRunSessionId,
daemonUrl,
daemonToken,
artifact,
logger,
});
uploadedArtifactIds.add(artifact.id);
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not upload agent-device remote session artifact', error);
logger.warn({ err: error }, 'Could not upload agent-device remote session artifact.');
}
}
} catch (err) {
if (err instanceof AgentDeviceArtifactsUnsupportedError) {
logger.warn(
'agent-device daemon does not expose artifact inventory; remote session artifact uploads are disabled.'
);
return;
}
const error = err instanceof Error ? err : new Error(String(err));
listArtifactsErrorCount += 1;
if (listArtifactsErrorCount === 1 || listArtifactsErrorCount % 5 === 0) {
Sentry.capture('Could not list agent-device remote session artifacts', error);
logger.warn(
{ err: error, failedArtifactListCount: listArtifactsErrorCount },
'Could not list agent-device remote session artifacts.'
);
}
}
await sleepAsync(AGENT_DEVICE_ARTIFACT_UPLOAD_POLL_INTERVAL_MS);
}
}
export async function listAgentDeviceArtifactsAsync({
daemonUrl,
daemonToken,
}: {
daemonUrl: string;
daemonToken: string;
}): Promise<AgentDeviceArtifact[]> {
const response = await fetch(new URL('/artifacts', daemonUrl).toString(), {
headers: { Authorization: `Bearer ${daemonToken}` },
});
if (response.status === 404) {
throw new AgentDeviceArtifactsUnsupportedError();
}
if (!response.ok) {
throw new SystemError(
`Failed to list agent-device artifacts: ${response.status} ${response.statusText}`
);
}
const result = AgentDeviceArtifactsListResponseSchema.safeParse(await response.json());
if (!result.success) {
throw new SystemError(`Invalid agent-device artifacts response: ${result.error.message}`);
}
return result.data.artifacts;
}
export async function uploadAgentDeviceArtifactAsync(
ctx: CustomBuildContext,
{
deviceRunSessionId,
daemonUrl,
daemonToken,
artifact,
logger,
}: {
deviceRunSessionId: string;
daemonUrl: string;
daemonToken: string;
artifact: AgentDeviceArtifact;
logger: bunyan;
}
): Promise<void> {
logger.info(`Downloading artifact ${artifact.filename}.`);
const temporaryDirectory = await mkdtemp(path.join(os.tmpdir(), 'agent-device-artifact-'));
try {
const temporaryArtifactPath = path.join(temporaryDirectory, path.basename(artifact.filename));
await downloadAgentDeviceArtifactToFileAsync({
artifact,
daemonUrl,
daemonToken,
destinationPath: temporaryArtifactPath,
});
const { size } = await stat(temporaryArtifactPath);
logger.info(`Uploading artifact ${artifact.filename} (${formatBytes(size)}).`);
await uploadDeviceRunSessionArtifactAsync(ctx, {
deviceRunSessionId,
artifactId: artifact.id,
name: `${artifact.filename} (${artifact.id})`,
filename: artifact.filename,
kind: artifact.artifactType ?? undefined,
size,
stream: createReadStream(temporaryArtifactPath),
});
} finally {
await rm(temporaryDirectory, { recursive: true, force: true });
}
}
async function downloadAgentDeviceArtifactToFileAsync({
artifact,
daemonUrl,
daemonToken,
destinationPath,
}: {
artifact: AgentDeviceArtifact;
daemonUrl: string;
daemonToken: string;
destinationPath: string;
}): Promise<void> {
const response = await fetch(
new URL(`/artifacts/${encodeURIComponent(artifact.id)}`, daemonUrl).toString(),
{
headers: { Authorization: `Bearer ${daemonToken}` },
}
);
if (!response.ok) {
throw new SystemError(
`Failed to download agent-device artifact ${artifact.id}: ${response.status} ${response.statusText}`
);
}
if (!response.body) {
throw new SystemError(
`Agent-device artifact ${artifact.id} response did not include a readable body.`
);
}
await pipeline(response.body, createWriteStream(destinationPath));
}
@@ -1,8 +1,7 @@
import { Android } from '@expo/eas-build-job';
import { bunyan } from '@expo/logger';
import { BuildStepEnv } from '@expo/steps';
import spawn, { SpawnPromise, SpawnResult } from '@expo/turtle-spawn';
import assert from 'assert';
import spawn from '@expo/turtle-spawn';
import fs from 'fs-extra';
import path from 'path';
@@ -23,10 +22,18 @@ export async function runGradleCommand({
logger.info(`Running 'gradlew ${gradleCommand} ${verboseFlag}' in ${androidDir}`);
await fs.chmod(path.join(androidDir, 'gradlew'), 0o755);
const shouldResetOOMScore = env.EAS_BUILD_RUNNER === 'eas-build' && process.platform === 'linux';
const spawnPromise = spawn(
await spawn(
'bash',
['-c', `./gradlew ${gradleCommand} --profile ${verboseFlag}`],
[
'-c',
getGradleShellCommand({
gradleCommand,
oomScoreAdj: shouldResetOOMScore ? 0 : undefined,
verboseFlag,
}),
],
{
cwd: androidDir,
logger,
@@ -44,73 +51,20 @@ export async function runGradleCommand({
},
}
);
if (env.EAS_BUILD_RUNNER === 'eas-build' && process.platform === 'linux') {
adjustOOMScore(spawnPromise, logger);
}
await spawnPromise;
}
/**
* OOM Killer sometimes kills worker server while build is exceeding memory limits.
* `oom_score_adj` is a value between -1000 and 1000 and it defaults to 0.
* It defines which process is more likely to get killed (higher value more likely).
*
* This function sets oom_score_adj for Gradle process and all its child processes.
*/
function adjustOOMScore(spawnPromise: SpawnPromise<SpawnResult>, logger: bunyan): void {
setTimeout(
async () => {
try {
assert(spawnPromise.child.pid);
const pids = await getParentAndDescendantProcessPidsAsync(spawnPromise.child.pid);
await Promise.all(
pids.map(async (pid: number) => {
// Value 800 is just a guess here. It's probably higher than most other
// process. I didn't want to set it any higher, because I'm not sure if OOM Killer
// can start killing processes when there is still enough memory left.
const oomScoreOverride = 800;
await fs.writeFile(`/proc/${pid}/oom_score_adj`, `${oomScoreOverride}\n`);
})
);
} catch (err: any) {
logger.debug({ err, stderr: err?.stderr }, 'Failed to override oom_score_adj');
}
},
// Wait 20 seconds to make sure all child processes are started
20000
);
}
async function getChildrenPidsAsync(parentPids: number[]): Promise<number[]> {
try {
const result = await spawn('pgrep', ['-P', parentPids.join(',')], {
stdio: 'pipe',
});
return result.stdout
.toString()
.split('\n')
.map(i => Number(i.trim()))
.filter(i => i);
} catch {
return [];
}
}
async function getParentAndDescendantProcessPidsAsync(ppid: number): Promise<number[]> {
const children = new Set<number>([ppid]);
let shouldCheckAgain = true;
while (shouldCheckAgain) {
const pids = await getChildrenPidsAsync([...children]);
shouldCheckAgain = false;
for (const pid of pids) {
if (!children.has(pid)) {
shouldCheckAgain = true;
children.add(pid);
}
}
}
return [...children];
export function getGradleShellCommand({
gradleCommand,
oomScoreAdj,
verboseFlag,
}: {
gradleCommand: string;
oomScoreAdj?: number;
verboseFlag: string;
}): string {
const oomScoreAdjPrefix =
oomScoreAdj === undefined ? '' : `echo ${oomScoreAdj} > /proc/$$/oom_score_adj || true; `;
return `${oomScoreAdjPrefix}exec ./gradlew ${gradleCommand} --profile ${verboseFlag}`;
}
export function resolveGradleCommand(job: Android.Job, command?: string): string {
@@ -6,15 +6,17 @@ import fetch from 'node-fetch';
import os from 'node:os';
import path from 'node:path';
import { pipeline } from 'node:stream/promises';
import { setTimeout as setTimeoutAsync } from 'node:timers/promises';
import { z } from 'zod';
import { CustomBuildContext } from '../../customBuildContext';
import { Sentry } from '../../sentry';
import { formatBytes } from '../../utils/artifacts';
import { sleepAsync } from '../../utils/retry';
import { uploadDeviceRunSessionArtifactAsync } from './deviceRunSessionArtifacts';
const ARGENT_ARTIFACT_UPLOAD_POLL_INTERVAL_MS = 5_000;
const ARGENT_ARTIFACT_UPLOAD_CLEANUP_TIMEOUT_MS = 30_000;
const ARGENT_ARTIFACT_FETCH_TIMEOUT_MS = 10_000;
const ArgentArtifactSchema = z.object({
id: z.string(),
@@ -35,39 +37,56 @@ export async function pollArgentArtifactsForUploadAsync(
toolsUrl,
toolsAuthToken,
logger,
signal,
}: {
deviceRunSessionId: string;
toolsUrl: string;
toolsAuthToken?: string;
logger: bunyan;
signal: AbortSignal;
}
): Promise<never> {
): Promise<void> {
logger.info('Started polling Argent tool-server for artifacts.');
const seenArtifactIds = new Set<string>();
const pendingUploads = new Set<Promise<void>>();
let listArtifactsErrorCount = 0;
for (;;) {
const queueArtifactUpload = (artifact: ArgentArtifact): void => {
if (seenArtifactIds.has(artifact.id)) {
return;
}
seenArtifactIds.add(artifact.id);
const uploadPromise = uploadArgentArtifactAsync(ctx, {
deviceRunSessionId,
toolsUrl,
toolsAuthToken,
artifact,
logger,
}).catch(err => {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not upload Argent remote session artifact', error);
logger.warn({ err: error }, 'Could not upload Argent remote session artifact.');
});
pendingUploads.add(uploadPromise);
void uploadPromise.finally(() => {
pendingUploads.delete(uploadPromise);
});
};
const listArtifactsForUploadAsync = async ({
signal,
}: {
signal?: AbortSignal;
} = {}): Promise<ArgentArtifact[]> => {
try {
const artifacts = await listArgentArtifactsAsync({ toolsUrl, toolsAuthToken });
const artifacts = await listArgentArtifactsAsync({ toolsUrl, toolsAuthToken, signal });
listArtifactsErrorCount = 0;
for (const artifact of artifacts) {
if (seenArtifactIds.has(artifact.id)) {
continue;
}
seenArtifactIds.add(artifact.id);
void uploadArgentArtifactAsync(ctx, {
deviceRunSessionId,
toolsUrl,
toolsAuthToken,
artifact,
logger,
}).catch(err => {
const error = err instanceof Error ? err : new Error(String(err));
Sentry.capture('Could not upload Argent remote session artifact', error);
logger.warn({ err: error }, 'Could not upload Argent remote session artifact.');
});
}
return artifacts;
} catch (err) {
if (signal?.aborted) {
return [];
}
const error = err instanceof Error ? err : new Error(String(err));
listArtifactsErrorCount += 1;
if (listArtifactsErrorCount === 1 || listArtifactsErrorCount % 5 === 0) {
@@ -77,20 +96,47 @@ export async function pollArgentArtifactsForUploadAsync(
'Could not list Argent remote session artifacts.'
);
}
return [];
}
await sleepAsync(ARGENT_ARTIFACT_UPLOAD_POLL_INTERVAL_MS);
};
const listAndQueueArtifactUploadsAsync = async ({
signal,
}: {
signal?: AbortSignal;
} = {}): Promise<void> => {
const artifacts = await listArtifactsForUploadAsync({ signal });
for (const artifact of artifacts) {
queueArtifactUpload(artifact);
}
};
while (!signal.aborted) {
await listAndQueueArtifactUploadsAsync({ signal });
await sleepUntilAbortedAsync(ARGENT_ARTIFACT_UPLOAD_POLL_INTERVAL_MS, signal);
}
logger.info('Argent artifact polling stopped; draining remaining artifacts.');
await listAndQueueArtifactUploadsAsync();
await waitForPendingUploadsAsync({
pendingUploads,
timeoutMs: ARGENT_ARTIFACT_UPLOAD_CLEANUP_TIMEOUT_MS,
logger,
});
}
export async function listArgentArtifactsAsync({
toolsUrl,
toolsAuthToken,
signal,
}: {
toolsUrl: string;
toolsAuthToken?: string;
signal?: AbortSignal;
}): Promise<ArgentArtifact[]> {
const response = await fetch(new URL('/artifacts', toolsUrl).toString(), {
headers: toolsAuthToken ? { Authorization: `Bearer ${toolsAuthToken}` } : {},
signal: createTimeoutSignal({ signal, timeoutMs: ARGENT_ARTIFACT_FETCH_TIMEOUT_MS }),
});
if (!response.ok) {
throw new SystemError(
@@ -138,6 +184,7 @@ export async function uploadArgentArtifactAsync(
artifactId: artifact.id,
name: `${filename} (${artifact.id})`,
filename,
kind: undefined,
size,
stream: createReadStream(temporaryArtifactPath),
});
@@ -159,6 +206,10 @@ async function downloadArgentArtifactToFileAsync({
}): Promise<void> {
const response = await fetch(new URL(`/artifacts/${artifact.id}`, toolsUrl).toString(), {
headers: toolsAuthToken ? { Authorization: `Bearer ${toolsAuthToken}` } : {},
signal: createTimeoutSignal({
signal: undefined,
timeoutMs: ARGENT_ARTIFACT_FETCH_TIMEOUT_MS,
}),
});
if (!response.ok) {
throw new SystemError(
@@ -172,3 +223,62 @@ async function downloadArgentArtifactToFileAsync({
}
await pipeline(response.body, createWriteStream(destinationPath));
}
async function sleepUntilAbortedAsync(timeoutMs: number, signal: AbortSignal): Promise<void> {
try {
await setTimeoutAsync(timeoutMs, undefined, { signal });
} catch (err) {
if (!signal.aborted) {
throw err;
}
}
}
function createTimeoutSignal({
signal,
timeoutMs,
}: {
signal: AbortSignal | undefined;
timeoutMs: number;
}): AbortSignal {
const timeoutSignal = AbortSignal.timeout(timeoutMs);
return signal ? AbortSignal.any([signal, timeoutSignal]) : timeoutSignal;
}
async function waitForPendingUploadsAsync({
pendingUploads,
timeoutMs,
logger,
}: {
pendingUploads: Set<Promise<void>>;
timeoutMs: number;
logger: bunyan;
}): Promise<void> {
if (pendingUploads.size === 0) {
return;
}
logger.info(`Waiting for ${pendingUploads.size} Argent artifact upload(s) to finish.`);
let timeout: NodeJS.Timeout | undefined;
try {
await Promise.race([
Promise.allSettled([...pendingUploads]),
new Promise<never>((_resolve, reject) => {
timeout = setTimeout(() => {
reject(
new SystemError(
`Timed out after ${Math.round(timeoutMs / 1000)}s waiting for Argent artifact uploads.`,
{
trackingCode: 'SIMULATOR_ARTIFACT_SLOW_UPLOAD',
}
)
);
}, timeoutMs);
}),
]);
} finally {
if (timeout) {
clearTimeout(timeout);
}
}
}
@@ -27,6 +27,8 @@ export async function uploadDeviceRunSessionArtifactAsync(
artifactId,
name,
filename,
kind,
metadata,
size,
stream,
}: {
@@ -34,6 +36,8 @@ export async function uploadDeviceRunSessionArtifactAsync(
artifactId: string;
name: string;
filename: string;
kind: string | undefined;
metadata?: Record<string, unknown>;
size: number;
stream: NodeJS.ReadableStream;
}
@@ -43,6 +47,8 @@ export async function uploadDeviceRunSessionArtifactAsync(
artifactId,
name,
filename,
kind,
metadata,
size,
});
const response = await fetch(uploadSession.url, {
@@ -65,12 +71,16 @@ async function createDeviceRunSessionArtifactUploadSessionAsync(
artifactId,
name,
filename,
kind,
metadata,
size,
}: {
deviceRunSessionId: string;
artifactId: string;
name: string;
filename: string;
kind: string | undefined;
metadata?: Record<string, unknown>;
size: number;
}
) {
@@ -80,6 +90,8 @@ async function createDeviceRunSessionArtifactUploadSessionAsync(
input: {
name,
filename,
...(kind !== undefined ? { kind } : {}),
...(metadata !== undefined ? { metadata } : {}),
size,
},
})
@@ -8,6 +8,7 @@ import nullthrows from 'nullthrows';
import { z } from 'zod';
import { randomBytes } from 'node:crypto';
import fs from 'node:fs';
import { setTimeout as setTimeoutAsync } from 'node:timers/promises';
import { CustomBuildContext } from '../../customBuildContext';
import { Sentry } from '../../sentry';
@@ -30,6 +31,19 @@ const START_DEVICE_RUN_SESSION_MUTATION = graphql(`
}
`);
const DEVICE_RUN_SESSION_STATUS_QUERY = graphql(`
query DeviceRunSessionStatus($deviceRunSessionId: ID!) {
deviceRunSessions {
byId(deviceRunSessionId: $deviceRunSessionId) {
id
status
}
}
}
`);
const DEVICE_RUN_SESSION_STATUS_POLL_INTERVAL_MS = 5_000;
export function getDeviceRunSessionIdOrThrow(env: BuildStepEnv): string {
const deviceRunSessionId = env.DEVICE_RUN_SESSION_ID;
if (!deviceRunSessionId) {
@@ -96,6 +110,75 @@ export async function selectXcodeDeveloperDirectoryAsync({
});
}
export async function waitForDeviceRunSessionStoppedAsync({
ctx,
deviceRunSessionId,
logger,
signal,
}: {
ctx: CustomBuildContext;
deviceRunSessionId: string;
logger: bunyan;
signal?: AbortSignal;
}): Promise<void> {
logger.info(
`Remote session is live. Polling device run session ${deviceRunSessionId} until it is stopped.`
);
let pollErrorCount = 0;
while (!signal?.aborted) {
try {
const result = await ctx.graphqlClient
.query(DEVICE_RUN_SESSION_STATUS_QUERY, { deviceRunSessionId })
.toPromise();
if (result.error) {
throw result.error;
}
const status = result.data?.deviceRunSessions?.byId?.status;
if (!status) {
throw new Error(`Device run session ${deviceRunSessionId} status response was missing.`);
}
pollErrorCount = 0;
if (status === 'STOPPED') {
logger.info(`Device run session ${deviceRunSessionId} was stopped.`);
return;
}
if (status === 'ERRORED') {
throw new SystemError(`Device run session ${deviceRunSessionId} errored.`);
}
} catch (err) {
if (err instanceof SystemError) {
throw err;
}
const error = err instanceof Error ? err : new Error(String(err));
pollErrorCount += 1;
if (pollErrorCount === 1 || pollErrorCount % 5 === 0) {
Sentry.capture('Could not poll device run session status', error, { level: 'warning' });
logger.warn(
{ err: error, failedStatusPollCount: pollErrorCount },
'Could not poll device run session status; will retry.'
);
}
}
await sleepUntilAbortedAsync(DEVICE_RUN_SESSION_STATUS_POLL_INTERVAL_MS, signal);
}
}
async function sleepUntilAbortedAsync(
timeoutMs: number,
signal: AbortSignal | undefined
): Promise<void> {
try {
await setTimeoutAsync(timeoutMs, undefined, signal ? { signal } : undefined);
} catch (err) {
if (!signal?.aborted) {
throw err;
}
}
}
const TurnIceServersResponseSchema = z.object({
data: z.object({
iceServers: TurnIceServersSchema,
@@ -210,6 +293,8 @@ export async function uploadRemoteSessionConfigAsync({
}
export type DetachedProcessHandle = {
/** PID of the directly spawned process, if the OS assigned one. */
pid: number | undefined;
getOutput: () => string;
};
@@ -242,7 +327,7 @@ export function spawnDetached({
promise.child.stdout?.on('data', appendChunk);
promise.child.stderr?.on('data', appendChunk);
return { getOutput: () => output };
return { pid: promise.child.pid, getOutput: () => output };
}
export async function startServeSimWithTunnelAsync(
@@ -258,7 +343,7 @@ export async function startServeSimWithTunnelAsync(
logger: bunyan;
timeoutMs: number;
}
): Promise<{ previewUrl: string; streamUrl: string }> {
): Promise<{ previewUrl: string }> {
logger.info('Launching serve-sim with tunnel.');
const turnArgs = await fetchServeSimTurnArgsAsync(ctx, { env, logger });
const serveSim = spawnDetached({
@@ -281,33 +366,30 @@ export async function startServeSimWithTunnelAsync(
env,
});
logger.info('Waiting for serve-sim to report tunnel and stream URLs.');
logger.info('Waiting for serve-sim to report tunnel URL.');
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
const output = serveSim.getOutput();
const previewUrl = matchLabeledUrl({ output, label: 'Tunnel', baseDomain });
const streamUrl = matchLabeledUrl({ output, label: 'Stream', baseDomain });
if (previewUrl && streamUrl) {
return { previewUrl, streamUrl };
const previewUrl = matchTunnelUrl({ output, baseDomain });
if (previewUrl) {
return { previewUrl };
}
await sleepAsync(1_000);
}
throw new SystemError(
`Timed out waiting for serve-sim to report Tunnel and Stream URLs. Last output:\n${serveSim.getOutput() || '<empty>'}`
`Timed out waiting for serve-sim to report Tunnel URL. Last output:\n${serveSim.getOutput() || '<empty>'}`
);
}
function matchLabeledUrl({
function matchTunnelUrl({
output,
label,
baseDomain,
}: {
output: string;
label: string;
baseDomain: string;
}): string | null {
const labelPattern = new RegExp(
`${label}:\\s*(https:\\/\\/[a-z0-9-]+\\.${escapeRegExp(baseDomain)})`
`Tunnel:\\s*(https:\\/\\/[a-z0-9-]+\\.${escapeRegExp(baseDomain)})`
);
const match = labelPattern.exec(output);
return match ? match[1] : null;
@@ -1,10 +1,8 @@
import { randomBytes } from 'crypto';
import fs from 'fs-extra';
import fetch, { RequestInit, Response } from 'node-fetch';
import path from 'path';
import { Readable } from 'stream';
import GCS from '../client';
import { type SignedUrl, uploadWithSignedUrl } from '../uploadWithSignedUrl';
jest.mock('node-fetch');
@@ -29,18 +27,15 @@ class DNSError extends ErrorWithCode {
const TEST_BUCKET = 'turtle-v2-test';
let googleApplicationCredentials: string | undefined;
beforeAll(() => {
googleApplicationCredentials = process.env.GOOGLE_APPLICATION_CREDENTIALS;
process.env.GOOGLE_APPLICATION_CREDENTIALS = path.join(__dirname, 'mock-credentials.json');
});
function createSignedUrl(key: string, contentType: string): SignedUrl {
return {
url: `https://storage.googleapis.com/${TEST_BUCKET}/${key}?signature=test`,
headers: { 'content-type': contentType },
};
}
afterAll(() => {
process.env.GOOGLE_APPLICATION_CREDENTIALS = googleApplicationCredentials;
});
describe('GCS client', () => {
describe('uploadWithPresignedURL function', () => {
describe('signed URL uploader', () => {
describe('uploadWithSignedUrl', () => {
it('should throw an error if upload fails', async () => {
const fetchMock = jest.mocked(fetch);
const res = {
@@ -48,22 +43,16 @@ describe('GCS client', () => {
status: 500,
} as Response;
const localImagePath = path.join(__dirname, 'cat.jpg');
const key = 'cat.jpg';
const gcs = new GCS(TEST_BUCKET);
const signedUrl = await gcs.createSignedUploadUrl({
key,
expirationTime: 30000,
contentType: 'image/jpeg',
});
const signedUrl = createSignedUrl(key, 'image/jpeg');
fetchMock.mockImplementation(async () => res);
await expect(
GCS.uploadWithSignedUrl({
uploadWithSignedUrl({
signedUrl,
srcGeneratorAsync: async () => fs.createReadStream(localImagePath),
retryIntervalMs: 500,
srcGeneratorAsync: async () => Readable.from('image'),
retryIntervalMs: 0,
})
).rejects.toThrow();
});
@@ -75,21 +64,15 @@ describe('GCS client', () => {
status: 200,
} as Response;
const localImagePath = path.join(__dirname, 'cat.jpg');
const key = 'cat.jpg';
const gcs = new GCS(TEST_BUCKET);
const signedUrl = await gcs.createSignedUploadUrl({
key,
expirationTime: 30000,
contentType: 'image/jpeg',
});
const signedUrl = createSignedUrl(key, 'image/jpeg');
fetchMock.mockImplementation(async () => res);
const result = await GCS.uploadWithSignedUrl({
const result = await uploadWithSignedUrl({
signedUrl,
srcGeneratorAsync: async () => fs.createReadStream(localImagePath),
retryIntervalMs: 500,
srcGeneratorAsync: async () => Readable.from('image'),
retryIntervalMs: 0,
});
expect(result).toEqual('https://storage.googleapis.com/turtle-v2-test/cat.jpg');
});
@@ -101,15 +84,9 @@ describe('GCS client', () => {
status: 200,
} as Response;
const localImagePath = path.join(__dirname, 'cat.jpg');
const key = 'cat.jpg';
const gcs = new GCS(TEST_BUCKET);
const signedUrl = await gcs.createSignedUploadUrl({
key,
expirationTime: 30000,
contentType: 'image/jpeg',
});
const signedUrl = createSignedUrl(key, 'image/jpeg');
fetchMock
.mockImplementationOnce(async () => {
@@ -119,10 +96,10 @@ describe('GCS client', () => {
throw new DNSError('failed twice', 'EAI_AGAIN');
})
.mockImplementation(async () => res);
const result = await GCS.uploadWithSignedUrl({
const result = await uploadWithSignedUrl({
signedUrl,
srcGeneratorAsync: async () => fs.createReadStream(localImagePath),
retryIntervalMs: 500,
srcGeneratorAsync: async () => Readable.from('image'),
retryIntervalMs: 0,
});
expect(result).toEqual('https://storage.googleapis.com/turtle-v2-test/cat.jpg');
expect(fetchMock).toHaveBeenCalledTimes(3);
@@ -135,15 +112,9 @@ describe('GCS client', () => {
status: 200,
} as Response;
const localImagePath = path.join(__dirname, 'cat.jpg');
const key = 'cat.jpg';
const gcs = new GCS(TEST_BUCKET);
const signedUrl = await gcs.createSignedUploadUrl({
key,
expirationTime: 30000,
contentType: 'image/jpeg',
});
const signedUrl = createSignedUrl(key, 'image/jpeg');
fetchMock
.mockImplementationOnce(async () => {
@@ -159,10 +130,10 @@ describe('GCS client', () => {
} as Response;
})
.mockImplementation(async () => res);
const result = await GCS.uploadWithSignedUrl({
const result = await uploadWithSignedUrl({
signedUrl,
srcGeneratorAsync: async () => fs.createReadStream(localImagePath),
retryIntervalMs: 500,
srcGeneratorAsync: async () => Readable.from('image'),
retryIntervalMs: 0,
});
expect(result).toEqual('https://storage.googleapis.com/turtle-v2-test/cat.jpg');
expect(fetchMock).toHaveBeenCalledTimes(3);
@@ -175,15 +146,9 @@ describe('GCS client', () => {
status: 200,
} as Response;
const localImagePath = path.join(__dirname, 'cat.jpg');
const key = 'cat.jpg';
const gcs = new GCS(TEST_BUCKET);
const signedUrl = await gcs.createSignedUploadUrl({
key,
expirationTime: 30000,
contentType: 'image/jpeg',
});
const signedUrl = createSignedUrl(key, 'image/jpeg');
fetchMock
.mockImplementationOnce(async () => {
@@ -196,10 +161,10 @@ describe('GCS client', () => {
throw new DNSError('failed once');
})
.mockImplementation(async () => res);
const result = await GCS.uploadWithSignedUrl({
const result = await uploadWithSignedUrl({
signedUrl,
srcGeneratorAsync: async () => fs.createReadStream(localImagePath),
retryIntervalMs: 500,
srcGeneratorAsync: async () => Readable.from('image'),
retryIntervalMs: 0,
});
expect(result).toEqual('https://storage.googleapis.com/turtle-v2-test/cat.jpg');
expect(fetchMock).toHaveBeenCalledTimes(3);
@@ -212,15 +177,9 @@ describe('GCS client', () => {
status: 200,
} as Response;
const localImagePath = path.join(__dirname, 'cat.jpg');
const key = 'cat.jpg';
const gcs = new GCS(TEST_BUCKET);
const signedUrl = await gcs.createSignedUploadUrl({
key,
expirationTime: 30000,
contentType: 'image/jpeg',
});
const signedUrl = createSignedUrl(key, 'image/jpeg');
const lastDNSError = new DNSError('failed thrice');
fetchMock
@@ -235,10 +194,10 @@ describe('GCS client', () => {
})
.mockImplementation(async () => res);
await expect(
GCS.uploadWithSignedUrl({
uploadWithSignedUrl({
signedUrl,
srcGeneratorAsync: async () => fs.createReadStream(localImagePath),
retryIntervalMs: 500,
srcGeneratorAsync: async () => Readable.from('image'),
retryIntervalMs: 0,
})
).rejects.toThrow(lastDNSError);
expect(fetchMock).toHaveBeenCalledTimes(3);
@@ -251,15 +210,9 @@ describe('GCS client', () => {
status: 200,
} as Response;
const localImagePath = path.join(__dirname, 'cat.jpg');
const key = 'cat.jpg';
const gcs = new GCS(TEST_BUCKET);
const signedUrl = await gcs.createSignedUploadUrl({
key,
expirationTime: 30000,
contentType: 'image/jpeg',
});
const signedUrl = createSignedUrl(key, 'image/jpeg');
fetchMock
.mockImplementationOnce(async () => {
@@ -282,10 +235,10 @@ describe('GCS client', () => {
})
.mockImplementation(async () => res);
await expect(
GCS.uploadWithSignedUrl({
uploadWithSignedUrl({
signedUrl,
srcGeneratorAsync: async () => fs.createReadStream(localImagePath),
retryIntervalMs: 500,
srcGeneratorAsync: async () => Readable.from('image'),
retryIntervalMs: 0,
})
).rejects.toThrow();
expect(fetchMock).toHaveBeenCalledTimes(3);
@@ -298,15 +251,9 @@ describe('GCS client', () => {
status: 200,
} as Response;
const localImagePath = path.join(__dirname, 'cat.jpg');
const key = 'cat.jpg';
const gcs = new GCS(TEST_BUCKET);
const signedUrl = await gcs.createSignedUploadUrl({
key,
expirationTime: 30000,
contentType: 'image/jpeg',
});
const signedUrl = createSignedUrl(key, 'image/jpeg');
const nonDNSError = new ErrorWithCode('failed once', 'A_DIFFERENT_CODE');
fetchMock
@@ -315,10 +262,10 @@ describe('GCS client', () => {
})
.mockImplementation(async () => res);
await expect(
GCS.uploadWithSignedUrl({
uploadWithSignedUrl({
signedUrl,
srcGeneratorAsync: async () => fs.createReadStream(localImagePath),
retryIntervalMs: 500,
srcGeneratorAsync: async () => Readable.from('image'),
retryIntervalMs: 0,
})
).rejects.toThrow(nonDNSError);
expect(fetchMock).toHaveBeenCalledTimes(1);
@@ -354,19 +301,14 @@ describe('GCS client', () => {
} as Response;
});
const gcs = new GCS(TEST_BUCKET);
const signedUrl = await gcs.createSignedUploadUrl({
key: 'text.txt',
expirationTime: 30000,
contentType: 'text/plain',
});
const signedUrl = createSignedUrl('text.txt', 'text/plain');
const bufferToUpload = randomBytes(16);
const result = await GCS.uploadWithSignedUrl({
const result = await uploadWithSignedUrl({
signedUrl,
srcGeneratorAsync: async () => Readable.from(bufferToUpload),
retryIntervalMs: 500,
retryIntervalMs: 0,
});
expect(result).toEqual('https://storage.googleapis.com/turtle-v2-test/text.txt');
expect(fetchMock).toHaveBeenCalledTimes(3);
@@ -1,14 +1,14 @@
import { Response } from 'node-fetch';
export interface RetryOptions {
type RetryOptions = {
retries: number;
retryIntervalMs: number;
shouldRetryOnError: (error: any) => boolean;
shouldRetryOnError: (error: unknown) => boolean;
shouldRetryOnResponse: (response: Response) => boolean;
}
};
// based on https://github.com/googleapis/nodejs-storage/blob/8ab50804fc7bae3bbd159bbb4adf65c02215b11b/src/storage.ts#L284-L320
export async function retryOnGCSUploadFailure(
export async function retryOnUploadFailure(
fn: (attemptCount: number) => Promise<Response>,
{
retries,
@@ -18,13 +18,14 @@ export async function retryOnGCSUploadFailure(
return await retry(fn, {
retries,
retryIntervalMs,
shouldRetryOnError: e => {
shouldRetryOnError: err => {
return (
e.code === 'ENOTFOUND' ||
e.code === 'EAI_AGAIN' ||
e.code === 'ECONNRESET' ||
e.code === 'ETIMEDOUT' ||
e.code === 'EPIPE'
isErrorWithCode(err) &&
(err.code === 'ENOTFOUND' ||
err.code === 'EAI_AGAIN' ||
err.code === 'ECONNRESET' ||
err.code === 'ETIMEDOUT' ||
err.code === 'EPIPE')
);
},
shouldRetryOnResponse: resp => {
@@ -33,8 +34,12 @@ export async function retryOnGCSUploadFailure(
});
}
function isErrorWithCode(error: unknown): error is { code: string } {
return typeof error === 'object' && error !== null && 'code' in error;
}
/**
* Wrapper used to execute an inner function and possibly retry it if it throws and error
* Wrapper used to execute an inner function and possibly retry it if it throws an error
* @param fn Function to be executed and retried in case of error
* @param retries How many times at most should the function be retried
* @param retryIntervalMs Time interval between the retries
@@ -55,7 +60,7 @@ async function retry(
} else {
return resp;
}
} catch (err: any) {
} catch (err: unknown) {
if (attemptCount === retries || !shouldRetryOnError(err)) {
throw err;
}
@@ -0,0 +1,57 @@
import fetch, { FetchError } from 'node-fetch';
import { Readable } from 'stream';
import { URL } from 'url';
import { retryOnUploadFailure } from './retry';
export type SignedUrl = {
url: string;
headers: Record<string, string>;
};
export type UploadWithSignedUrlParams = {
signedUrl: SignedUrl;
srcGeneratorAsync: () => Promise<Readable>;
retryIntervalMs?: number;
retries?: number;
};
export async function uploadWithSignedUrl({
signedUrl,
srcGeneratorAsync,
retries = 2,
retryIntervalMs = 30_000,
}: UploadWithSignedUrlParams): Promise<string> {
let resp;
try {
resp = await retryOnUploadFailure(
async () => {
const src = await srcGeneratorAsync();
return await fetch(signedUrl.url, {
method: 'PUT',
headers: signedUrl.headers,
body: src,
});
},
{ retries, retryIntervalMs }
);
} catch (err: unknown) {
if (err instanceof FetchError) {
throw new Error(`Failed to upload the file, reason: ${err.code}`);
}
throw err;
}
if (!resp.ok) {
let body: string | undefined;
try {
body = await resp.text();
} catch {}
throw new Error(
`Failed to upload file: status: ${resp.status} status text: ${resp.statusText}, body: ${body}`
);
}
const url = new URL(signedUrl.url);
return `${url.protocol}//${url.host}${url.pathname}`;
}
@@ -329,6 +329,9 @@ export namespace AndroidEmulatorUtils {
await emulatorPromise;
}
emulatorPromise.child.unref();
// The emulator is detached and managed through adb. Observe its process promise
// immediately so expected exits during retry cleanup do not become unhandled rejections.
void asyncResult(emulatorPromise);
const serialId = await retryAsync(
async () => {
@@ -32,7 +32,11 @@ export namespace IosSimulatorUtils {
lastBootedAt?: string;
};
type SimulatorDevice = XcrunSimctlDevice & { runtime: string; displayName: string };
type SimulatorDevice = XcrunSimctlDevice & {
runtime: string;
runtimeDisplayName: string;
displayName: string;
};
type XcrunSimctlListDevicesJsonOutput = {
devices: {
@@ -60,6 +64,7 @@ export namespace IosSimulatorUtils {
...devices.map(device => ({
...device,
runtime,
runtimeDisplayName: formatRuntimeDisplayName(runtime),
displayName: `${device.name} (${device.udid}) on ${runtime}`,
}))
);
@@ -226,16 +231,32 @@ export namespace IosSimulatorUtils {
udid: IosSimulatorUuid;
env: NodeJS.ProcessEnv;
}): Promise<void> {
await spawn(
'xcrun',
['simctl', 'spawn', udid, 'launchctl', 'disable', 'system/com.apple.apsd'],
{ env }
);
await spawn(
'xcrun',
['simctl', 'spawn', udid, 'launchctl', 'bootout', 'system/com.apple.apsd'],
{ env }
);
const launchctlDomains = ['user/foreground', 'system'];
let lastError: unknown;
for (const domain of launchctlDomains) {
const service = `${domain}/com.apple.apsd`;
try {
await spawn('xcrun', ['simctl', 'spawn', udid, 'launchctl', 'disable', service], { env });
try {
await spawn('xcrun', ['simctl', 'spawn', udid, 'launchctl', 'bootout', service], {
env,
});
} catch (err) {
// bootout can fail when apsd is already gone; verify the service state below.
lastError = err;
}
if (!(await isLaunchctlServiceLoadedAsync({ udid, env, serviceLabel: 'com.apple.apsd' }))) {
return;
}
} catch (err) {
lastError = err;
}
}
throw lastError ?? new SystemError('Unable to disable apsd in the Simulator.');
}
export async function collectLogsAsync({
@@ -358,6 +379,31 @@ export namespace IosSimulatorUtils {
}
}
function formatRuntimeDisplayName(runtimeIdentifier: string): string {
const match = /^com\.apple\.CoreSimulator\.SimRuntime\.([^-]+)-(.+)$/.exec(runtimeIdentifier);
return match ? `${match[1]} ${match[2].replaceAll('-', '.')}` : runtimeIdentifier;
}
async function isLaunchctlServiceLoadedAsync({
udid,
env,
serviceLabel,
}: {
udid: IosSimulatorUuid;
env: NodeJS.ProcessEnv;
serviceLabel: string;
}): Promise<boolean> {
try {
await spawn('xcrun', ['simctl', 'spawn', udid, 'launchctl', 'list', serviceLabel], { env });
return true;
} catch (err) {
if (err instanceof Error && 'status' in err && (err as { status: unknown }).status === 113) {
return false;
}
throw err;
}
}
function parseUdidFromBootstatusStdout(stdout: string): IosSimulatorUuid | null {
const matches = stdout.match(/^Monitoring boot status for .+ \((.+)\)\.$/m);
if (!matches) {
@@ -23,6 +23,39 @@ describe('IosSimulatorUtils', () => {
mockedSpawn.mockResolvedValue({ stdout: '', stderr: '' } as any);
});
describe(IosSimulatorUtils.getAvailableDevicesAsync, () => {
it('adds a human-readable runtime display name', async () => {
mockedSpawn.mockResolvedValue({
stdout: JSON.stringify({
devices: {
'com.apple.CoreSimulator.SimRuntime.iOS-18-6': [
{
dataPath: '/tmp/eas-test-simulator-data',
dataPathSize: 1,
logPath: '/tmp/eas-test-simulator-logs',
udid: 'test-udid',
isAvailable: true,
deviceTypeIdentifier: 'com.apple.CoreSimulator.SimDeviceType.iPhone-16',
state: 'Booted',
name: 'iPhone 16',
},
],
},
}),
stderr: '',
} as any);
await expect(
IosSimulatorUtils.getAvailableDevicesAsync({ env: process.env, filter: 'booted' })
).resolves.toEqual([
expect.objectContaining({
name: 'iPhone 16',
runtimeDisplayName: 'iOS 18.6',
}),
]);
});
});
describe(IosSimulatorUtils.startAsync, () => {
it('waits for boot completion without writing accessibility prefs', async () => {
mockedSpawn.mockImplementation((async (command: string, args: string[]) => {
@@ -178,7 +211,75 @@ describe('IosSimulatorUtils', () => {
});
describe(IosSimulatorUtils.disableApsdAsync, () => {
it('disables and boots out apsd in the simulator', async () => {
it('disables and boots out apsd in the simulator foreground user domain', async () => {
mockedSpawn.mockImplementation((async (_command: string, args: string[]) => {
if (args.join(' ') === 'simctl spawn test-udid launchctl list com.apple.apsd') {
throw Object.assign(new Error('apsd not loaded'), { status: 113 });
}
return { stdout: '', stderr: '' } as any;
}) as any);
await IosSimulatorUtils.disableApsdAsync({
udid: 'test-udid' as any,
env: process.env,
});
expect(mockedSpawn).toHaveBeenCalledWith(
'xcrun',
['simctl', 'spawn', 'test-udid', 'launchctl', 'disable', 'user/foreground/com.apple.apsd'],
{ env: process.env }
);
expect(mockedSpawn).toHaveBeenCalledWith(
'xcrun',
['simctl', 'spawn', 'test-udid', 'launchctl', 'bootout', 'user/foreground/com.apple.apsd'],
{ env: process.env }
);
expect(mockedSpawn).toHaveBeenCalledWith(
'xcrun',
['simctl', 'spawn', 'test-udid', 'launchctl', 'list', 'com.apple.apsd'],
{ env: process.env }
);
expect(mockedSpawn).not.toHaveBeenCalledWith(
'xcrun',
['simctl', 'spawn', 'test-udid', 'launchctl', 'disable', 'system/com.apple.apsd'],
{ env: process.env }
);
});
it('succeeds when bootout fails after apsd is already stopped', async () => {
mockedSpawn.mockImplementation((async (_command: string, args: string[]) => {
if (
args.join(' ') ===
'simctl spawn test-udid launchctl bootout user/foreground/com.apple.apsd'
) {
throw Object.assign(new Error('bootout failed'), { status: 3 });
}
if (args.join(' ') === 'simctl spawn test-udid launchctl list com.apple.apsd') {
throw Object.assign(new Error('apsd not loaded'), { status: 113 });
}
return { stdout: '', stderr: '' } as any;
}) as any);
await expect(
IosSimulatorUtils.disableApsdAsync({
udid: 'test-udid' as any,
env: process.env,
})
).resolves.toBeUndefined();
});
it('falls back to the system domain when the foreground user domain keeps apsd running', async () => {
let listCount = 0;
mockedSpawn.mockImplementation((async (_command: string, args: string[]) => {
if (args.join(' ') === 'simctl spawn test-udid launchctl list com.apple.apsd') {
listCount += 1;
if (listCount === 2) {
throw Object.assign(new Error('apsd not loaded'), { status: 113 });
}
}
return { stdout: '', stderr: '' } as any;
}) as any);
await IosSimulatorUtils.disableApsdAsync({
udid: 'test-udid' as any,
env: process.env,
@@ -189,11 +290,19 @@ describe('IosSimulatorUtils', () => {
['simctl', 'spawn', 'test-udid', 'launchctl', 'disable', 'system/com.apple.apsd'],
{ env: process.env }
);
expect(mockedSpawn).toHaveBeenCalledWith(
'xcrun',
['simctl', 'spawn', 'test-udid', 'launchctl', 'bootout', 'system/com.apple.apsd'],
{ env: process.env }
);
});
it('throws a SystemError when apsd is still running after all disable attempts', async () => {
mockedSpawn.mockImplementation((async () => {
return { stdout: '', stderr: '' } as any;
}) as any);
await expect(
IosSimulatorUtils.disableApsdAsync({
udid: 'test-udid' as any,
env: process.env,
})
).rejects.toThrow(SystemError);
});
});
});
@@ -32,6 +32,7 @@ function zipEntryMap(entries: Record<string, true>): Record<string, { name: stri
function makeCtx(overrides: {
platform: Platform;
simulator?: boolean;
developmentClient?: boolean;
channel?: string;
env?: Record<string, string>;
}): BuildContext<any> {
@@ -40,10 +41,12 @@ function makeCtx(overrides: {
? {
platform: Platform.IOS,
simulator: overrides.simulator ?? false,
developmentClient: overrides.developmentClient ?? false,
updates: overrides.channel ? { channel: overrides.channel } : undefined,
}
: {
platform: Platform.ANDROID,
developmentClient: overrides.developmentClient ?? false,
updates: overrides.channel ? { channel: overrides.channel } : undefined,
};
@@ -192,6 +195,20 @@ describe('uploadEmbeddedBundleAsync', () => {
expect(artifacts.findArtifacts).not.toHaveBeenCalled();
});
it('skips development client builds', async () => {
const ctx = makeCtx({
platform: Platform.ANDROID,
developmentClient: true,
channel: 'development',
});
await uploadEmbeddedBundleAsync(ctx);
expect(ctx.markBuildPhaseSkipped).toHaveBeenCalled();
expect(ctx.markBuildPhaseHasWarnings).not.toHaveBeenCalled();
expect(artifacts.findArtifacts).not.toHaveBeenCalled();
});
it('warns when bundle or manifest is missing from the archive', async () => {
jest.mocked(artifacts.findArtifacts).mockResolvedValue(['/tmp/app-release.apk']);
mockZipEntries.mockResolvedValue(
@@ -0,0 +1,39 @@
import { Datadog } from '../../datadog';
import { reportWorkflowHookMetricToDatadog } from '../hookMetrics';
describe(reportWorkflowHookMetricToDatadog, () => {
const distributionSpy = jest.spyOn(Datadog, 'distribution').mockImplementation(() => {});
beforeEach(() => {
distributionSpy.mockClear();
});
it('emits one eas.workflow.hook distribution tagged by anchor/timing/result', () => {
reportWorkflowHookMetricToDatadog({
anchor: 'install_node_modules',
timing: 'before',
result: 'success',
});
expect(distributionSpy).toHaveBeenCalledTimes(1);
expect(distributionSpy).toHaveBeenCalledWith('eas.workflow.hook', 1, {
anchor: 'install_node_modules',
timing: 'before',
result: 'success',
});
});
it('adds the anchor_result tag on after events', () => {
reportWorkflowHookMetricToDatadog({
anchor: 'submit',
timing: 'after',
result: 'failed',
anchorResult: 'failed',
});
expect(distributionSpy).toHaveBeenCalledWith('eas.workflow.hook', 1, {
anchor: 'submit',
timing: 'after',
result: 'failed',
anchor_result: 'failed',
});
});
});
@@ -18,6 +18,11 @@ export async function uploadEmbeddedBundleAsync(ctx: BuildContext<BuildJob>): Pr
return;
}
if (ctx.job.developmentClient) {
ctx.markBuildPhaseSkipped();
return;
}
const { platform } = ctx.job;
if (platform === Platform.IOS && (ctx.job as Ios.Job).simulator) {
ctx.markBuildPhaseSkipped();
@@ -0,0 +1,18 @@
import { WorkflowHookMetric } from '@expo/steps';
import { Datadog } from '../datadog';
/**
* The engine's provider callback is the ONLY `eas.workflow.hook` emitter —
* never emit directly on top. The turtle-job-runs proxy forwards these tags
* as-is; the turtle-builds proxy (custom builds) also adds the standard build
* tags (image, sdk_version, build_mode, …), the same as every build metric.
*/
export function reportWorkflowHookMetricToDatadog(metric: WorkflowHookMetric): void {
Datadog.distribution('eas.workflow.hook', 1, {
anchor: metric.anchor,
timing: metric.timing,
result: metric.result,
...(metric.anchorResult !== undefined ? { anchor_result: metric.anchorResult } : null),
});
}
@@ -30,3 +30,29 @@ export async function getParentAndDescendantProcessPidsAsync(ppid: number): Prom
}
return [...children];
}
export async function isProcessDescendantOfAsync(
pid: number,
ancestorPid: number
): Promise<boolean> {
let currentPid = pid;
while (currentPid > 0) {
if (currentPid === ancestorPid) {
return true;
}
try {
const result = await spawn('ps', ['-p', String(currentPid), '-o', 'ppid='], {
stdio: 'pipe',
});
const parentPid = Number(result.stdout.toString().trim());
if (!Number.isInteger(parentPid) || parentPid <= 0 || parentPid === currentPid) {
return false;
}
currentPid = parentPid;
} catch {
return false;
}
}
return false;
}
@@ -1,6 +1,6 @@
{
"name": "create-eas-build-function",
"version": "20.1.0",
"version": "21.0.3",
"description": "Create functions for use in EAS Build custom builds.",
"keywords": [
"expo",
@@ -35,7 +35,7 @@
"clean": "rimraf ./build/ \"*.tsbuildinfo\""
},
"dependencies": {
"@expo/steps": "20.1.0"
"@expo/steps": "21.0.3"
},
"devDependencies": {
"@expo/package-manager": "1.9.10",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/downloader",
"version": "20.0.0",
"version": "21.0.0",
"bugs": "https://github.com/expo/eas-cli/issues",
"license": "BUSL-1.1",
"author": "Expo <[email protected]>",
Loaded 100 of 243 files, more files were not shown because too many files have changed in this diff. Show more