Compare commits

...
Author SHA1 Message Date
AbbanMustafa 80494c297f redact 2026-05-11 17:31:31 -04:00
AbbanMustafa 9052e3e446 redact 2026-05-11 16:20:31 -04:00
AbbanMustafa ec6772317a redact 2026-05-11 15:07:25 -04:00
Mustafa Abban d9ff560dab [build-internal] - handle errors when running internal build (#3706)
* [build-internal] - handle errors when running internal build

* [build-internal] - handle errors when running internal build

* [build-internal] - handle errors when running internal build
2026-05-11 13:15:49 -04:00
Ash Wu 1f53177ecf [build-tools] bump xclogparser default version (#3703)
Signed-off-by: Ash Wu <[email protected]>
2026-05-11 21:55:38 +08:00
Ash Wu b1f606979a [build-tools] improve xcactivitylog output (#3702)
Signed-off-by: Ash Wu <[email protected]>
2026-05-11 21:37:01 +08:00
Ash Wu 71f968e971 [build-tools] add smart_retry to maestro job (#3688)
Signed-off-by: Ash Wu <[email protected]>
2026-05-11 21:18:51 +08:00
Szymon Dziedzic 687e8897c1 [build-tools] Add Android support to start_agent_device_remote_session (#3697)
<!-- If this PR requires a changelog entry, add it by commenting the PR with the command `/changelog-entry [breaking-change|new-feature|bug-fix|chore] [message]`. -->
<!-- You can skip the changelog check by labeling the PR with "no changelog". -->

# Why

`eas/start_agent_device_remote_session` only worked on macOS (iOS) workers — it ran `xcode-select` and used `brew` to install `cloudflared`, so an Android workflow that tried to call it would fail. We want the same agent-device remote session functionality on Android (Linux) workers so a single step can drive both platforms with identical outputs.

# How

- Made the function platform-aware by reading `global.runtimePlatform`.
- Darwin path is unchanged: `xcode-select`, then `brew install cloudflared`.
- Linux path: skip `xcode-select` and, if `cloudflared` isn't already on `PATH`, download it from GitHub releases via `sudo curl` into `/usr/local/bin/cloudflared` (arch-aware: `linux-amd64` / `linux-arm64`) and `chmod +x`.
- `bun` is pre-installed on both worker images, so no install logic is needed.
- Daemon launch (`bun run src/daemon.ts`), `~/.agent-device/daemon.json` polling, the cloudflared tunnel, and the emitted `AGENT_DEVICE_DAEMON_BASE_URL` / `AGENT_DEVICE_DAEMON_AUTH_TOKEN` are unchanged — same final effect on both platforms.

# Test Plan

- `yarn typecheck` — clean
- `yarn lint` — 0 new errors (pre-existing warnings only)
- `yarn fmt:check` — clean
- `yarn test` in `packages/build-tools` — 480 tests pass
- Still to verify on a real Android worker: confirm `cloudflared` gets installed on a fresh image, the daemon writes credentials, the tunnel returns a `*.trycloudflare.com` URL, and the exported env vars work end-to-end against a connected Android device/emulator.
- iOS path on macOS workers should be unaffected (no behavioral change).
2026-05-11 14:24:54 +02:00
Stanisław Chmiela 0d5ef2ee89 [eas-build-job] Preserve workflow job outputs (#3704)
## Summary

This adds a shared `JobOutputs` type/schema and preserves declared workflow job outputs in Android, iOS, and generic job definitions.

Before this change, `outputs` was stripped from several build job shapes during Joi validation, including normal build jobs and custom builds that use `customBuildConfig.path`. That meant downstream worker code could not upload `jobs.<name>.outputs` for workflow `type: build` jobs.

## Changes

- Add shared `JobOutputs`, `JobOutputsSchema`, and `JobOutputsSchemaZ` definitions in `common.ts`.
- Use the shared output type/schema from Android, iOS, and Generic job definitions.
- Validate `outputs` directly on the Android and iOS top-level job schemas.
- Remove output handling from `CustomBuildConfigSchema`, so that schema only controls custom config vs inline step shape.
- Add Android and iOS schema coverage for normal build jobs, config-path custom jobs, and iOS resign jobs.

## Validation

- `packages/eas-build-job`: `yarn test android.test.ts ios.test.ts generic.test.ts`
- `packages/eas-build-job`: `yarn build`
- `yarn oxfmt --check ...`
2026-05-11 13:56:44 +02:00
Stanisław Chmiela f856d4022f Enable precompiled modules proxying for pod install (#3681) 2026-05-11 10:21:35 +02:00
Juwan Wheatley 5585ef99c8 [eas-cli] set the Convex env var on EAS automatically (#3685)
# Why

In our Convex docs (https://docs.expo.dev/guides/using-convex/#save-the-convex-url-as-an-eas-environment-variable), we tell people to set an ENV var with EAS to use in their deployments. With our Convex integration, we can automate this step and make the setup UX even smoother. 

Fixes ENG-21068

# How

I added a step to the convex:connect command to set the `EXPO_PUBLIC_CONVEX_URL` EAS env var. If it already exists, we prompt the user to prevent accidental rewrites. 

# Test Plan

`yarn build` and using the local `easd` alias, run `EXPO_STAGING=true easd integrations:convex:connect`

You should see the new output talking about the env var, and the env var should also be visible in the env var dashboard page:

<img width="1564" height="264" alt="Screenshot 2026-05-08 at 04 29 16" src="https://github.com/user-attachments/assets/b9a0340b-8993-4198-b9c0-971fabe464e0" />

<img width="656" height="162" alt="Screenshot 2026-05-08 at 04 29 35" src="https://github.com/user-attachments/assets/a3721b98-30b3-4055-b8a2-276760605c4c" />
2026-05-09 05:51:47 +09:00
20 changed files with 695 additions and 113 deletions

No files matched your search

+2
View File
@@ -8,6 +8,8 @@ This is the log of notable changes to EAS CLI and related packages.
### 🎉 New features
- [eas-cli] Automatically save the Convex deployment URL as an EAS environment variable when connecting a Convex project. ([#3685](https://github.com/expo/eas-cli/pull/3685) by [@fiberjw](https://github.com/fiberjw))
### 🐛 Bug fixes
- [build-tools] Update the minimum Expo version required for iOS precompiled modules. ([#3677](https://github.com/expo/eas-cli/pull/3677) by [@sjchmiela](https://github.com/sjchmiela))
@@ -42,38 +42,42 @@ export async function runEasBuildInternalAsync<TJob extends BuildJob>({
autoSubmitArgs.push('--auto-submit');
}
const result = await spawn(
cmd,
[
...args,
'build:internal',
'--platform',
job.platform,
'--profile',
buildProfile,
...autoSubmitArgs,
],
{
cwd,
env: {
...env,
EXPO_TOKEN: nullthrows(job.secrets, 'Secrets must be defined for non-custom builds')
.robotAccessToken,
...extraEnv,
EAS_PROJECT_ROOT: projectRootOverride,
},
logger,
// This prevents printing stdout with job secrets and credentials to logs.
mode: PipeMode.STDERR_ONLY_AS_STDOUT,
}
);
try {
const result = await spawn(
cmd,
[
...args,
'build:internal',
'--platform',
job.platform,
'--profile',
buildProfile,
...autoSubmitArgs,
],
{
cwd,
env: {
...env,
EXPO_TOKEN: nullthrows(job.secrets, 'Secrets must be defined for non-custom builds')
.robotAccessToken,
...extraEnv,
EAS_PROJECT_ROOT: projectRootOverride,
},
logger,
// This prevents printing stdout with job secrets and credentials to logs.
mode: PipeMode.STDERR_ONLY_AS_STDOUT,
}
);
const stdout = result.stdout.toString();
const parsed = JSON.parse(stdout);
return validateEasBuildInternalResult({
result: parsed,
oldJob: job,
});
const stdout = result.stdout.toString();
const parsed = JSON.parse(stdout);
return validateEasBuildInternalResult({
result: parsed,
oldJob: job,
});
} catch (err: any) {
throw new Error('Failed to run eas build:internal');
}
}
export async function resolveEnvFromBuildProfileAsync<TJob extends BuildJob>(
@@ -19,9 +19,11 @@ const IOS_DIR = path.join(PROJECT_DIR, 'ios');
function makeIosBuildContext({
expoVersion,
cocoapodsCacheUrl,
usePrecompiledModules = true,
}: {
expoVersion?: string;
cocoapodsCacheUrl?: string;
usePrecompiledModules?: boolean;
}): BuildContext<Ios.Job> {
vol.fromJSON({
@@ -48,7 +50,10 @@ function makeIosBuildContext({
workingdir: WORKING_DIR,
logger: createMockLogger(),
logBuffer: { getLogs: () => [], getPhaseLogs: () => [] },
env: { __API_SERVER_URL: 'http://api.expo.test' },
env: {
__API_SERVER_URL: 'http://api.expo.test',
...(cocoapodsCacheUrl ? { EAS_BUILD_COCOAPODS_CACHE_URL: cocoapodsCacheUrl } : {}),
},
uploadArtifact: jest.fn(),
});
}
@@ -89,14 +94,38 @@ describe(installPods, () => {
cwd: IOS_DIR,
env: expect.objectContaining({
EXPO_USE_PRECOMPILED_MODULES: '1',
EXPO_PRECOMPILED_MODULES_BASE_URL:
'https://storage.googleapis.com/eas-build-precompiled-modules/',
}),
})
);
expect(
(spawn as jest.Mock).mock.calls[1][2].env.EXPO_PRECOMPILED_MODULES_BASE_URL
).toBeUndefined();
expect(ctx.logger.info).toHaveBeenCalledWith(
'Detected expo=999.0.0; enabling precompiled modules use. Installing pods with additional environment variables.\nEXPO_USE_PRECOMPILED_MODULES=1\nPrecompiled modules pod install environment is configured.'
'Detected expo=999.0.0; enabling precompiled modules use. Installing pods with additional environment variables.\nEXPO_USE_PRECOMPILED_MODULES=1\nEXPO_PRECOMPILED_MODULES_BASE_URL=https://storage.googleapis.com/eas-build-precompiled-modules/\nPrecompiled modules pod install environment is configured.'
);
});
it('proxies precompiled modules through CocoaPods cache when enabled', async () => {
const ctx = makeIosBuildContext({
expoVersion: '999.0.0',
cocoapodsCacheUrl: 'https://cocoapods-cache.expo.test/',
});
await installPods(ctx, {});
expect(spawn).toHaveBeenCalledWith(
'pod',
['install'],
expect.objectContaining({
cwd: IOS_DIR,
env: expect.objectContaining({
EXPO_USE_PRECOMPILED_MODULES: '1',
EXPO_PRECOMPILED_MODULES_BASE_URL:
'https://cocoapods-cache.expo.test/storage.googleapis.com/eas-build-precompiled-modules/',
}),
})
);
expect(ctx.logger.info).toHaveBeenCalledWith(
'Detected expo=999.0.0; enabling precompiled modules use. Installing pods with additional environment variables.\nEXPO_USE_PRECOMPILED_MODULES=1\nEXPO_PRECOMPILED_MODULES_BASE_URL=https://cocoapods-cache.expo.test/storage.googleapis.com/eas-build-precompiled-modules/\nPrecompiled modules pod install environment is configured.'
);
});
@@ -146,6 +175,9 @@ describe(installPods, () => {
expect((spawn as jest.Mock).mock.calls[0][2].env.EAS_USE_PRECOMPILED_MODULES).toBeUndefined();
expect((spawn as jest.Mock).mock.calls[0][2].env.EXPO_USE_PRECOMPILED_MODULES).toBeUndefined();
expect(
(spawn as jest.Mock).mock.calls[0][2].env.EXPO_PRECOMPILED_MODULES_BASE_URL
).toBeUndefined();
expect(ctx.logger.info).not.toHaveBeenCalled();
});
});
+14 -14
View File
@@ -7,7 +7,8 @@ import semver from 'semver';
import { BuildContext } from '../context';
const MIN_PRECOMPILED_MODULES_EXPO_VERSION = '55.0.21';
// const PRECOMPILED_MODULES_BASE_URL = 'https://storage.googleapis.com/eas-build-precompiled-modules/';
const PRECOMPILED_MODULES_BASE_URL =
'https://storage.googleapis.com/eas-build-precompiled-modules/';
export async function installPods<TJob extends Ios.Job>(
ctx: BuildContext<TJob>,
@@ -76,10 +77,9 @@ async function resolvePrecompiledModulesPodInstallEnvAsync<TJob extends Ios.Job>
return {};
}
// Start rollout with Expo precompiled modules only. Add third-party modules after this is stable.
const env: Env = {
EXPO_USE_PRECOMPILED_MODULES: '1',
// EXPO_PRECOMPILED_MODULES_BASE_URL: getPrecompiledModulesBaseUrl(),
EXPO_PRECOMPILED_MODULES_BASE_URL: getPrecompiledModulesBaseUrl(ctx),
};
ctx.logger.info(
@@ -105,14 +105,14 @@ async function getInstalledExpoPackageVersionAsync<TJob extends Ios.Job>(
return (await fs.readJson(expoPackageJsonPath)).version;
}
// function getPrecompiledModulesBaseUrl<TJob extends Ios.Job>(ctx: BuildContext<TJob>): string {
// if (!ctx.env.EAS_BUILD_COCOAPODS_CACHE_URL) {
// return PRECOMPILED_MODULES_BASE_URL;
// }
//
// const parsedUrl = new URL(PRECOMPILED_MODULES_BASE_URL);
// return PRECOMPILED_MODULES_BASE_URL.replace(
// `${parsedUrl.protocol}//${parsedUrl.host}`,
// `${ctx.env.EAS_BUILD_COCOAPODS_CACHE_URL.replace(/\/$/, '')}/${parsedUrl.host}`
// );
// }
function getPrecompiledModulesBaseUrl<TJob extends Ios.Job>(ctx: BuildContext<TJob>): string {
if (!ctx.env.EAS_BUILD_COCOAPODS_CACHE_URL) {
return PRECOMPILED_MODULES_BASE_URL;
}
const parsedUrl = new URL(PRECOMPILED_MODULES_BASE_URL);
return PRECOMPILED_MODULES_BASE_URL.replace(
`${parsedUrl.protocol}//${parsedUrl.host}`,
`${ctx.env.EAS_BUILD_COCOAPODS_CACHE_URL.replace(/\/$/, '')}/${parsedUrl.host}`
);
}
@@ -3,6 +3,7 @@ import spawn from '@expo/turtle-spawn';
import { createGlobalContextMock } from '../../../__tests__/utils/context';
import { createMockLogger } from '../../../__tests__/utils/logger';
import * as discovery from '../maestroFlowDiscovery';
import * as parser from '../maestroResultParser';
import { createMaestroTestsBuildFunction } from '../maestroTests';
@@ -207,6 +208,108 @@ describe('createMaestroTestsBuildFunction', () => {
);
});
it('runs all original flows on every retry when smart_retry=false', async () => {
mockedSpawn.mockRejectedValueOnce(rejectExit1()).mockResolvedValueOnce(SPAWN_SUCCESS);
const parseSpy = jest
.spyOn(parser, 'parseFailedFlowsFromJUnit')
.mockResolvedValue(['flows/b.yaml']);
const step = createStep({
flow_path: ['flows/a.yaml', 'flows/b.yaml'],
retries: 1,
output_format: 'junit',
smart_retry: false,
platform: 'android',
});
await step.executeAsync();
// Both attempts must see both flows in the original input order. Tail-slice
// equality locks order, not just membership — guards against any future
// failed-first reshuffling of the dumb-retry list.
const a0 = mockedSpawn.mock.calls[0][1]!;
const a1 = mockedSpawn.mock.calls[1][1]!;
expect(a0.slice(-2)).toEqual(['flows/a.yaml', 'flows/b.yaml']);
expect(a1.slice(-2)).toEqual(['flows/a.yaml', 'flows/b.yaml']);
expect(parseSpy).not.toHaveBeenCalled();
});
it('subsets to failing flows on retry when smart_retry=true (default behaviour)', async () => {
mockedSpawn.mockRejectedValueOnce(rejectExit1()).mockResolvedValueOnce(SPAWN_SUCCESS);
jest.spyOn(parser, 'parseFailedFlowsFromJUnit').mockResolvedValue(['flows/b.yaml']);
const step = createStep({
flow_path: ['flows/a.yaml', 'flows/b.yaml'],
retries: 1,
output_format: 'junit',
smart_retry: true,
platform: 'android',
});
await step.executeAsync();
expect(mockedSpawn.mock.calls[0][1]).toEqual(
expect.arrayContaining(['flows/a.yaml', 'flows/b.yaml'])
);
const a1 = mockedSpawn.mock.calls[1][1]!;
expect(a1).toContain('flows/b.yaml');
expect(a1).not.toContain('flows/a.yaml');
});
it('falls back to all-flows retry when smart_retry=true but output_format!=junit', async () => {
mockedSpawn.mockRejectedValueOnce(rejectExit1()).mockResolvedValueOnce(SPAWN_SUCCESS);
const parseSpy = jest.spyOn(parser, 'parseFailedFlowsFromJUnit');
const step = createStep({
flow_path: ['flows/a.yaml', 'flows/b.yaml'],
retries: 1,
output_format: 'html',
smart_retry: true,
platform: 'android',
});
await step.executeAsync();
expect(parseSpy).not.toHaveBeenCalled();
expect(mockedSpawn.mock.calls[1][1]).toEqual(
expect.arrayContaining(['flows/a.yaml', 'flows/b.yaml'])
);
});
it('falls back to all-flows retry when nameToPath is null (duplicate flow names)', async () => {
mockedSpawn.mockRejectedValueOnce(rejectExit1()).mockResolvedValueOnce(SPAWN_SUCCESS);
jest.spyOn(discovery, 'buildFlowNameToPathMap').mockResolvedValue(null);
const parseSpy = jest.spyOn(parser, 'parseFailedFlowsFromJUnit');
const step = createStep({
flow_path: ['flows/a.yaml', 'flows/b.yaml'],
retries: 1,
output_format: 'junit',
smart_retry: true,
platform: 'android',
});
await step.executeAsync();
expect(parseSpy).not.toHaveBeenCalled();
expect(mockedSpawn.mock.calls[1][1]).toEqual(
expect.arrayContaining(['flows/a.yaml', 'flows/b.yaml'])
);
});
it('defaults smart_retry to true when omitted from inputs', async () => {
mockedSpawn.mockRejectedValueOnce(rejectExit1()).mockResolvedValueOnce(SPAWN_SUCCESS);
jest.spyOn(parser, 'parseFailedFlowsFromJUnit').mockResolvedValue(['flows/b.yaml']);
const step = createStep({
flow_path: ['flows/a.yaml', 'flows/b.yaml'],
retries: 1,
output_format: 'junit',
platform: 'android',
});
await step.executeAsync();
const a1 = mockedSpawn.mock.calls[1][1]!;
expect(a1).toContain('flows/b.yaml');
expect(a1).not.toContain('flows/a.yaml');
});
it('writes merged junit on success', async () => {
mockedSpawn.mockResolvedValue(SPAWN_SUCCESS);
const mergeSpy = jest.spyOn(parser, 'mergeJUnitReports').mockResolvedValue();
@@ -97,6 +97,12 @@ export function createMaestroTestsBuildFunction(): BuildFunction {
defaultValue: 0,
allowedValueTypeName: BuildStepInputValueTypeName.NUMBER,
}),
BuildStepInput.createProvider({
id: 'smart_retry',
required: false,
defaultValue: true,
allowedValueTypeName: BuildStepInputValueTypeName.BOOLEAN,
}),
BuildStepInput.createProvider({
id: 'shards',
required: false,
@@ -186,6 +192,7 @@ export function createMaestroTestsBuildFunction(): BuildFunction {
inputs.shards.value,
'shards must be a positive integer.'
);
const smartRetry = inputs.smart_retry.value as boolean;
try {
await fs.mkdir(junitReportDirectory, { recursive: true });
@@ -256,7 +263,7 @@ export function createMaestroTestsBuildFunction(): BuildFunction {
break;
}
if (outputFormat === 'junit' && outputPath && nameToPath) {
if (smartRetry && outputFormat === 'junit' && outputPath && nameToPath) {
const failed = await parseFailedFlowsFromJUnit({
junitFile: outputPath,
nameToPath,
@@ -1,5 +1,10 @@
import { bunyan } from '@expo/logger';
import { BuildFunction, BuildStepInput, BuildStepInputValueTypeName } from '@expo/steps';
import {
BuildFunction,
BuildRuntimePlatform,
BuildStepInput,
BuildStepInputValueTypeName,
} from '@expo/steps';
import spawn from '@expo/turtle-spawn';
import childProcess from 'node:child_process';
import fs from 'node:fs';
@@ -15,6 +20,7 @@ const DAEMON_LOG = path.join(RUN_DIR, 'daemon.log');
const TUNNEL_LOG = path.join(RUN_DIR, 'cloudflared.log');
const DAEMON_JSON_PATH = path.join(os.homedir(), '.agent-device', 'daemon.json');
const XCODE_DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer';
const CLOUDFLARED_LINUX_INSTALL_PATH = '/usr/local/bin/cloudflared';
const STARTUP_TIMEOUT_MS = 60_000;
export function createStartAgentDeviceRemoteSessionBuildFunction(): BuildFunction {
@@ -30,21 +36,27 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(): BuildFunctio
allowedValueTypeName: BuildStepInputValueTypeName.STRING,
}),
],
fn: async ({ logger }, { inputs, env }) => {
fn: async ({ logger, global }, { inputs, env }) => {
const packageVersion = inputs.package_version.value as string | undefined;
logger.info(`Starting agent-device remote session (version: ${packageVersion ?? 'latest'}).`);
const { runtimePlatform } = global;
logger.info(
`Starting agent-device remote session (version: ${packageVersion ?? 'latest'}, runtime: ${runtimePlatform}).`
);
logger.info(`Preparing runtime directory at ${RUN_DIR}.`);
await fs.promises.mkdir(RUN_DIR, { recursive: true });
logger.info(`Selecting Xcode developer directory: ${XCODE_DEVELOPER_DIR}.`);
await spawn('sudo', ['xcode-select', '-s', XCODE_DEVELOPER_DIR], { env, logger });
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
logger.info(`Selecting Xcode developer directory: ${XCODE_DEVELOPER_DIR}.`);
await spawn('sudo', ['xcode-select', '-s', XCODE_DEVELOPER_DIR], { env, logger });
}
logger.info('Ensuring cloudflared is installed.');
await ensureCloudflaredInstalledAsync({ env, logger });
logger.info('Ensuring bun is installed.');
await ensureBunInstalledAsync({ env, logger });
const cloudflaredCommand = await ensureCloudflaredInstalledAsync({
runtimePlatform,
env,
logger,
});
logger.info(
packageVersion
@@ -82,7 +94,7 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(): BuildFunctio
`Starting cloudflared tunnel to http://localhost:${daemonPort} (log file: ${TUNNEL_LOG}).`
);
await spawnDetachedAsync({
command: `cloudflared tunnel --url "http://localhost:${daemonPort}"`,
command: `${cloudflaredCommand} tunnel --url "http://localhost:${daemonPort}"`,
logFile: TUNNEL_LOG,
env,
logger,
@@ -110,23 +122,44 @@ export function createStartAgentDeviceRemoteSessionBuildFunction(): BuildFunctio
}
async function ensureCloudflaredInstalledAsync({
runtimePlatform,
env,
logger,
}: {
runtimePlatform: BuildRuntimePlatform;
env: NodeJS.ProcessEnv;
logger: bunyan;
}): Promise<void> {
await ensureBrewPackageInstalledAsync({ name: 'cloudflared', env, logger });
}): Promise<string> {
if (runtimePlatform === BuildRuntimePlatform.DARWIN) {
await ensureBrewPackageInstalledAsync({ name: 'cloudflared', env, logger });
return 'cloudflared';
}
if (await isCommandAvailableAsync({ command: 'cloudflared', env })) {
return 'cloudflared';
}
const cloudflaredArch = cloudflaredLinuxArchForNodeArch(os.arch());
const downloadUrl = `https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-${cloudflaredArch}`;
logger.info(`Downloading cloudflared from ${downloadUrl} to ${CLOUDFLARED_LINUX_INSTALL_PATH}.`);
await spawn('sudo', ['curl', '-fsSL', '-o', CLOUDFLARED_LINUX_INSTALL_PATH, downloadUrl], {
env,
logger,
});
await spawn('sudo', ['chmod', '+x', CLOUDFLARED_LINUX_INSTALL_PATH], { env, logger });
// Return the absolute install path so the tunnel command works even when
// /usr/local/bin is not on the step's PATH.
return CLOUDFLARED_LINUX_INSTALL_PATH;
}
async function ensureBunInstalledAsync({
env,
logger,
}: {
env: NodeJS.ProcessEnv;
logger: bunyan;
}): Promise<void> {
await ensureBrewPackageInstalledAsync({ name: 'bun', env, logger });
function cloudflaredLinuxArchForNodeArch(arch: string): 'amd64' | 'arm64' {
if (arch === 'x64') {
return 'amd64';
}
if (arch === 'arm64') {
return 'arm64';
}
throw new Error(
`Unsupported architecture for cloudflared on Linux: "${arch}". Expected "x64" or "arm64".`
);
}
async function ensureBrewPackageInstalledAsync({
@@ -145,6 +178,21 @@ async function ensureBrewPackageInstalledAsync({
);
}
async function isCommandAvailableAsync({
command,
env,
}: {
command: string;
env: NodeJS.ProcessEnv;
}): Promise<boolean> {
try {
await spawn('bash', ['-c', `command -v ${command}`], { env, ignoreStdio: true });
return true;
} catch {
return false;
}
}
async function cloneAgentDeviceAsync({
packageVersion,
env,
@@ -146,6 +146,44 @@ const FIXTURE_NESTED = {
],
};
// Overlapping + gapped intervals with pairwise-distinct totals
// (taskSeconds=11, wallSpan=12, activeWallTime=8). Distinct values catch a
// regression that leaks wallSpan or taskSeconds into the rendered Active cell.
const FIXTURE_OVERLAP_WITH_GAP = {
schema: { name: 'OverlapWithGap' },
subSteps: [
{
title: 'Build target GappedModule',
subSteps: [
{
detailStepType: 'cCompilation',
duration: 5.0,
startTimestamp: 100.0,
endTimestamp: 105.0,
signature: 'CompileC a.c',
subSteps: [],
},
{
detailStepType: 'cCompilation',
duration: 4.0,
startTimestamp: 102.0,
endTimestamp: 106.0,
signature: 'CompileC b.c',
subSteps: [],
},
{
detailStepType: 'cCompilation',
duration: 2.0,
startTimestamp: 110.0,
endTimestamp: 112.0,
signature: 'CompileC c.c',
subSteps: [],
},
],
},
],
};
const mockedDownloadFile = jest.mocked(downloadFile);
const mockedSpawn = jest.mocked(spawn);
@@ -191,14 +229,14 @@ describe('parseAndReportXcactivitylog', () => {
});
expect(mockedDownloadFile).toHaveBeenCalledWith(
'https://storage.googleapis.com/turtle-v2/xclogparser/XCLogParser-macOS-x86-64-arm64-v0.2.46.zip',
path.join(tempDir, 'XCLogParser-macOS-x86-64-arm64-v0.2.46.zip'),
'https://storage.googleapis.com/turtle-v2/xclogparser/XCLogParser-macOS-x86-64-arm64-v0.2.47.zip',
path.join(tempDir, 'XCLogParser-macOS-x86-64-arm64-v0.2.47.zip'),
{ retry: 3, timeout: 20_000 }
);
expect(mockedSpawn).toHaveBeenNthCalledWith(
1,
'unzip',
['-q', path.join(tempDir, 'XCLogParser-macOS-x86-64-arm64-v0.2.46.zip'), '-d', tempDir],
['-q', path.join(tempDir, 'XCLogParser-macOS-x86-64-arm64-v0.2.47.zip'), '-d', tempDir],
{ stdio: 'pipe' }
);
expect(mockedSpawn).toHaveBeenNthCalledWith(
@@ -244,14 +282,14 @@ describe('parseAndReportXcactivitylog', () => {
expect(mockedDownloadFile).toHaveBeenNthCalledWith(
1,
'https://cache.example.com/storage.googleapis.com/turtle-v2/xclogparser/XCLogParser-macOS-x86-64-arm64-v0.2.46.zip',
'/tmp/xclogparser-123/XCLogParser-macOS-x86-64-arm64-v0.2.46.zip',
'https://cache.example.com/storage.googleapis.com/turtle-v2/xclogparser/XCLogParser-macOS-x86-64-arm64-v0.2.47.zip',
'/tmp/xclogparser-123/XCLogParser-macOS-x86-64-arm64-v0.2.47.zip',
{ retry: 3, timeout: 20_000 }
);
expect(mockedDownloadFile).toHaveBeenNthCalledWith(
2,
'https://storage.googleapis.com/turtle-v2/xclogparser/XCLogParser-macOS-x86-64-arm64-v0.2.46.zip',
'/tmp/xclogparser-123/XCLogParser-macOS-x86-64-arm64-v0.2.46.zip',
'https://storage.googleapis.com/turtle-v2/xclogparser/XCLogParser-macOS-x86-64-arm64-v0.2.47.zip',
'/tmp/xclogparser-123/XCLogParser-macOS-x86-64-arm64-v0.2.47.zip',
{ retry: 3, timeout: 20_000 }
);
expect(logger.debug).toHaveBeenCalledWith(
@@ -280,14 +318,14 @@ describe('parseAndReportXcactivitylog', () => {
expect(mockedDownloadFile).toHaveBeenNthCalledWith(
1,
'https://cache.example.com/storage.googleapis.com/turtle-v2/xclogparser/XCLogParser-macOS-x86-64-arm64-v0.2.46.zip',
'/tmp/xclogparser-123/XCLogParser-macOS-x86-64-arm64-v0.2.46.zip',
'https://cache.example.com/storage.googleapis.com/turtle-v2/xclogparser/XCLogParser-macOS-x86-64-arm64-v0.2.47.zip',
'/tmp/xclogparser-123/XCLogParser-macOS-x86-64-arm64-v0.2.47.zip',
{ retry: 3, timeout: 20_000 }
);
expect(mockedDownloadFile).toHaveBeenNthCalledWith(
2,
'https://storage.googleapis.com/turtle-v2/xclogparser/XCLogParser-macOS-x86-64-arm64-v0.2.46.zip',
'/tmp/xclogparser-123/XCLogParser-macOS-x86-64-arm64-v0.2.46.zip',
'https://storage.googleapis.com/turtle-v2/xclogparser/XCLogParser-macOS-x86-64-arm64-v0.2.47.zip',
'/tmp/xclogparser-123/XCLogParser-macOS-x86-64-arm64-v0.2.47.zip',
{ retry: 3, timeout: 20_000 }
);
expect(logger.debug).toHaveBeenCalledWith(
@@ -478,17 +516,38 @@ describe('buildTargetMetrics', () => {
expect(mod).toBeDefined();
expect(mod!.taskSeconds).toBeCloseTo(3.0);
expect(mod!.wallSpan).toBe(0); // no valid intervals
expect(mod!.activeWallTime).toBe(0); // no valid intervals
});
it('computes activeWallTime by merging overlapping intervals and excluding gaps', () => {
const { results } = buildTargetMetrics(FIXTURE_OVERLAP_WITH_GAP);
const mod = results.find(r => r.moduleName === 'GappedModule');
expect(mod).toBeDefined();
expect(mod!.taskSeconds).toBeCloseTo(11.0); // 5 + 4 + 2
expect(mod!.wallSpan).toBeCloseTo(12.0); // 112 - 100
expect(mod!.activeWallTime).toBeCloseTo(8.0); // (106-100) + (112-110)
});
});
describe('formatReport', () => {
it('produces table with header and module rows', () => {
it('produces table with header, legend, and module rows', () => {
const report = formatReport(FIXTURE_TWO_MODULES);
expect(report).toContain('Xcode Build — Compile Metrics by Module');
expect(report).toContain('ModuleA');
expect(report).toContain('ModuleB');
expect(report).toContain('TOTAL');
expect(report).toContain('% Task');
expect(report).toContain('% Sum');
expect(report).toContain('Sum = sum of compile-step wall durations');
expect(report).toContain('Active = merged compile-step wall time');
expect(report).not.toContain('% Task');
expect(report).not.toContain('Wall = ');
});
it('renders Active using activeWallTime, not wallSpan', () => {
const report = formatReport(FIXTURE_OVERLAP_WITH_GAP);
expect(report).toContain('11.0s');
expect(report).toContain('8.0s');
expect(report).not.toContain('12.0s');
});
it('does not include modules below threshold', () => {
@@ -7,7 +7,7 @@ import os from 'os';
import path from 'path';
import { z } from 'zod';
const DEFAULT_XCLOGPARSER_VERSION = 'v0.2.46';
const DEFAULT_XCLOGPARSER_VERSION = 'v0.2.47';
const XCLOGPARSER_DOWNLOAD_URL = 'https://storage.googleapis.com/turtle-v2/xclogparser';
const XCLOGPARSER_DOWNLOAD_TIMEOUT_MS = 20_000;
const XCLOGPARSER_OUTPUT_FILENAME = 'xcactivitylog.json';
@@ -183,6 +183,7 @@ type XcactivitylogData = z.infer<typeof XcactivitylogDataSchemaZ>;
interface TargetMetric {
moduleName: string;
taskSeconds: number;
// Not rendered; retained as secondary sort tie-breaker in buildTargetMetrics.
wallSpan: number;
activeWallTime: number;
}
@@ -358,19 +359,22 @@ export function formatReport(data: XcactivitylogData): string {
lines.push(
`Schema: ${typeof data.schema === 'string' ? data.schema : (data.schema?.name ?? 'unknown')}`
);
lines.push('% Task = share of total Compile Task Seconds');
lines.push('Wall = first compile start to last compile end');
lines.push(
'Sum = sum of compile-step wall durations within the target; overlapping steps are counted separately'
);
lines.push('% Sum = share of total Sum');
lines.push('Active = merged compile-step wall time, excluding idle gaps between compile steps');
lines.push('');
lines.push(header);
lines.push(
'│ ' +
'Module'.padEnd(nameWidth) +
' │ ' +
'Task'.padStart(taskWidth) +
'Sum'.padStart(taskWidth) +
' │ ' +
'% Task'.padStart(pctWidth) +
'% Sum'.padStart(pctWidth) +
' │ ' +
'Wall'.padStart(wallWidth) +
'Active'.padStart(wallWidth) +
' │ ' +
' '.repeat(barMaxWidth) +
' │'
@@ -390,7 +394,7 @@ export function formatReport(data: XcactivitylogData): string {
' │ ' +
`${pct.toFixed(1)}%`.padStart(pctWidth) +
' │ ' +
formatSeconds(result.wallSpan).padStart(wallWidth) +
formatSeconds(result.activeWallTime).padStart(wallWidth) +
' │ ' +
bar +
' │'
@@ -266,6 +266,9 @@ describe('Android.JobSchema', () => {
customBuildConfig: {
path: 'production.android.yml',
},
outputs: {
android_build_type: 'apk',
},
initiatingUserId: randomUUID(),
appId: randomUUID(),
};
@@ -340,6 +343,28 @@ describe('Android.JobSchema', () => {
expect(error).toBeFalsy();
});
test('preserves outputs for build jobs', () => {
const job = {
mode: BuildMode.BUILD,
type: Workflow.GENERIC,
platform: Platform.ANDROID,
projectArchive: {
type: ArchiveSourceType.URL,
url: 'https://expo.dev/builds/123',
},
projectRootDirectory: '.',
outputs: {
android_build_type: 'apk',
},
secrets,
initiatingUserId: randomUUID(),
appId: randomUUID(),
};
const { value, error } = Android.JobSchema.validate(job, joiOptions);
expect(value).toMatchObject(job);
expect(error).toBeFalsy();
});
test('can set hooks with arbitrary names', () => {
const job = {
mode: BuildMode.BUILD,
@@ -77,6 +77,9 @@ describe('Ios.JobSchema', () => {
},
initiatingUserId: randomUUID(),
appId: randomUUID(),
outputs: {
resign_job: 'true',
},
};
const { value, error } = Ios.JobSchema.validate(genericJob, joiOptions);
@@ -180,6 +183,9 @@ describe('Ios.JobSchema', () => {
customBuildConfig: {
path: 'production.ios.yml',
},
outputs: {
ios_build_type: 'simulator',
},
initiatingUserId: randomUUID(),
appId: randomUUID(),
};
@@ -254,6 +260,30 @@ describe('Ios.JobSchema', () => {
expect(error).toBeFalsy();
});
test('preserves outputs for build jobs', () => {
const job = {
mode: BuildMode.BUILD,
type: Workflow.UNKNOWN,
platform: Platform.IOS,
projectArchive: {
type: ArchiveSourceType.URL,
url: 'https://expo.dev/builds/123',
},
projectRootDirectory: '.',
outputs: {
ios_build_type: 'simulator',
},
secrets: {
buildCredentials,
},
initiatingUserId: randomUUID(),
appId: randomUUID(),
};
const { value, error } = Ios.JobSchema.validate(job, joiOptions);
expect(value).toMatchObject(job);
expect(error).toBeFalsy();
});
test('invalid generic job', () => {
const genericJob = {
secrets: {
+4 -1
View File
@@ -15,6 +15,8 @@ import {
EnvironmentSecretsSchema,
Hooks,
HooksSchema,
JobOutputs,
JobOutputsSchema,
Platform,
StaticWorkflowInterpolationContext,
StaticWorkflowInterpolationContextZ,
@@ -110,7 +112,7 @@ export interface Job {
};
hooks?: Hooks;
steps?: Step[];
outputs?: Record<string, string>;
outputs?: JobOutputs;
experimental?: {
prebuildCommand?: string;
@@ -176,6 +178,7 @@ export const JobSchema = Joi.object({
buildType: Joi.string().valid(...Object.values(BuildType)),
username: Joi.string(),
hooks: HooksSchema,
outputs: JobOutputsSchema,
experimental: Joi.object({
prebuildCommand: Joi.string(),
+4 -3
View File
@@ -352,6 +352,10 @@ export type DynamicInterpolationContext = {
export type WorkflowInterpolationContext = StaticWorkflowInterpolationContext &
DynamicInterpolationContext;
export type JobOutputs = Record<string, string>;
export const JobOutputsSchema = Joi.object<JobOutputs>().pattern(Joi.string(), Joi.string());
export const JobOutputsSchemaZ = z.record(z.string(), z.string());
export const CustomBuildConfigSchema = Joi.object().when('.mode', {
is: [BuildMode.CUSTOM, BuildMode.REPACK],
then: Joi.object().when('.customBuildConfig.path', {
@@ -361,7 +365,6 @@ export const CustomBuildConfigSchema = Joi.object().when('.mode', {
path: Joi.string().required(),
}).required(),
steps: Joi.any().strip(),
outputs: Joi.any().strip(),
}),
otherwise: Joi.object({
customBuildConfig: Joi.any().strip(),
@@ -369,13 +372,11 @@ export const CustomBuildConfigSchema = Joi.object().when('.mode', {
.items(Joi.any())
.required()
.custom(steps => validateSteps(steps), 'steps validation'),
outputs: Joi.object().pattern(Joi.string(), Joi.string()).required(),
}),
}),
otherwise: Joi.object({
customBuildConfig: Joi.any().strip(),
steps: Joi.any().strip(),
outputs: Joi.any().strip(),
}),
});
+2 -1
View File
@@ -6,6 +6,7 @@ import {
BuildTrigger,
EnvironmentSecretZ,
HooksZ,
JobOutputsSchemaZ,
StaticWorkflowInterpolationContextZ,
} from './common';
import { StepZ } from './step';
@@ -48,7 +49,7 @@ export namespace Generic {
hooks: HooksZ.optional(),
steps: z.array(StepZ).min(1),
outputs: z.record(z.string(), z.string()).optional(),
outputs: JobOutputsSchemaZ.optional(),
});
export type PartialJob = z.infer<typeof PartialJobZ>;
+4 -1
View File
@@ -15,6 +15,8 @@ import {
EnvironmentSecretsSchema,
Hooks,
HooksSchema,
JobOutputs,
JobOutputsSchema,
Platform,
StaticWorkflowInterpolationContext,
StaticWorkflowInterpolationContextZ,
@@ -124,7 +126,7 @@ export interface Job {
};
hooks?: Hooks;
steps?: Step[];
outputs?: Record<string, string>;
outputs?: JobOutputs;
experimental?: {
prebuildCommand?: string;
@@ -210,6 +212,7 @@ export const JobSchema = Joi.object({
username: Joi.string(),
hooks: HooksSchema,
outputs: JobOutputsSchema,
experimental: Joi.object({
prebuildCommand: Joi.string(),
@@ -2,10 +2,18 @@ import spawnAsync from '@expo/spawn-async';
import * as fs from 'fs-extra';
import { getMockOclifConfig } from '../../../../__tests__/commands/utils';
import { DefaultEnvironment } from '../../../../build/utils/environment';
import { ExpoGraphqlClient } from '../../../../commandUtils/context/contextUtils/createGraphqlClient';
import { testProjectId } from '../../../../credentials/__tests__/fixtures-constants';
import {
EnvironmentSecretType,
EnvironmentVariableScope,
EnvironmentVariableVisibility,
} from '../../../../graphql/generated';
import { ConvexMutation } from '../../../../graphql/mutations/ConvexMutation';
import { EnvironmentVariableMutation } from '../../../../graphql/mutations/EnvironmentVariableMutation';
import { ConvexQuery } from '../../../../graphql/queries/ConvexQuery';
import { EnvironmentVariablesQuery } from '../../../../graphql/queries/EnvironmentVariablesQuery';
import {
ConvexTeamConnectionData,
SetupConvexProjectResultData,
@@ -17,7 +25,9 @@ import { Actor } from '../../../../user/User';
import IntegrationsConvexConnect from '../connect';
jest.mock('../../../../graphql/queries/ConvexQuery');
jest.mock('../../../../graphql/queries/EnvironmentVariablesQuery');
jest.mock('../../../../graphql/mutations/ConvexMutation');
jest.mock('../../../../graphql/mutations/EnvironmentVariableMutation');
jest.mock('../../../../project/projectUtils');
jest.mock('../../../../prompts');
jest.mock('@expo/spawn-async');
@@ -132,6 +142,37 @@ describe(IntegrationsConvexConnect, () => {
jest.mocked(ConvexMutation.createConvexTeamConnectionAsync).mockResolvedValue(mockConnection);
jest.mocked(ConvexMutation.setupConvexProjectAsync).mockResolvedValue(mockSetupResult);
jest.mocked(ConvexMutation.sendConvexTeamInviteToVerifiedEmailAsync).mockResolvedValue(true);
jest.mocked(EnvironmentVariablesQuery.byAppIdAsync).mockResolvedValue([]);
jest.mocked(EnvironmentVariableMutation.createForAppAsync).mockResolvedValue({
id: 'env-var-1',
name: 'EXPO_PUBLIC_CONVEX_URL',
value: mockSetupResult.convexDeploymentUrl,
environments: [
DefaultEnvironment.Production,
DefaultEnvironment.Preview,
DefaultEnvironment.Development,
],
createdAt: '2024-01-01T00:00:00.000Z',
updatedAt: '2024-01-01T00:00:00.000Z',
scope: EnvironmentVariableScope.Project,
visibility: EnvironmentVariableVisibility.Public,
type: EnvironmentSecretType.String,
});
jest.mocked(EnvironmentVariableMutation.updateAsync).mockResolvedValue({
id: 'env-var-1',
name: 'EXPO_PUBLIC_CONVEX_URL',
value: mockSetupResult.convexDeploymentUrl,
environments: [
DefaultEnvironment.Production,
DefaultEnvironment.Preview,
DefaultEnvironment.Development,
],
createdAt: '2024-01-01T00:00:00.000Z',
updatedAt: '2024-01-01T00:00:00.000Z',
scope: EnvironmentVariableScope.Project,
visibility: EnvironmentVariableVisibility.Public,
type: EnvironmentSecretType.String,
});
});
it('creates a Convex team and project with the new mutation shape', async () => {
@@ -287,6 +328,108 @@ describe(IntegrationsConvexConnect, () => {
expect(writeCall?.[1]).toContain('EXPO_PUBLIC_CONVEX_URL=https://happy-otter-123.convex.cloud');
});
it('creates the Convex URL as a project EAS environment variable for all default environments', async () => {
jest
.mocked(ConvexQuery.getConvexTeamConnectionsByAccountIdAsync)
.mockResolvedValue([mockConnection]);
await createCommand([]).runAsync();
expect(EnvironmentVariablesQuery.byAppIdAsync).toHaveBeenCalledWith(graphqlClient, {
appId: testProjectId,
filterNames: ['EXPO_PUBLIC_CONVEX_URL'],
});
expect(EnvironmentVariableMutation.createForAppAsync).toHaveBeenCalledWith(
graphqlClient,
{
name: 'EXPO_PUBLIC_CONVEX_URL',
value: 'https://happy-otter-123.convex.cloud',
environments: [
DefaultEnvironment.Production,
DefaultEnvironment.Preview,
DefaultEnvironment.Development,
],
visibility: EnvironmentVariableVisibility.Public,
type: EnvironmentSecretType.String,
},
testProjectId
);
expect(EnvironmentVariableMutation.updateAsync).not.toHaveBeenCalled();
});
it('updates an existing project EAS environment variable for the Convex URL', async () => {
jest
.mocked(ConvexQuery.getConvexTeamConnectionsByAccountIdAsync)
.mockResolvedValue([mockConnection]);
jest.mocked(EnvironmentVariablesQuery.byAppIdAsync).mockResolvedValue([
{
id: 'env-var-1',
name: 'EXPO_PUBLIC_CONVEX_URL',
value: 'https://old-deployment.convex.cloud',
environments: [DefaultEnvironment.Production],
createdAt: '2024-01-01T00:00:00.000Z',
updatedAt: '2024-01-01T00:00:00.000Z',
scope: EnvironmentVariableScope.Project,
visibility: EnvironmentVariableVisibility.Public,
type: EnvironmentSecretType.String,
},
]);
await createCommand([]).runAsync();
expect(confirmAsync).toHaveBeenCalledWith({
message: expect.stringContaining(
'EAS already has an EXPO_PUBLIC_CONVEX_URL environment variable'
),
});
expect(EnvironmentVariableMutation.updateAsync).toHaveBeenCalledWith(graphqlClient, {
id: 'env-var-1',
name: 'EXPO_PUBLIC_CONVEX_URL',
value: 'https://happy-otter-123.convex.cloud',
environments: [
DefaultEnvironment.Production,
DefaultEnvironment.Preview,
DefaultEnvironment.Development,
],
visibility: EnvironmentVariableVisibility.Public,
type: EnvironmentSecretType.String,
});
expect(EnvironmentVariableMutation.createForAppAsync).not.toHaveBeenCalled();
});
it('skips updating an existing project EAS environment variable for the Convex URL when declined', async () => {
jest
.mocked(ConvexQuery.getConvexTeamConnectionsByAccountIdAsync)
.mockResolvedValue([mockConnection]);
jest.mocked(EnvironmentVariablesQuery.byAppIdAsync).mockResolvedValue([
{
id: 'env-var-1',
name: 'EXPO_PUBLIC_CONVEX_URL',
value: 'https://old-deployment.convex.cloud',
environments: [DefaultEnvironment.Production],
createdAt: '2024-01-01T00:00:00.000Z',
updatedAt: '2024-01-01T00:00:00.000Z',
scope: EnvironmentVariableScope.Project,
visibility: EnvironmentVariableVisibility.Public,
type: EnvironmentSecretType.String,
},
]);
jest.mocked(confirmAsync).mockResolvedValue(false);
await createCommand([]).runAsync();
expect(confirmAsync).toHaveBeenCalledWith({
message: expect.stringContaining(
'EAS already has an EXPO_PUBLIC_CONVEX_URL environment variable'
),
});
expect(EnvironmentVariableMutation.updateAsync).not.toHaveBeenCalled();
expect(EnvironmentVariableMutation.createForAppAsync).not.toHaveBeenCalled();
expect(Log.warn).toHaveBeenCalledWith(
expect.stringContaining('Skipped updating EAS environment variable')
);
});
it('prompts before overwriting existing Convex .env.local values', async () => {
jest
.mocked(ConvexQuery.getConvexTeamConnectionsByAccountIdAsync)
@@ -5,11 +5,20 @@ import dotenv from 'dotenv';
import * as fs from 'fs-extra';
import path from 'path';
import { DefaultEnvironment } from '../../../build/utils/environment';
import EasCommand from '../../../commandUtils/EasCommand';
import { ExpoGraphqlClient } from '../../../commandUtils/context/contextUtils/createGraphqlClient';
import { confirmRecentConvexInviteAsync, formatConvexTeam } from '../../../commandUtils/convex';
import { EASNonInteractiveFlag } from '../../../commandUtils/flags';
import {
EnvironmentSecretType,
EnvironmentVariableScope,
EnvironmentVariableVisibility,
} from '../../../graphql/generated';
import { ConvexMutation } from '../../../graphql/mutations/ConvexMutation';
import { EnvironmentVariableMutation } from '../../../graphql/mutations/EnvironmentVariableMutation';
import { ConvexQuery } from '../../../graphql/queries/ConvexQuery';
import { EnvironmentVariablesQuery } from '../../../graphql/queries/EnvironmentVariablesQuery';
import { ConvexTeamConnectionData } from '../../../graphql/types/ConvexTeamConnection';
import Log from '../../../log';
import { ora } from '../../../ora';
@@ -23,6 +32,12 @@ const CONVEX_REGIONS = [
];
const DEFAULT_REGION = 'aws-us-east-1';
const EAS_CONVEX_ENV_VAR_NAME = 'EXPO_PUBLIC_CONVEX_URL';
const EAS_CONVEX_ENVIRONMENTS = [
DefaultEnvironment.Production,
DefaultEnvironment.Preview,
DefaultEnvironment.Development,
];
type TeamInviteResult = 'sent' | 'skipped' | 'failed';
@@ -132,12 +147,20 @@ export default class IntegrationsConvexConnect extends EasCommand {
throw error;
}
// 6. Send team invite (non-fatal)
// 6. Save the Convex URL as an EAS environment variable for builds
await this.upsertConvexUrlEasEnvVarAsync(
graphqlClient,
projectId,
setupResult.convexDeploymentUrl,
nonInteractive
);
// 7. Send team invite (non-fatal)
const teamInviteResult = await this.sendTeamInviteAsync(graphqlClient, connection, actor, {
nonInteractive,
});
// 7. Write deploy key and URL to .env.local
// 8. Write deploy key and URL to .env.local
await this.writeEnvLocalAsync(
projectDir,
setupResult.deployKey,
@@ -145,7 +168,7 @@ export default class IntegrationsConvexConnect extends EasCommand {
nonInteractive
);
// 8. Success message
// 9. Success message
Log.addNewLineIfNone();
Log.log(chalk.green('Convex is ready!'));
Log.newLine();
@@ -163,6 +186,63 @@ export default class IntegrationsConvexConnect extends EasCommand {
}
}
private async upsertConvexUrlEasEnvVarAsync(
graphqlClient: ExpoGraphqlClient,
projectId: string,
convexUrl: string,
nonInteractive: boolean
): Promise<void> {
const existingVariables = await EnvironmentVariablesQuery.byAppIdAsync(graphqlClient, {
appId: projectId,
filterNames: [EAS_CONVEX_ENV_VAR_NAME],
});
const existingProjectVariable = existingVariables.find(
variable => variable.scope === EnvironmentVariableScope.Project
);
if (existingProjectVariable) {
if (!nonInteractive) {
const overwrite = await confirmAsync({
message: `EAS already has an ${EAS_CONVEX_ENV_VAR_NAME} environment variable for this project. Overwrite it?`,
});
if (!overwrite) {
Log.warn(
`Skipped updating EAS environment variable ${chalk.bold(EAS_CONVEX_ENV_VAR_NAME)}.`
);
return;
}
}
await EnvironmentVariableMutation.updateAsync(graphqlClient, {
id: existingProjectVariable.id,
name: EAS_CONVEX_ENV_VAR_NAME,
value: convexUrl,
environments: EAS_CONVEX_ENVIRONMENTS,
visibility: EnvironmentVariableVisibility.Public,
type: EnvironmentSecretType.String,
});
Log.withTick(
`Updated EAS environment variable ${chalk.bold(EAS_CONVEX_ENV_VAR_NAME)} for builds`
);
return;
}
await EnvironmentVariableMutation.createForAppAsync(
graphqlClient,
{
name: EAS_CONVEX_ENV_VAR_NAME,
value: convexUrl,
environments: EAS_CONVEX_ENVIRONMENTS,
visibility: EnvironmentVariableVisibility.Public,
type: EnvironmentSecretType.String,
},
projectId
);
Log.withTick(
`Created EAS environment variable ${chalk.bold(EAS_CONVEX_ENV_VAR_NAME)} for builds`
);
}
private async resolveRegionAsync(
flagValue: string | undefined,
nonInteractive: boolean
+2 -12
View File
@@ -4,7 +4,7 @@ import { LoggerLevel, bunyan, createLogger } from '@expo/logger';
import { Readable, Transform, TransformCallback, Writable } from 'stream';
import config from './config';
import { maybeStringBase64Decode, simpleSecretsWhitelist } from './secrets';
import { redactSecrets } from './secrets';
import { uuidv7 } from 'uuidv7';
import HttpLogStream from './utils/HttpLogStream';
@@ -58,12 +58,6 @@ export class WorkerLogBuffer extends Writable implements LogBuffer {
}
function createTransformStream(secrets: EnvironmentSecret[]): Transform {
const secretValues = secrets.map(({ value }) => value);
const secretList: string[] = [
...secretValues,
...secretValues.map(maybeStringBase64Decode).filter((i): i is string => !!i),
].filter(i => i.length > 1 && !simpleSecretsWhitelist.includes(i));
return new Transform({
readableObjectMode: true,
writableObjectMode: true,
@@ -80,13 +74,9 @@ function createTransformStream(secrets: EnvironmentSecret[]): Transform {
}
if (chunk && typeof chunk === 'object' && chunk.msg) {
const msgWithoutSecrets = secretList.reduce((acc: string, pattern: string): string => {
return acc.replaceAll(pattern, '*'.repeat(pattern.length));
}, chunk.msg as string);
callback(null, {
...chunk,
msg: msgWithoutSecrets,
msg: redactSecrets(chunk.msg as string, secrets),
});
} else {
callback(null, chunk);
+22
View File
@@ -1,3 +1,5 @@
import { EnvironmentSecret } from '@expo/eas-build-job';
export function maybeStringBase64Decode(maybeEncodedString: string): string | null {
try {
const buffer = Buffer.from(maybeEncodedString, 'base64');
@@ -10,6 +12,26 @@ export function maybeStringBase64Decode(maybeEncodedString: string): string | nu
}
}
export function redactSecrets(
message: string,
secrets: EnvironmentSecret[],
additionalSecrets?: string[]
): string {
const secretValues = [
...secrets.map(({ value }) => value),
...(additionalSecrets ?? []),
];
const secretList = [
...secretValues,
...secretValues.map(maybeStringBase64Decode).filter((i): i is string => !!i),
].filter((i) => i.length > 1 && !simpleSecretsWhitelist.includes(i));
return secretList.reduce(
(acc, pattern) => acc.replaceAll(pattern, '*'.repeat(pattern.length)),
message
);
}
export const simpleSecretsWhitelist = [
'production',
'prod',
+26 -1
View File
@@ -31,6 +31,7 @@ import { createBuildContext } from './context';
import { Analytics } from './external/analytics';
import { LauncherMessage, Worker, WorkerMessage } from './external/turtle';
import logger, { createBuildLoggerWithSecretsFilter } from './logger';
import { redactSecrets } from './secrets';
import sentry from './sentry';
import State from './state';
import { WebSocketServer } from './utils/WebSocketServer';
@@ -353,13 +354,37 @@ export default class BuildService {
}
try {
const additionalSecrets: string[] = [];
if (robotAccessToken) {
additionalSecrets.push(robotAccessToken);
}
const secrets = job.secrets as any;
const keystore = secrets?.buildCredentials?.keystore;
additionalSecrets.push(
keystore?.dataBase64,
keystore?.keystorePassword,
keystore?.keyPassword,
);
for (const targetCreds of Object.values(secrets?.buildCredentials ?? {})) {
const creds = targetCreds as any;
additionalSecrets.push(
creds?.provisioningProfileBase64,
creds?.distributionCertificate?.dataBase64,
creds?.distributionCertificate?.password,
);
}
const redactedErrorMessage = redactSecrets(
rawErrorMessage,
job.secrets?.environmentSecrets ?? [],
additionalSecrets.filter((s): s is string => typeof s === 'string')
);
await turtleFetch(
new URL('turtle-builds/logs', config.wwwApiV2BaseUrl).toString(),
'POST',
{
json: {
buildId: this.buildId,
message: rawErrorMessage,
message: redactedErrorMessage,
level: 'error',
tags: {
build_phase: err.buildPhase ?? null,