mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-07 06:16:06 -10:00
fix(codex): gate models on the Codex CLI floor before the ChatGPT backend rejects them (#15399)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Codex adapter and the native runner run Codex in local hosts and in managed sandboxes, and the model catalog lists the models an operator can select > - The Codex backend accepts a new model with ChatGPT sign-in only from a recent enough Codex CLI. An older CLI fails every turn with `The '<model>' model is not supported when using Codex with a ChatGPT account.` > - After #14942 added `gpt-6.1-sol`, an operator selected it for an agent in a managed Daytona sandbox. The sandbox image still shipped Codex 0.156.0. The connection test and runs failed with that sentence, which reads like an account problem > - Paperclip only checked the Codex compatibility window (`>=0.149.0 <0.161.0`), so 0.156.0 passed and the failure surfaced from the backend without a cause > - This pull request records the verified Codex CLI floor for each gated model, compares the installed `codex --version` with that floor in the environment Test and in the remote runner, and names the backend rejection when it still happens > - The benefit is a precise, actionable message ("gpt-6.1-sol requires Codex CLI 0.159.0 or newer; detected 0.156.0; promote a sandbox image with Codex 0.160.0") instead of an opaque 400, and no doomed hello probe ## Linked Issues or Issue Description Refs #14942 **Bug: selecting `gpt-6.1-sol` in a managed sandbox fails with a ChatGPT account error.** **Steps to reproduce** 1. Promote a sandbox image that ships Codex CLI 0.156.0. 2. Create a Codex agent with a ChatGPT sign-in connection, select `gpt-6.1-sol`, and run the environment Test or a task in that sandbox. **Expected behavior** The Test or the run tells the operator that the Codex CLI in the sandbox is older than the model needs. **Actual behavior** The Test reports `codex_hello_probe_failed` with `{"type":"error","status":400,"error":{"type":"invalid_request_error","message":"The 'gpt-6.1-sol' model is not supported when using Codex with a ChatGPT account."}}`. A run fails with "The selected model is not supported by the current ChatGPT connection." **Evidence** - Codex 0.156.1 and older are rejected for `gpt-6.1-sol` with ChatGPT sign-in: https://github.com/openai/codex/issues/49396 - Codex 0.159.0 and 0.159.2 are accepted: https://github.com/openai/codex/issues/49464 and https://github.com/decolua/9router/issues/4471 (same error, fixed by raising the client version identity from 0.155.0 to 0.159.0) - Codex 0.157.0 release notes: "Add GPT-6 Sol and Luna to the model catalog": https://github.com/openai/codex/releases/tag/rust-v0.157.0 - GPT-6.1 Sol is included for Plus, Pro, Business, Enterprise, and Edu with ChatGPT sign-in: https://learn.chatgpt.com/docs/models ## What Changed - `packages/adapters/codex-local/src/index.ts`: add `minimumCodexCliVersionForModel` (`gpt-6.1-sol` → 0.159.0, `gpt-6-sol` and `gpt-6-luna` → 0.157.0, with the sources above), `parseCodexCliVersionOutput`, `codexCliVersionAtLeast`, and `CODEX_CHATGPT_MODEL_REJECTION_RE`. Models without a verified floor return `null`, so nothing probes them. The agent configuration doc describes the floors. - `packages/adapters/codex-local/src/server/cli-version.ts` (new): run `codex --version` where the run would execute it (local, SSH, or sandbox), compare it with the model floor, and build the `codex_cli_version_compatible` / `codex_cli_version_incompatible` checks. Map the backend rejection sentence to `codex_hello_probe_model_rejected` with the detected CLI version and a hint that separates a stale CLI from a plan that does not include the model. - `packages/adapters/codex-local/src/server/test.ts` (CLI lane Test): run the version check before the hello probe when the model has a floor. Skip the hello probe when the CLI is too old (`codex_hello_probe_skipped_cli_version`). When the probe still fails with the backend sentence, report `codex_hello_probe_model_rejected` instead of the generic `codex_hello_probe_failed`. The new codes do not match the auth-failure patterns, so a managed connection is not invalidated. - `packages/adapters/codex-local/src/server/acp.ts` (ACP lane Test): for remote targets, run the same version check against the shared `codex` the ACP server spawns. - `server/src/services/native-runtime/native-session-executor.ts`: after the compatibility-window check, compare the remote Codex with the configured model's floor and fail with `runner_remote_provider_artifact_incompatible: <model> requires Codex <floor> or newer with ChatGPT sign-in, received <version> from the sandbox image; promote a sandbox image with Codex 0.160.0 or configure PAPERCLIP_RUNNER_REMOTE_CODEX_NPM_SPEC=...`. When a preinstalled Codex fails this check and an npm spec is configured, the existing fallback installs the pinned release. - Tests: adapter metadata, CLI-lane Test (too old, compatible, no floor, backend rejection), ACP-lane Test (too old, compatible, no floor), and the remote runner floor (nine version/model cases). - `doc/adapter-model-audit-2026-10-02.md`: record the floors and the reason. No catalog entry, default model, saved agent configuration, pin, lockfile, or image definition changes. ## Verification Local (Node 25.9, pnpm 9.15.4 via corepack): ```sh pnpm --filter @paperclipai/adapter-codex-local typecheck pnpm --filter @paperclipai/adapter-codex-local exec vitest run # 81 passed pnpm --filter @paperclipai/server exec vitest run \ src/services/native-runtime/native-session-executor.test.ts \ src/services/native-runtime/codex-runtime-compatibility.test.ts -t Codex # 84 passed ``` Also run locally: the full `native-session-executor.test.ts` file (533 passed) and the full Codex adapter suite (81 passed). Follow-up commit `9fd4d3b25` (Greptile P2: the ACP-lane version probe ignored the agent's configured env): the probe now receives the adapter's string-valued `env` entries, the same ones `buildCodexAcpConfig` hands to remote ACP runs, so a `PATH` override selects the same `codex` for the Test as for the run. New test covers a `PATH` + `CODEX_HOME` override and a dropped non-string entry. Re-run on that head: `pnpm --filter @paperclipai/adapter-codex-local typecheck` clean; full Codex adapter suite 503 passed (31 files). Not run here: `pnpm -r typecheck` for `server` (the direct `tsc --noEmit` was killed by the sandbox memory cap; the adapter package typecheck passes and CI covers the server), `pnpm build`, browser suites, and a live sandbox probe. No UI files changed, so token gates do not apply. Manual check for a reviewer: set an agent's Codex model to `gpt-6.1-sol`, point its environment at a sandbox whose `codex --version` prints `codex-cli 0.156.0`, and run the environment Test. The result contains `codex_cli_version_incompatible` with `Detected Codex CLI 0.156.0.` and no hello probe check. With Codex 0.160.0 the Test contains `codex_cli_version_compatible` and runs the hello probe. ## Risks - A floor is enforced for all authentication modes, because the runner does not know the credential method at verification time. An OpenAI API key on Codex 0.157.0 or 0.158.0 with `gpt-6.1-sol` is now rejected before launch. Paperclip pins Codex 0.160.0 everywhere, so this only affects installs that lag the pin, and the message names the fix. - The 0.159.0 floor for `gpt-6.1-sol` is the oldest stable release verified to work; 0.157.0 and 0.158.0 were not verified either way. If OpenAI accepts an older client, the floor can be lowered in one table. - The `codex --version` probe adds one short process run to the Test only for models with a floor. - Rollback: revert this pull request. No data or configuration migrates. ## Model Used Claude Fable 5.1 (`claude-fable-5-1`), Anthropic, with extended thinking, tool use, and web research, operating as a Paperclip agent through Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Bender (Fable) <[email protected]> Co-authored-by: Claude Fable 5.1 <[email protected]>
This commit is contained in:
10 files changed
+699
-3
No files matched your search
@@ -10,7 +10,7 @@ No agent defaults or saved model selections are migrated.
|
||||
|
||||
| Adapter | Changes from the audit |
|
||||
| --- | --- |
|
||||
| Codex and the Codex runner catalog | Add GPT-6.1 Sol (`gpt-6.1-sol`). Expose efforts through Ultra and Fast mode, as documented. Keep `gpt-5.6-sol` as the default. |
|
||||
| Codex and the Codex runner catalog | Add GPT-6.1 Sol (`gpt-6.1-sol`). Expose efforts through Ultra and Fast mode, as documented. Keep `gpt-5.6-sol` as the default. Follow-up (October 6, 2026): with ChatGPT sign-in the Codex backend accepts `gpt-6.1-sol` only from Codex CLI 0.159.0 or newer (0.156.1 and older are rejected with "The 'gpt-6.1-sol' model is not supported when using Codex with a ChatGPT account."), and `gpt-6-sol` / `gpt-6-luna` entered the bundled catalog in 0.157.0. The adapter now records these floors; the environment Test and the remote runner compare the installed `codex --version` against them, so a sandbox image baked before the pin moved reports the stale CLI instead of an account error. |
|
||||
| Claude on Bedrock | Add Sonnet 5.5 (`us.anthropic.claude-sonnet-5-5`), using the catalog's existing US inference-profile convention. Sonnet 5.5 IDs (direct or Bedrock-qualified) get the documented `xhigh` and `max` efforts and require Claude Code 2.1.284 or later on the CLI lane. |
|
||||
| OpenCode | Add `openai/gpt-6.1-sol` and `anthropic/claude-sonnet-5-5` to the static fallback used by remote environments. Both IDs are present in the OpenCode model registry. |
|
||||
| Claude Code (direct) | No change in this audit. [#14993](https://github.com/paperclipai/paperclip/pull/14993) (merged October 5, 2026, superseding #14816) adds Sonnet 5.5 (`claude-sonnet-5-5`, Claude Code 2.1.284 or later) and refreshes the Claude runtime. |
|
||||
|
||||
@@ -1,12 +1,58 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
CODEX_CHATGPT_MODEL_REJECTION_RE,
|
||||
codexCliVersionAtLeast,
|
||||
codexLocalReasoningEffortsForModel,
|
||||
DEFAULT_CODEX_LOCAL_MODEL,
|
||||
isCodexLocalFastModeSupported,
|
||||
minimumCodexCliVersionForModel,
|
||||
models,
|
||||
normalizeCodexModel,
|
||||
parseCodexCliVersionOutput,
|
||||
} from "./index.js";
|
||||
|
||||
describe("codex model CLI floors", () => {
|
||||
it("records the oldest Codex CLI verified to run each gated model with ChatGPT sign-in", () => {
|
||||
expect(minimumCodexCliVersionForModel("gpt-6.1-sol")).toBe("0.159.0");
|
||||
expect(minimumCodexCliVersionForModel(" gpt-6.1-sol ")).toBe("0.159.0");
|
||||
expect(minimumCodexCliVersionForModel("gpt-6-sol")).toBe("0.157.0");
|
||||
expect(minimumCodexCliVersionForModel("gpt-6-luna")).toBe("0.157.0");
|
||||
});
|
||||
|
||||
it.each([DEFAULT_CODEX_LOCAL_MODEL, "gpt-5.6", "gpt-6-astra", "gpt-5.5", "custom-model", "", null, undefined])(
|
||||
"has no floor for %s so callers skip the version probe", (model) => {
|
||||
expect(minimumCodexCliVersionForModel(model)).toBeNull();
|
||||
},
|
||||
);
|
||||
|
||||
it("compares stable versions numerically and rejects prerelease or ambiguous input", () => {
|
||||
expect(codexCliVersionAtLeast("0.159.0", "0.159.0")).toBe(true);
|
||||
expect(codexCliVersionAtLeast("0.160.1", "0.159.0")).toBe(true);
|
||||
expect(codexCliVersionAtLeast("1.0.0", "0.159.0")).toBe(true);
|
||||
expect(codexCliVersionAtLeast("0.156.1", "0.159.0")).toBe(false);
|
||||
expect(codexCliVersionAtLeast("0.16.0", "0.159.0")).toBe(false);
|
||||
expect(codexCliVersionAtLeast("0.160.0-alpha.1", "0.159.0")).toBe(false);
|
||||
expect(codexCliVersionAtLeast("", "0.159.0")).toBe(false);
|
||||
});
|
||||
|
||||
it("parses only a single stable `codex-cli x.y.z` line from --version output", () => {
|
||||
expect(parseCodexCliVersionOutput("codex-cli 0.160.0\n")).toBe("0.160.0");
|
||||
expect(parseCodexCliVersionOutput("WARNING: PATH unchanged\r\ncodex-cli 0.156.0\r\n")).toBe("0.156.0");
|
||||
expect(parseCodexCliVersionOutput("codex-cli 0.162.0-alpha.17")).toBeNull();
|
||||
expect(parseCodexCliVersionOutput("codex 0.160.0")).toBeNull();
|
||||
expect(parseCodexCliVersionOutput("codex-cli 0.149.0\ncodex-cli 0.160.0")).toBeNull();
|
||||
expect(parseCodexCliVersionOutput("")).toBeNull();
|
||||
});
|
||||
|
||||
it("matches the backend's ChatGPT model rejection and captures the model", () => {
|
||||
const match = CODEX_CHATGPT_MODEL_REJECTION_RE.exec(
|
||||
"{\"type\":\"error\",\"status\":400,\"error\":{\"type\":\"invalid_request_error\",\"message\":\"The 'gpt-6.1-sol' model is not supported when using Codex with a ChatGPT account.\"}}",
|
||||
);
|
||||
expect(match?.[1]).toBe("gpt-6.1-sol");
|
||||
expect(CODEX_CHATGPT_MODEL_REJECTION_RE.test("Model metadata for `gpt-6.1-sol` not found")).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("codex local adapter metadata", () => {
|
||||
it("advertises current Codex-capable OpenAI models without changing the default", () => {
|
||||
const modelIds = models.map((model) => model.id);
|
||||
|
||||
@@ -85,6 +85,60 @@ export function codexLocalReasoningEffortsForModel(
|
||||
}
|
||||
}
|
||||
|
||||
// The Codex backend accepts a model for ChatGPT sign-in only from a client that identifies as a
|
||||
// recent enough CLI (the request carries the CLI version). An older `codex` fails every turn with
|
||||
// "The '<model>' model is not supported when using Codex with a ChatGPT account.", which reads
|
||||
// like an account problem when it is really a stale install, typically a sandbox image baked
|
||||
// before the catalog gained the model. Record the oldest stable release verified to work so the
|
||||
// Test and the remote runner can name the actual gap. Omit models with no verified floor; the
|
||||
// install pin in the runner and images is always at or above every floor listed here.
|
||||
const CODEX_LOCAL_MODEL_MINIMUM_CLI_VERSIONS: Readonly<Record<string, string>> = {
|
||||
// 0.156.1 and older are rejected (openai/codex#49396); 0.159.0 and 0.159.2 are accepted
|
||||
// (openai/codex#49464). No stable release between those was verified either way.
|
||||
"gpt-6.1-sol": "0.159.0",
|
||||
// Codex 0.157.0 release notes: "Add GPT-6 Sol and Luna to the model catalog".
|
||||
"gpt-6-sol": "0.157.0",
|
||||
"gpt-6-luna": "0.157.0",
|
||||
};
|
||||
|
||||
export function minimumCodexCliVersionForModel(model: string | null | undefined): string | null {
|
||||
return CODEX_LOCAL_MODEL_MINIMUM_CLI_VERSIONS[normalizeCodexModel(model)] ?? null;
|
||||
}
|
||||
|
||||
const STABLE_CODEX_CLI_VERSION = /^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)$/;
|
||||
|
||||
function stableCodexCliVersionParts(version: string): number[] | null {
|
||||
const match = STABLE_CODEX_CLI_VERSION.exec(version.trim());
|
||||
if (!match) return null;
|
||||
const parts = match.slice(1).map(Number);
|
||||
return parts.every(Number.isSafeInteger) ? parts : null;
|
||||
}
|
||||
|
||||
/** Read the version from `codex --version` output; prerelease or ambiguous output yields null. */
|
||||
export function parseCodexCliVersionOutput(output: string): string | null {
|
||||
const versions = output
|
||||
.split(/\r?\n/)
|
||||
.map((line) => /^codex-cli (\S+)$/.exec(line.trim())?.[1])
|
||||
.filter((version): version is string => version !== undefined);
|
||||
if (versions.length !== 1 || !stableCodexCliVersionParts(versions[0]!)) return null;
|
||||
return versions[0]!;
|
||||
}
|
||||
|
||||
export function codexCliVersionAtLeast(version: string, minimum: string): boolean {
|
||||
const parsedVersion = stableCodexCliVersionParts(version);
|
||||
const parsedMinimum = stableCodexCliVersionParts(minimum);
|
||||
if (!parsedVersion || !parsedMinimum) return false;
|
||||
for (let index = 0; index < parsedMinimum.length; index += 1) {
|
||||
if (parsedVersion[index] !== parsedMinimum[index]) {
|
||||
return parsedVersion[index]! > parsedMinimum[index]!;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
export const CODEX_CHATGPT_MODEL_REJECTION_RE =
|
||||
/The '([^']+)' model is not supported when using Codex with a ChatGPT account\./;
|
||||
|
||||
export function isCodexLocalKnownModel(model: string | null | undefined): boolean {
|
||||
const normalizedModel = normalizeModelId(model);
|
||||
if (!normalizedModel) return false;
|
||||
@@ -175,6 +229,7 @@ Notes:
|
||||
- Paperclip injects desired local skills into the effective CODEX_HOME/skills/ directory at execution time so Codex can discover "$paperclip" and related skills without polluting the project working directory. For new and updated agents, Paperclip assigns an isolated managed home at ~/.paperclip/instances/<id>/companies/<companyId>/agents/<agentId>/codex-home/skills/; when CODEX_HOME is explicitly overridden in adapter config, that override is used instead.
|
||||
- New and updated codex_local agents persist an empty OPENAI_API_KEY override by default so a host-level OPENAI_API_KEY cannot leak into Codex runs through process inheritance. Explicit CODEX_HOME overrides must not point at the shared company codex-home, $CODEX_HOME, or ~/.codex.
|
||||
- Some model/tool combinations reject certain effort levels (for example minimal with web search enabled).
|
||||
- With ChatGPT sign-in, the Codex backend accepts a model only from a recent enough Codex CLI: GPT-6.1 Sol needs 0.159.0 or newer, GPT-6 Sol and GPT-6 Luna need 0.157.0 or newer. An older CLI fails every turn with "The '<model>' model is not supported when using Codex with a ChatGPT account." The environment Test and the remote runner compare the installed \`codex --version\` against these floors before running; in a managed sandbox, the fix is a sandbox image that ships the Codex version Paperclip pins.
|
||||
- Fast mode is supported on GPT-6 (astra/sol/luna), GPT-6.1 Sol, GPT-5.6 (sol/terra/luna), GPT-5.5, GPT-5.4 and manual model IDs. When enabled for those models, Paperclip applies \`service_tier="fast"\` and \`features.fast_mode=true\`.
|
||||
- When Paperclip realizes a workspace/runtime for a run, it injects PAPERCLIP_WORKSPACE_* and PAPERCLIP_RUNTIME_* env vars for agent-side tooling.
|
||||
- The ACP engine keeps its workspace sandbox and enables network access on each turn. Explicit sandbox_workspace_write.network_access overrides in extraArgs (or env.PAPERCLIP_CODEX_ACP_NETWORK_ACCESS="false") disable it; execution-target network denial wins. The bundled ACP patch is needed because upstream mode presets override Codex config.toml on every turn.
|
||||
|
||||
@@ -0,0 +1,182 @@
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import type { AdapterExecutionTarget } from "@paperclipai/adapter-utils/execution-target";
|
||||
|
||||
const { ensureAdapterExecutionTargetCommandResolvable, runAdapterExecutionTargetProcess } = vi.hoisted(() => ({
|
||||
ensureAdapterExecutionTargetCommandResolvable: vi.fn(async () => {}),
|
||||
runAdapterExecutionTargetProcess: vi.fn(async () => ({
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
stdout: "codex-cli 0.156.0\n",
|
||||
stderr: "",
|
||||
pid: 1,
|
||||
startedAt: new Date().toISOString(),
|
||||
})),
|
||||
}));
|
||||
|
||||
vi.mock("@paperclipai/adapter-utils/execution-target", async () => {
|
||||
const actual = await vi.importActual<typeof import("@paperclipai/adapter-utils/execution-target")>(
|
||||
"@paperclipai/adapter-utils/execution-target",
|
||||
);
|
||||
return { ...actual, ensureAdapterExecutionTargetCommandResolvable, runAdapterExecutionTargetProcess };
|
||||
});
|
||||
|
||||
import { testCodexAcpEnvironment } from "./acp.js";
|
||||
|
||||
describe("codex ACP lane model CLI floor", () => {
|
||||
let cwd: string;
|
||||
|
||||
beforeEach(async () => {
|
||||
cwd = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-acp-version-"));
|
||||
vi.stubEnv("CODEX_HOME", await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-acp-home-")));
|
||||
vi.stubEnv("PAPERCLIP_HOME", await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-acp-instance-")));
|
||||
vi.stubEnv("PAPERCLIP_INSTANCE_ID", "default");
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
vi.clearAllMocks();
|
||||
vi.unstubAllEnvs();
|
||||
await fs.rm(cwd, { recursive: true, force: true }).catch(() => undefined);
|
||||
});
|
||||
|
||||
function sandboxTarget(): AdapterExecutionTarget {
|
||||
return {
|
||||
kind: "remote",
|
||||
transport: "sandbox",
|
||||
providerKey: "daytona",
|
||||
remoteCwd: "/remote/workspace",
|
||||
runner: {
|
||||
execute: async () => ({
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
stdout: "",
|
||||
stderr: "",
|
||||
pid: null,
|
||||
startedAt: new Date().toISOString(),
|
||||
}),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
it("fails the sandbox Test when the shared codex is older than the model's floor", async () => {
|
||||
const result = await testCodexAcpEnvironment({
|
||||
companyId: "company-1",
|
||||
adapterType: "codex_local",
|
||||
config: { cwd, model: "gpt-6.1-sol", env: { OPENAI_API_KEY: "sk-test" } },
|
||||
executionTarget: sandboxTarget(),
|
||||
environmentName: "QA Daytona",
|
||||
});
|
||||
|
||||
expect(result.status).toBe("fail");
|
||||
expect(result.checks.find((check) => check.code === "codex_cli_version_incompatible")).toMatchObject({
|
||||
level: "error",
|
||||
message: "gpt-6.1-sol requires Codex CLI 0.159.0 or newer with ChatGPT sign-in.",
|
||||
detail: "Detected Codex CLI 0.156.0.",
|
||||
});
|
||||
expect(runAdapterExecutionTargetProcess).toHaveBeenCalledTimes(1);
|
||||
const call = runAdapterExecutionTargetProcess.mock.calls[0] as unknown as [
|
||||
string,
|
||||
AdapterExecutionTarget,
|
||||
string,
|
||||
string[],
|
||||
{ env: Record<string, string> },
|
||||
];
|
||||
expect(call[1]).toMatchObject({ kind: "remote", transport: "sandbox" });
|
||||
expect(call[2]).toBe("codex");
|
||||
expect(call[3]).toEqual(["--version"]);
|
||||
expect(call[4].env).toEqual({ OPENAI_API_KEY: "sk-test" });
|
||||
});
|
||||
|
||||
it("probes the codex the agent's configured env selects, not the image default", async () => {
|
||||
runAdapterExecutionTargetProcess.mockResolvedValueOnce({
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
stdout: "codex-cli 0.160.0\n",
|
||||
stderr: "",
|
||||
pid: 1,
|
||||
startedAt: new Date().toISOString(),
|
||||
});
|
||||
|
||||
const result = await testCodexAcpEnvironment({
|
||||
companyId: "company-1",
|
||||
adapterType: "codex_local",
|
||||
config: {
|
||||
cwd,
|
||||
model: "gpt-6.1-sol",
|
||||
env: {
|
||||
OPENAI_API_KEY: "sk-test",
|
||||
PATH: "/opt/codex-0.160.0/bin:/usr/local/bin:/usr/bin",
|
||||
CODEX_HOME: "/remote/codex-home",
|
||||
PAPERCLIP_CODEX_ACP_NETWORK_ACCESS: false,
|
||||
},
|
||||
},
|
||||
executionTarget: sandboxTarget(),
|
||||
environmentName: "QA Daytona",
|
||||
});
|
||||
|
||||
expect(result.status).toBe("pass");
|
||||
expect(result.checks.find((check) => check.code === "codex_cli_version_compatible")).toMatchObject({
|
||||
message: "Codex CLI 0.160.0 satisfies the 0.159.0 minimum for gpt-6.1-sol.",
|
||||
});
|
||||
expect(runAdapterExecutionTargetProcess).toHaveBeenCalledTimes(1);
|
||||
const call = runAdapterExecutionTargetProcess.mock.calls[0] as unknown as [
|
||||
string,
|
||||
AdapterExecutionTarget,
|
||||
string,
|
||||
string[],
|
||||
{ env: Record<string, string> },
|
||||
];
|
||||
expect(call[3]).toEqual(["--version"]);
|
||||
// String-valued entries are forwarded exactly as the ACP run receives them;
|
||||
// non-string values are dropped the same way buildCodexAcpConfig consumers do.
|
||||
expect(call[4].env).toEqual({
|
||||
OPENAI_API_KEY: "sk-test",
|
||||
PATH: "/opt/codex-0.160.0/bin:/usr/local/bin:/usr/bin",
|
||||
CODEX_HOME: "/remote/codex-home",
|
||||
});
|
||||
});
|
||||
|
||||
it("passes the sandbox Test when the shared codex satisfies the floor", async () => {
|
||||
runAdapterExecutionTargetProcess.mockResolvedValueOnce({
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
stdout: "codex-cli 0.160.0\n",
|
||||
stderr: "",
|
||||
pid: 1,
|
||||
startedAt: new Date().toISOString(),
|
||||
});
|
||||
|
||||
const result = await testCodexAcpEnvironment({
|
||||
companyId: "company-1",
|
||||
adapterType: "codex_local",
|
||||
config: { cwd, model: "gpt-6.1-sol", env: { OPENAI_API_KEY: "sk-test" } },
|
||||
executionTarget: sandboxTarget(),
|
||||
environmentName: "QA Daytona",
|
||||
});
|
||||
|
||||
expect(result.status).toBe("pass");
|
||||
expect(result.checks.find((check) => check.code === "codex_cli_version_compatible")).toMatchObject({
|
||||
message: "Codex CLI 0.160.0 satisfies the 0.159.0 minimum for gpt-6.1-sol.",
|
||||
});
|
||||
});
|
||||
|
||||
it("does not probe the sandbox for models without a verified floor", async () => {
|
||||
const result = await testCodexAcpEnvironment({
|
||||
companyId: "company-1",
|
||||
adapterType: "codex_local",
|
||||
config: { cwd, model: "gpt-5.6-sol", env: { OPENAI_API_KEY: "sk-test" } },
|
||||
executionTarget: sandboxTarget(),
|
||||
environmentName: "QA Daytona",
|
||||
});
|
||||
|
||||
expect(result.status).toBe("pass");
|
||||
expect(runAdapterExecutionTargetProcess).not.toHaveBeenCalled();
|
||||
expect(result.checks.some((check) => check.code.startsWith("codex_cli_version_"))).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -49,6 +49,7 @@ import {
|
||||
stageCodexHomeForSync,
|
||||
} from "./codex-home.js";
|
||||
import { ADAPTER_AUTH_MISSING_CHECK_CODE } from "./auth-check.js";
|
||||
import { checkCodexCliVersionForModel } from "./cli-version.js";
|
||||
|
||||
const moduleDir = path.dirname(fileURLToPath(import.meta.url));
|
||||
const packageRootDir = path.resolve(moduleDir, "../..");
|
||||
@@ -648,6 +649,32 @@ export async function testCodexAcpEnvironment(
|
||||
}
|
||||
}
|
||||
|
||||
if (targetIsRemote && commandResolvable) {
|
||||
// The ACP server spawns the shared `codex` installed in the environment.
|
||||
// Models with a verified CLI floor are checked against that executable so
|
||||
// a stale sandbox image is named here, before a run fails on every turn
|
||||
// with the backend's "not supported when using Codex with a ChatGPT
|
||||
// account" rejection. Models without a floor skip the probe.
|
||||
//
|
||||
// Remote ACP runs receive the adapter's string-valued env entries (see
|
||||
// buildCodexAcpConfig), so the probe gets the same ones. An agent that
|
||||
// selects its Codex install through PATH is then measured against that
|
||||
// install rather than the image default.
|
||||
const probeEnv: Record<string, string> = {};
|
||||
for (const [key, value] of Object.entries(envConfig)) {
|
||||
if (typeof value === "string") probeEnv[key] = value;
|
||||
}
|
||||
const versionCheck = await checkCodexCliVersionForModel({
|
||||
runId: `codex-acp-envtest-${Date.now()}-${Math.random().toString(16).slice(2)}`,
|
||||
model: asString(config.model, ""),
|
||||
command: "codex",
|
||||
target,
|
||||
cwd: resolveAdapterExecutionTargetCwd(target, asString(config.cwd, ""), process.cwd()),
|
||||
env: probeEnv,
|
||||
});
|
||||
if (versionCheck) checks.push(versionCheck.check);
|
||||
}
|
||||
|
||||
const mode = firstNonEmptyString(config.mode, config.acpMode) ?? DEFAULT_ACP_ENGINE_MODE;
|
||||
const warmHandleIdleMs = asNumber(
|
||||
config.warmHandleIdleMs ?? config.acpWarmHandleIdleMs,
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
import type { AdapterEnvironmentCheck } from "@paperclipai/adapter-utils";
|
||||
import type { AdapterExecutionTarget } from "@paperclipai/adapter-utils/execution-target";
|
||||
import { runAdapterExecutionTargetProcess } from "@paperclipai/adapter-utils/execution-target";
|
||||
import {
|
||||
CODEX_CHATGPT_MODEL_REJECTION_RE,
|
||||
codexCliVersionAtLeast,
|
||||
minimumCodexCliVersionForModel,
|
||||
normalizeCodexModel,
|
||||
parseCodexCliVersionOutput,
|
||||
} from "../index.js";
|
||||
|
||||
export const CODEX_CLI_VERSION_INCOMPATIBLE_CHECK_CODE = "codex_cli_version_incompatible";
|
||||
export const CODEX_CLI_VERSION_COMPATIBLE_CHECK_CODE = "codex_cli_version_compatible";
|
||||
export const CODEX_HELLO_PROBE_MODEL_REJECTED_CHECK_CODE = "codex_hello_probe_model_rejected";
|
||||
|
||||
/**
|
||||
* Run `codex --version` where the run would execute it. Never cached: an
|
||||
* operator may upgrade Codex, or promote a new sandbox image, while the
|
||||
* Paperclip server keeps running, and the next Test should see the change.
|
||||
*/
|
||||
export async function readCodexCommandVersion(input: {
|
||||
runId: string;
|
||||
command: string;
|
||||
target: AdapterExecutionTarget | null | undefined;
|
||||
cwd: string;
|
||||
env: Record<string, string>;
|
||||
}): Promise<string | null> {
|
||||
try {
|
||||
const result = await runAdapterExecutionTargetProcess(
|
||||
input.runId,
|
||||
input.target,
|
||||
input.command,
|
||||
["--version"],
|
||||
{
|
||||
cwd: input.cwd,
|
||||
env: input.env,
|
||||
timeoutSec: 20,
|
||||
graceSec: 5,
|
||||
onLog: async () => {},
|
||||
},
|
||||
);
|
||||
if (result.timedOut || result.exitCode !== 0) return null;
|
||||
return parseCodexCliVersionOutput(`${result.stdout}\n${result.stderr}`);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function sandboxUpgradeHint(targetIsSandbox: boolean): string {
|
||||
return targetIsSandbox
|
||||
? "Promote a sandbox image that ships the Codex CLI version Paperclip pins, then retry the Test. Adding a model to the catalog does not update the Codex CLI baked into existing sandbox images."
|
||||
: "Upgrade Codex (for example `npm install -g @openai/codex@latest`) where this agent runs, then retry the Test.";
|
||||
}
|
||||
|
||||
/**
|
||||
* Compare the installed Codex CLI with the floor the configured model needs
|
||||
* for ChatGPT sign-in. Returns null when the model has no verified floor, so
|
||||
* callers skip the version probe entirely for such models.
|
||||
*/
|
||||
export async function checkCodexCliVersionForModel(input: {
|
||||
runId: string;
|
||||
model: string | null | undefined;
|
||||
command: string;
|
||||
target: AdapterExecutionTarget | null | undefined;
|
||||
cwd: string;
|
||||
env: Record<string, string>;
|
||||
}): Promise<{ check: AdapterEnvironmentCheck; compatible: boolean; detectedVersion: string | null } | null> {
|
||||
const minimum = minimumCodexCliVersionForModel(input.model);
|
||||
if (!minimum) return null;
|
||||
const model = normalizeCodexModel(input.model);
|
||||
const targetIsSandbox = input.target?.kind === "remote" && input.target.transport === "sandbox";
|
||||
const detected = await readCodexCommandVersion(input);
|
||||
if (detected && codexCliVersionAtLeast(detected, minimum)) {
|
||||
return {
|
||||
compatible: true,
|
||||
detectedVersion: detected,
|
||||
check: {
|
||||
code: CODEX_CLI_VERSION_COMPATIBLE_CHECK_CODE,
|
||||
level: "info",
|
||||
message: `Codex CLI ${detected} satisfies the ${minimum} minimum for ${model}.`,
|
||||
},
|
||||
};
|
||||
}
|
||||
return {
|
||||
compatible: false,
|
||||
detectedVersion: detected,
|
||||
check: {
|
||||
code: CODEX_CLI_VERSION_INCOMPATIBLE_CHECK_CODE,
|
||||
level: "error",
|
||||
message: `${model} requires Codex CLI ${minimum} or newer with ChatGPT sign-in.`,
|
||||
detail: detected
|
||||
? `Detected Codex CLI ${detected}.`
|
||||
: "Could not determine the installed Codex CLI version from `codex --version`.",
|
||||
hint: sandboxUpgradeHint(targetIsSandbox),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The backend rejects a model for ChatGPT sign-in with one fixed sentence.
|
||||
* Map it to a dedicated check so the cause (stale CLI, or a plan that does
|
||||
* not include the model) is named instead of a generic probe failure.
|
||||
*/
|
||||
export function codexHelloProbeModelRejectionCheck(input: {
|
||||
evidence: string;
|
||||
detectedVersion: string | null;
|
||||
targetIsSandbox: boolean;
|
||||
}): AdapterEnvironmentCheck | null {
|
||||
const match = CODEX_CHATGPT_MODEL_REJECTION_RE.exec(input.evidence);
|
||||
if (!match) return null;
|
||||
const model = match[1]!;
|
||||
const minimum = minimumCodexCliVersionForModel(model);
|
||||
const versionNote = input.detectedVersion
|
||||
? ` Detected Codex CLI ${input.detectedVersion}.`
|
||||
: "";
|
||||
return {
|
||||
code: CODEX_HELLO_PROBE_MODEL_REJECTED_CHECK_CODE,
|
||||
level: "error",
|
||||
message: `The Codex backend rejected ${model} for this ChatGPT account.`,
|
||||
detail: `${match[0]}${versionNote}`,
|
||||
hint: minimum
|
||||
? `${model} needs Codex CLI ${minimum} or newer with ChatGPT sign-in. ${sandboxUpgradeHint(input.targetIsSandbox)} If the CLI is already new enough, the plan does not include this model (Enterprise and Edu administrators must enable it) or the rollout has not reached the account; choose another model or use an OpenAI API key.`
|
||||
: `The ChatGPT plan does not include ${model}, the rollout has not reached the account, or the installed Codex CLI is older than the model requires. ${sandboxUpgradeHint(input.targetIsSandbox)} Otherwise choose another model or use an OpenAI API key.`,
|
||||
};
|
||||
}
|
||||
@@ -253,6 +253,125 @@ describe("codex remote environment diagnostics", () => {
|
||||
expect(probeCall?.[3]).toContain("--skip-git-repo-check");
|
||||
});
|
||||
|
||||
// `sandboxTarget()` is declared later in this describe block; function
|
||||
// declarations hoist, so the version-floor cases below reuse it.
|
||||
function versionProbeResult(stdout: string) {
|
||||
return {
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
stdout,
|
||||
stderr: "",
|
||||
pid: 111,
|
||||
startedAt: new Date().toISOString(),
|
||||
};
|
||||
}
|
||||
|
||||
it("fails before the hello probe when the sandbox Codex CLI is older than the model's floor", async () => {
|
||||
// A sandbox image baked before gpt-6.1-sol reached the catalog still runs
|
||||
// Codex 0.156.0. The backend would reject every turn with "not supported
|
||||
// when using Codex with a ChatGPT account"; the Test must name the real
|
||||
// gap (the CLI in the image) instead of running that doomed probe.
|
||||
runAdapterExecutionTargetProcess.mockResolvedValueOnce(versionProbeResult("codex-cli 0.156.0\n"));
|
||||
|
||||
const result = await testEnvironment({
|
||||
companyId: "company-1",
|
||||
adapterType: "codex_local",
|
||||
config: { engine: "cli", command: "codex", model: "gpt-6.1-sol" },
|
||||
executionTarget: sandboxTarget(),
|
||||
environmentName: "QA Daytona",
|
||||
});
|
||||
|
||||
expect(result.status).toBe("fail");
|
||||
const incompatible = result.checks.find((check) => check.code === "codex_cli_version_incompatible");
|
||||
expect(incompatible).toMatchObject({
|
||||
level: "error",
|
||||
message: "gpt-6.1-sol requires Codex CLI 0.159.0 or newer with ChatGPT sign-in.",
|
||||
detail: "Detected Codex CLI 0.156.0.",
|
||||
});
|
||||
expect(incompatible?.hint).toContain("sandbox image");
|
||||
expect(result.checks.some((check) => check.code === "codex_hello_probe_skipped_cli_version")).toBe(true);
|
||||
expect(result.checks.some((check) => check.code.startsWith("codex_hello_probe_") && !check.code.includes("skipped"))).toBe(false);
|
||||
expect(runAdapterExecutionTargetProcess).toHaveBeenCalledTimes(1);
|
||||
const versionCall = runAdapterExecutionTargetProcess.mock.calls[0] as unknown as [string, AdapterExecutionTarget, string, string[]];
|
||||
expect(versionCall[2]).toBe("codex");
|
||||
expect(versionCall[3]).toEqual(["--version"]);
|
||||
expect(prepareAdapterExecutionTargetRuntime).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("runs the hello probe when the sandbox Codex CLI satisfies the model's floor", async () => {
|
||||
runAdapterExecutionTargetProcess.mockResolvedValueOnce(versionProbeResult("codex-cli 0.160.0\n"));
|
||||
|
||||
const result = await testEnvironment({
|
||||
companyId: "company-1",
|
||||
adapterType: "codex_local",
|
||||
config: { engine: "cli", command: "codex", model: "gpt-6.1-sol" },
|
||||
executionTarget: sandboxTarget(),
|
||||
environmentName: "QA Daytona",
|
||||
});
|
||||
|
||||
expect(result.status).toBe("pass");
|
||||
expect(result.checks.find((check) => check.code === "codex_cli_version_compatible")).toMatchObject({
|
||||
level: "info",
|
||||
message: "Codex CLI 0.160.0 satisfies the 0.159.0 minimum for gpt-6.1-sol.",
|
||||
});
|
||||
expect(result.checks.some((check) => check.code === "codex_hello_probe_passed")).toBe(true);
|
||||
expect(runAdapterExecutionTargetProcess).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it("skips the version probe for models without a verified floor", async () => {
|
||||
const result = await testEnvironment({
|
||||
companyId: "company-1",
|
||||
adapterType: "codex_local",
|
||||
config: { engine: "cli", command: "codex", model: "gpt-5.6-sol" },
|
||||
executionTarget: sandboxTarget(),
|
||||
environmentName: "QA Daytona",
|
||||
});
|
||||
|
||||
expect(result.status).toBe("pass");
|
||||
expect(result.checks.some((check) => check.code.startsWith("codex_cli_version_"))).toBe(false);
|
||||
expect(runAdapterExecutionTargetProcess).toHaveBeenCalledTimes(1);
|
||||
const helloCall = runAdapterExecutionTargetProcess.mock.calls[0] as unknown as [string, AdapterExecutionTarget, string, string[]];
|
||||
expect(helloCall[3]).not.toEqual(["--version"]);
|
||||
});
|
||||
|
||||
it("names the backend's ChatGPT model rejection instead of a generic probe failure", async () => {
|
||||
// The CLI is new enough, but the account's plan or rollout does not include
|
||||
// the model. The backend answers with one fixed sentence; surface it with
|
||||
// the detected CLI version so the operator can tell the two causes apart.
|
||||
runAdapterExecutionTargetProcess.mockResolvedValueOnce(versionProbeResult("codex-cli 0.160.0\n"));
|
||||
runAdapterExecutionTargetProcess.mockResolvedValueOnce({
|
||||
exitCode: 1,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
stdout: "",
|
||||
stderr: "ERROR: {\"type\":\"error\",\"status\":400,\"error\":{\"type\":\"invalid_request_error\",\"message\":\"The 'gpt-6.1-sol' model is not supported when using Codex with a ChatGPT account.\"}}",
|
||||
pid: 222,
|
||||
startedAt: new Date().toISOString(),
|
||||
});
|
||||
|
||||
const result = await testEnvironment({
|
||||
companyId: "company-1",
|
||||
adapterType: "codex_local",
|
||||
config: { engine: "cli", command: "codex", model: "gpt-6.1-sol" },
|
||||
executionTarget: sandboxTarget(),
|
||||
environmentName: "QA Daytona",
|
||||
});
|
||||
|
||||
expect(result.status).toBe("fail");
|
||||
const rejected = result.checks.find((check) => check.code === "codex_hello_probe_model_rejected");
|
||||
expect(rejected).toMatchObject({
|
||||
level: "error",
|
||||
message: "The Codex backend rejected gpt-6.1-sol for this ChatGPT account.",
|
||||
detail: "The 'gpt-6.1-sol' model is not supported when using Codex with a ChatGPT account. Detected Codex CLI 0.160.0.",
|
||||
});
|
||||
expect(rejected?.hint).toContain("0.159.0");
|
||||
expect(rejected?.hint).toContain("plan");
|
||||
expect(result.checks.some((check) => check.code === "codex_hello_probe_failed")).toBe(false);
|
||||
// Not an auth failure: the managed-connection flow must not invalidate the credential.
|
||||
expect(result.checks.some((check) => check.code === "codex_hello_probe_auth_required")).toBe(false);
|
||||
});
|
||||
|
||||
it("emits the canonical adapter_auth_missing check when a sandbox hello probe reports missing auth", async () => {
|
||||
// The sandbox has no seedable credentials, so the hello probe returns an
|
||||
// authentication-required error. The Test must emit the neutral canonical
|
||||
|
||||
@@ -37,6 +37,7 @@ import {
|
||||
} from "./codex-auth-cache.js";
|
||||
import { resolveCodexExecutionEngineForRun, testCodexAcpEnvironment } from "./acp.js";
|
||||
import { ADAPTER_AUTH_MISSING_CHECK_CODE } from "./auth-check.js";
|
||||
import { checkCodexCliVersionForModel, codexHelloProbeModelRejectionCheck } from "./cli-version.js";
|
||||
|
||||
function summarizeStatus(checks: AdapterEnvironmentCheck[]): AdapterEnvironmentTestResult["status"] {
|
||||
if (checks.some((check) => check.level === "error")) return "fail";
|
||||
@@ -383,7 +384,34 @@ export async function testEnvironment(
|
||||
|
||||
const canRunProbe =
|
||||
checks.every((check) => check.code !== "codex_cwd_invalid" && check.code !== "codex_command_unresolvable");
|
||||
if (canRunProbe) {
|
||||
// Models with a verified CLI floor are compared against the installed Codex
|
||||
// before the hello probe. A probe on an older CLI only produces the backend's
|
||||
// "not supported when using Codex with a ChatGPT account" rejection, which
|
||||
// hides the real gap: the CLI (often a stale sandbox image), not the account.
|
||||
let detectedCliVersion: string | null = null;
|
||||
let configuredModelIsCompatible = true;
|
||||
if (canRunProbe && commandLooksLike(command, "codex")) {
|
||||
const versionCheck = await checkCodexCliVersionForModel({
|
||||
runId,
|
||||
model: asString(config.model, ""),
|
||||
command,
|
||||
target,
|
||||
cwd,
|
||||
env,
|
||||
});
|
||||
if (versionCheck) {
|
||||
checks.push(versionCheck.check);
|
||||
configuredModelIsCompatible = versionCheck.compatible;
|
||||
detectedCliVersion = versionCheck.detectedVersion;
|
||||
}
|
||||
}
|
||||
if (canRunProbe && !configuredModelIsCompatible) {
|
||||
checks.push({
|
||||
code: "codex_hello_probe_skipped_cli_version",
|
||||
level: "info",
|
||||
message: "Skipped hello probe because the installed Codex CLI cannot use the configured model.",
|
||||
});
|
||||
} else if (canRunProbe) {
|
||||
if (!commandLooksLike(command, "codex")) {
|
||||
checks.push({
|
||||
code: "codex_hello_probe_skipped_custom_command",
|
||||
@@ -525,7 +553,12 @@ export async function testEnvironment(
|
||||
});
|
||||
}
|
||||
} else {
|
||||
checks.push({
|
||||
const modelRejection = codexHelloProbeModelRejectionCheck({
|
||||
evidence: `${parsed.errorMessage ?? ""}\n${providerStderr}\n${probe.stdout}`,
|
||||
detectedVersion: detectedCliVersion,
|
||||
targetIsSandbox,
|
||||
});
|
||||
checks.push(modelRejection ?? {
|
||||
code: "codex_hello_probe_failed",
|
||||
level: "error",
|
||||
message: "Codex hello probe failed.",
|
||||
|
||||
@@ -11968,6 +11968,95 @@ describe("runnerd provider runtime wiring", () => {
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ version: "0.156.0", model: "gpt-6.1-sol", floor: "0.159.0" },
|
||||
{ version: "0.158.0", model: "gpt-6.1-sol", floor: "0.159.0" },
|
||||
{ version: "0.156.0", model: "gpt-6-sol", floor: "0.157.0" },
|
||||
{ version: "0.156.0", model: "gpt-6-luna", floor: "0.157.0" },
|
||||
{ version: "0.159.0", model: "gpt-6.1-sol", floor: null },
|
||||
{ version: "0.160.0", model: "gpt-6.1-sol", floor: null },
|
||||
{ version: "0.157.0", model: "gpt-6-sol", floor: null },
|
||||
{ version: "0.156.0", model: "gpt-5.6-sol", floor: null },
|
||||
{ version: "0.156.0", model: null, floor: null },
|
||||
])("rejects a sandbox Codex $version below the ChatGPT floor for $model before launch (floor=$floor)", async ({ version, model, floor }) => {
|
||||
// A preinstalled Codex inside the compatibility window can still be too
|
||||
// old for the configured model: the ChatGPT backend rejects every turn
|
||||
// with "not supported when using Codex with a ChatGPT account". The
|
||||
// verifier names the stale image instead of letting the run fail as an
|
||||
// account problem.
|
||||
const gatedExecution = {
|
||||
...execution,
|
||||
provider: { kind: "codex", model, approvalPolicy: "never" },
|
||||
binding: { ...execution.binding, runId: `run-codex-floor-${version}-${model ?? "default"}` },
|
||||
} as NativeExecutionInputV1;
|
||||
const onLog = vi.fn(async () => undefined);
|
||||
const syncIn = vi.fn(async () => undefined);
|
||||
const remoteExecute = vi.fn(
|
||||
async (command: { command: string; args?: string[] }) => {
|
||||
let stdout = "";
|
||||
const script = command.args?.[1] ?? "";
|
||||
if (command.args?.[0] === "--build-metadata") {
|
||||
stdout = JSON.stringify({
|
||||
schema: "paperclip-runner/runnerd-build-metadata/v1",
|
||||
binaryName: "paperclip-runnerd",
|
||||
packageName: "@paperclipai/paperclip-runner",
|
||||
binaryContractVersion: 2,
|
||||
durableSessionCapabilities: ["unlimited_runtime", "connection_lease_renewal"],
|
||||
prpTransportModes: ["listen_ws"],
|
||||
});
|
||||
} else if (command.args?.[0] === "--version") {
|
||||
if (command.command.endsWith("/.paperclip-runtime/paperclip-runner/bin/codex")) {
|
||||
throw new Error("reached-preinstalled-codex-verification");
|
||||
}
|
||||
stdout = `codex-cli ${version}`;
|
||||
} else if (script === "uname -s; uname -m") {
|
||||
stdout = `${process.platform === "darwin" ? "Darwin" : "Linux"}\n${process.arch === "arm64" ? "arm64" : "x86_64"}\n`;
|
||||
} else if (script.includes("command -v paperclip-runnerd")) {
|
||||
stdout = "/usr/local/bin/paperclip-runnerd\n";
|
||||
} else if (script.includes("command -v codex")) {
|
||||
stdout = "/opt/paperclip-runner/bin/codex\n";
|
||||
} else if (!script.includes("ln -sfn") && !script.includes("paperclip_codex_launcher_tmp")) {
|
||||
throw new Error(`unexpected command: ${command.command}`);
|
||||
}
|
||||
return { exitCode: 0, signal: null, timedOut: false, stderr: "", stdout };
|
||||
},
|
||||
);
|
||||
await createRunnerdBackend({
|
||||
db: leaseDb(gatedExecution),
|
||||
execution: gatedExecution,
|
||||
runnerInstanceId: "runner-image-runtime",
|
||||
onLog,
|
||||
runnerIngressAuthorized: true,
|
||||
runnerExecutionTarget: {
|
||||
kind: "remote",
|
||||
transport: "sandbox",
|
||||
remoteCwd: "/workspace",
|
||||
environmentId: "environment",
|
||||
leaseId: "lease",
|
||||
providerKey: "daytona",
|
||||
effectiveCapabilities: { runnerWebSocketIngress: true },
|
||||
runner: { execute: remoteExecute, syncIn },
|
||||
} as never,
|
||||
});
|
||||
state.createTransport.mockClear();
|
||||
state.createBackend.mock.calls.at(-1)![1].codexTransportFactory!();
|
||||
const transport = state.createTransport.mock
|
||||
.calls[0]![0] as RunnerTransportOptions & {
|
||||
controlPlaneRegistration: (authority: unknown) => Promise<unknown>;
|
||||
};
|
||||
await expect(transport.controlPlaneRegistration({})).rejects.toThrow(
|
||||
floor
|
||||
? `runner_remote_provider_artifact_incompatible: ${model} requires Codex ${floor} or newer with ChatGPT sign-in, received ${version} from the sandbox image; promote a sandbox image with Codex 0.160.0 or configure PAPERCLIP_RUNNER_REMOTE_CODEX_NPM_SPEC=@openai/[email protected]`
|
||||
: "reached-preinstalled-codex-verification",
|
||||
);
|
||||
if (floor) {
|
||||
// No npm spec is configured, so there is no fallback install; the run
|
||||
// fails before launch instead of running against the stale CLI.
|
||||
expect(remoteExecute.mock.calls.some(([call]) => call.command === "npm")).toBe(false);
|
||||
expect(remoteExecute.mock.calls.some(([call]) => call.args?.[1]?.includes("paperclip_codex_launcher_tmp"))).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
it("binds a remote launch to the configured controller-owned runner artifact", async () => {
|
||||
const remoteCwd = "/home/daytona/paperclip-workspace";
|
||||
const controllerArtifact = "/controller/artifacts/paperclip-runnerd";
|
||||
|
||||
@@ -27,6 +27,11 @@ import { bindManagedNativeCredentialTurn, completeManagedNativeCredentialTurn }
|
||||
import { createLocalNativeQuestionBridge } from "./local-native-question-bridge.js";
|
||||
import { readVerifiedRemoteWorkspaceFile } from "./remote-deliverable-file.js";
|
||||
import { copyBackCodexAuth } from "@paperclipai/adapter-codex-local/server";
|
||||
import {
|
||||
codexCliVersionAtLeast,
|
||||
minimumCodexCliVersionForModel,
|
||||
normalizeCodexModel,
|
||||
} from "@paperclipai/adapter-codex-local";
|
||||
import { nativeCompletionFeedback } from "./native-completion-feedback.js";
|
||||
import { hasAcknowledgedNativeReassignmentStopIntent, hasAcknowledgedNativeStopIntent } from "../acknowledged-native-stop.js";
|
||||
import { stoppedCodexTurnIsTextOnly } from "./stopped-codex-turn.js";
|
||||
@@ -11333,6 +11338,21 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
`runner_remote_provider_artifact_incompatible: supported Codex versions ${REMOTE_CODEX_SUPPORTED_RANGE}, received ${version ?? "an unrecognized or prerelease version"}; install a supported stable Codex release or configure PAPERCLIP_RUNNER_REMOTE_CODEX_NPM_SPEC=@openai/codex@${REMOTE_PROVIDER_PACK_PINS.codex}`,
|
||||
);
|
||||
}
|
||||
// A Codex inside the compatibility window can still be too old for the
|
||||
// configured model: the ChatGPT backend rejects a model from clients
|
||||
// below the model's floor on every turn. Fail before launch with the
|
||||
// exact gap, so a stale sandbox image is not reported as an account
|
||||
// problem. When a preinstalled Codex fails here and an npm spec is
|
||||
// configured, the caller falls back to installing the pinned release.
|
||||
const configuredModel = input.execution.provider.kind === "codex"
|
||||
? input.execution.provider.model
|
||||
: null;
|
||||
const modelMinimum = minimumCodexCliVersionForModel(configuredModel);
|
||||
if (modelMinimum && !codexCliVersionAtLeast(version, modelMinimum)) {
|
||||
throw new Error(
|
||||
`runner_remote_provider_artifact_incompatible: ${normalizeCodexModel(configuredModel)} requires Codex ${modelMinimum} or newer with ChatGPT sign-in, received ${version} from the sandbox image; promote a sandbox image with Codex ${REMOTE_PROVIDER_PACK_PINS.codex} or configure PAPERCLIP_RUNNER_REMOTE_CODEX_NPM_SPEC=@openai/codex@${REMOTE_PROVIDER_PACK_PINS.codex}`,
|
||||
);
|
||||
}
|
||||
if (version !== REMOTE_PROVIDER_PACK_PINS.codex && !reportedCodexVersions.has(version)) {
|
||||
reportedCodexVersions.add(version);
|
||||
await input.onLog?.(
|
||||
|
||||
Reference in new issue
Block a user