mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-07 06:16:06 -10:00
feat(codex): default Codex to gpt-6.1-sol, half the gpt-5.6-sol price (#19623)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Codex agents run through Paperclip Runner or the legacy
`codex_local` harness, and the runner E2E suites run paid Codex cells
> - The shared Codex default and many selections preferred `gpt-5.6-sol`
> - The reviewed OpenAI rate card in
`server/src/services/codex-pricing.ts` (`openai-standard-2026-09-30`)
prices `gpt-6.1-sol` at $2 input / $0.10 cached / $10 output per 1M
tokens, and `gpt-5.6-sol` at $4 / $0.40 / $20
> - So every default run and every paid Codex eval cell cost twice as
much as necessary
> - This pull request moves every place that prefers or selects
`gpt-5.6-sol` to `gpt-6.1-sol`, and keeps historical facts, rate cards
and compatibility fallbacks on `gpt-5.6-sol`
> - The benefit is half the token price for new default Codex agents and
for the Codex E2E and TestOps cells
## Linked Issues or Issue Description
**What existing behavior does this improve?**
The default Codex model for new agents and the Codex model that the
runner E2E suites and TestOps plans select.
**Subsystem affected**
Adapters (`adapter-utils`, `codex-local`), server native runtime
fallback, UI AI connection pools, Paperclip Runner devtools and live
evals, runner E2E catalog and TestOps mappings.
**Current behavior**
New Codex agents without an explicit model store `gpt-5.6-sol`. The
runner E2E `runner-codex` profile, the Reliability `queue-steer` plan
cell, the Product regression TestOps mapping and the ACPX Codex
qualification selection all use `gpt-5.6-sol`.
**Proposed behavior**
They use `gpt-6.1-sol`. Existing agents that store a model keep it.
`gpt-5.6-sol` stays selectable.
**Reason and benefit**
`gpt-6.1-sol` costs half as much per token on the in-repo reviewed rate
card. The current OpenAI pricing page (retrieved 2026-11-06) shows the
same short-context rates and also lists long-context rates of $4 / $0.20
/ $15 for `gpt-6.1-sol` against $8 / $0.80 / $30 for `gpt-5.6-sol`.
**Breaking changes**
None for stored configurations. See Risks.
## What Changed
Changed defaults and selections (all now `gpt-6.1-sol`):
- `PAPERCLIP_RUNNER_DEFAULT_MODELS.codex` in
`packages/adapter-utils/src/paperclip-runner-permissions.ts`. This also
changes `DEFAULT_CODEX_LOCAL_MODEL`,
`resolvePaperclipRunnerModel("codex", …)`, runner config normalization,
`codex_local` → Paperclip Runner conversion, the UI new-agent and
agent-config defaults, the composer default model and the public MCP
effort lookup.
- `codex-local` `models`: the 5.6 entry is now the literal
`gpt-5.6-sol`, because it used to be `DEFAULT_CODEX_LOCAL_MODEL`. The
list order and contents do not change.
- AI connection pool: a new OpenAI member profile uses
`PAPERCLIP_RUNNER_DEFAULT_MODELS.codex` instead of a hard-coded
`gpt-5.6-sol`.
- Paperclip Runner devtools ACPX Codex default and preset, the
`runner-codex-sol` local provider smoke, and the `acpx-codex-sol` live
workflow candidate.
- `tests/acpx-qualification-models.ts` Codex selection (runner E2E and
runner acceptance ACPX Codex profiles).
- Runner E2E: the `runner-codex` profile follows the default, so all
native Codex cells move. This includes the `runner-codex` cells among
the 52 standard `everyday-workflows` cells and the seven Codex
reliability cells. `runner-codex-mini` stays on `gpt-5.4-mini`. The
Reliability plan `queue-steer` `steeringProfile` is now
`['runner-codex', 'gpt-6.1-sol']`. Its executionId
(`everyday-workflows.runner-codex.daytona.queue-steer`), its fullName
and its TestOps case identity do not change. No separate profile is
necessary.
- `product-regression-testops.mjs`: the declared model for every
`runner-codex` case is now `gpt-6.1-sol` (it was already that for
`configured-repository`).
- `tests/runner-e2e/installed-cli.ts`: the credential-free installed
startup proof opens a session on the default model.
- `tests/runner-e2e/scheduled-prices.json`: add a `gpt-6.1-sol` price
(`openai-standard-2026-11-06`, including 30m/12h cache writes and
long-context rates from the OpenAI pricing page). The weekly scheduled
Codex cells need an exact price for their model, or admission fails
closed.
- Storybook: the composer model picker fixtures, the connection pool
stories and host, and the agent settings preview show `gpt-6.1-sol` as
the default.
- Docs: `doc/DEVELOPING.md` (the Codex conversion default) and
`tests/runner-e2e/EVERYDAY-WORKFLOWS.md` (local matrix, reliability
extras, `queue-steer` row).
- Tests that assert the default now expect `gpt-6.1-sol`. New
assertions: the default has the `0.159.0` CLI floor, the compatibility
fallback is floor-free, the `queue-steer` plan cell runs `runner-codex`
on `gpt-6.1-sol`, and all native Codex everyday cells (standard and
reliability) use `gpt-6.1-sol`.
Deliberately kept on `gpt-5.6-sol`:
- Rate cards: `server/src/services/codex-pricing.ts` and
`packages/paperclip-runner/src/evals/model-pricing.ts`. Old receipts
need them. (The runner eval rate card has no `gpt-6.1-sol` entry. An
unpriced candidate is reported as unpriced, not as zero. I did not add a
rate there because that card has its own version and source.)
- The remote Codex CLI compatibility fallback. A new
`CODEX_LOCAL_COMPATIBILITY_FALLBACK_MODEL = "gpt-5.6-sol"` replaces the
default in `codexModelFallbackCandidates`. `gpt-6.1-sol` needs Codex
0.159.0 or newer, so it cannot be the floor-free last resort. Fallback
behavior is unchanged.
- `tests/runner-e2e/provider-usage.ts`: the implicit request
cache-policy attribution stays qualified only for `gpt-5.6-sol`. I did
not verify that policy for `gpt-6.1-sol`. A `gpt-6.1-sol` response that
omits the write-TTL split keeps its cache-write cost unknown, which is
the existing behavior for unqualified models.
- `tests/runner-e2e/scheduled-measurements.json`: historical
`gpt-5.6-sol` measurements. They no longer match the current suite
definitions, so the forecast reports them as historical only. The
forecast tests now build their synthetic cohorts on the current
definition hash.
- Model catalogs that list `gpt-5.6-sol` as available (`codex-local`,
`cursor-local`, `opencode-local`, Speko voice diagnostics, Pi picker
fixture).
- The `gpt-5.6` → `gpt-5.6-sol` alias for old stored configs.
- Dated or evidence docs: `doc/plans/*`, `doc/adapter-model-audit-*`,
`QUALIFICATION-*`, `doc/connections/MEMORY.md`,
`doc/connections/TASK-REVIEWS-VERIFICATION.md`,
`tests/runner-e2e/CHAT-FEATURES.md`, the Slack eval evidence, and the
S49 soak signature text (the soak failure happened on `gpt-5.6-sol`).
- Test fixtures that parse, price, migrate or round-trip an existing
`gpt-5.6-sol` value (Rust runner tests, driver tests, pricing tests,
stored-config tests, stories that show an existing agent).
## Verification
- `npx vitest run` on every server, UI, adapter-utils and codex-local
test file that mentions `gpt-5.6-sol` or the default, plus all of
`packages/adapter-utils` and `packages/adapters/codex-local`: 5,483
passed. Three timeouts under load (`sandbox-managed-runtime`,
`acp.quota`) passed when rerun.
- `packages/paperclip-runner`: `vitest run` for `src/eval`, `src/evals`,
`src/drivers/acpx`, `src/drivers/codex`, `src/cli`, `src/contracts`,
`src/backends`: 2,641 passed.
- `node --test tests/runner-e2e/reliability-testops.test.mjs
tests/runner-e2e/product-regression-testops.test.mjs`: 28 passed.
- `node tests/runner-e2e/run-unit.mjs`: all model, catalog, reliability,
scheduled and TestOps tests pass. Failures remain only in
`copilot-protection-flow`, `denied-target-isolation-flow`,
`pi-controls-flow`, `workflow-target` and `unit-launcher`. These five
also fail on unmodified master in this local environment (filesystem
watch and git environment), and they do not touch models.
- `vitest run --config tests/runner-acceptance/vitest.config.ts`: 19
passed.
- `pnpm test:e2e:runner -- --list --suite everyday-workflows --group
reliability`: the seven `everyday-workflows.runner-codex.daytona.<case>`
cells show `gpt-6.1-sol` with `OPENAI_API_KEY,DAYTONA_API_KEY`.
`--matrix-json` gives 16 reliability cells with the same credentials as
before. `--list --all` shows no `gpt-5.6-sol` cell.
- Typecheck: `adapter-utils`, `adapter-codex-local`, `server`, `ui`,
`paperclip-runner` (TypeScript), and `pnpm test:e2e:runner:typecheck`.
- `pnpm check:token-gates`: clean.
- Not run: paid cells, Rust tests (no Rust change), the browser suites,
and the full `pnpm -r typecheck` / `pnpm build` (CI runs them).
## Risks
- Agents without an explicit model change model on their next save or
normalization, and Paperclip Runner rows with a blank model use
`gpt-6.1-sol` at execution. Stored models, including `gpt-5.6-sol`, are
never rewritten. No migration is necessary.
- `gpt-6.1-sol` needs Codex CLI 0.159.0 or newer with ChatGPT sign-in.
The runner pins `@openai/[email protected].0`. On a remote target with an
older image CLI, the existing compatibility check falls back to
`gpt-6-sol` and then `gpt-5.6-sol`, and that probe now runs for
default-model agents too. A legacy `codex_local` agent created with the
new default on a host with an older Codex CLI gets the existing
actionable "upgrade Codex" Test result.
- Qualification: the native Codex profile uses `adapter_constant` model
qualification, so it follows the default. No qualification gate is
dropped. The ACPX Codex qualification selection moves to `gpt-6.1-sol`.
Its next qualification run uses the new model. The provider-usage
cache-policy attribution stays limited to `gpt-5.6-sol`.
- Runner E2E history: the Codex cells change their suite definition
hashes, so earlier `gpt-5.6-sol` results are a separate cohort and do
not count as current.
- The `gpt-6.1-sol` scheduled price uses the file's existing `expiresAt`
(2026-11-19). It needs the same review as the other entries before then.
- No `.github/` change. The Reliability queue-steer cell keeps the
`runner-codex` profile id, so the existing workflow credential and setup
steps still match.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, running in
Claude Code with tool use (shell, file edits, web fetch of the OpenAI
pricing page).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Related: #19604 (commits the Reliability TestOps identities; this PR
changes only `steeringProfile` in the same file).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5.5 <[email protected]>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed
1 parent
05a19768ae
commit
02a2299173
31 files changed
+114
-60
No files matched your search
+6
-2
@@ -3149,8 +3149,12 @@ their existing preparation order.
|
||||
OpenCode defaults to `allow`, with explicit `ask` and `deny` options; ACPX
|
||||
defaults to `approve-all`, with explicit `approve-paperclip`, `approve-reads`,
|
||||
and `deny-all` options. Codex conversion keeps a non-empty model and
|
||||
otherwise stores the shared `gpt-5.6-sol` default. The native execution boundary
|
||||
applies the same default to older runner rows whose model is missing or blank.
|
||||
otherwise stores the shared `gpt-6.1-sol` default (half the `gpt-5.6-sol` token
|
||||
price on the reviewed OpenAI rate card in `server/src/services/codex-pricing.ts`).
|
||||
The native execution boundary applies the same default to older runner rows whose
|
||||
model is missing or blank; a stored model, including `gpt-5.6-sol`, is never
|
||||
rewritten. A remote Codex CLI older than 0.159.0 falls back to an older compatible
|
||||
Sol model (`gpt-6-sol`, then `gpt-5.6-sol`).
|
||||
|
||||
For an Agent Chat test drive, configure two agents with Paperclip Runner: native Codex and ACPX Claude. Connect
|
||||
the Claude account through the agent's **AI connection** section (or supply an
|
||||
|
||||
@@ -11,7 +11,7 @@ export type PaperclipRunnerPermissionMode =
|
||||
export const PAPERCLIP_RUNNER_IDLE_TIMEOUT_DEFAULT_MS = 300_000;
|
||||
export const PAPERCLIP_RUNNER_IDLE_TIMEOUT_MAX_MS = 86_400_000;
|
||||
export const PAPERCLIP_RUNNER_DEFAULT_MODELS = {
|
||||
codex: "gpt-5.6-sol",
|
||||
codex: "gpt-6.1-sol",
|
||||
acpx: "claude-opus-5-5",
|
||||
opencode: "openrouter/deepseek/deepseek-v4-flash-0731",
|
||||
} as const;
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
CODEX_CHATGPT_MODEL_REJECTION_RE,
|
||||
CODEX_LOCAL_COMPATIBILITY_FALLBACK_MODEL,
|
||||
codexChatGptRetirementForModel,
|
||||
codexCliVersionAtLeast,
|
||||
codexLocalReasoningEffortsForModel,
|
||||
@@ -20,7 +21,12 @@ describe("codex model CLI floors", () => {
|
||||
expect(minimumCodexCliVersionForModel("gpt-6-luna")).toBe("0.157.0");
|
||||
});
|
||||
|
||||
it.each([DEFAULT_CODEX_LOCAL_MODEL, "gpt-5.6", "gpt-6-astra", "gpt-5.5", "custom-model", "", null, undefined])(
|
||||
it("gates the default model on its CLI floor and keeps the compatibility fallback floor-free", () => {
|
||||
expect(minimumCodexCliVersionForModel(DEFAULT_CODEX_LOCAL_MODEL)).toBe("0.159.0");
|
||||
expect(minimumCodexCliVersionForModel(CODEX_LOCAL_COMPATIBILITY_FALLBACK_MODEL)).toBeNull();
|
||||
});
|
||||
|
||||
it.each([CODEX_LOCAL_COMPATIBILITY_FALLBACK_MODEL, "gpt-5.6", "gpt-6-astra", "gpt-5.5", "custom-model", "", null, undefined])(
|
||||
"has no floor for %s so callers skip the version probe", (model) => {
|
||||
expect(minimumCodexCliVersionForModel(model)).toBeNull();
|
||||
},
|
||||
@@ -72,9 +78,10 @@ describe("codex local adapter metadata", () => {
|
||||
it("advertises current Codex-capable OpenAI models without changing the default", () => {
|
||||
const modelIds = models.map((model) => model.id);
|
||||
|
||||
// Default to the concrete gpt-5.6-sol slug — Codex ships no metadata for the bare gpt-5.6
|
||||
// alias, so it must not be advertised or used as the default (it triggers a fallback warning).
|
||||
expect(DEFAULT_CODEX_LOCAL_MODEL).toBe("gpt-5.6-sol");
|
||||
// Default to gpt-6.1-sol, half the gpt-5.6-sol price on the reviewed rate card. Bare family
|
||||
// aliases such as gpt-5.6 have no Codex metadata, so they are never advertised or the default.
|
||||
expect(DEFAULT_CODEX_LOCAL_MODEL).toBe("gpt-6.1-sol");
|
||||
expect(CODEX_LOCAL_COMPATIBILITY_FALLBACK_MODEL).toBe("gpt-5.6-sol");
|
||||
// Ordered like the ChatGPT app: newest version first, then by capability within a version (#14878).
|
||||
expect(modelIds.slice(0, 13)).toEqual([
|
||||
"gpt-6.1-sol",
|
||||
|
||||
@@ -8,10 +8,15 @@ export const label = "Codex";
|
||||
|
||||
export const SANDBOX_INSTALL_COMMAND = "npm install -g @openai/codex";
|
||||
|
||||
// Use the concrete `gpt-5.6-sol` slug (Codex's own default for the 5.6 family) rather than the
|
||||
// bare `gpt-5.6` alias: OpenAI ships no model metadata for the bare slug, so passing it makes the
|
||||
// Codex CLI warn ("Model metadata for `gpt-5.6` not found") and fall back to generic context limits.
|
||||
// The default is `gpt-6.1-sol`: the reviewed OpenAI rate card in server/src/services/codex-pricing.ts
|
||||
// prices it at half of `gpt-5.6-sol`. It applies only to agents without an explicit model; a stored
|
||||
// model is never rewritten. Always use a concrete slug rather than a bare family alias such as
|
||||
// `gpt-5.6`: OpenAI ships no model metadata for bare slugs, so the Codex CLI warns and falls back to
|
||||
// generic context limits.
|
||||
export const DEFAULT_CODEX_LOCAL_MODEL = PAPERCLIP_RUNNER_DEFAULT_MODELS.codex;
|
||||
// The floor-free model a remote Codex CLI too old for the requested model falls back to. It stays
|
||||
// `gpt-5.6-sol` because the default `gpt-6.1-sol` itself needs Codex 0.159.0 or newer.
|
||||
export const CODEX_LOCAL_COMPATIBILITY_FALLBACK_MODEL = "gpt-5.6-sol";
|
||||
export const DEFAULT_CODEX_LOCAL_BYPASS_APPROVALS_AND_SANDBOX = true;
|
||||
|
||||
export function resolveCodexLocalCliPolicy(config: unknown, options: { networkAccess?: boolean } = {}) {
|
||||
@@ -219,8 +224,7 @@ export const models = [
|
||||
{ id: "gpt-6-astra", label: "gpt-6-astra" },
|
||||
{ id: "gpt-6-sol", label: "gpt-6-sol" },
|
||||
{ id: "gpt-6-luna", label: "gpt-6-luna" },
|
||||
// DEFAULT_CODEX_LOCAL_MODEL is gpt-5.6-sol, the first 5.6 entry.
|
||||
{ id: DEFAULT_CODEX_LOCAL_MODEL, label: DEFAULT_CODEX_LOCAL_MODEL },
|
||||
{ id: "gpt-5.6-sol", label: "gpt-5.6-sol" },
|
||||
{ id: "gpt-5.6-terra", label: "gpt-5.6-terra" },
|
||||
{ id: "gpt-5.6-luna", label: "gpt-5.6-luna" },
|
||||
{ id: "gpt-5.5", label: "gpt-5.5" },
|
||||
|
||||
@@ -378,7 +378,7 @@ describe("buildPaperclipRunnerConfig", () => {
|
||||
it("uses the Codex default when no model was selected", () => {
|
||||
expect(buildPaperclipRunnerConfig(makeValues({ model: "" }))).toMatchObject({
|
||||
provider: "codex",
|
||||
model: "gpt-5.6-sol",
|
||||
model: "gpt-6.1-sol",
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -55,11 +55,11 @@ const CHAT_HARNESS_KEY = "paperclip-runner.capability.chat.harness.v1";
|
||||
const MODEL_PRESETS = {
|
||||
codex: ["gpt-5.4-mini", "gpt-5.4"],
|
||||
opencode: ["openrouter/deepseek/deepseek-v4-flash-0731"],
|
||||
acpx: ["openrouter/deepseek/deepseek-v4-flash-0731", "claude-sonnet-5", "gpt-5.6-sol"],
|
||||
acpx: ["openrouter/deepseek/deepseek-v4-flash-0731", "claude-sonnet-5", "gpt-6.1-sol"],
|
||||
} as const;
|
||||
const ACPX_DEFAULT_MODELS = {
|
||||
claude: "claude-sonnet-5",
|
||||
codex: "gpt-5.6-sol",
|
||||
codex: "gpt-6.1-sol",
|
||||
} as const;
|
||||
|
||||
function acpxLabel(configuration: CapabilityHarnessConfiguration | null | undefined): string {
|
||||
|
||||
@@ -53,7 +53,7 @@ function liveCandidate(id, candidateSlots) {
|
||||
source && {
|
||||
...source,
|
||||
id: "runner-codex-sol",
|
||||
model: "gpt-5.6-sol",
|
||||
model: "gpt-6.1-sol",
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
@@ -202,7 +202,7 @@ export const RUNNER_LIVE_CANDIDATE_SLOTS: readonly RunnerLiveCandidateSlot[] =
|
||||
slotId: "acpx-codex",
|
||||
adapter: "acpx_runtime",
|
||||
provider: "acpx",
|
||||
model: "gpt-5.6-sol",
|
||||
model: "gpt-6.1-sol",
|
||||
tier: "strong",
|
||||
reasoningEffort: "medium",
|
||||
qualification: {
|
||||
|
||||
@@ -1133,7 +1133,7 @@ describe("agent routes adapter validation", () => {
|
||||
expect.objectContaining({
|
||||
adapterConfig: expect.objectContaining({
|
||||
provider: "codex",
|
||||
model: "gpt-5.6-sol",
|
||||
model: "gpt-6.1-sol",
|
||||
codexPermissionMode: "never",
|
||||
lifecycleMode: "per_turn",
|
||||
paperclipSkillSync: { desiredSkills: ["company-1/reviewer"] },
|
||||
@@ -1426,7 +1426,7 @@ describe("agent routes adapter validation", () => {
|
||||
expect(mockAgentService.update).toHaveBeenCalledWith(
|
||||
"11111111-1111-4111-8111-111111111111",
|
||||
expect.objectContaining({
|
||||
adapterConfig: expect.objectContaining({ model: "gpt-5.6-sol" }),
|
||||
adapterConfig: expect.objectContaining({ model: "gpt-6.1-sol" }),
|
||||
}),
|
||||
expect.any(Object),
|
||||
);
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { posix } from "node:path";
|
||||
import {
|
||||
codexCliVersionAtLeast,
|
||||
DEFAULT_CODEX_LOCAL_MODEL,
|
||||
CODEX_LOCAL_COMPATIBILITY_FALLBACK_MODEL,
|
||||
minimumCodexCliVersionForModel,
|
||||
models,
|
||||
normalizeCodexModel,
|
||||
@@ -11,7 +11,7 @@ import type { CommandManagedRuntimeRunner } from "@paperclipai/adapter-utils/com
|
||||
import { createNativeSshCommandRunner } from "./native-ssh-command-runner.js";
|
||||
import { isSupportedRemoteCodexVersion, parseCodexCliVersion } from "./codex-runtime-compatibility.js";
|
||||
|
||||
/** Newest older model in the same class first, then the stable Runner default. */
|
||||
/** Newest older model in the same class first, then the floor-free compatibility model. */
|
||||
export function codexModelFallbackCandidates(requestedModel: string): string[] {
|
||||
const requested = normalizeCodexModel(requestedModel);
|
||||
const identity = /^gpt-(\d+(?:\.\d+)?)-(astra|sol|terra|luna)$/.exec(requested);
|
||||
@@ -20,7 +20,7 @@ export function codexModelFallbackCandidates(requestedModel: string): string[] {
|
||||
return candidate && candidate[2] === identity[2] && Number(candidate[1]) < Number(identity[1])
|
||||
? [id] : [];
|
||||
}) : [];
|
||||
return [...new Set([...sameClass, DEFAULT_CODEX_LOCAL_MODEL])].filter((model) => model !== requested);
|
||||
return [...new Set([...sameClass, CODEX_LOCAL_COMPATIBILITY_FALLBACK_MODEL])].filter((model) => model !== requested);
|
||||
}
|
||||
|
||||
export function compatibleCodexModel(model: string | null, version: string | null): string | null {
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
/** Explicit evaluation selections, not runtime defaults or model allowlists. */
|
||||
export const ACPX_QUALIFICATION_MODELS = {
|
||||
claude: "claude-sonnet-5",
|
||||
codex: "gpt-5.6-sol",
|
||||
codex: "gpt-6.1-sol",
|
||||
grok: "grok-4.7",
|
||||
pi: "openrouter/anthropic/claude-sonnet-4.6",
|
||||
} as const;
|
||||
@@ -138,7 +138,7 @@ native Codex Daytona cell.
|
||||
| `three-turn-revise` | `runner-opencode-deepseek-1017.daytona` (per-turn lease) | S49 hit turn 3 on an ephemeral Daytona lease. |
|
||||
| `long-silent-command` | `runner-opencode-deepseek-1017.daytona` (warm lease) | S52/S52b hung on remote runners during a silent command. |
|
||||
| `tree-pause-release` | `runner-opencode-deepseek-1017.local` | S50/S50b is tree-hold and parked-retry scheduling. |
|
||||
| `queue-steer` | `runner-codex.daytona` (warm lease) | H2 needs a steering-capable driver; OpenCode has no steering. |
|
||||
| `queue-steer` | `runner-codex.daytona` (warm lease, `gpt-6.1-sol`) | H2 needs a steering-capable driver; OpenCode has no steering. |
|
||||
| `queued-across-restart` | `runner-opencode-deepseek-1017.daytona` (warm lease) | S55/U9 happened across a restart with a remote lease. |
|
||||
|
||||
The local plan cells cannot reproduce Daytona-only signatures: the remote
|
||||
@@ -150,7 +150,7 @@ S47, S48 and S50.
|
||||
|
||||
The extras are outside the TestOps plan. Every case except `queue-steer` (whose
|
||||
Codex cell is the plan cell) also runs once on native Codex (`runner-codex`,
|
||||
`gpt-5.6-sol`) on Daytona: the warm environment, except `three-turn-revise` on
|
||||
`gpt-6.1-sol`) on Daytona: the warm environment, except `three-turn-revise` on
|
||||
the per-turn lease. `long-silent-command` also runs on
|
||||
the released ACPX facades Grok (`runner-acpx-grok`), Kimi (`runner-acpx-kimi`)
|
||||
and OpenRouter/Pi (`runner-acpx-pi`), which hung 3/3 before the fix. They run
|
||||
@@ -221,7 +221,7 @@ cause" contract is checked only if a loss happens naturally.
|
||||
|
||||
## Matrix and running
|
||||
|
||||
The local matrix has fourteen cases on native Codex `gpt-5.6-sol`, native ACPX Claude
|
||||
The local matrix has fourteen cases on native Codex `gpt-6.1-sol`, native ACPX Claude
|
||||
`claude-sonnet-5`, and native Codex `gpt-5.4-mini`: 42 cells. The two core profiles
|
||||
also declare build/revise, delegation, controller-restart, and skill-creation cases
|
||||
on Daytona: eight cells. OpenCode adds only local hiring/reuse and delegation,
|
||||
|
||||
@@ -160,7 +160,7 @@ async function startupFixture(mode: "normal" | "wrong-model" | "ambient-codex" |
|
||||
constructor(options){this.options=options;}
|
||||
async openSession(){state.calls.push("open");save();
|
||||
${mode === "startup-and-cleanup-failed" ? 'throw new Error("fixture original provider startup failure");' : ''}
|
||||
const model=${JSON.stringify(mode === "wrong-model" ? "wrong-model" : "gpt-5.6-sol")};
|
||||
const model=${JSON.stringify(mode === "wrong-model" ? "wrong-model" : "gpt-6.1-sol")};
|
||||
return {ids:()=>({driverSessionId:"PRIVATE-DRIVER-ID",providerSessionId:"PRIVATE-PROVIDER-ID"}),
|
||||
events:()=>({async *[Symbol.asyncIterator](){yield {eventType:"session.started",payload:{context:{model,sandbox:{permissionProfile:{id:"paperclip-runner-workspace-read-only"}}}}};}}),
|
||||
startTurn(){state.calls.push("FORBIDDEN-TURN");save();throw new Error("No provider turn is permitted");},
|
||||
|
||||
@@ -140,7 +140,7 @@ export async function probeInstalledCodexStartup(input: {
|
||||
const driver = new runtime.CodexAppServerDriver({
|
||||
taskEnvelope: { schema: "paperclip.skillless_task.v1", objective: "Credential-free installed startup proof.",
|
||||
completionContract: { revision: "installed-startup-v1", criteria: [] }, constraints: [], expectedResultSchema: "paperclip.run_result.v1" },
|
||||
model: "gpt-5.6-sol", conversationMode: "direct", requestedCollaborationMode: "plan", approvalPolicy: "never",
|
||||
model: "gpt-6.1-sol", conversationMode: "direct", requestedCollaborationMode: "plan", approvalPolicy: "never",
|
||||
includeCollaborationModeInstructions: false, dynamicTools: [], environment,
|
||||
transportFactory: () => bundle.transport, requireProviderSessionIdentity: true,
|
||||
});
|
||||
@@ -148,7 +148,7 @@ export async function probeInstalledCodexStartup(input: {
|
||||
const event = await session.events()[Symbol.asyncIterator]().next();
|
||||
const identity = session.ids();
|
||||
const evidence = bundle.evidence();
|
||||
if (event.value?.eventType !== "session.started" || event.value.payload?.context?.model !== "gpt-5.6-sol" ||
|
||||
if (event.value?.eventType !== "session.started" || event.value.payload?.context?.model !== "gpt-6.1-sol" ||
|
||||
event.value.payload?.context?.sandbox?.permissionProfile?.id !== "paperclip-runner-workspace-read-only" ||
|
||||
!identity.driverSessionId || !identity.providerSessionId || evidence.providerDriver !== "codex_app_server" ||
|
||||
!(evidence.runnerPid > 0) || !(evidence.codexPid > 0)) throw new Error("Installed Codex startup returned no matching native identity or read-only model context");
|
||||
@@ -157,7 +157,7 @@ export async function probeInstalledCodexStartup(input: {
|
||||
if (!bundle.evidence().runnerExited) throw new Error("Installed Codex startup cleanup is incomplete");
|
||||
cleanupConfirmed = true;
|
||||
return { schema: "paperclip.e2e.installed-codex-startup/v1", platform: `${process.platform}-${process.arch}`,
|
||||
runnerSha256, codexSha256, codexVersion, providerDriver: "codex_app_server", model: "gpt-5.6-sol",
|
||||
runnerSha256, codexSha256, codexVersion, providerDriver: "codex_app_server", model: "gpt-6.1-sol",
|
||||
nativeSessionStarted: true, cleanupConfirmed: true, providerTurns: 0, credentialsUsed: false,
|
||||
defaultRuntimeResolution: true, reasoningEffortExercised: false };
|
||||
} catch (error) {
|
||||
|
||||
@@ -72,7 +72,7 @@ it("runs every standing Product regression case through its existing full-stack
|
||||
expect(projected.records[1]!.value.checks).toHaveLength(matcherResults.length);
|
||||
expect(projected.records[1]!.value.checks?.map(check => check.passed)).toEqual(matcherResults.map(check => check.passed));
|
||||
if (firstTask) {
|
||||
const alias = "gpt-5.6-sol-2026-11-01", secret = "sk-proj-syntheticcredentialabcdefghijklmnop";
|
||||
const alias = "gpt-6.1-sol-2026-11-01", secret = "sk-proj-syntheticcredentialabcdefghijklmnop";
|
||||
for (const [configuredModel, observedModels] of [[row.model, [alias]], [null, []], [row.model, [secret]]] as const) {
|
||||
const reported = observedModels[0] ?? configuredModel ?? "provider-default (unreported)";
|
||||
const output = project({ ...raw, model: reported, firstTask: { ...firstTask, configuredModel, observedModels: [...observedModels] } });
|
||||
@@ -83,7 +83,7 @@ it("runs every standing Product regression case through its existing full-stack
|
||||
expect(output.records[1]!.value.reportedModelSha256).toBe(createHash("sha256").update(reported).digest("hex"));
|
||||
expect(JSON.stringify(output)).not.toContain(reported);
|
||||
}
|
||||
for (const configuredModel of ["gpt-6.1-sol", secret]) {
|
||||
for (const configuredModel of ["gpt-5.6-sol", secret]) {
|
||||
expect(() => project({ ...raw, model: alias, firstTask: { ...firstTask, configuredModel, observedModels: [alias] } }))
|
||||
.toThrow("product_regression_testops_metadata_invalid");
|
||||
}
|
||||
|
||||
@@ -26,8 +26,7 @@ export const productRegressionCases = [
|
||||
[168, 'agent-chat', 'new-session'],
|
||||
].map(([id, suite, name, profile = 'runner-codex']) => Object.freeze({
|
||||
id, suite, name, profile,
|
||||
model: suite === 'task-workspaces' ? 'openrouter/deepseek/deepseek-v4-flash-0731'
|
||||
: suite === 'configured-repository' ? 'gpt-6.1-sol' : 'gpt-5.6-sol',
|
||||
model: suite === 'task-workspaces' ? 'openrouter/deepseek/deepseek-v4-flash-0731' : 'gpt-6.1-sol',
|
||||
fullName: `paperclip.Product E2E.${suite}.${name}`,
|
||||
executionId: `${suite}.${profile}.local.${name}`,
|
||||
plan: 'product-regression', environment: 'local',
|
||||
|
||||
@@ -20,7 +20,7 @@ const selection = cases => ({ schema: 'paperclip.product-regression-testops-sele
|
||||
const result = (overrides = {}) => ({
|
||||
executionId: selected.executionId, suiteId: selected.suite, caseId: selected.name,
|
||||
profileId: selected.profile, environmentId: 'local', source: { sha: targetSha }, attempt: 1,
|
||||
status: 'passed', cleanup: 'passed', model: 'gpt-5.6-sol', suiteDefinitionHash: 'c'.repeat(64),
|
||||
status: 'passed', cleanup: 'passed', model: 'gpt-6.1-sol', suiteDefinitionHash: 'c'.repeat(64),
|
||||
matcherResults: [{ matcher: { kind: 'json_path', path: 'observed.outputVerified', expected: true }, passed: true }],
|
||||
...overrides,
|
||||
});
|
||||
@@ -133,7 +133,7 @@ test('the transmission projection excludes raw errors, provider IDs, narratives
|
||||
|
||||
test('secret-shaped and unknown metadata cannot cross the reporting boundary', () => {
|
||||
const sentinel = 'sk-proj-syntheticcredentialabcdefghijklmnop';
|
||||
for (const mutation of [{ model: sentinel }, { model: 'gpt-6.1-sol' }, { failureClass: sentinel }, { cleanup: sentinel },
|
||||
for (const mutation of [{ model: sentinel }, { model: 'gpt-5.6-sol' }, { failureClass: sentinel }, { cleanup: sentinel },
|
||||
{ matcherResults: [{ passed: true, matcher: { kind: sentinel } }] }]) {
|
||||
assert.throws(() => projectProductResults(campaign([result(mutation)]), selection([selected]), targetSha),
|
||||
error => error.message === 'product_regression_testops_metadata_invalid');
|
||||
|
||||
@@ -27,7 +27,9 @@ it("binds every TestOps Reliability case to its existing explicit-only cell", ()
|
||||
expect(execution.environment.id).toBe(row.environment);
|
||||
expect(row.environment).toBe(cell.environment);
|
||||
// Six cells use DeepSeek 1017 on OpenCode; queue-steer needs a steering-capable driver (Codex app server).
|
||||
if (row.name === "queue-steer") expect([execution.profile.provider, execution.profile.id]).toEqual(["codex", "runner-codex"]);
|
||||
// It runs the cheaper gpt-6.1-sol, half the gpt-5.6-sol price on the reviewed rate card.
|
||||
if (row.name === "queue-steer") expect([execution.profile.provider, execution.profile.id, execution.profile.model])
|
||||
.toEqual(["codex", "runner-codex", "gpt-6.1-sol"]);
|
||||
else expect([execution.profile.provider, execution.profile.model]).toEqual(["opencode", reliabilityModel]);
|
||||
expect(row.signature).toBe(everydayReliabilitySignatures[row.name as keyof typeof everydayReliabilitySignatures].signature);
|
||||
// OpenRouter (or the OpenAI key for the Codex steering cell), plus Daytona only for sandbox cells.
|
||||
@@ -41,6 +43,13 @@ it("binds every TestOps Reliability case to its existing explicit-only cell", ()
|
||||
for (const job of buildMatrixJobs(executions)) expect(job.timeoutMinutes).toBeGreaterThan(20);
|
||||
});
|
||||
|
||||
it("runs every native Codex everyday cell, standard and reliability, on the gpt-6.1-sol default", () => {
|
||||
const codex = runnerMatrix.filter(execution => execution.suite.id === "everyday-workflows" && execution.profile.id === "runner-codex");
|
||||
expect(codex.filter(execution => everydayReliabilityCaseIds.includes(execution.task.id as never))).toHaveLength(7);
|
||||
expect(codex.filter(execution => !everydayReliabilityCaseIds.includes(execution.task.id as never)).length).toBeGreaterThan(0);
|
||||
expect(new Set(codex.map(execution => execution.profile.model))).toEqual(new Set(["gpt-6.1-sol"]));
|
||||
});
|
||||
|
||||
it("keeps a case that steers an active turn on a steering-capable driver", async () => {
|
||||
// queue-steer requires a 2xx same-turn steer. The OpenCode server driver
|
||||
// declares steering false, so the route answers steering_unsupported there.
|
||||
|
||||
@@ -13,7 +13,7 @@ export const reliabilityPlanName = 'Reliability — staging soak';
|
||||
export const reliabilitySuite = 'everyday-workflows';
|
||||
export const reliabilityProfile = 'runner-opencode-deepseek-1017';
|
||||
export const reliabilityModel = 'openrouter/deepseek/deepseek-v4.1-flash-1017';
|
||||
const steeringProfile = ['runner-codex', 'gpt-5.6-sol'];
|
||||
const steeringProfile = ['runner-codex', 'gpt-6.1-sol'];
|
||||
|
||||
// Daytona only where the soak failure was a sandbox lease failure (S49, S52/S52b,
|
||||
// S55). The catalog test binds this table to everyday-reliability-cases.ts.
|
||||
|
||||
@@ -39,7 +39,7 @@ test('the plan is six DeepSeek 1017 cells plus native Codex for steering, Dayton
|
||||
assert.ok(reliabilityDefinitions.every(row => row.fullName === `paperclip.Product E2E.everyday-workflows.${row.name}`));
|
||||
// OpenCode declares steering: false, so only queue-steer uses the steering-capable native Codex cell.
|
||||
assert.deepEqual(reliabilityDefinitions.filter(row => row.profile !== reliabilityProfile || row.model !== reliabilityModel)
|
||||
.map(row => [row.name, row.profile, row.model, row.environment]), [['queue-steer', 'runner-codex', 'gpt-5.6-sol', 'daytona']]);
|
||||
.map(row => [row.name, row.profile, row.model, row.environment]), [['queue-steer', 'runner-codex', 'gpt-6.1-sol', 'daytona']]);
|
||||
assert.deepEqual(reliabilityDefinitions.filter(row => row.profile === reliabilityProfile && row.environment === 'daytona').map(row => row.name),
|
||||
['three-turn-revise', 'long-silent-command', 'queued-across-restart']);
|
||||
assert.equal(testOpsPlanDetails.reliability.size, 7);
|
||||
|
||||
@@ -58,7 +58,7 @@ it("computes distributions within one cohort and does not sum P95s", () => {
|
||||
const snapshot = structuredClone(measurements);
|
||||
const cell = selected("daily").find(cell => cell.id.startsWith("core-"))!;
|
||||
const sample = snapshot.samples.find(sample => sample.executionId === cell.id)!;
|
||||
snapshot.samples = Array.from({ length: 20 }, (_, index) => ({ ...sample, attemptId: `attempt-${index}`,
|
||||
snapshot.samples = Array.from({ length: 20 }, (_, index) => ({ ...sample, definitionHash: cell.suiteDefinitionHash, attemptId: `attempt-${index}`,
|
||||
knownModelUsd: String(index + 1), attemptDurationMs: (index + 1) * 1000 }));
|
||||
const forecast = scheduledForecast([cell], models, snapshot);
|
||||
expect(forecast.cells[0]!.compatibleSuccessful).toEqual({ samples: 20,
|
||||
@@ -85,7 +85,7 @@ it("preserves nanodollar costs and genuine measured zero", () => {
|
||||
const snapshot = structuredClone(measurements);
|
||||
const cell = selected("daily").find(cell => cell.id.startsWith("core-"))!;
|
||||
const sample = snapshot.samples.find(sample => sample.executionId === cell.id)!;
|
||||
snapshot.samples = [{ ...sample, knownModelUsd: "0.000000001", attemptDurationMs: 0 }];
|
||||
snapshot.samples = [{ ...sample, definitionHash: cell.suiteDefinitionHash, knownModelUsd: "0.000000001", attemptDurationMs: 0 }];
|
||||
expect(scheduledForecast([cell], models, snapshot).compatibleSuccessfulSelection)
|
||||
.toEqual({ modelUsd: "0.000000001", summedAttemptMs: 0 });
|
||||
snapshot.samples[0]!.knownModelUsd = "0";
|
||||
@@ -95,7 +95,7 @@ it("preserves nanodollar costs and genuine measured zero", () => {
|
||||
it("keeps hosts separate and requires hosted evidence for the whole selection", () => {
|
||||
const snapshot = structuredClone(measurements);
|
||||
const cell = selected("daily").find(cell => cell.id.startsWith("core-"))!;
|
||||
const sample = snapshot.samples.find(sample => sample.executionId === cell.id)!;
|
||||
const sample = { ...snapshot.samples.find(sample => sample.executionId === cell.id)!, definitionHash: cell.suiteDefinitionHash };
|
||||
snapshot.samples = [sample, { ...sample, attemptId: "different-host", measurementHost: "local-docker-linux-arm64" }];
|
||||
const forecast = scheduledForecast([cell], models, snapshot);
|
||||
expect(forecast.cells[0]!.historicalCohorts).toHaveLength(2);
|
||||
|
||||
@@ -26,10 +26,12 @@ it.each(["daily", "weekly"] as const)("runs the exact %s selection through one s
|
||||
expect(manifest.models.every(model => Number(evalRequestReservation({ ...model, mode: "braintrust", apiKey: "fixture" })) <= Number(manifest.limits.maxUsd))).toBe(true);
|
||||
});
|
||||
|
||||
it("does not reduce Codex's verified context ceiling to fit the daily allowance", () => {
|
||||
it("keeps Codex's verified context ceiling and long-context reservation without joining the daily selection", () => {
|
||||
const codex = scheduledManifest({ ...input, cadence: "weekly" }).models.find(row => row.provider === "openai")!;
|
||||
expect(codex.price.model).toBe("gpt-6.1-sol");
|
||||
expect(codex.maximumInputTokens).toBe(1_050_000);
|
||||
expect(Number(evalRequestReservation({ ...codex, mode: "braintrust", apiKey: "fixture" }))).toBeGreaterThan(20);
|
||||
// The full 1.05M-token request reserves at the long-context rates; gpt-6.1-sol halves the gpt-5.6-sol hold.
|
||||
expect(Number(evalRequestReservation({ ...codex, mode: "braintrust", apiKey: "fixture" }))).toBeCloseTo(10.74576, 5);
|
||||
expect(scheduledManifest(input).models.map(model => model.provider)).toEqual(["anthropic"]);
|
||||
});
|
||||
|
||||
|
||||
@@ -3,6 +3,33 @@
|
||||
"verifiedAt": "2026-10-20T00:00:00Z",
|
||||
"expiresAt": "2026-11-19T00:00:00Z",
|
||||
"models": [
|
||||
{
|
||||
"provider": "openai",
|
||||
"maximumInputTokens": 1050000,
|
||||
"maximumOutputTokens": 16384,
|
||||
"timeoutMs": 180000,
|
||||
"price": {
|
||||
"provider": "openai",
|
||||
"model": "gpt-6.1-sol",
|
||||
"version": "openai-standard-2026-11-06",
|
||||
"sourceUrl": "https://developers.openai.com/api/docs/pricing",
|
||||
"inputUsdPerMillion": "2",
|
||||
"cachedReadUsdPerMillion": "0.1",
|
||||
"outputUsdPerMillion": "10",
|
||||
"cacheWrite30mUsdPerMillion": "2.5",
|
||||
"cacheWrite12hUsdPerMillion": "5",
|
||||
"longContext": {
|
||||
"aboveInputTokens": 272000,
|
||||
"rates": {
|
||||
"inputUsdPerMillion": "4",
|
||||
"cachedReadUsdPerMillion": "0.2",
|
||||
"outputUsdPerMillion": "15",
|
||||
"cacheWrite30mUsdPerMillion": "5",
|
||||
"cacheWrite12hUsdPerMillion": "10"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"provider": "openai",
|
||||
"maximumInputTokens": 1050000,
|
||||
|
||||
@@ -31,7 +31,7 @@ describe("resolvePaperclipRunnerTransitionModel", () => {
|
||||
|
||||
it("uses the current Codex default when the source model is blank", () => {
|
||||
expect(resolvePaperclipRunnerTransitionModel("codex_local", ""))
|
||||
.toBe("gpt-5.6-sol");
|
||||
.toBe("gpt-6.1-sol");
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -2,6 +2,7 @@ import { useEffect, useState } from "react";
|
||||
import { useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { ArrowDown, ArrowUp, ExternalLink, Loader2, MoreHorizontal, Plus, Search, Trash2, X } from "lucide-react";
|
||||
import { aiConnectionRouterAppDefinition, type AiConnectionPool, type AiConnectionPoolConfig, type AiConnectionPoolMember, type AiManagedConnectionSummary, type ToolConnection } from "@paperclipai/shared";
|
||||
import { PAPERCLIP_RUNNER_DEFAULT_MODELS } from "@paperclipai/adapter-utils";
|
||||
import { aiConnectionsApi } from "@/api/ai-connections";
|
||||
import { aiConnectionPoolsApi, type PoolInspection } from "@/api/ai-connection-pools";
|
||||
import { describeError } from "@/api/errors";
|
||||
@@ -29,7 +30,7 @@ import { AI_PROVIDERS, aiMethodLabel } from "./model";
|
||||
const configOf = ({ name, enabled, members, mode, thresholdPercent }: AiConnectionPoolConfig): AiConnectionPoolConfig => ({ name, enabled, members, mode, thresholdPercent });
|
||||
const emptyConfig = (): AiConnectionPoolConfig => ({ name: "AI connection pool", enabled: false, members: [], mode: "round_robin", thresholdPercent: 90 });
|
||||
function memberOf(account: AiManagedConnectionSummary): AiConnectionPoolMember {
|
||||
const profile: AiConnectionPoolMember["profile"] = account.provider === "openai" ? { provider: "codex", model: "gpt-5.6-sol" }
|
||||
const profile: AiConnectionPoolMember["profile"] = account.provider === "openai" ? { provider: "codex", model: PAPERCLIP_RUNNER_DEFAULT_MODELS.codex }
|
||||
: account.provider === "anthropic" ? { provider: "acpx", acpxAgent: "claude", model: "claude-sonnet-5" }
|
||||
: account.provider === "xai" ? { provider: "acpx", acpxAgent: "grok", model: "grok-4.7" }
|
||||
: { provider: "opencode", model: "openrouter/anthropic/claude-sonnet-4.6" };
|
||||
|
||||
@@ -19,7 +19,7 @@ describe("composer run settings", () => {
|
||||
expect(composerDefaultModel({ ...agent("claude_local"), adapterConfig: { env: { ANTHROPIC_MODEL: { type: "secret_ref", secretId: "model-secret" } } } })).toBe("");
|
||||
expect(composerDefaultModel({ ...agent("codex_local"), adapterConfig: { model: " private-codex " } })).toBe("private-codex");
|
||||
expect(composerDefaultModel(agent("codex_local"))).toBe("");
|
||||
expect(composerDefaultModel(agent("paperclip_runner", "codex"))).toBe("gpt-5.6-sol");
|
||||
expect(composerDefaultModel(agent("paperclip_runner", "codex"))).toBe("gpt-6.1-sol");
|
||||
expect(composerDefaultModel(agent("grok_local"))).toBe("grok-build");
|
||||
expect(composerDefaultModel(undefined)).toBe("");
|
||||
});
|
||||
|
||||
@@ -26,7 +26,7 @@ export function ConnectionPoolConnectorHost({ pluginKey = "example.pool-router",
|
||||
useEffect(() => {
|
||||
const previous = window.fetch;
|
||||
const catalog = [aiConnectionRouterAppDefinition(pluginKey, descriptor, scenario === "unavailable" ? { available: false, reason: "Ask your instance operator to enable AI connection routing." } : { available: true }), ...["openai", "anthropic", "openrouter", "github"].map(slug => getAppStoreDefinition(slug)!)];
|
||||
let pools: AiConnectionPool[] = ["setup", "empty", "catalog", "unavailable"].includes(scenario) ? [] : [{ id: poolId, companyId, pluginKey, name: "Research accounts", enabled: false, mode: ["usage", "exhausted", "unknown"].includes(scenario) ? "usage_aware" : "round_robin", thresholdPercent: 90, revision: 1, members: poolStoryAccounts.slice(0, 2).map((account, index) => ({ id: `20000000-0000-4000-8000-00000000000${index + 1}`, binding: { mode: "delegated", provider: account.provider, method: account.method, connectionId: account.id, grantId: account.grantId }, profile: index === 0 ? { provider: "codex", model: "gpt-5.6-sol" } : { provider: "acpx", acpxAgent: "claude", model: "claude-sonnet-5" } })) }];
|
||||
let pools: AiConnectionPool[] = ["setup", "empty", "catalog", "unavailable"].includes(scenario) ? [] : [{ id: poolId, companyId, pluginKey, name: "Research accounts", enabled: false, mode: ["usage", "exhausted", "unknown"].includes(scenario) ? "usage_aware" : "round_robin", thresholdPercent: 90, revision: 1, members: poolStoryAccounts.slice(0, 2).map((account, index) => ({ id: `20000000-0000-4000-8000-00000000000${index + 1}`, binding: { mode: "delegated", provider: account.provider, method: account.method, connectionId: account.id, grantId: account.grantId }, profile: index === 0 ? { provider: "codex", model: "gpt-6.1-sol" } : { provider: "acpx", acpxAgent: "claude", model: "claude-sonnet-5" } })) }];
|
||||
const accounts = scenario === "empty" ? [] : scenario === "revoked" ? poolStoryAccounts.slice(1) : poolStoryAccounts;
|
||||
const applications = (): ToolApplication[] => catalog.map(app => ({ id: `app-${app.slug}`, companyId, applicationKey: app.aiConnectionRouter ? `plugin:${pluginKey}:ai-router` : `app-gallery:${app.slug}`, name: app.name, description: app.description, type: app.aiConnectionRouter ? "paperclip_plugin" : "mcp_http", status: "active", pluginId: app.aiConnectionRouter ? "plugin-storybook" : null, ownerAgentId: null, ownerUserId: "dotta", metadata: { sourceTemplateKey: app.slug }, archivedAt: null, createdAt: date, updatedAt: date }));
|
||||
const connections = (): ToolConnection[] => [...accounts.map(account => ({ id: account.id, name: account.name, enabled: true, slug: account.provider, config: { sourceTemplateKey: account.provider, ai: { provider: account.provider, method: account.method }, aiIsolatedSubscription: true } })), ...pools.map(pool => ({ id: pool.id, name: pool.name, enabled: pool.enabled, slug: aiConnectionRouterSlug(pluginKey), config: { aiRouter: { pluginKey }, sourceTemplateKey: aiConnectionRouterSlug(pluginKey) } }))].map(row => ({ id: row.id, companyId, applicationId: `app-${row.slug}`, name: row.name, uid: row.id, connectionKind: "managed", ownership: "customer", connectionPurpose: "ai", transport: "runtime_auth", authKind: "none", credentialSource: "paperclip_vault", credentialPolicy: "per_user", status: "active", enabled: row.enabled, transportConfig: {}, config: row.config, credentialSecretRefs: [], healthStatus: "ok", healthCheckedAt: date, healthMessage: null, lastError: null, createdByAgentId: null, createdByUserId: "dotta", createdAt: date, updatedAt: date }));
|
||||
|
||||
@@ -86,7 +86,7 @@ export function AgentSettingsPreview({
|
||||
const [fixtures] = useState(() => {
|
||||
const fixtures = createSettingsFixtures(adapterType, testOutcome, saveFails);
|
||||
if (adapterType === "codex_local" || adapterType === "paperclip_runner") {
|
||||
fixtures.agent.adapterConfig = { ...fixtures.agent.adapterConfig, model: "gpt-5.6-sol", modelReasoningEffort: "high", ...(adapterType === "paperclip_runner" ? { provider: "codex" } : {}) };
|
||||
fixtures.agent.adapterConfig = { ...fixtures.agent.adapterConfig, model: "gpt-6.1-sol", modelReasoningEffort: "high", ...(adapterType === "paperclip_runner" ? { provider: "codex" } : {}) };
|
||||
}
|
||||
return fixtures;
|
||||
});
|
||||
|
||||
@@ -21,7 +21,8 @@ export type ComposerAgent = {
|
||||
};
|
||||
|
||||
const codexModels: ModelOption[] = [
|
||||
{ id: "gpt-5.6-sol", label: "GPT-5.6 Sol", detail: "Agent default" },
|
||||
{ id: "gpt-6.1-sol", label: "GPT-6.1 Sol", detail: "Agent default" },
|
||||
{ id: "gpt-5.6-sol", label: "GPT-5.6 Sol" },
|
||||
{ id: "gpt-6-astra", label: "GPT-6 Astra", detail: "Extended effort range" },
|
||||
{ id: "gpt-5.6-terra", label: "GPT-5.6 Terra" },
|
||||
{ id: "gpt-5.6-luna", label: "GPT-5.6 Luna" },
|
||||
@@ -30,7 +31,7 @@ const codexModels: ModelOption[] = [
|
||||
];
|
||||
|
||||
export const composerAgents: ComposerAgent[] = [
|
||||
{ id: "codex", name: "Codie", role: "Engineering", harness: "Codex", adapterType: "codex_local", defaultModel: "gpt-5.6-sol", models: codexModels, manualPattern: "Model ID, e.g. gpt-5.6-sol" },
|
||||
{ id: "codex", name: "Codie", role: "Engineering", harness: "Codex", adapterType: "codex_local", defaultModel: "gpt-6.1-sol", models: codexModels, manualPattern: "Model ID, e.g. gpt-6.1-sol" },
|
||||
{ id: "claude", name: "Clara", role: "Research", harness: "Claude Code", adapterType: "claude_local", defaultModel: "claude-sonnet-5", models: [
|
||||
{ id: "claude-sonnet-5", label: "Claude Sonnet 5" },
|
||||
{ id: "claude-opus-5", label: "Claude Opus 5" },
|
||||
@@ -62,7 +63,7 @@ export const composerAgents: ComposerAgent[] = [
|
||||
{ id: "sonnet-4.6", label: "Sonnet 4.6" },
|
||||
], manualPattern: "Cursor model ID" },
|
||||
{ id: "cursor-cloud", name: "Cloudy", role: "Remote engineering", harness: "Cursor Cloud", adapterType: "cursor_cloud", defaultLabel: "Account default", models: [], manualPattern: "Cursor Cloud model ID" },
|
||||
{ id: "runner", name: "Runa", role: "Operations", harness: "Paperclip Runner", adapterType: "paperclip_runner", provider: "Codex profile", defaultModel: "gpt-5.6-sol", models: codexModels, manualPattern: "Codex model ID" },
|
||||
{ id: "runner", name: "Runa", role: "Operations", harness: "Paperclip Runner", adapterType: "paperclip_runner", provider: "Codex profile", defaultModel: "gpt-6.1-sol", models: codexModels, manualPattern: "Codex model ID" },
|
||||
{ id: "grok", name: "Grok", role: "Investigation", harness: "Grok CLI", adapterType: "grok_local", defaultModel: "grok-build", models: [
|
||||
{ id: "grok-build", label: "Grok Build" },
|
||||
{ id: "grok-4.7", label: "Grok 4.7" },
|
||||
@@ -86,9 +87,9 @@ export const composerAgents: ComposerAgent[] = [
|
||||
{ id: "kimi-code/kimi-for-coding", label: "K2.8 Preview" },
|
||||
{ id: "kimi-code/k3", label: "K3" },
|
||||
], manualPattern: "kimi-code/model" },
|
||||
{ id: "long-labels", name: "Alexandra Engineering Coordinator", role: "Engineering", harness: "Codex", adapterType: "codex_local", defaultModel: "gpt-5.6-sol", models: [
|
||||
{ id: "gpt-5.6-sol", label: "GPT-5.6 Sol Extended Context Preview", detail: "Agent default" },
|
||||
], manualPattern: "Model ID, e.g. gpt-5.6-sol" },
|
||||
{ id: "long-labels", name: "Alexandra Engineering Coordinator", role: "Engineering", harness: "Codex", adapterType: "codex_local", defaultModel: "gpt-6.1-sol", models: [
|
||||
{ id: "gpt-6.1-sol", label: "GPT-6.1 Sol Extended Context Preview", detail: "Agent default" },
|
||||
], manualPattern: "Model ID, e.g. gpt-6.1-sol" },
|
||||
];
|
||||
|
||||
/** Share the capsule-avatar palettes used by the agent persona stories. */
|
||||
|
||||
@@ -18,19 +18,19 @@ const pool: AiConnectionPool = {
|
||||
id: poolId, companyId, pluginKey: "paperclip-cloud.plugin-connection-pool", revision: 3,
|
||||
name: "Research accounts", enabled: true, mode: "round_robin", thresholdPercent: 90,
|
||||
members: [
|
||||
{ id: "20000000-0000-4000-8000-000000000001", binding: { mode: "delegated", provider: "openai", method: "subscription", connectionId: "30000000-0000-4000-8000-000000000001", grantId: "40000000-0000-4000-8000-000000000001" }, profile: { provider: "codex", model: "gpt-5.6-sol", effort: "high" } },
|
||||
{ id: "20000000-0000-4000-8000-000000000001", binding: { mode: "delegated", provider: "openai", method: "subscription", connectionId: "30000000-0000-4000-8000-000000000001", grantId: "40000000-0000-4000-8000-000000000001" }, profile: { provider: "codex", model: "gpt-6.1-sol", effort: "high" } },
|
||||
{ id: "20000000-0000-4000-8000-000000000002", binding: { mode: "delegated", provider: "anthropic", method: "subscription", connectionId: "30000000-0000-4000-8000-000000000002", grantId: "40000000-0000-4000-8000-000000000002" }, profile: { provider: "acpx", acpxAgent: "claude", model: "claude-sonnet-5" } },
|
||||
],
|
||||
};
|
||||
const accounts: AiConnectionList = { currentUserId: "user-board", canManageConnections: true, pools: [pool], connections: pool.members.map((m, index) => ({
|
||||
id: m.binding.connectionId!, grantId: m.binding.grantId!, companyId, provider: m.binding.provider, method: m.binding.method!, name: index === 0 ? "My OpenAI subscription" : "My Claude subscription", ownership: "personal", ownerUserId: "user-board", isDefault: true, status: "connected",
|
||||
})) };
|
||||
const models = [{ id: "gpt-5.6-sol", label: "GPT-5.6 Sol · Codex" }, { id: "claude-sonnet-5", label: "Claude Sonnet 5 · Claude ACP" }];
|
||||
const agent: Agent = { ...storybookAgents.find(a => a.id === agentId)!, adapterType: "paperclip_runner", adapterConfig: { provider: "codex", model: "gpt-5.6-sol" }, runtimeConfig: { aiConnection: { mode: "router", connectionId: poolId } } };
|
||||
const models = [{ id: "gpt-6.1-sol", label: "GPT-6.1 Sol · Codex" }, { id: "claude-sonnet-5", label: "Claude Sonnet 5 · Claude ACP" }];
|
||||
const agent: Agent = { ...storybookAgents.find(a => a.id === agentId)!, adapterType: "paperclip_runner", adapterConfig: { provider: "codex", model: "gpt-6.1-sol" }, runtimeConfig: { aiConnection: { mode: "router", connectionId: poolId } } };
|
||||
type Scenario = "overview" | "selector" | "legacy" | "adoption" | "unavailable" | "read-only" | "models" | "effort" | "mobile" | "claude-model" | "quota" | "checking" | "run-codex" | "run-claude" | "fallback" | "retry";
|
||||
|
||||
function PoolComposer({ scenario }: { scenario: Scenario }) {
|
||||
const [settings, setSettings] = useState<ComposerRunSettings>({ model: scenario === "claude-model" ? "claude-sonnet-5" : "gpt-5.6-sol", effort: null, fast: false });
|
||||
const [settings, setSettings] = useState<ComposerRunSettings>({ model: scenario === "claude-model" ? "claude-sonnet-5" : "gpt-6.1-sol", effort: null, fast: false });
|
||||
return <div className="space-y-3 rounded-xl border bg-card p-4">
|
||||
<p className="text-sm">Choose settings for the next turn. The task keeps its account and harness.</p>
|
||||
<ComposerRunSettingsPicker companyId={companyId} assigneeValue={`agent:${agentId}`} currentAssigneeValue={`agent:${agentId}`}
|
||||
|
||||
@@ -81,7 +81,7 @@ export const ResetToAgentDefault: Story = {
|
||||
const screen = within(canvasElement.ownerDocument.body);
|
||||
await userEvent.click(screen.getByRole("button", { name: "Reset to agent default" }));
|
||||
await expect(screen.getByTestId("selected-effort")).toHaveTextContent("Default");
|
||||
await expect(screen.getByRole("button", { name: "Choose exact model" })).toHaveTextContent("GPT-5.6 Sol");
|
||||
await expect(screen.getByRole("button", { name: "Choose exact model" })).toHaveTextContent("GPT-6.1 Sol");
|
||||
await expect(screen.getByRole("button", { name: "Fast mode" })).toHaveAttribute("aria-pressed", "false");
|
||||
},
|
||||
};
|
||||
|
||||
Reference in new issue
Block a user