Compare commits

...

6 Commits

Author SHA1 Message Date
Kit Langton ca7f7c215e refactor(ai): centralize credential lowering 2026-08-19 16:15:45 -04:00
Kit Langton 930541010c fix(ai): preserve provider credential behavior 2026-08-19 14:46:54 -04:00
Kit Langton bb4b9b1ed4 test(core): provide native model credentials 2026-08-19 14:34:59 -04:00
Kit Langton f498cf26eb test(core): isolate provider auth config 2026-08-19 14:18:16 -04:00
Kit Langton 5141e60bf7 fix(ai): preserve credential precedence 2026-08-19 14:14:56 -04:00
Kit Langton 0fda7fe231 refactor(ai): simplify provider boundaries 2026-08-19 14:05:07 -04:00
73 changed files with 1282 additions and 2745 deletions
+20 -17
View File
@@ -49,7 +49,7 @@ Filter or narrow `LLMEvent` streams with `LLMEvent.is.*` (camelCase guards, e.g.
### Routes
A route is the registered, runnable composition of four orthogonal pieces:
A route is the runnable composition of four orthogonal pieces:
- **`Protocol`** (`src/route/protocol.ts`) — semantic API contract. Owns request body construction (`body.from`), the body schema (`body.schema`), the streaming-event schema (`stream.event`), and the event-to-`LLMEvent` state machine (`stream.step`). `Route.make(...)` validates and JSON-encodes the body from `body.schema` and decodes frames with `stream.event`. Examples: `OpenAIChat.protocol`, `OpenResponses.protocol`, `OpenAIResponses.protocol`, `AnthropicMessages.protocol`, `Gemini.protocol`, `BedrockConverse.protocol`.
- **`Endpoint`** (`src/route/endpoint.ts`) — URL construction. The host, path, and route query live on the endpoint. `Endpoint.path("/chat/completions", { baseURL })` is the common case; pass a function for paths that embed the model id or a body field (e.g. `Endpoint.path(({ body }) => `/model/${body.modelId}/converse-stream`)`).
@@ -66,12 +66,12 @@ export const route = Route.make({
endpoint: Endpoint.path("/chat/completions", {
baseURL: "https://api.openai.com/v1",
}),
auth: Auth.bearer(),
auth: Auth.bearer(Auth.config("OPENAI_API_KEY")),
framing: Framing.sse,
})
```
Route defaults are request-shaping defaults such as `headers`, `limits`, `generation`, `providerOptions`, and `http`. Endpoint host/query belongs on the route endpoint. Selected `LanguageModel` values carry only model id, provider id, and the configured route value. Model capability/catalog metadata lives outside this package; protocol support is enforced by request lowering and typed `AIError`s.
Route defaults are request-shaping defaults such as `headers`, `limits`, `generation`, `providerOptions`, and `http`. Endpoint host/query belongs on the route endpoint. Selected `LanguageModel` values carry model identity and the configured route; low-level callers may also attach model-specific defaults and compatibility metadata. Model capability/catalog metadata lives outside this package; protocol support is enforced by request lowering and typed `AIError`s.
The four-axis decomposition is the reason DeepSeek, TogetherAI, Cerebras, Baseten, Fireworks, and DeepInfra all reuse `OpenAIChat.protocol` verbatim — each provider deployment is a 5-15 line `Route.make(...)` call instead of a 300-400 line route clone. Bug fixes in one protocol propagate to every consumer of that protocol in a single commit.
@@ -79,7 +79,7 @@ When a provider supports multiple physical transports, selection remains executi
### URL Construction
`Endpoint` owns `{ baseURL, path, query }`. Each protocol route includes a canonical endpoint when the provider has one (e.g. `https://api.openai.com/v1`); provider helpers override endpoint fields by configuring the route before selecting a model. Routes that have no canonical URL (OpenAI-compatible Chat, GitHub Copilot) require configuration before execution.
`Endpoint` owns `{ baseURL, path, query }`. Each protocol route includes a canonical endpoint when the provider has one (e.g. `https://api.openai.com/v1`); provider helpers override endpoint fields by configuring the route before selecting a model. Generic OpenAI-compatible routes have no canonical URL and require configuration before execution.
For providers where the URL is derived from typed inputs (Azure resource name, Bedrock region), the provider helper configures the route endpoint before calling `.model(...)`. Use `AtLeastOne<T>` from `route/auth-options.ts` for inputs that accept either of two derivation paths (Azure: `resourceName` or `baseURL`).
@@ -88,7 +88,7 @@ For providers where the URL is derived from typed inputs (Azure resource name, B
Provider-facing APIs are configured facades over route values. Endpoint/auth/resource/API-version setup happens before model selection, and model selectors accept only a model or deployment id:
```ts
const openai = OpenAI.configure({ apiKey, baseURL })
const openai = OpenAI.configure({ apiKey, baseURL, store: false })
const model = openai.responses("gpt-4o-mini")
const azure = Azure.configure({ resourceName, apiKey, apiVersion: "v1" })
@@ -108,17 +108,22 @@ Keep provider facades small and explicit:
- Resolve `apiKey``Auth` with `AuthOptions.bearer(options, "<PROVIDER>_API_KEY")` (it honors an explicit `auth` override and falls back to `Auth.config(envVar)` so missing keys surface a typed `Authentication` error rather than a runtime crash).
- Use separate top-level facades for products with different required setup, such as `CloudflareAIGateway` and `CloudflareWorkersAI`.
Provider facades and model-derived `LLMRequest.providerOptions` are provider-specific, so expose typed native options flat at those boundaries. Provider package settings keep deployment configuration separate from their typed `providerOptions` field, except facades such as OpenAI whose settings are already unambiguous when flat. The selected `LanguageModel<Options>` carries request-option typing; the route decodes the flat runtime record. Keep provider metadata namespaced because replay may contain metadata from multiple layers.
`Provider.make(...)` remains available for simple static provider definitions, but new built-in providers should prefer plain configured facades unless a helper removes real duplication without adding runtime behavior.
### Provider Package Entrypoints
Catalog-selected native providers use package-like export paths from `@opencode-ai/ai`. They are internal entrypoints in one npm package, not separately published provider packages. Every entrypoint implements `ProviderPackage.Definition` and exposes `model(modelID, settings)`, where settings are serializable provider configuration plus common `headers`, `body`, and `limits` overlays.
Catalog-selected native providers use package-like export paths from `@opencode-ai/ai`. They are internal entrypoints in one npm package, not separately published provider packages. Every entrypoint implements `ProviderPackage.Definition` and exposes `model({ id, settings, credential, defaults })`. Core selects and refreshes the optional `key | oauth` credential; the provider package interprets it as route authentication. Serializable provider settings remain separate from common `headers`, `body`, and `limits` defaults.
```ts
import { model } from "@opencode-ai/ai/providers/openai/responses"
const selected = model("gpt-5", {
apiKey,
const selected = model({
id: "gpt-5",
settings: {},
credential: { type: "key", value: apiKey },
defaults: {},
})
```
@@ -144,7 +149,7 @@ packages/ai/src/
executor.ts RequestExecutor service + transport error mapping
protocol.ts Protocol type + Protocol.make
endpoint.ts Endpoint type + Endpoint.path
auth.ts Auth type + Auth.bearer / Auth.apiKeyHeader / Auth.passthrough
auth.ts Auth type + Auth.bearer / Auth.header / Auth.none
auth-options.ts ProviderAuthOption shape, AuthOptions.bearer, AtLeastOne helper
framing.ts Framing type + Framing.sse
transport/ transport implementations
@@ -169,8 +174,8 @@ packages/ai/src/
openai-compatible.ts generic Chat helper + family model helpers
openai-compatible-responses.ts generic Responses helper
openai-compatible-profile.ts family defaults (deepseek, togetherai, ...)
azure.ts / amazon-bedrock.ts / cloudflare.ts / github-copilot.ts / google.ts / xai.ts / openai.ts / anthropic.ts / openrouter.ts
tool.ts typed tool() helper
azure.ts / amazon-bedrock.ts / cloudflare.ts / google.ts / xai.ts / openai.ts / anthropic.ts / openrouter.ts
tool.ts typed Tool.make helper
tool-runtime.ts narrow one-call typed tool dispatcher
```
@@ -221,19 +226,17 @@ Routes lower these into provider-native assistant tool-call messages and tool-re
### Tool dispatch
`LLM.stream(request)` and `LLM.generate(request)` each run exactly one provider turn. Add tool schemas to `request.tools` with `Tool.toDefinitions(tools)`. When a caller wants the package's typed one-call execution behavior, pass each canonical local `tool-call` event to `ToolRuntime.dispatch(tools, call)`.
`LLM.stream(request)` and `LLM.generate(request)` each run exactly one model call. Add tool schemas to `request.tools` with `Tool.toDefinitions(tools)`. When a caller wants the package's typed one-call execution behavior, pass each canonical local `tool-call` event to `ToolRuntime.dispatch(tools, call)`.
```ts
const get_weather = tool({
const get_weather = Tool.make({
description: "Get current weather for a city",
parameters: Schema.Struct({ city: Schema.String }),
success: Schema.Struct({ temperature: Schema.Number, condition: Schema.String }),
execute: ({ city }) =>
execute: (input) =>
Effect.gen(function* () {
// city: string — typed from parameters Schema
const data = yield* WeatherApi.fetch(city)
const data = yield* WeatherApi.fetch(input.city)
return { temperature: data.temp, condition: data.cond }
// return type checked against success Schema
}),
})
File diff suppressed because it is too large Load Diff
+80 -18
View File
@@ -34,8 +34,10 @@ Run `LLMClient.stream(request)` instead of `generate` when you want incremental
Use `Image.generate` with an image model for direct asset generation:
```ts
import { Image, ImageInput } from "@opencode-ai/ai"
import { Effect, Layer } from "effect"
import { Image, ImageClient, ImageInput } from "@opencode-ai/ai"
import { OpenAI } from "@opencode-ai/ai/providers"
import { RequestExecutor } from "@opencode-ai/ai/route"
const program = Effect.gen(function* () {
const response = yield* Image.generate({
@@ -52,6 +54,10 @@ const program = Effect.gen(function* () {
return response.images // GeneratedImage[] with owned bytes or a provider URL
})
const imageLayer = ImageClient.layer.pipe(Layer.provide(RequestExecutor.fetchLayer))
await Effect.runPromise(program.pipe(Effect.provide(imageLayer)))
```
Pass ordered image inputs to the same method for editing, composition, or image-conditioned generation:
@@ -199,7 +205,7 @@ The hosted result is represented as a provider-executed tool call and tool resul
- **`LLM.request({...})`** — build a provider-neutral `LLMRequest`. Accepts ergonomic inputs (`system: string`, `prompt: string`) that normalize into the canonical Schema classes.
- **`LLM.generate` / `LLM.stream`** — re-exported from `LLMClient` for one-import use.
- **`Message.user(...)` / `Message.assistant(...)` / `Message.tool(...)`** — message constructors from the canonical schema model.
- **`Message.system(...)` / `Message.user(...)` / `Message.assistant(...)` / `Message.tool(...)`** — message constructors from the canonical schema model. Top-level `request.system` is the initial prompt; a system message in history is a chronological operator update.
- **`LanguageModel.make(...)` / `ToolCallPart.make(...)` / `ToolResultPart.make(...)` / `ToolDefinition.make(...)`** — model and tool-related constructors from the canonical schema model.
- **`LLMEvent.is.*`** — typed guards (`is.textDelta`, `is.toolCall`, `is.finish`, …) for filtering streams.
- **`Image.generate({...})`** — generate images through a provider-neutral image request and response model.
@@ -237,11 +243,11 @@ Prompt caching is **on by default**. Every `LLMRequest` resolves to `cache: "aut
### Auto placement
`"auto"` places up to four breakpoints — the last tool definition, the first system part, the last system part when distinct, and the final message boundary. These expose successively larger reusable prefixes for tools, the base agent, project instructions, and the active conversation. The rolling final-message boundary is the load-bearing detail in tool loops: it advances on every request so the previous cache entry stays within Anthropic's 20-block lookback.
`"auto"` places up to four breakpoints — the last tool definition, the first system part, the last system part when distinct, and the final message boundary. These expose successively larger reusable prefixes for tools, the base agent, project instructions, and the active conversation. The rolling final-message boundary advances on every request so recent conversation prefixes remain reusable during tool loops.
Tools precede every system and conversation block in the provider prefix, so tool definitions must remain byte-stable and deterministically ordered for downstream breakpoints to remain reusable.
The math justifies the default: Anthropic's 5-minute cache write is 1.25× base, read is 0.1×, so a single reuse within 5 minutes already wins. One-shot completions below the per-model minimum-cacheable-token threshold silently no-op on the wire, so the worst case is harmless.
Requests below a provider's minimum cacheable size simply do not produce a reusable cache entry.
### Opting out
@@ -285,6 +291,7 @@ LLM.request({
| ----------------------- | ------------------------------------------------------------------------- |
| Anthropic Messages | emits up to 4 `cache_control` markers (4-breakpoint cap enforced) |
| Bedrock Converse | emits up to 4 `cachePoint` blocks (4-breakpoint cap enforced) |
| OpenRouter | emits up to 4 `cache_control` markers |
| OpenAI Chat / Responses | no-op (implicit caching above 1024 tokens) |
| Gemini | no-op (implicit caching on 2.5+; explicit `CachedContent` is out-of-band) |
@@ -304,19 +311,26 @@ const gateway = CloudflareAIGateway.configure({
}).model("workers-ai/@cf/meta/llama-3.1-8b-instruct")
```
Included providers: OpenAI, Anthropic, Google (Gemini), Google Vertex Gemini and Anthropic, Amazon Bedrock, Azure OpenAI, Cloudflare AI Gateway, Cloudflare Workers AI, GitHub Copilot, OpenRouter, xAI, Z.ai, plus generic OpenAI-compatible Chat and Responses entrypoints and an Anthropic Messages-compatible entrypoint.
Included providers: OpenAI, Anthropic, Google (Gemini), Google Vertex Gemini and Anthropic, Amazon Bedrock, Azure OpenAI, Cloudflare AI Gateway, Cloudflare Workers AI, OpenRouter, xAI, Z.ai, plus generic OpenAI-compatible Chat and Responses entrypoints and an Anthropic Messages-compatible entrypoint. GitHub Copilot remains a Core-owned AI SDK integration rather than an AI-package provider.
### Package-like entrypoints
Native catalog integrations load provider behavior through package-like entrypoints. These are export paths from the same `@opencode-ai/ai` npm package, not independently published packages. Each entrypoint exports the same `model(modelID, settings)` contract, and `settings` contains serializable provider configuration plus common `headers`, `body`, and `limits` overlays.
Native catalog integrations load provider behavior through package-like entrypoints. These are export paths from the same `@opencode-ai/ai` npm package, not independently published packages. Each entrypoint exports the same `model({ id, settings, credential, defaults })` contract. Core selects and refreshes the optional `key | oauth` credential, while the provider package interprets it as route authentication. Serializable provider settings remain separate from common `headers`, `body`, and `limits` defaults.
```ts
import { model } from "@opencode-ai/ai/providers/openai/responses"
const selected = model("gpt-5", {
apiKey: process.env.OPENAI_API_KEY,
headers: { "x-application": "opencode" },
limits: { context: 200_000, output: 64_000 },
const apiKey = process.env.OPENAI_API_KEY
if (!apiKey) throw new Error("OPENAI_API_KEY is required")
const selected = model({
id: "gpt-5",
settings: {},
credential: { type: "key", value: apiKey },
defaults: {
headers: { "x-application": "opencode" },
limits: { context: 200_000, output: 64_000 },
},
})
```
@@ -340,30 +354,57 @@ Tuned Vertex Gemini deployments use model ids shaped like `endpoints/1234567890`
```ts
import { model } from "@opencode-ai/ai/providers/google-vertex/gemini"
model("gemini-3.5-flash", { project: "my-project", location: "global" })
model({
id: "gemini-3.5-flash",
settings: { project: "my-project", location: "global" },
defaults: {},
})
```
```ts
import { model } from "@opencode-ai/ai/providers/google-vertex/chat"
model("deepseek-ai/deepseek-v3.2-maas", { project: "my-project", location: "global" })
model({
id: "deepseek-ai/deepseek-v3.2-maas",
settings: { project: "my-project", location: "global" },
defaults: {},
})
```
```ts
import { model } from "@opencode-ai/ai/providers/google-vertex/responses"
model("xai/grok-4.20-reasoning", { project: "my-project", location: "global" })
model({
id: "xai/grok-4.20-reasoning",
settings: { project: "my-project", location: "global" },
defaults: {},
})
```
```ts
import { model } from "@opencode-ai/ai/providers/google-vertex/messages"
model("claude-sonnet-4-6", { project: "my-project", location: "global" })
model({
id: "claude-sonnet-4-6",
settings: { project: "my-project", location: "global" },
defaults: {},
})
```
Provider facades such as `OpenAI.configure(...).responses(...)` remain the direct application API. Package-like entrypoints are the self-similar loading contract used when a catalog selects behavior by export path.
Provider facades such as `OpenAI.configure(...).responses(...)` remain the direct application API. Package-like entrypoints are the self-similar loading contract used when a catalog selects behavior by export path. The entrypoints listed above implement that contract and are covered by `test/provider-package.test.ts`.
Other provider exports listed above remain direct facades until they explicitly implement the package-like contract. Exporting a provider facade does not implicitly make it a catalog-loadable provider package.
## How OpenCode uses this package
OpenCode does not call provider facades directly from the CLI or server. Core owns the integration:
1. `packages/core/src/model-resolver.ts` resolves catalog metadata and an active integration credential into a `LanguageModel`. Native package entrypoints expose `model({ id, settings, credential, defaults })`; catalog packages without a native mapping fall back through Core's AI SDK adapter.
2. `packages/core/src/session/model-request.ts` lowers Session state, instructions, tools, and plugin hooks into one canonical `LLMRequest`.
3. `packages/core/src/session/runner/llm.ts` calls the yielded `LLMClient.Service` once per physical attempt and persists provider-neutral `LLMEvent`s.
4. Core owns retries, continuation, compaction, permissions, durable tool execution, and Session history. None of that orchestration belongs in this package.
Title generation, compaction, standalone generation, and transient Session generation also build `LLMRequest`s and use the same `LLMClient.Service`. Core's `AISDK` adapter wraps remaining Vercel AI SDK models in executable routes so native and fallback providers present the same request and event model to callers.
This separation is intentional: `@opencode-ai/ai` owns one model call, provider protocols, and transport; Core owns the durable agent runtime.
## Provider options & HTTP overlays
@@ -371,11 +412,33 @@ Request options in order of stability:
1. **`generation`** — portable knobs (`maxTokens`, `temperature`, `topP`, `topK`, penalties, seed, stop).
2. **`promptCacheKey`** — stable cache affinity lowered by every protocol that supports it.
3. **`providerOptions: { <provider>: {...} }`** — typed-at-the-facade provider-specific knobs (OpenAI `store`, Anthropic `thinking`, Gemini `thinkingConfig`, OpenRouter routing).
3. **`providerOptions: { ... }`** — flat options inferred from the selected model (OpenAI `store`, Anthropic `thinking`, Gemini `thinkingConfig`, OpenRouter routing).
4. **`http: { body, headers, query }`** — last-resort serializable overlays merged into the final HTTP request. Reach for this only when a stable typed path doesn't yet exist.
Route/provider defaults are overridden by request-level values for each axis.
Provider-specific facades accept their own options directly because the provider is already known:
```ts
const model = OpenAI.configure({
apiKey,
store: false,
reasoningEffort: "high",
}).responses("gpt-5")
```
The selected model supplies the provider-specific option type, so per-request overrides stay flat while the canonical runtime request remains provider-neutral:
```ts
LLM.request({
model,
prompt,
providerOptions: {
reasoningEffort: "low",
},
})
```
## Routes
Adding a new model or deployment is usually 5-15 lines using `Route.make({ protocol, endpoint, auth, framing, ... })`. The route owns endpoint/auth/framing and the protocol owns body construction plus stream parsing. Transports are reusable IO templates that receive route endpoint/auth at compile time. Capability/catalog metadata lives outside this low-level package; unsupported request shapes fail during protocol lowering. See `AGENTS.md` for the architectural detail.
@@ -387,6 +450,5 @@ This package is built on Effect. Public methods return `Effect` or `Stream`; pro
## See also
- `AGENTS.md` — architecture, route construction, contributor guide
- `STATUS.md` — native provider parity status and AI SDK migration gaps
- `example/tutorial.ts` — runnable end-to-end walkthrough
- `test/provider/*.test.ts` — fixture-first protocol tests; `*.recorded.test.ts` files cover live cassettes
-107
View File
@@ -1,107 +0,0 @@
# LLM Provider Parity Status
Last reviewed: 2026-08-07
This file tracks the gap between the native `@opencode-ai/ai` package and the AI SDK provider packages that opencode still depends on for many catalog/runtime paths.
## Existing Status Sources
| File | What it tracks | Limitation |
| ----------------------------------- | ----------------------------------------------------------------------- | ------------------------------------------------------- |
| `packages/ai/DESIGN.md` | Future clean-break API proposal for `@opencode-ai/ai`. | Not a provider parity tracker. |
| `packages/ai/example/call-sites.md` | Route/value/provider-facade migration checklist and call-site sketches. | Architecture migration only; not AI SDK package parity. |
## Current Implementation Snapshot
| Native slice | Source | Current state | Main gaps |
| ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| OpenAI Chat | `src/protocols/openai-chat.ts`, `src/providers/openai.ts` | Usable. Streams text, reasoning deltas, tool calls, usage, images, and common generation controls. | No typed structured-output / `response_format` path. Limited typed OpenAI option surface compared with SDK escape hatches. |
| OpenAI Responses | `src/protocols/open-responses.ts`, `src/protocols/openai-responses.ts`, `src/providers/openai.ts` | Usable over HTTP by default, with optional per-call WebSocket channel execution on the same model and route identity. | No incremental `previous_response_id` path or persistent Session channel manager yet. Typed options cover only a subset of Responses fields. Structured output is still mostly synthetic-tool based. |
| OpenAI-compatible Chat | `src/protocols/openai-compatible-chat.ts`, `src/providers/openai-compatible.ts` | Usable for generic Chat and several profiles: Baseten, Cerebras, DeepInfra, DeepSeek, Fireworks, Groq, TogetherAI. | Family quirks are mostly endpoint defaults, not full typed behavior. |
| Open Responses-compatible | `src/protocols/open-responses.ts`, `src/protocols/openai-compatible-responses.ts`, `src/providers/openai-compatible-responses.ts` | Usable for deployments that implement the provider-neutral Open Responses protocol. The deployment adapter does not inherit OpenAI tools, events, metadata, or defaults. | No named family profiles or recorded deployment coverage yet. |
| Anthropic-compatible Messages | `src/protocols/anthropic-messages.ts`, `src/providers/anthropic-compatible.ts` | Usable for deployments that implement the Anthropic Messages wire protocol. Named Anthropic composes this base; MiniMax M3 has recorded text and tool-loop coverage. | No named compatible family profiles yet. |
| Anthropic Messages | `src/protocols/anthropic-messages.ts`, `src/providers/anthropic.ts` | Usable. Supports tools, thinking, cache control, images, server-hosted tool events, and usage. | Provider option surface is small. Beta/header handling, metadata, and newer Messages fields need a typed parity pass. |
| Gemini Developer API | `src/protocols/gemini.ts`, `src/providers/google.ts` | Usable for Google API key flow. Supports text, images, tools, thinking signatures, and cache usage. | This is not Vertex. Typed provider options are narrow; many Gemini request fields currently require raw `http.body` overlays. |
| Vertex Gemini | `src/protocols/gemini.ts`, `src/providers/google-vertex.ts` | Usable through API-key express mode, explicit OAuth tokens, or ADC with project/location endpoint derivation, including tuned `endpoints/...` deployments. | Core runner/catalog mapping and recorded provider coverage are missing. |
| Vertex Chat | `src/protocols/openai-chat.ts`, `src/providers/google-vertex-chat.ts` | Usable for MaaS models through OpenAI-compatible Chat Completions with explicit OAuth tokens or ADC and project/location endpoint derivation. | Core runner/catalog mapping and recorded provider coverage are missing; MaaS family-specific request parity needs review. |
| Vertex Responses | `src/protocols/open-responses.ts`, `src/providers/google-vertex-responses.ts` | Usable for Grok models through Open Responses with explicit OAuth tokens or ADC, project/location endpoint derivation, and an explicit `store: false` Vertex default. | Core runner/catalog mapping and recorded provider coverage are missing; stateful continuation is not supported by Vertex. |
| Vertex Messages | `src/protocols/anthropic-messages.ts`, `src/providers/google-vertex-messages.ts` | Usable through explicit OAuth tokens or ADC, including global, regional, and `eu`/`us` multi-region endpoints. | Core runner/catalog mapping and recorded provider coverage are missing; Vertex-specific hosted-tool parity needs review. |
| Bedrock Converse | `src/protocols/bedrock-converse.ts`, `src/providers/amazon-bedrock.ts` | Partial but real. Supports AWS event-stream framing, SigV4 with supplied credentials, bearer auth, tools, reasoning signatures, media, cache points, and recorded tests. | Native facade does not mirror the AI SDK plugin's default AWS credential chain/profile behavior. Runner/catalog mapping is missing. Guardrails, inference profiles, region-specific model ID fixes, and model-specific request fields need a parity pass. |
| Azure OpenAI | `src/providers/azure.ts` using OpenAI Chat/Responses protocols | Partial. Supports resource/base URL setup, API key auth, API version query, Chat, and Responses selectors. | Core runner does not map `@ai-sdk/azure` to this native facade. AAD/token auth and Azure-specific endpoint variants need review. |
| Cloudflare AI Gateway / Workers AI | `src/providers/cloudflare.ts` | Present via OpenAI-compatible Chat routes. | Useful but not part of the critical AI SDK replacement set yet. Needs per-product recorded coverage before relying on it broadly. |
| OpenRouter | `src/providers/openrouter.ts` | Present with OpenRouter-specific usage/reasoning/prompt-cache options over Chat. | Responses-style OpenRouter support is absent. |
| xAI | `src/providers/xai.ts` | Present with Responses and Chat selectors. | Needs package-parity review against the AI SDK xAI provider. |
| GitHub Copilot | `src/providers/github-copilot.ts` | Present as explicit-base-URL OpenAI Chat/Responses facade. | Runtime/catalog integration remains specialized and should stay separate from public OpenAI-compatible defaults. |
## V2 Runner Status
`packages/core/src/session/runner/model.ts` currently resolves only this native subset from catalog `aisdk` metadata:
| Catalog API | Native route used today |
| --------------------------------------------------- | ---------------------------- |
| `aisdk:@ai-sdk/openai` | `OpenAIResponses.route` |
| `aisdk:@ai-sdk/anthropic` | `AnthropicMessages.route` |
| `aisdk:@ai-sdk/openai-compatible` with explicit URL | `OpenAICompatibleChat.route` |
Other `aisdk:` packages, including Google Vertex, Azure, and Bedrock, currently fall back through the AI SDK loader in the production runner. The dependency-free resolver seam rejects them with `SessionRunnerModel.UnsupportedPackageError`; they are not native route mappings yet.
## AI SDK Package Parity Matrix
| AI SDK package | Intended native target | Status | Biggest gaps |
| --------------------------------- | --------------------------------------------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `@ai-sdk/openai` | `OpenAI.chat`, `OpenAI.responses` | Partial / usable | Add complete typed option coverage, structured output strategy, explicit Responses continuation support, and runner execution policy for optional WebSocket channels. |
| `@ai-sdk/openai-compatible` | Generic OpenAI-compatible Chat and Responses | Partial / usable | Decide per-family namespace/profile behavior and runner API selection for providers that support Responses versus Chat only. |
| `@ai-sdk/anthropic` | `AnthropicMessages` | Partial / usable | Finish Messages API parity for headers/betas/metadata/newer fields and document hosted-tool continuation expectations. |
| `@ai-sdk/google` | Gemini Developer API | Partial / usable | Add typed options for safety, response schema/modalities, cached content, grounding/search/code execution, and non-text output modes where supported. |
| `@ai-sdk/google-vertex` | Vertex Gemini namespace/facade | Partial / usable | Add runner/catalog mapping, recorded coverage, and broader provider-option parity. |
| `@ai-sdk/google-vertex/anthropic` | Anthropic Messages over Vertex namespace/facade | Partial / usable | Add runner/catalog mapping, recorded coverage, and Vertex-specific hosted-tool parity. |
| `@ai-sdk/google-vertex/maas` | Vertex Chat | Partial / usable | Add runner/catalog mapping, recorded coverage, and MaaS family-specific request parity. |
| `@ai-sdk/google-vertex/xai` | Vertex Chat / Responses | Partial / usable | Decide Chat/Responses selection for catalog models, add runner mapping and recorded coverage, and review xAI-specific request options. |
| `@ai-sdk/azure` | Azure OpenAI Chat/Responses facade | Partial | Map runner/catalog metadata to native Azure, handle resourceName/baseURL/apiVersion variants, add AAD/token auth story, and verify Chat vs Responses deployment selection. |
| `@ai-sdk/amazon-bedrock` | Bedrock Converse | Partial | Add default AWS credential chain/profile support, region/inference-profile model ID handling, provider option parity via `additionalModelRequestFields`, guardrails/performance config, and runner/catalog mapping. |
| `@ai-sdk/amazon-bedrock/mantle` | Bedrock Mantle OpenAI-compatible Chat/Responses namespace | Partial / usable | Add default AWS credential chain/profile support; native catalog mapping currently requires bearer auth or explicit static credentials. |
## Highest-Risk Gaps
1. Runner support is narrower than the LLM package. The package has native provider facades for Google, Azure, and Bedrock, but the V2 Session runner only maps OpenAI, Anthropic, and explicit OpenAI-compatible Chat from `aisdk` catalog metadata.
2. The Open Responses adapter is available through a separate package entrypoint, but the V2 runner still maps `@ai-sdk/openai-compatible` to Chat only. Catalog selection must become API-aware before Responses deployments can use it.
3. Bedrock native auth is not AI SDK parity. The AI SDK plugin uses the default AWS provider chain, profile, container credentials, and Bedrock bearer token env behavior. Native Bedrock currently expects explicit credentials or bearer auth on the facade.
4. Vertex Gemini, Vertex Chat, Vertex Responses, and Vertex Messages now have native package entrypoints, but the core runner does not map catalog metadata to them yet and recorded provider coverage is still missing.
5. Azure is only a provider facade, not a full runtime replacement. Native Azure exists, but the catalog runner does not select it, and token auth/resource variants need review.
6. Provider option typing is uneven. OpenAI, Anthropic, Gemini, Bedrock, and OpenRouter each expose a small typed subset plus raw HTTP overlays; this is useful but not equivalent to AI SDK provider option coverage.
7. Structured output is not provider-native yet. `LLM.generateObject` still uses a synthetic tool strategy, while the future design expects native structured output where reliable and tool fallback where needed.
8. Package/namespace boundaries for the current native loading set are explicit in docs and exports. Other exported provider facades are not catalog package entrypoints until they implement the contract. Vertex xAI still needs catalog API selection.
9. Recorded coverage is uneven. OpenAI, Anthropic, Gemini, Bedrock Converse, Bedrock Mantle, Cloudflare, OpenRouter, and several OpenAI-compatible Chat providers have cassettes. Azure and Vertex still need first-class recorded scenarios before switching defaults.
## Native Namespace Shape
These are implementation/API slices, not separate npm packages.
| API slice | Package-like entrypoint | Purpose |
| ----------------------------- | ----------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| OpenAI Chat | `@opencode-ai/ai/providers/openai/chat` | OpenAI `/chat/completions` semantics. |
| OpenAI Responses | `@opencode-ai/ai/providers/openai/responses` | OpenAI `/responses` semantics with HTTP default and optional per-call WebSocket execution. |
| OpenAI-compatible Chat | `@opencode-ai/ai/providers/openai-compatible` | Generic OpenAI-compatible `/chat/completions`. |
| Open Responses-compatible | `@opencode-ai/ai/providers/openai-compatible/responses` | Generic provider-neutral `/responses`. |
| Anthropic-compatible Messages | `@opencode-ai/ai/providers/anthropic-compatible` | Generic Anthropic-compatible `/messages`. |
| Anthropic Messages | `@opencode-ai/ai/providers/anthropic` | Anthropic Messages API. |
| Gemini Developer API | `@opencode-ai/ai/providers/google` | Google AI Studio Gemini API. |
| Vertex Gemini | `@opencode-ai/ai/providers/google-vertex/gemini` | Vertex Gemini API; `providers/google-vertex` is the default alias. |
| Vertex Chat | `@opencode-ai/ai/providers/google-vertex/chat` | Vertex OpenAI-compatible Chat Completions for MaaS models. |
| Vertex Responses | `@opencode-ai/ai/providers/google-vertex/responses` | Vertex Open Responses for Grok models. |
| Vertex Messages | `@opencode-ai/ai/providers/google-vertex/messages` | Vertex-hosted Anthropic Messages API. |
| Bedrock Converse | `@opencode-ai/ai/providers/amazon-bedrock` | AWS Bedrock Converse API. |
| Bedrock Mantle Chat | `@opencode-ai/ai/providers/amazon-bedrock/mantle/chat` | AWS Bedrock Mantle OpenAI-compatible Chat API. |
| Bedrock Mantle Responses | `@opencode-ai/ai/providers/amazon-bedrock/mantle/responses` | AWS Bedrock Mantle OpenAI-compatible Responses API. |
| Azure OpenAI Chat | `@opencode-ai/ai/providers/azure/chat` | Azure specialization of OpenAI Chat. |
| Azure OpenAI Responses | `@opencode-ai/ai/providers/azure/responses` | Azure specialization of OpenAI Responses. |
## Suggested Next Work Slices
1. Add native runner/catalog mappings for `@ai-sdk/azure`, `@ai-sdk/google`, and `@ai-sdk/amazon-bedrock` where the existing native facades are already close.
2. Add API-aware runner/catalog selection between OpenAI-compatible Chat and Responses.
3. Bring Bedrock native auth/config to AI SDK parity: region, profile, default AWS credential chain, bearer token env, endpoint override, and cross-region inference profile handling.
4. Add runner/catalog mappings and recorded scenarios for the native Vertex Gemini, Chat, Responses, and Messages entrypoints.
5. Decide Chat/Responses selection for `@ai-sdk/google-vertex/xai` catalog models.
6. Expand typed provider options from the existing V1 lowerer knowledge in `packages/core/src/v1/config/provider-options.ts` before adding more raw overlay examples.
7. Add recorded provider tests for Azure, Vertex Gemini, Vertex Chat, Vertex Responses, Vertex Messages, and Bedrock credential-chain behavior before making native runtime the default for those packages.
-606
View File
@@ -1,606 +0,0 @@
# LLM Call Site Sketches
Scratchpad for examples first, abstractions second. Current direction: routes
execute, provider facades organize configured route sets, and models carry route
values directly.
## Conversation Summary
Kit and Aidan want provider-specific LLM behavior to move out of opencode's AI
SDK transform path and into `packages/ai` where possible. The goal is not a big
generic transform layer; the goal is small composable route definitions backed by
recorded golden tests.
Things to keep testing against:
- Cache placement: `cache: "auto"`, manual cache breakpoints, provider cache usage.
- Images: golden image tests for providers/protocols that claim image support.
- Reasoning: canonical reasoning parts/events versus provider-native knobs.
- Auth: bearer, custom headers, multiple credentials, query auth, SigV4, OAuth, no auth.
- OpenAI-compatible providers: DeepSeek, Together, Groq, Alibaba/DashScope, custom routers.
- Provider switching: stale signatures, encrypted reasoning, provider metadata, incompatible parts.
- Error quality: typed errors instead of generic SDK/server failures.
## Final Guide: Routes Execute, Providers Organize
Do not introduce a first-class `Deployment` abstraction unless it gains real
semantics. Provider facades are ergonomic configured route groups, not execution
registries. The executable/composable thing is still a route. Do not make route
construction publish to a global registry; models should carry their route value
directly.
Keep durable identity separate from runtime capability:
- Durable identity is small serializable data like `{ providerID, modelID }` for
config, sessions, logs, and catalogs.
- Runtime capability is a `LanguageModel` with a route value, protocol, transport, auth,
and defaults. It is allowed to contain functions and schemas.
- If persisted identity needs to become executable, resolve it through an app
boundary first. Do not make `LLMRequest` recover behavior from a global route
side table.
Keep unconfigured behavior values as values, not factories. A transport like
`HttpTransport.sseJson` should be a reusable immutable value. Use a function only
when the caller supplies options or when construction needs fresh state.
Use constants to remove repetition before inventing abstractions. Provider ids
are branded once per provider facade and reused across routes; a plain exported
object is enough for the provider-facing API unless a helper earns its keep by
removing repeated route projection.
Expose default configured provider instances, and put provider-specific setup on
`.configure(...)`. Model selectors stay pure: `model(id)`, `responses(id)`,
`chat(id)`, etc. Endpoint/auth/resource/api-version configuration happens before
model selection, not as a second argument to model selection.
Use provider/product facades consistently:
- One coherent provider/product config surface gets one top-level facade.
- APIs/model kinds that share that config are methods on the facade.
- Different products with different required config get separate top-level
facades, not a shared namespace with unrelated children.
- Default facades are exposed only when concrete defaults or lazy env/credential
defaults make the facade valid.
Examples:
```ts
OpenAI.responses("gpt-4o")
OpenAI.chat("gpt-4o")
Azure.configure({ resourceName, apiKey }).responses("my-deployment")
AmazonBedrock.configure({ region, credentials }).model("anthropic.claude-3-5-sonnet-20241022-v2:0")
CloudflareAIGateway.configure({ accountId, gatewayId, gatewayApiKey, apiKey }).model("openai/gpt-4o")
CloudflareWorkersAI.configure({ accountId, apiKey }).model("@cf/meta/llama-3.1-8b-instruct")
OpenAICompatible.configure({
provider: "custom",
baseURL: "https://custom.example/v1",
auth: Auth.bearer(apiKey),
}).model("custom-model")
```
Standardize the provider facade contract before abstracting construction. A
plain object is enough at first; add a helper only if repeated route projection
starts hiding the real provider-specific config.
`Route.with(...)` patch semantics should be boring and explicit:
- Omitted fields inherit from the original route.
- `endpoint` patches merge with the existing endpoint, so overriding `baseURL`
keeps the existing `path`.
- `endpoint.query` merges by default; later values win.
- `auth` replaces.
- `headers` merge by default; undefined values are omitted.
- `id` is optional in patches. Route ids are diagnostic/provider API labels, not
global runtime registry keys.
1. **Route**
- route id
- provider id
- protocol
- body schema
- body builder
- stream event schema
- parser/state machine
- transport
- method / IO shape
- framing
- request preparation
- constants when unconfigured; functions only when configured
- endpoint
- base URL
- static path
- body/model-derived path
- query params
- auth
- bearer
- custom header
- multiple credentials
- SigV4
- none
- defaults
- headers
- generation defaults
- provider options
- limits
2. **Provider Facade**
- default configured provider instance
- provider-specific `.configure(...)`
- plain object/function facade over one or more routes
- top-level export only when it represents one coherent config surface
- no passive `Provider.make(...)` wrapper unless it gains runtime behavior
3. **Model Selector**
- route/provider-owned selector
- accepts model id only
- returns executable models
- does not accept endpoint/auth/deployment overrides
4. **Language Model**
- model id
- route value
- provider id
- configured route value at selection time
5. **LLM Request**
- model
- messages/tools
- generation/cache/reasoning/response-format options
- request-level HTTP overlays for per-request headers/query/body additions,
not provider endpoint/auth reconfiguration
6. **Compile**
- read route from model
- merge route defaults and request overrides
- build final URL from route endpoint
- apply auth from the configured route
- build body with protocol
- execute with transport and parse with protocol
## Provider Facade Shape
The provider abstraction is a facade over configured routes, not the runtime
execution mechanism:
```ts
type ProviderFacade<APIs, Config> = {
readonly id: ProviderID
readonly model: (id: string) => LanguageModel
readonly configure: (input?: Config) => ProviderFacade<APIs, Config>
} & APIs
```
Manual construction is fine and should be the default until duplication earns a
helper:
```ts
export const OpenAI = {
id: openAIProvider,
model: openAIResponses.model,
responses: openAIResponses.model,
chat: openAIChat.model,
configure: configureOpenAI,
} satisfies ProviderFacade<
{
responses: (id: string) => LanguageModel
chat: (id: string) => LanguageModel
},
OpenAIConfig
>
```
If several providers repeat the same projection from route values to model
methods, the helper can stay deliberately tiny:
```ts
const configureOpenAI = (input: OpenAIConfig = {}) =>
Provider.define({
id: openAIProvider,
routes: {
responses: openAIResponses.with(openAIConfig(input)),
chat: openAIChat.with(openAIConfig(input)),
},
default: "responses",
configure: configureOpenAI,
})
export const OpenAI = configureOpenAI()
```
`Provider.define(...)` would only project route methods and preserve types:
```ts
OpenAI.model("gpt-4o")
OpenAI.responses("gpt-4o")
OpenAI.chat("gpt-4o")
OpenAI.configure({ apiKey }).responses("gpt-4o")
```
It must not register routes, select routes dynamically, or participate in
execution. Execution still reads the route value carried by the model.
## Ideal Call Sites
Define concrete routes for a native provider, then project them through a
provider facade:
```ts
const openAIProvider = ProviderID.make("openai")
const openAIResponses = Route.make({
id: "openai-responses",
provider: openAIProvider,
protocol: OpenAIResponses.protocol,
transport: HttpTransport.sseJson,
endpoint: {
baseURL: "https://api.openai.com/v1",
path: "/responses",
},
auth: Auth.envBearer("OPENAI_API_KEY"),
})
const openAIChat = Route.make({
id: "openai-chat",
provider: openAIProvider,
protocol: OpenAIChat.protocol,
transport: HttpTransport.sseJson,
endpoint: {
baseURL: "https://api.openai.com/v1",
path: "/chat/completions",
},
auth: Auth.envBearer("OPENAI_API_KEY"),
})
const openAIConfig = (input: OpenAIConfig) => ({
endpoint: input.endpoint,
auth: input.auth ?? (input.apiKey ? Auth.bearer(input.apiKey) : undefined),
headers: {
"OpenAI-Organization": input.organization,
"OpenAI-Project": input.project,
},
})
const configureOpenAI = (input: OpenAIConfig = {}) => {
const responses = openAIResponses.with(openAIConfig(input))
const chat = openAIChat.with(openAIConfig(input))
return {
id: openAIProvider,
responses: responses.model,
chat: chat.model,
model: responses.model,
configure: configureOpenAI,
}
}
export const OpenAI = configureOpenAI()
```
Specialize it functionally for concrete providers:
```ts
const deepSeekProvider = ProviderID.make("deepseek")
const deepseekChat = openAIChat.with({
id: "deepseek-chat",
provider: deepSeekProvider,
endpoint: {
baseURL: "https://api.deepseek.com/v1",
},
auth: Auth.envBearer("DEEPSEEK_API_KEY"),
})
const configureDeepSeek = (input: OpenAICompatibleConfig = {}) => {
const route = deepseekChat.with({
endpoint: input.endpoint,
auth: input.auth ?? (input.apiKey ? Auth.bearer(input.apiKey) : undefined),
})
return {
id: deepSeekProvider,
model: route.model,
configure: configureDeepSeek,
}
}
export const DeepSeek = {
id: deepSeekProvider,
model: deepseekChat.model,
configure: configureDeepSeek,
}
```
Provider-specific configuration happens before model selection:
```ts
const deepseek = DeepSeek.configure({
endpoint: {
baseURL: "https://proxy.example.com/v1",
},
auth: Auth.bearer(apiKey),
})
const model = deepseek.model("deepseek-chat")
```
Final request call site stays boring:
```ts
const response =
yield *
LLM.generate(
LLM.request({
model: DeepSeek.model("deepseek-chat"),
prompt: "Hello.",
}),
)
```
For direct provider-facade calls, Responses has one semantic model and route:
```ts
OpenAI.responses("gpt-4o")
```
The package-like OpenAI Responses entrypoint has the same transport-neutral
`model(...)` contract:
```ts
import { model } from "@opencode-ai/ai/providers/openai/responses"
model("gpt-4o", { apiKey })
```
Vertex keeps Gemini, Chat, Responses, and Messages as separate package-like entrypoints,
while sharing project/location resolution and ADC authentication internally:
```ts
import { model } from "@opencode-ai/ai/providers/google-vertex/gemini"
model("gemini-3.5-flash", { project, location: "global" })
```
```ts
import { model } from "@opencode-ai/ai/providers/google-vertex/chat"
model("deepseek-ai/deepseek-v3.2-maas", { project, location: "global" })
```
```ts
import { model } from "@opencode-ai/ai/providers/google-vertex/responses"
model("xai/grok-4.20-reasoning", { project, location: "global" })
```
```ts
import { model } from "@opencode-ai/ai/providers/google-vertex/messages"
model("claude-sonnet-4-6", { project, location: "global" })
```
The client does not require a different public layer for WebSocket execution.
Responses routes use HTTP by default, and callers may pass a channel executor per
call. Routes without channel support simply ignore that execution capability.
Azure is a route specialization with auth/path/default changes plus input
mapping. The public API configures the Azure resource once, then selects
deployment ids with pure model selectors:
```ts
const azureProvider = ProviderID.make("azure")
const azureResponses = openAIResponses.with({
id: "azure-openai-responses",
provider: azureProvider,
auth: Auth.envHeader("api-key", "AZURE_OPENAI_API_KEY"),
})
const configureAzure = (input: AzureConfig = {}) => {
const route = azureResponses.with({
endpoint: {
baseURL:
input.baseURL ??
Endpoint.envBaseURL(
"AZURE_RESOURCE_NAME",
(resourceName) => `https://${resourceName}.openai.azure.com/openai/v1`,
),
query: { "api-version": input.apiVersion ?? "v1" },
},
auth: input.apiKey ? Auth.header("api-key", input.apiKey) : Auth.envHeader("api-key", "AZURE_OPENAI_API_KEY"),
})
return {
id: azureProvider,
model: route.model,
responses: route.model,
configure: configureAzure,
}
}
export const Azure = configureAzure()
const azure = Azure.configure({
resourceName: "my-resource",
apiVersion: "v1",
})
const model = azure.responses("my-deployment")
```
Default provider facades are only valid when required configuration has a lazy
default source. `Azure.responses("my-deployment")` can be valid if endpoint
resolution reads `AZURE_RESOURCE_NAME` lazily and fails with a typed
configuration error when missing. If a provider has no sensible lazy default,
do not expose a default model selector; expose only a configured entrypoint.
Cloudflare AI Gateway and Workers AI are separate product facades because their
configuration surfaces differ. Do not make a root `Cloudflare.configure(...)`
pretend there is one coherent Cloudflare provider configuration:
```ts
const cloudflareProvider = ProviderID.make("cloudflare-ai-gateway")
const cloudflareOpenAIChat = openAIChat.with({
id: "cloudflare-ai-gateway-openai-chat",
provider: cloudflareProvider,
auth: Auth.bearerHeader("cf-aig-authorization").andThen(Auth.bearer()),
})
const configureCloudflareAIGateway = (input: CloudflareAIGatewayConfig) => {
const route = cloudflareOpenAIChat.with({
endpoint: {
baseURL: `https://gateway.ai.cloudflare.com/v1/${input.accountId}/${input.gatewayId}/openai`,
},
auth: Auth.bearerHeader("cf-aig-authorization", input.gatewayApiKey).andThen(Auth.bearer(input.apiKey)),
})
return {
id: cloudflareProvider,
model: (modelID: string) => route.model({ id: modelID }),
configure: configureCloudflareAIGateway,
}
}
export const CloudflareAIGateway = {
id: cloudflareProvider,
configure: configureCloudflareAIGateway,
}
const gateway = CloudflareAIGateway.configure({
accountId: "account",
gatewayId: "gateway",
gatewayApiKey,
apiKey,
})
const model = gateway.model("openai/gpt-4o")
```
If a Cloudflare product gains a full lazy env default, it can expose a direct
selector too. Until then, omitting `CloudflareAIGateway.model(...)` makes missing
account/gateway configuration unrepresentable.
opencode's dynamic runtime should construct executable models at its app
boundary instead of exposing a giant unstructured public model constructor or a
generic dynamic resolver:
```ts
const model =
providerID === "azure" ? Azure.configure(resolvedAzureConfig).responses(apiModelID) : OpenAI.responses(apiModelID)
```
That boundary can branch on durable config/catalog metadata and call typed
provider APIs directly. Transport selection remains execution policy: a Session
or other caller may pass a WebSocket channel executor per call without changing
the model constructed by this boundary.
## Competitive Shape
This follows the strongest parts of adjacent libraries:
- AI SDK: configured provider instances expose provider-specific model methods.
- Effect AI: executable models carry provider requirements and can be resolved by
an app boundary.
- LiteLLM/opencode config: dynamic `providerID/modelID` branching belongs at the
app boundary, not in the typed public provider API or a global runtime
resolver.
- LangChain/LlamaIndex: constructor-style config plus model id is convenient,
but we avoid making model selection also configure endpoint/auth.
The chosen split is:
```txt
Route = execution mechanics
Provider facade = configured route group
LanguageModel = selected executable model carrying route value
App boundary = explicit durable-config -> typed-provider call
```
## What This Removes
- No `Provider.make(...)` as a core abstraction.
- No `Provider.make(...)` wrapper just to bind an id to model functions. Use a
branded provider id constant and a plain exported provider facade.
- No `Deployment.define(...)` unless future examples force it.
- No global route registry as the normal execution path.
- No import side effects required before a model can execute.
- No duplicate `provider.id` object when selected models already carry provider
id.
- No `model(id, overrides)` escape hatch. Model selection takes the model id;
endpoint/auth/deployment customization happens by configuring the route first.
- No transport setting on a provider or executable model. OpenAI Responses uses
HTTP by default and accepts an optional per-call channel executor as execution policy.
- No separate public `LLMClient.layerWithWebSocket`. The runtime should expose one
client layer with the available transport capabilities.
- No executable `ModelRef`. The executable handle is `LanguageModel`; durable model
identity stays separate and cannot execute on its own.
## Implementation Todo
- [x] Replace the current executable `ModelRef` with `LanguageModel`.
- [x] Change `LanguageModel.route` to carry a route value, not a `RouteID` string.
- [ ] Keep a separate durable model identity type for persisted/session/catalog
data, likely `{ providerID, modelID }`, and make it clear that it cannot
execute without resolver context.
- [x] Change route model selectors so `route.model(id)` returns an executable
model with the route value attached, not a globally registered route id.
- [x] Remove the standalone `Route.model(route, defaults, mapInput)` helper;
configured route instances own model selection.
- [x] Remove endpoint/auth escape hatches from route model selection; callers must
configure endpoint/auth through `route.with(...)` or provider facades before
calling `.model(...)`.
- [x] Remove request-shaping defaults from `LanguageModel`; selected models now carry only
id, provider, and configured route while defaults live on routes or requests.
- [x] Rework `LLMClient.stream` / `generate` to read
`request.model.route` directly instead of calling `registeredRoute(...)`.
- [x] Remove `Route.make(...)` global registration from the normal execution
path; keep route ids only as diagnostics/provider API labels.
- [x] Model endpoint as `{ baseURL, path, query }` on routes, then remove the
current split where host/query live on the model and path lives in route
transport setup.
- [x] Define `Route.with(...)` with explicit patch semantics for endpoint merge,
query merge, header merge, auth replacement, and optional diagnostic id.
- [x] Make unconfigured transports reusable constants such as
`HttpTransport.sseJson`; keep transport functions only for configured/fresh
state construction.
- [x] Collapse the public WebSocket runtime split so one `LLMClient.layer` accepts
optional per-call channel execution without changing route identity.
- [x] Convert OpenAI provider APIs to provider-facade shape:
`OpenAI.configure(config).responses(id)` and `.chat(id)`.
- [x] Convert Azure to a configured facade where resource/base URL/api version
setup happens before selecting deployment ids.
- [x] Split Cloudflare products into separate facades such as
`CloudflareAIGateway` and `CloudflareWorkersAI`; do not expose a shared root
config surface unless one product actually exists.
- [x] Migrate remaining built-in provider facades one at a time so configuration
happens before model selection and selectors accept only ids:
xAI, GitHub Copilot, OpenRouter, OpenAI-compatible families, Anthropic,
Google/Gemini, and Amazon Bedrock now use configured facades such as
`Provider.configure(options).model(id)` with named selectors where needed.
- [ ] Decide whether a tiny `Provider.define(...)` helper is warranted after two
or three provider conversions; start with plain objects if duplication is not
yet painful.
- [x] Keep executable model construction transport-neutral at the Session boundary;
Session-scoped execution policy supplies channel capability separately.
- [ ] Update tests so direct route/provider tests assert route values are carried
by executable models, and opencode/native tests assert boundary-based route
selection.
- [ ] Remove compatibility exports or stale docs only after internal call sites
are migrated; do not keep duplicate constructor paths without an external
compatibility need.
## Open Questions
- Default facades with required setup: should providers like Azure and Bedrock
expose default model selectors only when all required setup has lazy env or
credential-chain defaults? If not, omit the default selector so missing config
is impossible at the type/API level.
- Lazy endpoint/auth values: should `Endpoint.envBaseURL(...)` and env-backed
auth produce typed configuration/authentication errors at compile/prepare time
or only when executing the transport?
- `Route.with(...)` clearing semantics: endpoint/query/header patches merge by
default, but what is the explicit way to remove an inherited value?
- Provider facade helper: keep plain objects until duplication hurts, or add a
tiny `Provider.define(...)` immediately to enforce shape and method projection?
- Auth shape: should auth stay as today's composable `Auth`, or split into an
auth placement/strategy and credential sources?
- Naming: is `baseURL` still the right endpoint field name, or should it be
`origin` / `urlPrefix` to clarify that route `path` is appended?
+8 -10
View File
@@ -17,13 +17,11 @@ import { OpenAI } from "@opencode-ai/ai/providers"
const apiKey = Config.redacted("OPENAI_API_KEY")
// 1. Pick a model. The provider helper records provider identity, protocol
// choice, capabilities, deployment options, authentication, and defaults.
// choice, deployment options, authentication, and defaults. Catalog capabilities
// remain application-owned and are not part of LanguageModel.
const model = OpenAI.configure({
apiKey,
generation: { maxTokens: 160 },
providerOptions: {
openai: { store: false },
},
}).model("gpt-4o-mini")
// 2. Build a provider-neutral request. This is useful when reusing one request
@@ -34,7 +32,7 @@ const model = OpenAI.configure({
// - `generation`: common controls such as max tokens, temperature, topP/topK,
// penalties, seed, and stop sequences.
// - `promptCacheKey`: stable cache affinity for protocols that support it.
// - `providerOptions`: namespaced provider-native behavior. For example,
// - `providerOptions`: model-typed provider-native behavior. For example,
// OpenAI store behavior, Anthropic thinking, Gemini thinking config, or
// OpenRouter routing/reasoning.
// - `http`: last-resort serializable overlays for final request body, headers,
@@ -74,8 +72,8 @@ const streamText = LLM.stream(request).pipe(
Stream.runDrain,
)
// 5. Tools are typed with Effect Schema. Provider turns remain explicit:
// advertise definitions on the request, stream one turn, dispatch local calls,
// 5. Tools are typed with Effect Schema. Model calls remain explicit:
// advertise definitions on the request, stream one call, dispatch local calls,
// then persist/build follow-up history in the enclosing product flow.
const tools = {
get_weather: Tool.make({
@@ -102,7 +100,7 @@ const streamWithTools = Effect.gen(function* () {
console.log("tool result", event.name, dispatched.result)
// A durable agent would persist these messages before starting another
// raw model turn. This tutorial keeps the boundary visible instead.
// model call. This tutorial keeps the boundary visible instead.
const followUp = LLMRequest.update(request, {
messages: [
...request.messages,
@@ -188,14 +186,14 @@ const FakeProtocol = Protocol.make<FakeBody, string, string, void>({
},
})
// An route is the runnable binding for that protocol. It adds the deployment
// A route is the runnable binding for that protocol. It adds the deployment
// axes that the protocol deliberately does not know: URL, auth, and framing.
const FakeAdapter = Route.make({
id: "fake-echo",
provider: "fake-echo",
protocol: FakeProtocol,
endpoint: Endpoint.path("/v1/echo", { baseURL: "https://fake.local" }),
auth: Auth.passthrough,
auth: Auth.none,
framing: Framing.sse,
})
+3 -5
View File
@@ -5,8 +5,8 @@
// The default `"auto"` shape places breakpoints at the last tool definition,
// the first and last distinct system parts, and the conversation tail. This
// exposes reusable tool, base-agent, project, and session prefixes while
// advancing the tail after each tool result keeps the previous cache entry
// within Anthropic's 20-block lookback during long agent turns.
// advancing the tail after each tool result keeps recent conversation prefixes
// reusable during long agent runs.
//
// Manual `cache: CacheHint` placements on individual parts are preserved and
// count against the four-breakpoint budget; auto only fills remaining slots.
@@ -23,9 +23,7 @@ const NONE: CachePolicyObject = {}
const BREAKPOINT_CAP = 4
// Resolution rules:
// - undefined → "auto" — caching is on by default. The math favors it:
// Anthropic 5m-cache write is 1.25x base, read is 0.1x,
// so a single reuse within 5 minutes already wins.
// - undefined → "auto" — caching is on by default.
// - "auto" → tools + first/last system + final message boundary.
// - "none" → no auto placement; manual `CacheHint`s still flow.
// - object form → exactly what the caller asked for.
+3
View File
@@ -37,6 +37,9 @@ export type {
LanguageModelOptions as ProviderLanguageModelOptions,
} from "./provider.js"
export type {
Credential as ProviderPackageCredential,
Defaults as ProviderPackageDefaults,
Definition as ProviderPackageDefinition,
ModelInput as ProviderPackageModelInput,
Settings as ProviderPackageSettings,
} from "./provider-package.js"
@@ -16,7 +16,6 @@ import {
type JsonSchema,
type LLMRequest,
type MediaPart,
type ProviderOptions,
type ProviderMetadata,
type ToolCallPart,
type ToolDefinition,
@@ -52,9 +51,7 @@ export interface OptionsInput {
readonly effort?: string
}
export type ProviderOptionsInput = ProviderOptions & {
readonly anthropic?: OptionsInput
}
export type ProviderOptionsInput = OptionsInput
// =============================================================================
// Request Body Schema
@@ -593,7 +590,7 @@ const lowerMessages = Effect.fn("AnthropicMessages.lowerMessages")(function* (
})
const resolveOptions = Effect.fn("AnthropicMessages.resolveOptions")(function* (request: LLMRequest) {
const input = request.providerOptions?.anthropic
const input = request.providerOptions
return {
thinking: yield* resolveThinking(input?.thinking),
effort: typeof input?.effort === "string" ? input.effort : undefined,
@@ -1015,8 +1012,8 @@ const step = (state: ParserState, event: AnthropicEvent) => {
// =============================================================================
/**
* The Anthropic Messages protocol — request body construction, body schema,
* and the streaming-event state machine. Used by native Anthropic Cloud and
* (once registered) Vertex Anthropic / Bedrock-hosted Anthropic passthrough.
* and the streaming-event state machine shared by Anthropic-compatible and
* Vertex-hosted Messages routes.
*/
export const protocol = Protocol.make({
id: ADAPTER,
+3 -7
View File
@@ -12,7 +12,6 @@ import {
type JsonSchema,
type LLMRequest,
type MediaPart,
type ProviderOptions,
type ProviderMetadata,
type TextPart,
type ToolCallPart,
@@ -67,9 +66,7 @@ export interface OptionsInput {
}
}
export type ProviderOptionsInput = ProviderOptions & {
readonly gemini?: OptionsInput
}
export type ProviderOptionsInput = OptionsInput
// =============================================================================
// Request Body Schema
@@ -387,7 +384,7 @@ const lowerMessages = Effect.fn("Gemini.lowerMessages")(function* (request: LLMR
})
const resolveOptions = (request: LLMRequest) => {
const input = request.providerOptions?.gemini
const input = request.providerOptions
const value = input?.thinkingConfig
const thinkingConfig = {
thinkingBudget:
@@ -630,8 +627,7 @@ const step = (state: ParserState, event: GeminiEvent) => {
// =============================================================================
/**
* The Gemini protocol — request body construction, body schema, and the
* streaming-event state machine. Used by Google AI Studio Gemini and (once
* registered) Vertex Gemini.
* streaming-event state machine shared by Google AI Studio and Vertex Gemini.
*/
export const protocol = Protocol.make({
id: ADAPTER,
+1 -1
View File
@@ -340,7 +340,7 @@ export const lowerTool = Effect.fn("OpenResponses.lowerTool")(function* (
name: tool.name,
description: tool.description,
parameters: ToolSchemaProjection.responses(inputSchema),
// TODO: Read this from Responses tool options so direct LLM callers can opt into strict schemas.
// The common tool definition does not currently express Responses strict-schema policy.
strict: false,
}
})
@@ -261,7 +261,7 @@ export const route = Route.make({
endpoint,
auth,
transport,
defaults: { providerOptions: { openai: { store: false } } },
defaults: { providerOptions: { store: false } },
})
export * as OpenAIResponses from "./openai-responses.js"
+8 -14
View File
@@ -41,12 +41,10 @@ export interface ToolAccumulator {
* when at least one is defined. Returns `undefined` when neither input nor
* output is known so routes don't publish a misleading `0`.
*
* Under the additive `AI.Usage` contract, `inputTokens` and `outputTokens`
* are the non-cached input and visible output only. The provider-supplied
* `total` is the source of truth when present; the computed fallback
* under-counts cache and reasoning by design and exists mainly so
* Anthropic-style providers (which don't surface a total) still get a
* sensible aggregate on the input + output axes.
* Under the inclusive `AI.Usage` contract, `inputTokens` includes cached input
* and `outputTokens` includes reasoning. Protocol mappers normalize those
* inclusive values before calling this helper. The provider-supplied total is
* the source of truth when present; otherwise their sum is the canonical total.
*/
export const totalTokens = (
inputTokens: number | undefined,
@@ -67,7 +65,7 @@ export const totalTokens = (
*
* If `total` is `undefined`, returns `undefined` (we don't fabricate
* counts). If `subtrahend` is `undefined`, returns `total` unchanged. The
* provider-native breakdown stays available on `Usage.native` for debugging.
* provider-native breakdown stays available on `Usage.providerMetadata` for debugging.
*/
export const subtractTokens = (total: number | undefined, subtrahend: number | undefined): number | undefined => {
if (total === undefined) return undefined
@@ -199,8 +197,8 @@ export const errorText = (error: unknown) => {
/**
* `framing` step for Server-Sent Events. Decodes UTF-8, runs the SSE channel
* decoder, and drops empty / `[DONE]` keep-alive events so the downstream
* `decodeChunk` sees one JSON string per element. The SSE channel emits a
* decoder, and drops empty / `[DONE]` keep-alive events so the protocol event
* schema sees one JSON string per element. The SSE channel emits a
* `Retry` control event on its error channel; we drop it here (we don't
* implement client-driven retries). Decoder failures become provider output
* errors so the public error channel stays `AIError`.
@@ -216,11 +214,7 @@ export const sseFraming = (bytes: Stream.Stream<Uint8Array, AIError>): Stream.St
)
/**
* Canonical invalid-request constructor. Lift one-line `const invalid =
* (message) => invalidRequest(message)` aliases out of every
* route so the error constructor lives in one place. If we ever extend
* `InvalidRequestReason` with route context or trace metadata, the change
* lands here.
* Canonical invalid-request constructor shared by protocol lowering.
*/
export const invalidRequest = (message: string) =>
new AIError({
@@ -4,7 +4,7 @@ import { newBreakpoints, ttlBucket, type Breakpoints } from "./cache.js"
// Bedrock cache markers are positional: emit a `cachePoint` block immediately
// after the content the caller wants treated as a cacheable prefix. Bedrock
// accepts optional `ttl: "5m" | "1h"` on cachePoint, mirroring Anthropic.
// accepts optional `ttl: "5m" | "1h"` on cachePoint.
export const CachePointBlock = Schema.Struct({
cachePoint: Schema.Struct({
type: Schema.tag("default"),
@@ -13,9 +13,8 @@ export const CachePointBlock = Schema.Struct({
})
export type CachePointBlock = Schema.Schema.Type<typeof CachePointBlock>
// Bedrock-Claude enforces the same 4-breakpoint cap as the Anthropic Messages
// API. Callers pass a shared counter through every `block()` call site so the
// budget is respected across `system`, `messages`, and `tools`.
// Callers pass a shared counter through every `block()` call site so the
// four-breakpoint budget is respected across `system`, `messages`, and `tools`.
export const BEDROCK_BREAKPOINT_CAP = 4
export type { Breakpoints } from "./cache.js"
+3 -6
View File
@@ -1,6 +1,4 @@
// Shared helpers for provider cache-marker lowering. Anthropic and Bedrock
// both enforce a 4-breakpoint cap per request and accept the same `5m`/`1h`
// TTL buckets, so the counter and TTL mapping live here.
// Shared counter and TTL mapping for provider cache-marker lowering.
export interface Breakpoints {
remaining: number
@@ -9,8 +7,7 @@ export interface Breakpoints {
export const newBreakpoints = (cap: number): Breakpoints => ({ remaining: cap, dropped: 0 })
// Returns `"1h"` for any `ttlSeconds >= 3600`, otherwise `undefined` (the
// provider default 5m). Anthropic & Bedrock both treat anything shorter than
// an hour as 5m.
// Requests of at least one hour use the explicit `"1h"` bucket; shorter
// requests omit the wire TTL and use the provider default.
export const ttlBucket = (ttlSeconds: number | undefined): "1h" | undefined =>
ttlSeconds !== undefined && ttlSeconds >= 3600 ? "1h" : undefined
@@ -56,9 +56,7 @@ export type Resolved = Omit<Options, "allowedTools"> & {
const decodeOptions = Schema.decodeUnknownOption(Options)
export const resolve = (request: LLMRequest): Resolved => {
const input = Option.getOrUndefined(
decodeOptions(request.providerOptions?.[request.model.route.providerMetadataKey ?? "openresponses"]),
)
const input = Option.getOrUndefined(decodeOptions(request.providerOptions))
if (!input) return {}
return {
...input,
+42 -3
View File
@@ -1,7 +1,21 @@
import { Auth } from "./route/auth.js"
import type { AuthOverride, RequiredApiKeyAuth } from "./route/auth-options.js"
import type { LanguageModel, ProviderOptions } from "./schema/index.js"
export interface Settings extends Readonly<Record<string, unknown>> {
readonly baseURL?: string
export interface Settings {}
export type Credential =
| {
readonly type: "key"
readonly value: string
readonly configuration?: Readonly<Record<string, unknown>>
}
| {
readonly type: "oauth"
readonly accessToken: string
}
export interface Defaults {
readonly headers?: Readonly<Record<string, string>>
readonly body?: Readonly<Record<string, unknown>>
readonly limits?: {
@@ -11,11 +25,36 @@ export interface Settings extends Readonly<Record<string, unknown>> {
}
}
export interface ModelInput<ProviderSettings extends Settings = Settings> {
readonly id: string
readonly settings: ProviderSettings
readonly credential?: Credential
readonly defaults: Defaults
}
export const routeDefaults = (input: Defaults) => ({
headers: input.headers,
http: input.body === undefined ? undefined : { body: input.body },
limits: input.limits,
})
export const bearerCredentialValue = (input: Credential) => (input.type === "key" ? input.value : input.accessToken)
export const bearerAuthOption = (input: Credential): AuthOverride => ({ auth: Auth.bearer(bearerCredentialValue(input)) })
export const apiKeyOrBearerAuthOption = (
input: Credential,
competingKeyHeader: string,
): RequiredApiKeyAuth | AuthOverride =>
input.type === "key"
? { apiKey: input.value }
: { auth: Auth.remove(competingKeyHeader).andThen(Auth.bearer(input.accessToken)) }
export interface Definition<
ProviderSettings extends Settings = Settings,
Options extends ProviderOptions = ProviderOptions,
> {
readonly model: (modelID: string, settings: ProviderSettings) => LanguageModel<Options>
readonly model: (input: ModelInput<ProviderSettings>) => LanguageModel<Options>
}
export * as ProviderPackage from "./provider-package.js"
@@ -1,6 +1,6 @@
import { Auth } from "../route/auth.js"
import type { Route as RouteDef, RouteDefaultsInput } from "../route/client.js"
import type { ProviderPackage } from "../provider-package.js"
import { ProviderPackage } from "../provider-package.js"
import { OpenAIChat } from "../protocols/openai-chat.js"
import { OpenAIResponses } from "../protocols/openai-responses.js"
import { BedrockAuth, type Credentials } from "../protocols/utils/bedrock-auth.js"
@@ -79,29 +79,27 @@ export const configure = (input: Config = {}) => {
export const provider = configure()
const config = (settings: Settings): Config => {
if (settings.auth === "bearer" && settings.apiKey === undefined)
const config = (input: ProviderPackage.ModelInput<Settings>): Config => {
if (!input.credential && input.settings.auth === "bearer" && input.settings.apiKey === undefined)
throw new Error("Amazon Bedrock Mantle bearer auth requires apiKey")
if (settings.auth === "sigv4" && settings.apiKey !== undefined)
if (!input.credential && input.settings.auth === "sigv4" && input.settings.apiKey !== undefined)
throw new Error("Amazon Bedrock Mantle SigV4 auth does not accept apiKey")
return {
apiKey: settings.auth === "sigv4" ? undefined : settings.apiKey,
baseURL: settings.baseURL,
credentials: settings.credentials,
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
providerOptions: settings.providerOptions,
region: settings.region,
...ProviderPackage.routeDefaults(input.defaults),
apiKey: input.credential
? ProviderPackage.bearerCredentialValue(input.credential)
: input.settings.auth === "sigv4"
? undefined
: input.settings.apiKey,
baseURL: input.settings.baseURL,
credentials: input.settings.credentials,
providerOptions: input.settings.providerOptions,
region: input.settings.region,
}
}
export const chatModel: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (
modelID,
settings,
) => configure(config(settings)).chat(modelID)
export const responsesModel: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (
modelID,
settings,
) => configure(config(settings)).responses(modelID)
export const chatModel: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (input) =>
configure(config(input)).chat(input.id)
export const responsesModel: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (input) =>
configure(config(input)).responses(input.id)
export const model = chatModel
+15 -13
View File
@@ -1,6 +1,6 @@
import type { RouteDefaultsInput } from "../route/client.js"
import { Auth } from "../route/auth.js"
import type { ProviderPackage } from "../provider-package.js"
import { ProviderPackage } from "../provider-package.js"
import { ProviderID, type ModelID } from "../schema/index.js"
import * as BedrockConverse from "../protocols/bedrock-converse.js"
import type { BedrockCredentials } from "../protocols/bedrock-converse.js"
@@ -50,19 +50,21 @@ export const configure = (input: Config = {}) => {
}
export const provider = configure()
export const model: ProviderPackage.Definition<Settings>["model"] = (modelID, settings) => {
if (settings.auth === "bearer" && settings.apiKey === undefined)
export const model: ProviderPackage.Definition<Settings>["model"] = (input) => {
if (!input.credential && input.settings.auth === "bearer" && input.settings.apiKey === undefined)
throw new Error("Amazon Bedrock bearer auth requires apiKey")
if (settings.auth === "sigv4" && settings.apiKey !== undefined)
if (!input.credential && input.settings.auth === "sigv4" && input.settings.apiKey !== undefined)
throw new Error("Amazon Bedrock SigV4 auth does not accept apiKey")
return configure({
apiKey: settings.auth === "sigv4" ? undefined : settings.apiKey,
baseURL: settings.baseURL,
credentials: settings.credentials,
generation: settings.topP === undefined ? undefined : { topP: settings.topP },
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
region: settings.region,
}).model(modelID)
...ProviderPackage.routeDefaults(input.defaults),
apiKey: input.credential
? ProviderPackage.bearerCredentialValue(input.credential)
: input.settings.auth === "sigv4"
? undefined
: input.settings.apiKey,
baseURL: input.settings.baseURL,
credentials: input.settings.credentials,
generation: input.settings.topP === undefined ? undefined : { topP: input.settings.topP },
region: input.settings.region,
}).model(input.id)
}
@@ -1,4 +1,4 @@
import type { ProviderPackage } from "../provider-package.js"
import { ProviderPackage } from "../provider-package.js"
import { AnthropicMessages } from "../protocols/anthropic-messages.js"
import { Auth } from "../route/auth.js"
import type { ProviderAuthOption } from "../route/auth-options.js"
@@ -32,7 +32,9 @@ export const routes = [AnthropicMessages.route]
const auth = (input: ProviderAuthOption<"optional">) => {
if ("auth" in input && input.auth) return input.auth
return Auth.optional("apiKey" in input ? input.apiKey : undefined, "apiKey").pipe(Auth.header("x-api-key"))
return Auth.remove("authorization").andThen(
Auth.optional("apiKey" in input ? input.apiKey : undefined, "apiKey").pipe(Auth.header("x-api-key")),
)
}
export const configure = (input: Config) => {
@@ -57,21 +59,20 @@ export const provider = {
configure,
}
export const model: ProviderPackage.Definition<Settings, AnthropicMessages.ProviderOptionsInput>["model"] = (
modelID,
settings,
) => {
if (settings.apiKey !== undefined && settings.authToken !== undefined)
export const model: ProviderPackage.Definition<Settings, AnthropicMessages.ProviderOptionsInput>["model"] = (input) => {
if (!input.credential && input.settings.apiKey !== undefined && input.settings.authToken !== undefined)
throw new Error("Anthropic-compatible apiKey cannot be combined with authToken")
return configure({
...(settings.authToken === undefined ? { apiKey: settings.apiKey } : { auth: Auth.bearer(settings.authToken) }),
baseURL: settings.baseURL,
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
provider: settings.provider,
providerOptions: settings.providerOptions,
}).model(modelID)
...ProviderPackage.routeDefaults(input.defaults),
...(input.credential
? ProviderPackage.apiKeyOrBearerAuthOption(input.credential, "x-api-key")
: input.settings.authToken === undefined
? { apiKey: input.settings.apiKey }
: { auth: Auth.bearer(input.settings.authToken) }),
baseURL: input.settings.baseURL,
provider: input.settings.provider,
providerOptions: input.settings.providerOptions,
}).model(input.id)
}
export * as AnthropicCompatible from "./anthropic-compatible.js"
+17 -16
View File
@@ -1,7 +1,7 @@
import type { RouteDefaultsInput } from "../route/client.js"
import { Auth } from "../route/auth.js"
import type { ProviderAuthOption } from "../route/auth-options.js"
import type { ProviderPackage } from "../provider-package.js"
import { ProviderPackage } from "../provider-package.js"
import { ProviderID, type ModelID } from "../schema/index.js"
import { AnthropicMessages } from "../protocols/anthropic-messages.js"
import { AnthropicCompatible } from "./anthropic-compatible.js"
@@ -31,9 +31,11 @@ export type Settings = ProviderPackage.Settings &
const auth = (options: ProviderAuthOption<"optional">) => {
if ("auth" in options && options.auth) return options.auth
return Auth.optional("apiKey" in options ? options.apiKey : undefined, "apiKey")
.orElse(Auth.config("ANTHROPIC_API_KEY"))
.pipe(Auth.header("x-api-key"))
return Auth.remove("authorization").andThen(
Auth.optional("apiKey" in options ? options.apiKey : undefined, "apiKey")
.orElse(Auth.config("ANTHROPIC_API_KEY"))
.pipe(Auth.header("x-api-key")),
)
}
export const configure = (input: Config = {}) => {
@@ -52,18 +54,17 @@ export const configure = (input: Config = {}) => {
}
export const provider = configure()
export const model: ProviderPackage.Definition<Settings, AnthropicMessages.ProviderOptionsInput>["model"] = (
modelID,
settings,
) => {
if (settings.apiKey !== undefined && settings.authToken !== undefined)
export const model: ProviderPackage.Definition<Settings, AnthropicMessages.ProviderOptionsInput>["model"] = (input) => {
if (!input.credential && input.settings.apiKey !== undefined && input.settings.authToken !== undefined)
throw new Error("Anthropic apiKey cannot be combined with authToken")
return configure({
...(settings.authToken === undefined ? { apiKey: settings.apiKey } : { auth: Auth.bearer(settings.authToken) }),
baseURL: settings.baseURL,
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
providerOptions: settings.providerOptions,
}).model(modelID)
...ProviderPackage.routeDefaults(input.defaults),
...(input.credential
? ProviderPackage.apiKeyOrBearerAuthOption(input.credential, "x-api-key")
: input.settings.authToken === undefined
? { apiKey: input.settings.apiKey }
: { auth: Auth.bearer(input.settings.authToken) }),
baseURL: input.settings.baseURL,
providerOptions: input.settings.providerOptions,
}).model(input.id)
}
+17 -16
View File
@@ -1,7 +1,7 @@
import { Auth } from "../route/auth.js"
import { type AtLeastOne, type ProviderAuthOption } from "../route/auth-options.js"
import type { Route as RouteDef, RouteDefaultsInput } from "../route/client.js"
import type { ProviderPackage } from "../provider-package.js"
import { ProviderPackage } from "../provider-package.js"
import { ProviderID, type ModelID } from "../schema/index.js"
import * as OpenAIChat from "../protocols/openai-chat.js"
import * as OpenAIResponses from "../protocols/openai-responses.js"
@@ -120,28 +120,29 @@ export const provider = {
configure,
}
const config = (settings: Settings): Config => {
const config = (input: ProviderPackage.ModelInput<Settings>): Config => {
const settings = input.settings
const configuration = input.credential?.type === "key" ? input.credential.configuration : undefined
const baseURL = settings.baseURL ?? (typeof configuration?.baseURL === "string" ? configuration.baseURL : undefined)
const resourceName =
settings.resourceName ?? (typeof configuration?.resourceName === "string" ? configuration.resourceName : undefined)
const common = {
apiKey: settings.apiKey,
...ProviderPackage.routeDefaults(input.defaults),
...(input.credential
? ProviderPackage.apiKeyOrBearerAuthOption(input.credential, "api-key")
: { apiKey: settings.apiKey }),
apiVersion: settings.apiVersion,
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
providerOptions: settings.providerOptions,
queryParams: settings.queryParams === undefined ? undefined : { ...settings.queryParams },
useDeploymentBasedUrls: settings.useDeploymentBasedUrls,
}
if (settings.baseURL !== undefined) return { ...common, baseURL: settings.baseURL }
if (settings.resourceName !== undefined) return { ...common, resourceName: settings.resourceName }
if (baseURL !== undefined) return { ...common, baseURL }
if (resourceName !== undefined) return { ...common, resourceName }
throw new Error("Azure requires resourceName or baseURL")
}
export const responsesModel: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (
modelID,
settings,
) => configure(config(settings)).responses(modelID)
export const chatModel: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (
modelID,
settings,
) => configure(config(settings)).chat(modelID)
export const responsesModel: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (input) =>
configure(config(input)).responses(input.id)
export const chatModel: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (input) =>
configure(config(input)).chat(input.id)
export const model = responsesModel
+11 -12
View File
@@ -1,4 +1,4 @@
import type { ProviderPackage } from "../provider-package.js"
import { ProviderPackage } from "../provider-package.js"
import { OpenAICompatibleChat } from "../protocols/openai-compatible-chat.js"
import type { RouteDefaultsInput } from "../route/client.js"
import { ProviderID, type ModelID } from "../schema/index.js"
@@ -68,16 +68,15 @@ export const provider = {
configure,
}
export const model: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (modelID, settings) => {
if (settings.apiKey !== undefined) throw new Error("Google Vertex Chat does not support API keys")
export const model: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (input) => {
if (input.credential?.type === "key" || (!input.credential && input.settings.apiKey !== undefined))
throw new Error("Google Vertex Chat does not support API keys")
return configure({
accessToken: settings.accessToken,
baseURL: settings.baseURL,
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
location: settings.location,
project: settings.project,
providerOptions: settings.providerOptions,
}).model(modelID)
...ProviderPackage.routeDefaults(input.defaults),
accessToken: input.credential?.type === "oauth" ? input.credential.accessToken : input.settings.accessToken,
baseURL: input.settings.baseURL,
location: input.settings.location,
project: input.settings.project,
providerOptions: input.settings.providerOptions,
}).model(input.id)
}
@@ -1,5 +1,5 @@
import { Effect, Schema, Struct } from "effect"
import type { ProviderPackage } from "../provider-package.js"
import { ProviderPackage } from "../provider-package.js"
import { AnthropicMessages } from "../protocols/anthropic-messages.js"
import { Auth } from "../route/auth.js"
import { Route, type RouteDefaultsInput } from "../route/client.js"
@@ -100,19 +100,15 @@ export const provider = {
configure,
}
export const model: ProviderPackage.Definition<Settings, AnthropicMessages.ProviderOptionsInput>["model"] = (
modelID,
settings,
) => {
if (settings.apiKey !== undefined) throw new Error("Google Vertex Messages does not support API keys")
export const model: ProviderPackage.Definition<Settings, AnthropicMessages.ProviderOptionsInput>["model"] = (input) => {
if (input.credential?.type === "key" || (!input.credential && input.settings.apiKey !== undefined))
throw new Error("Google Vertex Messages does not support API keys")
return configure({
accessToken: settings.accessToken,
baseURL: settings.baseURL,
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
location: settings.location,
project: settings.project,
providerOptions: settings.providerOptions,
}).model(modelID)
...ProviderPackage.routeDefaults(input.defaults),
accessToken: input.credential?.type === "oauth" ? input.credential.accessToken : input.settings.accessToken,
baseURL: input.settings.baseURL,
location: input.settings.location,
project: input.settings.project,
providerOptions: input.settings.providerOptions,
}).model(input.id)
}
@@ -1,4 +1,4 @@
import type { ProviderPackage } from "../provider-package.js"
import { ProviderPackage } from "../provider-package.js"
import { OpenAICompatibleResponses } from "../protocols/openai-compatible-responses.js"
import type { RouteDefaultsInput } from "../route/client.js"
import { ProviderID, type ModelID } from "../schema/index.js"
@@ -27,7 +27,7 @@ export interface Settings extends ProviderPackage.Settings {
const route = OpenAICompatibleResponses.route.with({
id: "google-vertex-responses",
provider: id,
providerOptions: { openresponses: { store: false } },
providerOptions: { store: false },
})
export const routes = [route]
@@ -70,19 +70,15 @@ export const provider = {
configure,
}
export const model: ProviderPackage.Definition<Settings, OpenResponsesProviderOptionsInput>["model"] = (
modelID,
settings,
) => {
if (settings.apiKey !== undefined) throw new Error("Google Vertex Responses does not support API keys")
export const model: ProviderPackage.Definition<Settings, OpenResponsesProviderOptionsInput>["model"] = (input) => {
if (input.credential?.type === "key" || (!input.credential && input.settings.apiKey !== undefined))
throw new Error("Google Vertex Responses does not support API keys")
return configure({
accessToken: settings.accessToken,
baseURL: settings.baseURL,
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
location: settings.location,
project: settings.project,
providerOptions: settings.providerOptions,
}).model(modelID)
...ProviderPackage.routeDefaults(input.defaults),
accessToken: input.credential?.type === "oauth" ? input.credential.accessToken : input.settings.accessToken,
baseURL: input.settings.baseURL,
location: input.settings.location,
project: input.settings.project,
providerOptions: input.settings.providerOptions,
}).model(input.id)
}
@@ -69,9 +69,8 @@ const adc = (project?: string) => {
export const oauth = (input: OAuthOptions, project?: string) => {
if (input.accessToken !== undefined && input.auth !== undefined)
throw new Error("Google Vertex accessToken cannot be combined with auth")
if (input.auth) return input.auth
if (input.accessToken !== undefined) return Auth.bearer(input.accessToken)
return adc(project)
const auth = input.auth ?? (input.accessToken !== undefined ? Auth.bearer(input.accessToken) : adc(project))
return Auth.remove("x-goog-api-key").andThen(auth)
}
export * as GoogleVertexShared from "./google-vertex-shared.js"
+23 -18
View File
@@ -1,21 +1,19 @@
import { Effect } from "effect"
import type { ProviderPackage } from "../provider-package.js"
import { ProviderPackage } from "../provider-package.js"
import { Gemini } from "../protocols/gemini.js"
import { ProviderShared } from "../protocols/shared.js"
import { Auth } from "../route/auth.js"
import { Route, type RouteDefaultsInput } from "../route/client.js"
import { Endpoint } from "../route/endpoint.js"
import { Framing } from "../route/framing.js"
import { ProviderID, type LLMRequest, type ModelID, type ProviderOptions } from "../schema/index.js"
import { ProviderID, type LLMRequest, type ModelID } from "../schema/index.js"
import { GoogleVertexShared } from "./google-vertex-shared.js"
export interface GeminiOptionsInput extends Gemini.OptionsInput {
readonly labels?: Readonly<Record<string, string>>
}
export type GeminiProviderOptionsInput = ProviderOptions & {
readonly gemini?: GeminiOptionsInput
}
export type GeminiProviderOptionsInput = GeminiOptionsInput
export const id = ProviderID.make("google-vertex")
@@ -40,7 +38,7 @@ export type Settings = ProviderPackage.Settings &
const fromRequest = Effect.fn("GoogleVertex.fromRequest")(function* (request: LLMRequest) {
const body = yield* Gemini.protocol.body.from(request)
const value = request.providerOptions?.gemini?.labels
const value = request.providerOptions?.labels
const labels = ProviderShared.isRecord(value)
? Object.fromEntries(
Object.entries(value).filter((entry): entry is [string, string] => typeof entry[1] === "string"),
@@ -96,7 +94,10 @@ const configuredRoute = (input: Config, modelID: string | ModelID) => {
return route.with({
...rest,
endpoint: { baseURL: endpoint },
auth: apiKey === undefined ? GoogleVertexShared.oauth(input, project) : Auth.header("x-goog-api-key", apiKey),
auth:
apiKey === undefined
? GoogleVertexShared.oauth(input, project)
: Auth.remove("authorization").andThen(Auth.header("x-goog-api-key", apiKey)),
})
}
@@ -113,17 +114,21 @@ export const provider = {
id,
configure,
}
export const model: ProviderPackage.Definition<Settings, GeminiProviderOptionsInput>["model"] = (modelID, settings) => {
if (settings.apiKey !== undefined && settings.accessToken !== undefined)
export const model: ProviderPackage.Definition<Settings, GeminiProviderOptionsInput>["model"] = (input) => {
if (!input.credential && input.settings.apiKey !== undefined && input.settings.accessToken !== undefined)
throw new Error("Google Vertex apiKey cannot be combined with accessToken or auth")
return configure({
...(settings.apiKey === undefined ? { accessToken: settings.accessToken } : { apiKey: settings.apiKey }),
baseURL: settings.baseURL,
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
location: settings.location,
project: settings.project,
providerOptions: settings.providerOptions,
}).model(modelID)
...ProviderPackage.routeDefaults(input.defaults),
...(input.credential
? input.credential.type === "key"
? { apiKey: input.credential.value }
: { accessToken: input.credential.accessToken }
: input.settings.apiKey === undefined
? { accessToken: input.settings.accessToken }
: { apiKey: input.settings.apiKey }),
baseURL: input.settings.baseURL,
location: input.settings.location,
project: input.settings.project,
providerOptions: input.settings.providerOptions,
}).model(input.id)
}
+14 -12
View File
@@ -1,7 +1,7 @@
import type { RouteDefaultsInput } from "../route/client.js"
import { Auth } from "../route/auth.js"
import type { ProviderAuthOption } from "../route/auth-options.js"
import type { ProviderPackage } from "../provider-package.js"
import { ProviderPackage } from "../provider-package.js"
import { HttpOptions, ProviderID, mergeHttpOptions, type ModelID } from "../schema/index.js"
import { Gemini } from "../protocols/gemini.js"
import { GoogleImages } from "../protocols/google-images.js"
@@ -28,9 +28,11 @@ export interface Settings extends ProviderPackage.Settings {
const auth = (options: ProviderAuthOption<"optional">) => {
if ("auth" in options && options.auth) return options.auth
return Auth.optional("apiKey" in options ? options.apiKey : undefined, "apiKey")
.orElse(Auth.config("GOOGLE_GENERATIVE_AI_API_KEY"))
.pipe(Auth.header("x-goog-api-key"))
return Auth.remove("authorization").andThen(
Auth.optional("apiKey" in options ? options.apiKey : undefined, "apiKey")
.orElse(Auth.config("GOOGLE_GENERATIVE_AI_API_KEY"))
.pipe(Auth.header("x-goog-api-key")),
)
}
const configuredRoute = (input: Config) => {
@@ -57,14 +59,14 @@ export const configure = (input: Config = {}) => {
}
export const provider = configure()
export const model: ProviderPackage.Definition<Settings, Gemini.ProviderOptionsInput>["model"] = (modelID, settings) =>
export const model: ProviderPackage.Definition<Settings, Gemini.ProviderOptionsInput>["model"] = (input) =>
configure({
apiKey: settings.apiKey,
baseURL: settings.baseURL,
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
providerOptions: settings.providerOptions,
}).model(modelID)
...ProviderPackage.routeDefaults(input.defaults),
...(input.credential
? ProviderPackage.apiKeyOrBearerAuthOption(input.credential, "x-goog-api-key")
: { apiKey: input.settings.apiKey }),
baseURL: input.settings.baseURL,
providerOptions: input.settings.providerOptions,
}).model(input.id)
export const image = provider.image
@@ -1,10 +1,6 @@
import type { Options } from "../protocols/utils/open-responses-options.js"
import type { ProviderOptions } from "../schema/index.js"
export type OpenResponsesOptionsInput = Options & { readonly [key: string]: unknown }
export type OpenResponsesProviderOptionsInput = ProviderOptions & {
readonly openresponses?: OpenResponsesOptionsInput
}
export type OpenResponsesProviderOptionsInput = OpenResponsesOptionsInput
export * as OpenResponsesProviderOptions from "./open-responses-options.js"
@@ -1,4 +1,4 @@
import type { ProviderPackage } from "../provider-package.js"
import { ProviderPackage } from "../provider-package.js"
import { OpenAICompatibleResponses } from "../protocols/openai-compatible-responses.js"
import { AuthOptions, type ProviderAuthOption } from "../route/auth-options.js"
import type { RouteDefaultsInput } from "../route/client.js"
@@ -46,16 +46,13 @@ export const provider = {
configure,
}
export const model: ProviderPackage.Definition<Settings, OpenResponsesProviderOptionsInput>["model"] = (
modelID,
settings,
) =>
export const model: ProviderPackage.Definition<Settings, OpenResponsesProviderOptionsInput>["model"] = (input) =>
configure({
apiKey: settings.apiKey,
baseURL: settings.baseURL,
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
provider: settings.provider,
providerOptions: settings.providerOptions,
}).model(modelID)
...ProviderPackage.routeDefaults(input.defaults),
...(input.credential
? ProviderPackage.bearerAuthOption(input.credential)
: { apiKey: input.settings.apiKey }),
baseURL: input.settings.baseURL,
provider: input.settings.provider,
providerOptions: input.settings.providerOptions,
}).model(input.id)
+10 -10
View File
@@ -2,7 +2,7 @@ import { ProviderID, type ModelID } from "../schema/index.js"
import * as OpenAICompatibleChat from "../protocols/openai-compatible-chat.js"
import type { RouteDefaultsInput } from "../route/client.js"
import { AuthOptions, type ProviderAuthOption } from "../route/auth-options.js"
import type { ProviderPackage } from "../provider-package.js"
import { ProviderPackage } from "../provider-package.js"
import { profiles, type OpenAICompatibleProfile } from "./openai-compatible-profile.js"
import type { OpenAIProviderOptionsInput } from "./openai-options.js"
@@ -68,16 +68,16 @@ export const provider = {
configure,
}
export const model: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (modelID, settings) =>
export const model: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (input) =>
configure({
apiKey: settings.apiKey,
baseURL: settings.baseURL,
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
provider: settings.provider,
providerOptions: settings.providerOptions,
}).model(modelID)
...ProviderPackage.routeDefaults(input.defaults),
...(input.credential
? ProviderPackage.bearerAuthOption(input.credential)
: { apiKey: input.settings.apiKey }),
baseURL: input.settings.baseURL,
provider: input.settings.provider,
providerOptions: input.settings.providerOptions,
}).model(input.id)
export const baseten = define(profiles.baseten)
export const cerebras = define(profiles.cerebras)
+7 -27
View File
@@ -1,32 +1,13 @@
import type { ProviderOptions } from "../schema/index.js"
import { mergeProviderOptions } from "../schema/index.js"
import { mergeProviderOptions, type ProviderOptions } from "../schema/index.js"
import type { OpenResponsesOptionsInput } from "./open-responses-options.js"
import type { Options } from "../protocols/utils/open-responses-options.js"
export type { OpenAIResponseIncludable, OpenAIServiceTier } from "../protocols/utils/openai-options.js"
export type OpenAIOptionsInput = OpenResponsesOptionsInput
export type OpenAIConfigOptions = Options
export type OpenAIProviderOptionsInput = ProviderOptions & {
readonly openai?: OpenAIOptionsInput
}
const definedEntries = (input: Record<string, unknown>) =>
Object.entries(input).filter((entry) => entry[1] !== undefined)
const openAIProviderOptions = (options: OpenAIOptionsInput | undefined): ProviderOptions | undefined => {
const openai = Object.fromEntries(
definedEntries({
store: options?.store,
reasoningEffort: options?.reasoningEffort,
reasoningSummary: options?.reasoningSummary,
include: options?.include,
textVerbosity: options?.textVerbosity,
serviceTier: options?.serviceTier,
}),
)
if (Object.keys(openai).length === 0) return undefined
return { openai }
}
export type OpenAIProviderOptionsInput = OpenAIOptionsInput
export const gpt5DefaultOptions = (
modelID: string,
@@ -34,7 +15,7 @@ export const gpt5DefaultOptions = (
): ProviderOptions | undefined => {
const id = modelID.toLowerCase()
if (!id.includes("gpt-5") || id.includes("gpt-5-chat") || id.includes("gpt-5-pro")) return undefined
return openAIProviderOptions({
return {
reasoningEffort: "medium",
reasoningSummary: "auto",
// GPT-5 reasoning models are configured stateless (`store: false`) by
@@ -47,14 +28,13 @@ export const gpt5DefaultOptions = (
options.textVerbosity === true && id.includes("gpt-5.") && !id.includes("codex") && !id.includes("-chat")
? "low"
: undefined,
})
}
}
export const openAIDefaultOptions = (
modelID: string,
options: { readonly textVerbosity?: boolean } = {},
): ProviderOptions | undefined =>
mergeProviderOptions(openAIProviderOptions({ store: false }), gpt5DefaultOptions(modelID, options))
): ProviderOptions | undefined => mergeProviderOptions({ store: false }, gpt5DefaultOptions(modelID, options))
export const withOpenAIOptions = <Options extends { readonly providerOptions?: OpenAIProviderOptionsInput }>(
modelID: string,
+53 -19
View File
@@ -1,10 +1,10 @@
import { AuthOptions, type ProviderAuthOption } from "../route/auth-options.js"
import type { Route, RouteDefaultsInput } from "../route/client.js"
import type { ProviderPackage } from "../provider-package.js"
import { ProviderPackage } from "../provider-package.js"
import { HttpOptions, ProviderID, ToolDefinition, mergeHttpOptions, type ModelID } from "../schema/index.js"
import * as OpenAIChat from "../protocols/openai-chat.js"
import * as OpenAIResponses from "../protocols/openai-responses.js"
import { withOpenAIOptions, type OpenAIProviderOptionsInput } from "./openai-options.js"
import { withOpenAIOptions, type OpenAIConfigOptions, type OpenAIProviderOptionsInput } from "./openai-options.js"
import { OpenAIImages, type OpenAIImageString } from "../protocols/openai-images.js"
export type { OpenAIOptionsInput, OpenAIResponseIncludable } from "./openai-options.js"
@@ -17,11 +17,11 @@ export const routes = [OpenAIResponses.route, OpenAIChat.route]
// This provider facade wraps the lower-level Responses and Chat model factories
// with OpenAI-specific conveniences: typed options, API-key sugar, env fallback,
// and default option normalization.
export type Config = RouteDefaultsInput &
export type Config = Omit<RouteDefaultsInput, "providerOptions"> &
OpenAIConfigOptions &
ProviderAuthOption<"optional"> & {
readonly baseURL?: string
readonly queryParams?: Record<string, string>
readonly providerOptions?: OpenAIProviderOptionsInput
}
export interface ImageGenerationOptions {
@@ -57,13 +57,12 @@ export const imageGeneration = (options: ImageGenerationOptions = {}) =>
},
})
export interface Settings extends ProviderPackage.Settings {
export interface Settings extends ProviderPackage.Settings, OpenAIConfigOptions {
readonly apiKey?: string
readonly baseURL?: string
readonly organization?: string
readonly project?: string
readonly queryParams?: Readonly<Record<string, string>>
readonly providerOptions?: OpenAIProviderOptionsInput
}
const auth = (options: ProviderAuthOption<"optional">) => AuthOptions.bearer(options, "OPENAI_API_KEY")
@@ -73,6 +72,39 @@ const defaults = (input: Config) => {
return rest
}
const splitConfigOptions = <Input extends OpenAIConfigOptions>(input: Input) => {
const {
instructions,
store,
reasoningEffort,
reasoningSummary,
include,
textVerbosity,
serviceTier,
truncation,
allowedTools,
maxToolCalls,
parallelToolCalls,
...rest
} = input
return {
options: {
instructions,
store,
reasoningEffort,
reasoningSummary,
include,
textVerbosity,
serviceTier,
truncation,
allowedTools,
maxToolCalls,
parallelToolCalls,
},
rest,
}
}
const configuredRoute = <Body, Prepared>(route: Route<Body, Prepared>, input: Config) =>
route.with({
auth: auth(input),
@@ -82,7 +114,8 @@ const configuredRoute = <Body, Prepared>(route: Route<Body, Prepared>, input: Co
export const configure = (input: Config = {}) => {
const responsesRoute = configuredRoute(OpenAIResponses.route, input)
const chatRoute = configuredRoute(OpenAIChat.route, input)
const modelDefaults = defaults(input)
const split = splitConfigOptions(defaults(input))
const modelDefaults = { ...split.rest, providerOptions: split.options }
const responses = (id: string | ModelID) =>
responsesRoute
.with(withOpenAIOptions(id, modelDefaults, { textVerbosity: true }))
@@ -113,31 +146,32 @@ export const configure = (input: Config = {}) => {
export const provider = configure()
const config = (settings: Settings): Config => {
const config = (input: ProviderPackage.ModelInput<Settings>): Config => {
const settings = input.settings
const options = splitConfigOptions(settings).options
const headers = {
...(settings.organization === undefined ? {} : { "OpenAI-Organization": settings.organization }),
...(settings.project === undefined ? {} : { "OpenAI-Project": settings.project }),
...settings.headers,
...input.defaults.headers,
}
return {
apiKey: settings.apiKey,
...ProviderPackage.routeDefaults(input.defaults),
...(input.credential
? ProviderPackage.bearerAuthOption(input.credential)
: { apiKey: settings.apiKey }),
baseURL: settings.baseURL,
headers: Object.keys(headers).length === 0 ? undefined : headers,
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
providerOptions: settings.providerOptions,
queryParams: settings.queryParams === undefined ? undefined : { ...settings.queryParams },
...options,
}
}
export const model: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (modelID, settings) => {
return configure(config(settings)).responses(modelID)
export const model: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (input) => {
return configure(config(input)).responses(input.id)
}
export const chatModel: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (
modelID,
settings,
) => configure(config(settings)).chat(modelID)
export const chatModel: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (input) =>
configure(config(input)).chat(input.id)
export const responses = provider.responses
export const chat = provider.chat
export const image = provider.image
+12 -17
View File
@@ -4,8 +4,8 @@ import { Endpoint } from "../route/endpoint.js"
import { Framing } from "../route/framing.js"
import { Protocol } from "../route/protocol.js"
import { AuthOptions, type ProviderAuthOption } from "../route/auth-options.js"
import { ProviderID, type CacheHint, type ModelID, type ProviderOptions } from "../schema/index.js"
import type { ProviderPackage } from "../provider-package.js"
import { ProviderID, type CacheHint, type ModelID } from "../schema/index.js"
import { ProviderPackage } from "../provider-package.js"
import * as OpenAICompatibleProfiles from "./openai-compatible-profile.js"
import * as OpenAIChat from "../protocols/openai-chat.js"
import { newBreakpoints, ttlBucket } from "../protocols/utils/cache.js"
@@ -71,9 +71,7 @@ export interface OpenRouterOptions {
}>
}
export type OpenRouterProviderOptionsInput = ProviderOptions & {
readonly openrouter?: OpenRouterOptions
}
export type OpenRouterProviderOptionsInput = OpenRouterOptions
export type LanguageModelOptions = Omit<RouteDefaultsInput, "providerOptions"> &
ProviderAuthOption<"optional"> & {
@@ -120,7 +118,7 @@ export const protocol = Protocol.make({
return {
...body,
messages,
...bodyOptions(request.providerOptions?.openrouter),
...bodyOptions(request.providerOptions),
...(request.promptCacheKey ? { prompt_cache_key: request.promptCacheKey } : {}),
} as OpenRouterBody
}),
@@ -193,15 +191,12 @@ export const configure = (input: LanguageModelOptions = {}) => {
}
export const provider = configure()
export const model: ProviderPackage.Definition<Settings, OpenRouterProviderOptionsInput>["model"] = (
modelID,
settings,
) =>
export const model: ProviderPackage.Definition<Settings, OpenRouterProviderOptionsInput>["model"] = (input) =>
configure({
apiKey: settings.apiKey,
baseURL: settings.baseURL,
headers: settings.headers,
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
providerOptions: settings.providerOptions,
}).model(modelID)
...ProviderPackage.routeDefaults(input.defaults),
...(input.credential
? ProviderPackage.bearerAuthOption(input.credential)
: { apiKey: input.settings.apiKey }),
baseURL: input.settings.baseURL,
providerOptions: input.settings.providerOptions,
}).model(input.id)
+12 -14
View File
@@ -1,20 +1,18 @@
import { AuthOptions, type ProviderAuthOption } from "../route/auth-options.js"
import { Route, type RouteDefaultsInput } from "../route/client.js"
import { Endpoint } from "../route/endpoint.js"
import { HttpOptions, ProviderID, type ModelID, type ProviderOptions } from "../schema/index.js"
import { HttpOptions, ProviderID, type ModelID } from "../schema/index.js"
import * as OpenAICompatibleProfiles from "./openai-compatible-profile.js"
import * as OpenAICompatibleChat from "../protocols/openai-compatible-chat.js"
import * as OpenAIChat from "../protocols/openai-chat.js"
import * as OpenAIResponses from "../protocols/openai-responses.js"
import { XAIImages } from "../protocols/xai-images.js"
import type { OpenAIOptionsInput } from "./openai-options.js"
import type { ProviderPackage } from "../provider-package.js"
import { ProviderPackage } from "../provider-package.js"
export const id = ProviderID.make("xai")
export type XAIProviderOptionsInput = ProviderOptions & {
readonly xai?: OpenAIOptionsInput
}
export type XAIProviderOptionsInput = OpenAIOptionsInput
export type LanguageModelOptions = Omit<RouteDefaultsInput, "providerOptions"> &
ProviderAuthOption<"optional"> & {
@@ -37,7 +35,7 @@ const responsesRoute = Route.make({
protocol: OpenAIResponses.protocol,
endpoint: Endpoint.path("/responses", { baseURL: OpenAICompatibleProfiles.profiles.xai.baseURL }),
transport: OpenAIResponses.httpTransport,
defaults: { providerOptions: { xai: { store: false } } },
defaults: { providerOptions: { store: false } },
})
const chatRoute = Route.make({
@@ -97,15 +95,15 @@ export const configure = (input: LanguageModelOptions = {}) => {
}
export const provider = configure()
export const model: ProviderPackage.Definition<Settings, XAIProviderOptionsInput>["model"] = (modelID, settings) =>
export const model: ProviderPackage.Definition<Settings, XAIProviderOptionsInput>["model"] = (input) =>
configure({
apiKey: settings.apiKey,
baseURL: settings.baseURL,
headers: settings.headers,
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
providerOptions: settings.providerOptions,
}).model(modelID)
...ProviderPackage.routeDefaults(input.defaults),
...(input.credential
? ProviderPackage.bearerAuthOption(input.credential)
: { apiKey: input.settings.apiKey }),
baseURL: input.settings.baseURL,
providerOptions: input.settings.providerOptions,
}).model(input.id)
export const responses = provider.responses
export const chat = provider.chat
export const image = provider.image
+2 -2
View File
@@ -13,8 +13,8 @@ import type { AIError } from "../schema/index.js"
* - AWS event stream — length-prefixed binary frames with CRC checksums.
* Each emitted frame is one parsed binary event record.
*
* The frame type is opaque to this layer; the protocol's `decode` step turns
* a frame into a typed chunk.
* The frame type is opaque to this layer; the protocol's event schema decodes
* each frame before its state machine handles it.
*/
export interface Definition<Frame> {
readonly id: string
+1 -2
View File
@@ -73,8 +73,7 @@ export interface ProtocolStream<Frame, Event, State> {
*
* Provider implementations should usually call `Protocol.make({ ... })`
* without explicit type arguments; the schemas and parser functions are the
* source of truth. The constructor remains as the public seam for future
* cross-cutting concerns such as tracing or instrumentation.
* source of truth.
*/
export const make = <Body, Frame, Event, State>(
input: Protocol<Body, Frame, Event, State>,
+4 -15
View File
@@ -33,22 +33,12 @@ const mergeStringRecords = (
return Object.keys(result).length === 0 ? undefined : result
}
export const ProviderOptions = Schema.Record(Schema.String, Schema.Record(Schema.String, Schema.Unknown))
export const ProviderOptions = Schema.Record(Schema.String, Schema.Unknown)
export type ProviderOptions = Schema.Schema.Type<typeof ProviderOptions>
export const mergeProviderOptions = (
...items: ReadonlyArray<ProviderOptions | undefined>
): ProviderOptions | undefined => {
const result: Record<string, Record<string, unknown>> = {}
for (const item of items) {
if (!item) continue
for (const [provider, options] of Object.entries(item)) {
const merged = mergeJsonRecords(result[provider], options)
if (merged) result[provider] = merged
}
}
return Object.keys(result).length === 0 ? undefined : result
}
): ProviderOptions | undefined => mergeJsonRecords(...items)
export class HttpOptions extends Schema.Class<HttpOptions>("AI.HttpOptions")({
body: Schema.optional(JsonSchema),
@@ -269,10 +259,9 @@ export class CacheHint extends Schema.Class<CacheHint>("LLM.CacheHint")({
// Auto-placement policy for prompt caching. The protocol-neutral lowering step
// reads this and injects `CacheHint`s at the configured boundaries; the
// per-protocol body builders then translate those hints into wire markers as
// usual. `"auto"` is the recommended default for agent loops — it places
// usual. `"auto"` is the default for agent loops — it places
// breakpoints at the last tool definition, the first and last distinct system
// parts, and the conversation tail. The rolling message breakpoint keeps a
// prior cache entry within Anthropic/Bedrock's 20-block lookback during long
// parts, and the conversation tail so recent prefixes remain reusable during
// tool loops.
//
// Pass `"none"` to opt out entirely (the legacy behavior). Pass the granular
+64 -25
View File
@@ -81,8 +81,22 @@ OpenAI.configure({
}).responses("gpt-4.1-mini")
OpenAI.configure({
generation: { maxTokens: 100 },
providerOptions: { openai: { store: false } },
store: false,
}).responses("gpt-4.1-mini")
OpenAI.model({
id: "gpt-5",
settings: {},
credential: { type: "key", value: "sk-test" },
defaults: { headers: { "x-test": "value" } },
})
OpenAI.model({
id: "gpt-5",
settings: {
// @ts-expect-error Common request defaults belong under input.defaults.
headers: { "x-test": "value" },
},
defaults: {},
})
// @ts-expect-error OpenAI model selectors only accept model ids.
OpenAI.configure({ apiKey: "sk-test" }).responses("gpt-4.1-mini", {})
@@ -97,7 +111,7 @@ OpenAI.configure({ bogus: true })
OpenAI.configure({ generation: { maxTokens: "many" } })
// @ts-expect-error provider-native options remain typed.
OpenAI.configure({ providerOptions: { openai: { store: "false" } } })
OpenAI.configure({ store: "false" })
// @ts-expect-error auth is an override, so OpenAI rejects apiKey with auth.
OpenAI.configure({ apiKey: "sk-test", auth: Auth.bearer("oauth-token") })
@@ -139,48 +153,57 @@ Anthropic.configure({ apiKey: "anthropic-key" }).model("claude-haiku")
Anthropic.configure({
apiKey: "anthropic-key",
providerOptions: {
anthropic: { thinking: { type: "enabled", budgetTokens: 1_024 }, effort: "high" },
thinking: { type: "enabled", budgetTokens: 1_024 },
effort: "high",
},
}).model("claude-haiku")
// @ts-expect-error Anthropic model selectors only accept model ids.
Anthropic.configure({ apiKey: "anthropic-key" }).model("claude-haiku", {})
// @ts-expect-error Anthropic package settings accept only one auth source.
Anthropic.model("claude-sonnet-4-6", { apiKey: "anthropic-key", authToken: "anthropic-token" })
Anthropic.model({
id: "claude-sonnet-4-6",
// @ts-expect-error Anthropic package settings accept only one auth source.
settings: { apiKey: "anthropic-key", authToken: "anthropic-token" },
defaults: {},
})
// @ts-expect-error Enabled Anthropic thinking requires a token budget.
Anthropic.configure({ providerOptions: { anthropic: { thinking: { type: "enabled" } } } })
Anthropic.configure({ providerOptions: { thinking: { type: "enabled" } } })
// @ts-expect-error Anthropic thinking budgets must be numbers.
Anthropic.configure({ providerOptions: { anthropic: { thinking: { type: "enabled", budgetTokens: "large" } } } })
Anthropic.configure({ providerOptions: { thinking: { type: "enabled", budgetTokens: "large" } } })
AnthropicCompatible.configure({
apiKey: "messages-key",
baseURL: "https://messages.example.com/v1",
provider: "example",
providerOptions: { anthropic: { thinking: { type: "disabled" } } },
providerOptions: { thinking: { type: "disabled" } },
}).model("compatible-model")
// @ts-expect-error Anthropic-compatible providers require a base URL.
AnthropicCompatible.configure({ apiKey: "messages-key" })
// @ts-expect-error Anthropic-compatible model selectors only accept model ids.
AnthropicCompatible.configure({ baseURL: "https://messages.example.com/v1" }).model("compatible-model", {})
// @ts-expect-error Anthropic-compatible package settings accept only one auth source.
AnthropicCompatible.model("compatible-model", {
apiKey: "messages-key",
authToken: "messages-token",
baseURL: "https://messages.example.com/v1",
AnthropicCompatible.model({
id: "compatible-model",
// @ts-expect-error Anthropic-compatible package settings accept only one auth source.
settings: {
apiKey: "messages-key",
authToken: "messages-token",
baseURL: "https://messages.example.com/v1",
},
defaults: {},
})
Google.configure({ apiKey: "google-key" }).model("gemini-2.5-flash")
Google.configure({
apiKey: "google-key",
providerOptions: { gemini: { thinkingConfig: { thinkingBudget: 0, includeThoughts: false } } },
providerOptions: { thinkingConfig: { thinkingBudget: 0, includeThoughts: false } },
}).model("gemini-2.5-flash")
// @ts-expect-error Google model selectors only accept model ids.
Google.configure({ apiKey: "google-key" }).model("gemini-2.5-flash", {})
// @ts-expect-error Gemini thinking budgets must be numbers.
Google.configure({ providerOptions: { gemini: { thinkingConfig: { thinkingBudget: "large" } } } })
Google.configure({ providerOptions: { thinkingConfig: { thinkingBudget: "large" } } })
GoogleVertex.configure({
apiKey: "vertex-key",
providerOptions: { gemini: { thinkingConfig: { thinkingBudget: 1_024 } } },
providerOptions: { thinkingConfig: { thinkingBudget: 1_024 } },
}).model("gemini-3.5-flash")
GoogleVertex.configure({ accessToken: "vertex-token", project: "project" }).model("gemini-3.5-flash")
GoogleVertex.configure({ auth: Auth.bearer("vertex-token"), project: "project" }).model("gemini-3.5-flash")
@@ -188,15 +211,23 @@ GoogleVertex.configure({ auth: Auth.bearer("vertex-token"), project: "project" }
GoogleVertex.configure({ apiKey: "vertex-key" }).model("gemini-3.5-flash", {})
// @ts-expect-error Vertex Gemini config accepts only one auth source.
GoogleVertex.configure({ accessToken: "vertex-token", apiKey: "vertex-key", project: "project" })
// @ts-expect-error Vertex Gemini package settings accept only one auth source.
GoogleVertex.model("gemini-3.5-flash", { accessToken: "vertex-token", apiKey: "vertex-key", project: "project" })
GoogleVertex.model({
id: "gemini-3.5-flash",
// @ts-expect-error Vertex Gemini package settings accept only one auth source.
settings: { accessToken: "vertex-token", apiKey: "vertex-key", project: "project" },
defaults: {},
})
GoogleVertexChat.configure({ accessToken: "vertex-token", project: "project" }).model("deepseek-ai/deepseek-v3.2-maas")
GoogleVertexChat.configure({ auth: Auth.bearer("vertex-token"), project: "project" }).model(
"deepseek-ai/deepseek-v3.2-maas",
)
// @ts-expect-error Vertex Chat package settings do not accept API keys.
GoogleVertexChat.model("deepseek-ai/deepseek-v3.2-maas", { apiKey: "vertex-key", project: "project" })
GoogleVertexChat.model({
id: "deepseek-ai/deepseek-v3.2-maas",
// @ts-expect-error Vertex Chat package settings do not accept API keys.
settings: { apiKey: "vertex-key", project: "project" },
defaults: {},
})
GoogleVertexChat.configure({ accessToken: "vertex-token", project: "project" }).model(
"deepseek-ai/deepseek-v3.2-maas",
// @ts-expect-error Vertex Chat model selectors only accept model ids.
@@ -213,8 +244,12 @@ GoogleVertexResponses.configure({ accessToken: "vertex-token", project: "project
GoogleVertexResponses.configure({ auth: Auth.bearer("vertex-token"), project: "project" }).model(
"xai/grok-4.20-reasoning",
)
// @ts-expect-error Vertex Responses package settings do not accept API keys.
GoogleVertexResponses.model("xai/grok-4.20-reasoning", { apiKey: "vertex-key", project: "project" })
GoogleVertexResponses.model({
id: "xai/grok-4.20-reasoning",
// @ts-expect-error Vertex Responses package settings do not accept API keys.
settings: { apiKey: "vertex-key", project: "project" },
defaults: {},
})
GoogleVertexResponses.configure({ accessToken: "vertex-token", project: "project" }).model(
"xai/grok-4.20-reasoning",
// @ts-expect-error Vertex Responses model selectors only accept model ids.
@@ -230,10 +265,14 @@ GoogleVertexResponses.configure({
GoogleVertexMessages.configure({
accessToken: "vertex-token",
project: "project",
providerOptions: { anthropic: { thinking: { type: "adaptive", display: "omitted" }, effort: "low" } },
providerOptions: { thinking: { type: "adaptive", display: "omitted" }, effort: "low" },
}).model("claude-sonnet-4-6")
// @ts-expect-error Vertex Messages package settings do not accept API keys.
GoogleVertexMessages.model("claude-sonnet-4-6", { apiKey: "vertex-key", project: "project" })
GoogleVertexMessages.model({
id: "claude-sonnet-4-6",
// @ts-expect-error Vertex Messages package settings do not accept API keys.
settings: { apiKey: "vertex-key", project: "project" },
defaults: {},
})
GoogleVertexMessages.configure({ auth: Auth.bearer("vertex-token"), project: "project" }).model("claude-sonnet-4-6")
GoogleVertexMessages.configure({ accessToken: "vertex-token", project: "project" }).model(
"claude-sonnet-4-6",
+3 -3
View File
@@ -51,13 +51,13 @@ describe("request option precedence", () => {
endpoint: { baseURL: "https://api.openai.test/v1/" },
auth: Auth.bearer("test"),
generation: { maxTokens: 10, temperature: 1, stop: ["route"] },
providerOptions: { openai: { store: false, reasoningEffort: "low" } },
providerOptions: { store: false, reasoningEffort: "low" },
})
const model = route.model({
id: "gpt-4o-mini",
defaults: {
generation: { maxTokens: 20, temperature: 0.5, frequencyPenalty: 0.25, stop: ["model"] },
providerOptions: { openai: { reasoningEffort: "medium" } },
providerOptions: { reasoningEffort: "medium" },
},
})
const prepared = yield* compileRequest(
@@ -65,7 +65,7 @@ describe("request option precedence", () => {
model,
prompt: "Say hello.",
generation: { maxTokens: 30, topP: 0.9, stop: ["request"] },
providerOptions: { openai: { store: true } },
providerOptions: { store: true },
}),
)
+1 -1
View File
@@ -105,7 +105,7 @@ export function continuationRequest(input: {
tools: features.has("tool-call") ? [continuationTool] : [],
cache: "none",
providerOptions: features.has("encrypted-reasoning")
? { openai: { store: false, include: ["reasoning.encrypted_content"], reasoningSummary: "auto" } }
? { store: false, include: ["reasoning.encrypted_content"], reasoningSummary: "auto" }
: undefined,
generation: { maxTokens: 80, temperature: 0 },
})
+6 -8
View File
@@ -13,9 +13,7 @@ interface ExampleOptions {
readonly mode?: "fast" | "thorough"
}
type ExampleProviderOptions = ProviderOptions & {
readonly example?: ExampleOptions
}
type ExampleProviderOptions = ProviderOptions & ExampleOptions
const model = OpenAIChat.route
.with({ endpoint: { baseURL: "https://example.com/v1" } })
@@ -26,7 +24,7 @@ type StreamRequirements<T> = T extends Stream.Stream<infer _A, infer _E, infer R
type Equal<A, B> = [A, B] extends [B, A] ? true : false
type Assert<T extends true> = T
LLM.request({ model, prompt: "Hello", providerOptions: { example: { mode: "fast" } } })
LLM.request({ model, prompt: "Hello", providerOptions: { mode: "fast" } })
LLM.request({ model, prompt: "Hello", providerOptions: { future: { option: true } } })
const generated = LLM.generate(LLM.request({ model, prompt: "Hello" }))
@@ -38,14 +36,14 @@ LLM.request({
model,
prompt: "Hello",
// @ts-expect-error Known provider options preserve their value types.
providerOptions: { example: { mode: "slow" } },
providerOptions: { mode: "slow" },
})
const generatedObject = LLM.generateObject({
model,
prompt: "Hello",
schema: Schema.Struct({ answer: Schema.String }),
providerOptions: { example: { mode: "thorough" } },
providerOptions: { mode: "thorough" },
})
type GenerateObjectRequirements = Assert<Equal<Requirements<typeof generatedObject>, LLMClientService>>
@@ -61,13 +59,13 @@ LLM.generateObject({
prompt: "Hello",
jsonSchema: { type: "object" },
// @ts-expect-error Dynamic object generation uses the selected model's provider options.
providerOptions: { example: { mode: false } },
providerOptions: { mode: false },
})
declare const generic: LanguageModel
LLM.request({ model: generic, prompt: "Hello", providerOptions: { arbitrary: { option: true } } })
const options: LanguageModelProviderOptions<typeof model> = { example: { mode: "fast" } }
const options: LanguageModelProviderOptions<typeof model> = { mode: "fast" }
void (options satisfies LanguageModelProviderOptions<typeof model>)
void (true satisfies GenerateRequirements)
void (true satisfies StreamClientRequirements)
+5 -5
View File
@@ -59,18 +59,18 @@ describe("llm constructors", () => {
provider: "fake",
route: chatRoute.with({
generation: { maxTokens: 100, temperature: 1 },
providerOptions: { openai: { store: false, metadata: { model: true } } },
providerOptions: { store: false, metadata: { model: true } },
http: { body: { metadata: { model: true } }, headers: { "x-shared": "model" }, query: { model: "1" } },
}),
}),
prompt: "Say hello.",
generation: { temperature: 0 },
providerOptions: { openai: { store: true, metadata: { request: true } } },
providerOptions: { store: true, metadata: { request: true } },
http: { body: { metadata: { request: true } }, headers: { "x-shared": "request" }, query: { request: "1" } },
})
expect(request.generation).toEqual({ temperature: 0 })
expect(request.providerOptions).toEqual({ openai: { store: true, metadata: { request: true } } })
expect(request.providerOptions).toEqual({ store: true, metadata: { request: true } })
expect(request.http).toEqual({
body: { metadata: { request: true } },
headers: { "x-shared": "request" },
@@ -123,7 +123,7 @@ describe("llm constructors", () => {
defaults: {
limits: { context: 128_000, output: 8_192 },
generation: { maxTokens: 1_024, stop: ["END"] },
providerOptions: { openai: { parallelToolCalls: false } },
providerOptions: { parallelToolCalls: false },
http: { body: { extra_body: true } },
},
compatibility: { toolSchema: "moonshot" },
@@ -132,7 +132,7 @@ describe("llm constructors", () => {
expect(request.model.defaults?.limits).toEqual({ context: 128_000, output: 8_192 })
expect(request.model.defaults?.generation).toEqual({ maxTokens: 1_024, stop: ["END"] })
expect(request.model.defaults?.providerOptions).toEqual({ openai: { parallelToolCalls: false } })
expect(request.model.defaults?.providerOptions).toEqual({ parallelToolCalls: false })
expect(request.model.defaults?.http).toEqual({ body: { extra_body: true } })
expect(request.model.compatibility).toEqual({ toolSchema: "moonshot" })
expect(request.generation).toBeUndefined()
@@ -3,11 +3,11 @@ import { AnthropicCompatible } from "../../src/providers.js"
const model = AnthropicCompatible.configure({ baseURL: "https://example.com" }).model("claude")
LLM.request({ model, prompt: "Hello", providerOptions: { anthropic: { effort: "high" } } })
LLM.request({ model, prompt: "Hello", providerOptions: { effort: "high" } })
LLM.request({
model,
prompt: "Hello",
// @ts-expect-error Anthropic effort must be a string.
providerOptions: { anthropic: { effort: 1 } },
providerOptions: { effort: 1 },
})
@@ -3,11 +3,11 @@ import { Anthropic } from "../../src/providers.js"
const model = Anthropic.provider.model("claude-sonnet-4-5")
LLM.request({ model, prompt: "Hello", providerOptions: { anthropic: { thinking: { type: "adaptive" } } } })
LLM.request({ model, prompt: "Hello", providerOptions: { thinking: { type: "adaptive" } } })
LLM.request({
model,
prompt: "Hello",
// @ts-expect-error Anthropic thinking modes are a fixed union.
providerOptions: { anthropic: { thinking: { type: "automatic" } } },
providerOptions: { thinking: { type: "automatic" } },
})
@@ -3,11 +3,11 @@ import { Azure } from "../../src/providers.js"
const model = Azure.configure({ resourceName: "example" }).responses("deployment")
LLM.request({ model, prompt: "Hello", providerOptions: { openai: { store: false } } })
LLM.request({ model, prompt: "Hello", providerOptions: { store: false } })
LLM.request({
model,
prompt: "Hello",
// @ts-expect-error Azure OpenAI store must be boolean.
providerOptions: { openai: { store: "false" } },
providerOptions: { store: "false" },
})
@@ -3,11 +3,11 @@ import { GoogleVertexChat } from "../../src/providers.js"
const model = GoogleVertexChat.configure({ accessToken: "test", project: "project" }).model("gemini")
LLM.request({ model, prompt: "Hello", providerOptions: { openai: { serviceTier: "priority" } } })
LLM.request({ model, prompt: "Hello", providerOptions: { serviceTier: "priority" } })
LLM.request({
model,
prompt: "Hello",
// @ts-expect-error Vertex OpenAI-compatible service tiers use the OpenAI union.
providerOptions: { openai: { serviceTier: "premium" } },
providerOptions: { serviceTier: "premium" },
})
@@ -3,11 +3,11 @@ import { GoogleVertexMessages } from "../../src/providers.js"
const model = GoogleVertexMessages.configure({ accessToken: "test", project: "project" }).model("claude")
LLM.request({ model, prompt: "Hello", providerOptions: { anthropic: { effort: "medium" } } })
LLM.request({ model, prompt: "Hello", providerOptions: { effort: "medium" } })
LLM.request({
model,
prompt: "Hello",
// @ts-expect-error Vertex Anthropic effort must be a string.
providerOptions: { anthropic: { effort: false } },
providerOptions: { effort: false },
})
@@ -3,11 +3,11 @@ import { GoogleVertexResponses } from "../../src/providers.js"
const model = GoogleVertexResponses.configure({ accessToken: "test", project: "project" }).model("gemini")
LLM.request({ model, prompt: "Hello", providerOptions: { openresponses: { textVerbosity: "high" } } })
LLM.request({ model, prompt: "Hello", providerOptions: { textVerbosity: "high" } })
LLM.request({
model,
prompt: "Hello",
// @ts-expect-error Vertex Responses verbosity uses the Open Responses union.
providerOptions: { openresponses: { textVerbosity: "verbose" } },
providerOptions: { textVerbosity: "verbose" },
})
@@ -6,12 +6,12 @@ const model = GoogleVertex.provider.configure({ apiKey: "test" }).model("gemini-
LLM.request({
model,
prompt: "Hello",
providerOptions: { gemini: { thinkingConfig: { includeThoughts: true } } },
providerOptions: { thinkingConfig: { includeThoughts: true } },
})
LLM.request({
model,
prompt: "Hello",
// @ts-expect-error Vertex Gemini includeThoughts must be boolean.
providerOptions: { gemini: { thinkingConfig: { includeThoughts: "yes" } } },
providerOptions: { thinkingConfig: { includeThoughts: "yes" } },
})
@@ -6,17 +6,15 @@ const model = Google.provider.model("gemini-2.5-pro")
LLM.request({
model,
prompt: "Hello",
providerOptions: { gemini: { thinkingConfig: { thinkingBudget: 1024 } } },
providerOptions: { thinkingConfig: { thinkingBudget: 1024 } },
})
LLM.request({
model,
prompt: "Hello",
providerOptions: {
gemini: {
// @ts-expect-error Gemini safety settings require a threshold for every category.
safetySettings: [{ category: "HARM_CATEGORY_HATE_SPEECH" }],
},
// @ts-expect-error Gemini safety settings require a threshold for every category.
safetySettings: [{ category: "HARM_CATEGORY_HATE_SPEECH" }],
},
})
@@ -24,12 +22,10 @@ LLM.request({
model,
prompt: "Hello",
providerOptions: {
gemini: {
cachedContent: "cachedContents/example",
safetySettings: [{ category: "HARM_CATEGORY_HATE_SPEECH", threshold: "BLOCK_ONLY_HIGH" }],
serviceTier: "future-tier",
thinkingConfig: { thinkingLevel: "high", includeThoughts: true },
},
cachedContent: "cachedContents/example",
safetySettings: [{ category: "HARM_CATEGORY_HATE_SPEECH", threshold: "BLOCK_ONLY_HIGH" }],
serviceTier: "future-tier",
thinkingConfig: { thinkingLevel: "high", includeThoughts: true },
},
})
@@ -37,11 +33,11 @@ LLM.request({
model,
prompt: "Hello",
// @ts-expect-error Gemini thinking budgets must be numeric.
providerOptions: { gemini: { thinkingConfig: { thinkingBudget: "large" } } },
providerOptions: { thinkingConfig: { thinkingBudget: "large" } },
})
LLM.request({
model,
prompt: "Hello",
providerOptions: { gemini: { thinkingConfig: { thinkingLevel: "maximum" } } },
providerOptions: { thinkingConfig: { thinkingLevel: "maximum" } },
})
@@ -3,11 +3,11 @@ import { OpenAICompatibleResponses } from "../../src/providers.js"
const model = OpenAICompatibleResponses.configure({ baseURL: "https://example.com" }).model("model")
LLM.request({ model, prompt: "Hello", providerOptions: { openresponses: { reasoningSummary: "detailed" } } })
LLM.request({ model, prompt: "Hello", providerOptions: { reasoningSummary: "detailed" } })
LLM.request({
model,
prompt: "Hello",
// @ts-expect-error Open Responses reasoning summaries use a fixed union.
providerOptions: { openresponses: { reasoningSummary: "full" } },
providerOptions: { reasoningSummary: "full" },
})
@@ -3,11 +3,11 @@ import { OpenAICompatible } from "../../src/providers.js"
const model = OpenAICompatible.deepseek.model("deepseek-chat")
LLM.request({ model, prompt: "Hello", providerOptions: { openai: { store: false } } })
LLM.request({ model, prompt: "Hello", providerOptions: { store: false } })
LLM.request({
model,
prompt: "Hello",
// @ts-expect-error OpenAI-compatible store must be boolean.
providerOptions: { openai: { store: "false" } },
providerOptions: { store: "false" },
})
@@ -3,13 +3,13 @@ import { OpenAI } from "../../src/providers.js"
const selected = OpenAI.responses("gpt-5")
LLM.request({ model: selected, prompt: "Hello", providerOptions: { openai: { reasoningEffort: "high" } } })
LLM.request({ model: selected, prompt: "Hello", providerOptions: { reasoningEffort: "high" } })
LLM.request({
model: selected,
prompt: "Hello",
// @ts-expect-error OpenAI reasoning effort must be a string.
providerOptions: { openai: { reasoningEffort: 1 } },
providerOptions: { reasoningEffort: 1 },
})
OpenAI.configure({
@@ -3,27 +3,25 @@ import { OpenRouter } from "../../src/providers.js"
const model = OpenRouter.provider.model("anthropic/claude-sonnet-4.5")
LLM.request({ model, prompt: "Hello", providerOptions: { openrouter: { usage: true } } })
LLM.request({ model, prompt: "Hello", providerOptions: { usage: true } })
LLM.request({
model,
prompt: "Hello",
providerOptions: {
openrouter: {
models: ["google/gemini-3.1-pro"],
provider: {
order: ["anthropic"],
require_parameters: true,
data_collection: "future-policy",
sort: "future-sort",
max_price: { prompt: "0.50" },
},
reasoning: { effort: "future-effort", exclude: false },
plugins: [{ id: "future-plugin", enabled: true }],
web_search_options: { engine: "future-engine" },
debug: { echo_upstream_body: true },
user: "user_123",
models: ["google/gemini-3.1-pro"],
provider: {
order: ["anthropic"],
require_parameters: true,
data_collection: "future-policy",
sort: "future-sort",
max_price: { prompt: "0.50" },
},
reasoning: { effort: "future-effort", exclude: false },
plugins: [{ id: "future-plugin", enabled: true }],
web_search_options: { engine: "future-engine" },
debug: { echo_upstream_body: true },
user: "user_123",
},
})
@@ -31,5 +29,5 @@ LLM.request({
model,
prompt: "Hello",
// @ts-expect-error OpenRouter usage must be boolean or an option record.
providerOptions: { openrouter: { usage: "yes" } },
providerOptions: { usage: "yes" },
})
@@ -3,11 +3,11 @@ import { XAI } from "../../src/providers.js"
const model = XAI.provider.model("grok-4")
LLM.request({ model, prompt: "Hello", providerOptions: { xai: { reasoningEffort: "high" } } })
LLM.request({ model, prompt: "Hello", providerOptions: { reasoningEffort: "high" } })
LLM.request({
model,
prompt: "Hello",
// @ts-expect-error xAI's OpenAI-compatible reasoning effort must be a string.
providerOptions: { xai: { reasoningEffort: true } },
providerOptions: { reasoningEffort: true },
})
+322 -116
View File
@@ -1,6 +1,72 @@
import { describe, expect, test } from "bun:test"
import { ConfigProvider, Effect } from "effect"
import { Headers } from "effect/unstable/http"
import { LLM, ProviderPackage } from "@opencode-ai/ai"
import { model } from "@opencode-ai/ai/providers/openai"
const packageInput = <Input extends Record<string, unknown>>(id: string, input: Input) => {
const { headers, body, limits, ...settings } = input
return { id, settings, defaults: { headers, body, limits } }
}
const authHeaders = (
selected: ReturnType<typeof model>,
headers: Record<string, string> = {},
env: Record<string, string> = {},
) =>
Effect.runPromise(
selected.route.auth.apply({
request: LLM.request({ model: selected, prompt: "hello" }),
method: "POST",
url: "https://example.test/v1",
body: "{}",
headers: Headers.fromInput(headers),
}).pipe(Effect.provide(ConfigProvider.layer(ConfigProvider.fromEnv({ env })))),
)
const applyAuth = (
option: ReturnType<typeof ProviderPackage.bearerAuthOption>,
headers: Record<string, string> = {},
) => {
const selected = model(packageInput("gpt-5", { apiKey: "fixture" }))
return Effect.runPromise(
option.auth.apply({
request: LLM.request({ model: selected, prompt: "hello" }),
method: "POST",
url: "https://example.test/v1",
body: "{}",
headers: Headers.fromInput(headers),
}),
)
}
describe("provider package credential lowering", () => {
test("intentionally renders keys and OAuth credentials as bearer auth", async () => {
const key = await applyAuth(ProviderPackage.bearerAuthOption({ type: "key", value: "provider-key" }))
const oauth = await applyAuth(
ProviderPackage.bearerAuthOption({ type: "oauth", accessToken: "provider-token" }),
)
expect(key.authorization).toBe("Bearer provider-key")
expect(oauth.authorization).toBe("Bearer provider-token")
})
test("keeps key-header credentials configurable and removes stale keys for OAuth", async () => {
expect(
ProviderPackage.apiKeyOrBearerAuthOption({ type: "key", value: "provider-key" }, "x-api-key"),
).toEqual({ apiKey: "provider-key" })
const oauth = ProviderPackage.apiKeyOrBearerAuthOption(
{ type: "oauth", accessToken: "provider-token" },
"x-api-key",
)
if (!("auth" in oauth)) throw new Error("Expected OAuth credential to lower to auth")
const headers = await applyAuth(oauth, { "x-api-key": "stale" })
expect(headers.authorization).toBe("Bearer provider-token")
expect(headers["x-api-key"]).toBeUndefined()
})
})
describe("provider package entrypoints", () => {
test("semantic API aliases expose the same contract", async () => {
const modules = await Promise.all([
@@ -45,14 +111,18 @@ describe("provider package entrypoints", () => {
body: { service_tier: "priority" },
limits: { context: 200_000, output: 64_000 },
}
const openrouter = OpenRouter.model("anthropic/claude-sonnet-4", {
...settings,
providerOptions: { openrouter: { usage: true } },
})
const xai = XAI.model("grok-4", {
...settings,
providerOptions: { xai: { reasoningEffort: "high" } },
})
const openrouter = OpenRouter.model(
packageInput("anthropic/claude-sonnet-4", {
...settings,
providerOptions: { usage: true },
}),
)
const xai = XAI.model(
packageInput("grok-4", {
...settings,
providerOptions: { reasoningEffort: "high" },
}),
)
for (const selected of [openrouter, xai]) {
expect(selected.route.endpoint.baseURL).toBe(settings.baseURL)
@@ -60,37 +130,156 @@ describe("provider package entrypoints", () => {
expect(selected.route.defaults.http?.body).toEqual(settings.body)
expect(selected.route.defaults.limits).toEqual(settings.limits)
}
expect(openrouter.route.defaults.providerOptions).toEqual({ openrouter: { usage: true } })
expect(xai.route.defaults.providerOptions).toMatchObject({ xai: { reasoningEffort: "high", store: false } })
expect(openrouter.route.defaults.providerOptions).toEqual({ usage: true })
expect(xai.route.defaults.providerOptions).toMatchObject({ reasoningEffort: "high", store: false })
})
test("maps package settings onto the executable model", () => {
const selected = model("gpt-5", {
apiKey: "fixture",
baseURL: "https://api.openai.test/v1",
headers: { "x-application": "opencode" },
body: { service_tier: "priority" },
limits: { context: 200_000, output: 64_000 },
unrelatedInheritedSetting: true,
})
const selected = model(
packageInput("gpt-5", {
apiKey: "fixture",
baseURL: "https://api.openai.test/v1",
headers: { "x-application": "opencode" },
body: { service_tier: "priority" },
limits: { context: 200_000, output: 64_000 },
reasoningEffort: "high",
unrelatedInheritedSetting: true,
}),
)
expect(selected.route.id).toBe("openai-responses")
expect(selected.route.defaults.headers).toEqual({ "x-application": "opencode" })
expect(selected.route.defaults.http?.body).toEqual({ service_tier: "priority" })
expect(selected.route.defaults.limits).toEqual({ context: 200_000, output: 64_000 })
expect(selected.route.defaults.providerOptions).toEqual({
store: false,
reasoningEffort: "high",
reasoningSummary: "auto",
include: ["reasoning.encrypted_content"],
})
})
test("lets provider packages interpret resolved credentials", async () => {
const Anthropic = await import("@opencode-ai/ai/providers/anthropic")
const Azure = await import("@opencode-ai/ai/providers/azure")
const Google = await import("@opencode-ai/ai/providers/google")
const GoogleVertex = await import("@opencode-ai/ai/providers/google-vertex")
const GoogleVertexChat = await import("@opencode-ai/ai/providers/google-vertex/chat")
const openai = model({
id: "gpt-5",
settings: {},
credential: { type: "oauth", accessToken: "openai-token" },
defaults: {},
})
const anthropicKey = Anthropic.model({
id: "claude-sonnet-4-6",
settings: {},
credential: { type: "key", value: "anthropic-key" },
defaults: {},
})
const anthropicOAuth = Anthropic.model({
id: "claude-sonnet-4-6",
settings: {},
credential: { type: "oauth", accessToken: "anthropic-token" },
defaults: {},
})
const anthropicEmptyKey = Anthropic.model({
id: "claude-sonnet-4-6",
settings: {},
credential: { type: "key", value: "" },
defaults: {},
})
const azureKey = Azure.model({
id: "deployment",
settings: { resourceName: "opencode-test" },
credential: { type: "key", value: "azure-key" },
defaults: {},
})
const azureOAuth = Azure.model({
id: "deployment",
settings: { resourceName: "opencode-test" },
credential: { type: "oauth", accessToken: "azure-token" },
defaults: {},
})
const googleKey = Google.model({
id: "gemini-2.5-flash",
settings: {},
credential: { type: "key", value: "google-key" },
defaults: {},
})
const googleOAuth = Google.model({
id: "gemini-2.5-flash",
settings: {},
credential: { type: "oauth", accessToken: "google-token" },
defaults: {},
})
const vertexKey = GoogleVertex.model({
id: "gemini-3.5-flash",
settings: {},
credential: { type: "key", value: "vertex-key" },
defaults: {},
})
const vertexOAuth = GoogleVertex.model({
id: "gemini-3.5-flash",
settings: { project: "vertex-project" },
credential: { type: "oauth", accessToken: "vertex-token" },
defaults: {},
})
const vertexChatOAuth = GoogleVertexChat.model({
id: "deepseek-ai/deepseek-v3.2-maas",
settings: { apiKey: "configured-key", project: "vertex-project" },
credential: { type: "oauth", accessToken: "vertex-chat-token" },
defaults: {},
})
expect((await authHeaders(openai)).authorization).toBe("Bearer openai-token")
const anthropicKeyHeaders = await authHeaders(anthropicKey, { authorization: "Bearer stale" })
const anthropicOAuthHeaders = await authHeaders(anthropicOAuth, { "x-api-key": "stale" })
const anthropicEmptyKeyHeaders = await authHeaders(
anthropicEmptyKey,
{ authorization: "Bearer stale" },
{ ANTHROPIC_API_KEY: "environment-key" },
)
const azureKeyHeaders = await authHeaders(azureKey, { authorization: "Bearer stale" })
const azureOAuthHeaders = await authHeaders(azureOAuth, { "api-key": "stale" })
const googleKeyHeaders = await authHeaders(googleKey, { authorization: "Bearer stale" })
const googleOAuthHeaders = await authHeaders(googleOAuth, { "x-goog-api-key": "stale" })
const vertexKeyHeaders = await authHeaders(vertexKey, { authorization: "Bearer stale" })
const vertexOAuthHeaders = await authHeaders(vertexOAuth, { "x-goog-api-key": "stale" })
expect(anthropicKeyHeaders["x-api-key"]).toBe("anthropic-key")
expect(anthropicKeyHeaders.authorization).toBeUndefined()
expect(anthropicOAuthHeaders.authorization).toBe("Bearer anthropic-token")
expect(anthropicOAuthHeaders["x-api-key"]).toBeUndefined()
expect(anthropicEmptyKeyHeaders["x-api-key"]).toBe("environment-key")
expect(anthropicEmptyKeyHeaders.authorization).toBeUndefined()
expect(azureKeyHeaders["api-key"]).toBe("azure-key")
expect(azureKeyHeaders.authorization).toBeUndefined()
expect(azureOAuthHeaders.authorization).toBe("Bearer azure-token")
expect(azureOAuthHeaders["api-key"]).toBeUndefined()
expect(googleKeyHeaders["x-goog-api-key"]).toBe("google-key")
expect(googleKeyHeaders.authorization).toBeUndefined()
expect(googleOAuthHeaders.authorization).toBe("Bearer google-token")
expect(googleOAuthHeaders["x-goog-api-key"]).toBeUndefined()
expect(vertexKeyHeaders["x-goog-api-key"]).toBe("vertex-key")
expect(vertexKeyHeaders.authorization).toBeUndefined()
expect(vertexOAuthHeaders.authorization).toBe("Bearer vertex-token")
expect(vertexOAuthHeaders["x-goog-api-key"]).toBeUndefined()
expect((await authHeaders(vertexChatOAuth)).authorization).toBe("Bearer vertex-chat-token")
})
test("maps OpenAI-compatible Responses settings onto the executable model", async () => {
const OpenAICompatibleResponses = await import("@opencode-ai/ai/providers/openai-compatible/responses")
const selected = OpenAICompatibleResponses.model("custom-model", {
apiKey: "fixture",
baseURL: "https://responses.example.test/v1",
provider: "example",
headers: { "x-application": "opencode" },
body: { service_tier: "priority" },
limits: { context: 200_000, output: 64_000 },
providerOptions: { openresponses: { reasoningEffort: "low", store: true } },
})
const selected = OpenAICompatibleResponses.model(
packageInput("custom-model", {
apiKey: "fixture",
baseURL: "https://responses.example.test/v1",
provider: "example",
headers: { "x-application": "opencode" },
body: { service_tier: "priority" },
limits: { context: 200_000, output: 64_000 },
providerOptions: { reasoningEffort: "low", store: true },
}),
)
expect(String(selected.provider)).toBe("example")
expect(selected.route.id).toBe("openai-compatible-responses")
@@ -101,22 +290,22 @@ describe("provider package entrypoints", () => {
expect(selected.route.defaults.headers).toEqual({ "x-application": "opencode" })
expect(selected.route.defaults.http?.body).toEqual({ service_tier: "priority" })
expect(selected.route.defaults.limits).toEqual({ context: 200_000, output: 64_000 })
expect(selected.route.defaults.providerOptions).toEqual({
openresponses: { reasoningEffort: "low", store: true },
})
expect(selected.route.defaults.providerOptions).toEqual({ reasoningEffort: "low", store: true })
})
test("maps Anthropic-compatible settings onto the executable model", async () => {
const AnthropicCompatible = await import("@opencode-ai/ai/providers/anthropic-compatible")
const selected = AnthropicCompatible.model("compatible-model", {
apiKey: "fixture",
baseURL: "https://messages.example.test/v1",
provider: "example",
headers: { "x-application": "opencode" },
body: { metadata: { user_id: "user_1" } },
limits: { context: 200_000, output: 64_000 },
providerOptions: { anthropic: { effort: "low" } },
})
const selected = AnthropicCompatible.model(
packageInput("compatible-model", {
apiKey: "fixture",
baseURL: "https://messages.example.test/v1",
provider: "example",
headers: { "x-application": "opencode" },
body: { metadata: { user_id: "user_1" } },
limits: { context: 200_000, output: 64_000 },
providerOptions: { effort: "low" },
}),
)
expect(String(selected.provider)).toBe("example")
expect(selected.route.id).toBe("anthropic-messages")
@@ -127,25 +316,25 @@ describe("provider package entrypoints", () => {
expect(selected.route.defaults.headers).toEqual({ "x-application": "opencode" })
expect(selected.route.defaults.http?.body).toEqual({ metadata: { user_id: "user_1" } })
expect(selected.route.defaults.limits).toEqual({ context: 200_000, output: 64_000 })
expect(selected.route.defaults.providerOptions).toEqual({ anthropic: { effort: "low" } })
expect(selected.route.defaults.providerOptions).toEqual({ effort: "low" })
})
test("maps Anthropic provider options onto the executable model", async () => {
const Anthropic = await import("@opencode-ai/ai/providers/anthropic")
const selected = Anthropic.model("claude-sonnet-4-6", {
apiKey: "fixture",
providerOptions: { anthropic: { thinking: { type: "adaptive" } } },
})
const selected = Anthropic.model(
packageInput("claude-sonnet-4-6", {
apiKey: "fixture",
providerOptions: { thinking: { type: "adaptive" } },
}),
)
expect(selected.route.defaults.providerOptions).toEqual({
anthropic: { thinking: { type: "adaptive" } },
})
expect(selected.route.defaults.providerOptions).toEqual({ thinking: { type: "adaptive" } })
})
test("requires an Anthropic-compatible base URL at runtime", async () => {
const AnthropicCompatible = await import("@opencode-ai/ai/providers/anthropic-compatible")
expect(() =>
Reflect.apply(AnthropicCompatible.model, undefined, ["compatible-model", { apiKey: "fixture" }]),
Reflect.apply(AnthropicCompatible.model, undefined, [packageInput("compatible-model", { apiKey: "fixture" })]),
).toThrow("Anthropic-compatible providers require a baseURL")
})
@@ -154,25 +343,28 @@ describe("provider package entrypoints", () => {
const AnthropicCompatible = await import("@opencode-ai/ai/providers/anthropic-compatible")
expect(() =>
Reflect.apply(AnthropicCompatible.model, undefined, [
"compatible-model",
{
packageInput("compatible-model", {
apiKey: "fixture",
authToken: "token",
baseURL: "https://messages.example.test/v1",
},
}),
]),
).toThrow("Anthropic-compatible apiKey cannot be combined with authToken")
expect(() =>
Reflect.apply(Anthropic.model, undefined, ["claude-sonnet-4-6", { apiKey: "fixture", authToken: "token" }]),
Reflect.apply(Anthropic.model, undefined, [
packageInput("claude-sonnet-4-6", { apiKey: "fixture", authToken: "token" }),
]),
).toThrow("Anthropic apiKey cannot be combined with authToken")
})
test("maps legacy OpenAI organization and project settings to headers", () => {
const selected = model("gpt-5", {
apiKey: "fixture",
organization: "org_123",
project: "proj_123",
})
const selected = model(
packageInput("gpt-5", {
apiKey: "fixture",
organization: "org_123",
project: "proj_123",
}),
)
expect(selected.route.defaults.headers).toMatchObject({
"OpenAI-Organization": "org_123",
@@ -192,10 +384,10 @@ describe("provider package entrypoints", () => {
limits: { context: 200_000, output: 64_000 },
}
const responses = AzureResponses.model("deployment", settings)
const chat = AzureChat.model("deployment", settings)
const responses = AzureResponses.model(packageInput("deployment", settings))
const chat = AzureChat.model(packageInput("deployment", settings))
expect(Azure.model("deployment", settings).route.id).toBe("azure-openai-responses")
expect(Azure.model(packageInput("deployment", settings)).route.id).toBe("azure-openai-responses")
expect(responses.route.id).toBe("azure-openai-responses")
expect(responses.route.endpoint.baseURL).toBe("https://opencode-test.openai.azure.com/openai/v1")
expect(responses.route.defaults.headers).toEqual({ "x-application": "opencode" })
@@ -206,16 +398,20 @@ describe("provider package entrypoints", () => {
test("constructs Azure deployment URLs and preserves custom gateway URLs", async () => {
const Azure = await import("@opencode-ai/ai/providers/azure")
const deployment = Azure.model("custom-deployment", {
apiKey: "fixture",
resourceName: "opencode-test",
apiVersion: "2025-01-01-preview",
useDeploymentBasedUrls: true,
})
const gateway = Azure.model("gateway-model", {
apiKey: "fixture",
baseURL: "https://gateway.example/azure/",
})
const deployment = Azure.model(
packageInput("custom-deployment", {
apiKey: "fixture",
resourceName: "opencode-test",
apiVersion: "2025-01-01-preview",
useDeploymentBasedUrls: true,
}),
)
const gateway = Azure.model(
packageInput("gateway-model", {
apiKey: "fixture",
baseURL: "https://gateway.example/azure/",
}),
)
expect(deployment.route.endpoint).toMatchObject({
baseURL: "https://opencode-test.openai.azure.com/openai/deployments/custom-deployment",
@@ -227,23 +423,23 @@ describe("provider package entrypoints", () => {
test("maps Google package settings onto the Gemini model", async () => {
const Google = await import("@opencode-ai/ai/providers/google")
const selected = Google.model("gemini-2.5-flash", {
apiKey: "fixture",
baseURL: "https://generativelanguage.test/v1beta",
headers: { "x-application": "opencode" },
body: { safetySettings: [] },
limits: { context: 1_000_000, output: 65_536 },
providerOptions: { gemini: { thinkingConfig: { thinkingBudget: 1_024 } } },
})
const selected = Google.model(
packageInput("gemini-2.5-flash", {
apiKey: "fixture",
baseURL: "https://generativelanguage.test/v1beta",
headers: { "x-application": "opencode" },
body: { safetySettings: [] },
limits: { context: 1_000_000, output: 65_536 },
providerOptions: { thinkingConfig: { thinkingBudget: 1_024 } },
}),
)
expect(selected.route.id).toBe("gemini")
expect(selected.route.endpoint.baseURL).toBe("https://generativelanguage.test/v1beta")
expect(selected.route.defaults.headers).toEqual({ "x-application": "opencode" })
expect(selected.route.defaults.http?.body).toEqual({ safetySettings: [] })
expect(selected.route.defaults.limits).toEqual({ context: 1_000_000, output: 65_536 })
expect(selected.route.defaults.providerOptions).toEqual({
gemini: { thinkingConfig: { thinkingBudget: 1_024 } },
})
expect(selected.route.defaults.providerOptions).toEqual({ thinkingConfig: { thinkingBudget: 1_024 } })
})
test("selects Vertex entrypoints with the same model contract", async () => {
@@ -252,27 +448,35 @@ describe("provider package entrypoints", () => {
const GoogleVertexChat = await import("@opencode-ai/ai/providers/google-vertex/chat")
const GoogleVertexResponses = await import("@opencode-ai/ai/providers/google-vertex/responses")
const GoogleVertexMessages = await import("@opencode-ai/ai/providers/google-vertex/messages")
const gemini = GoogleVertex.model("gemini-3.5-flash", {
apiKey: "fixture",
headers: { "x-application": "opencode" },
body: { safetySettings: [] },
limits: { context: 1_000_000, output: 65_536 },
})
const messages = GoogleVertexMessages.model("claude-sonnet-4-6", {
accessToken: "fixture",
location: "global",
project: "vertex-project",
})
const chat = GoogleVertexChat.model("deepseek-ai/deepseek-v3.2-maas", {
accessToken: "fixture",
location: "global",
project: "vertex-project",
})
const responses = GoogleVertexResponses.model("xai/grok-4.20-reasoning", {
accessToken: "fixture",
location: "global",
project: "vertex-project",
})
const gemini = GoogleVertex.model(
packageInput("gemini-3.5-flash", {
apiKey: "fixture",
headers: { "x-application": "opencode" },
body: { safetySettings: [] },
limits: { context: 1_000_000, output: 65_536 },
}),
)
const messages = GoogleVertexMessages.model(
packageInput("claude-sonnet-4-6", {
accessToken: "fixture",
location: "global",
project: "vertex-project",
}),
)
const chat = GoogleVertexChat.model(
packageInput("deepseek-ai/deepseek-v3.2-maas", {
accessToken: "fixture",
location: "global",
project: "vertex-project",
}),
)
const responses = GoogleVertexResponses.model(
packageInput("xai/grok-4.20-reasoning", {
accessToken: "fixture",
location: "global",
project: "vertex-project",
}),
)
expect(GoogleVertexGemini.model).toBe(GoogleVertex.model)
expect(gemini.route.id).toBe("google-vertex-gemini")
@@ -282,11 +486,13 @@ describe("provider package entrypoints", () => {
expect(gemini.route.defaults.http?.body).toEqual({ safetySettings: [] })
expect(gemini.route.defaults.limits).toEqual({ context: 1_000_000, output: 65_536 })
expect(
GoogleVertex.model("gemini-3.5-flash", {
accessToken: "fixture",
location: "eu",
project: "vertex-project",
}).route.endpoint.baseURL,
GoogleVertex.model(
packageInput("gemini-3.5-flash", {
accessToken: "fixture",
location: "eu",
project: "vertex-project",
}),
).route.endpoint.baseURL,
).toBe("https://aiplatform.eu.rep.googleapis.com/v1beta1/projects/vertex-project/locations/eu/publishers/google")
expect(messages.route.id).toBe("google-vertex-messages")
expect(messages.route.protocol).toBe("anthropic-messages")
@@ -305,7 +511,7 @@ describe("provider package entrypoints", () => {
baseURL: "https://aiplatform.googleapis.com/v1/projects/vertex-project/locations/global/endpoints/openapi",
path: "/responses",
})
expect(responses.route.defaults.providerOptions).toEqual({ openresponses: { store: false } })
expect(responses.route.defaults.providerOptions).toEqual({ store: false })
})
test("rejects conflicting Vertex auth settings at runtime", async () => {
@@ -316,8 +522,11 @@ describe("provider package entrypoints", () => {
const Providers = await import("@opencode-ai/ai/providers")
expect(() =>
Reflect.apply(GoogleVertex.model, undefined, [
"gemini-3.5-flash",
{ accessToken: "token", apiKey: "fixture", project: "vertex-project" },
packageInput("gemini-3.5-flash", {
accessToken: "token",
apiKey: "fixture",
project: "vertex-project",
}),
]),
).toThrow("Google Vertex apiKey cannot be combined with accessToken or auth")
const configured = Reflect.apply(GoogleVertex.configure, undefined, [
@@ -326,8 +535,7 @@ describe("provider package entrypoints", () => {
expect(() => configured.model("gemini-3.5-flash")).toThrow("Google Vertex accessToken cannot be combined with auth")
expect(() =>
Reflect.apply(GoogleVertexMessages.model, undefined, [
"claude-sonnet-4-6",
{ apiKey: "fixture", project: "vertex-project" },
packageInput("claude-sonnet-4-6", { apiKey: "fixture", project: "vertex-project" }),
]),
).toThrow("Google Vertex Messages does not support API keys")
expect(() =>
@@ -337,8 +545,7 @@ describe("provider package entrypoints", () => {
).toThrow("Google Vertex Messages does not support API keys")
expect(() =>
Reflect.apply(GoogleVertexChat.model, undefined, [
"deepseek-ai/deepseek-v3.2-maas",
{ apiKey: "fixture", project: "vertex-project" },
packageInput("deepseek-ai/deepseek-v3.2-maas", { apiKey: "fixture", project: "vertex-project" }),
]),
).toThrow("Google Vertex Chat does not support API keys")
expect(() =>
@@ -348,8 +555,7 @@ describe("provider package entrypoints", () => {
).toThrow("Google Vertex Chat does not support API keys")
expect(() =>
Reflect.apply(GoogleVertexResponses.model, undefined, [
"xai/grok-4.20-reasoning",
{ apiKey: "fixture", project: "vertex-project" },
packageInput("xai/grok-4.20-reasoning", { apiKey: "fixture", project: "vertex-project" }),
]),
).toThrow("Google Vertex Responses does not support API keys")
expect(() =>
@@ -63,7 +63,8 @@ describe("Anthropic Messages route", () => {
const prepared = yield* compileRequest(
LLMRequest.update(request, {
providerOptions: {
anthropic: { thinking: { type: "adaptive", display: "summarized" }, effort: "low" },
thinking: { type: "adaptive", display: "summarized" },
effort: "low",
},
}),
)
@@ -79,17 +80,17 @@ describe("Anthropic Messages route", () => {
Effect.gen(function* () {
const enabled = yield* compileRequest(
LLMRequest.update(request, {
providerOptions: { anthropic: { thinking: { type: "enabled", budgetTokens: 1_024 } } },
providerOptions: { thinking: { type: "enabled", budgetTokens: 1_024 } },
}),
)
const legacy = yield* compileRequest(
LLMRequest.update(request, {
providerOptions: { anthropic: { thinking: { type: "enabled", budget_tokens: 2_048 } } },
providerOptions: { thinking: { type: "enabled", budget_tokens: 2_048 } },
}),
)
const disabled = yield* compileRequest(
LLMRequest.update(request, {
providerOptions: { anthropic: { thinking: { type: "disabled" } } },
providerOptions: { thinking: { type: "disabled" } },
}),
)
@@ -103,7 +104,7 @@ describe("Anthropic Messages route", () => {
Effect.gen(function* () {
const error = yield* compileRequest(
LLMRequest.update(request, {
providerOptions: { anthropic: { thinking: { type: "enabled" } } },
providerOptions: { thinking: { type: "enabled" } },
}),
).pipe(Effect.flip)
@@ -1062,9 +1063,7 @@ describe("Anthropic Messages route", () => {
),
)
expect(response.toolCalls).toMatchObject([
{ id: "call_1", name: "lookup", input: { query: "weather" } },
])
expect(response.toolCalls).toMatchObject([{ id: "call_1", name: "lookup", input: { query: "weather" } }])
expect(response.finishReason).toEqual({ normalized: "tool-calls", raw: "tool_use" })
}),
)
+7 -9
View File
@@ -48,28 +48,26 @@ describe("Gemini route", () => {
const prepared = yield* compileRequest(
LLMRequest.update(request, {
providerOptions: {
gemini: {
cachedContent: "cachedContents/example",
safetySettings: [{ category: "HARM_CATEGORY_HATE_SPEECH", threshold: "BLOCK_ONLY_HIGH" }],
serviceTier: "priority",
thinkingConfig: { thinkingBudget: 0, includeThoughts: false, thinkingLevel: "high" },
},
cachedContent: "cachedContents/example",
safetySettings: [{ category: "HARM_CATEGORY_HATE_SPEECH", threshold: "BLOCK_ONLY_HIGH" }],
serviceTier: "priority",
thinkingConfig: { thinkingBudget: 0, includeThoughts: false, thinkingLevel: "high" },
},
}),
)
const filtered = yield* compileRequest(
LLMRequest.update(request, {
providerOptions: { gemini: { thinkingConfig: { thinkingBudget: "invalid", includeThoughts: false } } },
providerOptions: { thinkingConfig: { thinkingBudget: "invalid", includeThoughts: false } },
}),
)
const defaulted = yield* compileRequest(
LLMRequest.update(request, {
providerOptions: { gemini: { thinkingConfig: { thinkingLevel: "high" } } },
providerOptions: { thinkingConfig: { thinkingLevel: "high" } },
}),
)
const emptySafetySettings = yield* compileRequest(
LLMRequest.update(request, {
providerOptions: { gemini: { safetySettings: [] } },
providerOptions: { safetySettings: [] },
}),
)
@@ -62,7 +62,7 @@ describe("Google Vertex providers", () => {
accessToken: "vertex-token",
project: "vertex-project",
providerOptions: {
gemini: { labels: { component: "opencode", environment: "test" } },
labels: { component: "opencode", environment: "test" },
},
}).model("gemini-3.5-flash"),
prompt: "Say hello.",
@@ -15,7 +15,7 @@ const cases = [
model: LanguageModel.update(
OpenRouter.configure({
apiKey: process.env.OPENROUTER_API_KEY ?? "fixture",
providerOptions: { openrouter: { reasoning: { max_tokens: 1024 } } },
providerOptions: { reasoning: { max_tokens: 1024 } },
}).model("anthropic/claude-sonnet-4.6"),
{ compatibility: { reasoningField: "reasoning" } },
),
@@ -165,7 +165,7 @@ describe("OpenAI Chat route", () => {
LLM.request({
model: OpenAI.configure({ baseURL: "https://api.openai.test/v1/", apiKey: "test" }).chat("gpt-4o-mini"),
prompt: "think",
providerOptions: { openai: { reasoningEffort: "max" } },
providerOptions: { reasoningEffort: "max" },
}),
)
@@ -221,7 +221,7 @@ describe("OpenAI Chat route", () => {
LLM.request({
model,
prompt: "think",
providerOptions: { openai: { reasoningEffort: "experimental" } },
providerOptions: { reasoningEffort: "experimental" },
}),
)
@@ -118,20 +118,18 @@ describe("Open Responses-compatible route", () => {
}),
)
it.effect("reads standard options from the Open Responses namespace", () =>
it.effect("reads standard Open Responses options", () =>
Effect.gen(function* () {
const model = configure({
apiKey: "test-key",
baseURL: "https://responses.example.test/v1",
providerOptions: {
openresponses: {
reasoningEffort: "low",
store: true,
truncation: "auto",
allowedTools: { toolNames: ["lookup"] },
maxToolCalls: 2,
parallelToolCalls: false,
},
reasoningEffort: "low",
store: true,
truncation: "auto",
allowedTools: { toolNames: ["lookup"] },
maxToolCalls: 2,
parallelToolCalls: false,
},
}).model("example-model")
const prepared = yield* compileRequest(
@@ -159,8 +159,8 @@ describe("OpenAI Responses route", () => {
it.effect("lowers semantic service tier options", () =>
Effect.gen(function* () {
const input = LLMRequest.update(request, { providerOptions: { openai: { serviceTier: "priority" } } })
expect(input.providerOptions).toEqual({ openai: { serviceTier: "priority" } })
const input = LLMRequest.update(request, { providerOptions: { serviceTier: "priority" } })
expect(input.providerOptions).toEqual({ serviceTier: "priority" })
const prepared = yield* compileRequest(input)
expect(prepared.body).toMatchObject({ service_tier: "priority" })
@@ -171,7 +171,7 @@ describe("OpenAI Responses route", () => {
it.effect("passes through custom OpenAI reasoning effort strings", () =>
Effect.gen(function* () {
const prepared = yield* compileRequest(
LLMRequest.update(request, { providerOptions: { openai: { reasoningEffort: "experimental" } } }),
LLMRequest.update(request, { providerOptions: { reasoningEffort: "experimental" } }),
)
expect(prepared.body.reasoning).toEqual({ effort: "experimental" })
@@ -181,7 +181,7 @@ describe("OpenAI Responses route", () => {
it.effect("omits unsupported semantic service tiers", () =>
Effect.gen(function* () {
const prepared = yield* compileRequest(
LLMRequest.update(request, { providerOptions: { openai: { serviceTier: "unsupported" } } }),
LLMRequest.update(request, { providerOptions: { serviceTier: "unsupported" } }),
)
expect(prepared.body).not.toHaveProperty("service_tier")
@@ -1280,15 +1280,13 @@ describe("OpenAI Responses route", () => {
],
toolChoice: "none",
providerOptions: {
openai: {
reasoningEffort: "high",
reasoningSummary: "auto",
include: ["reasoning.encrypted_content"],
truncation: "disabled",
allowedTools: { toolNames: ["read", "grep"], mode: "required" },
maxToolCalls: 4,
parallelToolCalls: false,
},
reasoningEffort: "high",
reasoningSummary: "auto",
include: ["reasoning.encrypted_content"],
truncation: "disabled",
allowedTools: { toolNames: ["read", "grep"], mode: "required" },
maxToolCalls: 4,
parallelToolCalls: false,
},
}),
)
@@ -1319,9 +1317,7 @@ describe("OpenAI Responses route", () => {
model,
prompt: "hi",
providerOptions: {
openai: {
include: ["reasoning.encrypted_content", "code_interpreter_call.outputs", "web_search_call.results"],
},
include: ["reasoning.encrypted_content", "code_interpreter_call.outputs", "web_search_call.results"],
},
}),
)
@@ -1340,7 +1336,7 @@ describe("OpenAI Responses route", () => {
LLM.request({
model,
prompt: "hi",
providerOptions: { openai: { include: ["reasoning.encrypted_content", "bogus.thing"] } },
providerOptions: { include: ["reasoning.encrypted_content", "bogus.thing"] },
}),
)
@@ -1350,9 +1346,7 @@ describe("OpenAI Responses route", () => {
it.effect("treats an explicit empty include as no include at all", () =>
Effect.gen(function* () {
const prepared = yield* compileRequest(
LLM.request({ model, prompt: "hi", providerOptions: { openai: { include: [] } } }),
)
const prepared = yield* compileRequest(LLM.request({ model, prompt: "hi", providerOptions: { include: [] } }))
expect(prepared.body.include).toBeUndefined()
}),
@@ -1361,7 +1355,7 @@ describe("OpenAI Responses route", () => {
it.effect("passes an unknown includable value through", () =>
Effect.gen(function* () {
const prepared = yield* compileRequest(
LLM.request({ model, prompt: "hi", providerOptions: { openai: { include: ["bogus.thing"] } } }),
LLM.request({ model, prompt: "hi", providerOptions: { include: ["bogus.thing"] } }),
)
expect(prepared.body.include).toEqual(["bogus.thing"])
@@ -1370,9 +1364,7 @@ describe("OpenAI Responses route", () => {
it.effect("omits include when no include is set", () =>
Effect.gen(function* () {
const prepared = yield* compileRequest(
LLM.request({ model, prompt: "hi", providerOptions: { openai: { store: false } } }),
)
const prepared = yield* compileRequest(LLM.request({ model, prompt: "hi", providerOptions: { store: false } }))
expect(prepared.body.include).toBeUndefined()
}),
@@ -1403,7 +1395,7 @@ describe("OpenAI Responses route", () => {
LLM.request({
model: OpenAI.configure({ baseURL: "https://api.openai.test/v1/", apiKey: "test" }).responses("gpt-5.2"),
prompt: "hi",
providerOptions: { openai: { include: [] } },
providerOptions: { include: [] },
}),
)
@@ -1727,7 +1719,7 @@ describe("OpenAI Responses route", () => {
it.effect("streams each reasoning summary part as a separate block", () =>
Effect.gen(function* () {
const response = yield* LLMClient.generate(
LLMRequest.update(request, { providerOptions: { openai: { store: false } } }),
LLMRequest.update(request, { providerOptions: { store: false } }),
).pipe(
Effect.provide(
fixedResponse(
@@ -1781,9 +1773,7 @@ describe("OpenAI Responses route", () => {
it.effect("closes reasoning summary parts when storage is not disabled", () =>
Effect.gen(function* () {
const response = yield* LLMClient.generate(
LLMRequest.update(request, { providerOptions: { openai: { store: true } } }),
).pipe(
const response = yield* LLMClient.generate(LLMRequest.update(request, { providerOptions: { store: true } })).pipe(
Effect.provide(
fixedResponse(
sseEvents(
@@ -1837,7 +1827,7 @@ describe("OpenAI Responses route", () => {
]),
Message.user("Summarize it."),
],
providerOptions: { openai: { store: false } },
providerOptions: { store: false },
}),
).pipe(
Effect.provide(
@@ -1896,7 +1886,7 @@ describe("OpenAI Responses route", () => {
{ type: "text", text: "After." },
]),
],
providerOptions: { openai: { store: false } },
providerOptions: { store: false },
}),
)
@@ -1926,7 +1916,7 @@ describe("OpenAI Responses route", () => {
},
]),
],
providerOptions: { openai: { store: true } },
providerOptions: { store: true },
}),
)
@@ -1959,7 +1949,7 @@ describe("OpenAI Responses route", () => {
]),
Message.user("Continue."),
],
providerOptions: { openai: { store: true } },
providerOptions: { store: true },
}),
)
@@ -2032,7 +2022,7 @@ describe("OpenAI Responses route", () => {
},
]),
],
providerOptions: { openai: { store: false } },
providerOptions: { store: false },
}),
)
@@ -2072,7 +2062,7 @@ describe("OpenAI Responses route", () => {
]),
Message.user("Summarize it."),
],
providerOptions: { openai: { store: false } },
providerOptions: { store: false },
}),
)
+11 -13
View File
@@ -141,7 +141,7 @@ describe("OpenRouter", () => {
LLM.request({
model: OpenRouter.configure({
apiKey: "test-key",
providerOptions: { openrouter: { usage: false } },
providerOptions: { usage: false },
}).model("openai/gpt-4o-mini"),
cache: "none",
prompt: "Hello",
@@ -159,17 +159,15 @@ describe("OpenRouter", () => {
model: OpenRouter.configure({
apiKey: "test-key",
providerOptions: {
openrouter: {
usage: true,
reasoning: { effort: "high" },
models: ["anthropic/claude-sonnet-4.6", "google/gemini-3.1-pro"],
provider: { order: ["anthropic", "google"], require_parameters: true },
plugins: [{ id: "response-healing" }],
web_search_options: { engine: "native", max_results: 3 },
debug: { echo_upstream_body: true },
user: "user_123",
future_option: { enabled: true },
},
usage: true,
reasoning: { effort: "high" },
models: ["anthropic/claude-sonnet-4.6", "google/gemini-3.1-pro"],
provider: { order: ["anthropic", "google"], require_parameters: true },
plugins: [{ id: "response-healing" }],
web_search_options: { engine: "native", max_results: 3 },
debug: { echo_upstream_body: true },
user: "user_123",
future_option: { enabled: true },
},
}).model("anthropic/claude-3.7-sonnet:thinking"),
prompt: "Think briefly.",
@@ -210,7 +208,7 @@ describe("OpenRouter", () => {
LLM.request({
model: OpenRouter.configure({
apiKey: "test-key",
providerOptions: { openrouter: invalid },
providerOptions: invalid,
}).model("openai/gpt-4o-mini"),
prompt: "Hello",
}),
+6 -9
View File
@@ -181,12 +181,10 @@ const normalizeImageText = (value: string) =>
.trim()
const encryptedReasoningOptions = {
openai: {
store: false,
include: ["reasoning.encrypted_content"],
reasoningEffort: "low",
reasoningSummary: "auto",
},
store: false,
include: ["reasoning.encrypted_content"],
reasoningEffort: "low",
reasoningSummary: "auto",
} as const
type AssistantTextExpectation = string | RegExp
@@ -304,8 +302,7 @@ const runTextScenario = (context: GoldenScenarioContext) =>
assistant.expectText(/^Hello!?$/, {
system: "You are concise.",
maxTokens: context.maxTokens ?? 40,
providerOptions:
context.model.route.id === "gemini" ? { gemini: { thinkingConfig: { thinkingBudget: 0 } } } : undefined,
providerOptions: context.model.route.id === "gemini" ? { thinkingConfig: { thinkingBudget: 0 } } : undefined,
}),
])
@@ -388,7 +385,7 @@ const runReasoningScenario = (context: GoldenScenarioContext) =>
user("Think briefly, then reply exactly with: Hello!"),
assistant.expectText(/^Hello!?$/, {
system: "Show concise reasoning when the provider supports visible reasoning summaries.",
providerOptions: { openai: { reasoningEffort: "low", reasoningSummary: "auto" } },
providerOptions: { reasoningEffort: "low", reasoningSummary: "auto" },
maxTokens: context.maxTokens ?? 120,
assert: (response) => expect(response.usage?.reasoningTokens ?? 0).toBeGreaterThan(0),
}),
+2 -2
View File
@@ -55,7 +55,7 @@ const schema_only_weather = Tool.make({
})
describe("LLMClient tools", () => {
it.effect("uses the registered model route when adding runtime tools", () =>
it.effect("uses the selected model route when adding runtime tools", () =>
Effect.gen(function* () {
const layer = scriptedResponses([
sseEvents(deltaChunk({ role: "assistant", content: "Done." }), finishChunk("stop")),
@@ -636,7 +636,7 @@ describe("LLMClient tools", () => {
.with({ endpoint: { baseURL: "https://api.openai.test/v1/" }, auth: Auth.bearer("test") })
.model({ id: "gpt-5.5" }),
prompt: "Use the tool.",
providerOptions: { openai: { store: false, include: ["reasoning.encrypted_content"] } },
providerOptions: { store: false, include: ["reasoning.encrypted_content"] },
}),
tools: { get_weather },
}).pipe(Stream.runCollect, Effect.provide(layer))
+53 -61
View File
@@ -20,16 +20,48 @@ export interface MapInput {
export function map(input: MapInput): Mapping | undefined {
const baseSettings = mapBaseSettings(input.settings)
switch (input.packageName) {
case "@ai-sdk/anthropic":
case "@ai-sdk/openai":
return {
package: "@opencode-ai/ai/providers/openai",
settings: {
...baseSettings,
...mapAPIKey(input.settings),
...(typeof input.settings.organization === "string" ? { organization: input.settings.organization } : {}),
...(typeof input.settings.project === "string" ? { project: input.settings.project } : {}),
...(isStringRecord(input.settings.queryParams) ? { queryParams: input.settings.queryParams } : {}),
...openAIOptions(input.settings),
},
...(isStringRecord(input.settings.headers) ? { headers: input.settings.headers } : {}),
}
case "@ai-sdk/anthropic": {
const providerOptions = {
...(isRecord(input.settings.thinking) ? { thinking: input.settings.thinking } : {}),
...(typeof input.settings.effort === "string" ? { effort: input.settings.effort } : {}),
}
return {
package: "@opencode-ai/ai/providers/anthropic",
settings: {
...baseSettings,
...mapAPIKey(input.settings),
...(typeof input.settings.authToken === "string" ? { authToken: input.settings.authToken } : {}),
...mapAnthropicOptions(input.settings),
...(Object.keys(providerOptions).length === 0 ? {} : { providerOptions }),
},
...(isStringRecord(input.settings.headers) ? { headers: input.settings.headers } : {}),
}
}
case "@ai-sdk/openai-compatible":
return typeof input.settings.baseURL !== "string"
? undefined
: {
package: "@opencode-ai/ai/providers/openai-compatible",
settings: {
...baseSettings,
...mapAPIKey(input.settings),
provider: input.providerID,
...mapOpenAIOptions(input.settings),
},
...(isStringRecord(input.settings.headers) ? { headers: input.settings.headers } : {}),
}
case "@ai-sdk/amazon-bedrock":
return {
package: "@opencode-ai/ai/providers/amazon-bedrock",
@@ -71,10 +103,7 @@ export function map(input: MapInput): Mapping | undefined {
...mapAPIKey(input.settings),
...(typeof input.settings.location === "string" ? { location: input.settings.location } : {}),
...(typeof input.settings.project === "string" ? { project: input.settings.project } : {}),
...mapGoogleOptions(
input.settings,
isStringRecord(input.settings.labels) ? { labels: input.settings.labels } : {},
),
...mapGoogleOptions(input.settings),
},
...(isStringRecord(input.settings.headers) ? { headers: input.settings.headers } : {}),
}
@@ -89,45 +118,14 @@ export function map(input: MapInput): Mapping | undefined {
...(isRecord(input.settings.thinking) || typeof input.settings.effort === "string"
? {
providerOptions: {
anthropic: {
...(isRecord(input.settings.thinking) ? { thinking: input.settings.thinking } : {}),
...(typeof input.settings.effort === "string" ? { effort: input.settings.effort } : {}),
},
...(isRecord(input.settings.thinking) ? { thinking: input.settings.thinking } : {}),
...(typeof input.settings.effort === "string" ? { effort: input.settings.effort } : {}),
},
}
: {}),
},
...(isStringRecord(input.settings.headers) ? { headers: input.settings.headers } : {}),
}
case "@ai-sdk/openai":
return {
package: "@opencode-ai/ai/providers/openai",
settings: {
...baseSettings,
...mapAPIKey(input.settings),
...(typeof input.settings.organization === "string" ? { organization: input.settings.organization } : {}),
...(typeof input.settings.project === "string" ? { project: input.settings.project } : {}),
...(isStringRecord(input.settings.queryParams) ? { queryParams: input.settings.queryParams } : {}),
...mapProviderOptions(input.settings, "openai", [
"apiKey",
"baseURL",
"organization",
"project",
"queryParams",
]),
},
}
case "@ai-sdk/openai-compatible":
if (typeof input.settings.baseURL !== "string") return
return {
package: "@opencode-ai/ai/providers/openai-compatible",
settings: {
...baseSettings,
...mapAPIKey(input.settings),
provider: input.providerID,
...mapProviderOptions(input.settings, "openai", ["apiKey", "baseURL"]),
},
}
case "@openrouter/ai-sdk-provider":
return mapOpenRouter(input.settings, baseSettings)
case "@ai-sdk/xai":
@@ -142,20 +140,6 @@ export function map(input: MapInput): Mapping | undefined {
}
}
function mapAnthropicOptions(settings: Readonly<Record<string, unknown>>) {
return mapProviderOptions(settings, "anthropic", ["apiKey", "authToken", "baseURL"])
}
function mapProviderOptions(
settings: Readonly<Record<string, unknown>>,
key: string,
excluded: ReadonlyArray<string>,
) {
const options = Object.fromEntries(Object.entries(settings).filter(([name]) => !excluded.includes(name)))
if (Object.keys(options).length === 0) return {}
return { providerOptions: { [key]: options } }
}
function mapBedrockMantle(input: MapInput, baseSettings: Readonly<Record<string, unknown>>): Mapping | undefined {
const settings = input.settings
const chat = input.modelID === "openai.gpt-oss-safeguard-20b" || input.modelID === "openai.gpt-oss-safeguard-120b"
@@ -276,17 +260,26 @@ function bedrockRegion(settings: Readonly<Record<string, unknown>>) {
}
function mapOpenAIOptions(settings: Readonly<Record<string, unknown>>) {
const options = openAIOptions(settings)
if (Object.keys(options).length === 0) return {}
return { providerOptions: options }
}
function openAIOptions(settings: Readonly<Record<string, unknown>>) {
const options = {
...(typeof settings.instructions === "string" ? { instructions: settings.instructions } : {}),
...(typeof settings.reasoningEffort === "string" ? { reasoningEffort: settings.reasoningEffort } : {}),
...(typeof settings.reasoningSummary === "string" ? { reasoningSummary: settings.reasoningSummary } : {}),
...(Array.isArray(settings.include) ? { include: settings.include } : {}),
...(typeof settings.store === "boolean" ? { store: settings.store } : {}),
...(typeof settings.promptCacheKey === "string" ? { promptCacheKey: settings.promptCacheKey } : {}),
...(typeof settings.textVerbosity === "string" ? { textVerbosity: settings.textVerbosity } : {}),
...(typeof settings.serviceTier === "string" ? { serviceTier: settings.serviceTier } : {}),
...(typeof settings.truncation === "string" ? { truncation: settings.truncation } : {}),
...(isRecord(settings.allowedTools) ? { allowedTools: settings.allowedTools } : {}),
...(typeof settings.maxToolCalls === "number" ? { maxToolCalls: settings.maxToolCalls } : {}),
...(typeof settings.parallelToolCalls === "boolean" ? { parallelToolCalls: settings.parallelToolCalls } : {}),
}
if (Object.keys(options).length === 0) return {}
return { providerOptions: { openai: options } }
return options
}
function mapBaseSettings(settings: Readonly<Record<string, unknown>>) {
@@ -299,7 +292,7 @@ function mapAPIKey(settings: Readonly<Record<string, unknown>>) {
return typeof settings.apiKey === "string" ? { apiKey: settings.apiKey } : {}
}
function mapGoogleOptions(settings: Readonly<Record<string, unknown>>, extra: Readonly<Record<string, unknown>> = {}) {
function mapGoogleOptions(settings: Readonly<Record<string, unknown>>) {
const input = settings.thinkingConfig
const thinkingConfig = {
...(isRecord(input) && typeof input.thinkingBudget === "number" ? { thinkingBudget: input.thinkingBudget } : {}),
@@ -314,10 +307,9 @@ function mapGoogleOptions(settings: Readonly<Record<string, unknown>>, extra: Re
...(Array.isArray(settings.safetySettings) ? { safetySettings: settings.safetySettings } : {}),
...(typeof settings.serviceTier === "string" ? { serviceTier: settings.serviceTier } : {}),
...(Object.keys(thinkingConfig).length > 0 ? { thinkingConfig } : {}),
...extra,
}
if (Object.keys(options).length === 0) return {}
return { providerOptions: { gemini: options } }
return { providerOptions: options }
}
function mapOpenRouter(
@@ -369,7 +361,7 @@ function mapOpenRouterOptions(settings: Readonly<Record<string, unknown>>) {
),
)
if (Object.keys(options).length === 0) return {}
return { providerOptions: { openrouter: options } }
return { providerOptions: options }
}
function isStringRecord(value: unknown): value is Readonly<Record<string, string>> {
@@ -382,5 +374,5 @@ function mapXAIOptions(settings: Readonly<Record<string, unknown>>) {
...(typeof settings.store === "boolean" ? { store: settings.store } : {}),
}
if (Object.keys(options).length === 0) return {}
return { providerOptions: { xai: options } }
return { providerOptions: options }
}
+26 -14
View File
@@ -307,12 +307,6 @@ function modelFromLanguage(info: Info, language: LanguageModelV3) {
const packageName = Provider.packageName(info.package!)
const projected = mapBodyToProviderOptions(info, packageName)
const optionKey = providerOptionKey(packageName, info.providerID)
const providerOptions = (() => {
if (projected.settings === undefined) return
if (packageName === "@ai-sdk/gateway") return gatewayProviderOptions(info.modelID ?? info.id, projected.settings)
if (packageName === "@ai-sdk/azure") return { openai: projected.settings, azure: projected.settings }
return { [optionKey]: projected.settings }
})()
const route: AnyRoute = {
id: `ai-sdk:${packageName}`,
provider: ProviderID.make(info.providerID),
@@ -335,11 +329,11 @@ function modelFromLanguage(info: Info, language: LanguageModelV3) {
headers: info.headers,
},
limits: { context: info.limit.context, input: info.limit.input, output: info.limit.output },
providerOptions,
providerOptions: projected.settings,
},
body: {
schema: Schema.Unknown,
from: (request) => Effect.succeed(callOptions(request)),
from: (request) => Effect.succeed(callOptions(request, packageName, info.modelID ?? info.id, optionKey)),
},
with: () => route,
model: (input) =>
@@ -414,7 +408,12 @@ function mapBodyToProviderOptions(model: Info, packageName: string) {
}
}
function callOptions(request: LLMRequest): LanguageModelV3CallOptions {
function callOptions(
request: LLMRequest,
packageName: string | undefined,
modelID: ID,
optionKey: string,
): LanguageModelV3CallOptions {
return {
prompt: prompt(request),
maxOutputTokens: request.generation?.maxTokens,
@@ -428,7 +427,7 @@ function callOptions(request: LLMRequest): LanguageModelV3CallOptions {
tools: request.tools.map(tool),
toolChoice: toolChoice(request.toolChoice),
headers: request.http?.headers,
providerOptions: providerOptions(request.providerOptions),
providerOptions: requestProviderOptions(request.providerOptions, packageName, modelID, optionKey),
}
}
@@ -526,7 +525,7 @@ function assistantPart(part: ContentPart): AssistantContent {
case "media":
return [{ type: "file", mediaType: part.mediaType, data: part.data, filename: part.filename }]
case "reasoning":
return [{ type: "reasoning", text: part.text, providerOptions: providerOptions(part.providerMetadata) }]
return [{ type: "reasoning", text: part.text, providerOptions: metadataProviderOptions(part.providerMetadata) }]
case "tool-call":
return [
{
@@ -535,7 +534,7 @@ function assistantPart(part: ContentPart): AssistantContent {
toolName: part.name,
input: part.input,
providerExecuted: part.providerExecuted,
providerOptions: providerOptions(part.providerMetadata),
providerOptions: metadataProviderOptions(part.providerMetadata),
},
]
case "tool-result":
@@ -551,7 +550,7 @@ function toolResultPart(part: ContentPart): ToolResultContent[] {
toolCallId: part.id,
toolName: part.name,
output: toolOutput(part.result),
providerOptions: providerOptions(part.providerMetadata),
providerOptions: metadataProviderOptions(part.providerMetadata),
},
]
}
@@ -595,7 +594,20 @@ function toolChoice(input: LLMRequest["toolChoice"]): LanguageModelV3ToolChoice
return { type: input.type }
}
function providerOptions(input: LLMRequest["providerOptions"]): SharedV3ProviderOptions | undefined {
function requestProviderOptions(
input: LLMRequest["providerOptions"],
packageName: string | undefined,
modelID: ID,
optionKey: string,
): SharedV3ProviderOptions | undefined {
if (!input) return undefined
const options = jsonObject(input)
if (packageName === "@ai-sdk/gateway") return gatewayProviderOptions(modelID, options)
if (packageName === "@ai-sdk/azure") return { openai: options, azure: options }
return { [optionKey]: options }
}
function metadataProviderOptions(input: ProviderMetadata | undefined): SharedV3ProviderOptions | undefined {
if (!input) return undefined
return Object.fromEntries(Object.entries(input).map(([key, value]) => [key, jsonObject(value)]))
}
+25 -25
View File
@@ -1,7 +1,7 @@
export * as ModelResolver from "./model-resolver.js"
import { makeLocationNode } from "@opencode-ai/util/effect/app-node"
import { LanguageModel } from "@opencode-ai/ai"
import { LanguageModel, ProviderPackage } from "@opencode-ai/ai"
import { Auth } from "@opencode-ai/ai/route"
import { Context, Effect, Layer, Schema } from "effect"
import { produce } from "immer"
@@ -124,7 +124,7 @@ const resolveCatalogModel = Effect.fn("ModelResolver.resolveCatalogModel")(funct
const resolved = prepareRuntimeModel(model, credential)
const packageName = Provider.packageName(resolved.package)
const configuration = credential?.type === "key" ? credential.configuration : undefined
const configured = { ...resolved.settings, ...credential?.metadata, ...configuration }
const configured = Provider.mergeOverlay(resolved.settings, configuration) ?? {}
const mapping = Provider.isAISDK(resolved.package)
? AISDKNative.map({
packageName,
@@ -140,7 +140,7 @@ const resolveCatalogModel = Effect.fn("ModelResolver.resolveCatalogModel")(funct
const settings = yield* prepareProviderSettings(
resolved,
Provider.mergeOverlay(resolved.settings, {
...nativeCredentialSettings(resolved.package ?? "", credential),
...legacyCredentialSettings(credential),
...credential?.metadata,
...configuration,
}) ?? {},
@@ -157,16 +157,18 @@ const resolveCatalogModel = Effect.fn("ModelResolver.resolveCatalogModel")(funct
const module = yield* (dependencies?.loadPackage ?? Provider.loadPackage)(specifier).pipe(
Effect.mapError(() => unsupported(resolved)),
)
const settings = {
...(credential ? withoutNativeAuthSettings(mapped) : mapped),
...nativeCredentialSettings(specifier, credential),
headers: Provider.mergeHeaders(mapping?.headers, resolved.headers),
body: Provider.mergeOverlay(mapping?.body, resolved.body),
limits: { context: resolved.limit.context, input: resolved.limit.input, output: resolved.limit.output },
}
return yield* Effect.try({
try: () => {
const runtime = module.model(resolved.modelID ?? resolved.id, settings)
const runtime = module.model({
id: resolved.modelID ?? resolved.id,
settings: mapped,
credential: providerCredential(credential),
defaults: {
headers: Provider.mergeHeaders(mapping?.headers, resolved.headers),
body: Provider.mergeOverlay(mapping?.body, resolved.body),
limits: { context: resolved.limit.context, input: resolved.limit.input, output: resolved.limit.output },
},
})
return LanguageModel.update(runtime, {
provider: resolved.providerID,
compatibility: resolved.compatibility
@@ -225,25 +227,23 @@ function unresolvedProviderVariables(model: Info, baseURL: string) {
})
}
const nativeCredentialSettings = (specifier: string, credential: Credential.Value | undefined) => {
const legacyCredentialSettings = (credential: Credential.Value | undefined) => {
if (!credential) return {}
if (credential.type === "key") return { apiKey: credential.key }
if (
specifier === "@opencode-ai/ai/providers/anthropic" ||
specifier === "@opencode-ai/ai/providers/anthropic-compatible"
)
return { authToken: credential.access }
if (
specifier === "@opencode-ai/ai/providers/google-vertex" ||
specifier.startsWith("@opencode-ai/ai/providers/google-vertex/")
)
return { accessToken: credential.access }
return { apiKey: credential.access }
}
const withoutNativeAuthSettings = (settings: Record<string, unknown>) => {
const { accessToken: _accessToken, apiKey: _apiKey, authToken: _authToken, ...rest } = settings
return rest
const providerCredential = (credential: Credential.Value | undefined): ProviderPackage.Credential | undefined => {
if (!credential) return undefined
if (credential.type === "key" && credential.key.length === 0) return undefined
if (credential.type === "oauth" && credential.access.length === 0) return undefined
if (credential.type === "key")
return {
type: "key",
value: credential.key,
configuration: credential.configuration,
}
return { type: "oauth", accessToken: credential.access }
}
const unsupported = (model: Info) =>
+1 -1
View File
@@ -171,7 +171,7 @@ const importPackage = Effect.fn("Provider.importPackage")(function* (
if (typeof module !== "object" || module === null || typeof (module as { model?: unknown }).model !== "function") {
return yield* new LoadError({
package: specifier,
cause: new Error(`Provider package ${specifier} does not export model(modelID, settings)`),
cause: new Error(`Provider package ${specifier} does not export model(input)`),
})
}
return module as ProviderPackageDefinition
+36 -55
View File
@@ -23,15 +23,11 @@ describe("AISDKNative", () => {
apiKey: "secret",
baseURL: "https://api.meta.ai/v1",
organization: "org",
providerOptions: {
openai: {
reasoningEffort: "xhigh",
reasoningSummary: "auto",
include: ["reasoning.encrypted_content"],
instructions: "Follow the repository instructions.",
truncation: "auto",
},
},
reasoningEffort: "xhigh",
reasoningSummary: "auto",
include: ["reasoning.encrypted_content"],
instructions: "Follow the repository instructions.",
truncation: "auto",
},
})
expect(map("@ai-sdk/openai-compatible", { baseURL: "https://example.com/v1", reasoningEffort: "high" })).toEqual({
@@ -39,7 +35,7 @@ describe("AISDKNative", () => {
settings: {
baseURL: "https://example.com/v1",
provider: "test-provider",
providerOptions: { openai: { reasoningEffort: "high" } },
providerOptions: { reasoningEffort: "high" },
},
})
})
@@ -58,10 +54,8 @@ describe("AISDKNative", () => {
authToken: "token",
baseURL: "https://anthropic.example/v1",
providerOptions: {
anthropic: {
thinking: { type: "adaptive", display: "summarized" },
effort: "high",
},
thinking: { type: "adaptive", display: "summarized" },
effort: "high",
},
},
})
@@ -81,7 +75,8 @@ describe("AISDKNative", () => {
project: "project",
location: "us-central1",
providerOptions: {
gemini: { labels: { environment: "test" }, thinkingConfig: { thinkingLevel: "high" } },
labels: { environment: "test" },
thinkingConfig: { thinkingLevel: "high" },
},
},
})
@@ -115,7 +110,7 @@ describe("AISDKNative", () => {
apiVersion: "2025-01-01-preview",
queryParams: { feature: "enabled" },
useDeploymentBasedUrls: true,
providerOptions: { openai: { reasoningEffort: "high" } },
providerOptions: { reasoningEffort: "high" },
},
})
expect(map("@ai-sdk/azure", { ...settings, useCompletionUrls: true }, "custom-deployment")?.package).toBe(
@@ -193,11 +188,9 @@ describe("AISDKNative", () => {
baseURL: "https://mantle.test/v1",
region: "us-west-2",
providerOptions: {
openai: {
reasoningEffort: "high",
reasoningSummary: "auto",
include: ["reasoning.encrypted_content"],
},
reasoningEffort: "high",
reasoningSummary: "auto",
include: ["reasoning.encrypted_content"],
},
},
headers: { "x-test": "value" },
@@ -246,7 +239,7 @@ describe("AISDKNative", () => {
region: "eu-west-1",
},
baseURL: "https://bedrock-mantle.eu-west-1.api.aws/v1",
providerOptions: { openai: { store: false } },
providerOptions: { store: false },
},
})
})
@@ -278,12 +271,10 @@ describe("AISDKNative", () => {
package: "@opencode-ai/ai/providers/openrouter",
settings: {
providerOptions: {
openrouter: {
models: ["anthropic/claude-sonnet-4.6"],
provider: { only: ["anthropic"], require_parameters: true },
reasoning: { effort: "high" },
future_option: { enabled: true },
},
models: ["anthropic/claude-sonnet-4.6"],
provider: { only: ["anthropic"], require_parameters: true },
reasoning: { effort: "high" },
future_option: { enabled: true },
},
},
headers: {
@@ -312,15 +303,13 @@ describe("AISDKNative", () => {
package: "@opencode-ai/ai/providers/google",
settings: {
providerOptions: {
gemini: {
cachedContent: "cachedContents/example",
safetySettings: [{ category: "HARM_CATEGORY_HARASSMENT", threshold: "BLOCK_NONE" }],
serviceTier: "flex",
thinkingConfig: {
thinkingBudget: 0,
includeThoughts: false,
thinkingLevel: "high",
},
cachedContent: "cachedContents/example",
safetySettings: [{ category: "HARM_CATEGORY_HARASSMENT", threshold: "BLOCK_NONE" }],
serviceTier: "flex",
thinkingConfig: {
thinkingBudget: 0,
includeThoughts: false,
thinkingLevel: "high",
},
},
},
@@ -330,7 +319,7 @@ describe("AISDKNative", () => {
test("maps Google thinking settings independently", () => {
for (const thinkingConfig of [{ thinkingBudget: -1 }, { includeThoughts: true }, { thinkingLevel: "medium" }]) {
expect(map("@ai-sdk/google", { thinkingConfig })).toMatchObject({
settings: { providerOptions: { gemini: { thinkingConfig } } },
settings: { providerOptions: { thinkingConfig } },
})
}
})
@@ -345,11 +334,9 @@ describe("AISDKNative", () => {
).toMatchObject({
settings: {
providerOptions: {
gemini: {
cachedContent: "cachedContents/example",
safetySettings: [{ category: "HARM_CATEGORY_HARASSMENT", threshold: "BLOCK_NONE" }],
serviceTier: "future-tier",
},
cachedContent: "cachedContents/example",
safetySettings: [{ category: "HARM_CATEGORY_HARASSMENT", threshold: "BLOCK_NONE" }],
serviceTier: "future-tier",
},
},
})
@@ -374,10 +361,8 @@ describe("AISDKNative", () => {
location: "eu",
project: "vertex-project",
providerOptions: {
gemini: {
labels: { component: "opencode", environment: "test" },
thinkingConfig: { thinkingLevel: "high" },
},
labels: { component: "opencode", environment: "test" },
thinkingConfig: { thinkingLevel: "high" },
},
},
headers: { "x-test": "value" },
@@ -403,10 +388,8 @@ describe("AISDKNative", () => {
location: "eu",
project: "vertex-project",
providerOptions: {
anthropic: {
thinking: { type: "adaptive", display: "summarized" },
effort: "high",
},
thinking: { type: "adaptive", display: "summarized" },
effort: "high",
},
},
headers: { "x-test": "value" },
@@ -427,10 +410,8 @@ describe("AISDKNative", () => {
apiKey: "secret",
baseURL: "https://xai.example/v1",
providerOptions: {
xai: {
reasoningEffort: "custom",
store: true,
},
reasoningEffort: "custom",
store: true,
},
},
})
+178 -88
View File
@@ -2,7 +2,7 @@ import { describe, expect } from "bun:test"
import { LLM, LanguageModel } from "@opencode-ai/ai"
import { OpenAIChat } from "@opencode-ai/ai/protocols"
import { compileRequest } from "@opencode-ai/ai/route/client"
import { Effect, Layer } from "effect"
import { ConfigProvider, Effect, Layer } from "effect"
import { Headers } from "effect/unstable/http"
import { Credential } from "@opencode-ai/core/credential"
import { Integration } from "@opencode-ai/core/integration"
@@ -77,6 +77,22 @@ describe("ModelResolver", () => {
}),
Credential.Key.make({ type: "key", key: "secret" }),
)
const configuredCredential = yield* ModelResolver.fromCatalogModel(
model(Provider.aisdk("@ai-sdk/azure"), {
providerID: Provider.ID.azure,
modelID: "configured-deployment",
settings: { resourceName: "catalog-resource", apiVersion: "catalog-version" },
}),
Credential.Key.make({
type: "key",
key: "secret",
configuration: {
resourceName: "configured-resource",
apiVersion: "configured-version",
useDeploymentBasedUrls: true,
},
}),
)
const chat = yield* ModelResolver.fromCatalogModel(
model(Provider.aisdk("@ai-sdk/azure"), {
providerID: Provider.ID.azure,
@@ -114,6 +130,10 @@ describe("ModelResolver", () => {
query: { "api-version": "2025-01-01-preview" },
},
})
expect(configuredCredential.route.endpoint).toMatchObject({
baseURL: "https://configured-resource.openai.azure.com/openai/deployments/configured-deployment",
query: { "api-version": "configured-version" },
})
expect(chat).toMatchObject({ id: "chat-deployment", provider: "azure" })
expect(chat.route.id).toBe("azure-openai-chat")
expect(deployment).toMatchObject({ id: "legacy-url-deployment", provider: "azure" })
@@ -135,11 +155,22 @@ describe("ModelResolver", () => {
settings: { baseURL: "https://${AZURE_HOST}/openai" },
}),
)
const configured = yield* ModelResolver.fromCatalogModel(
model(Provider.aisdk("@ai-sdk/azure"), {
providerID: Provider.ID.azure,
}),
Credential.Key.make({
type: "key",
key: "secret",
configuration: { baseURL: "https://${AZURE_HOST}/openai" },
}),
)
expect(resolved.route.endpoint).toMatchObject({
baseURL: "https://resource.openai.azure.com/openai/v1",
query: { "api-version": "v1" },
})
expect(configured.route.endpoint.baseURL).toBe("https://resource.openai.azure.com/openai/v1")
}),
),
)
@@ -256,25 +287,33 @@ describe("ModelResolver", () => {
}),
)
it.effect("treats an empty configured API key as omitted", () =>
withEnv({ OPENAI_API_KEY: "environment-key" }, () =>
Effect.gen(function* () {
const resolved = yield* ModelResolver.fromCatalogModel(
model(Provider.aisdk("@ai-sdk/openai"), {
settings: { apiKey: "", baseURL: "https://openai.example/v1" },
}),
)
const headers = yield* resolved.route.auth.apply({
request: LLM.request({ model: resolved, prompt: "Hello" }),
it.effect("treats empty configured and selected API keys as omitted", () =>
Effect.gen(function* () {
const resolved = yield* ModelResolver.fromCatalogModel(
model(Provider.aisdk("@ai-sdk/openai"), {
settings: { apiKey: "", baseURL: "https://openai.example/v1" },
}),
)
const selected = yield* ModelResolver.fromCatalogModel(
model(Provider.aisdk("@ai-sdk/openai"), {
settings: { baseURL: "https://openai.example/v1" },
}),
Credential.Key.make({ type: "key", key: "" }),
)
const headers = yield* Effect.forEach([resolved, selected], (model) =>
model.route.auth.apply({
request: LLM.request({ model, prompt: "Hello" }),
method: "POST",
url: "https://openai.example/v1/responses",
body: "{}",
headers: Headers.empty,
})
}),
).pipe(
Effect.provide(ConfigProvider.layer(ConfigProvider.fromEnv({ env: { OPENAI_API_KEY: "environment-key" } }))),
)
expect(headers.authorization).toBe("Bearer environment-key")
}),
),
expect(headers.map((item) => item.authorization)).toEqual(["Bearer environment-key", "Bearer environment-key"])
}),
)
it.effect("uses no native API-key auth for an explicitly enabled provider without credentials", () => {
@@ -477,7 +516,11 @@ describe("ModelResolver", () => {
},
],
})
const resolved = yield* ModelResolver.resolveModel(catalog, VariantID.make("xhigh"))
const resolved = yield* ModelResolver.resolveModel(
catalog,
VariantID.make("xhigh"),
Credential.Key.make({ type: "key", key: "secret" }),
)
expect(resolved.route.defaults.headers).toMatchObject({ "x-test": "header", "x-variant": "high" })
expect(resolved.route.defaults.http?.body).toEqual({
@@ -487,12 +530,10 @@ describe("ModelResolver", () => {
temperature: 0.2,
})
expect(resolved.route.defaults.providerOptions).toEqual({
openai: {
store: false,
reasoningEffort: "xhigh",
reasoningSummary: "auto",
include: ["reasoning.encrypted_content"],
},
store: false,
reasoningEffort: "xhigh",
reasoningSummary: "auto",
include: ["reasoning.encrypted_content"],
})
const prepared = yield* compileRequest(LLM.request({ model: resolved, prompt: "Hello" }))
expect(prepared.body).toMatchObject({
@@ -561,7 +602,7 @@ describe("ModelResolver", () => {
custom_extension: { enabled: true },
})
expect(resolved.route.defaults.providerOptions).toEqual({
anthropic: { thinking: { type: "enabled", budgetTokens: 12000 } },
thinking: { type: "enabled", budgetTokens: 12000 },
})
}),
)
@@ -582,6 +623,43 @@ describe("ModelResolver", () => {
}),
)
it.effect("lets the native Anthropic package distinguish key and OAuth credentials", () =>
Effect.gen(function* () {
const catalog = model(Provider.aisdk("@ai-sdk/anthropic"), {
settings: { baseURL: "https://anthropic.example/v1" },
})
const key = yield* ModelResolver.fromCatalogModel(
catalog,
Credential.Key.make({ type: "key", key: "anthropic-key" }),
)
const oauth = yield* ModelResolver.fromCatalogModel(
catalog,
Credential.OAuth.make({
type: "oauth",
methodID: Integration.MethodID.make("device"),
access: "anthropic-token",
refresh: "refresh",
expires: Date.now() + 60_000,
}),
)
const input = (resolved: LanguageModel) => ({
request: LLM.request({ model: resolved, prompt: "Hello" }),
method: "POST" as const,
url: "https://anthropic.example/v1/messages",
body: "{}",
headers: Headers.empty,
})
const keyHeaders = yield* key.route.auth.apply(input(key))
const oauthHeaders = yield* oauth.route.auth.apply(input(oauth))
expect(keyHeaders["x-api-key"]).toBe("anthropic-key")
expect(keyHeaders.authorization).toBeUndefined()
expect(oauthHeaders.authorization).toBe("Bearer anthropic-token")
expect(oauthHeaders["x-api-key"]).toBeUndefined()
}),
)
it.effect("uses resolved credentials for bearer auth", () =>
Effect.gen(function* () {
const resolved = yield* ModelResolver.fromCatalogModel(
@@ -686,6 +764,24 @@ describe("ModelResolver", () => {
}),
)
it.effect("maps flat native OpenAI settings into provider options", () =>
Effect.gen(function* () {
const resolved = yield* ModelResolver.fromCatalogModel(
model("@opencode-ai/ai/providers/openai", {
modelID: "gpt-5",
settings: { reasoningEffort: "high", store: true },
}),
)
expect(resolved.route.defaults.providerOptions).toEqual({
store: true,
reasoningEffort: "high",
reasoningSummary: "auto",
include: ["reasoning.encrypted_content"],
})
}),
)
it.effect("does not route native OpenAI-compatible packages to the codex backend", () =>
Effect.gen(function* () {
const resolved = yield* ModelResolver.fromCatalogModel(
@@ -772,15 +868,18 @@ describe("ModelResolver", () => {
loadPackage: (specifier) => {
expect(specifier).toBe("@opencode-ai/ai/providers/custom")
return Effect.succeed({
model: (modelID, settings) => {
expect(modelID).toBe("api-test-model")
expect(settings).toEqual({
region: "test",
headers: { "x-package": "header" },
body: { custom: true },
limits: { context: 100, output: 20 },
model: (input) => {
expect(input).toEqual({
id: "api-test-model",
settings: { region: "test" },
credential: undefined,
defaults: {
headers: { "x-package": "header" },
body: { custom: true },
limits: { context: 100, output: 20 },
},
})
return LanguageModel.make({ id: modelID, provider: "package-provider", route: native.route })
return LanguageModel.make({ id: input.id, provider: "package-provider", route: native.route })
},
})
},
@@ -791,7 +890,7 @@ describe("ModelResolver", () => {
}),
)
it.effect("maps OAuth credentials to native provider auth settings", () =>
it.effect("passes OAuth credentials to native provider packages without interpreting them", () =>
Effect.gen(function* () {
const native = yield* ModelResolver.fromCatalogModel(
model(Provider.aisdk("@ai-sdk/openai"), {
@@ -806,23 +905,23 @@ describe("ModelResolver", () => {
expires: Date.now() + 60_000,
})
const packages = [
["@opencode-ai/ai/providers/google-vertex", "accessToken"],
["@opencode-ai/ai/providers/google-vertex/gemini", "accessToken"],
["@opencode-ai/ai/providers/google-vertex/chat", "accessToken"],
["@opencode-ai/ai/providers/google-vertex/responses", "accessToken"],
["@opencode-ai/ai/providers/google-vertex/messages", "accessToken"],
["@opencode-ai/ai/providers/anthropic", "authToken"],
["@opencode-ai/ai/providers/anthropic-compatible", "authToken"],
"@opencode-ai/ai/providers/google-vertex",
"@opencode-ai/ai/providers/google-vertex/gemini",
"@opencode-ai/ai/providers/google-vertex/chat",
"@opencode-ai/ai/providers/google-vertex/responses",
"@opencode-ai/ai/providers/google-vertex/messages",
"@opencode-ai/ai/providers/anthropic",
"@opencode-ai/ai/providers/anthropic-compatible",
] as const
yield* Effect.forEach(packages, ([specifier, key]) =>
yield* Effect.forEach(packages, (specifier) =>
ModelResolver.fromCatalogModel(model(specifier, { settings: { apiKey: "configured-key" } }), credential, {
loadPackage: () =>
Effect.succeed({
model: (modelID, settings) => {
expect(settings).toMatchObject({ [key]: "oauth-token" })
expect(settings).not.toHaveProperty("apiKey")
return LanguageModel.make({ id: modelID, provider: "package-provider", route: native.route })
model: (input) => {
expect(input.settings).toEqual({ apiKey: "configured-key" })
expect(input.credential).toEqual({ type: "oauth", accessToken: "oauth-token" })
return LanguageModel.make({ id: input.id, provider: "package-provider", route: native.route })
},
}),
}),
@@ -843,52 +942,50 @@ describe("ModelResolver", () => {
include: ["reasoning.encrypted_content"],
},
{
openai: {
reasoningEffort: "xhigh",
reasoningSummary: "auto",
include: ["reasoning.encrypted_content"],
},
reasoningEffort: "xhigh",
reasoningSummary: "auto",
include: ["reasoning.encrypted_content"],
},
],
[
"@ai-sdk/anthropic",
"@opencode-ai/ai/providers/anthropic",
{ thinking: { type: "adaptive", display: "summarized" }, effort: "high" },
{ anthropic: { thinking: { type: "adaptive", display: "summarized" }, effort: "high" } },
{ providerOptions: { thinking: { type: "adaptive", display: "summarized" }, effort: "high" } },
],
[
"@ai-sdk/openai-compatible",
"@opencode-ai/ai/providers/openai-compatible",
{ reasoningEffort: "high" },
{ openai: { reasoningEffort: "high" } },
{ provider: "test-provider", providerOptions: { reasoningEffort: "high" } },
],
[
"@ai-sdk/google",
"@opencode-ai/ai/providers/google",
{ thinkingConfig: { thinkingLevel: "high" } },
{ gemini: { thinkingConfig: { thinkingLevel: "high" } } },
{ providerOptions: { thinkingConfig: { thinkingLevel: "high" } } },
],
[
"@ai-sdk/google-vertex",
"@opencode-ai/ai/providers/google-vertex",
{ thinkingConfig: { thinkingLevel: "high" } },
{ gemini: { thinkingConfig: { thinkingLevel: "high" } } },
{ providerOptions: { thinkingConfig: { thinkingLevel: "high" } } },
],
[
"@openrouter/ai-sdk-provider",
"@opencode-ai/ai/providers/openrouter",
{ reasoning: { effort: "high" } },
{ openrouter: { reasoning: { effort: "high" } } },
{ providerOptions: { reasoning: { effort: "high" } } },
],
[
"@ai-sdk/xai",
"@opencode-ai/ai/providers/xai",
{ reasoningEffort: "high" },
{ xai: { reasoningEffort: "high" } },
{ providerOptions: { reasoningEffort: "high" } },
],
] as const
yield* Effect.forEach(packages, ([catalogPackage, nativePackage, sourceOptions, providerOptions]) =>
yield* Effect.forEach(packages, ([catalogPackage, nativePackage, sourceOptions, mappedSettings]) =>
ModelResolver.fromCatalogModel(
model(Provider.aisdk(catalogPackage), {
modelID: "api-model",
@@ -901,17 +998,19 @@ describe("ModelResolver", () => {
loadPackage: (specifier) => {
expect(specifier).toBe(nativePackage)
return Effect.succeed({
model: (modelID, settings) => {
expect(modelID).toBe("api-model")
expect(settings).toMatchObject({
apiKey: "secret",
model: (input) => {
expect(input.id).toBe("api-model")
expect(input.settings).toMatchObject({
baseURL: "https://provider.example/v1",
...mappedSettings,
})
expect(input.credential).toEqual({ type: "key", value: "secret" })
expect(input.defaults).toEqual({
headers: { "x-provider": "header" },
body: { custom: true },
limits: { context: 100, output: 20 },
providerOptions,
})
return LanguageModel.make({ id: modelID, provider: "native-provider", route: native.route })
return LanguageModel.make({ id: input.id, provider: "native-provider", route: native.route })
},
})
},
@@ -935,11 +1034,7 @@ describe("ModelResolver", () => {
["@ai-sdk/azure", "@opencode-ai/ai/providers/azure/responses", "api-model"],
["@ai-sdk/google", "@opencode-ai/ai/providers/google", "api-model"],
["@ai-sdk/google-vertex", "@opencode-ai/ai/providers/google-vertex", "api-model"],
[
"@ai-sdk/google-vertex/anthropic",
"@opencode-ai/ai/providers/google-vertex/messages",
"claude-sonnet-4-6",
],
["@ai-sdk/google-vertex/anthropic", "@opencode-ai/ai/providers/google-vertex/messages", "claude-sonnet-4-6"],
["@ai-sdk/openai", "@opencode-ai/ai/providers/openai", "api-model"],
["@ai-sdk/openai-compatible", "@opencode-ai/ai/providers/openai-compatible", "api-model"],
["@openrouter/ai-sdk-provider", "@opencode-ai/ai/providers/openrouter", "api-model"],
@@ -957,7 +1052,8 @@ describe("ModelResolver", () => {
loadPackage: (specifier) => {
expect(specifier).toBe(nativePackage)
return Effect.succeed({
model: (id) => LanguageModel.make({ id, provider: "native-provider", route: OpenAIChat.route }),
model: (input) =>
LanguageModel.make({ id: input.id, provider: "native-provider", route: OpenAIChat.route }),
})
},
loadAISDK: () => Effect.die(`AI SDK loader called for ${catalogPackage}`),
@@ -993,20 +1089,18 @@ describe("ModelResolver", () => {
loadPackage: (specifier) => {
expect(specifier).toBe("@opencode-ai/ai/providers/google-vertex/messages")
return Effect.succeed({
model: (modelID, settings) => {
expect(modelID).toBe("claude-sonnet-4-6")
expect(settings).toMatchObject({
accessToken: "vertex-token",
model: (input) => {
expect(input.id).toBe("claude-sonnet-4-6")
expect(input.settings).toMatchObject({
location: "eu",
project: "vertex-project",
providerOptions: {
anthropic: {
thinking: { type: "adaptive", display: "summarized" },
effort: "high",
},
thinking: { type: "adaptive", display: "summarized" },
effort: "high",
},
})
return LanguageModel.make({ id: modelID, provider: "native-provider", route: native.route })
expect(input.credential).toEqual({ type: "oauth", accessToken: "vertex-token" })
return LanguageModel.make({ id: input.id, provider: "native-provider", route: native.route })
},
})
},
@@ -1033,16 +1127,16 @@ describe("ModelResolver", () => {
{
loadPackage: () =>
Effect.succeed({
model: (modelID, settings) => {
expect(settings.headers).toEqual({
model: (input) => {
expect(input.defaults.headers).toEqual({
"HTTP-Referer": "https://opencode.ai",
"X-OpenRouter-Title": "Custom",
})
expect(settings.body).toEqual({
expect(input.defaults.body).toEqual({
transforms: ["middle-out"],
provider: { sort: "price", only: ["anthropic"] },
})
return LanguageModel.make({ id: modelID, provider: "openrouter", route: OpenAIChat.route })
return LanguageModel.make({ id: input.id, provider: "openrouter", route: OpenAIChat.route })
},
}),
},
@@ -1076,15 +1170,11 @@ describe("ModelResolver", () => {
)
expect(google.route.id).toBe("gemini")
expect(google.route.defaults.providerOptions).toEqual({
gemini: { thinkingConfig: { thinkingBudget: 1_024 } },
})
expect(google.route.defaults.providerOptions).toEqual({ thinkingConfig: { thinkingBudget: 1_024 } })
expect(openrouter.route.id).toBe("openrouter")
expect(openrouter.route.defaults.providerOptions).toEqual({ openrouter: { reasoning: { effort: "high" } } })
expect(openrouter.route.defaults.providerOptions).toEqual({ reasoning: { effort: "high" } })
expect(xai.route.id).toBe("openai-responses")
expect(xai.route.defaults.providerOptions).toEqual({
xai: { reasoningEffort: "high", store: false },
})
expect(xai.route.defaults.providerOptions).toEqual({ reasoningEffort: "high", store: false })
expect(bedrock.route.id).toBe("bedrock-converse")
expect(bedrock.route.defaults.generation).toEqual({ topP: 0.8 })
expect(bedrock.route.defaults.http?.body).toEqual({ serviceTier: { type: "priority" } })