diff --git a/packages/ai/AGENTS.md b/packages/ai/AGENTS.md index 3fc328099ec..fc807ddb9b7 100644 --- a/packages/ai/AGENTS.md +++ b/packages/ai/AGENTS.md @@ -80,7 +80,7 @@ Route defaults are request-shaping defaults such as `headers`, `limits`, `genera The four-axis decomposition is the reason DeepSeek, TogetherAI, Cerebras, Baseten, Fireworks, and DeepInfra all reuse `OpenAIChat.protocol` verbatim — each provider deployment is a 5-15 line `Route.make(...)` call instead of a 300-400 line route clone. Bug fixes in one protocol propagate to every consumer of that protocol in a single commit. -When a provider ships a non-HTTP transport (OpenAI's WebSocket Responses backend, hypothetical bidirectional streaming APIs), the seam is `Transport` — `WebSocketTransport.jsonTransport.with(...)` constructs an IO template whose `prepare` receives the route endpoint/auth at compile time, builds a WebSocket URL and message, and whose `frames` yields decoded text from the socket. Same protocol and endpoint source, different transport. +When a provider supports multiple physical transports, selection remains execution policy below its semantic route. OpenAI Responses uses a purpose-built hybrid transport that prepares one final request, executes HTTP by default, and passes a generic channel exchange to a per-call `WebSocketChannelExecutor` when supplied. `Route.streamPrepared` owns decoding and acknowledges channel completion only after successful full consumption. ### URL Construction @@ -106,7 +106,7 @@ const proxied = gateway.model("openai/gpt-4o-mini") Keep provider facades small and explicit: - Use branded `ProviderID.make(...)` and `ModelID.make(...)` where ids are constructed directly. -- Use `model` for the default API path and named methods for provider-native alternatives such as OpenAI `responses`, `responsesWebSocket`, and `chat`. +- Use `model` for the default API path and named methods for provider-native alternatives such as OpenAI `responses` and `chat`. - Put provider-specific setup on `.configure(...)`; do not add `model(id, overrides)` as a duplicate construction path. - Export lower-level `routes` arrays separately only when advanced internal wiring needs them. - Prefer `apiKey` as provider-specific sugar and `auth` as the explicit override; keep them mutually exclusive in provider option types with `ProviderAuthOption`. @@ -124,11 +124,10 @@ import { model } from "@opencode-ai/ai/providers/openai/responses" const selected = model("gpt-5", { apiKey, - transport: "websocket", }) ``` -Keep semantic APIs as separate entrypoints, such as OpenAI `chat` and `responses`. Keep transport choices inside the semantic entrypoint settings, so OpenAI Responses HTTP and WebSocket share one entrypoint. Provider facades may still expose named selectors such as `responsesWebSocket` for direct typed call sites; the package-like contract maps its settings to those selectors before returning an executable `LanguageModel`. +Keep semantic APIs as separate entrypoints, such as OpenAI `chat` and `responses`. Transport is execution policy: OpenAI Responses uses HTTP by default and may receive a per-call WebSocket channel executor through `StreamOptions` without changing model or route identity. Do not expose `Route` in provider package settings. Route composition stays an implementation detail behind `model(...)`. @@ -154,9 +153,10 @@ packages/ai/src/ auth-options.ts ProviderAuthOption shape, AuthOptions.bearer, AtLeastOne helper framing.ts Framing type + Framing.sse transport/ transport implementations - index.ts Transport type + HttpTransport / WebSocketTransport namespaces + index.ts Transport execution types + HttpTransport / WebSocketTransport namespaces + websocket-channel.ts generic sequential channel executor/driver contract http.ts HttpTransport.httpJson — POST + framing - websocket.ts WebSocketTransport.json + WebSocketExecutor service + websocket.ts direct one-request channel executor + raw socket adapter protocols/ shared.ts ProviderShared toolkit used inside protocol impls openai-chat.ts protocol + route (compose OpenAIChat.protocol) diff --git a/packages/ai/README.md b/packages/ai/README.md index 6f088d2b2a4..d2b0b970330 100644 --- a/packages/ai/README.md +++ b/packages/ai/README.md @@ -315,7 +315,6 @@ import { model } from "@opencode-ai/ai/providers/openai/responses" const selected = model("gpt-5", { apiKey: process.env.OPENAI_API_KEY, - transport: "websocket", headers: { "x-application": "opencode" }, limits: { context: 200_000, output: 64_000 }, }) @@ -332,7 +331,7 @@ OpenAI Chat and OpenAI Responses are separate semantic entrypoints: - `@opencode-ai/ai/providers/google-vertex/responses` - `@opencode-ai/ai/providers/google-vertex/messages` -Responses HTTP versus WebSocket is a scoped `transport` setting on the OpenAI Responses entrypoint, not another entrypoint. Azure follows the same Chat/Responses split at `providers/azure/chat` and `providers/azure/responses`. Generic OpenAI-compatible Chat remains at `providers/openai-compatible`; the Responses adapter at `providers/openai-compatible/responses` uses the provider-neutral Open Responses protocol. OpenAI Responses extends that baseline with OpenAI tools, event variants, metadata, defaults, and transports. Generic Anthropic Messages-compatible providers use `providers/anthropic-compatible`, which the named Anthropic provider composes. Google Gemini and Amazon Bedrock expose their single native API through their existing provider paths. +OpenAI Responses has one semantic route and uses HTTP by default. Advanced callers may supply a per-call WebSocket channel executor through `StreamOptions`; transport policy does not change provider settings, model identity, or route identity. Azure follows the same Chat/Responses split at `providers/azure/chat` and `providers/azure/responses`. Generic OpenAI-compatible Chat remains at `providers/openai-compatible`; the Responses adapter at `providers/openai-compatible/responses` uses the provider-neutral Open Responses protocol. OpenAI Responses extends that baseline with OpenAI tools, event variants, metadata, and defaults. Generic Anthropic Messages-compatible providers use `providers/anthropic-compatible`, which the named Anthropic provider composes. Google Gemini and Amazon Bedrock expose their single native API through their existing provider paths. Vertex Gemini, Vertex Chat, Vertex Responses, and Vertex Messages are separate API entrypoints. All accept `project`, `location`, and an optional `accessToken`; when no explicit token or auth override is supplied they lazily use Google Application Default Credentials. Vertex Gemini instead selects express mode when `apiKey` or `GOOGLE_VERTEX_API_KEY` is present. Vertex Chat targets MaaS models through the OpenAI-compatible Chat Completions endpoint, while Vertex Responses targets Grok models and defaults `store` to `false` as required by Vertex. `providers/google-vertex` remains the default alias for `providers/google-vertex/gemini`. diff --git a/packages/ai/STATUS.md b/packages/ai/STATUS.md index bb94db657bf..564804ebe24 100644 --- a/packages/ai/STATUS.md +++ b/packages/ai/STATUS.md @@ -1,6 +1,6 @@ # LLM Provider Parity Status -Last reviewed: 2026-07-24 +Last reviewed: 2026-08-07 This file tracks the gap between the native `@opencode-ai/ai` package and the AI SDK provider packages that opencode still depends on for many catalog/runtime paths. @@ -16,8 +16,7 @@ This file tracks the gap between the native `@opencode-ai/ai` package and the AI | Native slice | Source | Current state | Main gaps | | ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | OpenAI Chat | `src/protocols/openai-chat.ts`, `src/providers/openai.ts` | Usable. Streams text, reasoning deltas, tool calls, usage, images, and common generation controls. | No typed structured-output / `response_format` path. Limited typed OpenAI option surface compared with SDK escape hatches. | -| OpenAI Responses HTTP | `src/protocols/open-responses.ts`, `src/protocols/openai-responses.ts`, `src/providers/openai.ts` | Usable. Extends the Open Responses baseline with hosted-tool event surfacing, reasoning replay metadata, GPT-5 defaults, and cache usage. | No explicit `previous_response_id` path. Typed options cover only a subset of Responses fields. Structured output is still mostly synthetic-tool based. | -| OpenAI Responses WebSocket | `src/protocols/openai-responses.ts`, `src/route/transport/websocket.ts` | Present as `OpenAI.responsesWebSocket(...)`. | Runner/catalog support explicitly must not downgrade WebSocket routes; broader runtime selection is not complete. | +| OpenAI Responses | `src/protocols/open-responses.ts`, `src/protocols/openai-responses.ts`, `src/providers/openai.ts` | Usable over HTTP by default, with optional per-call WebSocket channel execution on the same model and route identity. | No incremental `previous_response_id` path or persistent Session channel manager yet. Typed options cover only a subset of Responses fields. Structured output is still mostly synthetic-tool based. | | OpenAI-compatible Chat | `src/protocols/openai-compatible-chat.ts`, `src/providers/openai-compatible.ts` | Usable for generic Chat and several profiles: Baseten, Cerebras, DeepInfra, DeepSeek, Fireworks, Groq, TogetherAI. | Family quirks are mostly endpoint defaults, not full typed behavior. | | Open Responses-compatible | `src/protocols/open-responses.ts`, `src/protocols/openai-compatible-responses.ts`, `src/providers/openai-compatible-responses.ts` | Usable for deployments that implement the provider-neutral Open Responses protocol. The deployment adapter does not inherit OpenAI tools, events, metadata, or defaults. | No named family profiles or recorded deployment coverage yet. | | Anthropic-compatible Messages | `src/protocols/anthropic-messages.ts`, `src/providers/anthropic-compatible.ts` | Usable for deployments that implement the Anthropic Messages wire protocol. Named Anthropic composes this base; MiniMax M3 has recorded text and tool-loop coverage. | No named compatible family profiles yet. | @@ -48,19 +47,19 @@ Other `aisdk:` packages, including Google Vertex, Azure, and Bedrock, currently ## AI SDK Package Parity Matrix -| AI SDK package | Intended native target | Status | Biggest gaps | -| --------------------------------- | -------------------------------------------------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `@ai-sdk/openai` | `OpenAI.chat`, `OpenAI.responses`, `OpenAI.responsesWebSocket` | Partial / usable | Add complete typed option coverage, structured output strategy, explicit Responses continuation support, and runner route selection between Chat/Responses/WebSocket. | -| `@ai-sdk/openai-compatible` | Generic OpenAI-compatible Chat and Responses | Partial / usable | Decide per-family namespace/profile behavior and runner API selection for providers that support Responses versus Chat only. | -| `@ai-sdk/anthropic` | `AnthropicMessages` | Partial / usable | Finish Messages API parity for headers/betas/metadata/newer fields and document hosted-tool continuation expectations. | -| `@ai-sdk/google` | Gemini Developer API | Partial / usable | Add typed options for safety, response schema/modalities, cached content, grounding/search/code execution, and non-text output modes where supported. | -| `@ai-sdk/google-vertex` | Vertex Gemini namespace/facade | Partial / usable | Add runner/catalog mapping, recorded coverage, and broader provider-option parity. | -| `@ai-sdk/google-vertex/anthropic` | Anthropic Messages over Vertex namespace/facade | Partial / usable | Add runner/catalog mapping, recorded coverage, and Vertex-specific hosted-tool parity. | -| `@ai-sdk/google-vertex/maas` | Vertex Chat | Partial / usable | Add runner/catalog mapping, recorded coverage, and MaaS family-specific request parity. | -| `@ai-sdk/google-vertex/xai` | Vertex Chat / Responses | Partial / usable | Decide Chat/Responses selection for catalog models, add runner mapping and recorded coverage, and review xAI-specific request options. | -| `@ai-sdk/azure` | Azure OpenAI Chat/Responses facade | Partial | Map runner/catalog metadata to native Azure, handle resourceName/baseURL/apiVersion variants, add AAD/token auth story, and verify Chat vs Responses deployment selection. | -| `@ai-sdk/amazon-bedrock` | Bedrock Converse | Partial | Add default AWS credential chain/profile support, region/inference-profile model ID handling, provider option parity via `additionalModelRequestFields`, guardrails/performance config, and runner/catalog mapping. | -| `@ai-sdk/amazon-bedrock/mantle` | Bedrock Mantle OpenAI-compatible Chat/Responses namespace | Partial / usable | Add default AWS credential chain/profile support; native catalog mapping currently requires bearer auth or explicit static credentials. | +| AI SDK package | Intended native target | Status | Biggest gaps | +| --------------------------------- | --------------------------------------------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `@ai-sdk/openai` | `OpenAI.chat`, `OpenAI.responses` | Partial / usable | Add complete typed option coverage, structured output strategy, explicit Responses continuation support, and runner execution policy for optional WebSocket channels. | +| `@ai-sdk/openai-compatible` | Generic OpenAI-compatible Chat and Responses | Partial / usable | Decide per-family namespace/profile behavior and runner API selection for providers that support Responses versus Chat only. | +| `@ai-sdk/anthropic` | `AnthropicMessages` | Partial / usable | Finish Messages API parity for headers/betas/metadata/newer fields and document hosted-tool continuation expectations. | +| `@ai-sdk/google` | Gemini Developer API | Partial / usable | Add typed options for safety, response schema/modalities, cached content, grounding/search/code execution, and non-text output modes where supported. | +| `@ai-sdk/google-vertex` | Vertex Gemini namespace/facade | Partial / usable | Add runner/catalog mapping, recorded coverage, and broader provider-option parity. | +| `@ai-sdk/google-vertex/anthropic` | Anthropic Messages over Vertex namespace/facade | Partial / usable | Add runner/catalog mapping, recorded coverage, and Vertex-specific hosted-tool parity. | +| `@ai-sdk/google-vertex/maas` | Vertex Chat | Partial / usable | Add runner/catalog mapping, recorded coverage, and MaaS family-specific request parity. | +| `@ai-sdk/google-vertex/xai` | Vertex Chat / Responses | Partial / usable | Decide Chat/Responses selection for catalog models, add runner mapping and recorded coverage, and review xAI-specific request options. | +| `@ai-sdk/azure` | Azure OpenAI Chat/Responses facade | Partial | Map runner/catalog metadata to native Azure, handle resourceName/baseURL/apiVersion variants, add AAD/token auth story, and verify Chat vs Responses deployment selection. | +| `@ai-sdk/amazon-bedrock` | Bedrock Converse | Partial | Add default AWS credential chain/profile support, region/inference-profile model ID handling, provider option parity via `additionalModelRequestFields`, guardrails/performance config, and runner/catalog mapping. | +| `@ai-sdk/amazon-bedrock/mantle` | Bedrock Mantle OpenAI-compatible Chat/Responses namespace | Partial / usable | Add default AWS credential chain/profile support; native catalog mapping currently requires bearer auth or explicit static credentials. | ## Highest-Risk Gaps diff --git a/packages/ai/example/call-sites.md b/packages/ai/example/call-sites.md index 06b766cb28b..f8099fdbeee 100644 --- a/packages/ai/example/call-sites.md +++ b/packages/ai/example/call-sites.md @@ -67,7 +67,6 @@ Examples: ```ts OpenAI.responses("gpt-4o") OpenAI.chat("gpt-4o") -OpenAI.responsesWebSocket("gpt-4o") Azure.configure({ resourceName, apiKey }).responses("my-deployment") AmazonBedrock.configure({ region, credentials }).model("anthropic.claude-3-5-sonnet-20241022-v2:0") @@ -250,11 +249,6 @@ const openAIChat = Route.make({ auth: Auth.envBearer("OPENAI_API_KEY"), }) -const openAIResponsesWebSocket = openAIResponses.with({ - id: "openai-responses-websocket", - transport: WebSocketTransport.json, -}) - const openAIConfig = (input: OpenAIConfig) => ({ endpoint: input.endpoint, auth: input.auth ?? (input.apiKey ? Auth.bearer(input.apiKey) : undefined), @@ -266,13 +260,11 @@ const openAIConfig = (input: OpenAIConfig) => ({ const configureOpenAI = (input: OpenAIConfig = {}) => { const responses = openAIResponses.with(openAIConfig(input)) - const responsesWebSocket = openAIResponsesWebSocket.with(openAIConfig(input)) const chat = openAIChat.with(openAIConfig(input)) return { id: openAIProvider, responses: responses.model, - responsesWebSocket: responsesWebSocket.model, chat: chat.model, model: responses.model, configure: configureOpenAI, @@ -342,22 +334,19 @@ const response = ) ``` -For direct provider-facade calls, HTTP versus WebSocket is represented as named -route selectors, not as model or request overrides. Same protocol, different -transport, different route: +For direct provider-facade calls, Responses has one semantic model and route: ```ts OpenAI.responses("gpt-4o") -OpenAI.responsesWebSocket("gpt-4o") ``` -The package-like OpenAI Responses entrypoint instead keeps transport scoped to -Responses settings while preserving the same `model(...)` contract: +The package-like OpenAI Responses entrypoint has the same transport-neutral +`model(...)` contract: ```ts import { model } from "@opencode-ai/ai/providers/openai/responses" -model("gpt-4o", { apiKey, transport: "websocket" }) +model("gpt-4o", { apiKey }) ``` Vertex keeps Gemini, Chat, Responses, and Messages as separate package-like entrypoints, @@ -499,16 +488,13 @@ generic dynamic resolver: const model = providerID === "azure" ? Azure.configure(resolvedAzureConfig).responses(apiModelID) - : endpoint.websocket - ? OpenAI.responsesWebSocket(apiModelID) - : OpenAI.responses(apiModelID) + : OpenAI.responses(apiModelID) ``` That boundary can branch on durable config/catalog metadata and call typed -provider APIs directly. A direct provider-facade boundary maps metadata like -`endpoint.websocket` to `OpenAI.responsesWebSocket(apiModelID)`. A package-loading -boundary passes `transport: "websocket"` to the OpenAI Responses entrypoint. -The client runtime only executes the route carried by the resulting model. +provider APIs directly. Transport selection remains execution policy: a Session +or other caller may pass a WebSocket channel executor per call without changing +the model constructed by this boundary. ## Competitive Shape @@ -544,9 +530,8 @@ App boundary = explicit durable-config -> typed-provider call id. - No `model(id, overrides)` escape hatch. Model selection takes the model id; endpoint/auth/deployment customization happens by configuring the route first. -- No transport override on an executable model or request. Direct provider - facades use `responses` versus `responsesWebSocket`; the package-like Responses - entrypoint maps its scoped `transport` setting before constructing the model. +- No transport setting on a provider or executable model. OpenAI Responses uses + HTTP by default and accepts an optional per-call channel executor as execution policy. - No separate public `LLMClient.layerWithWebSocket`. The runtime should expose one client layer with the available transport capabilities. - No executable `ModelRef`. The executable handle is `LanguageModel`; durable model @@ -580,12 +565,10 @@ App boundary = explicit durable-config -> typed-provider call - [x] Make unconfigured transports reusable constants such as `HttpTransport.sseJson`; keep transport functions only for configured/fresh state construction. -- [x] Collapse the public WebSocket runtime split so one `LLMClient.layer` - exposes available transport capabilities and selected routes fail with typed - transport config errors when a required capability is missing. +- [x] Collapse the public WebSocket runtime split so one `LLMClient.layer` accepts + optional per-call channel execution without changing route identity. - [x] Convert OpenAI provider APIs to provider-facade shape: - `OpenAI.configure(config).responses(id)`, `.chat(id)`, and - `.responsesWebSocket(id)`. + `OpenAI.configure(config).responses(id)` and `.chat(id)`. - [x] Convert Azure to a configured facade where resource/base URL/api version setup happens before selecting deployment ids. - [x] Split Cloudflare products into separate facades such as diff --git a/packages/ai/src/protocols/open-responses.ts b/packages/ai/src/protocols/open-responses.ts index 7a987d07371..906be7784c6 100644 --- a/packages/ai/src/protocols/open-responses.ts +++ b/packages/ai/src/protocols/open-responses.ts @@ -233,7 +233,7 @@ export const WebSocketErrorEvent = Schema.StructWithRest( ) const decodeWebSocketErrorEvent = Schema.decodeUnknownEffect(WebSocketErrorEvent) -const decodeKnownErrorEvent = (event: Event) => +export const decodeKnownErrorEvent = (event: Event) => decodeWebSocketErrorEvent({ ...event, status: typeof event.status === "number" ? event.status : undefined, @@ -1001,7 +1001,7 @@ const providerErrorMessage = (event: Event, fallback: string): string => { return message || code || fallback } -const providerError = (state: ParserState, event: Event, fallback: string) => { +export const providerFailure = (id: string, event: Event, fallback: string) => { const code = event.code || event.error?.code || event.response?.error?.code || undefined const message = providerErrorMessage(event, fallback) const status = @@ -1011,12 +1011,14 @@ const providerError = (state: ParserState, event: Event, fallback: string) => { ? event.status_code : undefined return new AIError({ - module: state.id, + module: id, method: "stream", reason: classifyProviderFailure({ message, code, status }), }) } +const providerError = (state: ParserState, event: Event, fallback: string) => providerFailure(state.id, event, fallback) + export const step = (state: ParserState, event: Event) => { if (event.type === "response.output_text.delta" || event.type === "response.output_text.done") { if (!event.item_id) return ProviderShared.eventError(state.id, `${event.type} is missing item_id`) diff --git a/packages/ai/src/protocols/openai-responses-channel.ts b/packages/ai/src/protocols/openai-responses-channel.ts new file mode 100644 index 00000000000..3d06a8c57c1 --- /dev/null +++ b/packages/ai/src/protocols/openai-responses-channel.ts @@ -0,0 +1,37 @@ +import { Effect, Schema } from "effect" +import type { WebSocketChannelDriver } from "../route/transport" +import * as ProviderShared from "./shared" +import { OpenResponses } from "./open-responses" + +const ADAPTER = "openai-responses" +const NAME = "OpenAI Responses" +const decodeEvent = Schema.decodeUnknownEffect(OpenResponses.protocol.stream.event) + +export const make = (message: string): WebSocketChannelDriver => ({ + create: () => Effect.succeed({ message, mode: "full" }), + observe: (_create, frame) => + Effect.gen(function* () { + const event = yield* decodeEvent(frame).pipe( + Effect.mapError(() => ProviderShared.eventError(ADAPTER, "Invalid OpenAI Responses WebSocket event", frame)), + ) + if (event.type === "response.completed") return { type: "completed", frame } + if (event.type === "response.incomplete") return { type: "incomplete", frame } + if (event.type === "response.failed") + return { + type: "provider-failure", + error: OpenResponses.providerFailure(ADAPTER, event, `${NAME} response failed`), + } + if (event.type === "error") { + yield* OpenResponses.decodeKnownErrorEvent(event).pipe( + Effect.mapError(() => ProviderShared.eventError(ADAPTER, `${NAME} returned a malformed error event`, frame)), + ) + return { + type: "provider-failure", + error: OpenResponses.providerFailure(ADAPTER, event, `${NAME} stream error`), + } + } + return { type: "frame", frame } + }), +}) + +export const OpenAIResponsesChannel = { make } as const diff --git a/packages/ai/src/protocols/openai-responses.ts b/packages/ai/src/protocols/openai-responses.ts index 43237e89f60..d0b7d919d0a 100644 --- a/packages/ai/src/protocols/openai-responses.ts +++ b/packages/ai/src/protocols/openai-responses.ts @@ -1,15 +1,24 @@ -import { Effect, Encoding, Schema } from "effect" +import { Effect, Encoding, Schema, Stream } from "effect" +import { Headers } from "effect/unstable/http" import { Route } from "../route/client" import { Auth } from "../route/auth" import { Endpoint } from "../route/endpoint" +import { Framing } from "../route/framing" import { Protocol } from "../route/protocol" -import { HttpTransport, WebSocketTransport } from "../route/transport" +import { + HttpTransport, + WebSocketTransport, + type Transport, + type WebSocketChannelDriver, + type WebSocketChannelExchange, +} from "../route/transport" import { LLMEvent, LLMRequest, type JsonSchema, type ToolDefinition } from "../schema" import { OpenResponses } from "./open-responses" import { optionalArray, ProviderShared } from "./shared" import { Lifecycle } from "./utils/lifecycle" import { OpenAIImage } from "./utils/openai-image" import { ToolSchemaProjection } from "./utils/tool-schema" +import { OpenAIResponsesChannel } from "./openai-responses-channel" const ADAPTER = "openai-responses" const NAME = "OpenAI Responses" @@ -250,17 +259,6 @@ const auth = Auth.none export const httpTransport = HttpTransport.sseJson.with() -export const route = Route.make({ - id: ADAPTER, - provider: "openai", - providerMetadataKey: "openai", - protocol, - endpoint, - auth, - transport: httpTransport, - defaults: { providerOptions: { openai: { store: false } } }, -}) - const decodeWebSocketMessage = ProviderShared.validateWith(Schema.decodeUnknownEffect(OpenAIResponsesWebSocketMessage)) const webSocketMessage = (body: OpenAIResponsesBody | Record) => @@ -271,22 +269,58 @@ const webSocketMessage = (body: OpenAIResponsesBody | Record) = return yield* decodeWebSocketMessage({ ...message, type: "response.create" }) }) -export const webSocketTransport = WebSocketTransport.jsonTransport.with< - OpenAIResponsesBody, - OpenAIResponsesWebSocketMessage ->({ - toMessage: webSocketMessage, - encodeMessage: encodeWebSocketMessage, -}) +export interface OpenAIResponsesPrepared { + readonly http: HttpTransport.HttpPrepared + readonly channel?: { + readonly url: string + readonly headers: Headers.Headers + readonly driver: WebSocketChannelDriver + } +} -export const webSocketRoute = Route.make({ - id: `${ADAPTER}-websocket`, +export const transport: Transport = { + id: httpTransport.id, + prepare: (input) => + Effect.gen(function* () { + const parts = yield* HttpTransport.jsonRequestParts(input) + return { + http: { + request: ProviderShared.jsonPost({ url: parts.url, body: parts.bodyText, headers: parts.headers }), + framing: Framing.sse, + middleware: input.middleware, + }, + channel: input.webSocket + ? { + url: yield* WebSocketTransport.toWebSocketUrl(parts.url), + headers: parts.headers, + driver: OpenAIResponsesChannel.make(encodeWebSocketMessage(yield* webSocketMessage(parts.jsonBody))), + } + : undefined, + } + }), + execute: (prepared, request, runtime, options) => { + if (!options?.webSocket || !prepared.channel) return httpTransport.execute(prepared.http, request, runtime) + const exchange: WebSocketChannelExchange = { + id: request.id ?? "request", + connect: { url: prepared.channel.url, headers: prepared.channel.headers }, + fallback: () => + Stream.unwrap( + httpTransport.execute(prepared.http, request, runtime).pipe(Effect.map((execution) => execution.frames)), + ), + driver: prepared.channel.driver, + } + return options.webSocket.execute(exchange) + }, +} + +export const route = Route.make({ + id: ADAPTER, provider: "openai", providerMetadataKey: "openai", protocol, endpoint, auth, - transport: webSocketTransport, + transport, defaults: { providerOptions: { openai: { store: false } } }, }) diff --git a/packages/ai/src/providers/openai.ts b/packages/ai/src/providers/openai.ts index c8213644b8f..08e977a2c52 100644 --- a/packages/ai/src/providers/openai.ts +++ b/packages/ai/src/providers/openai.ts @@ -12,7 +12,7 @@ export type { OpenAIImageOptions } from "../protocols/openai-images" export const id = ProviderID.make("openai") -export const routes = [OpenAIResponses.route, OpenAIResponses.webSocketRoute, OpenAIChat.route] +export const routes = [OpenAIResponses.route, OpenAIChat.route] // This provider facade wraps the lower-level Responses and Chat model factories // with OpenAI-specific conveniences: typed options, API-key sugar, env fallback, @@ -63,7 +63,6 @@ export interface Settings extends ProviderPackage.Settings { readonly organization?: string readonly project?: string readonly queryParams?: Readonly> - readonly transport?: "http" | "websocket" readonly providerOptions?: OpenAIProviderOptionsInput } @@ -82,17 +81,12 @@ const configuredRoute = (route: Route, input: Co export const configure = (input: Config = {}) => { const responsesRoute = configuredRoute(OpenAIResponses.route, input) - const responsesWebSocketRoute = configuredRoute(OpenAIResponses.webSocketRoute, input) const chatRoute = configuredRoute(OpenAIChat.route, input) const modelDefaults = defaults(input) const responses = (id: string | ModelID) => responsesRoute .with(withOpenAIOptions(id, modelDefaults, { textVerbosity: true })) .model({ id }) - const responsesWebSocket = (id: string | ModelID) => - responsesWebSocketRoute - .with(withOpenAIOptions(id, modelDefaults, { textVerbosity: true })) - .model({ id }) const chat = (id: string | ModelID) => chatRoute.with(withOpenAIOptions(id, modelDefaults)).model({ id }) const image = (modelID: string | ModelID) => @@ -111,7 +105,6 @@ export const configure = (input: Config = {}) => { id, model: responses, responses, - responsesWebSocket, chat, image, configure, @@ -138,10 +131,7 @@ const config = (settings: Settings): Config => { } export const model: ProviderPackage.Definition["model"] = (modelID, settings) => { - const configured = configure(config(settings)) - if (settings.transport === undefined || settings.transport === "http") return configured.responses(modelID) - if (settings.transport === "websocket") return configured.responsesWebSocket(modelID) - throw new Error(`Unsupported OpenAI Responses transport: ${String(settings.transport)}`) + return configure(config(settings)).responses(modelID) } export const chatModel: ProviderPackage.Definition["model"] = ( @@ -149,6 +139,5 @@ export const chatModel: ProviderPackage.Definition configure(config(settings)).chat(modelID) export const responses = provider.responses -export const responsesWebSocket = provider.responsesWebSocket export const chat = provider.chat export const image = provider.image diff --git a/packages/ai/src/route/client.ts b/packages/ai/src/route/client.ts index c6eb0476ce6..6227c8cd106 100644 --- a/packages/ai/src/route/client.ts +++ b/packages/ai/src/route/client.ts @@ -314,6 +314,7 @@ function makeFromTransport( encodeBody, headers: routeInput.headers, middleware: options?.http, + webSocket: options?.webSocket, }), streamPrepared: (prepared: Prepared, request: LLMRequest, runtime: TransportRuntime, options?: StreamOptions) => { const route = `${request.model.provider}/${request.model.route.id}` diff --git a/packages/ai/src/route/transport/index.ts b/packages/ai/src/route/transport/index.ts index 646790e61ea..55d960b493e 100644 --- a/packages/ai/src/route/transport/index.ts +++ b/packages/ai/src/route/transport/index.ts @@ -38,6 +38,7 @@ export interface TransportPrepareInput { readonly encodeBody: (body: Body) => string readonly headers?: (input: { readonly request: LLMRequest }) => Record readonly middleware?: HttpMiddleware + readonly webSocket?: WebSocketChannelExecutor } export * as HttpTransport from "./http" diff --git a/packages/ai/src/route/transport/websocket.ts b/packages/ai/src/route/transport/websocket.ts index 914ae6b682f..aaf27ea0eca 100644 --- a/packages/ai/src/route/transport/websocket.ts +++ b/packages/ai/src/route/transport/websocket.ts @@ -146,7 +146,7 @@ const waitOpen = (ws: globalThis.WebSocket, input: WebSocketRequest) => { }) } -const webSocketUrl = (value: string) => +export const toWebSocketUrl = (value: string) => Effect.try({ try: () => { const url = new URL(value) @@ -351,7 +351,7 @@ export const json = (input: JsonInput): JsonTransp ...prepareInput, }) return { - url: yield* webSocketUrl(parts.url), + url: yield* toWebSocketUrl(parts.url), headers: parts.headers, message: input.encodeMessage(yield* input.toMessage(parts.jsonBody)), } @@ -403,4 +403,5 @@ export const WebSocketTransport = { open, fromWebSocket, messageText, + toWebSocketUrl, } as const diff --git a/packages/ai/test/executor.test.ts b/packages/ai/test/executor.test.ts index 1e16efd6a8f..37dd5ad3fe6 100644 --- a/packages/ai/test/executor.test.ts +++ b/packages/ai/test/executor.test.ts @@ -416,9 +416,7 @@ describe("RequestExecutor", () => { }) describe("WebSocket channel execution", () => { - const model = OpenAI.configure({ baseURL: "https://api.openai.test/v1/", apiKey: "test" }).responsesWebSocket( - "gpt-4.1-mini", - ) + const model = OpenAI.configure({ baseURL: "https://api.openai.test/v1/", apiKey: "test" }).responses("gpt-4.1-mini") const request = LLM.request({ model, prompt: "Say hello." }) const frames = [ JSON.stringify({ type: "response.output_text.delta", item_id: "msg_1", delta: "Hi" }), diff --git a/packages/ai/test/exports.test.ts b/packages/ai/test/exports.test.ts index 809cc314a7e..ec63d302727 100644 --- a/packages/ai/test/exports.test.ts +++ b/packages/ai/test/exports.test.ts @@ -45,7 +45,6 @@ describe("public exports", () => { expect(OpenAI.model).toBeFunction() expect(OpenAI.provider.responses).toBe(OpenAI.responses) - expect(OpenAI.provider.responsesWebSocket).toBe(OpenAI.responsesWebSocket) expect(OpenAI.configure({ apiKey: "fixture" }).responses).toBeFunction() expect(OpenAICompatible.deepseek.model).toBeFunction() expect( @@ -87,7 +86,6 @@ describe("public exports", () => { expect(OpenAICompatibleResponses.route.id).toBe("openai-compatible-responses") expect(OpenAICompatibleResponses.route.protocol).toBe("open-responses") expect(OpenAIResponses.route.id).toBe("openai-responses") - expect(OpenAIResponses.webSocketRoute.id).toBe("openai-responses-websocket") expect(AnthropicMessages.route.id).toBe("anthropic-messages") }) }) diff --git a/packages/ai/test/provider-options/openai.types.ts b/packages/ai/test/provider-options/openai.types.ts index 9546e1271b3..ece8e021c32 100644 --- a/packages/ai/test/provider-options/openai.types.ts +++ b/packages/ai/test/provider-options/openai.types.ts @@ -1,13 +1,18 @@ import { LLM } from "../../src" import { OpenAI } from "../../src/providers" -const model = OpenAI.responses("gpt-5") +const selected = OpenAI.responses("gpt-5") -LLM.request({ model, prompt: "Hello", providerOptions: { openai: { reasoningEffort: "high" } } }) +LLM.request({ model: selected, prompt: "Hello", providerOptions: { openai: { reasoningEffort: "high" } } }) LLM.request({ - model, + model: selected, prompt: "Hello", // @ts-expect-error OpenAI reasoning effort must be a string. providerOptions: { openai: { reasoningEffort: 1 } }, }) + +OpenAI.configure({ + // @ts-expect-error Transport is execution policy, not provider configuration. + transport: "websocket", +}) diff --git a/packages/ai/test/provider-package.test.ts b/packages/ai/test/provider-package.test.ts index 1e5caf308cd..43fd1b488c4 100644 --- a/packages/ai/test/provider-package.test.ts +++ b/packages/ai/test/provider-package.test.ts @@ -80,11 +80,6 @@ describe("provider package entrypoints", () => { expect(selected.route.defaults.limits).toEqual({ context: 200_000, output: 64_000 }) }) - test("selects transport without changing the semantic API", () => { - expect(model("gpt-5", { apiKey: "fixture" }).route.id).toBe("openai-responses") - expect(model("gpt-5", { apiKey: "fixture", transport: "websocket" }).route.id).toBe("openai-responses-websocket") - }) - test("maps OpenAI-compatible Responses settings onto the executable model", async () => { const OpenAICompatibleResponses = await import("@opencode-ai/ai/providers/openai-compatible/responses") const selected = OpenAICompatibleResponses.model("custom-model", { diff --git a/packages/ai/test/provider/openai-responses.test.ts b/packages/ai/test/provider/openai-responses.test.ts index 167e69737b6..4f0d022a81e 100644 --- a/packages/ai/test/provider/openai-responses.test.ts +++ b/packages/ai/test/provider/openai-responses.test.ts @@ -4,6 +4,7 @@ import { Headers, HttpClientRequest } from "effect/unstable/http" import { LLM, AIError, + HttpOptions, LLMEvent, LLMRequest, Message, @@ -217,19 +218,19 @@ describe("OpenAI Responses route", () => { }), ) - it.effect("prepares OpenAI Responses WebSocket target", () => + it.effect("prepares one OpenAI Responses route for either transport", () => Effect.gen(function* () { const prepared = yield* compileRequest( LLMRequest.update(request, { - model: OpenAIResponses.webSocketRoute + model: OpenAIResponses.route .with({ endpoint: { baseURL: "https://api.openai.test/v1/" }, auth: Auth.bearer("test") }) .model({ id: "gpt-4.1-mini" }), }), ) - expect(prepared.route).toBe("openai-responses-websocket") + expect(prepared.route).toBe("openai-responses") expect(prepared.protocol).toBe("openai-responses") - expect(prepared.metadata).toEqual({ transport: "websocket-json" }) + expect(prepared.metadata).toEqual({ transport: "http-json" }) expect(prepared.body).toMatchObject({ model: "gpt-4.1-mini", store: false, stream: true }) }), ) @@ -264,9 +265,7 @@ describe("OpenAI Responses route", () => { }) const response = yield* LLMClient.generate( LLM.request({ - model: OpenAI.configure({ baseURL: "https://api.openai.test/v1/", apiKey: "test" }).responsesWebSocket( - "gpt-4.1-mini", - ), + model: OpenAI.configure({ baseURL: "https://api.openai.test/v1/", apiKey: "test" }).responses("gpt-4.1-mini"), prompt: "Say hello.", }), { webSocket }, @@ -285,6 +284,85 @@ describe("OpenAI Responses route", () => { }), ) + it.effect("builds WebSocket and HTTP fallback from the same final request", () => + Effect.gen(function* () { + const attempts = yield* Ref.make(0) + const message = yield* Ref.make("") + const body = yield* Ref.make("") + const response = yield* LLMClient.generate( + LLM.request({ + model: OpenAI.configure({ baseURL: "https://api.openai.test/v1/", apiKey: "test" }).responses("gpt-4.1-mini"), + prompt: "Say hello.", + http: { + body: { model: "overlaid-model", metadata: { source: "overlay" } }, + headers: { "x-request": "request" }, + query: { mode: "test" }, + }, + }), + { + webSocket: { + execute: (exchange) => + Effect.gen(function* () { + yield* exchange.driver + .create(undefined) + .pipe(Effect.flatMap((create) => Ref.set(message, create.message))) + return { frames: exchange.fallback(), complete: Effect.void } + }), + }, + }, + ).pipe( + Effect.provide( + dynamicResponse((input) => + Effect.gen(function* () { + yield* Ref.update(attempts, (value) => value + 1) + yield* Ref.set(body, input.text) + expect(input.request.url).toBe("https://api.openai.test/v1/responses?mode=test") + expect(input.request.headers.authorization).toBe("Bearer test") + expect(input.request.headers["x-request"]).toBe("request") + return input.respond(sseEvents({ type: "response.completed", response: {} }), { + headers: { "content-type": "text/event-stream" }, + }) + }), + ), + ), + ) + + const httpBody = JSON.parse(yield* Ref.get(body)) + const { stream: _stream, ...shared } = httpBody + expect(response.finishReason?.normalized).toBe("stop") + expect(yield* Ref.get(attempts)).toBe(1) + expect(JSON.parse(yield* Ref.get(message))).toEqual({ type: "response.create", ...shared }) + expect(httpBody).toMatchObject({ + model: "overlaid-model", + metadata: { source: "overlay" }, + stream: true, + }) + }), + ) + + it.effect("uses exactly one HTTP request when no WebSocket executor is supplied", () => + Effect.gen(function* () { + const attempts = yield* Ref.make(0) + yield* LLMClient.generate( + LLMRequest.update(request, { http: new HttpOptions({ body: { input: "raw-http-input" } }) }), + ).pipe( + Effect.provide( + dynamicResponse((input) => + Effect.gen(function* () { + yield* Ref.update(attempts, (value) => value + 1) + expect(JSON.parse(input.text).input).toBe("raw-http-input") + return input.respond(sseEvents({ type: "response.completed", response: {} }), { + headers: { "content-type": "text/event-stream" }, + }) + }), + ), + ), + ) + + expect(yield* Ref.get(attempts)).toBe(1) + }), + ) + it.effect("closes a direct WebSocket execution after partial consumption", () => Effect.gen(function* () { const closed = yield* Ref.make(false) @@ -302,9 +380,7 @@ describe("OpenAI Responses route", () => { yield* LLMClient.stream( LLM.request({ - model: OpenAI.configure({ baseURL: "https://api.openai.test/v1/", apiKey: "test" }).responsesWebSocket( - "gpt-4.1-mini", - ), + model: OpenAI.configure({ baseURL: "https://api.openai.test/v1/", apiKey: "test" }).responses("gpt-4.1-mini"), prompt: "Say hello.", }), { webSocket }, @@ -347,7 +423,7 @@ describe("OpenAI Responses route", () => { const errors = yield* Effect.forEach(events, (event) => LLMClient.generate( LLM.request({ - model: OpenAI.configure({ baseURL: "https://api.openai.test/v1/", apiKey: "test" }).responsesWebSocket( + model: OpenAI.configure({ baseURL: "https://api.openai.test/v1/", apiKey: "test" }).responses( "gpt-4.1-mini", ), prompt: "Say hello.", @@ -409,7 +485,7 @@ describe("OpenAI Responses route", () => { close: Effect.void, }), }) - const model = OpenAI.configure({ baseURL: "https://api.openai.test/v1/", apiKey: "test" }).responsesWebSocket( + const model = OpenAI.configure({ baseURL: "https://api.openai.test/v1/", apiKey: "test" }).responses( "gpt-4.1-mini", )