[PR #6376] feat: experimental durable streaming architecture #11867

Open
opened 2026-02-16 18:16:48 -05:00 by yindo · 0 comments
Owner

Original Pull Request: https://github.com/anomalyco/opencode/pull/6376

State: open
Merged: No


    ╔═══════════════════════════════════════════════════════════════╗
    ║                                                               ║
    ║   ██████╗ ██╗   ██╗██████╗  █████╗ ██████╗ ██╗     ███████╗   ║
    ║   ██╔══██╗██║   ██║██╔══██╗██╔══██╗██╔══██╗██║     ██╔════╝   ║
    ║   ██║  ██║██║   ██║██████╔╝███████║██████╔╝██║     █████╗     ║
    ║   ██║  ██║██║   ██║██╔══██╗██╔══██║██╔══██╗██║     ██╔══╝     ║
    ║   ██████╔╝╚██████╔╝██║  ██║██║  ██║██████╔╝███████╗███████╗   ║
    ║   ╚═════╝  ╚═════╝ ╚═╝  ╚═╝╚═╝  ╚═╝╚═════╝ ╚══════╝╚══════╝   ║
    ║                                                               ║
    ║   ███████╗████████╗██████╗ ███████╗ █████╗ ███╗   ███╗███████╗║
    ║   ██╔════╝╚══██╔══╝██╔══██╗██╔════╝██╔══██╗████╗ ████║██╔════╝║
    ║   ███████╗   ██║   ██████╔╝█████╗  ███████║██╔████╔██║███████╗║
    ║   ╚════██║   ██║   ██╔══██╗██╔══╝  ██╔══██║██║╚██╔╝██║╚════██║║
    ║   ███████║   ██║   ██║  ██║███████╗██║  ██║██║ ╚═╝ ██║███████║║
    ║   ╚══════╝   ╚═╝   ╚═╝  ╚═╝╚══════╝╚═╝  ╚═╝╚═╝     ╚═╝╚══════╝║
    ║                                                               ║
    ╚═══════════════════════════════════════════════════════════════╝

Why

SSE connections drop. Networks flake. Clients reconnect. When they do, they miss events.

Durable streams solve this: persist events with monotonic offsets, let clients catch up from where they left off. No lost messages, no polling, no complexity on the client side.

How

┌─────────────────────────────────────────────────────────────┐
│                        DATA FLOW                            │
├───���─────────────────────────────────────────────────────────┤
│                                                             │
│   Bus.publish(event)                                        │
│         │                                                   │
│         ▼                                                   │
│   EventStore.append() ──► SQLite + ULID offset              │
│         │                                                   │
│         ▼                                                   │
│   GET /stream/events?offset=X                               │
│         │                                                   │
│         ├──► Catch-up: JSON array of missed events          │
│         └──► Live: SSE stream from offset onwards           │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Config-gated. Zero behavior change when disabled:

// opencode.jsonc
{
  "experimental": {
    "durableStreams": true
  }
}

What's in the box

Component File Purpose
Config flag config.ts experimental.durableStreams (default: false)
EventStore event-store/ SQLite persistence, ULID offsets, catch-up queries
ServerRegistry server/registry.ts JSON file-based multi-server discovery
Stream endpoint server/stream.ts GET /stream/events (catch-up + live SSE)
Discovery server/discovery.ts GET/POST/DELETE /servers
Persistence hook bus/index.ts Auto-persist on Bus.publish()
Self-registration cli/cmd/serve.ts 15s heartbeat, graceful shutdown

Offset semantics

  • Offsets are opaque ULID strings (lexicographically sortable)
  • offset="-1" = start from beginning
  • Response includes Stream-Next-Offset header for resumption
  • Client stores offset, reconnects with it, gets exactly what they missed

Tests

407 pass | 0 fail

Full coverage on EventStore and ServerRegistry with edge cases for catch-up semantics, payload robustness, and ULID monotonicity.

**Original Pull Request:** https://github.com/anomalyco/opencode/pull/6376 **State:** open **Merged:** No --- ``` ╔═══════════════════════════════════════════════════════════════╗ ║ ║ ║ ██████╗ ██╗ ██╗██████╗ █████╗ ██████╗ ██╗ ███████╗ ║ ║ ██╔══██╗██║ ██║██╔══██╗██╔══██╗██╔══██╗██║ ██╔════╝ ║ ║ ██║ ██║██║ ██║██████╔╝███████║██████╔╝██║ █████╗ ║ ║ ██║ ██║██║ ██║██╔══██╗██╔══██║██╔══██╗██║ ██╔══╝ ║ ║ ██████╔╝╚██████╔╝██║ ██║██║ ██║██████╔╝███████╗███████╗ ║ ║ ╚═════╝ ╚═════╝ ╚═╝ ╚═╝╚═╝ ╚═╝╚═════╝ ╚══════╝╚══════╝ ║ ║ ║ ║ ███████╗████████╗██████╗ ███████╗ █████╗ ███╗ ███╗███████╗║ ║ ██╔════╝╚══██╔══╝██╔══██╗██╔════╝██╔══██╗████╗ ████║██╔════╝║ ║ ███████╗ ██║ ██████╔╝█████╗ ███████║██╔████╔██║███████╗║ ║ ╚════██║ ██║ ██╔══██╗██╔══╝ ██╔══██║██║╚██╔╝██║╚════██║║ ║ ███████║ ██║ ██║ ██║███████╗██║ ██║██║ ╚═╝ ██║███████║║ ║ ╚══════╝ ╚═╝ ╚═╝ ╚═╝╚══════╝╚═╝ ╚═╝╚═╝ ╚═╝╚══════╝║ ║ ║ ╚═══════════════════════════════════════════════════════════════╝ ``` ## Why SSE connections drop. Networks flake. Clients reconnect. When they do, they miss events. Durable streams solve this: persist events with monotonic offsets, let clients catch up from where they left off. No lost messages, no polling, no complexity on the client side. ## How ``` ┌─────────────────────────────────────────────────────────────┐ │ DATA FLOW │ ├───���─────────────────────────────────────────────────────────┤ │ │ │ Bus.publish(event) │ │ │ │ │ ▼ │ │ EventStore.append() ──► SQLite + ULID offset │ │ │ │ │ ▼ │ │ GET /stream/events?offset=X │ │ │ │ │ ├──► Catch-up: JSON array of missed events │ │ └──► Live: SSE stream from offset onwards │ │ │ └─────────────────────────────────────────────────────────────┘ ``` **Config-gated.** Zero behavior change when disabled: ```jsonc // opencode.jsonc { "experimental": { "durableStreams": true } } ``` ## What's in the box | Component | File | Purpose | |-----------|------|---------| | Config flag | `config.ts` | `experimental.durableStreams` (default: false) | | EventStore | `event-store/` | SQLite persistence, ULID offsets, catch-up queries | | ServerRegistry | `server/registry.ts` | JSON file-based multi-server discovery | | Stream endpoint | `server/stream.ts` | `GET /stream/events` (catch-up + live SSE) | | Discovery | `server/discovery.ts` | `GET/POST/DELETE /servers` | | Persistence hook | `bus/index.ts` | Auto-persist on Bus.publish() | | Self-registration | `cli/cmd/serve.ts` | 15s heartbeat, graceful shutdown | ## Offset semantics - Offsets are opaque ULID strings (lexicographically sortable) - `offset="-1"` = start from beginning - Response includes `Stream-Next-Offset` header for resumption - Client stores offset, reconnects with it, gets exactly what they missed ## Tests ``` 407 pass | 0 fail ``` Full coverage on EventStore and ServerRegistry with edge cases for catch-up semantics, payload robustness, and ULID monotonicity.
yindo added the pull-request label 2026-02-16 18:16:48 -05:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#11867