anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 02:05:26 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

[PR #11328] fix(opencode): Throttle storage writes during streaming to reduce I/O overhead #13739

New Issue
Open
opened 2026-02-16 18:18:34 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 18:18:34 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/anomalyco/opencode/pull/11328

State: open
Merged: No

Problem

During streaming responses, OpenCode writes to storage (JSON files) on every reasoning-delta event. With fast streaming APIs, this results in hundreds of file writes per second, causing:

  1. High I/O overhead from frequent file system operations
  2. Increased latency between receiving content and displaying it
  3. Slower perceived response time compared to native CLI tools (e.g., kiro-cli)

Fixes #11329

Analysis

I compared OpenCode with kiro-cli using the same Kiro API:

Aspect kiro-cli OpenCode
API endpoint Same (q.us-east-1.amazonaws.com) Same
TTFB 2-10s 2-10s
Delta handling Memory only File write per delta
Perceived speed Fast Slower

The bottleneck is in processor.ts:

case "reasoning-delta":
  if (part.text) await Session.updatePart({ part, delta: value.text })
  // This writes to file system on EVERY delta!

Session.updatePart calls Storage.write which does:

await Bun.write(target, JSON.stringify(content, null, 2))

Solution

Implement throttled storage writes for reasoning-delta events:

  • Accumulate content in memory (already happening via part.text +=)
  • Only flush to storage every 50ms (STREAMING_UPDATE_THROTTLE_MS)
  • Always flush on reasoning-end to ensure data persistence
  • No change to final content or persistence guarantees

Expected Impact

  • ~20x fewer file writes during streaming (assuming 1 delta/2.5ms average)
  • Reduced I/O overhead and CPU usage
  • Faster perceived response time
  • No change to final message content

Changes

  • Added STREAMING_UPDATE_THROTTLE_MS constant (50ms)
  • Added throttle state tracking (lastUpdateTime, pendingReasoningUpdates)
  • Modified reasoning-delta handler to throttle storage writes
  • Ensured reasoning-end always flushes pending updates
**Original Pull Request:** https://github.com/anomalyco/opencode/pull/11328 **State:** open **Merged:** No --- ## Problem During streaming responses, OpenCode writes to storage (JSON files) on **every reasoning-delta event**. With fast streaming APIs, this results in hundreds of file writes per second, causing: 1. High I/O overhead from frequent file system operations 2. Increased latency between receiving content and displaying it 3. Slower perceived response time compared to native CLI tools (e.g., kiro-cli) Fixes #11329 ## Analysis I compared OpenCode with kiro-cli using the same Kiro API: | Aspect | kiro-cli | OpenCode | |--------|----------|----------| | API endpoint | Same (`q.us-east-1.amazonaws.com`) | Same | | TTFB | 2-10s | 2-10s | | Delta handling | Memory only | **File write per delta** | | Perceived speed | Fast | Slower | The bottleneck is in `processor.ts`: ```typescript case "reasoning-delta": if (part.text) await Session.updatePart({ part, delta: value.text }) // This writes to file system on EVERY delta! ``` `Session.updatePart` calls `Storage.write` which does: ```typescript await Bun.write(target, JSON.stringify(content, null, 2)) ``` ## Solution Implement throttled storage writes for reasoning-delta events: - Accumulate content in memory (already happening via `part.text +=`) - Only flush to storage every **50ms** (`STREAMING_UPDATE_THROTTLE_MS`) - Always flush on `reasoning-end` to ensure data persistence - No change to final content or persistence guarantees ## Expected Impact - **~20x fewer file writes** during streaming (assuming 1 delta/2.5ms average) - Reduced I/O overhead and CPU usage - Faster perceived response time - No change to final message content ## Changes - Added `STREAMING_UPDATE_THROTTLE_MS` constant (50ms) - Added throttle state tracking (`lastUpdateTime`, `pendingReasoningUpdates`) - Modified `reasoning-delta` handler to throttle storage writes - Ensured `reasoning-end` always flushes pending updates
yindo added the pull-request label 2026-02-16 18:18:34 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#13739
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?