anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 10:15:31 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

SSE Stream Bug: Out-of-Order thinking_delta via LiteLLM → AWS Bedrock #2376

New Issue
Open
opened 2026-02-16 17:35:23 -05:00 by yindo · 5 comments
yindo commented 2026-02-16 17:35:23 -05:00
Owner
Copy Link
Copy Source

Originally created by @zenyr on GitHub (Oct 30, 2025).

Reported with help from Claude Haiku 4.5 and OpenCode v0.15.29

Issue Summary

Proxying Claude Haiku 4.5 through LiteLLM → AWS Bedrock Converse API occasionally delivers thinking_delta blocks out-of-order, causing OpenCode stream processor crash.

  • Problem: thinking_delta arrives BEFORE message_start (intermittent, unpredictable)
  • Impact: state machine violation → stream crash → incomplete response
  • Frequency: rare, unknown cause
  • NOT: Anthropic or Bedrock official API issue

Root Cause Analysis

Request Path

OpenCode → LiteLLM → AWS Bedrock Converse API
  ↓
LiteLLM reconstructs Bedrock response as Anthropic SSE
  ↓
Out-of-order blocks, missing cache tokens (evidence: cache_read_input_tokens = 0)
  ↓
OpenCode stream processor crashes

Evidence LiteLLM Is Reconstructing

  • Cache tokens all zero: indicates response recomposition, not direct forwarding
  • Block order violation: Bedrock typically maintains order for direct clients
  • Specific to LiteLLM proxy: doesn't occur with direct API calls
  • Conclusion: LiteLLM's stream handler buffers → reconstructs → emits SSE with modified order

Suspected Mechanism

LiteLLM's stream() method:

  1. Buffers chunks from Bedrock
  2. Reconstructs Anthropic-compatible structure
  3. Emits SSE with reordered blocks
  4. Loses metadata (cache tokens)

Failure Scenario

Observed sequence (malformed):

1. content_block_delta (index: 1, type: thinking_delta) ❌ BEFORE message_start
2. message_start
3. content_block_start (index: 0, type: text)
4. content_block_delta (index: 0, type: text_delta) × N
5. ...

Expected (correct):

1. message_start
2. content_block_start (index: 0, type: text)
3. content_block_delta (index: 0, type: text_delta) × N
4. ...

OpenCode processor behavior:

  • Stream handler expects: text-start → text-delta+ → text-end
  • Receives: mixed text-delta + reasoning-delta interleaved
  • State machine violation → updatePart() fails → crash

Impact

Component Severity Symptom
Stream processing HIGH Out-of-order blocks crash state machine
Response HIGH Incomplete/corrupted output
Token tracking MEDIUM Reasoning tokens lost
User experience MEDIUM Unpredictable stream failures

Proposed Solutions

Option 1 (Simple): Skip unhandled delta types (e.g., thinking_delta) in stream processor, continue processing valid blocks → solves immediately.

Option 2 (Robust): Initialize missing start events on-demand + wrap updatePart in try-catch → handles multiple failure modes.

Option 3 (Recommended): Combine both → prevent issues upstream + graceful degradation downstream.

Context

  • Not Anthropic official API: confirmed different behavior with direct API
  • Not Bedrock issue: Converse API maintains order for direct clients
  • LiteLLM proxy layer bug: response reconstruction doesn't preserve ordering
  • Intermittent: cause unknown, no reliable reproduction steps
  • Components affected: stream processor
  • Related: cache token metadata loss confirms response recomposition

Repro cases

[Non-fatal] Example of invalid tool-use out of nowhere
  • I'll share another fatal one if I happened to encounter one, similar to this format:
event: message_start
data: {"type":"message_start","message":{"id":"msg_test","type":"message","role":"assistant","content":[],"model":"claude-haiku-4.5","stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":0,"output_tokens":0}}}

event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"The architecture"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" has three layers:"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"\n\n1. Parser layer - handles input"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"\n2. Type guard layer - validates types"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"\n3. Domain model layer - business logic"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"\n\nThis pattern is useful for UI builders"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" and query optimization."}}

event: content_block_stop
data: {"type":"content_block_stop","index":0}

event: content_block_start
data: {"type":"content_block_start","index":1,"content_block":{"type":"tool_use","id":"tooluse_test123","name":"TodoWrite","input":{}}}

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":""}}

event: content_block_stop
data: {"type":"content_block_stop","index":1}

event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"input_tokens":1000,"output_tokens":300}}

event: message_stop
data: {"type":"message_stop"}
Originally created by @zenyr on GitHub (Oct 30, 2025). **Reported with help from Claude Haiku 4.5 and OpenCode v0.15.29** --- ## Issue Summary Proxying Claude Haiku 4.5 through **LiteLLM → AWS Bedrock Converse API** occasionally delivers `thinking_delta` blocks **out-of-order**, causing OpenCode stream processor crash. - **Problem**: thinking_delta arrives BEFORE message_start (intermittent, unpredictable) - **Impact**: state machine violation → stream crash → incomplete response - **Frequency**: rare, unknown cause - **NOT**: Anthropic or Bedrock official API issue --- ## Root Cause Analysis ### Request Path ``` OpenCode → LiteLLM → AWS Bedrock Converse API ↓ LiteLLM reconstructs Bedrock response as Anthropic SSE ↓ Out-of-order blocks, missing cache tokens (evidence: cache_read_input_tokens = 0) ↓ OpenCode stream processor crashes ``` ### Evidence LiteLLM Is Reconstructing - **Cache tokens all zero**: indicates response recomposition, not direct forwarding - **Block order violation**: Bedrock typically maintains order for direct clients - **Specific to LiteLLM proxy**: doesn't occur with direct API calls - **Conclusion**: LiteLLM's stream handler buffers → reconstructs → emits SSE with modified order ### Suspected Mechanism LiteLLM's `stream()` method: 1. Buffers chunks from Bedrock 2. Reconstructs Anthropic-compatible structure 3. Emits SSE with reordered blocks 4. Loses metadata (cache tokens) --- ## Failure Scenario **Observed sequence** (malformed): ``` 1. content_block_delta (index: 1, type: thinking_delta) ❌ BEFORE message_start 2. message_start 3. content_block_start (index: 0, type: text) 4. content_block_delta (index: 0, type: text_delta) × N 5. ... ``` **Expected** (correct): ``` 1. message_start 2. content_block_start (index: 0, type: text) 3. content_block_delta (index: 0, type: text_delta) × N 4. ... ``` **OpenCode processor behavior**: - Stream handler expects: `text-start → text-delta+ → text-end` - Receives: mixed `text-delta` + `reasoning-delta` interleaved - State machine violation → `updatePart()` fails → crash --- ## Impact | Component | Severity | Symptom | | ----------------- | -------- | --------------------------------------- | | Stream processing | HIGH | Out-of-order blocks crash state machine | | Response | HIGH | Incomplete/corrupted output | | Token tracking | MEDIUM | Reasoning tokens lost | | User experience | MEDIUM | Unpredictable stream failures | --- ## Proposed Solutions **Option 1 (Simple)**: Skip unhandled delta types (e.g., thinking_delta) in stream processor, continue processing valid blocks → solves immediately. **Option 2 (Robust)**: Initialize missing start events on-demand + wrap updatePart in try-catch → handles multiple failure modes. **Option 3 (Recommended)**: Combine both → prevent issues upstream + graceful degradation downstream. --- ## Context - **Not Anthropic official API**: confirmed different behavior with direct API - **Not Bedrock issue**: Converse API maintains order for direct clients - **LiteLLM proxy layer bug**: response reconstruction doesn't preserve ordering - **Intermittent**: cause unknown, no reliable reproduction steps - **Components affected**: stream processor - **Related**: cache token metadata loss confirms response recomposition ## Repro cases <details><summary>[Non-fatal] Example of invalid tool-use out of nowhere</summary> - I'll share another fatal one if I happened to encounter one, similar to this format: ``` event: message_start data: {"type":"message_start","message":{"id":"msg_test","type":"message","role":"assistant","content":[],"model":"claude-haiku-4.5","stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":0,"output_tokens":0}}} event: content_block_start data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}} event: content_block_delta data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"The architecture"}} event: content_block_delta data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" has three layers:"}} event: content_block_delta data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"\n\n1. Parser layer - handles input"}} event: content_block_delta data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"\n2. Type guard layer - validates types"}} event: content_block_delta data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"\n3. Domain model layer - business logic"}} event: content_block_delta data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"\n\nThis pattern is useful for UI builders"}} event: content_block_delta data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" and query optimization."}} event: content_block_stop data: {"type":"content_block_stop","index":0} event: content_block_start data: {"type":"content_block_start","index":1,"content_block":{"type":"tool_use","id":"tooluse_test123","name":"TodoWrite","input":{}}} event: content_block_delta data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":""}} event: content_block_stop data: {"type":"content_block_stop","index":1} event: message_delta data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"input_tokens":1000,"output_tokens":300}} event: message_stop data: {"type":"message_stop"} ``` </details>
yindo commented 2026-02-16 17:35:23 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Oct 30, 2025):

Can u show the error you see?

@rekram1-node commented on GitHub (Oct 30, 2025): Can u show the error you see?
yindo commented 2026-02-16 17:35:23 -05:00
Author
Owner
Copy Link
Copy Source

@zenyr commented on GitHub (Oct 31, 2025):

Can u show the error you see?

It originated from message-v2.ts, but I couldn't copy the "broken UI" since it didn't respond to mouse events. I'll update it here once it happens again. I searched through .local/share/opencode but didn't find useful error logs.

@zenyr commented on GitHub (Oct 31, 2025): > Can u show the error you see? It originated from [message-v2.ts](https://github.com/sst/opencode/blob/dev/packages/opencode/src/session/message-v2.ts), but I couldn't copy the "broken UI" since it didn't respond to mouse events. I'll update it here once it happens again. I searched through `.local/share/opencode` but didn't find useful error logs.
yindo commented 2026-02-16 17:35:23 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Oct 31, 2025):

Oof okay good to know we need to be better at preventing the broken ui states yuck!

@rekram1-node commented on GitHub (Oct 31, 2025): Oof okay good to know we need to be better at preventing the broken ui states yuck!
yindo commented 2026-02-16 17:35:23 -05:00
Author
Owner
Copy Link
Copy Source

@zenyr commented on GitHub (Oct 31, 2025):

Oof okay good to know we need to be better at preventing the broken ui states yuck!

For that matter I wonder if this could fall into "not-a-bug" category, since, you know, garbage got in and the garbage broke our UI. 🤷 Better be safe though!

@zenyr commented on GitHub (Oct 31, 2025): > Oof okay good to know we need to be better at preventing the broken ui states yuck! For that matter I wonder if this could fall into "not-a-bug" category, since, you know, _garbage got in_ and the garbage broke our UI. 🤷 Better be safe though!
yindo commented 2026-02-16 17:35:23 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Oct 31, 2025):

I think it definitely shouldn't break the ui so we should fix that, bug works for me

I'd understand if it got to a weird state but ik the corruption ur describing we dont want any of that

@rekram1-node commented on GitHub (Oct 31, 2025): I think it definitely shouldn't break the ui so we should fix that, bug works for me I'd understand if it got to a weird state but ik the corruption ur describing we dont want any of that
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#2376
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?