[PR #3531] Fix "bufio.Scanner token too long" error by replacing Scanner with Reader in SSE #10687

Closed
opened 2026-02-16 18:15:25 -05:00 by yindo · 0 comments
Owner

Original Pull Request: https://github.com/anomalyco/opencode/pull/3531

State: closed
Merged: Yes


Related issues:

Fix bufio.Scanner token too long error in SSE decoder

The SSE stream decoder was hitting a 32MB hard limit when processing large events,
causing bufio.Scanner: token too long errors. This blocked users from reading large
files through the TUI.

Changes:

• Register UnboundedDecoder for the text/event-stream

    ssestream.RegisterDecoder("text/event-stream", decoders.NewUnboundedDecoder)

• Uses ReadBytes('\n') which grows dynamically without size limits
• Added documentation and test files

Testing: Verified with events up to 50MB. All tests pass.

**Original Pull Request:** https://github.com/anomalyco/opencode/pull/3531 **State:** closed **Merged:** Yes --- ## Related issues: - #3409 - #3502 - #3506 ## Fix bufio.Scanner token too long error in SSE decoder The SSE stream decoder was hitting a 32MB hard limit when processing large events, causing bufio.Scanner: token too long errors. This blocked users from reading large files through the TUI. Changes: • Register UnboundedDecoder for the `text/event-stream` ``` ssestream.RegisterDecoder("text/event-stream", decoders.NewUnboundedDecoder) ``` • Uses ReadBytes('\n') which grows dynamically without size limits • Added documentation and test files Testing: Verified with events up to 50MB. All tests pass.
yindo added the pull-request label 2026-02-16 18:15:25 -05:00
yindo closed this issue 2026-02-16 18:15:25 -05:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#10687