anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 10:15:31 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

[PR #10190] fix(opencode): resolve performance bottleneck in prompt loop #13363

New Issue
Open
opened 2026-02-16 18:18:13 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 18:18:13 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/anomalyco/opencode/pull/10190

State: open
Merged: No

Replace expensive remeda clone() with lazy shallow copying that only copies messages/parts that actually need modification.

This fixes 30+ second delays when sending messages in sessions with 200+ messages.

What does this PR do?

This PR replaces the expensive remeda.clone() call in [packages/opencode/src/session/prompt.ts] with a performance-optimized lazy shallow copy pattern.

The Problem:
In the main prompt loop, the code was performing a full deep clone of the entire message history on every iteration. For large sessions (200+ messages), this expensive operation caused extreme performance degradation, resulting in 30-180 second delays before the LLM could start generating tokens.

The Solution:
Implemented a lazy mapping strategy. Instead of cloning everything defensively:

  • We keep the original message references by default.
  • We only clone a specific message and its parts if modifications are actually required (e.g., when wrapping un-processed user messages with <system-reminder>).
  • This maintains immutability where it matters without the O(n*m) deep-clone overhead.

How did you verify your code works?

  1. Performance Test: Verified on a session with ~250 messages.
    • Before: 30 - 180 seconds delay (CPU pinned during clone).
    • After: Response starts in < 5 seconds.
  2. Logic Verification: Confirmed that the system-reminder wrapper is still correctly applied to new user messages as expected.
  3. Immutability Check: Verified that the original message objects in the history remain untouched unless they are the ones being specifically modified for the prompt.

Fixes #10187

**Original Pull Request:** https://github.com/anomalyco/opencode/pull/10190 **State:** open **Merged:** No --- Replace expensive remeda clone() with lazy shallow copying that only copies messages/parts that actually need modification. This fixes 30+ second delays when sending messages in sessions with 200+ messages. ### What does this PR do? This PR replaces the expensive `remeda.clone()` call in [packages/opencode/src/session/prompt.ts] with a performance-optimized lazy shallow copy pattern. **The Problem:** In the main prompt loop, the code was performing a full deep clone of the entire message history on every iteration. For large sessions (200+ messages), this expensive operation caused extreme performance degradation, resulting in 30-180 second delays before the LLM could start generating tokens. **The Solution:** Implemented a lazy mapping strategy. Instead of cloning everything defensively: - We keep the original message references by default. - We only clone a specific message and its parts if modifications are actually required (e.g., when wrapping un-processed user messages with `<system-reminder>`). - This maintains immutability where it matters without the O(n*m) deep-clone overhead. ### How did you verify your code works? 1. **Performance Test**: Verified on a session with ~250 messages. - **Before**: 30 - 180 seconds delay (CPU pinned during clone). - **After**: Response starts in < 5 seconds. 2. **Logic Verification**: Confirmed that the `system-reminder` wrapper is still correctly applied to new user messages as expected. 3. **Immutability Check**: Verified that the original message objects in the history remain untouched unless they are the ones being specifically modified for the prompt. Fixes #10187
yindo added the pull-request label 2026-02-16 18:18:13 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#13363
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?