anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 02:05:26 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

[Feature Request] Configuration to Limit Input Tokens per Request #1738

New Issue
Closed
opened 2026-02-16 17:32:24 -05:00 by yindo · 4 comments
yindo commented 2026-02-16 17:32:24 -05:00
Owner
Copy Link
Copy Source

Originally created by @JDR42 on GitHub (Sep 16, 2025).

Originally assigned to: @thdxr on GitHub.

Is your feature request related to a problem? Please describe.

Yes. Currently, when in a long chat session, opencode appears to send the entire chat history with every new prompt. For very long sessions (e.g., sessions with over 1 million tokens of history), this results in significant and often unnecessary costs.

For many tasks, only the most recent portion of the conversation is relevant for generating the next response. For example, I may only need the last 50,000 tokens of context, not the full 1 million. As it stands, I am paying for the processing of the entire chat history on every single request, which is financially wasteful and increases API latency. There is currently no way to control this behavior without manually clearing the session, which causes the entire local context to be lost.

Describe the solution you'd like

I would like a configuration setting in opencode.json that allows users to specify a maximum number of input tokens to be sent to the model for any given request.

For example, a user could set:

{
  "maxInputTokens": 50000
}

When this setting is active, the opencode client would intelligently truncate the chat history before sending it to the API, ensuring that only the most recent and relevant parts of the conversation (up to the specified token limit) are included in the payload.

This would give users direct control over their API costs and latency without forcing them to lose the valuable context available locally in the terminal.

As LLM models continue to increase their maximum context window, I imagine the need for this type of feature will only increase over time.

Describe alternatives you've considered

  1. Manually clearing the chat history (/clear): This is a destructive action. It forces the user to erase valuable context that they may want to scroll back and refer to locally. The goal is to preserve the local chat history while limiting the data sent over the API.
  2. Continuing as is: This is not a sustainable option for users who engage in long, complex coding sessions, as costs can quickly become prohibitive.

Additional context

This feature would be a critical improvement for power users and anyone on a budget. It provides essential control over token consumption, which is a key factor in using any LLM-based tool.

This type of context management is a standard feature in many other LLM interfaces (for example, aider has a max_input_tokens advanced model setting), and adding it to opencode would make it an even more powerful and economically viable tool for professional developers. Thank you for considering this feature.

Slight variation

Perhaps a better name should be maxChatInputLength. And instead of using tokens, we would just do some kind of simple truncation of the chat session by character length. All of the context related to the system prompt, tools, and what-not can be kept at 100%, but the chat would be truncated. This variant captures the spirit of the idea, which is to limit the massive amounts of waste that are typical for many of our agentic sessions which require only a shallow context.

Originally created by @JDR42 on GitHub (Sep 16, 2025). Originally assigned to: @thdxr on GitHub. **Is your feature request related to a problem? Please describe.** Yes. Currently, when in a long chat session, `opencode` appears to send the entire chat history with every new prompt. For very long sessions (e.g., sessions with over 1 million tokens of history), this results in significant and often unnecessary costs. For many tasks, only the most recent portion of the conversation is relevant for generating the next response. For example, I may only need the last 50,000 tokens of context, not the full 1 million. As it stands, I am paying for the processing of the entire chat history on every single request, which is financially wasteful and increases API latency. There is currently no way to control this behavior without manually clearing the session, which causes the entire local context to be lost. **Describe the solution you'd like** I would like a configuration setting in `opencode.json` that allows users to specify a maximum number of input tokens to be sent to the model for any given request. For example, a user could set: ```json { "maxInputTokens": 50000 } ``` When this setting is active, the `opencode` client would intelligently truncate the chat history before sending it to the API, ensuring that only the most recent and relevant parts of the conversation (up to the specified token limit) are included in the payload. This would give users direct control over their API costs and latency without forcing them to lose the valuable context available locally in the terminal. As LLM models continue to increase their maximum context window, I imagine the need for this type of feature will only increase over time. **Describe alternatives you've considered** 1. **Manually clearing the chat history (`/clear`):** This is a destructive action. It forces the user to erase valuable context that they may want to scroll back and refer to locally. The goal is to preserve the local chat history while limiting the data sent over the API. 2. **Continuing as is:** This is not a sustainable option for users who engage in long, complex coding sessions, as costs can quickly become prohibitive. **Additional context** This feature would be a critical improvement for power users and anyone on a budget. It provides essential control over token consumption, which is a key factor in using any LLM-based tool. This type of context management is a standard feature in many other LLM interfaces (for example, **aider** has a `max_input_tokens` advanced model setting), and adding it to `opencode` would make it an even more powerful and economically viable tool for professional developers. Thank you for considering this feature. **Slight variation** Perhaps a better name should be `maxChatInputLength`. And instead of using tokens, we would just do some kind of simple truncation of the chat session by character length. All of the context related to the system prompt, tools, and what-not can be kept at 100%, but the chat would be truncated. This variant captures the spirit of the idea, which is to limit the massive amounts of waste that are typical for many of our agentic sessions which require only a shallow context.
yindo closed this issue 2026-02-16 17:32:24 -05:00
yindo commented 2026-02-16 17:32:24 -05:00
Author
Owner
Copy Link
Copy Source

@github-actions[bot] commented on GitHub (Sep 16, 2025):

This issue might be a duplicate of existing issues. Please check:

  • #1990: [Feature Request] Add User Controls for Context Management - This proposes similar context management controls including token usage breakdown and context size management
  • #924: Dramatic increase in token consumption - Reports similar concerns about high token usage and costs in long sessions
  • #1172: high input token usage on a small project - Describes similar issues with excessive input token usage leading to blocked workflows
  • #1573: [suggestion] save tokens by adding an ASK MODE - Proposes token-saving features for situations where full context isn't needed

Feel free to ignore if none of these address your specific case.

@github-actions[bot] commented on GitHub (Sep 16, 2025): This issue might be a duplicate of existing issues. Please check: - #1990: [Feature Request] Add User Controls for Context Management - This proposes similar context management controls including token usage breakdown and context size management - #924: Dramatic increase in token consumption - Reports similar concerns about high token usage and costs in long sessions - #1172: high input token usage on a small project - Describes similar issues with excessive input token usage leading to blocked workflows - #1573: [suggestion] save tokens by adding an ASK MODE - Proposes token-saving features for situations where full context isn't needed Feel free to ignore if none of these address your specific case.
yindo commented 2026-02-16 17:32:24 -05:00
Author
Owner
Copy Link
Copy Source

@thdxr commented on GitHub (Sep 17, 2025):

what model do you use?

@thdxr commented on GitHub (Sep 17, 2025): what model do you use?
yindo commented 2026-02-16 17:32:24 -05:00
Author
Owner
Copy Link
Copy Source

@JDR42 commented on GitHub (Sep 17, 2025):

Many but the real standouts for this feature request would be those setups that lack caching such as Qwen3 Coder 480B A35B on Cerebras.

Consider long-running chats with numerous simple, "shallow" prompts.

For example, each of these...

  • "Change it back to 28 pixels"
  • "Show me all staged files in this folder"
  • "Commit all outstanding changes to the repo"

...doesn't require 128K of chat context

@JDR42 commented on GitHub (Sep 17, 2025): Many but the real standouts for this feature request would be those setups that lack caching such as `Qwen3 Coder 480B A35B` on Cerebras. Consider long-running chats with numerous simple, "shallow" prompts. For example, each of these... * "Change it back to 28 pixels" * "Show me all staged files in this folder" * "Commit all outstanding changes to the repo" ...doesn't require 128K of chat context
yindo commented 2026-02-16 17:32:24 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Dec 27, 2025):

[automated] Closing due to 90+ days of inactivity. Feel free to reopen if you still need this!

@rekram1-node commented on GitHub (Dec 27, 2025): [automated] Closing due to 90+ days of inactivity. Feel free to reopen if you still need this!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#1738
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?