anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-21 17:56:33 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

[PR #4799] FIX: "the prompt is too long" -- character aware token estimation #11123

New Issue
Closed
opened 2026-02-16 18:15:55 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 18:15:55 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/anomalyco/opencode/pull/4799

State: closed
Merged: No

Background

I continue to get "the prompt is too long" errors on a variety of Claude models. It's frequent enough to be noticeable as an issue for my team.

Findings

  1. It happens on requests when I was at ~25% context fill prior to sending the request
  2. It still happens despite modifying my opencode.json to have 10% lower context limits across all models.

Most recently, after changing Opus 4.1 to have 180k instead of 200k context, I still got "226170 > 200000" which is a full 25% higher than the 180k limit that was configured via opencode.

Hypothesis

Since most token counts are based actual from the LLM response, I judge that we are underestimating tokens when estimating. I think 1:4 is both very aggressive and not sensitive enough to differences in types of content. While for English-language text, my earlier estimates of 1:3.5 are roughly in agreement with 1:4, we diverge when it comes to numeric characters and delimiters.

Changes

I've replaced the original estimate of 4 characters per token, to a weighted estimate that counts different partial token amounts per character, depending on whether the character is a digit, letter, punctuation, or whitespace. These combine to create a weighted estimate which can then be multiplied by a factor provided in the environment variable, OPENCODE_TOKEN_FACTOR, in order to make the estimate more or less conservative.

Other approaches considered

  • I declined to add tiktoken though I think it would be wise, given that it would add dependencies for a non-functional upgrade.
  • Make the weights and overall factor available in the config file. Decided this would be heavy-handed until if/when different users might have enough data on their workloads, to be able effectively to tune those values.
  • I'd also be interested in making token estimation into a pluggable behavior, so that users could provide a function, e.g. (provider string, model string, text string) => number in order to be able to choose how these estimations are done. If one of y'all would give pointers on an acceptable way to do this, I'd be interested to do that.
**Original Pull Request:** https://github.com/anomalyco/opencode/pull/4799 **State:** closed **Merged:** No --- ## Background I continue to get "the prompt is too long" errors on a variety of Claude models. It's frequent enough to be noticeable as an issue for my team. ## Findings 1. It happens on requests when I was at ~25% context fill prior to sending the request 2. It still happens despite modifying my opencode.json to have 10% lower context limits across all models. Most recently, after changing Opus 4.1 to have 180k instead of 200k context, I still got "226170 > 200000" which is a full 25% higher than the 180k limit that was configured via opencode. ## Hypothesis Since most token counts are based _actual_ from the LLM response, I judge that we are underestimating tokens when estimating. I think 1:4 is both very aggressive and not sensitive enough to differences in types of content. While for English-language text, my earlier estimates of 1:3.5 are roughly in agreement with 1:4, we diverge when it comes to numeric characters and delimiters. ## Changes I've replaced the original estimate of 4 characters per token, to a weighted estimate that counts different partial token amounts per character, depending on whether the character is a digit, letter, punctuation, or whitespace. These combine to create a weighted estimate which can then be multiplied by a factor provided in the environment variable, OPENCODE_TOKEN_FACTOR, in order to make the estimate more or less conservative. ## Other approaches considered * I declined to add tiktoken though I think it would be wise, given that it would add dependencies for a non-functional upgrade. * Make the weights and overall factor available in the config file. Decided this would be heavy-handed until if/when different users might have enough data on their workloads, to be able effectively to tune those values. * I'd also be interested in making token estimation into a pluggable behavior, so that users could provide a function, e.g. `(provider string, model string, text string) => number` in order to be able to choose how these estimations are done. If one of y'all would give pointers on an acceptable way to do this, I'd be interested to do that.
yindo added the pull-request label 2026-02-16 18:15:55 -05:00
yindo closed this issue 2026-02-16 18:15:55 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#11123
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?