anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 02:05:26 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

[FEATURE]: Add local token estimation fallback when API usage is empty #9073

New Issue
Open
opened 2026-02-16 18:11:34 -05:00 by yindo · 1 comment
yindo commented 2026-02-16 18:11:34 -05:00
Owner
Copy Link
Copy Source

Originally created by @FurryWolfX on GitHub (Feb 11, 2026).

Originally assigned to: @thdxr on GitHub.

Feature hasn't been suggested before.

  • I have verified this feature I'm about to request hasn't been suggested before.

Describe the enhancement you want to request

Problem

Some custom providers' OpenAI-compatible APIs (streaming) don't return usage information in their responses. This causes two critical issues:

  1. Context usage not displayed: opencode cannot show token consumption to users
  2. Automatic compact not triggered: The compact mechanism relies on usage data to know when to trigger, so it fails entirely when usage is missing

Context

Users are integrating with various custom OpenAI-compatible providers that may not include usage metadata in their streaming response chunks. This is a common scenario for:

  • Self-hosted language model services
  • Custom proxy implementations
  • Alternative LLM providers that don't follow the full OpenAI API specification

Without accurate usage tracking, users lose visibility into their token consumption and, more importantly, the automatic compact feature becomes non-functional, potentially leading to context overflow.

Request

Add a fallback mechanism to locally estimate token usage when the API response doesn't include usage information. This should:

  • Apply primarily to streaming responses (where usage is most commonly missing)
  • Maintain compatibility with providers that do return usage (prefer actual usage when available)
  • Ensure the automatic compact mechanism continues to work even with estimated data
  • Be reasonably accurate for practical purposes

Proposed Solution

  1. Implement a local token counter that tracks tokens as streaming chunks arrive
  2. When API response usage is empty or missing:
    • Use the local estimation as a fallback
    • Log a warning that usage is estimated (not from API)
  3. Keep using actual API usage when it's available (for accuracy)
  4. The estimation should handle both prompt tokens and completion tokens separately

Additional Notes

Token estimation libraries like tiktoken or simple character-based heuristics could be used for the fallback implementation. The estimation doesn't need to be perfect—just accurate enough to enable the compact functionality to work reliably.

Originally created by @FurryWolfX on GitHub (Feb 11, 2026). Originally assigned to: @thdxr on GitHub. ### Feature hasn't been suggested before. - [x] I have verified this feature I'm about to request hasn't been suggested before. ### Describe the enhancement you want to request ## Problem Some custom providers' OpenAI-compatible APIs (streaming) don't return usage information in their responses. This causes two critical issues: 1. **Context usage not displayed**: opencode cannot show token consumption to users 2. **Automatic compact not triggered**: The compact mechanism relies on usage data to know when to trigger, so it fails entirely when usage is missing ## Context Users are integrating with various custom OpenAI-compatible providers that may not include usage metadata in their streaming response chunks. This is a common scenario for: - Self-hosted language model services - Custom proxy implementations - Alternative LLM providers that don't follow the full OpenAI API specification Without accurate usage tracking, users lose visibility into their token consumption and, more importantly, the automatic compact feature becomes non-functional, potentially leading to context overflow. ## Request Add a fallback mechanism to locally estimate token usage when the API response doesn't include usage information. This should: - Apply primarily to streaming responses (where usage is most commonly missing) - Maintain compatibility with providers that do return usage (prefer actual usage when available) - Ensure the automatic compact mechanism continues to work even with estimated data - Be reasonably accurate for practical purposes ## Proposed Solution 1. Implement a local token counter that tracks tokens as streaming chunks arrive 2. When API response usage is empty or missing: - Use the local estimation as a fallback - Log a warning that usage is estimated (not from API) 3. Keep using actual API usage when it's available (for accuracy) 4. The estimation should handle both prompt tokens and completion tokens separately ## Additional Notes Token estimation libraries like `tiktoken` or simple character-based heuristics could be used for the fallback implementation. The estimation doesn't need to be perfect—just accurate enough to enable the compact functionality to work reliably.
yindo added the discussion label 2026-02-16 18:11:34 -05:00
yindo commented 2026-02-16 18:11:34 -05:00
Author
Owner
Copy Link
Copy Source

@github-actions[bot] commented on GitHub (Feb 11, 2026):

This issue might be a duplicate of or closely related to existing issues. Please check:

  • #6152: [FEATURE] Session context usage (similar token estimation with fallback approach)
  • #10634: Compaction overflow check doesn't account for large tool outputs (token estimation and context overflow handling)
  • #5210: Custom OpenAI-compatible provider returns no text content (missing usage information in streaming responses)

You may want to review these issues to see if they address your use case or if there's overlap in the proposed solutions.

@github-actions[bot] commented on GitHub (Feb 11, 2026): --- This issue might be a duplicate of or closely related to existing issues. Please check: - #6152: [FEATURE] Session context usage (similar token estimation with fallback approach) - #10634: Compaction overflow check doesn't account for large tool outputs (token estimation and context overflow handling) - #5210: Custom OpenAI-compatible provider returns no text content (missing usage information in streaming responses) You may want to review these issues to see if they address your use case or if there's overlap in the proposed solutions.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label discussion
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#9073
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?