anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 02:05:26 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

[FEATURE]:Feature: Local LLM Title Generation with @xenova/transformers #2847

New Issue
Open
opened 2026-02-16 17:37:31 -05:00 by yindo · 1 comment
yindo commented 2026-02-16 17:37:31 -05:00
Owner
Copy Link
Copy Source

Originally created by @riatzukiza on GitHub (Nov 13, 2025).

Feature hasn't been suggested before.

  • I have verified this feature I'm about to request hasn't been suggested before.

Describe the enhancement you want to request

Story

Users report "mystery" token usage from title generation despite documentation. Replace/augment cloud-based title generation with local inference using @xenova/transformers to eliminate external API calls and enable offline operation (when hardware will will allow)

Tasks

  1. Add @huggingface/transformers dependency and create local title generator module
  2. Implement hardware auto-detection (NVIDIA/AMD/Intel GPU, CPU fallback)
  3. Add configuration options for local title generation with cloud fallback
  4. Integrate local generator into existing title generation flow
  5. Test on target hardware and ensure <2s generation time

Definition of Done

  • Local title generation working with at least one small model (distilgpt2/t5-small)
  • Hardware acceleration auto-detection implemented
  • Graceful fallback to cloud models when local fails (preferring free models from zen when available)
  • No external API calls for title generation when enabled
  • Documentation updated with configuration and hardware requirements, only attempting local model when hardware reasonably permits (check ram, gpu, npu, corecount, etc)
Originally created by @riatzukiza on GitHub (Nov 13, 2025). ### Feature hasn't been suggested before. - [x] I have verified this feature I'm about to request hasn't been suggested before. ### Describe the enhancement you want to request ## Story Users report "mystery" token usage from title generation despite documentation. Replace/augment cloud-based title generation with local inference using @xenova/transformers to eliminate external API calls and enable offline operation (when hardware will will allow) ## Tasks 1. Add @huggingface/transformers dependency and create local title generator module 2. Implement hardware auto-detection (NVIDIA/AMD/Intel GPU, CPU fallback) 3. Add configuration options for local title generation with cloud fallback 4. Integrate local generator into existing title generation flow 5. Test on target hardware and ensure <2s generation time ## Definition of Done - Local title generation working with at least one small model (distilgpt2/t5-small) - Hardware acceleration auto-detection implemented - Graceful fallback to cloud models when local fails (preferring free models from zen when available) - No external API calls for title generation when enabled - Documentation updated with configuration and hardware requirements, only attempting local model when hardware reasonably permits (check ram, gpu, npu, corecount, etc)
yindo added the discussion label 2026-02-16 17:37:31 -05:00
yindo commented 2026-02-16 17:37:31 -05:00
Author
Owner
Copy Link
Copy Source

@github-actions[bot] commented on GitHub (Nov 13, 2025):

This issue might be a duplicate of existing issues. Please check:

  • #4133: [FEATURE]: Use a cheap model to name sessions - This addresses the exact same concern about "mystery token usage from title generation" and proposes using a cheaper model for session naming, with async execution and tool-free requests.

  • #3862: bad title generation - Reports issues with title generation functionality

  • #4255: OpenCode v1.0.25 Hangs Indefinitely with LM Studio + Qwen Models - Extensive discussion about local model support, mentions "preferring free models from zen when available" which aligns with your fallback strategy

  • #4002: [FEATURE]: Use GPU for Rendering the TUI - Discusses hardware detection (GPU detection) which is part of your hardware auto-detection requirement

  • #1480: Local LLM Support on MAC with LM Studio - Addresses local LLM configuration without API providers

  • #2224: Suppport for airgapped installation - Requests preventing external API calls, similar to your "no external API calls when enabled" requirement

Feel free to ignore if none of these address your specific case of implementing @xenova/transformers for local title generation.

@github-actions[bot] commented on GitHub (Nov 13, 2025): This issue might be a duplicate of existing issues. Please check: - #4133: [FEATURE]: Use a cheap model to name sessions - This addresses the exact same concern about "mystery token usage from title generation" and proposes using a cheaper model for session naming, with async execution and tool-free requests. - #3862: bad title generation - Reports issues with title generation functionality - #4255: OpenCode v1.0.25 Hangs Indefinitely with LM Studio + Qwen Models - Extensive discussion about local model support, mentions "preferring free models from zen when available" which aligns with your fallback strategy - #4002: [FEATURE]: Use GPU for Rendering the TUI - Discusses hardware detection (GPU detection) which is part of your hardware auto-detection requirement - #1480: Local LLM Support on MAC with LM Studio - Addresses local LLM configuration without API providers - #2224: Suppport for airgapped installation - Requests preventing external API calls, similar to your "no external API calls when enabled" requirement Feel free to ignore if none of these address your specific case of implementing @xenova/transformers for local title generation.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label discussion
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#2847
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?