anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 10:15:31 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

[Feature Request] MCP Tool Search - Lazy Loading for 85% Token Reduction #6757

New Issue
Open
opened 2026-02-16 18:05:10 -05:00 by yindo · 4 comments
yindo commented 2026-02-16 18:05:10 -05:00
Owner
Copy Link
Copy Source

Originally created by @dewu0224 on GitHub (Jan 19, 2026).

Originally assigned to: @thdxr on GitHub.

Feature Request: MCP Tool Search - Lazy Loading for Token Optimization

Summary

Implement MCP Tool Search functionality to enable lazy loading of MCP server tools, dramatically reducing token consumption and improving context efficiency for multi-MCP server setups.

Problem Statement

Currently, OpenCode loads all tool definitions from all connected MCP servers at session startup. This causes severe context window bloat:

  • 7+ MCP servers: Consume 67,000+ tokens before any user interaction
  • Single Docker MCP server (135 tools): Consumes 125,000 tokens
  • Typical 4-server setup: Burns 51,000 tokens (46.9% of context window)
  • Result: Users lose 50-70% of their 200K context limit before writing a single prompt

This creates a brutal tradeoff: either limit MCP servers to 2-3 core tools, or accept that half your context budget disappears before work begins.

Proposed Solution: MCP Tool Search (Lazy Loading)

Implement a lazy loading system that:

  1. Loads only a lightweight search index (~5K tokens) at startup instead of all tool definitions
  2. Dynamically fetches tool definitions only when needed for a specific task
  3. Automatically activates when tool descriptions would exceed 10% of available context
  4. Maintains backward compatibility with all existing MCP servers

Expected Benefits

Metric Before After Improvement
MCP Tools Token Consumption 39.8K tokens (19.9%) ~5K tokens (2.5%) 85% reduction
Available Context 92K tokens (46%) 195K tokens (97.5%) 112% increase
4-Server Setup 51K tokens 8.5K tokens 46.9% reduction
Tool Selection Accuracy (Opus 4.5) 79.5% 88.1% 10.8% increase
Tool Selection Accuracy (Opus 4) 49% 74% 51% increase

Implementation Approach

1. Core Architecture

  • Create a lightweight tool registry/index at session startup
  • Implement on-demand tool definition loading via similarity search (Regex or BM25)
  • Use serverInstructions field metadata for intelligent tool discovery

2. Configuration Options

// Global settings
{
  "enable_tool_search": true  // Auto-enabled when beneficial
}

// Per-MCP server configuration
{
  "mcpServers": {
    "my-server": {
      "command": "node",
      "args": ["/path/to/server.js"],
      "serverInstructions": "Database operations for PostgreSQL including queries, schema management, and data migrations. Use for any database-related tasks."
    }
  }
}

3. Activation Logic

  • Automatic: Activates when tool descriptions > 10% of context window
  • Manual Override: Allow users to enable/disable via settings
  • Per-Server Control: Option to disable for specific high-frequency tools

4. User Experience

  • Transparent operation - no workflow changes required
  • Monitor usage via /context and /mcp commands
  • Clear indication of which tools are loaded on-demand

Technical Implementation Details

Search Index Creation

  • Index tool names, descriptions, and serverInstructions
  • Use BM25 or Regex-based similarity search for matching user queries to tools
  • Keep index under 5K tokens total

On-Demand Loading

  • Intercept tool invocation requests
  • Query index for relevant tools
  • Load only matched tool definitions into context
  • Cache loaded tools for session duration

Server Instructions Enhancement

{
  "serverInstructions": "Database operations for PostgreSQL including queries, schema management, and data migrations. Use for any database-related tasks. Supports SQL queries, table creation, schema migration, and backup operations."
}

References

Use Cases

  1. Multi-MCP Users: Run 7+ servers without context exhaustion
  2. Complex Workflows: Maintain conversation history across extended sessions
  3. Tool-Rich Environments: Access 100+ tools without performance penalty
  4. Enterprise Deployments: Scale tool integrations without token constraints

Priority

High - This is a critical feature for scalability and user experience. It removes the primary constraint limiting MCP adoption and enables more sophisticated agent workflows.

Additional Considerations

  • Backward Compatibility: Must work with all existing MCP servers
  • Performance: Search operations should be fast (<100ms)
  • Monitoring: Provide tools to track token savings
  • Migration Path: Smooth transition from current behavior

Related Features

  • Programmatic Tool Use
  • Advanced Tool Discovery
  • Context Optimization
  • MCP Server Management

This feature would significantly enhance OpenCode's ability to handle complex, multi-tool workflows while maintaining optimal context efficiency. The implementation approach follows established patterns from Anthropic's Claude Code and represents a significant leap forward in AI coding agent architecture.

Originally created by @dewu0224 on GitHub (Jan 19, 2026). Originally assigned to: @thdxr on GitHub. ## Feature Request: MCP Tool Search - Lazy Loading for Token Optimization ### Summary Implement MCP Tool Search functionality to enable lazy loading of MCP server tools, dramatically reducing token consumption and improving context efficiency for multi-MCP server setups. ### Problem Statement Currently, OpenCode loads all tool definitions from all connected MCP servers at session startup. This causes severe context window bloat: - **7+ MCP servers**: Consume 67,000+ tokens before any user interaction - **Single Docker MCP server** (135 tools): Consumes 125,000 tokens - **Typical 4-server setup**: Burns 51,000 tokens (46.9% of context window) - **Result**: Users lose 50-70% of their 200K context limit before writing a single prompt This creates a brutal tradeoff: either limit MCP servers to 2-3 core tools, or accept that half your context budget disappears before work begins. ### Proposed Solution: MCP Tool Search (Lazy Loading) Implement a lazy loading system that: 1. **Loads only a lightweight search index** (~5K tokens) at startup instead of all tool definitions 2. **Dynamically fetches tool definitions** only when needed for a specific task 3. **Automatically activates** when tool descriptions would exceed 10% of available context 4. **Maintains backward compatibility** with all existing MCP servers ### Expected Benefits | Metric | Before | After | Improvement | |--------|--------|-------|-------------| | MCP Tools Token Consumption | 39.8K tokens (19.9%) | ~5K tokens (2.5%) | **85% reduction** | | Available Context | 92K tokens (46%) | 195K tokens (97.5%) | **112% increase** | | 4-Server Setup | 51K tokens | 8.5K tokens | **46.9% reduction** | | Tool Selection Accuracy (Opus 4.5) | 79.5% | 88.1% | **10.8% increase** | | Tool Selection Accuracy (Opus 4) | 49% | 74% | **51% increase** | ### Implementation Approach #### 1. Core Architecture - Create a lightweight tool registry/index at session startup - Implement on-demand tool definition loading via similarity search (Regex or BM25) - Use `serverInstructions` field metadata for intelligent tool discovery #### 2. Configuration Options ```json // Global settings { "enable_tool_search": true // Auto-enabled when beneficial } // Per-MCP server configuration { "mcpServers": { "my-server": { "command": "node", "args": ["/path/to/server.js"], "serverInstructions": "Database operations for PostgreSQL including queries, schema management, and data migrations. Use for any database-related tasks." } } } ``` #### 3. Activation Logic - **Automatic**: Activates when tool descriptions > 10% of context window - **Manual Override**: Allow users to enable/disable via settings - **Per-Server Control**: Option to disable for specific high-frequency tools #### 4. User Experience - Transparent operation - no workflow changes required - Monitor usage via `/context` and `/mcp` commands - Clear indication of which tools are loaded on-demand ### Technical Implementation Details #### Search Index Creation - Index tool names, descriptions, and `serverInstructions` - Use BM25 or Regex-based similarity search for matching user queries to tools - Keep index under 5K tokens total #### On-Demand Loading - Intercept tool invocation requests - Query index for relevant tools - Load only matched tool definitions into context - Cache loaded tools for session duration #### Server Instructions Enhancement ```json { "serverInstructions": "Database operations for PostgreSQL including queries, schema management, and data migrations. Use for any database-related tasks. Supports SQL queries, table creation, schema migration, and backup operations." } ``` ### References - **Anthropic's MCP Tool Search**: Released January 2026 - **Original Feature Request**: GitHub anthropics/claude-code#7336 - **Documentation**: https://claudefa.st/blog/tools/mcp-extensions/mcp-tool-search - **Analysis**: https://venturebeat.com/orchestration/claude-code-just-got-updated-with-one-of-the-most-requested-user-features - **Performance Data**: https://medium.com/@joe.njenga/claude-code-just-cut-mcp-context-bloat-by-46-9-51k-tokens-down-to-8-5k-with-new-tool-search-ddf9e905f734 ### Use Cases 1. **Multi-MCP Users**: Run 7+ servers without context exhaustion 2. **Complex Workflows**: Maintain conversation history across extended sessions 3. **Tool-Rich Environments**: Access 100+ tools without performance penalty 4. **Enterprise Deployments**: Scale tool integrations without token constraints ### Priority **High** - This is a critical feature for scalability and user experience. It removes the primary constraint limiting MCP adoption and enables more sophisticated agent workflows. ### Additional Considerations - **Backward Compatibility**: Must work with all existing MCP servers - **Performance**: Search operations should be fast (<100ms) - **Monitoring**: Provide tools to track token savings - **Migration Path**: Smooth transition from current behavior ### Related Features - Programmatic Tool Use - Advanced Tool Discovery - Context Optimization - MCP Server Management --- This feature would significantly enhance OpenCode's ability to handle complex, multi-tool workflows while maintaining optimal context efficiency. The implementation approach follows established patterns from Anthropic's Claude Code and represents a significant leap forward in AI coding agent architecture.
yindo commented 2026-02-16 18:05:10 -05:00
Author
Owner
Copy Link
Copy Source

@github-actions[bot] commented on GitHub (Jan 19, 2026):

This issue might be a duplicate of existing issues. Please check:

  • #8277: [FEATURE]: Lazy/dynamic loading for mcp tools - proposes lazy loading pattern with a meta-tool for on-demand tool loading
  • #8625: [FEATURE]:Add mcp search tool, reduce mcp tool occupying a lot of context - directly mentions implementing MCP search tool similar to Claude Code
  • #7406: [FEATURE]: Support MCP-CLI for on-demand loading of MCP tools for reduced token consumption - discusses MCP-CLI for on-demand tool loading
  • #7399: [FEATURE]: Filter which tools are loaded from MCP servers (includeTools/excludeTools) - addresses tool filtering to reduce context overhead

Feel free to ignore if this issue includes additional details or a different implementation approach than those listed above.

@github-actions[bot] commented on GitHub (Jan 19, 2026): This issue might be a duplicate of existing issues. Please check: - #8277: [FEATURE]: Lazy/dynamic loading for mcp tools - proposes lazy loading pattern with a meta-tool for on-demand tool loading - #8625: [FEATURE]:Add mcp search tool, reduce mcp tool occupying a lot of context - directly mentions implementing MCP search tool similar to Claude Code - #7406: [FEATURE]: Support MCP-CLI for on-demand loading of MCP tools for reduced token consumption - discusses MCP-CLI for on-demand tool loading - #7399: [FEATURE]: Filter which tools are loaded from MCP servers (includeTools/excludeTools) - addresses tool filtering to reduce context overhead Feel free to ignore if this issue includes additional details or a different implementation approach than those listed above.
yindo commented 2026-02-16 18:05:10 -05:00
Author
Owner
Copy Link
Copy Source

@famitzsy8 commented on GitHub (Jan 19, 2026):

A fork is being implemented partially addressing this problem here. #9461 discusses the issue in a more general way

@famitzsy8 commented on GitHub (Jan 19, 2026): A fork is being implemented partially addressing this problem [here](https://github.com/famitzsy8/opencode-tool-search-tool). #9461 discusses the issue in a more general way
yindo commented 2026-02-16 18:05:10 -05:00
Author
Owner
Copy Link
Copy Source

@TheOutdoorProgrammer commented on GitHub (Feb 6, 2026):

Hey — thanks for filing this! There are several open issues requesting essentially the same thing (tool search / deferred tool loading / lazy MCP loading). To consolidate discussion and improve visibility, it'd be great if this could be closed in favor of https://github.com/anomalyco/opencode/issues/8625 which has the most thumbs up on the topic. Consolidating makes it easier for maintainers to see community demand in one place.

Related issues tracking the same feature:

Semi related issues that would compliment this (for visibility):

I've rebased #8771 here. I've been running it for a few days its working amazing. Looking forward to this being part of OpenCode.

@TheOutdoorProgrammer commented on GitHub (Feb 6, 2026): Hey — thanks for filing this! There are several open issues requesting essentially the same thing (tool search / deferred tool loading / lazy MCP loading). To consolidate discussion and improve visibility, it'd be great if this could be closed in favor of https://github.com/anomalyco/opencode/issues/8625 which has the most thumbs up on the topic. Consolidating makes it easier for maintainers to see community demand in one place. Related issues tracking the same feature: - #9461 - #9350 - #8277 Semi related issues that would compliment this (for visibility): - #7399 I've rebased #8771 [here](https://github.com/anomalyco/opencode/pull/12520). I've been running it for a few days its working amazing. Looking forward to this being part of OpenCode.
yindo commented 2026-02-16 18:05:10 -05:00
Author
Owner
Copy Link
Copy Source

@Dumbris commented on GitHub (Feb 10, 2026):

fwiw this already exists as a standalone proxy. https://github.com/smart-mcp-proxy/mcpproxy-go sits between your client and MCP servers, does BM25 search over tool descriptions per query, and only returns the relevant tools. works with opencode or any MCP client without requiring code changes on the client side. the 85% number in the title is conservative — in my setup it's closer to 97% reduction

@Dumbris commented on GitHub (Feb 10, 2026): fwiw this already exists as a standalone proxy. https://github.com/smart-mcp-proxy/mcpproxy-go sits between your client and MCP servers, does BM25 search over tool descriptions per query, and only returns the relevant tools. works with opencode or any MCP client without requiring code changes on the client side. the 85% number in the title is conservative — in my setup it's closer to 97% reduction
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#6757
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?