anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 18:25:32 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

[FEATURE] Add a batch read tool #1285

New Issue
Closed
opened 2026-02-16 17:30:18 -05:00 by yindo · 9 comments
yindo commented 2026-02-16 17:30:18 -05:00
Owner
Copy Link
Copy Source

Originally created by @leeweisern on GitHub (Aug 13, 2025).

Originally assigned to: @thdxr on GitHub.

I think gemini have something similar:

Image

Passing a folder instead of a file, the LLM batch reads everything in the folder. This could be a great tool too for subagents. Currently they read 1 by 1, maybe having a tool that can batch read files, using subagents will be even faster.

Originally created by @leeweisern on GitHub (Aug 13, 2025). Originally assigned to: @thdxr on GitHub. I think gemini have something similar: <img width="829" height="420" alt="Image" src="https://github.com/user-attachments/assets/59bc0d0a-7b92-40b7-a1b4-ea924d4040ad" /> Passing a folder instead of a file, the LLM batch reads everything in the folder. This could be a great tool too for subagents. Currently they read 1 by 1, maybe having a tool that can batch read files, using subagents will be even faster.
yindo closed this issue 2026-02-16 17:30:18 -05:00
yindo commented 2026-02-16 17:30:18 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Aug 14, 2025):

So you are looking for attachments to have directory support correct?

leaving as note to self, related: #1464, #1466

@rekram1-node commented on GitHub (Aug 14, 2025): So you are looking for attachments to have directory support correct? leaving as note to self, related: #1464, #1466
yindo commented 2026-02-16 17:30:18 -05:00
Author
Owner
Copy Link
Copy Source

@leeweisern commented on GitHub (Aug 14, 2025):

Yes, Gemini cli does this well. It concatenate all nested files recursively.

I am also thinking this could be a separate tool, like a more powerful read tool. I find subagents usually read file one by one. If this tool exists in their arsenal, they can gather context more quickly?

@leeweisern commented on GitHub (Aug 14, 2025): Yes, Gemini cli does this well. It concatenate all nested files recursively. I am also thinking this could be a separate tool, like a more powerful read tool. I find subagents usually read file one by one. If this tool exists in their arsenal, they can gather context more quickly?
yindo commented 2026-02-16 17:30:18 -05:00
Author
Owner
Copy Link
Copy Source

@DaleLJefferson commented on GitHub (Aug 20, 2025):

Roocode has this feature as well, massively reduces time to read context.

@DaleLJefferson commented on GitHub (Aug 20, 2025): Roocode has this feature as well, massively reduces time to read context.
yindo commented 2026-02-16 17:30:18 -05:00
Author
Owner
Copy Link
Copy Source

@Sewer56 commented on GitHub (Sep 6, 2025):

With the advent of faster models running on beefier hardware, tool call overhead begins to dominate during codebase investigation/discovery phases- such as when using the PLAN agent, or creating your own plan via a /slash command.

As prompt processing tends to run at 10s of thousands of tokens per second, even the mainstream models (e.g. Claude) can become severely slown down by unnecessarily alternating between input and output during discovery. After reading a file, the LLM will often decide it wants to read 3-4 more files, to investigate how existing code interacts with a component that needs to be modified. However, in opencode. there is currently no native way to describe this. So precious seconds wind up wasted, and costs- for users without prompt caching.

Some of the super fast, low latency models/providers also contain rate limits, such as requests per minute or tokens per second. Cerebras in particular comes to mind, with 400k TPM and 30 RPS on the 'Pro' plan. (Recently increased, but still heavily limiting). The output is instant at around 1.5k-2.2k tokens/second, but the tool call overhead completely dominates the discovery phase.

It's also worth noting, that the multiple files to read aren't always in the same folder.

[On a positive note, Cerebras' completions API started returning prompt caching directives `"prompt_tokens_details":{"cached_tokens":0}` last night- presumably, more details will be announced on Monday]
@Sewer56 commented on GitHub (Sep 6, 2025): With the advent of faster models running on beefier hardware, tool call overhead begins to dominate during codebase investigation/discovery phases- such as when using the `PLAN` agent, or creating your own plan via a `/slash` command. As prompt processing tends to run at 10s of thousands of tokens per second, even the mainstream models (e.g. Claude) can become severely slown down by unnecessarily alternating between `input` and `output` during discovery. After reading a file, the LLM will often decide it wants to read 3-4 more files, to investigate how existing code interacts with a component that needs to be modified. However, in `opencode`. there is currently no native way to describe this. So precious seconds wind up wasted, and costs- for users without prompt caching. Some of the super fast, low latency models/providers also contain rate limits, such as `requests per minute` or `tokens per second`. Cerebras in particular comes to mind, with 400k TPM and 30 RPS on the 'Pro' plan. (Recently increased, but still heavily limiting). The output is instant at around 1.5k-2.2k tokens/second, but the tool call overhead completely dominates the discovery phase. It's also worth noting, that the multiple files to read aren't always in the same folder. <details> [On a positive note, Cerebras' completions API started returning prompt caching directives `"prompt_tokens_details":{"cached_tokens":0}` last night- presumably, more details will be announced on Monday] </details>
yindo commented 2026-02-16 17:30:18 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Sep 6, 2025):

yeah this makes sense

@rekram1-node commented on GitHub (Sep 6, 2025): yeah this makes sense
yindo commented 2026-02-16 17:30:18 -05:00
Author
Owner
Copy Link
Copy Source

@Sewer56 commented on GitHub (Sep 6, 2025):

Probably worth noting.

There's 2 approaches I can think of here:

  • Making Read multi file by default
  • Making ReadFiles a separate tool.

I'm not sure which one of the two is better.
Technically the second one can already be achieved by MCP, but there's always a risk of the LLM using the single read/write tool instead of the batch one when available.

One of the tradeoffs is separate ReadFiles call would duplicate instructions in initial context. On the other hand, a Read that's multi-file by default would use some extra tokens for every read call on a single file. I imagine over the long run, they would probably even out to about the same.

In the same vein, a batch 'edit' would also be useful. Two use cases come to mind:

  1. If you ask 'refine this plan.md', and give it a list of changes, then multiple sections of it would be edited in a single call, saving time and tokens.

  2. When adding or amending multiple methods in a file, a batch edit would be able to emplace them in one call. That would be especially handy for Cerebras and Groq, Grok Fast given their inference speed.

Note we already have a 'replace all' but not 'replace multiple diffs'.

(Edit: I made a few edits, since original comment was poorly worded)

@Sewer56 commented on GitHub (Sep 6, 2025): Probably worth noting. There's 2 approaches I can think of here: - Making `Read` multi file by default - Making `ReadFiles` a separate tool. I'm not sure which one of the two is better. Technically the second one can already be achieved by MCP, but there's always a risk of the LLM using the single read/write tool instead of the batch one when available. One of the tradeoffs is separate `ReadFiles` call would duplicate instructions in initial context. On the other hand, a `Read` that's multi-file by default would use some extra tokens for every read call on a single file. I imagine over the long run, they would probably even out to about the same. -------- In the same vein, a batch 'edit' would also be useful. Two use cases come to mind: 1. If you ask 'refine this plan.md', and give it a list of changes, then multiple sections of it would be edited in a single call, saving time and tokens. 2. When adding or amending multiple methods in a file, a batch edit would be able to emplace them in one call. That would be especially handy for `Cerebras` and `Groq`, `Grok Fast` given their inference speed. Note we already have a 'replace all' but not 'replace multiple diffs'. (Edit: I made a few edits, since original comment was poorly worded)
yindo commented 2026-02-16 17:30:19 -05:00
Author
Owner
Copy Link
Copy Source

@Sewer56 commented on GitHub (Sep 11, 2025):

I left some notes about this on the OpenCode discord.
https://discord.com/channels/1391832426048651334/1394667004979445931/1415894027022106746

Quoted below:

Being a little bit excited, I kind of pulled an all nighter after work yesterday- playing around, and tried some things.

Cerebras' hosted models have a very unique problem. When you can output >1000 tokens/sec, tool calls start to become a large bottleneck-
as a result of the various overheads, from time to first token serverside, to network latency, local processing, etc.

So I was wondering if there was an 'easy' way to shave some tool calls-

Quick Experiment

I used a new (not yet documented) plugin feature to add extra system tools to opencode:

  • A MultiRead system tool (read 1/more files at once)
  • A MultiEdit system tool. (edit 1/more parts in 1/more files)
  • A MultiWrite system tool. (write multiple files at once)

Usually in agentic coding, when the user provides a prompt- the LLM will use a lot of tool calls to gather
context required to figure out how to perform the user's request.

It is often true, when observing, that if an LLM will read 1 file, it will likely want to read 2-3
more files that are referenced from the first file.
i.e. I know I need to edit this file, now I need to look at this file's dependencies sort of thing.

In the case of Cerebras, that would also help with easing the Tokens Per Minute (TPM) limitations which
unfortunately make Cerebras Code's use cases limited.

Observations

I used Cerebras Code (i.e. cerebras/qwen3-coder-480b), since this is the model that would benefit
the most from this. I also did the experiments on sonnet from Anthropic, but just for sanity tests.

Models Aren't Keen on using the Multi-Tools out of the Box

When available alongside regular Read/Write/Edit stool, the multi versions are very rarely used by LLMs.
This is true for both qwen on Cerebras and sonnet on Anthropic.

This is true even when given precise instructions and very strong directives to use them aggressively.
For example:
- You MUST use if more than 1 edit is required to a file and
- You MUST batch file edits into one and use 'multi-edit' tool

If you take away regular Read/Write/Edit tools and only give the LLM the multi versions,
most (but not all) of the time LLMs will still only read/write 1 file at a time.

Models Are however able to use the tools, if explicitly asked.

Models will use the tools well if you explicitly ask them to.
But there is a recency bias, it will only do it if reminded to fairly recently.

Some Examples are:

  • Before you begin, use "multi-read' tool to read all files present in the user's prompt.

    • You can add this in a slash command, or similar.

  • Group all needed changes by file and use the 'multi-edit' tool to implement the planned changes.

    • Useful when switching from Plan Mode to Build/Act Mode

However, further down the road, the LLMs will unfortunately forget and default to only operating on
one file at a time. For instance, if it edits multiple files- and there's more than 1 error, it will
try to solve the errors one at a time, even if trivial to fix; meaning they default back to only 1 edit
at a time.

Performance Notes

Prefetching in Plan Mode

For the PLAN mod directive Before you begin, use 'multi-read' tool to read all files present in the user's prompt:

  • ✅ Recommend for non-thinking models including cerebras/qwen3-coder-480b on Cerebras.
  • ❌ NOT RECOMMENDED for some thinking models like sonnet (thinking enabled).

For the latter case of thinking models, normally the model would read a file via tool call, comment
on what it just read in the thinking block and then comment on it in the non-thinking output.

The problem I encountered is, when the files are prefetched, the remarks/content in the <think> tags
and non-think output is duplicated. Basically, the <think> section will have the answer for the current
TODO point in the TODO tools, and then put a same or shorter version of that in the non-thinking output.

This leads to more tokens being used in the end, and more time planning, without a discernable gain.

Note: I haven't verified if this is a sonnet specific issue or also present in other thinking models.
I would guess the latter, but that's purely speculation.

Non-thinking sonnet does not suffer the regression, but then again, it doesn't benefit much time wise
when output is sub 100 tokens/sec.

🔥 Note: During one of the tests, I managed to get a working plan in my benchmark in 30 seconds (jaw dropping!).
(It usually takes sonnet 3:30 - 4 minutes) However, this relies on having a list of related files in your prompt for
MultiRead. In my case, my /prompt-refiner command outputs that when I use it to make a concrete,
well defined set of objectives from vague/incomplete information.

Grouping Edits after Plan Mode

For the directive Group all needed changes by file and use the 'multi-edit' tool to implement the planned changes, which
you would use when switching from Plan mode to Act/Edit mode.

This usually works well with limited side effects.
But, the LLM wouldn't always use it.

I would recommend it in a slash command though if you are working with Cerebras models
so you don't manually need to switch the model to qwen every time and then type out the instruction.

⚠️ Note: More complex tool calls will increase risk of failed calls.
Based on my experience so far, I'd recommend to limiting it to multiple edits per file in a single
tool call. Alternatively, edits in multiple files but all within a single 'TODO' objective.

I think it's the best tradeoff in error rate vs speed.
Also might allow it to be called more often, as more strict/explicit directives could be used.
Failed tool calls persist in the context, so you don't want to fire too huge diffs in case they fail to apply;
as you know, sometimes models fail even single edits.

📔 Note 2: I would consider this worthwhile only for Cerebras. It's a speed/error tradeoff in cases where
tool call overhead dominates. I'd probably consider something like 500 tokens/second the baseline at which
point you'd want to multiedit for considerable speed gain.

✅ Note 3: My PLAN tool output includes a quick summary of new/modified APIs for each file. That likely helps
the LLM group together changes it needs into multiedits.

Conclusion

General Conclusion

Though I found 2 good use cases above, the gains overall aren't that great actually.
(It may be surprising to hear that only now.)

I will be using the 2 tricks above with Cerebras models from now on.
In practice, they probably save ~15s in an otherwise 2 minute session (assuming no throttling);
and help to avoid, well, TPM throttling. However, at the end of the day;
you're still going to have a lot of tool calls:

  • Updating the TODO list.
  • Search/Grep & File Listing.
  • Single Edits to Fix Errors.
  • Everything else.

A Hypothesis

As for the LLMs not being able to efficiently use the multi read/write/edit tools without explicit
user prompting; I have a hypothesis.

From my observation; the models generally try to do the following:

  • Break up complex problems into smaller objectives
  • Try to solve each objective one at a time

(The TODO tools which they are trained to use also help them reinforce this.)

The caveat is, solving different steps of a bigger problem will often require edits to the same files-
sometimes at a very granular level, e.g. Add Periodic Refresh, Add Public API for Explicit Refresh.

Because the LLM is trained to sequentially focus at one task at a time, it will often miss
opportunities to solve 2 tasks at once, which is now possible with MultiEdit tool and the like.

It clearly can do it, if explicitly prompted to, but would not default to it- even if explicitly
instructed by system prompt.

@Sewer56 commented on GitHub (Sep 11, 2025): I left some notes about this on the `OpenCode` discord. https://discord.com/channels/1391832426048651334/1394667004979445931/1415894027022106746 Quoted below: -------- Being a little bit excited, I kind of pulled an all nighter after work yesterday- playing around, and tried some things. Cerebras' hosted models have a very unique problem. When you can output >1000 tokens/sec, tool calls start to become a large bottleneck- as a result of the various overheads, from time to first token serverside, to network latency, local processing, etc. So I was wondering if there was an 'easy' way to shave some tool calls- ## Quick Experiment I used a new (not yet documented) [plugin feature to add extra system tools](<https://github.com/sst/opencode/pull/2420>) to `opencode`: - A `MultiRead` system tool (read 1/more files at once) - A `MultiEdit` system tool. (edit 1/more parts in 1/more files) - A `MultiWrite` system tool. (write multiple files at once) Usually in agentic coding, when the user provides a prompt- the LLM will use a lot of tool calls to gather context required to figure out how to perform the user's request. It is often true, when observing, that if an LLM will read 1 file, it will likely want to read 2-3 more files that are referenced from the first file. i.e. `I know I need to edit this file, now I need to look at this file's dependencies` sort of thing. In the case of Cerebras, that would also help with easing the Tokens Per Minute (TPM) limitations which unfortunately make Cerebras Code's use cases limited. ## Observations I used Cerebras Code (i.e. `cerebras/qwen3-coder-480b`), since this is the model that would benefit the most from this. I also did the experiments on `sonnet` from `Anthropic`, but just for sanity tests. ### Models Aren't Keen on using the Multi-Tools out of the Box When available alongside regular `Read`/`Write`/`Edit` stool, the multi versions are very rarely used by LLMs. This is true for both `qwen` on Cerebras and `sonnet` on Anthropic. This is true even when given precise instructions and very strong directives to use them aggressively. For example: - `You MUST use if more than 1 edit is required to a file` and - `You MUST batch file edits into one and use 'multi-edit' tool` If you take away regular `Read`/`Write`/`Edit` tools and only give the LLM the multi versions, most (but not all) of the time LLMs will still only read/write 1 file at a time. ### Models Are however able to use the tools, if explicitly asked. Models will use the tools well ***if you explicitly ask them to***. But there is a *recency bias*, it will only do it if reminded to fairly recently. Some Examples are: - `Before you begin, use "multi-read' tool to read all files present in the user's prompt`. - You can add this in a slash command, or similar. - `Group all needed changes by file and use the 'multi-edit' tool to implement the planned changes`. - Useful when switching from `Plan` Mode to `Build/Act` Mode However, further down the road, the LLMs will unfortunately forget and default to only operating on one file at a time. For instance, if it edits multiple files- and there's more than 1 error, it will try to solve the errors one at a time, even if trivial to fix; meaning they default back to only 1 edit at a time. ## Performance Notes ### Prefetching in Plan Mode For the PLAN mod directive `Before you begin, use 'multi-read' tool to read all files present in the user's prompt`: - :white_check_mark: Recommend for non-thinking models including `cerebras/qwen3-coder-480b` on Cerebras. - :x: ***NOT RECOMMENDED*** for some thinking models like `sonnet` (thinking enabled). For the latter case of thinking models, normally the model would read a file via tool call, comment on what it just read in the thinking block and then comment on it in the non-thinking output. The problem I encountered is, when the files are prefetched, the remarks/content in the `<think>` tags and non-think output is duplicated. Basically, the `<think>` section will have the answer for the current TODO point in the TODO tools, and then put a same or shorter version of that in the non-thinking output. This leads to more tokens being used in the end, and more time planning, without a discernable gain. Note: I haven't verified if this is a `sonnet` specific issue or also present in other thinking models. I would guess the latter, but that's purely speculation. Non-thinking `sonnet` does not suffer the regression, but then again, it doesn't benefit much time wise when output is sub 100 tokens/sec. :fire: Note: During one of the tests, I managed to get a working plan in my benchmark in 30 seconds (*jaw dropping!*). (It usually takes `sonnet` 3:30 - 4 minutes) However, this relies on having a list of related files in your prompt for `MultiRead`. In my case, my `/prompt-refiner` command outputs that when I use it to make a concrete, well defined set of objectives from vague/incomplete information. ### Grouping Edits after Plan Mode For the directive `Group all needed changes by file and use the 'multi-edit' tool to implement the planned changes`, which you would use when switching from `Plan` mode to `Act/Edit` mode. ***This usually works well*** with limited side effects. But, the LLM wouldn't always use it. ***I would recommend it*** in a slash command though ***if you are working with Cerebras models*** so you don't manually need to switch the model to qwen every time and then type out the instruction. :warning: Note: More complex tool calls will increase risk of failed calls. Based on my experience so far, I'd recommend to limiting it to multiple edits per file in a single tool call. Alternatively, edits in multiple files but all within a single 'TODO' objective. I think it's the best tradeoff in error rate vs speed. Also might allow it to be called more often, as more strict/explicit directives could be used. Failed tool calls persist in the context, so you don't want to fire too huge diffs in case they fail to apply; as you know, sometimes models fail even single edits. :notebook_with_decorative_cover: Note 2: I would consider this worthwhile only for Cerebras. It's a speed/error tradeoff in cases where tool call overhead dominates. I'd probably consider something like 500 tokens/second the baseline at which point you'd want to multiedit for considerable speed gain. :white_check_mark: Note 3: My `PLAN` tool output includes a quick summary of new/modified APIs for each file. That likely helps the LLM group together changes it needs into multiedits. ## Conclusion ### General Conclusion Though I found 2 good use cases above, the gains overall ***aren't that great actually.*** (It may be surprising to hear that only now.) I will be using the 2 tricks above with Cerebras models from now on. In practice, they probably save ~15s in an otherwise 2 minute session (assuming no throttling); and help to avoid, well, TPM throttling. However, at the end of the day; you're still going to have a lot of tool calls: - Updating the TODO list. - Search/Grep & File Listing. - Single Edits to Fix Errors. - Everything else. ### A Hypothesis As for the LLMs not being able to efficiently use the multi read/write/edit tools without explicit user prompting; I have a hypothesis. From my observation; the models generally try to do the following: - Break up complex problems into smaller objectives - Try to solve each objective one at a time (The TODO tools which they are trained to use also help them reinforce this.) The caveat is, ***solving different steps of a bigger problem*** will often require edits to the same files- sometimes at a very granular level, e.g. `Add Periodic Refresh`, `Add Public API for Explicit Refresh`. Because the LLM is trained to sequentially focus at one task at a time, it will often miss opportunities to solve 2 tasks at once, which is now possible with `MultiEdit` tool and the like. It clearly can do it, *if explicitly prompted to*, but would not default to it- even if explicitly instructed by system prompt.
yindo commented 2026-02-16 17:30:19 -05:00
Author
Owner
Copy Link
Copy Source

@Sewer56 commented on GitHub (Sep 11, 2025):

Anyway if anyone wants to faff around, here's the system tools: https://github.com/Sewer56/nixos-setup/tree/d6e6f566270fb6b0ec4e67ba618937e57a59104e/users/sewer/home-manager/programs/opencode/config/plugin , which use the newly added (not yet documented) plugin system from like 3 days ago.

@Sewer56 commented on GitHub (Sep 11, 2025): Anyway if anyone wants to faff around, here's the system tools: https://github.com/Sewer56/nixos-setup/tree/d6e6f566270fb6b0ec4e67ba618937e57a59104e/users/sewer/home-manager/programs/opencode/config/plugin , which use the newly added (not yet documented) plugin system from like 3 days ago.
yindo commented 2026-02-16 17:30:19 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Dec 27, 2025):

[automated] Closing due to 90+ days of inactivity. Feel free to reopen if you still need this!

@rekram1-node commented on GitHub (Dec 27, 2025): [automated] Closing due to 90+ days of inactivity. Feel free to reopen if you still need this!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#1285
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?