Make the directory listing limit configurable in the system prompt #1682

Open
opened 2026-02-16 17:32:08 -05:00 by yindo · 6 comments
Owner

Originally created by @Cluster444 on GitHub (Sep 10, 2025).

Originally assigned to: @Cluster444 on GitHub.

Right now the files pushed into the system prompt as output from Ripgrep.tree are pretty much useless on a large enough project. The breadth-first nature of it makes it prioritize shallow files, and most useful files are more deeply nested in a lot of projects. For example here's some stats from my rails app

git ls-files - 1168
git ls-files app - 667
git ls-files app/models app/controllers - 228 (100% of this gets ignored currently)

And on top of that, the output is kind of nonsensical, i'm not sure the model is going to make heads of tails of this. For example this is what my /app directory ends up looking like in the prompt

app/
assets/
builds/
[1 truncated]
[2 truncated]
channels/
application_cable/
[2 truncated]
[7 truncated]
constants/
date_formats.rb
controllers/
baskets/
[3 truncated]
[41 truncated]
errors/
user_facing_error.rb
helpers/
content_filters/
[1 truncated]
[37 truncated]
javascript/
actions/
[1 truncated]
[6 truncated]
jobs/
chat/
[1 truncated]
[4 truncated]
lib/
assertion.rb
mailers/
application_mailer.rb
[2 truncated]
models/
[64 truncated]
views/
[32 truncated]

There's at least two config options that would help improve this:

  1. Make the limit configurable
  2. Allow for directory scopes to be applied

And some kind of better formatting that gives a relationship between directories and files as this is all disjointed. Maybe something like

# app/controllers
- this_controller.rb
- that_controller.rb
# app/models
- this.rb
- that.rb
# app/models/this
- taggable.rb
- joinable.rb

this would keep the paths from being repeated, but also give them some element of emphasis that would actually match things like error output.

Originally created by @Cluster444 on GitHub (Sep 10, 2025). Originally assigned to: @Cluster444 on GitHub. Right now the files pushed into the system prompt as output from Ripgrep.tree are pretty much useless on a large enough project. The breadth-first nature of it makes it prioritize shallow files, and most useful files are more deeply nested in a lot of projects. For example here's some stats from my rails app ``` git ls-files - 1168 git ls-files app - 667 git ls-files app/models app/controllers - 228 (100% of this gets ignored currently) ``` And on top of that, the output is kind of nonsensical, i'm not sure the model is going to make heads of tails of this. For example this is what my `/app` directory ends up looking like in the prompt ``` app/ assets/ builds/ [1 truncated] [2 truncated] channels/ application_cable/ [2 truncated] [7 truncated] constants/ date_formats.rb controllers/ baskets/ [3 truncated] [41 truncated] errors/ user_facing_error.rb helpers/ content_filters/ [1 truncated] [37 truncated] javascript/ actions/ [1 truncated] [6 truncated] jobs/ chat/ [1 truncated] [4 truncated] lib/ assertion.rb mailers/ application_mailer.rb [2 truncated] models/ [64 truncated] views/ [32 truncated] ``` There's at least two config options that would help improve this: 1. Make the limit configurable 2. Allow for directory scopes to be applied And some kind of better formatting that gives a relationship between directories and files as this is all disjointed. Maybe something like ``` # app/controllers - this_controller.rb - that_controller.rb # app/models - this.rb - that.rb # app/models/this - taggable.rb - joinable.rb ``` this would keep the paths from being repeated, but also give them some element of emphasis that would actually match things like error output.
Author
Owner

@github-actions[bot] commented on GitHub (Sep 10, 2025):

This issue might be a duplicate of existing issues. Please check:

  • #1868: Request to make Bash line output configurable - similar request for making output limits configurable
  • #2375: Glob/grep Tool Max Lines (Limit output of all tools?) - discusses limiting tool outputs to prevent context window issues
  • #1028: Improve AGENTS.md structure for task-specific instructions and reduced context crowding - addresses context crowding issues with large projects
  • #924: Dramatic increase in token consumption - related to context/token consumption problems with large projects

Feel free to ignore if none of these address your specific case.

@github-actions[bot] commented on GitHub (Sep 10, 2025): This issue might be a duplicate of existing issues. Please check: - #1868: Request to make Bash line output configurable - similar request for making output limits configurable - #2375: Glob/grep Tool Max Lines (Limit output of all tools?) - discusses limiting tool outputs to prevent context window issues - #1028: Improve AGENTS.md structure for task-specific instructions and reduced context crowding - addresses context crowding issues with large projects - #924: Dramatic increase in token consumption - related to context/token consumption problems with large projects Feel free to ignore if none of these address your specific case.
Author
Owner

@rekram1-node commented on GitHub (Sep 10, 2025):

just going to assign it to you since I saw the exchange in discord

@rekram1-node commented on GitHub (Sep 10, 2025): just going to assign it to you since I saw the exchange in discord
Author
Owner

@taqtiqa-mark commented on GitHub (Oct 14, 2025):

@Cluster444 IIUC, this will be affected by allowing users more control of the context via /add and /drop, see #1990. Is that correct ?
There is also the question of whether the raw files are added to the context or a tree-sitter extract?

@taqtiqa-mark commented on GitHub (Oct 14, 2025): @Cluster444 IIUC, this will be affected by allowing users more control of the context via `/add` and `/drop`, see #1990. Is that correct ? There is also the question of whether the raw files are added to the context or a tree-sitter extract?
Author
Owner

@Cluster444 commented on GitHub (Oct 14, 2025):

No I dont think that would affect what I was getting at here. This is a builtin that is dumping a directory tree directly into system instructions, and on even small projects it renders it quite useless because it saturates the 200 file limit with all the 1st and 2nd tier files, where most source code is 3-5 dirs deep in many projects, so they never show up.

@Cluster444 commented on GitHub (Oct 14, 2025): No I dont think that would affect what I was getting at here. This is a builtin that is dumping a directory tree directly into system instructions, and on even small projects it renders it quite useless because it saturates the 200 file limit with all the 1st and 2nd tier files, where most source code is 3-5 dirs deep in many projects, so they never show up.
Author
Owner

@taqtiqa-mark commented on GitHub (Oct 14, 2025):

This is a builtin that is dumping a directory tree directly into system instructions...

Yes, that destination is just plain, flat-out wrong.

... where most source code is 3-5 dirs deep in many projects, so they never show up.

Correct so a agent or command /add folder/needed/one/*.ts and /add folder/two/*.html, where the tree-sitter data from those folders is added to the Context-Text, and not the User-Prompt-text, and definitely not the System-Prompt-Text.

That would be a sane approach.

@taqtiqa-mark commented on GitHub (Oct 14, 2025): > This is a builtin that is dumping a directory tree directly into system instructions... Yes, that destination is just plain, flat-out wrong. > ... where most source code is 3-5 dirs deep in many projects, so they never show up. Correct so a agent or command `/add folder/needed/one/*.ts` and `/add folder/two/*.html`, where the tree-sitter data from those folders is added to the Context-Text, and not the User-Prompt-text, and **definitely not** the System-Prompt-Text. That would be a sane approach.
Author
Owner

@taqtiqa-mark commented on GitHub (Oct 15, 2025):

Substantiating this claim:

Yes, that destination is just plain, flat-out wrong.

See: https://github.com/sst/opencode/issues/2108#issuecomment-3403600775

@taqtiqa-mark commented on GitHub (Oct 15, 2025): Substantiating this claim: > Yes, that destination is just plain, flat-out wrong. See: https://github.com/sst/opencode/issues/2108#issuecomment-3403600775
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#1682