[GH-ISSUE #5292] Few anything llm issues and suggestions #5002

Closed
opened 2026-06-05 14:51:24 -04:00 by yindo · 4 comments
Owner

Originally created by @CreativeCoder0 on GitHub (Mar 28, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5292

I am using anythingllm desktop version in linux btw. more specifically fedora (with gnome desktop environment).

  1. can create a folder after clicking on upload icon on the right side of workspace name. but can't move existing files to those folders.
  2. even you can't directly upload to the new folder u just created.
  3. to improve performance i made a custom model (i don't know what it is exactly called, so i am calling it custom model). basically i created a Modelfile and used parameter num_ctx to lower the context size. when i use this custom model, it is still using 16k context unless u go to settings and there u specify the context default size, but by doing so it is the new default for all chats. we should have a input box or slider to change context size in the chat interface, so that we can change it in every thread in every workspace, as we can choose different model in every thread in every workspace. also add option to enable/disable kv cache.
  4. either publish a flatpak version or give us option to scale the ui. the current situation is u can't use it comfortably unless u choose "large text" at the top right corner in the chat interface. but this is not large, this is the normal, so rename it to "Normal". for people with visibility challenge, u should give more larger options or slider to scale (in the interface or may be in the settings).
    one thing, i am not sure, but wayland may be one reason for the first two issues i mentioned. may be flatpak solves this.
  5. i mentioned one problem earlier, that it requests google even if i have choose duckduckgo for web search. u replied that i should try turning of web scraping and use just web search. first of all, this should not be like that. but i thought may be upcoming updates will fix it, for now i can do as u said. so i tried but the problem still exists.
  6. this is not an issue(probably), just curious. i see requests to "cdn.anythingllm.com" and "app.posthog.com" each time u open this app. why these requests are made? or i configured something wrong in settings?
  7. for users like me who don't have big gpus, can u please somehow integrate it with ide like via some extension or some other way? currently anything llm can generate the code as u instructed, but because we don't have expensive hardware, we can't expect that it will generate the whole thing properly and that too with few documentation rag. currently i use continue.dev extension, but i would like to integrate anythingllm as i get all the features here at one place like- web scraping docs, rag, choosing model with context size etc. It may have many use cases, for example -
    suppose new version of next.js released. i need to make a website. i will say scrape next.js latest version docs. then i explain this is my project idea, so extract which concepts are useful from docs and summarise them so that we can rag it further to use with the model used with coding agent or with autocompletion model. we can continue the chat exactly where we left in anythingllm interface, we can run it in the ide, we can manually change in code in the ide and these changes are noticed by the llm. so it will give me code according to those changes. in this way, we will be use optimize local ai to increase the overall productivity without expensive hardware. plus turboquant is on the way, so we will be able to use bigger models. so even with 4-8gb of gpu we can do pretty exciting things.
Originally created by @CreativeCoder0 on GitHub (Mar 28, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5292 I am using anythingllm desktop version in linux btw. more specifically fedora (with gnome desktop environment). 1. can create a folder after clicking on upload icon on the right side of workspace name. but can't move existing files to those folders. 2. even you can't directly upload to the new folder u just created. 3. to improve performance i made a custom model (i don't know what it is exactly called, so i am calling it custom model). basically i created a Modelfile and used parameter num_ctx to lower the context size. when i use this custom model, it is still using 16k context unless u go to settings and there u specify the context default size, but by doing so it is the new default for all chats. we should have a input box or slider to change context size in the chat interface, so that we can change it in every thread in every workspace, as we can choose different model in every thread in every workspace. also add option to enable/disable kv cache. 4. either publish a flatpak version or give us option to scale the ui. the current situation is u can't use it comfortably unless u choose "large text" at the top right corner in the chat interface. but this is not large, this is the normal, so rename it to "Normal". for people with visibility challenge, u should give more larger options or slider to scale (in the interface or may be in the settings). one thing, i am not sure, but wayland may be one reason for the first two issues i mentioned. may be flatpak solves this. 5. i mentioned one problem earlier, that it requests google even if i have choose duckduckgo for web search. u replied that i should try turning of web scraping and use just web search. first of all, this should not be like that. but i thought may be upcoming updates will fix it, for now i can do as u said. so i tried but the problem still exists. 6. this is not an issue(probably), just curious. i see requests to "cdn.anythingllm.com" and "app.posthog.com" each time u open this app. why these requests are made? or i configured something wrong in settings? 7. for users like me who don't have big gpus, can u please somehow integrate it with ide like via some extension or some other way? currently anything llm can generate the code as u instructed, but because we don't have expensive hardware, we can't expect that it will generate the whole thing properly and that too with few documentation rag. currently i use continue.dev extension, but i would like to integrate anythingllm as i get all the features here at one place like- web scraping docs, rag, choosing model with context size etc. It may have many use cases, for example - suppose new version of next.js released. i need to make a website. i will say scrape next.js latest version docs. then i explain this is my project idea, so extract which concepts are useful from docs and summarise them so that we can rag it further to use with the model used with coding agent or with autocompletion model. we can continue the chat exactly where we left in anythingllm interface, we can run it in the ide, we can manually change in code in the ide and these changes are noticed by the llm. so it will give me code according to those changes. in this way, we will be use optimize local ai to increase the overall productivity without expensive hardware. plus turboquant is on the way, so we will be able to use bigger models. so even with 4-8gb of gpu we can do pretty exciting things.
yindo closed this issue 2026-06-05 14:51:24 -04:00
Author
Owner

@bulrush15 commented on GitHub (Mar 28, 2026):

for users like me who don't have big gpus, can u please somehow integrate it with ide like via some extension or some other way?

As a developer I use Visual Studio Code to interact with Gemini AI via an extension. There is possibly an extension to use AnythingLLM directly inside VSC as well. Go to the VSC extension marketplace to see what they have. https://code.visualstudio.com/docs/configure/extensions/extension-marketplace

I can also use VSC to interact with Ollama but I haven't tried it. But Ollama will support many more AI models. Just run Ollama as a server.

<!-- gh-comment-id:4148241140 --> @bulrush15 commented on GitHub (Mar 28, 2026): > for users like me who don't have big gpus, can u please somehow integrate it with ide like via some extension or some other way? As a developer I use Visual Studio Code to interact with Gemini AI via an extension. There is possibly an extension to use AnythingLLM directly inside VSC as well. Go to the VSC extension marketplace to see what they have. https://code.visualstudio.com/docs/configure/extensions/extension-marketplace I can also use VSC to interact with Ollama but I haven't tried it. But Ollama will support many more AI models. Just run Ollama as a server.
Author
Owner

@timothycarambat commented on GitHub (Mar 28, 2026):

can create a folder after clicking on upload icon on the right side of workspace name. but can't move existing files to those folders.

Am I misunderstanding the bug here? Let me know or send a screenshot so I can see what is going on and we can address it

https://github.com/user-attachments/assets/14f63793-c72d-40be-a089-74b14e327772

even you can't directly upload to the new folder u just created.

This is true, and annoying, mostly because the uploader is the box below the window so it is hard to know what a "target" folder would be during upload. I am open to any feedback around this - maybe folders can be targets?

to improve performance i made a custom model (i don't know what it is exactly called, so i am calling it custom model). basically i created a Modelfile and used parameter num_ctx to lower the context size. when i use this custom model, it is still using 16k context unless u go to settings and there u specify the context default size, but by doing so it is the new default for all chats. we should have a input box or slider to change context size in the chat interface, so that we can change it in every thread in every workspace, as we can choose different model in every thread in every workspace. also add option to enable/disable kv cache.

👍

either publish a flatpak version or give us option to scale the ui. the current situation is u can't use it comfortably unless u choose "large text" at the top right corner in the chat interface. but this is not large, this is the normal, so rename it to "Normal". for people with visibility challenge, u should give more larger options or slider to scale (in the interface or may be in the settings).
one thing, i am not sure, but wayland may be one reason for the first two issues i mentioned. may be flatpak solves this.

related #2823.

mentioned one problem earlier, that it requests google even if i have choose duckduckgo for web search. u replied that i should try turning of web scraping and use just web search. first of all, this should not be like that. but i thought may be upcoming updates will fix it, for now i can do as u said. so i tried but the problem still exists.

In issue #5265 unless I am misunderstanding the issue at hand it was the LLM attempting to call the web-scrape tool as opposed to web-search - which is what you actually want. AnythingLLM just passes tools to the model, if the model decides trying to scrape google is a better option than searching then that is a model issue and not a bug we can really fix. That being said a screenshot/video of the issue would make me understand if that is 100% the case or I am just reading the issue poorly.

this is not an issue(probably), just curious. i see requests to "cdn.anythingllm.com" and "app.posthog.com" each time u open this app. why these requests are made? or i configured something wrong in settings?

So we have an announcements JSON file and also the version check txt file on our CDN so there is a central place to version check the software. We also mirror some critical assets like the embedder model so people blocked by HuggingFace (China, etc) can still use the app.

For PostHog, the only reason you would see that is Telemetry, which you can turn off! In the UI go to Settings > Privacy & Data and there is a toggle for that. It is basic events - no IP, data, or identifying information at all

for users like me who don't have big gpus, can u please somehow integrate it with ide like via some extension or some other way? currently anything llm can generate the code as u instructed, but because we don't have expensive hardware, we can't expect that it will generate the whole thing properly and that too with few documentation rag. currently i use continue.dev extension, but i would like to integrate anythingllm as i get all the features here at one place like- web scraping docs, rag, choosing model with context size etc. It may have many use cases, for example -
suppose new version of next.js released. i need to make a website. i will say scrape next.js latest version docs. then i explain this is my project idea, so extract which concepts are useful from docs and summarise them so that we can rag it further to use with the model used with coding agent or with autocompletion model. we can continue the chat exactly where we left in anythingllm interface, we can run it in the ide, we can manually change in code in the ide and these changes are noticed by the llm. so it will give me code according to those changes. in this way, we will be use optimize local ai to increase the overall productivity without expensive hardware. plus turboquant is on the way, so we will be able to use bigger models. so even with 4-8gb of gpu we can do pretty exciting things.

I totally get this idea. Our hesitation has always been that coding is a deep rabbit hole with very specific tooling needs that diverge pretty quickly from our core focus. That said, I don't want to dismiss this entirely - if there's a lighter-touch way to bridge AnythingLLM and an IDE (e.g. via our API rather than a full extension), that feels more possible.

I'd like to understand the minimum viable integration that would actually be useful for you because we do have the developer API and I imagine this would just be tacking onto that.

<!-- gh-comment-id:4149204727 --> @timothycarambat commented on GitHub (Mar 28, 2026): > can create a folder after clicking on upload icon on the right side of workspace name. but can't move existing files to those folders. Am I misunderstanding the bug here? Let me know or send a screenshot so I can see what is going on and we can address it https://github.com/user-attachments/assets/14f63793-c72d-40be-a089-74b14e327772 > even you can't directly upload to the new folder u just created. This is true, and annoying, mostly because the uploader is the box below the window so it is hard to know what a "target" folder would be during upload. I am open to any feedback around this - maybe folders can be targets? > to improve performance i made a custom model (i don't know what it is exactly called, so i am calling it custom model). basically i created a Modelfile and used parameter num_ctx to lower the context size. when i use this custom model, it is still using 16k context unless u go to settings and there u specify the context default size, but by doing so it is the new default for all chats. we should have a input box or slider to change context size in the chat interface, so that we can change it in every thread in every workspace, as we can choose different model in every thread in every workspace. also add option to enable/disable kv cache. 👍 > either publish a flatpak version or give us option to scale the ui. the current situation is u can't use it comfortably unless u choose "large text" at the top right corner in the chat interface. but this is not large, this is the normal, so rename it to "Normal". for people with visibility challenge, u should give more larger options or slider to scale (in the interface or may be in the settings). one thing, i am not sure, but wayland may be one reason for the first two issues i mentioned. may be flatpak solves this. related #2823. > mentioned one problem earlier, that it requests google even if i have choose duckduckgo for web search. u replied that i should try turning of web scraping and use just web search. first of all, this should not be like that. but i thought may be upcoming updates will fix it, for now i can do as u said. so i tried but the problem still exists. In issue #5265 unless I am misunderstanding the issue at hand it was the LLM attempting to call the `web-scrape` tool as opposed to `web-search` - which is what you actually want. AnythingLLM just passes tools to the model, if the model decides trying to scrape google is a better option than searching then that is a model issue and not a bug we can really fix. That being said a screenshot/video of the issue would make me understand if that is 100% the case or I am just reading the issue poorly. > this is not an issue(probably), just curious. i see requests to "cdn.anythingllm.com" and "app.posthog.com" each time u open this app. why these requests are made? or i configured something wrong in settings? So we have an announcements JSON file and also the version check txt file on our CDN so there is a central place to version check the software. We also mirror some critical assets like the embedder model so people blocked by HuggingFace (China, etc) can still use the app. For PostHog, the only reason you would see that is [Telemetry](https://github.com/Mintplex-Labs/anything-llm?tab=readme-ov-file#opting-out), which you can turn off! In the UI go to Settings > Privacy & Data and there is a toggle for that. It is basic events - no IP, data, or identifying information at all > for users like me who don't have big gpus, can u please somehow integrate it with ide like via some extension or some other way? currently anything llm can generate the code as u instructed, but because we don't have expensive hardware, we can't expect that it will generate the whole thing properly and that too with few documentation rag. currently i use continue.dev extension, but i would like to integrate anythingllm as i get all the features here at one place like- web scraping docs, rag, choosing model with context size etc. It may have many use cases, for example - suppose new version of next.js released. i need to make a website. i will say scrape next.js latest version docs. then i explain this is my project idea, so extract which concepts are useful from docs and summarise them so that we can rag it further to use with the model used with coding agent or with autocompletion model. we can continue the chat exactly where we left in anythingllm interface, we can run it in the ide, we can manually change in code in the ide and these changes are noticed by the llm. so it will give me code according to those changes. in this way, we will be use optimize local ai to increase the overall productivity without expensive hardware. plus turboquant is on the way, so we will be able to use bigger models. so even with 4-8gb of gpu we can do pretty exciting things. I totally get this idea. Our hesitation has always been that coding is a deep rabbit hole with very specific tooling needs that diverge pretty quickly from our core focus. That said, I don't want to dismiss this entirely - if there's a lighter-touch way to bridge AnythingLLM and an IDE (e.g. via our API rather than a full extension), that feels more possible. I'd like to understand the minimum viable integration that would actually be useful for you because we do have the developer API and I imagine this would just be tacking onto that.
Author
Owner

@CreativeCoder0 commented on GitHub (Mar 29, 2026):

Am I misunderstanding the bug here? Let me know or send a screenshot so I can see what is going on and we can address it

no, actually i could not understand the ui. after selecting files if you want to move to a new folder, u have to click on the icon between "move to workspace" and "delete" buttons. i tried to move them by dragging 🤣, please bring tutorial videos soon.

This is true, and annoying, mostly because the uploader is the box below the window so it is hard to know what a "target" folder would be during upload. I am open to any feedback around this - maybe folders can be targets?

may be we can do something like, if the user selects a folder then drops file/folders, those things should go to that folder. if user doesn't select any folder, files can go in the root folder or "custom document" folder like now. probably this name can be changed to something like "others" or "unorganised" folder, i am not sure though.

or may be we can take a completely different approach. once the files/folders are dropped or selected, anythingllm should raise a pop up saying "where u want to save these files?" and give a dropdown of available folders with a "create new folder" button.

In issue https://github.com/Mintplex-Labs/anything-llm/issues/5265 unless I am misunderstanding the issue at hand it was the LLM attempting to call the web-scrape tool as opposed to web-search - which is what you actually want. AnythingLLM just passes tools to the model, if the model decides trying to scrape google is a better option than searching then that is a model issue and not a bug we can really fix. That being said a screenshot/video of the issue would make me understand if that is 100% the case or I am just reading the issue poorly.

i clearly asked the chat to search what is date today on internet. but i will still try again with that model + some other model, and then inform you the result. also i will attach some screenshot or recordings if needed as u suggested.

For PostHog, the only reason you would see that is Telemetry, which you can turn off! In the UI go to Settings > Privacy & Data and there is a toggle for that. It is basic events - no IP, data, or identifying information at all

i found the toggle button. actually i didn't click on privacy as it looked like a link. may be you should make it look like other settings. if it was intentional, you want people don't disable telemetry so that you can improve anythingllm, that is also ok, everybody will not come to github to give feedback, i can understand. btw you wrote those lines that you don't collect any chat or any other personal things, u just want to improve anythingllm - those sentences were very humble. i don't think anybody will disable if they read those lines (except who is absolutely obsessed with privacy). may be you should add a feedback page in settings. this way you will get the feedback from many people even from them who hesitate to keep telemetry enabled but want to help the project. may be in the welcome pop ups, you can include this issue that we need to improve it so don't disable it, but even if you do, there is a feedback page in settings, help us sending feedback to help this project.

I totally get this idea. Our hesitation has always been that coding is a deep rabbit hole with very specific tooling needs that diverge pretty quickly from our core focus. That said, I don't want to dismiss this entirely - if there's a lighter-touch way to bridge AnythingLLM and an IDE (e.g. via our API rather than a full extension), that feels more possible.

i understand, i will suggest leave it for now. if some idea comes to my mind i will inform you. no need to distract, this project is a blessing for common people, it must keep improving and for that all resources including efforts should be focused. otherwise i know open source projects, they got request and complaint both for a basic feature 4 years ago but still couldn't do it, every developer is working on different things, no project roadmap, no priorities...nothing.

Btw i have a new question. suppose there is an website i want to web scrape. assume it has 800 pages, each page has 10 articles listed. u can click on any link and read the article. i want to just save them as markdown, or may be i want to save them as html, later i will make them markdown so that i can read them in obsidian. so if i create an agent flow, and use the block called "write file", will anythingllm prompt me to ask where i want to save files 800*10 = 8000 times? or u can send the folder path where you want to save in prompt.

please bring the tutorials, there are a lot of confusions. as u are making this tool for everybody, u can't cover every use cases for everybody, but you can cover each tools. for example, if your last video on agent flow include the block "write file" i would not face this confusion today, but in that video you just explained only "api call" and "web scraping" blocks. i could try myself to discover how these things behave, but then i could not think how will i call 800 pages, as your video shows request to one link. if i make an url pattern of the website using variables and let the model handel the pagination, i fear if it behave weird. as i said, having a medium level pc, i am using low parameter model, so it can take wrong decisions. also my heath is not good, otherwise i could still try. but even if i tried and i was successful, a common person who doesn't have any coding knowledge can't do this.

<!-- gh-comment-id:4150427678 --> @CreativeCoder0 commented on GitHub (Mar 29, 2026): > Am I misunderstanding the bug here? Let me know or send a screenshot so I can see what is going on and we can address it no, actually i could not understand the ui. after selecting files if you want to move to a new folder, u have to click on the icon between "move to workspace" and "delete" buttons. i tried to move them by dragging 🤣, please bring tutorial videos soon. > This is true, and annoying, mostly because the uploader is the box below the window so it is hard to know what a "target" folder would be during upload. I am open to any feedback around this - maybe folders can be targets? may be we can do something like, if the user selects a folder then drops file/folders, those things should go to that folder. if user doesn't select any folder, files can go in the root folder or "custom document" folder like now. probably this name can be changed to something like "others" or "unorganised" folder, i am not sure though. or may be we can take a completely different approach. once the files/folders are dropped or selected, anythingllm should raise a pop up saying "where u want to save these files?" and give a dropdown of available folders with a "create new folder" button. > In issue https://github.com/Mintplex-Labs/anything-llm/issues/5265 unless I am misunderstanding the issue at hand it was the LLM attempting to call the web-scrape tool as opposed to web-search - which is what you actually want. AnythingLLM just passes tools to the model, if the model decides trying to scrape google is a better option than searching then that is a model issue and not a bug we can really fix. That being said a screenshot/video of the issue would make me understand if that is 100% the case or I am just reading the issue poorly. i clearly asked the chat to search what is date today on internet. but i will still try again with that model + some other model, and then inform you the result. also i will attach some screenshot or recordings if needed as u suggested. > For PostHog, the only reason you would see that is [Telemetry](https://github.com/Mintplex-Labs/anything-llm?tab=readme-ov-file#opting-out), which you can turn off! In the UI go to Settings > Privacy & Data and there is a toggle for that. It is basic events - no IP, data, or identifying information at all i found the toggle button. actually i didn't click on privacy as it looked like a link. may be you should make it look like other settings. if it was intentional, you want people don't disable telemetry so that you can improve anythingllm, that is also ok, everybody will not come to github to give feedback, i can understand. btw you wrote those lines that you don't collect any chat or any other personal things, u just want to improve anythingllm - those sentences were very humble. i don't think anybody will disable if they read those lines (except who is absolutely obsessed with privacy). may be you should add a feedback page in settings. this way you will get the feedback from many people even from them who hesitate to keep telemetry enabled but want to help the project. may be in the welcome pop ups, you can include this issue that we need to improve it so don't disable it, but even if you do, there is a feedback page in settings, help us sending feedback to help this project. > I totally get this idea. Our hesitation has always been that coding is a deep rabbit hole with very specific tooling needs that diverge pretty quickly from our core focus. That said, I don't want to dismiss this entirely - if there's a lighter-touch way to bridge AnythingLLM and an IDE (e.g. via our API rather than a full extension), that feels more possible. i understand, i will suggest leave it for now. if some idea comes to my mind i will inform you. no need to distract, this project is a blessing for common people, it must keep improving and for that all resources including efforts should be focused. otherwise i know open source projects, they got request and complaint both for a basic feature 4 years ago but still couldn't do it, every developer is working on different things, no project roadmap, no priorities...nothing. Btw i have a new question. suppose there is an website i want to web scrape. assume it has 800 pages, each page has 10 articles listed. u can click on any link and read the article. i want to just save them as markdown, or may be i want to save them as html, later i will make them markdown so that i can read them in obsidian. so if i create an agent flow, and use the block called "write file", will anythingllm prompt me to ask where i want to save files 800*10 = 8000 times? or u can send the folder path where you want to save in prompt. please bring the tutorials, there are a lot of confusions. as u are making this tool for everybody, u can't cover every use cases for everybody, but you can cover each tools. for example, if your last video on agent flow include the block "write file" i would not face this confusion today, but in that video you just explained only "api call" and "web scraping" blocks. i could try myself to discover how these things behave, but then i could not think how will i call 800 pages, as your video shows request to one link. if i make an url pattern of the website using variables and let the model handel the pagination, i fear if it behave weird. as i said, having a medium level pc, i am using low parameter model, so it can take wrong decisions. also my heath is not good, otherwise i could still try. but even if i tried and i was successful, a common person who doesn't have any coding knowledge can't do this.
Author
Owner

@roboticarmgroup commented on GitHub (Apr 3, 2026):

are you using a chinese model? i dont even know wtf anything llm is, it sounds retarded or targeted towards the type of retards who google chatgpt and sign up for some wrapper reseller and think they signed up for chatgpt. it looks to me like they're just reselling cheap chinese LLMs like Qwen or tailoring to people who are self hosting similar models.

anyway just a heads up, chinese models struggle with markdown and html because they still use chinese characters that we don't see in their english responses. you can just google it for more info

<!-- gh-comment-id:4182481973 --> @roboticarmgroup commented on GitHub (Apr 3, 2026): are you using a chinese model? i dont even know wtf anything llm is, it sounds retarded or targeted towards the type of retards who google chatgpt and sign up for some wrapper reseller and think they signed up for chatgpt. it looks to me like they're just reselling cheap chinese LLMs like Qwen or tailoring to people who are self hosting similar models. anyway just a heads up, chinese models struggle with markdown and html because they still use chinese characters that we don't see in their english responses. you can just google it for more info
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5002