Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1329] [BUG]: agents do not work with LM Studio #836

New Issue
Closed
opened 2026-02-22 18:21:39 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:21:39 -05:00
Owner
Copy Link
Copy Source

Originally created by @FBTCAD on GitHub (May 10, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1329

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

I loaded Model NousResearch/Meta-Llama-3-8B-Instruct-GGUF/Meta-Llama-3-8B-Instruct-Q5_K_M.gguf to summarize a document in pdf format - while the RAG feature works the summary reports that the document cannot be seen.
@agent can you summarize MBThesis.pdf

I apologize again, but I still don’t have access to the file named “MBThesis.pdf”. Could you please share the contents of the PDF file with me? Alternatively, if you could provide a brief summary or abstract of the thesis, I’d be happy to help you summarize it.

If you’re referring to a specific section or chapter within the thesis, please let me know and I’ll do my best to assist you.

LLM window shows:
[2024-05-10 13:35:38.697] [INFO] [NousResearch/Meta-Llama-3-8B-Instruct-GGUF/Meta-Llama-3-8B-Instruct-Q5_K_M.gguf] Accumulated 70 tokens: I apologize again, but I still don't have access to the file named "MBThesis.pdf". Could you please share the contents of the PDF file with me? Alternatively, if you could provide a brief summary or abstract of the thesis, I'd be happy to help you summarize it.\n\nIf you're referring to a specific section or chapter within

If I use RAG I can see the context is checked as we can see the chapters from the Thesis

<|start_header_id|>system<|end_header_id|>

Given the following conversation, relevant context, and a follow up question, reply with an answer to the current question the user is asking. Return only your response to the question given the above information following the users instructions as needed. Context: [CONTEXT 0]: been consistently providing support to Chinese semiconductorR&Dby establishing research centers and transferring sensitive technologies (Shihua & Costaglione,2023). Lastly, due to the implementation of the October 7 Export Control Regulations, many Amer- ican companies have been forbidden from doing business with their Chinese customers. This situation created a void in the market that will be filled over time by other companies, either from South Korea, Japan, or Europe. This may ultimately strike back and result in a diminution of the American position in the global semiconductor market. 75Chapter 5 Future Advancement Capacity of the Semiconductor Industry in China This chapter serves as a summary of the research and analyses conducted in the preceding parts of the thesis. It aims to evaluate the level of readiness of individual segments within the Chinese semiconductor industry to compete with their direct Western counterparts. The first section is solely devoted to the evaluation and temporal quantification of the tech- nological gap in various segments of the supply chain, beginning with theICdesign stage and concluding with the assembly and testing process. The subsequent section identifies three po- tential scenarios for the advancement of the chip-making industry, wherein the availability of advanced lithography has been used as a variable. 5.1. Progress Towards a Point of Parity with Semiconductor In- dustry Leaders This section examines the position of the various segments within the Chinese chip-making industry in relation to the global technological status. The following paragraphs provide a brief overview of the major advances and key existing challenges. Ultimately, the section reviews the progress, potential deviations from the mainstream, and evaluates the magnitude of the gap between China and specific members of the semiconductor Western world alliance. As the analysis is solely based on publicly available data, assumptions derived from expe- [END CONTEXT 0]

Are there known steps to reproduce?

just use LM studio , load NousResearch/Meta-Llama-3-8B-Instruct-GGUF/Meta-Llama-3-8B-Instruct-Q5_K_M.gguf , use nomonic-ai for embeddings in LM studio and anythingLLM and try to use agent

Originally created by @FBTCAD on GitHub (May 10, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1329 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? I loaded Model NousResearch/Meta-Llama-3-8B-Instruct-GGUF/Meta-Llama-3-8B-Instruct-Q5_K_M.gguf to summarize a document in pdf format - while the RAG feature works the summary reports that the document cannot be seen. @agent can you summarize MBThesis.pdf I apologize again, but I still don’t have access to the file named “MBThesis.pdf”. Could you please share the contents of the PDF file with me? Alternatively, if you could provide a brief summary or abstract of the thesis, I’d be happy to help you summarize it. If you’re referring to a specific section or chapter within the thesis, please let me know and I’ll do my best to assist you. LLM window shows: [2024-05-10 13:35:38.697] [INFO] [NousResearch/Meta-Llama-3-8B-Instruct-GGUF/Meta-Llama-3-8B-Instruct-Q5_K_M.gguf] Accumulated 70 tokens: I apologize again, but I still don't have access to the file named "MBThesis.pdf". Could you please share the contents of the PDF file with me? Alternatively, if you could provide a brief summary or abstract of the thesis, I'd be happy to help you summarize it.\n\nIf you're referring to a specific section or chapter within If I use RAG I can see the context is checked as we can see the chapters from the Thesis <|start_header_id|>system<|end_header_id|> Given the following conversation, relevant context, and a follow up question, reply with an answer to the current question the user is asking. Return only your response to the question given the above information following the users instructions as needed. Context: [CONTEXT 0]: been consistently providing support to Chinese semiconductorR&Dby establishing research centers and transferring sensitive technologies (Shihua & Costaglione,2023). Lastly, due to the implementation of the October 7 Export Control Regulations, many Amer- ican companies have been forbidden from doing business with their Chinese customers. This situation created a void in the market that will be filled over time by other companies, either from South Korea, Japan, or Europe. This may ultimately strike back and result in a diminution of the American position in the global semiconductor market. 75Chapter 5 Future Advancement Capacity of the Semiconductor Industry in China This chapter serves as a summary of the research and analyses conducted in the preceding parts of the thesis. It aims to evaluate the level of readiness of individual segments within the Chinese semiconductor industry to compete with their direct Western counterparts. The first section is solely devoted to the evaluation and temporal quantification of the tech- nological gap in various segments of the supply chain, beginning with theICdesign stage and concluding with the assembly and testing process. The subsequent section identifies three po- tential scenarios for the advancement of the chip-making industry, wherein the availability of advanced lithography has been used as a variable. 5.1. Progress Towards a Point of Parity with Semiconductor In- dustry Leaders This section examines the position of the various segments within the Chinese chip-making industry in relation to the global technological status. The following paragraphs provide a brief overview of the major advances and key existing challenges. Ultimately, the section reviews the progress, potential deviations from the mainstream, and evaluates the magnitude of the gap between China and specific members of the semiconductor Western world alliance. As the analysis is solely based on publicly available data, assumptions derived from expe- [END CONTEXT 0] ### Are there known steps to reproduce? just use LM studio , load NousResearch/Meta-Llama-3-8B-Instruct-GGUF/Meta-Llama-3-8B-Instruct-Q5_K_M.gguf , use nomonic-ai for embeddings in LM studio and anythingLLM and try to use agent
yindo added the possible bug label 2026-02-22 18:21:39 -05:00
yindo closed this issue 2026-02-22 18:21:39 -05:00
yindo commented 2026-02-22 18:21:40 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 10, 2024):

This is really, unfortunately, how models work. Its more in line with a "behavior" issue. I notice you are using a Q5, which should certainly be OK, but not great. The reason is a Q5 quantization on an already small 8B model can really throw random responses and more importantly - not follow instructions.

This is really just an artifact of the model. I bet if you were even more explicit in your direction to call out the specific tool, it may do better.
example: Using rag-memory to summarize MBThesis.pdf

Its annoying because in an ideal world, it should just work. It does do better with higher quantization values. The model should not only follow instructions better but also imply your commands via natural language better with something like a Q8.

Just the current state of LLMs!

@timothycarambat commented on GitHub (May 10, 2024): This is really, unfortunately, how models work. Its more in line with a "behavior" issue. I notice you are using a Q5, which should certainly be OK, but not great. The reason is a Q5 quantization on an already small 8B model can really throw random responses and more importantly - not follow instructions. This is really just an artifact of the model. I bet if you were even more explicit in your direction to call out the specific tool, it may do better. example: `Using rag-memory to summarize MBThesis.pdf` Its annoying because in an ideal world, it should just work. It _does_ do better with higher quantization values. The model should not only follow instructions better but also imply your commands via natural language better with something like a Q8. Just the current state of LLMs!
yindo changed title from [BUG]: agents do not work with LM Studio to [GH-ISSUE #1329] [BUG]: agents do not work with LM Studio 2026-06-05 14:37:27 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#836
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?