Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1335] [FEAT]: Expand summarization to generic LLM Provider #841

New Issue
Closed
opened 2026-02-22 18:21:43 -05:00 by yindo · 7 comments
yindo commented 2026-02-22 18:21:43 -05:00
Owner
Copy Link
Copy Source

Originally created by @FBTCAD on GitHub (May 10, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1335

Originally assigned to: @timothycarambat on GitHub.

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

using LM Studio Community/Meta-Llama-3-8B-Instruct-GGUF/Meta-Llama-3-8B-Instruct-Q4_K_M.gguf in LM Studio when trying to summarize a pdf file asked for API key.
[2024-05-10 19:57:37.216] [INFO] [LM Studio Community/Meta-Llama-3-8B-Instruct-GGUF/Meta-Llama-3-8B-Instruct-Q4_K_M.gguf] Accumulated 108 tokens: I apologize for the inconvenience, but I was unable to summarize the document MBThesis.pdf due to an error.\n\nError:\nAn error occurred while attempting to process the file. The necessary API keys (OpenAI or Azure OpenAI) were not found, which prevented me from accessing and analyzing the document.\n\nIf you would like to provide me with a valid API key or an alternative method to summarize the document, I'll be happy to assist you further. Alternatively, please let me know if there's anything else I can help you

Here is the output from AnythingLLM:
@agent summary of MBThesis.pdf

Agent @agent invoked.
Swapping over to agent chat. Type /exit to exit agent execution loop early.
[debug]: @agent is attempting to call document-summarizer tool
@agent: Looking at the available documents.
@agent: Found 1 documents
@agent: Grabbing all content for MBThesis.pdf
@agent: Summarizing MBThesis.pdf...
I apologize for the inconvenience, but I was unable to summarize the document MBThesis.pdf due to an error.

Error: An error occurred while attempting to process the file. The necessary API keys (OpenAI or Azure OpenAI) were not found, which prevented me from accessing and analyzing the document.

If you would like to provide me with a valid API key or an alternative method to summarize the document, I’ll be happy to assist you further. Alternatively, please let me know if there’s anything else I can help you with.

Are there known steps to reproduce?

No response

Originally created by @FBTCAD on GitHub (May 10, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1335 Originally assigned to: @timothycarambat on GitHub. ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? using LM Studio Community/Meta-Llama-3-8B-Instruct-GGUF/Meta-Llama-3-8B-Instruct-Q4_K_M.gguf in LM Studio when trying to summarize a pdf file asked for API key. [2024-05-10 19:57:37.216] [INFO] [LM Studio Community/Meta-Llama-3-8B-Instruct-GGUF/Meta-Llama-3-8B-Instruct-Q4_K_M.gguf] Accumulated 108 tokens: I apologize for the inconvenience, but I was unable to summarize the document MBThesis.pdf due to an error.\n\n**Error:**\nAn error occurred while attempting to process the file. The necessary API keys (OpenAI or Azure OpenAI) were not found, which prevented me from accessing and analyzing the document.\n\nIf you would like to provide me with a valid API key or an alternative method to summarize the document, I'll be happy to assist you further. Alternatively, please let me know if there's anything else I can help you Here is the output from AnythingLLM: @agent summary of MBThesis.pdf Agent @agent invoked. Swapping over to agent chat. Type /exit to exit agent execution loop early. [debug]: @agent is attempting to call `document-summarizer` tool @agent: Looking at the available documents. @agent: Found 1 documents @agent: Grabbing all content for MBThesis.pdf @agent: Summarizing MBThesis.pdf... I apologize for the inconvenience, but I was unable to summarize the document MBThesis.pdf due to an error. Error: An error occurred while attempting to process the file. The necessary API keys (OpenAI or Azure OpenAI) were not found, which prevented me from accessing and analyzing the document. If you would like to provide me with a valid API key or an alternative method to summarize the document, I’ll be happy to assist you further. Alternatively, please let me know if there’s anything else I can help you with. ### Are there known steps to reproduce? _No response_
yindo added the enhancementcore-team-only labels 2026-02-22 18:21:43 -05:00
yindo closed this issue 2026-02-22 18:21:43 -05:00
yindo commented 2026-02-22 18:21:44 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 10, 2024):

I think this is a limitation with the doc-summarize tool. We need to make it more generic to be able to just use the current client. Will address this today

@timothycarambat commented on GitHub (May 10, 2024): I think this is a limitation with the `doc-summarize` tool. We need to make it more generic to be able to just use the current client. Will address this today
yindo commented 2026-02-22 18:21:44 -05:00
Author
Owner
Copy Link
Copy Source

@eriksonssilva commented on GitHub (May 16, 2024):

@timothycarambat I was facing the same issue. Then I decided to start AnythingLLM from terminal (I'm on Linux). Before that, I made sure to use 'export OPENAI_API_KEY="dummy-key"'.
Once I did that, the agent always gave me error 401 complaining about the API Key not being valid:
image
or
image

Note that in my case I am using Ollama and trying to mimic what you have explained here: https://www.youtube.com/watch?v=4UFrVvy7VlA

I appologize if I should not be bumping this topic since you told you're already working on it. I was just making sure to give you some more context.

Thanks a lot!

@eriksonssilva commented on GitHub (May 16, 2024): @timothycarambat I was facing the same issue. Then I decided to start AnythingLLM from terminal (I'm on Linux). Before that, I made sure to use 'export OPENAI_API_KEY="dummy-key"'. Once I did that, the agent always gave me error 401 complaining about the API Key not being valid: ![image](https://github.com/Mintplex-Labs/anything-llm/assets/43449566/ac086dab-7afb-49cc-b441-1bab2a41902a) or ![image](https://github.com/Mintplex-Labs/anything-llm/assets/43449566/27a88ad0-70fe-4e84-a05c-91acf53128e6) Note that in my case I am using Ollama and trying to mimic what you have explained here: https://www.youtube.com/watch?v=4UFrVvy7VlA I appologize if I should not be bumping this topic since you told you're already working on it. I was just making sure to give you some more context. Thanks a lot!
yindo commented 2026-02-22 18:21:44 -05:00
Author
Owner
Copy Link
Copy Source

@Propheticus commented on GitHub (May 23, 2024):

Found the same and reported here https://discord.com/channels/1114740394715004990/1243182212170645544

@Propheticus commented on GitHub (May 23, 2024): Found the same and reported here https://discord.com/channels/1114740394715004990/1243182212170645544
yindo commented 2026-02-22 18:21:44 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 23, 2024):

https://github.com/Mintplex-Labs/anything-llm/blob/acbad2f2cf7c13e03e0a0dc1067041ba4c59930c/server/utils/agents/aibitat/utils/summarize.js#L13

@timothycarambat commented on GitHub (May 23, 2024): https://github.com/Mintplex-Labs/anything-llm/blob/acbad2f2cf7c13e03e0a0dc1067041ba4c59930c/server/utils/agents/aibitat/utils/summarize.js#L13
yindo commented 2026-02-22 18:21:44 -05:00
Author
Owner
Copy Link
Copy Source

@Propheticus commented on GitHub (May 23, 2024):

https://github.com/Mintplex-Labs/anything-llm/blob/acbad2f2cf7c13e03e0a0dc1067041ba4c59930c/server/utils/agents/aibitat/utils/summarize.js#L13

Well that's as clear cut an explanation as they come 😄 No words needed.

@Propheticus commented on GitHub (May 23, 2024): > https://github.com/Mintplex-Labs/anything-llm/blob/acbad2f2cf7c13e03e0a0dc1067041ba4c59930c/server/utils/agents/aibitat/utils/summarize.js#L13 Well that's as clear cut an explanation as they come 😄 No words needed.
yindo commented 2026-02-22 18:21:44 -05:00
Author
Owner
Copy Link
Copy Source

@estcap2 commented on GitHub (Jun 6, 2024):

I am getting the same response as @FBTCAD .
I tried latest native macOS and Docker versions, different PDFs, diferent models etc.

Example using Ollama and llama3:8b-instruct-q8_0:

@agent can you please summarize DavydovTayar.pdf?
Agent @agent invoked. Swapping over to agent chat. Type /exit to exit agent execution loop early.
[debug]: @agent is attempting to call `document-summarizer` tool
@agent: Looking at the available documents.
@agent: Found 1 documents
@agent: Grabbing all content for DavydovTayar.pdf
@agent: Summarizing DavydovTayar.pdf...
I apologize, but it seems that I encountered an issue while attempting to summarize the file “DavydovTayar.pdf”. Unfortunately, I was unable to access the necessary information due to a missing OpenAI or Azure OpenAI API key.

It also asks for OpenAI key, the weird part is that the same document can be accessed via traditional Rag in the same chat without the agent, but I am looking to replicate the results seen here https://www.youtube.com/watch?v=4UFrVvy7VlA

Update:
Trying to run the same with a disabled internet connection, it throws a timeout:

Screenshot 2024-06-06 at 8 32 24 PM

@timothycarambat could it be that the tool is not 100% run locally?

@estcap2 commented on GitHub (Jun 6, 2024): I am getting the same response as @FBTCAD . I tried latest native macOS and Docker versions, different PDFs, diferent models etc. Example using Ollama and llama3:8b-instruct-q8_0: ``` @agent can you please summarize DavydovTayar.pdf? Agent @agent invoked. Swapping over to agent chat. Type /exit to exit agent execution loop early. [debug]: @agent is attempting to call `document-summarizer` tool @agent: Looking at the available documents. @agent: Found 1 documents @agent: Grabbing all content for DavydovTayar.pdf @agent: Summarizing DavydovTayar.pdf... I apologize, but it seems that I encountered an issue while attempting to summarize the file “DavydovTayar.pdf”. Unfortunately, I was unable to access the necessary information due to a missing OpenAI or Azure OpenAI API key. ``` It also asks for OpenAI key, the weird part is that the same document can be accessed via traditional Rag in the same chat without the agent, but I am looking to replicate the results seen here https://www.youtube.com/watch?v=4UFrVvy7VlA Update: Trying to run the same with a disabled internet connection, it throws a timeout: <img width="836" alt="Screenshot 2024-06-06 at 8 32 24 PM" src="https://github.com/Mintplex-Labs/anything-llm/assets/21089308/bb470dd6-4102-490f-aa00-521f58b15aaf"> @timothycarambat could it be that the tool is not 100% run locally?
yindo commented 2026-02-22 18:21:44 -05:00
Author
Owner
Copy Link
Copy Source

@estcap2 commented on GitHub (Jun 6, 2024):

I am getting the same response as @FBTCAD . I tried latest native macOS and Docker versions, different PDFs, diferent models etc.

Example using Ollama and llama3:8b-instruct-q8_0:

@agent can you please summarize DavydovTayar.pdf?
Agent @agent invoked. Swapping over to agent chat. Type /exit to exit agent execution loop early.
[debug]: @agent is attempting to call `document-summarizer` tool
@agent: Looking at the available documents.
@agent: Found 1 documents
@agent: Grabbing all content for DavydovTayar.pdf
@agent: Summarizing DavydovTayar.pdf...
I apologize, but it seems that I encountered an issue while attempting to summarize the file “DavydovTayar.pdf”. Unfortunately, I was unable to access the necessary information due to a missing OpenAI or Azure OpenAI API key.

It also asks for OpenAI key, the weird part is that the same document can be accessed via traditional Rag in the same chat without the agent, but I am looking to replicate the results seen here https://www.youtube.com/watch?v=4UFrVvy7VlA

Update: Trying to run the same with a disabled internet connection, it throws a timeout:

Screenshot 2024-06-06 at 8 32 24 PM @timothycarambat could it be that the tool is not 100% run locally?

🤦‍♂️ nevermind I just took a look to the tool's code. For anyone wondering, at the moment the summarization is not run locally, it uses OpenAI's GPT.
My confusion arose from the cited video and the notion I had that everything was run locally

@estcap2 commented on GitHub (Jun 6, 2024): > I am getting the same response as @FBTCAD . I tried latest native macOS and Docker versions, different PDFs, diferent models etc. > > Example using Ollama and llama3:8b-instruct-q8_0: > > ``` > @agent can you please summarize DavydovTayar.pdf? > Agent @agent invoked. Swapping over to agent chat. Type /exit to exit agent execution loop early. > [debug]: @agent is attempting to call `document-summarizer` tool > @agent: Looking at the available documents. > @agent: Found 1 documents > @agent: Grabbing all content for DavydovTayar.pdf > @agent: Summarizing DavydovTayar.pdf... > I apologize, but it seems that I encountered an issue while attempting to summarize the file “DavydovTayar.pdf”. Unfortunately, I was unable to access the necessary information due to a missing OpenAI or Azure OpenAI API key. > ``` > > It also asks for OpenAI key, the weird part is that the same document can be accessed via traditional Rag in the same chat without the agent, but I am looking to replicate the results seen here https://www.youtube.com/watch?v=4UFrVvy7VlA > > Update: Trying to run the same with a disabled internet connection, it throws a timeout: > > <img alt="Screenshot 2024-06-06 at 8 32 24 PM" width="836" src="https://private-user-images.githubusercontent.com/21089308/337466524-bb470dd6-4102-490f-aa00-521f58b15aaf.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTc3MTc4OTQsIm5iZiI6MTcxNzcxNzU5NCwicGF0aCI6Ii8yMTA4OTMwOC8zMzc0NjY1MjQtYmI0NzBkZDYtNDEwMi00OTBmLWFhMDAtNTIxZjU4YjE1YWFmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MDYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjA2VDIzNDYzNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWRjZjM1NjhjYTQxODEyOWFmMDY0ZjMxNjFiYzJiNmViMWRmZjYwOGRmNTBiNWE5NTE4MGVjZTU3ZTAyNDhhNTQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0._ntCy4cXoGQWrMPnVCiMAl3LRIFUP_Z20Jr8j_cP5c0"> > @timothycarambat could it be that the tool is not 100% run locally? 🤦‍♂️ nevermind I just took a look to the tool's code. For anyone wondering, at the moment the summarization is not run locally, it uses OpenAI's GPT. My confusion arose from the cited video and the notion I had that everything was run locally
yindo changed title from [FEAT]: Expand summarization to generic LLM Provider to [GH-ISSUE #1335] [FEAT]: Expand summarization to generic LLM Provider 2026-06-05 14:37:30 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label core-team-only enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#841
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?