Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1379] Agent not working #877

New Issue
Closed
opened 2026-02-22 18:21:57 -05:00 by yindo · 23 comments
yindo commented 2026-02-22 18:21:57 -05:00
Owner
Copy Link
Copy Source

Originally created by @Nunsferatus on GitHub (May 13, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1379

How are you running AnythingLLM?

Docker (local)

What happened?

image

This is the response that I receive no matter what.

Am I doing something wrong or is this a bug?!

Are there known steps to reproduce?

No response

Originally created by @Nunsferatus on GitHub (May 13, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1379 ### How are you running AnythingLLM? Docker (local) ### What happened? ![image](https://github.com/Mintplex-Labs/anything-llm/assets/83787364/8d63f436-1274-4a83-bbf7-cff1a462f9f2) This is the response that I receive no matter what. Am I doing something wrong or is this a bug?! ### Are there known steps to reproduce? _No response_
yindo added the possible bug label 2026-02-22 18:21:57 -05:00
yindo closed this issue 2026-02-22 18:21:57 -05:00
yindo commented 2026-02-22 18:21:57 -05:00
Author
Owner
Copy Link
Copy Source

@SamuelEnzi commented on GitHub (May 13, 2024):

As far as i know agents only work with OpenAI as an LLM provider for now. What are your provider settings?

@SamuelEnzi commented on GitHub (May 13, 2024): As far as i know agents only work with OpenAI as an LLM provider for now. What are your provider settings?
yindo commented 2026-02-22 18:21:58 -05:00
Author
Owner
Copy Link
Copy Source

@SamuelEnzi commented on GitHub (May 13, 2024):

@Nunsferatus here is a screenshot
image

@SamuelEnzi commented on GitHub (May 13, 2024): @Nunsferatus here is a screenshot ![image](https://github.com/Mintplex-Labs/anything-llm/assets/62288386/1a71441f-d60a-40e4-b465-3ecc4194b0c8)
yindo commented 2026-02-22 18:21:58 -05:00
Author
Owner
Copy Link
Copy Source

@Nunsferatus commented on GitHub (May 13, 2024):

Thank you for your quick response.

image

Here is the agent's configuration that I'm using.

@Nunsferatus commented on GitHub (May 13, 2024): Thank you for your quick response. ![image](https://github.com/Mintplex-Labs/anything-llm/assets/83787364/982865fd-062a-47a0-bf45-a8070a12a90b) Here is the agent's configuration that I'm using.
yindo commented 2026-02-22 18:21:58 -05:00
Author
Owner
Copy Link
Copy Source

@SamuelEnzi commented on GitHub (May 13, 2024):

@Nunsferatus Yes it seems like you use Ollama which does currently not support agent as far as i know. You should switch to OpenAI Provider or wait until this feature is implemented by the team.

@SamuelEnzi commented on GitHub (May 13, 2024): @Nunsferatus Yes it seems like you use Ollama which does currently not support agent as far as i know. You should switch to OpenAI Provider or wait until this feature is implemented by the team.
yindo commented 2026-02-22 18:21:58 -05:00
Author
Owner
Copy Link
Copy Source

@Nunsferatus commented on GitHub (May 13, 2024):

I don't think that you are correct.
As you can see in the following video at minute 9:53, Tim Carambat uses, as far as I understand, the same configuration that I am trying to use.

https://www.youtube.com/watch?v=4UFrVvy7VlA

@Nunsferatus commented on GitHub (May 13, 2024): I don't think that you are correct. As you can see in the following video at minute 9:53, Tim Carambat uses, as far as I understand, the same configuration that I am trying to use. https://www.youtube.com/watch?v=4UFrVvy7VlA
yindo commented 2026-02-22 18:21:58 -05:00
Author
Owner
Copy Link
Copy Source

@SamuelEnzi commented on GitHub (May 13, 2024):

You are right it is shown in the video. Either there is conflicting Information or Tim is using a different version. For me it does not work either when i try other providers.

@SamuelEnzi commented on GitHub (May 13, 2024): You are right it is shown in the video. Either there is conflicting Information or Tim is using a different version. For me it does not work either when i try other providers.
yindo commented 2026-02-22 18:21:58 -05:00
Author
Owner
Copy Link
Copy Source

@Nunsferatus commented on GitHub (May 13, 2024):

Thank you Samuel for your help.

Hopefully, someone who is more knowledgeable than we can figure out what is happening...

@Nunsferatus commented on GitHub (May 13, 2024): Thank you Samuel for your help. Hopefully, someone who is more knowledgeable than we can figure out what is happening...
yindo commented 2026-02-22 18:21:58 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 13, 2024):

@SamuelEnzi Yeah, the latest docker and desktop versions now support OSS model providers like Ollama.

@Nunsferatus, this is worth me writing an article on for our docs page because we get this question at least twice a day and I touched on it in the video at bit.

TLDR: LLMs are not deterministic and its ability to know and call a tool is dependent on many factors, but the main on is the model size + quantization. In that video i am using LLama3-8B Q8, which is okay, but not perfect!

We can tell and LLM to do x,y,z but that does not mean it will listen and its steerability and coherence of a question is always a little random. We currently even set the temperature of every tool call to zero, but that still does not mean it will call the tools.

This is also me assuming that the PDF you want to summarize is in the workspace you are asking. Sometimes the LLM is so hard-headed that you even need to tell it the exact tool. Use doc-summarizer to summarize mypdf.pdf and it still may not work.

This is just a reality of LLMs, they certainly are not coherent and using bigger and better models produces better results but that is still not the golden ticket. The reason some LLMs like OpenAI do tool calls separately is to "force" them to use a tool, whereas with our wrapper we cannot exert that level on control on the LLM. Which also could be a bad idea to "force" this as it may just constantly loop, or never call anything, and a whole bunch of other issues.

Basically, it is not magic and its really model dependent, but neither of those guarantee any OSS LLM will always 100% understand and comprehend what your question was and leverage tools to answer that.

What model and quantization are you running? Closing for now, but we can continue disucssion

@timothycarambat commented on GitHub (May 13, 2024): @SamuelEnzi Yeah, the latest docker and desktop versions now support OSS model providers like Ollama. @Nunsferatus, this is worth me writing an article on for our docs page because we get this question at least twice a day and I touched on it in the video at bit. TLDR: LLMs are **not deterministic** and its ability to know and call a tool is dependent on many factors, but the main on is the model size + quantization. In that video i am using LLama3-8B Q8, which is okay, but not perfect! We can tell and LLM to do x,y,z but that does not mean it will listen and its steerability and coherence of a question is always a little random. We currently even set the temperature of every tool call to zero, but that still does not mean it **will call** the tools. This is also me assuming that the PDF you want to summarize is in the workspace you are asking. Sometimes the LLM is so hard-headed that you even need to tell it the exact tool. `Use doc-summarizer to summarize mypdf.pdf` and it still may not work. This is just a reality of LLMs, they certainly are not coherent and using bigger and better models produces better results but that is still not the golden ticket. The reason some LLMs like OpenAI do tool calls separately is to "force" them to use a tool, whereas with our wrapper we cannot exert that level on control on the LLM. Which also could be a bad idea to "force" this as it may just constantly loop, or never call anything, and a whole bunch of other issues. Basically, it is not magic and its really model dependent, but neither of those guarantee any OSS LLM will always 100% understand and comprehend what your question was and leverage tools to answer that. What model and quantization are you running? Closing for now, but we can continue disucssion
yindo commented 2026-02-22 18:21:58 -05:00
Author
Owner
Copy Link
Copy Source

@tiangao88 commented on GitHub (May 14, 2024):

Sorry to reopen this discussion.
I Just installed the latest docker image and have the same issue while using gpt-4o

image image image image

How can I see which version of AnythingLLM I am actually running? Maybe I did something wrong in the configuration.

@tiangao88 commented on GitHub (May 14, 2024): Sorry to reopen this discussion. I Just installed the latest docker image and have the same issue while using gpt-4o <img width="593" alt="image" src="https://github.com/Mintplex-Labs/anything-llm/assets/53409436/9046816b-2241-421c-90c5-c9373f7cf91e"> <img width="985" alt="image" src="https://github.com/Mintplex-Labs/anything-llm/assets/53409436/b8916e0d-8007-4af2-a13f-a7c3e72ab2a0"> <img width="990" alt="image" src="https://github.com/Mintplex-Labs/anything-llm/assets/53409436/cd1ec220-d56e-43c1-ac4d-011e30c4543e"> <img width="570" alt="image" src="https://github.com/Mintplex-Labs/anything-llm/assets/53409436/6bf5674b-27a1-402b-b962-c32c6bb2df43"> How can I see which version of AnythingLLM I am actually running? Maybe I did something wrong in the configuration.
yindo commented 2026-02-22 18:21:58 -05:00
Author
Owner
Copy Link
Copy Source

@Nunsferatus commented on GitHub (May 14, 2024):

Hi Timothy,
Thank you for your answer and I think that either an article or a new video on it would be just fantastic.

As you can see from my previous post I am using llama3:8b-instruct-q8_0
Which if I'm not mistaken is exactly the one that you use in the video.

I have also tried mistral:7b-instruct-q8_0 with the same result.

image

I have tried many ways of making the question to see if that would solve the problem none of my tries worked.

I am running it in Docker do you think that's the issue?
I have it that way because I would prefer to have it in a way that I can use on whatever platform/Device I want as long as I'm in my network without having to install it on every device.

Thanks again

@Nunsferatus commented on GitHub (May 14, 2024): Hi Timothy, Thank you for your answer and I think that either an article or a new video on it would be just fantastic. As you can see from my previous post I am using llama3:8b-instruct-q8_0 Which if I'm not mistaken is exactly the one that you use in the video. I have also tried mistral:7b-instruct-q8_0 with the same result. ![image](https://github.com/Mintplex-Labs/anything-llm/assets/83787364/3ee9b019-8148-4f34-ba45-1a2aa58432e9) I have tried many ways of making the question to see if that would solve the problem none of my tries worked. I am running it in Docker do you think that's the issue? I have it that way because I would prefer to have it in a way that I can use on whatever platform/Device I want as long as I'm in my network without having to install it on every device. Thanks again
yindo commented 2026-02-22 18:21:58 -05:00
Author
Owner
Copy Link
Copy Source

@Anto79-ops commented on GitHub (May 14, 2024):

@tiangao88 im experieing the same as you, it just closes the agent immediately wthiout any actions. I did get a log

Failed to run validation function on agent_search_provider Invalid SERP provider.

Which is strange because I had web-search disabled. Then I got proper Google search crendentaisl, but still exiting almost immeadiately from the agent chat.

I'm not sure what is happened, but what makes this harder is that there are no logs being produced. You may have to restart your instance (if in docker) and try again, and look into your logs. It logs the error above (if that is the error) only once.

Sorry to reopen this discussion. I Just installed the latest docker image and have the same issue while using gpt-4o

image image image image
How can I see which version of AnythingLLM I am actually running? Maybe I did something wrong in the configuration.

@Anto79-ops commented on GitHub (May 14, 2024): @tiangao88 im experieing the same as you, it just closes the agent immediately wthiout any actions. I did get a log `Failed to run validation function on agent_search_provider Invalid SERP provider.` Which is strange because I had web-search disabled. Then I got proper Google search crendentaisl, but still exiting almost immeadiately from the agent chat. I'm not sure what is happened, but what makes this harder is that there are no logs being produced. You may have to restart your instance (if in docker) and try again, and look into your logs. It logs the error above (if that is the error) only once. > Sorry to reopen this discussion. I Just installed the latest docker image and have the same issue while using gpt-4o > > <img alt="image" width="593" src="https://private-user-images.githubusercontent.com/53409436/330412076-9046816b-2241-421c-90c5-c9373f7cf91e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTU3MTYxOTAsIm5iZiI6MTcxNTcxNTg5MCwicGF0aCI6Ii81MzQwOTQzNi8zMzA0MTIwNzYtOTA0NjgxNmItMjI0MS00MjFjLTkwYzUtYzkzNzNmN2NmOTFlLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA1MTQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNTE0VDE5NDQ1MFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWIxZDM5Y2U0MjRkYzU0N2YxODU2Mjk2ZTQ3YzAxMjkyNmI0MjU2ZDBmOGIxYjgwMGY5NjZmMmFkMzA2OWExMWEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.XZXuCODLzmRBB00pU8KGxPSx-PIuTBQ4twUQxrz-Org"> <img alt="image" width="985" src="https://private-user-images.githubusercontent.com/53409436/330412205-b8916e0d-8007-4af2-a13f-a7c3e72ab2a0.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTU3MTYxOTAsIm5iZiI6MTcxNTcxNTg5MCwicGF0aCI6Ii81MzQwOTQzNi8zMzA0MTIyMDUtYjg5MTZlMGQtODAwNy00YWYyLWExM2YtYTdjM2U3MmFiMmEwLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA1MTQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNTE0VDE5NDQ1MFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTFmYzAwNTlhZGZkN2QxMWJmNWNlZTQwM2QxOWY1Y2Q2MTUwMGJhMjdmMDg3YzI3OTMwMjI0NmVmYzhiMmI3ODImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.LM1mDbtRbdgmLZdslkJRmO45RjKHU4aQoSofdAwP9p0"> <img alt="image" width="990" src="https://private-user-images.githubusercontent.com/53409436/330412295-cd1ec220-d56e-43c1-ac4d-011e30c4543e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTU3MTYxOTAsIm5iZiI6MTcxNTcxNTg5MCwicGF0aCI6Ii81MzQwOTQzNi8zMzA0MTIyOTUtY2QxZWMyMjAtZDU2ZS00M2MxLWFjNGQtMDExZTMwYzQ1NDNlLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA1MTQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNTE0VDE5NDQ1MFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWMzMjExZWQ5YmI5OWFkMzYxZmRlYzAxYTdiNTNiMjc2NTlmMDNmODllZWIyMTBhYjI3MmQzZTk1MmRjMWMxZTUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.ChhoF4w55YKJeLrnHsr8aPpx9pGhHXvYk3D1fdgfstE"> <img alt="image" width="570" src="https://private-user-images.githubusercontent.com/53409436/330412759-6bf5674b-27a1-402b-b962-c32c6bb2df43.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTU3MTYxOTAsIm5iZiI6MTcxNTcxNTg5MCwicGF0aCI6Ii81MzQwOTQzNi8zMzA0MTI3NTktNmJmNTY3NGItMjdhMS00MDJiLWI5NjItYzMyYzZiYjJkZjQzLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA1MTQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNTE0VDE5NDQ1MFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTgzYWJiZGZhYjdkMmE5NWYyNTYzMDVmOTk0ZGRlZjJjNzA5MDZmOTA2NTBmZjIzNTQ5OGMwMzg0ZWFjMjM1OTEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0._BkayxjszCqUs9s9DxAcvbqJMxkpQAxufGL6v-4x6EE"> > How can I see which version of AnythingLLM I am actually running? Maybe I did something wrong in the configuration.
yindo commented 2026-02-22 18:21:58 -05:00
Author
Owner
Copy Link
Copy Source

@SamuelEnzi commented on GitHub (May 15, 2024):

@Anto79-ops @tiangao88 It seems like the issue has something todo with the docker host. I host my instance on a server (docker) within my local network and i have the same issue as @tiangao88 where it instantly exits from the agent.

To confirm my theory i installed the windows desktop application and used the OLLAMA instance hosted on the same server than before. Locally the agent works without any issue.

Here are two screenshots. First is the Desktop application and second the docker instance

image
image

@SamuelEnzi commented on GitHub (May 15, 2024): @Anto79-ops @tiangao88 It seems like the issue has something todo with the docker host. I host my instance on a server (docker) within my local network and i have the same issue as @tiangao88 where it instantly exits from the agent. To confirm my theory i installed the windows desktop application and used the OLLAMA instance hosted on the same server than before. Locally the agent works without any issue. Here are two screenshots. First is the Desktop application and second the docker instance ![image](https://github.com/Mintplex-Labs/anything-llm/assets/62288386/c98c1057-af14-4e19-b609-8d281330e1ae) ![image](https://github.com/Mintplex-Labs/anything-llm/assets/62288386/dbbff943-2eaf-4189-926b-11c49a12b154)
yindo commented 2026-02-22 18:21:58 -05:00
Author
Owner
Copy Link
Copy Source

@max-zirkonzahn commented on GitHub (May 15, 2024):

If you are running Apache as a reverse proxy, the problem seems to be Apache. The frontend tries to establish a WSS connection when calling the agent and fails to do so, as can be seen in the webbrowser console.
grafik
When you try to access the frontend directly without the Apache proxy, the agent chat works.

I tried to add

        RewriteEngine on
        RewriteCond ${HTTP:Upgrade} websocket [NC]
        RewriteCond ${HTTP:Connection} upgrade [NC]
        RewriteRule .* "ws:/<ip of AnythingLLMserver>/$1" [P,L]

to my VirtualHost, but that didn't fix the problem. (If I made some errors here, please correct me)

I switched from Apache to HAproxy, based on my experience with WSS and Apache.
Agent chat is working now. I hope this helps

@max-zirkonzahn commented on GitHub (May 15, 2024): If you are running Apache as a reverse proxy, the problem seems to be Apache. The frontend tries to establish a WSS connection when calling the agent and fails to do so, as can be seen in the webbrowser console. ![grafik](https://github.com/Mintplex-Labs/anything-llm/assets/137179419/4ad39b3f-9536-4695-970f-7b617309143c) When you try to access the frontend directly without the Apache proxy, the agent chat works. I tried to add ```apache RewriteEngine on RewriteCond ${HTTP:Upgrade} websocket [NC] RewriteCond ${HTTP:Connection} upgrade [NC] RewriteRule .* "ws:/<ip of AnythingLLMserver>/$1" [P,L] ``` to my VirtualHost, but that didn't fix the problem. (If I made some errors here, please correct me) I switched from Apache to HAproxy, based on my experience with WSS and Apache. Agent chat is working now. I hope this helps
yindo commented 2026-02-22 18:21:59 -05:00
Author
Owner
Copy Link
Copy Source

@tiangao88 commented on GitHub (May 15, 2024):

If you are running Apache as a reverse proxy, the problem seems to be Apache. The frontend tries to establish a WSS connection when calling the agent and fails to do so, as can be seen in the webbrowser console. grafik When you try to access the frontend directly without the Apache proxy, the agent chat works.

I tried to add

        RewriteEngine on
        RewriteCond ${HTTP:Upgrade} websocket [NC]
        RewriteCond ${HTTP:Connection} upgrade [NC]
        RewriteRule .* "ws:/<ip of AnythingLLMserver>/$1" [P,L]

to my VirtualHost, but that didn't fix the problem. (If I made some errors here, please correct me)

I switched from Apache to HAproxy, based on my experience with WSS and Apache. Agent chat is working now. I hope this helps

You have indeed found the root cause, thanks!
When accessing directly the docker container IP, bypassing all proxy, the Agent functionalities are working.
Now I have to find how to customize Nginx configuration to let websocket connections through.

@tiangao88 commented on GitHub (May 15, 2024): > If you are running Apache as a reverse proxy, the problem seems to be Apache. The frontend tries to establish a WSS connection when calling the agent and fails to do so, as can be seen in the webbrowser console. ![grafik](https://private-user-images.githubusercontent.com/137179419/330732264-4ad39b3f-9536-4695-970f-7b617309143c.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTU3NzU1MzMsIm5iZiI6MTcxNTc3NTIzMywicGF0aCI6Ii8xMzcxNzk0MTkvMzMwNzMyMjY0LTRhZDM5YjNmLTk1MzYtNDY5NS05NzBmLTdiNjE3MzA5MTQzYy5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNTE1JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDUxNVQxMjEzNTNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1mNWU5MjIyMjU1ZmU1NWZiMDhmZTA3YmYxOGFmNzI5MWYyNjdmYmU5MDA3ZGE3ZmFjNmQ0YWYzNzNhZTU1NTljJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.DLdX76LcSFyZrldLTOMCRzRrxiIVB0iqyZzAFvEand8) When you try to access the frontend directly without the Apache proxy, the agent chat works. > > I tried to add > > ```apache-config > RewriteEngine on > RewriteCond ${HTTP:Upgrade} websocket [NC] > RewriteCond ${HTTP:Connection} upgrade [NC] > RewriteRule .* "ws:/<ip of AnythingLLMserver>/$1" [P,L] > ``` > > to my VirtualHost, but that didn't fix the problem. (If I made some errors here, please correct me) > > I switched from Apache to HAproxy, based on my experience with WSS and Apache. Agent chat is working now. I hope this helps You have indeed found the root cause, thanks! When accessing directly the docker container IP, bypassing all proxy, the Agent functionalities are working. Now I have to find how to customize Nginx configuration to let websocket connections through.
yindo commented 2026-02-22 18:21:59 -05:00
Author
Owner
Copy Link
Copy Source

@Anto79-ops commented on GitHub (May 15, 2024):

can I ask, is Apache something that is already running and installed? I don't recall installing/using this. thanks

@Anto79-ops commented on GitHub (May 15, 2024): can I ask, is Apache something that is already running and installed? I don't recall installing/using this. thanks
yindo commented 2026-02-22 18:21:59 -05:00
Author
Owner
Copy Link
Copy Source

@max-zirkonzahn commented on GitHub (May 15, 2024):

can I ask, is Apache something that is already running and installed? I don't recall installing/using this. thanks

Sure, Apache is not installed by default, but some people, like myself, add it to add SSL/TLS to the connection to protect the data in transit.

@max-zirkonzahn commented on GitHub (May 15, 2024): > can I ask, is Apache something that is already running and installed? I don't recall installing/using this. thanks Sure, Apache is not installed by default, but some people, like myself, add it to add SSL/TLS to the connection to protect the data in transit.
yindo commented 2026-02-22 18:21:59 -05:00
Author
Owner
Copy Link
Copy Source

@Anto79-ops commented on GitHub (May 15, 2024):

thank you, I can confirm I'm getting the same problem as shown here with my Chrome logs

image

I will see if switching to HAproxy fixes it. thank you!

@Anto79-ops commented on GitHub (May 15, 2024): thank you, I can confirm I'm getting the same problem as shown here with my Chrome logs ![image](https://github.com/Mintplex-Labs/anything-llm/assets/79593761/19b960ff-4f87-4ca5-ad9a-a83c7a20dfc8) I will see if switching to HAproxy fixes it. thank you!
yindo commented 2026-02-22 18:21:59 -05:00
Author
Owner
Copy Link
Copy Source

@man2004 commented on GitHub (May 16, 2024):

@tiangao88 , please see if https://github.com/Mintplex-Labs/anything-llm/issues/1257#issuecomment-2089399353 helps.

@man2004 commented on GitHub (May 16, 2024): @tiangao88 , please see if https://github.com/Mintplex-Labs/anything-llm/issues/1257#issuecomment-2089399353 helps.
yindo commented 2026-02-22 18:21:59 -05:00
Author
Owner
Copy Link
Copy Source

@greigmarshall commented on GitHub (Jun 2, 2024):

I had the same error shown in the screenshots from @max-zirkonzahn and @Anto79-ops. I'm using nginx for reverse proxy and was able to resolve the issue by adding the following to the appropriate server block:

    location /api {
        proxy_pass http://<server IP:port>/api/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_connect_timeout 7d;
        proxy_send_timeout 7d;
        proxy_read_timeout 7d;
    }

This configuration can be applied to anything that uses websocket.

@greigmarshall commented on GitHub (Jun 2, 2024): I had the same error shown in the screenshots from @max-zirkonzahn and @Anto79-ops. I'm using nginx for reverse proxy and was able to resolve the issue by adding the following to the appropriate server block: ``` location /api { proxy_pass http://<server IP:port>/api/; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_set_header Host $host; proxy_connect_timeout 7d; proxy_send_timeout 7d; proxy_read_timeout 7d; } ``` This configuration can be applied to anything that uses websocket.
yindo commented 2026-02-22 18:21:59 -05:00
Author
Owner
Copy Link
Copy Source

@tnypxl commented on GitHub (Sep 7, 2024):

@greigmarshall Was running into this issue and your suggestion fix it! You rock!

@tnypxl commented on GitHub (Sep 7, 2024): @greigmarshall Was running into this issue and your suggestion fix it! You rock!
yindo commented 2026-02-22 18:21:59 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Sep 9, 2024):

FWIW; this is buried in the docs, we should duplicate it to the BARE_METAL docs at least as an addendum for proxies since this issue often gets overlooked and makes agents look broken

https://github.com/Mintplex-Labs/anything-llm/blob/master/cloud-deployments/aws/cloudformation/aws_https_instructions.md#step-7-create-simple-http-proxy-configuration-for-anythingllm

@timothycarambat commented on GitHub (Sep 9, 2024): FWIW; this is buried in the docs, we should duplicate it to the BARE_METAL docs at least as an addendum for proxies since this issue often gets overlooked and makes agents look broken https://github.com/Mintplex-Labs/anything-llm/blob/master/cloud-deployments/aws/cloudformation/aws_https_instructions.md#step-7-create-simple-http-proxy-configuration-for-anythingllm
yindo commented 2026-02-22 18:21:59 -05:00
Author
Owner
Copy Link
Copy Source

@nasierjaffer commented on GitHub (Jan 26, 2025):

See below the config in apache2 that fixes the problem

Apache Modules Required

sudo a2enmod proxy
sudo a2enmod proxy_http
sudo a2enmod proxy_wstunnel
sudo a2enmod rewrite
sudo a2enmod headers
sudo systemctl restart apache2

Modify your virtualhost as below

<VirtualHost *:80>
ServerName example.domain.com
Redirect / https://example.domain.com/
ErrorLog ${APACHE_LOG_DIR}/example.domain.com.error.log
CustomLog ${APACHE_LOG_DIR}/example.domain.com.log combined
RewriteEngine on
RewriteCond %{SERVER_NAME} =example.domain.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]

ServerName example.domain.com ServerAdmin webmaster@localhost DocumentRoot /var/www/html/ 
    # Enable required modules
    LoadModule proxy_module modules/mod_proxy.so
    LoadModule proxy_http_module modules/mod_proxy_http.so
    LoadModule proxy_wstunnel_module modules/mod_proxy_wstunnel.so

    # WebSocket support for agent protocol
    ProxyRequests Off
    ProxyPreserveHost On
    
    # Special handling for WebSocket connections
    RewriteEngine On
    RewriteCond %{HTTP:Upgrade} websocket [NC]
    RewriteCond %{HTTP:Connection} upgrade [NC]
    RewriteRule ^/api/agent-invocation/(.*) ws://localhost:3001/api/agent-invocation/$1 [P,L]

    # Proxy settings for long-running requests
    ProxyTimeout 605
    TimeOut 605
    
    # Main proxy configuration
    ProxyPass / http://localhost:3001/ timeout=605 keepalive=On
    ProxyPassReverse / http://localhost:3001/

    # Disable buffering for streaming responses
    SetEnv proxy-nokeepalive 1
    SetEnv proxy-initial-not-pooled 1
    
    # SSL Configuration
    SSLEngine on
    SSLCertificateFile /path/to/your/fullchain.pem
    SSLCertificateKeyFile /path/to/your/privkey.pem
    Include /etc/letsencrypt/options-ssl-apache.conf

    # Additional headers for WebSocket
    Header always set Access-Control-Allow-Origin "*"
    Header always set Access-Control-Allow-Methods "GET, POST, OPTIONS"
    Header always set Access-Control-Allow-Headers "DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization"
</VirtualHost>
@nasierjaffer commented on GitHub (Jan 26, 2025): See below the config in apache2 that fixes the problem Apache Modules Required sudo a2enmod proxy sudo a2enmod proxy_http sudo a2enmod proxy_wstunnel sudo a2enmod rewrite sudo a2enmod headers sudo systemctl restart apache2 Modify your virtualhost as below <VirtualHost *:80> ServerName example.domain.com Redirect / https://example.domain.com/ ErrorLog ${APACHE_LOG_DIR}/example.domain.com.error.log CustomLog ${APACHE_LOG_DIR}/example.domain.com.log combined RewriteEngine on RewriteCond %{SERVER_NAME} =example.domain.com RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent] </VirtualHost> <IfModule mod_ssl.c> <VirtualHost *:443> ServerName example.domain.com ServerAdmin webmaster@localhost DocumentRoot /var/www/html/ # Enable required modules LoadModule proxy_module modules/mod_proxy.so LoadModule proxy_http_module modules/mod_proxy_http.so LoadModule proxy_wstunnel_module modules/mod_proxy_wstunnel.so # WebSocket support for agent protocol ProxyRequests Off ProxyPreserveHost On # Special handling for WebSocket connections RewriteEngine On RewriteCond %{HTTP:Upgrade} websocket [NC] RewriteCond %{HTTP:Connection} upgrade [NC] RewriteRule ^/api/agent-invocation/(.*) ws://localhost:3001/api/agent-invocation/$1 [P,L] # Proxy settings for long-running requests ProxyTimeout 605 TimeOut 605 # Main proxy configuration ProxyPass / http://localhost:3001/ timeout=605 keepalive=On ProxyPassReverse / http://localhost:3001/ # Disable buffering for streaming responses SetEnv proxy-nokeepalive 1 SetEnv proxy-initial-not-pooled 1 # SSL Configuration SSLEngine on SSLCertificateFile /path/to/your/fullchain.pem SSLCertificateKeyFile /path/to/your/privkey.pem Include /etc/letsencrypt/options-ssl-apache.conf # Additional headers for WebSocket Header always set Access-Control-Allow-Origin "*" Header always set Access-Control-Allow-Methods "GET, POST, OPTIONS" Header always set Access-Control-Allow-Headers "DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization" </VirtualHost> </IfModule>
yindo changed title from Agent not working to [GH-ISSUE #1379] Agent not working 2026-06-05 14:37:42 -04:00
yindo commented 2026-06-05 14:37:42 -04:00
Author
Owner
Copy Link
Copy Source

@gitwittidbit commented on GitHub (May 12, 2026):

In my setup, the agent doesn't exit immediately, but it does so -- after a while -- without presenting any outcome.

It seems that this is the same issue neverthelesse, because it only happens when anythingllm is accessed via my reverse proxy (haproxy) and not when it is being accessed directly via the IP address.

Would someone who has got this working with haproxy please be able to tell me what settings I need to make this work for me, too?

Thanks.

<!-- gh-comment-id:4435145155 --> @gitwittidbit commented on GitHub (May 12, 2026): In my setup, the agent doesn't exit immediately, but it does so -- after a while -- without presenting any outcome. It seems that this is the same issue neverthelesse, because it only happens when anythingllm is accessed via my reverse proxy (haproxy) and not when it is being accessed directly via the IP address. Would someone who has got this working with haproxy please be able to tell me what settings I need to make this work for me, too? Thanks.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#877
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?