[GH-ISSUE #1586] [BUG]: Error after embedding document ""Could not respond to message" #1029

Closed
opened 2026-02-22 18:22:45 -05:00 by yindo · 25 comments
Owner

Originally created by @LLMuser on GitHub (May 31, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1586

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

Everytime I upload and embed a document I get the error ""Could not respond to message. An error occurred while streaming response. network error"".

Doesn´t matter which model I use. Happens even with fresh installation. After error occurs the application is frozen. After restart same error occurs.

I use Windows Server 2022 with NVIDIA Quadro K6000

Are there known steps to reproduce?

Install AnythingLLM. Upload and embed document (PDF). Type any message.

Originally created by @LLMuser on GitHub (May 31, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1586 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? Everytime I upload and embed a document I get the error ""Could not respond to message. An error occurred while streaming response. network error"". Doesn´t matter which model I use. Happens even with fresh installation. After error occurs the application is frozen. After restart same error occurs. I use Windows Server 2022 with NVIDIA Quadro K6000 ### Are there known steps to reproduce? Install AnythingLLM. Upload and embed document (PDF). Type any message.
yindo added the possible buginvestigating labels 2026-02-22 18:22:45 -05:00
yindo closed this issue 2026-02-22 18:22:45 -05:00
Author
Owner

@shatfield4 commented on GitHub (May 31, 2024):

If this is happening with every LLM model you are using to try and chat, it sounds like this is probably an issue with Windows Server firewall blocking the connection to whatever model you are using. You will need to investigate and see if your firewall settings are blocking you.

@shatfield4 commented on GitHub (May 31, 2024): If this is happening with every LLM model you are using to try and chat, it sounds like this is probably an issue with Windows Server firewall blocking the connection to whatever model you are using. You will need to investigate and see if your firewall settings are blocking you.
Author
Owner

@LLMuser commented on GitHub (Jun 3, 2024):

Thanx @shatfield4 but firewall is disabled. Without embedding documents all models work fine. So IMHO it seems that is has to do with the embedding model.

If I choose another embedder (e.g. Ollama instead of AnyghingLLM) in the preferences it a) takes much longer to embed a document and b) I get a different error: "Could not respond to message. LanceDBError: No vector column found to create index".

@LLMuser commented on GitHub (Jun 3, 2024): Thanx @shatfield4 but firewall is disabled. Without embedding documents all models work fine. So IMHO it seems that is has to do with the embedding model. If I choose another embedder (e.g. Ollama instead of AnyghingLLM) in the preferences it a) takes much longer to embed a document and b) I get a different error: "Could not respond to message. LanceDBError: No vector column found to create index".
Author
Owner

@timothycarambat commented on GitHub (Jun 3, 2024):

That response would typically indicate that the embedder failed to embed the content. With the default embedder can you send a chat on a brand new workspace with no documents? That should have no embedder call required.

Next, I would open C:\Users\user\AppData\Roaming\anythingllm-desktop\storage and see if there is a folder with files in it named storage\models\Xenova\ and there should be a subfolder for all-miniLM-l6 or something to the effect.

@timothycarambat commented on GitHub (Jun 3, 2024): That response would typically indicate that the embedder failed to embed the content. With the default embedder can you send a chat on a brand new workspace with no documents? That should have no embedder call required. Next, I would open `C:\Users\user\AppData\Roaming\anythingllm-desktop\storage` and see if there is a folder with files in it named `storage\models\Xenova\` and there should be a subfolder for `all-miniLM-l6` or something to the effect.
Author
Owner

@LLMuser commented on GitHub (Jun 4, 2024):

Thanks @timothycarambat!

With the default embedder can you send a chat on a brand new workspace with no documents? That should have no embedder call required.

If I set the default embedder and make a brand new chat with no documents, there is no error message.

Next, I would open C:\Users\user\AppData\Roaming\anythingllm-desktop\storage and see if there is a folder with files in it named storage\models\Xenova\ and there should be a subfolder for all-miniLM-l6 or something to the effect.

This folder exists. But I don´t understand what it indicates or what do do with it. Should I delete it? It has not created along with the brand new workspace though.

Thanks again for your support and your efforts on AnythingLLM!

@LLMuser commented on GitHub (Jun 4, 2024): Thanks @timothycarambat! > With the default embedder can you send a chat on a brand new workspace with no documents? That should have no embedder call required. If I set the default embedder and make a brand new chat with no documents, there is no error message. > Next, I would open `C:\Users\user\AppData\Roaming\anythingllm-desktop\storage` and see if there is a folder with files in it named `storage\models\Xenova\` and there should be a subfolder for `all-miniLM-l6` or something to the effect. This folder exists. But I don´t understand what it indicates or what do do with it. Should I delete it? It has not created along with the brand new workspace though. Thanks again for your support and your efforts on AnythingLLM!
Author
Owner

@timothycarambat commented on GitHub (Jun 4, 2024):

lets try deleting the Xenova folder and then try to re-embed a file on a fresh and empty workspace. Be sure to close and re-open app after deleting that folder. Please test a smaller text file just to limit variables in debugging. The embedder downloads on the first use so it does not bloat the app if unused. Its only 25MB so it should not take long, however if it still fails then there is a possibility the MSFT C++ Distributable might not be installed on your system - which is possible, but really unlikely but it has happened before.

https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170#latest-microsoft-visual-c-redistributable-version

@timothycarambat commented on GitHub (Jun 4, 2024): lets try deleting the `Xenova` folder and then try to re-embed a file on a fresh and empty workspace. Be sure to close and re-open app after deleting that folder. Please test a smaller text file just to limit variables in debugging. The embedder downloads on the first use so it does not bloat the app if unused. Its only 25MB so it should not take long, however if it still fails then there is a possibility the MSFT C++ Distributable might not be installed on your system - which _is_ possible, but really unlikely but it has happened before. https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170#latest-microsoft-visual-c-redistributable-version
Author
Owner

@LLMuser commented on GitHub (Jun 4, 2024):

lets try deleting the Xenova folder and then try to re-embed a file on a fresh and empty workspace. Be sure to close and re-open app after deleting that folder. Please test a smaller text file just to limit variables in debugging.

Did that. Same error.

however if it still fails then there is a possibility the MSFT C++ Distributable might not be installed on your system - which is possible, but really unlikely but it has happened before.

I had MS Visual C++ installed, reinstalled vc_redist.x64.exe though but error still occurs after deleting the "Xenova" and re-embedding in a fresh and empty workspace.

@LLMuser commented on GitHub (Jun 4, 2024): > lets try deleting the `Xenova` folder and then try to re-embed a file on a fresh and empty workspace. Be sure to close and re-open app after deleting that folder. Please test a smaller text file just to limit variables in debugging. Did that. Same error. > however if it still fails then there is a possibility the MSFT C++ Distributable might not be installed on your system - which _is_ possible, but really unlikely but it has happened before. I had MS Visual C++ installed, reinstalled vc_redist.x64.exe though but error still occurs after deleting the "Xenova" and re-embedding in a fresh and empty workspace.
Author
Owner

@LucBruz commented on GitHub (Jun 6, 2024):

Hey, i got the exact same error as LLMuser.

I don't get why the ticket have been closed since the error is still there

@LucBruz commented on GitHub (Jun 6, 2024): Hey, i got the exact same error as LLMuser. I don't get why the ticket have been closed since the error is still there
Author
Owner

@timothycarambat commented on GitHub (Jun 6, 2024):

Because this is not a bug and is a client-side specific issue. If the issue was a bug 100% of users would be impacted and that is not the case - so now it is a support ticket in Github and we prematurely close them as it crowds the repo.

@timothycarambat commented on GitHub (Jun 6, 2024): Because this is not a bug and is a client-side specific issue. If the issue was a bug 100% of users would be impacted and that is not the case - so now it is a support ticket in Github and we prematurely close them as it crowds the repo.
Author
Owner

@AppliactionsZealZone commented on GitHub (Jun 11, 2024):

Hey, i got the exact same error as LLMuser.

I don't get why the ticket have been closed since the error is still there
meto~~

@AppliactionsZealZone commented on GitHub (Jun 11, 2024): Hey, i got the exact same error as LLMuser. I don't get why the ticket have been closed since the error is still there meto~~
Author
Owner

@AppliactionsZealZone commented on GitHub (Jun 11, 2024):

Maybe my windows system computer installed too many environment variables led to this problem, how to troubleshoot become a very egg pain problem

@AppliactionsZealZone commented on GitHub (Jun 11, 2024): Maybe my windows system computer installed too many environment variables led to this problem, how to troubleshoot become a very egg pain problem
Author
Owner

@timothycarambat commented on GitHub (Jun 12, 2024):

Closed because it cannot be replicated. If anyone is willing to get any good logs you can do so: https://docs.useanything.com/debug

@timothycarambat commented on GitHub (Jun 12, 2024): Closed because it cannot be replicated. If anyone is willing to get any good logs you can do so: https://docs.useanything.com/debug
Author
Owner

@ojoaustral commented on GitHub (Aug 16, 2024):

Hi, I'm experiencing the same problem on an ASUS Vivobook with Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU, 3417 Mhz, 12 Core(s), 32GB RAM. Windows 11.

I have tried several different LLMs and the problem begins when saving and embedding a context document. I get the same error as described above and the software freezes.

@ojoaustral commented on GitHub (Aug 16, 2024): Hi, I'm experiencing the same problem on an ASUS Vivobook with Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU, 3417 Mhz, 12 Core(s), 32GB RAM. Windows 11. I have tried several different LLMs and the problem begins when saving and embedding a context document. I get the same error as described above and the software freezes.
Author
Owner

@timothycarambat commented on GitHub (Aug 16, 2024):

@ojoaustral That is a brand-new machine so should not be an issue with AVX2 support on that CPU. The LLM has no impact on the upload/embedding section. I am assuming you have both the default vectorDB and Embedder model selected in ANythingLLM?

@timothycarambat commented on GitHub (Aug 16, 2024): @ojoaustral That is a brand-new machine so should not be an issue with AVX2 support on that CPU. The LLM has no impact on the upload/embedding section. I am assuming you have both the default vectorDB and Embedder model selected in ANythingLLM?
Author
Owner

@ojoaustral commented on GitHub (Aug 16, 2024):

Hey, thanks!

I have LanceDB as the Vector DB, and AnythingLLM embedder. I don't see an
AnythingLMM vector database option.

C

On Fri, Aug 16, 2024 at 2:29 PM Timothy Carambat @.***>
wrote:

@ojoaustral https://github.com/ojoaustral That is a brand-new machine
so should not be an issue with AVX2 support on that CPU. The LLM has no
impact on the upload/embedding section. I am assuming you have both the
default vectorDB and Embedder model selected in ANythingLLM?


Reply to this email directly, view it on GitHub
https://github.com/Mintplex-Labs/anything-llm/issues/1586#issuecomment-2293968213,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKLXMVRNXD3OSQSXIVLZHIDZRZAJHAVCNFSM6AAAAABISY4TZ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJTHE3DQMRRGM
.
You are receiving this because you were mentioned.Message ID:
@.***>

@ojoaustral commented on GitHub (Aug 16, 2024): Hey, thanks! I have LanceDB as the Vector DB, and AnythingLLM embedder. I don't see an AnythingLMM vector database option. C On Fri, Aug 16, 2024 at 2:29 PM Timothy Carambat ***@***.***> wrote: > @ojoaustral <https://github.com/ojoaustral> That is a brand-new machine > so should not be an issue with AVX2 support on that CPU. The LLM has no > impact on the upload/embedding section. I am assuming you have both the > default vectorDB and Embedder model selected in ANythingLLM? > > — > Reply to this email directly, view it on GitHub > <https://github.com/Mintplex-Labs/anything-llm/issues/1586#issuecomment-2293968213>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AKLXMVRNXD3OSQSXIVLZHIDZRZAJHAVCNFSM6AAAAABISY4TZ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJTHE3DQMRRGM> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >
Author
Owner

@FlashGorshkov commented on GitHub (Aug 17, 2024):

I have the same error. MacOS 14.6 (23G80) Sonoma. Llama 3.1 70B.

@FlashGorshkov commented on GitHub (Aug 17, 2024): I have the same error. MacOS 14.6 (23G80) Sonoma. Llama 3.1 70B.
Author
Owner

@ojoaustral commented on GitHub (Aug 19, 2024):

Hi Tim,

Yes, I have the default vectorDB (LanceDB) and Embedder (AnythingLLM). The
same problem also happens when I try scraping a webpage. I checked the
Windows firewall and AnythingLLM.exe is enabled. Do you have any idea what
is going on? What can I try now?

Best,

C

On Fri, Aug 16, 2024 at 2:29 PM Timothy Carambat @.***>
wrote:

@ojoaustral https://github.com/ojoaustral That is a brand-new machine
so should not be an issue with AVX2 support on that CPU. The LLM has no
impact on the upload/embedding section. I am assuming you have both the
default vectorDB and Embedder model selected in ANythingLLM?


Reply to this email directly, view it on GitHub
https://github.com/Mintplex-Labs/anything-llm/issues/1586#issuecomment-2293968213,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKLXMVRNXD3OSQSXIVLZHIDZRZAJHAVCNFSM6AAAAABISY4TZ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJTHE3DQMRRGM
.
You are receiving this because you were mentioned.Message ID:
@.***>

@ojoaustral commented on GitHub (Aug 19, 2024): Hi Tim, Yes, I have the default vectorDB (LanceDB) and Embedder (AnythingLLM). The same problem also happens when I try scraping a webpage. I checked the Windows firewall and AnythingLLM.exe is enabled. Do you have any idea what is going on? What can I try now? Best, C On Fri, Aug 16, 2024 at 2:29 PM Timothy Carambat ***@***.***> wrote: > @ojoaustral <https://github.com/ojoaustral> That is a brand-new machine > so should not be an issue with AVX2 support on that CPU. The LLM has no > impact on the upload/embedding section. I am assuming you have both the > default vectorDB and Embedder model selected in ANythingLLM? > > — > Reply to this email directly, view it on GitHub > <https://github.com/Mintplex-Labs/anything-llm/issues/1586#issuecomment-2293968213>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AKLXMVRNXD3OSQSXIVLZHIDZRZAJHAVCNFSM6AAAAABISY4TZ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJTHE3DQMRRGM> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >
Author
Owner

@timothycarambat commented on GitHub (Aug 19, 2024):

Nothing useful from https://docs.useanything.com/debug?

@timothycarambat commented on GitHub (Aug 19, 2024): Nothing useful from https://docs.useanything.com/debug?
Author
Owner

@FlowDownTheRiver commented on GitHub (Sep 11, 2024):

windows 10 64 bit log.txt
Using desktop app. I did everything this page suggested. I have run the program in cmd and got this log. when using a file in workspace.
it shows :
Could not respond to message.
An error occurred while streaming response. network error

in web browser.

it force closes the program in cli,so the client no longer works. That is why nothing works after we try to use a file for the embedder.

lets try deleting the Xenova folder and then try to re-embed a file on a fresh and empty workspace. Be sure to close and re-open app after deleting that folder. Please test a smaller text file just to limit variables in debugging. The embedder downloads on the first use so it does not bloat the app if unused. Its only 25MB so it should not take long, however if it still fails then there is a possibility the MSFT C++ Distributable might not be installed on your system - which is possible, but really unlikely but it has happened before.

https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170#latest-microsoft-visual-c-redistributable-version
I also have done this .

Next, I would open C:\Users\user\AppData\Roaming\anythingllm-desktop\storage and see if there is a folder with files in it named storage\models\Xenova\ and there should be a subfolder for all-miniLM-l6 or something to the effect.

and deleted this as well.

Something is force closing the program when an embedding is used.

my cpu is avx intel i7 3930k, 1080 ti (11 gb vram) ,latest nvidia driver.
windows 10 , 22h2 build 19045.3758.

@timothycarambat if you need me to do some more stuff,let me know..

@FlowDownTheRiver commented on GitHub (Sep 11, 2024): [windows 10 64 bit log.txt](https://github.com/user-attachments/files/16965581/windows.10.64.bit.log.txt) Using desktop app. I did everything this page suggested. I have run the program in cmd and got this log. when using a file in workspace. it shows : **Could not respond to message. An error occurred while streaming response. network error** in web browser. it force closes the program in cli,so the client no longer works. That is why nothing works after we try to use a file for the embedder. > lets try deleting the `Xenova` folder and then try to re-embed a file on a fresh and empty workspace. Be sure to close and re-open app after deleting that folder. Please test a smaller text file just to limit variables in debugging. The embedder downloads on the first use so it does not bloat the app if unused. Its only 25MB so it should not take long, however if it still fails then there is a possibility the MSFT C++ Distributable might not be installed on your system - which _is_ possible, but really unlikely but it has happened before. > > https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170#latest-microsoft-visual-c-redistributable-version I also have done this . > Next, I would open `C:\Users\user\AppData\Roaming\anythingllm-desktop\storage` and see if there is a folder with files in it named `storage\models\Xenova\` and there should be a subfolder for `all-miniLM-l6` or something to the effect. and deleted this as well. Something is force closing the program when an embedding is used. my cpu is avx intel i7 3930k, 1080 ti (11 gb vram) ,latest nvidia driver. windows 10 , 22h2 build 19045.3758. @timothycarambat if you need me to do some more stuff,let me know..
Author
Owner

@PascalLaube commented on GitHub (Sep 24, 2024):

For me the Error went away as I started Anything LLM with administrative rights.

@PascalLaube commented on GitHub (Sep 24, 2024): For me the Error went away as I started Anything LLM with administrative rights.
Author
Owner

@FlowDownTheRiver commented on GitHub (Sep 24, 2024):

For me the Error went away as I started Anything LLM with administrative rights.

Unfortunately it didn't help. However I have installed chromadb ,and hosted chroma server locally. then when I choose chroma inside anything llm and put the localhost ip address it worked. At least this way I can use RAG.

the downside is I have to start my chroma server outside of anything llm. So I made a bat file which call chroma server and anything llm.

example main bat file :

@echo off
rem First command
start cmd /c call "Run_chroma_server.bat"

rem Second command runs in a new cmd instance simultaneously
start cmd /c call "C:\Users\USERNAME\AppData\Local\Programs\AnythingLLM\AnythingLLM.exe"

and :
Run_chroma_server.bat :

chroma run --path my_database

here {my_database} is the folder where you store your vector. it can be any folder you create.

I have tested the internal lancedb option in a friend's pc vs chromadb .In comparison I found that chroma worked faster as speedwise.

you can try chromadb with : pip install chromadb

inside anything llm by default chroma db runs on this local host : http://localhost:8000 , fill that in and you are good to go.

I have written this in case anybody wants to test.

@FlowDownTheRiver commented on GitHub (Sep 24, 2024): > For me the Error went away as I started Anything LLM with administrative rights. Unfortunately it didn't help. However I have installed chromadb ,and hosted chroma server locally. then when I choose chroma inside anything llm and put the localhost ip address it worked. At least this way I can use RAG. the downside is I have to start my chroma server outside of anything llm. So I made a bat file which call chroma server and anything llm. example main bat file : @echo off rem First command start cmd /c call "Run_chroma_server.bat" rem Second command runs in a new cmd instance simultaneously start cmd /c call "C:\Users\USERNAME\AppData\Local\Programs\AnythingLLM\AnythingLLM.exe" --------------------------- and : Run_chroma_server.bat : chroma run --path my_database here {my_database} is the folder where you store your vector. it can be any folder you create. I have tested the internal lancedb option in a friend's pc vs chromadb .In comparison I found that chroma worked faster as speedwise. you can try chromadb with : pip install chromadb inside anything llm by default chroma db runs on this local host : http://localhost:8000 , fill that in and you are good to go. I have written this in case anybody wants to test.
Author
Owner

@FlowDownTheRiver commented on GitHub (Sep 24, 2024):

@timothycarambat Is there a way for windows client when chroma database is selected it shall install chromadb internally and start its server internally and locally ,so we don't have to depend on external installation of chroma and start it outside anything llm or can you pack chromadb for the installation(windows at least). Why ,cos it seems to work on even with avx supported gpus. This can end all the problems for old machines . Please :)

@FlowDownTheRiver commented on GitHub (Sep 24, 2024): @timothycarambat Is there a way for windows client when chroma database is selected it shall install chromadb internally and start its server internally and locally ,so we don't have to depend on external installation of chroma and start it outside anything llm or can you pack chromadb for the installation(windows at least). Why ,cos it seems to work on even with avx supported gpus. This can end all the problems for old machines . Please :)
Author
Owner

@timothycarambat commented on GitHub (Sep 24, 2024):

@FlowDownTheRiver I wish it was that simple. IIRC Chroma was going to explore embedded Chroma, but the AVX2 CPU support is really kind of helpful because AVX2 support has been around for over ten years! No doubt whatever we get up to in the future will benefit from running a somewhat modern machine.

For now, it will have to be external, its not as easy as installing chroma automatically and that's a lot of complexity to support machine that are frankly a bit out of scope for us

@timothycarambat commented on GitHub (Sep 24, 2024): @FlowDownTheRiver I wish it was that simple. IIRC Chroma was going to explore embedded Chroma, but the AVX2 CPU support is really kind of helpful because AVX2 support has been around for over ten years! No doubt whatever we get up to in the future will benefit from running a somewhat modern machine. For now, it will have to be external, its not as easy as installing chroma automatically and that's a lot of complexity to support machine that are frankly a bit out of scope for us
Author
Owner

@FlowDownTheRiver commented on GitHub (Sep 24, 2024):

@timothycarambat Thank you for your time and explaining. This is a great project regardless. I can work with external server since it is at least working for my needs. Thanks for this amazing project to you,to the team and to contributers as well.

@FlowDownTheRiver commented on GitHub (Sep 24, 2024): @timothycarambat Thank you for your time and explaining. This is a great project regardless. I can work with external server since it is at least working for my needs. Thanks for this amazing project to you,to the team and to contributers as well.
Author
Owner

@yetkind commented on GitHub (Sep 27, 2024):

I have the same error

Could not respond to message.
An error occurred while streaming response. network error
@yetkind commented on GitHub (Sep 27, 2024): I have the same error ``` Could not respond to message. An error occurred while streaming response. network error ```
Author
Owner

@timothycarambat commented on GitHub (Sep 27, 2024):

https://docs.anythingllm.com/fetch-failed-on-upload

@timothycarambat commented on GitHub (Sep 27, 2024): https://docs.anythingllm.com/fetch-failed-on-upload
yindo changed title from [BUG]: Error after embedding document ""Could not respond to message" to [GH-ISSUE #1586] [BUG]: Error after embedding document ""Could not respond to message" 2026-06-05 14:38:35 -04:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1029