Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1161] [BUG]: Ollama fetch failed when chatting with docs #715

New Issue
Closed
opened 2026-02-22 18:20:58 -05:00 by yindo · 11 comments
yindo commented 2026-02-22 18:20:58 -05:00
Owner
Copy Link
Copy Source

Originally created by @danielrosly on GitHub (Apr 21, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1161

How are you running AnythingLLM?

AnythingLLM desktop app in Windows 10 x64.

What happened?

The procedure that was reproduced is shown here: https://www.youtube.com/watch?v=IJYC6zf86lU&list=WL&index=3&t=331s.
It was done using the newest versions of Ollama and AnythingLLM (1.5.0). Llama2:7b was used as the base model (although the problem occurred in the same form for Llama3), and all other settings were default. The embedded URL was https://useanything.com. Fetching and embedding data was executed without problem.

I encountered an error shown in the picture below when trying to chat with the model. For a fraction of second, error message contains 'network error' is printed before it changes into 'Failed to fetch'.

This error rendered AnythingLLM unusable. The UI responded erratically, i.e. subsequent attempts to send a message resulted in a blinking fragment of the screen related to the message and the response.

After that the tool had to be restarted. After the restart, the conversation that caused these errors was not present. But if one removes fetched documents from workspace, this workspace can be used to chat without an error.

2024-04-21_11h57_18

Before all the above tests, when I had used the workspace Test1, which didn't contain any embedded data, I managed to use the model without any problems.

Regarding relation with #821 and #946.

I am quite sure that it is related somehow, because not full contents from HuggingFace has been downloaded.
But also this page either shows misleading information, or there is really something wrong with the configuration:
2024-04-21_17h16_08
This directory structure is located at c:\Users\********\AppData\Roaming\anythingllm-desktop\storage\models\Xenova\all-MiniLM-L6-v2
As it is visible, to the contrary of https://github.com/Mintplex-Labs/anything-llm/issues/946 and https://github.com/Mintplex-Labs/anything-llm/issues/821 model has been downloaded. I did download model manually from the same IP as where my tools are working, so this is NOT a problem with blocked IP as described in #946. Perhaps your tool tries to download model too aggresively and what is why the connections are broken.
In the end, if this in picture above is correct, then text shown in italic in blue area should disappear or be changed.

Are there known steps to reproduce?

No response

Originally created by @danielrosly on GitHub (Apr 21, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1161 ### How are you running AnythingLLM? AnythingLLM desktop app in Windows 10 x64. ### What happened? The procedure that was reproduced is shown here: [https://www.youtube.com/watch?v=IJYC6zf86lU&list=WL&index=3&t=331s](https://www.youtube.com/watch?v=IJYC6zf86lU&list=WL&index=3&t=331s). It was done using the newest versions of Ollama and AnythingLLM (1.5.0). Llama2:7b was used as the base model (although the problem occurred in the same form for Llama3), and all other settings were default. The embedded URL was [https://useanything.com](https://useanything.com). Fetching and embedding data was executed without problem. I encountered an error shown in the picture below when trying to chat with the model. For a fraction of second, error message contains 'network error' is printed before it changes into 'Failed to fetch'. This error rendered AnythingLLM unusable. The UI responded erratically, i.e. subsequent attempts to send a message resulted in a blinking fragment of the screen related to the message and the response. After that the tool had to be restarted. After the restart, the conversation that caused these errors was not present. But if one removes fetched documents from workspace, this workspace can be used to chat without an error. ![2024-04-21_11h57_18](https://github.com/Mintplex-Labs/anything-llm/assets/137312125/bd043921-0f93-4734-9c66-55db2c6f430f) Before all the above tests, when I had used the workspace `Test1`, which didn't contain any embedded data, I managed to use the model without any problems. **Regarding relation with #821 and #946.** I am quite sure that it is related somehow, because not full contents from HuggingFace has been downloaded. But also this page either shows misleading information, or there is really something wrong with the configuration: ![2024-04-21_17h16_08](https://github.com/Mintplex-Labs/anything-llm/assets/137312125/1fcb58bd-02e5-44c5-98b1-7aa0de454cdd) This directory structure is located at `c:\Users\********\AppData\Roaming\anythingllm-desktop\storage\models\Xenova\all-MiniLM-L6-v2` As it is visible, to the contrary of https://github.com/Mintplex-Labs/anything-llm/issues/946 and https://github.com/Mintplex-Labs/anything-llm/issues/821 model has been downloaded. I did download model manually from the same IP as where my tools are working, so this is NOT a problem with blocked IP as described in #946. Perhaps your tool tries to download model too aggresively and what is why the connections are broken. In the end, if this in picture above is correct, then text shown in italic in blue area should disappear or be changed. ### Are there known steps to reproduce? _No response_
yindo added the needs info / can't replicate label 2026-02-22 18:20:58 -05:00
yindo closed this issue 2026-02-22 18:20:58 -05:00
yindo commented 2026-02-22 18:20:58 -05:00
Author
Owner
Copy Link
Copy Source

@danielrosly commented on GitHub (Apr 21, 2024):

Update.
I downloaded and built the newest version from master.
It works with embedded data when ran in development mode. Downloads proper data structure as below:
2024-04-21_18h40_24

Unfortunately, puting just the same data (binary identical) to c:\Users\********\AppData\Roaming\anythingllm-desktop\storage\models\Xenova\all-MiniLM-L6-v2 doesn't make 1.5.0 distribution work.

So, it is not problem with downloading, because replacing model data didn't help.

@danielrosly commented on GitHub (Apr 21, 2024): Update. I downloaded and built the newest version from `master`. It works with embedded data when ran in development mode. Downloads proper data structure as below: ![2024-04-21_18h40_24](https://github.com/Mintplex-Labs/anything-llm/assets/137312125/8921ea08-bf2f-4517-85b2-268162b9bbb3) Unfortunately, puting just the same data (binary identical) to `c:\Users\********\AppData\Roaming\anythingllm-desktop\storage\models\Xenova\all-MiniLM-L6-v2` doesn't make 1.5.0 distribution work. So, it is not problem with downloading, because replacing model data didn't help.
yindo commented 2026-02-22 18:20:59 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 21, 2024):

Did you, at any point, change your embedding model after embedding documents? Unable to replicate currently. Also this fetch failed example is exactly what happens when "ollama serve" is not running and try to send a chat and/or the URL is wrong (using localhost vs 127.0.0.1)

@timothycarambat commented on GitHub (Apr 21, 2024): Did you, at any point, change your embedding model after embedding documents? Unable to replicate currently. Also this fetch failed example is exactly what happens when "ollama serve" is not running and try to send a chat and/or the URL is wrong (using localhost vs 127.0.0.1)
yindo commented 2026-02-22 18:20:59 -05:00
Author
Owner
Copy Link
Copy Source

@CorbenYkt commented on GitHub (Apr 21, 2024):

Hi everyone
The same problem :(
Even adding one document gives me a error

@CorbenYkt commented on GitHub (Apr 21, 2024): Hi everyone The same problem :( Even adding one document gives me a error
yindo commented 2026-02-22 18:20:59 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 21, 2024):

@CorbenYkt are you positive Ollama is running & the connection information you gave AnythingLLM is the same? If you can see models in the dropdown for the LLM preference, it is correct. Additionally, what settings did you provide for the Context Window during the LLM preference?

Lastly, what model are you using with ollama

@timothycarambat commented on GitHub (Apr 21, 2024): @CorbenYkt are you positive Ollama is running & the connection information you gave AnythingLLM is the same? If you can see models in the dropdown for the LLM preference, it is correct. Additionally, what settings did you provide for the `Context Window` during the LLM preference? Lastly, what model are you using with ollama
yindo commented 2026-02-22 18:20:59 -05:00
Author
Owner
Copy Link
Copy Source

@CorbenYkt commented on GitHub (Apr 22, 2024):

Hello @timothycarambat
Here is my settings
image
image
image

VerctorDB is LanceDB
And I'm loading only one PDF for testing.
image
After adding PDF any promts are gives this error:
image

@CorbenYkt commented on GitHub (Apr 22, 2024): Hello @timothycarambat Here is my settings ![image](https://github.com/Mintplex-Labs/anything-llm/assets/117908636/b8dce668-44c1-42b1-826a-f7a165268737) ![image](https://github.com/Mintplex-Labs/anything-llm/assets/117908636/54baa411-285d-4470-9155-b90eee9d4157) ![image](https://github.com/Mintplex-Labs/anything-llm/assets/117908636/5101179f-1089-4ac3-b07c-52510826085d) VerctorDB is LanceDB And I'm loading only one PDF for testing. ![image](https://github.com/Mintplex-Labs/anything-llm/assets/117908636/30c719b4-57f5-4520-b975-3c1f770d67fa) After adding PDF any promts are gives this error: ![image](https://github.com/Mintplex-Labs/anything-llm/assets/117908636/5c14ab07-aa3c-4a59-9155-b385a4144504)
yindo commented 2026-02-22 18:20:59 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 22, 2024):

@CorbenYkt
The issue here is that you have llama3:latest as your embedding provider. LLama3 is an LLM, not an embedder. Change your embedder model back to native (which is probably what you embed that document with) and it will work.

@timothycarambat commented on GitHub (Apr 22, 2024): @CorbenYkt The issue here is that you have `llama3:latest` as your **embedding** provider. LLama3 is an LLM, not an embedder. Change your embedder model back to native (which is probably what you embed that document with) and it will work.
yindo commented 2026-02-22 18:20:59 -05:00
Author
Owner
Copy Link
Copy Source

@CorbenYkt commented on GitHub (Apr 22, 2024):

@CorbenYkt The issue here is that you have llama3:latest as your embedding provider. LLama3 is an LLM, not an embedder. Change your embedder model back to native (which is probably what you embed that document with) and it will work.

Ohh... that is so simple)) Thx!

@CorbenYkt commented on GitHub (Apr 22, 2024): > @CorbenYkt The issue here is that you have `llama3:latest` as your **embedding** provider. LLama3 is an LLM, not an embedder. Change your embedder model back to native (which is probably what you embed that document with) and it will work. Ohh... that is so simple)) Thx!
yindo commented 2026-02-22 18:20:59 -05:00
Author
Owner
Copy Link
Copy Source

@danielrosly commented on GitHub (Apr 27, 2024):

Unfortunately, this above is not my case.

I did following to try to locate the reason:

  1. Built app from souces in my local machine (where error occurs) and ran it in debug mode. Worked. I did that from WSL. Pictures are attached below.
  2. Installed and used desktop application 1.5.0 with Ollama in my laptop. Also worked (error didn't occur). Installation and configuration were identical to these presented by me at the beginning of this issue.
  3. Removed completly desktop app 1.5.0 from machine where error occurs and installed 1.5.1 Model was downloaded properly (I it compared to p.1). Errror the same as reported in the beginnig of this issue.

I am starting to think that my configuration (especially processor) might be TOO OLD.
Machine that causes problem is based on CPU XEON E5-2697v2 with 64GB RAM and GTX1080. Win10.
I.e. the processor doesn't have AVX2 instruction set.

Could that be ?

Two other questions:

  • Is there any way to get more meaningful logs from AnythingLLM Windows desktop application ?
  • Is there any description how to build desktop applicaion for Windows out of sources ?

Local dev run (as recommended by yarn setup):
2024-04-27_17h55_06
2024-04-27_17h55_41

@danielrosly commented on GitHub (Apr 27, 2024): Unfortunately, this above is not my case. I did following to try to locate the reason: 1. Built app from souces in my local machine (where error occurs) and ran it in debug mode. Worked. I did that from WSL. Pictures are attached below. 2. Installed and used desktop application 1.5.0 with Ollama in my laptop. Also worked (error didn't occur). Installation and configuration were identical to these presented by me at the beginning of this issue. 3. Removed completly desktop app 1.5.0 from machine where error occurs and installed 1.5.1 Model was downloaded properly (I it compared to p.1). Errror the same as reported in the beginnig of this issue. **I am starting to think that my configuration (especially processor) might be TOO OLD. Machine that causes problem is based on CPU XEON E5-2697v2 with 64GB RAM and GTX1080. Win10. I.e. the processor doesn't have AVX2 instruction set.** Could that be ? Two other questions: - Is there any way to get more meaningful logs from AnythingLLM Windows desktop application ? - Is there any description how to build desktop applicaion for Windows out of sources ? Local dev run (as recommended by `yarn setup`): ![2024-04-27_17h55_06](https://github.com/Mintplex-Labs/anything-llm/assets/137312125/d1e37826-d27e-45fc-ae5d-5f652717f554) ![2024-04-27_17h55_41](https://github.com/Mintplex-Labs/anything-llm/assets/137312125/ce55a18b-b0d9-4ef3-98d0-36a66c6964db)
yindo commented 2026-02-22 18:20:59 -05:00
Author
Owner
Copy Link
Copy Source

@stanribilir commented on GitHub (Apr 28, 2024):

@CorbenYkt The issue here is that you have llama3:latest as your embedding provider. LLama3 is an LLM, not an embedder. Change your embedder model back to native (which is probably what you embed that document with) and it will work.

can i use nomic embedder from ollama though with above approach?

@stanribilir commented on GitHub (Apr 28, 2024): > @CorbenYkt The issue here is that you have `llama3:latest` as your **embedding** provider. LLama3 is an LLM, not an embedder. Change your embedder model back to native (which is probably what you embed that document with) and it will work. can i use nomic embedder from ollama though with above approach?
yindo commented 2026-02-22 18:20:59 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 28, 2024):

Yes, nomic is an embedding model. So will be no issue if you select that for embedding. Just dont also set nomic as your LLM because then you'll still have the same problem - just reverse :)

@timothycarambat commented on GitHub (Apr 28, 2024): Yes, `nomic` is an embedding model. So will be no issue if you select that for embedding. Just dont also set `nomic` as your LLM because then you'll still have the same problem - just reverse :)
yindo commented 2026-02-22 18:20:59 -05:00
Author
Owner
Copy Link
Copy Source

@danielrosly commented on GitHub (Jun 2, 2024):

Summary of my findings regarding my first comment in this issue.
The problem described there occurs in this location: https://github.com/Mintplex-Labs/anything-llm/blob/9a38b32c746b790531975d27888fea5552050062/server/utils/vectorDbProviders/lance/index.js#L74

This is call to searching in LanceDB.

It is somehow related with both data within AnythingLLM and method .metricType("cosine") When I debugged above call with different (test) data and/or without explicitly calling method .metricType("cosine") - the error didn't occur.

The error also does not occur, when vector database is changed to ChromaDB.

So, my conclustion stated somewhere above:
I am starting to think that my configuration (especially processor) might be TOO OLD.
Machine that causes problem is based on CPU XEON E5-2697v2 with 64GB RAM and GTX1080. Win10.
I.e. the processor doesn't have AVX2 instruction set.
seems to be probable - but not for AnythingLLM, but for LanceDB.

@danielrosly commented on GitHub (Jun 2, 2024): Summary of my findings regarding my first comment in this issue. The problem described there occurs in this location: https://github.com/Mintplex-Labs/anything-llm/blob/9a38b32c746b790531975d27888fea5552050062/server/utils/vectorDbProviders/lance/index.js#L74 This is call to searching in LanceDB. It is somehow related with both data within AnythingLLM and method `.metricType("cosine")` When I debugged above call with different (test) data and/or without explicitly calling method `.metricType("cosine")` - the error didn't occur. The error also does not occur, when vector database is changed to ChromaDB. So, my conclustion stated somewhere above: _I am starting to think that my configuration (especially processor) might be TOO OLD. Machine that causes problem is based on CPU XEON E5-2697v2 with 64GB RAM and GTX1080. Win10. I.e. the processor doesn't have AVX2 instruction set._ seems to be probable - but not for AnythingLLM, **but for LanceDB**.
yindo changed title from [BUG]: Ollama fetch failed when chatting with docs to [GH-ISSUE #1161] [BUG]: Ollama fetch failed when chatting with docs 2026-06-05 14:36:49 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label needs info / can't replicate
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#715
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?