Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1355] Ollama embeddings "crash" when embedding a repo #857

New Issue
Closed
opened 2026-02-22 18:21:49 -05:00 by yindo · 9 comments
yindo commented 2026-02-22 18:21:49 -05:00
Owner
Copy Link
Copy Source

Originally created by @Freffles on GitHub (May 10, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1355

Using Ollama embeddings nomic-embed-text on a github repo, embeddings stop after about 5 minutes. Anything LLM briefly displays an error message about 108 documents failed to load (there are 120 in the repo) then the message disappears. I managed to capture a screenshot (attached). There is no information available in the "event logs" within Anything LLM as theses appear to only deal with workspace documents added or removed. I have not been able to locate any other Anything LLM log to give any other information.

The vectorDC is LanceDB.

This has happened three times now with Anything LLM. FYI, the Ollama server log is attached.

I have successfully (but slowly) embedded this repo several times into a local FAISS DB using my own code.

image
ollama_logs.txt

Originally created by @Freffles on GitHub (May 10, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1355 Using Ollama embeddings nomic-embed-text on a github repo, embeddings stop after about 5 minutes. Anything LLM briefly displays an error message about 108 documents failed to load (there are 120 in the repo) then the message disappears. I managed to capture a screenshot (attached). There is no information available in the "event logs" within Anything LLM as theses appear to only deal with workspace documents added or removed. I have not been able to locate any other Anything LLM log to give any other information. The vectorDC is LanceDB. This has happened three times now with Anything LLM. FYI, the Ollama server log is attached. I have successfully (but slowly) embedded this repo several times into a local FAISS DB using my own code. ![image](https://github.com/Mintplex-Labs/anything-llm/assets/122719748/89158ab4-aaae-44c7-bdb3-8c1199585e0c) [ollama_logs.txt](https://github.com/Mintplex-Labs/anything-llm/files/15281162/ollama_logs.txt)
yindo closed this issue 2026-02-22 18:21:49 -05:00
yindo commented 2026-02-22 18:21:49 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 11, 2024):

A couple questions to help figure out where this is coming from

  • Did this happen on a workspace that had ever had anything embedded in it prior
  • were any of the documents to be embedded, already cached
  • did you ever use any embedder other than the ollama embedder with nomic?
  • Does this problem persist with a single document, or only when uploaded 100+?
@timothycarambat commented on GitHub (May 11, 2024): A couple questions to help figure out where this is coming from - Did this happen on a workspace that had ever had anything embedded in it prior - were any of the documents to be embedded, already cached - did you ever use any embedder other than the ollama embedder with nomic? - Does this problem persist with a single document, or only when uploaded 100+?
yindo commented 2026-02-22 18:21:49 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 11, 2024):

I do see this in the logs

time=2024-05-11T11:23:05.819+10:00 level=INFO source=routes.go:405 msg="embedding generation failed: no slots available after 10 retries"

This would indicate that a 0 length vector could be being returned due to a failure. That could be the issue and LanceDB is just say "i wont embed a 0 length vector when the dimensions should be 768"

@timothycarambat commented on GitHub (May 11, 2024): I do see this in the logs > time=2024-05-11T11:23:05.819+10:00 level=INFO source=routes.go:405 msg="embedding generation failed: no slots available after 10 retries" This would indicate that a 0 length vector could be being returned due to a failure. That could be the issue and LanceDB is just say "i wont embed a 0 length vector when the dimensions should be 768"
yindo commented 2026-02-22 18:21:50 -05:00
Author
Owner
Copy Link
Copy Source

@Freffles commented on GitHub (May 11, 2024):

What would be a zero length vector? In regards to you questions:

I had the error on two workspaces. The first may have had embeddings with the default embedded but I deleted everything and started again. The most recent attempt was a clean workspace.

In terms of caching, I don't know. What I do know is that I retrieved the repo multiple times.

I did manage, I think, to embed the same repo with the default embeddings model.

I did get ollama embeddings to work with a much smaller repo that had only 3 files.

@Freffles commented on GitHub (May 11, 2024): What would be a zero length vector? In regards to you questions: I had the error on two workspaces. The first may have had embeddings with the default embedded but I deleted everything and started again. The most recent attempt was a clean workspace. In terms of caching, I don't know. What I do know is that I retrieved the repo multiple times. I did manage, I think, to embed the same repo with the default embeddings model. I did get ollama embeddings to work with a much smaller repo that had only 3 files.
yindo commented 2026-02-22 18:21:50 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 11, 2024):

The error from ollama also seems to indicate the ollama ran out of memory and started to fail to return embeddings. This would make sense because when we sent 108 files at once to ollama it probably crashed.

Going to close for now, as I think this might just be a usage issue since it does till work at lower volumes?

@timothycarambat commented on GitHub (May 11, 2024): The error from ollama also seems to indicate the ollama ran out of memory and started to fail to return embeddings. This would make sense because when we sent 108 files at once to ollama it probably crashed. Going to close for now, as I think this might just be a usage issue since it does till work at lower volumes?
yindo commented 2026-02-22 18:21:50 -05:00
Author
Owner
Copy Link
Copy Source

@Freffles commented on GitHub (May 11, 2024):

Ok but ollama embeds this without failure outside of Anything LLM.

@Freffles commented on GitHub (May 11, 2024): Ok but ollama embeds this without failure outside of Anything LLM.
yindo commented 2026-02-22 18:21:50 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 11, 2024):

How are you doing this outside of AnythingLLM? sending each document one at a time in order?

@timothycarambat commented on GitHub (May 11, 2024): How are you doing this outside of AnythingLLM? sending each document one at a time in order?
yindo commented 2026-02-22 18:21:50 -05:00
Author
Owner
Copy Link
Copy Source

@Freffles commented on GitHub (May 12, 2024):

FYI, also failed the same way with Chroma inside Anything LLM.

The attached dogs breakfast (Lang_main.py) is how I've done it with the same repo outside of Anything LLM with FAISS. Have also successfully (and very slowly) processed the much larger Autogen repo (~800 docs) the same way. The settings in my config file are pretty much boilerplate as I'm having trouble with speed of the embeddings (~20 minutes to embed the repo I'm trying on Anything LLM) so there is room for improvement.

config.txt

Lang_main.txt

@Freffles commented on GitHub (May 12, 2024): FYI, also failed the same way with Chroma inside Anything LLM. The attached dogs breakfast (Lang_main.py) is how I've done it with the same repo outside of Anything LLM with FAISS. Have also successfully (and very slowly) processed the much larger Autogen repo (~800 docs) the same way. The settings in my config file are pretty much boilerplate as I'm having trouble with speed of the embeddings (~20 minutes to embed the repo I'm trying on Anything LLM) so there is room for improvement. [config.txt](https://github.com/Mintplex-Labs/anything-llm/files/15285498/config.txt) [Lang_main.txt](https://github.com/Mintplex-Labs/anything-llm/files/15285478/Lang_main.txt)
yindo commented 2026-02-22 18:21:50 -05:00
Author
Owner
Copy Link
Copy Source

@LindsayRex commented on GitHub (Jun 6, 2024):

The error from ollama also seems to indicate the ollama ran out of memory and started to fail to return embeddings. This would make sense because when we sent 108 files at once to ollama it probably crashed.

Going to close for now, as I think this might just be a usage issue since it does till work at lower volumes?

I am gettign similar results.. if i send more than 5 files at a time to embeed.. i get a similar issue with an intel PC & Nvidia card, Pc is only 2 year sold with a nift new nvidia 4070 super in it.. The computer reboots like the "reset" switch was hit.

@LindsayRex commented on GitHub (Jun 6, 2024): > The error from ollama also seems to indicate the ollama ran out of memory and started to fail to return embeddings. This would make sense because when we sent 108 files at once to ollama it probably crashed. > > Going to close for now, as I think this might just be a usage issue since it does till work at lower volumes? I am gettign similar results.. if i send more than 5 files at a time to embeed.. i get a similar issue with an intel PC & Nvidia card, Pc is only 2 year sold with a nift new nvidia 4070 super in it.. The computer reboots like the "reset" switch was hit.
yindo commented 2026-02-22 18:21:50 -05:00
Author
Owner
Copy Link
Copy Source

@Freffles commented on GitHub (Jun 7, 2024):

The idea that somehow documents are NOT sent in sequence is baffling. I have not pulled this string all the way yet. I suspect there may be some "documents" in that repo that don't work with the one size fits all splitting and chunking that would be used by Anything LLM. I will note the following for now:

Importing a GitHub repo includes a lot of crap you don't need. In anything LLM you can "exclude" files and folders. When I looked at what was in the repo and excluded stuff I didn't need (reducing the file count to 68), it worked. I don't think the issue is the number of files, that's just the easiest thing to blame. As Anything LLM users, there are a lot of things we can't control and don't have visibility of. When I tried the entire repo of 112 files: Fail. When I eliminated files that I didn't need and embedded 68 files: Pass. Outside of Anything LLM embedding the ENTIRE repo works without the need to remove any files.

Ollama embeddings, using nomic-embed-text, are faster inside Anything LLM as compared to outside for the exact same set of documents. I have not looked into this for a while but I do intend to come back to this and run some experiments to gather some more information. There was, and maybe still is, an issue with Ollama not using GPU (or enough GPU) when doing the embeddings.

@Freffles commented on GitHub (Jun 7, 2024): The idea that somehow documents are NOT sent in sequence is baffling. I have not pulled this string all the way yet. I suspect there may be some "documents" in that repo that don't work with the one size fits all splitting and chunking that would be used by Anything LLM. I will note the following for now: Importing a GitHub repo includes a lot of crap you don't need. In anything LLM you can "exclude" files and folders. When I looked at what was in the repo and excluded stuff I didn't need (reducing the file count to 68), it worked. I don't think the issue is the number of files, that's just the easiest thing to blame. As Anything LLM users, there are a lot of things we can't control and don't have visibility of. When I tried the entire repo of 112 files: Fail. When I eliminated files that I didn't need and embedded 68 files: Pass. Outside of Anything LLM embedding the ENTIRE repo works without the need to remove any files. Ollama embeddings, using nomic-embed-text, are faster inside Anything LLM as compared to outside for the exact same set of documents. I have not looked into this for a while but I do intend to come back to this and run some experiments to gather some more information. There was, and maybe still is, an issue with Ollama not using GPU (or enough GPU) when doing the embeddings.
yindo changed title from Ollama embeddings "crash" when embedding a repo to [GH-ISSUE #1355] Ollama embeddings "crash" when embedding a repo 2026-06-05 14:37:35 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#857
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?