Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4712] [BUG]: Cannot embed in to ChromaDB after updating to 1.9.0 #2983

New Issue
Closed
opened 2026-02-22 18:32:08 -05:00 by yindo · 8 comments
yindo commented 2026-02-22 18:32:08 -05:00
Owner
Copy Link
Copy Source

Originally created by @Mark-Petley on GitHub (Dec 5, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4712

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

After updating Anything LLM to 1.9.0 to gain access to the CometAPI integration I have an issue that document that would previous embed leveraging LM Studio and the text-embedding-nomic-embed-test-v1.5 LLM the same document will now not embed into a backend ChromaDB. When I attempt to do this via API I get a 404 error and when I do this in the GUI of AnythingLLM I get served an error after about 5 minutes of "error embedding into chromaDB. unexpected token 'p' payload too large invalid json". I have attempted using the same setting as before and the same document that worked before the upgrade. I have also reduce the text chunking from 512 to 256 and the embedding from 512 / 50 overlap to 256 / 25 overlap. Yet still the same error occurs.

Image

Are there known steps to reproduce?

Configure ChromaDB with the following settings:

collection: TEST
distance_metric: cosine
chunking:
chunk_size_tokens: 512
chunk_overlap_tokens: 20
splitter: hierarchical
heading_first: true
sentence_fallback: true
metadata:

  • doc_title
  • section_title
  • heading_path
  • clause_id
  • page_start
  • page_end
  • version_date
    retrieval:
    top_k: 12
    mmr_lambda: 0.5
    hybrid_search: true
    rerank:
    enabled: true
    initial_k: 12
    final_k: 5

set Anythinkg LLM as outlined above.

Take any document even 2mb in size that is purely text in either docx, md or json and try to upload.

Originally created by @Mark-Petley on GitHub (Dec 5, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4712 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? After updating Anything LLM to 1.9.0 to gain access to the CometAPI integration I have an issue that document that would previous embed leveraging LM Studio and the text-embedding-nomic-embed-test-v1.5 LLM the same document will now not embed into a backend ChromaDB. When I attempt to do this via API I get a 404 error and when I do this in the GUI of AnythingLLM I get served an error after about 5 minutes of "error embedding into chromaDB. unexpected token 'p' payload too large invalid json". I have attempted using the same setting as before and the same document that worked before the upgrade. I have also reduce the text chunking from 512 to 256 and the embedding from 512 / 50 overlap to 256 / 25 overlap. Yet still the same error occurs. <img width="347" height="144" alt="Image" src="https://github.com/user-attachments/assets/f57cf303-23c2-493e-b6e1-344dc952ed9f" /> ### Are there known steps to reproduce? Configure ChromaDB with the following settings: collection: TEST distance_metric: cosine chunking: chunk_size_tokens: 512 chunk_overlap_tokens: 20 splitter: hierarchical heading_first: true sentence_fallback: true metadata: - doc_title - section_title - heading_path - clause_id - page_start - page_end - version_date retrieval: top_k: 12 mmr_lambda: 0.5 hybrid_search: true rerank: enabled: true initial_k: 12 final_k: 5 set Anythinkg LLM as outlined above. Take any document even 2mb in size that is purely text in either docx, md or json and try to upload.
yindo added the possible bug label 2026-02-22 18:32:08 -05:00
yindo closed this issue 2026-02-22 18:32:08 -05:00
yindo commented 2026-02-22 18:32:09 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Dec 5, 2025):

https://github.com/Mintplex-Labs/anything-llm/pull/4705

Already patched, not in desktop yet since PR was merged post 1.9.0

@timothycarambat commented on GitHub (Dec 5, 2025): https://github.com/Mintplex-Labs/anything-llm/pull/4705 Already patched, not in desktop yet since PR was merged post 1.9.0
yindo commented 2026-02-22 18:32:09 -05:00
Author
Owner
Copy Link
Copy Source

@Mark-Petley commented on GitHub (Dec 5, 2025):

any timeline you could share as to when we can expect the one with the bug fix in it to be released?

@Mark-Petley commented on GitHub (Dec 5, 2025): any timeline you could share as to when we can expect the one with the bug fix in it to be released?
yindo commented 2026-02-22 18:32:09 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Dec 9, 2025):

In 1.9.1 release - this week

@timothycarambat commented on GitHub (Dec 9, 2025): In 1.9.1 release - this week
yindo commented 2026-02-22 18:32:09 -05:00
Author
Owner
Copy Link
Copy Source

@Mark-Petley commented on GitHub (Dec 11, 2025):

Even after updating to 1.9.1 we are still seeing the same issue with the documents we are trying to embed into Chroma. Is the anything that we can do to work around this?

@Mark-Petley commented on GitHub (Dec 11, 2025): Even after updating to 1.9.1 we are still seeing the same issue with the documents we are trying to embed into Chroma. Is the anything that we can do to work around this?
yindo commented 2026-02-22 18:32:09 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Dec 11, 2025):

@Mark-Petley are you on Chroma cloud? I am able to upload a document to AnythingLLM with Chroma cloud and the default embedder. This results in over 1000 vectors - all of which appear in the collection.

I then send a simple query to do some retrieval and it works without payload issues - which were previously replicable in 1.9.0.

@timothycarambat commented on GitHub (Dec 11, 2025): @Mark-Petley are you on Chroma cloud? I am able to upload a document to AnythingLLM with Chroma cloud and the default embedder. This results in over 1000 vectors - all of which appear in the collection. I then send a simple query to do some retrieval and it works without payload issues - which were previously replicable in 1.9.0.
yindo commented 2026-02-22 18:32:09 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Dec 11, 2025):

Also you mention this

heading_first: true
sentence_fallback: true
metadata:

doc_title
section_title
heading_path
clause_id
page_start
page_end
version_date
retrieval:
top_k: 12
mmr_lambda: 0.5
hybrid_search: true
rerank:
enabled: true
initial_k: 12
final_k: 5

Where are you even defining these settings. Half of them cannot even be set in AnythingLLM and also your metadata keys do not match what we define for a "document". There is no text key in metadata and you cannot import existing collections into AnythingLLM. Because the schema can be anything we can only allow AnythingLLM to upsert/query collections created by AnythingLLM. Whatever you have done in Chroma previously cannot be co-opted into AnythingLLM. That is your issue - we dont know the shape of your collection and because of that any request fails. Likely because you are missing the critical text key in your metadata since we need it.

@timothycarambat commented on GitHub (Dec 11, 2025): Also you mention this > heading_first: true sentence_fallback: true metadata: > > doc_title section_title heading_path clause_id page_start page_end version_date retrieval: top_k: 12 mmr_lambda: 0.5 hybrid_search: true rerank: enabled: true initial_k: 12 final_k: 5 Where are you even defining these settings. Half of them cannot even be set in AnythingLLM and also your metadata keys do not match what we define for a "document". There is no `text` key in metadata and you cannot _import_ existing collections into AnythingLLM. Because the schema can be anything we can only allow AnythingLLM to upsert/query collections created by AnythingLLM. Whatever you have done in Chroma previously cannot be co-opted into AnythingLLM. That is your issue - we dont know the shape of your collection and because of that any request fails. Likely because you are missing the critical `text` key in your metadata since we need it.
yindo commented 2026-02-22 18:32:09 -05:00
Author
Owner
Copy Link
Copy Source

@Mark-Petley commented on GitHub (Dec 12, 2025):

Thanks for the feedback on this, I can tell you the following. We are using a local instance of Chroma and all the settings I have shared are the default ones that Chroma configures on setup. This configuration worked prior to 1.9.1 with these setting and a particular document which now does not embed into Chroma now but did pre 1.9.1. We are going to test the default Vector provider within Anything LLM to see if that mitigates the issue. However the default vector provider within Anything LLM wouldn't allow us to scale should we need to, hence the decision on Chroma.

I can tell you that when the document worked prior to the upgrade I was getting in excess of 4000 vectors from it so it is a large document. With text chunking of 300 and an embed of 300 with a 30 overlap.

@Mark-Petley commented on GitHub (Dec 12, 2025): Thanks for the feedback on this, I can tell you the following. We are using a local instance of Chroma and all the settings I have shared are the default ones that Chroma configures on setup. This configuration worked prior to 1.9.1 with these setting and a particular document which now does not embed into Chroma now but did pre 1.9.1. We are going to test the default Vector provider within Anything LLM to see if that mitigates the issue. However the default vector provider within Anything LLM wouldn't allow us to scale should we need to, hence the decision on Chroma. I can tell you that when the document worked prior to the upgrade I was getting in excess of 4000 vectors from it so it is a large document. With text chunking of 300 and an embed of 300 with a 30 overlap.
yindo commented 2026-02-22 18:32:09 -05:00
Author
Owner
Copy Link
Copy Source

@Mark-Petley commented on GitHub (Dec 12, 2025):

Quick update for you, we have tested with other Vector Database Providers using the same document and using the bge-large LLM model hosted in LM Studio, which was our previous configuration that was working and found that using a different vector provider it is working and creating the vectors. This to me points to it still being an issue in the local hosted Chroma interface between anything llm and Chroma.

@Mark-Petley commented on GitHub (Dec 12, 2025): Quick update for you, we have tested with other Vector Database Providers using the same document and using the bge-large LLM model hosted in LM Studio, which was our previous configuration that was working and found that using a different vector provider it is working and creating the vectors. This to me points to it still being an issue in the local hosted Chroma interface between anything llm and Chroma.
yindo changed title from [BUG]: Cannot embed in to ChromaDB after updating to 1.9.0 to [GH-ISSUE #4712] [BUG]: Cannot embed in to ChromaDB after updating to 1.9.0 2026-06-05 14:49:38 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2983
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?