Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #645] The accuracy of data retrieval is not high #369

New Issue
Closed
opened 2026-02-22 18:19:09 -05:00 by yindo · 15 comments
yindo commented 2026-02-22 18:19:09 -05:00
Owner
Copy Link
Copy Source

Originally created by @huicewang on GitHub (Jan 24, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/645

What would you like to see?

During the testing process, it was found that the accuracy of data retrieval is not high, and there are often illusions when dealing with large amounts of data. Additionally, the answers provided are too brief. I suggest referring to this open-source platform: https://github.com/danswer-ai/danswer. The project utilizes intent recognition with a large model and re-ranking techniques, which significantly improve accuracy. Can you make some optimizations in this regard? The accuracy of data retrieval is the most critical part of the system. Thank you.

Originally created by @huicewang on GitHub (Jan 24, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/645 ### What would you like to see? During the testing process, it was found that the accuracy of data retrieval is not high, and there are often illusions when dealing with large amounts of data. Additionally, the answers provided are too brief. I suggest referring to this open-source platform: https://github.com/danswer-ai/danswer. The project utilizes intent recognition with a large model and re-ranking techniques, which significantly improve accuracy. Can you make some optimizations in this regard? The accuracy of data retrieval is the most critical part of the system. Thank you.
yindo added the questionneeds info / can't replicate labels 2026-02-22 18:19:09 -05:00
yindo closed this issue 2026-02-22 18:19:09 -05:00
yindo commented 2026-02-22 18:19:09 -05:00
Author
Owner
Copy Link
Copy Source

@GANJAC commented on GitHub (Jan 25, 2024):

Hi huicewang, do you use native embedder ?

@GANJAC commented on GitHub (Jan 25, 2024): Hi huicewang, do you use native embedder ?
yindo commented 2026-02-22 18:19:09 -05:00
Author
Owner
Copy Link
Copy Source

@huicewang commented on GitHub (Jan 25, 2024):

Yes，bge-large

------------------ Original ------------------
From: GANJAC @.>
Date: Fri,Jan 26,2024 1:08 AM
To: Mintplex-Labs/anything-llm @.>
Cc: huicewang @.>, Author @.>
Subject: Re: [Mintplex-Labs/anything-llm] The accuracy of data retrieval isnot high (Issue #645)

Hi huicewang, do you use native embedder ?

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

@huicewang commented on GitHub (Jan 25, 2024): Yes，bge-large ------------------ Original ------------------ From: GANJAC ***@***.***> Date: Fri,Jan 26,2024 1:08 AM To: Mintplex-Labs/anything-llm ***@***.***> Cc: huicewang ***@***.***>, Author ***@***.***> Subject: Re: [Mintplex-Labs/anything-llm] The accuracy of data retrieval isnot high (Issue #645) Hi huicewang, do you use native embedder ? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>
yindo commented 2026-02-22 18:19:09 -05:00
Author
Owner
Copy Link
Copy Source

@huicewang commented on GitHub (Jan 25, 2024):

But the openai embedding is the same result

------------------ Original ------------------
From: GANJAC @.>
Date: Fri,Jan 26,2024 1:08 AM
To: Mintplex-Labs/anything-llm @.>
Cc: huicewang @.>, Author @.>
Subject: Re: [Mintplex-Labs/anything-llm] The accuracy of data retrieval isnot high (Issue #645)

Hi huicewang, do you use native embedder ?

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

@huicewang commented on GitHub (Jan 25, 2024): But the openai embedding is the same result ------------------ Original ------------------ From: GANJAC ***@***.***> Date: Fri,Jan 26,2024 1:08 AM To: Mintplex-Labs/anything-llm ***@***.***> Cc: huicewang ***@***.***>, Author ***@***.***> Subject: Re: [Mintplex-Labs/anything-llm] The accuracy of data retrieval isnot high (Issue #645) Hi huicewang, do you use native embedder ? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>
yindo commented 2026-02-22 18:19:10 -05:00
Author
Owner
Copy Link
Copy Source

@sumitsodhi88 commented on GitHub (Jan 27, 2024):

same issue for me while using lance db and native embedder.

even if i copy the same line from the pdf uploaded, the chunk having same line is not found/ selected.

this happens for large size pdfs for specific on specific data. for eg. im using a constitution but whole language seems to be same. the model is not able to choose correct chunk i have tried all settings and prompts. However, using word file solved it somewhat but still persists.

@sumitsodhi88 commented on GitHub (Jan 27, 2024): same issue for me while using lance db and native embedder. even if i copy the same line from the pdf uploaded, the chunk having same line is not found/ selected. this happens for large size pdfs for specific on specific data. for eg. im using a constitution but whole language seems to be same. the model is not able to choose correct chunk i have tried all settings and prompts. However, using word file solved it somewhat but still persists.
yindo commented 2026-02-22 18:19:10 -05:00
Author
Owner
Copy Link
Copy Source

@GANJAC commented on GitHub (Jan 27, 2024):

Hi guys, excuse me for the delayed response, i have 95% mitigated this issue with this model:

https://huggingface.co/Xenova/multilingual-e5-large

as native embedder an qdrant vector db.

Also i use openorca mistral 7b served by llama-cpp-python as generative ai.

I hope this can help.

@GANJAC commented on GitHub (Jan 27, 2024): Hi guys, excuse me for the delayed response, i have 95% mitigated this issue with this model: https://huggingface.co/Xenova/multilingual-e5-large as native embedder an qdrant vector db. Also i use openorca mistral 7b served by llama-cpp-python as generative ai. I hope this can help.
yindo commented 2026-02-22 18:19:10 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jan 27, 2024):

@GANJAC This is an interesting model. I wonder if we could have this as an option for the native embedder as we currently use the all-MiniLM-L6-v2 model, but that model does not perform well with other languages (esp Mandarin).

What language are your documents? If they are English and the models are the same dimension, I would be interested to see how that model performs unless it is too large to download on the fly

@timothycarambat commented on GitHub (Jan 27, 2024): @GANJAC This is an interesting model. I wonder if we could have this as an option for the native embedder as we currently use the [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) model, but that model does not perform well with other languages (esp Mandarin). What language are your documents? If they are English and the models are the same dimension, I would be interested to see how that model performs unless it is too large to download on the fly
yindo commented 2026-02-22 18:19:10 -05:00
Author
Owner
Copy Link
Copy Source

@sumitsodhi88 commented on GitHub (Jan 28, 2024):

Screenshot_20240128_114540_Chrome

tested all-MiniLM-L6-v2. Is this same in all other embedding types.

@sumitsodhi88 commented on GitHub (Jan 28, 2024): ![Screenshot_20240128_114540_Chrome](https://github.com/Mintplex-Labs/anything-llm/assets/149290101/08075b32-f444-4a14-b5ed-fc6fc60fc23f) tested all-MiniLM-L6-v2. Is this same in all other embedding types.
yindo commented 2026-02-22 18:19:10 -05:00
Author
Owner
Copy Link
Copy Source

@GANJAC commented on GitHub (Jan 30, 2024):

@GANJAC This is an interesting model. I wonder if we could have this as an option for the native embedder as we currently use the all-MiniLM-L6-v2 model, but that model does not perform well with other languages (esp Mandarin).

What language are your documents? If they are English and the models are the same dimension, I would be interested to see how that model performs unless it is too large to download on the fly

Hi @timothycarambat, first of all thank you so much for your awesome work!!! My documents is Italian and English (50/50), from what i see the model multilingual-e5-large is more and more accurate than all-MiniLM-L6-v2 in any languages, and despite it's size (2.7 GB) it's very fast in embeddings.

Hope this help.

@GANJAC commented on GitHub (Jan 30, 2024): > @GANJAC This is an interesting model. I wonder if we could have this as an option for the native embedder as we currently use the [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) model, but that model does not perform well with other languages (esp Mandarin). > > What language are your documents? If they are English and the models are the same dimension, I would be interested to see how that model performs unless it is too large to download on the fly Hi @timothycarambat, first of all thank you so much for your awesome work!!! My documents is Italian and English (50/50), from what i see the model **multilingual-e5-large** is more and more accurate than **all-MiniLM-L6-v2** in any languages, and despite it's size (2.7 GB) it's very fast in embeddings. Hope this help.
yindo commented 2026-02-22 18:19:10 -05:00
Author
Owner
Copy Link
Copy Source

@huicewang commented on GitHub (Jan 30, 2024):

If a PDF file contains images, is there a good solution? Can OCR recognition be used to achieve this?

------------------ Original ------------------
From: GANJAC @.>
Date: Tue,Jan 30,2024 4:11 PM
To: Mintplex-Labs/anything-llm @.>
Cc: huicewang @.>, Author @.>
Subject: Re: [Mintplex-Labs/anything-llm] The accuracy of data retrieval isnot high (Issue #645)

@GANJAC This is an interesting model. I wonder if we could have this as an option for the native embedder as we currently use the all-MiniLM-L6-v2 model, but that model does not perform well with other languages (esp Mandarin).

What language are your documents? If they are English and the models are the same dimension, I would be interested to see how that model performs unless it is too large to download on the fly

Hi @timothycarambat, first of all thank you so much for your awesome work!!! My documents is Italian and English (50/50), from what i see the model multilingual-e5-large is more and more accurate than all-MiniLM-L6-v2 in any languages, and despite it's size (2.7 GB) it's very fast in embeddings.

Hope this help.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

@huicewang commented on GitHub (Jan 30, 2024): If a PDF file contains images, is there a good solution? Can OCR recognition be used to achieve this? ------------------ Original ------------------ From: GANJAC ***@***.***> Date: Tue,Jan 30,2024 4:11 PM To: Mintplex-Labs/anything-llm ***@***.***> Cc: huicewang ***@***.***>, Author ***@***.***> Subject: Re: [Mintplex-Labs/anything-llm] The accuracy of data retrieval isnot high (Issue #645) @GANJAC This is an interesting model. I wonder if we could have this as an option for the native embedder as we currently use the all-MiniLM-L6-v2 model, but that model does not perform well with other languages (esp Mandarin). What language are your documents? If they are English and the models are the same dimension, I would be interested to see how that model performs unless it is too large to download on the fly Hi @timothycarambat, first of all thank you so much for your awesome work!!! My documents is Italian and English (50/50), from what i see the model multilingual-e5-large is more and more accurate than all-MiniLM-L6-v2 in any languages, and despite it's size (2.7 GB) it's very fast in embeddings. Hope this help. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>
yindo commented 2026-02-22 18:19:10 -05:00
Author
Owner
Copy Link
Copy Source

@Tiberius1313 commented on GitHub (Mar 6, 2024):

Hi guys, excuse me for the delayed response, i have 95% mitigated this issue with this model:

https://huggingface.co/Xenova/multilingual-e5-large

as native embedder an qdrant vector db.

Also i use openorca mistral 7b served by llama-cpp-python as generative ai.

I hope this can help.

How can I switch the default embedding model?

@Tiberius1313 commented on GitHub (Mar 6, 2024): > Hi guys, excuse me for the delayed response, i have 95% mitigated this issue with this model: > > https://huggingface.co/Xenova/multilingual-e5-large > > as native embedder an qdrant vector db. > > Also i use openorca mistral 7b served by llama-cpp-python as generative ai. > > I hope this can help. How can I switch the default embedding model?
yindo commented 2026-02-22 18:19:10 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 6, 2024):

@Tiberius1313
Its in the settings on the sidebar under "Embedding preference"

@timothycarambat commented on GitHub (Mar 6, 2024): @Tiberius1313 Its in the settings on the sidebar under "Embedding preference"
yindo commented 2026-02-22 18:19:10 -05:00
Author
Owner
Copy Link
Copy Source

@Tiberius1313 commented on GitHub (Mar 6, 2024):

@timothycarambat thank you for your fast reply.

I'm on Windows10 and AnythingLLM v.1.2.0. Left sidebar -> Settings -> Embedding preference. Do you change AnythingLMM Embedder to a different embedding model or did you use "LocalAI" / "Ollama"? I was able to run in LM Studio a local model (Mixtral 8x7b) but not an embedding model. In my UI there's no option to change the default embedder.

PS: Problem also describe in #856

Screenshot (6108)

@Tiberius1313 commented on GitHub (Mar 6, 2024): @timothycarambat thank you for your fast reply. I'm on Windows10 and AnythingLLM v.1.2.0. Left sidebar -> Settings -> Embedding preference. Do you change AnythingLMM Embedder to a different embedding model or did you use "LocalAI" / "Ollama"? I was able to run in LM Studio a local model (Mixtral 8x7b) but not an embedding model. In my UI there's no option to change the default embedder. PS: Problem also describe in #856 ![Screenshot (6108)](https://github.com/Mintplex-Labs/anything-llm/assets/97293733/12b20ec2-1ad2-4114-ae00-71d2d6534b94)
yindo commented 2026-02-22 18:19:10 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 6, 2024):

@Tiberius1313 in that above UI you just can click on any other provider and there will be options to load an embedder via OpenAi,Azure,LocalAI, and Ollama (requires additional setup on your end). The default app only ships with one embedding model due and is meant to be quite general and it notable does not perform well in multi-lingual documents. Basically only english. It is a small model (all-miniLM)

@timothycarambat commented on GitHub (Mar 6, 2024): @Tiberius1313 in that above UI you just can click on any other provider and there will be options to load an embedder via OpenAi,Azure,LocalAI, and Ollama (requires additional setup on your end). The default app only ships with one embedding model due and is meant to be quite general and it notable does **not** perform well in multi-lingual documents. Basically only english. It is a small model (all-miniLM)
yindo commented 2026-02-22 18:19:10 -05:00
Author
Owner
Copy Link
Copy Source

@Tiberius1313 commented on GitHub (Mar 6, 2024):

@timothycarambat yes I've tried but I can't get anything else running. Could you please describe how you did make the multilingual-e5-large model work?

@Tiberius1313 commented on GitHub (Mar 6, 2024): @timothycarambat yes I've tried but I can't get anything else running. Could you please describe how you did make the multilingual-e5-large model work?
yindo commented 2026-02-22 18:19:10 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 7, 2024):

see https://github.com/Mintplex-Labs/anything-llm/issues/780
@Tiberius1313 the multi-lingual model is not yet supported. That must be custom code?

Moving conversation to the above issue!

@timothycarambat commented on GitHub (Mar 7, 2024): see https://github.com/Mintplex-Labs/anything-llm/issues/780 @Tiberius1313 the multi-lingual model is not yet supported. That must be custom code? Moving conversation to the above issue!
yindo changed title from The accuracy of data retrieval is not high to [GH-ISSUE #645] The accuracy of data retrieval is not high 2026-06-05 14:34:56 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label needs info / can't replicate question
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#369
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?