Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1775] [BUG]: OpenAI Compatible Endpoints completion without chunk #1161

New Issue
Closed
opened 2026-02-22 18:23:25 -05:00 by yindo · 4 comments
yindo commented 2026-02-22 18:23:25 -05:00
Owner
Copy Link
Copy Source

Originally created by @nicho2 on GitHub (Jun 27, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1775

How are you running AnythingLLM?

Docker (local)

What happened?

I tried your new interestring functionnaly "OpenAI Compatible Endpoints" , the path : /v1/openai/chat/completions

you have write : Send a prompt to the workspace with full use of documents as if sending a chat in AnythingLLM. Only supports some values of OpenAI API.

the system prompt is sending to the llm, but no documents contexts are inserted.

Are there known steps to reproduce?

{
"messages": [
{
"role": "user",
"content": "qu'est ce qu'un système BMS"
}
],
"model": "atelier_bms",
"stream": false,
"temperature": 0
}

Originally created by @nicho2 on GitHub (Jun 27, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1775 ### How are you running AnythingLLM? Docker (local) ### What happened? I tried your new interestring functionnaly "OpenAI Compatible Endpoints" , the path : /v1/openai/chat/completions you have write : Send a prompt to the workspace with full use of documents as if sending a chat in AnythingLLM. Only supports some values of OpenAI API. the system prompt is sending to the llm, but no documents contexts are inserted. ### Are there known steps to reproduce? { "messages": [ { "role": "user", "content": "qu'est ce qu'un système BMS" } ], "model": "atelier_bms", "stream": false, "temperature": 0 }
yindo added the possible bug label 2026-02-22 18:23:25 -05:00
yindo closed this issue 2026-02-22 18:23:25 -05:00
yindo commented 2026-02-22 18:23:26 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 27, 2024):

In the docker logs you would see if any context was retrieved, but the case here is likely that there just simply are no matches to the prompt given in the vector database.

The search does occur
https://github.com/Mintplex-Labs/anything-llm/blob/7a78ad3960e069dbc783971dc7ff13cf1868c767/server/utils/chats/openaiCompatible.js#L83-L97

@timothycarambat commented on GitHub (Jun 27, 2024): In the docker logs you would see if any context was retrieved, but the case here is likely that there just simply are no matches to the prompt given in the vector database. The search _does occur_ https://github.com/Mintplex-Labs/anything-llm/blob/7a78ad3960e069dbc783971dc7ff13cf1868c767/server/utils/chats/openaiCompatible.js#L83-L97
yindo commented 2026-02-22 18:23:26 -05:00
Author
Owner
Copy Link
Copy Source

@nicho2 commented on GitHub (Jun 28, 2024):

I checked by sending the same question from the standard UI and by the API

In both cases, the vector base is well questioned and returns many pieces of documents. On the other hand, since the OPENAI_Like API, these pieces of documents are not added to the context

The docker log does not say much:

[OllamaEmbedder] Embedding 1 chunks of text with nomic-embed-text:latest.
[TELEMETRY SENT] {
event: 'sent_chat',
distinctId: '98297666-e886-4011-9303-5720b1f81b92',
properties: {
LLMSelection: 'lmstudio',
Embedder: 'ollama',
VectorDbSelection: 'chroma',
runtime: 'docker'
}
}
[Event Logged] - api_sent_cha

The distances are almost the same:

{"ids":"a85d269b-5da2-4f4c-9069-a49cfd6316da","adeb8a8c-95b4-4099-8912-fa2e22455eea","c2cfa314-96f0-495c-bc45-82f580d5c18f","cecfbcd9-c9ab-45e3-a595-becc01d506a7","b7dd0ad5-1313-4db7-aea9-15f7a1a47b56","distances":369.6019287109375,371.1099548339844,371.14044189453125,371.15972900390625,371.2540588378906,"metadatas":[[{"....

@nicho2 commented on GitHub (Jun 28, 2024): I checked by sending the same question from the standard UI and by the API In both cases, the vector base is well questioned and returns many pieces of documents. On the other hand, since the OPENAI_Like API, these pieces of documents are not added to the context The docker log does not say much: [OllamaEmbedder] Embedding 1 chunks of text with nomic-embed-text:latest. [TELEMETRY SENT] { event: 'sent_chat', distinctId: '98297666-e886-4011-9303-5720b1f81b92', properties: { LLMSelection: 'lmstudio', Embedder: 'ollama', VectorDbSelection: 'chroma', runtime: 'docker' } } [Event Logged] - api_sent_cha **The distances are almost the same:** {"ids":[["a85d269b-5da2-4f4c-9069-a49cfd6316da","adeb8a8c-95b4-4099-8912-fa2e22455eea","c2cfa314-96f0-495c-bc45-82f580d5c18f","cecfbcd9-c9ab-45e3-a595-becc01d506a7","b7dd0ad5-1313-4db7-aea9-15f7a1a47b56"]],"distances":[[369.6019287109375,371.1099548339844,371.14044189453125,371.15972900390625,371.2540588378906]],"metadatas":[[{"....
yindo commented 2026-02-22 18:23:26 -05:00
Author
Owner
Copy Link
Copy Source

@nicho2 commented on GitHub (Jun 28, 2024):

if i use only Query, i have the answer ::
There is no relevant information in this workspace to answer your query.

but i see the chromadb answer , i have similarity >=0.25 :

{"ids":"135a37f6-8eb4-4452-9297-2cd8a0069975","790a133c-2967-4820-a44a-c2e0beabc418","4849c503-4538-4741-b152-150f91455e20","e3b5c32f-8d75-4971-95ac-bb6a3cf02808","distances":0.3664718270301819,0.3721543550491333,0.3739575743675232,0.37604087591171265,"metadatas":[[{"chunkSource":"","description":"Unknown","docAuthor":"Unknown","docSource":"a text file uploaded by the user.","id":"9753b7ca-7218-410b-8273-2ccdb94fe162","published":"6/21/2024, 2:21:04 PM","title":"Guide technique AREA MANAGER - partie 1.md","token_count_estimate":16438,"url":"file:///app/collector/hotdir/Guide technique AREA MANAGER - partie 1.md","wordCount":12105},{"chunkSource":"","description":"Unknown","docAuthor":"Unknown","docSource":"a text file uploaded by the user.","id":"9753b7ca-7218-410b-8273-2ccdb94fe162","published":"6/21/2024, 2:21:04 PM","title":"Guide technique AREA MANAGER - partie 1.md","token_count_estimate":16438,"url":"file:///app/collector/hotdir/Guide technique A

@nicho2 commented on GitHub (Jun 28, 2024): if i use only Query, i have the answer :: There is no relevant information in this workspace to answer your query. but i see the chromadb answer , i have similarity >=0.25 : {"ids":[["135a37f6-8eb4-4452-9297-2cd8a0069975","790a133c-2967-4820-a44a-c2e0beabc418","4849c503-4538-4741-b152-150f91455e20","e3b5c32f-8d75-4971-95ac-bb6a3cf02808"]],"distances":[[0.3664718270301819,0.3721543550491333,0.3739575743675232,0.37604087591171265]],"metadatas":[[{"chunkSource":"","description":"Unknown","docAuthor":"Unknown","docSource":"a text file uploaded by the user.","id":"9753b7ca-7218-410b-8273-2ccdb94fe162","published":"6/21/2024, 2:21:04 PM","title":"Guide technique AREA MANAGER - partie 1.md","token_count_estimate":16438,"url":"file:///app/collector/hotdir/Guide technique AREA MANAGER - partie 1.md","wordCount":12105},{"chunkSource":"","description":"Unknown","docAuthor":"Unknown","docSource":"a text file uploaded by the user.","id":"9753b7ca-7218-410b-8273-2ccdb94fe162","published":"6/21/2024, 2:21:04 PM","title":"Guide technique AREA MANAGER - partie 1.md","token_count_estimate":16438,"url":"file:///app/collector/hotdir/Guide technique A
yindo commented 2026-02-22 18:23:26 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 28, 2024):

Odd, i dont see a "documents" key in there. There should be ids,distances,metadatas and documents

@timothycarambat commented on GitHub (Jun 28, 2024): Odd, i dont see a "documents" key in there. There should be `ids`,`distances`,`metadatas` and `documents`
yindo changed title from [BUG]: OpenAI Compatible Endpoints completion without chunk to [GH-ISSUE #1775] [BUG]: OpenAI Compatible Endpoints completion without chunk 2026-06-05 14:39:16 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1161
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?