Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[PR #5589] feat: add query/passage prefix env vars to GenericOpenAi embedder #5490

New Issue
Open
opened 2026-06-05 15:21:31 -04:00 by yindo · 0 comments
yindo commented 2026-06-05 15:21:31 -04:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/Mintplex-Labs/anything-llm/pull/5589
Author: @jhsmith409
Created: 5/7/2026
Status: 🔄 Open

Base: master ← Head: feat/embedder-query-passage-prefix

📝 Commits (1)

  • 771ffd3 feat: add query/passage prefix env vars to GenericOpenAi embedder

📊 Changes

3 files changed (+171 additions, -4 deletions)

View changed files

📝 server/.env.example (+7 -0)
➕ server/__tests__/utils/EmbeddingEngines/genericOpenAi/index.test.js (+135 -0)
📝 server/utils/EmbeddingEngines/genericOpenAi/index.js (+29 -4)

📄 Description

Pull Request Type

  • ✨ feat (New feature)

Relevant Issues

connects #5403

Description

The generic-openai embedder currently sends every input through unwrapped — both queries (via embedTextInput) and passages (via embedChunks) hit the same input: chunk POST body. That's correct for symmetric models like text-embedding-ada-002, but it leaves a measurable accuracy gap for asymmetric embedders (Qwen3-Embedding family, BGE, E5-instruct, multilingual-e5-*, etc.) that were trained to expect a prefix on queries.

Per the Qwen3-Embedding model card, queries should be wrapped as:

Instruct: Given a web search query, retrieve relevant passages that answer the query
Query:{query}

…with passages sent raw. The model card cites ~1–5% MTEB regression when the wrap is omitted. There's currently no way to apply this in AnythingLLM. Verification of the unwrapped behavior is in #5403 (comment) — env grep, source snippet, and a wire-capture from the spied OpenAI client.

This PR adds two optional env vars on EMBEDDING_ENGINE=generic-openai:

Env var Applied to Default Typical Qwen3 value
EMBEDDING_QUERY_PREFIX embedTextInput (queries) "" "Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery:"
EMBEDDING_PASSAGE_PREFIX embedChunks (ingest) "" "" (Qwen3 leaves passages raw)

Both empty by default, so the wire payload is bitwise-identical to the current behavior unless a user opts in. The architecture made this easy: every vector-DB provider in server/utils/vectorDbProviders/* already calls embedTextInput only for queries and embedChunks only for ingest, so the change is local to one file.

A new embedChunks(textChunks, { isPassage = true }) second arg lets embedTextInput request the query path internally without re-prefixing. Existing call sites (lance, chroma, qdrant, pinecone, milvus, weaviate, astra, pgvector) all pass a single arg and continue getting the passage default — no upstream churn.

Visuals (if applicable)

n/a — env-only change.

Additional Information

Question for @timothycarambat on scoping — I deliberately kept this PR minimal so it's easy to review, but I floated three scope options before writing it. Happy to expand if you'd prefer one of the larger options:

  1. (this PR) genericOpenAi only, env-driven. Smallest, most reviewable; closes the prefix half of #5403; chunking-strategy half stays open for a follow-up.
  2. All OpenAI-compatible embedders — apply the same env vars across genericOpenAi, openai, lmstudio, localai, ollama, voyageai, litellm, lemonade. Bigger PR but consistent behavior wherever someone might plug in a Qwen3-style asymmetric model. Could be a follow-up that builds on the helper this PR establishes.
  3. Full UI + env (matches #5403 exactly) — UI fields in the Embedder Config screen plus env vars plus per-embedder plumbing, with persistence on the EmbeddingPreference model. Largest PR; requires frontend work and likely a small Prisma migration; matches what @chrisjunlee originally asked for.

If you'd rather I roll (2) or (3) into this PR before merge, let me know and I'll push more commits to this branch. If (1) lands as-is, I'm happy to follow up with (2) or (3) in a separate PR — your call which direction.

The chunking-strategy half of #5403 is intentionally out of scope here — it's a separable change with its own design questions (which strategies to expose, where to apply them in the splitter, etc.) and I don't want to couple them.

Developer Validations

  • I ran yarn lint from the root of the repo & committed changes
  • Relevant documentation has been updated (if applicable)
  • I have tested my code functionality
  • Docker build succeeds locally

Notes on validation:

  • New Jest suite at server/__tests__/utils/EmbeddingEngines/genericOpenAi/index.test.js (5 tests, all passing) covers: defaults preserve old behavior; query prefix applied only to query path; passage prefix applied only to ingest path; query prefix doesn't leak to passage path when both are set; array inputs to embedTextInput get each element prefixed.
  • yarn lint:check from server/ passes.
  • Verified end-to-end against a live deployment serving Qwen3-Embedding-0.6B through a LiteLLM gateway: with the env vars set, the wire-capture inside the container shows {"input":["Instruct: ...\nQuery:<user query>"]} on the query path and raw chunks on the ingest path, matching the model card.
  • server/.env.example updated with both new env vars and a Qwen3-Embedding example, including a hint on dotenv \n quoting.
  • Docker build verified locally: docker build --platform=linux/amd64 -f docker/Dockerfile . from a fresh clone of this branch built successfully through the production-build stage (exit 0).

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/Mintplex-Labs/anything-llm/pull/5589 **Author:** [@jhsmith409](https://github.com/jhsmith409) **Created:** 5/7/2026 **Status:** 🔄 Open **Base:** `master` ← **Head:** `feat/embedder-query-passage-prefix` --- ### 📝 Commits (1) - [`771ffd3`](https://github.com/Mintplex-Labs/anything-llm/commit/771ffd379b3f91b0a624aec2a7b4afc748c55252) feat: add query/passage prefix env vars to GenericOpenAi embedder ### 📊 Changes **3 files changed** (+171 additions, -4 deletions) <details> <summary>View changed files</summary> 📝 `server/.env.example` (+7 -0) ➕ `server/__tests__/utils/EmbeddingEngines/genericOpenAi/index.test.js` (+135 -0) 📝 `server/utils/EmbeddingEngines/genericOpenAi/index.js` (+29 -4) </details> ### 📄 Description ### Pull Request Type - [x] ✨ feat (New feature) ### Relevant Issues connects #5403 ### Description The `generic-openai` embedder currently sends every input through unwrapped — both queries (via `embedTextInput`) and passages (via `embedChunks`) hit the same `input: chunk` POST body. That's correct for symmetric models like `text-embedding-ada-002`, but it leaves a measurable accuracy gap for **asymmetric** embedders (Qwen3-Embedding family, BGE, E5-instruct, multilingual-e5-*, etc.) that were trained to expect a prefix on queries. Per the [Qwen3-Embedding model card](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B), queries should be wrapped as: ``` Instruct: Given a web search query, retrieve relevant passages that answer the query Query:{query} ``` …with passages sent raw. The model card cites ~1–5% MTEB regression when the wrap is omitted. There's currently no way to apply this in AnythingLLM. Verification of the unwrapped behavior is in [#5403 (comment)](https://github.com/Mintplex-Labs/anything-llm/issues/5403#issuecomment-4396775933) — env grep, source snippet, and a wire-capture from the spied OpenAI client. This PR adds two optional env vars on `EMBEDDING_ENGINE=generic-openai`: | Env var | Applied to | Default | Typical Qwen3 value | |----------------------------|---------------------------|---------|-------------------------------------------------------------------------------------------| | `EMBEDDING_QUERY_PREFIX` | `embedTextInput` (queries) | `""` | `"Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery:"` | | `EMBEDDING_PASSAGE_PREFIX` | `embedChunks` (ingest) | `""` | `""` (Qwen3 leaves passages raw) | Both empty by default, so the wire payload is **bitwise-identical to the current behavior** unless a user opts in. The architecture made this easy: every vector-DB provider in `server/utils/vectorDbProviders/*` already calls `embedTextInput` only for queries and `embedChunks` only for ingest, so the change is local to one file. A new `embedChunks(textChunks, { isPassage = true })` second arg lets `embedTextInput` request the query path internally without re-prefixing. Existing call sites (lance, chroma, qdrant, pinecone, milvus, weaviate, astra, pgvector) all pass a single arg and continue getting the passage default — no upstream churn. ### Visuals (if applicable) n/a — env-only change. ### Additional Information **Question for @timothycarambat on scoping** — I deliberately kept this PR minimal so it's easy to review, but I floated three scope options before writing it. Happy to expand if you'd prefer one of the larger options: 1. **(this PR)** `genericOpenAi` only, env-driven. Smallest, most reviewable; closes the prefix half of #5403; chunking-strategy half stays open for a follow-up. 2. **All OpenAI-compatible embedders** — apply the same env vars across `genericOpenAi`, `openai`, `lmstudio`, `localai`, `ollama`, `voyageai`, `litellm`, `lemonade`. Bigger PR but consistent behavior wherever someone might plug in a Qwen3-style asymmetric model. Could be a follow-up that builds on the helper this PR establishes. 3. **Full UI + env (matches #5403 exactly)** — UI fields in the Embedder Config screen plus env vars plus per-embedder plumbing, with persistence on the `EmbeddingPreference` model. Largest PR; requires frontend work and likely a small Prisma migration; matches what @chrisjunlee originally asked for. If you'd rather I roll (2) or (3) into this PR before merge, let me know and I'll push more commits to this branch. If (1) lands as-is, I'm happy to follow up with (2) or (3) in a separate PR — your call which direction. The chunking-strategy half of #5403 is intentionally **out of scope** here — it's a separable change with its own design questions (which strategies to expose, where to apply them in the splitter, etc.) and I don't want to couple them. ### Developer Validations - [x] I ran `yarn lint` from the root of the repo & committed changes - [x] Relevant documentation has been updated (if applicable) - [x] I have tested my code functionality - [x] Docker build succeeds locally Notes on validation: - New Jest suite at `server/__tests__/utils/EmbeddingEngines/genericOpenAi/index.test.js` (5 tests, all passing) covers: defaults preserve old behavior; query prefix applied only to query path; passage prefix applied only to ingest path; query prefix doesn't leak to passage path when both are set; array inputs to `embedTextInput` get each element prefixed. - `yarn lint:check` from `server/` passes. - Verified end-to-end against a live deployment serving Qwen3-Embedding-0.6B through a LiteLLM gateway: with the env vars set, the wire-capture inside the container shows `{"input":["Instruct: ...\nQuery:<user query>"]}` on the query path and raw chunks on the ingest path, matching the model card. - `server/.env.example` updated with both new env vars and a Qwen3-Embedding example, including a hint on dotenv `\n` quoting. - Docker build verified locally: `docker build --platform=linux/amd64 -f docker/Dockerfile .` from a fresh clone of this branch built successfully through the `production-build` stage (exit 0). --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-06-05 15:21:31 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5490
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?