Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4610] [BUG]: Error with multilingual-e5-large Embedding Model - "Cannot read properties of undefined (reading 'user')" #2934

New Issue
Closed
opened 2026-02-22 18:31:54 -05:00 by yindo · 5 comments
yindo commented 2026-02-22 18:31:54 -05:00
Owner
Copy Link
Copy Source

Originally created by @juangig6 on GitHub (Nov 4, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4610

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

Hello,
I am experiencing an issue when using an external Embedding Model (Model Name: multilingual-e5-large). After vectorizing documents, I receive the following error message:
Could not respond to message.
Cannot read properties of undefined (reading 'user')
However, when I switch to other built-in models such as multilingual-e5-small, the system works normally without any errors.
My Configuration:

Embedding Provider: Local AI
Model Name: multilingual-e5-large
LocalAI Base URL: http://127.0.0.1:8080/v1
AnythingLLM Version: v1.9.0

Additional Context:
I am trying to promote the adoption of AnythingLLM within our organization for legal document retrieval applications, specifically for searching court judgment documents using RAG (Retrieval-Augmented Generation).
However, the multilingual-e5-small model does not provide satisfactory results for our use case. Therefore, I am attempting to use the multilingual-e5-large model to improve the retrieval accuracy for legal documents.
This issue is blocking our evaluation and potential deployment of AnythingLLM in our judicial system.
Could you please help resolve this issue?
Thank you for your assistance.

Are there known steps to reproduce?

Steps to Reproduce:

Go to Settings → Vector Database → Embedding Provider
Select "Local AI" and set Model Name to multilingual-e5-large
Set LocalAI Base URL to http://127.0.0.1:8080/v1
Save configuration
Upload documents to a workspace
Try to send a query in the chat

Result:
Error appears: "Could not respond to message. Cannot read properties of undefined (reading 'user')"
Note:
Switching to multilingual-e5-small works normally. The issue only occurs with the large model.

here is my code:

embedding_server.py

from flask import Flask, request, jsonify
from sentence_transformers import SentenceTransformer

app = Flask(name)

MODEL_PATH = r"D:\anythingllm-models\multilingual-e5-large"

print("="*60)
print("🔄 正在載入 multilingual-e5-large 模型...")
print(f"📁 模型路徑: {MODEL_PATH}")

model = SentenceTransformer(MODEL_PATH)

print("✅ 模型載入完成！")
print("="*60)

@app.route('/v1/embeddings', methods=['POST'])
@app.route('/embeddings', methods=['POST'])
@app.route('/v1/embedding', methods=['POST'])
@app.route('/embedding', methods=['POST'])
def create_embeddings():
try:
data = request.json

    if 'input' in data:
        texts = data['input'] if isinstance(data['input'], list) else [data['input']]
    elif 'inputs' in data:
        texts = data['inputs'] if isinstance(data['inputs'], list) else [data['inputs']]
    else:
        return jsonify({"error": "Missing input field"}), 400
    
    print(f"🔍 處理 {len(texts)} 個文本...")
    
    embeddings = model.encode(texts, convert_to_numpy=True, show_progress_bar=False)
    
    response = {
        "object": "list",
        "data": [
            {
                "object": "embedding",
                "embedding": emb.tolist(),
                "index": idx
            }
            for idx, emb in enumerate(embeddings)
        ],
        "model": "multilingual-e5-large",
        "usage": {
            "prompt_tokens": sum(len(t.split()) for t in texts),
            "total_tokens": sum(len(t.split()) for t in texts)
        }
    }
    
    print(f"✅ 完成！")
    return jsonify(response)

except Exception as e:
    print(f"❌ 錯誤: {str(e)}")
    import traceback
    traceback.print_exc()
    return jsonify({"error": str(e)}), 500

@app.route('/models', methods=['GET'])
@app.route('/v1/models', methods=['GET'])
def list_models():
return jsonify({
"data": [
{
"id": "multilingual-e5-large",
"object": "model",
"owned_by": "local"
}
]
})

@app.route('/health', methods=['GET'])
@app.route('/', methods=['GET'])
def health():
return jsonify({
"status": "ok",
"model": "multilingual-e5-large",
"model_path": MODEL_PATH,
"vector_dimension": 1024
})

if name == 'main':
print("\n" + "="*60)
print("🚀 Embedding API Server 已啟動！")
print("📍 URL: http://localhost:8080")
print("🔍 Health Check: http://localhost:8080/health")
print("="*60 + "\n")

app.run(host='0.0.0.0', port=8080, debug=False, threaded=True)
Originally created by @juangig6 on GitHub (Nov 4, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4610 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? Hello, I am experiencing an issue when using an external Embedding Model (Model Name: multilingual-e5-large). After vectorizing documents, I receive the following error message: Could not respond to message. Cannot read properties of undefined (reading 'user') However, when I switch to other built-in models such as multilingual-e5-small, the system works normally without any errors. My Configuration: Embedding Provider: Local AI Model Name: multilingual-e5-large LocalAI Base URL: http://127.0.0.1:8080/v1 AnythingLLM Version: v1.9.0 Additional Context: I am trying to promote the adoption of AnythingLLM within our organization for legal document retrieval applications, specifically for searching court judgment documents using RAG (Retrieval-Augmented Generation). However, the multilingual-e5-small model does not provide satisfactory results for our use case. Therefore, I am attempting to use the multilingual-e5-large model to improve the retrieval accuracy for legal documents. This issue is blocking our evaluation and potential deployment of AnythingLLM in our judicial system. Could you please help resolve this issue? Thank you for your assistance. ### Are there known steps to reproduce? Steps to Reproduce: Go to Settings → Vector Database → Embedding Provider Select "Local AI" and set Model Name to multilingual-e5-large Set LocalAI Base URL to http://127.0.0.1:8080/v1 Save configuration Upload documents to a workspace Try to send a query in the chat Result: Error appears: "Could not respond to message. Cannot read properties of undefined (reading 'user')" Note: Switching to multilingual-e5-small works normally. The issue only occurs with the large model. ================= here is my code: ================= # embedding_server.py from flask import Flask, request, jsonify from sentence_transformers import SentenceTransformer app = Flask(__name__) MODEL_PATH = r"D:\anythingllm-models\multilingual-e5-large" print("="*60) print("🔄 正在載入 multilingual-e5-large 模型...") print(f"📁 模型路徑: {MODEL_PATH}") model = SentenceTransformer(MODEL_PATH) print("✅ 模型載入完成！") print("="*60) @app.route('/v1/embeddings', methods=['POST']) @app.route('/embeddings', methods=['POST']) @app.route('/v1/embedding', methods=['POST']) @app.route('/embedding', methods=['POST']) def create_embeddings(): try: data = request.json if 'input' in data: texts = data['input'] if isinstance(data['input'], list) else [data['input']] elif 'inputs' in data: texts = data['inputs'] if isinstance(data['inputs'], list) else [data['inputs']] else: return jsonify({"error": "Missing input field"}), 400 print(f"🔍 處理 {len(texts)} 個文本...") embeddings = model.encode(texts, convert_to_numpy=True, show_progress_bar=False) response = { "object": "list", "data": [ { "object": "embedding", "embedding": emb.tolist(), "index": idx } for idx, emb in enumerate(embeddings) ], "model": "multilingual-e5-large", "usage": { "prompt_tokens": sum(len(t.split()) for t in texts), "total_tokens": sum(len(t.split()) for t in texts) } } print(f"✅ 完成！") return jsonify(response) except Exception as e: print(f"❌ 錯誤: {str(e)}") import traceback traceback.print_exc() return jsonify({"error": str(e)}), 500 @app.route('/models', methods=['GET']) @app.route('/v1/models', methods=['GET']) def list_models(): return jsonify({ "data": [ { "id": "multilingual-e5-large", "object": "model", "owned_by": "local" } ] }) @app.route('/health', methods=['GET']) @app.route('/', methods=['GET']) def health(): return jsonify({ "status": "ok", "model": "multilingual-e5-large", "model_path": MODEL_PATH, "vector_dimension": 1024 }) if __name__ == '__main__': print("\n" + "="*60) print("🚀 Embedding API Server 已啟動！") print("📍 URL: http://localhost:8080") print("🔍 Health Check: http://localhost:8080/health") print("="*60 + "\n") app.run(host='0.0.0.0', port=8080, debug=False, threaded=True)
yindo added the possible bug label 2026-02-22 18:31:54 -05:00
yindo closed this issue 2026-02-22 18:31:54 -05:00
yindo commented 2026-02-22 18:31:55 -05:00
Author
Owner
Copy Link
Copy Source

@Yiming-M commented on GitHub (Nov 9, 2025):

I got this error as well when I was trying to send my fourth message in a thread.

@Yiming-M commented on GitHub (Nov 9, 2025): I got this error as well when I was trying to send my fourth message in a thread.
yindo commented 2026-02-22 18:31:55 -05:00
Author
Owner
Copy Link
Copy Source

@wil-low commented on GitHub (Nov 10, 2025):

I'm getting this error after about 13-15 questions in a thread. LLMs are from external Ollama. Model is Mistral:7b. Chat history is 20, performance mode is Basic (i.e. 2048 tokens). So I got the error, then reduced chat history to 10 and resubmitted my question. LLM responded.
My understanding is that System Prompt + 20 chat histories exceeded token limit, and AnythingLLM incorrectly truncated the request dropping 'user' variable.

I reverted Chat History to 20 and the model is working normally. So there might be a more involved issue...

@wil-low commented on GitHub (Nov 10, 2025): I'm getting this error after about 13-15 questions in a thread. LLMs are from external Ollama. Model is Mistral:7b. Chat history is 20, performance mode is Basic (i.e. 2048 tokens). So I got the error, then reduced chat history to 10 and resubmitted my question. LLM responded. My understanding is that System Prompt + 20 chat histories exceeded token limit, and AnythingLLM incorrectly truncated the request dropping 'user' variable. I reverted Chat History to 20 and the model is working normally. So there might be a more involved issue...
yindo commented 2026-02-22 18:31:55 -05:00
Author
Owner
Copy Link
Copy Source

@wil-low commented on GitHub (Nov 19, 2025):

I found that even a simple entry into a chat settings helps, although the question that caused the error gets removed from the chat.

@wil-low commented on GitHub (Nov 19, 2025): I found that even a simple entry into a chat settings helps, although the question that caused the error gets removed from the chat.
yindo commented 2026-02-22 18:31:55 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Nov 20, 2025):

We need the logs from the backend in order to resolve this - the frontend error message will not help - however some follow up questions.

  • You get this error during document embedding or during prompting?

This is likely related to the issues resolved by #4669 - which was caused by a race condition due to us trying to determine the token context window size for your loaded model. This is now patched.

It comes from this line
https://github.com/Mintplex-Labs/anything-llm/blob/4ec85418c448a2c0e7b565c8b43c54dff668d53b/server/utils/helpers/chat/index.js#L67-L68

Which only becomes relevant when you are estimated to have more content in your chat window than the model can support and we have to begin truncation. This specifically impacted both Ollama and LM Studio

@timothycarambat commented on GitHub (Nov 20, 2025): We need the logs from the backend in order to resolve this - the frontend error message will not help - however some follow up questions. - You get this error during document embedding or during prompting? This is likely related to the issues resolved by #4669 - which was caused by a race condition due to us trying to determine the token context window size for your loaded model. This is now patched. It comes from this line https://github.com/Mintplex-Labs/anything-llm/blob/4ec85418c448a2c0e7b565c8b43c54dff668d53b/server/utils/helpers/chat/index.js#L67-L68 Which only becomes relevant when you are estimated to have more content in your chat window than the model can support and we have to begin truncation. This specifically impacted both Ollama and LM Studio
yindo commented 2026-02-22 18:31:55 -05:00
Author
Owner
Copy Link
Copy Source

@juangig6 commented on GitHub (Nov 22, 2025):

Thanks everyone for the clarification!

@juangig6 commented on GitHub (Nov 22, 2025): Thanks everyone for the clarification!
yindo changed title from [BUG]: Error with multilingual-e5-large Embedding Model - "Cannot read properties of undefined (reading 'user')" to [GH-ISSUE #4610] [BUG]: Error with multilingual-e5-large Embedding Model - "Cannot read properties of undefined (reading 'user')" 2026-06-05 14:49:21 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2934
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?