langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-08-24 12:32:54 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

版本升级后数据集命中测试出现异常 #242

New Issue
Closed
opened 2026-02-21 17:26:30 -05:00 by yindo · 5 comments
yindo commented 2026-02-21 17:26:30 -05:00
Owner
Copy Link
Copy Source

Originally created by @jeanlyn on GitHub (Jul 4, 2023).

Originally assigned to: @takatost on GitHub.

Dify version: Self Host

Steps To Reproduce

版本由0.3.1升级到0.3.6后，使用旧的数据集进行命中测试的时候会报以下的错误:
image

API报错如下:

ValueError: Error during query: [{'locations': [{'column': 6, 'line': 1}], 'message': "explorer: get class: vector search: object vector search at index vector_index_178a48f5_3f50_4765_a6b7_1e7dc986412c_node: shard vector_index_178a48f5_3f50_4765_a6b7_1e7dc986412c_node_trp4Y9iz0jxH: vector search: knn search: distance between entrypoint and query node: vector lengths don't match: 4096 vs 768", 'path': ['Get', 'Vector_index_178a48f5_3f50_4765_a6b7_1e7dc986412c_Node']}]
ERROR:app:Exception on /console/api/datasets/178a48f5-3f50-4765-a6b7-1e7dc986412c/hit-testing [POST]
Traceback (most recent call last):
  File "/app/api/controllers/console/datasets/hit_testing.py", line 88, in post
    response = HitTestingService.retrieve(
  File "/app/api/services/hit_testing_service.py", line 50, in retrieve
    documents = vector_index.search(
  File "/app/api/core/index/vector_index/base.py", line 58, in search
    docs_with_similarity = vector_store.similarity_search_with_relevance_scores(
  File "/usr/local/lib/python3.10/site-packages/langchain/vectorstores/base.py", line 141, in similarity_search_with_relevance_scores
    docs_and_similarities = self._similarity_search_with_relevance_scores(
  File "/usr/local/lib/python3.10/site-packages/langchain/vectorstores/weaviate.py", line 375, in _similarity_search_with_relevance_scores
    docs_and_scores = self.similarity_search_with_score(query, k=k, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/langchain/vectorstores/weaviate.py", line 349, in similarity_search_with_score
    raise ValueError(f"Error during query: {result['errors']}")
ValueError: Error during query: [{'locations': [{'column': 6, 'line': 1}], 'message': "explorer: get class: vector search: object vector search at index vector_index_178a48f5_3f50_4765_a6b7_1e7dc986412c_node: shard vector_index_178a48f5_3f50_4765_a6b7_1e7dc986412c_node_trp4Y9iz0jxH: vector search: knn search: distance between entrypoint and query node: vector lengths don't match: 4096 vs 768", 'path': ['Get', 'Vector_index_178a48f5_3f50_4765_a6b7_1e7dc986412c_Node']}]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1484, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1469, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/usr/local/lib/python3.10/site-packages/flask_restful/__init__.py", line 467, in wrapper
    resp = resource(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/flask/views.py", line 109, in view
    return current_app.ensure_sync(self.dispatch_request)(**kwargs)
  File "/usr/local/lib/python3.10/site-packages/flask_restful/__init__.py", line 582, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/app/api/controllers/console/setup.py", line 82, in decorated
    return view(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/flask_login/utils.py", line 290, in decorated_view
    return current_app.ensure_sync(func)(*args, **kwargs)
  File "/app/api/controllers/console/wraps.py", line 19, in decorated
    return view(*args, **kwargs)
  File "/app/api/controllers/console/datasets/hit_testing.py", line 106, in post
    raise InternalServerError(str(e))
werkzeug.exceptions.InternalServerError: 500 Internal Server Error: Error during query: [{'locations': [{'column': 6, 'line': 1}], 'message': "explorer: get class: vector search: object vector search at index vector_index_178a48f5_3f50_4765_a6b7_1e7dc986412c_node: shard vector_index_178a48f5_3f50_4765_a6b7_1e7dc986412c_node_trp4Y9iz0jxH: vector search: knn search: distance between entrypoint and query node: vector lengths don't match: 4096 vs 768", 'path': ['Get', 'Vector_index_178a48f5_3f50_4765_a6b7_1e7dc986412c_Node']}]
Originally created by @jeanlyn on GitHub (Jul 4, 2023). Originally assigned to: @takatost on GitHub. <!-- Please provide a clear and concise description of what the bug is. Include screenshots if needed. Please test using the latest version of the relevant Dify packages to make sure your issue has not already been fixed. --> Dify version: Self Host ## Steps To Reproduce <!-- Your bug will get fixed much faster if we can run your code and it doesn't have dependencies other than Dify. Issues without reproduction steps or code examples may be immediately closed as not actionable. --> 版本由0.3.1升级到0.3.6后，使用旧的数据集进行命中测试的时候会报以下的错误: <img width="1500" alt="image" src="https://github.com/langgenius/dify/assets/3426093/ed0d1b55-fe38-4711-8349-29cafb5e2680"> API报错如下: ``` ValueError: Error during query: [{'locations': [{'column': 6, 'line': 1}], 'message': "explorer: get class: vector search: object vector search at index vector_index_178a48f5_3f50_4765_a6b7_1e7dc986412c_node: shard vector_index_178a48f5_3f50_4765_a6b7_1e7dc986412c_node_trp4Y9iz0jxH: vector search: knn search: distance between entrypoint and query node: vector lengths don't match: 4096 vs 768", 'path': ['Get', 'Vector_index_178a48f5_3f50_4765_a6b7_1e7dc986412c_Node']}] ERROR:app:Exception on /console/api/datasets/178a48f5-3f50-4765-a6b7-1e7dc986412c/hit-testing [POST] Traceback (most recent call last): File "/app/api/controllers/console/datasets/hit_testing.py", line 88, in post response = HitTestingService.retrieve( File "/app/api/services/hit_testing_service.py", line 50, in retrieve documents = vector_index.search( File "/app/api/core/index/vector_index/base.py", line 58, in search docs_with_similarity = vector_store.similarity_search_with_relevance_scores( File "/usr/local/lib/python3.10/site-packages/langchain/vectorstores/base.py", line 141, in similarity_search_with_relevance_scores docs_and_similarities = self._similarity_search_with_relevance_scores( File "/usr/local/lib/python3.10/site-packages/langchain/vectorstores/weaviate.py", line 375, in _similarity_search_with_relevance_scores docs_and_scores = self.similarity_search_with_score(query, k=k, **kwargs) File "/usr/local/lib/python3.10/site-packages/langchain/vectorstores/weaviate.py", line 349, in similarity_search_with_score raise ValueError(f"Error during query: {result['errors']}") ValueError: Error during query: [{'locations': [{'column': 6, 'line': 1}], 'message': "explorer: get class: vector search: object vector search at index vector_index_178a48f5_3f50_4765_a6b7_1e7dc986412c_node: shard vector_index_178a48f5_3f50_4765_a6b7_1e7dc986412c_node_trp4Y9iz0jxH: vector search: knn search: distance between entrypoint and query node: vector lengths don't match: 4096 vs 768", 'path': ['Get', 'Vector_index_178a48f5_3f50_4765_a6b7_1e7dc986412c_Node']}] During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1484, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1469, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) File "/usr/local/lib/python3.10/site-packages/flask_restful/__init__.py", line 467, in wrapper resp = resource(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/flask/views.py", line 109, in view return current_app.ensure_sync(self.dispatch_request)(**kwargs) File "/usr/local/lib/python3.10/site-packages/flask_restful/__init__.py", line 582, in dispatch_request resp = meth(*args, **kwargs) File "/app/api/controllers/console/setup.py", line 82, in decorated return view(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/flask_login/utils.py", line 290, in decorated_view return current_app.ensure_sync(func)(*args, **kwargs) File "/app/api/controllers/console/wraps.py", line 19, in decorated return view(*args, **kwargs) File "/app/api/controllers/console/datasets/hit_testing.py", line 106, in post raise InternalServerError(str(e)) werkzeug.exceptions.InternalServerError: 500 Internal Server Error: Error during query: [{'locations': [{'column': 6, 'line': 1}], 'message': "explorer: get class: vector search: object vector search at index vector_index_178a48f5_3f50_4765_a6b7_1e7dc986412c_node: shard vector_index_178a48f5_3f50_4765_a6b7_1e7dc986412c_node_trp4Y9iz0jxH: vector search: knn search: distance between entrypoint and query node: vector lengths don't match: 4096 vs 768", 'path': ['Get', 'Vector_index_178a48f5_3f50_4765_a6b7_1e7dc986412c_Node']}] ```
yindo closed this issue 2026-02-21 17:26:30 -05:00
yindo commented 2026-02-21 17:26:30 -05:00
Author
Owner
Copy Link
Copy Source

@jeanlyn commented on GitHub (Jul 4, 2023):

/cc @takatost

@jeanlyn commented on GitHub (Jul 4, 2023): /cc @takatost
yindo commented 2026-02-21 17:26:30 -05:00
Author
Owner
Copy Link
Copy Source

@takatost commented on GitHub (Jul 4, 2023):

Have u made any changes to the code related to the embedding model? The error looks like it's caused by the vector length in the vector database being 4096, while the query length is 768.

@takatost commented on GitHub (Jul 4, 2023): Have u made any changes to the code related to the embedding model? The error looks like it's caused by the vector length in the vector database being 4096, while the query length is 768.
yindo commented 2026-02-21 17:26:30 -05:00
Author
Owner
Copy Link
Copy Source

@jeanlyn commented on GitHub (Jul 5, 2023):

We upgrade our model from ChatGLM to ChatGLM2, and wrap api to openai style, let me check more detail.

@jeanlyn commented on GitHub (Jul 5, 2023): We upgrade our model from ChatGLM to ChatGLM2, and wrap api to openai style, let me check more detail.
yindo commented 2026-02-21 17:26:30 -05:00
Author
Owner
Copy Link
Copy Source

@takatost commented on GitHub (Jul 5, 2023):

Looks like it's caused by changing the embedding model, the vector length is different from before

@takatost commented on GitHub (Jul 5, 2023): Looks like it's caused by changing the embedding model, the vector length is different from before
yindo commented 2026-02-21 17:26:30 -05:00
Author
Owner
Copy Link
Copy Source

@jeanlyn commented on GitHub (Jul 5, 2023):

Thanks @takatost , it is indeed the issue you mentioned.

@jeanlyn commented on GitHub (Jul 5, 2023): Thanks @takatost , it is indeed the issue you mentioned.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#242
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?