langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-21 17:45:23 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

Azure_openai text embedding not working #42

New Issue
Closed
opened 2026-02-16 10:17:50 -05:00 by yindo · 1 comment
yindo commented 2026-02-16 10:17:50 -05:00
Owner
Copy Link
Copy Source

Originally created by @snicholas on GitHub (Mar 5, 2025).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues Dify issues & Dify Official Plugins, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.0.0

Plugin version

0.04

Cloud or Self Hosted

Self Hosted (Source), Self Hosted (Docker)

Steps to reproduce

After setting up ada2-embedding the indexing fail with error. It seems to expect a list of integers, but gets back an int.

✔️ Error log

2025-03-05 12:30:48 2025-03-05 11:30:48.928 ERROR [Dummy-1] [app.py:875] - Exception on /console/api/datasets/indexing-estimate [POST]
2025-03-05 12:30:48 Traceback (most recent call last):
2025-03-05 12:30:48 File "/app/api/controllers/console/datasets/datasets.py", line 452, in post
2025-03-05 12:30:48 response = indexing_runner.indexing_estimate(
2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 12:30:48 File "/app/api/core/indexing_runner.py", line 290, in indexing_estimate
2025-03-05 12:30:48 documents = index_processor.transform(
2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 12:30:48 File "/app/api/core/rag/index_processor/processor/paragraph_index_processor.py", line 58, in transform
2025-03-05 12:30:48 document_nodes = splitter.split_documents([document])
2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 12:30:48 File "/app/api/core/rag/splitter/text_splitter.py", line 96, in split_documents
2025-03-05 12:30:48 return self.create_documents(texts, metadatas=metadatas)
2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 12:30:48 File "/app/api/core/rag/splitter/text_splitter.py", line 81, in create_documents
2025-03-05 12:30:48 for chunk in self.split_text(text):
2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^
2025-03-05 12:30:48 File "/app/api/core/rag/splitter/fixed_text_splitter.py", line 68, in split_text
2025-03-05 12:30:48 chunks_lengths = self._length_function(chunks)
2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 12:30:48 File "/app/api/core/rag/splitter/fixed_text_splitter.py", line 38, in _token_encoder
2025-03-05 12:30:48 return embedding_model_instance.get_text_embedding_num_tokens(texts=texts)
2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 12:30:48 File "/app/api/core/model_manager.py", line 244, in get_text_embedding_num_tokens
2025-03-05 12:30:48 self._round_robin_invoke(
2025-03-05 12:30:48 File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke
2025-03-05 12:30:48 return function(*args, **kwargs)
2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 12:30:48 File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 65, in get_num_tokens
2025-03-05 12:30:48 return plugin_model_manager.get_text_embedding_num_tokens(
2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 12:30:48 File "/app/api/core/plugin/manager/model.py", line 313, in get_text_embedding_num_tokens
2025-03-05 12:30:48 for resp in response:
2025-03-05 12:30:48 ^^^^^^^^
2025-03-05 12:30:48 File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream
2025-03-05 12:30:48 self._handle_plugin_daemon_error(error.error_type, error.message)
2025-03-05 12:30:48 File "/app/api/core/plugin/manager/base.py", line 221, in handle_plugin_daemon_error
2025-03-05 12:30:48 raise PluginInvokeError(description=message)
2025-03-05 12:30:48 core.plugin.manager.exc.PluginInvokeError: PluginInvokeError: {"error_type":"unmarshal_error","message":"unmarshal json failed: json: cannot unmarshal number into Go struct field GetTextEmbeddingNumTokensResponse.num_tokens of type []int"}
2025-03-05 12:30:48
2025-03-05 12:30:48 During handling of the above exception, another exception occurred:
2025-03-05 12:30:48
2025-03-05 12:30:48 Traceback (most recent call last):
2025-03-05 12:30:48 File "/app/api/.venv/lib/python3.12/site-packages/flask/app.py", line 917, in full_dispatch_request
2025-03-05 12:30:48 rv = self.dispatch_request()
2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 12:30:48 File "/app/api/.venv/lib/python3.12/site-packages/flask/app.py", line 902, in dispatch_request
2025-03-05 12:30:48 return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 12:30:48 File "/app/api/.venv/lib/python3.12/site-packages/flask_restful/__init__.py", line 489, in wrapper
2025-03-05 12:30:48 resp = resource(*args, **kwargs)
2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 12:30:48 File "/app/api/.venv/lib/python3.12/site-packages/flask/views.py", line 110, in view
2025-03-05 12:30:48 return current_app.ensure_sync(self.dispatch_request)(**kwargs) # type: ignore[no-any-return]
2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 12:30:48 File "/app/api/.venv/lib/python3.12/site-packages/flask_restful/__init_.py", line 604, in dispatch_request
2025-03-05 12:30:48 resp = meth(*args, **kwargs)
2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^
2025-03-05 12:30:48 File "/app/api/controllers/console/wraps.py", line 147, in decorated
2025-03-05 12:30:48 return view(*args, **kwargs)
2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^
2025-03-05 12:30:48 File "/app/api/libs/login.py", line 94, in decorated_view
2025-03-05 12:30:48 return current_app.ensure_sync(func)(*args, **kwargs)
2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-05 12:30:48 File "/app/api/controllers/console/wraps.py", line 27, in decorated
2025-03-05 12:30:48 return view(*args, **kwargs)
2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^
2025-03-05 12:30:48 File "/app/api/controllers/console/datasets/datasets.py", line 468, in post
2025-03-05 12:30:48 raise IndexingEstimateError(str(e))
2025-03-05 12:30:48 controllers.console.datasets.error.IndexingEstimateError: 500 Internal Server Error: PluginInvokeError: {"error_type":"unmarshal_error","message":"unmarshal json failed: json: cannot unmarshal number into Go struct field GetTextEmbeddingNumTokensResponse.num_tokens of type []int"}

Originally created by @snicholas on GitHub (Mar 5, 2025). ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [Dify issues](https://github.com/langgenius/dify/issues) & [Dify Official Plugins](https://github.com/langgenius/dify-official-plugins/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.0.0 ### Plugin version 0.04 ### Cloud or Self Hosted Self Hosted (Source), Self Hosted (Docker) ### Steps to reproduce After setting up ada2-embedding the indexing fail with error. It seems to expect a list of integers, but gets back an int. ### ✔️ Error log 2025-03-05 12:30:48 2025-03-05 11:30:48.928 ERROR [Dummy-1] [app.py:875] - Exception on /console/api/datasets/indexing-estimate [POST] 2025-03-05 12:30:48 Traceback (most recent call last): 2025-03-05 12:30:48 File "/app/api/controllers/console/datasets/datasets.py", line 452, in post 2025-03-05 12:30:48 response = indexing_runner.indexing_estimate( 2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-05 12:30:48 File "/app/api/core/indexing_runner.py", line 290, in indexing_estimate 2025-03-05 12:30:48 documents = index_processor.transform( 2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-05 12:30:48 File "/app/api/core/rag/index_processor/processor/paragraph_index_processor.py", line 58, in transform 2025-03-05 12:30:48 document_nodes = splitter.split_documents([document]) 2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-05 12:30:48 File "/app/api/core/rag/splitter/text_splitter.py", line 96, in split_documents 2025-03-05 12:30:48 return self.create_documents(texts, metadatas=metadatas) 2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-05 12:30:48 File "/app/api/core/rag/splitter/text_splitter.py", line 81, in create_documents 2025-03-05 12:30:48 for chunk in self.split_text(text): 2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^ 2025-03-05 12:30:48 File "/app/api/core/rag/splitter/fixed_text_splitter.py", line 68, in split_text 2025-03-05 12:30:48 chunks_lengths = self._length_function(chunks) 2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-05 12:30:48 File "/app/api/core/rag/splitter/fixed_text_splitter.py", line 38, in _token_encoder 2025-03-05 12:30:48 return embedding_model_instance.get_text_embedding_num_tokens(texts=texts) 2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-05 12:30:48 File "/app/api/core/model_manager.py", line 244, in get_text_embedding_num_tokens 2025-03-05 12:30:48 self._round_robin_invoke( 2025-03-05 12:30:48 File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke 2025-03-05 12:30:48 return function(*args, **kwargs) 2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-05 12:30:48 File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 65, in get_num_tokens 2025-03-05 12:30:48 return plugin_model_manager.get_text_embedding_num_tokens( 2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-05 12:30:48 File "/app/api/core/plugin/manager/model.py", line 313, in get_text_embedding_num_tokens 2025-03-05 12:30:48 for resp in response: 2025-03-05 12:30:48 ^^^^^^^^ 2025-03-05 12:30:48 File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream 2025-03-05 12:30:48 self._handle_plugin_daemon_error(error.error_type, error.message) 2025-03-05 12:30:48 File "/app/api/core/plugin/manager/base.py", line 221, in _handle_plugin_daemon_error 2025-03-05 12:30:48 raise PluginInvokeError(description=message) 2025-03-05 12:30:48 core.plugin.manager.exc.PluginInvokeError: PluginInvokeError: {"error_type":"unmarshal_error","message":"unmarshal json failed: json: cannot unmarshal number into Go struct field GetTextEmbeddingNumTokensResponse.num_tokens of type []int"} 2025-03-05 12:30:48 2025-03-05 12:30:48 During handling of the above exception, another exception occurred: 2025-03-05 12:30:48 2025-03-05 12:30:48 Traceback (most recent call last): 2025-03-05 12:30:48 File "/app/api/.venv/lib/python3.12/site-packages/flask/app.py", line 917, in full_dispatch_request 2025-03-05 12:30:48 rv = self.dispatch_request() 2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-05 12:30:48 File "/app/api/.venv/lib/python3.12/site-packages/flask/app.py", line 902, in dispatch_request 2025-03-05 12:30:48 return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return] 2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-05 12:30:48 File "/app/api/.venv/lib/python3.12/site-packages/flask_restful/__init__.py", line 489, in wrapper 2025-03-05 12:30:48 resp = resource(*args, **kwargs) 2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-05 12:30:48 File "/app/api/.venv/lib/python3.12/site-packages/flask/views.py", line 110, in view 2025-03-05 12:30:48 return current_app.ensure_sync(self.dispatch_request)(**kwargs) # type: ignore[no-any-return] 2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-05 12:30:48 File "/app/api/.venv/lib/python3.12/site-packages/flask_restful/__init__.py", line 604, in dispatch_request 2025-03-05 12:30:48 resp = meth(*args, **kwargs) 2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^ 2025-03-05 12:30:48 File "/app/api/controllers/console/wraps.py", line 147, in decorated 2025-03-05 12:30:48 return view(*args, **kwargs) 2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^ 2025-03-05 12:30:48 File "/app/api/libs/login.py", line 94, in decorated_view 2025-03-05 12:30:48 return current_app.ensure_sync(func)(*args, **kwargs) 2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-05 12:30:48 File "/app/api/controllers/console/wraps.py", line 27, in decorated 2025-03-05 12:30:48 return view(*args, **kwargs) 2025-03-05 12:30:48 ^^^^^^^^^^^^^^^^^^^^^ 2025-03-05 12:30:48 File "/app/api/controllers/console/datasets/datasets.py", line 468, in post 2025-03-05 12:30:48 raise IndexingEstimateError(str(e)) 2025-03-05 12:30:48 controllers.console.datasets.error.IndexingEstimateError: 500 Internal Server Error: PluginInvokeError: {"error_type":"unmarshal_error","message":"unmarshal json failed: json: cannot unmarshal number into Go struct field GetTextEmbeddingNumTokensResponse.num_tokens of type []int"}
yindo added the bug label 2026-02-16 10:17:50 -05:00
yindo closed this issue 2026-02-16 10:17:50 -05:00
yindo commented 2026-02-16 10:17:50 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Mar 5, 2025):

Should be fixed in https://github.com/langgenius/dify-official-plugins/pull/385, you can upgrade your plugin version now.

@crazywoola commented on GitHub (Mar 5, 2025): Should be fixed in https://github.com/langgenius/dify-official-plugins/pull/385, you can upgrade your plugin version now.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#42
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?