mirror of
https://github.com/langgenius/dify-official-plugins.git
synced 2026-07-21 17:45:23 -04:00
The type get_num_tokens exception should be List[int] #25
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @AIgniteLUO on GitHub (Feb 27, 2025).
{"error_type":"unmarshal_error","message":"unmarshal json failed: json: cannot unmarshal number into Go struct field GetTextEmbeddingNumTokensResponse.num_tokens of type []int"}
@AIgniteLUO commented on GitHub (Mar 1, 2025):
Fixed
@huanghuajie1997 commented on GitHub (Apr 28, 2025):
dify version 1.3
i meet " PluginInvokeError: {"error_type":"unmarshal_error","message":"unmarshal json failed: json: cannot unmarshal number into Go struct field GetTextEmbeddingNumTokensResponse.num_tokens of type []int"}" when using my own text-embedding plugin, how to fit it out ,thanks! @AIgniteLUO
the whole exception:
worker-1 | 2025-04-28T08:58:20.559104519Z 2025-04-28 08:58:20.557 ERROR [Dummy-32] [indexing_runner.py:96] - consume document failed
worker-1 | 2025-04-28T08:58:20.559138730Z Traceback (most recent call last):
worker-1 | 2025-04-28T08:58:20.559143837Z File "/app/api/core/indexing_runner.py", line 77, in run
worker-1 | 2025-04-28T08:58:20.559148304Z self._load_segments(dataset, dataset_document, documents)
worker-1 | 2025-04-28T08:58:20.559152440Z File "/app/api/core/indexing_runner.py", line 723, in _load_segments
worker-1 | 2025-04-28T08:58:20.559157304Z doc_store.add_documents(docs=documents, save_child=dataset_document.doc_form == IndexType.PARENT_CHILD_INDEX)
worker-1 | 2025-04-28T08:58:20.559161619Z File "/app/api/core/rag/docstore/dataset_docstore.py", line 84, in add_documents
worker-1 | 2025-04-28T08:58:20.559165869Z tokens_list = embedding_model.get_text_embedding_num_tokens(page_content_list)
worker-1 | 2025-04-28T08:58:20.559170051Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1 | 2025-04-28T08:58:20.559176072Z File "/app/api/core/model_manager.py", line 244, in get_text_embedding_num_tokens
worker-1 | 2025-04-28T08:58:20.559182828Z self._round_robin_invoke(
worker-1 | 2025-04-28T08:58:20.559186780Z File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke
worker-1 | 2025-04-28T08:58:20.559190892Z return function(*args, **kwargs)
worker-1 | 2025-04-28T08:58:20.559195034Z ^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1 | 2025-04-28T08:58:20.559198922Z File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 65, in get_num_tokens
worker-1 | 2025-04-28T08:58:20.559203198Z return plugin_model_manager.get_text_embedding_num_tokens(
worker-1 | 2025-04-28T08:58:20.559223977Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1 | 2025-04-28T08:58:20.559227820Z File "/app/api/core/plugin/manager/model.py", line 313, in get_text_embedding_num_tokens
worker-1 | 2025-04-28T08:58:20.559232333Z for resp in response:
worker-1 | 2025-04-28T08:58:20.559235688Z ^^^^^^^^
worker-1 | 2025-04-28T08:58:20.559239022Z File "/app/api/core/plugin/manager/base.py", line 191, in _request_with_plugin_daemon_response_stream
worker-1 | 2025-04-28T08:58:20.559242805Z self._handle_plugin_daemon_error(error.error_type, error.message)
worker-1 | 2025-04-28T08:58:20.559246292Z File "/app/api/core/plugin/manager/base.py", line 223, in _handle_plugin_daemon_error
worker-1 | 2025-04-28T08:58:20.559249848Z raise PluginInvokeError(description=message)
worker-1 | 2025-04-28T08:58:20.559253312Z core.plugin.manager.exc.PluginInvokeError: PluginInvokeError: {"error_type":"unmarshal_error","message":"unmarshal json failed: json: cannot unmarshal number into Go struct field GetTextEmbeddingNumTokensResponse.num_tokens of type []int"}