langgenius/dify-plugin-sdks
1
0
Fork 0
mirror of https://github.com/langgenius/dify-plugin-sdks.git synced 2026-07-22 02:15:22 -04:00
Code Issues 58 Packages Projects Releases 10 Wiki Activity

[Offline Deployment] Token Count Calculation Failure Due to GPT-2 Dependency #20

New Issue
Open
opened 2026-02-15 21:15:19 -05:00 by yindo · 0 comments
yindo commented 2026-02-15 21:15:19 -05:00
Owner
Copy Link
Copy Source

Originally created by @toddlerya on GitHub (Mar 20, 2025).

Bug Description

When deploying in an offline environment using docker-compose, an error occurs in token count calculation. The local model service does not provide token length calculation capabilities, so the system falls back to the _get_num_tokens_by_gpt2() method. However, in the offline environment, accessing the openaipublic.blob.core.windows.net service fails, resulting in network request errors .

Temporary Workaround

Modify the get_llm_num_tokens() and get_text_embedding_num_tokens() functions in dify_plugin/core_plugin_executor.py to directly use the len() method to calculate num_tokens and return the result.

Expectation

Provide an offline-friendly method for calculating token lengths without relying on GPT-2, enabling fully self-contained offline deployment .

Originally created by @toddlerya on GitHub (Mar 20, 2025). # Bug Description When deploying in an offline environment using docker-compose, an error occurs in token count calculation. The local model service does not provide token length calculation capabilities, so the system falls back to the `_get_num_tokens_by_gpt2()` method. However, in the offline environment, accessing the `openaipublic.blob.core.windows.net` service fails, resulting in network request errors . # Temporary Workaround Modify the `get_llm_num_tokens()` and `get_text_embedding_num_tokens()` functions in dify_plugin/core_plugin_executor.py to directly use the `len()` method to calculate num_tokens and return the result. # Expectation Provide an offline-friendly method for calculating token lengths without relying on GPT-2, enabling fully self-contained offline deployment .
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
documentation
enhancement
pull-request
Mirrored from GitHub Pull Request
 question
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-plugin-sdks#20
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?