langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-22 10:05:23 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

Align Gemini validate_credentials to use LLM generation API instead of count_tokens #958

New Issue
Closed
opened 2026-02-16 10:21:07 -05:00 by yindo · 1 comment
yindo commented 2026-02-16 10:21:07 -05:00
Owner
Copy Link
Copy Source

Originally created by @WH-2099 on GitHub (Jan 28, 2026).

Originally assigned to: @WH-2099 on GitHub.

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

The current Gemini plugin uses genai_client.models.count_tokens() API for credential validation, while other model providers (OpenAI, Anthropic, etc.) use the actual LLM generation API. This inconsistency may lead to situations where credentials pass validation but fail during actual usage.

2. Additional context or comments

No response

3. Can you help us with this feature?

  • I am interested in contributing to this feature.
Originally created by @WH-2099 on GitHub (Jan 28, 2026). Originally assigned to: @WH-2099 on GitHub. ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify-official-plugins/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] Please do not modify this template :) and fill in all the required fields. ### 1. Is this request related to a challenge you're experiencing? Tell me about your story. The current Gemini plugin uses `genai_client.models.count_tokens()` API for credential validation, while other model providers (OpenAI, Anthropic, etc.) use the actual LLM generation API. This inconsistency may lead to situations where credentials pass validation but fail during actual usage. ### 2. Additional context or comments _No response_ ### 3. Can you help us with this feature? - [x] I am interested in contributing to this feature.
yindo added the enhancement label 2026-02-16 10:21:07 -05:00
yindo closed this issue 2026-02-16 10:21:07 -05:00
yindo commented 2026-02-16 10:21:07 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jan 28, 2026):

To align Gemini's credential validation with other providers, update the validate_credentials method in models/gemini/models/llm/llm.py to use the LLM generation API instead of count_tokens. Currently, it calls genai_client.models.count_tokens(model=model, contents="ping") for validation, which can let invalid credentials pass if they only fail during actual LLM usage (source).

You should replace the count_tokens call with a minimal LLM generation call, such as genai_client.models.generate_content(model=model, contents=["ping"]), similar to how OpenAI and Anthropic providers do it (OpenAI example, Anthropic example). Make sure to handle exceptions by raising CredentialsValidateFailedError if validation fails, and log other exceptions as in the current pattern.

After making this change, test with both valid and invalid credentials to ensure the new validation logic correctly reflects real-world LLM API usage. This will prevent credentials that only work for token counting but fail for actual generation from being accepted. Update any related comments or documentation to clarify the new approach.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

Open in Cursor Join Discord Share on X

@dosubot[bot] commented on GitHub (Jan 28, 2026): <!-- Answer --> To align Gemini's credential validation with other providers, update the validate_credentials method in models/gemini/models/llm/llm.py to use the LLM generation API instead of count_tokens. Currently, it calls genai_client.models.count_tokens(model=model, contents="ping") for validation, which can let invalid credentials pass if they only fail during actual LLM usage ([source](https://github.com/langgenius/dify-official-plugins/blob/86735a97132b1a22a0d2d8b9e0944cbffea1b674/models/gemini/models/llm/llm.py#L61-L1093)). You should replace the count_tokens call with a minimal LLM generation call, such as genai_client.models.generate_content(model=model, contents=["ping"]), similar to how OpenAI and Anthropic providers do it ([OpenAI example](https://github.com/langgenius/dify-official-plugins/blob/86735a97132b1a22a0d2d8b9e0944cbffea1b674/models/openai/provider/openai.py#L11-L33), [Anthropic example](https://github.com/langgenius/dify-official-plugins/blob/86735a97132b1a22a0d2d8b9e0944cbffea1b674/models/anthropic/provider/anthropic.py#L9-L26)). Make sure to handle exceptions by raising CredentialsValidateFailedError if validation fails, and log other exceptions as in the current pattern. After making this change, test with both valid and invalid credentials to ensure the new validation logic correctly reflects real-world LLM API usage. This will prevent credentials that only work for token counting but fail for actual generation from being accepted. Update any related comments or documentation to clarify the new approach. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/c9c6aba2-1de4-4a06-bc72-f5f660e00467?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/c9c6aba2-1de4-4a06-bc72-f5f660e00467?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/c9c6aba2-1de4-4a06-bc72-f5f660e00467?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/c9c6aba2-1de4-4a06-bc72-f5f660e00467?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/c9c6aba2-1de4-4a06-bc72-f5f660e00467?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/c9c6aba2-1de4-4a06-bc72-f5f660e00467?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/c9c6aba2-1de4-4a06-bc72-f5f660e00467?feedback_type=other)</sup> [![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=To%20align%20Gemini%27s%20credential%20validation%20with%20other%20providers%2C%20update%20the%20validate_credentials%20method%20in%20models/gemini/models/llm/llm.py%20to%20use%20the%20LLM%20generation%20API%20instead%20of%20count_tokens.%20Currently%2C%20it%20calls%20genai_client.models.count_tokens%28model%3Dmodel%2C%20contents%3D%22ping%22%29%20for%20validation%2C%20which%20can%20let%20invalid%20credentials%20pass%20if%20they%20only%20fail%20during%20actual%20LLM%20usage%C2%A0%28%5Bsource%5D%28https%3A//github.com/langgenius/dify-official-plugins/blob/86735a97132b1a22a0d2d8b9e0944cbffea1b674/models/gemini/models/llm/llm.py%23L61-L1093%29%29.%0A%0AYou%20should%20replace%20the%20count_tokens%20call%20with%20a%20minimal%20LLM%20generation%20call%2C%20such%20as%20genai_client.models.generate_content%28model%3Dmodel%2C%20contents%3D%5B%22ping%22%5D%29%2C%20similar%20to%20how%20OpenAI%20and%20Anthropic%20providers%20do%20it%20%28%5BOpenAI%20example%5D%28https%3A//github.com/langgenius/dify-official-plugins/blob/86735a97132b1a22a0d2d8b9e0944cbffea1b674/models/openai/provider/openai.py%23L11-L33%29%2C%20%5BAnthropic%20example%5D%28https%3A//github.com/langgenius/dify-official-plugins/blob/86735a97132b1a22a0d2d8b9e0944cbffea1b674/models/anthropic/provider/anthropic.py%23L9-L26%29%29.%20Make%20sure%20to%20handle%20exceptions%20by%20raising%20CredentialsValidateFailedError%20if%20validation%20fails%2C%20and%20log%20other%20exceptions%20as%20in%20the%20current%20pattern.%0A%0AAfter%20making%20this%20change%2C%20test%20with%20both%20valid%20and%20invalid%20credentials%20to%20ensure%20the%20new%20validation%20logic%20correctly%20reflects%20real-world%20LLM%20API%20usage.%20This%20will%20prevent%20credentials%20that%20only%20work%20for%20token%20counting%20but%20fail%20for%20actual%20generation%20from%20being%20accepted.%20Update%20any%20related%20comments%20or%20documentation%20to%20clarify%20the%20new%20approach.)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify-official-plugins/issues/2516)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#958
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?