langgenius/dify-plugin-sdks
1
0
Fork 0
mirror of https://github.com/langgenius/dify-plugin-sdks.git synced 2026-07-22 02:15:22 -04:00
Code Issues 58 Packages Projects Releases 10 Wiki Activity

Is there a way to access session.storage inside a Model Provider? #64

New Issue
Closed
opened 2026-02-15 21:15:31 -05:00 by yindo · 4 comments
yindo commented 2026-02-15 21:15:31 -05:00
Owner
Copy Link
Copy Source

Originally created by @AkiraVoid on GitHub (Dec 4, 2025).

I want to persist the API key provided in a provider credential and reuse it in model invocation. But it seems that there's no self.session property inside ModelProvider, is there a way to access the storage or any other solution to persist the API key in provider credential?

Originally created by @AkiraVoid on GitHub (Dec 4, 2025). I want to persist the API key provided in a provider credential and reuse it in model invocation. But it seems that there's no self.session property inside ModelProvider, is there a way to access the storage or any other solution to persist the API key in provider credential?
yindo added the question label 2026-02-15 21:15:31 -05:00
yindo closed this issue 2026-02-15 21:15:31 -05:00
yindo commented 2026-02-15 21:15:32 -05:00
Author
Owner
Copy Link
Copy Source

@AkiraVoid commented on GitHub (Dec 7, 2025):

Also posted as a discussion in dify's main repo: https://github.com/langgenius/dify/discussions/29249

@AkiraVoid commented on GitHub (Dec 7, 2025): Also posted as a discussion in dify's main repo: https://github.com/langgenius/dify/discussions/29249
yindo commented 2026-02-15 21:15:32 -05:00
Author
Owner
Copy Link
Copy Source

@Mairuis commented on GitHub (Dec 8, 2025):

Hi @AkiraVoid,

You're correct that ModelProvider and AIModel classes do not have access to session.storage. This is by design - Model providers are intended to be stateless.

Why Model Providers don't have session access

Looking at the SDK architecture:

  • Tool class has self.session → can access session.storage
  • Endpoint class has self.session → can access session.storage
  • ModelProvider class does NOT have session → no storage access
  • AIModel (LargeLanguageModel, etc.) does NOT have session → no storage access

This is intentional because model invocations are designed to be stateless and credentials are already passed through the credentials parameter in each method call.

Your use case

If you want to persist an API key from provider credentials and reuse it in model invocations, you actually already have access to it:

  1. In validate_provider_credentials(credentials) - you receive the credentials dict
  2. In LargeLanguageModel._invoke(model, credentials, ...) - you also receive the credentials dict

The credentials are passed through every invocation, so you don't need to persist them separately. The credentials dict should contain your API key that was configured when the user added the model provider.

class MyLLM(LargeLanguageModel):
    def _invoke(self, model: str, credentials: dict, prompt_messages: list[PromptMessage], ...):
        api_key = credentials.get("api_key")
        # Use the api_key directly - no need to store it

If you truly need persistent storage

If your use case requires persistent storage (e.g., caching tokens, storing session state), you would need to:

  1. Use a Tool or Endpoint instead, which have session access
  2. Implement your own external storage (e.g., file-based, Redis, etc.) outside the SDK's session.storage

Could you share more details about why you need to persist the API key separately? There might be a better solution depending on your specific use case.

@Mairuis commented on GitHub (Dec 8, 2025): Hi @AkiraVoid, You're correct that `ModelProvider` and `AIModel` classes do not have access to `session.storage`. This is by design - Model providers are intended to be stateless. ## Why Model Providers don't have session access Looking at the SDK architecture: - **`Tool`** class has `self.session` → can access `session.storage` - **`Endpoint`** class has `self.session` → can access `session.storage` - **`ModelProvider`** class does NOT have session → no storage access - **`AIModel`** (LargeLanguageModel, etc.) does NOT have session → no storage access This is intentional because model invocations are designed to be stateless and credentials are already passed through the `credentials` parameter in each method call. ## Your use case If you want to persist an API key from provider credentials and reuse it in model invocations, you actually already have access to it: 1. **In `validate_provider_credentials(credentials)`** - you receive the credentials dict 2. **In `LargeLanguageModel._invoke(model, credentials, ...)`** - you also receive the credentials dict The credentials are passed through every invocation, so you don't need to persist them separately. The credentials dict should contain your API key that was configured when the user added the model provider. ```python class MyLLM(LargeLanguageModel): def _invoke(self, model: str, credentials: dict, prompt_messages: list[PromptMessage], ...): api_key = credentials.get("api_key") # Use the api_key directly - no need to store it ``` ## If you truly need persistent storage If your use case requires persistent storage (e.g., caching tokens, storing session state), you would need to: 1. **Use a Tool or Endpoint** instead, which have session access 2. **Implement your own external storage** (e.g., file-based, Redis, etc.) outside the SDK's session.storage Could you share more details about why you need to persist the API key separately? There might be a better solution depending on your specific use case.
yindo commented 2026-02-15 21:15:32 -05:00
Author
Owner
Copy Link
Copy Source

@AkiraVoid commented on GitHub (Dec 9, 2025):

Thanks, @Mairuis

But actually, the credentials dict passed to AIModel._invoke does not have the API key parameter defined in provider_credential_schema of provider.yaml, this is confirmed by kurokobo in discussion https://github.com/langgenius/dify/discussions/29249.

What I need is an ability to access the API key configured in provider level, so the users can configure it once and add multiple models which use that key to send requests (which means the API key input cannot be defined in a model_credential_schema). Also, as mentioned in that discussion, the API key input defined in the model_credential_schema is not safe.

@AkiraVoid commented on GitHub (Dec 9, 2025): Thanks, @Mairuis But actually, the `credentials` dict passed to `AIModel._invoke` does not have the API key parameter defined in `provider_credential_schema` of `provider.yaml`, this is confirmed by kurokobo in discussion https://github.com/langgenius/dify/discussions/29249. What I need is an ability to access the API key **configured in provider level**, so the users can configure it once and add multiple models which use that key to send requests (which means the API key input cannot be defined in a `model_credential_schema`). Also, as mentioned in that discussion, the API key input defined in the `model_credential_schema` is not safe.
yindo commented 2026-02-15 21:15:32 -05:00
Author
Owner
Copy Link
Copy Source

@AkiraVoid commented on GitHub (Dec 10, 2025):

Hi, @Mairuis

I am wondering if there's any plan for us to achieve this? Or do I need to open another issue in another repository for this problem?

@AkiraVoid commented on GitHub (Dec 10, 2025): Hi, @Mairuis I am wondering if there's any plan for us to achieve this? Or do I need to open another issue in another repository for this problem?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
documentation
enhancement
pull-request
Mirrored from GitHub Pull Request
 question
No Label question
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-plugin-sdks#64
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?