* fix latency calc
* feat: add timing context management to AIModel for latency tracking
- Introduced a new TimingContextRaceConditionError to handle race conditions in multi-threaded environments.
- Implemented a timing_context method in AIModel to track request timing and prevent concurrent usage.
- Updated various model classes (LargeLanguageModel, ModerationModel, RerankModel, Speech2TextModel, TextEmbeddingModel, TTSModel) to utilize the new timing context for latency calculations.
- Added tests to validate timing context behavior under concurrent and sequential usage scenarios.
* feat: implement ModelFactory for dynamic model instance creation
- Added ModelFactory class to generate stateless model instances based on provider configurations and model classes.
- Updated PluginRegistration to utilize ModelFactory for managing model instances, enhancing code organization and maintainability.
* feat: add unit tests for model registry and mock model provider
- Introduced a new test file for validating model registry functionality.
- Implemented mock classes for model provider and LLM to facilitate testing.
- Added tests to ensure correct model instance retrieval from the registry.
* apply ruff
---------
Co-authored-by: Yeuoly <admin@srmxy.cn>
* feat: add filename support to multi-modal prompt messages
- Add filename field to MultiModalPromptMessageContent class
- Field defaults to empty string for backward compatibility
- Inherited by all multi-modal content types (Image, Video, Audio, Document)
- Aligns with main Dify repository implementation (PR #24777)
Closes#196🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(README.md): update version binding
Signed-off-by: -LAN- <laipz8200@outlook.com>
---------
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: Claude <noreply@anthropic.com>
* chore: fix ruff issue
* feat(oauth): implement OAuth
* feat(invoke-message): refactor message handling and introduce InvokeMessage class
* feat(plugin-oauth): add credential_id and credential_type to tool parameters
* feat(plugin-oauth): add credential_id and credential_type to tool parameters
* chore: update dify_plugin version to 0.5.0b4 and clean up github.yaml
* chore: update plugin version to 0.1.2 in manifest.yaml
* feat(session): session context and tool backwards invocation credential support
* feat(oauth): session context and tool backwards invocation credential support
* feat: update README and requirements for OAuth support in version 0.4.2
* feat: add .gitignore to exclude IDE files and secret keys
* chore: apply ruff
* feat: bump version to 0.4.2b1
* feat: update GitHub plugin configuration for OAuth support and improve credential handling
* feat: update .gitignore to exclude dify plugin files and public keys
* feat: fix credential validation for GitHub API and bump version to 0.2.1
* feat: update GitHub plugin to support multiple access tokens and bump version to 0.2.5
* chore: apply ruff
* feat: add ToolProviderOAuthError for improved OAuth error handling in GitHub plugin
* chore: apply ruff
* chore: bump version to 0.4.2
* chore: update examples sdk version to 0.4.2
* fix: thread deadlock in PluginRunner when running tests without gevent monkey patching
* feat: add support for refreshing OAuth credentials in Plugin and GitHub provider
* feat: refactor OAuth credential handling to return structured OAuthCredentials object
* apply ruff
* feat: refactor OAuth credential handling to use ToolOAuthCredentials for improved structure
* feat: reorganize imports in __init__.py for improved clarity and structure
* feat: add Microsoft To Do plugin for refresh token example
* chore: apply ruff
* fix: update author in GitHub configuration and clean up Microsoft To Do schema
* chore: bump version to 0.4.2b2 in pyproject.toml
* feat: update Microsoft To Do plugin to handle OAuth token encoding and version bump
* feat:remove inelegant example
* chore: update dify_plugin version to 0.4.2
* chore: bump version to 0.4.2 in pyproject.toml
---------
Co-authored-by: Yeuoly <admin@srmxy.cn>
* docs: update README.md to include support for `dark-icon` field in manifest versioning
* feat: add dark mode icon fields to Tool and SimpleProviderEntity models
- Introduced `icon_dark` field in ToolProviderIdentity for dark mode support.
- Added `icon_small_dark` and `icon_large_dark` fields in SimpleProviderEntity for dark mode icons.
* fix: Add response body to CredentialsValidateFailedError exceptions to aid in error diagnosis, especially useful for locally provided models
* fix: linter
---------
Co-authored-by: liull <liulili@thunisoft.com>
Co-authored-by: Yeuoly <admin@srmxy.cn>
* feat: add dynamic select support
- Introduced ParameterOption class for parameter options with value, label, and optional icon.
- Updated ToolParameterOption to inherit from ParameterOption.
- Added DYNAMIC_SELECT to CommonParameterType enum.
- Implemented fetch_parameter_options method in Tool class for fetching parameter options.
- Created DynamicSelectProtocol for dynamic select functionality.
- Added tests for Tool construction and parameter options fetching.
* feat: implement dynamic parameter fetching functionality
- Added DynamicParameterFetchParameterOptionsRequest class for dynamic parameter requests.
- Introduced DynamicParameterActions enum for action types related to dynamic parameters.
- Enhanced PluginExecutor with methods to handle dynamic parameter fetching.
- Updated Tool interface to include a method for fetching parameter options.
- Documented the DynamicSelectProtocol for clarity on its usage.
* feat: enhance plugin functionality with dynamic parameter options and trigger interface
- Added dynamic parameter fetching capabilities to the Plugin class.
- Introduced TriggerProvider and Trigger classes for managing trigger functionalities.
- Updated Tool interface to enforce implementation of credential validation and OAuth methods.
- Enhanced error messages for NotImplementedError in ToolProvider and Trigger classes.
- Removed deprecated tools and their configurations from the codebase.
* docs: update README.md to include support for `dynamic-select` parameter type in manifest versioning
* chore: remove code_based_workflow
* feat: add LLMStructuredOutput and LLMStructuredOutputInvocation for structured output handling
- Introduced LLMStructuredOutput model class for structured output representation.
- Added LLMStructuredOutputInvocation class to handle invocation with structured output.
- Updated ModelInvocations to include LLMStructuredOutputInvocation.
- Modified InvokeType enum to support LLMStructuredOutput type.
* docs: update README.md to include support for LLM structured output in manifest versioning
* docs: update README.md with manifest versioning details and enhance setup.py field descriptions for version compatibility
* docs: enhance README.md with manifest versioning details and update Python SDK README for version constraints
* feat: enhance PluginRunner to build plugins from source directories
- Added functionality to the PluginRunner to detect if the plugin package path is a directory and build the plugin accordingly.
- Introduced a new method `_build_plugin` to handle the building process and log the output.
- Implemented resource cleanup for temporary directories used during the build process.
- Updated integration tests to reflect changes in the mock server configuration by using a constant for the server port.
* fix: linter
* feat(oauth): implement OAuth authorization URL and credentials retrieval
- Added OAuthGetAuthorizationUrlRequest and OAuthGetCredentialsRequest models for handling OAuth requests.
- Introduced methods in PluginExecutor for generating OAuth authorization URLs and retrieving OAuth credentials.
- Updated ToolProvider interface to include OAuth methods for authorization URL and credentials handling.
- Refactored PluginInvokeType and related action enums to support OAuth actions.
* feat: add integration configuration and plugin runner functionality
- Introduced `IntegrationConfig` class for managing plugin testing configurations, including validation for the `dify_cli_path`.
- Added `PluginInvokeRequest` and `PluginGenericResponse` models to handle plugin invocation requests and responses.
- Implemented `PluginRunner` class to facilitate running plugins locally, including asynchronous message handling and process management.
- Merged access actions into a single `PluginAccessAction` for streamlined access control.
* apply ruff
* feat: enhance plugin runner with new response handling and exception management
- Added `PLUGIN_INVOKE_END` response type to manage the end of plugin invocations.
- Introduced `PluginStopped` exception to handle plugin termination gracefully.
- Updated `PluginRunner` to support stopping the plugin process and handling asynchronous message reading with improved error management.
- Modified message queue to accommodate `None` responses for completed invocations.
* refactor: rename PluginStopped exception to PluginStoppedError
- Renamed `PluginStopped` to `PluginStoppedError` for clarity and consistency.
- Updated references in the `run.py` file to reflect the new exception name.
- Added a docstring to `PluginStoppedError` to describe its purpose.
* fix: update version check for dify CLI in IntegrationConfig
- Modified the version check in the IntegrationConfig class to require a minimum version of 0.1.0 instead of 0.4.0.
- Ensured that the version validation logic is clearer and more robust.
* chore: update dependencies and lock file
- Updated the `content_hash` in `pdm.lock` to reflect changes in dependencies.
- Added `packaging>=25.0` to the dependencies in `pyproject.toml` to ensure compatibility with the latest packaging features.
* feat: enhance IntegrationConfig with improved CLI path validation
- Added a list of potential plugin names to search for the dify CLI executable.
- Updated the validation logic to check for the CLI path against multiple plugin names.
- Changed error handling to use ValidationError for invalid CLI path and version checks.
* feat: add setup script for dify-plugin-cli installation
- Introduced a new script to automate the installation of dify-plugin-cli, including OS and architecture detection.
- Updated the pull request workflow to include the setup of the dify-plugin-cli before installing dependencies.
- Refactored error handling in IntegrationConfig to use ValueError instead of ValidationError for CLI path and version checks.
* feat: add integration test for invoking LLM with mocked OpenAI server
- Introduced a new integration test in `test_invoke_llm.py` to validate the invocation of the LLM plugin using a mocked OpenAI server.
- Implemented a mock server to simulate OpenAI's chat completions API, supporting both streaming and non-streaming responses.
- The test downloads the latest `langgenius-openai` plugin, runs the plugin, and asserts the expected response from the LLM.
* fix: add timeout to requests in LLM integration test
- Updated the `test_invoke_llm.py` file to include a timeout of 10 seconds for both POST and GET requests to the marketplace API.
- This change aims to improve the reliability of the integration test by preventing indefinite blocking on network calls.
* feat: enhance dify-plugin-cli setup script with installation verification and PATH configuration
- Added verification to check if dify-plugin is available in the user's PATH after installation.
- Implemented logic to add dify-plugin to the system PATH for all users when run as root, or to the user's profile for non-root users.
- Created symlinks in common bin directories and provided usage instructions for the dify-plugin commands.
* refactor: update dify-plugin-cli setup script for improved PATH management
- Enhanced the script to add the installation directory to the GITHUB_PATH for GitHub Actions.
- Modified the PATH export for local development to ensure the installation directory is prioritized.
- Removed redundant installation verification and symlink creation logic, streamlining the setup process.
* test: add SSH support to pull request workflow
- Integrated SSH functionality into the pull request workflow using the mxschmitt/action-tmate@v3 action.
- This addition allows for interactive debugging and terminal access during pull request checks.
* chore: remove SSH step from pull request workflow and fix import in integration test
- Eliminated the SSH step from the pull request workflow to streamline the process.
- Fixed the import statement for the `requests` library in `test_invoke_llm.py` to ensure proper functionality during the test execution.
* wtf
* feat: implement OpenAI mock server for integration testing
- Added a new mock server in `python/tests/__mock_server` to simulate OpenAI's chat completions API, supporting both streaming and non-streaming responses.
- Updated the pull request workflow to install the `uv` tool and launch the mock server during testing.
- Refactored `test_invoke_llm.py` to utilize the mock server, ensuring reliable integration tests without external dependencies.
* apply ruff
* refactor: streamline mock server launch in pull request workflow
- Replaced inline mock server launch commands in the pull request workflow with a dedicated script `launch_mock_server.sh` for improved readability and maintainability.
- The new script handles the execution of the mock server, ensuring a cleaner workflow configuration.
* fix: update mock server launch path in pull request workflow
- Changed the path for launching the mock server script from `./scripts/launch_mock_server.sh` to `./python/scripts/launch_mock_server.sh` to ensure correct execution within the workflow.
* fix: run mock server in background during pull request workflow
- Updated the pull request workflow to launch the mock server script in the background by appending `&` to the command. This change allows subsequent steps to execute without waiting for the mock server to terminate.
* chore: clean up dify-plugin-cli setup script by removing outdated installation instructions
- Removed outdated comments regarding the download and installation of dify-plugin-cli, streamlining the script for clarity.
* feat: enhance PluginRunner with graceful shutdown and thread safety
- Introduced a stop flag and a lock to ensure thread-safe access to the stop flag in the PluginRunner class.
- Implemented a _close method to handle graceful termination of the plugin process, including sending a SIGTERM signal and closing pipes.
- Refactored the _read_async method to ensure proper cleanup and handling of messages, maintaining the integrity of the message queue.
* support REMOTE_INSTALL_URL
* nit
* nit
* passing remote_install_host and remote_install_port to TCPReaderWriter
* fix the host and port parsing approach without yarl
* extract method for parsing the host and port
* nit
* nit
* nit
* nit
* improve _get_remote_install_host_and_port method
* fix style
* fix style
- Introduced an optional `repo` field in the `PluginConfiguration` class to store the repository URL of the plugin, enhancing the configuration capabilities.
- Added `@final` decorator to the constructors of `AgentStrategy`, `Endpoint`, `AIModel`, and `Tool` classes to prevent overriding.
- Included documentation notes in each constructor to indicate that they should not be overridden.
* fix: improve buffer handling
* tests:add tests for StdioRequestReader
- Introduced a new private method `_read_async` to handle asynchronous reading from stdin.
- Updated `_read_stream` to utilize the new `_read_async` method for improved code clarity and maintainability.
- Added a new test file `test_stdio.py` to validate the functionality of the StdioRequestReader, ensuring correct handling of input data streams.
* fix: remove useless code
* fix: handle empty lines in StdioRequestReader
- Updated the StdioRequestReader to strip and skip empty lines during input processing.
- Added a new test to validate the handling of empty lines, ensuring the reader correctly processes a stream with mixed content.
* apply ruff
---------
Co-authored-by: Yeuoly <admin@srmxy.cn>
- Introduced a class method `_load_yaml_file` to encapsulate YAML file loading logic.
- Enhanced the `validate_endpoints` method to support both direct endpoint configurations and YAML file paths.
- Added unit tests for loading endpoint configurations from YAML strings and files.
- Added exception handling for unsupported protocols in the `get_blob` method.
- Raises a ValueError with a clear message if the URL is invalid, guiding users to check the `FILES_URL` environment variable.
- Maintained existing functionality while improving robustness against HTTP request failures.
- Added CLI for generating documentation with `generate-docs` command.
- Introduced `SchemaDocumentationGenerator` to create structured documentation from schemas.
- Implemented `SchemaDoc` class for schema metadata and documentation management.
- Updated `PluginConfiguration` to include descriptions for plugin components.
- Added support for outside reference fields in the SchemaDocumentationGenerator to improve documentation clarity.
- Updated SchemaDoc to include outside_reference_fields parameter.
- Modified various entity classes to utilize outside_reference_fields for better schema representation.
- Introduced new container type checks and helper methods for handling dynamic fields in documentation generation.
* feat(http_parser): add functions to parse raw HTTP requests and convert Request objects to raw data
- Implemented `parse_raw_request` to convert raw HTTP data into a Request object, including support for headers and body.
- Added `convert_request_to_raw_data` to transform a Request object back into raw HTTP data.
- Created unit tests for both functions to ensure correct parsing and conversion behavior.
* apply ruff
* feat: support var-selector params & surpport retriever_resources messsage
* opt: rename var-select to any
* fix: line too long
* opt: Add RetrieverResource Type
- Changed minimum_dify_version from required to optional in PluginConfiguration.
- Added unit tests to verify behavior when minimum_dify_version is not provided.
* refactor: replace json.loads with TypeAdapter for JSON validation in request readers and model
- Updated JSON parsing in BackwardsInvocation, StdioRequestReader, and TCPReaderWriter to use TypeAdapter for improved validation.
- Enhanced error handling in OAICompatLargeLanguageModel by replacing json.loads with TypeAdapter, ensuring better compliance with expected JSON schema.
* fix: linter
* feat(invocation): add FetchAppInvocation and update type hints for improved clarity
- Introduced FetchAppInvocation class to handle fetching app data.
- Updated type hints in ToolInvokeMessage and related classes to use Mapping for better type safety.
- Added FetchApp enum value to InvokeType for consistency in invocation types.
* apply ruff
* feat(python): Add specific error classes for StorageInvocation
Replace generic `Exception` raises with `NotFoundError` when the key is missing,
and `StorageInvocationError` for other response errors. This improves error
handling at call sites by avoiding overly generic exceptions.
# Conflicts:
# python/dify_plugin/invocations/storage.py
* test(python): Add test cases for `StorageInvocation`
- Ensure `StorageInvocation` raises the correct exceptions when data is missing
or `_backwards_invoke` returns an invalid response.
- Verify that `StorageInvocation` correctly processes the result returned
by `_backwards_invoke`.
* chore(python): fix linter violations
* feat(python): Change the exception `StorageInvocation.exist` raised
* test(python): Add test cases for `StorageInvocation.exist`.
* chore(python): temporarily disable `UP007` rule in Ruff
Disable the `UP007` rule temporarily due to numerous existing
violations in the current codebase.
* ci(python): add lint and test for Python SDK
* ci: Add mega-linter
* chore(python): Add final newline to markdown files
* fix(python): Fix linter issues by using ruff's auto fix
* fix(python): Fix TRY400 linter violations
Ruff suggests using `logger.exception` to log when
logging an exception, instead of using `logging.error`.
The former logs the exception and the traceback
automatically.
ref: https://docs.astral.sh/ruff/rules/error-instead-of-exception/
* fix(python): fix ruff linter violations
* test(python): Add Test cases for LLM entities
Test the initialization of `LLMResult` and `LLMResultChunk`.
* test(python): Add test for `AgentModelConfig`
Ensures that the `history_prompt_messages` attribute is
not shared between instances of `AgentModelConfig`.
* chore(python): output diffs in Ruff's format check
* style(python): format code with Ruff
* chore(python): Add a bash script for fix and format python code
* chore: add `persist-credentials: false` to checkout action
Enhances security by preventing subsequent
steps from accessing GitHub credentials, reducing
potential attack surfaces.
Ref: https://github.com/actions/checkout#checkout-v4
immortal-wm