* refactor(plugin_manager): enhance HTTP client timeout handling in serverless runtime with DialContext
- Updated the HTTP client in the ServerlessPluginRuntime to use a context-aware DialContext for better timeout management.
- Removed the write timeout option from the HTTP request builder, streamlining the request handling process.
- Improved connection handling by setting write deadlines based on the PluginMaxExecutionTimeout.
This change enhances the reliability of network operations within the serverless runtime environment.
* refactor(plugin_manager): remove write deadline setting in serverless runtime connection initialization
- Eliminated the write deadline setting from the connection initialization in the ServerlessPluginRuntime.
- This change simplifies the connection handling process and aligns with the recent enhancements to timeout management.
This update contributes to a more streamlined and efficient network operation within the serverless environment.
* refactor(plugin_manager): adjust HTTP client timeout settings in serverless runtime
- Modified the HTTP client configuration in the ServerlessPluginRuntime to set the TLS handshake timeout based on PluginMaxExecutionTimeout.
- Retained the IdleConnTimeout setting to ensure consistent connection management.
This update improves the timeout handling for secure connections, enhancing overall network reliability in the serverless environment.
- Refactored the RemapAssets function to streamline the remapping of icon fields for both models and tools, including support for dark mode icons.
- Introduced new fields IconSmallDark and IconLargeDark in the ModelProviderDeclaration and added IconDark in ToolProviderIdentity to accommodate dark mode assets.
- Improved error handling during the remapping process for better clarity and maintainability.
* feat(plugin_manager): optimize local plugin startup with concurrent control
- Add semaphore-based concurrency control for plugin launches
- Implement parallel plugin startup using goroutines
- Optimize error handling to prevent goroutine blocking
- Add concurrency metrics logging
Note: handleNewLocalPlugins now accepts config parameter with default concurrency limit
* feat(plugin_manager): make local plugin launching concurrency configurable
* fix(plugin_manager): optimize comment and error handling
- Updated comments to clarify the concurrent plugin launching configuration.
- Added a nil check for the error channel during plugin startup to improve code robustness.
* refactor(plugin_manager): refactor plugin startup logic
- Remove the semaphore mechanism and switch to using routine.Submit for concurrency management
* fix(plugin_manager): Optimize plugin startup logs and concurrency control
- Added log output for maximum concurrency when starting local plugins
- Implemented a channel-based concurrency control mechanism to ensure limits are not exceeded
- Fixed closure variable capture issue to prevent incorrect plugin information
- Improved error handling to avoid deadlocks during startup
* fix(plugin_manager): simplify error channel handling and semaphore release logic
---------
Co-authored-by: jim02.he <jim02.he@vipshop.com>
* feat: add support for dark icon
- Introduced IconDark field in PluginDeclaration and related structures to support dark mode icons.
- Updated the installation process to handle dark icons.
- Enhanced asset validation to check for the presence of dark icons.
This change improves the visual consistency of plugins in dark mode environments.
* enhance(cli/icon): add plugin icon support with multiple categories
- Added support for light and dark icons for various plugin categories including agent, datasource, extension, model, tool, and trigger.
- Replaced the previous single icon implementation with a structured map for better organization and retrieval of icons based on category and theme.
- Removed the old Python icon file to streamline asset management.
This update improves the visual representation of plugins across different themes, enhancing user experience.
* change icons
* fix
* fix
* comments
Directly fetch and invoke the listener for a given session_id instead of iterating over the entire listener map.
Signed-off-by: guanz42 <guanz42@163.com>
* refactor: update PluginManager to use configuration for various configurations
- Replaced hardcoded values in PluginManager methods with values from the configuration.
- Updated serverless plugin launch timeout and working paths to utilize the new configuration structure.
- Enhanced local plugin runtime initialization to pull settings from the configuration, improving maintainability and flexibility.
* refactor: clean up PluginManager by removing unused fields and updating platform check
- Removed commented-out fields from PluginManager to enhance code clarity.
- Updated platform check to utilize the configuration structure instead of a direct field reference, improving maintainability.
- Added `pluginMaxExecutionTimeout` to `PluginManager` for configurable execution limits.
- Updated `ServerlessPluginRuntime` to utilize the new timeout setting in HTTP requests.
- Refactored AWSPluginRuntime references to ServerlessPluginRuntime for consistency across the codebase.
* fix: errChan failed to write response because of panic nil (#296)
* fix: join err and er into a single error using errors.Join, thanks @Yeuoly (#296)
---------
Co-authored-by: NeatGuyCoding <cto@sb>
* feat(redis): Add support for Redis Sentinel mode
Added support for Redis Sentinel mode to the Redis client, enabling automatic discovery and connection to the primary node through Sentinel. Updated relevant configuration files and initialization logic to support Sentinel mode configuration and connection.
* add lost RedisUser.
* feat: introduce 'run' command for local plugin execution
- Added a new command `run` to launch plugins locally, allowing communication through stdin/stdout.
- Removed the previous `test` command and its associated functionality to streamline the plugin testing process.
* feat: enhance 'run' command with TCP support
- Introduced a new `RunPluginPayload` structure to encapsulate plugin execution parameters.
- Added TCP communication mode to the `run` command, allowing multiple client connections.
- Updated command flags to configure run mode and logging options.
- Implemented client handling and server creation for both stdin/stdout and TCP modes.
* docs: improve comments in RunPlugin function for clarity
- Enhanced comments to provide clearer explanations of the plugin decoding process and the creation of client streams for both stdin/stdout and TCP modes.
- Updated comments to reflect the functionality and behavior of the plugin execution flow.
* refactor: update command structure and enhance plugin invocation handling
- Changed the command structure to add `runPluginCommand` under `pluginCommand` for better organization.
- Introduced `InvokePluginPayload` type to encapsulate plugin invocation details.
- Enhanced `RunPlugin` function to handle responses and errors more effectively, including logging to stdout.
- Updated client handling to support session management and improved error handling during plugin invocation.
- Renamed TCP server creation function for consistency.
* feat: enhance plugin response handling and logging capabilities
- Added a new flag `--response-format` to specify the output format (text or json) for plugin responses.
- Introduced a `logger` to manage logging output to stdout with timestamps and file information.
- Updated `logResponse` and `systemLog` functions to handle different response formats.
- Enhanced `handleClient` and `RunPlugin` functions to utilize the new response format feature.
- Implemented signal handling to clean up temporary directories on shutdown.
* feat: enhance plugin response structure and logging
- Added `InvokeID` to `InvokePluginPayload` and `GenericResponse` for better tracking of plugin invocations.
- Updated `logResponse` to include `InvokeID` in error responses for improved debugging.
- Enhanced client handling in `handleClient` to log plugin readiness and received requests.
- Refactored client stream creation for better readability and consistency.
* feat: add plugin invoke end response type and logging
- Introduced `GENERIC_RESPONSE_TYPE_PLUGIN_INVOKE_END` to enhance response tracking for plugin invocations.
- Updated `handleClient` to log the end of plugin invocation, improving visibility into the plugin lifecycle.
* chore: remove fullfeature tags
- Introduced a new test file to implement a fake OpenAI server that streams responses for benchmarking purposes.
- Updated the existing benchmark test to utilize the new fake server, enhancing the testing environment for local plugin runtime invocations.
- Refactored the package name in the benchmark test file for clarity and consistency.
* feat: add plugin reinstallation functionality and admin API support
- Implemented ReinstallToAWSFromPkg method to allow reinstallation of plugins on AWS Lambda, updating function URL and name.
- Added clearServerlessRuntimeCache method to manage serverless runtime cache.
- Enhanced LaunchPlugin to support an ignoreIdempotent flag for forced reinstallation.
- Introduced admin API endpoints for plugin reinstallation, secured with an API key validation middleware.
- Updated configuration to include AdminApiEnabled and AdminApiKey settings.
* refactor: update plugin reinstallation endpoint and improve unauthorized response
- Changed the plugin reinstallation endpoint from "/plugins/reinstall" to "/plugin/serverless/reinstall" for better clarity.
- Modified the unauthorized response in the AdminAPIKey middleware to return a more descriptive JSON message.
- Implemented validation in NewInstalledBucket to log warnings for non-alphanumeric starting characters and empty paths for installedPath.
- Updated parameter naming for consistency across methods in InstalledBucket, enhancing code clarity.
- Adjusted the buffer size settings in the stdioHolder to utilize configurable values for stdout buffer size and maximum buffer size, enhancing flexibility in plugin output handling.
* fix: improve line processing in StdioRequestReader
- Replaced synchronous stdin reading with asynchronous data retrieval.
- Enhanced line processing to skip empty lines and handle incomplete data more effectively, improving overall robustness of the request reading mechanism.
* feat: add asynchronous reading to StdioRequestReader
- Implemented the _read_async method to read data from stdin in 64KB chunks, optimizing data retrieval.
- This enhancement improves the efficiency of the request reading process in the StdioRequestReader class.
* feat: add read & write timeout config
* refactor: update Dify invocation configuration to use structured payload
- Changed the Dify invocation daemon to accept a structured payload for initialization, improving clarity and maintainability.
- Updated related configuration variables in the .env.example file to reflect the new naming convention for backwards invocation timeouts.
- Adjusted tests and plugin manager to accommodate the new payload structure.
---------
Co-authored-by: Yeuoly <admin@srmxy.cn>
- Introduced new configuration options for plugin stdio buffer sizes in the .env.example file.
- Updated the PluginManager and LocalPluginRuntime to utilize these new buffer size settings.
- Enhanced the stdioHolder to accept buffer size configurations, improving plugin output handling.
- Modified related tests to accommodate the new stdioHolder configuration structure.
* feat: enhance integration testing for specific plugins
- Added integration tests for the official agent, including test data for agent strategy invocation.
- Introduced JSON schema generation and validation utilities to ensure proper request formatting.
- Enhanced mock invocation handling in the plugin manager to support tool parameters.
- Added new test utilities for simulating OpenAI server responses and managing plugin runtime.
* fix: update RunOnce function to return response stream and enhance integration test
- Modified the RunOnce function to return a response stream instead of an error, allowing for better handling of streamed responses.
- Updated the integration test for the official agent to read from the response stream, ensuring proper validation of the agent strategy invocation.
- Introduced two new patch files: `0.1.1.llm.py.patch` and `0.1.1.request_reader.py.patch` to enhance the plugin's functionality.
- Updated the `environment_python.go` file to reference the new patches and apply them conditionally based on the plugin SDK version.
- Improved the handling of LLM usage and request reading in the plugin environment.
* refactor(plugin_daemon): simplify InvokeTool function and enhance stream handling
- Removed unnecessary complexity from the InvokeTool function by eliminating the blob handling logic and directly returning the response.
- Updated the stream implementation to support blocking writes, allowing for better management of data flow when the buffer is full.
- Added tests for the new blocking write functionality to ensure reliability and performance under high load conditions.
* fix: buffer size
- Updated the getPluginSdkVersion function to parse version ranges with multiple constraints, returning the highest version found.
- Improved error handling to provide clearer feedback when no valid version is found.
- Added unit tests to validate the new functionality and ensure correctness across various requirement formats.
* refactor(plugin_manager): replace stdio handling with stdioHolder for improved management and cleanup
- Introduced stdioHolder to encapsulate stdio operations, enhancing modularity.
- Updated LocalPluginRuntime methods to utilize stdioHolder for event listening and error handling.
- Removed legacy stdio handling functions to streamline the codebase.
* feat(plugin_manager): add tests for stdioHolder functionality and enhance error handling
- Introduced comprehensive unit tests for stdioHolder, covering various scenarios including event listening, error writing, and cleanup.
- Added constants for maximum error message length and heartbeat interval to improve code maintainability.
- Updated stdioHolder methods to utilize these constants for better readability and consistency.
Updated the error handling in the InitPythonEnvironment function to log pre-compilation errors instead of returning them. This change addresses issues with plugins referencing broken SDKs while still functioning correctly. Additionally, corrected the log message for pre-loading plugins to reflect the accurate state.
Introduce `pythonCompileAllExtraArgs` to allow users to pass extra arguments to the Python `compileall` command. This enhancement enables more flexible compilation options for plugins by extending the configuration in the plugin manager and local runtime components.
- Updated `launcher.go` to include `PythonCompileAllExtraArgs` in the runtime configuration.
- Modified `environment_python.go` to handle the additional compile arguments.
- Added new configuration fields in `type.go` and `config.go`.
- Adjusted initialization in `manager.go` to accept the new configuration parameter.
Yeuoly