* feat(datasource): Implement datasource validation and invocation steps
- Added new functionality for validating datasource credentials and invoking the first and second steps of the datasource process.
- Introduced new API endpoints for datasource operations in the HTTP server.
- Created corresponding service and controller methods to handle datasource requests.
- Defined new request and response structures for datasource operations in the entities package.
* feat: add routes
* feat(datasource): Add initial datasource models and validation
- Introduced new `DatasourceInstallation` model to represent datasource installations with relevant fields.
- Created `datasource.go` file for future datasource service implementation.
- Enhanced `datasource_declaration.go` with new types and validation functions for datasource provider and parameters.
- Updated `plugin_declaration.go` to include datasource support in plugin structures.
- Refactored `agent_declaration.go` and `tool_declaration.go` to use a unified `ParameterOption` type for options.
* refactor(datasource): Update request and response types for online document content retrieval
- Renamed and updated request and response types for the `DatasourceGetOnlineDocumentPageContent` function to improve clarity and consistency.
- Introduced `RequestInvokeOnlineDocumentDatasourceGetContent` and `DatasourceInvokeOnlineDocumentGetContentResponse` types.
- Adjusted related function signatures and dispatchers to reflect the new types across the datasource service and controller implementations.
* feat(datasource): Implement datasource installation handling
- Added functionality to create and update `DatasourceInstallation` records during plugin installation and upgrade processes.
- Enhanced the `InstallPlugin` function to create a new datasource installation if a datasource declaration is present.
- Updated the `UpgradePlugin` function to handle the deletion of the original datasource installation and creation of a new one if the datasource declaration changes.
* feat(datasource): Add datasource registration handling in plugin runtime
- Introduced handling for datasource declarations in the plugin runtime.
- Updated `RemotePluginRuntime` to track datasource registration status.
- Enhanced message processing to include validation and assignment of datasource declarations.
* feat(datasource): Implement datasource listing and retrieval endpoints
- Added `ListDatasources` and `GetDatasource` functions to the service layer for handling datasource queries.
- Created corresponding controller methods to process HTTP requests for listing and retrieving datasources.
- Implemented request validation for tenant ID, page, page size, plugin ID, and provider parameters.
* add datasource routes
* feat(datasource): Add icon remapping for datasource declarations
- Implemented functionality to remap icons for both the main datasource and its individual datasources within the plugin declaration.
- Enhanced error handling to provide clearer feedback when remapping fails.
* fix(datasource): update OAuthSchema validation to remove unnecessary 'dive' tag
- Modified the validation tag for OAuthSchema in DatasourceProviderDeclaration to simplify the validation process by removing the 'dive' requirement.
* feat(tests): add datasource declaration parsing and output
- Introduced a new main.go file for testing datasource declaration parsing.
- Implemented JSON unmarshalling for RemotePluginRegisterPayload and DatasourceProviderDeclaration.
- Added a sample datasource declaration for testing purposes.
* fix(plugin): improve error handling in UninstallPlugin and add datasource deletion
- Enhanced error handling to return a specific message when a plugin is not installed.
- Added functionality to delete the associated datasource installation during the uninstallation process.
* feat(datasource): add support for decoding datasource provider declaration
- Added custom JSON marshalling and unmarshalling methods to handle CredentialsSchema and Datasources more effectively.
- Improved error handling during YAML unmarshalling to support both object and array formats for CredentialsSchema.
- Ensured proper initialization of DatasourceFiles and Tags to prevent nil references.
* fix: provider type
* refactor(datasource): simplify JSON unmarshalling for CredentialsSchema
- Removed complex handling of CredentialsSchema in the UnmarshalJSON method, focusing on the Datasources field.
- Streamlined the code to improve readability and maintainability by eliminating unnecessary checks and logic related to CredentialsSchema.
* feat: streaming datasource
* feat: datasource
* feat: datasource
* feat:datasource
* feat:datasource
* feat:datasource
* feat: add redirect_uri field to OAuth request structs
* feat: add online driver file request and response structures
* feat: add online driver file request and response structures
* feat: add online_driver datasource type to validation
* feat: rename online driver to online drive and update related classes and methods :)
* refactor: rename OnlineDocumentPageChunk to DatasourceGetPagesResponse and update related references
* feat: update request types for online drive browsing and downloading
* feat: add metadata field to OAuthGetCredentialsResult
* feat: built-in json schema definations
* feat(plugin_entities): add built-in schema definitions and processing for datasource YAML
* test(plugin_entities): add unit tests for schema definitions and YAML processing
* refactor(plugin_entities): centralize built-in schema definitions and processing
* refactor(plugin_entities): remove unused properties from built-in schema definitions
* refactor(plugin_entities): built-in schema & new datasource structure
* refactor(plugin_entities): enhance schema processing with checks and error handling
* refactor(plugin_entities): update validation for OnlineDriveBrowseFilesRequest prefix field to be optional
* refactor(json_schema): remove json schema definitions and validation
* fix(plugin_entities): remove output_schema validation tests
* feat: add rag tag
---------
Co-authored-by: Harry <xh001x@hotmail.com>
Co-authored-by: Dongyu Li <544104925@qq.com>
Co-authored-by: Novice <novice12185727@gmail.com>
- Introduced TriggerInstallation model for managing trigger installations in the database.
- Updated autoMigrate function to include trigger installations in the migration process.
- Added new HTTP routes for listing and retrieving triggers in the HTTP server.
- Implemented ListTriggers and GetTrigger controller functions for handling trigger requests.
- Enhanced plugin management functions to create, update, and delete trigger installations during plugin lifecycle events.
These changes integrate trigger capabilities into the system, improving the overall plugin functionality and management.
- Updated .env.example to include DB_EXTRAS and DB_CHARSET variables.
- Refactored InitPluginDB functions for PostgreSQL and MySQL to accept a configuration struct, allowing for more flexible database connection settings.
- Adjusted connection pool settings to utilize new configuration options for charset and extras.
This change improves the configurability of database connections and prepares the codebase for future enhancements.
- Add support for connecting to a default database when target database connection fails
- Introduce new configuration parameter `DBDefaultDatabase` with default value "postgres"
- Modify `initDifyPluginDB` to attempt connection to default database and create target database if needed
- Update test case to include default database parameter
Stream