mirror of
https://github.com/langgenius/dify-plugin-daemon.git
synced 2026-07-23 10:15:22 -04:00
878edde455
* feat(datasource): Implement datasource validation and invocation steps - Added new functionality for validating datasource credentials and invoking the first and second steps of the datasource process. - Introduced new API endpoints for datasource operations in the HTTP server. - Created corresponding service and controller methods to handle datasource requests. - Defined new request and response structures for datasource operations in the entities package. * feat: add routes * feat(datasource): Add initial datasource models and validation - Introduced new `DatasourceInstallation` model to represent datasource installations with relevant fields. - Created `datasource.go` file for future datasource service implementation. - Enhanced `datasource_declaration.go` with new types and validation functions for datasource provider and parameters. - Updated `plugin_declaration.go` to include datasource support in plugin structures. - Refactored `agent_declaration.go` and `tool_declaration.go` to use a unified `ParameterOption` type for options. * refactor(datasource): Update request and response types for online document content retrieval - Renamed and updated request and response types for the `DatasourceGetOnlineDocumentPageContent` function to improve clarity and consistency. - Introduced `RequestInvokeOnlineDocumentDatasourceGetContent` and `DatasourceInvokeOnlineDocumentGetContentResponse` types. - Adjusted related function signatures and dispatchers to reflect the new types across the datasource service and controller implementations. * feat(datasource): Implement datasource installation handling - Added functionality to create and update `DatasourceInstallation` records during plugin installation and upgrade processes. - Enhanced the `InstallPlugin` function to create a new datasource installation if a datasource declaration is present. - Updated the `UpgradePlugin` function to handle the deletion of the original datasource installation and creation of a new one if the datasource declaration changes. * feat(datasource): Add datasource registration handling in plugin runtime - Introduced handling for datasource declarations in the plugin runtime. - Updated `RemotePluginRuntime` to track datasource registration status. - Enhanced message processing to include validation and assignment of datasource declarations. * feat(datasource): Implement datasource listing and retrieval endpoints - Added `ListDatasources` and `GetDatasource` functions to the service layer for handling datasource queries. - Created corresponding controller methods to process HTTP requests for listing and retrieving datasources. - Implemented request validation for tenant ID, page, page size, plugin ID, and provider parameters. * add datasource routes * feat(datasource): Add icon remapping for datasource declarations - Implemented functionality to remap icons for both the main datasource and its individual datasources within the plugin declaration. - Enhanced error handling to provide clearer feedback when remapping fails. * fix(datasource): update OAuthSchema validation to remove unnecessary 'dive' tag - Modified the validation tag for OAuthSchema in DatasourceProviderDeclaration to simplify the validation process by removing the 'dive' requirement. * feat(tests): add datasource declaration parsing and output - Introduced a new main.go file for testing datasource declaration parsing. - Implemented JSON unmarshalling for RemotePluginRegisterPayload and DatasourceProviderDeclaration. - Added a sample datasource declaration for testing purposes. * fix(plugin): improve error handling in UninstallPlugin and add datasource deletion - Enhanced error handling to return a specific message when a plugin is not installed. - Added functionality to delete the associated datasource installation during the uninstallation process. * feat(datasource): add support for decoding datasource provider declaration - Added custom JSON marshalling and unmarshalling methods to handle CredentialsSchema and Datasources more effectively. - Improved error handling during YAML unmarshalling to support both object and array formats for CredentialsSchema. - Ensured proper initialization of DatasourceFiles and Tags to prevent nil references. * fix: provider type * refactor(datasource): simplify JSON unmarshalling for CredentialsSchema - Removed complex handling of CredentialsSchema in the UnmarshalJSON method, focusing on the Datasources field. - Streamlined the code to improve readability and maintainability by eliminating unnecessary checks and logic related to CredentialsSchema. * feat: streaming datasource * feat: datasource * feat: datasource * feat:datasource * feat:datasource * feat:datasource * feat: add redirect_uri field to OAuth request structs * feat: add online driver file request and response structures * feat: add online driver file request and response structures * feat: add online_driver datasource type to validation * feat: rename online driver to online drive and update related classes and methods :) * refactor: rename OnlineDocumentPageChunk to DatasourceGetPagesResponse and update related references * feat: update request types for online drive browsing and downloading * feat: add metadata field to OAuthGetCredentialsResult * feat: built-in json schema definations * feat(plugin_entities): add built-in schema definitions and processing for datasource YAML * test(plugin_entities): add unit tests for schema definitions and YAML processing * refactor(plugin_entities): centralize built-in schema definitions and processing * refactor(plugin_entities): remove unused properties from built-in schema definitions * refactor(plugin_entities): built-in schema & new datasource structure * refactor(plugin_entities): enhance schema processing with checks and error handling * refactor(plugin_entities): update validation for OnlineDriveBrowseFilesRequest prefix field to be optional * refactor(json_schema): remove json schema definitions and validation * fix(plugin_entities): remove output_schema validation tests * feat: add rag tag --------- Co-authored-by: Harry <xh001x@hotmail.com> Co-authored-by: Dongyu Li <544104925@qq.com> Co-authored-by: Novice <novice12185727@gmail.com>
89 lines
3.4 KiB
Go
89 lines
3.4 KiB
Go
package datasource_entities
|
|
|
|
import (
|
|
"github.com/go-playground/validator/v10"
|
|
"github.com/langgenius/dify-plugin-daemon/pkg/validators"
|
|
)
|
|
|
|
type DataSourceValidateCredentialsResponse struct {
|
|
Result bool `json:"result"`
|
|
}
|
|
|
|
type DataSourceResponseChunkType string
|
|
|
|
const (
|
|
DataSourceResponseChunkTypeText DataSourceResponseChunkType = "text"
|
|
DataSourceResponseChunkTypeFile DataSourceResponseChunkType = "file"
|
|
DataSourceResponseChunkTypeBlob DataSourceResponseChunkType = "blob"
|
|
DataSourceResponseChunkTypeBlobChunk DataSourceResponseChunkType = "blob_chunk"
|
|
DataSourceResponseChunkTypeJson DataSourceResponseChunkType = "json"
|
|
DataSourceResponseChunkTypeLink DataSourceResponseChunkType = "link"
|
|
DataSourceResponseChunkTypeImage DataSourceResponseChunkType = "image"
|
|
DataSourceResponseChunkTypeImageLink DataSourceResponseChunkType = "image_link"
|
|
DataSourceResponseChunkTypeVariable DataSourceResponseChunkType = "variable"
|
|
DataSourceResponseChunkTypeLog DataSourceResponseChunkType = "log"
|
|
DataSourceResponseChunkTypeRetrieverResources DataSourceResponseChunkType = "retriever_resources"
|
|
)
|
|
|
|
func IsValidDataSourceResponseChunkType(fl validator.FieldLevel) bool {
|
|
t := fl.Field().String()
|
|
switch DataSourceResponseChunkType(t) {
|
|
case DataSourceResponseChunkTypeText,
|
|
DataSourceResponseChunkTypeFile,
|
|
DataSourceResponseChunkTypeBlob,
|
|
DataSourceResponseChunkTypeBlobChunk,
|
|
DataSourceResponseChunkTypeJson,
|
|
DataSourceResponseChunkTypeLink,
|
|
DataSourceResponseChunkTypeImage,
|
|
DataSourceResponseChunkTypeImageLink,
|
|
DataSourceResponseChunkTypeVariable,
|
|
DataSourceResponseChunkTypeLog,
|
|
DataSourceResponseChunkTypeRetrieverResources:
|
|
return true
|
|
default:
|
|
return false
|
|
}
|
|
}
|
|
|
|
func init() {
|
|
err := validators.GlobalEntitiesValidator.RegisterValidation(
|
|
"is_valid_data_source_response_chunk_type",
|
|
IsValidDataSourceResponseChunkType,
|
|
)
|
|
if err != nil {
|
|
panic(err)
|
|
}
|
|
}
|
|
|
|
type DataSourceResponseChunk struct {
|
|
Type DataSourceResponseChunkType `json:"type" validate:"required,is_valid_data_source_response_chunk_type"`
|
|
Message map[string]any `json:"message"`
|
|
Meta map[string]any `json:"meta"`
|
|
}
|
|
|
|
type WebsiteCrawlChunk struct {
|
|
Result map[string]any `json:"result"`
|
|
}
|
|
|
|
type DatasourceGetPagesResponse struct {
|
|
Result []map[string]any `json:"result"`
|
|
}
|
|
|
|
type OnlineDriveFile struct {
|
|
ID string `json:"id" validate:"required"` // The file ID
|
|
Name string `json:"name" validate:"required"` // The file name
|
|
Size int `json:"size" validate:"required"` // The file size
|
|
Type string `json:"type" validate:"required"` // The file type: folder or file
|
|
}
|
|
|
|
type OnlineDriveFileBucket struct {
|
|
Bucket *string `json:"bucket" validate:"omitempty"` // The file bucket (optional)
|
|
Files []OnlineDriveFile `json:"files" validate:"required"` // The file list
|
|
IsTruncated bool `json:"is_truncated" validate:"required"` // Whether the result is truncated
|
|
NextPageParameters map[string]interface{} `json:"next_page_parameters" validate:"omitempty"` // Parameters for fetching the next page
|
|
}
|
|
|
|
type GetOnlineDriveBrowseFilesResponse struct {
|
|
Result []OnlineDriveFileBucket `json:"result" validate:"required"` // The list of file buckets
|
|
}
|