Commit Graph

975 Commits

Author SHA1 Message Date
Harry 2856d56513 Merge remote-tracking branch 'origin/main' into feat/datasource
# Conflicts:
#	internal/types/models/curd/atomic.go
2025-09-16 15:56:03 +08:00
Yunlu Wen 9e3081bd17 feat: support orphan plugin (#440)
* support orphan plugin in serverless mode

* add validation to UpgradePlugin

* log reinstalls
2025-09-12 20:34:38 +08:00
Yunlu Wen e6bb4967a8 fix lock release (#447) 2025-09-12 19:30:01 +08:00
kenwoodjw bb83075acd feat: add max-size flag (#445)
Signed-off-by: kenwoodjw <blackxin55+@gmail.com>
2025-09-12 19:26:29 +08:00
Byron.wang b4da677c5e bump dify-cloud-kit version to v0.1.0 (#442) 2025-09-03 18:08:28 +08:00
Yunlu Wen a4c32d32ea feat: propagate plugin unique identifier to serverless control plane (#428)
* feat: propagate plugin unique identifier to serverless

* resolve conflict
2025-09-03 11:01:48 +08:00
Novice 71cc259338 feat: add rag tag 2025-09-02 22:12:20 +08:00
Yeuoly 5c57624f5d docs: add comprehensive development documentation (#433) 2025-09-01 13:30:42 +08:00
toli-ch 880dc70345 Fix OS and architecture detection for Windows (#432) 2025-09-01 13:21:29 +08:00
Yeuoly 604da5d495 refactor: rename AWS references to Serverless in plugin daemon (#437)
- Updated AWS-related types and functions to reflect Serverless terminology, including renaming `AWSTransactionHandler` to `ServerlessTransactionHandler` and `AWSTransactionWriter` to `ServerlessTransactionWriter`.
- Adjusted error messages and comments to align with the new naming conventions.
- Modified installation functions to use Serverless terminology for clarity and consistency across the codebase.
2025-08-28 18:24:22 +08:00
Blackoutta b86f12fbaa Add Redis caching to FetchPluginInstallation middleware for performance optimization (#400)
* add caching to fetch plugin installation middleware

* invalidate plugin inst cache after successfully upgrading or uninstalling operations

* optimize: refactored cache invalidation ops from curd layer to service layer

---------

Co-authored-by: yangyi.hu593671 <yangyi.hu593671@seres.cn>
2025-08-27 14:16:52 +08:00
kuangzhiqiang 6c03c1b3c8 fix Frequent plugin pulls (#420)
Co-authored-by: kuangzhiqiang <kuangzhiqiang@xiaomi.com>
2025-08-27 14:16:31 +08:00
Yeuoly ab7540eed9 Add Claude Code GitHub Workflow (#434)
* "Claude PR Assistant workflow"

* "Claude Code Review workflow"
2025-08-27 14:03:59 +08:00
Yeuoly b286be778e chore: remove JSON schema validation from tool and agent services (#430)
* chore: remove JSON schema validation from tool and agent services

- Remove bindAgentStrategyValidator and bindToolValidator functions
- Remove gojsonschema dependency
- Simplify InvokeAgentStrategy by removing validation logic
- Consolidate tool invocation into generated code
- Remove redundant tool_service.go and related test files
- Move InvokeTool to generated files for consistency

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: remove duplicated routes

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-08-26 20:36:25 +08:00
Harry 867b61c213 fix(plugin_entities): remove output_schema validation tests 2025-08-26 12:29:41 +08:00
Harry e758592e5a refactor(json_schema): remove json schema definitions and validation 2025-08-25 19:20:03 +08:00
Tianyi Jing 526addc778 fix: add missing types of tool_parameter (#421)
Signed-off-by: jingfelix <jingfelix@outlook.com>
2025-08-23 18:52:32 +08:00
Harry cd091378ce refactor(plugin_entities): update validation for OnlineDriveBrowseFilesRequest prefix field to be optional 2025-08-11 15:24:52 +08:00
Betula-L 21015d4474 fix credential type (#419) 2025-08-08 21:56:34 +08:00
Harry 9ce516a3cd refactor(plugin_entities): enhance schema processing with checks and error handling 2025-08-08 17:13:04 +08:00
Harry 7c40929ddd refactor(plugin_entities): built-in schema & new datasource structure 2025-08-08 15:56:08 +08:00
Harry 7699ab00a9 refactor(plugin_entities): remove unused properties from built-in schema definitions 2025-08-08 11:02:11 +08:00
Harry 50b81c0232 refactor(plugin_entities): centralize built-in schema definitions and processing 2025-08-07 17:19:08 +08:00
Harry 0e9ec24f16 Merge remote-tracking branch 'origin/main' into feat/datasource
# Conflicts:
#	internal/core/plugin_daemon/access_types/access.go
#	internal/server/http_server.gen.go
2025-08-07 12:01:06 +08:00
Harry e6f0bd7973 test(plugin_entities): add unit tests for schema definitions and YAML processing 2025-08-07 00:38:53 +08:00
Harry 99fcbd138d feat(plugin_entities): add built-in schema definitions and processing for datasource YAML 2025-08-07 00:33:00 +08:00
Harry 71edd1a7aa feat: built-in json schema definations 2025-08-01 15:13:32 +08:00
Yeuoly 222ab80a6b refactor(session_manager): update GetSession function to return error (#413)
- Modified GetSession to return an error alongside the session, improving error handling when a session is not found.
- Updated invocation of GetSession in AWS event handler to handle the new error return value.
- Removed logging statements in favor of returning errors directly for better error propagation.
2025-07-24 13:40:38 +08:00
Yeuoly 178e7649ba fix(env): update REMOTE_INSTALL_URL format in .env.example (#412)
- Combined REMOTE_INSTALL_URL and REMOTE_INSTALL_PORT into a single line for clarity.
- This change improves the configuration format for easier understanding and usage.
2025-07-23 18:21:42 +08:00
Yeuoly 80389be55b chore(deps): update go-git and golang.org/x/tools dependencies (#410)
- Updated github.com/go-git/go-git from v4.7.0 to v5.16.2.
- Bumped golang.org/x/tools from v0.22.0 to v0.35.0.
- Added indirect dependencies for github.com/go-git/gcfg and github.com/go-git/go-billy.
- Updated various indirect dependencies including golang.org/x/net, golang.org/x/sys, and golang.org/x/text.
2025-07-23 16:22:48 +08:00
dependabot[bot] 9234aed479 chore(deps): bump github.com/go-jose/go-jose/v4 from 4.0.4 to 4.0.5 (#409)
Bumps [github.com/go-jose/go-jose/v4](https://github.com/go-jose/go-jose) from 4.0.4 to 4.0.5.
- [Release notes](https://github.com/go-jose/go-jose/releases)
- [Commits](https://github.com/go-jose/go-jose/compare/v4.0.4...v4.0.5)

---
updated-dependencies:
- dependency-name: github.com/go-jose/go-jose/v4
  dependency-version: 4.0.5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
0.2.0
2025-07-23 13:51:27 +08:00
Yeuoly bace3bfb58 feat(oauth): implement refresh credentials functionality (#408)
* feat(oauth): implement refresh credentials functionality

- Added RefreshCredentials endpoint to handle OAuth credential refresh requests.
- Introduced RequestOAuthRefreshCredentials structure for request validation.
- Updated access types and actions to include refresh credentials.
- Enhanced server routing and controller logic to support the new functionality.
- Updated OAuth entities to include expiration handling for refreshed credentials.

* feat(oauth): add metadata field to OAuthGetCredentialsResult

---------

Co-authored-by: Harry <xh001x@hotmail.com>
2025-07-23 13:11:36 +08:00
Yeuoly 7f463e32f6 feat(plugin_decoder): add support for internationalized readme files (#393)
* feat(plugin_decoder): add support for internationalized readme files

- Introduced the AvailableI18nReadme method in the PluginDecoder interface to retrieve available readme files in multiple languages.
- Implemented the method in FSPluginDecoder and ZipPluginDecoder to read localized readme files from the filesystem and zip archives.
- Enhanced UnixPluginDecoder to handle readme files in a structured manner, including support for reading from a dedicated "readme" directory.
- Added unit tests to verify the functionality of the AvailableI18nReadme method and ensure correct retrieval of localized readme content.

* feat(plugin): add support for multilingual README generation

- Introduced functionality to create README files in multiple languages (Simplified Chinese, Japanese, Portuguese) based on user selection.
- Enhanced the profile management to include options for enabling internationalized README and selecting languages.
- Added new language choice structure to manage language options and their selection state.
- Implemented rendering and writing of language-specific README files during plugin creation.
- Included new README template files for each supported language.

* feat(plugin): add README command and list functionality

- Introduced a new `readme` command to the plugin CLI for managing README files.
- Added `list` subcommand to display available README languages for a specified plugin path.
- Implemented functionality to read and list supported README languages in a tabular format.
- Enhanced error handling for plugin file reading and decoding processes.
2025-07-21 16:02:26 +08:00
Maries 3b0a8679f4 feat/tool oauth cli template (#407)
* feat(cli): update OAuth handling and requirements for dify_plugin

* feat(oauth): update OAuth support and adjust dify_plugin version constraints
2025-07-21 15:37:13 +08:00
Harry 9eefb38552 feat: add metadata field to OAuthGetCredentialsResult 2025-07-18 18:33:43 +08:00
Harry eb2d0e917c Merge remote-tracking branch 'origin' into feat/datasource 2025-07-18 12:57:01 +08:00
Maries a0414b30e6 0.2.0 (#402)
* feat(oauth): add RedirectURI field to OAuth request structures

* feat(oauth): update OAuthSchema validation

* feat: add Context field to request and session structures

* feat: add CredentialType field to Credentials and InvokeToolRequest structures

* fix: handle unhandled default case in basic_type.go

* feat: add support for build branches in build-push.yml
2025-07-17 16:28:43 +08:00
Yeuoly 33c023b375 refactor(plugin_manager): enhance HTTP client timeout handling in ser… (#385)
* refactor(plugin_manager): enhance HTTP client timeout handling in serverless runtime with DialContext

- Updated the HTTP client in the ServerlessPluginRuntime to use a context-aware DialContext for better timeout management.
- Removed the write timeout option from the HTTP request builder, streamlining the request handling process.
- Improved connection handling by setting write deadlines based on the PluginMaxExecutionTimeout.

This change enhances the reliability of network operations within the serverless runtime environment.

* refactor(plugin_manager): remove write deadline setting in serverless runtime connection initialization

- Eliminated the write deadline setting from the connection initialization in the ServerlessPluginRuntime.
- This change simplifies the connection handling process and aligns with the recent enhancements to timeout management.

This update contributes to a more streamlined and efficient network operation within the serverless environment.

* refactor(plugin_manager): adjust HTTP client timeout settings in serverless runtime

- Modified the HTTP client configuration in the ServerlessPluginRuntime to set the TLS handshake timeout based on PluginMaxExecutionTimeout.
- Retained the IdleConnTimeout setting to ensure consistent connection management.

This update improves the timeout handling for secure connections, enhancing overall network reliability in the serverless environment.
2025-07-09 17:43:24 +08:00
Yeuoly 7bc3b7565e feat(plugin_manager): enhance asset remapping for icons (#392)
- Refactored the RemapAssets function to streamline the remapping of icon fields for both models and tools, including support for dark mode icons.
- Introduced new fields IconSmallDark and IconLargeDark in the ModelProviderDeclaration and added IconDark in ToolProviderIdentity to accommodate dark mode assets.
- Improved error handling during the remapping process for better clarity and maintainability.
2025-07-09 17:02:57 +08:00
homejim 6ae762ba11 feat(plugin_manager): optimize local plugin startup with concurrency (#375)
* feat(plugin_manager): optimize local plugin startup with concurrent control

- Add semaphore-based concurrency control for plugin launches
- Implement parallel plugin startup using goroutines
- Optimize error handling to prevent goroutine blocking
- Add concurrency metrics logging

Note: handleNewLocalPlugins now accepts config parameter with default concurrency limit

* feat(plugin_manager): make local plugin launching concurrency configurable

* fix(plugin_manager): optimize comment and error handling

- Updated comments to clarify the concurrent plugin launching configuration.
- Added a nil check for the error channel during plugin startup to improve code robustness.

* refactor(plugin_manager): refactor plugin startup logic

- Remove the semaphore mechanism and switch to using routine.Submit for concurrency management

* fix(plugin_manager): Optimize plugin startup logs and concurrency control

- Added log output for maximum concurrency when starting local plugins
- Implemented a channel-based concurrency control mechanism to ensure limits are not exceeded
- Fixed closure variable capture issue to prevent incorrect plugin information
- Improved error handling to avoid deadlocks during startup

* fix(plugin_manager): simplify error channel handling and semaphore release logic

---------

Co-authored-by: jim02.he <jim02.he@vipshop.com>
2025-07-08 19:09:31 +08:00
Yeuoly b97cce7167 enhance(cli/icon): add multiple categories default plugin icons (#388)
* feat: add support for dark icon

- Introduced IconDark field in PluginDeclaration and related structures to support dark mode icons.
- Updated the installation process to handle dark icons.
- Enhanced asset validation to check for the presence of dark icons.

This change improves the visual consistency of plugins in dark mode environments.

* enhance(cli/icon): add plugin icon support with multiple categories

- Added support for light and dark icons for various plugin categories including agent, datasource, extension, model, tool, and trigger.
- Replaced the previous single icon implementation with a structured map for better organization and retrieval of icons based on category and theme.
- Removed the old Python icon file to streamline asset management.

This update improves the visual representation of plugins across different themes, enhancing user experience.

* change icons

* fix

* fix

* comments
2025-07-08 12:48:48 +08:00
Blackoutta 18d4151883 optimize: skip sleep for remote plugin runtime during restart, making the debugging experience smoother (#387) 2025-07-07 18:17:02 +08:00
Yeuoly 56fcd686e1 feat: add active request tracking to health check and dispatch routes (#384)
- Implemented middleware to track active requests and active dispatch requests using atomic counters.
- Updated health check response to include counts of active requests and active dispatch requests.
- Integrated the new middleware into the HTTP server and plugin dispatch group for improved monitoring.
2025-07-04 19:56:42 +08:00
Harry d39e38e6cd feat: update request types for online drive browsing and downloading 2025-07-04 11:34:27 +08:00
AkisAya 412589f94a skip error plugin names (#381) 2025-07-03 23:43:03 +08:00
Harry 57fd5221b7 refactor: rename OnlineDocumentPageChunk to DatasourceGetPagesResponse and update related references 2025-07-02 23:11:39 +08:00
Harry 9321ebdc02 Merge remote-tracking branch 'origin/main' into feat/datasource
# Conflicts:
#	internal/core/plugin_daemon/access_types/access.go
#	internal/server/controllers/definitions/definitions.go
#	internal/server/http_server.gen.go
2025-07-02 22:53:35 +08:00
Yeuoly 6d6fb3892e feat: add InvokeLLMWithStructuredOutput functionality (#369)
* feat: add InvokeLLMWithStructuredOutput functionality

- Introduced a new method InvokeLLMWithStructuredOutput to the BackwardsInvocation interface for handling structured output requests.
- Added corresponding request and response types to support structured output.
- Implemented the method in both RealBackwardsInvocation and MockedDifyInvocation for testing purposes.
- Updated permission handling and task execution for the new structured output invocation type.

This enhancement allows for more flexible and detailed responses from the LLM, improving the overall functionality of the invocation system.

* refactor: enhance LLMResultChunkWithStructuredOutput structure

- Updated the LLMResultChunkWithStructuredOutput type to include additional fields: Model, SystemFingerprint, and Delta.
- Added comments to clarify the reasoning behind the structure and the use of type embedding for JSON marshaling.

This change improves the clarity and functionality of the LLMResultChunkWithStructuredOutput type, ensuring proper JSON serialization.

* refactor: streamline LLMResultChunk construction in InvokeLLMWithStructuredOutput

- Simplified the construction of LLMResultChunk and LLMResultChunkWithStructuredOutput by removing unnecessary type embedding.
- Enhanced readability and maintainability of the code while preserving functionality.

This change contributes to cleaner code and improved clarity in the handling of structured output responses.
0.1.3
2025-06-30 12:07:46 +08:00
@defstream 7a7848b3ae Update README.md (#372) 2025-06-28 12:31:57 +08:00
Harry c1e5ecd06b feat: rename online driver to online drive and update related classes and methods :) 2025-06-27 20:11:48 +08:00