Commit Graph

40 Commits

Author SHA1 Message Date
Yeuoly 0eb2cbe284 Fix/remove tenant id reference in readme service (#530)
* fix: remove tenant ID reference from plugin readme service

* fix: rename PluginReadme to PluginReadmeRecord to remove tenant ID reference

* fix: handle error when saving plugin readme map to database

* fix: remove TenantId from FetchPluginReadme request binding

* fix: standardize error message casing for plugin unique identifier validation
2025-12-12 15:35:06 +08:00
Yunlu Wen f954688930 enterprise related adjustment (#510)
* remove enterprise logics

* release lock if runtime already installed

* support setting serverless endpoint by api

* remove global tenant id totally

* scan timeout tasks

* adding comments

* use index in loop

* add log
2025-12-03 18:13:54 +08:00
Yeuoly 888ad788bc refactor: plugin lifecycle control panel (#499)
* refactor: introduce local plugin control panel and cleanup environment setup process

* fix: args

* refactor: new local runtime

* temp: stash work for refactor on RemotePluginServer

* refactor: unify local runtime lifetime and sperate init environment process

* chore: add missing files

* stash

* refactor: local plugin lifetime control

* refactor: complete installation process of control panel

* refactor: adapt service layer to new controlpanel

* refactor: pluginManager.Install

* fix: add routine wrap to InstallServerless, avoid blocking main thread

* feat: reinstall serverless runtime

* chore: add comments to Reinstall and update confusing naming

* refactor: unify install plugin service

* refactor: add labels to debugging runtime

* refactor: add getters to plugin manager

* refactor: split install service to decode/install_task/install service

* ???

* refactor: adapt controllers

* refactor: session write

* refactor: session runtime

* Refine install task orchestration (#501)

* refactor: installing task

* refactor cluster management, decouple lifetime management and cluster

* fix cli test command

* fix: cleanup TODO comments and implement GracefulStop for instance

* feat: add logger to control panel

* fix: multiple nil references

* refactor: better lifetime control

* refactor: better cycle interval

* fix(LocalPluginRuntime): prevent returning err when it's not  error

* fix: avoid adding empty PipExtraArgs

* fix: missing errors in Environment init

* fix: add truncateMessage to avoid db explosion

* cleanup: better lifecycle management

* fix: init status at the beginning of installation

* optimize: GracefulStop for pluginInstance

* refactor: tests

* refactor: centralize routine labels (#504)

* cleanup: RoutineKey

* fix: init routine pool

* fix: correctly handle cluster register error

* fix: memory leak

* fix: add \n to instance write

* fix(installer.go): set success to true after succeed for defer func

* refactor

* fix: missing cwd in testutils

* fix: scaleup default runtime nums to 1 when testing

* fix: localruntime appconfig in testing module

* Update internal/core/local_runtime/load_balancing.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* fix: more efficiency implement in installer_local.go

* fix: returns after failing in onDebuggingRuntimeDisconnected

* fix: returns after failing in onDebuggingRuntimeDisconnected

* fix: splits tests

* refactor: naming

* refactor: manifest.VersionX

* fix: adapt SetDefault to tests

* fix: enforce use constants in DBType

* fix: generate

* fix: linter

* cleanup tests

* refactor: change  package to

* cleanup: useless codes

* Update internal/cluster/plugin.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* cleanup

* refactor: decouple connection_key management from debugging_time

* refactor: confused naming

* feat: recycle resources to adapt to https://github.com/langgenius/dify-plugin-daemon/pull/500

* refactor: confusing redirecting

* fix: support get serverless runtime

* fix: race condition in Launching

* fix: avoid ManifestValidate in first step of debugging handshake

* fix: adding ReleaseAllLocks to finalizers

* wtf: what a beautiful code

* refactor: rename Stream.Async to Stream.Process

* fix: kill process if daed instance was detected

* fix: correctly handle failures

* fix: consistence of difference interfaces

* fix: add stacktrace to panic

* fix: only trigger once  event

* fix: ensure plugin runtime was shutdown

* feat: cleanup install tasks

* fix: add scale logs

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-11-18 17:28:02 +08:00
Maries 4589841b0c feat: introduce trigger (#482)
* feat(plugins): add FetchPluginReadme endpoint and update launch configurations

* feat: add PluginReadme database model

* feat: implement readme extracting and storage

* feat: implement readme endpoint

* feat: add plugin asset extraction endpoint with caching support

* Implement trigger functionality and clean up dynamic select code

- Added new trigger-related access types and actions in access.go.
- Introduced new HTTP routes for trigger operations in http_server.gen.go.
- Updated plugin declaration to include triggers in plugin_entities.
- Removed unused dynamic select service and controller files.
- Enhanced message handling in debugging_runtime to support trigger registration.

This update enhances the plugin system by integrating trigger capabilities while cleaning up legacy code.

* Refactor trigger-related types and enhance dynamic select functionality

- Updated TriggerProviderIdentity and TriggerProviderConfiguration to improve structure and validation.
- Renamed TriggerConfiguration to TriggerDeclaration for clarity.
- Added CredentialType to RequestDynamicParameterSelect for better request handling.
- Enhanced PluginDecoderHelper to read and unmarshal trigger files correctly.

These changes streamline the trigger system and improve the overall request handling in the plugin architecture.

* Add trigger functionality and enhance database integration

- Introduced TriggerInstallation model for managing trigger installations in the database.
- Updated autoMigrate function to include trigger installations in the migration process.
- Added new HTTP routes for listing and retrieving triggers in the HTTP server.
- Implemented ListTriggers and GetTrigger controller functions for handling trigger requests.
- Enhanced plugin management functions to create, update, and delete trigger installations during plugin lifecycle events.

These changes integrate trigger capabilities into the system, improving the overall plugin functionality and management.

* feat: add remapping for trigger icons in MediaBucket

- Enhanced the RemapAssets function to include remapping of trigger identity icons and dark icons.
- Added error handling for remapping failures to ensure robust asset management.

These changes improve the handling of trigger assets within the plugin system, ensuring icons are correctly remapped as needed.

* feat: add Multiple field to TriggerParameter for enhanced configuration

- Introduced a new Multiple field in the TriggerParameter struct to allow for multiple values in trigger configurations.
- This addition improves the flexibility of trigger parameters within the plugin system.

These changes enhance the capability of trigger parameters, enabling more complex configurations.

* feat: add Multiple field to ProviderConfig for enhanced configuration

- Introduced a new Multiple field in the ProviderConfig struct to allow for multiple values in provider configurations.
- This addition improves the flexibility of provider options within the plugin system.

These changes enhance the capability of provider configurations, enabling more complex setups.

* fix(plugin): update validation error messages in ManifestValidate method

- Enhanced error messages in the ManifestValidate function to include 'trigger' in the validation checks for plugin declarations.
- Updated logic to ensure that all relevant fields are considered when validating the presence of mutually exclusive parameters.

* feat(trigger): add CHECKBOX parameter type to plugin entities  and refactor the trigger provider strcuture

- Introduced a new CHECKBOX parameter type in constant.go for plugin entities.
- Updated tool_declaration.go and trigger_declaration.go to include TOOL_PARAMETER_TYPE_CHECKBOX and TRIGGER_PARAMETER_TYPE_CHECKBOX respectively.
- Enhanced validation logic to accommodate the new CHECKBOX type in parameter checks.

* fix(trigger): update SubscriptionSchema validation in TriggerProviderDeclaration

- Changed SubscriptionSchema validation from 'omitempty' to 'required' in TriggerProviderDeclaration to ensure it is always provided.
- Updated SubscriptionConstructor field to be a pointer to allow for optional inclusion in the trigger provider configuration.

* fix(trigger): rename ParametersSchema to Parameters in SubscriptionConstructor

- Updated the SubscriptionConstructor struct to rename the ParametersSchema field to Parameters for consistency.
- Adjusted related JSON and YAML marshaling logic to reflect the new field name, ensuring proper handling of trigger parameters.

* refactor(trigger): enhance YAML unmarshalling for SubscriptionConstructor and SubscriptionSchema

- Introduced a new helper function to convert YAML nodes to ProviderConfig lists, improving the handling of subscription_schema and credentials_schema.
- Updated the UnmarshalYAML method to utilize the new function, simplifying the logic for parsing different YAML formats.
- Ensured proper initialization of SubscriptionConstructor fields to prevent nil pointer dereferences.

* fix(trigger): update SubscriptionConstructor validation in TriggerProviderDeclaration

- Changed the validation for SubscriptionConstructor in TriggerProviderDeclaration from 'omitempty,dive' to 'omitempty' to simplify the validation logic.
- Ensured that the SubscriptionConstructor field remains optional while maintaining its intended functionality.

* refactor(trigger): rename Trigger to Event in plugin entities and related structures

- Updated the naming conventions in trigger_declaration.go to replace 'Trigger' with 'Event' for better clarity and consistency.
- Adjusted related types, validation functions, and unmarshalling logic to reflect the new 'Event' terminology.
- Ensured that all references to triggers in the codebase are updated to events, including in the SubscriptionConstructor and response structures.

* refactor(trigger): rename TriggerInvoke to TriggerInvokeEvent and update related structures

- Renamed TriggerInvoke function and associated request/response types to TriggerInvokeEvent for improved clarity.
- Updated routing and controller methods to reflect the new naming convention.
- Ensured all references to the trigger invoke functionality are consistent with the new event terminology.

* refactor(trigger): remove Subscription struct from trigger_declaration.go and update TriggerDispatchEventRequest

- Removed the Subscription struct from trigger_declaration.go to streamline the codebase.
- Added Credentials field to TriggerDispatchEventRequest for enhanced functionality and clarity.
- Ensured that the changes maintain consistency with the existing naming conventions and structures.

* fix(trigger): improve nil checks for SubscriptionConstructor in TriggerProviderDeclaration

- Added nil checks for SubscriptionConstructor before accessing its fields to prevent potential nil pointer dereferences.
- Ensured that Parameters and CredentialsSchema are initialized only if SubscriptionConstructor is not nil, enhancing code robustness.

* fix(plugin): add recovery mechanism in OnTraffic to handle panics

- Introduced a deferred function in OnTraffic to recover from panics, logging the error and stack trace for better debugging.
- This enhancement improves the stability of the DifyServer by preventing crashes due to unexpected runtime errors.

* feat(trigger): add Subscription field to TriggerInvokeEventRequest

- Introduced a new Subscription field in the TriggerInvokeEventRequest struct to accommodate subscription data.
- Ensured the field is marked as required, enhancing the request's functionality and validation requirements.

* refactor(event): simplify EventDescription structure in EventDeclaration

- Removed the EventDescription struct and replaced it with a direct I18nObject field in EventDeclaration.
- This change streamlines the event configuration by reducing complexity while maintaining required validation for the description.

* feat(trigger): add UserID field to TriggerDispatchEventResponse

- Introduced a new UserID field in the TriggerDispatchEventResponse struct to include user identification in the response.
- The field is marked as optional, enhancing the response's flexibility while maintaining existing functionality.

* feat: add payload to TriggerDispatchEventResponse

* fix

* feat(trigger): update TriggerDispatchEventResponse structure

* fix: avoid path collusion

* fix: missing )

* fix: query param

* fix: form param

* fix: remove redundant dynamic parameter access type

* fix: remove dynamic parameter access type from validation

* Update internal/server/controllers/plugins.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: Stream <Stream_2@qq.com>
Co-authored-by: Yeuoly <admin@srmxy.cn>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-10-29 13:56:58 +08:00
Yunlu Wen 9e3081bd17 feat: support orphan plugin (#440)
* support orphan plugin in serverless mode

* add validation to UpgradePlugin

* log reinstalls
2025-09-12 20:34:38 +08:00
Yeuoly a6c8fae9c7 feat: add decode plugin from identifier endpoint (#349)
* feat: add decode plugin from identifier endpoint

- Introduced a new endpoint to decode a plugin from a unique identifier.
- Implemented the DecodePluginFromIdentifier function to handle decoding and verification of plugin signatures.
- Updated the HTTP server routes to include the new decode endpoint.

* refactor: update decode plugin endpoint path

- Moved the decode plugin from identifier endpoint to a new path for better organization.
- Updated the HTTP server routes accordingly to reflect the new endpoint structure.
2025-06-18 16:01:15 +08:00
Yeuoly d884cab8ae feat: add plugin reinstallation functionality and admin API support (#285)
* feat: add plugin reinstallation functionality and admin API support

- Implemented ReinstallToAWSFromPkg method to allow reinstallation of plugins on AWS Lambda, updating function URL and name.
- Added clearServerlessRuntimeCache method to manage serverless runtime cache.
- Enhanced LaunchPlugin to support an ignoreIdempotent flag for forced reinstallation.
- Introduced admin API endpoints for plugin reinstallation, secured with an API key validation middleware.
- Updated configuration to include AdminApiEnabled and AdminApiKey settings.

* refactor: update plugin reinstallation endpoint and improve unauthorized response

- Changed the plugin reinstallation endpoint from "/plugins/reinstall" to "/plugin/serverless/reinstall" for better clarity.
- Modified the unauthorized response in the AdminAPIKey middleware to return a more descriptive JSON message.
2025-05-15 15:22:25 +08:00
Yeuoly b19140ca23 refactor: optimize plugin declaration caching and encoding (#34) 2025-02-25 20:52:51 +08:00
Yeuoly 15060ee312 refactor: move entities to pkg 2025-01-06 14:52:02 +08:00
Yeuoly f6aa10a9bc fix: multiple plugin installation 2024-12-31 19:04:58 +08:00
Yeuoly dd5325b681 feat: support delete all install tasks 2024-11-25 17:01:25 +08:00
Yeuoly ef1ffa9f6c refactor: change to use generic errors in controllers 2024-11-22 16:44:44 +08:00
Yeuoly b0680f9403 fix: delete plugin item from install task 2024-11-15 15:29:25 +08:00
Yeuoly 9c9a6c26c0 feat: support upload bundle 2024-11-14 22:59:51 +08:00
Yeuoly 44a2de1f62 feat: support upload bundle 2024-11-14 21:23:33 +08:00
Yeuoly 26fd94ebcd refactor: convert snakecase to camelcase 2024-11-12 21:24:41 +08:00
Yeuoly c57b554245 feat: checking missing dependencies 2024-11-12 19:51:28 +08:00
Yeuoly eeadb3e29c refactor: support upgrade plugin to any verions 2024-10-25 17:41:14 +08:00
Yeuoly f339a21001 feat: support remove single item from installation task 2024-10-25 13:19:54 +08:00
Yeuoly a55f6f507f refactor: install plugin from local 2024-10-18 20:29:02 +08:00
Yeuoly 090f1e004d fix: install progress has bad handing logics 2024-10-16 14:36:48 +08:00
Yeuoly fc7bfc91b4 fix: async install plugins task 2024-10-16 14:01:08 +08:00
Yeuoly 3de55a81ce refactor: installing plugin 2024-10-14 21:07:56 +08:00
Yeuoly a74a1a4801 refactor: plugin installation changed to sync tasks 2024-10-14 18:11:22 +08:00
Yeuoly 9055ccf69d feat: support install source 2024-10-10 15:01:14 +08:00
Yeuoly a12518dd7f feat: support verify signature 2024-10-09 23:11:23 +08:00
Yeuoly f66f763392 fix: missing model parameter rules default template transformers in default plugin decoder 2024-10-08 22:33:11 +08:00
Yeuoly 69072bd8d5 feat: support force verifying sig 2024-10-08 21:28:22 +08:00
Yeuoly 37a5b12c86 feat: support fetch tool entity 2024-09-29 17:09:09 +08:00
Yeuoly d0bc14a515 feat: installation lifetime 2024-09-20 21:52:35 +08:00
Yeuoly e7172076f9 fix: controller validates 2024-09-19 17:25:35 +08:00
Yeuoly e7b57878d2 refactor: uri 2024-09-18 13:01:28 +08:00
Yeuoly 91fd7c3e7f feat: plugin managment 2024-09-14 18:39:02 +08:00
Yeuoly 026c76ba1d refactor: plugin identifier 2024-09-14 17:23:06 +08:00
Yeuoly b52cbe0815 refactor 2024-09-14 16:48:11 +08:00
Yeuoly 1df861f9fc refactor: checksum 2024-09-11 14:46:16 +08:00
Yeuoly cda65741b1 feat: install 2024-09-10 18:26:22 +08:00
Yeuoly 50b7c6d138 feat: plugin install 2024-09-09 12:07:11 +08:00
Yeuoly 1c2d5e6bca feat: support plugin assets 2024-09-04 20:35:56 +08:00
Yeuoly dc402cd948 refactor: plugin id/identity and plugin installation 2024-09-03 22:38:19 +08:00