Commit Graph

7 Commits

Author SHA1 Message Date
L1nSn0w 98b24b5588 fix(cache): pass watch keys to Transaction so optimistic lock actually works (#701)
* feat(redis): add Redis Cluster client path to plugin-daemon

Wire the missing third branch so operators can point the daemon at a
Redis Cluster without falling back to dialing an empty REDIS_HOST:PORT
and panicking at startup.

* introduce REDIS_USE_CLUSTERS / REDIS_CLUSTERS /
  REDIS_CLUSTERS_PASSWORD envs, aligned with dify api naming so Helm
  can set a single env group
* branch in PluginManager.Launch for Cluster ahead of the existing
  Sentinel / standalone paths; falls back to REDIS_PASSWORD if the
  cluster-specific password is not set
* implement cache.InitRedisClusterClient via redis.NewClusterClient;
  downstream cache / lock / pub-sub helpers keep working because
  `client` is declared as redis.UniversalClient

Redis Cluster disables SELECT DB, so the cluster branch does not
plumb RedisDB. This is intentional and documented in the release
note accompanying the overall Redis Cluster support work.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(redis): validate Redis cluster addresses and improve transaction handling

* feat(redis): implement CSV parsing utility and refactor Redis transaction handling

* feat(redis): scope plugin-daemon to standalone + sentinel only

Narrow the plugin-daemon Redis client to standalone and sentinel modes;
remove the cluster code paths that were briefly exercised on this branch.

Changes:
- Remove REDIS_USE_CLUSTERS / REDIS_CLUSTERS / REDIS_CLUSTERS_PASSWORD
  env fields from app.Config.
- Remove the cluster branch (plus the two cluster-specific fail-fast
  guards on REDIS_USE_CLUSTERS + REDIS_USE_SENTINEL and REDIS_DB != 0)
  from PluginManager.Launch. The init block is now just sentinel or
  standalone.
- Delete cache.InitRedisClusterClient. The main redis client + all
  helpers (Transaction, pub/sub, lock, etc.) already route through
  redis.UniversalClient so downstream code needs no adjustment.

Intentionally kept:
- Transaction(fn, watchKeys...) signature: the variadic signature is
  backwards-compatible with every existing Transaction(fn) call and the
  one new-style caller (debugging_service) benefits from real WATCH
  semantics even on standalone — this is correctness-independent of
  cluster support.
- parser.SplitAndTrimCSV: the sentinel branch migrated to it in 617e7a5f
  and still uses it to parse REDIS_SENTINELS, so the helper has a
  standalone/sentinel consumer and stays.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 09:33:28 +08:00
Blackoutta e1cb3cf0bb feat: support configurable redis key prefix (#692) 2026-04-14 15:58:37 +08:00
wangxiaolei 35825a7925 feat: support opentelemetry (#583) 2026-01-27 13:36:13 +08:00
wangxiaolei e8f8f17f1a feat: add redis distribute lock (#564)
* feat: add redis distribute lock

* refactor: use structured logging for python env

---------

Co-authored-by: Yeuoly <admin@srmxy.cn>
2026-01-15 17:06:02 +08:00
Oscaner Miao 2be1f651a9 feat(#450): add Redis SSL/TLS configuration support (#558)
* feat(#450): add Redis SSL/TLS configuration support

Add comprehensive SSL/TLS support for Redis connections with configurable certificate verification modes. Introduces new environment variables for SSL configuration including REDIS_USE_SSL, REDIS_SSL_CERT_REQS (supporting CERT_NONE, CERT_OPTIONAL, CERT_REQUIRED), and REDIS_SSL_CA_CERTS for custom CA certificates.

Changes:
- Add Redis SSL configuration options to .env.example
- Implement RedisTLSConfig() method to build tls.Config based on environment settings
- Pass TLS config to both standard Redis and Sentinel mode initializers
- Support custom CA certificate loading and verification modes
- Set minimum TLS version to 1.2 for security
- Minor whitespace cleanup in existing config comments

This enables secure Redis connections in production environments with flexible certificate verification options.

* fix(#450): prevent reference cycle in TLS config and simplify SSL setup

- Capture only RootCAs in VerifyConnection closure to avoid retaining
  entire tlsConf and potential reference cycles
- Remove redundant nil checks for tlsConf in Redis client initialization
  since tlsConf is guaranteed to be non-nil when useSsl is true
- Update comments to reflect actual behavior and constraints

* fix(#450): improve Redis TLS certificate verification logic for optional certificates

* fix(#450): simplify Redis TLS certificate verification logic for optional and required certificates

* docs(#450): add note for CA certificate file path in Redis SSL configuration

* test(#450): add comprehensive tests for Redis TLS configuration

* fix(#450): enhance Redis SSL configuration documentation and enforce CA cert requirement

* fix(#450): add nil TLS parameter to InitRedisClient calls in tests

Update all InitRedisClient function calls across test files to include the new nil parameter for TLS configuration. This change maintains backward compatibility by explicitly passing nil for TLS settings in non-TLS test scenarios.

* fix(#450): add default TLS configuration for Redis client when no tlsConf is provided
2026-01-13 16:27:16 +08:00
Byron.wang ca3d00229e Feat/Implement structured logging and Trace ID propagation (#552)
* use slog instead of log package and format to new log schema

* update the environment name to LOG_OUTPUT_FORMAT

* add the env to .env.example

* fix log reference error

* change the order of milldlewares

* delete unused code

* fix the concurrently session potential race condition

* fix the log format in tests

* update the duplicate code

* refactor: convert log functions to slog structured format

- Change log.Error/Info/Warn/Debug/Panic to accept msg + key-value pairs
- Remove printf-style formatting from log functions
- Update log calls in internal/cluster, internal/db, internal/core/session_manager
- Remove unused 'initialized' variable from log package
- Remaining files will be updated in follow-up commits

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: update all log call sites to use slog structured format

Convert all log.Error, log.Info, log.Warn, log.Debug, and log.Panic
calls from printf-style formatting to slog key-value pairs.

Before: log.Error("failed to do something: %s", err.Error())
After:  log.Error("failed to do something", "error", err)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: update cmd/ log calls to use slog structured format

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: implement GnetLogger for structured logging in gnet

* refactor: remove deprecated log visibility functions and related calls

* feat: enhance session management with trace and identity context propagation

* feat: implement serverless transaction handler and writer for plugin runtime

* refactor: rename context field to traceCtx in RealBackwardsInvocation

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Yeuoly <admin@srmxy.cn>
2025-12-30 11:00:48 +08:00
Yeuoly 888ad788bc refactor: plugin lifecycle control panel (#499)
* refactor: introduce local plugin control panel and cleanup environment setup process

* fix: args

* refactor: new local runtime

* temp: stash work for refactor on RemotePluginServer

* refactor: unify local runtime lifetime and sperate init environment process

* chore: add missing files

* stash

* refactor: local plugin lifetime control

* refactor: complete installation process of control panel

* refactor: adapt service layer to new controlpanel

* refactor: pluginManager.Install

* fix: add routine wrap to InstallServerless, avoid blocking main thread

* feat: reinstall serverless runtime

* chore: add comments to Reinstall and update confusing naming

* refactor: unify install plugin service

* refactor: add labels to debugging runtime

* refactor: add getters to plugin manager

* refactor: split install service to decode/install_task/install service

* ???

* refactor: adapt controllers

* refactor: session write

* refactor: session runtime

* Refine install task orchestration (#501)

* refactor: installing task

* refactor cluster management, decouple lifetime management and cluster

* fix cli test command

* fix: cleanup TODO comments and implement GracefulStop for instance

* feat: add logger to control panel

* fix: multiple nil references

* refactor: better lifetime control

* refactor: better cycle interval

* fix(LocalPluginRuntime): prevent returning err when it's not  error

* fix: avoid adding empty PipExtraArgs

* fix: missing errors in Environment init

* fix: add truncateMessage to avoid db explosion

* cleanup: better lifecycle management

* fix: init status at the beginning of installation

* optimize: GracefulStop for pluginInstance

* refactor: tests

* refactor: centralize routine labels (#504)

* cleanup: RoutineKey

* fix: init routine pool

* fix: correctly handle cluster register error

* fix: memory leak

* fix: add \n to instance write

* fix(installer.go): set success to true after succeed for defer func

* refactor

* fix: missing cwd in testutils

* fix: scaleup default runtime nums to 1 when testing

* fix: localruntime appconfig in testing module

* Update internal/core/local_runtime/load_balancing.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* fix: more efficiency implement in installer_local.go

* fix: returns after failing in onDebuggingRuntimeDisconnected

* fix: returns after failing in onDebuggingRuntimeDisconnected

* fix: splits tests

* refactor: naming

* refactor: manifest.VersionX

* fix: adapt SetDefault to tests

* fix: enforce use constants in DBType

* fix: generate

* fix: linter

* cleanup tests

* refactor: change  package to

* cleanup: useless codes

* Update internal/cluster/plugin.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* cleanup

* refactor: decouple connection_key management from debugging_time

* refactor: confused naming

* feat: recycle resources to adapt to https://github.com/langgenius/dify-plugin-daemon/pull/500

* refactor: confusing redirecting

* fix: support get serverless runtime

* fix: race condition in Launching

* fix: avoid ManifestValidate in first step of debugging handshake

* fix: adding ReleaseAllLocks to finalizers

* wtf: what a beautiful code

* refactor: rename Stream.Async to Stream.Process

* fix: kill process if daed instance was detected

* fix: correctly handle failures

* fix: consistence of difference interfaces

* fix: add stacktrace to panic

* fix: only trigger once  event

* fix: ensure plugin runtime was shutdown

* feat: cleanup install tasks

* fix: add scale logs

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-11-18 17:28:02 +08:00