[PR #185] [MERGED] Multi-Provider Expansion & Testing Infrastructure #221

New Issue

2026-06-06T22:09:45-04:00

yindo commented

2026-06-06 22:09:45 -04:00

📋 Pull Request Information

Original PR: https://github.com/vxcontrol/pentagi/pull/185
Author: @asdek
Created: 3/9/2026
Status: ✅ Merged
Merged: 3/9/2026
Merged by: @asdek

Base: master ← Head: feature/next_release

📝 Commits (10+)

c68f933 fix: treat screenshot failure as non-critical in browser tool
5720402 fix: propagate container lookup error in GetTool for terminal/file tools
e36f47f fix: prevent http.DefaultClient mutation in search tools
c716ec6 test: remove unused test server and dead imports in search tool tests
ddd0b0e test: add unit tests for perplexity and google search tools
c538b80 test: address review feedback on search tool tests
539f6c2 feat: add CN LLM providers (DeepSeek, GLM, Kimi, Qwen) with bug fixes
8354f7e chore(deps): bump rollup
f4da45e chore(deps): bump go.opentelemetry.io/otel/sdk
1c3f149 feat: add Novita AI as optional LLM provider

📊 Changes

122 files changed (+19200 additions, -3653 deletions)

View changed files

📝 .env.example (+31 -2)
📝 .gitignore (+9 -0)
📝 .vscode/launch.json (+12 -0)
📝 .vscode/settings.json (+1 -0)
➕ CLAUDE.md (+150 -0)
📝 Dockerfile (+4 -1)
📝 README.md (+848 -151)
📝 backend/cmd/ctester/main.go (+50 -3)
📝 backend/cmd/ftester/main.go (+1 -1)
📝 backend/cmd/ftester/worker/executor.go (+15 -32)
📝 backend/cmd/installer/wizard/controller/controller.go (+189 -12)
📝 backend/cmd/installer/wizard/locale/locale.go (+231 -66)
📝 backend/cmd/installer/wizard/models/llm_provider_form.go (+133 -9)
📝 backend/cmd/installer/wizard/models/llm_providers.go (+4 -0)
📝 backend/cmd/installer/wizard/models/search_engines_form.go (+50 -0)
📝 backend/cmd/installer/wizard/models/types.go (+8 -0)
📝 backend/cmd/installer/wizard/registry/registry.go (+4 -0)
📝 backend/docs/config.md (+295 -23)
📝 backend/docs/database.md (+2 -2)
📝 backend/docs/flow_execution.md (+4 -0)

...and 80 more files

📄 Description

Description

Problem

PentAGI's custom provider architecture only supported a single OpenAI-compatible endpoint at a time, preventing users from simultaneously configuring multiple cloud services. Users had to choose one custom provider at startup and couldn't switch between different LLM services (DeepSeek, GLM, Kimi, Qwen) through the UI. AWS Bedrock authentication was inflexible, forcing users into a single credential model. The codebase had insufficient test coverage for critical tool components, multiple security vulnerabilities in resource management, and several quality issues affecting production stability.

Solution

This preparation release consolidates 27 commits across 14 merged pull requests, delivering:

LLM Provider Ecosystem Expansion: Native first-class support for DeepSeek, GLM (智谱清言), Kimi (月之暗面), and Qwen (通义千问) as individual providers. Users can now configure all four Chinese cloud services simultaneously with dedicated API keys and switch between them instantly through the Settings UI, eliminating the previous single-custom-provider limitation
Flexible AWS Bedrock Authentication: Three authentication modes (AWS SDK default chain, bearer token, static credentials) with priority-based selection for diverse deployment scenarios
Comprehensive Testing Infrastructure: 15+ new unit test files covering tool registries, agent contexts, custom JSON types, executor logic, and search tool integrations - significantly improving code reliability and maintenance confidence
Security Hardening: CA private key cleanup post-signing, resource leak fixes (browser HTTP clients, terminal tar operations), Bedrock toolConfig nil safety
Code Quality Improvements: Fixed proxy configuration bugs, eliminated debug output, corrected typos, expanded .gitignore patterns

Closes #166, #167, #168, #170, #171, #172

Type of Change

🚀 New feature (non-breaking change which adds functionality)
🐛 Bug fix (non-breaking change which fixes an issue)
🧪 Test update
🛡️ Security update
📚 Documentation update

Areas Affected

Core Services (Backend API)
AI Agents (Researcher/Developer/Executor)
Security Tools Integration
External Integrations (LLM/Search APIs)
Documentation
Infrastructure/DevOps

Testing and Verification

Test Configuration

PentAGI Version: 1.2.1-dev (post-1.2.0)
Docker Version: 24.0+
Host OS: Linux/macOS/Windows
LLM Providers: OpenAI, Anthropic, DeepSeek, GLM, Kimi, Qwen
Enabled Features: Core + new providers

Test Steps

Run all new unit tests (go test ./...) - 100% pass rate
Verify DeepSeek/GLM/Kimi/Qwen provider initialization with valid API keys
Test AWS Bedrock with all three authentication methods
Confirm Google Search proxy configuration applies correctly
Test browser tool graceful screenshot failure handling
Verify terminal tar operations handle large files without OOM
Validate CA key cleanup after certificate signing

Test Results

✅ 15+ new test files with comprehensive coverage:
- args_test.go: Bool/Int64 custom JSON unmarshalling (20+ test cases)
- context_test.go: Agent context chaining and isolation
- registry_test.go: Tool type mapping and completeness validation
- executor_test.go: Custom executor tool definitions and execution
- terminal_test.go: Terminal formatting and name generation
- google_test.go: Google Search service creation and parsing
- perplexity_test.go: Perplexity API error handling and formatting
✅ All existing tests pass - no regressions introduced
✅ Security fixes verified: CA key removed, resource leaks eliminated
✅ Proxy configuration fixed in Google/Tavily/Traversaal search tools
✅ Nil pointer safety in Bedrock provider toolConfig handling
✅ Provider validation returns clear errors instead of misleading OpenAI fallback messages

Security Considerations

Fixes Implemented:

Certificate Authority Key Management (PR #168)
- CA private key (service_ca.key) deleted immediately after signing server certificate
- Eliminates attack surface if container filesystem is compromised
- Attacker can no longer issue arbitrary trusted certificates post-breach
Resource Leak Prevention (PRs #150, #167, #168)
- Browser tool: Screenshot failure no longer discards successfully-fetched page content
- Terminal tar: 50MB file size validation prevents memory exhaustion attacks
- HTTP clients: Proper client isolation in search tools (no global mutation)
Bedrock Runtime Safety (PR #166)
- Nil check prevents panic when toolConfig is undefined
- Graceful degradation instead of agent crash
Provider Authentication Security
- DeepSeek/GLM/Kimi/Qwen: Explicit API key validation with clear error messages
- Bedrock: Priority-based auth selection with secure credential handling
- No credential leakage in logs or error messages

No New Vulnerabilities Introduced:

All new providers follow existing secure credential management patterns
Test code uses isolated contexts and mock data (no real API keys)
.gitignore expansion prevents accidental IDE/OS file commits

Performance Impact

Improvements:

Testing Performance: t.Parallel() in new test files enables concurrent test execution
HTTP Client Optimization: Search tools now create dedicated clients instead of mutating global http.DefaultClient (eliminates race conditions)
Browser Tool Efficiency: Screenshot failures no longer block content retrieval - faster response times when image capture fails

No Degradation:

Authentication method selection adds <1ms overhead (priority checks are simple conditionals)
Test suite expansion does not affect runtime performance (tests don't run in production)
Langchaingo update (v0.1.14-update.2) maintains compatibility with existing performance characteristics

Resource Usage:

Chinese providers use same connection pooling as other LLM integrations
Terminal tar size validation prevents unbounded memory allocations

Documentation Updates

Configuration documentation updates - DeepSeek/GLM/Kimi/Qwen env vars in .env.example and docker-compose.yml
Example configurations - Added novita.provider.yml custom provider example for Novita AI aggregator
GraphQL schema updates - No changes (provider management uses existing schema)
Other: Fixed multiple typos ("bellow" → "below" in terminal output), expanded .gitignore with IDE/OS patterns

Deployment Notes

New Environment Variables (Optional):

# Chinese Mainland LLM Providers (all optional)
DEEPSEEK_API_KEY=sk-...
DEEPSEEK_SERVER_URL=https://api.deepseek.com  # optional, uses default if not set

GLM_API_KEY=...
GLM_SERVER_URL=https://open.bigmodel.cn/api/paas  # optional

KIMI_API_KEY=...
KIMI_SERVER_URL=https://api.moonshot.cn  # optional

QWEN_API_KEY=sk-...
QWEN_SERVER_URL=https://dashscope.aliyuncs.com/compatible-mode  # optional

# AWS Bedrock Enhanced Authentication (optional)
BEDROCK_DEFAULT_AUTH=true  # Use AWS SDK credential chain (IAM roles, env vars, etc.)
BEDROCK_BEARER_TOKEN=...   # Alternative: bearer token auth
# Existing BEDROCK_AWS_ACCESS_KEY_ID and BEDROCK_AWS_SECRET_ACCESS_KEY still work

Database Migrations:

20260301_add_chinese_llm_providers.sql - Adds deepseek, glm, kimi, qwen to PROVIDER_TYPE enum
Migrations run automatically on startup

Breaking Changes:

❌ None - All changes are backward compatible
Existing provider configurations remain functional
New providers are opt-in via environment variables

Compatibility:

✅ Fully backward compatible with v1.2.0
✅ No API changes to GraphQL or REST endpoints
✅ Existing deployments work without configuration changes
✅ langchaingo v0.1.14-update.2 compatible with existing provider implementations

Upgrade Steps:

Pull latest changes
(Optional) Add API keys for new providers to .env
Run docker compose up -d - migrations apply automatically
New providers appear in Settings UI if API keys are configured

Checklist

Code Quality

My code follows the project's coding standards
I have added/updated necessary documentation
I have added tests to cover my changes
All new and existing tests pass
I have run go fmt and go vet (for Go code)
I have run npm run lint (for TypeScript/JavaScript code)

Security

I have considered security implications
Changes maintain or improve the security model
Sensitive information has been properly handled

Compatibility

Changes are backward compatible
Breaking changes are clearly marked and documented
Dependencies are properly updated

Documentation

Documentation is clear and complete
Comments are added for non-obvious code
API changes are documented

Additional Notes

Key Changes by Category

🚀 New LLM Providers (PR #154)

Chinese Cloud Service Providers - First-class provider support enabling simultaneous multi-service configuration:

DeepSeek: Cost-effective reasoning models with competitive performance
GLM (智谱清言): Zhipu AI's general-purpose language models
Kimi (月之暗面): Moonshot AI's long-context models
Qwen (通义千问): Alibaba Cloud's Qianwen family

Key Improvement: Previously, users could only configure ONE custom OpenAI-compatible provider at a time, requiring application restart to switch between Chinese cloud services. Now all four providers can be enabled simultaneously with individual API keys, and users can instantly switch between them through the Settings UI.

Implementation Highlights:

Each provider has dedicated backend implementation with independent configuration
All providers follow OpenAI-compatible API pattern for easy integration
Explicit API key validation with clear error messages (no misleading "missing OpenAI API key" fallback)
Proper whitelisting in ProviderType.Valid() to prevent 422 Unprocessable Entity errors
Frontend GraphQL queries updated to load default agent configurations for each provider
Docker Compose env var declarations ensure keys are injected into containers

Custom Provider Example (PR #162) - Added novita.provider.yml configuration example:

Demonstrates custom provider setup for Novita AI aggregator service
Provides access to DeepSeek V3.2, Minimax M2.5, GLM-5 through single API endpoint
Test report included (novita-report.md) with performance validation
Shows model configuration pattern using moonshotai/kimi-k2.5

Contributor: @niuqun2003 (DeepSeek/GLM/Kimi/Qwen native providers), @Alex-wuhu (Novita custom provider example)

🔐 AWS Bedrock Authentication Enhancement

Multiple Authentication Methods - Flexible credential management for diverse deployment scenarios:

AWS SDK Default Chain (BEDROCK_DEFAULT_AUTH=true)
- IAM instance profiles (EC2, ECS, Lambda)
- Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
- Shared credentials file (~/.aws/credentials)
- SSO/SAML federation
Bearer Token (BEDROCK_BEARER_TOKEN=...)
- Pre-signed AWS Signature v4 tokens
- Temporary credentials from STS
Static Credentials (existing)
- BEDROCK_AWS_ACCESS_KEY_ID + BEDROCK_AWS_SECRET_ACCESS_KEY

Priority Order: Default Chain → Bearer Token → Static Credentials

Benefits:

Container orchestration platforms can use instance profiles (no hardcoded keys)
CI/CD pipelines can inject temporary STS tokens
Development environments retain simple static key configuration

Contributor: @asdek

🧪 Testing Infrastructure Expansion

15+ New Unit Test Files - Comprehensive coverage for previously untested components:

Tool Registry & Context (PR #171):

context_test.go: Agent context chaining, parent/current promotion, isolation verification
registry_test.go: Tool type mapping completeness, enum string representation, mutation safety

Custom JSON Types (PR #170):

args_test.go: Bool/Int64 unmarshalling from bare values, quoted strings, single quotes, whitespace trimming, overflow/underflow boundaries, nil safety

Executor & Terminal (PR #172):

executor_test.go: Custom executor tool definitions, execution behavior, unknown tool handling
terminal_test.go: Terminal name generation, ANSI formatting, input/output colorization

Search Tools (PR #153):

google_test.go: Service creation, proxy configuration, result parsing, special character handling
perplexity_test.go: API key validation, error response handling (10 HTTP codes), citation formatting

Test Quality Features:

t.Parallel() for concurrent execution (faster CI/CD)
context.Background() / t.Context() consistency across test files
Comprehensive edge case coverage (null, overflow, whitespace, unicode)
Mock providers for screenshot/search logs (test isolation)

Contributors: @mason5052 (primary author of all test infrastructure)

🛡️ Security Fixes

Certificate Authority Key Cleanup (PR #168):

Problem: CA private key left on disk after signing server certificate
Risk: Compromised container filesystem allows attacker to issue arbitrary trusted certificates
Fix: Delete service_ca.key, service.csr, service_ca.srl immediately after signing
Impact: Reduces attack surface in multi-tenant deployments

Resource Leak Prevention:

Browser Screenshot Failure (PR #150)
- Old behavior: Screenshot error discards successfully-fetched HTML/Markdown content
- New behavior: Log warning, return content with empty screenshot name
- Benefit: More reliable page content extraction, faster agent progress
Terminal Tar Operations (PR #168)
- Old behavior: No file size limit when creating tar archives
- New behavior: 50MB validation before io.Copy (prevents OOM attacks)
- Benefit: Protects against memory exhaustion from malicious large files
HTTP Client Mutation (PR #151)
- Old behavior: http.DefaultClient.Transport mutated when setting proxy in Tavily/Traversaal
- New behavior: Create dedicated http.Client instance for proxied requests
- Benefit: Eliminates race conditions in concurrent search operations

Bedrock Runtime Safety (PR #166):

Nil check on toolConfig before dereferencing in langchaingo integration
Prevents panic when Bedrock responses omit tool configuration

Contributors: @mason5052 (resource leaks, HTTP client), @Priyanka-2725 (Bedrock nil safety)

🐛 Bug Fixes

Proxy Configuration (PR #167):

Google Search: newSearchService() ignored constructed opts slice (proxy had no effect)
Fix: Pass opts... to NewService() call instead of hardcoded option.WithAPIKey()

Terminal Output (PR #164):

Typo: "keeps bellow" → "shown below" in file content header

Container Lookup Error (PR #152):

GetTool silently ignored GetFlowPrimaryContainer errors
Result: Cryptic "container is not running" instead of clear "missing container" message
Fix: Propagate error immediately with flow ID context

VS Code Settings (PR #163):

Expanded .gitignore with IDE patterns (.vscode/settings.json, .idea/, *.swp) and OS files (Thumbs.db)

Frontend Debug Cleanup (PR #141):

Removed 7 leftover console.log() statements from production code

Contributors: @mason5052 (proxy, container, frontend cleanup), @haosenwang1018 (gitignore)

📦 Dependency Updates

OpenTelemetry SDK (PR #161):

go.opentelemetry.io/otel/sdk: 1.36.0 → 1.40.0
4 minor version bump with performance improvements and bug fixes

Rollup (PR #155):

rollup: 4.53.1 → 4.59.0 (frontend build tool security and performance updates)

langchaingo (commit 274ec52):

Updated to stable release v0.1.14-update.2

Contributors: @dependabot[bot] (automated dependency PRs), @asdek (langchaingo)

Contributors

This preparation release includes contributions from:

@asdek (Dmitry Ng) - Project maintainer, Bedrock auth enhancement, langchaingo update, test coordination
@niuqun2003 - DeepSeek/GLM/Kimi/Qwen provider implementation and integration
@mason5052 - Testing infrastructure (15+ test files), security fixes, proxy bugs, code cleanup
@Alex-wuhu - Novita AI provider integration and model configuration
@Priyanka-2725 (Priyanka Singh) - Bedrock toolConfig nil safety fix
@haosenwang1018 - .gitignore expansion with IDE/OS patterns
@dependabot[bot] - Automated dependency security updates

Special thanks to the community for improving code quality, test coverage, and deployment flexibility! 🎉

Merged Pull Requests

#154 - feat: add Chinese mainland LLM providers (DeepSeek, GLM, Kimi, Qwen)
#162 - feat: add Novita AI as optional LLM provider
#166 - fix(bedrock): prevent runtime failure when toolConfig is undefined
#167 - fix: apply constructed opts to Google search service creation
#168 - fix: remove CA private key after certificate signing in entrypoint
#170 - test: add unit tests for Bool and Int64 custom JSON types
#171 - test: add unit tests for agent context and tool registry
#172 - test: add unit tests for executor helpers and terminal utilities
#150 - fix: treat screenshot failure as non-critical in browser tool
#151 - fix: prevent http.DefaultClient mutation in search tools
#152 - fix: propagate container lookup error in GetTool
#153 - test: add unit tests for perplexity and google search tools
#161 - chore(deps): bump go.opentelemetry.io/otel/sdk from 1.36.0 to 1.40.0
#155 - chore(deps): bump rollup from 4.53.1 to 4.59.0
#163 - chore: expand .gitignore with IDE and OS patterns
#164 - fix: correct typo in terminal ReadFile output message
#141 - chore: remove debug console.log statements from frontend

Migration Path

For existing deployments upgrading from v1.2.0:

Pull latest changes
```
docker compose pull
```
Review new environment variables (all optional):
- Add Chinese provider API keys (DeepSeek/GLM/Kimi/Qwen) for multi-service access
- Configure Bedrock auth method if using AWS infrastructure
Apply changes:
```
docker compose up -d
```
- Database migrations run automatically
- New providers appear in Settings UI when API keys are configured
Test (if using new providers):
- Create test flow
- Select DeepSeek/GLM/Kimi/Qwen model
- Verify agent initialization and response generation

No downtime required - Migrations are additive only (new enum values).

Future Work

This preparation release establishes infrastructure for upcoming features:

Extended Provider Testing: Comprehensive performance benchmarks for Chinese providers
Multi-Region Deployment Guide: Documentation for geo-distributed PentAGI instances
Provider Failover: Automatic fallback between providers for resilience
Cost Optimization: Per-provider pricing configuration and usage analytics
Enhanced Testing Dashboard: Visual test coverage reports and CI/CD integration

Note

Medium Risk
Medium risk because it expands provider/auth configuration paths (Bedrock auth modes, new providers, installer wiring) and refactors tool construction in ftester/search tools, which could impact runtime behavior if misconfigured.

Overview
Expands LLM provider support and configuration by adding env/config + installer UI wiring for deepseek, glm, kimi, and qwen, plus new Bedrock auth options (BEDROCK_DEFAULT_AUTH, BEDROCK_BEARER_TOKEN) and an Ollama Cloud API key (OLLAMA_SERVER_API_KEY). ctester/ftester are updated to recognize the new providers and enforce Bedrock auth presence.

Refactors tool initialization in function testing: ftester’s GetTool now only fetches the primary container when required (terminal/file) and fails fast on missing container, and search tools are constructed from the shared config.Config (enabling new DuckDuckGo params and SEARXNG_TIMEOUT).

Packaging/docs/dev quality updates: .env.example, README.md, and installer locale text are updated for the new providers and settings; Docker image now normalizes entrypoint.sh line endings and includes additional example provider configs; .gitignore and VS Code settings/launch configs are refreshed, and a new CLAUDE.md repo guidance file is added.

^{Written by Cursor Bugbot for commit da51f17973. This will update automatically on new commits. Configure here.}

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/vxcontrol/pentagi/pull/185 **Author:** [@asdek](https://github.com/asdek) **Created:** 3/9/2026 **Status:** ✅ Merged **Merged:** 3/9/2026 **Merged by:** [@asdek](https://github.com/asdek) **Base:** `master` ← **Head:** `feature/next_release` --- ### 📝 Commits (10+) - [`c68f933`](https://github.com/vxcontrol/pentagi/commit/c68f9335e06ae39606c4376a777a19cf624882ca) fix: treat screenshot failure as non-critical in browser tool - [`5720402`](https://github.com/vxcontrol/pentagi/commit/5720402301ac6fedadb2005f592e877436f81fda) fix: propagate container lookup error in GetTool for terminal/file tools - [`e36f47f`](https://github.com/vxcontrol/pentagi/commit/e36f47ffd379ea93c944c7a5303e990884834cc0) fix: prevent http.DefaultClient mutation in search tools - [`c716ec6`](https://github.com/vxcontrol/pentagi/commit/c716ec6a7430fb47df5de3b2bc114afb9676c418) test: remove unused test server and dead imports in search tool tests - [`ddd0b0e`](https://github.com/vxcontrol/pentagi/commit/ddd0b0e16e168abafa48bac04909201ce95f30f8) test: add unit tests for perplexity and google search tools - [`c538b80`](https://github.com/vxcontrol/pentagi/commit/c538b803dcd39c34e37f1b17d783d2767e3f044e) test: address review feedback on search tool tests - [`539f6c2`](https://github.com/vxcontrol/pentagi/commit/539f6c2cda0f08802ef82fc8ac05c2739cf7b41b) feat: add CN LLM providers (DeepSeek, GLM, Kimi, Qwen) with bug fixes - [`8354f7e`](https://github.com/vxcontrol/pentagi/commit/8354f7e3a992a6bde892d791daaf40beb6a5e331) chore(deps): bump rollup - [`f4da45e`](https://github.com/vxcontrol/pentagi/commit/f4da45e63b6f9e7b47994fd90eca58a95201d48c) chore(deps): bump go.opentelemetry.io/otel/sdk - [`1c3f149`](https://github.com/vxcontrol/pentagi/commit/1c3f14971feeda7b65852dbb81fac6a4d1ab9b31) feat: add Novita AI as optional LLM provider ### 📊 Changes **122 files changed** (+19200 additions, -3653 deletions) <details> <summary>View changed files</summary> 📝 `.env.example` (+31 -2) 📝 `.gitignore` (+9 -0) 📝 `.vscode/launch.json` (+12 -0) 📝 `.vscode/settings.json` (+1 -0) ➕ `CLAUDE.md` (+150 -0) 📝 `Dockerfile` (+4 -1) 📝 `README.md` (+848 -151) 📝 `backend/cmd/ctester/main.go` (+50 -3) 📝 `backend/cmd/ftester/main.go` (+1 -1) 📝 `backend/cmd/ftester/worker/executor.go` (+15 -32) 📝 `backend/cmd/installer/wizard/controller/controller.go` (+189 -12) 📝 `backend/cmd/installer/wizard/locale/locale.go` (+231 -66) 📝 `backend/cmd/installer/wizard/models/llm_provider_form.go` (+133 -9) 📝 `backend/cmd/installer/wizard/models/llm_providers.go` (+4 -0) 📝 `backend/cmd/installer/wizard/models/search_engines_form.go` (+50 -0) 📝 `backend/cmd/installer/wizard/models/types.go` (+8 -0) 📝 `backend/cmd/installer/wizard/registry/registry.go` (+4 -0) 📝 `backend/docs/config.md` (+295 -23) 📝 `backend/docs/database.md` (+2 -2) 📝 `backend/docs/flow_execution.md` (+4 -0) _...and 80 more files_ </details> ### 📄 Description ## Description ### Problem PentAGI's custom provider architecture only supported a single OpenAI-compatible endpoint at a time, preventing users from simultaneously configuring multiple cloud services. Users had to choose one custom provider at startup and couldn't switch between different LLM services (DeepSeek, GLM, Kimi, Qwen) through the UI. AWS Bedrock authentication was inflexible, forcing users into a single credential model. The codebase had insufficient test coverage for critical tool components, multiple security vulnerabilities in resource management, and several quality issues affecting production stability. ### Solution This preparation release consolidates 27 commits across 14 merged pull requests, delivering: - **LLM Provider Ecosystem Expansion**: Native first-class support for DeepSeek, GLM (智谱清言), Kimi (月之暗面), and Qwen (通义千问) as individual providers. Users can now configure all four Chinese cloud services simultaneously with dedicated API keys and switch between them instantly through the Settings UI, eliminating the previous single-custom-provider limitation - **Flexible AWS Bedrock Authentication**: Three authentication modes (AWS SDK default chain, bearer token, static credentials) with priority-based selection for diverse deployment scenarios - **Comprehensive Testing Infrastructure**: 15+ new unit test files covering tool registries, agent contexts, custom JSON types, executor logic, and search tool integrations - significantly improving code reliability and maintenance confidence - **Security Hardening**: CA private key cleanup post-signing, resource leak fixes (browser HTTP clients, terminal tar operations), Bedrock toolConfig nil safety - **Code Quality Improvements**: Fixed proxy configuration bugs, eliminated debug output, corrected typos, expanded `.gitignore` patterns Closes #166, #167, #168, #170, #171, #172 ## Type of Change - [x] 🚀 New feature (non-breaking change which adds functionality) - [x] 🐛 Bug fix (non-breaking change which fixes an issue) - [x] 🧪 Test update - [x] 🛡️ Security update - [x] 📚 Documentation update ## Areas Affected - [x] Core Services (Backend API) - [x] AI Agents (Researcher/Developer/Executor) - [x] Security Tools Integration - [x] External Integrations (LLM/Search APIs) - [x] Documentation - [x] Infrastructure/DevOps ## Testing and Verification ### Test Configuration ```yaml PentAGI Version: 1.2.1-dev (post-1.2.0) Docker Version: 24.0+ Host OS: Linux/macOS/Windows LLM Providers: OpenAI, Anthropic, DeepSeek, GLM, Kimi, Qwen Enabled Features: Core + new providers ``` ### Test Steps 1. Run all new unit tests (`go test ./...`) - 100% pass rate 2. Verify DeepSeek/GLM/Kimi/Qwen provider initialization with valid API keys 3. Test AWS Bedrock with all three authentication methods 4. Confirm Google Search proxy configuration applies correctly 5. Test browser tool graceful screenshot failure handling 6. Verify terminal tar operations handle large files without OOM 7. Validate CA key cleanup after certificate signing ### Test Results - ✅ **15+ new test files** with comprehensive coverage: - `args_test.go`: Bool/Int64 custom JSON unmarshalling (20+ test cases) - `context_test.go`: Agent context chaining and isolation - `registry_test.go`: Tool type mapping and completeness validation - `executor_test.go`: Custom executor tool definitions and execution - `terminal_test.go`: Terminal formatting and name generation - `google_test.go`: Google Search service creation and parsing - `perplexity_test.go`: Perplexity API error handling and formatting - ✅ **All existing tests pass** - no regressions introduced - ✅ **Security fixes verified**: CA key removed, resource leaks eliminated - ✅ **Proxy configuration fixed** in Google/Tavily/Traversaal search tools - ✅ **Nil pointer safety** in Bedrock provider toolConfig handling - ✅ **Provider validation** returns clear errors instead of misleading OpenAI fallback messages ## Security Considerations **Fixes Implemented:** 1. **Certificate Authority Key Management** (PR #168) - CA private key (`service_ca.key`) deleted immediately after signing server certificate - Eliminates attack surface if container filesystem is compromised - Attacker can no longer issue arbitrary trusted certificates post-breach 2. **Resource Leak Prevention** (PRs #150, #167, #168) - Browser tool: Screenshot failure no longer discards successfully-fetched page content - Terminal tar: 50MB file size validation prevents memory exhaustion attacks - HTTP clients: Proper client isolation in search tools (no global mutation) 3. **Bedrock Runtime Safety** (PR #166) - Nil check prevents panic when `toolConfig` is undefined - Graceful degradation instead of agent crash 4. **Provider Authentication Security** - DeepSeek/GLM/Kimi/Qwen: Explicit API key validation with clear error messages - Bedrock: Priority-based auth selection with secure credential handling - No credential leakage in logs or error messages **No New Vulnerabilities Introduced:** - All new providers follow existing secure credential management patterns - Test code uses isolated contexts and mock data (no real API keys) - `.gitignore` expansion prevents accidental IDE/OS file commits ## Performance Impact **Improvements:** - **Testing Performance**: `t.Parallel()` in new test files enables concurrent test execution - **HTTP Client Optimization**: Search tools now create dedicated clients instead of mutating global `http.DefaultClient` (eliminates race conditions) - **Browser Tool Efficiency**: Screenshot failures no longer block content retrieval - faster response times when image capture fails **No Degradation:** - Authentication method selection adds <1ms overhead (priority checks are simple conditionals) - Test suite expansion does not affect runtime performance (tests don't run in production) - Langchaingo update (v0.1.14-update.2) maintains compatibility with existing performance characteristics **Resource Usage:** - Chinese providers use same connection pooling as other LLM integrations - Terminal tar size validation prevents unbounded memory allocations ## Documentation Updates - [x] Configuration documentation updates - DeepSeek/GLM/Kimi/Qwen env vars in `.env.example` and `docker-compose.yml` - [x] Example configurations - Added `novita.provider.yml` custom provider example for Novita AI aggregator - [x] GraphQL schema updates - No changes (provider management uses existing schema) - [x] Other: Fixed multiple typos ("bellow" → "below" in terminal output), expanded `.gitignore` with IDE/OS patterns ## Deployment Notes **New Environment Variables (Optional):** ```bash # Chinese Mainland LLM Providers (all optional) DEEPSEEK_API_KEY=sk-... DEEPSEEK_SERVER_URL=https://api.deepseek.com # optional, uses default if not set GLM_API_KEY=... GLM_SERVER_URL=https://open.bigmodel.cn/api/paas # optional KIMI_API_KEY=... KIMI_SERVER_URL=https://api.moonshot.cn # optional QWEN_API_KEY=sk-... QWEN_SERVER_URL=https://dashscope.aliyuncs.com/compatible-mode # optional # AWS Bedrock Enhanced Authentication (optional) BEDROCK_DEFAULT_AUTH=true # Use AWS SDK credential chain (IAM roles, env vars, etc.) BEDROCK_BEARER_TOKEN=... # Alternative: bearer token auth # Existing BEDROCK_AWS_ACCESS_KEY_ID and BEDROCK_AWS_SECRET_ACCESS_KEY still work ``` **Database Migrations:** - `20260301_add_chinese_llm_providers.sql` - Adds `deepseek`, `glm`, `kimi`, `qwen` to `PROVIDER_TYPE` enum - Migrations run automatically on startup **Breaking Changes:** - ❌ **None** - All changes are backward compatible - Existing provider configurations remain functional - New providers are opt-in via environment variables **Compatibility:** - ✅ Fully backward compatible with v1.2.0 - ✅ No API changes to GraphQL or REST endpoints - ✅ Existing deployments work without configuration changes - ✅ langchaingo v0.1.14-update.2 compatible with existing provider implementations **Upgrade Steps:** 1. Pull latest changes 2. (Optional) Add API keys for new providers to `.env` 3. Run `docker compose up -d` - migrations apply automatically 4. New providers appear in Settings UI if API keys are configured ## Checklist ### Code Quality - [x] My code follows the project's coding standards - [x] I have added/updated necessary documentation - [x] I have added tests to cover my changes - [x] All new and existing tests pass - [x] I have run `go fmt` and `go vet` (for Go code) - [x] I have run `npm run lint` (for TypeScript/JavaScript code) ### Security - [x] I have considered security implications - [x] Changes maintain or improve the security model - [x] Sensitive information has been properly handled ### Compatibility - [x] Changes are backward compatible - [x] Breaking changes are clearly marked and documented - [x] Dependencies are properly updated ### Documentation - [x] Documentation is clear and complete - [x] Comments are added for non-obvious code - [x] API changes are documented ## Additional Notes ### Key Changes by Category #### 🚀 New LLM Providers (PR #154) **Chinese Cloud Service Providers** - First-class provider support enabling simultaneous multi-service configuration: - **DeepSeek**: Cost-effective reasoning models with competitive performance - **GLM (智谱清言)**: Zhipu AI's general-purpose language models - **Kimi (月之暗面)**: Moonshot AI's long-context models - **Qwen (通义千问)**: Alibaba Cloud's Qianwen family **Key Improvement**: Previously, users could only configure ONE custom OpenAI-compatible provider at a time, requiring application restart to switch between Chinese cloud services. Now all four providers can be enabled simultaneously with individual API keys, and users can instantly switch between them through the Settings UI. **Implementation Highlights:** - Each provider has dedicated backend implementation with independent configuration - All providers follow OpenAI-compatible API pattern for easy integration - Explicit API key validation with clear error messages (no misleading "missing OpenAI API key" fallback) - Proper whitelisting in `ProviderType.Valid()` to prevent 422 Unprocessable Entity errors - Frontend GraphQL queries updated to load default agent configurations for each provider - Docker Compose env var declarations ensure keys are injected into containers **Custom Provider Example** (PR #162) - Added `novita.provider.yml` configuration example: - Demonstrates custom provider setup for Novita AI aggregator service - Provides access to DeepSeek V3.2, Minimax M2.5, GLM-5 through single API endpoint - Test report included (`novita-report.md`) with performance validation - Shows model configuration pattern using `moonshotai/kimi-k2.5` Contributor: @niuqun2003 (DeepSeek/GLM/Kimi/Qwen native providers), @Alex-wuhu (Novita custom provider example) #### 🔐 AWS Bedrock Authentication Enhancement **Multiple Authentication Methods** - Flexible credential management for diverse deployment scenarios: 1. **AWS SDK Default Chain** (`BEDROCK_DEFAULT_AUTH=true`) - IAM instance profiles (EC2, ECS, Lambda) - Environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`) - Shared credentials file (`~/.aws/credentials`) - SSO/SAML federation 2. **Bearer Token** (`BEDROCK_BEARER_TOKEN=...`) - Pre-signed AWS Signature v4 tokens - Temporary credentials from STS 3. **Static Credentials** (existing) - `BEDROCK_AWS_ACCESS_KEY_ID` + `BEDROCK_AWS_SECRET_ACCESS_KEY` **Priority Order**: Default Chain → Bearer Token → Static Credentials **Benefits**: - Container orchestration platforms can use instance profiles (no hardcoded keys) - CI/CD pipelines can inject temporary STS tokens - Development environments retain simple static key configuration Contributor: @asdek #### 🧪 Testing Infrastructure Expansion **15+ New Unit Test Files** - Comprehensive coverage for previously untested components: **Tool Registry & Context** (PR #171): - `context_test.go`: Agent context chaining, parent/current promotion, isolation verification - `registry_test.go`: Tool type mapping completeness, enum string representation, mutation safety **Custom JSON Types** (PR #170): - `args_test.go`: Bool/Int64 unmarshalling from bare values, quoted strings, single quotes, whitespace trimming, overflow/underflow boundaries, nil safety **Executor & Terminal** (PR #172): - `executor_test.go`: Custom executor tool definitions, execution behavior, unknown tool handling - `terminal_test.go`: Terminal name generation, ANSI formatting, input/output colorization **Search Tools** (PR #153): - `google_test.go`: Service creation, proxy configuration, result parsing, special character handling - `perplexity_test.go`: API key validation, error response handling (10 HTTP codes), citation formatting **Test Quality Features**: - `t.Parallel()` for concurrent execution (faster CI/CD) - `context.Background()` / `t.Context()` consistency across test files - Comprehensive edge case coverage (null, overflow, whitespace, unicode) - Mock providers for screenshot/search logs (test isolation) Contributors: @mason5052 (primary author of all test infrastructure) #### 🛡️ Security Fixes **Certificate Authority Key Cleanup** (PR #168): - Problem: CA private key left on disk after signing server certificate - Risk: Compromised container filesystem allows attacker to issue arbitrary trusted certificates - Fix: Delete `service_ca.key`, `service.csr`, `service_ca.srl` immediately after signing - Impact: Reduces attack surface in multi-tenant deployments **Resource Leak Prevention**: 1. **Browser Screenshot Failure** (PR #150) - Old behavior: Screenshot error discards successfully-fetched HTML/Markdown content - New behavior: Log warning, return content with empty screenshot name - Benefit: More reliable page content extraction, faster agent progress 2. **Terminal Tar Operations** (PR #168) - Old behavior: No file size limit when creating tar archives - New behavior: 50MB validation before `io.Copy` (prevents OOM attacks) - Benefit: Protects against memory exhaustion from malicious large files 3. **HTTP Client Mutation** (PR #151) - Old behavior: `http.DefaultClient.Transport` mutated when setting proxy in Tavily/Traversaal - New behavior: Create dedicated `http.Client` instance for proxied requests - Benefit: Eliminates race conditions in concurrent search operations **Bedrock Runtime Safety** (PR #166): - Nil check on `toolConfig` before dereferencing in langchaingo integration - Prevents panic when Bedrock responses omit tool configuration Contributors: @mason5052 (resource leaks, HTTP client), @Priyanka-2725 (Bedrock nil safety) #### 🐛 Bug Fixes **Proxy Configuration** (PR #167): - Google Search: `newSearchService()` ignored constructed `opts` slice (proxy had no effect) - Fix: Pass `opts...` to `NewService()` call instead of hardcoded `option.WithAPIKey()` **Terminal Output** (PR #164): - Typo: "keeps bellow" → "shown below" in file content header **Container Lookup Error** (PR #152): - `GetTool` silently ignored `GetFlowPrimaryContainer` errors - Result: Cryptic "container is not running" instead of clear "missing container" message - Fix: Propagate error immediately with flow ID context **VS Code Settings** (PR #163): - Expanded `.gitignore` with IDE patterns (`.vscode/settings.json`, `.idea/`, `*.swp`) and OS files (`Thumbs.db`) **Frontend Debug Cleanup** (PR #141): - Removed 7 leftover `console.log()` statements from production code Contributors: @mason5052 (proxy, container, frontend cleanup), @haosenwang1018 (gitignore) #### 📦 Dependency Updates **OpenTelemetry SDK** (PR #161): - `go.opentelemetry.io/otel/sdk`: 1.36.0 → 1.40.0 - 4 minor version bump with performance improvements and bug fixes **Rollup** (PR #155): - `rollup`: 4.53.1 → 4.59.0 (frontend build tool security and performance updates) **langchaingo** (commit 274ec52): - Updated to stable release `v0.1.14-update.2` Contributors: @dependabot[bot] (automated dependency PRs), @asdek (langchaingo) ### Contributors This preparation release includes contributions from: - **@asdek** (Dmitry Ng) - Project maintainer, Bedrock auth enhancement, langchaingo update, test coordination - **@niuqun2003** - DeepSeek/GLM/Kimi/Qwen provider implementation and integration - **@mason5052** - Testing infrastructure (15+ test files), security fixes, proxy bugs, code cleanup - **@Alex-wuhu** - Novita AI provider integration and model configuration - **@Priyanka-2725** (Priyanka Singh) - Bedrock toolConfig nil safety fix - **@haosenwang1018** - .gitignore expansion with IDE/OS patterns - **@dependabot[bot]** - Automated dependency security updates Special thanks to the community for improving code quality, test coverage, and deployment flexibility! 🎉 ## Merged Pull Requests - #154 - feat: add Chinese mainland LLM providers (DeepSeek, GLM, Kimi, Qwen) - #162 - feat: add Novita AI as optional LLM provider - #166 - fix(bedrock): prevent runtime failure when toolConfig is undefined - #167 - fix: apply constructed opts to Google search service creation - #168 - fix: remove CA private key after certificate signing in entrypoint - #170 - test: add unit tests for Bool and Int64 custom JSON types - #171 - test: add unit tests for agent context and tool registry - #172 - test: add unit tests for executor helpers and terminal utilities - #150 - fix: treat screenshot failure as non-critical in browser tool - #151 - fix: prevent http.DefaultClient mutation in search tools - #152 - fix: propagate container lookup error in GetTool - #153 - test: add unit tests for perplexity and google search tools - #161 - chore(deps): bump go.opentelemetry.io/otel/sdk from 1.36.0 to 1.40.0 - #155 - chore(deps): bump rollup from 4.53.1 to 4.59.0 - #163 - chore: expand .gitignore with IDE and OS patterns - #164 - fix: correct typo in terminal ReadFile output message - #141 - chore: remove debug console.log statements from frontend ## Migration Path For existing deployments upgrading from v1.2.0: 1. **Pull latest changes** ```bash docker compose pull ``` 2. **Review new environment variables** (all optional): - Add Chinese provider API keys (DeepSeek/GLM/Kimi/Qwen) for multi-service access - Configure Bedrock auth method if using AWS infrastructure 3. **Apply changes**: ```bash docker compose up -d ``` - Database migrations run automatically - New providers appear in Settings UI when API keys are configured 3. **Test** (if using new providers): - Create test flow - Select DeepSeek/GLM/Kimi/Qwen model - Verify agent initialization and response generation **No downtime required** - Migrations are additive only (new enum values). ## Future Work This preparation release establishes infrastructure for upcoming features: - **Extended Provider Testing**: Comprehensive performance benchmarks for Chinese providers - **Multi-Region Deployment Guide**: Documentation for geo-distributed PentAGI instances - **Provider Failover**: Automatic fallback between providers for resilience - **Cost Optimization**: Per-provider pricing configuration and usage analytics - **Enhanced Testing Dashboard**: Visual test coverage reports and CI/CD integration  --- > [!NOTE] > **Medium Risk** > Medium risk because it expands provider/auth configuration paths (Bedrock auth modes, new providers, installer wiring) and refactors tool construction in `ftester`/search tools, which could impact runtime behavior if misconfigured. > > **Overview** > **Expands LLM provider support and configuration** by adding env/config + installer UI wiring for `deepseek`, `glm`, `kimi`, and `qwen`, plus new Bedrock auth options (`BEDROCK_DEFAULT_AUTH`, `BEDROCK_BEARER_TOKEN`) and an Ollama Cloud API key (`OLLAMA_SERVER_API_KEY`). `ctester`/`ftester` are updated to recognize the new providers and enforce Bedrock auth presence. > > **Refactors tool initialization in function testing**: `ftester`’s `GetTool` now only fetches the primary container when required (terminal/file) and fails fast on missing container, and search tools are constructed from the shared `config.Config` (enabling new DuckDuckGo params and `SEARXNG_TIMEOUT`). > > **Packaging/docs/dev quality updates**: `.env.example`, `README.md`, and installer locale text are updated for the new providers and settings; Docker image now normalizes `entrypoint.sh` line endings and includes additional example provider configs; `.gitignore` and VS Code settings/launch configs are refreshed, and a new `CLAUDE.md` repo guidance file is added. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit da51f179734594fe2a2681d7133715a74b1d34c8. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup>  --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

yindo added the pull-request label 2026-06-06 22:09:45 -04:00

yindo closed this issue

2026-06-06 22:09:46 -04:00

yindo referenced this issue

2026-06-06 22:09:51 -04:00

[PR #221] [CLOSED] Add configuration for self-evolving agent system #243

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: vxcontrol/pentagi#221