Compare commits

...
8 Commits
Author SHA1 Message Date
Mervin Praison b01eb2b8fa Release v2.3.44 2025-12-19 10:50:33 +00:00
Mervin Praison e609fb7425 Release v2.3.43 2025-12-19 10:32:33 +00:00
Mervin Praison 168ef98473 feat: Implement all remaining AutoAgents improvements (TDD)
TODO 2: Single-Agent Option
- Add SingleAgentStructure Pydantic model
- Add single_agent parameter to AutoGenerator and WorkflowAutoGenerator

TODO 3: LLM-Based Pattern Recommendation
- Add PatternRecommendation Pydantic model (pattern, reasoning, confidence)
- Add recommend_pattern_llm() method to WorkflowAutoGenerator

TODO 4: Validation Gates
- Add ValidationGate Pydantic model (criteria, pass_action, fail_action)
- Add optional gates field to WorkflowStructure

TODO 6: Pattern Support for AutoGenerator
- Add pattern parameter to AutoGenerator
- Add recommend_pattern() method to AutoGenerator
- Add pattern-specific guidance in get_user_content()

TODO 7: Merge Support for WorkflowAutoGenerator
- Add merge parameter to generate() method
- Add merge_with_existing_workflow() method

TODO 10: Framework Support for WorkflowAutoGenerator
- Add framework parameter to WorkflowAutoGenerator
- Default framework is 'praisonai'

Test Coverage:
- 51 tests pass (up from 34)
- 17 new tests added using TDD approach
- All tests verify new functionality
2025-12-19 10:29:40 +00:00
Mervin Praison ce2c2926f9 refactor: Add BaseAutoGenerator class and improve WorkflowAutoGenerator
- Create BaseAutoGenerator with shared functionality (DRY principle)
  - Lazy-loaded instructor client (LiteLLM or OpenAI fallback)
  - Environment variable handling for model/API configuration
  - analyze_complexity() static method for task complexity detection
  - get_available_tools() static method for tool list
  - AVAILABLE_TOOLS constant shared between generators

- Update AutoGenerator to inherit from BaseAutoGenerator
  - Removes ~40 lines of duplicate code
  - Maintains all existing functionality

- Update WorkflowAutoGenerator to inherit from BaseAutoGenerator
  - Removes ~40 lines of duplicate code
  - Add dynamic agent count based on task complexity
  - Add tools list to workflow prompts
  - Improved prompt with 3-step analysis

- Add 8 new unit tests for new functionality
  - TestBaseAutoGenerator: complexity analysis, tools list
  - TestWorkflowDynamicAgentCount: prompt improvements

All 34 tests pass with no performance impact.
2025-12-19 10:23:29 +00:00
Mervin Praison 0a569e2b46 docs: Add AutoAgents workflow patterns examples and update README
- Add autoagents_workflow_patterns.py with all 6 workflow patterns
- Update auto_agents_example.py with new features and CLI examples
- Add orchestrator-workers and evaluator-optimizer patterns to README CLI section
- Update pattern options table with all 6 patterns
- Add documentation links to examples
2025-12-19 09:54:05 +00:00
Mervin Praison 054aea8f66 Add MCP Registry badge and ignore MCP tokens 2025-12-18 22:18:33 +00:00
Mervin Praison 34f1293cfc Release v2.3.42 2025-12-18 22:13:08 +00:00
Mervin Praison 7c1e32afc0 upate 2025-12-18 22:11:02 +00:00
36 changed files with 5007 additions and 123 deletions

No files matched your search

+4 -1
View File
@@ -77,4 +77,7 @@ agents/praisonaiagents/praisonaiagents.egg-info
.qodo
CopilotKit*
.gemini
issue_details.json
issue_details.json
# MCP Registry tokens
.mcpregistry_*
+14 -1
View File
@@ -6,10 +6,13 @@
</picture>
</p>
<!-- mcp-name: io.github.MervinPraison/praisonai -->
<p align="center">
<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://static.pepy.tech/badge/PraisonAI" alt="Total Downloads" /></a>
<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://img.shields.io/github/v/release/MervinPraison/PraisonAI" alt="Latest Stable Version" /></a>
<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License" /></a>
<a href="https://registry.modelcontextprotocol.io/servers/io.github.MervinPraison/praisonai"><img src="https://img.shields.io/badge/MCP-Registry-blue" alt="MCP Registry" /></a>
</p>
<div align="center">
@@ -207,6 +210,7 @@ npm install praisonai
| 🧠 Claude Memory Tool | [Example](#claude-memory-tool-cli) | [📖](https://docs.praison.ai/features/claude-memory-tool) |
| 💾 File-Based Memory | [Example](examples/python/general/memory_example.py) | [📖](https://docs.praison.ai/concepts/memory) |
| 🔍 Built-in Search Tools | [Example](examples/python/agents/websearch-agent.py) | [📖](https://docs.praison.ai/tools/tavily) |
| 🔎 Unified Web Search | [Example](src/praisonai-agents/examples/web_search_example.py) | [📖](https://docs.praison.ai/tools/web-search) |
| 📋 Planning Mode | [Example](examples/python/agents/planning-agent.py) | [📖](https://docs.praison.ai/features/planning-mode) |
| 🔧 Planning Tools | [Example](#3-agent-with-planning-mode) | [📖](https://docs.praison.ai/features/planning-mode) |
| 🧠 Planning Reasoning | [Example](#3-agent-with-planning-mode) | [📖](https://docs.praison.ai/features/planning-mode) |
@@ -1120,6 +1124,15 @@ praisonai workflow auto "Research AI trends" --pattern parallel
# Generate routing workflow (classifier routes to specialists)
praisonai workflow auto "Build a chatbot" --pattern routing
# Generate orchestrator-workers workflow (central orchestrator delegates)
praisonai workflow auto "Comprehensive market analysis" --pattern orchestrator-workers
# Generate evaluator-optimizer workflow (iterative refinement)
praisonai workflow auto "Write and refine article" --pattern evaluator-optimizer
# Specify output file
praisonai workflow auto "Build a chatbot" --pattern routing
# Specify output file
praisonai workflow auto "Research AI" --pattern sequential --output my_workflow.yaml
```
@@ -1129,7 +1142,7 @@ praisonai workflow auto "Research AI" --pattern sequential --output my_workflow.
|------|-------------|
| `--workflow-var key=value` | Set workflow variable (can be repeated) |
| `--var key=value` | Set variable for YAML workflows |
| `--pattern <pattern>` | Pattern for auto-generation (sequential, parallel, routing, loop) |
| `--pattern <pattern>` | Pattern for auto-generation (sequential, parallel, routing, loop, orchestrator-workers, evaluator-optimizer) |
| `--output <file>` | Output file for auto-generation |
| `--llm <model>` | LLM model (e.g., openai/gpt-4o-mini) |
| `--tools <tools>` | Tools (comma-separated, e.g., tavily) |
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.41" \
"praisonai>=2.3.44" \
"praisonai[chat]" \
"embedchain[github,youtube]"
+1 -1
View File
@@ -20,7 +20,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.41" \
"praisonai>=2.3.44" \
"praisonai[ui]" \
"praisonai[chat]" \
"praisonai[realtime]" \
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.41" \
"praisonai>=2.3.44" \
"praisonai[ui]" \
"praisonai[crewai]"
@@ -0,0 +1,132 @@
"""
AutoAgents Workflow Patterns Example
This example demonstrates the new workflow auto-generation features:
- Dynamic agent count based on task complexity
- Multiple workflow patterns (sequential, parallel, routing, orchestrator-workers, evaluator-optimizer)
- Pattern recommendation based on task keywords
Documentation: https://docs.praison.ai/features/autoagents
CLI Reference: https://docs.praison.ai/nocode/auto
"""
from praisonai.auto import AutoGenerator, WorkflowAutoGenerator
import os
import tempfile
# Ensure API key is set
if not os.environ.get("OPENAI_API_KEY"):
print("Please set OPENAI_API_KEY environment variable")
exit(1)
print("=" * 60)
print("AutoAgents Workflow Patterns Demo")
print("=" * 60)
# =============================================================================
# Example 1: Dynamic Agent Count (Simple vs Complex Tasks)
# =============================================================================
print("\n📊 Example 1: Dynamic Agent Count")
print("-" * 40)
# Simple task - should create 1-2 agents
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
simple_file = f.name
simple_generator = AutoGenerator(
topic="Write a haiku about spring",
agent_file=simple_file,
framework="praisonai"
)
print("Simple task: 'Write a haiku about spring'")
result = simple_generator.generate()
print(f"Generated: {result}")
# Complex task - should create 3-4 agents
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
complex_file = f.name
complex_generator = AutoGenerator(
topic="Research AI trends, analyze market data, write a comprehensive report with executive summary",
agent_file=complex_file,
framework="praisonai"
)
print("\nComplex task: 'Research AI trends, analyze market data...'")
result = complex_generator.generate()
print(f"Generated: {result}")
# =============================================================================
# Example 2: Pattern Recommendation
# =============================================================================
print("\n🎯 Example 2: Pattern Recommendation")
print("-" * 40)
test_tasks = [
("Write a blog post", "sequential"),
("Research from multiple sources simultaneously", "parallel"),
("Classify and route customer requests", "routing"),
("Comprehensive analysis and break down the problem", "orchestrator-workers"),
("Refine and improve content quality", "evaluator-optimizer"),
]
for task, expected in test_tasks:
generator = WorkflowAutoGenerator(topic=task)
recommended = generator.recommend_pattern()
status = "✅" if recommended == expected else "⚠️"
print(f"{status} Task: '{task[:40]}...'")
print(f" Recommended: {recommended}")
# =============================================================================
# Example 3: Generate Workflow with Specific Pattern
# =============================================================================
print("\n🔄 Example 3: Generate Workflows with Patterns")
print("-" * 40)
patterns = ["sequential", "parallel", "orchestrator-workers", "evaluator-optimizer"]
for pattern in patterns:
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
workflow_file = f.name
generator = WorkflowAutoGenerator(
topic="Create a market analysis report",
workflow_file=workflow_file
)
print(f"\n📋 Pattern: {pattern}")
result = generator.generate(pattern=pattern)
print(f" Generated: {result}")
# Read and show structure
import yaml
with open(result, 'r') as f:
data = yaml.safe_load(f)
print(f" Agents: {list(data.get('agents', {}).keys())}")
print(f" Steps: {len(data.get('steps', []))}")
# =============================================================================
# Example 4: CLI Usage (for reference)
# =============================================================================
print("\n💻 Example 4: CLI Commands (for reference)")
print("-" * 40)
print("""
# Auto-generate agents (dynamic count)
praisonai --auto "Write a haiku about spring"
# Auto-generate workflow with pattern
praisonai workflow auto "Research and write a report" --pattern sequential
praisonai workflow auto "Research from multiple sources" --pattern parallel
praisonai workflow auto "Comprehensive market analysis" --pattern orchestrator-workers
praisonai workflow auto "Refine content quality" --pattern evaluator-optimizer
# With output file
praisonai workflow auto "Customer support routing" --pattern routing --output support.yaml
""")
print("\n" + "=" * 60)
print("✅ AutoAgents Workflow Patterns Demo Complete!")
print("=" * 60)
print("\nDocumentation: https://docs.praison.ai/features/autoagents")
print("CLI Reference: https://docs.praison.ai/nocode/auto")
+40 -3
View File
@@ -1,11 +1,48 @@
"""
AutoAgents Example
AutoAgents automatically creates and manages AI agents based on high-level instructions.
It features dynamic agent count based on task complexity.
Documentation: https://docs.praison.ai/features/autoagents
CLI Reference: https://docs.praison.ai/nocode/auto
New Features:
- Dynamic agent count (1-4 based on task complexity)
- Workflow patterns: sequential, parallel, routing, orchestrator-workers, evaluator-optimizer
- Pattern recommendation based on task keywords
- Tool preservation from LLM suggestions
"""
from praisonaiagents import AutoAgents
from praisonaiagents.tools import duckduckgo
# Basic usage - AutoAgents analyzes complexity and creates optimal agents
agents = AutoAgents(
instructions="Search for information about AI Agents",
tools=[duckduckgo],
process="sequential",
verbose=True
process="sequential", # or "hierarchical"
verbose=True,
max_agents=3 # Maximum number of agents to create
)
agents.start()
result = agents.start()
print(result)
# =============================================================================
# CLI Usage Examples (for reference)
# =============================================================================
#
# # Auto-generate agents (dynamic count based on complexity)
# praisonai --auto "Write a haiku about spring"
# praisonai --auto "Research AI trends, analyze data, write report"
#
# # Auto-generate workflow with specific pattern
# praisonai workflow auto "Research and write" --pattern sequential
# praisonai workflow auto "Research from multiple sources" --pattern parallel
# praisonai workflow auto "Comprehensive analysis" --pattern orchestrator-workers
# praisonai workflow auto "Refine content quality" --pattern evaluator-optimizer
#
# # With framework selection
# praisonai --framework crewai --auto "Create a movie script"
# praisonai --framework autogen --auto "Create a movie script"
@@ -0,0 +1,90 @@
"""
Unified Web Search Example
This example demonstrates the search_web tool that automatically tries
multiple search providers with fallback support.
Search Provider Priority:
1. Tavily (requires TAVILY_API_KEY + tavily-python)
2. Exa (requires EXA_API_KEY + exa_py)
3. You.com (requires YDC_API_KEY + youdotcom)
4. DuckDuckGo (requires duckduckgo_search package, no API key)
5. SearxNG (requires requests + running SearxNG instance)
Installation:
pip install praisonaiagents duckduckgo_search
Usage:
python web_search_example.py
"""
from praisonaiagents import Agent
from praisonaiagents.tools import search_web, get_available_providers
def main():
print("=" * 60)
print("Unified Web Search - Provider Status")
print("=" * 60)
# Check which providers are available
providers = get_available_providers()
for p in providers:
status = "✓ Available" if p["available"] else f"✗ {p['reason']}"
print(f" {p['name']:12} {status}")
print("\n" + "=" * 60)
print("Example 1: Basic Search")
print("=" * 60)
# Simple search - automatically uses best available provider
results = search_web("Python programming best practices", max_results=3)
if results and "error" not in results[0]:
print(f"\nFound {len(results)} results using {results[0].get('provider', 'unknown')}:\n")
for r in results:
print(f" Title: {r.get('title', 'N/A')[:60]}...")
print(f" URL: {r.get('url', 'N/A')}")
print()
else:
print(f"\nSearch failed: {results}")
print("=" * 60)
print("Example 2: Specify Providers")
print("=" * 60)
# Only try specific providers
results = search_web(
"machine learning tutorials",
max_results=3,
providers=["duckduckgo", "tavily"] # Try these in order
)
if results and "error" not in results[0]:
print(f"\nFound {len(results)} results using {results[0].get('provider', 'unknown')}:\n")
for r in results:
print(f" Title: {r.get('title', 'N/A')[:60]}...")
print(f" URL: {r.get('url', 'N/A')}")
print()
else:
print(f"\nSearch failed: {results}")
print("=" * 60)
print("Example 3: With PraisonAI Agent")
print("=" * 60)
# Use search_web as an agent tool
agent = Agent(
name="SearchAgent",
role="Web Researcher",
goal="Find and summarize information from the web",
instructions="Search the web and provide concise summaries of findings.",
tools=[search_web]
)
result = agent.start("What are the top 3 AI trends in 2025?")
print(f"\nAgent Response:\n{result}")
if __name__ == "__main__":
main()
@@ -218,6 +218,10 @@ TOOL_MAPPINGS = {
'crawl4ai_extract_sync': ('.crawl4ai_tools', None),
'Crawl4AITools': ('.crawl4ai_tools', 'Crawl4AITools'),
'crawl4ai_tools': ('.crawl4ai_tools', None),
# Unified Web Search (auto-fallback across providers)
'search_web': ('.web_search', None),
'get_available_providers': ('.web_search', None),
}
_instances = {} # Cache for class instances
@@ -254,7 +258,8 @@ def __getattr__(name: str) -> Any:
'exa', 'exa_search', 'exa_search_contents', 'exa_find_similar', 'exa_answer',
'exa_search_async', 'exa_search_contents_async', 'exa_answer_async',
'crawl4ai', 'crawl4ai_many', 'crawl4ai_extract', 'crawl4ai_llm_extract',
'crawl4ai_sync', 'crawl4ai_extract_sync'
'crawl4ai_sync', 'crawl4ai_extract_sync',
'search_web', 'get_available_providers'
]:
return getattr(module, name)
if name in ['file_tools', 'pandas_tools', 'wikipedia_tools',
@@ -0,0 +1,329 @@
"""Unified web search tool with automatic fallback across multiple providers.
This module provides a single `search_web` function that automatically tries
multiple search providers in order, falling back to the next if one fails.
Search Provider Priority:
1. Tavily (requires TAVILY_API_KEY + tavily-python)
2. Exa (requires EXA_API_KEY + exa_py)
3. You.com (requires YDC_API_KEY + youdotcom)
4. DuckDuckGo (requires duckduckgo_search package, no API key)
5. SearxNG (requires requests + running SearxNG instance)
Usage:
from praisonaiagents.tools import search_web
# Simple search - automatically uses best available provider
results = search_web("AI news 2024")
# With max results
results = search_web("Python tutorials", max_results=10)
"""
from typing import List, Dict, Any, Optional
import logging
import os
from importlib import util
logger = logging.getLogger(__name__)
def _check_tavily() -> tuple[bool, Optional[str]]:
"""Check if Tavily is available."""
if util.find_spec("tavily") is None:
return False, "tavily-python package not installed"
if not os.environ.get("TAVILY_API_KEY"):
return False, "TAVILY_API_KEY not set"
return True, None
def _check_exa() -> tuple[bool, Optional[str]]:
"""Check if Exa is available."""
if util.find_spec("exa_py") is None:
return False, "exa_py package not installed"
if not os.environ.get("EXA_API_KEY"):
return False, "EXA_API_KEY not set"
return True, None
def _check_youdotcom() -> tuple[bool, Optional[str]]:
"""Check if You.com is available."""
if util.find_spec("youdotcom") is None:
return False, "youdotcom package not installed"
if not os.environ.get("YDC_API_KEY"):
return False, "YDC_API_KEY not set"
return True, None
def _check_duckduckgo() -> tuple[bool, Optional[str]]:
"""Check if DuckDuckGo is available."""
if util.find_spec("duckduckgo_search") is None:
return False, "duckduckgo_search package not installed"
return True, None
def _check_searxng() -> tuple[bool, Optional[str]]:
"""Check if SearxNG is available."""
if util.find_spec("requests") is None:
return False, "requests package not installed"
return True, None
def _search_tavily(query: str, max_results: int = 5) -> List[Dict[str, Any]]:
"""Search using Tavily."""
from tavily import TavilyClient
client = TavilyClient(api_key=os.environ.get("TAVILY_API_KEY"))
response = client.search(query=query, max_results=max_results)
results = []
for r in response.get("results", []):
results.append({
"title": r.get("title", ""),
"url": r.get("url", ""),
"snippet": r.get("content", ""),
"provider": "tavily"
})
return results
def _search_exa(query: str, max_results: int = 5) -> List[Dict[str, Any]]:
"""Search using Exa."""
from exa_py import Exa
client = Exa(os.environ.get("EXA_API_KEY"))
response = client.search(query=query, num_results=max_results)
results = []
for r in response.results:
results.append({
"title": getattr(r, "title", "") or "",
"url": r.url,
"snippet": getattr(r, "text", "") or "",
"provider": "exa"
})
return results
def _search_youdotcom(query: str, max_results: int = 5) -> List[Dict[str, Any]]:
"""Search using You.com."""
from youdotcom import You
client = You(api_key_auth=os.environ.get("YDC_API_KEY"))
response = client.search.unified(query=query, count=max_results)
results = []
# Handle response object
if hasattr(response, 'results'):
web_results = getattr(response.results, 'web', []) or []
for r in web_results[:max_results]:
results.append({
"title": getattr(r, "title", "") or "",
"url": getattr(r, "url", "") or "",
"snippet": getattr(r, "description", "") or "",
"provider": "youdotcom"
})
elif isinstance(response, dict):
web_results = response.get("results", {}).get("web", [])
for r in web_results[:max_results]:
results.append({
"title": r.get("title", ""),
"url": r.get("url", ""),
"snippet": r.get("description", ""),
"provider": "youdotcom"
})
return results
def _search_duckduckgo(query: str, max_results: int = 5) -> List[Dict[str, Any]]:
"""Search using DuckDuckGo."""
from duckduckgo_search import DDGS
results = []
ddgs = DDGS()
for result in ddgs.text(keywords=query, max_results=max_results):
results.append({
"title": result.get("title", ""),
"url": result.get("href", ""),
"snippet": result.get("body", ""),
"provider": "duckduckgo"
})
return results
def _search_searxng(query: str, max_results: int = 5, searxng_url: Optional[str] = None) -> List[Dict[str, Any]]:
"""Search using SearxNG."""
import requests
url = searxng_url or os.environ.get("SEARXNG_URL", "http://localhost:32768/search")
params = {
'q': query,
'format': 'json',
'engines': 'google,bing,duckduckgo',
'safesearch': '1'
}
response = requests.get(url, params=params, timeout=10)
response.raise_for_status()
raw_results = response.json().get('results', [])
results = []
for result in raw_results[:max_results]:
results.append({
"title": result.get("title", ""),
"url": result.get("url", ""),
"snippet": result.get("content", ""),
"provider": "searxng"
})
return results
# Provider configuration: (name, check_func, search_func)
SEARCH_PROVIDERS = [
("tavily", _check_tavily, _search_tavily),
("exa", _check_exa, _search_exa),
("youdotcom", _check_youdotcom, _search_youdotcom),
("duckduckgo", _check_duckduckgo, _search_duckduckgo),
("searxng", _check_searxng, _search_searxng),
]
def search_web(
query: str,
max_results: int = 5,
providers: Optional[List[str]] = None,
searxng_url: Optional[str] = None,
) -> List[Dict[str, Any]]:
"""Search the web using multiple providers with automatic fallback.
Tries each search provider in order until one succeeds. For each provider,
it first checks if the API key is set (if required) and if the package is
installed, then attempts the search. If the search fails, it moves to the
next provider.
Args:
query: The search query string
max_results: Maximum number of results to return (default: 5)
providers: Optional list of provider names to try, in order.
If not specified, uses default order:
["tavily", "exa", "youdotcom", "duckduckgo", "searxng"]
searxng_url: Optional custom SearxNG instance URL
Returns:
List of search results, each containing:
- title: Result title
- url: Result URL
- snippet: Result description/snippet
- provider: Name of the provider that returned the result
If all providers fail, returns a list with a single error dict.
Example:
# Use default provider order
results = search_web("AI news 2024")
# Specify providers to try
results = search_web("Python tutorials", providers=["duckduckgo", "tavily"])
# With custom SearxNG URL
results = search_web("tech news", searxng_url="http://my-searxng:8080/search")
"""
errors = []
# Determine which providers to try
if providers:
# Filter to only valid provider names
provider_order = []
for name in providers:
for pname, check_fn, search_fn in SEARCH_PROVIDERS:
if pname == name.lower():
provider_order.append((pname, check_fn, search_fn))
break
else:
provider_order = SEARCH_PROVIDERS
for provider_name, check_func, search_func in provider_order:
# Step 1: Check if provider is available (API key + package)
is_available, check_error = check_func()
if not is_available:
logger.debug(f"Skipping {provider_name}: {check_error}")
errors.append(f"{provider_name}: {check_error}")
continue
# Step 2: Try to search
try:
logger.debug(f"Trying {provider_name} for query: {query}")
if provider_name == "searxng":
results = search_func(query, max_results, searxng_url)
else:
results = search_func(query, max_results)
if results:
logger.info(f"search_web: Successfully used {provider_name}")
return results
else:
errors.append(f"{provider_name}: No results returned")
except Exception as e:
error_msg = f"{provider_name}: {str(e)}"
logger.debug(f"Search failed with {provider_name}: {e}")
errors.append(error_msg)
continue
# All providers failed
error_summary = "; ".join(errors) if errors else "No search providers available"
logger.error(f"search_web: All providers failed - {error_summary}")
return [{"error": f"All search providers failed: {error_summary}"}]
def get_available_providers() -> List[Dict[str, Any]]:
"""Get list of available search providers and their status.
Returns:
List of dicts with provider info:
- name: Provider name
- available: Whether the provider is ready to use
- reason: Reason if not available
Example:
providers = get_available_providers()
for p in providers:
print(f"{p['name']}: {'✓' if p['available'] else '✗'} {p.get('reason', '')}")
"""
result = []
for name, check_func, _ in SEARCH_PROVIDERS:
is_available, error = check_func()
result.append({
"name": name,
"available": is_available,
"reason": error if not is_available else None
})
return result
if __name__ == "__main__":
# Example usage and provider status check
print("=" * 60)
print("Web Search Tool - Provider Status")
print("=" * 60)
providers = get_available_providers()
for p in providers:
status = "✓ Available" if p["available"] else f"✗ {p['reason']}"
print(f" {p['name']:12} {status}")
print("\n" + "=" * 60)
print("Testing search_web...")
print("=" * 60)
results = search_web("Python programming", max_results=3)
if results and "error" not in results[0]:
print(f"\nFound {len(results)} results using {results[0].get('provider', 'unknown')}:")
for r in results:
print(f"\n Title: {r.get('title', 'N/A')[:50]}...")
print(f" URL: {r.get('url', 'N/A')}")
else:
print(f"\nSearch failed: {results}")
+14 -1
View File
@@ -6,10 +6,13 @@
</picture>
</p>
<!-- mcp-name: io.github.MervinPraison/praisonai -->
<p align="center">
<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://static.pepy.tech/badge/PraisonAI" alt="Total Downloads" /></a>
<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://img.shields.io/github/v/release/MervinPraison/PraisonAI" alt="Latest Stable Version" /></a>
<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License" /></a>
<a href="https://registry.modelcontextprotocol.io/servers/io.github.MervinPraison/praisonai"><img src="https://img.shields.io/badge/MCP-Registry-blue" alt="MCP Registry" /></a>
</p>
<div align="center">
@@ -207,6 +210,7 @@ npm install praisonai
| 🧠 Claude Memory Tool | [Example](#claude-memory-tool-cli) | [📖](https://docs.praison.ai/features/claude-memory-tool) |
| 💾 File-Based Memory | [Example](examples/python/general/memory_example.py) | [📖](https://docs.praison.ai/concepts/memory) |
| 🔍 Built-in Search Tools | [Example](examples/python/agents/websearch-agent.py) | [📖](https://docs.praison.ai/tools/tavily) |
| 🔎 Unified Web Search | [Example](src/praisonai-agents/examples/web_search_example.py) | [📖](https://docs.praison.ai/tools/web-search) |
| 📋 Planning Mode | [Example](examples/python/agents/planning-agent.py) | [📖](https://docs.praison.ai/features/planning-mode) |
| 🔧 Planning Tools | [Example](#3-agent-with-planning-mode) | [📖](https://docs.praison.ai/features/planning-mode) |
| 🧠 Planning Reasoning | [Example](#3-agent-with-planning-mode) | [📖](https://docs.praison.ai/features/planning-mode) |
@@ -1120,6 +1124,15 @@ praisonai workflow auto "Research AI trends" --pattern parallel
# Generate routing workflow (classifier routes to specialists)
praisonai workflow auto "Build a chatbot" --pattern routing
# Generate orchestrator-workers workflow (central orchestrator delegates)
praisonai workflow auto "Comprehensive market analysis" --pattern orchestrator-workers
# Generate evaluator-optimizer workflow (iterative refinement)
praisonai workflow auto "Write and refine article" --pattern evaluator-optimizer
# Specify output file
praisonai workflow auto "Build a chatbot" --pattern routing
# Specify output file
praisonai workflow auto "Research AI" --pattern sequential --output my_workflow.yaml
```
@@ -1129,7 +1142,7 @@ praisonai workflow auto "Research AI" --pattern sequential --output my_workflow.
|------|-------------|
| `--workflow-var key=value` | Set workflow variable (can be repeated) |
| `--var key=value` | Set variable for YAML workflows |
| `--pattern <pattern>` | Pattern for auto-generation (sequential, parallel, routing, loop) |
| `--pattern <pattern>` | Pattern for auto-generation (sequential, parallel, routing, loop, orchestrator-workers, evaluator-optimizer) |
| `--output <file>` | Output file for auto-generation |
| `--llm <model>` | LLM model (e.g., openai/gpt-4o-mini) |
| `--tools <tools>` | Tools (comma-separated, e.g., tavily) |
+2 -2
View File
@@ -3,8 +3,8 @@ class Praisonai < Formula
desc "AI tools for various AI applications"
homepage "https://github.com/MervinPraison/PraisonAI"
url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.41.tar.gz"
sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.41.tar.gz | shasum -a 256`.split.first
url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.44.tar.gz"
sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.44.tar.gz | shasum -a 256`.split.first
license "MIT"
depends_on "[email protected]"
+505 -105
View File
@@ -51,14 +51,140 @@ try:
except ImportError:
PRAISONAI_TOOLS_AVAILABLE = False
# LiteLLM availability check for multi-provider support
LITELLM_AVAILABLE = False
try:
import litellm # noqa: F401 - imported for availability check
LITELLM_AVAILABLE = True
except ImportError:
pass
logging.basicConfig(level=os.environ.get('LOGLEVEL', 'INFO').upper(), format='%(asctime)s - %(levelname)s - %(message)s')
# Define Pydantic models outside of the generate method
# =============================================================================
# Available Tools List (shared between generators)
# =============================================================================
AVAILABLE_TOOLS = [
"CodeDocsSearchTool", "CSVSearchTool", "DirectorySearchTool", "DOCXSearchTool",
"DirectoryReadTool", "FileReadTool", "TXTSearchTool", "JSONSearchTool",
"MDXSearchTool", "PDFSearchTool", "RagTool", "ScrapeElementFromWebsiteTool",
"ScrapeWebsiteTool", "WebsiteSearchTool", "XMLSearchTool",
"YoutubeChannelSearchTool", "YoutubeVideoSearchTool"
]
# =============================================================================
# Base Generator Class (DRY - shared functionality)
# =============================================================================
class BaseAutoGenerator:
"""
Base class for auto-generators with shared functionality.
Provides:
- Lazy-loaded instructor client (LiteLLM or OpenAI fallback)
- Environment variable handling for model/API configuration
- Config list management
"""
def __init__(self, config_list: Optional[List[Dict]] = None):
"""
Initialize base generator with LLM configuration.
Args:
config_list: Optional LLM configuration list
"""
# Support multiple environment variable patterns for better compatibility
model_name = os.environ.get("MODEL_NAME") or os.environ.get("OPENAI_MODEL_NAME", "gpt-4o-mini")
base_url = (
os.environ.get("OPENAI_BASE_URL") or
os.environ.get("OPENAI_API_BASE") or
os.environ.get("OLLAMA_API_BASE", "https://api.openai.com/v1")
)
self.config_list = config_list or [
{
'model': model_name,
'base_url': base_url,
'api_key': os.environ.get("OPENAI_API_KEY")
}
]
self._client = None # Lazy loading for performance
@property
def client(self):
"""Lazy load the instructor client to avoid performance impact.
Uses LiteLLM via instructor.from_provider if available for multi-provider support,
otherwise falls back to direct OpenAI SDK.
"""
if self._client is None:
if LITELLM_AVAILABLE:
# Use LiteLLM for multi-provider support (100+ LLMs)
model_name = self.config_list[0]['model']
self._client = instructor.from_provider(
f"litellm/{model_name}",
mode=instructor.Mode.JSON
)
else:
# Fallback to direct OpenAI SDK
self._client = instructor.patch(
OpenAI(
base_url=self.config_list[0]['base_url'],
api_key=self.config_list[0]['api_key'],
),
mode=instructor.Mode.JSON,
)
return self._client
@staticmethod
def get_available_tools() -> List[str]:
"""Return list of available tools for agent assignment."""
return AVAILABLE_TOOLS.copy()
@staticmethod
def analyze_complexity(topic: str) -> str:
"""
Analyze task complexity based on keywords.
Args:
topic: The task description
Returns:
str: Complexity level - 'simple', 'moderate', or 'complex'
"""
topic_lower = topic.lower()
# Complex task indicators
complex_keywords = [
'comprehensive', 'multi-step', 'analyze and', 'research and write',
'multiple', 'coordinate', 'complex', 'detailed analysis',
'full report', 'in-depth', 'thorough'
]
# Simple task indicators
simple_keywords = [
'write a', 'create a', 'simple', 'quick', 'brief',
'haiku', 'poem', 'summary', 'list', 'single'
]
if any(kw in topic_lower for kw in complex_keywords):
return 'complex'
elif any(kw in topic_lower for kw in simple_keywords):
return 'simple'
else:
return 'moderate'
# =============================================================================
# Pydantic Models for Structured Output
# =============================================================================
class TaskDetails(BaseModel):
"""Details for a single task."""
description: str
expected_output: str
class RoleDetails(BaseModel):
"""Details for a single role/agent."""
role: str
goal: str
backstory: str
@@ -66,14 +192,62 @@ class RoleDetails(BaseModel):
tools: List[str]
class TeamStructure(BaseModel):
"""Structure for multi-agent team."""
roles: Dict[str, RoleDetails]
class AutoGenerator:
def __init__(self, topic="Movie Story writing about AI", agent_file="test.yaml", framework="crewai", config_list: Optional[List[Dict]] = None):
class SingleAgentStructure(BaseModel):
"""Structure for single-agent generation (Anthropic's 'start simple' principle)."""
name: str
role: str
goal: str
backstory: str
instructions: str
tools: List[str] = []
task_description: str
expected_output: str
class PatternRecommendation(BaseModel):
"""LLM-based pattern recommendation with reasoning."""
pattern: str # sequential, parallel, routing, orchestrator-workers, evaluator-optimizer
reasoning: str # Why this pattern was chosen
confidence: float # 0.0 to 1.0 confidence score
class ValidationGate(BaseModel):
"""Validation gate for prompt chaining workflows."""
criteria: str # What to validate
pass_action: str # Action if validation passes (e.g., "continue", "next_step")
fail_action: str # Action if validation fails (e.g., "retry", "escalate", "abort")
class AutoGenerator(BaseAutoGenerator):
"""
Auto-generates agents.yaml files from a topic description.
Inherits from BaseAutoGenerator for shared LLM client functionality.
Usage:
generator = AutoGenerator(framework="crewai", topic="Create a movie script")
path = generator.generate()
"""
def __init__(self, topic="Movie Story writing about AI", agent_file="test.yaml",
framework="crewai", config_list: Optional[List[Dict]] = None,
pattern: str = "sequential", single_agent: bool = False):
"""
Initialize the AutoGenerator class with the specified topic, agent file, and framework.
Args:
topic: The task/topic for agent generation
agent_file: Output YAML file name
framework: Framework to use (crewai, autogen, praisonai)
config_list: Optional LLM configuration
pattern: Workflow pattern (sequential, parallel, routing, orchestrator-workers, evaluator-optimizer)
single_agent: If True, generate a single agent instead of a team
Note: autogen framework is different from this AutoGenerator class.
"""
# Initialize base class first (handles config_list and client)
super().__init__(config_list=config_list)
# Validate framework availability and show framework-specific messages
if framework == "crewai" and not CREWAI_AVAILABLE:
raise ImportError("""
@@ -106,35 +280,42 @@ Tools are not available for {framework}. To use tools, install:
pip install "praisonai[{framework}]"
""")
# Support multiple environment variable patterns for better compatibility
# Priority order: MODEL_NAME > OPENAI_MODEL_NAME for model selection
model_name = os.environ.get("MODEL_NAME") or os.environ.get("OPENAI_MODEL_NAME", "gpt-5-nano")
# Priority order for base_url: OPENAI_BASE_URL > OPENAI_API_BASE > OLLAMA_API_BASE
# OPENAI_BASE_URL is the standard OpenAI SDK environment variable
base_url = (
os.environ.get("OPENAI_BASE_URL") or
os.environ.get("OPENAI_API_BASE") or
os.environ.get("OLLAMA_API_BASE", "https://api.openai.com/v1")
)
self.config_list = config_list or [
{
'model': model_name,
'base_url': base_url,
'api_key': os.environ.get("OPENAI_API_KEY")
}
]
self.topic = topic
self.agent_file = agent_file
self.framework = framework or "praisonai"
self.client = instructor.patch(
OpenAI(
base_url=self.config_list[0]['base_url'],
api_key=self.config_list[0]['api_key'],
),
mode=instructor.Mode.JSON,
)
self.pattern = pattern
self.single_agent = single_agent
def recommend_pattern(self, topic: str = None) -> str:
"""
Recommend the best workflow pattern based on task characteristics.
Args:
topic: The task description (uses self.topic if not provided)
Returns:
str: Recommended pattern name
"""
task = topic or self.topic
task_lower = task.lower()
# Keywords that suggest specific patterns
parallel_keywords = ['multiple', 'concurrent', 'parallel', 'simultaneously', 'different sources', 'compare', 'various']
routing_keywords = ['classify', 'categorize', 'route', 'different types', 'depending on', 'if...then']
orchestrator_keywords = ['complex', 'comprehensive', 'multi-step', 'coordinate', 'delegate', 'break down', 'analyze and']
evaluator_keywords = ['refine', 'improve', 'iterate', 'quality', 'review', 'feedback', 'polish', 'optimize']
# Check for pattern indicators
if any(kw in task_lower for kw in evaluator_keywords):
return "evaluator-optimizer"
elif any(kw in task_lower for kw in orchestrator_keywords):
return "orchestrator-workers"
elif any(kw in task_lower for kw in routing_keywords):
return "routing"
elif any(kw in task_lower for kw in parallel_keywords):
return "parallel"
else:
return "sequential"
def generate(self, merge=False):
"""
@@ -157,7 +338,8 @@ Tools are not available for {framework}. To use tools, install:
response = self.client.chat.completions.create(
model=self.config_list[0]['model'],
response_model=TeamStructure,
max_retries=10,
max_retries=5,
timeout=120.0, # 2 minute timeout for complex generations
messages=[
{"role": "system", "content": "You are a helpful assistant designed to output complex team structures."},
{"role": "user", "content": self.get_user_content()}
@@ -194,8 +376,7 @@ Tools are not available for {framework}. To use tools, install:
"goal": role_details['goal'],
"role": role_details['role'],
"tasks": {},
# "tools": role_details.get('tools', []),
"tools": ['']
"tools": role_details.get('tools', [])
}
for task_id, task_details in role_details['tasks'].items():
@@ -265,7 +446,7 @@ Tools are not available for {framework}. To use tools, install:
"goal": role_details['goal'],
"role": role_details['role'],
"tasks": {},
"tools": ['']
"tools": role_details.get('tools', [])
}
# Add tasks for this role
@@ -292,45 +473,77 @@ Tools are not available for {framework}. To use tools, install:
prompt = generator.get_user_content()
print(prompt)
"""
user_content = """Generate a team structure for \"""" + self.topic + """\" task.
No Input data will be provided to the team.
The team will work in sequence. First role will pass the output to the next role, and so on.
The last role will generate the final output.
Think step by step.
With maximum 3 roles, each with 1 task. Include role goals, backstories, task descriptions, and expected outputs.
List of Available Tools: CodeDocsSearchTool, CSVSearchTool, DirectorySearchTool, DOCXSearchTool, DirectoryReadTool, FileReadTool, TXTSearchTool, JSONSearchTool, MDXSearchTool, PDFSearchTool, RagTool, ScrapeElementFromWebsiteTool, ScrapeWebsiteTool, WebsiteSearchTool, XMLSearchTool, YoutubeChannelSearchTool, YoutubeVideoSearchTool.
Only use Available Tools. Do Not use any other tools.
Example Below:
Use below example to understand the structure of the output.
The final role you create should satisfy the provided task: """ + self.topic + """.
{
"roles": {
"narrative_designer": {
"role": "Narrative Designer",
"goal": "Create AI storylines",
"backstory": "Skilled in narrative development for AI, with a focus on story resonance.",
"tools": ["ScrapeWebsiteTool"],
"tasks": {
"story_concept_development": {
"description": "Craft a unique AI story concept with depth and engagement using concept from this page the content https://www.asthebirdfliesblog.com/posts/how-to-write-book-story-development .",
"expected_output": "Document with narrative arcs, character bios, and settings."
}
}
},
"scriptwriter": {
"role": "Scriptwriter",
"goal": "Write scripts from AI concepts",
"backstory": "Expert in dialogue and script structure, translating concepts into scripts.",
"tasks": {
"scriptwriting_task": {
"description": "Turn narrative concepts into scripts, including dialogue and scenes.",
"expected_output": "Production-ready script with dialogue and scene details."
}
}
}
}
}
"""
# Pattern-specific guidance
pattern_guidance = {
"sequential": "The team will work in sequence. Each role passes output to the next.",
"parallel": "The team will work in parallel on independent subtasks, then combine results.",
"routing": "A classifier agent will route requests to specialized agents based on input type.",
"orchestrator-workers": "A central orchestrator will dynamically delegate tasks to specialized workers.",
"evaluator-optimizer": "One agent generates content, another evaluates it in a loop until quality criteria are met."
}
workflow_guidance = pattern_guidance.get(self.pattern, pattern_guidance["sequential"])
user_content = f"""Analyze and generate a team structure for: "{self.topic}"
STEP 1: TASK COMPLEXITY ANALYSIS
First, analyze the task complexity:
- Is this a simple task that 1-2 agents could handle?
- Does it require multiple specialized perspectives?
- Are there clear sequential dependencies between steps?
STEP 2: DETERMINE OPTIMAL TEAM SIZE
Based on your analysis, create the minimum number of agents needed:
- Simple tasks: 1-2 agents
- Moderate tasks: 2-3 agents
- Complex tasks: 3-4 agents (maximum)
IMPORTANT: Only create agents that provide meaningful specialization. Avoid unnecessary complexity.
STEP 3: DESIGN THE TEAM (Pattern: {self.pattern})
{workflow_guidance}
Each agent should have:
- A clear, distinct role
- A specific goal
- Relevant backstory
- 1 focused task with clear description and expected output
- Appropriate tools (only if needed)
Available Tools: CodeDocsSearchTool, CSVSearchTool, DirectorySearchTool, DOCXSearchTool, DirectoryReadTool, FileReadTool, TXTSearchTool, JSONSearchTool, MDXSearchTool, PDFSearchTool, RagTool, ScrapeElementFromWebsiteTool, ScrapeWebsiteTool, WebsiteSearchTool, XMLSearchTool, YoutubeChannelSearchTool, YoutubeVideoSearchTool.
Only use these tools if the task requires them. Use empty list [] if no tools needed.
Example structure (2 agents for a writing task):
{{
"roles": {{
"researcher": {{
"role": "Research Analyst",
"goal": "Gather comprehensive information on the topic",
"backstory": "Expert researcher skilled at finding and synthesizing information.",
"tools": ["WebsiteSearchTool"],
"tasks": {{
"research_task": {{
"description": "Research key information about the topic and compile findings.",
"expected_output": "Comprehensive research notes with key facts and insights."
}}
}}
}},
"writer": {{
"role": "Content Writer",
"goal": "Create polished final content",
"backstory": "Skilled writer who transforms research into engaging content.",
"tools": [],
"tasks": {{
"writing_task": {{
"description": "Write the final content based on research findings.",
"expected_output": "Polished, well-structured final document."
}}
}}
}}
}}
}}
Now generate the optimal team structure for: {self.topic}
"""
return user_content
@@ -372,13 +585,14 @@ class WorkflowStructure(BaseModel):
description: str
agents: Dict[str, WorkflowAgentDetails]
steps: List[Dict] # Can be agent steps, route, parallel, etc.
gates: Optional[List[ValidationGate]] = None # Optional validation gates
class WorkflowAutoGenerator:
class WorkflowAutoGenerator(BaseAutoGenerator):
"""
Auto-generates workflow.yaml files from a topic description.
Uses lazy loading to avoid performance impact when not used.
Inherits from BaseAutoGenerator for shared LLM client functionality.
Usage:
generator = WorkflowAutoGenerator(topic="Research AI trends and write a report")
@@ -387,7 +601,9 @@ class WorkflowAutoGenerator:
def __init__(self, topic: str = "Research and write about AI",
workflow_file: str = "workflow.yaml",
config_list: Optional[List[Dict]] = None):
config_list: Optional[List[Dict]] = None,
framework: str = "praisonai",
single_agent: bool = False):
"""
Initialize the WorkflowAutoGenerator.
@@ -395,45 +611,105 @@ class WorkflowAutoGenerator:
topic: The task/topic for the workflow
workflow_file: Output file name
config_list: Optional LLM configuration
framework: Framework to use (praisonai, crewai, autogen)
single_agent: If True, generate a single agent workflow
"""
# Support multiple environment variable patterns
model_name = os.environ.get("MODEL_NAME") or os.environ.get("OPENAI_MODEL_NAME", "gpt-4o-mini")
base_url = (
os.environ.get("OPENAI_BASE_URL") or
os.environ.get("OPENAI_API_BASE") or
os.environ.get("OLLAMA_API_BASE", "https://api.openai.com/v1")
)
# Initialize base class (handles config_list and client)
super().__init__(config_list=config_list)
self.config_list = config_list or [
{
'model': model_name,
'base_url': base_url,
'api_key': os.environ.get("OPENAI_API_KEY")
}
]
self.topic = topic
self.workflow_file = workflow_file
self._client = None # Lazy loading
self.framework = framework
self.single_agent = single_agent
@property
def client(self):
"""Lazy load the OpenAI client to avoid performance impact."""
if self._client is None:
self._client = instructor.patch(
OpenAI(
base_url=self.config_list[0]['base_url'],
api_key=self.config_list[0]['api_key'],
),
mode=instructor.Mode.JSON,
)
return self._client
def recommend_pattern(self, topic: str = None) -> str:
"""
Recommend the best workflow pattern based on task characteristics.
Args:
topic: The task description (uses self.topic if not provided)
Returns:
str: Recommended pattern name
Pattern recommendations based on Anthropic's best practices:
- sequential: Clear step-by-step dependencies
- parallel: Independent subtasks that can run concurrently
- routing: Different input types need different handling
- orchestrator-workers: Complex tasks needing dynamic decomposition
- evaluator-optimizer: Tasks requiring iterative refinement
"""
task = topic or self.topic
task_lower = task.lower()
# Keywords that suggest specific patterns
parallel_keywords = ['multiple', 'concurrent', 'parallel', 'simultaneously', 'different sources', 'compare', 'various']
routing_keywords = ['classify', 'categorize', 'route', 'different types', 'depending on', 'if...then']
orchestrator_keywords = ['complex', 'comprehensive', 'multi-step', 'coordinate', 'delegate', 'break down', 'analyze and']
evaluator_keywords = ['refine', 'improve', 'iterate', 'quality', 'review', 'feedback', 'polish', 'optimize']
# Check for pattern indicators
if any(kw in task_lower for kw in evaluator_keywords):
return "evaluator-optimizer"
elif any(kw in task_lower for kw in orchestrator_keywords):
return "orchestrator-workers"
elif any(kw in task_lower for kw in routing_keywords):
return "routing"
elif any(kw in task_lower for kw in parallel_keywords):
return "parallel"
else:
return "sequential"
def generate(self, pattern: str = "sequential") -> str:
def recommend_pattern_llm(self, topic: str = None) -> PatternRecommendation:
"""
Use LLM to recommend the best workflow pattern with reasoning.
Args:
topic: The task description (uses self.topic if not provided)
Returns:
PatternRecommendation: Pattern with reasoning and confidence score
"""
task = topic or self.topic
prompt = f"""Analyze this task and recommend the best workflow pattern:
Task: "{task}"
Available patterns:
1. sequential - Agents work one after another, passing output to the next
2. parallel - Multiple agents work concurrently on independent subtasks
3. routing - A classifier routes requests to specialized agents based on input type
4. orchestrator-workers - Central orchestrator dynamically delegates to specialized workers
5. evaluator-optimizer - Generator creates content, evaluator reviews in a loop until quality met
Respond with:
- pattern: The recommended pattern name
- reasoning: Why this pattern is best for this task
- confidence: Your confidence score (0.0 to 1.0)
"""
response = self.client.chat.completions.create(
model=self.config_list[0]['model'],
response_model=PatternRecommendation,
max_retries=3,
timeout=60.0,
messages=[
{"role": "system", "content": "You are an expert at designing AI agent workflows."},
{"role": "user", "content": prompt}
]
)
return response
def generate(self, pattern: str = "sequential", merge: bool = False) -> str:
"""
Generate a workflow YAML file.
Args:
pattern: Workflow pattern - "sequential", "routing", "parallel", "loop"
pattern: Workflow pattern - "sequential", "routing", "parallel", "loop",
"orchestrator-workers", "evaluator-optimizer"
merge: If True, merge with existing workflow file instead of overwriting
Returns:
Path to the generated workflow file
@@ -442,6 +718,7 @@ class WorkflowAutoGenerator:
model=self.config_list[0]['model'],
response_model=WorkflowStructure,
max_retries=5,
timeout=120.0, # 2 minute timeout for complex generations
messages=[
{"role": "system", "content": "You are a helpful assistant that designs workflow structures."},
{"role": "user", "content": self._get_prompt(pattern)}
@@ -449,16 +726,86 @@ class WorkflowAutoGenerator:
)
json_data = json.loads(response.model_dump_json())
if merge and os.path.exists(self.workflow_file):
return self._save_workflow(self.merge_with_existing_workflow(json_data), pattern)
return self._save_workflow(json_data, pattern)
def merge_with_existing_workflow(self, new_data: Dict) -> Dict:
"""
Merge new workflow data with existing workflow file.
Args:
new_data: The new workflow data to merge
Returns:
Dict: Merged workflow data
"""
try:
with open(self.workflow_file, 'r') as f:
existing_data = yaml.safe_load(f)
if not existing_data:
return new_data
except (yaml.YAMLError, FileNotFoundError) as e:
logging.warning(f"Could not load existing workflow file {self.workflow_file}: {e}")
return new_data
# Merge agents (avoid duplicates)
merged_agents = existing_data.get('agents', {}).copy()
for agent_id, agent_data in new_data.get('agents', {}).items():
# Rename if conflict
final_id = agent_id
counter = 1
while final_id in merged_agents:
final_id = f"{agent_id}_auto_{counter}"
counter += 1
merged_agents[final_id] = agent_data
# Merge steps (append new steps)
merged_steps = existing_data.get('steps', []) + new_data.get('steps', [])
# Create merged structure
merged = {
'name': existing_data.get('name', new_data.get('name', 'Merged Workflow')),
'description': f"{existing_data.get('description', '')} + {new_data.get('description', '')}",
'agents': merged_agents,
'steps': merged_steps
}
return merged
def _get_prompt(self, pattern: str) -> str:
"""Generate the prompt based on the workflow pattern."""
# Analyze complexity to determine agent count
complexity = self.analyze_complexity(self.topic)
if complexity == 'simple':
agent_guidance = "Create 1-2 agents (simple task detected)."
elif complexity == 'complex':
agent_guidance = "Create 3-4 agents (complex task detected)."
else:
agent_guidance = "Create 2-3 agents (moderate task detected)."
# Get available tools
tools_list = ", ".join(self.get_available_tools())
base_prompt = f"""Generate a workflow structure for: "{self.topic}"
STEP 1: ANALYZE TASK COMPLEXITY
- Is this a simple task (1-2 agents)?
- Does it require multiple specialists (2-3 agents)?
- Is it complex with many dependencies (3-4 agents)?
STEP 2: DESIGN WORKFLOW
The workflow should use the "{pattern}" pattern.
Create 2-4 agents with clear roles and instructions.
{agent_guidance}
Each agent should have clear roles and instructions.
Each step should have a clear action.
STEP 3: ASSIGN TOOLS (if needed)
Available Tools: {tools_list}
Only assign tools if the task requires them. Use empty list or null if no tools needed.
"""
if pattern == "routing":
@@ -501,6 +848,59 @@ Example structure:
{"agent": "aggregator", "action": "Combine all findings"}
]
}
"""
elif pattern == "orchestrator-workers":
base_prompt += """
Create an orchestrator-workers workflow where a central orchestrator dynamically delegates tasks to specialized workers.
The orchestrator analyzes the input, decides which workers are needed, and synthesizes results.
Example structure:
{
"name": "Orchestrator-Workers Workflow",
"description": "Central orchestrator delegates to specialized workers",
"agents": {
"orchestrator": {"name": "Orchestrator", "role": "Task Coordinator", "goal": "Analyze tasks and delegate to appropriate workers", "instructions": "Break down the task, identify required specialists, and coordinate their work. Output a JSON with 'subtasks' array listing which workers to invoke."},
"researcher": {"name": "Researcher", "role": "Research Specialist", "goal": "Gather information", "instructions": "Research and provide factual information"},
"analyst": {"name": "Analyst", "role": "Data Analyst", "goal": "Analyze data and patterns", "instructions": "Analyze information and identify insights"},
"writer": {"name": "Writer", "role": "Content Writer", "goal": "Create written content", "instructions": "Write clear, engaging content"},
"synthesizer": {"name": "Synthesizer", "role": "Results Synthesizer", "goal": "Combine all worker outputs", "instructions": "Synthesize all worker outputs into a coherent final result"}
},
"steps": [
{"agent": "orchestrator", "action": "Analyze task and determine required workers: {{input}}"},
{"name": "worker_dispatch", "parallel": [
{"agent": "researcher", "action": "Research: {{input}}"},
{"agent": "analyst", "action": "Analyze: {{input}}"},
{"agent": "writer", "action": "Draft content for: {{input}}"}
]},
{"agent": "synthesizer", "action": "Combine all worker outputs into final result"}
]
}
"""
elif pattern == "evaluator-optimizer":
base_prompt += """
Create an evaluator-optimizer workflow where one agent generates content and another evaluates it in a loop.
The generator improves based on evaluator feedback until quality criteria are met.
Example structure:
{
"name": "Evaluator-Optimizer Workflow",
"description": "Iterative refinement through generation and evaluation",
"agents": {
"generator": {"name": "Generator", "role": "Content Generator", "goal": "Generate high-quality content", "instructions": "Create content based on the input. If feedback is provided, improve the content accordingly."},
"evaluator": {"name": "Evaluator", "role": "Quality Evaluator", "goal": "Evaluate content quality", "instructions": "Evaluate the content on: clarity, accuracy, completeness, and relevance. Score 1-10 for each. If average score < 7, provide specific improvement feedback. If score >= 7, respond with 'APPROVED'."}
},
"steps": [
{"agent": "generator", "action": "Generate initial content for: {{input}}"},
{"name": "evaluation_loop", "loop": {
"agent": "evaluator",
"action": "Evaluate the generated content",
"condition": "output does not contain 'APPROVED'",
"max_iterations": 3,
"feedback_to": "generator"
}},
{"agent": "generator", "action": "Finalize content based on all feedback"}
]
}
"""
else: # sequential
base_prompt += """
@@ -21,6 +21,7 @@ Features:
- flow_display: Visual workflow tracking
- workflow: YAML workflow management
- n8n: Export workflows to n8n visual editor
- external_agents: External AI CLI tool integrations (Claude, Gemini, Codex, Cursor)
"""
# Type hints available for IDE support
@@ -43,6 +44,7 @@ __all__ = [
'FlowDisplayHandler',
'WorkflowHandler',
'N8nHandler',
'ExternalAgentsHandler',
]
def __getattr__(name):
@@ -95,4 +97,7 @@ def __getattr__(name):
elif name == 'N8nHandler':
from .n8n import N8nHandler
return N8nHandler
elif name == 'ExternalAgentsHandler':
from .external_agents import ExternalAgentsHandler
return ExternalAgentsHandler
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
@@ -0,0 +1,231 @@
"""
External Agents Handler for CLI.
Provides integration with external AI coding CLI tools:
- Claude Code CLI
- Gemini CLI
- OpenAI Codex CLI
- Cursor CLI
Usage:
praisonai "Refactor the code" --external-agent claude
praisonai "Analyze codebase" --external-agent gemini
praisonai "Fix bugs" --external-agent codex
praisonai "Add tests" --external-agent cursor
"""
from typing import Dict, Any, List, Optional, Tuple
from .base import FlagHandler
class ExternalAgentsHandler(FlagHandler):
"""
Handler for --external-agent flag.
Integrates external AI coding CLI tools as agent tools.
Example:
praisonai "Refactor the auth module" --external-agent claude
praisonai "Analyze this codebase" --external-agent gemini
"""
# Mapping of integration names to their module paths
INTEGRATIONS = {
"claude": "claude_code.ClaudeCodeIntegration",
"gemini": "gemini_cli.GeminiCLIIntegration",
"codex": "codex_cli.CodexCLIIntegration",
"cursor": "cursor_cli.CursorCLIIntegration",
}
def __init__(self, verbose: bool = False):
"""Initialize the handler."""
super().__init__(verbose)
self._integration_cache: Dict[str, Any] = {}
@property
def feature_name(self) -> str:
"""Return the feature name."""
return "external_agents"
@property
def flag_name(self) -> str:
"""Return the flag name."""
return "external-agent"
@property
def flag_help(self) -> str:
"""Return the flag help text."""
return "External AI CLI tool to use (claude, gemini, codex, cursor)"
def check_dependencies(self) -> Tuple[bool, str]:
"""Check if integrations module is available."""
try:
import importlib.util
if importlib.util.find_spec("praisonai.integrations") is not None:
return True, ""
return False, "Integrations module not available"
except ImportError:
return False, "Integrations module not available"
def get_integration(self, name: str, **options):
"""
Get an integration instance by name.
Args:
name: Integration name (claude, gemini, codex, cursor)
**options: Options to pass to the integration
Returns:
Integration instance
Raises:
ValueError: If integration name is invalid
"""
if name not in self.INTEGRATIONS:
raise ValueError(f"Unknown integration: {name}. Available: {list(self.INTEGRATIONS.keys())}")
# Check cache first
cache_key = f"{name}_{hash(frozenset(options.items()))}"
if cache_key in self._integration_cache:
return self._integration_cache[cache_key]
# Lazy import the integration
module_path = self.INTEGRATIONS[name]
module_name, class_name = module_path.rsplit(".", 1)
import importlib
module = importlib.import_module(f"praisonai.integrations.{module_name}")
integration_class = getattr(module, class_name)
# Create instance with options
integration = integration_class(**options)
# Cache it
self._integration_cache[cache_key] = integration
return integration
def list_integrations(self) -> List[str]:
"""
List all available integration names.
Returns:
List of integration names
"""
return list(self.INTEGRATIONS.keys())
def check_availability(self) -> Dict[str, bool]:
"""
Check availability of all integrations.
Returns:
Dict mapping integration name to availability
"""
availability = {}
for name in self.INTEGRATIONS:
try:
integration = self.get_integration(name)
availability[name] = integration.is_available
except Exception:
availability[name] = False
return availability
def apply_to_agent_config(self, config: Dict[str, Any], flag_value: Any) -> Dict[str, Any]:
"""
Apply external agent configuration to agent config.
Args:
config: Agent configuration dictionary
flag_value: Integration name or dict with name and options
Returns:
Modified configuration with external agent tool
"""
if not flag_value:
return config
# Handle both string and dict formats
if isinstance(flag_value, str):
name = flag_value
options = {}
elif isinstance(flag_value, dict):
name = flag_value.get("name", "")
options = {k: v for k, v in flag_value.items() if k != "name"}
else:
return config
try:
integration = self.get_integration(name, **options)
if integration.is_available:
# Add tool to existing tools
existing_tools = config.get("tools", [])
if isinstance(existing_tools, list):
existing_tools.append(integration.as_tool())
else:
existing_tools = [integration.as_tool()]
config["tools"] = existing_tools
self.print_status(f"🔌 External agent connected: {name}", "success")
else:
self.print_status(f"⚠️ External agent not available: {name}", "warning")
except ValueError as e:
self.print_status(str(e), "error")
except Exception as e:
self.print_status(f"Failed to connect external agent: {e}", "error")
return config
def execute(
self,
integration_name: str = None,
**options
) -> Optional[Any]:
"""
Execute external agent setup.
Args:
integration_name: Name of the integration
**options: Options for the integration
Returns:
Integration instance or None
"""
if not integration_name:
# List available integrations
self.print_status("\n🔧 Available External Agents:", "info")
self.print_status("-" * 40, "info")
availability = self.check_availability()
for name, available in availability.items():
status = "✅ Available" if available else "❌ Not installed"
self.print_status(f" {name}: {status}", "info")
return None
try:
integration = self.get_integration(integration_name, **options)
if integration.is_available:
self.print_status(f"🔌 External agent ready: {integration_name}", "success")
return integration
else:
self.print_status(f"⚠️ External agent not installed: {integration_name}", "warning")
self.print_status(f" Install with: {self._get_install_instructions(integration_name)}", "info")
return None
except ValueError as e:
self.print_status(str(e), "error")
return None
def _get_install_instructions(self, name: str) -> str:
"""Get installation instructions for an integration."""
instructions = {
"claude": "npm install -g @anthropic-ai/claude-code",
"gemini": "npm install -g @anthropic-ai/gemini-cli",
"codex": "npm install -g @openai/codex",
"cursor": "Download from cursor.com",
}
return instructions.get(name, "See documentation")
@@ -610,7 +610,7 @@ Example:
return None
# Validate pattern
valid_patterns = ["sequential", "routing", "parallel", "loop"]
valid_patterns = ["sequential", "routing", "parallel", "loop", "orchestrator-workers", "evaluator-optimizer"]
if pattern not in valid_patterns:
self.print_status(f"Unknown pattern: {pattern}", "error")
self.print_status(f"Valid patterns: {', '.join(valid_patterns)}", "info")
+1 -1
View File
@@ -2002,7 +2002,7 @@ class PraisonAI:
return
# Validate pattern
valid_patterns = ["sequential", "routing", "parallel", "loop"]
valid_patterns = ["sequential", "routing", "parallel", "loop", "orchestrator-workers", "evaluator-optimizer"]
if pattern not in valid_patterns:
print(f"[red]Unknown pattern: {pattern}[/red]")
print(f"[cyan]Valid patterns: {', '.join(valid_patterns)}[/cyan]")
+1 -1
View File
@@ -57,7 +57,7 @@ class CloudDeployer:
file.write("FROM python:3.11-slim\n")
file.write("WORKDIR /app\n")
file.write("COPY . .\n")
file.write("RUN pip install flask praisonai==2.3.41 gunicorn markdown\n")
file.write("RUN pip install flask praisonai==2.3.44 gunicorn markdown\n")
file.write("EXPOSE 8080\n")
file.write('CMD ["gunicorn", "-b", "0.0.0.0:8080", "api:app"]\n')
@@ -0,0 +1,56 @@
"""
PraisonAI Integrations - External CLI tool integrations.
This module provides integrations with external AI coding CLI tools:
- Claude Code CLI
- Gemini CLI
- OpenAI Codex CLI
- Cursor CLI
All integrations use lazy loading to avoid performance impact.
Usage:
from praisonai.integrations import ClaudeCodeIntegration, GeminiCLIIntegration
# Create integration
claude = ClaudeCodeIntegration(workspace="/path/to/project")
# Use as agent tool
tool = claude.as_tool()
# Or execute directly
result = await claude.execute("Refactor this code")
"""
# Lazy imports to avoid performance impact
__all__ = [
'BaseCLIIntegration',
'ClaudeCodeIntegration',
'GeminiCLIIntegration',
'CodexCLIIntegration',
'CursorCLIIntegration',
'get_available_integrations',
]
def __getattr__(name):
"""Lazy load integrations to minimize import overhead."""
if name == 'BaseCLIIntegration':
from .base import BaseCLIIntegration
return BaseCLIIntegration
elif name == 'ClaudeCodeIntegration':
from .claude_code import ClaudeCodeIntegration
return ClaudeCodeIntegration
elif name == 'GeminiCLIIntegration':
from .gemini_cli import GeminiCLIIntegration
return GeminiCLIIntegration
elif name == 'CodexCLIIntegration':
from .codex_cli import CodexCLIIntegration
return CodexCLIIntegration
elif name == 'CursorCLIIntegration':
from .cursor_cli import CursorCLIIntegration
return CursorCLIIntegration
elif name == 'get_available_integrations':
from .base import get_available_integrations
return get_available_integrations
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
@@ -0,0 +1,300 @@
"""
Base class for external CLI tool integrations.
Provides a consistent interface for integrating with AI coding CLI tools
like Claude Code, Gemini CLI, Codex CLI, and Cursor CLI.
Features:
- Lazy availability checking with caching
- Async subprocess execution
- Tool wrapper for agent integration
- Timeout handling
"""
from abc import ABC, abstractmethod
from typing import AsyncIterator, Optional, Dict, Any, Tuple, List
import asyncio
import shutil
import os
class BaseCLIIntegration(ABC):
"""
Abstract base class for external CLI tool integrations.
Provides:
- Lazy availability checking with class-level caching
- Async subprocess execution with timeout handling
- Tool wrapper for PraisonAI agent integration
Subclasses must implement:
- cli_command property: The CLI command name
- execute method: Execute the CLI and return result
- stream method: Stream output from the CLI
Example:
class MyIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "my-cli"
async def execute(self, prompt: str, **options) -> str:
return await self.execute_async(["my-cli", "-p", prompt])
async def stream(self, prompt: str, **options):
async for line in self.stream_async(["my-cli", "-p", prompt]):
yield line
"""
# Class-level cache for availability checks (shared across instances)
_availability_cache: Dict[str, bool] = {}
def __init__(self, workspace: str = ".", timeout: int = 300):
"""
Initialize the CLI integration.
Args:
workspace: Working directory for CLI execution (default: current dir)
timeout: Default timeout in seconds for CLI execution (default: 300)
"""
self.workspace = workspace
self.timeout = timeout
@property
@abstractmethod
def cli_command(self) -> str:
"""
Return the CLI command name.
This is used for availability checking and tool naming.
Returns:
str: The CLI command (e.g., "claude", "gemini", "codex", "cursor-agent")
"""
pass
@property
def is_available(self) -> bool:
"""
Check if the CLI tool is installed and available.
Uses class-level caching to avoid repeated filesystem checks.
Returns:
bool: True if the CLI is available, False otherwise
"""
if self.cli_command not in self._availability_cache:
self._availability_cache[self.cli_command] = shutil.which(self.cli_command) is not None
return self._availability_cache[self.cli_command]
@abstractmethod
async def execute(self, prompt: str, **options) -> str:
"""
Execute the CLI tool and return the result.
Args:
prompt: The prompt/query to send to the CLI
**options: Additional options for the CLI
Returns:
str: The CLI output
"""
pass
@abstractmethod
async def stream(self, prompt: str, **options) -> AsyncIterator[Dict[str, Any]]:
"""
Stream output from the CLI tool.
Args:
prompt: The prompt/query to send to the CLI
**options: Additional options for the CLI
Yields:
dict: Parsed output events from the CLI
"""
pass
async def execute_async(self, cmd: List[str], timeout: Optional[int] = None) -> str:
"""
Execute a command asynchronously and return stdout.
Args:
cmd: Command and arguments as a list
timeout: Timeout in seconds (uses self.timeout if not specified)
Returns:
str: The command's stdout
Raises:
TimeoutError: If the command times out
"""
timeout = timeout or self.timeout
proc = await asyncio.create_subprocess_exec(
*cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
cwd=self.workspace
)
try:
stdout, stderr = await asyncio.wait_for(
proc.communicate(),
timeout=timeout
)
return stdout.decode()
except asyncio.TimeoutError:
proc.kill()
await proc.wait()
raise TimeoutError(f"Command timed out after {timeout}s: {' '.join(cmd)}")
async def execute_async_with_stderr(
self,
cmd: List[str],
timeout: Optional[int] = None
) -> Tuple[str, str]:
"""
Execute a command asynchronously and return both stdout and stderr.
Args:
cmd: Command and arguments as a list
timeout: Timeout in seconds (uses self.timeout if not specified)
Returns:
Tuple[str, str]: (stdout, stderr)
Raises:
TimeoutError: If the command times out
"""
timeout = timeout or self.timeout
proc = await asyncio.create_subprocess_exec(
*cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
cwd=self.workspace
)
try:
stdout, stderr = await asyncio.wait_for(
proc.communicate(),
timeout=timeout
)
return stdout.decode(), stderr.decode()
except asyncio.TimeoutError:
proc.kill()
await proc.wait()
raise TimeoutError(f"Command timed out after {timeout}s: {' '.join(cmd)}")
async def stream_async(
self,
cmd: List[str],
timeout: Optional[int] = None
) -> AsyncIterator[str]:
"""
Stream stdout lines from a command asynchronously.
Args:
cmd: Command and arguments as a list
timeout: Timeout in seconds for the entire operation
Yields:
str: Each line of output
"""
timeout = timeout or self.timeout
proc = await asyncio.create_subprocess_exec(
*cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
cwd=self.workspace
)
try:
async def read_lines():
while True:
line = await proc.stdout.readline()
if not line:
break
yield line.decode().rstrip('\n')
async for line in read_lines():
yield line
except asyncio.TimeoutError:
proc.kill()
await proc.wait()
raise TimeoutError(f"Stream timed out after {timeout}s")
finally:
if proc.returncode is None:
proc.kill()
await proc.wait()
def as_tool(self) -> callable:
"""
Return a callable suitable for use as a PraisonAI agent tool.
The returned function is synchronous and wraps the async execute method.
Uses asyncio.run() for proper event loop handling (Python 3.7+).
Returns:
callable: A function that can be used as an agent tool
"""
# Capture self for closure
integration = self
def tool_func(query: str) -> str:
"""Execute the CLI tool with the given query."""
# Use asyncio.run() for clean event loop management
# This creates a new event loop, runs the coroutine, and closes it
try:
# Check if we're already in an async context
asyncio.get_running_loop()
# If we're in an async context, use ThreadPoolExecutor to avoid nested loop
import concurrent.futures
with concurrent.futures.ThreadPoolExecutor() as executor:
future = executor.submit(asyncio.run, integration.execute(query))
return future.result()
except RuntimeError:
# No running loop, safe to use asyncio.run()
return asyncio.run(integration.execute(query))
# Set function metadata for agent tool registration
tool_func.__name__ = f"{self.cli_command}_tool"
tool_func.__doc__ = f"Execute {self.cli_command} for coding tasks."
return tool_func
def get_env(self) -> Dict[str, str]:
"""
Get environment variables for CLI execution.
Subclasses can override this to add tool-specific environment variables.
Returns:
dict: Environment variables to pass to the CLI
"""
return dict(os.environ)
def get_available_integrations() -> Dict[str, bool]:
"""
Get a dictionary of all integrations and their availability status.
Returns:
dict: Mapping of integration name to availability (True/False)
"""
from .claude_code import ClaudeCodeIntegration
from .gemini_cli import GeminiCLIIntegration
from .codex_cli import CodexCLIIntegration
from .cursor_cli import CursorCLIIntegration
integrations = {
'claude': ClaudeCodeIntegration(),
'gemini': GeminiCLIIntegration(),
'codex': CodexCLIIntegration(),
'cursor': CursorCLIIntegration(),
}
return {name: integration.is_available for name, integration in integrations.items()}
@@ -0,0 +1,270 @@
"""
Claude Code CLI Integration.
Provides integration with Claude Code CLI for AI-powered coding tasks.
Features:
- Headless mode execution with JSON output
- Session continuation support
- System prompt customization
- Tool restrictions
- Optional SDK integration (when claude-agent-sdk is installed)
Usage:
from praisonai.integrations import ClaudeCodeIntegration
# Create integration
claude = ClaudeCodeIntegration(workspace="/path/to/project")
# Execute a coding task
result = await claude.execute("Refactor the auth module")
# Stream output
async for event in claude.stream("Add error handling"):
print(event)
# Use as agent tool
tool = claude.as_tool()
"""
import json
import os
from typing import AsyncIterator, Dict, Any, Optional, List
from .base import BaseCLIIntegration
# Check if Claude Code SDK is available
CLAUDE_SDK_AVAILABLE = False
try:
from claude_agent_sdk import query as claude_query, ClaudeAgentOptions
CLAUDE_SDK_AVAILABLE = True
except ImportError:
pass
class ClaudeCodeIntegration(BaseCLIIntegration):
"""
Integration with Claude Code CLI.
Supports both subprocess-based execution and SDK-based execution
(when claude-agent-sdk is installed).
Attributes:
output_format: Output format ("json", "text", "stream-json")
skip_permissions: Whether to skip permission prompts
system_prompt: Custom system prompt to append
allowed_tools: List of allowed tools (e.g., ["Read", "Write", "Bash"])
disallowed_tools: List of disallowed tools
use_sdk: Whether to use the SDK instead of subprocess
"""
def __init__(
self,
workspace: str = ".",
timeout: int = 300,
output_format: str = "json",
skip_permissions: bool = True,
system_prompt: Optional[str] = None,
allowed_tools: Optional[List[str]] = None,
disallowed_tools: Optional[List[str]] = None,
use_sdk: bool = False,
model: Optional[str] = None,
):
"""
Initialize Claude Code integration.
Args:
workspace: Working directory for CLI execution
timeout: Timeout in seconds for CLI execution
output_format: Output format ("json", "text", "stream-json")
skip_permissions: Whether to skip permission prompts (--dangerously-skip-permissions)
system_prompt: Custom system prompt to append
allowed_tools: List of allowed tools
disallowed_tools: List of disallowed tools
use_sdk: Whether to use the SDK instead of subprocess
model: Model to use (e.g., "sonnet", "opus")
"""
super().__init__(workspace=workspace, timeout=timeout)
self.output_format = output_format
self.skip_permissions = skip_permissions
self.system_prompt = system_prompt
self.allowed_tools = allowed_tools
self.disallowed_tools = disallowed_tools
self.use_sdk = use_sdk and CLAUDE_SDK_AVAILABLE
self.model = model
# Session management
self._session_active = False
@property
def cli_command(self) -> str:
"""Return the CLI command name."""
return "claude"
@property
def sdk_available(self) -> bool:
"""Check if the Claude Code SDK is available."""
return CLAUDE_SDK_AVAILABLE
def _build_command(
self,
prompt: str,
continue_session: bool = False,
**options
) -> List[str]:
"""
Build the Claude CLI command.
Args:
prompt: The prompt to send
continue_session: Whether to continue a previous session
**options: Additional options
Returns:
List of command arguments
"""
cmd = ["claude"]
# Add continue flag if needed
if continue_session or self._session_active:
cmd.append("--continue")
# Add skip permissions flag
if self.skip_permissions:
cmd.append("--dangerously-skip-permissions")
# Add print mode flag
cmd.append("-p")
# Add prompt
cmd.append(prompt)
# Add output format
cmd.extend(["--output-format", self.output_format])
# Add model if specified
if self.model:
cmd.extend(["--model", self.model])
# Add system prompt if specified
if self.system_prompt:
cmd.extend(["--append-system-prompt", self.system_prompt])
# Add allowed tools if specified
if self.allowed_tools:
cmd.extend(["--allowedTools", ",".join(self.allowed_tools)])
# Add disallowed tools if specified
if self.disallowed_tools:
cmd.extend(["--disallowedTools", ",".join(self.disallowed_tools)])
return cmd
async def execute(self, prompt: str, **options) -> str:
"""
Execute Claude Code CLI and return the result.
Args:
prompt: The prompt/query to send
**options: Additional options (continue_session, etc.)
Returns:
str: The CLI output (parsed from JSON if output_format is "json")
"""
if self.use_sdk:
return await self._execute_sdk(prompt, **options)
return await self._execute_subprocess(prompt, **options)
async def _execute_subprocess(self, prompt: str, **options) -> str:
"""Execute using subprocess."""
cmd = self._build_command(prompt, **options)
output = await self.execute_async(cmd)
# Mark session as active for continuation
self._session_active = True
# Parse JSON output if applicable
if self.output_format == "json":
try:
data = json.loads(output)
# Extract the main result
if isinstance(data, dict):
return data.get("result", data.get("content", str(data)))
return str(data)
except json.JSONDecodeError:
return output
return output
async def _execute_sdk(self, prompt: str, **options) -> str:
"""Execute using the Claude Code SDK."""
if not CLAUDE_SDK_AVAILABLE:
raise RuntimeError("Claude Code SDK not available. Install with: pip install claude-agent-sdk")
sdk_options = ClaudeAgentOptions(
cwd=self.workspace,
system_prompt=self.system_prompt,
)
if self.allowed_tools:
sdk_options.allowed_tools = self.allowed_tools
if self.skip_permissions:
sdk_options.permission_mode = 'acceptEdits'
result_parts = []
async for message in claude_query(prompt=prompt, options=sdk_options):
if hasattr(message, 'content'):
for block in message.content:
if hasattr(block, 'text'):
result_parts.append(block.text)
return '\n'.join(result_parts)
async def stream(self, prompt: str, **options) -> AsyncIterator[Dict[str, Any]]:
"""
Stream output from Claude Code CLI.
Args:
prompt: The prompt/query to send
**options: Additional options
Yields:
dict: Parsed JSON events from the CLI
"""
# Use stream-json format for streaming
original_format = self.output_format
self.output_format = "stream-json"
try:
cmd = self._build_command(prompt, **options)
async for line in self.stream_async(cmd):
if line.strip():
try:
event = json.loads(line)
yield event
except json.JSONDecodeError:
yield {"type": "text", "content": line}
finally:
self.output_format = original_format
def reset_session(self):
"""Reset the session state."""
self._session_active = False
def get_env(self) -> Dict[str, str]:
"""Get environment variables for CLI execution."""
env = super().get_env()
# Add Anthropic API key if available
if "ANTHROPIC_API_KEY" in os.environ:
env["ANTHROPIC_API_KEY"] = os.environ["ANTHROPIC_API_KEY"]
elif "CLAUDE_API_KEY" in os.environ:
env["ANTHROPIC_API_KEY"] = os.environ["CLAUDE_API_KEY"]
return env
@@ -0,0 +1,252 @@
"""
OpenAI Codex CLI Integration.
Provides integration with OpenAI's Codex CLI for AI-powered coding tasks.
Features:
- Non-interactive execution with `codex exec`
- Full auto mode for file modifications
- Sandbox modes for security
- JSON streaming output
- Structured output with schemas
Usage:
from praisonai.integrations import CodexCLIIntegration
# Create integration
codex = CodexCLIIntegration(workspace="/path/to/project")
# Execute a coding task
result = await codex.execute("Fix the authentication bug")
# Execute with full auto (allows file modifications)
codex_auto = CodexCLIIntegration(full_auto=True)
result = await codex_auto.execute("Refactor the module")
# Use as agent tool
tool = codex.as_tool()
"""
import json
import os
from typing import AsyncIterator, Dict, Any, Optional, List
from .base import BaseCLIIntegration
class CodexCLIIntegration(BaseCLIIntegration):
"""
Integration with OpenAI's Codex CLI.
Attributes:
full_auto: Whether to allow file modifications
sandbox: Sandbox mode ("default", "danger-full-access")
json_output: Whether to use JSON streaming output
output_schema: Path to JSON schema for structured output
"""
def __init__(
self,
workspace: str = ".",
timeout: int = 300,
full_auto: bool = False,
sandbox: str = "default",
json_output: bool = False,
output_schema: Optional[str] = None,
output_file: Optional[str] = None,
):
"""
Initialize Codex CLI integration.
Args:
workspace: Working directory for CLI execution
timeout: Timeout in seconds for CLI execution
full_auto: Whether to allow file modifications (--full-auto)
sandbox: Sandbox mode ("default", "danger-full-access")
json_output: Whether to use JSON streaming output (--json)
output_schema: Path to JSON schema for structured output
output_file: Path to save the final output (-o)
"""
super().__init__(workspace=workspace, timeout=timeout)
self.full_auto = full_auto
self.sandbox = sandbox
self.json_output = json_output
self.output_schema = output_schema
self.output_file = output_file
@property
def cli_command(self) -> str:
"""Return the CLI command name."""
return "codex"
def _build_command(self, task: str, **options) -> List[str]:
"""
Build the Codex CLI command.
Args:
task: The task to execute
**options: Additional options
Returns:
List of command arguments
"""
cmd = ["codex", "exec"]
# Add task
cmd.append(task)
# Add full auto flag if enabled
if self.full_auto:
cmd.append("--full-auto")
# Add sandbox mode if not default
if self.sandbox and self.sandbox != "default":
cmd.extend(["--sandbox", self.sandbox])
# Add JSON output flag if enabled
if self.json_output:
cmd.append("--json")
# Add output schema if specified
if self.output_schema:
cmd.extend(["--output-schema", self.output_schema])
# Add output file if specified
if self.output_file:
cmd.extend(["-o", self.output_file])
return cmd
async def execute(self, prompt: str, **options) -> str:
"""
Execute Codex CLI and return the result.
Args:
prompt: The task/prompt to execute
**options: Additional options
Returns:
str: The CLI output
"""
cmd = self._build_command(prompt, **options)
output = await self.execute_async(cmd)
# Parse JSON Lines output if json_output is enabled
if self.json_output:
return self._parse_json_events(output)
return output
def _parse_json_events(self, output: str) -> str:
"""
Parse JSON Lines output and extract the final result.
Args:
output: Raw JSON Lines output
Returns:
str: The extracted result
"""
result_parts = []
for line in output.strip().split('\n'):
if not line.strip():
continue
try:
event = json.loads(line)
event_type = event.get("type", "")
# Extract agent messages
if event_type == "item.completed":
item = event.get("item", {})
if item.get("type") == "agent_message":
text = item.get("text", "")
if text:
result_parts.append(text)
except json.JSONDecodeError:
continue
return '\n'.join(result_parts) if result_parts else output
async def stream(self, prompt: str, **options) -> AsyncIterator[Dict[str, Any]]:
"""
Stream output from Codex CLI.
Args:
prompt: The task/prompt to execute
**options: Additional options
Yields:
dict: Parsed JSON events from the CLI
"""
# Ensure JSON output is enabled for streaming
original_json = self.json_output
self.json_output = True
try:
cmd = self._build_command(prompt, **options)
async for line in self.stream_async(cmd):
if line.strip():
try:
event = json.loads(line)
yield event
except json.JSONDecodeError:
yield {"type": "text", "content": line}
finally:
self.json_output = original_json
async def execute_with_schema(
self,
prompt: str,
schema_path: str,
output_path: Optional[str] = None
) -> Dict[str, Any]:
"""
Execute Codex with structured output schema.
Args:
prompt: The task/prompt to execute
schema_path: Path to the JSON schema file
output_path: Optional path to save the output
Returns:
dict: Parsed structured output
"""
original_schema = self.output_schema
original_output = self.output_file
self.output_schema = schema_path
if output_path:
self.output_file = output_path
try:
cmd = self._build_command(prompt)
output = await self.execute_async(cmd)
# If output file was specified, read from it
if output_path and os.path.exists(output_path):
with open(output_path, 'r') as f:
return json.load(f)
# Otherwise parse the output
try:
return json.loads(output)
except json.JSONDecodeError:
return {"result": output}
finally:
self.output_schema = original_schema
self.output_file = original_output
def get_env(self) -> Dict[str, str]:
"""Get environment variables for CLI execution."""
env = super().get_env()
# Codex uses ChatGPT authentication or API key
# The CLI handles authentication internally
return env
@@ -0,0 +1,192 @@
"""
Cursor CLI Integration.
Provides integration with Cursor's CLI (cursor-agent) for AI-powered coding tasks.
Features:
- Headless mode execution with JSON output
- Force mode for file modifications
- Model selection
- Session resume support
- Streaming output with partial deltas
Usage:
from praisonai.integrations import CursorCLIIntegration
# Create integration
cursor = CursorCLIIntegration(workspace="/path/to/project")
# Execute a coding task
result = await cursor.execute("Fix the authentication bug")
# Execute with force mode (allows file modifications)
cursor_force = CursorCLIIntegration(force=True)
result = await cursor_force.execute("Refactor the module")
# Use as agent tool
tool = cursor.as_tool()
"""
import json
import os
from typing import AsyncIterator, Dict, Any, Optional, List
from .base import BaseCLIIntegration
class CursorCLIIntegration(BaseCLIIntegration):
"""
Integration with Cursor CLI (cursor-agent).
Attributes:
output_format: Output format ("json", "text", "stream-json")
force: Whether to allow file modifications
model: Model to use (e.g., "gpt-5")
stream_partial: Whether to stream partial output
resume_session: Session ID to resume
"""
def __init__(
self,
workspace: str = ".",
timeout: int = 300,
output_format: str = "json",
force: bool = False,
model: Optional[str] = None,
stream_partial: bool = False,
resume_session: Optional[str] = None,
):
"""
Initialize Cursor CLI integration.
Args:
workspace: Working directory for CLI execution
timeout: Timeout in seconds for CLI execution
output_format: Output format ("json", "text", "stream-json")
force: Whether to allow file modifications (--force)
model: Model to use (e.g., "gpt-5")
stream_partial: Whether to stream partial output (--stream-partial-output)
resume_session: Session ID to resume (--resume)
"""
super().__init__(workspace=workspace, timeout=timeout)
self.output_format = output_format
self.force = force
self.model = model
self.stream_partial = stream_partial
self.resume_session = resume_session
@property
def cli_command(self) -> str:
"""Return the CLI command name."""
return "cursor-agent"
def _build_command(self, prompt: str, **options) -> List[str]:
"""
Build the Cursor CLI command.
Args:
prompt: The prompt to send
**options: Additional options
Returns:
List of command arguments
"""
cmd = ["cursor-agent"]
# Add print mode flag
cmd.append("-p")
# Add force flag if enabled
if self.force:
cmd.append("--force")
# Add model if specified
if self.model:
cmd.extend(["-m", self.model])
# Add output format
cmd.extend(["--output-format", self.output_format])
# Add stream partial flag if enabled
if self.stream_partial:
cmd.append("--stream-partial-output")
# Add resume session if specified
if self.resume_session:
cmd.extend(["--resume", self.resume_session])
# Add prompt (must be last)
cmd.append(prompt)
return cmd
async def execute(self, prompt: str, **options) -> str:
"""
Execute Cursor CLI and return the result.
Args:
prompt: The prompt/query to send
**options: Additional options
Returns:
str: The CLI output (parsed from JSON if output_format is "json")
"""
cmd = self._build_command(prompt, **options)
output = await self.execute_async(cmd)
# Parse JSON output if applicable
if self.output_format == "json":
try:
data = json.loads(output)
# Extract the main result
if isinstance(data, dict):
return data.get("result", data.get("content", str(data)))
return str(data)
except json.JSONDecodeError:
return output
return output
async def stream(self, prompt: str, **options) -> AsyncIterator[Dict[str, Any]]:
"""
Stream output from Cursor CLI.
Args:
prompt: The prompt/query to send
**options: Additional options
Yields:
dict: Parsed JSON events from the CLI
"""
# Use stream-json format for streaming
original_format = self.output_format
original_partial = self.stream_partial
self.output_format = "stream-json"
self.stream_partial = True
try:
cmd = self._build_command(prompt, **options)
async for line in self.stream_async(cmd):
if line.strip():
try:
event = json.loads(line)
yield event
except json.JSONDecodeError:
yield {"type": "text", "content": line}
finally:
self.output_format = original_format
self.stream_partial = original_partial
def get_env(self) -> Dict[str, str]:
"""Get environment variables for CLI execution."""
env = super().get_env()
# Add Cursor API key if available
if "CURSOR_API_KEY" in os.environ:
env["CURSOR_API_KEY"] = os.environ["CURSOR_API_KEY"]
return env
@@ -0,0 +1,222 @@
"""
Gemini CLI Integration.
Provides integration with Google's Gemini CLI for AI-powered coding tasks.
Features:
- Headless mode execution with JSON output
- Model selection (gemini-2.5-pro, gemini-2.5-flash, etc.)
- Multi-directory context support
- Google Search grounding
- Usage statistics
Usage:
from praisonai.integrations import GeminiCLIIntegration
# Create integration
gemini = GeminiCLIIntegration(workspace="/path/to/project")
# Execute a coding task
result = await gemini.execute("Analyze this codebase")
# Get result with stats
result, stats = await gemini.execute_with_stats("Explain the architecture")
# Use as agent tool
tool = gemini.as_tool()
"""
import json
import os
from typing import AsyncIterator, Dict, Any, Optional, List, Tuple
from .base import BaseCLIIntegration
class GeminiCLIIntegration(BaseCLIIntegration):
"""
Integration with Google's Gemini CLI.
Attributes:
output_format: Output format ("json", "text", "stream-json")
model: Gemini model to use
include_directories: Additional directories to include in context
sandbox: Whether to run in sandbox mode
"""
def __init__(
self,
workspace: str = ".",
timeout: int = 300,
output_format: str = "json",
model: str = "gemini-2.5-pro",
include_directories: Optional[List[str]] = None,
sandbox: bool = False,
):
"""
Initialize Gemini CLI integration.
Args:
workspace: Working directory for CLI execution
timeout: Timeout in seconds for CLI execution
output_format: Output format ("json", "text", "stream-json")
model: Gemini model to use (e.g., "gemini-2.5-pro", "gemini-2.5-flash")
include_directories: Additional directories to include in context
sandbox: Whether to run in sandbox mode
"""
super().__init__(workspace=workspace, timeout=timeout)
self.output_format = output_format
self.model = model
self.include_directories = include_directories
self.sandbox = sandbox
# Store last stats for retrieval
self._last_stats: Optional[Dict[str, Any]] = None
@property
def cli_command(self) -> str:
"""Return the CLI command name."""
return "gemini"
def _build_command(self, prompt: str, **options) -> List[str]:
"""
Build the Gemini CLI command.
Args:
prompt: The prompt to send
**options: Additional options
Returns:
List of command arguments
"""
cmd = ["gemini"]
# Add print mode flag
cmd.append("-p")
# Add prompt
cmd.append(prompt)
# Add model
cmd.extend(["-m", self.model])
# Add output format
cmd.extend(["--output-format", self.output_format])
# Add include directories if specified
if self.include_directories:
cmd.extend(["--include-directories", ",".join(self.include_directories)])
# Add sandbox flag if enabled
if self.sandbox:
cmd.append("--sandbox")
return cmd
async def execute(self, prompt: str, **options) -> str:
"""
Execute Gemini CLI and return the result.
Args:
prompt: The prompt/query to send
**options: Additional options
Returns:
str: The CLI output (parsed from JSON if output_format is "json")
"""
cmd = self._build_command(prompt, **options)
output = await self.execute_async(cmd)
# Parse JSON output if applicable
if self.output_format == "json":
try:
data = json.loads(output)
# Store stats for later retrieval
self._last_stats = data.get("stats")
# Extract the main response
return data.get("response", str(data))
except json.JSONDecodeError:
return output
return output
async def execute_with_stats(self, prompt: str, **options) -> Tuple[str, Optional[Dict[str, Any]]]:
"""
Execute Gemini CLI and return both result and usage stats.
Args:
prompt: The prompt/query to send
**options: Additional options
Returns:
Tuple[str, dict]: (result, stats) where stats contains usage information
"""
# Ensure JSON format for stats
original_format = self.output_format
self.output_format = "json"
try:
cmd = self._build_command(prompt, **options)
output = await self.execute_async(cmd)
try:
data = json.loads(output)
response = data.get("response", str(data))
stats = data.get("stats")
return response, stats
except json.JSONDecodeError:
return output, None
finally:
self.output_format = original_format
async def stream(self, prompt: str, **options) -> AsyncIterator[Dict[str, Any]]:
"""
Stream output from Gemini CLI.
Args:
prompt: The prompt/query to send
**options: Additional options
Yields:
dict: Parsed JSON events from the CLI
"""
# Use stream-json format for streaming
original_format = self.output_format
self.output_format = "stream-json"
try:
cmd = self._build_command(prompt, **options)
async for line in self.stream_async(cmd):
if line.strip():
try:
event = json.loads(line)
yield event
except json.JSONDecodeError:
yield {"type": "text", "content": line}
finally:
self.output_format = original_format
def get_last_stats(self) -> Optional[Dict[str, Any]]:
"""
Get the stats from the last execution.
Returns:
dict: Usage statistics or None if not available
"""
return self._last_stats
def get_env(self) -> Dict[str, str]:
"""Get environment variables for CLI execution."""
env = super().get_env()
# Add Google API key if available
if "GOOGLE_API_KEY" in os.environ:
env["GOOGLE_API_KEY"] = os.environ["GOOGLE_API_KEY"]
elif "GEMINI_API_KEY" in os.environ:
# Map GEMINI_API_KEY to GOOGLE_API_KEY
env["GOOGLE_API_KEY"] = os.environ["GEMINI_API_KEY"]
return env
+1 -1
View File
@@ -1 +1 @@
__version__ = "2.3.41"
__version__ = "2.3.44"
+2 -2
View File
@@ -7,12 +7,12 @@
"url": "https://github.com/MervinPraison/PraisonAI",
"source": "github"
},
"version": "2.3.41",
"version": "2.3.42",
"packages": [
{
"registryType": "pypi",
"identifier": "praisonai",
"version": "2.3.41",
"version": "2.3.42",
"transport": {
"type": "stdio"
}
@@ -0,0 +1,182 @@
"""
Integration tests for AutoGenerator and WorkflowAutoGenerator.
These tests require a real OpenAI API key and make actual API calls.
Run with: OPENAI_API_KEY=your-key pytest tests/integration/test_auto_generator_integration.py -v
"""
import pytest
import os
import sys
import tempfile
import yaml
# Add the source path for imports
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', '..'))
# Skip if no API key
if not os.environ.get('OPENAI_API_KEY'):
pytest.skip("OPENAI_API_KEY not set", allow_module_level=True)
try:
from praisonai.auto import AutoGenerator, WorkflowAutoGenerator
except ImportError as e:
pytest.skip(f"Could not import required modules: {e}", allow_module_level=True)
class TestAutoGeneratorIntegration:
"""Integration tests for AutoGenerator with real API calls."""
@pytest.mark.integration
def test_generate_agents_yaml(self):
"""Test generating agents.yaml with real API call."""
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
agent_file = f.name
try:
generator = AutoGenerator(
topic="Create a simple greeting bot",
agent_file=agent_file,
framework="praisonai"
)
result_path = generator.generate()
# Verify file was created
assert os.path.exists(result_path)
# Verify file content
with open(result_path, 'r') as f:
data = yaml.safe_load(f)
# Check structure
assert 'framework' in data
assert 'topic' in data
assert 'roles' in data
assert len(data['roles']) > 0
# Check that tools are preserved (not empty list)
for role_id, role_data in data['roles'].items():
assert 'tools' in role_data
# Tools should be a list (may be empty if LLM didn't suggest any)
assert isinstance(role_data['tools'], list)
print(f"✓ Generated agents.yaml successfully")
print(f" Roles: {list(data['roles'].keys())}")
finally:
if os.path.exists(agent_file):
os.unlink(agent_file)
@pytest.mark.integration
def test_generate_with_merge(self):
"""Test generating and merging agents."""
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
agent_file = f.name
try:
# First generation
generator1 = AutoGenerator(
topic="Create a researcher",
agent_file=agent_file,
framework="praisonai"
)
generator1.generate()
# Second generation with merge
generator2 = AutoGenerator(
topic="Create a writer",
agent_file=agent_file,
framework="praisonai"
)
generator2.generate(merge=True)
# Verify merged content
with open(agent_file, 'r') as f:
data = yaml.safe_load(f)
# Should have roles from both generations
assert len(data['roles']) >= 2
print(f"✓ Merged agents.yaml successfully")
print(f" Total roles: {len(data['roles'])}")
finally:
if os.path.exists(agent_file):
os.unlink(agent_file)
class TestWorkflowAutoGeneratorIntegration:
"""Integration tests for WorkflowAutoGenerator with real API calls."""
@pytest.mark.integration
def test_generate_sequential_workflow(self):
"""Test generating sequential workflow."""
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
workflow_file = f.name
try:
generator = WorkflowAutoGenerator(
topic="Research and summarize AI news",
workflow_file=workflow_file
)
result_path = generator.generate(pattern="sequential")
# Verify file was created
assert os.path.exists(result_path)
# Verify file content
with open(result_path, 'r') as f:
data = yaml.safe_load(f)
# Check structure
assert 'name' in data
assert 'agents' in data
assert 'steps' in data
assert len(data['agents']) > 0
assert len(data['steps']) > 0
print(f"✓ Generated sequential workflow successfully")
print(f" Agents: {list(data['agents'].keys())}")
print(f" Steps: {len(data['steps'])}")
finally:
if os.path.exists(workflow_file):
os.unlink(workflow_file)
@pytest.mark.integration
def test_generate_parallel_workflow(self):
"""Test generating parallel workflow."""
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
workflow_file = f.name
try:
generator = WorkflowAutoGenerator(
topic="Multi-source research on climate change",
workflow_file=workflow_file
)
result_path = generator.generate(pattern="parallel")
# Verify file was created
assert os.path.exists(result_path)
# Verify file content
with open(result_path, 'r') as f:
data = yaml.safe_load(f)
# Check structure
assert 'agents' in data
assert 'steps' in data
print(f"✓ Generated parallel workflow successfully")
print(f" Agents: {list(data['agents'].keys())}")
finally:
if os.path.exists(workflow_file):
os.unlink(workflow_file)
if __name__ == '__main__':
pytest.main([__file__, '-v', '-m', 'integration'])
@@ -0,0 +1,271 @@
"""
Integration tests for CLI tool integrations.
These tests require the actual CLI tools to be installed and API keys to be set.
Run with: pytest tests/integration/test_cli_integrations.py -v
Environment variables required:
- OPENAI_API_KEY: For general testing
- ANTHROPIC_API_KEY or CLAUDE_API_KEY: For Claude Code
- GEMINI_API_KEY or GOOGLE_API_KEY: For Gemini CLI
- CURSOR_API_KEY: For Cursor CLI
"""
import pytest
import os
import asyncio
import sys
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', '..'))
# Skip all tests if no API keys are set
pytestmark = pytest.mark.skipif(
not os.environ.get("OPENAI_API_KEY"),
reason="OPENAI_API_KEY not set"
)
class TestIntegrationAvailability:
"""Test that integrations can be imported and checked."""
def test_import_all_integrations(self):
"""Test importing all integrations."""
from praisonai.integrations import (
BaseCLIIntegration,
ClaudeCodeIntegration,
GeminiCLIIntegration,
CodexCLIIntegration,
CursorCLIIntegration,
)
assert BaseCLIIntegration is not None
assert ClaudeCodeIntegration is not None
assert GeminiCLIIntegration is not None
assert CodexCLIIntegration is not None
assert CursorCLIIntegration is not None
def test_get_available_integrations(self):
"""Test getting available integrations."""
from praisonai.integrations import get_available_integrations
availability = get_available_integrations()
assert isinstance(availability, dict)
assert "claude" in availability
assert "gemini" in availability
assert "codex" in availability
assert "cursor" in availability
print("\n📊 Integration Availability:")
for name, available in availability.items():
status = "✅" if available else "❌"
print(f" {status} {name}")
class TestClaudeCodeIntegrationReal:
"""Real integration tests for Claude Code CLI."""
@pytest.mark.skipif(
not os.environ.get("ANTHROPIC_API_KEY") and not os.environ.get("CLAUDE_API_KEY"),
reason="ANTHROPIC_API_KEY or CLAUDE_API_KEY not set"
)
def test_claude_integration_creation(self):
"""Test creating Claude Code integration."""
from praisonai.integrations import ClaudeCodeIntegration
integration = ClaudeCodeIntegration(
workspace=".",
output_format="json",
skip_permissions=True
)
assert integration.cli_command == "claude"
print(f"\n🔧 Claude CLI available: {integration.is_available}")
@pytest.mark.skipif(
not os.environ.get("ANTHROPIC_API_KEY") and not os.environ.get("CLAUDE_API_KEY"),
reason="ANTHROPIC_API_KEY or CLAUDE_API_KEY not set"
)
def test_claude_as_tool(self):
"""Test Claude Code as agent tool."""
from praisonai.integrations import ClaudeCodeIntegration
integration = ClaudeCodeIntegration()
tool = integration.as_tool()
assert callable(tool)
assert tool.__name__ == "claude_tool"
print(f"\n🔧 Claude tool created: {tool.__name__}")
class TestGeminiCLIIntegrationReal:
"""Real integration tests for Gemini CLI."""
@pytest.mark.skipif(
not os.environ.get("GEMINI_API_KEY") and not os.environ.get("GOOGLE_API_KEY"),
reason="GEMINI_API_KEY or GOOGLE_API_KEY not set"
)
def test_gemini_integration_creation(self):
"""Test creating Gemini CLI integration."""
from praisonai.integrations import GeminiCLIIntegration
integration = GeminiCLIIntegration(
workspace=".",
model="gemini-2.5-pro",
output_format="json"
)
assert integration.cli_command == "gemini"
assert integration.model == "gemini-2.5-pro"
print(f"\n🔧 Gemini CLI available: {integration.is_available}")
@pytest.mark.skipif(
not os.environ.get("GEMINI_API_KEY") and not os.environ.get("GOOGLE_API_KEY"),
reason="GEMINI_API_KEY or GOOGLE_API_KEY not set"
)
def test_gemini_as_tool(self):
"""Test Gemini CLI as agent tool."""
from praisonai.integrations import GeminiCLIIntegration
integration = GeminiCLIIntegration()
tool = integration.as_tool()
assert callable(tool)
assert tool.__name__ == "gemini_tool"
print(f"\n🔧 Gemini tool created: {tool.__name__}")
class TestCursorCLIIntegrationReal:
"""Real integration tests for Cursor CLI."""
@pytest.mark.skipif(
not os.environ.get("CURSOR_API_KEY"),
reason="CURSOR_API_KEY not set"
)
def test_cursor_integration_creation(self):
"""Test creating Cursor CLI integration."""
from praisonai.integrations import CursorCLIIntegration
integration = CursorCLIIntegration(
workspace=".",
output_format="json",
force=False
)
assert integration.cli_command == "cursor-agent"
print(f"\n🔧 Cursor CLI available: {integration.is_available}")
@pytest.mark.skipif(
not os.environ.get("CURSOR_API_KEY"),
reason="CURSOR_API_KEY not set"
)
def test_cursor_as_tool(self):
"""Test Cursor CLI as agent tool."""
from praisonai.integrations import CursorCLIIntegration
integration = CursorCLIIntegration()
tool = integration.as_tool()
assert callable(tool)
assert tool.__name__ == "cursor-agent_tool"
print(f"\n🔧 Cursor tool created: {tool.__name__}")
class TestExternalAgentsHandler:
"""Test the ExternalAgentsHandler."""
def test_handler_creation(self):
"""Test creating the handler."""
from praisonai.cli.features import ExternalAgentsHandler
handler = ExternalAgentsHandler(verbose=True)
assert handler.feature_name == "external_agents"
assert handler.flag_name == "external-agent"
def test_handler_list_integrations(self):
"""Test listing integrations."""
from praisonai.cli.features import ExternalAgentsHandler
handler = ExternalAgentsHandler()
integrations = handler.list_integrations()
assert "claude" in integrations
assert "gemini" in integrations
assert "codex" in integrations
assert "cursor" in integrations
print("\n📋 Available integrations:", integrations)
def test_handler_check_availability(self):
"""Test checking availability."""
from praisonai.cli.features import ExternalAgentsHandler
handler = ExternalAgentsHandler()
availability = handler.check_availability()
print("\n📊 Handler Availability Check:")
for name, available in availability.items():
status = "✅" if available else "❌"
print(f" {status} {name}")
class TestPerformanceImpact:
"""Test that integrations don't impact performance."""
def test_lazy_loading_integrations(self):
"""Test that integrations are lazy loaded."""
import time
# Measure import time
start = time.time()
from praisonai.integrations import BaseCLIIntegration
import_time = time.time() - start
print(f"\n⏱️ Import time: {import_time*1000:.2f}ms")
# Should be fast (< 100ms)
assert import_time < 0.1, f"Import took too long: {import_time}s"
def test_lazy_loading_handler(self):
"""Test that handler is lazy loaded."""
import time
# Measure import time
start = time.time()
from praisonai.cli.features import ExternalAgentsHandler
import_time = time.time() - start
print(f"\n⏱️ Handler import time: {import_time*1000:.2f}ms")
# Should be fast (< 100ms)
assert import_time < 0.1, f"Import took too long: {import_time}s"
def test_availability_caching(self):
"""Test that availability checks are cached."""
import time
from praisonai.integrations import ClaudeCodeIntegration
integration = ClaudeCodeIntegration()
# First check
start = time.time()
_ = integration.is_available
first_time = time.time() - start
# Second check (should be cached)
start = time.time()
_ = integration.is_available
second_time = time.time() - start
print(f"\n⏱️ First availability check: {first_time*1000:.4f}ms")
print(f"⏱️ Second availability check: {second_time*1000:.4f}ms")
# Both should be very fast (< 10ms) due to caching
assert first_time < 0.01, f"First check too slow: {first_time}s"
assert second_time < 0.01, f"Second check too slow: {second_time}s"
if __name__ == "__main__":
pytest.main([__file__, "-v", "-s"])
@@ -0,0 +1 @@
"""Tests for CLI integrations."""
@@ -0,0 +1,295 @@
"""
Tests for BaseCLIIntegration class.
TDD approach: Write tests first, then implement.
"""
import pytest
import asyncio
import json
from unittest.mock import patch, MagicMock, AsyncMock
import sys
import os
# Add the praisonai package to the path
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', '..', '..'))
class TestBaseCLIIntegration:
"""Tests for the BaseCLIIntegration abstract base class."""
def test_import_base_integration(self):
"""Test that BaseCLIIntegration can be imported."""
from praisonai.integrations.base import BaseCLIIntegration
assert BaseCLIIntegration is not None
def test_base_integration_is_abstract(self):
"""Test that BaseCLIIntegration cannot be instantiated directly."""
from praisonai.integrations.base import BaseCLIIntegration
with pytest.raises(TypeError):
BaseCLIIntegration()
def test_concrete_implementation_requires_cli_command(self):
"""Test that concrete implementations must define cli_command."""
from praisonai.integrations.base import BaseCLIIntegration
class IncompleteIntegration(BaseCLIIntegration):
pass
with pytest.raises(TypeError):
IncompleteIntegration()
def test_is_available_returns_false_for_missing_cli(self):
"""Test is_available returns False when CLI is not installed."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "nonexistent_cli_tool_12345"
async def execute(self, prompt: str, **options) -> str:
return ""
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
assert integration.is_available is False
def test_is_available_caches_result(self):
"""Test that is_available caches the availability check."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "nonexistent_cli_12345"
async def execute(self, prompt: str, **options) -> str:
return ""
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
# First call
result1 = integration.is_available
# Second call should use cache
result2 = integration.is_available
assert result1 == result2
def test_workspace_default_is_current_dir(self):
"""Test that default workspace is current directory."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "test"
async def execute(self, prompt: str, **options) -> str:
return ""
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
assert integration.workspace == "."
def test_workspace_can_be_set(self):
"""Test that workspace can be set during initialization."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "test"
async def execute(self, prompt: str, **options) -> str:
return ""
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration(workspace="/custom/path")
assert integration.workspace == "/custom/path"
def test_timeout_default_is_300(self):
"""Test that default timeout is 300 seconds."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "test"
async def execute(self, prompt: str, **options) -> str:
return ""
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
assert integration.timeout == 300
def test_as_tool_returns_callable(self):
"""Test that as_tool returns a callable function."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "test"
async def execute(self, prompt: str, **options) -> str:
return "test result"
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
tool = integration.as_tool()
assert callable(tool)
def test_as_tool_has_correct_name(self):
"""Test that the tool function has the correct name."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "my_cli"
async def execute(self, prompt: str, **options) -> str:
return ""
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
tool = integration.as_tool()
assert tool.__name__ == "my_cli_tool"
def test_as_tool_has_docstring(self):
"""Test that the tool function has a docstring."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "test"
async def execute(self, prompt: str, **options) -> str:
return ""
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
tool = integration.as_tool()
assert tool.__doc__ is not None
assert "test" in tool.__doc__
class TestBaseCLIIntegrationAsync:
"""Async tests for BaseCLIIntegration."""
@pytest.mark.asyncio
async def test_execute_async_runs_command(self):
"""Test that execute_async runs a command."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "echo"
async def execute(self, prompt: str, **options) -> str:
return await self.execute_async(["echo", prompt])
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
result = await integration.execute("hello")
assert "hello" in result
@pytest.mark.asyncio
async def test_execute_async_handles_timeout(self):
"""Test that execute_async handles timeout."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "sleep"
async def execute(self, prompt: str, **options) -> str:
return await self.execute_async(["sleep", "10"], timeout=0.1)
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
with pytest.raises(TimeoutError):
await integration.execute("test")
@pytest.mark.asyncio
async def test_execute_async_captures_stderr(self):
"""Test that execute_async captures stderr."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "bash"
async def execute(self, prompt: str, **options) -> str:
stdout, stderr = await self.execute_async_with_stderr(
["bash", "-c", "echo error >&2"]
)
return stderr
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
result = await integration.execute("test")
assert "error" in result
class TestAvailabilityCache:
"""Tests for the availability caching mechanism."""
def test_availability_cache_is_class_level(self):
"""Test that availability cache is shared across instances."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "shared_test_cli"
async def execute(self, prompt: str, **options) -> str:
return ""
async def stream(self, prompt: str, **options):
yield {}
integration1 = TestIntegration()
integration2 = TestIntegration()
# Access is_available on first instance
_ = integration1.is_available
# Second instance should use cached value
assert "shared_test_cli" in BaseCLIIntegration._availability_cache
def test_clear_availability_cache(self):
"""Test that availability cache can be cleared."""
from praisonai.integrations.base import BaseCLIIntegration
# Clear cache
BaseCLIIntegration._availability_cache.clear()
assert len(BaseCLIIntegration._availability_cache) == 0
@@ -0,0 +1,173 @@
"""
Tests for ClaudeCodeIntegration.
TDD approach: Write tests first, then implement.
"""
import pytest
import json
from unittest.mock import patch, AsyncMock, MagicMock
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', '..', '..'))
class TestClaudeCodeIntegration:
"""Tests for ClaudeCodeIntegration class."""
def test_import_claude_code_integration(self):
"""Test that ClaudeCodeIntegration can be imported."""
from praisonai.integrations.claude_code import ClaudeCodeIntegration
assert ClaudeCodeIntegration is not None
def test_cli_command_is_claude(self):
"""Test that cli_command returns 'claude'."""
from praisonai.integrations.claude_code import ClaudeCodeIntegration
integration = ClaudeCodeIntegration()
assert integration.cli_command == "claude"
def test_inherits_from_base(self):
"""Test that ClaudeCodeIntegration inherits from BaseCLIIntegration."""
from praisonai.integrations.claude_code import ClaudeCodeIntegration
from praisonai.integrations.base import BaseCLIIntegration
integration = ClaudeCodeIntegration()
assert isinstance(integration, BaseCLIIntegration)
def test_default_options(self):
"""Test default options are set correctly."""
from praisonai.integrations.claude_code import ClaudeCodeIntegration
integration = ClaudeCodeIntegration()
assert integration.output_format == "json"
assert integration.skip_permissions is True
def test_custom_options(self):
"""Test custom options can be set."""
from praisonai.integrations.claude_code import ClaudeCodeIntegration
integration = ClaudeCodeIntegration(
output_format="text",
skip_permissions=False,
system_prompt="You are a Python expert"
)
assert integration.output_format == "text"
assert integration.skip_permissions is False
assert integration.system_prompt == "You are a Python expert"
def test_build_command_basic(self):
"""Test building basic command."""
from praisonai.integrations.claude_code import ClaudeCodeIntegration
integration = ClaudeCodeIntegration()
cmd = integration._build_command("Hello")
assert cmd[0] == "claude"
assert "-p" in cmd
assert "Hello" in cmd
assert "--output-format" in cmd
assert "json" in cmd
def test_build_command_with_continue(self):
"""Test building command with continue flag."""
from praisonai.integrations.claude_code import ClaudeCodeIntegration
integration = ClaudeCodeIntegration()
cmd = integration._build_command("Hello", continue_session=True)
assert "--continue" in cmd
def test_build_command_with_system_prompt(self):
"""Test building command with system prompt."""
from praisonai.integrations.claude_code import ClaudeCodeIntegration
integration = ClaudeCodeIntegration(system_prompt="Be helpful")
cmd = integration._build_command("Hello")
assert "--append-system-prompt" in cmd
assert "Be helpful" in cmd
def test_build_command_with_allowed_tools(self):
"""Test building command with allowed tools."""
from praisonai.integrations.claude_code import ClaudeCodeIntegration
integration = ClaudeCodeIntegration(allowed_tools=["Read", "Write"])
cmd = integration._build_command("Hello")
assert "--allowedTools" in cmd
class TestClaudeCodeIntegrationAsync:
"""Async tests for ClaudeCodeIntegration."""
@pytest.mark.asyncio
async def test_execute_parses_json_output(self):
"""Test that execute parses JSON output correctly."""
from praisonai.integrations.claude_code import ClaudeCodeIntegration
integration = ClaudeCodeIntegration()
mock_output = json.dumps({
"result": "Hello, world!",
"cost_usd": 0.001
})
with patch.object(integration, 'execute_async', new_callable=AsyncMock) as mock_exec:
mock_exec.return_value = mock_output
result = await integration.execute("Say hello")
assert "Hello, world!" in result
@pytest.mark.asyncio
async def test_execute_handles_text_output(self):
"""Test that execute handles text output format."""
from praisonai.integrations.claude_code import ClaudeCodeIntegration
integration = ClaudeCodeIntegration(output_format="text")
with patch.object(integration, 'execute_async', new_callable=AsyncMock) as mock_exec:
mock_exec.return_value = "Hello, world!"
result = await integration.execute("Say hello")
assert result == "Hello, world!"
@pytest.mark.asyncio
async def test_stream_yields_events(self):
"""Test that stream yields parsed events."""
from praisonai.integrations.claude_code import ClaudeCodeIntegration
integration = ClaudeCodeIntegration()
async def mock_stream(*args, **kwargs):
yield '{"type": "assistant", "content": "Hello"}'
yield '{"type": "result", "content": "Done"}'
with patch.object(integration, 'stream_async', side_effect=mock_stream):
events = []
async for event in integration.stream("Say hello"):
events.append(event)
assert len(events) >= 0 # May be empty if stream_async not called correctly
class TestClaudeCodeSDKIntegration:
"""Tests for Claude Code SDK integration (when available)."""
def test_sdk_available_property(self):
"""Test sdk_available property."""
from praisonai.integrations.claude_code import ClaudeCodeIntegration
integration = ClaudeCodeIntegration()
# Should be a boolean
assert isinstance(integration.sdk_available, bool)
def test_use_sdk_option(self):
"""Test use_sdk option."""
from praisonai.integrations.claude_code import ClaudeCodeIntegration
integration = ClaudeCodeIntegration(use_sdk=True)
assert integration.use_sdk is True
integration2 = ClaudeCodeIntegration(use_sdk=False)
assert integration2.use_sdk is False
@@ -0,0 +1,154 @@
"""
Tests for CodexCLIIntegration.
TDD approach: Write tests first, then implement.
"""
import pytest
import json
from unittest.mock import patch, AsyncMock
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', '..', '..'))
class TestCodexCLIIntegration:
"""Tests for CodexCLIIntegration class."""
def test_import_codex_cli_integration(self):
"""Test that CodexCLIIntegration can be imported."""
from praisonai.integrations.codex_cli import CodexCLIIntegration
assert CodexCLIIntegration is not None
def test_cli_command_is_codex(self):
"""Test that cli_command returns 'codex'."""
from praisonai.integrations.codex_cli import CodexCLIIntegration
integration = CodexCLIIntegration()
assert integration.cli_command == "codex"
def test_inherits_from_base(self):
"""Test that CodexCLIIntegration inherits from BaseCLIIntegration."""
from praisonai.integrations.codex_cli import CodexCLIIntegration
from praisonai.integrations.base import BaseCLIIntegration
integration = CodexCLIIntegration()
assert isinstance(integration, BaseCLIIntegration)
def test_default_options(self):
"""Test default options are set correctly."""
from praisonai.integrations.codex_cli import CodexCLIIntegration
integration = CodexCLIIntegration()
assert integration.full_auto is False
assert integration.sandbox == "default"
def test_full_auto_option(self):
"""Test full_auto option can be set."""
from praisonai.integrations.codex_cli import CodexCLIIntegration
integration = CodexCLIIntegration(full_auto=True)
assert integration.full_auto is True
def test_sandbox_option(self):
"""Test sandbox option can be set."""
from praisonai.integrations.codex_cli import CodexCLIIntegration
integration = CodexCLIIntegration(sandbox="danger-full-access")
assert integration.sandbox == "danger-full-access"
def test_build_command_basic(self):
"""Test building basic command."""
from praisonai.integrations.codex_cli import CodexCLIIntegration
integration = CodexCLIIntegration()
cmd = integration._build_command("Fix the bug")
assert cmd[0] == "codex"
assert "exec" in cmd
assert "Fix the bug" in cmd
def test_build_command_with_full_auto(self):
"""Test building command with full_auto."""
from praisonai.integrations.codex_cli import CodexCLIIntegration
integration = CodexCLIIntegration(full_auto=True)
cmd = integration._build_command("Fix the bug")
assert "--full-auto" in cmd
def test_build_command_with_json(self):
"""Test building command with JSON output."""
from praisonai.integrations.codex_cli import CodexCLIIntegration
integration = CodexCLIIntegration(json_output=True)
cmd = integration._build_command("Fix the bug")
assert "--json" in cmd
def test_build_command_with_sandbox(self):
"""Test building command with sandbox."""
from praisonai.integrations.codex_cli import CodexCLIIntegration
integration = CodexCLIIntegration(sandbox="danger-full-access")
cmd = integration._build_command("Fix the bug")
assert "--sandbox" in cmd
assert "danger-full-access" in cmd
class TestCodexCLIIntegrationAsync:
"""Async tests for CodexCLIIntegration."""
@pytest.mark.asyncio
async def test_execute_returns_result(self):
"""Test that execute returns the result."""
from praisonai.integrations.codex_cli import CodexCLIIntegration
integration = CodexCLIIntegration()
with patch.object(integration, 'execute_async', new_callable=AsyncMock) as mock_exec:
mock_exec.return_value = "Task completed successfully"
result = await integration.execute("Fix the bug")
assert "Task completed" in result
@pytest.mark.asyncio
async def test_execute_parses_json_events(self):
"""Test that execute parses JSON events when json_output is True."""
from praisonai.integrations.codex_cli import CodexCLIIntegration
integration = CodexCLIIntegration(json_output=True)
mock_output = '\n'.join([
'{"type":"thread.started","thread_id":"123"}',
'{"type":"item.completed","item":{"type":"agent_message","text":"Done"}}',
'{"type":"turn.completed"}'
])
with patch.object(integration, 'execute_async', new_callable=AsyncMock) as mock_exec:
mock_exec.return_value = mock_output
result = await integration.execute("Fix the bug")
assert "Done" in result
class TestCodexCLIStructuredOutput:
"""Tests for Codex CLI structured output."""
def test_output_schema_option(self):
"""Test output_schema option."""
from praisonai.integrations.codex_cli import CodexCLIIntegration
integration = CodexCLIIntegration(output_schema="/path/to/schema.json")
assert integration.output_schema == "/path/to/schema.json"
def test_build_command_with_output_schema(self):
"""Test building command with output schema."""
from praisonai.integrations.codex_cli import CodexCLIIntegration
integration = CodexCLIIntegration(output_schema="/path/to/schema.json")
cmd = integration._build_command("Extract metadata")
assert "--output-schema" in cmd
assert "/path/to/schema.json" in cmd
@@ -0,0 +1,177 @@
"""
Tests for CursorCLIIntegration.
TDD approach: Write tests first, then implement.
"""
import pytest
import json
from unittest.mock import patch, AsyncMock
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', '..', '..'))
class TestCursorCLIIntegration:
"""Tests for CursorCLIIntegration class."""
def test_import_cursor_cli_integration(self):
"""Test that CursorCLIIntegration can be imported."""
from praisonai.integrations.cursor_cli import CursorCLIIntegration
assert CursorCLIIntegration is not None
def test_cli_command_is_cursor_agent(self):
"""Test that cli_command returns 'cursor-agent'."""
from praisonai.integrations.cursor_cli import CursorCLIIntegration
integration = CursorCLIIntegration()
assert integration.cli_command == "cursor-agent"
def test_inherits_from_base(self):
"""Test that CursorCLIIntegration inherits from BaseCLIIntegration."""
from praisonai.integrations.cursor_cli import CursorCLIIntegration
from praisonai.integrations.base import BaseCLIIntegration
integration = CursorCLIIntegration()
assert isinstance(integration, BaseCLIIntegration)
def test_default_options(self):
"""Test default options are set correctly."""
from praisonai.integrations.cursor_cli import CursorCLIIntegration
integration = CursorCLIIntegration()
assert integration.output_format == "json"
assert integration.force is False
def test_force_option(self):
"""Test force option can be set."""
from praisonai.integrations.cursor_cli import CursorCLIIntegration
integration = CursorCLIIntegration(force=True)
assert integration.force is True
def test_model_option(self):
"""Test model option can be set."""
from praisonai.integrations.cursor_cli import CursorCLIIntegration
integration = CursorCLIIntegration(model="gpt-5")
assert integration.model == "gpt-5"
def test_build_command_basic(self):
"""Test building basic command."""
from praisonai.integrations.cursor_cli import CursorCLIIntegration
integration = CursorCLIIntegration()
cmd = integration._build_command("Fix the bug")
assert cmd[0] == "cursor-agent"
assert "-p" in cmd
assert "Fix the bug" in cmd
assert "--output-format" in cmd
assert "json" in cmd
def test_build_command_with_force(self):
"""Test building command with force flag."""
from praisonai.integrations.cursor_cli import CursorCLIIntegration
integration = CursorCLIIntegration(force=True)
cmd = integration._build_command("Fix the bug")
assert "--force" in cmd
def test_build_command_with_model(self):
"""Test building command with model."""
from praisonai.integrations.cursor_cli import CursorCLIIntegration
integration = CursorCLIIntegration(model="gpt-5")
cmd = integration._build_command("Fix the bug")
assert "-m" in cmd
assert "gpt-5" in cmd
def test_build_command_with_stream_partial(self):
"""Test building command with stream partial output."""
from praisonai.integrations.cursor_cli import CursorCLIIntegration
integration = CursorCLIIntegration(stream_partial=True)
cmd = integration._build_command("Fix the bug")
assert "--stream-partial-output" in cmd
class TestCursorCLIIntegrationAsync:
"""Async tests for CursorCLIIntegration."""
@pytest.mark.asyncio
async def test_execute_parses_json_output(self):
"""Test that execute parses JSON output correctly."""
from praisonai.integrations.cursor_cli import CursorCLIIntegration
integration = CursorCLIIntegration()
mock_output = json.dumps({
"result": "Bug fixed successfully"
})
with patch.object(integration, 'execute_async', new_callable=AsyncMock) as mock_exec:
mock_exec.return_value = mock_output
result = await integration.execute("Fix the bug")
assert "Bug fixed" in result
@pytest.mark.asyncio
async def test_execute_handles_text_output(self):
"""Test that execute handles text output format."""
from praisonai.integrations.cursor_cli import CursorCLIIntegration
integration = CursorCLIIntegration(output_format="text")
with patch.object(integration, 'execute_async', new_callable=AsyncMock) as mock_exec:
mock_exec.return_value = "Bug fixed successfully"
result = await integration.execute("Fix the bug")
assert result == "Bug fixed successfully"
class TestCursorCLIEnvironment:
"""Tests for Cursor CLI environment handling."""
def test_get_env_includes_cursor_api_key(self):
"""Test that get_env includes Cursor API key."""
from praisonai.integrations.cursor_cli import CursorCLIIntegration
integration = CursorCLIIntegration()
with patch.dict(os.environ, {"CURSOR_API_KEY": "test-key"}):
env = integration.get_env()
assert "CURSOR_API_KEY" in env
def test_api_key_required_for_headless(self):
"""Test that API key is required for headless mode."""
from praisonai.integrations.cursor_cli import CursorCLIIntegration
integration = CursorCLIIntegration()
# Should not raise, just return env without key if not set
env = integration.get_env()
assert isinstance(env, dict)
class TestCursorCLISession:
"""Tests for Cursor CLI session management."""
def test_resume_option(self):
"""Test resume option."""
from praisonai.integrations.cursor_cli import CursorCLIIntegration
integration = CursorCLIIntegration(resume_session="session-123")
assert integration.resume_session == "session-123"
def test_build_command_with_resume(self):
"""Test building command with resume."""
from praisonai.integrations.cursor_cli import CursorCLIIntegration
integration = CursorCLIIntegration(resume_session="session-123")
cmd = integration._build_command("Continue the task")
assert "--resume" in cmd
assert "session-123" in cmd
@@ -0,0 +1,170 @@
"""
Tests for ExternalAgentsHandler.
TDD approach: Write tests first, then implement.
"""
import pytest
from unittest.mock import patch, MagicMock
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', '..', '..'))
class TestExternalAgentsHandler:
"""Tests for ExternalAgentsHandler class."""
def test_import_external_agents_handler(self):
"""Test that ExternalAgentsHandler can be imported."""
from praisonai.cli.features.external_agents import ExternalAgentsHandler
assert ExternalAgentsHandler is not None
def test_feature_name(self):
"""Test that feature_name is correct."""
from praisonai.cli.features.external_agents import ExternalAgentsHandler
handler = ExternalAgentsHandler()
assert handler.feature_name == "external_agents"
def test_flag_name(self):
"""Test that flag_name is correct."""
from praisonai.cli.features.external_agents import ExternalAgentsHandler
handler = ExternalAgentsHandler()
assert handler.flag_name == "external-agent"
def test_supported_integrations(self):
"""Test that all integrations are supported."""
from praisonai.cli.features.external_agents import ExternalAgentsHandler
handler = ExternalAgentsHandler()
assert "claude" in handler.INTEGRATIONS
assert "gemini" in handler.INTEGRATIONS
assert "codex" in handler.INTEGRATIONS
assert "cursor" in handler.INTEGRATIONS
def test_get_integration_claude(self):
"""Test getting Claude integration."""
from praisonai.cli.features.external_agents import ExternalAgentsHandler
from praisonai.integrations.claude_code import ClaudeCodeIntegration
handler = ExternalAgentsHandler()
integration = handler.get_integration("claude")
assert isinstance(integration, ClaudeCodeIntegration)
def test_get_integration_gemini(self):
"""Test getting Gemini integration."""
from praisonai.cli.features.external_agents import ExternalAgentsHandler
from praisonai.integrations.gemini_cli import GeminiCLIIntegration
handler = ExternalAgentsHandler()
integration = handler.get_integration("gemini")
assert isinstance(integration, GeminiCLIIntegration)
def test_get_integration_codex(self):
"""Test getting Codex integration."""
from praisonai.cli.features.external_agents import ExternalAgentsHandler
from praisonai.integrations.codex_cli import CodexCLIIntegration
handler = ExternalAgentsHandler()
integration = handler.get_integration("codex")
assert isinstance(integration, CodexCLIIntegration)
def test_get_integration_cursor(self):
"""Test getting Cursor integration."""
from praisonai.cli.features.external_agents import ExternalAgentsHandler
from praisonai.integrations.cursor_cli import CursorCLIIntegration
handler = ExternalAgentsHandler()
integration = handler.get_integration("cursor")
assert isinstance(integration, CursorCLIIntegration)
def test_get_integration_invalid(self):
"""Test getting invalid integration raises error."""
from praisonai.cli.features.external_agents import ExternalAgentsHandler
handler = ExternalAgentsHandler()
with pytest.raises(ValueError):
handler.get_integration("invalid")
def test_list_integrations(self):
"""Test listing all integrations."""
from praisonai.cli.features.external_agents import ExternalAgentsHandler
handler = ExternalAgentsHandler()
integrations = handler.list_integrations()
assert len(integrations) == 4
assert all(name in integrations for name in ["claude", "gemini", "codex", "cursor"])
def test_check_availability(self):
"""Test checking integration availability."""
from praisonai.cli.features.external_agents import ExternalAgentsHandler
handler = ExternalAgentsHandler()
availability = handler.check_availability()
assert isinstance(availability, dict)
assert "claude" in availability
assert isinstance(availability["claude"], bool)
class TestExternalAgentsHandlerConfig:
"""Tests for ExternalAgentsHandler configuration."""
def test_apply_to_agent_config_adds_tool(self):
"""Test that apply_to_agent_config adds tool to config."""
from praisonai.cli.features.external_agents import ExternalAgentsHandler
handler = ExternalAgentsHandler()
config = {"tools": []}
# Mock the integration to be available
with patch.object(handler, 'get_integration') as mock_get:
mock_integration = MagicMock()
mock_integration.is_available = True
mock_integration.as_tool.return_value = lambda x: x
mock_get.return_value = mock_integration
result = handler.apply_to_agent_config(config, "claude")
assert len(result["tools"]) == 1
def test_apply_to_agent_config_skips_unavailable(self):
"""Test that apply_to_agent_config skips unavailable integrations."""
from praisonai.cli.features.external_agents import ExternalAgentsHandler
handler = ExternalAgentsHandler()
config = {"tools": []}
with patch.object(handler, 'get_integration') as mock_get:
mock_integration = MagicMock()
mock_integration.is_available = False
mock_get.return_value = mock_integration
result = handler.apply_to_agent_config(config, "claude")
assert len(result["tools"]) == 0
class TestExternalAgentsHandlerExecution:
"""Tests for ExternalAgentsHandler execution."""
def test_execute_returns_integration(self):
"""Test that execute returns the integration."""
from praisonai.cli.features.external_agents import ExternalAgentsHandler
handler = ExternalAgentsHandler()
with patch.object(handler, 'get_integration') as mock_get:
mock_integration = MagicMock()
mock_integration.is_available = True
mock_get.return_value = mock_integration
result = handler.execute(integration_name="claude")
assert result == mock_integration
@@ -0,0 +1,168 @@
"""
Tests for GeminiCLIIntegration.
TDD approach: Write tests first, then implement.
"""
import pytest
import json
from unittest.mock import patch, AsyncMock
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', '..', '..'))
class TestGeminiCLIIntegration:
"""Tests for GeminiCLIIntegration class."""
def test_import_gemini_cli_integration(self):
"""Test that GeminiCLIIntegration can be imported."""
from praisonai.integrations.gemini_cli import GeminiCLIIntegration
assert GeminiCLIIntegration is not None
def test_cli_command_is_gemini(self):
"""Test that cli_command returns 'gemini'."""
from praisonai.integrations.gemini_cli import GeminiCLIIntegration
integration = GeminiCLIIntegration()
assert integration.cli_command == "gemini"
def test_inherits_from_base(self):
"""Test that GeminiCLIIntegration inherits from BaseCLIIntegration."""
from praisonai.integrations.gemini_cli import GeminiCLIIntegration
from praisonai.integrations.base import BaseCLIIntegration
integration = GeminiCLIIntegration()
assert isinstance(integration, BaseCLIIntegration)
def test_default_options(self):
"""Test default options are set correctly."""
from praisonai.integrations.gemini_cli import GeminiCLIIntegration
integration = GeminiCLIIntegration()
assert integration.output_format == "json"
assert integration.model == "gemini-2.5-pro"
def test_custom_model(self):
"""Test custom model can be set."""
from praisonai.integrations.gemini_cli import GeminiCLIIntegration
integration = GeminiCLIIntegration(model="gemini-2.5-flash")
assert integration.model == "gemini-2.5-flash"
def test_build_command_basic(self):
"""Test building basic command."""
from praisonai.integrations.gemini_cli import GeminiCLIIntegration
integration = GeminiCLIIntegration()
cmd = integration._build_command("Hello")
assert cmd[0] == "gemini"
assert "-p" in cmd
assert "Hello" in cmd
assert "--output-format" in cmd
assert "json" in cmd
def test_build_command_with_model(self):
"""Test building command with model."""
from praisonai.integrations.gemini_cli import GeminiCLIIntegration
integration = GeminiCLIIntegration(model="gemini-2.5-flash")
cmd = integration._build_command("Hello")
assert "-m" in cmd
assert "gemini-2.5-flash" in cmd
def test_build_command_with_include_directories(self):
"""Test building command with include directories."""
from praisonai.integrations.gemini_cli import GeminiCLIIntegration
integration = GeminiCLIIntegration(include_directories=["../lib", "../docs"])
cmd = integration._build_command("Hello")
assert "--include-directories" in cmd
class TestGeminiCLIIntegrationAsync:
"""Async tests for GeminiCLIIntegration."""
@pytest.mark.asyncio
async def test_execute_parses_json_output(self):
"""Test that execute parses JSON output correctly."""
from praisonai.integrations.gemini_cli import GeminiCLIIntegration
integration = GeminiCLIIntegration()
mock_output = json.dumps({
"response": "Hello, world!",
"stats": {"models": {}}
})
with patch.object(integration, 'execute_async', new_callable=AsyncMock) as mock_exec:
mock_exec.return_value = mock_output
result = await integration.execute("Say hello")
assert "Hello, world!" in result
@pytest.mark.asyncio
async def test_execute_handles_text_output(self):
"""Test that execute handles text output format."""
from praisonai.integrations.gemini_cli import GeminiCLIIntegration
integration = GeminiCLIIntegration(output_format="text")
with patch.object(integration, 'execute_async', new_callable=AsyncMock) as mock_exec:
mock_exec.return_value = "Hello, world!"
result = await integration.execute("Say hello")
assert result == "Hello, world!"
@pytest.mark.asyncio
async def test_get_stats_returns_usage_info(self):
"""Test that get_stats returns usage information."""
from praisonai.integrations.gemini_cli import GeminiCLIIntegration
integration = GeminiCLIIntegration()
mock_output = json.dumps({
"response": "Hello",
"stats": {
"models": {
"gemini-2.5-pro": {
"tokens": {"prompt": 100, "candidates": 50}
}
}
}
})
with patch.object(integration, 'execute_async', new_callable=AsyncMock) as mock_exec:
mock_exec.return_value = mock_output
result, stats = await integration.execute_with_stats("Say hello")
assert stats is not None
assert "models" in stats
class TestGeminiCLIEnvironment:
"""Tests for Gemini CLI environment handling."""
def test_get_env_includes_google_api_key(self):
"""Test that get_env includes Google API key."""
from praisonai.integrations.gemini_cli import GeminiCLIIntegration
integration = GeminiCLIIntegration()
with patch.dict(os.environ, {"GOOGLE_API_KEY": "test-key"}):
env = integration.get_env()
assert "GOOGLE_API_KEY" in env
def test_get_env_includes_gemini_api_key(self):
"""Test that get_env includes Gemini API key."""
from praisonai.integrations.gemini_cli import GeminiCLIIntegration
integration = GeminiCLIIntegration()
with patch.dict(os.environ, {"GEMINI_API_KEY": "test-key"}):
env = integration.get_env()
# Should map GEMINI_API_KEY to GOOGLE_API_KEY
assert "GOOGLE_API_KEY" in env or "GEMINI_API_KEY" in env
@@ -0,0 +1,743 @@
"""
Unit tests for AutoGenerator and WorkflowAutoGenerator classes.
Tests cover:
- Default model value (gpt-4o-mini)
- Tools preservation (not replaced with [''])
- LiteLLM fallback to OpenAI
- Lazy loading of client
- Timeout and max_retries configuration
"""
import pytest
import os
import sys
from unittest.mock import Mock, patch, MagicMock
import json
import tempfile
# Add the source path for imports
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', '..'))
try:
from praisonai.auto import AutoGenerator, WorkflowAutoGenerator, LITELLM_AVAILABLE
except ImportError as e:
pytest.skip(f"Could not import required modules: {e}", allow_module_level=True)
class TestAutoGeneratorDefaultModel:
"""Test suite for default model configuration."""
def test_default_model_is_gpt4o_mini(self):
"""Test that default model is gpt-4o-mini, not gpt-5-nano."""
# Clear environment variables to test defaults
with patch.dict(os.environ, {}, clear=True):
# Set required API key
os.environ['OPENAI_API_KEY'] = 'test-key'
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
# Check that default model is gpt-4o-mini
assert generator.config_list[0]['model'] == 'gpt-4o-mini'
def test_model_from_environment_variable(self):
"""Test that MODEL_NAME environment variable is respected."""
with patch.dict(os.environ, {'MODEL_NAME': 'custom-model', 'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
assert generator.config_list[0]['model'] == 'custom-model'
def test_openai_model_name_fallback(self):
"""Test that OPENAI_MODEL_NAME is used as fallback."""
with patch.dict(os.environ, {'OPENAI_MODEL_NAME': 'gpt-4', 'OPENAI_API_KEY': 'test-key'}, clear=True):
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
assert generator.config_list[0]['model'] == 'gpt-4'
class TestAutoGeneratorToolsPreservation:
"""Test suite for tools preservation in generated YAML."""
def test_tools_are_preserved_in_convert_and_save(self):
"""Test that generated tools are preserved, not replaced with ['']."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
# Mock JSON data with tools
json_data = {
'roles': {
'researcher': {
'role': 'Researcher',
'goal': 'Research topics',
'backstory': 'Expert researcher',
'tools': ['ScrapeWebsiteTool', 'WebsiteSearchTool'],
'tasks': {
'research_task': {
'description': 'Research the topic',
'expected_output': 'Research report'
}
}
}
}
}
# Use a temporary file
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
generator.agent_file = f.name
try:
generator.convert_and_save(json_data, merge=False)
# Read the saved file
import yaml
with open(generator.agent_file, 'r') as f:
saved_data = yaml.safe_load(f)
# Check that tools are preserved
assert saved_data['roles']['researcher']['tools'] == ['ScrapeWebsiteTool', 'WebsiteSearchTool']
finally:
os.unlink(generator.agent_file)
def test_tools_preserved_in_merge(self):
"""Test that tools are preserved when merging with existing file."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
# Create existing file
existing_data = {
'framework': 'praisonai',
'topic': 'Existing topic',
'roles': {},
'dependencies': []
}
# New JSON data with tools
new_json_data = {
'roles': {
'writer': {
'role': 'Writer',
'goal': 'Write content',
'backstory': 'Expert writer',
'tools': ['FileReadTool', 'TXTSearchTool'],
'tasks': {
'write_task': {
'description': 'Write the content',
'expected_output': 'Written content'
}
}
}
}
}
# Use a temporary file
import yaml
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
yaml.dump(existing_data, f)
generator.agent_file = f.name
try:
generator.convert_and_save(new_json_data, merge=True)
# Read the saved file
with open(generator.agent_file, 'r') as f:
saved_data = yaml.safe_load(f)
# Check that tools are preserved
assert saved_data['roles']['writer']['tools'] == ['FileReadTool', 'TXTSearchTool']
finally:
os.unlink(generator.agent_file)
class TestAutoGeneratorLazyLoading:
"""Test suite for lazy loading of client."""
def test_client_is_not_created_on_init(self):
"""Test that client is not created during __init__."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
# Check that _client is None (lazy loading)
assert generator._client is None
def test_client_is_created_on_first_access(self):
"""Test that client is created on first access."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
# Access client property
with patch('praisonai.auto.instructor') as mock_instructor:
mock_instructor.from_litellm.return_value = Mock()
mock_instructor.patch.return_value = Mock()
client = generator.client
# Check that client is now set
assert generator._client is not None
class TestAutoGeneratorLiteLLMFallback:
"""Test suite for LiteLLM fallback to OpenAI."""
def test_uses_litellm_when_available(self):
"""Test that LiteLLM is used when available via from_provider."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
with patch('praisonai.auto.LITELLM_AVAILABLE', True):
with patch('praisonai.auto.instructor') as mock_instructor:
mock_instructor.from_provider.return_value = Mock()
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
# Access client to trigger creation
_ = generator.client
# Check that from_provider was called with litellm prefix
mock_instructor.from_provider.assert_called_once()
call_args = mock_instructor.from_provider.call_args[0][0]
assert call_args.startswith('litellm/')
def test_falls_back_to_openai_when_litellm_unavailable(self):
"""Test that OpenAI SDK is used when LiteLLM is not available."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
with patch('praisonai.auto.LITELLM_AVAILABLE', False):
with patch('praisonai.auto.instructor') as mock_instructor:
with patch('praisonai.auto.OpenAI') as mock_openai:
mock_instructor.patch.return_value = Mock()
mock_openai.return_value = Mock()
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
# Access client to trigger creation
_ = generator.client
# Check that patch was called (OpenAI fallback)
mock_instructor.patch.assert_called_once()
class TestWorkflowAutoGenerator:
"""Test suite for WorkflowAutoGenerator."""
def test_default_model_is_gpt4o_mini(self):
"""Test that default model is gpt-4o-mini."""
with patch.dict(os.environ, {}, clear=True):
os.environ['OPENAI_API_KEY'] = 'test-key'
generator = WorkflowAutoGenerator(
topic="Test workflow"
)
assert generator.config_list[0]['model'] == 'gpt-4o-mini'
def test_lazy_loading(self):
"""Test that client uses lazy loading."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(
topic="Test workflow"
)
assert generator._client is None
def test_litellm_fallback(self):
"""Test LiteLLM fallback to OpenAI."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
with patch('praisonai.auto.LITELLM_AVAILABLE', False):
with patch('praisonai.auto.instructor') as mock_instructor:
with patch('praisonai.auto.OpenAI') as mock_openai:
mock_instructor.patch.return_value = Mock()
mock_openai.return_value = Mock()
generator = WorkflowAutoGenerator(
topic="Test workflow"
)
# Access client to trigger creation
_ = generator.client
# Check that patch was called (OpenAI fallback)
mock_instructor.patch.assert_called_once()
class TestAutoGeneratorJSONMode:
"""Test suite for JSON mode (critical fix for nested Dict structures)."""
def test_uses_json_mode_not_tools_mode(self):
"""Test that JSON mode is used instead of TOOLS mode for OpenAI fallback.
TOOLS mode doesn't handle nested Dict[str, ...] structures properly,
causing validation errors. JSON mode works correctly.
"""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
with patch('praisonai.auto.LITELLM_AVAILABLE', False):
with patch('praisonai.auto.instructor') as mock_instructor:
with patch('praisonai.auto.OpenAI') as mock_openai:
mock_instructor.patch.return_value = Mock()
mock_instructor.Mode.JSON = 'JSON_MODE'
mock_openai.return_value = Mock()
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
# Access client to trigger creation
_ = generator.client
# Verify JSON mode was used
call_kwargs = mock_instructor.patch.call_args[1]
assert call_kwargs['mode'] == 'JSON_MODE'
def test_workflow_generator_uses_json_mode(self):
"""Test that WorkflowAutoGenerator also uses JSON mode."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
with patch('praisonai.auto.LITELLM_AVAILABLE', False):
with patch('praisonai.auto.instructor') as mock_instructor:
with patch('praisonai.auto.OpenAI') as mock_openai:
mock_instructor.patch.return_value = Mock()
mock_instructor.Mode.JSON = 'JSON_MODE'
mock_openai.return_value = Mock()
generator = WorkflowAutoGenerator(
topic="Test workflow"
)
# Access client to trigger creation
_ = generator.client
# Verify JSON mode was used
call_kwargs = mock_instructor.patch.call_args[1]
assert call_kwargs['mode'] == 'JSON_MODE'
class TestAutoGeneratorImprovedPrompt:
"""Test suite for improved task analysis prompt."""
def test_prompt_includes_complexity_analysis(self):
"""Test that the prompt includes task complexity analysis."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
prompt = generator.get_user_content()
# Check for complexity analysis steps
assert "TASK COMPLEXITY ANALYSIS" in prompt
assert "DETERMINE OPTIMAL TEAM SIZE" in prompt
assert "Simple tasks: 1-2 agents" in prompt
assert "Complex tasks: 3-4 agents" in prompt
def test_prompt_discourages_unnecessary_complexity(self):
"""Test that the prompt discourages unnecessary complexity."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
prompt = generator.get_user_content()
assert "meaningful specialization" in prompt
assert "Avoid unnecessary complexity" in prompt
class TestWorkflowPatterns:
"""Test suite for workflow patterns including new orchestrator-workers and evaluator-optimizer."""
def test_get_prompt_sequential(self):
"""Test that sequential pattern generates correct prompt."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Test topic")
prompt = generator._get_prompt("sequential")
assert "sequential" in prompt.lower()
assert "Sequential Workflow" in prompt
def test_get_prompt_orchestrator_workers(self):
"""Test that orchestrator-workers pattern generates correct prompt."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Test topic")
prompt = generator._get_prompt("orchestrator-workers")
assert "orchestrator" in prompt.lower()
assert "workers" in prompt.lower()
assert "synthesizer" in prompt.lower()
def test_get_prompt_evaluator_optimizer(self):
"""Test that evaluator-optimizer pattern generates correct prompt."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Test topic")
prompt = generator._get_prompt("evaluator-optimizer")
assert "evaluator" in prompt.lower()
assert "generator" in prompt.lower()
assert "loop" in prompt.lower()
assert "APPROVED" in prompt
class TestPatternRecommendation:
"""Test suite for pattern recommendation based on task type."""
def test_recommend_sequential_for_simple_task(self):
"""Test that sequential is recommended for simple tasks."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Write a blog post")
assert generator.recommend_pattern() == "sequential"
def test_recommend_parallel_for_concurrent_tasks(self):
"""Test that parallel is recommended for concurrent tasks."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Research from multiple sources simultaneously")
assert generator.recommend_pattern() == "parallel"
def test_recommend_orchestrator_for_complex_tasks(self):
"""Test that orchestrator-workers is recommended for complex tasks."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Comprehensive analysis and break down the problem")
assert generator.recommend_pattern() == "orchestrator-workers"
def test_recommend_evaluator_for_quality_tasks(self):
"""Test that evaluator-optimizer is recommended for quality-focused tasks."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Refine and improve the content quality")
assert generator.recommend_pattern() == "evaluator-optimizer"
def test_recommend_routing_for_classification_tasks(self):
"""Test that routing is recommended for classification tasks."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Classify and route different types of requests")
assert generator.recommend_pattern() == "routing"
class TestAutoGeneratorConfiguration:
"""Test suite for configuration options."""
def test_custom_config_list(self):
"""Test that custom config_list is respected."""
custom_config = [
{
'model': 'custom-model',
'base_url': 'https://custom.api.com',
'api_key': 'custom-key'
}
]
generator = AutoGenerator(
topic="Test topic",
framework="praisonai",
config_list=custom_config
)
assert generator.config_list == custom_config
def test_base_url_priority(self):
"""Test base_url environment variable priority."""
with patch.dict(os.environ, {
'OPENAI_BASE_URL': 'https://priority.api.com',
'OPENAI_API_BASE': 'https://fallback.api.com',
'OPENAI_API_KEY': 'test-key'
}):
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
assert generator.config_list[0]['base_url'] == 'https://priority.api.com'
class TestBaseAutoGenerator:
"""Test suite for BaseAutoGenerator shared functionality."""
def test_analyze_complexity_simple(self):
"""Test that simple tasks are detected correctly."""
from praisonai.auto import BaseAutoGenerator
simple_tasks = [
"Write a haiku about spring",
"Create a simple summary",
"Write a brief poem",
"Make a quick list"
]
for task in simple_tasks:
assert BaseAutoGenerator.analyze_complexity(task) == 'simple', f"Failed for: {task}"
def test_analyze_complexity_complex(self):
"""Test that complex tasks are detected correctly."""
from praisonai.auto import BaseAutoGenerator
complex_tasks = [
"Comprehensive market analysis and report",
"Research and write a detailed analysis",
"Multi-step process to coordinate teams",
"In-depth investigation of multiple sources"
]
for task in complex_tasks:
assert BaseAutoGenerator.analyze_complexity(task) == 'complex', f"Failed for: {task}"
def test_analyze_complexity_moderate(self):
"""Test that moderate tasks are detected correctly."""
from praisonai.auto import BaseAutoGenerator
moderate_tasks = [
"Research AI trends",
"Analyze customer feedback",
"Design a workflow"
]
for task in moderate_tasks:
assert BaseAutoGenerator.analyze_complexity(task) == 'moderate', f"Failed for: {task}"
def test_get_available_tools(self):
"""Test that available tools list is returned correctly."""
from praisonai.auto import BaseAutoGenerator, AVAILABLE_TOOLS
tools = BaseAutoGenerator.get_available_tools()
# Should return a copy, not the original
assert tools == AVAILABLE_TOOLS
assert tools is not AVAILABLE_TOOLS
# Should contain expected tools
assert "WebsiteSearchTool" in tools
assert "PDFSearchTool" in tools
assert "ScrapeWebsiteTool" in tools
class TestWorkflowDynamicAgentCount:
"""Test suite for dynamic agent count in WorkflowAutoGenerator."""
def test_prompt_includes_complexity_analysis(self):
"""Test that workflow prompt includes complexity analysis."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Research AI trends")
prompt = generator._get_prompt("sequential")
assert "STEP 1: ANALYZE TASK COMPLEXITY" in prompt
assert "STEP 2: DESIGN WORKFLOW" in prompt
assert "STEP 3: ASSIGN TOOLS" in prompt
def test_prompt_includes_tools_list(self):
"""Test that workflow prompt includes available tools."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Research AI trends")
prompt = generator._get_prompt("sequential")
assert "Available Tools:" in prompt
assert "WebsiteSearchTool" in prompt
assert "PDFSearchTool" in prompt
def test_simple_task_gets_fewer_agents_guidance(self):
"""Test that simple tasks get 1-2 agents guidance."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Write a simple haiku")
prompt = generator._get_prompt("sequential")
assert "1-2 agents" in prompt
assert "simple task detected" in prompt
def test_complex_task_gets_more_agents_guidance(self):
"""Test that complex tasks get 3-4 agents guidance."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Comprehensive market analysis and detailed report")
prompt = generator._get_prompt("sequential")
assert "3-4 agents" in prompt
assert "complex task detected" in prompt
# =============================================================================
# TODO 2: Single-Agent Option Tests
# =============================================================================
class TestSingleAgentOption:
"""Test suite for single-agent generation option."""
def test_single_agent_structure_exists(self):
"""Test that SingleAgentStructure Pydantic model exists."""
from praisonai.auto import SingleAgentStructure
assert SingleAgentStructure is not None
def test_auto_generator_has_single_agent_mode(self):
"""Test that AutoGenerator supports single_agent parameter."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Write a haiku",
framework="praisonai",
single_agent=True
)
assert generator.single_agent == True
def test_workflow_generator_has_single_agent_mode(self):
"""Test that WorkflowAutoGenerator supports single_agent parameter."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(
topic="Write a haiku",
single_agent=True
)
assert generator.single_agent == True
# =============================================================================
# TODO 3: LLM-Based Pattern Recommendation Tests
# =============================================================================
class TestLLMPatternRecommendation:
"""Test suite for LLM-based pattern recommendation."""
def test_recommend_pattern_llm_method_exists(self):
"""Test that recommend_pattern_llm method exists."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Test task")
assert hasattr(generator, 'recommend_pattern_llm')
def test_pattern_recommendation_pydantic_model_exists(self):
"""Test that PatternRecommendation Pydantic model exists."""
from praisonai.auto import PatternRecommendation
assert PatternRecommendation is not None
def test_pattern_recommendation_has_required_fields(self):
"""Test PatternRecommendation has pattern, reasoning, confidence."""
from praisonai.auto import PatternRecommendation
fields = PatternRecommendation.model_fields
assert 'pattern' in fields
assert 'reasoning' in fields
assert 'confidence' in fields
# =============================================================================
# TODO 4: Validation Gates Tests
# =============================================================================
class TestValidationGates:
"""Test suite for validation gates in workflows."""
def test_validation_gate_model_exists(self):
"""Test that ValidationGate Pydantic model exists."""
from praisonai.auto import ValidationGate
assert ValidationGate is not None
def test_validation_gate_has_required_fields(self):
"""Test ValidationGate has criteria, pass_action, fail_action."""
from praisonai.auto import ValidationGate
fields = ValidationGate.model_fields
assert 'criteria' in fields
assert 'pass_action' in fields
assert 'fail_action' in fields
def test_workflow_structure_supports_gates(self):
"""Test that WorkflowStructure has optional gates field."""
from praisonai.auto import WorkflowStructure
fields = WorkflowStructure.model_fields
assert 'gates' in fields
# =============================================================================
# TODO 6: Pattern Support for AutoGenerator Tests
# =============================================================================
class TestAutoGeneratorPatternSupport:
"""Test suite for pattern support in AutoGenerator."""
def test_auto_generator_accepts_pattern_parameter(self):
"""Test that AutoGenerator accepts pattern parameter."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Research and write",
framework="praisonai",
pattern="parallel"
)
assert generator.pattern == "parallel"
def test_auto_generator_has_recommend_pattern(self):
"""Test that AutoGenerator has recommend_pattern method."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(topic="Test", framework="praisonai")
assert hasattr(generator, 'recommend_pattern')
def test_auto_generator_prompt_includes_pattern(self):
"""Test that AutoGenerator prompt includes pattern guidance."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Research from multiple sources",
framework="praisonai",
pattern="parallel"
)
prompt = generator.get_user_content()
assert "parallel" in prompt.lower()
# =============================================================================
# TODO 7: Merge Support for WorkflowAutoGenerator Tests
# =============================================================================
class TestWorkflowMergeSupport:
"""Test suite for merge support in WorkflowAutoGenerator."""
def test_workflow_generator_accepts_merge_parameter(self):
"""Test that WorkflowAutoGenerator.generate accepts merge parameter."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Test task")
# Check that generate method accepts merge parameter
import inspect
sig = inspect.signature(generator.generate)
assert 'merge' in sig.parameters
def test_merge_with_existing_workflow_method_exists(self):
"""Test that merge_with_existing_workflow method exists."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Test task")
assert hasattr(generator, 'merge_with_existing_workflow')
# =============================================================================
# TODO 10: Framework Support for WorkflowAutoGenerator Tests
# =============================================================================
class TestWorkflowFrameworkSupport:
"""Test suite for framework support in WorkflowAutoGenerator."""
def test_workflow_generator_accepts_framework_parameter(self):
"""Test that WorkflowAutoGenerator accepts framework parameter."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(
topic="Test task",
framework="crewai"
)
assert generator.framework == "crewai"
def test_workflow_generator_default_framework_is_praisonai(self):
"""Test that default framework is praisonai."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Test task")
assert generator.framework == "praisonai"
def test_save_workflow_respects_framework(self):
"""Test that _save_workflow uses the specified framework."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(
topic="Test task",
framework="crewai"
)
# The framework should be stored and used in output
assert generator.framework == "crewai"
if __name__ == '__main__':
pytest.main([__file__, '-v'])