Compare commits

...
7 Commits
Author SHA1 Message Date
Mervin Praison e609fb7425 Release v2.3.43 2025-12-19 10:32:33 +00:00
Mervin Praison 168ef98473 feat: Implement all remaining AutoAgents improvements (TDD)
TODO 2: Single-Agent Option
- Add SingleAgentStructure Pydantic model
- Add single_agent parameter to AutoGenerator and WorkflowAutoGenerator

TODO 3: LLM-Based Pattern Recommendation
- Add PatternRecommendation Pydantic model (pattern, reasoning, confidence)
- Add recommend_pattern_llm() method to WorkflowAutoGenerator

TODO 4: Validation Gates
- Add ValidationGate Pydantic model (criteria, pass_action, fail_action)
- Add optional gates field to WorkflowStructure

TODO 6: Pattern Support for AutoGenerator
- Add pattern parameter to AutoGenerator
- Add recommend_pattern() method to AutoGenerator
- Add pattern-specific guidance in get_user_content()

TODO 7: Merge Support for WorkflowAutoGenerator
- Add merge parameter to generate() method
- Add merge_with_existing_workflow() method

TODO 10: Framework Support for WorkflowAutoGenerator
- Add framework parameter to WorkflowAutoGenerator
- Default framework is 'praisonai'

Test Coverage:
- 51 tests pass (up from 34)
- 17 new tests added using TDD approach
- All tests verify new functionality
2025-12-19 10:29:40 +00:00
Mervin Praison ce2c2926f9 refactor: Add BaseAutoGenerator class and improve WorkflowAutoGenerator
- Create BaseAutoGenerator with shared functionality (DRY principle)
  - Lazy-loaded instructor client (LiteLLM or OpenAI fallback)
  - Environment variable handling for model/API configuration
  - analyze_complexity() static method for task complexity detection
  - get_available_tools() static method for tool list
  - AVAILABLE_TOOLS constant shared between generators

- Update AutoGenerator to inherit from BaseAutoGenerator
  - Removes ~40 lines of duplicate code
  - Maintains all existing functionality

- Update WorkflowAutoGenerator to inherit from BaseAutoGenerator
  - Removes ~40 lines of duplicate code
  - Add dynamic agent count based on task complexity
  - Add tools list to workflow prompts
  - Improved prompt with 3-step analysis

- Add 8 new unit tests for new functionality
  - TestBaseAutoGenerator: complexity analysis, tools list
  - TestWorkflowDynamicAgentCount: prompt improvements

All 34 tests pass with no performance impact.
2025-12-19 10:23:29 +00:00
Mervin Praison 0a569e2b46 docs: Add AutoAgents workflow patterns examples and update README
- Add autoagents_workflow_patterns.py with all 6 workflow patterns
- Update auto_agents_example.py with new features and CLI examples
- Add orchestrator-workers and evaluator-optimizer patterns to README CLI section
- Update pattern options table with all 6 patterns
- Add documentation links to examples
2025-12-19 09:54:05 +00:00
Mervin Praison 054aea8f66 Add MCP Registry badge and ignore MCP tokens 2025-12-18 22:18:33 +00:00
Mervin Praison 34f1293cfc Release v2.3.42 2025-12-18 22:13:08 +00:00
Mervin Praison 7c1e32afc0 upate 2025-12-18 22:11:02 +00:00
23 changed files with 2422 additions and 123 deletions

No files matched your search

+4 -1
View File
@@ -77,4 +77,7 @@ agents/praisonaiagents/praisonaiagents.egg-info
.qodo
CopilotKit*
.gemini
issue_details.json
issue_details.json
# MCP Registry tokens
.mcpregistry_*
+14 -1
View File
@@ -6,10 +6,13 @@
</picture>
</p>
<!-- mcp-name: io.github.MervinPraison/praisonai -->
<p align="center">
<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://static.pepy.tech/badge/PraisonAI" alt="Total Downloads" /></a>
<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://img.shields.io/github/v/release/MervinPraison/PraisonAI" alt="Latest Stable Version" /></a>
<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License" /></a>
<a href="https://registry.modelcontextprotocol.io/servers/io.github.MervinPraison/praisonai"><img src="https://img.shields.io/badge/MCP-Registry-blue" alt="MCP Registry" /></a>
</p>
<div align="center">
@@ -207,6 +210,7 @@ npm install praisonai
| 🧠 Claude Memory Tool | [Example](#claude-memory-tool-cli) | [📖](https://docs.praison.ai/features/claude-memory-tool) |
| 💾 File-Based Memory | [Example](examples/python/general/memory_example.py) | [📖](https://docs.praison.ai/concepts/memory) |
| 🔍 Built-in Search Tools | [Example](examples/python/agents/websearch-agent.py) | [📖](https://docs.praison.ai/tools/tavily) |
| 🔎 Unified Web Search | [Example](src/praisonai-agents/examples/web_search_example.py) | [📖](https://docs.praison.ai/tools/web-search) |
| 📋 Planning Mode | [Example](examples/python/agents/planning-agent.py) | [📖](https://docs.praison.ai/features/planning-mode) |
| 🔧 Planning Tools | [Example](#3-agent-with-planning-mode) | [📖](https://docs.praison.ai/features/planning-mode) |
| 🧠 Planning Reasoning | [Example](#3-agent-with-planning-mode) | [📖](https://docs.praison.ai/features/planning-mode) |
@@ -1120,6 +1124,15 @@ praisonai workflow auto "Research AI trends" --pattern parallel
# Generate routing workflow (classifier routes to specialists)
praisonai workflow auto "Build a chatbot" --pattern routing
# Generate orchestrator-workers workflow (central orchestrator delegates)
praisonai workflow auto "Comprehensive market analysis" --pattern orchestrator-workers
# Generate evaluator-optimizer workflow (iterative refinement)
praisonai workflow auto "Write and refine article" --pattern evaluator-optimizer
# Specify output file
praisonai workflow auto "Build a chatbot" --pattern routing
# Specify output file
praisonai workflow auto "Research AI" --pattern sequential --output my_workflow.yaml
```
@@ -1129,7 +1142,7 @@ praisonai workflow auto "Research AI" --pattern sequential --output my_workflow.
|------|-------------|
| `--workflow-var key=value` | Set workflow variable (can be repeated) |
| `--var key=value` | Set variable for YAML workflows |
| `--pattern <pattern>` | Pattern for auto-generation (sequential, parallel, routing, loop) |
| `--pattern <pattern>` | Pattern for auto-generation (sequential, parallel, routing, loop, orchestrator-workers, evaluator-optimizer) |
| `--output <file>` | Output file for auto-generation |
| `--llm <model>` | LLM model (e.g., openai/gpt-4o-mini) |
| `--tools <tools>` | Tools (comma-separated, e.g., tavily) |
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.41" \
"praisonai>=2.3.43" \
"praisonai[chat]" \
"embedchain[github,youtube]"
+1 -1
View File
@@ -20,7 +20,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.41" \
"praisonai>=2.3.43" \
"praisonai[ui]" \
"praisonai[chat]" \
"praisonai[realtime]" \
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.41" \
"praisonai>=2.3.43" \
"praisonai[ui]" \
"praisonai[crewai]"
@@ -0,0 +1,132 @@
"""
AutoAgents Workflow Patterns Example
This example demonstrates the new workflow auto-generation features:
- Dynamic agent count based on task complexity
- Multiple workflow patterns (sequential, parallel, routing, orchestrator-workers, evaluator-optimizer)
- Pattern recommendation based on task keywords
Documentation: https://docs.praison.ai/features/autoagents
CLI Reference: https://docs.praison.ai/nocode/auto
"""
from praisonai.auto import AutoGenerator, WorkflowAutoGenerator
import os
import tempfile
# Ensure API key is set
if not os.environ.get("OPENAI_API_KEY"):
print("Please set OPENAI_API_KEY environment variable")
exit(1)
print("=" * 60)
print("AutoAgents Workflow Patterns Demo")
print("=" * 60)
# =============================================================================
# Example 1: Dynamic Agent Count (Simple vs Complex Tasks)
# =============================================================================
print("\n📊 Example 1: Dynamic Agent Count")
print("-" * 40)
# Simple task - should create 1-2 agents
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
simple_file = f.name
simple_generator = AutoGenerator(
topic="Write a haiku about spring",
agent_file=simple_file,
framework="praisonai"
)
print("Simple task: 'Write a haiku about spring'")
result = simple_generator.generate()
print(f"Generated: {result}")
# Complex task - should create 3-4 agents
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
complex_file = f.name
complex_generator = AutoGenerator(
topic="Research AI trends, analyze market data, write a comprehensive report with executive summary",
agent_file=complex_file,
framework="praisonai"
)
print("\nComplex task: 'Research AI trends, analyze market data...'")
result = complex_generator.generate()
print(f"Generated: {result}")
# =============================================================================
# Example 2: Pattern Recommendation
# =============================================================================
print("\n🎯 Example 2: Pattern Recommendation")
print("-" * 40)
test_tasks = [
("Write a blog post", "sequential"),
("Research from multiple sources simultaneously", "parallel"),
("Classify and route customer requests", "routing"),
("Comprehensive analysis and break down the problem", "orchestrator-workers"),
("Refine and improve content quality", "evaluator-optimizer"),
]
for task, expected in test_tasks:
generator = WorkflowAutoGenerator(topic=task)
recommended = generator.recommend_pattern()
status = "✅" if recommended == expected else "⚠️"
print(f"{status} Task: '{task[:40]}...'")
print(f" Recommended: {recommended}")
# =============================================================================
# Example 3: Generate Workflow with Specific Pattern
# =============================================================================
print("\n🔄 Example 3: Generate Workflows with Patterns")
print("-" * 40)
patterns = ["sequential", "parallel", "orchestrator-workers", "evaluator-optimizer"]
for pattern in patterns:
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
workflow_file = f.name
generator = WorkflowAutoGenerator(
topic="Create a market analysis report",
workflow_file=workflow_file
)
print(f"\n📋 Pattern: {pattern}")
result = generator.generate(pattern=pattern)
print(f" Generated: {result}")
# Read and show structure
import yaml
with open(result, 'r') as f:
data = yaml.safe_load(f)
print(f" Agents: {list(data.get('agents', {}).keys())}")
print(f" Steps: {len(data.get('steps', []))}")
# =============================================================================
# Example 4: CLI Usage (for reference)
# =============================================================================
print("\n💻 Example 4: CLI Commands (for reference)")
print("-" * 40)
print("""
# Auto-generate agents (dynamic count)
praisonai --auto "Write a haiku about spring"
# Auto-generate workflow with pattern
praisonai workflow auto "Research and write a report" --pattern sequential
praisonai workflow auto "Research from multiple sources" --pattern parallel
praisonai workflow auto "Comprehensive market analysis" --pattern orchestrator-workers
praisonai workflow auto "Refine content quality" --pattern evaluator-optimizer
# With output file
praisonai workflow auto "Customer support routing" --pattern routing --output support.yaml
""")
print("\n" + "=" * 60)
print("✅ AutoAgents Workflow Patterns Demo Complete!")
print("=" * 60)
print("\nDocumentation: https://docs.praison.ai/features/autoagents")
print("CLI Reference: https://docs.praison.ai/nocode/auto")
+40 -3
View File
@@ -1,11 +1,48 @@
"""
AutoAgents Example
AutoAgents automatically creates and manages AI agents based on high-level instructions.
It features dynamic agent count based on task complexity.
Documentation: https://docs.praison.ai/features/autoagents
CLI Reference: https://docs.praison.ai/nocode/auto
New Features:
- Dynamic agent count (1-4 based on task complexity)
- Workflow patterns: sequential, parallel, routing, orchestrator-workers, evaluator-optimizer
- Pattern recommendation based on task keywords
- Tool preservation from LLM suggestions
"""
from praisonaiagents import AutoAgents
from praisonaiagents.tools import duckduckgo
# Basic usage - AutoAgents analyzes complexity and creates optimal agents
agents = AutoAgents(
instructions="Search for information about AI Agents",
tools=[duckduckgo],
process="sequential",
verbose=True
process="sequential", # or "hierarchical"
verbose=True,
max_agents=3 # Maximum number of agents to create
)
agents.start()
result = agents.start()
print(result)
# =============================================================================
# CLI Usage Examples (for reference)
# =============================================================================
#
# # Auto-generate agents (dynamic count based on complexity)
# praisonai --auto "Write a haiku about spring"
# praisonai --auto "Research AI trends, analyze data, write report"
#
# # Auto-generate workflow with specific pattern
# praisonai workflow auto "Research and write" --pattern sequential
# praisonai workflow auto "Research from multiple sources" --pattern parallel
# praisonai workflow auto "Comprehensive analysis" --pattern orchestrator-workers
# praisonai workflow auto "Refine content quality" --pattern evaluator-optimizer
#
# # With framework selection
# praisonai --framework crewai --auto "Create a movie script"
# praisonai --framework autogen --auto "Create a movie script"
@@ -0,0 +1,90 @@
"""
Unified Web Search Example
This example demonstrates the search_web tool that automatically tries
multiple search providers with fallback support.
Search Provider Priority:
1. Tavily (requires TAVILY_API_KEY + tavily-python)
2. Exa (requires EXA_API_KEY + exa_py)
3. You.com (requires YDC_API_KEY + youdotcom)
4. DuckDuckGo (requires duckduckgo_search package, no API key)
5. SearxNG (requires requests + running SearxNG instance)
Installation:
pip install praisonaiagents duckduckgo_search
Usage:
python web_search_example.py
"""
from praisonaiagents import Agent
from praisonaiagents.tools import search_web, get_available_providers
def main():
print("=" * 60)
print("Unified Web Search - Provider Status")
print("=" * 60)
# Check which providers are available
providers = get_available_providers()
for p in providers:
status = "✓ Available" if p["available"] else f"✗ {p['reason']}"
print(f" {p['name']:12} {status}")
print("\n" + "=" * 60)
print("Example 1: Basic Search")
print("=" * 60)
# Simple search - automatically uses best available provider
results = search_web("Python programming best practices", max_results=3)
if results and "error" not in results[0]:
print(f"\nFound {len(results)} results using {results[0].get('provider', 'unknown')}:\n")
for r in results:
print(f" Title: {r.get('title', 'N/A')[:60]}...")
print(f" URL: {r.get('url', 'N/A')}")
print()
else:
print(f"\nSearch failed: {results}")
print("=" * 60)
print("Example 2: Specify Providers")
print("=" * 60)
# Only try specific providers
results = search_web(
"machine learning tutorials",
max_results=3,
providers=["duckduckgo", "tavily"] # Try these in order
)
if results and "error" not in results[0]:
print(f"\nFound {len(results)} results using {results[0].get('provider', 'unknown')}:\n")
for r in results:
print(f" Title: {r.get('title', 'N/A')[:60]}...")
print(f" URL: {r.get('url', 'N/A')}")
print()
else:
print(f"\nSearch failed: {results}")
print("=" * 60)
print("Example 3: With PraisonAI Agent")
print("=" * 60)
# Use search_web as an agent tool
agent = Agent(
name="SearchAgent",
role="Web Researcher",
goal="Find and summarize information from the web",
instructions="Search the web and provide concise summaries of findings.",
tools=[search_web]
)
result = agent.start("What are the top 3 AI trends in 2025?")
print(f"\nAgent Response:\n{result}")
if __name__ == "__main__":
main()
@@ -218,6 +218,10 @@ TOOL_MAPPINGS = {
'crawl4ai_extract_sync': ('.crawl4ai_tools', None),
'Crawl4AITools': ('.crawl4ai_tools', 'Crawl4AITools'),
'crawl4ai_tools': ('.crawl4ai_tools', None),
# Unified Web Search (auto-fallback across providers)
'search_web': ('.web_search', None),
'get_available_providers': ('.web_search', None),
}
_instances = {} # Cache for class instances
@@ -254,7 +258,8 @@ def __getattr__(name: str) -> Any:
'exa', 'exa_search', 'exa_search_contents', 'exa_find_similar', 'exa_answer',
'exa_search_async', 'exa_search_contents_async', 'exa_answer_async',
'crawl4ai', 'crawl4ai_many', 'crawl4ai_extract', 'crawl4ai_llm_extract',
'crawl4ai_sync', 'crawl4ai_extract_sync'
'crawl4ai_sync', 'crawl4ai_extract_sync',
'search_web', 'get_available_providers'
]:
return getattr(module, name)
if name in ['file_tools', 'pandas_tools', 'wikipedia_tools',
@@ -0,0 +1,329 @@
"""Unified web search tool with automatic fallback across multiple providers.
This module provides a single `search_web` function that automatically tries
multiple search providers in order, falling back to the next if one fails.
Search Provider Priority:
1. Tavily (requires TAVILY_API_KEY + tavily-python)
2. Exa (requires EXA_API_KEY + exa_py)
3. You.com (requires YDC_API_KEY + youdotcom)
4. DuckDuckGo (requires duckduckgo_search package, no API key)
5. SearxNG (requires requests + running SearxNG instance)
Usage:
from praisonaiagents.tools import search_web
# Simple search - automatically uses best available provider
results = search_web("AI news 2024")
# With max results
results = search_web("Python tutorials", max_results=10)
"""
from typing import List, Dict, Any, Optional
import logging
import os
from importlib import util
logger = logging.getLogger(__name__)
def _check_tavily() -> tuple[bool, Optional[str]]:
"""Check if Tavily is available."""
if util.find_spec("tavily") is None:
return False, "tavily-python package not installed"
if not os.environ.get("TAVILY_API_KEY"):
return False, "TAVILY_API_KEY not set"
return True, None
def _check_exa() -> tuple[bool, Optional[str]]:
"""Check if Exa is available."""
if util.find_spec("exa_py") is None:
return False, "exa_py package not installed"
if not os.environ.get("EXA_API_KEY"):
return False, "EXA_API_KEY not set"
return True, None
def _check_youdotcom() -> tuple[bool, Optional[str]]:
"""Check if You.com is available."""
if util.find_spec("youdotcom") is None:
return False, "youdotcom package not installed"
if not os.environ.get("YDC_API_KEY"):
return False, "YDC_API_KEY not set"
return True, None
def _check_duckduckgo() -> tuple[bool, Optional[str]]:
"""Check if DuckDuckGo is available."""
if util.find_spec("duckduckgo_search") is None:
return False, "duckduckgo_search package not installed"
return True, None
def _check_searxng() -> tuple[bool, Optional[str]]:
"""Check if SearxNG is available."""
if util.find_spec("requests") is None:
return False, "requests package not installed"
return True, None
def _search_tavily(query: str, max_results: int = 5) -> List[Dict[str, Any]]:
"""Search using Tavily."""
from tavily import TavilyClient
client = TavilyClient(api_key=os.environ.get("TAVILY_API_KEY"))
response = client.search(query=query, max_results=max_results)
results = []
for r in response.get("results", []):
results.append({
"title": r.get("title", ""),
"url": r.get("url", ""),
"snippet": r.get("content", ""),
"provider": "tavily"
})
return results
def _search_exa(query: str, max_results: int = 5) -> List[Dict[str, Any]]:
"""Search using Exa."""
from exa_py import Exa
client = Exa(os.environ.get("EXA_API_KEY"))
response = client.search(query=query, num_results=max_results)
results = []
for r in response.results:
results.append({
"title": getattr(r, "title", "") or "",
"url": r.url,
"snippet": getattr(r, "text", "") or "",
"provider": "exa"
})
return results
def _search_youdotcom(query: str, max_results: int = 5) -> List[Dict[str, Any]]:
"""Search using You.com."""
from youdotcom import You
client = You(api_key_auth=os.environ.get("YDC_API_KEY"))
response = client.search.unified(query=query, count=max_results)
results = []
# Handle response object
if hasattr(response, 'results'):
web_results = getattr(response.results, 'web', []) or []
for r in web_results[:max_results]:
results.append({
"title": getattr(r, "title", "") or "",
"url": getattr(r, "url", "") or "",
"snippet": getattr(r, "description", "") or "",
"provider": "youdotcom"
})
elif isinstance(response, dict):
web_results = response.get("results", {}).get("web", [])
for r in web_results[:max_results]:
results.append({
"title": r.get("title", ""),
"url": r.get("url", ""),
"snippet": r.get("description", ""),
"provider": "youdotcom"
})
return results
def _search_duckduckgo(query: str, max_results: int = 5) -> List[Dict[str, Any]]:
"""Search using DuckDuckGo."""
from duckduckgo_search import DDGS
results = []
ddgs = DDGS()
for result in ddgs.text(keywords=query, max_results=max_results):
results.append({
"title": result.get("title", ""),
"url": result.get("href", ""),
"snippet": result.get("body", ""),
"provider": "duckduckgo"
})
return results
def _search_searxng(query: str, max_results: int = 5, searxng_url: Optional[str] = None) -> List[Dict[str, Any]]:
"""Search using SearxNG."""
import requests
url = searxng_url or os.environ.get("SEARXNG_URL", "http://localhost:32768/search")
params = {
'q': query,
'format': 'json',
'engines': 'google,bing,duckduckgo',
'safesearch': '1'
}
response = requests.get(url, params=params, timeout=10)
response.raise_for_status()
raw_results = response.json().get('results', [])
results = []
for result in raw_results[:max_results]:
results.append({
"title": result.get("title", ""),
"url": result.get("url", ""),
"snippet": result.get("content", ""),
"provider": "searxng"
})
return results
# Provider configuration: (name, check_func, search_func)
SEARCH_PROVIDERS = [
("tavily", _check_tavily, _search_tavily),
("exa", _check_exa, _search_exa),
("youdotcom", _check_youdotcom, _search_youdotcom),
("duckduckgo", _check_duckduckgo, _search_duckduckgo),
("searxng", _check_searxng, _search_searxng),
]
def search_web(
query: str,
max_results: int = 5,
providers: Optional[List[str]] = None,
searxng_url: Optional[str] = None,
) -> List[Dict[str, Any]]:
"""Search the web using multiple providers with automatic fallback.
Tries each search provider in order until one succeeds. For each provider,
it first checks if the API key is set (if required) and if the package is
installed, then attempts the search. If the search fails, it moves to the
next provider.
Args:
query: The search query string
max_results: Maximum number of results to return (default: 5)
providers: Optional list of provider names to try, in order.
If not specified, uses default order:
["tavily", "exa", "youdotcom", "duckduckgo", "searxng"]
searxng_url: Optional custom SearxNG instance URL
Returns:
List of search results, each containing:
- title: Result title
- url: Result URL
- snippet: Result description/snippet
- provider: Name of the provider that returned the result
If all providers fail, returns a list with a single error dict.
Example:
# Use default provider order
results = search_web("AI news 2024")
# Specify providers to try
results = search_web("Python tutorials", providers=["duckduckgo", "tavily"])
# With custom SearxNG URL
results = search_web("tech news", searxng_url="http://my-searxng:8080/search")
"""
errors = []
# Determine which providers to try
if providers:
# Filter to only valid provider names
provider_order = []
for name in providers:
for pname, check_fn, search_fn in SEARCH_PROVIDERS:
if pname == name.lower():
provider_order.append((pname, check_fn, search_fn))
break
else:
provider_order = SEARCH_PROVIDERS
for provider_name, check_func, search_func in provider_order:
# Step 1: Check if provider is available (API key + package)
is_available, check_error = check_func()
if not is_available:
logger.debug(f"Skipping {provider_name}: {check_error}")
errors.append(f"{provider_name}: {check_error}")
continue
# Step 2: Try to search
try:
logger.debug(f"Trying {provider_name} for query: {query}")
if provider_name == "searxng":
results = search_func(query, max_results, searxng_url)
else:
results = search_func(query, max_results)
if results:
logger.info(f"search_web: Successfully used {provider_name}")
return results
else:
errors.append(f"{provider_name}: No results returned")
except Exception as e:
error_msg = f"{provider_name}: {str(e)}"
logger.debug(f"Search failed with {provider_name}: {e}")
errors.append(error_msg)
continue
# All providers failed
error_summary = "; ".join(errors) if errors else "No search providers available"
logger.error(f"search_web: All providers failed - {error_summary}")
return [{"error": f"All search providers failed: {error_summary}"}]
def get_available_providers() -> List[Dict[str, Any]]:
"""Get list of available search providers and their status.
Returns:
List of dicts with provider info:
- name: Provider name
- available: Whether the provider is ready to use
- reason: Reason if not available
Example:
providers = get_available_providers()
for p in providers:
print(f"{p['name']}: {'✓' if p['available'] else '✗'} {p.get('reason', '')}")
"""
result = []
for name, check_func, _ in SEARCH_PROVIDERS:
is_available, error = check_func()
result.append({
"name": name,
"available": is_available,
"reason": error if not is_available else None
})
return result
if __name__ == "__main__":
# Example usage and provider status check
print("=" * 60)
print("Web Search Tool - Provider Status")
print("=" * 60)
providers = get_available_providers()
for p in providers:
status = "✓ Available" if p["available"] else f"✗ {p['reason']}"
print(f" {p['name']:12} {status}")
print("\n" + "=" * 60)
print("Testing search_web...")
print("=" * 60)
results = search_web("Python programming", max_results=3)
if results and "error" not in results[0]:
print(f"\nFound {len(results)} results using {results[0].get('provider', 'unknown')}:")
for r in results:
print(f"\n Title: {r.get('title', 'N/A')[:50]}...")
print(f" URL: {r.get('url', 'N/A')}")
else:
print(f"\nSearch failed: {results}")
+14 -1
View File
@@ -6,10 +6,13 @@
</picture>
</p>
<!-- mcp-name: io.github.MervinPraison/praisonai -->
<p align="center">
<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://static.pepy.tech/badge/PraisonAI" alt="Total Downloads" /></a>
<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://img.shields.io/github/v/release/MervinPraison/PraisonAI" alt="Latest Stable Version" /></a>
<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License" /></a>
<a href="https://registry.modelcontextprotocol.io/servers/io.github.MervinPraison/praisonai"><img src="https://img.shields.io/badge/MCP-Registry-blue" alt="MCP Registry" /></a>
</p>
<div align="center">
@@ -207,6 +210,7 @@ npm install praisonai
| 🧠 Claude Memory Tool | [Example](#claude-memory-tool-cli) | [📖](https://docs.praison.ai/features/claude-memory-tool) |
| 💾 File-Based Memory | [Example](examples/python/general/memory_example.py) | [📖](https://docs.praison.ai/concepts/memory) |
| 🔍 Built-in Search Tools | [Example](examples/python/agents/websearch-agent.py) | [📖](https://docs.praison.ai/tools/tavily) |
| 🔎 Unified Web Search | [Example](src/praisonai-agents/examples/web_search_example.py) | [📖](https://docs.praison.ai/tools/web-search) |
| 📋 Planning Mode | [Example](examples/python/agents/planning-agent.py) | [📖](https://docs.praison.ai/features/planning-mode) |
| 🔧 Planning Tools | [Example](#3-agent-with-planning-mode) | [📖](https://docs.praison.ai/features/planning-mode) |
| 🧠 Planning Reasoning | [Example](#3-agent-with-planning-mode) | [📖](https://docs.praison.ai/features/planning-mode) |
@@ -1120,6 +1124,15 @@ praisonai workflow auto "Research AI trends" --pattern parallel
# Generate routing workflow (classifier routes to specialists)
praisonai workflow auto "Build a chatbot" --pattern routing
# Generate orchestrator-workers workflow (central orchestrator delegates)
praisonai workflow auto "Comprehensive market analysis" --pattern orchestrator-workers
# Generate evaluator-optimizer workflow (iterative refinement)
praisonai workflow auto "Write and refine article" --pattern evaluator-optimizer
# Specify output file
praisonai workflow auto "Build a chatbot" --pattern routing
# Specify output file
praisonai workflow auto "Research AI" --pattern sequential --output my_workflow.yaml
```
@@ -1129,7 +1142,7 @@ praisonai workflow auto "Research AI" --pattern sequential --output my_workflow.
|------|-------------|
| `--workflow-var key=value` | Set workflow variable (can be repeated) |
| `--var key=value` | Set variable for YAML workflows |
| `--pattern <pattern>` | Pattern for auto-generation (sequential, parallel, routing, loop) |
| `--pattern <pattern>` | Pattern for auto-generation (sequential, parallel, routing, loop, orchestrator-workers, evaluator-optimizer) |
| `--output <file>` | Output file for auto-generation |
| `--llm <model>` | LLM model (e.g., openai/gpt-4o-mini) |
| `--tools <tools>` | Tools (comma-separated, e.g., tavily) |
+2 -2
View File
@@ -3,8 +3,8 @@ class Praisonai < Formula
desc "AI tools for various AI applications"
homepage "https://github.com/MervinPraison/PraisonAI"
url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.41.tar.gz"
sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.41.tar.gz | shasum -a 256`.split.first
url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.43.tar.gz"
sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.43.tar.gz | shasum -a 256`.split.first
license "MIT"
depends_on "[email protected]"
+505 -105
View File
@@ -51,14 +51,140 @@ try:
except ImportError:
PRAISONAI_TOOLS_AVAILABLE = False
# LiteLLM availability check for multi-provider support
LITELLM_AVAILABLE = False
try:
import litellm # noqa: F401 - imported for availability check
LITELLM_AVAILABLE = True
except ImportError:
pass
logging.basicConfig(level=os.environ.get('LOGLEVEL', 'INFO').upper(), format='%(asctime)s - %(levelname)s - %(message)s')
# Define Pydantic models outside of the generate method
# =============================================================================
# Available Tools List (shared between generators)
# =============================================================================
AVAILABLE_TOOLS = [
"CodeDocsSearchTool", "CSVSearchTool", "DirectorySearchTool", "DOCXSearchTool",
"DirectoryReadTool", "FileReadTool", "TXTSearchTool", "JSONSearchTool",
"MDXSearchTool", "PDFSearchTool", "RagTool", "ScrapeElementFromWebsiteTool",
"ScrapeWebsiteTool", "WebsiteSearchTool", "XMLSearchTool",
"YoutubeChannelSearchTool", "YoutubeVideoSearchTool"
]
# =============================================================================
# Base Generator Class (DRY - shared functionality)
# =============================================================================
class BaseAutoGenerator:
"""
Base class for auto-generators with shared functionality.
Provides:
- Lazy-loaded instructor client (LiteLLM or OpenAI fallback)
- Environment variable handling for model/API configuration
- Config list management
"""
def __init__(self, config_list: Optional[List[Dict]] = None):
"""
Initialize base generator with LLM configuration.
Args:
config_list: Optional LLM configuration list
"""
# Support multiple environment variable patterns for better compatibility
model_name = os.environ.get("MODEL_NAME") or os.environ.get("OPENAI_MODEL_NAME", "gpt-4o-mini")
base_url = (
os.environ.get("OPENAI_BASE_URL") or
os.environ.get("OPENAI_API_BASE") or
os.environ.get("OLLAMA_API_BASE", "https://api.openai.com/v1")
)
self.config_list = config_list or [
{
'model': model_name,
'base_url': base_url,
'api_key': os.environ.get("OPENAI_API_KEY")
}
]
self._client = None # Lazy loading for performance
@property
def client(self):
"""Lazy load the instructor client to avoid performance impact.
Uses LiteLLM via instructor.from_provider if available for multi-provider support,
otherwise falls back to direct OpenAI SDK.
"""
if self._client is None:
if LITELLM_AVAILABLE:
# Use LiteLLM for multi-provider support (100+ LLMs)
model_name = self.config_list[0]['model']
self._client = instructor.from_provider(
f"litellm/{model_name}",
mode=instructor.Mode.JSON
)
else:
# Fallback to direct OpenAI SDK
self._client = instructor.patch(
OpenAI(
base_url=self.config_list[0]['base_url'],
api_key=self.config_list[0]['api_key'],
),
mode=instructor.Mode.JSON,
)
return self._client
@staticmethod
def get_available_tools() -> List[str]:
"""Return list of available tools for agent assignment."""
return AVAILABLE_TOOLS.copy()
@staticmethod
def analyze_complexity(topic: str) -> str:
"""
Analyze task complexity based on keywords.
Args:
topic: The task description
Returns:
str: Complexity level - 'simple', 'moderate', or 'complex'
"""
topic_lower = topic.lower()
# Complex task indicators
complex_keywords = [
'comprehensive', 'multi-step', 'analyze and', 'research and write',
'multiple', 'coordinate', 'complex', 'detailed analysis',
'full report', 'in-depth', 'thorough'
]
# Simple task indicators
simple_keywords = [
'write a', 'create a', 'simple', 'quick', 'brief',
'haiku', 'poem', 'summary', 'list', 'single'
]
if any(kw in topic_lower for kw in complex_keywords):
return 'complex'
elif any(kw in topic_lower for kw in simple_keywords):
return 'simple'
else:
return 'moderate'
# =============================================================================
# Pydantic Models for Structured Output
# =============================================================================
class TaskDetails(BaseModel):
"""Details for a single task."""
description: str
expected_output: str
class RoleDetails(BaseModel):
"""Details for a single role/agent."""
role: str
goal: str
backstory: str
@@ -66,14 +192,62 @@ class RoleDetails(BaseModel):
tools: List[str]
class TeamStructure(BaseModel):
"""Structure for multi-agent team."""
roles: Dict[str, RoleDetails]
class AutoGenerator:
def __init__(self, topic="Movie Story writing about AI", agent_file="test.yaml", framework="crewai", config_list: Optional[List[Dict]] = None):
class SingleAgentStructure(BaseModel):
"""Structure for single-agent generation (Anthropic's 'start simple' principle)."""
name: str
role: str
goal: str
backstory: str
instructions: str
tools: List[str] = []
task_description: str
expected_output: str
class PatternRecommendation(BaseModel):
"""LLM-based pattern recommendation with reasoning."""
pattern: str # sequential, parallel, routing, orchestrator-workers, evaluator-optimizer
reasoning: str # Why this pattern was chosen
confidence: float # 0.0 to 1.0 confidence score
class ValidationGate(BaseModel):
"""Validation gate for prompt chaining workflows."""
criteria: str # What to validate
pass_action: str # Action if validation passes (e.g., "continue", "next_step")
fail_action: str # Action if validation fails (e.g., "retry", "escalate", "abort")
class AutoGenerator(BaseAutoGenerator):
"""
Auto-generates agents.yaml files from a topic description.
Inherits from BaseAutoGenerator for shared LLM client functionality.
Usage:
generator = AutoGenerator(framework="crewai", topic="Create a movie script")
path = generator.generate()
"""
def __init__(self, topic="Movie Story writing about AI", agent_file="test.yaml",
framework="crewai", config_list: Optional[List[Dict]] = None,
pattern: str = "sequential", single_agent: bool = False):
"""
Initialize the AutoGenerator class with the specified topic, agent file, and framework.
Args:
topic: The task/topic for agent generation
agent_file: Output YAML file name
framework: Framework to use (crewai, autogen, praisonai)
config_list: Optional LLM configuration
pattern: Workflow pattern (sequential, parallel, routing, orchestrator-workers, evaluator-optimizer)
single_agent: If True, generate a single agent instead of a team
Note: autogen framework is different from this AutoGenerator class.
"""
# Initialize base class first (handles config_list and client)
super().__init__(config_list=config_list)
# Validate framework availability and show framework-specific messages
if framework == "crewai" and not CREWAI_AVAILABLE:
raise ImportError("""
@@ -106,35 +280,42 @@ Tools are not available for {framework}. To use tools, install:
pip install "praisonai[{framework}]"
""")
# Support multiple environment variable patterns for better compatibility
# Priority order: MODEL_NAME > OPENAI_MODEL_NAME for model selection
model_name = os.environ.get("MODEL_NAME") or os.environ.get("OPENAI_MODEL_NAME", "gpt-5-nano")
# Priority order for base_url: OPENAI_BASE_URL > OPENAI_API_BASE > OLLAMA_API_BASE
# OPENAI_BASE_URL is the standard OpenAI SDK environment variable
base_url = (
os.environ.get("OPENAI_BASE_URL") or
os.environ.get("OPENAI_API_BASE") or
os.environ.get("OLLAMA_API_BASE", "https://api.openai.com/v1")
)
self.config_list = config_list or [
{
'model': model_name,
'base_url': base_url,
'api_key': os.environ.get("OPENAI_API_KEY")
}
]
self.topic = topic
self.agent_file = agent_file
self.framework = framework or "praisonai"
self.client = instructor.patch(
OpenAI(
base_url=self.config_list[0]['base_url'],
api_key=self.config_list[0]['api_key'],
),
mode=instructor.Mode.JSON,
)
self.pattern = pattern
self.single_agent = single_agent
def recommend_pattern(self, topic: str = None) -> str:
"""
Recommend the best workflow pattern based on task characteristics.
Args:
topic: The task description (uses self.topic if not provided)
Returns:
str: Recommended pattern name
"""
task = topic or self.topic
task_lower = task.lower()
# Keywords that suggest specific patterns
parallel_keywords = ['multiple', 'concurrent', 'parallel', 'simultaneously', 'different sources', 'compare', 'various']
routing_keywords = ['classify', 'categorize', 'route', 'different types', 'depending on', 'if...then']
orchestrator_keywords = ['complex', 'comprehensive', 'multi-step', 'coordinate', 'delegate', 'break down', 'analyze and']
evaluator_keywords = ['refine', 'improve', 'iterate', 'quality', 'review', 'feedback', 'polish', 'optimize']
# Check for pattern indicators
if any(kw in task_lower for kw in evaluator_keywords):
return "evaluator-optimizer"
elif any(kw in task_lower for kw in orchestrator_keywords):
return "orchestrator-workers"
elif any(kw in task_lower for kw in routing_keywords):
return "routing"
elif any(kw in task_lower for kw in parallel_keywords):
return "parallel"
else:
return "sequential"
def generate(self, merge=False):
"""
@@ -157,7 +338,8 @@ Tools are not available for {framework}. To use tools, install:
response = self.client.chat.completions.create(
model=self.config_list[0]['model'],
response_model=TeamStructure,
max_retries=10,
max_retries=5,
timeout=120.0, # 2 minute timeout for complex generations
messages=[
{"role": "system", "content": "You are a helpful assistant designed to output complex team structures."},
{"role": "user", "content": self.get_user_content()}
@@ -194,8 +376,7 @@ Tools are not available for {framework}. To use tools, install:
"goal": role_details['goal'],
"role": role_details['role'],
"tasks": {},
# "tools": role_details.get('tools', []),
"tools": ['']
"tools": role_details.get('tools', [])
}
for task_id, task_details in role_details['tasks'].items():
@@ -265,7 +446,7 @@ Tools are not available for {framework}. To use tools, install:
"goal": role_details['goal'],
"role": role_details['role'],
"tasks": {},
"tools": ['']
"tools": role_details.get('tools', [])
}
# Add tasks for this role
@@ -292,45 +473,77 @@ Tools are not available for {framework}. To use tools, install:
prompt = generator.get_user_content()
print(prompt)
"""
user_content = """Generate a team structure for \"""" + self.topic + """\" task.
No Input data will be provided to the team.
The team will work in sequence. First role will pass the output to the next role, and so on.
The last role will generate the final output.
Think step by step.
With maximum 3 roles, each with 1 task. Include role goals, backstories, task descriptions, and expected outputs.
List of Available Tools: CodeDocsSearchTool, CSVSearchTool, DirectorySearchTool, DOCXSearchTool, DirectoryReadTool, FileReadTool, TXTSearchTool, JSONSearchTool, MDXSearchTool, PDFSearchTool, RagTool, ScrapeElementFromWebsiteTool, ScrapeWebsiteTool, WebsiteSearchTool, XMLSearchTool, YoutubeChannelSearchTool, YoutubeVideoSearchTool.
Only use Available Tools. Do Not use any other tools.
Example Below:
Use below example to understand the structure of the output.
The final role you create should satisfy the provided task: """ + self.topic + """.
{
"roles": {
"narrative_designer": {
"role": "Narrative Designer",
"goal": "Create AI storylines",
"backstory": "Skilled in narrative development for AI, with a focus on story resonance.",
"tools": ["ScrapeWebsiteTool"],
"tasks": {
"story_concept_development": {
"description": "Craft a unique AI story concept with depth and engagement using concept from this page the content https://www.asthebirdfliesblog.com/posts/how-to-write-book-story-development .",
"expected_output": "Document with narrative arcs, character bios, and settings."
}
}
},
"scriptwriter": {
"role": "Scriptwriter",
"goal": "Write scripts from AI concepts",
"backstory": "Expert in dialogue and script structure, translating concepts into scripts.",
"tasks": {
"scriptwriting_task": {
"description": "Turn narrative concepts into scripts, including dialogue and scenes.",
"expected_output": "Production-ready script with dialogue and scene details."
}
}
}
}
}
"""
# Pattern-specific guidance
pattern_guidance = {
"sequential": "The team will work in sequence. Each role passes output to the next.",
"parallel": "The team will work in parallel on independent subtasks, then combine results.",
"routing": "A classifier agent will route requests to specialized agents based on input type.",
"orchestrator-workers": "A central orchestrator will dynamically delegate tasks to specialized workers.",
"evaluator-optimizer": "One agent generates content, another evaluates it in a loop until quality criteria are met."
}
workflow_guidance = pattern_guidance.get(self.pattern, pattern_guidance["sequential"])
user_content = f"""Analyze and generate a team structure for: "{self.topic}"
STEP 1: TASK COMPLEXITY ANALYSIS
First, analyze the task complexity:
- Is this a simple task that 1-2 agents could handle?
- Does it require multiple specialized perspectives?
- Are there clear sequential dependencies between steps?
STEP 2: DETERMINE OPTIMAL TEAM SIZE
Based on your analysis, create the minimum number of agents needed:
- Simple tasks: 1-2 agents
- Moderate tasks: 2-3 agents
- Complex tasks: 3-4 agents (maximum)
IMPORTANT: Only create agents that provide meaningful specialization. Avoid unnecessary complexity.
STEP 3: DESIGN THE TEAM (Pattern: {self.pattern})
{workflow_guidance}
Each agent should have:
- A clear, distinct role
- A specific goal
- Relevant backstory
- 1 focused task with clear description and expected output
- Appropriate tools (only if needed)
Available Tools: CodeDocsSearchTool, CSVSearchTool, DirectorySearchTool, DOCXSearchTool, DirectoryReadTool, FileReadTool, TXTSearchTool, JSONSearchTool, MDXSearchTool, PDFSearchTool, RagTool, ScrapeElementFromWebsiteTool, ScrapeWebsiteTool, WebsiteSearchTool, XMLSearchTool, YoutubeChannelSearchTool, YoutubeVideoSearchTool.
Only use these tools if the task requires them. Use empty list [] if no tools needed.
Example structure (2 agents for a writing task):
{{
"roles": {{
"researcher": {{
"role": "Research Analyst",
"goal": "Gather comprehensive information on the topic",
"backstory": "Expert researcher skilled at finding and synthesizing information.",
"tools": ["WebsiteSearchTool"],
"tasks": {{
"research_task": {{
"description": "Research key information about the topic and compile findings.",
"expected_output": "Comprehensive research notes with key facts and insights."
}}
}}
}},
"writer": {{
"role": "Content Writer",
"goal": "Create polished final content",
"backstory": "Skilled writer who transforms research into engaging content.",
"tools": [],
"tasks": {{
"writing_task": {{
"description": "Write the final content based on research findings.",
"expected_output": "Polished, well-structured final document."
}}
}}
}}
}}
}}
Now generate the optimal team structure for: {self.topic}
"""
return user_content
@@ -372,13 +585,14 @@ class WorkflowStructure(BaseModel):
description: str
agents: Dict[str, WorkflowAgentDetails]
steps: List[Dict] # Can be agent steps, route, parallel, etc.
gates: Optional[List[ValidationGate]] = None # Optional validation gates
class WorkflowAutoGenerator:
class WorkflowAutoGenerator(BaseAutoGenerator):
"""
Auto-generates workflow.yaml files from a topic description.
Uses lazy loading to avoid performance impact when not used.
Inherits from BaseAutoGenerator for shared LLM client functionality.
Usage:
generator = WorkflowAutoGenerator(topic="Research AI trends and write a report")
@@ -387,7 +601,9 @@ class WorkflowAutoGenerator:
def __init__(self, topic: str = "Research and write about AI",
workflow_file: str = "workflow.yaml",
config_list: Optional[List[Dict]] = None):
config_list: Optional[List[Dict]] = None,
framework: str = "praisonai",
single_agent: bool = False):
"""
Initialize the WorkflowAutoGenerator.
@@ -395,45 +611,105 @@ class WorkflowAutoGenerator:
topic: The task/topic for the workflow
workflow_file: Output file name
config_list: Optional LLM configuration
framework: Framework to use (praisonai, crewai, autogen)
single_agent: If True, generate a single agent workflow
"""
# Support multiple environment variable patterns
model_name = os.environ.get("MODEL_NAME") or os.environ.get("OPENAI_MODEL_NAME", "gpt-4o-mini")
base_url = (
os.environ.get("OPENAI_BASE_URL") or
os.environ.get("OPENAI_API_BASE") or
os.environ.get("OLLAMA_API_BASE", "https://api.openai.com/v1")
)
# Initialize base class (handles config_list and client)
super().__init__(config_list=config_list)
self.config_list = config_list or [
{
'model': model_name,
'base_url': base_url,
'api_key': os.environ.get("OPENAI_API_KEY")
}
]
self.topic = topic
self.workflow_file = workflow_file
self._client = None # Lazy loading
self.framework = framework
self.single_agent = single_agent
@property
def client(self):
"""Lazy load the OpenAI client to avoid performance impact."""
if self._client is None:
self._client = instructor.patch(
OpenAI(
base_url=self.config_list[0]['base_url'],
api_key=self.config_list[0]['api_key'],
),
mode=instructor.Mode.JSON,
)
return self._client
def recommend_pattern(self, topic: str = None) -> str:
"""
Recommend the best workflow pattern based on task characteristics.
Args:
topic: The task description (uses self.topic if not provided)
Returns:
str: Recommended pattern name
Pattern recommendations based on Anthropic's best practices:
- sequential: Clear step-by-step dependencies
- parallel: Independent subtasks that can run concurrently
- routing: Different input types need different handling
- orchestrator-workers: Complex tasks needing dynamic decomposition
- evaluator-optimizer: Tasks requiring iterative refinement
"""
task = topic or self.topic
task_lower = task.lower()
# Keywords that suggest specific patterns
parallel_keywords = ['multiple', 'concurrent', 'parallel', 'simultaneously', 'different sources', 'compare', 'various']
routing_keywords = ['classify', 'categorize', 'route', 'different types', 'depending on', 'if...then']
orchestrator_keywords = ['complex', 'comprehensive', 'multi-step', 'coordinate', 'delegate', 'break down', 'analyze and']
evaluator_keywords = ['refine', 'improve', 'iterate', 'quality', 'review', 'feedback', 'polish', 'optimize']
# Check for pattern indicators
if any(kw in task_lower for kw in evaluator_keywords):
return "evaluator-optimizer"
elif any(kw in task_lower for kw in orchestrator_keywords):
return "orchestrator-workers"
elif any(kw in task_lower for kw in routing_keywords):
return "routing"
elif any(kw in task_lower for kw in parallel_keywords):
return "parallel"
else:
return "sequential"
def generate(self, pattern: str = "sequential") -> str:
def recommend_pattern_llm(self, topic: str = None) -> PatternRecommendation:
"""
Use LLM to recommend the best workflow pattern with reasoning.
Args:
topic: The task description (uses self.topic if not provided)
Returns:
PatternRecommendation: Pattern with reasoning and confidence score
"""
task = topic or self.topic
prompt = f"""Analyze this task and recommend the best workflow pattern:
Task: "{task}"
Available patterns:
1. sequential - Agents work one after another, passing output to the next
2. parallel - Multiple agents work concurrently on independent subtasks
3. routing - A classifier routes requests to specialized agents based on input type
4. orchestrator-workers - Central orchestrator dynamically delegates to specialized workers
5. evaluator-optimizer - Generator creates content, evaluator reviews in a loop until quality met
Respond with:
- pattern: The recommended pattern name
- reasoning: Why this pattern is best for this task
- confidence: Your confidence score (0.0 to 1.0)
"""
response = self.client.chat.completions.create(
model=self.config_list[0]['model'],
response_model=PatternRecommendation,
max_retries=3,
timeout=60.0,
messages=[
{"role": "system", "content": "You are an expert at designing AI agent workflows."},
{"role": "user", "content": prompt}
]
)
return response
def generate(self, pattern: str = "sequential", merge: bool = False) -> str:
"""
Generate a workflow YAML file.
Args:
pattern: Workflow pattern - "sequential", "routing", "parallel", "loop"
pattern: Workflow pattern - "sequential", "routing", "parallel", "loop",
"orchestrator-workers", "evaluator-optimizer"
merge: If True, merge with existing workflow file instead of overwriting
Returns:
Path to the generated workflow file
@@ -442,6 +718,7 @@ class WorkflowAutoGenerator:
model=self.config_list[0]['model'],
response_model=WorkflowStructure,
max_retries=5,
timeout=120.0, # 2 minute timeout for complex generations
messages=[
{"role": "system", "content": "You are a helpful assistant that designs workflow structures."},
{"role": "user", "content": self._get_prompt(pattern)}
@@ -449,16 +726,86 @@ class WorkflowAutoGenerator:
)
json_data = json.loads(response.model_dump_json())
if merge and os.path.exists(self.workflow_file):
return self._save_workflow(self.merge_with_existing_workflow(json_data), pattern)
return self._save_workflow(json_data, pattern)
def merge_with_existing_workflow(self, new_data: Dict) -> Dict:
"""
Merge new workflow data with existing workflow file.
Args:
new_data: The new workflow data to merge
Returns:
Dict: Merged workflow data
"""
try:
with open(self.workflow_file, 'r') as f:
existing_data = yaml.safe_load(f)
if not existing_data:
return new_data
except (yaml.YAMLError, FileNotFoundError) as e:
logging.warning(f"Could not load existing workflow file {self.workflow_file}: {e}")
return new_data
# Merge agents (avoid duplicates)
merged_agents = existing_data.get('agents', {}).copy()
for agent_id, agent_data in new_data.get('agents', {}).items():
# Rename if conflict
final_id = agent_id
counter = 1
while final_id in merged_agents:
final_id = f"{agent_id}_auto_{counter}"
counter += 1
merged_agents[final_id] = agent_data
# Merge steps (append new steps)
merged_steps = existing_data.get('steps', []) + new_data.get('steps', [])
# Create merged structure
merged = {
'name': existing_data.get('name', new_data.get('name', 'Merged Workflow')),
'description': f"{existing_data.get('description', '')} + {new_data.get('description', '')}",
'agents': merged_agents,
'steps': merged_steps
}
return merged
def _get_prompt(self, pattern: str) -> str:
"""Generate the prompt based on the workflow pattern."""
# Analyze complexity to determine agent count
complexity = self.analyze_complexity(self.topic)
if complexity == 'simple':
agent_guidance = "Create 1-2 agents (simple task detected)."
elif complexity == 'complex':
agent_guidance = "Create 3-4 agents (complex task detected)."
else:
agent_guidance = "Create 2-3 agents (moderate task detected)."
# Get available tools
tools_list = ", ".join(self.get_available_tools())
base_prompt = f"""Generate a workflow structure for: "{self.topic}"
STEP 1: ANALYZE TASK COMPLEXITY
- Is this a simple task (1-2 agents)?
- Does it require multiple specialists (2-3 agents)?
- Is it complex with many dependencies (3-4 agents)?
STEP 2: DESIGN WORKFLOW
The workflow should use the "{pattern}" pattern.
Create 2-4 agents with clear roles and instructions.
{agent_guidance}
Each agent should have clear roles and instructions.
Each step should have a clear action.
STEP 3: ASSIGN TOOLS (if needed)
Available Tools: {tools_list}
Only assign tools if the task requires them. Use empty list or null if no tools needed.
"""
if pattern == "routing":
@@ -501,6 +848,59 @@ Example structure:
{"agent": "aggregator", "action": "Combine all findings"}
]
}
"""
elif pattern == "orchestrator-workers":
base_prompt += """
Create an orchestrator-workers workflow where a central orchestrator dynamically delegates tasks to specialized workers.
The orchestrator analyzes the input, decides which workers are needed, and synthesizes results.
Example structure:
{
"name": "Orchestrator-Workers Workflow",
"description": "Central orchestrator delegates to specialized workers",
"agents": {
"orchestrator": {"name": "Orchestrator", "role": "Task Coordinator", "goal": "Analyze tasks and delegate to appropriate workers", "instructions": "Break down the task, identify required specialists, and coordinate their work. Output a JSON with 'subtasks' array listing which workers to invoke."},
"researcher": {"name": "Researcher", "role": "Research Specialist", "goal": "Gather information", "instructions": "Research and provide factual information"},
"analyst": {"name": "Analyst", "role": "Data Analyst", "goal": "Analyze data and patterns", "instructions": "Analyze information and identify insights"},
"writer": {"name": "Writer", "role": "Content Writer", "goal": "Create written content", "instructions": "Write clear, engaging content"},
"synthesizer": {"name": "Synthesizer", "role": "Results Synthesizer", "goal": "Combine all worker outputs", "instructions": "Synthesize all worker outputs into a coherent final result"}
},
"steps": [
{"agent": "orchestrator", "action": "Analyze task and determine required workers: {{input}}"},
{"name": "worker_dispatch", "parallel": [
{"agent": "researcher", "action": "Research: {{input}}"},
{"agent": "analyst", "action": "Analyze: {{input}}"},
{"agent": "writer", "action": "Draft content for: {{input}}"}
]},
{"agent": "synthesizer", "action": "Combine all worker outputs into final result"}
]
}
"""
elif pattern == "evaluator-optimizer":
base_prompt += """
Create an evaluator-optimizer workflow where one agent generates content and another evaluates it in a loop.
The generator improves based on evaluator feedback until quality criteria are met.
Example structure:
{
"name": "Evaluator-Optimizer Workflow",
"description": "Iterative refinement through generation and evaluation",
"agents": {
"generator": {"name": "Generator", "role": "Content Generator", "goal": "Generate high-quality content", "instructions": "Create content based on the input. If feedback is provided, improve the content accordingly."},
"evaluator": {"name": "Evaluator", "role": "Quality Evaluator", "goal": "Evaluate content quality", "instructions": "Evaluate the content on: clarity, accuracy, completeness, and relevance. Score 1-10 for each. If average score < 7, provide specific improvement feedback. If score >= 7, respond with 'APPROVED'."}
},
"steps": [
{"agent": "generator", "action": "Generate initial content for: {{input}}"},
{"name": "evaluation_loop", "loop": {
"agent": "evaluator",
"action": "Evaluate the generated content",
"condition": "output does not contain 'APPROVED'",
"max_iterations": 3,
"feedback_to": "generator"
}},
{"agent": "generator", "action": "Finalize content based on all feedback"}
]
}
"""
else: # sequential
base_prompt += """
@@ -610,7 +610,7 @@ Example:
return None
# Validate pattern
valid_patterns = ["sequential", "routing", "parallel", "loop"]
valid_patterns = ["sequential", "routing", "parallel", "loop", "orchestrator-workers", "evaluator-optimizer"]
if pattern not in valid_patterns:
self.print_status(f"Unknown pattern: {pattern}", "error")
self.print_status(f"Valid patterns: {', '.join(valid_patterns)}", "info")
+1 -1
View File
@@ -2002,7 +2002,7 @@ class PraisonAI:
return
# Validate pattern
valid_patterns = ["sequential", "routing", "parallel", "loop"]
valid_patterns = ["sequential", "routing", "parallel", "loop", "orchestrator-workers", "evaluator-optimizer"]
if pattern not in valid_patterns:
print(f"[red]Unknown pattern: {pattern}[/red]")
print(f"[cyan]Valid patterns: {', '.join(valid_patterns)}[/cyan]")
+1 -1
View File
@@ -57,7 +57,7 @@ class CloudDeployer:
file.write("FROM python:3.11-slim\n")
file.write("WORKDIR /app\n")
file.write("COPY . .\n")
file.write("RUN pip install flask praisonai==2.3.41 gunicorn markdown\n")
file.write("RUN pip install flask praisonai==2.3.43 gunicorn markdown\n")
file.write("EXPOSE 8080\n")
file.write('CMD ["gunicorn", "-b", "0.0.0.0:8080", "api:app"]\n')
@@ -0,0 +1,56 @@
"""
PraisonAI Integrations - External CLI tool integrations.
This module provides integrations with external AI coding CLI tools:
- Claude Code CLI
- Gemini CLI
- OpenAI Codex CLI
- Cursor CLI
All integrations use lazy loading to avoid performance impact.
Usage:
from praisonai.integrations import ClaudeCodeIntegration, GeminiCLIIntegration
# Create integration
claude = ClaudeCodeIntegration(workspace="/path/to/project")
# Use as agent tool
tool = claude.as_tool()
# Or execute directly
result = await claude.execute("Refactor this code")
"""
# Lazy imports to avoid performance impact
__all__ = [
'BaseCLIIntegration',
'ClaudeCodeIntegration',
'GeminiCLIIntegration',
'CodexCLIIntegration',
'CursorCLIIntegration',
'get_available_integrations',
]
def __getattr__(name):
"""Lazy load integrations to minimize import overhead."""
if name == 'BaseCLIIntegration':
from .base import BaseCLIIntegration
return BaseCLIIntegration
elif name == 'ClaudeCodeIntegration':
from .claude_code import ClaudeCodeIntegration
return ClaudeCodeIntegration
elif name == 'GeminiCLIIntegration':
from .gemini_cli import GeminiCLIIntegration
return GeminiCLIIntegration
elif name == 'CodexCLIIntegration':
from .codex_cli import CodexCLIIntegration
return CodexCLIIntegration
elif name == 'CursorCLIIntegration':
from .cursor_cli import CursorCLIIntegration
return CursorCLIIntegration
elif name == 'get_available_integrations':
from .base import get_available_integrations
return get_available_integrations
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+1 -1
View File
@@ -1 +1 @@
__version__ = "2.3.41"
__version__ = "2.3.43"
+2 -2
View File
@@ -7,12 +7,12 @@
"url": "https://github.com/MervinPraison/PraisonAI",
"source": "github"
},
"version": "2.3.41",
"version": "2.3.42",
"packages": [
{
"registryType": "pypi",
"identifier": "praisonai",
"version": "2.3.41",
"version": "2.3.42",
"transport": {
"type": "stdio"
}
@@ -0,0 +1,182 @@
"""
Integration tests for AutoGenerator and WorkflowAutoGenerator.
These tests require a real OpenAI API key and make actual API calls.
Run with: OPENAI_API_KEY=your-key pytest tests/integration/test_auto_generator_integration.py -v
"""
import pytest
import os
import sys
import tempfile
import yaml
# Add the source path for imports
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', '..'))
# Skip if no API key
if not os.environ.get('OPENAI_API_KEY'):
pytest.skip("OPENAI_API_KEY not set", allow_module_level=True)
try:
from praisonai.auto import AutoGenerator, WorkflowAutoGenerator
except ImportError as e:
pytest.skip(f"Could not import required modules: {e}", allow_module_level=True)
class TestAutoGeneratorIntegration:
"""Integration tests for AutoGenerator with real API calls."""
@pytest.mark.integration
def test_generate_agents_yaml(self):
"""Test generating agents.yaml with real API call."""
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
agent_file = f.name
try:
generator = AutoGenerator(
topic="Create a simple greeting bot",
agent_file=agent_file,
framework="praisonai"
)
result_path = generator.generate()
# Verify file was created
assert os.path.exists(result_path)
# Verify file content
with open(result_path, 'r') as f:
data = yaml.safe_load(f)
# Check structure
assert 'framework' in data
assert 'topic' in data
assert 'roles' in data
assert len(data['roles']) > 0
# Check that tools are preserved (not empty list)
for role_id, role_data in data['roles'].items():
assert 'tools' in role_data
# Tools should be a list (may be empty if LLM didn't suggest any)
assert isinstance(role_data['tools'], list)
print(f"✓ Generated agents.yaml successfully")
print(f" Roles: {list(data['roles'].keys())}")
finally:
if os.path.exists(agent_file):
os.unlink(agent_file)
@pytest.mark.integration
def test_generate_with_merge(self):
"""Test generating and merging agents."""
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
agent_file = f.name
try:
# First generation
generator1 = AutoGenerator(
topic="Create a researcher",
agent_file=agent_file,
framework="praisonai"
)
generator1.generate()
# Second generation with merge
generator2 = AutoGenerator(
topic="Create a writer",
agent_file=agent_file,
framework="praisonai"
)
generator2.generate(merge=True)
# Verify merged content
with open(agent_file, 'r') as f:
data = yaml.safe_load(f)
# Should have roles from both generations
assert len(data['roles']) >= 2
print(f"✓ Merged agents.yaml successfully")
print(f" Total roles: {len(data['roles'])}")
finally:
if os.path.exists(agent_file):
os.unlink(agent_file)
class TestWorkflowAutoGeneratorIntegration:
"""Integration tests for WorkflowAutoGenerator with real API calls."""
@pytest.mark.integration
def test_generate_sequential_workflow(self):
"""Test generating sequential workflow."""
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
workflow_file = f.name
try:
generator = WorkflowAutoGenerator(
topic="Research and summarize AI news",
workflow_file=workflow_file
)
result_path = generator.generate(pattern="sequential")
# Verify file was created
assert os.path.exists(result_path)
# Verify file content
with open(result_path, 'r') as f:
data = yaml.safe_load(f)
# Check structure
assert 'name' in data
assert 'agents' in data
assert 'steps' in data
assert len(data['agents']) > 0
assert len(data['steps']) > 0
print(f"✓ Generated sequential workflow successfully")
print(f" Agents: {list(data['agents'].keys())}")
print(f" Steps: {len(data['steps'])}")
finally:
if os.path.exists(workflow_file):
os.unlink(workflow_file)
@pytest.mark.integration
def test_generate_parallel_workflow(self):
"""Test generating parallel workflow."""
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
workflow_file = f.name
try:
generator = WorkflowAutoGenerator(
topic="Multi-source research on climate change",
workflow_file=workflow_file
)
result_path = generator.generate(pattern="parallel")
# Verify file was created
assert os.path.exists(result_path)
# Verify file content
with open(result_path, 'r') as f:
data = yaml.safe_load(f)
# Check structure
assert 'agents' in data
assert 'steps' in data
print(f"✓ Generated parallel workflow successfully")
print(f" Agents: {list(data['agents'].keys())}")
finally:
if os.path.exists(workflow_file):
os.unlink(workflow_file)
if __name__ == '__main__':
pytest.main([__file__, '-v', '-m', 'integration'])
@@ -0,0 +1 @@
"""Tests for CLI integrations."""
@@ -0,0 +1,295 @@
"""
Tests for BaseCLIIntegration class.
TDD approach: Write tests first, then implement.
"""
import pytest
import asyncio
import json
from unittest.mock import patch, MagicMock, AsyncMock
import sys
import os
# Add the praisonai package to the path
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', '..', '..'))
class TestBaseCLIIntegration:
"""Tests for the BaseCLIIntegration abstract base class."""
def test_import_base_integration(self):
"""Test that BaseCLIIntegration can be imported."""
from praisonai.integrations.base import BaseCLIIntegration
assert BaseCLIIntegration is not None
def test_base_integration_is_abstract(self):
"""Test that BaseCLIIntegration cannot be instantiated directly."""
from praisonai.integrations.base import BaseCLIIntegration
with pytest.raises(TypeError):
BaseCLIIntegration()
def test_concrete_implementation_requires_cli_command(self):
"""Test that concrete implementations must define cli_command."""
from praisonai.integrations.base import BaseCLIIntegration
class IncompleteIntegration(BaseCLIIntegration):
pass
with pytest.raises(TypeError):
IncompleteIntegration()
def test_is_available_returns_false_for_missing_cli(self):
"""Test is_available returns False when CLI is not installed."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "nonexistent_cli_tool_12345"
async def execute(self, prompt: str, **options) -> str:
return ""
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
assert integration.is_available is False
def test_is_available_caches_result(self):
"""Test that is_available caches the availability check."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "nonexistent_cli_12345"
async def execute(self, prompt: str, **options) -> str:
return ""
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
# First call
result1 = integration.is_available
# Second call should use cache
result2 = integration.is_available
assert result1 == result2
def test_workspace_default_is_current_dir(self):
"""Test that default workspace is current directory."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "test"
async def execute(self, prompt: str, **options) -> str:
return ""
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
assert integration.workspace == "."
def test_workspace_can_be_set(self):
"""Test that workspace can be set during initialization."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "test"
async def execute(self, prompt: str, **options) -> str:
return ""
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration(workspace="/custom/path")
assert integration.workspace == "/custom/path"
def test_timeout_default_is_300(self):
"""Test that default timeout is 300 seconds."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "test"
async def execute(self, prompt: str, **options) -> str:
return ""
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
assert integration.timeout == 300
def test_as_tool_returns_callable(self):
"""Test that as_tool returns a callable function."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "test"
async def execute(self, prompt: str, **options) -> str:
return "test result"
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
tool = integration.as_tool()
assert callable(tool)
def test_as_tool_has_correct_name(self):
"""Test that the tool function has the correct name."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "my_cli"
async def execute(self, prompt: str, **options) -> str:
return ""
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
tool = integration.as_tool()
assert tool.__name__ == "my_cli_tool"
def test_as_tool_has_docstring(self):
"""Test that the tool function has a docstring."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "test"
async def execute(self, prompt: str, **options) -> str:
return ""
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
tool = integration.as_tool()
assert tool.__doc__ is not None
assert "test" in tool.__doc__
class TestBaseCLIIntegrationAsync:
"""Async tests for BaseCLIIntegration."""
@pytest.mark.asyncio
async def test_execute_async_runs_command(self):
"""Test that execute_async runs a command."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "echo"
async def execute(self, prompt: str, **options) -> str:
return await self.execute_async(["echo", prompt])
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
result = await integration.execute("hello")
assert "hello" in result
@pytest.mark.asyncio
async def test_execute_async_handles_timeout(self):
"""Test that execute_async handles timeout."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "sleep"
async def execute(self, prompt: str, **options) -> str:
return await self.execute_async(["sleep", "10"], timeout=0.1)
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
with pytest.raises(TimeoutError):
await integration.execute("test")
@pytest.mark.asyncio
async def test_execute_async_captures_stderr(self):
"""Test that execute_async captures stderr."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "bash"
async def execute(self, prompt: str, **options) -> str:
stdout, stderr = await self.execute_async_with_stderr(
["bash", "-c", "echo error >&2"]
)
return stderr
async def stream(self, prompt: str, **options):
yield {}
integration = TestIntegration()
result = await integration.execute("test")
assert "error" in result
class TestAvailabilityCache:
"""Tests for the availability caching mechanism."""
def test_availability_cache_is_class_level(self):
"""Test that availability cache is shared across instances."""
from praisonai.integrations.base import BaseCLIIntegration
class TestIntegration(BaseCLIIntegration):
@property
def cli_command(self) -> str:
return "shared_test_cli"
async def execute(self, prompt: str, **options) -> str:
return ""
async def stream(self, prompt: str, **options):
yield {}
integration1 = TestIntegration()
integration2 = TestIntegration()
# Access is_available on first instance
_ = integration1.is_available
# Second instance should use cached value
assert "shared_test_cli" in BaseCLIIntegration._availability_cache
def test_clear_availability_cache(self):
"""Test that availability cache can be cleared."""
from praisonai.integrations.base import BaseCLIIntegration
# Clear cache
BaseCLIIntegration._availability_cache.clear()
assert len(BaseCLIIntegration._availability_cache) == 0
@@ -0,0 +1,743 @@
"""
Unit tests for AutoGenerator and WorkflowAutoGenerator classes.
Tests cover:
- Default model value (gpt-4o-mini)
- Tools preservation (not replaced with [''])
- LiteLLM fallback to OpenAI
- Lazy loading of client
- Timeout and max_retries configuration
"""
import pytest
import os
import sys
from unittest.mock import Mock, patch, MagicMock
import json
import tempfile
# Add the source path for imports
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', '..'))
try:
from praisonai.auto import AutoGenerator, WorkflowAutoGenerator, LITELLM_AVAILABLE
except ImportError as e:
pytest.skip(f"Could not import required modules: {e}", allow_module_level=True)
class TestAutoGeneratorDefaultModel:
"""Test suite for default model configuration."""
def test_default_model_is_gpt4o_mini(self):
"""Test that default model is gpt-4o-mini, not gpt-5-nano."""
# Clear environment variables to test defaults
with patch.dict(os.environ, {}, clear=True):
# Set required API key
os.environ['OPENAI_API_KEY'] = 'test-key'
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
# Check that default model is gpt-4o-mini
assert generator.config_list[0]['model'] == 'gpt-4o-mini'
def test_model_from_environment_variable(self):
"""Test that MODEL_NAME environment variable is respected."""
with patch.dict(os.environ, {'MODEL_NAME': 'custom-model', 'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
assert generator.config_list[0]['model'] == 'custom-model'
def test_openai_model_name_fallback(self):
"""Test that OPENAI_MODEL_NAME is used as fallback."""
with patch.dict(os.environ, {'OPENAI_MODEL_NAME': 'gpt-4', 'OPENAI_API_KEY': 'test-key'}, clear=True):
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
assert generator.config_list[0]['model'] == 'gpt-4'
class TestAutoGeneratorToolsPreservation:
"""Test suite for tools preservation in generated YAML."""
def test_tools_are_preserved_in_convert_and_save(self):
"""Test that generated tools are preserved, not replaced with ['']."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
# Mock JSON data with tools
json_data = {
'roles': {
'researcher': {
'role': 'Researcher',
'goal': 'Research topics',
'backstory': 'Expert researcher',
'tools': ['ScrapeWebsiteTool', 'WebsiteSearchTool'],
'tasks': {
'research_task': {
'description': 'Research the topic',
'expected_output': 'Research report'
}
}
}
}
}
# Use a temporary file
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
generator.agent_file = f.name
try:
generator.convert_and_save(json_data, merge=False)
# Read the saved file
import yaml
with open(generator.agent_file, 'r') as f:
saved_data = yaml.safe_load(f)
# Check that tools are preserved
assert saved_data['roles']['researcher']['tools'] == ['ScrapeWebsiteTool', 'WebsiteSearchTool']
finally:
os.unlink(generator.agent_file)
def test_tools_preserved_in_merge(self):
"""Test that tools are preserved when merging with existing file."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
# Create existing file
existing_data = {
'framework': 'praisonai',
'topic': 'Existing topic',
'roles': {},
'dependencies': []
}
# New JSON data with tools
new_json_data = {
'roles': {
'writer': {
'role': 'Writer',
'goal': 'Write content',
'backstory': 'Expert writer',
'tools': ['FileReadTool', 'TXTSearchTool'],
'tasks': {
'write_task': {
'description': 'Write the content',
'expected_output': 'Written content'
}
}
}
}
}
# Use a temporary file
import yaml
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
yaml.dump(existing_data, f)
generator.agent_file = f.name
try:
generator.convert_and_save(new_json_data, merge=True)
# Read the saved file
with open(generator.agent_file, 'r') as f:
saved_data = yaml.safe_load(f)
# Check that tools are preserved
assert saved_data['roles']['writer']['tools'] == ['FileReadTool', 'TXTSearchTool']
finally:
os.unlink(generator.agent_file)
class TestAutoGeneratorLazyLoading:
"""Test suite for lazy loading of client."""
def test_client_is_not_created_on_init(self):
"""Test that client is not created during __init__."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
# Check that _client is None (lazy loading)
assert generator._client is None
def test_client_is_created_on_first_access(self):
"""Test that client is created on first access."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
# Access client property
with patch('praisonai.auto.instructor') as mock_instructor:
mock_instructor.from_litellm.return_value = Mock()
mock_instructor.patch.return_value = Mock()
client = generator.client
# Check that client is now set
assert generator._client is not None
class TestAutoGeneratorLiteLLMFallback:
"""Test suite for LiteLLM fallback to OpenAI."""
def test_uses_litellm_when_available(self):
"""Test that LiteLLM is used when available via from_provider."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
with patch('praisonai.auto.LITELLM_AVAILABLE', True):
with patch('praisonai.auto.instructor') as mock_instructor:
mock_instructor.from_provider.return_value = Mock()
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
# Access client to trigger creation
_ = generator.client
# Check that from_provider was called with litellm prefix
mock_instructor.from_provider.assert_called_once()
call_args = mock_instructor.from_provider.call_args[0][0]
assert call_args.startswith('litellm/')
def test_falls_back_to_openai_when_litellm_unavailable(self):
"""Test that OpenAI SDK is used when LiteLLM is not available."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
with patch('praisonai.auto.LITELLM_AVAILABLE', False):
with patch('praisonai.auto.instructor') as mock_instructor:
with patch('praisonai.auto.OpenAI') as mock_openai:
mock_instructor.patch.return_value = Mock()
mock_openai.return_value = Mock()
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
# Access client to trigger creation
_ = generator.client
# Check that patch was called (OpenAI fallback)
mock_instructor.patch.assert_called_once()
class TestWorkflowAutoGenerator:
"""Test suite for WorkflowAutoGenerator."""
def test_default_model_is_gpt4o_mini(self):
"""Test that default model is gpt-4o-mini."""
with patch.dict(os.environ, {}, clear=True):
os.environ['OPENAI_API_KEY'] = 'test-key'
generator = WorkflowAutoGenerator(
topic="Test workflow"
)
assert generator.config_list[0]['model'] == 'gpt-4o-mini'
def test_lazy_loading(self):
"""Test that client uses lazy loading."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(
topic="Test workflow"
)
assert generator._client is None
def test_litellm_fallback(self):
"""Test LiteLLM fallback to OpenAI."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
with patch('praisonai.auto.LITELLM_AVAILABLE', False):
with patch('praisonai.auto.instructor') as mock_instructor:
with patch('praisonai.auto.OpenAI') as mock_openai:
mock_instructor.patch.return_value = Mock()
mock_openai.return_value = Mock()
generator = WorkflowAutoGenerator(
topic="Test workflow"
)
# Access client to trigger creation
_ = generator.client
# Check that patch was called (OpenAI fallback)
mock_instructor.patch.assert_called_once()
class TestAutoGeneratorJSONMode:
"""Test suite for JSON mode (critical fix for nested Dict structures)."""
def test_uses_json_mode_not_tools_mode(self):
"""Test that JSON mode is used instead of TOOLS mode for OpenAI fallback.
TOOLS mode doesn't handle nested Dict[str, ...] structures properly,
causing validation errors. JSON mode works correctly.
"""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
with patch('praisonai.auto.LITELLM_AVAILABLE', False):
with patch('praisonai.auto.instructor') as mock_instructor:
with patch('praisonai.auto.OpenAI') as mock_openai:
mock_instructor.patch.return_value = Mock()
mock_instructor.Mode.JSON = 'JSON_MODE'
mock_openai.return_value = Mock()
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
# Access client to trigger creation
_ = generator.client
# Verify JSON mode was used
call_kwargs = mock_instructor.patch.call_args[1]
assert call_kwargs['mode'] == 'JSON_MODE'
def test_workflow_generator_uses_json_mode(self):
"""Test that WorkflowAutoGenerator also uses JSON mode."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
with patch('praisonai.auto.LITELLM_AVAILABLE', False):
with patch('praisonai.auto.instructor') as mock_instructor:
with patch('praisonai.auto.OpenAI') as mock_openai:
mock_instructor.patch.return_value = Mock()
mock_instructor.Mode.JSON = 'JSON_MODE'
mock_openai.return_value = Mock()
generator = WorkflowAutoGenerator(
topic="Test workflow"
)
# Access client to trigger creation
_ = generator.client
# Verify JSON mode was used
call_kwargs = mock_instructor.patch.call_args[1]
assert call_kwargs['mode'] == 'JSON_MODE'
class TestAutoGeneratorImprovedPrompt:
"""Test suite for improved task analysis prompt."""
def test_prompt_includes_complexity_analysis(self):
"""Test that the prompt includes task complexity analysis."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
prompt = generator.get_user_content()
# Check for complexity analysis steps
assert "TASK COMPLEXITY ANALYSIS" in prompt
assert "DETERMINE OPTIMAL TEAM SIZE" in prompt
assert "Simple tasks: 1-2 agents" in prompt
assert "Complex tasks: 3-4 agents" in prompt
def test_prompt_discourages_unnecessary_complexity(self):
"""Test that the prompt discourages unnecessary complexity."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
prompt = generator.get_user_content()
assert "meaningful specialization" in prompt
assert "Avoid unnecessary complexity" in prompt
class TestWorkflowPatterns:
"""Test suite for workflow patterns including new orchestrator-workers and evaluator-optimizer."""
def test_get_prompt_sequential(self):
"""Test that sequential pattern generates correct prompt."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Test topic")
prompt = generator._get_prompt("sequential")
assert "sequential" in prompt.lower()
assert "Sequential Workflow" in prompt
def test_get_prompt_orchestrator_workers(self):
"""Test that orchestrator-workers pattern generates correct prompt."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Test topic")
prompt = generator._get_prompt("orchestrator-workers")
assert "orchestrator" in prompt.lower()
assert "workers" in prompt.lower()
assert "synthesizer" in prompt.lower()
def test_get_prompt_evaluator_optimizer(self):
"""Test that evaluator-optimizer pattern generates correct prompt."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Test topic")
prompt = generator._get_prompt("evaluator-optimizer")
assert "evaluator" in prompt.lower()
assert "generator" in prompt.lower()
assert "loop" in prompt.lower()
assert "APPROVED" in prompt
class TestPatternRecommendation:
"""Test suite for pattern recommendation based on task type."""
def test_recommend_sequential_for_simple_task(self):
"""Test that sequential is recommended for simple tasks."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Write a blog post")
assert generator.recommend_pattern() == "sequential"
def test_recommend_parallel_for_concurrent_tasks(self):
"""Test that parallel is recommended for concurrent tasks."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Research from multiple sources simultaneously")
assert generator.recommend_pattern() == "parallel"
def test_recommend_orchestrator_for_complex_tasks(self):
"""Test that orchestrator-workers is recommended for complex tasks."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Comprehensive analysis and break down the problem")
assert generator.recommend_pattern() == "orchestrator-workers"
def test_recommend_evaluator_for_quality_tasks(self):
"""Test that evaluator-optimizer is recommended for quality-focused tasks."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Refine and improve the content quality")
assert generator.recommend_pattern() == "evaluator-optimizer"
def test_recommend_routing_for_classification_tasks(self):
"""Test that routing is recommended for classification tasks."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Classify and route different types of requests")
assert generator.recommend_pattern() == "routing"
class TestAutoGeneratorConfiguration:
"""Test suite for configuration options."""
def test_custom_config_list(self):
"""Test that custom config_list is respected."""
custom_config = [
{
'model': 'custom-model',
'base_url': 'https://custom.api.com',
'api_key': 'custom-key'
}
]
generator = AutoGenerator(
topic="Test topic",
framework="praisonai",
config_list=custom_config
)
assert generator.config_list == custom_config
def test_base_url_priority(self):
"""Test base_url environment variable priority."""
with patch.dict(os.environ, {
'OPENAI_BASE_URL': 'https://priority.api.com',
'OPENAI_API_BASE': 'https://fallback.api.com',
'OPENAI_API_KEY': 'test-key'
}):
generator = AutoGenerator(
topic="Test topic",
framework="praisonai"
)
assert generator.config_list[0]['base_url'] == 'https://priority.api.com'
class TestBaseAutoGenerator:
"""Test suite for BaseAutoGenerator shared functionality."""
def test_analyze_complexity_simple(self):
"""Test that simple tasks are detected correctly."""
from praisonai.auto import BaseAutoGenerator
simple_tasks = [
"Write a haiku about spring",
"Create a simple summary",
"Write a brief poem",
"Make a quick list"
]
for task in simple_tasks:
assert BaseAutoGenerator.analyze_complexity(task) == 'simple', f"Failed for: {task}"
def test_analyze_complexity_complex(self):
"""Test that complex tasks are detected correctly."""
from praisonai.auto import BaseAutoGenerator
complex_tasks = [
"Comprehensive market analysis and report",
"Research and write a detailed analysis",
"Multi-step process to coordinate teams",
"In-depth investigation of multiple sources"
]
for task in complex_tasks:
assert BaseAutoGenerator.analyze_complexity(task) == 'complex', f"Failed for: {task}"
def test_analyze_complexity_moderate(self):
"""Test that moderate tasks are detected correctly."""
from praisonai.auto import BaseAutoGenerator
moderate_tasks = [
"Research AI trends",
"Analyze customer feedback",
"Design a workflow"
]
for task in moderate_tasks:
assert BaseAutoGenerator.analyze_complexity(task) == 'moderate', f"Failed for: {task}"
def test_get_available_tools(self):
"""Test that available tools list is returned correctly."""
from praisonai.auto import BaseAutoGenerator, AVAILABLE_TOOLS
tools = BaseAutoGenerator.get_available_tools()
# Should return a copy, not the original
assert tools == AVAILABLE_TOOLS
assert tools is not AVAILABLE_TOOLS
# Should contain expected tools
assert "WebsiteSearchTool" in tools
assert "PDFSearchTool" in tools
assert "ScrapeWebsiteTool" in tools
class TestWorkflowDynamicAgentCount:
"""Test suite for dynamic agent count in WorkflowAutoGenerator."""
def test_prompt_includes_complexity_analysis(self):
"""Test that workflow prompt includes complexity analysis."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Research AI trends")
prompt = generator._get_prompt("sequential")
assert "STEP 1: ANALYZE TASK COMPLEXITY" in prompt
assert "STEP 2: DESIGN WORKFLOW" in prompt
assert "STEP 3: ASSIGN TOOLS" in prompt
def test_prompt_includes_tools_list(self):
"""Test that workflow prompt includes available tools."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Research AI trends")
prompt = generator._get_prompt("sequential")
assert "Available Tools:" in prompt
assert "WebsiteSearchTool" in prompt
assert "PDFSearchTool" in prompt
def test_simple_task_gets_fewer_agents_guidance(self):
"""Test that simple tasks get 1-2 agents guidance."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Write a simple haiku")
prompt = generator._get_prompt("sequential")
assert "1-2 agents" in prompt
assert "simple task detected" in prompt
def test_complex_task_gets_more_agents_guidance(self):
"""Test that complex tasks get 3-4 agents guidance."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Comprehensive market analysis and detailed report")
prompt = generator._get_prompt("sequential")
assert "3-4 agents" in prompt
assert "complex task detected" in prompt
# =============================================================================
# TODO 2: Single-Agent Option Tests
# =============================================================================
class TestSingleAgentOption:
"""Test suite for single-agent generation option."""
def test_single_agent_structure_exists(self):
"""Test that SingleAgentStructure Pydantic model exists."""
from praisonai.auto import SingleAgentStructure
assert SingleAgentStructure is not None
def test_auto_generator_has_single_agent_mode(self):
"""Test that AutoGenerator supports single_agent parameter."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Write a haiku",
framework="praisonai",
single_agent=True
)
assert generator.single_agent == True
def test_workflow_generator_has_single_agent_mode(self):
"""Test that WorkflowAutoGenerator supports single_agent parameter."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(
topic="Write a haiku",
single_agent=True
)
assert generator.single_agent == True
# =============================================================================
# TODO 3: LLM-Based Pattern Recommendation Tests
# =============================================================================
class TestLLMPatternRecommendation:
"""Test suite for LLM-based pattern recommendation."""
def test_recommend_pattern_llm_method_exists(self):
"""Test that recommend_pattern_llm method exists."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Test task")
assert hasattr(generator, 'recommend_pattern_llm')
def test_pattern_recommendation_pydantic_model_exists(self):
"""Test that PatternRecommendation Pydantic model exists."""
from praisonai.auto import PatternRecommendation
assert PatternRecommendation is not None
def test_pattern_recommendation_has_required_fields(self):
"""Test PatternRecommendation has pattern, reasoning, confidence."""
from praisonai.auto import PatternRecommendation
fields = PatternRecommendation.model_fields
assert 'pattern' in fields
assert 'reasoning' in fields
assert 'confidence' in fields
# =============================================================================
# TODO 4: Validation Gates Tests
# =============================================================================
class TestValidationGates:
"""Test suite for validation gates in workflows."""
def test_validation_gate_model_exists(self):
"""Test that ValidationGate Pydantic model exists."""
from praisonai.auto import ValidationGate
assert ValidationGate is not None
def test_validation_gate_has_required_fields(self):
"""Test ValidationGate has criteria, pass_action, fail_action."""
from praisonai.auto import ValidationGate
fields = ValidationGate.model_fields
assert 'criteria' in fields
assert 'pass_action' in fields
assert 'fail_action' in fields
def test_workflow_structure_supports_gates(self):
"""Test that WorkflowStructure has optional gates field."""
from praisonai.auto import WorkflowStructure
fields = WorkflowStructure.model_fields
assert 'gates' in fields
# =============================================================================
# TODO 6: Pattern Support for AutoGenerator Tests
# =============================================================================
class TestAutoGeneratorPatternSupport:
"""Test suite for pattern support in AutoGenerator."""
def test_auto_generator_accepts_pattern_parameter(self):
"""Test that AutoGenerator accepts pattern parameter."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Research and write",
framework="praisonai",
pattern="parallel"
)
assert generator.pattern == "parallel"
def test_auto_generator_has_recommend_pattern(self):
"""Test that AutoGenerator has recommend_pattern method."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(topic="Test", framework="praisonai")
assert hasattr(generator, 'recommend_pattern')
def test_auto_generator_prompt_includes_pattern(self):
"""Test that AutoGenerator prompt includes pattern guidance."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = AutoGenerator(
topic="Research from multiple sources",
framework="praisonai",
pattern="parallel"
)
prompt = generator.get_user_content()
assert "parallel" in prompt.lower()
# =============================================================================
# TODO 7: Merge Support for WorkflowAutoGenerator Tests
# =============================================================================
class TestWorkflowMergeSupport:
"""Test suite for merge support in WorkflowAutoGenerator."""
def test_workflow_generator_accepts_merge_parameter(self):
"""Test that WorkflowAutoGenerator.generate accepts merge parameter."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Test task")
# Check that generate method accepts merge parameter
import inspect
sig = inspect.signature(generator.generate)
assert 'merge' in sig.parameters
def test_merge_with_existing_workflow_method_exists(self):
"""Test that merge_with_existing_workflow method exists."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Test task")
assert hasattr(generator, 'merge_with_existing_workflow')
# =============================================================================
# TODO 10: Framework Support for WorkflowAutoGenerator Tests
# =============================================================================
class TestWorkflowFrameworkSupport:
"""Test suite for framework support in WorkflowAutoGenerator."""
def test_workflow_generator_accepts_framework_parameter(self):
"""Test that WorkflowAutoGenerator accepts framework parameter."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(
topic="Test task",
framework="crewai"
)
assert generator.framework == "crewai"
def test_workflow_generator_default_framework_is_praisonai(self):
"""Test that default framework is praisonai."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(topic="Test task")
assert generator.framework == "praisonai"
def test_save_workflow_respects_framework(self):
"""Test that _save_workflow uses the specified framework."""
with patch.dict(os.environ, {'OPENAI_API_KEY': 'test-key'}):
generator = WorkflowAutoGenerator(
topic="Test task",
framework="crewai"
)
# The framework should be stored and used in output
assert generator.framework == "crewai"
if __name__ == '__main__':
pytest.main([__file__, '-v'])