Compare commits

...
14 Commits
Author SHA1 Message Date
Mervin Praison 43a15e6049 Release v2.3.36 2025-12-18 16:42:58 +00:00
Mervin Praison 8bf5a08246 Release v2.3.35 2025-12-18 16:28:02 +00:00
Mervin Praison aa81094ac3 Release v2.3.34 2025-12-18 14:55:33 +00:00
Mervin Praison c74fd7e925 Release v2.3.33 2025-12-18 14:41:36 +00:00
Mervin Praison 249acf7965 docs: Add performance benchmarks and auto-update functionality
Add performance comparison table to README showing PraisonAI as fastest framework for agent instantiation. Include benchmarks for LangGraph and CrewAI frameworks. Implement auto-save functionality to generate BENCHMARK_RESULTS.md and update README.md with latest benchmark results.
2025-12-18 13:30:06 +00:00
Mervin Praison ab68b2d296 Release v2.3.32 2025-12-18 13:17:32 +00:00
Mervin Praison abcc5b3b04 Release v2.3.31 2025-12-18 12:57:05 +00:00
Mervin Praison e584f56099 Release v2.3.30 2025-12-18 11:59:22 +00:00
Mervin Praison b6b01af066 Release v2.3.33 2025-12-18 11:19:29 +00:00
Mervin Praison 9aae3eff86 Release v2.3.32 2025-12-18 10:48:09 +00:00
Mervin Praison f90582c66b Release v2.3.31 2025-12-18 10:27:03 +00:00
Mervin Praison 67a8d65d8e Release v2.3.30 2025-12-18 09:07:42 +00:00
Mervin Praison d60c31a86d Release v2.3.29 2025-12-18 02:47:04 +00:00
Mervin Praison fcb7b2a0d0 fix(tests): Mark test_phase2_refactor tests as @pytest.mark.real
These tests make real API calls and should be excluded from CI.
2025-12-17 13:31:01 +00:00
53 changed files with 7386 additions and 256 deletions

No files matched your search

+72
View File
@@ -34,6 +34,7 @@ PraisonAI is a production-ready Multi-AI Agents framework with self-reflection,
## 📑 Table of Contents
- [⚡ Performance](#-performance)
- [🚀 Quick Start](#-quick-start)
- [📦 Installation](#-installation)
- [Python SDK](#python-sdk)
@@ -73,6 +74,32 @@ PraisonAI is a production-ready Multi-AI Agents framework with self-reflection,
---
## ⚡ Performance
PraisonAI Agents is the **fastest AI agent framework** for agent instantiation.
| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
| **PraisonAI** | **3.77** | **1.00x (fastest)** |
| OpenAI Agents SDK | 5.26 | 1.39x |
| Agno | 5.64 | 1.49x |
| PraisonAI (LiteLLM) | 7.56 | 2.00x |
| PydanticAI | 226.94 | 60.16x |
| LangGraph | 4,558.71 | 1,209x |
| CrewAI | 15,607.92 | 4,138x |
<details>
<summary>Run benchmarks yourself</summary>
```bash
cd praisonai-agents
python benchmarks/simple_benchmark.py
```
</details>
---
## 🚀 Quick Start
Get started with PraisonAI in under 1 minute:
@@ -1392,6 +1419,36 @@ praisonai commit
praisonai commit --push
```
### Serve CLI (API Server):
```bash
# Start API server for agents defined in YAML
praisonai serve agents.yaml
# With custom port and host
praisonai serve agents.yaml --port 8005 --host 0.0.0.0
# Alternative flag style
praisonai agents.yaml --serve
# The server provides:
# POST /agents - Run all agents sequentially
# POST /agents/{name} - Run specific agent (e.g., /agents/researcher)
# GET /agents/list - List available agents
```
### n8n Integration CLI:
```bash
# Export workflow to n8n and open in browser
praisonai agents.yaml --n8n
# With custom n8n URL
praisonai agents.yaml --n8n --n8n-url http://localhost:5678
# Set N8N_API_KEY for auto-import
export N8N_API_KEY="your-api-key"
praisonai agents.yaml --n8n
```
### @Mentions in Prompts:
```bash
# Include file content in prompt
@@ -1523,6 +1580,21 @@ agent = Agent(
env={"BRAVE_API_KEY": "your-key"}
)
)
# Multiple MCP servers + regular functions
def my_custom_tool(query: str) -> str:
"""Custom tool function."""
return f"Result: {query}"
agent = Agent(
name="MultiToolAgent",
instructions="Agent with multiple MCP servers",
tools=[
MCP("uvx mcp-server-time"), # Time tools
MCP("npx @modelcontextprotocol/server-memory"), # Memory tools
my_custom_tool # Regular function
]
)
```
### MCP Server (Expose Tools as MCP Server)
+2 -1
View File
@@ -14,9 +14,10 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
# Note: praisonai is installed from PyPI - remove version pin to use latest available
RUN pip install --no-cache-dir \
flask \
"praisonai>=2.3.28" \
praisonai \
"praisonai[api]" \
gunicorn \
markdown
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.28" \
"praisonai>=2.3.36" \
"praisonai[chat]" \
"embedchain[github,youtube]"
+1 -1
View File
@@ -20,7 +20,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.28" \
"praisonai>=2.3.36" \
"praisonai[ui]" \
"praisonai[chat]" \
"praisonai[realtime]" \
+25
View File
@@ -0,0 +1,25 @@
# Dockerfile for PraisonAI Serve API
# Deploy to Railway, Render, Fly.io, or any container platform
#
# Build: docker build -f Dockerfile.serve -t praisonai-serve .
# Run: docker run -p 8005:8005 -e OPENAI_API_KEY=sk-xxx praisonai-serve
FROM python:3.11-slim
WORKDIR /app
# Install dependencies
RUN pip install --no-cache-dir praisonai praisonaiagents
# Copy agents configuration
COPY agents.yaml .
# Expose the API port
EXPOSE 8005
# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD curl -f http://localhost:8005/health || exit 1
# Start the API server
CMD ["praisonai", "serve", "agents.yaml", "--host", "0.0.0.0", "--port", "8005"]
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.28" \
"praisonai>=2.3.36" \
"praisonai[ui]" \
"praisonai[crewai]"
+108
View File
@@ -0,0 +1,108 @@
"""
Multiple MCP Servers Example
This example demonstrates how to use multiple MCP servers with a single agent.
You can combine different MCP servers (time, memory, filesystem, etc.) along
with regular Python functions as tools.
Requirements:
- pip install praisonaiagents
- uvx (for mcp-server-time)
- npx (for @modelcontextprotocol servers)
"""
from praisonaiagents import Agent
from praisonaiagents.mcp import MCP
def my_custom_tool(query: str) -> str:
"""
A custom Python function that can be used alongside MCP tools.
Args:
query: The search query
Returns:
A formatted result string
"""
return f"Custom tool result for: {query}"
def main():
# Example 1: Single MCP server (basic usage)
print("=" * 60)
print("Example 1: Single MCP Server")
print("=" * 60)
agent_single = Agent(
name="TimeAgent",
instructions="You are a helpful assistant that can tell time.",
tools=MCP("uvx mcp-server-time")
)
response = agent_single.start("What time is it in Tokyo?")
print(f"Response: {response}\n")
# Example 2: Multiple MCP servers
print("=" * 60)
print("Example 2: Multiple MCP Servers")
print("=" * 60)
agent_multi = Agent(
name="MultiMCPAgent",
instructions="""You are a helpful assistant with access to multiple tools:
- Time tools for getting current time and converting timezones
- Memory tools for storing and retrieving information
Use the appropriate tool based on the user's request.""",
tools=[
MCP("uvx mcp-server-time"), # Time tools
MCP("npx @modelcontextprotocol/server-memory"), # Memory tools
]
)
response = agent_multi.start("What time is it in New York and London?")
print(f"Response: {response}\n")
# Example 3: Multiple MCP servers + custom functions
print("=" * 60)
print("Example 3: MCP Servers + Custom Functions")
print("=" * 60)
agent_mixed = Agent(
name="MixedToolsAgent",
instructions="""You are a helpful assistant with access to:
- Time tools for timezone operations
- A custom search tool for general queries
Choose the right tool for each task.""",
tools=[
MCP("uvx mcp-server-time"), # MCP server
my_custom_tool # Regular Python function
]
)
response = agent_mixed.start("What time is it in Paris?")
print(f"Response: {response}\n")
# Example 4: MCP with environment variables
print("=" * 60)
print("Example 4: MCP with Environment Variables")
print("=" * 60)
# Note: Replace with your actual API key
# agent_with_env = Agent(
# name="SearchAgent",
# instructions="You can search the web for information.",
# tools=[
# MCP(
# command="npx",
# args=["-y", "@modelcontextprotocol/server-brave-search"],
# env={"BRAVE_API_KEY": "your-api-key-here"}
# ),
# MCP("uvx mcp-server-time")
# ]
# )
print("(Skipped - requires BRAVE_API_KEY)")
if __name__ == "__main__":
main()
+1 -1
View File
@@ -11,7 +11,7 @@
framework: praisonai
process: workflow
topic: "Research AI trends and create a comprehensive report"
input: "Research AI trends and create a comprehensive report" # The input passed to {{input}} in steps
# Workflow configuration (new)
workflow:
+1 -1
View File
@@ -39,4 +39,4 @@ steps:
over: topics
- agent: summarizer
action: "Create a comprehensive summary of all the analyzed topics"
action: "Create a comprehensive summary of all the analyzed topics based on these results: {{previous_output}}"
@@ -3,6 +3,7 @@
name: Parallel Research Workflow
description: Multiple agents research different aspects concurrently, then an aggregator combines results
input: "AI-powered customer service solutions" # The input passed to {{input}} in steps
workflow:
verbose: true
@@ -43,4 +44,4 @@ steps:
action: "Research customer needs for: {{input}}"
- agent: aggregator
action: "Synthesize all the research findings into a comprehensive report"
action: "Synthesize all the research findings into a comprehensive report: {{previous_output}}"
@@ -3,6 +3,7 @@
name: Agentic Routing Workflow
description: A classifier agent routes requests to specialized handler agents
input: "Write a poem about the ocean" # The input passed to {{input}} in steps
workflow:
verbose: true
@@ -3,6 +3,7 @@
name: Simple Research Workflow
description: A simple workflow where a researcher gathers info and a writer summarizes it
input: "The benefits of renewable energy" # The input passed to {{input}} in steps
workflow:
verbose: true
+1
View File
@@ -87,3 +87,4 @@ graph.py
chroma_db/
.qodo
*_COMPARISON_ANALYSIS.md
*_REPORT.md
@@ -0,0 +1,35 @@
# PraisonAI Agents - Benchmark Results
**Generated:** 2025-12-18 14:40:22
**Iterations:** 100
**Test:** Agent instantiation (without tools)
## Results
| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
| **PraisonAI** | **3.77** | **1.00x (fastest)** |
| OpenAI Agents SDK | 5.26 | 1.39x |
| Agno | 5.64 | 1.49x |
| PraisonAI (LiteLLM) | 7.56 | 2.00x |
| PydanticAI | 226.94 | 60.16x |
| LangGraph | 4,558.71 | 1,209x |
| CrewAI | 15,607.92 | 4,138x |
## Package Versions
| Package | Version |
|---------|--------|
| PraisonAI | 0.1.5 |
| Agno | 2.3.14 |
| PydanticAI | 1.35.0 |
| OpenAI Agents SDK | 0.6.3 |
| LangGraph | 1.0.5 |
| CrewAI | 1.6.1 |
## How to Reproduce
```bash
cd praisonai-agents
python benchmarks/simple_benchmark.py
```
@@ -0,0 +1,31 @@
# PraisonAI Agents - Real Execution Benchmark
**Generated:** 2025-12-18 14:43:52
**Model:** gpt-4o-mini
**Iterations:** 3
**Prompt:** "What is 2+2? Reply with just the number."
## Results
| Framework | Method | Avg Time | Relative |
|-----------|--------|----------|----------|
| **PraisonAI** | `agent.start()` | **0.45s** | **1.00x (fastest)** |
| PraisonAI (LiteLLM) | `agent.start()` | 0.55s | 1.22x |
| CrewAI | `crew.kickoff()` | 0.58s | 1.28x |
| Agno | `agent.run()` | 0.92s | 2.05x |
## Package Versions
| Package | Version |
|---------|--------|
| PraisonAI | 0.1.5 |
| Agno | 2.3.14 |
| CrewAI | 1.6.1 |
## How to Reproduce
```bash
export OPENAI_API_KEY=your_key
cd praisonai-agents
python benchmarks/execution_benchmark.py
```
@@ -0,0 +1,72 @@
# PraisonAI Agents - Real-World Execution Benchmark
**Generated:** 2025-12-18 14:33:49
**Model:** gpt-4o-mini
**Iterations:** 3 per test
## Results Summary
| Test | PraisonAI (avg) | Agno (avg) | Difference |
|------|-----------------|------------|------------|
| Single Agent | 2.04s | 3.18s | PraisonAI 1.56x faster |
| 2 Agents | 2.49s | 20.49s | PraisonAI 8.21x faster |
| 3 Agents | 6.52s | 13.97s | PraisonAI 2.14x faster |
## Detailed Results
### praisonai_single
- Average: 2.04s
- Min: 1.95s
- Max: 2.19s
- Std Dev: 0.13s
- Times: ['2.19s', '1.95s', '2.00s']
### agno_single
- Average: 3.18s
- Min: 3.00s
- Max: 3.51s
- Std Dev: 0.28s
- Times: ['3.51s', '3.00s', '3.04s']
### praisonai_two_agents
- Average: 2.49s
- Min: 2.22s
- Max: 2.65s
- Std Dev: 0.24s
- Times: ['2.65s', '2.22s', '2.62s']
### agno_two_agents
- Average: 20.49s
- Min: 9.96s
- Max: 35.05s
- Std Dev: 13.02s
- Times: ['16.46s', '9.96s', '35.05s']
### praisonai_three_agents
- Average: 6.52s
- Min: 5.62s
- Max: 7.39s
- Std Dev: 0.88s
- Times: ['5.62s', '6.54s', '7.39s']
### agno_three_agents
- Average: 13.97s
- Min: 11.69s
- Max: 15.32s
- Std Dev: 1.99s
- Times: ['14.91s', '11.69s', '15.32s']
## Package Versions
| Package | Version |
|---------|--------|
| PraisonAI | 0.1.5 |
| Agno | 2.3.14 |
## How to Reproduce
```bash
export OPENAI_API_KEY=your_key
cd praisonai-agents
python benchmarks/real_benchmark.py
```
@@ -0,0 +1,22 @@
# PraisonAI Agents - Benchmark Results
**Generated:** 2025-12-18 13:23:43
## Agent Instantiation Time
| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
| **PraisonAI** | **4.16** | **1.00x (fastest)** |
| OpenAI Agents SDK | 5.55 | 1.33x |
| Agno | 5.58 | 1.34x |
| PraisonAI (LiteLLM) | 8.52 | 2.05x |
| PydanticAI | 225.87 | 54.25x |
| LangGraph | 4,349.54 | 1,045x |
| CrewAI | 15,911.11 | 3,821x |
## How to Reproduce
```bash
cd praisonai-agents
python benchmarks/simple_benchmark.py
```
@@ -0,0 +1,33 @@
# PraisonAI Agents - Tools Benchmark Results
**Generated:** 2025-12-18 14:40:35
**Iterations:** 100
**Test:** Agent instantiation WITH TOOLS
## Results
| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
| **PraisonAI** | **3.24** | **1.00x (fastest)** |
| Agno | 5.12 | 1.58x |
| PraisonAI (LiteLLM) | 8.59 | 2.65x |
| OpenAI Agents SDK | 279.95 | 86.44x |
| LangGraph | 2,310.82 | 713x |
| CrewAI | 15,773.44 | 4,870x |
## Package Versions
| Package | Version |
|---------|--------|
| PraisonAI | 0.1.5 |
| Agno | 2.3.14 |
| OpenAI Agents SDK | 0.6.3 |
| LangGraph | 1.0.5 |
| CrewAI | 1.6.1 |
## How to Reproduce
```bash
cd praisonai-agents
python benchmarks/tools_benchmark.py
```
@@ -0,0 +1,368 @@
#!/usr/bin/env python3
"""
PraisonAI Agents - Deep Profiling Benchmark
Profiles each step of agent initialization to identify performance bottlenecks.
Key Optimizations Applied:
1. Lazy RulesManager - filesystem operations deferred until first access
2. Lazy os.getcwd() - fast_context_path resolved only when needed
3. Lazy console initialization - Rich Console created on demand
Usage:
python benchmarks/deep_profiling.py
"""
import time
import tracemalloc
import cProfile
import pstats
import io
from typing import Literal, Callable, Dict, List
from dataclasses import dataclass
# ============================================================================
# Timing Utilities
# ============================================================================
@dataclass
class StepTiming:
name: str
duration_us: float # microseconds
memory_kb: float # kilobytes
class StepProfiler:
"""Profile individual steps during initialization"""
def __init__(self):
self.steps: List[StepTiming] = []
self._start_time = None
self._start_memory = None
def start_step(self, name: str):
tracemalloc.start()
self._start_time = time.perf_counter()
self._step_name = name
def end_step(self):
end_time = time.perf_counter()
current, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()
duration_us = (end_time - self._start_time) * 1_000_000
memory_kb = peak / 1024
self.steps.append(StepTiming(
name=self._step_name,
duration_us=duration_us,
memory_kb=memory_kb
))
def print_report(self, title: str):
print(f"\n{'='*70}")
print(f"STEP-BY-STEP PROFILING: {title}")
print(f"{'='*70}")
total_time = sum(s.duration_us for s in self.steps)
total_memory = sum(s.memory_kb for s in self.steps)
print(f"\n{'Step':<40} {'Time (μs)':<15} {'Memory (KB)':<15} {'% Time':<10}")
print("-" * 80)
for step in sorted(self.steps, key=lambda x: x.duration_us, reverse=True):
pct = (step.duration_us / total_time * 100) if total_time > 0 else 0
print(f"{step.name:<40} {step.duration_us:<15.2f} {step.memory_kb:<15.2f} {pct:<10.1f}%")
print("-" * 80)
print(f"{'TOTAL':<40} {total_time:<15.2f} {total_memory:<15.2f} {'100.0':<10}%")
def time_function(func: Callable, iterations: int = 100) -> Dict:
"""Time a function with detailed statistics"""
times = []
# Warmup
for _ in range(10):
func()
# Measure
for _ in range(iterations):
start = time.perf_counter()
func()
end = time.perf_counter()
times.append((end - start) * 1_000_000) # Convert to microseconds
return {
'avg': sum(times) / len(times),
'min': min(times),
'max': max(times),
'median': sorted(times)[len(times)//2]
}
# ============================================================================
# Profile PraisonAI Agent Initialization - Step by Step
# ============================================================================
def profile_praisonai_detailed():
"""Profile each step of PraisonAI Agent initialization"""
print("\n" + "="*70)
print("DETAILED PROFILING: PraisonAI Agent")
print("="*70)
profiler = StepProfiler()
# Step 1: Import
profiler.start_step("1. Import praisonaiagents")
from praisonaiagents import Agent as PraisonAgent
profiler.end_step()
# Step 2: Define tool
profiler.start_step("2. Define tool function")
def get_weather(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
tools = [get_weather]
profiler.end_step()
# Step 3: Full instantiation
profiler.start_step("3. Full Agent instantiation")
agent = PraisonAgent(
name="Test Agent",
instructions="Be concise, reply with one sentence.",
llm="gpt-4o-mini",
tools=tools,
verbose=False
)
profiler.end_step()
profiler.print_report("PraisonAI Agent")
# Now profile the __init__ internals using cProfile
print("\n" + "="*70)
print("cProfile ANALYSIS: PraisonAI Agent.__init__")
print("="*70)
pr = cProfile.Profile()
pr.enable()
for _ in range(100):
agent = PraisonAgent(
name="Test Agent",
instructions="Be concise, reply with one sentence.",
llm="gpt-4o-mini",
tools=tools,
verbose=False
)
pr.disable()
# Print top 30 functions by cumulative time
s = io.StringIO()
ps = pstats.Stats(pr, stream=s).sort_stats('cumulative')
ps.print_stats(30)
print(s.getvalue())
return agent
# ============================================================================
# Profile Agno Agent Initialization - Step by Step
# ============================================================================
def profile_agno_detailed():
"""Profile each step of Agno Agent initialization"""
print("\n" + "="*70)
print("DETAILED PROFILING: Agno Agent")
print("="*70)
profiler = StepProfiler()
# Step 1: Import
profiler.start_step("1. Import agno")
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
profiler.end_step()
# Step 2: Define tool
profiler.start_step("2. Define tool function")
def get_weather(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
tools = [get_weather]
profiler.end_step()
# Step 3: Full instantiation
profiler.start_step("3. Full Agent instantiation")
agent = AgnoAgent(model=OpenAIChat(id="gpt-4o-mini"), tools=tools)
profiler.end_step()
profiler.print_report("Agno Agent")
# Now profile the __init__ internals using cProfile
print("\n" + "="*70)
print("cProfile ANALYSIS: Agno Agent.__init__")
print("="*70)
pr = cProfile.Profile()
pr.enable()
for _ in range(100):
agent = AgnoAgent(model=OpenAIChat(id="gpt-4o-mini"), tools=tools)
pr.disable()
# Print top 30 functions by cumulative time
s = io.StringIO()
ps = pstats.Stats(pr, stream=s).sort_stats('cumulative')
ps.print_stats(30)
print(s.getvalue())
return agent
# ============================================================================
# Profile Individual Components of PraisonAI
# ============================================================================
def profile_praisonai_components():
"""Profile individual components of PraisonAI Agent initialization"""
print("\n" + "="*70)
print("COMPONENT-LEVEL PROFILING: PraisonAI Agent")
print("="*70)
from praisonaiagents import Agent as PraisonAgent
def get_weather(city: Literal["nyc", "sf"]):
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
tools = [get_weather]
# Test different configurations
configs = [
("Minimal (no tools, no llm)", {"name": "Test", "verbose": False}),
("With name only", {"name": "Test Agent", "verbose": False}),
("With instructions", {"name": "Test", "instructions": "Be concise", "verbose": False}),
("With llm (gpt-4o-mini)", {"name": "Test", "llm": "gpt-4o-mini", "verbose": False}),
("With llm (openai/gpt-4o-mini)", {"name": "Test", "llm": "openai/gpt-4o-mini", "verbose": False}),
("With tools", {"name": "Test", "tools": tools, "verbose": False}),
("Full config (OpenAI SDK)", {"name": "Test", "instructions": "Be concise", "llm": "gpt-4o-mini", "tools": tools, "verbose": False}),
("Full config (LiteLLM)", {"name": "Test", "instructions": "Be concise", "llm": "openai/gpt-4o-mini", "tools": tools, "verbose": False}),
]
print(f"\n{'Configuration':<40} {'Avg Time (μs)':<15} {'Min (μs)':<15} {'Max (μs)':<15}")
print("-" * 85)
for name, config in configs:
def create_agent():
return PraisonAgent(**config)
stats = time_function(create_agent, iterations=100)
print(f"{name:<40} {stats['avg']:<15.2f} {stats['min']:<15.2f} {stats['max']:<15.2f}")
# ============================================================================
# Compare Specific Operations
# ============================================================================
def compare_specific_operations():
"""Compare specific operations between PraisonAI and Agno"""
print("\n" + "="*70)
print("OPERATION COMPARISON: PraisonAI vs Agno")
print("="*70)
from praisonaiagents import Agent as PraisonAgent
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
def get_weather(city: Literal["nyc", "sf"]):
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
tools = [get_weather]
operations = []
# Operation 1: Bare minimum instantiation
def praisonai_minimal():
return PraisonAgent(name="Test", verbose=False)
def agno_minimal():
return AgnoAgent(model=OpenAIChat(id="gpt-4o-mini"))
praison_stats = time_function(praisonai_minimal, 100)
agno_stats = time_function(agno_minimal, 100)
operations.append(("Minimal instantiation", praison_stats['avg'], agno_stats['avg']))
# Operation 2: With tools
def praisonai_with_tools():
return PraisonAgent(name="Test", tools=tools, verbose=False)
def agno_with_tools():
return AgnoAgent(model=OpenAIChat(id="gpt-4o-mini"), tools=tools)
praison_stats = time_function(praisonai_with_tools, 100)
agno_stats = time_function(agno_with_tools, 100)
operations.append(("With tools", praison_stats['avg'], agno_stats['avg']))
# Operation 3: Full config
def praisonai_full():
return PraisonAgent(
name="Test Agent",
instructions="Be concise",
llm="gpt-4o-mini",
tools=tools,
verbose=False
)
def agno_full():
return AgnoAgent(
model=OpenAIChat(id="gpt-4o-mini"),
tools=tools,
instructions=["Be concise"]
)
praison_stats = time_function(praisonai_full, 100)
agno_stats = time_function(agno_full, 100)
operations.append(("Full config", praison_stats['avg'], agno_stats['avg']))
print(f"\n{'Operation':<30} {'PraisonAI (μs)':<18} {'Agno (μs)':<18} {'Ratio':<10}")
print("-" * 76)
for op_name, praison_time, agno_time in operations:
ratio = praison_time / agno_time if agno_time > 0 else 0
print(f"{op_name:<30} {praison_time:<18.2f} {agno_time:<18.2f} {ratio:<10.1f}x")
# ============================================================================
# Main
# ============================================================================
if __name__ == "__main__":
import os
os.environ.setdefault("OPENAI_API_KEY", "sk-test")
print("="*70)
print("PraisonAI Agents - Deep Profiling Benchmark")
print("="*70)
print("\nThis analysis identifies performance bottlenecks in agent initialization.\n")
# Run all profiling
profile_praisonai_detailed()
profile_agno_detailed()
profile_praisonai_components()
compare_specific_operations()
print("\n" + "="*70)
print("PROFILING COMPLETE")
print("="*70)
@@ -0,0 +1,286 @@
#!/usr/bin/env python3
"""
PraisonAI Agents - Real Execution Benchmark
Benchmarks ACTUAL agent execution with LLM API calls.
Tests single agent running a simple task across frameworks.
Usage:
export OPENAI_API_KEY=your_key
python benchmarks/execution_benchmark.py
python benchmarks/execution_benchmark.py --no-save
"""
import time
import os
import argparse
from importlib.metadata import version as get_version
# Check API key
if not os.environ.get('OPENAI_API_KEY'):
print("ERROR: Set OPENAI_API_KEY environment variable")
exit(1)
MODEL = "gpt-4o-mini"
ITERATIONS = 3
PROMPT = "What is 2+2? Reply with just the number."
def get_package_versions():
"""Get version numbers for benchmarked packages."""
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
'CrewAI': 'crewai',
}
versions = {}
for name, pkg in packages.items():
try:
versions[name] = get_version(pkg)
except:
versions[name] = 'not installed'
return versions
def benchmark_praisonai():
"""Benchmark PraisonAI agent execution using agent.start()"""
from praisonaiagents import Agent
print("\n--- PraisonAI ---")
print(f"Method: agent.start()")
times = []
for i in range(ITERATIONS):
agent = Agent(
name="Calculator",
instructions="You are a helpful assistant. Be very brief.",
llm=MODEL,
verbose=False
)
start = time.perf_counter()
response = agent.start(PROMPT)
elapsed = time.perf_counter() - start
times.append(elapsed)
print(f" Run {i+1}: {elapsed:.2f}s - Response: {str(response)[:50]}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def benchmark_praisonai_litellm():
"""Benchmark PraisonAI with LiteLLM backend using agent.start()"""
from praisonaiagents import Agent
print("\n--- PraisonAI (LiteLLM) ---")
print(f"Method: agent.start()")
times = []
for i in range(ITERATIONS):
agent = Agent(
name="Calculator",
instructions="You are a helpful assistant. Be very brief.",
llm=f"openai/{MODEL}",
verbose=False
)
start = time.perf_counter()
response = agent.start(PROMPT)
elapsed = time.perf_counter() - start
times.append(elapsed)
print(f" Run {i+1}: {elapsed:.2f}s - Response: {str(response)[:50]}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def benchmark_agno():
"""Benchmark Agno agent execution using agent.run()"""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
print("\n--- Agno ---")
print(f"Method: agent.run()")
times = []
for i in range(ITERATIONS):
agent = Agent(
model=OpenAIChat(id=MODEL),
instructions=["You are a helpful assistant. Be very brief."],
)
start = time.perf_counter()
response = agent.run(PROMPT)
elapsed = time.perf_counter() - start
times.append(elapsed)
# Extract content from response
content = str(response.content)[:50] if hasattr(response, 'content') else str(response)[:50]
print(f" Run {i+1}: {elapsed:.2f}s - Response: {content}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def benchmark_crewai():
"""Benchmark CrewAI agent execution using crew.kickoff()"""
from crewai import Agent, Task, Crew
print("\n--- CrewAI ---")
print(f"Method: crew.kickoff()")
times = []
for i in range(ITERATIONS):
agent = Agent(
role="Calculator",
goal="Answer math questions",
backstory="You are a helpful assistant.",
llm=MODEL,
verbose=False
)
task = Task(
description=PROMPT,
expected_output="A number",
agent=agent
)
crew = Crew(
agents=[agent],
tasks=[task],
verbose=False
)
start = time.perf_counter()
response = crew.kickoff()
elapsed = time.perf_counter() - start
times.append(elapsed)
print(f" Run {i+1}: {elapsed:.2f}s - Response: {str(response)[:50]}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def save_results(results: dict):
"""Save results to markdown file."""
import os
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), 'EXECUTION_BENCHMARK_RESULTS.md')
versions = get_package_versions()
# Sort by time
sorted_results = sorted(results.items(), key=lambda x: x[1])
fastest = sorted_results[0][1]
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Real Execution Benchmark\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write(f'**Model:** {MODEL}\n')
f.write(f'**Iterations:** {ITERATIONS}\n')
f.write(f'**Prompt:** "{PROMPT}"\n\n')
f.write('## Results\n\n')
f.write('| Framework | Method | Avg Time | Relative |\n')
f.write('|-----------|--------|----------|----------|\n')
methods = {
'PraisonAI': 'agent.start()',
'PraisonAI (LiteLLM)': 'agent.start()',
'Agno': 'agent.run()',
'CrewAI': 'crew.kickoff()',
}
for name, avg in sorted_results:
ratio = avg / fastest
method = methods.get(name, 'unknown')
if ratio == 1.0:
f.write(f'| **{name}** | `{method}` | **{avg:.2f}s** | **1.00x (fastest)** |\n')
else:
f.write(f'| {name} | `{method}` | {avg:.2f}s | {ratio:.2f}x |\n')
f.write('\n## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('export OPENAI_API_KEY=your_key\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/execution_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='PraisonAI Agents - Real Execution Benchmark')
parser.add_argument('--no-save', action='store_true', help='Do not save results to file')
args = parser.parse_args()
print('=' * 60)
print('PraisonAI Agents - Real Execution Benchmark')
print('=' * 60)
print(f'\nModel: {MODEL}')
print(f'Iterations: {ITERATIONS}')
print(f'Prompt: "{PROMPT}"')
versions = get_package_versions()
print('\nPackage versions:')
for pkg, ver in versions.items():
print(f' {pkg}: {ver}')
results = {}
# Run benchmarks
try:
results['PraisonAI'] = benchmark_praisonai()
except Exception as e:
print(f"PraisonAI error: {e}")
try:
results['PraisonAI (LiteLLM)'] = benchmark_praisonai_litellm()
except Exception as e:
print(f"PraisonAI (LiteLLM) error: {e}")
try:
results['Agno'] = benchmark_agno()
except Exception as e:
print(f"Agno error: {e}")
try:
results['CrewAI'] = benchmark_crewai()
except Exception as e:
print(f"CrewAI error: {e}")
# Summary
print('\n' + '=' * 60)
print('SUMMARY')
print('=' * 60)
if results:
sorted_results = sorted(results.items(), key=lambda x: x[1])
fastest = sorted_results[0][1]
print(f"\n{'Framework':<25} {'Avg Time':<12} {'Relative':<10}")
print('-' * 47)
for name, avg in sorted_results:
ratio = avg / fastest
print(f'{name:<25} {avg:<12.2f}s {ratio:.2f}x')
print('\n' + '=' * 60)
# Save results (unless --no-save)
if results and not args.no_save:
save_results(results)
elif args.no_save:
print('\nResults not saved (--no-save flag used)')
@@ -0,0 +1,523 @@
"""
PraisonAI Agents - Comprehensive Performance Benchmark
Measures agent instantiation time and memory footprint across
multiple AI agent frameworks using industry-standard methodology.
Usage:
python benchmarks/performance_benchmark.py
"""
import gc
import time
import tracemalloc
import statistics
from typing import Literal, List, Callable
from dataclasses import dataclass, field
@dataclass
class PerformanceResult:
"""Performance measurement results"""
run_times: List[float] = field(default_factory=list)
memory_usages: List[float] = field(default_factory=list)
@property
def avg_run_time(self) -> float:
return statistics.mean(self.run_times) if self.run_times else 0
@property
def avg_memory_usage(self) -> float:
return statistics.mean(self.memory_usages) if self.memory_usages else 0
@property
def min_run_time(self) -> float:
return min(self.run_times) if self.run_times else 0
@property
def max_run_time(self) -> float:
return max(self.run_times) if self.run_times else 0
@property
def std_dev_run_time(self) -> float:
return statistics.stdev(self.run_times) if len(self.run_times) > 1 else 0
@property
def median_run_time(self) -> float:
return statistics.median(self.run_times) if self.run_times else 0
def compute_tracemalloc_baseline(samples: int = 3) -> float:
"""Compute baseline memory usage for tracemalloc"""
def empty_func():
return
results = []
for _ in range(samples):
gc.collect()
tracemalloc.start()
empty_func()
_, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()
results.append(peak / 1024 / 1024) # Convert to MiB
return sum(results) / len(results) if results else 0
def measure_time(func: Callable, iterations: int = 1000, warmup: int = 10) -> List[float]:
"""Measure execution time for a function"""
# Warmup runs
for _ in range(warmup):
func()
# Measured runs
times = []
for _ in range(iterations):
start = time.perf_counter()
func()
end = time.perf_counter()
times.append(end - start)
return times
def measure_memory(func: Callable, iterations: int = 1000, warmup: int = 10) -> List[float]:
"""Measure memory usage for a function"""
baseline = compute_tracemalloc_baseline()
# Warmup runs
for _ in range(warmup):
func()
# Measured runs
memory_usages = []
for _ in range(iterations):
gc.collect()
tracemalloc.start()
func()
_, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()
peak_mib = peak / 1024 / 1024
adjusted = max(0, peak_mib - baseline)
memory_usages.append(adjusted)
return memory_usages
def run_benchmark(func: Callable, name: str, iterations: int = 1000, warmup: int = 10) -> PerformanceResult:
"""Run full benchmark for a function"""
print(f"\n{'='*60}")
print(f"Benchmarking: {name}")
print(f"{'='*60}")
print(f"Warmup runs: {warmup}")
print(f"Measured iterations: {iterations}")
print("\nMeasuring runtime...")
run_times = measure_time(func, iterations, warmup)
print("Measuring memory...")
memory_usages = measure_memory(func, iterations, warmup)
result = PerformanceResult(run_times=run_times, memory_usages=memory_usages)
print(f"\nResults for {name}:")
print(f" Average Time: {result.avg_run_time:.6f} seconds ({result.avg_run_time * 1_000_000:.2f} μs)")
print(f" Min Time: {result.min_run_time:.6f} seconds")
print(f" Max Time: {result.max_run_time:.6f} seconds")
print(f" Std Dev: {result.std_dev_run_time:.6f} seconds")
print(f" Median Time: {result.median_run_time:.6f} seconds")
print(f" Avg Memory: {result.avg_memory_usage:.6f} MiB ({result.avg_memory_usage * 1024:.2f} KiB)")
return result
# ============================================================================
# Framework Imports
# ============================================================================
from praisonaiagents import Agent as PraisonAgent
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
# Additional frameworks for comparison
try:
from pydantic_ai import Agent as PydanticAgent
PYDANTIC_AVAILABLE = True
except ImportError:
PYDANTIC_AVAILABLE = False
try:
from crewai.agent import Agent as CrewAgent
from crewai.tools import tool as crewai_tool
CREWAI_AVAILABLE = True
except ImportError:
CREWAI_AVAILABLE = False
try:
from langchain_core.tools import tool as langchain_tool
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
LANGGRAPH_AVAILABLE = True
except ImportError:
LANGGRAPH_AVAILABLE = False
try:
from agents import Agent as OpenAIAgent, function_tool
OPENAI_AGENTS_AVAILABLE = True
except ImportError:
OPENAI_AGENTS_AVAILABLE = False
def get_weather(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
tools = [get_weather]
def instantiate_praisonai_agent():
"""Instantiate a PraisonAI agent with one tool."""
return PraisonAgent(
name="Test Agent",
instructions="Be concise, reply with one sentence.",
llm="gpt-4o-mini",
tools=tools,
verbose=False
)
def instantiate_praisonai_litellm_agent():
"""Instantiate a PraisonAI agent with LiteLLM backend."""
return PraisonAgent(
name="Test Agent",
instructions="Be concise, reply with one sentence.",
llm="openai/gpt-4o-mini",
tools=tools,
verbose=False
)
def instantiate_agno_agent():
"""Instantiate an Agno agent with one tool."""
return AgnoAgent(model=OpenAIChat(id="gpt-4o-mini"), tools=tools)
def instantiate_pydantic_agent():
"""Instantiate a PydanticAI agent with one tool."""
agent = PydanticAgent("openai:gpt-4o-mini", system_prompt="Be concise, reply with one sentence.")
@agent.tool_plain
def get_weather_pydantic(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
return agent
def instantiate_openai_agents():
"""Instantiate an OpenAI Agents SDK agent with one tool."""
return OpenAIAgent(
name="Test agent",
instructions="Be concise, reply with one sentence.",
model="gpt-4o-mini",
tools=[function_tool(get_weather)],
)
def instantiate_langgraph_agent():
"""Instantiate a LangGraph agent with one tool."""
@langchain_tool
def get_weather_lg(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
return create_react_agent(model=ChatOpenAI(model="gpt-4o-mini"), tools=[get_weather_lg])
def instantiate_crewai_agent():
"""Instantiate a CrewAI agent with one tool."""
@crewai_tool("Weather Tool")
def get_weather_crew(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
return CrewAgent(
llm="gpt-4o-mini",
role="Test Agent",
goal="Be concise, reply with one sentence.",
tools=[get_weather_crew],
backstory="Test",
)
def get_package_versions():
"""Get version numbers for all benchmarked packages."""
from importlib.metadata import version as get_version
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
'PydanticAI': 'pydantic-ai',
'OpenAI Agents SDK': 'openai-agents',
'LangGraph': 'langgraph',
'CrewAI': 'crewai'
}
versions = {}
for display_name, pkg_name in packages.items():
try:
versions[display_name] = get_version(pkg_name)
except Exception:
versions[display_name] = 'not installed'
return versions
def save_benchmark_results(results: dict, baseline):
"""Save benchmark results to BENCHMARK_RESULTS.md and update README.md"""
import os
import re
from datetime import datetime
benchmarks_dir = os.path.dirname(__file__)
filepath = os.path.join(benchmarks_dir, 'BENCHMARK_RESULTS.md')
# Get package versions
versions = get_package_versions()
display_names = {
'agno': 'Agno',
'praisonai': 'PraisonAI',
'praisonai_litellm': 'PraisonAI (LiteLLM)',
'pydantic': 'PydanticAI',
'openai_agents': 'OpenAI Agents SDK',
'langgraph': 'LangGraph',
'crewai': 'CrewAI'
}
# Build table rows
table_rows = []
sorted_results = sorted(results.items(), key=lambda x: x[1].avg_run_time)
for name, result in sorted_results:
display_name = display_names.get(name, name)
avg_us = result.avg_run_time * 1e6
ratio = result.avg_run_time / baseline.avg_run_time if baseline.avg_run_time > 0 else 0
if name == 'praisonai':
table_rows.append(f'| **{display_name}** | **{avg_us:.2f}** | **1.00x (fastest)** |')
else:
avg_str = f'{avg_us:,.2f}' if avg_us >= 1000 else f'{avg_us:.2f}'
ratio_str = f'{ratio:,.0f}x' if ratio >= 100 else f'{ratio:.2f}x'
table_rows.append(f'| {display_name} | {avg_str} | {ratio_str} |')
# Save to BENCHMARK_RESULTS.md
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Benchmark Results\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write('**Methodology:** 10 warmup runs + 1000 measured iterations\n\n')
f.write('## Agent Instantiation Time\n\n')
f.write('| Framework | Avg Time (μs) | Relative |\n')
f.write('|-----------|---------------|----------|\n')
f.write('\n'.join(table_rows) + '\n')
f.write('\n## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/performance_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
# Update README.md
readme_path = os.path.join(benchmarks_dir, '..', '..', '..', 'README.md')
if os.path.exists(readme_path):
with open(readme_path, 'r') as f:
content = f.read()
new_table = '''| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
''' + '\n'.join(table_rows)
pattern = r'(\| Framework \| Avg Time \(μs\) \| Relative \|\n\|[-|]+\|\n)(\|[^\n]+\|\n)+'
if re.search(pattern, content):
content = re.sub(pattern, new_table + '\n', content)
with open(readme_path, 'w') as f:
f.write(content)
print(f'README.md updated: {readme_path}')
if __name__ == "__main__":
print("="*70)
print("PraisonAI Agents - Comprehensive Performance Benchmark")
print("="*70)
print("\nThis benchmark measures:")
print(" 1. Agent instantiation time (how fast an agent object is created)")
print(" 2. Memory footprint (how much memory an agent uses)")
print("\nMethodology:")
print(" - 10 warmup runs (not measured)")
print(" - 1000 measured iterations")
print(" - tracemalloc for memory measurement")
print(" - Baseline subtraction for accuracy")
print("\nFrameworks available:")
print(" - PraisonAI: ✅")
print(f" - Agno: ✅")
print(f" - PydanticAI: {'✅' if PYDANTIC_AVAILABLE else '❌'}")
print(f" - OpenAI Agents SDK: {'✅' if OPENAI_AGENTS_AVAILABLE else '❌'}")
print(f" - LangGraph: {'✅' if LANGGRAPH_AVAILABLE else '❌'}")
print(f" - CrewAI: {'✅' if CREWAI_AVAILABLE else '❌'}")
results = {}
# Test PraisonAI first
try:
print("\n" + "="*70)
print("Testing PraisonAI Agents")
print("="*70)
results['praisonai'] = run_benchmark(
instantiate_praisonai_agent,
"PraisonAI",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing PraisonAI: {e}")
try:
print("\n" + "="*70)
print("Testing PraisonAI Agents (LiteLLM)")
print("="*70)
results['praisonai_litellm'] = run_benchmark(
instantiate_praisonai_litellm_agent,
"PraisonAI (LiteLLM)",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing PraisonAI (LiteLLM): {e}")
# Test Agno
try:
print("\n" + "="*70)
print("Testing Agno")
print("="*70)
results['agno'] = run_benchmark(
instantiate_agno_agent,
"Agno",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing Agno: {e}")
# Test PydanticAI
if PYDANTIC_AVAILABLE:
try:
print("\n" + "="*70)
print("Testing PydanticAI")
print("="*70)
results['pydantic'] = run_benchmark(
instantiate_pydantic_agent,
"PydanticAI Agent Instantiation",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing PydanticAI: {e}")
# Test OpenAI Agents SDK
if OPENAI_AGENTS_AVAILABLE:
try:
print("\n" + "="*70)
print("Testing OpenAI Agents SDK")
print("="*70)
results['openai_agents'] = run_benchmark(
instantiate_openai_agents,
"OpenAI Agents SDK Instantiation",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing OpenAI Agents SDK: {e}")
# Test LangGraph
if LANGGRAPH_AVAILABLE:
try:
print("\n" + "="*70)
print("Testing LangGraph")
print("="*70)
results['langgraph'] = run_benchmark(
instantiate_langgraph_agent,
"LangGraph Agent Instantiation",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing LangGraph: {e}")
# Test CrewAI
if CREWAI_AVAILABLE:
try:
print("\n" + "="*70)
print("Testing CrewAI")
print("="*70)
results['crewai'] = run_benchmark(
instantiate_crewai_agent,
"CrewAI Agent Instantiation",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing CrewAI: {e}")
# Final Comparison Table
print("\n" + "="*70)
print("RESULTS SUMMARY")
print("="*70)
if 'praisonai' in results:
baseline = results['praisonai']
print(f"\n{'Framework':<25} {'Avg Time (μs)':<15} {'Avg Memory (KiB)':<18} {'Relative':<15}")
print("-" * 73)
# Sort by time
sorted_results = sorted(results.items(), key=lambda x: x[1].avg_run_time)
for name, result in sorted_results:
time_ratio = result.avg_run_time / baseline.avg_run_time if baseline.avg_run_time > 0 else 0
time_str = f"{time_ratio:.2f}x"
display_name = {
'agno': 'Agno',
'praisonai': 'PraisonAI',
'praisonai_litellm': 'PraisonAI (LiteLLM)',
'pydantic': 'PydanticAI',
'openai_agents': 'OpenAI Agents SDK',
'langgraph': 'LangGraph',
'crewai': 'CrewAI'
}.get(name, name)
print(f"{display_name:<25} {result.avg_run_time*1e6:<15.2f} {result.avg_memory_usage*1024:<18.2f} {time_str:<15}")
print("\n" + "="*70)
# Save results to files
save_benchmark_results(results, baseline)
@@ -0,0 +1,481 @@
#!/usr/bin/env python3
"""
PraisonAI Agents - Real-World Execution Benchmark
Benchmarks actual agent execution with LLM calls across frameworks.
Tests single agent, 2-agent, and 3-agent workflows.
Usage:
python benchmarks/real_benchmark.py
"""
import time
import statistics
import os
from typing import Literal, List
from dataclasses import dataclass, field
from importlib.metadata import version as get_version
# Ensure API key is set
if not os.environ.get('OPENAI_API_KEY'):
print("ERROR: OPENAI_API_KEY environment variable not set")
exit(1)
@dataclass
class ExecutionResult:
"""Execution benchmark results"""
times: List[float] = field(default_factory=list)
@property
def avg_time(self) -> float:
return statistics.mean(self.times) if self.times else 0
@property
def min_time(self) -> float:
return min(self.times) if self.times else 0
@property
def max_time(self) -> float:
return max(self.times) if self.times else 0
@property
def std_dev(self) -> float:
return statistics.stdev(self.times) if len(self.times) > 1 else 0
def get_package_versions():
"""Get version numbers for benchmarked packages."""
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
}
versions = {}
for display_name, pkg_name in packages.items():
try:
versions[display_name] = get_version(pkg_name)
except Exception:
versions[display_name] = 'not installed'
return versions
# ============================================================================
# Tool Definitions
# ============================================================================
def get_weather(city: Literal["nyc", "sf"]) -> str:
"""Get weather information for a city."""
if city == "nyc":
return "New York: Cloudy, 45°F, 60% humidity"
return "San Francisco: Sunny, 72°F, 40% humidity"
def get_time(timezone: Literal["est", "pst"]) -> str:
"""Get current time in a timezone."""
if timezone == "est":
return "Eastern Time: 2:30 PM"
return "Pacific Time: 11:30 AM"
def calculate(expression: str) -> str:
"""Calculate a math expression."""
try:
result = eval(expression)
return f"Result: {result}"
except Exception:
return "Error: Invalid expression"
# ============================================================================
# PraisonAI Benchmarks
# ============================================================================
def benchmark_praisonai_single(iterations: int = 3) -> ExecutionResult:
"""Benchmark single PraisonAI agent execution."""
from praisonaiagents import Agent
result = ExecutionResult()
for i in range(iterations):
print(f" PraisonAI single agent iteration {i+1}/{iterations}...")
agent = Agent(
name="Assistant",
instructions="You are a helpful assistant. Be very brief.",
llm="gpt-4o-mini",
tools=[get_weather],
verbose=False
)
start = time.perf_counter()
response = agent.start("What's the weather in NYC? Reply in one sentence.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
def benchmark_praisonai_two_agents(iterations: int = 3) -> ExecutionResult:
"""Benchmark 2 PraisonAI agents working together."""
from praisonaiagents import Agent, PraisonAIAgents
result = ExecutionResult()
for i in range(iterations):
print(f" PraisonAI 2-agent iteration {i+1}/{iterations}...")
researcher = Agent(
name="Researcher",
role="Research Assistant",
goal="Gather weather information",
instructions="Get weather data and pass to reporter. Be brief.",
llm="gpt-4o-mini",
tools=[get_weather],
verbose=False
)
reporter = Agent(
name="Reporter",
role="News Reporter",
goal="Create weather report",
instructions="Create a one-sentence weather summary.",
llm="gpt-4o-mini",
verbose=False
)
agents = PraisonAIAgents(
agents=[researcher, reporter],
process="sequential",
verbose=False
)
start = time.perf_counter()
response = agents.start("Get NYC weather and create a brief report.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
def benchmark_praisonai_three_agents(iterations: int = 3) -> ExecutionResult:
"""Benchmark 3 PraisonAI agents working together."""
from praisonaiagents import Agent, PraisonAIAgents
result = ExecutionResult()
for i in range(iterations):
print(f" PraisonAI 3-agent iteration {i+1}/{iterations}...")
weather_agent = Agent(
name="WeatherAgent",
role="Weather Specialist",
goal="Get weather data",
instructions="Fetch weather information. Be brief.",
llm="gpt-4o-mini",
tools=[get_weather],
verbose=False
)
time_agent = Agent(
name="TimeAgent",
role="Time Specialist",
goal="Get time information",
instructions="Fetch timezone information. Be brief.",
llm="gpt-4o-mini",
tools=[get_time],
verbose=False
)
summarizer = Agent(
name="Summarizer",
role="Summary Writer",
goal="Combine information",
instructions="Create a one-sentence summary combining weather and time.",
llm="gpt-4o-mini",
verbose=False
)
agents = PraisonAIAgents(
agents=[weather_agent, time_agent, summarizer],
process="sequential",
verbose=False
)
start = time.perf_counter()
response = agents.start("Get NYC weather and EST time, then summarize.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
# ============================================================================
# Agno Benchmarks
# ============================================================================
def benchmark_agno_single(iterations: int = 3) -> ExecutionResult:
"""Benchmark single Agno agent execution."""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
result = ExecutionResult()
for i in range(iterations):
print(f" Agno single agent iteration {i+1}/{iterations}...")
agent = Agent(
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["You are a helpful assistant. Be very brief."],
tools=[get_weather],
)
start = time.perf_counter()
response = agent.run("What's the weather in NYC? Reply in one sentence.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
def benchmark_agno_two_agents(iterations: int = 3) -> ExecutionResult:
"""Benchmark 2 Agno agents working together (using Team)."""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.team import Team
result = ExecutionResult()
for i in range(iterations):
print(f" Agno 2-agent iteration {i+1}/{iterations}...")
researcher = Agent(
name="Researcher",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Get weather data. Be brief."],
tools=[get_weather],
)
reporter = Agent(
name="Reporter",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Create a one-sentence weather summary."],
)
team = Team(
members=[researcher, reporter],
model=OpenAIChat(id="gpt-4o-mini"),
)
start = time.perf_counter()
response = team.run("Get NYC weather and create a brief report.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
def benchmark_agno_three_agents(iterations: int = 3) -> ExecutionResult:
"""Benchmark 3 Agno agents working together."""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.team import Team
result = ExecutionResult()
for i in range(iterations):
print(f" Agno 3-agent iteration {i+1}/{iterations}...")
weather_agent = Agent(
name="WeatherAgent",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Fetch weather information. Be brief."],
tools=[get_weather],
)
time_agent = Agent(
name="TimeAgent",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Fetch timezone information. Be brief."],
tools=[get_time],
)
summarizer = Agent(
name="Summarizer",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Create a one-sentence summary combining weather and time."],
)
team = Team(
members=[weather_agent, time_agent, summarizer],
model=OpenAIChat(id="gpt-4o-mini"),
)
start = time.perf_counter()
response = team.run("Get NYC weather and EST time, then summarize.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
# ============================================================================
# Save Results
# ============================================================================
def save_results(results: dict, versions: dict):
"""Save benchmark results to file."""
import os
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), 'REAL_BENCHMARK_RESULTS.md')
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Real-World Execution Benchmark\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write('**Model:** gpt-4o-mini\n')
f.write('**Iterations:** 3 per test\n\n')
f.write('## Results Summary\n\n')
f.write('| Test | PraisonAI (avg) | Agno (avg) | Difference |\n')
f.write('|------|-----------------|------------|------------|\n')
tests = ['single', 'two_agents', 'three_agents']
test_names = ['Single Agent', '2 Agents', '3 Agents']
for test, name in zip(tests, test_names):
praison_key = f'praisonai_{test}'
agno_key = f'agno_{test}'
if praison_key in results and agno_key in results:
praison_avg = results[praison_key].avg_time
agno_avg = results[agno_key].avg_time
if praison_avg < agno_avg:
diff = f'PraisonAI {agno_avg/praison_avg:.2f}x faster'
else:
diff = f'Agno {praison_avg/agno_avg:.2f}x faster'
f.write(f'| {name} | {praison_avg:.2f}s | {agno_avg:.2f}s | {diff} |\n')
f.write('\n## Detailed Results\n\n')
for key, result in results.items():
f.write(f'### {key}\n')
f.write(f'- Average: {result.avg_time:.2f}s\n')
f.write(f'- Min: {result.min_time:.2f}s\n')
f.write(f'- Max: {result.max_time:.2f}s\n')
f.write(f'- Std Dev: {result.std_dev:.2f}s\n')
f.write(f'- Times: {[f"{t:.2f}s" for t in result.times]}\n\n')
f.write('## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('export OPENAI_API_KEY=your_key\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/real_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
# ============================================================================
# Main
# ============================================================================
if __name__ == "__main__":
print("="*70)
print("PraisonAI Agents - Real-World Execution Benchmark")
print("="*70)
print("\nThis benchmark measures actual agent execution time including:")
print(" - LLM API calls")
print(" - Tool execution")
print(" - Multi-agent coordination")
print("\nModel: gpt-4o-mini")
print("Iterations: 3 per test")
versions = get_package_versions()
print(f"\nPackage versions:")
for pkg, ver in versions.items():
print(f" {pkg}: {ver}")
results = {}
# Single Agent Tests
print("\n" + "="*70)
print("SINGLE AGENT BENCHMARK")
print("="*70)
print("\nTesting PraisonAI single agent...")
results['praisonai_single'] = benchmark_praisonai_single()
print("\nTesting Agno single agent...")
results['agno_single'] = benchmark_agno_single()
# 2-Agent Tests
print("\n" + "="*70)
print("2-AGENT BENCHMARK")
print("="*70)
print("\nTesting PraisonAI 2 agents...")
results['praisonai_two_agents'] = benchmark_praisonai_two_agents()
print("\nTesting Agno 2 agents (Team)...")
results['agno_two_agents'] = benchmark_agno_two_agents()
# 3-Agent Tests
print("\n" + "="*70)
print("3-AGENT BENCHMARK")
print("="*70)
print("\nTesting PraisonAI 3 agents...")
results['praisonai_three_agents'] = benchmark_praisonai_three_agents()
print("\nTesting Agno 3 agents (Team)...")
results['agno_three_agents'] = benchmark_agno_three_agents()
# Summary
print("\n" + "="*70)
print("RESULTS SUMMARY")
print("="*70)
print(f"\n{'Test':<20} {'PraisonAI (avg)':<18} {'Agno (avg)':<15} {'Difference':<20}")
print("-" * 73)
tests = [
('Single Agent', 'praisonai_single', 'agno_single'),
('2 Agents', 'praisonai_two_agents', 'agno_two_agents'),
('3 Agents', 'praisonai_three_agents', 'agno_three_agents'),
]
for name, praison_key, agno_key in tests:
praison_avg = results[praison_key].avg_time
agno_avg = results[agno_key].avg_time
if praison_avg < agno_avg:
diff = f'PraisonAI {agno_avg/praison_avg:.2f}x faster'
else:
diff = f'Agno {praison_avg/agno_avg:.2f}x faster'
print(f"{name:<20} {praison_avg:<18.2f}s {agno_avg:<15.2f}s {diff:<20}")
print("\n" + "="*70)
# Save results
save_results(results, versions)
@@ -0,0 +1,235 @@
"""
PraisonAI Agents - Response Time Benchmark
Measures actual LLM response time with gpt-4o-mini across frameworks.
Usage:
python benchmarks/response_benchmark.py
"""
import time
import statistics
import os
from typing import List, Literal
from dataclasses import dataclass, field
# Ensure we use gpt-4o-mini
os.environ['OPENAI_MODEL_NAME'] = 'gpt-4o-mini'
@dataclass
class ResponseResult:
"""Response time measurement results"""
response_times: List[float] = field(default_factory=list)
@property
def avg_time(self) -> float:
return statistics.mean(self.response_times) if self.response_times else 0
@property
def min_time(self) -> float:
return min(self.response_times) if self.response_times else 0
@property
def max_time(self) -> float:
return max(self.response_times) if self.response_times else 0
@property
def std_dev(self) -> float:
return statistics.stdev(self.response_times) if len(self.response_times) > 1 else 0
def get_weather(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
def benchmark_praisonai_response(iterations: int = 5) -> ResponseResult:
"""Benchmark PraisonAI agent response time"""
from praisonaiagents import Agent
result = ResponseResult()
for i in range(iterations):
print(f" PraisonAI iteration {i+1}/{iterations}...")
# Create fresh agent each time
agent = Agent(
name="Test Agent",
instructions="Be concise, reply with one sentence.",
llm="gpt-4o-mini",
verbose=False
)
start = time.perf_counter()
response = agent.start("What is the capital of France?")
end = time.perf_counter()
result.response_times.append(end - start)
print(f" Response time: {end - start:.3f}s")
return result
def benchmark_agno_response(iterations: int = 5) -> ResponseResult:
"""Benchmark Agno agent response time"""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
result = ResponseResult()
for i in range(iterations):
print(f" Agno iteration {i+1}/{iterations}...")
# Create fresh agent each time
agent = Agent(
model=OpenAIChat(id="gpt-4o-mini"),
system_message="Be concise, reply with one sentence.",
)
start = time.perf_counter()
response = agent.run("What is the capital of France?")
end = time.perf_counter()
result.response_times.append(end - start)
print(f" Response time: {end - start:.3f}s")
return result
def benchmark_praisonai_with_tool(iterations: int = 5) -> ResponseResult:
"""Benchmark PraisonAI agent response time with tool calling"""
from praisonaiagents import Agent
result = ResponseResult()
for i in range(iterations):
print(f" PraisonAI (with tool) iteration {i+1}/{iterations}...")
agent = Agent(
name="Weather Agent",
instructions="You are a weather assistant. Use the get_weather tool to answer questions.",
llm="gpt-4o-mini",
tools=[get_weather],
verbose=False
)
start = time.perf_counter()
response = agent.start("What's the weather in NYC?")
end = time.perf_counter()
result.response_times.append(end - start)
print(f" Response time: {end - start:.3f}s")
return result
def benchmark_agno_with_tool(iterations: int = 5) -> ResponseResult:
"""Benchmark Agno agent response time with tool calling"""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
result = ResponseResult()
for i in range(iterations):
print(f" Agno (with tool) iteration {i+1}/{iterations}...")
agent = Agent(
model=OpenAIChat(id="gpt-4o-mini"),
system_message="You are a weather assistant. Use the get_weather tool to answer questions.",
tools=[get_weather],
)
start = time.perf_counter()
response = agent.run("What's the weather in NYC?")
end = time.perf_counter()
result.response_times.append(end - start)
print(f" Response time: {end - start:.3f}s")
return result
if __name__ == "__main__":
print("="*70)
print("PraisonAI Agents - Response Time Benchmark")
print("="*70)
print("\nModel: gpt-4o-mini")
print("Measures end-to-end response time including:")
print(" - Agent instantiation")
print(" - API call to OpenAI")
print(" - Response processing")
iterations = 5
results = {}
# Test 1: Simple response (no tools)
print("\n" + "="*70)
print("TEST 1: Simple Response (No Tools)")
print("="*70)
print("\nBenchmarking PraisonAI...")
try:
results['praisonai_simple'] = benchmark_praisonai_response(iterations)
except Exception as e:
print(f"Error: {e}")
import traceback
traceback.print_exc()
print("\nBenchmarking Agno...")
try:
results['agno_simple'] = benchmark_agno_response(iterations)
except Exception as e:
print(f"Error: {e}")
import traceback
traceback.print_exc()
# Test 2: Response with tool calling
print("\n" + "="*70)
print("TEST 2: Response with Tool Calling")
print("="*70)
print("\nBenchmarking PraisonAI with tool...")
try:
results['praisonai_tool'] = benchmark_praisonai_with_tool(iterations)
except Exception as e:
print(f"Error: {e}")
import traceback
traceback.print_exc()
print("\nBenchmarking Agno with tool...")
try:
results['agno_tool'] = benchmark_agno_with_tool(iterations)
except Exception as e:
print(f"Error: {e}")
import traceback
traceback.print_exc()
# Summary
print("\n" + "="*70)
print("SUMMARY RESULTS")
print("="*70)
print(f"\n{'Test':<30} {'PraisonAI (avg)':<20} {'Agno (avg)':<20} {'Ratio':<15}")
print("-" * 85)
if 'praisonai_simple' in results and 'agno_simple' in results:
p_time = results['praisonai_simple'].avg_time
a_time = results['agno_simple'].avg_time
ratio = p_time / a_time if a_time > 0 else float('inf')
print(f"{'Simple Response':<30} {p_time:<20.3f}s {a_time:<20.3f}s {ratio:.2f}x")
if 'praisonai_tool' in results and 'agno_tool' in results:
p_time = results['praisonai_tool'].avg_time
a_time = results['agno_tool'].avg_time
ratio = p_time / a_time if a_time > 0 else float('inf')
print(f"{'With Tool Calling':<30} {p_time:<20.3f}s {a_time:<20.3f}s {ratio:.2f}x")
print("\n" + "="*70)
print("NOTE")
print("="*70)
print("\nResponse time is dominated by OpenAI API latency.")
print("Agent instantiation overhead is negligible in real-world usage.")
@@ -0,0 +1,194 @@
❯ python benchmarks/performance_benchmark.py
/Users/praison/praisonai-package/src/praisonai-agents/praisonaiagents/_warning_patch.py:47: UserWarning: Mixing V1 models and V2 models (or constructs, like `TypeAdapter`) is not supported. Please upgrade `Settings` to V2.
_original_warn(message, category, stacklevel, source)
[13:04:15] INFO [13:04:15] config.py:50
config.py:50 INFO
Using config path:
/Users/praison/.con
fig/crewai/settings
.json
======================================================================
PraisonAI Agents - Comprehensive Performance Benchmark
======================================================================
This benchmark measures:
1. Agent instantiation time (how fast an agent object is created)
2. Memory footprint (how much memory an agent uses)
Methodology:
- 10 warmup runs (not measured)
- 1000 measured iterations
- tracemalloc for memory measurement
- Baseline subtraction for accuracy
Frameworks available:
- PraisonAI: ✅
- Agno: ✅
- PydanticAI: ✅
- OpenAI Agents SDK: ✅
- LangGraph: ✅
- CrewAI: ✅
======================================================================
Testing PraisonAI Agents
======================================================================
============================================================
Benchmarking: PraisonAI
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for PraisonAI:
Average Time: 0.000003 seconds (3.20 μs)
Min Time: 0.000003 seconds
Max Time: 0.000025 seconds
Std Dev: 0.000001 seconds
Median Time: 0.000003 seconds
Avg Memory: 0.003678 MiB (3.77 KiB)
======================================================================
Testing PraisonAI Agents (LiteLLM)
======================================================================
============================================================
Benchmarking: PraisonAI (LiteLLM)
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for PraisonAI (LiteLLM):
Average Time: 0.000007 seconds (7.31 μs)
Min Time: 0.000007 seconds
Max Time: 0.000060 seconds
Std Dev: 0.000002 seconds
Median Time: 0.000007 seconds
Avg Memory: 0.006419 MiB (6.57 KiB)
======================================================================
Testing Agno
======================================================================
============================================================
Benchmarking: Agno
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for Agno:
Average Time: 0.000005 seconds (5.30 μs)
Min Time: 0.000005 seconds
Max Time: 0.000039 seconds
Std Dev: 0.000002 seconds
Median Time: 0.000005 seconds
Avg Memory: 0.006599 MiB (6.76 KiB)
======================================================================
Testing PydanticAI
======================================================================
============================================================
Benchmarking: PydanticAI Agent Instantiation
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for PydanticAI Agent Instantiation:
Average Time: 0.000247 seconds (247.34 μs)
Min Time: 0.000218 seconds
Max Time: 0.000552 seconds
Std Dev: 0.000031 seconds
Median Time: 0.000237 seconds
Avg Memory: 0.028486 MiB (29.17 KiB)
======================================================================
Testing OpenAI Agents SDK
======================================================================
============================================================
Benchmarking: OpenAI Agents SDK Instantiation
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for OpenAI Agents SDK Instantiation:
Average Time: 0.000282 seconds (282.25 μs)
Min Time: 0.000247 seconds
Max Time: 0.001110 seconds
Std Dev: 0.000055 seconds
Median Time: 0.000263 seconds
Avg Memory: 0.029032 MiB (29.73 KiB)
======================================================================
Testing LangGraph
======================================================================
============================================================
Benchmarking: LangGraph Agent Instantiation
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for LangGraph Agent Instantiation:
Average Time: 0.003200 seconds (3200.15 μs)
Min Time: 0.002808 seconds
Max Time: 0.116363 seconds
Std Dev: 0.003628 seconds
Median Time: 0.002972 seconds
Avg Memory: 0.207509 MiB (212.49 KiB)
======================================================================
Testing CrewAI
======================================================================
============================================================
Benchmarking: CrewAI Agent Instantiation
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for CrewAI Agent Instantiation:
Average Time: 0.016772 seconds (16771.69 μs)
Min Time: 0.014404 seconds
Max Time: 0.091841 seconds
Std Dev: 0.003445 seconds
Median Time: 0.015816 seconds
Avg Memory: 0.065050 MiB (66.61 KiB)
======================================================================
RESULTS SUMMARY
======================================================================
Framework Avg Time (μs) Avg Memory (KiB) Relative
-------------------------------------------------------------------------
PraisonAI 3.20 3.77 1.00x
Agno 5.30 6.76 1.66x
PraisonAI (LiteLLM) 7.31 6.57 2.28x
PydanticAI 247.34 29.17 77.25x
OpenAI Agents SDK 282.25 29.73 88.15x
LangGraph 3200.15 212.49 999.46x
CrewAI 16771.69 66.61 5238.06x
======================================================================
❯
@@ -0,0 +1,243 @@
#!/usr/bin/env python3
"""
PraisonAI Agents - Performance Benchmark
Compares agent instantiation times across popular AI agent frameworks.
Usage:
python benchmarks/simple_benchmark.py
"""
import time
from typing import Literal
ITERATIONS = 100
def sample_tool(city: Literal['nyc', 'sf']):
"""Sample tool for benchmark testing."""
if city == 'nyc':
return 'cloudy'
return 'sunny'
def measure_instantiation(create_fn, iterations=ITERATIONS):
"""Measure average instantiation time in microseconds."""
times = []
for _ in range(iterations):
start = time.perf_counter()
create_fn()
times.append((time.perf_counter() - start) * 1_000_000)
return sum(times) / len(times)
def run_benchmark():
"""Run the benchmark across all available frameworks (without tools)."""
results = {}
print('=' * 60)
print('PraisonAI Agents - Performance Benchmark')
print('=' * 60)
print(f'\nIterations: {ITERATIONS}')
print('Metric: Agent instantiation time (microseconds)\n')
# PraisonAI (without tools)
print("Testing PraisonAI...")
from praisonaiagents import Agent as PraisonAgent
results['PraisonAI'] = measure_instantiation(
lambda: PraisonAgent(name='Test', llm='gpt-4o-mini', verbose=False)
)
results['PraisonAI (LiteLLM)'] = measure_instantiation(
lambda: PraisonAgent(name='Test', llm='openai/gpt-4o-mini', verbose=False)
)
# Other frameworks for comparison
print("Testing other frameworks...")
try:
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
results['Agno'] = measure_instantiation(
lambda: AgnoAgent(model=OpenAIChat(id='gpt-4o-mini'))
)
except ImportError:
pass
try:
from pydantic_ai import Agent as PydanticAgent
results['PydanticAI'] = measure_instantiation(
lambda: PydanticAgent('openai:gpt-4o-mini')
)
except ImportError:
pass
try:
from agents import Agent as OpenAIAgent
results['OpenAI Agents SDK'] = measure_instantiation(
lambda: OpenAIAgent(name='Test', model='gpt-4o-mini')
)
except ImportError:
pass
try:
from langchain_core.tools import tool as langchain_tool
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
@langchain_tool
def get_weather_lg(city: Literal['nyc', 'sf']):
"""Get weather info."""
return 'sunny' if city == 'sf' else 'cloudy'
results['LangGraph'] = measure_instantiation(
lambda: create_react_agent(model=ChatOpenAI(model='gpt-4o-mini'), tools=[get_weather_lg])
)
except ImportError:
pass
try:
from crewai.agent import Agent as CrewAgent
from crewai.tools import tool as crewai_tool
@crewai_tool("Weather Tool")
def get_weather_crew(city: Literal['nyc', 'sf']):
"""Get weather info."""
return 'sunny' if city == 'sf' else 'cloudy'
results['CrewAI'] = measure_instantiation(
lambda: CrewAgent(
role='Weather Agent',
goal='Provide weather info',
backstory='A weather expert',
tools=[get_weather_crew],
verbose=False
)
)
except ImportError:
pass
# Print results
print('\n' + '=' * 60)
print('RESULTS')
print('=' * 60)
baseline = results.get('PraisonAI', 1)
print(f"\n{'Framework':<25} {'Avg Time (μs)':<15} {'Relative':<10}")
print('-' * 50)
for name, avg in sorted(results.items(), key=lambda x: x[1]):
ratio = avg / baseline
print(f'{name:<25} {avg:<15.2f} {ratio:.2f}x')
print('\n' + '=' * 60)
return results
def get_package_versions():
"""Get version numbers for benchmarked packages."""
from importlib.metadata import version as get_version
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
'PydanticAI': 'pydantic-ai',
'OpenAI Agents SDK': 'openai-agents',
'LangGraph': 'langgraph',
'CrewAI': 'crewai'
}
versions = {}
for display_name, pkg_name in packages.items():
try:
versions[display_name] = get_version(pkg_name)
except Exception:
versions[display_name] = 'not installed'
return versions
def save_results(results: dict, filename: str = 'BENCHMARK_RESULTS.md'):
"""Save benchmark results to a markdown file and update README."""
import os
import re
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), filename)
versions = get_package_versions()
# Find the fastest result (baseline)
sorted_results = sorted(results.items(), key=lambda x: x[1])
fastest_time = sorted_results[0][1] if sorted_results else 1
# Build the table rows
table_rows = []
for name, avg in sorted_results:
ratio = avg / fastest_time
if ratio == 1.0:
table_rows.append(f'| **{name}** | **{avg:.2f}** | **1.00x (fastest)** |')
else:
avg_str = f'{avg:,.2f}' if avg >= 1000 else f'{avg:.2f}'
ratio_str = f'{ratio:,.0f}x' if ratio >= 100 else f'{ratio:.2f}x'
table_rows.append(f'| {name} | {avg_str} | {ratio_str} |')
# Save to BENCHMARK_RESULTS.md
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Benchmark Results\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write(f'**Iterations:** {ITERATIONS}\n')
f.write('**Test:** Agent instantiation (without tools)\n\n')
f.write('## Results\n\n')
f.write('| Framework | Avg Time (μs) | Relative |\n')
f.write('|-----------|---------------|----------|\n')
f.write('\n'.join(table_rows) + '\n')
f.write('\n## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/simple_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
# Also update the main README.md
readme_path = os.path.join(os.path.dirname(__file__), '..', '..', '..', 'README.md')
if os.path.exists(readme_path):
update_readme(readme_path, table_rows)
return filepath
def update_readme(readme_path: str, table_rows: list):
"""Update the performance section in README.md with latest results."""
import re
with open(readme_path, 'r') as f:
content = f.read()
# Build the new table
new_table = '''| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
''' + '\n'.join(table_rows)
# Pattern to match the performance table
pattern = r'(\| Framework \| Avg Time \(μs\) \| Relative \|\n\|[-|]+\|\n)(\|[^\n]+\|\n)+'
if re.search(pattern, content):
content = re.sub(pattern, new_table + '\n', content)
with open(readme_path, 'w') as f:
f.write(content)
print(f'README.md updated: {readme_path}')
if __name__ == '__main__':
results = run_benchmark()
save_results(results)
@@ -0,0 +1,213 @@
#!/usr/bin/env python3
"""
PraisonAI Agents - Tools Benchmark
Compares agent instantiation times WITH TOOLS across frameworks.
Usage:
python benchmarks/tools_benchmark.py
"""
import time
from typing import Literal
from importlib.metadata import version as get_version
ITERATIONS = 100
def sample_tool(city: Literal['nyc', 'sf']):
"""Sample tool for benchmark testing."""
if city == 'nyc':
return 'cloudy'
return 'sunny'
def measure_instantiation(create_fn, iterations=ITERATIONS):
"""Measure average instantiation time in microseconds."""
times = []
# Warmup
for _ in range(10):
create_fn()
# Measure
for _ in range(iterations):
start = time.perf_counter()
create_fn()
elapsed = time.perf_counter() - start
times.append(elapsed * 1_000_000) # Convert to microseconds
return sum(times) / len(times)
def run_benchmark():
"""Run the benchmark across all available frameworks WITH TOOLS."""
tools = [sample_tool]
results = {}
print('=' * 60)
print('PraisonAI Agents - Tools Benchmark')
print('=' * 60)
print(f'\nIterations: {ITERATIONS}')
print('Metric: Agent instantiation time WITH TOOLS (microseconds)\n')
# PraisonAI with tools
print("Testing PraisonAI...")
from praisonaiagents import Agent as PraisonAgent
results['PraisonAI'] = measure_instantiation(
lambda: PraisonAgent(name='Test', llm='gpt-4o-mini', tools=tools, verbose=False)
)
results['PraisonAI (LiteLLM)'] = measure_instantiation(
lambda: PraisonAgent(name='Test', llm='openai/gpt-4o-mini', tools=tools, verbose=False)
)
# Other frameworks with tools
print("Testing other frameworks...")
try:
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
results['Agno'] = measure_instantiation(
lambda: AgnoAgent(model=OpenAIChat(id='gpt-4o-mini'), tools=tools)
)
except ImportError:
pass
try:
from agents import Agent as OpenAIAgent, function_tool
results['OpenAI Agents SDK'] = measure_instantiation(
lambda: OpenAIAgent(name='Test', model='gpt-4o-mini', tools=[function_tool(sample_tool)])
)
except ImportError:
pass
try:
from langchain_core.tools import tool as langchain_tool
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
@langchain_tool
def get_weather_lg(city: Literal['nyc', 'sf']):
"""Get weather info."""
return 'sunny' if city == 'sf' else 'cloudy'
results['LangGraph'] = measure_instantiation(
lambda: create_react_agent(model=ChatOpenAI(model='gpt-4o-mini'), tools=[get_weather_lg])
)
except ImportError:
pass
try:
from crewai.agent import Agent as CrewAgent
from crewai.tools import tool as crewai_tool
@crewai_tool("Weather Tool")
def get_weather_crew(city: Literal['nyc', 'sf']):
"""Get weather info."""
return 'sunny' if city == 'sf' else 'cloudy'
results['CrewAI'] = measure_instantiation(
lambda: CrewAgent(
role='Weather Agent',
goal='Provide weather info',
backstory='A weather expert',
tools=[get_weather_crew],
verbose=False
)
)
except ImportError:
pass
# Print results
print('\n' + '=' * 60)
print('RESULTS (WITH TOOLS)')
print('=' * 60)
sorted_results = sorted(results.items(), key=lambda x: x[1])
baseline = sorted_results[0][1] if sorted_results else 1
print(f"\n{'Framework':<25} {'Avg Time (μs)':<15} {'Relative':<10}")
print('-' * 50)
for name, avg in sorted_results:
ratio = avg / baseline
print(f'{name:<25} {avg:<15.2f} {ratio:.2f}x')
print('\n' + '=' * 60)
return results
def get_package_versions():
"""Get version numbers for benchmarked packages."""
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
'OpenAI Agents SDK': 'openai-agents',
'LangGraph': 'langgraph',
'CrewAI': 'crewai'
}
versions = {}
for display_name, pkg_name in packages.items():
try:
versions[display_name] = get_version(pkg_name)
except Exception:
versions[display_name] = 'not installed'
return versions
def save_results(results: dict, filename: str = 'TOOLS_BENCHMARK_RESULTS.md'):
"""Save benchmark results to a markdown file."""
import os
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), filename)
versions = get_package_versions()
# Find the fastest result (baseline)
sorted_results = sorted(results.items(), key=lambda x: x[1])
fastest_time = sorted_results[0][1] if sorted_results else 1
# Build the table rows
table_rows = []
for name, avg in sorted_results:
ratio = avg / fastest_time
if ratio == 1.0:
table_rows.append(f'| **{name}** | **{avg:.2f}** | **1.00x (fastest)** |')
else:
avg_str = f'{avg:,.2f}' if avg >= 1000 else f'{avg:.2f}'
ratio_str = f'{ratio:,.0f}x' if ratio >= 100 else f'{ratio:.2f}x'
table_rows.append(f'| {name} | {avg_str} | {ratio_str} |')
# Save to file
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Tools Benchmark Results\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write(f'**Iterations:** {ITERATIONS}\n')
f.write('**Test:** Agent instantiation WITH TOOLS\n\n')
f.write('## Results\n\n')
f.write('| Framework | Avg Time (μs) | Relative |\n')
f.write('|-----------|---------------|----------|\n')
f.write('\n'.join(table_rows) + '\n')
f.write('\n## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/tools_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
return filepath
if __name__ == '__main__':
results = run_benchmark()
save_results(results)
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,9 @@
Starting test with verbose=False
CALLBACK EXECUTED!
Message:
You need to do the following task: Say hello world.
Expected Output: A greeting.
Please provide only the final result of your work. Do not add any conversation or extra explanation.
Response: Hello, World!
Other kwargs: {'markdown': True, 'generation_time': 2.685833215713501, 'agent_name': 'TestAgent', 'agent_role': 'Assistant', 'agent_tools': None, 'task_name': 'simple_task', 'task_description': 'Say hello world', 'task_id': 0}
--------------------------------------------------
@@ -551,8 +551,9 @@ Your Goal: {self.goal}
self.auto_save = auto_save # Session name for auto-saving
# Initialize rules manager for persistent context (like Cursor/Windsurf)
# NOTE: Lazy initialization - rules are loaded only when accessed (performance optimization)
self._rules_manager = None
self._init_rules_manager()
self._rules_manager_initialized = False
# Handle web_search fallback: inject DuckDuckGo tool for unsupported models
if web_search and not self._model_supports_web_search():
@@ -599,7 +600,7 @@ Your Goal: {self.goal}
# Fast Context configuration (lazy loaded)
self.fast_context_enabled = fast_context
self.fast_context_path = fast_context_path or os.getcwd()
self._fast_context_path = fast_context_path # Store raw, resolve lazily
self.fast_context_model = fast_context_model
self.fast_context_max_turns = fast_context_max_turns
self.fast_context_parallelism = fast_context_parallelism
@@ -614,6 +615,17 @@ Your Goal: {self.goal}
self._console = Console()
return self._console
@property
def fast_context_path(self):
"""Lazily resolve fast_context_path - avoids os.getcwd() on every init."""
if self._fast_context_path is None:
self._fast_context_path = os.getcwd()
return self._fast_context_path
@fast_context_path.setter
def fast_context_path(self, value):
self._fast_context_path = value
@property
def _openai_client(self):
"""Lazily initialize OpenAI client only when needed."""
@@ -785,6 +797,21 @@ Your Goal: {self.goal}
logging.warning(f"FastContext search failed: {e}")
return None
@property
def rules_manager(self):
"""
Lazy-initialized RulesManager for persistent rules/instructions.
This property initializes the RulesManager only when first accessed,
avoiding expensive filesystem operations during agent instantiation.
Returns:
RulesManager instance or None if not available
"""
if not self._rules_manager_initialized:
self._init_rules_manager()
return self._rules_manager
def _init_rules_manager(self):
"""
Initialize RulesManager for persistent rules/instructions.
@@ -793,7 +820,10 @@ Your Goal: {self.goal}
- ~/.praison/rules/ (global)
- .praison/rules/ (workspace)
- Subdirectory rules
NOTE: This is called lazily via the rules_manager property for performance.
"""
self._rules_manager_initialized = True
try:
from ..memory.rules_manager import RulesManager
import os
@@ -828,10 +858,10 @@ Your Goal: {self.goal}
Returns:
Formatted rules context string
"""
if not self._rules_manager:
if not self.rules_manager:
return ""
return self._rules_manager.build_rules_context(
return self.rules_manager.build_rules_context(
file_path=file_path,
include_manual=include_manual
)
@@ -1239,8 +1269,8 @@ Your Goal: {self.goal}
Your Role: {self.role}\n
Your Goal: {self.goal}"""
# Add rules context if rules manager is enabled
if self._rules_manager:
# Add rules context if rules manager is enabled (lazy initialization)
if self._rules_manager_initialized and self._rules_manager:
rules_context = self.get_rules_context()
if rules_context:
system_prompt += f"\n\n## Rules (Guidelines you must follow)\n{rules_context}"
@@ -1539,23 +1569,54 @@ Your Goal: {self.goal}"""
# Special handling for MCP tools
# Check if tools is an MCP instance with the requested function name
from ..mcp.mcp import MCP
# Helper function to execute MCP tool
def _execute_mcp_tool(mcp_instance, func_name, args):
"""Execute a tool from an MCP instance."""
# Handle SSE MCP client
if hasattr(mcp_instance, 'is_sse') and mcp_instance.is_sse:
if hasattr(mcp_instance, 'sse_client'):
for tool in mcp_instance.sse_client.tools:
if tool.name == func_name:
logging.debug(f"Found matching SSE MCP tool: {func_name}")
return True, tool(**args)
# Handle HTTP Stream MCP client
if hasattr(mcp_instance, 'is_http_stream') and mcp_instance.is_http_stream:
if hasattr(mcp_instance, 'http_stream_client'):
for tool in mcp_instance.http_stream_client.tools:
if tool.name == func_name:
logging.debug(f"Found matching HTTP Stream MCP tool: {func_name}")
return True, tool(**args)
# Handle WebSocket MCP client
if hasattr(mcp_instance, 'is_websocket') and mcp_instance.is_websocket:
if hasattr(mcp_instance, 'websocket_client'):
for tool in mcp_instance.websocket_client.tools:
if tool.name == func_name:
logging.debug(f"Found matching WebSocket MCP tool: {func_name}")
return True, tool(**args)
# Handle stdio MCP client
if hasattr(mcp_instance, 'runner'):
for mcp_tool in mcp_instance.runner.tools:
if hasattr(mcp_tool, 'name') and mcp_tool.name == func_name:
logging.debug(f"Found matching MCP tool: {func_name}")
return True, mcp_instance.runner.call_tool(func_name, args)
return False, None
# Check if tools is a single MCP instance
if isinstance(self.tools, MCP):
logging.debug(f"Looking for MCP tool {function_name}")
# Handle SSE MCP client
if hasattr(self.tools, 'is_sse') and self.tools.is_sse:
if hasattr(self.tools, 'sse_client'):
for tool in self.tools.sse_client.tools:
if tool.name == function_name:
logging.debug(f"Found matching SSE MCP tool: {function_name}")
return tool(**arguments)
# Handle stdio MCP client
elif hasattr(self.tools, 'runner'):
# Check if any of the MCP tools match the function name
for mcp_tool in self.tools.runner.tools:
if hasattr(mcp_tool, 'name') and mcp_tool.name == function_name:
logging.debug(f"Found matching MCP tool: {function_name}")
return self.tools.runner.call_tool(function_name, arguments)
found, result = _execute_mcp_tool(self.tools, function_name, arguments)
if found:
return result
# Check if tools is a list that may contain MCP instances
if isinstance(self.tools, (list, tuple)):
for tool in self.tools:
if isinstance(tool, MCP):
logging.debug(f"Looking for MCP tool {function_name} in MCP instance")
found, result = _execute_mcp_tool(tool, function_name, arguments)
if found:
return result
# Try to find the function in the agent's tools list first
func = None
@@ -1870,7 +1931,8 @@ Your Goal: {self.goal}"""
if tool_param is not None:
from ..mcp.mcp import MCP
if isinstance(tool_param, MCP) and hasattr(tool_param, 'to_openai_tool'):
logging.debug("Converting MCP tool to OpenAI format")
# Single MCP instance
logging.debug("Converting single MCP tool to OpenAI format")
openai_tool = tool_param.to_openai_tool()
if openai_tool:
# Handle both single tool and list of tools
@@ -1879,6 +1941,22 @@ Your Goal: {self.goal}"""
else:
tool_param = [openai_tool]
logging.debug(f"Converted MCP tool: {tool_param}")
elif isinstance(tool_param, (list, tuple)):
# List that may contain MCP instances - convert each MCP to OpenAI format
converted_tools = []
for t in tool_param:
if isinstance(t, MCP) and hasattr(t, 'to_openai_tool'):
logging.debug("Converting MCP instance in list to OpenAI format")
openai_tools = t.to_openai_tool()
if isinstance(openai_tools, list):
converted_tools.extend(openai_tools)
elif openai_tools:
converted_tools.append(openai_tools)
else:
# Keep non-MCP tools as-is
converted_tools.append(t)
tool_param = converted_tools
logging.debug(f"Converted {len(converted_tools)} tools from list")
# Store chat history length for potential rollback
chat_history_length = len(self.chat_history)
@@ -1370,6 +1370,61 @@ Context:
agent_names = ", ".join([agent.name for agent in self.agents])
print(f"📊 Available agents for this endpoint ({len(self.agents)}): {agent_names}")
# Create per-agent endpoints for individual agent access
# This allows n8n and other tools to call specific agents
agents_dict = {agent.name.lower().replace(' ', '_'): agent for agent in self.agents}
# Add GET endpoint to list available agents
@_agents_shared_apps[port].get(f"{path}/list")
async def list_agents():
return {
"agents": [
{"name": agent.name, "id": agent.name.lower().replace(' ', '_')}
for agent in self.agents
]
}
# Add per-agent POST endpoints
for agent_id, agent_instance in agents_dict.items():
agent_path = f"{path}/{agent_id}"
# Create a closure to capture the agent instance
def create_agent_handler(agent):
async def handle_single_agent(request: Request):
try:
request_data = await request.json()
query = request_data.get("query", "")
if not query:
raise HTTPException(status_code=400, detail="Missing 'query' field")
except:
raise HTTPException(status_code=400, detail="Invalid JSON body")
try:
if asyncio.iscoroutinefunction(agent.chat):
response = await agent.achat(query)
else:
loop = asyncio.get_running_loop()
response = await loop.run_in_executor(None, lambda q=query: agent.chat(q))
return {
"agent": agent.name,
"query": query,
"response": response
}
except Exception as e:
logging.error(f"Error with agent {agent.name}: {str(e)}", exc_info=True)
return JSONResponse(
status_code=500,
content={"error": f"Agent error: {str(e)}"}
)
return handle_single_agent
# Register the endpoint
_agents_shared_apps[port].post(agent_path)(create_agent_handler(agent_instance))
_agents_registered_endpoints[port][agent_path] = f"{endpoint_id}_{agent_id}"
print(f"🔗 Per-agent endpoints: {', '.join([f'{path}/{aid}' for aid in agents_dict.keys()])}")
# Start the server if it's not already running for this port
if not _agents_server_started.get(port, False):
# Mark the server as started first to prevent duplicate starts
@@ -440,7 +440,7 @@ class Workflow:
previous_output = output
if verbose:
print(f"✅ {step.name}: {str(output)[:100]}...")
print(f"✅ {step.name}: {str(output)}")
# Handle early stop
if stop:
@@ -571,7 +571,7 @@ class Workflow:
output = f"Error: {e}"
if verbose:
print(f"✅ {normalized.name}: {str(output)[:100]}...")
print(f"✅ {normalized.name}: {str(output)}")
return {
"step": normalized.name,
@@ -286,6 +286,7 @@ class Workflow:
planning_llm: Optional[str] = None # LLM for planning
reasoning: bool = False # Enable chain-of-thought reasoning
verbose: bool = False # Enable verbose output
stream: bool = True # Enable streaming responses (default True like PraisonAIAgents)
# Callbacks (like process="workflow")
on_workflow_start: Optional[Callable[['Workflow', str], None]] = None # (workflow, input)
@@ -316,7 +317,8 @@ class Workflow:
self,
input: str = "",
llm: Optional[str] = None,
verbose: bool = False
verbose: bool = False,
stream: bool = None
) -> Dict[str, Any]:
"""
Run the workflow with the given input.
@@ -340,16 +342,22 @@ class Workflow:
input: The input text/prompt for the workflow
llm: LLM model to use (default: gpt-4o-mini)
verbose: Print step outputs
stream: Enable streaming responses (default: use workflow's stream setting)
Returns:
Dict with 'output' (final result) and 'steps' (all step results)
"""
# Use default LLM if not specified
model = llm or self.default_llm or "gpt-4o-mini"
logger.debug(f"Workflow using model: {model} (llm={llm}, default_llm={self.default_llm})")
# Use workflow verbose setting if not overridden
verbose = verbose or self.verbose
# Use workflow stream setting if not overridden
if stream is None:
stream = self.stream
# Add input to variables
all_variables = {**self.variables, "input": input}
@@ -382,7 +390,7 @@ class Workflow:
if isinstance(step, Route):
# Decision-based routing
route_result = self._execute_route(
step, previous_output, input, all_variables, model, verbose
step, previous_output, input, all_variables, model, verbose, stream
)
results.extend(route_result["steps"])
previous_output = route_result["output"]
@@ -393,7 +401,7 @@ class Workflow:
elif isinstance(step, Parallel):
# Parallel execution
parallel_result = self._execute_parallel(
step, previous_output, input, all_variables, model, verbose
step, previous_output, input, all_variables, model, verbose, stream
)
results.extend(parallel_result["steps"])
previous_output = parallel_result["output"]
@@ -404,7 +412,7 @@ class Workflow:
elif isinstance(step, Loop):
# Loop over items
loop_result = self._execute_loop(
step, previous_output, input, all_variables, model, verbose
step, previous_output, input, all_variables, model, verbose, stream
)
results.extend(loop_result["steps"])
previous_output = loop_result["output"]
@@ -415,7 +423,7 @@ class Workflow:
elif isinstance(step, Repeat):
# Repeat until condition
repeat_result = self._execute_repeat(
step, previous_output, input, all_variables, model, verbose
step, previous_output, input, all_variables, model, verbose, stream
)
results.extend(repeat_result["steps"])
previous_output = repeat_result["output"]
@@ -494,7 +502,11 @@ class Workflow:
for key, value in all_variables.items():
action = action.replace(f"{{{{{key}}}}}", str(value))
if previous_output:
action = action.replace("{{previous_output}}", str(previous_output))
if "{{previous_output}}" in action:
action = action.replace("{{previous_output}}", str(previous_output))
else:
# Auto-append context if not explicitly referenced
action = f"{action}\n\nContext from previous step:\n{previous_output}"
action = action.replace("{{input}}", input)
# Add reasoning prompt if enabled
@@ -508,9 +520,9 @@ class Workflow:
# Handle images if present
step_images = getattr(step, 'images', None)
if step_images:
output = step.agent.chat(action, images=step_images)
output = step.agent.chat(action, images=step_images, stream=stream)
else:
output = step.agent.chat(action)
output = step.agent.chat(action, stream=stream)
# Handle output_pydantic if present
output_pydantic = getattr(step, 'output_pydantic', None)
@@ -538,14 +550,19 @@ class Workflow:
llm=config.get("llm", model),
tools=step_tools if step_tools else None,
verbose=verbose,
reasoning=self.reasoning
reasoning=self.reasoning,
stream=stream
)
# Substitute variables in action
action = step.action
for key, value in all_variables.items():
action = action.replace(f"{{{{{key}}}}}", str(value))
if previous_output:
action = action.replace("{{previous_output}}", str(previous_output))
if "{{previous_output}}" in action:
action = action.replace("{{previous_output}}", str(previous_output))
else:
# Auto-append context if not explicitly referenced
action = f"{action}\n\nContext from previous step:\n{previous_output}"
action = action.replace("{{input}}", input)
# Add reasoning prompt if enabled
@@ -554,7 +571,8 @@ class Workflow:
if validation_feedback:
action = f"{action}\n\nPrevious attempt feedback: {validation_feedback}"
output = temp_agent.chat(action)
output = temp_agent.chat(action, stream=stream)
except Exception as e:
step_error = e
@@ -629,7 +647,7 @@ class Workflow:
previous_output = output
if verbose:
print(f"✅ {step.name}: {str(output)[:100]}...")
print(f"✅ {step.name}: {str(output)}")
# Handle early stop
if stop:
@@ -828,7 +846,8 @@ Create a brief execution plan (2-3 sentences) describing how to best accomplish
all_variables: Dict[str, Any],
model: str,
verbose: bool,
index: int = 0
index: int = 0,
stream: bool = True
) -> Dict[str, Any]:
"""Execute a single step and return result."""
normalized = self._normalize_single_step(step, index)
@@ -862,9 +881,13 @@ Create a brief execution plan (2-3 sentences) describing how to best accomplish
for key, value in all_variables.items():
action = action.replace(f"{{{{{key}}}}}", str(value))
if previous_output:
action = action.replace("{{previous_output}}", str(previous_output))
if "{{previous_output}}" in action:
action = action.replace("{{previous_output}}", str(previous_output))
else:
# Auto-append context if not explicitly referenced
action = f"{action}\n\nContext from previous step:\n{previous_output}"
action = action.replace("{{input}}", input)
output = normalized.agent.chat(action)
output = normalized.agent.chat(action, stream=stream)
except Exception as e:
output = f"Error: {e}"
elif normalized.action:
@@ -876,19 +899,25 @@ Create a brief execution plan (2-3 sentences) describing how to best accomplish
role=config.get("role", "Assistant"),
goal=config.get("goal", "Complete the task"),
llm=config.get("llm", model),
verbose=verbose
verbose=verbose,
stream=stream
)
action = normalized.action
for key, value in all_variables.items():
action = action.replace(f"{{{{{key}}}}}", str(value))
if previous_output:
action = action.replace("{{previous_output}}", str(previous_output))
output = temp_agent.chat(action)
if "{{previous_output}}" in action:
action = action.replace("{{previous_output}}", str(previous_output))
else:
# Auto-append context if not explicitly referenced
action = f"{action}\n\nContext from previous step:\n{previous_output}"
output = temp_agent.chat(action, stream=stream)
except Exception as e:
output = f"Error: {e}"
if verbose:
print(f"✅ {normalized.name}: {str(output)[:100]}...")
print(f"✅ {normalized.name}: {str(output)}")
return {
"step": normalized.name,
@@ -904,7 +933,8 @@ Create a brief execution plan (2-3 sentences) describing how to best accomplish
input: str,
all_variables: Dict[str, Any],
model: str,
verbose: bool
verbose: bool,
stream: bool = True
) -> Dict[str, Any]:
"""Execute routing based on previous output."""
results = []
@@ -930,7 +960,7 @@ Create a brief execution plan (2-3 sentences) describing how to best accomplish
# Execute matched route steps
for idx, step in enumerate(matched_route):
step_result = self._execute_single_step_internal(
step, output, input, all_variables, model, verbose, idx
step, output, input, all_variables, model, verbose, idx, stream=stream
)
results.append({"step": step_result["step"], "output": step_result["output"]})
output = step_result["output"]
@@ -948,7 +978,8 @@ Create a brief execution plan (2-3 sentences) describing how to best accomplish
input: str,
all_variables: Dict[str, Any],
model: str,
verbose: bool
verbose: bool,
stream: bool = True
) -> Dict[str, Any]:
"""Execute steps in parallel (simulated with sequential for now)."""
import concurrent.futures
@@ -965,7 +996,7 @@ Create a brief execution plan (2-3 sentences) describing how to best accomplish
for idx, step in enumerate(parallel_step.steps):
future = executor.submit(
self._execute_single_step_internal,
step, previous_output, input, all_variables.copy(), model, False, idx
step, previous_output, input, all_variables.copy(), model, False, idx, stream
)
futures.append((idx, future))
@@ -994,7 +1025,8 @@ Create a brief execution plan (2-3 sentences) describing how to best accomplish
input: str,
all_variables: Dict[str, Any],
model: str,
verbose: bool
verbose: bool,
stream: bool = True
) -> Dict[str, Any]:
"""Execute step for each item in loop."""
import csv
@@ -1034,7 +1066,7 @@ Create a brief execution plan (2-3 sentences) describing how to best accomplish
loop_vars["loop_index"] = idx
step_result = self._execute_single_step_internal(
loop_step.step, previous_output, input, loop_vars, model, verbose, idx
loop_step.step, previous_output, input, loop_vars, model, verbose, idx, stream=stream
)
results.append({"step": f"{step_result['step']}_{idx}", "output": step_result["output"]})
outputs.append(step_result["output"])
@@ -1052,7 +1084,8 @@ Create a brief execution plan (2-3 sentences) describing how to best accomplish
input: str,
all_variables: Dict[str, Any],
model: str,
verbose: bool
verbose: bool,
stream: bool = True
) -> Dict[str, Any]:
"""Repeat step until condition is met."""
results = []
@@ -1063,7 +1096,7 @@ Create a brief execution plan (2-3 sentences) describing how to best accomplish
for iteration in range(repeat_step.max_iterations):
step_result = self._execute_single_step_internal(
repeat_step.step, output, input, all_variables, model, verbose, iteration
repeat_step.step, output, input, all_variables, model, verbose, iteration, stream=stream
)
results.append({"step": f"{step_result['step']}_{iteration}", "output": step_result["output"]})
output = step_result["output"]
@@ -131,11 +131,18 @@ class YAMLWorkflowParser:
normalized = data.copy()
# 1. Normalize workflow name: topic -> name
if 'topic' in normalized and 'name' not in normalized:
# 1. Normalize workflow name: topic -> name (only if topic is used as name, not as input)
# Note: 'topic' is legacy, 'input' is canonical for workflow input data
if 'topic' in normalized and 'name' not in normalized and 'input' not in normalized:
# Legacy behavior: topic was used as both name and input
normalized['name'] = normalized.get('topic', 'Unnamed Workflow')
# 2. Normalize container: roles -> agents
# 2. Normalize workflow input: 'input' is canonical, 'topic' is alias
# Priority: input > topic (input takes precedence if both exist)
if 'input' not in normalized and 'topic' in normalized:
normalized['input'] = normalized['topic']
# 3. Normalize container: roles -> agents
# Note: _convert_roles_to_agents handles backstory -> instructions
if 'roles' in normalized and 'agents' not in normalized:
normalized['agents'] = self._convert_roles_to_agents(normalized['roles'])
@@ -143,7 +150,7 @@ class YAMLWorkflowParser:
if 'steps' not in normalized:
normalized['steps'] = self._extract_steps_from_roles(normalized['roles'])
# 3. Normalize agent fields: backstory -> instructions (for agents section)
# 4. Normalize agent fields: backstory -> instructions (for agents section)
if 'agents' in normalized:
for agent_id, agent_config in normalized['agents'].items():
if isinstance(agent_config, dict):
@@ -151,7 +158,7 @@ class YAMLWorkflowParser:
if 'backstory' in agent_config and 'instructions' not in agent_config:
agent_config['instructions'] = agent_config['backstory']
# 4. Normalize step fields: description -> action
# 5. Normalize step fields: description -> action
if 'steps' in normalized:
for step in normalized['steps']:
if isinstance(step, dict):
@@ -225,6 +232,7 @@ class YAMLWorkflowParser:
workflow_config = data.get('workflow', {})
planning = workflow_config.get('planning', False)
planning_llm = workflow_config.get('planning_llm')
default_llm = workflow_config.get('default_llm')
reasoning = workflow_config.get('reasoning', False)
verbose = workflow_config.get('verbose', False)
memory_config = workflow_config.get('memory_config')
@@ -254,6 +262,7 @@ class YAMLWorkflowParser:
variables=variables,
planning=planning,
planning_llm=planning_llm,
default_llm=default_llm,
reasoning=reasoning,
verbose=verbose,
memory_config=memory_config,
@@ -268,6 +277,10 @@ class YAMLWorkflowParser:
workflow.process = process
workflow.manager_llm = manager_llm
# Store workflow input (from 'input' or 'topic' field)
# This is the default input passed to workflow.start() if no input is provided
workflow.default_input = data.get('input', data.get('topic', ''))
return workflow
def _convert_roles_to_agents(self, roles: Dict[str, Dict]) -> Dict[str, Dict]:
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "praisonaiagents"
version = "0.1.7"
version = "0.1.14"
description = "Praison AI agents for completing complex tasks with Self Reflection Agents"
readme = "README.md"
requires-python = ">=3.10"
@@ -0,0 +1,391 @@
"""
Test-Driven Development tests for Multiple MCP Tools support.
This test file verifies that:
1. Single MCP instance works (baseline)
2. Multiple MCP instances in a list work for tool definition conversion
3. Multiple MCP instances in a list work for tool execution
4. Mixed tools (MCP + regular functions) work together
Uses mcp-server-time as the test MCP server (simple, no API keys needed).
"""
import pytest
import os
import sys
from unittest.mock import Mock
# Add the package to path for testing
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
from praisonaiagents.mcp.mcp import MCP
from praisonaiagents.agent.agent import Agent
# ============================================================================
# Test Fixtures
# ============================================================================
@pytest.fixture
def time_mcp():
"""Create a Time MCP server instance."""
return MCP("uvx mcp-server-time", timeout=30)
@pytest.fixture
def mock_mcp_with_tools():
"""Create a mock MCP instance with predefined tools."""
mock = Mock(spec=MCP)
mock.is_sse = False
mock.is_http_stream = False
mock.is_websocket = False
# Create mock runner with tools
mock.runner = Mock()
mock_tool1 = Mock()
mock_tool1.name = "get_current_time"
mock_tool1.description = "Get current time in a timezone"
mock_tool1.inputSchema = {
"type": "object",
"properties": {
"timezone": {"type": "string", "description": "IANA timezone name"}
},
"required": ["timezone"]
}
mock_tool2 = Mock()
mock_tool2.name = "convert_time"
mock_tool2.description = "Convert time between timezones"
mock_tool2.inputSchema = {
"type": "object",
"properties": {
"source_timezone": {"type": "string"},
"time": {"type": "string"},
"target_timezone": {"type": "string"}
},
"required": ["source_timezone", "time", "target_timezone"]
}
mock.runner.tools = [mock_tool1, mock_tool2]
mock._tools = []
# Mock to_openai_tool method
def to_openai_tool():
return [
{
"type": "function",
"function": {
"name": "get_current_time",
"description": "Get current time in a timezone",
"parameters": mock_tool1.inputSchema
}
},
{
"type": "function",
"function": {
"name": "convert_time",
"description": "Convert time between timezones",
"parameters": mock_tool2.inputSchema
}
}
]
mock.to_openai_tool = to_openai_tool
# Mock __iter__ to return tool functions
def get_current_time(timezone: str) -> str:
return f"Time in {timezone}: 12:00"
def convert_time(source_timezone: str, time: str, target_timezone: str) -> str:
return f"Converted {time} from {source_timezone} to {target_timezone}"
get_current_time.__name__ = "get_current_time"
convert_time.__name__ = "convert_time"
mock._tools = [get_current_time, convert_time]
mock.__iter__ = lambda self: iter(self._tools)
return mock
@pytest.fixture
def mock_mcp_filesystem():
"""Create a mock MCP instance simulating filesystem server."""
mock = Mock(spec=MCP)
mock.is_sse = False
mock.is_http_stream = False
mock.is_websocket = False
mock.runner = Mock()
mock_tool = Mock()
mock_tool.name = "read_file"
mock_tool.description = "Read a file"
mock_tool.inputSchema = {
"type": "object",
"properties": {
"path": {"type": "string"}
},
"required": ["path"]
}
mock.runner.tools = [mock_tool]
mock._tools = []
def to_openai_tool():
return [
{
"type": "function",
"function": {
"name": "read_file",
"description": "Read a file",
"parameters": mock_tool.inputSchema
}
}
]
mock.to_openai_tool = to_openai_tool
def read_file(path: str) -> str:
return f"Contents of {path}"
read_file.__name__ = "read_file"
mock._tools = [read_file]
mock.__iter__ = lambda self: iter(self._tools)
return mock
def sample_function(query: str) -> str:
"""A sample regular function tool."""
return f"Result for: {query}"
# ============================================================================
# Test 1: Single MCP Instance (Baseline)
# ============================================================================
class TestSingleMCPInstance:
"""Tests for single MCP instance - baseline functionality."""
def test_single_mcp_creates_tools(self, mock_mcp_with_tools):
"""Test that a single MCP instance properly exposes tools."""
tools = mock_mcp_with_tools.to_openai_tool()
assert isinstance(tools, list)
assert len(tools) == 2
assert tools[0]["function"]["name"] == "get_current_time"
assert tools[1]["function"]["name"] == "convert_time"
def test_single_mcp_is_iterable(self, mock_mcp_with_tools):
"""Test that MCP instance is iterable."""
tools_list = mock_mcp_with_tools._tools
assert len(tools_list) == 2
assert callable(tools_list[0])
assert callable(tools_list[1])
def test_agent_accepts_single_mcp(self, mock_mcp_with_tools):
"""Test that Agent accepts a single MCP instance as tools."""
# This should not raise an error
agent = Agent(
name="TestAgent",
instructions="Test agent",
tools=mock_mcp_with_tools
)
# Agent wraps tools in a list if not already a list
assert mock_mcp_with_tools in agent.tools or agent.tools == mock_mcp_with_tools
# ============================================================================
# Test 2: Multiple MCP Instances - Tool Definition Conversion
# ============================================================================
class TestMultipleMCPToolDefinition:
"""Tests for multiple MCP instances - tool definition conversion."""
def test_agent_accepts_multiple_mcp_in_list(self, mock_mcp_with_tools, mock_mcp_filesystem):
"""Test that Agent accepts multiple MCP instances in a list."""
agent = Agent(
name="TestAgent",
instructions="Test agent",
tools=[mock_mcp_with_tools, mock_mcp_filesystem]
)
assert isinstance(agent.tools, list)
assert len(agent.tools) == 2
def test_format_tools_handles_multiple_mcp(self, mock_mcp_with_tools, mock_mcp_filesystem):
"""Test that _format_tools_for_llm handles multiple MCP instances."""
agent = Agent(
name="TestAgent",
instructions="Test agent",
tools=[mock_mcp_with_tools, mock_mcp_filesystem]
)
# Call the internal method to format tools
formatted = agent._format_tools_for_completion(agent.tools)
# Should have 3 tools total (2 from time + 1 from filesystem)
assert isinstance(formatted, list)
assert len(formatted) == 3
# Check tool names
tool_names = [t["function"]["name"] for t in formatted]
assert "get_current_time" in tool_names
assert "convert_time" in tool_names
assert "read_file" in tool_names
def test_system_prompt_includes_all_mcp_tools(self, mock_mcp_with_tools, mock_mcp_filesystem):
"""Test that system prompt includes tools from all MCP instances."""
agent = Agent(
name="TestAgent",
instructions="Test agent",
tools=[mock_mcp_with_tools, mock_mcp_filesystem]
)
system_prompt = agent._build_system_prompt(agent.tools)
# Should mention all tools
assert "get_current_time" in system_prompt
assert "convert_time" in system_prompt
assert "read_file" in system_prompt
# ============================================================================
# Test 3: Multiple MCP Instances - Tool Execution
# ============================================================================
class TestMultipleMCPToolExecution:
"""Tests for multiple MCP instances - tool execution."""
def test_execute_tool_finds_mcp_tool_in_list(self, mock_mcp_with_tools, mock_mcp_filesystem):
"""Test that execute_tool_call finds tools from MCP instances in a list."""
agent = Agent(
name="TestAgent",
instructions="Test agent",
tools=[mock_mcp_with_tools, mock_mcp_filesystem]
)
# Mock the runner.call_tool method
mock_mcp_with_tools.runner.call_tool = Mock(return_value='{"timezone": "UTC", "datetime": "2024-01-01T12:00:00Z"}')
# This should find and execute the tool from the first MCP instance
result = agent.execute_tool("get_current_time", {"timezone": "UTC"})
# Verify the tool was called
mock_mcp_with_tools.runner.call_tool.assert_called_once_with("get_current_time", {"timezone": "UTC"})
def test_execute_tool_finds_tool_from_second_mcp(self, mock_mcp_with_tools, mock_mcp_filesystem):
"""Test that execute_tool_call finds tools from the second MCP instance."""
agent = Agent(
name="TestAgent",
instructions="Test agent",
tools=[mock_mcp_with_tools, mock_mcp_filesystem]
)
# Mock the runner.call_tool method
mock_mcp_filesystem.runner.call_tool = Mock(return_value='Contents of /test/file.txt')
# This should find and execute the tool from the second MCP instance
result = agent.execute_tool("read_file", {"path": "/test/file.txt"})
# Verify the tool was called
mock_mcp_filesystem.runner.call_tool.assert_called_once_with("read_file", {"path": "/test/file.txt"})
# ============================================================================
# Test 4: Mixed Tools (MCP + Regular Functions)
# ============================================================================
class TestMixedTools:
"""Tests for mixed tools - MCP instances and regular functions."""
def test_agent_accepts_mixed_tools(self, mock_mcp_with_tools):
"""Test that Agent accepts a mix of MCP instances and regular functions."""
agent = Agent(
name="TestAgent",
instructions="Test agent",
tools=[mock_mcp_with_tools, sample_function]
)
assert isinstance(agent.tools, list)
assert len(agent.tools) == 2
def test_format_tools_handles_mixed(self, mock_mcp_with_tools):
"""Test that _format_tools_for_llm handles mixed tools."""
agent = Agent(
name="TestAgent",
instructions="Test agent",
tools=[mock_mcp_with_tools, sample_function]
)
formatted = agent._format_tools_for_completion(agent.tools)
# Should have 3 tools (2 from MCP + 1 regular function)
assert isinstance(formatted, list)
assert len(formatted) == 3
tool_names = [t["function"]["name"] for t in formatted]
assert "get_current_time" in tool_names
assert "convert_time" in tool_names
assert "sample_function" in tool_names
def test_execute_mixed_tools(self, mock_mcp_with_tools):
"""Test executing both MCP tools and regular functions."""
agent = Agent(
name="TestAgent",
instructions="Test agent",
tools=[mock_mcp_with_tools, sample_function]
)
# Execute regular function
result = agent.execute_tool("sample_function", {"query": "test"})
assert "Result for: test" in str(result)
# Mock and execute MCP tool
mock_mcp_with_tools.runner.call_tool = Mock(return_value='{"timezone": "UTC"}')
result = agent.execute_tool("get_current_time", {"timezone": "UTC"})
mock_mcp_with_tools.runner.call_tool.assert_called_once()
# ============================================================================
# Integration Tests (requires actual MCP server)
# ============================================================================
@pytest.mark.integration
@pytest.mark.skipif(
os.environ.get("SKIP_MCP_INTEGRATION") == "1",
reason="MCP integration tests skipped"
)
class TestMCPIntegration:
"""Integration tests using actual MCP servers."""
def test_real_time_mcp_server(self):
"""Test with real mcp-server-time."""
try:
mcp = MCP("uvx mcp-server-time", timeout=30)
tools = mcp.to_openai_tool()
assert isinstance(tools, list)
assert len(tools) >= 1
tool_names = [t["function"]["name"] for t in tools]
assert "get_current_time" in tool_names
except Exception as e:
pytest.skip(f"MCP server not available: {e}")
def test_multiple_real_mcp_servers(self):
"""Test with multiple real MCP servers."""
try:
time_mcp = MCP("uvx mcp-server-time", timeout=30)
# Create agent with multiple MCP instances
agent = Agent(
name="MultiMCPAgent",
instructions="Test agent with multiple MCP tools",
tools=[time_mcp]
)
# Verify tools are available
formatted = agent._format_tools_for_completion(agent.tools)
assert len(formatted) >= 1
except Exception as e:
pytest.skip(f"MCP servers not available: {e}")
# ============================================================================
# Run tests
# ============================================================================
if __name__ == "__main__":
pytest.main([__file__, "-v", "--tb=short"])
@@ -1032,7 +1032,7 @@ class TestAgentWorkflows:
self.name = name
self.tools = tools or []
def chat(self, message):
def chat(self, message, **kwargs):
return f"Response from {self.name}: {message}"
agent = MockAgent("TestAgent", tools=["tool1", "tool2"])
@@ -1051,7 +1051,7 @@ class TestAgentWorkflows:
def __init__(self, name):
self.name = name
def chat(self, message):
def chat(self, message, **kwargs):
return f"Processed: {message}"
agent = MockAgent("Processor")
@@ -1069,7 +1069,7 @@ class TestAgentWorkflows:
self.name = name
self.prefix = prefix
def chat(self, message):
def chat(self, message, **kwargs):
return f"{self.prefix}: {message}"
agent1 = MockAgent("Agent1", "First")
@@ -1090,7 +1090,7 @@ class TestAgentWorkflows:
self.name = name
self.output = output
def chat(self, message):
def chat(self, message, **kwargs):
return self.output
agent1 = MockAgent("Agent1", "Result A")
@@ -1116,7 +1116,7 @@ class TestAgentWorkflows:
self.name = name
self.output = output
def chat(self, message):
def chat(self, message, **kwargs):
return self.output
tech_agent = MockAgent("TechAgent", "Technical response")
@@ -1149,7 +1149,7 @@ class TestAgentWorkflows:
def __init__(self, name):
self.name = name
def chat(self, message):
def chat(self, message, **kwargs):
return f"Processed: {message}"
processor = MockAgent("Processor")
@@ -1463,7 +1463,7 @@ class TestToolsPerStep:
self.name = name
self.tools = tools or []
def chat(self, message):
def chat(self, message, **kwargs):
return f"Response with {len(self.tools)} tools"
def mock_tool():
@@ -1389,6 +1389,79 @@ steps:
assert workflow is not None
assert workflow.name == "My Topic Workflow"
def test_input_field_canonical(self):
"""Test that 'input' field is the canonical way to specify workflow input."""
from praisonaiagents.workflows import YAMLWorkflowParser
yaml_content = """
name: Input Test Workflow
input: "This is the workflow input"
agents:
researcher:
name: Researcher
role: Research Analyst
goal: Research topics
instructions: "Provide research"
steps:
- agent: researcher
action: "Research {{input}}"
"""
parser = YAMLWorkflowParser()
workflow = parser.parse_string(yaml_content)
assert workflow is not None
assert workflow.default_input == "This is the workflow input"
def test_topic_as_input_alias(self):
"""Test that 'topic' works as an alias for 'input' (backward compatibility)."""
from praisonaiagents.workflows import YAMLWorkflowParser
yaml_content = """
name: Topic Alias Test
topic: "This is the topic as input"
agents:
researcher:
name: Researcher
role: Research Analyst
goal: Research topics
instructions: "Provide research"
steps:
- agent: researcher
action: "Research {{input}}"
"""
parser = YAMLWorkflowParser()
workflow = parser.parse_string(yaml_content)
assert workflow is not None
assert workflow.default_input == "This is the topic as input"
def test_input_takes_precedence_over_topic(self):
"""Test that 'input' takes precedence over 'topic' when both are present."""
from praisonaiagents.workflows import YAMLWorkflowParser
yaml_content = """
name: Precedence Test
input: "Input value wins"
topic: "Topic value loses"
agents:
researcher:
name: Researcher
role: Research Analyst
goal: Research topics
instructions: "Provide research"
steps:
- agent: researcher
action: "Research {{input}}"
"""
parser = YAMLWorkflowParser()
workflow = parser.parse_string(yaml_content)
assert workflow is not None
assert workflow.default_input == "Input value wins"
def test_backstory_normalized_to_instructions(self):
"""Test that 'backstory' is normalized to 'instructions'."""
from praisonaiagents.workflows import YAMLWorkflowParser
+1 -1
View File
@@ -2422,7 +2422,7 @@ wheels = [
[[package]]
name = "praisonaiagents"
version = "0.1.7"
version = "0.1.14"
source = { editable = "." }
dependencies = [
{ name = "aiohttp" },
+72
View File
@@ -34,6 +34,7 @@ PraisonAI is a production-ready Multi-AI Agents framework with self-reflection,
## 📑 Table of Contents
- [⚡ Performance](#-performance)
- [🚀 Quick Start](#-quick-start)
- [📦 Installation](#-installation)
- [Python SDK](#python-sdk)
@@ -73,6 +74,32 @@ PraisonAI is a production-ready Multi-AI Agents framework with self-reflection,
---
## ⚡ Performance
PraisonAI Agents is the **fastest AI agent framework** for agent instantiation.
| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
| **PraisonAI** | **3.77** | **1.00x (fastest)** |
| OpenAI Agents SDK | 5.26 | 1.39x |
| Agno | 5.64 | 1.49x |
| PraisonAI (LiteLLM) | 7.56 | 2.00x |
| PydanticAI | 226.94 | 60.16x |
| LangGraph | 4,558.71 | 1,209x |
| CrewAI | 15,607.92 | 4,138x |
<details>
<summary>Run benchmarks yourself</summary>
```bash
cd praisonai-agents
python benchmarks/simple_benchmark.py
```
</details>
---
## 🚀 Quick Start
Get started with PraisonAI in under 1 minute:
@@ -1392,6 +1419,36 @@ praisonai commit
praisonai commit --push
```
### Serve CLI (API Server):
```bash
# Start API server for agents defined in YAML
praisonai serve agents.yaml
# With custom port and host
praisonai serve agents.yaml --port 8005 --host 0.0.0.0
# Alternative flag style
praisonai agents.yaml --serve
# The server provides:
# POST /agents - Run all agents sequentially
# POST /agents/{name} - Run specific agent (e.g., /agents/researcher)
# GET /agents/list - List available agents
```
### n8n Integration CLI:
```bash
# Export workflow to n8n and open in browser
praisonai agents.yaml --n8n
# With custom n8n URL
praisonai agents.yaml --n8n --n8n-url http://localhost:5678
# Set N8N_API_KEY for auto-import
export N8N_API_KEY="your-api-key"
praisonai agents.yaml --n8n
```
### @Mentions in Prompts:
```bash
# Include file content in prompt
@@ -1523,6 +1580,21 @@ agent = Agent(
env={"BRAVE_API_KEY": "your-key"}
)
)
# Multiple MCP servers + regular functions
def my_custom_tool(query: str) -> str:
"""Custom tool function."""
return f"Result: {query}"
agent = Agent(
name="MultiToolAgent",
instructions="Agent with multiple MCP servers",
tools=[
MCP("uvx mcp-server-time"), # Time tools
MCP("npx @modelcontextprotocol/server-memory"), # Memory tools
my_custom_tool # Regular function
]
)
```
### MCP Server (Expose Tools as MCP Server)
+41 -36
View File
@@ -1,44 +1,49 @@
# Movie Script Workflow - Cat in Mars
# Uses the modern agents + steps format
name: Create Movie Script About Cat in Mars
description: A sequential workflow to research, design narrative, and write a movie script
framework: praisonai
process: sequential
topic: create movie script about cat in mars
memory: false
roles:
workflow:
verbose: true
agents:
researcher:
backstory: Skilled in research, with a focus on gathering accurate and relevant
information.
name: Researcher
role: Research Analyst
goal: Gather information about Mars and cats
role: Researcher
min_reflect: 1
max_reflect: 2
tasks:
research_task:
description: Research about Mars, its environment, and the feasibility of
a cat being on Mars. Also, research about cat behavior and characteristics.
expected_output: Document with research findings on Mars and cats.
backstory: Skilled in research, with a focus on gathering accurate and relevant information.
instructions: |
Research about Mars, its environment, and the feasibility of a cat being on Mars.
Also, research about cat behavior and characteristics.
tools:
- search_tool
- search_tool
narrative_designer:
backstory: Skilled in narrative development, with a focus on creating engaging
stories.
name: Narrative Designer
role: Story Concept Developer
goal: Create a story concept for a movie about a cat in Mars
role: Narrative Designer
tasks:
story_concept_development:
description: Use the research findings to craft a unique story concept for
a movie about a cat in Mars.
expected_output: Document with narrative arcs, character bios, and settings.
tools:
- ''
backstory: Skilled in narrative development, with a focus on creating engaging stories.
instructions: |
Use the research findings to craft a unique story concept for a movie about a cat in Mars.
Create narrative arcs, character bios, and settings.
scriptwriter:
backstory: Expert in dialogue and script structure, translating concepts into
scripts.
name: Scriptwriter
role: Script Writer
goal: Write a movie script about a cat in Mars
role: Scriptwriter
tasks:
scriptwriting_task:
description: Turn the story concept into a production-ready movie script,
including dialogue and scene details.
expected_output: Final movie script with dialogue and scene details.
tools:
- ''
dependencies: []
backstory: Expert in dialogue and script structure, translating concepts into scripts.
instructions: |
Turn the story concept into a production-ready movie script,
including dialogue and scene details.
steps:
- agent: researcher
action: "Research about Mars environment and cat behavior for the movie concept"
- agent: narrative_designer
action: "Create story concept based on research: {{previous_output}}"
- agent: scriptwriter
action: "Write the final movie script based on the story concept: {{previous_output}}"
+133
View File
@@ -0,0 +1,133 @@
{
"id": "e022a3a1",
"name": "Create Movie Script About Cat in Mars",
"nodes": [
{
"id": "trigger",
"name": "Webhook",
"type": "n8n-nodes-base.webhook",
"typeVersion": 2,
"position": [
250,
300
],
"webhookId": "dd7cf5cb72d19eae",
"parameters": {
"path": "create-movie-script-about-cat-in-mars",
"httpMethod": "POST",
"responseMode": "lastNode",
"options": {}
}
},
{
"id": "agent_0",
"name": "Researcher",
"type": "n8n-nodes-base.httpRequest",
"typeVersion": 4.2,
"position": [
450,
300
],
"parameters": {
"method": "POST",
"url": "http://127.0.0.1:8005/agents/researcher",
"sendBody": true,
"specifyBody": "json",
"jsonBody": "={{ JSON.stringify({ query: $json.body?.query || $json.query || 'Research about Mars environment and cat behavior for the movie concept' }) }}",
"options": {
"timeout": 300000
}
},
"notes": "Agent: Researcher\nTask: Research about Mars environment and cat behavior for the movie concept"
},
{
"id": "agent_1",
"name": "Narrative Designer",
"type": "n8n-nodes-base.httpRequest",
"typeVersion": 4.2,
"position": [
700,
300
],
"parameters": {
"method": "POST",
"url": "http://127.0.0.1:8005/agents/narrative_designer",
"sendBody": true,
"specifyBody": "json",
"jsonBody": "={{ JSON.stringify({ query: $json.response || 'Create story concept based on research: previous_output' }) }}",
"options": {
"timeout": 300000
}
},
"notes": "Agent: Narrative Designer\nTask: Create story concept based on research: {{previous_output}}"
},
{
"id": "agent_2",
"name": "Scriptwriter",
"type": "n8n-nodes-base.httpRequest",
"typeVersion": 4.2,
"position": [
950,
300
],
"parameters": {
"method": "POST",
"url": "http://127.0.0.1:8005/agents/scriptwriter",
"sendBody": true,
"specifyBody": "json",
"jsonBody": "={{ JSON.stringify({ query: $json.response || 'Write the final movie script based on the story concept: previous_output' }) }}",
"options": {
"timeout": 300000
}
},
"notes": "Agent: Scriptwriter\nTask: Write the final movie script based on the story concept: {{previous_output}}"
}
],
"connections": {
"Webhook": {
"main": [
[
{
"node": "Researcher",
"type": "main",
"index": 0
}
]
]
},
"Researcher": {
"main": [
[
{
"node": "Narrative Designer",
"type": "main",
"index": 0
}
]
]
},
"Narrative Designer": {
"main": [
[
{
"node": "Scriptwriter",
"type": "main",
"index": 0
}
]
]
}
},
"active": false,
"settings": {
"executionOrder": "v1"
},
"versionId": "1",
"pinData": {},
"meta": {
"instanceId": "praisonai-export",
"templateCredsSetupCompleted": true,
"description": "A sequential workflow to research, design narrative, and write a movie script"
},
"tags": []
}
+2 -2
View File
@@ -3,8 +3,8 @@ class Praisonai < Formula
desc "AI tools for various AI applications"
homepage "https://github.com/MervinPraison/PraisonAI"
url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.28.tar.gz"
sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.28.tar.gz | shasum -a 256`.split.first
url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.36.tar.gz"
sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.36.tar.gz | shasum -a 256`.split.first
license "MIT"
depends_on "[email protected]"
+14 -4
View File
@@ -327,7 +327,8 @@ class AgentsGenerator:
# Route to YAMLWorkflowParser for advanced workflow patterns
return self._run_yaml_workflow(config)
topic = config.get('topic', '')
# Get workflow input: 'input' is canonical, 'topic' is alias for backward compatibility
topic = config.get('input', config.get('topic', ''))
tools_dict = {}
# Only try to use praisonai_tools if it's available and needed
@@ -441,6 +442,15 @@ class AgentsGenerator:
if 'name' not in config:
config['name'] = config.get('topic', 'Workflow')
# Pass model from config_list to workflow as default_llm
if self.config_list and self.config_list[0].get('model'):
model_from_cli = self.config_list[0]['model']
# Set default_llm in workflow config if not already set
if 'workflow' not in config:
config['workflow'] = {}
if 'default_llm' not in config['workflow']:
config['workflow']['default_llm'] = model_from_cli
# Convert config back to YAML string for parser
# Note: YAMLWorkflowParser handles 'roles' to 'agents' conversion internally
import yaml as yaml_module
@@ -450,11 +460,11 @@ class AgentsGenerator:
parser = YAMLWorkflowParser()
workflow = parser.parse_string(yaml_content)
# Get input from topic or variables
input_data = config.get('topic', '')
# Get input: 'input' is canonical, 'topic' is alias for backward compatibility
input_data = config.get('input', config.get('topic', ''))
# Execute workflow
self.logger.info(f"Running workflow: {workflow.name}")
self.logger.debug(f"Running workflow: {workflow.name}")
result = workflow.start(input_data)
if result.get("status") == "completed":
@@ -20,6 +20,7 @@ Features:
- router: Smart model selection
- flow_display: Visual workflow tracking
- workflow: YAML workflow management
- n8n: Export workflows to n8n visual editor
"""
# Type hints available for IDE support
@@ -41,6 +42,7 @@ __all__ = [
'RouterHandler',
'FlowDisplayHandler',
'WorkflowHandler',
'N8nHandler',
]
def __getattr__(name):
@@ -90,4 +92,7 @@ def __getattr__(name):
elif name == 'WorkflowHandler':
from .workflow import WorkflowHandler
return WorkflowHandler
elif name == 'N8nHandler':
from .n8n import N8nHandler
return N8nHandler
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
@@ -1,25 +1,31 @@
"""
Flow Display Handler for CLI.
Provides visual workflow tracking.
Shows a visual flow diagram of the workflow WITHOUT executing it.
Usage: praisonai agents.yaml --flow-display
"""
from typing import Any, Dict, Tuple, List
from typing import Any, Dict, Tuple
from .base import FlagHandler
import os
import yaml
class FlowDisplayHandler(FlagHandler):
"""
Handler for --flow-display flag.
Enables visual workflow tracking for multi-agent executions.
Shows a visual flow diagram/chart of agents and tasks WITHOUT executing.
This allows users to preview the workflow structure before running it.
Example:
praisonai agents.yaml --flow-display
praisonai "Complex task" --flow-display
"""
def __init__(self, verbose: bool = False):
super().__init__(verbose)
self._console = None
@property
def feature_name(self) -> str:
return "flow_display"
@@ -30,160 +36,531 @@ class FlowDisplayHandler(FlagHandler):
@property
def flag_help(self) -> str:
return "Enable visual workflow tracking"
return "Show workflow flow diagram without executing"
def _get_console(self):
"""Get or create Rich Console."""
if self._console is None:
try:
from rich.console import Console
self._console = Console()
except ImportError:
pass
return self._console
def check_dependencies(self) -> Tuple[bool, str]:
"""Check if FlowDisplay is available."""
"""Check if Rich is available."""
try:
import importlib.util
if importlib.util.find_spec("praisonaiagents") is not None:
return True, ""
return False, "praisonaiagents not installed"
rich_available = importlib.util.find_spec("rich") is not None
if not rich_available:
return False, "rich not installed. Install with: pip install rich"
return True, ""
except ImportError:
return False, "praisonaiagents not installed. Install with: pip install praisonaiagents"
def _get_flow_display(self):
"""Get FlowDisplay class lazily."""
try:
from praisonaiagents import FlowDisplay
return FlowDisplay
except ImportError:
return None
def create_flow_display(self, title: str = "Workflow") -> Any:
"""
Create a FlowDisplay instance.
Args:
title: Title for the workflow display
Returns:
FlowDisplay instance or None
"""
FlowDisplay = self._get_flow_display()
if not FlowDisplay:
self.print_status(
"FlowDisplay requires praisonaiagents. Install with: pip install praisonaiagents",
"error"
)
return None
try:
return FlowDisplay(title=title)
except Exception as e:
self.log(f"Failed to create FlowDisplay: {e}", "error")
return None
return False, "rich not installed. Install with: pip install rich"
def apply_to_agent_config(self, config: Dict[str, Any], flag_value: Any) -> Dict[str, Any]:
"""
Apply flow display configuration.
Args:
config: Agent configuration dictionary
flag_value: Boolean indicating flow display
Returns:
Modified configuration
"""
"""Apply flow display configuration."""
if flag_value:
config['flow_display'] = True
return config
def display_workflow_start(self, workflow_name: str, agents: List[str] = None):
def display_flow_diagram(self, yaml_file: str, show_footer: bool = True) -> bool:
"""
Display workflow start.
Display a visual flow diagram of the workflow from YAML file.
Args:
workflow_name: Name of the workflow
agents: List of agent names
"""
self.print_status(f"\n🚀 Starting Workflow: {workflow_name}", "info")
self.print_status("=" * 50, "info")
if agents:
self.print_status("Agents:", "info")
for i, agent in enumerate(agents, 1):
self.print_status(f" {i}. {agent}", "info")
self.print_status("=" * 50, "info")
def display_step(self, step_num: int, step_name: str, status: str = "running"):
"""
Display a workflow step.
Args:
step_num: Step number
step_name: Name of the step
status: Step status (running, completed, failed)
"""
icons = {
'running': '⏳',
'completed': '✅',
'failed': '❌',
'pending': '⬜'
}
icon = icons.get(status, '•')
self.print_status(f" {icon} Step {step_num}: {step_name}", "info")
def display_workflow_end(self, success: bool = True, duration: float = None):
"""
Display workflow end.
Args:
success: Whether workflow completed successfully
duration: Workflow duration in seconds
"""
self.print_status("=" * 50, "info")
if success:
msg = "✅ Workflow completed successfully"
else:
msg = "❌ Workflow failed"
if duration:
msg += f" ({duration:.2f}s)"
self.print_status(msg, "success" if success else "error")
def track_execution(self, func):
"""
Decorator to track function execution in flow display.
Args:
func: Function to track
yaml_file: Path to the agents.yaml file
show_footer: Whether to show the "run without --flow-display" footer
Returns:
Wrapped function
True if displayed successfully, False otherwise
"""
import functools
console = self._get_console()
if not console:
print("Rich library not available for flow display")
return False
@functools.wraps(func)
def wrapper(*args, **kwargs):
func_name = func.__name__
# Load YAML config
try:
if not os.path.exists(yaml_file):
console.print(f"[red]Error: File not found: {yaml_file}[/red]")
return False
self.display_step(1, func_name, "running")
with open(yaml_file, 'r') as f:
config = yaml.safe_load(f)
except Exception as e:
console.print(f"[red]Error loading YAML: {e}[/red]")
return False
# Detect YAML format: 'roles' (old) or 'agents'+'steps' (new workflow)
has_roles = 'roles' in config and config['roles']
has_agents = 'agents' in config and config['agents']
has_steps = 'steps' in config and config['steps']
if not has_roles and not has_agents:
console.print("[red]Error: Invalid YAML format. No 'roles' or 'agents' found.[/red]")
return False
try:
from rich.panel import Panel
from rich.table import Table
from rich.text import Text
from rich import box
# Determine format type
is_workflow_format = has_agents and has_steps
# Extract workflow info based on format
if is_workflow_format:
name = config.get('name', 'Workflow')
description = config.get('description', '')
framework = config.get('framework', 'praisonai')
workflow_config = config.get('workflow', {})
memory = workflow_config.get('memory_config', {}).get('persist', False)
num_agents = len(config['agents'])
num_steps = len(config['steps'])
else:
name = config.get('topic', 'Workflow')
description = ''
framework = config.get('framework', 'praisonai')
process = config.get('process', 'sequential')
memory = config.get('memory', False)
num_agents = len(config['roles'])
num_steps = sum(len(details.get('tasks', {})) for details in config['roles'].values())
# Build the flow diagram
console.print()
# === HEADER ===
header = Text()
header.append("📊 WORKFLOW FLOW DIAGRAM\n", style="bold cyan")
header.append("─" * 40, style="dim")
header_panel = Panel(
header,
border_style="cyan",
box=box.DOUBLE_EDGE,
padding=(0, 2)
)
console.print(header_panel)
# === WORKFLOW INFO ===
info_table = Table(show_header=False, box=None, padding=(0, 2))
info_table.add_column("Key", style="bold yellow")
info_table.add_column("Value", style="white")
info_table.add_row("📋 Name:", name)
if description:
info_table.add_row("📝 Description:", description)
# Show input if present ('input' is canonical, 'topic' is alias)
workflow_input = config.get('input', config.get('topic', ''))
if workflow_input:
info_table.add_row("💬 Input:", workflow_input)
info_table.add_row("⚙️ Framework:", framework)
if is_workflow_format:
info_table.add_row("🔄 Format:", "WORKFLOW (steps-based)")
planning = workflow_config.get('planning', False)
info_table.add_row("🧠 Planning:", "✅ Enabled" if planning else "❌ Disabled")
else:
info_table.add_row("🔄 Process:", process.upper())
info_table.add_row("💾 Memory:", "✅ Enabled" if memory else "❌ Disabled")
info_table.add_row("👥 Agents:", str(num_agents))
info_table.add_row("📝 Steps:", str(num_steps))
console.print(Panel(info_table, title="[bold white]Workflow Configuration[/bold white]",
border_style="blue", box=box.ROUNDED))
console.print()
# === FLOW DIAGRAM ===
if is_workflow_format:
self._display_steps_flow(console, config)
else:
process = config.get('process', 'sequential')
if process == 'sequential':
self._display_sequential_flow(console, config)
elif process == 'hierarchical':
self._display_hierarchical_flow(console, config)
else:
self._display_sequential_flow(console, config)
# === DETAILED AGENT INFO ===
console.print()
if is_workflow_format:
self._display_workflow_agent_details(console, config)
else:
self._display_agent_details(console, config)
# === FOOTER ===
if show_footer:
console.print()
footer = Text()
footer.append("💡 ", style="bold")
footer.append("To execute this workflow, run without --flow-display flag", style="dim italic")
console.print(Panel(footer, border_style="dim", box=box.ROUNDED))
console.print()
return True
except ImportError as e:
console.print(f"[red]Error: Rich components not available: {e}[/red]")
return False
def _display_sequential_flow(self, console, config: Dict):
"""Display sequential flow diagram."""
from rich.panel import Panel
from rich.text import Text
from rich import box
roles = list(config['roles'].items())
# Box width (content area)
BOX_WIDTH = 36
def pad_line(content: str, width: int = BOX_WIDTH) -> str:
"""Pad content to fixed width."""
padding = width - len(content)
if padding > 0:
return content + " " * padding
return content[:width]
flow_text = Text()
flow_text.append(" SEQUENTIAL EXECUTION FLOW\n\n", style="bold magenta")
for i, (role_key, details) in enumerate(roles):
role_name = details.get('role', role_key)
tasks = list(details.get('tasks', {}).keys())
# Agent box
if i == 0:
flow_text.append(" ┌── START ──┐\n", style="green")
flow_text.append(" │\n", style="dim")
flow_text.append(" ▼\n", style="dim")
# Agent node
flow_text.append(" ╔════════════════════════════════════╗\n", style="cyan")
line = f" Agent {i+1}: {role_name}"
flow_text.append(f" ║{pad_line(line)}║\n", style="cyan")
# Tasks
for task in tasks:
line = f" -> {task}"
flow_text.append(f" ║{pad_line(line)}║\n", style="dim white")
flow_text.append(" ╚════════════════════════════════════╝\n", style="cyan")
# Arrow to next
if i < len(roles) - 1:
flow_text.append(" │\n", style="dim")
flow_text.append(" output\n", style="dim italic")
flow_text.append(" ▼\n", style="dim")
else:
flow_text.append(" │\n", style="dim")
flow_text.append(" ▼\n", style="dim")
flow_text.append(" └── END ────┘\n", style="red")
console.print(Panel(flow_text, border_style="magenta", box=box.DOUBLE_EDGE,
title="[bold white]Execution Flow[/bold white]"))
def _display_hierarchical_flow(self, console, config: Dict):
"""Display hierarchical flow diagram."""
from rich.panel import Panel
from rich.text import Text
from rich import box
manager_llm = config.get('manager_llm', 'default')
flow_text = Text()
flow_text.append("👑 HIERARCHICAL EXECUTION FLOW\n\n", style="bold magenta")
# Manager at top
flow_text.append(" ┌──────────────────────┐\n", style="yellow")
flow_text.append(" │ 👑 MANAGER AGENT │\n", style="bold yellow")
flow_text.append(f" │ LLM: {manager_llm:<14}│\n", style="dim yellow")
flow_text.append(" └──────────┬───────────┘\n", style="yellow")
flow_text.append(" │\n", style="dim")
flow_text.append(" ┌──────────┴───────────┐\n", style="dim")
flow_text.append(" │ delegates to │\n", style="dim italic")
flow_text.append(" └──────────┬───────────┘\n", style="dim")
roles = list(config['roles'].items())
num_agents = len(roles)
# Draw branches
if num_agents > 0:
branch_line = " "
for i in range(num_agents):
if i == 0:
branch_line += "┌" + "─" * 8
elif i == num_agents - 1:
branch_line += "┬" + "─" * 8 + "┐"
else:
branch_line += "┬" + "─" * 8
flow_text.append(branch_line + "\n", style="dim")
# Agent boxes
for i, (role_key, details) in enumerate(roles):
flow_text.append("\n ╔═══════════════╗", style="cyan")
flow_text.append("\n")
for i, (role_key, details) in enumerate(roles):
role_name = details.get('role', role_key)[:12]
flow_text.append(f" ║ 🤖 {role_name:<10} ║", style="cyan")
flow_text.append("\n")
for i, (role_key, details) in enumerate(roles):
flow_text.append(" ╚═══════════════╝", style="cyan")
flow_text.append("\n")
console.print(Panel(flow_text, border_style="magenta", box=box.DOUBLE_EDGE,
title="[bold white]Execution Flow[/bold white]"))
def _display_agent_details(self, console, config: Dict):
"""Display detailed agent information."""
from rich.table import Table
from rich import box
table = Table(
title="[bold cyan]📋 Agent & Task Details[/bold cyan]",
box=box.ROUNDED,
show_header=True,
header_style="bold magenta",
border_style="blue",
padding=(0, 1),
expand=True
)
table.add_column("#", style="dim", width=3, justify="center")
table.add_column("Agent", style="bold yellow", min_width=15)
table.add_column("Role", style="cyan", min_width=15)
table.add_column("Goal", style="white", min_width=25, overflow="fold")
table.add_column("Tasks", style="green", min_width=20)
table.add_column("Tools", style="magenta", min_width=15)
for i, (role_key, details) in enumerate(config['roles'].items(), 1):
role_name = details.get('role', role_key)
goal = details.get('goal', '-')
if len(goal) > 50:
goal = goal[:47] + "..."
tasks = list(details.get('tasks', {}).keys())
tasks_str = "\n".join(f"• {t}" for t in tasks) if tasks else "-"
tools = details.get('tools', [])
tools_str = ", ".join(t for t in tools if t) if tools else "-"
table.add_row(
str(i),
role_name,
role_key,
goal,
tasks_str,
tools_str
)
console.print(table)
def _display_steps_flow(self, console, config: Dict):
"""Display steps-based workflow flow diagram."""
from rich.panel import Panel
from rich.text import Text
from rich import box
steps = config.get('steps', [])
agents = config.get('agents', {})
# Box width (content area)
BOX_WIDTH = 36
def pad_line(content: str, width: int = BOX_WIDTH) -> str:
"""Pad content to fixed width, accounting for emoji width."""
# Emojis typically take 2 character widths
emoji_count = sum(1 for c in content if ord(c) > 0x1F300)
actual_len = len(content) + emoji_count # Add extra for emoji width
padding = width - actual_len
if padding > 0:
return content + " " * padding
return content[:width]
flow_text = Text()
flow_text.append(" STEPS-BASED EXECUTION FLOW\n\n", style="bold magenta")
flow_text.append(" ┌── START ──┐\n", style="green")
flow_text.append(" │\n", style="dim")
flow_text.append(" ▼\n", style="dim")
for i, step in enumerate(steps):
# Determine step type
agent_key = step.get('agent', '')
step_name = step.get('name', '')
# Check for special step types
has_route = 'route' in step
has_parallel = 'parallel' in step
has_loop = 'loop' in step
has_repeat = 'repeat' in step
if has_route:
# Route step
flow_text.append(" ╔════════════════════════════════════╗\n", style="yellow")
line = f" ROUTE: {step_name or 'routing'}"
flow_text.append(f" ║{pad_line(line)}║\n", style="yellow")
routes = step['route']
for route_name, route_agents in routes.items():
agents_str = ", ".join(route_agents) if isinstance(route_agents, list) else str(route_agents)
line = f" -> {route_name}: {agents_str}"
flow_text.append(f" ║{pad_line(line)}║\n", style="yellow")
flow_text.append(" ╚════════════════════════════════════╝\n", style="yellow")
elif has_parallel:
# Parallel step
flow_text.append(" ╔════════════════════════════════════╗\n", style="green")
line = f" PARALLEL: {step_name or 'parallel'}"
flow_text.append(f" ║{pad_line(line)}║\n", style="green")
for p_step in step['parallel']:
p_agent = p_step.get('agent', '?')
line = f" |-- {p_agent}"
flow_text.append(f" ║{pad_line(line)}║\n", style="green")
flow_text.append(" ╚════════════════════════════════════╝\n", style="green")
else:
# Regular agent step
agent_info = agents.get(agent_key, {})
agent_name = agent_info.get('name', agent_key)
flow_text.append(" ╔════════════════════════════════════╗\n", style="cyan")
line = f" Step {i+1}: {agent_name}"
flow_text.append(f" ║{pad_line(line)}║\n", style="cyan")
# Show modifiers
if has_loop:
loop_over = step['loop'].get('over', '?')
line = f" [loop over: {loop_over}]"
flow_text.append(f" ║{pad_line(line)}║\n", style="dim magenta")
if has_repeat:
until = step['repeat'].get('until', '?')
max_iter = step['repeat'].get('max_iterations', '?')
line = f" [repeat until: {until} max:{max_iter}]"
flow_text.append(f" ║{pad_line(line)}║\n", style="dim magenta")
flow_text.append(" ╚════════════════════════════════════╝\n", style="cyan")
# Arrow to next
if i < len(steps) - 1:
flow_text.append(" │\n", style="dim")
flow_text.append(" ▼\n", style="dim")
flow_text.append(" │\n", style="dim")
flow_text.append(" ▼\n", style="dim")
flow_text.append(" └── END ────┘\n", style="red")
console.print(Panel(flow_text, border_style="magenta", box=box.DOUBLE_EDGE,
title="[bold white]Execution Flow[/bold white]"))
def _display_workflow_agent_details(self, console, config: Dict):
"""Display detailed agent information for workflow format."""
from rich.table import Table
from rich import box
table = Table(
title="[bold cyan]📋 Agent Details[/bold cyan]",
box=box.ROUNDED,
show_header=True,
header_style="bold magenta",
border_style="blue",
padding=(0, 1),
expand=True
)
table.add_column("#", style="dim", width=3, justify="center")
table.add_column("Agent Key", style="bold yellow", min_width=15)
table.add_column("Name", style="cyan", min_width=15)
table.add_column("Role", style="white", min_width=20)
table.add_column("Goal", style="green", min_width=25, overflow="fold")
for i, (agent_key, details) in enumerate(config['agents'].items(), 1):
name = details.get('name', agent_key)
role = details.get('role', '-')
goal = details.get('goal', '-')
if len(goal) > 40:
goal = goal[:37] + "..."
table.add_row(
str(i),
agent_key,
name,
role,
goal
)
console.print(table)
def display_workflow_start(self, workflow_name: str, agents: list) -> None:
"""
Display workflow start message.
Args:
workflow_name: Name of the workflow being started
agents: List of agent names in the workflow
"""
console = self._get_console()
if console:
try:
result = func(*args, **kwargs)
self.display_step(1, func_name, "completed")
return result
except Exception:
self.display_step(1, func_name, "failed")
raise
return wrapper
from rich.panel import Panel
from rich import box
agents_str = ", ".join(agents) if agents else "No agents"
content = f"🚀 Starting: {workflow_name}\n👥 Agents: {agents_str}"
console.print(Panel(content, title="[bold cyan]Workflow Started[/bold cyan]",
border_style="cyan", box=box.ROUNDED))
except ImportError:
print(f"Starting workflow: {workflow_name} with agents: {agents}")
def execute(self, workflow_name: str = None, **kwargs) -> Any:
def display_workflow_end(self, success: bool = True) -> None:
"""
Execute flow display setup.
Display workflow end message.
Args:
workflow_name: Name of the workflow
success: Whether the workflow completed successfully
"""
console = self._get_console()
if console:
try:
from rich.panel import Panel
from rich import box
if success:
content = "✅ Workflow completed successfully"
style = "green"
else:
content = "❌ Workflow failed"
style = "red"
console.print(Panel(content, title=f"[bold {style}]Workflow Ended[/bold {style}]",
border_style=style, box=box.ROUNDED))
except ImportError:
status = "successfully" if success else "with errors"
print(f"Workflow ended {status}")
def execute(self, yaml_file: str = None, **kwargs) -> bool:
"""
Execute flow display - shows diagram without running workflow.
Args:
yaml_file: Path to the YAML file
Returns:
FlowDisplay instance or None
True if displayed successfully
"""
if workflow_name:
self.display_workflow_start(workflow_name)
return self.create_flow_display(workflow_name or "Workflow")
if yaml_file:
return self.display_flow_diagram(yaml_file)
return False
+673
View File
@@ -0,0 +1,673 @@
"""
n8n Integration Handler for PraisonAI CLI.
Converts PraisonAI agents.yaml workflows to n8n workflow JSON format
and opens them in the n8n UI.
Usage:
praisonai agents.yaml --n8n
praisonai agents.yaml --n8n --n8n-url http://localhost:5678
"""
import json
import yaml
import webbrowser
import hashlib
import os
from typing import Any, Dict, List, Optional
from pathlib import Path
try:
import requests
REQUESTS_AVAILABLE = True
except ImportError:
REQUESTS_AVAILABLE = False
from .base import FlagHandler
class N8nHandler(FlagHandler):
"""
Handler for n8n workflow export and visualization.
Converts PraisonAI YAML workflows to n8n-compatible JSON format
and opens them in the n8n UI for visual editing.
"""
def __init__(self, verbose: bool = False, n8n_url: str = "http://localhost:5678",
use_execute_command: bool = False, api_url: str = "http://127.0.0.1:8005"):
"""
Initialize the n8n handler.
Args:
verbose: Enable verbose output
n8n_url: Base URL of the n8n instance
use_execute_command: Use Execute Command nodes (runs praisonai directly)
instead of HTTP Request nodes
api_url: PraisonAI API URL that n8n will call (for tunnel/cloud deployments)
"""
super().__init__(verbose=verbose)
self.n8n_url = n8n_url
self.praisonai_api_url = api_url
self.use_execute_command = use_execute_command
@property
def feature_name(self) -> str:
return "n8n"
@property
def flag_name(self) -> str:
return "n8n"
@property
def flag_help(self) -> str:
return "Export workflow to n8n and open in browser"
def execute(self, yaml_path: str, n8n_url: Optional[str] = None,
open_browser: bool = True, **kwargs) -> Dict[str, Any]:
"""
Execute the n8n export workflow.
Args:
yaml_path: Path to the agents.yaml file
n8n_url: Optional custom n8n URL
open_browser: Whether to open the workflow in browser
Returns:
Dictionary with workflow JSON and URL
"""
if n8n_url:
self.n8n_url = n8n_url
# Convert YAML to n8n workflow JSON
workflow_json = self.convert_yaml_to_n8n(yaml_path)
# Save workflow JSON to file
output_path = self._save_workflow_json(workflow_json, yaml_path)
self.print_status("✅ Workflow converted successfully!", "success")
self.print_status(f"📄 JSON saved to: {output_path}", "info")
# Try to create workflow via n8n API
api_key = os.environ.get('N8N_API_KEY')
workflow_id = None
if api_key and REQUESTS_AVAILABLE:
workflow_id = self._create_workflow_via_api(workflow_json, api_key)
webhook_url = None
if workflow_id:
# Successfully created via API - open directly
url = f"{self.n8n_url}/workflow/{workflow_id}"
self.print_status("🚀 Workflow created in n8n!", "success")
# Activate the workflow so webhook is available
if self._activate_workflow(workflow_id, api_key):
self.print_status("✅ Workflow activated!", "success")
# Get webhook path from the workflow
trigger_node = next((n for n in workflow_json["nodes"]
if n.get("type") == "n8n-nodes-base.webhook"), None)
if trigger_node:
webhook_path = trigger_node["parameters"].get("path", "praisonai")
webhook_url = f"{self.n8n_url}/webhook/{webhook_path}"
self.print_status("", "info")
self.print_status("🔗 Webhook URL (to trigger workflow):", "info")
self.print_status(f" POST {webhook_url}", "info")
else:
self.print_status("⚠️ Could not activate workflow (activate manually in n8n)", "warning")
if open_browser:
self.open_in_browser(url)
self.print_status(f"🌐 Opening: {url}", "info")
else:
# Fallback to manual import instructions
url = self.generate_n8n_url(workflow_json, self.n8n_url)
if not api_key:
self.print_status("", "info")
self.print_status("💡 Tip: Set N8N_API_KEY env var for auto-import", "info")
self.print_status(" Generate API key in n8n: Settings → API", "info")
self.print_status("", "info")
self.print_status("📋 To import into n8n:", "info")
self.print_status(" 1. Open n8n in your browser", "info")
self.print_status(" 2. Click the three dots menu (⋮) in the top right", "info")
self.print_status(" 3. Select 'Import from File'", "info")
self.print_status(f" 4. Choose: {output_path}", "info")
if open_browser:
self.open_in_browser(url)
self.print_status(f"🌐 Opening n8n: {url}", "info")
return {
"workflow": workflow_json,
"url": url,
"output_path": output_path,
"workflow_id": workflow_id,
"webhook_url": webhook_url
}
def _create_workflow_via_api(self, workflow_json: Dict[str, Any],
api_key: str) -> Optional[str]:
"""
Create workflow in n8n via REST API.
Args:
workflow_json: The n8n workflow JSON
api_key: n8n API key
Returns:
Workflow ID if successful, None otherwise
"""
if not REQUESTS_AVAILABLE:
return None
try:
# Only include fields allowed by n8n API schema
# See: n8n/packages/cli/src/public-api/v1/handlers/workflows/spec/schemas/workflow.yml
workflow_data = {
"name": workflow_json.get("name", "PraisonAI Workflow"),
"nodes": workflow_json.get("nodes", []),
"connections": workflow_json.get("connections", {}),
"settings": workflow_json.get("settings", {"executionOrder": "v1"}),
}
# Optional fields
if "staticData" in workflow_json:
workflow_data["staticData"] = workflow_json["staticData"]
response = requests.post(
f"{self.n8n_url}/api/v1/workflows",
headers={
"X-N8N-API-KEY": api_key,
"Content-Type": "application/json"
},
json=workflow_data,
timeout=30
)
if response.status_code == 200:
result = response.json()
return result.get('id')
else:
self.log(f"API error: {response.status_code} - {response.text}", "warning")
return None
except Exception as e:
self.log(f"Failed to create workflow via API: {e}", "warning")
return None
def _activate_workflow(self, workflow_id: str, api_key: str) -> bool:
"""
Activate a workflow via n8n API.
Args:
workflow_id: The workflow ID to activate
api_key: n8n API key
Returns:
True if successful, False otherwise
"""
if not REQUESTS_AVAILABLE:
return False
try:
response = requests.post(
f"{self.n8n_url}/api/v1/workflows/{workflow_id}/activate",
headers={
"X-N8N-API-KEY": api_key,
"Content-Type": "application/json"
},
timeout=30
)
if response.status_code == 200:
return True
else:
self.log(f"Activation error: {response.status_code} - {response.text}", "warning")
return False
except Exception as e:
self.log(f"Failed to activate workflow: {e}", "warning")
return False
def convert_yaml_to_n8n(self, yaml_path: str, use_webhook: bool = True) -> Dict[str, Any]:
"""
Convert a PraisonAI YAML workflow to n8n JSON format.
The n8n workflow will have:
- A webhook trigger to receive requests
- One HTTP Request node per agent (calls /agents/{agent_name})
- Each agent node passes its output to the next agent
Args:
yaml_path: Path to the agents.yaml file
use_webhook: Use webhook trigger for programmatic execution (default: True)
Returns:
n8n workflow JSON as dictionary
"""
# Load YAML file
with open(yaml_path, 'r') as f:
yaml_content = yaml.safe_load(f)
# Extract workflow metadata
workflow_name = yaml_content.get('name', 'PraisonAI Workflow')
description = yaml_content.get('description', '')
# Generate a unique webhook path from workflow name
webhook_path = workflow_name.lower().replace(' ', '-').replace('/', '-')[:50]
# Get agents and steps
agents = yaml_content.get('agents', yaml_content.get('roles', {}))
steps = yaml_content.get('steps', [])
# If no steps defined, create steps from agents
if not steps and agents:
steps = self._create_steps_from_agents(agents)
# Build n8n nodes and connections
nodes = []
connections = {}
# Add trigger node (webhook for programmatic execution, manual otherwise)
trigger_node = self._create_trigger_node(use_webhook=use_webhook, webhook_path=webhook_path)
nodes.append(trigger_node)
# Track previous node for connections
prev_node_name = trigger_node["name"]
x_position = 450
# Create one HTTP node per agent step
for i, step in enumerate(steps):
if isinstance(step, dict):
agent_id = step.get('agent', '')
action = step.get('action', step.get('description', ''))
agent_config = agents.get(agent_id, {})
# Create HTTP node that calls the specific agent endpoint
agent_node = self._create_per_agent_node(
agent_id=agent_id,
agent_config=agent_config,
action=action,
position=[x_position, 300],
index=i,
is_first=(i == 0)
)
nodes.append(agent_node)
# Connect to previous node
if prev_node_name not in connections:
connections[prev_node_name] = {"main": [[]]}
connections[prev_node_name]["main"][0].append({
"node": agent_node["name"],
"type": "main",
"index": 0
})
prev_node_name = agent_node["name"]
x_position += 250
# Build the complete workflow
# Generate a unique ID based on workflow name
workflow_id = hashlib.md5(workflow_name.encode()).hexdigest()[:8]
workflow = {
"id": workflow_id,
"name": workflow_name,
"nodes": nodes,
"connections": connections,
"active": False,
"settings": {
"executionOrder": "v1"
},
"versionId": "1",
"pinData": {},
"meta": {
"instanceId": "praisonai-export",
"templateCredsSetupCompleted": True
},
"tags": []
}
if description:
workflow["meta"]["description"] = description
return workflow
def _create_steps_from_agents(self, agents: Dict[str, Any]) -> List[Dict]:
"""Create steps from agents when no steps are defined."""
steps = []
for agent_id, agent_config in agents.items():
if isinstance(agent_config, dict):
# Check for tasks nested in agent
tasks = agent_config.get('tasks', {})
if tasks:
for task_id, task_config in tasks.items():
steps.append({
'agent': agent_id,
'action': task_config.get('description', task_config.get('action', f'Execute {task_id}'))
})
else:
# Create a default step for the agent
steps.append({
'agent': agent_id,
'action': agent_config.get('instructions', agent_config.get('goal', f'Execute {agent_id}'))
})
return steps
def _create_trigger_node(self, use_webhook: bool = False,
webhook_path: str = "praisonai") -> Dict[str, Any]:
"""Create a trigger node (webhook or manual).
Args:
use_webhook: If True, creates a webhook trigger for programmatic execution
webhook_path: Path for the webhook URL (only used if use_webhook=True)
"""
if use_webhook:
# Generate a unique webhook ID
webhook_id = hashlib.md5(webhook_path.encode()).hexdigest()[:16]
return {
"id": "trigger",
"name": "Webhook",
"type": "n8n-nodes-base.webhook",
"typeVersion": 2,
"position": [250, 300],
"webhookId": webhook_id,
"parameters": {
"path": webhook_path,
"httpMethod": "POST",
"responseMode": "lastNode",
"options": {}
}
}
return {
"id": "trigger",
"name": "Manual Trigger",
"type": "n8n-nodes-base.manualTrigger",
"typeVersion": 1,
"position": [250, 300],
"parameters": {}
}
def _create_per_agent_node(self, agent_id: str, agent_config: Dict,
action: str, position: List[int],
index: int, is_first: bool = False) -> Dict[str, Any]:
"""
Create an HTTP Request node that calls a specific agent endpoint.
Args:
agent_id: Agent identifier (used in URL path)
agent_config: Agent configuration from YAML
action: The action/task description for this agent
position: Node position [x, y]
index: Node index
is_first: Whether this is the first agent (uses webhook input)
Returns:
n8n HTTP Request node configuration
"""
agent_name = agent_config.get('name', agent_id.title())
# Convert agent_id to URL-safe format (lowercase, underscores)
agent_url_id = agent_id.lower().replace(' ', '_')
# Build the query - first agent uses webhook input, others use previous response
# Escape single quotes and double braces (n8n expression syntax)
escaped_action = action.replace("'", "\\'").replace("{{", "").replace("}}", "")
if is_first:
# First agent: use query from webhook body
query_expr = "$json.body?.query || $json.query || '" + escaped_action + "'"
else:
# Subsequent agents: use previous agent's response + original action context
query_expr = "$json.response || '" + escaped_action + "'"
return {
"id": f"agent_{index}",
"name": agent_name,
"type": "n8n-nodes-base.httpRequest",
"typeVersion": 4.2,
"position": position,
"parameters": {
"method": "POST",
"url": f"{self.praisonai_api_url}/agents/{agent_url_id}",
"sendBody": True,
"specifyBody": "json",
"jsonBody": f"={{{{ JSON.stringify({{ query: {query_expr} }}) }}}}",
"options": {
"timeout": 300000 # 5 minute timeout per agent
}
},
"notes": f"Agent: {agent_name}\nTask: {action[:100]}"
}
def _create_praisonai_workflow_node(self, workflow_name: str,
agent_names: List[str],
position: List[int]) -> Dict[str, Any]:
"""
Create a single HTTP Request node that triggers the entire PraisonAI workflow.
The PraisonAI API will run all agents sequentially when called.
Args:
workflow_name: Name of the workflow
agent_names: List of agent names (for documentation)
position: Node position [x, y]
Returns:
n8n HTTP Request node configuration
"""
# The query will be passed from the webhook trigger
# Using n8n expression to get the body from the webhook
return {
"id": "praisonai_workflow",
"name": f"PraisonAI: {workflow_name}",
"type": "n8n-nodes-base.httpRequest",
"typeVersion": 4.2,
"position": position,
"parameters": {
"method": "POST",
"url": f"{self.praisonai_api_url}/agents",
"sendBody": True,
"specifyBody": "json",
# Pass the query from webhook body, or use a default
"jsonBody": "={{ JSON.stringify({ query: $json.body?.query || $json.query || 'Start workflow' }) }}",
"options": {
"timeout": 600000 # 10 minute timeout for full workflow
}
},
"notes": f"Runs agents: {', '.join(agent_names)}"
}
def _create_agent_node(self, agent_id: str, agent_config: Dict,
action: str, position: List[int],
index: int) -> Dict[str, Any]:
"""Create a node for an agent - either Execute Command or HTTP Request."""
if self.use_execute_command:
return self._create_execute_command_node(
agent_id, agent_config, action, position, index
)
else:
return self._create_http_request_node(
agent_id, agent_config, action, position, index
)
def _create_execute_command_node(self, agent_id: str, agent_config: Dict,
action: str, position: List[int],
index: int) -> Dict[str, Any]:
"""Create an Execute Command node that runs praisonai directly."""
agent_name = agent_config.get('name', agent_id.title())
# Escape quotes in the action for shell command
escaped_action = action.replace('"', '\\"').replace("'", "\\'")
# Build the praisonai command
# Uses the input from previous node if available
command = f'praisonai "{escaped_action}"'
return {
"id": f"agent_{index}",
"name": agent_name,
"type": "n8n-nodes-base.executeCommand",
"typeVersion": 1,
"position": position,
"parameters": {
"command": command,
"executeOnce": True
}
}
def _create_http_request_node(self, agent_id: str, agent_config: Dict,
action: str, position: List[int],
index: int) -> Dict[str, Any]:
"""Create an HTTP Request node for an agent."""
agent_name = agent_config.get('name', agent_id.title())
# Build the request body
request_body = {
"query": action,
"agent": agent_id
}
return {
"id": f"agent_{index}",
"name": agent_name,
"type": "n8n-nodes-base.httpRequest",
"typeVersion": 4.2,
"position": position,
"parameters": {
"method": "POST",
"url": f"{self.praisonai_api_url}/agents",
"sendBody": True,
"specifyBody": "json",
"jsonBody": json.dumps(request_body),
"options": {
"timeout": 300000 # 5 minute timeout for LLM calls
}
}
}
def _create_parallel_nodes(self, parallel_steps: List[Dict],
x_position: int,
agents: Dict[str, Any]) -> tuple:
"""Create nodes for parallel execution."""
nodes = []
connections = {}
for i, step in enumerate(parallel_steps):
agent_id = step.get('agent', '')
action = step.get('action', step.get('description', ''))
agent_config = agents.get(agent_id, {})
node = self._create_agent_node(
agent_id=agent_id,
agent_config=agent_config,
action=action,
position=[x_position + (i * 200), 300 + (i * 100)],
index=100 + i # Offset index for parallel nodes
)
nodes.append(node)
return nodes, connections
def _create_route_node(self, step: Dict, x_position: int,
index: int) -> Dict[str, Any]:
"""Create an IF node for routing decisions."""
route_config = step.get('route', {})
conditions = list(route_config.keys())
return {
"id": f"route_{index}",
"name": f"Route Decision {index}",
"type": "n8n-nodes-base.if",
"typeVersion": 2,
"position": [x_position, 300],
"parameters": {
"conditions": {
"options": {
"caseSensitive": True,
"leftValue": "",
"typeValidation": "strict"
},
"conditions": [
{
"id": f"condition_{i}",
"leftValue": "={{ $json.result }}",
"rightValue": condition,
"operator": {
"type": "string",
"operation": "contains"
}
}
for i, condition in enumerate(conditions) if condition != 'default'
]
}
}
}
def generate_n8n_url(self, workflow_json: Dict[str, Any],
base_url: str) -> str:
"""
Generate a URL to open n8n workflow editor.
Note: n8n doesn't support URL hash-based import. Users need to:
1. Open n8n
2. Use "Import from File" to load the saved JSON
Args:
workflow_json: The n8n workflow JSON (unused, kept for API compatibility)
base_url: Base URL of the n8n instance
Returns:
URL to open n8n workflow editor
"""
# n8n doesn't support URL-based workflow import via hash
# Return the new workflow URL - user will need to import manually
return f"{base_url}/workflow/new"
def open_in_browser(self, url: str) -> None:
"""
Open a URL in the default web browser.
Args:
url: URL to open
"""
try:
webbrowser.open(url)
except Exception as e:
self.print_status(f"Could not open browser: {e}", "warning")
self.print_status(f"Please open this URL manually: {url}", "info")
def _save_workflow_json(self, workflow_json: Dict[str, Any],
yaml_path: str) -> str:
"""
Save the workflow JSON to a file.
Args:
workflow_json: The n8n workflow JSON
yaml_path: Original YAML file path
Returns:
Path to the saved JSON file
"""
# Create output path based on input YAML
yaml_file = Path(yaml_path)
output_path = yaml_file.parent / f"{yaml_file.stem}_n8n.json"
with open(output_path, 'w') as f:
json.dump(workflow_json, f, indent=2)
return str(output_path)
def get_workflow_json_for_clipboard(self, yaml_path: str) -> str:
"""
Get the workflow JSON as a string for clipboard copy.
Args:
yaml_path: Path to the agents.yaml file
Returns:
JSON string ready for clipboard
"""
workflow_json = self.convert_yaml_to_n8n(yaml_path)
return json.dumps(workflow_json, indent=2)
+261 -1
View File
@@ -210,6 +210,10 @@ class PraisonAI:
version_string = f"PraisonAI version {__version__}"
self.framework = args.framework or self.framework
# Update config_list model if --model flag is provided
if getattr(args, 'model', None):
self.config_list[0]['model'] = args.model
# Check for piped input from stdin
stdin_input = self.read_stdin_if_available()
@@ -495,6 +499,32 @@ class PraisonAI:
print(result)
return result
else:
# n8n Integration - Export workflow to n8n and open in browser
if getattr(args, 'n8n', False):
from .features.n8n import N8nHandler
n8n_handler = N8nHandler(
verbose=getattr(args, 'verbose', False),
n8n_url=getattr(args, 'n8n_url', 'http://localhost:5678'),
api_url=getattr(args, 'api_url', 'http://127.0.0.1:8005')
)
result = n8n_handler.execute(self.agent_file)
return result
# Flow Display - Show flow diagram WITHOUT executing (--flow-display flag)
if getattr(args, 'flow_display', False):
from .features.flow_display import FlowDisplayHandler
flow_handler = FlowDisplayHandler(verbose=getattr(args, 'verbose', False))
flow_handler.display_flow_diagram(self.agent_file)
return # Exit without executing
# Always show flow diagram before execution (default behavior)
try:
from .features.flow_display import FlowDisplayHandler
flow_handler = FlowDisplayHandler(verbose=getattr(args, 'verbose', False))
flow_handler.display_flow_diagram(self.agent_file, show_footer=False)
except Exception:
pass # Continue even if flow display fails
agents_generator = AgentsGenerator(
self.agent_file,
self.framework,
@@ -549,7 +579,7 @@ class PraisonAI:
return default_args
# Define special commands
special_commands = ['chat', 'code', 'call', 'realtime', 'train', 'ui', 'context', 'research', 'memory', 'rules', 'workflow', 'hooks', 'knowledge', 'session', 'tools', 'todo', 'docs', 'mcp', 'commit']
special_commands = ['chat', 'code', 'call', 'realtime', 'train', 'ui', 'context', 'research', 'memory', 'rules', 'workflow', 'hooks', 'knowledge', 'session', 'tools', 'todo', 'docs', 'mcp', 'commit', 'serve']
parser = argparse.ArgumentParser(prog="praisonai", description="praisonAI command-line interface")
parser.add_argument("--framework", choices=["crewai", "autogen", "praisonai"], help="Specify the framework")
@@ -650,6 +680,16 @@ class PraisonAI:
# Flow Display - visual workflow
parser.add_argument("--flow-display", action="store_true", help="Enable visual workflow tracking")
# n8n Integration - export workflow to n8n
parser.add_argument("--n8n", action="store_true", help="Export workflow to n8n and open in browser")
parser.add_argument("--n8n-url", type=str, default="http://localhost:5678", help="n8n instance URL (default: http://localhost:5678)")
parser.add_argument("--api-url", type=str, default="http://127.0.0.1:8005", help="PraisonAI API URL for n8n to call (default: http://127.0.0.1:8005)")
# Serve - start API server for agents
parser.add_argument("--serve", action="store_true", help="Start API server for agents (use with agents.yaml)")
parser.add_argument("--port", type=int, default=8005, help="Server port (default: 8005)")
parser.add_argument("--host", type=str, default="127.0.0.1", help="Server host (default: 127.0.0.1)")
# If we're in a test environment, parse with empty args to avoid pytest interference
if in_test_env:
args, unknown_args = parser.parse_known_args([])
@@ -677,6 +717,13 @@ class PraisonAI:
args.realtime = True
if args.command == 'call':
args.call = True
if args.command == 'serve':
args.serve = True
# Handle serve command - start API server for agents
if args.command == 'serve' or args.serve:
self._handle_serve_command(args, unknown_args)
sys.exit(0)
# Handle both command and flag versions for call
if args.command == 'call' or args.call:
@@ -3011,6 +3058,219 @@ Now, {final_instruction.lower()}:"""
print("pip install praisonaiagents # For PraisonAIAgents\n")
sys.exit(1)
def _handle_serve_command(self, args, unknown_args):
"""
Handle the serve command - start API server for agents.
Supports:
- Sequential workflows with agents and steps
- Workflow patterns: route, parallel, loop, repeat
- All process types: sequential, workflow, hierarchical
Usage:
praisonai serve agents.yaml
praisonai serve agents.yaml --port 8005 --host 0.0.0.0
praisonai agents.yaml --serve
"""
import yaml
from praisonaiagents import Agent, PraisonAIAgents
# Determine the YAML file path
yaml_file = None
# Check if command is 'serve' and there's a file in unknown_args
if args.command == 'serve' and unknown_args:
yaml_file = unknown_args[0]
# Check if command itself is a yaml file (praisonai agents.yaml --serve)
elif args.command and args.command.endswith('.yaml'):
yaml_file = args.command
# Default to agents.yaml
else:
yaml_file = 'agents.yaml'
# Check if file exists
if not os.path.exists(yaml_file):
print(f"[red]ERROR: File not found: {yaml_file}[/red]")
print("\nUsage:")
print(" praisonai serve agents.yaml")
print(" praisonai serve agents.yaml --port 8005")
print(" praisonai agents.yaml --serve")
sys.exit(1)
print(f"📄 Loading workflow from: {yaml_file}")
# Load YAML file
with open(yaml_file, 'r') as f:
config = yaml.safe_load(f)
# Extract agents
agents_config = config.get('agents', config.get('roles', {}))
agents_dict = {} # Map agent_id to Agent instance
agents_list = []
for agent_id, agent_config in agents_config.items():
if isinstance(agent_config, dict):
agent = Agent(
name=agent_config.get('name', agent_id.title()),
role=agent_config.get('role', ''),
goal=agent_config.get('goal', ''),
backstory=agent_config.get('backstory', ''),
instructions=agent_config.get('instructions', ''),
llm=agent_config.get('llm', 'gpt-4o-mini'),
verbose=False
)
agents_dict[agent_id] = agent
agents_list.append(agent)
print(f" ✓ Loaded agent: {agent.name}")
if not agents_list:
print("[red]ERROR: No agents found in YAML file[/red]")
sys.exit(1)
# Extract tasks from steps (if defined)
steps = config.get('steps', [])
tasks_list = []
# Detect advanced workflow patterns
has_route = any(isinstance(s, dict) and 'route' in s for s in steps)
has_parallel = any(isinstance(s, dict) and 'parallel' in s for s in steps)
has_loop = any(isinstance(s, dict) and 'loop' in s for s in steps)
has_repeat = any(isinstance(s, dict) and 'repeat' in s for s in steps)
if has_route or has_parallel or has_loop or has_repeat:
patterns = []
if has_route: patterns.append("route")
if has_parallel: patterns.append("parallel")
if has_loop: patterns.append("loop")
if has_repeat: patterns.append("repeat")
print(f"\n🔀 Detected workflow patterns: {', '.join(patterns)}")
if steps:
from praisonaiagents import Task
print(f"\n📋 Loading workflow steps...")
# First pass: create all tasks
task_name_map = {} # Map step names to Task objects for context/next_tasks
for i, step in enumerate(steps):
if isinstance(step, dict):
# Handle advanced patterns
if 'route' in step:
print(f" ✓ Loaded routing step: {step.get('name', f'route_{i+1}')}")
continue # Route steps are handled by workflow process
if 'parallel' in step:
parallel_tasks = step.get('parallel', [])
print(f" ✓ Loaded parallel step: {len(parallel_tasks)} concurrent tasks")
for j, pt in enumerate(parallel_tasks):
if isinstance(pt, dict):
agent_id = pt.get('agent', '')
action = pt.get('action', pt.get('description', ''))
if agent_id in agents_dict:
task = Task(
name=f"parallel_{i}_{j}",
description=action,
agent=agents_dict[agent_id],
expected_output=pt.get('expected_output', 'Completed task output'),
async_execution=True, # Parallel tasks run async
)
tasks_list.append(task)
task_name_map[f"parallel_{i}_{j}"] = task
continue
if 'loop' in step:
loop_config = step.get('loop', {})
print(f" ✓ Loaded loop step: over '{loop_config.get('over', 'items')}'")
continue # Loop steps are handled by workflow process
if 'repeat' in step:
repeat_config = step.get('repeat', {})
print(f" ✓ Loaded repeat step: until '{repeat_config.get('until', 'done')}'")
continue # Repeat steps are handled by workflow process
# Standard step
agent_id = step.get('agent', '')
action = step.get('action', step.get('description', ''))
task_name = step.get('name', f"step_{i+1}")
if agent_id in agents_dict:
task = Task(
name=task_name,
description=action,
agent=agents_dict[agent_id],
expected_output=step.get('expected_output', 'Completed task output'),
task_type=step.get('task_type', 'task'),
is_start=step.get('is_start', i == 0),
async_execution=step.get('async_execution', False),
output_file=step.get('output_file'),
)
tasks_list.append(task)
task_name_map[task_name] = task
print(f" ✓ Loaded step {i+1}: {agent_id} → {action[:50]}...")
# Second pass: set up next_tasks and context relationships
for i, step in enumerate(steps):
if isinstance(step, dict) and 'agent' in step:
task_name = step.get('name', f"step_{i+1}")
if task_name in task_name_map:
task = task_name_map[task_name]
# Set next_tasks if specified
next_tasks = step.get('next_tasks', [])
if next_tasks:
task.next_tasks = next_tasks
# Set context if specified
context_refs = step.get('context', [])
if context_refs:
task.context = [task_name_map[ref] for ref in context_refs if ref in task_name_map]
# Set condition for workflow branching
condition = step.get('condition', {})
if condition:
task.condition = condition
port = args.port
host = args.host
# Extract process type from workflow config (default: sequential)
workflow_config = config.get('workflow', {})
process_type = workflow_config.get('process', 'sequential')
verbose = workflow_config.get('verbose', False)
# Also check top-level process key
if 'process' in config:
process_type = config['process']
print(f"\n🚀 Starting PraisonAI API server...")
print(f" Host: {host}")
print(f" Port: {port}")
print(f" Agents: {len(agents_list)}")
if tasks_list:
print(f" Tasks: {len(tasks_list)}")
print(f" Process: {process_type}")
# Create and launch - with tasks if defined
if tasks_list:
praison = PraisonAIAgents(
agents=agents_list,
tasks=tasks_list,
process=process_type,
verbose=1 if verbose else 0
)
else:
praison = PraisonAIAgents(
agents=agents_list,
process=process_type,
verbose=1 if verbose else 0
)
praison.launch(port=port, host=host)
# Keep the main thread alive to prevent exit
print("\n✅ Server running. Press Ctrl+C to stop.")
try:
import time
while True:
time.sleep(1)
except KeyboardInterrupt:
print("\n👋 Server stopped.")
def create_chainlit_chat_interface(self):
"""
Create a Chainlit interface for the chat application.
+1 -1
View File
@@ -57,7 +57,7 @@ class CloudDeployer:
file.write("FROM python:3.11-slim\n")
file.write("WORKDIR /app\n")
file.write("COPY . .\n")
file.write("RUN pip install flask praisonai==2.3.28 gunicorn markdown\n")
file.write("RUN pip install flask praisonai==2.3.36 gunicorn markdown\n")
file.write("EXPOSE 8080\n")
file.write('CMD ["gunicorn", "-b", "0.0.0.0:8080", "api:app"]\n')
+1 -1
View File
@@ -1 +1 @@
__version__ = "2.3.28"
__version__ = "2.3.36"
+1 -1
View File
@@ -12,7 +12,7 @@ dependencies = [
"rich>=13.7",
"markdown>=3.5",
"pyparsing>=3.0.0",
"praisonaiagents>=0.1.7",
"praisonaiagents>=0.1.14",
"python-dotenv>=0.19.0",
"instructor>=1.3.3",
"PyYAML>=6.0",
+246
View File
@@ -0,0 +1,246 @@
"""
Test-Driven Development tests for n8n integration.
Tests for converting PraisonAI agents.yaml to n8n workflow JSON format.
"""
import pytest
import json
import os
import tempfile
from pathlib import Path
class TestN8nConverter:
"""Tests for YAML to n8n JSON conversion."""
@pytest.fixture
def sample_agents_yaml(self):
"""Sample agents.yaml content for testing."""
return """
name: Test Workflow
description: A test workflow for n8n integration
agents:
researcher:
name: Researcher
role: Research Analyst
goal: Research topics
instructions: Research and gather information
writer:
name: Writer
role: Content Writer
goal: Write content
instructions: Write based on research
steps:
- agent: researcher
action: "Research the topic"
- agent: writer
action: "Write content based on research"
"""
@pytest.fixture
def sample_yaml_file(self, sample_agents_yaml):
"""Create a temporary YAML file for testing."""
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
f.write(sample_agents_yaml)
f.flush()
yield f.name
os.unlink(f.name)
def test_n8n_handler_import(self):
"""Test that N8nHandler can be imported."""
from praisonai.cli.features.n8n import N8nHandler
assert N8nHandler is not None
def test_n8n_handler_initialization(self):
"""Test N8nHandler initialization."""
from praisonai.cli.features.n8n import N8nHandler
handler = N8nHandler(verbose=True)
assert handler.feature_name == "n8n"
assert handler.verbose is True
def test_convert_yaml_to_n8n_returns_dict(self, sample_yaml_file):
"""Test that convert_yaml_to_n8n returns a dictionary."""
from praisonai.cli.features.n8n import N8nHandler
handler = N8nHandler()
result = handler.convert_yaml_to_n8n(sample_yaml_file)
assert isinstance(result, dict)
def test_n8n_workflow_has_required_keys(self, sample_yaml_file):
"""Test that n8n workflow JSON has required keys."""
from praisonai.cli.features.n8n import N8nHandler
handler = N8nHandler()
result = handler.convert_yaml_to_n8n(sample_yaml_file)
assert "name" in result
assert "nodes" in result
assert "connections" in result
def test_n8n_workflow_has_trigger_node(self, sample_yaml_file):
"""Test that n8n workflow has a trigger node (webhook or manual)."""
from praisonai.cli.features.n8n import N8nHandler
handler = N8nHandler()
result = handler.convert_yaml_to_n8n(sample_yaml_file)
nodes = result["nodes"]
# Check for webhook or manual trigger nodes
trigger_nodes = [n for n in nodes if "trigger" in n.get("type", "").lower()
or "webhook" in n.get("type", "").lower()
or "trigger" in n.get("name", "").lower()
or "webhook" in n.get("name", "").lower()]
assert len(trigger_nodes) >= 1
def test_n8n_workflow_has_per_agent_nodes(self, sample_yaml_file):
"""Test that n8n workflow has HTTP Request nodes for each agent."""
from praisonai.cli.features.n8n import N8nHandler
handler = N8nHandler()
result = handler.convert_yaml_to_n8n(sample_yaml_file)
nodes = result["nodes"]
# Should have HTTP Request nodes for each agent
http_nodes = [n for n in nodes if "httpRequest" in n.get("type", "")]
assert len(http_nodes) >= 2 # At least 2 agents
# Each node should call a specific agent endpoint
for node in http_nodes:
assert "/agents/" in node["parameters"]["url"]
def test_n8n_workflow_nodes_are_connected(self, sample_yaml_file):
"""Test that nodes are properly connected."""
from praisonai.cli.features.n8n import N8nHandler
handler = N8nHandler()
result = handler.convert_yaml_to_n8n(sample_yaml_file)
connections = result["connections"]
assert len(connections) > 0
def test_n8n_workflow_name_from_yaml(self, sample_yaml_file):
"""Test that workflow name is taken from YAML."""
from praisonai.cli.features.n8n import N8nHandler
handler = N8nHandler()
result = handler.convert_yaml_to_n8n(sample_yaml_file)
assert result["name"] == "Test Workflow"
def test_generate_n8n_url(self, sample_yaml_file):
"""Test URL generation for n8n."""
from praisonai.cli.features.n8n import N8nHandler
handler = N8nHandler()
workflow_json = handler.convert_yaml_to_n8n(sample_yaml_file)
url = handler.generate_n8n_url(workflow_json, "http://localhost:5678")
assert url.startswith("http://localhost:5678")
assert "workflow" in url.lower()
def test_node_positions_are_sequential(self, sample_yaml_file):
"""Test that node positions are laid out sequentially."""
from praisonai.cli.features.n8n import N8nHandler
handler = N8nHandler()
result = handler.convert_yaml_to_n8n(sample_yaml_file)
nodes = result["nodes"]
positions = [n.get("position", [0, 0]) for n in nodes]
# X positions should increase for sequential workflow
x_positions = [p[0] for p in positions]
assert x_positions == sorted(x_positions), "Nodes should be positioned left to right"
class TestN8nCLIIntegration:
"""Tests for CLI integration of n8n feature."""
def test_n8n_flag_in_argparser(self):
"""Test that --n8n flag is recognized by argparser."""
import argparse
# This will be tested after we add the flag to main.py
pass
def test_execute_method_exists(self):
"""Test that execute method exists."""
from praisonai.cli.features.n8n import N8nHandler
handler = N8nHandler()
assert hasattr(handler, 'execute')
assert callable(handler.execute)
class TestN8nWorkflowPatterns:
"""Tests for different workflow patterns."""
@pytest.fixture
def parallel_workflow_yaml(self):
"""YAML with parallel steps."""
return """
name: Parallel Workflow
agents:
agent1:
name: Agent1
role: Role1
goal: Goal1
agent2:
name: Agent2
role: Role2
goal: Goal2
steps:
- parallel:
- agent: agent1
action: "Do task 1"
- agent: agent2
action: "Do task 2"
"""
@pytest.fixture
def route_workflow_yaml(self):
"""YAML with route/decision steps."""
return """
name: Route Workflow
agents:
classifier:
name: Classifier
role: Classifier
goal: Classify input
handler_a:
name: HandlerA
role: Handler
goal: Handle type A
handler_b:
name: HandlerB
role: Handler
goal: Handle type B
steps:
- agent: classifier
action: "Classify the input"
- route:
type_a: [handler_a]
type_b: [handler_b]
"""
def test_parallel_workflow_creates_split_node(self, parallel_workflow_yaml):
"""Test that parallel steps create appropriate n8n structure."""
import tempfile
import os
from praisonai.cli.features.n8n import N8nHandler
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
f.write(parallel_workflow_yaml)
f.flush()
handler = N8nHandler()
result = handler.convert_yaml_to_n8n(f.name)
# Should have nodes for parallel execution
assert len(result["nodes"]) >= 2
os.unlink(f.name)
if __name__ == "__main__":
pytest.main([__file__, "-v"])
+9 -1
View File
@@ -1,15 +1,21 @@
#!/usr/bin/env python3
"""Test script to verify Phase 2 refactoring maintains backward compatibility"""
"""Test script to verify Phase 2 refactoring maintains backward compatibility
NOTE: These tests make real API calls and require valid API keys.
They are marked with @pytest.mark.real to be excluded from CI.
"""
import asyncio
import sys
import os
import pytest
# Add the correct path for imports
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'src/praisonai-agents'))
from praisonaiagents.llm import LLM
@pytest.mark.real
def test_sync_methods():
"""Test synchronous methods still work correctly"""
print("Testing synchronous methods...")
@@ -52,6 +58,7 @@ def test_sync_methods():
print("✅ Synchronous methods working correctly\n")
@pytest.mark.real
async def test_async_methods():
"""Test asynchronous methods still work correctly"""
print("Testing asynchronous methods...")
@@ -94,6 +101,7 @@ async def test_async_methods():
print("✅ Asynchronous methods working correctly\n")
@pytest.mark.real
def test_edge_cases():
"""Test edge cases and special scenarios"""
print("Testing edge cases...")
+4 -4
View File
@@ -4197,7 +4197,7 @@ requires-dist = [
{ name = "praisonai-tools", marker = "extra == 'autogen'", specifier = ">=0.0.22" },
{ name = "praisonai-tools", marker = "extra == 'autogen-v4'", specifier = ">=0.0.22" },
{ name = "praisonai-tools", marker = "extra == 'crewai'", specifier = ">=0.0.22" },
{ name = "praisonaiagents", specifier = ">=0.1.7" },
{ name = "praisonaiagents", specifier = ">=0.1.14" },
{ name = "pyautogen", marker = "extra == 'autogen'", specifier = "==0.2.29" },
{ name = "pydantic", marker = "extra == 'chat'", specifier = "<=2.10.1" },
{ name = "pydantic", marker = "extra == 'code'", specifier = "<=2.10.1" },
@@ -4252,7 +4252,7 @@ wheels = [
[[package]]
name = "praisonaiagents"
version = "0.1.7"
version = "0.1.14"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "aiohttp" },
@@ -4261,9 +4261,9 @@ dependencies = [
{ name = "pydantic" },
{ name = "rich" },
]
sdist = { url = "https://files.pythonhosted.org/packages/07/19/d59400d701c3433c1b0c08024f34d9e17d63bb6257c0f56e39c9715f66a8/praisonaiagents-0.1.7.tar.gz", hash = "sha256:42ac442ff80bdbcca0f25987d8ac722e4ce9480a650acecde1c6fc7fa124acdf", size = 615505, upload-time = "2025-12-17T13:15:11.644Z" }
sdist = { url = "https://files.pythonhosted.org/packages/aa/19/3493e61f00afc8f2dc6beed5407dd5d18e36f70c96107aa0d8f23c136f26/praisonaiagents-0.1.14.tar.gz", hash = "sha256:3e86b548fa6a44f37b84fc9588d275ede34a303d2f5ac3d0c596f4e2464131b9", size = 621584, upload-time = "2025-12-18T14:39:37.194Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b1/cb/cdd00ec34abb60925133ed8643432f93f45f4c679ccd3f599d01a26fc354/praisonaiagents-0.1.7-py3-none-any.whl", hash = "sha256:b07ecc2b088f9bcbd4dab352685f35142b422d002581aaa8b992a8f7033d6d13", size = 545981, upload-time = "2025-12-17T13:15:09.434Z" },
{ url = "https://files.pythonhosted.org/packages/0b/f7/618f7278370091eb62c118331646010d139dd88a8477c8bea91e18ac274e/praisonaiagents-0.1.14-py3-none-any.whl", hash = "sha256:034c862bb293248c29f8bf6ce4a910cb17bc8b980dd5cc38d790a4cb16a08729", size = 548184, upload-time = "2025-12-18T14:39:34.989Z" },
]
[[package]]