Compare commits

...
6 Commits
Author SHA1 Message Date
Mervin Praison afa37b0782 Release v2.3.37 2025-12-18 17:41:16 +00:00
Mervin Praison 43a15e6049 Release v2.3.36 2025-12-18 16:42:58 +00:00
Mervin Praison 8bf5a08246 Release v2.3.35 2025-12-18 16:28:02 +00:00
Mervin Praison aa81094ac3 Release v2.3.34 2025-12-18 14:55:33 +00:00
Mervin Praison c74fd7e925 Release v2.3.33 2025-12-18 14:41:36 +00:00
Mervin Praison 249acf7965 docs: Add performance benchmarks and auto-update functionality
Add performance comparison table to README showing PraisonAI as fastest framework for agent instantiation. Include benchmarks for LangGraph and CrewAI frameworks. Implement auto-save functionality to generate BENCHMARK_RESULTS.md and update README.md with latest benchmark results.
2025-12-18 13:30:06 +00:00
44 changed files with 5003 additions and 128 deletions

No files matched your search

+57
View File
@@ -34,6 +34,7 @@ PraisonAI is a production-ready Multi-AI Agents framework with self-reflection,
## 📑 Table of Contents
- [⚡ Performance](#-performance)
- [🚀 Quick Start](#-quick-start)
- [📦 Installation](#-installation)
- [Python SDK](#python-sdk)
@@ -73,6 +74,32 @@ PraisonAI is a production-ready Multi-AI Agents framework with self-reflection,
---
## ⚡ Performance
PraisonAI Agents is the **fastest AI agent framework** for agent instantiation.
| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
| **PraisonAI** | **3.77** | **1.00x (fastest)** |
| OpenAI Agents SDK | 5.26 | 1.39x |
| Agno | 5.64 | 1.49x |
| PraisonAI (LiteLLM) | 7.56 | 2.00x |
| PydanticAI | 226.94 | 60.16x |
| LangGraph | 4,558.71 | 1,209x |
| CrewAI | 15,607.92 | 4,138x |
<details>
<summary>Run benchmarks yourself</summary>
```bash
cd praisonai-agents
python benchmarks/simple_benchmark.py
```
</details>
---
## 🚀 Quick Start
Get started with PraisonAI in under 1 minute:
@@ -1392,6 +1419,36 @@ praisonai commit
praisonai commit --push
```
### Serve CLI (API Server):
```bash
# Start API server for agents defined in YAML
praisonai serve agents.yaml
# With custom port and host
praisonai serve agents.yaml --port 8005 --host 0.0.0.0
# Alternative flag style
praisonai agents.yaml --serve
# The server provides:
# POST /agents - Run all agents sequentially
# POST /agents/{name} - Run specific agent (e.g., /agents/researcher)
# GET /agents/list - List available agents
```
### n8n Integration CLI:
```bash
# Export workflow to n8n and open in browser
praisonai agents.yaml --n8n
# With custom n8n URL
praisonai agents.yaml --n8n --n8n-url http://localhost:5678
# Set N8N_API_KEY for auto-import
export N8N_API_KEY="your-api-key"
praisonai agents.yaml --n8n
```
### @Mentions in Prompts:
```bash
# Include file content in prompt
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.32" \
"praisonai>=2.3.37" \
"praisonai[chat]" \
"embedchain[github,youtube]"
+1 -1
View File
@@ -20,7 +20,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.32" \
"praisonai>=2.3.37" \
"praisonai[ui]" \
"praisonai[chat]" \
"praisonai[realtime]" \
+25
View File
@@ -0,0 +1,25 @@
# Dockerfile for PraisonAI Serve API
# Deploy to Railway, Render, Fly.io, or any container platform
#
# Build: docker build -f Dockerfile.serve -t praisonai-serve .
# Run: docker run -p 8005:8005 -e OPENAI_API_KEY=sk-xxx praisonai-serve
FROM python:3.11-slim
WORKDIR /app
# Install dependencies
RUN pip install --no-cache-dir praisonai praisonaiagents
# Copy agents configuration
COPY agents.yaml .
# Expose the API port
EXPOSE 8005
# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD curl -f http://localhost:8005/health || exit 1
# Start the API server
CMD ["praisonai", "serve", "agents.yaml", "--host", "0.0.0.0", "--port", "8005"]
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.32" \
"praisonai>=2.3.37" \
"praisonai[ui]" \
"praisonai[crewai]"
@@ -0,0 +1,35 @@
# PraisonAI Agents - Benchmark Results
**Generated:** 2025-12-18 14:40:22
**Iterations:** 100
**Test:** Agent instantiation (without tools)
## Results
| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
| **PraisonAI** | **3.77** | **1.00x (fastest)** |
| OpenAI Agents SDK | 5.26 | 1.39x |
| Agno | 5.64 | 1.49x |
| PraisonAI (LiteLLM) | 7.56 | 2.00x |
| PydanticAI | 226.94 | 60.16x |
| LangGraph | 4,558.71 | 1,209x |
| CrewAI | 15,607.92 | 4,138x |
## Package Versions
| Package | Version |
|---------|--------|
| PraisonAI | 0.1.5 |
| Agno | 2.3.14 |
| PydanticAI | 1.35.0 |
| OpenAI Agents SDK | 0.6.3 |
| LangGraph | 1.0.5 |
| CrewAI | 1.6.1 |
## How to Reproduce
```bash
cd praisonai-agents
python benchmarks/simple_benchmark.py
```
@@ -0,0 +1,31 @@
# PraisonAI Agents - Real Execution Benchmark
**Generated:** 2025-12-18 14:43:52
**Model:** gpt-4o-mini
**Iterations:** 3
**Prompt:** "What is 2+2? Reply with just the number."
## Results
| Framework | Method | Avg Time | Relative |
|-----------|--------|----------|----------|
| **PraisonAI** | `agent.start()` | **0.45s** | **1.00x (fastest)** |
| PraisonAI (LiteLLM) | `agent.start()` | 0.55s | 1.22x |
| CrewAI | `crew.kickoff()` | 0.58s | 1.28x |
| Agno | `agent.run()` | 0.92s | 2.05x |
## Package Versions
| Package | Version |
|---------|--------|
| PraisonAI | 0.1.5 |
| Agno | 2.3.14 |
| CrewAI | 1.6.1 |
## How to Reproduce
```bash
export OPENAI_API_KEY=your_key
cd praisonai-agents
python benchmarks/execution_benchmark.py
```
@@ -0,0 +1,72 @@
# PraisonAI Agents - Real-World Execution Benchmark
**Generated:** 2025-12-18 14:33:49
**Model:** gpt-4o-mini
**Iterations:** 3 per test
## Results Summary
| Test | PraisonAI (avg) | Agno (avg) | Difference |
|------|-----------------|------------|------------|
| Single Agent | 2.04s | 3.18s | PraisonAI 1.56x faster |
| 2 Agents | 2.49s | 20.49s | PraisonAI 8.21x faster |
| 3 Agents | 6.52s | 13.97s | PraisonAI 2.14x faster |
## Detailed Results
### praisonai_single
- Average: 2.04s
- Min: 1.95s
- Max: 2.19s
- Std Dev: 0.13s
- Times: ['2.19s', '1.95s', '2.00s']
### agno_single
- Average: 3.18s
- Min: 3.00s
- Max: 3.51s
- Std Dev: 0.28s
- Times: ['3.51s', '3.00s', '3.04s']
### praisonai_two_agents
- Average: 2.49s
- Min: 2.22s
- Max: 2.65s
- Std Dev: 0.24s
- Times: ['2.65s', '2.22s', '2.62s']
### agno_two_agents
- Average: 20.49s
- Min: 9.96s
- Max: 35.05s
- Std Dev: 13.02s
- Times: ['16.46s', '9.96s', '35.05s']
### praisonai_three_agents
- Average: 6.52s
- Min: 5.62s
- Max: 7.39s
- Std Dev: 0.88s
- Times: ['5.62s', '6.54s', '7.39s']
### agno_three_agents
- Average: 13.97s
- Min: 11.69s
- Max: 15.32s
- Std Dev: 1.99s
- Times: ['14.91s', '11.69s', '15.32s']
## Package Versions
| Package | Version |
|---------|--------|
| PraisonAI | 0.1.5 |
| Agno | 2.3.14 |
## How to Reproduce
```bash
export OPENAI_API_KEY=your_key
cd praisonai-agents
python benchmarks/real_benchmark.py
```
@@ -0,0 +1,22 @@
# PraisonAI Agents - Benchmark Results
**Generated:** 2025-12-18 13:23:43
## Agent Instantiation Time
| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
| **PraisonAI** | **4.16** | **1.00x (fastest)** |
| OpenAI Agents SDK | 5.55 | 1.33x |
| Agno | 5.58 | 1.34x |
| PraisonAI (LiteLLM) | 8.52 | 2.05x |
| PydanticAI | 225.87 | 54.25x |
| LangGraph | 4,349.54 | 1,045x |
| CrewAI | 15,911.11 | 3,821x |
## How to Reproduce
```bash
cd praisonai-agents
python benchmarks/simple_benchmark.py
```
@@ -0,0 +1,33 @@
# PraisonAI Agents - Tools Benchmark Results
**Generated:** 2025-12-18 14:40:35
**Iterations:** 100
**Test:** Agent instantiation WITH TOOLS
## Results
| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
| **PraisonAI** | **3.24** | **1.00x (fastest)** |
| Agno | 5.12 | 1.58x |
| PraisonAI (LiteLLM) | 8.59 | 2.65x |
| OpenAI Agents SDK | 279.95 | 86.44x |
| LangGraph | 2,310.82 | 713x |
| CrewAI | 15,773.44 | 4,870x |
## Package Versions
| Package | Version |
|---------|--------|
| PraisonAI | 0.1.5 |
| Agno | 2.3.14 |
| OpenAI Agents SDK | 0.6.3 |
| LangGraph | 1.0.5 |
| CrewAI | 1.6.1 |
## How to Reproduce
```bash
cd praisonai-agents
python benchmarks/tools_benchmark.py
```
@@ -0,0 +1,296 @@
#!/usr/bin/env python3
"""
PraisonAI Agents - Real Execution Benchmark
Benchmarks ACTUAL agent execution with LLM API calls.
Tests single agent running a simple task across frameworks.
Usage:
export OPENAI_API_KEY=your_key
python benchmarks/execution_benchmark.py
python benchmarks/execution_benchmark.py --no-save
python benchmarks/execution_benchmark.py --iterations 5
python benchmarks/execution_benchmark.py --model gpt-4o --iterations 10
"""
import time
import os
import argparse
from importlib.metadata import version as get_version
# Defaults (can be overridden via CLI)
DEFAULT_MODEL = "gpt-4o-mini"
DEFAULT_ITERATIONS = 3
DEFAULT_PROMPT = "What is 2+2? Reply with just the number."
def get_package_versions():
"""Get version numbers for benchmarked packages."""
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
'CrewAI': 'crewai',
}
versions = {}
for name, pkg in packages.items():
try:
versions[name] = get_version(pkg)
except:
versions[name] = 'not installed'
return versions
def benchmark_praisonai(model, iterations, prompt):
"""Benchmark PraisonAI agent execution using agent.start()"""
from praisonaiagents import Agent
print("\n--- PraisonAI ---")
print("Method: agent.start()")
times = []
for i in range(iterations):
agent = Agent(
name="Calculator",
instructions="You are a helpful assistant. Be very brief.",
llm=model,
verbose=False
)
start = time.perf_counter()
response = agent.start(prompt)
elapsed = time.perf_counter() - start
times.append(elapsed)
print(f" Run {i+1}: {elapsed:.2f}s - Response: {str(response)[:50]}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def benchmark_praisonai_litellm(model, iterations, prompt):
"""Benchmark PraisonAI with LiteLLM backend using agent.start()"""
from praisonaiagents import Agent
print("\n--- PraisonAI (LiteLLM) ---")
print("Method: agent.start()")
times = []
for i in range(iterations):
agent = Agent(
name="Calculator",
instructions="You are a helpful assistant. Be very brief.",
llm=f"openai/{model}",
verbose=False
)
start = time.perf_counter()
response = agent.start(prompt)
elapsed = time.perf_counter() - start
times.append(elapsed)
print(f" Run {i+1}: {elapsed:.2f}s - Response: {str(response)[:50]}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def benchmark_agno(model, iterations, prompt):
"""Benchmark Agno agent execution using agent.run()"""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
print("\n--- Agno ---")
print("Method: agent.run()")
times = []
for i in range(iterations):
agent = Agent(
model=OpenAIChat(id=model),
instructions=["You are a helpful assistant. Be very brief."],
)
start = time.perf_counter()
response = agent.run(prompt)
elapsed = time.perf_counter() - start
times.append(elapsed)
# Extract content from response
content = str(response.content)[:50] if hasattr(response, 'content') else str(response)[:50]
print(f" Run {i+1}: {elapsed:.2f}s - Response: {content}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def benchmark_crewai(model, iterations, prompt):
"""Benchmark CrewAI agent execution using crew.kickoff()"""
from crewai import Agent, Task, Crew
print("\n--- CrewAI ---")
print("Method: crew.kickoff()")
times = []
for i in range(iterations):
agent = Agent(
role="Calculator",
goal="Answer math questions",
backstory="You are a helpful assistant.",
llm=model,
verbose=False
)
task = Task(
description=prompt,
expected_output="A number",
agent=agent
)
crew = Crew(
agents=[agent],
tasks=[task],
verbose=False
)
start = time.perf_counter()
response = crew.kickoff()
elapsed = time.perf_counter() - start
times.append(elapsed)
print(f" Run {i+1}: {elapsed:.2f}s - Response: {str(response)[:50]}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def save_results(results: dict, model: str, iterations: int, prompt: str):
"""Save results to markdown file."""
import os
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), 'EXECUTION_BENCHMARK_RESULTS.md')
versions = get_package_versions()
# Sort by time
sorted_results = sorted(results.items(), key=lambda x: x[1])
fastest = sorted_results[0][1]
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Real Execution Benchmark\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write(f'**Model:** {model}\n')
f.write(f'**Iterations:** {iterations}\n')
f.write(f'**Prompt:** "{prompt}"\n\n')
f.write('## Results\n\n')
f.write('| Framework | Method | Avg Time | Relative |\n')
f.write('|-----------|--------|----------|----------|\n')
methods = {
'PraisonAI': 'agent.start()',
'PraisonAI (LiteLLM)': 'agent.start()',
'Agno': 'agent.run()',
'CrewAI': 'crew.kickoff()',
}
for name, avg in sorted_results:
ratio = avg / fastest
method = methods.get(name, 'unknown')
if ratio == 1.0:
f.write(f'| **{name}** | `{method}` | **{avg:.2f}s** | **1.00x (fastest)** |\n')
else:
f.write(f'| {name} | `{method}` | {avg:.2f}s | {ratio:.2f}x |\n')
f.write('\n## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('export OPENAI_API_KEY=your_key\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/execution_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='PraisonAI Agents - Real Execution Benchmark')
parser.add_argument('--no-save', action='store_true', help='Do not save results to file')
parser.add_argument('--model', '-m', type=str, default=DEFAULT_MODEL, help=f'Model to use (default: {DEFAULT_MODEL})')
parser.add_argument('--iterations', '-i', type=int, default=DEFAULT_ITERATIONS, help=f'Number of iterations (default: {DEFAULT_ITERATIONS})')
parser.add_argument('--prompt', '-p', type=str, default=DEFAULT_PROMPT, help=f'Prompt to use (default: "{DEFAULT_PROMPT}")')
args = parser.parse_args()
# Check API key
if not os.environ.get('OPENAI_API_KEY'):
print("ERROR: Set OPENAI_API_KEY environment variable")
exit(1)
model = args.model
iterations = args.iterations
prompt = args.prompt
print('=' * 60)
print('PraisonAI Agents - Real Execution Benchmark')
print('=' * 60)
print(f'\nModel: {model}')
print(f'Iterations: {iterations}')
print(f'Prompt: "{prompt}"')
versions = get_package_versions()
print('\nPackage versions:')
for pkg, ver in versions.items():
print(f' {pkg}: {ver}')
results = {}
# Run benchmarks
try:
results['PraisonAI'] = benchmark_praisonai(model, iterations, prompt)
except Exception as e:
print(f"PraisonAI error: {e}")
try:
results['PraisonAI (LiteLLM)'] = benchmark_praisonai_litellm(model, iterations, prompt)
except Exception as e:
print(f"PraisonAI (LiteLLM) error: {e}")
try:
results['Agno'] = benchmark_agno(model, iterations, prompt)
except Exception as e:
print(f"Agno error: {e}")
try:
results['CrewAI'] = benchmark_crewai(model, iterations, prompt)
except Exception as e:
print(f"CrewAI error: {e}")
# Summary
print('\n' + '=' * 60)
print('SUMMARY')
print('=' * 60)
if results:
sorted_results = sorted(results.items(), key=lambda x: x[1])
fastest = sorted_results[0][1]
print(f"\n{'Framework':<25} {'Avg Time':<12} {'Relative':<10}")
print('-' * 47)
for name, avg in sorted_results:
ratio = avg / fastest
print(f'{name:<25} {avg:<12.2f}s {ratio:.2f}x')
print('\n' + '=' * 60)
# Save results (unless --no-save)
if results and not args.no_save:
save_results(results, model, iterations, prompt)
elif args.no_save:
print('\nResults not saved (--no-save flag used)')
@@ -259,6 +259,110 @@ def instantiate_crewai_agent():
)
def get_package_versions():
"""Get version numbers for all benchmarked packages."""
from importlib.metadata import version as get_version
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
'PydanticAI': 'pydantic-ai',
'OpenAI Agents SDK': 'openai-agents',
'LangGraph': 'langgraph',
'CrewAI': 'crewai'
}
versions = {}
for display_name, pkg_name in packages.items():
try:
versions[display_name] = get_version(pkg_name)
except Exception:
versions[display_name] = 'not installed'
return versions
def save_benchmark_results(results: dict, baseline):
"""Save benchmark results to BENCHMARK_RESULTS.md and update README.md"""
import os
import re
from datetime import datetime
benchmarks_dir = os.path.dirname(__file__)
filepath = os.path.join(benchmarks_dir, 'BENCHMARK_RESULTS.md')
# Get package versions
versions = get_package_versions()
display_names = {
'agno': 'Agno',
'praisonai': 'PraisonAI',
'praisonai_litellm': 'PraisonAI (LiteLLM)',
'pydantic': 'PydanticAI',
'openai_agents': 'OpenAI Agents SDK',
'langgraph': 'LangGraph',
'crewai': 'CrewAI'
}
# Build table rows
table_rows = []
sorted_results = sorted(results.items(), key=lambda x: x[1].avg_run_time)
for name, result in sorted_results:
display_name = display_names.get(name, name)
avg_us = result.avg_run_time * 1e6
ratio = result.avg_run_time / baseline.avg_run_time if baseline.avg_run_time > 0 else 0
if name == 'praisonai':
table_rows.append(f'| **{display_name}** | **{avg_us:.2f}** | **1.00x (fastest)** |')
else:
avg_str = f'{avg_us:,.2f}' if avg_us >= 1000 else f'{avg_us:.2f}'
ratio_str = f'{ratio:,.0f}x' if ratio >= 100 else f'{ratio:.2f}x'
table_rows.append(f'| {display_name} | {avg_str} | {ratio_str} |')
# Save to BENCHMARK_RESULTS.md
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Benchmark Results\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write('**Methodology:** 10 warmup runs + 1000 measured iterations\n\n')
f.write('## Agent Instantiation Time\n\n')
f.write('| Framework | Avg Time (μs) | Relative |\n')
f.write('|-----------|---------------|----------|\n')
f.write('\n'.join(table_rows) + '\n')
f.write('\n## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/performance_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
# Update README.md
readme_path = os.path.join(benchmarks_dir, '..', '..', '..', 'README.md')
if os.path.exists(readme_path):
with open(readme_path, 'r') as f:
content = f.read()
new_table = '''| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
''' + '\n'.join(table_rows)
pattern = r'(\| Framework \| Avg Time \(μs\) \| Relative \|\n\|[-|]+\|\n)(\|[^\n]+\|\n)+'
if re.search(pattern, content):
content = re.sub(pattern, new_table + '\n', content)
with open(readme_path, 'w') as f:
f.write(content)
print(f'README.md updated: {readme_path}')
if __name__ == "__main__":
print("="*70)
print("PraisonAI Agents - Comprehensive Performance Benchmark")
@@ -414,3 +518,6 @@ if __name__ == "__main__":
print(f"{display_name:<25} {result.avg_run_time*1e6:<15.2f} {result.avg_memory_usage*1024:<18.2f} {time_str:<15}")
print("\n" + "="*70)
# Save results to files
save_benchmark_results(results, baseline)
@@ -0,0 +1,481 @@
#!/usr/bin/env python3
"""
PraisonAI Agents - Real-World Execution Benchmark
Benchmarks actual agent execution with LLM calls across frameworks.
Tests single agent, 2-agent, and 3-agent workflows.
Usage:
python benchmarks/real_benchmark.py
"""
import time
import statistics
import os
from typing import Literal, List
from dataclasses import dataclass, field
from importlib.metadata import version as get_version
# Ensure API key is set
if not os.environ.get('OPENAI_API_KEY'):
print("ERROR: OPENAI_API_KEY environment variable not set")
exit(1)
@dataclass
class ExecutionResult:
"""Execution benchmark results"""
times: List[float] = field(default_factory=list)
@property
def avg_time(self) -> float:
return statistics.mean(self.times) if self.times else 0
@property
def min_time(self) -> float:
return min(self.times) if self.times else 0
@property
def max_time(self) -> float:
return max(self.times) if self.times else 0
@property
def std_dev(self) -> float:
return statistics.stdev(self.times) if len(self.times) > 1 else 0
def get_package_versions():
"""Get version numbers for benchmarked packages."""
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
}
versions = {}
for display_name, pkg_name in packages.items():
try:
versions[display_name] = get_version(pkg_name)
except Exception:
versions[display_name] = 'not installed'
return versions
# ============================================================================
# Tool Definitions
# ============================================================================
def get_weather(city: Literal["nyc", "sf"]) -> str:
"""Get weather information for a city."""
if city == "nyc":
return "New York: Cloudy, 45°F, 60% humidity"
return "San Francisco: Sunny, 72°F, 40% humidity"
def get_time(timezone: Literal["est", "pst"]) -> str:
"""Get current time in a timezone."""
if timezone == "est":
return "Eastern Time: 2:30 PM"
return "Pacific Time: 11:30 AM"
def calculate(expression: str) -> str:
"""Calculate a math expression."""
try:
result = eval(expression)
return f"Result: {result}"
except Exception:
return "Error: Invalid expression"
# ============================================================================
# PraisonAI Benchmarks
# ============================================================================
def benchmark_praisonai_single(iterations: int = 3) -> ExecutionResult:
"""Benchmark single PraisonAI agent execution."""
from praisonaiagents import Agent
result = ExecutionResult()
for i in range(iterations):
print(f" PraisonAI single agent iteration {i+1}/{iterations}...")
agent = Agent(
name="Assistant",
instructions="You are a helpful assistant. Be very brief.",
llm="gpt-4o-mini",
tools=[get_weather],
verbose=False
)
start = time.perf_counter()
response = agent.start("What's the weather in NYC? Reply in one sentence.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
def benchmark_praisonai_two_agents(iterations: int = 3) -> ExecutionResult:
"""Benchmark 2 PraisonAI agents working together."""
from praisonaiagents import Agent, PraisonAIAgents
result = ExecutionResult()
for i in range(iterations):
print(f" PraisonAI 2-agent iteration {i+1}/{iterations}...")
researcher = Agent(
name="Researcher",
role="Research Assistant",
goal="Gather weather information",
instructions="Get weather data and pass to reporter. Be brief.",
llm="gpt-4o-mini",
tools=[get_weather],
verbose=False
)
reporter = Agent(
name="Reporter",
role="News Reporter",
goal="Create weather report",
instructions="Create a one-sentence weather summary.",
llm="gpt-4o-mini",
verbose=False
)
agents = PraisonAIAgents(
agents=[researcher, reporter],
process="sequential",
verbose=False
)
start = time.perf_counter()
response = agents.start("Get NYC weather and create a brief report.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
def benchmark_praisonai_three_agents(iterations: int = 3) -> ExecutionResult:
"""Benchmark 3 PraisonAI agents working together."""
from praisonaiagents import Agent, PraisonAIAgents
result = ExecutionResult()
for i in range(iterations):
print(f" PraisonAI 3-agent iteration {i+1}/{iterations}...")
weather_agent = Agent(
name="WeatherAgent",
role="Weather Specialist",
goal="Get weather data",
instructions="Fetch weather information. Be brief.",
llm="gpt-4o-mini",
tools=[get_weather],
verbose=False
)
time_agent = Agent(
name="TimeAgent",
role="Time Specialist",
goal="Get time information",
instructions="Fetch timezone information. Be brief.",
llm="gpt-4o-mini",
tools=[get_time],
verbose=False
)
summarizer = Agent(
name="Summarizer",
role="Summary Writer",
goal="Combine information",
instructions="Create a one-sentence summary combining weather and time.",
llm="gpt-4o-mini",
verbose=False
)
agents = PraisonAIAgents(
agents=[weather_agent, time_agent, summarizer],
process="sequential",
verbose=False
)
start = time.perf_counter()
response = agents.start("Get NYC weather and EST time, then summarize.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
# ============================================================================
# Agno Benchmarks
# ============================================================================
def benchmark_agno_single(iterations: int = 3) -> ExecutionResult:
"""Benchmark single Agno agent execution."""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
result = ExecutionResult()
for i in range(iterations):
print(f" Agno single agent iteration {i+1}/{iterations}...")
agent = Agent(
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["You are a helpful assistant. Be very brief."],
tools=[get_weather],
)
start = time.perf_counter()
response = agent.run("What's the weather in NYC? Reply in one sentence.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
def benchmark_agno_two_agents(iterations: int = 3) -> ExecutionResult:
"""Benchmark 2 Agno agents working together (using Team)."""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.team import Team
result = ExecutionResult()
for i in range(iterations):
print(f" Agno 2-agent iteration {i+1}/{iterations}...")
researcher = Agent(
name="Researcher",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Get weather data. Be brief."],
tools=[get_weather],
)
reporter = Agent(
name="Reporter",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Create a one-sentence weather summary."],
)
team = Team(
members=[researcher, reporter],
model=OpenAIChat(id="gpt-4o-mini"),
)
start = time.perf_counter()
response = team.run("Get NYC weather and create a brief report.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
def benchmark_agno_three_agents(iterations: int = 3) -> ExecutionResult:
"""Benchmark 3 Agno agents working together."""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.team import Team
result = ExecutionResult()
for i in range(iterations):
print(f" Agno 3-agent iteration {i+1}/{iterations}...")
weather_agent = Agent(
name="WeatherAgent",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Fetch weather information. Be brief."],
tools=[get_weather],
)
time_agent = Agent(
name="TimeAgent",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Fetch timezone information. Be brief."],
tools=[get_time],
)
summarizer = Agent(
name="Summarizer",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Create a one-sentence summary combining weather and time."],
)
team = Team(
members=[weather_agent, time_agent, summarizer],
model=OpenAIChat(id="gpt-4o-mini"),
)
start = time.perf_counter()
response = team.run("Get NYC weather and EST time, then summarize.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
# ============================================================================
# Save Results
# ============================================================================
def save_results(results: dict, versions: dict):
"""Save benchmark results to file."""
import os
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), 'REAL_BENCHMARK_RESULTS.md')
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Real-World Execution Benchmark\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write('**Model:** gpt-4o-mini\n')
f.write('**Iterations:** 3 per test\n\n')
f.write('## Results Summary\n\n')
f.write('| Test | PraisonAI (avg) | Agno (avg) | Difference |\n')
f.write('|------|-----------------|------------|------------|\n')
tests = ['single', 'two_agents', 'three_agents']
test_names = ['Single Agent', '2 Agents', '3 Agents']
for test, name in zip(tests, test_names):
praison_key = f'praisonai_{test}'
agno_key = f'agno_{test}'
if praison_key in results and agno_key in results:
praison_avg = results[praison_key].avg_time
agno_avg = results[agno_key].avg_time
if praison_avg < agno_avg:
diff = f'PraisonAI {agno_avg/praison_avg:.2f}x faster'
else:
diff = f'Agno {praison_avg/agno_avg:.2f}x faster'
f.write(f'| {name} | {praison_avg:.2f}s | {agno_avg:.2f}s | {diff} |\n')
f.write('\n## Detailed Results\n\n')
for key, result in results.items():
f.write(f'### {key}\n')
f.write(f'- Average: {result.avg_time:.2f}s\n')
f.write(f'- Min: {result.min_time:.2f}s\n')
f.write(f'- Max: {result.max_time:.2f}s\n')
f.write(f'- Std Dev: {result.std_dev:.2f}s\n')
f.write(f'- Times: {[f"{t:.2f}s" for t in result.times]}\n\n')
f.write('## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('export OPENAI_API_KEY=your_key\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/real_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
# ============================================================================
# Main
# ============================================================================
if __name__ == "__main__":
print("="*70)
print("PraisonAI Agents - Real-World Execution Benchmark")
print("="*70)
print("\nThis benchmark measures actual agent execution time including:")
print(" - LLM API calls")
print(" - Tool execution")
print(" - Multi-agent coordination")
print("\nModel: gpt-4o-mini")
print("Iterations: 3 per test")
versions = get_package_versions()
print(f"\nPackage versions:")
for pkg, ver in versions.items():
print(f" {pkg}: {ver}")
results = {}
# Single Agent Tests
print("\n" + "="*70)
print("SINGLE AGENT BENCHMARK")
print("="*70)
print("\nTesting PraisonAI single agent...")
results['praisonai_single'] = benchmark_praisonai_single()
print("\nTesting Agno single agent...")
results['agno_single'] = benchmark_agno_single()
# 2-Agent Tests
print("\n" + "="*70)
print("2-AGENT BENCHMARK")
print("="*70)
print("\nTesting PraisonAI 2 agents...")
results['praisonai_two_agents'] = benchmark_praisonai_two_agents()
print("\nTesting Agno 2 agents (Team)...")
results['agno_two_agents'] = benchmark_agno_two_agents()
# 3-Agent Tests
print("\n" + "="*70)
print("3-AGENT BENCHMARK")
print("="*70)
print("\nTesting PraisonAI 3 agents...")
results['praisonai_three_agents'] = benchmark_praisonai_three_agents()
print("\nTesting Agno 3 agents (Team)...")
results['agno_three_agents'] = benchmark_agno_three_agents()
# Summary
print("\n" + "="*70)
print("RESULTS SUMMARY")
print("="*70)
print(f"\n{'Test':<20} {'PraisonAI (avg)':<18} {'Agno (avg)':<15} {'Difference':<20}")
print("-" * 73)
tests = [
('Single Agent', 'praisonai_single', 'agno_single'),
('2 Agents', 'praisonai_two_agents', 'agno_two_agents'),
('3 Agents', 'praisonai_three_agents', 'agno_three_agents'),
]
for name, praison_key, agno_key in tests:
praison_avg = results[praison_key].avg_time
agno_avg = results[agno_key].avg_time
if praison_avg < agno_avg:
diff = f'PraisonAI {agno_avg/praison_avg:.2f}x faster'
else:
diff = f'Agno {praison_avg/agno_avg:.2f}x faster'
print(f"{name:<20} {praison_avg:<18.2f}s {agno_avg:<15.2f}s {diff:<20}")
print("\n" + "="*70)
# Save results
save_results(results, versions)
@@ -0,0 +1,194 @@
❯ python benchmarks/performance_benchmark.py
/Users/praison/praisonai-package/src/praisonai-agents/praisonaiagents/_warning_patch.py:47: UserWarning: Mixing V1 models and V2 models (or constructs, like `TypeAdapter`) is not supported. Please upgrade `Settings` to V2.
_original_warn(message, category, stacklevel, source)
[13:04:15] INFO [13:04:15] config.py:50
config.py:50 INFO
Using config path:
/Users/praison/.con
fig/crewai/settings
.json
======================================================================
PraisonAI Agents - Comprehensive Performance Benchmark
======================================================================
This benchmark measures:
1. Agent instantiation time (how fast an agent object is created)
2. Memory footprint (how much memory an agent uses)
Methodology:
- 10 warmup runs (not measured)
- 1000 measured iterations
- tracemalloc for memory measurement
- Baseline subtraction for accuracy
Frameworks available:
- PraisonAI: ✅
- Agno: ✅
- PydanticAI: ✅
- OpenAI Agents SDK: ✅
- LangGraph: ✅
- CrewAI: ✅
======================================================================
Testing PraisonAI Agents
======================================================================
============================================================
Benchmarking: PraisonAI
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for PraisonAI:
Average Time: 0.000003 seconds (3.20 μs)
Min Time: 0.000003 seconds
Max Time: 0.000025 seconds
Std Dev: 0.000001 seconds
Median Time: 0.000003 seconds
Avg Memory: 0.003678 MiB (3.77 KiB)
======================================================================
Testing PraisonAI Agents (LiteLLM)
======================================================================
============================================================
Benchmarking: PraisonAI (LiteLLM)
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for PraisonAI (LiteLLM):
Average Time: 0.000007 seconds (7.31 μs)
Min Time: 0.000007 seconds
Max Time: 0.000060 seconds
Std Dev: 0.000002 seconds
Median Time: 0.000007 seconds
Avg Memory: 0.006419 MiB (6.57 KiB)
======================================================================
Testing Agno
======================================================================
============================================================
Benchmarking: Agno
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for Agno:
Average Time: 0.000005 seconds (5.30 μs)
Min Time: 0.000005 seconds
Max Time: 0.000039 seconds
Std Dev: 0.000002 seconds
Median Time: 0.000005 seconds
Avg Memory: 0.006599 MiB (6.76 KiB)
======================================================================
Testing PydanticAI
======================================================================
============================================================
Benchmarking: PydanticAI Agent Instantiation
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for PydanticAI Agent Instantiation:
Average Time: 0.000247 seconds (247.34 μs)
Min Time: 0.000218 seconds
Max Time: 0.000552 seconds
Std Dev: 0.000031 seconds
Median Time: 0.000237 seconds
Avg Memory: 0.028486 MiB (29.17 KiB)
======================================================================
Testing OpenAI Agents SDK
======================================================================
============================================================
Benchmarking: OpenAI Agents SDK Instantiation
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for OpenAI Agents SDK Instantiation:
Average Time: 0.000282 seconds (282.25 μs)
Min Time: 0.000247 seconds
Max Time: 0.001110 seconds
Std Dev: 0.000055 seconds
Median Time: 0.000263 seconds
Avg Memory: 0.029032 MiB (29.73 KiB)
======================================================================
Testing LangGraph
======================================================================
============================================================
Benchmarking: LangGraph Agent Instantiation
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for LangGraph Agent Instantiation:
Average Time: 0.003200 seconds (3200.15 μs)
Min Time: 0.002808 seconds
Max Time: 0.116363 seconds
Std Dev: 0.003628 seconds
Median Time: 0.002972 seconds
Avg Memory: 0.207509 MiB (212.49 KiB)
======================================================================
Testing CrewAI
======================================================================
============================================================
Benchmarking: CrewAI Agent Instantiation
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for CrewAI Agent Instantiation:
Average Time: 0.016772 seconds (16771.69 μs)
Min Time: 0.014404 seconds
Max Time: 0.091841 seconds
Std Dev: 0.003445 seconds
Median Time: 0.015816 seconds
Avg Memory: 0.065050 MiB (66.61 KiB)
======================================================================
RESULTS SUMMARY
======================================================================
Framework Avg Time (μs) Avg Memory (KiB) Relative
-------------------------------------------------------------------------
PraisonAI 3.20 3.77 1.00x
Agno 5.30 6.76 1.66x
PraisonAI (LiteLLM) 7.31 6.57 2.28x
PydanticAI 247.34 29.17 77.25x
OpenAI Agents SDK 282.25 29.73 88.15x
LangGraph 3200.15 212.49 999.46x
CrewAI 16771.69 66.61 5238.06x
======================================================================
❯
@@ -33,8 +33,7 @@ def measure_instantiation(create_fn, iterations=ITERATIONS):
def run_benchmark():
"""Run the benchmark across all available frameworks."""
tools = [sample_tool]
"""Run the benchmark across all available frameworks (without tools)."""
results = {}
print('=' * 60)
@@ -43,15 +42,16 @@ def run_benchmark():
print(f'\nIterations: {ITERATIONS}')
print('Metric: Agent instantiation time (microseconds)\n')
# PraisonAI
# PraisonAI (without tools)
print("Testing PraisonAI...")
from praisonaiagents import Agent as PraisonAgent
results['PraisonAI'] = measure_instantiation(
lambda: PraisonAgent(name='Test', llm='gpt-4o-mini', tools=tools, verbose=False)
lambda: PraisonAgent(name='Test', llm='gpt-4o-mini', verbose=False)
)
results['PraisonAI (LiteLLM)'] = measure_instantiation(
lambda: PraisonAgent(name='Test', llm='openai/gpt-4o-mini', tools=tools, verbose=False)
lambda: PraisonAgent(name='Test', llm='openai/gpt-4o-mini', verbose=False)
)
# Other frameworks for comparison
@@ -61,7 +61,7 @@ def run_benchmark():
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
results['Agno'] = measure_instantiation(
lambda: AgnoAgent(model=OpenAIChat(id='gpt-4o-mini'), tools=tools)
lambda: AgnoAgent(model=OpenAIChat(id='gpt-4o-mini'))
)
except ImportError:
pass
@@ -82,6 +82,43 @@ def run_benchmark():
except ImportError:
pass
try:
from langchain_core.tools import tool as langchain_tool
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
@langchain_tool
def get_weather_lg(city: Literal['nyc', 'sf']):
"""Get weather info."""
return 'sunny' if city == 'sf' else 'cloudy'
results['LangGraph'] = measure_instantiation(
lambda: create_react_agent(model=ChatOpenAI(model='gpt-4o-mini'), tools=[get_weather_lg])
)
except ImportError:
pass
try:
from crewai.agent import Agent as CrewAgent
from crewai.tools import tool as crewai_tool
@crewai_tool("Weather Tool")
def get_weather_crew(city: Literal['nyc', 'sf']):
"""Get weather info."""
return 'sunny' if city == 'sf' else 'cloudy'
results['CrewAI'] = measure_instantiation(
lambda: CrewAgent(
role='Weather Agent',
goal='Provide weather info',
backstory='A weather expert',
tools=[get_weather_crew],
verbose=False
)
)
except ImportError:
pass
# Print results
print('\n' + '=' * 60)
print('RESULTS')
@@ -99,5 +136,108 @@ def run_benchmark():
return results
def get_package_versions():
"""Get version numbers for benchmarked packages."""
from importlib.metadata import version as get_version
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
'PydanticAI': 'pydantic-ai',
'OpenAI Agents SDK': 'openai-agents',
'LangGraph': 'langgraph',
'CrewAI': 'crewai'
}
versions = {}
for display_name, pkg_name in packages.items():
try:
versions[display_name] = get_version(pkg_name)
except Exception:
versions[display_name] = 'not installed'
return versions
def save_results(results: dict, filename: str = 'BENCHMARK_RESULTS.md'):
"""Save benchmark results to a markdown file and update README."""
import os
import re
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), filename)
versions = get_package_versions()
# Find the fastest result (baseline)
sorted_results = sorted(results.items(), key=lambda x: x[1])
fastest_time = sorted_results[0][1] if sorted_results else 1
# Build the table rows
table_rows = []
for name, avg in sorted_results:
ratio = avg / fastest_time
if ratio == 1.0:
table_rows.append(f'| **{name}** | **{avg:.2f}** | **1.00x (fastest)** |')
else:
avg_str = f'{avg:,.2f}' if avg >= 1000 else f'{avg:.2f}'
ratio_str = f'{ratio:,.0f}x' if ratio >= 100 else f'{ratio:.2f}x'
table_rows.append(f'| {name} | {avg_str} | {ratio_str} |')
# Save to BENCHMARK_RESULTS.md
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Benchmark Results\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write(f'**Iterations:** {ITERATIONS}\n')
f.write('**Test:** Agent instantiation (without tools)\n\n')
f.write('## Results\n\n')
f.write('| Framework | Avg Time (μs) | Relative |\n')
f.write('|-----------|---------------|----------|\n')
f.write('\n'.join(table_rows) + '\n')
f.write('\n## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/simple_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
# Also update the main README.md
readme_path = os.path.join(os.path.dirname(__file__), '..', '..', '..', 'README.md')
if os.path.exists(readme_path):
update_readme(readme_path, table_rows)
return filepath
def update_readme(readme_path: str, table_rows: list):
"""Update the performance section in README.md with latest results."""
import re
with open(readme_path, 'r') as f:
content = f.read()
# Build the new table
new_table = '''| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
''' + '\n'.join(table_rows)
# Pattern to match the performance table
pattern = r'(\| Framework \| Avg Time \(μs\) \| Relative \|\n\|[-|]+\|\n)(\|[^\n]+\|\n)+'
if re.search(pattern, content):
content = re.sub(pattern, new_table + '\n', content)
with open(readme_path, 'w') as f:
f.write(content)
print(f'README.md updated: {readme_path}')
if __name__ == '__main__':
run_benchmark()
results = run_benchmark()
save_results(results)
@@ -0,0 +1,213 @@
#!/usr/bin/env python3
"""
PraisonAI Agents - Tools Benchmark
Compares agent instantiation times WITH TOOLS across frameworks.
Usage:
python benchmarks/tools_benchmark.py
"""
import time
from typing import Literal
from importlib.metadata import version as get_version
ITERATIONS = 100
def sample_tool(city: Literal['nyc', 'sf']):
"""Sample tool for benchmark testing."""
if city == 'nyc':
return 'cloudy'
return 'sunny'
def measure_instantiation(create_fn, iterations=ITERATIONS):
"""Measure average instantiation time in microseconds."""
times = []
# Warmup
for _ in range(10):
create_fn()
# Measure
for _ in range(iterations):
start = time.perf_counter()
create_fn()
elapsed = time.perf_counter() - start
times.append(elapsed * 1_000_000) # Convert to microseconds
return sum(times) / len(times)
def run_benchmark():
"""Run the benchmark across all available frameworks WITH TOOLS."""
tools = [sample_tool]
results = {}
print('=' * 60)
print('PraisonAI Agents - Tools Benchmark')
print('=' * 60)
print(f'\nIterations: {ITERATIONS}')
print('Metric: Agent instantiation time WITH TOOLS (microseconds)\n')
# PraisonAI with tools
print("Testing PraisonAI...")
from praisonaiagents import Agent as PraisonAgent
results['PraisonAI'] = measure_instantiation(
lambda: PraisonAgent(name='Test', llm='gpt-4o-mini', tools=tools, verbose=False)
)
results['PraisonAI (LiteLLM)'] = measure_instantiation(
lambda: PraisonAgent(name='Test', llm='openai/gpt-4o-mini', tools=tools, verbose=False)
)
# Other frameworks with tools
print("Testing other frameworks...")
try:
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
results['Agno'] = measure_instantiation(
lambda: AgnoAgent(model=OpenAIChat(id='gpt-4o-mini'), tools=tools)
)
except ImportError:
pass
try:
from agents import Agent as OpenAIAgent, function_tool
results['OpenAI Agents SDK'] = measure_instantiation(
lambda: OpenAIAgent(name='Test', model='gpt-4o-mini', tools=[function_tool(sample_tool)])
)
except ImportError:
pass
try:
from langchain_core.tools import tool as langchain_tool
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
@langchain_tool
def get_weather_lg(city: Literal['nyc', 'sf']):
"""Get weather info."""
return 'sunny' if city == 'sf' else 'cloudy'
results['LangGraph'] = measure_instantiation(
lambda: create_react_agent(model=ChatOpenAI(model='gpt-4o-mini'), tools=[get_weather_lg])
)
except ImportError:
pass
try:
from crewai.agent import Agent as CrewAgent
from crewai.tools import tool as crewai_tool
@crewai_tool("Weather Tool")
def get_weather_crew(city: Literal['nyc', 'sf']):
"""Get weather info."""
return 'sunny' if city == 'sf' else 'cloudy'
results['CrewAI'] = measure_instantiation(
lambda: CrewAgent(
role='Weather Agent',
goal='Provide weather info',
backstory='A weather expert',
tools=[get_weather_crew],
verbose=False
)
)
except ImportError:
pass
# Print results
print('\n' + '=' * 60)
print('RESULTS (WITH TOOLS)')
print('=' * 60)
sorted_results = sorted(results.items(), key=lambda x: x[1])
baseline = sorted_results[0][1] if sorted_results else 1
print(f"\n{'Framework':<25} {'Avg Time (μs)':<15} {'Relative':<10}")
print('-' * 50)
for name, avg in sorted_results:
ratio = avg / baseline
print(f'{name:<25} {avg:<15.2f} {ratio:.2f}x')
print('\n' + '=' * 60)
return results
def get_package_versions():
"""Get version numbers for benchmarked packages."""
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
'OpenAI Agents SDK': 'openai-agents',
'LangGraph': 'langgraph',
'CrewAI': 'crewai'
}
versions = {}
for display_name, pkg_name in packages.items():
try:
versions[display_name] = get_version(pkg_name)
except Exception:
versions[display_name] = 'not installed'
return versions
def save_results(results: dict, filename: str = 'TOOLS_BENCHMARK_RESULTS.md'):
"""Save benchmark results to a markdown file."""
import os
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), filename)
versions = get_package_versions()
# Find the fastest result (baseline)
sorted_results = sorted(results.items(), key=lambda x: x[1])
fastest_time = sorted_results[0][1] if sorted_results else 1
# Build the table rows
table_rows = []
for name, avg in sorted_results:
ratio = avg / fastest_time
if ratio == 1.0:
table_rows.append(f'| **{name}** | **{avg:.2f}** | **1.00x (fastest)** |')
else:
avg_str = f'{avg:,.2f}' if avg >= 1000 else f'{avg:.2f}'
ratio_str = f'{ratio:,.0f}x' if ratio >= 100 else f'{ratio:.2f}x'
table_rows.append(f'| {name} | {avg_str} | {ratio_str} |')
# Save to file
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Tools Benchmark Results\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write(f'**Iterations:** {ITERATIONS}\n')
f.write('**Test:** Agent instantiation WITH TOOLS\n\n')
f.write('## Results\n\n')
f.write('| Framework | Avg Time (μs) | Relative |\n')
f.write('|-----------|---------------|----------|\n')
f.write('\n'.join(table_rows) + '\n')
f.write('\n## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/tools_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
return filepath
if __name__ == '__main__':
results = run_benchmark()
save_results(results)
@@ -305,3 +305,12 @@ __all__.extend([
'LineRange'
])
# AG-UI support (optional, lazy loaded)
try:
from praisonaiagents.ui.agui import AGUI
__all__.append('AGUI')
_agui_available = True
except ImportError:
_agui_available = False
AGUI = None
@@ -1370,6 +1370,61 @@ Context:
agent_names = ", ".join([agent.name for agent in self.agents])
print(f"📊 Available agents for this endpoint ({len(self.agents)}): {agent_names}")
# Create per-agent endpoints for individual agent access
# This allows n8n and other tools to call specific agents
agents_dict = {agent.name.lower().replace(' ', '_'): agent for agent in self.agents}
# Add GET endpoint to list available agents
@_agents_shared_apps[port].get(f"{path}/list")
async def list_agents():
return {
"agents": [
{"name": agent.name, "id": agent.name.lower().replace(' ', '_')}
for agent in self.agents
]
}
# Add per-agent POST endpoints
for agent_id, agent_instance in agents_dict.items():
agent_path = f"{path}/{agent_id}"
# Create a closure to capture the agent instance
def create_agent_handler(agent):
async def handle_single_agent(request: Request):
try:
request_data = await request.json()
query = request_data.get("query", "")
if not query:
raise HTTPException(status_code=400, detail="Missing 'query' field")
except:
raise HTTPException(status_code=400, detail="Invalid JSON body")
try:
if asyncio.iscoroutinefunction(agent.chat):
response = await agent.achat(query)
else:
loop = asyncio.get_running_loop()
response = await loop.run_in_executor(None, lambda q=query: agent.chat(q))
return {
"agent": agent.name,
"query": query,
"response": response
}
except Exception as e:
logging.error(f"Error with agent {agent.name}: {str(e)}", exc_info=True)
return JSONResponse(
status_code=500,
content={"error": f"Agent error: {str(e)}"}
)
return handle_single_agent
# Register the endpoint
_agents_shared_apps[port].post(agent_path)(create_agent_handler(agent_instance))
_agents_registered_endpoints[port][agent_path] = f"{endpoint_id}_{agent_id}"
print(f"🔗 Per-agent endpoints: {', '.join([f'{path}/{aid}' for aid in agents_dict.keys()])}")
# Start the server if it's not already running for this port
if not _agents_server_started.get(port, False):
# Mark the server as started first to prevent duplicate starts
@@ -0,0 +1,13 @@
"""
UI Integrations for PraisonAI Agents
This module provides UI protocol integrations for exposing PraisonAI Agents
via various frontend protocols.
Available integrations:
- agui: AG-UI (Agent-User Interface) protocol for CopilotKit and compatible frontends
"""
from praisonaiagents.ui.agui import AGUI
__all__ = ["AGUI"]
@@ -0,0 +1,22 @@
"""
AG-UI Protocol Integration for PraisonAI Agents
This module provides AG-UI (Agent-User Interface) protocol support,
enabling PraisonAI Agents to be exposed via a standardized streaming API
compatible with CopilotKit and other AG-UI frontends.
Usage:
from praisonaiagents import Agent
from praisonaiagents.ui.agui import AGUI
from fastapi import FastAPI
agent = Agent(name="Assistant", role="Helper", goal="Help users")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
"""
from praisonaiagents.ui.agui.agui import AGUI
__all__ = ["AGUI"]
@@ -0,0 +1,203 @@
"""
AGUI - Main AG-UI Interface Class
Exposes PraisonAI Agents via the AG-UI protocol.
"""
import logging
import uuid
from typing import AsyncIterator, List, Optional, TYPE_CHECKING
from praisonaiagents.ui.agui.types import (
BaseEvent,
RunAgentInput,
)
from praisonaiagents.ui.agui.conversion import (
agui_messages_to_praisonai,
validate_state,
extract_user_input,
)
from praisonaiagents.ui.agui.streaming import (
create_run_error_event,
async_stream_agent_response,
async_stream_agents_response,
)
from praisonaiagents.ui.agui.encoder import EventEncoder
if TYPE_CHECKING:
from praisonaiagents import Agent, PraisonAIAgents
from fastapi import APIRouter
logger = logging.getLogger(__name__)
class AGUI:
"""
AG-UI Interface for PraisonAI Agents.
Exposes a PraisonAI Agent or PraisonAIAgents workflow via the AG-UI protocol,
enabling integration with CopilotKit and other AG-UI compatible frontends.
Usage:
from praisonaiagents import Agent
from praisonaiagents.ui.agui import AGUI
from fastapi import FastAPI
agent = Agent(name="Assistant", role="Helper", goal="Help users")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
Args:
agent: Single PraisonAI Agent instance
agents: PraisonAIAgents instance for multi-agent workflows
name: Name for the AG-UI endpoint (defaults to agent name)
description: Description of the agent
prefix: URL prefix for the router (e.g., "/api/v1")
tags: OpenAPI tags for the router
"""
def __init__(
self,
agent: Optional["Agent"] = None,
agents: Optional["PraisonAIAgents"] = None,
name: Optional[str] = None,
description: Optional[str] = None,
prefix: str = "",
tags: Optional[List[str]] = None,
):
if agent is None and agents is None:
raise ValueError("AGUI requires an agent or agents instance")
self.agent = agent
self.agents = agents
self.prefix = prefix
self.tags = tags or ["AGUI"]
# Set name from agent if not provided
if name:
self.name = name
elif agent and hasattr(agent, 'name'):
self.name = agent.name
elif agents and hasattr(agents, 'name'):
self.name = agents.name
else:
self.name = "PraisonAI Agent"
# Set description
if description:
self.description = description
elif agent and hasattr(agent, 'role'):
self.description = agent.role
else:
self.description = "PraisonAI Agent via AG-UI"
self._router: Optional["APIRouter"] = None
def get_router(self) -> "APIRouter":
"""
Get the FastAPI router for this AGUI instance.
Returns:
FastAPI APIRouter with AG-UI endpoints
"""
from fastapi import APIRouter
from fastapi.responses import StreamingResponse
if self._router is not None:
return self._router
self._router = APIRouter(prefix=self.prefix, tags=self.tags)
# Attach routes
self._attach_routes(self._router)
return self._router
def _attach_routes(self, router: "APIRouter") -> None:
"""Attach AG-UI routes to the router."""
from fastapi.responses import StreamingResponse
encoder = EventEncoder()
@router.post("/agui")
async def run_agent_agui(run_input: RunAgentInput):
"""Run the agent via AG-UI protocol."""
async def event_generator():
async for event in self._run_agent(run_input):
yield encoder.encode(event)
return StreamingResponse(
event_generator(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Methods": "POST, GET, OPTIONS",
"Access-Control-Allow-Headers": "*",
},
)
@router.get("/status")
async def get_status():
"""Get agent status."""
return {"status": "available"}
async def _run_agent(self, run_input: RunAgentInput) -> AsyncIterator[BaseEvent]:
"""
Run the agent with the given input.
Args:
run_input: AG-UI run input
Yields:
AG-UI events
"""
run_id = run_input.run_id or str(uuid.uuid4())
thread_id = run_input.thread_id
try:
# Convert messages
messages = run_input.messages or []
# Extract user input
user_input = extract_user_input(messages)
# Validate state
session_state = validate_state(run_input.state, thread_id)
# Get user_id from forwarded_props if available
user_id = None
if run_input.forwarded_props and isinstance(run_input.forwarded_props, dict):
user_id = run_input.forwarded_props.get("user_id")
# Run appropriate handler
if self.agent:
async for event in async_stream_agent_response(
agent=self.agent,
user_input=user_input,
thread_id=thread_id,
run_id=run_id,
session_state=session_state,
messages=agui_messages_to_praisonai(messages),
):
yield event
elif self.agents:
async for event in async_stream_agents_response(
agents=self.agents,
user_input=user_input,
thread_id=thread_id,
run_id=run_id,
session_state=session_state,
):
yield event
else:
yield create_run_error_event("No agent or agents configured")
except Exception as e:
logger.error(f"Error running agent: {e}", exc_info=True)
yield create_run_error_event(str(e))
@@ -0,0 +1,211 @@
"""
AG-UI Message Conversion
Converts between AG-UI message format and PraisonAI message format.
"""
import logging
from typing import Any, Dict, List, Optional, Set
from pydantic import BaseModel
from praisonaiagents.ui.agui.types import Message, ToolCall, FunctionCall
logger = logging.getLogger(__name__)
def agui_messages_to_praisonai(messages: List[Message]) -> List[Dict[str, Any]]:
"""
Convert AG-UI messages to PraisonAI format.
Args:
messages: List of AG-UI Message objects
Returns:
List of message dictionaries in PraisonAI format
"""
if not messages:
return []
# Track tool_call_ids that have results to avoid duplicates
tool_call_ids_with_results: Set[str] = set()
for msg in messages:
if msg.role == "tool" and msg.tool_call_id:
tool_call_ids_with_results.add(msg.tool_call_id)
result: List[Dict[str, Any]] = []
seen_tool_call_ids: Set[str] = set()
for msg in messages:
# Skip system messages - agent builds its own
if msg.role == "system":
continue
if msg.role == "tool":
# Deduplicate tool results
if msg.tool_call_id in seen_tool_call_ids:
logger.debug(f"Skipping duplicate tool result: {msg.tool_call_id}")
continue
if msg.tool_call_id:
seen_tool_call_ids.add(msg.tool_call_id)
result.append({
"role": "tool",
"content": msg.content or "",
"tool_call_id": msg.tool_call_id,
})
elif msg.role == "assistant":
msg_dict: Dict[str, Any] = {
"role": "assistant",
"content": msg.content or "",
}
if msg.tool_calls:
# Include all tool_calls (filtering only when there are results to match)
tool_calls_list = [
{
"id": tc.id,
"function": {
"name": tc.function.name,
"arguments": tc.function.arguments,
},
"type": "function",
}
for tc in msg.tool_calls
]
if tool_calls_list:
msg_dict["tool_calls"] = tool_calls_list
result.append(msg_dict)
elif msg.role == "user":
result.append({
"role": "user",
"content": msg.content or "",
})
else:
logger.warning(f"Unknown message role: {msg.role}")
return result
def praisonai_messages_to_agui(messages: List[Dict[str, Any]]) -> List[Message]:
"""
Convert PraisonAI messages to AG-UI format.
Args:
messages: List of message dictionaries in PraisonAI format
Returns:
List of AG-UI Message objects
"""
if not messages:
return []
result: List[Message] = []
for msg in messages:
role = msg.get("role", "user")
content = msg.get("content", "")
if role == "tool":
result.append(Message(
role="tool",
content=content,
tool_call_id=msg.get("tool_call_id"),
))
elif role == "assistant":
tool_calls = None
if "tool_calls" in msg and msg["tool_calls"]:
tool_calls = [
ToolCall(
id=tc.get("id", ""),
function=FunctionCall(
name=tc.get("function", {}).get("name", ""),
arguments=tc.get("function", {}).get("arguments", "{}"),
),
)
for tc in msg["tool_calls"]
]
result.append(Message(
role="assistant",
content=content,
tool_calls=tool_calls,
))
elif role == "user":
result.append(Message(
role="user",
content=content,
))
elif role == "system":
result.append(Message(
role="system",
content=content,
))
return result
def validate_state(state: Any, thread_id: str) -> Optional[Dict[str, Any]]:
"""
Validate and convert state to a dictionary.
Args:
state: The state to validate (can be dict, Pydantic model, or other)
thread_id: Thread ID for logging
Returns:
State as a dictionary, or None if invalid
"""
if state is None:
return None
if isinstance(state, dict):
return state
if isinstance(state, BaseModel):
try:
return state.model_dump()
except Exception:
pass
# Try to_dict method
if hasattr(state, "to_dict") and callable(getattr(state, "to_dict")):
try:
result = state.to_dict()
if isinstance(result, dict):
return result
except Exception:
pass
logger.warning(
f"State must be a dict, got {type(state).__name__}. "
f"State will be ignored. Thread: {thread_id}"
)
return None
def extract_user_input(messages: List[Message]) -> str:
"""
Extract the last user message content.
Args:
messages: List of AG-UI messages
Returns:
Content of the last user message, or empty string
"""
if not messages:
return ""
for msg in reversed(messages):
if msg.role == "user" and msg.content:
return msg.content
return ""
@@ -0,0 +1,30 @@
"""
AG-UI Event Encoder
Encodes AG-UI events to Server-Sent Events (SSE) format.
"""
from praisonaiagents.ui.agui.types import BaseEvent
class EventEncoder:
"""Encodes AG-UI events to SSE format."""
def __init__(self):
pass
def get_content_type(self) -> str:
"""Returns the content type for SSE."""
return "text/event-stream"
def encode(self, event: BaseEvent) -> str:
"""
Encode an event to SSE format.
Args:
event: The event to encode
Returns:
SSE formatted string
"""
return f"data: {event.model_dump_json(by_alias=True, exclude_none=True)}\n\n"
@@ -0,0 +1,372 @@
"""
AG-UI Event Streaming
Provides utilities for streaming PraisonAI agent responses as AG-UI events.
"""
import uuid
from dataclasses import dataclass, field
from typing import Any, AsyncIterator, Dict, Iterator, List, Optional, Set
from praisonaiagents.ui.agui.types import (
BaseEvent,
TextMessageStartEvent,
TextMessageContentEvent,
TextMessageEndEvent,
ToolCallStartEvent,
ToolCallArgsEvent,
ToolCallEndEvent,
ToolCallResultEvent,
RunStartedEvent,
RunFinishedEvent,
RunErrorEvent,
StepStartedEvent,
StepFinishedEvent,
StateSnapshotEvent,
)
@dataclass
class EventBuffer:
"""Buffer to manage event ordering constraints."""
active_tool_call_ids: Set[str] = field(default_factory=set)
ended_tool_call_ids: Set[str] = field(default_factory=set)
current_text_message_id: str = ""
next_text_message_id: str = field(default_factory=lambda: str(uuid.uuid4()))
pending_tool_calls_parent_id: str = ""
def start_tool_call(self, tool_call_id: str) -> None:
"""Start a new tool call."""
self.active_tool_call_ids.add(tool_call_id)
def end_tool_call(self, tool_call_id: str) -> None:
"""End a tool call."""
self.active_tool_call_ids.discard(tool_call_id)
self.ended_tool_call_ids.add(tool_call_id)
def start_text_message(self) -> str:
"""Start a new text message and return its ID."""
self.current_text_message_id = self.next_text_message_id
self.next_text_message_id = str(uuid.uuid4())
return self.current_text_message_id
def get_parent_message_id_for_tool_call(self) -> str:
"""Get the message ID to use as parent for tool calls."""
if self.pending_tool_calls_parent_id:
return self.pending_tool_calls_parent_id
return self.current_text_message_id
# Event Creation Functions
def create_text_message_events(
content: str,
message_id: Optional[str] = None,
role: str = "assistant"
) -> Iterator[BaseEvent]:
"""
Create text message events for a complete message.
Args:
content: The message content
message_id: Optional message ID (generated if not provided)
role: Message role (default: assistant)
Yields:
TextMessageStartEvent, TextMessageContentEvent(s), TextMessageEndEvent
"""
msg_id = message_id or str(uuid.uuid4())
yield TextMessageStartEvent(message_id=msg_id, role=role)
if content:
yield TextMessageContentEvent(message_id=msg_id, delta=content)
yield TextMessageEndEvent(message_id=msg_id)
def stream_text_chunks(
chunks: Iterator[str],
message_id: Optional[str] = None,
role: str = "assistant"
) -> Iterator[BaseEvent]:
"""
Stream text chunks as AG-UI events.
Args:
chunks: Iterator of text chunks
message_id: Optional message ID
role: Message role
Yields:
AG-UI events for the text stream
"""
msg_id = message_id or str(uuid.uuid4())
yield TextMessageStartEvent(message_id=msg_id, role=role)
for chunk in chunks:
if chunk:
yield TextMessageContentEvent(message_id=msg_id, delta=chunk)
yield TextMessageEndEvent(message_id=msg_id)
def create_tool_call_events(
tool_call_id: str,
tool_name: str,
arguments: str,
parent_message_id: Optional[str] = None
) -> Iterator[BaseEvent]:
"""
Create tool call events.
Args:
tool_call_id: Unique ID for the tool call
tool_name: Name of the tool being called
arguments: JSON string of arguments
parent_message_id: Optional parent message ID
Yields:
ToolCallStartEvent, ToolCallArgsEvent, ToolCallEndEvent
"""
yield ToolCallStartEvent(
tool_call_id=tool_call_id,
tool_call_name=tool_name,
parent_message_id=parent_message_id
)
yield ToolCallArgsEvent(
tool_call_id=tool_call_id,
delta=arguments
)
yield ToolCallEndEvent(tool_call_id=tool_call_id)
def create_tool_result_event(
tool_call_id: str,
content: str,
message_id: Optional[str] = None
) -> ToolCallResultEvent:
"""
Create a tool result event.
Args:
tool_call_id: ID of the tool call this is a result for
content: Result content
message_id: Optional message ID
Returns:
ToolCallResultEvent
"""
return ToolCallResultEvent(
message_id=message_id or str(uuid.uuid4()),
tool_call_id=tool_call_id,
content=content
)
# Run Lifecycle Events
def create_run_started_event(thread_id: str, run_id: str) -> RunStartedEvent:
"""Create a run started event."""
return RunStartedEvent(thread_id=thread_id, run_id=run_id)
def create_run_finished_event(
thread_id: str,
run_id: str,
result: Optional[Any] = None
) -> RunFinishedEvent:
"""Create a run finished event."""
return RunFinishedEvent(thread_id=thread_id, run_id=run_id, result=result)
def create_run_error_event(
message: str,
code: Optional[str] = None
) -> RunErrorEvent:
"""Create a run error event."""
return RunErrorEvent(message=message, code=code)
# Step Events
def create_step_started_event(step_name: str) -> StepStartedEvent:
"""Create a step started event."""
return StepStartedEvent(step_name=step_name)
def create_step_finished_event(step_name: str) -> StepFinishedEvent:
"""Create a step finished event."""
return StepFinishedEvent(step_name=step_name)
# State Events
def create_state_snapshot_event(state: Dict[str, Any]) -> StateSnapshotEvent:
"""Create a state snapshot event."""
return StateSnapshotEvent(snapshot=state)
# Async Streaming
async def async_stream_response(
response_stream: AsyncIterator[str],
thread_id: str,
run_id: str
) -> AsyncIterator[BaseEvent]:
"""
Stream an async response as AG-UI events.
Args:
response_stream: Async iterator of response chunks
thread_id: Thread ID
run_id: Run ID
Yields:
AG-UI events
"""
# Emit run started
yield create_run_started_event(thread_id, run_id)
message_id = str(uuid.uuid4())
message_started = False
try:
async for chunk in response_stream:
if not message_started:
yield TextMessageStartEvent(message_id=message_id, role="assistant")
message_started = True
if chunk:
yield TextMessageContentEvent(message_id=message_id, delta=chunk)
if message_started:
yield TextMessageEndEvent(message_id=message_id)
# Emit run finished
yield create_run_finished_event(thread_id, run_id)
except Exception as e:
if message_started:
yield TextMessageEndEvent(message_id=message_id)
yield create_run_error_event(str(e))
async def async_stream_agent_response(
agent,
user_input: str,
thread_id: str,
run_id: str,
session_state: Optional[Dict[str, Any]] = None,
messages: Optional[List[Dict[str, Any]]] = None
) -> AsyncIterator[BaseEvent]:
"""
Stream an agent's response as AG-UI events.
Args:
agent: PraisonAI Agent instance
user_input: User input string
thread_id: Thread ID
run_id: Run ID
session_state: Optional session state
messages: Optional message history
Yields:
AG-UI events
"""
import asyncio
# Emit run started
yield create_run_started_event(thread_id, run_id)
message_id = str(uuid.uuid4())
try:
# Check if agent has async chat method
if hasattr(agent, 'achat'):
response = await agent.achat(user_input)
elif hasattr(agent, 'chat'):
# Run sync chat in executor
loop = asyncio.get_event_loop()
response = await loop.run_in_executor(None, agent.chat, user_input)
else:
raise ValueError("Agent must have 'chat' or 'achat' method")
# Emit text message events
yield TextMessageStartEvent(message_id=message_id, role="assistant")
if response:
yield TextMessageContentEvent(message_id=message_id, delta=str(response))
yield TextMessageEndEvent(message_id=message_id)
# Emit run finished
yield create_run_finished_event(thread_id, run_id)
except Exception as e:
yield create_run_error_event(str(e))
async def async_stream_agents_response(
agents,
user_input: str,
thread_id: str,
run_id: str,
session_state: Optional[Dict[str, Any]] = None
) -> AsyncIterator[BaseEvent]:
"""
Stream a PraisonAIAgents workflow response as AG-UI events.
Args:
agents: PraisonAIAgents instance
user_input: User input string
thread_id: Thread ID
run_id: Run ID
session_state: Optional session state
Yields:
AG-UI events
"""
import asyncio
# Emit run started
yield create_run_started_event(thread_id, run_id)
message_id = str(uuid.uuid4())
try:
# Check if agents has async start method
if hasattr(agents, 'astart'):
result = await agents.astart(user_input)
elif hasattr(agents, 'start'):
# Run sync start in executor
loop = asyncio.get_event_loop()
result = await loop.run_in_executor(None, agents.start, user_input)
else:
raise ValueError("Agents must have 'start' or 'astart' method")
# Emit text message events with result
yield TextMessageStartEvent(message_id=message_id, role="assistant")
if result:
# Handle different result types
if isinstance(result, dict):
content = result.get("output", str(result))
elif hasattr(result, "raw"):
content = result.raw
else:
content = str(result)
yield TextMessageContentEvent(message_id=message_id, delta=content)
yield TextMessageEndEvent(message_id=message_id)
# Emit run finished
yield create_run_finished_event(thread_id, run_id)
except Exception as e:
yield create_run_error_event(str(e))
@@ -0,0 +1,183 @@
"""
AG-UI Protocol Types
This module defines the event types and message types for the AG-UI protocol,
following the ag-ui-protocol specification.
"""
from enum import Enum
from typing import Any, Dict, List, Literal, Optional, Union
from pydantic import BaseModel, Field
class EventType(str, Enum):
"""AG-UI Event Types."""
TEXT_MESSAGE_START = "TEXT_MESSAGE_START"
TEXT_MESSAGE_CONTENT = "TEXT_MESSAGE_CONTENT"
TEXT_MESSAGE_END = "TEXT_MESSAGE_END"
TOOL_CALL_START = "TOOL_CALL_START"
TOOL_CALL_ARGS = "TOOL_CALL_ARGS"
TOOL_CALL_END = "TOOL_CALL_END"
TOOL_CALL_RESULT = "TOOL_CALL_RESULT"
STATE_SNAPSHOT = "STATE_SNAPSHOT"
STATE_DELTA = "STATE_DELTA"
RUN_STARTED = "RUN_STARTED"
RUN_FINISHED = "RUN_FINISHED"
RUN_ERROR = "RUN_ERROR"
STEP_STARTED = "STEP_STARTED"
STEP_FINISHED = "STEP_FINISHED"
CUSTOM = "CUSTOM"
class BaseEvent(BaseModel):
"""Base event for all AG-UI events."""
type: EventType
timestamp: Optional[int] = None
model_config = {"use_enum_values": True}
# Text Message Events
class TextMessageStartEvent(BaseEvent):
"""Event indicating the start of a text message."""
type: Literal["TEXT_MESSAGE_START"] = "TEXT_MESSAGE_START"
message_id: str
role: str = "assistant"
class TextMessageContentEvent(BaseEvent):
"""Event containing text message content."""
type: Literal["TEXT_MESSAGE_CONTENT"] = "TEXT_MESSAGE_CONTENT"
message_id: str
delta: str
class TextMessageEndEvent(BaseEvent):
"""Event indicating the end of a text message."""
type: Literal["TEXT_MESSAGE_END"] = "TEXT_MESSAGE_END"
message_id: str
# Tool Call Events
class ToolCallStartEvent(BaseEvent):
"""Event indicating the start of a tool call."""
type: Literal["TOOL_CALL_START"] = "TOOL_CALL_START"
tool_call_id: str
tool_call_name: str
parent_message_id: Optional[str] = None
class ToolCallArgsEvent(BaseEvent):
"""Event containing tool call arguments."""
type: Literal["TOOL_CALL_ARGS"] = "TOOL_CALL_ARGS"
tool_call_id: str
delta: str
class ToolCallEndEvent(BaseEvent):
"""Event indicating the end of a tool call."""
type: Literal["TOOL_CALL_END"] = "TOOL_CALL_END"
tool_call_id: str
class ToolCallResultEvent(BaseEvent):
"""Event containing tool call result."""
type: Literal["TOOL_CALL_RESULT"] = "TOOL_CALL_RESULT"
message_id: str
tool_call_id: str
content: str
role: Optional[Literal["tool"]] = "tool"
# Run Lifecycle Events
class RunStartedEvent(BaseEvent):
"""Event indicating a run has started."""
type: Literal["RUN_STARTED"] = "RUN_STARTED"
thread_id: str
run_id: str
parent_run_id: Optional[str] = None
class RunFinishedEvent(BaseEvent):
"""Event indicating a run has finished."""
type: Literal["RUN_FINISHED"] = "RUN_FINISHED"
thread_id: str
run_id: str
result: Optional[Any] = None
class RunErrorEvent(BaseEvent):
"""Event indicating a run error."""
type: Literal["RUN_ERROR"] = "RUN_ERROR"
message: str
code: Optional[str] = None
# Step Events
class StepStartedEvent(BaseEvent):
"""Event indicating a step has started."""
type: Literal["STEP_STARTED"] = "STEP_STARTED"
step_name: str
class StepFinishedEvent(BaseEvent):
"""Event indicating a step has finished."""
type: Literal["STEP_FINISHED"] = "STEP_FINISHED"
step_name: str
# State Events
class StateSnapshotEvent(BaseEvent):
"""Event containing a state snapshot."""
type: Literal["STATE_SNAPSHOT"] = "STATE_SNAPSHOT"
snapshot: Dict[str, Any]
# Message Types
class FunctionCall(BaseModel):
"""Function call in a tool call."""
name: str
arguments: str
class ToolCall(BaseModel):
"""Tool call in a message."""
id: str
function: FunctionCall
type: str = "function"
class Message(BaseModel):
"""AG-UI Message type."""
role: Literal["user", "assistant", "system", "tool"]
content: Optional[str] = None
tool_calls: Optional[List[ToolCall]] = None
tool_call_id: Optional[str] = None
name: Optional[str] = None
class RunAgentInput(BaseModel):
"""Input for running an agent."""
thread_id: str
run_id: Optional[str] = None
messages: Optional[List[Message]] = Field(default_factory=list)
state: Optional[Dict[str, Any]] = None
forwarded_props: Optional[Dict[str, Any]] = None
# Union type for all events
Event = Union[
TextMessageStartEvent,
TextMessageContentEvent,
TextMessageEndEvent,
ToolCallStartEvent,
ToolCallArgsEvent,
ToolCallEndEvent,
ToolCallResultEvent,
StateSnapshotEvent,
RunStartedEvent,
RunFinishedEvent,
RunErrorEvent,
StepStartedEvent,
StepFinishedEvent,
]
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "praisonaiagents"
version = "0.1.13"
version = "0.1.15"
description = "Praison AI agents for completing complex tasks with Self Reflection Agents"
readme = "README.md"
requires-python = ">=3.10"
@@ -0,0 +1 @@
"""Tests for AG-UI integration."""
@@ -0,0 +1,188 @@
"""
Test AGUI Class - TDD Tests for Main Interface Class
Phase 5: AGUI Class Tests
- Test AGUI initialization with Agent
- Test AGUI initialization with PraisonAIAgents
- Test get_router returns FastAPI router
- Test validation errors
"""
class TestAGUIInitialization:
"""Test AGUI class initialization."""
def test_agui_init_with_agent(self):
"""Test AGUI initialization with single Agent."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
assert agui.agent == agent
assert agui.agents is None
def test_agui_init_with_agents(self):
"""Test AGUI initialization with PraisonAIAgents."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent, Task, PraisonAIAgents
agent = Agent(name="Test", role="Tester", goal="Test things")
task = Task(description="Test task", expected_output="Result", agent=agent)
agents = PraisonAIAgents(agents=[agent], tasks=[task])
agui = AGUI(agents=agents)
assert agui.agents == agents
assert agui.agent is None
def test_agui_requires_agent_or_agents(self):
"""Test AGUI raises error if neither agent nor agents provided."""
from praisonaiagents.ui.agui import AGUI
import pytest
with pytest.raises(ValueError, match="requires an agent or agents"):
AGUI()
def test_agui_custom_prefix(self):
"""Test AGUI with custom prefix."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent, prefix="/api/v1")
assert agui.prefix == "/api/v1"
def test_agui_custom_tags(self):
"""Test AGUI with custom tags."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent, tags=["Custom", "Tags"])
assert agui.tags == ["Custom", "Tags"]
def test_agui_default_tags(self):
"""Test AGUI default tags."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
assert agui.tags == ["AGUI"]
class TestAGUIRouter:
"""Test AGUI router generation."""
def test_get_router_returns_api_router(self):
"""Test get_router returns FastAPI APIRouter."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import APIRouter
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
router = agui.get_router()
assert isinstance(router, APIRouter)
def test_router_has_agui_endpoint(self):
"""Test router has /agui POST endpoint."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
router = agui.get_router()
# Check routes
route_paths = [route.path for route in router.routes]
assert "/agui" in route_paths
def test_router_has_status_endpoint(self):
"""Test router has /status GET endpoint."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
router = agui.get_router()
# Check routes
route_paths = [route.path for route in router.routes]
assert "/status" in route_paths
def test_router_with_prefix(self):
"""Test router respects prefix."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent, prefix="/api/v1")
router = agui.get_router()
assert router.prefix == "/api/v1"
class TestAGUIIntegration:
"""Test AGUI integration with FastAPI."""
def test_include_router_in_fastapi(self):
"""Test including AGUI router in FastAPI app."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
# Check app has routes
route_paths = [route.path for route in app.routes]
assert "/agui" in route_paths
assert "/status" in route_paths
class TestAGUIName:
"""Test AGUI name and description."""
def test_agui_name(self):
"""Test AGUI name property."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent, name="My Agent")
assert agui.name == "My Agent"
def test_agui_description(self):
"""Test AGUI description property."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent, description="A test agent")
assert agui.description == "A test agent"
def test_agui_default_name_from_agent(self):
"""Test AGUI uses agent name by default."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="TestAgent", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
assert agui.name == "TestAgent"
@@ -0,0 +1,286 @@
"""
Test AG-UI Message Conversion - TDD Tests for Message Format Conversion
Phase 3: Message Conversion Tests
- Test converting AG-UI messages to PraisonAI format
- Test converting PraisonAI messages to AG-UI format
- Test handling of tool calls
- Test handling of tool results
- Test edge cases (empty messages, missing fields)
"""
class TestAGUIToPraisonAIConversion:
"""Test converting AG-UI messages to PraisonAI format."""
def test_convert_user_message(self):
"""Test converting a user message."""
from praisonaiagents.ui.agui.conversion import agui_messages_to_praisonai
from praisonaiagents.ui.agui.types import Message
agui_messages = [Message(role="user", content="Hello")]
result = agui_messages_to_praisonai(agui_messages)
assert len(result) == 1
assert result[0]["role"] == "user"
assert result[0]["content"] == "Hello"
def test_convert_assistant_message(self):
"""Test converting an assistant message."""
from praisonaiagents.ui.agui.conversion import agui_messages_to_praisonai
from praisonaiagents.ui.agui.types import Message
agui_messages = [Message(role="assistant", content="Hi there!")]
result = agui_messages_to_praisonai(agui_messages)
assert len(result) == 1
assert result[0]["role"] == "assistant"
assert result[0]["content"] == "Hi there!"
def test_convert_tool_message(self):
"""Test converting a tool result message."""
from praisonaiagents.ui.agui.conversion import agui_messages_to_praisonai
from praisonaiagents.ui.agui.types import Message
agui_messages = [Message(role="tool", content="Search results", tool_call_id="tc-123")]
result = agui_messages_to_praisonai(agui_messages)
assert len(result) == 1
assert result[0]["role"] == "tool"
assert result[0]["content"] == "Search results"
assert result[0]["tool_call_id"] == "tc-123"
def test_convert_assistant_with_tool_calls(self):
"""Test converting assistant message with tool calls."""
from praisonaiagents.ui.agui.conversion import agui_messages_to_praisonai
from praisonaiagents.ui.agui.types import Message, ToolCall, FunctionCall
agui_messages = [
Message(
role="assistant",
content="",
tool_calls=[
ToolCall(
id="tc-123",
function=FunctionCall(name="search", arguments='{"query": "test"}')
)
]
)
]
result = agui_messages_to_praisonai(agui_messages)
assert len(result) == 1
assert result[0]["role"] == "assistant"
assert "tool_calls" in result[0]
assert len(result[0]["tool_calls"]) == 1
assert result[0]["tool_calls"][0]["function"]["name"] == "search"
def test_convert_multiple_messages(self):
"""Test converting multiple messages."""
from praisonaiagents.ui.agui.conversion import agui_messages_to_praisonai
from praisonaiagents.ui.agui.types import Message
agui_messages = [
Message(role="user", content="Hello"),
Message(role="assistant", content="Hi!"),
Message(role="user", content="How are you?"),
]
result = agui_messages_to_praisonai(agui_messages)
assert len(result) == 3
assert result[0]["role"] == "user"
assert result[1]["role"] == "assistant"
assert result[2]["role"] == "user"
def test_convert_empty_messages(self):
"""Test converting empty message list."""
from praisonaiagents.ui.agui.conversion import agui_messages_to_praisonai
result = agui_messages_to_praisonai([])
assert result == []
def test_skip_system_messages(self):
"""Test that system messages are skipped (agent builds its own)."""
from praisonaiagents.ui.agui.conversion import agui_messages_to_praisonai
from praisonaiagents.ui.agui.types import Message
agui_messages = [
Message(role="system", content="You are helpful"),
Message(role="user", content="Hello"),
]
result = agui_messages_to_praisonai(agui_messages)
# System message should be skipped
assert len(result) == 1
assert result[0]["role"] == "user"
class TestPraisonAIToAGUIConversion:
"""Test converting PraisonAI messages to AG-UI format."""
def test_convert_user_message(self):
"""Test converting a user message."""
from praisonaiagents.ui.agui.conversion import praisonai_messages_to_agui
praison_messages = [{"role": "user", "content": "Hello"}]
result = praisonai_messages_to_agui(praison_messages)
assert len(result) == 1
assert result[0].role == "user"
assert result[0].content == "Hello"
def test_convert_assistant_message(self):
"""Test converting an assistant message."""
from praisonaiagents.ui.agui.conversion import praisonai_messages_to_agui
praison_messages = [{"role": "assistant", "content": "Hi there!"}]
result = praisonai_messages_to_agui(praison_messages)
assert len(result) == 1
assert result[0].role == "assistant"
assert result[0].content == "Hi there!"
def test_convert_tool_result(self):
"""Test converting a tool result message."""
from praisonaiagents.ui.agui.conversion import praisonai_messages_to_agui
praison_messages = [
{"role": "tool", "content": "Search results", "tool_call_id": "tc-123"}
]
result = praisonai_messages_to_agui(praison_messages)
assert len(result) == 1
assert result[0].role == "tool"
assert result[0].content == "Search results"
assert result[0].tool_call_id == "tc-123"
def test_convert_assistant_with_tool_calls(self):
"""Test converting assistant message with tool calls."""
from praisonaiagents.ui.agui.conversion import praisonai_messages_to_agui
praison_messages = [
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "tc-123",
"function": {
"name": "search",
"arguments": '{"query": "test"}'
}
}
]
}
]
result = praisonai_messages_to_agui(praison_messages)
assert len(result) == 1
assert result[0].role == "assistant"
assert result[0].tool_calls is not None
assert len(result[0].tool_calls) == 1
assert result[0].tool_calls[0].function.name == "search"
def test_convert_empty_messages(self):
"""Test converting empty message list."""
from praisonaiagents.ui.agui.conversion import praisonai_messages_to_agui
result = praisonai_messages_to_agui([])
assert result == []
class TestStateConversion:
"""Test state conversion utilities."""
def test_validate_dict_state(self):
"""Test validating dict state."""
from praisonaiagents.ui.agui.conversion import validate_state
state = {"key": "value"}
result = validate_state(state, "thread-123")
assert result == {"key": "value"}
def test_validate_none_state(self):
"""Test validating None state."""
from praisonaiagents.ui.agui.conversion import validate_state
result = validate_state(None, "thread-123")
assert result is None
def test_validate_pydantic_state(self):
"""Test validating Pydantic model state."""
from praisonaiagents.ui.agui.conversion import validate_state
from pydantic import BaseModel
class TestState(BaseModel):
key: str = "value"
state = TestState()
result = validate_state(state, "thread-123")
assert result == {"key": "value"}
def test_validate_invalid_state_returns_none(self):
"""Test that invalid state returns None."""
from praisonaiagents.ui.agui.conversion import validate_state
# A string is not a valid state
result = validate_state("invalid", "thread-123")
assert result is None
class TestToolCallDeduplication:
"""Test tool call deduplication logic."""
def test_deduplicate_tool_results(self):
"""Test that duplicate tool results are deduplicated."""
from praisonaiagents.ui.agui.conversion import agui_messages_to_praisonai
from praisonaiagents.ui.agui.types import Message
# Same tool_call_id appearing twice
agui_messages = [
Message(role="tool", content="Result 1", tool_call_id="tc-123"),
Message(role="tool", content="Result 2", tool_call_id="tc-123"), # Duplicate
]
result = agui_messages_to_praisonai(agui_messages)
# Should only have one result
assert len(result) == 1
assert result[0]["content"] == "Result 1" # First one wins
class TestExtractUserInput:
"""Test extracting user input from messages."""
def test_extract_last_user_message(self):
"""Test extracting the last user message."""
from praisonaiagents.ui.agui.conversion import extract_user_input
from praisonaiagents.ui.agui.types import Message
messages = [
Message(role="user", content="First message"),
Message(role="assistant", content="Response"),
Message(role="user", content="Last message"),
]
result = extract_user_input(messages)
assert result == "Last message"
def test_extract_from_empty_messages(self):
"""Test extracting from empty messages."""
from praisonaiagents.ui.agui.conversion import extract_user_input
result = extract_user_input([])
assert result == ""
def test_extract_no_user_messages(self):
"""Test extracting when no user messages exist."""
from praisonaiagents.ui.agui.conversion import extract_user_input
from praisonaiagents.ui.agui.types import Message
messages = [
Message(role="assistant", content="Hello"),
]
result = extract_user_input(messages)
assert result == ""
@@ -0,0 +1,311 @@
"""
Test AG-UI Integration - TDD Tests for End-to-End Integration
Phase 7: Integration Tests
- Test full flow with single Agent
- Test full flow with PraisonAIAgents
- Test tool calling flow
- Test error recovery
- Test state persistence
"""
class TestSingleAgentIntegration:
"""Test integration with single Agent."""
def test_single_agent_chat_flow(self):
"""Test complete chat flow with single agent."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
import json
agent = Agent(
name="Assistant",
role="Helpful Assistant",
goal="Help users with their questions",
llm="gpt-4o-mini"
)
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
# Send a message
response = client.post("/agui", json={
"thread_id": "test-thread",
"messages": [{"role": "user", "content": "Say hello"}]
})
assert response.status_code == 200
# Parse SSE events
events = []
for line in response.text.split("\n"):
if line.startswith("data: "):
event_data = json.loads(line[6:])
events.append(event_data)
# Should have run started event
event_types = [e.get("type") for e in events]
assert "RUN_STARTED" in event_types
assert "RUN_FINISHED" in event_types or "RUN_ERROR" in event_types
class TestMultiAgentIntegration:
"""Test integration with PraisonAIAgents."""
def test_multi_agent_workflow_flow(self):
"""Test complete workflow with multiple agents."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent, Task, PraisonAIAgents
from fastapi import FastAPI
import json
# Create agents
researcher = Agent(
name="Researcher",
role="Research Analyst",
goal="Research topics thoroughly",
llm="gpt-4o-mini"
)
writer = Agent(
name="Writer",
role="Content Writer",
goal="Write engaging content",
llm="gpt-4o-mini"
)
# Create tasks
research_task = Task(
description="Research the topic: {topic}",
expected_output="Research findings",
agent=researcher
)
write_task = Task(
description="Write about the research",
expected_output="Written article",
agent=writer
)
# Create PraisonAIAgents
agents = PraisonAIAgents(
agents=[researcher, writer],
tasks=[research_task, write_task]
)
agui = AGUI(agents=agents)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "test-thread",
"messages": [{"role": "user", "content": "Research AI"}]
})
assert response.status_code == 200
class TestToolCallingIntegration:
"""Test integration with tool calling."""
def test_agent_with_tools(self):
"""Test agent with tools integration."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
import json
def search_tool(query: str) -> str:
"""Search for information."""
return f"Search results for: {query}"
agent = Agent(
name="SearchAgent",
role="Search Assistant",
goal="Help users search for information",
tools=[search_tool],
llm="gpt-4o-mini"
)
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "test-thread",
"messages": [{"role": "user", "content": "Search for Python tutorials"}]
})
assert response.status_code == 200
# Parse events
events = []
for line in response.text.split("\n"):
if line.startswith("data: "):
event_data = json.loads(line[6:])
events.append(event_data)
# Check for tool call events (may or may not have them depending on LLM response)
event_types = [e.get("type") for e in events]
assert "RUN_STARTED" in event_types
class TestStateManagement:
"""Test state management in AG-UI."""
def test_state_passed_to_agent(self):
"""Test that state is passed to agent."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(
name="StatefulAgent",
role="Stateful Assistant",
goal="Remember context",
llm="gpt-4o-mini"
)
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "test-thread",
"messages": [{"role": "user", "content": "Hello"}],
"state": {"user_name": "John", "preferences": {"language": "en"}}
})
assert response.status_code == 200
class TestConversationHistory:
"""Test conversation history handling."""
def test_multiple_messages_in_history(self):
"""Test handling multiple messages in conversation history."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(
name="ChatAgent",
role="Chat Assistant",
goal="Have conversations",
llm="gpt-4o-mini"
)
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "test-thread",
"messages": [
{"role": "user", "content": "My name is John"},
{"role": "assistant", "content": "Hello John!"},
{"role": "user", "content": "What is my name?"}
]
})
assert response.status_code == 200
class TestErrorRecovery:
"""Test error recovery scenarios."""
def test_graceful_error_handling(self):
"""Test graceful error handling."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
import json
# Agent with invalid LLM should handle error gracefully
agent = Agent(
name="ErrorAgent",
role="Error Test",
goal="Test error handling",
llm="invalid-model-name"
)
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "test-thread",
"messages": [{"role": "user", "content": "Hello"}]
})
# Should not crash, should return error event
assert response.status_code == 200
# Parse events
events = []
for line in response.text.split("\n"):
if line.startswith("data: "):
try:
event_data = json.loads(line[6:])
events.append(event_data)
except json.JSONDecodeError:
pass
# Should have error event
event_types = [e.get("type") for e in events]
assert "RUN_ERROR" in event_types or "RUN_FINISHED" in event_types
class TestPerformance:
"""Test performance - ensure no impact on existing functionality."""
def test_agui_import_does_not_slow_package(self):
"""Test that importing AGUI doesn't slow down package import."""
import time
start = time.time()
from praisonaiagents.ui.agui import AGUI
import_time = time.time() - start
# Import should be fast (< 1 second)
assert import_time < 1.0
def test_agui_lazy_loading(self):
"""Test that AGUI uses lazy loading where possible."""
# This test ensures we don't import heavy dependencies until needed
import sys
# Remove agui from cache if present
modules_to_remove = [k for k in sys.modules.keys() if 'agui' in k]
for mod in modules_to_remove:
del sys.modules[mod]
# Import should not load FastAPI until get_router is called
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test")
agui = AGUI(agent=agent)
# FastAPI should be loaded when get_router is called
router = agui.get_router()
assert router is not None
@@ -0,0 +1,266 @@
"""
Test AG-UI Router - TDD Tests for FastAPI HTTP Endpoints
Phase 6: FastAPI Router Tests
- Test /agui POST endpoint
- Test /status GET endpoint
- Test request validation
- Test streaming response
- Test error handling
"""
class TestStatusEndpoint:
"""Test /status endpoint."""
def test_status_endpoint_returns_available(self):
"""Test status endpoint returns available status."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.get("/status")
assert response.status_code == 200
assert response.json() == {"status": "available"}
class TestAGUIEndpoint:
"""Test /agui POST endpoint."""
def test_agui_endpoint_accepts_post(self):
"""Test /agui endpoint accepts POST requests."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
# Minimal valid request
response = client.post("/agui", json={
"thread_id": "thread-123",
"messages": [{"role": "user", "content": "Hello"}]
})
# Should return streaming response (200)
assert response.status_code == 200
def test_agui_endpoint_returns_streaming_response(self):
"""Test /agui endpoint returns streaming response."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "thread-123",
"messages": [{"role": "user", "content": "Hello"}]
})
# Check content type is event-stream
assert "text/event-stream" in response.headers.get("content-type", "")
def test_agui_endpoint_requires_thread_id(self):
"""Test /agui endpoint requires thread_id."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
# Missing thread_id
response = client.post("/agui", json={
"messages": [{"role": "user", "content": "Hello"}]
})
# Should return validation error
assert response.status_code == 422
class TestAGUIEndpointWithState:
"""Test /agui endpoint with state."""
def test_agui_endpoint_accepts_state(self):
"""Test /agui endpoint accepts state parameter."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "thread-123",
"messages": [{"role": "user", "content": "Hello"}],
"state": {"custom_key": "custom_value"}
})
assert response.status_code == 200
class TestAGUIEndpointWithRunId:
"""Test /agui endpoint with run_id."""
def test_agui_endpoint_accepts_run_id(self):
"""Test /agui endpoint accepts run_id parameter."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "thread-123",
"run_id": "run-456",
"messages": [{"role": "user", "content": "Hello"}]
})
assert response.status_code == 200
def test_agui_endpoint_generates_run_id_if_missing(self):
"""Test /agui endpoint generates run_id if not provided."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "thread-123",
"messages": [{"role": "user", "content": "Hello"}]
})
# Should succeed even without run_id
assert response.status_code == 200
class TestAGUIEndpointCORS:
"""Test /agui endpoint CORS headers."""
def test_agui_endpoint_cors_headers(self):
"""Test /agui endpoint includes CORS headers."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "thread-123",
"messages": [{"role": "user", "content": "Hello"}]
})
# Check CORS headers
assert response.headers.get("access-control-allow-origin") == "*"
class TestAGUIEndpointErrorHandling:
"""Test /agui endpoint error handling."""
def test_agui_endpoint_handles_agent_error(self):
"""Test /agui endpoint handles agent errors gracefully."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
# Create agent that will fail
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
# This should not crash the server
response = client.post("/agui", json={
"thread_id": "thread-123",
"messages": [{"role": "user", "content": "Hello"}]
})
# Should return 200 with error event in stream
assert response.status_code == 200
class TestAGUIEndpointWithForwardedProps:
"""Test /agui endpoint with forwarded_props."""
def test_agui_endpoint_accepts_forwarded_props(self):
"""Test /agui endpoint accepts forwarded_props."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "thread-123",
"messages": [{"role": "user", "content": "Hello"}],
"forwarded_props": {"user_id": "user-123"}
})
assert response.status_code == 200
@@ -0,0 +1,240 @@
"""
Test AG-UI Event Streaming - TDD Tests for Streaming PraisonAI Responses
Phase 4: Event Streaming Tests
- Test streaming text content as AG-UI events
- Test streaming tool calls as AG-UI events
- Test run lifecycle events (start, finish, error)
- Test step events
- Test event ordering
"""
import asyncio
class TestTextMessageStreaming:
"""Test streaming text messages."""
def test_stream_simple_text(self):
"""Test streaming simple text content."""
from praisonaiagents.ui.agui.streaming import create_text_message_events
events = list(create_text_message_events(
content="Hello, world!",
message_id="msg-123"
))
# Should have start, content, end events
assert len(events) == 3
assert events[0].type == "TEXT_MESSAGE_START"
assert events[0].message_id == "msg-123"
assert events[1].type == "TEXT_MESSAGE_CONTENT"
assert events[1].delta == "Hello, world!"
assert events[2].type == "TEXT_MESSAGE_END"
def test_stream_chunked_text(self):
"""Test streaming text in chunks."""
from praisonaiagents.ui.agui.streaming import stream_text_chunks
chunks = ["Hello", ", ", "world", "!"]
events = list(stream_text_chunks(chunks, message_id="msg-123"))
# Should have start, 4 content events, end
assert len(events) == 6
assert events[0].type == "TEXT_MESSAGE_START"
assert events[1].delta == "Hello"
assert events[2].delta == ", "
assert events[3].delta == "world"
assert events[4].delta == "!"
assert events[5].type == "TEXT_MESSAGE_END"
def test_stream_empty_text(self):
"""Test streaming empty text."""
from praisonaiagents.ui.agui.streaming import create_text_message_events
events = list(create_text_message_events(content="", message_id="msg-123"))
# Should still have start and end, but no content
assert len(events) == 2
assert events[0].type == "TEXT_MESSAGE_START"
assert events[1].type == "TEXT_MESSAGE_END"
class TestToolCallStreaming:
"""Test streaming tool calls."""
def test_stream_tool_call(self):
"""Test streaming a tool call."""
from praisonaiagents.ui.agui.streaming import create_tool_call_events
events = list(create_tool_call_events(
tool_call_id="tc-123",
tool_name="search",
arguments='{"query": "test"}',
parent_message_id="msg-123"
))
# Should have start, args, end events
assert len(events) == 3
assert events[0].type == "TOOL_CALL_START"
assert events[0].tool_call_id == "tc-123"
assert events[0].tool_call_name == "search"
assert events[1].type == "TOOL_CALL_ARGS"
assert events[1].delta == '{"query": "test"}'
assert events[2].type == "TOOL_CALL_END"
def test_stream_tool_result(self):
"""Test streaming a tool result."""
from praisonaiagents.ui.agui.streaming import create_tool_result_event
event = create_tool_result_event(
tool_call_id="tc-123",
content="Search results...",
message_id="msg-456"
)
assert event.type == "TOOL_CALL_RESULT"
assert event.tool_call_id == "tc-123"
assert event.content == "Search results..."
class TestRunLifecycleEvents:
"""Test run lifecycle events."""
def test_create_run_started_event(self):
"""Test creating run started event."""
from praisonaiagents.ui.agui.streaming import create_run_started_event
event = create_run_started_event(thread_id="thread-123", run_id="run-456")
assert event.type == "RUN_STARTED"
assert event.thread_id == "thread-123"
assert event.run_id == "run-456"
def test_create_run_finished_event(self):
"""Test creating run finished event."""
from praisonaiagents.ui.agui.streaming import create_run_finished_event
event = create_run_finished_event(thread_id="thread-123", run_id="run-456")
assert event.type == "RUN_FINISHED"
assert event.thread_id == "thread-123"
assert event.run_id == "run-456"
def test_create_run_error_event(self):
"""Test creating run error event."""
from praisonaiagents.ui.agui.streaming import create_run_error_event
event = create_run_error_event(message="Something went wrong", code="ERR001")
assert event.type == "RUN_ERROR"
assert event.message == "Something went wrong"
assert event.code == "ERR001"
class TestStepEvents:
"""Test step events."""
def test_create_step_started_event(self):
"""Test creating step started event."""
from praisonaiagents.ui.agui.streaming import create_step_started_event
event = create_step_started_event(step_name="research")
assert event.type == "STEP_STARTED"
assert event.step_name == "research"
def test_create_step_finished_event(self):
"""Test creating step finished event."""
from praisonaiagents.ui.agui.streaming import create_step_finished_event
event = create_step_finished_event(step_name="research")
assert event.type == "STEP_FINISHED"
assert event.step_name == "research"
class TestEventBuffer:
"""Test EventBuffer for managing event ordering."""
def test_event_buffer_creation(self):
"""Test EventBuffer creation."""
from praisonaiagents.ui.agui.streaming import EventBuffer
buffer = EventBuffer()
assert buffer.active_tool_call_ids == set()
assert buffer.current_text_message_id == ""
def test_event_buffer_start_tool_call(self):
"""Test starting a tool call in buffer."""
from praisonaiagents.ui.agui.streaming import EventBuffer
buffer = EventBuffer()
buffer.start_tool_call("tc-123")
assert "tc-123" in buffer.active_tool_call_ids
def test_event_buffer_end_tool_call(self):
"""Test ending a tool call in buffer."""
from praisonaiagents.ui.agui.streaming import EventBuffer
buffer = EventBuffer()
buffer.start_tool_call("tc-123")
buffer.end_tool_call("tc-123")
assert "tc-123" not in buffer.active_tool_call_ids
assert "tc-123" in buffer.ended_tool_call_ids
def test_event_buffer_start_text_message(self):
"""Test starting a text message in buffer."""
from praisonaiagents.ui.agui.streaming import EventBuffer
buffer = EventBuffer()
msg_id = buffer.start_text_message()
assert msg_id != ""
assert buffer.current_text_message_id == msg_id
class TestAsyncStreaming:
"""Test async streaming functionality."""
def test_async_stream_response(self):
"""Test async streaming of agent response."""
async def run_test():
from praisonaiagents.ui.agui.streaming import async_stream_response
# Mock response chunks
async def mock_chunks():
yield "Hello"
yield ", "
yield "world!"
events = []
async for event in async_stream_response(
response_stream=mock_chunks(),
thread_id="thread-123",
run_id="run-456"
):
events.append(event)
# Should have run_started, text events, run_finished
assert len(events) > 0
assert events[0].type == "RUN_STARTED"
assert events[-1].type == "RUN_FINISHED"
asyncio.run(run_test())
class TestStateSnapshotEvents:
"""Test state snapshot events."""
def test_create_state_snapshot_event(self):
"""Test creating state snapshot event."""
from praisonaiagents.ui.agui.streaming import create_state_snapshot_event
state = {"task": "research", "progress": 50}
event = create_state_snapshot_event(state)
assert event.type == "STATE_SNAPSHOT"
assert event.snapshot == state
@@ -0,0 +1,327 @@
"""
Test AG-UI Types - TDD Tests for Event and Message Types
Phase 2: Core Types Tests
- Test EventType enum values
- Test BaseEvent creation
- Test all event type classes
- Test message type classes
- Test RunAgentInput validation
"""
import json
class TestEventType:
"""Test EventType enum."""
def test_event_type_text_message_start(self):
"""Test TEXT_MESSAGE_START event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.TEXT_MESSAGE_START == "TEXT_MESSAGE_START"
def test_event_type_text_message_content(self):
"""Test TEXT_MESSAGE_CONTENT event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.TEXT_MESSAGE_CONTENT == "TEXT_MESSAGE_CONTENT"
def test_event_type_text_message_end(self):
"""Test TEXT_MESSAGE_END event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.TEXT_MESSAGE_END == "TEXT_MESSAGE_END"
def test_event_type_tool_call_start(self):
"""Test TOOL_CALL_START event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.TOOL_CALL_START == "TOOL_CALL_START"
def test_event_type_tool_call_args(self):
"""Test TOOL_CALL_ARGS event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.TOOL_CALL_ARGS == "TOOL_CALL_ARGS"
def test_event_type_tool_call_end(self):
"""Test TOOL_CALL_END event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.TOOL_CALL_END == "TOOL_CALL_END"
def test_event_type_tool_call_result(self):
"""Test TOOL_CALL_RESULT event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.TOOL_CALL_RESULT == "TOOL_CALL_RESULT"
def test_event_type_run_started(self):
"""Test RUN_STARTED event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.RUN_STARTED == "RUN_STARTED"
def test_event_type_run_finished(self):
"""Test RUN_FINISHED event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.RUN_FINISHED == "RUN_FINISHED"
def test_event_type_run_error(self):
"""Test RUN_ERROR event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.RUN_ERROR == "RUN_ERROR"
def test_event_type_step_started(self):
"""Test STEP_STARTED event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.STEP_STARTED == "STEP_STARTED"
def test_event_type_step_finished(self):
"""Test STEP_FINISHED event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.STEP_FINISHED == "STEP_FINISHED"
def test_event_type_state_snapshot(self):
"""Test STATE_SNAPSHOT event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.STATE_SNAPSHOT == "STATE_SNAPSHOT"
class TestBaseEvent:
"""Test BaseEvent class."""
def test_base_event_creation(self):
"""Test BaseEvent can be created."""
from praisonaiagents.ui.agui.types import BaseEvent, EventType
event = BaseEvent(type=EventType.RUN_STARTED)
assert event.type == EventType.RUN_STARTED
def test_base_event_timestamp_optional(self):
"""Test BaseEvent timestamp is optional."""
from praisonaiagents.ui.agui.types import BaseEvent, EventType
event = BaseEvent(type=EventType.RUN_STARTED)
assert event.timestamp is None
def test_base_event_to_dict(self):
"""Test BaseEvent can be converted to dict."""
from praisonaiagents.ui.agui.types import BaseEvent, EventType
event = BaseEvent(type=EventType.RUN_STARTED)
data = event.model_dump()
assert data["type"] == "RUN_STARTED"
class TestTextMessageEvents:
"""Test text message event classes."""
def test_text_message_start_event(self):
"""Test TextMessageStartEvent creation."""
from praisonaiagents.ui.agui.types import TextMessageStartEvent
event = TextMessageStartEvent(message_id="msg-123", role="assistant")
assert event.message_id == "msg-123"
assert event.role == "assistant"
assert event.type == "TEXT_MESSAGE_START"
def test_text_message_content_event(self):
"""Test TextMessageContentEvent creation."""
from praisonaiagents.ui.agui.types import TextMessageContentEvent
event = TextMessageContentEvent(message_id="msg-123", delta="Hello")
assert event.message_id == "msg-123"
assert event.delta == "Hello"
assert event.type == "TEXT_MESSAGE_CONTENT"
def test_text_message_end_event(self):
"""Test TextMessageEndEvent creation."""
from praisonaiagents.ui.agui.types import TextMessageEndEvent
event = TextMessageEndEvent(message_id="msg-123")
assert event.message_id == "msg-123"
assert event.type == "TEXT_MESSAGE_END"
class TestToolCallEvents:
"""Test tool call event classes."""
def test_tool_call_start_event(self):
"""Test ToolCallStartEvent creation."""
from praisonaiagents.ui.agui.types import ToolCallStartEvent
event = ToolCallStartEvent(
tool_call_id="tc-123",
tool_call_name="search",
parent_message_id="msg-123"
)
assert event.tool_call_id == "tc-123"
assert event.tool_call_name == "search"
assert event.parent_message_id == "msg-123"
assert event.type == "TOOL_CALL_START"
def test_tool_call_args_event(self):
"""Test ToolCallArgsEvent creation."""
from praisonaiagents.ui.agui.types import ToolCallArgsEvent
event = ToolCallArgsEvent(tool_call_id="tc-123", delta='{"query": "test"}')
assert event.tool_call_id == "tc-123"
assert event.delta == '{"query": "test"}'
assert event.type == "TOOL_CALL_ARGS"
def test_tool_call_end_event(self):
"""Test ToolCallEndEvent creation."""
from praisonaiagents.ui.agui.types import ToolCallEndEvent
event = ToolCallEndEvent(tool_call_id="tc-123")
assert event.tool_call_id == "tc-123"
assert event.type == "TOOL_CALL_END"
def test_tool_call_result_event(self):
"""Test ToolCallResultEvent creation."""
from praisonaiagents.ui.agui.types import ToolCallResultEvent
event = ToolCallResultEvent(
message_id="msg-456",
tool_call_id="tc-123",
content="Search results..."
)
assert event.message_id == "msg-456"
assert event.tool_call_id == "tc-123"
assert event.content == "Search results..."
assert event.type == "TOOL_CALL_RESULT"
class TestRunEvents:
"""Test run lifecycle event classes."""
def test_run_started_event(self):
"""Test RunStartedEvent creation."""
from praisonaiagents.ui.agui.types import RunStartedEvent
event = RunStartedEvent(thread_id="thread-123", run_id="run-456")
assert event.thread_id == "thread-123"
assert event.run_id == "run-456"
assert event.type == "RUN_STARTED"
def test_run_finished_event(self):
"""Test RunFinishedEvent creation."""
from praisonaiagents.ui.agui.types import RunFinishedEvent
event = RunFinishedEvent(thread_id="thread-123", run_id="run-456")
assert event.thread_id == "thread-123"
assert event.run_id == "run-456"
assert event.type == "RUN_FINISHED"
def test_run_error_event(self):
"""Test RunErrorEvent creation."""
from praisonaiagents.ui.agui.types import RunErrorEvent
event = RunErrorEvent(message="Something went wrong")
assert event.message == "Something went wrong"
assert event.type == "RUN_ERROR"
class TestStepEvents:
"""Test step event classes."""
def test_step_started_event(self):
"""Test StepStartedEvent creation."""
from praisonaiagents.ui.agui.types import StepStartedEvent
event = StepStartedEvent(step_name="research")
assert event.step_name == "research"
assert event.type == "STEP_STARTED"
def test_step_finished_event(self):
"""Test StepFinishedEvent creation."""
from praisonaiagents.ui.agui.types import StepFinishedEvent
event = StepFinishedEvent(step_name="research")
assert event.step_name == "research"
assert event.type == "STEP_FINISHED"
class TestStateEvents:
"""Test state event classes."""
def test_state_snapshot_event(self):
"""Test StateSnapshotEvent creation."""
from praisonaiagents.ui.agui.types import StateSnapshotEvent
event = StateSnapshotEvent(snapshot={"key": "value"})
assert event.snapshot == {"key": "value"}
assert event.type == "STATE_SNAPSHOT"
class TestMessageTypes:
"""Test message type classes."""
def test_user_message(self):
"""Test UserMessage creation."""
from praisonaiagents.ui.agui.types import Message
msg = Message(role="user", content="Hello")
assert msg.role == "user"
assert msg.content == "Hello"
def test_assistant_message(self):
"""Test AssistantMessage creation."""
from praisonaiagents.ui.agui.types import Message
msg = Message(role="assistant", content="Hi there!")
assert msg.role == "assistant"
assert msg.content == "Hi there!"
def test_tool_message(self):
"""Test ToolMessage creation."""
from praisonaiagents.ui.agui.types import Message
msg = Message(role="tool", content="Result", tool_call_id="tc-123")
assert msg.role == "tool"
assert msg.content == "Result"
assert msg.tool_call_id == "tc-123"
class TestToolCall:
"""Test ToolCall type."""
def test_tool_call_creation(self):
"""Test ToolCall creation."""
from praisonaiagents.ui.agui.types import ToolCall, FunctionCall
tc = ToolCall(
id="tc-123",
function=FunctionCall(name="search", arguments='{"query": "test"}')
)
assert tc.id == "tc-123"
assert tc.function.name == "search"
assert tc.function.arguments == '{"query": "test"}'
class TestRunAgentInput:
"""Test RunAgentInput type."""
def test_run_agent_input_creation(self):
"""Test RunAgentInput creation."""
from praisonaiagents.ui.agui.types import RunAgentInput, Message
input_data = RunAgentInput(
thread_id="thread-123",
run_id="run-456",
messages=[Message(role="user", content="Hello")]
)
assert input_data.thread_id == "thread-123"
assert input_data.run_id == "run-456"
assert len(input_data.messages) == 1
def test_run_agent_input_optional_fields(self):
"""Test RunAgentInput optional fields."""
from praisonaiagents.ui.agui.types import RunAgentInput
input_data = RunAgentInput(thread_id="thread-123")
assert input_data.thread_id == "thread-123"
assert input_data.run_id is None
assert input_data.messages is None or len(input_data.messages) == 0
assert input_data.state is None
class TestEventSerialization:
"""Test event serialization to JSON."""
def test_event_json_serialization(self):
"""Test events can be serialized to JSON."""
from praisonaiagents.ui.agui.types import TextMessageStartEvent
event = TextMessageStartEvent(message_id="msg-123", role="assistant")
json_str = event.model_dump_json()
data = json.loads(json_str)
assert data["type"] == "TEXT_MESSAGE_START"
assert data["message_id"] == "msg-123"
def test_event_sse_format(self):
"""Test events can be formatted as SSE."""
from praisonaiagents.ui.agui.types import TextMessageStartEvent
from praisonaiagents.ui.agui.encoder import EventEncoder
encoder = EventEncoder()
event = TextMessageStartEvent(message_id="msg-123", role="assistant")
sse = encoder.encode(event)
assert sse.startswith("data: ")
assert sse.endswith("\n\n")
# Parse the JSON part
json_part = sse[6:-2] # Remove "data: " prefix and "\n\n" suffix
data = json.loads(json_part)
assert data["type"] == "TEXT_MESSAGE_START"
+1 -1
View File
@@ -2422,7 +2422,7 @@ wheels = [
[[package]]
name = "praisonaiagents"
version = "0.1.13"
version = "0.1.15"
source = { editable = "." }
dependencies = [
{ name = "aiohttp" },
+57
View File
@@ -34,6 +34,7 @@ PraisonAI is a production-ready Multi-AI Agents framework with self-reflection,
## 📑 Table of Contents
- [⚡ Performance](#-performance)
- [🚀 Quick Start](#-quick-start)
- [📦 Installation](#-installation)
- [Python SDK](#python-sdk)
@@ -73,6 +74,32 @@ PraisonAI is a production-ready Multi-AI Agents framework with self-reflection,
---
## ⚡ Performance
PraisonAI Agents is the **fastest AI agent framework** for agent instantiation.
| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
| **PraisonAI** | **3.77** | **1.00x (fastest)** |
| OpenAI Agents SDK | 5.26 | 1.39x |
| Agno | 5.64 | 1.49x |
| PraisonAI (LiteLLM) | 7.56 | 2.00x |
| PydanticAI | 226.94 | 60.16x |
| LangGraph | 4,558.71 | 1,209x |
| CrewAI | 15,607.92 | 4,138x |
<details>
<summary>Run benchmarks yourself</summary>
```bash
cd praisonai-agents
python benchmarks/simple_benchmark.py
```
</details>
---
## 🚀 Quick Start
Get started with PraisonAI in under 1 minute:
@@ -1392,6 +1419,36 @@ praisonai commit
praisonai commit --push
```
### Serve CLI (API Server):
```bash
# Start API server for agents defined in YAML
praisonai serve agents.yaml
# With custom port and host
praisonai serve agents.yaml --port 8005 --host 0.0.0.0
# Alternative flag style
praisonai agents.yaml --serve
# The server provides:
# POST /agents - Run all agents sequentially
# POST /agents/{name} - Run specific agent (e.g., /agents/researcher)
# GET /agents/list - List available agents
```
### n8n Integration CLI:
```bash
# Export workflow to n8n and open in browser
praisonai agents.yaml --n8n
# With custom n8n URL
praisonai agents.yaml --n8n --n8n-url http://localhost:5678
# Set N8N_API_KEY for auto-import
export N8N_API_KEY="your-api-key"
praisonai agents.yaml --n8n
```
### @Mentions in Prompts:
```bash
# Include file content in prompt
+49 -22
View File
@@ -4,60 +4,87 @@
"nodes": [
{
"id": "trigger",
"name": "Manual Trigger",
"type": "n8n-nodes-base.manualTrigger",
"typeVersion": 1,
"name": "Webhook",
"type": "n8n-nodes-base.webhook",
"typeVersion": 2,
"position": [
250,
300
],
"parameters": {}
"webhookId": "dd7cf5cb72d19eae",
"parameters": {
"path": "create-movie-script-about-cat-in-mars",
"httpMethod": "POST",
"responseMode": "lastNode",
"options": {}
}
},
{
"id": "agent_0",
"name": "Researcher",
"type": "n8n-nodes-base.executeCommand",
"typeVersion": 1,
"type": "n8n-nodes-base.httpRequest",
"typeVersion": 4.2,
"position": [
450,
300
],
"parameters": {
"command": "praisonai \"Research about Mars environment and cat behavior for the movie concept\"",
"executeOnce": true
}
"method": "POST",
"url": "http://127.0.0.1:8005/agents/researcher",
"sendBody": true,
"specifyBody": "json",
"jsonBody": "={{ JSON.stringify({ query: $json.body?.query || $json.query || 'Research about Mars environment and cat behavior for the movie concept' }) }}",
"options": {
"timeout": 300000
}
},
"notes": "Agent: Researcher\nTask: Research about Mars environment and cat behavior for the movie concept"
},
{
"id": "agent_1",
"name": "Narrative Designer",
"type": "n8n-nodes-base.executeCommand",
"typeVersion": 1,
"type": "n8n-nodes-base.httpRequest",
"typeVersion": 4.2,
"position": [
650,
700,
300
],
"parameters": {
"command": "praisonai \"Create story concept based on research: {{previous_output}}\"",
"executeOnce": true
}
"method": "POST",
"url": "http://127.0.0.1:8005/agents/narrative_designer",
"sendBody": true,
"specifyBody": "json",
"jsonBody": "={{ JSON.stringify({ query: $json.response || 'Create story concept based on research: previous_output' }) }}",
"options": {
"timeout": 300000
}
},
"notes": "Agent: Narrative Designer\nTask: Create story concept based on research: {{previous_output}}"
},
{
"id": "agent_2",
"name": "Scriptwriter",
"type": "n8n-nodes-base.executeCommand",
"typeVersion": 1,
"type": "n8n-nodes-base.httpRequest",
"typeVersion": 4.2,
"position": [
850,
950,
300
],
"parameters": {
"command": "praisonai \"Write the final movie script based on the story concept: {{previous_output}}\"",
"executeOnce": true
}
"method": "POST",
"url": "http://127.0.0.1:8005/agents/scriptwriter",
"sendBody": true,
"specifyBody": "json",
"jsonBody": "={{ JSON.stringify({ query: $json.response || 'Write the final movie script based on the story concept: previous_output' }) }}",
"options": {
"timeout": 300000
}
},
"notes": "Agent: Scriptwriter\nTask: Write the final movie script based on the story concept: {{previous_output}}"
}
],
"connections": {
"Manual Trigger": {
"Webhook": {
"main": [
[
{
+2 -2
View File
@@ -3,8 +3,8 @@ class Praisonai < Formula
desc "AI tools for various AI applications"
homepage "https://github.com/MervinPraison/PraisonAI"
url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.32.tar.gz"
sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.32.tar.gz | shasum -a 256`.split.first
url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.37.tar.gz"
sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.37.tar.gz | shasum -a 256`.split.first
license "MIT"
depends_on "[email protected]"
+209 -75
View File
@@ -35,7 +35,7 @@ class N8nHandler(FlagHandler):
"""
def __init__(self, verbose: bool = False, n8n_url: str = "http://localhost:5678",
use_execute_command: bool = True):
use_execute_command: bool = False, api_url: str = "http://127.0.0.1:8005"):
"""
Initialize the n8n handler.
@@ -44,10 +44,11 @@ class N8nHandler(FlagHandler):
n8n_url: Base URL of the n8n instance
use_execute_command: Use Execute Command nodes (runs praisonai directly)
instead of HTTP Request nodes
api_url: PraisonAI API URL that n8n will call (for tunnel/cloud deployments)
"""
super().__init__(verbose=verbose)
self.n8n_url = n8n_url
self.praisonai_api_url = "http://localhost:8000"
self.praisonai_api_url = api_url
self.use_execute_command = use_execute_command
@property
@@ -94,11 +95,28 @@ class N8nHandler(FlagHandler):
if api_key and REQUESTS_AVAILABLE:
workflow_id = self._create_workflow_via_api(workflow_json, api_key)
webhook_url = None
if workflow_id:
# Successfully created via API - open directly
url = f"{self.n8n_url}/workflow/{workflow_id}"
self.print_status("🚀 Workflow created in n8n!", "success")
# Activate the workflow so webhook is available
if self._activate_workflow(workflow_id, api_key):
self.print_status("✅ Workflow activated!", "success")
# Get webhook path from the workflow
trigger_node = next((n for n in workflow_json["nodes"]
if n.get("type") == "n8n-nodes-base.webhook"), None)
if trigger_node:
webhook_path = trigger_node["parameters"].get("path", "praisonai")
webhook_url = f"{self.n8n_url}/webhook/{webhook_path}"
self.print_status("", "info")
self.print_status("🔗 Webhook URL (to trigger workflow):", "info")
self.print_status(f" POST {webhook_url}", "info")
else:
self.print_status("⚠️ Could not activate workflow (activate manually in n8n)", "warning")
if open_browser:
self.open_in_browser(url)
self.print_status(f"🌐 Opening: {url}", "info")
@@ -126,7 +144,8 @@ class N8nHandler(FlagHandler):
"workflow": workflow_json,
"url": url,
"output_path": output_path,
"workflow_id": workflow_id
"workflow_id": workflow_id,
"webhook_url": webhook_url
}
def _create_workflow_via_api(self, workflow_json: Dict[str, Any],
@@ -179,12 +198,52 @@ class N8nHandler(FlagHandler):
self.log(f"Failed to create workflow via API: {e}", "warning")
return None
def convert_yaml_to_n8n(self, yaml_path: str) -> Dict[str, Any]:
def _activate_workflow(self, workflow_id: str, api_key: str) -> bool:
"""
Activate a workflow via n8n API.
Args:
workflow_id: The workflow ID to activate
api_key: n8n API key
Returns:
True if successful, False otherwise
"""
if not REQUESTS_AVAILABLE:
return False
try:
response = requests.post(
f"{self.n8n_url}/api/v1/workflows/{workflow_id}/activate",
headers={
"X-N8N-API-KEY": api_key,
"Content-Type": "application/json"
},
timeout=30
)
if response.status_code == 200:
return True
else:
self.log(f"Activation error: {response.status_code} - {response.text}", "warning")
return False
except Exception as e:
self.log(f"Failed to activate workflow: {e}", "warning")
return False
def convert_yaml_to_n8n(self, yaml_path: str, use_webhook: bool = True) -> Dict[str, Any]:
"""
Convert a PraisonAI YAML workflow to n8n JSON format.
The n8n workflow will have:
- A webhook trigger to receive requests
- One HTTP Request node per agent (calls /agents/{agent_name})
- Each agent node passes its output to the next agent
Args:
yaml_path: Path to the agents.yaml file
use_webhook: Use webhook trigger for programmatic execution (default: True)
Returns:
n8n workflow JSON as dictionary
@@ -197,6 +256,9 @@ class N8nHandler(FlagHandler):
workflow_name = yaml_content.get('name', 'PraisonAI Workflow')
description = yaml_content.get('description', '')
# Generate a unique webhook path from workflow name
webhook_path = workflow_name.lower().replace(' ', '-').replace('/', '-')[:50]
# Get agents and steps
agents = yaml_content.get('agents', yaml_content.get('roles', {}))
steps = yaml_content.get('steps', [])
@@ -209,82 +271,43 @@ class N8nHandler(FlagHandler):
nodes = []
connections = {}
# Add manual trigger node
trigger_node = self._create_trigger_node()
# Add trigger node (webhook for programmatic execution, manual otherwise)
trigger_node = self._create_trigger_node(use_webhook=use_webhook, webhook_path=webhook_path)
nodes.append(trigger_node)
# Track previous node for connections
prev_node_name = trigger_node["name"]
x_position = 450
# Process each step
x_position = 450 # Start position after trigger
# Create one HTTP node per agent step
for i, step in enumerate(steps):
if isinstance(step, dict):
if 'parallel' in step:
# Handle parallel steps
parallel_nodes, parallel_connections = self._create_parallel_nodes(
step['parallel'], x_position, agents
)
nodes.extend(parallel_nodes)
connections.update(parallel_connections)
# Connect previous node to first parallel node
if prev_node_name not in connections:
connections[prev_node_name] = {"main": [[]]}
for pnode in parallel_nodes:
connections[prev_node_name]["main"][0].append({
"node": pnode["name"],
"type": "main",
"index": 0
})
x_position += 200 * len(parallel_nodes)
prev_node_name = parallel_nodes[-1]["name"] if parallel_nodes else prev_node_name
elif 'route' in step:
# Handle route/decision steps
route_node = self._create_route_node(step, x_position, i)
nodes.append(route_node)
# Connect previous node
if prev_node_name not in connections:
connections[prev_node_name] = {"main": [[]]}
connections[prev_node_name]["main"][0].append({
"node": route_node["name"],
"type": "main",
"index": 0
})
prev_node_name = route_node["name"]
x_position += 200
else:
# Regular agent step
agent_id = step.get('agent', '')
action = step.get('action', step.get('description', ''))
agent_config = agents.get(agent_id, {})
agent_node = self._create_agent_node(
agent_id=agent_id,
agent_config=agent_config,
action=action,
position=[x_position, 300],
index=i
)
nodes.append(agent_node)
# Connect to previous node
if prev_node_name not in connections:
connections[prev_node_name] = {"main": [[]]}
connections[prev_node_name]["main"][0].append({
"node": agent_node["name"],
"type": "main",
"index": 0
})
prev_node_name = agent_node["name"]
x_position += 200
agent_id = step.get('agent', '')
action = step.get('action', step.get('description', ''))
agent_config = agents.get(agent_id, {})
# Create HTTP node that calls the specific agent endpoint
agent_node = self._create_per_agent_node(
agent_id=agent_id,
agent_config=agent_config,
action=action,
position=[x_position, 300],
index=i,
is_first=(i == 0)
)
nodes.append(agent_node)
# Connect to previous node
if prev_node_name not in connections:
connections[prev_node_name] = {"main": [[]]}
connections[prev_node_name]["main"][0].append({
"node": agent_node["name"],
"type": "main",
"index": 0
})
prev_node_name = agent_node["name"]
x_position += 250
# Build the complete workflow
# Generate a unique ID based on workflow name
@@ -334,8 +357,31 @@ class N8nHandler(FlagHandler):
})
return steps
def _create_trigger_node(self) -> Dict[str, Any]:
"""Create a manual trigger node."""
def _create_trigger_node(self, use_webhook: bool = False,
webhook_path: str = "praisonai") -> Dict[str, Any]:
"""Create a trigger node (webhook or manual).
Args:
use_webhook: If True, creates a webhook trigger for programmatic execution
webhook_path: Path for the webhook URL (only used if use_webhook=True)
"""
if use_webhook:
# Generate a unique webhook ID
webhook_id = hashlib.md5(webhook_path.encode()).hexdigest()[:16]
return {
"id": "trigger",
"name": "Webhook",
"type": "n8n-nodes-base.webhook",
"typeVersion": 2,
"position": [250, 300],
"webhookId": webhook_id,
"parameters": {
"path": webhook_path,
"httpMethod": "POST",
"responseMode": "lastNode",
"options": {}
}
}
return {
"id": "trigger",
"name": "Manual Trigger",
@@ -345,6 +391,94 @@ class N8nHandler(FlagHandler):
"parameters": {}
}
def _create_per_agent_node(self, agent_id: str, agent_config: Dict,
action: str, position: List[int],
index: int, is_first: bool = False) -> Dict[str, Any]:
"""
Create an HTTP Request node that calls a specific agent endpoint.
Args:
agent_id: Agent identifier (used in URL path)
agent_config: Agent configuration from YAML
action: The action/task description for this agent
position: Node position [x, y]
index: Node index
is_first: Whether this is the first agent (uses webhook input)
Returns:
n8n HTTP Request node configuration
"""
agent_name = agent_config.get('name', agent_id.title())
# Convert agent_id to URL-safe format (lowercase, underscores)
agent_url_id = agent_id.lower().replace(' ', '_')
# Build the query - first agent uses webhook input, others use previous response
# Escape single quotes and double braces (n8n expression syntax)
escaped_action = action.replace("'", "\\'").replace("{{", "").replace("}}", "")
if is_first:
# First agent: use query from webhook body
query_expr = "$json.body?.query || $json.query || '" + escaped_action + "'"
else:
# Subsequent agents: use previous agent's response + original action context
query_expr = "$json.response || '" + escaped_action + "'"
return {
"id": f"agent_{index}",
"name": agent_name,
"type": "n8n-nodes-base.httpRequest",
"typeVersion": 4.2,
"position": position,
"parameters": {
"method": "POST",
"url": f"{self.praisonai_api_url}/agents/{agent_url_id}",
"sendBody": True,
"specifyBody": "json",
"jsonBody": f"={{{{ JSON.stringify({{ query: {query_expr} }}) }}}}",
"options": {
"timeout": 300000 # 5 minute timeout per agent
}
},
"notes": f"Agent: {agent_name}\nTask: {action[:100]}"
}
def _create_praisonai_workflow_node(self, workflow_name: str,
agent_names: List[str],
position: List[int]) -> Dict[str, Any]:
"""
Create a single HTTP Request node that triggers the entire PraisonAI workflow.
The PraisonAI API will run all agents sequentially when called.
Args:
workflow_name: Name of the workflow
agent_names: List of agent names (for documentation)
position: Node position [x, y]
Returns:
n8n HTTP Request node configuration
"""
# The query will be passed from the webhook trigger
# Using n8n expression to get the body from the webhook
return {
"id": "praisonai_workflow",
"name": f"PraisonAI: {workflow_name}",
"type": "n8n-nodes-base.httpRequest",
"typeVersion": 4.2,
"position": position,
"parameters": {
"method": "POST",
"url": f"{self.praisonai_api_url}/agents",
"sendBody": True,
"specifyBody": "json",
# Pass the query from webhook body, or use a default
"jsonBody": "={{ JSON.stringify({ query: $json.body?.query || $json.query || 'Start workflow' }) }}",
"options": {
"timeout": 600000 # 10 minute timeout for full workflow
}
},
"notes": f"Runs agents: {', '.join(agent_names)}"
}
def _create_agent_node(self, agent_id: str, agent_config: Dict,
action: str, position: List[int],
index: int) -> Dict[str, Any]:
+229 -2
View File
@@ -504,7 +504,8 @@ class PraisonAI:
from .features.n8n import N8nHandler
n8n_handler = N8nHandler(
verbose=getattr(args, 'verbose', False),
n8n_url=getattr(args, 'n8n_url', 'http://localhost:5678')
n8n_url=getattr(args, 'n8n_url', 'http://localhost:5678'),
api_url=getattr(args, 'api_url', 'http://127.0.0.1:8005')
)
result = n8n_handler.execute(self.agent_file)
return result
@@ -578,7 +579,7 @@ class PraisonAI:
return default_args
# Define special commands
special_commands = ['chat', 'code', 'call', 'realtime', 'train', 'ui', 'context', 'research', 'memory', 'rules', 'workflow', 'hooks', 'knowledge', 'session', 'tools', 'todo', 'docs', 'mcp', 'commit']
special_commands = ['chat', 'code', 'call', 'realtime', 'train', 'ui', 'context', 'research', 'memory', 'rules', 'workflow', 'hooks', 'knowledge', 'session', 'tools', 'todo', 'docs', 'mcp', 'commit', 'serve']
parser = argparse.ArgumentParser(prog="praisonai", description="praisonAI command-line interface")
parser.add_argument("--framework", choices=["crewai", "autogen", "praisonai"], help="Specify the framework")
@@ -682,6 +683,12 @@ class PraisonAI:
# n8n Integration - export workflow to n8n
parser.add_argument("--n8n", action="store_true", help="Export workflow to n8n and open in browser")
parser.add_argument("--n8n-url", type=str, default="http://localhost:5678", help="n8n instance URL (default: http://localhost:5678)")
parser.add_argument("--api-url", type=str, default="http://127.0.0.1:8005", help="PraisonAI API URL for n8n to call (default: http://127.0.0.1:8005)")
# Serve - start API server for agents
parser.add_argument("--serve", action="store_true", help="Start API server for agents (use with agents.yaml)")
parser.add_argument("--port", type=int, default=8005, help="Server port (default: 8005)")
parser.add_argument("--host", type=str, default="127.0.0.1", help="Server host (default: 127.0.0.1)")
# If we're in a test environment, parse with empty args to avoid pytest interference
if in_test_env:
@@ -710,6 +717,13 @@ class PraisonAI:
args.realtime = True
if args.command == 'call':
args.call = True
if args.command == 'serve':
args.serve = True
# Handle serve command - start API server for agents
if args.command == 'serve' or args.serve:
self._handle_serve_command(args, unknown_args)
sys.exit(0)
# Handle both command and flag versions for call
if args.command == 'call' or args.call:
@@ -3044,6 +3058,219 @@ Now, {final_instruction.lower()}:"""
print("pip install praisonaiagents # For PraisonAIAgents\n")
sys.exit(1)
def _handle_serve_command(self, args, unknown_args):
"""
Handle the serve command - start API server for agents.
Supports:
- Sequential workflows with agents and steps
- Workflow patterns: route, parallel, loop, repeat
- All process types: sequential, workflow, hierarchical
Usage:
praisonai serve agents.yaml
praisonai serve agents.yaml --port 8005 --host 0.0.0.0
praisonai agents.yaml --serve
"""
import yaml
from praisonaiagents import Agent, PraisonAIAgents
# Determine the YAML file path
yaml_file = None
# Check if command is 'serve' and there's a file in unknown_args
if args.command == 'serve' and unknown_args:
yaml_file = unknown_args[0]
# Check if command itself is a yaml file (praisonai agents.yaml --serve)
elif args.command and args.command.endswith('.yaml'):
yaml_file = args.command
# Default to agents.yaml
else:
yaml_file = 'agents.yaml'
# Check if file exists
if not os.path.exists(yaml_file):
print(f"[red]ERROR: File not found: {yaml_file}[/red]")
print("\nUsage:")
print(" praisonai serve agents.yaml")
print(" praisonai serve agents.yaml --port 8005")
print(" praisonai agents.yaml --serve")
sys.exit(1)
print(f"📄 Loading workflow from: {yaml_file}")
# Load YAML file
with open(yaml_file, 'r') as f:
config = yaml.safe_load(f)
# Extract agents
agents_config = config.get('agents', config.get('roles', {}))
agents_dict = {} # Map agent_id to Agent instance
agents_list = []
for agent_id, agent_config in agents_config.items():
if isinstance(agent_config, dict):
agent = Agent(
name=agent_config.get('name', agent_id.title()),
role=agent_config.get('role', ''),
goal=agent_config.get('goal', ''),
backstory=agent_config.get('backstory', ''),
instructions=agent_config.get('instructions', ''),
llm=agent_config.get('llm', 'gpt-4o-mini'),
verbose=False
)
agents_dict[agent_id] = agent
agents_list.append(agent)
print(f" ✓ Loaded agent: {agent.name}")
if not agents_list:
print("[red]ERROR: No agents found in YAML file[/red]")
sys.exit(1)
# Extract tasks from steps (if defined)
steps = config.get('steps', [])
tasks_list = []
# Detect advanced workflow patterns
has_route = any(isinstance(s, dict) and 'route' in s for s in steps)
has_parallel = any(isinstance(s, dict) and 'parallel' in s for s in steps)
has_loop = any(isinstance(s, dict) and 'loop' in s for s in steps)
has_repeat = any(isinstance(s, dict) and 'repeat' in s for s in steps)
if has_route or has_parallel or has_loop or has_repeat:
patterns = []
if has_route: patterns.append("route")
if has_parallel: patterns.append("parallel")
if has_loop: patterns.append("loop")
if has_repeat: patterns.append("repeat")
print(f"\n🔀 Detected workflow patterns: {', '.join(patterns)}")
if steps:
from praisonaiagents import Task
print(f"\n📋 Loading workflow steps...")
# First pass: create all tasks
task_name_map = {} # Map step names to Task objects for context/next_tasks
for i, step in enumerate(steps):
if isinstance(step, dict):
# Handle advanced patterns
if 'route' in step:
print(f" ✓ Loaded routing step: {step.get('name', f'route_{i+1}')}")
continue # Route steps are handled by workflow process
if 'parallel' in step:
parallel_tasks = step.get('parallel', [])
print(f" ✓ Loaded parallel step: {len(parallel_tasks)} concurrent tasks")
for j, pt in enumerate(parallel_tasks):
if isinstance(pt, dict):
agent_id = pt.get('agent', '')
action = pt.get('action', pt.get('description', ''))
if agent_id in agents_dict:
task = Task(
name=f"parallel_{i}_{j}",
description=action,
agent=agents_dict[agent_id],
expected_output=pt.get('expected_output', 'Completed task output'),
async_execution=True, # Parallel tasks run async
)
tasks_list.append(task)
task_name_map[f"parallel_{i}_{j}"] = task
continue
if 'loop' in step:
loop_config = step.get('loop', {})
print(f" ✓ Loaded loop step: over '{loop_config.get('over', 'items')}'")
continue # Loop steps are handled by workflow process
if 'repeat' in step:
repeat_config = step.get('repeat', {})
print(f" ✓ Loaded repeat step: until '{repeat_config.get('until', 'done')}'")
continue # Repeat steps are handled by workflow process
# Standard step
agent_id = step.get('agent', '')
action = step.get('action', step.get('description', ''))
task_name = step.get('name', f"step_{i+1}")
if agent_id in agents_dict:
task = Task(
name=task_name,
description=action,
agent=agents_dict[agent_id],
expected_output=step.get('expected_output', 'Completed task output'),
task_type=step.get('task_type', 'task'),
is_start=step.get('is_start', i == 0),
async_execution=step.get('async_execution', False),
output_file=step.get('output_file'),
)
tasks_list.append(task)
task_name_map[task_name] = task
print(f" ✓ Loaded step {i+1}: {agent_id} → {action[:50]}...")
# Second pass: set up next_tasks and context relationships
for i, step in enumerate(steps):
if isinstance(step, dict) and 'agent' in step:
task_name = step.get('name', f"step_{i+1}")
if task_name in task_name_map:
task = task_name_map[task_name]
# Set next_tasks if specified
next_tasks = step.get('next_tasks', [])
if next_tasks:
task.next_tasks = next_tasks
# Set context if specified
context_refs = step.get('context', [])
if context_refs:
task.context = [task_name_map[ref] for ref in context_refs if ref in task_name_map]
# Set condition for workflow branching
condition = step.get('condition', {})
if condition:
task.condition = condition
port = args.port
host = args.host
# Extract process type from workflow config (default: sequential)
workflow_config = config.get('workflow', {})
process_type = workflow_config.get('process', 'sequential')
verbose = workflow_config.get('verbose', False)
# Also check top-level process key
if 'process' in config:
process_type = config['process']
print(f"\n🚀 Starting PraisonAI API server...")
print(f" Host: {host}")
print(f" Port: {port}")
print(f" Agents: {len(agents_list)}")
if tasks_list:
print(f" Tasks: {len(tasks_list)}")
print(f" Process: {process_type}")
# Create and launch - with tasks if defined
if tasks_list:
praison = PraisonAIAgents(
agents=agents_list,
tasks=tasks_list,
process=process_type,
verbose=1 if verbose else 0
)
else:
praison = PraisonAIAgents(
agents=agents_list,
process=process_type,
verbose=1 if verbose else 0
)
praison.launch(port=port, host=host)
# Keep the main thread alive to prevent exit
print("\n✅ Server running. Press Ctrl+C to stop.")
try:
import time
while True:
time.sleep(1)
except KeyboardInterrupt:
print("\n👋 Server stopped.")
def create_chainlit_chat_interface(self):
"""
Create a Chainlit interface for the chat application.
+1 -1
View File
@@ -57,7 +57,7 @@ class CloudDeployer:
file.write("FROM python:3.11-slim\n")
file.write("WORKDIR /app\n")
file.write("COPY . .\n")
file.write("RUN pip install flask praisonai==2.3.32 gunicorn markdown\n")
file.write("RUN pip install flask praisonai==2.3.37 gunicorn markdown\n")
file.write("EXPOSE 8080\n")
file.write('CMD ["gunicorn", "-b", "0.0.0.0:8080", "api:app"]\n')
+1 -1
View File
@@ -1 +1 @@
__version__ = "2.3.32"
__version__ = "2.3.37"
+1 -1
View File
@@ -12,7 +12,7 @@ dependencies = [
"rich>=13.7",
"markdown>=3.5",
"pyparsing>=3.0.0",
"praisonaiagents>=0.1.13",
"praisonaiagents>=0.1.15",
"python-dotenv>=0.19.0",
"instructor>=1.3.3",
"PyYAML>=6.0",
+15 -8
View File
@@ -81,26 +81,33 @@ steps:
assert "connections" in result
def test_n8n_workflow_has_trigger_node(self, sample_yaml_file):
"""Test that n8n workflow has a manual trigger node."""
"""Test that n8n workflow has a trigger node (webhook or manual)."""
from praisonai.cli.features.n8n import N8nHandler
handler = N8nHandler()
result = handler.convert_yaml_to_n8n(sample_yaml_file)
nodes = result["nodes"]
trigger_nodes = [n for n in nodes if "trigger" in n.get("type", "").lower() or "trigger" in n.get("name", "").lower()]
# Check for webhook or manual trigger nodes
trigger_nodes = [n for n in nodes if "trigger" in n.get("type", "").lower()
or "webhook" in n.get("type", "").lower()
or "trigger" in n.get("name", "").lower()
or "webhook" in n.get("name", "").lower()]
assert len(trigger_nodes) >= 1
def test_n8n_workflow_has_agent_nodes(self, sample_yaml_file):
"""Test that n8n workflow has agent nodes (Execute Command or HTTP Request)."""
def test_n8n_workflow_has_per_agent_nodes(self, sample_yaml_file):
"""Test that n8n workflow has HTTP Request nodes for each agent."""
from praisonai.cli.features.n8n import N8nHandler
handler = N8nHandler()
result = handler.convert_yaml_to_n8n(sample_yaml_file)
nodes = result["nodes"]
# Agent nodes can be either executeCommand or httpRequest
agent_nodes = [n for n in nodes if "executeCommand" in n.get("type", "") or "httpRequest" in n.get("type", "")]
# Should have at least 2 agent nodes (one per agent step)
assert len(agent_nodes) >= 2
# Should have HTTP Request nodes for each agent
http_nodes = [n for n in nodes if "httpRequest" in n.get("type", "")]
assert len(http_nodes) >= 2 # At least 2 agents
# Each node should call a specific agent endpoint
for node in http_nodes:
assert "/agents/" in node["parameters"]["url"]
def test_n8n_workflow_nodes_are_connected(self, sample_yaml_file):
"""Test that nodes are properly connected."""
+4 -4
View File
@@ -4197,7 +4197,7 @@ requires-dist = [
{ name = "praisonai-tools", marker = "extra == 'autogen'", specifier = ">=0.0.22" },
{ name = "praisonai-tools", marker = "extra == 'autogen-v4'", specifier = ">=0.0.22" },
{ name = "praisonai-tools", marker = "extra == 'crewai'", specifier = ">=0.0.22" },
{ name = "praisonaiagents", specifier = ">=0.1.13" },
{ name = "praisonaiagents", specifier = ">=0.1.15" },
{ name = "pyautogen", marker = "extra == 'autogen'", specifier = "==0.2.29" },
{ name = "pydantic", marker = "extra == 'chat'", specifier = "<=2.10.1" },
{ name = "pydantic", marker = "extra == 'code'", specifier = "<=2.10.1" },
@@ -4252,7 +4252,7 @@ wheels = [
[[package]]
name = "praisonaiagents"
version = "0.1.13"
version = "0.1.15"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "aiohttp" },
@@ -4261,9 +4261,9 @@ dependencies = [
{ name = "pydantic" },
{ name = "rich" },
]
sdist = { url = "https://files.pythonhosted.org/packages/79/9e/076a981f4d53b0fa58888748971797ab4ab968bf1d183e846e2b76653b19/praisonaiagents-0.1.13.tar.gz", hash = "sha256:1547fd38eef596c5b9bfffec86ce425935990e951605b143c17950ade23e2452", size = 620335, upload-time = "2025-12-18T13:15:34.77Z" }
sdist = { url = "https://files.pythonhosted.org/packages/76/1b/9a6352623f2f2c52373a890d04db1d4adc852e164c33840c0071f7d93a50/praisonaiagents-0.1.15.tar.gz", hash = "sha256:45c2ea0a699df4c28a10bf192ef9f6fde1bf9178e966a9892148ed02f0ddb142", size = 628803, upload-time = "2025-12-18T17:38:14.311Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/35/50/968fde58d1e02a41411c357251ab6f9f3f07a95167081c71a0a8ee873594/praisonaiagents-0.1.13-py3-none-any.whl", hash = "sha256:4d63873db8fcfaddffec35006864bd73aa55021a4a5e99b6ce9ba818b2ac0311", size = 547445, upload-time = "2025-12-18T13:15:32.935Z" },
{ url = "https://files.pythonhosted.org/packages/a4/e1/d0a98166a357da2ff94c79f3c311b070a8f37cd7bbec23f7648a29efa417/praisonaiagents-0.1.15-py3-none-any.whl", hash = "sha256:596991445a271e70831a89d300a25f8e482d75ef0405cab22eb5f53293880af1", size = 557414, upload-time = "2025-12-18T17:38:12.296Z" },
]
[[package]]