Compare commits

...
89 Commits
Author SHA1 Message Date
Mervin Praison ebb9b24bff Release v2.4.2 2025-12-25 00:29:49 +00:00
Mervin Praison 759ef2c655 Update database initialization to use db() shortcut and expand tools table with 100+ tools 2025-12-25 00:24:48 +00:00
Mervin Praison cb4318f6e8 Release v2.4.1 2025-12-24 15:00:45 +00:00
Mervin Praison d9bb869d5e Add documentation for 6 new features: Shadow Git Checkpoints, Background Tasks, Policy Engine, Thinking Budgets, Output Styles, and Context Compaction 2025-12-24 08:02:48 +00:00
Mervin Praison 34b6aeb76b Release v2.4.0 2025-12-24 07:47:53 +00:00
Mervin Praison 078b7801e1 Add 'response' field to workflow API for consistency with single-agent endpoint 2025-12-23 17:50:07 +00:00
Mervin Praison a970e849b4 Release v2.3.87 2025-12-23 16:08:35 +00:00
Mervin Praison 8e68f4e71c Release v2.3.86 2025-12-23 15:23:31 +00:00
Mervin Praison 308758cdc2 Release v2.3.85 2025-12-23 15:07:46 +00:00
Mervin Praison f8fa61b082 Release v2.3.84 2025-12-23 14:57:15 +00:00
Mervin Praison 955d236924 Release v2.3.84 2025-12-23 13:20:08 +00:00
Mervin Praison d9d63c3a13 Release v2.3.83 2025-12-23 13:06:42 +00:00
Mervin Praison 5bee6eb470 Release v2.3.83 2025-12-23 11:42:47 +00:00
Mervin Praison 7cb1676b7c Release v2.3.82 2025-12-23 09:41:09 +00:00
Mervin Praison e2e9c3f9ee Release v2.3.81 2025-12-23 08:11:31 +00:00
Mervin Praison cbe8beeca6 Release v2.3.80 2025-12-23 07:54:07 +00:00
Mervin Praison f7b66a52b7 Release v2.3.79 2025-12-23 07:27:35 +00:00
Mervin Praison 6dbf194312 Release v2.3.78 2025-12-23 06:46:15 +00:00
Mervin Praison 335e8d7e47 Release v2.3.77 2025-12-23 06:28:26 +00:00
Mervin Praison d65840e092 Release v2.3.76 2025-12-23 05:40:52 +00:00
Mervin Praison 5a13571fdd Release v2.3.75 2025-12-22 19:11:32 +00:00
Mervin Praison e3b48c2526 Release v2.3.74 2025-12-22 17:01:50 +00:00
Mervin Praison 8d32238b4c Release v2.3.73 2025-12-22 15:47:51 +00:00
Mervin Praison c17b1298bb Reorganize README feature table with improved categorization and hierarchy 2025-12-22 14:50:37 +00:00
Mervin Praison ec6c150315 Release v2.3.72 2025-12-22 13:44:48 +00:00
Mervin Praison 3d26cedd5d Release v2.3.71 2025-12-22 11:59:12 +00:00
Mervin Praison 8c735a60b2 Release v2.3.70 2025-12-22 10:43:48 +00:00
Mervin Praison 2f5f2f1c13 Release v2.3.69 2025-12-22 10:40:53 +00:00
Mervin Praison 0253073c68 Release v2.3.68 2025-12-22 10:35:45 +00:00
Mervin Praison 7f0e334e8c Release v2.3.67 2025-12-22 09:14:33 +00:00
Mervin Praison cacf1097ef Release v2.3.66 2025-12-22 08:50:02 +00:00
Mervin Praison 09f01919d2 Fix duplicate --max-retries argument conflict in CLI parser 2025-12-21 05:19:08 +00:00
Mervin Praison 92397615f8 Release v2.3.65 2025-12-20 21:49:04 +00:00
Mervin Praison 4b21943a53 Release v2.3.64 2025-12-20 19:57:39 +00:00
Mervin Praison af56a753bd Release v2.3.63 2025-12-20 12:02:44 +00:00
Mervin Praison 15c2559a3c Release v2.3.62 2025-12-20 11:59:46 +00:00
Mervin Praison 3215bdc0a6 Release v2.3.61 2025-12-20 11:15:22 +00:00
Mervin Praison 0ff0d83744 fix: Preserve real API keys for integration tests in conftest.py
- Check for @pytest.mark.integration marker
- Check if test file is in integration/ directory
- This prevents conftest from overriding real API keys with 'test-key'
2025-12-20 03:58:13 +00:00
Mervin Praison 8f956f91dd fix: Improve API key validation in integration test skip conditions
- Check for 'sk-' prefix and minimum length (40 chars) to detect invalid keys
- This ensures tests are skipped when using placeholder/test API keys in CI
2025-12-20 03:54:58 +00:00
Mervin Praison 954f6ac33d fix: Skip integration tests when using test API key
- test_auto_generator_integration.py: Skip when OPENAI_API_KEY contains 'test'
- test_model_routing_real.py: Skip test_real_openai_call when using test key
- test_cli_integrations.py: Skip all tests when using test key

These tests require real API credentials and should not run in CI
when only a placeholder test key is available.
2025-12-20 03:51:22 +00:00
Mervin Praison 264d11d8d9 fix: Fix failing unit tests for ClaudeCodeSDK and QueryRewriter CLI
- test_use_sdk_option: Updated test to match actual behavior where use_sdk
  is only True when SDK is available (use_sdk and CLAUDE_SDK_AVAILABLE)
- test_applies_rewrite_before_agent: Fixed mocking to properly mock
  _run_with_status_display and praisonaiagents submodules
2025-12-20 03:47:28 +00:00
Mervin Praison 88da2bdd7b Release v2.3.60 2025-12-20 03:35:28 +00:00
Mervin Praison b84e6c2460 Release v2.3.59 2025-12-20 03:28:48 +00:00
Mervin Praison 0434b23ada Release v2.3.58 2025-12-20 03:08:31 +00:00
Mervin Praison a9951e0694 Release v2.3.57 2025-12-19 16:47:09 +00:00
Mervin Praison 073062cd96 Release v2.3.56 2025-12-19 16:14:58 +00:00
Mervin Praison 936b422876 Release v2.3.55 2025-12-19 16:05:38 +00:00
Mervin Praison 99ab421ce2 Release v2.3.54 2025-12-19 16:04:43 +00:00
Mervin Praison c1f334905e Release v2.3.53 2025-12-19 15:38:30 +00:00
Mervin Praison 6d3ba7cdf8 Release v2.3.52 2025-12-19 15:34:19 +00:00
Mervin Praison 96616498d2 Release v2.3.51 2025-12-19 15:25:30 +00:00
Mervin Praison 1cad345415 Release v2.3.50 2025-12-19 15:19:29 +00:00
Mervin Praison 802e7e52f9 fix(readme): correct CLI docs URLs to include /docs/ prefix
Changed all https://docs.praison.ai/cli/* URLs to https://docs.praison.ai/docs/cli/*
Fixed 39 URLs total
2025-12-19 13:01:53 +00:00
Mervin Praison 53704f234c Release v2.3.49 2025-12-19 12:49:12 +00:00
Mervin Praison b575994d91 docs(readme): update AI commit CLI with security scanning and auto mode
- Add full auto mode (-a) with stage, security check, commit, push
- Document security scanning features
- Add --no-verify flag documentation
- List detected sensitive patterns
2025-12-19 12:35:49 +00:00
Mervin Praison 7ed8596261 Release v2.3.48 2025-12-19 12:35:02 +00:00
Mervin Praison deef65c4bb Release v2.3.47 2025-12-19 12:23:05 +00:00
Mervin Praison d51327effd Release v2.3.46 2025-12-19 12:22:29 +00:00
Mervin Praison 30ce8857e8 fix: suppress crewai.cli.config INFO log at startup 2025-12-19 12:18:23 +00:00
Mervin Praison 397874ccfc Release v2.3.45 2025-12-19 12:16:12 +00:00
Mervin Praison 2016902e9a docs: Add examples README with quick links to docs 2025-12-19 11:14:24 +00:00
Mervin Praison d2d19511da docs: Update README with Model Router info and fix links
- Rename 'Router Agent' to 'Model Router' for consistency
- Fix docs link to point to /features/model-router
- Add model-routing template CLI command
- Add link to Model Router documentation
2025-12-19 11:13:48 +00:00
Mervin Praison bc9579b497 Add Code category with individual examples for external CLI integrations
- Add Code category in Key Features table (like Workflows)
- Add individual examples for each external agent:
  - claude_code_example.py
  - gemini_cli_example.py
  - codex_cli_example.py
  - cursor_cli_example.py
  - external_agents_example.py (all integrations)
  - code_editing_example.py
- Link to individual docs pages for each integration
- Organize code-related features under Code category
2025-12-19 11:07:25 +00:00
Mervin Praison 3161389341 Add External Agents CLI documentation to README
- Add External Agents feature to Key Features table
- Update Code Editing docs link to /code/editing
- Add External Agents CLI section with examples
- Include Python API usage examples
- Add environment variables documentation
- Link to detailed docs at docs.praison.ai/code/external-agents
2025-12-19 11:01:24 +00:00
Mervin Praison b01eb2b8fa Release v2.3.44 2025-12-19 10:50:33 +00:00
Mervin Praison e609fb7425 Release v2.3.43 2025-12-19 10:32:33 +00:00
Mervin Praison 168ef98473 feat: Implement all remaining AutoAgents improvements (TDD)
TODO 2: Single-Agent Option
- Add SingleAgentStructure Pydantic model
- Add single_agent parameter to AutoGenerator and WorkflowAutoGenerator

TODO 3: LLM-Based Pattern Recommendation
- Add PatternRecommendation Pydantic model (pattern, reasoning, confidence)
- Add recommend_pattern_llm() method to WorkflowAutoGenerator

TODO 4: Validation Gates
- Add ValidationGate Pydantic model (criteria, pass_action, fail_action)
- Add optional gates field to WorkflowStructure

TODO 6: Pattern Support for AutoGenerator
- Add pattern parameter to AutoGenerator
- Add recommend_pattern() method to AutoGenerator
- Add pattern-specific guidance in get_user_content()

TODO 7: Merge Support for WorkflowAutoGenerator
- Add merge parameter to generate() method
- Add merge_with_existing_workflow() method

TODO 10: Framework Support for WorkflowAutoGenerator
- Add framework parameter to WorkflowAutoGenerator
- Default framework is 'praisonai'

Test Coverage:
- 51 tests pass (up from 34)
- 17 new tests added using TDD approach
- All tests verify new functionality
2025-12-19 10:29:40 +00:00
Mervin Praison ce2c2926f9 refactor: Add BaseAutoGenerator class and improve WorkflowAutoGenerator
- Create BaseAutoGenerator with shared functionality (DRY principle)
  - Lazy-loaded instructor client (LiteLLM or OpenAI fallback)
  - Environment variable handling for model/API configuration
  - analyze_complexity() static method for task complexity detection
  - get_available_tools() static method for tool list
  - AVAILABLE_TOOLS constant shared between generators

- Update AutoGenerator to inherit from BaseAutoGenerator
  - Removes ~40 lines of duplicate code
  - Maintains all existing functionality

- Update WorkflowAutoGenerator to inherit from BaseAutoGenerator
  - Removes ~40 lines of duplicate code
  - Add dynamic agent count based on task complexity
  - Add tools list to workflow prompts
  - Improved prompt with 3-step analysis

- Add 8 new unit tests for new functionality
  - TestBaseAutoGenerator: complexity analysis, tools list
  - TestWorkflowDynamicAgentCount: prompt improvements

All 34 tests pass with no performance impact.
2025-12-19 10:23:29 +00:00
Mervin Praison 0a569e2b46 docs: Add AutoAgents workflow patterns examples and update README
- Add autoagents_workflow_patterns.py with all 6 workflow patterns
- Update auto_agents_example.py with new features and CLI examples
- Add orchestrator-workers and evaluator-optimizer patterns to README CLI section
- Update pattern options table with all 6 patterns
- Add documentation links to examples
2025-12-19 09:54:05 +00:00
Mervin Praison 054aea8f66 Add MCP Registry badge and ignore MCP tokens 2025-12-18 22:18:33 +00:00
Mervin Praison 34f1293cfc Release v2.3.42 2025-12-18 22:13:08 +00:00
Mervin Praison 7c1e32afc0 upate 2025-12-18 22:11:02 +00:00
Mervin Praison 1125c835ec Release v2.3.41 2025-12-18 22:09:58 +00:00
Mervin Praison 889d04a843 Release v2.3.40 2025-12-18 22:07:54 +00:00
Mervin Praison fba811c477 Release v2.3.39 2025-12-18 21:54:19 +00:00
Mervin Praison df687d86c3 Release v2.3.38 2025-12-18 21:26:34 +00:00
Mervin Praison afa37b0782 Release v2.3.37 2025-12-18 17:41:16 +00:00
Mervin Praison 43a15e6049 Release v2.3.36 2025-12-18 16:42:58 +00:00
Mervin Praison 8bf5a08246 Release v2.3.35 2025-12-18 16:28:02 +00:00
Mervin Praison aa81094ac3 Release v2.3.34 2025-12-18 14:55:33 +00:00
Mervin Praison c74fd7e925 Release v2.3.33 2025-12-18 14:41:36 +00:00
Mervin Praison 249acf7965 docs: Add performance benchmarks and auto-update functionality
Add performance comparison table to README showing PraisonAI as fastest framework for agent instantiation. Include benchmarks for LangGraph and CrewAI frameworks. Implement auto-save functionality to generate BENCHMARK_RESULTS.md and update README.md with latest benchmark results.
2025-12-18 13:30:06 +00:00
Mervin Praison ab68b2d296 Release v2.3.32 2025-12-18 13:17:32 +00:00
Mervin Praison abcc5b3b04 Release v2.3.31 2025-12-18 12:57:05 +00:00
Mervin Praison e584f56099 Release v2.3.30 2025-12-18 11:59:22 +00:00
Mervin Praison b6b01af066 Release v2.3.33 2025-12-18 11:19:29 +00:00
Mervin Praison 9aae3eff86 Release v2.3.32 2025-12-18 10:48:09 +00:00
Mervin Praison f90582c66b Release v2.3.31 2025-12-18 10:27:03 +00:00
Mervin Praison 67a8d65d8e Release v2.3.30 2025-12-18 09:07:42 +00:00
399 changed files with 80493 additions and 861 deletions

No files matched your search

+4 -1
View File
@@ -77,4 +77,7 @@ agents/praisonaiagents/praisonaiagents.egg-info
.qodo
CopilotKit*
.gemini
issue_details.json
issue_details.json
# MCP Registry tokens
.mcpregistry_*
+813 -167
View File
File diff suppressed because it is too large. Load diff
+2 -1
View File
@@ -14,9 +14,10 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
# Note: praisonai is installed from PyPI - remove version pin to use latest available
RUN pip install --no-cache-dir \
flask \
"praisonai>=2.3.29" \
praisonai \
"praisonai[api]" \
gunicorn \
markdown
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.29" \
"praisonai>=2.4.2" \
"praisonai[chat]" \
"embedchain[github,youtube]"
+1 -1
View File
@@ -20,7 +20,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.29" \
"praisonai>=2.4.2" \
"praisonai[ui]" \
"praisonai[chat]" \
"praisonai[realtime]" \
+25
View File
@@ -0,0 +1,25 @@
# Dockerfile for PraisonAI Serve API
# Deploy to Railway, Render, Fly.io, or any container platform
#
# Build: docker build -f Dockerfile.serve -t praisonai-serve .
# Run: docker run -p 8005:8005 -e OPENAI_API_KEY=sk-xxx praisonai-serve
FROM python:3.11-slim
WORKDIR /app
# Install dependencies
RUN pip install --no-cache-dir praisonai praisonaiagents
# Copy agents configuration
COPY agents.yaml .
# Expose the API port
EXPOSE 8005
# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD curl -f http://localhost:8005/health || exit 1
# Start the API server
CMD ["praisonai", "serve", "agents.yaml", "--host", "0.0.0.0", "--port", "8005"]
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.29" \
"praisonai>=2.4.2" \
"praisonai[ui]" \
"praisonai[crewai]"
+49
View File
@@ -0,0 +1,49 @@
# PraisonAI Examples
This folder contains examples for PraisonAI. For detailed documentation, visit [docs.praison.ai](https://docs.praison.ai).
## Structure
```
examples/
├── python/ # Python examples
│ ├── agents/ # Agent examples (single, multi, router, etc.)
│ ├── workflows/ # Workflow patterns (routing, parallel, loop)
│ ├── tools/ # Custom tools examples
│ ├── mcp/ # MCP protocol examples
│ ├── memory/ # Memory and sessions
│ ├── code/ # Code editing and external CLI tools
│ └── ...
├── yaml/ # YAML workflow examples
└── cookbooks/ # Complete use-case examples
```
## Quick Links
| Category | Examples | Docs |
|----------|----------|------|
| **Agents** | [python/agents/](python/agents/) | [📖](https://docs.praison.ai/concepts/agents) |
| **Workflows** | [python/workflows/](python/workflows/) | [📖](https://docs.praison.ai/features/workflows) |
| **Model Router** | [python/agents/router-agent-cost-optimization.py](python/agents/router-agent-cost-optimization.py) | [📖](https://docs.praison.ai/features/model-router) |
| **MCP** | [python/mcp/](python/mcp/) | [📖](https://docs.praison.ai/mcp) |
| **Memory** | [python/memory/](python/memory/) | [📖](https://docs.praison.ai/concepts/memory) |
| **Tools** | [python/tools/](python/tools/) | [📖](https://docs.praison.ai/tools) |
| **Code** | [python/code/](python/code/) | [📖](https://docs.praison.ai/code) |
| **YAML** | [yaml/](yaml/) | [📖](https://docs.praison.ai/features/yaml-workflows) |
## Running Examples
```bash
# Install PraisonAI
pip install praisonai
# Set API key
export OPENAI_API_KEY=your_key_here
# Run an example
python examples/python/agents/single-agent.py
```
## CLI Commands
See the main [README.md](../README.md#-cli--no-code-interface) for all CLI commands.
+92
View File
@@ -0,0 +1,92 @@
#!/usr/bin/env python3
"""
Basic Background Tasks Example for PraisonAI Agents.
This example demonstrates how to use the background task system to:
1. Run tasks asynchronously
2. Track task progress
3. Cancel running tasks
4. Handle task completion
Usage:
python basic_background.py
"""
import asyncio
import time
from praisonaiagents.background import BackgroundRunner, BackgroundConfig, TaskStatus
async def main():
print("=" * 60)
print("Background Tasks Demo")
print("=" * 60)
# Create a background runner with config
config = BackgroundConfig(max_concurrent_tasks=3)
runner = BackgroundRunner(config=config)
# Define some example tasks
def slow_task(duration: float, name: str) -> str:
"""A slow task that simulates work."""
print(f" 🔄 Task '{name}' starting (will take {duration}s)")
time.sleep(duration)
return f"Task '{name}' completed after {duration}s"
async def async_task(duration: float, name: str) -> str:
"""An async task that simulates work."""
print(f" 🔄 Async task '{name}' starting (will take {duration}s)")
await asyncio.sleep(duration)
return f"Async task '{name}' completed after {duration}s"
# Submit tasks
print("\n--- Submitting Tasks ---")
task1 = await runner.submit(slow_task, args=(1.0, "quick-job"), name="task1")
print(f"✅ Submitted task 1: {task1.id[:8]}")
task2 = await runner.submit(slow_task, args=(2.0, "medium-job"), name="task2")
print(f"✅ Submitted task 2: {task2.id[:8]}")
task3 = await runner.submit(async_task, args=(1.5, "async-job"), name="task3")
print(f"✅ Submitted async task 3: {task3.id[:8]}")
# List all tasks
print("\n--- Task Status ---")
for task in runner.tasks:
print(f" [{task.id[:8]}] {task.status.value}")
# Wait for task 1 to complete
print("\n--- Waiting for Task 1 ---")
await task1.wait(timeout=5.0)
print(f"✅ Task 1 result: {task1.result}")
# Check status of all tasks
print("\n--- Updated Task Status ---")
for task in [task1, task2, task3]:
status = "✅" if task.status == TaskStatus.COMPLETED else "🔄"
print(f" {status} [{task.id[:8]}] {task.status.value}")
# Wait for all remaining tasks
print("\n--- Waiting for All Tasks ---")
await task2.wait(timeout=10.0)
await task3.wait(timeout=10.0)
# Final status
print("\n--- Final Results ---")
for task in [task1, task2, task3]:
if task.result:
print(f" ✅ {task.result}")
# Cleanup
runner.clear_completed()
print("\n✅ Cleared completed tasks")
print("\n" + "=" * 60)
print("Demo Complete!")
print("=" * 60)
if __name__ == "__main__":
asyncio.run(main())
+110
View File
@@ -0,0 +1,110 @@
#!/usr/bin/env python3
"""
Basic Checkpoints Example for PraisonAI Agents.
This example demonstrates how to use the shadow git checkpointing system to:
1. Save checkpoints before making changes
2. List available checkpoints
3. View diffs between checkpoints
4. Restore to a previous checkpoint
Usage:
python basic_checkpoints.py
"""
import asyncio
import tempfile
from pathlib import Path
from praisonaiagents.checkpoints import CheckpointService
async def main():
# Create a temporary workspace for demonstration
with tempfile.TemporaryDirectory() as workspace:
print("=" * 60)
print("Shadow Git Checkpointing Demo")
print("=" * 60)
print(f"\nWorkspace: {workspace}")
# Create some initial files
test_file = Path(workspace) / "example.py"
test_file.write_text("# Initial content\nprint('Hello, World!')\n")
print(f"\n✅ Created initial file: {test_file.name}")
# Initialize the checkpoint service
service = CheckpointService(workspace_dir=workspace)
try:
# Initialize the shadow git repository
print("\n--- Initializing Checkpoint Service ---")
await service.initialize()
print("✅ Shadow git repository initialized")
# Save first checkpoint
print("\n--- Saving Checkpoint 1: Initial State ---")
result1 = await service.save("Initial state")
print(f"✅ Checkpoint saved: {result1.checkpoint.id[:8]}")
print(f" Message: {result1.checkpoint.message}")
# Make some changes
print("\n--- Making Changes ---")
test_file.write_text("# Modified content\nprint('Hello, PraisonAI!')\nprint('Checkpoints are awesome!')\n")
print("✅ Modified example.py")
# Create a new file
new_file = Path(workspace) / "utils.py"
new_file.write_text("def helper():\n return 'I help!'\n")
print("✅ Created utils.py")
# Save second checkpoint
print("\n--- Saving Checkpoint 2: After Changes ---")
result2 = await service.save("Added features")
print(f"✅ Checkpoint saved: {result2.checkpoint.id[:8]}")
# List all checkpoints
print("\n--- Listing All Checkpoints ---")
checkpoints = await service.list_checkpoints()
for i, cp in enumerate(checkpoints, 1):
print(f" {i}. [{cp.id[:8]}] {cp.message} ({cp.timestamp})")
# Show diff
print("\n--- Current Diff (uncommitted changes) ---")
diff = await service.diff()
if diff.files:
for f in diff.files:
print(f" {f.status}: {f.path}")
else:
print(" No uncommitted changes")
# Make more changes to show diff
test_file.write_text("# Even more changes\nprint('This will be reverted')\n")
print("\n✅ Made more changes to example.py")
diff = await service.diff()
print("\n--- New Diff ---")
if diff.files:
for f in diff.files:
print(f" {f.status}: {f.path}")
# Restore to checkpoint 2
print(f"\n--- Restoring to Checkpoint: {result2.checkpoint.id[:8]} ---")
await service.restore(result2.checkpoint.id)
print("✅ Restored successfully!")
# Verify restoration
content = test_file.read_text()
print(f"\n--- Verified Content After Restore ---")
print(content)
except Exception as e:
print(f"\n❌ Error: {e}")
raise
print("\n" + "=" * 60)
print("Demo Complete!")
print("=" * 60)
if __name__ == "__main__":
asyncio.run(main())
+93
View File
@@ -0,0 +1,93 @@
# Persistence CLI Commands
## Quick Reference
```bash
# Check database connectivity
praisonai persistence doctor \
--conversation-url postgresql://localhost:5432/praisonai \
--state-url redis://localhost:6379
# Run agent with persistence
praisonai persistence run \
--conversation-url postgresql://localhost:5432/praisonai \
--session-id my-session
# Resume a session
praisonai persistence resume \
--conversation-url postgresql://localhost:5432/praisonai \
--session-id my-session
# Export session to file
praisonai persistence export \
--conversation-url postgresql://localhost:5432/praisonai \
--session-id my-session \
--output session_backup.jsonl
# Import session from file
praisonai persistence import \
--conversation-url postgresql://localhost:5432/praisonai \
--file session_backup.jsonl
# Check schema status
praisonai persistence status \
--conversation-url postgresql://localhost:5432/praisonai \
--state-url redis://localhost:6379
# Apply migrations
praisonai persistence migrate \
--conversation-url postgresql://localhost:5432/praisonai \
--state-url redis://localhost:6379
```
## Environment Variables
Instead of CLI flags, you can use environment variables:
```bash
export PRAISON_CONVERSATION_URL=postgresql://localhost:5432/praisonai
export PRAISON_STATE_URL=redis://localhost:6379
export PRAISON_KNOWLEDGE_URL=http://localhost:6333
# Then run without URL flags
praisonai persistence doctor
praisonai persistence status
```
## Docker Setup
```bash
# Start all services
docker run -d --name praison-postgres \
-e POSTGRES_PASSWORD=praison123 \
-e POSTGRES_DB=praisonai \
-p 5432:5432 postgres:16
docker run -d --name praison-redis \
-p 6379:6379 redis:7
docker run -d --name praison-qdrant \
-p 6333:6333 -p 6334:6334 qdrant/qdrant
# Verify
docker ps
```
## Example Workflow
```bash
# 1. Start services
docker-compose up -d
# 2. Check connectivity
praisonai persistence doctor
# 3. Run agent
praisonai persistence run --session-id demo-session
# 4. Export for backup
praisonai persistence export --session-id demo-session --output backup.jsonl
# 5. Import to another environment
praisonai persistence import --file backup.jsonl
```
+124
View File
@@ -0,0 +1,124 @@
# PraisonAI Persistence CLI - End-to-End Guide
This guide demonstrates the CLI commands for database persistence.
## Prerequisites
```bash
pip install "praisonai[tools]"
# Start Docker containers
docker run -d --name praison-postgres -p 5432:5432 \
-e POSTGRES_PASSWORD=praison123 \
-e POSTGRES_DB=praisonai \
postgres:16
docker run -d --name praison-qdrant -p 6333:6333 qdrant/qdrant
docker run -d --name praison-redis -p 6379:6379 redis:7
```
## 1. Doctor Command - Validate Connectivity
```bash
# Check all stores
praisonai persistence doctor \
--conversation-url "postgresql://postgres:praison123@localhost:5432/praisonai" \
--knowledge-url "http://localhost:6333" \
--state-url "redis://localhost:6379"
```
**Expected Output:**
```
==================================================
PraisonAI Persistence Doctor
==================================================
[Conversation Store] Testing: postgresql://postgres:****@localhost:5432/praisonai
✅ Connected (postgres)
[Knowledge Store] Testing: http://localhost:6333
✅ Connected (qdrant)
[State Store] Testing: redis://localhost:6379
✅ Connected (redis)
==================================================
Results: 3/3 stores connected successfully
==================================================
```
## 2. Run Command - Execute Agent with Persistence
```bash
# Run agent with persistence (dry-run first)
praisonai persistence run \
--session-id "cli-demo-001" \
--conversation-url "postgresql://postgres:praison123@localhost:5432/praisonai" \
--dry-run \
"Hello, my name is Bob"
# Actually run it
praisonai persistence run \
--session-id "cli-demo-001" \
--conversation-url "postgresql://postgres:praison123@localhost:5432/praisonai" \
"Hello, my name is Bob"
```
**Expected Output:**
```
[Session: cli-demo-001]
Agent: Hello Bob! Nice to meet you. How can I help you today?
```
## 3. Resume Command - Continue a Session
```bash
# Show history
praisonai persistence resume \
--session-id "cli-demo-001" \
--conversation-url "postgresql://postgres:praison123@localhost:5432/praisonai" \
--show-history
# Continue conversation
praisonai persistence resume \
--session-id "cli-demo-001" \
--conversation-url "postgresql://postgres:praison123@localhost:5432/praisonai" \
--continue "What's my name?"
```
**Expected Output:**
```
[Session: cli-demo-001]
Messages in history: 2
--- Conversation History ---
[USER] Hello, my name is Bob
[ASSISTANT] Hello Bob! Nice to meet you...
--- End History ---
Agent: Your name is Bob!
```
## Environment Variables
Instead of passing URLs on command line, use environment variables:
```bash
export PRAISON_CONVERSATION_URL="postgresql://postgres:praison123@localhost:5432/praisonai"
export PRAISON_KNOWLEDGE_URL="http://localhost:6333"
export PRAISON_STATE_URL="redis://localhost:6379"
export OPENAI_API_KEY="your-key"
# Now commands are simpler
praisonai persistence doctor --all
praisonai persistence run --session-id "my-session" "Hello!"
```
## Full CLI Reference
```
praisonai persistence --help
praisonai persistence doctor --help
praisonai persistence run --help
praisonai persistence resume --help
```
+146
View File
@@ -0,0 +1,146 @@
#!/usr/bin/env python3
"""
Basic Context Compaction Example for PraisonAI Agents.
This example demonstrates how to use context compaction to:
1. Manage conversation context length
2. Use different compaction strategies
3. Track compaction results
4. Preserve important messages
Usage:
python basic_compaction.py
"""
from praisonaiagents.compaction import (
ContextCompactor, CompactionConfig, CompactionStrategy, CompactionResult
)
def main():
print("=" * 60)
print("Context Compaction Demo")
print("=" * 60)
# ==========================================================================
# Sample Messages
# ==========================================================================
messages = [
{"role": "system", "content": "You are a helpful AI assistant."},
{"role": "user", "content": "Hello! Can you help me with Python?"},
{"role": "assistant", "content": "Of course! I'd be happy to help you with Python. What would you like to know?"},
{"role": "user", "content": "How do I read a file in Python?"},
{"role": "assistant", "content": "To read a file in Python, you can use the built-in open() function. Here's an example:\n\nwith open('filename.txt', 'r') as f:\n content = f.read()\n\nThis opens the file, reads its content, and automatically closes it."},
{"role": "user", "content": "What about writing to a file?"},
{"role": "assistant", "content": "To write to a file, use the 'w' mode:\n\nwith open('filename.txt', 'w') as f:\n f.write('Hello, World!')\n\nUse 'a' mode to append instead of overwriting."},
{"role": "user", "content": "Can you explain list comprehensions?"},
{"role": "assistant", "content": "List comprehensions are a concise way to create lists. Instead of:\n\nresult = []\nfor x in range(10):\n result.append(x * 2)\n\nYou can write:\n\nresult = [x * 2 for x in range(10)]"},
]
print(f"\n--- Original Messages ---")
print(f" Message count: {len(messages)}")
# ==========================================================================
# Basic Compaction
# ==========================================================================
print("\n--- Basic Compaction (Truncate Strategy) ---")
compactor = ContextCompactor(
max_tokens=50, # Low limit to trigger compaction
strategy=CompactionStrategy.TRUNCATE,
preserve_system=True,
preserve_recent=2
)
# Check stats before
stats = compactor.get_stats(messages)
print(f" Total tokens: {stats['total_tokens']}")
print(f" Max tokens: {stats['max_tokens']}")
print(f" Needs compaction: {stats['needs_compaction']}")
# Compact
compacted, result = compactor.compact(messages)
print(f"\n After compaction:")
print(f" Messages: {len(messages)} -> {len(compacted)}")
print(f" Tokens: {result.original_tokens} -> {result.compacted_tokens}")
print(f" Saved: {result.tokens_saved} tokens")
print(f" Compression: {result.compression_ratio:.1%}")
# ==========================================================================
# Sliding Window Strategy
# ==========================================================================
print("\n--- Sliding Window Strategy ---")
compactor = ContextCompactor(
max_tokens=50,
strategy=CompactionStrategy.SLIDING,
preserve_system=True,
preserve_recent=3
)
compacted, result = compactor.compact(messages)
print(f" Messages kept: {result.messages_kept}")
print(f" Messages removed: {result.messages_removed}")
print(f" Strategy: {result.strategy_used.value}")
# ==========================================================================
# Summarize Strategy
# ==========================================================================
print("\n--- Summarize Strategy ---")
compactor = ContextCompactor(
max_tokens=50,
strategy=CompactionStrategy.SUMMARIZE,
preserve_system=True,
preserve_recent=2
)
compacted, result = compactor.compact(messages)
print(f" Messages kept: {result.messages_kept}")
print(f" Strategy: {result.strategy_used.value}")
# Show what messages remain
print(f"\n Remaining message roles:")
for msg in compacted:
role = msg.get("role", "unknown")
content_preview = msg.get("content", "")[:50]
print(f" [{role}] {content_preview}...")
# ==========================================================================
# No Compaction Needed
# ==========================================================================
print("\n--- No Compaction Needed ---")
compactor = ContextCompactor(
max_tokens=10000, # High limit
strategy=CompactionStrategy.TRUNCATE
)
compacted, result = compactor.compact(messages)
print(f" Was compacted: {result.was_compacted}")
print(f" Messages unchanged: {len(compacted) == len(messages)}")
# ==========================================================================
# Result Serialization
# ==========================================================================
print("\n--- Result Serialization ---")
compactor = ContextCompactor(max_tokens=50)
_, result = compactor.compact(messages)
data = result.to_dict()
print(f" Original tokens: {data['original_tokens']}")
print(f" Compacted tokens: {data['compacted_tokens']}")
print(f" Compression ratio: {data['compression_ratio']:.2f}")
print("\n" + "=" * 60)
print("Demo Complete!")
print("=" * 60)
if __name__ == "__main__":
main()
+85
View File
@@ -0,0 +1,85 @@
"""
CLI Compare Example - Advanced Usage
This example demonstrates advanced comparison features including
custom mode configurations and result analysis.
"""
from praisonai.cli.features.compare import (
CompareHandler,
CompareResult,
ModeResult,
get_mode_config,
save_compare_result,
format_comparison_table,
)
# Create custom mode configurations
custom_modes = {
"fast": {"llm": "gpt-4o-mini"},
"accurate": {"llm": "gpt-4o", "planning": True},
"research": {"research": True, "web_search": True},
}
# Create handler with verbose output
handler = CompareHandler(verbose=True)
# Example: Manually create comparison results for analysis
# (useful for testing without API calls)
mock_result = CompareResult(
query="What are the latest AI trends?",
comparisons=[
ModeResult(
mode="basic",
output="AI trends include LLMs, multimodal models, and agents.",
execution_time_ms=1234.5,
model_used="gpt-4o-mini",
tools_used=[]
),
ModeResult(
mode="tools",
output="Based on recent search results, AI trends in 2024 include...",
execution_time_ms=2567.8,
model_used="gpt-4o-mini",
tools_used=["internet_search"]
),
ModeResult(
mode="research",
output="Comprehensive research shows that AI trends...",
execution_time_ms=5432.1,
model_used="gpt-4o-mini",
tools_used=["internet_search", "web_scraper"]
),
]
)
# Get summary statistics
summary = mock_result.get_summary()
print("Comparison Summary:")
print(f" Fastest mode: {summary['fastest']} ({summary['fastest_time_ms']:.1f}ms)")
print(f" Slowest mode: {summary['slowest']} ({summary['slowest_time_ms']:.1f}ms)")
# Format as table
print("\n" + format_comparison_table(mock_result))
# Convert to JSON for storage
print("\nJSON Output:")
print(mock_result.to_json())
# Save to file
# save_compare_result(mock_result, "advanced_comparison.json")
print("\n--- CLI Usage Examples ---")
print("""
# Basic comparison
praisonai "What is AI?" --compare "basic,tools"
# Compare with specific model
praisonai "Explain quantum computing" --compare "basic,planning,research" --model gpt-4o
# Save results to file
praisonai "Latest tech news" --compare "basic,tools,web_search" --compare-output results.json
# Verbose output
praisonai "Write a poem" --compare "basic,planning" --verbose
""")
+45
View File
@@ -0,0 +1,45 @@
"""
CLI Compare Example - Basic Usage
This example demonstrates how to compare different CLI modes programmatically.
"""
from praisonai.cli.features.compare import (
CompareHandler,
get_mode_config,
list_available_modes,
parse_modes,
)
# List all available comparison modes
print("Available modes:")
for mode in list_available_modes():
config = get_mode_config(mode)
print(f" - {mode}: {config}")
# Parse modes from comma-separated string
modes = parse_modes("basic,tools,planning")
print(f"\nParsed modes: {modes}")
# Create compare handler
handler = CompareHandler(verbose=True)
# Compare modes (this will run actual agent calls)
# Uncomment to run:
# result = handler.compare(
# query="What is artificial intelligence?",
# modes=["basic", "tools"],
# model="gpt-4o-mini"
# )
# handler.print_result(result)
# Save results to file
# result = handler.execute(
# query="Explain machine learning",
# modes_str="basic,planning",
# output_path="comparison_results.json"
# )
print("\nTo run comparison via CLI:")
print(' praisonai "What is AI?" --compare "basic,tools,planning"')
print(' praisonai "What is AI?" --compare "basic,research" --compare-output results.json')
@@ -0,0 +1,23 @@
# Example: Data Collection Agent with 30-minute Schedule
# Collects and analyzes data every 30 minutes
framework: praisonai
agents:
- name: "Data Collector"
role: "Data Analysis Specialist"
goal: "Collect and analyze market data at regular intervals"
instructions: |
You are a data analysis specialist focused on collecting and analyzing
market trends and metrics. Provide actionable insights from the data.
tools:
- search_tool
verbose: false
task: "Collect latest market data and identify key trends"
# Schedule: Every 30 minutes with immediate execution
schedule:
interval: "*/30m"
max_retries: 5
run_immediately: true
@@ -0,0 +1,53 @@
# Example: 24/7 AI News Monitor with Scheduler
# This agent runs continuously at regular intervals to monitor AI news
framework: praisonai
agents:
- name: "AI News Monitor"
role: "Technology News Analyst"
goal: "Monitor and summarize the latest AI developments"
instructions: |
You are an expert technology news analyst specializing in AI and machine learning.
Your job is to search for the latest AI news, identify the most important developments,
and provide clear, concise summaries with sources.
Focus on:
- Major AI product launches
- Research breakthroughs
- Industry trends
- Policy and regulation updates
tools:
- search_tool
verbose: true
# Task to execute on each scheduled run
task: |
Search for the latest AI news from the past 24 hours.
Provide the top 3 most important stories with:
1. Headline
2. Brief summary (2-3 sentences)
3. Source URL
4. Why it's significant
# Schedule configuration
schedule:
# Interval options:
# - "hourly" - Every hour
# - "daily" - Every 24 hours
# - "*/30m" - Every 30 minutes
# - "*/6h" - Every 6 hours
# - "3600" - Custom seconds
interval: "hourly"
# Maximum retry attempts if execution fails
max_retries: 3
# Run immediately when scheduler starts (before first scheduled interval)
run_immediately: true
# Optional: Variables for customization
variables:
news_category: "AI and Machine Learning"
region: "Global"
time_range: "24 hours"
+69
View File
@@ -0,0 +1,69 @@
"""
Accuracy Evaluation Example
This example demonstrates how to evaluate agent output accuracy
by comparing against expected outputs using LLM-as-judge.
"""
import os
from praisonaiagents import Agent
from praisonaiagents.eval import AccuracyEvaluator
# Check if we have an API key
has_api_key = os.getenv("OPENAI_API_KEY") is not None
if has_api_key:
# Create a simple math tutor agent
agent = Agent(
instructions="You are a math tutor. Answer math questions concisely with just the number."
)
# Create accuracy evaluator
evaluator = AccuracyEvaluator(
agent=agent,
input_text="What is 2 + 2?",
expected_output="4",
num_iterations=3, # Run 3 times for statistical significance
verbose=True
)
# Run evaluation
result = evaluator.run(print_summary=True)
# Check results
print(f"\nAverage Score: {result.avg_score}/10")
print(f"Passed: {result.passed}")
else:
print("⚠️ No OPENAI_API_KEY found. Using mock evaluation...")
# Create evaluator without agent for mock testing
evaluator = AccuracyEvaluator(
func=lambda x: "4", # Mock function
input_text="What is 2 + 2?",
expected_output="4",
num_iterations=1,
verbose=True
)
# Test with mock output
try:
result = evaluator.run(print_summary=True)
print(f"\nMock Average Score: {result.avg_score}/10")
print(f"Mock Passed: {result.passed}")
except Exception as e:
print(f"Mock evaluation failed: {e}")
print("This is expected without an API key")
# You can also evaluate pre-generated outputs
print("\n--- Testing pre-generated output evaluation ---")
output = "The answer is 4"
if has_api_key:
try:
result2 = evaluator.evaluate_output(output, print_summary=True)
print(f"Pre-generated output score: {result2.avg_score}/10")
except Exception as e:
print(f"Pre-generated evaluation failed: {e}")
else:
print("⚠️ Skipping pre-generated evaluation (no API key)")
print("To run full evaluation, set OPENAI_API_KEY environment variable")
+105
View File
@@ -0,0 +1,105 @@
"""
Batch Evaluation Example
This example demonstrates how to run batch evaluations
from a JSON test file.
"""
import os
import json
import tempfile
from praisonaiagents.eval import AccuracyEvaluator
# Check if we have an API key
has_api_key = os.getenv("OPENAI_API_KEY") is not None
# Create a test file with multiple test cases
test_cases = [
{
"input": "What is 2 + 2?",
"expected": "4"
},
{
"input": "What is the capital of France?",
"expected": "Paris"
},
{
"input": "What color is the sky?",
"expected": "Blue"
}
]
# Save test cases to a temporary file
with tempfile.NamedTemporaryFile(mode='w', suffix='.json', delete=False) as f:
json.dump(test_cases, f)
test_file = f.name
print(f"Test file created: {test_file}")
print(f"Running {len(test_cases)} test cases...")
if has_api_key:
# You can use the CLI for batch evaluation:
# praisonai eval batch --agent agents.yaml --test-file tests.json --batch-type accuracy
# Or programmatically:
from praisonaiagents import Agent
agent = Agent(
instructions="You are a helpful assistant. Answer questions concisely."
)
results = []
for i, test_case in enumerate(test_cases):
print(f"\nTest {i + 1}: {test_case['input']}")
evaluator = AccuracyEvaluator(
agent=agent,
input_text=test_case["input"],
expected_output=test_case["expected"],
num_iterations=1
)
result = evaluator.run()
results.append({
"input": test_case["input"],
"expected": test_case["expected"],
"score": result.avg_score,
"passed": result.passed
})
print(f" Score: {result.avg_score}/10")
print(f" Passed: {result.passed}")
# Summary
passed = sum(1 for r in results if r["passed"])
print(f"\n{'='*50}")
print(f"Summary: {passed}/{len(results)} tests passed")
print(f"Pass Rate: {passed/len(results):.1%}")
else:
print("⚠️ No OPENAI_API_KEY found. Running mock batch evaluation...")
# Mock evaluation without API key
results = []
for i, test_case in enumerate(test_cases):
print(f"\nTest {i + 1}: {test_case['input']}")
# Mock result (all would fail without API key)
mock_score = 0.0
mock_passed = False
results.append({
"input": test_case["input"],
"expected": test_case["expected"],
"score": mock_score,
"passed": mock_passed
})
print(f" Score: {mock_score}/10 (mock)")
print(f" Passed: {mock_passed} (expected without API key)")
# Summary
passed = sum(1 for r in results if r["passed"])
print(f"\n{'='*50}")
print(f"Summary: {passed}/{len(results)} tests passed (mock evaluation)")
print(f"Pass Rate: {passed/len(results):.1%}")
print("\nTo run real batch evaluation, set OPENAI_API_KEY environment variable")
+82
View File
@@ -0,0 +1,82 @@
"""
Criteria Evaluation Example
This example demonstrates how to evaluate agent outputs
against custom criteria using LLM-as-judge.
"""
import os
from praisonaiagents import Agent
from praisonaiagents.eval import CriteriaEvaluator
# Check if we have an API key
has_api_key = os.getenv("OPENAI_API_KEY") is not None
if has_api_key:
print("--- Testing Criteria Evaluation with Agent ---")
# Create a customer service agent
agent = Agent(
instructions="You are a friendly customer service agent. Be helpful and empathetic."
)
# Create criteria evaluator with numeric scoring
evaluator = CriteriaEvaluator(
criteria="Response is helpful, empathetic, and provides a clear solution",
agent=agent,
input_text="I'm frustrated because my order hasn't arrived yet.",
scoring_type="numeric", # Score 1-10
threshold=7.0, # Pass if score >= 7
num_iterations=2,
verbose=True
)
# Run evaluation
result = evaluator.run(print_summary=True)
print("\nNumeric Scoring Results:")
print(f" Average Score: {result.avg_score}/10")
print(f" Pass Rate: {result.pass_rate:.1%}")
print(f" Passed: {result.passed}")
# Binary scoring example (pass/fail)
print("\n--- Testing Binary Scoring ---")
binary_evaluator = CriteriaEvaluator(
criteria="Response does not contain any offensive language",
agent=agent,
input_text="Tell me a joke",
scoring_type="binary", # Pass or Fail
verbose=True
)
binary_result = binary_evaluator.run(print_summary=True)
# With on_fail callback
print("\n--- Testing Failure Callback ---")
def handle_failure(score):
print(f"ALERT: Evaluation failed with score {score.score}")
print(f"Reasoning: {score.reasoning}")
callback_evaluator = CriteriaEvaluator(
criteria="Response is professional and helpful",
agent=agent,
input_text="Help me",
on_fail=handle_failure,
threshold=8.0
)
callback_evaluator.run()
else:
print("⚠️ No OPENAI_API_KEY found. Skipping agent-based criteria evaluation...")
# Test pre-generated output evaluation (doesn't need agent)
print("\n--- Testing Pre-generated Output Evaluation ---")
if has_api_key:
output = "I understand your frustration. Let me check on your order right away."
try:
result2 = evaluator.evaluate_output(output, print_summary=True)
print(f"Pre-generated output score: {result2.avg_score}/10")
except Exception as e:
print(f"Pre-generated evaluation failed: {e}")
else:
print("⚠️ Skipping pre-generated evaluation (no API key)")
print("To run full evaluation, set OPENAI_API_KEY environment variable")
+59
View File
@@ -0,0 +1,59 @@
"""
Performance Evaluation Example
This example demonstrates how to benchmark agent performance
by measuring runtime and memory usage across multiple iterations.
"""
import os
from praisonaiagents import Agent
from praisonaiagents.eval import PerformanceEvaluator
# Check if we have an API key
has_api_key = os.getenv("OPENAI_API_KEY") is not None
if has_api_key:
print("--- Testing Agent Performance ---")
# Create a simple agent
agent = Agent(
instructions="You are a helpful assistant. Answer questions briefly."
)
# Create performance evaluator
evaluator = PerformanceEvaluator(
agent=agent,
input_text="What is the capital of France?",
num_iterations=10, # Run 10 benchmark iterations
warmup_runs=2, # 2 warmup runs before measurement
track_memory=True, # Track memory usage
verbose=True
)
# Run evaluation
result = evaluator.run(print_summary=True)
# Access detailed metrics
print("\nAgent Performance Results:")
print(f" Average Time: {result.avg_run_time:.4f}s")
print(f" Min Time: {result.min_run_time:.4f}s")
print(f" Max Time: {result.max_run_time:.4f}s")
print(f" Median Time: {result.median_run_time:.4f}s")
print(f" P95 Time: {result.p95_run_time:.4f}s")
print(f" Avg Memory: {result.avg_memory:.2f} MB")
else:
print("⚠️ No OPENAI_API_KEY found. Skipping agent performance test...")
# You can also benchmark any function
def my_function():
import time
time.sleep(0.1)
return "done"
func_evaluator = PerformanceEvaluator(
func=my_function,
num_iterations=5,
warmup_runs=1,
track_memory=False
)
func_result = func_evaluator.run(print_summary=True)
+72
View File
@@ -0,0 +1,72 @@
"""
Reliability Evaluation Example
This example demonstrates how to verify that agents call
the expected tools during execution.
"""
import os
from praisonaiagents import Agent
from praisonaiagents.eval import ReliabilityEvaluator
def search_web(query: str) -> str:
"""Search the web for information."""
return f"Search results for: {query}"
def calculate(expression: str) -> str:
"""Calculate a math expression."""
return str(eval(expression))
# Check if we have an API key
has_api_key = os.getenv("OPENAI_API_KEY") is not None
if has_api_key:
print("--- Testing Agent Tool Call Reliability ---")
# Create an agent with tools
agent = Agent(
instructions="You are a helpful assistant with access to search and calculator tools.",
tools=[search_web, calculate]
)
# Create reliability evaluator
evaluator = ReliabilityEvaluator(
agent=agent,
input_text="Search for the weather in Paris and calculate 25 * 4",
expected_tools=["search_web", "calculate"], # Tools that should be called
forbidden_tools=["delete_file"], # Tools that should NOT be called
verbose=True
)
# Run evaluation
result = evaluator.run(print_summary=True)
# Check results
print("\nAgent Reliability Results:")
print(f" Passed: {result.passed}")
print(f" Pass Rate: {result.pass_rate:.1%}")
else:
print("⚠️ No OPENAI_API_KEY found. Skipping agent reliability test...")
print("Agent would fail to call tools without API key (expected behavior)")
# You can also evaluate pre-recorded tool calls (doesn't need API key)
print("\n--- Testing Pre-recorded Tool Call Evaluation ---")
# Create a mock agent for tool call evaluation
mock_agent = Agent(instructions="Mock agent")
evaluator = ReliabilityEvaluator(
agent=mock_agent,
expected_tools=["search_web", "calculate"], # Tools that should be called
forbidden_tools=["delete_file"], # Tools that should NOT be called
verbose=True
)
result2 = evaluator.evaluate_tool_calls(
actual_tools=["search_web", "calculate"],
print_summary=True
)
print("\nPre-recorded Tool Call Results:")
print(f" Passed: {result2.passed}")
print(f" Pass Rate: {result2.pass_rate:.1%}")
+142
View File
@@ -0,0 +1,142 @@
#!/usr/bin/env python3
"""
Basic Hooks Example for PraisonAI Agents.
This example demonstrates how to use the hooks system to:
1. Log tool calls before execution
2. Block dangerous operations
3. Add context to agent responses
4. Monitor session lifecycle
Usage:
python basic_hooks.py
"""
import asyncio
from praisonaiagents import Agent
from praisonaiagents.hooks import (
HookRegistry, HookRunner, HookEvent, HookResult,
BeforeToolInput, AfterToolInput
)
def main():
# Create a hook registry
registry = HookRegistry()
# ==========================================================================
# Hook 1: Log all tool calls
# ==========================================================================
@registry.on(HookEvent.BEFORE_TOOL)
def log_tool_calls(event_data: BeforeToolInput) -> HookResult:
"""Log every tool call before execution."""
print(f"\n📝 [LOG] Tool: {event_data.tool_name}")
print(f" Input: {event_data.tool_input}")
return HookResult.allow()
# ==========================================================================
# Hook 2: Block dangerous file operations (using function logic)
# ==========================================================================
@registry.on(HookEvent.BEFORE_TOOL)
def block_delete_operations(event_data: BeforeToolInput) -> HookResult:
"""Block any delete or remove operations."""
dangerous_keywords = ['delete', 'remove', 'rm', 'unlink']
tool_lower = event_data.tool_name.lower()
if any(kw in tool_lower for kw in dangerous_keywords):
print(f"\n🚫 [BLOCKED] Dangerous operation: {event_data.tool_name}")
return HookResult.deny(
f"Operation '{event_data.tool_name}' is blocked by security policy"
)
return HookResult.allow()
# ==========================================================================
# Hook 3: Require confirmation for write operations
# ==========================================================================
@registry.on(HookEvent.BEFORE_TOOL)
def confirm_write_operations(event_data: BeforeToolInput) -> HookResult:
"""Request confirmation for write operations."""
write_keywords = ['write', 'save', 'create', 'update']
tool_lower = event_data.tool_name.lower()
if any(kw in tool_lower for kw in write_keywords):
print(f"\n⚠️ [CONFIRM] Write operation: {event_data.tool_name}")
# In a real scenario, you might prompt the user here
# For this example, we'll auto-approve
return HookResult.allow("Auto-approved for demo")
return HookResult.allow()
# ==========================================================================
# Hook 4: Add timing information after tool execution
# ==========================================================================
@registry.on(HookEvent.AFTER_TOOL)
def log_tool_completion(event_data: AfterToolInput) -> HookResult:
"""Log tool completion with timing."""
print(f"\n✅ [DONE] Tool: {event_data.tool_name}")
print(f" Duration: {event_data.execution_time_ms:.2f}ms")
if event_data.tool_error:
print(f" Error: {event_data.tool_error}")
return HookResult.allow()
# ==========================================================================
# Test the hooks
# ==========================================================================
print("=" * 60)
print("Testing Hooks System")
print("=" * 60)
runner = HookRunner(registry)
# Test 1: Normal tool call (should be logged and allowed)
print("\n--- Test 1: Normal read operation ---")
input1 = BeforeToolInput(
session_id="test",
cwd="/tmp",
event_name="before_tool",
timestamp="2024-01-01T00:00:00",
tool_name="read_file",
tool_input={"path": "/tmp/test.txt"}
)
results = asyncio.run(runner.execute(HookEvent.BEFORE_TOOL, input1))
print(f"Result: {'ALLOWED' if not HookRunner.is_blocked(results) else 'BLOCKED'}")
# Test 2: Delete operation (should be blocked)
print("\n--- Test 2: Delete operation ---")
input2 = BeforeToolInput(
session_id="test",
cwd="/tmp",
event_name="before_tool",
timestamp="2024-01-01T00:00:00",
tool_name="delete_file",
tool_input={"path": "/important/data.txt"}
)
results = asyncio.run(runner.execute(HookEvent.BEFORE_TOOL, input2))
print(f"Result: {'ALLOWED' if not HookRunner.is_blocked(results) else 'BLOCKED'}")
if HookRunner.is_blocked(results):
print(f"Reason: {HookRunner.get_blocking_reason(results)}")
# Test 3: Write operation (should be confirmed)
print("\n--- Test 3: Write operation ---")
input3 = BeforeToolInput(
session_id="test",
cwd="/tmp",
event_name="before_tool",
timestamp="2024-01-01T00:00:00",
tool_name="write_file",
tool_input={"path": "/tmp/output.txt", "content": "Hello"}
)
results = asyncio.run(runner.execute(HookEvent.BEFORE_TOOL, input3))
print(f"Result: {'ALLOWED' if not HookRunner.is_blocked(results) else 'BLOCKED'}")
# Show registered hooks
print("\n" + "=" * 60)
print("Registered Hooks:")
print("=" * 60)
for event, hooks in registry.list_hooks().items():
print(f"\n{event}:")
for hook in hooks:
print(f" - {hook['name']} (matcher: {hook['matcher'] or '*'})")
if __name__ == "__main__":
main()
+79
View File
@@ -0,0 +1,79 @@
"""
Qdrant Knowledge Store Example for PraisonAI.
Demonstrates vector insert + search through the knowledge store.
Requirements:
pip install "praisonai[tools]"
Docker Setup:
docker run -d --name praison-qdrant -p 6333:6333 qdrant/qdrant
Run:
python knowledge_qdrant.py
Expected Output:
=== Qdrant Knowledge Store Demo ===
Created collection: demo_knowledge
Inserted 3 documents
Search results: 2 documents found
"""
from praisonai.persistence.factory import create_knowledge_store
from praisonai.persistence.knowledge.base import KnowledgeDocument
import random
# Create Qdrant knowledge store
store = create_knowledge_store("qdrant", url="http://localhost:6333")
collection_name = "demo_knowledge"
dimension = 384 # Embedding dimension
# Create collection
print("=== Qdrant Knowledge Store Demo ===")
if store.collection_exists(collection_name):
store.delete_collection(collection_name)
store.create_collection(collection_name, dimension=dimension)
print(f"Created collection: {collection_name}")
# Create sample documents with embeddings
random.seed(42)
documents = [
KnowledgeDocument(
id="doc-1",
content="Python is a versatile programming language used for web development, data science, and AI.",
embedding=[random.random() for _ in range(dimension)],
metadata={"category": "programming", "language": "python"}
),
KnowledgeDocument(
id="doc-2",
content="Machine learning enables computers to learn patterns from data without explicit programming.",
embedding=[random.random() for _ in range(dimension)],
metadata={"category": "ai", "topic": "ml"}
),
KnowledgeDocument(
id="doc-3",
content="PostgreSQL is a powerful open-source relational database system.",
embedding=[random.random() for _ in range(dimension)],
metadata={"category": "database", "type": "sql"}
),
]
# Insert documents
ids = store.insert(collection_name, documents)
print(f"Inserted {len(ids)} documents")
# Search for similar documents
query_embedding = [random.random() for _ in range(dimension)]
results = store.search(collection_name, query_embedding, limit=2)
print(f"Search results: {len(results)} documents found")
for doc in results:
print(f" - {doc.id}: {doc.content[:50]}...")
# Cleanup
store.delete_collection(collection_name)
store.close()
print("\n=== Demo Complete ===")
+43
View File
@@ -0,0 +1,43 @@
"""
Minimal Agent with Database Persistence
This is the simplest way to add persistence to your agent.
Messages, runs, and traces are automatically saved.
Run:
python minimal_agent_db.py
Expected output:
- Agent responds to your question
- Data is persisted to PostgreSQL
- Session can be resumed later with same session_id
"""
from praisonaiagents import Agent, db
# Create database connection (PostgreSQL + Redis)
my_db = db(
database_url="postgresql://postgres:praison123@localhost:5432/praisonai",
state_url="redis://localhost:6379" # Optional: for state/tracing
)
# Create agent with persistence
agent = Agent(
name="Assistant",
instructions="You are a helpful assistant. Be concise.",
db=my_db,
# session_id is optional - defaults to per-hour ID (YYYYMMDDHH-hash)
# Set explicitly for session continuity across runs:
session_id="my-persistent-session"
)
# Chat - messages are automatically persisted
response = agent.chat("What is the capital of France?")
print(f"Response: {response}")
# Verify persistence
data = my_db.export_session("my-persistent-session")
print(f"\nMessages stored: {len(data.get('messages', []))}")
# Clean up
my_db.close()
@@ -0,0 +1,61 @@
"""
PostgreSQL with Runs and Traces
Demonstrates full persistence with run tracking and tracing.
Run:
python postgres_runs_traces.py
Expected output:
- Agent responds with tool usage
- Runs and traces are persisted
- Export shows full session data
"""
from praisonaiagents import Agent, db
def get_weather(city: str) -> str:
"""Get weather for a city."""
return f"Weather in {city}: Sunny, 72°F"
# Create database with PostgreSQL + Redis
my_db = db(
database_url="postgresql://postgres:praison123@localhost:5432/praisonai",
state_url="redis://localhost:6379"
)
# Create agent with tools
agent = Agent(
name="WeatherBot",
instructions="You help with weather. Use the get_weather tool when asked.",
tools=[get_weather],
db=my_db,
session_id="weather-session-001"
)
# First chat - triggers run tracking
print("=== First Question ===")
response1 = agent.chat("What's the weather in Tokyo?")
print(f"Response: {response1}\n")
# Second chat - same session, new run
print("=== Second Question ===")
response2 = agent.chat("How about New York?")
print(f"Response: {response2}\n")
# Export session data
print("=== Session Data ===")
data = my_db.export_session("weather-session-001")
print(f"Session ID: {data.get('session_id')}")
print(f"Messages: {len(data.get('messages', []))}")
print(f"Agent: {data.get('agent_name')}")
# Show messages
print("\n=== Messages ===")
for msg in data.get('messages', []):
role = msg.get('role', 'unknown')
content = msg.get('content', '')[:60]
print(f" [{role}] {content}...")
my_db.close()
print("\n✅ Done")
+83
View File
@@ -0,0 +1,83 @@
"""
Redis State Store
Use Redis for state management, tracing, and caching.
Run:
python redis_state.py
Expected output:
- State operations (set/get/delete)
- Tracing operations
"""
from praisonaiagents import db
# Create Redis-only database (for state/tracing)
redis_db = db.RedisDB(host="localhost", port=6379)
print("=== Redis State Store ===")
# Test state operations
print("\n1. State Operations:")
redis_db._init_stores() # Initialize stores
# Set a value
redis_db._state_store.set("user:123:preferences", {
"theme": "dark",
"language": "en"
})
print(" Set user preferences")
# Get the value
prefs = redis_db._state_store.get("user:123:preferences")
print(f" Got preferences: {prefs}")
# Set with TTL (expires in 60 seconds)
redis_db._state_store.set("temp:token", "abc123", ttl=60)
print(" Set temporary token (60s TTL)")
# Delete
redis_db._state_store.delete("user:123:preferences")
print(" Deleted preferences")
# Test tracing
print("\n2. Tracing Operations:")
# Start a trace
trace_id = "trace-example-001"
redis_db.on_trace_start(
trace_id=trace_id,
session_id="example-session",
agent_name="ExampleAgent",
metadata={"purpose": "demo"}
)
print(f" Started trace: {trace_id}")
# Add spans
span_id = "span-llm-001"
redis_db.on_span_start(
span_id=span_id,
trace_id=trace_id,
name="llm_call",
attributes={"model": "gpt-4", "temperature": 0.7}
)
print(f" Started span: {span_id}")
redis_db.on_span_end(
span_id=span_id,
status="ok",
attributes={"tokens": 150, "latency_ms": 450}
)
print(f" Ended span: {span_id}")
# End trace
redis_db.on_trace_end(
trace_id=trace_id,
status="ok",
metadata={"total_spans": 1}
)
print(f" Ended trace: {trace_id}")
redis_db.close()
print("\n✅ Done")
+66
View File
@@ -0,0 +1,66 @@
"""
Session Resume Example
Demonstrates how to resume a conversation from a previous session.
Run:
python session_resume.py
Expected output:
- First run: Creates new session
- Second run: Resumes with history
"""
from praisonaiagents import Agent, db
# Create database
my_db = db(
database_url="postgresql://postgres:praison123@localhost:5432/praisonai",
state_url="redis://localhost:6379"
)
SESSION_ID = "resume-demo-session"
# Create agent with explicit session_id
agent = Agent(
name="MemoryBot",
instructions="You remember our conversation. Be helpful and concise.",
db=my_db,
session_id=SESSION_ID
)
# Check if this is a new or resumed session
data = my_db.export_session(SESSION_ID)
existing_messages = len(data.get('messages', []))
if existing_messages == 0:
print("=== NEW SESSION ===")
print("This is a new conversation.\n")
response = agent.chat("My name is Alice and I love Python programming.")
print(f"Response: {response}\n")
response = agent.chat("What's my favorite programming language?")
print(f"Response: {response}\n")
else:
print(f"=== RESUMED SESSION ({existing_messages} previous messages) ===")
print("Continuing previous conversation.\n")
# Show previous messages
print("Previous messages:")
for msg in data.get('messages', [])[:4]:
role = msg.get('role', 'unknown')
content = msg.get('content', '')[:60]
print(f" [{role}] {content}...")
print()
# Continue conversation
response = agent.chat("What's my name and what do I like?")
print(f"Response: {response}\n")
# Show final state
final_data = my_db.export_session(SESSION_ID)
print(f"Total messages now: {len(final_data.get('messages', []))}")
my_db.close()
print("✅ Done")
+63
View File
@@ -0,0 +1,63 @@
"""
Simple Database Persistence Example for PraisonAI Agents.
This example shows how to enable automatic conversation persistence
with just 2 lines of code.
Requirements:
pip install "praisonai[tools]"
Docker Setup:
docker run -d --name praison-postgres -p 5432:5432 \
-e POSTGRES_PASSWORD=praison123 \
-e POSTGRES_DB=praisonai \
postgres:16
Environment:
export OPENAI_API_KEY=your-key
Run:
python simple_agent_postgres.py
Expected Output:
=== First Run ===
Agent: Nice to meet you, Alice! Python is a great choice...
Session ID: demo-session-001
Messages persisted: 2
"""
from praisonaiagents import Agent
from praisonai.db import PostgresDB
# Create database adapter
db = PostgresDB(
host="localhost",
port=5432,
database="praisonai",
user="postgres",
password="praison123"
)
# Create agent with persistence enabled
agent = Agent(
name="Assistant",
instructions="You are a helpful assistant. Keep responses brief.",
db=db,
session_id="demo-session-001", # Same session_id = resume conversation
verbose=False
)
# Chat with persistence
print("=== First Run ===")
response = agent.chat("My name is Alice and I love Python programming.")
print(f"Agent: {response}\n")
# Show session info
print(f"Session ID: {agent.session_id}")
print(f"Messages in history: {len(agent.chat_history)}")
# Close DB connection
db.close()
print("\n=== Session persisted! ===")
print("Run again with same session_id to resume.")
+41
View File
@@ -0,0 +1,41 @@
"""
SQLite Local Database
Use SQLite for simple local persistence without external services.
Run:
python sqlite_local.py
Expected output:
- Agent responds
- Data persisted to local SQLite file
"""
from praisonaiagents import Agent, db
import tempfile
import os
# Create SQLite database (local file)
db_path = os.path.join(tempfile.gettempdir(), "my_agent.db")
my_db = db.SQLiteDB(path=db_path)
print(f"Database: {db_path}")
# Create agent
agent = Agent(
name="LocalBot",
instructions="You are a helpful assistant.",
db=my_db,
session_id="local-session"
)
# Chat
response = agent.chat("Hello! What can you help me with?")
print(f"Response: {response}")
# Verify
data = my_db.export_session("local-session")
print(f"\nMessages stored: {len(data.get('messages', []))}")
my_db.close()
print("✅ Done")
+67
View File
@@ -0,0 +1,67 @@
"""
Redis State Store Example for PraisonAI.
Demonstrates set/get/delete + TTL operations.
Requirements:
pip install "praisonai[tools]"
Docker Setup:
docker run -d --name praison-redis -p 6379:6379 redis:7
Run:
python state_redis.py
Expected Output:
=== Redis State Store Demo ===
Set key: session:user123:preferences
Get value: {'theme': 'dark', 'language': 'en'}
TTL remaining: 3600 seconds
Hash operations: {'name': 'Alice', 'role': 'admin'}
"""
from praisonai.persistence.factory import create_state_store
# Create Redis state store
store = create_state_store("redis", url="redis://localhost:6379")
print("=== Redis State Store Demo ===")
# Basic set/get
key = "session:user123:preferences"
value = {"theme": "dark", "language": "en", "notifications": True}
store.set(key, value)
print(f"Set key: {key}")
retrieved = store.get(key)
print(f"Get value: {retrieved}")
# Set with TTL (expires in 1 hour)
ttl_key = "session:user123:token"
store.set(ttl_key, "abc123xyz", ttl=3600)
remaining = store.ttl(ttl_key)
print(f"TTL remaining: {remaining} seconds")
# Hash operations (for structured data)
hash_key = "user:123:profile"
store.hset(hash_key, "name", "Alice")
store.hset(hash_key, "role", "admin")
store.hset(hash_key, "last_login", "2024-12-24")
profile = store.hgetall(hash_key)
print(f"Hash operations: {profile}")
# Check existence
exists = store.exists(key)
print(f"Key exists: {exists}")
# Delete keys
store.delete(key)
store.delete(ttl_key)
store.delete(hash_key)
print("Cleaned up test keys")
store.close()
print("\n=== Demo Complete ===")
+142
View File
@@ -0,0 +1,142 @@
#!/usr/bin/env python3
"""
Basic Policy Engine Example for PraisonAI Agents.
This example demonstrates how to use the policy engine to:
1. Create policies with rules
2. Check if actions are allowed
3. Block dangerous operations
4. Use convenience policy creators
Usage:
python basic_policy.py
"""
from praisonaiagents.policy import (
PolicyEngine, Policy, PolicyRule, PolicyAction, PolicyConfig,
create_deny_tools_policy, create_read_only_policy
)
def main():
print("=" * 60)
print("Policy Engine Demo")
print("=" * 60)
# Create a policy engine
engine = PolicyEngine()
# ==========================================================================
# Policy 1: Block dangerous tools
# ==========================================================================
print("\n--- Creating Policies ---")
dangerous_tools_policy = Policy(
name="block_dangerous",
description="Block dangerous file operations",
rules=[
PolicyRule(
action=PolicyAction.DENY,
resource="tool:delete_*",
reason="Delete operations are blocked",
name="deny_delete"
),
PolicyRule(
action=PolicyAction.DENY,
resource="tool:rm_*",
reason="Remove operations are blocked",
name="deny_rm"
),
],
priority=100
)
engine.add_policy(dangerous_tools_policy)
print(f"✅ Added policy: {dangerous_tools_policy.name}")
# ==========================================================================
# Policy 2: Allow read operations
# ==========================================================================
read_policy = Policy(
name="allow_read",
description="Allow read operations",
rules=[
PolicyRule(
action=PolicyAction.ALLOW,
resource="tool:read_*",
name="allow_read"
),
PolicyRule(
action=PolicyAction.ALLOW,
resource="tool:list_*",
name="allow_list"
),
]
)
engine.add_policy(read_policy)
print(f"✅ Added policy: {read_policy.name}")
# ==========================================================================
# Test policy checks
# ==========================================================================
print("\n--- Testing Policy Checks ---")
test_cases = [
("tool:read_file", "Reading a file"),
("tool:list_directory", "Listing directory"),
("tool:delete_file", "Deleting a file"),
("tool:rm_directory", "Removing directory"),
("tool:write_file", "Writing a file"),
]
for resource, description in test_cases:
result = engine.check(resource, {})
status = "✅ ALLOWED" if result.allowed else "🚫 DENIED"
reason = f" ({result.reason})" if result.reason else ""
print(f" {status}: {description}{reason}")
# ==========================================================================
# Using convenience functions
# ==========================================================================
print("\n--- Using Convenience Functions ---")
# Create a read-only policy
read_only = create_read_only_policy()
print(f"✅ Created read-only policy with {len(read_only.rules)} rules")
# Create a deny tools policy
deny_tools = create_deny_tools_policy(
["execute_*", "shell_*", "system_*"],
reason="System commands are blocked"
)
print(f"✅ Created deny-tools policy with {len(deny_tools.rules)} rules")
# ==========================================================================
# Strict mode example
# ==========================================================================
print("\n--- Strict Mode Example ---")
strict_engine = PolicyEngine(PolicyConfig(strict_mode=True))
# In strict mode, unknown resources are denied
result = strict_engine.check("tool:unknown_operation", {})
print(f" Unknown operation in strict mode: {'DENIED' if not result.allowed else 'ALLOWED'}")
# ==========================================================================
# Policy serialization
# ==========================================================================
print("\n--- Policy Serialization ---")
policy_dict = dangerous_tools_policy.to_dict()
print(f" Serialized policy: {policy_dict['name']} with {len(policy_dict['rules'])} rules")
# Recreate from dict
restored = Policy.from_dict(policy_dict)
print(f" Restored policy: {restored.name}")
print("\n" + "=" * 60)
print("Demo Complete!")
print("=" * 60)
if __name__ == "__main__":
main()
+53
View File
@@ -0,0 +1,53 @@
"""
PraisonAI A2A Server Example
Expose a PraisonAI Agent as an A2A (Agent2Agent) Server.
This enables other AI agents to discover and communicate with your agent.
Run:
uvicorn a2a-server:app --reload
Endpoints:
GET /.well-known/agent.json - Agent Card for discovery
GET /status - Server status
"""
from praisonaiagents import Agent, A2A
from fastapi import FastAPI
# Create an agent with tools
def search_web(query: str) -> str:
"""Search the web for information."""
return f"Search results for: {query}"
def calculate(expression: str) -> str:
"""Calculate a mathematical expression."""
try:
return f"Result: {eval(expression)}"
except Exception:
return "Invalid expression"
agent = Agent(
name="Research Assistant",
role="Research Analyst",
goal="Help users research topics and answer questions",
tools=[search_web, calculate]
)
# Expose as A2A Server
a2a = A2A(
agent=agent,
url="http://localhost:8000/a2a",
version="1.0.0"
)
# Create FastAPI app
app = FastAPI(
title="PraisonAI A2A Server",
description="A2A-compatible agent server"
)
app.include_router(a2a.get_router())
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8000)
@@ -0,0 +1,132 @@
"""
AutoAgents Workflow Patterns Example
This example demonstrates the new workflow auto-generation features:
- Dynamic agent count based on task complexity
- Multiple workflow patterns (sequential, parallel, routing, orchestrator-workers, evaluator-optimizer)
- Pattern recommendation based on task keywords
Documentation: https://docs.praison.ai/features/autoagents
CLI Reference: https://docs.praison.ai/nocode/auto
"""
from praisonai.auto import AutoGenerator, WorkflowAutoGenerator
import os
import tempfile
# Ensure API key is set
if not os.environ.get("OPENAI_API_KEY"):
print("Please set OPENAI_API_KEY environment variable")
exit(1)
print("=" * 60)
print("AutoAgents Workflow Patterns Demo")
print("=" * 60)
# =============================================================================
# Example 1: Dynamic Agent Count (Simple vs Complex Tasks)
# =============================================================================
print("\n📊 Example 1: Dynamic Agent Count")
print("-" * 40)
# Simple task - should create 1-2 agents
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
simple_file = f.name
simple_generator = AutoGenerator(
topic="Write a haiku about spring",
agent_file=simple_file,
framework="praisonai"
)
print("Simple task: 'Write a haiku about spring'")
result = simple_generator.generate()
print(f"Generated: {result}")
# Complex task - should create 3-4 agents
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
complex_file = f.name
complex_generator = AutoGenerator(
topic="Research AI trends, analyze market data, write a comprehensive report with executive summary",
agent_file=complex_file,
framework="praisonai"
)
print("\nComplex task: 'Research AI trends, analyze market data...'")
result = complex_generator.generate()
print(f"Generated: {result}")
# =============================================================================
# Example 2: Pattern Recommendation
# =============================================================================
print("\n🎯 Example 2: Pattern Recommendation")
print("-" * 40)
test_tasks = [
("Write a blog post", "sequential"),
("Research from multiple sources simultaneously", "parallel"),
("Classify and route customer requests", "routing"),
("Comprehensive analysis and break down the problem", "orchestrator-workers"),
("Refine and improve content quality", "evaluator-optimizer"),
]
for task, expected in test_tasks:
generator = WorkflowAutoGenerator(topic=task)
recommended = generator.recommend_pattern()
status = "✅" if recommended == expected else "⚠️"
print(f"{status} Task: '{task[:40]}...'")
print(f" Recommended: {recommended}")
# =============================================================================
# Example 3: Generate Workflow with Specific Pattern
# =============================================================================
print("\n🔄 Example 3: Generate Workflows with Patterns")
print("-" * 40)
patterns = ["sequential", "parallel", "orchestrator-workers", "evaluator-optimizer"]
for pattern in patterns:
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
workflow_file = f.name
generator = WorkflowAutoGenerator(
topic="Create a market analysis report",
workflow_file=workflow_file
)
print(f"\n📋 Pattern: {pattern}")
result = generator.generate(pattern=pattern)
print(f" Generated: {result}")
# Read and show structure
import yaml
with open(result, 'r') as f:
data = yaml.safe_load(f)
print(f" Agents: {list(data.get('agents', {}).keys())}")
print(f" Steps: {len(data.get('steps', []))}")
# =============================================================================
# Example 4: CLI Usage (for reference)
# =============================================================================
print("\n💻 Example 4: CLI Commands (for reference)")
print("-" * 40)
print("""
# Auto-generate agents (dynamic count)
praisonai --auto "Write a haiku about spring"
# Auto-generate workflow with pattern
praisonai workflow auto "Research and write a report" --pattern sequential
praisonai workflow auto "Research from multiple sources" --pattern parallel
praisonai workflow auto "Comprehensive market analysis" --pattern orchestrator-workers
praisonai workflow auto "Refine content quality" --pattern evaluator-optimizer
# With output file
praisonai workflow auto "Customer support routing" --pattern routing --output support.yaml
""")
print("\n" + "=" * 60)
print("✅ AutoAgents Workflow Patterns Demo Complete!")
print("=" * 60)
print("\nDocumentation: https://docs.praison.ai/features/autoagents")
print("CLI Reference: https://docs.praison.ai/nocode/auto")
@@ -0,0 +1,38 @@
"""
Autonomy Modes Example for PraisonAI CLI.
Control AI autonomy: suggest, auto_edit, full_auto.
Docs: https://docs.praison.ai/cli/autonomy-modes
"""
from praisonai.cli.features import AutonomyModeHandler
from praisonai.cli.features.autonomy_mode import ActionRequest, ActionType
# Initialize with suggest mode (safest)
handler = AutonomyModeHandler()
manager = handler.initialize(mode="suggest")
print(f"Current mode: {handler.get_mode()}")
print("Available modes: suggest, auto_edit, full_auto")
# Create an action request
action = ActionRequest(
action_type=ActionType.FILE_WRITE,
description="Edit src/main.py to add logging",
details={"file": "src/main.py"}
)
# In suggest mode, this would require approval
# For demo, we'll just show the action
print(f"\nAction: {action.description}")
print(f"Type: {action.action_type.value}")
print("Would require approval in 'suggest' mode")
# Change to auto_edit mode
handler.set_mode("auto_edit")
print(f"\nChanged to: {handler.get_mode()}")
print("File edits now auto-approved, shell commands still require approval")
# Get statistics
stats = handler.get_stats()
print(f"\nStats: {stats}")
@@ -0,0 +1,34 @@
"""
Cost Tracking Example for PraisonAI CLI.
Real-time token usage and cost monitoring.
Docs: https://docs.praison.ai/cli/cost-tracking
"""
from praisonai.cli.features import CostTrackerHandler
# Initialize tracker
handler = CostTrackerHandler()
tracker = handler.initialize(session_id="demo-session")
# Track some requests
handler.track_request("gpt-4o", input_tokens=1000, output_tokens=500)
handler.track_request("gpt-4o-mini", input_tokens=2000, output_tokens=800)
handler.track_request("claude-3-5-sonnet", input_tokens=1500, output_tokens=600)
# Get totals
print(f"Total tokens: {handler.get_tokens():,}")
print(f"Total cost: ${handler.get_cost():.4f}")
# Get detailed summary
summary = handler.get_summary()
print("\n=== Session Summary ===")
print(f"Session ID: {summary['session_id']}")
print(f"Total requests: {summary['total_requests']}")
print(f"Input tokens: {summary['total_input_tokens']:,}")
print(f"Output tokens: {summary['total_output_tokens']:,}")
print(f"Total cost: ${summary['total_cost']:.4f}")
print(f"Avg cost/request: ${summary['avg_cost_per_request']:.4f}")
# Display formatted (uses Rich if available)
handler.display_summary()
@@ -0,0 +1,45 @@
"""
Git Integration Example for PraisonAI CLI.
Auto-commit with AI messages, diff viewing, undo.
Docs: https://docs.praison.ai/cli/git-integration
"""
from praisonai.cli.features import GitIntegrationHandler
# Initialize
handler = GitIntegrationHandler(verbose=True)
git = handler.initialize(repo_path=".")
# Check if it's a git repo
if not git.is_repo:
print("Not a git repository!")
exit(1)
# Show status
print("=== Git Status ===")
status = handler.show_status()
print(f"Branch: {status.branch}")
print(f"Staged: {len(status.staged_files)}")
print(f"Modified: {len(status.modified_files)}")
print(f"Untracked: {len(status.untracked_files)}")
# Show diff (if any changes)
if status.has_changes:
print("\n=== Diff ===")
diff = handler.show_diff()
if diff:
print(diff[:300] + "..." if len(diff) > 300 else diff)
# Show recent commits
print("\n=== Recent Commits ===")
commits = handler.show_log(count=5)
for c in commits:
print(f" {c.short_hash} {c.message[:50]}")
# To commit with AI message:
# commit = handler.commit() # Auto-generates message
# commit = handler.commit(message="Custom message")
# To undo last commit:
# handler.undo(soft=True) # Keeps changes staged
@@ -0,0 +1,49 @@
"""
Interactive TUI Example for PraisonAI CLI.
Rich terminal interface with completions and history.
Docs: https://docs.praison.ai/cli/interactive-tui
"""
from praisonai.cli.features import InteractiveTUIHandler
from praisonai.cli.features.interactive_tui import InteractiveConfig
# Custom configuration
config = InteractiveConfig(
prompt="ai> ",
multiline=True,
enable_completions=True,
show_status_bar=True
)
# Define callbacks
def on_input(text):
"""Handle regular input."""
return f"You said: {text}"
def on_command(cmd):
"""Handle slash commands."""
if cmd == "/exit":
return {"type": "exit"}
return {"type": "command", "message": f"Executed: {cmd}"}
# Initialize
handler = InteractiveTUIHandler()
session = handler.initialize(
config=config,
on_input=on_input,
on_command=on_command
)
# Add commands for completion
session.add_commands(["help", "exit", "cost", "model", "plan"])
# Add symbols from codebase
session.add_symbols(["MyClass", "my_function"])
print("Interactive TUI initialized!")
print("Commands available: /help, /exit, /cost, /model, /plan")
print("Run handler.run() to start interactive session")
# Uncomment to run interactive session:
# handler.run()
+24
View File
@@ -0,0 +1,24 @@
"""
Repository Map Example for PraisonAI CLI.
Intelligent codebase mapping with symbol extraction.
Docs: https://docs.praison.ai/cli/repo-map
"""
from praisonai.cli.features import RepoMapHandler
# Initialize with current directory
handler = RepoMapHandler(verbose=True)
repo_map = handler.initialize(root=".")
# Get the repository map
print("=== Repository Map ===")
map_str = handler.get_map()
print(map_str[:500] if len(map_str) > 500 else map_str)
print("..." if len(map_str) > 500 else "")
# Refresh after file changes
handler.refresh()
# Get symbols for a specific file (if exists)
# symbols = handler.get_context("MyClass")
@@ -0,0 +1,42 @@
"""
Sandbox Execution Example for PraisonAI CLI.
Secure isolated command execution (--sandbox flag).
Docs: https://docs.praison.ai/cli/sandbox-execution
"""
from praisonai.cli.features import SandboxExecutorHandler
from praisonai.cli.features.sandbox_executor import SandboxMode
# Initialize with basic sandbox
handler = SandboxExecutorHandler(verbose=True)
sandbox = handler.initialize(mode="basic")
print(f"Sandbox mode: {handler.get_mode()}")
print(f"Sandbox enabled: {handler.is_enabled}")
# Execute a safe command
print("\n=== Safe Command ===")
result = handler.execute("echo 'Hello from sandbox!'")
print(f"Success: {result.success}")
print(f"Output: {result.stdout.strip()}")
print(f"Sandboxed: {result.was_sandboxed}")
# Try a blocked command
print("\n=== Blocked Command ===")
result = handler.execute("rm -rf /")
print(f"Success: {result.success}")
print(f"Violations: {result.policy_violations}")
# Validate command before execution
print("\n=== Command Validation ===")
violations = handler.validate_command("sudo apt install something")
if violations:
print(f"Command blocked: {violations}")
else:
print("Command allowed")
# Available modes
print("\n=== Available Modes ===")
for mode in SandboxMode:
print(f" {mode.value}")
@@ -0,0 +1,51 @@
"""
Slash Commands Example for PraisonAI CLI.
Demonstrates interactive slash commands like /help, /cost, /model.
Docs: https://docs.praison.ai/cli/slash-commands
"""
from praisonai.cli.features import SlashCommandHandler
from praisonai.cli.features.slash_commands import SlashCommand, CommandKind
# Create handler
handler = SlashCommandHandler()
# Execute commands
print("=== /help ===")
result = handler.execute("/help")
print(result.get("message", ""))
print("\n=== /cost ===")
result = handler.execute("/cost")
print(result.get("message", ""))
print("\n=== /model ===")
result = handler.execute("/model")
print(result.get("message", ""))
# Check if input is a command
print("\n=== Command Detection ===")
print(f"/help is command: {handler.is_command('/help')}")
print(f"hello is command: {handler.is_command('hello')}")
# Get completions for auto-complete
print("\n=== Completions for '/he' ===")
completions = handler.get_completions("/he")
print(completions)
# Register custom command
def my_custom_handler(args, context):
return {"type": "custom", "message": f"Custom command with args: {args}"}
custom_cmd = SlashCommand(
name="mycommand",
description="My custom command",
handler=my_custom_handler,
kind=CommandKind.ACTION,
aliases=["mc"]
)
handler.register(custom_cmd)
result = handler.execute("/mycommand arg1 arg2")
print(f"\n=== Custom Command ===\n{result.get('message', '')}")
@@ -0,0 +1,34 @@
"""
Claude Code CLI Integration Example
Demonstrates how to use Claude Code CLI as an agent tool in PraisonAI.
"""
import asyncio
from praisonai.integrations import ClaudeCodeIntegration
async def main():
# Create Claude Code integration
claude = ClaudeCodeIntegration(
workspace=".",
output_format="json",
skip_permissions=True,
system_prompt="Be concise and follow best practices",
allowed_tools=["Read", "Write", "Bash"]
)
# Check availability
print(f"Claude CLI available: {claude.is_available}")
print(f"Claude SDK available: {claude.sdk_available}")
# Execute a coding task
result = await claude.execute("List all Python files in the current directory")
print(f"Result: {result}")
# Stream output
print("\nStreaming output:")
async for event in claude.stream("Explain the project structure"):
print(event)
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,60 @@
"""
Code Editing Tools Example
Demonstrates how to use PraisonAI's code editing tools with SEARCH/REPLACE diffs.
"""
from praisonai.code import (
set_workspace,
code_read_file,
code_write_file,
code_apply_diff,
code_list_files,
code_execute_command,
CODE_TOOLS
)
# Set workspace
set_workspace(".")
# List files
print("Files in current directory:")
files = code_list_files(".", recursive=False)
for f in files:
print(f" {f}")
# Read a file
print("\nReading example file:")
try:
content = code_read_file("README.md", start_line=1, end_line=10)
print(content)
except Exception as e:
print(f" File not found: {e}")
# Write a file
print("\nWriting test file:")
code_write_file("test_output.txt", "Hello from PraisonAI Code Tools!")
print(" Created test_output.txt")
# Apply a diff
print("\nApplying diff:")
diff = """
<<<<<<< SEARCH
Hello from PraisonAI Code Tools!
=======
Hello from PraisonAI Code Tools!
This line was added by a diff.
>>>>>>> REPLACE
"""
result = code_apply_diff("test_output.txt", diff)
print(f" {result}")
# Execute command
print("\nExecuting command:")
output = code_execute_command("cat test_output.txt")
print(f" {output}")
# Cleanup
import os
os.remove("test_output.txt")
print("\nCleanup complete.")
+36
View File
@@ -0,0 +1,36 @@
"""
Codex CLI Integration Example
Demonstrates how to use OpenAI Codex CLI as an agent tool in PraisonAI.
"""
import asyncio
from praisonai.integrations import CodexCLIIntegration
async def main():
# Create Codex CLI integration
codex = CodexCLIIntegration(
workspace=".",
full_auto=True,
sandbox="danger-full-access",
json_output=True
)
# Check availability
print(f"Codex CLI available: {codex.is_available}")
print(f"Full Auto: {codex.full_auto}")
print(f"Sandbox: {codex.sandbox}")
# Execute a coding task
result = await codex.execute("List files in the current directory")
print(f"Result: {result}")
# Stream JSON events
print("\nStreaming events:")
async for event in codex.stream("Explain the project"):
event_type = event.get("type", "")
if event_type == "item.completed":
print(f" {event}")
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,37 @@
"""
Cursor CLI Integration Example
Demonstrates how to use Cursor CLI as an agent tool in PraisonAI.
"""
import asyncio
from praisonai.integrations import CursorCLIIntegration
async def main():
# Create Cursor CLI integration
cursor = CursorCLIIntegration(
workspace=".",
output_format="json",
force=True,
model="gpt-5",
stream_partial=True
)
# Check availability
print(f"Cursor CLI available: {cursor.is_available}")
print(f"Force mode: {cursor.force}")
print(f"Model: {cursor.model}")
# Execute a coding task
result = await cursor.execute("List files in the current directory")
print(f"Result: {result}")
# Stream output
print("\nStreaming output:")
async for event in cursor.stream("Explain the project"):
content = event.get("content", "")
if content:
print(content, end="", flush=True)
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,58 @@
"""
External Agents Integration Example
Demonstrates how to use all external CLI integrations as agent tools.
"""
import asyncio
from praisonai.integrations import (
ClaudeCodeIntegration,
GeminiCLIIntegration,
CodexCLIIntegration,
CursorCLIIntegration,
get_available_integrations
)
async def main():
# Check availability of all integrations
print("=== External CLI Integrations ===\n")
availability = get_available_integrations()
print("Availability:")
for name, avail in availability.items():
status = "✅" if avail else "❌"
print(f" {status} {name}")
# Create integrations
print("\n--- Claude Code ---")
claude = ClaudeCodeIntegration(workspace=".", skip_permissions=True)
print(f"CLI: {claude.cli_command}")
print(f"Available: {claude.is_available}")
tool = claude.as_tool()
print(f"Tool: {tool.__name__}")
print("\n--- Gemini CLI ---")
gemini = GeminiCLIIntegration(workspace=".", model="gemini-2.5-flash")
print(f"CLI: {gemini.cli_command}")
print(f"Available: {gemini.is_available}")
tool = gemini.as_tool()
print(f"Tool: {tool.__name__}")
print("\n--- Codex CLI ---")
codex = CodexCLIIntegration(workspace=".", full_auto=True)
print(f"CLI: {codex.cli_command}")
print(f"Available: {codex.is_available}")
tool = codex.as_tool()
print(f"Tool: {tool.__name__}")
print("\n--- Cursor CLI ---")
cursor = CursorCLIIntegration(workspace=".", force=True)
print(f"CLI: {cursor.cli_command}")
print(f"Available: {cursor.is_available}")
tool = cursor.as_tool()
print(f"Tool: {tool.__name__}")
print("\n=== All integrations ready ===")
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,33 @@
"""
Gemini CLI Integration Example
Demonstrates how to use Gemini CLI as an agent tool in PraisonAI.
"""
import asyncio
from praisonai.integrations import GeminiCLIIntegration
async def main():
# Create Gemini CLI integration
gemini = GeminiCLIIntegration(
workspace=".",
model="gemini-2.5-flash",
output_format="json",
include_directories=["../lib", "../docs"]
)
# Check availability
print(f"Gemini CLI available: {gemini.is_available}")
print(f"Model: {gemini.model}")
# Execute a coding task
result = await gemini.execute("Analyze the code structure")
print(f"Result: {result}")
# Execute with stats
result, stats = await gemini.execute_with_stats("Explain main.py")
print(f"Result: {result}")
print(f"Stats: {stats}")
if __name__ == "__main__":
asyncio.run(main())
+40 -3
View File
@@ -1,11 +1,48 @@
"""
AutoAgents Example
AutoAgents automatically creates and manages AI agents based on high-level instructions.
It features dynamic agent count based on task complexity.
Documentation: https://docs.praison.ai/features/autoagents
CLI Reference: https://docs.praison.ai/nocode/auto
New Features:
- Dynamic agent count (1-4 based on task complexity)
- Workflow patterns: sequential, parallel, routing, orchestrator-workers, evaluator-optimizer
- Pattern recommendation based on task keywords
- Tool preservation from LLM suggestions
"""
from praisonaiagents import AutoAgents
from praisonaiagents.tools import duckduckgo
# Basic usage - AutoAgents analyzes complexity and creates optimal agents
agents = AutoAgents(
instructions="Search for information about AI Agents",
tools=[duckduckgo],
process="sequential",
verbose=True
process="sequential", # or "hierarchical"
verbose=True,
max_agents=3 # Maximum number of agents to create
)
agents.start()
result = agents.start()
print(result)
# =============================================================================
# CLI Usage Examples (for reference)
# =============================================================================
#
# # Auto-generate agents (dynamic count based on complexity)
# praisonai --auto "Write a haiku about spring"
# praisonai --auto "Research AI trends, analyze data, write report"
#
# # Auto-generate workflow with specific pattern
# praisonai workflow auto "Research and write" --pattern sequential
# praisonai workflow auto "Research from multiple sources" --pattern parallel
# praisonai workflow auto "Comprehensive analysis" --pattern orchestrator-workers
# praisonai workflow auto "Refine content quality" --pattern evaluator-optimizer
#
# # With framework selection
# praisonai --framework crewai --auto "Create a movie script"
# praisonai --framework autogen --auto "Create a movie script"
+108
View File
@@ -0,0 +1,108 @@
"""
Multiple MCP Servers Example
This example demonstrates how to use multiple MCP servers with a single agent.
You can combine different MCP servers (time, memory, filesystem, etc.) along
with regular Python functions as tools.
Requirements:
- pip install praisonaiagents
- uvx (for mcp-server-time)
- npx (for @modelcontextprotocol servers)
"""
from praisonaiagents import Agent
from praisonaiagents.mcp import MCP
def my_custom_tool(query: str) -> str:
"""
A custom Python function that can be used alongside MCP tools.
Args:
query: The search query
Returns:
A formatted result string
"""
return f"Custom tool result for: {query}"
def main():
# Example 1: Single MCP server (basic usage)
print("=" * 60)
print("Example 1: Single MCP Server")
print("=" * 60)
agent_single = Agent(
name="TimeAgent",
instructions="You are a helpful assistant that can tell time.",
tools=MCP("uvx mcp-server-time")
)
response = agent_single.start("What time is it in Tokyo?")
print(f"Response: {response}\n")
# Example 2: Multiple MCP servers
print("=" * 60)
print("Example 2: Multiple MCP Servers")
print("=" * 60)
agent_multi = Agent(
name="MultiMCPAgent",
instructions="""You are a helpful assistant with access to multiple tools:
- Time tools for getting current time and converting timezones
- Memory tools for storing and retrieving information
Use the appropriate tool based on the user's request.""",
tools=[
MCP("uvx mcp-server-time"), # Time tools
MCP("npx @modelcontextprotocol/server-memory"), # Memory tools
]
)
response = agent_multi.start("What time is it in New York and London?")
print(f"Response: {response}\n")
# Example 3: Multiple MCP servers + custom functions
print("=" * 60)
print("Example 3: MCP Servers + Custom Functions")
print("=" * 60)
agent_mixed = Agent(
name="MixedToolsAgent",
instructions="""You are a helpful assistant with access to:
- Time tools for timezone operations
- A custom search tool for general queries
Choose the right tool for each task.""",
tools=[
MCP("uvx mcp-server-time"), # MCP server
my_custom_tool # Regular Python function
]
)
response = agent_mixed.start("What time is it in Paris?")
print(f"Response: {response}\n")
# Example 4: MCP with environment variables
print("=" * 60)
print("Example 4: MCP with Environment Variables")
print("=" * 60)
# Note: Replace with your actual API key
# agent_with_env = Agent(
# name="SearchAgent",
# instructions="You can search the web for information.",
# tools=[
# MCP(
# command="npx",
# args=["-y", "@modelcontextprotocol/server-brave-search"],
# env={"BRAVE_API_KEY": "your-api-key-here"}
# ),
# MCP("uvx mcp-server-time")
# ]
# )
print("(Skipped - requires BRAVE_API_KEY)")
if __name__ == "__main__":
main()
@@ -0,0 +1,115 @@
"""
24/7 News Checker Agent Example
This example demonstrates how to create a continuously running agent
that checks for AI news every hour and prints the results.
Usage:
python news_checker_agent.py
The agent will:
1. Run immediately on start
2. Check for latest AI news every 1 hour
3. Print results to console
4. Continue running until stopped (Ctrl+C)
"""
import os
import sys
from datetime import datetime
# Add parent directory to path for imports
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '../../..'))
from praisonaiagents import Agent
from praisonai.agent_scheduler import AgentScheduler
from tools import search_tool
def on_success(result):
"""Callback function called when agent execution succeeds."""
print("\n" + "="*80)
print(f"✅ SUCCESS - {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
print("="*80)
print(result)
print("="*80 + "\n")
def on_failure(error):
"""Callback function called when agent execution fails."""
print("\n" + "="*80)
print(f"❌ FAILURE - {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
print("="*80)
print(f"Error: {error}")
print("="*80 + "\n")
def main():
"""Main function to set up and run the scheduled news checker agent."""
print("\n🤖 Starting 24/7 News Checker Agent")
print("="*80)
# Create the agent with web search capability
agent = Agent(
name="NewsChecker",
role="AI News Monitor",
goal="Monitor and summarize the latest AI and technology news",
backstory="""You are an expert AI news analyst who stays up-to-date
with the latest developments in artificial intelligence, machine learning,
and technology. You provide concise, accurate summaries of important news.""",
tools=[search_tool],
verbose=True
)
# Define the task
task = """Search for the latest AI news from today.
Focus on major developments, breakthroughs, or announcements.
Provide a summary of the top 3-5 most important news items.
Include the source and key details for each item."""
# Create scheduler with callbacks
scheduler = AgentScheduler(
agent=agent,
task=task,
on_success=on_success,
on_failure=on_failure
)
print(f"Agent: {agent.name}")
print(f"Task: {task}")
print(f"Schedule: Every 1 hour")
print("="*80)
print("\n⏰ Starting scheduler... (Press Ctrl+C to stop)\n")
# Start the scheduler
# - "hourly" or "*/1h" = every 1 hour
# - run_immediately=True = run once before starting the schedule
scheduler.start(
schedule_expr="hourly", # Run every hour
max_retries=3, # Retry up to 3 times on failure
run_immediately=True # Run immediately on start
)
try:
# Keep the main thread alive
while scheduler.is_running:
import time
time.sleep(1)
except KeyboardInterrupt:
print("\n\n🛑 Stopping scheduler...")
scheduler.stop()
# Print final statistics
stats = scheduler.get_stats()
print("\n📊 Final Statistics:")
print(f" Total Executions: {stats['total_executions']}")
print(f" Successful: {stats['successful_executions']}")
print(f" Failed: {stats['failed_executions']}")
print(f" Success Rate: {stats['success_rate']:.1f}%")
print("\n✅ Agent stopped successfully\n")
if __name__ == "__main__":
main()
@@ -0,0 +1,143 @@
"""
Live News Checker Agent - 24/7 Scheduled Agent
This example demonstrates a real working agent that checks AI news
every hour using the AgentScheduler with actual API calls.
Requirements:
- OPENAI_API_KEY environment variable set
- praisonaiagents package installed
Usage:
python news_checker_live.py
"""
import os
import sys
from datetime import datetime
# Add parent directory to path
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '../../..'))
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '../../../..'))
from praisonaiagents import Agent
from praisonai.scheduler import AgentScheduler
# Import search_tool from root tools.py
try:
from tools import search_tool
except ImportError:
# Fallback: create inline search tool
from duckduckgo_search import DDGS
def search_tool(query: str) -> list:
"""Search the web using DuckDuckGo."""
try:
results = []
ddgs = DDGS()
for result in ddgs.text(keywords=query, max_results=5):
results.append({
"title": result.get("title", ""),
"url": result.get("href", ""),
"snippet": result.get("body", ""),
})
return results
except Exception as e:
print(f"Search error: {e}")
return []
def on_success(result):
"""Callback when agent execution succeeds."""
print("\n" + "="*80)
print(f"✅ SUCCESS - {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
print("="*80)
print(result)
print("="*80 + "\n")
def on_failure(error):
"""Callback when agent execution fails."""
print("\n" + "="*80)
print(f"❌ FAILURE - {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
print("="*80)
print(f"Error: {error}")
print("="*80 + "\n")
def main():
"""Main function to run the scheduled news checker."""
# Check for API key
if not os.getenv("OPENAI_API_KEY"):
print("❌ Error: OPENAI_API_KEY environment variable not set")
print("Please set it with: export OPENAI_API_KEY=your_key_here")
sys.exit(1)
print("\n🤖 Starting 24/7 AI News Checker Agent")
print("="*80)
# Create the agent with web search capability
agent = Agent(
name="AI News Monitor",
role="Technology News Analyst",
goal="Monitor and summarize the latest AI and technology news",
backstory="""You are an expert AI news analyst who stays up-to-date
with the latest developments in artificial intelligence, machine learning,
and technology. You provide concise, accurate summaries of important news.""",
tools=[search_tool],
verbose=True,
self_reflect=False # Disable reflection for faster execution
)
# Define the task
task = """Search for the latest AI news from today.
Focus on major developments, breakthroughs, or announcements.
Provide a summary of the top 3 most important news items.
Include the source and key details for each item."""
# Create scheduler with callbacks
scheduler = AgentScheduler(
agent=agent,
task=task,
on_success=on_success,
on_failure=on_failure
)
print(f"Agent: {agent.name}")
print(f"Task: {task}")
print(f"Schedule: Every 1 hour")
print("="*80)
print("\n⏰ Starting scheduler... (Press Ctrl+C to stop)\n")
# Start the scheduler
# For testing: use "*/2m" (every 2 minutes) instead of "hourly"
# For production: use "hourly"
scheduler.start(
schedule_expr="*/2m", # Every 2 minutes for testing
max_retries=3,
run_immediately=True # Run once immediately
)
try:
# Keep the main thread alive
while scheduler.is_running:
import time
time.sleep(1)
except KeyboardInterrupt:
print("\n\n🛑 Stopping scheduler...")
scheduler.stop()
# Print final statistics
stats = scheduler.get_stats()
print("\n📊 Final Statistics:")
print(f" Total Executions: {stats['total_executions']}")
print(f" Successful: {stats['successful_executions']}")
print(f" Failed: {stats['failed_executions']}")
print(f" Success Rate: {stats['success_rate']:.1f}%")
print("\n✅ Agent stopped successfully\n")
if __name__ == "__main__":
main()
@@ -0,0 +1,61 @@
"""
Simple test script for 24/7 Agent Scheduler
This script demonstrates the basic usage of AgentScheduler
for running agents periodically.
Usage:
python simple_test.py
"""
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '../../..'))
from unittest.mock import Mock
from praisonai.scheduler import AgentScheduler
# Create a mock agent for testing
mock_agent = Mock()
mock_agent.name = "TestAgent"
mock_agent.start = Mock(return_value="Test execution successful!")
# Create scheduler
scheduler = AgentScheduler(
agent=mock_agent,
task="Test task execution"
)
print("🤖 Testing AgentScheduler")
print("=" * 60)
# Test 1: Execute once
print("\n✅ Test 1: Execute once")
result = scheduler.execute_once()
print(f"Result: {result}")
# Test 2: Get stats
print("\n✅ Test 2: Get statistics")
stats = scheduler.get_stats()
print(f"Stats: {stats}")
# Test 3: Start scheduler (will run every 5 seconds)
print("\n✅ Test 3: Start scheduler (*/5s)")
scheduler.start("*/5s", run_immediately=True)
import time
print("Running for 12 seconds...")
time.sleep(12)
# Stop scheduler
print("\n✅ Test 4: Stop scheduler")
scheduler.stop()
# Final stats
print("\n📊 Final Statistics:")
final_stats = scheduler.get_stats()
for key, value in final_stats.items():
print(f" {key}: {value}")
print("\n✅ All tests passed!")
print("=" * 60)
@@ -0,0 +1,14 @@
framework: praisonai
agents:
- name: "5 Second Timer"
role: "Timer Agent"
instructions: "Display the current time"
verbose: true
task: "Show current time"
schedule:
interval: "*/5s"
max_retries: 3
run_immediately: true
+14
View File
@@ -0,0 +1,14 @@
framework: praisonai
agents:
- name: "Counter Agent"
role: "Counter"
instructions: "Count from 1 to 5, incrementing by 1 each time. Just output the numbers clearly."
verbose: true
task: "Print numbers 1, 2, 3, 4, 5 (one number per line)"
schedule:
interval: "*/5s"
max_retries: 3
run_immediately: true
@@ -0,0 +1,16 @@
framework: praisonai
agents:
- name: "Counter Agent with Limits"
role: "Counter"
instructions: "Count from 1 to 5, incrementing by 1 each time. Just output the numbers clearly."
verbose: true
task: "Print numbers 1, 2, 3, 4, 5 (one number per line)"
schedule:
interval: "*/5s" # Run every 5 seconds
max_retries: 3 # Retry up to 3 times on failure
run_immediately: true # Run once immediately on start
timeout: 30 # Max 30 seconds per execution
max_cost: 0.01 # Stop after spending $0.01 USD
@@ -0,0 +1,15 @@
framework: praisonai
agents:
- name: "Budget Test Agent"
role: "Tester"
instructions: "Count from 1 to 3"
verbose: true
task: "Print numbers 1, 2, 3"
schedule:
interval: "*/3s"
max_retries: 2
run_immediately: true
# No max_cost specified - should default to $1.00
+73
View File
@@ -0,0 +1,73 @@
"""
Basic Agent Skills Usage Example
This example demonstrates how to use Agent Skills with PraisonAI Agents.
Skills are modular capabilities that can be loaded and used by agents.
"""
from praisonaiagents import Agent, SkillManager
# Example 1: Using SkillManager directly
print("=" * 50)
print("Example 1: Using SkillManager directly")
print("=" * 50)
# Create a skill manager
manager = SkillManager()
# Discover skills from a directory
# This scans the directory for subdirectories containing SKILL.md files
skill_count = manager.discover(["./pdf-processing"], include_defaults=False)
print(f"Discovered {skill_count} skill(s)")
# List available skills
for skill in manager:
print(f" - {skill.properties.name}: {skill.properties.description[:50]}...")
# Generate prompt XML for system prompt injection
prompt_xml = manager.to_prompt()
print("\nGenerated prompt XML:")
print(prompt_xml)
# Example 2: Using skills with an Agent
print("\n" + "=" * 50)
print("Example 2: Using skills with an Agent")
print("=" * 50)
# Create an agent with skills
# Skills are lazy-loaded only when accessed
agent = Agent(
name="PDF Assistant",
instructions="You are a helpful assistant that can process PDF documents.",
skills=["./pdf-processing"], # Direct skill paths
)
# Access the skill manager through the agent
if agent.skill_manager:
print(f"Agent has {len(agent.skill_manager)} skill(s) loaded")
# Get the skills prompt for system prompt injection
skills_prompt = agent.get_skills_prompt()
print("\nAgent skills prompt:")
print(skills_prompt)
# Example 3: Discovering skills from multiple directories
print("\n" + "=" * 50)
print("Example 3: Using skills_dirs for discovery")
print("=" * 50)
# Create an agent that discovers skills from directories
agent_with_discovery = Agent(
name="Multi-Skill Agent",
instructions="You are a versatile assistant with multiple skills.",
skills_dirs=["./"], # Scan current directory for skill subdirectories
)
if agent_with_discovery.skill_manager:
print(f"Discovered {len(agent_with_discovery.skill_manager)} skill(s)")
for name in agent_with_discovery.skill_manager.skill_names:
print(f" - {name}")
print("\n" + "=" * 50)
print("Examples complete!")
print("=" * 50)
+94
View File
@@ -0,0 +1,94 @@
#!/usr/bin/env python3
"""
Example: Creating Skills with PraisonAI CLI
This example demonstrates how to create Agent Skills using the CLI.
"""
import subprocess
import tempfile
import os
from pathlib import Path
def example_create_skill_with_template():
"""Create a skill using template mode (no API key required)."""
print("=" * 60)
print("Example 1: Create skill with template")
print("=" * 60)
with tempfile.TemporaryDirectory() as tmpdir:
result = subprocess.run([
"praisonai", "skills", "create",
"--name", "my-template-skill",
"--description", "A skill for processing text files",
"--author", "example-team",
"--license", "MIT",
"--template",
"--output-dir", tmpdir
], capture_output=True, text=True)
print(result.stdout)
if result.returncode == 0:
skill_md = Path(tmpdir) / "my-template-skill" / "SKILL.md"
print(f"\nGenerated SKILL.md:\n{skill_md.read_text()[:500]}...")
else:
print(f"Error: {result.stderr}")
def example_create_skill_with_ai():
"""Create a skill using AI generation (requires API key)."""
print("\n" + "=" * 60)
print("Example 2: Create skill with AI generation")
print("=" * 60)
if not os.environ.get("OPENAI_API_KEY"):
print("Skipping: OPENAI_API_KEY not set")
return
with tempfile.TemporaryDirectory() as tmpdir:
result = subprocess.run([
"praisonai", "skills", "create",
"--name", "csv-analyzer",
"--description", "Analyze CSV files and generate statistical reports",
"--output-dir", tmpdir
], capture_output=True, text=True)
print(result.stdout)
if result.returncode == 0:
skill_md = Path(tmpdir) / "csv-analyzer" / "SKILL.md"
print(f"\nGenerated SKILL.md:\n{skill_md.read_text()[:800]}...")
# Check if script was generated
script_path = Path(tmpdir) / "csv-analyzer" / "scripts" / "skill.py"
if script_path.exists():
print(f"\nGenerated skill.py:\n{script_path.read_text()[:500]}...")
def example_create_skill_with_script():
"""Create a skill with auto-generated Python script."""
print("\n" + "=" * 60)
print("Example 3: Create skill with script")
print("=" * 60)
with tempfile.TemporaryDirectory() as tmpdir:
result = subprocess.run([
"praisonai", "skills", "create",
"--name", "data-processor",
"--description", "Process and transform data files",
"--template",
"--script",
"--output-dir", tmpdir
], capture_output=True, text=True)
print(result.stdout)
if result.returncode == 0:
script_path = Path(tmpdir) / "data-processor" / "scripts" / "skill.py"
if script_path.exists():
print(f"\nGenerated skill.py:\n{script_path.read_text()}")
if __name__ == "__main__":
example_create_skill_with_template()
example_create_skill_with_ai()
example_create_skill_with_script()
+163
View File
@@ -0,0 +1,163 @@
"""
Custom Skill Creation Example
This example demonstrates how to programmatically create and validate
custom Agent Skills using PraisonAI Agents.
"""
import tempfile
from pathlib import Path
from praisonaiagents.skills import (
SkillManager,
SkillLoader,
validate,
read_properties,
)
def create_custom_skill(
name: str,
description: str,
instructions: str,
base_dir: str = None
) -> Path:
"""Create a custom skill programmatically.
Args:
name: Skill name (kebab-case)
description: What the skill does
instructions: Markdown instructions for the skill
base_dir: Directory to create skill in (default: temp dir)
Returns:
Path to the created skill directory
"""
if base_dir is None:
base_dir = tempfile.mkdtemp()
skill_dir = Path(base_dir) / name
skill_dir.mkdir(parents=True, exist_ok=True)
# Create SKILL.md
skill_md_content = f"""---
name: {name}
description: {description}
license: Apache-2.0
metadata:
author: custom
version: "1.0"
---
{instructions}
"""
(skill_dir / "SKILL.md").write_text(skill_md_content)
# Create optional directories
(skill_dir / "scripts").mkdir(exist_ok=True)
(skill_dir / "references").mkdir(exist_ok=True)
(skill_dir / "assets").mkdir(exist_ok=True)
return skill_dir
def main():
print("=" * 60)
print("Custom Skill Creation Example")
print("=" * 60)
# Create a custom skill
print("\n1. Creating a custom skill...")
skill_path = create_custom_skill(
name="code-review",
description="Review code for best practices, bugs, and improvements. Use when asked to review or analyze code quality.",
instructions="""
# Code Review Skill
## Overview
This skill enables thorough code review with focus on:
- Code quality and readability
- Potential bugs and edge cases
- Performance considerations
- Security vulnerabilities
- Best practices adherence
## Instructions
When reviewing code:
1. **Read the code carefully** - Understand the purpose and flow
2. **Check for bugs** - Look for null checks, edge cases, error handling
3. **Evaluate readability** - Variable names, comments, structure
4. **Consider performance** - Time/space complexity, unnecessary operations
5. **Security review** - Input validation, injection risks, data exposure
6. **Suggest improvements** - Provide actionable recommendations
## Output Format
Structure your review as:
- Summary: Brief overview of the code
- Issues: List of problems found (severity: high/medium/low)
- Suggestions: Recommended improvements
- Positive aspects: What's done well
"""
)
print(f" Created skill at: {skill_path}")
# Validate the skill
print("\n2. Validating the skill...")
errors = validate(skill_path)
if errors:
print(" Validation errors:")
for error in errors:
print(f" - {error}")
else:
print(" ✓ Skill is valid!")
# Load and inspect the skill
print("\n3. Loading skill properties...")
props = read_properties(skill_path)
print(f" Name: {props.name}")
print(f" Description: {props.description[:60]}...")
print(f" License: {props.license}")
# Use with SkillLoader for progressive loading
print("\n4. Using SkillLoader for progressive disclosure...")
loader = SkillLoader()
# Level 1: Load metadata only
skill = loader.load_metadata(str(skill_path))
print(f" Level 1 - Metadata loaded: {skill.properties.name}")
print(f" Instructions loaded: {skill.is_activated}")
# Level 2: Activate (load instructions)
loader.activate(skill)
print(f" Level 2 - Activated: {skill.is_activated}")
print(f" Instructions preview: {skill.instructions[:100]}...")
# Use with SkillManager
print("\n5. Using with SkillManager...")
manager = SkillManager()
manager.add_skill(str(skill_path))
# Generate prompt XML
prompt = manager.to_prompt()
print(" Generated prompt XML:")
print(prompt)
# Clean up
print("\n6. Cleanup...")
import shutil
shutil.rmtree(skill_path.parent)
print(" Temporary files removed.")
print("\n" + "=" * 60)
print("Example complete!")
print("=" * 60)
if __name__ == "__main__":
main()
+137
View File
@@ -0,0 +1,137 @@
#!/usr/bin/env python3
"""
Example: Multi-Agent CLI Usage
This example demonstrates how to use the praisonai agents command
to define and run multiple agents from the command line.
"""
import subprocess
import os
def example_single_agent():
"""Run a single agent with a simple task."""
print("=" * 60)
print("Example 1: Single Agent")
print("=" * 60)
result = subprocess.run([
"praisonai", "agents", "run",
"--agent", "assistant:Helper",
"--task", "What is the capital of France?"
], capture_output=True, text=True)
print(result.stdout)
if result.returncode != 0:
print(f"Error: {result.stderr}")
def example_agent_with_tools():
"""Run an agent with tools."""
print("\n" + "=" * 60)
print("Example 2: Agent with Tools")
print("=" * 60)
result = subprocess.run([
"praisonai", "agents", "run",
"--agent", "researcher:Research Analyst:internet_search",
"--task", "Find the latest news about renewable energy"
], capture_output=True, text=True)
print(result.stdout)
def example_multiple_agents_sequential():
"""Run multiple agents in sequential mode."""
print("\n" + "=" * 60)
print("Example 3: Multiple Agents (Sequential)")
print("=" * 60)
result = subprocess.run([
"praisonai", "agents", "run",
"--agent", "researcher:Research Analyst:internet_search",
"--agent", "writer:Content Writer:write_file",
"--task", "Research AI trends and write a brief summary"
], capture_output=True, text=True)
print(result.stdout)
def example_multiple_agents_parallel():
"""Run multiple agents in parallel mode."""
print("\n" + "=" * 60)
print("Example 4: Multiple Agents (Parallel)")
print("=" * 60)
result = subprocess.run([
"praisonai", "agents", "run",
"--agent", "analyst1:Market Analyst",
"--agent", "analyst2:Tech Analyst",
"--process", "parallel",
"--task", "Analyze the current state of AI industry"
], capture_output=True, text=True)
print(result.stdout)
def example_with_custom_llm():
"""Run agents with a custom LLM model."""
print("\n" + "=" * 60)
print("Example 5: Custom LLM Model")
print("=" * 60)
result = subprocess.run([
"praisonai", "agents", "run",
"--agent", "coder:Developer:execute_command",
"--llm", "gpt-4o-mini",
"--task", "Write a simple hello world Python script"
], capture_output=True, text=True)
print(result.stdout)
def example_with_instructions():
"""Run agents with additional instructions."""
print("\n" + "=" * 60)
print("Example 6: With Additional Instructions")
print("=" * 60)
result = subprocess.run([
"praisonai", "agents", "run",
"--agent", "analyst:Data Analyst",
"--instructions", "Be concise and provide only key insights",
"--task", "Explain the benefits of using AI in healthcare"
], capture_output=True, text=True)
print(result.stdout)
def example_list_tools():
"""List available tools for agents."""
print("\n" + "=" * 60)
print("Example 7: List Available Tools")
print("=" * 60)
result = subprocess.run([
"praisonai", "agents", "list"
], capture_output=True, text=True)
print(result.stdout)
if __name__ == "__main__":
# Check for API key
if not os.environ.get("OPENAI_API_KEY"):
print("Warning: OPENAI_API_KEY not set. Some examples may fail.")
print()
# Run examples
example_list_tools()
example_single_agent()
# Uncomment to run more examples:
# example_agent_with_tools()
# example_multiple_agents_sequential()
# example_multiple_agents_parallel()
# example_with_custom_llm()
# example_with_instructions()
+38
View File
@@ -0,0 +1,38 @@
---
name: pdf-processing
description: Process and extract information from PDF documents. Use this skill when the user asks to read, analyze, or extract data from PDF files.
license: Apache-2.0
compatibility: Works with PraisonAI Agents
metadata:
author: praisonai
version: "1.0"
---
# PDF Processing Skill
## Overview
This skill enables agents to process PDF documents, extract text content, and analyze the information within them.
## When to Use
Activate this skill when:
- User asks to read or analyze a PDF file
- User needs to extract text from a PDF
- User wants to summarize a PDF document
- User needs to search for information within a PDF
## Instructions
1. First, verify the PDF file exists at the specified path
2. Use appropriate tools to read the PDF content
3. Extract text while preserving structure where possible
4. For large PDFs, process in chunks to manage context
5. Summarize or analyze based on user's specific request
## Best Practices
- Always confirm the file path before processing
- Handle encrypted PDFs gracefully with appropriate error messages
- For scanned PDFs, note that OCR may be required
- Preserve important formatting like tables and lists
+137
View File
@@ -0,0 +1,137 @@
#!/usr/bin/env python3
"""
Basic Thinking Budgets Example for PraisonAI Agents.
This example demonstrates how to use thinking budgets to:
1. Configure token budgets for extended thinking
2. Use predefined budget levels
3. Track thinking usage
4. Adapt budgets based on complexity
Usage:
python basic_thinking.py
"""
from praisonaiagents.thinking import (
ThinkingBudget, ThinkingConfig, ThinkingUsage, ThinkingTracker
)
from praisonaiagents.thinking.budget import BudgetLevel
def main():
print("=" * 60)
print("Thinking Budgets Demo")
print("=" * 60)
# ==========================================================================
# Budget Levels
# ==========================================================================
print("\n--- Budget Levels ---")
levels = [
("Minimal", ThinkingBudget.minimal()),
("Low", ThinkingBudget.low()),
("Medium", ThinkingBudget.medium()),
("High", ThinkingBudget.high()),
("Maximum", ThinkingBudget.maximum()),
]
for name, budget in levels:
print(f" {name}: {budget.max_tokens:,} tokens")
# ==========================================================================
# Custom Budget
# ==========================================================================
print("\n--- Custom Budget ---")
custom_budget = ThinkingBudget(
max_tokens=12000,
min_tokens=2000,
adaptive=True,
max_time_seconds=120.0
)
print(f" Max tokens: {custom_budget.max_tokens:,}")
print(f" Min tokens: {custom_budget.min_tokens:,}")
print(f" Adaptive: {custom_budget.adaptive}")
print(f" Time limit: {custom_budget.max_time_seconds}s")
# ==========================================================================
# Complexity-Based Scaling
# ==========================================================================
print("\n--- Complexity-Based Token Allocation ---")
budget = ThinkingBudget(
max_tokens=16000,
min_tokens=2000,
adaptive=True
)
complexities = [0.0, 0.25, 0.5, 0.75, 1.0]
for complexity in complexities:
tokens = budget.get_tokens_for_complexity(complexity)
bar = "█" * int(tokens / 1000)
print(f" Complexity {complexity:.2f}: {tokens:,} tokens {bar}")
# ==========================================================================
# Usage Tracking
# ==========================================================================
print("\n--- Usage Tracking ---")
usage = ThinkingUsage(
budget_tokens=8000,
tokens_used=5000,
time_seconds=30.0,
budget_time=60.0
)
print(f" Tokens used: {usage.tokens_used:,} / {usage.budget_tokens:,}")
print(f" Tokens remaining: {usage.tokens_remaining:,}")
print(f" Utilization: {usage.token_utilization:.1%}")
print(f" Over budget: {usage.is_over_budget}")
print(f" Time remaining: {usage.time_remaining}s")
# ==========================================================================
# Session Tracking
# ==========================================================================
print("\n--- Session Tracking ---")
tracker = ThinkingTracker()
# Simulate multiple thinking sessions
sessions = [
(8000, 4000, 20.0),
(8000, 6000, 35.0),
(16000, 12000, 60.0),
(8000, 9000, 45.0), # Over budget
]
for budget_tokens, tokens_used, time_seconds in sessions:
session = tracker.start_session(budget_tokens=budget_tokens)
tracker.end_session(session, tokens_used=tokens_used, time_seconds=time_seconds)
summary = tracker.get_summary()
print(f" Total sessions: {summary['session_count']}")
print(f" Total tokens used: {summary['total_tokens_used']:,}")
print(f" Average tokens/session: {summary['average_tokens_per_session']:,.0f}")
print(f" Average utilization: {summary['average_utilization']:.1%}")
print(f" Over-budget sessions: {summary['over_budget_count']}")
# ==========================================================================
# Serialization
# ==========================================================================
print("\n--- Serialization ---")
budget = ThinkingBudget.from_level(BudgetLevel.HIGH)
data = budget.to_dict()
print(f" Serialized: level={data['level']}, max_tokens={data['max_tokens']}")
restored = ThinkingBudget.from_dict(data)
print(f" Restored: {restored.max_tokens:,} tokens")
print("\n" + "=" * 60)
print("Demo Complete!")
print("=" * 60)
if __name__ == "__main__":
main()
+44
View File
@@ -0,0 +1,44 @@
"""ArXiv Tool Example.
This example demonstrates how to use the ArXiv tool with PraisonAI agents.
Requirements:
pip install "praisonai[tools]"
Usage:
python tool.py
Note: No API key required!
"""
from praisonai_tools import ArxivTool
def main():
# Initialize ArXiv tool (no API key needed)
arxiv = ArxivTool()
# Example 1: Search ArXiv
print("=" * 60)
print("Example 1: Search ArXiv")
print("=" * 60)
results = arxiv.search("transformer neural networks", max_results=3)
if results and "error" in results[0]:
print(f"Error: {results[0]['error']}")
else:
print(f"Found {len(results)} papers:")
for paper in results:
print(f"\n Title: {paper.get('title', 'N/A')[:60]}...")
authors = paper.get('authors', [])
if authors:
print(f" Authors: {', '.join(authors[:2])}...")
print(f" Published: {paper.get('published', 'N/A')}")
print(f" PDF: {paper.get('pdf_url', 'N/A')}")
print("\n✅ ArXiv tool working correctly!")
if __name__ == "__main__":
main()
+53
View File
@@ -0,0 +1,53 @@
"""Calculator Tool Example.
This example demonstrates how to use the Calculator tool with PraisonAI agents.
Requirements:
pip install "praisonai[tools]"
Usage:
python tool.py
Note: No API key required!
"""
from praisonai_tools import CalculatorTool
def main():
# Initialize Calculator tool
calc = CalculatorTool()
# Example 1: Basic arithmetic
print("=" * 60)
print("Example 1: Basic Arithmetic")
print("=" * 60)
expressions = [
"2 + 2",
"10 * 5",
"100 / 4",
"2 ** 10",
"(10 + 5) * 2 - 3"
]
for expr in expressions:
result = calc.evaluate(expr)
print(f" {expr} = {result.get('result', result)}")
# Example 2: Percentage calculation
print("\n" + "=" * 60)
print("Example 2: Percentage Calculation")
print("=" * 60)
result = calc.evaluate("250 * 0.15")
print(f" 15% of 250 = {result.get('result', result)}")
result = calc.evaluate("1000 * (1 + 0.05) ** 3")
print(f" $1000 at 5% for 3 years = ${result.get('result', 0):.2f}")
print("\n✅ Calculator tool working correctly!")
if __name__ == "__main__":
main()
+75
View File
@@ -0,0 +1,75 @@
"""DuckDuckGo Search Tool Example.
This example demonstrates how to use the DuckDuckGo search tool with PraisonAI agents.
Requirements:
pip install "praisonai[tools]"
Usage:
python tool.py
Note: No API key required!
"""
from praisonai_tools import DuckDuckGoTool
def main():
# Initialize DuckDuckGo tool (no API key needed)
ddg = DuckDuckGoTool()
# Example 1: Web search
print("=" * 60)
print("Example 1: Web Search")
print("=" * 60)
results = ddg.search("Python programming best practices", max_results=3)
if results and "error" in results[0]:
print(f"Error: {results[0]['error']}")
else:
print(f"Found {len(results)} results:")
for r in results:
print(f" - {r['title']}")
print(f" URL: {r['url']}")
print(f" Snippet: {r['snippet'][:100]}...")
print()
# Example 2: News search
print("=" * 60)
print("Example 2: News Search")
print("=" * 60)
news = ddg.news("artificial intelligence", max_results=3)
if news and "error" in news[0]:
print(f"Error: {news[0]['error']}")
else:
print(f"Found {len(news)} news articles:")
for n in news:
print(f" - {n['title']}")
print(f" Source: {n.get('source', 'N/A')}")
print(f" Date: {n.get('date', 'N/A')}")
print()
# Example 3: Image search
print("=" * 60)
print("Example 3: Image Search")
print("=" * 60)
images = ddg.images("sunset landscape", max_results=3)
if images and "error" in images[0]:
print(f"Error: {images[0]['error']}")
else:
print(f"Found {len(images)} images:")
for img in images:
print(f" - {img['title']}")
print(f" Image URL: {img['image_url'][:60]}...")
print()
print("✅ DuckDuckGo tool working correctly!")
if __name__ == "__main__":
main()
+48
View File
@@ -0,0 +1,48 @@
"""Exa Search Tool Example.
This example demonstrates how to use the Exa search tool with PraisonAI agents.
Requirements:
pip install "praisonai[tools]"
export EXA_API_KEY=your_api_key
Usage:
python tool.py
"""
import os
from praisonai_tools import ExaTool
def main():
# Check for API key
if not os.getenv("EXA_API_KEY"):
print("Error: EXA_API_KEY environment variable not set")
print("Get your API key from https://exa.ai/")
return
# Initialize Exa tool
exa = ExaTool()
# Example 1: Neural search
print("=" * 60)
print("Example 1: Neural Search")
print("=" * 60)
results = exa.search("best practices for Python development", num_results=3)
if results and "error" in results[0]:
print(f"Error: {results[0]['error']}")
else:
print(f"Found {len(results)} results:")
for r in results:
print(f" - {r.get('title', 'N/A')[:50]}...")
print(f" URL: {r.get('url', 'N/A')[:60]}...")
print(f" Score: {r.get('score', 'N/A')}")
print()
print("✅ Exa tool working correctly!")
if __name__ == "__main__":
main()
+45
View File
@@ -0,0 +1,45 @@
"""Firecrawl Tool Example.
This example demonstrates how to use the Firecrawl tool with PraisonAI agents.
Requirements:
pip install "praisonai[tools]"
export FIRECRAWL_API_KEY=your_api_key
Usage:
python tool.py
"""
import os
from praisonai_tools import FirecrawlTool
def main():
# Check for API key
if not os.getenv("FIRECRAWL_API_KEY"):
print("Error: FIRECRAWL_API_KEY environment variable not set")
print("Get your API key from https://firecrawl.dev/")
return
# Initialize Firecrawl tool
firecrawl = FirecrawlTool()
# Example 1: Scrape a page
print("=" * 60)
print("Example 1: Scrape a Page")
print("=" * 60)
result = firecrawl.scrape("https://example.com")
if "error" in result:
print(f"Error: {result['error']}")
else:
print(f"URL: {result.get('url', 'N/A')}")
markdown = result.get('markdown', '')
print(f"Content preview: {markdown[:300]}...")
print("\n✅ Firecrawl tool working correctly!")
if __name__ == "__main__":
main()
+42
View File
@@ -0,0 +1,42 @@
"""Hacker News Tool Example.
This example demonstrates how to use the Hacker News tool with PraisonAI agents.
Requirements:
pip install "praisonai[tools]"
Usage:
python tool.py
Note: No API key required!
"""
from praisonai_tools import HackerNewsTool
def main():
# Initialize Hacker News tool
hn = HackerNewsTool()
# Example 1: Get top stories
print("=" * 60)
print("Example 1: Top Stories")
print("=" * 60)
stories = hn.get_top_stories(limit=5)
if stories and "error" in stories[0]:
print(f"Error: {stories[0]['error']}")
else:
print(f"Found {len(stories)} top stories:")
for story in stories:
print(f" - {story.get('title', 'N/A')[:50]}...")
print(f" Score: {story.get('score', 'N/A')} | Comments: {story.get('descendants', 'N/A')}")
print(f" URL: {story.get('url', 'N/A')[:60]}...")
print()
print("✅ Hacker News tool working correctly!")
if __name__ == "__main__":
main()
+40
View File
@@ -0,0 +1,40 @@
"""Jina Reader Tool Example.
This example demonstrates how to use the Jina Reader tool with PraisonAI agents.
Requirements:
pip install "praisonai[tools]"
Usage:
python tool.py
Note: Works without API key (with rate limits)
"""
from praisonai_tools import JinaTool
def main():
# Initialize Jina tool (works without API key)
jina = JinaTool()
# Example 1: Read a URL
print("=" * 60)
print("Example 1: Read URL Content")
print("=" * 60)
result = jina.read("https://example.com")
if "error" in result:
print(f"Error: {result['error']}")
else:
print(f"URL: {result.get('url', 'N/A')}")
print(f"Title: {result.get('title', 'N/A')}")
content = result.get('content', '')
print(f"Content preview: {content[:300]}...")
print("\n✅ Jina tool working correctly!")
if __name__ == "__main__":
main()
+56
View File
@@ -0,0 +1,56 @@
"""PostgreSQL Tool Example.
This example demonstrates how to use the PostgreSQL tool with PraisonAI agents.
Requirements:
pip install "praisonai[tools]"
Docker: docker run -d --name postgres -e POSTGRES_PASSWORD=praison123 -e POSTGRES_DB=praisonai -p 5432:5432 postgres:16
Usage:
python tool.py
"""
from praisonai_tools import PostgresTool
def main():
# Initialize PostgreSQL tool
pg = PostgresTool(
host="localhost",
port=5432,
database="praisonai",
user="postgres",
password="praison123"
)
# Example 1: List tables
print("=" * 60)
print("Example 1: List Tables")
print("=" * 60)
tables = pg.list_tables()
if isinstance(tables, dict) and "error" in tables:
print(f"Error: {tables['error']}")
else:
print(f"Found {len(tables)} tables:")
for t in tables[:10]:
print(f" - {t.get('table_name', t)}")
# Example 2: Query
print("\n" + "=" * 60)
print("Example 2: Execute Query")
print("=" * 60)
result = pg.query("SELECT version()")
if isinstance(result, dict) and "error" in result:
print(f"Error: {result['error']}")
else:
print(f"PostgreSQL version: {result}")
print("\n✅ PostgreSQL tool working correctly!")
if __name__ == "__main__":
main()
+47
View File
@@ -0,0 +1,47 @@
"""Serper Tool Example.
This example demonstrates how to use the Serper (Google Search) tool with PraisonAI agents.
Requirements:
pip install "praisonai[tools]"
export SERPER_API_KEY=your_api_key
Usage:
python tool.py
"""
import os
from praisonai_tools import SerperTool
def main():
# Check for API key
if not os.getenv("SERPER_API_KEY"):
print("Error: SERPER_API_KEY environment variable not set")
print("Get your API key from https://serper.dev/")
return
# Initialize Serper tool
serper = SerperTool()
# Example 1: Web search
print("=" * 60)
print("Example 1: Google Web Search")
print("=" * 60)
results = serper.search("Python programming tutorials", max_results=3)
if results and "error" in results[0]:
print(f"Error: {results[0]['error']}")
else:
print(f"Found {len(results)} results:")
for r in results:
print(f" - {r.get('title', 'N/A')[:50]}...")
print(f" URL: {r.get('link', 'N/A')[:60]}...")
print()
print("✅ Serper tool working correctly!")
if __name__ == "__main__":
main()
+57
View File
@@ -0,0 +1,57 @@
"""Tavily Search Tool Example.
This example demonstrates how to use the Tavily search tool with PraisonAI agents.
Requirements:
pip install "praisonai[tools]"
export TAVILY_API_KEY=your_api_key
Usage:
python tool.py
"""
import os
from praisonai_tools import TavilyTool
def main():
# Check for API key
if not os.getenv("TAVILY_API_KEY"):
print("Error: TAVILY_API_KEY environment variable not set")
print("Get your API key from https://tavily.com/")
return
# Initialize Tavily tool
tavily = TavilyTool()
# Example 1: Basic search
print("=" * 60)
print("Example 1: Basic Search")
print("=" * 60)
results = tavily.search("What is quantum computing?", max_results=3)
if "error" in results:
print(f"Error: {results['error']}")
else:
print(f"Query: {results.get('query')}")
if "answer" in results:
print(f"\nAI Answer: {results['answer'][:200]}...")
print(f"\nResults ({len(results.get('results', []))}):")
for r in results.get("results", []):
print(f" - {r['title']}")
print(f" URL: {r['url']}")
print(f" Score: {r.get('score', 'N/A')}")
# Example 2: Search context for RAG
print("\n" + "=" * 60)
print("Example 2: Search Context (for RAG)")
print("=" * 60)
context = tavily.search_context("Python best practices 2024")
print(f"Context (first 300 chars): {context[:300]}...")
print("\n✅ Tavily tool working correctly!")
if __name__ == "__main__":
main()
+69
View File
@@ -0,0 +1,69 @@
"""Wikipedia Tool Example.
This example demonstrates how to use the Wikipedia tool with PraisonAI agents.
Requirements:
pip install "praisonai[tools]"
Usage:
python tool.py
Note: No API key required!
"""
from praisonai_tools import WikipediaTool
def main():
# Initialize Wikipedia tool (no API key needed)
wiki = WikipediaTool()
# Example 1: Search Wikipedia
print("=" * 60)
print("Example 1: Search Wikipedia")
print("=" * 60)
results = wiki.search("Python programming", max_results=5)
if results and "error" in results[0]:
print(f"Error: {results[0]['error']}")
else:
print(f"Found {len(results)} results:")
for r in results:
print(f" - {r['title']}")
# Example 2: Get page summary
print("\n" + "=" * 60)
print("Example 2: Get Page Summary")
print("=" * 60)
summary = wiki.summary("Artificial intelligence", sentences=3)
if "error" in summary:
print(f"Error: {summary['error']}")
else:
print(f"Title: {summary['title']}")
print(f"Summary: {summary['summary'][:300]}...")
# Example 3: Get full page
print("\n" + "=" * 60)
print("Example 3: Get Full Page")
print("=" * 60)
page = wiki.get_page("Machine learning")
if "error" in page:
print(f"Error: {page['error']}")
if "options" in page:
print(f"Options: {page['options'][:5]}")
else:
print(f"Title: {page['title']}")
print(f"URL: {page['url']}")
print(f"Categories: {page['categories'][:3]}")
print(f"Content preview: {page['content'][:200]}...")
print("\n✅ Wikipedia tool working correctly!")
if __name__ == "__main__":
main()
+1 -1
View File
@@ -11,7 +11,7 @@
framework: praisonai
process: workflow
topic: "Research AI trends and create a comprehensive report"
input: "Research AI trends and create a comprehensive report" # The input passed to {{input}} in steps
# Workflow configuration (new)
workflow:
+1 -1
View File
@@ -39,4 +39,4 @@ steps:
over: topics
- agent: summarizer
action: "Create a comprehensive summary of all the analyzed topics"
action: "Create a comprehensive summary of all the analyzed topics based on these results: {{previous_output}}"
@@ -3,6 +3,7 @@
name: Parallel Research Workflow
description: Multiple agents research different aspects concurrently, then an aggregator combines results
input: "AI-powered customer service solutions" # The input passed to {{input}} in steps
workflow:
verbose: true
@@ -43,4 +44,4 @@ steps:
action: "Research customer needs for: {{input}}"
- agent: aggregator
action: "Synthesize all the research findings into a comprehensive report"
action: "Synthesize all the research findings into a comprehensive report: {{previous_output}}"
@@ -3,6 +3,7 @@
name: Agentic Routing Workflow
description: A classifier agent routes requests to specialized handler agents
input: "Write a poem about the ocean" # The input passed to {{input}} in steps
workflow:
verbose: true
@@ -3,6 +3,7 @@
name: Simple Research Workflow
description: A simple workflow where a researcher gathers info and a writer summarizes it
input: "The benefits of renewable energy" # The input passed to {{input}} in steps
workflow:
verbose: true
+1
View File
@@ -87,3 +87,4 @@ graph.py
chroma_db/
.qodo
*_COMPARISON_ANALYSIS.md
*_REPORT.md
@@ -0,0 +1,35 @@
# PraisonAI Agents - Benchmark Results
**Generated:** 2025-12-18 14:40:22
**Iterations:** 100
**Test:** Agent instantiation (without tools)
## Results
| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
| **PraisonAI** | **3.77** | **1.00x (fastest)** |
| OpenAI Agents SDK | 5.26 | 1.39x |
| Agno | 5.64 | 1.49x |
| PraisonAI (LiteLLM) | 7.56 | 2.00x |
| PydanticAI | 226.94 | 60.16x |
| LangGraph | 4,558.71 | 1,209x |
| CrewAI | 15,607.92 | 4,138x |
## Package Versions
| Package | Version |
|---------|--------|
| PraisonAI | 0.1.5 |
| Agno | 2.3.14 |
| PydanticAI | 1.35.0 |
| OpenAI Agents SDK | 0.6.3 |
| LangGraph | 1.0.5 |
| CrewAI | 1.6.1 |
## How to Reproduce
```bash
cd praisonai-agents
python benchmarks/simple_benchmark.py
```
@@ -0,0 +1,31 @@
# PraisonAI Agents - Real Execution Benchmark
**Generated:** 2025-12-18 14:43:52
**Model:** gpt-4o-mini
**Iterations:** 3
**Prompt:** "What is 2+2? Reply with just the number."
## Results
| Framework | Method | Avg Time | Relative |
|-----------|--------|----------|----------|
| **PraisonAI** | `agent.start()` | **0.45s** | **1.00x (fastest)** |
| PraisonAI (LiteLLM) | `agent.start()` | 0.55s | 1.22x |
| CrewAI | `crew.kickoff()` | 0.58s | 1.28x |
| Agno | `agent.run()` | 0.92s | 2.05x |
## Package Versions
| Package | Version |
|---------|--------|
| PraisonAI | 0.1.5 |
| Agno | 2.3.14 |
| CrewAI | 1.6.1 |
## How to Reproduce
```bash
export OPENAI_API_KEY=your_key
cd praisonai-agents
python benchmarks/execution_benchmark.py
```
@@ -0,0 +1,72 @@
# PraisonAI Agents - Real-World Execution Benchmark
**Generated:** 2025-12-18 14:33:49
**Model:** gpt-4o-mini
**Iterations:** 3 per test
## Results Summary
| Test | PraisonAI (avg) | Agno (avg) | Difference |
|------|-----------------|------------|------------|
| Single Agent | 2.04s | 3.18s | PraisonAI 1.56x faster |
| 2 Agents | 2.49s | 20.49s | PraisonAI 8.21x faster |
| 3 Agents | 6.52s | 13.97s | PraisonAI 2.14x faster |
## Detailed Results
### praisonai_single
- Average: 2.04s
- Min: 1.95s
- Max: 2.19s
- Std Dev: 0.13s
- Times: ['2.19s', '1.95s', '2.00s']
### agno_single
- Average: 3.18s
- Min: 3.00s
- Max: 3.51s
- Std Dev: 0.28s
- Times: ['3.51s', '3.00s', '3.04s']
### praisonai_two_agents
- Average: 2.49s
- Min: 2.22s
- Max: 2.65s
- Std Dev: 0.24s
- Times: ['2.65s', '2.22s', '2.62s']
### agno_two_agents
- Average: 20.49s
- Min: 9.96s
- Max: 35.05s
- Std Dev: 13.02s
- Times: ['16.46s', '9.96s', '35.05s']
### praisonai_three_agents
- Average: 6.52s
- Min: 5.62s
- Max: 7.39s
- Std Dev: 0.88s
- Times: ['5.62s', '6.54s', '7.39s']
### agno_three_agents
- Average: 13.97s
- Min: 11.69s
- Max: 15.32s
- Std Dev: 1.99s
- Times: ['14.91s', '11.69s', '15.32s']
## Package Versions
| Package | Version |
|---------|--------|
| PraisonAI | 0.1.5 |
| Agno | 2.3.14 |
## How to Reproduce
```bash
export OPENAI_API_KEY=your_key
cd praisonai-agents
python benchmarks/real_benchmark.py
```
@@ -0,0 +1,22 @@
# PraisonAI Agents - Benchmark Results
**Generated:** 2025-12-18 13:23:43
## Agent Instantiation Time
| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
| **PraisonAI** | **4.16** | **1.00x (fastest)** |
| OpenAI Agents SDK | 5.55 | 1.33x |
| Agno | 5.58 | 1.34x |
| PraisonAI (LiteLLM) | 8.52 | 2.05x |
| PydanticAI | 225.87 | 54.25x |
| LangGraph | 4,349.54 | 1,045x |
| CrewAI | 15,911.11 | 3,821x |
## How to Reproduce
```bash
cd praisonai-agents
python benchmarks/simple_benchmark.py
```
@@ -0,0 +1,33 @@
# PraisonAI Agents - Tools Benchmark Results
**Generated:** 2025-12-18 14:40:35
**Iterations:** 100
**Test:** Agent instantiation WITH TOOLS
## Results
| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
| **PraisonAI** | **3.24** | **1.00x (fastest)** |
| Agno | 5.12 | 1.58x |
| PraisonAI (LiteLLM) | 8.59 | 2.65x |
| OpenAI Agents SDK | 279.95 | 86.44x |
| LangGraph | 2,310.82 | 713x |
| CrewAI | 15,773.44 | 4,870x |
## Package Versions
| Package | Version |
|---------|--------|
| PraisonAI | 0.1.5 |
| Agno | 2.3.14 |
| OpenAI Agents SDK | 0.6.3 |
| LangGraph | 1.0.5 |
| CrewAI | 1.6.1 |
## How to Reproduce
```bash
cd praisonai-agents
python benchmarks/tools_benchmark.py
```
@@ -0,0 +1,368 @@
#!/usr/bin/env python3
"""
PraisonAI Agents - Deep Profiling Benchmark
Profiles each step of agent initialization to identify performance bottlenecks.
Key Optimizations Applied:
1. Lazy RulesManager - filesystem operations deferred until first access
2. Lazy os.getcwd() - fast_context_path resolved only when needed
3. Lazy console initialization - Rich Console created on demand
Usage:
python benchmarks/deep_profiling.py
"""
import time
import tracemalloc
import cProfile
import pstats
import io
from typing import Literal, Callable, Dict, List
from dataclasses import dataclass
# ============================================================================
# Timing Utilities
# ============================================================================
@dataclass
class StepTiming:
name: str
duration_us: float # microseconds
memory_kb: float # kilobytes
class StepProfiler:
"""Profile individual steps during initialization"""
def __init__(self):
self.steps: List[StepTiming] = []
self._start_time = None
self._start_memory = None
def start_step(self, name: str):
tracemalloc.start()
self._start_time = time.perf_counter()
self._step_name = name
def end_step(self):
end_time = time.perf_counter()
current, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()
duration_us = (end_time - self._start_time) * 1_000_000
memory_kb = peak / 1024
self.steps.append(StepTiming(
name=self._step_name,
duration_us=duration_us,
memory_kb=memory_kb
))
def print_report(self, title: str):
print(f"\n{'='*70}")
print(f"STEP-BY-STEP PROFILING: {title}")
print(f"{'='*70}")
total_time = sum(s.duration_us for s in self.steps)
total_memory = sum(s.memory_kb for s in self.steps)
print(f"\n{'Step':<40} {'Time (μs)':<15} {'Memory (KB)':<15} {'% Time':<10}")
print("-" * 80)
for step in sorted(self.steps, key=lambda x: x.duration_us, reverse=True):
pct = (step.duration_us / total_time * 100) if total_time > 0 else 0
print(f"{step.name:<40} {step.duration_us:<15.2f} {step.memory_kb:<15.2f} {pct:<10.1f}%")
print("-" * 80)
print(f"{'TOTAL':<40} {total_time:<15.2f} {total_memory:<15.2f} {'100.0':<10}%")
def time_function(func: Callable, iterations: int = 100) -> Dict:
"""Time a function with detailed statistics"""
times = []
# Warmup
for _ in range(10):
func()
# Measure
for _ in range(iterations):
start = time.perf_counter()
func()
end = time.perf_counter()
times.append((end - start) * 1_000_000) # Convert to microseconds
return {
'avg': sum(times) / len(times),
'min': min(times),
'max': max(times),
'median': sorted(times)[len(times)//2]
}
# ============================================================================
# Profile PraisonAI Agent Initialization - Step by Step
# ============================================================================
def profile_praisonai_detailed():
"""Profile each step of PraisonAI Agent initialization"""
print("\n" + "="*70)
print("DETAILED PROFILING: PraisonAI Agent")
print("="*70)
profiler = StepProfiler()
# Step 1: Import
profiler.start_step("1. Import praisonaiagents")
from praisonaiagents import Agent as PraisonAgent
profiler.end_step()
# Step 2: Define tool
profiler.start_step("2. Define tool function")
def get_weather(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
tools = [get_weather]
profiler.end_step()
# Step 3: Full instantiation
profiler.start_step("3. Full Agent instantiation")
agent = PraisonAgent(
name="Test Agent",
instructions="Be concise, reply with one sentence.",
llm="gpt-4o-mini",
tools=tools,
verbose=False
)
profiler.end_step()
profiler.print_report("PraisonAI Agent")
# Now profile the __init__ internals using cProfile
print("\n" + "="*70)
print("cProfile ANALYSIS: PraisonAI Agent.__init__")
print("="*70)
pr = cProfile.Profile()
pr.enable()
for _ in range(100):
agent = PraisonAgent(
name="Test Agent",
instructions="Be concise, reply with one sentence.",
llm="gpt-4o-mini",
tools=tools,
verbose=False
)
pr.disable()
# Print top 30 functions by cumulative time
s = io.StringIO()
ps = pstats.Stats(pr, stream=s).sort_stats('cumulative')
ps.print_stats(30)
print(s.getvalue())
return agent
# ============================================================================
# Profile Agno Agent Initialization - Step by Step
# ============================================================================
def profile_agno_detailed():
"""Profile each step of Agno Agent initialization"""
print("\n" + "="*70)
print("DETAILED PROFILING: Agno Agent")
print("="*70)
profiler = StepProfiler()
# Step 1: Import
profiler.start_step("1. Import agno")
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
profiler.end_step()
# Step 2: Define tool
profiler.start_step("2. Define tool function")
def get_weather(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
tools = [get_weather]
profiler.end_step()
# Step 3: Full instantiation
profiler.start_step("3. Full Agent instantiation")
agent = AgnoAgent(model=OpenAIChat(id="gpt-4o-mini"), tools=tools)
profiler.end_step()
profiler.print_report("Agno Agent")
# Now profile the __init__ internals using cProfile
print("\n" + "="*70)
print("cProfile ANALYSIS: Agno Agent.__init__")
print("="*70)
pr = cProfile.Profile()
pr.enable()
for _ in range(100):
agent = AgnoAgent(model=OpenAIChat(id="gpt-4o-mini"), tools=tools)
pr.disable()
# Print top 30 functions by cumulative time
s = io.StringIO()
ps = pstats.Stats(pr, stream=s).sort_stats('cumulative')
ps.print_stats(30)
print(s.getvalue())
return agent
# ============================================================================
# Profile Individual Components of PraisonAI
# ============================================================================
def profile_praisonai_components():
"""Profile individual components of PraisonAI Agent initialization"""
print("\n" + "="*70)
print("COMPONENT-LEVEL PROFILING: PraisonAI Agent")
print("="*70)
from praisonaiagents import Agent as PraisonAgent
def get_weather(city: Literal["nyc", "sf"]):
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
tools = [get_weather]
# Test different configurations
configs = [
("Minimal (no tools, no llm)", {"name": "Test", "verbose": False}),
("With name only", {"name": "Test Agent", "verbose": False}),
("With instructions", {"name": "Test", "instructions": "Be concise", "verbose": False}),
("With llm (gpt-4o-mini)", {"name": "Test", "llm": "gpt-4o-mini", "verbose": False}),
("With llm (openai/gpt-4o-mini)", {"name": "Test", "llm": "openai/gpt-4o-mini", "verbose": False}),
("With tools", {"name": "Test", "tools": tools, "verbose": False}),
("Full config (OpenAI SDK)", {"name": "Test", "instructions": "Be concise", "llm": "gpt-4o-mini", "tools": tools, "verbose": False}),
("Full config (LiteLLM)", {"name": "Test", "instructions": "Be concise", "llm": "openai/gpt-4o-mini", "tools": tools, "verbose": False}),
]
print(f"\n{'Configuration':<40} {'Avg Time (μs)':<15} {'Min (μs)':<15} {'Max (μs)':<15}")
print("-" * 85)
for name, config in configs:
def create_agent():
return PraisonAgent(**config)
stats = time_function(create_agent, iterations=100)
print(f"{name:<40} {stats['avg']:<15.2f} {stats['min']:<15.2f} {stats['max']:<15.2f}")
# ============================================================================
# Compare Specific Operations
# ============================================================================
def compare_specific_operations():
"""Compare specific operations between PraisonAI and Agno"""
print("\n" + "="*70)
print("OPERATION COMPARISON: PraisonAI vs Agno")
print("="*70)
from praisonaiagents import Agent as PraisonAgent
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
def get_weather(city: Literal["nyc", "sf"]):
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
tools = [get_weather]
operations = []
# Operation 1: Bare minimum instantiation
def praisonai_minimal():
return PraisonAgent(name="Test", verbose=False)
def agno_minimal():
return AgnoAgent(model=OpenAIChat(id="gpt-4o-mini"))
praison_stats = time_function(praisonai_minimal, 100)
agno_stats = time_function(agno_minimal, 100)
operations.append(("Minimal instantiation", praison_stats['avg'], agno_stats['avg']))
# Operation 2: With tools
def praisonai_with_tools():
return PraisonAgent(name="Test", tools=tools, verbose=False)
def agno_with_tools():
return AgnoAgent(model=OpenAIChat(id="gpt-4o-mini"), tools=tools)
praison_stats = time_function(praisonai_with_tools, 100)
agno_stats = time_function(agno_with_tools, 100)
operations.append(("With tools", praison_stats['avg'], agno_stats['avg']))
# Operation 3: Full config
def praisonai_full():
return PraisonAgent(
name="Test Agent",
instructions="Be concise",
llm="gpt-4o-mini",
tools=tools,
verbose=False
)
def agno_full():
return AgnoAgent(
model=OpenAIChat(id="gpt-4o-mini"),
tools=tools,
instructions=["Be concise"]
)
praison_stats = time_function(praisonai_full, 100)
agno_stats = time_function(agno_full, 100)
operations.append(("Full config", praison_stats['avg'], agno_stats['avg']))
print(f"\n{'Operation':<30} {'PraisonAI (μs)':<18} {'Agno (μs)':<18} {'Ratio':<10}")
print("-" * 76)
for op_name, praison_time, agno_time in operations:
ratio = praison_time / agno_time if agno_time > 0 else 0
print(f"{op_name:<30} {praison_time:<18.2f} {agno_time:<18.2f} {ratio:<10.1f}x")
# ============================================================================
# Main
# ============================================================================
if __name__ == "__main__":
import os
os.environ.setdefault("OPENAI_API_KEY", "sk-test")
print("="*70)
print("PraisonAI Agents - Deep Profiling Benchmark")
print("="*70)
print("\nThis analysis identifies performance bottlenecks in agent initialization.\n")
# Run all profiling
profile_praisonai_detailed()
profile_agno_detailed()
profile_praisonai_components()
compare_specific_operations()
print("\n" + "="*70)
print("PROFILING COMPLETE")
print("="*70)
@@ -0,0 +1,296 @@
#!/usr/bin/env python3
"""
PraisonAI Agents - Real Execution Benchmark
Benchmarks ACTUAL agent execution with LLM API calls.
Tests single agent running a simple task across frameworks.
Usage:
export OPENAI_API_KEY=your_key
python benchmarks/execution_benchmark.py
python benchmarks/execution_benchmark.py --no-save
python benchmarks/execution_benchmark.py --iterations 5
python benchmarks/execution_benchmark.py --model gpt-4o --iterations 10
"""
import time
import os
import argparse
from importlib.metadata import version as get_version
# Defaults (can be overridden via CLI)
DEFAULT_MODEL = "gpt-4o-mini"
DEFAULT_ITERATIONS = 3
DEFAULT_PROMPT = "What is 2+2? Reply with just the number."
def get_package_versions():
"""Get version numbers for benchmarked packages."""
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
'CrewAI': 'crewai',
}
versions = {}
for name, pkg in packages.items():
try:
versions[name] = get_version(pkg)
except:
versions[name] = 'not installed'
return versions
def benchmark_praisonai(model, iterations, prompt):
"""Benchmark PraisonAI agent execution using agent.start()"""
from praisonaiagents import Agent
print("\n--- PraisonAI ---")
print("Method: agent.start()")
times = []
for i in range(iterations):
agent = Agent(
name="Calculator",
instructions="You are a helpful assistant. Be very brief.",
llm=model,
verbose=False
)
start = time.perf_counter()
response = agent.start(prompt)
elapsed = time.perf_counter() - start
times.append(elapsed)
print(f" Run {i+1}: {elapsed:.2f}s - Response: {str(response)[:50]}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def benchmark_praisonai_litellm(model, iterations, prompt):
"""Benchmark PraisonAI with LiteLLM backend using agent.start()"""
from praisonaiagents import Agent
print("\n--- PraisonAI (LiteLLM) ---")
print("Method: agent.start()")
times = []
for i in range(iterations):
agent = Agent(
name="Calculator",
instructions="You are a helpful assistant. Be very brief.",
llm=f"openai/{model}",
verbose=False
)
start = time.perf_counter()
response = agent.start(prompt)
elapsed = time.perf_counter() - start
times.append(elapsed)
print(f" Run {i+1}: {elapsed:.2f}s - Response: {str(response)[:50]}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def benchmark_agno(model, iterations, prompt):
"""Benchmark Agno agent execution using agent.run()"""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
print("\n--- Agno ---")
print("Method: agent.run()")
times = []
for i in range(iterations):
agent = Agent(
model=OpenAIChat(id=model),
instructions=["You are a helpful assistant. Be very brief."],
)
start = time.perf_counter()
response = agent.run(prompt)
elapsed = time.perf_counter() - start
times.append(elapsed)
# Extract content from response
content = str(response.content)[:50] if hasattr(response, 'content') else str(response)[:50]
print(f" Run {i+1}: {elapsed:.2f}s - Response: {content}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def benchmark_crewai(model, iterations, prompt):
"""Benchmark CrewAI agent execution using crew.kickoff()"""
from crewai import Agent, Task, Crew
print("\n--- CrewAI ---")
print("Method: crew.kickoff()")
times = []
for i in range(iterations):
agent = Agent(
role="Calculator",
goal="Answer math questions",
backstory="You are a helpful assistant.",
llm=model,
verbose=False
)
task = Task(
description=prompt,
expected_output="A number",
agent=agent
)
crew = Crew(
agents=[agent],
tasks=[task],
verbose=False
)
start = time.perf_counter()
response = crew.kickoff()
elapsed = time.perf_counter() - start
times.append(elapsed)
print(f" Run {i+1}: {elapsed:.2f}s - Response: {str(response)[:50]}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def save_results(results: dict, model: str, iterations: int, prompt: str):
"""Save results to markdown file."""
import os
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), 'EXECUTION_BENCHMARK_RESULTS.md')
versions = get_package_versions()
# Sort by time
sorted_results = sorted(results.items(), key=lambda x: x[1])
fastest = sorted_results[0][1]
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Real Execution Benchmark\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write(f'**Model:** {model}\n')
f.write(f'**Iterations:** {iterations}\n')
f.write(f'**Prompt:** "{prompt}"\n\n')
f.write('## Results\n\n')
f.write('| Framework | Method | Avg Time | Relative |\n')
f.write('|-----------|--------|----------|----------|\n')
methods = {
'PraisonAI': 'agent.start()',
'PraisonAI (LiteLLM)': 'agent.start()',
'Agno': 'agent.run()',
'CrewAI': 'crew.kickoff()',
}
for name, avg in sorted_results:
ratio = avg / fastest
method = methods.get(name, 'unknown')
if ratio == 1.0:
f.write(f'| **{name}** | `{method}` | **{avg:.2f}s** | **1.00x (fastest)** |\n')
else:
f.write(f'| {name} | `{method}` | {avg:.2f}s | {ratio:.2f}x |\n')
f.write('\n## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('export OPENAI_API_KEY=your_key\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/execution_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='PraisonAI Agents - Real Execution Benchmark')
parser.add_argument('--no-save', action='store_true', help='Do not save results to file')
parser.add_argument('--model', '-m', type=str, default=DEFAULT_MODEL, help=f'Model to use (default: {DEFAULT_MODEL})')
parser.add_argument('--iterations', '-i', type=int, default=DEFAULT_ITERATIONS, help=f'Number of iterations (default: {DEFAULT_ITERATIONS})')
parser.add_argument('--prompt', '-p', type=str, default=DEFAULT_PROMPT, help=f'Prompt to use (default: "{DEFAULT_PROMPT}")')
args = parser.parse_args()
# Check API key
if not os.environ.get('OPENAI_API_KEY'):
print("ERROR: Set OPENAI_API_KEY environment variable")
exit(1)
model = args.model
iterations = args.iterations
prompt = args.prompt
print('=' * 60)
print('PraisonAI Agents - Real Execution Benchmark')
print('=' * 60)
print(f'\nModel: {model}')
print(f'Iterations: {iterations}')
print(f'Prompt: "{prompt}"')
versions = get_package_versions()
print('\nPackage versions:')
for pkg, ver in versions.items():
print(f' {pkg}: {ver}')
results = {}
# Run benchmarks
try:
results['PraisonAI'] = benchmark_praisonai(model, iterations, prompt)
except Exception as e:
print(f"PraisonAI error: {e}")
try:
results['PraisonAI (LiteLLM)'] = benchmark_praisonai_litellm(model, iterations, prompt)
except Exception as e:
print(f"PraisonAI (LiteLLM) error: {e}")
try:
results['Agno'] = benchmark_agno(model, iterations, prompt)
except Exception as e:
print(f"Agno error: {e}")
try:
results['CrewAI'] = benchmark_crewai(model, iterations, prompt)
except Exception as e:
print(f"CrewAI error: {e}")
# Summary
print('\n' + '=' * 60)
print('SUMMARY')
print('=' * 60)
if results:
sorted_results = sorted(results.items(), key=lambda x: x[1])
fastest = sorted_results[0][1]
print(f"\n{'Framework':<25} {'Avg Time':<12} {'Relative':<10}")
print('-' * 47)
for name, avg in sorted_results:
ratio = avg / fastest
print(f'{name:<25} {avg:<12.2f}s {ratio:.2f}x')
print('\n' + '=' * 60)
# Save results (unless --no-save)
if results and not args.no_save:
save_results(results, model, iterations, prompt)
elif args.no_save:
print('\nResults not saved (--no-save flag used)')
@@ -0,0 +1,523 @@
"""
PraisonAI Agents - Comprehensive Performance Benchmark
Measures agent instantiation time and memory footprint across
multiple AI agent frameworks using industry-standard methodology.
Usage:
python benchmarks/performance_benchmark.py
"""
import gc
import time
import tracemalloc
import statistics
from typing import Literal, List, Callable
from dataclasses import dataclass, field
@dataclass
class PerformanceResult:
"""Performance measurement results"""
run_times: List[float] = field(default_factory=list)
memory_usages: List[float] = field(default_factory=list)
@property
def avg_run_time(self) -> float:
return statistics.mean(self.run_times) if self.run_times else 0
@property
def avg_memory_usage(self) -> float:
return statistics.mean(self.memory_usages) if self.memory_usages else 0
@property
def min_run_time(self) -> float:
return min(self.run_times) if self.run_times else 0
@property
def max_run_time(self) -> float:
return max(self.run_times) if self.run_times else 0
@property
def std_dev_run_time(self) -> float:
return statistics.stdev(self.run_times) if len(self.run_times) > 1 else 0
@property
def median_run_time(self) -> float:
return statistics.median(self.run_times) if self.run_times else 0
def compute_tracemalloc_baseline(samples: int = 3) -> float:
"""Compute baseline memory usage for tracemalloc"""
def empty_func():
return
results = []
for _ in range(samples):
gc.collect()
tracemalloc.start()
empty_func()
_, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()
results.append(peak / 1024 / 1024) # Convert to MiB
return sum(results) / len(results) if results else 0
def measure_time(func: Callable, iterations: int = 1000, warmup: int = 10) -> List[float]:
"""Measure execution time for a function"""
# Warmup runs
for _ in range(warmup):
func()
# Measured runs
times = []
for _ in range(iterations):
start = time.perf_counter()
func()
end = time.perf_counter()
times.append(end - start)
return times
def measure_memory(func: Callable, iterations: int = 1000, warmup: int = 10) -> List[float]:
"""Measure memory usage for a function"""
baseline = compute_tracemalloc_baseline()
# Warmup runs
for _ in range(warmup):
func()
# Measured runs
memory_usages = []
for _ in range(iterations):
gc.collect()
tracemalloc.start()
func()
_, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()
peak_mib = peak / 1024 / 1024
adjusted = max(0, peak_mib - baseline)
memory_usages.append(adjusted)
return memory_usages
def run_benchmark(func: Callable, name: str, iterations: int = 1000, warmup: int = 10) -> PerformanceResult:
"""Run full benchmark for a function"""
print(f"\n{'='*60}")
print(f"Benchmarking: {name}")
print(f"{'='*60}")
print(f"Warmup runs: {warmup}")
print(f"Measured iterations: {iterations}")
print("\nMeasuring runtime...")
run_times = measure_time(func, iterations, warmup)
print("Measuring memory...")
memory_usages = measure_memory(func, iterations, warmup)
result = PerformanceResult(run_times=run_times, memory_usages=memory_usages)
print(f"\nResults for {name}:")
print(f" Average Time: {result.avg_run_time:.6f} seconds ({result.avg_run_time * 1_000_000:.2f} μs)")
print(f" Min Time: {result.min_run_time:.6f} seconds")
print(f" Max Time: {result.max_run_time:.6f} seconds")
print(f" Std Dev: {result.std_dev_run_time:.6f} seconds")
print(f" Median Time: {result.median_run_time:.6f} seconds")
print(f" Avg Memory: {result.avg_memory_usage:.6f} MiB ({result.avg_memory_usage * 1024:.2f} KiB)")
return result
# ============================================================================
# Framework Imports
# ============================================================================
from praisonaiagents import Agent as PraisonAgent
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
# Additional frameworks for comparison
try:
from pydantic_ai import Agent as PydanticAgent
PYDANTIC_AVAILABLE = True
except ImportError:
PYDANTIC_AVAILABLE = False
try:
from crewai.agent import Agent as CrewAgent
from crewai.tools import tool as crewai_tool
CREWAI_AVAILABLE = True
except ImportError:
CREWAI_AVAILABLE = False
try:
from langchain_core.tools import tool as langchain_tool
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
LANGGRAPH_AVAILABLE = True
except ImportError:
LANGGRAPH_AVAILABLE = False
try:
from agents import Agent as OpenAIAgent, function_tool
OPENAI_AGENTS_AVAILABLE = True
except ImportError:
OPENAI_AGENTS_AVAILABLE = False
def get_weather(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
tools = [get_weather]
def instantiate_praisonai_agent():
"""Instantiate a PraisonAI agent with one tool."""
return PraisonAgent(
name="Test Agent",
instructions="Be concise, reply with one sentence.",
llm="gpt-4o-mini",
tools=tools,
verbose=False
)
def instantiate_praisonai_litellm_agent():
"""Instantiate a PraisonAI agent with LiteLLM backend."""
return PraisonAgent(
name="Test Agent",
instructions="Be concise, reply with one sentence.",
llm="openai/gpt-4o-mini",
tools=tools,
verbose=False
)
def instantiate_agno_agent():
"""Instantiate an Agno agent with one tool."""
return AgnoAgent(model=OpenAIChat(id="gpt-4o-mini"), tools=tools)
def instantiate_pydantic_agent():
"""Instantiate a PydanticAI agent with one tool."""
agent = PydanticAgent("openai:gpt-4o-mini", system_prompt="Be concise, reply with one sentence.")
@agent.tool_plain
def get_weather_pydantic(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
return agent
def instantiate_openai_agents():
"""Instantiate an OpenAI Agents SDK agent with one tool."""
return OpenAIAgent(
name="Test agent",
instructions="Be concise, reply with one sentence.",
model="gpt-4o-mini",
tools=[function_tool(get_weather)],
)
def instantiate_langgraph_agent():
"""Instantiate a LangGraph agent with one tool."""
@langchain_tool
def get_weather_lg(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
return create_react_agent(model=ChatOpenAI(model="gpt-4o-mini"), tools=[get_weather_lg])
def instantiate_crewai_agent():
"""Instantiate a CrewAI agent with one tool."""
@crewai_tool("Weather Tool")
def get_weather_crew(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
return CrewAgent(
llm="gpt-4o-mini",
role="Test Agent",
goal="Be concise, reply with one sentence.",
tools=[get_weather_crew],
backstory="Test",
)
def get_package_versions():
"""Get version numbers for all benchmarked packages."""
from importlib.metadata import version as get_version
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
'PydanticAI': 'pydantic-ai',
'OpenAI Agents SDK': 'openai-agents',
'LangGraph': 'langgraph',
'CrewAI': 'crewai'
}
versions = {}
for display_name, pkg_name in packages.items():
try:
versions[display_name] = get_version(pkg_name)
except Exception:
versions[display_name] = 'not installed'
return versions
def save_benchmark_results(results: dict, baseline):
"""Save benchmark results to BENCHMARK_RESULTS.md and update README.md"""
import os
import re
from datetime import datetime
benchmarks_dir = os.path.dirname(__file__)
filepath = os.path.join(benchmarks_dir, 'BENCHMARK_RESULTS.md')
# Get package versions
versions = get_package_versions()
display_names = {
'agno': 'Agno',
'praisonai': 'PraisonAI',
'praisonai_litellm': 'PraisonAI (LiteLLM)',
'pydantic': 'PydanticAI',
'openai_agents': 'OpenAI Agents SDK',
'langgraph': 'LangGraph',
'crewai': 'CrewAI'
}
# Build table rows
table_rows = []
sorted_results = sorted(results.items(), key=lambda x: x[1].avg_run_time)
for name, result in sorted_results:
display_name = display_names.get(name, name)
avg_us = result.avg_run_time * 1e6
ratio = result.avg_run_time / baseline.avg_run_time if baseline.avg_run_time > 0 else 0
if name == 'praisonai':
table_rows.append(f'| **{display_name}** | **{avg_us:.2f}** | **1.00x (fastest)** |')
else:
avg_str = f'{avg_us:,.2f}' if avg_us >= 1000 else f'{avg_us:.2f}'
ratio_str = f'{ratio:,.0f}x' if ratio >= 100 else f'{ratio:.2f}x'
table_rows.append(f'| {display_name} | {avg_str} | {ratio_str} |')
# Save to BENCHMARK_RESULTS.md
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Benchmark Results\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write('**Methodology:** 10 warmup runs + 1000 measured iterations\n\n')
f.write('## Agent Instantiation Time\n\n')
f.write('| Framework | Avg Time (μs) | Relative |\n')
f.write('|-----------|---------------|----------|\n')
f.write('\n'.join(table_rows) + '\n')
f.write('\n## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/performance_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
# Update README.md
readme_path = os.path.join(benchmarks_dir, '..', '..', '..', 'README.md')
if os.path.exists(readme_path):
with open(readme_path, 'r') as f:
content = f.read()
new_table = '''| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
''' + '\n'.join(table_rows)
pattern = r'(\| Framework \| Avg Time \(μs\) \| Relative \|\n\|[-|]+\|\n)(\|[^\n]+\|\n)+'
if re.search(pattern, content):
content = re.sub(pattern, new_table + '\n', content)
with open(readme_path, 'w') as f:
f.write(content)
print(f'README.md updated: {readme_path}')
if __name__ == "__main__":
print("="*70)
print("PraisonAI Agents - Comprehensive Performance Benchmark")
print("="*70)
print("\nThis benchmark measures:")
print(" 1. Agent instantiation time (how fast an agent object is created)")
print(" 2. Memory footprint (how much memory an agent uses)")
print("\nMethodology:")
print(" - 10 warmup runs (not measured)")
print(" - 1000 measured iterations")
print(" - tracemalloc for memory measurement")
print(" - Baseline subtraction for accuracy")
print("\nFrameworks available:")
print(" - PraisonAI: ✅")
print(f" - Agno: ✅")
print(f" - PydanticAI: {'✅' if PYDANTIC_AVAILABLE else '❌'}")
print(f" - OpenAI Agents SDK: {'✅' if OPENAI_AGENTS_AVAILABLE else '❌'}")
print(f" - LangGraph: {'✅' if LANGGRAPH_AVAILABLE else '❌'}")
print(f" - CrewAI: {'✅' if CREWAI_AVAILABLE else '❌'}")
results = {}
# Test PraisonAI first
try:
print("\n" + "="*70)
print("Testing PraisonAI Agents")
print("="*70)
results['praisonai'] = run_benchmark(
instantiate_praisonai_agent,
"PraisonAI",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing PraisonAI: {e}")
try:
print("\n" + "="*70)
print("Testing PraisonAI Agents (LiteLLM)")
print("="*70)
results['praisonai_litellm'] = run_benchmark(
instantiate_praisonai_litellm_agent,
"PraisonAI (LiteLLM)",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing PraisonAI (LiteLLM): {e}")
# Test Agno
try:
print("\n" + "="*70)
print("Testing Agno")
print("="*70)
results['agno'] = run_benchmark(
instantiate_agno_agent,
"Agno",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing Agno: {e}")
# Test PydanticAI
if PYDANTIC_AVAILABLE:
try:
print("\n" + "="*70)
print("Testing PydanticAI")
print("="*70)
results['pydantic'] = run_benchmark(
instantiate_pydantic_agent,
"PydanticAI Agent Instantiation",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing PydanticAI: {e}")
# Test OpenAI Agents SDK
if OPENAI_AGENTS_AVAILABLE:
try:
print("\n" + "="*70)
print("Testing OpenAI Agents SDK")
print("="*70)
results['openai_agents'] = run_benchmark(
instantiate_openai_agents,
"OpenAI Agents SDK Instantiation",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing OpenAI Agents SDK: {e}")
# Test LangGraph
if LANGGRAPH_AVAILABLE:
try:
print("\n" + "="*70)
print("Testing LangGraph")
print("="*70)
results['langgraph'] = run_benchmark(
instantiate_langgraph_agent,
"LangGraph Agent Instantiation",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing LangGraph: {e}")
# Test CrewAI
if CREWAI_AVAILABLE:
try:
print("\n" + "="*70)
print("Testing CrewAI")
print("="*70)
results['crewai'] = run_benchmark(
instantiate_crewai_agent,
"CrewAI Agent Instantiation",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing CrewAI: {e}")
# Final Comparison Table
print("\n" + "="*70)
print("RESULTS SUMMARY")
print("="*70)
if 'praisonai' in results:
baseline = results['praisonai']
print(f"\n{'Framework':<25} {'Avg Time (μs)':<15} {'Avg Memory (KiB)':<18} {'Relative':<15}")
print("-" * 73)
# Sort by time
sorted_results = sorted(results.items(), key=lambda x: x[1].avg_run_time)
for name, result in sorted_results:
time_ratio = result.avg_run_time / baseline.avg_run_time if baseline.avg_run_time > 0 else 0
time_str = f"{time_ratio:.2f}x"
display_name = {
'agno': 'Agno',
'praisonai': 'PraisonAI',
'praisonai_litellm': 'PraisonAI (LiteLLM)',
'pydantic': 'PydanticAI',
'openai_agents': 'OpenAI Agents SDK',
'langgraph': 'LangGraph',
'crewai': 'CrewAI'
}.get(name, name)
print(f"{display_name:<25} {result.avg_run_time*1e6:<15.2f} {result.avg_memory_usage*1024:<18.2f} {time_str:<15}")
print("\n" + "="*70)
# Save results to files
save_benchmark_results(results, baseline)
@@ -0,0 +1,481 @@
#!/usr/bin/env python3
"""
PraisonAI Agents - Real-World Execution Benchmark
Benchmarks actual agent execution with LLM calls across frameworks.
Tests single agent, 2-agent, and 3-agent workflows.
Usage:
python benchmarks/real_benchmark.py
"""
import time
import statistics
import os
from typing import Literal, List
from dataclasses import dataclass, field
from importlib.metadata import version as get_version
# Ensure API key is set
if not os.environ.get('OPENAI_API_KEY'):
print("ERROR: OPENAI_API_KEY environment variable not set")
exit(1)
@dataclass
class ExecutionResult:
"""Execution benchmark results"""
times: List[float] = field(default_factory=list)
@property
def avg_time(self) -> float:
return statistics.mean(self.times) if self.times else 0
@property
def min_time(self) -> float:
return min(self.times) if self.times else 0
@property
def max_time(self) -> float:
return max(self.times) if self.times else 0
@property
def std_dev(self) -> float:
return statistics.stdev(self.times) if len(self.times) > 1 else 0
def get_package_versions():
"""Get version numbers for benchmarked packages."""
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
}
versions = {}
for display_name, pkg_name in packages.items():
try:
versions[display_name] = get_version(pkg_name)
except Exception:
versions[display_name] = 'not installed'
return versions
# ============================================================================
# Tool Definitions
# ============================================================================
def get_weather(city: Literal["nyc", "sf"]) -> str:
"""Get weather information for a city."""
if city == "nyc":
return "New York: Cloudy, 45°F, 60% humidity"
return "San Francisco: Sunny, 72°F, 40% humidity"
def get_time(timezone: Literal["est", "pst"]) -> str:
"""Get current time in a timezone."""
if timezone == "est":
return "Eastern Time: 2:30 PM"
return "Pacific Time: 11:30 AM"
def calculate(expression: str) -> str:
"""Calculate a math expression."""
try:
result = eval(expression)
return f"Result: {result}"
except Exception:
return "Error: Invalid expression"
# ============================================================================
# PraisonAI Benchmarks
# ============================================================================
def benchmark_praisonai_single(iterations: int = 3) -> ExecutionResult:
"""Benchmark single PraisonAI agent execution."""
from praisonaiagents import Agent
result = ExecutionResult()
for i in range(iterations):
print(f" PraisonAI single agent iteration {i+1}/{iterations}...")
agent = Agent(
name="Assistant",
instructions="You are a helpful assistant. Be very brief.",
llm="gpt-4o-mini",
tools=[get_weather],
verbose=False
)
start = time.perf_counter()
response = agent.start("What's the weather in NYC? Reply in one sentence.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
def benchmark_praisonai_two_agents(iterations: int = 3) -> ExecutionResult:
"""Benchmark 2 PraisonAI agents working together."""
from praisonaiagents import Agent, PraisonAIAgents
result = ExecutionResult()
for i in range(iterations):
print(f" PraisonAI 2-agent iteration {i+1}/{iterations}...")
researcher = Agent(
name="Researcher",
role="Research Assistant",
goal="Gather weather information",
instructions="Get weather data and pass to reporter. Be brief.",
llm="gpt-4o-mini",
tools=[get_weather],
verbose=False
)
reporter = Agent(
name="Reporter",
role="News Reporter",
goal="Create weather report",
instructions="Create a one-sentence weather summary.",
llm="gpt-4o-mini",
verbose=False
)
agents = PraisonAIAgents(
agents=[researcher, reporter],
process="sequential",
verbose=False
)
start = time.perf_counter()
response = agents.start("Get NYC weather and create a brief report.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
def benchmark_praisonai_three_agents(iterations: int = 3) -> ExecutionResult:
"""Benchmark 3 PraisonAI agents working together."""
from praisonaiagents import Agent, PraisonAIAgents
result = ExecutionResult()
for i in range(iterations):
print(f" PraisonAI 3-agent iteration {i+1}/{iterations}...")
weather_agent = Agent(
name="WeatherAgent",
role="Weather Specialist",
goal="Get weather data",
instructions="Fetch weather information. Be brief.",
llm="gpt-4o-mini",
tools=[get_weather],
verbose=False
)
time_agent = Agent(
name="TimeAgent",
role="Time Specialist",
goal="Get time information",
instructions="Fetch timezone information. Be brief.",
llm="gpt-4o-mini",
tools=[get_time],
verbose=False
)
summarizer = Agent(
name="Summarizer",
role="Summary Writer",
goal="Combine information",
instructions="Create a one-sentence summary combining weather and time.",
llm="gpt-4o-mini",
verbose=False
)
agents = PraisonAIAgents(
agents=[weather_agent, time_agent, summarizer],
process="sequential",
verbose=False
)
start = time.perf_counter()
response = agents.start("Get NYC weather and EST time, then summarize.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
# ============================================================================
# Agno Benchmarks
# ============================================================================
def benchmark_agno_single(iterations: int = 3) -> ExecutionResult:
"""Benchmark single Agno agent execution."""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
result = ExecutionResult()
for i in range(iterations):
print(f" Agno single agent iteration {i+1}/{iterations}...")
agent = Agent(
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["You are a helpful assistant. Be very brief."],
tools=[get_weather],
)
start = time.perf_counter()
response = agent.run("What's the weather in NYC? Reply in one sentence.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
def benchmark_agno_two_agents(iterations: int = 3) -> ExecutionResult:
"""Benchmark 2 Agno agents working together (using Team)."""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.team import Team
result = ExecutionResult()
for i in range(iterations):
print(f" Agno 2-agent iteration {i+1}/{iterations}...")
researcher = Agent(
name="Researcher",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Get weather data. Be brief."],
tools=[get_weather],
)
reporter = Agent(
name="Reporter",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Create a one-sentence weather summary."],
)
team = Team(
members=[researcher, reporter],
model=OpenAIChat(id="gpt-4o-mini"),
)
start = time.perf_counter()
response = team.run("Get NYC weather and create a brief report.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
def benchmark_agno_three_agents(iterations: int = 3) -> ExecutionResult:
"""Benchmark 3 Agno agents working together."""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.team import Team
result = ExecutionResult()
for i in range(iterations):
print(f" Agno 3-agent iteration {i+1}/{iterations}...")
weather_agent = Agent(
name="WeatherAgent",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Fetch weather information. Be brief."],
tools=[get_weather],
)
time_agent = Agent(
name="TimeAgent",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Fetch timezone information. Be brief."],
tools=[get_time],
)
summarizer = Agent(
name="Summarizer",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Create a one-sentence summary combining weather and time."],
)
team = Team(
members=[weather_agent, time_agent, summarizer],
model=OpenAIChat(id="gpt-4o-mini"),
)
start = time.perf_counter()
response = team.run("Get NYC weather and EST time, then summarize.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
# ============================================================================
# Save Results
# ============================================================================
def save_results(results: dict, versions: dict):
"""Save benchmark results to file."""
import os
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), 'REAL_BENCHMARK_RESULTS.md')
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Real-World Execution Benchmark\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write('**Model:** gpt-4o-mini\n')
f.write('**Iterations:** 3 per test\n\n')
f.write('## Results Summary\n\n')
f.write('| Test | PraisonAI (avg) | Agno (avg) | Difference |\n')
f.write('|------|-----------------|------------|------------|\n')
tests = ['single', 'two_agents', 'three_agents']
test_names = ['Single Agent', '2 Agents', '3 Agents']
for test, name in zip(tests, test_names):
praison_key = f'praisonai_{test}'
agno_key = f'agno_{test}'
if praison_key in results and agno_key in results:
praison_avg = results[praison_key].avg_time
agno_avg = results[agno_key].avg_time
if praison_avg < agno_avg:
diff = f'PraisonAI {agno_avg/praison_avg:.2f}x faster'
else:
diff = f'Agno {praison_avg/agno_avg:.2f}x faster'
f.write(f'| {name} | {praison_avg:.2f}s | {agno_avg:.2f}s | {diff} |\n')
f.write('\n## Detailed Results\n\n')
for key, result in results.items():
f.write(f'### {key}\n')
f.write(f'- Average: {result.avg_time:.2f}s\n')
f.write(f'- Min: {result.min_time:.2f}s\n')
f.write(f'- Max: {result.max_time:.2f}s\n')
f.write(f'- Std Dev: {result.std_dev:.2f}s\n')
f.write(f'- Times: {[f"{t:.2f}s" for t in result.times]}\n\n')
f.write('## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('export OPENAI_API_KEY=your_key\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/real_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
# ============================================================================
# Main
# ============================================================================
if __name__ == "__main__":
print("="*70)
print("PraisonAI Agents - Real-World Execution Benchmark")
print("="*70)
print("\nThis benchmark measures actual agent execution time including:")
print(" - LLM API calls")
print(" - Tool execution")
print(" - Multi-agent coordination")
print("\nModel: gpt-4o-mini")
print("Iterations: 3 per test")
versions = get_package_versions()
print(f"\nPackage versions:")
for pkg, ver in versions.items():
print(f" {pkg}: {ver}")
results = {}
# Single Agent Tests
print("\n" + "="*70)
print("SINGLE AGENT BENCHMARK")
print("="*70)
print("\nTesting PraisonAI single agent...")
results['praisonai_single'] = benchmark_praisonai_single()
print("\nTesting Agno single agent...")
results['agno_single'] = benchmark_agno_single()
# 2-Agent Tests
print("\n" + "="*70)
print("2-AGENT BENCHMARK")
print("="*70)
print("\nTesting PraisonAI 2 agents...")
results['praisonai_two_agents'] = benchmark_praisonai_two_agents()
print("\nTesting Agno 2 agents (Team)...")
results['agno_two_agents'] = benchmark_agno_two_agents()
# 3-Agent Tests
print("\n" + "="*70)
print("3-AGENT BENCHMARK")
print("="*70)
print("\nTesting PraisonAI 3 agents...")
results['praisonai_three_agents'] = benchmark_praisonai_three_agents()
print("\nTesting Agno 3 agents (Team)...")
results['agno_three_agents'] = benchmark_agno_three_agents()
# Summary
print("\n" + "="*70)
print("RESULTS SUMMARY")
print("="*70)
print(f"\n{'Test':<20} {'PraisonAI (avg)':<18} {'Agno (avg)':<15} {'Difference':<20}")
print("-" * 73)
tests = [
('Single Agent', 'praisonai_single', 'agno_single'),
('2 Agents', 'praisonai_two_agents', 'agno_two_agents'),
('3 Agents', 'praisonai_three_agents', 'agno_three_agents'),
]
for name, praison_key, agno_key in tests:
praison_avg = results[praison_key].avg_time
agno_avg = results[agno_key].avg_time
if praison_avg < agno_avg:
diff = f'PraisonAI {agno_avg/praison_avg:.2f}x faster'
else:
diff = f'Agno {praison_avg/agno_avg:.2f}x faster'
print(f"{name:<20} {praison_avg:<18.2f}s {agno_avg:<15.2f}s {diff:<20}")
print("\n" + "="*70)
# Save results
save_results(results, versions)
@@ -0,0 +1,235 @@
"""
PraisonAI Agents - Response Time Benchmark
Measures actual LLM response time with gpt-4o-mini across frameworks.
Usage:
python benchmarks/response_benchmark.py
"""
import time
import statistics
import os
from typing import List, Literal
from dataclasses import dataclass, field
# Ensure we use gpt-4o-mini
os.environ['OPENAI_MODEL_NAME'] = 'gpt-4o-mini'
@dataclass
class ResponseResult:
"""Response time measurement results"""
response_times: List[float] = field(default_factory=list)
@property
def avg_time(self) -> float:
return statistics.mean(self.response_times) if self.response_times else 0
@property
def min_time(self) -> float:
return min(self.response_times) if self.response_times else 0
@property
def max_time(self) -> float:
return max(self.response_times) if self.response_times else 0
@property
def std_dev(self) -> float:
return statistics.stdev(self.response_times) if len(self.response_times) > 1 else 0
def get_weather(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
def benchmark_praisonai_response(iterations: int = 5) -> ResponseResult:
"""Benchmark PraisonAI agent response time"""
from praisonaiagents import Agent
result = ResponseResult()
for i in range(iterations):
print(f" PraisonAI iteration {i+1}/{iterations}...")
# Create fresh agent each time
agent = Agent(
name="Test Agent",
instructions="Be concise, reply with one sentence.",
llm="gpt-4o-mini",
verbose=False
)
start = time.perf_counter()
response = agent.start("What is the capital of France?")
end = time.perf_counter()
result.response_times.append(end - start)
print(f" Response time: {end - start:.3f}s")
return result
def benchmark_agno_response(iterations: int = 5) -> ResponseResult:
"""Benchmark Agno agent response time"""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
result = ResponseResult()
for i in range(iterations):
print(f" Agno iteration {i+1}/{iterations}...")
# Create fresh agent each time
agent = Agent(
model=OpenAIChat(id="gpt-4o-mini"),
system_message="Be concise, reply with one sentence.",
)
start = time.perf_counter()
response = agent.run("What is the capital of France?")
end = time.perf_counter()
result.response_times.append(end - start)
print(f" Response time: {end - start:.3f}s")
return result
def benchmark_praisonai_with_tool(iterations: int = 5) -> ResponseResult:
"""Benchmark PraisonAI agent response time with tool calling"""
from praisonaiagents import Agent
result = ResponseResult()
for i in range(iterations):
print(f" PraisonAI (with tool) iteration {i+1}/{iterations}...")
agent = Agent(
name="Weather Agent",
instructions="You are a weather assistant. Use the get_weather tool to answer questions.",
llm="gpt-4o-mini",
tools=[get_weather],
verbose=False
)
start = time.perf_counter()
response = agent.start("What's the weather in NYC?")
end = time.perf_counter()
result.response_times.append(end - start)
print(f" Response time: {end - start:.3f}s")
return result
def benchmark_agno_with_tool(iterations: int = 5) -> ResponseResult:
"""Benchmark Agno agent response time with tool calling"""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
result = ResponseResult()
for i in range(iterations):
print(f" Agno (with tool) iteration {i+1}/{iterations}...")
agent = Agent(
model=OpenAIChat(id="gpt-4o-mini"),
system_message="You are a weather assistant. Use the get_weather tool to answer questions.",
tools=[get_weather],
)
start = time.perf_counter()
response = agent.run("What's the weather in NYC?")
end = time.perf_counter()
result.response_times.append(end - start)
print(f" Response time: {end - start:.3f}s")
return result
if __name__ == "__main__":
print("="*70)
print("PraisonAI Agents - Response Time Benchmark")
print("="*70)
print("\nModel: gpt-4o-mini")
print("Measures end-to-end response time including:")
print(" - Agent instantiation")
print(" - API call to OpenAI")
print(" - Response processing")
iterations = 5
results = {}
# Test 1: Simple response (no tools)
print("\n" + "="*70)
print("TEST 1: Simple Response (No Tools)")
print("="*70)
print("\nBenchmarking PraisonAI...")
try:
results['praisonai_simple'] = benchmark_praisonai_response(iterations)
except Exception as e:
print(f"Error: {e}")
import traceback
traceback.print_exc()
print("\nBenchmarking Agno...")
try:
results['agno_simple'] = benchmark_agno_response(iterations)
except Exception as e:
print(f"Error: {e}")
import traceback
traceback.print_exc()
# Test 2: Response with tool calling
print("\n" + "="*70)
print("TEST 2: Response with Tool Calling")
print("="*70)
print("\nBenchmarking PraisonAI with tool...")
try:
results['praisonai_tool'] = benchmark_praisonai_with_tool(iterations)
except Exception as e:
print(f"Error: {e}")
import traceback
traceback.print_exc()
print("\nBenchmarking Agno with tool...")
try:
results['agno_tool'] = benchmark_agno_with_tool(iterations)
except Exception as e:
print(f"Error: {e}")
import traceback
traceback.print_exc()
# Summary
print("\n" + "="*70)
print("SUMMARY RESULTS")
print("="*70)
print(f"\n{'Test':<30} {'PraisonAI (avg)':<20} {'Agno (avg)':<20} {'Ratio':<15}")
print("-" * 85)
if 'praisonai_simple' in results and 'agno_simple' in results:
p_time = results['praisonai_simple'].avg_time
a_time = results['agno_simple'].avg_time
ratio = p_time / a_time if a_time > 0 else float('inf')
print(f"{'Simple Response':<30} {p_time:<20.3f}s {a_time:<20.3f}s {ratio:.2f}x")
if 'praisonai_tool' in results and 'agno_tool' in results:
p_time = results['praisonai_tool'].avg_time
a_time = results['agno_tool'].avg_time
ratio = p_time / a_time if a_time > 0 else float('inf')
print(f"{'With Tool Calling':<30} {p_time:<20.3f}s {a_time:<20.3f}s {ratio:.2f}x")
print("\n" + "="*70)
print("NOTE")
print("="*70)
print("\nResponse time is dominated by OpenAI API latency.")
print("Agent instantiation overhead is negligible in real-world usage.")
@@ -0,0 +1,194 @@
❯ python benchmarks/performance_benchmark.py
/Users/praison/praisonai-package/src/praisonai-agents/praisonaiagents/_warning_patch.py:47: UserWarning: Mixing V1 models and V2 models (or constructs, like `TypeAdapter`) is not supported. Please upgrade `Settings` to V2.
_original_warn(message, category, stacklevel, source)
[13:04:15] INFO [13:04:15] config.py:50
config.py:50 INFO
Using config path:
/Users/praison/.con
fig/crewai/settings
.json
======================================================================
PraisonAI Agents - Comprehensive Performance Benchmark
======================================================================
This benchmark measures:
1. Agent instantiation time (how fast an agent object is created)
2. Memory footprint (how much memory an agent uses)
Methodology:
- 10 warmup runs (not measured)
- 1000 measured iterations
- tracemalloc for memory measurement
- Baseline subtraction for accuracy
Frameworks available:
- PraisonAI: ✅
- Agno: ✅
- PydanticAI: ✅
- OpenAI Agents SDK: ✅
- LangGraph: ✅
- CrewAI: ✅
======================================================================
Testing PraisonAI Agents
======================================================================
============================================================
Benchmarking: PraisonAI
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for PraisonAI:
Average Time: 0.000003 seconds (3.20 μs)
Min Time: 0.000003 seconds
Max Time: 0.000025 seconds
Std Dev: 0.000001 seconds
Median Time: 0.000003 seconds
Avg Memory: 0.003678 MiB (3.77 KiB)
======================================================================
Testing PraisonAI Agents (LiteLLM)
======================================================================
============================================================
Benchmarking: PraisonAI (LiteLLM)
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for PraisonAI (LiteLLM):
Average Time: 0.000007 seconds (7.31 μs)
Min Time: 0.000007 seconds
Max Time: 0.000060 seconds
Std Dev: 0.000002 seconds
Median Time: 0.000007 seconds
Avg Memory: 0.006419 MiB (6.57 KiB)
======================================================================
Testing Agno
======================================================================
============================================================
Benchmarking: Agno
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for Agno:
Average Time: 0.000005 seconds (5.30 μs)
Min Time: 0.000005 seconds
Max Time: 0.000039 seconds
Std Dev: 0.000002 seconds
Median Time: 0.000005 seconds
Avg Memory: 0.006599 MiB (6.76 KiB)
======================================================================
Testing PydanticAI
======================================================================
============================================================
Benchmarking: PydanticAI Agent Instantiation
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for PydanticAI Agent Instantiation:
Average Time: 0.000247 seconds (247.34 μs)
Min Time: 0.000218 seconds
Max Time: 0.000552 seconds
Std Dev: 0.000031 seconds
Median Time: 0.000237 seconds
Avg Memory: 0.028486 MiB (29.17 KiB)
======================================================================
Testing OpenAI Agents SDK
======================================================================
============================================================
Benchmarking: OpenAI Agents SDK Instantiation
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for OpenAI Agents SDK Instantiation:
Average Time: 0.000282 seconds (282.25 μs)
Min Time: 0.000247 seconds
Max Time: 0.001110 seconds
Std Dev: 0.000055 seconds
Median Time: 0.000263 seconds
Avg Memory: 0.029032 MiB (29.73 KiB)
======================================================================
Testing LangGraph
======================================================================
============================================================
Benchmarking: LangGraph Agent Instantiation
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for LangGraph Agent Instantiation:
Average Time: 0.003200 seconds (3200.15 μs)
Min Time: 0.002808 seconds
Max Time: 0.116363 seconds
Std Dev: 0.003628 seconds
Median Time: 0.002972 seconds
Avg Memory: 0.207509 MiB (212.49 KiB)
======================================================================
Testing CrewAI
======================================================================
============================================================
Benchmarking: CrewAI Agent Instantiation
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for CrewAI Agent Instantiation:
Average Time: 0.016772 seconds (16771.69 μs)
Min Time: 0.014404 seconds
Max Time: 0.091841 seconds
Std Dev: 0.003445 seconds
Median Time: 0.015816 seconds
Avg Memory: 0.065050 MiB (66.61 KiB)
======================================================================
RESULTS SUMMARY
======================================================================
Framework Avg Time (μs) Avg Memory (KiB) Relative
-------------------------------------------------------------------------
PraisonAI 3.20 3.77 1.00x
Agno 5.30 6.76 1.66x
PraisonAI (LiteLLM) 7.31 6.57 2.28x
PydanticAI 247.34 29.17 77.25x
OpenAI Agents SDK 282.25 29.73 88.15x
LangGraph 3200.15 212.49 999.46x
CrewAI 16771.69 66.61 5238.06x
======================================================================
❯
@@ -0,0 +1,243 @@
#!/usr/bin/env python3
"""
PraisonAI Agents - Performance Benchmark
Compares agent instantiation times across popular AI agent frameworks.
Usage:
python benchmarks/simple_benchmark.py
"""
import time
from typing import Literal
ITERATIONS = 100
def sample_tool(city: Literal['nyc', 'sf']):
"""Sample tool for benchmark testing."""
if city == 'nyc':
return 'cloudy'
return 'sunny'
def measure_instantiation(create_fn, iterations=ITERATIONS):
"""Measure average instantiation time in microseconds."""
times = []
for _ in range(iterations):
start = time.perf_counter()
create_fn()
times.append((time.perf_counter() - start) * 1_000_000)
return sum(times) / len(times)
def run_benchmark():
"""Run the benchmark across all available frameworks (without tools)."""
results = {}
print('=' * 60)
print('PraisonAI Agents - Performance Benchmark')
print('=' * 60)
print(f'\nIterations: {ITERATIONS}')
print('Metric: Agent instantiation time (microseconds)\n')
# PraisonAI (without tools)
print("Testing PraisonAI...")
from praisonaiagents import Agent as PraisonAgent
results['PraisonAI'] = measure_instantiation(
lambda: PraisonAgent(name='Test', llm='gpt-4o-mini', verbose=False)
)
results['PraisonAI (LiteLLM)'] = measure_instantiation(
lambda: PraisonAgent(name='Test', llm='openai/gpt-4o-mini', verbose=False)
)
# Other frameworks for comparison
print("Testing other frameworks...")
try:
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
results['Agno'] = measure_instantiation(
lambda: AgnoAgent(model=OpenAIChat(id='gpt-4o-mini'))
)
except ImportError:
pass
try:
from pydantic_ai import Agent as PydanticAgent
results['PydanticAI'] = measure_instantiation(
lambda: PydanticAgent('openai:gpt-4o-mini')
)
except ImportError:
pass
try:
from agents import Agent as OpenAIAgent
results['OpenAI Agents SDK'] = measure_instantiation(
lambda: OpenAIAgent(name='Test', model='gpt-4o-mini')
)
except ImportError:
pass
try:
from langchain_core.tools import tool as langchain_tool
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
@langchain_tool
def get_weather_lg(city: Literal['nyc', 'sf']):
"""Get weather info."""
return 'sunny' if city == 'sf' else 'cloudy'
results['LangGraph'] = measure_instantiation(
lambda: create_react_agent(model=ChatOpenAI(model='gpt-4o-mini'), tools=[get_weather_lg])
)
except ImportError:
pass
try:
from crewai.agent import Agent as CrewAgent
from crewai.tools import tool as crewai_tool
@crewai_tool("Weather Tool")
def get_weather_crew(city: Literal['nyc', 'sf']):
"""Get weather info."""
return 'sunny' if city == 'sf' else 'cloudy'
results['CrewAI'] = measure_instantiation(
lambda: CrewAgent(
role='Weather Agent',
goal='Provide weather info',
backstory='A weather expert',
tools=[get_weather_crew],
verbose=False
)
)
except ImportError:
pass
# Print results
print('\n' + '=' * 60)
print('RESULTS')
print('=' * 60)
baseline = results.get('PraisonAI', 1)
print(f"\n{'Framework':<25} {'Avg Time (μs)':<15} {'Relative':<10}")
print('-' * 50)
for name, avg in sorted(results.items(), key=lambda x: x[1]):
ratio = avg / baseline
print(f'{name:<25} {avg:<15.2f} {ratio:.2f}x')
print('\n' + '=' * 60)
return results
def get_package_versions():
"""Get version numbers for benchmarked packages."""
from importlib.metadata import version as get_version
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
'PydanticAI': 'pydantic-ai',
'OpenAI Agents SDK': 'openai-agents',
'LangGraph': 'langgraph',
'CrewAI': 'crewai'
}
versions = {}
for display_name, pkg_name in packages.items():
try:
versions[display_name] = get_version(pkg_name)
except Exception:
versions[display_name] = 'not installed'
return versions
def save_results(results: dict, filename: str = 'BENCHMARK_RESULTS.md'):
"""Save benchmark results to a markdown file and update README."""
import os
import re
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), filename)
versions = get_package_versions()
# Find the fastest result (baseline)
sorted_results = sorted(results.items(), key=lambda x: x[1])
fastest_time = sorted_results[0][1] if sorted_results else 1
# Build the table rows
table_rows = []
for name, avg in sorted_results:
ratio = avg / fastest_time
if ratio == 1.0:
table_rows.append(f'| **{name}** | **{avg:.2f}** | **1.00x (fastest)** |')
else:
avg_str = f'{avg:,.2f}' if avg >= 1000 else f'{avg:.2f}'
ratio_str = f'{ratio:,.0f}x' if ratio >= 100 else f'{ratio:.2f}x'
table_rows.append(f'| {name} | {avg_str} | {ratio_str} |')
# Save to BENCHMARK_RESULTS.md
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Benchmark Results\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write(f'**Iterations:** {ITERATIONS}\n')
f.write('**Test:** Agent instantiation (without tools)\n\n')
f.write('## Results\n\n')
f.write('| Framework | Avg Time (μs) | Relative |\n')
f.write('|-----------|---------------|----------|\n')
f.write('\n'.join(table_rows) + '\n')
f.write('\n## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/simple_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
# Also update the main README.md
readme_path = os.path.join(os.path.dirname(__file__), '..', '..', '..', 'README.md')
if os.path.exists(readme_path):
update_readme(readme_path, table_rows)
return filepath
def update_readme(readme_path: str, table_rows: list):
"""Update the performance section in README.md with latest results."""
import re
with open(readme_path, 'r') as f:
content = f.read()
# Build the new table
new_table = '''| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
''' + '\n'.join(table_rows)
# Pattern to match the performance table
pattern = r'(\| Framework \| Avg Time \(μs\) \| Relative \|\n\|[-|]+\|\n)(\|[^\n]+\|\n)+'
if re.search(pattern, content):
content = re.sub(pattern, new_table + '\n', content)
with open(readme_path, 'w') as f:
f.write(content)
print(f'README.md updated: {readme_path}')
if __name__ == '__main__':
results = run_benchmark()
save_results(results)
@@ -0,0 +1,213 @@
#!/usr/bin/env python3
"""
PraisonAI Agents - Tools Benchmark
Compares agent instantiation times WITH TOOLS across frameworks.
Usage:
python benchmarks/tools_benchmark.py
"""
import time
from typing import Literal
from importlib.metadata import version as get_version
ITERATIONS = 100
def sample_tool(city: Literal['nyc', 'sf']):
"""Sample tool for benchmark testing."""
if city == 'nyc':
return 'cloudy'
return 'sunny'
def measure_instantiation(create_fn, iterations=ITERATIONS):
"""Measure average instantiation time in microseconds."""
times = []
# Warmup
for _ in range(10):
create_fn()
# Measure
for _ in range(iterations):
start = time.perf_counter()
create_fn()
elapsed = time.perf_counter() - start
times.append(elapsed * 1_000_000) # Convert to microseconds
return sum(times) / len(times)
def run_benchmark():
"""Run the benchmark across all available frameworks WITH TOOLS."""
tools = [sample_tool]
results = {}
print('=' * 60)
print('PraisonAI Agents - Tools Benchmark')
print('=' * 60)
print(f'\nIterations: {ITERATIONS}')
print('Metric: Agent instantiation time WITH TOOLS (microseconds)\n')
# PraisonAI with tools
print("Testing PraisonAI...")
from praisonaiagents import Agent as PraisonAgent
results['PraisonAI'] = measure_instantiation(
lambda: PraisonAgent(name='Test', llm='gpt-4o-mini', tools=tools, verbose=False)
)
results['PraisonAI (LiteLLM)'] = measure_instantiation(
lambda: PraisonAgent(name='Test', llm='openai/gpt-4o-mini', tools=tools, verbose=False)
)
# Other frameworks with tools
print("Testing other frameworks...")
try:
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
results['Agno'] = measure_instantiation(
lambda: AgnoAgent(model=OpenAIChat(id='gpt-4o-mini'), tools=tools)
)
except ImportError:
pass
try:
from agents import Agent as OpenAIAgent, function_tool
results['OpenAI Agents SDK'] = measure_instantiation(
lambda: OpenAIAgent(name='Test', model='gpt-4o-mini', tools=[function_tool(sample_tool)])
)
except ImportError:
pass
try:
from langchain_core.tools import tool as langchain_tool
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
@langchain_tool
def get_weather_lg(city: Literal['nyc', 'sf']):
"""Get weather info."""
return 'sunny' if city == 'sf' else 'cloudy'
results['LangGraph'] = measure_instantiation(
lambda: create_react_agent(model=ChatOpenAI(model='gpt-4o-mini'), tools=[get_weather_lg])
)
except ImportError:
pass
try:
from crewai.agent import Agent as CrewAgent
from crewai.tools import tool as crewai_tool
@crewai_tool("Weather Tool")
def get_weather_crew(city: Literal['nyc', 'sf']):
"""Get weather info."""
return 'sunny' if city == 'sf' else 'cloudy'
results['CrewAI'] = measure_instantiation(
lambda: CrewAgent(
role='Weather Agent',
goal='Provide weather info',
backstory='A weather expert',
tools=[get_weather_crew],
verbose=False
)
)
except ImportError:
pass
# Print results
print('\n' + '=' * 60)
print('RESULTS (WITH TOOLS)')
print('=' * 60)
sorted_results = sorted(results.items(), key=lambda x: x[1])
baseline = sorted_results[0][1] if sorted_results else 1
print(f"\n{'Framework':<25} {'Avg Time (μs)':<15} {'Relative':<10}")
print('-' * 50)
for name, avg in sorted_results:
ratio = avg / baseline
print(f'{name:<25} {avg:<15.2f} {ratio:.2f}x')
print('\n' + '=' * 60)
return results
def get_package_versions():
"""Get version numbers for benchmarked packages."""
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
'OpenAI Agents SDK': 'openai-agents',
'LangGraph': 'langgraph',
'CrewAI': 'crewai'
}
versions = {}
for display_name, pkg_name in packages.items():
try:
versions[display_name] = get_version(pkg_name)
except Exception:
versions[display_name] = 'not installed'
return versions
def save_results(results: dict, filename: str = 'TOOLS_BENCHMARK_RESULTS.md'):
"""Save benchmark results to a markdown file."""
import os
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), filename)
versions = get_package_versions()
# Find the fastest result (baseline)
sorted_results = sorted(results.items(), key=lambda x: x[1])
fastest_time = sorted_results[0][1] if sorted_results else 1
# Build the table rows
table_rows = []
for name, avg in sorted_results:
ratio = avg / fastest_time
if ratio == 1.0:
table_rows.append(f'| **{name}** | **{avg:.2f}** | **1.00x (fastest)** |')
else:
avg_str = f'{avg:,.2f}' if avg >= 1000 else f'{avg:.2f}'
ratio_str = f'{ratio:,.0f}x' if ratio >= 100 else f'{ratio:.2f}x'
table_rows.append(f'| {name} | {avg_str} | {ratio_str} |')
# Save to file
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Tools Benchmark Results\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write(f'**Iterations:** {ITERATIONS}\n')
f.write('**Test:** Agent instantiation WITH TOOLS\n\n')
f.write('## Results\n\n')
f.write('| Framework | Avg Time (μs) | Relative |\n')
f.write('|-----------|---------------|----------|\n')
f.write('\n'.join(table_rows) + '\n')
f.write('\n## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/tools_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
return filepath
if __name__ == '__main__':
results = run_benchmark()
save_results(results)
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,9 @@
Starting test with verbose=False
CALLBACK EXECUTED!
Message:
You need to do the following task: Say hello world.
Expected Output: A greeting.
Please provide only the final result of your work. Do not add any conversation or extra explanation.
Response: Hello, World!
Other kwargs: {'markdown': True, 'generation_time': 2.685833215713501, 'agent_name': 'TestAgent', 'agent_role': 'Assistant', 'agent_tools': None, 'task_name': 'simple_task', 'task_description': 'Say hello world', 'task_id': 0}
--------------------------------------------------
@@ -0,0 +1,90 @@
"""
Unified Web Search Example
This example demonstrates the search_web tool that automatically tries
multiple search providers with fallback support.
Search Provider Priority:
1. Tavily (requires TAVILY_API_KEY + tavily-python)
2. Exa (requires EXA_API_KEY + exa_py)
3. You.com (requires YDC_API_KEY + youdotcom)
4. DuckDuckGo (requires duckduckgo_search package, no API key)
5. SearxNG (requires requests + running SearxNG instance)
Installation:
pip install praisonaiagents duckduckgo_search
Usage:
python web_search_example.py
"""
from praisonaiagents import Agent
from praisonaiagents.tools import search_web, get_available_providers
def main():
print("=" * 60)
print("Unified Web Search - Provider Status")
print("=" * 60)
# Check which providers are available
providers = get_available_providers()
for p in providers:
status = "✓ Available" if p["available"] else f"✗ {p['reason']}"
print(f" {p['name']:12} {status}")
print("\n" + "=" * 60)
print("Example 1: Basic Search")
print("=" * 60)
# Simple search - automatically uses best available provider
results = search_web("Python programming best practices", max_results=3)
if results and "error" not in results[0]:
print(f"\nFound {len(results)} results using {results[0].get('provider', 'unknown')}:\n")
for r in results:
print(f" Title: {r.get('title', 'N/A')[:60]}...")
print(f" URL: {r.get('url', 'N/A')}")
print()
else:
print(f"\nSearch failed: {results}")
print("=" * 60)
print("Example 2: Specify Providers")
print("=" * 60)
# Only try specific providers
results = search_web(
"machine learning tutorials",
max_results=3,
providers=["duckduckgo", "tavily"] # Try these in order
)
if results and "error" not in results[0]:
print(f"\nFound {len(results)} results using {results[0].get('provider', 'unknown')}:\n")
for r in results:
print(f" Title: {r.get('title', 'N/A')[:60]}...")
print(f" URL: {r.get('url', 'N/A')}")
print()
else:
print(f"\nSearch failed: {results}")
print("=" * 60)
print("Example 3: With PraisonAI Agent")
print("=" * 60)
# Use search_web as an agent tool
agent = Agent(
name="SearchAgent",
role="Web Researcher",
goal="Find and summarize information from the web",
instructions="Search the web and provide concise summaries of findings.",
tools=[search_web]
)
result = agent.start("What are the top 3 AI trends in 2025?")
print(f"\nAgent Response:\n{result}")
if __name__ == "__main__":
main()
@@ -3,7 +3,7 @@ Praison AI Agents - A package for hierarchical AI agent task execution
"""
# Apply warning patch BEFORE any imports to intercept warnings at the source
from . import _warning_patch
from . import _warning_patch # noqa: F401
# Import centralized logging configuration FIRST
from . import _logging
@@ -54,6 +54,7 @@ except ImportError:
MCP = None
from .session import Session
from .memory.memory import Memory
from .db import db
# Workflows - import from dedicated workflows module
from .workflows import (
Workflow, WorkflowStep, WorkflowContext, StepResult,
@@ -78,6 +79,19 @@ def __getattr__(name):
elif name == "LineRange":
from praisonaiagents.context.fast import LineRange
return LineRange
# Agent Skills support (lazy loaded for zero performance impact)
elif name == "SkillManager":
from praisonaiagents.skills import SkillManager
return SkillManager
elif name == "SkillProperties":
from praisonaiagents.skills import SkillProperties
return SkillProperties
elif name == "SkillMetadata":
from praisonaiagents.skills import SkillMetadata
return SkillMetadata
elif name == "SkillLoader":
from praisonaiagents.skills import SkillLoader
return SkillLoader
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
# Planning mode support
@@ -244,6 +258,7 @@ __all__ = [
'AutoAgents',
'Session',
'Memory',
'db',
'display_interaction',
'display_self_reflection',
'display_instruction',
@@ -289,6 +304,23 @@ __all__ = [
'ExpandResult'
]
# Add workflow exports
__all__.extend([
'Workflow',
'WorkflowStep',
'WorkflowContext',
'StepResult',
'Route',
'Parallel',
'Loop',
'Repeat',
'route',
'parallel',
'loop',
'repeat',
'Pipeline'
])
# Add MCP to __all__ if available
if _mcp_available:
__all__.append('MCP')
@@ -305,3 +337,29 @@ __all__.extend([
'LineRange'
])
# Add Agent Skills exports (lazy loaded)
__all__.extend([
'SkillManager',
'SkillProperties',
'SkillMetadata',
'SkillLoader'
])
# AG-UI support (optional, lazy loaded)
try:
from praisonaiagents.ui.agui import AGUI
__all__.append('AGUI')
_agui_available = True
except ImportError:
_agui_available = False
AGUI = None
# A2A support (optional, lazy loaded)
try:
from praisonaiagents.ui.a2a import A2A
__all__.append('A2A')
_a2a_available = True
except ImportError:
_a2a_available = False
A2A = None
@@ -55,10 +55,10 @@ def _get_all_noisy_loggers() -> List[str]:
def _configure_loggers():
"""Configure all loggers based on LOGLEVEL environment variable."""
loglevel = os.environ.get('LOGLEVEL', 'INFO').upper()
loglevel = os.environ.get('LOGLEVEL', 'WARNING').upper()
# When DEBUG is set, allow some HTTP logging for API endpoints
if loglevel == 'DEBUG':
# When DEBUG or INFO is set, allow some HTTP logging for API endpoints
if loglevel in ('DEBUG', 'INFO'):
allowed_debug_loggers = {"httpx", "httpx._client", "openai._client"}
for logger_name in _get_all_noisy_loggers():
@@ -109,10 +109,10 @@ def configure_root_logger():
"""Configure the root logger with RichHandler."""
from rich.logging import RichHandler
loglevel = os.environ.get('LOGLEVEL', 'INFO').upper()
loglevel = os.environ.get('LOGLEVEL', 'WARNING').upper()
logging.basicConfig(
level=getattr(logging, loglevel, logging.INFO),
level=getattr(logging, loglevel, logging.WARNING),
format="%(asctime)s %(filename)s:%(lineno)d %(levelname)s %(message)s",
datefmt="[%X]",
handlers=[RichHandler(rich_tracebacks=True)],
@@ -8,9 +8,10 @@ import functools
import sys
# Apply aggressive warning filters first
warnings.filterwarnings("ignore", message=".*There is no current event loop.*", category=DeprecationWarning)
warnings.filterwarnings("ignore", message=".*Pydantic serializer warnings.*")
warnings.filterwarnings("ignore", message=".*PydanticSerializationUnexpectedValue.*")
warnings.filterwarnings("ignore", message=".*Expected 9 fields but got.*")
warnings.filterwarnings("ignore", message=".*Expected \\d+ fields but got.*")
warnings.filterwarnings("ignore", message=".*Expected `StreamingChoices` but got.*")
warnings.filterwarnings("ignore", message=".*serialized value may not be as expected.*")
warnings.filterwarnings("ignore", message=".*Use 'content=<...>' to upload raw bytes/text content.*")
@@ -27,9 +28,12 @@ SUPPRESSED_PATTERNS = [
"The `dict` method is deprecated; use `model_dump` instead",
"Pydantic serializer warnings",
"PydanticSerializationUnexpectedValue",
"Expected 9 fields but got 5 for type `Message`",
"Expected", # Catches all "Expected N fields but got M" patterns
"Expected `StreamingChoices` but got `Choices`",
"serialized value may not be as expected"
"serialized value may not be as expected",
"Mixing V1 models and V2 models",
"Please upgrade `Settings` to V2",
"There is no current event loop"
]
@functools.wraps(_original_warn)
@@ -41,7 +45,7 @@ def _patched_warn(message, category=None, stacklevel=1, source=None):
if pattern in msg_str:
return
if category == UserWarning and "pydantic" in msg_str.lower():
if category is UserWarning and "pydantic" in msg_str.lower():
return
_original_warn(message, category, stacklevel, source)
@@ -55,7 +59,7 @@ def _patched_warn_explicit(message, category, filename, lineno, module=None, reg
if pattern in msg_str:
return
if category == UserWarning and "pydantic" in msg_str.lower():
if category is UserWarning and "pydantic" in msg_str.lower():
return
if module and "pydantic" in str(module):
@@ -250,7 +250,11 @@ class Agent:
fast_context_parallelism: int = 8,
fast_context_timeout: float = 30.0,
history_in_context: Optional[int] = None,
auto_save: Optional[str] = None
auto_save: Optional[str] = None,
skills: Optional[List[str]] = None,
skills_dirs: Optional[List[str]] = None,
db: Optional[Any] = None,
session_id: Optional[str] = None
):
"""Initialize an Agent instance.
@@ -551,8 +555,9 @@ Your Goal: {self.goal}
self.auto_save = auto_save # Session name for auto-saving
# Initialize rules manager for persistent context (like Cursor/Windsurf)
# NOTE: Lazy initialization - rules are loaded only when accessed (performance optimization)
self._rules_manager = None
self._init_rules_manager()
self._rules_manager_initialized = False
# Handle web_search fallback: inject DuckDuckGo tool for unsupported models
if web_search and not self._model_supports_web_search():
@@ -599,13 +604,24 @@ Your Goal: {self.goal}
# Fast Context configuration (lazy loaded)
self.fast_context_enabled = fast_context
self.fast_context_path = fast_context_path or os.getcwd()
self._fast_context_path = fast_context_path # Store raw, resolve lazily
self.fast_context_model = fast_context_model
self.fast_context_max_turns = fast_context_max_turns
self.fast_context_parallelism = fast_context_parallelism
self.fast_context_timeout = fast_context_timeout
self._fast_context_instance = None # Lazy loaded
# Agent Skills configuration (lazy loaded for zero performance impact)
self._skills = skills
self._skills_dirs = skills_dirs
self._skill_manager = None # Lazy loaded
self._skills_initialized = False
# Database persistence (lazy - no imports until used)
self._db = db
self._session_id = session_id
self._db_initialized = False
@property
def console(self):
"""Lazily initialize Rich Console only when needed."""
@@ -614,6 +630,83 @@ Your Goal: {self.goal}
self._console = Console()
return self._console
@property
def fast_context_path(self):
"""Lazily resolve fast_context_path - avoids os.getcwd() on every init."""
if self._fast_context_path is None:
self._fast_context_path = os.getcwd()
return self._fast_context_path
@fast_context_path.setter
def fast_context_path(self, value):
self._fast_context_path = value
@property
def skill_manager(self):
"""Lazily initialize SkillManager only when skills are accessed."""
if self._skill_manager is None and (self._skills or self._skills_dirs):
from ..skills import SkillManager
self._skill_manager = SkillManager()
# Add explicit skill paths
if self._skills:
for skill_path in self._skills:
self._skill_manager.add_skill(skill_path)
# Discover skills from directories
if self._skills_dirs:
self._skill_manager.discover(self._skills_dirs, include_defaults=False)
self._skills_initialized = True
# Auto-add skill execution tools if not already present
self._add_skill_tools()
return self._skill_manager
def _add_skill_tools(self):
"""Add tools required for skill execution (read_file, run_skill_script).
Uses lazy imports from praisonaiagents.tools to avoid performance impact
when skills are not used.
"""
# Check if tools already include required capabilities
tool_names = set()
for tool in self.tools:
if callable(tool) and hasattr(tool, '__name__'):
tool_names.add(tool.__name__)
elif hasattr(tool, 'name'):
tool_names.add(tool.name)
# Add read_file if not present
if 'read_file' not in tool_names:
try:
from ..tools import read_file
self.tools.append(read_file)
logging.debug("Added read_file tool for skill support")
except ImportError:
logging.warning("Could not import read_file tool for skills")
# Add run_skill_script from skill_tools module
if 'run_skill_script' not in tool_names:
try:
from ..tools.skill_tools import create_skill_tools
# Create skill tools with current working directory
skill_tools = create_skill_tools()
self.tools.append(skill_tools.run_skill_script)
logging.debug("Added run_skill_script tool for skill support")
except ImportError as e:
logging.warning(f"Could not import skill_tools: {e}")
def get_skills_prompt(self) -> str:
"""Get the XML prompt for available skills.
Returns:
XML string with <available_skills> block, or empty string if no skills
"""
if self.skill_manager is None:
return ""
return self.skill_manager.to_prompt()
@property
def _openai_client(self):
"""Lazily initialize OpenAI client only when needed."""
@@ -785,6 +878,21 @@ Your Goal: {self.goal}
logging.warning(f"FastContext search failed: {e}")
return None
@property
def rules_manager(self):
"""
Lazy-initialized RulesManager for persistent rules/instructions.
This property initializes the RulesManager only when first accessed,
avoiding expensive filesystem operations during agent instantiation.
Returns:
RulesManager instance or None if not available
"""
if not self._rules_manager_initialized:
self._init_rules_manager()
return self._rules_manager
def _init_rules_manager(self):
"""
Initialize RulesManager for persistent rules/instructions.
@@ -793,7 +901,10 @@ Your Goal: {self.goal}
- ~/.praison/rules/ (global)
- .praison/rules/ (workspace)
- Subdirectory rules
NOTE: This is called lazily via the rules_manager property for performance.
"""
self._rules_manager_initialized = True
try:
from ..memory.rules_manager import RulesManager
import os
@@ -828,10 +939,10 @@ Your Goal: {self.goal}
Returns:
Formatted rules context string
"""
if not self._rules_manager:
if not self.rules_manager:
return ""
return self._rules_manager.build_rules_context(
return self.rules_manager.build_rules_context(
file_path=file_path,
include_manual=include_manual
)
@@ -1239,8 +1350,8 @@ Your Goal: {self.goal}
Your Role: {self.role}\n
Your Goal: {self.goal}"""
# Add rules context if rules manager is enabled
if self._rules_manager:
# Add rules context if rules manager is enabled (lazy initialization)
if self._rules_manager_initialized and self._rules_manager:
rules_context = self.get_rules_context()
if rules_context:
system_prompt += f"\n\n## Rules (Guidelines you must follow)\n{rules_context}"
@@ -1254,6 +1365,13 @@ Your Goal: {self.goal}"""
if self.verbose:
self._display_memory_info()
# Add skills prompt if skills are configured
if self._skills or self._skills_dirs:
skills_prompt = self.get_skills_prompt()
if skills_prompt:
system_prompt += f"\n\n## Available Skills\n{skills_prompt}"
system_prompt += "\n\nWhen a skill is relevant to the task, read its SKILL.md file to get detailed instructions. If the skill has scripts in its scripts/ directory, you can execute them using the execute_code or run_script tool."
# Add tool usage instructions if tools are available
# Use provided tools or fall back to self.tools
tools_to_use = tools if tools is not None else self.tools
@@ -1508,13 +1626,13 @@ Your Goal: {self.goal}"""
logging.debug(f"{self.name} executing tool {function_name} with arguments: {arguments}")
# Check if approval is required for this tool
from ..approval import is_approval_required, console_approval_callback, get_risk_level, mark_approved, ApprovalDecision
from ..approval import is_approval_required, console_approval_callback, get_risk_level, mark_approved, ApprovalDecision, get_approval_callback
if is_approval_required(function_name):
risk_level = get_risk_level(function_name)
logging.info(f"Tool {function_name} requires approval (risk level: {risk_level})")
logging.debug(f"Tool {function_name} requires approval (risk level: {risk_level})")
# Use global approval callback or default console callback
callback = approval_callback or console_approval_callback
callback = get_approval_callback() or console_approval_callback
try:
decision = callback(function_name, arguments, risk_level)
@@ -1539,23 +1657,54 @@ Your Goal: {self.goal}"""
# Special handling for MCP tools
# Check if tools is an MCP instance with the requested function name
from ..mcp.mcp import MCP
# Helper function to execute MCP tool
def _execute_mcp_tool(mcp_instance, func_name, args):
"""Execute a tool from an MCP instance."""
# Handle SSE MCP client
if hasattr(mcp_instance, 'is_sse') and mcp_instance.is_sse:
if hasattr(mcp_instance, 'sse_client'):
for tool in mcp_instance.sse_client.tools:
if tool.name == func_name:
logging.debug(f"Found matching SSE MCP tool: {func_name}")
return True, tool(**args)
# Handle HTTP Stream MCP client
if hasattr(mcp_instance, 'is_http_stream') and mcp_instance.is_http_stream:
if hasattr(mcp_instance, 'http_stream_client'):
for tool in mcp_instance.http_stream_client.tools:
if tool.name == func_name:
logging.debug(f"Found matching HTTP Stream MCP tool: {func_name}")
return True, tool(**args)
# Handle WebSocket MCP client
if hasattr(mcp_instance, 'is_websocket') and mcp_instance.is_websocket:
if hasattr(mcp_instance, 'websocket_client'):
for tool in mcp_instance.websocket_client.tools:
if tool.name == func_name:
logging.debug(f"Found matching WebSocket MCP tool: {func_name}")
return True, tool(**args)
# Handle stdio MCP client
if hasattr(mcp_instance, 'runner'):
for mcp_tool in mcp_instance.runner.tools:
if hasattr(mcp_tool, 'name') and mcp_tool.name == func_name:
logging.debug(f"Found matching MCP tool: {func_name}")
return True, mcp_instance.runner.call_tool(func_name, args)
return False, None
# Check if tools is a single MCP instance
if isinstance(self.tools, MCP):
logging.debug(f"Looking for MCP tool {function_name}")
# Handle SSE MCP client
if hasattr(self.tools, 'is_sse') and self.tools.is_sse:
if hasattr(self.tools, 'sse_client'):
for tool in self.tools.sse_client.tools:
if tool.name == function_name:
logging.debug(f"Found matching SSE MCP tool: {function_name}")
return tool(**arguments)
# Handle stdio MCP client
elif hasattr(self.tools, 'runner'):
# Check if any of the MCP tools match the function name
for mcp_tool in self.tools.runner.tools:
if hasattr(mcp_tool, 'name') and mcp_tool.name == function_name:
logging.debug(f"Found matching MCP tool: {function_name}")
return self.tools.runner.call_tool(function_name, arguments)
found, result = _execute_mcp_tool(self.tools, function_name, arguments)
if found:
return result
# Check if tools is a list that may contain MCP instances
if isinstance(self.tools, (list, tuple)):
for tool in self.tools:
if isinstance(tool, MCP):
logging.debug(f"Looking for MCP tool {function_name} in MCP instance")
found, result = _execute_mcp_tool(tool, function_name, arguments)
if found:
return result
# Try to find the function in the agent's tools list first
func = None
@@ -1815,7 +1964,108 @@ Your Goal: {self.goal}"""
# expand=False
# )
def _init_db_session(self):
"""Initialize DB session if db adapter is provided (lazy, first chat only)."""
if self._db is None or self._db_initialized:
return
# Generate session_id if not provided: default to per-hour ID (YYYYMMDDHH-agentname)
if self._session_id is None:
import hashlib
from datetime import datetime, timezone
# Per-hour session ID: YYYYMMDDHH (UTC) + agent name hash for uniqueness
hour_str = datetime.now(timezone.utc).strftime("%Y%m%d%H")
agent_hash = hashlib.md5(self.name.encode()).hexdigest()[:6]
self._session_id = f"{hour_str}-{agent_hash}"
# Call db adapter's on_agent_start to get previous messages
try:
history = self._db.on_agent_start(
agent_name=self.name,
session_id=self._session_id,
user_id=self.user_id,
metadata={"role": self.role, "goal": self.goal}
)
# Restore chat history from previous session
if history:
for msg in history:
self.chat_history.append({
"role": msg.role,
"content": msg.content
})
logging.info(f"Resumed session {self._session_id} with {len(history)} messages")
except Exception as e:
logging.warning(f"Failed to initialize DB session: {e}")
self._db_initialized = True
self._current_run_id = None # Track current run
def _start_run(self, input_content: str):
"""Start a new run (turn) for persistence tracking."""
if self._db is None:
return
import uuid
self._current_run_id = f"run-{uuid.uuid4().hex[:12]}"
try:
if hasattr(self._db, 'on_run_start'):
self._db.on_run_start(
session_id=self._session_id,
run_id=self._current_run_id,
input_content=input_content,
metadata={"agent_name": self.name}
)
except Exception as e:
logging.warning(f"Failed to start run: {e}")
def _end_run(self, output_content: str, status: str = "completed", metrics: dict = None):
"""End the current run (turn)."""
if self._db is None or self._current_run_id is None:
return
try:
if hasattr(self._db, 'on_run_end'):
self._db.on_run_end(
session_id=self._session_id,
run_id=self._current_run_id,
output_content=output_content,
status=status,
metrics=metrics or {},
metadata={"agent_name": self.name}
)
except Exception as e:
logging.warning(f"Failed to end run: {e}")
self._current_run_id = None
def _persist_message(self, role: str, content: str):
"""Persist a message to the DB if adapter is provided."""
if self._db is None:
return
try:
if role == "user":
self._db.on_user_message(self._session_id, content)
elif role == "assistant":
self._db.on_agent_message(self._session_id, content)
except Exception as e:
logging.warning(f"Failed to persist message: {e}")
@property
def session_id(self) -> Optional[str]:
"""Get the current session ID."""
return self._session_id
def chat(self, prompt, temperature=1.0, tools=None, output_json=None, output_pydantic=None, reasoning_steps=False, stream=None, task_name=None, task_description=None, task_id=None):
# Initialize DB session on first chat (lazy)
self._init_db_session()
# Start a new run for this chat turn
prompt_str = prompt if isinstance(prompt, str) else str(prompt)
self._start_run(prompt_str)
# Reset the final display flag for each new conversation
self._final_display_shown = False
@@ -1870,7 +2120,8 @@ Your Goal: {self.goal}"""
if tool_param is not None:
from ..mcp.mcp import MCP
if isinstance(tool_param, MCP) and hasattr(tool_param, 'to_openai_tool'):
logging.debug("Converting MCP tool to OpenAI format")
# Single MCP instance
logging.debug("Converting single MCP tool to OpenAI format")
openai_tool = tool_param.to_openai_tool()
if openai_tool:
# Handle both single tool and list of tools
@@ -1879,6 +2130,22 @@ Your Goal: {self.goal}"""
else:
tool_param = [openai_tool]
logging.debug(f"Converted MCP tool: {tool_param}")
elif isinstance(tool_param, (list, tuple)):
# List that may contain MCP instances - convert each MCP to OpenAI format
converted_tools = []
for t in tool_param:
if isinstance(t, MCP) and hasattr(t, 'to_openai_tool'):
logging.debug("Converting MCP instance in list to OpenAI format")
openai_tools = t.to_openai_tool()
if isinstance(openai_tools, list):
converted_tools.extend(openai_tools)
elif openai_tools:
converted_tools.append(openai_tools)
else:
# Keep non-MCP tools as-is
converted_tools.append(t)
tool_param = converted_tools
logging.debug(f"Converted {len(converted_tools)} tools from list")
# Store chat history length for potential rollback
chat_history_length = len(self.chat_history)
@@ -1895,6 +2162,8 @@ Your Goal: {self.goal}"""
self.chat_history[-1].get("content") == normalized_content):
# Add user message to chat history BEFORE LLM call so handoffs can access it
self.chat_history.append({"role": "user", "content": normalized_content})
# Persist user message to DB
self._persist_message("user", normalized_content)
try:
# Pass everything to LLM class
@@ -1924,6 +2193,8 @@ Your Goal: {self.goal}"""
)
self.chat_history.append({"role": "assistant", "content": response_text})
# Persist assistant message to DB
self._persist_message("assistant", response_text)
# Log completion time if in debug mode
if logging.getLogger().getEffectiveLevel() == logging.DEBUG:
@@ -1968,6 +2239,8 @@ Your Goal: {self.goal}"""
self.chat_history[-1].get("content") == normalized_content):
# Add user message to chat history BEFORE LLM call so handoffs can access it
self.chat_history.append({"role": "user", "content": normalized_content})
# Persist user message to DB (OpenAI path)
self._persist_message("user", normalized_content)
reflection_count = 0
start_time = time.time()
@@ -2007,6 +2280,8 @@ Your Goal: {self.goal}"""
# Add to chat history and return raw response
# User message already added before LLM call via _build_messages
self.chat_history.append({"role": "assistant", "content": response_text})
# Persist assistant message to DB
self._persist_message("assistant", response_text)
# Apply guardrail validation even for JSON output
try:
validated_response = self._apply_guardrail_with_retry(response_text, original_prompt, temperature, tools, task_name, task_description, task_id)
@@ -2022,6 +2297,8 @@ Your Goal: {self.goal}"""
if not self.self_reflect:
# User message already added before LLM call via _build_messages
self.chat_history.append({"role": "assistant", "content": response_text})
# Persist assistant message to DB (non-reflect path)
self._persist_message("assistant", response_text)
if self.verbose:
logging.debug(f"Agent {self.name} final response: {response_text}")
# Return only reasoning content if reasoning_steps is True
@@ -2110,11 +2387,13 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
validated_response = self._apply_guardrail_with_retry(response_text, original_prompt, temperature, tools, task_name, task_description, task_id)
# Execute callback after validation
self._execute_callback_and_display(original_prompt, validated_response, time.time() - start_time, task_name, task_description, task_id)
self._end_run(validated_response, "completed", {"duration_ms": (time.time() - start_time) * 1000})
return validated_response
except Exception as e:
logging.error(f"Agent {self.name}: Guardrail validation failed after reflection: {e}")
# Rollback chat history on guardrail failure
self.chat_history = self.chat_history[:chat_history_length]
self._end_run(None, "error", {"error": str(e)})
return None
# Check if we've hit max reflections
@@ -1370,6 +1370,61 @@ Context:
agent_names = ", ".join([agent.name for agent in self.agents])
print(f"📊 Available agents for this endpoint ({len(self.agents)}): {agent_names}")
# Create per-agent endpoints for individual agent access
# This allows n8n and other tools to call specific agents
agents_dict = {agent.name.lower().replace(' ', '_'): agent for agent in self.agents}
# Add GET endpoint to list available agents
@_agents_shared_apps[port].get(f"{path}/list")
async def list_agents():
return {
"agents": [
{"name": agent.name, "id": agent.name.lower().replace(' ', '_')}
for agent in self.agents
]
}
# Add per-agent POST endpoints
for agent_id, agent_instance in agents_dict.items():
agent_path = f"{path}/{agent_id}"
# Create a closure to capture the agent instance
def create_agent_handler(agent):
async def handle_single_agent(request: Request):
try:
request_data = await request.json()
query = request_data.get("query", "")
if not query:
raise HTTPException(status_code=400, detail="Missing 'query' field")
except:
raise HTTPException(status_code=400, detail="Invalid JSON body")
try:
if asyncio.iscoroutinefunction(agent.chat):
response = await agent.achat(query)
else:
loop = asyncio.get_running_loop()
response = await loop.run_in_executor(None, lambda q=query: agent.chat(q))
return {
"agent": agent.name,
"query": query,
"response": response
}
except Exception as e:
logging.error(f"Error with agent {agent.name}: {str(e)}", exc_info=True)
return JSONResponse(
status_code=500,
content={"error": f"Agent error: {str(e)}"}
)
return handle_single_agent
# Register the endpoint
_agents_shared_apps[port].post(agent_path)(create_agent_handler(agent_instance))
_agents_registered_endpoints[port][agent_path] = f"{endpoint_id}_{agent_id}"
print(f"🔗 Per-agent endpoints: {', '.join([f'{path}/{aid}' for aid in agents_dict.keys()])}")
# Start the server if it's not already running for this port
if not _agents_server_started.get(port, False):
# Mark the server as started first to prevent duplicate starts
Loaded 100 of 399 files, more files were not shown because too many files have changed in this diff. Show more