Compare commits

...
67 Commits
Author SHA1 Message Date
Mervin Praison afa37b0782 Release v2.3.37 2025-12-18 17:41:16 +00:00
Mervin Praison 43a15e6049 Release v2.3.36 2025-12-18 16:42:58 +00:00
Mervin Praison 8bf5a08246 Release v2.3.35 2025-12-18 16:28:02 +00:00
Mervin Praison aa81094ac3 Release v2.3.34 2025-12-18 14:55:33 +00:00
Mervin Praison c74fd7e925 Release v2.3.33 2025-12-18 14:41:36 +00:00
Mervin Praison 249acf7965 docs: Add performance benchmarks and auto-update functionality
Add performance comparison table to README showing PraisonAI as fastest framework for agent instantiation. Include benchmarks for LangGraph and CrewAI frameworks. Implement auto-save functionality to generate BENCHMARK_RESULTS.md and update README.md with latest benchmark results.
2025-12-18 13:30:06 +00:00
Mervin Praison ab68b2d296 Release v2.3.32 2025-12-18 13:17:32 +00:00
Mervin Praison abcc5b3b04 Release v2.3.31 2025-12-18 12:57:05 +00:00
Mervin Praison e584f56099 Release v2.3.30 2025-12-18 11:59:22 +00:00
Mervin Praison b6b01af066 Release v2.3.33 2025-12-18 11:19:29 +00:00
Mervin Praison 9aae3eff86 Release v2.3.32 2025-12-18 10:48:09 +00:00
Mervin Praison f90582c66b Release v2.3.31 2025-12-18 10:27:03 +00:00
Mervin Praison 67a8d65d8e Release v2.3.30 2025-12-18 09:07:42 +00:00
Mervin Praison d60c31a86d Release v2.3.29 2025-12-18 02:47:04 +00:00
Mervin Praison fcb7b2a0d0 fix(tests): Mark test_phase2_refactor tests as @pytest.mark.real
These tests make real API calls and should be excluded from CI.
2025-12-17 13:31:01 +00:00
Mervin Praison 81ad7db679 Release v2.3.28 2025-12-17 13:22:14 +00:00
Mervin Praison 0482c3ebb0 chore: Update dependencies and versioning for PraisonAI
- Bump PraisonAI version to 2.3.28 across Dockerfiles and deployment scripts.
- Update praisonaiagents version to 0.1.7 in pyproject.toml.
- Enhance README with a structured Table of Contents and improved section headers for better navigation.
- Remove outdated README.md from praisonai-agents and link to the main README.
2025-12-17 13:20:29 +00:00
Mervin Praison 3107458016 docs: Enhance README with emoji for section headers for better readability 2025-12-17 13:09:58 +00:00
Mervin Praison 44c8ceebae fix: Fix 15 test failures across multiple test files
Test fixes:
1. Import Errors (5 tests):
   - test_llm_self_reflection_direct.py: Define calculator locally
   - test_self_reflection_comprehensive.py: Define duckduckgo_search locally
   - test_self_reflection_fix_simple.py: Define calculator locally
   - test_self_reflection_fix_verification.py: Define calculator locally
   - test_fix_comprehensive.py: Convert to proper pytest with skipif

2. Workflow Assertion (1 test):
   - test_workflow_agents.py: Check error in results array

3. Ollama Type Assertion (1 test):
   - test_ollama_sequential_fix.py: Accept both str and int types

4. Ollama Missing Fixture (3 tests):
   - test_ollama_fix.py: Add proper pytest functions with skipif
   - test_ollama_async_fix.py: Add proper pytest functions with skipif

5. Memory Stale Data (2 tests):
   - test_file_memory.py: Use unique user_id per test

6. Planning Async Mocks (3 tests):
   - test_planning.py: Use get_response_async instead of achat

Results: 921 passed, 9 failed (down from 12), 16 skipped
2025-12-17 13:05:12 +00:00
Mervin Praison 20c0b02528 fix(ci): Fix Docker publish workflow - trigger on release, update actions 2025-12-17 13:00:51 +00:00
Mervin Praison cfc886b591 docs: Update examples to use canonical field names (agents, instructions) 2025-12-17 12:34:32 +00:00
Mervin Praison 77ac4b7a09 fix: Use _yaml_action from YAML parser when executing workflow steps
The action/description field from YAML was being stored on the agent but not
used when executing the workflow. Now the _normalize_single_step method
checks for _yaml_action and uses it as the prompt.

This fixes the issue where workflow steps were not using the action as the prompt.
2025-12-17 12:33:07 +00:00
Mervin Praison 5897cedc45 docs: Update README examples to use canonical field names 2025-12-17 12:29:47 +00:00
Mervin Praison a4d1079813 feat: Add unified field names with A-I-G-S mnemonic
- Add canonical field name suggestions to CLI validate command
- Document A-I-G-S mnemonic (Agents, Instructions, Goal, Steps)
- Add field name reference table to README
- Add comprehensive tests for field name normalization
- Accept both old and new names: roles/agents, backstory/instructions, description/action, topic/name

Implements 'Accept liberally, suggest canonically' principle
2025-12-17 12:24:11 +00:00
Mervin Praison 4eed985c4d Release v2.3.27 2025-12-17 12:17:02 +00:00
Mervin Praison 0658491072 docs: Update README with workflow features and feature parity
- Add YAML workflow file documentation
- Add workflow auto-generation documentation
- Add complete workflow.yaml reference with all features
- Add feature parity note between agents.yaml and workflow.yaml
- Update workflow CLI options table
2025-12-17 11:52:33 +00:00
Mervin Praison 9f82368b4c feat: Add workflow auto-generation (--auto for workflow.yaml)
- Add WorkflowAutoGenerator class in auto.py with lazy loading
- Support sequential, routing, parallel, loop patterns
- Add 'praisonai workflow auto' CLI command
- Add action_auto to WorkflowHandler
- Add _auto_generate_workflow to main.py
- Uses instructor for structured output generation
- Lazy loading ensures no performance impact when not used

Usage:
  praisonai workflow auto "Research AI trends" --pattern parallel
  praisonai workflow auto "Build a chatbot" --pattern routing
2025-12-17 11:50:02 +00:00
Mervin Praison c267cdda87 Release v2.3.26 2025-12-17 11:47:04 +00:00
Mervin Praison b88bb04090 feat: Add feature parity between agents.yaml and workflow.yaml
Workflow.yaml now supports all agents.yaml features:

Agent-level features:
- framework (praisonai, crewai, autogen)
- process (sequential, hierarchical, workflow)
- manager_llm for hierarchical process
- function_calling_llm
- max_rpm (rate limiting)
- max_execution_time (timeout)
- reflect_llm, min_reflect, max_reflect
- system_template, prompt_template, response_template
- backstory (alias for instructions)

Step-level features:
- expected_output (like agents.yaml tasks)
- context (task dependencies)
- output_json, output_pydantic (structured output)
- create_directory
- callback
- async_execution

Added 14 new TDD tests for feature parity verification.
All 41 YAML parser tests pass.
2025-12-17 11:40:38 +00:00
Mervin Praison 709a35fbdc refactor: Remove duplicate code
- Remove duplicate _convert_roles_to_agents from agents_generator.py
  (YAMLWorkflowParser already handles this conversion)
- Refactor _create_workflow_from_template in main.py to use
  WorkflowHandler.TEMPLATES instead of duplicating templates

This reduces code duplication and ensures single source of truth for:
1. Roles-to-agents conversion logic (in yaml_parser.py)
2. Workflow templates (in workflow.py)
2025-12-17 11:16:51 +00:00
Mervin Praison 8b792424b6 feat: Extend agents.yaml with workflow patterns (Option D)
- Add workflow detection in agents_generator.py (process: workflow)
- Route to YAMLWorkflowParser when workflow mode detected
- Support both 'roles' and 'agents' keys for backward compatibility
- Map 'backstory' to 'instructions' automatically
- Add _convert_roles_to_agents method in YAMLWorkflowParser
- Add _run_yaml_workflow method in AgentsGenerator
- Add 7 new tests for roles-to-agents conversion
- Add extended agents.yaml example with all workflow patterns
- Update README with extended agents.yaml documentation

This allows users to use advanced workflow patterns (route, parallel,
loop, repeat) directly in agents.yaml while maintaining full backward
compatibility with existing agents.yaml files.

Usage:
  praisonai agents.yaml  # Works for both standard and workflow mode
2025-12-17 11:11:46 +00:00
Mervin Praison ef2a8cd658 feat(cli): Add YAML workflow CLI commands
- Add WorkflowHandler for 'praisonai workflow' commands
- Add 'praisonai workflow run <file.yaml>' to run YAML workflows
- Add 'praisonai workflow validate <file.yaml>' to validate YAML workflows
- Add 'praisonai workflow template <name>' to create from templates
- Templates: simple, routing, parallel, loop, evaluator-optimizer
- Support --var key=value for passing variables
- Add unit tests for WorkflowHandler (8 tests)
- Update main.py to support YAML workflow execution
2025-12-17 10:52:23 +00:00
Mervin Praison d41d2dcba2 docs: Add YAML workflow loading documentation to README 2025-12-17 10:43:20 +00:00
Mervin Praison 3bc1786b49 feat(workflows): Add YAMLWorkflowParser for YAML-based workflow definitions
- Add YAMLWorkflowParser class to parse YAML workflow files
- Support all workflow patterns: route(), parallel(), loop(), repeat()
- Support agent definitions with planning, reasoning, tools
- Support workflow config: planning, reasoning, memory_config, verbose
- Support variables, callbacks, and guardrails
- Add load_yaml(), execute_yaml(), aexecute_yaml() to WorkflowManager
- Add comprehensive test suite (20 tests)
- Add example YAML workflow files:
  - simple_workflow.yaml
  - routing_workflow.yaml
  - parallel_workflow.yaml
  - loop_workflow.yaml
  - evaluator_optimizer_workflow.yaml
  - complete_workflow.yaml
- Add example_usage.py demonstrating YAML workflow usage
2025-12-17 10:42:44 +00:00
Mervin Praison 2eec3e2359 fix: exclude real tests from all pattern and fix session test assertions
- test_runner.py: Add '-m not real' to exclude tests requiring API keys when running 'all' pattern
- test_cli_features.py: Fix test_session_get_actions to match actual SessionHandler.get_actions() implementation (list, show, resume, delete instead of start)
2025-12-17 08:33:24 +00:00
Mervin Praison 6168401a69 feat(docs): Simplify workflow documentation to focus on agent-first approach
- Replace WorkflowManager examples with Workflow class
- Emphasize Agent objects as workflow steps
- Streamline pattern examples (route, parallel, loop, repeat) with agent-based usage
- Remove complex WorkflowStep configuration examples
- Consolidate workflow features into clearer, more concise examples
- Remove workflow comparison table and memory config examples
- Focus on core workflow patterns with practical agent examples
2025-12-17 08:30:10 +00:00
Mervin Praison 473fda653d Release v2.3.25 2025-12-17 08:17:15 +00:00
Mervin Praison 30f1a440b6 Release v2.3.24 2025-12-17 08:04:24 +00:00
Mervin Praison f991f37cfe Release v2.3.23 2025-12-17 00:41:48 +00:00
Mervin Praison a43b3ac72c Release v2.3.22 2025-12-17 00:25:14 +00:00
Mervin Praison 06f82249e1 Release v2.3.21 2025-12-17 00:05:40 +00:00
Mervin Praison d3aa78241a Release v2.3.20 2025-12-16 23:46:47 +00:00
Mervin Praison dfb77c39f9 Release v2.3.19 2025-12-16 23:26:53 +00:00
Mervin Praison 7e8f9b0574 Release v2.3.18 2025-12-16 23:18:36 +00:00
Mervin Praison 19bcdcd000 Release v2.3.16 2025-12-16 22:59:29 +00:00
Mervin Praison 8f5160f827 feat(workflows): Add branching, loops, callbacks, and per-step variables
- Add branching support with next_steps and branch_condition fields
- Add loop support with loop_over and loop_var for iterating over variables
- Add on_task_start and on_task_complete callbacks to PraisonAIAgents
- Add variables parameter for global variable substitution across tasks
- Add per-step agent creation from agent_config when no agent assigned
- Add task-level variable substitution in task descriptions
- Refactor workflow
2025-12-16 22:58:56 +00:00
Mervin Praison 3574a4d2c0 Release v2.3.15 2025-12-16 22:50:37 +00:00
Mervin Praison b552843fa9 docs: Update README workflow CLI with new features
- Add inline workflow examples (--workflow 'step1,step2')
- Add planning mode example (--planning)
- Add tools and save examples (--tools, --save)
- Update to use global flags instead of workflow-specific flags
2025-12-16 21:46:30 +00:00
Mervin Praison 0461a133df fix(workflow): Actually use planning mode when --planning is enabled
Before: --planning flag was stored but never used, agent.chat() was called
After: --planning flag creates agent with planning=True and calls agent.start()

The agent.start() method checks if planning is enabled and calls
_start_with_planning() which:
1. Creates a plan using PlanningAgent
2. Displays the generated plan
3. Executes each step with context passing
4. Compiles results into final output
2025-12-16 19:12:28 +00:00
Mervin Praison b33857de33 refactor(cli): Use global flags for workflow instead of workflow-specific flags
BREAKING CHANGE: Removed workflow-specific flags in favor of global flags

Before:
  praisonai workflow run 'name' --workflow-tools tavily --workflow-save --workflow-verbose

After:
  praisonai workflow run 'name' --tools tavily --save --verbose

This simplifies the CLI by reusing existing global flags:
- --llm (instead of --workflow-llm)
- --tools (instead of --workflow-tools)
- --planning (instead of --workflow-planning)
- --memory (instead of --workflow-memory)
- --save (instead of --workflow-save)
- --verbose (instead of --workflow-verbose)

Only --workflow-var remains workflow-specific for variable substitution.
2025-12-16 19:09:52 +00:00
Mervin Praison 433c643f6c feat(cli): Add inline workflow support without .md template
Usage: praisonai 'prompt' --workflow 'step1:action1,step2:action2'

Example:
  praisonai 'What is AI?' --workflow 'Research:Research the topic,Summarize:Write a summary' --workflow-save

Features:
- Parse inline workflow steps from --workflow flag
- Sequential execution with context passing
- All workflow options work: --workflow-tools, --workflow-save, etc.
- No .md template file required
2025-12-16 19:05:20 +00:00
Mervin Praison 214119a063 feat(cli): Add complete workflow CLI options
- Add --workflow-llm for LLM model selection
- Add --workflow-tools for tools (comma-separated)
- Add --workflow-planning for planning mode
- Add --workflow-memory for memory support
- Add --workflow-stream for streaming output
- Add --workflow-verbose for verbose output
- Add --workflow-save to save output to file
- Show final output after workflow completion
- Save all step outputs to markdown file
- Update help text with all options
2025-12-16 19:01:57 +00:00
Mervin Praison 989a9f6fc5 Release v2.3.14 2025-12-16 18:50:59 +00:00
Mervin Praison cf276a5640 feat(workflows): Add context passing, per-step agents, async execution
## New Features:
- Context passing between workflow steps with {{previous_output}} and {{step_name_output}}
- Per-step agent configuration with role, goal, backstory, and tools
- Per-step tools support
- Async execution with aexecute() method
- Planning mode at workflow level
- Enhanced markdown parser for agent/tools blocks

## WorkflowStep Enhancements:
- agent_config: Optional[Dict] - Per-step agent configuration
- tools: Optional[List] - Per-step tools
- context_from: Optional[List[str]] - Specific steps to include context from
- retain_full_context: bool - Include all previous outputs vs only last
- output_variable: Optional[str] - Custom variable name for step output

## Workflow Enhancements:
- default_llm: Optional[str] - Default LLM for all steps
- planning: bool - Enable planning mode
- planning_llm: Optional[str] - LLM for planning
- memory_config: Optional[Dict] - Memory configuration

## Execute Method Enhancements:
- default_agent parameter (executor now optional)
- default_llm, memory, planning, stream, verbose parameters

## Tests:
- 74 new tests covering all features
2025-12-16 18:49:43 +00:00
Mervin Praison b5134ffb2f Add --final-agent flag for post-processing output with specialized agent 2025-12-16 16:40:22 +00:00
Mervin Praison 5664036712 Release v2.3.13 2025-12-16 16:33:03 +00:00
Mervin Praison 6c2ca64cbe Remove unused --compile-results flag 2025-12-16 16:32:42 +00:00
Mervin Praison 08c742287c Fix: Pass max_tokens as dict config to Agent 2025-12-16 16:27:29 +00:00
Mervin Praison 4ee32b8eb2 Improve planning mode: compile all step results into comprehensive final report 2025-12-16 16:22:44 +00:00
Mervin Praison 35f07e5653 Add --max-tokens flag and improve planning mode output compilation 2025-12-16 16:22:37 +00:00
Mervin Praison 6481aec376 Add --tools support for all agents including planning mode 2025-12-16 16:14:39 +00:00
Mervin Praison 98c846ef78 Improve: Dynamic tool instructions from docstrings instead of hardcoded search 2025-12-16 16:01:30 +00:00
Mervin Praison 6e4a9faaf8 Fix: Instruct PromptExpanderAgent to use tools for context gathering 2025-12-16 15:59:46 +00:00
Mervin Praison 05d62db355 Reorganize README.md by moving Prompt Expansion, Web Search/Fetch, and MCP sections after CLI Features table, adding comprehensive programmatic usage examples and feature comparison tables for better documentation structure and clarity. 2025-12-16 15:52:36 +00:00
Mervin Praison 375c8c9826 Fix CLI docs links to use /docs/cli/ path 2025-12-16 15:41:42 +00:00
Mervin Praison c6977c8221 Add CLI Features table with docs links 2025-12-16 15:19:29 +00:00
Mervin Praison d407c83cf3 Add comprehensive CLI feature documentation to README.md, including Guardrail, Metrics, Image Processing, Telemetry, MCP, Fast Context, Knowledge, Session, Tools, Handoff, Auto Memory, Todo, Router, Flow Display, and Prompt Expansion with usage examples and command options for each feature. 2025-12-16 15:06:41 +00:00
128 changed files with 28550 additions and 2256 deletions

No files matched your search

+37 -28
View File
@@ -1,45 +1,54 @@
name: Daily Publish Docker
name: Publish Docker
on:
push:
branches:
- main
schedule:
- cron: '0 0 * * *'
release:
types: [published]
workflow_dispatch:
inputs:
tag:
description: 'Docker image tag (default: latest)'
required: false
default: 'latest'
jobs:
check-and-build:
build-and-push:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Check last commit time
id: check_commit
run: |
last_commit_time=$(git log -1 --format=%ct)
current_time=$(date +%s)
time_diff=$((current_time - last_commit_time))
if [ $time_diff -ge 120 ] && [ $time_diff -lt 240 ]; then
echo "should_run=true" >> $GITHUB_OUTPUT
else
echo "should_run=false" >> $GITHUB_OUTPUT
fi
uses: actions/checkout@v4
- name: Set up Docker Buildx
if: steps.check_commit.outputs.should_run == 'true'
uses: docker/setup-buildx-action@v1
uses: docker/setup-buildx-action@v3
- name: Log in to the GitHub Container Registry
if: steps.check_commit.outputs.should_run == 'true'
run: echo ${{ secrets.GH_TOKEN }} | docker login ghcr.io -u USERNAME --password-stdin
- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: ghcr.io/mervinpraison/praisonai
tags: |
type=raw,value=latest,enable=${{ github.event_name == 'release' }}
type=semver,pattern={{version}},enable=${{ github.event_name == 'release' }}
type=semver,pattern={{major}}.{{minor}},enable=${{ github.event_name == 'release' }}
type=raw,value=${{ github.event.inputs.tag || 'latest' }},enable=${{ github.event_name == 'workflow_dispatch' }}
- name: Build and push Docker image
if: steps.check_commit.outputs.should_run == 'true'
uses: docker/build-push-action@v2
uses: docker/build-push-action@v5
with:
context: .
file: ./docker/Dockerfile
push: true
tags: ghcr.io/mervinpraison/praisonai:latest
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
+1267 -295
View File
File diff suppressed because it is too large. Load diff
+2 -1
View File
@@ -14,9 +14,10 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
# Note: praisonai is installed from PyPI - remove version pin to use latest available
RUN pip install --no-cache-dir \
flask \
"praisonai>=2.3.12" \
praisonai \
"praisonai[api]" \
gunicorn \
markdown
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.12" \
"praisonai>=2.3.37" \
"praisonai[chat]" \
"embedchain[github,youtube]"
+1 -1
View File
@@ -20,7 +20,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.12" \
"praisonai>=2.3.37" \
"praisonai[ui]" \
"praisonai[chat]" \
"praisonai[realtime]" \
+25
View File
@@ -0,0 +1,25 @@
# Dockerfile for PraisonAI Serve API
# Deploy to Railway, Render, Fly.io, or any container platform
#
# Build: docker build -f Dockerfile.serve -t praisonai-serve .
# Run: docker run -p 8005:8005 -e OPENAI_API_KEY=sk-xxx praisonai-serve
FROM python:3.11-slim
WORKDIR /app
# Install dependencies
RUN pip install --no-cache-dir praisonai praisonaiagents
# Copy agents configuration
COPY agents.yaml .
# Expose the API port
EXPOSE 8005
# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD curl -f http://localhost:8005/health || exit 1
# Start the API server
CMD ["praisonai", "serve", "agents.yaml", "--host", "0.0.0.0", "--port", "8005"]
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.12" \
"praisonai>=2.3.37" \
"praisonai[ui]" \
"praisonai[crewai]"
@@ -29,7 +29,7 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -71,7 +71,6 @@
"import pandas as pd\n",
"\n",
"print(\"📦 Dependencies installed!\")\n",
"print(\"🔄 Converting from AGNO → PraisonAI\")\n",
"print(\"🎯 Building Job Hunting Agent System\")"
]
},
@@ -1,10 +1,10 @@
framework: crewai
topic: "Technology Research and Content Creation using AWS Bedrock"
roles:
agents: # Canonical: use 'agents' instead of 'roles'
research_specialist:
role: "Research Specialist"
backstory: |
instructions: # Canonical |
You are an experienced technology researcher with deep expertise in analyzing
emerging technologies and market trends. You excel at gathering comprehensive
information from various sources and synthesizing it into actionable insights.
@@ -35,7 +35,7 @@ roles:
content_creator:
role: "Technical Content Creator"
backstory: |
instructions: # Canonical |
You are a skilled technical writer and content creator who specializes in
translating complex technology concepts into engaging, accessible content
for various audiences including developers, business leaders, and end users.
@@ -64,7 +64,7 @@ roles:
quality_reviewer:
role: "Content Quality Reviewer"
backstory: |
instructions: # Canonical |
You are a meticulous editor and quality assurance specialist with extensive
experience in technical content review. You ensure accuracy, clarity, and
consistency while maintaining the intended tone and message.
@@ -8,10 +8,10 @@ variables:
industry: "Electric Vehicles"
research_focus: "comprehensive market analysis"
roles:
agents: # Canonical: use 'agents' instead of 'roles'
market_overview_specialist:
role: "Market Overview Specialist"
backstory: |
instructions: # Canonical |
You are a seasoned market research analyst with 10+ years of experience
in analyzing market dynamics, trends, and growth patterns across various
industries. You excel at synthesizing complex market data into clear,
@@ -64,7 +64,7 @@ roles:
competitive_intelligence_analyst:
role: "Competitive Intelligence Analyst"
backstory: |
instructions: # Canonical |
You are a competitive intelligence expert specializing in analyzing
competitive landscapes, market positioning, and strategic intelligence.
You have deep expertise in competitor analysis, market share assessment,
@@ -119,7 +119,7 @@ roles:
financial_performance_analyst:
role: "Financial Performance Analyst"
backstory: |
instructions: # Canonical |
You are a financial analyst with expertise in corporate finance,
financial statement analysis, and industry benchmarking. You specialize
in analyzing financial performance metrics, trends, and comparative
@@ -173,7 +173,7 @@ roles:
growth_opportunities_researcher:
role: "Growth Opportunities Researcher"
backstory: |
instructions: # Canonical |
You are a strategic business analyst specializing in identifying growth
opportunities, market expansion strategies, and emerging business models.
You excel at spotting trends, gaps, and untapped potential in markets
@@ -228,7 +228,7 @@ roles:
risk_assessment_specialist:
role: "Risk Assessment Specialist"
backstory: |
instructions: # Canonical |
You are a risk management expert with extensive experience in identifying,
analyzing, and assessing business risks across various industries. You
specialize in market risks, competitive threats, regulatory challenges,
@@ -287,7 +287,7 @@ roles:
research_report_synthesizer:
role: "Research Report Synthesizer"
backstory: |
instructions: # Canonical |
You are an expert business report writer and analyst with extensive
experience in synthesizing complex research findings into clear,
actionable business intelligence reports. You excel at creating
+108
View File
@@ -0,0 +1,108 @@
"""
Multiple MCP Servers Example
This example demonstrates how to use multiple MCP servers with a single agent.
You can combine different MCP servers (time, memory, filesystem, etc.) along
with regular Python functions as tools.
Requirements:
- pip install praisonaiagents
- uvx (for mcp-server-time)
- npx (for @modelcontextprotocol servers)
"""
from praisonaiagents import Agent
from praisonaiagents.mcp import MCP
def my_custom_tool(query: str) -> str:
"""
A custom Python function that can be used alongside MCP tools.
Args:
query: The search query
Returns:
A formatted result string
"""
return f"Custom tool result for: {query}"
def main():
# Example 1: Single MCP server (basic usage)
print("=" * 60)
print("Example 1: Single MCP Server")
print("=" * 60)
agent_single = Agent(
name="TimeAgent",
instructions="You are a helpful assistant that can tell time.",
tools=MCP("uvx mcp-server-time")
)
response = agent_single.start("What time is it in Tokyo?")
print(f"Response: {response}\n")
# Example 2: Multiple MCP servers
print("=" * 60)
print("Example 2: Multiple MCP Servers")
print("=" * 60)
agent_multi = Agent(
name="MultiMCPAgent",
instructions="""You are a helpful assistant with access to multiple tools:
- Time tools for getting current time and converting timezones
- Memory tools for storing and retrieving information
Use the appropriate tool based on the user's request.""",
tools=[
MCP("uvx mcp-server-time"), # Time tools
MCP("npx @modelcontextprotocol/server-memory"), # Memory tools
]
)
response = agent_multi.start("What time is it in New York and London?")
print(f"Response: {response}\n")
# Example 3: Multiple MCP servers + custom functions
print("=" * 60)
print("Example 3: MCP Servers + Custom Functions")
print("=" * 60)
agent_mixed = Agent(
name="MixedToolsAgent",
instructions="""You are a helpful assistant with access to:
- Time tools for timezone operations
- A custom search tool for general queries
Choose the right tool for each task.""",
tools=[
MCP("uvx mcp-server-time"), # MCP server
my_custom_tool # Regular Python function
]
)
response = agent_mixed.start("What time is it in Paris?")
print(f"Response: {response}\n")
# Example 4: MCP with environment variables
print("=" * 60)
print("Example 4: MCP with Environment Variables")
print("=" * 60)
# Note: Replace with your actual API key
# agent_with_env = Agent(
# name="SearchAgent",
# instructions="You can search the web for information.",
# tools=[
# MCP(
# command="npx",
# args=["-y", "@modelcontextprotocol/server-brave-search"],
# env={"BRAVE_API_KEY": "your-api-key-here"}
# ),
# MCP("uvx mcp-server-time")
# ]
# )
print("(Skipped - requires BRAVE_API_KEY)")
if __name__ == "__main__":
main()
+4 -4
View File
@@ -1,8 +1,8 @@
framework: praisonai
topic: crawl and extract all data from each url from this page https://news.ycombinator.com/
roles:
agents: # Canonical: use 'agents' instead of 'roles'
web_scraper:
backstory: An expert in data extraction from websites, adept at navigating and
instructions: # Canonical An expert in data extraction from websites, adept at navigating and
retrieving detailed information.
goal: Extract URLs from the source page
role: Web Scraper
@@ -15,7 +15,7 @@ roles:
tools:
- 'code_interpret'
url_data_extractor:
backstory: Specializes in crawling websites to gather comprehensive data, ensuring
instructions: # Canonical Specializes in crawling websites to gather comprehensive data, ensuring
nothing is missed from each link.
goal: Crawl each URL for data extraction
role: URL Data Extractor
@@ -29,7 +29,7 @@ roles:
tools:
- 'code_interpret'
data_organizer:
backstory: Known for keen attention to detail, capable of structuring vast amounts
instructions: # Canonical Known for keen attention to detail, capable of structuring vast amounts
of information into usable formats efficiently.
goal: Organize extracted data into a structured format
role: Data Organizer
@@ -0,0 +1,38 @@
"""
Simple Agentic Workflow Example
Demonstrates using Agent instances directly as workflow steps.
Each agent processes the input and passes output to the next agent.
"""
from praisonaiagents import Agent, Workflow
# Create agents with specific roles
researcher = Agent(
name="Researcher",
role="Research Analyst",
goal="Research and provide information about topics",
instructions="You are a research analyst. Provide concise, factual information."
)
writer = Agent(
name="Writer",
role="Content Writer",
goal="Write engaging content based on research",
instructions="You are a content writer. Write clear, engaging content based on the research provided."
)
# Create workflow with agents as steps
workflow = Workflow(
name="Simple Agentic Pipeline",
steps=[researcher, writer]
)
if __name__ == "__main__":
# Run workflow - agents process sequentially
result = workflow.start(
"What are the key benefits of AI agents?",
verbose=True
)
print(f"\nFinal output:\n{result['output']}")
@@ -0,0 +1,36 @@
"""
Simple Workflow Example
The easiest way to create and run workflows in PraisonAI.
Just pass functions or agents as steps - no complex configuration needed!
"""
from praisonaiagents import Workflow, WorkflowContext, StepResult
# Define simple handler functions
def validate(ctx: WorkflowContext) -> StepResult:
"""Validate the input data."""
if not ctx.input:
return StepResult(output="No input provided", stop_workflow=True)
return StepResult(output=f"Validated: {ctx.input}")
def process(ctx: WorkflowContext) -> StepResult:
"""Process the validated data."""
return StepResult(output=f"Processed: {ctx.previous_result}")
def format_output(ctx: WorkflowContext) -> StepResult:
"""Format the final output."""
return StepResult(output=f"✅ Final: {ctx.previous_result}")
# Create workflow - just list your functions!
workflow = Workflow(
name="Simple Pipeline",
steps=[validate, process, format_output]
)
if __name__ == "__main__":
# Run the workflow - PraisonAI style uses .start()
result = workflow.start("Hello, World!", verbose=True)
print(f"\nFinal output: {result['output']}")
print(f"Steps completed: {len(result['steps'])}")
@@ -0,0 +1,57 @@
"""
Task Callbacks Example
Demonstrates using on_task_start and on_task_complete callbacks
with PraisonAIAgents for workflow monitoring and logging.
"""
from praisonaiagents import Agent, Task, PraisonAIAgents
# Create an agent
researcher = Agent(
name="Researcher",
role="Research Assistant",
goal="Find and summarize information",
llm="gpt-4o-mini"
)
# Create tasks with variables
task1 = Task(
name="research",
description="Research the topic: {{topic}}",
expected_output="A brief summary of the topic",
agent=researcher,
variables={"topic": "artificial intelligence"}
)
task2 = Task(
name="summarize",
description="Create a one-paragraph summary of the research.",
expected_output="A concise summary",
agent=researcher
)
# Define callbacks
def on_start(task, task_id):
"""Called before each task starts."""
print(f"🚀 Starting task: {task.name} (ID: {task_id})")
def on_complete(task, output):
"""Called after each task completes."""
print(f"✅ Completed task: {task.name}")
print(f" Output preview: {str(output)[:100]}...")
if __name__ == "__main__":
# Create agents with callbacks
agents = PraisonAIAgents(
agents=[researcher],
tasks=[task1, task2],
process="workflow",
on_task_start=on_start,
on_task_complete=on_complete,
variables={"global_var": "shared_value"} # Global variables
)
# Run the workflow
result = agents.start()
print("\n🎉 Workflow completed!")
@@ -0,0 +1,48 @@
"""
Workflow Branching Example
Demonstrates conditional branching in workflows where steps can route
to different next steps based on output content.
"""
from praisonaiagents.memory.workflows import WorkflowManager, WorkflowStep, Workflow
# Create a workflow with branching
workflow = Workflow(
name="Decision Workflow",
description="A workflow that branches based on validation result",
steps=[
WorkflowStep(
name="validate",
action="Check if the number 42 is positive. Reply with 'valid' or 'invalid'.",
next_steps=["success_handler", "error_handler"],
branch_condition={
"valid": ["success_handler"],
"invalid": ["error_handler"]
}
),
WorkflowStep(
name="success_handler",
action="The validation passed! Generate a success message."
),
WorkflowStep(
name="error_handler",
action="The validation failed. Generate an error message."
)
]
)
if __name__ == "__main__":
# Create manager and register workflow
manager = WorkflowManager()
manager.workflows["Decision Workflow"] = workflow
# Execute - will branch to success_handler since 42 is positive
result = manager.execute(
"Decision Workflow",
default_llm="gpt-4o-mini"
)
print("Workflow completed!")
for step_result in result["results"]:
print(f" {step_result['step']}: {step_result['status']}")
@@ -0,0 +1,63 @@
"""
Workflow Checkpoints Example
Demonstrates saving and resuming workflow execution using checkpoints.
Useful for long-running workflows that may be interrupted.
"""
from praisonaiagents.memory.workflows import WorkflowManager, WorkflowStep, Workflow
# Create a multi-step workflow
workflow = Workflow(
name="Long Process",
description="A workflow with checkpoints for resumability",
steps=[
WorkflowStep(
name="step1",
action="Initialize the process and prepare data."
),
WorkflowStep(
name="step2",
action="Process the first batch of data."
),
WorkflowStep(
name="step3",
action="Process the second batch of data."
),
WorkflowStep(
name="step4",
action="Finalize and generate report."
)
]
)
if __name__ == "__main__":
manager = WorkflowManager()
manager.workflows["Long Process"] = workflow
# Execute with checkpoint - saves after each step
print("=== Starting workflow with checkpoint ===")
result = manager.execute(
"Long Process",
default_llm="gpt-4o-mini",
checkpoint="my-checkpoint" # Saves progress after each step
)
print(f"Completed {len(result['results'])} steps")
# List available checkpoints
print("\n=== Available checkpoints ===")
checkpoints = manager.list_checkpoints()
for cp in checkpoints:
print(f" {cp['name']}: {cp['completed_steps']} steps completed")
# Resume from checkpoint (if workflow was interrupted)
# result = manager.execute(
# "Long Process",
# default_llm="gpt-4o-mini",
# resume="my-checkpoint" # Resumes from last saved state
# )
# Clean up checkpoint
manager.delete_checkpoint("my-checkpoint")
print("\n✅ Checkpoint deleted")
@@ -0,0 +1,83 @@
"""
Agentic Conditional Workflow Example
Demonstrates using should_run to conditionally execute agent steps
based on input or previous results.
"""
from praisonaiagents import Agent, Workflow, WorkflowStep
# Condition functions
def is_technical(ctx) -> bool:
"""Only run technical review for technical content."""
keywords = ["code", "programming", "algorithm", "database", "api"]
return any(keyword in ctx.input.lower() for keyword in keywords)
def needs_creative(ctx) -> bool:
"""Only run creative enhancement for creative content."""
keywords = ["story", "poem", "creative", "write", "narrative"]
return any(keyword in ctx.input.lower() for keyword in keywords)
# Create agents
analyzer = Agent(
name="Analyzer",
role="Content Analyzer",
goal="Analyze and process content",
instructions="Analyze the given content and provide insights."
)
tech_reviewer = Agent(
name="TechReviewer",
role="Technical Reviewer",
goal="Review technical content",
instructions="Review the technical aspects and provide technical feedback."
)
creative_enhancer = Agent(
name="CreativeEnhancer",
role="Creative Enhancer",
goal="Enhance creative content",
instructions="Enhance the creative aspects and make it more engaging."
)
finalizer = Agent(
name="Finalizer",
role="Content Finalizer",
goal="Finalize the content",
instructions="Provide the final polished version of the content."
)
# Create workflow with conditional steps
workflow = Workflow(
name="Conditional Agentic Pipeline",
steps=[
analyzer, # Always runs
WorkflowStep(
name="tech_review",
agent=tech_reviewer,
should_run=is_technical # Only runs for technical content
),
WorkflowStep(
name="creative_enhance",
agent=creative_enhancer,
should_run=needs_creative # Only runs for creative content
),
finalizer # Always runs
]
)
if __name__ == "__main__":
# Test 1: Technical content
print("=== Test 1: Technical Content ===")
result = workflow.start("Review this Python code for best practices", verbose=True)
print(f"Output: {result['output'][:200]}...\n")
# Test 2: Creative content
print("=== Test 2: Creative Content ===")
result = workflow.start("Write a short story about a robot", verbose=True)
print(f"Output: {result['output'][:200]}...\n")
# Test 3: General content (neither technical nor creative)
print("=== Test 3: General Content ===")
result = workflow.start("What is the weather like today?", verbose=True)
print(f"Output: {result['output'][:200]}...")
@@ -0,0 +1,68 @@
"""
Workflow Conditional Execution Example
Demonstrates using should_run to conditionally execute steps
based on input or previous results.
"""
from praisonaiagents import Workflow, WorkflowStep, WorkflowContext, StepResult
# Condition functions - return True to run the step, False to skip
def is_sensitive_content(ctx: WorkflowContext) -> bool:
"""Only run compliance check for sensitive topics."""
sensitive_keywords = ["legal", "medical", "financial", "security"]
return any(keyword in ctx.input.lower() for keyword in sensitive_keywords)
def needs_translation(ctx: WorkflowContext) -> bool:
"""Only translate if content is not in English."""
return ctx.variables.get("language", "en") != "en"
# Handler functions
def process_content(ctx: WorkflowContext) -> StepResult:
return StepResult(output=f"Processed: {ctx.input}")
def compliance_check(ctx: WorkflowContext) -> StepResult:
return StepResult(output=f"✅ Compliance verified for: {ctx.previous_result}")
def translate(ctx: WorkflowContext) -> StepResult:
lang = ctx.variables.get("language", "unknown")
return StepResult(output=f"🌐 Translated to {lang}: {ctx.previous_result}")
def finalize(ctx: WorkflowContext) -> StepResult:
return StepResult(output=f"📄 Final: {ctx.previous_result}")
# Create workflow with conditional steps
workflow = Workflow(
name="Conditional Pipeline",
steps=[
WorkflowStep(name="process", handler=process_content),
WorkflowStep(
name="compliance",
handler=compliance_check,
should_run=is_sensitive_content # Only runs for sensitive content
),
WorkflowStep(
name="translate",
handler=translate,
should_run=needs_translation # Only runs if language != "en"
),
WorkflowStep(name="finalize", handler=finalize)
]
)
if __name__ == "__main__":
# Test 1: Normal content (skips compliance)
print("=== Test 1: Normal content ===")
result = workflow.start("Hello world", verbose=True)
print(f"Output: {result['output']}\n")
# Test 2: Sensitive content (runs compliance)
print("=== Test 2: Legal content ===")
result = workflow.start("Review this legal document", verbose=True)
print(f"Output: {result['output']}\n")
# Test 3: Non-English content (runs translation)
print("=== Test 3: Non-English content ===")
workflow.variables = {"language": "es"}
result = workflow.start("Hola mundo", verbose=True)
print(f"Output: {result['output']}")
@@ -0,0 +1,69 @@
"""
Workflow Early Stop Example
Demonstrates how to stop a workflow early using custom handler
functions that return StepResult with stop_workflow=True.
"""
from praisonaiagents.memory.workflows import (
WorkflowManager, WorkflowStep, Workflow,
WorkflowContext, StepResult
)
# Custom validator that can stop the workflow
def validate_data(context: WorkflowContext) -> StepResult:
"""Check if data is valid and stop workflow if not."""
data = context.variables.get("data", {})
if data.get("value", 0) < 0:
return StepResult(
output="❌ Validation failed: negative value detected. Stopping workflow.",
stop_workflow=True # This stops the entire workflow
)
return StepResult(
output="✅ Validation passed: data is valid.",
stop_workflow=False # Continue to next step
)
# Create workflow with early stop capability
workflow = Workflow(
name="Data Processing",
description="Process data with validation gate",
variables={
"data": {"value": -5} # Invalid data - will trigger early stop
},
steps=[
WorkflowStep(
name="validate",
handler=validate_data # Custom function
),
WorkflowStep(
name="process",
action="Process the validated data." # Won't run if validation fails
),
WorkflowStep(
name="report",
action="Generate final report." # Won't run if validation fails
)
]
)
if __name__ == "__main__":
manager = WorkflowManager()
manager.workflows["Data Processing"] = workflow
# Test with invalid data (will stop early)
print("=== Testing with invalid data ===")
result = manager.execute("Data Processing", default_llm="gpt-4o-mini")
for step_result in result["results"]:
print(f" {step_result['step']}: {step_result['output']}")
# Test with valid data (will complete)
print("\n=== Testing with valid data ===")
workflow.variables["data"] = {"value": 42}
result = manager.execute("Data Processing", default_llm="gpt-4o-mini")
for step_result in result["results"]:
print(f" {step_result['step']}: {step_result['status']}")
@@ -0,0 +1,59 @@
"""
Agentic Loop with CSV Workflow Example
Demonstrates iterating over a CSV file with an agent processing each row.
"""
from praisonaiagents import Agent, Workflow
from praisonaiagents.workflows import loop
import tempfile
import os
# Create sample CSV file content
csv_content = """topic,description
AI Ethics,Ethical considerations in artificial intelligence
Machine Learning,Algorithms that learn from data
Neural Networks,Computing systems inspired by biological neural networks
Natural Language Processing,AI understanding human language"""
# Create processor agent
processor = Agent(
name="TopicProcessor",
role="Topic Analyst",
goal="Analyze and explain topics",
instructions="Provide a brief, insightful analysis of the given topic."
)
# Create summarizer agent
summarizer = Agent(
name="Summarizer",
role="Results Summarizer",
goal="Summarize all processed results",
instructions="Create a comprehensive summary of all the analyzed topics."
)
if __name__ == "__main__":
# Create temp CSV file
with tempfile.NamedTemporaryFile(mode='w', suffix='.csv', delete=False) as f:
f.write(csv_content)
csv_path = f.name
try:
print("=== Testing Agentic Loop with CSV ===\n")
print(f"Processing CSV: {csv_path}\n")
# Create workflow with loop over CSV
workflow = Workflow(
name="CSV Topic Processor",
steps=[
loop(processor, from_csv=csv_path),
summarizer
]
)
result = workflow.start("Analyze these AI topics", verbose=True)
print(f"\nFinal Summary:\n{result['output']}")
finally:
os.unlink(csv_path)
@@ -0,0 +1,55 @@
"""
Workflow Loop with CSV Example
Demonstrates iterating over a CSV file, processing each row.
"""
from praisonaiagents import Workflow, WorkflowContext, StepResult
from praisonaiagents.workflows import loop
import tempfile
import os
# Create sample CSV file
csv_content = """name,email,task
Alice,[email protected],Write documentation
Bob,[email protected],Review pull requests
Charlie,[email protected],Fix critical bug
Diana,[email protected],Deploy to production"""
# Processor for each CSV row
def process_task(ctx: WorkflowContext) -> StepResult:
"""Process a single task from CSV."""
item = ctx.variables.get("item", {})
name = item.get("name", "Unknown")
task = item.get("task", "No task")
index = ctx.variables.get("loop_index", 0)
return StepResult(
output=f"[{index+1}] {name}: {task} ✅",
variables={"last_processed": name}
)
if __name__ == "__main__":
# Create temp CSV file
with tempfile.NamedTemporaryFile(mode='w', suffix='.csv', delete=False) as f:
f.write(csv_content)
csv_path = f.name
try:
print("=== Testing Loop with CSV ===\n")
print(f"CSV file: {csv_path}\n")
# Create workflow
workflow = Workflow(
name="CSV Task Processor",
steps=[loop(process_task, from_csv=csv_path)]
)
result = workflow.start("Process all tasks", verbose=True)
print(f"\nAll outputs:\n{result['output']}")
print(f"\nProcessed {len(result['variables']['loop_outputs'])} items")
print(f"Last processed: {result['variables'].get('last_processed')}")
finally:
os.unlink(csv_path)
@@ -0,0 +1,52 @@
"""
Workflow Loop with List Example
Demonstrates iterating over a list of items, processing each one.
"""
from praisonaiagents import Workflow, WorkflowContext, StepResult
from praisonaiagents.workflows import loop
# Sample data
fruits = ["apple", "banana", "cherry", "date", "elderberry"]
# Processor for each item
def process_fruit(ctx: WorkflowContext) -> StepResult:
"""Process a single fruit from the list."""
fruit = ctx.variables.get("item", "unknown")
index = ctx.variables.get("loop_index", 0)
# Simulate processing
result = f"🍎 [{index + 1}] {fruit.upper()} - processed successfully"
return StepResult(
output=result,
variables={"last_fruit": fruit}
)
# Aggregator to summarize results
def summarize(ctx: WorkflowContext) -> StepResult:
"""Summarize all processed fruits."""
outputs = ctx.variables.get("loop_outputs", [])
summary = f"📊 Processed {len(outputs)} fruits:\n"
summary += "\n".join(f" {o}" for o in outputs)
return StepResult(output=summary)
# Create workflow
workflow = Workflow(
name="Fruit Processor",
steps=[
loop(process_fruit, over="fruits"),
summarize
],
variables={"fruits": fruits}
)
if __name__ == "__main__":
print("=== Testing Loop with List ===\n")
result = workflow.start("Process all fruits", verbose=True)
print(f"\nFinal Output:\n{result['output']}")
print(f"\nLoop outputs count: {len(result['variables']['loop_outputs'])}")
print(f"Last processed: {result['variables'].get('last_fruit')}")
@@ -0,0 +1,46 @@
"""
Workflow Loops Example
Demonstrates iterating over a list of items in a workflow step.
Each item is processed individually with the loop variable.
"""
from praisonaiagents.memory.workflows import WorkflowManager, WorkflowStep, Workflow
# Create a workflow with loop
workflow = Workflow(
name="Process Items",
description="Process a list of items using loop",
variables={
"items": ["apple", "banana", "cherry"] # List to iterate over
},
steps=[
WorkflowStep(
name="process_each",
action="Describe the fruit: {{item}}",
loop_over="items", # Variable name containing the list
loop_var="item" # Variable name for current item
),
WorkflowStep(
name="summarize",
action="Summarize all the fruits that were processed."
)
]
)
if __name__ == "__main__":
# Create manager and register workflow
manager = WorkflowManager()
manager.workflows["Process Items"] = workflow
# Execute - will process each fruit in the list
result = manager.execute(
"Process Items",
default_llm="gpt-4o-mini"
)
print("Workflow completed!")
for step_result in result["results"]:
print(f" {step_result['step']}: {step_result['status']}")
if isinstance(step_result.get("output"), list):
print(f" Loop processed {len(step_result['output'])} items")
@@ -0,0 +1,68 @@
"""
Mixed Steps Workflow Example
Demonstrates combining different step types in a single workflow:
- Agent instances
- Handler functions
- WorkflowStep with action strings
"""
from praisonaiagents import Workflow, WorkflowStep, WorkflowContext, StepResult, Agent
# Create an agent
analyzer = Agent(
name="Analyzer",
role="Data Analyst",
goal="Analyze and summarize data",
llm="gpt-4o-mini"
)
# Custom handler functions
def validate_input(ctx: WorkflowContext) -> StepResult:
"""Validate the input before processing."""
if len(ctx.input) < 5:
return StepResult(
output="Input too short. Minimum 5 characters required.",
stop_workflow=True
)
return StepResult(output=f"✅ Valid input: {ctx.input}")
def format_output(ctx: WorkflowContext) -> StepResult:
"""Format the final output."""
return StepResult(
output=f"📊 Report:\n{ctx.previous_result}",
variables={"report_generated": True}
)
# Create workflow with mixed step types
workflow = Workflow(
name="Mixed Pipeline",
steps=[
# Step 1: Function - validates input
validate_input,
# Step 2: Agent - analyzes content
analyzer,
# Step 3: WorkflowStep with action string
WorkflowStep(
name="enhance",
action="Add a conclusion to this analysis: {{previous_output}}"
),
# Step 4: Function - formats output
format_output
],
default_llm="gpt-4o-mini" # Used for action-based steps
)
if __name__ == "__main__":
# Test with valid input
print("=== Running mixed workflow ===")
result = workflow.start(
"Analyze the trends in AI adoption for 2024",
verbose=True
)
print(f"\nFinal output:\n{result['output']}")
print(f"\nVariables: {result['variables']}")
@@ -0,0 +1,56 @@
"""
Agentic Parallel Workflow Example
Demonstrates running multiple agents concurrently and
combining their results with an aggregator agent.
"""
from praisonaiagents import Agent, Workflow
from praisonaiagents.workflows import parallel
# Create parallel research agents
market_researcher = Agent(
name="MarketResearcher",
role="Market Research Analyst",
goal="Research market trends and opportunities",
instructions="Analyze market trends. Provide concise market insights."
)
competitor_researcher = Agent(
name="CompetitorResearcher",
role="Competitive Intelligence Analyst",
goal="Research competitor strategies",
instructions="Analyze competitors. Provide key competitive insights."
)
customer_researcher = Agent(
name="CustomerResearcher",
role="Customer Research Analyst",
goal="Research customer needs and behaviors",
instructions="Analyze customer segments. Provide customer insights."
)
# Create aggregator agent
aggregator = Agent(
name="Aggregator",
role="Research Synthesizer",
goal="Synthesize research findings",
instructions="Combine all research findings into a comprehensive summary."
)
# Create workflow with parallel execution
workflow = Workflow(
name="Parallel Research Pipeline",
steps=[
parallel([market_researcher, competitor_researcher, customer_researcher]),
aggregator
]
)
if __name__ == "__main__":
print("=== Testing Agentic Parallel Workflow ===\n")
# Run workflow - all researchers work in parallel, then aggregator summarizes
result = workflow.start("Research the AI industry", verbose=True)
print(f"\nFinal Summary:\n{result['output']}")
@@ -0,0 +1,53 @@
"""
Workflow Parallel Execution Example
Demonstrates running multiple steps concurrently and
combining their results.
"""
from praisonaiagents import Workflow, WorkflowContext, StepResult
from praisonaiagents.workflows import parallel
import time
# Parallel workers - each does independent work
def research_market(ctx: WorkflowContext) -> StepResult:
"""Research market trends."""
time.sleep(0.1) # Simulate work
return StepResult(output="📊 Market: Growth expected 15% YoY")
def research_competitors(ctx: WorkflowContext) -> StepResult:
"""Research competitor analysis."""
time.sleep(0.1) # Simulate work
return StepResult(output="🏢 Competitors: 3 major players identified")
def research_customers(ctx: WorkflowContext) -> StepResult:
"""Research customer feedback."""
time.sleep(0.1) # Simulate work
return StepResult(output="👥 Customers: 85% satisfaction rate")
# Aggregator - combines parallel results
def summarize_research(ctx: WorkflowContext) -> StepResult:
"""Summarize all research findings."""
outputs = ctx.variables.get("parallel_outputs", [])
summary = "📋 RESEARCH SUMMARY:\n" + "\n".join(f" • {o}" for o in outputs)
return StepResult(output=summary)
# Create workflow with parallel execution
workflow = Workflow(
name="Parallel Research",
steps=[
parallel([research_market, research_competitors, research_customers]),
summarize_research
]
)
if __name__ == "__main__":
print("=== Testing Parallel Workflow ===\n")
start_time = time.time()
result = workflow.start("Analyze the business landscape", verbose=True)
elapsed = time.time() - start_time
print(f"\nFinal Output:\n{result['output']}")
print(f"\nCompleted in {elapsed:.2f}s (parallel execution)")
print(f"Individual outputs: {len(result['variables']['parallel_outputs'])}")
@@ -0,0 +1,53 @@
"""
Agentic Evaluator-Optimizer (Repeat) Workflow Example
Demonstrates the evaluator-optimizer pattern where an agent
generates content and another evaluates it, repeating until approved.
"""
from praisonaiagents import Agent, Workflow
from praisonaiagents.workflows import repeat
# Create generator agent
generator = Agent(
name="Generator",
role="Content Generator",
goal="Generate high-quality content",
instructions="Generate content based on the topic. Improve based on feedback if provided."
)
# Create evaluator agent
evaluator = Agent(
name="Evaluator",
role="Content Evaluator",
goal="Evaluate and provide feedback on content",
instructions="Evaluate the content quality. If good, respond with 'APPROVED'. Otherwise provide specific feedback for improvement."
)
# Evaluation function to check if content is approved
def is_approved(ctx) -> bool:
return "approved" in ctx.previous_result.lower()
# Create workflow with evaluator-optimizer pattern
workflow = Workflow(
name="Evaluator-Optimizer Pipeline",
steps=[
generator, # First, generate content
repeat(
evaluator, # Evaluate and provide feedback
until=is_approved,
max_iterations=3
)
]
)
if __name__ == "__main__":
print("=== Testing Agentic Evaluator-Optimizer Workflow ===\n")
# Run optimization workflow
result = workflow.start(
"Write a compelling product description for an AI assistant",
verbose=True
)
print(f"\nFinal Result:\n{result['output']}")
@@ -0,0 +1,69 @@
"""
Workflow Repeat (Evaluator-Optimizer) Example
Demonstrates repeating a step until a condition is met.
This is useful for iterative improvement patterns.
"""
from praisonaiagents import Workflow, WorkflowContext, StepResult
from praisonaiagents.workflows import repeat
# Simulated content generator that improves each iteration
class ContentGenerator:
def __init__(self):
self.iteration = 0
self.content = []
def generate(self, ctx: WorkflowContext) -> StepResult:
"""Generate content, adding more each iteration."""
self.iteration += 1
# Add new content each iteration
new_items = [
f"Point {self.iteration}.1: Key insight about the topic",
f"Point {self.iteration}.2: Supporting evidence"
]
self.content.extend(new_items)
output = f"Iteration {self.iteration}: Generated {len(self.content)} points total"
return StepResult(
output=output,
variables={
"total_points": len(self.content),
"content": self.content.copy()
}
)
# Condition: stop when we have enough content
def has_enough_content(ctx: WorkflowContext) -> bool:
"""Check if we have generated enough content."""
total = ctx.variables.get("total_points", 0)
return total >= 6 # Stop when we have 6+ points
# Create generator instance
generator = ContentGenerator()
# Create workflow with repeat pattern
workflow = Workflow(
name="Content Generator",
steps=[
repeat(
generator.generate,
until=has_enough_content,
max_iterations=10
)
]
)
if __name__ == "__main__":
print("=== Testing Repeat (Evaluator-Optimizer) Pattern ===\n")
result = workflow.start("Generate comprehensive content", verbose=True)
print(f"\nFinal output: {result['output']}")
print(f"Total iterations: {result['variables']['repeat_iterations']}")
print(f"Total points generated: {result['variables']['total_points']}")
print("\nGenerated content:")
for point in result['variables']['content']:
print(f" • {point}")
@@ -0,0 +1,65 @@
"""
Agentic Routing Workflow Example
Demonstrates decision-based routing where an agent classifier
routes to specialized handler agents based on the request type.
"""
from praisonaiagents import Agent, Workflow
from praisonaiagents.workflows import route
# Create classifier agent
classifier = Agent(
name="Classifier",
role="Request Classifier",
goal="Classify incoming requests",
instructions="Classify the request. Respond with ONLY 'technical', 'creative', or 'general'."
)
# Create specialized handler agents
tech_agent = Agent(
name="TechExpert",
role="Technical Expert",
goal="Handle technical questions",
instructions="You are a technical expert. Provide detailed technical answers."
)
creative_agent = Agent(
name="CreativeWriter",
role="Creative Writer",
goal="Handle creative requests",
instructions="You are a creative writer. Write engaging, creative content."
)
general_agent = Agent(
name="GeneralAssistant",
role="General Assistant",
goal="Handle general requests",
instructions="You are a helpful assistant. Provide clear, helpful responses."
)
# Create workflow with routing
workflow = Workflow(
name="Agentic Router",
steps=[
classifier,
route({
"technical": [tech_agent],
"creative": [creative_agent],
"default": [general_agent]
})
]
)
if __name__ == "__main__":
print("=== Testing Agentic Routing Workflow ===\n")
# Test 1: Technical question
print("--- Technical Request ---")
result = workflow.start("How does machine learning work?", verbose=True)
print(f"Result: {result['output'][:200]}...\n")
# Test 2: Creative request
print("--- Creative Request ---")
result = workflow.start("Write a poem about the ocean", verbose=True)
print(f"Result: {result['output'][:200]}...")
@@ -0,0 +1,61 @@
"""
Workflow Routing Example
Demonstrates decision-based routing where the workflow
takes different paths based on the output of a decision step.
"""
from praisonaiagents import Workflow, WorkflowContext, StepResult
from praisonaiagents.workflows import route
# Decision maker - determines which route to take
def classify_request(ctx: WorkflowContext) -> StepResult:
"""Classify the input and decide the route."""
input_lower = ctx.input.lower()
if "urgent" in input_lower or "emergency" in input_lower:
return StepResult(output="priority: high")
elif "question" in input_lower or "help" in input_lower:
return StepResult(output="priority: support")
else:
return StepResult(output="priority: normal")
# Route handlers
def handle_high_priority(ctx: WorkflowContext) -> StepResult:
return StepResult(output="🚨 HIGH PRIORITY: Escalating to senior team immediately!")
def handle_support(ctx: WorkflowContext) -> StepResult:
return StepResult(output="💬 SUPPORT: Routing to help desk for assistance.")
def handle_normal(ctx: WorkflowContext) -> StepResult:
return StepResult(output="📋 NORMAL: Added to standard processing queue.")
# Create workflow with routing
workflow = Workflow(
name="Request Router",
steps=[
classify_request,
route({
"high": [handle_high_priority],
"support": [handle_support],
"normal": [handle_normal],
"default": [handle_normal]
})
]
)
if __name__ == "__main__":
# Test different inputs
print("=== Testing Routing Workflow ===\n")
# Test 1: Urgent request
result = workflow.start("This is an URGENT matter!", verbose=True)
print(f"Result: {result['output']}\n")
# Test 2: Support request
result = workflow.start("I have a question about my account", verbose=True)
print(f"Result: {result['output']}\n")
# Test 3: Normal request
result = workflow.start("Please process my order", verbose=True)
print(f"Result: {result['output']}")
@@ -0,0 +1,45 @@
"""
Workflow with Agents Example
Demonstrates using Agent instances directly as workflow steps.
Agents are automatically wrapped and executed in sequence.
"""
from praisonaiagents import Workflow, Agent
# Create agents
researcher = Agent(
name="Researcher",
role="Research Specialist",
goal="Find accurate information",
llm="gpt-4o-mini"
)
writer = Agent(
name="Writer",
role="Content Writer",
goal="Write engaging content",
llm="gpt-4o-mini"
)
editor = Agent(
name="Editor",
role="Content Editor",
goal="Polish and improve content",
llm="gpt-4o-mini"
)
# Create workflow with agents as steps
workflow = Workflow(
name="Content Pipeline",
steps=[researcher, writer, editor]
)
if __name__ == "__main__":
# Run the workflow
result = workflow.start(
"Write a short paragraph about artificial intelligence",
verbose=True
)
print(f"\nFinal output:\n{result['output']}")
+117
View File
@@ -0,0 +1,117 @@
# Extended agents.yaml with Workflow Patterns
# This file demonstrates the extended agents.yaml format that supports
# advanced workflow patterns while maintaining backward compatibility.
#
# Usage: praisonai agents_workflow.yaml
#
# Key differences from standard agents.yaml:
# - process: workflow (triggers workflow mode)
# - workflow: section for planning, reasoning, memory
# - steps: section for workflow patterns (route, parallel, loop, repeat)
framework: praisonai
process: workflow
input: "Research AI trends and create a comprehensive report" # The input passed to {{input}} in steps
# Workflow configuration (new)
workflow:
planning: true
reasoning: true
verbose: true
memory_config:
provider: chroma
persist: true
# Variables for template substitution
variables:
topic: AI trends
topics:
- Machine Learning
- Neural Networks
- Natural Language Processing
# Agent definitions using familiar 'roles' format
agents: # Canonical: use 'agents' instead of 'roles'
classifier:
role: Request Classifier
instructions: # Canonical |
You are an expert at classifying requests into categories.
Respond with ONLY one word: 'technical', 'creative', or 'research'.
goal: Classify incoming requests accurately
researcher:
role: Research Analyst
instructions: # Canonical |
You are a skilled research analyst with expertise in technology trends.
You provide concise, factual research findings with citations.
goal: Research topics thoroughly and provide insights
tools:
- tavily_search
llm:
model: gpt-4o-mini
tech_expert:
role: Technical Expert
instructions: # Canonical |
You are a technical expert with deep knowledge in AI and ML.
You provide detailed technical explanations.
goal: Handle technical questions with expertise
creative_writer:
role: Creative Writer
instructions: # Canonical |
You are a creative writer who can make complex topics engaging.
You write in a clear, accessible style.
goal: Create engaging content from research
market_analyst:
role: Market Analyst
instructions: # Canonical |
You analyze market trends and competitive landscapes.
You provide actionable business insights.
goal: Analyze market trends and opportunities
aggregator:
role: Synthesizer
instructions: # Canonical |
You combine findings from multiple sources into coherent reports.
You identify patterns and key insights.
goal: Synthesize all findings into comprehensive reports
# Workflow steps with patterns (new)
steps:
# Step 1: Classify the request
- agent: classifier
action: "Classify this request: {{topic}}"
# Step 2: Route to appropriate handler based on classification
- name: routing
route:
technical: [tech_expert]
creative: [creative_writer]
default: [researcher]
# Step 3: Parallel research on multiple aspects
- name: parallel_research
parallel:
- agent: researcher
action: "Research technology trends for {{topic}}"
- agent: market_analyst
action: "Analyze market opportunities for {{topic}}"
# Step 4: Loop over specific topics
- agent: researcher
action: "Deep dive into {{item}}"
loop:
over: topics
# Step 5: Aggregate all findings
- agent: aggregator
action: "Synthesize all research findings into a comprehensive report"
repeat:
until: "comprehensive"
max_iterations: 2
# Optional callbacks
callbacks:
on_step_complete: log_step
@@ -0,0 +1,116 @@
# Complete Workflow Example
# Demonstrates all workflow patterns in a single comprehensive workflow
name: Complete Research Workflow
description: A comprehensive workflow demonstrating all patterns - route, parallel, loop, repeat
framework: praisonai
workflow:
planning: true
reasoning: true
verbose: true
memory_config:
provider: chroma
persist: true
variables:
topic: AI trends
topics:
- Machine Learning
- Neural Networks
- Natural Language Processing
agents:
classifier:
name: Classifier
role: Request Classifier
goal: Classify incoming requests
instructions: |
Classify the request. Respond with ONLY one word:
- 'technical' for technical questions
- 'creative' for creative requests
- 'research' for research tasks
researcher:
name: Researcher
role: Research Analyst
goal: Research topics thoroughly
instructions: "You are a research analyst. Provide concise, factual research findings."
tech_expert:
name: TechExpert
role: Technical Expert
goal: Handle technical questions
instructions: "You are a technical expert. Provide detailed technical answers."
creative_writer:
name: CreativeWriter
role: Creative Writer
goal: Handle creative requests
instructions: "You are a creative writer. Write engaging, creative content."
market_analyst:
name: MarketAnalyst
role: Market Analyst
goal: Analyze market trends
instructions: "You are a market analyst. Provide market insights."
competitor_analyst:
name: CompetitorAnalyst
role: Competitor Analyst
goal: Analyze competitors
instructions: "You are a competitor analyst. Provide competitive insights."
aggregator:
name: Aggregator
role: Synthesizer
goal: Combine and synthesize findings
instructions: "You are a synthesizer. Combine all findings into a comprehensive report."
evaluator:
name: Evaluator
role: Quality Evaluator
goal: Evaluate content quality
instructions: |
Evaluate the content. If it's comprehensive and well-structured, respond with "APPROVED".
Otherwise, provide specific feedback for improvement.
steps:
# Step 1: Classify the request
- agent: classifier
action: "Classify: {{input}}"
# Step 2: Route to appropriate handler
- name: routing
route:
technical: [tech_expert]
creative: [creative_writer]
default: [researcher]
# Step 3: Parallel research
- name: parallel_research
parallel:
- agent: market_analyst
action: "Analyze market for {{topic}}"
- agent: competitor_analyst
action: "Analyze competitors for {{topic}}"
# Step 4: Loop over topics
- agent: researcher
action: "Research {{item}}"
loop:
over: topics
# Step 5: Aggregate findings
- agent: aggregator
action: "Synthesize all findings into a comprehensive report"
# Step 6: Evaluator-optimizer
- agent: evaluator
action: "Evaluate the report quality"
repeat:
until: "approved"
max_iterations: 2
callbacks:
on_step_complete: log_step
@@ -0,0 +1,43 @@
# Evaluator-Optimizer Workflow
# Demonstrates the repeat() pattern for iterative improvement
name: Evaluator Optimizer Workflow
description: A generator creates content, an evaluator reviews it repeatedly until quality standards are met
workflow:
verbose: true
agents:
generator:
name: Generator
role: Content Generator
goal: Generate high-quality content
instructions: |
You are a content generator. Generate content based on the request.
If feedback is provided, improve your content based on that feedback.
Always strive for clarity, accuracy, and engagement.
evaluator:
name: Evaluator
role: Content Evaluator
goal: Evaluate content quality and provide feedback
instructions: |
You are a content evaluator. Review the content provided.
If the content is high quality (clear, accurate, engaging), respond with:
"APPROVED: [brief reason]"
If the content needs improvement, respond with:
"NEEDS IMPROVEMENT: [specific feedback for improvement]"
Be constructive and specific in your feedback.
steps:
- agent: generator
action: "Generate content for: {{input}}"
- agent: evaluator
action: "Evaluate this content and provide feedback: {{previous_output}}"
repeat:
until: "approved"
max_iterations: 3
+152
View File
@@ -0,0 +1,152 @@
"""
Example usage of YAML Workflow Parser.
This script demonstrates how to load and execute YAML workflow files.
"""
import os
from pathlib import Path
# Ensure OPENAI_API_KEY is set
if not os.getenv("OPENAI_API_KEY"):
print("Please set OPENAI_API_KEY environment variable")
print("export OPENAI_API_KEY='your-api-key'")
exit(1)
from praisonaiagents.workflows import YAMLWorkflowParser, WorkflowManager
def example_1_parse_and_run():
"""Example 1: Parse YAML and run workflow directly."""
print("=" * 60)
print("Example 1: Parse YAML and run workflow")
print("=" * 60)
yaml_content = """
name: Quick Research
description: A quick research workflow
agents:
researcher:
name: Researcher
role: Research Analyst
goal: Research topics
instructions: "Provide brief, factual information."
steps:
- agent: researcher
action: "Research: {{input}}"
"""
parser = YAMLWorkflowParser()
workflow = parser.parse_string(yaml_content)
print(f"Workflow Name: {workflow.name}")
print(f"Steps: {len(workflow.steps)}")
# Run the workflow
result = workflow.start("What is Python?")
print(f"Result: {result['output'][:200]}...")
print()
def example_2_load_yaml_file():
"""Example 2: Load workflow from YAML file using WorkflowManager."""
print("=" * 60)
print("Example 2: Load YAML file with WorkflowManager")
print("=" * 60)
# Get the directory of this script
script_dir = Path(__file__).parent
yaml_file = script_dir / "simple_workflow.yaml"
if not yaml_file.exists():
print(f"YAML file not found: {yaml_file}")
return
manager = WorkflowManager()
workflow = manager.load_yaml(yaml_file)
print(f"Loaded workflow: {workflow.name}")
print(f"Steps: {len(workflow.steps)}")
# Run the workflow
result = workflow.start("Benefits of renewable energy")
print(f"Result: {result['output'][:200]}...")
print()
def example_3_execute_yaml_directly():
"""Example 3: Execute YAML file directly with WorkflowManager."""
print("=" * 60)
print("Example 3: Execute YAML directly")
print("=" * 60)
script_dir = Path(__file__).parent
yaml_file = script_dir / "simple_workflow.yaml"
if not yaml_file.exists():
print(f"YAML file not found: {yaml_file}")
return
manager = WorkflowManager()
result = manager.execute_yaml(
yaml_file,
input_data="The future of AI",
verbose=True
)
print(f"Status: {result['status']}")
print(f"Result: {result['output'][:200]}...")
print()
def example_4_with_variables():
"""Example 4: YAML workflow with variables."""
print("=" * 60)
print("Example 4: Workflow with variables")
print("=" * 60)
yaml_content = """
name: Variable Workflow
variables:
topic: Machine Learning
style: concise
agents:
writer:
name: Writer
role: Content Writer
goal: Write content
instructions: "Write {{style}} content about the topic."
steps:
- agent: writer
action: "Write about {{topic}}"
"""
parser = YAMLWorkflowParser()
workflow = parser.parse_string(yaml_content)
print(f"Variables: {workflow.variables}")
# Run with default variables
result = workflow.start("")
print(f"Result: {result['output'][:200]}...")
print()
if __name__ == "__main__":
print("YAML Workflow Examples")
print("=" * 60)
print()
# Run examples
example_1_parse_and_run()
example_2_load_yaml_file()
example_3_execute_yaml_directly()
example_4_with_variables()
print("=" * 60)
print("All examples completed!")
print("=" * 60)
@@ -0,0 +1,42 @@
# Loop Workflow
# Demonstrates the loop() pattern for iterating over items
name: Repetitive Processing Workflow
description: An agent processes multiple items in a loop, then a summarizer combines results
workflow:
verbose: true
variables:
topics:
- Artificial Intelligence
- Machine Learning
- Neural Networks
- Natural Language Processing
agents:
processor:
name: Processor
role: Topic Processor
goal: Process and analyze each topic thoroughly
instructions: |
You are a topic processor. For each topic given, provide:
1. A brief definition
2. Key concepts
3. Real-world applications
Keep your response concise but informative.
summarizer:
name: Summarizer
role: Results Summarizer
goal: Summarize all processed results into a final report
instructions: "You are a summarizer. Combine all the processed topics into a cohesive final report."
steps:
- agent: processor
action: "Analyze this topic: {{item}}"
loop:
over: topics
- agent: summarizer
action: "Create a comprehensive summary of all the analyzed topics based on these results: {{previous_output}}"
@@ -0,0 +1,47 @@
# Parallel Workflow
# Demonstrates the parallel() pattern for concurrent execution
name: Parallel Research Workflow
description: Multiple agents research different aspects concurrently, then an aggregator combines results
input: "AI-powered customer service solutions" # The input passed to {{input}} in steps
workflow:
verbose: true
agents:
market_researcher:
name: MarketResearcher
role: Market Research Analyst
goal: Research market trends and opportunities
instructions: "You are a market research analyst. Analyze market trends and provide concise insights."
competitor_researcher:
name: CompetitorResearcher
role: Competitive Intelligence Analyst
goal: Research competitor strategies and positioning
instructions: "You are a competitive intelligence analyst. Analyze competitors and provide key insights."
customer_researcher:
name: CustomerResearcher
role: Customer Research Analyst
goal: Research customer needs and behaviors
instructions: "You are a customer research analyst. Analyze customer segments and provide insights."
aggregator:
name: Aggregator
role: Research Synthesizer
goal: Synthesize all research findings into a comprehensive summary
instructions: "You are a research synthesizer. Combine all research findings into a clear, comprehensive summary."
steps:
- name: parallel_research
parallel:
- agent: market_researcher
action: "Research market trends for: {{input}}"
- agent: competitor_researcher
action: "Research competitors for: {{input}}"
- agent: customer_researcher
action: "Research customer needs for: {{input}}"
- agent: aggregator
action: "Synthesize all the research findings into a comprehensive report: {{previous_output}}"
@@ -0,0 +1,50 @@
# Routing Workflow
# Demonstrates the route() pattern for decision-based branching
name: Agentic Routing Workflow
description: A classifier agent routes requests to specialized handler agents
input: "Write a poem about the ocean" # The input passed to {{input}} in steps
workflow:
verbose: true
agents:
classifier:
name: Classifier
role: Request Classifier
goal: Classify incoming requests by type
instructions: |
You are a request classifier. Analyze the request and respond with ONLY one word:
- 'technical' for technical/coding questions
- 'creative' for creative writing requests
- 'general' for everything else
Respond with ONLY the classification word, nothing else.
tech_expert:
name: TechExpert
role: Technical Expert
goal: Answer technical questions with expertise
instructions: "You are a technical expert. Provide detailed, accurate technical answers."
creative_writer:
name: CreativeWriter
role: Creative Writer
goal: Create engaging creative content
instructions: "You are a creative writer. Write engaging, imaginative content."
general_assistant:
name: GeneralAssistant
role: General Assistant
goal: Help with general requests
instructions: "You are a helpful assistant. Provide clear, helpful responses."
steps:
- agent: classifier
action: "Classify this request: {{input}}"
- name: routing
route:
technical: [tech_expert]
creative: [creative_writer]
default: [general_assistant]
@@ -0,0 +1,29 @@
# Simple Sequential Workflow
# Demonstrates basic agent-to-agent workflow
name: Simple Research Workflow
description: A simple workflow where a researcher gathers info and a writer summarizes it
input: "The benefits of renewable energy" # The input passed to {{input}} in steps
workflow:
verbose: true
agents:
researcher:
name: Researcher
role: Research Analyst
goal: Research and gather information about topics
instructions: "You are a research analyst. Provide concise, factual information about the given topic."
writer:
name: Writer
role: Content Writer
goal: Write clear and engaging summaries
instructions: "You are a content writer. Take the research provided and write a clear, engaging summary."
steps:
- agent: researcher
action: "Research the following topic: {{input}}"
- agent: writer
action: "Write a summary based on this research: {{previous_output}}"
+1
View File
@@ -87,3 +87,4 @@ graph.py
chroma_db/
.qodo
*_COMPARISON_ANALYSIS.md
*_REPORT.md
File diff suppressed because it is too large. Load diff
+1
View File
@@ -0,0 +1 @@
../../README.md
@@ -0,0 +1,35 @@
# PraisonAI Agents - Benchmark Results
**Generated:** 2025-12-18 14:40:22
**Iterations:** 100
**Test:** Agent instantiation (without tools)
## Results
| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
| **PraisonAI** | **3.77** | **1.00x (fastest)** |
| OpenAI Agents SDK | 5.26 | 1.39x |
| Agno | 5.64 | 1.49x |
| PraisonAI (LiteLLM) | 7.56 | 2.00x |
| PydanticAI | 226.94 | 60.16x |
| LangGraph | 4,558.71 | 1,209x |
| CrewAI | 15,607.92 | 4,138x |
## Package Versions
| Package | Version |
|---------|--------|
| PraisonAI | 0.1.5 |
| Agno | 2.3.14 |
| PydanticAI | 1.35.0 |
| OpenAI Agents SDK | 0.6.3 |
| LangGraph | 1.0.5 |
| CrewAI | 1.6.1 |
## How to Reproduce
```bash
cd praisonai-agents
python benchmarks/simple_benchmark.py
```
@@ -0,0 +1,31 @@
# PraisonAI Agents - Real Execution Benchmark
**Generated:** 2025-12-18 14:43:52
**Model:** gpt-4o-mini
**Iterations:** 3
**Prompt:** "What is 2+2? Reply with just the number."
## Results
| Framework | Method | Avg Time | Relative |
|-----------|--------|----------|----------|
| **PraisonAI** | `agent.start()` | **0.45s** | **1.00x (fastest)** |
| PraisonAI (LiteLLM) | `agent.start()` | 0.55s | 1.22x |
| CrewAI | `crew.kickoff()` | 0.58s | 1.28x |
| Agno | `agent.run()` | 0.92s | 2.05x |
## Package Versions
| Package | Version |
|---------|--------|
| PraisonAI | 0.1.5 |
| Agno | 2.3.14 |
| CrewAI | 1.6.1 |
## How to Reproduce
```bash
export OPENAI_API_KEY=your_key
cd praisonai-agents
python benchmarks/execution_benchmark.py
```
@@ -0,0 +1,72 @@
# PraisonAI Agents - Real-World Execution Benchmark
**Generated:** 2025-12-18 14:33:49
**Model:** gpt-4o-mini
**Iterations:** 3 per test
## Results Summary
| Test | PraisonAI (avg) | Agno (avg) | Difference |
|------|-----------------|------------|------------|
| Single Agent | 2.04s | 3.18s | PraisonAI 1.56x faster |
| 2 Agents | 2.49s | 20.49s | PraisonAI 8.21x faster |
| 3 Agents | 6.52s | 13.97s | PraisonAI 2.14x faster |
## Detailed Results
### praisonai_single
- Average: 2.04s
- Min: 1.95s
- Max: 2.19s
- Std Dev: 0.13s
- Times: ['2.19s', '1.95s', '2.00s']
### agno_single
- Average: 3.18s
- Min: 3.00s
- Max: 3.51s
- Std Dev: 0.28s
- Times: ['3.51s', '3.00s', '3.04s']
### praisonai_two_agents
- Average: 2.49s
- Min: 2.22s
- Max: 2.65s
- Std Dev: 0.24s
- Times: ['2.65s', '2.22s', '2.62s']
### agno_two_agents
- Average: 20.49s
- Min: 9.96s
- Max: 35.05s
- Std Dev: 13.02s
- Times: ['16.46s', '9.96s', '35.05s']
### praisonai_three_agents
- Average: 6.52s
- Min: 5.62s
- Max: 7.39s
- Std Dev: 0.88s
- Times: ['5.62s', '6.54s', '7.39s']
### agno_three_agents
- Average: 13.97s
- Min: 11.69s
- Max: 15.32s
- Std Dev: 1.99s
- Times: ['14.91s', '11.69s', '15.32s']
## Package Versions
| Package | Version |
|---------|--------|
| PraisonAI | 0.1.5 |
| Agno | 2.3.14 |
## How to Reproduce
```bash
export OPENAI_API_KEY=your_key
cd praisonai-agents
python benchmarks/real_benchmark.py
```
@@ -0,0 +1,22 @@
# PraisonAI Agents - Benchmark Results
**Generated:** 2025-12-18 13:23:43
## Agent Instantiation Time
| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
| **PraisonAI** | **4.16** | **1.00x (fastest)** |
| OpenAI Agents SDK | 5.55 | 1.33x |
| Agno | 5.58 | 1.34x |
| PraisonAI (LiteLLM) | 8.52 | 2.05x |
| PydanticAI | 225.87 | 54.25x |
| LangGraph | 4,349.54 | 1,045x |
| CrewAI | 15,911.11 | 3,821x |
## How to Reproduce
```bash
cd praisonai-agents
python benchmarks/simple_benchmark.py
```
@@ -0,0 +1,33 @@
# PraisonAI Agents - Tools Benchmark Results
**Generated:** 2025-12-18 14:40:35
**Iterations:** 100
**Test:** Agent instantiation WITH TOOLS
## Results
| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
| **PraisonAI** | **3.24** | **1.00x (fastest)** |
| Agno | 5.12 | 1.58x |
| PraisonAI (LiteLLM) | 8.59 | 2.65x |
| OpenAI Agents SDK | 279.95 | 86.44x |
| LangGraph | 2,310.82 | 713x |
| CrewAI | 15,773.44 | 4,870x |
## Package Versions
| Package | Version |
|---------|--------|
| PraisonAI | 0.1.5 |
| Agno | 2.3.14 |
| OpenAI Agents SDK | 0.6.3 |
| LangGraph | 1.0.5 |
| CrewAI | 1.6.1 |
## How to Reproduce
```bash
cd praisonai-agents
python benchmarks/tools_benchmark.py
```
@@ -0,0 +1,368 @@
#!/usr/bin/env python3
"""
PraisonAI Agents - Deep Profiling Benchmark
Profiles each step of agent initialization to identify performance bottlenecks.
Key Optimizations Applied:
1. Lazy RulesManager - filesystem operations deferred until first access
2. Lazy os.getcwd() - fast_context_path resolved only when needed
3. Lazy console initialization - Rich Console created on demand
Usage:
python benchmarks/deep_profiling.py
"""
import time
import tracemalloc
import cProfile
import pstats
import io
from typing import Literal, Callable, Dict, List
from dataclasses import dataclass
# ============================================================================
# Timing Utilities
# ============================================================================
@dataclass
class StepTiming:
name: str
duration_us: float # microseconds
memory_kb: float # kilobytes
class StepProfiler:
"""Profile individual steps during initialization"""
def __init__(self):
self.steps: List[StepTiming] = []
self._start_time = None
self._start_memory = None
def start_step(self, name: str):
tracemalloc.start()
self._start_time = time.perf_counter()
self._step_name = name
def end_step(self):
end_time = time.perf_counter()
current, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()
duration_us = (end_time - self._start_time) * 1_000_000
memory_kb = peak / 1024
self.steps.append(StepTiming(
name=self._step_name,
duration_us=duration_us,
memory_kb=memory_kb
))
def print_report(self, title: str):
print(f"\n{'='*70}")
print(f"STEP-BY-STEP PROFILING: {title}")
print(f"{'='*70}")
total_time = sum(s.duration_us for s in self.steps)
total_memory = sum(s.memory_kb for s in self.steps)
print(f"\n{'Step':<40} {'Time (μs)':<15} {'Memory (KB)':<15} {'% Time':<10}")
print("-" * 80)
for step in sorted(self.steps, key=lambda x: x.duration_us, reverse=True):
pct = (step.duration_us / total_time * 100) if total_time > 0 else 0
print(f"{step.name:<40} {step.duration_us:<15.2f} {step.memory_kb:<15.2f} {pct:<10.1f}%")
print("-" * 80)
print(f"{'TOTAL':<40} {total_time:<15.2f} {total_memory:<15.2f} {'100.0':<10}%")
def time_function(func: Callable, iterations: int = 100) -> Dict:
"""Time a function with detailed statistics"""
times = []
# Warmup
for _ in range(10):
func()
# Measure
for _ in range(iterations):
start = time.perf_counter()
func()
end = time.perf_counter()
times.append((end - start) * 1_000_000) # Convert to microseconds
return {
'avg': sum(times) / len(times),
'min': min(times),
'max': max(times),
'median': sorted(times)[len(times)//2]
}
# ============================================================================
# Profile PraisonAI Agent Initialization - Step by Step
# ============================================================================
def profile_praisonai_detailed():
"""Profile each step of PraisonAI Agent initialization"""
print("\n" + "="*70)
print("DETAILED PROFILING: PraisonAI Agent")
print("="*70)
profiler = StepProfiler()
# Step 1: Import
profiler.start_step("1. Import praisonaiagents")
from praisonaiagents import Agent as PraisonAgent
profiler.end_step()
# Step 2: Define tool
profiler.start_step("2. Define tool function")
def get_weather(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
tools = [get_weather]
profiler.end_step()
# Step 3: Full instantiation
profiler.start_step("3. Full Agent instantiation")
agent = PraisonAgent(
name="Test Agent",
instructions="Be concise, reply with one sentence.",
llm="gpt-4o-mini",
tools=tools,
verbose=False
)
profiler.end_step()
profiler.print_report("PraisonAI Agent")
# Now profile the __init__ internals using cProfile
print("\n" + "="*70)
print("cProfile ANALYSIS: PraisonAI Agent.__init__")
print("="*70)
pr = cProfile.Profile()
pr.enable()
for _ in range(100):
agent = PraisonAgent(
name="Test Agent",
instructions="Be concise, reply with one sentence.",
llm="gpt-4o-mini",
tools=tools,
verbose=False
)
pr.disable()
# Print top 30 functions by cumulative time
s = io.StringIO()
ps = pstats.Stats(pr, stream=s).sort_stats('cumulative')
ps.print_stats(30)
print(s.getvalue())
return agent
# ============================================================================
# Profile Agno Agent Initialization - Step by Step
# ============================================================================
def profile_agno_detailed():
"""Profile each step of Agno Agent initialization"""
print("\n" + "="*70)
print("DETAILED PROFILING: Agno Agent")
print("="*70)
profiler = StepProfiler()
# Step 1: Import
profiler.start_step("1. Import agno")
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
profiler.end_step()
# Step 2: Define tool
profiler.start_step("2. Define tool function")
def get_weather(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
tools = [get_weather]
profiler.end_step()
# Step 3: Full instantiation
profiler.start_step("3. Full Agent instantiation")
agent = AgnoAgent(model=OpenAIChat(id="gpt-4o-mini"), tools=tools)
profiler.end_step()
profiler.print_report("Agno Agent")
# Now profile the __init__ internals using cProfile
print("\n" + "="*70)
print("cProfile ANALYSIS: Agno Agent.__init__")
print("="*70)
pr = cProfile.Profile()
pr.enable()
for _ in range(100):
agent = AgnoAgent(model=OpenAIChat(id="gpt-4o-mini"), tools=tools)
pr.disable()
# Print top 30 functions by cumulative time
s = io.StringIO()
ps = pstats.Stats(pr, stream=s).sort_stats('cumulative')
ps.print_stats(30)
print(s.getvalue())
return agent
# ============================================================================
# Profile Individual Components of PraisonAI
# ============================================================================
def profile_praisonai_components():
"""Profile individual components of PraisonAI Agent initialization"""
print("\n" + "="*70)
print("COMPONENT-LEVEL PROFILING: PraisonAI Agent")
print("="*70)
from praisonaiagents import Agent as PraisonAgent
def get_weather(city: Literal["nyc", "sf"]):
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
tools = [get_weather]
# Test different configurations
configs = [
("Minimal (no tools, no llm)", {"name": "Test", "verbose": False}),
("With name only", {"name": "Test Agent", "verbose": False}),
("With instructions", {"name": "Test", "instructions": "Be concise", "verbose": False}),
("With llm (gpt-4o-mini)", {"name": "Test", "llm": "gpt-4o-mini", "verbose": False}),
("With llm (openai/gpt-4o-mini)", {"name": "Test", "llm": "openai/gpt-4o-mini", "verbose": False}),
("With tools", {"name": "Test", "tools": tools, "verbose": False}),
("Full config (OpenAI SDK)", {"name": "Test", "instructions": "Be concise", "llm": "gpt-4o-mini", "tools": tools, "verbose": False}),
("Full config (LiteLLM)", {"name": "Test", "instructions": "Be concise", "llm": "openai/gpt-4o-mini", "tools": tools, "verbose": False}),
]
print(f"\n{'Configuration':<40} {'Avg Time (μs)':<15} {'Min (μs)':<15} {'Max (μs)':<15}")
print("-" * 85)
for name, config in configs:
def create_agent():
return PraisonAgent(**config)
stats = time_function(create_agent, iterations=100)
print(f"{name:<40} {stats['avg']:<15.2f} {stats['min']:<15.2f} {stats['max']:<15.2f}")
# ============================================================================
# Compare Specific Operations
# ============================================================================
def compare_specific_operations():
"""Compare specific operations between PraisonAI and Agno"""
print("\n" + "="*70)
print("OPERATION COMPARISON: PraisonAI vs Agno")
print("="*70)
from praisonaiagents import Agent as PraisonAgent
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
def get_weather(city: Literal["nyc", "sf"]):
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
tools = [get_weather]
operations = []
# Operation 1: Bare minimum instantiation
def praisonai_minimal():
return PraisonAgent(name="Test", verbose=False)
def agno_minimal():
return AgnoAgent(model=OpenAIChat(id="gpt-4o-mini"))
praison_stats = time_function(praisonai_minimal, 100)
agno_stats = time_function(agno_minimal, 100)
operations.append(("Minimal instantiation", praison_stats['avg'], agno_stats['avg']))
# Operation 2: With tools
def praisonai_with_tools():
return PraisonAgent(name="Test", tools=tools, verbose=False)
def agno_with_tools():
return AgnoAgent(model=OpenAIChat(id="gpt-4o-mini"), tools=tools)
praison_stats = time_function(praisonai_with_tools, 100)
agno_stats = time_function(agno_with_tools, 100)
operations.append(("With tools", praison_stats['avg'], agno_stats['avg']))
# Operation 3: Full config
def praisonai_full():
return PraisonAgent(
name="Test Agent",
instructions="Be concise",
llm="gpt-4o-mini",
tools=tools,
verbose=False
)
def agno_full():
return AgnoAgent(
model=OpenAIChat(id="gpt-4o-mini"),
tools=tools,
instructions=["Be concise"]
)
praison_stats = time_function(praisonai_full, 100)
agno_stats = time_function(agno_full, 100)
operations.append(("Full config", praison_stats['avg'], agno_stats['avg']))
print(f"\n{'Operation':<30} {'PraisonAI (μs)':<18} {'Agno (μs)':<18} {'Ratio':<10}")
print("-" * 76)
for op_name, praison_time, agno_time in operations:
ratio = praison_time / agno_time if agno_time > 0 else 0
print(f"{op_name:<30} {praison_time:<18.2f} {agno_time:<18.2f} {ratio:<10.1f}x")
# ============================================================================
# Main
# ============================================================================
if __name__ == "__main__":
import os
os.environ.setdefault("OPENAI_API_KEY", "sk-test")
print("="*70)
print("PraisonAI Agents - Deep Profiling Benchmark")
print("="*70)
print("\nThis analysis identifies performance bottlenecks in agent initialization.\n")
# Run all profiling
profile_praisonai_detailed()
profile_agno_detailed()
profile_praisonai_components()
compare_specific_operations()
print("\n" + "="*70)
print("PROFILING COMPLETE")
print("="*70)
@@ -0,0 +1,296 @@
#!/usr/bin/env python3
"""
PraisonAI Agents - Real Execution Benchmark
Benchmarks ACTUAL agent execution with LLM API calls.
Tests single agent running a simple task across frameworks.
Usage:
export OPENAI_API_KEY=your_key
python benchmarks/execution_benchmark.py
python benchmarks/execution_benchmark.py --no-save
python benchmarks/execution_benchmark.py --iterations 5
python benchmarks/execution_benchmark.py --model gpt-4o --iterations 10
"""
import time
import os
import argparse
from importlib.metadata import version as get_version
# Defaults (can be overridden via CLI)
DEFAULT_MODEL = "gpt-4o-mini"
DEFAULT_ITERATIONS = 3
DEFAULT_PROMPT = "What is 2+2? Reply with just the number."
def get_package_versions():
"""Get version numbers for benchmarked packages."""
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
'CrewAI': 'crewai',
}
versions = {}
for name, pkg in packages.items():
try:
versions[name] = get_version(pkg)
except:
versions[name] = 'not installed'
return versions
def benchmark_praisonai(model, iterations, prompt):
"""Benchmark PraisonAI agent execution using agent.start()"""
from praisonaiagents import Agent
print("\n--- PraisonAI ---")
print("Method: agent.start()")
times = []
for i in range(iterations):
agent = Agent(
name="Calculator",
instructions="You are a helpful assistant. Be very brief.",
llm=model,
verbose=False
)
start = time.perf_counter()
response = agent.start(prompt)
elapsed = time.perf_counter() - start
times.append(elapsed)
print(f" Run {i+1}: {elapsed:.2f}s - Response: {str(response)[:50]}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def benchmark_praisonai_litellm(model, iterations, prompt):
"""Benchmark PraisonAI with LiteLLM backend using agent.start()"""
from praisonaiagents import Agent
print("\n--- PraisonAI (LiteLLM) ---")
print("Method: agent.start()")
times = []
for i in range(iterations):
agent = Agent(
name="Calculator",
instructions="You are a helpful assistant. Be very brief.",
llm=f"openai/{model}",
verbose=False
)
start = time.perf_counter()
response = agent.start(prompt)
elapsed = time.perf_counter() - start
times.append(elapsed)
print(f" Run {i+1}: {elapsed:.2f}s - Response: {str(response)[:50]}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def benchmark_agno(model, iterations, prompt):
"""Benchmark Agno agent execution using agent.run()"""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
print("\n--- Agno ---")
print("Method: agent.run()")
times = []
for i in range(iterations):
agent = Agent(
model=OpenAIChat(id=model),
instructions=["You are a helpful assistant. Be very brief."],
)
start = time.perf_counter()
response = agent.run(prompt)
elapsed = time.perf_counter() - start
times.append(elapsed)
# Extract content from response
content = str(response.content)[:50] if hasattr(response, 'content') else str(response)[:50]
print(f" Run {i+1}: {elapsed:.2f}s - Response: {content}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def benchmark_crewai(model, iterations, prompt):
"""Benchmark CrewAI agent execution using crew.kickoff()"""
from crewai import Agent, Task, Crew
print("\n--- CrewAI ---")
print("Method: crew.kickoff()")
times = []
for i in range(iterations):
agent = Agent(
role="Calculator",
goal="Answer math questions",
backstory="You are a helpful assistant.",
llm=model,
verbose=False
)
task = Task(
description=prompt,
expected_output="A number",
agent=agent
)
crew = Crew(
agents=[agent],
tasks=[task],
verbose=False
)
start = time.perf_counter()
response = crew.kickoff()
elapsed = time.perf_counter() - start
times.append(elapsed)
print(f" Run {i+1}: {elapsed:.2f}s - Response: {str(response)[:50]}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def save_results(results: dict, model: str, iterations: int, prompt: str):
"""Save results to markdown file."""
import os
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), 'EXECUTION_BENCHMARK_RESULTS.md')
versions = get_package_versions()
# Sort by time
sorted_results = sorted(results.items(), key=lambda x: x[1])
fastest = sorted_results[0][1]
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Real Execution Benchmark\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write(f'**Model:** {model}\n')
f.write(f'**Iterations:** {iterations}\n')
f.write(f'**Prompt:** "{prompt}"\n\n')
f.write('## Results\n\n')
f.write('| Framework | Method | Avg Time | Relative |\n')
f.write('|-----------|--------|----------|----------|\n')
methods = {
'PraisonAI': 'agent.start()',
'PraisonAI (LiteLLM)': 'agent.start()',
'Agno': 'agent.run()',
'CrewAI': 'crew.kickoff()',
}
for name, avg in sorted_results:
ratio = avg / fastest
method = methods.get(name, 'unknown')
if ratio == 1.0:
f.write(f'| **{name}** | `{method}` | **{avg:.2f}s** | **1.00x (fastest)** |\n')
else:
f.write(f'| {name} | `{method}` | {avg:.2f}s | {ratio:.2f}x |\n')
f.write('\n## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('export OPENAI_API_KEY=your_key\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/execution_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='PraisonAI Agents - Real Execution Benchmark')
parser.add_argument('--no-save', action='store_true', help='Do not save results to file')
parser.add_argument('--model', '-m', type=str, default=DEFAULT_MODEL, help=f'Model to use (default: {DEFAULT_MODEL})')
parser.add_argument('--iterations', '-i', type=int, default=DEFAULT_ITERATIONS, help=f'Number of iterations (default: {DEFAULT_ITERATIONS})')
parser.add_argument('--prompt', '-p', type=str, default=DEFAULT_PROMPT, help=f'Prompt to use (default: "{DEFAULT_PROMPT}")')
args = parser.parse_args()
# Check API key
if not os.environ.get('OPENAI_API_KEY'):
print("ERROR: Set OPENAI_API_KEY environment variable")
exit(1)
model = args.model
iterations = args.iterations
prompt = args.prompt
print('=' * 60)
print('PraisonAI Agents - Real Execution Benchmark')
print('=' * 60)
print(f'\nModel: {model}')
print(f'Iterations: {iterations}')
print(f'Prompt: "{prompt}"')
versions = get_package_versions()
print('\nPackage versions:')
for pkg, ver in versions.items():
print(f' {pkg}: {ver}')
results = {}
# Run benchmarks
try:
results['PraisonAI'] = benchmark_praisonai(model, iterations, prompt)
except Exception as e:
print(f"PraisonAI error: {e}")
try:
results['PraisonAI (LiteLLM)'] = benchmark_praisonai_litellm(model, iterations, prompt)
except Exception as e:
print(f"PraisonAI (LiteLLM) error: {e}")
try:
results['Agno'] = benchmark_agno(model, iterations, prompt)
except Exception as e:
print(f"Agno error: {e}")
try:
results['CrewAI'] = benchmark_crewai(model, iterations, prompt)
except Exception as e:
print(f"CrewAI error: {e}")
# Summary
print('\n' + '=' * 60)
print('SUMMARY')
print('=' * 60)
if results:
sorted_results = sorted(results.items(), key=lambda x: x[1])
fastest = sorted_results[0][1]
print(f"\n{'Framework':<25} {'Avg Time':<12} {'Relative':<10}")
print('-' * 47)
for name, avg in sorted_results:
ratio = avg / fastest
print(f'{name:<25} {avg:<12.2f}s {ratio:.2f}x')
print('\n' + '=' * 60)
# Save results (unless --no-save)
if results and not args.no_save:
save_results(results, model, iterations, prompt)
elif args.no_save:
print('\nResults not saved (--no-save flag used)')
@@ -0,0 +1,523 @@
"""
PraisonAI Agents - Comprehensive Performance Benchmark
Measures agent instantiation time and memory footprint across
multiple AI agent frameworks using industry-standard methodology.
Usage:
python benchmarks/performance_benchmark.py
"""
import gc
import time
import tracemalloc
import statistics
from typing import Literal, List, Callable
from dataclasses import dataclass, field
@dataclass
class PerformanceResult:
"""Performance measurement results"""
run_times: List[float] = field(default_factory=list)
memory_usages: List[float] = field(default_factory=list)
@property
def avg_run_time(self) -> float:
return statistics.mean(self.run_times) if self.run_times else 0
@property
def avg_memory_usage(self) -> float:
return statistics.mean(self.memory_usages) if self.memory_usages else 0
@property
def min_run_time(self) -> float:
return min(self.run_times) if self.run_times else 0
@property
def max_run_time(self) -> float:
return max(self.run_times) if self.run_times else 0
@property
def std_dev_run_time(self) -> float:
return statistics.stdev(self.run_times) if len(self.run_times) > 1 else 0
@property
def median_run_time(self) -> float:
return statistics.median(self.run_times) if self.run_times else 0
def compute_tracemalloc_baseline(samples: int = 3) -> float:
"""Compute baseline memory usage for tracemalloc"""
def empty_func():
return
results = []
for _ in range(samples):
gc.collect()
tracemalloc.start()
empty_func()
_, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()
results.append(peak / 1024 / 1024) # Convert to MiB
return sum(results) / len(results) if results else 0
def measure_time(func: Callable, iterations: int = 1000, warmup: int = 10) -> List[float]:
"""Measure execution time for a function"""
# Warmup runs
for _ in range(warmup):
func()
# Measured runs
times = []
for _ in range(iterations):
start = time.perf_counter()
func()
end = time.perf_counter()
times.append(end - start)
return times
def measure_memory(func: Callable, iterations: int = 1000, warmup: int = 10) -> List[float]:
"""Measure memory usage for a function"""
baseline = compute_tracemalloc_baseline()
# Warmup runs
for _ in range(warmup):
func()
# Measured runs
memory_usages = []
for _ in range(iterations):
gc.collect()
tracemalloc.start()
func()
_, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()
peak_mib = peak / 1024 / 1024
adjusted = max(0, peak_mib - baseline)
memory_usages.append(adjusted)
return memory_usages
def run_benchmark(func: Callable, name: str, iterations: int = 1000, warmup: int = 10) -> PerformanceResult:
"""Run full benchmark for a function"""
print(f"\n{'='*60}")
print(f"Benchmarking: {name}")
print(f"{'='*60}")
print(f"Warmup runs: {warmup}")
print(f"Measured iterations: {iterations}")
print("\nMeasuring runtime...")
run_times = measure_time(func, iterations, warmup)
print("Measuring memory...")
memory_usages = measure_memory(func, iterations, warmup)
result = PerformanceResult(run_times=run_times, memory_usages=memory_usages)
print(f"\nResults for {name}:")
print(f" Average Time: {result.avg_run_time:.6f} seconds ({result.avg_run_time * 1_000_000:.2f} μs)")
print(f" Min Time: {result.min_run_time:.6f} seconds")
print(f" Max Time: {result.max_run_time:.6f} seconds")
print(f" Std Dev: {result.std_dev_run_time:.6f} seconds")
print(f" Median Time: {result.median_run_time:.6f} seconds")
print(f" Avg Memory: {result.avg_memory_usage:.6f} MiB ({result.avg_memory_usage * 1024:.2f} KiB)")
return result
# ============================================================================
# Framework Imports
# ============================================================================
from praisonaiagents import Agent as PraisonAgent
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
# Additional frameworks for comparison
try:
from pydantic_ai import Agent as PydanticAgent
PYDANTIC_AVAILABLE = True
except ImportError:
PYDANTIC_AVAILABLE = False
try:
from crewai.agent import Agent as CrewAgent
from crewai.tools import tool as crewai_tool
CREWAI_AVAILABLE = True
except ImportError:
CREWAI_AVAILABLE = False
try:
from langchain_core.tools import tool as langchain_tool
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
LANGGRAPH_AVAILABLE = True
except ImportError:
LANGGRAPH_AVAILABLE = False
try:
from agents import Agent as OpenAIAgent, function_tool
OPENAI_AGENTS_AVAILABLE = True
except ImportError:
OPENAI_AGENTS_AVAILABLE = False
def get_weather(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
tools = [get_weather]
def instantiate_praisonai_agent():
"""Instantiate a PraisonAI agent with one tool."""
return PraisonAgent(
name="Test Agent",
instructions="Be concise, reply with one sentence.",
llm="gpt-4o-mini",
tools=tools,
verbose=False
)
def instantiate_praisonai_litellm_agent():
"""Instantiate a PraisonAI agent with LiteLLM backend."""
return PraisonAgent(
name="Test Agent",
instructions="Be concise, reply with one sentence.",
llm="openai/gpt-4o-mini",
tools=tools,
verbose=False
)
def instantiate_agno_agent():
"""Instantiate an Agno agent with one tool."""
return AgnoAgent(model=OpenAIChat(id="gpt-4o-mini"), tools=tools)
def instantiate_pydantic_agent():
"""Instantiate a PydanticAI agent with one tool."""
agent = PydanticAgent("openai:gpt-4o-mini", system_prompt="Be concise, reply with one sentence.")
@agent.tool_plain
def get_weather_pydantic(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
return agent
def instantiate_openai_agents():
"""Instantiate an OpenAI Agents SDK agent with one tool."""
return OpenAIAgent(
name="Test agent",
instructions="Be concise, reply with one sentence.",
model="gpt-4o-mini",
tools=[function_tool(get_weather)],
)
def instantiate_langgraph_agent():
"""Instantiate a LangGraph agent with one tool."""
@langchain_tool
def get_weather_lg(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
return create_react_agent(model=ChatOpenAI(model="gpt-4o-mini"), tools=[get_weather_lg])
def instantiate_crewai_agent():
"""Instantiate a CrewAI agent with one tool."""
@crewai_tool("Weather Tool")
def get_weather_crew(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
return CrewAgent(
llm="gpt-4o-mini",
role="Test Agent",
goal="Be concise, reply with one sentence.",
tools=[get_weather_crew],
backstory="Test",
)
def get_package_versions():
"""Get version numbers for all benchmarked packages."""
from importlib.metadata import version as get_version
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
'PydanticAI': 'pydantic-ai',
'OpenAI Agents SDK': 'openai-agents',
'LangGraph': 'langgraph',
'CrewAI': 'crewai'
}
versions = {}
for display_name, pkg_name in packages.items():
try:
versions[display_name] = get_version(pkg_name)
except Exception:
versions[display_name] = 'not installed'
return versions
def save_benchmark_results(results: dict, baseline):
"""Save benchmark results to BENCHMARK_RESULTS.md and update README.md"""
import os
import re
from datetime import datetime
benchmarks_dir = os.path.dirname(__file__)
filepath = os.path.join(benchmarks_dir, 'BENCHMARK_RESULTS.md')
# Get package versions
versions = get_package_versions()
display_names = {
'agno': 'Agno',
'praisonai': 'PraisonAI',
'praisonai_litellm': 'PraisonAI (LiteLLM)',
'pydantic': 'PydanticAI',
'openai_agents': 'OpenAI Agents SDK',
'langgraph': 'LangGraph',
'crewai': 'CrewAI'
}
# Build table rows
table_rows = []
sorted_results = sorted(results.items(), key=lambda x: x[1].avg_run_time)
for name, result in sorted_results:
display_name = display_names.get(name, name)
avg_us = result.avg_run_time * 1e6
ratio = result.avg_run_time / baseline.avg_run_time if baseline.avg_run_time > 0 else 0
if name == 'praisonai':
table_rows.append(f'| **{display_name}** | **{avg_us:.2f}** | **1.00x (fastest)** |')
else:
avg_str = f'{avg_us:,.2f}' if avg_us >= 1000 else f'{avg_us:.2f}'
ratio_str = f'{ratio:,.0f}x' if ratio >= 100 else f'{ratio:.2f}x'
table_rows.append(f'| {display_name} | {avg_str} | {ratio_str} |')
# Save to BENCHMARK_RESULTS.md
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Benchmark Results\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write('**Methodology:** 10 warmup runs + 1000 measured iterations\n\n')
f.write('## Agent Instantiation Time\n\n')
f.write('| Framework | Avg Time (μs) | Relative |\n')
f.write('|-----------|---------------|----------|\n')
f.write('\n'.join(table_rows) + '\n')
f.write('\n## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/performance_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
# Update README.md
readme_path = os.path.join(benchmarks_dir, '..', '..', '..', 'README.md')
if os.path.exists(readme_path):
with open(readme_path, 'r') as f:
content = f.read()
new_table = '''| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
''' + '\n'.join(table_rows)
pattern = r'(\| Framework \| Avg Time \(μs\) \| Relative \|\n\|[-|]+\|\n)(\|[^\n]+\|\n)+'
if re.search(pattern, content):
content = re.sub(pattern, new_table + '\n', content)
with open(readme_path, 'w') as f:
f.write(content)
print(f'README.md updated: {readme_path}')
if __name__ == "__main__":
print("="*70)
print("PraisonAI Agents - Comprehensive Performance Benchmark")
print("="*70)
print("\nThis benchmark measures:")
print(" 1. Agent instantiation time (how fast an agent object is created)")
print(" 2. Memory footprint (how much memory an agent uses)")
print("\nMethodology:")
print(" - 10 warmup runs (not measured)")
print(" - 1000 measured iterations")
print(" - tracemalloc for memory measurement")
print(" - Baseline subtraction for accuracy")
print("\nFrameworks available:")
print(" - PraisonAI: ✅")
print(f" - Agno: ✅")
print(f" - PydanticAI: {'✅' if PYDANTIC_AVAILABLE else '❌'}")
print(f" - OpenAI Agents SDK: {'✅' if OPENAI_AGENTS_AVAILABLE else '❌'}")
print(f" - LangGraph: {'✅' if LANGGRAPH_AVAILABLE else '❌'}")
print(f" - CrewAI: {'✅' if CREWAI_AVAILABLE else '❌'}")
results = {}
# Test PraisonAI first
try:
print("\n" + "="*70)
print("Testing PraisonAI Agents")
print("="*70)
results['praisonai'] = run_benchmark(
instantiate_praisonai_agent,
"PraisonAI",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing PraisonAI: {e}")
try:
print("\n" + "="*70)
print("Testing PraisonAI Agents (LiteLLM)")
print("="*70)
results['praisonai_litellm'] = run_benchmark(
instantiate_praisonai_litellm_agent,
"PraisonAI (LiteLLM)",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing PraisonAI (LiteLLM): {e}")
# Test Agno
try:
print("\n" + "="*70)
print("Testing Agno")
print("="*70)
results['agno'] = run_benchmark(
instantiate_agno_agent,
"Agno",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing Agno: {e}")
# Test PydanticAI
if PYDANTIC_AVAILABLE:
try:
print("\n" + "="*70)
print("Testing PydanticAI")
print("="*70)
results['pydantic'] = run_benchmark(
instantiate_pydantic_agent,
"PydanticAI Agent Instantiation",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing PydanticAI: {e}")
# Test OpenAI Agents SDK
if OPENAI_AGENTS_AVAILABLE:
try:
print("\n" + "="*70)
print("Testing OpenAI Agents SDK")
print("="*70)
results['openai_agents'] = run_benchmark(
instantiate_openai_agents,
"OpenAI Agents SDK Instantiation",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing OpenAI Agents SDK: {e}")
# Test LangGraph
if LANGGRAPH_AVAILABLE:
try:
print("\n" + "="*70)
print("Testing LangGraph")
print("="*70)
results['langgraph'] = run_benchmark(
instantiate_langgraph_agent,
"LangGraph Agent Instantiation",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing LangGraph: {e}")
# Test CrewAI
if CREWAI_AVAILABLE:
try:
print("\n" + "="*70)
print("Testing CrewAI")
print("="*70)
results['crewai'] = run_benchmark(
instantiate_crewai_agent,
"CrewAI Agent Instantiation",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing CrewAI: {e}")
# Final Comparison Table
print("\n" + "="*70)
print("RESULTS SUMMARY")
print("="*70)
if 'praisonai' in results:
baseline = results['praisonai']
print(f"\n{'Framework':<25} {'Avg Time (μs)':<15} {'Avg Memory (KiB)':<18} {'Relative':<15}")
print("-" * 73)
# Sort by time
sorted_results = sorted(results.items(), key=lambda x: x[1].avg_run_time)
for name, result in sorted_results:
time_ratio = result.avg_run_time / baseline.avg_run_time if baseline.avg_run_time > 0 else 0
time_str = f"{time_ratio:.2f}x"
display_name = {
'agno': 'Agno',
'praisonai': 'PraisonAI',
'praisonai_litellm': 'PraisonAI (LiteLLM)',
'pydantic': 'PydanticAI',
'openai_agents': 'OpenAI Agents SDK',
'langgraph': 'LangGraph',
'crewai': 'CrewAI'
}.get(name, name)
print(f"{display_name:<25} {result.avg_run_time*1e6:<15.2f} {result.avg_memory_usage*1024:<18.2f} {time_str:<15}")
print("\n" + "="*70)
# Save results to files
save_benchmark_results(results, baseline)
@@ -0,0 +1,481 @@
#!/usr/bin/env python3
"""
PraisonAI Agents - Real-World Execution Benchmark
Benchmarks actual agent execution with LLM calls across frameworks.
Tests single agent, 2-agent, and 3-agent workflows.
Usage:
python benchmarks/real_benchmark.py
"""
import time
import statistics
import os
from typing import Literal, List
from dataclasses import dataclass, field
from importlib.metadata import version as get_version
# Ensure API key is set
if not os.environ.get('OPENAI_API_KEY'):
print("ERROR: OPENAI_API_KEY environment variable not set")
exit(1)
@dataclass
class ExecutionResult:
"""Execution benchmark results"""
times: List[float] = field(default_factory=list)
@property
def avg_time(self) -> float:
return statistics.mean(self.times) if self.times else 0
@property
def min_time(self) -> float:
return min(self.times) if self.times else 0
@property
def max_time(self) -> float:
return max(self.times) if self.times else 0
@property
def std_dev(self) -> float:
return statistics.stdev(self.times) if len(self.times) > 1 else 0
def get_package_versions():
"""Get version numbers for benchmarked packages."""
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
}
versions = {}
for display_name, pkg_name in packages.items():
try:
versions[display_name] = get_version(pkg_name)
except Exception:
versions[display_name] = 'not installed'
return versions
# ============================================================================
# Tool Definitions
# ============================================================================
def get_weather(city: Literal["nyc", "sf"]) -> str:
"""Get weather information for a city."""
if city == "nyc":
return "New York: Cloudy, 45°F, 60% humidity"
return "San Francisco: Sunny, 72°F, 40% humidity"
def get_time(timezone: Literal["est", "pst"]) -> str:
"""Get current time in a timezone."""
if timezone == "est":
return "Eastern Time: 2:30 PM"
return "Pacific Time: 11:30 AM"
def calculate(expression: str) -> str:
"""Calculate a math expression."""
try:
result = eval(expression)
return f"Result: {result}"
except Exception:
return "Error: Invalid expression"
# ============================================================================
# PraisonAI Benchmarks
# ============================================================================
def benchmark_praisonai_single(iterations: int = 3) -> ExecutionResult:
"""Benchmark single PraisonAI agent execution."""
from praisonaiagents import Agent
result = ExecutionResult()
for i in range(iterations):
print(f" PraisonAI single agent iteration {i+1}/{iterations}...")
agent = Agent(
name="Assistant",
instructions="You are a helpful assistant. Be very brief.",
llm="gpt-4o-mini",
tools=[get_weather],
verbose=False
)
start = time.perf_counter()
response = agent.start("What's the weather in NYC? Reply in one sentence.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
def benchmark_praisonai_two_agents(iterations: int = 3) -> ExecutionResult:
"""Benchmark 2 PraisonAI agents working together."""
from praisonaiagents import Agent, PraisonAIAgents
result = ExecutionResult()
for i in range(iterations):
print(f" PraisonAI 2-agent iteration {i+1}/{iterations}...")
researcher = Agent(
name="Researcher",
role="Research Assistant",
goal="Gather weather information",
instructions="Get weather data and pass to reporter. Be brief.",
llm="gpt-4o-mini",
tools=[get_weather],
verbose=False
)
reporter = Agent(
name="Reporter",
role="News Reporter",
goal="Create weather report",
instructions="Create a one-sentence weather summary.",
llm="gpt-4o-mini",
verbose=False
)
agents = PraisonAIAgents(
agents=[researcher, reporter],
process="sequential",
verbose=False
)
start = time.perf_counter()
response = agents.start("Get NYC weather and create a brief report.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
def benchmark_praisonai_three_agents(iterations: int = 3) -> ExecutionResult:
"""Benchmark 3 PraisonAI agents working together."""
from praisonaiagents import Agent, PraisonAIAgents
result = ExecutionResult()
for i in range(iterations):
print(f" PraisonAI 3-agent iteration {i+1}/{iterations}...")
weather_agent = Agent(
name="WeatherAgent",
role="Weather Specialist",
goal="Get weather data",
instructions="Fetch weather information. Be brief.",
llm="gpt-4o-mini",
tools=[get_weather],
verbose=False
)
time_agent = Agent(
name="TimeAgent",
role="Time Specialist",
goal="Get time information",
instructions="Fetch timezone information. Be brief.",
llm="gpt-4o-mini",
tools=[get_time],
verbose=False
)
summarizer = Agent(
name="Summarizer",
role="Summary Writer",
goal="Combine information",
instructions="Create a one-sentence summary combining weather and time.",
llm="gpt-4o-mini",
verbose=False
)
agents = PraisonAIAgents(
agents=[weather_agent, time_agent, summarizer],
process="sequential",
verbose=False
)
start = time.perf_counter()
response = agents.start("Get NYC weather and EST time, then summarize.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
# ============================================================================
# Agno Benchmarks
# ============================================================================
def benchmark_agno_single(iterations: int = 3) -> ExecutionResult:
"""Benchmark single Agno agent execution."""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
result = ExecutionResult()
for i in range(iterations):
print(f" Agno single agent iteration {i+1}/{iterations}...")
agent = Agent(
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["You are a helpful assistant. Be very brief."],
tools=[get_weather],
)
start = time.perf_counter()
response = agent.run("What's the weather in NYC? Reply in one sentence.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
def benchmark_agno_two_agents(iterations: int = 3) -> ExecutionResult:
"""Benchmark 2 Agno agents working together (using Team)."""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.team import Team
result = ExecutionResult()
for i in range(iterations):
print(f" Agno 2-agent iteration {i+1}/{iterations}...")
researcher = Agent(
name="Researcher",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Get weather data. Be brief."],
tools=[get_weather],
)
reporter = Agent(
name="Reporter",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Create a one-sentence weather summary."],
)
team = Team(
members=[researcher, reporter],
model=OpenAIChat(id="gpt-4o-mini"),
)
start = time.perf_counter()
response = team.run("Get NYC weather and create a brief report.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
def benchmark_agno_three_agents(iterations: int = 3) -> ExecutionResult:
"""Benchmark 3 Agno agents working together."""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.team import Team
result = ExecutionResult()
for i in range(iterations):
print(f" Agno 3-agent iteration {i+1}/{iterations}...")
weather_agent = Agent(
name="WeatherAgent",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Fetch weather information. Be brief."],
tools=[get_weather],
)
time_agent = Agent(
name="TimeAgent",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Fetch timezone information. Be brief."],
tools=[get_time],
)
summarizer = Agent(
name="Summarizer",
model=OpenAIChat(id="gpt-4o-mini"),
instructions=["Create a one-sentence summary combining weather and time."],
)
team = Team(
members=[weather_agent, time_agent, summarizer],
model=OpenAIChat(id="gpt-4o-mini"),
)
start = time.perf_counter()
response = team.run("Get NYC weather and EST time, then summarize.")
elapsed = time.perf_counter() - start
result.times.append(elapsed)
print(f" Time: {elapsed:.2f}s")
return result
# ============================================================================
# Save Results
# ============================================================================
def save_results(results: dict, versions: dict):
"""Save benchmark results to file."""
import os
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), 'REAL_BENCHMARK_RESULTS.md')
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Real-World Execution Benchmark\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write('**Model:** gpt-4o-mini\n')
f.write('**Iterations:** 3 per test\n\n')
f.write('## Results Summary\n\n')
f.write('| Test | PraisonAI (avg) | Agno (avg) | Difference |\n')
f.write('|------|-----------------|------------|------------|\n')
tests = ['single', 'two_agents', 'three_agents']
test_names = ['Single Agent', '2 Agents', '3 Agents']
for test, name in zip(tests, test_names):
praison_key = f'praisonai_{test}'
agno_key = f'agno_{test}'
if praison_key in results and agno_key in results:
praison_avg = results[praison_key].avg_time
agno_avg = results[agno_key].avg_time
if praison_avg < agno_avg:
diff = f'PraisonAI {agno_avg/praison_avg:.2f}x faster'
else:
diff = f'Agno {praison_avg/agno_avg:.2f}x faster'
f.write(f'| {name} | {praison_avg:.2f}s | {agno_avg:.2f}s | {diff} |\n')
f.write('\n## Detailed Results\n\n')
for key, result in results.items():
f.write(f'### {key}\n')
f.write(f'- Average: {result.avg_time:.2f}s\n')
f.write(f'- Min: {result.min_time:.2f}s\n')
f.write(f'- Max: {result.max_time:.2f}s\n')
f.write(f'- Std Dev: {result.std_dev:.2f}s\n')
f.write(f'- Times: {[f"{t:.2f}s" for t in result.times]}\n\n')
f.write('## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('export OPENAI_API_KEY=your_key\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/real_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
# ============================================================================
# Main
# ============================================================================
if __name__ == "__main__":
print("="*70)
print("PraisonAI Agents - Real-World Execution Benchmark")
print("="*70)
print("\nThis benchmark measures actual agent execution time including:")
print(" - LLM API calls")
print(" - Tool execution")
print(" - Multi-agent coordination")
print("\nModel: gpt-4o-mini")
print("Iterations: 3 per test")
versions = get_package_versions()
print(f"\nPackage versions:")
for pkg, ver in versions.items():
print(f" {pkg}: {ver}")
results = {}
# Single Agent Tests
print("\n" + "="*70)
print("SINGLE AGENT BENCHMARK")
print("="*70)
print("\nTesting PraisonAI single agent...")
results['praisonai_single'] = benchmark_praisonai_single()
print("\nTesting Agno single agent...")
results['agno_single'] = benchmark_agno_single()
# 2-Agent Tests
print("\n" + "="*70)
print("2-AGENT BENCHMARK")
print("="*70)
print("\nTesting PraisonAI 2 agents...")
results['praisonai_two_agents'] = benchmark_praisonai_two_agents()
print("\nTesting Agno 2 agents (Team)...")
results['agno_two_agents'] = benchmark_agno_two_agents()
# 3-Agent Tests
print("\n" + "="*70)
print("3-AGENT BENCHMARK")
print("="*70)
print("\nTesting PraisonAI 3 agents...")
results['praisonai_three_agents'] = benchmark_praisonai_three_agents()
print("\nTesting Agno 3 agents (Team)...")
results['agno_three_agents'] = benchmark_agno_three_agents()
# Summary
print("\n" + "="*70)
print("RESULTS SUMMARY")
print("="*70)
print(f"\n{'Test':<20} {'PraisonAI (avg)':<18} {'Agno (avg)':<15} {'Difference':<20}")
print("-" * 73)
tests = [
('Single Agent', 'praisonai_single', 'agno_single'),
('2 Agents', 'praisonai_two_agents', 'agno_two_agents'),
('3 Agents', 'praisonai_three_agents', 'agno_three_agents'),
]
for name, praison_key, agno_key in tests:
praison_avg = results[praison_key].avg_time
agno_avg = results[agno_key].avg_time
if praison_avg < agno_avg:
diff = f'PraisonAI {agno_avg/praison_avg:.2f}x faster'
else:
diff = f'Agno {praison_avg/agno_avg:.2f}x faster'
print(f"{name:<20} {praison_avg:<18.2f}s {agno_avg:<15.2f}s {diff:<20}")
print("\n" + "="*70)
# Save results
save_results(results, versions)
@@ -0,0 +1,235 @@
"""
PraisonAI Agents - Response Time Benchmark
Measures actual LLM response time with gpt-4o-mini across frameworks.
Usage:
python benchmarks/response_benchmark.py
"""
import time
import statistics
import os
from typing import List, Literal
from dataclasses import dataclass, field
# Ensure we use gpt-4o-mini
os.environ['OPENAI_MODEL_NAME'] = 'gpt-4o-mini'
@dataclass
class ResponseResult:
"""Response time measurement results"""
response_times: List[float] = field(default_factory=list)
@property
def avg_time(self) -> float:
return statistics.mean(self.response_times) if self.response_times else 0
@property
def min_time(self) -> float:
return min(self.response_times) if self.response_times else 0
@property
def max_time(self) -> float:
return max(self.response_times) if self.response_times else 0
@property
def std_dev(self) -> float:
return statistics.stdev(self.response_times) if len(self.response_times) > 1 else 0
def get_weather(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
def benchmark_praisonai_response(iterations: int = 5) -> ResponseResult:
"""Benchmark PraisonAI agent response time"""
from praisonaiagents import Agent
result = ResponseResult()
for i in range(iterations):
print(f" PraisonAI iteration {i+1}/{iterations}...")
# Create fresh agent each time
agent = Agent(
name="Test Agent",
instructions="Be concise, reply with one sentence.",
llm="gpt-4o-mini",
verbose=False
)
start = time.perf_counter()
response = agent.start("What is the capital of France?")
end = time.perf_counter()
result.response_times.append(end - start)
print(f" Response time: {end - start:.3f}s")
return result
def benchmark_agno_response(iterations: int = 5) -> ResponseResult:
"""Benchmark Agno agent response time"""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
result = ResponseResult()
for i in range(iterations):
print(f" Agno iteration {i+1}/{iterations}...")
# Create fresh agent each time
agent = Agent(
model=OpenAIChat(id="gpt-4o-mini"),
system_message="Be concise, reply with one sentence.",
)
start = time.perf_counter()
response = agent.run("What is the capital of France?")
end = time.perf_counter()
result.response_times.append(end - start)
print(f" Response time: {end - start:.3f}s")
return result
def benchmark_praisonai_with_tool(iterations: int = 5) -> ResponseResult:
"""Benchmark PraisonAI agent response time with tool calling"""
from praisonaiagents import Agent
result = ResponseResult()
for i in range(iterations):
print(f" PraisonAI (with tool) iteration {i+1}/{iterations}...")
agent = Agent(
name="Weather Agent",
instructions="You are a weather assistant. Use the get_weather tool to answer questions.",
llm="gpt-4o-mini",
tools=[get_weather],
verbose=False
)
start = time.perf_counter()
response = agent.start("What's the weather in NYC?")
end = time.perf_counter()
result.response_times.append(end - start)
print(f" Response time: {end - start:.3f}s")
return result
def benchmark_agno_with_tool(iterations: int = 5) -> ResponseResult:
"""Benchmark Agno agent response time with tool calling"""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
result = ResponseResult()
for i in range(iterations):
print(f" Agno (with tool) iteration {i+1}/{iterations}...")
agent = Agent(
model=OpenAIChat(id="gpt-4o-mini"),
system_message="You are a weather assistant. Use the get_weather tool to answer questions.",
tools=[get_weather],
)
start = time.perf_counter()
response = agent.run("What's the weather in NYC?")
end = time.perf_counter()
result.response_times.append(end - start)
print(f" Response time: {end - start:.3f}s")
return result
if __name__ == "__main__":
print("="*70)
print("PraisonAI Agents - Response Time Benchmark")
print("="*70)
print("\nModel: gpt-4o-mini")
print("Measures end-to-end response time including:")
print(" - Agent instantiation")
print(" - API call to OpenAI")
print(" - Response processing")
iterations = 5
results = {}
# Test 1: Simple response (no tools)
print("\n" + "="*70)
print("TEST 1: Simple Response (No Tools)")
print("="*70)
print("\nBenchmarking PraisonAI...")
try:
results['praisonai_simple'] = benchmark_praisonai_response(iterations)
except Exception as e:
print(f"Error: {e}")
import traceback
traceback.print_exc()
print("\nBenchmarking Agno...")
try:
results['agno_simple'] = benchmark_agno_response(iterations)
except Exception as e:
print(f"Error: {e}")
import traceback
traceback.print_exc()
# Test 2: Response with tool calling
print("\n" + "="*70)
print("TEST 2: Response with Tool Calling")
print("="*70)
print("\nBenchmarking PraisonAI with tool...")
try:
results['praisonai_tool'] = benchmark_praisonai_with_tool(iterations)
except Exception as e:
print(f"Error: {e}")
import traceback
traceback.print_exc()
print("\nBenchmarking Agno with tool...")
try:
results['agno_tool'] = benchmark_agno_with_tool(iterations)
except Exception as e:
print(f"Error: {e}")
import traceback
traceback.print_exc()
# Summary
print("\n" + "="*70)
print("SUMMARY RESULTS")
print("="*70)
print(f"\n{'Test':<30} {'PraisonAI (avg)':<20} {'Agno (avg)':<20} {'Ratio':<15}")
print("-" * 85)
if 'praisonai_simple' in results and 'agno_simple' in results:
p_time = results['praisonai_simple'].avg_time
a_time = results['agno_simple'].avg_time
ratio = p_time / a_time if a_time > 0 else float('inf')
print(f"{'Simple Response':<30} {p_time:<20.3f}s {a_time:<20.3f}s {ratio:.2f}x")
if 'praisonai_tool' in results and 'agno_tool' in results:
p_time = results['praisonai_tool'].avg_time
a_time = results['agno_tool'].avg_time
ratio = p_time / a_time if a_time > 0 else float('inf')
print(f"{'With Tool Calling':<30} {p_time:<20.3f}s {a_time:<20.3f}s {ratio:.2f}x")
print("\n" + "="*70)
print("NOTE")
print("="*70)
print("\nResponse time is dominated by OpenAI API latency.")
print("Agent instantiation overhead is negligible in real-world usage.")
@@ -0,0 +1,194 @@
❯ python benchmarks/performance_benchmark.py
/Users/praison/praisonai-package/src/praisonai-agents/praisonaiagents/_warning_patch.py:47: UserWarning: Mixing V1 models and V2 models (or constructs, like `TypeAdapter`) is not supported. Please upgrade `Settings` to V2.
_original_warn(message, category, stacklevel, source)
[13:04:15] INFO [13:04:15] config.py:50
config.py:50 INFO
Using config path:
/Users/praison/.con
fig/crewai/settings
.json
======================================================================
PraisonAI Agents - Comprehensive Performance Benchmark
======================================================================
This benchmark measures:
1. Agent instantiation time (how fast an agent object is created)
2. Memory footprint (how much memory an agent uses)
Methodology:
- 10 warmup runs (not measured)
- 1000 measured iterations
- tracemalloc for memory measurement
- Baseline subtraction for accuracy
Frameworks available:
- PraisonAI: ✅
- Agno: ✅
- PydanticAI: ✅
- OpenAI Agents SDK: ✅
- LangGraph: ✅
- CrewAI: ✅
======================================================================
Testing PraisonAI Agents
======================================================================
============================================================
Benchmarking: PraisonAI
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for PraisonAI:
Average Time: 0.000003 seconds (3.20 μs)
Min Time: 0.000003 seconds
Max Time: 0.000025 seconds
Std Dev: 0.000001 seconds
Median Time: 0.000003 seconds
Avg Memory: 0.003678 MiB (3.77 KiB)
======================================================================
Testing PraisonAI Agents (LiteLLM)
======================================================================
============================================================
Benchmarking: PraisonAI (LiteLLM)
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for PraisonAI (LiteLLM):
Average Time: 0.000007 seconds (7.31 μs)
Min Time: 0.000007 seconds
Max Time: 0.000060 seconds
Std Dev: 0.000002 seconds
Median Time: 0.000007 seconds
Avg Memory: 0.006419 MiB (6.57 KiB)
======================================================================
Testing Agno
======================================================================
============================================================
Benchmarking: Agno
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for Agno:
Average Time: 0.000005 seconds (5.30 μs)
Min Time: 0.000005 seconds
Max Time: 0.000039 seconds
Std Dev: 0.000002 seconds
Median Time: 0.000005 seconds
Avg Memory: 0.006599 MiB (6.76 KiB)
======================================================================
Testing PydanticAI
======================================================================
============================================================
Benchmarking: PydanticAI Agent Instantiation
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for PydanticAI Agent Instantiation:
Average Time: 0.000247 seconds (247.34 μs)
Min Time: 0.000218 seconds
Max Time: 0.000552 seconds
Std Dev: 0.000031 seconds
Median Time: 0.000237 seconds
Avg Memory: 0.028486 MiB (29.17 KiB)
======================================================================
Testing OpenAI Agents SDK
======================================================================
============================================================
Benchmarking: OpenAI Agents SDK Instantiation
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for OpenAI Agents SDK Instantiation:
Average Time: 0.000282 seconds (282.25 μs)
Min Time: 0.000247 seconds
Max Time: 0.001110 seconds
Std Dev: 0.000055 seconds
Median Time: 0.000263 seconds
Avg Memory: 0.029032 MiB (29.73 KiB)
======================================================================
Testing LangGraph
======================================================================
============================================================
Benchmarking: LangGraph Agent Instantiation
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for LangGraph Agent Instantiation:
Average Time: 0.003200 seconds (3200.15 μs)
Min Time: 0.002808 seconds
Max Time: 0.116363 seconds
Std Dev: 0.003628 seconds
Median Time: 0.002972 seconds
Avg Memory: 0.207509 MiB (212.49 KiB)
======================================================================
Testing CrewAI
======================================================================
============================================================
Benchmarking: CrewAI Agent Instantiation
============================================================
Warmup runs: 10
Measured iterations: 1000
Measuring runtime...
Measuring memory...
Results for CrewAI Agent Instantiation:
Average Time: 0.016772 seconds (16771.69 μs)
Min Time: 0.014404 seconds
Max Time: 0.091841 seconds
Std Dev: 0.003445 seconds
Median Time: 0.015816 seconds
Avg Memory: 0.065050 MiB (66.61 KiB)
======================================================================
RESULTS SUMMARY
======================================================================
Framework Avg Time (μs) Avg Memory (KiB) Relative
-------------------------------------------------------------------------
PraisonAI 3.20 3.77 1.00x
Agno 5.30 6.76 1.66x
PraisonAI (LiteLLM) 7.31 6.57 2.28x
PydanticAI 247.34 29.17 77.25x
OpenAI Agents SDK 282.25 29.73 88.15x
LangGraph 3200.15 212.49 999.46x
CrewAI 16771.69 66.61 5238.06x
======================================================================
❯
@@ -0,0 +1,243 @@
#!/usr/bin/env python3
"""
PraisonAI Agents - Performance Benchmark
Compares agent instantiation times across popular AI agent frameworks.
Usage:
python benchmarks/simple_benchmark.py
"""
import time
from typing import Literal
ITERATIONS = 100
def sample_tool(city: Literal['nyc', 'sf']):
"""Sample tool for benchmark testing."""
if city == 'nyc':
return 'cloudy'
return 'sunny'
def measure_instantiation(create_fn, iterations=ITERATIONS):
"""Measure average instantiation time in microseconds."""
times = []
for _ in range(iterations):
start = time.perf_counter()
create_fn()
times.append((time.perf_counter() - start) * 1_000_000)
return sum(times) / len(times)
def run_benchmark():
"""Run the benchmark across all available frameworks (without tools)."""
results = {}
print('=' * 60)
print('PraisonAI Agents - Performance Benchmark')
print('=' * 60)
print(f'\nIterations: {ITERATIONS}')
print('Metric: Agent instantiation time (microseconds)\n')
# PraisonAI (without tools)
print("Testing PraisonAI...")
from praisonaiagents import Agent as PraisonAgent
results['PraisonAI'] = measure_instantiation(
lambda: PraisonAgent(name='Test', llm='gpt-4o-mini', verbose=False)
)
results['PraisonAI (LiteLLM)'] = measure_instantiation(
lambda: PraisonAgent(name='Test', llm='openai/gpt-4o-mini', verbose=False)
)
# Other frameworks for comparison
print("Testing other frameworks...")
try:
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
results['Agno'] = measure_instantiation(
lambda: AgnoAgent(model=OpenAIChat(id='gpt-4o-mini'))
)
except ImportError:
pass
try:
from pydantic_ai import Agent as PydanticAgent
results['PydanticAI'] = measure_instantiation(
lambda: PydanticAgent('openai:gpt-4o-mini')
)
except ImportError:
pass
try:
from agents import Agent as OpenAIAgent
results['OpenAI Agents SDK'] = measure_instantiation(
lambda: OpenAIAgent(name='Test', model='gpt-4o-mini')
)
except ImportError:
pass
try:
from langchain_core.tools import tool as langchain_tool
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
@langchain_tool
def get_weather_lg(city: Literal['nyc', 'sf']):
"""Get weather info."""
return 'sunny' if city == 'sf' else 'cloudy'
results['LangGraph'] = measure_instantiation(
lambda: create_react_agent(model=ChatOpenAI(model='gpt-4o-mini'), tools=[get_weather_lg])
)
except ImportError:
pass
try:
from crewai.agent import Agent as CrewAgent
from crewai.tools import tool as crewai_tool
@crewai_tool("Weather Tool")
def get_weather_crew(city: Literal['nyc', 'sf']):
"""Get weather info."""
return 'sunny' if city == 'sf' else 'cloudy'
results['CrewAI'] = measure_instantiation(
lambda: CrewAgent(
role='Weather Agent',
goal='Provide weather info',
backstory='A weather expert',
tools=[get_weather_crew],
verbose=False
)
)
except ImportError:
pass
# Print results
print('\n' + '=' * 60)
print('RESULTS')
print('=' * 60)
baseline = results.get('PraisonAI', 1)
print(f"\n{'Framework':<25} {'Avg Time (μs)':<15} {'Relative':<10}")
print('-' * 50)
for name, avg in sorted(results.items(), key=lambda x: x[1]):
ratio = avg / baseline
print(f'{name:<25} {avg:<15.2f} {ratio:.2f}x')
print('\n' + '=' * 60)
return results
def get_package_versions():
"""Get version numbers for benchmarked packages."""
from importlib.metadata import version as get_version
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
'PydanticAI': 'pydantic-ai',
'OpenAI Agents SDK': 'openai-agents',
'LangGraph': 'langgraph',
'CrewAI': 'crewai'
}
versions = {}
for display_name, pkg_name in packages.items():
try:
versions[display_name] = get_version(pkg_name)
except Exception:
versions[display_name] = 'not installed'
return versions
def save_results(results: dict, filename: str = 'BENCHMARK_RESULTS.md'):
"""Save benchmark results to a markdown file and update README."""
import os
import re
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), filename)
versions = get_package_versions()
# Find the fastest result (baseline)
sorted_results = sorted(results.items(), key=lambda x: x[1])
fastest_time = sorted_results[0][1] if sorted_results else 1
# Build the table rows
table_rows = []
for name, avg in sorted_results:
ratio = avg / fastest_time
if ratio == 1.0:
table_rows.append(f'| **{name}** | **{avg:.2f}** | **1.00x (fastest)** |')
else:
avg_str = f'{avg:,.2f}' if avg >= 1000 else f'{avg:.2f}'
ratio_str = f'{ratio:,.0f}x' if ratio >= 100 else f'{ratio:.2f}x'
table_rows.append(f'| {name} | {avg_str} | {ratio_str} |')
# Save to BENCHMARK_RESULTS.md
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Benchmark Results\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write(f'**Iterations:** {ITERATIONS}\n')
f.write('**Test:** Agent instantiation (without tools)\n\n')
f.write('## Results\n\n')
f.write('| Framework | Avg Time (μs) | Relative |\n')
f.write('|-----------|---------------|----------|\n')
f.write('\n'.join(table_rows) + '\n')
f.write('\n## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/simple_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
# Also update the main README.md
readme_path = os.path.join(os.path.dirname(__file__), '..', '..', '..', 'README.md')
if os.path.exists(readme_path):
update_readme(readme_path, table_rows)
return filepath
def update_readme(readme_path: str, table_rows: list):
"""Update the performance section in README.md with latest results."""
import re
with open(readme_path, 'r') as f:
content = f.read()
# Build the new table
new_table = '''| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
''' + '\n'.join(table_rows)
# Pattern to match the performance table
pattern = r'(\| Framework \| Avg Time \(μs\) \| Relative \|\n\|[-|]+\|\n)(\|[^\n]+\|\n)+'
if re.search(pattern, content):
content = re.sub(pattern, new_table + '\n', content)
with open(readme_path, 'w') as f:
f.write(content)
print(f'README.md updated: {readme_path}')
if __name__ == '__main__':
results = run_benchmark()
save_results(results)
@@ -0,0 +1,213 @@
#!/usr/bin/env python3
"""
PraisonAI Agents - Tools Benchmark
Compares agent instantiation times WITH TOOLS across frameworks.
Usage:
python benchmarks/tools_benchmark.py
"""
import time
from typing import Literal
from importlib.metadata import version as get_version
ITERATIONS = 100
def sample_tool(city: Literal['nyc', 'sf']):
"""Sample tool for benchmark testing."""
if city == 'nyc':
return 'cloudy'
return 'sunny'
def measure_instantiation(create_fn, iterations=ITERATIONS):
"""Measure average instantiation time in microseconds."""
times = []
# Warmup
for _ in range(10):
create_fn()
# Measure
for _ in range(iterations):
start = time.perf_counter()
create_fn()
elapsed = time.perf_counter() - start
times.append(elapsed * 1_000_000) # Convert to microseconds
return sum(times) / len(times)
def run_benchmark():
"""Run the benchmark across all available frameworks WITH TOOLS."""
tools = [sample_tool]
results = {}
print('=' * 60)
print('PraisonAI Agents - Tools Benchmark')
print('=' * 60)
print(f'\nIterations: {ITERATIONS}')
print('Metric: Agent instantiation time WITH TOOLS (microseconds)\n')
# PraisonAI with tools
print("Testing PraisonAI...")
from praisonaiagents import Agent as PraisonAgent
results['PraisonAI'] = measure_instantiation(
lambda: PraisonAgent(name='Test', llm='gpt-4o-mini', tools=tools, verbose=False)
)
results['PraisonAI (LiteLLM)'] = measure_instantiation(
lambda: PraisonAgent(name='Test', llm='openai/gpt-4o-mini', tools=tools, verbose=False)
)
# Other frameworks with tools
print("Testing other frameworks...")
try:
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
results['Agno'] = measure_instantiation(
lambda: AgnoAgent(model=OpenAIChat(id='gpt-4o-mini'), tools=tools)
)
except ImportError:
pass
try:
from agents import Agent as OpenAIAgent, function_tool
results['OpenAI Agents SDK'] = measure_instantiation(
lambda: OpenAIAgent(name='Test', model='gpt-4o-mini', tools=[function_tool(sample_tool)])
)
except ImportError:
pass
try:
from langchain_core.tools import tool as langchain_tool
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
@langchain_tool
def get_weather_lg(city: Literal['nyc', 'sf']):
"""Get weather info."""
return 'sunny' if city == 'sf' else 'cloudy'
results['LangGraph'] = measure_instantiation(
lambda: create_react_agent(model=ChatOpenAI(model='gpt-4o-mini'), tools=[get_weather_lg])
)
except ImportError:
pass
try:
from crewai.agent import Agent as CrewAgent
from crewai.tools import tool as crewai_tool
@crewai_tool("Weather Tool")
def get_weather_crew(city: Literal['nyc', 'sf']):
"""Get weather info."""
return 'sunny' if city == 'sf' else 'cloudy'
results['CrewAI'] = measure_instantiation(
lambda: CrewAgent(
role='Weather Agent',
goal='Provide weather info',
backstory='A weather expert',
tools=[get_weather_crew],
verbose=False
)
)
except ImportError:
pass
# Print results
print('\n' + '=' * 60)
print('RESULTS (WITH TOOLS)')
print('=' * 60)
sorted_results = sorted(results.items(), key=lambda x: x[1])
baseline = sorted_results[0][1] if sorted_results else 1
print(f"\n{'Framework':<25} {'Avg Time (μs)':<15} {'Relative':<10}")
print('-' * 50)
for name, avg in sorted_results:
ratio = avg / baseline
print(f'{name:<25} {avg:<15.2f} {ratio:.2f}x')
print('\n' + '=' * 60)
return results
def get_package_versions():
"""Get version numbers for benchmarked packages."""
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
'OpenAI Agents SDK': 'openai-agents',
'LangGraph': 'langgraph',
'CrewAI': 'crewai'
}
versions = {}
for display_name, pkg_name in packages.items():
try:
versions[display_name] = get_version(pkg_name)
except Exception:
versions[display_name] = 'not installed'
return versions
def save_results(results: dict, filename: str = 'TOOLS_BENCHMARK_RESULTS.md'):
"""Save benchmark results to a markdown file."""
import os
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), filename)
versions = get_package_versions()
# Find the fastest result (baseline)
sorted_results = sorted(results.items(), key=lambda x: x[1])
fastest_time = sorted_results[0][1] if sorted_results else 1
# Build the table rows
table_rows = []
for name, avg in sorted_results:
ratio = avg / fastest_time
if ratio == 1.0:
table_rows.append(f'| **{name}** | **{avg:.2f}** | **1.00x (fastest)** |')
else:
avg_str = f'{avg:,.2f}' if avg >= 1000 else f'{avg:.2f}'
ratio_str = f'{ratio:,.0f}x' if ratio >= 100 else f'{ratio:.2f}x'
table_rows.append(f'| {name} | {avg_str} | {ratio_str} |')
# Save to file
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Tools Benchmark Results\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write(f'**Iterations:** {ITERATIONS}\n')
f.write('**Test:** Agent instantiation WITH TOOLS\n\n')
f.write('## Results\n\n')
f.write('| Framework | Avg Time (μs) | Relative |\n')
f.write('|-----------|---------------|----------|\n')
f.write('\n'.join(table_rows) + '\n')
f.write('\n## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/tools_benchmark.py\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
return filepath
if __name__ == '__main__':
results = run_benchmark()
save_results(results)
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,9 @@
Starting test with verbose=False
CALLBACK EXECUTED!
Message:
You need to do the following task: Say hello world.
Expected Output: A greeting.
Please provide only the final result of your work. Do not add any conversation or extra explanation.
Response: Hello, World!
Other kwargs: {'markdown': True, 'generation_time': 2.685833215713501, 'agent_name': 'TestAgent', 'agent_role': 'Assistant', 'agent_tools': None, 'task_name': 'simple_task', 'task_description': 'Say hello world', 'task_id': 0}
--------------------------------------------------
@@ -54,6 +54,13 @@ except ImportError:
MCP = None
from .session import Session
from .memory.memory import Memory
# Workflows - import from dedicated workflows module
from .workflows import (
Workflow, WorkflowStep, WorkflowContext, StepResult,
Route, Parallel, Loop, Repeat,
route, parallel, loop, repeat,
Pipeline # Alias for Workflow
)
from .guardrails import GuardrailResult, LLMGuardrail
# Fast Context support (lazy loaded to avoid performance impact)
@@ -298,3 +305,12 @@ __all__.extend([
'LineRange'
])
# AG-UI support (optional, lazy loaded)
try:
from praisonaiagents.ui.agui import AGUI
__all__.append('AGUI')
_agui_available = True
except ImportError:
_agui_available = False
AGUI = None
@@ -248,7 +248,9 @@ class Agent:
fast_context_model: str = "gpt-4o-mini",
fast_context_max_turns: int = 4,
fast_context_parallelism: int = 8,
fast_context_timeout: float = 30.0
fast_context_timeout: float = 30.0,
history_in_context: Optional[int] = None,
auto_save: Optional[str] = None
):
"""Initialize an Agent instance.
@@ -544,9 +546,14 @@ Your Goal: {self.goal}
self.prompt_caching = prompt_caching
self.claude_memory = claude_memory
# Session management
self.history_in_context = history_in_context # Number of past sessions to include
self.auto_save = auto_save # Session name for auto-saving
# Initialize rules manager for persistent context (like Cursor/Windsurf)
# NOTE: Lazy initialization - rules are loaded only when accessed (performance optimization)
self._rules_manager = None
self._init_rules_manager()
self._rules_manager_initialized = False
# Handle web_search fallback: inject DuckDuckGo tool for unsupported models
if web_search and not self._model_supports_web_search():
@@ -593,7 +600,7 @@ Your Goal: {self.goal}
# Fast Context configuration (lazy loaded)
self.fast_context_enabled = fast_context
self.fast_context_path = fast_context_path or os.getcwd()
self._fast_context_path = fast_context_path # Store raw, resolve lazily
self.fast_context_model = fast_context_model
self.fast_context_max_turns = fast_context_max_turns
self.fast_context_parallelism = fast_context_parallelism
@@ -608,6 +615,17 @@ Your Goal: {self.goal}
self._console = Console()
return self._console
@property
def fast_context_path(self):
"""Lazily resolve fast_context_path - avoids os.getcwd() on every init."""
if self._fast_context_path is None:
self._fast_context_path = os.getcwd()
return self._fast_context_path
@fast_context_path.setter
def fast_context_path(self, value):
self._fast_context_path = value
@property
def _openai_client(self):
"""Lazily initialize OpenAI client only when needed."""
@@ -779,6 +797,21 @@ Your Goal: {self.goal}
logging.warning(f"FastContext search failed: {e}")
return None
@property
def rules_manager(self):
"""
Lazy-initialized RulesManager for persistent rules/instructions.
This property initializes the RulesManager only when first accessed,
avoiding expensive filesystem operations during agent instantiation.
Returns:
RulesManager instance or None if not available
"""
if not self._rules_manager_initialized:
self._init_rules_manager()
return self._rules_manager
def _init_rules_manager(self):
"""
Initialize RulesManager for persistent rules/instructions.
@@ -787,7 +820,10 @@ Your Goal: {self.goal}
- ~/.praison/rules/ (global)
- .praison/rules/ (workspace)
- Subdirectory rules
NOTE: This is called lazily via the rules_manager property for performance.
"""
self._rules_manager_initialized = True
try:
from ..memory.rules_manager import RulesManager
import os
@@ -822,10 +858,10 @@ Your Goal: {self.goal}
Returns:
Formatted rules context string
"""
if not self._rules_manager:
if not self.rules_manager:
return ""
return self._rules_manager.build_rules_context(
return self.rules_manager.build_rules_context(
file_path=file_path,
include_manual=include_manual
)
@@ -1233,8 +1269,8 @@ Your Goal: {self.goal}
Your Role: {self.role}\n
Your Goal: {self.goal}"""
# Add rules context if rules manager is enabled
if self._rules_manager:
# Add rules context if rules manager is enabled (lazy initialization)
if self._rules_manager_initialized and self._rules_manager:
rules_context = self.get_rules_context()
if rules_context:
system_prompt += f"\n\n## Rules (Guidelines you must follow)\n{rules_context}"
@@ -1533,23 +1569,54 @@ Your Goal: {self.goal}"""
# Special handling for MCP tools
# Check if tools is an MCP instance with the requested function name
from ..mcp.mcp import MCP
# Helper function to execute MCP tool
def _execute_mcp_tool(mcp_instance, func_name, args):
"""Execute a tool from an MCP instance."""
# Handle SSE MCP client
if hasattr(mcp_instance, 'is_sse') and mcp_instance.is_sse:
if hasattr(mcp_instance, 'sse_client'):
for tool in mcp_instance.sse_client.tools:
if tool.name == func_name:
logging.debug(f"Found matching SSE MCP tool: {func_name}")
return True, tool(**args)
# Handle HTTP Stream MCP client
if hasattr(mcp_instance, 'is_http_stream') and mcp_instance.is_http_stream:
if hasattr(mcp_instance, 'http_stream_client'):
for tool in mcp_instance.http_stream_client.tools:
if tool.name == func_name:
logging.debug(f"Found matching HTTP Stream MCP tool: {func_name}")
return True, tool(**args)
# Handle WebSocket MCP client
if hasattr(mcp_instance, 'is_websocket') and mcp_instance.is_websocket:
if hasattr(mcp_instance, 'websocket_client'):
for tool in mcp_instance.websocket_client.tools:
if tool.name == func_name:
logging.debug(f"Found matching WebSocket MCP tool: {func_name}")
return True, tool(**args)
# Handle stdio MCP client
if hasattr(mcp_instance, 'runner'):
for mcp_tool in mcp_instance.runner.tools:
if hasattr(mcp_tool, 'name') and mcp_tool.name == func_name:
logging.debug(f"Found matching MCP tool: {func_name}")
return True, mcp_instance.runner.call_tool(func_name, args)
return False, None
# Check if tools is a single MCP instance
if isinstance(self.tools, MCP):
logging.debug(f"Looking for MCP tool {function_name}")
# Handle SSE MCP client
if hasattr(self.tools, 'is_sse') and self.tools.is_sse:
if hasattr(self.tools, 'sse_client'):
for tool in self.tools.sse_client.tools:
if tool.name == function_name:
logging.debug(f"Found matching SSE MCP tool: {function_name}")
return tool(**arguments)
# Handle stdio MCP client
elif hasattr(self.tools, 'runner'):
# Check if any of the MCP tools match the function name
for mcp_tool in self.tools.runner.tools:
if hasattr(mcp_tool, 'name') and mcp_tool.name == function_name:
logging.debug(f"Found matching MCP tool: {function_name}")
return self.tools.runner.call_tool(function_name, arguments)
found, result = _execute_mcp_tool(self.tools, function_name, arguments)
if found:
return result
# Check if tools is a list that may contain MCP instances
if isinstance(self.tools, (list, tuple)):
for tool in self.tools:
if isinstance(tool, MCP):
logging.debug(f"Looking for MCP tool {function_name} in MCP instance")
found, result = _execute_mcp_tool(tool, function_name, arguments)
if found:
return result
# Try to find the function in the agent's tools list first
func = None
@@ -1864,7 +1931,8 @@ Your Goal: {self.goal}"""
if tool_param is not None:
from ..mcp.mcp import MCP
if isinstance(tool_param, MCP) and hasattr(tool_param, 'to_openai_tool'):
logging.debug("Converting MCP tool to OpenAI format")
# Single MCP instance
logging.debug("Converting single MCP tool to OpenAI format")
openai_tool = tool_param.to_openai_tool()
if openai_tool:
# Handle both single tool and list of tools
@@ -1873,6 +1941,22 @@ Your Goal: {self.goal}"""
else:
tool_param = [openai_tool]
logging.debug(f"Converted MCP tool: {tool_param}")
elif isinstance(tool_param, (list, tuple)):
# List that may contain MCP instances - convert each MCP to OpenAI format
converted_tools = []
for t in tool_param:
if isinstance(t, MCP) and hasattr(t, 'to_openai_tool'):
logging.debug("Converting MCP instance in list to OpenAI format")
openai_tools = t.to_openai_tool()
if isinstance(openai_tools, list):
converted_tools.extend(openai_tools)
elif openai_tools:
converted_tools.append(openai_tools)
else:
# Keep non-MCP tools as-is
converted_tools.append(t)
tool_param = converted_tools
logging.debug(f"Converted {len(converted_tools)} tools from list")
# Store chat history length for potential rollback
chat_history_length = len(self.chat_history)
@@ -2627,10 +2711,10 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
# Execute the step
result = self.chat(step_prompt, **kwargs)
results.append(result)
results.append({"step": i + 1, "description": step.description, "result": result})
# Update context for next step
context += f"\n\nStep {i + 1} result: {result[:500] if result else 'No result'}"
# Update context for next step (use full result, not truncated)
context += f"\n\nStep {i + 1} result: {result if result else 'No result'}"
console.print(f" [green]✅ Completed[/green]")
@@ -2638,26 +2722,116 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
console.print(f"[dim]Progress: [{'█' * bar_length}] 100%[/dim]")
console.print(f"Completed {len(plan.steps)}/{len(plan.steps)} steps!\n")
# Return the final result
return results[-1] if results else None
# Compile all results into a comprehensive final output
if len(results) > 1:
# Create a compilation prompt
all_results_text = "\n\n".join([
f"## Step {r['step']}: {r['description']}\n\n{r['result']}"
for r in results
])
compilation_prompt = f"""You are tasked with compiling a comprehensive, detailed report from the following research steps.
IMPORTANT: Write a DETAILED and COMPREHENSIVE report. Do NOT summarize or compress the information.
Include ALL relevant details, data, statistics, and findings from each step.
Organize the information logically with clear sections and subsections.
## Research Results to Compile:
{all_results_text}
## Instructions:
1. Combine all the information into a single, well-organized document
2. Preserve ALL details, numbers, statistics, and specific findings
3. Use clear headings and subheadings
4. Do not omit any important information
5. Make it comprehensive and detailed
Write the complete compiled report:"""
console.print("\n[bold blue]📝 COMPILING FINAL REPORT[/bold blue]")
console.print("[dim]Creating comprehensive output from all steps...[/dim]\n")
final_result = self.chat(compilation_prompt, **kwargs)
return final_result
# Return the single result if only one step
return results[0]["result"] if results else None
def start(self, prompt: str, **kwargs):
"""Start the agent with a prompt. This is a convenience method that wraps chat()."""
# Load history from past sessions if history_in_context is set
self._load_history_context()
# Check if planning mode is enabled
if self.planning:
return self._start_with_planning(prompt, **kwargs)
# Check if streaming is enabled (either from kwargs or agent's stream attribute)
stream_enabled = kwargs.get('stream', getattr(self, 'stream', False))
if stream_enabled:
result = self._start_with_planning(prompt, **kwargs)
elif kwargs.get('stream', getattr(self, 'stream', False)):
# Return a generator for streaming response
return self._start_stream(prompt, **kwargs)
result = self._start_stream(prompt, **kwargs)
else:
# Return regular chat response for backward compatibility
# Explicitly pass the resolved stream parameter to avoid chat() method default
kwargs['stream'] = stream_enabled
return self.chat(prompt, **kwargs)
kwargs['stream'] = False
result = self.chat(prompt, **kwargs)
# Auto-save session if enabled
self._auto_save_session()
return result
def _load_history_context(self):
"""Load history from past sessions into context if history_in_context is set."""
if not self.history_in_context or not self._memory_instance:
return
try:
sessions = self._memory_instance.list_sessions()
if not sessions:
return
# Get the last N sessions
sessions_to_load = sessions[:self.history_in_context]
for session_info in reversed(sessions_to_load):
try:
session_data = self._memory_instance.resume_session(session_info["name"])
history = session_data.get("conversation_history", [])
# Add history to chat_history with a marker
for msg in history:
if msg not in self.chat_history:
# Mark as from history to avoid duplication
msg_copy = msg.copy()
msg_copy["_from_history"] = True
self.chat_history.append(msg_copy)
except Exception:
continue
if sessions_to_load:
logging.debug(f"Loaded history from {len(sessions_to_load)} past session(s)")
except Exception as e:
logging.debug(f"Error loading history context: {e}")
def _auto_save_session(self):
"""Auto-save session if auto_save is enabled."""
if not self.auto_save or not self._memory_instance:
return
try:
# Filter out history markers before saving
clean_history = [
{k: v for k, v in msg.items() if k != "_from_history"}
for msg in self.chat_history
]
self._memory_instance.save_session(
name=self.auto_save,
conversation_history=clean_history,
metadata={"agent_name": self.name, "user_id": self.user_id}
)
logging.debug(f"Auto-saved session: {self.auto_save}")
except Exception as e:
logging.debug(f"Error auto-saving session: {e}")
def _start_stream(self, prompt: str, **kwargs) -> Generator[str, None, None]:
"""Stream generator for real-time response chunks."""
@@ -347,6 +347,37 @@ Expanded prompt:"""
current_date=current_date
)
# Add tool usage instructions if tools are available
if self.tools:
# Build dynamic tool descriptions from docstrings
tool_descriptions = []
for tool in self.tools:
tool_name = getattr(tool, '__name__', str(tool))
tool_doc = getattr(tool, '__doc__', '')
# Get first line of docstring as description
if tool_doc:
first_line = tool_doc.strip().split('\n')[0].strip()
tool_descriptions.append(f"- {tool_name}: {first_line}")
else:
tool_descriptions.append(f"- {tool_name}")
tools_list = '\n'.join(tool_descriptions)
tool_instruction = f"""
IMPORTANT: You have the following tools available:
{tools_list}
Before expanding this prompt, FIRST use the available tools to gather relevant current information about the topic. This will help you create a more informed and accurate expanded prompt.
Steps:
1. Use the appropriate tool(s) to gather relevant information about the topic
2. Use the gathered information to inform your expansion
3. Create an expanded prompt that incorporates the latest context
"""
expansion_prompt = tool_instruction + expansion_prompt
# Call the agent
expanded = self._call_agent(expansion_prompt).strip()
@@ -2,7 +2,7 @@ import os
import time
import json
import logging
from typing import Any, Dict, Optional, List
from typing import Any, Dict, Optional, List, Callable
from pydantic import BaseModel
from rich.text import Text
from rich.panel import Panel
@@ -127,7 +127,7 @@ def process_task_context(context_item, verbose=0, user_id=None):
return str(context_item) # Fallback for unknown types
class PraisonAIAgents:
def __init__(self, agents, tasks=None, verbose=0, completion_checker=None, max_retries=5, process="sequential", manager_llm=None, memory=False, memory_config=None, embedder=None, user_id=None, max_iter=10, stream=True, name: Optional[str] = None, planning: bool = False, planning_llm: Optional[str] = None, auto_approve_plan: bool = False, planning_tools: Optional[List] = None, planning_reasoning: bool = False):
def __init__(self, agents, tasks=None, verbose=0, completion_checker=None, max_retries=5, process="sequential", manager_llm=None, memory=False, memory_config=None, embedder=None, user_id=None, max_iter=10, stream=True, name: Optional[str] = None, planning: bool = False, planning_llm: Optional[str] = None, auto_approve_plan: bool = False, planning_tools: Optional[List] = None, planning_reasoning: bool = False, on_task_start: Optional[Callable] = None, on_task_complete: Optional[Callable] = None, variables: Optional[Dict[str, Any]] = None):
# Add check at the start if memory is requested
if memory:
try:
@@ -162,6 +162,11 @@ class PraisonAIAgents:
self.stream = stream
self.name = name # Store the name for the Agents collection
# Callbacks for workflow execution
self.on_task_start = on_task_start # Called before each task
self.on_task_complete = on_task_complete # Called after each task
self.variables = variables if variables else {} # Global variables for substitution
# Check for manager_llm in environment variable if not provided
self.manager_llm = manager_llm or os.getenv('OPENAI_MODEL_NAME', 'gpt-5-nano')
@@ -661,13 +666,24 @@ Context:
executor_agent = task.agent
# Create agent from agent_config if provided and no agent assigned
if executor_agent is None and getattr(task, 'agent_config', None):
executor_agent = self._create_agent_from_config(task.agent_config)
task.agent = executor_agent
# Set current agent for token tracking
llm = getattr(executor_agent, 'llm', None) or getattr(executor_agent, 'llm_instance', None)
if llm and hasattr(llm, 'set_current_agent'):
llm.set_current_agent(executor_agent.name)
# Substitute variables in task description if provided
task_description = task.description
if getattr(task, 'variables', None):
for key, value in task.variables.items():
task_description = task_description.replace(f"{{{{{key}}}}}", str(value))
task_prompt = f"""
You need to do the following task: {task.description}.
You need to do the following task: {task_description}.
Expected Output: {task.expected_output}.
"""
if task.context:
@@ -825,6 +841,17 @@ Context:
logger.info(f"Task with ID {task_id} is already completed")
return
# Call on_task_start callback if provided
if self.on_task_start:
try:
self.on_task_start(task, task_id)
except Exception as e:
logger.error(f"Error in on_task_start callback: {e}")
# Apply global variables to task if not already set
if self.variables and not getattr(task, 'variables', None):
task.variables = self.variables
retries = 0
while task.status != "completed" and retries < self.max_retries:
logger.debug(f"Attempt {retries+1} for task {task_id}")
@@ -857,6 +884,14 @@ Context:
logger.exception(e)
self.save_output_to_file(task, task_output)
# Call on_task_complete callback if provided
if self.on_task_complete:
try:
self.on_task_complete(task, task_output)
except Exception as e:
logger.error(f"Error in on_task_complete callback: {e}")
if self.verbose >= 1:
logger.info(f"Task {task_id} completed successfully.")
else:
@@ -1335,6 +1370,61 @@ Context:
agent_names = ", ".join([agent.name for agent in self.agents])
print(f"📊 Available agents for this endpoint ({len(self.agents)}): {agent_names}")
# Create per-agent endpoints for individual agent access
# This allows n8n and other tools to call specific agents
agents_dict = {agent.name.lower().replace(' ', '_'): agent for agent in self.agents}
# Add GET endpoint to list available agents
@_agents_shared_apps[port].get(f"{path}/list")
async def list_agents():
return {
"agents": [
{"name": agent.name, "id": agent.name.lower().replace(' ', '_')}
for agent in self.agents
]
}
# Add per-agent POST endpoints
for agent_id, agent_instance in agents_dict.items():
agent_path = f"{path}/{agent_id}"
# Create a closure to capture the agent instance
def create_agent_handler(agent):
async def handle_single_agent(request: Request):
try:
request_data = await request.json()
query = request_data.get("query", "")
if not query:
raise HTTPException(status_code=400, detail="Missing 'query' field")
except:
raise HTTPException(status_code=400, detail="Invalid JSON body")
try:
if asyncio.iscoroutinefunction(agent.chat):
response = await agent.achat(query)
else:
loop = asyncio.get_running_loop()
response = await loop.run_in_executor(None, lambda q=query: agent.chat(q))
return {
"agent": agent.name,
"query": query,
"response": response
}
except Exception as e:
logging.error(f"Error with agent {agent.name}: {str(e)}", exc_info=True)
return JSONResponse(
status_code=500,
content={"error": f"Agent error: {str(e)}"}
)
return handle_single_agent
# Register the endpoint
_agents_shared_apps[port].post(agent_path)(create_agent_handler(agent_instance))
_agents_registered_endpoints[port][agent_path] = f"{endpoint_id}_{agent_id}"
print(f"🔗 Per-agent endpoints: {', '.join([f'{path}/{aid}' for aid in agents_dict.keys()])}")
# Start the server if it's not already running for this port
if not _agents_server_started.get(port, False):
# Mark the server as started first to prevent duplicate starts
@@ -1561,6 +1651,29 @@ Context:
"""Get the current todo list."""
return self._todo_list
def _create_agent_from_config(self, agent_config: dict) -> 'Agent':
"""
Create an Agent from a configuration dictionary.
Args:
agent_config: Dict with keys like 'name', 'role', 'goal', 'backstory', 'llm', 'tools'
Returns:
Agent instance
"""
from ..agent.agent import Agent
return Agent(
name=agent_config.get('name', 'TaskAgent'),
role=agent_config.get('role'),
goal=agent_config.get('goal'),
backstory=agent_config.get('backstory'),
llm=agent_config.get('llm'),
tools=agent_config.get('tools'),
verbose=self.verbose >= 1,
memory=self.memory
)
def _get_planning_agent(self):
"""Lazy load PlanningAgent."""
if self._planning_agent is None and self.planning:
@@ -15,6 +15,8 @@ This module provides memory management capabilities including:
- Auto-generated memories (like Windsurf Cascade)
- Workflows (like Windsurf)
- Hooks (like Windsurf Cascade Hooks)
- Docs management (like Cursor docs)
- MCP config management (like Cursor .cursor/mcp/)
Memory Providers:
- FileMemory: Zero-dependency JSON file-based storage (default)
@@ -23,10 +25,14 @@ Memory Providers:
- AutoMemory: Automatic memory extraction from conversations
- WorkflowManager: Multi-step workflow execution
- HooksManager: Pre/post operation hooks
- DocsManager: Documentation context management
- MCPConfigManager: MCP server configuration management
"""
from .file_memory import FileMemory, create_memory
from .rules_manager import RulesManager, Rule, create_rules_manager
from .docs_manager import DocsManager, Doc
from .mcp_config import MCPConfigManager, MCPConfig
# Lazy imports for optional modules to avoid dependency issues and improve startup time
def __getattr__(name):
@@ -42,17 +48,58 @@ def __getattr__(name):
if name == "create_auto_memory":
from .auto_memory import create_auto_memory
return create_auto_memory
# Workflows - now in dedicated module, re-export for backward compatibility
if name == "WorkflowManager":
from .workflows import WorkflowManager
from ..workflows import WorkflowManager
return WorkflowManager
if name == "Workflow":
from .workflows import Workflow
from ..workflows import Workflow
return Workflow
if name == "WorkflowStep":
from .workflows import WorkflowStep
from ..workflows import WorkflowStep
return WorkflowStep
if name == "WorkflowContext":
from ..workflows import WorkflowContext
return WorkflowContext
if name == "StepResult":
from ..workflows import StepResult
return StepResult
if name == "Pipeline":
from ..workflows import Pipeline
return Pipeline
if name == "Route":
from ..workflows import Route
return Route
if name == "Parallel":
from ..workflows import Parallel
return Parallel
if name == "Loop":
from ..workflows import Loop
return Loop
if name == "Repeat":
from ..workflows import Repeat
return Repeat
if name == "route":
from ..workflows import route
return route
if name == "parallel":
from ..workflows import parallel
return parallel
if name == "loop":
from ..workflows import loop
return loop
if name == "repeat":
from ..workflows import repeat
return repeat
# Backward compatibility aliases
if name == "StepInput":
from ..workflows import WorkflowContext
return WorkflowContext
if name == "StepOutput":
from ..workflows import StepResult
return StepResult
if name == "create_workflow_manager":
from .workflows import create_workflow_manager
from ..workflows.workflows import create_workflow_manager
return create_workflow_manager
if name == "HooksManager":
from .hooks import HooksManager
@@ -63,6 +110,7 @@ def __getattr__(name):
if name == "create_hooks_manager":
from .hooks import create_hooks_manager
return create_hooks_manager
# DocsManager and MCPConfigManager are already imported at module level
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
__all__ = [
@@ -78,13 +126,31 @@ __all__ = [
"AutoMemory",
"AutoMemoryExtractor",
"create_auto_memory",
# Workflows
# Workflows (re-exported from praisonaiagents.workflows for backward compatibility)
"WorkflowManager",
"Workflow",
"Pipeline",
"WorkflowStep",
"WorkflowContext",
"StepResult",
"create_workflow_manager",
# Workflow patterns
"Route",
"Parallel",
"Loop",
"Repeat",
"route",
"parallel",
"loop",
"repeat",
# Hooks
"HooksManager",
"HookResult",
"create_hooks_manager",
# Docs management
"DocsManager",
"Doc",
# MCP config management
"MCPConfigManager",
"MCPConfig",
]
@@ -0,0 +1,431 @@
"""
Docs Manager for PraisonAI Agents.
Provides reusable documentation context support similar to:
- Cursor Notepads (deprecated in favor of docs)
- Cursor .cursor/docs/
- Project documentation for AI context
Features:
- Auto-discovery of docs from .praison/docs/
- Support for markdown, text, and code files
- @doc mentions for including specific docs
- Priority ordering for docs
- Character limits for context window management
Storage Structure:
.praison/docs/
├── project-overview.md # Project overview
├── architecture.md # Architecture decisions
├── api-reference.md # API documentation
└── coding-standards.md # Coding guidelines
Doc File Format (YAML frontmatter + Markdown):
---
description: Project architecture overview
priority: 10
tags: [architecture, design]
---
# Architecture Overview
This project uses...
"""
import os
import logging
from pathlib import Path
from typing import Any, Dict, List, Optional, Literal
from dataclasses import dataclass, field
logger = logging.getLogger(__name__)
@dataclass
class Doc:
"""A single documentation file with metadata and content."""
name: str
content: str
description: str = ""
priority: int = 0
tags: List[str] = field(default_factory=list)
file_path: Optional[str] = None
def matches_tag(self, tag: str) -> bool:
"""Check if this doc has a specific tag."""
return tag.lower() in [t.lower() for t in self.tags]
def to_dict(self) -> Dict[str, Any]:
return {
"name": self.name,
"content": self.content,
"description": self.description,
"priority": self.priority,
"tags": self.tags,
"file_path": self.file_path
}
@classmethod
def from_dict(cls, data: Dict[str, Any]) -> 'Doc':
return cls(**data)
class DocsManager:
"""
Manages documentation context for AI agents.
Provides:
- Auto-discovery of docs from .praison/docs/
- Global docs from ~/.praison/docs/
- @doc mentions for including specific docs
- Priority-based ordering
"""
DOCS_DIR_NAME = ".praison/docs"
SUPPORTED_EXTENSIONS = [".md", ".txt", ".rst"]
# Maximum characters per doc (context window management)
MAX_DOC_CHARS = 12000
def __init__(
self,
workspace_path: Optional[str] = None,
global_docs_path: Optional[str] = None,
verbose: int = 0
):
"""
Initialize DocsManager.
Args:
workspace_path: Path to workspace/project root
global_docs_path: Path to global docs (default: ~/.praison/docs)
verbose: Verbosity level
"""
self.workspace_path = Path(workspace_path) if workspace_path else Path.cwd()
self.global_docs_path = Path(global_docs_path) if global_docs_path else Path.home() / ".praison" / "docs"
self.verbose = verbose
self._docs: Dict[str, Doc] = {}
self._load_all_docs()
def _log(self, msg: str, level: int = logging.INFO):
"""Log message if verbose."""
if self.verbose >= 1:
logger.log(level, msg)
def _parse_frontmatter(self, content: str) -> tuple:
"""
Parse YAML frontmatter from content.
Returns:
Tuple of (frontmatter_dict, remaining_content)
"""
frontmatter = {}
remaining = content
# Check for YAML frontmatter
if content.startswith("---"):
parts = content.split("---", 2)
if len(parts) >= 3:
try:
import yaml
frontmatter = yaml.safe_load(parts[1]) or {}
remaining = parts[2].strip()
except Exception:
# If YAML parsing fails, use content as-is
pass
return frontmatter, remaining
def _load_doc_file(self, file_path: Path, scope: str = "workspace") -> Optional[Doc]:
"""Load a single doc file."""
try:
content = file_path.read_text(encoding="utf-8")
frontmatter, doc_content = self._parse_frontmatter(content)
name = file_path.stem
# Determine priority based on scope
base_priority = 100 if scope == "workspace" else 0
doc = Doc(
name=name,
content=doc_content,
description=frontmatter.get("description", ""),
priority=frontmatter.get("priority", base_priority),
tags=frontmatter.get("tags", []),
file_path=str(file_path)
)
return doc
except Exception as e:
self._log(f"Failed to load doc {file_path}: {e}", logging.WARNING)
return None
def _load_docs_from_dir(self, docs_dir: Path, scope: str = "workspace"):
"""Load all docs from a directory."""
if not docs_dir.exists():
return
for ext in self.SUPPORTED_EXTENSIONS:
for file_path in docs_dir.glob(f"*{ext}"):
if file_path.is_file():
doc = self._load_doc_file(file_path, scope)
if doc:
key = f"{scope}:{doc.name}"
self._docs[key] = doc
self._log(f"Loaded doc: {key}")
def _load_all_docs(self):
"""Load all docs from global and workspace directories."""
self._docs.clear()
# 1. Load global docs (lowest priority)
if self.global_docs_path.exists():
self._load_docs_from_dir(self.global_docs_path, "global")
for key, doc in self._docs.items():
if key.startswith("global:"):
doc.priority = doc.priority - 1000 # Lower priority for global
# 2. Load workspace docs
workspace_docs_dir = self.workspace_path / self.DOCS_DIR_NAME.replace("/", os.sep)
if workspace_docs_dir.exists():
self._load_docs_from_dir(workspace_docs_dir, "workspace")
self._log(f"Loaded {len(self._docs)} docs total")
def reload(self):
"""Reload all docs from disk."""
self._load_all_docs()
def get_all_docs(self) -> List[Doc]:
"""Get all loaded docs, sorted by priority."""
docs = list(self._docs.values())
docs.sort(key=lambda d: d.priority, reverse=True)
return docs
def get_doc(self, name: str) -> Optional[Doc]:
"""Get a specific doc by name."""
# Try workspace first, then global
for scope in ["workspace", "global"]:
key = f"{scope}:{name}"
if key in self._docs:
return self._docs[key]
return None
def get_docs_by_tag(self, tag: str) -> List[Doc]:
"""Get all docs with a specific tag."""
docs = [d for d in self._docs.values() if d.matches_tag(tag)]
docs.sort(key=lambda d: d.priority, reverse=True)
return docs
def get_docs_for_context(
self,
include_docs: Optional[List[str]] = None,
include_tags: Optional[List[str]] = None,
max_chars: int = 50000
) -> List[Doc]:
"""
Get docs for inclusion in agent context.
Args:
include_docs: Specific doc names to include (via @doc mentions)
include_tags: Tags to filter by
max_chars: Maximum total characters
Returns:
List of docs sorted by priority
"""
active = []
# Include specific docs by name
if include_docs:
for name in include_docs:
doc = self.get_doc(name)
if doc and doc not in active:
active.append(doc)
# Include docs by tag
if include_tags:
for tag in include_tags:
for doc in self.get_docs_by_tag(tag):
if doc not in active:
active.append(doc)
# If no specific docs requested, include all high-priority docs
if not include_docs and not include_tags:
for doc in self.get_all_docs():
if doc.priority >= 100: # Only high-priority docs by default
active.append(doc)
# Sort by priority
active.sort(key=lambda d: d.priority, reverse=True)
return active
def format_docs_for_prompt(
self,
include_docs: Optional[List[str]] = None,
include_tags: Optional[List[str]] = None,
max_chars: int = 50000
) -> str:
"""
Format docs as a string for inclusion in prompts.
Args:
include_docs: Specific doc names to include
include_tags: Tags to filter by
max_chars: Maximum total characters
Returns:
Formatted docs string
"""
active_docs = self.get_docs_for_context(include_docs, include_tags, max_chars)
if not active_docs:
return ""
parts = ["# Project Documentation\n"]
total_chars = len(parts[0])
for doc in active_docs:
if total_chars >= max_chars:
break
doc_text = doc.content.strip()
if not doc_text:
continue
# Add doc with header
if doc.description:
header = f"## {doc.name}: {doc.description}"
else:
header = f"## {doc.name}"
section = f"{header}\n{doc_text}\n"
if total_chars + len(section) <= max_chars:
parts.append(section)
total_chars += len(section)
else:
# Truncate last doc
remaining = max_chars - total_chars
if remaining > 100:
parts.append(section[:remaining] + "\n... (truncated)")
break
return "\n".join(parts)
def create_doc(
self,
name: str,
content: str,
description: str = "",
priority: int = 0,
tags: Optional[List[str]] = None,
scope: Literal["global", "workspace"] = "workspace"
) -> Doc:
"""
Create a new doc file.
Args:
name: Doc name (used as filename)
content: Doc content (markdown)
description: Short description
priority: Priority (higher = applied first)
tags: Tags for categorization
scope: Where to save (global or workspace)
Returns:
Created Doc object
"""
# Determine save path
if scope == "global":
docs_dir = self.global_docs_path
else:
docs_dir = self.workspace_path / self.DOCS_DIR_NAME.replace("/", os.sep)
docs_dir.mkdir(parents=True, exist_ok=True)
file_path = docs_dir / f"{name}.md"
# Build frontmatter
frontmatter_lines = ["---"]
if description:
frontmatter_lines.append(f'description: "{description}"')
if priority != 0:
frontmatter_lines.append(f"priority: {priority}")
if tags:
tags_str = ", ".join(f'"{t}"' for t in tags)
frontmatter_lines.append(f"tags: [{tags_str}]")
frontmatter_lines.append("---")
frontmatter_lines.append("")
# Write file
full_content = "\n".join(frontmatter_lines) + content
file_path.write_text(full_content, encoding="utf-8")
# Create and register doc
doc = Doc(
name=name,
content=content,
description=description,
priority=priority,
tags=tags or [],
file_path=str(file_path)
)
self._docs[f"{scope}:{name}"] = doc
self._log(f"Created doc '{name}' at {file_path}")
return doc
def delete_doc(self, name: str, scope: Optional[str] = None) -> bool:
"""
Delete a doc file.
Args:
name: Doc name to delete
scope: Scope to delete from (None = try both)
Returns:
True if deleted, False if not found
"""
scopes = [scope] if scope else ["workspace", "global"]
for s in scopes:
key = f"{s}:{name}"
if key in self._docs:
doc = self._docs[key]
if doc.file_path:
try:
Path(doc.file_path).unlink()
del self._docs[key]
self._log(f"Deleted doc '{name}' from {s}")
return True
except Exception as e:
self._log(f"Failed to delete doc file: {e}", logging.ERROR)
return False
def list_docs(self) -> List[Dict[str, Any]]:
"""
List all docs with metadata.
Returns:
List of doc info dicts
"""
docs = []
for key, doc in self._docs.items():
scope = key.split(":")[0]
docs.append({
"name": doc.name,
"scope": scope,
"description": doc.description,
"priority": doc.priority,
"tags": doc.tags,
"file_path": doc.file_path,
"content_length": len(doc.content)
})
# Sort by priority
docs.sort(key=lambda d: d["priority"], reverse=True)
return docs
@@ -0,0 +1,369 @@
"""
MCP Config Manager for PraisonAI Agents.
Provides JSON-based MCP server configuration support similar to:
- Cursor .cursor/mcp/*.mcpjson
- Windsurf MCP configuration
Features:
- Auto-discovery of MCP configs from .praison/mcp/
- JSON config files for MCP servers
- Environment variable support
- Easy server management
Storage Structure:
.praison/mcp/
├── filesystem.json # Filesystem MCP server
├── brave-search.json # Brave search MCP server
└── postgres.json # PostgreSQL MCP server
Config File Format (JSON):
{
"name": "filesystem",
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-filesystem", "."],
"env": {
"SOME_VAR": "value"
},
"enabled": true,
"description": "Filesystem access for the agent"
}
"""
import os
import json
import logging
from pathlib import Path
from typing import Any, Dict, List, Optional
from dataclasses import dataclass, field
logger = logging.getLogger(__name__)
@dataclass
class MCPConfig:
"""A single MCP server configuration."""
name: str
command: str
args: List[str] = field(default_factory=list)
env: Dict[str, str] = field(default_factory=dict)
enabled: bool = True
description: str = ""
file_path: Optional[str] = None
def to_dict(self) -> Dict[str, Any]:
return {
"name": self.name,
"command": self.command,
"args": self.args,
"env": self.env,
"enabled": self.enabled,
"description": self.description
}
@classmethod
def from_dict(cls, data: Dict[str, Any], file_path: Optional[str] = None) -> 'MCPConfig':
return cls(
name=data.get("name", ""),
command=data.get("command", ""),
args=data.get("args", []),
env=data.get("env", {}),
enabled=data.get("enabled", True),
description=data.get("description", ""),
file_path=file_path
)
def to_mcp_instance(self):
"""
Convert this config to an MCP instance.
Returns:
MCP instance ready for use with agents
"""
try:
from praisonaiagents.tools.mcp import MCP
# Resolve environment variables
resolved_env = {}
for key, value in self.env.items():
if value.startswith("$"):
# Environment variable reference
env_var = value[1:]
resolved_env[key] = os.environ.get(env_var, value)
else:
resolved_env[key] = value
return MCP(
command=self.command,
args=self.args,
env=resolved_env if resolved_env else None
)
except ImportError:
logger.warning("MCP tools not available")
return None
class MCPConfigManager:
"""
Manages MCP server configurations for AI agents.
Provides:
- Auto-discovery of MCP configs from .praison/mcp/
- Global configs from ~/.praison/mcp/
- Easy server management
"""
MCP_DIR_NAME = ".praison/mcp"
SUPPORTED_EXTENSIONS = [".json", ".mcpjson"]
def __init__(
self,
workspace_path: Optional[str] = None,
global_mcp_path: Optional[str] = None,
verbose: int = 0
):
"""
Initialize MCPConfigManager.
Args:
workspace_path: Path to workspace/project root
global_mcp_path: Path to global MCP configs (default: ~/.praison/mcp)
verbose: Verbosity level
"""
self.workspace_path = Path(workspace_path) if workspace_path else Path.cwd()
self.global_mcp_path = Path(global_mcp_path) if global_mcp_path else Path.home() / ".praison" / "mcp"
self.verbose = verbose
self._configs: Dict[str, MCPConfig] = {}
self._load_all_configs()
def _log(self, msg: str, level: int = logging.INFO):
"""Log message if verbose."""
if self.verbose >= 1:
logger.log(level, msg)
def _load_config_file(self, file_path: Path, scope: str = "workspace") -> Optional[MCPConfig]:
"""Load a single MCP config file."""
try:
content = file_path.read_text(encoding="utf-8")
data = json.loads(content)
# Use filename as name if not specified
if not data.get("name"):
data["name"] = file_path.stem
config = MCPConfig.from_dict(data, str(file_path))
return config
except json.JSONDecodeError as e:
self._log(f"Invalid JSON in {file_path}: {e}", logging.WARNING)
return None
except Exception as e:
self._log(f"Failed to load MCP config {file_path}: {e}", logging.WARNING)
return None
def _load_configs_from_dir(self, mcp_dir: Path, scope: str = "workspace"):
"""Load all MCP configs from a directory."""
if not mcp_dir.exists():
return
for ext in self.SUPPORTED_EXTENSIONS:
for file_path in mcp_dir.glob(f"*{ext}"):
if file_path.is_file():
config = self._load_config_file(file_path, scope)
if config:
key = f"{scope}:{config.name}"
self._configs[key] = config
self._log(f"Loaded MCP config: {key}")
def _load_all_configs(self):
"""Load all MCP configs from global and workspace directories."""
self._configs.clear()
# 1. Load global configs
if self.global_mcp_path.exists():
self._load_configs_from_dir(self.global_mcp_path, "global")
# 2. Load workspace configs (override global)
workspace_mcp_dir = self.workspace_path / self.MCP_DIR_NAME.replace("/", os.sep)
if workspace_mcp_dir.exists():
self._load_configs_from_dir(workspace_mcp_dir, "workspace")
self._log(f"Loaded {len(self._configs)} MCP configs total")
def reload(self):
"""Reload all configs from disk."""
self._load_all_configs()
def get_all_configs(self) -> List[MCPConfig]:
"""Get all loaded MCP configs."""
return list(self._configs.values())
def get_enabled_configs(self) -> List[MCPConfig]:
"""Get all enabled MCP configs."""
return [c for c in self._configs.values() if c.enabled]
def get_config(self, name: str) -> Optional[MCPConfig]:
"""Get a specific MCP config by name."""
# Try workspace first, then global
for scope in ["workspace", "global"]:
key = f"{scope}:{name}"
if key in self._configs:
return self._configs[key]
return None
def get_mcp_tools(self, names: Optional[List[str]] = None) -> List:
"""
Get MCP instances for use with agents.
Args:
names: Specific config names to load (None = all enabled)
Returns:
List of MCP instances
"""
tools = []
if names:
configs = [self.get_config(name) for name in names]
configs = [c for c in configs if c and c.enabled]
else:
configs = self.get_enabled_configs()
for config in configs:
mcp = config.to_mcp_instance()
if mcp:
tools.append(mcp)
return tools
def create_config(
self,
name: str,
command: str,
args: Optional[List[str]] = None,
env: Optional[Dict[str, str]] = None,
enabled: bool = True,
description: str = "",
scope: str = "workspace"
) -> MCPConfig:
"""
Create a new MCP config file.
Args:
name: Config name (used as filename)
command: Command to run (e.g., "npx")
args: Command arguments
env: Environment variables
enabled: Whether the config is enabled
description: Description of the MCP server
scope: Where to save (global or workspace)
Returns:
Created MCPConfig object
"""
# Determine save path
if scope == "global":
mcp_dir = self.global_mcp_path
else:
mcp_dir = self.workspace_path / self.MCP_DIR_NAME.replace("/", os.sep)
mcp_dir.mkdir(parents=True, exist_ok=True)
file_path = mcp_dir / f"{name}.json"
# Build config
config = MCPConfig(
name=name,
command=command,
args=args or [],
env=env or {},
enabled=enabled,
description=description,
file_path=str(file_path)
)
# Write file
file_path.write_text(
json.dumps(config.to_dict(), indent=2),
encoding="utf-8"
)
self._configs[f"{scope}:{name}"] = config
self._log(f"Created MCP config '{name}' at {file_path}")
return config
def delete_config(self, name: str, scope: Optional[str] = None) -> bool:
"""
Delete an MCP config file.
Args:
name: Config name to delete
scope: Scope to delete from (None = try both)
Returns:
True if deleted, False if not found
"""
scopes = [scope] if scope else ["workspace", "global"]
for s in scopes:
key = f"{s}:{name}"
if key in self._configs:
config = self._configs[key]
if config.file_path:
try:
Path(config.file_path).unlink()
del self._configs[key]
self._log(f"Deleted MCP config '{name}' from {s}")
return True
except Exception as e:
self._log(f"Failed to delete config file: {e}", logging.ERROR)
return False
def enable_config(self, name: str) -> bool:
"""Enable an MCP config."""
config = self.get_config(name)
if config:
config.enabled = True
self._save_config(config)
return True
return False
def disable_config(self, name: str) -> bool:
"""Disable an MCP config."""
config = self.get_config(name)
if config:
config.enabled = False
self._save_config(config)
return True
return False
def _save_config(self, config: MCPConfig):
"""Save a config back to its file."""
if config.file_path:
Path(config.file_path).write_text(
json.dumps(config.to_dict(), indent=2),
encoding="utf-8"
)
def list_configs(self) -> List[Dict[str, Any]]:
"""
List all MCP configs with metadata.
Returns:
List of config info dicts
"""
configs = []
for key, config in self._configs.items():
scope = key.split(":")[0]
configs.append({
"name": config.name,
"scope": scope,
"command": config.command,
"args": config.args,
"enabled": config.enabled,
"description": config.description,
"file_path": config.file_path
})
return configs
File diff suppressed because it is too large. Load diff
@@ -826,7 +826,34 @@ Provide a JSON with the structure:
logging.info("Hierarchical task execution finished")
def workflow(self):
"""Synchronous version of workflow method"""
"""Synchronous version of workflow method.
DEPRECATED: This method is deprecated. Use the Workflow class instead:
```python
from praisonaiagents import Workflow, StepResult
from praisonaiagents.workflows import route, parallel, loop, repeat
workflow = Workflow(steps=[step1, step2])
result = workflow.start("input")
```
The Workflow class provides a simpler API with more features:
- route() for decision-based branching
- parallel() for concurrent execution
- loop() for iteration over lists/CSV
- repeat() for evaluator-optimizer patterns
- Callbacks (on_step_start, on_step_complete, etc.)
- Guardrails with validation feedback
- Status tracking
"""
import warnings
warnings.warn(
"process='workflow' is deprecated. Use the Workflow class instead: "
"from praisonaiagents import Workflow; workflow = Workflow(steps=[...]); workflow.start()",
DeprecationWarning,
stacklevel=2
)
current_iter = 0 # Track how many times we've looped
# Build workflow relationships first
for task in self.tasks.values():
@@ -44,7 +44,9 @@ class Task:
retain_full_context: bool = False, # By default, only use previous task output, not all previous tasks
guardrail: Optional[Union[Callable[[TaskOutput], Tuple[bool, Any]], str]] = None,
max_retries: int = 3,
retry_count: int = 0
retry_count: int = 0,
agent_config: Optional[Dict[str, Any]] = None,
variables: Optional[Dict[str, Any]] = None
):
# Add check if memory config is provided
if memory is not None or (config and config.get('memory_config')):
@@ -89,6 +91,8 @@ class Task:
self.retry_count = retry_count
self._guardrail_fn = None
self.validation_feedback = None # Store validation failure feedback for retry attempts
self.agent_config = agent_config # Per-task agent configuration {role, goal, backstory, llm}
self.variables = variables if variables else {} # Variables for substitution in description
# Set logger level based on config verbose level
verbose = self.config.get("verbose", 0)
@@ -0,0 +1,319 @@
"""
Mentions Parser for PraisonAI Agents.
Provides @mention syntax support similar to Cursor IDE:
- @file:path/to/file.py - Include file content
- @web:query - Search the web
- @doc:name - Include doc from .praison/docs/
- @rule:name - Include specific rule
Usage:
from praisonaiagents.tools.mentions import MentionsParser
parser = MentionsParser()
context, cleaned_prompt = parser.process("@file:main.py explain this code")
"""
import re
import logging
from pathlib import Path
from typing import Dict, List, Optional, Tuple
logger = logging.getLogger(__name__)
class MentionsParser:
"""
Parse and process @mentions in prompts.
Supports:
- @file:path - Include file content
- @web:query - Web search (requires web search tool)
- @doc:name - Include doc from .praison/docs/
- @rule:name - Include specific rule
- @url:https://... - Fetch URL content
"""
# Patterns for different mention types
PATTERNS = {
"file": re.compile(r'@file:([^\s]+)'),
"web": re.compile(r'@web:([^\s]+(?:\s+[^\s@]+)*)'),
"doc": re.compile(r'@doc:([^\s]+)'),
"rule": re.compile(r'@rule:([^\s]+)'),
"url": re.compile(r'@url:(https?://[^\s]+)'),
}
def __init__(
self,
workspace_path: Optional[str] = None,
verbose: int = 0
):
"""
Initialize MentionsParser.
Args:
workspace_path: Path to workspace/project root
verbose: Verbosity level
"""
self.workspace_path = Path(workspace_path) if workspace_path else Path.cwd()
self.verbose = verbose
# Lazy-loaded managers
self._docs_manager = None
self._rules_manager = None
def _log(self, msg: str, level: int = logging.INFO):
"""Log message if verbose."""
if self.verbose >= 1:
logger.log(level, msg)
def _get_docs_manager(self):
"""Lazy load DocsManager."""
if self._docs_manager is None:
try:
from praisonaiagents.memory.docs_manager import DocsManager
self._docs_manager = DocsManager(workspace_path=str(self.workspace_path))
except ImportError:
self._log("DocsManager not available", logging.WARNING)
return self._docs_manager
def _get_rules_manager(self):
"""Lazy load RulesManager."""
if self._rules_manager is None:
try:
from praisonaiagents.memory.rules_manager import RulesManager
self._rules_manager = RulesManager(workspace_path=str(self.workspace_path))
except ImportError:
self._log("RulesManager not available", logging.WARNING)
return self._rules_manager
def process(self, prompt: str) -> Tuple[str, str]:
"""
Process a prompt and extract @mentions.
Args:
prompt: The input prompt with @mentions
Returns:
Tuple of (context_string, cleaned_prompt)
"""
context_parts = []
cleaned_prompt = prompt
# Process each mention type
for mention_type, pattern in self.PATTERNS.items():
matches = pattern.findall(prompt)
for match in matches:
context = self._process_mention(mention_type, match)
if context:
context_parts.append(context)
# Remove the mention from the prompt
cleaned_prompt = pattern.sub('', cleaned_prompt, count=1)
# Clean up extra whitespace
cleaned_prompt = ' '.join(cleaned_prompt.split())
# Build context string
context_string = ""
if context_parts:
context_string = "\n\n".join(context_parts) + "\n\n"
return context_string, cleaned_prompt
def _process_mention(self, mention_type: str, value: str) -> Optional[str]:
"""
Process a single mention.
Args:
mention_type: Type of mention (file, web, doc, rule, url)
value: The mention value
Returns:
Context string or None
"""
if mention_type == "file":
return self._process_file_mention(value)
elif mention_type == "web":
return self._process_web_mention(value)
elif mention_type == "doc":
return self._process_doc_mention(value)
elif mention_type == "rule":
return self._process_rule_mention(value)
elif mention_type == "url":
return self._process_url_mention(value)
return None
def _process_file_mention(self, file_path: str) -> Optional[str]:
"""Process @file:path mention."""
try:
# Resolve path relative to workspace
full_path = self.workspace_path / file_path
if not full_path.exists():
# Try as absolute path
full_path = Path(file_path)
if not full_path.exists():
self._log(f"File not found: {file_path}", logging.WARNING)
return f"# File: {file_path}\n[File not found]"
content = full_path.read_text(encoding="utf-8")
# Limit content size
if len(content) > 50000:
content = content[:50000] + "\n... (truncated)"
# Detect language for code fence
ext = full_path.suffix.lower()
lang_map = {
".py": "python",
".js": "javascript",
".ts": "typescript",
".jsx": "jsx",
".tsx": "tsx",
".java": "java",
".cpp": "cpp",
".c": "c",
".go": "go",
".rs": "rust",
".rb": "ruby",
".php": "php",
".swift": "swift",
".kt": "kotlin",
".md": "markdown",
".json": "json",
".yaml": "yaml",
".yml": "yaml",
".xml": "xml",
".html": "html",
".css": "css",
".sql": "sql",
".sh": "bash",
}
lang = lang_map.get(ext, "")
return f"# File: {file_path}\n```{lang}\n{content}\n```"
except Exception as e:
self._log(f"Error reading file {file_path}: {e}", logging.ERROR)
return f"# File: {file_path}\n[Error reading file: {e}]"
def _process_web_mention(self, query: str) -> Optional[str]:
"""Process @web:query mention."""
try:
# Try to use web search
from praisonaiagents.tools import duckduckgo_search
results = duckduckgo_search(query, max_results=3)
if results:
return f"# Web Search: {query}\n{results}"
else:
return f"# Web Search: {query}\n[No results found]"
except ImportError:
self._log("Web search not available", logging.WARNING)
return f"# Web Search: {query}\n[Web search not available. Install with: pip install duckduckgo-search]"
except Exception as e:
self._log(f"Error in web search: {e}", logging.ERROR)
return f"# Web Search: {query}\n[Error: {e}]"
def _process_doc_mention(self, doc_name: str) -> Optional[str]:
"""Process @doc:name mention."""
docs = self._get_docs_manager()
if not docs:
return f"# Doc: {doc_name}\n[DocsManager not available]"
doc = docs.get_doc(doc_name)
if doc:
return f"# Doc: {doc.name}\n{doc.content}"
else:
return f"# Doc: {doc_name}\n[Doc not found]"
def _process_rule_mention(self, rule_name: str) -> Optional[str]:
"""Process @rule:name mention."""
rules = self._get_rules_manager()
if not rules:
return f"# Rule: {rule_name}\n[RulesManager not available]"
rule = rules.get_rule(rule_name)
if rule:
return f"# Rule: {rule.name}\n{rule.content}"
else:
return f"# Rule: {rule_name}\n[Rule not found]"
def _process_url_mention(self, url: str) -> Optional[str]:
"""Process @url:https://... mention."""
try:
import urllib.request
req = urllib.request.Request(
url,
headers={'User-Agent': 'Mozilla/5.0 (compatible; PraisonAI/1.0)'}
)
with urllib.request.urlopen(req, timeout=10) as response:
content = response.read().decode('utf-8', errors='ignore')
# Limit content size
if len(content) > 50000:
content = content[:50000] + "\n... (truncated)"
# Try to extract text from HTML
if '<html' in content.lower():
# Simple HTML text extraction
import re
# Remove scripts and styles
content = re.sub(r'<script[^>]*>.*?</script>', '', content, flags=re.DOTALL | re.IGNORECASE)
content = re.sub(r'<style[^>]*>.*?</style>', '', content, flags=re.DOTALL | re.IGNORECASE)
# Remove tags
content = re.sub(r'<[^>]+>', ' ', content)
# Clean whitespace
content = ' '.join(content.split())
return f"# URL: {url}\n{content[:10000]}"
except Exception as e:
self._log(f"Error fetching URL {url}: {e}", logging.ERROR)
return f"# URL: {url}\n[Error fetching URL: {e}]"
def extract_mentions(self, prompt: str) -> Dict[str, List[str]]:
"""
Extract all mentions from a prompt without processing them.
Args:
prompt: The input prompt
Returns:
Dict mapping mention types to lists of values
"""
mentions = {}
for mention_type, pattern in self.PATTERNS.items():
matches = pattern.findall(prompt)
if matches:
mentions[mention_type] = matches
return mentions
def has_mentions(self, prompt: str) -> bool:
"""Check if a prompt contains any @mentions."""
for pattern in self.PATTERNS.values():
if pattern.search(prompt):
return True
return False
def process_mentions(prompt: str, workspace_path: Optional[str] = None) -> Tuple[str, str]:
"""
Convenience function to process mentions in a prompt.
Args:
prompt: The input prompt with @mentions
workspace_path: Optional workspace path
Returns:
Tuple of (context_string, cleaned_prompt)
"""
parser = MentionsParser(workspace_path=workspace_path)
return parser.process(prompt)
# Export
__all__ = ["MentionsParser", "process_mentions"]
@@ -0,0 +1,265 @@
"""
Rules Tools for PraisonAI Agents.
Provides tools for agents to create, read, and manage rules dynamically.
This enables Agent-Requested rules similar to Cursor's agent-requested rules feature.
Usage:
from praisonaiagents.tools.rules_tools import (
create_rule_tool,
list_rules_tool,
get_rule_tool,
delete_rule_tool
)
agent = Agent(
name="Assistant",
tools=[create_rule_tool, list_rules_tool, get_rule_tool]
)
"""
import logging
from typing import Optional
logger = logging.getLogger(__name__)
# Singleton RulesManager instance
_rules_manager = None
def _get_rules_manager():
"""Get or create the RulesManager singleton."""
global _rules_manager
if _rules_manager is None:
from praisonaiagents.memory.rules_manager import RulesManager
_rules_manager = RulesManager()
return _rules_manager
def create_rule_tool(
name: str,
content: str,
description: str = "",
globs: Optional[str] = None,
activation: str = "always",
priority: int = 0,
scope: str = "workspace"
) -> str:
"""
Create a new rule for the AI agent to follow.
This tool allows the agent to create persistent rules that will be
applied to future conversations and tasks.
Args:
name: Rule name (used as filename, no spaces or special chars)
content: The rule content in markdown format
description: Short description of what the rule does
globs: Comma-separated glob patterns for file matching (e.g., "**/*.py,**/*.js")
activation: When to apply the rule:
- "always": Always apply this rule
- "glob": Apply when working with files matching globs
- "manual": Only apply when explicitly mentioned
- "ai_decision": Let AI decide when to apply
priority: Priority (higher = applied first, default 0)
scope: Where to save ("workspace" or "global")
Returns:
Success message with rule details
Example:
create_rule_tool(
name="python-style",
content="- Use type hints for all functions\\n- Follow PEP 8",
description="Python coding standards",
globs="**/*.py",
activation="glob"
)
"""
try:
manager = _get_rules_manager()
# Parse globs
glob_list = None
if globs:
glob_list = [g.strip() for g in globs.split(",") if g.strip()]
# Validate activation
valid_activations = ["always", "glob", "manual", "ai_decision"]
if activation not in valid_activations:
return f"Error: activation must be one of {valid_activations}"
# Validate scope
if scope not in ["workspace", "global"]:
return "Error: scope must be 'workspace' or 'global'"
# Create the rule
rule = manager.create_rule(
name=name,
content=content,
description=description,
globs=glob_list,
activation=activation,
priority=priority,
scope=scope
)
return f"""Successfully created rule '{name}'
Location: {rule.file_path}
Activation: {activation}
Priority: {priority}
Description: {description or 'None'}
Globs: {glob_list or 'None'}
The rule will be applied according to its activation mode."""
except Exception as e:
logger.error(f"Error creating rule: {e}")
return f"Error creating rule: {str(e)}"
def list_rules_tool() -> str:
"""
List all available rules.
Returns a formatted list of all rules with their metadata.
Returns:
Formatted string with rule information
"""
try:
manager = _get_rules_manager()
rules = manager.get_all_rules()
if not rules:
return "No rules found. Use create_rule_tool to create new rules."
lines = ["# Available Rules\n"]
for rule in rules:
scope = "global" if "global:" in (rule.file_path or "") else "workspace"
lines.append(f"## {rule.name}")
lines.append(f"- **Activation**: {rule.activation}")
lines.append(f"- **Priority**: {rule.priority}")
if rule.description:
lines.append(f"- **Description**: {rule.description}")
if rule.globs:
lines.append(f"- **Globs**: {', '.join(rule.globs)}")
lines.append(f"- **Scope**: {scope}")
lines.append("")
return "\n".join(lines)
except Exception as e:
logger.error(f"Error listing rules: {e}")
return f"Error listing rules: {str(e)}"
def get_rule_tool(name: str) -> str:
"""
Get the content of a specific rule.
Args:
name: The name of the rule to retrieve
Returns:
The rule content and metadata
"""
try:
manager = _get_rules_manager()
rule = manager.get_rule(name)
if not rule:
return f"Rule '{name}' not found. Use list_rules_tool to see available rules."
return f"""# Rule: {rule.name}
**Description**: {rule.description or 'None'}
**Activation**: {rule.activation}
**Priority**: {rule.priority}
**Globs**: {', '.join(rule.globs) if rule.globs else 'None'}
**File**: {rule.file_path}
## Content:
{rule.content}
"""
except Exception as e:
logger.error(f"Error getting rule: {e}")
return f"Error getting rule: {str(e)}"
def delete_rule_tool(name: str, scope: Optional[str] = None) -> str:
"""
Delete a rule.
Args:
name: The name of the rule to delete
scope: Optional scope to delete from ("workspace" or "global")
Returns:
Success or error message
"""
try:
manager = _get_rules_manager()
if scope and scope not in ["workspace", "global"]:
return "Error: scope must be 'workspace' or 'global'"
success = manager.delete_rule(name, scope)
if success:
return f"Successfully deleted rule '{name}'"
else:
return f"Rule '{name}' not found or could not be deleted"
except Exception as e:
logger.error(f"Error deleting rule: {e}")
return f"Error deleting rule: {str(e)}"
def get_active_rules_tool(file_path: Optional[str] = None) -> str:
"""
Get all rules that are currently active for a given file or context.
Args:
file_path: Optional file path to check which rules apply
Returns:
Formatted string with active rules
"""
try:
manager = _get_rules_manager()
if file_path:
rules = manager.get_rules_for_file(file_path)
else:
rules = manager.get_active_rules()
if not rules:
return "No active rules for this context."
lines = ["# Active Rules\n"]
for rule in rules:
lines.append(f"## {rule.name}")
if rule.description:
lines.append(f"*{rule.description}*\n")
lines.append(rule.content)
lines.append("")
return "\n".join(lines)
except Exception as e:
logger.error(f"Error getting active rules: {e}")
return f"Error getting active rules: {str(e)}"
# Export all tools
__all__ = [
"create_rule_tool",
"list_rules_tool",
"get_rule_tool",
"delete_rule_tool",
"get_active_rules_tool"
]
@@ -0,0 +1,13 @@
"""
UI Integrations for PraisonAI Agents
This module provides UI protocol integrations for exposing PraisonAI Agents
via various frontend protocols.
Available integrations:
- agui: AG-UI (Agent-User Interface) protocol for CopilotKit and compatible frontends
"""
from praisonaiagents.ui.agui import AGUI
__all__ = ["AGUI"]
@@ -0,0 +1,22 @@
"""
AG-UI Protocol Integration for PraisonAI Agents
This module provides AG-UI (Agent-User Interface) protocol support,
enabling PraisonAI Agents to be exposed via a standardized streaming API
compatible with CopilotKit and other AG-UI frontends.
Usage:
from praisonaiagents import Agent
from praisonaiagents.ui.agui import AGUI
from fastapi import FastAPI
agent = Agent(name="Assistant", role="Helper", goal="Help users")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
"""
from praisonaiagents.ui.agui.agui import AGUI
__all__ = ["AGUI"]
@@ -0,0 +1,203 @@
"""
AGUI - Main AG-UI Interface Class
Exposes PraisonAI Agents via the AG-UI protocol.
"""
import logging
import uuid
from typing import AsyncIterator, List, Optional, TYPE_CHECKING
from praisonaiagents.ui.agui.types import (
BaseEvent,
RunAgentInput,
)
from praisonaiagents.ui.agui.conversion import (
agui_messages_to_praisonai,
validate_state,
extract_user_input,
)
from praisonaiagents.ui.agui.streaming import (
create_run_error_event,
async_stream_agent_response,
async_stream_agents_response,
)
from praisonaiagents.ui.agui.encoder import EventEncoder
if TYPE_CHECKING:
from praisonaiagents import Agent, PraisonAIAgents
from fastapi import APIRouter
logger = logging.getLogger(__name__)
class AGUI:
"""
AG-UI Interface for PraisonAI Agents.
Exposes a PraisonAI Agent or PraisonAIAgents workflow via the AG-UI protocol,
enabling integration with CopilotKit and other AG-UI compatible frontends.
Usage:
from praisonaiagents import Agent
from praisonaiagents.ui.agui import AGUI
from fastapi import FastAPI
agent = Agent(name="Assistant", role="Helper", goal="Help users")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
Args:
agent: Single PraisonAI Agent instance
agents: PraisonAIAgents instance for multi-agent workflows
name: Name for the AG-UI endpoint (defaults to agent name)
description: Description of the agent
prefix: URL prefix for the router (e.g., "/api/v1")
tags: OpenAPI tags for the router
"""
def __init__(
self,
agent: Optional["Agent"] = None,
agents: Optional["PraisonAIAgents"] = None,
name: Optional[str] = None,
description: Optional[str] = None,
prefix: str = "",
tags: Optional[List[str]] = None,
):
if agent is None and agents is None:
raise ValueError("AGUI requires an agent or agents instance")
self.agent = agent
self.agents = agents
self.prefix = prefix
self.tags = tags or ["AGUI"]
# Set name from agent if not provided
if name:
self.name = name
elif agent and hasattr(agent, 'name'):
self.name = agent.name
elif agents and hasattr(agents, 'name'):
self.name = agents.name
else:
self.name = "PraisonAI Agent"
# Set description
if description:
self.description = description
elif agent and hasattr(agent, 'role'):
self.description = agent.role
else:
self.description = "PraisonAI Agent via AG-UI"
self._router: Optional["APIRouter"] = None
def get_router(self) -> "APIRouter":
"""
Get the FastAPI router for this AGUI instance.
Returns:
FastAPI APIRouter with AG-UI endpoints
"""
from fastapi import APIRouter
from fastapi.responses import StreamingResponse
if self._router is not None:
return self._router
self._router = APIRouter(prefix=self.prefix, tags=self.tags)
# Attach routes
self._attach_routes(self._router)
return self._router
def _attach_routes(self, router: "APIRouter") -> None:
"""Attach AG-UI routes to the router."""
from fastapi.responses import StreamingResponse
encoder = EventEncoder()
@router.post("/agui")
async def run_agent_agui(run_input: RunAgentInput):
"""Run the agent via AG-UI protocol."""
async def event_generator():
async for event in self._run_agent(run_input):
yield encoder.encode(event)
return StreamingResponse(
event_generator(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Methods": "POST, GET, OPTIONS",
"Access-Control-Allow-Headers": "*",
},
)
@router.get("/status")
async def get_status():
"""Get agent status."""
return {"status": "available"}
async def _run_agent(self, run_input: RunAgentInput) -> AsyncIterator[BaseEvent]:
"""
Run the agent with the given input.
Args:
run_input: AG-UI run input
Yields:
AG-UI events
"""
run_id = run_input.run_id or str(uuid.uuid4())
thread_id = run_input.thread_id
try:
# Convert messages
messages = run_input.messages or []
# Extract user input
user_input = extract_user_input(messages)
# Validate state
session_state = validate_state(run_input.state, thread_id)
# Get user_id from forwarded_props if available
user_id = None
if run_input.forwarded_props and isinstance(run_input.forwarded_props, dict):
user_id = run_input.forwarded_props.get("user_id")
# Run appropriate handler
if self.agent:
async for event in async_stream_agent_response(
agent=self.agent,
user_input=user_input,
thread_id=thread_id,
run_id=run_id,
session_state=session_state,
messages=agui_messages_to_praisonai(messages),
):
yield event
elif self.agents:
async for event in async_stream_agents_response(
agents=self.agents,
user_input=user_input,
thread_id=thread_id,
run_id=run_id,
session_state=session_state,
):
yield event
else:
yield create_run_error_event("No agent or agents configured")
except Exception as e:
logger.error(f"Error running agent: {e}", exc_info=True)
yield create_run_error_event(str(e))
@@ -0,0 +1,211 @@
"""
AG-UI Message Conversion
Converts between AG-UI message format and PraisonAI message format.
"""
import logging
from typing import Any, Dict, List, Optional, Set
from pydantic import BaseModel
from praisonaiagents.ui.agui.types import Message, ToolCall, FunctionCall
logger = logging.getLogger(__name__)
def agui_messages_to_praisonai(messages: List[Message]) -> List[Dict[str, Any]]:
"""
Convert AG-UI messages to PraisonAI format.
Args:
messages: List of AG-UI Message objects
Returns:
List of message dictionaries in PraisonAI format
"""
if not messages:
return []
# Track tool_call_ids that have results to avoid duplicates
tool_call_ids_with_results: Set[str] = set()
for msg in messages:
if msg.role == "tool" and msg.tool_call_id:
tool_call_ids_with_results.add(msg.tool_call_id)
result: List[Dict[str, Any]] = []
seen_tool_call_ids: Set[str] = set()
for msg in messages:
# Skip system messages - agent builds its own
if msg.role == "system":
continue
if msg.role == "tool":
# Deduplicate tool results
if msg.tool_call_id in seen_tool_call_ids:
logger.debug(f"Skipping duplicate tool result: {msg.tool_call_id}")
continue
if msg.tool_call_id:
seen_tool_call_ids.add(msg.tool_call_id)
result.append({
"role": "tool",
"content": msg.content or "",
"tool_call_id": msg.tool_call_id,
})
elif msg.role == "assistant":
msg_dict: Dict[str, Any] = {
"role": "assistant",
"content": msg.content or "",
}
if msg.tool_calls:
# Include all tool_calls (filtering only when there are results to match)
tool_calls_list = [
{
"id": tc.id,
"function": {
"name": tc.function.name,
"arguments": tc.function.arguments,
},
"type": "function",
}
for tc in msg.tool_calls
]
if tool_calls_list:
msg_dict["tool_calls"] = tool_calls_list
result.append(msg_dict)
elif msg.role == "user":
result.append({
"role": "user",
"content": msg.content or "",
})
else:
logger.warning(f"Unknown message role: {msg.role}")
return result
def praisonai_messages_to_agui(messages: List[Dict[str, Any]]) -> List[Message]:
"""
Convert PraisonAI messages to AG-UI format.
Args:
messages: List of message dictionaries in PraisonAI format
Returns:
List of AG-UI Message objects
"""
if not messages:
return []
result: List[Message] = []
for msg in messages:
role = msg.get("role", "user")
content = msg.get("content", "")
if role == "tool":
result.append(Message(
role="tool",
content=content,
tool_call_id=msg.get("tool_call_id"),
))
elif role == "assistant":
tool_calls = None
if "tool_calls" in msg and msg["tool_calls"]:
tool_calls = [
ToolCall(
id=tc.get("id", ""),
function=FunctionCall(
name=tc.get("function", {}).get("name", ""),
arguments=tc.get("function", {}).get("arguments", "{}"),
),
)
for tc in msg["tool_calls"]
]
result.append(Message(
role="assistant",
content=content,
tool_calls=tool_calls,
))
elif role == "user":
result.append(Message(
role="user",
content=content,
))
elif role == "system":
result.append(Message(
role="system",
content=content,
))
return result
def validate_state(state: Any, thread_id: str) -> Optional[Dict[str, Any]]:
"""
Validate and convert state to a dictionary.
Args:
state: The state to validate (can be dict, Pydantic model, or other)
thread_id: Thread ID for logging
Returns:
State as a dictionary, or None if invalid
"""
if state is None:
return None
if isinstance(state, dict):
return state
if isinstance(state, BaseModel):
try:
return state.model_dump()
except Exception:
pass
# Try to_dict method
if hasattr(state, "to_dict") and callable(getattr(state, "to_dict")):
try:
result = state.to_dict()
if isinstance(result, dict):
return result
except Exception:
pass
logger.warning(
f"State must be a dict, got {type(state).__name__}. "
f"State will be ignored. Thread: {thread_id}"
)
return None
def extract_user_input(messages: List[Message]) -> str:
"""
Extract the last user message content.
Args:
messages: List of AG-UI messages
Returns:
Content of the last user message, or empty string
"""
if not messages:
return ""
for msg in reversed(messages):
if msg.role == "user" and msg.content:
return msg.content
return ""
@@ -0,0 +1,30 @@
"""
AG-UI Event Encoder
Encodes AG-UI events to Server-Sent Events (SSE) format.
"""
from praisonaiagents.ui.agui.types import BaseEvent
class EventEncoder:
"""Encodes AG-UI events to SSE format."""
def __init__(self):
pass
def get_content_type(self) -> str:
"""Returns the content type for SSE."""
return "text/event-stream"
def encode(self, event: BaseEvent) -> str:
"""
Encode an event to SSE format.
Args:
event: The event to encode
Returns:
SSE formatted string
"""
return f"data: {event.model_dump_json(by_alias=True, exclude_none=True)}\n\n"
@@ -0,0 +1,372 @@
"""
AG-UI Event Streaming
Provides utilities for streaming PraisonAI agent responses as AG-UI events.
"""
import uuid
from dataclasses import dataclass, field
from typing import Any, AsyncIterator, Dict, Iterator, List, Optional, Set
from praisonaiagents.ui.agui.types import (
BaseEvent,
TextMessageStartEvent,
TextMessageContentEvent,
TextMessageEndEvent,
ToolCallStartEvent,
ToolCallArgsEvent,
ToolCallEndEvent,
ToolCallResultEvent,
RunStartedEvent,
RunFinishedEvent,
RunErrorEvent,
StepStartedEvent,
StepFinishedEvent,
StateSnapshotEvent,
)
@dataclass
class EventBuffer:
"""Buffer to manage event ordering constraints."""
active_tool_call_ids: Set[str] = field(default_factory=set)
ended_tool_call_ids: Set[str] = field(default_factory=set)
current_text_message_id: str = ""
next_text_message_id: str = field(default_factory=lambda: str(uuid.uuid4()))
pending_tool_calls_parent_id: str = ""
def start_tool_call(self, tool_call_id: str) -> None:
"""Start a new tool call."""
self.active_tool_call_ids.add(tool_call_id)
def end_tool_call(self, tool_call_id: str) -> None:
"""End a tool call."""
self.active_tool_call_ids.discard(tool_call_id)
self.ended_tool_call_ids.add(tool_call_id)
def start_text_message(self) -> str:
"""Start a new text message and return its ID."""
self.current_text_message_id = self.next_text_message_id
self.next_text_message_id = str(uuid.uuid4())
return self.current_text_message_id
def get_parent_message_id_for_tool_call(self) -> str:
"""Get the message ID to use as parent for tool calls."""
if self.pending_tool_calls_parent_id:
return self.pending_tool_calls_parent_id
return self.current_text_message_id
# Event Creation Functions
def create_text_message_events(
content: str,
message_id: Optional[str] = None,
role: str = "assistant"
) -> Iterator[BaseEvent]:
"""
Create text message events for a complete message.
Args:
content: The message content
message_id: Optional message ID (generated if not provided)
role: Message role (default: assistant)
Yields:
TextMessageStartEvent, TextMessageContentEvent(s), TextMessageEndEvent
"""
msg_id = message_id or str(uuid.uuid4())
yield TextMessageStartEvent(message_id=msg_id, role=role)
if content:
yield TextMessageContentEvent(message_id=msg_id, delta=content)
yield TextMessageEndEvent(message_id=msg_id)
def stream_text_chunks(
chunks: Iterator[str],
message_id: Optional[str] = None,
role: str = "assistant"
) -> Iterator[BaseEvent]:
"""
Stream text chunks as AG-UI events.
Args:
chunks: Iterator of text chunks
message_id: Optional message ID
role: Message role
Yields:
AG-UI events for the text stream
"""
msg_id = message_id or str(uuid.uuid4())
yield TextMessageStartEvent(message_id=msg_id, role=role)
for chunk in chunks:
if chunk:
yield TextMessageContentEvent(message_id=msg_id, delta=chunk)
yield TextMessageEndEvent(message_id=msg_id)
def create_tool_call_events(
tool_call_id: str,
tool_name: str,
arguments: str,
parent_message_id: Optional[str] = None
) -> Iterator[BaseEvent]:
"""
Create tool call events.
Args:
tool_call_id: Unique ID for the tool call
tool_name: Name of the tool being called
arguments: JSON string of arguments
parent_message_id: Optional parent message ID
Yields:
ToolCallStartEvent, ToolCallArgsEvent, ToolCallEndEvent
"""
yield ToolCallStartEvent(
tool_call_id=tool_call_id,
tool_call_name=tool_name,
parent_message_id=parent_message_id
)
yield ToolCallArgsEvent(
tool_call_id=tool_call_id,
delta=arguments
)
yield ToolCallEndEvent(tool_call_id=tool_call_id)
def create_tool_result_event(
tool_call_id: str,
content: str,
message_id: Optional[str] = None
) -> ToolCallResultEvent:
"""
Create a tool result event.
Args:
tool_call_id: ID of the tool call this is a result for
content: Result content
message_id: Optional message ID
Returns:
ToolCallResultEvent
"""
return ToolCallResultEvent(
message_id=message_id or str(uuid.uuid4()),
tool_call_id=tool_call_id,
content=content
)
# Run Lifecycle Events
def create_run_started_event(thread_id: str, run_id: str) -> RunStartedEvent:
"""Create a run started event."""
return RunStartedEvent(thread_id=thread_id, run_id=run_id)
def create_run_finished_event(
thread_id: str,
run_id: str,
result: Optional[Any] = None
) -> RunFinishedEvent:
"""Create a run finished event."""
return RunFinishedEvent(thread_id=thread_id, run_id=run_id, result=result)
def create_run_error_event(
message: str,
code: Optional[str] = None
) -> RunErrorEvent:
"""Create a run error event."""
return RunErrorEvent(message=message, code=code)
# Step Events
def create_step_started_event(step_name: str) -> StepStartedEvent:
"""Create a step started event."""
return StepStartedEvent(step_name=step_name)
def create_step_finished_event(step_name: str) -> StepFinishedEvent:
"""Create a step finished event."""
return StepFinishedEvent(step_name=step_name)
# State Events
def create_state_snapshot_event(state: Dict[str, Any]) -> StateSnapshotEvent:
"""Create a state snapshot event."""
return StateSnapshotEvent(snapshot=state)
# Async Streaming
async def async_stream_response(
response_stream: AsyncIterator[str],
thread_id: str,
run_id: str
) -> AsyncIterator[BaseEvent]:
"""
Stream an async response as AG-UI events.
Args:
response_stream: Async iterator of response chunks
thread_id: Thread ID
run_id: Run ID
Yields:
AG-UI events
"""
# Emit run started
yield create_run_started_event(thread_id, run_id)
message_id = str(uuid.uuid4())
message_started = False
try:
async for chunk in response_stream:
if not message_started:
yield TextMessageStartEvent(message_id=message_id, role="assistant")
message_started = True
if chunk:
yield TextMessageContentEvent(message_id=message_id, delta=chunk)
if message_started:
yield TextMessageEndEvent(message_id=message_id)
# Emit run finished
yield create_run_finished_event(thread_id, run_id)
except Exception as e:
if message_started:
yield TextMessageEndEvent(message_id=message_id)
yield create_run_error_event(str(e))
async def async_stream_agent_response(
agent,
user_input: str,
thread_id: str,
run_id: str,
session_state: Optional[Dict[str, Any]] = None,
messages: Optional[List[Dict[str, Any]]] = None
) -> AsyncIterator[BaseEvent]:
"""
Stream an agent's response as AG-UI events.
Args:
agent: PraisonAI Agent instance
user_input: User input string
thread_id: Thread ID
run_id: Run ID
session_state: Optional session state
messages: Optional message history
Yields:
AG-UI events
"""
import asyncio
# Emit run started
yield create_run_started_event(thread_id, run_id)
message_id = str(uuid.uuid4())
try:
# Check if agent has async chat method
if hasattr(agent, 'achat'):
response = await agent.achat(user_input)
elif hasattr(agent, 'chat'):
# Run sync chat in executor
loop = asyncio.get_event_loop()
response = await loop.run_in_executor(None, agent.chat, user_input)
else:
raise ValueError("Agent must have 'chat' or 'achat' method")
# Emit text message events
yield TextMessageStartEvent(message_id=message_id, role="assistant")
if response:
yield TextMessageContentEvent(message_id=message_id, delta=str(response))
yield TextMessageEndEvent(message_id=message_id)
# Emit run finished
yield create_run_finished_event(thread_id, run_id)
except Exception as e:
yield create_run_error_event(str(e))
async def async_stream_agents_response(
agents,
user_input: str,
thread_id: str,
run_id: str,
session_state: Optional[Dict[str, Any]] = None
) -> AsyncIterator[BaseEvent]:
"""
Stream a PraisonAIAgents workflow response as AG-UI events.
Args:
agents: PraisonAIAgents instance
user_input: User input string
thread_id: Thread ID
run_id: Run ID
session_state: Optional session state
Yields:
AG-UI events
"""
import asyncio
# Emit run started
yield create_run_started_event(thread_id, run_id)
message_id = str(uuid.uuid4())
try:
# Check if agents has async start method
if hasattr(agents, 'astart'):
result = await agents.astart(user_input)
elif hasattr(agents, 'start'):
# Run sync start in executor
loop = asyncio.get_event_loop()
result = await loop.run_in_executor(None, agents.start, user_input)
else:
raise ValueError("Agents must have 'start' or 'astart' method")
# Emit text message events with result
yield TextMessageStartEvent(message_id=message_id, role="assistant")
if result:
# Handle different result types
if isinstance(result, dict):
content = result.get("output", str(result))
elif hasattr(result, "raw"):
content = result.raw
else:
content = str(result)
yield TextMessageContentEvent(message_id=message_id, delta=content)
yield TextMessageEndEvent(message_id=message_id)
# Emit run finished
yield create_run_finished_event(thread_id, run_id)
except Exception as e:
yield create_run_error_event(str(e))
@@ -0,0 +1,183 @@
"""
AG-UI Protocol Types
This module defines the event types and message types for the AG-UI protocol,
following the ag-ui-protocol specification.
"""
from enum import Enum
from typing import Any, Dict, List, Literal, Optional, Union
from pydantic import BaseModel, Field
class EventType(str, Enum):
"""AG-UI Event Types."""
TEXT_MESSAGE_START = "TEXT_MESSAGE_START"
TEXT_MESSAGE_CONTENT = "TEXT_MESSAGE_CONTENT"
TEXT_MESSAGE_END = "TEXT_MESSAGE_END"
TOOL_CALL_START = "TOOL_CALL_START"
TOOL_CALL_ARGS = "TOOL_CALL_ARGS"
TOOL_CALL_END = "TOOL_CALL_END"
TOOL_CALL_RESULT = "TOOL_CALL_RESULT"
STATE_SNAPSHOT = "STATE_SNAPSHOT"
STATE_DELTA = "STATE_DELTA"
RUN_STARTED = "RUN_STARTED"
RUN_FINISHED = "RUN_FINISHED"
RUN_ERROR = "RUN_ERROR"
STEP_STARTED = "STEP_STARTED"
STEP_FINISHED = "STEP_FINISHED"
CUSTOM = "CUSTOM"
class BaseEvent(BaseModel):
"""Base event for all AG-UI events."""
type: EventType
timestamp: Optional[int] = None
model_config = {"use_enum_values": True}
# Text Message Events
class TextMessageStartEvent(BaseEvent):
"""Event indicating the start of a text message."""
type: Literal["TEXT_MESSAGE_START"] = "TEXT_MESSAGE_START"
message_id: str
role: str = "assistant"
class TextMessageContentEvent(BaseEvent):
"""Event containing text message content."""
type: Literal["TEXT_MESSAGE_CONTENT"] = "TEXT_MESSAGE_CONTENT"
message_id: str
delta: str
class TextMessageEndEvent(BaseEvent):
"""Event indicating the end of a text message."""
type: Literal["TEXT_MESSAGE_END"] = "TEXT_MESSAGE_END"
message_id: str
# Tool Call Events
class ToolCallStartEvent(BaseEvent):
"""Event indicating the start of a tool call."""
type: Literal["TOOL_CALL_START"] = "TOOL_CALL_START"
tool_call_id: str
tool_call_name: str
parent_message_id: Optional[str] = None
class ToolCallArgsEvent(BaseEvent):
"""Event containing tool call arguments."""
type: Literal["TOOL_CALL_ARGS"] = "TOOL_CALL_ARGS"
tool_call_id: str
delta: str
class ToolCallEndEvent(BaseEvent):
"""Event indicating the end of a tool call."""
type: Literal["TOOL_CALL_END"] = "TOOL_CALL_END"
tool_call_id: str
class ToolCallResultEvent(BaseEvent):
"""Event containing tool call result."""
type: Literal["TOOL_CALL_RESULT"] = "TOOL_CALL_RESULT"
message_id: str
tool_call_id: str
content: str
role: Optional[Literal["tool"]] = "tool"
# Run Lifecycle Events
class RunStartedEvent(BaseEvent):
"""Event indicating a run has started."""
type: Literal["RUN_STARTED"] = "RUN_STARTED"
thread_id: str
run_id: str
parent_run_id: Optional[str] = None
class RunFinishedEvent(BaseEvent):
"""Event indicating a run has finished."""
type: Literal["RUN_FINISHED"] = "RUN_FINISHED"
thread_id: str
run_id: str
result: Optional[Any] = None
class RunErrorEvent(BaseEvent):
"""Event indicating a run error."""
type: Literal["RUN_ERROR"] = "RUN_ERROR"
message: str
code: Optional[str] = None
# Step Events
class StepStartedEvent(BaseEvent):
"""Event indicating a step has started."""
type: Literal["STEP_STARTED"] = "STEP_STARTED"
step_name: str
class StepFinishedEvent(BaseEvent):
"""Event indicating a step has finished."""
type: Literal["STEP_FINISHED"] = "STEP_FINISHED"
step_name: str
# State Events
class StateSnapshotEvent(BaseEvent):
"""Event containing a state snapshot."""
type: Literal["STATE_SNAPSHOT"] = "STATE_SNAPSHOT"
snapshot: Dict[str, Any]
# Message Types
class FunctionCall(BaseModel):
"""Function call in a tool call."""
name: str
arguments: str
class ToolCall(BaseModel):
"""Tool call in a message."""
id: str
function: FunctionCall
type: str = "function"
class Message(BaseModel):
"""AG-UI Message type."""
role: Literal["user", "assistant", "system", "tool"]
content: Optional[str] = None
tool_calls: Optional[List[ToolCall]] = None
tool_call_id: Optional[str] = None
name: Optional[str] = None
class RunAgentInput(BaseModel):
"""Input for running an agent."""
thread_id: str
run_id: Optional[str] = None
messages: Optional[List[Message]] = Field(default_factory=list)
state: Optional[Dict[str, Any]] = None
forwarded_props: Optional[Dict[str, Any]] = None
# Union type for all events
Event = Union[
TextMessageStartEvent,
TextMessageContentEvent,
TextMessageEndEvent,
ToolCallStartEvent,
ToolCallArgsEvent,
ToolCallEndEvent,
ToolCallResultEvent,
StateSnapshotEvent,
RunStartedEvent,
RunFinishedEvent,
RunErrorEvent,
StepStartedEvent,
StepFinishedEvent,
]
@@ -0,0 +1,60 @@
"""
Workflows module for PraisonAI Agents.
Provides workflow/pipeline patterns for orchestrating agents and functions.
"""
from .workflows import (
# Core classes
Workflow,
Pipeline, # Alias for Workflow
WorkflowStep,
WorkflowContext,
StepResult,
WorkflowManager,
# Pattern helpers
Route,
Parallel,
Loop,
Repeat,
# Convenience functions
route,
parallel,
loop,
repeat,
# Backward compatibility aliases
StepInput,
StepOutput,
)
from .yaml_parser import YAMLWorkflowParser
__all__ = [
# Core
"Workflow",
"Pipeline",
"WorkflowStep",
"WorkflowContext",
"StepResult",
"WorkflowManager",
# YAML Parser
"YAMLWorkflowParser",
# Patterns
"Route",
"Parallel",
"Loop",
"Repeat",
"route",
"parallel",
"loop",
"repeat",
# Aliases
"StepInput",
"StepOutput",
]
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,752 @@
"""
YAML Workflow Parser for PraisonAI Agents.
Parses YAML workflow files into Workflow objects with support for:
- Agent definitions with planning, reasoning, tools
- Workflow patterns: route(), parallel(), loop(), repeat()
- Variables and template substitution
- Callbacks and guardrails
- Memory configuration
"""
import yaml
from typing import Any, Callable, Dict, List, Optional, Union
from pathlib import Path
from ..agent.agent import Agent
from .workflows import Workflow, WorkflowStep, route, parallel, loop, repeat
class YAMLWorkflowParser:
"""
Parser for YAML workflow files.
Converts YAML workflow definitions into Workflow objects that can be executed.
Example YAML format:
```yaml
name: My Workflow
description: A multi-agent workflow
workflow:
planning: true
reasoning: true
verbose: true
memory_config:
provider: chroma
persist: true
variables:
topic: AI trends
agents:
researcher:
name: Researcher
role: Research Analyst
goal: Research topics
instructions: "Provide research findings"
tools:
- tavily_search
steps:
- agent: researcher
action: "Research {{topic}}"
- name: routing
route:
technical: [tech_agent]
default: [general_agent]
callbacks:
on_step_complete: log_step
```
"""
def __init__(self, tool_registry: Optional[Dict[str, Callable]] = None):
"""
Initialize the YAML workflow parser.
Args:
tool_registry: Optional dictionary mapping tool names to callable functions
"""
self.tool_registry = tool_registry or {}
self._agents: Dict[str, Agent] = {}
self._callbacks: Dict[str, Callable] = {}
def parse_file(self, file_path: Union[str, Path]) -> Workflow:
"""
Parse a YAML workflow file.
Args:
file_path: Path to the YAML file
Returns:
Workflow object ready for execution
"""
file_path = Path(file_path)
if not file_path.exists():
raise FileNotFoundError(f"Workflow file not found: {file_path}")
with open(file_path, 'r') as f:
yaml_content = f.read()
return self.parse_string(yaml_content)
def parse_string(self, yaml_content: str) -> Workflow:
"""
Parse a YAML workflow string.
Args:
yaml_content: YAML content as string
Returns:
Workflow object ready for execution
"""
data = yaml.safe_load(yaml_content)
# Normalize to canonical format (accept both old and new field names)
normalized_data = self._normalize_yaml_config(data)
return self._parse_workflow_data(normalized_data)
def _normalize_yaml_config(self, data: Dict[str, Any]) -> Dict[str, Any]:
"""
Normalize YAML config to canonical format.
Accepts both old (agents.yaml) and new (workflow.yaml) field names.
This implements "Accept liberally, suggest canonically" principle.
Canonical names (preferred):
- agents (not roles)
- instructions (not backstory)
- action (not description)
- steps (not tasks)
- name (not topic)
Args:
data: Raw parsed YAML data
Returns:
Normalized data with canonical field names
"""
if not data:
return data
normalized = data.copy()
# 1. Normalize workflow name: topic -> name (only if topic is used as name, not as input)
# Note: 'topic' is legacy, 'input' is canonical for workflow input data
if 'topic' in normalized and 'name' not in normalized and 'input' not in normalized:
# Legacy behavior: topic was used as both name and input
normalized['name'] = normalized.get('topic', 'Unnamed Workflow')
# 2. Normalize workflow input: 'input' is canonical, 'topic' is alias
# Priority: input > topic (input takes precedence if both exist)
if 'input' not in normalized and 'topic' in normalized:
normalized['input'] = normalized['topic']
# 3. Normalize container: roles -> agents
# Note: _convert_roles_to_agents handles backstory -> instructions
if 'roles' in normalized and 'agents' not in normalized:
normalized['agents'] = self._convert_roles_to_agents(normalized['roles'])
# Extract steps from tasks nested in roles if no steps defined
if 'steps' not in normalized:
normalized['steps'] = self._extract_steps_from_roles(normalized['roles'])
# 4. Normalize agent fields: backstory -> instructions (for agents section)
if 'agents' in normalized:
for agent_id, agent_config in normalized['agents'].items():
if isinstance(agent_config, dict):
# backstory -> instructions
if 'backstory' in agent_config and 'instructions' not in agent_config:
agent_config['instructions'] = agent_config['backstory']
# 5. Normalize step fields: description -> action
if 'steps' in normalized:
for step in normalized['steps']:
if isinstance(step, dict):
# description -> action
if 'description' in step and 'action' not in step:
step['action'] = step['description']
# Handle parallel steps
if 'parallel' in step:
for parallel_step in step['parallel']:
if isinstance(parallel_step, dict):
if 'description' in parallel_step and 'action' not in parallel_step:
parallel_step['action'] = parallel_step['description']
return normalized
def _extract_steps_from_roles(self, roles: Dict[str, Dict]) -> List[Dict]:
"""
Extract steps from tasks nested within roles (agents.yaml format).
Args:
roles: The roles section from agents.yaml
Returns:
List of step definitions
"""
steps = []
for role_id, role_config in roles.items():
if 'tasks' in role_config:
for task_id, task_config in role_config['tasks'].items():
step = {
'name': task_id,
'agent': role_id,
}
# description -> action
if 'description' in task_config:
step['action'] = task_config['description']
elif 'action' in task_config:
step['action'] = task_config['action']
# Copy other task fields
for field in ['expected_output', 'context', 'output_file',
'output_json', 'create_directory', 'callback',
'async_execution', 'guardrail', 'max_retries']:
if field in task_config:
step[field] = task_config[field]
steps.append(step)
return steps
def _parse_workflow_data(self, data: Dict[str, Any]) -> Workflow:
"""
Parse workflow data dictionary into a Workflow object.
Args:
data: Parsed YAML data dictionary
Returns:
Workflow object
"""
# Extract workflow metadata
name = data.get('name', 'Unnamed Workflow')
description = data.get('description', '')
# Extract framework and process (for feature parity with agents.yaml)
framework = data.get('framework', 'praisonai')
process = data.get('process', 'sequential')
manager_llm = data.get('manager_llm')
# Parse workflow configuration
workflow_config = data.get('workflow', {})
planning = workflow_config.get('planning', False)
planning_llm = workflow_config.get('planning_llm')
default_llm = workflow_config.get('default_llm')
reasoning = workflow_config.get('reasoning', False)
verbose = workflow_config.get('verbose', False)
memory_config = workflow_config.get('memory_config')
# Parse variables
variables = data.get('variables', {})
# Parse agents - support both 'agents' and 'roles' keys for backward compatibility
agents_data = data.get('agents', {})
if not agents_data and 'roles' in data:
# Convert roles format to agents format
agents_data = self._convert_roles_to_agents(data['roles'])
self._agents = self._parse_agents(agents_data)
# Parse callbacks
callbacks_data = data.get('callbacks', {})
self._parse_callbacks(callbacks_data)
# Parse steps
steps_data = data.get('steps', [])
steps = self._parse_steps(steps_data)
# Create workflow
workflow = Workflow(
name=name,
steps=steps,
variables=variables,
planning=planning,
planning_llm=planning_llm,
default_llm=default_llm,
reasoning=reasoning,
verbose=verbose,
memory_config=memory_config,
on_workflow_start=self._callbacks.get('on_workflow_start'),
on_step_complete=self._callbacks.get('on_step_complete'),
on_workflow_complete=self._callbacks.get('on_workflow_complete'),
)
# Store additional attributes for feature parity with agents.yaml
workflow.description = description
workflow.framework = framework
workflow.process = process
workflow.manager_llm = manager_llm
# Store workflow input (from 'input' or 'topic' field)
# This is the default input passed to workflow.start() if no input is provided
workflow.default_input = data.get('input', data.get('topic', ''))
return workflow
def _convert_roles_to_agents(self, roles: Dict[str, Dict]) -> Dict[str, Dict]:
"""
Convert agents.yaml 'roles' format to workflow 'agents' format.
This enables backward compatibility with the existing agents.yaml format.
Mapping:
- backstory -> instructions
- role -> role
- goal -> goal
- tools -> tools
- llm -> llm
- tasks -> (used to auto-generate steps if no steps defined)
Args:
roles: The roles section from agents.yaml
Returns:
Converted agents dictionary in workflow format
"""
agents = {}
for role_id, role_config in roles.items():
agent = {
'name': role_config.get('role', role_id),
'role': role_config.get('role', role_id),
'goal': role_config.get('goal', ''),
'instructions': role_config.get('backstory', ''),
}
# Copy optional fields
if 'llm' in role_config:
llm_config = role_config['llm']
if isinstance(llm_config, dict):
agent['llm'] = llm_config.get('model', 'gpt-4o-mini')
else:
agent['llm'] = llm_config
if 'tools' in role_config:
agent['tools'] = [t for t in role_config['tools'] if t]
if 'verbose' in role_config:
agent['verbose'] = role_config['verbose']
if 'max_iter' in role_config:
agent['max_iter'] = role_config['max_iter']
if 'planning' in role_config:
agent['planning'] = role_config['planning']
if 'reasoning' in role_config:
agent['reasoning'] = role_config['reasoning']
agents[role_id] = agent
return agents
def _parse_agents(self, agents_data: Dict[str, Dict]) -> Dict[str, Agent]:
"""
Parse agent definitions from YAML.
Args:
agents_data: Dictionary of agent definitions
Returns:
Dictionary mapping agent names to Agent objects
"""
agents = {}
for agent_id, agent_config in agents_data.items():
agent = self._create_agent(agent_id, agent_config)
agents[agent_id] = agent
return agents
def _create_agent(self, agent_id: str, config: Dict[str, Any]) -> Agent:
"""
Create an Agent from configuration.
Args:
agent_id: Identifier for the agent
config: Agent configuration dictionary
Returns:
Agent object
"""
# Extract agent parameters
name = config.get('name', agent_id)
role = config.get('role', 'Assistant')
goal = config.get('goal', '')
instructions = config.get('instructions', '')
backstory = config.get('backstory', '')
# LLM configuration
llm = config.get('llm')
# Tools
tools_config = config.get('tools', [])
tools = self._resolve_tools(tools_config)
# Advanced parameters (feature parity with agents.yaml)
planning = config.get('planning', False)
reasoning = config.get('reasoning', False)
verbose = config.get('verbose', False)
allow_delegation = config.get('allow_delegation', False)
max_iter = config.get('max_iter')
cache = config.get('cache', True)
# Additional agents.yaml fields
function_calling_llm = config.get('function_calling_llm')
max_rpm = config.get('max_rpm')
max_execution_time = config.get('max_execution_time')
reflect_llm = config.get('reflect_llm')
min_reflect = config.get('min_reflect')
max_reflect = config.get('max_reflect')
system_template = config.get('system_template')
prompt_template = config.get('prompt_template')
response_template = config.get('response_template')
# Create agent
agent = Agent(
name=name,
role=role,
goal=goal,
instructions=instructions,
backstory=backstory,
llm=llm,
tools=tools if tools else None,
verbose=verbose,
)
# Store additional attributes for later use
agent._yaml_planning = planning
agent._yaml_reasoning = reasoning
agent._yaml_allow_delegation = allow_delegation
agent._yaml_max_iter = max_iter
agent._yaml_cache = cache
# Store additional agents.yaml fields for feature parity
agent._yaml_function_calling_llm = function_calling_llm
agent._yaml_max_rpm = max_rpm
agent._yaml_max_execution_time = max_execution_time
agent._yaml_reflect_llm = reflect_llm
agent._yaml_min_reflect = min_reflect
agent._yaml_max_reflect = max_reflect
agent._yaml_system_template = system_template
agent._yaml_prompt_template = prompt_template
agent._yaml_response_template = response_template
return agent
def _resolve_tools(self, tools_config: List[str]) -> List[Callable]:
"""
Resolve tool names to callable functions.
Args:
tools_config: List of tool names
Returns:
List of callable tool functions
"""
tools = []
for tool_name in tools_config:
if tool_name in self.tool_registry:
tools.append(self.tool_registry[tool_name])
# If tool not in registry, skip it (will be resolved later or is a string reference)
return tools
def _parse_callbacks(self, callbacks_data: Dict[str, str]) -> None:
"""
Parse callback definitions.
Args:
callbacks_data: Dictionary of callback name to function name
"""
# For now, store callback names - they can be resolved later
# In a full implementation, these would be resolved to actual functions
for callback_name, func_name in callbacks_data.items():
# Store as None for now - actual resolution happens at runtime
self._callbacks[callback_name] = None
def _parse_steps(self, steps_data: List[Dict]) -> List:
"""
Parse workflow steps from YAML.
Args:
steps_data: List of step definitions
Returns:
List of workflow steps (Agent, WorkflowStep, or pattern objects)
"""
steps = []
for step_data in steps_data:
step = self._parse_single_step(step_data)
if step is not None:
steps.append(step)
return steps
def _parse_single_step(self, step_data: Dict) -> Any:
"""
Parse a single step definition.
Args:
step_data: Step definition dictionary
Returns:
Step object (Agent, WorkflowStep, or pattern)
"""
# Check for pattern types
if 'route' in step_data:
return self._parse_route_step(step_data)
elif 'parallel' in step_data:
return self._parse_parallel_step(step_data)
elif 'loop' in step_data:
return self._parse_loop_step(step_data)
elif 'repeat' in step_data:
return self._parse_repeat_step(step_data)
elif 'agent' in step_data:
return self._parse_agent_step(step_data)
else:
# Generic step
return self._parse_generic_step(step_data)
def _parse_agent_step(self, step_data: Dict) -> Agent:
"""
Parse an agent step.
Args:
step_data: Step definition with 'agent' key
Returns:
Agent object
"""
agent_id = step_data['agent']
if agent_id in self._agents:
agent = self._agents[agent_id]
# Store step name if provided
step_name = step_data.get('name')
if step_name:
agent._yaml_step_name = step_name
# If there's an action, we need to handle it
action = step_data.get('action')
if action:
# Store action as task for the agent
agent._yaml_action = action
# Handle guardrail
guardrail = step_data.get('guardrail')
if guardrail:
agent._yaml_guardrail = guardrail
# Handle max_retries
max_retries = step_data.get('max_retries')
if max_retries:
agent._yaml_max_retries = max_retries
# Handle expected_output (feature parity with agents.yaml tasks)
expected_output = step_data.get('expected_output')
if expected_output:
agent._yaml_expected_output = expected_output
# Handle context/dependencies (feature parity with agents.yaml tasks)
context = step_data.get('context')
if context:
agent._yaml_context = context
# Handle output_file
output_file = step_data.get('output_file')
if output_file:
agent._yaml_output_file = output_file
# Handle output_json (structured output)
output_json = step_data.get('output_json')
if output_json:
agent._yaml_output_json = output_json
# Handle output_pydantic
output_pydantic = step_data.get('output_pydantic')
if output_pydantic:
agent._yaml_output_pydantic = output_pydantic
# Handle create_directory
create_directory = step_data.get('create_directory')
if create_directory is not None:
agent._yaml_create_directory = create_directory
# Handle callback
callback = step_data.get('callback')
if callback:
agent._yaml_callback = callback
# Handle async_execution
async_execution = step_data.get('async_execution')
if async_execution is not None:
agent._yaml_async_execution = async_execution
return agent
else:
raise ValueError(f"Agent '{agent_id}' not defined in agents section")
def _parse_route_step(self, step_data: Dict) -> Dict:
"""
Parse a route pattern step.
Args:
step_data: Step definition with 'route' key
Returns:
Route pattern object
"""
route_config = step_data['route']
# Build routes dictionary with resolved agents
routes = {}
for route_key, agent_ids in route_config.items():
if isinstance(agent_ids, list):
routes[route_key] = [self._agents[aid] for aid in agent_ids if aid in self._agents]
else:
if agent_ids in self._agents:
routes[route_key] = [self._agents[agent_ids]]
return route(routes)
def _parse_parallel_step(self, step_data: Dict) -> Dict:
"""
Parse a parallel pattern step.
Args:
step_data: Step definition with 'parallel' key
Returns:
Parallel pattern object
"""
parallel_config = step_data['parallel']
# Build list of parallel steps
parallel_steps = []
for item in parallel_config:
if 'agent' in item:
agent_id = item['agent']
if agent_id in self._agents:
agent = self._agents[agent_id]
# Store action if present
if 'action' in item:
agent._yaml_action = item['action']
parallel_steps.append(agent)
return parallel(parallel_steps)
def _parse_loop_step(self, step_data: Dict) -> Dict:
"""
Parse a loop pattern step.
Args:
step_data: Step definition with 'loop' key
Returns:
Loop pattern object
"""
loop_config = step_data['loop']
agent_id = step_data.get('agent')
if agent_id and agent_id in self._agents:
agent = self._agents[agent_id]
# Store action if present
if 'action' in step_data:
agent._yaml_action = step_data['action']
# Get loop parameters
over = loop_config.get('over')
from_csv = loop_config.get('from_csv')
return loop(agent, over=over, from_csv=from_csv)
raise ValueError("Loop step requires an agent")
def _parse_repeat_step(self, step_data: Dict) -> Dict:
"""
Parse a repeat pattern step.
Args:
step_data: Step definition with 'repeat' key
Returns:
Repeat pattern object
"""
repeat_config = step_data['repeat']
agent_id = step_data.get('agent')
if agent_id and agent_id in self._agents:
agent = self._agents[agent_id]
# Store action if present
if 'action' in step_data:
agent._yaml_action = step_data['action']
# Get repeat parameters
until = repeat_config.get('until')
max_iterations = repeat_config.get('max_iterations', 5)
# Create condition function from 'until' string
if isinstance(until, str):
condition = self._create_condition_from_string(until)
else:
condition = until
return repeat(agent, until=condition, max_iterations=max_iterations)
raise ValueError("Repeat step requires an agent")
def _create_condition_from_string(self, condition_str: str) -> Callable:
"""
Create a condition function from a string.
Args:
condition_str: Condition string (e.g., "approved", "comprehensive")
Returns:
Callable condition function
"""
def condition(ctx) -> bool:
result = str(ctx.previous_result).lower()
return condition_str.lower() in result
return condition
def _parse_generic_step(self, step_data: Dict) -> WorkflowStep:
"""
Parse a generic workflow step.
Args:
step_data: Step definition dictionary
Returns:
WorkflowStep object
"""
name = step_data.get('name', 'step')
action = step_data.get('action', '')
return WorkflowStep(
name=name,
action=action,
)
def register_tool(self, name: str, tool: Callable) -> None:
"""
Register a tool for use in workflows.
Args:
name: Tool name as referenced in YAML
tool: Callable tool function
"""
self.tool_registry[name] = tool
def register_callback(self, name: str, callback: Callable) -> None:
"""
Register a callback function.
Args:
name: Callback name as referenced in YAML
callback: Callable callback function
"""
self._callbacks[name] = callback
+2 -1
View File
@@ -4,8 +4,9 @@ build-backend = "setuptools.build_meta"
[project]
name = "praisonaiagents"
version = "0.0.191"
version = "0.1.15"
description = "Praison AI agents for completing complex tasks with Self Reflection Agents"
readme = "README.md"
requires-python = ">=3.10"
authors = [
{ name="Mervin Praison" }
@@ -0,0 +1 @@
"""Tests for AG-UI integration."""
@@ -0,0 +1,188 @@
"""
Test AGUI Class - TDD Tests for Main Interface Class
Phase 5: AGUI Class Tests
- Test AGUI initialization with Agent
- Test AGUI initialization with PraisonAIAgents
- Test get_router returns FastAPI router
- Test validation errors
"""
class TestAGUIInitialization:
"""Test AGUI class initialization."""
def test_agui_init_with_agent(self):
"""Test AGUI initialization with single Agent."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
assert agui.agent == agent
assert agui.agents is None
def test_agui_init_with_agents(self):
"""Test AGUI initialization with PraisonAIAgents."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent, Task, PraisonAIAgents
agent = Agent(name="Test", role="Tester", goal="Test things")
task = Task(description="Test task", expected_output="Result", agent=agent)
agents = PraisonAIAgents(agents=[agent], tasks=[task])
agui = AGUI(agents=agents)
assert agui.agents == agents
assert agui.agent is None
def test_agui_requires_agent_or_agents(self):
"""Test AGUI raises error if neither agent nor agents provided."""
from praisonaiagents.ui.agui import AGUI
import pytest
with pytest.raises(ValueError, match="requires an agent or agents"):
AGUI()
def test_agui_custom_prefix(self):
"""Test AGUI with custom prefix."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent, prefix="/api/v1")
assert agui.prefix == "/api/v1"
def test_agui_custom_tags(self):
"""Test AGUI with custom tags."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent, tags=["Custom", "Tags"])
assert agui.tags == ["Custom", "Tags"]
def test_agui_default_tags(self):
"""Test AGUI default tags."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
assert agui.tags == ["AGUI"]
class TestAGUIRouter:
"""Test AGUI router generation."""
def test_get_router_returns_api_router(self):
"""Test get_router returns FastAPI APIRouter."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import APIRouter
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
router = agui.get_router()
assert isinstance(router, APIRouter)
def test_router_has_agui_endpoint(self):
"""Test router has /agui POST endpoint."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
router = agui.get_router()
# Check routes
route_paths = [route.path for route in router.routes]
assert "/agui" in route_paths
def test_router_has_status_endpoint(self):
"""Test router has /status GET endpoint."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
router = agui.get_router()
# Check routes
route_paths = [route.path for route in router.routes]
assert "/status" in route_paths
def test_router_with_prefix(self):
"""Test router respects prefix."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent, prefix="/api/v1")
router = agui.get_router()
assert router.prefix == "/api/v1"
class TestAGUIIntegration:
"""Test AGUI integration with FastAPI."""
def test_include_router_in_fastapi(self):
"""Test including AGUI router in FastAPI app."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
# Check app has routes
route_paths = [route.path for route in app.routes]
assert "/agui" in route_paths
assert "/status" in route_paths
class TestAGUIName:
"""Test AGUI name and description."""
def test_agui_name(self):
"""Test AGUI name property."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent, name="My Agent")
assert agui.name == "My Agent"
def test_agui_description(self):
"""Test AGUI description property."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent, description="A test agent")
assert agui.description == "A test agent"
def test_agui_default_name_from_agent(self):
"""Test AGUI uses agent name by default."""
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="TestAgent", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
assert agui.name == "TestAgent"
@@ -0,0 +1,286 @@
"""
Test AG-UI Message Conversion - TDD Tests for Message Format Conversion
Phase 3: Message Conversion Tests
- Test converting AG-UI messages to PraisonAI format
- Test converting PraisonAI messages to AG-UI format
- Test handling of tool calls
- Test handling of tool results
- Test edge cases (empty messages, missing fields)
"""
class TestAGUIToPraisonAIConversion:
"""Test converting AG-UI messages to PraisonAI format."""
def test_convert_user_message(self):
"""Test converting a user message."""
from praisonaiagents.ui.agui.conversion import agui_messages_to_praisonai
from praisonaiagents.ui.agui.types import Message
agui_messages = [Message(role="user", content="Hello")]
result = agui_messages_to_praisonai(agui_messages)
assert len(result) == 1
assert result[0]["role"] == "user"
assert result[0]["content"] == "Hello"
def test_convert_assistant_message(self):
"""Test converting an assistant message."""
from praisonaiagents.ui.agui.conversion import agui_messages_to_praisonai
from praisonaiagents.ui.agui.types import Message
agui_messages = [Message(role="assistant", content="Hi there!")]
result = agui_messages_to_praisonai(agui_messages)
assert len(result) == 1
assert result[0]["role"] == "assistant"
assert result[0]["content"] == "Hi there!"
def test_convert_tool_message(self):
"""Test converting a tool result message."""
from praisonaiagents.ui.agui.conversion import agui_messages_to_praisonai
from praisonaiagents.ui.agui.types import Message
agui_messages = [Message(role="tool", content="Search results", tool_call_id="tc-123")]
result = agui_messages_to_praisonai(agui_messages)
assert len(result) == 1
assert result[0]["role"] == "tool"
assert result[0]["content"] == "Search results"
assert result[0]["tool_call_id"] == "tc-123"
def test_convert_assistant_with_tool_calls(self):
"""Test converting assistant message with tool calls."""
from praisonaiagents.ui.agui.conversion import agui_messages_to_praisonai
from praisonaiagents.ui.agui.types import Message, ToolCall, FunctionCall
agui_messages = [
Message(
role="assistant",
content="",
tool_calls=[
ToolCall(
id="tc-123",
function=FunctionCall(name="search", arguments='{"query": "test"}')
)
]
)
]
result = agui_messages_to_praisonai(agui_messages)
assert len(result) == 1
assert result[0]["role"] == "assistant"
assert "tool_calls" in result[0]
assert len(result[0]["tool_calls"]) == 1
assert result[0]["tool_calls"][0]["function"]["name"] == "search"
def test_convert_multiple_messages(self):
"""Test converting multiple messages."""
from praisonaiagents.ui.agui.conversion import agui_messages_to_praisonai
from praisonaiagents.ui.agui.types import Message
agui_messages = [
Message(role="user", content="Hello"),
Message(role="assistant", content="Hi!"),
Message(role="user", content="How are you?"),
]
result = agui_messages_to_praisonai(agui_messages)
assert len(result) == 3
assert result[0]["role"] == "user"
assert result[1]["role"] == "assistant"
assert result[2]["role"] == "user"
def test_convert_empty_messages(self):
"""Test converting empty message list."""
from praisonaiagents.ui.agui.conversion import agui_messages_to_praisonai
result = agui_messages_to_praisonai([])
assert result == []
def test_skip_system_messages(self):
"""Test that system messages are skipped (agent builds its own)."""
from praisonaiagents.ui.agui.conversion import agui_messages_to_praisonai
from praisonaiagents.ui.agui.types import Message
agui_messages = [
Message(role="system", content="You are helpful"),
Message(role="user", content="Hello"),
]
result = agui_messages_to_praisonai(agui_messages)
# System message should be skipped
assert len(result) == 1
assert result[0]["role"] == "user"
class TestPraisonAIToAGUIConversion:
"""Test converting PraisonAI messages to AG-UI format."""
def test_convert_user_message(self):
"""Test converting a user message."""
from praisonaiagents.ui.agui.conversion import praisonai_messages_to_agui
praison_messages = [{"role": "user", "content": "Hello"}]
result = praisonai_messages_to_agui(praison_messages)
assert len(result) == 1
assert result[0].role == "user"
assert result[0].content == "Hello"
def test_convert_assistant_message(self):
"""Test converting an assistant message."""
from praisonaiagents.ui.agui.conversion import praisonai_messages_to_agui
praison_messages = [{"role": "assistant", "content": "Hi there!"}]
result = praisonai_messages_to_agui(praison_messages)
assert len(result) == 1
assert result[0].role == "assistant"
assert result[0].content == "Hi there!"
def test_convert_tool_result(self):
"""Test converting a tool result message."""
from praisonaiagents.ui.agui.conversion import praisonai_messages_to_agui
praison_messages = [
{"role": "tool", "content": "Search results", "tool_call_id": "tc-123"}
]
result = praisonai_messages_to_agui(praison_messages)
assert len(result) == 1
assert result[0].role == "tool"
assert result[0].content == "Search results"
assert result[0].tool_call_id == "tc-123"
def test_convert_assistant_with_tool_calls(self):
"""Test converting assistant message with tool calls."""
from praisonaiagents.ui.agui.conversion import praisonai_messages_to_agui
praison_messages = [
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "tc-123",
"function": {
"name": "search",
"arguments": '{"query": "test"}'
}
}
]
}
]
result = praisonai_messages_to_agui(praison_messages)
assert len(result) == 1
assert result[0].role == "assistant"
assert result[0].tool_calls is not None
assert len(result[0].tool_calls) == 1
assert result[0].tool_calls[0].function.name == "search"
def test_convert_empty_messages(self):
"""Test converting empty message list."""
from praisonaiagents.ui.agui.conversion import praisonai_messages_to_agui
result = praisonai_messages_to_agui([])
assert result == []
class TestStateConversion:
"""Test state conversion utilities."""
def test_validate_dict_state(self):
"""Test validating dict state."""
from praisonaiagents.ui.agui.conversion import validate_state
state = {"key": "value"}
result = validate_state(state, "thread-123")
assert result == {"key": "value"}
def test_validate_none_state(self):
"""Test validating None state."""
from praisonaiagents.ui.agui.conversion import validate_state
result = validate_state(None, "thread-123")
assert result is None
def test_validate_pydantic_state(self):
"""Test validating Pydantic model state."""
from praisonaiagents.ui.agui.conversion import validate_state
from pydantic import BaseModel
class TestState(BaseModel):
key: str = "value"
state = TestState()
result = validate_state(state, "thread-123")
assert result == {"key": "value"}
def test_validate_invalid_state_returns_none(self):
"""Test that invalid state returns None."""
from praisonaiagents.ui.agui.conversion import validate_state
# A string is not a valid state
result = validate_state("invalid", "thread-123")
assert result is None
class TestToolCallDeduplication:
"""Test tool call deduplication logic."""
def test_deduplicate_tool_results(self):
"""Test that duplicate tool results are deduplicated."""
from praisonaiagents.ui.agui.conversion import agui_messages_to_praisonai
from praisonaiagents.ui.agui.types import Message
# Same tool_call_id appearing twice
agui_messages = [
Message(role="tool", content="Result 1", tool_call_id="tc-123"),
Message(role="tool", content="Result 2", tool_call_id="tc-123"), # Duplicate
]
result = agui_messages_to_praisonai(agui_messages)
# Should only have one result
assert len(result) == 1
assert result[0]["content"] == "Result 1" # First one wins
class TestExtractUserInput:
"""Test extracting user input from messages."""
def test_extract_last_user_message(self):
"""Test extracting the last user message."""
from praisonaiagents.ui.agui.conversion import extract_user_input
from praisonaiagents.ui.agui.types import Message
messages = [
Message(role="user", content="First message"),
Message(role="assistant", content="Response"),
Message(role="user", content="Last message"),
]
result = extract_user_input(messages)
assert result == "Last message"
def test_extract_from_empty_messages(self):
"""Test extracting from empty messages."""
from praisonaiagents.ui.agui.conversion import extract_user_input
result = extract_user_input([])
assert result == ""
def test_extract_no_user_messages(self):
"""Test extracting when no user messages exist."""
from praisonaiagents.ui.agui.conversion import extract_user_input
from praisonaiagents.ui.agui.types import Message
messages = [
Message(role="assistant", content="Hello"),
]
result = extract_user_input(messages)
assert result == ""
@@ -0,0 +1,311 @@
"""
Test AG-UI Integration - TDD Tests for End-to-End Integration
Phase 7: Integration Tests
- Test full flow with single Agent
- Test full flow with PraisonAIAgents
- Test tool calling flow
- Test error recovery
- Test state persistence
"""
class TestSingleAgentIntegration:
"""Test integration with single Agent."""
def test_single_agent_chat_flow(self):
"""Test complete chat flow with single agent."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
import json
agent = Agent(
name="Assistant",
role="Helpful Assistant",
goal="Help users with their questions",
llm="gpt-4o-mini"
)
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
# Send a message
response = client.post("/agui", json={
"thread_id": "test-thread",
"messages": [{"role": "user", "content": "Say hello"}]
})
assert response.status_code == 200
# Parse SSE events
events = []
for line in response.text.split("\n"):
if line.startswith("data: "):
event_data = json.loads(line[6:])
events.append(event_data)
# Should have run started event
event_types = [e.get("type") for e in events]
assert "RUN_STARTED" in event_types
assert "RUN_FINISHED" in event_types or "RUN_ERROR" in event_types
class TestMultiAgentIntegration:
"""Test integration with PraisonAIAgents."""
def test_multi_agent_workflow_flow(self):
"""Test complete workflow with multiple agents."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent, Task, PraisonAIAgents
from fastapi import FastAPI
import json
# Create agents
researcher = Agent(
name="Researcher",
role="Research Analyst",
goal="Research topics thoroughly",
llm="gpt-4o-mini"
)
writer = Agent(
name="Writer",
role="Content Writer",
goal="Write engaging content",
llm="gpt-4o-mini"
)
# Create tasks
research_task = Task(
description="Research the topic: {topic}",
expected_output="Research findings",
agent=researcher
)
write_task = Task(
description="Write about the research",
expected_output="Written article",
agent=writer
)
# Create PraisonAIAgents
agents = PraisonAIAgents(
agents=[researcher, writer],
tasks=[research_task, write_task]
)
agui = AGUI(agents=agents)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "test-thread",
"messages": [{"role": "user", "content": "Research AI"}]
})
assert response.status_code == 200
class TestToolCallingIntegration:
"""Test integration with tool calling."""
def test_agent_with_tools(self):
"""Test agent with tools integration."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
import json
def search_tool(query: str) -> str:
"""Search for information."""
return f"Search results for: {query}"
agent = Agent(
name="SearchAgent",
role="Search Assistant",
goal="Help users search for information",
tools=[search_tool],
llm="gpt-4o-mini"
)
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "test-thread",
"messages": [{"role": "user", "content": "Search for Python tutorials"}]
})
assert response.status_code == 200
# Parse events
events = []
for line in response.text.split("\n"):
if line.startswith("data: "):
event_data = json.loads(line[6:])
events.append(event_data)
# Check for tool call events (may or may not have them depending on LLM response)
event_types = [e.get("type") for e in events]
assert "RUN_STARTED" in event_types
class TestStateManagement:
"""Test state management in AG-UI."""
def test_state_passed_to_agent(self):
"""Test that state is passed to agent."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(
name="StatefulAgent",
role="Stateful Assistant",
goal="Remember context",
llm="gpt-4o-mini"
)
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "test-thread",
"messages": [{"role": "user", "content": "Hello"}],
"state": {"user_name": "John", "preferences": {"language": "en"}}
})
assert response.status_code == 200
class TestConversationHistory:
"""Test conversation history handling."""
def test_multiple_messages_in_history(self):
"""Test handling multiple messages in conversation history."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(
name="ChatAgent",
role="Chat Assistant",
goal="Have conversations",
llm="gpt-4o-mini"
)
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "test-thread",
"messages": [
{"role": "user", "content": "My name is John"},
{"role": "assistant", "content": "Hello John!"},
{"role": "user", "content": "What is my name?"}
]
})
assert response.status_code == 200
class TestErrorRecovery:
"""Test error recovery scenarios."""
def test_graceful_error_handling(self):
"""Test graceful error handling."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
import json
# Agent with invalid LLM should handle error gracefully
agent = Agent(
name="ErrorAgent",
role="Error Test",
goal="Test error handling",
llm="invalid-model-name"
)
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "test-thread",
"messages": [{"role": "user", "content": "Hello"}]
})
# Should not crash, should return error event
assert response.status_code == 200
# Parse events
events = []
for line in response.text.split("\n"):
if line.startswith("data: "):
try:
event_data = json.loads(line[6:])
events.append(event_data)
except json.JSONDecodeError:
pass
# Should have error event
event_types = [e.get("type") for e in events]
assert "RUN_ERROR" in event_types or "RUN_FINISHED" in event_types
class TestPerformance:
"""Test performance - ensure no impact on existing functionality."""
def test_agui_import_does_not_slow_package(self):
"""Test that importing AGUI doesn't slow down package import."""
import time
start = time.time()
from praisonaiagents.ui.agui import AGUI
import_time = time.time() - start
# Import should be fast (< 1 second)
assert import_time < 1.0
def test_agui_lazy_loading(self):
"""Test that AGUI uses lazy loading where possible."""
# This test ensures we don't import heavy dependencies until needed
import sys
# Remove agui from cache if present
modules_to_remove = [k for k in sys.modules.keys() if 'agui' in k]
for mod in modules_to_remove:
del sys.modules[mod]
# Import should not load FastAPI until get_router is called
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
agent = Agent(name="Test", role="Tester", goal="Test")
agui = AGUI(agent=agent)
# FastAPI should be loaded when get_router is called
router = agui.get_router()
assert router is not None
@@ -0,0 +1,266 @@
"""
Test AG-UI Router - TDD Tests for FastAPI HTTP Endpoints
Phase 6: FastAPI Router Tests
- Test /agui POST endpoint
- Test /status GET endpoint
- Test request validation
- Test streaming response
- Test error handling
"""
class TestStatusEndpoint:
"""Test /status endpoint."""
def test_status_endpoint_returns_available(self):
"""Test status endpoint returns available status."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.get("/status")
assert response.status_code == 200
assert response.json() == {"status": "available"}
class TestAGUIEndpoint:
"""Test /agui POST endpoint."""
def test_agui_endpoint_accepts_post(self):
"""Test /agui endpoint accepts POST requests."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
# Minimal valid request
response = client.post("/agui", json={
"thread_id": "thread-123",
"messages": [{"role": "user", "content": "Hello"}]
})
# Should return streaming response (200)
assert response.status_code == 200
def test_agui_endpoint_returns_streaming_response(self):
"""Test /agui endpoint returns streaming response."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "thread-123",
"messages": [{"role": "user", "content": "Hello"}]
})
# Check content type is event-stream
assert "text/event-stream" in response.headers.get("content-type", "")
def test_agui_endpoint_requires_thread_id(self):
"""Test /agui endpoint requires thread_id."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
# Missing thread_id
response = client.post("/agui", json={
"messages": [{"role": "user", "content": "Hello"}]
})
# Should return validation error
assert response.status_code == 422
class TestAGUIEndpointWithState:
"""Test /agui endpoint with state."""
def test_agui_endpoint_accepts_state(self):
"""Test /agui endpoint accepts state parameter."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "thread-123",
"messages": [{"role": "user", "content": "Hello"}],
"state": {"custom_key": "custom_value"}
})
assert response.status_code == 200
class TestAGUIEndpointWithRunId:
"""Test /agui endpoint with run_id."""
def test_agui_endpoint_accepts_run_id(self):
"""Test /agui endpoint accepts run_id parameter."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "thread-123",
"run_id": "run-456",
"messages": [{"role": "user", "content": "Hello"}]
})
assert response.status_code == 200
def test_agui_endpoint_generates_run_id_if_missing(self):
"""Test /agui endpoint generates run_id if not provided."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "thread-123",
"messages": [{"role": "user", "content": "Hello"}]
})
# Should succeed even without run_id
assert response.status_code == 200
class TestAGUIEndpointCORS:
"""Test /agui endpoint CORS headers."""
def test_agui_endpoint_cors_headers(self):
"""Test /agui endpoint includes CORS headers."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "thread-123",
"messages": [{"role": "user", "content": "Hello"}]
})
# Check CORS headers
assert response.headers.get("access-control-allow-origin") == "*"
class TestAGUIEndpointErrorHandling:
"""Test /agui endpoint error handling."""
def test_agui_endpoint_handles_agent_error(self):
"""Test /agui endpoint handles agent errors gracefully."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
# Create agent that will fail
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
# This should not crash the server
response = client.post("/agui", json={
"thread_id": "thread-123",
"messages": [{"role": "user", "content": "Hello"}]
})
# Should return 200 with error event in stream
assert response.status_code == 200
class TestAGUIEndpointWithForwardedProps:
"""Test /agui endpoint with forwarded_props."""
def test_agui_endpoint_accepts_forwarded_props(self):
"""Test /agui endpoint accepts forwarded_props."""
from fastapi.testclient import TestClient
from praisonaiagents.ui.agui import AGUI
from praisonaiagents import Agent
from fastapi import FastAPI
agent = Agent(name="Test", role="Tester", goal="Test things")
agui = AGUI(agent=agent)
app = FastAPI()
app.include_router(agui.get_router())
client = TestClient(app)
response = client.post("/agui", json={
"thread_id": "thread-123",
"messages": [{"role": "user", "content": "Hello"}],
"forwarded_props": {"user_id": "user-123"}
})
assert response.status_code == 200
@@ -0,0 +1,240 @@
"""
Test AG-UI Event Streaming - TDD Tests for Streaming PraisonAI Responses
Phase 4: Event Streaming Tests
- Test streaming text content as AG-UI events
- Test streaming tool calls as AG-UI events
- Test run lifecycle events (start, finish, error)
- Test step events
- Test event ordering
"""
import asyncio
class TestTextMessageStreaming:
"""Test streaming text messages."""
def test_stream_simple_text(self):
"""Test streaming simple text content."""
from praisonaiagents.ui.agui.streaming import create_text_message_events
events = list(create_text_message_events(
content="Hello, world!",
message_id="msg-123"
))
# Should have start, content, end events
assert len(events) == 3
assert events[0].type == "TEXT_MESSAGE_START"
assert events[0].message_id == "msg-123"
assert events[1].type == "TEXT_MESSAGE_CONTENT"
assert events[1].delta == "Hello, world!"
assert events[2].type == "TEXT_MESSAGE_END"
def test_stream_chunked_text(self):
"""Test streaming text in chunks."""
from praisonaiagents.ui.agui.streaming import stream_text_chunks
chunks = ["Hello", ", ", "world", "!"]
events = list(stream_text_chunks(chunks, message_id="msg-123"))
# Should have start, 4 content events, end
assert len(events) == 6
assert events[0].type == "TEXT_MESSAGE_START"
assert events[1].delta == "Hello"
assert events[2].delta == ", "
assert events[3].delta == "world"
assert events[4].delta == "!"
assert events[5].type == "TEXT_MESSAGE_END"
def test_stream_empty_text(self):
"""Test streaming empty text."""
from praisonaiagents.ui.agui.streaming import create_text_message_events
events = list(create_text_message_events(content="", message_id="msg-123"))
# Should still have start and end, but no content
assert len(events) == 2
assert events[0].type == "TEXT_MESSAGE_START"
assert events[1].type == "TEXT_MESSAGE_END"
class TestToolCallStreaming:
"""Test streaming tool calls."""
def test_stream_tool_call(self):
"""Test streaming a tool call."""
from praisonaiagents.ui.agui.streaming import create_tool_call_events
events = list(create_tool_call_events(
tool_call_id="tc-123",
tool_name="search",
arguments='{"query": "test"}',
parent_message_id="msg-123"
))
# Should have start, args, end events
assert len(events) == 3
assert events[0].type == "TOOL_CALL_START"
assert events[0].tool_call_id == "tc-123"
assert events[0].tool_call_name == "search"
assert events[1].type == "TOOL_CALL_ARGS"
assert events[1].delta == '{"query": "test"}'
assert events[2].type == "TOOL_CALL_END"
def test_stream_tool_result(self):
"""Test streaming a tool result."""
from praisonaiagents.ui.agui.streaming import create_tool_result_event
event = create_tool_result_event(
tool_call_id="tc-123",
content="Search results...",
message_id="msg-456"
)
assert event.type == "TOOL_CALL_RESULT"
assert event.tool_call_id == "tc-123"
assert event.content == "Search results..."
class TestRunLifecycleEvents:
"""Test run lifecycle events."""
def test_create_run_started_event(self):
"""Test creating run started event."""
from praisonaiagents.ui.agui.streaming import create_run_started_event
event = create_run_started_event(thread_id="thread-123", run_id="run-456")
assert event.type == "RUN_STARTED"
assert event.thread_id == "thread-123"
assert event.run_id == "run-456"
def test_create_run_finished_event(self):
"""Test creating run finished event."""
from praisonaiagents.ui.agui.streaming import create_run_finished_event
event = create_run_finished_event(thread_id="thread-123", run_id="run-456")
assert event.type == "RUN_FINISHED"
assert event.thread_id == "thread-123"
assert event.run_id == "run-456"
def test_create_run_error_event(self):
"""Test creating run error event."""
from praisonaiagents.ui.agui.streaming import create_run_error_event
event = create_run_error_event(message="Something went wrong", code="ERR001")
assert event.type == "RUN_ERROR"
assert event.message == "Something went wrong"
assert event.code == "ERR001"
class TestStepEvents:
"""Test step events."""
def test_create_step_started_event(self):
"""Test creating step started event."""
from praisonaiagents.ui.agui.streaming import create_step_started_event
event = create_step_started_event(step_name="research")
assert event.type == "STEP_STARTED"
assert event.step_name == "research"
def test_create_step_finished_event(self):
"""Test creating step finished event."""
from praisonaiagents.ui.agui.streaming import create_step_finished_event
event = create_step_finished_event(step_name="research")
assert event.type == "STEP_FINISHED"
assert event.step_name == "research"
class TestEventBuffer:
"""Test EventBuffer for managing event ordering."""
def test_event_buffer_creation(self):
"""Test EventBuffer creation."""
from praisonaiagents.ui.agui.streaming import EventBuffer
buffer = EventBuffer()
assert buffer.active_tool_call_ids == set()
assert buffer.current_text_message_id == ""
def test_event_buffer_start_tool_call(self):
"""Test starting a tool call in buffer."""
from praisonaiagents.ui.agui.streaming import EventBuffer
buffer = EventBuffer()
buffer.start_tool_call("tc-123")
assert "tc-123" in buffer.active_tool_call_ids
def test_event_buffer_end_tool_call(self):
"""Test ending a tool call in buffer."""
from praisonaiagents.ui.agui.streaming import EventBuffer
buffer = EventBuffer()
buffer.start_tool_call("tc-123")
buffer.end_tool_call("tc-123")
assert "tc-123" not in buffer.active_tool_call_ids
assert "tc-123" in buffer.ended_tool_call_ids
def test_event_buffer_start_text_message(self):
"""Test starting a text message in buffer."""
from praisonaiagents.ui.agui.streaming import EventBuffer
buffer = EventBuffer()
msg_id = buffer.start_text_message()
assert msg_id != ""
assert buffer.current_text_message_id == msg_id
class TestAsyncStreaming:
"""Test async streaming functionality."""
def test_async_stream_response(self):
"""Test async streaming of agent response."""
async def run_test():
from praisonaiagents.ui.agui.streaming import async_stream_response
# Mock response chunks
async def mock_chunks():
yield "Hello"
yield ", "
yield "world!"
events = []
async for event in async_stream_response(
response_stream=mock_chunks(),
thread_id="thread-123",
run_id="run-456"
):
events.append(event)
# Should have run_started, text events, run_finished
assert len(events) > 0
assert events[0].type == "RUN_STARTED"
assert events[-1].type == "RUN_FINISHED"
asyncio.run(run_test())
class TestStateSnapshotEvents:
"""Test state snapshot events."""
def test_create_state_snapshot_event(self):
"""Test creating state snapshot event."""
from praisonaiagents.ui.agui.streaming import create_state_snapshot_event
state = {"task": "research", "progress": 50}
event = create_state_snapshot_event(state)
assert event.type == "STATE_SNAPSHOT"
assert event.snapshot == state
@@ -0,0 +1,327 @@
"""
Test AG-UI Types - TDD Tests for Event and Message Types
Phase 2: Core Types Tests
- Test EventType enum values
- Test BaseEvent creation
- Test all event type classes
- Test message type classes
- Test RunAgentInput validation
"""
import json
class TestEventType:
"""Test EventType enum."""
def test_event_type_text_message_start(self):
"""Test TEXT_MESSAGE_START event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.TEXT_MESSAGE_START == "TEXT_MESSAGE_START"
def test_event_type_text_message_content(self):
"""Test TEXT_MESSAGE_CONTENT event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.TEXT_MESSAGE_CONTENT == "TEXT_MESSAGE_CONTENT"
def test_event_type_text_message_end(self):
"""Test TEXT_MESSAGE_END event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.TEXT_MESSAGE_END == "TEXT_MESSAGE_END"
def test_event_type_tool_call_start(self):
"""Test TOOL_CALL_START event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.TOOL_CALL_START == "TOOL_CALL_START"
def test_event_type_tool_call_args(self):
"""Test TOOL_CALL_ARGS event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.TOOL_CALL_ARGS == "TOOL_CALL_ARGS"
def test_event_type_tool_call_end(self):
"""Test TOOL_CALL_END event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.TOOL_CALL_END == "TOOL_CALL_END"
def test_event_type_tool_call_result(self):
"""Test TOOL_CALL_RESULT event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.TOOL_CALL_RESULT == "TOOL_CALL_RESULT"
def test_event_type_run_started(self):
"""Test RUN_STARTED event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.RUN_STARTED == "RUN_STARTED"
def test_event_type_run_finished(self):
"""Test RUN_FINISHED event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.RUN_FINISHED == "RUN_FINISHED"
def test_event_type_run_error(self):
"""Test RUN_ERROR event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.RUN_ERROR == "RUN_ERROR"
def test_event_type_step_started(self):
"""Test STEP_STARTED event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.STEP_STARTED == "STEP_STARTED"
def test_event_type_step_finished(self):
"""Test STEP_FINISHED event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.STEP_FINISHED == "STEP_FINISHED"
def test_event_type_state_snapshot(self):
"""Test STATE_SNAPSHOT event type exists."""
from praisonaiagents.ui.agui.types import EventType
assert EventType.STATE_SNAPSHOT == "STATE_SNAPSHOT"
class TestBaseEvent:
"""Test BaseEvent class."""
def test_base_event_creation(self):
"""Test BaseEvent can be created."""
from praisonaiagents.ui.agui.types import BaseEvent, EventType
event = BaseEvent(type=EventType.RUN_STARTED)
assert event.type == EventType.RUN_STARTED
def test_base_event_timestamp_optional(self):
"""Test BaseEvent timestamp is optional."""
from praisonaiagents.ui.agui.types import BaseEvent, EventType
event = BaseEvent(type=EventType.RUN_STARTED)
assert event.timestamp is None
def test_base_event_to_dict(self):
"""Test BaseEvent can be converted to dict."""
from praisonaiagents.ui.agui.types import BaseEvent, EventType
event = BaseEvent(type=EventType.RUN_STARTED)
data = event.model_dump()
assert data["type"] == "RUN_STARTED"
class TestTextMessageEvents:
"""Test text message event classes."""
def test_text_message_start_event(self):
"""Test TextMessageStartEvent creation."""
from praisonaiagents.ui.agui.types import TextMessageStartEvent
event = TextMessageStartEvent(message_id="msg-123", role="assistant")
assert event.message_id == "msg-123"
assert event.role == "assistant"
assert event.type == "TEXT_MESSAGE_START"
def test_text_message_content_event(self):
"""Test TextMessageContentEvent creation."""
from praisonaiagents.ui.agui.types import TextMessageContentEvent
event = TextMessageContentEvent(message_id="msg-123", delta="Hello")
assert event.message_id == "msg-123"
assert event.delta == "Hello"
assert event.type == "TEXT_MESSAGE_CONTENT"
def test_text_message_end_event(self):
"""Test TextMessageEndEvent creation."""
from praisonaiagents.ui.agui.types import TextMessageEndEvent
event = TextMessageEndEvent(message_id="msg-123")
assert event.message_id == "msg-123"
assert event.type == "TEXT_MESSAGE_END"
class TestToolCallEvents:
"""Test tool call event classes."""
def test_tool_call_start_event(self):
"""Test ToolCallStartEvent creation."""
from praisonaiagents.ui.agui.types import ToolCallStartEvent
event = ToolCallStartEvent(
tool_call_id="tc-123",
tool_call_name="search",
parent_message_id="msg-123"
)
assert event.tool_call_id == "tc-123"
assert event.tool_call_name == "search"
assert event.parent_message_id == "msg-123"
assert event.type == "TOOL_CALL_START"
def test_tool_call_args_event(self):
"""Test ToolCallArgsEvent creation."""
from praisonaiagents.ui.agui.types import ToolCallArgsEvent
event = ToolCallArgsEvent(tool_call_id="tc-123", delta='{"query": "test"}')
assert event.tool_call_id == "tc-123"
assert event.delta == '{"query": "test"}'
assert event.type == "TOOL_CALL_ARGS"
def test_tool_call_end_event(self):
"""Test ToolCallEndEvent creation."""
from praisonaiagents.ui.agui.types import ToolCallEndEvent
event = ToolCallEndEvent(tool_call_id="tc-123")
assert event.tool_call_id == "tc-123"
assert event.type == "TOOL_CALL_END"
def test_tool_call_result_event(self):
"""Test ToolCallResultEvent creation."""
from praisonaiagents.ui.agui.types import ToolCallResultEvent
event = ToolCallResultEvent(
message_id="msg-456",
tool_call_id="tc-123",
content="Search results..."
)
assert event.message_id == "msg-456"
assert event.tool_call_id == "tc-123"
assert event.content == "Search results..."
assert event.type == "TOOL_CALL_RESULT"
class TestRunEvents:
"""Test run lifecycle event classes."""
def test_run_started_event(self):
"""Test RunStartedEvent creation."""
from praisonaiagents.ui.agui.types import RunStartedEvent
event = RunStartedEvent(thread_id="thread-123", run_id="run-456")
assert event.thread_id == "thread-123"
assert event.run_id == "run-456"
assert event.type == "RUN_STARTED"
def test_run_finished_event(self):
"""Test RunFinishedEvent creation."""
from praisonaiagents.ui.agui.types import RunFinishedEvent
event = RunFinishedEvent(thread_id="thread-123", run_id="run-456")
assert event.thread_id == "thread-123"
assert event.run_id == "run-456"
assert event.type == "RUN_FINISHED"
def test_run_error_event(self):
"""Test RunErrorEvent creation."""
from praisonaiagents.ui.agui.types import RunErrorEvent
event = RunErrorEvent(message="Something went wrong")
assert event.message == "Something went wrong"
assert event.type == "RUN_ERROR"
class TestStepEvents:
"""Test step event classes."""
def test_step_started_event(self):
"""Test StepStartedEvent creation."""
from praisonaiagents.ui.agui.types import StepStartedEvent
event = StepStartedEvent(step_name="research")
assert event.step_name == "research"
assert event.type == "STEP_STARTED"
def test_step_finished_event(self):
"""Test StepFinishedEvent creation."""
from praisonaiagents.ui.agui.types import StepFinishedEvent
event = StepFinishedEvent(step_name="research")
assert event.step_name == "research"
assert event.type == "STEP_FINISHED"
class TestStateEvents:
"""Test state event classes."""
def test_state_snapshot_event(self):
"""Test StateSnapshotEvent creation."""
from praisonaiagents.ui.agui.types import StateSnapshotEvent
event = StateSnapshotEvent(snapshot={"key": "value"})
assert event.snapshot == {"key": "value"}
assert event.type == "STATE_SNAPSHOT"
class TestMessageTypes:
"""Test message type classes."""
def test_user_message(self):
"""Test UserMessage creation."""
from praisonaiagents.ui.agui.types import Message
msg = Message(role="user", content="Hello")
assert msg.role == "user"
assert msg.content == "Hello"
def test_assistant_message(self):
"""Test AssistantMessage creation."""
from praisonaiagents.ui.agui.types import Message
msg = Message(role="assistant", content="Hi there!")
assert msg.role == "assistant"
assert msg.content == "Hi there!"
def test_tool_message(self):
"""Test ToolMessage creation."""
from praisonaiagents.ui.agui.types import Message
msg = Message(role="tool", content="Result", tool_call_id="tc-123")
assert msg.role == "tool"
assert msg.content == "Result"
assert msg.tool_call_id == "tc-123"
class TestToolCall:
"""Test ToolCall type."""
def test_tool_call_creation(self):
"""Test ToolCall creation."""
from praisonaiagents.ui.agui.types import ToolCall, FunctionCall
tc = ToolCall(
id="tc-123",
function=FunctionCall(name="search", arguments='{"query": "test"}')
)
assert tc.id == "tc-123"
assert tc.function.name == "search"
assert tc.function.arguments == '{"query": "test"}'
class TestRunAgentInput:
"""Test RunAgentInput type."""
def test_run_agent_input_creation(self):
"""Test RunAgentInput creation."""
from praisonaiagents.ui.agui.types import RunAgentInput, Message
input_data = RunAgentInput(
thread_id="thread-123",
run_id="run-456",
messages=[Message(role="user", content="Hello")]
)
assert input_data.thread_id == "thread-123"
assert input_data.run_id == "run-456"
assert len(input_data.messages) == 1
def test_run_agent_input_optional_fields(self):
"""Test RunAgentInput optional fields."""
from praisonaiagents.ui.agui.types import RunAgentInput
input_data = RunAgentInput(thread_id="thread-123")
assert input_data.thread_id == "thread-123"
assert input_data.run_id is None
assert input_data.messages is None or len(input_data.messages) == 0
assert input_data.state is None
class TestEventSerialization:
"""Test event serialization to JSON."""
def test_event_json_serialization(self):
"""Test events can be serialized to JSON."""
from praisonaiagents.ui.agui.types import TextMessageStartEvent
event = TextMessageStartEvent(message_id="msg-123", role="assistant")
json_str = event.model_dump_json()
data = json.loads(json_str)
assert data["type"] == "TEXT_MESSAGE_START"
assert data["message_id"] == "msg-123"
def test_event_sse_format(self):
"""Test events can be formatted as SSE."""
from praisonaiagents.ui.agui.types import TextMessageStartEvent
from praisonaiagents.ui.agui.encoder import EventEncoder
encoder = EventEncoder()
event = TextMessageStartEvent(message_id="msg-123", role="assistant")
sse = encoder.encode(event)
assert sse.startswith("data: ")
assert sse.endswith("\n\n")
# Parse the JSON part
json_part = sse[6:-2] # Remove "data: " prefix and "\n\n" suffix
data = json.loads(json_part)
assert data["type"] == "TEXT_MESSAGE_START"
+13 -6
View File
@@ -493,17 +493,22 @@ class TestAgentIntegration:
def test_agent_store_memory(self):
"""Test Agent store_memory method."""
import uuid
from praisonaiagents import Agent
# Use unique user_id to avoid stale data from previous test runs
unique_user_id = f"test_user_{uuid.uuid4().hex[:8]}"
agent = Agent(
name="Test Agent",
memory=True
memory=True,
user_id=unique_user_id
)
agent.store_memory("User prefers dark mode", memory_type="short_term")
items = agent._memory_instance.get_short_term()
assert len(items) == 1
assert len(items) == 1, f"Expected 1 item, got {len(items)} items"
assert items[0].content == "User prefers dark mode"
def test_agent_get_memory_context(self):
@@ -658,10 +663,12 @@ class TestMultiAgentMemorySharing:
def test_agents_share_memory_with_same_user_id(self):
"""Test that agents with same user_id share memory."""
import uuid
from praisonaiagents import Agent
with tempfile.TemporaryDirectory() as tmpdir:
user_id = "shared_user"
# Use unique user_id to avoid stale data from previous test runs
user_id = f"shared_user_{uuid.uuid4().hex[:8]}"
base_path = f"{tmpdir}/memory"
# Agent 1 stores memories
@@ -676,9 +683,9 @@ class TestMultiAgentMemorySharing:
agent1._memory_instance.add_entity("Alice", "person", {"role": "developer"})
# Verify Agent1 stored memories
assert len(agent1._memory_instance.get_short_term()) == 1
assert len(agent1._memory_instance.get_long_term()) == 1
assert len(agent1._memory_instance.get_all_entities()) == 1
assert len(agent1._memory_instance.get_short_term()) == 1, f"Expected 1 short_term, got {len(agent1._memory_instance.get_short_term())}"
assert len(agent1._memory_instance.get_long_term()) == 1, f"Expected 1 long_term, got {len(agent1._memory_instance.get_long_term())}"
assert len(agent1._memory_instance.get_all_entities()) == 1, f"Expected 1 entity, got {len(agent1._memory_instance.get_all_entities())}"
# Agent 2 with same user_id should load memories
agent2 = Agent(
@@ -6,6 +6,8 @@ The agent should:
3. Return the final result (200)
"""
import pytest
import os
from praisonaiagents import Agent
def get_stock_price(company_name: str) -> str:
@@ -35,41 +37,60 @@ def multiply(a: int, b: int) -> int:
print(f"[Tool Called] multiply({a}, {b})")
return a * b
# Test with Gemini
print("=" * 60)
print("Testing with Gemini model")
print("=" * 60)
agent_gemini = Agent(
instructions="You are a helpful assistant. You can use the tools provided to you to help the user. When asked to multiply a stock price, first get the stock price, then multiply it.",
llm="gemini/gemini-2.5-pro",
tools=[get_stock_price, multiply],
verbose=True
@pytest.mark.skipif(
not os.environ.get("OPENAI_API_KEY"),
reason="OPENAI_API_KEY not set - skipping integration test"
)
def test_sequential_tool_calling_gpt():
"""Test sequential tool calling with GPT model"""
print("=" * 60)
print("Testing with GPT model")
print("=" * 60)
result_gemini = agent_gemini.start("what is the stock price of Google? multiply the Google stock price with 2")
print(f"\nFinal Result (Gemini): {result_gemini}")
agent_gpt = Agent(
instructions="You are a helpful assistant. When asked to multiply a stock price, first get the stock price, then multiply it.",
llm="gpt-4o-mini",
tools=[get_stock_price, multiply],
verbose=True
)
# Test with GPT-4
print("\n" + "=" * 60)
print("Testing with GPT-4 model")
print("=" * 60)
result = agent_gpt.start("what is the stock price of Google? multiply the Google stock price with 2")
print(f"\nFinal Result (GPT): {result}")
# Verify result
assert result, "GPT returned empty result"
print(f"GPT result contains '200': {'200' in str(result)}")
agent_gpt4 = Agent(
instructions="You are a helpful assistant. You can use the tools provided to you to help the user. When asked to multiply a stock price, first get the stock price, then multiply it.",
llm="gpt-5-nano",
tools=[get_stock_price, multiply],
verbose=True
@pytest.mark.skipif(
not os.environ.get("GOOGLE_API_KEY"),
reason="GOOGLE_API_KEY not set - skipping integration test"
)
def test_sequential_tool_calling_gemini():
"""Test sequential tool calling with Gemini model"""
print("=" * 60)
print("Testing with Gemini model")
print("=" * 60)
result_gpt4 = agent_gpt4.start("what is the stock price of Google? multiply the Google stock price with 2")
print(f"\nFinal Result (GPT-4): {result_gpt4}")
agent_gemini = Agent(
instructions="You are a helpful assistant. When asked to multiply a stock price, first get the stock price, then multiply it.",
llm="gemini/gemini-2.0-flash",
tools=[get_stock_price, multiply],
verbose=True
)
# Verify results
print("\n" + "=" * 60)
print("Test Results Summary")
print("=" * 60)
print(f"Gemini result contains '200': {'200' in str(result_gemini) if result_gemini else False}")
print(f"GPT-4 result contains '200': {'200' in str(result_gpt4) if result_gpt4 else False}")
print(f"Gemini returned empty: {not result_gemini or result_gemini == ''}")
print(f"GPT-4 returned empty: {not result_gpt4 or result_gpt4 == ''}")
result = agent_gemini.start("what is the stock price of Google? multiply the Google stock price with 2")
print(f"\nFinal Result (Gemini): {result}")
# Verify result
assert result, "Gemini returned empty result"
print(f"Gemini result contains '200': {'200' in str(result)}")
if __name__ == "__main__":
# Run tests manually
if os.environ.get("OPENAI_API_KEY"):
test_sequential_tool_calling_gpt()
if os.environ.get("GOOGLE_API_KEY"):
test_sequential_tool_calling_gemini()
@@ -2,7 +2,23 @@
"""Test LLM class directly to verify self-reflection fix"""
from praisonaiagents.llm import LLM
from praisonaiagents.tools import calculator
# Define calculator tool locally to avoid import issues
def calculator(expression: str) -> str:
"""
Evaluate a mathematical expression.
Args:
expression: A mathematical expression to evaluate (e.g., "2 + 2", "15 * 23")
Returns:
The result of the calculation as a string
"""
try:
result = eval(expression)
return str(result)
except Exception as e:
return f"Error: {str(e)}"
def test_llm_direct():
"""Test LLM class directly with self-reflection and tools"""
@@ -0,0 +1,391 @@
"""
Test-Driven Development tests for Multiple MCP Tools support.
This test file verifies that:
1. Single MCP instance works (baseline)
2. Multiple MCP instances in a list work for tool definition conversion
3. Multiple MCP instances in a list work for tool execution
4. Mixed tools (MCP + regular functions) work together
Uses mcp-server-time as the test MCP server (simple, no API keys needed).
"""
import pytest
import os
import sys
from unittest.mock import Mock
# Add the package to path for testing
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
from praisonaiagents.mcp.mcp import MCP
from praisonaiagents.agent.agent import Agent
# ============================================================================
# Test Fixtures
# ============================================================================
@pytest.fixture
def time_mcp():
"""Create a Time MCP server instance."""
return MCP("uvx mcp-server-time", timeout=30)
@pytest.fixture
def mock_mcp_with_tools():
"""Create a mock MCP instance with predefined tools."""
mock = Mock(spec=MCP)
mock.is_sse = False
mock.is_http_stream = False
mock.is_websocket = False
# Create mock runner with tools
mock.runner = Mock()
mock_tool1 = Mock()
mock_tool1.name = "get_current_time"
mock_tool1.description = "Get current time in a timezone"
mock_tool1.inputSchema = {
"type": "object",
"properties": {
"timezone": {"type": "string", "description": "IANA timezone name"}
},
"required": ["timezone"]
}
mock_tool2 = Mock()
mock_tool2.name = "convert_time"
mock_tool2.description = "Convert time between timezones"
mock_tool2.inputSchema = {
"type": "object",
"properties": {
"source_timezone": {"type": "string"},
"time": {"type": "string"},
"target_timezone": {"type": "string"}
},
"required": ["source_timezone", "time", "target_timezone"]
}
mock.runner.tools = [mock_tool1, mock_tool2]
mock._tools = []
# Mock to_openai_tool method
def to_openai_tool():
return [
{
"type": "function",
"function": {
"name": "get_current_time",
"description": "Get current time in a timezone",
"parameters": mock_tool1.inputSchema
}
},
{
"type": "function",
"function": {
"name": "convert_time",
"description": "Convert time between timezones",
"parameters": mock_tool2.inputSchema
}
}
]
mock.to_openai_tool = to_openai_tool
# Mock __iter__ to return tool functions
def get_current_time(timezone: str) -> str:
return f"Time in {timezone}: 12:00"
def convert_time(source_timezone: str, time: str, target_timezone: str) -> str:
return f"Converted {time} from {source_timezone} to {target_timezone}"
get_current_time.__name__ = "get_current_time"
convert_time.__name__ = "convert_time"
mock._tools = [get_current_time, convert_time]
mock.__iter__ = lambda self: iter(self._tools)
return mock
@pytest.fixture
def mock_mcp_filesystem():
"""Create a mock MCP instance simulating filesystem server."""
mock = Mock(spec=MCP)
mock.is_sse = False
mock.is_http_stream = False
mock.is_websocket = False
mock.runner = Mock()
mock_tool = Mock()
mock_tool.name = "read_file"
mock_tool.description = "Read a file"
mock_tool.inputSchema = {
"type": "object",
"properties": {
"path": {"type": "string"}
},
"required": ["path"]
}
mock.runner.tools = [mock_tool]
mock._tools = []
def to_openai_tool():
return [
{
"type": "function",
"function": {
"name": "read_file",
"description": "Read a file",
"parameters": mock_tool.inputSchema
}
}
]
mock.to_openai_tool = to_openai_tool
def read_file(path: str) -> str:
return f"Contents of {path}"
read_file.__name__ = "read_file"
mock._tools = [read_file]
mock.__iter__ = lambda self: iter(self._tools)
return mock
def sample_function(query: str) -> str:
"""A sample regular function tool."""
return f"Result for: {query}"
# ============================================================================
# Test 1: Single MCP Instance (Baseline)
# ============================================================================
class TestSingleMCPInstance:
"""Tests for single MCP instance - baseline functionality."""
def test_single_mcp_creates_tools(self, mock_mcp_with_tools):
"""Test that a single MCP instance properly exposes tools."""
tools = mock_mcp_with_tools.to_openai_tool()
assert isinstance(tools, list)
assert len(tools) == 2
assert tools[0]["function"]["name"] == "get_current_time"
assert tools[1]["function"]["name"] == "convert_time"
def test_single_mcp_is_iterable(self, mock_mcp_with_tools):
"""Test that MCP instance is iterable."""
tools_list = mock_mcp_with_tools._tools
assert len(tools_list) == 2
assert callable(tools_list[0])
assert callable(tools_list[1])
def test_agent_accepts_single_mcp(self, mock_mcp_with_tools):
"""Test that Agent accepts a single MCP instance as tools."""
# This should not raise an error
agent = Agent(
name="TestAgent",
instructions="Test agent",
tools=mock_mcp_with_tools
)
# Agent wraps tools in a list if not already a list
assert mock_mcp_with_tools in agent.tools or agent.tools == mock_mcp_with_tools
# ============================================================================
# Test 2: Multiple MCP Instances - Tool Definition Conversion
# ============================================================================
class TestMultipleMCPToolDefinition:
"""Tests for multiple MCP instances - tool definition conversion."""
def test_agent_accepts_multiple_mcp_in_list(self, mock_mcp_with_tools, mock_mcp_filesystem):
"""Test that Agent accepts multiple MCP instances in a list."""
agent = Agent(
name="TestAgent",
instructions="Test agent",
tools=[mock_mcp_with_tools, mock_mcp_filesystem]
)
assert isinstance(agent.tools, list)
assert len(agent.tools) == 2
def test_format_tools_handles_multiple_mcp(self, mock_mcp_with_tools, mock_mcp_filesystem):
"""Test that _format_tools_for_llm handles multiple MCP instances."""
agent = Agent(
name="TestAgent",
instructions="Test agent",
tools=[mock_mcp_with_tools, mock_mcp_filesystem]
)
# Call the internal method to format tools
formatted = agent._format_tools_for_completion(agent.tools)
# Should have 3 tools total (2 from time + 1 from filesystem)
assert isinstance(formatted, list)
assert len(formatted) == 3
# Check tool names
tool_names = [t["function"]["name"] for t in formatted]
assert "get_current_time" in tool_names
assert "convert_time" in tool_names
assert "read_file" in tool_names
def test_system_prompt_includes_all_mcp_tools(self, mock_mcp_with_tools, mock_mcp_filesystem):
"""Test that system prompt includes tools from all MCP instances."""
agent = Agent(
name="TestAgent",
instructions="Test agent",
tools=[mock_mcp_with_tools, mock_mcp_filesystem]
)
system_prompt = agent._build_system_prompt(agent.tools)
# Should mention all tools
assert "get_current_time" in system_prompt
assert "convert_time" in system_prompt
assert "read_file" in system_prompt
# ============================================================================
# Test 3: Multiple MCP Instances - Tool Execution
# ============================================================================
class TestMultipleMCPToolExecution:
"""Tests for multiple MCP instances - tool execution."""
def test_execute_tool_finds_mcp_tool_in_list(self, mock_mcp_with_tools, mock_mcp_filesystem):
"""Test that execute_tool_call finds tools from MCP instances in a list."""
agent = Agent(
name="TestAgent",
instructions="Test agent",
tools=[mock_mcp_with_tools, mock_mcp_filesystem]
)
# Mock the runner.call_tool method
mock_mcp_with_tools.runner.call_tool = Mock(return_value='{"timezone": "UTC", "datetime": "2024-01-01T12:00:00Z"}')
# This should find and execute the tool from the first MCP instance
result = agent.execute_tool("get_current_time", {"timezone": "UTC"})
# Verify the tool was called
mock_mcp_with_tools.runner.call_tool.assert_called_once_with("get_current_time", {"timezone": "UTC"})
def test_execute_tool_finds_tool_from_second_mcp(self, mock_mcp_with_tools, mock_mcp_filesystem):
"""Test that execute_tool_call finds tools from the second MCP instance."""
agent = Agent(
name="TestAgent",
instructions="Test agent",
tools=[mock_mcp_with_tools, mock_mcp_filesystem]
)
# Mock the runner.call_tool method
mock_mcp_filesystem.runner.call_tool = Mock(return_value='Contents of /test/file.txt')
# This should find and execute the tool from the second MCP instance
result = agent.execute_tool("read_file", {"path": "/test/file.txt"})
# Verify the tool was called
mock_mcp_filesystem.runner.call_tool.assert_called_once_with("read_file", {"path": "/test/file.txt"})
# ============================================================================
# Test 4: Mixed Tools (MCP + Regular Functions)
# ============================================================================
class TestMixedTools:
"""Tests for mixed tools - MCP instances and regular functions."""
def test_agent_accepts_mixed_tools(self, mock_mcp_with_tools):
"""Test that Agent accepts a mix of MCP instances and regular functions."""
agent = Agent(
name="TestAgent",
instructions="Test agent",
tools=[mock_mcp_with_tools, sample_function]
)
assert isinstance(agent.tools, list)
assert len(agent.tools) == 2
def test_format_tools_handles_mixed(self, mock_mcp_with_tools):
"""Test that _format_tools_for_llm handles mixed tools."""
agent = Agent(
name="TestAgent",
instructions="Test agent",
tools=[mock_mcp_with_tools, sample_function]
)
formatted = agent._format_tools_for_completion(agent.tools)
# Should have 3 tools (2 from MCP + 1 regular function)
assert isinstance(formatted, list)
assert len(formatted) == 3
tool_names = [t["function"]["name"] for t in formatted]
assert "get_current_time" in tool_names
assert "convert_time" in tool_names
assert "sample_function" in tool_names
def test_execute_mixed_tools(self, mock_mcp_with_tools):
"""Test executing both MCP tools and regular functions."""
agent = Agent(
name="TestAgent",
instructions="Test agent",
tools=[mock_mcp_with_tools, sample_function]
)
# Execute regular function
result = agent.execute_tool("sample_function", {"query": "test"})
assert "Result for: test" in str(result)
# Mock and execute MCP tool
mock_mcp_with_tools.runner.call_tool = Mock(return_value='{"timezone": "UTC"}')
result = agent.execute_tool("get_current_time", {"timezone": "UTC"})
mock_mcp_with_tools.runner.call_tool.assert_called_once()
# ============================================================================
# Integration Tests (requires actual MCP server)
# ============================================================================
@pytest.mark.integration
@pytest.mark.skipif(
os.environ.get("SKIP_MCP_INTEGRATION") == "1",
reason="MCP integration tests skipped"
)
class TestMCPIntegration:
"""Integration tests using actual MCP servers."""
def test_real_time_mcp_server(self):
"""Test with real mcp-server-time."""
try:
mcp = MCP("uvx mcp-server-time", timeout=30)
tools = mcp.to_openai_tool()
assert isinstance(tools, list)
assert len(tools) >= 1
tool_names = [t["function"]["name"] for t in tools]
assert "get_current_time" in tool_names
except Exception as e:
pytest.skip(f"MCP server not available: {e}")
def test_multiple_real_mcp_servers(self):
"""Test with multiple real MCP servers."""
try:
time_mcp = MCP("uvx mcp-server-time", timeout=30)
# Create agent with multiple MCP instances
agent = Agent(
name="MultiMCPAgent",
instructions="Test agent with multiple MCP tools",
tools=[time_mcp]
)
# Verify tools are available
formatted = agent._format_tools_for_completion(agent.tools)
assert len(formatted) >= 1
except Exception as e:
pytest.skip(f"MCP servers not available: {e}")
# ============================================================================
# Run tests
# ============================================================================
if __name__ == "__main__":
pytest.main([__file__, "-v", "--tb=short"])
@@ -5,7 +5,9 @@ Test script to verify Ollama empty response fix for both sync and async methods.
import asyncio
import logging
from praisonaiagents import Agent, Task, PraisonAIAgents, TaskOutput
import os
import pytest
from praisonaiagents import Agent, Task, PraisonAIAgents
from typing import Dict, Any
# Enable debug logging
@@ -25,8 +27,8 @@ def search_tool(query: str) -> Dict[str, Any]:
logging.debug(f"[TEST] Search tool returning: {results}")
return results
# Test with both sync and async workflows
async def test_model_async(model_name: str):
# Helper async function to test a model
async def _test_model_async_helper(model_name: str):
print(f"\n{'='*60}")
print(f"Testing ASYNC with model: {model_name}")
print('='*60)
@@ -78,7 +80,7 @@ async def test_model_async(model_name: str):
traceback.print_exc()
return False
def test_model_sync(model_name: str):
def _test_model_sync_helper(model_name: str):
print(f"\n{'='*60}")
print(f"Testing SYNC with model: {model_name}")
print('='*60)
@@ -130,18 +132,60 @@ def test_model_sync(model_name: str):
traceback.print_exc()
return False
@pytest.mark.skipif(
not os.environ.get("OPENAI_API_KEY"),
reason="OPENAI_API_KEY not set - skipping integration test"
)
@pytest.mark.asyncio
async def test_ollama_async_openai():
"""Test async with OpenAI model as baseline."""
result = await _test_model_async_helper("gpt-4o-mini")
assert result, "OpenAI async model test failed"
@pytest.mark.skipif(
not os.environ.get("OPENAI_API_KEY"),
reason="OPENAI_API_KEY not set - skipping integration test"
)
def test_ollama_sync_openai():
"""Test sync with OpenAI model as baseline."""
result = _test_model_sync_helper("gpt-4o-mini")
assert result, "OpenAI sync model test failed"
@pytest.mark.skipif(
not os.environ.get("OLLAMA_HOST", "").strip(),
reason="OLLAMA_HOST not set - skipping Ollama integration test"
)
@pytest.mark.asyncio
async def test_ollama_async_ollama():
"""Test async with Ollama model."""
result = await _test_model_async_helper("ollama/llama3.2")
assert result, "Ollama async model test failed"
@pytest.mark.skipif(
not os.environ.get("OLLAMA_HOST", "").strip(),
reason="OLLAMA_HOST not set - skipping Ollama integration test"
)
def test_ollama_sync_ollama():
"""Test sync with Ollama model."""
result = _test_model_sync_helper("ollama/llama3.2")
assert result, "Ollama sync model test failed"
async def main():
print("Testing Ollama empty response fix for both sync and async...")
# Test sync methods
print("\n1. Testing SYNC methods:")
openai_sync_success = test_model_sync("openai/gpt-5-nano")
ollama_sync_success = test_model_sync("ollama/llama3.2")
openai_sync_success = _test_model_sync_helper("gpt-4o-mini")
ollama_sync_success = _test_model_sync_helper("ollama/llama3.2")
# Test async methods
print("\n2. Testing ASYNC methods:")
openai_async_success = await test_model_async("openai/gpt-5-nano")
ollama_async_success = await test_model_async("ollama/llama3.2")
openai_async_success = await _test_model_async_helper("gpt-4o-mini")
ollama_async_success = await _test_model_async_helper("ollama/llama3.2")
# Summary
print(f"\n{'='*60}")
+27 -5
View File
@@ -4,7 +4,9 @@ Test script to verify Ollama empty response fix.
"""
import logging
from praisonaiagents import Agent, Task, PraisonAIAgents, TaskOutput
import os
import pytest
from praisonaiagents import Agent, Task, PraisonAIAgents
from typing import Dict, Any
# Enable debug logging
@@ -24,8 +26,8 @@ def search_tool(query: str) -> Dict[str, Any]:
logging.debug(f"[TEST] Search tool returning: {results}")
return results
# Test with both OpenAI and Ollama models
def test_model(model_name: str):
# Helper function to test a model
def _test_model_helper(model_name: str):
print(f"\n{'='*60}")
print(f"Testing with model: {model_name}")
print('='*60)
@@ -77,16 +79,36 @@ def test_model(model_name: str):
traceback.print_exc()
return False
@pytest.mark.skipif(
not os.environ.get("OPENAI_API_KEY"),
reason="OPENAI_API_KEY not set - skipping integration test"
)
def test_ollama_fix_openai():
"""Test with OpenAI model as baseline."""
result = _test_model_helper("gpt-4o-mini")
assert result, "OpenAI model test failed"
@pytest.mark.skipif(
not os.environ.get("OLLAMA_HOST", "").strip(),
reason="OLLAMA_HOST not set - skipping Ollama integration test"
)
def test_ollama_fix_ollama():
"""Test with Ollama model."""
result = _test_model_helper("ollama/llama3.2")
assert result, "Ollama model test failed"
if __name__ == "__main__":
print("Testing Ollama empty response fix...")
# Test with OpenAI first (as baseline)
print("\n1. Testing with OpenAI (baseline):")
openai_success = test_model("openai/gpt-5-nano")
openai_success = _test_model_helper("gpt-4o-mini")
# Test with Ollama
print("\n2. Testing with Ollama:")
ollama_success = test_model("ollama/llama3.2")
ollama_success = _test_model_helper("ollama/llama3.2")
# Summary
print(f"\n{'='*60}")
@@ -53,11 +53,14 @@ def test_ollama_argument_validation():
print("\n2. Testing mixed arguments (the main issue):")
mixed_args = {"a": "get_stock_price", "company_name": "Google", "b": "2"}
filtered_args = llm._validate_and_filter_ollama_arguments("multiply", mixed_args, tools)
expected_filtered = {"a": "get_stock_price", "b": "2"} # Should remove 'company_name'
# Note: The method may coerce "2" to int 2 based on type hints - both are acceptable
expected_filtered_str = {"a": "get_stock_price", "b": "2"} # String version
expected_filtered_int = {"a": "get_stock_price", "b": 2} # Int version (coerced)
print(f"Original: {mixed_args}")
print(f"Filtered: {filtered_args}")
print(f"Expected: {expected_filtered}")
assert filtered_args == expected_filtered, f"Expected {expected_filtered}, got {filtered_args}"
print(f"Expected: {expected_filtered_str} or {expected_filtered_int}")
assert filtered_args == expected_filtered_str or filtered_args == expected_filtered_int, \
f"Expected {expected_filtered_str} or {expected_filtered_int}, got {filtered_args}"
print("✅ Mixed arguments filtering test passed")
# Test case 3: All invalid arguments
+6 -3
View File
@@ -704,7 +704,8 @@ class TestPlanningAgent:
planner = PlanningAgent()
mock_llm = MagicMock()
mock_llm.achat = AsyncMock(return_value=json.dumps({
# Mock get_response_async which is the method actually called
mock_llm.get_response_async = AsyncMock(return_value=json.dumps({
"name": "Test Plan",
"description": "A test plan",
"steps": [
@@ -733,7 +734,8 @@ class TestPlanningAgent:
planner = PlanningAgent()
mock_llm = MagicMock()
mock_llm.achat = AsyncMock(return_value=json.dumps({
# Mock get_response_async which is the method actually called
mock_llm.get_response_async = AsyncMock(return_value=json.dumps({
"name": "Refined Plan",
"description": "Updated plan",
"steps": [
@@ -765,7 +767,8 @@ class TestPlanningAgent:
planner = PlanningAgent()
mock_llm = MagicMock()
mock_llm.achat = AsyncMock(return_value="Analysis: Python project with FastAPI")
# Mock get_response_async which is the method actually called
mock_llm.get_response_async = AsyncMock(return_value="Analysis: Python project with FastAPI")
planner._llm = mock_llm
@@ -2,7 +2,29 @@
"""Comprehensive test script to verify self-reflection works with tools after the fix"""
from praisonaiagents import Agent, Task, PraisonAIAgents
from praisonaiagents.tools import duckduckgo_search
# Define duckduckgo_search tool locally to avoid import issues
def duckduckgo_search(query: str) -> str:
"""
Search the web using DuckDuckGo.
Args:
query: The search query string
Returns:
Search results as a string
"""
try:
from duckduckgo_search import DDGS
with DDGS() as ddgs:
results = list(ddgs.text(query, max_results=3))
if results:
return "\n".join([f"- {r.get('title', '')}: {r.get('body', '')}" for r in results])
return "No results found"
except ImportError:
return f"Mock search results for: {query}"
except Exception as e:
return f"Search error: {str(e)}"
def test_self_reflection_with_tools():
"""Test self-reflection with tools - should work after the fix"""
@@ -2,7 +2,23 @@
"""Simple test to verify the self-reflection fix works"""
from praisonaiagents import Agent
from praisonaiagents.tools import calculator
# Define calculator tool locally to avoid import issues
def calculator(expression: str) -> str:
"""
Evaluate a mathematical expression.
Args:
expression: A mathematical expression to evaluate (e.g., "2 + 2", "25 * 17")
Returns:
The result of the calculation as a string
"""
try:
result = eval(expression)
return str(result)
except Exception as e:
return f"Error: {str(e)}"
def test_self_reflection_fix():
"""Test that self-reflection works with tools after the fix"""
@@ -2,7 +2,23 @@
"""Test to verify the self-reflection fix works with tools"""
from praisonaiagents import Agent, Task, PraisonAIAgents
from praisonaiagents.tools import calculator
# Define calculator tool locally to avoid import issues
def calculator(expression: str) -> str:
"""
Evaluate a mathematical expression.
Args:
expression: A mathematical expression to evaluate (e.g., "123 * 456")
Returns:
The result of the calculation as a string
"""
try:
result = eval(expression)
return str(result)
except Exception as e:
return f"Error: {str(e)}"
def test_self_reflection_fix_verification():
"""Test that self-reflection now works with tools"""
@@ -0,0 +1,397 @@
"""
Test suite for Workflow Per-Step Agent Configuration.
Tests the enhanced workflow system's ability to configure different agents
for each step, including custom roles, goals, backstories, and LLM settings.
"""
import pytest
from unittest.mock import Mock, patch
import sys
import os
# Add the package to path for testing
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
from praisonaiagents.memory.workflows import (
WorkflowStep,
Workflow,
WorkflowManager
)
class TestWorkflowStepAgentFields:
"""Test WorkflowStep dataclass agent-related fields."""
def test_workflow_step_has_agent_config_field(self):
"""WorkflowStep should have agent_config field."""
step = WorkflowStep(
name="test_step",
action="do something",
agent_config={
"role": "Researcher",
"goal": "Find information",
"backstory": "Expert researcher"
}
)
assert hasattr(step, 'agent_config')
assert step.agent_config["role"] == "Researcher"
def test_workflow_step_has_tools_field(self):
"""WorkflowStep should have tools field."""
step = WorkflowStep(
name="test_step",
action="do something",
tools=["search", "calculator"]
)
assert hasattr(step, 'tools')
assert "search" in step.tools
def test_workflow_step_agent_config_default_none(self):
"""agent_config should default to None."""
step = WorkflowStep(name="test_step", action="do something")
assert step.agent_config is None
def test_workflow_step_tools_default_none(self):
"""tools should default to None."""
step = WorkflowStep(name="test_step", action="do something")
assert step.tools is None
def test_workflow_step_to_dict_includes_agent_fields(self):
"""to_dict() should include agent-related fields."""
step = WorkflowStep(
name="test_step",
action="do something",
agent_config={"role": "Writer"},
tools=["grammar_check"]
)
d = step.to_dict()
assert "agent_config" in d
assert "tools" in d
class TestWorkflowDefaultAgentConfig:
"""Test Workflow dataclass default agent configuration."""
def test_workflow_has_default_agent_config(self):
"""Workflow should have default_agent_config field."""
workflow = Workflow(
name="test_workflow",
default_agent_config={
"role": "Assistant",
"goal": "Help with tasks"
}
)
assert hasattr(workflow, 'default_agent_config')
assert workflow.default_agent_config["role"] == "Assistant"
def test_workflow_has_default_llm(self):
"""Workflow should have default_llm field."""
workflow = Workflow(
name="test_workflow",
default_llm="gpt-4o-mini"
)
assert hasattr(workflow, 'default_llm')
assert workflow.default_llm == "gpt-4o-mini"
def test_workflow_to_dict_includes_defaults(self):
"""to_dict() should include default agent config fields."""
workflow = Workflow(
name="test_workflow",
default_agent_config={"role": "Helper"},
default_llm="gpt-4o"
)
d = workflow.to_dict()
assert "default_agent_config" in d
assert "default_llm" in d
class TestCreateStepAgent:
"""Test _create_step_agent method."""
@staticmethod
def _register_workflow(manager, workflow):
"""Helper to register workflow and mark as loaded."""
manager._workflows[workflow.name.lower()] = workflow
manager._loaded = True
def test_create_step_agent_with_config(self):
"""Should create agent from step's agent_config."""
manager = WorkflowManager()
step = WorkflowStep(
name="research",
action="Research topic",
agent_config={
"role": "Researcher",
"goal": "Find comprehensive information",
"backstory": "Expert in research"
}
)
with patch('praisonaiagents.agent.agent.Agent') as MockAgent:
MockAgent.return_value = Mock()
agent = manager._create_step_agent(
step=step,
default_llm="gpt-4o-mini",
verbose=0
)
# Verify Agent was called with config
MockAgent.assert_called_once()
call_kwargs = MockAgent.call_args[1]
assert call_kwargs["role"] == "Researcher"
assert call_kwargs["goal"] == "Find comprehensive information"
def test_create_step_agent_uses_default_agent(self):
"""Should return default_agent when no agent_config."""
manager = WorkflowManager()
step = WorkflowStep(
name="simple_step",
action="Do something"
)
default_agent = Mock()
agent = manager._create_step_agent(
step=step,
default_agent=default_agent,
default_llm="gpt-4o-mini"
)
assert agent == default_agent
def test_create_step_agent_includes_tools(self):
"""Should include tools in agent config."""
manager = WorkflowManager()
mock_tool = Mock()
step = WorkflowStep(
name="research",
action="Research topic",
agent_config={"role": "Researcher"},
tools=[mock_tool]
)
with patch('praisonaiagents.agent.agent.Agent') as MockAgent:
MockAgent.return_value = Mock()
manager._create_step_agent(
step=step,
default_llm="gpt-4o-mini"
)
call_kwargs = MockAgent.call_args[1]
assert "tools" in call_kwargs
assert mock_tool in call_kwargs["tools"]
def test_create_step_agent_includes_memory(self):
"""Should include memory in agent config."""
manager = WorkflowManager()
step = WorkflowStep(
name="research",
action="Research topic",
agent_config={"role": "Researcher"}
)
mock_memory = Mock()
with patch('praisonaiagents.agent.agent.Agent') as MockAgent:
MockAgent.return_value = Mock()
manager._create_step_agent(
step=step,
memory=mock_memory
)
call_kwargs = MockAgent.call_args[1]
assert "memory" in call_kwargs
assert call_kwargs["memory"] == mock_memory
class TestExecuteWithAgents:
"""Test execute() with per-step agent configuration."""
@staticmethod
def _register_workflow(manager, workflow):
"""Helper to register workflow and mark as loaded."""
manager._workflows[workflow.name.lower()] = workflow
manager._loaded = True
def test_execute_with_default_agent(self):
"""Execute should use default_agent when provided."""
manager = WorkflowManager()
workflow = Workflow(
name="test_workflow",
steps=[
WorkflowStep(name="step1", action="Action 1"),
WorkflowStep(name="step2", action="Action 2")
]
)
self._register_workflow(manager, workflow)
# Create mock default agent
mock_agent = Mock()
mock_agent.chat = Mock(return_value="Agent response")
result = manager.execute(
"test_workflow",
default_agent=mock_agent
)
assert result["success"]
assert mock_agent.chat.call_count == 2
def test_execute_with_step_agent_config(self):
"""Execute should create agent from step's agent_config."""
manager = WorkflowManager()
workflow = Workflow(
name="test_workflow",
steps=[
WorkflowStep(
name="research",
action="Research AI",
agent_config={
"role": "Researcher",
"goal": "Find information"
}
)
]
)
self._register_workflow(manager, workflow)
with patch('praisonaiagents.agent.agent.Agent') as MockAgent:
mock_agent_instance = Mock()
mock_agent_instance.chat = Mock(return_value="Research results")
MockAgent.return_value = mock_agent_instance
result = manager.execute(
"test_workflow",
default_llm="gpt-4o-mini"
)
assert result["success"]
MockAgent.assert_called_once()
def test_execute_with_executor_overrides_agent(self):
"""Executor function should override agent creation."""
manager = WorkflowManager()
workflow = Workflow(
name="test_workflow",
steps=[
WorkflowStep(
name="step1",
action="Action 1",
agent_config={"role": "Should not be used"}
)
]
)
self._register_workflow(manager, workflow)
executor_called = [False]
def custom_executor(action):
executor_called[0] = True
return "Custom executor result"
result = manager.execute(
"test_workflow",
executor=custom_executor
)
assert result["success"]
assert executor_called[0]
def test_execute_fails_without_executor_or_agent(self):
"""Execute should fail if no executor or agent provided."""
manager = WorkflowManager()
workflow = Workflow(
name="test_workflow",
steps=[
WorkflowStep(name="step1", action="Action 1")
]
)
self._register_workflow(manager, workflow)
result = manager.execute("test_workflow")
assert not result["success"]
# Error can be at top level or in results array
error_found = "No executor available" in result.get("error", "")
if not error_found and "results" in result:
error_found = any(
"No executor available" in r.get("error", "")
for r in result.get("results", [])
)
assert error_found, f"Expected 'No executor available' error, got: {result}"
class TestWorkflowLevelDefaults:
"""Test workflow-level default configurations."""
@staticmethod
def _register_workflow(manager, workflow):
"""Helper to register workflow and mark as loaded."""
manager._workflows[workflow.name.lower()] = workflow
manager._loaded = True
def test_workflow_default_llm_used(self):
"""Workflow's default_llm should be used for agent creation."""
manager = WorkflowManager()
workflow = Workflow(
name="test_workflow",
default_llm="gpt-4o",
steps=[
WorkflowStep(
name="step1",
action="Action 1",
agent_config={"role": "Helper"}
)
]
)
self._register_workflow(manager, workflow)
with patch('praisonaiagents.agent.agent.Agent') as MockAgent:
mock_agent_instance = Mock()
mock_agent_instance.chat = Mock(return_value="Response")
MockAgent.return_value = mock_agent_instance
manager.execute("test_workflow")
call_kwargs = MockAgent.call_args[1]
assert call_kwargs.get("llm") == "gpt-4o"
def test_execute_default_llm_overrides_workflow(self):
"""Execute's default_llm should override workflow's default_llm."""
manager = WorkflowManager()
workflow = Workflow(
name="test_workflow",
default_llm="gpt-3.5-turbo",
steps=[
WorkflowStep(
name="step1",
action="Action 1",
agent_config={"role": "Helper"}
)
]
)
self._register_workflow(manager, workflow)
with patch('praisonaiagents.agent.agent.Agent') as MockAgent:
mock_agent_instance = Mock()
mock_agent_instance.chat = Mock(return_value="Response")
MockAgent.return_value = mock_agent_instance
manager.execute("test_workflow", default_llm="gpt-4o-mini")
call_kwargs = MockAgent.call_args[1]
assert call_kwargs.get("llm") == "gpt-4o-mini"
if __name__ == "__main__":
pytest.main([__file__, "-v"])
Loaded 100 of 128 files, more files were not shown because too many files have changed in this diff. Show more