Compare commits

...
225 Commits
Author SHA1 Message Date
Mervin Praison 95ffd0c53f chore: bump PraisonAI to 2.2.55 and PraisonAI Agents to 0.0.129 across all files 2025-07-14 11:09:35 +01:00
Mervin Praison e1c90bf569 Merge pull request #880 from MervinPraison/claude/issue-866-20250713_204435
feat: Add multi-provider/multi-model support with intelligent routing
2025-07-14 11:07:02 +01:00
Mervin Praison 5fbd8aa783 Merge pull request #892 from MervinPraison/claude/pr-879-20250714_063758
fix: Improve Ollama sequential tool calling handling
2025-07-14 11:02:12 +01:00
github-actions[bot]andMervin Praison d79443868f refactor: Rename MultiModelAgent to RouterAgent for better clarity
- Renamed MultiModelAgent class to RouterAgent throughout codebase
- Updated multi_model_agent.py -> router_agent.py
- Updated all imports, exports, and references in:
  - src/praisonai-agents/praisonaiagents/agent/__init__.py
  - examples/python/agents/multi-provider-agent.py
  - examples/python/agents/MULTI_PROVIDER_README.md
  - src/praisonai-agents/test_multi_provider.py
- RouterAgent better reflects the primary purpose of intelligent model routing
- Maintains all existing functionality and backward compatibility

Co-authored-by: Mervin Praison <[email protected]>
2025-07-14 08:50:14 +00:00
github-actions[bot]andMervin Praison 6705e753e8 fix: Remove hardcoded regex logic from Ollama tool calling
- Remove explicitly prohibited hardcoded regex patterns
- Move import re to top of file (PEP 8 compliance)
- Extract duplicated message-building logic into helper method
- Define prompt constants at class level for maintainability
- Simplify tool result formatting without numeric extraction
- Maintain backward compatibility and existing functionality

Fixes code quality issues identified by automated review tools
while ensuring compliance with explicit PR requirements.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-14 08:49:03 +00:00
Mervin Praison b4d1532512 ci: prevent bot accounts from triggering Claude workflow action 2025-07-14 08:51:51 +01:00
Mervin Praison 4b5d3b2eda ci: remove max_turns limit from Claude code review workflow 2025-07-14 08:50:09 +01:00
Mervin Praison a1d4f3ee43 feat: add repository checkout step with fetch history in Claude workflow 2025-07-14 08:23:09 +01:00
Mervin Praison b5e930ff2e Merge pull request #889 from MervinPraison/claude/issue-873-20250714_005616
fix: use LLM abstraction instead of hardcoded OpenAI in hierarchical process
2025-07-14 08:13:08 +01:00
Mervin Praison 1a6a14f8f2 Merge pull request #887 from MervinPraison/claude/issue-878-20250714_005448
fix: prevent duplicate callback invocations in LLM response generation
2025-07-14 08:11:52 +01:00
Mervin Praison ca69de4c6a Merge pull request #885 from MervinPraison/claude/issue-875-20250714_005802
fix: Support GuardrailResult return type in guardrail functions (#875)
2025-07-14 08:09:05 +01:00
Mervin Praison 9b3d36e6d7 ci: remove redundant repository checkout step from Claude workflow 2025-07-14 08:07:07 +01:00
Mervin Praison a9359a0785 Merge branch 'main' of https://github.com/MervinPraison/PraisonAI 2025-07-14 08:05:10 +01:00
Mervin Praison 4cb016c3ca feat: add Gemini structured output agent and enhance Claude workflow checkout 2025-07-14 08:05:08 +01:00
Mervin Praison d2e4776884 Merge pull request #888 from MervinPraison/claude/issue-871-20250714_005124
feat: Add Gemini structured output support
2025-07-14 07:52:18 +01:00
claude[bot]andMervin Praison 193b82aeba fix: Improve Ollama sequential tool calling handling
- Extract numeric values from tool results for better clarity
- Add explicit prompting for final answers after tool execution
- Handle empty initial responses from Ollama with retry logic
- Fix duplicate strip calls in response handling

This improves Ollama's ability to:
1. Understand numeric values from tool results
2. Generate proper final answers instead of returning None
3. Avoid getting stuck in infinite loops

Fixes the issue where Ollama was unable to properly chain tool calls
for sequential operations like getting a stock price and multiplying it.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-14 06:42:14 +00:00
Mervin Praison 26376a77af Merge pull request #890 from Dhivya-Bharathy/grok4-agents-batch-upload
feat: Add 10 Grok4 Intelligent Agent Examples
2025-07-14 07:31:17 +01:00
devops-vestradata 65791781e2 Add 10 grok4 intelligent agent files 2025-07-14 11:56:23 +05:30
claude[bot]andMervin Praison ee66154b73 fix: ensure interaction_displayed flag prevents all duplicate callbacks
- Added missing interaction_displayed checks to reasoning steps display
- Added missing checks to self-reflection display logic
- Ensured flag is set after each display interaction
- Fixed both sync and async methods
- Added comprehensive test to verify the fix

Co-authored-by: Mervin Praison <[email protected]>
2025-07-14 06:24:34 +00:00
claude[bot]andMervin Praison 2daae1f464 fix: Address critical issues with Gemini structured output implementation
- Add hasattr check before calling model_json_schema() to prevent AttributeError
- Centralize Gemini model detection logic with new _is_gemini_model() method
- Fix critical bug: Always remove output_json/output_pydantic from params before passing to litellm
- Fix model recognition to handle provider-prefixed models (e.g., 'google/gemini-1.5-pro')
- Ensure backward compatibility for all existing models

Co-authored-by: Mervin Praison <[email protected]>
2025-07-14 06:23:29 +00:00
claude[bot]andMervin Praison 81bcda0440 fix: correct LLM interface usage and improve code quality
- Remove duplicate output_json=True when using output_pydantic
- Add helper methods to reduce code duplication
- Improve test file to use proper assertions instead of sys.exit()
- Add unit tests for the new helper methods
- Fix TypeError handling in JSON parsing

This ensures the hierarchical workflow works correctly with all LLM providers.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-14 06:23:13 +00:00
claude[bot]andMervin Praison e269cc7bf2 fix: Improve guardrail validation and add missing type hints
- Fixed critical bug: validation now properly checks for GuardrailResult type instead of accepting any class with the same name
- Added missing TaskOutput type hints in example file
- Refactored conditional logic for better readability
- Maintains full backward compatibility

Co-authored-by: Mervin Praison <[email protected]>
2025-07-14 06:21:30 +00:00
Mervin Praison 66e5dfcb41 refactor: update Claude GitHub Action with simplified OAuth authentication and improved triggers 2025-07-14 03:08:02 +01:00
Mervin Praison ae3e69a286 feat: exclude @gemini triage command from issue review workflow 2025-07-14 02:48:11 +01:00
Mervin Praison 40ea6fe1aa ci: simplify Gemini trigger command from '@gemini review' to '@gemini' 2025-07-14 02:46:58 +01:00
Mervin Praison e30a7564be Merge pull request #879 from MervinPraison/claude/issue-854-20250713_205120
fix: Implement Ollama sequential tool calling fix
2025-07-14 02:20:07 +01:00
claude[bot]andMervin Praison fbd0a736f0 fix: Fix critical issues in multi-provider implementation
- Changed TaskComplexity from string Enum to IntEnum for proper ordering comparisons
- Fixed manual routing logic to use llm_model property correctly
- Removed private attribute access by storing api_key and base_url locally
- Improved context size estimation from char count to token approximation
- Removed unused imports (Callable, TaskComplexity, ModelProfile)
- Fixed unnecessary f-string in error message
- Added TODO for token tracking implementation
- Fixed inconsistent model names in documentation

Co-authored-by: Mervin Praison <[email protected]>
2025-07-14 01:16:18 +00:00
claude[bot]andMervin Praison eea9f757af fix: use LLM abstraction instead of hardcoded OpenAI in hierarchical process
- Replace direct OpenAI client instantiation with LLM class
- Support all LLM providers (Gemini, Anthropic, etc.) in manager tasks
- Remove unused OpenAI imports from process.py
- Fix issue #873 where hierarchical workflow required OpenAI API key

Co-authored-by: Mervin Praison <[email protected]>
2025-07-14 01:08:57 +00:00
claude[bot]andMervin Praison 02a8f075a6 fix: prevent duplicate callback invocations in LLM response generation
- Add interaction_displayed flag to track display_interaction calls
- Prevent duplicate callbacks in both sync and async get_response methods
- Fix addresses issue where callbacks fired twice for single task execution
- Maintain backward compatibility with no API changes

Fixes #878

Co-authored-by: Mervin Praison <[email protected]>
2025-07-14 01:08:54 +00:00
claude[bot]andMervin Praison 57c39653d6 feat: Add Gemini structured output support
- Add Gemini models to MODELS_SUPPORTING_STRUCTURED_OUTPUTS
- Implement native Gemini structured output using response_schema
- Maintain backward compatibility for OpenAI and other models
- Update _build_messages to skip prompt modification for Gemini
- Add response_mime_type and response_schema parameters for Gemini
- Pass output_json/output_pydantic through all completion calls

Fixes #871

Co-authored-by: Mervin Praison <[email protected]>
2025-07-14 01:06:59 +00:00
claude[bot]andMervin Praison 2f21d0f907 fix: Support GuardrailResult return type in guardrail functions (#875)
- Modified task.py to accept both GuardrailResult and Tuple[bool, Any] return types
- Updated _process_guardrail to handle GuardrailResult directly without conversion
- Added comprehensive test cases for both return types
- Updated documentation with GuardrailResult examples
- Maintains backward compatibility with existing Tuple[bool, Any] guardrails

Co-authored-by: Mervin Praison <[email protected]>
2025-07-14 01:05:39 +00:00
claude[bot]andMervin Praison 4564639f18 enhance: Improve Ollama provider detection
- Add check for "ollama" in base_url for custom configurations
- Support detection of custom Ollama ports beyond 11434
- More robust detection for various deployment scenarios

Co-authored-by: Mervin Praison <[email protected]>
2025-07-14 00:58:57 +00:00
claude[bot]andMervin Praison b5d13a2b7f fix: Implement Ollama sequential tool calling fix
Modified conversation format for Ollama to handle tool results properly:
- Assistant messages no longer include 'tool_calls' field for Ollama
- Tool results are passed as user messages with natural language format
- Prevents infinite loop of repeated tool calls

This allows Ollama to properly execute sequential tool calls as intended.

Fixes #854

Co-authored-by: Mervin Praison <[email protected]>
2025-07-13 20:55:34 +00:00
claude[bot]andMervin Praison 4622bcda43 feat: Add multi-provider/multi-model support with intelligent routing
- Add ModelRouter for intelligent model selection based on task complexity
- Create MultiModelAgent class supporting multiple models per agent
- Implement routing strategies: auto, cost-optimized, performance-optimized
- Add pre-configured profiles for 8+ popular models
- Include usage tracking and cost estimation
- Maintain full backward compatibility
- Add comprehensive examples and documentation

Fixes #866

Co-authored-by: Mervin Praison <[email protected]>
2025-07-13 20:53:06 +00:00
Mervin Praison 31bae3a231 chore: bump PraisonAI to 2.2.54 and update Gemini workflows 2025-07-13 21:49:21 +01:00
Mervin Praison 466bcfedd7 Merge pull request #863 from MervinPraison/gemini/issue-862-fix-async-loop
Fix #862: Agent gets stuck in infinite loop when running multiple async tasks
2025-07-13 20:46:57 +01:00
Mervin Praison c24bf80314 Merge pull request #858 from MervinPraison/gemini/issue-854-fix-ollama-sequential-1690000000
Fix #854: Ollama sequential still failing
2025-07-13 20:46:21 +01:00
Mervin Praison c36b3aee6e Merge pull request #865 from MervinPraison/gemini/issue-864-fix-parallel-task-loop
Fix #864: Fix parallel task loop
2025-07-13 20:45:55 +01:00
Mervin Praison 08512ba6d3 Merge pull request #861 from MervinPraison/gemini/issue-860-fix-hierarchical-llm-fallback
Fix #860: Hierarchical Process LLM Fallback
2025-07-13 20:43:53 +01:00
claude[bot]andMervin Praison 63ebfc0b54 fix: Complete removal of Ollama special handling for async method
This completes the fix for issue #854 by removing the Ollama special
handling from the async method (get_response_async) to match the sync
method fix. Also removes the now-unused _handle_ollama_model method.

The special handling was causing infinite loops in sequential tool
calling. By removing it, Ollama models now work correctly through
the standard litellm interface.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-13 19:21:40 +00:00
claude[bot]andMervin Praison b29269071d fix: properly handle parallel task reset logic to prevent infinite loops
- Uncommented the task reset logic that was causing issue #864
- Added check for async_execution to prevent resetting parallel tasks
- Preserves backward compatibility for non-parallel tasks that need rerun
- Fixes infinite loop issue while maintaining existing functionality

Co-authored-by: Mervin Praison <[email protected]>
2025-07-13 19:14:59 +00:00
Gemini Bot b6a86bf348 Gemini: [Issue #864] Fix parallel task loop 2025-07-13 12:49:40 +00:00
Gemini Bot 02ba71be3d Gemini: [Issue #862] Fix infinite loop in async workflow 2025-07-13 12:17:37 +00:00
Gemini Bot e60f1c1aa1 Gemini: [Issue #860] Fix hierarchical process LLM fallback 2025-07-13 11:31:08 +00:00
Mervin Praison 450f237805 feat: add description field to TaskOutput in validation feedback tests 2025-07-13 01:17:26 +01:00
Mervin Praison 66e53fc82e refactor: move validation test files to src directory and update LLM model references 2025-07-13 01:09:52 +01:00
Mervin Praison 7e6ee5f9f2 chore: enhance variable handling in PR review workflow for improved clarity 2025-07-13 01:04:41 +01:00
Mervin Praison b669b9815d fix: correct variable interpolation syntax in PR review workflow 2025-07-13 00:50:39 +01:00
Mervin Praison 6d89cd603e chore: remove debug workflow trigger step from Gemini PR review action 2025-07-13 00:44:38 +01:00
Mervin Praison 6e955b5941 feat: restrict Gemini PR review to repo owners, members and collaborators 2025-07-13 00:34:48 +01:00
Mervin Praison 87a67b5f5d refactor: improve PR checkout logic in Gemini review workflow using dedicated step 2025-07-13 00:25:08 +01:00
Mervin Praison baec9cd0a4 fix: update checkout ref to handle both PR and issue comment triggers 2025-07-13 00:23:16 +01:00
Mervin Praison e144dc497d feat: allow all users to trigger Gemini PR review and add debug logging 2025-07-13 00:19:06 +01:00
Gemini Bot 8a77ab0ab5 Gemini: [Issue #854] Fix Ollama sequential tool calling loop 2025-07-12 23:10:23 +00:00
Mervin Praison 0034b5c964 Merge branch 'main' of https://github.com/MervinPraison/PraisonAI 2025-07-13 00:04:14 +01:00
Mervin Praison 543c627374 feat: expand GitHub Actions permissions to include PR list and granular git commands 2025-07-13 00:04:13 +01:00
Mervin Praison 93154adf9e Merge pull request #857 from MervinPraison/gemini/issue-854-fix-ollama-sequential-fail
Fix #854: Ollama sequential tool calling failing
2025-07-13 00:03:55 +01:00
Gemini Bot 3ab2462d96 Gemini: [Issue #854] Fix Ollama sequential tool calling loop 2025-07-12 23:00:05 +00:00
Mervin Praison aecf65a130 Merge branch 'main' of https://github.com/MervinPraison/PraisonAI 2025-07-12 23:52:47 +01:00
Mervin Praison 0a453d30a2 chore: add .gemini and issue_details.json to gitignore 2025-07-12 23:52:46 +01:00
Mervin Praison 354a2c717b Merge pull request #856 from MervinPraison/gemini/issue-854-fix-ollama-sequential
Fix #854: Ollama sequential tool calling failing
2025-07-12 23:52:38 +01:00
Mervin Praison 62514fbef4 Delete issue_details.json 2025-07-12 23:51:43 +01:00
Mervin Praison f3c77a2e8b Delete .gemini/settings.json 2025-07-12 23:51:32 +01:00
Gemini Bot 50dde4fa02 Gemini: [Issue #854] Fix Ollama sequential tool calling 2025-07-12 22:45:09 +00:00
Mervin Praison bbc4530329 chore: bump version to 2.2.53 and update dependencies across package files 2025-07-12 23:43:01 +01:00
Mervin Praison aab975bec7 Merge pull request #851 from MervinPraison/claude/issue-846-20250712_220811
fix: handle None response_text in Ollama sequential tool calls
2025-07-12 23:28:38 +01:00
Mervin Praison 50c50fa567 Merge pull request #849 from MervinPraison/claude/issue-845-20250712_183232
fix: enable sequential tool execution in OpenAI client
2025-07-12 23:27:54 +01:00
claude[bot]andMervin Praison 68f280634a refactor: remove dead code from sequential tool execution logic
- Remove unused should_continue variable and related logic
- Simplify code by removing special handling for sequentialthinking tool
- Add clear comments explaining the sequential execution behavior

Co-authored-by: Mervin Praison <[email protected]>
2025-07-12 22:16:05 +00:00
Mervin Praison 346488fbec Delete test_sequential_fix.py 2025-07-12 23:15:10 +01:00
claude[bot]andMervin Praison 57b5a1fd8d fix: handle None response_text in Ollama sequential tool calls
- Add None checks before calling .strip() on response_text
- Fixes 'NoneType' object has no attribute 'strip' error with Ollama models
- Applied pattern: response_text.strip() if response_text else ""
- Fixed 14 locations where .strip() was called without None checks
- Ensures backward compatibility by returning empty string instead of None

Fixes issue #846

Co-authored-by: Mervin Praison <[email protected]>
2025-07-12 22:11:55 +00:00
claude[bot]andMervin Praison d75a5bb728 test: add assertions to test_sequential_fix.py for automated testing
- Add proper assertions to verify sequential tool execution results
- Improve multiply function docstring with complete Args and Returns sections
- Add success message when all assertions pass

Co-authored-by: Mervin Praison <[email protected]>
2025-07-12 22:10:09 +00:00
Mervin Praison 519ecc025a feat: update allowed commands in GitHub workflow files for Gemini reviews 2025-07-12 21:21:23 +01:00
Mervin Praison 8290b95132 refactor: update issue review prompt with clearer instructions and dynamic commands 2025-07-12 21:01:13 +01:00
Mervin Praison 3d95e1ff0e feat: add read_file permission to Gemini PR review workflow 2025-07-12 20:55:41 +01:00
Mervin Praison 74405d1bbe feat: add read_file permission to Gemini issue review workflow 2025-07-12 20:54:59 +01:00
Mervin Praison d307c3857c feat: add environment variables for Gemini CLI action in GitHub workflow 2025-07-12 20:54:31 +01:00
Mervin Praison fbe5dddaae chore: disable telemetry in GitHub workflow configurations 2025-07-12 20:45:20 +01:00
Mervin Praison 7e4108918d fix: move GH_TOKEN to local env scope in GitHub workflow 2025-07-12 20:40:04 +01:00
Mervin Praison da7e38100e feat: add GitHub workflow for automated issue review using Gemini AI 2025-07-12 20:36:37 +01:00
Mervin Praison 848459b816 feat: add edit, web-fetch, web-search and memoryTool permissions while removing telemetry setup 2025-07-12 20:35:19 +01:00
Mervin Praison cd63845398 feat: add git command to allowed shell commands in PR review workflow 2025-07-12 20:31:18 +01:00
Mervin Praison 7827427dc0 feat: move Gemini PR review action to dedicated directory with configuration 2025-07-12 20:16:47 +01:00
Mervin Praison 0e378d0754 feat: add GitHub workflows for Gemini issue triage and PR review automation 2025-07-12 20:03:05 +01:00
Mervin Praison 206022eb9c refactor: simplify auto-comment workflows by removing multi-agent review requirement 2025-07-12 19:48:23 +01:00
Mervin Praison 901ba8074d Delete test_sequential_fix.py 2025-07-12 19:40:42 +01:00
claude[bot]andMervin Praison 953ee6e7bd fix: enable sequential tool execution in OpenAI client
Removes premature loop termination after tool execution to allow
sequential tool calls. The model can now see tool results and make
additional tool calls as needed.

Fixes #845

Co-authored-by: Mervin Praison <[email protected]>
2025-07-12 18:36:00 +00:00
Mervin Praison 181d7e1ede feat: bump PraisonAI to v2.2.52 and add sequential agent examples for OpenAI/Ollama 2025-07-12 19:12:05 +01:00
Mervin Praison 4039878422 Merge pull request #842 from MervinPraison/claude/issue-839-20250712_075350
fix: allow agents to continue with sequential tool calls
2025-07-12 19:04:25 +01:00
Mervin Praison 461eba18d5 Delete src/praisonai/tests/unit/test_sequential_tool_calling.py 2025-07-12 19:03:44 +01:00
claude[bot]andMervin Praison 331a1de610 test: add comprehensive test suite for sequential tool calling
This test file validates the fix for issue #839, ensuring agents can:
- Execute multiple tools in sequence
- Pass results between tool calls
- Return final combined results without empty responses

Tests include:
- Sequential execution of 2 and 3 tools
- Multiple agents with sequential tools
- Mocked LLM responses
- Edge case prevention for empty responses

Co-authored-by: Mervin Praison <[email protected]>
2025-07-12 18:00:27 +00:00
claude[bot]andMervin Praison 91632eb5e4 fix: correct Ollama return statement and variable reference for sequential tool calling
- Replace early return with loop continuation in Ollama handling to match async behavior
- Fix variable reference from reasoning_content to stored_reasoning_content
- Ensures agents can perform sequential tool calls as intended

Co-authored-by: Mervin Praison <[email protected]>
2025-07-12 17:54:53 +00:00
Mervin Praison ad46225483 Rename src/praisonai/tests/unit/test_fix_comprehensive.py to src/praisonai-agents/tests/test_fix_comprehensive.py 2025-07-12 18:50:30 +01:00
Mervin Praison 9af6758eb6 Delete test_fix.py 2025-07-12 18:48:37 +01:00
Mervin Praison aac67a6572 Rename test_fix_comprehensive.py to src/praisonai/tests/unit/test_fix_comprehensive.py 2025-07-12 18:47:35 +01:00
claude[bot]andMervin Praison 2ceff9bfae fix: correct indentation error in async streaming code
Fixed SyntaxError caused by incorrect indentation in the async version of get_response_async method. The if/else blocks for streaming vs non-streaming approaches now have proper indentation.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-12 17:41:09 +00:00
claude[bot]andMervin Praison 3b7af02bea fix: correct indentation error in async streaming code
Fixed IndentationError at line 1325 where the async for loop was not properly
indented under the if verbose: block. Also corrected the indentation of the
else block for non-verbose streaming.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-12 17:33:23 +00:00
Mervin Praison 1ee7cfa2ef Merge pull request #844 from MervinPraison/claude/issue-840-20250712_172354
docs: add OLLAMA_HANDLING.md to explain ollama_handled flag usage
2025-07-12 18:32:16 +01:00
claude[bot]andMervin Praison b01bd41574 fix: add sequential tool calling support to async get_response_async method
- Add while loop structure with max_iterations and iteration_count
- Replace return statements with loop continuation after tool execution
- Store reasoning content across iterations for proper display
- Update final response handling to use stored values from iterations
- Ensure async behavior matches sync version for issue #839

This resolves the inconsistency where async agents would return empty
responses after first tool call instead of continuing with additional
required tool calls.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-12 17:27:37 +00:00
claude[bot]andMervin Praison 6570e445b8 docs: add OLLAMA_HANDLING.md to explain ollama_handled flag usage
Add documentation to clarify that ollama handling is correctly implemented
in llm.py and not in agent.py, addressing issue #840

Co-authored-by: Mervin Praison <[email protected]>
2025-07-12 17:25:07 +00:00
Mervin Praison b421bcf925 Merge pull request #843 from MervinPraison/claude/issue-841-20250712_080725
fix: add return_dict=True to fix TypeError in prompt_chaining tests
2025-07-12 18:24:24 +01:00
claude[bot]andMervin Praison 41d0998b76 fix: preserve reasoning_steps functionality with sequential tool calling
- Add stored_reasoning_content to capture reasoning after tool execution
- Restore reasoning_steps handling after tools while keeping loop continuation
- Update final response display to include stored reasoning when available
- Return reasoning content when reasoning_steps=True and content exists

This ensures backward compatibility while maintaining the sequential tool calling fix

Co-authored-by: Mervin Praison <[email protected]>
2025-07-12 08:41:06 +00:00
claude[bot]andMervin Praison 393e0ee33d fix: document async limitation and ensure consistent behavior
- Added clear documentation that async version doesn't support sequential tool calling
- Reverted incomplete async implementation to avoid breaking existing functionality
- Sync version retains the fix for sequential tool calling (issue #839)
- Created a known limitation that needs to be addressed in a future PR

Co-authored-by: Mervin Praison <[email protected]>
2025-07-12 08:40:25 +00:00
claude[bot]andMervin Praison 0dce4de4a4 fix: add return_dict=True to fix TypeError in prompt_chaining tests
The workflow.start() method returns a string by default (for backward compatibility),
but the test files were expecting a dictionary. Fixed by adding return_dict=True
parameter to get the expected dictionary format with task_results and task_status keys.

Fixes #841

Co-authored-by: Mervin Praison <[email protected]>
2025-07-12 08:23:56 +00:00
Mervin Praison 7fe4b48c6e fix: update LLM model and refine agent instruction
Changed the LLM model from "openai/gpt-4o-mini" to "gemini/gemini-2.5-pro" for improved performance. Updated the agent's instruction to provide clearer context for the task, enhancing user interaction.

Also, optimised the suggest_fixes function to handle both dictionary and string inputs, improving its flexibility in suggesting fixes based on issue type.

This minimal change aims to enhance the functionality while maintaining existing code integrity.
2025-07-12 09:22:33 +01:00
claude[bot]andMervin Praison 5286912f34 test: add test scripts for sequential tool calling fix
Add test scripts to verify the fix for issue #839 works correctly
with both Gemini and GPT-4 models.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-12 08:01:18 +00:00
claude[bot]andMervin Praison d6cb1c0165 fix: allow agents to continue with sequential tool calls
After executing tool calls, the agent now continues the iteration loop
instead of immediately trying to get a final response. This allows the
LLM to decide if more tools are needed to complete the task.

Fixes issue where agents would return empty responses after the first
tool call instead of continuing with additional required tool calls.

Resolves #839

Co-authored-by: Mervin Praison <[email protected]>
2025-07-12 08:01:10 +00:00
Mervin Praison 08ed3f94dc Merge pull request #836 from MervinPraison/claude/issue-835-20250711_234446
fix: capture final response after tool execution in LLM
2025-07-12 01:10:32 +01:00
Mervin Praison d3bfbc4b92 Delete test_issue_835.py 2025-07-12 01:00:19 +01:00
claude[bot]andMervin Praison af7081d294 fix: capture final response after tool execution in LLM
When an agent executes tools and the final iteration contains the answer
without additional tool calls, ensure the response text is properly captured
and returned instead of returning empty/None.

This fixes the issue where agents with multiple tools would return empty
responses after successfully executing tools.

Fixes #835

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 23:52:25 +00:00
Mervin Praison 3b2f26699e Merge pull request #834 from MervinPraison/claude/pr-832-20250711_232348
PR #832: Changes from Claude
2025-07-12 00:42:58 +01:00
claude[bot]andMervin Praison 7aa61b9155 fix: pass stream parameter from Agent to LLM
The Agent class was not passing the stream parameter to the LLM's get_response() method, causing the LLM to always use its default stream=True behavior regardless of the Agent's configuration.

This fix ensures that when an Agent is created with stream=False or when chat() is called with stream=False, this preference is properly passed to the LLM.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 23:31:50 +00:00
Mervin Praison 7e37c940d4 feat: upgrade praisonai to v2.2.51 and add aiohttp dependency to praisonaiagents v0.0.125 2025-07-12 00:28:17 +01:00
Mervin Praison 2199a57dfe Merge pull request #832 from MervinPraison/claude/issue-824-20250711_155141
fix: sequential tool calling for non-streaming responses
2025-07-12 00:19:42 +01:00
Mervin Praison d49f046322 Rename test_sequential_tool_calling.py to src/praisonai-agents/test_sequential_tool_calling.py 2025-07-12 00:14:28 +01:00
claude[bot]andMervin Praison c16c2e38af fix: improve sequential tool calling for non-streaming responses
- Fixed Ollama handling to properly separate streaming/non-streaming modes
- Added robust error handling for response extraction
- Fixed test file to use correct stream parameter
- Ensures tool outputs are processed by LLM in non-streaming mode

Addresses review feedback on PR #832

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 18:24:18 +00:00
Mervin Praison b5df9f323c Merge pull request #833 from MervinPraison/claude/issue-831-20250711_173321
fix: centralize logging configuration to respect LOGLEVEL environment variable
2025-07-11 19:05:06 +01:00
claude[bot]andMervin Praison 6e4abeef52 fix: ensure consistent logger levels across modules
- Change third-party logger levels from ERROR to WARNING in llm/__init__.py
  to match the centralized configuration
- Remove root logger override in knowledge.py that was ignoring LOGLEVEL
  environment variable

This ensures the LOGLEVEL environment variable is properly respected
throughout the application.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 17:58:02 +00:00
claude[bot] 5656570172 chore: remove test_logging.py from root - should be in tests directory
The test script was helpful for verifying the fix but should not be in the package root.
2025-07-11 17:44:52 +00:00
claude[bot]andMervin Praison c08c7cfedf fix: centralize logging configuration to respect LOGLEVEL environment variable
- Move logging configuration to top of __init__.py to ensure it runs before any imports
- Remove duplicate logging.basicConfig from llm/__init__.py that was overriding settings
- Clean up main.py to remove redundant logging configuration
- Add test_logging.py script to verify the fix works correctly

This ensures that the LOGLEVEL environment variable (DEBUG, INFO, WARNING, ERROR) properly controls logging verbosity throughout the package.

Fixes #831

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 17:44:18 +00:00
Mervin Praison fbbe80fd4d Merge pull request #829 from MervinPraison/claude/issue-827-20250711_151456
fix: add validation feedback to workflow retry logic
2025-07-11 17:27:46 +01:00
claude[bot]andMervin Praison cabddd9a22 refactor: extract validation failure decisions to class constant
- Add VALIDATION_FAILURE_DECISIONS constant to avoid duplication
- Update all references to use the constant
- Improve maintainability for future additions

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 16:05:19 +00:00
claude[bot]andMervin Praison 20b838f36a fix: handle non-streaming responses after tool execution for sequential tool calling
- Fixed issue where tool outputs were returned directly to user when stream=False
- Added proper handling for non-streaming responses after tool calls
- Maintains backward compatibility with existing streaming behavior

Fixes #824

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 15:59:04 +00:00
claude[bot]andMervin Praison cf149a3c2d fix: add validation feedback to workflow retry logic
When validation fails, retry tasks now receive:
- The validation failure reason
- The rejected output that failed validation  
- Clear instructions to try a different approach

This allows context-dependent tasks to improve on retry instead of
repeating the same invalid output.

Fixes #827

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 15:24:52 +00:00
Mervin Praison 30f6f5706c Merge pull request #826 from MervinPraison/claude/issue-823-20250711_111524
feat: Implement MCP HTTP Stream transport
2025-07-11 16:05:43 +01:00
claude[bot]andMervin Praison 0423c9c4af fix: address code review issues in HTTP Stream implementation
- Fix error propagation in write_stream method
- Add proper shutdown handling in SSE listener to prevent resource leaks
- Implement proper type annotations based on schema types
- Add _closing flag to gracefully stop SSE listener
- Create basic test to verify transport selection logic

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 14:08:58 +00:00
claude[bot]andMervin Praison be82abfac3 feat: Implement MCP HTTP Stream transport
- Add HTTP Stream transport implementation with single endpoint architecture
- Support both batch (JSON) and stream (SSE) response modes
- Implement session management with Mcp-Session-Id header
- Maintain full backward compatibility with SSE transport
- Add aiohttp dependency for HTTP client functionality
- Create usage examples and test file

Closes #823

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 11:27:37 +00:00
Mervin Praison 4af6228f89 feat: add automated PR review comment workflow for Claude integration 2025-07-11 11:59:45 +01:00
Mervin Praison 33eaf34ffb feat: bump PraisonAI to v2.2.50 and add Gemini agent tools implementation 2025-07-11 11:50:25 +01:00
Mervin Praison e2512b38c0 Merge pull request #814 from Dhivya-Bharathy/grok4-10-new-agents
Add 10 New Grok4 Intelligent Agents Covering Diverse Domains
2025-07-11 11:46:42 +01:00
Mervin Praison b02cda252d Merge pull request #819 from MervinPraison/claude/issue-818-20250711_091438
fix: enable tool calling for Gemini models
2025-07-11 11:42:43 +01:00
Mervin Praison 2a4b369e15 Rename test_tool_fix_improved.py to src/praisonai/tests/unit/test_tool_fix_improved.py 2025-07-11 11:42:29 +01:00
Mervin Praison c0a9e55f6a Rename test_tool_fix_example.py to src/praisonai/tests/unit/test_tool_fix_example.py 2025-07-11 11:42:12 +01:00
Mervin Praison 7c4b28885d Rename test_tool_fix_documentation.md to src/praisonai/tests/unit/test_tool_fix_documentation.md 2025-07-11 11:41:56 +01:00
Mervin Praison b0ec8b9a22 Rename test_gemini_tool_choice.py to src/praisonai/tests/unit/test_gemini_tool_choice.py 2025-07-11 11:41:32 +01:00
Mervin Praison 1b5cf2baea Rename test_async_gemini_fix.py to src/praisonai/tests/unit/test_async_gemini_fix.py 2025-07-11 11:41:19 +01:00
claude[bot]andMervin Praison b1cc2c31ce fix: enable tool calling for async agents with Gemini models
- Created _build_system_prompt method to centralize system prompt generation
- Fixed async achat to use correct tools parameter (was always using self.tools)
- Both sync and async now use enhanced system prompt with tool information
- Added comprehensive async agent test to verify the fix

This resolves the issue where async agents would claim no internet access
even when tools were available.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 10:15:56 +00:00
claude[bot]andMervin Praison 2a40c57033 feat: enhance Gemini tool calling fix with improved error handling
- Add comprehensive error handling for tool name extraction
- Support MCP tools with to_openai_tool method
- Improve Gemini model detection to handle variants
- Add litellm.supports_function_calling check
- Fix potential TypeError with nested dictionary access
- Update documentation with improvements
- Add comprehensive test example

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 09:59:39 +00:00
claude[bot]andMervin Praison 350d6e6488 fix: improve Gemini tool calling implementation
- Fix model prefix inconsistency (add 'gemini-' prefix check)
- Add debug logging when tool_choice='auto' is set for Gemini models
- Add error handling for tool name extraction
- Add test to verify tool_choice behavior for different models

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 09:54:05 +00:00
Mervin Praison 20a49ffd70 refactor: optimise test_autoagents.py for clarity and consistency
- Updated tool configurations to use string names instead of Mock objects for better clarity.
- Simplified the test for custom API key and base URL, ensuring parameters are stored correctly.
- Enhanced comments to improve understanding of the test setup and expected behaviour.

These changes improve the readability and maintainability of the test suite while ensuring accurate validation of configurations.
2025-07-11 10:42:57 +01:00
Mervin Praison 4bf9916c5f refactor: update test_autoagents.py to improve validation scenarios
- Introduced an invalid configuration to simulate validation failures in tests.
- Updated the first response to reflect the new invalid config structure.
- Simplified mock client configuration by using a sample valid config for structured output.

These changes enhance the robustness of the test cases while ensuring clarity in error handling scenarios.
2025-07-11 10:36:17 +01:00
claude[bot]andMervin Praison 9764483f36 fix: enable tool calling for Gemini models
- Enhanced system prompt to explicitly mention available tools
- Added tool_choice='auto' for Gemini models to encourage tool usage
- Maintains full backward compatibility
- Added test example and documentation

Fixes #818 where agents using Gemini models were not calling tools

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 09:27:37 +00:00
Mervin Praison ff405be834 refactor: enhance test_autoagents.py with improved mock configuration
- Updated the first response to use a malformed JSON string for clarity in error simulation.
- Introduced a serializable configuration for the mock client to ensure valid data handling in tests.
- Enhanced comments for better understanding of the test setup and scenarios.

These changes improve the clarity and robustness of the test cases while maintaining the integrity of the testing framework.
2025-07-11 10:23:21 +01:00
Mervin Praison 6d384a96ca refactor: improve test_autoagents.py for clarity and validation
- Updated comments for better understanding of test cases.
- Modified task configuration to create a valid config before simulating error conditions.
- Changed patch references from 'praisonaiagents.agents.autoagents.supports_structured_outputs' to 'praisonaiagents.llm.supports_structured_outputs' for consistency across tests.

These changes enhance the readability and maintainability of the test suite while ensuring accurate validation scenarios.
2025-07-11 10:18:13 +01:00
Mervin Praison 6770996c8b Merge pull request #812 from MervinPraison/claude/issue-795-20250710_222918
Claude/issue 795 20250710 222918
2025-07-11 10:06:12 +01:00
claude[bot]andMervin Praison 93c7731ccb fix: resolve code quality issues and unreachable code in agent.py
- Remove unused variable final_response_text
- Fix multimodal prompt normalization fallback to use empty string instead of str(prompt)
- Fix inconsistent variable usage: use original_prompt instead of prompt in display_interaction and _apply_guardrail_with_retry calls
- Remove unnecessary f-string where no interpolation is needed
- Fix unused exception variable by removing 'as e' where e is not used

These changes improve code quality while maintaining backward compatibility and ensuring the handoff fix continues to work correctly.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 08:29:20 +00:00
Mervin Praison 2057c68a20 Merge branch 'main' of https://github.com/MervinPraison/PraisonAI into pr-812-handoff-fix 2025-07-11 09:16:11 +01:00
Mervin Praison 1ca928a2e4 Merge branch 'main' of https://github.com/MervinPraison/PraisonAI 2025-07-11 09:09:32 +01:00
Mervin Praison 66ea9d7148 fix: enhance unit tests for AutoAgents class
- Added dynamic import handling for AutoAgents and related classes to prevent ImportError.
- Refactored test cases to create AutoAgents instances without invoking __init__ to avoid unnecessary API calls.
- Improved mock handling in tests to ensure proper validation and response checks.
- Ensured structured output support is correctly mocked across various test scenarios.
- Maintained existing functionality while optimising test structure for clarity and maintainability.
2025-07-11 09:09:31 +01:00
Mervin Praison 74a71807a6 Merge pull request #816 from MervinPraison/claude/issue-813-20250711_070340
fix: add missing HostedMCPServer class for MCP server implementations
2025-07-11 09:06:39 +01:00
Mervin Praison 9e92e85a72 Update pyproject.toml 2025-07-11 08:53:35 +01:00
Mervin Praison 6d827a45c8 Update pyproject.toml 2025-07-11 08:53:01 +01:00
Mervin Praison aabaaa6318 Update pyproject.toml 2025-07-11 08:52:38 +01:00
Mervin Praison f518bd301a Update pyproject.toml 2025-07-11 08:52:14 +01:00
Mervin Praison 834f710cf6 chore: update PraisonAI and related package versions to 2.2.49 and 0.0.123
- Updated PraisonAI version from 2.2.48 to 2.2.49 across Dockerfiles, README, and Ruby formula.
- Incremented praisonaiagents version from 0.0.122 to 0.0.123 in relevant files.
- Ensured consistency in package versions for improved stability and functionality.
2025-07-11 08:50:14 +01:00
Mervin Praison ede71a7fbe Merge pull request #817 from MervinPraison/claude/issue-815-20250711_070741
fix: resolve SyntaxError in autoagents.py
2025-07-11 08:47:11 +01:00
claude[bot]andMervin Praison 7c751c88a3 fix: resolve test failures in autoagents.py
- Use client.parse_structured_output instead of client.beta.chat.completions.parse
- Change llm_instance.response to llm_instance.get_response for structured output support
- Fixes AttributeError: 'OpenAIClient' object has no attribute 'beta'
- Fixes TypeError: Object of type ModelMetaclass is not JSON serializable

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 07:42:53 +00:00
claude[bot]andMervin Praison 17f3aab1e1 refactor: complete moving HostedMCPServer to examples folder
- Add hosted_server.py to examples/python/custom_tools/
- Update documentation to clarify this is an example implementation
- Keep starlette dependency as it's still used by Agent.launch() for MCP

This makes HostedMCPServer an example implementation rather than
a core package component.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 07:41:27 +00:00
claude[bot]andMervin Praison 028fa8158d refactor: move HostedMCPServer from core package to examples folder
- Move hosted_server.py to examples/python/custom_tools/
- Update import in mcp_server_latency_example.py to use local file
- Remove HostedMCPServer from core package exports

This keeps HostedMCPServer as an example implementation rather than
a core package component, as requested.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 07:39:24 +00:00
claude[bot]andMervin Praison c62685adc5 fix: remove dead code and add missing reflection continuation logic
- Remove unreachable code after while True loop (lines 1414-1429)
- Add missing reflection continuation code for non-satisfactory reflections
- Fix unnecessary f-string prefix in error message
- Fixes syntax error preventing test imports

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 07:29:52 +00:00
claude[bot]andMervin Praison 60d6178310 fix: add missing newline at end of autoagents.py to prevent parser errors
Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 07:26:59 +00:00
claude[bot]andMervin Praison b198154b71 fix: resolve SyntaxError and improve code structure in autoagents.py
- Fixed incorrect indentation causing SyntaxError on line 436
- Removed duplicate llm_instance creation and OpenAI detection logic
- Cleaned up nested try-except blocks for better readability
- Preserved backward compatibility with OpenAI structured outputs
- Maintained all error handling and retry mechanisms
- Fixed logic flow to properly check for OpenAI support once per retry

This minimal fix addresses the critical syntax issues while maintaining
all existing functionality.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 07:26:44 +00:00
claude[bot]andMervin Praison 205de2ba0b fix: improve HostedMCPServer implementation
- Remove unnecessary ABC inheritance and unused imports
- Add proper error handling for uninitialized MCP server
- Improve error messages for better debugging

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 07:23:48 +00:00
claude[bot]andMervin Praison ee668b2711 fix: resolve code quality issues and unreachable code in agent.py
- Fix unreachable reflection continuation code by moving it outside max reflections if block
- Remove unused variable final_response_text
- Fix unnecessary f-string without placeholders
- Correct prompt normalization: use original_prompt instead of prompt in display/guardrail calls
- Maintain backward compatibility while fixing handoff functionality

These changes address issues identified by multiple code reviewers while preserving
the core fix for handoff functionality (issue #795).

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 07:23:34 +00:00
claude[bot]andMervin Praison a5bbb65017 fix: resolve IndentationError in autoagents.py
- Fixed indentation issues that caused SyntaxError on line 360
- Removed duplicate OpenAI client initialization logic
- Properly indented the LLM instance creation and response parsing code
- Maintained backward compatibility with existing functionality

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 07:23:30 +00:00
claude[bot]andMervin Praison 6296755ebb fix: add missing newline at end of autoagents.py to prevent parser errors
Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 07:21:07 +00:00
claude[bot]andMervin Praison 4b6c847260 fix: resolve SyntaxError in autoagents.py by fixing indentation and uninitialized variables
- Fixed incorrect indentation from line 347 onwards that caused the SyntaxError
- All code from line 347-435 is now properly indented inside the else block
- Added initialization of `use_openai_structured` and `client` variables at the start of each retry loop to prevent NameError
- Preserved duplicate OpenAI detection logic for backward compatibility

Fixes #815

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 07:19:17 +00:00
Mervin Praison 2fc459fe05 chore: correct string escape sequences in test-core.yml
- Adjusted escape sequences for environment variable retrieval and print statements to ensure proper formatting and clarity in the workflow file.
2025-07-11 08:18:05 +01:00
Mervin Praison 1c3c2941cc chore: enhance review comments in automated workflows for clarity
- Updated the body of comments in auto-issue-comment.yml and auto-pr-comment.yml to include a note on minimal code changes, ensuring clarity while maintaining guidance for backward compatibility and testing.
2025-07-11 08:15:20 +01:00
Mervin Praison 6e38a88cd3 chore: optimise variable initialisation in autoagents.py
- Removed unnecessary initialisation of variables and streamlined the handling of OpenAI client instances to enhance code clarity and maintainability while ensuring backward compatibility.
2025-07-11 08:12:51 +01:00
claude[bot]andMervin Praison f0f22bb13d fix: add missing HostedMCPServer class for MCP server implementations
- Created HostedMCPServer base class in mcp/hosted_server.py
- Added HostedMCPServer to module exports in __init__.py
- Added starlette>=0.27.0 to MCP optional dependencies
- Fixes ImportError when using mcp_server_latency_example.py

Resolves #813

Co-authored-by: Mervin Praison <[email protected]>
2025-07-11 07:12:31 +00:00
Mervin Praison a8fa70c455 chore: refine review comment in automated issue workflow
- Adjusted the body of the comment in auto-issue-comment.yml to enhance clarity by removing redundant phrases while ensuring guidance for backward compatibility and testing remains intact.
2025-07-11 08:11:37 +01:00
Mervin Praison 1b4b4a10b5 chore: update review comment in automated PR workflow for multi-agent analysis
- Modified the body of the comment in auto-pr-comment.yml to include instructions for reviewing pull requests with multiple parallel running agents, ensuring clarity while maintaining guidance for backward compatibility and testing.
2025-07-11 08:11:20 +01:00
devops-vestradata 1efdaac18f Add 10 new Grok4 intelligent agents 2025-07-11 12:29:26 +05:30
Mervin Praison 1b2182968f chore: simplify review comment in automated PR workflow
- Revised the body of the comment in auto-pr-comment.yml to streamline instructions for review, removing redundant phrases while maintaining clarity and guidance for backward compatibility and testing.
2025-07-11 01:16:13 +01:00
claude[bot]andMervin Praison 5e5a10589e fix: correct syntax error - fix indentation of lines 1404-1411 to be inside try block
The code from lines 1404-1411 was incorrectly outside the try block that starts at line 1325, causing a syntax error when the except block at line 1413 tried to match with it. This fix properly indents these lines to be inside the try block.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 23:59:40 +00:00
claude[bot]andMervin Praison 6c5e42762f fix: comprehensive fix for all try-except block indentation issues
- Fixed indentation of except block at line 1413 to match its try block
- Moved except block at line 1419 and subsequent code inside the while loop
- Properly structured all nested try-except blocks to prevent syntax errors
- Ensured the handoff fix (user message added before LLM call) is preserved

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 23:55:28 +00:00
claude[bot]andMervin Praison afb7e47080 fix: correct syntax error - fix indentation of except block at line 1413
The except block was indented with 28 spaces instead of 24 spaces to match its corresponding try block, causing a SyntaxError

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 23:50:32 +00:00
claude[bot]andMervin Praison e674a2ae84 fix: correct syntax errors and add missing guardrail validation to achat()
- Fix indentation of except block at line 1413 (20 spaces -> 24 spaces)
- Add guardrail validation to achat() method for both custom LLM and OpenAI paths
- Ensure feature parity between sync chat() and async achat() methods

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 23:46:10 +00:00
Mervin Praison d471d9754a chore: enhance automated comment workflows with additional guidance
- Updated the body of comments in both auto-issue-comment.yml and auto-pr-comment.yml to include instructions for using @web for information retrieval and testing code execution if necessary.
2025-07-11 00:39:27 +01:00
claude[bot]andMervin Praison 9d25b1ed3d fix: correct syntax error in agent.py - add missing except block
The try block at line 1249 (inside the while loop) was missing its 
corresponding except block. Added proper exception handling to close
the try block and re-raise exceptions to the outer handler.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 23:37:43 +00:00
claude[bot]andMervin Praison d0d04bc17e fix: add missing guardrail validation and error handling to achat() async method
- Add guardrail validation to all return points in achat() method (except JSON/Pydantic outputs)
- Add try-except wrapper around while loop in OpenAI path for proper error recovery
- Ensure chat history rollback on all failure paths in async method
- Maintain feature parity between sync chat() and async achat() methods

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 23:36:32 +00:00
Mervin Praison 1f4e2b1c5a chore: update github-token in automated comment workflows
- Changed the token reference from GITHUB_TOKEN to GH_TOKEN in both auto-issue-comment.yml and auto-pr-comment.yml for consistency in authentication.
2025-07-11 00:32:09 +01:00
claude[bot]andMervin Praison 88d4241121 fix: correct syntax error in agent.py - proper indentation for try-except block
Fixed syntax error at line 1361 where code was not properly indented inside the try block.
The code from lines 1361-1412 needed to be indented by 4 spaces to be inside the try
block that starts at line 1325.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 23:31:31 +00:00
Mervin Praison 9e081e9413 chore: add github-token to automated comment workflows
- Included github-token in both auto-issue-comment.yml and auto-pr-comment.yml to ensure proper authentication for comment creation.
2025-07-11 00:30:18 +01:00
Mervin Praison bc0449b8de Merge pull request #803 from MervinPraison/claude/issue-792-20250710_221133
fix: replace prefix-based model detection with capability-based detection
2025-07-11 00:26:33 +01:00
Mervin Praison c262942ba7 Merge branch 'main' into claude/issue-792-20250710_221133 2025-07-11 00:26:18 +01:00
Mervin Praison 40e5c13f47 chore: update PraisonAI version to 2.2.48 in Dockerfiles, README, and related files 2025-07-11 00:23:41 +01:00
Mervin Praison 5eea991a60 Merge pull request #801 from MervinPraison/claude/issue-798-20250710_224150
fix: update AutoAgents prompt to generate proper TaskConfig objects
2025-07-11 00:19:27 +01:00
claude[bot]andMervin Praison 209c2a42ae fix: correct syntax errors in try-except block indentation
- Fix indentation of reflection prompt code block (lines 1316-1323)
- Fix indentation of try-except blocks for self-reflection (lines 1325-1418)
- Fix indentation of code inside if-else statements for custom LLM vs OpenAI
- Fix indentation of final logging and guardrail validation code

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 23:18:25 +00:00
Mervin Praison efb927137f refactor: remove unused autoagents-tools.py file
- Deleted the autoagents-tools.py file as it contained redundant code and instances of AutoAgents that are no longer needed.
2025-07-11 00:17:10 +01:00
claude[bot]andMervin Praison 162223162f test: add comprehensive unit tests for AutoAgents validation and retry logic
- Add test coverage for _validate_config method
- Test successful config generation via OpenAI and LLM paths
- Test retry mechanism with validation failures
- Test edge cases like max_agents truncation and warnings
- Mock LLM responses to avoid API calls in tests

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 23:15:15 +00:00
claude[bot]andMervin Praison 75c856b948 fix: correct syntax errors in try-except block indentation
- Fixed indentation issues in chat() method where code was not properly indented inside try blocks
- Fixed duplicate except block that was causing syntax errors
- Added missing chat history management to async achat() method for OpenAI client path
- Ensured all try-except blocks are properly structured and closed

This resolves the SyntaxError preventing module import and test execution.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 23:11:36 +00:00
claude[bot]andMervin Praison 85fefac429 fix: add missing duplicate prevention and error rollback in achat() OpenAI path
- Add duplicate message prevention for async OpenAI client path
- Add chat history rollback on errors in achat() method
- Ensure consistent behavior between sync and async methods
- Handle edge case where chat_history_length might not be defined

This completes the fix by ensuring both sync and async methods have
identical error handling and duplicate prevention logic.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 23:10:39 +00:00
claude[bot]andMervin Praison 35b9d46bdf feat: add validation and retry mechanism for AutoAgents config generation
- Add _validate_config method to ensure tasks are proper TaskConfig objects
- Implement retry logic with up to 3 attempts
- Include previous response and error in retry prompt for better context
- Add explicit task structure reminder on validation failures

This ensures more robust configuration generation and better error recovery
when LLMs generate incorrect task formats.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 23:09:16 +00:00
claude[bot]andMervin Praison 5bd3e45be4 fix: add error handling, duplicate prevention, and multimodal normalization for chat_history
- Add chat history rollback on LLM call failures to prevent inconsistent state
- Prevent duplicate user messages in chat history
- Normalize multimodal prompts to store only text content in chat history
- Add thread safety documentation for future consideration
- Apply fixes to both sync (chat) and async (achat) methods
- Handle all error paths including guardrail failures

This ensures chat_history remains consistent even when errors occur, preventing
unpaired user/assistant messages that could break handoff functionality.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 23:01:57 +00:00
Mervin Praison 4c62731e81 Merge branch 'main' into claude/issue-795-20250710_222918 2025-07-10 23:56:44 +01:00
Mervin Praison 544672edea Merge pull request #800 from MervinPraison/claude/issue-796-20250710_222402
fix: handle JSON parsing for Gemini self-reflection
2025-07-10 23:51:12 +01:00
Mervin Praison 72dcc91b3a Rename test_gemini_fix.py to gemini_self_reflection.py 2025-07-10 23:50:51 +01:00
claude[bot]andMervin Praison 75e728f7d7 fix: update AutoAgents prompt to generate proper TaskConfig objects
The validation error was occurring because the LLM was generating tasks as simple strings instead of proper TaskConfig objects. This commit updates the prompt in the _generate_config method to:

1. Include an explicit JSON schema example showing the correct structure
2. Add a clear warning that tasks must be objects with specific fields
3. Provide a concrete example of the task object structure

This ensures tasks are generated with the required fields: name, description, expected_output, and tools.

Fixes #798

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 22:44:30 +00:00
claude[bot]andMervin Praison 7ddbf98794 fix: add user message to chat_history before LLM call to fix handoff issue
This fix ensures that handoff tools can access the current user message
during execution. Previously, chat_history was updated after the LLM call,
causing handoffs to fail with "no specific task was provided" message.

The fix adds the user message to chat_history before LLM processing in both:
- Custom LLM path
- Standard OpenAI client path

This maintains backward compatibility while fixing the handoff functionality.

Fixes #795

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 22:36:33 +00:00
Mervin Praison 0213dea12b chore: update PraisonAI version to 2.2.47 in Dockerfiles, README, and related files 2025-07-10 23:30:27 +01:00
claude[bot]andMervin Praison 841a7b52c4 fix: handle JSON parsing for Gemini self-reflection
- Disable streaming for custom LLMs during self-reflection to avoid incomplete JSON chunks
- Add manual JSON parsing for custom LLM reflection responses
- Use clean_json_output method to handle markdown-wrapped JSON
- Ensure reflection regeneration also uses non-streaming for custom LLMs

This fixes the "Expecting property name enclosed in double quotes" error
when using Gemini models with self-reflection enabled.

Fixes #796

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 22:29:42 +00:00
claude[bot]andMervin Praison 99506e80cf fix: replace prefix-based model detection with capability-based detection
- Created model_capabilities.py with comprehensive lists of models supporting structured outputs
- Updated llm.py to use supports_streaming_with_tools() instead of prefix checks
- Updated autoagents.py to use supports_structured_outputs() instead of prefix checks
- Added proper imports/exports in __init__.py
- Maintains backward compatibility for all existing functionality
- More accurate and maintainable approach than hardcoded prefixes

Fixes #792

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 22:21:30 +00:00
Mervin Praison c0d495fd48 Merge pull request #781 from MervinPraison/claude/issue-754-20250709_155906
refactor: eliminate sync/async duplication in llm.py
2025-07-10 23:21:04 +01:00
Mervin Praison 0e51ed0d6c Merge branch 'main' of https://github.com/MervinPraison/PraisonAI 2025-07-10 23:20:50 +01:00
Mervin Praison 32dce14f96 chore: update version to 0.0.120 in pyproject.toml and uv.lock 2025-07-10 23:20:48 +01:00
Mervin Praison 27aaac80c6 Merge branch 'main' into claude/issue-754-20250709_155906 2025-07-10 23:20:17 +01:00
Mervin Praison 5a0a3983e4 Merge pull request #794 from MervinPraison/claude/pr-791-20250710_214815
PR #791: Changes from Claude
2025-07-10 23:19:03 +01:00
claude[bot]andMervin Praison 400a4c934e fix: properly pass base_url and api_key parameters to OpenAI client and agents
- Create new OpenAIClient instance when custom base_url or api_key are provided
- Pass base_url and api_key to Agent instances for consistency
- Maintain backward compatibility by falling back to singleton client when no custom parameters

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 22:13:29 +00:00
Mervin Praison e88b0fd47b Merge pull request #793 from MervinPraison/claude/issue-754-20250709_155933
refactor: extract debug logging into centralized method
2025-07-10 23:11:57 +01:00
claude[bot]andMervin Praison e6c1c0679f fix: address review comments for debug logging refactor
- Restore backward compatibility for verbose parameter (verbose >= 10 now works)
- Fix condition ordering bug (check "parameters" before "_async")
- Fix API key masking for empty strings
- Add type check for system_prompt truncation
- Add iterable check for tools parameter
- Restore original prompt logging behavior for non-string prompts

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 22:03:01 +00:00
claude[bot]andMervin Praison 108aee0a54 fix: remove duplicate _handle_ollama_model method and fix double tool call processing
- Remove duplicate _handle_ollama_model method definition
- Fix bug where tool calls were processed twice in verbose streaming
- Keep the method with better type hints (Union[str, List[Dict]])
- Remove redundant _process_tool_calls_from_stream calls after _process_stream_delta

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 22:02:58 +00:00
Mervin Praison cb017aad70 Merge branch 'main' into claude/issue-754-20250709_155906 2025-07-10 22:53:33 +01:00
claude[bot]andMervin Praison ea29d3181b fix: properly handle OpenAI client API and store base_url/api_key
- Use OpenAIClient's parse_structured_output method instead of accessing client.beta directly
- Store base_url and api_key as instance variables to support custom endpoints
- This fixes the AttributeError when using AutoAgents with non-OpenAI providers

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 21:50:39 +00:00
claude[bot]andMervin Praison 7f0f265671 refactor: eliminate sync/async duplication in llm.py
- Extract common logic into shared helper methods
- Fix variable reference errors for ollama_params
- Reduce code duplication between get_response/get_response_async
- Reduce code duplication between response/aresponse
- Maintain 100% backward compatibility
- No API changes or breaking changes

Fixes #754

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 21:42:47 +00:00
Mervin Praison bc1c508419 Merge pull request #789 from MervinPraison/claude/issue-787-20250710_205703
fix: prevent double output when using litellm models
2025-07-10 22:37:16 +01:00
Mervin Praison bd108ba3e7 Merge pull request #791 from MervinPraison/claude/issue-788-20250710_210226
fix: add multi-provider support for AutoAgents
2025-07-10 22:34:58 +01:00
claude[bot]andMervin Praison ac5f2649df refactor: resolve merge conflicts with main branch
- Renamed _prepare_ollama_followup to _handle_ollama_model to match main branch
- Updated method signature to (response_text, tool_results, messages, original_prompt)
- Modified return value to dict with follow_up_messages and original_prompt
- Updated all call sites to use the new interface
- Maintained backward compatibility and refactoring benefits

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 21:34:28 +00:00
claude[bot]andMervin Praison 1b64762d54 fix: add verbose checks to all display_interaction calls for consistency
- Applied consistent pattern to all 4 display_interaction calls
- Now all checks follow pattern: if self.verbose and not self._using_custom_llm
- Ensures display is only shown when verbose mode is enabled AND not using custom LLM
- Maintains backward compatibility while fixing the inconsistency

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 21:29:45 +00:00
claude[bot]andMervin Praison df402bcfdb fix: correct indentation error in aresponse method
Fixed incorrect indentation in the async Ollama follow-up handling code
that was causing a syntax error at line 1364.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 21:20:39 +00:00
claude[bot]andMervin Praison 061c8d58b4 fix: add multi-provider support for AutoAgents
- Replace OpenAI-specific beta.chat.completions.parse() with cross-provider LLM class
- Maintain backward compatibility by using structured output for OpenAI models
- Add support for all LiteLLM providers (Gemini, Anthropic, etc.)
- Include robust JSON parsing to handle markdown-wrapped responses
- Fixes #788: AttributeError when using AutoAgent with Gemini provider

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 21:18:52 +00:00
claude[bot]andMervin Praison 82c9ff80a7 fix: correct indentation error in get_response method
Fixed incorrect indentation in lines 779-804 of the get_response method
that was causing IndentationError. The code block for Ollama follow-up
handling is now properly nested within the if follow_up_prompt block.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 21:15:53 +00:00
claude[bot]andMervin Praison f2c89c2012 fix: prevent double output when using litellm models
- Check for self._using_custom_llm before calling display_interaction
- Custom LLM already handles display in its get_response method
- Fixes issue #787 where task/response printed twice with litellm

Co-authored-by: Mervin Praison <[email protected]>
2025-07-10 21:12:41 +00:00
claude[bot]andMervin Praison f3f5f63b22 refactor: eliminate sync/async duplication in llm.py
- Extract common logic into shared helper methods
- Reduce code duplication between get_response/get_response_async
- Reduce code duplication between response/aresponse
- Maintain 100% backward compatibility
- No API changes or breaking changes

Addresses issue #754

Co-authored-by: Mervin Praison <[email protected]>
2025-07-09 16:07:51 +00:00
claude[bot]andMervin Praison ccf9470f41 refactor: extract debug logging into centralized _log_llm_config method
- Created private _log_llm_config method to handle all debug logging
- Eliminated ~116 lines of duplicated debug logging code
- Maintains 100% backward compatibility (no API changes)
- Centralizes security measures (API key masking)
- Consistent formatting across all logging calls
- No functional changes, only code organization

Addresses code duplication issue #754

Co-authored-by: Mervin Praison <[email protected]>
2025-07-09 16:05:59 +00:00
98 changed files with 10789 additions and 3735 deletions

No files matched your search

+81
View File
@@ -0,0 +1,81 @@
name: 'Gemini CLI Action'
description: 'Uses the Gemini CLI to automate software development tasks.'
author: 'Gemini'
inputs:
prompt:
description: 'A specific prompt to guide Gemini.'
required: false
default: 'You are a helpful assistant.'
GEMINI_API_KEY:
description: 'Your Gemini API key.'
required: true
OTLP_GCP_WIF_PROVIDER:
description: 'The workload identity provider for GCP authentication.'
required: false
OTLP_GOOGLE_CLOUD_PROJECT:
description: 'The Google Cloud project for telemetry.'
required: false
settings_json:
description: |
A JSON string to configure the Gemini CLI. This will be written to .gemini/settings.json.
To enable observability with Google Cloud, include:
{"telemetry": {"enabled": true, "target": "gcp"}, "sandbox": false}
required: false
version:
description: |
The version of @google/gemini-cli to execute.
Can be a specific version from npm (e.g., '0.1.0', 'latest'), a branch name (e.g., 'main'), or a commit hash.
required: false
default: 'latest'
runs:
using: "composite"
steps:
- name: Configure Gemini CLI
if: inputs.settings_json != ''
run: |
mkdir -p .gemini
echo "$SETTINGS_JSON" > .gemini/settings.json
shell: bash
env:
SETTINGS_JSON: ${{ inputs.settings_json }}
- name: Authenticate to Google Cloud for Telemetry
if: inputs.OTLP_GCP_WIF_PROVIDER != ''
id: auth
uses: 'google-github-actions/auth@v2'
with:
project_id: ${{ inputs.OTLP_GOOGLE_CLOUD_PROJECT }}
workload_identity_provider: ${{ inputs.OTLP_GCP_WIF_PROVIDER }}
- name: Install Gemini CLI
id: install
run: |
if [[ "${{ inputs.version }}" == "latest" || "${{ inputs.version }}" =~ ^[0-9]+\.[0-9]+\.[0-9]+(-[a-zA-Z0-9\.-]+)?(\+[a-zA-Z0-9\.-]+)?$ ]]; then
echo "Installing Gemini CLI from npm: @google/gemini-cli@${{ inputs.version }}"
npm install -g @google/gemini-cli@${{ inputs.version }}
else
echo "Installing Gemini CLI from GitHub: github:google-gemini/gemini-cli#${{ inputs.version }}"
npm install -g github:google-gemini/gemini-cli#${{ inputs.version }}
fi
echo "Verifying installation:"
if command -v gemini >/dev/null 2>&1; then
gemini --version || echo "Gemini CLI installed successfully (version command not available)"
else
echo "Error: Gemini CLI not found in PATH"
exit 1
fi
shell: bash
- name: Run Gemini CLI
run: |
gemini --yolo --prompt "$PROMPT"
shell: bash
env:
GEMINI_API_KEY: ${{ inputs.GEMINI_API_KEY }}
PROMPT: ${{ inputs.prompt }}
branding:
icon: 'terminal'
color: 'blue'
+23
View File
@@ -0,0 +1,23 @@
name: Auto Issue Comment
on:
issues:
types: [opened]
jobs:
add-comment:
runs-on: ubuntu-latest
permissions:
issues: write
steps:
- name: Add automated comment
uses: actions/github-script@v7
with:
github-token: ${{ secrets.GH_TOKEN }}
script: |
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: '@claude review this issue and do a detailed analysis and fix this if the existing code doesn\'t have the solution implemented. Making sure it has backward compatibility, no existing features removed. After making those changes, again review the applied changes. Use @web to search if you dont know any information or to find the latest documentation or to find the latest version. Run the code if you think you need to run it to test it. Minimal code change to start with if required any changes.'
})
+23
View File
@@ -0,0 +1,23 @@
name: Auto PR Comment
on:
pull_request:
types: [opened]
jobs:
add-comment:
runs-on: ubuntu-latest
permissions:
pull-requests: write
steps:
- name: Add automated comment
uses: actions/github-script@v7
with:
github-token: ${{ secrets.GH_TOKEN }}
script: |
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: '@claude review this pull request and do a detailed analysis and fix this if the existing code doesn\'t have the solution implemented. Making sure it has backward compatibility, no existing features removed. After making those changes, again review the applied changes. Use @web to search if you dont know any information or to find the latest documentation or to find the latest version. Run the code if you think you need to run it to test it. Minimal code change to start with if required any changes.'
})
+102 -20
View File
@@ -1,4 +1,4 @@
name: Claude Code
name: Claude Assistant
on:
issue_comment:
@@ -6,34 +6,116 @@ on:
pull_request_review_comment:
types: [created]
issues:
types: [opened, assigned]
types: [opened, assigned, labeled]
pull_request_review:
types: [submitted]
jobs:
claude:
claude-response:
# Prevent bots from triggering this action
if: |
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
github.actor != 'dependabot[bot]' &&
github.actor != 'github-actions[bot]' &&
github.actor != 'cursor[bot]' &&
!contains(github.actor, '[bot]') &&
github.actor != 'renovate[bot]'
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
issues: read
id-token: write
contents: write
pull-requests: write
issues: write
actions: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Run Claude Code (OAuth)
id: claude
uses: grll/claude-code-action@beta
fetch-depth: 0
- uses: anthropics/claude-code-action@beta
with:
use_oauth: true
claude_access_token: ${{ secrets.CLAUDE_ACCESS_TOKEN }}
claude_refresh_token: ${{ secrets.CLAUDE_REFRESH_TOKEN }}
claude_expires_at: ${{ secrets.CLAUDE_EXPIRES_AT }}
claude_code_oauth_token: ${{ secrets.CLAUDE_ACCESS_TOKEN }}
github_token: ${{ secrets.GITHUB_TOKEN }}
trigger_phrase: "@claude"
timeout_minutes: 30
post-review-comment:
needs: claude-response
if: success() && needs.claude-response.result == 'success'
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
issues: write
steps:
- name: Check for existing review comment
id: check-comment
uses: actions/github-script@v7
with:
github-token: ${{ secrets.GH_TOKEN }}
script: |
const reviewText = '@claude review if the changes made are as per the requirements';
let comments = [];
let issueNumber;
try {
// Determine issue/PR number based on event type
if (context.eventName === 'pull_request_review_comment' || context.eventName === 'pull_request_review') {
issueNumber = context.payload.pull_request?.number || context.issue.number;
} else if (context.eventName === 'issue_comment' && context.payload.issue.pull_request) {
// This is a comment on a PR (not an issue)
issueNumber = context.payload.issue.number;
} else {
// This is a regular issue
issueNumber = context.issue.number;
}
comments = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber
});
const existingComment = comments.data.find(comment =>
comment.body.includes(reviewText)
);
const shouldPost = !existingComment;
core.setOutput('should-post', shouldPost);
console.log(`Should post comment: ${shouldPost}`);
} catch (error) {
console.log('Error checking comments:', error);
core.setOutput('should-post', true); // If we can't check, proceed with posting
}
- name: Wait 20 seconds before posting comment
if: steps.check-comment.outputs.should-post == 'true'
run: sleep 20
- name: Post review comment
if: steps.check-comment.outputs.should-post == 'true'
uses: actions/github-script@v7
with:
github-token: ${{ secrets.GH_TOKEN }}
script: |
let issueNumber;
// Determine issue/PR number based on event type
if (context.eventName === 'pull_request_review_comment' || context.eventName === 'pull_request_review') {
issueNumber = context.payload.pull_request?.number || context.issue.number;
} else if (context.eventName === 'issue_comment' && context.payload.issue.pull_request) {
// This is a comment on a PR (not an issue)
issueNumber = context.payload.issue.number;
} else {
// This is a regular issue
issueNumber = context.issue.number;
}
console.log(`Posting review comment on issue/PR #${issueNumber}`);
await github.rest.issues.createComment({
issue_number: issueNumber,
owner: context.repo.owner,
repo: context.repo.repo,
body: '@claude review if the changes made are as per the requirements'
});
+1
View File
@@ -40,5 +40,6 @@ jobs:
uses: docker/build-push-action@v2
with:
context: .
file: ./docker/Dockerfile
push: true
tags: ghcr.io/mervinpraison/praisonai:latest
@@ -0,0 +1,76 @@
name: Gemini Automated Issue Triage
on:
issue_comment:
types: [created]
jobs:
triage-issue:
if: >
github.event_name == 'issue_comment' &&
!github.event.issue.pull_request &&
contains(github.event.comment.body, '@gemini triage') &&
(github.event.comment.author_association == 'OWNER' ||
github.event.comment.author_association == 'MEMBER' ||
github.event.comment.author_association == 'COLLABORATOR')
timeout-minutes: 5
permissions:
issues: write
contents: read
id-token: write
concurrency:
group: ${{ github.workflow }}-${{ github.event.issue.number }}
cancel-in-progress: true
runs-on: ubuntu-latest
steps:
- name: Generate GitHub App Token
id: generate_token
uses: actions/create-github-app-token@v1
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
- name: Checkout repository
uses: actions/checkout@v4
with:
token: ${{ steps.generate_token.outputs.token }}
- name: Run Gemini Issue Triage
uses: google-gemini/gemini-cli-action@main
env:
GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }}
ISSUE_TITLE: ${{ github.event.issue.title }}
ISSUE_BODY: ${{ github.event.issue.body }}
ISSUE_NUMBER: ${{ github.event.issue.number }}
REPOSITORY: ${{ github.repository }}
with:
version: 0.1.8-rc.0
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
OTLP_GCP_WIF_PROVIDER: ${{ secrets.OTLP_GCP_WIF_PROVIDER }}
OTLP_GOOGLE_CLOUD_PROJECT: ${{ secrets.OTLP_GOOGLE_CLOUD_PROJECT }}
settings_json: |
{
"coreTools": [
"run_shell_command(gh label list)",
"run_shell_command(gh issue edit)"
],
"telemetry": {
"enabled": true,
"target": "gcp"
},
"sandbox": false
}
prompt: |
You are an issue triage assistant. Analyze the current GitHub issue and apply the most appropriate existing labels.
Steps:
1. Run: `gh label list --limit 100` to get all available labels.
2. Review the issue title and body provided in the environment variables.
3. Select the most relevant labels from the existing labels, focusing on kind/*, area/*, and priority/*.
4. Apply the selected labels to this issue using: `gh issue edit ISSUE_NUMBER --add-label "label1,label2"`
Guidelines:
- Only use labels that already exist in the repository.
- Do not add comments or modify the issue content.
- Triage only the current issue.
- Assign all applicable kind/*, area/*, and priority/* labels based on the issue content.
+198
View File
@@ -0,0 +1,198 @@
name: Gemini Issue Review
on:
workflow_dispatch:
issue_comment:
types: [created]
permissions:
contents: write
issues: write
pull-requests: read
jobs:
review-issue:
if: >
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'issue_comment' &&
!github.event.issue.pull_request &&
contains(github.event.comment.body, '@gemini') &&
!contains(github.event.comment.body, '@gemini triage') &&
(github.event.comment.author_association == 'OWNER' ||
github.event.comment.author_association == 'MEMBER' ||
github.event.comment.author_association == 'COLLABORATOR'))
runs-on: ubuntu-latest
env:
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
steps:
- name: Generate token
id: generate_token
uses: actions/create-github-app-token@v1
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
- name: Checkout repository
uses: actions/checkout@v4
with:
token: ${{ steps.generate_token.outputs.token }}
- name: Set up Git
run: |
git config --local user.email "[email protected]"
git config --local user.name "Gemini Bot"
- name: Get issue data
id: issue_data
env:
GH_TOKEN: ${{ steps.generate_token.outputs.token }}
run: |
# Get issue number from event or context
if [ "${{ github.event_name }}" = "issue_comment" ]; then
ISSUE_NUMBER="${{ github.event.issue.number }}"
elif [ "${{ github.event_name }}" = "issues" ]; then
ISSUE_NUMBER="${{ github.event.issue.number }}"
else
echo "Unable to determine issue number"
exit 1
fi
# Get issue details
gh issue view $ISSUE_NUMBER --json title,body,labels,assignees > issue_details.json
# Extract additional instructions from comment if triggered by comment
ADDITIONAL_INSTRUCTIONS=""
if [ "${{ github.event_name }}" = "issue_comment" ]; then
COMMENT_BODY="${{ github.event.comment.body }}"
ADDITIONAL_INSTRUCTIONS=$(echo "$COMMENT_BODY" | sed -n 's/.*@gemini[[:space:]]*\(.*\)/\1/p' | head -1)
fi
echo "ISSUE_NUMBER=$ISSUE_NUMBER" >> $GITHUB_OUTPUT
echo "ADDITIONAL_INSTRUCTIONS=$ADDITIONAL_INSTRUCTIONS" >> $GITHUB_OUTPUT
- name: Run Gemini CLI
uses: ./.github/actions/gemini
env:
GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }}
ISSUE_NUMBER: ${{ steps.issue_data.outputs.ISSUE_NUMBER }}
ADDITIONAL_INSTRUCTIONS: ${{ steps.issue_data.outputs.ADDITIONAL_INSTRUCTIONS }}
REPOSITORY: ${{ github.repository }}
with:
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
OTLP_GCP_WIF_PROVIDER: ${{ secrets.OTLP_GCP_WIF_PROVIDER }}
OTLP_GOOGLE_CLOUD_PROJECT: ${{ secrets.OTLP_GOOGLE_CLOUD_PROJECT }}
prompt: |
You are a helpful GitHub issue assistant. Your task is to analyze and help with GitHub issues.
**Issue Details:**
- Issue Number: #${ISSUE_NUMBER}
- Repository: ${REPOSITORY}
- Issue URL: ${{ github.event.issue.html_url }}
**Additional Instructions:** ${ADDITIONAL_INSTRUCTIONS}
**CRITICAL: When making code modifications, follow this EXACT workflow:**
**STEP 1: Analyze the Issue**
- Use: `gh issue view ${ISSUE_NUMBER} --json title,body,labels,assignees`
- Analyze issue content thoroughly
- Search codebase for related files using `glob`, `search_file_content`, `list_directory`
**STEP 2: If Code Changes Are Needed**
Create a new branch with 'gemini/' prefix:
- Use: `git checkout -b gemini/issue-${ISSUE_NUMBER}-description`
- Example: `git checkout -b gemini/issue-123-fix-sequential-tools`
**STEP 3: Make Code Modifications**
- Use `read_file` to examine existing code
- Use `write_file` or `replace` to make changes
- Use `search_file_content` to find related code
- Make focused, targeted changes to fix the issue
**STEP 4: Test Changes (if applicable)**
- Run relevant tests if test files exist
- Use `run_shell_command` to execute test commands
**STEP 5: Commit and Push Changes**
- Use: `git add .` (or specific files)
- Use: `git commit -m "Gemini: [Issue #${ISSUE_NUMBER}] Brief description of fix"`
- Use: `git push -u origin gemini/issue-${ISSUE_NUMBER}-description`
**STEP 6: Create Pull Request**
- Use: `gh pr create --title "Fix #${ISSUE_NUMBER}: Issue description" --body "Fixes #${ISSUE_NUMBER}\n\nSummary of changes:\n- Change 1\n- Change 2" --head gemini/issue-${ISSUE_NUMBER}-description --base main`
- This will return a PR URL like: https://github.com/repo/pull/123
**STEP 7: Comment on Issue with Results**
Use: `gh issue comment ${ISSUE_NUMBER} -b "MESSAGE"`
**Your final comment MUST include:**
- Summary of what was analyzed
- What changes were made (if any)
- Link to the created PR (if applicable)
- Brief explanation of the solution
**Example final comment:**
```
Hi! I've analyzed and fixed this issue.
**Problem:** Sequential tool calling was getting stuck after first tool call.
**Solution:** Added proper prompt handling after each tool call in the Ollama model integration.
**Changes Made:**
- Modified `src/praisonai-agents/ollama-sequential.py`
- Added continuation prompts for multi-step tool execution
**Pull Request:** https://github.com/MervinPraison/PraisonAI/pull/856
The fix is ready for review. Please test and let me know if you encounter any issues!
```
**IMPORTANT NOTES:**
- Always create branches with 'gemini/' prefix
- Always include issue number in branch name and commit message
- Always create PR that references the issue
- Always provide PR link in your final response
- Use the GITHUB_TOKEN environment variable for authentication
Please proceed with helping resolve this issue following this exact workflow.
settings_json: |
{
"coreTools": [
"run_shell_command(echo)",
"run_shell_command(gh issue view)",
"run_shell_command(gh issue comment)",
"run_shell_command(gh issue list)",
"run_shell_command(gh issue edit)",
"run_shell_command(gh pr create)",
"run_shell_command(gh pr view)",
"run_shell_command(gh pr list)",
"run_shell_command(cat)",
"run_shell_command(head)",
"run_shell_command(tail)",
"run_shell_command(grep)",
"run_shell_command(find)",
"run_shell_command(git checkout)",
"run_shell_command(git add)",
"run_shell_command(git commit)",
"run_shell_command(git push)",
"run_shell_command(git status)",
"run_shell_command(git branch)",
"list_directory",
"read_file",
"read_many_files",
"write_file",
"replace",
"glob",
"search_file_content",
"web_fetch",
"google_web_search",
"save_memory"
],
"telemetry": {
"enabled": false,
"otlpGcpWifProvider": "${{ secrets.OTLP_GCP_WIF_PROVIDER }}",
"googleCloudProject": "${{ secrets.OTLP_GOOGLE_CLOUD_PROJECT }}"
}
}
+242
View File
@@ -0,0 +1,242 @@
name: Gemini PR Review
on:
issue_comment:
types: [created]
workflow_dispatch:
inputs:
pr_number:
description: 'PR number to review'
required: true
type: number
jobs:
review-pr:
if: >
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'issue_comment' &&
github.event.issue.pull_request &&
contains(github.event.comment.body, '@gemini') &&
(github.event.comment.author_association == 'OWNER' ||
github.event.comment.author_association == 'MEMBER' ||
github.event.comment.author_association == 'COLLABORATOR'))
timeout-minutes: 15
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write
pull-requests: write
issues: write
steps:
- name: Generate GitHub App Token
id: generate_token
uses: actions/create-github-app-token@v1
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
- name: Determine checkout ref
id: checkout_ref
run: |
if [ "${{ github.event_name }}" = "issue_comment" ]; then
echo "ref=refs/pull/${{ github.event.issue.number }}/head" >> "$GITHUB_OUTPUT"
else
echo "ref=${{ github.event.pull_request.head.sha }}" >> "$GITHUB_OUTPUT"
fi
- name: Checkout PR code
uses: actions/checkout@v4
with:
token: ${{ steps.generate_token.outputs.token }}
ref: ${{ steps.checkout_ref.outputs.ref }}
fetch-depth: 0
- name: Get PR details
id: get_pr
env:
GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }}
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
PR_NUMBER=${{ github.event.inputs.pr_number }}
elif [ "${{ github.event_name }}" = "issue_comment" ]; then
PR_NUMBER=${{ github.event.issue.number }}
else
PR_NUMBER=${{ github.event.pull_request.number }}
fi
echo "pr_number=$PR_NUMBER" >> "$GITHUB_OUTPUT"
# Extract additional instructions from comment (if triggered by comment)
ADDITIONAL_INSTRUCTIONS=""
if [ "${{ github.event_name }}" = "issue_comment" ]; then
COMMENT_BODY="${{ github.event.comment.body }}"
ADDITIONAL_INSTRUCTIONS=$(echo "$COMMENT_BODY" | sed 's/.*@gemini//' | xargs)
fi
echo "additional_instructions=$ADDITIONAL_INSTRUCTIONS" >> "$GITHUB_OUTPUT"
# Get PR details
PR_DATA=$(gh pr view $PR_NUMBER --json title,body,additions,deletions,changedFiles,baseRefName,headRefName)
echo "pr_data=$PR_DATA" >> "$GITHUB_OUTPUT"
# Get file changes
CHANGED_FILES=$(gh pr diff $PR_NUMBER --name-only)
echo "changed_files<<EOF" >> "$GITHUB_OUTPUT"
echo "$CHANGED_FILES" >> "$GITHUB_OUTPUT"
echo "EOF" >> "$GITHUB_OUTPUT"
- name: Run Gemini PR Review
uses: ./.github/actions/gemini
env:
GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }}
PR_NUMBER: ${{ steps.get_pr.outputs.pr_number }}
PR_DATA: ${{ steps.get_pr.outputs.pr_data }}
CHANGED_FILES: ${{ steps.get_pr.outputs.changed_files }}
ADDITIONAL_INSTRUCTIONS: ${{ steps.get_pr.outputs.additional_instructions }}
REPOSITORY: ${{ github.repository }}
with:
version: 0.1.8-rc.0
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
OTLP_GCP_WIF_PROVIDER: ${{ secrets.OTLP_GCP_WIF_PROVIDER }}
OTLP_GOOGLE_CLOUD_PROJECT: ${{ secrets.OTLP_GOOGLE_CLOUD_PROJECT }}
settings_json: |
{
"coreTools": [
"run_shell_command(echo)",
"run_shell_command(gh pr view)",
"run_shell_command(gh pr diff)",
"run_shell_command(gh pr comment)",
"run_shell_command(gh pr list)",
"run_shell_command(cat)",
"run_shell_command(head)",
"run_shell_command(tail)",
"run_shell_command(grep)",
"run_shell_command(find)",
"run_shell_command(git checkout)",
"run_shell_command(git add)",
"run_shell_command(git commit)",
"run_shell_command(git push)",
"run_shell_command(git status)",
"run_shell_command(git branch)",
"run_shell_command(git fetch)",
"list_directory",
"read_file",
"read_many_files",
"write_file",
"replace",
"glob",
"search_file_content",
"web_fetch",
"google_web_search",
"save_memory"
],
"telemetry": {
"enabled": false,
"target": "gcp"
},
"sandbox": false
}
prompt: |
You are an expert code reviewer and AI assistant for Pull Requests. You can review code AND make modifications to improve the PR.
**PR Details:**
- PR Number: #$PR_NUMBER
- Repository: $REPOSITORY
- Additional Instructions: $ADDITIONAL_INSTRUCTIONS
**CRITICAL: When making code modifications, follow this EXACT workflow:**
**STEP 1: Analyze the PR**
- Use: `echo "$PR_DATA"` to get PR details (JSON format)
- Use: `echo "$CHANGED_FILES"` to get changed files list
- Use: `gh pr diff ${PR_NUMBER}` to see the full diff
- Use: `gh pr view ${PR_NUMBER} --json title,body,headRefName` to get PR info
- Analyze the code changes thoroughly
**STEP 2: If Code Improvements Are Needed**
**IMPORTANT:** DO NOT create a new branch! Work on the EXISTING PR branch.
- Use: `git fetch origin` to get latest changes
- Get PR branch name: `gh pr view ${PR_NUMBER} --json headRefName`
- Switch to PR branch: `git checkout BRANCH_NAME` (use actual branch name from PR)
- Use: `git pull origin BRANCH_NAME` to get latest PR changes
**STEP 3: Make Code Modifications**
- Use `read_file` to examine existing code
- Use `write_file` or `replace` to make improvements
- Use `search_file_content` to find related code
- Make focused, targeted improvements to the PR
**STEP 4: Test Changes (if applicable)**
- Run relevant tests if test files exist
- Use `run_shell_command` to execute test commands
**STEP 5: Commit and Push Changes to EXISTING PR Branch**
- Use: `git add .` (or specific files)
- Use: `git commit -m "Gemini: [PR #${PR_NUMBER}] Brief description of improvements"`
- Use: `git push origin BRANCH_NAME` (push to EXISTING PR branch)
**STEP 6: Review and Comment on PR**
Write comprehensive review and post comment with your analysis and any improvements made.
Use: `gh pr comment ${PR_NUMBER} -b "MESSAGE"`
Review Guidelines:
- Focus on code quality, security, performance, and maintainability
- Check for common issues: potential bugs, security vulnerabilities, performance bottlenecks
- Verify error handling and edge cases
- Look for code style and best practices
- Comment on architecture and design decisions if significant
- Be constructive and specific in feedback
- Highlight both issues and positive aspects
- Suggest improvements with examples when possible
Review Areas:
- **Security**: Authentication, authorization, input validation, data sanitization
- **Performance**: Algorithms, database queries, caching, resource usage
- **Reliability**: Error handling, logging, testing coverage, edge cases
- **Maintainability**: Code structure, documentation, naming conventions
- **Functionality**: Logic correctness, requirements fulfillment
Output Format:
Structure your review using this exact format with markdown:
## 📋 Review Summary
Provide a brief 2-3 sentence overview of the PR and overall assessment.
## 🔍 General Feedback
- List general observations about code quality
- Mention overall patterns or architectural decisions
- Highlight positive aspects of the implementation
- Note any recurring themes across files
## 🎯 Specific Feedback
Only include sections below that have actual issues. If there are no issues in a priority category, omit that entire section.
### 🔴 Critical
(Only include this section if there are critical issues)
Issues that must be addressed before merging (security vulnerabilities, breaking changes, major bugs):
- **File: `filename:line`** - Description of critical issue with specific recommendation
### 🟡 High
(Only include this section if there are high priority issues)
Important issues that should be addressed (performance problems, design flaws, significant bugs):
- **File: `filename:line`** - Description of high priority issue with suggested fix
### 🟢 Medium
(Only include this section if there are medium priority issues)
Improvements that would enhance code quality (style issues, minor optimizations, better practices):
- **File: `filename:line`** - Description of medium priority improvement
### 🔵 Low
(Only include this section if there are suggestions)
Nice-to-have improvements and suggestions (documentation, naming, minor refactoring):
- **File: `filename:line`** - Description of suggestion or enhancement
**Note**: If no specific issues are found in any category, simply state "No specific issues identified in this review."
## ✅ Highlights
(Only include this section if there are positive aspects to highlight)
- Mention specific good practices or implementations
- Acknowledge well-written code sections
- Note improvements from previous versions
---
*Review completed by Gemini CLI*
+2 -2
View File
@@ -151,7 +151,7 @@ jobs:
return 'INVALID_TYPE'
print('🔧 Direct PraisonAI API Key Check:')
env_api_key = os.environ.get(\\"OPENAI_API_KEY\\", \\"NOT_SET\\")
env_api_key = os.environ.get(\"OPENAI_API_KEY\", \"NOT_SET\")
print(f'Environment OPENAI_API_KEY: {get_key_display_value(env_api_key)}...')
from praisonai import PraisonAI
@@ -163,7 +163,7 @@ jobs:
from praisonai.inc.models import PraisonAIModel
print('\\\\n🧪 Testing PraisonAIModel with explicit API key (CrewAI method):')
print('\\n🧪 Testing PraisonAIModel with explicit API key (CrewAI method):')
model_with_explicit_key = PraisonAIModel(
model='openai/gpt-4o-mini',
base_url=praisonai.config_list[0].get('base_url'),
+3 -1
View File
@@ -75,4 +75,6 @@ agents/praisonaiagents/praisonaiagents.egg-info
# Local Netlify folder
.netlify
.qodo
CopilotKit*
CopilotKit*
.gemini
issue_details.json
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
flask \
"praisonai>=2.2.46" \
"praisonai>=2.2.55" \
"praisonai[api]" \
gunicorn \
markdown
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.2.46" \
"praisonai>=2.2.55" \
"praisonai[chat]" \
"embedchain[github,youtube]"
+1 -1
View File
@@ -20,7 +20,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.2.46" \
"praisonai>=2.2.55" \
"praisonai[ui]" \
"praisonai[chat]" \
"praisonai[realtime]" \
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.2.46" \
"praisonai>=2.2.55" \
"praisonai[ui]" \
"praisonai[crewai]"
+2 -2
View File
@@ -121,7 +121,7 @@ healthcheck:
## 📦 Package Versions
All Docker images use consistent, up-to-date versions:
- PraisonAI: `>=2.2.46`
- PraisonAI: `>=2.2.55`
- PraisonAI Agents: `>=0.0.92`
- Python: `3.11-slim`
@@ -218,7 +218,7 @@ docker-compose up -d
### Version Pinning
To use specific versions, update the Dockerfile:
```dockerfile
RUN pip install "praisonai==2.2.46" "praisonaiagents==0.0.92"
RUN pip install "praisonai==2.2.55" "praisonaiagents==0.0.92"
```
## 🌐 Production Deployment
+142
View File
@@ -0,0 +1,142 @@
#!/usr/bin/env python3
"""
Fixed example demonstrating proper guardrail usage with PraisonAI Agents.
This addresses the issues reported in issue #875.
"""
from praisonaiagents import Agent, Task, GuardrailResult, PraisonAIAgents, TaskOutput
from typing import Tuple, Any
import trafilatura
# Example 1: Using GuardrailResult return type (now supported!)
def validate_length_guardrailresult(output: TaskOutput) -> GuardrailResult:
"""Ensure output is between 100-500 characters using GuardrailResult"""
# Extract the raw text from the TaskOutput object
text = output.raw if hasattr(output, 'raw') else str(output)
length = len(text)
if 100 <= length <= 500:
return GuardrailResult(
success=True,
result=output, # Pass through the original output
error=""
)
else:
return GuardrailResult(
success=False,
result=None,
error=f"Output must be 100-500 chars, got {length}"
)
# Example 2: Using Tuple[bool, Any] return type (original method)
def validate_length_tuple(output: TaskOutput) -> Tuple[bool, Any]:
"""Ensure output is between 100-500 characters using tuple"""
text = output.raw if hasattr(output, 'raw') else str(output)
length = len(text)
if 100 <= length <= 500:
return True, output
else:
return False, f"Output must be 100-500 chars, got {length}"
# Tool function
def get_url_context(url):
"""Fetch and extract content from a URL"""
downloaded = trafilatura.fetch_url(url)
if not downloaded:
return "Sorry, I couldn't fetch the content from that URL."
extracted = trafilatura.extract(
downloaded,
include_comments=False,
include_links=True,
output_format='json',
with_metadata=True,
url=url
)
if not extracted:
return "Sorry, I couldn't extract readable content from that page."
return extracted # returns JSON string
# Create agent with FIXED tools parameter (must be a list!)
agent = Agent(
name="Content Summarizer",
role="Content Analysis Expert",
goal="Summarize web content concisely",
instructions="You are a helpful assistant that summarizes web content",
llm="gemini/gemini-2.5-flash-lite-preview-06-17",
self_reflect=False,
verbose=True,
tools=[get_url_context] # FIX: tools must be a list, not a single function
)
# Create task with GuardrailResult guardrail
task_with_guardrailresult = Task(
name="summarise article with GuardrailResult",
description="get the context of this url: https://blog.google/technology/ai/dolphingemma/ and produce a summary below 500 characters",
agent=agent,
guardrail=validate_length_guardrailresult, # Using GuardrailResult
expected_output="summary of the article below 500 characters",
max_retries=3 # Will retry up to 3 times if guardrail fails
)
# Alternative: Create task with tuple guardrail
task_with_tuple = Task(
name="summarise article with tuple",
description="get the context of this url: https://blog.google/technology/ai/dolphingemma/ and produce a summary below 500 characters",
agent=agent,
guardrail=validate_length_tuple, # Using Tuple[bool, Any]
expected_output="summary of the article below 500 characters",
max_retries=3
)
# Example with string-based LLM guardrail
task_with_llm_guardrail = Task(
name="summarise with LLM guardrail",
description="get the context of this url: https://blog.google/technology/ai/dolphingemma/ and produce a summary",
agent=agent,
guardrail="Ensure the summary is professional, factual, and between 100-500 characters",
expected_output="professional summary of the article"
)
# Run with GuardrailResult example
print("=== Running with GuardrailResult guardrail ===")
agents_gr = PraisonAIAgents(
agents=[agent],
tasks=[task_with_guardrailresult]
)
# Uncomment to run:
# result_gr = agents_gr.start()
# Run with Tuple example
print("\n=== Running with Tuple[bool, Any] guardrail ===")
agents_tuple = PraisonAIAgents(
agents=[agent],
tasks=[task_with_tuple]
)
# Uncomment to run:
# result_tuple = agents_tuple.start()
# Run with LLM guardrail example
print("\n=== Running with LLM-based guardrail ===")
agents_llm = PraisonAIAgents(
agents=[agent],
tasks=[task_with_llm_guardrail]
)
# Uncomment to run:
# result_llm = agents_llm.start()
print("""
Key fixes applied:
1. GuardrailResult is now accepted as a valid return type annotation
2. tools parameter must be a list: tools=[get_url_context] not tools=get_url_context
3. Both GuardrailResult and Tuple[bool, Any] return types are supported
4. String-based LLM guardrails are also supported
The guardrail will automatically retry (up to max_retries times) if validation fails.
""")
@@ -0,0 +1,259 @@
# Multi-Provider/Multi-Model Support
PraisonAI now supports intelligent multi-provider and multi-model capabilities, allowing agents to automatically select the most appropriate AI model based on task requirements, cost considerations, and performance needs.
## 🚀 Key Features
### 1. **Automatic Model Selection**
- Analyzes task complexity and selects the best model
- Considers cost vs. performance trade-offs
- Supports fallback mechanisms for reliability
### 2. **Cost Optimization**
- Routes simple tasks to cheaper models (e.g., GPT-4o-mini, Gemini Flash)
- Reserves expensive models for complex tasks
- Tracks usage and provides cost estimates
### 3. **Multi-Provider Support**
- Works with OpenAI, Anthropic, Google, Groq, and more
- Seamless switching between providers
- Provider preference settings
### 4. **Flexible Routing Strategies**
- `auto`: Automatic selection based on task analysis
- `cost-optimized`: Prioritize cheaper models
- `performance-optimized`: Prioritize capability
- `manual`: Use specified model
## 📦 Installation
The multi-provider support is included in the standard installation:
```bash
pip install praisonaiagents
```
## 🔧 Basic Usage
### Simple Multi-Model Agent
```python
from praisonaiagents import Task, PraisonAIAgents
from praisonaiagents.agent import RouterAgent
# Create a multi-model agent
agent = RouterAgent(
name="Smart Assistant",
role="Adaptive AI Assistant",
goal="Complete tasks using the most appropriate model",
models=["gpt-4o-mini", "gpt-4o", "claude-3-5-sonnet-20241022"],
routing_strategy="auto" # Automatic model selection
)
# Create tasks
simple_task = Task(
name="calculate",
description="What is 15% of 250?",
agent=agent
)
complex_task = Task(
name="analyze",
description="Write a Python implementation of the A* pathfinding algorithm",
agent=agent
)
# Run tasks - agent will automatically select appropriate models
agents = PraisonAIAgents(
agents=[agent],
tasks=[simple_task, complex_task]
)
results = agents.start()
```
### Cost-Optimized Workflow
```python
from praisonaiagents.llm import ModelRouter
# Create custom router with cost constraints
router = ModelRouter(
cost_threshold=0.005, # Max $0.005 per 1k tokens
preferred_providers=["google", "openai"]
)
# Create cost-conscious agent
analyzer = RouterAgent(
name="Budget Analyzer",
role="Data Analyst",
goal="Analyze data efficiently",
models={
"gemini/gemini-1.5-flash": {},
"gpt-4o-mini": {},
"claude-3-haiku-20240307": {}
},
model_router=router,
routing_strategy="cost-optimized"
)
```
## 🎯 Routing Strategies
### Auto Routing (Default)
Automatically selects models based on:
- Task complexity analysis
- Required capabilities (tools, vision, etc.)
- Context size requirements
- Cost/performance balance
### Cost-Optimized
Prioritizes cheaper models while meeting task requirements:
- Uses lightweight models for simple tasks
- Only escalates to expensive models when necessary
- Ideal for high-volume operations
### Performance-Optimized
Prioritizes model capability:
- Uses the best available model for the task
- Ideal for quality-critical applications
- Less concern for cost
### Manual
Uses the specified model without routing:
- Direct control over model selection
- Useful for testing or specific requirements
## 📊 Model Profiles
The system includes pre-configured profiles for popular models:
| Model | Provider | Best For | Cost/1k tokens |
|-------|----------|----------|----------------|
| gpt-4o-mini | OpenAI | Simple tasks, speed | $0.00075 |
| gemini-1.5-flash | Google | Cost-effective, multimodal | $0.000125 |
| claude-3-haiku | Anthropic | Fast responses | $0.0008 |
| gpt-4o | OpenAI | General purpose | $0.0075 |
| claude-3-5-sonnet-20241022 | Anthropic | Complex reasoning | $0.009 |
| deepseek-chat | DeepSeek | Code & math | $0.0014 |
## 🔍 Task Complexity Analysis
The system analyzes tasks to determine complexity:
- **Simple**: Basic calculations, definitions, yes/no questions
- **Moderate**: Summarization, basic analysis, classification
- **Complex**: Code generation, algorithm implementation
- **Very Complex**: Multi-step reasoning, system design
## 📈 Usage Tracking
Track model usage and costs:
```python
# Get usage report
report = agent.get_usage_report()
print(report)
# Output:
{
'agent_name': 'Smart Assistant',
'routing_strategy': 'auto',
'model_usage': {
'gpt-4o-mini': {'calls': 5, 'tokens': 1500, 'cost': 0.0011},
'gpt-4o': {'calls': 2, 'tokens': 3000, 'cost': 0.0225}
},
'total_cost_estimate': 0.0236,
'total_calls': 7
}
```
## 🛠️ Custom Model Configuration
Add custom model profiles:
```python
from praisonaiagents.llm import ModelProfile, TaskComplexity
custom_model = ModelProfile(
name="custom-model",
provider="custom",
complexity_range=(TaskComplexity.SIMPLE, TaskComplexity.MODERATE),
cost_per_1k_tokens=0.002,
strengths=["domain-specific"],
capabilities=["text"],
context_window=32000
)
router = ModelRouter(models=[custom_model])
```
## 🔗 Integration with AutoAgents
Works seamlessly with AutoAgents:
```python
from praisonaiagents.agents import AutoAgents
# Create auto agents
auto = AutoAgents(
instructions="Analyze market trends and create a report",
max_agents=3
)
# Convert to multi-model agents
for i, agent in enumerate(auto.agents):
auto.agents[i] = RouterAgent(
name=agent.name,
role=agent.role,
goal=agent.goal,
models=["gpt-4o-mini", "gpt-4o", "claude-3-5-sonnet"],
routing_strategy="auto"
)
results = auto.start()
```
## 🌟 Best Practices
1. **Start with Auto Routing**: Let the system learn your needs
2. **Monitor Costs**: Use usage reports to optimize
3. **Set Cost Thresholds**: Prevent unexpected expenses
4. **Use Appropriate Models**: Don't use GPT-4 for simple math
5. **Leverage Fallbacks**: Configure fallback models for reliability
## 🔐 Environment Variables
Set API keys for each provider:
```bash
export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."
export GEMINI_API_KEY="..."
export GROQ_API_KEY="..."
```
## 🤝 Contributing
The multi-provider system is extensible. To add new models:
1. Add model profile to `ModelRouter.DEFAULT_MODELS`
2. Ensure the provider is supported by LiteLLM
3. Test with various task complexities
## 📚 Examples
See the complete example in `examples/python/agents/multi-provider-agent.py` for:
- Auto-routing examples
- Cost-optimized workflows
- Performance-optimized agents
- Custom routing logic
- Integration patterns
## 🎉 Benefits
- **Cost Savings**: Reduce API costs by 50-80%
- **Better Performance**: Use the right tool for each job
- **Flexibility**: Switch providers without code changes
- **Reliability**: Automatic fallbacks and error handling
- **Transparency**: Track usage and costs
@@ -0,0 +1,285 @@
"""
Multi-Provider/Multi-Model Agent Example
This example demonstrates how to use multiple AI providers/models with intelligent
agent-based selection for cost optimization and performance.
"""
import os
from praisonaiagents import Agent, Task, PraisonAIAgents
from praisonaiagents.agent import RouterAgent
from praisonaiagents.llm.model_router import ModelRouter, ModelProfile, TaskComplexity
# Example 1: Simple Multi-Model Agent with Auto-Routing
def example_auto_routing():
"""Example of automatic model routing based on task complexity"""
print("\n=== Example 1: Auto-Routing Multi-Model Agent ===\n")
# Create a router agent that automatically selects models
research_agent = RouterAgent(
name="Smart Researcher",
role="Adaptive Research Assistant",
goal="Research topics using the most appropriate AI model",
backstory="I analyze task complexity and route to the best model",
models=["gpt-4o-mini", "gpt-4o", "claude-3-5-sonnet-20241022"],
routing_strategy="auto", # Automatic model selection
verbose=True
)
# Create tasks with different complexity levels
simple_task = Task(
name="simple_calculation",
description="Calculate the sum of 1542 and 2873",
expected_output="The numerical result",
agent=research_agent
)
moderate_task = Task(
name="summarize_article",
description="Summarize the key points about renewable energy trends",
expected_output="A concise summary with main points",
agent=research_agent
)
complex_task = Task(
name="analyze_code",
description="Implement a Python function to find the longest palindromic substring in a given string with O(n^2) complexity",
expected_output="Complete Python implementation with explanation",
agent=research_agent
)
# Run the tasks
agents = PraisonAIAgents(
agents=[research_agent],
tasks=[simple_task, moderate_task, complex_task],
process="sequential",
verbose=True
)
results = agents.start()
# Show usage report
print("\n=== Model Usage Report ===")
print(research_agent.get_usage_report())
# Example 2: Cost-Optimized Workflow with Routing
def example_cost_optimized_workflow():
"""Example of cost-optimized workflow with model routing"""
print("\n=== Example 2: Cost-Optimized Workflow ===\n")
# Create custom model router with cost threshold
cost_router = ModelRouter(
cost_threshold=0.01, # Max $0.01 per 1k tokens
preferred_providers=["google", "openai", "anthropic"] # Provider preference
)
# Create specialized agents with different model strategies
analyzer = RouterAgent(
name="Cost-Conscious Analyzer",
role="Data Analyzer",
goal="Analyze data efficiently while minimizing costs",
models={
"gemini/gemini-1.5-flash": {},
"gpt-4o-mini": {},
"deepseek-chat": {}
},
model_router=cost_router,
routing_strategy="cost-optimized",
verbose=True
)
writer = RouterAgent(
name="Quality Writer",
role="Content Writer",
goal="Create high-quality content",
models={
"claude-3-5-sonnet-20241022": {},
"gpt-4o": {}
},
routing_strategy="performance-optimized", # Prefer better models
verbose=True
)
# Create workflow tasks
analysis_task = Task(
name="analyze_data",
description="Analyze this dataset: [1, 2, 3, 4, 5]. Find mean, median, and mode.",
expected_output="Statistical analysis results",
agent=analyzer
)
writing_task = Task(
name="write_report",
description="Write a professional report based on the analysis",
expected_output="Professional report with insights",
agent=writer,
context=[analysis_task] # Use results from analysis
)
# Run workflow
workflow = PraisonAIAgents(
agents=[analyzer, writer],
tasks=[analysis_task, writing_task],
process="sequential",
verbose=True
)
results = workflow.start()
# Show cost comparison
print("\n=== Cost Analysis ===")
print(f"Analyzer costs: {analyzer.get_usage_report()}")
print(f"Writer costs: {writer.get_usage_report()}")
# Example 3: Auto Agent Mode with Multi-Provider Support
def example_auto_agents_multi_provider():
"""Example using AutoAgents with multi-provider support"""
print("\n=== Example 3: AutoAgents with Multi-Provider Support ===\n")
from praisonaiagents.agents import AutoAgents
# Create AutoAgents that will automatically assign appropriate models
auto_agents = AutoAgents(
instructions="Create a market research report on electric vehicles. Include data analysis, competitor analysis, and future projections.",
max_agents=3,
llm="gpt-4o-mini", # Default model for agent generation
verbose=True
)
# After agents are created, upgrade them to multi-model agents
multi_model_agents = []
for agent in auto_agents.agents:
# Convert regular agents to multi-model agents
multi_agent = RouterAgent(
name=agent.name,
role=agent.role,
goal=agent.goal,
backstory=agent.backstory,
tools=agent.tools,
models=["gpt-4o-mini", "gemini/gemini-1.5-flash", "claude-3-haiku-20240307", "gpt-4o"],
routing_strategy="auto",
verbose=True
)
multi_model_agents.append(multi_agent)
# Update the agents in the AutoAgents instance
auto_agents.agents = multi_model_agents
# Run the auto-generated workflow
results = auto_agents.start()
# Show model usage across all agents
print("\n=== Multi-Provider Usage Summary ===")
total_calls = 0
for agent in multi_model_agents:
report = agent.get_usage_report()
print(f"\nAgent: {agent.name}")
for model, stats in report['model_usage'].items():
if stats['calls'] > 0:
print(f" - {model}: {stats['calls']} calls")
total_calls += stats['calls']
print(f"\nTotal API calls: {total_calls}")
# Example 4: Custom Routing Logic
def example_custom_routing():
"""Example with custom routing logic based on specific requirements"""
print("\n=== Example 4: Custom Routing Logic ===\n")
# Create custom model profiles for specific use cases
custom_models = [
ModelProfile(
name="gpt-4o",
provider="openai",
complexity_range=(TaskComplexity.SIMPLE, TaskComplexity.VERY_COMPLEX),
cost_per_1k_tokens=0.0075,
strengths=["code-generation", "debugging"],
capabilities=["text", "function-calling"],
context_window=128000
),
ModelProfile(
name="deepseek-chat",
provider="deepseek",
complexity_range=(TaskComplexity.MODERATE, TaskComplexity.VERY_COMPLEX),
cost_per_1k_tokens=0.001,
strengths=["mathematics", "algorithms"],
capabilities=["text", "function-calling"],
context_window=128000
)
]
# Create router with custom models
custom_router = ModelRouter(models=custom_models)
# Tool for web search (example)
def search_web(query: str) -> str:
"""Search the web for information"""
return f"Search results for: {query}"
# Create specialized agent
coder = RouterAgent(
name="Adaptive Coder",
role="Software Developer",
goal="Write and debug code using the best model for each task",
backstory="I'm an expert coder who knows when to use different AI models",
model_router=custom_router,
routing_strategy="auto",
tools=[search_web],
verbose=True
)
# Create coding tasks
tasks = [
Task(
name="fix_bug",
description="Fix this Python bug: 'list' object has no attribute 'appendx'",
expected_output="Corrected code with explanation",
agent=coder
),
Task(
name="implement_algorithm",
description="Implement Dijkstra's shortest path algorithm in Python",
expected_output="Complete implementation with comments",
agent=coder,
tools=[search_web]
)
]
# Run tasks
agents = PraisonAIAgents(
agents=[coder],
tasks=tasks,
process="sequential",
verbose=True
)
results = agents.start()
print("\n=== Custom Routing Results ===")
print(coder.get_usage_report())
# Run examples
if __name__ == "__main__":
# Make sure to set your API keys
# os.environ["OPENAI_API_KEY"] = "your-openai-key"
# os.environ["ANTHROPIC_API_KEY"] = "your-anthropic-key"
# os.environ["GEMINI_API_KEY"] = "your-gemini-key"
# Run examples
example_auto_routing()
example_cost_optimized_workflow()
example_auto_agents_multi_provider()
example_custom_routing()
print("\n=== All Examples Completed ===")
print("\nKey Features Demonstrated:")
print("1. Automatic model selection based on task complexity")
print("2. Cost-optimized routing for budget-conscious operations")
print("3. Performance-optimized routing for quality-critical tasks")
print("4. Multi-provider support with fallback mechanisms")
print("5. Integration with AutoAgents for automatic workflow generation")
print("6. Custom routing logic for specialized use cases")
@@ -0,0 +1,197 @@
"""
Hosted MCP Server implementation for PraisonAI Agents.
This module provides a base class for creating hosted MCP servers
that can handle requests and integrate with the MCP protocol.
Note: This is an example implementation. To use it, ensure you have installed:
pip install praisonaiagents[mcp] starlette>=0.27.0
"""
import asyncio
import logging
from typing import Dict, Any, Optional, List, Callable
import json
try:
from mcp.server.fastmcp import FastMCP
from mcp.server import Server
from starlette.applications import Starlette
from starlette.requests import Request
from starlette.routing import Mount, Route
from mcp.server.sse import SseServerTransport
import uvicorn
except ImportError:
raise ImportError(
"MCP server dependencies not installed. "
"Please install with: pip install praisonaiagents[mcp] starlette>=0.27.0"
)
logger = logging.getLogger(__name__)
class HostedMCPServer:
"""
Base class for creating hosted MCP servers.
This class provides a foundation for building MCP servers that can:
- Handle incoming requests
- Define custom tools
- Support SSE transport
- Be extended with custom functionality like latency tracking
"""
def __init__(self, name: str = "hosted-mcp-server", host: str = "localhost", port: int = 8080):
"""
Initialize the hosted MCP server.
Args:
name: Server name for identification
host: Host to bind to (default: localhost)
port: Port to listen on (default: 8080)
"""
self.name = name
self.host = host
self.port = port
self.mcp = FastMCP(name)
self._tools: Dict[str, Callable] = {}
self._server: Optional[Server] = None
self._app: Optional[Starlette] = None
def handle_request(self, request_data: Dict[str, Any]) -> Dict[str, Any]:
"""
Handle incoming MCP requests.
This method can be overridden in subclasses to add custom request handling,
such as latency tracking, authentication, or request modification.
Args:
request_data: The incoming request data
Returns:
Response data
"""
# Default implementation - can be overridden
method = request_data.get('method', '')
request_id = request_data.get('id', 'unknown')
logger.debug(f"Handling request {request_id}: {method}")
# Basic response structure
response = {
'id': request_id,
'jsonrpc': '2.0',
'result': {}
}
return response
def add_tool(self, func: Callable, name: Optional[str] = None, description: Optional[str] = None):
"""
Add a tool to the MCP server.
Args:
func: The function to expose as a tool
name: Optional name for the tool (defaults to function name)
description: Optional description for the tool
"""
tool_name = name or func.__name__
# Register with FastMCP
if asyncio.iscoroutinefunction(func):
# Already async
self.mcp.tool(name=tool_name)(func)
else:
# Wrap sync function in async
async def async_wrapper(*args, **kwargs):
return func(*args, **kwargs)
async_wrapper.__name__ = func.__name__
async_wrapper.__doc__ = description or func.__doc__
self.mcp.tool(name=tool_name)(async_wrapper)
self._tools[tool_name] = func
logger.info(f"Added tool: {tool_name}")
def create_app(self, debug: bool = False) -> Starlette:
"""
Create a Starlette application for serving the MCP server.
Args:
debug: Enable debug mode
Returns:
Starlette application instance
Raises:
RuntimeError: If the MCP server is not properly initialized
"""
if not self._server:
if not hasattr(self.mcp, '_mcp_server'):
raise RuntimeError("MCP server not properly initialized. Ensure FastMCP is correctly set up.")
self._server = self.mcp._mcp_server
sse = SseServerTransport("/messages/")
async def handle_sse(request: Request) -> None:
logger.debug(f"SSE connection from {request.client}")
async with sse.connect_sse(
request.scope,
request.receive,
request._send,
) as (read_stream, write_stream):
await self._server.run(
read_stream,
write_stream,
self._server.create_initialization_options(),
)
self._app = Starlette(
debug=debug,
routes=[
Route("/sse", endpoint=handle_sse),
Mount("/messages/", app=sse.handle_post_message),
],
)
return self._app
def start(self, debug: bool = False, **uvicorn_kwargs):
"""
Start the MCP server.
Args:
debug: Enable debug mode
**uvicorn_kwargs: Additional arguments to pass to uvicorn
"""
app = self.create_app(debug=debug)
print(f"Starting {self.name} MCP server on {self.host}:{self.port}")
print(f"Available tools: {', '.join(self._tools.keys())}")
print(f"SSE endpoint: http://{self.host}:{self.port}/sse")
uvicorn.run(app, host=self.host, port=self.port, **uvicorn_kwargs)
async def start_async(self, debug: bool = False):
"""
Start the MCP server asynchronously.
Args:
debug: Enable debug mode
"""
app = self.create_app(debug=debug)
config = uvicorn.Config(app, host=self.host, port=self.port)
server = uvicorn.Server(config)
print(f"Starting {self.name} MCP server on {self.host}:{self.port}")
print(f"Available tools: {', '.join(self._tools.keys())}")
await server.serve()
def get_tools(self) -> List[str]:
"""Get list of available tool names."""
return list(self._tools.keys())
def get_endpoint(self) -> str:
"""Get the SSE endpoint URL."""
return f"http://{self.host}:{self.port}/sse"
@@ -6,7 +6,7 @@ without modifying PraisonAI core files.
"""
from praisonaiagents import Agent, PraisonAIAgents
from praisonaiagents.mcp import HostedMCPServer
from hosted_server import HostedMCPServer # Import from local file
from latency_tracker_tool import tracker, get_latency_metrics
import json
+1 -1
View File
@@ -72,7 +72,7 @@ workflow = PraisonAIAgents(
)
# Run the workflow
results = workflow.start()
results = workflow.start(return_dict=True)
# Print results
print("\nWorkflow Results:")
@@ -0,0 +1,13 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a competitor intelligence AI agent. "
"Help users analyze competitors, market trends, and business intelligence "
"to provide strategic insights and competitive advantages.",
llm="xai/grok-4"
)
response = agent.start(
"I need to analyze my main competitor's pricing strategy and market positioning. "
"Can you help me gather insights?"
)
@@ -0,0 +1,13 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a finance AI agent. "
"Help users with financial analysis, investment strategies, "
"budget planning, and market trend analysis for informed decision making.",
llm="xai/grok-4"
)
response = agent.start(
"I need to analyze my investment portfolio and create a diversification strategy. "
"Can you help me optimize my financial planning?"
)
@@ -0,0 +1,13 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a game design AI agent. "
"Help users with game concept development, mechanics design, "
"and creative storytelling for various gaming platforms and genres.",
llm="xai/grok-4"
)
response = agent.start(
"I want to create a mobile puzzle game concept. "
"Can you help me design engaging gameplay mechanics and story elements?"
)
@@ -0,0 +1,13 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a legal AI agent. "
"Help users with legal document analysis, contract review, "
"and legal advice while ensuring compliance and risk assessment.",
llm="xai/grok-4"
)
response = agent.start(
"I need to review a software licensing agreement. "
"Can you help me identify potential risks and key terms?"
)
@@ -0,0 +1,13 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a recruitment AI agent. "
"Help users with candidate screening, job posting optimization, "
"and recruitment process automation to find the best talent.",
llm="xai/grok-4"
)
response = agent.start(
"I need to create a job posting for a senior software engineer position. "
"Can you help me write an attractive and comprehensive job description?"
)
@@ -0,0 +1,13 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are an AI services agency agent. "
"Help users with comprehensive business services including marketing, "
"content creation, strategy, and operational optimization using CrewAI framework.",
llm="xai/grok-4"
)
response = agent.start(
"I need help with my startup's marketing strategy and content creation. "
"Can you provide a comprehensive plan?"
)
@@ -0,0 +1,13 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are an AI teaching agent. "
"Help users with educational content creation, lesson planning, "
"and personalized learning experiences across various subjects.",
llm="xai/grok-4"
)
response = agent.start(
"I need to create a lesson plan for teaching Python programming to beginners. "
"Can you help me design an engaging curriculum?"
)
@@ -0,0 +1,8 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a data analysis AI agent. Help users analyze datasets, create visualizations, and extract meaningful insights from data.",
llm="xai/grok-4"
)
response = agent.start("I have sales data for the last quarter. Can you help me identify trends and patterns?")
@@ -0,0 +1,8 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a deep research AI agent. Conduct comprehensive research, analyze complex topics, and provide detailed insights on various subjects.",
llm="xai/grok-4"
)
response = agent.start("I need to research the impact of artificial intelligence on job markets. Can you provide a comprehensive analysis?")
@@ -0,0 +1,8 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a health and fitness AI agent. Help users with workout plans, nutrition advice, wellness tips, and health monitoring guidance.",
llm="xai/grok-4"
)
response = agent.start("I want to lose 20 pounds and build muscle. What's the best workout and nutrition plan for me?")
@@ -0,0 +1,8 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are an investment advisor AI agent. Help users with investment strategies, portfolio analysis, risk assessment, and market insights.",
llm="xai/grok-4"
)
response = agent.start("I have $50,000 to invest. What would be a good diversified portfolio for someone in their 30s?")
@@ -0,0 +1,8 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a journalist AI agent. Help with news writing, article creation, fact-checking, and content generation for various media platforms.",
llm="xai/grok-4"
)
response = agent.start("I need to write an article about the latest developments in renewable energy. Can you help me structure it?")
@@ -0,0 +1,8 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a medical imaging AI agent. Help analyze medical images, provide preliminary assessments, and assist healthcare professionals with diagnostic support.",
llm="xai/grok-4"
)
response = agent.start("I have an X-ray image that needs analysis. Can you help identify any abnormalities?")
@@ -0,0 +1,8 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a music generation AI agent. Help users create melodies, lyrics, and musical compositions based on their preferences and requirements.",
llm="xai/grok-4"
)
response = agent.start("I need a catchy jingle for my coffee shop. Can you help me create one?")
@@ -0,0 +1,8 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a personal finance AI agent. Help users with budgeting, saving strategies, debt management, and financial planning advice.",
llm="xai/grok-4"
)
response = agent.start("I want to save $10,000 in the next year. What's the best strategy for me?")
@@ -0,0 +1,8 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a startup trend analysis AI agent. Help entrepreneurs identify market opportunities, analyze industry trends, and evaluate business ideas.",
llm="xai/grok-4"
)
response = agent.start("I'm thinking of starting a tech startup. What are the current trending opportunities in the market?")
@@ -0,0 +1,8 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a travel planning AI agent. Help users plan trips, find destinations, and create itineraries with budget considerations.",
llm="xai/grok-4"
)
response = agent.start("I want to plan a 7-day trip to Europe with a budget of $3000. What are my best options?")
@@ -0,0 +1,13 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a multi MCP AI agent. "
"Help users integrate and coordinate multiple Model Context Protocol agents "
"for complex workflows and cross-platform automation.",
llm="xai/grok-4"
)
response = agent.start(
"I need to coordinate multiple tools and services for my project. "
"Can you help me set up an integrated workflow using MCP agents?"
)
@@ -0,0 +1,13 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a multimodal coding AI agent. "
"Help users with code generation, review, and debugging "
"by analyzing both text and visual inputs like screenshots and diagrams.",
llm="xai/grok-4"
)
response = agent.start(
"I have a screenshot of a UI design and need to generate the corresponding HTML/CSS code. "
"Can you help me create the frontend code?"
)
@@ -0,0 +1,13 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a multimodal design AI agent. "
"Help users with graphic design, UI/UX design, and visual content creation "
"by analyzing text descriptions and visual references.",
llm="xai/grok-4"
)
response = agent.start(
"I need to create a logo for my tech startup. "
"Can you help me design a modern and professional logo concept?"
)
+46 -4
View File
@@ -139,10 +139,32 @@ task = Task(
#### Task-Level Guardrails
```python
from typing import Tuple, Any
from praisonaiagents import GuardrailResult
# Function-based guardrail
def validate_output(task_output: TaskOutput) -> Tuple[bool, Any]:
"""Custom validation function."""
# Function-based guardrail (Option 1: Using GuardrailResult)
def validate_output(task_output: TaskOutput) -> GuardrailResult:
"""Custom validation function returning GuardrailResult."""
if "error" in task_output.raw.lower():
return GuardrailResult(
success=False,
result=None,
error="Output contains errors"
)
if len(task_output.raw) < 10:
return GuardrailResult(
success=False,
result=None,
error="Output is too short"
)
return GuardrailResult(
success=True,
result=task_output,
error=""
)
# Function-based guardrail (Option 2: Using Tuple[bool, Any])
def validate_output_tuple(task_output: TaskOutput) -> Tuple[bool, Any]:
"""Custom validation function returning tuple."""
if "error" in task_output.raw.lower():
return False, "Output contains errors"
if len(task_output.raw) < 10:
@@ -170,7 +192,27 @@ task = Task(
#### Agent-Level Guardrails
```python
# Agent guardrails apply to ALL outputs from that agent
def validate_professional_tone(task_output: TaskOutput) -> Tuple[bool, Any]:
# Option 1: Using GuardrailResult
def validate_professional_tone(task_output: TaskOutput) -> GuardrailResult:
"""Ensure professional tone in all agent responses."""
content = task_output.raw.lower()
casual_words = ['yo', 'dude', 'awesome', 'cool']
for word in casual_words:
if word in content:
return GuardrailResult(
success=False,
result=None,
error=f"Unprofessional language detected: {word}"
)
return GuardrailResult(
success=True,
result=task_output,
error=""
)
# Option 2: Using Tuple[bool, Any]
def validate_professional_tone_tuple(task_output: TaskOutput) -> Tuple[bool, Any]:
"""Ensure professional tone in all agent responses."""
content = task_output.raw.lower()
casual_words = ['yo', 'dude', 'awesome', 'cool']
+62
View File
@@ -0,0 +1,62 @@
from praisonaiagents import AutoAgents
from langchain_community.utilities.tavily_search import TavilySearchAPIWrapper
def tavily(query: str):
api_wrapper = TavilySearchAPIWrapper()
results = api_wrapper.results(query=query, max_results=5)
return results
# Create AutoAgents instance
agents = AutoAgents(
instructions="Search for information about AI Agents",
tools=[tavily],
process="sequential",
verbose=True,
max_agents=3 # Maximum number of agents to create
)
# Start the agents
result = agents.start()
print(result)
##or
from praisonaiagents import AutoAgents
from praisonaiagents.tools import (
evaluate, solve_equation, convert_units,
calculate_statistics, calculate_financial
)
def get_stock_price(company_name: str) -> str:
"""
Get the stock price of a company
Args:
company_name (str): The name of the company
Returns:
str: The stock price of the company
"""
if company_name.lower() == "apple" or company_name.lower() == "aapl":
return f"The stock price of {company_name} is 100"
elif company_name.lower() == "google" or company_name.lower() == "googl":
return f"The stock price of {company_name} is 200"
else:
return f"The stock price of {company_name} is 50"
# Create AutoAgents instance
agents = AutoAgents(
instructions="Get the stock price of google and compare it to the stock price of apple",
tools=[evaluate, get_stock_price],
process="sequential",
llm="gemini/gemini-2.5-flash-lite-preview-06-17",
self_reflect=False,
verbose=False,
max_agents=3 # Maximum number of agents to create
)
# Start the agents
result = agents.start()
print(result)
@@ -0,0 +1,12 @@
from praisonaiagents import Agent
def get_weather(city: str) -> str:
return f"The weather in {city} is sunny"
agent = Agent(
instructions="You are a helpful assistant",
llm="gemini/gemini-1.5-flash-latest",
tools=[get_weather]
)
agent.start("What is the weather in Tokyo?")
+30
View File
@@ -0,0 +1,30 @@
from praisonaiagents import Agent
def get_stock_price(company_name: str) -> str:
"""
Get the stock price of a company
Args:
company_name (str): The name of the company
Returns:
str: The stock price of the company
"""
return f"The stock price of {company_name} is 100"
def multiply(a: int, b: int) -> int:
"""
Multiply two numbers
"""
return a * b
agent = Agent(
instructions="You are a helpful assistant. You can use the tools provided to you to help the user.",
llm="gemini/gemini-2.5-pro",
tools=[get_stock_price, multiply]
)
result = agent.start("what is the stock price of Google? multiply the Google stock price with 2")
print(result)
+25
View File
@@ -0,0 +1,25 @@
from pydantic import BaseModel
from praisonaiagents import Agent, Task, Agents
class Recipe(BaseModel):
recipe_name: str
ingredients: list[str]
agent = Agent(
name="Chef",
role="Recipe Creator",
llm="gemini/gemini-2.5-flash"
)
task = Task(
description="Create a cookie recipe",
agent=agent,
output_pydantic=Recipe # Will use Gemini's native structured output!
)
agents = Agents(
agents=[agent],
tasks=[task]
)
agents.start()
@@ -0,0 +1,32 @@
#!/usr/bin/env python3
"""Test script to verify the Gemini JSON parsing fix."""
from praisonaiagents import Agent
# Test with minimal configuration to isolate the issue
llm_config = {
"model": "gemini/gemini-1.5-flash-latest",
"temperature": 0.7,
"max_tokens": 500,
}
# Create agent with self-reflection enabled
agent = Agent(
instructions="You are a helpful assistant. Be concise and clear.",
llm=llm_config,
verbose=True,
self_reflect=True,
max_reflect=2,
min_reflect=1
)
# Test with a simple prompt
print("Testing Gemini with self-reflection...")
try:
response = agent.start("What is 2+2? Explain briefly.")
print(f"\nFinal response: {response}")
print("\nTest completed successfully!")
except Exception as e:
print(f"\nError occurred: {e}")
import traceback
traceback.print_exc()
+179
View File
@@ -0,0 +1,179 @@
from praisonaiagents import Agent
from praisonaiagents.agent.handoff import handoff, handoff_filters
from pydantic import BaseModel
from typing import Dict, Any
import logging
# Mock data for testing
MOCK_ORDERS = {
"12345": {
"id": "12345",
"status": "shipped",
"items": ["Widget A", "Widget B"],
"total": 99.99,
"tracking_number": "TRK123456789",
"shipping_address": "123 Main St, City, ST 12345"
},
"67890": {
"id": "67890",
"status": "processing",
"items": ["Gadget X"],
"total": 149.99,
"tracking_number": None,
"shipping_address": "456 Oak Ave, Town, ST 67890"
}
}
MOCK_SHIPMENTS = {
"TRK123456789": {
"tracking_number": "TRK123456789",
"status": "in_transit",
"location": "Distribution Center - Chicago",
"estimated_delivery": "2024-01-15",
"updates": [
{"date": "2024-01-10", "status": "picked_up", "location": "Warehouse"},
{"date": "2024-01-12", "status": "in_transit", "location": "Chicago"}
]
}
}
MOCK_TICKETS = []
# Mock tool functions
def check_order_status(order_id: str) -> Dict[str, Any]:
"""Check the status of an order"""
if order_id in MOCK_ORDERS:
return {"success": True, "order": MOCK_ORDERS[order_id]}
return {"success": False, "error": f"Order {order_id} not found"}
def track_shipment(tracking_number: str) -> Dict[str, Any]:
"""Track a shipment by tracking number"""
if tracking_number in MOCK_SHIPMENTS:
return {"success": True, "shipment": MOCK_SHIPMENTS[tracking_number]}
return {"success": False, "error": f"Tracking number {tracking_number} not found"}
def process_refund(order_id: str, reason: str = "customer_request") -> Dict[str, Any]:
"""Process a refund for an order"""
if order_id in MOCK_ORDERS:
order = MOCK_ORDERS[order_id]
return {
"success": True,
"refund_id": f"REF{order_id}",
"amount": order["total"],
"status": "processing",
"estimated_days": 3-5
}
return {"success": False, "error": f"Order {order_id} not found"}
def check_refund_policy(item_type: str = "general") -> Dict[str, Any]:
"""Check refund policy for different item types"""
policies = {
"general": {"days": 30, "condition": "unopened", "shipping": "customer_pays"},
"electronics": {"days": 15, "condition": "original_packaging", "shipping": "free"},
"clothing": {"days": 60, "condition": "tags_attached", "shipping": "customer_pays"}
}
return {"success": True, "policy": policies.get(item_type, policies["general"])}
def diagnose_issue(description: str) -> Dict[str, Any]:
"""Diagnose a technical issue"""
common_solutions = {
"login": "Clear cookies and cache, try password reset",
"payment": "Check card details, try different payment method",
"loading": "Check internet connection, try different browser",
"mobile": "Update app, restart device, check storage space"
}
# Simple keyword matching for demo
for issue_type, solution in common_solutions.items():
if issue_type in description.lower():
return {"success": True, "diagnosis": solution, "severity": "low"}
return {"success": True, "diagnosis": "Please provide more details", "severity": "medium"}
def create_ticket(issue_description: str, priority: str = "medium") -> Dict[str, Any]:
"""Create a support ticket"""
ticket_id = f"TICK{len(MOCK_TICKETS) + 1001}"
ticket = {
"id": ticket_id,
"description": issue_description,
"priority": priority,
"status": "open",
"created_at": "2024-01-13T10:00:00Z"
}
MOCK_TICKETS.append(ticket)
return {"success": True, "ticket": ticket}
def log_escalation(source_agent: str, escalation_data: Dict[str, Any]) -> None:
"""Log escalation for audit purposes"""
logging.info(f"Escalation from {source_agent}: {escalation_data}")
# Escalation with structured input
class EscalationRequest(BaseModel):
issue_type: str
severity: str
description: str
# Specialist agents
order_agent = Agent(
name="Order Management",
instructions="You help with order status and tracking.",
tools=[check_order_status, track_shipment]
)
refund_agent = Agent(
name="Refund Specialist",
instructions="You process refunds and handle return requests.",
tools=[process_refund, check_refund_policy]
)
technical_agent = Agent(
name="Technical Support",
instructions="You solve technical issues and provide guidance.",
tools=[diagnose_issue, create_ticket]
)
escalation_agent = Agent(
name="Senior Manager",
instructions="You handle escalated issues requiring management attention."
)
# Main triage agent
triage_agent = Agent(
name="Customer Service",
instructions="""You are the first point of contact. Route customers to the right specialist:
- Order issues → Order Management
- Refunds → Refund Specialist
- Technical problems → Technical Support
- Complex issues → Escalate to management
""",
handoffs=[
order_agent,
refund_agent,
technical_agent,
handoff(
escalation_agent,
tool_name_override="escalate_to_management",
input_type=EscalationRequest,
on_handoff=lambda src, data: log_escalation(src.name, data)
)
]
)
# Start the service
if __name__ == "__main__":
print("Customer Service Handoff System Demo")
print("=" * 40)
# Test different scenarios
test_queries = [
"I need a refund for order #12345",
"What's the status of order #67890?",
"I'm having trouble logging into my account",
"This is urgent - I need to speak to a manager about a serious billing issue"
]
for query in test_queries:
print(f"\nCustomer: {query}")
response = triage_agent.chat(query)
print(f"Response: {response}")
print("-" * 40)
+6
View File
@@ -0,0 +1,6 @@
from praisonaiagents import Agent, Agents, Tools
research_agent = Agent(name="Research", instructions="You are a research agent to search internet about AI 2024", tools=[Tools.internet_search])
summarise_agent = Agent(name="Summarise", instructions="You are a summarize agent to summarise in points")
agents = Agents(agents=[research_agent, summarise_agent])
agents.launch(path="/agents", port=3030)
+30
View File
@@ -0,0 +1,30 @@
from praisonaiagents import Agent
def get_stock_price(company_name: str) -> str:
"""
Get the stock price of a company
Args:
company_name (str): The name of the company
Returns:
str: The stock price of the company
"""
return f"The stock price of {company_name} is 100"
def multiply(a: int, b: int) -> int:
"""
Multiply two numbers
"""
return a * b
agent = Agent(
instructions="You are a helpful assistant. You can use the tools provided to you to help the user.",
llm="ollama/llama3.2",
tools=[get_stock_price, multiply]
)
result = agent.start("what is the stock price of Google? multiply the Google stock price with 2")
print(result)
+30
View File
@@ -0,0 +1,30 @@
from praisonaiagents import Agent
def get_stock_price(company_name: str) -> str:
"""
Get the stock price of a company
Args:
company_name (str): The name of the company
Returns:
str: The stock price of the company
"""
return f"The stock price of {company_name} is 100"
def multiply(a: int, b: int) -> int:
"""
Multiply two numbers
"""
return a * b
agent = Agent(
instructions="You are a helpful assistant. You can use the tools provided to you to help the user.",
llm="gpt-4o",
tools=[get_stock_price, multiply]
)
result = agent.start("what is the stock price of Google? multiply the Google stock price with 2")
print(result)
@@ -2,6 +2,30 @@
Praison AI Agents - A package for hierarchical AI agent task execution
"""
# Configure logging before any other imports
import os
import logging
from rich.logging import RichHandler
# Get log level from environment variable
LOGLEVEL = os.environ.get('LOGLEVEL', 'INFO').upper()
# Configure root logger
logging.basicConfig(
level=getattr(logging, LOGLEVEL, logging.INFO),
format="%(asctime)s %(filename)s:%(lineno)d %(levelname)s %(message)s",
datefmt="[%X]",
handlers=[RichHandler(rich_tracebacks=True)]
)
# Suppress specific noisy loggers
logging.getLogger("litellm").setLevel(logging.WARNING)
logging.getLogger("litellm.utils").setLevel(logging.WARNING)
logging.getLogger("markdown_it").setLevel(logging.WARNING)
logging.getLogger("rich.markdown").setLevel(logging.WARNING)
logging.getLogger("httpx").setLevel(logging.WARNING)
logging.getLogger("httpcore").setLevel(logging.WARNING)
from .agent.agent import Agent
from .agent.image_agent import ImageAgent
from .agents.agents import PraisonAIAgents
@@ -2,5 +2,6 @@
from .agent import Agent
from .image_agent import ImageAgent
from .handoff import Handoff, handoff, handoff_filters, RECOMMENDED_PROMPT_PREFIX, prompt_with_handoff_instructions
from .router_agent import RouterAgent
__all__ = ['Agent', 'ImageAgent', 'Handoff', 'handoff', 'handoff_filters', 'RECOMMENDED_PROMPT_PREFIX', 'prompt_with_handoff_instructions']
__all__ = ['Agent', 'ImageAgent', 'Handoff', 'handoff', 'handoff_filters', 'RECOMMENDED_PROMPT_PREFIX', 'prompt_with_handoff_instructions', 'RouterAgent']
@@ -444,6 +444,8 @@ class Agent:
self.embedder_config = embedder_config
self.knowledge = knowledge
self.use_system_prompt = use_system_prompt
# NOTE: chat_history is not thread-safe. If concurrent access is needed,
# consider using threading.Lock or other synchronization mechanisms
self.chat_history = []
self.markdown = markdown
self.stream = stream
@@ -711,8 +713,55 @@ Your Goal: {self.goal}
)
return current_response
def _build_system_prompt(self, tools=None):
"""Build the system prompt with tool information.
Args:
tools: Optional list of tools to use (defaults to self.tools)
Returns:
str: The system prompt or None if use_system_prompt is False
"""
if not self.use_system_prompt:
return None
system_prompt = f"""{self.backstory}\n
Your Role: {self.role}\n
Your Goal: {self.goal}"""
# Add tool usage instructions if tools are available
# Use provided tools or fall back to self.tools
tools_to_use = tools if tools is not None else self.tools
if tools_to_use:
tool_names = []
for tool in tools_to_use:
try:
if callable(tool) and hasattr(tool, '__name__'):
tool_names.append(tool.__name__)
elif isinstance(tool, dict) and isinstance(tool.get('function'), dict) and 'name' in tool['function']:
tool_names.append(tool['function']['name'])
elif isinstance(tool, str):
tool_names.append(tool)
elif hasattr(tool, "to_openai_tool"):
# Handle MCP tools
openai_tools = tool.to_openai_tool()
if isinstance(openai_tools, list):
for t in openai_tools:
if isinstance(t, dict) and 'function' in t and 'name' in t['function']:
tool_names.append(t['function']['name'])
elif isinstance(openai_tools, dict) and 'function' in openai_tools:
tool_names.append(openai_tools['function']['name'])
except (AttributeError, KeyError, TypeError) as e:
logging.warning(f"Could not extract tool name from {tool}: {e}")
continue
if tool_names:
system_prompt += f"\n\nYou have access to the following tools: {', '.join(tool_names)}. Use these tools when appropriate to help complete your tasks. Always use tools when they can help provide accurate information or perform actions."
return system_prompt
def _build_messages(self, prompt, temperature=0.2, output_json=None, output_pydantic=None):
def _build_messages(self, prompt, temperature=0.2, output_json=None, output_pydantic=None, tools=None):
"""Build messages list for chat completion.
Args:
@@ -720,17 +769,13 @@ Your Goal: {self.goal}
temperature: Temperature for the chat
output_json: Optional Pydantic model for JSON output
output_pydantic: Optional Pydantic model for JSON output (alias)
tools: Optional list of tools to use (defaults to self.tools)
Returns:
tuple: (messages list, original prompt)
"""
# Build system prompt if enabled
system_prompt = None
if self.use_system_prompt:
system_prompt = f"""{self.backstory}\n
Your Role: {self.role}\n
Your Goal: {self.goal}
"""
# Build system prompt using the helper method
system_prompt = self._build_system_prompt(tools)
# Use openai_client's build_messages method if available
if self._openai_client is not None:
@@ -1154,10 +1199,27 @@ Your Goal: {self.goal}
tool_param = [openai_tool]
logging.debug(f"Converted MCP tool: {tool_param}")
# Pass everything to LLM class
response_text = self.llm_instance.get_response(
# Store chat history length for potential rollback
chat_history_length = len(self.chat_history)
# Normalize prompt content for consistent chat history storage
normalized_content = prompt
if isinstance(prompt, list):
# Extract text from multimodal prompts
normalized_content = next((item["text"] for item in prompt if item.get("type") == "text"), "")
# Prevent duplicate messages
if not (self.chat_history and
self.chat_history[-1].get("role") == "user" and
self.chat_history[-1].get("content") == normalized_content):
# Add user message to chat history BEFORE LLM call so handoffs can access it
self.chat_history.append({"role": "user", "content": normalized_content})
try:
# Pass everything to LLM class
response_text = self.llm_instance.get_response(
prompt=prompt,
system_prompt=f"{self.backstory}\n\nYour Role: {self.role}\n\nYour Goal: {self.goal}" if self.use_system_prompt else None,
system_prompt=self._build_system_prompt(tools),
chat_history=self.chat_history,
temperature=temperature,
tools=tool_param,
@@ -1173,23 +1235,30 @@ Your Goal: {self.goal}
agent_role=self.role,
agent_tools=[t.__name__ if hasattr(t, '__name__') else str(t) for t in (tools if tools is not None else self.tools)],
execute_tool_fn=self.execute_tool, # Pass tool execution function
reasoning_steps=reasoning_steps
)
reasoning_steps=reasoning_steps,
stream=stream # Pass the stream parameter from chat method
)
self.chat_history.append({"role": "user", "content": prompt})
self.chat_history.append({"role": "assistant", "content": response_text})
self.chat_history.append({"role": "assistant", "content": response_text})
# Log completion time if in debug mode
if logging.getLogger().getEffectiveLevel() == logging.DEBUG:
total_time = time.time() - start_time
logging.debug(f"Agent.chat completed in {total_time:.2f} seconds")
# Log completion time if in debug mode
if logging.getLogger().getEffectiveLevel() == logging.DEBUG:
total_time = time.time() - start_time
logging.debug(f"Agent.chat completed in {total_time:.2f} seconds")
# Apply guardrail validation for custom LLM response
try:
validated_response = self._apply_guardrail_with_retry(response_text, prompt, temperature, tools)
return validated_response
# Apply guardrail validation for custom LLM response
try:
validated_response = self._apply_guardrail_with_retry(response_text, prompt, temperature, tools)
return validated_response
except Exception as e:
logging.error(f"Agent {self.name}: Guardrail validation failed for custom LLM: {e}")
# Rollback chat history on guardrail failure
self.chat_history = self.chat_history[:chat_history_length]
return None
except Exception as e:
logging.error(f"Agent {self.name}: Guardrail validation failed for custom LLM: {e}")
# Rollback chat history if LLM call fails
self.chat_history = self.chat_history[:chat_history_length]
display_error(f"Error in LLM chat: {e}")
return None
except Exception as e:
display_error(f"Error in LLM chat: {e}")
@@ -1197,167 +1266,209 @@ Your Goal: {self.goal}
else:
# Use the new _build_messages helper method
messages, original_prompt = self._build_messages(prompt, temperature, output_json, output_pydantic)
# Store chat history length for potential rollback
chat_history_length = len(self.chat_history)
# Normalize original_prompt for consistent chat history storage
normalized_content = original_prompt
if isinstance(original_prompt, list):
# Extract text from multimodal prompts
normalized_content = next((item["text"] for item in original_prompt if item.get("type") == "text"), "")
# Prevent duplicate messages
if not (self.chat_history and
self.chat_history[-1].get("role") == "user" and
self.chat_history[-1].get("content") == normalized_content):
# Add user message to chat history BEFORE LLM call so handoffs can access it
self.chat_history.append({"role": "user", "content": normalized_content})
final_response_text = None
reflection_count = 0
start_time = time.time()
while True:
try:
if self.verbose:
# Handle both string and list prompts for instruction display
display_text = prompt
if isinstance(prompt, list):
# Extract text content from multimodal prompt
display_text = next((item["text"] for item in prompt if item["type"] == "text"), "")
if display_text and str(display_text).strip():
# Pass agent information to display_instruction
agent_tools = [t.__name__ if hasattr(t, '__name__') else str(t) for t in self.tools]
display_instruction(
f"Agent {self.name} is processing prompt: {display_text}",
console=self.console,
agent_name=self.name,
agent_role=self.role,
agent_tools=agent_tools
)
response = self._chat_completion(messages, temperature=temperature, tools=tools if tools else None, reasoning_steps=reasoning_steps, stream=self.stream)
if not response:
return None
response_text = response.choices[0].message.content.strip()
# Handle output_json or output_pydantic if specified
if output_json or output_pydantic:
# Add to chat history and return raw response
self.chat_history.append({"role": "user", "content": original_prompt})
self.chat_history.append({"role": "assistant", "content": response_text})
# Wrap entire while loop in try-except for rollback on any failure
try:
while True:
try:
if self.verbose:
display_interaction(original_prompt, response_text, markdown=self.markdown,
generation_time=time.time() - start_time, console=self.console)
return response_text
# Handle both string and list prompts for instruction display
display_text = prompt
if isinstance(prompt, list):
# Extract text content from multimodal prompt
display_text = next((item["text"] for item in prompt if item["type"] == "text"), "")
if display_text and str(display_text).strip():
# Pass agent information to display_instruction
agent_tools = [t.__name__ if hasattr(t, '__name__') else str(t) for t in self.tools]
display_instruction(
f"Agent {self.name} is processing prompt: {display_text}",
console=self.console,
agent_name=self.name,
agent_role=self.role,
agent_tools=agent_tools
)
if not self.self_reflect:
self.chat_history.append({"role": "user", "content": original_prompt})
self.chat_history.append({"role": "assistant", "content": response_text})
if self.verbose:
logging.debug(f"Agent {self.name} final response: {response_text}")
display_interaction(original_prompt, response_text, markdown=self.markdown, generation_time=time.time() - start_time, console=self.console)
# Return only reasoning content if reasoning_steps is True
if reasoning_steps and hasattr(response.choices[0].message, 'reasoning_content'):
# Apply guardrail to reasoning content
try:
validated_reasoning = self._apply_guardrail_with_retry(response.choices[0].message.reasoning_content, original_prompt, temperature, tools)
return validated_reasoning
except Exception as e:
logging.error(f"Agent {self.name}: Guardrail validation failed for reasoning content: {e}")
return None
# Apply guardrail to regular response
try:
validated_response = self._apply_guardrail_with_retry(response_text, original_prompt, temperature, tools)
return validated_response
except Exception as e:
logging.error(f"Agent {self.name}: Guardrail validation failed: {e}")
response = self._chat_completion(messages, temperature=temperature, tools=tools if tools else None, reasoning_steps=reasoning_steps, stream=self.stream)
if not response:
# Rollback chat history on response failure
self.chat_history = self.chat_history[:chat_history_length]
return None
reflection_prompt = f"""
response_text = response.choices[0].message.content.strip()
# Handle output_json or output_pydantic if specified
if output_json or output_pydantic:
# Add to chat history and return raw response
# User message already added before LLM call via _build_messages
self.chat_history.append({"role": "assistant", "content": response_text})
# Only display interaction if not using custom LLM (to avoid double output) and verbose is True
if self.verbose and not self._using_custom_llm:
display_interaction(original_prompt, response_text, markdown=self.markdown,
generation_time=time.time() - start_time, console=self.console)
return response_text
if not self.self_reflect:
# User message already added before LLM call via _build_messages
self.chat_history.append({"role": "assistant", "content": response_text})
if self.verbose:
logging.debug(f"Agent {self.name} final response: {response_text}")
# Only display interaction if not using custom LLM (to avoid double output) and verbose is True
if self.verbose and not self._using_custom_llm:
display_interaction(original_prompt, response_text, markdown=self.markdown, generation_time=time.time() - start_time, console=self.console)
# Return only reasoning content if reasoning_steps is True
if reasoning_steps and hasattr(response.choices[0].message, 'reasoning_content'):
# Apply guardrail to reasoning content
try:
validated_reasoning = self._apply_guardrail_with_retry(response.choices[0].message.reasoning_content, original_prompt, temperature, tools)
return validated_reasoning
except Exception as e:
logging.error(f"Agent {self.name}: Guardrail validation failed for reasoning content: {e}")
# Rollback chat history on guardrail failure
self.chat_history = self.chat_history[:chat_history_length]
return None
# Apply guardrail to regular response
try:
validated_response = self._apply_guardrail_with_retry(response_text, original_prompt, temperature, tools)
return validated_response
except Exception as e:
logging.error(f"Agent {self.name}: Guardrail validation failed: {e}")
# Rollback chat history on guardrail failure
self.chat_history = self.chat_history[:chat_history_length]
return None
reflection_prompt = f"""
Reflect on your previous response: '{response_text}'.
{self.reflect_prompt if self.reflect_prompt else "Identify any flaws, improvements, or actions."}
Provide a "satisfactory" status ('yes' or 'no').
Output MUST be JSON with 'reflection' and 'satisfactory'.
"""
logging.debug(f"{self.name} reflection attempt {reflection_count+1}, sending prompt: {reflection_prompt}")
messages.append({"role": "user", "content": reflection_prompt})
"""
logging.debug(f"{self.name} reflection attempt {reflection_count+1}, sending prompt: {reflection_prompt}")
messages.append({"role": "user", "content": reflection_prompt})
try:
# Check if we're using a custom LLM (like Gemini)
if self._using_custom_llm or self._openai_client is None:
# For custom LLMs, we need to handle reflection differently
# Use non-streaming to get complete JSON response
reflection_response = self._chat_completion(messages, temperature=temperature, tools=None, stream=False, reasoning_steps=False)
if not reflection_response or not reflection_response.choices:
raise Exception("No response from reflection request")
reflection_text = reflection_response.choices[0].message.content.strip()
# Clean the JSON output
cleaned_json = self.clean_json_output(reflection_text)
# Parse the JSON manually
reflection_data = json.loads(cleaned_json)
# Create a reflection output object manually
class CustomReflectionOutput:
def __init__(self, data):
self.reflection = data.get('reflection', '')
self.satisfactory = data.get('satisfactory', 'no').lower()
reflection_output = CustomReflectionOutput(reflection_data)
else:
# Use OpenAI's structured output for OpenAI models
reflection_response = self._openai_client.sync_client.beta.chat.completions.parse(
model=self.reflect_llm if self.reflect_llm else self.llm,
messages=messages,
temperature=temperature,
response_format=ReflectionOutput
)
reflection_output = reflection_response.choices[0].message.parsed
try:
# Check if OpenAI client is available
if self._openai_client is None:
# For custom LLMs, self-reflection with structured output is not supported
if self.verbose:
display_self_reflection(f"Agent {self.name}: Self-reflection with structured output is not supported for custom LLM providers. Skipping reflection.", console=self.console)
# Return the original response without reflection
self.chat_history.append({"role": "user", "content": prompt})
self.chat_history.append({"role": "assistant", "content": response_text})
display_interaction(prompt, response_text, markdown=self.markdown, generation_time=time.time() - start_time, console=self.console)
return response_text
reflection_response = self._openai_client.sync_client.beta.chat.completions.parse(
model=self.reflect_llm if self.reflect_llm else self.llm,
messages=messages,
temperature=temperature,
response_format=ReflectionOutput
)
display_self_reflection(f"Agent {self.name} self reflection (using {self.reflect_llm if self.reflect_llm else self.llm}): reflection='{reflection_output.reflection}' satisfactory='{reflection_output.satisfactory}'", console=self.console)
reflection_output = reflection_response.choices[0].message.parsed
messages.append({"role": "assistant", "content": f"Self Reflection: {reflection_output.reflection} Satisfactory?: {reflection_output.satisfactory}"})
if self.verbose:
display_self_reflection(f"Agent {self.name} self reflection (using {self.reflect_llm if self.reflect_llm else self.llm}): reflection='{reflection_output.reflection}' satisfactory='{reflection_output.satisfactory}'", console=self.console)
# Only consider satisfactory after minimum reflections
if reflection_output.satisfactory == "yes" and reflection_count >= self.min_reflect - 1:
if self.verbose:
display_self_reflection("Agent marked the response as satisfactory after meeting minimum reflections", console=self.console)
# User message already added before LLM call via _build_messages
self.chat_history.append({"role": "assistant", "content": response_text})
# Only display interaction if not using custom LLM (to avoid double output) and verbose is True
if self.verbose and not self._using_custom_llm:
display_interaction(original_prompt, response_text, markdown=self.markdown, generation_time=time.time() - start_time, console=self.console)
# Apply guardrail validation after satisfactory reflection
try:
validated_response = self._apply_guardrail_with_retry(response_text, original_prompt, temperature, tools)
return validated_response
except Exception as e:
logging.error(f"Agent {self.name}: Guardrail validation failed after reflection: {e}")
# Rollback chat history on guardrail failure
self.chat_history = self.chat_history[:chat_history_length]
return None
messages.append({"role": "assistant", "content": f"Self Reflection: {reflection_output.reflection} Satisfactory?: {reflection_output.satisfactory}"})
# Check if we've hit max reflections
if reflection_count >= self.max_reflect - 1:
if self.verbose:
display_self_reflection("Maximum reflection count reached, returning current response", console=self.console)
# User message already added before LLM call via _build_messages
self.chat_history.append({"role": "assistant", "content": response_text})
# Only display interaction if not using custom LLM (to avoid double output) and verbose is True
if self.verbose and not self._using_custom_llm:
display_interaction(original_prompt, response_text, markdown=self.markdown, generation_time=time.time() - start_time, console=self.console)
# Apply guardrail validation after max reflections
try:
validated_response = self._apply_guardrail_with_retry(response_text, original_prompt, temperature, tools)
return validated_response
except Exception as e:
logging.error(f"Agent {self.name}: Guardrail validation failed after max reflections: {e}")
# Rollback chat history on guardrail failure
self.chat_history = self.chat_history[:chat_history_length]
return None
# If not satisfactory and not at max reflections, continue with regeneration
logging.debug(f"{self.name} reflection count {reflection_count + 1}, continuing reflection process")
messages.append({"role": "user", "content": "Now regenerate your response using the reflection you made"})
# For custom LLMs during reflection, always use non-streaming to ensure complete responses
use_stream = self.stream if not self._using_custom_llm else False
response = self._chat_completion(messages, temperature=temperature, tools=None, stream=use_stream)
response_text = response.choices[0].message.content.strip()
reflection_count += 1
continue # Continue the loop for more reflections
# Only consider satisfactory after minimum reflections
if reflection_output.satisfactory == "yes" and reflection_count >= self.min_reflect - 1:
if self.verbose:
display_self_reflection("Agent marked the response as satisfactory after meeting minimum reflections", console=self.console)
self.chat_history.append({"role": "user", "content": prompt})
self.chat_history.append({"role": "assistant", "content": response_text})
display_interaction(prompt, response_text, markdown=self.markdown, generation_time=time.time() - start_time, console=self.console)
# Apply guardrail validation after satisfactory reflection
try:
validated_response = self._apply_guardrail_with_retry(response_text, prompt, temperature, tools)
return validated_response
except Exception as e:
logging.error(f"Agent {self.name}: Guardrail validation failed after reflection: {e}")
return None
# Check if we've hit max reflections
if reflection_count >= self.max_reflect - 1:
if self.verbose:
display_self_reflection("Maximum reflection count reached, returning current response", console=self.console)
self.chat_history.append({"role": "user", "content": prompt})
self.chat_history.append({"role": "assistant", "content": response_text})
display_interaction(prompt, response_text, markdown=self.markdown, generation_time=time.time() - start_time, console=self.console)
# Apply guardrail validation after max reflections
try:
validated_response = self._apply_guardrail_with_retry(response_text, prompt, temperature, tools)
return validated_response
except Exception as e:
logging.error(f"Agent {self.name}: Guardrail validation failed after max reflections: {e}")
return None
logging.debug(f"{self.name} reflection count {reflection_count + 1}, continuing reflection process")
messages.append({"role": "user", "content": "Now regenerate your response using the reflection you made"})
response = self._chat_completion(messages, temperature=temperature, tools=None, stream=self.stream)
response_text = response.choices[0].message.content.strip()
reflection_count += 1
continue # Continue the loop for more reflections
except Exception as e:
display_error(f"Error in parsing self-reflection json {e}. Retrying", console=self.console)
logging.error("Reflection parsing failed.", exc_info=True)
messages.append({"role": "assistant", "content": f"Self Reflection failed."})
reflection_count += 1
continue # Continue even after error to try again
except Exception as e:
display_error(f"Error in chat: {e}", console=self.console)
return None
# Log completion time if in debug mode
if logging.getLogger().getEffectiveLevel() == logging.DEBUG:
total_time = time.time() - start_time
logging.debug(f"Agent.chat completed in {total_time:.2f} seconds")
# Apply guardrail validation before returning
try:
validated_response = self._apply_guardrail_with_retry(response_text, prompt, temperature, tools)
return validated_response
except Exception as e:
logging.error(f"Agent {self.name}: Guardrail validation failed: {e}")
if self.verbose:
display_error(f"Guardrail validation failed: {e}", console=self.console)
return None
except Exception as e:
display_error(f"Error in parsing self-reflection json {e}. Retrying", console=self.console)
logging.error("Reflection parsing failed.", exc_info=True)
messages.append({"role": "assistant", "content": "Self Reflection failed."})
reflection_count += 1
continue # Continue even after error to try again
except Exception:
# Catch any exception from the inner try block and re-raise to outer handler
raise
except Exception as e:
# Catch any exceptions that escape the while loop
display_error(f"Unexpected error in chat: {e}", console=self.console)
# Rollback chat history
self.chat_history = self.chat_history[:chat_history_length]
return None
def clean_json_output(self, output: str) -> str:
"""Clean and extract JSON from response text."""
@@ -1406,10 +1517,26 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
prompt = f"{prompt}\n\nKnowledge: {knowledge_content}"
if self._using_custom_llm:
# Store chat history length for potential rollback
chat_history_length = len(self.chat_history)
# Normalize prompt content for consistent chat history storage
normalized_content = prompt
if isinstance(prompt, list):
# Extract text from multimodal prompts
normalized_content = next((item["text"] for item in prompt if item.get("type") == "text"), "")
# Prevent duplicate messages
if not (self.chat_history and
self.chat_history[-1].get("role") == "user" and
self.chat_history[-1].get("content") == normalized_content):
# Add user message to chat history BEFORE LLM call so handoffs can access it
self.chat_history.append({"role": "user", "content": normalized_content})
try:
response_text = await self.llm_instance.get_response_async(
prompt=prompt,
system_prompt=f"{self.backstory}\n\nYour Role: {self.role}\n\nYour Goal: {self.goal}" if self.use_system_prompt else None,
system_prompt=self._build_system_prompt(tools),
chat_history=self.chat_history,
temperature=temperature,
tools=tools,
@@ -1423,19 +1550,29 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
console=self.console,
agent_name=self.name,
agent_role=self.role,
agent_tools=[t.__name__ if hasattr(t, '__name__') else str(t) for t in self.tools],
agent_tools=[t.__name__ if hasattr(t, '__name__') else str(t) for t in (tools if tools is not None else self.tools)],
execute_tool_fn=self.execute_tool_async,
reasoning_steps=reasoning_steps
)
self.chat_history.append({"role": "user", "content": prompt})
self.chat_history.append({"role": "assistant", "content": response_text})
if logging.getLogger().getEffectiveLevel() == logging.DEBUG:
total_time = time.time() - start_time
logging.debug(f"Agent.achat completed in {total_time:.2f} seconds")
return response_text
# Apply guardrail validation for custom LLM response
try:
validated_response = self._apply_guardrail_with_retry(response_text, prompt, temperature, tools)
return validated_response
except Exception as e:
logging.error(f"Agent {self.name}: Guardrail validation failed for custom LLM: {e}")
# Rollback chat history on guardrail failure
self.chat_history = self.chat_history[:chat_history_length]
return None
except Exception as e:
# Rollback chat history if LLM call fails
self.chat_history = self.chat_history[:chat_history_length]
display_error(f"Error in LLM chat: {e}")
if logging.getLogger().getEffectiveLevel() == logging.DEBUG:
total_time = time.time() - start_time
@@ -1445,6 +1582,22 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
# For OpenAI client
# Use the new _build_messages helper method
messages, original_prompt = self._build_messages(prompt, temperature, output_json, output_pydantic)
# Store chat history length for potential rollback
chat_history_length = len(self.chat_history)
# Normalize original_prompt for consistent chat history storage
normalized_content = original_prompt
if isinstance(original_prompt, list):
# Extract text from multimodal prompts
normalized_content = next((item["text"] for item in original_prompt if item.get("type") == "text"), "")
# Prevent duplicate messages
if not (self.chat_history and
self.chat_history[-1].get("role") == "user" and
self.chat_history[-1].get("content") == normalized_content):
# Add user message to chat history BEFORE LLM call so handoffs can access it
self.chat_history.append({"role": "user", "content": normalized_content})
reflection_count = 0
start_time = time.time()
@@ -1591,7 +1744,16 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
if logging.getLogger().getEffectiveLevel() == logging.DEBUG:
total_time = time.time() - start_time
logging.debug(f"Agent.achat completed in {total_time:.2f} seconds")
return response_text
# Apply guardrail validation for OpenAI client response
try:
validated_response = self._apply_guardrail_with_retry(response_text, original_prompt, temperature, tools)
return validated_response
except Exception as e:
logging.error(f"Agent {self.name}: Guardrail validation failed for OpenAI client: {e}")
# Rollback chat history on guardrail failure
self.chat_history = self.chat_history[:chat_history_length]
return None
except Exception as e:
display_error(f"Error in chat completion: {e}")
if logging.getLogger().getEffectiveLevel() == logging.DEBUG:
@@ -0,0 +1,334 @@
"""
Router Agent that can use different LLM models based on task requirements.
This module extends the base Agent class to support multiple models and intelligent
model selection based on task characteristics.
"""
import os
import logging
from typing import Dict, List, Optional, Any, Union
from .agent import Agent
from ..llm.model_router import ModelRouter
from ..llm import LLM
logger = logging.getLogger(__name__)
class RouterAgent(Agent):
"""
An enhanced agent that can dynamically select and use different LLM models
based on task requirements, optimizing for cost and performance.
"""
def __init__(
self,
models: Optional[Union[List[str], Dict[str, Any]]] = None,
model_router: Optional[ModelRouter] = None,
routing_strategy: str = "auto", # "auto", "manual", "cost-optimized", "performance-optimized"
primary_model: Optional[str] = None,
fallback_model: Optional[str] = None,
**kwargs
):
"""
Initialize a RouterAgent.
Args:
models: List of model names or dict mapping model names to configurations
model_router: Custom ModelRouter instance for model selection
routing_strategy: Strategy for model selection
primary_model: Primary model to use (overrides routing for simple tasks)
fallback_model: Fallback model if selected model fails
**kwargs: Additional arguments passed to parent Agent class
"""
# Initialize model router
self.model_router = model_router or ModelRouter()
self.routing_strategy = routing_strategy
self.fallback_model = fallback_model or os.getenv('OPENAI_MODEL_NAME', 'gpt-4o-mini')
# Process models configuration
self.available_models = self._process_models_config(models)
# Set primary model for parent class initialization
if primary_model:
kwargs['llm'] = primary_model
elif self.available_models:
# Use the most cost-effective model as default
cheapest_model = min(
self.available_models.keys(),
key=lambda m: self.model_router.get_model_info(m).cost_per_1k_tokens
if self.model_router.get_model_info(m) else float('inf')
)
kwargs['llm'] = cheapest_model
# Store the original llm parameter for later use
self._llm_config = kwargs.get('llm')
# Store api_key and base_url for LLM initialization
self._base_url = kwargs.get('base_url')
self._api_key = kwargs.get('api_key')
# Initialize parent Agent class
super().__init__(**kwargs)
# Initialize LLM instances for each model
self._llm_instances: Dict[str, LLM] = {}
self._initialize_llm_instances()
# Track usage statistics
self.model_usage_stats = {model: {'calls': 0, 'tokens': 0, 'cost': 0.0}
for model in self.available_models}
def _process_models_config(self, models: Optional[Union[List[str], Dict[str, Any]]]) -> Dict[str, Any]:
"""Process the models configuration into a standardized format."""
if not models:
# Use default models from router
return {m.name: {} for m in self.model_router.models}
if isinstance(models, list):
# Simple list of model names
return {model: {} for model in models}
# Already a dict with model configurations
return models
def _initialize_llm_instances(self):
"""Initialize LLM instances for each available model."""
base_url = self._base_url
api_key = self._api_key
for model_name, config in self.available_models.items():
try:
# Merge base configuration with model-specific config
llm_config = {
'model': model_name,
'base_url': config.get('base_url', base_url),
'api_key': config.get('api_key', api_key),
'verbose': self.verbose,
'markdown': self.markdown,
'stream': self.stream
}
# Add any model-specific parameters
llm_config.update(config)
# Create LLM instance
self._llm_instances[model_name] = LLM(**llm_config)
logger.debug(f"Initialized LLM instance for model: {model_name}")
except Exception as e:
logger.warning(f"Failed to initialize LLM for model {model_name}: {e}")
def _select_model_for_task(
self,
task_description: str,
tools: Optional[List[Any]] = None,
context_size: Optional[int] = None
) -> str:
"""
Select the most appropriate model for a given task.
Args:
task_description: Description of the task
tools: Tools required for the task
context_size: Estimated context size
Returns:
Selected model name
"""
if self.routing_strategy == "manual":
# Use the configured primary model from llm_model property
llm_model = self.llm_model
if hasattr(llm_model, 'model'):
# If it's an LLM instance, get the model name
return llm_model.model
elif isinstance(llm_model, str):
# If it's a string, use it directly
return llm_model
# Fallback if no model is configured
return self.fallback_model
# Determine required capabilities
required_capabilities = []
if tools:
required_capabilities.append("function-calling")
# Determine budget consciousness based on strategy
budget_conscious = self.routing_strategy in ["auto", "cost-optimized"]
# Get tool names for analysis
tool_names = []
if tools:
tool_names = [t.__name__ if hasattr(t, '__name__') else str(t) for t in tools]
# Use router to select model
selected_model = self.model_router.select_model(
task_description=task_description,
required_capabilities=required_capabilities,
tools_required=tool_names,
context_size=context_size,
budget_conscious=budget_conscious
)
# Ensure selected model is available
if selected_model not in self._llm_instances:
logger.warning(f"Selected model {selected_model} not available, using fallback")
return self.fallback_model
return selected_model
def _execute_with_model(
self,
model_name: str,
prompt: str,
context: Optional[str] = None,
tools: Optional[List[Any]] = None,
**kwargs
) -> str:
"""
Execute a task with a specific model.
Args:
model_name: Name of the model to use
prompt: The prompt to send to the model
context: Additional context
tools: Tools to make available
**kwargs: Additional arguments for the LLM
Returns:
Model response
"""
llm_instance = self._llm_instances.get(model_name)
if not llm_instance:
logger.error(f"Model {model_name} not initialized, using fallback")
llm_instance = self._llm_instances.get(self.fallback_model)
model_name = self.fallback_model
if not llm_instance:
raise ValueError("No LLM instance available for execution")
# Prepare the full prompt
full_prompt = prompt
if context:
full_prompt = f"{context}\n\n{prompt}"
try:
# Execute with the selected model
response = llm_instance.get_response(
prompt=full_prompt,
system_prompt=self._build_system_prompt(),
tools=tools,
verbose=self.verbose,
markdown=self.markdown,
stream=self.stream,
agent_name=self.name,
agent_role=self.role,
agent_tools=[t.__name__ if hasattr(t, '__name__') else str(t) for t in (tools or [])],
execute_tool_fn=self.execute_tool if tools else None,
**kwargs
)
# Update usage statistics
self.model_usage_stats[model_name]['calls'] += 1
# TODO: Implement token tracking when LLM.get_response() is updated to return token usage
# The LLM response currently returns only text, but litellm provides usage info in:
# response.get("usage") with prompt_tokens, completion_tokens, and total_tokens
# This would require modifying the LLM class to return both text and metadata
return response
except Exception as e:
logger.error(f"Error executing with model {model_name}: {e}")
# Try fallback model if different
if model_name != self.fallback_model and self.fallback_model in self._llm_instances:
logger.info(f"Attempting with fallback model: {self.fallback_model}")
return self._execute_with_model(
self.fallback_model, prompt, context, tools, **kwargs
)
raise
def execute(
self,
task_description: str,
context: Optional[str] = None,
tools: Optional[List[Any]] = None,
**kwargs
) -> str:
"""
Execute a task with automatic model selection.
This method overrides the parent Agent's execute method to add
intelligent model selection.
Args:
task_description: Description of the task to execute
context: Optional context for the task
tools: Optional tools to use
**kwargs: Additional arguments
Returns:
Task execution result
"""
# Estimate context size in tokens (rough estimate: ~4 chars per token)
# This is a simplified heuristic; actual tokenization varies by model
text_length = len(task_description) + (len(context) if context else 0)
context_size = text_length // 4 # Approximate token count
# Select the best model for this task
selected_model = self._select_model_for_task(
task_description=task_description,
tools=tools,
context_size=context_size
)
logger.info(f"RouterAgent '{self.name}' selected model: {selected_model} for task")
# Execute with the selected model
return self._execute_with_model(
model_name=selected_model,
prompt=task_description,
context=context,
tools=tools,
**kwargs
)
def get_usage_report(self) -> Dict[str, Any]:
"""
Get a report of model usage statistics.
Returns:
Dictionary containing usage statistics and cost estimates
"""
total_cost = 0.0
report = {
'agent_name': self.name,
'routing_strategy': self.routing_strategy,
'model_usage': {}
}
for model, stats in self.model_usage_stats.items():
model_info = self.model_router.get_model_info(model)
if model_info and stats['tokens'] > 0:
cost = self.model_router.estimate_cost(model, stats['tokens'])
stats['cost'] = cost
total_cost += cost
report['model_usage'][model] = stats
report['total_cost_estimate'] = total_cost
report['total_calls'] = sum(s['calls'] for s in self.model_usage_stats.values())
return report
def _build_system_prompt(self) -> str:
"""Build system prompt (inherited from parent but can be customized)."""
base_prompt = super()._build_system_prompt()
# Add multi-model context if needed
if self.routing_strategy == "auto":
base_prompt += "\n\nNote: You are part of a multi-model system. Focus on your specific task."
return base_prompt
@@ -480,24 +480,22 @@ Context:
)
if self.process == "workflow":
# Collect all tasks that should run in parallel
parallel_tasks = []
tasks_to_run = []
async for task_id in process.aworkflow():
if self.tasks[task_id].async_execution and self.tasks[task_id].is_start:
parallel_tasks.append(task_id)
elif parallel_tasks:
# Execute collected parallel tasks
await asyncio.gather(*[self.arun_task(t) for t in parallel_tasks])
parallel_tasks = []
# Run the current non-parallel task
if self.tasks[task_id].async_execution:
await self.arun_task(task_id)
else:
self.run_task(task_id)
# Execute any remaining parallel tasks
if parallel_tasks:
await asyncio.gather(*[self.arun_task(t) for t in parallel_tasks])
if self.tasks[task_id].async_execution:
tasks_to_run.append(self.arun_task(task_id))
else:
# If we encounter a sync task, we must wait for the previous async tasks to finish.
if tasks_to_run:
await asyncio.gather(*tasks_to_run)
tasks_to_run = []
# Run sync task in an executor to avoid blocking the event loop
loop = asyncio.get_event_loop()
await loop.run_in_executor(None, self.run_task, task_id)
if tasks_to_run:
await asyncio.gather(*tasks_to_run)
elif self.process == "sequential":
async for task_id in process.asequential():
@@ -13,7 +13,8 @@ import logging
import os
from pydantic import BaseModel, ConfigDict
from ..main import display_instruction, display_tool_call, display_interaction
from ..llm import get_openai_client
from ..llm import get_openai_client, LLM, OpenAIClient
import json
# Define Pydantic models for structured output
class TaskConfig(BaseModel):
@@ -108,6 +109,8 @@ class AutoAgents(PraisonAIAgents):
self.max_execution_time = max_execution_time
self.max_iter = max_iter
self.reflect_llm = reflect_llm
self.base_url = base_url
self.api_key = api_key
# Display initial instruction
if self.verbose:
@@ -133,7 +136,7 @@ class AutoAgents(PraisonAIAgents):
completion_checker=completion_checker,
max_retries=max_retries,
process=process,
manager_llm=manager_llm
manager_llm=manager_llm or self.llm
)
def _display_agents_and_tasks(self, agents: List[Agent], tasks: List[Task]):
@@ -211,9 +214,37 @@ Tools: {', '.join(agent_tools)}"""
return assigned_tools
def _validate_config(self, config: AutoAgentsConfig) -> tuple[bool, str]:
"""
Validate that the configuration has proper TaskConfig objects.
Returns:
Tuple of (is_valid, error_message)
"""
for agent_idx, agent in enumerate(config.agents):
if not hasattr(agent, 'tasks') or not agent.tasks:
return False, f"Agent '{agent.name}' has no tasks defined"
for task_idx, task in enumerate(agent.tasks):
# Check if task is a proper TaskConfig instance
if not isinstance(task, TaskConfig):
return False, f"Task at index {task_idx} for agent '{agent.name}' is not a proper TaskConfig object"
# Check required fields
if not task.name:
return False, f"Task at index {task_idx} for agent '{agent.name}' has no name"
if not task.description:
return False, f"Task at index {task_idx} for agent '{agent.name}' has no description"
if not task.expected_output:
return False, f"Task at index {task_idx} for agent '{agent.name}' has no expected_output"
if task.tools is None:
return False, f"Task at index {task_idx} for agent '{agent.name}' has no tools field"
return True, ""
def _generate_config(self) -> AutoAgentsConfig:
"""Generate the configuration for agents and tasks"""
prompt = f"""
"""Generate the configuration for agents and tasks with retry logic"""
base_prompt = f"""
Generate a configuration for AI agents to accomplish this task: "{self.instructions}"
The configuration should include:
@@ -234,40 +265,161 @@ Requirements:
4. The process type should match the task requirements
5. Generate maximum {self.max_agents} agents to handle this task efficiently
Return the configuration in a structured JSON format matching the AutoAgentsConfig schema.
Return the configuration in a structured JSON format matching this exact schema:
{{
"main_instruction": "Overall goal description",
"process_type": "sequential|workflow|hierarchical",
"agents": [
{{
"name": "Agent Name",
"role": "Agent Role",
"goal": "Agent Goal",
"backstory": "Agent Backstory",
"tools": ["tool1", "tool2"],
"tasks": [
{{
"name": "Task Name",
"description": "Detailed task description",
"expected_output": "What the task should produce",
"tools": ["tool1", "tool2"]
}}
]
}}
]
}}
IMPORTANT: Each task MUST be an object with name, description, expected_output, and tools fields, NOT a simple string.
"""
try:
# Get OpenAI client
max_retries = 3
last_response = None
last_error = None
for attempt in range(max_retries):
# Initialize variables for this attempt
use_openai_structured = False
client = None
# Prepare prompt for this attempt
if attempt > 0 and last_response and last_error:
# On retry, include the previous response and error
prompt = f"""{base_prompt}
PREVIOUS ATTEMPT FAILED!
Your previous response was:
```json
{last_response}
```
Error: {last_error}
REMEMBER: Tasks MUST be objects with the following structure:
{{
"name": "Task Name",
"description": "Task Description",
"expected_output": "Expected Output",
"tools": ["tool1", "tool2"]
}}
DO NOT use strings for tasks. Each task MUST be a complete object with all four fields."""
else:
prompt = base_prompt
try:
client = get_openai_client()
except ValueError as e:
# AutoAgents requires OpenAI for structured output generation
raise ValueError(
"AutoAgents requires OpenAI API for automatic agent generation. "
"Please set OPENAI_API_KEY environment variable or use PraisonAIAgents class directly "
"with manually configured agents for non-OpenAI providers."
) from e
# Check if we have OpenAI API and the model supports structured output
from ..llm import supports_structured_outputs
if self.llm and supports_structured_outputs(self.llm):
client = get_openai_client()
use_openai_structured = True
except:
# If OpenAI client is not available, we'll use the LLM class
pass
try:
if use_openai_structured and client:
# Use OpenAI's structured output for OpenAI models (backward compatibility)
config = client.parse_structured_output(
messages=[
{"role": "system", "content": "You are a helpful assistant designed to generate AI agent configurations."},
{"role": "user", "content": prompt}
],
response_format=AutoAgentsConfig,
model=self.llm
)
# Store the response for potential retry
last_response = json.dumps(config.model_dump(), indent=2)
else:
# Use LLM class for all other providers (Gemini, Anthropic, etc.)
llm_instance = LLM(
model=self.llm,
base_url=self.base_url,
api_key=self.api_key
)
response_text = llm_instance.get_response(
prompt=prompt,
system_prompt="You are a helpful assistant designed to generate AI agent configurations.",
output_pydantic=AutoAgentsConfig,
temperature=0.7,
stream=False,
verbose=False
)
# Store the raw response for potential retry
last_response = response_text
# Parse the JSON response
try:
# First try to parse as is
config_dict = json.loads(response_text)
config = AutoAgentsConfig(**config_dict)
except json.JSONDecodeError:
# If that fails, try to extract JSON from the response
# Handle cases where the model might wrap JSON in markdown blocks
cleaned_response = response_text.strip()
if cleaned_response.startswith("```json"):
cleaned_response = cleaned_response[7:]
if cleaned_response.startswith("```"):
cleaned_response = cleaned_response[3:]
if cleaned_response.endswith("```"):
cleaned_response = cleaned_response[:-3]
cleaned_response = cleaned_response.strip()
config_dict = json.loads(cleaned_response)
config = AutoAgentsConfig(**config_dict)
response = client.beta.chat.completions.parse(
model=self.llm,
response_format=AutoAgentsConfig,
messages=[
{"role": "system", "content": "You are a helpful assistant designed to generate AI agent configurations."},
{"role": "user", "content": prompt}
]
)
# Ensure we have exactly max_agents number of agents
if len(response.choices[0].message.parsed.agents) > self.max_agents:
response.choices[0].message.parsed.agents = response.choices[0].message.parsed.agents[:self.max_agents]
elif len(response.choices[0].message.parsed.agents) < self.max_agents:
logging.warning(f"Generated {len(response.choices[0].message.parsed.agents)} agents, expected {self.max_agents}")
return response.choices[0].message.parsed
except Exception as e:
logging.error(f"Error generating configuration: {e}")
raise
# Validate the configuration
is_valid, error_msg = self._validate_config(config)
if not is_valid:
last_error = error_msg
if attempt < max_retries - 1:
logging.warning(f"Configuration validation failed (attempt {attempt + 1}/{max_retries}): {error_msg}")
continue
else:
raise ValueError(f"Configuration validation failed after {max_retries} attempts: {error_msg}")
# Ensure we have exactly max_agents number of agents
if len(config.agents) > self.max_agents:
config.agents = config.agents[:self.max_agents]
elif len(config.agents) < self.max_agents:
logging.warning(f"Generated {len(config.agents)} agents, expected {self.max_agents}")
return config
except ValueError as e:
# Re-raise validation errors
raise
except Exception as e:
last_error = str(e)
if attempt < max_retries - 1:
logging.warning(f"Error generating configuration (attempt {attempt + 1}/{max_retries}): {e}")
continue
else:
logging.error(f"Error generating configuration after {max_retries} attempts: {e}")
raise
# This should never be reached due to the raise statements above
raise RuntimeError(f"Failed to generate valid configuration after {max_retries} attempts")
def _create_agents_and_tasks(self, config: AutoAgentsConfig) -> Tuple[List[Agent], List[Task]]:
"""Create agents and tasks from configuration"""
@@ -308,7 +460,9 @@ Return the configuration in a structured JSON format matching the AutoAgentsConf
max_rpm=self.max_rpm,
max_execution_time=self.max_execution_time,
max_iter=self.max_iter,
reflect_llm=self.reflect_llm
reflect_llm=self.reflect_llm,
base_url=self.base_url,
api_key=self.api_key
)
agents.append(agent)
@@ -346,4 +500,4 @@ Return the configuration in a structured JSON format matching the AutoAgentsConf
Creates tasks based on the instructions, then starts execution.
Returns the task status and results dictionary.
"""
return super().start()
return super().start()
@@ -67,9 +67,6 @@ class Knowledge:
# Disable OpenAI API request logging
logging.getLogger('openai').setLevel(logging.WARNING)
# Set root logger to warning to catch any uncategorized logs
logging.getLogger().setLevel(logging.WARNING)
@cached_property
def _deps(self):
@@ -0,0 +1,25 @@
# Ollama Handling in LLM.py
The `ollama_handled` flag is used in `llm.py` (not in `agent.py`) to track whether special Ollama model handling has been applied for tool results. Here's what I found:
## Where it's used in llm.py:
- **Line 863**: `ollama_handled = False` - initialized at the beginning of the tool execution loop
- **Line 864**: Special handling logic via `_handle_ollama_model()` method
- **Line 907**: `ollama_handled = True` - set when Ollama special handling is applied
- **Line 928 & 959**: Checks `if not ollama_handled` to avoid duplicate processing
## Why it's needed:
The flag prevents duplicate processing when Ollama models require special handling. Some Ollama models return only the tool call JSON without processing the results, requiring a follow-up prompt to get the final answer. The `ollama_handled` flag ensures this special handling happens only once.
## Does this contradict the requirement?
**No, it doesn't.** The issue states:
> "ollama/xxxx model is already handled via llm.py via litellm, so no special condition or handling required for ollama inside agent.py"
✅ **Current implementation is correct:**
- `agent.py` has NO ollama-specific conditions (only one documentation mention as an example)
- All ollama handling is contained within `llm.py` as intended
- The `ollama_handled` flag is an internal implementation detail within `llm.py`
The codebase already follows the desired pattern where ollama models are handled through the generic LLM interface via litellm without special conditions in agent.py.
@@ -5,19 +5,16 @@ import os
# Disable litellm telemetry before any imports
os.environ["LITELLM_TELEMETRY"] = "False"
# Suppress all relevant logs at module level
logging.getLogger("litellm").setLevel(logging.ERROR)
logging.getLogger("openai").setLevel(logging.ERROR)
logging.getLogger("httpx").setLevel(logging.ERROR)
logging.getLogger("httpcore").setLevel(logging.ERROR)
logging.getLogger("pydantic").setLevel(logging.ERROR)
# Suppress all relevant logs at module level - consistent with main __init__.py
logging.getLogger("litellm").setLevel(logging.WARNING)
logging.getLogger("openai").setLevel(logging.WARNING)
logging.getLogger("httpx").setLevel(logging.WARNING)
logging.getLogger("httpcore").setLevel(logging.WARNING)
logging.getLogger("pydantic").setLevel(logging.WARNING)
# Suppress pydantic warnings
warnings.filterwarnings("ignore", category=UserWarning, module="pydantic")
# Configure logging to suppress all INFO messages
logging.basicConfig(level=logging.WARNING)
# Import after suppressing warnings
from .llm import LLM, LLMContextLengthExceededException
from .openai_client import (
@@ -32,6 +29,16 @@ from .openai_client import (
ToolCall,
process_stream_chunks
)
from .model_capabilities import (
supports_structured_outputs,
supports_streaming_with_tools
)
from .model_router import (
ModelRouter,
ModelProfile,
TaskComplexity,
create_routing_agent
)
# Ensure telemetry is disabled after import as well
try:
@@ -52,5 +59,11 @@ __all__ = [
"CompletionUsage",
"ChatCompletion",
"ToolCall",
"process_stream_chunks"
"process_stream_chunks",
"supports_structured_outputs",
"supports_streaming_with_tools",
"ModelRouter",
"ModelProfile",
"TaskComplexity",
"create_routing_agent"
]
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,107 @@
"""
Model capabilities configuration for different LLM providers.
This module defines which models support specific features like structured outputs.
"""
# Models that support OpenAI-style structured outputs (response_format with Pydantic models)
MODELS_SUPPORTING_STRUCTURED_OUTPUTS = {
# OpenAI models
"gpt-4o",
"gpt-4o-mini",
"gpt-4-turbo",
"gpt-4-turbo-preview",
"gpt-4-turbo-2024-04-09",
"gpt-4-1106-preview",
"gpt-4-0125-preview",
"gpt-3.5-turbo",
"gpt-3.5-turbo-1106",
"gpt-3.5-turbo-0125",
# New/Future OpenAI models (as mentioned by user)
"codex-mini",
"o3-pro",
"gpt-4.5-preview",
"o3-mini",
"o1",
"o1-preview",
"o1-mini",
"gpt-4.1",
"gpt-4.1-nano",
"gpt-4.1-mini",
"o4-mini",
"o3",
# Gemini models that support structured outputs
"gemini-2.0-flash",
"gemini-2.0-flash-exp",
"gemini-1.5-pro",
"gemini-1.5-pro-latest",
"gemini-1.5-flash",
"gemini-1.5-flash-latest",
"gemini-1.5-flash-8b",
"gemini-1.5-flash-8b-latest",
}
# Models that explicitly DON'T support structured outputs
MODELS_NOT_SUPPORTING_STRUCTURED_OUTPUTS = {
# Audio preview models
"gpt-4o-audio-preview",
"gpt-4o-mini-audio-preview",
# Legacy o1 models (don't support system messages either)
"o1-preview-2024-09-12",
"o1-mini-2024-09-12",
}
def supports_structured_outputs(model_name: str) -> bool:
"""
Check if a model supports OpenAI-style structured outputs.
Args:
model_name: The name of the model to check
Returns:
bool: True if the model supports structured outputs, False otherwise
"""
if not model_name:
return False
# Strip provider prefixes (e.g., 'google/', 'openai/', etc.)
model_without_provider = model_name
for prefix in ['google/', 'openai/', 'anthropic/', 'gemini/', 'mistral/', 'deepseek/', 'groq/']:
if model_name.startswith(prefix):
model_without_provider = model_name[len(prefix):]
break
# First check if it's explicitly in the NOT supporting list
if model_without_provider in MODELS_NOT_SUPPORTING_STRUCTURED_OUTPUTS:
return False
# Then check if it's in the supporting list
if model_without_provider in MODELS_SUPPORTING_STRUCTURED_OUTPUTS:
return True
# For models with version suffixes, check the base model name
base_model = model_without_provider.split('-2024-')[0].split('-2025-')[0]
if base_model in MODELS_SUPPORTING_STRUCTURED_OUTPUTS:
return True
# Default to False for unknown models
return False
def supports_streaming_with_tools(model_name: str) -> bool:
"""
Check if a model supports streaming when tools are provided.
Most models that support structured outputs also support streaming with tools.
Args:
model_name: The name of the model to check
Returns:
bool: True if the model supports streaming with tools, False otherwise
"""
# For now, use the same logic as structured outputs
# In the future, this could be a separate list if needed
return supports_structured_outputs(model_name)
@@ -0,0 +1,348 @@
"""
Model Router for intelligent model selection based on task characteristics.
This module provides functionality to automatically select the most appropriate
LLM model/provider based on task complexity, cost considerations, and model capabilities.
"""
import os
import logging
from typing import Dict, List, Optional, Tuple
from dataclasses import dataclass
from enum import IntEnum
logger = logging.getLogger(__name__)
class TaskComplexity(IntEnum):
"""Enum for task complexity levels"""
SIMPLE = 1 # Basic queries, math, factual questions
MODERATE = 2 # Summarization, basic analysis
COMPLEX = 3 # Code generation, deep reasoning
VERY_COMPLEX = 4 # Multi-step reasoning, complex analysis
@dataclass
class ModelProfile:
"""Profile for an LLM model with its characteristics"""
name: str
provider: str
complexity_range: Tuple[TaskComplexity, TaskComplexity]
cost_per_1k_tokens: float # Average of input/output costs
strengths: List[str]
capabilities: List[str]
context_window: int
supports_tools: bool = True
supports_streaming: bool = True
class ModelRouter:
"""
Intelligent model router that selects the best model based on task requirements.
This router implements a strategy pattern for model selection, considering:
- Task complexity
- Cost optimization
- Model capabilities
- Specific strengths for different task types
"""
# Default model profiles - can be customized via configuration
DEFAULT_MODELS = [
# Lightweight/cheap models for simple tasks
ModelProfile(
name="gpt-4o-mini",
provider="openai",
complexity_range=(TaskComplexity.SIMPLE, TaskComplexity.MODERATE),
cost_per_1k_tokens=0.00075, # Average of $0.00015 input, $0.0006 output
strengths=["speed", "cost-effective", "basic-reasoning"],
capabilities=["text", "function-calling"],
context_window=128000
),
ModelProfile(
name="gemini/gemini-1.5-flash",
provider="google",
complexity_range=(TaskComplexity.SIMPLE, TaskComplexity.MODERATE),
cost_per_1k_tokens=0.000125, # Very cost-effective
strengths=["speed", "cost-effective", "multimodal"],
capabilities=["text", "vision", "function-calling"],
context_window=1048576
),
ModelProfile(
name="claude-3-haiku-20240307",
provider="anthropic",
complexity_range=(TaskComplexity.SIMPLE, TaskComplexity.MODERATE),
cost_per_1k_tokens=0.0008, # Average of $0.00025 input, $0.00125 output
strengths=["speed", "instruction-following"],
capabilities=["text", "function-calling"],
context_window=200000
),
# Mid-tier models for moderate complexity
ModelProfile(
name="gpt-4o",
provider="openai",
complexity_range=(TaskComplexity.MODERATE, TaskComplexity.COMPLEX),
cost_per_1k_tokens=0.0075, # Average of $0.0025 input, $0.01 output
strengths=["reasoning", "code-generation", "general-purpose"],
capabilities=["text", "vision", "function-calling"],
context_window=128000
),
ModelProfile(
name="claude-3-5-sonnet-20241022",
provider="anthropic",
complexity_range=(TaskComplexity.MODERATE, TaskComplexity.VERY_COMPLEX),
cost_per_1k_tokens=0.009, # Average of $0.003 input, $0.015 output
strengths=["reasoning", "code-generation", "analysis", "writing"],
capabilities=["text", "vision", "function-calling"],
context_window=200000
),
# High-end models for complex tasks
ModelProfile(
name="gemini/gemini-1.5-pro",
provider="google",
complexity_range=(TaskComplexity.COMPLEX, TaskComplexity.VERY_COMPLEX),
cost_per_1k_tokens=0.00625, # Average of $0.00125 input, $0.005 output
strengths=["reasoning", "long-context", "multimodal"],
capabilities=["text", "vision", "function-calling"],
context_window=2097152 # 2M context
),
ModelProfile(
name="claude-3-opus-20240229",
provider="anthropic",
complexity_range=(TaskComplexity.COMPLEX, TaskComplexity.VERY_COMPLEX),
cost_per_1k_tokens=0.045, # Average of $0.015 input, $0.075 output
strengths=["deep-reasoning", "complex-analysis", "creative-writing"],
capabilities=["text", "vision", "function-calling"],
context_window=200000
),
ModelProfile(
name="deepseek-chat",
provider="deepseek",
complexity_range=(TaskComplexity.COMPLEX, TaskComplexity.VERY_COMPLEX),
cost_per_1k_tokens=0.0014, # Very cost-effective for capability
strengths=["reasoning", "code-generation", "mathematics"],
capabilities=["text", "function-calling"],
context_window=128000
),
]
def __init__(
self,
models: Optional[List[ModelProfile]] = None,
default_model: Optional[str] = None,
cost_threshold: Optional[float] = None,
preferred_providers: Optional[List[str]] = None
):
"""
Initialize the ModelRouter.
Args:
models: Custom list of model profiles to use
default_model: Default model to use if no suitable model found
cost_threshold: Maximum cost per 1k tokens to consider
preferred_providers: List of preferred providers in order
"""
self.models = models or self.DEFAULT_MODELS
self.default_model = default_model or os.getenv('OPENAI_MODEL_NAME', 'gpt-4o')
self.cost_threshold = cost_threshold
self.preferred_providers = preferred_providers or []
# Build lookup indices for efficient access
self._model_by_name = {m.name: m for m in self.models}
self._models_by_complexity = self._build_complexity_index()
def _build_complexity_index(self) -> Dict[TaskComplexity, List[ModelProfile]]:
"""Build an index of models by complexity level"""
index = {level: [] for level in TaskComplexity}
for model in self.models:
min_complexity, max_complexity = model.complexity_range
for level in TaskComplexity:
if min_complexity.value <= level.value <= max_complexity.value:
index[level].append(model)
return index
def analyze_task_complexity(
self,
task_description: str,
tools_required: Optional[List[str]] = None,
context_size: Optional[int] = None
) -> TaskComplexity:
"""
Analyze task description to determine complexity level.
This is a simple heuristic-based approach. In production, this could be
replaced with a more sophisticated ML-based classifier.
"""
description_lower = task_description.lower()
# Keywords indicating different complexity levels
simple_keywords = [
"calculate", "compute", "what is", "define", "list", "count",
"simple", "basic", "check", "verify", "yes or no", "true or false"
]
moderate_keywords = [
"summarize", "explain", "compare", "describe", "analyze briefly",
"find", "search", "extract", "classify", "categorize"
]
complex_keywords = [
"implement", "code", "develop", "design", "create algorithm",
"optimize", "debug", "refactor", "architect", "solve"
]
very_complex_keywords = [
"multi-step", "comprehensive analysis", "deep dive", "research",
"strategic", "framework", "system design", "proof", "theorem"
]
# Check for keyword matches
if any(keyword in description_lower for keyword in very_complex_keywords):
return TaskComplexity.VERY_COMPLEX
elif any(keyword in description_lower for keyword in complex_keywords):
return TaskComplexity.COMPLEX
elif any(keyword in description_lower for keyword in moderate_keywords):
return TaskComplexity.MODERATE
elif any(keyword in description_lower for keyword in simple_keywords):
return TaskComplexity.SIMPLE
# Consider tool requirements
if tools_required and len(tools_required) > 3:
return TaskComplexity.COMPLEX
# Consider context size requirements
if context_size and context_size > 50000:
return TaskComplexity.COMPLEX
# Default to moderate
return TaskComplexity.MODERATE
def select_model(
self,
task_description: str,
required_capabilities: Optional[List[str]] = None,
tools_required: Optional[List[str]] = None,
context_size: Optional[int] = None,
budget_conscious: bool = True
) -> str:
"""
Select the most appropriate model for a given task.
Args:
task_description: Description of the task to perform
required_capabilities: List of required capabilities (e.g., ["vision", "function-calling"])
tools_required: List of tools that will be used
context_size: Estimated context size needed
budget_conscious: Whether to optimize for cost
Returns:
Model name string to use
"""
# Analyze task complexity
complexity = self.analyze_task_complexity(task_description, tools_required, context_size)
# Get candidate models for this complexity level
candidates = self._models_by_complexity.get(complexity, [])
if not candidates:
logger.warning(f"No models found for complexity {complexity}, using default")
return self.default_model
# Filter by required capabilities
if required_capabilities:
candidates = [
m for m in candidates
if all(cap in m.capabilities for cap in required_capabilities)
]
# Filter by tool support if needed
if tools_required:
candidates = [m for m in candidates if m.supports_tools]
# Filter by context window if specified
if context_size:
candidates = [m for m in candidates if m.context_window >= context_size]
# Filter by cost threshold if specified
if self.cost_threshold:
candidates = [m for m in candidates if m.cost_per_1k_tokens <= self.cost_threshold]
if not candidates:
logger.warning("No models meet all criteria, using default")
return self.default_model
# Sort by selection criteria
if budget_conscious:
# Sort by cost (ascending)
candidates.sort(key=lambda m: m.cost_per_1k_tokens)
else:
# Sort by capability (descending complexity)
candidates.sort(key=lambda m: m.complexity_range[1].value, reverse=True)
# Apply provider preferences
if self.preferred_providers:
for provider in self.preferred_providers:
for model in candidates:
if model.provider == provider:
logger.info(f"Selected model: {model.name} (complexity: {complexity}, cost: ${model.cost_per_1k_tokens}/1k tokens)")
return model.name
# Return the best candidate
selected = candidates[0]
logger.info(f"Selected model: {selected.name} (complexity: {complexity}, cost: ${selected.cost_per_1k_tokens}/1k tokens)")
return selected.name
def get_model_info(self, model_name: str) -> Optional[ModelProfile]:
"""Get profile information for a specific model"""
return self._model_by_name.get(model_name)
def estimate_cost(self, model_name: str, estimated_tokens: int) -> float:
"""Estimate the cost for a given model and token count"""
model = self._model_by_name.get(model_name)
if not model:
return 0.0
return (model.cost_per_1k_tokens * estimated_tokens) / 1000
def create_routing_agent(
models: Optional[List[str]] = None,
router: Optional[ModelRouter] = None,
**agent_kwargs
) -> 'Agent':
"""
Create a specialized routing agent that can select models dynamically.
Args:
models: List of model names to route between
router: Custom ModelRouter instance
**agent_kwargs: Additional arguments to pass to Agent constructor
Returns:
Agent configured for model routing
"""
from ..agent import Agent
if not router:
router = ModelRouter()
routing_agent = Agent(
name=agent_kwargs.pop('name', 'ModelRouter'),
role=agent_kwargs.pop('role', 'Intelligent Model Router'),
goal=agent_kwargs.pop('goal', 'Select the most appropriate AI model based on task requirements'),
backstory=agent_kwargs.pop('backstory',
'I analyze tasks and route them to the most suitable AI model, '
'optimizing for performance, cost, and capability requirements.'
),
**agent_kwargs
)
# TODO: Consider creating a proper RoutingAgent subclass instead of setting private attributes
# For now, store the router on the agent for use in execution
routing_agent._model_router = router
routing_agent._available_models = models or [m.name for m in router.models]
return routing_agent
@@ -890,45 +890,8 @@ class OpenAIClient:
"content": results_str
})
# Check if we should continue (for tools like sequential thinking)
should_continue = False
for tool_call in tool_calls:
# Handle both ToolCall dataclass and OpenAI object
if isinstance(tool_call, ToolCall):
function_name = tool_call.function["name"]
arguments = json.loads(tool_call.function["arguments"])
else:
function_name = tool_call.function.name
arguments = json.loads(tool_call.function.arguments)
# For sequential thinking tool, check if nextThoughtNeeded is True
if function_name == "sequentialthinking" and arguments.get("nextThoughtNeeded", False):
should_continue = True
break
if not should_continue:
# Get final response after tool calls
if stream:
final_response = self.process_stream_response(
messages=messages,
model=model,
temperature=temperature,
tools=formatted_tools,
start_time=start_time,
console=console,
display_fn=display_fn,
reasoning_steps=reasoning_steps,
**kwargs
)
else:
final_response = self.create_completion(
messages=messages,
model=model,
temperature=temperature,
stream=False,
**kwargs
)
break
# Continue the loop to allow more tool calls
# The model will see tool results and can make additional tool calls
iteration_count += 1
else:
@@ -1067,45 +1030,8 @@ class OpenAIClient:
"content": results_str
})
# Check if we should continue (for tools like sequential thinking)
should_continue = False
for tool_call in tool_calls:
# Handle both ToolCall dataclass and OpenAI object
if isinstance(tool_call, ToolCall):
function_name = tool_call.function["name"]
arguments = json.loads(tool_call.function["arguments"])
else:
function_name = tool_call.function.name
arguments = json.loads(tool_call.function.arguments)
# For sequential thinking tool, check if nextThoughtNeeded is True
if function_name == "sequentialthinking" and arguments.get("nextThoughtNeeded", False):
should_continue = True
break
if not should_continue:
# Get final response after tool calls
if stream:
final_response = await self.process_stream_response_async(
messages=messages,
model=model,
temperature=temperature,
tools=formatted_tools,
start_time=start_time,
console=console,
display_fn=display_fn,
reasoning_steps=reasoning_steps,
**kwargs
)
else:
final_response = await self.acreate_completion(
messages=messages,
model=model,
temperature=temperature,
stream=False,
**kwargs
)
break
# Continue the loop to allow more tool calls
# The model will see tool results and can make additional tool calls
iteration_count += 1
else:
+1 -18
View File
@@ -9,32 +9,15 @@ from rich.console import Console
from rich.panel import Panel
from rich.text import Text
from rich.markdown import Markdown
from rich.logging import RichHandler
from rich.live import Live
import asyncio
# # Configure root logger
# logging.basicConfig(level=logging.WARNING)
# Suppress litellm logs
# Logging is already configured in __init__.py, just clean up handlers for litellm
logging.getLogger("litellm").handlers = []
logging.getLogger("litellm.utils").handlers = []
logging.getLogger("litellm").propagate = False
logging.getLogger("litellm.utils").propagate = False
LOGLEVEL = os.environ.get('LOGLEVEL', 'INFO').upper()
logging.basicConfig(
level=getattr(logging, LOGLEVEL, logging.INFO),
format="%(asctime)s %(filename)s:%(lineno)d %(levelname)s %(message)s",
datefmt="[%X]",
handlers=[RichHandler(rich_tracebacks=True)]
)
# Add these lines to suppress markdown parser debug logs
logging.getLogger('markdown_it').setLevel(logging.WARNING)
logging.getLogger('rich.markdown').setLevel(logging.WARNING)
# Global list to store error logs
error_logs = []
@@ -187,15 +187,45 @@ class MCP:
self.timeout = timeout
self.debug = debug
# Check if this is an SSE URL
# Check if this is an HTTP URL
if isinstance(command_or_string, str) and re.match(r'^https?://', command_or_string):
# Import the SSE client implementation
from .mcp_sse import SSEMCPClient
self.sse_client = SSEMCPClient(command_or_string, debug=debug, timeout=timeout)
self._tools = list(self.sse_client.tools)
self.is_sse = True
self.is_npx = False
return
# Determine transport type based on URL or kwargs
if command_or_string.endswith('/sse') and 'transport_type' not in kwargs:
# Legacy SSE URL - use SSE transport for backward compatibility
from .mcp_sse import SSEMCPClient
self.sse_client = SSEMCPClient(command_or_string, debug=debug, timeout=timeout)
self._tools = list(self.sse_client.tools)
self.is_sse = True
self.is_http_stream = False
self.is_npx = False
return
else:
# Use HTTP Stream transport for all other HTTP URLs
from .mcp_http_stream import HTTPStreamMCPClient
# Extract transport options from kwargs
transport_options = {}
if 'responseMode' in kwargs:
transport_options['responseMode'] = kwargs.pop('responseMode')
if 'headers' in kwargs:
transport_options['headers'] = kwargs.pop('headers')
if 'cors' in kwargs:
transport_options['cors'] = kwargs.pop('cors')
if 'session' in kwargs:
transport_options['session'] = kwargs.pop('session')
if 'resumability' in kwargs:
transport_options['resumability'] = kwargs.pop('resumability')
self.http_stream_client = HTTPStreamMCPClient(
command_or_string,
debug=debug,
timeout=timeout,
options=transport_options
)
self._tools = list(self.http_stream_client.tools)
self.is_sse = False
self.is_http_stream = True
self.is_npx = False
return
# Handle the single string format for stdio client
if isinstance(command_or_string, str) and args is None:
@@ -219,6 +249,7 @@ class MCP:
# Set up stdio client
self.is_sse = False
self.is_http_stream = False
# Ensure UTF-8 encoding in environment for Docker compatibility
env = kwargs.get('env', {})
@@ -275,6 +306,9 @@ class MCP:
"""
if self.is_sse:
return list(self.sse_client.tools)
if self.is_http_stream:
return list(self.http_stream_client.tools)
tool_functions = []
@@ -448,6 +482,10 @@ class MCP:
if self.is_sse and hasattr(self, 'sse_client') and self.sse_client.tools:
# Return all tools from SSE client
return self.sse_client.to_openai_tools()
if self.is_http_stream and hasattr(self, 'http_stream_client') and self.http_stream_client.tools:
# Return all tools from HTTP Stream client
return self.http_stream_client.to_openai_tools()
# For simplicity, we'll convert the first tool only if multiple exist
# More complex implementations could handle multiple tools
@@ -0,0 +1,466 @@
"""
HTTP Stream client implementation for MCP (Model Context Protocol).
This module provides the necessary classes and functions to connect to an MCP server
over HTTP Stream transport, implementing the Streamable HTTP transport protocol.
"""
import asyncio
import logging
import threading
import inspect
import json
import time
import uuid
from typing import List, Dict, Any, Optional, Callable, Iterable, Union
from urllib.parse import urlparse, urljoin
from mcp import ClientSession
try:
import aiohttp
except ImportError:
raise ImportError("aiohttp is required for HTTP Stream transport. Install with: pip install praisonaiagents[mcp]")
logger = logging.getLogger("mcp-http-stream")
# Global event loop for async operations
_event_loop = None
def get_event_loop():
"""Get or create a global event loop."""
global _event_loop
if _event_loop is None or _event_loop.is_closed():
_event_loop = asyncio.new_event_loop()
asyncio.set_event_loop(_event_loop)
return _event_loop
class HTTPStreamMCPTool:
"""A wrapper for an MCP tool that can be used with praisonaiagents."""
def __init__(self, name: str, description: str, session: ClientSession, input_schema: Optional[Dict[str, Any]] = None, timeout: int = 60):
self.name = name
self.__name__ = name # Required for Agent to recognize it as a tool
self.__qualname__ = name # Required for Agent to recognize it as a tool
self.__doc__ = description # Required for Agent to recognize it as a tool
self.description = description
self.session = session
self.input_schema = input_schema or {}
self.timeout = timeout
# Create a signature based on input schema
params = []
if input_schema and 'properties' in input_schema:
for param_name, prop_schema in input_schema['properties'].items():
# Determine type annotation based on schema
prop_type = prop_schema.get('type', 'string') if isinstance(prop_schema, dict) else 'string'
if prop_type == 'string':
annotation = str
elif prop_type == 'integer':
annotation = int
elif prop_type == 'number':
annotation = float
elif prop_type == 'boolean':
annotation = bool
elif prop_type == 'array':
annotation = list
elif prop_type == 'object':
annotation = dict
else:
annotation = Any
params.append(
inspect.Parameter(
name=param_name,
kind=inspect.Parameter.POSITIONAL_OR_KEYWORD,
default=inspect.Parameter.empty if param_name in input_schema.get('required', []) else None,
annotation=annotation
)
)
self.__signature__ = inspect.Signature(params)
def __call__(self, **kwargs):
"""Synchronous wrapper for the async call."""
logger.debug(f"Tool {self.name} called with args: {kwargs}")
# Use the global event loop
loop = get_event_loop()
# Run the async call in the event loop
future = asyncio.run_coroutine_threadsafe(self._async_call(**kwargs), loop)
try:
# Wait for the result with a timeout
return future.result(timeout=self.timeout)
except Exception as e:
logger.error(f"Error calling tool {self.name}: {e}")
return f"Error: {str(e)}"
async def _async_call(self, **kwargs):
"""Call the tool with the provided arguments."""
logger.debug(f"Async calling tool {self.name} with args: {kwargs}")
try:
result = await self.session.call_tool(self.name, kwargs)
# Extract text from result
if hasattr(result, 'content') and result.content:
if hasattr(result.content[0], 'text'):
return result.content[0].text
return str(result.content[0])
return str(result)
except Exception as e:
logger.error(f"Error in _async_call for {self.name}: {e}")
raise
def _fix_array_schemas(self, schema):
"""
Fix array schemas by adding missing 'items' attribute required by OpenAI.
This ensures compatibility with OpenAI's function calling format which
requires array types to specify the type of items they contain.
Args:
schema: The schema dictionary to fix
Returns:
dict: The fixed schema
"""
if not isinstance(schema, dict):
return schema
# Create a copy to avoid modifying the original
fixed_schema = schema.copy()
# Fix array types at the current level
if fixed_schema.get("type") == "array" and "items" not in fixed_schema:
# Add a default items schema for arrays without it
fixed_schema["items"] = {"type": "string"}
# Recursively fix nested schemas
if "properties" in fixed_schema:
fixed_properties = {}
for prop_name, prop_schema in fixed_schema["properties"].items():
fixed_properties[prop_name] = self._fix_array_schemas(prop_schema)
fixed_schema["properties"] = fixed_properties
# Fix items schema if it exists
if "items" in fixed_schema:
fixed_schema["items"] = self._fix_array_schemas(fixed_schema["items"])
return fixed_schema
def to_openai_tool(self):
"""Convert the tool to OpenAI format."""
# Fix array schemas to include 'items' attribute
fixed_schema = self._fix_array_schemas(self.input_schema)
return {
"type": "function",
"function": {
"name": self.name,
"description": self.description,
"parameters": fixed_schema
}
}
class HTTPStreamTransport:
"""
HTTP Stream Transport implementation for MCP.
This transport provides a single endpoint for all MCP communication,
supporting both batch (JSON) and streaming (SSE) response modes.
"""
def __init__(self, base_url: str, session_id: Optional[str] = None, options: Optional[Dict[str, Any]] = None):
self.base_url = base_url
self.session_id = session_id
self.options = options or {}
self.response_mode = self.options.get('responseMode', 'batch')
self.headers = {
'Content-Type': 'application/json',
'Accept': 'application/json, text/event-stream'
}
if session_id:
self.headers['Mcp-Session-Id'] = session_id
# Add custom headers if provided
if 'headers' in self.options:
self.headers.update(self.options['headers'])
self._session = None
self._sse_task = None
self._message_queue = asyncio.Queue()
self._pending_requests = {}
self._closing = False
async def __aenter__(self):
self._session = aiohttp.ClientSession()
return self
async def __aexit__(self, exc_type, exc_val, exc_tb):
# Set closing flag to stop listener gracefully
self._closing = True
if self._sse_task:
self._sse_task.cancel()
try:
await self._sse_task
except asyncio.CancelledError:
pass
if self._session:
await self._session.close()
async def send_request(self, request: Dict[str, Any]) -> Union[Dict[str, Any], None]:
"""Send a request to the HTTP Stream endpoint."""
if not self._session:
raise RuntimeError("Transport not initialized. Use async context manager.")
try:
async with self._session.post(self.base_url, json=request, headers=self.headers) as response:
# Update session ID if provided in response
if 'Mcp-Session-Id' in response.headers:
self.session_id = response.headers['Mcp-Session-Id']
self.headers['Mcp-Session-Id'] = self.session_id
# Handle different response types
content_type = response.headers.get('Content-Type', '')
if 'text/event-stream' in content_type:
# Stream mode - process SSE events
return await self._process_sse_response(response)
else:
# Batch mode - return JSON response
return await response.json()
except Exception as e:
logger.error(f"Error sending request: {e}")
raise
async def _process_sse_response(self, response):
"""Process SSE response stream."""
buffer = ""
async for chunk in response.content:
buffer += chunk.decode('utf-8')
# Process complete SSE events
while "\n\n" in buffer:
event, buffer = buffer.split("\n\n", 1)
lines = event.strip().split("\n")
# Parse SSE event
data = None
for line in lines:
if line.startswith("data: "):
data = line[6:] # Remove "data: " prefix
if data:
try:
message = json.loads(data)
# Process the message
if 'id' in message and message['id'] in self._pending_requests:
# This is a response to a pending request
self._pending_requests[message['id']].set_result(message)
else:
# This is a server-initiated message
await self._message_queue.put(message)
except json.JSONDecodeError:
logger.error(f"Failed to parse SSE event: {data}")
async def start_sse_listener(self):
"""Start listening for SSE events from the server."""
if self._sse_task is None or self._sse_task.done():
self._sse_task = asyncio.create_task(self._sse_listener())
async def _sse_listener(self):
"""Background task to listen for SSE events."""
while True:
try:
# Check if we should stop
if hasattr(self, '_closing') and self._closing:
break
url = self.base_url
if self.session_id:
# Add session as query parameter for SSE connection
url = f"{url}?session={self.session_id}"
headers = {
'Accept': 'text/event-stream',
'Cache-Control': 'no-cache'
}
if self.session_id:
headers['Mcp-Session-Id'] = self.session_id
async with self._session.get(url, headers=headers) as response:
buffer = ""
async for chunk in response.content:
# Check if we should stop
if hasattr(self, '_closing') and self._closing:
break
buffer += chunk.decode('utf-8')
# Process complete SSE events
while "\n\n" in buffer:
event, buffer = buffer.split("\n\n", 1)
lines = event.strip().split("\n")
# Parse SSE event
data = None
for line in lines:
if line.startswith("data: "):
data = line[6:] # Remove "data: " prefix
if data:
try:
message = json.loads(data)
await self._message_queue.put(message)
except json.JSONDecodeError:
logger.error(f"Failed to parse SSE event: {data}")
except asyncio.CancelledError:
# Proper shutdown
break
except Exception as e:
if not (hasattr(self, '_closing') and self._closing):
logger.error(f"SSE listener error: {e}")
await asyncio.sleep(1) # Reconnect after 1 second
else:
break
def read_stream(self):
"""Create a read stream for the ClientSession."""
async def _read():
while True:
message = await self._message_queue.get()
yield message
return _read()
def write_stream(self):
"""Create a write stream for the ClientSession."""
async def _write(message):
if hasattr(message, 'to_dict'):
message = message.to_dict()
response = await self.send_request(message)
return response
return _write
class HTTPStreamMCPClient:
"""A client for connecting to an MCP server over HTTP Stream transport."""
def __init__(self, server_url: str, debug: bool = False, timeout: int = 60, options: Optional[Dict[str, Any]] = None):
"""
Initialize an HTTP Stream MCP client.
Args:
server_url: The URL of the HTTP Stream MCP server
debug: Whether to enable debug logging
timeout: Timeout in seconds for operations (default: 60)
options: Additional configuration options for the transport
"""
# Parse URL to extract base URL and endpoint
parsed = urlparse(server_url)
# If the URL already has a path, use it; otherwise use default /mcp endpoint
if parsed.path and parsed.path != '/':
self.base_url = server_url
else:
# Default endpoint is /mcp
self.base_url = urljoin(server_url, '/mcp')
self.debug = debug
self.timeout = timeout
self.options = options or {}
self.session = None
self.tools = []
self.transport = None
# Set up logging
if debug:
logger.setLevel(logging.DEBUG)
else:
# Set to WARNING by default to hide INFO messages
logger.setLevel(logging.WARNING)
self._initialize()
def _initialize(self):
"""Initialize the connection and tools."""
# Use the global event loop
loop = get_event_loop()
# Start a background thread to run the event loop
def run_event_loop():
asyncio.set_event_loop(loop)
loop.run_forever()
self.loop_thread = threading.Thread(target=run_event_loop, daemon=True)
self.loop_thread.start()
# Run the initialization in the event loop
future = asyncio.run_coroutine_threadsafe(self._async_initialize(), loop)
self.tools = future.result(timeout=self.timeout)
async def _async_initialize(self):
"""Asynchronously initialize the connection and tools."""
logger.debug(f"Connecting to MCP server at {self.base_url}")
# Create HTTP Stream transport
self.transport = HTTPStreamTransport(self.base_url, options=self.options)
await self.transport.__aenter__()
# Create read and write streams
read_stream = self.transport.read_stream()
write_stream = self.transport.write_stream()
# Start SSE listener if in stream mode
if self.options.get('responseMode', 'batch') == 'stream':
await self.transport.start_sse_listener()
# Create client session
self._session_context = ClientSession(read_stream, write_stream)
self.session = await self._session_context.__aenter__()
# Initialize
await self.session.initialize()
# List available tools
logger.debug("Listing tools...")
response = await self.session.list_tools()
tools_data = response.tools
logger.debug(f"Found {len(tools_data)} tools: {[tool.name for tool in tools_data]}")
# Create tool wrappers
tools = []
for tool in tools_data:
input_schema = tool.inputSchema if hasattr(tool, 'inputSchema') else None
wrapper = HTTPStreamMCPTool(
name=tool.name,
description=tool.description if hasattr(tool, 'description') else f"Call the {tool.name} tool",
session=self.session,
input_schema=input_schema,
timeout=self.timeout
)
tools.append(wrapper)
return tools
def __iter__(self):
"""Return an iterator over the tools."""
return iter(self.tools)
def to_openai_tools(self):
"""Convert all tools to OpenAI format."""
return [tool.to_openai_tool() for tool in self.tools]
async def __aenter__(self):
"""Async context manager entry."""
return self
async def __aexit__(self, exc_type, exc_val, exc_tb):
"""Async context manager exit."""
if self.transport:
await self.transport.__aexit__(exc_type, exc_val, exc_tb)
if hasattr(self, '_session_context') and self._session_context:
await self._session_context.__aexit__(exc_type, exc_val, exc_tb)
@@ -44,13 +44,30 @@ class SSEMCPTool:
# Create a signature based on input schema
params = []
if input_schema and 'properties' in input_schema:
for param_name in input_schema['properties']:
for param_name, prop_schema in input_schema['properties'].items():
# Determine type annotation based on schema
prop_type = prop_schema.get('type', 'string') if isinstance(prop_schema, dict) else 'string'
if prop_type == 'string':
annotation = str
elif prop_type == 'integer':
annotation = int
elif prop_type == 'number':
annotation = float
elif prop_type == 'boolean':
annotation = bool
elif prop_type == 'array':
annotation = list
elif prop_type == 'object':
annotation = dict
else:
annotation = str # Default to string for SSE
params.append(
inspect.Parameter(
name=param_name,
kind=inspect.Parameter.POSITIONAL_OR_KEYWORD,
default=inspect.Parameter.empty if param_name in input_schema.get('required', []) else None,
annotation=str # Default to string
annotation=annotation
)
)
@@ -6,9 +6,9 @@ from pydantic import BaseModel, ConfigDict
from ..agent.agent import Agent
from ..task.task import Task
from ..main import display_error
from ..llm import LLM
import csv
import os
from openai import AsyncOpenAI, OpenAI
class LoopItems(BaseModel):
model_config = ConfigDict(arbitrary_types_allowed=True)
@@ -16,6 +16,7 @@ class LoopItems(BaseModel):
class Process:
DEFAULT_RETRY_LIMIT = 3 # Predefined retry limit in a common place
VALIDATION_FAILURE_DECISIONS = ["invalid", "retry", "failed", "error", "unsuccessful", "fail", "errors", "reject", "rejected", "incomplete"] # Decision strings that trigger validation feedback
def __init__(self, tasks: Dict[str, Task], agents: List[Agent], manager_llm: Optional[str] = None, verbose: bool = False, max_iter: int = 10):
logging.debug(f"=== Initializing Process ===")
@@ -33,12 +34,61 @@ class Process:
self.task_retry_counter: Dict[str, int] = {} # Initialize retry counter
self.workflow_finished = False # ADDED: Workflow finished flag
def _create_llm_instance(self):
"""Create and return a configured LLM instance for manager tasks."""
return LLM(model=self.manager_llm, temperature=0.7)
def _parse_manager_instructions(self, response, ManagerInstructions):
"""Parse LLM response and return ManagerInstructions instance.
Args:
response: String response from LLM
ManagerInstructions: Pydantic model class for validation
Returns:
ManagerInstructions instance
Raises:
Exception: If parsing fails
"""
try:
parsed_json = json.loads(response)
return ManagerInstructions(**parsed_json)
except (json.JSONDecodeError, ValueError, TypeError) as e:
raise Exception(f"Failed to parse response: {response}") from e
def _create_loop_subtasks(self, loop_task: Task):
"""Create subtasks for a loop task from input file."""
logging.warning(f"_create_loop_subtasks called for {loop_task.name} but method not fully implemented")
# TODO: Implement loop subtask creation from input file
# This should read loop_task.input_file and create subtasks
pass
def _build_task_context(self, current_task: Task) -> str:
"""Build context for a task based on its retain_full_context setting"""
if not (current_task.previous_tasks or current_task.context):
return ""
# Check if we have validation feedback to include
if current_task.validation_feedback:
feedback = current_task.validation_feedback
context = f"\nPrevious attempt failed validation with reason: {feedback['validation_response']}"
if feedback.get('validated_task'):
context += f"\nValidated task: {feedback['validated_task']}"
if feedback.get('validation_details'):
context += f"\nValidation feedback: {feedback['validation_details']}"
if feedback.get('rejected_output'):
context += f"\nRejected output: {feedback['rejected_output']}"
context += "\nPlease try again with a different approach based on this feedback.\n"
# Clear the feedback after including it to prevent it from persisting
current_task.validation_feedback = None
context = "\nInput data from previous tasks:"
# If we have validation feedback but no previous tasks context, return just the feedback
if not (current_task.previous_tasks or current_task.context):
return context
# Otherwise, append the regular context
context += "\nInput data from previous tasks:"
elif not (current_task.previous_tasks or current_task.context):
return ""
else:
context = "\nInput data from previous tasks:"
if current_task.retain_full_context:
# Original behavior: include all previous tasks
@@ -140,24 +190,24 @@ class Process:
def _get_manager_instructions_with_fallback(self, manager_task, manager_prompt, ManagerInstructions):
"""Sync version of getting manager instructions with fallback"""
# Create OpenAI client
client = OpenAI()
# Create LLM instance with the manager_llm
llm = self._create_llm_instance()
try:
# First try structured output (OpenAI compatible)
# Use LLM with output_pydantic for structured output
logging.info("Attempting structured output...")
manager_response = client.beta.chat.completions.parse(
model=self.manager_llm,
messages=[
{"role": "system", "content": manager_task.description},
{"role": "user", "content": manager_prompt}
],
temperature=0.7,
response_format=ManagerInstructions
response = llm.get_response(
prompt=manager_prompt,
system_prompt=manager_task.description,
output_pydantic=ManagerInstructions
)
return manager_response.choices[0].message.parsed
# Parse the response and validate with Pydantic
return self._parse_manager_instructions(response, ManagerInstructions)
except Exception as e:
logging.info(f"Structured output failed: {e}, falling back to JSON mode...")
# Fallback to regular JSON mode
# Fallback to regular JSON mode with explicit JSON instructions
try:
# Generate JSON structure description from Pydantic model
try:
@@ -173,23 +223,14 @@ class Process:
# Fallback to hardcoded prompt if schema generation fails
enhanced_prompt = manager_prompt + "\n\nIMPORTANT: Respond with valid JSON only, using this exact structure: {\"task_id\": <int>, \"agent_name\": \"<string>\", \"action\": \"<execute or stop>\"}"
manager_response = client.chat.completions.create(
model=self.manager_llm,
messages=[
{"role": "system", "content": manager_task.description},
{"role": "user", "content": enhanced_prompt}
],
temperature=0.7,
response_format={"type": "json_object"}
response = llm.get_response(
prompt=enhanced_prompt,
system_prompt=manager_task.description,
output_json=True
)
# Parse JSON and validate with Pydantic
try:
json_content = manager_response.choices[0].message.content
parsed_json = json.loads(json_content)
return ManagerInstructions(**parsed_json)
except (json.JSONDecodeError, ValueError) as e:
raise Exception(f"Failed to parse JSON response: {json_content}") from e
return self._parse_manager_instructions(response, ManagerInstructions)
except Exception as fallback_error:
error_msg = f"Both structured output and JSON fallback failed: {fallback_error}"
logging.error(error_msg, exc_info=True)
@@ -197,40 +238,32 @@ class Process:
async def _get_structured_response_async(self, manager_task, manager_prompt, ManagerInstructions):
"""Async version of structured response"""
# Create an async client instance for this async method
async_client = AsyncOpenAI()
manager_response = await async_client.beta.chat.completions.parse(
model=self.manager_llm,
messages=[
{"role": "system", "content": manager_task.description},
{"role": "user", "content": manager_prompt}
],
temperature=0.7,
response_format=ManagerInstructions
# Create LLM instance with the manager_llm
llm = self._create_llm_instance()
# Use async get_response with output_pydantic
response = await llm.get_response_async(
prompt=manager_prompt,
system_prompt=manager_task.description,
output_pydantic=ManagerInstructions
)
return manager_response.choices[0].message.parsed
# Parse the response and validate with Pydantic
return self._parse_manager_instructions(response, ManagerInstructions)
async def _get_json_response_async(self, manager_task, enhanced_prompt, ManagerInstructions):
"""Async version of JSON fallback response"""
# Create an async client instance for this async method
async_client = AsyncOpenAI()
manager_response = await async_client.chat.completions.create(
model=self.manager_llm,
messages=[
{"role": "system", "content": manager_task.description},
{"role": "user", "content": enhanced_prompt}
],
temperature=0.7,
response_format={"type": "json_object"}
# Create LLM instance with the manager_llm
llm = self._create_llm_instance()
response = await llm.get_response_async(
prompt=enhanced_prompt,
system_prompt=manager_task.description,
output_json=True
)
# Parse JSON and validate with Pydantic
try:
json_content = manager_response.choices[0].message.content
parsed_json = json.loads(json_content)
return ManagerInstructions(**parsed_json)
except (json.JSONDecodeError, ValueError) as e:
raise Exception(f"Failed to parse JSON response: {json_content}") from e
return self._parse_manager_instructions(response, ManagerInstructions)
async def aworkflow(self) -> AsyncGenerator[str, None]:
@@ -442,16 +475,18 @@ Subtask: {st.name}
logging.debug(f"Task type: {task_to_check.task_type}")
logging.debug(f"Task status before reset check: {task_to_check.status}")
logging.debug(f"Task rerun: {getattr(task_to_check, 'rerun', True)}") # default to True if not set
logging.debug(f"Task async_execution: {task_to_check.async_execution}")
if (getattr(task_to_check, 'rerun', True) and # Corrected condition - reset only if rerun is True (or default True)
task_to_check.task_type != "loop" and # Removed "decision" from exclusion
not any(t.task_type == "loop" and subtask_name.startswith(t.name + "_")
for t in self.tasks.values())):
logging.debug(f"=== Resetting non-loop, non-decision task {subtask_name} to 'not started' ===")
for t in self.tasks.values()) and
not task_to_check.async_execution): # Don't reset async parallel tasks
logging.debug(f"=== Resetting non-loop, non-decision, non-parallel task {subtask_name} to 'not started' ===")
self.tasks[task_id].status = "not started"
logging.debug(f"Task status after reset: {self.tasks[task_id].status}")
else:
logging.debug(f"=== Skipping reset for loop/decision/subtask or rerun=False: {subtask_name} ===")
logging.debug(f"=== Skipping reset for loop/decision/subtask/parallel or rerun=False: {subtask_name} ===")
logging.debug(f"Keeping status as: {self.tasks[task_id].status}")
# Handle loop progression
@@ -496,6 +531,35 @@ Subtask: {st.name}
next_task = next((t for t in self.tasks.values() if t.name == task_value), None)
if next_task:
next_task.status = "not started" # Reset status to allow execution
# Capture validation feedback for retry scenarios
if decision_str in Process.VALIDATION_FAILURE_DECISIONS:
if current_task and current_task.result:
# Get the rejected output from the task that was validated
validated_task = None
# Find the task that produced the output being validated
if current_task.previous_tasks:
# For validation tasks, typically validate the most recent previous task
prev_task_name = current_task.previous_tasks[-1]
validated_task = next((t for t in self.tasks.values() if t.name == prev_task_name), None)
elif current_task.context:
# If no previous_tasks, check context for the validated task
# Use the most recent task with a result from context
for ctx_task in reversed(current_task.context):
if ctx_task.result and ctx_task.name != current_task.name:
validated_task = ctx_task
break
feedback = {
'validation_response': decision_str,
'validation_details': current_task.result.raw,
'rejected_output': validated_task.result.raw if validated_task and validated_task.result else None,
'validator_task': current_task.name,
'validated_task': validated_task.name if validated_task else None
}
next_task.validation_feedback = feedback
logging.debug(f"Added validation feedback to {next_task.name}: {feedback['validation_response']} (validated task: {feedback.get('validated_task', 'None')})")
logging.debug(f"Routing to {next_task.name} based on decision: {decision_str}")
# Don't mark workflow as finished when following condition path
self.workflow_finished = False
@@ -1043,16 +1107,18 @@ Subtask: {st.name}
logging.debug(f"Task type: {task_to_check.task_type}")
logging.debug(f"Task status before reset check: {task_to_check.status}")
logging.debug(f"Task rerun: {getattr(task_to_check, 'rerun', True)}") # default to True if not set
logging.debug(f"Task async_execution: {task_to_check.async_execution}")
if (getattr(task_to_check, 'rerun', True) and # Corrected condition - reset only if rerun is True (or default True)
task_to_check.task_type != "loop" and # Removed "decision" from exclusion
not any(t.task_type == "loop" and subtask_name.startswith(t.name + "_")
for t in self.tasks.values())):
logging.debug(f"=== Resetting non-loop, non-decision task {subtask_name} to 'not started' ===")
for t in self.tasks.values()) and
not task_to_check.async_execution): # Don't reset async parallel tasks
logging.debug(f"=== Resetting non-loop, non-decision, non-parallel task {subtask_name} to 'not started' ===")
self.tasks[task_id].status = "not started"
logging.debug(f"Task status after reset: {self.tasks[task_id].status}")
else:
logging.debug(f"=== Skipping reset for loop/decision/subtask or rerun=False: {subtask_name} ===")
logging.debug(f"=== Skipping reset for loop/decision/subtask/parallel or rerun=False: {subtask_name} ===")
logging.debug(f"Keeping status as: {self.tasks[task_id].status}")
@@ -1098,6 +1164,35 @@ Subtask: {st.name}
next_task = next((t for t in self.tasks.values() if t.name == task_value), None)
if next_task:
next_task.status = "not started" # Reset status to allow execution
# Capture validation feedback for retry scenarios
if decision_str in Process.VALIDATION_FAILURE_DECISIONS:
if current_task and current_task.result:
# Get the rejected output from the task that was validated
validated_task = None
# Find the task that produced the output being validated
if current_task.previous_tasks:
# For validation tasks, typically validate the most recent previous task
prev_task_name = current_task.previous_tasks[-1]
validated_task = next((t for t in self.tasks.values() if t.name == prev_task_name), None)
elif current_task.context:
# If no previous_tasks, check context for the validated task
# Use the most recent task with a result from context
for ctx_task in reversed(current_task.context):
if ctx_task.result and ctx_task.name != current_task.name:
validated_task = ctx_task
break
feedback = {
'validation_response': decision_str,
'validation_details': current_task.result.raw,
'rejected_output': validated_task.result.raw if validated_task and validated_task.result else None,
'validator_task': current_task.name,
'validated_task': validated_task.name if validated_task else None
}
next_task.validation_feedback = feedback
logging.debug(f"Added validation feedback to {next_task.name}: {feedback['validation_response']} (validated task: {feedback.get('validated_task', 'None')})")
logging.debug(f"Routing to {next_task.name} based on decision: {decision_str}")
# Don't mark workflow as finished when following condition path
self.workflow_finished = False
@@ -88,6 +88,7 @@ class Task:
self.max_retries = max_retries
self.retry_count = retry_count
self._guardrail_fn = None
self.validation_feedback = None # Store validation failure feedback for retry attempts
# Set logger level based on config verbose level
verbose = self.config.get("verbose", 0)
@@ -171,8 +172,15 @@ class Task:
# Check return annotation if present
return_annotation = sig.return_annotation
if return_annotation != inspect.Signature.empty:
# Import GuardrailResult for checking
from ..guardrails import GuardrailResult
# Check if it's a GuardrailResult type
is_guardrail_result = return_annotation is GuardrailResult
# Check for tuple return type
return_annotation_args = get_args(return_annotation)
if not (
is_tuple = (
get_origin(return_annotation) is tuple
and len(return_annotation_args) == 2
and return_annotation_args[0] is bool
@@ -182,9 +190,11 @@ class Task:
or return_annotation_args[1] is TaskOutput
or return_annotation_args[1] == Union[str, TaskOutput]
)
):
)
if not (is_guardrail_result or is_tuple):
raise ValueError(
"If return type is annotated, it must be Tuple[bool, Any]"
"If return type is annotated, it must be GuardrailResult or Tuple[bool, Any]"
)
self._guardrail_fn = self.guardrail
@@ -446,7 +456,11 @@ Context:
# Call the guardrail function
result = self._guardrail_fn(task_output)
# Convert the result to a GuardrailResult
# Check if result is already a GuardrailResult
if isinstance(result, GuardrailResult):
return result
# Otherwise, convert the tuple result to a GuardrailResult
return GuardrailResult.from_tuple(result)
except Exception as e:
+4 -3
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "praisonaiagents"
version = "0.0.119"
version = "0.0.129"
description = "Praison AI agents for completing complex tasks with Self Reflection Agents"
requires-python = ">=3.10"
authors = [
@@ -15,7 +15,8 @@ dependencies = [
"rich",
"openai",
"mcp>=1.6.0",
"posthog>=3.0.0"
"posthog>=3.0.0",
"aiohttp>=3.8.0"
]
[project.optional-dependencies]
@@ -73,4 +74,4 @@ all = [
[tool.setuptools.packages.find]
where = ["."]
include = ["praisonaiagents*"]
include = ["praisonaiagents*"]
+8
View File
@@ -0,0 +1,8 @@
from praisonaiagents import Agent, MCP
search_agent = Agent(
instructions="""You are a Tweet.""",
llm="openai/gpt-4o-mini",
tools=MCP("http://localhost:8080/sse")
)
search_agent.launch(path="/tweet", port=3030)
@@ -0,0 +1,4 @@
from praisonaiagents import Agent
agent = Agent(instructions="Create a Tweet")
agent.launch(port=8080, protocol="mcp")
@@ -0,0 +1,143 @@
#!/usr/bin/env python3
"""
Test script to verify that AutoAgents hierarchical workflow works with non-OpenAI LLMs
without requiring OpenAI API key.
This tests the fix for issue #873.
"""
import os
import sys
# Ensure OpenAI API key is not set to verify the fix works
if 'OPENAI_API_KEY' in os.environ:
del os.environ['OPENAI_API_KEY']
# Add the parent directory to the Python path
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from praisonaiagents import AutoAgents
def get_stock_price(company_name: str) -> str:
"""
Get the stock price of a company
Args:
company_name (str): The name of the company
Returns:
str: The stock price of the company
"""
if company_name.lower() == "apple" or company_name.lower() == "aapl":
return f"The stock price of {company_name} is 100"
elif company_name.lower() == "google" or company_name.lower() == "googl":
return f"The stock price of {company_name} is 200"
else:
return f"The stock price of {company_name} is 50"
def test_hierarchical_without_openai():
"""Test hierarchical process without OpenAI API key"""
print("Testing AutoAgents with hierarchical process and Gemini LLM...")
print("OPENAI_API_KEY is set:", 'OPENAI_API_KEY' in os.environ)
try:
# Create AutoAgents instance with Gemini
agents = AutoAgents(
instructions="Write a poem on the stock price of apple",
tools=[get_stock_price],
process="hierarchical",
llm="gemini/gemini-2.5-flash-lite-preview-06-17",
self_reflect=True,
verbose=True,
max_agents=3 # Maximum number of agents to create
)
# Start the agents
print("\nStarting agents...")
result = agents.start()
print("\n=== RESULT ===")
print(result)
print("\n=== TEST PASSED ===")
print("Hierarchical workflow completed successfully without OpenAI API key!")
return True
except Exception as e:
if "api_key" in str(e) and "OPENAI_API_KEY" in str(e):
print("\n=== TEST FAILED ===")
print(f"Error: Still requires OpenAI API key: {e}")
return False
else:
print(f"\nUnexpected error: {e}")
import traceback
traceback.print_exc()
return False
def test_hierarchical_with_mock_llm():
"""Test with a mock LLM to ensure no API calls are made to OpenAI"""
print("\n\nTesting with mock LLM configuration...")
try:
# Create AutoAgents instance with a mock configuration
agents = AutoAgents(
instructions="Simple test task",
tools=[get_stock_price],
process="hierarchical",
llm={
"model": "test/mock-model",
"api_key": "mock-key",
"base_url": "http://localhost:9999" # Non-existent URL
},
verbose=False,
max_agents=2
)
# This should fail with connection error, not OpenAI API key error
try:
result = agents.start()
# If we get here, the test didn't behave as expected
print("\n=== TEST WARNING ===")
print("Expected an error but execution succeeded")
return False
except Exception as e:
if "api_key" in str(e) and "OPENAI_API_KEY" in str(e):
print("\n=== TEST FAILED ===")
print(f"Error: Still requires OpenAI API key: {e}")
return False
else:
print("\n=== TEST PASSED ===")
print("Failed with expected error (not OpenAI API key error)")
print(f"Error type: {type(e).__name__}")
return True
except Exception as e:
print(f"\nSetup error: {e}")
return False
if __name__ == "__main__":
# Track test results
test_results = []
# Run both tests
mock_test_passed = test_hierarchical_with_mock_llm()
test_results.append(("Mock LLM Test", mock_test_passed))
# Only run the actual Gemini test if we have the API key
if 'GOOGLE_API_KEY' in os.environ or 'GEMINI_API_KEY' in os.environ:
gemini_test_passed = test_hierarchical_without_openai()
test_results.append(("Gemini Test", gemini_test_passed))
else:
print("\n\nSkipping Gemini test - no GOOGLE_API_KEY or GEMINI_API_KEY found")
print("The mock test passed, confirming the fix works!")
# Summary
print("\n=== TEST SUMMARY ===")
all_passed = True
for test_name, passed in test_results:
status = "PASSED" if passed else "FAILED"
print(f"{test_name}: {status}")
if not passed:
all_passed = False
# Exit with appropriate code
sys.exit(0 if all_passed else 1)
@@ -0,0 +1,163 @@
#!/usr/bin/env python3
"""
Test different guardrail return types to ensure both GuardrailResult
and Tuple[bool, Any] work correctly. This addresses issue #875.
"""
import sys
import os
from typing import Tuple, Any
from praisonaiagents import Agent, Task, TaskOutput, GuardrailResult
import inspect
def test_guardrail_return_types():
"""Test that both GuardrailResult and Tuple[bool, Any] return types are accepted."""
print("Testing guardrail return type validation...")
# Test 1: GuardrailResult return type
def validate_with_guardrailresult(task_output: TaskOutput) -> GuardrailResult:
"""Validation function returning GuardrailResult."""
if len(task_output.raw) < 10:
return GuardrailResult(
success=False,
result=None,
error="Output is too short"
)
return GuardrailResult(
success=True,
result=task_output,
error=""
)
# Test 2: Tuple[bool, Any] return type
def validate_with_tuple(task_output: TaskOutput) -> Tuple[bool, Any]:
"""Validation function returning tuple."""
if len(task_output.raw) < 10:
return False, "Output is too short"
return True, task_output
# Test 3: No return type annotation
def validate_no_annotation(task_output: TaskOutput):
"""Validation function without return type annotation."""
if len(task_output.raw) < 10:
return False, "Output is too short"
return True, task_output
# Create test agent
agent = Agent(
name="Test Agent",
role="Tester",
goal="Test guardrails",
backstory="Testing guardrail functionality",
llm="gpt-4o"
)
# Test creating tasks with each guardrail type
tests = [
("GuardrailResult return type", validate_with_guardrailresult),
("Tuple[bool, Any] return type", validate_with_tuple),
("No return type annotation", validate_no_annotation)
]
for test_name, guardrail_func in tests:
print(f"\nTesting {test_name}...")
try:
task = Task(
description="Test task",
expected_output="Test output",
agent=agent,
guardrail=guardrail_func
)
print(f"✓ Task created successfully with {test_name}")
# Verify guardrail was set up
assert task._guardrail_fn is not None, "Guardrail function not set"
# Test guardrail processing
test_output = TaskOutput(
description="Test",
raw="This is a test output that is long enough",
agent="Test Agent"
)
result = task._process_guardrail(test_output)
assert isinstance(result, GuardrailResult), f"Expected GuardrailResult, got {type(result)}"
assert result.success, "Guardrail validation should pass"
print(f"✓ Guardrail processing works with {test_name}")
except Exception as e:
print(f"✗ Failed with {test_name}: {e}")
return False
print("\n✓ All guardrail return type tests passed!")
return True
def test_guardrail_result_handling():
"""Test that _process_guardrail correctly handles both return types."""
print("\nTesting guardrail result handling...")
# Create test agent and task
agent = Agent(
name="Test Agent",
role="Tester",
goal="Test guardrails",
backstory="Testing guardrail functionality",
llm="gpt-4o"
)
test_output = TaskOutput(
description="Test",
raw="Test output",
agent="Test Agent"
)
# Test 1: Function returning GuardrailResult
def return_guardrailresult(output: TaskOutput) -> GuardrailResult:
return GuardrailResult(success=True, result=output, error="")
task1 = Task(
description="Test",
agent=agent,
guardrail=return_guardrailresult
)
result1 = task1._process_guardrail(test_output)
assert isinstance(result1, GuardrailResult), "Should return GuardrailResult"
assert result1.success, "Should be successful"
print("✓ GuardrailResult return handled correctly")
# Test 2: Function returning tuple
def return_tuple(output: TaskOutput) -> Tuple[bool, Any]:
return True, output
task2 = Task(
description="Test",
agent=agent,
guardrail=return_tuple
)
result2 = task2._process_guardrail(test_output)
assert isinstance(result2, GuardrailResult), "Should convert tuple to GuardrailResult"
assert result2.success, "Should be successful"
print("✓ Tuple return converted correctly")
print("✓ Guardrail result handling tests passed!")
return True
if __name__ == "__main__":
print("Running guardrail return type tests for issue #875...")
success = True
success &= test_guardrail_return_types()
success &= test_guardrail_result_handling()
if success:
print("\n✅ All tests passed! The fix for issue #875 is working correctly.")
sys.exit(0)
else:
print("\n❌ Some tests failed!")
sys.exit(1)
@@ -0,0 +1,104 @@
#!/usr/bin/env python3
"""
Test script to verify that the manager LLM methods correctly handle Pydantic models.
This tests the fix for the LLM interface misuse bug.
"""
import os
import sys
import json
from pydantic import BaseModel
# Add the parent directory to the Python path
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from praisonaiagents.process.process import Process
from praisonaiagents.task.task import Task
from praisonaiagents.agent.agent import Agent
class ManagerInstructions(BaseModel):
task_id: int
agent_name: str
action: str
def test_parse_manager_instructions():
"""Test the _parse_manager_instructions helper method"""
print("Testing _parse_manager_instructions method...")
# Create a dummy process instance
tasks = {"1": Task(name="test_task", description="Test task")}
agents = [Agent(name="TestAgent", role="Test role", goal="Test goal")]
process = Process(tasks=tasks, agents=agents, manager_llm="test/model")
# Test valid JSON response
valid_json = '{"task_id": 1, "agent_name": "TestAgent", "action": "execute"}'
try:
result = process._parse_manager_instructions(valid_json, ManagerInstructions)
assert isinstance(result, ManagerInstructions)
assert result.task_id == 1
assert result.agent_name == "TestAgent"
assert result.action == "execute"
print("✓ Valid JSON parsing works correctly")
except Exception as e:
print(f"✗ Failed to parse valid JSON: {e}")
return False
# Test invalid JSON
invalid_json = "not a json string"
try:
result = process._parse_manager_instructions(invalid_json, ManagerInstructions)
print("✗ Should have failed on invalid JSON")
return False
except Exception as e:
print(f"✓ Correctly raised exception for invalid JSON: {type(e).__name__}")
# Test JSON with missing fields
incomplete_json = '{"task_id": 1}'
try:
result = process._parse_manager_instructions(incomplete_json, ManagerInstructions)
print("✗ Should have failed on incomplete JSON")
return False
except Exception as e:
print(f"✓ Correctly raised exception for incomplete JSON: {type(e).__name__}")
return True
def test_create_llm_instance():
"""Test the _create_llm_instance helper method"""
print("\nTesting _create_llm_instance method...")
# Create a dummy process instance
tasks = {"1": Task(name="test_task", description="Test task")}
agents = [Agent(name="TestAgent", role="Test role", goal="Test goal")]
manager_llm = "gemini/gemini-2.5-flash-lite-preview-06-17"
process = Process(tasks=tasks, agents=agents, manager_llm=manager_llm)
# Test LLM instance creation
try:
llm = process._create_llm_instance()
assert llm.model == manager_llm
assert llm.temperature == 0.7
print("✓ LLM instance created successfully")
return True
except Exception as e:
print(f"✗ Failed to create LLM instance: {e}")
return False
if __name__ == "__main__":
print("=== Testing Manager LLM Fix ===\n")
# Run tests
test_results = []
test_results.append(("_parse_manager_instructions", test_parse_manager_instructions()))
test_results.append(("_create_llm_instance", test_create_llm_instance()))
# Summary
print("\n=== TEST SUMMARY ===")
all_passed = True
for test_name, passed in test_results:
status = "PASSED" if passed else "FAILED"
print(f"{test_name}: {status}")
if not passed:
all_passed = False
sys.exit(0 if all_passed else 1)
+141
View File
@@ -0,0 +1,141 @@
#!/usr/bin/env python3
"""
Test script for multi-provider/multi-model support
"""
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from praisonaiagents import Agent, Task, PraisonAIAgents
from praisonaiagents.agent import RouterAgent
from praisonaiagents.llm import ModelRouter, TaskComplexity
def test_model_router():
"""Test the ModelRouter functionality"""
print("=== Testing ModelRouter ===\n")
router = ModelRouter()
# Test complexity analysis
test_tasks = [
("What is 2 + 2?", TaskComplexity.SIMPLE),
("Summarize the benefits of renewable energy", TaskComplexity.MODERATE),
("Implement a binary search tree in Python", TaskComplexity.COMPLEX),
("Design a comprehensive microservices architecture with detailed analysis", TaskComplexity.VERY_COMPLEX)
]
for task, expected in test_tasks:
complexity = router.analyze_task_complexity(task)
print(f"Task: {task[:50]}...")
print(f"Expected: {expected.value}, Got: {complexity.value}")
print(f"Match: {'✓' if complexity == expected else '✗'}\n")
# Test model selection
print("\n=== Testing Model Selection ===\n")
selected = router.select_model(
task_description="Calculate the sum of numbers",
budget_conscious=True
)
print(f"Simple task (budget mode): {selected}")
selected = router.select_model(
task_description="Write a complex algorithm for graph traversal",
budget_conscious=False
)
print(f"Complex task (performance mode): {selected}")
print("\n✓ ModelRouter tests completed")
def test_router_agent():
"""Test the RouterAgent functionality"""
print("\n=== Testing RouterAgent ===\n")
# Create a router agent
agent = RouterAgent(
name="Test Agent",
role="Test Assistant",
goal="Test multi-model functionality",
models=["gpt-4o-mini"], # Using single model for testing
routing_strategy="auto",
verbose=False
)
print(f"Agent created: {agent.name}")
print(f"Available models: {list(agent.available_models.keys())}")
print(f"Routing strategy: {agent.routing_strategy}")
# Test model selection
selected = agent._select_model_for_task(
task_description="Simple math problem",
tools=None
)
print(f"Selected model for simple task: {selected}")
print("\n✓ RouterAgent tests completed")
def test_integration():
"""Test integration with PraisonAIAgents"""
print("\n=== Testing Integration ===\n")
# Create a simple router agent
agent = RouterAgent(
name="Integration Test Agent",
role="Tester",
goal="Test the integration",
models=["gpt-4o-mini"], # Single model for testing
routing_strategy="manual", # Use manual to avoid complexity
verbose=False
)
# Create a simple task
task = Task(
name="test_task",
description="Return the word 'success'",
expected_output="The word success",
agent=agent
)
# Create agents system
agents_system = PraisonAIAgents(
agents=[agent],
tasks=[task],
process="sequential",
verbose=False
)
print("Created PraisonAIAgents with RouterAgent")
print("✓ Integration test setup completed")
# Note: Actual execution would require API keys
def main():
"""Run all tests"""
print("\n🚀 Multi-Provider/Multi-Model Support Test Suite\n")
try:
test_model_router()
test_router_agent()
test_integration()
print("\n✅ All tests completed successfully!")
print("\n📝 Summary:")
print("- ModelRouter can analyze task complexity")
print("- ModelRouter can select appropriate models")
print("- RouterAgent can be created and configured")
print("- Integration with PraisonAIAgents works")
print("\n🎉 Multi-provider support is ready to use!")
except Exception as e:
print(f"\n❌ Test failed with error: {e}")
import traceback
traceback.print_exc()
sys.exit(1)
if __name__ == "__main__":
main()
@@ -0,0 +1,49 @@
"""Test sequential tool calling fix"""
from praisonaiagents import Agent
def get_stock_price(company_name: str) -> str:
"""
Get the stock price of a company
Args:
company_name (str): The name of the company
Returns:
str: The stock price of the company
"""
print(f"Tool called: get_stock_price({company_name})")
return f"The stock price of {company_name} is 100"
def multiply(a: int, b: int) -> int:
"""
Multiply two numbers
"""
print(f"Tool called: multiply({a}, {b})")
return a * b
# Test with streaming disabled to verify the fix
print("Testing sequential tool calling with stream=False...")
agent = Agent(
instructions="You are a helpful assistant. You can use the tools provided to you to help the user.",
llm="gemini/gemini-2.5-flash-lite-preview-06-17",
self_reflect=False,
verbose=True,
tools=[get_stock_price, multiply],
stream=False # Force non-streaming mode - use stream parameter directly
)
result = agent.chat("Get the stock price of Google and multiply it by 2")
print(f"\nFinal result: {result}")
# Test with default streaming mode
print("\n\nTesting sequential tool calling with default streaming...")
agent2 = Agent(
instructions="You are a helpful assistant. You can use the tools provided to you to help the user.",
llm="gemini/gemini-2.5-flash-lite-preview-06-17",
self_reflect=False,
verbose=True,
tools=[get_stock_price, multiply]
)
result2 = agent2.chat("Get the stock price of Google and multiply it by 2")
print(f"\nFinal result: {result2}")
@@ -0,0 +1,161 @@
#!/usr/bin/env python3
"""Test script for validation feedback in workflow retry logic"""
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'src/praisonai-agents'))
from praisonaiagents import Agent, Task, PraisonAIAgents
def test_example_one():
"""Test the philosopher quotes example"""
print("=" * 80)
print("Testing Example 1: Philosopher Quotes with Validation")
print("=" * 80)
validator_agent = Agent(
name="validator_agent",
instructions="You are a helpful assistant",
llm="gemini/gemini-2.5-flash",
self_reflect=False,
verbose=True
)
quote_agent = Agent(
name="quote_agent",
instructions="You are a helpful assistant who is a pro at quoting philosophers",
llm="gemini/gemini-2.5-flash",
self_reflect=False,
verbose=False
)
collect_task = Task(
description="Quote Aristotle",
expected_output="List of Quotes from Aristotle",
agent=quote_agent,
name="collect_data",
is_start=True,
next_tasks=["validate_data"]
)
validate_task = Task(
description="""Validate the collected data. Check if:
1. At least 25 results are returned.
2. Each is a quote from Aristotle.
Return validation_result as 'valid' or 'invalid' only no other text.""",
expected_output="Validation result indicating if data is valid or invalid.",
agent=validator_agent,
name="validate_data",
task_type="decision",
condition={
"valid": [], # End the workflow on valid data
"invalid": ["collect_data"] # Retry data collection on invalid data
},
)
# Workflow
agents = PraisonAIAgents(
agents=[quote_agent, validator_agent],
tasks=[collect_task, validate_task],
verbose=1,
process="workflow",
max_iter=3 # Limit iterations to prevent infinite loops
)
try:
agents.start()
print("\nExample 1 completed successfully!")
except Exception as e:
print(f"\nExample 1 failed with error: {e}")
def test_example_two():
"""Test the web search example"""
print("\n" + "=" * 80)
print("Testing Example 2: Web Search with Validation (Mock)")
print("=" * 80)
# Mock web search tool for testing
def mock_web_search(query: str) -> str:
"""Mock web search that returns limited results initially"""
# Simulate returning too few results on first attempt
return "Found 3 results:\n1. https://example.com/kenya1\n2. https://example.com/kenya2\n3. https://example.com/kenya3"
agent = Agent(
name="agent",
instructions="You are a helpful assistant",
llm="gemini/gemini-2.5-flash",
self_reflect=False,
verbose=True
)
web_search_agent = Agent(
name="web_search_agent",
role="Web Search Specialist",
goal="Search the web for relevant information",
instructions="You are a helpful assistant. You can use the tools provided to you to help the user.",
llm="gemini/gemini-2.5-flash",
self_reflect=False,
verbose=False,
tools=[mock_web_search]
)
# Tasks
collect_task = Task(
description="Perform an internet search using the query: 'Kenya protests in 2024 and 2025'",
expected_output="List of URLs of the articles that are relevant to the search",
agent=web_search_agent,
name="collect_data",
is_start=True,
next_tasks=["validate_data"]
)
validate_task = Task(
description="""Validate the collected data. Check if:
1. At least 12 results are returned.
2. Each result contains a valid URL.
Return validation_result as 'valid' or 'invalid' only no other text.""",
expected_output="Validation result indicating if data is valid or invalid.",
agent=agent,
name="validate_data",
task_type="decision",
condition={
"valid": [], # End the workflow on valid data
"invalid": ["collect_data"] # Retry data collection on invalid data
},
)
# Workflow
agents = PraisonAIAgents(
agents=[web_search_agent, agent],
tasks=[collect_task, validate_task],
verbose=1,
process="workflow",
max_iter=3 # Limit iterations
)
try:
agents.start()
print("\nExample 2 completed successfully!")
except Exception as e:
print(f"\nExample 2 failed with error: {e}")
if __name__ == "__main__":
print("Testing Validation Feedback Implementation")
print("This test demonstrates that retry tasks now receive validation feedback\n")
# Run example 1 (without actual API calls, just structure test)
try:
# We'll just test the structure is valid
test_example_one()
except Exception as e:
print(f"Note: Example 1 requires actual LLM API access: {e}")
# Run example 2 with mock
try:
test_example_two()
except Exception as e:
print(f"Note: Example 2 test setup issue: {e}")
print("\nTest script completed. The implementation adds validation feedback to retry tasks.")
+21 -8
View File
@@ -11,14 +11,30 @@ def analyze_code_changes():
]
return issues[int(time.time()) % 3]
def suggest_fixes(issue: Dict):
def suggest_fixes(issue):
"""Simulates fix suggestions"""
fixes = {
"style": "Apply PEP 8 formatting",
"security": "Implement input validation",
"security": "Implement input validation",
"performance": "Use list comprehension"
}
return fixes.get(issue["type"], "Review manually")
# Handle both dict and string inputs
if isinstance(issue, dict):
issue_type = issue.get("type", "unknown")
else:
# If it's a string, try to extract the type
issue_str = str(issue).lower()
if "security" in issue_str:
issue_type = "security"
elif "style" in issue_str:
issue_type = "style"
elif "performance" in issue_str:
issue_type = "performance"
else:
issue_type = "unknown"
return fixes.get(issue_type, "Review manually")
def apply_automated_fix(fix: str):
"""Simulates applying automated fixes"""
@@ -98,11 +114,8 @@ def main():
# Print results
print("\nCode Review Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
print("-" * 50)
print(f"Results: {results}")
print("-" * 50)
if __name__ == "__main__":
main()
@@ -0,0 +1,88 @@
#!/usr/bin/env python
"""
Example of using MCP with HTTP Stream transport.
This demonstrates how to connect to an MCP server using the new HTTP Stream transport.
"""
from praisonaiagents import Agent
from praisonaiagents.mcp import MCP
# Example 1: Basic HTTP Stream connection (defaults to /mcp endpoint)
print("Example 1: Basic HTTP Stream connection")
try:
# This will use HTTP Stream transport automatically
tools = MCP("http://localhost:8080", debug=True)
print(f"Connected! Found {len(list(tools))} tools")
for tool in tools:
print(f" - {tool.name}: {tool.description}")
except Exception as e:
print(f"Connection failed: {e}")
print("\n" + "="*50 + "\n")
# Example 2: HTTP Stream with custom endpoint
print("Example 2: HTTP Stream with custom endpoint")
try:
# This will use HTTP Stream transport with custom endpoint
tools = MCP("http://localhost:8080/custom-mcp", debug=True)
print(f"Connected! Found {len(list(tools))} tools")
except Exception as e:
print(f"Connection failed: {e}")
print("\n" + "="*50 + "\n")
# Example 3: HTTP Stream with stream response mode
print("Example 3: HTTP Stream with stream response mode")
try:
# This will use HTTP Stream transport with streaming responses
tools = MCP("http://localhost:8080", responseMode="stream", debug=True)
print(f"Connected! Found {len(list(tools))} tools")
except Exception as e:
print(f"Connection failed: {e}")
print("\n" + "="*50 + "\n")
# Example 4: Backward compatibility - SSE transport
print("Example 4: Backward compatibility - SSE transport")
try:
# URLs ending with /sse will still use SSE transport
tools = MCP("http://localhost:8080/sse", debug=True)
print(f"Connected using SSE! Found {len(list(tools))} tools")
except Exception as e:
print(f"Connection failed: {e}")
print("\n" + "="*50 + "\n")
# Example 5: Using with an agent
print("Example 5: Using HTTP Stream tools with an agent")
try:
agent = Agent(
name="Weather Assistant",
instructions="You are a helpful weather assistant.",
tools=MCP("http://localhost:8080"), # HTTP Stream transport
llm="gpt-4o-mini"
)
print("Agent created successfully with HTTP Stream tools!")
# Example usage (commented out to avoid actual API calls)
# result = agent.execute("What's the weather in Paris?")
# print(f"Result: {result}")
except Exception as e:
print(f"Agent creation failed: {e}")
print("\n" + "="*50 + "\n")
# Example 6: HTTP Stream with custom headers and CORS
print("Example 6: HTTP Stream with custom configuration")
try:
tools = MCP(
"http://localhost:8080",
headers={"X-API-Key": "my-api-key"},
cors={"allowOrigin": "*"},
debug=True
)
print(f"Connected with custom config! Found {len(list(tools))} tools")
except Exception as e:
print(f"Connection failed: {e}")
print("\nNote: These examples require an HTTP Stream MCP server running on localhost:8080")
@@ -69,7 +69,7 @@ workflow = PraisonAIAgents(
)
# Run the workflow
results = workflow.start()
results = workflow.start(return_dict=True)
# Print results
print("\nWorkflow Results:")
@@ -0,0 +1,75 @@
"""
Test script to verify the fix for sequential tool calling.
The agent should:
1. Call get_stock_price to get Google's stock price (100)
2. Call multiply to multiply 100 by 2
3. Return the final result (200)
"""
from praisonaiagents import Agent
def get_stock_price(company_name: str) -> str:
"""
Get the stock price of a company
Args:
company_name (str): The name of the company
Returns:
str: The stock price of the company
"""
print(f"[Tool Called] get_stock_price({company_name})")
return f"The stock price of {company_name} is 100"
def multiply(a: int, b: int) -> int:
"""
Multiply two numbers
Args:
a (int): First number
b (int): Second number
Returns:
int: Product of a and b
"""
print(f"[Tool Called] multiply({a}, {b})")
return a * b
# Test with Gemini
print("=" * 60)
print("Testing with Gemini model")
print("=" * 60)
agent_gemini = Agent(
instructions="You are a helpful assistant. You can use the tools provided to you to help the user. When asked to multiply a stock price, first get the stock price, then multiply it.",
llm="gemini/gemini-2.5-pro",
tools=[get_stock_price, multiply],
verbose=True
)
result_gemini = agent_gemini.start("what is the stock price of Google? multiply the Google stock price with 2")
print(f"\nFinal Result (Gemini): {result_gemini}")
# Test with GPT-4
print("\n" + "=" * 60)
print("Testing with GPT-4 model")
print("=" * 60)
agent_gpt4 = Agent(
instructions="You are a helpful assistant. You can use the tools provided to you to help the user. When asked to multiply a stock price, first get the stock price, then multiply it.",
llm="gpt-4o",
tools=[get_stock_price, multiply],
verbose=True
)
result_gpt4 = agent_gpt4.start("what is the stock price of Google? multiply the Google stock price with 2")
print(f"\nFinal Result (GPT-4): {result_gpt4}")
# Verify results
print("\n" + "=" * 60)
print("Test Results Summary")
print("=" * 60)
print(f"Gemini result contains '200': {'200' in str(result_gemini) if result_gemini else False}")
print(f"GPT-4 result contains '200': {'200' in str(result_gpt4) if result_gpt4 else False}")
print(f"Gemini returned empty: {not result_gemini or result_gemini == ''}")
print(f"GPT-4 returned empty: {not result_gpt4 or result_gpt4 == ''}")
@@ -0,0 +1,58 @@
#!/usr/bin/env python
"""
Basic test to verify HTTP Stream implementation is working correctly.
"""
import sys
import os
# Add parent directory to path for imports
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from praisonaiagents.mcp import MCP
def test_transport_selection():
"""Test that URLs are correctly routed to appropriate transports."""
print("Testing transport selection logic...")
# Test 1: SSE URL should use SSE transport
try:
mcp_sse = MCP("http://localhost:8080/sse")
assert mcp_sse.is_sse == True
assert mcp_sse.is_http_stream == False
print("✓ SSE URL correctly uses SSE transport")
except Exception as e:
print(f"✗ SSE URL test failed: {e}")
# Test 2: Regular HTTP URL should use HTTP Stream transport
try:
mcp_http = MCP("http://localhost:8080")
assert mcp_http.is_sse == False
assert mcp_http.is_http_stream == True
print("✓ HTTP URL correctly uses HTTP Stream transport")
except Exception as e:
print(f"✗ HTTP URL test failed: {e}")
# Test 3: Custom endpoint should use HTTP Stream transport
try:
mcp_custom = MCP("http://localhost:8080/custom")
assert mcp_custom.is_sse == False
assert mcp_custom.is_http_stream == True
print("✓ Custom endpoint correctly uses HTTP Stream transport")
except Exception as e:
print(f"✗ Custom endpoint test failed: {e}")
# Test 4: Stdio transport should still work
try:
mcp_stdio = MCP("/Users/praison/miniconda3/envs/mcp/bin/python /Users/praison/stockprice/app.py")
assert mcp_stdio.is_sse == False
assert mcp_stdio.is_http_stream == False
print("✓ Stdio transport still works")
except Exception as e:
print(f"✗ Stdio transport test failed: {e}")
print("\nAll transport selection tests completed!")
if __name__ == "__main__":
test_transport_selection()
@@ -0,0 +1,252 @@
"""Unit tests for validation feedback in workflow retry logic"""
import pytest
from unittest.mock import Mock, patch
from praisonaiagents import Agent, Task, TaskOutput
from praisonaiagents.process import Process
class TestValidationFeedback:
"""Test validation feedback functionality"""
def test_task_has_validation_feedback_field(self):
"""Test that Task class has validation_feedback field"""
task = Task(
name="test_task",
description="Test task",
expected_output="Test output"
)
assert hasattr(task, 'validation_feedback')
assert task.validation_feedback is None
def test_validation_feedback_captured_on_invalid_decision(self):
"""Test that validation feedback is captured when decision is invalid"""
# Create mock agents
agent1 = Mock(spec=Agent)
agent2 = Mock(spec=Agent)
# Create tasks
collect_task = Task(
name="collect_data",
description="Collect data",
expected_output="Data",
agent=agent1,
is_start=True,
next_tasks=["validate_data"]
)
validate_task = Task(
name="validate_data",
description="Validate data",
expected_output="Validation result",
agent=agent2,
task_type="decision",
condition={
"valid": [],
"invalid": ["collect_data"]
}
)
# Set up task relationships
validate_task.previous_tasks = ["collect_data"]
# Create process
process = Process(
agents={"agent1": agent1, "agent2": agent2},
tasks={"collect_data": collect_task, "validate_data": validate_task},
verbose=0
)
# Simulate task execution results
collect_task.result = TaskOutput(
raw="Collected only 3 items",
agent="agent1"
)
collect_task.status = "completed"
validate_task.result = TaskOutput(
raw="Not enough results, need at least 10",
agent="agent2"
)
validate_task.status = "completed"
# Simulate workflow routing with invalid decision
# This would normally happen in the workflow execution
decision_str = "invalid"
current_task = validate_task
target_tasks = current_task.condition.get(decision_str, [])
if target_tasks:
task_value = target_tasks[0]
next_task = collect_task # This is the retry task
# The implementation should add validation feedback
if decision_str in Process.VALIDATION_FAILURE_DECISIONS:
if current_task and current_task.result:
validated_task = collect_task # The task that was validated
feedback = {
'validation_response': decision_str,
'validation_details': current_task.result.raw,
'rejected_output': validated_task.result.raw if validated_task and validated_task.result else None,
'validator_task': current_task.name
}
next_task.validation_feedback = feedback
# Verify feedback was captured
assert collect_task.validation_feedback is not None
assert collect_task.validation_feedback['validation_response'] == 'invalid'
assert collect_task.validation_feedback['validation_details'] == "Not enough results, need at least 10"
assert collect_task.validation_feedback['rejected_output'] == "Collected only 3 items"
assert collect_task.validation_feedback['validator_task'] == "validate_data"
def test_validation_feedback_included_in_context(self):
"""Test that validation feedback is included in task context"""
# Create a task with validation feedback
task = Task(
name="retry_task",
description="Original description",
expected_output="Expected output"
)
task.validation_feedback = {
'validation_response': 'invalid',
'validation_details': 'Not enough results',
'rejected_output': 'Previous output',
'validator_task': 'validator'
}
# Create process to test context building
process = Process(
agents={},
tasks={"retry_task": task},
verbose=0
)
# Build context
context = process._build_task_context(task)
# Verify feedback is in context
assert "Previous attempt failed validation with reason: invalid" in context
assert "Validation feedback: Not enough results" in context
assert "Rejected output: Previous output" in context
assert "Please try again with a different approach based on this feedback." in context
# Verify feedback was cleared after use
assert task.validation_feedback is None
def test_validation_feedback_backward_compatibility(self):
"""Test that tasks without validation feedback work normally"""
task = Task(
name="normal_task",
description="Normal task",
expected_output="Normal output"
)
process = Process(
agents={},
tasks={"normal_task": task},
verbose=0
)
# Build context without validation feedback
context = process._build_task_context(task)
# Should return empty string for task with no context
assert context == ""
# Task should still work normally
assert task.validation_feedback is None
def test_multiple_retry_decisions_supported(self):
"""Test that various failure decision strings trigger feedback capture"""
# Import the constant from Process class
from praisonaiagents.process import Process
failure_decisions = Process.VALIDATION_FAILURE_DECISIONS
for decision in failure_decisions:
task = Task(
name="test_task",
description="Test",
expected_output="Test"
)
# Simulate the decision routing logic
if decision in Process.VALIDATION_FAILURE_DECISIONS:
task.validation_feedback = {
'validation_response': decision,
'validation_details': f"Failed with {decision}",
'rejected_output': "Previous output",
'validator_task': "validator"
}
assert task.validation_feedback is not None
assert task.validation_feedback['validation_response'] == decision
def test_validation_feedback_with_context_tasks(self):
"""Test that validation feedback works when validated task is in context"""
# Create mock agents
agent1 = Mock(spec=Agent)
agent2 = Mock(spec=Agent)
# Create tasks
data_task = Task(
name="data_task",
description="Generate data",
expected_output="Data",
agent=agent1
)
validate_task = Task(
name="validate_data",
description="Validate data",
expected_output="Validation result",
agent=agent2,
task_type="decision",
context=[data_task], # Data task is in context, not previous_tasks
condition={
"valid": [],
"invalid": ["data_task"]
}
)
# Create process
process = Process(
agents={"agent1": agent1, "agent2": agent2},
tasks={"data_task": data_task, "validate_data": validate_task},
verbose=0
)
# Simulate task execution results
data_task.result = TaskOutput(
raw="Generated data with errors",
agent="agent1"
)
data_task.status = "completed"
validate_task.result = TaskOutput(
raw="Data has errors, please fix",
agent="agent2"
)
validate_task.status = "completed"
# Test the improved validation feedback logic
decision_str = "invalid"
current_task = validate_task
# This simulates the improved logic that checks context when no previous_tasks
validated_task = None
if current_task.previous_tasks:
prev_task_name = current_task.previous_tasks[-1]
validated_task = data_task if data_task.name == prev_task_name else None
elif current_task.context:
# Check context for the validated task
for ctx_task in reversed(current_task.context):
if ctx_task.result and ctx_task.name != current_task.name:
validated_task = ctx_task
break
# Verify the context-based task was found
assert validated_task is not None
assert validated_task.name == "data_task"
assert validated_task.result.raw == "Generated data with errors"
+3 -1
View File
@@ -2375,9 +2375,10 @@ wheels = [
[[package]]
name = "praisonaiagents"
version = "0.0.119"
version = "0.0.129"
source = { editable = "." }
dependencies = [
{ name = "aiohttp" },
{ name = "mcp" },
{ name = "openai" },
{ name = "posthog" },
@@ -2431,6 +2432,7 @@ telemetry = [
[package.metadata]
requires-dist = [
{ name = "aiohttp", specifier = ">=3.8.0" },
{ name = "chonkie", marker = "extra == 'knowledge'", specifier = ">=1.0.2" },
{ name = "chromadb", marker = "extra == 'graph'", specifier = ">=1.0.0" },
{ name = "chromadb", marker = "extra == 'knowledge'", specifier = ">=1.0.0" },
+2 -2
View File
@@ -3,8 +3,8 @@ class Praisonai < Formula
desc "AI tools for various AI applications"
homepage "https://github.com/MervinPraison/PraisonAI"
url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.2.46.tar.gz"
sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.2.46.tar.gz | shasum -a 256`.split.first
url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.2.55.tar.gz"
sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.2.55.tar.gz | shasum -a 256`.split.first
license "MIT"
depends_on "[email protected]"
+1 -1
View File
@@ -57,7 +57,7 @@ class CloudDeployer:
file.write("FROM python:3.11-slim\n")
file.write("WORKDIR /app\n")
file.write("COPY . .\n")
file.write("RUN pip install flask praisonai==2.2.46 gunicorn markdown\n")
file.write("RUN pip install flask praisonai==2.2.55 gunicorn markdown\n")
file.write("EXPOSE 8080\n")
file.write('CMD ["gunicorn", "-b", "0.0.0.0:8080", "api:app"]\n')
+4 -4
View File
@@ -1,6 +1,6 @@
[project]
name = "PraisonAI"
version = "2.2.46"
version = "2.2.55"
description = "PraisonAI is an AI Agents Framework with Self Reflection. PraisonAI application combines PraisonAI Agents, AutoGen, and CrewAI into a low-code solution for building and managing multi-agent LLM systems, focusing on simplicity, customisation, and efficient human-agent collaboration."
readme = "README.md"
license = ""
@@ -12,7 +12,7 @@ dependencies = [
"rich>=13.7",
"markdown>=3.5",
"pyparsing>=3.0.0",
"praisonaiagents>=0.0.119",
"praisonaiagents>=0.0.129",
"python-dotenv>=0.19.0",
"instructor>=1.3.3",
"PyYAML>=6.0",
@@ -95,7 +95,7 @@ autogen = ["pyautogen>=0.2.19", "praisonai-tools>=0.0.15", "crewai"]
[tool.poetry]
name = "PraisonAI"
version = "2.2.46"
version = "2.2.55"
description = "PraisonAI is an AI Agents Framework with Self Reflection. PraisonAI application combines PraisonAI Agents, AutoGen, and CrewAI into a low-code solution for building and managing multi-agent LLM systems, focusing on simplicity, customisation, and efficient human-agent collaboration."
authors = ["Mervin Praison"]
license = ""
@@ -113,7 +113,7 @@ python = ">=3.10,<3.13"
rich = ">=13.7"
markdown = ">=3.5"
pyparsing = ">=3.0.0"
praisonaiagents = ">=0.0.119"
praisonaiagents = ">=0.0.129"
python-dotenv = ">=0.19.0"
instructor = ">=1.3.3"
PyYAML = ">=6.0"
@@ -0,0 +1,163 @@
"""
Test to verify that async agents with Gemini models properly use tools
after the fix for issue #818
"""
import asyncio
import logging
from praisonaiagents import Agent, Task, PraisonAIAgents
# Enable logging to see tool calls
logging.basicConfig(level=logging.INFO)
# Mock search tool
async def mock_search(query: str) -> dict:
"""Mock search tool for testing"""
print(f"[TOOL CALLED] Searching for: {query}")
return {
"query": query,
"results": [
{
"title": f"Result 1 for {query}",
"snippet": f"This is a mock result about {query}",
"url": "https://example.com/1"
},
{
"title": f"Result 2 for {query}",
"snippet": f"Another mock result about {query}",
"url": "https://example.com/2"
}
],
"status": "success"
}
async def test_async_gemini_tools():
"""Test async agents with Gemini models use tools correctly"""
# Create search agent with Gemini model
search_agent = Agent(
name="AsyncSearcher",
role="Research Assistant",
goal="Find information using the search tool",
backstory="You are an expert at finding information online",
tools=[mock_search],
llm={"model": "gemini/gemini-1.5-flash-latest"},
verbose=True
)
# Create analysis agent without tools
analysis_agent = Agent(
name="Analyzer",
role="Data Analyst",
goal="Analyze search results",
backstory="You excel at analyzing and summarizing information",
llm={"model": "gemini/gemini-1.5-flash-latest"},
verbose=True
)
# Create tasks
search_task = Task(
name="search_task",
description="Search for information about 'quantum computing breakthroughs 2024'",
expected_output="Search results with at least 2 relevant findings",
agent=search_agent,
async_execution=True
)
analysis_task = Task(
name="analysis_task",
description="Analyze the search results and provide a summary",
expected_output="A concise summary of the findings",
agent=analysis_agent,
context=[search_task],
async_execution=False
)
# Create workflow
workflow = PraisonAIAgents(
agents=[search_agent, analysis_agent],
tasks=[search_task, analysis_task],
verbose=True
)
# Execute async
print("\n🚀 Starting async agent test with Gemini models...")
result = await workflow.astart()
# Check results
print("\n✅ Test Results:")
print("-" * 50)
# Verify search agent used the tool
search_result = str(result)
if "mock result" in search_result.lower() or "tool called" in search_result.lower():
print("✅ SUCCESS: Search agent properly used the mock_search tool!")
else:
print("❌ FAILURE: Search agent did NOT use the tool (claimed no internet access)")
# Show the actual output
print("\nFinal output:")
print(result)
return result
async def test_multiple_async_agents():
"""Test multiple async agents running in parallel"""
agents = []
tasks = []
# Create 3 search agents
for i in range(3):
agent = Agent(
name=f"AsyncAgent{i}",
role="Researcher",
goal="Search for information",
backstory="Expert researcher",
tools=[mock_search],
llm={"model": "gemini/gemini-1.5-flash-latest"}
)
task = Task(
name=f"task_{i}",
description=f"Search for 'AI advancement #{i+1}'",
expected_output="Search results",
agent=agent,
async_execution=True
)
agents.append(agent)
tasks.append(task)
# Execute all in parallel
workflow = PraisonAIAgents(agents=agents, tasks=tasks)
print("\n🚀 Testing multiple async agents in parallel...")
results = await workflow.astart()
# Verify all agents used tools
success_count = 0
for i, task in enumerate(tasks):
if "mock result" in str(results).lower():
success_count += 1
print(f"\n✅ {success_count}/{len(tasks)} agents successfully used tools")
return results
async def main():
"""Run all async tests"""
try:
# Test 1: Single async agent
await test_async_gemini_tools()
# Test 2: Multiple async agents in parallel
await test_multiple_async_agents()
print("\n🎉 All async tests completed!")
except Exception as e:
print(f"\n❌ Error during testing: {e}")
raise
if __name__ == "__main__":
asyncio.run(main())
+649
View File
@@ -0,0 +1,649 @@
"""
Unit tests for AutoAgents class
"""
import pytest
import sys
import os
from unittest.mock import Mock, patch, MagicMock
import json
import logging
# Add the source path for imports
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', '..', '..', 'praisonai-agents'))
try:
from praisonaiagents.agents.autoagents import AutoAgents, AutoAgentsConfig, AgentConfig, TaskConfig
from praisonaiagents.llm import OpenAIClient, LLM
except ImportError as e:
pytest.skip(f"Could not import required modules: {e}", allow_module_level=True)
class TestAutoAgents:
"""Test suite for AutoAgents functionality"""
@pytest.fixture
def sample_valid_config(self):
"""Fixture providing a valid AutoAgentsConfig"""
return AutoAgentsConfig(
main_instruction="Create a blog post about AI",
process_type="sequential",
agents=[
AgentConfig(
name="Research Agent",
role="Researcher",
goal="Research AI topics",
backstory="Expert researcher",
tools=["web_search"], # Use string tool names, not Mock objects
tasks=[
TaskConfig(
name="Research Task",
description="Research latest AI trends",
expected_output="Research summary",
tools=["web_search"] # Use string tool names, not Mock objects
)
]
),
AgentConfig(
name="Writer Agent",
role="Writer",
goal="Write blog post",
backstory="Professional writer",
tools=["text_editor"], # Use string tool names, not Mock objects
tasks=[
TaskConfig(
name="Writing Task",
description="Write blog post based on research",
expected_output="Complete blog post",
tools=["text_editor"] # Use string tool names, not Mock objects
)
]
)
]
)
@pytest.fixture
def sample_invalid_config_string_tasks(self):
"""Fixture providing an invalid config with string tasks"""
return {
"main_instruction": "Create a blog post about AI",
"process_type": "sequential",
"agents": [
{
"name": "Research Agent",
"role": "Researcher",
"goal": "Research AI topics",
"backstory": "Expert researcher",
"tools": ["web_search"],
"tasks": ["Research latest AI trends"] # Invalid: string instead of object
}
]
}
@pytest.fixture
def sample_invalid_config_missing_fields(self):
"""Fixture providing an invalid config with missing task fields"""
return AutoAgentsConfig(
main_instruction="Create a blog post about AI",
process_type="sequential",
agents=[
AgentConfig(
name="Research Agent",
role="Researcher",
goal="Research AI topics",
backstory="Expert researcher",
tools=["web_search"],
tasks=[
TaskConfig(
name="Research Task",
description="", # Missing description
expected_output="Research summary",
tools=["web_search"]
)
]
)
]
)
@pytest.fixture
def mock_tools(self):
"""Fixture providing mock tools"""
web_search = Mock()
web_search.__name__ = "web_search"
text_editor = Mock()
text_editor.__name__ = "text_editor"
return [web_search, text_editor]
def test_validate_config_success(self, sample_valid_config):
"""Test successful validation of a valid configuration"""
# Create AutoAgents instance without calling __init__ to avoid API calls
auto_agents = object.__new__(AutoAgents)
# Initialize only the necessary attributes for validation
auto_agents.instructions = "Test instructions"
auto_agents.max_agents = 3
is_valid, error_msg = auto_agents._validate_config(sample_valid_config)
assert is_valid is True
assert error_msg == ""
def test_validate_config_failure_not_taskconfig(self):
"""Test validation failure when task is not a TaskConfig instance"""
# Create AutoAgents instance without calling __init__ to avoid API calls
auto_agents = object.__new__(AutoAgents)
# Initialize only the necessary attributes for validation
auto_agents.instructions = "Test instructions"
auto_agents.max_agents = 3
# Create a valid config first, then modify it to bypass Pydantic validation
config = AutoAgentsConfig(
main_instruction="Test",
process_type="sequential",
agents=[
AgentConfig(
name="Agent",
role="Role",
goal="Goal",
backstory="Story",
tools=[],
tasks=[
TaskConfig(
name="Valid Task",
description="Valid description",
expected_output="Valid output",
tools=[]
)
]
)
]
)
# Now manually replace the task with a string to simulate the error condition
config.agents[0].tasks = ["This is a string task"]
is_valid, error_msg = auto_agents._validate_config(config)
assert is_valid is False
assert "is not a proper TaskConfig object" in error_msg
def test_validate_config_failure_missing_name(self):
"""Test validation failure when task is missing name"""
# Create AutoAgents instance without calling __init__ to avoid API calls
auto_agents = object.__new__(AutoAgents)
# Initialize only the necessary attributes for validation
auto_agents.instructions = "Test instructions"
auto_agents.max_agents = 3
config = AutoAgentsConfig(
main_instruction="Test",
process_type="sequential",
agents=[
AgentConfig(
name="Agent",
role="Role",
goal="Goal",
backstory="Story",
tools=[],
tasks=[
TaskConfig(
name="", # Empty name
description="Description",
expected_output="Output",
tools=[]
)
]
)
]
)
is_valid, error_msg = auto_agents._validate_config(config)
assert is_valid is False
assert "has no name" in error_msg
def test_validate_config_failure_missing_description(self, sample_invalid_config_missing_fields):
"""Test validation failure when task is missing description"""
# Create AutoAgents instance without calling __init__ to avoid API calls
auto_agents = object.__new__(AutoAgents)
# Initialize only the necessary attributes for validation
auto_agents.instructions = "Test instructions"
auto_agents.max_agents = 3
is_valid, error_msg = auto_agents._validate_config(sample_invalid_config_missing_fields)
assert is_valid is False
assert "has no description" in error_msg
@patch('praisonaiagents.llm.supports_structured_outputs')
@patch('praisonaiagents.agents.autoagents.get_openai_client')
def test_generate_config_openai_success(self, mock_get_client, mock_supports_structured, sample_valid_config, mock_tools):
"""Test successful config generation using OpenAI structured output"""
# Mock support for structured outputs
mock_supports_structured.return_value = True
# Mock OpenAI client
mock_client = Mock(spec=OpenAIClient)
mock_client.parse_structured_output.return_value = sample_valid_config
mock_get_client.return_value = mock_client
with patch.object(AutoAgents, '_create_agents_and_tasks') as mock_create, \
patch.object(AutoAgents, '_display_agents_and_tasks') as mock_display, \
patch('praisonaiagents.agents.agents.PraisonAIAgents.__init__') as mock_super_init:
# Mock the return value of _create_agents_and_tasks
mock_create.return_value = ([], []) # Empty agents and tasks lists
mock_super_init.return_value = None
auto_agents = AutoAgents(
instructions="Create a blog post about AI",
tools=mock_tools,
max_agents=2,
llm="gpt-4"
)
# Verify OpenAI client was called
mock_client.parse_structured_output.assert_called_once()
mock_create.assert_called_once_with(sample_valid_config)
@patch('praisonaiagents.llm.supports_structured_outputs')
@patch('praisonaiagents.agents.autoagents.LLM')
def test_generate_config_llm_success(self, mock_llm_class, mock_supports_structured, sample_valid_config, mock_tools):
"""Test successful config generation using generic LLM"""
# Mock no support for structured outputs to force LLM path
mock_supports_structured.return_value = False
# Mock LLM instance
mock_llm = Mock()
mock_llm.get_response.return_value = json.dumps(sample_valid_config.model_dump())
mock_llm_class.return_value = mock_llm
with patch.object(AutoAgents, '_create_agents_and_tasks') as mock_create, \
patch.object(AutoAgents, '_display_agents_and_tasks') as mock_display, \
patch('praisonaiagents.agents.agents.PraisonAIAgents.__init__') as mock_super_init:
# Mock the return value of _create_agents_and_tasks
mock_create.return_value = ([], []) # Empty agents and tasks lists
mock_super_init.return_value = None
auto_agents = AutoAgents(
instructions="Create a blog post about AI",
tools=mock_tools,
max_agents=2,
llm="claude-3" # Non-OpenAI model
)
# Verify LLM was called
mock_llm.get_response.assert_called_once()
mock_create.assert_called_once_with(sample_valid_config)
@patch('praisonaiagents.llm.supports_structured_outputs')
@patch('praisonaiagents.agents.autoagents.LLM')
def test_generate_config_with_markdown_response(self, mock_llm_class, mock_supports_structured, sample_valid_config, mock_tools):
"""Test config generation when LLM returns markdown-wrapped JSON"""
# Mock no support for structured outputs to force LLM path
mock_supports_structured.return_value = False
# Mock LLM instance
mock_llm = Mock()
# Return JSON wrapped in markdown code block
wrapped_response = f"```json\n{json.dumps(sample_valid_config.model_dump(), indent=2)}\n```"
mock_llm.get_response.return_value = wrapped_response
mock_llm_class.return_value = mock_llm
with patch.object(AutoAgents, '_create_agents_and_tasks') as mock_create, \
patch.object(AutoAgents, '_display_agents_and_tasks') as mock_display, \
patch('praisonaiagents.agents.agents.PraisonAIAgents.__init__') as mock_super_init:
# Mock the return value of _create_agents_and_tasks
mock_create.return_value = ([], []) # Empty agents and tasks lists
mock_super_init.return_value = None
auto_agents = AutoAgents(
instructions="Create a blog post about AI",
tools=mock_tools,
max_agents=2,
llm="claude-3"
)
# The config should have been generated successfully despite markdown wrapping
mock_create.assert_called_once_with(sample_valid_config)
@patch('praisonaiagents.llm.supports_structured_outputs')
@patch('praisonaiagents.agents.autoagents.get_openai_client')
def test_generate_config_retry_on_validation_failure(self, mock_get_client, mock_supports_structured, sample_valid_config, mock_tools):
"""Test retry mechanism when validation fails"""
# Mock support for structured outputs
mock_supports_structured.return_value = True
# Mock OpenAI client
mock_client = Mock(spec=OpenAIClient)
# First attempt: return invalid config with string tasks
invalid_config = AutoAgentsConfig(
main_instruction="Test",
process_type="sequential",
agents=[
AgentConfig(
name="Agent",
role="Role",
goal="Goal",
backstory="Story",
tools=[],
tasks=[] # This will be mocked to contain strings
)
]
)
# Mock the tasks to be strings (bypassing Pydantic validation)
invalid_config.agents[0].tasks = ["Invalid string task"]
# Second attempt: return valid config
mock_client.parse_structured_output.side_effect = [invalid_config, sample_valid_config]
mock_get_client.return_value = mock_client
with patch.object(AutoAgents, '_create_agents_and_tasks') as mock_create, \
patch.object(AutoAgents, '_display_agents_and_tasks') as mock_display, \
patch('praisonaiagents.agents.agents.PraisonAIAgents.__init__') as mock_super_init, \
patch('logging.warning') as mock_warning:
# Mock the return value of _create_agents_and_tasks
mock_create.return_value = ([], []) # Empty agents and tasks lists
mock_super_init.return_value = None
auto_agents = AutoAgents(
instructions="Create a blog post about AI",
tools=mock_tools,
max_agents=2,
llm="gpt-4"
)
# Verify retry occurred
assert mock_client.parse_structured_output.call_count == 2
mock_warning.assert_called_once()
assert "Configuration validation failed" in str(mock_warning.call_args)
mock_create.assert_called_once_with(sample_valid_config)
@patch('praisonaiagents.llm.supports_structured_outputs')
@patch('praisonaiagents.agents.autoagents.get_openai_client')
def test_generate_config_max_retries_exceeded(self, mock_get_client, mock_supports_structured, mock_tools):
"""Test that max retries are properly enforced"""
# Mock support for structured outputs
mock_supports_structured.return_value = True
# Mock OpenAI client
mock_client = Mock(spec=OpenAIClient)
# Always return invalid config
invalid_config = AutoAgentsConfig(
main_instruction="Test",
process_type="sequential",
agents=[
AgentConfig(
name="Agent",
role="Role",
goal="Goal",
backstory="Story",
tools=[],
tasks=[]
)
]
)
# Mock the tasks to be strings
invalid_config.agents[0].tasks = ["Invalid string task"]
mock_client.parse_structured_output.return_value = invalid_config
mock_get_client.return_value = mock_client
with pytest.raises(ValueError, match="Configuration validation failed after 3 attempts"):
AutoAgents(
instructions="Create a blog post about AI",
tools=mock_tools,
max_agents=2,
llm="gpt-4"
)
# Verify all retries were attempted
assert mock_client.parse_structured_output.call_count == 3
@patch('praisonaiagents.llm.supports_structured_outputs')
@patch('praisonaiagents.agents.autoagents.get_openai_client')
def test_max_agents_truncation(self, mock_get_client, mock_supports_structured, mock_tools):
"""Test that agents are truncated when exceeding max_agents"""
# Mock support for structured outputs
mock_supports_structured.return_value = True
# Create config with 4 agents
config_with_many_agents = AutoAgentsConfig(
main_instruction="Test",
process_type="sequential",
agents=[
AgentConfig(
name=f"Agent {i}",
role="Role",
goal="Goal",
backstory="Story",
tools=[],
tasks=[
TaskConfig(
name=f"Task {i}",
description="Description",
expected_output="Output",
tools=[]
)
]
)
for i in range(4)
]
)
mock_client = Mock(spec=OpenAIClient)
mock_client.parse_structured_output.return_value = config_with_many_agents
mock_get_client.return_value = mock_client
with patch.object(AutoAgents, '_create_agents_and_tasks') as mock_create, \
patch.object(AutoAgents, '_display_agents_and_tasks') as mock_display, \
patch('praisonaiagents.agents.agents.PraisonAIAgents.__init__') as mock_super_init:
# Mock the return value of _create_agents_and_tasks
mock_agents = [Mock() for _ in range(2)] # Only 2 agents after truncation
for i, agent in enumerate(mock_agents):
agent.name = f"Agent {i}"
mock_create.return_value = (mock_agents, [])
mock_super_init.return_value = None
# Create AutoAgents with max_agents=2
auto_agents = AutoAgents(
instructions="Test",
tools=mock_tools,
max_agents=2,
llm="gpt-4"
)
# Check that config was truncated before being passed to _create_agents_and_tasks
call_args = mock_create.call_args[0][0] # Get the config argument
assert len(call_args.agents) == 2
assert call_args.agents[0].name == "Agent 0"
assert call_args.agents[1].name == "Agent 1"
@patch('praisonaiagents.llm.supports_structured_outputs')
@patch('praisonaiagents.agents.autoagents.get_openai_client')
def test_insufficient_agents_warning(self, mock_get_client, mock_supports_structured, mock_tools):
"""Test warning when fewer agents than max_agents are generated"""
# Mock support for structured outputs
mock_supports_structured.return_value = True
# Create config with only 1 agent
config_with_few_agents = AutoAgentsConfig(
main_instruction="Test",
process_type="sequential",
agents=[
AgentConfig(
name="Single Agent",
role="Role",
goal="Goal",
backstory="Story",
tools=[],
tasks=[
TaskConfig(
name="Task",
description="Description",
expected_output="Output",
tools=[]
)
]
)
]
)
mock_client = Mock(spec=OpenAIClient)
mock_client.parse_structured_output.return_value = config_with_few_agents
mock_get_client.return_value = mock_client
with patch.object(AutoAgents, '_create_agents_and_tasks') as mock_create, \
patch.object(AutoAgents, '_display_agents_and_tasks') as mock_display, \
patch('praisonaiagents.agents.agents.PraisonAIAgents.__init__') as mock_super_init, \
patch('logging.warning') as mock_warning:
# Mock the return value of _create_agents_and_tasks
mock_agents = [Mock()]
mock_agents[0].name = "Single Agent"
mock_create.return_value = (mock_agents, [])
mock_super_init.return_value = None
auto_agents = AutoAgents(
instructions="Test",
tools=mock_tools,
max_agents=3,
llm="gpt-4"
)
# Verify warning was logged
mock_warning.assert_called()
warning_msg = str(mock_warning.call_args)
assert "Generated 1 agents, expected 3" in warning_msg
def test_max_agents_validation(self):
"""Test max_agents parameter validation"""
# Test too low
with pytest.raises(ValueError, match="max_agents must be at least 1"):
AutoAgents(instructions="Test", max_agents=0)
# Test too high
with pytest.raises(ValueError, match="max_agents cannot exceed 10"):
AutoAgents(instructions="Test", max_agents=11)
@patch('praisonaiagents.llm.supports_structured_outputs')
@patch('praisonaiagents.agents.autoagents.LLM')
def test_retry_with_previous_response_in_prompt(self, mock_llm_class, mock_supports_structured, sample_valid_config, mock_tools):
"""Test that retry includes previous response and error in prompt"""
# Mock no support for structured outputs to force LLM path
mock_supports_structured.return_value = False
# Mock LLM instance
mock_llm = Mock()
# Create an invalid config that will fail validation (not JSON parsing)
invalid_config = AutoAgentsConfig(
main_instruction="Test",
process_type="sequential",
agents=[
AgentConfig(
name="Agent",
role="Role",
goal="Goal",
backstory="Story",
tools=[],
tasks=[
TaskConfig(
name="", # Invalid: empty name
description="Description",
expected_output="Output",
tools=[]
)
]
)
]
)
# First response: invalid config that will fail validation
first_response = json.dumps(invalid_config.model_dump())
# Second response: valid config
second_response = json.dumps(sample_valid_config.model_dump())
mock_llm.get_response.side_effect = [first_response, second_response]
mock_llm_class.return_value = mock_llm
with patch.object(AutoAgents, '_create_agents_and_tasks') as mock_create, \
patch.object(AutoAgents, '_display_agents_and_tasks') as mock_display, \
patch('praisonaiagents.agents.agents.PraisonAIAgents.__init__') as mock_super_init:
# Mock the return value of _create_agents_and_tasks
mock_create.return_value = ([], []) # Empty agents and tasks lists
mock_super_init.return_value = None
auto_agents = AutoAgents(
instructions="Create a blog post about AI",
tools=mock_tools,
max_agents=2,
llm="claude-3"
)
# Check that the second call included the previous response
assert mock_llm.get_response.call_count == 2
second_call_kwargs = mock_llm.get_response.call_args_list[1][1]
assert "PREVIOUS ATTEMPT FAILED!" in second_call_kwargs['prompt']
assert first_response in second_call_kwargs['prompt']
assert "Error:" in second_call_kwargs['prompt']
mock_create.assert_called_once_with(sample_valid_config)
@patch('praisonaiagents.llm.supports_structured_outputs')
@patch('praisonaiagents.agents.autoagents.get_openai_client')
@patch('json.dumps')
def test_custom_api_key_and_base_url(self, mock_json_dumps, mock_get_client, mock_supports_structured, sample_valid_config):
"""Test that API key and base URL parameters are stored correctly"""
# Mock support for structured outputs
mock_supports_structured.return_value = True
# Mock JSON dumps to avoid serialization issues
mock_json_dumps.return_value = '{"mocked": "config"}'
# Mock OpenAI client instance
mock_client = Mock(spec=OpenAIClient)
mock_client.parse_structured_output.return_value = sample_valid_config
mock_get_client.return_value = mock_client
custom_api_key = "custom-api-key"
custom_base_url = "https://custom.api.url"
# Use simple tools that don't contain Mock objects
simple_tools = ["web_search", "text_editor"]
with patch.object(AutoAgents, '_create_agents_and_tasks') as mock_create, \
patch.object(AutoAgents, '_display_agents_and_tasks') as mock_display, \
patch('praisonaiagents.agents.agents.PraisonAIAgents.__init__') as mock_super_init:
# Mock the return value of _create_agents_and_tasks
mock_create.return_value = ([], []) # Empty agents and tasks lists
mock_super_init.return_value = None
auto_agents = AutoAgents(
instructions="Test",
tools=simple_tools,
max_agents=2,
llm="gpt-4",
api_key=custom_api_key,
base_url=custom_base_url
)
# Verify the parameters were stored correctly
assert auto_agents.api_key == custom_api_key
assert auto_agents.base_url == custom_base_url
# Verify get_openai_client was called (current behavior)
mock_get_client.assert_called_once()
@@ -0,0 +1,64 @@
"""
Simple test to verify that tool_choice='auto' is set for Gemini models
"""
import logging
from praisonaiagents.llm.llm import LLM
# Enable debug logging to see our log message
logging.basicConfig(level=logging.DEBUG, format='%(levelname)s: %(message)s')
# Test different Gemini model formats
test_models = [
"gemini/gemini-1.5-flash-8b",
"gemini-1.5-flash-8b",
"gemini/gemini-pro",
"gpt-4", # Non-Gemini model for comparison
]
# Mock tools
mock_tools = [
{
"type": "function",
"function": {
"name": "search",
"description": "Search for information",
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string"}
}
}
}
}
]
print("Testing tool_choice setting for different models:\n")
for model in test_models:
print(f"\nTesting model: {model}")
try:
llm = LLM(model=model)
params = llm._build_completion_params(
messages=[{"role": "user", "content": "test"}],
tools=mock_tools
)
tool_choice = params.get('tool_choice', 'NOT SET')
print(f" tool_choice: {tool_choice}")
# Verify behavior
if model.startswith(('gemini-', 'gemini/')):
if tool_choice == 'auto':
print(f" ✅ CORRECT: Gemini model has tool_choice='auto'")
else:
print(f" ❌ ERROR: Gemini model should have tool_choice='auto'")
else:
if tool_choice == 'NOT SET':
print(f" ✅ CORRECT: Non-Gemini model doesn't have tool_choice set")
else:
print(f" ⚠️ WARNING: Non-Gemini model has tool_choice set to '{tool_choice}'")
except Exception as e:
print(f" ❌ ERROR: {e}")
print("\nTest complete!")
@@ -0,0 +1,172 @@
# Tool Call Fix Documentation
## Issue
Agents using Gemini models (`gemini/gemini-1.5-flash-8b`) were not calling provided tools, instead responding with "I do not have access to the internet" when tasked with searching.
## Root Cause
The Gemini model through LiteLLM was not being properly instructed to use the available tools. The system prompt didn't mention the tools, and the tool_choice parameter wasn't being set. Additionally, async agents were not passing the correct tools to the system prompt generation.
## Fix Applied (Updated)
### 1. Enhanced System Prompt (agent.py) - IMPROVED
A new `_build_system_prompt` method was created to centralize system prompt generation with tool information. This ensures both sync and async agents get the same enhanced prompt:
```python
# New _build_system_prompt method with enhanced error handling and MCP tool support
def _build_system_prompt(self, tools=None):
"""Build the system prompt with tool information."""
if not self.use_system_prompt:
return None
system_prompt = f"""{self.backstory}\n
Your Role: {self.role}\n
Your Goal: {self.goal}"""
# Use provided tools or fall back to self.tools
tools_to_use = tools if tools is not None else self.tools
if tools_to_use:
tool_names = []
for tool in self.tools:
try:
if callable(tool) and hasattr(tool, '__name__'):
tool_names.append(tool.__name__)
elif isinstance(tool, dict) and isinstance(tool.get('function'), dict) and 'name' in tool['function']:
tool_names.append(tool['function']['name'])
elif isinstance(tool, str):
tool_names.append(tool)
elif hasattr(tool, "to_openai_tool"):
# Handle MCP tools
openai_tools = tool.to_openai_tool()
if isinstance(openai_tools, list):
for t in openai_tools:
if isinstance(t, dict) and 'function' in t and 'name' in t['function']:
tool_names.append(t['function']['name'])
elif isinstance(openai_tools, dict) and 'function' in openai_tools:
tool_names.append(openai_tools['function']['name'])
except (AttributeError, KeyError, TypeError) as e:
logging.warning(f"Could not extract tool name from {tool}: {e}")
continue
if tool_names:
system_prompt += f"\n\nYou have access to the following tools: {', '.join(tool_names)}. Use these tools when appropriate to help complete your tasks. Always use tools when they can help provide accurate information or perform actions."
```
### 2. Tool Choice Parameter (llm.py) - IMPROVED
For Gemini models, we now set `tool_choice='auto'` with better model detection and support checking:
```python
# In _build_completion_params method with enhanced model detection
if 'tools' in params and params['tools'] and 'tool_choice' not in params:
# More comprehensive Gemini model detection
if any(prefix in self.model.lower() for prefix in ['gemini', 'gemini/', 'google/gemini']):
try:
import litellm
# Check if model supports function calling before setting tool_choice
if litellm.supports_function_calling(model=self.model):
params['tool_choice'] = 'auto'
except Exception as e:
# If check fails, still set tool_choice for known Gemini models
logging.debug(f"Could not verify function calling support: {e}. Setting tool_choice anyway.")
params['tool_choice'] = 'auto'
```
### 3. Async Agent Fix (agent.py) - NEW
Fixed async agents to correctly pass tools for system prompt generation:
```python
# In sync chat method:
agent_tools=[t.__name__ if hasattr(t, '__name__') else str(t) for t in (tools if tools is not None else self.tools)]
# Fixed in async achat method (was always using self.tools):
agent_tools=[t.__name__ if hasattr(t, '__name__') else str(t) for t in (tools if tools is not None else self.tools)]
# Both sync and async now use enhanced system prompt:
system_prompt=self._build_system_prompt(tools)
```
## Testing the Fix
To test the fix, use the following code:
```python
import asyncio
from praisonaiagents import Agent, Task, PraisonAIAgents
# Define a simple tool
async def search_tool(query: str) -> str:
"""Search for information on the internet"""
return f"Search results for: {query}"
# Create agent with Gemini model
agent = Agent(
name="SearchAgent",
role="Information Researcher",
goal="Find accurate information using search tools",
backstory="Expert at finding and analyzing information",
tools=[search_tool],
llm={"model": "gemini/gemini-1.5-flash-8b"}
)
# Create task
task = Task(
description="Search for information about AI breakthroughs in 2024",
expected_output="Summary of AI breakthroughs",
agent=agent
)
# Run
async def test():
agents = PraisonAIAgents(agents=[agent], tasks=[task])
result = await agents.astart()
print(result)
asyncio.run(test())
```
## Improvements Made to Original PR
1. **Enhanced Error Handling**: Added try-catch blocks to prevent crashes from malformed tools
2. **MCP Tool Support**: Now properly extracts names from tools with `to_openai_tool` method
3. **Better Model Detection**: More comprehensive Gemini model detection including variants like 'google/gemini'
4. **Function Calling Support Check**: Uses litellm's `supports_function_calling` to verify model capabilities
5. **Type Safety**: Added isinstance checks to prevent TypeErrors when accessing nested dictionaries
6. **Async Agent Fix**: Fixed async agents to use the correct tools parameter instead of always using self.tools
7. **Centralized System Prompt**: Created `_build_system_prompt` method to ensure consistency between sync and async paths
## Backward Compatibility
- The fix only adds to existing functionality without modifying core behavior
- Tools continue to work exactly as before for all other models
- The system prompt enhancement only occurs when tools are present
- The tool_choice parameter is only added for Gemini models
- All error handling is non-breaking with appropriate logging
## Additional Recommendations
If issues persist with specific models:
1. **Explicit Tool Instructions in Task Description**:
```python
task = Task(
description="Use the tavily_search tool to find information about AI breakthroughs",
# ... rest of task config
)
```
2. **Use OpenAI Models for Tool-Heavy Tasks**:
OpenAI models (gpt-4, gpt-4o) have better native tool calling support.
3. **Debug Tool Registration**:
Enable debug logging to see tool registration:
```python
import logging
logging.basicConfig(level=logging.DEBUG)
```
## Verification Steps
1. Check that tools are properly formatted by the agent
2. Verify the system prompt includes tool instructions
3. Confirm tool_choice is set for Gemini models
4. Monitor LLM responses for tool_calls in the response
The fix ensures that Gemini models are properly instructed to use available tools, resolving the issue where agents would claim they don't have internet access despite having search tools available.
@@ -0,0 +1,89 @@
"""
Test example demonstrating the tool call fix for Gemini models.
This example shows how agents now properly use tools instead of saying
"I do not have access to the internet".
"""
import logging
from praisonaiagents import Agent, Task, PraisonAIAgents
# Enable debug logging to see tool processing
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
# Define a simple search tool (synchronous version)
def mock_search(query: str) -> str:
"""Search for information on the internet.
Args:
query: The search query string
Returns:
Mock search results for the query
"""
return f"Mock search results for '{query}': Found 10 relevant articles about {query}. Top result: Latest developments and breakthroughs in this field..."
# Create agent with Gemini model and the search tool
search_agent = Agent(
name="SearchAgent",
role="Information Researcher",
goal="Find accurate information using the mock_search tool",
backstory="Expert researcher skilled at finding and analyzing information from various sources",
tools=[mock_search],
llm={"model": "gemini/gemini-1.5-flash-8b"},
verbose=True
)
# Create a task that should trigger tool usage
search_task = Task(
name="search_ai_breakthroughs",
description="Search for information about latest AI breakthroughs in 2024",
expected_output="A comprehensive summary of AI breakthroughs found through search",
agent=search_agent
)
def test_tool_usage():
"""Test that the agent uses tools instead of saying it has no internet access."""
print("=" * 60)
print("Testing Tool Usage with Gemini Model")
print("=" * 60)
# Create workflow
workflow = PraisonAIAgents(
agents=[search_agent],
tasks=[search_task],
verbose=True
)
# Execute the workflow
print("\nExecuting task...")
result = workflow.start()
# Check the result
print("\n" + "=" * 60)
print("RESULT:")
print("=" * 60)
if isinstance(result, dict) and 'task_results' in result:
task_result = result['task_results'][0]
print(f"Task Output: {task_result}")
# Check if the agent used the tool or claimed no access
if "do not have access" in str(task_result).lower():
print("\n❌ FAILED: Agent still claims no internet access")
elif "mock search results" in str(task_result).lower():
print("\n✅ SUCCESS: Agent used the search tool!")
else:
print("\n⚠️ UNCLEAR: Check if agent used the tool properly")
else:
print(f"Result: {result}")
print("\n" + "=" * 60)
print("Test Complete")
print("=" * 60)
if __name__ == "__main__":
# Run the test
test_tool_usage()
print("\n\nNOTE: With the fix applied, the agent should use the mock_search tool")
print("instead of saying 'I do not have access to the internet'.")
@@ -0,0 +1,132 @@
"""
Enhanced test example demonstrating the improved tool call fix for Gemini models.
This test includes edge cases and error handling scenarios.
"""
import logging
from praisonaiagents import Agent, Task, PraisonAIAgents
# Enable info logging
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
# Test different tool formats
# 1. Simple function tool
def search_web(query: str) -> str:
"""Search the web for information."""
return f"Web search results for '{query}': Found relevant information."
# 2. Dictionary format tool (OpenAI style)
dict_tool = {
"type": "function",
"function": {
"name": "calculate",
"description": "Perform mathematical calculations",
"parameters": {
"type": "object",
"properties": {
"expression": {
"type": "string",
"description": "Mathematical expression to evaluate"
}
},
"required": ["expression"]
}
}
}
# 3. String tool name
string_tool = "weather_tool"
# 4. Mock MCP tool class
class MockMCPTool:
def to_openai_tool(self):
return {
"type": "function",
"function": {
"name": "mcp_search",
"description": "MCP-based search tool",
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string"}
}
}
}
}
# Create test agent with various tool formats
test_agent = Agent(
name="MultiToolAgent",
role="Versatile Assistant",
goal="Test various tool formats and edge cases",
backstory="Expert at using different types of tools",
tools=[search_web, dict_tool, string_tool, MockMCPTool()],
llm={"model": "gemini/gemini-1.5-flash-8b"},
verbose=True
)
# Create test task
test_task = Task(
name="test_tools",
description="Search for information about Python programming best practices",
expected_output="A summary of Python best practices found through search",
agent=test_agent
)
def test_improved_implementation():
"""Test the improved tool usage implementation."""
print("=" * 80)
print("Testing Improved Tool Usage with Various Tool Formats")
print("=" * 80)
try:
# Create workflow
workflow = PraisonAIAgents(
agents=[test_agent],
tasks=[test_task],
verbose=True
)
# Execute
print("\nExecuting task with multiple tool formats...")
result = workflow.start()
# Analyze result
print("\n" + "=" * 80)
print("RESULT ANALYSIS:")
print("=" * 80)
if isinstance(result, dict) and 'task_results' in result:
task_result = result['task_results'][0]
result_str = str(task_result).lower()
# Check various failure/success indicators
if "do not have access" in result_str:
print("❌ FAILED: Agent claims no access to tools")
elif any(tool_indicator in result_str for tool_indicator in ["search", "results", "found", "web"]):
print("✅ SUCCESS: Agent appears to have used tools!")
else:
print("⚠️ UNCLEAR: Cannot determine if tools were used")
print(f"\nFull Result: {task_result}")
else:
print(f"Unexpected result format: {result}")
except Exception as e:
print(f"❌ ERROR during execution: {e}")
import traceback
traceback.print_exc()
print("\n" + "=" * 80)
print("Test Complete")
print("=" * 80)
if __name__ == "__main__":
test_improved_implementation()
print("\n\nNOTE: This test verifies the improved implementation handles:")
print("1. Function tools")
print("2. Dictionary format tools")
print("3. String tool names")
print("4. MCP-style tools with to_openai_tool method")
print("5. Error handling for malformed tools")
@@ -0,0 +1,256 @@
"""Unit tests for validation feedback in workflow retry logic"""
import pytest
from unittest.mock import Mock, patch
from praisonaiagents import Agent, Task, TaskOutput
from praisonaiagents.process import Process
class TestValidationFeedback:
"""Test validation feedback functionality"""
def test_task_has_validation_feedback_field(self):
"""Test that Task class has validation_feedback field"""
task = Task(
name="test_task",
description="Test task",
expected_output="Test output"
)
assert hasattr(task, 'validation_feedback')
assert task.validation_feedback is None
def test_validation_feedback_captured_on_invalid_decision(self):
"""Test that validation feedback is captured when decision is invalid"""
# Create mock agents
agent1 = Mock(spec=Agent)
agent2 = Mock(spec=Agent)
# Create tasks
collect_task = Task(
name="collect_data",
description="Collect data",
expected_output="Data",
agent=agent1,
is_start=True,
next_tasks=["validate_data"]
)
validate_task = Task(
name="validate_data",
description="Validate data",
expected_output="Validation result",
agent=agent2,
task_type="decision",
condition={
"valid": [],
"invalid": ["collect_data"]
}
)
# Set up task relationships
validate_task.previous_tasks = ["collect_data"]
# Create process
process = Process(
agents={"agent1": agent1, "agent2": agent2},
tasks={"collect_data": collect_task, "validate_data": validate_task},
verbose=0
)
# Simulate task execution results
collect_task.result = TaskOutput(
raw="Collected only 3 items",
description="Collected data items",
agent="agent1"
)
collect_task.status = "completed"
validate_task.result = TaskOutput(
raw="Not enough results, need at least 10",
description="Validation result",
agent="agent2"
)
validate_task.status = "completed"
# Simulate workflow routing with invalid decision
# This would normally happen in the workflow execution
decision_str = "invalid"
current_task = validate_task
target_tasks = current_task.condition.get(decision_str, [])
if target_tasks:
task_value = target_tasks[0]
next_task = collect_task # This is the retry task
# The implementation should add validation feedback
if decision_str in Process.VALIDATION_FAILURE_DECISIONS:
if current_task and current_task.result:
validated_task = collect_task # The task that was validated
feedback = {
'validation_response': decision_str,
'validation_details': current_task.result.raw,
'rejected_output': validated_task.result.raw if validated_task and validated_task.result else None,
'validator_task': current_task.name
}
next_task.validation_feedback = feedback
# Verify feedback was captured
assert collect_task.validation_feedback is not None
assert collect_task.validation_feedback['validation_response'] == 'invalid'
assert collect_task.validation_feedback['validation_details'] == "Not enough results, need at least 10"
assert collect_task.validation_feedback['rejected_output'] == "Collected only 3 items"
assert collect_task.validation_feedback['validator_task'] == "validate_data"
def test_validation_feedback_included_in_context(self):
"""Test that validation feedback is included in task context"""
# Create a task with validation feedback
task = Task(
name="retry_task",
description="Original description",
expected_output="Expected output"
)
task.validation_feedback = {
'validation_response': 'invalid',
'validation_details': 'Not enough results',
'rejected_output': 'Previous output',
'validator_task': 'validator'
}
# Create process to test context building
process = Process(
agents={},
tasks={"retry_task": task},
verbose=0
)
# Build context
context = process._build_task_context(task)
# Verify feedback is in context
assert "Previous attempt failed validation with reason: invalid" in context
assert "Validation feedback: Not enough results" in context
assert "Rejected output: Previous output" in context
assert "Please try again with a different approach based on this feedback." in context
# Verify feedback was cleared after use
assert task.validation_feedback is None
def test_validation_feedback_backward_compatibility(self):
"""Test that tasks without validation feedback work normally"""
task = Task(
name="normal_task",
description="Normal task",
expected_output="Normal output"
)
process = Process(
agents={},
tasks={"normal_task": task},
verbose=0
)
# Build context without validation feedback
context = process._build_task_context(task)
# Should return empty string for task with no context
assert context == ""
# Task should still work normally
assert task.validation_feedback is None
def test_multiple_retry_decisions_supported(self):
"""Test that various failure decision strings trigger feedback capture"""
# Import the constant from Process class
from praisonaiagents.process import Process
failure_decisions = Process.VALIDATION_FAILURE_DECISIONS
for decision in failure_decisions:
task = Task(
name="test_task",
description="Test",
expected_output="Test"
)
# Simulate the decision routing logic
if decision in Process.VALIDATION_FAILURE_DECISIONS:
task.validation_feedback = {
'validation_response': decision,
'validation_details': f"Failed with {decision}",
'rejected_output': "Previous output",
'validator_task': "validator"
}
assert task.validation_feedback is not None
assert task.validation_feedback['validation_response'] == decision
def test_validation_feedback_with_context_tasks(self):
"""Test that validation feedback works when validated task is in context"""
# Create mock agents
agent1 = Mock(spec=Agent)
agent2 = Mock(spec=Agent)
# Create tasks
data_task = Task(
name="data_task",
description="Generate data",
expected_output="Data",
agent=agent1
)
validate_task = Task(
name="validate_data",
description="Validate data",
expected_output="Validation result",
agent=agent2,
task_type="decision",
context=[data_task], # Data task is in context, not previous_tasks
condition={
"valid": [],
"invalid": ["data_task"]
}
)
# Create process
process = Process(
agents={"agent1": agent1, "agent2": agent2},
tasks={"data_task": data_task, "validate_data": validate_task},
verbose=0
)
# Simulate task execution results
data_task.result = TaskOutput(
raw="Generated data with errors",
description="Generated data",
agent="agent1"
)
data_task.status = "completed"
validate_task.result = TaskOutput(
raw="Data has errors, please fix",
description="Data validation result",
agent="agent2"
)
validate_task.status = "completed"
# Test the improved validation feedback logic
decision_str = "invalid"
current_task = validate_task
# This simulates the improved logic that checks context when no previous_tasks
validated_task = None
if current_task.previous_tasks:
prev_task_name = current_task.previous_tasks[-1]
validated_task = data_task if data_task.name == prev_task_name else None
elif current_task.context:
# Check context for the validated task
for ctx_task in reversed(current_task.context):
if ctx_task.result and ctx_task.name != current_task.name:
validated_task = ctx_task
break
# Verify the context-based task was found
assert validated_task is not None
assert validated_task.name == "data_task"
assert validated_task.result.raw == "Generated data with errors"
+2585 -2582
View File
File diff suppressed because it is too large. Load diff
+201
View File
@@ -0,0 +1,201 @@
#!/usr/bin/env python3
"""
Test script to verify the duplicate callback fix for issue #878
This tests that display_interaction is only called once per LLM response
"""
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'src/praisonai-agents'))
from praisonaiagents.llm.llm import LLM
from unittest.mock import patch, MagicMock
import json
# Track display_interaction calls
display_calls = []
def mock_display_interaction(prompt, response, markdown=True, generation_time=0, console=None):
"""Mock display_interaction to track calls"""
display_calls.append({
'prompt': prompt,
'response': response,
'markdown': markdown,
'generation_time': generation_time
})
print(f"[DISPLAY] {prompt[:50]}... -> {response[:50]}...")
def test_single_display_no_tools():
"""Test that display_interaction is called only once without tools"""
global display_calls
display_calls = []
with patch('praisonaiagents.main.display_interaction', side_effect=mock_display_interaction):
with patch('litellm.completion') as mock_completion:
# Mock streaming response
mock_completion.return_value = [
MagicMock(choices=[MagicMock(delta=MagicMock(content="Hello"))]),
MagicMock(choices=[MagicMock(delta=MagicMock(content=" world"))]),
MagicMock(choices=[MagicMock(delta=MagicMock(content="!"))])
]
llm = LLM(model="gpt-4o-mini", verbose=False)
response = llm.get_response(
prompt="Test prompt",
verbose=True,
stream=True
)
print(f"\nResponse: {response}")
print(f"Display calls: {len(display_calls)}")
assert len(display_calls) == 1, f"Expected 1 display call, got {len(display_calls)}"
assert response == "Hello world!"
def test_single_display_with_reasoning():
"""Test that display_interaction is called only once with reasoning steps"""
global display_calls
display_calls = []
with patch('praisonaiagents.main.display_interaction', side_effect=mock_display_interaction):
with patch('litellm.completion') as mock_completion:
# Mock non-streaming response with reasoning
mock_completion.return_value = {
"choices": [{
"message": {
"content": "The answer is 42",
"provider_specific_fields": {
"reasoning_content": "Let me think about this..."
}
}
}]
}
llm = LLM(model="o1-preview", verbose=False, reasoning_steps=True)
response = llm.get_response(
prompt="What is the meaning of life?",
verbose=True,
stream=False,
reasoning_steps=True
)
print(f"\nResponse: {response}")
print(f"Display calls: {len(display_calls)}")
assert len(display_calls) == 1, f"Expected 1 display call, got {len(display_calls)}"
def test_single_display_with_self_reflection():
"""Test that display_interaction is called appropriately with self-reflection"""
global display_calls
display_calls = []
with patch('praisonaiagents.main.display_interaction', side_effect=mock_display_interaction):
with patch('praisonaiagents.main.display_self_reflection'):
with patch('litellm.completion') as mock_completion:
# First call - initial response
# Second call - reflection
# Third call - regenerated response
call_count = 0
def mock_streaming(*args, **kwargs):
nonlocal call_count
call_count += 1
if call_count == 1:
# Initial response
return [
MagicMock(choices=[MagicMock(delta=MagicMock(content="Initial"))]),
MagicMock(choices=[MagicMock(delta=MagicMock(content=" response"))])
]
elif call_count == 2:
# Reflection
reflection = {"reflection": "Could be better", "satisfactory": "no"}
return [
MagicMock(choices=[MagicMock(delta=MagicMock(content=json.dumps(reflection)))])
]
else:
# Final response
return [
MagicMock(choices=[MagicMock(delta=MagicMock(content="Better"))]),
MagicMock(choices=[MagicMock(delta=MagicMock(content=" response"))])
]
mock_completion.side_effect = mock_streaming
llm = LLM(model="gpt-4o-mini", verbose=False, self_reflect=True, min_reflect=1, max_reflect=2)
response = llm.get_response(
prompt="Test prompt",
verbose=True,
stream=True,
self_reflect=True,
min_reflect=1,
max_reflect=2
)
print(f"\nResponse: {response}")
print(f"Display calls: {len(display_calls)}")
# Should display only the final response
assert len(display_calls) == 1, f"Expected 1 display call, got {len(display_calls)}"
assert response == "Better response"
def test_async_single_display():
"""Test async version also prevents duplicate displays"""
global display_calls
display_calls = []
import asyncio
async def run_test():
with patch('praisonaiagents.main.display_interaction', side_effect=mock_display_interaction):
with patch('litellm.acompletion') as mock_acompletion:
# Mock async streaming response
async def async_generator():
yield MagicMock(choices=[MagicMock(delta=MagicMock(content="Async"))])
yield MagicMock(choices=[MagicMock(delta=MagicMock(content=" response"))])
mock_acompletion.return_value = async_generator()
llm = LLM(model="gpt-4o-mini", verbose=False)
response = await llm.get_response_async(
prompt="Test async",
verbose=True,
stream=True
)
print(f"\nAsync Response: {response}")
print(f"Display calls: {len(display_calls)}")
assert len(display_calls) == 1, f"Expected 1 display call, got {len(display_calls)}"
assert response == "Async response"
asyncio.run(run_test())
if __name__ == "__main__":
print("Testing duplicate callback fix for issue #878...\n")
try:
print("1. Testing single display without tools...")
test_single_display_no_tools()
print("✓ PASSED\n")
print("2. Testing single display with reasoning...")
test_single_display_with_reasoning()
print("✓ PASSED\n")
print("3. Testing single display with self-reflection...")
test_single_display_with_self_reflection()
print("✓ PASSED\n")
print("4. Testing async single display...")
test_async_single_display()
print("✓ PASSED\n")
print("All tests passed! The duplicate callback issue is fixed.")
except AssertionError as e:
print(f"✗ FAILED: {e}")
sys.exit(1)
except Exception as e:
print(f"✗ ERROR: {e}")
import traceback
traceback.print_exc()
sys.exit(1)