Compare commits

...
92 Commits
Author SHA1 Message Date
Mervin Praison fca801c3f7 chore: remove test files for display generating and streaming fixes 2025-07-23 23:57:08 +01:00
Mervin Praison 33c81861c0 feat: implement comprehensive agent monitoring system with streaming enhancements 2025-07-23 23:56:19 +01:00
Mervin Praison b49b6d567f Merge pull request #1050 from MervinPraison/claude/pr-1049-20250723-0902
fix: Remove display_generating when stream=false to prevent streaming-like behavior
2025-07-23 10:45:49 +01:00
github-actions[bot]andMervin Praison 8c1677556c fix: Complete display_generating fix for consistency across all code paths
- Fix inconsistency in custom LLM path (line 1116) to match OpenAI path behavior
- Update test script to verify all three fix locations (2 OpenAI + 1 custom LLM)
- Ensure no streaming-like behavior when stream=False regardless of LLM provider
- Maintain backward compatibility and existing functionality

Co-authored-by: Mervin Praison <[email protected]>
2025-07-23 09:34:50 +00:00
github-actions[bot]andMervin Praison a656ea2bca fix: Remove display_generating when stream=false to prevent streaming-like behavior
When stream=false but verbose=true, the system was incorrectly showing
streaming-like visual behavior (Generating... X.Xs) due to display_generating
being used even when users explicitly set stream=false.

Fixed by changing display_fn from display_generating to None when stream=false
in both locations (lines 1073 and 1172) in agent.py.

This ensures true non-streaming behavior when stream=false is set, regardless
of verbose setting.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-23 09:07:13 +00:00
Mervin Praison 02846b0efd Merge pull request #1049 from MervinPraison/claude/pr-1048-20250723-0615
Fix: Add display_generating support for OpenAI non-streaming mode
2025-07-23 07:30:21 +01:00
github-actions[bot]andMervin Praison 8d463e6897 Improve display_generating fix for OpenAI non-streaming mode
- Add missing display_fn support to async version (achat_completion_with_tools)
- Add proper error handling with try-catch around Live context manager
- Add safer content access with length checks for choices array
- Add console cleanup (console.print()) consistent with streaming mode
- Maintain backward compatibility with fallback to regular completion

Co-authored-by: Mervin Praison <[email protected]>
2025-07-23 06:23:50 +00:00
github-actions[bot]andClaude b47021d98f Fix: Add display_generating support for OpenAI non-streaming mode
🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
2025-07-23 06:19:44 +00:00
Mervin Praison ab4561d7a4 Merge pull request #1048 from MervinPraison/claude/pr-1046-20250722-2152
PR #1046: Changes from Claude
2025-07-22 23:19:32 +01:00
Mervin Praison 81af51f3e7 Merge pull request #1047 from MervinPraison/claude/pr-1045-20250722-2138
fix: resolve display_generating issue by ensuring stream parameter is correctly passed
2025-07-22 23:02:46 +01:00
github-actions[bot]andMervin Praison 550d9c30d4 fix: restore display_generating visibility in verbose streaming mode
- Modified streaming logic to show display_generating during verbose streaming
- Preserves non-verbose streaming behavior (no display_generating)
- Fixes user-reported issue where display_generating disappeared in basic-agents.py
- Maintains full backward compatibility with existing code
- Addresses display functionality regression from streaming fallback changes

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 21:59:47 +00:00
Mervin Praison 39a8942054 Merge pull request #1046 from MervinPraison/claude/fix-streaming-fallback-logic-20250722
fix: implement proper streaming fallback logic for JSON parsing errors
2025-07-22 22:46:55 +01:00
github-actions[bot]andMervin Praison 6ecb6e4887 fix: prevent duplicate API calls in streaming fallback logic
- Added fallback_completed flag to track successful fallback execution
- Modified non-streaming condition to only execute when fallback hasn''t completed
- Prevents duplicate litellm.completion calls that waste API quota and increase latency
- Addresses critical bug identified by Cursor bot review

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 21:43:35 +00:00
github-actions[bot]andMervin Praison d64c1381ed fix: resolve display_generating issue by ensuring stream parameter is correctly passed from start() to chat() method
- Fix stream parameter not being explicitly passed in start() method
- Ensures display_generating shows when agent.start() is called with default settings
- Agent defaults: verbose=True, stream=False should show ''Generating...'' message
- Resolves issue where chat() method default stream=True overrode agent''s stream=False setting
- Tested logic flow: (not stream and verbose) now correctly evaluates to True

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 21:43:19 +00:00
Mervin Praison 8e6bc3845b Merge pull request #1045 from MervinPraison/claude/pr-1042-20250722-2118
fix: correct display_generating logic to only show when stream=False AND verbose=True
2025-07-22 22:36:55 +01:00
github-actions[bot]andMervin Praison d2b2ad525e fix: implement proper streaming fallback logic for JSON parsing errors
- Fixed critical control flow bug where streaming errors set use_streaming=False
  but continued executing in streaming block instead of triggering fallback
- When recoverable streaming error occurs, immediately perform actual non-streaming
  API call instead of creating mock response with partial/empty data
- Added proper callback handling for both verbose/non-verbose modes
- Enhanced graceful degradation if both streaming and non-streaming fail
- Updated test to use agent.llm_model instead of agent.llm property

Fixes: Issue #1043 streaming JSON parsing errors causing "Response: None"

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Mervin Praison <[email protected]>
2025-07-22 21:35:49 +00:00
github-actions[bot]andMervin Praison 60508ba410 fix: correct hardcoded True to stream variable in display_generating logic
Fixed critical bug where (not True and self.verbose) was always False,
preventing display_generating from ever being called in one code path.
Changed to (not stream and self.verbose) to match intended logic.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 21:30:16 +00:00
Mervin Praison a2951184d7 Merge pull request #1044 from MervinPraison/claude/issue-1043-20250722-2015
fix: enhance LiteLLM streaming error handling for JSON parsing errors (Issue #1043)
2025-07-22 22:25:13 +01:00
github-actions[bot]andMervin Praison 4e26bbecfb fix: correct display_generating logic to only show when stream=False AND verbose=True
- Change logic from (stream or verbose) to (not stream and verbose)
- Ensure display_generating only appears for non-streaming verbose mode
- Remove streaming display blocks since no display needed when streaming
- Update test files to reflect correct expected behavior

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 21:24:01 +00:00
github-actions[bot]andMervin Praison f77583c64f fix: resolve critical fallback logic and display_generating behavior
- Fix broken streaming fallback: now properly calls non-streaming LLM instead of creating mock responses
- Update display_generating behavior per requirements:
  * stream=true (any verbose): no display_generating
  * stream=false + verbose=true: show display_generating
  * stream=false + verbose=false: no display_generating
- Clean up unnecessary f-strings without placeholders
- Maintain backward compatibility and existing functionality

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 21:23:42 +00:00
github-actions[bot]andMervin Praison 6071850401 fix: enhance LiteLLM streaming error handling for JSON parsing errors
- Add iterator-level error handling in get_response_stream method
- Wrap for chunk in stream_iterator loops with JSON parsing error recovery
- Add error handling to both verbose/non-verbose streaming in get_response
- Leverage existing _is_streaming_error_recoverable() logic for consistency
- Maintain automatic fallback to non-streaming mode on repeated errors
- Preserve backward compatibility with no breaking API changes
- Add comprehensive test script for Issue #1043 scenario

Fixes #1043: Error parsing chunk - JSON parsing error in Gemini streaming

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 20:23:51 +00:00
Mervin Praison 6ce4363e92 Merge pull request #1042 from MervinPraison/claude/issue-981-20250722-1946
fix: ensure display_generating is called when verbose=True regardless of streaming mode
2025-07-22 21:13:02 +01:00
github-actions[bot]andMervin Praison 3622838f9f fix: complete display_generating fix for both OpenAI and custom LLM paths
- Extend fix to custom LLM providers (Anthropic, Groq, local models)
- Add Live display wrapper for non-streaming custom LLM path
- Ensure consistent behavior: display when (stream OR verbose)
- Maintain backward compatibility for all existing functionality
- Add comprehensive test coverage for both agent paths

Fixes #981 completely for all LLM provider types

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 20:07:35 +00:00
github-actions[bot]andMervin Praison 60f0cb308a fix: ensure display_generating is called when verbose=True regardless of streaming mode
- Fix issue where display_generating was only called during streaming
- Now display_generating is called when either stream=True OR verbose=True
- Resolves user issue where verbose=True with default stream=False wasn''t showing ''Generating...'' display
- Add logic tests to verify the fix

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 19:49:49 +00:00
Mervin Praison a3afd4f7fd Merge pull request #1041 from MervinPraison/claude/pr-1039-20250722-1308
PR #1039: Logging
2025-07-22 20:42:51 +01:00
github-actions[bot]andMervin Praison 7118b9a47b perf: optimize litellm logger suppression using hierarchical logging
Remove redundant loops that iterate over all loggers to suppress litellm
submodules. Python''s hierarchical logging automatically propagates the
CRITICAL level from parent ''litellm'' logger to all child loggers
(litellm.utils, litellm.proxy, etc.).

Changes:
- Remove 3 redundant loops in __init__.py and llm/__init__.py
- Remove explicit configuration of child loggers (litellm.utils, etc.)
- Maintain litellm_logging as separate hierarchy (not a child of litellm)
- Add explanatory comments about hierarchical logging inheritance
- Preserve backward compatibility and DEBUG mode behavior

Performance: Eliminates unnecessary iteration over all existing loggers
while maintaining identical logging suppression behavior.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 19:35:02 +00:00
Mervin Praison 3fa0173bf8 refactor: reorganize test files and add MongoDB memory/knowledge integrations 2025-07-22 14:44:51 +01:00
Mervin Praison b9ea6accf9 Merge pull request #1040 from MervinPraison/claude/pr-1038-20250722-1255
PR #1038: Monitoring examples
2025-07-22 14:43:58 +01:00
github-actions[bot]andClaude 80d2491a59 fix: resolve critical issues in monitoring examples
- Fix ZeroDivisionError in 02_multi_agent_workflow_monitoring.py by adding zero-check for call counts
- Improve async handling in 04_async_agent_monitoring.py with better simulation and timing
- Fix UnboundLocalError in 05_error_handling_monitoring.py by correcting variable references
- Add zero-check protection in 07_hierarchical_agents_monitoring.py to prevent division errors
- Remove unused json import from 08_custom_tools_monitoring.py
- Remove unused timedelta import from 09_streaming_monitoring.py
- Fix UnboundLocalError in 10_comprehensive_dashboard.py by initializing avg_function_time variable

🤖 Generated with Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
2025-07-22 13:33:04 +00:00
github-actions[bot]andMervin Praison 52f23b87c9 fix: Remove logger disabling to allow DEBUG mode re-enabling
- Remove `disabled = True` from logger configuration to prevent permanent disabling
- Keep CRITICAL level suppression but allow dynamic re-enabling in DEBUG mode
- Maintain backward compatibility while fixing DEBUG mode functionality
- Add comprehensive test to verify dynamic mode switching works correctly
- Addresses reviewer concerns about permanent logger disabling

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 13:24:56 +00:00
github-actions[bot]andMervin Praison 70a3d2df34 feat: Add 10 comprehensive performance monitoring examples
- Create examples/python/monitoring/ directory with 10 diverse examples
- Cover basic monitoring, multi-agent workflows, tools, async, errors, memory, hierarchical, custom tools, streaming, and comprehensive dashboard
- Each example demonstrates different aspects of the performance monitoring system
- Include detailed README with usage patterns and best practices
- Show practical integration of @monitor_function and track_api_call
- Demonstrate performance analytics, reporting, and optimization insights

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 13:04:17 +00:00
Mervin Praison d3c08660f5 Merge pull request #1039 from MervinPraison/claude/fix-litellm-warnings-20250722-1151
Fix: Comprehensive LiteLLM deprecation warning suppression
2025-07-22 13:58:24 +01:00
github-actions[bot]andMervin Praison 4e09b7c878 fix: clean up warning suppression implementation
- Fix LLM module _should_suppress_warnings() consistency by adding missing test detection
- Remove redundant warning filters in main module (lines 74-76 duplicated lines 63-65)
- Remove duplicate sys import in main module
- All tests now pass: DEBUG mode properly respects LOGLEVEL=DEBUG
- Maintains backward compatibility and existing functionality

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 12:54:07 +00:00
Mervin Praison 6bc6ff79dd Merge pull request #1038 from MervinPraison/claude/issue-1035-20250722-1143
feat: Add comprehensive performance monitoring system
2025-07-22 13:44:08 +01:00
github-actions[bot]andMervin Praison ab78189601 fix: implement DEBUG mode support for warning suppression (issue #1033)
- Implement missing DEBUG mode functionality as specified in PR description
- Add _should_suppress_warnings() function to respect LOGLEVEL=DEBUG
- Fix overly broad global warning suppression (now module-specific)
- Remove unsafe pydantic.warnings.warn monkey-patching
- Improve test detection with pytest module checks
- Add comprehensive test scripts for validation

Key improvements:
• Warnings are suppressed in normal mode (LOGLEVEL=INFO)
• Warnings are shown when LOGLEVEL=DEBUG (as per PR requirement)
• More targeted warning suppression (module-specific vs global)
• Safer pydantic warning handling with specific filters
• Better backward compatibility and security

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Mervin Praison <[email protected]>
2025-07-22 12:35:18 +00:00
github-actions[bot]andMervin Praison 54c29a6ffe fix: address code quality issues in performance monitoring
- Remove unused imports (inspect, timedelta, time, deque, etc.)
- Fix division by zero error in performance trend analysis
- Improve call chain building algorithm for better efficiency and accuracy
- Replace bare except clauses with specific Exception handling
- Use collections.deque with maxlen instead of manual truncation
- Remove unused variables and loop variables
- Fix f-strings without placeholders
- Replace magic numbers with named constants
- Improve error handling specificity

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 12:33:33 +00:00
Mervin Praison afb32705dc Merge pull request #1037 from MervinPraison/claude/pr-1030-20250722-1138
fix: correct tool call argument parsing in streaming mode
2025-07-22 13:23:45 +01:00
github-actions[bot]andMervin Praison 9f9a373c31 feat: Add comprehensive performance monitoring system for function and API call analysis
This implementation addresses all requirements from Issue #1035:

Features Added:
- PerformanceMonitor class for function performance evaluation with detailed statistics
- FunctionFlowAnalyzer for execution flow visualization and bottleneck identification
- API call tracking and performance comparison (fast vs slow calls)
- Real-time performance reporting and comprehensive analytics
- CLI interface for easy command-line access to performance data
- Full backward compatibility with existing telemetry system

Key Capabilities:
1. Function Performance Evaluation: Decorators and context managers for automatic timing
2. Function Flow Analysis: Visual execution flow with bottleneck detection
3. Execution Speed Analysis: Min/max/average timing with trend analysis
4. API Call Performance: Success rates, response times, reliability metrics
5. API Call Visibility: Complete tracking and history of all API interactions

Files Modified:
- Enhanced telemetry/__init__.py to expose new performance tools
- Added performance_monitor.py with core monitoring functionality
- Added performance_utils.py with advanced analysis and visualization
- Added performance_cli.py for command-line interface
- Added comprehensive demo and examples

Integration:
- Seamless integration with existing PraisonAI telemetry infrastructure
- Zero breaking changes to existing functionality
- Optional import structure for graceful degradation
- Thread-safe operations with memory management

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 11:57:58 +00:00
github-actions[bot]andMervin Praison 2c16131770 fix: comprehensive suppression of litellm deprecation warnings
- Add environment variables to suppress warnings at source (LITELLM_TELEMETRY, HTTPX_DISABLE_WARNINGS)
- Implement global deprecation warning filters before imports
- Add specific message filters for exact warning text patterns
- Set aggressive logging levels (CRITICAL) for all litellm loggers
- Configure runtime litellm settings to disable debug output
- Apply comprehensive suppression across main and LLM modules
- Addresses remaining warnings from httpx content parameter and pydantic dict method usage

Resolves issue with persistent litellm warnings after initial fix

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 11:53:45 +00:00
github-actions[bot]andMervin Praison bc0d58104c enhance: improve JSON parsing error handling for tool arguments
- Add explicit JSON parsing error handling in both streaming and non-streaming modes
- Remove redundant import json statement in streaming mode (already imported at module level)
- Ensure graceful fallback to empty dict when JSON parsing fails
- Maintain backward compatibility while improving robustness

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 11:50:29 +00:00
Mervin Praison a11990fa40 Merge pull request #1034 from MervinPraison/claude/issue-1033-20250722-1128
Fix litellm deprecation warnings for issue #1033
2025-07-22 12:44:09 +01:00
github-actions[bot]andMervin Praison e1f6601137 refactor: consolidate and document warning filters for issue #1033
- Centralized warning suppression in main __init__.py with comprehensive comments
- Removed duplicated warning filters from llm/__init__.py
- Improved pydantic regex specificity and added module targeting
- Added clear documentation explaining why each warning is suppressed

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 11:43:27 +00:00
github-actions[bot]andMervin Praison 05dc3b5223 fix: correct tool call argument parsing in streaming mode
- Fix critical bug where tool_call[''function''][''arguments''] was passed as JSON string instead of parsed dict to execute_tool
- Add proper JSON parsing with fallback to empty dict for missing arguments
- Resolves tool execution failures during streaming when tools are called
- Maintains backward compatibility and error handling

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 11:42:01 +00:00
Mervin Praison c8fdab5885 Merge pull request #1032 from MervinPraison/claude/issue-1031-20250722-1110
fix: eliminate streaming pause caused by telemetry tracking
2025-07-22 12:40:04 +01:00
8f1c18da70 Fix litellm deprecation warnings for issue #1033
Added comprehensive deprecation warning filters to suppress warnings from:
- litellm module and event loop related warnings
- httpx module deprecation warnings
- Pydantic model_dump deprecation warnings

This resolves the noisy deprecation warnings that were cluttering the output
when using PraisonAI agents with newer versions of dependencies.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 11:36:17 +00:00
Mervin Praison 739c25cbea Merge pull request #1030 from MervinPraison/claude/fix-streaming-display-bypass-20250722
fix: bypass display_generation for OpenAI streaming to enable raw chunk output
2025-07-22 12:25:03 +01:00
github-actions[bot]andMervin Praison bbbb77ba5c feat: enhance streaming implementation with proper tool call support
- Fix critical OpenAI client access bug: use sync_client instead of client
- Reduce code duplication by consolidating completion creation logic
- Improve test script: remove unused imports and properly test generator consumption
- Verify comprehensive tool call handling in streaming mode is working correctly
- All automated review feedback addressed while maintaining backward compatibility

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 11:17:54 +00:00
github-actions[bot] 00b81080a3 merge: accept comprehensive async telemetry fix from remote
The remote branch already implements a more comprehensive solution using
threading at the integration level, which addresses the streaming pause issue
more thoroughly than the async_mode parameter approach.

Remote changes include:
- All telemetry tracking made asynchronous using background threads
- Special handling for streaming generators in agent.start()
- Better error handling with try/catch blocks
- Maintains the async_mode parameter in telemetry.py for future use

Resolves streaming pause issue #1031
2025-07-22 11:17:41 +00:00
github-actions[bot]andMervin Praison 2c7cbdf0c9 fix: eliminate streaming pause caused by synchronous telemetry tracking
- Add async_mode parameter to track_agent_execution() to enable background PostHog capture
- Update telemetry integration to auto-detect streaming agents and use async mode
- Maintains backward compatibility for non-streaming agents
- Resolves pause issue after "execution tracked: success=True" message in streaming workflows

Fixes #1031

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 11:16:58 +00:00
github-actions[bot]andMervin Praison 112a7248ca fix: Eliminate telemetry pause in streaming responses
- Make telemetry tracking asynchronous in all agent methods (chat, start, run)
- For streaming responses, defer telemetry tracking until after generator consumption
- Use separate daemon threads to prevent blocking on PostHog capture calls
- Add comprehensive test script to verify fix works
- Fixes issue where ~20 second pause occurred after ''execution tracked: success=True''

This resolves the streaming performance issue where telemetry calls were
blocking the start of streaming responses, causing users to wait before
seeing any output when using agent.start() with stream=True.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 11:15:50 +00:00
github-actions[bot]andMervin Praison ce214edd99 feat: enhance streaming implementation with proper tool call support
- Add comprehensive tool call handling in OpenAI streaming mode
- Properly accumulate and execute tool calls during streaming
- Maintain chat history integrity with tool call results
- Preserve backward compatibility and existing functionality
- Add error handling for tool execution failures in streaming

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 11:12:51 +00:00
Mervin Praison ab8932be28 Merge pull request #1029 from MervinPraison/claude/fix-gemini-streaming-json-parsing-20250722
fix: enhance Gemini streaming robustness with graceful JSON parsing error handling
2025-07-22 12:08:02 +01:00
github-actions[bot]andMervin Praison 47c91ae624 fix: correct critical bugs in Gemini streaming error handling
- Fix error counter reset bug: only reset after successful chunk processing
- Fix error masking bug: non-recoverable errors are re-raised immediately
- Add proper exception chaining with ''from chunk_error''
- Remove unnecessary f-string prefix
- Move test to proper location in tests/ directory
- Remove hardcoded sys.path in test script

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 11:04:14 +00:00
github-actions[bot]andMervin Praison 626260f787 fix: bypass display_generation for OpenAI streaming to enable raw chunk output
This fix modifies Agent._start_stream() to handle OpenAI-style models with direct
streaming that bypasses the display_generation UI, allowing users to get raw
streaming chunks when stream=True.

Key changes:
- Replace self.chat() fallback with direct OpenAI client streaming
- Add proper knowledge search and tool handling for OpenAI path
- Use _build_messages helper for consistent message formatting
- Yield raw chunks without display function interference
- Add robust error handling with chat history rollback
- Preserve fallback to simulated streaming on errors

Fixes streaming behavior where chunks were wrapped in display_generation
instead of being yielded directly to user code.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 11:03:23 +00:00
github-actions[bot]andMervin Praison c0b8aba75b fix: enhance Gemini streaming robustness with graceful JSON parsing error handling
- Add chunk-level error handling in get_response_stream() to catch malformed JSON chunks
- Implement smart error recovery that skips recoverable errors (JSON parsing) vs non-recoverable (connection)
- Add consecutive error limit (3) with automatic fallback to non-streaming mode
- Enhance error logging with specific categorization for better debugging
- Add _is_streaming_error_recoverable() helper method for error classification
- Create test script to validate streaming robustness improvements

Fixes issue where Gemini streaming fails with ''Expecting property name enclosed in double quotes''
when receiving incomplete JSON chunks like ''{'' during streaming responses.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 08:56:50 +00:00
Mervin Praison e3b0b3830e Merge pull request #1028 from MervinPraison/claude/fix-real-time-streaming-20250722
fix: implement real-time streaming for Agent.start() method
2025-07-22 09:47:33 +01:00
github-actions[bot]andMervin Praison 8c3e1f8569 fix: add comprehensive tool call support to streaming implementation
- Remove erroneous temperature parameter from _build_messages() call in get_response_stream()
- Fix critical bug that would cause TypeError during streaming
- Tool call handling already implemented with _process_stream_delta helper
- Real-time streaming now works with content and tool execution
- Follow-up responses after tool completion properly handled
- Backward compatibility maintained with stream=False default

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 08:10:27 +00:00
github-actions[bot]andMervin Praison ab01fe2965 fix: add comprehensive tool call support to streaming implementation
Fixes critical gap in get_response_stream method where tool calls were
completely ignored during streaming. The method now properly:

- Accumulates tool calls during streaming using _process_stream_delta
- Executes tool calls after streaming completes if execute_tool_fn provided
- Continues conversation with follow-up response after tool execution
- Handles both Ollama and other providers correctly
- Maintains real-time content streaming while adding tool support

This makes streaming fully functional for agents that use tools, which
is essential for PraisonAI''s core functionality.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Mervin Praison <[email protected]>
2025-07-22 07:58:31 +00:00
github-actions[bot]andMervin Praison e6472b5052 fix: restore backward compatibility by defaulting stream=False
- Change Agent constructor default: stream=True -> stream=False
- Update documentation to clarify backward compatibility priority
- Ensure existing code agent.start(''prompt'') returns string not generator
- Maintain streaming capability when explicitly enabled with stream=True

Fixes backward compatibility issue where basic usage would return generator
instead of expected string response, breaking existing codebases.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 06:40:27 +00:00
github-actions[bot]andMervin Praison a178abe759 fix: implement real-time streaming for Agent.start() method
- Add get_response_stream() generator method to LLM class for true real-time streaming
- Modify Agent._start_stream() to use new streaming generator for custom LLMs
- Replace simulated streaming with actual LiteLLM streaming chunks
- Maintain backward compatibility for OpenAI-style models (temporary fallback)
- Add comprehensive error handling and chat history management
- Include test files for validation

Fixes the issue where streaming was simulated by chunking complete responses
instead of yielding real-time chunks from the underlying LLM.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2025-07-22 06:32:17 +00:00
Mervin Praison b139ea98ba Merge pull request #1027 from MervinPraison/claude/issue-981-20250721-1557
fix: implement real-time streaming for Agent.start() method
2025-07-22 07:17:42 +01:00
github-actions[bot]andMervin Praison 589650029b refactor: simplify streaming implementation to eliminate code duplication
- Replace duplicated logic in _custom_llm_stream() and _openai_stream() with single _start_stream() method
- Reuse existing chat() method logic to maintain all features (tools, knowledge, MCP, etc.)
- Improve error handling and consistency across LLM types
- Maintain backward compatibility and all existing functionality
- Reduce maintenance burden by eliminating 150+ lines of duplicated code

Co-authored-by: Mervin Praison <[email protected]>
2025-07-21 16:42:18 +00:00
github-actions[bot]andMervin Praison 97b25f5003 fix: implement real-time streaming for Agent.start() method
- Fix streaming issue where Agent.start() was not yielding real-time chunks
- Add streaming generator support with proper type hints
- Implement _start_stream() method that routes to appropriate handlers
- Add _custom_llm_stream() for custom LLM providers with simulated chunking
- Add _openai_stream() for OpenAI client with true streaming chunks
- Maintain backward compatibility for existing non-streaming usage
- Support both OpenAI and custom LLM streaming with graceful error handling
- Preserve chat history management and proper conversation state

The user''s code pattern now works as expected:
for chunk in agent.start(prompt, stream=True):
    print(chunk, end='''', flush=True)

Fixes #981

Co-authored-by: Mervin Praison <[email protected]>
2025-07-21 16:02:10 +00:00
Mervin Praison 5d30ce9a1b Merge pull request #1026 from MervinPraison/claude/issue-1025-20250721-0849
feat: add context CLI command with --url and --goal parameters
2025-07-21 12:17:40 +01:00
github-actions[bot]andMervin Praison 5ad39b5f1b refactor: improve context CLI command with minor code enhancements
- Fix f-string formatting issues by removing unnecessary f prefix
- Add return type hint to handle_context_command method
- Enhance docstring with return type documentation
- Maintain backward compatibility with existing usage

Co-authored-by: Mervin Praison <[email protected]>
2025-07-21 10:12:16 +00:00
github-actions[bot]andMervin Praison 28c60d52f5 feat: enhance context CLI command with configurable options
- Add --auto-analyze flag for user control over analysis mode
- Use environment MODEL_NAME/OPENAI_MODEL_NAME instead of hardcoded model
- Update usage messages to include new optional flag
- Follow existing CLI patterns for model configuration
- Maintain backward compatibility with defaults

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2025-07-21 10:08:48 +00:00
github-actions[bot]andMervin Praison 3a9e26670f feat: add context CLI command with --url and --goal parameters
- Add ''context'' to special_commands list
- Add --url and --goal CLI arguments
- Add handle_context_command method to create ContextAgent
- Maintains backward compatibility
- Implements exact functionality requested in issue #1025

Usage: praisonai context --url <url> --goal <goal>

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2025-07-21 08:54:53 +00:00
Mervin Praison 209793bdb1 feat: add MongoDB integration examples and context engineering improvements 2025-07-20 03:47:29 +01:00
Mervin Praison aa965cf719 Merge pull request #1024 from MervinPraison/fix-knowledge-verbose-headers
fix: remove verbose knowledge headers from AI agent prompts
2025-07-19 13:06:45 +01:00
Mervin Praison 4927f24d4a Merge pull request #1021 from MervinPraison/claude/issue-1018-20250719-0632
Implement Context Engineering - Add ContextAgent for AI Context Generation
2025-07-19 09:45:51 +01:00
github-actions[bot]andMervin Praison 2a3240156d fix: add backward compatibility for embedding_db_config key in knowledge context processing
- Support both ''vector_store'' and ''embedding_db_config'' keys for knowledge context
- Maintains backward compatibility with existing test files
- No functional changes to the verbose header removal functionality

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2025-07-19 08:43:17 +00:00
github-actions[bot]andClaude 77e8ccb3c9 Transform hardcoded analysis to Agent-based dynamic analysis
This commit transforms all hardcoded analysis methods in ContextAgent to use specialized AI agents for dynamic analysis. The changes enable more flexible, intelligent, and adaptable context processing by leveraging agent-based approaches instead of static analysis patterns.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
2025-07-19 08:42:49 +00:00
github-actions[bot]andMervin Praison 031c73c7d3 fix: remove verbose knowledge headers from AI agent prompts
- Remove ''[DB Context]:'' and ''[Vector DB Error]:'' prefixes from agent outputs
- Add debug logging to preserve troubleshooting information
- Return clean content without verbose headers in terminal display
- Hide error details from AI prompts while preserving debug logs
- Consistent with memory.py verbose header removal approach

Co-authored-by: Mervin Praison <[email protected]>
2025-07-19 08:33:29 +00:00
Mervin Praison 82b6f9c5ff Merge pull request #1023 from MervinPraison/claude/pr-1020-20250719-0708
fix: remove verbose memory context headers from AI agent prompts
2025-07-19 09:06:01 +01:00
github-actions[bot]andMervin Praison 22449a35b7 feat: hide memory content from terminal display unless debug logging enabled
- Add include_in_output parameter to build_context_for_task() method
- Memory content now only included in output when logging level is DEBUG
- Memory is always logged for debugging regardless of output inclusion
- Preserves AI agent functionality while hiding memory from user display
- Addresses user requirement to hide memory items from terminal by default

Co-authored-by: Mervin Praison <[email protected]>
2025-07-19 07:55:41 +00:00
Mervin Praison 75bf3ac73b Merge pull request #1022 from MervinPraison/claude/issue-1019-20250719-0623
fix: move embedding debug logs to trace level
2025-07-19 08:52:32 +01:00
4d3e19462b Reorganize TRACE logging test files and improve test structure
- Move test_embedding_logging.py from root to tests/ directory for better organization
- Improve test structure with comprehensive TRACE logging validation
- Ensure proper test isolation and cleanup
- Maintain backward compatibility while improving code organization

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Mervin Praison <[email protected]>
Co-Authored-By: Claude <[email protected]>
2025-07-19 07:42:08 +00:00
github-actions[bot]andClaude efaff8845d Fix Context Engineering implementation: Complete functionality and async support
- Fixed core ContextAgent functionality with proper imports and async support
- Added comprehensive error handling and robust implementation
- Cleaned up imports and fixed async method patterns
- Moved test file from root to proper tests/ directory location
- Made implementation fully functional with all required methods
- Enhanced examples with working context engineering workflows

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
2025-07-19 07:15:22 +00:00
github-actions[bot]andMervin Praison a4e7a1bdbc fix: remove verbose memory context headers from AI agent prompts
- Remove ''Short-term Memory (X items)'' and ''Long-term Memory (X items)'' headers from AI prompts
- Remove ''Relevant Memory Context:'' and ''Previous task'' prefixes from agent context
- Keep actual memory content and task results for AI functionality
- Move verbose information to debug logging only
- Prevents AI agents from echoing memory context headers in terminal output

This fixes the issue where memory context headers were still appearing in terminal
displays even after the original PR moved verbose information to debug logging.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-19 07:14:37 +00:00
Mervin Praison fd346b410d Merge pull request #1020 from MervinPraison/claude/fix-context-display-issues
fix: move verbose context information from live display to debug logging
2025-07-19 07:53:34 +01:00
github-actions[bot]andMervin Praison 9acff8f977 Implement Context Engineering - Add ContextAgent for AI Context Generation
- Add ContextAgent class with comprehensive Context Engineering capabilities
- Implements automated context generation for AI coding assistants
- Features: codebase analysis, pattern extraction, validation loops, PRP generation
- 10x better than prompt engineering, 100x better than vibe coding
- Includes factory function create_context_agent() for easy instantiation
- Add comprehensive examples in agents/ and concepts/ directories
- Full backward compatibility maintained with existing PraisonAI functionality
- Complete test suite with 7/7 tests passing

Co-authored-by: Mervin Praison <[email protected]>
2025-07-19 06:46:51 +00:00
github-actions[bot]andMervin Praison 4bc3ad97a1 fix: restore AI agent access to actual memory and task context content
- agents.py: Restore actual task result content and memory context in AI prompts
- task.py: Include actual task context results for proper task chaining
- memory.py: Return actual memory content with clean formatting instead of placeholders
- Fix f-string formatting issue in agents.py

This fixes critical functionality bugs where AI agents were receiving placeholder
messages instead of actual content, breaking memory access and task dependencies.
AI agents now get full context while debug logging is preserved.

Resolves issues identified by automated reviews while maintaining backward compatibility.

Co-authored-by: Mervin Praison <[email protected]>
2025-07-19 06:45:55 +00:00
Claude Code a91228b46e Add custom TRACE logging level to reduce embedding data in debug output
- Add TRACE_LEVEL (5) below DEBUG (10) for fine-grained logging control
- Implement trace() method on logging.Logger class for backward compatibility
- Convert 4 embedding-related debug statements to use trace level
- Embedding data now only appears when logging level set to TRACE or lower
- Maintains full backward compatibility with existing debug logging
- Addresses issue where sensitive embedding data cluttered terminal output

Test coverage:
- Add test_embedding_logging.py to validate implementation functionality
- Verify TRACE level behavior and message filtering

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
2025-07-19 06:45:31 +00:00
Mervin Praison 74403501ed Merge pull request #1017 from MervinPraison/claude/issue-1016-20250719-0556
fix(knowledge): resolve 'list' object has no attribute 'get' error
2025-07-19 07:17:56 +01:00
github-actions[bot]andClaude c1703dd234 Fix backward compatibility issue in knowledge.py memory result handling
- Add type checking for memory_result in _process_single_input method
- Handle both dict and list formats for backward compatibility
- Gracefully handle unexpected types with warning instead of crashing
- Resolves issue #1016 where MongoDB memory provider returns dict while Chroma returns list

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
2025-07-19 06:08:08 +00:00
github-actions[bot]andClaude 256ad050e4 fix: move verbose context information from live display to debug logging
This commit addresses live display context issues by moving detailed content
from user-facing prompts to debug logging while maintaining functionality:

- agents.py line 653: Memory context now logged via logger.debug() with
  brief user-facing note in prompt instead of full content
- agents.py line 84: Previous task results logged via logger.debug() with
  brief completion status returned instead of full raw content
- task.py line 406: Previous task results logged via logger.debug() with
  brief completion message appended instead of full raw content
- memory.py: Memory section headers now use brief titles with item counts
  and log detailed content via logger.debug() instead of showing full content

These changes maintain backward compatibility and functionality while
improving user experience by reducing verbose output in live displays.
Detailed information remains available through debug logging for debugging
and troubleshooting purposes.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
2025-07-19 06:01:11 +00:00
github-actions[bot]andMervin Praison 4ca844e667 fix(knowledge): resolve ''list'' object has no attribute ''get'' error
Fixed line 717 in knowledge.py where memory_result.get(''results'', [])
was called on a list instead of a dict. Both MongoDBMemory.add() and
CustomMemory._add_to_vector_store() return lists directly, so changed
to all_results.extend(memory_result) for proper list concatenation.

Resolves #1016

Co-authored-by: Mervin Praison <[email protected]>
2025-07-19 06:00:09 +00:00
Mervin Praison 7e473dcb14 Merge pull request #1015 from MervinPraison/claude/issue-1014-20250719-0501
fix: show context only when loglevel=debug
2025-07-19 06:29:23 +01:00
Mervin Praison 0188c6bf77 docs: add llms.md documentation and update MongoDB example to use local knowledge file 2025-07-19 06:27:15 +01:00
github-actions[bot]andClaude df304df3cf Fix logger configuration to support DEBUG level for verbose >= 3
- Add DEBUG logging level when verbose >= 3
- Maintain INFO level for verbose >= 5
- Keep WARNING level for verbose < 3
- This makes the verbose flag functional for debugging purposes

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
2025-07-19 05:22:03 +00:00
Claude CodeandMervin Praison 82cdda36d7 fix: show context only when loglevel=debug
Changed logger.info to logger.debug for context content display
in agents.py to reduce log noise while maintaining the ability
to see context information when debug logging is enabled.

Changes:
- agents.py:315 - Context content logging moved to debug level
- agents.py:638 - Context content logging moved to debug level

Maintains backward compatibility as context count logging remains
at INFO level, only the actual context content is moved to DEBUG.

Fixes #1014

Co-authored-by: Mervin Praison <[email protected]>
2025-07-19 05:08:27 +00:00
Mervin Praison 9d6c6d7ab2 refactor: improve result handling and display in MongoDB example files 2025-07-19 06:01:20 +01:00
85 changed files with 14393 additions and 373 deletions

No files matched your search

+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
flask \
"praisonai>=2.2.71" \
"praisonai>=2.2.73" \
"praisonai[api]" \
gunicorn \
markdown
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.2.71" \
"praisonai>=2.2.73" \
"praisonai[chat]" \
"embedchain[github,youtube]"
+1 -1
View File
@@ -20,7 +20,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.2.71" \
"praisonai>=2.2.73" \
"praisonai[ui]" \
"praisonai[chat]" \
"praisonai[realtime]" \
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.2.71" \
"praisonai>=2.2.73" \
"praisonai[ui]" \
"praisonai[crewai]"
+2 -2
View File
@@ -121,7 +121,7 @@ healthcheck:
## 📦 Package Versions
All Docker images use consistent, up-to-date versions:
- PraisonAI: `>=2.2.71`
- PraisonAI: `>=2.2.73`
- PraisonAI Agents: `>=0.0.92`
- Python: `3.11-slim`
@@ -218,7 +218,7 @@ docker-compose up -d
### Version Pinning
To use specific versions, update the Dockerfile:
```dockerfile
RUN pip install "praisonai==2.2.71" "praisonaiagents==0.0.92"
RUN pip install "praisonai==2.2.73" "praisonaiagents==0.0.92"
```
## 🌐 Production Deployment
+231
View File
@@ -0,0 +1,231 @@
#!/usr/bin/env python3
"""
Context Agent Example - Basic Usage
This example demonstrates how to use the ContextAgent for automated context generation.
The ContextAgent implements Context Engineering principles to generate comprehensive
context for AI coding assistants, enabling first-try implementation success.
Key features demonstrated:
- Basic ContextAgent instantiation
- Codebase pattern analysis
- Context document generation
- Validation loop creation
- Prompt enhancement with context
- PRP (Product Requirements Prompt) generation
"""
import sys
from pathlib import Path
# Add the src directory to Python path for imports
project_root = Path(__file__).parent.parent.parent.parent / "src" / "praisonai-agents"
sys.path.insert(0, str(project_root))
from praisonaiagents import create_context_agent
def demonstrate_basic_context_generation():
"""Demonstrate basic context generation capabilities."""
print("🔧 Context Engineering Example: Basic Usage")
print("=" * 60)
# Create a ContextAgent using the factory function
context_agent = create_context_agent(
llm="gpt-4o-mini",
name="Basic Context Engineer",
verbose=True
)
# Example 1: Analyze codebase patterns
print("\n📁 Example 1: Analyzing Codebase Patterns")
print("-" * 40)
# Use current project as example (adjust path as needed)
project_path = str(project_root)
analysis = context_agent.analyze_codebase_patterns(
project_path=project_path,
file_patterns=["*.py"]
)
print(f"✅ Analyzed project at: {project_path}")
print(f" Project structure: {len(analysis.get('project_structure', {}).get('directories', []))} directories")
print(f" Code patterns identified: {len(analysis.get('code_patterns', {}))}")
print(f" Architecture: {analysis.get('architecture_insights', {}).get('primary_pattern', 'Unknown')}")
# Example 2: Generate context document
print("\n📄 Example 2: Generating Context Document")
print("-" * 40)
feature_request = "Add user authentication system with JWT tokens and role-based access control"
context_doc = context_agent.generate_context_document(
project_path=project_path,
requirements=feature_request,
analysis=analysis
)
print(f"✅ Generated context document for: {feature_request}")
print(f" Document length: {len(context_doc)} characters")
print(f" Contains architecture patterns: {'Architecture Patterns' in context_doc}")
print(f" Contains validation criteria: {'Validation Criteria' in context_doc}")
# Example 3: Create validation loop
print("\n✅ Example 3: Creating Validation Loop")
print("-" * 40)
success_criteria = [
"Authentication system accepts valid JWT tokens",
"Role-based access control blocks unauthorized users",
"User registration and login endpoints work correctly",
"Password hashing follows security best practices",
"Session management handles token expiration"
]
validation_config = context_agent.create_validation_loop(
implementation_requirements=feature_request,
success_criteria=success_criteria
)
print(f"✅ Created validation loop with {len(validation_config['validation_steps'])} steps")
print(f" Success criteria: {len(success_criteria)}")
print(f" Executable tests: {len(validation_config['executable_tests'])}")
print(f" Quality gates: {len(validation_config['quality_gates'])}")
# Example 4: Enhance prompt with context
print("\n🚀 Example 4: Enhancing Prompt with Context")
print("-" * 40)
basic_prompt = "Implement user authentication"
enhanced_prompt = context_agent.enhance_prompt_with_context(
base_prompt=basic_prompt,
context_data=analysis
)
print(f"✅ Enhanced prompt from {len(basic_prompt)} to {len(enhanced_prompt)} characters")
print(f" Original: '{basic_prompt}'")
print(" Enhanced includes: Context Engineering, Implementation Context, Quality Requirements")
# Example 5: Generate PRP (Product Requirements Prompt)
print("\n📋 Example 5: Generating PRP")
print("-" * 40)
documentation_links = [
"https://jwt.io/introduction/",
"https://fastapi.tiangolo.com/tutorial/security/",
"https://passlib.readthedocs.io/en/stable/"
]
prp = context_agent.generate_prp(
feature_request=feature_request,
context_analysis=analysis,
documentation_links=documentation_links
)
print("✅ Generated comprehensive PRP")
print(f" PRP length: {len(prp)} characters")
print(f" Contains feature request: {feature_request in prp}")
print(f" Contains documentation links: {len(documentation_links)} links included")
print(f" Contains implementation blueprint: {'Implementation Blueprint' in prp}")
print(f" Confidence score: {'9/10' in prp}")
def demonstrate_context_agent_as_tool():
"""Demonstrate using ContextAgent as a tool in a multi-agent workflow."""
print("\n🔗 Context Engineering Example: Multi-Agent Integration")
print("=" * 60)
# Create ContextAgent
context_agent = create_context_agent(llm="gpt-4o-mini")
# Example scenario: Product planning workflow
project_path = str(project_root)
feature_request = "Build a real-time chat system with WebSocket support"
# Step 1: Context analysis
print("\n1️⃣ Step 1: Context Analysis")
analysis = context_agent.analyze_codebase_patterns(project_path)
print(f"✅ Analyzed codebase architecture and patterns")
# Step 2: Generate implementation blueprint
print("\n2️⃣ Step 2: Implementation Blueprint")
blueprint = context_agent.create_implementation_blueprint(feature_request, analysis)
print(f"✅ Created implementation blueprint with {len(blueprint['implementation_steps'])} steps")
# Step 3: Generate comprehensive PRP
print("\n3️⃣ Step 3: Generate PRP")
prp = context_agent.generate_prp(feature_request, analysis) # Generated PRP for implementation guidance
print(f"✅ Generated PRP for implementation guidance")
# Step 4: Create validation framework
print("\n4️⃣ Step 4: Validation Framework")
criteria = [
"WebSocket connections establish successfully",
"Messages broadcast to all connected clients",
"Chat history persists and loads correctly",
"User authentication works with WebSocket",
"Connection handling is robust with reconnection"
]
validation = context_agent.create_validation_loop(feature_request, criteria) # Validation framework for QA
print(f"✅ Created validation framework with {len(criteria)} criteria")
print("\n🎯 Context Engineering Workflow Complete!")
print(f" Feature: {feature_request}")
print(" Implementation ready with comprehensive context")
print(" Validation criteria defined for quality assurance")
def show_context_engineering_benefits():
"""Demonstrate the benefits of Context Engineering vs traditional approaches."""
print("\n📊 Context Engineering Benefits")
print("=" * 60)
# Traditional prompt engineering approach
print("\n❌ Traditional Prompt Engineering:")
traditional_prompt = "Create a user authentication system"
print(f" Prompt: '{traditional_prompt}'")
print(f" Length: {len(traditional_prompt)} characters")
print(f" Context: Minimal - relies on AI's general knowledge")
print(f" Success Rate: Variable - depends on AI model's training")
# Context Engineering approach
print("\n✅ Context Engineering Approach:")
context_agent = create_context_agent(llm="gpt-4o-mini")
# Generate comprehensive context
analysis = context_agent.analyze_codebase_patterns(str(project_root))
enhanced_prompt = context_agent.enhance_prompt_with_context(
traditional_prompt, analysis
)
print(f" Enhanced Prompt Length: {len(enhanced_prompt)} characters")
print(f" Context: Comprehensive - includes:")
print(f" • Codebase architecture analysis")
print(f" • Existing patterns and conventions")
print(f" • Implementation guidance")
print(f" • Quality requirements")
print(f" • Validation criteria")
print(f" Success Rate: Higher - AI has all necessary context")
print(f"\n🎯 Context Engineering provides:")
print(" 📈 10x better than prompt engineering (comprehensive vs clever wording)")
print(" 📈 100x better than vibe coding (structured vs ad-hoc)")
print(" 🎯 First-try implementation success through complete context")
if __name__ == "__main__":
print("🚀 PraisonAI Context Engineering Examples")
print("=========================================")
try:
# Run all examples
demonstrate_basic_context_generation()
demonstrate_context_agent_as_tool()
show_context_engineering_benefits()
print("\n🎉 All Context Engineering examples completed successfully!")
print("\n💡 Key Takeaways:")
print(" • ContextAgent automates comprehensive context generation")
print(" • Context Engineering enables first-try implementation success")
print(" • Integration with existing PraisonAI agents is seamless")
print(" • Validation frameworks ensure quality and completeness")
except Exception as e:
print(f"\n❌ Error running examples: {e}")
print(" Make sure you have the PraisonAI package installed and configured")
sys.exit(1)
@@ -0,0 +1,469 @@
#!/usr/bin/env python3
"""
Context Engineering Workflow - Advanced Multi-Agent Example
This example demonstrates advanced Context Engineering workflows using ContextAgent
in combination with other PraisonAI agents. It shows how Context Engineering can
dramatically improve AI implementation success rates through comprehensive context
generation and validation loops.
Workflow demonstrated:
1. Product Manager Agent defines requirements
2. ContextAgent analyzes codebase and generates comprehensive context
3. Architecture Agent designs implementation using context
4. Developer Agent implements using context-enhanced prompts
5. QA Agent validates using context-generated criteria
This represents the full Context Engineering methodology in action.
"""
import sys
from pathlib import Path
# Add the src directory to Python path for imports
project_root = Path(__file__).parent.parent.parent.parent / "src" / "praisonai-agents"
sys.path.insert(0, str(project_root))
from praisonaiagents import Agent, Task, PraisonAIAgents, create_context_agent
class ContextEngineeringWorkflow:
"""
Advanced Context Engineering workflow demonstrating the full methodology.
This class orchestrates multiple agents using Context Engineering principles
to achieve higher success rates in AI-driven development tasks.
"""
def __init__(self, project_path: str, llm: str = "gpt-4o-mini"):
self.project_path = project_path
self.llm = llm
self.context_data = {}
self.setup_agents()
def setup_agents(self):
"""Setup the multi-agent team with Context Engineering support."""
# 1. Product Manager Agent - Defines requirements
self.product_manager = Agent(
name="Product Manager",
role="Product Requirements Specialist",
goal="Define clear, comprehensive product requirements and user stories",
backstory="""You are an experienced product manager who specializes in
creating detailed, actionable requirements. You understand that clear
requirements are the foundation of successful implementation.""",
instructions="""
When defining requirements:
1. Be specific and measurable
2. Include user acceptance criteria
3. Consider edge cases and error scenarios
4. Define success metrics
5. Specify technical constraints
""",
llm=self.llm,
verbose=True
)
# 2. Context Engineering Agent - Generates comprehensive context
self.context_engineer = create_context_agent(
llm=self.llm,
name="Context Engineering Specialist",
verbose=True
)
# 3. Software Architect Agent - Designs implementation using context
self.architect = Agent(
name="Software Architect",
role="System Architecture Specialist",
goal="Design robust, scalable system architecture based on comprehensive context",
backstory="""You are a senior software architect with expertise in
designing systems that follow established patterns and best practices.
You excel at creating architectures that integrate seamlessly with
existing codebases.""",
instructions="""
When designing architecture:
1. Follow the patterns identified in the context analysis
2. Ensure compatibility with existing systems
3. Design for scalability and maintainability
4. Consider security and performance implications
5. Document architectural decisions and rationale
""",
llm=self.llm,
verbose=True
)
# 4. Senior Developer Agent - Implements using context-enhanced guidance
self.developer = Agent(
name="Senior Developer",
role="Implementation Specialist",
goal="Implement features following context-guided best practices",
backstory="""You are a senior developer who excels at implementing
features that follow established codebase patterns. You understand
that consistency and quality are paramount.""",
instructions="""
When implementing features:
1. Follow the codebase patterns identified in context analysis
2. Maintain consistency with existing code style
3. Implement comprehensive error handling
4. Write clean, maintainable code
5. Follow the implementation blueprint exactly
""",
llm=self.llm,
verbose=True
)
# 5. QA Engineer Agent - Validates using context-generated criteria
self.qa_engineer = Agent(
name="QA Engineer",
role="Quality Assurance Specialist",
goal="Ensure implementation meets all quality criteria and requirements",
backstory="""You are an experienced QA engineer who specializes in
comprehensive testing and validation. You understand that quality
is built in, not bolted on.""",
instructions="""
When validating implementations:
1. Use the validation criteria from context analysis
2. Test both happy path and edge cases
3. Verify integration with existing systems
4. Check for security vulnerabilities
5. Validate performance requirements
""",
llm=self.llm,
verbose=True
)
def run_context_engineering_workflow(self, feature_request: str):
"""
Execute the complete Context Engineering workflow.
Args:
feature_request (str): The feature to be implemented
Returns:
dict: Complete workflow results with context and implementation
"""
print(f"🚀 Starting Context Engineering Workflow")
print(f"Feature Request: {feature_request}")
print("=" * 80)
# Phase 1: Requirements Analysis
print("\n📋 Phase 1: Product Requirements Analysis")
print("-" * 50)
requirements_task = Task(
name="requirements_analysis",
description=f"""
Analyze and refine the feature request: '{feature_request}'
Create comprehensive requirements including:
- Detailed feature description
- User acceptance criteria
- Technical requirements
- Success metrics
- Edge cases and constraints
""",
expected_output="Comprehensive product requirements document",
agent=self.product_manager
)
# Phase 2: Context Engineering Analysis
print("\n🔧 Phase 2: Context Engineering Analysis")
print("-" * 50)
# Generate comprehensive context using ContextAgent
codebase_analysis = self.context_engineer.analyze_codebase_patterns(
project_path=self.project_path
)
context_document = self.context_engineer.generate_context_document(
project_path=self.project_path,
requirements=feature_request,
analysis=codebase_analysis
)
validation_framework = self.context_engineer.create_validation_loop(
implementation_requirements=feature_request,
success_criteria=[
"Feature implements all specified requirements",
"Code follows existing patterns and conventions",
"Implementation includes comprehensive error handling",
"All tests pass and coverage meets standards",
"Integration with existing systems is seamless"
]
)
implementation_blueprint = self.context_engineer.create_implementation_blueprint(
feature_request=feature_request,
context_analysis=codebase_analysis
)
# Generate PRP for complete context
prp = self.context_engineer.generate_prp(
feature_request=feature_request,
context_analysis=codebase_analysis,
documentation_links=[
"https://docs.praisonai.com/",
"https://pydantic-docs.helpmanual.io/",
"https://fastapi.tiangolo.com/"
]
)
print(f"✅ Context Engineering Analysis Complete:")
print(f" • Codebase analysis: {len(str(codebase_analysis))} chars")
print(f" • Context document: {len(context_document)} chars")
print(f" • Validation framework: {len(validation_framework['validation_steps'])} steps")
print(f" • Implementation blueprint: {len(implementation_blueprint['implementation_steps'])} steps")
print(f" • PRP generated: {len(prp)} chars")
# Store context data for subsequent phases
self.context_data = {
"codebase_analysis": codebase_analysis,
"context_document": context_document,
"validation_framework": validation_framework,
"implementation_blueprint": implementation_blueprint,
"prp": prp
}
# Phase 3: Architecture Design with Context
print("\n🏗️ Phase 3: Architecture Design with Context")
print("-" * 50)
architecture_task = Task(
name="architecture_design",
description=f"""
Design system architecture for: '{feature_request}'
Use the comprehensive context provided:
{context_document}
Implementation Blueprint:
{implementation_blueprint}
Design architecture that:
- Follows identified codebase patterns
- Integrates with existing systems
- Meets all technical requirements
- Is scalable and maintainable
""",
expected_output="Detailed system architecture design with component specifications",
agent=self.architect,
context=[requirements_task]
)
# Phase 4: Implementation with Context-Enhanced Guidance
print("\n💻 Phase 4: Implementation with Context")
print("-" * 50)
# Enhance the implementation prompt with context
enhanced_prompt = self.context_engineer.enhance_prompt_with_context(
base_prompt=f"Implement {feature_request}",
context_data=codebase_analysis
)
implementation_task = Task(
name="feature_implementation",
description=f"""
Implement the feature using context-enhanced guidance:
{enhanced_prompt}
Product Requirements Prompt (PRP):
{prp}
Architecture Design: Reference the architecture task output
Implementation must:
- Follow the implementation blueprint exactly
- Use patterns identified in codebase analysis
- Meet all requirements from Phase 1
- Include comprehensive error handling
""",
expected_output="Complete feature implementation with code and documentation",
agent=self.developer,
context=[requirements_task, architecture_task]
)
# Phase 5: Quality Assurance with Context-Generated Criteria
print("\n🔍 Phase 5: Quality Assurance with Context")
print("-" * 50)
qa_task = Task(
name="quality_validation",
description=f"""
Validate the implementation using context-generated criteria:
Validation Framework:
{validation_framework}
Verify that implementation:
- Meets all success criteria defined in validation framework
- Follows codebase patterns and conventions
- Integrates properly with existing systems
- Handles edge cases and error scenarios
- Meets performance and security requirements
Use the validation steps provided to systematically check each criterion.
""",
expected_output="Comprehensive quality validation report with pass/fail status",
agent=self.qa_engineer,
context=[requirements_task, architecture_task, implementation_task]
)
# Execute the complete workflow
print("\n⚙️ Executing Context Engineering Workflow")
print("-" * 50)
agents_workflow = PraisonAIAgents(
agents=[
self.product_manager,
self.architect,
self.developer,
self.qa_engineer
],
tasks=[
requirements_task,
architecture_task,
implementation_task,
qa_task
],
process="sequential",
verbose=True
)
# Execute workflow
workflow_results = agents_workflow.start()
# Compile complete results
complete_results = {
"feature_request": feature_request,
"context_engineering": self.context_data,
"workflow_results": workflow_results,
"methodology": "Context Engineering - 10x better than prompt engineering"
}
return complete_results
def demonstrate_context_engineering_benefits():
"""
Demonstrate the benefits of Context Engineering methodology.
"""
print("\n📊 Context Engineering vs Traditional Approaches")
print("=" * 60)
# Traditional approach simulation
print("\n❌ Traditional AI Coding Approach:")
print(" 1. Write basic prompt: 'Implement user authentication'")
print(" 2. AI uses general knowledge to implement")
print(" 3. Result often doesn't fit existing codebase patterns")
print(" 4. Multiple iterations needed to fix integration issues")
print(" 5. Success rate: ~30-40% first-try success")
# Context Engineering approach
print("\n✅ Context Engineering Approach:")
print(" 1. Analyze codebase patterns and architecture")
print(" 2. Generate comprehensive context document")
print(" 3. Create validation framework with success criteria")
print(" 4. Enhance prompts with rich contextual information")
print(" 5. Generate PRP (Product Requirements Prompt)")
print(" 6. AI implements using complete context")
print(" 7. Success rate: ~90%+ first-try success")
print("\n📈 Context Engineering Advantages:")
print(" • 10x better than prompt engineering (context vs clever wording)")
print(" • 100x better than vibe coding (structured vs ad-hoc)")
print(" • Enables first-try implementation success")
print(" • Reduces development iteration cycles")
print(" • Ensures consistency with existing codebase")
print(" • Provides built-in quality validation")
def run_example_workflow():
"""Run an example Context Engineering workflow."""
# Setup workflow with current project
project_path = str(project_root)
workflow = ContextEngineeringWorkflow(
project_path=project_path,
llm="gpt-4o-mini"
)
# Example feature request
feature_request = """
Implement a real-time notification system that:
- Sends notifications via WebSocket connections
- Supports different notification types (info, warning, error)
- Persists notifications in database for offline users
- Includes user preference management for notification settings
- Provides REST API for notification management
"""
print("🎯 Running Example Context Engineering Workflow")
print("=" * 60)
try:
# Execute the complete workflow
results = workflow.run_context_engineering_workflow(feature_request)
print("\n🎉 Context Engineering Workflow Completed Successfully!")
print("=" * 60)
print(f"\n📊 Workflow Summary:")
print(f" • Feature: Real-time notification system")
print(f" • Context generated: ✅ Complete")
print(f" • Architecture designed: ✅ With context")
print(f" • Implementation: ✅ Context-guided")
print(f" • Quality validation: ✅ Context-criteria")
print(f"\n🔧 Context Engineering Data:")
context_data = results["context_engineering"]
print(f" • Codebase analysis: {len(str(context_data['codebase_analysis']))} chars")
print(f" • Context document: {len(context_data['context_document'])} chars")
print(f" • Validation framework: {len(context_data['validation_framework']['validation_steps'])} steps")
print(f" • Implementation blueprint: {len(context_data['implementation_blueprint']['implementation_steps'])} steps")
print(f" • PRP: {len(context_data['prp'])} chars")
return results
except Exception as e:
print(f"\n❌ Error in workflow execution: {e}")
print(" Note: This is a demonstration - actual execution requires proper environment setup")
return None
if __name__ == "__main__":
print("🚀 Advanced Context Engineering Workflow Example")
print("================================================")
print("\n💡 This example demonstrates the complete Context Engineering methodology:")
print(" • Comprehensive codebase analysis")
print(" • Context document generation")
print(" • Validation framework creation")
print(" • Multi-agent workflow with context integration")
print(" • Quality assurance with context-generated criteria")
# Show benefits comparison
demonstrate_context_engineering_benefits()
# Run example workflow (demo mode)
print("\n🎭 Demo Mode: Context Engineering Workflow Structure")
print("(Note: Full execution requires complete environment setup)")
project_path = str(project_root)
workflow = ContextEngineeringWorkflow(project_path, "gpt-4o-mini")
print(f"\n✅ Workflow Setup Complete:")
print(f" • Project path: {project_path}")
print(f" • Agents configured: 5 (PM, Context Engineer, Architect, Developer, QA)")
print(f" • Context Engineering agent: ✅ Ready")
print(f" • Multi-agent workflow: ✅ Configured")
print(f"\n🎯 Ready to execute Context Engineering workflow!")
print(f" Use workflow.run_context_engineering_workflow(feature_request)")
print(f" to execute the complete methodology.")
print(f"\n📚 Key Context Engineering Concepts Demonstrated:")
print(f" • Pattern extraction from existing codebase")
print(f" • Comprehensive context document generation")
print(f" • Validation loop creation with success criteria")
print(f" • PRP (Product Requirements Prompt) generation")
print(f" • Context-enhanced prompt engineering")
print(f" • Multi-agent workflow integration")
print(f"\n🎉 Context Engineering: 10x better than prompt engineering, 100x better than vibe coding!")
+20
View File
@@ -0,0 +1,20 @@
# PraisonAI Documentation
## Docs
- [AI Agents](https://docs.praison.ai/docs/agents/agents.md): Overview of all available PraisonAI agents and their capabilities
- [Data Analyst Agent](https://docs.praison.ai/docs/agents/data-analyst.md): Learn how to create AI agents for data analysis and insights generation.
- [Finance Agent](https://docs.praison.ai/docs/agents/finance.md): Learn how to create AI agents for financial analysis and investment recommendations.
- [Image Analysis Agent](https://docs.praison.ai/docs/agents/image.md): Learn how to create AI agents for image analysis and visual content understanding.
- [Image to Text Agent](https://docs.praison.ai/docs/agents/image-to-text.md): Learn how to create AI agents for converting images to textual descriptions and extracting text from images.
- [Markdown Agent](https://docs.praison.ai/docs/agents/markdown.md): Learn how to create AI agents for generating and formatting content in Markdown.
- [Planning Agent](https://docs.praison.ai/docs/agents/planning.md): Learn how to create AI agents for trip planning and itinerary generation.
- [Programming Agent](https://docs.praison.ai/docs/agents/programming.md): Learn how to create AI agents for code development, analysis, and execution.
- [Recommendation Agent](https://docs.praison.ai/docs/agents/recommendation.md): Learn how to create AI agents for personalized recommendations across various domains.
- [Research Agent](https://docs.praison.ai/docs/agents/research.md): Learn how to create AI agents for conducting comprehensive research and analysis.
- [Shopping Agent](https://docs.praison.ai/docs/agents/shopping.md): Learn how to create AI agents for price comparison and shopping assistance.
- [Single Agent](https://docs.praison.ai/docs/agents/single.md): Learn how to create a basic single-purpose AI agent for simple tasks.
- [Video Agent](https://docs.praison.ai/docs/agents/video.md): Learn how to create AI agents for video analysis and content understanding.
- [Web Search Agent](https://docs.praison.ai/docs/agents/websearch.md): Learn how to create AI agents for intelligent web searching and information gathering.
- [Wikipedia Agent](https://docs.praison.ai/docs/agents/wikipedia.md): Learn how to create AI agents for searching and extracting information from Wikipedia.
- [Agent Module](https://docs.praison.ai/docs/api/praisonaiagents/agent/agent.md): Documentation for the praisonaiagents.agent.agent module
@@ -82,7 +82,6 @@ def main():
backstory="""You are a knowledge management specialist who excels at
organizing, categorizing, and retrieving information from the knowledge base.
You ensure that knowledge is properly stored, indexed, and accessible.""",
knowledge=True,
knowledge_config=mongodb_knowledge_config,
memory=True,
verbose=True,
@@ -111,7 +110,6 @@ def main():
backstory="""You are a customer service representative who uses the
knowledge base and customer data to provide excellent support. You can
access customer history, product information, and troubleshooting guides.""",
knowledge=True,
knowledge_config=mongodb_knowledge_config,
tools=[mongodb_tools],
memory=True,
@@ -208,10 +206,18 @@ def main():
print("📊 Comprehensive Business System Results:")
print("=" * 60)
for i, result in enumerate(results, 1):
print(f"\n{i}. Task Result:")
print(f" Output: {result.raw[:200]}...")
print(f" Agent: {result.agent}")
# Handle results structure properly
if isinstance(results, (list, tuple)):
for i, result in enumerate(results, 1):
print(f"\n{i}. Task Result:")
if hasattr(result, 'raw') and hasattr(result, 'agent'):
print(f" Output: {result.raw[:200]}...")
print(f" Agent: {result.agent}")
else:
print(f" Output: {str(result)[:200]}...")
else:
print(f"\n1. Task Result:")
print(f" Output: {str(results)[:200]}...")
print("\n" + "=" * 60)
print("💾 Comprehensive MongoDB Integration Complete!")
@@ -222,10 +228,16 @@ def main():
print("-" * 40)
# Access different components
memory_system = business_system.memory
# Note: Direct memory access not available through business_system.memory
print("📋 Memory integration enabled through memory_config...")
print("Memory operations are handled internally by the agents")
# Access memory system directly
from praisonaiagents.memory import Memory
memory_system = Memory(config=mongodb_memory_config)
# Search business memories
print("📋 Searching business memories...")
print("\n📋 Searching business memories...")
business_memories = memory_system.search_long_term("business analysis", limit=3)
print(f"Found {len(business_memories)} business-related memories")
@@ -55,8 +55,8 @@ def main():
retrieve information from a comprehensive MongoDB knowledge base. You excel
at finding relevant information, synthesizing knowledge from multiple sources,
and providing accurate, context-aware responses.""",
knowledge=True,
knowledge_config=mongodb_knowledge_config,
knowledge=[os.path.join(os.path.dirname(__file__), "llms.md")],
memory=True,
verbose=True,
llm="gpt-4o-mini"
@@ -139,10 +139,18 @@ def main():
print("📊 Knowledge Management System Results:")
print("=" * 60)
for i, result in enumerate(results, 1):
print(f"\n{i}. Task Result:")
print(f" Output: {result.raw[:200]}...")
print(f" Agent: {result.agent}")
# Handle results structure properly
if isinstance(results, (list, tuple)):
for i, result in enumerate(results, 1):
print(f"\n{i}. Task Result:")
if hasattr(result, 'raw') and hasattr(result, 'agent'):
print(f" Output: {result.raw[:200]}...")
print(f" Agent: {result.agent}")
else:
print(f" Output: {str(result)[:200]}...")
else:
print(f"\n1. Task Result:")
print(f" Output: {str(results)[:200]}...")
print("\n" + "=" * 60)
print("💾 MongoDB Knowledge Integration Complete!")
@@ -107,10 +107,18 @@ def main():
print("📊 Research System Results:")
print("=" * 60)
for i, result in enumerate(results, 1):
print(f"\n{i}. Task Result:")
print(f" Output: {result.raw[:200]}...")
print(f" Agent: {result.agent}")
# Handle results structure properly
if isinstance(results, (list, tuple)):
for i, result in enumerate(results, 1):
print(f"\n{i}. Task Result:")
if hasattr(result, 'agent'):
print(f" Output: {str(result)[:200]}...")
print(f" Agent: {result.agent}")
else:
print(f" Output: {str(result)[:200]}...")
else:
print(f"\n1. Task Result:")
print(f" Output: {str(results)[:200]}...")
print("\n" + "=" * 60)
print("💾 MongoDB Memory Integration Complete!")
@@ -120,20 +128,36 @@ def main():
print("\n🔍 Demonstrating Memory Retrieval:")
print("-" * 40)
# Access the memory system directly
memory_system = research_system.memory
# Search for AI-related memories
ai_memories = memory_system.search_long_term("AI machine learning", limit=3)
print(f"Found {len(ai_memories)} AI-related memories:")
for memory in ai_memories:
print(f" - {memory.get('text', '')[:100]}...")
# Search for MongoDB-related memories
mongo_memories = memory_system.search_long_term("MongoDB vector search", limit=3)
print(f"\nFound {len(mongo_memories)} MongoDB-related memories:")
for memory in mongo_memories:
print(f" - {memory.get('text', '')[:100]}...")
# Note: Direct memory access not available through research_system.memory
# Instead, use Memory class directly for verification
try:
from praisonaiagents.memory import Memory
# Create Memory instance with MongoDB config for verification
memory_system = Memory(mongodb_memory_config)
# Search for AI-related memories
ai_memories = memory_system.search_long_term("AI machine learning", limit=3)
print(f"Found {len(ai_memories)} AI-related memories:")
for memory in ai_memories:
if hasattr(memory, 'get'):
print(f" - {memory.get('text', '')[:100]}...")
else:
print(f" - {str(memory)[:100]}...")
# Search for MongoDB-related memories
mongo_memories = memory_system.search_long_term("MongoDB vector search", limit=3)
print(f"\nFound {len(mongo_memories)} MongoDB-related memories:")
for memory in mongo_memories:
if hasattr(memory, 'get'):
print(f" - {memory.get('text', '')[:100]}...")
else:
print(f" - {str(memory)[:100]}...")
except ImportError:
print("Memory features not available - install with: pip install 'praisonaiagents[memory]'")
except Exception as mem_error:
print(f"Memory verification error: {mem_error}")
print("\n✅ MongoDB Memory Example Completed Successfully!")
@@ -121,16 +121,26 @@ def main():
# Execute the MongoDB operations pipeline
try:
results = mongodb_system.start()
results = mongodb_system.start(return_dict=True)
print("\n" + "=" * 60)
print("📊 MongoDB Tools System Results:")
print("=" * 60)
print(results)
for i, result in enumerate(results, 1):
print(f"\n{i}. Task Result:")
print(f" Output: {result.raw[:200]}...")
print(f" Agent: {result.agent}")
# Handle results structure properly
if isinstance(results, dict) and 'task_results' in results:
task_results = results['task_results']
for i, result in task_results.items():
print(f"\n{i}. Task Result:")
if hasattr(result, 'raw') and hasattr(result, 'agent'):
print(f" Output: {result.raw[:200]}...")
print(f" Agent: {result.agent}")
else:
print(f" Result: {str(result)[:200]}...")
else:
print(f"\n1. Task Result:")
print(f" Output: {str(results)[:200]}...")
print("\n" + "=" * 60)
print("💾 MongoDB Tools Integration Complete!")
@@ -0,0 +1,143 @@
"""
Multi-Agent Workflow with Performance Monitoring - Example 2
Demonstrates performance monitoring in a multi-agent workflow:
- Multiple agents with different roles
- Inter-agent communication tracking
- Workflow performance analysis
- Task delegation timing
Shows how to monitor complex multi-agent systems.
"""
from praisonaiagents import Agent, Task, PraisonAIAgents
from praisonaiagents.telemetry import (
monitor_function, track_api_call, get_function_stats,
analyze_function_flow, generate_comprehensive_report
)
import time
@monitor_function("research_task")
def setup_research_task():
"""Set up research task with monitoring."""
return Task(
name="research_task",
description="Research the topic of renewable energy trends",
expected_output="A comprehensive research summary"
)
@monitor_function("analysis_task")
def setup_analysis_task():
"""Set up analysis task with monitoring."""
return Task(
name="analysis_task",
description="Analyze the research data and identify key trends",
expected_output="Key insights and trend analysis"
)
@monitor_function("report_task")
def setup_report_task():
"""Set up report generation task with monitoring."""
return Task(
name="report_task",
description="Create a final report combining research and analysis",
expected_output="A well-structured final report"
)
@monitor_function("agent_creation")
def create_agents():
"""Create specialized agents with monitoring."""
researcher = Agent(
name="researcher",
role="Research Specialist",
goal="Gather comprehensive information on renewable energy",
backstory="Expert researcher with deep knowledge of energy sector",
instructions="You are a thorough researcher. Provide detailed, fact-based information.",
llm="gpt-4o-mini"
)
analyst = Agent(
name="analyst",
role="Data Analyst",
goal="Analyze trends and patterns in renewable energy data",
backstory="Experienced data analyst specializing in energy markets",
instructions="You are an analytical thinker. Focus on trends, patterns, and insights.",
llm="gpt-4o-mini"
)
writer = Agent(
name="writer",
role="Report Writer",
goal="Create clear, compelling reports",
backstory="Professional technical writer with expertise in energy sector",
instructions="You are a skilled writer. Create clear, well-structured reports.",
llm="gpt-4o-mini"
)
return researcher, analyst, writer
@monitor_function("workflow_execution")
def main():
"""Main function demonstrating multi-agent workflow with performance monitoring."""
print("=" * 70)
print("EXAMPLE 2: Multi-Agent Workflow with Performance Monitoring")
print("=" * 70)
# Create agents with monitoring
researcher, analyst, writer = create_agents()
# Set up tasks with monitoring
research_task = setup_research_task()
analysis_task = setup_analysis_task()
report_task = setup_report_task()
# Assign agents to tasks
research_task.agent = researcher
analysis_task.agent = analyst
report_task.agent = writer
# Create workflow
with track_api_call("multi_agent_workflow"):
print("\n🚀 Starting multi-agent workflow...")
workflow = PraisonAIAgents(
agents=[researcher, analyst, writer],
tasks=[research_task, analysis_task, report_task],
process="sequential",
verbose=True
)
result = workflow.start()
# Display comprehensive performance analysis
print("\n" + "=" * 70)
print("📊 MULTI-AGENT PERFORMANCE ANALYSIS")
print("=" * 70)
# Function statistics
print("\n📈 Function Performance Statistics:")
stats = get_function_stats()
for func_name, data in stats.items():
print(f" {func_name}:")
print(f" Calls: {data['call_count']}")
avg_time = data['total_time'] / data['call_count'] if data['call_count'] > 0 else 0
print(f" Avg Time: {avg_time:.3f}s")
print(f" Total Time: {data['total_time']:.3f}s")
# Flow analysis
print("\n🔄 Execution Flow Analysis:")
flow_analysis = analyze_function_flow()
print(f" Total Functions Monitored: {len(flow_analysis.get('functions', []))}")
print(f" Execution Chains: {len(flow_analysis.get('call_chains', []))}")
# Comprehensive report
print("\n📋 Comprehensive Performance Report:")
report = generate_comprehensive_report()
print(report[:500] + "..." if len(report) > 500 else report)
return result
if __name__ == "__main__":
result = main()
print(f"\n✅ Multi-agent workflow completed successfully!")
@@ -0,0 +1,119 @@
"""
Agent with Tools Performance Monitoring - Example 3
Demonstrates performance monitoring for agents using tools:
- Tool execution timing
- Search operation performance
- Tool call success/failure tracking
- Tool usage analysis
Shows how to monitor agent performance when using external tools.
"""
from praisonaiagents import Agent
from praisonaiagents.tools import duckduckgo, wikipedia
from praisonaiagents.telemetry import (
monitor_function, track_api_call, get_api_stats,
get_slowest_functions, performance_monitor
)
import time
@monitor_function("search_with_duckduckgo")
def perform_web_search(query):
"""Perform web search with performance monitoring."""
print(f"🔍 Searching web for: {query}")
with track_api_call("duckduckgo_search"):
# Simulate search time
time.sleep(0.2)
return f"Search results for: {query}"
@monitor_function("wikipedia_lookup")
def perform_wikipedia_search(topic):
"""Perform Wikipedia search with performance monitoring."""
print(f"📚 Looking up Wikipedia for: {topic}")
with track_api_call("wikipedia_search"):
# Simulate Wikipedia lookup
time.sleep(0.15)
return f"Wikipedia information about: {topic}"
@monitor_function("agent_with_tools_execution")
def main():
"""Main function demonstrating agent with tools and performance monitoring."""
print("=" * 70)
print("EXAMPLE 3: Agent with Tools Performance Monitoring")
print("=" * 70)
# Create agent with tools
agent = Agent(
instructions="""You are a research assistant that helps find information.
Use the available tools to search for information and provide comprehensive answers.
Be thorough in your research and cite your sources.""",
llm="gpt-4o-mini",
tools=[duckduckgo, wikipedia] # Add search tools
)
# Test queries to demonstrate tool performance monitoring
queries = [
"What are the latest developments in artificial intelligence?",
"Tell me about renewable energy innovations in 2024",
"How do neural networks work?"
]
results = []
for i, query in enumerate(queries, 1):
print(f"\n🎯 Query {i}: {query}")
# Monitor the search operations
perform_web_search(query)
perform_wikipedia_search(query)
# Track the full agent execution
with track_api_call(f"agent_query_{i}"):
print(f"\n🚀 Processing with agent...")
# Note: In a real implementation, the agent would use the tools automatically
# For demo purposes, we're simulating the tool usage
result = agent.start(f"Research and answer: {query}")
results.append(result)
print(f"📝 Agent Response: {result[:200]}..." if len(result) > 200 else f"📝 Agent Response: {result}")
# Performance Analysis
print("\n" + "=" * 70)
print("📊 TOOLS PERFORMANCE ANALYSIS")
print("=" * 70)
# API call statistics
print("\n📈 API Call Performance:")
api_stats = get_api_stats()
for api_name, stats in api_stats.items():
print(f" {api_name}:")
print(f" Total Calls: {stats['call_count']}")
print(f" Avg Response Time: {stats['total_time']/stats['call_count']:.3f}s")
print(f" Success Rate: {stats['success_count']/stats['call_count']*100:.1f}%")
print(f" Total Time: {stats['total_time']:.3f}s")
# Slowest functions analysis
print("\n🐌 Slowest Functions:")
slowest = get_slowest_functions()
for func_name, avg_time in slowest:
print(f" {func_name}: {avg_time:.3f}s average")
# Real-time performance data
print("\n⚡ Real-time Performance Data:")
current_stats = performance_monitor.get_function_performance()
print(f" Functions Monitored: {len(current_stats)}")
total_function_calls = sum(stats['call_count'] for stats in current_stats.values())
total_execution_time = sum(stats['total_time'] for stats in current_stats.values())
print(f" Total Function Calls: {total_function_calls}")
print(f" Total Execution Time: {total_execution_time:.3f}s")
print(f" Average Time per Call: {total_execution_time/total_function_calls:.3f}s" if total_function_calls > 0 else " No calls recorded")
return results
if __name__ == "__main__":
results = main()
print(f"\n✅ Tool monitoring example completed! Processed {len(results)} queries.")
@@ -0,0 +1,167 @@
"""
Async Agent with Performance Monitoring - Example 4
Demonstrates performance monitoring for asynchronous agent operations:
- Async function execution timing
- Concurrent task performance
- Async API call tracking
- Parallel operation analysis
Shows how to monitor performance in async/await workflows.
"""
import asyncio
import time
from praisonaiagents import Agent
from praisonaiagents.telemetry import (
monitor_function, track_api_call, get_performance_report,
get_function_stats, visualize_execution_flow
)
@monitor_function("async_data_processing")
async def process_data_async(data_id: int):
"""Async data processing with performance monitoring."""
print(f"⚙️ Processing data {data_id} asynchronously...")
await asyncio.sleep(0.1) # Simulate async processing
return f"Processed data {data_id}"
@monitor_function("async_api_call")
async def simulate_api_call(endpoint: str, delay: float = 0.2):
"""Simulate async API call with performance monitoring."""
print(f"🌐 Calling API endpoint: {endpoint}")
await asyncio.sleep(delay) # Simulate network delay
return f"Response from {endpoint}"
@monitor_function("batch_processing")
async def process_batch(batch_size: int = 3):
"""Process multiple items concurrently with monitoring."""
print(f"📦 Processing batch of {batch_size} items...")
# Create concurrent tasks
tasks = []
for i in range(batch_size):
tasks.append(process_data_async(i + 1))
# Wait for all tasks to complete
results = await asyncio.gather(*tasks)
print(f"✅ Batch processing completed: {len(results)} items processed")
return results
@monitor_function("async_agent_execution")
async def run_async_agent(query: str):
"""Run agent with async operations and monitoring."""
# Create agent
agent = Agent(
instructions="You are an efficient AI assistant that processes requests quickly",
llm="gpt-4o-mini"
)
# Track the agent execution
with track_api_call("async_agent_request"):
print(f"🚀 Running async agent for query: {query}")
# For demo purposes, we simulate async agent execution
# In real scenarios, you'd use agent.astart() or similar async method when available
await asyncio.sleep(0.1) # Simulate async processing time
result = agent.start(query)
# Simulate some async post-processing
await asyncio.sleep(0.05)
return result
@monitor_function("concurrent_agents")
async def run_concurrent_agents():
"""Run multiple agents concurrently with monitoring."""
print("🔄 Running multiple agents concurrently...")
queries = [
"What is machine learning?",
"Explain quantum computing",
"How does blockchain work?"
]
# Create concurrent agent tasks
agent_tasks = []
for i, query in enumerate(queries):
task = run_async_agent(query)
agent_tasks.append(task)
# Execute all agents concurrently
results = await asyncio.gather(*agent_tasks)
return results
async def main():
"""Main async function demonstrating async agent performance monitoring."""
print("=" * 70)
print("EXAMPLE 4: Async Agent Performance Monitoring")
print("=" * 70)
start_time = time.time()
# Run async operations with monitoring
print("\n📊 Phase 1: Batch Processing")
batch_results = await process_batch(3)
print("\n🌐 Phase 2: API Calls")
api_tasks = [
simulate_api_call("user-service", 0.1),
simulate_api_call("data-service", 0.15),
simulate_api_call("analytics-service", 0.2)
]
api_results = await asyncio.gather(*api_tasks)
print("\n🤖 Phase 3: Concurrent Agents")
agent_results = await run_concurrent_agents()
total_time = time.time() - start_time
# Performance Analysis
print("\n" + "=" * 70)
print("📊 ASYNC PERFORMANCE ANALYSIS")
print("=" * 70)
print(f"\n⏱️ Total Execution Time: {total_time:.3f}s")
# Function statistics
print("\n📈 Function Performance:")
stats = get_function_stats()
for func_name, data in stats.items():
avg_time = data['total_time'] / data['call_count'] if data['call_count'] > 0 else 0
print(f" {func_name}:")
print(f" Calls: {data['call_count']}")
print(f" Avg Time: {avg_time:.3f}s")
print(f" Total Time: {data['total_time']:.3f}s")
# Execution flow visualization
print("\n🔄 Execution Flow:")
try:
flow_viz = visualize_execution_flow()
print(" Flow visualization generated successfully")
print(f" Flow data length: {len(str(flow_viz))}")
except Exception as e:
print(f" Flow visualization: {str(e)}")
# Performance report
print("\n📋 Performance Report:")
report = get_performance_report()
print(report[:300] + "..." if len(report) > 300 else report)
# Results summary
print(f"\n✅ Results Summary:")
print(f" Batch items processed: {len(batch_results)}")
print(f" API calls completed: {len(api_results)}")
print(f" Agent queries processed: {len(agent_results)}")
return {
'batch_results': batch_results,
'api_results': api_results,
'agent_results': agent_results,
'total_time': total_time
}
if __name__ == "__main__":
# Run the async main function
result = asyncio.run(main())
print(f"\n🎉 Async monitoring example completed successfully!")
@@ -0,0 +1,195 @@
"""
Error Handling with Performance Monitoring - Example 5
Demonstrates performance monitoring for error scenarios:
- Error rate tracking
- Failed operation timing
- Recovery mechanism performance
- Exception handling with monitoring
Shows how to monitor performance when things go wrong.
"""
import random
import time
from praisonaiagents import Agent
from praisonaiagents.telemetry import (
monitor_function, track_api_call, get_function_stats,
get_api_stats, performance_monitor
)
@monitor_function("unreliable_operation")
def simulate_unreliable_operation(operation_id: int, failure_rate: float = 0.3):
"""Simulate an operation that sometimes fails."""
print(f"⚙️ Running operation {operation_id}...")
# Simulate processing time
time.sleep(random.uniform(0.05, 0.2))
# Random failure simulation
if random.random() < failure_rate:
raise RuntimeError(f"Operation {operation_id} failed!")
return f"Operation {operation_id} succeeded"
@monitor_function("retry_with_backoff")
def retry_operation(operation_id: int, max_retries: int = 3):
"""Retry an operation with exponential backoff and monitoring."""
for attempt in range(max_retries):
try:
print(f"🔄 Attempt {attempt + 1} for operation {operation_id}")
result = simulate_unreliable_operation(operation_id)
print(f"✅ Operation {operation_id} succeeded on attempt {attempt + 1}")
return result
except Exception as e:
print(f"❌ Attempt {attempt + 1} failed: {e}")
if attempt < max_retries - 1:
# Exponential backoff
sleep_time = 2 ** attempt * 0.1
print(f"⏳ Waiting {sleep_time:.1f}s before retry...")
time.sleep(sleep_time)
else:
print(f"💥 All {max_retries} attempts failed for operation {operation_id}")
raise
@monitor_function("robust_agent_execution")
def run_robust_agent(query: str, with_error_simulation: bool = True):
"""Run agent with error handling and performance monitoring."""
agent = Agent(
instructions="""You are a resilient AI assistant.
Even when facing challenges, you provide helpful responses.
If you encounter errors, acknowledge them and offer alternatives.""",
llm="gpt-4o-mini"
)
try:
with track_api_call("robust_agent_request"):
print(f"🚀 Processing query: {query}")
if with_error_simulation and random.random() < 0.2:
# Simulate API error
raise Exception("Simulated API error")
result = agent.start(query)
return result
except Exception as e:
print(f"⚠️ Error during agent execution: {e}")
# Fallback response with monitoring
with track_api_call("fallback_response"):
print("🔄 Using fallback response mechanism...")
time.sleep(0.1) # Simulate fallback processing
return f"I encountered an error while processing '{query}', but I'm designed to be resilient. Let me provide a general response about this topic."
@monitor_function("batch_processing_with_errors")
def process_batch_with_error_handling(batch_size: int = 5):
"""Process a batch of operations with comprehensive error handling."""
print(f"📦 Processing batch of {batch_size} operations with error monitoring...")
results = []
errors = []
for i in range(batch_size):
try:
result = retry_operation(i + 1)
results.append(result)
except Exception as e:
errors.append(f"Operation {i + 1}: {str(e)}")
return results, errors
def main():
"""Main function demonstrating error handling with performance monitoring."""
print("=" * 70)
print("EXAMPLE 5: Error Handling Performance Monitoring")
print("=" * 70)
# Phase 1: Batch processing with errors
print("\n📊 Phase 1: Batch Processing with Error Handling")
successes, failures = process_batch_with_error_handling(5)
print(f"\n📈 Batch Results:")
print(f" Successful operations: {len(successes)}")
print(f" Failed operations: {len(failures)}")
# Phase 2: Agent execution with error scenarios
print("\n🤖 Phase 2: Agent Execution with Error Scenarios")
test_queries = [
"What is artificial intelligence?",
"How do computers work?",
"Explain quantum physics",
]
agent_results = []
for query in test_queries:
try:
result = run_robust_agent(query, with_error_simulation=True)
agent_results.append(result)
print(f"✅ Query processed: '{query[:30]}...'")
except Exception as e:
print(f"❌ Query failed: '{query[:30]}...' - {e}")
# Performance Analysis with Error Metrics
print("\n" + "=" * 70)
print("📊 ERROR HANDLING PERFORMANCE ANALYSIS")
print("=" * 70)
# Function performance with error rates
print("\n📈 Function Performance (including errors):")
stats = get_function_stats()
for func_name, data in stats.items():
avg_time = data['total_time'] / data['call_count'] if data['call_count'] > 0 else 0
error_rate = data['error_count'] / data['call_count'] * 100 if data['call_count'] > 0 else 0
success_rate = 100 - error_rate
print(f" {func_name}:")
print(f" Calls: {data['call_count']}")
print(f" Errors: {data['error_count']}")
print(f" Success Rate: {success_rate:.1f}%")
print(f" Avg Time: {avg_time:.3f}s")
# API call reliability
print("\n🌐 API Call Reliability:")
api_stats = get_api_stats()
for api_name, stats in api_stats.items():
success_rate = stats['success_count'] / stats['call_count'] * 100 if stats['call_count'] > 0 else 0
avg_time = stats['total_time'] / stats['call_count'] if stats['call_count'] > 0 else 0
print(f" {api_name}:")
print(f" Success Rate: {success_rate:.1f}%")
print(f" Average Response Time: {avg_time:.3f}s")
print(f" Total Calls: {stats['call_count']}")
# Error patterns analysis
print("\n⚠️ Error Patterns:")
total_operations = sum(s['call_count'] for s in api_stats.values())
total_errors = sum(s['error_count'] for s in api_stats.values())
overall_success_rate = ((total_operations - total_errors) / total_operations * 100) if total_operations > 0 else 0
print(f" Total Operations: {total_operations}")
print(f" Total Errors: {total_errors}")
print(f" Overall Success Rate: {overall_success_rate:.1f}%")
# Recommendations
print(f"\n💡 Performance Recommendations:")
if overall_success_rate < 90:
print(" - Consider implementing more robust error handling")
print(" - Review retry strategies and backoff algorithms")
if total_errors > 0:
print(f" - Investigate {total_errors} errors for root cause analysis")
print(" - Monitor error patterns for system reliability improvements")
return {
'successful_operations': len(successes),
'failed_operations': len(failures),
'agent_results': len(agent_results),
'overall_success_rate': overall_success_rate
}
if __name__ == "__main__":
result = main()
print(f"\n✅ Error handling monitoring completed!")
print(f"Success rate: {result['overall_success_rate']:.1f}%")
@@ -0,0 +1,218 @@
"""
Memory Agent with Performance Monitoring - Example 6
Demonstrates performance monitoring for agents with memory capabilities:
- Memory storage and retrieval timing
- Knowledge base operations
- Session persistence performance
- Memory search optimization
Shows how to monitor performance in stateful agents.
"""
import time
from praisonaiagents import Agent
from praisonaiagents.telemetry import (
monitor_function, track_api_call, get_performance_report,
get_slowest_functions, analyze_performance_trends
)
# Simulate memory operations since actual memory might not be available
@monitor_function("memory_storage")
def store_memory(key: str, value: str, memory_type: str = "short_term"):
"""Simulate memory storage operation with monitoring."""
print(f"💾 Storing {memory_type} memory: {key}")
# Simulate storage time - short term is faster
storage_time = 0.05 if memory_type == "short_term" else 0.15
time.sleep(storage_time)
return f"Stored {key} in {memory_type} memory"
@monitor_function("memory_retrieval")
def retrieve_memory(key: str, memory_type: str = "short_term"):
"""Simulate memory retrieval operation with monitoring."""
print(f"🔍 Retrieving {memory_type} memory: {key}")
# Simulate retrieval time
retrieval_time = 0.03 if memory_type == "short_term" else 0.12
time.sleep(retrieval_time)
return f"Retrieved {key} from {memory_type} memory"
@monitor_function("knowledge_search")
def search_knowledge_base(query: str):
"""Simulate knowledge base search with monitoring."""
print(f"📚 Searching knowledge base for: {query}")
# Simulate vector search time
time.sleep(0.08)
return f"Knowledge search results for: {query}"
@monitor_function("memory_consolidation")
def consolidate_memories(session_id: str):
"""Simulate memory consolidation process."""
print(f"🔄 Consolidating memories for session: {session_id}")
# Simulate consolidation steps
store_memory("consolidated_session", session_id, "long_term")
retrieve_memory("recent_interactions", "short_term")
time.sleep(0.1)
return f"Memories consolidated for session: {session_id}"
@monitor_function("stateful_agent_interaction")
def run_stateful_agent(query: str, session_id: str, conversation_turn: int):
"""Run agent with memory operations and monitoring."""
# Simulate memory-enabled agent
agent = Agent(
instructions="""You are a memory-enabled AI assistant. You can remember
previous conversations and build on past interactions. Use your memory
to provide personalized and contextual responses.""",
llm="gpt-4o-mini"
)
# Pre-processing: retrieve relevant memories
with track_api_call("memory_preprocessing"):
print(f"🧠 Pre-processing memories for turn {conversation_turn}")
# Retrieve session context
if conversation_turn > 1:
retrieve_memory(f"session_{session_id}", "short_term")
# Search for relevant knowledge
search_knowledge_base(query)
time.sleep(0.05) # Simulate context preparation
# Main agent execution
with track_api_call("memory_agent_execution"):
print(f"🚀 Processing query with memory context...")
result = agent.start(query)
# Post-processing: store new memories
with track_api_call("memory_postprocessing"):
print(f"💭 Post-processing: storing interaction memories")
# Store the interaction
store_memory(f"interaction_{conversation_turn}", query, "short_term")
store_memory(f"response_{conversation_turn}", result, "short_term")
# Update session memory
store_memory(f"session_{session_id}", f"Turn {conversation_turn} completed", "short_term")
time.sleep(0.03) # Simulate memory update
return result
@monitor_function("multi_turn_conversation")
def simulate_multi_turn_conversation():
"""Simulate a multi-turn conversation with memory."""
session_id = "demo_session_001"
conversation = [
"Hello, I'm interested in learning about machine learning",
"What are the main types of machine learning algorithms?",
"Can you give me examples of supervised learning?",
"How does this relate to what we discussed about machine learning earlier?"
]
results = []
for turn, query in enumerate(conversation, 1):
print(f"\n💬 Conversation Turn {turn}")
print(f"User: {query}")
result = run_stateful_agent(query, session_id, turn)
results.append(result)
print(f"Agent: {result[:100]}..." if len(result) > 100 else f"Agent: {result}")
# Simulate user thinking time
time.sleep(0.1)
# End of conversation: consolidate memories
consolidate_memories(session_id)
return results
def main():
"""Main function demonstrating memory agent performance monitoring."""
print("=" * 70)
print("EXAMPLE 6: Memory Agent Performance Monitoring")
print("=" * 70)
# Warm up the memory system
print("\n🚀 Phase 1: Memory System Initialization")
store_memory("system_initialized", "true", "long_term")
search_knowledge_base("initialization")
# Run multi-turn conversation
print("\n💬 Phase 2: Multi-Turn Conversation with Memory")
conversation_results = simulate_multi_turn_conversation()
# Memory performance analysis
print("\n" + "=" * 70)
print("📊 MEMORY PERFORMANCE ANALYSIS")
print("=" * 70)
# Performance trends analysis
print("\n📈 Performance Trends:")
try:
trends = analyze_performance_trends()
print(f" Performance trend analysis completed")
if trends:
print(f" Functions analyzed: {len(trends.get('functions', []))}")
except Exception as e:
print(f" Trends analysis: {str(e)}")
# Slowest operations
print("\n🐌 Slowest Memory Operations:")
slowest = get_slowest_functions()
for func_name, avg_time in slowest[:5]: # Top 5 slowest
print(f" {func_name}: {avg_time:.3f}s average")
# Memory operation breakdown
print("\n💾 Memory Operation Breakdown:")
from praisonaiagents.telemetry import get_function_stats
stats = get_function_stats()
memory_operations = {k: v for k, v in stats.items() if 'memory' in k.lower()}
for op_name, data in memory_operations.items():
avg_time = data['total_time'] / data['call_count'] if data['call_count'] > 0 else 0
print(f" {op_name}:")
print(f" Operations: {data['call_count']}")
print(f" Avg Time: {avg_time:.3f}s")
print(f" Total Time: {data['total_time']:.3f}s")
# Session efficiency metrics
total_memory_ops = sum(data['call_count'] for data in memory_operations.values())
total_memory_time = sum(data['total_time'] for data in memory_operations.values())
print(f"\n🔍 Session Efficiency:")
print(f" Total Memory Operations: {total_memory_ops}")
print(f" Total Memory Time: {total_memory_time:.3f}s")
print(f" Avg Time per Memory Op: {total_memory_time/total_memory_ops:.3f}s" if total_memory_ops > 0 else " No memory operations")
# Performance report
print("\n📋 Comprehensive Performance Report:")
report = get_performance_report()
print(report[:400] + "..." if len(report) > 400 else report)
# Memory optimization suggestions
print(f"\n💡 Memory Optimization Suggestions:")
if total_memory_time > 1.0:
print(" - Consider implementing memory caching strategies")
print(" - Optimize vector search indexes for faster retrieval")
if total_memory_ops > 10:
print(" - Consider batching memory operations for efficiency")
print(" - Monitor memory consolidation frequency vs performance")
return {
'conversation_turns': len(conversation_results),
'memory_operations': total_memory_ops,
'total_memory_time': total_memory_time
}
if __name__ == "__main__":
result = main()
print(f"\n✅ Memory agent monitoring completed!")
print(f"Processed {result['conversation_turns']} conversation turns")
print(f"Executed {result['memory_operations']} memory operations in {result['total_memory_time']:.3f}s")
@@ -0,0 +1,264 @@
"""
Hierarchical Agents with Performance Monitoring - Example 7
Demonstrates performance monitoring for hierarchical agent systems:
- Manager-worker agent relationships
- Task delegation timing
- Hierarchical decision making performance
- Cross-level communication monitoring
Shows how to monitor complex organizational agent structures.
"""
import time
from praisonaiagents import Agent, Task, PraisonAIAgents
from praisonaiagents.telemetry import (
monitor_function, track_api_call, get_function_stats,
visualize_execution_flow, generate_comprehensive_report
)
@monitor_function("manager_decision_making")
def make_management_decision(task_description: str):
"""Simulate manager decision making process."""
print(f"👔 Manager analyzing task: {task_description}")
# Simulate decision complexity
time.sleep(0.1)
# Simple task routing logic
if "research" in task_description.lower():
return "research_worker"
elif "analysis" in task_description.lower():
return "analysis_worker"
elif "report" in task_description.lower():
return "writing_worker"
else:
return "general_worker"
@monitor_function("task_delegation")
def delegate_task(task: str, worker_type: str):
"""Simulate task delegation to workers."""
print(f"📤 Delegating to {worker_type}: {task}")
time.sleep(0.05) # Delegation overhead
return f"Task delegated to {worker_type}"
@monitor_function("worker_execution")
def execute_worker_task(worker_name: str, task_description: str):
"""Simulate worker executing assigned task."""
print(f"⚙️ {worker_name} executing: {task_description}")
# Different workers have different execution times
if "research" in worker_name:
time.sleep(0.2) # Research takes longer
elif "analysis" in worker_name:
time.sleep(0.15) # Analysis is moderate
elif "writing" in worker_name:
time.sleep(0.1) # Writing is relatively quick
else:
time.sleep(0.12) # General tasks
return f"{worker_name} completed: {task_description}"
@monitor_function("result_aggregation")
def aggregate_results(results: list):
"""Simulate aggregating results from multiple workers."""
print(f"📊 Aggregating {len(results)} worker results...")
time.sleep(0.08) # Aggregation time
return f"Aggregated {len(results)} results into final output"
@monitor_function("hierarchical_workflow")
def run_hierarchical_workflow():
"""Run a complete hierarchical workflow with monitoring."""
# Create manager agent
manager = Agent(
name="project_manager",
role="Project Manager",
goal="Efficiently coordinate team efforts and ensure project success",
backstory="Experienced project manager with expertise in team coordination",
instructions="""You are a project manager. Break down complex projects into tasks,
assign them to appropriate team members, and coordinate the overall effort.""",
llm="gpt-4o-mini"
)
# Create worker agents
research_worker = Agent(
name="research_specialist",
role="Research Specialist",
goal="Conduct thorough research on assigned topics",
backstory="Expert researcher with deep analytical skills",
instructions="You are a research specialist. Provide detailed, factual information.",
llm="gpt-4o-mini"
)
analysis_worker = Agent(
name="data_analyst",
role="Data Analyst",
goal="Analyze data and identify patterns and insights",
backstory="Experienced data analyst with strong analytical thinking",
instructions="You are a data analyst. Focus on patterns, trends, and insights.",
llm="gpt-4o-mini"
)
writing_worker = Agent(
name="technical_writer",
role="Technical Writer",
goal="Create clear, well-structured documentation and reports",
backstory="Professional writer specializing in technical content",
instructions="You are a technical writer. Create clear, comprehensive reports.",
llm="gpt-4o-mini"
)
# Project tasks
project_tasks = [
"Research current trends in artificial intelligence",
"Analyze the impact of AI on business productivity",
"Write a comprehensive report on AI implementation strategies"
]
# Execute hierarchical workflow
with track_api_call("hierarchical_project_execution"):
print("🏢 Starting hierarchical project execution...")
all_results = []
for task_desc in project_tasks:
print(f"\n📋 Processing project task: {task_desc}")
# Manager makes decision
assigned_worker = make_management_decision(task_desc)
# Delegate task
delegation_result = delegate_task(task_desc, assigned_worker)
# Select appropriate worker
if "research" in assigned_worker:
worker = research_worker
worker_name = "research_specialist"
elif "analysis" in assigned_worker:
worker = analysis_worker
worker_name = "data_analyst"
elif "writing" in assigned_worker:
worker = writing_worker
worker_name = "technical_writer"
else:
worker = research_worker # Default
worker_name = "research_specialist"
# Worker executes task
worker_result = execute_worker_task(worker_name, task_desc)
# Simulate actual agent execution
with track_api_call(f"{worker_name}_llm_execution"):
agent_result = worker.start(task_desc)
all_results.append(agent_result)
# Manager aggregates results
final_result = aggregate_results(all_results)
# Final manager review
with track_api_call("manager_final_review"):
print("👔 Manager conducting final review...")
manager_review = manager.start(
f"Review and synthesize these team outputs: {[r[:100] + '...' for r in all_results]}"
)
return all_results, manager_review
def main():
"""Main function demonstrating hierarchical agents performance monitoring."""
print("=" * 75)
print("EXAMPLE 7: Hierarchical Agents Performance Monitoring")
print("=" * 75)
# Run hierarchical workflow
print("\n🏢 Running Hierarchical Agent Workflow")
worker_results, manager_review = run_hierarchical_workflow()
print(f"\n✅ Workflow completed with {len(worker_results)} task results")
# Performance Analysis
print("\n" + "=" * 75)
print("📊 HIERARCHICAL PERFORMANCE ANALYSIS")
print("=" * 75)
# Role-based performance breakdown
print("\n👥 Role-Based Performance:")
stats = get_function_stats()
# Categorize functions by role
management_functions = [f for f in stats.keys() if 'manager' in f or 'decision' in f or 'delegation' in f or 'aggregation' in f]
worker_functions = [f for f in stats.keys() if 'worker' in f or 'execution' in f]
coordination_functions = [f for f in stats.keys() if 'hierarchical' in f or 'workflow' in f]
print(" 📋 Management Functions:")
for func in management_functions:
data = stats[func]
avg_time = data['total_time'] / data['call_count'] if data['call_count'] > 0 else 0
print(f" {func}: {data['call_count']} calls, {avg_time:.3f}s avg")
print("\n ⚙️ Worker Functions:")
for func in worker_functions:
data = stats[func]
avg_time = data['total_time'] / data['call_count'] if data['call_count'] > 0 else 0
print(f" {func}: {data['call_count']} calls, {avg_time:.3f}s avg")
print("\n 🔄 Coordination Functions:")
for func in coordination_functions:
data = stats[func]
avg_time = data['total_time'] / data['call_count'] if data['call_count'] > 0 else 0
print(f" {func}: {data['call_count']} calls, {avg_time:.3f}s avg")
# Hierarchy efficiency metrics
total_management_time = sum(stats[f]['total_time'] for f in management_functions)
total_worker_time = sum(stats[f]['total_time'] for f in worker_functions)
total_coordination_time = sum(stats[f]['total_time'] for f in coordination_functions)
print(f"\n⚖️ Hierarchy Efficiency Metrics:")
print(f" Management Overhead: {total_management_time:.3f}s")
print(f" Worker Execution Time: {total_worker_time:.3f}s")
print(f" Coordination Overhead: {total_coordination_time:.3f}s")
total_time = total_management_time + total_worker_time + total_coordination_time
if total_time > 0:
print(f" Worker Efficiency: {total_worker_time/total_time*100:.1f}%")
print(f" Management Overhead: {total_management_time/total_time*100:.1f}%")
print(f" Coordination Overhead: {total_coordination_time/total_time*100:.1f}%")
# Execution flow visualization
print("\n🔄 Execution Flow Analysis:")
try:
flow_viz = visualize_execution_flow()
print(" Hierarchical flow visualization generated")
print(f" Flow complexity: {len(str(flow_viz)) // 100} units")
except Exception as e:
print(f" Flow visualization: {str(e)}")
# Comprehensive report
print("\n📋 Comprehensive Performance Report:")
report = generate_comprehensive_report()
print(report[:500] + "..." if len(report) > 500 else report)
# Optimization recommendations
print(f"\n💡 Hierarchical Optimization Recommendations:")
if total_time > 0:
if total_management_time / total_time > 0.3:
print(" - High management overhead detected - consider task batching")
if total_coordination_time / total_time > 0.2:
print(" - Consider reducing coordination complexity")
if len(worker_results) > 0:
print(f" - Successfully processed {len(worker_results)} tasks hierarchically")
print(" - Monitor delegation efficiency vs direct execution trade-offs")
return {
'tasks_completed': len(worker_results),
'management_time': total_management_time,
'worker_time': total_worker_time,
'coordination_time': total_coordination_time,
'efficiency_ratio': total_worker_time / total_time if total_time > 0 else 0
}
if __name__ == "__main__":
result = main()
print(f"\n🎉 Hierarchical monitoring completed!")
print(f"Tasks: {result['tasks_completed']}, Efficiency: {result['efficiency_ratio']*100:.1f}%")
@@ -0,0 +1,314 @@
"""
Custom Tools with Performance Monitoring - Example 8
Demonstrates performance monitoring for agents using custom tools:
- Custom tool execution timing
- Tool creation and registration performance
- Tool usage pattern analysis
- Tool efficiency optimization
Shows how to monitor performance with custom-built tools.
"""
import time
import random
from praisonaiagents import Agent, tool
from praisonaiagents.telemetry import (
monitor_function, track_api_call, get_function_stats,
get_slowest_functions, performance_monitor
)
# Custom Tool 1: Data Processor
@tool
@monitor_function("data_processor_tool")
def process_data(data: str, operation: str = "analyze") -> str:
"""
Process data with specified operation.
Args:
data: The data to process
operation: Type of operation (analyze, transform, validate)
"""
print(f"🔧 Processing data with operation: {operation}")
# Simulate different processing times based on operation
processing_times = {
"analyze": 0.15,
"transform": 0.2,
"validate": 0.1
}
time.sleep(processing_times.get(operation, 0.12))
return f"Data processed with {operation}: {data[:50]}{'...' if len(data) > 50 else ''}"
# Custom Tool 2: API Simulator
@tool
@monitor_function("api_simulator_tool")
def call_external_api(endpoint: str, method: str = "GET", payload: str = "") -> str:
"""
Simulate calling an external API.
Args:
endpoint: API endpoint to call
method: HTTP method (GET, POST, PUT, DELETE)
payload: Request payload for POST/PUT
"""
print(f"🌐 Calling API: {method} {endpoint}")
# Simulate variable API response times
base_time = 0.1
if method == "POST":
base_time += 0.05
if payload:
base_time += len(payload) * 0.001
# Add random variation
response_time = base_time + random.uniform(0, 0.1)
time.sleep(response_time)
# Simulate occasional API failures
if random.random() < 0.1: # 10% failure rate
raise Exception(f"API call to {endpoint} failed")
return f"API Response from {endpoint}: {{'status': 'success', 'method': '{method}'}}"
# Custom Tool 3: Database Simulator
@tool
@monitor_function("database_simulator_tool")
def query_database(query: str, database: str = "main") -> str:
"""
Simulate database query execution.
Args:
query: SQL query to execute
database: Database name to query
"""
print(f"🗄️ Executing database query on {database}")
# Simulate query complexity based on keywords
complexity_keywords = {
"JOIN": 0.08,
"GROUP BY": 0.06,
"ORDER BY": 0.04,
"WHERE": 0.03
}
query_time = 0.05 # Base query time
for keyword, additional_time in complexity_keywords.items():
if keyword in query.upper():
query_time += additional_time
time.sleep(query_time)
return f"Query executed on {database}: Found {random.randint(1, 100)} results"
# Custom Tool 4: File Operations
@tool
@monitor_function("file_operations_tool")
def file_operation(operation: str, filename: str, content: str = "") -> str:
"""
Simulate file operations.
Args:
operation: Operation type (read, write, delete, copy)
filename: Target filename
content: Content for write operations
"""
print(f"📁 File operation: {operation} on {filename}")
# Simulate different operation times
operation_times = {
"read": 0.03,
"write": 0.05,
"delete": 0.02,
"copy": 0.07
}
time.sleep(operation_times.get(operation, 0.04))
if operation == "write" and content:
time.sleep(len(content) * 0.0001) # Simulate write time based on content size
return f"File {operation} completed for {filename}"
@monitor_function("tool_performance_test")
def test_tool_performance():
"""Test individual tool performance."""
print("🧪 Testing individual tool performance...")
# Test each tool multiple times
tools_results = []
# Test data processor
for op in ["analyze", "transform", "validate"]:
result = process_data("Sample data for testing", op)
tools_results.append(f"DataProcessor[{op}]: {result}")
# Test API simulator
for method in ["GET", "POST"]:
try:
result = call_external_api("/api/test", method, "test payload" if method == "POST" else "")
tools_results.append(f"APISimulator[{method}]: {result}")
except Exception as e:
tools_results.append(f"APISimulator[{method}]: Failed - {e}")
# Test database operations
queries = [
"SELECT * FROM users",
"SELECT COUNT(*) FROM orders WHERE date > '2024-01-01'",
"SELECT u.name, COUNT(o.id) FROM users u JOIN orders o GROUP BY u.id"
]
for query in queries:
result = query_database(query)
tools_results.append(f"Database: {result}")
# Test file operations
for op in ["read", "write", "copy"]:
result = file_operation(op, "test_file.txt", "Test content" if op == "write" else "")
tools_results.append(f"FileOps[{op}]: {result}")
return tools_results
@monitor_function("agent_with_custom_tools")
def run_agent_with_custom_tools():
"""Run agent using custom tools with performance monitoring."""
# Create agent with custom tools
agent = Agent(
instructions="""You are a technical assistant with access to various tools.
Use the appropriate tools to help with data processing, API calls, database queries,
and file operations. Always explain what tools you're using and why.""",
llm="gpt-4o-mini",
tools=[process_data, call_external_api, query_database, file_operation]
)
# Test scenarios for the agent
scenarios = [
"I need to analyze some customer data and store the results",
"Please fetch user information from the API and save it to a file",
"Query the database for recent orders and process the results"
]
results = []
for i, scenario in enumerate(scenarios, 1):
print(f"\n🎯 Scenario {i}: {scenario}")
with track_api_call(f"custom_tools_scenario_{i}"):
try:
result = agent.start(scenario)
results.append(result)
print(f"✅ Scenario {i} completed")
except Exception as e:
print(f"❌ Scenario {i} failed: {e}")
results.append(f"Failed: {e}")
return results
def main():
"""Main function demonstrating custom tools performance monitoring."""
print("=" * 70)
print("EXAMPLE 8: Custom Tools Performance Monitoring")
print("=" * 70)
# Phase 1: Individual tool performance testing
print("\n🔧 Phase 1: Individual Tool Performance Testing")
tool_test_results = test_tool_performance()
print(f"\n📊 Tool Test Summary:")
print(f" Total tool operations: {len(tool_test_results)}")
for result in tool_test_results[:5]: # Show first 5 results
print(f" - {result}")
if len(tool_test_results) > 5:
print(f" ... and {len(tool_test_results) - 5} more")
# Phase 2: Agent using custom tools
print("\n🤖 Phase 2: Agent with Custom Tools")
agent_results = run_agent_with_custom_tools()
# Performance Analysis
print("\n" + "=" * 70)
print("📊 CUSTOM TOOLS PERFORMANCE ANALYSIS")
print("=" * 70)
# Tool-specific performance
print("\n🔧 Tool Performance Breakdown:")
stats = get_function_stats()
tool_functions = {k: v for k, v in stats.items() if 'tool' in k.lower()}
for tool_name, data in tool_functions.items():
avg_time = data['total_time'] / data['call_count'] if data['call_count'] > 0 else 0
print(f" {tool_name}:")
print(f" Calls: {data['call_count']}")
print(f" Avg Time: {avg_time:.3f}s")
print(f" Total Time: {data['total_time']:.3f}s")
if data['error_count'] > 0:
print(f" Errors: {data['error_count']}")
# Tool efficiency ranking
print("\n🏆 Tool Efficiency Ranking (by average execution time):")
tool_efficiency = []
for tool_name, data in tool_functions.items():
if data['call_count'] > 0:
avg_time = data['total_time'] / data['call_count']
tool_efficiency.append((tool_name, avg_time))
tool_efficiency.sort(key=lambda x: x[1])
for rank, (tool_name, avg_time) in enumerate(tool_efficiency, 1):
print(f" {rank}. {tool_name}: {avg_time:.3f}s")
# Slowest operations
print("\n🐌 Slowest Tool Operations:")
slowest = get_slowest_functions()
tool_slowest = [(name, time) for name, time in slowest if 'tool' in name.lower()]
for tool_name, avg_time in tool_slowest[:3]:
print(f" {tool_name}: {avg_time:.3f}s")
# Usage patterns
total_tool_calls = sum(data['call_count'] for data in tool_functions.values())
total_tool_time = sum(data['total_time'] for data in tool_functions.values())
print(f"\n📈 Tool Usage Summary:")
print(f" Total Tool Calls: {total_tool_calls}")
print(f" Total Tool Execution Time: {total_tool_time:.3f}s")
print(f" Average Time per Tool Call: {total_tool_time/total_tool_calls:.3f}s" if total_tool_calls > 0 else " No tool calls")
# Tool reliability
total_tool_errors = sum(data['error_count'] for data in tool_functions.values())
reliability = ((total_tool_calls - total_tool_errors) / total_tool_calls * 100) if total_tool_calls > 0 else 100
print(f" Tool Reliability: {reliability:.1f}%")
if total_tool_errors > 0:
print(f" Total Errors: {total_tool_errors}")
# Optimization recommendations
print(f"\n💡 Custom Tools Optimization Recommendations:")
if tool_efficiency:
slowest_tool = tool_efficiency[-1]
if slowest_tool[1] > 0.2:
print(f" - Optimize {slowest_tool[0]} (slowest at {slowest_tool[1]:.3f}s)")
if total_tool_errors > 0:
print(f" - Investigate {total_tool_errors} tool errors for stability improvements")
if total_tool_calls > 20:
print(" - Consider tool caching for frequently used operations")
print(" - Monitor tool usage patterns for optimization opportunities")
return {
'tool_operations': len(tool_test_results),
'agent_scenarios': len(agent_results),
'total_tool_calls': total_tool_calls,
'tool_reliability': reliability
}
if __name__ == "__main__":
result = main()
print(f"\n✅ Custom tools monitoring completed!")
print(f"Operations: {result['tool_operations']}, Reliability: {result['tool_reliability']:.1f}%")
@@ -0,0 +1,286 @@
"""
Streaming & Real-Time Performance Monitoring - Example 9
Demonstrates performance monitoring for streaming and real-time operations:
- Real-time performance tracking
- Streaming response monitoring
- Live performance metrics
- Continuous performance analysis
Shows how to monitor performance during streaming operations.
"""
import time
import threading
from datetime import datetime
from praisonaiagents import Agent
from praisonaiagents.telemetry import (
monitor_function, track_api_call, performance_monitor,
get_performance_report, get_function_stats
)
# Global variables for streaming simulation
streaming_active = False
streaming_stats = {
'messages_processed': 0,
'total_latency': 0.0,
'errors': 0,
'start_time': None
}
@monitor_function("stream_message_processing")
def process_stream_message(message_id: int, content: str):
"""Process a single streaming message with monitoring."""
processing_start = time.time()
print(f"📨 Processing stream message {message_id}: {content[:30]}...")
# Simulate variable processing time
processing_time = 0.05 + (message_id % 3) * 0.02 # Simulate varying complexity
time.sleep(processing_time)
# Simulate occasional errors
if message_id % 20 == 0: # Every 20th message fails
raise Exception(f"Processing error for message {message_id}")
processing_end = time.time()
latency = processing_end - processing_start
# Update streaming stats
streaming_stats['messages_processed'] += 1
streaming_stats['total_latency'] += latency
return f"Processed message {message_id} in {latency:.3f}s"
@monitor_function("real_time_agent_response")
def get_real_time_agent_response(query: str, response_id: int):
"""Simulate real-time agent response with monitoring."""
print(f"🤖 Generating real-time response {response_id} for: {query}")
# Create a lightweight agent for streaming
agent = Agent(
instructions="You are a real-time assistant. Provide quick, helpful responses.",
llm="gpt-4o-mini"
)
with track_api_call(f"streaming_response_{response_id}"):
# Simulate streaming response generation
time.sleep(0.1) # Base response time
# In real implementation, this would be actual agent.start()
response = agent.start(f"Quickly respond to: {query}")
return response
@monitor_function("performance_metrics_collector")
def collect_live_metrics():
"""Collect live performance metrics during streaming."""
while streaming_active:
current_time = time.time()
# Collect current performance data
current_stats = performance_monitor.get_function_performance()
api_stats = performance_monitor.get_api_call_performance()
# Calculate streaming metrics
if streaming_stats['start_time']:
elapsed_time = current_time - streaming_stats['start_time']
messages_per_second = streaming_stats['messages_processed'] / elapsed_time if elapsed_time > 0 else 0
avg_latency = streaming_stats['total_latency'] / streaming_stats['messages_processed'] if streaming_stats['messages_processed'] > 0 else 0
print(f"📊 Live Metrics - Messages/sec: {messages_per_second:.2f}, Avg Latency: {avg_latency:.3f}s")
time.sleep(2) # Update every 2 seconds
@monitor_function("streaming_simulation")
def simulate_streaming_workload():
"""Simulate a streaming workload with continuous monitoring."""
global streaming_active, streaming_stats
print("🌊 Starting streaming simulation...")
streaming_active = True
streaming_stats['start_time'] = time.time()
# Start metrics collector in background
metrics_thread = threading.Thread(target=collect_live_metrics)
metrics_thread.daemon = True
metrics_thread.start()
# Simulate incoming stream messages
messages = [
"User question about AI",
"Request for weather information",
"Help with programming",
"General knowledge query",
"Technical support request",
"Creative writing assistance",
"Data analysis question",
"Research inquiry"
]
processed_responses = []
try:
for i in range(20): # Process 20 streaming messages
message = messages[i % len(messages)]
try:
# Process the stream message
result = process_stream_message(i + 1, message)
# Generate real-time response every 3rd message
if (i + 1) % 3 == 0:
response = get_real_time_agent_response(message, i + 1)
processed_responses.append(response)
# Brief pause between messages
time.sleep(0.1)
except Exception as e:
streaming_stats['errors'] += 1
print(f"❌ Error processing message {i + 1}: {e}")
finally:
streaming_active = False
print("🛑 Streaming simulation completed")
return processed_responses
@monitor_function("batch_vs_streaming_comparison")
def compare_batch_vs_streaming():
"""Compare batch vs streaming performance."""
print("\n🔄 Comparing Batch vs Streaming Performance...")
# Batch processing simulation
print("\n📦 Batch Processing Test:")
batch_start = time.time()
batch_messages = [f"Batch message {i}" for i in range(10)]
batch_results = []
with track_api_call("batch_processing"):
for i, message in enumerate(batch_messages):
# Simulate batch processing (all at once)
time.sleep(0.08) # Batch processing time
batch_results.append(f"Batch processed: {message}")
batch_time = time.time() - batch_start
# Streaming processing test (already done above)
streaming_time = streaming_stats['total_latency']
streaming_count = streaming_stats['messages_processed']
print(f"\n📊 Batch vs Streaming Comparison:")
print(f" Batch Processing:")
print(f" Messages: {len(batch_results)}")
print(f" Total Time: {batch_time:.3f}s")
print(f" Avg per Message: {batch_time/len(batch_results):.3f}s")
print(f" Streaming Processing:")
print(f" Messages: {streaming_count}")
print(f" Total Processing Time: {streaming_time:.3f}s")
print(f" Avg per Message: {streaming_time/streaming_count:.3f}s" if streaming_count > 0 else " No messages processed")
return batch_time, streaming_time
def main():
"""Main function demonstrating streaming performance monitoring."""
print("=" * 70)
print("EXAMPLE 9: Streaming & Real-Time Performance Monitoring")
print("=" * 70)
# Phase 1: Streaming simulation
print("\n🌊 Phase 1: Streaming Workload Simulation")
streaming_responses = simulate_streaming_workload()
# Phase 2: Batch vs Streaming comparison
print("\n📊 Phase 2: Batch vs Streaming Comparison")
batch_time, streaming_time = compare_batch_vs_streaming()
# Real-time Performance Analysis
print("\n" + "=" * 70)
print("📊 STREAMING PERFORMANCE ANALYSIS")
print("=" * 70)
# Streaming-specific metrics
print("\n🌊 Streaming Metrics:")
elapsed_total = time.time() - streaming_stats['start_time'] if streaming_stats['start_time'] else 0
throughput = streaming_stats['messages_processed'] / elapsed_total if elapsed_total > 0 else 0
avg_latency = streaming_stats['total_latency'] / streaming_stats['messages_processed'] if streaming_stats['messages_processed'] > 0 else 0
error_rate = streaming_stats['errors'] / streaming_stats['messages_processed'] * 100 if streaming_stats['messages_processed'] > 0 else 0
print(f" Messages Processed: {streaming_stats['messages_processed']}")
print(f" Total Processing Time: {streaming_stats['total_latency']:.3f}s")
print(f" Average Latency: {avg_latency:.3f}s")
print(f" Throughput: {throughput:.2f} messages/second")
print(f" Error Rate: {error_rate:.2f}%")
print(f" Errors: {streaming_stats['errors']}")
# Function performance during streaming
print("\n⚡ Real-Time Function Performance:")
stats = get_function_stats()
streaming_functions = {k: v for k, v in stats.items() if 'stream' in k.lower() or 'real_time' in k.lower()}
for func_name, data in streaming_functions.items():
avg_time = data['total_time'] / data['call_count'] if data['call_count'] > 0 else 0
print(f" {func_name}:")
print(f" Calls: {data['call_count']}")
print(f" Avg Time: {avg_time:.3f}s")
print(f" Total Time: {data['total_time']:.3f}s")
# Performance trend analysis
print("\n📈 Performance Trends:")
recent_performance = performance_monitor.get_function_performance()
total_functions = len(recent_performance)
total_calls = sum(data['call_count'] for data in recent_performance.values())
print(f" Functions Monitored: {total_functions}")
print(f" Total Function Calls: {total_calls}")
print(f" Average Calls per Function: {total_calls/total_functions:.1f}" if total_functions > 0 else " No functions monitored")
# Real-time vs Batch efficiency
print(f"\n⚖️ Efficiency Analysis:")
if batch_time > 0 and streaming_time > 0:
efficiency_gain = (batch_time - streaming_time) / batch_time * 100
print(f" Streaming vs Batch Efficiency: {efficiency_gain:+.1f}%")
if avg_latency > 0:
if avg_latency < 0.1:
print(" ✅ Excellent real-time performance (< 100ms)")
elif avg_latency < 0.2:
print(" ⚡ Good real-time performance (< 200ms)")
else:
print(" ⚠️ Consider optimization for better real-time performance")
# Current performance report
print("\n📋 Current Performance Report:")
report = get_performance_report()
print(report[:400] + "..." if len(report) > 400 else report)
# Real-time optimization suggestions
print(f"\n💡 Real-Time Optimization Recommendations:")
if avg_latency > 0.15:
print(f" - Optimize message processing (current avg: {avg_latency:.3f}s)")
if error_rate > 5:
print(f" - Investigate error patterns (current rate: {error_rate:.1f}%)")
if throughput < 10:
print(f" - Consider parallel processing for better throughput")
print(" - Monitor memory usage during extended streaming")
print(" - Implement adaptive batching for optimal performance")
return {
'streaming_responses': len(streaming_responses),
'messages_processed': streaming_stats['messages_processed'],
'throughput': throughput,
'average_latency': avg_latency,
'error_rate': error_rate
}
if __name__ == "__main__":
result = main()
print(f"\n🎉 Streaming monitoring completed!")
print(f"Throughput: {result['throughput']:.2f} msg/s, Latency: {result['average_latency']:.3f}s")
@@ -0,0 +1,433 @@
"""
Comprehensive Performance Dashboard - Example 10
Demonstrates comprehensive performance monitoring dashboard:
- Complete performance analytics
- Advanced reporting and visualization
- Performance trends and insights
- System-wide monitoring overview
Shows how to create a comprehensive performance monitoring solution.
"""
import time
import json
from datetime import datetime, timedelta
from praisonaiagents import Agent, Task, PraisonAIAgents
from praisonaiagents.telemetry import (
monitor_function, track_api_call, performance_monitor,
get_performance_report, get_function_stats, get_api_stats,
get_slowest_functions, get_slowest_apis,
analyze_function_flow, visualize_execution_flow,
analyze_performance_trends, generate_comprehensive_report,
clear_performance_data
)
@monitor_function("dashboard_data_collection")
def collect_comprehensive_data():
"""Collect comprehensive performance data for dashboard."""
print("📊 Collecting comprehensive performance data...")
# Generate diverse workload for demonstration
test_scenarios = [
("Quick query", "What is AI?"),
("Complex analysis", "Analyze the impact of artificial intelligence on modern business practices"),
("Research task", "Research the latest developments in quantum computing"),
("Creative task", "Write a creative story about space exploration"),
("Technical query", "Explain how neural networks process information")
]
# Create different types of agents for variety
agents = {
'quick_agent': Agent(
instructions="You are a quick-response agent. Provide concise answers.",
llm="gpt-4o-mini"
),
'analytical_agent': Agent(
instructions="You are an analytical agent. Provide detailed analysis.",
llm="gpt-4o-mini"
),
'research_agent': Agent(
instructions="You are a research agent. Provide comprehensive information.",
llm="gpt-4o-mini"
),
'creative_agent': Agent(
instructions="You are a creative agent. Provide imaginative responses.",
llm="gpt-4o-mini"
)
}
results = []
for i, (scenario_type, query) in enumerate(test_scenarios):
print(f"\n🎯 Scenario {i+1}: {scenario_type}")
# Select appropriate agent
agent_type = list(agents.keys())[i % len(agents)]
agent = agents[agent_type]
with track_api_call(f"{scenario_type.lower().replace(' ', '_')}_execution"):
try:
result = agent.start(query)
results.append({
'scenario': scenario_type,
'agent': agent_type,
'query': query,
'result': result[:200] + '...' if len(result) > 200 else result,
'success': True
})
except Exception as e:
results.append({
'scenario': scenario_type,
'agent': agent_type,
'query': query,
'error': str(e),
'success': False
})
return results
@monitor_function("multi_agent_workflow_simulation")
def simulate_complex_workflow():
"""Simulate complex multi-agent workflow for comprehensive data."""
print("🔄 Simulating complex multi-agent workflow...")
# Create specialized agents
planner = Agent(
name="planner",
instructions="You are a project planner. Break down tasks and plan execution.",
llm="gpt-4o-mini"
)
executor = Agent(
name="executor",
instructions="You are a task executor. Complete assigned tasks efficiently.",
llm="gpt-4o-mini"
)
reviewer = Agent(
name="reviewer",
instructions="You are a quality reviewer. Review and improve work.",
llm="gpt-4o-mini"
)
# Create workflow tasks
planning_task = Task(
name="planning",
description="Plan a comprehensive report on renewable energy trends",
expected_output="A detailed project plan with timeline and milestones"
)
execution_task = Task(
name="execution",
description="Execute the planned research and compile findings",
expected_output="Research findings and data compilation"
)
review_task = Task(
name="review",
description="Review the research and provide quality assessment",
expected_output="Quality review and improvement recommendations"
)
# Assign agents to tasks
planning_task.agent = planner
execution_task.agent = executor
review_task.agent = reviewer
# Execute workflow
with track_api_call("complex_workflow_execution"):
workflow = PraisonAIAgents(
agents=[planner, executor, reviewer],
tasks=[planning_task, execution_task, review_task],
process="sequential",
verbose=False
)
result = workflow.start()
return result
@monitor_function("performance_benchmarking")
def run_performance_benchmarks():
"""Run performance benchmarks for various operations."""
print("⚡ Running performance benchmarks...")
benchmark_results = {}
# Benchmark 1: Response time under different loads
print(" 📈 Benchmarking response times...")
response_times = []
for i in range(10):
start_time = time.time()
with track_api_call(f"benchmark_response_{i}"):
# Simulate quick agent response
time.sleep(0.05 + (i % 3) * 0.02) # Variable response time
response_times.append(time.time() - start_time)
benchmark_results['response_times'] = {
'min': min(response_times),
'max': max(response_times),
'avg': sum(response_times) / len(response_times),
'samples': len(response_times)
}
# Benchmark 2: Concurrent operations
print(" 🔄 Benchmarking concurrent operations...")
concurrent_start = time.time()
# Simulate concurrent operations
for i in range(5):
with track_api_call(f"concurrent_op_{i}"):
time.sleep(0.1) # Simulate concurrent work
concurrent_time = time.time() - concurrent_start
benchmark_results['concurrent_operations'] = {
'total_time': concurrent_time,
'operations': 5,
'avg_per_operation': concurrent_time / 5
}
# Benchmark 3: Memory usage simulation
print(" 💾 Benchmarking memory operations...")
memory_ops = []
for i in range(8):
start = time.time()
with track_api_call(f"memory_benchmark_{i}"):
# Simulate memory-intensive operation
time.sleep(0.03 + (i % 2) * 0.02)
memory_ops.append(time.time() - start)
benchmark_results['memory_operations'] = {
'operations': len(memory_ops),
'total_time': sum(memory_ops),
'avg_time': sum(memory_ops) / len(memory_ops)
}
return benchmark_results
@monitor_function("generate_dashboard_report")
def generate_dashboard_report():
"""Generate comprehensive dashboard report."""
print("📋 Generating comprehensive dashboard report...")
# Collect all performance data
function_stats = get_function_stats()
api_stats = get_api_stats()
slowest_functions = get_slowest_functions()
slowest_apis = get_slowest_apis()
# Generate reports
basic_report = get_performance_report()
comprehensive_report = generate_comprehensive_report()
# Analyze trends and flows
try:
flow_analysis = analyze_function_flow()
performance_trends = analyze_performance_trends()
flow_visualization = visualize_execution_flow()
except Exception as e:
flow_analysis = {"error": str(e)}
performance_trends = {"error": str(e)}
flow_visualization = {"error": str(e)}
dashboard_data = {
'timestamp': datetime.now().isoformat(),
'summary': {
'total_functions': len(function_stats),
'total_api_calls': len(api_stats),
'total_function_calls': sum(stats['call_count'] for stats in function_stats.values()),
'total_api_requests': sum(stats['call_count'] for stats in api_stats.values()),
'total_execution_time': sum(stats['total_time'] for stats in function_stats.values()),
'total_errors': sum(stats['error_count'] for stats in function_stats.values())
},
'performance_data': {
'function_stats': function_stats,
'api_stats': api_stats,
'slowest_functions': slowest_functions,
'slowest_apis': slowest_apis
},
'analysis': {
'flow_analysis': flow_analysis,
'performance_trends': performance_trends,
'flow_visualization': len(str(flow_visualization)) if flow_visualization else 0
},
'reports': {
'basic_report': basic_report,
'comprehensive_report': comprehensive_report[:500] + '...' if len(comprehensive_report) > 500 else comprehensive_report
}
}
return dashboard_data
def display_dashboard(dashboard_data, benchmark_results):
"""Display comprehensive performance dashboard."""
print("\n" + "=" * 80)
print("📊 COMPREHENSIVE PERFORMANCE DASHBOARD")
print("=" * 80)
# Executive Summary
summary = dashboard_data['summary']
print(f"\n📈 EXECUTIVE SUMMARY")
print(f" Report Generated: {dashboard_data['timestamp']}")
print(f" Functions Monitored: {summary['total_functions']}")
print(f" API Endpoints: {summary['total_api_calls']}")
print(f" Total Function Calls: {summary['total_function_calls']}")
print(f" Total API Requests: {summary['total_api_requests']}")
print(f" Total Execution Time: {summary['total_execution_time']:.3f}s")
print(f" Total Errors: {summary['total_errors']}")
# Performance Overview
print(f"\n⚡ PERFORMANCE OVERVIEW")
avg_function_time = 0
if summary['total_function_calls'] > 0:
avg_function_time = summary['total_execution_time'] / summary['total_function_calls']
print(f" Average Function Execution: {avg_function_time:.3f}s")
if summary['total_errors'] > 0 and summary['total_function_calls'] > 0:
error_rate = summary['total_errors'] / summary['total_function_calls'] * 100
print(f" Error Rate: {error_rate:.2f}%")
else:
print(f" Error Rate: 0.00%")
# Top Performers and Issues
print(f"\n🏆 TOP PERFORMERS")
perf_data = dashboard_data['performance_data']
print(" Fastest Functions:")
fastest_functions = sorted(perf_data['slowest_functions'], key=lambda x: x[1])[:3]
for name, time in fastest_functions:
print(f" {name}: {time:.3f}s")
print(" Slowest Functions:")
for name, time in perf_data['slowest_functions'][:3]:
print(f" {name}: {time:.3f}s")
# Benchmark Results
print(f"\n⚡ BENCHMARK RESULTS")
print(f" Response Time Benchmarks:")
rt = benchmark_results['response_times']
print(f" Min: {rt['min']:.3f}s, Max: {rt['max']:.3f}s, Avg: {rt['avg']:.3f}s")
print(f" Concurrent Operations:")
co = benchmark_results['concurrent_operations']
print(f" {co['operations']} operations in {co['total_time']:.3f}s")
print(f" Avg per operation: {co['avg_per_operation']:.3f}s")
print(f" Memory Operations:")
mo = benchmark_results['memory_operations']
print(f" {mo['operations']} operations, {mo['avg_time']:.3f}s average")
# System Health
print(f"\n🏥 SYSTEM HEALTH")
# Calculate health score
health_score = 100
if avg_function_time > 0.2:
health_score -= 20
if summary['total_errors'] > 0:
health_score -= summary['total_errors'] * 10
if rt['avg'] > 0.1:
health_score -= 15
health_score = max(0, health_score)
if health_score >= 90:
status = "🟢 Excellent"
elif health_score >= 70:
status = "🟡 Good"
elif health_score >= 50:
status = "🟠 Fair"
else:
status = "🔴 Needs Attention"
print(f" Overall Health Score: {health_score}/100 - {status}")
# Analysis Results
print(f"\n🔍 ANALYSIS RESULTS")
analysis = dashboard_data['analysis']
if 'error' not in analysis['flow_analysis']:
flow_data = analysis['flow_analysis']
print(f" Flow Analysis: {len(flow_data.get('functions', []))} functions analyzed")
else:
print(f" Flow Analysis: {analysis['flow_analysis']['error']}")
if analysis['flow_visualization']:
print(f" Flow Visualization: Generated ({analysis['flow_visualization']} chars)")
# Recommendations
print(f"\n💡 PERFORMANCE RECOMMENDATIONS")
if avg_function_time > 0.15:
print(" 🔧 Optimize slow functions for better response times")
if summary['total_errors'] > 0:
print(" 🚨 Investigate and resolve error sources")
if rt['avg'] > 0.08:
print(" ⚡ Consider response time optimization strategies")
if summary['total_function_calls'] > 50:
print(" 📊 Implement performance caching for high-volume functions")
print(" 📈 Continue monitoring trends for proactive optimization")
print(" 🔄 Regular performance reviews recommended")
def main():
"""Main function for comprehensive performance dashboard."""
print("=" * 80)
print("EXAMPLE 10: Comprehensive Performance Dashboard")
print("=" * 80)
print("This example demonstrates a complete performance monitoring solution")
print("with advanced analytics, benchmarking, and dashboard reporting.\n")
# Clear previous data for clean demo
clear_performance_data()
# Phase 1: Data Collection
print("📊 Phase 1: Comprehensive Data Collection")
collection_results = collect_comprehensive_data()
# Phase 2: Complex Workflow
print("\n🔄 Phase 2: Complex Multi-Agent Workflow")
workflow_result = simulate_complex_workflow()
# Phase 3: Performance Benchmarking
print("\n⚡ Phase 3: Performance Benchmarking")
benchmark_results = run_performance_benchmarks()
# Phase 4: Dashboard Generation
print("\n📋 Phase 4: Dashboard Report Generation")
dashboard_data = generate_dashboard_report()
# Phase 5: Display Comprehensive Dashboard
display_dashboard(dashboard_data, benchmark_results)
# Final Summary
print(f"\n🎯 EXECUTION SUMMARY")
print(f" Scenarios Executed: {len(collection_results)}")
successful_scenarios = sum(1 for r in collection_results if r['success'])
print(f" Successful Scenarios: {successful_scenarios}/{len(collection_results)}")
print(f" Workflow Completed: {'✅ Yes' if workflow_result else '❌ No'}")
print(f" Benchmarks Run: {len(benchmark_results)}")
print(f" Dashboard Generated: ✅ Complete")
return {
'scenarios': len(collection_results),
'success_rate': successful_scenarios / len(collection_results) * 100,
'dashboard_data': dashboard_data,
'benchmark_results': benchmark_results
}
if __name__ == "__main__":
result = main()
print(f"\n🎉 Comprehensive dashboard monitoring completed!")
print(f"Success Rate: {result['success_rate']:.1f}%")
print(f"Total Functions Monitored: {result['dashboard_data']['summary']['total_functions']}")
print(f"Performance Health Score Available in Dashboard Above ⬆️")
print("\n💡 This dashboard provides real-time insights into your PraisonAI system performance!")
+238
View File
@@ -0,0 +1,238 @@
# PraisonAI Performance Monitoring Examples
This directory contains 10 comprehensive examples demonstrating how to integrate performance monitoring with PraisonAI agents. Each example builds upon the basic pattern while showcasing different aspects of the monitoring system.
## 🚀 Quick Start
All examples follow this basic pattern:
```python
from praisonaiagents import Agent
from praisonaiagents.telemetry import monitor_function, track_api_call
@monitor_function("my_function")
def my_function():
# Your function code here
pass
agent = Agent(
instructions="You are a helpful assistant",
llm="gpt-4o-mini"
)
with track_api_call("agent_request"):
result = agent.start("Your question here")
```
## 📊 Examples Overview
### 1. Basic Agent with Monitoring (`01_basic_agent_with_monitoring.py`)
**Purpose:** Introduction to performance monitoring fundamentals
- Function execution timing with `@monitor_function` decorator
- API call tracking with `track_api_call` context manager
- Basic performance statistics retrieval
### 2. Multi-Agent Workflow Monitoring (`02_multi_agent_workflow_monitoring.py`)
**Purpose:** Monitor complex multi-agent systems
- Multiple agents with different roles
- Inter-agent communication tracking
- Workflow performance analysis
- Task delegation timing
### 3. Agent with Tools Monitoring (`03_agent_with_tools_monitoring.py`)
**Purpose:** Monitor performance when using external tools
- Tool execution timing
- Search operation performance
- Tool call success/failure tracking
- Tool usage analysis
### 4. Async Agent Monitoring (`04_async_agent_monitoring.py`)
**Purpose:** Performance monitoring in asynchronous workflows
- Async function execution timing
- Concurrent task performance
- Async API call tracking
- Parallel operation analysis
### 5. Error Handling Monitoring (`05_error_handling_monitoring.py`)
**Purpose:** Monitor performance during error scenarios
- Error rate tracking
- Failed operation timing
- Recovery mechanism performance
- Exception handling with monitoring
### 6. Memory Agent Monitoring (`06_memory_agent_monitoring.py`)
**Purpose:** Monitor stateful agents with memory
- Memory storage and retrieval timing
- Knowledge base operations
- Session persistence performance
- Memory search optimization
### 7. Hierarchical Agents Monitoring (`07_hierarchical_agents_monitoring.py`)
**Purpose:** Monitor complex organizational agent structures
- Manager-worker agent relationships
- Task delegation timing
- Hierarchical decision making performance
- Cross-level communication monitoring
### 8. Custom Tools Monitoring (`08_custom_tools_monitoring.py`)
**Purpose:** Monitor performance with custom-built tools
- Custom tool execution timing
- Tool creation and registration performance
- Tool usage pattern analysis
- Tool efficiency optimization
### 9. Streaming Monitoring (`09_streaming_monitoring.py`)
**Purpose:** Monitor real-time and streaming operations
- Real-time performance tracking
- Streaming response monitoring
- Live performance metrics
- Continuous performance analysis
### 10. Comprehensive Dashboard (`10_comprehensive_dashboard.py`)
**Purpose:** Complete performance monitoring solution
- Advanced analytics and reporting
- Performance trends and insights
- System-wide monitoring overview
- Executive dashboard with health scores
## 🎯 Key Monitoring Features Demonstrated
### Core Monitoring Tools
- **`@monitor_function(name)`** - Decorates functions for timing and statistics
- **`track_api_call(name)`** - Context manager for API call monitoring
- **`get_performance_report()`** - Generate comprehensive performance reports
- **`get_function_stats()`** - Retrieve detailed function performance data
- **`get_api_stats()`** - Get API call performance metrics
### Advanced Analytics
- **`analyze_function_flow()`** - Analyze execution flow and bottlenecks
- **`visualize_execution_flow()`** - Generate flow visualization
- **`analyze_performance_trends()`** - Identify performance trends
- **`generate_comprehensive_report()`** - Complete analysis with recommendations
### Performance Metrics Tracked
- **Execution Time:** Min, max, average, and total execution times
- **Call Counts:** Number of function calls and API requests
- **Error Rates:** Success/failure ratios and error tracking
- **Throughput:** Operations per second and message processing rates
- **Flow Analysis:** Function call chains and execution patterns
## 📈 Usage Patterns
### Basic Monitoring Pattern
```python
@monitor_function("function_name")
def your_function():
# Function implementation
pass
with track_api_call("api_operation"):
result = some_api_call()
```
### Performance Analysis Pattern
```python
# After running monitored operations
stats = get_function_stats()
report = get_performance_report()
trends = analyze_performance_trends()
```
### Real-time Monitoring Pattern
```python
# For streaming or real-time operations
while streaming_active:
with track_api_call("stream_message"):
process_message()
# Check performance in real-time
current_stats = performance_monitor.get_function_performance()
```
## 🔍 Performance Insights
Each example provides insights into different aspects:
- **Response Times:** How fast your agents respond
- **Bottlenecks:** Where delays occur in your workflow
- **Error Patterns:** What fails and how often
- **Resource Usage:** Efficiency of different operations
- **Scaling Behavior:** How performance changes with load
## 🚀 Running the Examples
1. **Install PraisonAI with performance monitoring support:**
```bash
pip install praisonaiagents
```
2. **Set up your environment:**
```bash
export OPENAI_API_KEY="your-api-key-here"
# or
export ANTHROPIC_API_KEY="your-anthropic-key-here"
```
3. **Run any example:**
```bash
python 01_basic_agent_with_monitoring.py
python 10_comprehensive_dashboard.py
```
## 📊 Sample Output
Each example provides detailed performance output like:
```
📊 PERFORMANCE MONITORING RESULTS
==================================================
📈 Function Performance Statistics:
question_processing:
Calls: 1
Avg Time: 0.100s
Total Time: 0.100s
agent_execution:
Calls: 1
Avg Time: 1.250s
Total Time: 1.250s
🌐 API Call Performance:
sky_explanation_request:
Success Rate: 100.0%
Average Response Time: 1.200s
Total Calls: 1
```
## 💡 Best Practices
1. **Strategic Monitoring:** Don't monitor everything - focus on critical paths
2. **Performance Baselines:** Establish baseline metrics for comparison
3. **Error Handling:** Always monitor error rates alongside performance
4. **Real-time Analysis:** Use streaming monitoring for production systems
5. **Regular Reviews:** Analyze trends and patterns regularly
## 🔧 Customization
You can customize monitoring by:
- **Custom Metrics:** Add your own performance counters
- **Alert Thresholds:** Set up alerts for performance degradation
- **Export Data:** Export performance data to external monitoring systems
- **Dashboard Integration:** Build custom dashboards using the monitoring APIs
## 📚 Further Reading
- [PraisonAI Documentation](https://docs.praisonai.com)
- [Performance Monitoring Guide](../../../src/praisonai-agents/praisonaiagents/telemetry/README.md)
- [API Reference](../../../src/praisonai-agents/praisonaiagents/telemetry/__init__.py)
## 🤝 Contributing
These examples are designed to be educational and extensible. Feel free to:
- Add new monitoring scenarios
- Improve existing examples
- Share your performance optimization discoveries
---
**Note:** Performance monitoring is optional and can be disabled by setting `PRAISONAI_TELEMETRY_DISABLED=true` if needed.
@@ -0,0 +1,321 @@
"""
PraisonAI Performance Monitoring Demo
This example demonstrates how to use PraisonAI's comprehensive performance monitoring
capabilities to answer the key questions from Issue #1035:
1. How to evaluate how each function is performing
2. Function flow analysis
3. How fast each function is executing
4. API call performance monitoring (fast vs slow calls)
5. How to see all API calls
Features Demonstrated:
- Function performance evaluation with detailed statistics
- Function execution flow analysis and visualization
- API call tracking and performance comparison
- Real-time performance monitoring
- Bottleneck identification and optimization recommendations
- Comprehensive performance reporting
"""
import time
import random
from praisonaiagents import Agent, Task, PraisonAIAgents
from praisonaiagents.tools import duckduckgo
from praisonaiagents.telemetry import (
monitor_function, track_api_call, performance_monitor,
analyze_function_flow,
visualize_execution_flow, generate_comprehensive_report
)
print("=" * 80)
print("PRAISONAI PERFORMANCE MONITORING DEMONSTRATION")
print("=" * 80)
print("This demo answers the key questions from Issue #1035:")
print("1. How to evaluate how each function is performing")
print("2. Function flow analysis")
print("3. How fast each function is executing")
print("4. API call performance monitoring (fast vs slow calls)")
print("5. How to see all API calls")
print("=" * 80)
print()
# Example 1: Function Performance Evaluation
print("📊 EXAMPLE 1: Function Performance Evaluation")
print("-" * 50)
# Demonstrate function monitoring with decorators
@monitor_function("data_processing")
def process_data(data_size: int):
"""Simulate data processing with variable performance."""
processing_time = 0.1 + (data_size * 0.01) # Simulate work
time.sleep(processing_time)
return f"Processed {data_size} items"
@monitor_function("model_inference")
def run_model_inference(complexity: str):
"""Simulate model inference with different complexities."""
if complexity == "simple":
time.sleep(0.05)
elif complexity == "medium":
time.sleep(0.2)
else: # complex
time.sleep(0.5)
return f"Inference completed for {complexity} model"
@monitor_function("database_query")
def query_database(query_type: str):
"""Simulate database queries with different performance characteristics."""
if query_type == "fast":
time.sleep(0.02)
elif query_type == "medium":
time.sleep(0.1)
else: # slow
time.sleep(0.3)
return f"Query {query_type} completed"
# Run functions with different performance characteristics
print("Running functions to generate performance data...")
for _ in range(5):
process_data(random.randint(10, 100))
run_model_inference(random.choice(["simple", "medium", "complex"]))
query_database(random.choice(["fast", "medium", "slow"]))
# Show function performance statistics
print("\n📈 Function Performance Statistics:")
function_stats = performance_monitor.get_function_performance()
for func_name, stats in function_stats.items():
print(f"\n🔧 {func_name}:")
print(f" Calls: {stats['call_count']}")
print(f" Average Time: {stats.get('average_time', 0):.3f}s")
print(f" Min/Max Time: {stats['min_time']:.3f}s / {stats['max_time']:.3f}s")
print(f" Success Rate: {stats.get('success_rate', 0)*100:.1f}%")
# Example 2: Function Flow Analysis
print("\n\n🔄 EXAMPLE 2: Function Flow Analysis")
print("-" * 50)
@monitor_function("workflow_step_1")
def workflow_step_1():
time.sleep(0.1)
workflow_step_2()
workflow_step_3()
@monitor_function("workflow_step_2")
def workflow_step_2():
time.sleep(0.05)
workflow_step_4()
@monitor_function("workflow_step_3")
def workflow_step_3():
time.sleep(0.08)
@monitor_function("workflow_step_4")
def workflow_step_4():
time.sleep(0.03)
print("Running workflow to demonstrate flow analysis...")
workflow_step_1()
# Analyze execution flow
print("\n📊 Function Execution Flow Analysis:")
flow_analysis = analyze_function_flow()
if "bottlenecks" in flow_analysis:
bottlenecks = flow_analysis["bottlenecks"]
if bottlenecks:
print("🚨 Identified Bottlenecks:")
for bottleneck in bottlenecks:
print(f" • {bottleneck['function']}: {bottleneck['average_duration']:.3f}s average")
else:
print("✅ No significant bottlenecks identified")
# Visualize execution flow
print("\n🎨 Function Execution Flow Visualization:")
flow_viz = visualize_execution_flow(format="text")
print(flow_viz)
# Example 3: API Call Performance Monitoring
print("\n\n🌐 EXAMPLE 3: API Call Performance Monitoring")
print("-" * 50)
# Simulate different API calls with various performance characteristics
def simulate_api_calls():
"""Simulate API calls to demonstrate tracking."""
# Fast API calls
with track_api_call("openai_api", "/v1/models"):
time.sleep(0.05) # Fast call
with track_api_call("openai_api", "/v1/chat/completions"):
time.sleep(0.3) # Medium call
# Slow API call
with track_api_call("anthropic_api", "/v1/messages"):
time.sleep(0.6) # Slow call
# Another fast call
with track_api_call("gemini_api", "/v1/generate"):
time.sleep(0.08) # Fast call
# Simulate error case
try:
with track_api_call("unreliable_api", "/v1/endpoint"):
time.sleep(0.2)
if random.random() < 0.3: # 30% chance of error
raise Exception("API Error")
except Exception:
pass # Error is already tracked
print("Simulating API calls with different performance characteristics...")
for _ in range(8):
simulate_api_calls()
# Show API performance statistics
print("\n📊 API Call Performance Statistics:")
api_stats = performance_monitor.get_api_call_performance()
for api_name, stats in api_stats.items():
print(f"\n🔗 {api_name}:")
print(f" Total Calls: {stats['call_count']}")
print(f" Average Time: {stats.get('average_time', 0):.3f}s")
print(f" Min/Max Time: {stats['min_time']:.3f}s / {stats['max_time']:.3f}s")
print(f" Success Rate: {stats.get('success_rate', 0)*100:.1f}%")
print(f" Successful: {stats['success_count']} | Failed: {stats['error_count']}")
# Show fastest vs slowest APIs
print("\n🏁 Fastest vs Slowest API Calls:")
slowest_apis = performance_monitor.get_slowest_api_calls(10)
for i, api in enumerate(slowest_apis, 1):
speed_indicator = "🐌" if api['average_time'] > 0.3 else "⚡" if api['average_time'] < 0.1 else "🚀"
print(f"{i}. {speed_indicator} {api['api']}: {api['average_time']:.3f}s avg (Success: {api['success_rate']*100:.1f}%)")
# Example 4: Real-time Performance Monitoring with Agents
print("\n\n🤖 EXAMPLE 4: Real-time Performance Monitoring with AI Agents")
print("-" * 50)
# Create agents with performance monitoring
research_agent = Agent(
name="PerformanceResearcher",
role="Performance Research Specialist",
goal="Research and analyze performance topics quickly",
backstory="You are an expert at researching performance optimization topics.",
tools=[duckduckgo],
verbose=False # Reduce output for cleaner demo
)
# Monitor agent operations
@monitor_function("agent_task_execution")
def execute_monitored_task(agent, task_description):
"""Execute an agent task with performance monitoring."""
task = Task(
description=task_description,
expected_output="Brief research summary with key insights",
agent=agent
)
agents_workflow = PraisonAIAgents(
agents=[agent],
tasks=[task],
verbose=False
)
# Simulate API calls during agent execution
with track_api_call("llm_provider", "/v1/chat/completions"):
time.sleep(0.2) # Simulate LLM call
result = agents_workflow.start()
return result
print("Running AI agent with performance monitoring...")
result = execute_monitored_task(
research_agent,
"Research the latest trends in AI performance optimization"
)
# Example 5: Comprehensive Performance Report
print("\n\n📋 EXAMPLE 5: Comprehensive Performance Report")
print("-" * 50)
# Generate comprehensive performance report
comprehensive_report = generate_comprehensive_report()
print(comprehensive_report)
# Example 6: Using CLI Interface Programmatically
print("\n\n💻 EXAMPLE 6: CLI Interface Usage")
print("-" * 50)
print("You can also use the CLI interface from command line:")
print()
print("# Show performance report")
print("python -m praisonaiagents.telemetry.performance_cli report")
print()
print("# Show function statistics")
print("python -m praisonaiagents.telemetry.performance_cli functions")
print()
print("# Show API call statistics")
print("python -m praisonaiagents.telemetry.performance_cli apis")
print()
print("# Show slowest functions")
print("python -m praisonaiagents.telemetry.performance_cli slowest-functions 5")
print()
print("# Analyze function flow")
print("python -m praisonaiagents.telemetry.performance_cli analyze-flow")
print()
print("# Show execution flow visualization")
print("python -m praisonaiagents.telemetry.performance_cli flow --format text")
print()
print("# Export all data to JSON")
print("python -m praisonaiagents.telemetry.performance_cli export --format json --output performance_data.json")
# Example 7: Integration with Existing Code
print("\n\n🔧 EXAMPLE 7: Easy Integration with Existing Code")
print("-" * 50)
print("To integrate performance monitoring into your existing PraisonAI code:")
print()
print("1. Import the performance monitoring tools:")
print(" from praisonaiagents.telemetry import monitor_function, track_api_call")
print()
print("2. Add decorators to your functions:")
print(" @monitor_function('my_function')")
print(" def my_function():")
print(" return 'result'")
print()
print("3. Track API calls:")
print(" with track_api_call('openai', '/v1/chat/completions'):")
print(" response = openai_client.chat.completions.create(...)")
print()
print("4. Get performance reports:")
print(" from praisonaiagents.telemetry import get_performance_report")
print(" print(get_performance_report())")
# Summary of capabilities
print("\n\n✅ SUMMARY: Questions from Issue #1035 - ANSWERED!")
print("=" * 80)
print("1. ✅ How to evaluate how each function is performing:")
print(" → Use @monitor_function decorator and get_function_stats()")
print(" → View detailed statistics: calls, timing, success rates, errors")
print()
print("2. ✅ Function flow analysis:")
print(" → Use analyze_function_flow() for bottleneck identification")
print(" → Use visualize_execution_flow() for flow visualization")
print(" → Get call chains and execution patterns")
print()
print("3. ✅ How fast each function is executing:")
print(" → Automatic timing collection with min/max/average/recent stats")
print(" → Use get_slowest_functions() to find performance bottlenecks")
print(" → Real-time active call monitoring")
print()
print("4. ✅ API call performance monitoring (fast vs slow calls):")
print(" → Use track_api_call() context manager for API monitoring")
print(" → Compare API performance with get_slowest_apis()")
print(" → Success/error rate tracking")
print()
print("5. ✅ How to see all API calls:")
print(" → Use get_api_stats() to see all tracked APIs")
print(" → View recent call history and detailed metrics")
print(" → CLI interface for easy access: performance_cli apis")
print()
print("🎉 All requested features are now available with backward compatibility!")
print("🔧 Easy integration with existing code using decorators and context managers")
print("📊 Comprehensive reporting and analysis capabilities")
print("💻 CLI interface for command-line access")
print("=" * 80)
@@ -0,0 +1,63 @@
"""
Basic Agent with Performance Monitoring - Example 1
Simple agent demonstrating performance monitoring basics:
- Function execution timing with @monitor_function decorator
- API call tracking with track_api_call context manager
- Basic performance statistics retrieval
This example shows the fundamental pattern for adding performance monitoring
to any PraisonAI agent workflow.
"""
from praisonaiagents import Agent
from praisonaiagents.telemetry import monitor_function, track_api_call, get_performance_report
import time
# Monitored function to demonstrate timing
@monitor_function("question_processing")
def process_question(question):
"""Process the incoming question with performance tracking."""
print(f"Processing question: {question}")
# Simulate some processing time
time.sleep(0.1)
return f"Processed: {question}"
# Main execution function with monitoring
@monitor_function("agent_execution")
def main():
"""Main function demonstrating basic agent with performance monitoring."""
print("=" * 60)
print("EXAMPLE 1: Basic Agent with Performance Monitoring")
print("=" * 60)
# Process the question with monitoring
question = "Why is the sky blue?"
processed_question = process_question(question)
# Create agent
agent = Agent(
instructions="You are a helpful assistant that explains scientific concepts simply",
llm="gpt-4o-mini"
)
# Track the API call performance
with track_api_call("sky_explanation_request"):
print("\n🚀 Starting agent execution...")
result = agent.start(processed_question)
print(f"\n📝 Agent Response:\n{result}")
# Display performance statistics
print("\n" + "=" * 60)
print("📊 PERFORMANCE MONITORING RESULTS")
print("=" * 60)
# Get and display performance report
report = get_performance_report()
print(report)
return result
if __name__ == "__main__":
result = main()
print(f"\n✅ Example completed successfully!")
@@ -0,0 +1,143 @@
"""
Multi-Agent Workflow with Performance Monitoring - Example 2
Demonstrates performance monitoring in a multi-agent workflow:
- Multiple agents with different roles
- Inter-agent communication tracking
- Workflow performance analysis
- Task delegation timing
Shows how to monitor complex multi-agent systems.
"""
from praisonaiagents import Agent, Task, PraisonAIAgents
from praisonaiagents.telemetry import (
monitor_function, track_api_call, get_function_stats,
analyze_function_flow, generate_comprehensive_report
)
import time
@monitor_function("research_task")
def setup_research_task():
"""Set up research task with monitoring."""
return Task(
name="research_task",
description="Research the topic of renewable energy trends",
expected_output="A comprehensive research summary"
)
@monitor_function("analysis_task")
def setup_analysis_task():
"""Set up analysis task with monitoring."""
return Task(
name="analysis_task",
description="Analyze the research data and identify key trends",
expected_output="Key insights and trend analysis"
)
@monitor_function("report_task")
def setup_report_task():
"""Set up report generation task with monitoring."""
return Task(
name="report_task",
description="Create a final report combining research and analysis",
expected_output="A well-structured final report"
)
@monitor_function("agent_creation")
def create_agents():
"""Create specialized agents with monitoring."""
researcher = Agent(
name="researcher",
role="Research Specialist",
goal="Gather comprehensive information on renewable energy",
backstory="Expert researcher with deep knowledge of energy sector",
instructions="You are a thorough researcher. Provide detailed, fact-based information.",
llm="gpt-4o-mini"
)
analyst = Agent(
name="analyst",
role="Data Analyst",
goal="Analyze trends and patterns in renewable energy data",
backstory="Experienced data analyst specializing in energy markets",
instructions="You are an analytical thinker. Focus on trends, patterns, and insights.",
llm="gpt-4o-mini"
)
writer = Agent(
name="writer",
role="Report Writer",
goal="Create clear, compelling reports",
backstory="Professional technical writer with expertise in energy sector",
instructions="You are a skilled writer. Create clear, well-structured reports.",
llm="gpt-4o-mini"
)
return researcher, analyst, writer
@monitor_function("workflow_execution")
def main():
"""Main function demonstrating multi-agent workflow with performance monitoring."""
print("=" * 70)
print("EXAMPLE 2: Multi-Agent Workflow with Performance Monitoring")
print("=" * 70)
# Create agents with monitoring
researcher, analyst, writer = create_agents()
# Set up tasks with monitoring
research_task = setup_research_task()
analysis_task = setup_analysis_task()
report_task = setup_report_task()
# Assign agents to tasks
research_task.agent = researcher
analysis_task.agent = analyst
report_task.agent = writer
# Create workflow
with track_api_call("multi_agent_workflow"):
print("\n🚀 Starting multi-agent workflow...")
workflow = PraisonAIAgents(
agents=[researcher, analyst, writer],
tasks=[research_task, analysis_task, report_task],
process="sequential",
verbose=True
)
result = workflow.start()
# Display comprehensive performance analysis
print("\n" + "=" * 70)
print("📊 MULTI-AGENT PERFORMANCE ANALYSIS")
print("=" * 70)
# Function statistics
print("\n📈 Function Performance Statistics:")
stats = get_function_stats()
for func_name, data in stats.items():
print(f" {func_name}:")
print(f" Calls: {data['call_count']}")
avg_time = data['total_time'] / data['call_count'] if data['call_count'] > 0 else 0
print(f" Avg Time: {avg_time:.3f}s")
print(f" Total Time: {data['total_time']:.3f}s")
# Flow analysis
print("\n🔄 Execution Flow Analysis:")
flow_analysis = analyze_function_flow()
print(f" Total Functions Monitored: {len(flow_analysis.get('functions', []))}")
print(f" Execution Chains: {len(flow_analysis.get('call_chains', []))}")
# Comprehensive report
print("\n📋 Comprehensive Performance Report:")
report = generate_comprehensive_report()
print(report[:500] + "..." if len(report) > 500 else report)
return result
if __name__ == "__main__":
result = main()
print(f"\n✅ Multi-agent workflow completed successfully!")
@@ -0,0 +1,119 @@
"""
Agent with Tools Performance Monitoring - Example 3
Demonstrates performance monitoring for agents using tools:
- Tool execution timing
- Search operation performance
- Tool call success/failure tracking
- Tool usage analysis
Shows how to monitor agent performance when using external tools.
"""
from praisonaiagents import Agent
from praisonaiagents.tools import duckduckgo, wikipedia
from praisonaiagents.telemetry import (
monitor_function, track_api_call, get_api_stats,
get_slowest_functions, performance_monitor
)
import time
@monitor_function("search_with_duckduckgo")
def perform_web_search(query):
"""Perform web search with performance monitoring."""
print(f"🔍 Searching web for: {query}")
with track_api_call("duckduckgo_search"):
# Simulate search time
time.sleep(0.2)
return f"Search results for: {query}"
@monitor_function("wikipedia_lookup")
def perform_wikipedia_search(topic):
"""Perform Wikipedia search with performance monitoring."""
print(f"📚 Looking up Wikipedia for: {topic}")
with track_api_call("wikipedia_search"):
# Simulate Wikipedia lookup
time.sleep(0.15)
return f"Wikipedia information about: {topic}"
@monitor_function("agent_with_tools_execution")
def main():
"""Main function demonstrating agent with tools and performance monitoring."""
print("=" * 70)
print("EXAMPLE 3: Agent with Tools Performance Monitoring")
print("=" * 70)
# Create agent with tools
agent = Agent(
instructions="""You are a research assistant that helps find information.
Use the available tools to search for information and provide comprehensive answers.
Be thorough in your research and cite your sources.""",
llm="gpt-4o-mini",
tools=[duckduckgo, wikipedia] # Add search tools
)
# Test queries to demonstrate tool performance monitoring
queries = [
"What are the latest developments in artificial intelligence?",
"Tell me about renewable energy innovations in 2024",
"How do neural networks work?"
]
results = []
for i, query in enumerate(queries, 1):
print(f"\n🎯 Query {i}: {query}")
# Monitor the search operations
perform_web_search(query)
perform_wikipedia_search(query)
# Track the full agent execution
with track_api_call(f"agent_query_{i}"):
print(f"\n🚀 Processing with agent...")
# Note: In a real implementation, the agent would use the tools automatically
# For demo purposes, we're simulating the tool usage
result = agent.start(f"Research and answer: {query}")
results.append(result)
print(f"📝 Agent Response: {result[:200]}..." if len(result) > 200 else f"📝 Agent Response: {result}")
# Performance Analysis
print("\n" + "=" * 70)
print("📊 TOOLS PERFORMANCE ANALYSIS")
print("=" * 70)
# API call statistics
print("\n📈 API Call Performance:")
api_stats = get_api_stats()
for api_name, stats in api_stats.items():
print(f" {api_name}:")
print(f" Total Calls: {stats['call_count']}")
print(f" Avg Response Time: {stats['total_time']/stats['call_count']:.3f}s")
print(f" Success Rate: {stats['success_count']/stats['call_count']*100:.1f}%")
print(f" Total Time: {stats['total_time']:.3f}s")
# Slowest functions analysis
print("\n🐌 Slowest Functions:")
slowest = get_slowest_functions()
for func_name, avg_time in slowest:
print(f" {func_name}: {avg_time:.3f}s average")
# Real-time performance data
print("\n⚡ Real-time Performance Data:")
current_stats = performance_monitor.get_function_performance()
print(f" Functions Monitored: {len(current_stats)}")
total_function_calls = sum(stats['call_count'] for stats in current_stats.values())
total_execution_time = sum(stats['total_time'] for stats in current_stats.values())
print(f" Total Function Calls: {total_function_calls}")
print(f" Total Execution Time: {total_execution_time:.3f}s")
print(f" Average Time per Call: {total_execution_time/total_function_calls:.3f}s" if total_function_calls > 0 else " No calls recorded")
return results
if __name__ == "__main__":
results = main()
print(f"\n✅ Tool monitoring example completed! Processed {len(results)} queries.")
@@ -0,0 +1,167 @@
"""
Async Agent with Performance Monitoring - Example 4
Demonstrates performance monitoring for asynchronous agent operations:
- Async function execution timing
- Concurrent task performance
- Async API call tracking
- Parallel operation analysis
Shows how to monitor performance in async/await workflows.
"""
import asyncio
import time
from praisonaiagents import Agent
from praisonaiagents.telemetry import (
monitor_function, track_api_call, get_performance_report,
get_function_stats, visualize_execution_flow
)
@monitor_function("async_data_processing")
async def process_data_async(data_id: int):
"""Async data processing with performance monitoring."""
print(f"⚙️ Processing data {data_id} asynchronously...")
await asyncio.sleep(0.1) # Simulate async processing
return f"Processed data {data_id}"
@monitor_function("async_api_call")
async def simulate_api_call(endpoint: str, delay: float = 0.2):
"""Simulate async API call with performance monitoring."""
print(f"🌐 Calling API endpoint: {endpoint}")
await asyncio.sleep(delay) # Simulate network delay
return f"Response from {endpoint}"
@monitor_function("batch_processing")
async def process_batch(batch_size: int = 3):
"""Process multiple items concurrently with monitoring."""
print(f"📦 Processing batch of {batch_size} items...")
# Create concurrent tasks
tasks = []
for i in range(batch_size):
tasks.append(process_data_async(i + 1))
# Wait for all tasks to complete
results = await asyncio.gather(*tasks)
print(f"✅ Batch processing completed: {len(results)} items processed")
return results
@monitor_function("async_agent_execution")
async def run_async_agent(query: str):
"""Run agent with async operations and monitoring."""
# Create agent
agent = Agent(
instructions="You are an efficient AI assistant that processes requests quickly",
llm="gpt-4o-mini"
)
# Track the agent execution
with track_api_call("async_agent_request"):
print(f"🚀 Running async agent for query: {query}")
# For demo purposes, we simulate async agent execution
# In real scenarios, you'd use agent.astart() or similar async method when available
await asyncio.sleep(0.1) # Simulate async processing time
result = agent.start(query)
# Simulate some async post-processing
await asyncio.sleep(0.05)
return result
@monitor_function("concurrent_agents")
async def run_concurrent_agents():
"""Run multiple agents concurrently with monitoring."""
print("🔄 Running multiple agents concurrently...")
queries = [
"What is machine learning?",
"Explain quantum computing",
"How does blockchain work?"
]
# Create concurrent agent tasks
agent_tasks = []
for i, query in enumerate(queries):
task = run_async_agent(query)
agent_tasks.append(task)
# Execute all agents concurrently
results = await asyncio.gather(*agent_tasks)
return results
async def main():
"""Main async function demonstrating async agent performance monitoring."""
print("=" * 70)
print("EXAMPLE 4: Async Agent Performance Monitoring")
print("=" * 70)
start_time = time.time()
# Run async operations with monitoring
print("\n📊 Phase 1: Batch Processing")
batch_results = await process_batch(3)
print("\n🌐 Phase 2: API Calls")
api_tasks = [
simulate_api_call("user-service", 0.1),
simulate_api_call("data-service", 0.15),
simulate_api_call("analytics-service", 0.2)
]
api_results = await asyncio.gather(*api_tasks)
print("\n🤖 Phase 3: Concurrent Agents")
agent_results = await run_concurrent_agents()
total_time = time.time() - start_time
# Performance Analysis
print("\n" + "=" * 70)
print("📊 ASYNC PERFORMANCE ANALYSIS")
print("=" * 70)
print(f"\n⏱️ Total Execution Time: {total_time:.3f}s")
# Function statistics
print("\n📈 Function Performance:")
stats = get_function_stats()
for func_name, data in stats.items():
avg_time = data['total_time'] / data['call_count'] if data['call_count'] > 0 else 0
print(f" {func_name}:")
print(f" Calls: {data['call_count']}")
print(f" Avg Time: {avg_time:.3f}s")
print(f" Total Time: {data['total_time']:.3f}s")
# Execution flow visualization
print("\n🔄 Execution Flow:")
try:
flow_viz = visualize_execution_flow()
print(" Flow visualization generated successfully")
print(f" Flow data length: {len(str(flow_viz))}")
except Exception as e:
print(f" Flow visualization: {str(e)}")
# Performance report
print("\n📋 Performance Report:")
report = get_performance_report()
print(report[:300] + "..." if len(report) > 300 else report)
# Results summary
print(f"\n✅ Results Summary:")
print(f" Batch items processed: {len(batch_results)}")
print(f" API calls completed: {len(api_results)}")
print(f" Agent queries processed: {len(agent_results)}")
return {
'batch_results': batch_results,
'api_results': api_results,
'agent_results': agent_results,
'total_time': total_time
}
if __name__ == "__main__":
# Run the async main function
result = asyncio.run(main())
print(f"\n🎉 Async monitoring example completed successfully!")
@@ -0,0 +1,195 @@
"""
Error Handling with Performance Monitoring - Example 5
Demonstrates performance monitoring for error scenarios:
- Error rate tracking
- Failed operation timing
- Recovery mechanism performance
- Exception handling with monitoring
Shows how to monitor performance when things go wrong.
"""
import random
import time
from praisonaiagents import Agent
from praisonaiagents.telemetry import (
monitor_function, track_api_call, get_function_stats,
get_api_stats, performance_monitor
)
@monitor_function("unreliable_operation")
def simulate_unreliable_operation(operation_id: int, failure_rate: float = 0.3):
"""Simulate an operation that sometimes fails."""
print(f"⚙️ Running operation {operation_id}...")
# Simulate processing time
time.sleep(random.uniform(0.05, 0.2))
# Random failure simulation
if random.random() < failure_rate:
raise RuntimeError(f"Operation {operation_id} failed!")
return f"Operation {operation_id} succeeded"
@monitor_function("retry_with_backoff")
def retry_operation(operation_id: int, max_retries: int = 3):
"""Retry an operation with exponential backoff and monitoring."""
for attempt in range(max_retries):
try:
print(f"🔄 Attempt {attempt + 1} for operation {operation_id}")
result = simulate_unreliable_operation(operation_id)
print(f"✅ Operation {operation_id} succeeded on attempt {attempt + 1}")
return result
except Exception as e:
print(f"❌ Attempt {attempt + 1} failed: {e}")
if attempt < max_retries - 1:
# Exponential backoff
sleep_time = 2 ** attempt * 0.1
print(f"⏳ Waiting {sleep_time:.1f}s before retry...")
time.sleep(sleep_time)
else:
print(f"💥 All {max_retries} attempts failed for operation {operation_id}")
raise
@monitor_function("robust_agent_execution")
def run_robust_agent(query: str, with_error_simulation: bool = True):
"""Run agent with error handling and performance monitoring."""
agent = Agent(
instructions="""You are a resilient AI assistant.
Even when facing challenges, you provide helpful responses.
If you encounter errors, acknowledge them and offer alternatives.""",
llm="gpt-4o-mini"
)
try:
with track_api_call("robust_agent_request"):
print(f"🚀 Processing query: {query}")
if with_error_simulation and random.random() < 0.2:
# Simulate API error
raise Exception("Simulated API error")
result = agent.start(query)
return result
except Exception as e:
print(f"⚠️ Error during agent execution: {e}")
# Fallback response with monitoring
with track_api_call("fallback_response"):
print("🔄 Using fallback response mechanism...")
time.sleep(0.1) # Simulate fallback processing
return f"I encountered an error while processing '{query}', but I'm designed to be resilient. Let me provide a general response about this topic."
@monitor_function("batch_processing_with_errors")
def process_batch_with_error_handling(batch_size: int = 5):
"""Process a batch of operations with comprehensive error handling."""
print(f"📦 Processing batch of {batch_size} operations with error monitoring...")
results = []
errors = []
for i in range(batch_size):
try:
result = retry_operation(i + 1)
results.append(result)
except Exception as e:
errors.append(f"Operation {i + 1}: {str(e)}")
return results, errors
def main():
"""Main function demonstrating error handling with performance monitoring."""
print("=" * 70)
print("EXAMPLE 5: Error Handling Performance Monitoring")
print("=" * 70)
# Phase 1: Batch processing with errors
print("\n📊 Phase 1: Batch Processing with Error Handling")
successes, failures = process_batch_with_error_handling(5)
print(f"\n📈 Batch Results:")
print(f" Successful operations: {len(successes)}")
print(f" Failed operations: {len(failures)}")
# Phase 2: Agent execution with error scenarios
print("\n🤖 Phase 2: Agent Execution with Error Scenarios")
test_queries = [
"What is artificial intelligence?",
"How do computers work?",
"Explain quantum physics",
]
agent_results = []
for query in test_queries:
try:
result = run_robust_agent(query, with_error_simulation=True)
agent_results.append(result)
print(f"✅ Query processed: '{query[:30]}...'")
except Exception as e:
print(f"❌ Query failed: '{query[:30]}...' - {e}")
# Performance Analysis with Error Metrics
print("\n" + "=" * 70)
print("📊 ERROR HANDLING PERFORMANCE ANALYSIS")
print("=" * 70)
# Function performance with error rates
print("\n📈 Function Performance (including errors):")
stats = get_function_stats()
for func_name, data in stats.items():
avg_time = data['total_time'] / data['call_count'] if data['call_count'] > 0 else 0
error_rate = data['error_count'] / data['call_count'] * 100 if data['call_count'] > 0 else 0
success_rate = 100 - error_rate
print(f" {func_name}:")
print(f" Calls: {data['call_count']}")
print(f" Errors: {data['error_count']}")
print(f" Success Rate: {success_rate:.1f}%")
print(f" Avg Time: {avg_time:.3f}s")
# API call reliability
print("\n🌐 API Call Reliability:")
api_stats = get_api_stats()
for api_name, stats in api_stats.items():
success_rate = stats['success_count'] / stats['call_count'] * 100 if stats['call_count'] > 0 else 0
avg_time = stats['total_time'] / stats['call_count'] if stats['call_count'] > 0 else 0
print(f" {api_name}:")
print(f" Success Rate: {success_rate:.1f}%")
print(f" Average Response Time: {avg_time:.3f}s")
print(f" Total Calls: {stats['call_count']}")
# Error patterns analysis
print("\n⚠️ Error Patterns:")
total_operations = sum(s['call_count'] for s in api_stats.values())
total_errors = sum(s['error_count'] for s in api_stats.values())
overall_success_rate = ((total_operations - total_errors) / total_operations * 100) if total_operations > 0 else 0
print(f" Total Operations: {total_operations}")
print(f" Total Errors: {total_errors}")
print(f" Overall Success Rate: {overall_success_rate:.1f}%")
# Recommendations
print(f"\n💡 Performance Recommendations:")
if overall_success_rate < 90:
print(" - Consider implementing more robust error handling")
print(" - Review retry strategies and backoff algorithms")
if total_errors > 0:
print(f" - Investigate {total_errors} errors for root cause analysis")
print(" - Monitor error patterns for system reliability improvements")
return {
'successful_operations': len(successes),
'failed_operations': len(failures),
'agent_results': len(agent_results),
'overall_success_rate': overall_success_rate
}
if __name__ == "__main__":
result = main()
print(f"\n✅ Error handling monitoring completed!")
print(f"Success rate: {result['overall_success_rate']:.1f}%")
@@ -0,0 +1,218 @@
"""
Memory Agent with Performance Monitoring - Example 6
Demonstrates performance monitoring for agents with memory capabilities:
- Memory storage and retrieval timing
- Knowledge base operations
- Session persistence performance
- Memory search optimization
Shows how to monitor performance in stateful agents.
"""
import time
from praisonaiagents import Agent
from praisonaiagents.telemetry import (
monitor_function, track_api_call, get_performance_report,
get_slowest_functions, analyze_performance_trends
)
# Simulate memory operations since actual memory might not be available
@monitor_function("memory_storage")
def store_memory(key: str, value: str, memory_type: str = "short_term"):
"""Simulate memory storage operation with monitoring."""
print(f"💾 Storing {memory_type} memory: {key}")
# Simulate storage time - short term is faster
storage_time = 0.05 if memory_type == "short_term" else 0.15
time.sleep(storage_time)
return f"Stored {key} in {memory_type} memory"
@monitor_function("memory_retrieval")
def retrieve_memory(key: str, memory_type: str = "short_term"):
"""Simulate memory retrieval operation with monitoring."""
print(f"🔍 Retrieving {memory_type} memory: {key}")
# Simulate retrieval time
retrieval_time = 0.03 if memory_type == "short_term" else 0.12
time.sleep(retrieval_time)
return f"Retrieved {key} from {memory_type} memory"
@monitor_function("knowledge_search")
def search_knowledge_base(query: str):
"""Simulate knowledge base search with monitoring."""
print(f"📚 Searching knowledge base for: {query}")
# Simulate vector search time
time.sleep(0.08)
return f"Knowledge search results for: {query}"
@monitor_function("memory_consolidation")
def consolidate_memories(session_id: str):
"""Simulate memory consolidation process."""
print(f"🔄 Consolidating memories for session: {session_id}")
# Simulate consolidation steps
store_memory("consolidated_session", session_id, "long_term")
retrieve_memory("recent_interactions", "short_term")
time.sleep(0.1)
return f"Memories consolidated for session: {session_id}"
@monitor_function("stateful_agent_interaction")
def run_stateful_agent(query: str, session_id: str, conversation_turn: int):
"""Run agent with memory operations and monitoring."""
# Simulate memory-enabled agent
agent = Agent(
instructions="""You are a memory-enabled AI assistant. You can remember
previous conversations and build on past interactions. Use your memory
to provide personalized and contextual responses.""",
llm="gpt-4o-mini"
)
# Pre-processing: retrieve relevant memories
with track_api_call("memory_preprocessing"):
print(f"🧠 Pre-processing memories for turn {conversation_turn}")
# Retrieve session context
if conversation_turn > 1:
retrieve_memory(f"session_{session_id}", "short_term")
# Search for relevant knowledge
search_knowledge_base(query)
time.sleep(0.05) # Simulate context preparation
# Main agent execution
with track_api_call("memory_agent_execution"):
print(f"🚀 Processing query with memory context...")
result = agent.start(query)
# Post-processing: store new memories
with track_api_call("memory_postprocessing"):
print(f"💭 Post-processing: storing interaction memories")
# Store the interaction
store_memory(f"interaction_{conversation_turn}", query, "short_term")
store_memory(f"response_{conversation_turn}", result, "short_term")
# Update session memory
store_memory(f"session_{session_id}", f"Turn {conversation_turn} completed", "short_term")
time.sleep(0.03) # Simulate memory update
return result
@monitor_function("multi_turn_conversation")
def simulate_multi_turn_conversation():
"""Simulate a multi-turn conversation with memory."""
session_id = "demo_session_001"
conversation = [
"Hello, I'm interested in learning about machine learning",
"What are the main types of machine learning algorithms?",
"Can you give me examples of supervised learning?",
"How does this relate to what we discussed about machine learning earlier?"
]
results = []
for turn, query in enumerate(conversation, 1):
print(f"\n💬 Conversation Turn {turn}")
print(f"User: {query}")
result = run_stateful_agent(query, session_id, turn)
results.append(result)
print(f"Agent: {result[:100]}..." if len(result) > 100 else f"Agent: {result}")
# Simulate user thinking time
time.sleep(0.1)
# End of conversation: consolidate memories
consolidate_memories(session_id)
return results
def main():
"""Main function demonstrating memory agent performance monitoring."""
print("=" * 70)
print("EXAMPLE 6: Memory Agent Performance Monitoring")
print("=" * 70)
# Warm up the memory system
print("\n🚀 Phase 1: Memory System Initialization")
store_memory("system_initialized", "true", "long_term")
search_knowledge_base("initialization")
# Run multi-turn conversation
print("\n💬 Phase 2: Multi-Turn Conversation with Memory")
conversation_results = simulate_multi_turn_conversation()
# Memory performance analysis
print("\n" + "=" * 70)
print("📊 MEMORY PERFORMANCE ANALYSIS")
print("=" * 70)
# Performance trends analysis
print("\n📈 Performance Trends:")
try:
trends = analyze_performance_trends()
print(f" Performance trend analysis completed")
if trends:
print(f" Functions analyzed: {len(trends.get('functions', []))}")
except Exception as e:
print(f" Trends analysis: {str(e)}")
# Slowest operations
print("\n🐌 Slowest Memory Operations:")
slowest = get_slowest_functions()
for func_name, avg_time in slowest[:5]: # Top 5 slowest
print(f" {func_name}: {avg_time:.3f}s average")
# Memory operation breakdown
print("\n💾 Memory Operation Breakdown:")
from praisonaiagents.telemetry import get_function_stats
stats = get_function_stats()
memory_operations = {k: v for k, v in stats.items() if 'memory' in k.lower()}
for op_name, data in memory_operations.items():
avg_time = data['total_time'] / data['call_count'] if data['call_count'] > 0 else 0
print(f" {op_name}:")
print(f" Operations: {data['call_count']}")
print(f" Avg Time: {avg_time:.3f}s")
print(f" Total Time: {data['total_time']:.3f}s")
# Session efficiency metrics
total_memory_ops = sum(data['call_count'] for data in memory_operations.values())
total_memory_time = sum(data['total_time'] for data in memory_operations.values())
print(f"\n🔍 Session Efficiency:")
print(f" Total Memory Operations: {total_memory_ops}")
print(f" Total Memory Time: {total_memory_time:.3f}s")
print(f" Avg Time per Memory Op: {total_memory_time/total_memory_ops:.3f}s" if total_memory_ops > 0 else " No memory operations")
# Performance report
print("\n📋 Comprehensive Performance Report:")
report = get_performance_report()
print(report[:400] + "..." if len(report) > 400 else report)
# Memory optimization suggestions
print(f"\n💡 Memory Optimization Suggestions:")
if total_memory_time > 1.0:
print(" - Consider implementing memory caching strategies")
print(" - Optimize vector search indexes for faster retrieval")
if total_memory_ops > 10:
print(" - Consider batching memory operations for efficiency")
print(" - Monitor memory consolidation frequency vs performance")
return {
'conversation_turns': len(conversation_results),
'memory_operations': total_memory_ops,
'total_memory_time': total_memory_time
}
if __name__ == "__main__":
result = main()
print(f"\n✅ Memory agent monitoring completed!")
print(f"Processed {result['conversation_turns']} conversation turns")
print(f"Executed {result['memory_operations']} memory operations in {result['total_memory_time']:.3f}s")
@@ -0,0 +1,264 @@
"""
Hierarchical Agents with Performance Monitoring - Example 7
Demonstrates performance monitoring for hierarchical agent systems:
- Manager-worker agent relationships
- Task delegation timing
- Hierarchical decision making performance
- Cross-level communication monitoring
Shows how to monitor complex organizational agent structures.
"""
import time
from praisonaiagents import Agent, Task, PraisonAIAgents
from praisonaiagents.telemetry import (
monitor_function, track_api_call, get_function_stats,
visualize_execution_flow, generate_comprehensive_report
)
@monitor_function("manager_decision_making")
def make_management_decision(task_description: str):
"""Simulate manager decision making process."""
print(f"👔 Manager analyzing task: {task_description}")
# Simulate decision complexity
time.sleep(0.1)
# Simple task routing logic
if "research" in task_description.lower():
return "research_worker"
elif "analysis" in task_description.lower():
return "analysis_worker"
elif "report" in task_description.lower():
return "writing_worker"
else:
return "general_worker"
@monitor_function("task_delegation")
def delegate_task(task: str, worker_type: str):
"""Simulate task delegation to workers."""
print(f"📤 Delegating to {worker_type}: {task}")
time.sleep(0.05) # Delegation overhead
return f"Task delegated to {worker_type}"
@monitor_function("worker_execution")
def execute_worker_task(worker_name: str, task_description: str):
"""Simulate worker executing assigned task."""
print(f"⚙️ {worker_name} executing: {task_description}")
# Different workers have different execution times
if "research" in worker_name:
time.sleep(0.2) # Research takes longer
elif "analysis" in worker_name:
time.sleep(0.15) # Analysis is moderate
elif "writing" in worker_name:
time.sleep(0.1) # Writing is relatively quick
else:
time.sleep(0.12) # General tasks
return f"{worker_name} completed: {task_description}"
@monitor_function("result_aggregation")
def aggregate_results(results: list):
"""Simulate aggregating results from multiple workers."""
print(f"📊 Aggregating {len(results)} worker results...")
time.sleep(0.08) # Aggregation time
return f"Aggregated {len(results)} results into final output"
@monitor_function("hierarchical_workflow")
def run_hierarchical_workflow():
"""Run a complete hierarchical workflow with monitoring."""
# Create manager agent
manager = Agent(
name="project_manager",
role="Project Manager",
goal="Efficiently coordinate team efforts and ensure project success",
backstory="Experienced project manager with expertise in team coordination",
instructions="""You are a project manager. Break down complex projects into tasks,
assign them to appropriate team members, and coordinate the overall effort.""",
llm="gpt-4o-mini"
)
# Create worker agents
research_worker = Agent(
name="research_specialist",
role="Research Specialist",
goal="Conduct thorough research on assigned topics",
backstory="Expert researcher with deep analytical skills",
instructions="You are a research specialist. Provide detailed, factual information.",
llm="gpt-4o-mini"
)
analysis_worker = Agent(
name="data_analyst",
role="Data Analyst",
goal="Analyze data and identify patterns and insights",
backstory="Experienced data analyst with strong analytical thinking",
instructions="You are a data analyst. Focus on patterns, trends, and insights.",
llm="gpt-4o-mini"
)
writing_worker = Agent(
name="technical_writer",
role="Technical Writer",
goal="Create clear, well-structured documentation and reports",
backstory="Professional writer specializing in technical content",
instructions="You are a technical writer. Create clear, comprehensive reports.",
llm="gpt-4o-mini"
)
# Project tasks
project_tasks = [
"Research current trends in artificial intelligence",
"Analyze the impact of AI on business productivity",
"Write a comprehensive report on AI implementation strategies"
]
# Execute hierarchical workflow
with track_api_call("hierarchical_project_execution"):
print("🏢 Starting hierarchical project execution...")
all_results = []
for task_desc in project_tasks:
print(f"\n📋 Processing project task: {task_desc}")
# Manager makes decision
assigned_worker = make_management_decision(task_desc)
# Delegate task
delegation_result = delegate_task(task_desc, assigned_worker)
# Select appropriate worker
if "research" in assigned_worker:
worker = research_worker
worker_name = "research_specialist"
elif "analysis" in assigned_worker:
worker = analysis_worker
worker_name = "data_analyst"
elif "writing" in assigned_worker:
worker = writing_worker
worker_name = "technical_writer"
else:
worker = research_worker # Default
worker_name = "research_specialist"
# Worker executes task
worker_result = execute_worker_task(worker_name, task_desc)
# Simulate actual agent execution
with track_api_call(f"{worker_name}_llm_execution"):
agent_result = worker.start(task_desc)
all_results.append(agent_result)
# Manager aggregates results
final_result = aggregate_results(all_results)
# Final manager review
with track_api_call("manager_final_review"):
print("👔 Manager conducting final review...")
manager_review = manager.start(
f"Review and synthesize these team outputs: {[r[:100] + '...' for r in all_results]}"
)
return all_results, manager_review
def main():
"""Main function demonstrating hierarchical agents performance monitoring."""
print("=" * 75)
print("EXAMPLE 7: Hierarchical Agents Performance Monitoring")
print("=" * 75)
# Run hierarchical workflow
print("\n🏢 Running Hierarchical Agent Workflow")
worker_results, manager_review = run_hierarchical_workflow()
print(f"\n✅ Workflow completed with {len(worker_results)} task results")
# Performance Analysis
print("\n" + "=" * 75)
print("📊 HIERARCHICAL PERFORMANCE ANALYSIS")
print("=" * 75)
# Role-based performance breakdown
print("\n👥 Role-Based Performance:")
stats = get_function_stats()
# Categorize functions by role
management_functions = [f for f in stats.keys() if 'manager' in f or 'decision' in f or 'delegation' in f or 'aggregation' in f]
worker_functions = [f for f in stats.keys() if 'worker' in f or 'execution' in f]
coordination_functions = [f for f in stats.keys() if 'hierarchical' in f or 'workflow' in f]
print(" 📋 Management Functions:")
for func in management_functions:
data = stats[func]
avg_time = data['total_time'] / data['call_count'] if data['call_count'] > 0 else 0
print(f" {func}: {data['call_count']} calls, {avg_time:.3f}s avg")
print("\n ⚙️ Worker Functions:")
for func in worker_functions:
data = stats[func]
avg_time = data['total_time'] / data['call_count'] if data['call_count'] > 0 else 0
print(f" {func}: {data['call_count']} calls, {avg_time:.3f}s avg")
print("\n 🔄 Coordination Functions:")
for func in coordination_functions:
data = stats[func]
avg_time = data['total_time'] / data['call_count'] if data['call_count'] > 0 else 0
print(f" {func}: {data['call_count']} calls, {avg_time:.3f}s avg")
# Hierarchy efficiency metrics
total_management_time = sum(stats[f]['total_time'] for f in management_functions)
total_worker_time = sum(stats[f]['total_time'] for f in worker_functions)
total_coordination_time = sum(stats[f]['total_time'] for f in coordination_functions)
print(f"\n⚖️ Hierarchy Efficiency Metrics:")
print(f" Management Overhead: {total_management_time:.3f}s")
print(f" Worker Execution Time: {total_worker_time:.3f}s")
print(f" Coordination Overhead: {total_coordination_time:.3f}s")
total_time = total_management_time + total_worker_time + total_coordination_time
if total_time > 0:
print(f" Worker Efficiency: {total_worker_time/total_time*100:.1f}%")
print(f" Management Overhead: {total_management_time/total_time*100:.1f}%")
print(f" Coordination Overhead: {total_coordination_time/total_time*100:.1f}%")
# Execution flow visualization
print("\n🔄 Execution Flow Analysis:")
try:
flow_viz = visualize_execution_flow()
print(" Hierarchical flow visualization generated")
print(f" Flow complexity: {len(str(flow_viz)) // 100} units")
except Exception as e:
print(f" Flow visualization: {str(e)}")
# Comprehensive report
print("\n📋 Comprehensive Performance Report:")
report = generate_comprehensive_report()
print(report[:500] + "..." if len(report) > 500 else report)
# Optimization recommendations
print(f"\n💡 Hierarchical Optimization Recommendations:")
if total_time > 0:
if total_management_time / total_time > 0.3:
print(" - High management overhead detected - consider task batching")
if total_coordination_time / total_time > 0.2:
print(" - Consider reducing coordination complexity")
if len(worker_results) > 0:
print(f" - Successfully processed {len(worker_results)} tasks hierarchically")
print(" - Monitor delegation efficiency vs direct execution trade-offs")
return {
'tasks_completed': len(worker_results),
'management_time': total_management_time,
'worker_time': total_worker_time,
'coordination_time': total_coordination_time,
'efficiency_ratio': total_worker_time / total_time if total_time > 0 else 0
}
if __name__ == "__main__":
result = main()
print(f"\n🎉 Hierarchical monitoring completed!")
print(f"Tasks: {result['tasks_completed']}, Efficiency: {result['efficiency_ratio']*100:.1f}%")
@@ -0,0 +1,314 @@
"""
Custom Tools with Performance Monitoring - Example 8
Demonstrates performance monitoring for agents using custom tools:
- Custom tool execution timing
- Tool creation and registration performance
- Tool usage pattern analysis
- Tool efficiency optimization
Shows how to monitor performance with custom-built tools.
"""
import time
import random
from praisonaiagents import Agent, tool
from praisonaiagents.telemetry import (
monitor_function, track_api_call, get_function_stats,
get_slowest_functions, performance_monitor
)
# Custom Tool 1: Data Processor
@tool
@monitor_function("data_processor_tool")
def process_data(data: str, operation: str = "analyze") -> str:
"""
Process data with specified operation.
Args:
data: The data to process
operation: Type of operation (analyze, transform, validate)
"""
print(f"🔧 Processing data with operation: {operation}")
# Simulate different processing times based on operation
processing_times = {
"analyze": 0.15,
"transform": 0.2,
"validate": 0.1
}
time.sleep(processing_times.get(operation, 0.12))
return f"Data processed with {operation}: {data[:50]}{'...' if len(data) > 50 else ''}"
# Custom Tool 2: API Simulator
@tool
@monitor_function("api_simulator_tool")
def call_external_api(endpoint: str, method: str = "GET", payload: str = "") -> str:
"""
Simulate calling an external API.
Args:
endpoint: API endpoint to call
method: HTTP method (GET, POST, PUT, DELETE)
payload: Request payload for POST/PUT
"""
print(f"🌐 Calling API: {method} {endpoint}")
# Simulate variable API response times
base_time = 0.1
if method == "POST":
base_time += 0.05
if payload:
base_time += len(payload) * 0.001
# Add random variation
response_time = base_time + random.uniform(0, 0.1)
time.sleep(response_time)
# Simulate occasional API failures
if random.random() < 0.1: # 10% failure rate
raise Exception(f"API call to {endpoint} failed")
return f"API Response from {endpoint}: {{'status': 'success', 'method': '{method}'}}"
# Custom Tool 3: Database Simulator
@tool
@monitor_function("database_simulator_tool")
def query_database(query: str, database: str = "main") -> str:
"""
Simulate database query execution.
Args:
query: SQL query to execute
database: Database name to query
"""
print(f"🗄️ Executing database query on {database}")
# Simulate query complexity based on keywords
complexity_keywords = {
"JOIN": 0.08,
"GROUP BY": 0.06,
"ORDER BY": 0.04,
"WHERE": 0.03
}
query_time = 0.05 # Base query time
for keyword, additional_time in complexity_keywords.items():
if keyword in query.upper():
query_time += additional_time
time.sleep(query_time)
return f"Query executed on {database}: Found {random.randint(1, 100)} results"
# Custom Tool 4: File Operations
@tool
@monitor_function("file_operations_tool")
def file_operation(operation: str, filename: str, content: str = "") -> str:
"""
Simulate file operations.
Args:
operation: Operation type (read, write, delete, copy)
filename: Target filename
content: Content for write operations
"""
print(f"📁 File operation: {operation} on {filename}")
# Simulate different operation times
operation_times = {
"read": 0.03,
"write": 0.05,
"delete": 0.02,
"copy": 0.07
}
time.sleep(operation_times.get(operation, 0.04))
if operation == "write" and content:
time.sleep(len(content) * 0.0001) # Simulate write time based on content size
return f"File {operation} completed for {filename}"
@monitor_function("tool_performance_test")
def test_tool_performance():
"""Test individual tool performance."""
print("🧪 Testing individual tool performance...")
# Test each tool multiple times
tools_results = []
# Test data processor
for op in ["analyze", "transform", "validate"]:
result = process_data("Sample data for testing", op)
tools_results.append(f"DataProcessor[{op}]: {result}")
# Test API simulator
for method in ["GET", "POST"]:
try:
result = call_external_api("/api/test", method, "test payload" if method == "POST" else "")
tools_results.append(f"APISimulator[{method}]: {result}")
except Exception as e:
tools_results.append(f"APISimulator[{method}]: Failed - {e}")
# Test database operations
queries = [
"SELECT * FROM users",
"SELECT COUNT(*) FROM orders WHERE date > '2024-01-01'",
"SELECT u.name, COUNT(o.id) FROM users u JOIN orders o GROUP BY u.id"
]
for query in queries:
result = query_database(query)
tools_results.append(f"Database: {result}")
# Test file operations
for op in ["read", "write", "copy"]:
result = file_operation(op, "test_file.txt", "Test content" if op == "write" else "")
tools_results.append(f"FileOps[{op}]: {result}")
return tools_results
@monitor_function("agent_with_custom_tools")
def run_agent_with_custom_tools():
"""Run agent using custom tools with performance monitoring."""
# Create agent with custom tools
agent = Agent(
instructions="""You are a technical assistant with access to various tools.
Use the appropriate tools to help with data processing, API calls, database queries,
and file operations. Always explain what tools you're using and why.""",
llm="gpt-4o-mini",
tools=[process_data, call_external_api, query_database, file_operation]
)
# Test scenarios for the agent
scenarios = [
"I need to analyze some customer data and store the results",
"Please fetch user information from the API and save it to a file",
"Query the database for recent orders and process the results"
]
results = []
for i, scenario in enumerate(scenarios, 1):
print(f"\n🎯 Scenario {i}: {scenario}")
with track_api_call(f"custom_tools_scenario_{i}"):
try:
result = agent.start(scenario)
results.append(result)
print(f"✅ Scenario {i} completed")
except Exception as e:
print(f"❌ Scenario {i} failed: {e}")
results.append(f"Failed: {e}")
return results
def main():
"""Main function demonstrating custom tools performance monitoring."""
print("=" * 70)
print("EXAMPLE 8: Custom Tools Performance Monitoring")
print("=" * 70)
# Phase 1: Individual tool performance testing
print("\n🔧 Phase 1: Individual Tool Performance Testing")
tool_test_results = test_tool_performance()
print(f"\n📊 Tool Test Summary:")
print(f" Total tool operations: {len(tool_test_results)}")
for result in tool_test_results[:5]: # Show first 5 results
print(f" - {result}")
if len(tool_test_results) > 5:
print(f" ... and {len(tool_test_results) - 5} more")
# Phase 2: Agent using custom tools
print("\n🤖 Phase 2: Agent with Custom Tools")
agent_results = run_agent_with_custom_tools()
# Performance Analysis
print("\n" + "=" * 70)
print("📊 CUSTOM TOOLS PERFORMANCE ANALYSIS")
print("=" * 70)
# Tool-specific performance
print("\n🔧 Tool Performance Breakdown:")
stats = get_function_stats()
tool_functions = {k: v for k, v in stats.items() if 'tool' in k.lower()}
for tool_name, data in tool_functions.items():
avg_time = data['total_time'] / data['call_count'] if data['call_count'] > 0 else 0
print(f" {tool_name}:")
print(f" Calls: {data['call_count']}")
print(f" Avg Time: {avg_time:.3f}s")
print(f" Total Time: {data['total_time']:.3f}s")
if data['error_count'] > 0:
print(f" Errors: {data['error_count']}")
# Tool efficiency ranking
print("\n🏆 Tool Efficiency Ranking (by average execution time):")
tool_efficiency = []
for tool_name, data in tool_functions.items():
if data['call_count'] > 0:
avg_time = data['total_time'] / data['call_count']
tool_efficiency.append((tool_name, avg_time))
tool_efficiency.sort(key=lambda x: x[1])
for rank, (tool_name, avg_time) in enumerate(tool_efficiency, 1):
print(f" {rank}. {tool_name}: {avg_time:.3f}s")
# Slowest operations
print("\n🐌 Slowest Tool Operations:")
slowest = get_slowest_functions()
tool_slowest = [(name, time) for name, time in slowest if 'tool' in name.lower()]
for tool_name, avg_time in tool_slowest[:3]:
print(f" {tool_name}: {avg_time:.3f}s")
# Usage patterns
total_tool_calls = sum(data['call_count'] for data in tool_functions.values())
total_tool_time = sum(data['total_time'] for data in tool_functions.values())
print(f"\n📈 Tool Usage Summary:")
print(f" Total Tool Calls: {total_tool_calls}")
print(f" Total Tool Execution Time: {total_tool_time:.3f}s")
print(f" Average Time per Tool Call: {total_tool_time/total_tool_calls:.3f}s" if total_tool_calls > 0 else " No tool calls")
# Tool reliability
total_tool_errors = sum(data['error_count'] for data in tool_functions.values())
reliability = ((total_tool_calls - total_tool_errors) / total_tool_calls * 100) if total_tool_calls > 0 else 100
print(f" Tool Reliability: {reliability:.1f}%")
if total_tool_errors > 0:
print(f" Total Errors: {total_tool_errors}")
# Optimization recommendations
print(f"\n💡 Custom Tools Optimization Recommendations:")
if tool_efficiency:
slowest_tool = tool_efficiency[-1]
if slowest_tool[1] > 0.2:
print(f" - Optimize {slowest_tool[0]} (slowest at {slowest_tool[1]:.3f}s)")
if total_tool_errors > 0:
print(f" - Investigate {total_tool_errors} tool errors for stability improvements")
if total_tool_calls > 20:
print(" - Consider tool caching for frequently used operations")
print(" - Monitor tool usage patterns for optimization opportunities")
return {
'tool_operations': len(tool_test_results),
'agent_scenarios': len(agent_results),
'total_tool_calls': total_tool_calls,
'tool_reliability': reliability
}
if __name__ == "__main__":
result = main()
print(f"\n✅ Custom tools monitoring completed!")
print(f"Operations: {result['tool_operations']}, Reliability: {result['tool_reliability']:.1f}%")
@@ -0,0 +1,286 @@
"""
Streaming & Real-Time Performance Monitoring - Example 9
Demonstrates performance monitoring for streaming and real-time operations:
- Real-time performance tracking
- Streaming response monitoring
- Live performance metrics
- Continuous performance analysis
Shows how to monitor performance during streaming operations.
"""
import time
import threading
from datetime import datetime
from praisonaiagents import Agent
from praisonaiagents.telemetry import (
monitor_function, track_api_call, performance_monitor,
get_performance_report, get_function_stats
)
# Global variables for streaming simulation
streaming_active = False
streaming_stats = {
'messages_processed': 0,
'total_latency': 0.0,
'errors': 0,
'start_time': None
}
@monitor_function("stream_message_processing")
def process_stream_message(message_id: int, content: str):
"""Process a single streaming message with monitoring."""
processing_start = time.time()
print(f"📨 Processing stream message {message_id}: {content[:30]}...")
# Simulate variable processing time
processing_time = 0.05 + (message_id % 3) * 0.02 # Simulate varying complexity
time.sleep(processing_time)
# Simulate occasional errors
if message_id % 20 == 0: # Every 20th message fails
raise Exception(f"Processing error for message {message_id}")
processing_end = time.time()
latency = processing_end - processing_start
# Update streaming stats
streaming_stats['messages_processed'] += 1
streaming_stats['total_latency'] += latency
return f"Processed message {message_id} in {latency:.3f}s"
@monitor_function("real_time_agent_response")
def get_real_time_agent_response(query: str, response_id: int):
"""Simulate real-time agent response with monitoring."""
print(f"🤖 Generating real-time response {response_id} for: {query}")
# Create a lightweight agent for streaming
agent = Agent(
instructions="You are a real-time assistant. Provide quick, helpful responses.",
llm="gpt-4o-mini"
)
with track_api_call(f"streaming_response_{response_id}"):
# Simulate streaming response generation
time.sleep(0.1) # Base response time
# In real implementation, this would be actual agent.start()
response = agent.start(f"Quickly respond to: {query}")
return response
@monitor_function("performance_metrics_collector")
def collect_live_metrics():
"""Collect live performance metrics during streaming."""
while streaming_active:
current_time = time.time()
# Collect current performance data
current_stats = performance_monitor.get_function_performance()
api_stats = performance_monitor.get_api_call_performance()
# Calculate streaming metrics
if streaming_stats['start_time']:
elapsed_time = current_time - streaming_stats['start_time']
messages_per_second = streaming_stats['messages_processed'] / elapsed_time if elapsed_time > 0 else 0
avg_latency = streaming_stats['total_latency'] / streaming_stats['messages_processed'] if streaming_stats['messages_processed'] > 0 else 0
print(f"📊 Live Metrics - Messages/sec: {messages_per_second:.2f}, Avg Latency: {avg_latency:.3f}s")
time.sleep(2) # Update every 2 seconds
@monitor_function("streaming_simulation")
def simulate_streaming_workload():
"""Simulate a streaming workload with continuous monitoring."""
global streaming_active, streaming_stats
print("🌊 Starting streaming simulation...")
streaming_active = True
streaming_stats['start_time'] = time.time()
# Start metrics collector in background
metrics_thread = threading.Thread(target=collect_live_metrics)
metrics_thread.daemon = True
metrics_thread.start()
# Simulate incoming stream messages
messages = [
"User question about AI",
"Request for weather information",
"Help with programming",
"General knowledge query",
"Technical support request",
"Creative writing assistance",
"Data analysis question",
"Research inquiry"
]
processed_responses = []
try:
for i in range(20): # Process 20 streaming messages
message = messages[i % len(messages)]
try:
# Process the stream message
result = process_stream_message(i + 1, message)
# Generate real-time response every 3rd message
if (i + 1) % 3 == 0:
response = get_real_time_agent_response(message, i + 1)
processed_responses.append(response)
# Brief pause between messages
time.sleep(0.1)
except Exception as e:
streaming_stats['errors'] += 1
print(f"❌ Error processing message {i + 1}: {e}")
finally:
streaming_active = False
print("🛑 Streaming simulation completed")
return processed_responses
@monitor_function("batch_vs_streaming_comparison")
def compare_batch_vs_streaming():
"""Compare batch vs streaming performance."""
print("\n🔄 Comparing Batch vs Streaming Performance...")
# Batch processing simulation
print("\n📦 Batch Processing Test:")
batch_start = time.time()
batch_messages = [f"Batch message {i}" for i in range(10)]
batch_results = []
with track_api_call("batch_processing"):
for i, message in enumerate(batch_messages):
# Simulate batch processing (all at once)
time.sleep(0.08) # Batch processing time
batch_results.append(f"Batch processed: {message}")
batch_time = time.time() - batch_start
# Streaming processing test (already done above)
streaming_time = streaming_stats['total_latency']
streaming_count = streaming_stats['messages_processed']
print(f"\n📊 Batch vs Streaming Comparison:")
print(f" Batch Processing:")
print(f" Messages: {len(batch_results)}")
print(f" Total Time: {batch_time:.3f}s")
print(f" Avg per Message: {batch_time/len(batch_results):.3f}s")
print(f" Streaming Processing:")
print(f" Messages: {streaming_count}")
print(f" Total Processing Time: {streaming_time:.3f}s")
print(f" Avg per Message: {streaming_time/streaming_count:.3f}s" if streaming_count > 0 else " No messages processed")
return batch_time, streaming_time
def main():
"""Main function demonstrating streaming performance monitoring."""
print("=" * 70)
print("EXAMPLE 9: Streaming & Real-Time Performance Monitoring")
print("=" * 70)
# Phase 1: Streaming simulation
print("\n🌊 Phase 1: Streaming Workload Simulation")
streaming_responses = simulate_streaming_workload()
# Phase 2: Batch vs Streaming comparison
print("\n📊 Phase 2: Batch vs Streaming Comparison")
batch_time, streaming_time = compare_batch_vs_streaming()
# Real-time Performance Analysis
print("\n" + "=" * 70)
print("📊 STREAMING PERFORMANCE ANALYSIS")
print("=" * 70)
# Streaming-specific metrics
print("\n🌊 Streaming Metrics:")
elapsed_total = time.time() - streaming_stats['start_time'] if streaming_stats['start_time'] else 0
throughput = streaming_stats['messages_processed'] / elapsed_total if elapsed_total > 0 else 0
avg_latency = streaming_stats['total_latency'] / streaming_stats['messages_processed'] if streaming_stats['messages_processed'] > 0 else 0
error_rate = streaming_stats['errors'] / streaming_stats['messages_processed'] * 100 if streaming_stats['messages_processed'] > 0 else 0
print(f" Messages Processed: {streaming_stats['messages_processed']}")
print(f" Total Processing Time: {streaming_stats['total_latency']:.3f}s")
print(f" Average Latency: {avg_latency:.3f}s")
print(f" Throughput: {throughput:.2f} messages/second")
print(f" Error Rate: {error_rate:.2f}%")
print(f" Errors: {streaming_stats['errors']}")
# Function performance during streaming
print("\n⚡ Real-Time Function Performance:")
stats = get_function_stats()
streaming_functions = {k: v for k, v in stats.items() if 'stream' in k.lower() or 'real_time' in k.lower()}
for func_name, data in streaming_functions.items():
avg_time = data['total_time'] / data['call_count'] if data['call_count'] > 0 else 0
print(f" {func_name}:")
print(f" Calls: {data['call_count']}")
print(f" Avg Time: {avg_time:.3f}s")
print(f" Total Time: {data['total_time']:.3f}s")
# Performance trend analysis
print("\n📈 Performance Trends:")
recent_performance = performance_monitor.get_function_performance()
total_functions = len(recent_performance)
total_calls = sum(data['call_count'] for data in recent_performance.values())
print(f" Functions Monitored: {total_functions}")
print(f" Total Function Calls: {total_calls}")
print(f" Average Calls per Function: {total_calls/total_functions:.1f}" if total_functions > 0 else " No functions monitored")
# Real-time vs Batch efficiency
print(f"\n⚖️ Efficiency Analysis:")
if batch_time > 0 and streaming_time > 0:
efficiency_gain = (batch_time - streaming_time) / batch_time * 100
print(f" Streaming vs Batch Efficiency: {efficiency_gain:+.1f}%")
if avg_latency > 0:
if avg_latency < 0.1:
print(" ✅ Excellent real-time performance (< 100ms)")
elif avg_latency < 0.2:
print(" ⚡ Good real-time performance (< 200ms)")
else:
print(" ⚠️ Consider optimization for better real-time performance")
# Current performance report
print("\n📋 Current Performance Report:")
report = get_performance_report()
print(report[:400] + "..." if len(report) > 400 else report)
# Real-time optimization suggestions
print(f"\n💡 Real-Time Optimization Recommendations:")
if avg_latency > 0.15:
print(f" - Optimize message processing (current avg: {avg_latency:.3f}s)")
if error_rate > 5:
print(f" - Investigate error patterns (current rate: {error_rate:.1f}%)")
if throughput < 10:
print(f" - Consider parallel processing for better throughput")
print(" - Monitor memory usage during extended streaming")
print(" - Implement adaptive batching for optimal performance")
return {
'streaming_responses': len(streaming_responses),
'messages_processed': streaming_stats['messages_processed'],
'throughput': throughput,
'average_latency': avg_latency,
'error_rate': error_rate
}
if __name__ == "__main__":
result = main()
print(f"\n🎉 Streaming monitoring completed!")
print(f"Throughput: {result['throughput']:.2f} msg/s, Latency: {result['average_latency']:.3f}s")
@@ -0,0 +1,433 @@
"""
Comprehensive Performance Dashboard - Example 10
Demonstrates comprehensive performance monitoring dashboard:
- Complete performance analytics
- Advanced reporting and visualization
- Performance trends and insights
- System-wide monitoring overview
Shows how to create a comprehensive performance monitoring solution.
"""
import time
import json
from datetime import datetime, timedelta
from praisonaiagents import Agent, Task, PraisonAIAgents
from praisonaiagents.telemetry import (
monitor_function, track_api_call, performance_monitor,
get_performance_report, get_function_stats, get_api_stats,
get_slowest_functions, get_slowest_apis,
analyze_function_flow, visualize_execution_flow,
analyze_performance_trends, generate_comprehensive_report,
clear_performance_data
)
@monitor_function("dashboard_data_collection")
def collect_comprehensive_data():
"""Collect comprehensive performance data for dashboard."""
print("📊 Collecting comprehensive performance data...")
# Generate diverse workload for demonstration
test_scenarios = [
("Quick query", "What is AI?"),
("Complex analysis", "Analyze the impact of artificial intelligence on modern business practices"),
("Research task", "Research the latest developments in quantum computing"),
("Creative task", "Write a creative story about space exploration"),
("Technical query", "Explain how neural networks process information")
]
# Create different types of agents for variety
agents = {
'quick_agent': Agent(
instructions="You are a quick-response agent. Provide concise answers.",
llm="gpt-4o-mini"
),
'analytical_agent': Agent(
instructions="You are an analytical agent. Provide detailed analysis.",
llm="gpt-4o-mini"
),
'research_agent': Agent(
instructions="You are a research agent. Provide comprehensive information.",
llm="gpt-4o-mini"
),
'creative_agent': Agent(
instructions="You are a creative agent. Provide imaginative responses.",
llm="gpt-4o-mini"
)
}
results = []
for i, (scenario_type, query) in enumerate(test_scenarios):
print(f"\n🎯 Scenario {i+1}: {scenario_type}")
# Select appropriate agent
agent_type = list(agents.keys())[i % len(agents)]
agent = agents[agent_type]
with track_api_call(f"{scenario_type.lower().replace(' ', '_')}_execution"):
try:
result = agent.start(query)
results.append({
'scenario': scenario_type,
'agent': agent_type,
'query': query,
'result': result[:200] + '...' if len(result) > 200 else result,
'success': True
})
except Exception as e:
results.append({
'scenario': scenario_type,
'agent': agent_type,
'query': query,
'error': str(e),
'success': False
})
return results
@monitor_function("multi_agent_workflow_simulation")
def simulate_complex_workflow():
"""Simulate complex multi-agent workflow for comprehensive data."""
print("🔄 Simulating complex multi-agent workflow...")
# Create specialized agents
planner = Agent(
name="planner",
instructions="You are a project planner. Break down tasks and plan execution.",
llm="gpt-4o-mini"
)
executor = Agent(
name="executor",
instructions="You are a task executor. Complete assigned tasks efficiently.",
llm="gpt-4o-mini"
)
reviewer = Agent(
name="reviewer",
instructions="You are a quality reviewer. Review and improve work.",
llm="gpt-4o-mini"
)
# Create workflow tasks
planning_task = Task(
name="planning",
description="Plan a comprehensive report on renewable energy trends",
expected_output="A detailed project plan with timeline and milestones"
)
execution_task = Task(
name="execution",
description="Execute the planned research and compile findings",
expected_output="Research findings and data compilation"
)
review_task = Task(
name="review",
description="Review the research and provide quality assessment",
expected_output="Quality review and improvement recommendations"
)
# Assign agents to tasks
planning_task.agent = planner
execution_task.agent = executor
review_task.agent = reviewer
# Execute workflow
with track_api_call("complex_workflow_execution"):
workflow = PraisonAIAgents(
agents=[planner, executor, reviewer],
tasks=[planning_task, execution_task, review_task],
process="sequential",
verbose=False
)
result = workflow.start()
return result
@monitor_function("performance_benchmarking")
def run_performance_benchmarks():
"""Run performance benchmarks for various operations."""
print("⚡ Running performance benchmarks...")
benchmark_results = {}
# Benchmark 1: Response time under different loads
print(" 📈 Benchmarking response times...")
response_times = []
for i in range(10):
start_time = time.time()
with track_api_call(f"benchmark_response_{i}"):
# Simulate quick agent response
time.sleep(0.05 + (i % 3) * 0.02) # Variable response time
response_times.append(time.time() - start_time)
benchmark_results['response_times'] = {
'min': min(response_times),
'max': max(response_times),
'avg': sum(response_times) / len(response_times),
'samples': len(response_times)
}
# Benchmark 2: Concurrent operations
print(" 🔄 Benchmarking concurrent operations...")
concurrent_start = time.time()
# Simulate concurrent operations
for i in range(5):
with track_api_call(f"concurrent_op_{i}"):
time.sleep(0.1) # Simulate concurrent work
concurrent_time = time.time() - concurrent_start
benchmark_results['concurrent_operations'] = {
'total_time': concurrent_time,
'operations': 5,
'avg_per_operation': concurrent_time / 5
}
# Benchmark 3: Memory usage simulation
print(" 💾 Benchmarking memory operations...")
memory_ops = []
for i in range(8):
start = time.time()
with track_api_call(f"memory_benchmark_{i}"):
# Simulate memory-intensive operation
time.sleep(0.03 + (i % 2) * 0.02)
memory_ops.append(time.time() - start)
benchmark_results['memory_operations'] = {
'operations': len(memory_ops),
'total_time': sum(memory_ops),
'avg_time': sum(memory_ops) / len(memory_ops)
}
return benchmark_results
@monitor_function("generate_dashboard_report")
def generate_dashboard_report():
"""Generate comprehensive dashboard report."""
print("📋 Generating comprehensive dashboard report...")
# Collect all performance data
function_stats = get_function_stats()
api_stats = get_api_stats()
slowest_functions = get_slowest_functions()
slowest_apis = get_slowest_apis()
# Generate reports
basic_report = get_performance_report()
comprehensive_report = generate_comprehensive_report()
# Analyze trends and flows
try:
flow_analysis = analyze_function_flow()
performance_trends = analyze_performance_trends()
flow_visualization = visualize_execution_flow()
except Exception as e:
flow_analysis = {"error": str(e)}
performance_trends = {"error": str(e)}
flow_visualization = {"error": str(e)}
dashboard_data = {
'timestamp': datetime.now().isoformat(),
'summary': {
'total_functions': len(function_stats),
'total_api_calls': len(api_stats),
'total_function_calls': sum(stats['call_count'] for stats in function_stats.values()),
'total_api_requests': sum(stats['call_count'] for stats in api_stats.values()),
'total_execution_time': sum(stats['total_time'] for stats in function_stats.values()),
'total_errors': sum(stats['error_count'] for stats in function_stats.values())
},
'performance_data': {
'function_stats': function_stats,
'api_stats': api_stats,
'slowest_functions': slowest_functions,
'slowest_apis': slowest_apis
},
'analysis': {
'flow_analysis': flow_analysis,
'performance_trends': performance_trends,
'flow_visualization': len(str(flow_visualization)) if flow_visualization else 0
},
'reports': {
'basic_report': basic_report,
'comprehensive_report': comprehensive_report[:500] + '...' if len(comprehensive_report) > 500 else comprehensive_report
}
}
return dashboard_data
def display_dashboard(dashboard_data, benchmark_results):
"""Display comprehensive performance dashboard."""
print("\n" + "=" * 80)
print("📊 COMPREHENSIVE PERFORMANCE DASHBOARD")
print("=" * 80)
# Executive Summary
summary = dashboard_data['summary']
print(f"\n📈 EXECUTIVE SUMMARY")
print(f" Report Generated: {dashboard_data['timestamp']}")
print(f" Functions Monitored: {summary['total_functions']}")
print(f" API Endpoints: {summary['total_api_calls']}")
print(f" Total Function Calls: {summary['total_function_calls']}")
print(f" Total API Requests: {summary['total_api_requests']}")
print(f" Total Execution Time: {summary['total_execution_time']:.3f}s")
print(f" Total Errors: {summary['total_errors']}")
# Performance Overview
print(f"\n⚡ PERFORMANCE OVERVIEW")
avg_function_time = 0
if summary['total_function_calls'] > 0:
avg_function_time = summary['total_execution_time'] / summary['total_function_calls']
print(f" Average Function Execution: {avg_function_time:.3f}s")
if summary['total_errors'] > 0 and summary['total_function_calls'] > 0:
error_rate = summary['total_errors'] / summary['total_function_calls'] * 100
print(f" Error Rate: {error_rate:.2f}%")
else:
print(f" Error Rate: 0.00%")
# Top Performers and Issues
print(f"\n🏆 TOP PERFORMERS")
perf_data = dashboard_data['performance_data']
print(" Fastest Functions:")
fastest_functions = sorted(perf_data['slowest_functions'], key=lambda x: x[1])[:3]
for name, time in fastest_functions:
print(f" {name}: {time:.3f}s")
print(" Slowest Functions:")
for name, time in perf_data['slowest_functions'][:3]:
print(f" {name}: {time:.3f}s")
# Benchmark Results
print(f"\n⚡ BENCHMARK RESULTS")
print(f" Response Time Benchmarks:")
rt = benchmark_results['response_times']
print(f" Min: {rt['min']:.3f}s, Max: {rt['max']:.3f}s, Avg: {rt['avg']:.3f}s")
print(f" Concurrent Operations:")
co = benchmark_results['concurrent_operations']
print(f" {co['operations']} operations in {co['total_time']:.3f}s")
print(f" Avg per operation: {co['avg_per_operation']:.3f}s")
print(f" Memory Operations:")
mo = benchmark_results['memory_operations']
print(f" {mo['operations']} operations, {mo['avg_time']:.3f}s average")
# System Health
print(f"\n🏥 SYSTEM HEALTH")
# Calculate health score
health_score = 100
if avg_function_time > 0.2:
health_score -= 20
if summary['total_errors'] > 0:
health_score -= summary['total_errors'] * 10
if rt['avg'] > 0.1:
health_score -= 15
health_score = max(0, health_score)
if health_score >= 90:
status = "🟢 Excellent"
elif health_score >= 70:
status = "🟡 Good"
elif health_score >= 50:
status = "🟠 Fair"
else:
status = "🔴 Needs Attention"
print(f" Overall Health Score: {health_score}/100 - {status}")
# Analysis Results
print(f"\n🔍 ANALYSIS RESULTS")
analysis = dashboard_data['analysis']
if 'error' not in analysis['flow_analysis']:
flow_data = analysis['flow_analysis']
print(f" Flow Analysis: {len(flow_data.get('functions', []))} functions analyzed")
else:
print(f" Flow Analysis: {analysis['flow_analysis']['error']}")
if analysis['flow_visualization']:
print(f" Flow Visualization: Generated ({analysis['flow_visualization']} chars)")
# Recommendations
print(f"\n💡 PERFORMANCE RECOMMENDATIONS")
if avg_function_time > 0.15:
print(" 🔧 Optimize slow functions for better response times")
if summary['total_errors'] > 0:
print(" 🚨 Investigate and resolve error sources")
if rt['avg'] > 0.08:
print(" ⚡ Consider response time optimization strategies")
if summary['total_function_calls'] > 50:
print(" 📊 Implement performance caching for high-volume functions")
print(" 📈 Continue monitoring trends for proactive optimization")
print(" 🔄 Regular performance reviews recommended")
def main():
"""Main function for comprehensive performance dashboard."""
print("=" * 80)
print("EXAMPLE 10: Comprehensive Performance Dashboard")
print("=" * 80)
print("This example demonstrates a complete performance monitoring solution")
print("with advanced analytics, benchmarking, and dashboard reporting.\n")
# Clear previous data for clean demo
clear_performance_data()
# Phase 1: Data Collection
print("📊 Phase 1: Comprehensive Data Collection")
collection_results = collect_comprehensive_data()
# Phase 2: Complex Workflow
print("\n🔄 Phase 2: Complex Multi-Agent Workflow")
workflow_result = simulate_complex_workflow()
# Phase 3: Performance Benchmarking
print("\n⚡ Phase 3: Performance Benchmarking")
benchmark_results = run_performance_benchmarks()
# Phase 4: Dashboard Generation
print("\n📋 Phase 4: Dashboard Report Generation")
dashboard_data = generate_dashboard_report()
# Phase 5: Display Comprehensive Dashboard
display_dashboard(dashboard_data, benchmark_results)
# Final Summary
print(f"\n🎯 EXECUTION SUMMARY")
print(f" Scenarios Executed: {len(collection_results)}")
successful_scenarios = sum(1 for r in collection_results if r['success'])
print(f" Successful Scenarios: {successful_scenarios}/{len(collection_results)}")
print(f" Workflow Completed: {'✅ Yes' if workflow_result else '❌ No'}")
print(f" Benchmarks Run: {len(benchmark_results)}")
print(f" Dashboard Generated: ✅ Complete")
return {
'scenarios': len(collection_results),
'success_rate': successful_scenarios / len(collection_results) * 100,
'dashboard_data': dashboard_data,
'benchmark_results': benchmark_results
}
if __name__ == "__main__":
result = main()
print(f"\n🎉 Comprehensive dashboard monitoring completed!")
print(f"Success Rate: {result['success_rate']:.1f}%")
print(f"Total Functions Monitored: {result['dashboard_data']['summary']['total_functions']}")
print(f"Performance Health Score Available in Dashboard Above ⬆️")
print("\n💡 This dashboard provides real-time insights into your PraisonAI system performance!")
+5
View File
@@ -0,0 +1,5 @@
from praisonaiagents import ContextAgent
agent = ContextAgent(llm="gpt-4o-mini", auto_analyze=False)
agent.start("https://github.com/MervinPraison/PraisonAI/ Need to add Authentication")
+3 -21
View File
@@ -1,25 +1,7 @@
from praisonaiagents import Agent
def get_stock_price(company_name: str) -> str:
"""
Get the stock price of a company
Args:
company_name (str): The name of the company
Returns:
str: The stock price of the company
"""
return f"The stock price of {company_name} is 100"
def multiply(a: int, b: int) -> int:
"""
Multiply two numbers
"""
return a * b
agent = Agent(
instructions="You are a helpful assistant. You can use the tools provided to you to help the user.",
llm="gemini/gemini-2.5-pro",
tools=[get_stock_price, multiply]
instructions="You are a helpful assistant",
llm="gemini/gemini-2.5-flash"
)
result = agent.start("what is the stock price of Google? multiply the Google stock price with 2")
agent.start("Why sky is Blue?")
+20
View File
@@ -0,0 +1,20 @@
# PraisonAI Documentation
## Docs
- [AI Agents](https://docs.praison.ai/docs/agents/agents.md): Overview of all available PraisonAI agents and their capabilities
- [Data Analyst Agent](https://docs.praison.ai/docs/agents/data-analyst.md): Learn how to create AI agents for data analysis and insights generation.
- [Finance Agent](https://docs.praison.ai/docs/agents/finance.md): Learn how to create AI agents for financial analysis and investment recommendations.
- [Image Analysis Agent](https://docs.praison.ai/docs/agents/image.md): Learn how to create AI agents for image analysis and visual content understanding.
- [Image to Text Agent](https://docs.praison.ai/docs/agents/image-to-text.md): Learn how to create AI agents for converting images to textual descriptions and extracting text from images.
- [Markdown Agent](https://docs.praison.ai/docs/agents/markdown.md): Learn how to create AI agents for generating and formatting content in Markdown.
- [Planning Agent](https://docs.praison.ai/docs/agents/planning.md): Learn how to create AI agents for trip planning and itinerary generation.
- [Programming Agent](https://docs.praison.ai/docs/agents/programming.md): Learn how to create AI agents for code development, analysis, and execution.
- [Recommendation Agent](https://docs.praison.ai/docs/agents/recommendation.md): Learn how to create AI agents for personalized recommendations across various domains.
- [Research Agent](https://docs.praison.ai/docs/agents/research.md): Learn how to create AI agents for conducting comprehensive research and analysis.
- [Shopping Agent](https://docs.praison.ai/docs/agents/shopping.md): Learn how to create AI agents for price comparison and shopping assistance.
- [Single Agent](https://docs.praison.ai/docs/agents/single.md): Learn how to create a basic single-purpose AI agent for simple tasks.
- [Video Agent](https://docs.praison.ai/docs/agents/video.md): Learn how to create AI agents for video analysis and content understanding.
- [Web Search Agent](https://docs.praison.ai/docs/agents/websearch.md): Learn how to create AI agents for intelligent web searching and information gathering.
- [Wikipedia Agent](https://docs.praison.ai/docs/agents/wikipedia.md): Learn how to create AI agents for searching and extracting information from Wikipedia.
- [Agent Module](https://docs.praison.ai/docs/api/praisonaiagents/agent/agent.md): Documentation for the praisonaiagents.agent.agent module
@@ -0,0 +1,297 @@
#!/usr/bin/env python3
"""
Comprehensive MongoDB Integration Example for PraisonAI Agents
This example demonstrates a complete MongoDB integration with PraisonAI agents,
combining memory, knowledge, and tools for a comprehensive data management system.
Prerequisites:
- Install MongoDB dependencies: pip install 'praisonaiagents[mongodb]'
- MongoDB server running (local or Atlas)
- OpenAI API key for embeddings
Features demonstrated:
- MongoDB as memory provider
- MongoDB as knowledge store
- MongoDB tools for data operations
- Vector search capabilities
- Multi-agent coordination with MongoDB
- Real-world business scenario simulation
"""
import os
from praisonaiagents import Agent, Task, PraisonAIAgents
from praisonaiagents.tools import mongodb_tools
# Ensure OpenAI API key is set
if not os.environ.get("OPENAI_API_KEY"):
raise ValueError("Please set the OPENAI_API_KEY environment variable")
def main():
# MongoDB configuration for different components
mongodb_memory_config = {
"provider": "mongodb",
"config": {
"connection_string": "mongodb://localhost:27017/", # Replace with your connection string
"database": "praisonai_comprehensive",
"use_vector_search": True,
"max_pool_size": 50
}
}
mongodb_knowledge_config = {
"vector_store": {
"provider": "mongodb",
"config": {
"connection_string": "mongodb://localhost:27017/",
"database": "praisonai_comprehensive",
"collection": "knowledge_base",
"use_vector_search": True
}
},
"embedder": {
"provider": "openai",
"config": {
"model": "text-embedding-3-small",
"api_key": os.getenv("OPENAI_API_KEY")
}
}
}
# Create specialized agents for different business functions
# 1. Data Manager Agent - handles data operations
data_manager = Agent(
name="Data Manager",
role="Senior Data Operations Manager",
goal="Manage and organize all data operations using MongoDB",
backstory="""You are an expert data operations manager who specializes in
MongoDB database management. You handle data ingestion, organization,
indexing, and ensure data quality and accessibility across the organization.""",
tools=[mongodb_tools],
memory=True,
verbose=True,
llm="gpt-4o-mini"
)
# 2. Knowledge Curator Agent - manages knowledge base
knowledge_curator = Agent(
name="Knowledge Curator",
role="Knowledge Management Specialist",
goal="Curate and maintain organizational knowledge using MongoDB",
backstory="""You are a knowledge management specialist who excels at
organizing, categorizing, and retrieving information from the knowledge base.
You ensure that knowledge is properly stored, indexed, and accessible.""",
knowledge_config=mongodb_knowledge_config,
memory=True,
verbose=True,
llm="gpt-4o-mini"
)
# 3. Business Analyst Agent - analyzes data and provides insights
business_analyst = Agent(
name="Business Analyst",
role="Senior Business Intelligence Analyst",
goal="Analyze business data and provide actionable insights",
backstory="""You are a senior business analyst who specializes in extracting
insights from data stored in MongoDB. You create reports, identify trends,
and provide strategic recommendations based on data analysis.""",
tools=[mongodb_tools],
memory=True,
verbose=True,
llm="gpt-4o-mini"
)
# 4. Customer Service Agent - handles customer interactions
customer_service = Agent(
name="Customer Service Agent",
role="Customer Service Representative",
goal="Provide excellent customer service using stored knowledge and data",
backstory="""You are a customer service representative who uses the
knowledge base and customer data to provide excellent support. You can
access customer history, product information, and troubleshooting guides.""",
knowledge_config=mongodb_knowledge_config,
tools=[mongodb_tools],
memory=True,
verbose=True,
llm="gpt-4o-mini"
)
# Create comprehensive business tasks
business_tasks = [
Task(
description="""Set up comprehensive business data infrastructure:
1. Create collections for customers, products, orders, and support tickets
2. Insert sample business data for an e-commerce platform
3. Set up appropriate indexes for performance optimization
4. Create vector indexes for product descriptions and customer support
5. Implement data validation and business rules
Focus on creating a realistic business data structure.
""",
expected_output="Complete business data infrastructure with sample data",
agent=data_manager
),
Task(
description="""Build organizational knowledge base:
1. Store product documentation and specifications
2. Create customer service guidelines and FAQs
3. Add troubleshooting guides and best practices
4. Include company policies and procedures
5. Organize knowledge by categories and tags
Ensure knowledge is searchable and well-organized.
""",
expected_output="Comprehensive knowledge base with categorized information",
agent=knowledge_curator
),
Task(
description="""Perform business intelligence analysis:
1. Analyze customer purchase patterns and behavior
2. Calculate key business metrics (revenue, conversion rates, etc.)
3. Identify top-selling products and categories
4. Analyze customer support ticket trends
5. Generate insights for business optimization
Use MongoDB aggregation and analytics capabilities.
""",
expected_output="Business intelligence report with actionable insights",
agent=business_analyst
),
Task(
description="""Handle customer service scenarios:
1. Process customer inquiries using knowledge base
2. Look up customer order history and preferences
3. Provide product recommendations based on customer data
4. Access troubleshooting guides for technical issues
5. Update customer records and interaction history
Demonstrate excellent customer service with data-driven support.
""",
expected_output="Customer service interactions with knowledge-based responses",
agent=customer_service
),
Task(
description="""Generate comprehensive business report:
1. Combine insights from all data sources
2. Create executive summary with key findings
3. Provide recommendations for business improvement
4. Include data visualizations and metrics
5. Store report in knowledge base for future reference
Create a comprehensive business intelligence report.
""",
expected_output="Executive business report with comprehensive insights",
agent=business_analyst
)
]
# Initialize the comprehensive business system
print("🚀 Starting Comprehensive MongoDB Business System...")
print("=" * 60)
business_system = PraisonAIAgents(
agents=[data_manager, knowledge_curator, business_analyst, customer_service],
tasks=business_tasks,
memory=True,
memory_config=mongodb_memory_config,
verbose=True
)
# Execute the comprehensive business pipeline
try:
results = business_system.start()
print("\n" + "=" * 60)
print("📊 Comprehensive Business System Results:")
print("=" * 60)
# Handle results structure properly
if isinstance(results, (list, tuple)):
for i, result in enumerate(results, 1):
print(f"\n{i}. Task Result:")
if hasattr(result, 'raw') and hasattr(result, 'agent'):
print(f" Output: {result.raw[:200]}...")
print(f" Agent: {result.agent}")
else:
print(f" Output: {str(result)[:200]}...")
else:
print(f"\n1. Task Result:")
print(f" Output: {str(results)[:200]}...")
print("\n" + "=" * 60)
print("💾 Comprehensive MongoDB Integration Complete!")
print("=" * 60)
# Demonstrate system integration
print("\n🔍 Demonstrating System Integration:")
print("-" * 40)
# Access different components
# Note: Direct memory access not available through business_system.memory
print("📋 Memory integration enabled through memory_config...")
print("Memory operations are handled internally by the agents")
# Access memory system directly
from praisonaiagents.memory import Memory
memory_system = Memory(config=mongodb_memory_config)
# Search business memories
print("\n📋 Searching business memories...")
business_memories = memory_system.search_long_term("business analysis", limit=3)
print(f"Found {len(business_memories)} business-related memories")
# Access knowledge system
from praisonaiagents.knowledge import Knowledge
knowledge_system = Knowledge(config=mongodb_knowledge_config)
# Search knowledge base
print("\n📚 Searching knowledge base...")
knowledge_results = knowledge_system.search("customer service", limit=3)
print(f"Found {len(knowledge_results)} knowledge entries")
# Direct MongoDB operations
print("\n🔧 Direct MongoDB operations...")
mongo_client = mongodb_tools.connect_mongodb(
"mongodb://localhost:27017/",
"praisonai_comprehensive"
)
# Get database statistics
stats = mongo_client.get_stats("customers")
print(f"Customer collection stats: {stats}")
print("\n" + "=" * 60)
print("🎉 Comprehensive MongoDB Example Completed Successfully!")
print("=" * 60)
print("\n📈 System Overview:")
print("- Memory System: MongoDB-based persistent memory")
print("- Knowledge Base: MongoDB vector search enabled")
print("- Tools Integration: Full MongoDB CRUD operations")
print("- Multi-Agent Coordination: Business workflow simulation")
print("- Vector Search: Semantic search capabilities")
print("- Data Analytics: Business intelligence insights")
except Exception as e:
print(f"\n❌ Error during execution: {e}")
print("Please check your MongoDB connection and API keys.")
if __name__ == "__main__":
print("Comprehensive MongoDB Integration Example")
print("=" * 50)
print("This example demonstrates complete MongoDB integration with PraisonAI")
print("Features:")
print("- MongoDB Memory Provider")
print("- MongoDB Knowledge Store")
print("- MongoDB Tools Integration")
print("- Vector Search Capabilities")
print("- Multi-Agent Business Simulation")
print("=" * 50)
print("Make sure you have:")
print("1. MongoDB server running (local or Atlas)")
print("2. OpenAI API key set in environment variables")
print("3. MongoDB dependencies installed: pip install 'praisonaiagents[mongodb]'")
print("=" * 50)
main()
@@ -0,0 +1,203 @@
#!/usr/bin/env python3
"""
MongoDB Knowledge Example for PraisonAI Agents
This example demonstrates how to use MongoDB as a knowledge store for PraisonAI agents.
MongoDB can store and retrieve knowledge documents with vector search capabilities.
Prerequisites:
- Install MongoDB dependencies: pip install 'praisonaiagents[mongodb]'
- MongoDB server running (local or Atlas)
- OpenAI API key for embeddings
Features demonstrated:
- MongoDB as knowledge vector store
- Document processing and storage
- Vector search for knowledge retrieval
- Knowledge-based agent interactions
- File processing with MongoDB storage
"""
import os
from praisonaiagents import Agent, Task, PraisonAIAgents
# Ensure OpenAI API key is set
if not os.environ.get("OPENAI_API_KEY"):
raise ValueError("Please set the OPENAI_API_KEY environment variable")
def main():
# MongoDB knowledge configuration
mongodb_knowledge_config = {
"vector_store": {
"provider": "mongodb",
"config": {
"connection_string": "mongodb://localhost:27017/", # Replace with your MongoDB connection string
"database": "praisonai_knowledge",
"collection": "knowledge_base",
"use_vector_search": True # Enable Atlas Vector Search
}
},
"embedder": {
"provider": "openai",
"config": {
"model": "text-embedding-3-small",
"api_key": os.getenv("OPENAI_API_KEY")
}
}
}
# Create a knowledge agent with MongoDB knowledge store
knowledge_agent = Agent(
name="MongoDB Knowledge Agent",
role="Knowledge Specialist",
goal="Provide accurate information from MongoDB knowledge base",
backstory="""You are an expert knowledge specialist who can access and
retrieve information from a comprehensive MongoDB knowledge base. You excel
at finding relevant information, synthesizing knowledge from multiple sources,
and providing accurate, context-aware responses.""",
knowledge_config=mongodb_knowledge_config,
knowledge=[os.path.join(os.path.dirname(__file__), "llms.md")],
memory=True,
verbose=True,
llm="gpt-4o-mini"
)
# Create a research assistant agent
research_agent = Agent(
name="Research Assistant",
role="Research Assistant",
goal="Gather information and store it in the knowledge base",
backstory="""You are a research assistant who specializes in gathering
information from various sources and organizing it for storage in the
knowledge base. You ensure information is accurate, well-structured,
and properly categorized.""",
memory=True,
verbose=True,
llm="gpt-4o-mini"
)
# Create tasks for knowledge management
knowledge_tasks = [
Task(
description="""Research and store information about MongoDB Atlas Vector Search:
1. Gather comprehensive information about MongoDB Atlas Vector Search
2. Include technical specifications, use cases, and best practices
3. Store the information in the MongoDB knowledge base
4. Organize information by categories (features, performance, integration)
""",
expected_output="MongoDB Atlas Vector Search information stored in knowledge base",
agent=research_agent
),
Task(
description="""Research and store information about AI agent frameworks:
1. Research popular AI agent frameworks (LangChain, AutoGen, etc.)
2. Compare their features, capabilities, and use cases
3. Store comparative analysis in the knowledge base
4. Include code examples and best practices
""",
expected_output="AI agent framework comparison stored in knowledge base",
agent=research_agent
),
Task(
description="""Query the knowledge base for MongoDB information:
1. Search for information about MongoDB Atlas Vector Search
2. Extract key features and capabilities
3. Provide a comprehensive summary
4. Include technical recommendations
""",
expected_output="Comprehensive MongoDB Atlas Vector Search summary from knowledge base",
agent=knowledge_agent
),
Task(
description="""Query the knowledge base for AI agent framework information:
1. Search for information about AI agent frameworks
2. Compare different frameworks based on stored knowledge
3. Provide recommendations for different use cases
4. Include best practices and examples
""",
expected_output="AI agent framework comparison and recommendations from knowledge base",
agent=knowledge_agent
)
]
# Initialize the multi-agent system with MongoDB knowledge
print("🚀 Starting MongoDB Knowledge Management System...")
print("=" * 60)
knowledge_system = PraisonAIAgents(
agents=[research_agent, knowledge_agent],
tasks=knowledge_tasks,
memory=True,
verbose=True
)
# Execute the knowledge management pipeline
try:
results = knowledge_system.start()
print("\n" + "=" * 60)
print("📊 Knowledge Management System Results:")
print("=" * 60)
# Handle results structure properly
if isinstance(results, (list, tuple)):
for i, result in enumerate(results, 1):
print(f"\n{i}. Task Result:")
if hasattr(result, 'raw') and hasattr(result, 'agent'):
print(f" Output: {result.raw[:200]}...")
print(f" Agent: {result.agent}")
else:
print(f" Output: {str(result)[:200]}...")
else:
print(f"\n1. Task Result:")
print(f" Output: {str(results)[:200]}...")
print("\n" + "=" * 60)
print("💾 MongoDB Knowledge Integration Complete!")
print("=" * 60)
# Demonstrate direct knowledge access
print("\n🔍 Demonstrating Direct Knowledge Access:")
print("-" * 40)
# Access knowledge system directly
from praisonaiagents.knowledge import Knowledge
knowledge_system = Knowledge(config=mongodb_knowledge_config)
# Store a sample document
sample_doc = """
MongoDB is a document-oriented database that provides high performance,
high availability, and easy scalability. It works on the concept of
collections and documents, using a flexible, JSON-like document model.
"""
print("📄 Storing sample document...")
knowledge_system.store(sample_doc, metadata={"category": "database", "type": "overview"})
# Search for stored knowledge
print("\n🔍 Searching knowledge base...")
results = knowledge_system.search("MongoDB document database", limit=3)
print(f"Found {len(results)} knowledge entries:")
for result in results:
print(f" - {result.get('memory', '')[:100]}...")
print(f" Score: {result.get('score', 'N/A')}")
print("\n✅ MongoDB Knowledge Example Completed Successfully!")
except Exception as e:
print(f"\n❌ Error during execution: {e}")
print("Please check your MongoDB connection and API keys.")
if __name__ == "__main__":
print("MongoDB Knowledge Example for PraisonAI Agents")
print("=" * 50)
print("This example demonstrates MongoDB integration for knowledge management")
print("Make sure you have:")
print("1. MongoDB server running (local or Atlas)")
print("2. OpenAI API key set in environment variables")
print("3. MongoDB dependencies installed: pip install 'praisonaiagents[mongodb]'")
print("=" * 50)
main()
@@ -0,0 +1,203 @@
#!/usr/bin/env python3
"""
MongoDB Knowledge Example for PraisonAI Agents
This example demonstrates how to use MongoDB as a knowledge store for PraisonAI agents.
MongoDB can store and retrieve knowledge documents with vector search capabilities.
Prerequisites:
- Install MongoDB dependencies: pip install 'praisonaiagents[mongodb]'
- MongoDB server running (local or Atlas)
- OpenAI API key for embeddings
Features demonstrated:
- MongoDB as knowledge vector store
- Document processing and storage
- Vector search for knowledge retrieval
- Knowledge-based agent interactions
- File processing with MongoDB storage
"""
import os
from praisonaiagents import Agent, Task, PraisonAIAgents
# Ensure OpenAI API key is set
if not os.environ.get("OPENAI_API_KEY"):
raise ValueError("Please set the OPENAI_API_KEY environment variable")
def main():
# MongoDB knowledge configuration
mongodb_knowledge_config = {
"vector_store": {
"provider": "mongodb",
"config": {
"connection_string": "mongodb://localhost:27017/", # Replace with your MongoDB connection string
"database": "praisonai_knowledge",
"collection": "knowledge_base",
"use_vector_search": True # Enable Atlas Vector Search
}
},
"embedder": {
"provider": "openai",
"config": {
"model": "text-embedding-3-small",
"api_key": os.getenv("OPENAI_API_KEY")
}
}
}
# Create a knowledge agent with MongoDB knowledge store
knowledge_agent = Agent(
name="MongoDB Knowledge Agent",
role="Knowledge Specialist",
goal="Provide accurate information from MongoDB knowledge base",
backstory="""You are an expert knowledge specialist who can access and
retrieve information from a comprehensive MongoDB knowledge base. You excel
at finding relevant information, synthesizing knowledge from multiple sources,
and providing accurate, context-aware responses.""",
knowledge_config=mongodb_knowledge_config,
knowledge=[os.path.join(os.path.dirname(__file__), "llms.md")],
memory=True,
verbose=True,
llm="gpt-4o-mini"
)
# Create a research assistant agent
research_agent = Agent(
name="Research Assistant",
role="Research Assistant",
goal="Gather information and store it in the knowledge base",
backstory="""You are a research assistant who specializes in gathering
information from various sources and organizing it for storage in the
knowledge base. You ensure information is accurate, well-structured,
and properly categorized.""",
memory=True,
verbose=True,
llm="gpt-4o-mini"
)
# Create tasks for knowledge management
knowledge_tasks = [
Task(
description="""Research and store information about MongoDB Atlas Vector Search:
1. Gather comprehensive information about MongoDB Atlas Vector Search
2. Include technical specifications, use cases, and best practices
3. Store the information in the MongoDB knowledge base
4. Organize information by categories (features, performance, integration)
""",
expected_output="MongoDB Atlas Vector Search information stored in knowledge base",
agent=research_agent
),
Task(
description="""Research and store information about AI agent frameworks:
1. Research popular AI agent frameworks (LangChain, AutoGen, etc.)
2. Compare their features, capabilities, and use cases
3. Store comparative analysis in the knowledge base
4. Include code examples and best practices
""",
expected_output="AI agent framework comparison stored in knowledge base",
agent=research_agent
),
Task(
description="""Query the knowledge base for MongoDB information:
1. Search for information about MongoDB Atlas Vector Search
2. Extract key features and capabilities
3. Provide a comprehensive summary
4. Include technical recommendations
""",
expected_output="Comprehensive MongoDB Atlas Vector Search summary from knowledge base",
agent=knowledge_agent
),
Task(
description="""Query the knowledge base for AI agent framework information:
1. Search for information about AI agent frameworks
2. Compare different frameworks based on stored knowledge
3. Provide recommendations for different use cases
4. Include best practices and examples
""",
expected_output="AI agent framework comparison and recommendations from knowledge base",
agent=knowledge_agent
)
]
# Initialize the multi-agent system with MongoDB knowledge
print("🚀 Starting MongoDB Knowledge Management System...")
print("=" * 60)
knowledge_system = PraisonAIAgents(
agents=[research_agent, knowledge_agent],
tasks=knowledge_tasks,
memory=True,
verbose=True
)
# Execute the knowledge management pipeline
try:
results = knowledge_system.start()
print("\n" + "=" * 60)
print("📊 Knowledge Management System Results:")
print("=" * 60)
# Handle results structure properly
if isinstance(results, (list, tuple)):
for i, result in enumerate(results, 1):
print(f"\n{i}. Task Result:")
if hasattr(result, 'raw') and hasattr(result, 'agent'):
print(f" Output: {result.raw[:200]}...")
print(f" Agent: {result.agent}")
else:
print(f" Output: {str(result)[:200]}...")
else:
print(f"\n1. Task Result:")
print(f" Output: {str(results)[:200]}...")
print("\n" + "=" * 60)
print("💾 MongoDB Knowledge Integration Complete!")
print("=" * 60)
# Demonstrate direct knowledge access
print("\n🔍 Demonstrating Direct Knowledge Access:")
print("-" * 40)
# Access knowledge system directly
from praisonaiagents.knowledge import Knowledge
knowledge_system = Knowledge(config=mongodb_knowledge_config)
# Store a sample document
sample_doc = """
MongoDB is a document-oriented database that provides high performance,
high availability, and easy scalability. It works on the concept of
collections and documents, using a flexible, JSON-like document model.
"""
print("📄 Storing sample document...")
knowledge_system.store(sample_doc, metadata={"category": "database", "type": "overview"})
# Search for stored knowledge
print("\n🔍 Searching knowledge base...")
results = knowledge_system.search("MongoDB document database", limit=3)
print(f"Found {len(results)} knowledge entries:")
for result in results:
print(f" - {result.get('memory', '')[:100]}...")
print(f" Score: {result.get('score', 'N/A')}")
print("\n✅ MongoDB Knowledge Example Completed Successfully!")
except Exception as e:
print(f"\n❌ Error during execution: {e}")
print("Please check your MongoDB connection and API keys.")
if __name__ == "__main__":
print("MongoDB Knowledge Example for PraisonAI Agents")
print("=" * 50)
print("This example demonstrates MongoDB integration for knowledge management")
print("Make sure you have:")
print("1. MongoDB server running (local or Atlas)")
print("2. OpenAI API key set in environment variables")
print("3. MongoDB dependencies installed: pip install 'praisonaiagents[mongodb]'")
print("=" * 50)
main()
@@ -0,0 +1,84 @@
from praisonaiagents import Agent, Task, PraisonAIAgents
from praisonaiagents.memory import Memory
from praisonaiagents.agent import ContextAgent
import pymongo
context_agent = ContextAgent(llm="gpt-4o-mini", auto_analyze=False)
context_output = context_agent.start("https://github.com/MervinPraison/PraisonAI/ Need to add Authentication")
mongodb_memory_config = {
"provider": "mongodb",
"config": {
"connection_string": "mongodb://localhost:27017/",
"database": "praisonai_memory",
"use_vector_search": True,
"max_pool_size": 50,
"min_pool_size": 10,
"server_selection_timeout": 5000
}
}
implementation_agent = Agent(
name="Implementation Agent",
role="Authentication Implementation Specialist",
goal="Implement authentication features based on project requirements",
backstory="Expert software implementer specializing in authentication systems, security features, and seamless integration with existing codebases",
memory=True,
llm="gpt-4o-mini",
)
implementation_task = Task(
description="Implement authentication features based on the project requirements from context analysis",
expected_output="Authentication implementation with code, configuration, and integration details",
agent=implementation_agent,
context=context_output,
)
implementation_system = PraisonAIAgents(
agents=[implementation_agent],
tasks=[implementation_task],
memory=True,
memory_config=mongodb_memory_config
)
results = implementation_system.start()
print(f"Results: {results}")
print("\n=== MEMORY VALIDATION ===")
try:
# Check MongoDB connection and collections
client = pymongo.MongoClient("mongodb://localhost:27017/")
db = client["praisonai_memory"]
collections = db.list_collection_names()
print(f"Available collections: {collections}")
for collection_name in collections:
count = db[collection_name].count_documents({})
print(f"{collection_name}: {count} documents")
if count > 0:
sample = db[collection_name].find_one()
print(f"Sample document: {sample}")
except Exception as e:
print(f"MongoDB direct check error: {e}")
try:
# Check memory system
memory_system = Memory(mongodb_memory_config)
# Try different search terms
search_terms = ["authentication", "security", "implementation", "requirements", "PraisonAI"]
for term in search_terms:
memories = memory_system.search_long_term(term, limit=3)
print(f"Search '{term}': {len(memories)} memories found")
if memories:
print(f" Sample: {str(memories[0])[:100]}...")
# Try to get all memories
all_memories = memory_system.get_all_memories()
print(f"Total memories: {len(all_memories)}")
except Exception as e:
print(f"Memory system error: {e}")
@@ -0,0 +1,178 @@
#!/usr/bin/env python3
"""
MongoDB Memory Example for PraisonAI Agents
This example demonstrates how to use MongoDB as a memory provider for PraisonAI agents.
MongoDB can be used as both a key-value store and a vector database for memory operations.
Prerequisites:
- Install MongoDB dependencies: pip install 'praisonaiagents[mongodb]'
- MongoDB server running (local or Atlas)
- OpenAI API key for embeddings
Features demonstrated:
- MongoDB as memory provider
- Vector search with Atlas Vector Search
- Quality scoring and filtering
- User-specific memory storage
- Persistent memory across sessions
"""
import os
from praisonaiagents import Agent, Task, PraisonAIAgents
# Ensure OpenAI API key is set
if not os.environ.get("OPENAI_API_KEY"):
raise ValueError("Please set the OPENAI_API_KEY environment variable")
def main():
# MongoDB memory configuration
mongodb_memory_config = {
"provider": "mongodb",
"config": {
"connection_string": "mongodb://localhost:27017/", # Replace with your MongoDB connection string
"database": "praisonai_memory",
"use_vector_search": True, # Enable Atlas Vector Search (requires MongoDB Atlas)
"max_pool_size": 50,
"min_pool_size": 10,
"server_selection_timeout": 5000
}
}
# Create a research agent with MongoDB memory
research_agent = Agent(
name="MongoDB Research Agent",
role="Senior Research Specialist",
goal="Research topics and maintain persistent memory using MongoDB",
backstory="""You are an expert researcher who specializes in collecting,
analyzing, and storing information using advanced MongoDB-based memory systems.
You excel at maintaining context across multiple research sessions and can
retrieve relevant information from your persistent memory store.""",
memory=True,
verbose=True,
llm="gpt-4o-mini"
)
# Create research tasks
research_tasks = [
Task(
description="""Research the latest developments in AI/ML for 2024:
1. Identify major AI breakthroughs and trends
2. Analyze the impact on various industries
3. Store key findings in MongoDB memory with quality scores
4. Identify emerging technologies and their potential applications
""",
expected_output="Comprehensive AI/ML research report with key findings stored in MongoDB",
agent=research_agent
),
Task(
description="""Research MongoDB Atlas Vector Search capabilities:
1. Explore vector search features and use cases
2. Compare with other vector databases
3. Analyze performance and scalability
4. Store technical insights in MongoDB memory
""",
expected_output="Technical analysis of MongoDB Atlas Vector Search with stored insights",
agent=research_agent
),
Task(
description="""Research AI agent frameworks and tools:
1. Compare popular AI agent frameworks
2. Analyze their strengths and weaknesses
3. Identify best practices for agent development
4. Store comparative analysis in MongoDB memory
""",
expected_output="Comparative analysis of AI agent frameworks with stored insights",
agent=research_agent
)
]
# Initialize the multi-agent system with MongoDB memory
print("🚀 Starting MongoDB Memory Research System...")
print("=" * 60)
research_system = PraisonAIAgents(
agents=[research_agent],
tasks=research_tasks,
memory=True,
memory_config=mongodb_memory_config,
verbose=True
)
# Execute the research pipeline
try:
results = research_system.start()
print("\n" + "=" * 60)
print("📊 Research System Results:")
print("=" * 60)
# Handle results structure properly
if isinstance(results, (list, tuple)):
for i, result in enumerate(results, 1):
print(f"\n{i}. Task Result:")
if hasattr(result, 'agent'):
print(f" Output: {str(result)[:200]}...")
print(f" Agent: {result.agent}")
else:
print(f" Output: {str(result)[:200]}...")
else:
print(f"\n1. Task Result:")
print(f" Output: {str(results)[:200]}...")
print("\n" + "=" * 60)
print("💾 MongoDB Memory Integration Complete!")
print("=" * 60)
# Demonstrate memory retrieval
print("\n🔍 Demonstrating Memory Retrieval:")
print("-" * 40)
# Note: Direct memory access not available through research_system.memory
# Instead, use Memory class directly for verification
try:
from praisonaiagents.memory import Memory
# Create Memory instance with MongoDB config for verification
memory_system = Memory(mongodb_memory_config)
# Search for AI-related memories
ai_memories = memory_system.search_long_term("AI machine learning", limit=3)
print(f"Found {len(ai_memories)} AI-related memories:")
for memory in ai_memories:
if hasattr(memory, 'get'):
print(f" - {memory.get('text', '')[:100]}...")
else:
print(f" - {str(memory)[:100]}...")
# Search for MongoDB-related memories
mongo_memories = memory_system.search_long_term("MongoDB vector search", limit=3)
print(f"\nFound {len(mongo_memories)} MongoDB-related memories:")
for memory in mongo_memories:
if hasattr(memory, 'get'):
print(f" - {memory.get('text', '')[:100]}...")
else:
print(f" - {str(memory)[:100]}...")
except ImportError:
print("Memory features not available - install with: pip install 'praisonaiagents[memory]'")
except Exception as mem_error:
print(f"Memory verification error: {mem_error}")
print("\n✅ MongoDB Memory Example Completed Successfully!")
except Exception as e:
print(f"\n❌ Error during execution: {e}")
print("Please check your MongoDB connection and API keys.")
if __name__ == "__main__":
print("MongoDB Memory Example for PraisonAI Agents")
print("=" * 50)
print("This example demonstrates MongoDB integration for persistent memory")
print("Make sure you have:")
print("1. MongoDB server running (local or Atlas)")
print("2. OpenAI API key set in environment variables")
print("3. MongoDB dependencies installed: pip install 'praisonaiagents[mongodb]'")
print("=" * 50)
main()
@@ -0,0 +1,205 @@
#!/usr/bin/env python3
"""
MongoDB Tools Example for PraisonAI Agents
This example demonstrates how to use MongoDB tools with PraisonAI agents.
Agents can perform MongoDB operations like inserting, querying, and vector search.
Prerequisites:
- Install MongoDB dependencies: pip install 'praisonaiagents[mongodb]'
- MongoDB server running (local or Atlas)
- OpenAI API key for embeddings
Features demonstrated:
- MongoDB tools integration
- Document CRUD operations
- Vector search with embeddings
- Data analysis with MongoDB
- Agent-driven database operations
"""
import os
from praisonaiagents import Agent, Task, PraisonAIAgents
from praisonaiagents.tools import mongodb_tools
# Ensure OpenAI API key is set
if not os.environ.get("OPENAI_API_KEY"):
raise ValueError("Please set the OPENAI_API_KEY environment variable")
def main():
# Create a MongoDB Database Agent with MongoDB tools
db_agent = Agent(
name="MongoDB Database Agent",
role="Database Operations Specialist",
goal="Manage MongoDB operations and data analysis",
backstory="""You are an expert database administrator who specializes in
MongoDB operations. You can perform complex database queries, manage
collections, and analyze data patterns. You excel at using MongoDB tools
for data management and retrieval.""",
tools=[mongodb_tools],
verbose=True,
llm="gpt-4o-mini"
)
# Create a Data Analysis Agent
analysis_agent = Agent(
name="Data Analysis Agent",
role="Data Analyst",
goal="Analyze data stored in MongoDB and provide insights",
backstory="""You are a data analyst who specializes in extracting insights
from MongoDB databases. You can query data, perform aggregations, and
identify patterns and trends in the stored information.""",
tools=[mongodb_tools],
verbose=True,
llm="gpt-4o-mini"
)
# Create tasks for MongoDB operations
mongodb_tasks = [
Task(
description="""Set up MongoDB collections and insert sample data:
1. Create a collection called 'products' for e-commerce data
2. Insert sample product documents with fields: name, price, category, description
3. Create a collection called 'users' for user data
4. Insert sample user documents with fields: name, email, age, preferences
5. Create appropriate indexes for better query performance
Use the MongoDB tools to perform these operations.
""",
expected_output="MongoDB collections created with sample data and indexes",
agent=db_agent
),
Task(
description="""Perform vector search operations:
1. Create a vector index for product descriptions
2. Store products with text embeddings for vector search
3. Demonstrate vector similarity search for product recommendations
4. Show how to find similar products based on description embeddings
Use the MongoDB vector search tools for these operations.
""",
expected_output="Vector search implementation with product recommendations",
agent=db_agent
),
Task(
description="""Query and analyze the stored data:
1. Find all products in a specific category
2. Calculate average price by category
3. Find users with specific preferences
4. Perform text search on product descriptions
5. Generate insights about the data patterns
Use MongoDB query tools to perform these analyses.
""",
expected_output="Data analysis report with insights from MongoDB queries",
agent=analysis_agent
),
Task(
description="""Demonstrate advanced MongoDB operations:
1. Update product prices based on specific criteria
2. Perform aggregation operations for statistical analysis
3. Create reports on user demographics and preferences
4. Implement data validation and error handling
5. Show MongoDB best practices for data management
Use MongoDB tools for these advanced operations.
""",
expected_output="Advanced MongoDB operations report with best practices",
agent=db_agent
)
]
# Initialize the multi-agent system with MongoDB tools
print("🚀 Starting MongoDB Tools Demo System...")
print("=" * 60)
mongodb_system = PraisonAIAgents(
agents=[db_agent, analysis_agent],
tasks=mongodb_tasks,
verbose=True
)
# Execute the MongoDB operations pipeline
try:
results = mongodb_system.start(return_dict=True)
print("\n" + "=" * 60)
print("📊 MongoDB Tools System Results:")
print("=" * 60)
print(results)
# Handle results structure properly
if isinstance(results, dict) and 'task_results' in results:
task_results = results['task_results']
for i, result in task_results.items():
print(f"\n{i}. Task Result:")
if hasattr(result, 'raw') and hasattr(result, 'agent'):
print(f" Output: {result.raw[:200]}...")
print(f" Agent: {result.agent}")
else:
print(f" Result: {str(result)[:200]}...")
else:
print(f"\n1. Task Result:")
print(f" Output: {str(results)[:200]}...")
print("\n" + "=" * 60)
print("💾 MongoDB Tools Integration Complete!")
print("=" * 60)
# Demonstrate direct MongoDB tools usage
print("\n🔍 Demonstrating Direct MongoDB Tools Usage:")
print("-" * 40)
# Connect to MongoDB
mongo_client = mongodb_tools.connect_mongodb(
"mongodb://localhost:27017/", # Replace with your connection string
"praisonai_tools_demo"
)
# Insert a sample document
print("📄 Inserting sample document...")
result = mongo_client.insert_document(
"demo_collection",
{
"title": "MongoDB Tools Demo",
"description": "Demonstration of MongoDB tools with PraisonAI",
"category": "database",
"tags": ["mongodb", "praisonai", "tools", "demo"]
},
metadata={"created_by": "demo", "version": "1.0"}
)
print(f" Result: {result}")
# Query documents
print("\n🔍 Querying documents...")
documents = mongo_client.find_documents(
"demo_collection",
{"category": "database"},
limit=5
)
print(f" Found {len(documents)} documents")
for doc in documents:
print(f" - {doc.get('title', 'N/A')}: {doc.get('description', '')[:50]}...")
# Get collection statistics
print("\n📈 Getting collection statistics...")
stats = mongo_client.get_stats("demo_collection")
print(f" Collection stats: {stats}")
print("\n✅ MongoDB Tools Example Completed Successfully!")
except Exception as e:
print(f"\n❌ Error during execution: {e}")
print("Please check your MongoDB connection and API keys.")
if __name__ == "__main__":
print("MongoDB Tools Example for PraisonAI Agents")
print("=" * 50)
print("This example demonstrates MongoDB tools integration")
print("Make sure you have:")
print("1. MongoDB server running (local or Atlas)")
print("2. OpenAI API key set in environment variables")
print("3. MongoDB dependencies installed: pip install 'praisonaiagents[mongodb]'")
print("=" * 50)
main()
@@ -0,0 +1,321 @@
"""
PraisonAI Performance Monitoring Demo
This example demonstrates how to use PraisonAI's comprehensive performance monitoring
capabilities to answer the key questions from Issue #1035:
1. How to evaluate how each function is performing
2. Function flow analysis
3. How fast each function is executing
4. API call performance monitoring (fast vs slow calls)
5. How to see all API calls
Features Demonstrated:
- Function performance evaluation with detailed statistics
- Function execution flow analysis and visualization
- API call tracking and performance comparison
- Real-time performance monitoring
- Bottleneck identification and optimization recommendations
- Comprehensive performance reporting
"""
import time
import random
from praisonaiagents import Agent, Task, PraisonAIAgents
from praisonaiagents.tools import duckduckgo
from praisonaiagents.telemetry import (
monitor_function, track_api_call, performance_monitor,
analyze_function_flow,
visualize_execution_flow, generate_comprehensive_report
)
print("=" * 80)
print("PRAISONAI PERFORMANCE MONITORING DEMONSTRATION")
print("=" * 80)
print("This demo answers the key questions from Issue #1035:")
print("1. How to evaluate how each function is performing")
print("2. Function flow analysis")
print("3. How fast each function is executing")
print("4. API call performance monitoring (fast vs slow calls)")
print("5. How to see all API calls")
print("=" * 80)
print()
# Example 1: Function Performance Evaluation
print("📊 EXAMPLE 1: Function Performance Evaluation")
print("-" * 50)
# Demonstrate function monitoring with decorators
@monitor_function("data_processing")
def process_data(data_size: int):
"""Simulate data processing with variable performance."""
processing_time = 0.1 + (data_size * 0.01) # Simulate work
time.sleep(processing_time)
return f"Processed {data_size} items"
@monitor_function("model_inference")
def run_model_inference(complexity: str):
"""Simulate model inference with different complexities."""
if complexity == "simple":
time.sleep(0.05)
elif complexity == "medium":
time.sleep(0.2)
else: # complex
time.sleep(0.5)
return f"Inference completed for {complexity} model"
@monitor_function("database_query")
def query_database(query_type: str):
"""Simulate database queries with different performance characteristics."""
if query_type == "fast":
time.sleep(0.02)
elif query_type == "medium":
time.sleep(0.1)
else: # slow
time.sleep(0.3)
return f"Query {query_type} completed"
# Run functions with different performance characteristics
print("Running functions to generate performance data...")
for _ in range(5):
process_data(random.randint(10, 100))
run_model_inference(random.choice(["simple", "medium", "complex"]))
query_database(random.choice(["fast", "medium", "slow"]))
# Show function performance statistics
print("\n📈 Function Performance Statistics:")
function_stats = performance_monitor.get_function_performance()
for func_name, stats in function_stats.items():
print(f"\n🔧 {func_name}:")
print(f" Calls: {stats['call_count']}")
print(f" Average Time: {stats.get('average_time', 0):.3f}s")
print(f" Min/Max Time: {stats['min_time']:.3f}s / {stats['max_time']:.3f}s")
print(f" Success Rate: {stats.get('success_rate', 0)*100:.1f}%")
# Example 2: Function Flow Analysis
print("\n\n🔄 EXAMPLE 2: Function Flow Analysis")
print("-" * 50)
@monitor_function("workflow_step_1")
def workflow_step_1():
time.sleep(0.1)
workflow_step_2()
workflow_step_3()
@monitor_function("workflow_step_2")
def workflow_step_2():
time.sleep(0.05)
workflow_step_4()
@monitor_function("workflow_step_3")
def workflow_step_3():
time.sleep(0.08)
@monitor_function("workflow_step_4")
def workflow_step_4():
time.sleep(0.03)
print("Running workflow to demonstrate flow analysis...")
workflow_step_1()
# Analyze execution flow
print("\n📊 Function Execution Flow Analysis:")
flow_analysis = analyze_function_flow()
if "bottlenecks" in flow_analysis:
bottlenecks = flow_analysis["bottlenecks"]
if bottlenecks:
print("🚨 Identified Bottlenecks:")
for bottleneck in bottlenecks:
print(f" • {bottleneck['function']}: {bottleneck['average_duration']:.3f}s average")
else:
print("✅ No significant bottlenecks identified")
# Visualize execution flow
print("\n🎨 Function Execution Flow Visualization:")
flow_viz = visualize_execution_flow(format="text")
print(flow_viz)
# Example 3: API Call Performance Monitoring
print("\n\n🌐 EXAMPLE 3: API Call Performance Monitoring")
print("-" * 50)
# Simulate different API calls with various performance characteristics
def simulate_api_calls():
"""Simulate API calls to demonstrate tracking."""
# Fast API calls
with track_api_call("openai_api", "/v1/models"):
time.sleep(0.05) # Fast call
with track_api_call("openai_api", "/v1/chat/completions"):
time.sleep(0.3) # Medium call
# Slow API call
with track_api_call("anthropic_api", "/v1/messages"):
time.sleep(0.6) # Slow call
# Another fast call
with track_api_call("gemini_api", "/v1/generate"):
time.sleep(0.08) # Fast call
# Simulate error case
try:
with track_api_call("unreliable_api", "/v1/endpoint"):
time.sleep(0.2)
if random.random() < 0.3: # 30% chance of error
raise Exception("API Error")
except Exception:
pass # Error is already tracked
print("Simulating API calls with different performance characteristics...")
for _ in range(8):
simulate_api_calls()
# Show API performance statistics
print("\n📊 API Call Performance Statistics:")
api_stats = performance_monitor.get_api_call_performance()
for api_name, stats in api_stats.items():
print(f"\n🔗 {api_name}:")
print(f" Total Calls: {stats['call_count']}")
print(f" Average Time: {stats.get('average_time', 0):.3f}s")
print(f" Min/Max Time: {stats['min_time']:.3f}s / {stats['max_time']:.3f}s")
print(f" Success Rate: {stats.get('success_rate', 0)*100:.1f}%")
print(f" Successful: {stats['success_count']} | Failed: {stats['error_count']}")
# Show fastest vs slowest APIs
print("\n🏁 Fastest vs Slowest API Calls:")
slowest_apis = performance_monitor.get_slowest_api_calls(10)
for i, api in enumerate(slowest_apis, 1):
speed_indicator = "🐌" if api['average_time'] > 0.3 else "⚡" if api['average_time'] < 0.1 else "🚀"
print(f"{i}. {speed_indicator} {api['api']}: {api['average_time']:.3f}s avg (Success: {api['success_rate']*100:.1f}%)")
# Example 4: Real-time Performance Monitoring with Agents
print("\n\n🤖 EXAMPLE 4: Real-time Performance Monitoring with AI Agents")
print("-" * 50)
# Create agents with performance monitoring
research_agent = Agent(
name="PerformanceResearcher",
role="Performance Research Specialist",
goal="Research and analyze performance topics quickly",
backstory="You are an expert at researching performance optimization topics.",
tools=[duckduckgo],
verbose=False # Reduce output for cleaner demo
)
# Monitor agent operations
@monitor_function("agent_task_execution")
def execute_monitored_task(agent, task_description):
"""Execute an agent task with performance monitoring."""
task = Task(
description=task_description,
expected_output="Brief research summary with key insights",
agent=agent
)
agents_workflow = PraisonAIAgents(
agents=[agent],
tasks=[task],
verbose=False
)
# Simulate API calls during agent execution
with track_api_call("llm_provider", "/v1/chat/completions"):
time.sleep(0.2) # Simulate LLM call
result = agents_workflow.start()
return result
print("Running AI agent with performance monitoring...")
result = execute_monitored_task(
research_agent,
"Research the latest trends in AI performance optimization"
)
# Example 5: Comprehensive Performance Report
print("\n\n📋 EXAMPLE 5: Comprehensive Performance Report")
print("-" * 50)
# Generate comprehensive performance report
comprehensive_report = generate_comprehensive_report()
print(comprehensive_report)
# Example 6: Using CLI Interface Programmatically
print("\n\n💻 EXAMPLE 6: CLI Interface Usage")
print("-" * 50)
print("You can also use the CLI interface from command line:")
print()
print("# Show performance report")
print("python -m praisonaiagents.telemetry.performance_cli report")
print()
print("# Show function statistics")
print("python -m praisonaiagents.telemetry.performance_cli functions")
print()
print("# Show API call statistics")
print("python -m praisonaiagents.telemetry.performance_cli apis")
print()
print("# Show slowest functions")
print("python -m praisonaiagents.telemetry.performance_cli slowest-functions 5")
print()
print("# Analyze function flow")
print("python -m praisonaiagents.telemetry.performance_cli analyze-flow")
print()
print("# Show execution flow visualization")
print("python -m praisonaiagents.telemetry.performance_cli flow --format text")
print()
print("# Export all data to JSON")
print("python -m praisonaiagents.telemetry.performance_cli export --format json --output performance_data.json")
# Example 7: Integration with Existing Code
print("\n\n🔧 EXAMPLE 7: Easy Integration with Existing Code")
print("-" * 50)
print("To integrate performance monitoring into your existing PraisonAI code:")
print()
print("1. Import the performance monitoring tools:")
print(" from praisonaiagents.telemetry import monitor_function, track_api_call")
print()
print("2. Add decorators to your functions:")
print(" @monitor_function('my_function')")
print(" def my_function():")
print(" return 'result'")
print()
print("3. Track API calls:")
print(" with track_api_call('openai', '/v1/chat/completions'):")
print(" response = openai_client.chat.completions.create(...)")
print()
print("4. Get performance reports:")
print(" from praisonaiagents.telemetry import get_performance_report")
print(" print(get_performance_report())")
# Summary of capabilities
print("\n\n✅ SUMMARY: Questions from Issue #1035 - ANSWERED!")
print("=" * 80)
print("1. ✅ How to evaluate how each function is performing:")
print(" → Use @monitor_function decorator and get_function_stats()")
print(" → View detailed statistics: calls, timing, success rates, errors")
print()
print("2. ✅ Function flow analysis:")
print(" → Use analyze_function_flow() for bottleneck identification")
print(" → Use visualize_execution_flow() for flow visualization")
print(" → Get call chains and execution patterns")
print()
print("3. ✅ How fast each function is executing:")
print(" → Automatic timing collection with min/max/average/recent stats")
print(" → Use get_slowest_functions() to find performance bottlenecks")
print(" → Real-time active call monitoring")
print()
print("4. ✅ API call performance monitoring (fast vs slow calls):")
print(" → Use track_api_call() context manager for API monitoring")
print(" → Compare API performance with get_slowest_apis()")
print(" → Success/error rate tracking")
print()
print("5. ✅ How to see all API calls:")
print(" → Use get_api_stats() to see all tracked APIs")
print(" → View recent call history and detailed metrics")
print(" → CLI interface for easy access: performance_cli apis")
print()
print("🎉 All requested features are now available with backward compatibility!")
print("🔧 Easy integration with existing code using decorators and context managers")
print("📊 Comprehensive reporting and analysis capabilities")
print("💻 CLI interface for command-line access")
print("=" * 80)
@@ -5,11 +5,27 @@ Praison AI Agents - A package for hierarchical AI agent task execution
# Configure logging before any other imports
import os
import logging
import warnings
import re
from rich.logging import RichHandler
# Set environment variables to suppress warnings at the source
os.environ["LITELLM_TELEMETRY"] = "False"
os.environ["LITELLM_DROP_PARAMS"] = "True"
# Disable httpx warnings
os.environ["HTTPX_DISABLE_WARNINGS"] = "True"
# Get log level from environment variable
LOGLEVEL = os.environ.get('LOGLEVEL', 'INFO').upper()
# Determine if warnings should be suppressed (not in DEBUG mode and not in tests)
def _should_suppress_warnings():
import sys
return (LOGLEVEL != 'DEBUG' and
not hasattr(sys, '_called_from_test') and
'pytest' not in sys.modules and
os.environ.get('PYTEST_CURRENT_TEST') is None)
# Configure root logger
logging.basicConfig(
level=getattr(logging, LOGLEVEL, logging.INFO),
@@ -18,16 +34,38 @@ logging.basicConfig(
handlers=[RichHandler(rich_tracebacks=True)]
)
# Suppress specific noisy loggers
logging.getLogger("litellm").setLevel(logging.WARNING)
logging.getLogger("litellm.utils").setLevel(logging.WARNING)
logging.getLogger("markdown_it").setLevel(logging.WARNING)
logging.getLogger("rich.markdown").setLevel(logging.WARNING)
logging.getLogger("httpx").setLevel(logging.WARNING)
logging.getLogger("httpcore").setLevel(logging.WARNING)
# Suppress specific noisy loggers - more aggressive suppression (only when not in DEBUG mode)
if _should_suppress_warnings():
logging.getLogger("litellm").setLevel(logging.CRITICAL)
logging.getLogger("litellm_logging").setLevel(logging.CRITICAL)
logging.getLogger("httpx").setLevel(logging.CRITICAL)
logging.getLogger("httpcore").setLevel(logging.CRITICAL)
logging.getLogger("pydantic").setLevel(logging.WARNING)
logging.getLogger("markdown_it").setLevel(logging.WARNING)
logging.getLogger("rich.markdown").setLevel(logging.WARNING)
# Note: litellm child loggers (litellm.utils, litellm.proxy, etc.) automatically inherit
# the CRITICAL level from the parent litellm logger due to Python's hierarchical logging
# Comprehensive warning suppression for litellm and dependencies (issue #1033)
# These warnings clutter output and are not actionable for users
# Set warning filter to suppress all warnings from problematic modules at import time
if _should_suppress_warnings():
# Module-specific warning suppression - applied before imports (only when not in DEBUG mode)
for module in ['litellm', 'httpx', 'httpcore', 'pydantic']:
warnings.filterwarnings("ignore", category=DeprecationWarning, module=module)
warnings.filterwarnings("ignore", category=UserWarning, module=module)
# Specific filters for known problematic warnings
warnings.filterwarnings("ignore", message="There is no current event loop")
warnings.filterwarnings("ignore", message=".*Use 'content=<...>' to upload raw bytes/text content.*")
warnings.filterwarnings("ignore", message=".*The `dict` method is deprecated; use `model_dump` instead.*")
warnings.filterwarnings("ignore", message=".*model_dump.*deprecated.*")
from .agent.agent import Agent
from .agent.image_agent import ImageAgent
from .agent.context_agent import ContextAgent, create_context_agent
from .agents.agents import PraisonAIAgents
from .task.task import Task
from .tools.tools import Tools
@@ -93,6 +131,30 @@ except ImportError:
# Add Agents as an alias for PraisonAIAgents
Agents = PraisonAIAgents
# Additional warning suppression after all imports (runtime suppression)
if _should_suppress_warnings():
# Try to import and configure litellm to suppress its warnings
try:
import litellm
# Disable all litellm logging and telemetry
litellm.telemetry = False
litellm.drop_params = True
# Set litellm to suppress warnings
litellm.suppress_debug_info = True
if hasattr(litellm, '_logging_obj'):
litellm._logging_obj.setLevel(logging.CRITICAL)
except (ImportError, AttributeError):
pass
# Suppress pydantic warnings that might occur at runtime (safer approach)
try:
warnings.filterwarnings("ignore", category=UserWarning, module="pydantic",
message=".*model_dump.*deprecated.*")
warnings.filterwarnings("ignore", category=UserWarning, module="pydantic",
message=".*dict.*method.*deprecated.*")
except Exception:
pass
# Apply telemetry auto-instrumentation after all imports
if _telemetry_available:
try:
@@ -108,6 +170,8 @@ if _telemetry_available:
__all__ = [
'Agent',
'ImageAgent',
'ContextAgent',
'create_context_agent',
'PraisonAIAgents',
'Agents',
'Tools',
@@ -1,7 +1,8 @@
"""Agent module for AI agents"""
from .agent import Agent
from .image_agent import ImageAgent
from .context_agent import ContextAgent, create_context_agent
from .handoff import Handoff, handoff, handoff_filters, RECOMMENDED_PROMPT_PREFIX, prompt_with_handoff_instructions
from .router_agent import RouterAgent
__all__ = ['Agent', 'ImageAgent', 'Handoff', 'handoff', 'handoff_filters', 'RECOMMENDED_PROMPT_PREFIX', 'prompt_with_handoff_instructions', 'RouterAgent']
__all__ = ['Agent', 'ImageAgent', 'ContextAgent', 'create_context_agent', 'Handoff', 'handoff', 'handoff_filters', 'RECOMMENDED_PROMPT_PREFIX', 'prompt_with_handoff_instructions', 'RouterAgent']
@@ -4,7 +4,7 @@ import json
import copy
import logging
import asyncio
from typing import List, Optional, Any, Dict, Union, Literal, TYPE_CHECKING, Callable, Tuple
from typing import List, Optional, Any, Dict, Union, Literal, TYPE_CHECKING, Callable, Tuple, Generator
from rich.console import Console
from rich.live import Live
from ..llm import (
@@ -206,7 +206,7 @@ class Agent:
knowledge_config: Optional[Dict[str, Any]] = None,
use_system_prompt: Optional[bool] = True,
markdown: bool = True,
stream: bool = True,
stream: bool = False,
self_reflect: bool = False,
max_reflect: int = 3,
min_reflect: int = 1,
@@ -281,8 +281,8 @@ class Agent:
conversations to establish agent behavior and context. Defaults to True.
markdown (bool, optional): Enable markdown formatting in agent responses for better
readability and structure. Defaults to True.
stream (bool, optional): Enable streaming responses from the language model. Set to False
for LLM providers that don't support streaming. Defaults to True.
stream (bool, optional): Enable streaming responses from the language model for real-time
output when using Agent.start() method. Defaults to False for backward compatibility.
self_reflect (bool, optional): Enable self-reflection capabilities where the agent
evaluates and improves its own responses. Defaults to False.
max_reflect (int, optional): Maximum number of self-reflection iterations to prevent
@@ -1070,7 +1070,7 @@ Your Goal: {self.goal}"""
tools=formatted_tools,
start_time=start_time,
console=self.console,
display_fn=display_generating,
display_fn=self.display_generating if self.verbose else None,
reasoning_steps=reasoning_steps
)
@@ -1109,25 +1109,51 @@ Your Goal: {self.goal}"""
reasoning_steps=reasoning_steps
)
else:
# Non-streaming with custom LLM
final_response = self.llm_instance.get_response(
prompt=messages[1:],
system_prompt=messages[0]['content'] if messages and messages[0]['role'] == 'system' else None,
temperature=temperature,
tools=formatted_tools if formatted_tools else None,
verbose=self.verbose,
markdown=self.markdown,
stream=stream,
console=self.console,
execute_tool_fn=self.execute_tool,
agent_name=self.name,
agent_role=self.role,
agent_tools=[t.__name__ for t in self.tools] if self.tools else None,
task_name=task_name,
task_description=task_description,
task_id=task_id,
reasoning_steps=reasoning_steps
)
# Non-streaming with custom LLM - don't show streaming-like behavior
if False: # Don't use display_generating when stream=False to avoid streaming-like behavior
# This block is disabled to maintain consistency with the OpenAI path fix
with Live(
display_generating("", start_time),
console=self.console,
refresh_per_second=4,
) as live:
final_response = self.llm_instance.get_response(
prompt=messages[1:],
system_prompt=messages[0]['content'] if messages and messages[0]['role'] == 'system' else None,
temperature=temperature,
tools=formatted_tools if formatted_tools else None,
verbose=self.verbose,
markdown=self.markdown,
stream=stream,
console=self.console,
execute_tool_fn=self.execute_tool,
agent_name=self.name,
agent_role=self.role,
agent_tools=[t.__name__ for t in self.tools] if self.tools else None,
task_name=task_name,
task_description=task_description,
task_id=task_id,
reasoning_steps=reasoning_steps
)
else:
final_response = self.llm_instance.get_response(
prompt=messages[1:],
system_prompt=messages[0]['content'] if messages and messages[0]['role'] == 'system' else None,
temperature=temperature,
tools=formatted_tools if formatted_tools else None,
verbose=self.verbose,
markdown=self.markdown,
stream=stream,
console=self.console,
execute_tool_fn=self.execute_tool,
agent_name=self.name,
agent_role=self.role,
agent_tools=[t.__name__ for t in self.tools] if self.tools else None,
task_name=task_name,
task_description=task_description,
task_id=task_id,
reasoning_steps=reasoning_steps
)
else:
# Use the standard OpenAI client approach with tool support
# Note: openai_client expects tools in various formats and will format them internally
@@ -1143,7 +1169,7 @@ Your Goal: {self.goal}"""
execute_tool_fn=self.execute_tool,
stream=stream,
console=self.console if (self.verbose or stream) else None,
display_fn=display_generating if stream else None,
display_fn=self.display_generating if self.verbose else None,
reasoning_steps=reasoning_steps,
verbose=self.verbose,
max_iterations=10
@@ -1187,8 +1213,32 @@ Your Goal: {self.goal}"""
task_description=None, # Not available in this context
task_id=None) # Not available in this context
self._final_display_shown = True
def display_generating(self, content: str, start_time: float):
"""Display function for generating animation with agent info."""
from rich.panel import Panel
from rich.markdown import Markdown
elapsed = time.time() - start_time
# Show content if provided (for both streaming and progressive display)
if content:
display_content = Markdown(content) if self.markdown else content
return Panel(
display_content,
title=f"[bold]{self.name}[/bold] - Generating... {elapsed:.1f}s",
border_style="green",
expand=False
)
else:
# No content yet: show generating message
return Panel(
f"[bold cyan]Generating response...[/bold cyan]",
title=f"[bold]{self.name}[/bold] - {elapsed:.1f}s",
border_style="cyan",
expand=False
)
def chat(self, prompt, temperature=0.2, tools=None, output_json=None, output_pydantic=None, reasoning_steps=False, stream=True, task_name=None, task_description=None, task_id=None):
def chat(self, prompt, temperature=0.2, tools=None, output_json=None, output_pydantic=None, reasoning_steps=False, stream=None, task_name=None, task_description=None, task_id=None):
# Reset the final display flag for each new conversation
self._final_display_shown = False
@@ -1209,6 +1259,9 @@ Your Goal: {self.goal}"""
start_time = time.time()
reasoning_steps = reasoning_steps or self.reasoning_steps
# Use agent's stream setting if not explicitly provided
if stream is None:
stream = self.stream
# Search for existing knowledge if any knowledge is provided
if self.knowledge:
search_results = self.knowledge.search(prompt, agent_id=self.agent_id)
@@ -1359,7 +1412,7 @@ Your Goal: {self.goal}"""
agent_tools=agent_tools
)
response = self._chat_completion(messages, temperature=temperature, tools=tools if tools else None, reasoning_steps=reasoning_steps, stream=self.stream, task_name=task_name, task_description=task_description, task_id=task_id)
response = self._chat_completion(messages, temperature=temperature, tools=tools if tools else None, reasoning_steps=reasoning_steps, stream=stream, task_name=task_name, task_description=task_description, task_id=task_id)
if not response:
# Rollback chat history on response failure
self.chat_history = self.chat_history[:chat_history_length]
@@ -1848,7 +1901,12 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
for tool_call in message.tool_calls:
try:
function_name = tool_call.function.name
arguments = json.loads(tool_call.function.arguments)
# Parse JSON arguments safely
try:
arguments = json.loads(tool_call.function.arguments)
except json.JSONDecodeError as json_error:
logging.error(f"Failed to parse tool arguments as JSON: {json_error}")
arguments = {}
# Find the matching tool
tool = next((t for t in tools if t.__name__ == function_name), None)
@@ -1890,25 +1948,16 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
chunks = []
start_time = time.time()
with Live(
display_generating("", start_time),
console=self.console,
refresh_per_second=4,
transient=True,
vertical_overflow="ellipsis",
auto_refresh=True
) as live:
async for chunk in final_response:
chunks.append(chunk)
if chunk.choices[0].delta.content:
full_response_text += chunk.choices[0].delta.content
live.update(display_generating(full_response_text, start_time))
if reasoning_steps and hasattr(chunk.choices[0].delta, "reasoning_content"):
rc = chunk.choices[0].delta.reasoning_content
if rc:
reasoning_content += rc
live.update(display_generating(f"{full_response_text}\n[Reasoning: {reasoning_content}]", start_time))
# Process stream without display_generating since streaming is active
async for chunk in final_response:
chunks.append(chunk)
if chunk.choices[0].delta.content:
full_response_text += chunk.choices[0].delta.content
if reasoning_steps and hasattr(chunk.choices[0].delta, "reasoning_content"):
rc = chunk.choices[0].delta.reasoning_content
if rc:
reasoning_content += rc
self.console.print()
@@ -1937,7 +1986,268 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
def start(self, prompt: str, **kwargs):
"""Start the agent with a prompt. This is a convenience method that wraps chat()."""
return self.chat(prompt, **kwargs)
# Check if streaming is enabled (either from kwargs or agent's stream attribute)
stream_enabled = kwargs.get('stream', getattr(self, 'stream', False))
if stream_enabled:
# Return a generator for streaming response
return self._start_stream(prompt, **kwargs)
else:
# Return regular chat response for backward compatibility
# Explicitly pass the resolved stream parameter to avoid chat() method default
kwargs['stream'] = stream_enabled
return self.chat(prompt, **kwargs)
def _start_stream(self, prompt: str, **kwargs) -> Generator[str, None, None]:
"""Stream generator for real-time response chunks."""
try:
# Reset the final display flag for each new conversation
self._final_display_shown = False
# Temporarily disable verbose mode to prevent console output conflicts during streaming
original_verbose = self.verbose
self.verbose = False
# For custom LLM path, use the new get_response_stream generator
if self._using_custom_llm:
# Handle knowledge search
actual_prompt = prompt
if self.knowledge:
search_results = self.knowledge.search(prompt, agent_id=self.agent_id)
if search_results:
if isinstance(search_results, dict) and 'results' in search_results:
knowledge_content = "\n".join([result['memory'] for result in search_results['results']])
else:
knowledge_content = "\n".join(search_results)
actual_prompt = f"{prompt}\n\nKnowledge: {knowledge_content}"
# Handle tools properly
tools = kwargs.get('tools', self.tools)
if tools is None or (isinstance(tools, list) and len(tools) == 0):
tool_param = self.tools
else:
tool_param = tools
# Convert MCP tools if needed
if tool_param is not None:
from ..mcp.mcp import MCP
if isinstance(tool_param, MCP) and hasattr(tool_param, 'to_openai_tool'):
openai_tool = tool_param.to_openai_tool()
if openai_tool:
if isinstance(openai_tool, list):
tool_param = openai_tool
else:
tool_param = [openai_tool]
# Store chat history length for potential rollback
chat_history_length = len(self.chat_history)
# Normalize prompt content for chat history
normalized_content = actual_prompt
if isinstance(actual_prompt, list):
normalized_content = next((item["text"] for item in actual_prompt if item.get("type") == "text"), "")
# Prevent duplicate messages in chat history
if not (self.chat_history and
self.chat_history[-1].get("role") == "user" and
self.chat_history[-1].get("content") == normalized_content):
self.chat_history.append({"role": "user", "content": normalized_content})
try:
# Use the new streaming generator from LLM class
response_content = ""
for chunk in self.llm_instance.get_response_stream(
prompt=actual_prompt,
system_prompt=self._build_system_prompt(tool_param),
chat_history=self.chat_history,
temperature=kwargs.get('temperature', 0.2),
tools=tool_param,
output_json=kwargs.get('output_json'),
output_pydantic=kwargs.get('output_pydantic'),
verbose=False, # Keep verbose false for streaming
markdown=self.markdown,
agent_name=self.name,
agent_role=self.role,
agent_tools=[t.__name__ if hasattr(t, '__name__') else str(t) for t in (tool_param or [])],
task_name=kwargs.get('task_name'),
task_description=kwargs.get('task_description'),
task_id=kwargs.get('task_id'),
execute_tool_fn=self.execute_tool
):
response_content += chunk
yield chunk
# Add complete response to chat history
if response_content:
self.chat_history.append({"role": "assistant", "content": response_content})
except Exception as e:
# Rollback chat history on error
self.chat_history = self.chat_history[:chat_history_length]
logging.error(f"Custom LLM streaming error: {e}")
raise
else:
# For OpenAI-style models, implement proper streaming without display
# Handle knowledge search
actual_prompt = prompt
if self.knowledge:
search_results = self.knowledge.search(prompt, agent_id=self.agent_id)
if search_results:
if isinstance(search_results, dict) and 'results' in search_results:
knowledge_content = "\n".join([result['memory'] for result in search_results['results']])
else:
knowledge_content = "\n".join(search_results)
actual_prompt = f"{prompt}\n\nKnowledge: {knowledge_content}"
# Handle tools properly
tools = kwargs.get('tools', self.tools)
if tools is None or (isinstance(tools, list) and len(tools) == 0):
tool_param = self.tools
else:
tool_param = tools
# Build messages using the helper method
messages, original_prompt = self._build_messages(actual_prompt, kwargs.get('temperature', 0.2),
kwargs.get('output_json'), kwargs.get('output_pydantic'))
# Store chat history length for potential rollback
chat_history_length = len(self.chat_history)
# Normalize original_prompt for consistent chat history storage
normalized_content = original_prompt
if isinstance(original_prompt, list):
normalized_content = next((item["text"] for item in original_prompt if item.get("type") == "text"), "")
# Prevent duplicate messages in chat history
if not (self.chat_history and
self.chat_history[-1].get("role") == "user" and
self.chat_history[-1].get("content") == normalized_content):
self.chat_history.append({"role": "user", "content": normalized_content})
try:
# Check if OpenAI client is available
if self._openai_client is None:
raise ValueError("OpenAI client is not initialized. Please provide OPENAI_API_KEY or use a custom LLM provider.")
# Format tools for OpenAI
formatted_tools = self._format_tools_for_completion(tool_param)
# Create streaming completion directly without display function
completion_args = {
"model": self.llm,
"messages": messages,
"temperature": kwargs.get('temperature', 0.2),
"stream": True
}
if formatted_tools:
completion_args["tools"] = formatted_tools
completion = self._openai_client.sync_client.chat.completions.create(**completion_args)
# Stream the response chunks without display
response_text = ""
tool_calls_data = []
for chunk in completion:
delta = chunk.choices[0].delta
# Handle text content
if delta.content is not None:
chunk_content = delta.content
response_text += chunk_content
yield chunk_content
# Handle tool calls (accumulate but don't yield as chunks)
if hasattr(delta, 'tool_calls') and delta.tool_calls:
for tool_call_delta in delta.tool_calls:
# Extend tool_calls_data list to accommodate the tool call index
while len(tool_calls_data) <= tool_call_delta.index:
tool_calls_data.append({'id': '', 'function': {'name': '', 'arguments': ''}})
# Accumulate tool call data
if tool_call_delta.id:
tool_calls_data[tool_call_delta.index]['id'] = tool_call_delta.id
if tool_call_delta.function.name:
tool_calls_data[tool_call_delta.index]['function']['name'] = tool_call_delta.function.name
if tool_call_delta.function.arguments:
tool_calls_data[tool_call_delta.index]['function']['arguments'] += tool_call_delta.function.arguments
# Handle any tool calls that were accumulated
if tool_calls_data:
# Add assistant message with tool calls to chat history
assistant_message = {"role": "assistant", "content": response_text}
if tool_calls_data:
assistant_message["tool_calls"] = [
{
"id": tc['id'],
"type": "function",
"function": tc['function']
} for tc in tool_calls_data if tc['id']
]
self.chat_history.append(assistant_message)
# Execute tool calls and add results to chat history
for tool_call in tool_calls_data:
if tool_call['id'] and tool_call['function']['name']:
try:
# Parse JSON arguments safely
try:
parsed_args = json.loads(tool_call['function']['arguments']) if tool_call['function']['arguments'] else {}
except json.JSONDecodeError as json_error:
logging.error(f"Failed to parse tool arguments as JSON: {json_error}")
parsed_args = {}
tool_result = self.execute_tool(
tool_call['function']['name'],
parsed_args
)
# Add tool result to chat history
self.chat_history.append({
"role": "tool",
"tool_call_id": tool_call['id'],
"content": str(tool_result)
})
except Exception as tool_error:
logging.error(f"Tool execution error in streaming: {tool_error}")
# Add error result to chat history
self.chat_history.append({
"role": "tool",
"tool_call_id": tool_call['id'],
"content": f"Error: {str(tool_error)}"
})
else:
# Add complete response to chat history (text-only response)
if response_text:
self.chat_history.append({"role": "assistant", "content": response_text})
except Exception as e:
# Rollback chat history on error
self.chat_history = self.chat_history[:chat_history_length]
logging.error(f"OpenAI streaming error: {e}")
# Fall back to simulated streaming
response = self.chat(prompt, **kwargs)
if response:
words = str(response).split()
chunk_size = max(1, len(words) // 20)
for i in range(0, len(words), chunk_size):
chunk_words = words[i:i + chunk_size]
chunk = ' '.join(chunk_words)
if i + chunk_size < len(words):
chunk += ' '
yield chunk
# Restore original verbose mode
self.verbose = original_verbose
except Exception as e:
# Restore verbose mode on any error
self.verbose = original_verbose
# Graceful fallback to non-streaming if streaming fails
logging.warning(f"Streaming failed, falling back to regular response: {e}")
response = self.chat(prompt, **kwargs)
if response:
yield response
def execute(self, task, context=None):
"""Execute a task synchronously - backward compatibility method"""
File diff suppressed because it is too large. Load diff
@@ -81,16 +81,19 @@ def process_task_context(context_item, verbose=0, user_id=None):
task_name = context_item.name if context_item.name else context_item.description
if context_item.result and task_status == TaskStatus.COMPLETED.value:
return f"Result of previous task {task_name}:\n{context_item.result.raw}"
# Log detailed result for debugging
logger.debug(f"Previous task '{task_name}' result: {context_item.result.raw}")
# Return actual result content without verbose label (essential for task chaining)
return context_item.result.raw
elif task_status == TaskStatus.COMPLETED.value and not context_item.result:
return f"Previous task {task_name} completed but produced no result."
return "" # No result to include
else:
return f"Previous task {task_name} is not yet completed (status: {task_status or TaskStatus.UNKNOWN.value})."
elif isinstance(context_item, dict) and "vector_store" in context_item:
return "" # Task not completed, no context to include
elif isinstance(context_item, dict) and ("vector_store" in context_item or "embedding_db_config" in context_item):
from ..knowledge.knowledge import Knowledge
try:
# Handle both string and dict configs
cfg = context_item["vector_store"]
# Handle both string and dict configs - support both vector_store and embedding_db_config keys for backward compatibility
cfg = context_item.get("vector_store") or context_item.get("embedding_db_config")
if isinstance(cfg, str):
cfg = json.loads(cfg)
@@ -101,9 +104,19 @@ def process_task_context(context_item, verbose=0, user_id=None):
context_item.get("query", ""), # Use query from context if available
user_id=user_id if user_id else None
)
return f"[DB Context]: {str(db_results)}"
# Log knowledge results for debugging (always available for troubleshooting)
logger.debug(f"Knowledge search results ({len(db_results)} items): {str(db_results)}")
# Return actual content without verbose "[DB Context]:" prefix
return str(db_results)
except Exception as e:
return f"[Vector DB Error]: {e}"
# Log error for debugging (always available for troubleshooting)
logger.debug(f"Vector DB Error: {e}")
# Return empty string to avoid exposing error details in AI prompts
# Error details are preserved in debug logs for troubleshooting
return ""
else:
return str(context_item) # Fallback for unknown types
@@ -148,7 +161,9 @@ class PraisonAIAgents:
# Set logger level based on verbose
if verbose >= 5:
logger.setLevel(logging.INFO)
logger.setLevel(logging.INFO) # keep everything ≥INFO
elif verbose >= 3:
logger.setLevel(logging.DEBUG) # surface DEBUG when user asks for it
else:
logger.setLevel(logging.WARNING)
@@ -312,7 +327,7 @@ Expected Output: {task.expected_output}.
if self.verbose >= 3:
logger.info(f"Task {task_id} context items: {len(unique_contexts)}")
for i, ctx in enumerate(unique_contexts):
logger.info(f"Context {i+1}: {ctx[:100]}...")
logger.debug(f"Context {i+1}: {ctx[:100]}...")
context_separator = '\n\n'
task_prompt += f"""
Context:
@@ -635,7 +650,7 @@ Expected Output: {task.expected_output}.
if self.verbose >= 3:
logger.info(f"Task {task_id} context items: {len(unique_contexts)}")
for i, ctx in enumerate(unique_contexts):
logger.info(f"Context {i+1}: {ctx[:100]}...")
logger.debug(f"Context {i+1}: {ctx[:100]}...")
context_separator = '\n\n'
task_prompt += f"""
Context:
@@ -648,7 +663,10 @@ Context:
try:
memory_context = task.memory.build_context_for_task(task.description)
if memory_context:
task_prompt += f"\n\nRelevant memory context:\n{memory_context}"
# Log detailed memory context for debugging
logger.debug(f"Memory context for task '{task.description}': {memory_context}")
# Include actual memory content without verbose headers (essential for AI agent functionality)
task_prompt += f"\n\n{memory_context}"
except Exception as e:
logger.error(f"Error getting memory context: {e}")
@@ -714,7 +714,15 @@ class Knowledge:
memory_result = self.store(memory, user_id=user_id, agent_id=agent_id,
run_id=run_id, metadata=metadata)
if memory_result:
all_results.extend(memory_result.get('results', []))
# Handle both dict and list formats for backward compatibility
if isinstance(memory_result, dict):
all_results.extend(memory_result.get('results', []))
elif isinstance(memory_result, list):
all_results.extend(memory_result)
else:
# Log warning for unexpected types but don't break
import logging
logging.warning(f"Unexpected memory_result type: {type(memory_result)}, skipping")
progress.advance(store_task)
return {'results': all_results, 'relations': []}
@@ -1,19 +1,36 @@
import logging
import warnings
import os
import re
# Disable litellm telemetry before any imports
os.environ["LITELLM_TELEMETRY"] = "False"
# Suppress all relevant logs at module level - consistent with main __init__.py
logging.getLogger("litellm").setLevel(logging.WARNING)
logging.getLogger("openai").setLevel(logging.WARNING)
logging.getLogger("httpx").setLevel(logging.WARNING)
logging.getLogger("httpcore").setLevel(logging.WARNING)
logging.getLogger("pydantic").setLevel(logging.WARNING)
# Check if warnings should be suppressed (consistent with main __init__.py)
def _should_suppress_warnings():
import sys
LOGLEVEL = os.environ.get('LOGLEVEL', 'INFO').upper()
return (LOGLEVEL != 'DEBUG' and
not hasattr(sys, '_called_from_test') and
'pytest' not in sys.modules and
os.environ.get('PYTEST_CURRENT_TEST') is None)
# Suppress pydantic warnings
warnings.filterwarnings("ignore", category=UserWarning, module="pydantic")
# Suppress all relevant logs at module level - more aggressive suppression consistent with main __init__.py (only when not in DEBUG mode)
if _should_suppress_warnings():
logging.getLogger("litellm").setLevel(logging.CRITICAL)
logging.getLogger("openai").setLevel(logging.WARNING)
logging.getLogger("httpx").setLevel(logging.CRITICAL)
logging.getLogger("httpcore").setLevel(logging.CRITICAL)
logging.getLogger("pydantic").setLevel(logging.WARNING)
# Note: litellm child loggers automatically inherit the CRITICAL level from the parent logger
# Warning filters are centrally managed in the main __init__.py file
# Apply additional local suppression for safety during LLM imports (only when not in DEBUG mode)
if _should_suppress_warnings():
for module in ['litellm', 'httpx', 'httpcore', 'pydantic']:
warnings.filterwarnings("ignore", category=DeprecationWarning, module=module)
warnings.filterwarnings("ignore", category=UserWarning, module=module)
# Import after suppressing warnings
from .llm import LLM, LLMContextLengthExceededException
@@ -40,12 +57,21 @@ from .model_router import (
create_routing_agent
)
# Ensure telemetry is disabled after import as well
try:
import litellm
litellm.telemetry = False
except ImportError:
pass
# Ensure comprehensive litellm configuration after import (only when not in DEBUG mode)
if _should_suppress_warnings():
try:
import litellm
# Disable all litellm logging and telemetry features
litellm.telemetry = False
litellm.drop_params = True
if hasattr(litellm, 'suppress_debug_info'):
litellm.suppress_debug_info = True
# Set all litellm loggers to CRITICAL level
if hasattr(litellm, '_logging_obj'):
litellm._logging_obj.setLevel(logging.CRITICAL)
# Note: Child loggers inherit from parent, no need to iterate over all loggers
except ImportError:
pass
__all__ = [
"LLM",
+485 -59
View File
@@ -971,13 +971,39 @@ class LLM:
# Provider doesn't support streaming with tools, use non-streaming
use_streaming = False
# Track whether fallback was successful to avoid duplicate API calls
fallback_completed = False
if use_streaming:
# Streaming approach (with or without tools)
tool_calls = []
response_text = ""
streaming_success = False
if verbose:
with Live(display_generating("", current_time), console=console, refresh_per_second=4) as live:
# Wrap streaming with error handling for LiteLLM JSON parsing errors
try:
if verbose:
# Verbose streaming: show display_generating during streaming
with Live(display_generating("", current_time), console=console, refresh_per_second=4) as live:
for chunk in litellm.completion(
**self._build_completion_params(
messages=messages,
tools=formatted_tools,
temperature=temperature,
stream=True,
output_json=output_json,
output_pydantic=output_pydantic,
**kwargs
)
):
if chunk and chunk.choices and chunk.choices[0].delta:
delta = chunk.choices[0].delta
response_text, tool_calls = self._process_stream_delta(
delta, response_text, tool_calls, formatted_tools
)
live.update(display_generating(response_text, current_time))
else:
# Non-verbose streaming: no display_generating during streaming
for chunk in litellm.completion(
**self._build_completion_params(
messages=messages,
@@ -994,74 +1020,205 @@ class LLM:
response_text, tool_calls = self._process_stream_delta(
delta, response_text, tool_calls, formatted_tools
)
if delta.content:
live.update(display_generating(response_text, current_time))
streaming_success = True
except Exception as streaming_error:
# Handle streaming errors with recovery logic
if self._is_streaming_error_recoverable(streaming_error):
if verbose:
logging.warning(f"Streaming error (recoverable): {streaming_error}")
logging.warning("Falling back to non-streaming mode")
# Immediately perform non-streaming fallback with actual API call
try:
if verbose:
# When verbose=True, always use streaming for better UX
with Live(display_generating("", current_time), console=console, refresh_per_second=4, transient=True) as live:
response_text = ""
# Use streaming when verbose for progressive display
for chunk in litellm.completion(
**self._build_completion_params(
messages=messages,
tools=formatted_tools,
temperature=temperature,
stream=True, # Always stream when verbose=True
output_json=output_json,
output_pydantic=output_pydantic,
**kwargs
)
):
if chunk and chunk.choices and chunk.choices[0].delta:
delta = chunk.choices[0].delta
if hasattr(delta, "content") and delta.content:
response_text += delta.content
live.update(display_generating(response_text, current_time))
# Clear the live display after completion
console.print()
# Create final response structure
final_response = {
"choices": [{
"message": {
"content": response_text,
"tool_calls": None
}
}]
}
else:
# For non-streaming + non-verbose: no display_generating (per user requirements)
final_response = litellm.completion(
**self._build_completion_params(
messages=messages,
tools=formatted_tools,
temperature=temperature,
stream=False,
output_json=output_json,
output_pydantic=output_pydantic,
**kwargs
)
)
response_text = final_response["choices"][0]["message"]["content"]
# Execute callbacks and display based on verbose setting
if verbose and not interaction_displayed:
# Display the complete response at once (this will trigger callbacks internally)
display_interaction(
original_prompt,
response_text,
markdown=markdown,
generation_time=time.time() - current_time,
console=console,
agent_name=agent_name,
agent_role=agent_role,
agent_tools=agent_tools,
task_name=task_name,
task_description=task_description,
task_id=task_id
)
interaction_displayed = True
callback_executed = True
elif not callback_executed:
# Only execute callback if display_interaction hasn't been called
execute_sync_callback(
'interaction',
message=original_prompt,
response=response_text,
markdown=markdown,
generation_time=time.time() - current_time,
agent_name=agent_name,
agent_role=agent_role,
agent_tools=agent_tools,
task_name=task_name,
task_description=task_description,
task_id=task_id
)
callback_executed = True
# Mark that fallback completed successfully
fallback_completed = True
streaming_success = False
except Exception as fallback_error:
# If non-streaming also fails, create a graceful fallback with partial streaming data
logging.warning(f"Non-streaming fallback also failed: {fallback_error}")
logging.warning("Using partial streaming response data")
response_text = response_text or ""
# Create a mock response with whatever partial data we have
final_response = {
"choices": [{
"message": {
"content": response_text,
"tool_calls": tool_calls if tool_calls else None
}
}]
}
fallback_completed = True
streaming_success = False
else:
# For non-recoverable errors, re-raise immediately
logging.error(f"Non-recoverable streaming error: {streaming_error}")
raise streaming_error
if streaming_success:
response_text = response_text.strip() if response_text else ""
# Execute callbacks after streaming completes (only if not verbose, since verbose will call display_interaction later)
if not verbose and not callback_executed:
execute_sync_callback(
'interaction',
message=original_prompt,
response=response_text,
markdown=markdown,
generation_time=time.time() - current_time,
agent_name=agent_name,
agent_role=agent_role,
agent_tools=agent_tools,
task_name=task_name,
task_description=task_description,
task_id=task_id
)
callback_executed = True
# Create a mock final_response with the captured data
final_response = {
"choices": [{
"message": {
"content": response_text,
"tool_calls": tool_calls if tool_calls else None
}
}]
}
# Only execute non-streaming if we haven't used streaming AND fallback hasn't completed
if not use_streaming and not fallback_completed:
# Non-streaming approach (when tools require it, streaming is disabled, or streaming fallback)
if verbose:
# When verbose=True, always use streaming for better UX
with Live(display_generating("", current_time), console=console, refresh_per_second=4, transient=True) as live:
response_text = ""
# Use streaming when verbose for progressive display
for chunk in litellm.completion(
**self._build_completion_params(
messages=messages,
tools=formatted_tools,
temperature=temperature,
stream=True, # Always stream when verbose=True
output_json=output_json,
output_pydantic=output_pydantic,
**kwargs
)
):
if chunk and chunk.choices and chunk.choices[0].delta:
delta = chunk.choices[0].delta
if hasattr(delta, "content") and delta.content:
response_text += delta.content
live.update(display_generating(response_text, current_time))
# Clear the live display after completion
console.print()
# Create final response structure
final_response = {
"choices": [{
"message": {
"content": response_text,
"tool_calls": None
}
}]
}
else:
# Non-verbose streaming
for chunk in litellm.completion(
# For non-streaming + non-verbose: no display_generating (per user requirements)
final_response = litellm.completion(
**self._build_completion_params(
messages=messages,
tools=formatted_tools,
temperature=temperature,
stream=True,
stream=False,
output_json=output_json,
output_pydantic=output_pydantic,
**kwargs
)
):
if chunk and chunk.choices and chunk.choices[0].delta:
delta = chunk.choices[0].delta
if delta.content:
response_text += delta.content
# Capture tool calls from streaming chunks if provider supports it
if formatted_tools and self._supports_streaming_tools():
tool_calls = self._process_tool_calls_from_stream(delta, tool_calls)
response_text = response_text.strip() if response_text else ""
# Execute callbacks after streaming completes (only if not verbose, since verbose will call display_interaction later)
if not verbose and not callback_executed:
execute_sync_callback(
'interaction',
message=original_prompt,
response=response_text,
markdown=markdown,
generation_time=time.time() - current_time,
agent_name=agent_name,
agent_role=agent_role,
agent_tools=agent_tools,
task_name=task_name,
task_description=task_description,
task_id=task_id
)
callback_executed = True
# Create a mock final_response with the captured data
final_response = {
"choices": [{
"message": {
"content": response_text,
"tool_calls": tool_calls if tool_calls else None
}
}]
}
else:
# Non-streaming approach (when tools require it or streaming is disabled)
final_response = litellm.completion(
**self._build_completion_params(
messages=messages,
tools=formatted_tools,
temperature=temperature,
stream=False,
output_json=output_json,
output_pydantic=output_pydantic,
**kwargs
)
)
response_text = final_response["choices"][0]["message"]["content"]
response_text = final_response["choices"][0]["message"]["content"]
# Execute callbacks and display based on verbose setting
if verbose and not interaction_displayed:
@@ -1557,11 +1714,280 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
total_time = time.time() - start_time
logging.debug(f"get_response completed in {total_time:.2f} seconds")
def get_response_stream(
self,
prompt: Union[str, List[Dict]],
system_prompt: Optional[str] = None,
chat_history: Optional[List[Dict]] = None,
temperature: float = 0.2,
tools: Optional[List[Any]] = None,
output_json: Optional[BaseModel] = None,
output_pydantic: Optional[BaseModel] = None,
verbose: bool = False, # Default to non-verbose for streaming
markdown: bool = True,
agent_name: Optional[str] = None,
agent_role: Optional[str] = None,
agent_tools: Optional[List[str]] = None,
task_name: Optional[str] = None,
task_description: Optional[str] = None,
task_id: Optional[str] = None,
execute_tool_fn: Optional[Callable] = None,
**kwargs
):
"""Generator that yields real-time response chunks from the LLM.
This method provides true streaming by yielding content chunks as they
are received from the underlying LLM, enabling real-time display of
responses without waiting for the complete response.
Args:
prompt: The prompt to send to the LLM
system_prompt: Optional system prompt
chat_history: Optional chat history
temperature: Sampling temperature
tools: Optional list of tools for function calling
output_json: Optional JSON schema for structured output
output_pydantic: Optional Pydantic model for structured output
verbose: Whether to enable verbose logging (default False for streaming)
markdown: Whether to enable markdown processing
agent_name: Optional agent name for logging
agent_role: Optional agent role for logging
agent_tools: Optional list of agent tools for logging
task_name: Optional task name for logging
task_description: Optional task description for logging
task_id: Optional task ID for logging
execute_tool_fn: Optional function for executing tools
**kwargs: Additional parameters
Yields:
str: Individual content chunks as they are received from the LLM
Raises:
Exception: If streaming fails or LLM call encounters an error
"""
try:
import litellm
# Build messages using existing logic
messages, original_prompt = self._build_messages(
prompt=prompt,
system_prompt=system_prompt,
chat_history=chat_history,
output_json=output_json,
output_pydantic=output_pydantic
)
# Format tools for litellm
formatted_tools = self._format_tools_for_litellm(tools)
# Determine if we should use streaming based on tool support
use_streaming = True
if formatted_tools and not self._supports_streaming_tools():
# Provider doesn't support streaming with tools, fall back to non-streaming
use_streaming = False
if use_streaming:
# Real-time streaming approach with tool call support
try:
tool_calls = []
response_text = ""
consecutive_errors = 0
max_consecutive_errors = 3 # Fallback to non-streaming after 3 consecutive errors
stream_iterator = litellm.completion(
**self._build_completion_params(
messages=messages,
tools=formatted_tools,
temperature=temperature,
stream=True,
output_json=output_json,
output_pydantic=output_pydantic,
**kwargs
)
)
# Wrap the iteration with additional error handling for LiteLLM JSON parsing errors
try:
for chunk in stream_iterator:
try:
if chunk and chunk.choices and chunk.choices[0].delta:
delta = chunk.choices[0].delta
# Process both content and tool calls using existing helper
response_text, tool_calls = self._process_stream_delta(
delta, response_text, tool_calls, formatted_tools
)
# Yield content chunks in real-time as they arrive
if delta.content:
yield delta.content
# Reset consecutive error counter only after successful chunk processing
consecutive_errors = 0
except Exception as chunk_error:
consecutive_errors += 1
# Log the specific error for debugging
if verbose:
logging.warning(f"Chunk processing error ({consecutive_errors}/{max_consecutive_errors}): {chunk_error}")
# Check if this error is recoverable using our helper method
if self._is_streaming_error_recoverable(chunk_error):
if verbose:
logging.warning("Recoverable streaming error detected, skipping malformed chunk and continuing")
# Skip this malformed chunk and continue if we haven't hit the limit
if consecutive_errors < max_consecutive_errors:
continue
else:
# Too many recoverable errors, fallback to non-streaming
logging.warning(f"Too many consecutive streaming errors ({consecutive_errors}), falling back to non-streaming mode")
raise Exception(f"Streaming failed with {consecutive_errors} consecutive errors") from chunk_error
else:
# For non-recoverable errors, re-raise immediately
logging.error(f"Non-recoverable streaming error: {chunk_error}")
raise chunk_error
except Exception as iterator_error:
# Handle errors that occur during stream iteration itself (e.g., JSON parsing in LiteLLM)
error_msg = str(iterator_error).lower()
# Check if this is a recoverable streaming error (including JSON parsing errors)
if self._is_streaming_error_recoverable(iterator_error):
if verbose:
logging.warning(f"Stream iterator error detected (recoverable): {iterator_error}")
logging.warning("Falling back to non-streaming mode due to stream iteration failure")
# Force fallback to non-streaming for iterator-level errors
raise Exception("Stream iteration failed with recoverable error, falling back to non-streaming") from iterator_error
else:
# For non-recoverable errors, re-raise immediately
logging.error(f"Non-recoverable stream iterator error: {iterator_error}")
raise iterator_error
# After streaming completes, handle tool calls if present
if tool_calls and execute_tool_fn:
# Add assistant message with tool calls to conversation
if self._is_ollama_provider():
messages.append({
"role": "assistant",
"content": response_text
})
else:
serializable_tool_calls = self._serialize_tool_calls(tool_calls)
messages.append({
"role": "assistant",
"content": response_text,
"tool_calls": serializable_tool_calls
})
# Execute tool calls and add results to conversation
for tool_call in tool_calls:
is_ollama = self._is_ollama_provider()
function_name, arguments, tool_call_id = self._extract_tool_call_info(tool_call, is_ollama)
try:
# Execute the tool
tool_result = execute_tool_fn(function_name, arguments)
# Add tool result to messages
tool_message = self._create_tool_message(function_name, tool_result, tool_call_id, is_ollama)
messages.append(tool_message)
except Exception as e:
logging.error(f"Tool execution error for {function_name}: {e}")
# Add error message to conversation
error_message = self._create_tool_message(
function_name, f"Error executing tool: {e}", tool_call_id, is_ollama
)
messages.append(error_message)
# Continue conversation after tool execution - get follow-up response
try:
follow_up_response = litellm.completion(
**self._build_completion_params(
messages=messages,
tools=formatted_tools,
temperature=temperature,
stream=False,
**kwargs
)
)
if follow_up_response and follow_up_response.choices:
follow_up_content = follow_up_response.choices[0].message.content
if follow_up_content:
# Yield the follow-up response after tool execution
yield follow_up_content
except Exception as e:
logging.error(f"Follow-up response failed: {e}")
except Exception as e:
error_msg = str(e).lower()
# Provide more specific error messages based on the error type
if any(keyword in error_msg for keyword in ['json', 'expecting property name', 'parse', 'decode']):
logging.warning(f"Streaming failed due to JSON parsing errors (likely malformed chunks from provider): {e}")
elif 'connection' in error_msg or 'timeout' in error_msg:
logging.warning(f"Streaming failed due to connection issues: {e}")
else:
logging.error(f"Streaming failed with unexpected error: {e}")
# Fall back to non-streaming if streaming fails
use_streaming = False
if not use_streaming:
# Fall back to non-streaming and yield the complete response
try:
response = litellm.completion(
**self._build_completion_params(
messages=messages,
tools=formatted_tools,
temperature=temperature,
stream=False,
output_json=output_json,
output_pydantic=output_pydantic,
**kwargs
)
)
if response and response.choices:
content = response.choices[0].message.content
if content:
# Yield the complete response as a single chunk
yield content
except Exception as e:
logging.error(f"Non-streaming fallback failed: {e}")
raise
except Exception as e:
logging.error(f"Error in get_response_stream: {e}")
raise
def _is_gemini_model(self) -> bool:
"""Check if the model is a Gemini model."""
if not self.model:
return False
return any(prefix in self.model.lower() for prefix in ['gemini', 'gemini/', 'google/gemini'])
def _is_streaming_error_recoverable(self, error: Exception) -> bool:
"""Check if a streaming error is recoverable (e.g., malformed chunk vs connection error)."""
error_msg = str(error).lower()
# JSON parsing errors are often recoverable (skip malformed chunk and continue)
json_error_keywords = ['json', 'expecting property name', 'parse', 'decode', 'invalid json']
if any(keyword in error_msg for keyword in json_error_keywords):
return True
# Connection errors might be temporary but are less recoverable in streaming context
connection_error_keywords = ['connection', 'timeout', 'network', 'http']
if any(keyword in error_msg for keyword in connection_error_keywords):
return False
# Other errors are generally not recoverable
return False
async def get_response_async(
self,
@@ -838,14 +838,55 @@ class OpenAIClient:
)
else:
# Process as regular non-streaming response
final_response = self.create_completion(
messages=messages,
model=model,
temperature=temperature,
tools=formatted_tools,
stream=False,
**kwargs
)
if display_fn and console:
# When verbose (display_fn provided), use streaming for better UX
try:
with Live(display_fn("", start_time), console=console, refresh_per_second=4, transient=True) as live:
# Use streaming when display_fn is provided for progressive display
response_stream = self.create_completion(
messages=messages,
model=model,
temperature=temperature,
tools=formatted_tools,
stream=True, # Always stream when verbose/display_fn
**kwargs
)
full_response_text = ""
chunks = []
# Process streaming response
for chunk in response_stream:
chunks.append(chunk)
if chunk.choices[0].delta.content:
full_response_text += chunk.choices[0].delta.content
live.update(display_fn(full_response_text, start_time))
# Process final response from chunks
final_response = process_stream_chunks(chunks)
# Clear the last generating display with a blank line
console.print()
except Exception as e:
self.logger.error(f"Error in Live display for non-streaming: {e}")
# Fallback to regular completion without display
final_response = self.create_completion(
messages=messages,
model=model,
temperature=temperature,
tools=formatted_tools,
stream=False,
**kwargs
)
else:
final_response = self.create_completion(
messages=messages,
model=model,
temperature=temperature,
tools=formatted_tools,
stream=False,
**kwargs
)
if not final_response:
return None
@@ -969,14 +1010,55 @@ class OpenAIClient:
)
else:
# Process as regular non-streaming response
final_response = await self.acreate_completion(
messages=messages,
model=model,
temperature=temperature,
tools=formatted_tools,
stream=False,
**kwargs
)
if display_fn and console:
# When verbose (display_fn provided), use streaming for better UX
try:
with Live(display_fn("", start_time), console=console, refresh_per_second=4, transient=True) as live:
# Use streaming when display_fn is provided for progressive display
response_stream = await self.acreate_completion(
messages=messages,
model=model,
temperature=temperature,
tools=formatted_tools,
stream=True, # Always stream when verbose/display_fn
**kwargs
)
full_response_text = ""
chunks = []
# Process streaming response
async for chunk in response_stream:
chunks.append(chunk)
if chunk.choices[0].delta.content:
full_response_text += chunk.choices[0].delta.content
live.update(display_fn(full_response_text, start_time))
# Process final response from chunks
final_response = process_stream_chunks(chunks)
# Clear the last generating display with a blank line
console.print()
except Exception as e:
self.logger.error(f"Error in Live display for async non-streaming: {e}")
# Fallback to regular completion without display
final_response = await self.acreate_completion(
messages=messages,
model=model,
temperature=temperature,
tools=formatted_tools,
stream=False,
**kwargs
)
else:
final_response = await self.acreate_completion(
messages=messages,
model=model,
temperature=temperature,
tools=formatted_tools,
stream=False,
**kwargs
)
if not final_response:
return None
@@ -10,9 +10,19 @@ from datetime import datetime
# Disable litellm telemetry before any imports
os.environ["LITELLM_TELEMETRY"] = "False"
# Set up logger
# Set up logger with custom TRACE level
logger = logging.getLogger(__name__)
# Add custom TRACE level (below DEBUG)
TRACE_LEVEL = 5
logging.addLevelName(TRACE_LEVEL, 'TRACE')
def trace(self, message, *args, **kwargs):
if self.isEnabledFor(TRACE_LEVEL):
self._log(TRACE_LEVEL, message, args, **kwargs)
logging.Logger.trace = trace
try:
import chromadb
from chromadb.config import Settings as ChromaSettings
@@ -128,6 +138,8 @@ class Memory:
logging.getLogger('httpx').setLevel(logging.WARNING)
logging.getLogger('httpcore').setLevel(logging.WARNING)
logging.getLogger('chromadb.segment.impl.vector.local_persistent_hnsw').setLevel(logging.ERROR)
logging.getLogger('utils').setLevel(logging.WARNING)
logging.getLogger('litellm.utils').setLevel(logging.WARNING)
self.provider = self.cfg.get("provider", "rag")
self.use_mem0 = (self.provider.lower() == "mem0") and MEM0_AVAILABLE
@@ -770,7 +782,7 @@ class Memory:
import litellm
logger.info("Getting embeddings from LiteLLM...")
logger.debug(f"Embedding input text: {text}")
logger.trace(f"Embedding input text: {text}")
response = litellm.embedding(
model=self.embedding_model,
@@ -778,7 +790,7 @@ class Memory:
)
embedding = response.data[0]["embedding"]
logger.info("Successfully got embeddings from LiteLLM")
logger.debug(f"Received embedding of length: {len(embedding)}")
logger.trace(f"Received embedding of length: {len(embedding)}")
elif OPENAI_AVAILABLE:
# Fallback to OpenAI client
@@ -786,7 +798,7 @@ class Memory:
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
logger.info("Getting embeddings from OpenAI...")
logger.debug(f"Embedding input text: {text}")
logger.trace(f"Embedding input text: {text}")
response = client.embeddings.create(
input=text,
@@ -794,7 +806,7 @@ class Memory:
)
embedding = response.data[0].embedding
logger.info("Successfully got embeddings from OpenAI")
logger.debug(f"Received embedding of length: {len(embedding)}")
logger.trace(f"Received embedding of length: {len(embedding)}")
else:
logger.warning("Neither litellm nor openai available for embeddings")
return
@@ -1243,12 +1255,22 @@ class Memory:
task_descr: str,
user_id: Optional[str] = None,
additional: str = "",
max_items: int = 3
max_items: int = 3,
include_in_output: Optional[bool] = None
) -> str:
"""
Merges relevant short-term, long-term, entity, user memories
into a single text block with deduplication and clean formatting.
Args:
include_in_output: If None, memory content is only included when debug logging is enabled.
If True, memory content is always included.
If False, memory content is never included (only logged for debugging).
"""
# Determine whether to include memory content in output based on logging level
if include_in_output is None:
include_in_output = logging.getLogger().getEffectiveLevel() == logging.DEBUG
q = (task_descr + " " + additional).strip()
lines = []
seen_contents = set() # Track unique contents
@@ -1306,16 +1328,20 @@ class Memory:
formatted_hits.append(formatted)
if formatted_hits:
# Add section header
if lines:
lines.append("") # Space before new section
lines.append(title)
lines.append("=" * len(title)) # Underline the title
lines.append("") # Space after title
# Log detailed memory content for debugging including section headers
brief_title = title.replace(" Context", "").replace("Memory ", "")
logger.debug(f"Memory section '{brief_title}' ({len(formatted_hits)} items): {formatted_hits}")
# Add formatted content with bullet points
for content in formatted_hits:
lines.append(f" • {content}")
# Only include memory content in output when specified (controlled by log level or explicit parameter)
if include_in_output:
# Add only the actual memory content for AI agent use (no headers)
if lines:
lines.append("") # Space before new section
# Include actual memory content without verbose section headers
for hit in formatted_hits:
lines.append(f"• {hit}")
lines.append("") # Space after content
# Add each section
# First get all results
@@ -1522,3 +1548,46 @@ class Memory:
logger.info(f"After quality filter: {len(filtered)} results")
return filtered
def get_all_memories(self) -> List[Dict[str, Any]]:
"""Get all memories from both short-term and long-term storage"""
all_memories = []
try:
# Get short-term memories
conn = sqlite3.connect(self.short_db)
c = conn.cursor()
rows = c.execute("SELECT id, content, meta, created_at FROM short_mem").fetchall()
conn.close()
for row in rows:
meta = json.loads(row[2] or "{}")
all_memories.append({
"id": row[0],
"text": row[1],
"metadata": meta,
"created_at": row[3],
"type": "short_term"
})
# Get long-term memories
conn = sqlite3.connect(self.long_db)
c = conn.cursor()
rows = c.execute("SELECT id, content, meta, created_at FROM long_mem").fetchall()
conn.close()
for row in rows:
meta = json.loads(row[2] or "{}")
all_memories.append({
"id": row[0],
"text": row[1],
"metadata": meta,
"created_at": row[3],
"type": "long_term"
})
return all_memories
except Exception as e:
self._log_verbose(f"Error getting all memories: {e}", logging.ERROR)
return []
@@ -402,13 +402,14 @@ Expected Output: {self.expected_output}.
context_results.append(f"Input Content: {' '.join(str(x) for x in context_item)}")
elif hasattr(context_item, 'result'): # Task object
if context_item.result:
context_results.append(
f"Result of previous task {context_item.name if context_item.name else context_item.description}:\n{context_item.result.raw}"
)
task_name = context_item.name if context_item.name else context_item.description
# Log detailed result for debugging
logger.debug(f"Previous task '{task_name}' result: {context_item.result.raw}")
# Include actual result content without verbose labels (essential for task chaining)
context_results.append(context_item.result.raw)
else:
context_results.append(
f"Previous task {context_item.name if context_item.name else context_item.description} has no result yet."
)
# Task has no result yet, don't include verbose status message
pass
# Join unique context results
unique_contexts = list(dict.fromkeys(context_results)) # Remove duplicates
@@ -1,13 +1,24 @@
"""
PraisonAI Agents Minimal Telemetry Module
PraisonAI Agents Telemetry & Performance Monitoring Module
This module provides:
1. Anonymous usage tracking with privacy-first design
2. User-friendly performance monitoring and analysis tools
This module provides anonymous usage tracking with privacy-first design.
Telemetry is opt-out and can be disabled via environment variables:
- PRAISONAI_TELEMETRY_DISABLED=true
- PRAISONAI_DISABLE_TELEMETRY=true
- DO_NOT_TRACK=true
No personal data, prompts, or responses are collected.
Performance Monitoring Features:
- Function performance tracking with detailed statistics
- API call monitoring and analysis
- Function execution flow visualization
- Performance bottleneck identification
- Real-time performance reporting
- CLI interface for easy access
"""
import os
@@ -20,15 +31,64 @@ if TYPE_CHECKING:
# Import the classes for real (not just type checking)
from .telemetry import MinimalTelemetry, TelemetryCollector
# Import performance monitoring tools
try:
from .performance_monitor import (
PerformanceMonitor, performance_monitor,
monitor_function, track_api_call, get_performance_report,
get_function_stats, get_api_stats, get_slowest_functions,
get_slowest_apis, clear_performance_data
)
from .performance_utils import (
FunctionFlowAnalyzer, PerformanceAnalyzer,
flow_analyzer, performance_analyzer,
analyze_function_flow, visualize_execution_flow,
analyze_performance_trends, generate_comprehensive_report
)
from .performance_cli import PerformanceCLI
PERFORMANCE_MONITORING_AVAILABLE = True
except ImportError:
PERFORMANCE_MONITORING_AVAILABLE = False
__all__ = [
# Core telemetry
'get_telemetry',
'enable_telemetry',
'disable_telemetry',
'disable_telemetry',
'force_shutdown_telemetry',
'MinimalTelemetry',
'TelemetryCollector', # For backward compatibility
]
# Add performance monitoring to __all__ if available
if PERFORMANCE_MONITORING_AVAILABLE:
__all__.extend([
# Performance monitoring classes
'PerformanceMonitor',
'FunctionFlowAnalyzer',
'PerformanceAnalyzer',
'PerformanceCLI',
# Global instances
'performance_monitor',
'flow_analyzer',
'performance_analyzer',
# Convenience functions
'monitor_function',
'track_api_call',
'get_performance_report',
'get_function_stats',
'get_api_stats',
'get_slowest_functions',
'get_slowest_apis',
'clear_performance_data',
'analyze_function_flow',
'visualize_execution_flow',
'analyze_performance_trends',
'generate_comprehensive_report',
# Availability flag
'PERFORMANCE_MONITORING_AVAILABLE'
])
def get_telemetry() -> 'MinimalTelemetry':
"""Get the global telemetry instance."""
@@ -42,13 +42,27 @@ def instrument_agent(agent: 'Agent', telemetry: Optional['MinimalTelemetry'] = N
if original_chat:
@wraps(original_chat)
def instrumented_chat(*args, **kwargs):
import threading
try:
result = original_chat(*args, **kwargs)
telemetry.track_agent_execution(agent.name, success=True)
# Track success asynchronously to prevent blocking
def track_async():
try:
telemetry.track_agent_execution(agent.name, success=True)
except:
pass # Ignore telemetry errors
threading.Thread(target=track_async, daemon=True).start()
return result
except Exception as e:
telemetry.track_agent_execution(agent.name, success=False)
telemetry.track_error(type(e).__name__)
# Track error asynchronously
def track_error_async():
try:
telemetry.track_agent_execution(agent.name, success=False)
telemetry.track_error(type(e).__name__)
except:
pass # Ignore telemetry errors
threading.Thread(target=track_error_async, daemon=True).start()
raise
agent.chat = instrumented_chat
@@ -57,13 +71,53 @@ def instrument_agent(agent: 'Agent', telemetry: Optional['MinimalTelemetry'] = N
if original_start:
@wraps(original_start)
def instrumented_start(*args, **kwargs):
import types
import threading
try:
result = original_start(*args, **kwargs)
telemetry.track_agent_execution(agent.name, success=True)
return result
# Check if result is a generator (streaming mode)
if isinstance(result, types.GeneratorType):
# For streaming, defer telemetry tracking to avoid blocking
def streaming_wrapper():
try:
for chunk in result:
yield chunk
# Track success only after streaming completes
# Use a separate thread to make it truly non-blocking
def track_async():
try:
telemetry.track_agent_execution(agent.name, success=True)
except:
pass # Ignore telemetry errors
threading.Thread(target=track_async, daemon=True).start()
except Exception as e:
# Track error immediately
threading.Thread(target=lambda: telemetry.track_agent_execution(agent.name, success=False), daemon=True).start()
threading.Thread(target=lambda: telemetry.track_error(type(e).__name__), daemon=True).start()
raise
return streaming_wrapper()
else:
# For non-streaming, track immediately but asynchronously
def track_async():
try:
telemetry.track_agent_execution(agent.name, success=True)
except:
pass # Ignore telemetry errors
threading.Thread(target=track_async, daemon=True).start()
return result
except Exception as e:
telemetry.track_agent_execution(agent.name, success=False)
telemetry.track_error(type(e).__name__)
# Track error immediately but asynchronously
def track_error_async():
try:
telemetry.track_agent_execution(agent.name, success=False)
telemetry.track_error(type(e).__name__)
except:
pass # Ignore telemetry errors
threading.Thread(target=track_error_async, daemon=True).start()
raise
agent.start = instrumented_start
@@ -72,13 +126,27 @@ def instrument_agent(agent: 'Agent', telemetry: Optional['MinimalTelemetry'] = N
if original_run:
@wraps(original_run)
def instrumented_run(*args, **kwargs):
import threading
try:
result = original_run(*args, **kwargs)
telemetry.track_agent_execution(agent.name, success=True)
# Track success asynchronously to prevent blocking
def track_async():
try:
telemetry.track_agent_execution(agent.name, success=True)
except:
pass # Ignore telemetry errors
threading.Thread(target=track_async, daemon=True).start()
return result
except Exception as e:
telemetry.track_agent_execution(agent.name, success=False)
telemetry.track_error(type(e).__name__)
# Track error asynchronously
def track_error_async():
try:
telemetry.track_agent_execution(agent.name, success=False)
telemetry.track_error(type(e).__name__)
except:
pass # Ignore telemetry errors
threading.Thread(target=track_error_async, daemon=True).start()
raise
agent.run = instrumented_run
@@ -0,0 +1,397 @@
"""
Performance Monitoring CLI for PraisonAI
Command-line interface for accessing performance monitoring features.
Provides easy access to function performance, API call tracking, and flow analysis.
Usage:
python -m praisonaiagents.telemetry.performance_cli [command] [options]
Or import and use programmatically:
from praisonaiagents.telemetry.performance_cli import PerformanceCLI
cli = PerformanceCLI()
cli.show_performance_report()
"""
import argparse
import json
import sys
from typing import Optional
import logging
try:
from .performance_monitor import performance_monitor, get_performance_report
from .performance_utils import (
analyze_function_flow, visualize_execution_flow,
analyze_performance_trends, generate_comprehensive_report
)
PERFORMANCE_TOOLS_AVAILABLE = True
except ImportError:
PERFORMANCE_TOOLS_AVAILABLE = False
logger = logging.getLogger(__name__)
class PerformanceCLI:
"""Command-line interface for performance monitoring."""
def __init__(self):
if not PERFORMANCE_TOOLS_AVAILABLE:
print("❌ Performance monitoring tools not available")
sys.exit(1)
def show_performance_report(self, detailed: bool = False) -> None:
"""Show performance monitoring report."""
print("📊 PraisonAI Performance Monitoring Report")
print("=" * 60)
if detailed:
report = generate_comprehensive_report()
else:
report = get_performance_report()
print(report)
def show_function_stats(self, function_name: Optional[str] = None) -> None:
"""Show function performance statistics."""
stats = performance_monitor.get_function_performance(function_name)
if not stats:
print("❌ No function performance data available")
return
print("🔧 Function Performance Statistics")
print("=" * 50)
for func, data in stats.items():
print(f"\n📝 Function: {func}")
print(f" Calls: {data['call_count']}")
print(f" Total Time: {data['total_time']:.3f}s")
if data['call_count'] > 0:
print(f" Average Time: {data.get('average_time', 0):.3f}s")
print(f" Min Time: {data['min_time']:.3f}s")
print(f" Max Time: {data['max_time']:.3f}s")
print(f" Success Rate: {data.get('success_rate', 0)*100:.1f}%")
print(f" Errors: {data['error_count']}")
def show_api_stats(self, api_name: Optional[str] = None) -> None:
"""Show API call performance statistics."""
stats = performance_monitor.get_api_call_performance(api_name)
if not stats:
print("❌ No API performance data available")
return
print("🌐 API Call Performance Statistics")
print("=" * 50)
for api, data in stats.items():
print(f"\n🔗 API: {api}")
print(f" Calls: {data['call_count']}")
print(f" Total Time: {data['total_time']:.3f}s")
if data['call_count'] > 0:
print(f" Average Time: {data.get('average_time', 0):.3f}s")
print(f" Min Time: {data['min_time']:.3f}s")
print(f" Max Time: {data['max_time']:.3f}s")
print(f" Success Rate: {data.get('success_rate', 0)*100:.1f}%")
print(f" Successful Calls: {data['success_count']}")
print(f" Failed Calls: {data['error_count']}")
def show_slowest_functions(self, limit: int = 10) -> None:
"""Show slowest performing functions."""
slowest = performance_monitor.get_slowest_functions(limit)
if not slowest:
print("❌ No function performance data available")
return
print(f"🐌 Top {limit} Slowest Functions")
print("=" * 50)
for i, func in enumerate(slowest, 1):
print(f"{i:2d}. {func['function']}")
print(f" Average: {func['average_time']:.3f}s")
print(f" Max: {func['max_time']:.3f}s")
print(f" Calls: {func['call_count']}")
print()
def show_slowest_apis(self, limit: int = 10) -> None:
"""Show slowest performing API calls."""
slowest = performance_monitor.get_slowest_api_calls(limit)
if not slowest:
print("❌ No API performance data available")
return
print(f"🐌 Top {limit} Slowest API Calls")
print("=" * 50)
for i, api in enumerate(slowest, 1):
print(f"{i:2d}. {api['api']}")
print(f" Average: {api['average_time']:.3f}s")
print(f" Max: {api['max_time']:.3f}s")
print(f" Success Rate: {api['success_rate']*100:.1f}%")
print(f" Calls: {api['call_count']}")
print()
def show_function_flow(self, format: str = "text", events: int = 50) -> None:
"""Show function execution flow."""
print(f"🔄 Function Execution Flow (Last {events} events)")
print("=" * 60)
flow_data = performance_monitor.get_function_flow(events)
if not flow_data:
print("❌ No flow data available")
return
if format == "json":
print(json.dumps(flow_data, indent=2))
else:
visualization = visualize_execution_flow(format=format)
print(visualization)
def analyze_flow(self) -> None:
"""Analyze function execution flow for bottlenecks and patterns."""
print("🔍 Function Flow Analysis")
print("=" * 50)
analysis = analyze_function_flow()
if "error" in analysis:
print(f"❌ {analysis['error']}")
return
if "message" in analysis:
print(f"ℹ️ {analysis['message']}")
return
# Show bottlenecks
bottlenecks = analysis.get("bottlenecks", [])
if bottlenecks:
print("🚨 Identified Bottlenecks:")
for bottleneck in bottlenecks:
severity_emoji = "🔴" if bottleneck["severity"] == "high" else "🟡"
print(f" {severity_emoji} {bottleneck['function']}")
print(f" Average: {bottleneck['average_duration']:.3f}s")
print(f" Max: {bottleneck['max_duration']:.3f}s")
print(f" Calls: {bottleneck['call_count']}")
print()
else:
print("✅ No significant bottlenecks identified")
# Show statistics
stats = analysis.get("statistics", {})
if stats:
print("\n📊 Flow Statistics:")
print(f" Total Function Calls: {stats.get('function_calls', 0)}")
print(f" Completed Calls: {stats.get('completed_calls', 0)}")
print(f" Success Rate: {stats.get('success_rate', 0)*100:.1f}%")
print(f" Total Execution Time: {stats.get('total_execution_time', 0):.3f}s")
print(f" Average Execution Time: {stats.get('average_execution_time', 0):.3f}s")
# Show parallelism info
parallelism = analysis.get("parallelism", {})
if parallelism:
print("\n🧵 Parallelism Analysis:")
print(f" Total Threads: {parallelism.get('total_threads', 0)}")
print(f" Peak Concurrency: {parallelism.get('peak_concurrency', 0)}")
def show_active_calls(self) -> None:
"""Show currently active function calls."""
active = performance_monitor.get_active_calls()
print("⚡ Currently Active Function Calls")
print("=" * 50)
if not active:
print("✅ No active function calls")
return
for _call_id, info in active.items():
print(f"🔄 {info['function']}")
print(f" Duration: {info['duration']:.1f}s")
print(f" Thread: {info['thread_id']}")
print(f" Started: {info['started_at']}")
print()
def show_trends(self) -> None:
"""Show performance trends analysis."""
print("📈 Performance Trends Analysis")
print("=" * 50)
trends = analyze_performance_trends()
if "error" in trends:
print(f"❌ {trends['error']}")
return
# Show recommendations
recommendations = trends.get("recommendations", [])
if recommendations:
print("💡 Performance Recommendations:")
for rec in recommendations:
print(f" {rec}")
print()
# Show function trends
func_trends = trends.get("function_trends", {})
improving = func_trends.get("improving", [])
degrading = func_trends.get("degrading", [])
if improving:
print("📈 Improving Functions:")
for trend in improving:
print(f" ✅ {trend['function']} ({trend['change_percent']:+.1f}%)")
if degrading:
print("\n📉 Degrading Functions:")
for trend in degrading:
print(f" ⚠️ {trend['function']} ({trend['change_percent']:+.1f}%)")
# Show API trends
api_trends = trends.get("api_trends", {})
slowest_apis = api_trends.get("slowest_apis", [])
least_reliable = api_trends.get("least_reliable", [])
if slowest_apis:
print("\n🐌 Slowest APIs:")
for api in slowest_apis[:3]:
print(f" • {api['api']}: {api['average_time']:.3f}s avg")
if least_reliable:
print("\n⚠️ Least Reliable APIs:")
for api in least_reliable[:3]:
if api['success_rate'] < 1.0:
print(f" • {api['api']}: {api['success_rate']*100:.1f}% success rate")
def export_data(self, format: str = "json", output_file: Optional[str] = None) -> None:
"""Export performance data to file or stdout."""
data = performance_monitor.export_data(format)
if output_file:
try:
with open(output_file, 'w') as f:
f.write(data)
print(f"✅ Data exported to {output_file}")
except (IOError, OSError) as e:
print(f"❌ Error exporting to file: {e}")
else:
print(data)
def clear_data(self) -> None:
"""Clear all performance monitoring data."""
performance_monitor.clear_statistics()
print("✅ All performance monitoring data cleared")
def main():
"""Main CLI entry point."""
parser = argparse.ArgumentParser(
description="PraisonAI Performance Monitoring CLI",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
python -m praisonaiagents.telemetry.performance_cli report
python -m praisonaiagents.telemetry.performance_cli functions
python -m praisonaiagents.telemetry.performance_cli apis
python -m praisonaiagents.telemetry.performance_cli slowest-functions 5
python -m praisonaiagents.telemetry.performance_cli flow
python -m praisonaiagents.telemetry.performance_cli analyze-flow
python -m praisonaiagents.telemetry.performance_cli trends
"""
)
subparsers = parser.add_subparsers(dest='command', help='Available commands')
# Report command
report_parser = subparsers.add_parser('report', help='Show performance report')
report_parser.add_argument('--detailed', action='store_true',
help='Show detailed comprehensive report')
# Functions command
func_parser = subparsers.add_parser('functions', help='Show function statistics')
func_parser.add_argument('--name', help='Specific function name')
# APIs command
api_parser = subparsers.add_parser('apis', help='Show API call statistics')
api_parser.add_argument('--name', help='Specific API name')
# Slowest functions command
slowest_func_parser = subparsers.add_parser('slowest-functions',
help='Show slowest functions')
slowest_func_parser.add_argument('limit', type=int, nargs='?', default=10,
help='Number of functions to show')
# Slowest APIs command
slowest_api_parser = subparsers.add_parser('slowest-apis',
help='Show slowest API calls')
slowest_api_parser.add_argument('limit', type=int, nargs='?', default=10,
help='Number of APIs to show')
# Flow command
flow_parser = subparsers.add_parser('flow', help='Show function execution flow')
flow_parser.add_argument('--format', choices=['text', 'json', 'mermaid'],
default='text', help='Output format')
flow_parser.add_argument('--events', type=int, default=50,
help='Number of recent events to show')
# Analyze flow command
subparsers.add_parser('analyze-flow', help='Analyze function flow for bottlenecks')
# Active calls command
subparsers.add_parser('active', help='Show currently active function calls')
# Trends command
subparsers.add_parser('trends', help='Show performance trends analysis')
# Export command
export_parser = subparsers.add_parser('export', help='Export performance data')
export_parser.add_argument('--format', choices=['json', 'dict'], default='json',
help='Export format')
export_parser.add_argument('--output', help='Output file (default: stdout)')
# Clear command
subparsers.add_parser('clear', help='Clear all performance data')
args = parser.parse_args()
if not args.command:
parser.print_help()
return
try:
cli = PerformanceCLI()
if args.command == 'report':
cli.show_performance_report(detailed=args.detailed)
elif args.command == 'functions':
cli.show_function_stats(args.name)
elif args.command == 'apis':
cli.show_api_stats(args.name)
elif args.command == 'slowest-functions':
cli.show_slowest_functions(args.limit)
elif args.command == 'slowest-apis':
cli.show_slowest_apis(args.limit)
elif args.command == 'flow':
cli.show_function_flow(args.format, args.events)
elif args.command == 'analyze-flow':
cli.analyze_flow()
elif args.command == 'active':
cli.show_active_calls()
elif args.command == 'trends':
cli.show_trends()
elif args.command == 'export':
cli.export_data(args.format, args.output)
elif args.command == 'clear':
cli.clear_data()
except KeyboardInterrupt:
print("\n❌ Operation cancelled by user")
sys.exit(1)
except Exception as e:
print(f"❌ Error: {e}")
sys.exit(1)
if __name__ == "__main__":
main()
@@ -0,0 +1,573 @@
"""
User-Friendly Performance Monitoring for PraisonAI
This module provides easy-to-use tools for monitoring function performance,
analyzing execution flow, and tracking API calls. Built on top of the existing
comprehensive telemetry infrastructure.
Features:
- Function performance evaluation with timing and statistics
- Function flow analysis and visualization
- API call tracking and performance monitoring
- Real-time performance reporting
- Easy integration with existing agents and workflows
"""
import time
import json
import threading
import functools
from collections import defaultdict, deque
from typing import Dict, Any, List, Optional, Callable, Union
from contextlib import contextmanager
from datetime import datetime
import logging
try:
from .telemetry import MinimalTelemetry
TELEMETRY_AVAILABLE = True
except ImportError:
TELEMETRY_AVAILABLE = False
logger = logging.getLogger(__name__)
class PerformanceMonitor:
"""
User-friendly performance monitoring for functions, API calls, and workflows.
Provides comprehensive performance tracking with simple APIs for:
- Function execution timing and statistics
- API call performance monitoring
- Function flow analysis
- Real-time performance reporting
"""
def __init__(self, max_entries: int = 10000):
"""
Initialize the performance monitor.
Args:
max_entries: Maximum number of performance entries to keep in memory
"""
self.max_entries = max_entries
self._lock = threading.RLock()
# Performance data storage
self._function_stats = defaultdict(lambda: {
'call_count': 0,
'total_time': 0.0,
'min_time': float('inf'),
'max_time': 0.0,
'recent_times': deque(maxlen=100),
'error_count': 0,
'last_called': None
})
self._api_calls = defaultdict(lambda: {
'call_count': 0,
'total_time': 0.0,
'min_time': float('inf'),
'max_time': 0.0,
'success_count': 0,
'error_count': 0,
'recent_calls': deque(maxlen=50)
})
self._function_flow = deque(maxlen=self.max_entries)
self._active_calls = {}
# Integration with existing telemetry
self._telemetry = None
if TELEMETRY_AVAILABLE:
try:
self._telemetry = MinimalTelemetry()
except Exception as e:
logger.debug(f"Could not initialize telemetry integration: {e}")
def monitor_function(self, func_name: Optional[str] = None):
"""
Decorator to monitor function performance.
Args:
func_name: Optional custom name for the function
Example:
@performance_monitor.monitor_function("my_critical_function")
def my_function():
return "result"
"""
def decorator(func: Callable) -> Callable:
name = func_name or f"{func.__module__}.{func.__qualname__}"
@functools.wraps(func)
def wrapper(*args, **kwargs):
start_time = time.time()
thread_id = threading.get_ident()
call_id = f"{name}_{thread_id}_{start_time}"
# Track active call
with self._lock:
self._active_calls[call_id] = {
'function': name,
'start_time': start_time,
'thread_id': thread_id
}
# Add to function flow
self._function_flow.append({
'function': name,
'timestamp': datetime.now(),
'event': 'start',
'thread_id': thread_id
})
try:
result = func(*args, **kwargs)
success = True
error = None
except Exception as e:
success = False
error = str(e)
raise
finally:
end_time = time.time()
execution_time = end_time - start_time
# Update performance statistics
self._record_function_performance(name, execution_time, success, error)
# Clean up active call
with self._lock:
self._active_calls.pop(call_id, None)
# Add to function flow
self._function_flow.append({
'function': name,
'timestamp': datetime.now(),
'event': 'end',
'duration': execution_time,
'success': success,
'thread_id': thread_id
})
return result
return wrapper
return decorator
@contextmanager
def track_api_call(self, api_name: str, endpoint: Optional[str] = None):
"""
Context manager to track API call performance.
Args:
api_name: Name of the API (e.g., "openai", "anthropic")
endpoint: Optional specific endpoint
Example:
with performance_monitor.track_api_call("openai", "/v1/chat/completions"):
response = openai_client.chat.completions.create(...)
"""
call_name = f"{api_name}:{endpoint}" if endpoint else api_name
start_time = time.time()
try:
yield
success = True
error = None
except Exception as e:
success = False
error = str(e)
raise
finally:
end_time = time.time()
execution_time = end_time - start_time
self._record_api_call(call_name, execution_time, success, error)
def _record_function_performance(self, func_name: str, execution_time: float,
success: bool, error: Optional[str] = None):
"""Record function performance statistics."""
with self._lock:
stats = self._function_stats[func_name]
stats['call_count'] += 1
stats['total_time'] += execution_time
stats['min_time'] = min(stats['min_time'], execution_time)
stats['max_time'] = max(stats['max_time'], execution_time)
stats['recent_times'].append(execution_time)
stats['last_called'] = datetime.now()
if not success:
stats['error_count'] += 1
# Integrate with existing telemetry
if self._telemetry:
self._telemetry.track_tool_usage(
tool_name=func_name,
success=success,
execution_time=execution_time
)
def _record_api_call(self, api_name: str, execution_time: float,
success: bool, error: Optional[str] = None):
"""Record API call performance statistics."""
with self._lock:
stats = self._api_calls[api_name]
stats['call_count'] += 1
stats['total_time'] += execution_time
stats['min_time'] = min(stats['min_time'], execution_time)
stats['max_time'] = max(stats['max_time'], execution_time)
if success:
stats['success_count'] += 1
else:
stats['error_count'] += 1
stats['recent_calls'].append({
'timestamp': datetime.now(),
'duration': execution_time,
'success': success,
'error': error
})
def get_function_performance(self, func_name: Optional[str] = None) -> Dict[str, Any]:
"""
Get performance statistics for functions.
Args:
func_name: Specific function name, or None for all functions
Returns:
Dictionary with performance statistics
"""
with self._lock:
if func_name:
if func_name not in self._function_stats:
return {}
stats = self._function_stats[func_name].copy()
# Convert deque to list for JSON serialization
stats['recent_times'] = list(stats['recent_times'])
# Calculate derived statistics
if stats['call_count'] > 0:
stats['average_time'] = stats['total_time'] / stats['call_count']
stats['success_rate'] = (stats['call_count'] - stats['error_count']) / stats['call_count']
if stats['recent_times']:
stats['recent_average'] = sum(stats['recent_times']) / len(stats['recent_times'])
return {func_name: stats}
else:
# Return all function statistics
result = {}
for name, stats in self._function_stats.items():
processed_stats = stats.copy()
processed_stats['recent_times'] = list(processed_stats['recent_times'])
if processed_stats['call_count'] > 0:
processed_stats['average_time'] = processed_stats['total_time'] / processed_stats['call_count']
processed_stats['success_rate'] = (processed_stats['call_count'] - processed_stats['error_count']) / processed_stats['call_count']
if processed_stats['recent_times']:
processed_stats['recent_average'] = sum(processed_stats['recent_times']) / len(processed_stats['recent_times'])
result[name] = processed_stats
return result
def get_api_call_performance(self, api_name: Optional[str] = None) -> Dict[str, Any]:
"""
Get performance statistics for API calls.
Args:
api_name: Specific API name, or None for all APIs
Returns:
Dictionary with API call performance statistics
"""
with self._lock:
if api_name:
if api_name not in self._api_calls:
return {}
stats = self._api_calls[api_name].copy()
# Convert deque to list for JSON serialization
recent_calls = []
for call in stats['recent_calls']:
call_copy = call.copy()
call_copy['timestamp'] = call_copy['timestamp'].isoformat()
recent_calls.append(call_copy)
stats['recent_calls'] = recent_calls
# Calculate derived statistics
if stats['call_count'] > 0:
stats['average_time'] = stats['total_time'] / stats['call_count']
stats['success_rate'] = stats['success_count'] / stats['call_count']
stats['error_rate'] = stats['error_count'] / stats['call_count']
return {api_name: stats}
else:
# Return all API statistics
result = {}
for name, stats in self._api_calls.items():
processed_stats = stats.copy()
# Convert recent calls
recent_calls = []
for call in processed_stats['recent_calls']:
call_copy = call.copy()
call_copy['timestamp'] = call_copy['timestamp'].isoformat()
recent_calls.append(call_copy)
processed_stats['recent_calls'] = recent_calls
if processed_stats['call_count'] > 0:
processed_stats['average_time'] = processed_stats['total_time'] / processed_stats['call_count']
processed_stats['success_rate'] = processed_stats['success_count'] / processed_stats['call_count']
processed_stats['error_rate'] = processed_stats['error_count'] / processed_stats['call_count']
result[name] = processed_stats
return result
def get_function_flow(self, last_n: Optional[int] = None) -> List[Dict[str, Any]]:
"""
Get function execution flow information.
Args:
last_n: Number of recent flow events to return, or None for all
Returns:
List of flow events showing function execution order and timing
"""
with self._lock:
flow = []
for event in self._function_flow.copy():
# Create a copy of the event to avoid modifying original
event_copy = event.copy()
# Convert timestamp to ISO format if it's a datetime object
timestamp = event_copy['timestamp']
if hasattr(timestamp, 'isoformat'):
event_copy['timestamp'] = timestamp.isoformat()
elif not isinstance(timestamp, str):
event_copy['timestamp'] = str(timestamp)
flow.append(event_copy)
if last_n:
return flow[-last_n:]
return flow
def get_active_calls(self) -> Dict[str, Any]:
"""Get information about currently executing functions."""
with self._lock:
active = {}
current_time = time.time()
for call_id, info in self._active_calls.items():
duration = current_time - info['start_time']
active[call_id] = {
'function': info['function'],
'duration': duration,
'thread_id': info['thread_id'],
'started_at': datetime.fromtimestamp(info['start_time']).isoformat()
}
return active
def get_slowest_functions(self, limit: int = 10) -> List[Dict[str, Any]]:
"""Get the slowest performing functions."""
with self._lock:
functions = []
for name, stats in self._function_stats.items():
if stats['call_count'] > 0:
avg_time = stats['total_time'] / stats['call_count']
functions.append({
'function': name,
'average_time': avg_time,
'max_time': stats['max_time'],
'call_count': stats['call_count']
})
# Sort by average time descending
functions.sort(key=lambda x: x['average_time'], reverse=True)
return functions[:limit]
def get_slowest_api_calls(self, limit: int = 10) -> List[Dict[str, Any]]:
"""Get the slowest performing API calls."""
with self._lock:
apis = []
for name, stats in self._api_calls.items():
if stats['call_count'] > 0:
avg_time = stats['total_time'] / stats['call_count']
apis.append({
'api': name,
'average_time': avg_time,
'max_time': stats['max_time'],
'call_count': stats['call_count'],
'success_rate': stats['success_count'] / stats['call_count']
})
# Sort by average time descending
apis.sort(key=lambda x: x['average_time'], reverse=True)
return apis[:limit]
def generate_performance_report(self) -> str:
"""
Generate a comprehensive performance report.
Returns:
Formatted string with performance analysis
"""
report = []
report.append("=" * 80)
report.append("PRAISONAI PERFORMANCE MONITORING REPORT")
report.append("=" * 80)
report.append(f"Generated at: {datetime.now().isoformat()}")
report.append("")
# Function performance summary
func_stats = self.get_function_performance()
if func_stats:
report.append("📊 FUNCTION PERFORMANCE SUMMARY")
report.append("-" * 40)
total_functions = len(func_stats)
total_calls = sum(stats['call_count'] for stats in func_stats.values())
total_errors = sum(stats['error_count'] for stats in func_stats.values())
report.append(f"Total Functions Monitored: {total_functions}")
report.append(f"Total Function Calls: {total_calls}")
report.append(f"Total Errors: {total_errors}")
report.append(f"Overall Success Rate: {((total_calls - total_errors) / total_calls * 100):.1f}%" if total_calls > 0 else "N/A")
report.append("")
# Slowest functions
slowest = self.get_slowest_functions(5)
if slowest:
report.append("🐌 SLOWEST FUNCTIONS (Top 5)")
for i, func in enumerate(slowest, 1):
report.append(f"{i}. {func['function']}")
report.append(f" Average: {func['average_time']:.3f}s | Max: {func['max_time']:.3f}s | Calls: {func['call_count']}")
report.append("")
# API performance summary
api_stats = self.get_api_call_performance()
if api_stats:
report.append("🌐 API CALL PERFORMANCE SUMMARY")
report.append("-" * 40)
total_apis = len(api_stats)
total_api_calls = sum(stats['call_count'] for stats in api_stats.values())
total_api_errors = sum(stats['error_count'] for stats in api_stats.values())
report.append(f"Total APIs Monitored: {total_apis}")
report.append(f"Total API Calls: {total_api_calls}")
report.append(f"Total API Errors: {total_api_errors}")
report.append(f"Overall API Success Rate: {((total_api_calls - total_api_errors) / total_api_calls * 100):.1f}%" if total_api_calls > 0 else "N/A")
report.append("")
# Slowest APIs
slowest_apis = self.get_slowest_api_calls(5)
if slowest_apis:
report.append("🐌 SLOWEST API CALLS (Top 5)")
for i, api in enumerate(slowest_apis, 1):
report.append(f"{i}. {api['api']}")
report.append(f" Average: {api['average_time']:.3f}s | Max: {api['max_time']:.3f}s | Success Rate: {api['success_rate']*100:.1f}%")
report.append("")
# Active calls
active = self.get_active_calls()
if active:
report.append("⚡ CURRENTLY ACTIVE FUNCTION CALLS")
report.append("-" * 40)
for _call_id, info in active.items():
report.append(f"• {info['function']} (running {info['duration']:.1f}s)")
report.append("")
# Function flow summary
flow = self.get_function_flow(10)
if flow:
report.append("🔄 RECENT FUNCTION FLOW (Last 10 Events)")
report.append("-" * 40)
for event in flow:
event_type = "🟢 START" if event['event'] == 'start' else "🔴 END"
duration_info = f" ({event.get('duration', 0):.3f}s)" if event['event'] == 'end' else ""
report.append(f"{event_type} {event['function']}{duration_info}")
report.append("")
report.append("=" * 80)
return "\n".join(report)
def clear_statistics(self):
"""Clear all performance statistics."""
with self._lock:
self._function_stats.clear()
self._api_calls.clear()
self._function_flow.clear()
self._active_calls.clear()
def export_data(self, format: str = "json") -> Union[str, Dict[str, Any]]:
"""
Export all performance data.
Args:
format: Export format ("json" or "dict")
Returns:
Performance data in requested format
"""
data = {
'functions': self.get_function_performance(),
'api_calls': self.get_api_call_performance(),
'flow': self.get_function_flow(),
'active_calls': self.get_active_calls(),
'export_timestamp': datetime.now().isoformat()
}
if format == "json":
return json.dumps(data, indent=2)
return data
# Global performance monitor instance
performance_monitor = PerformanceMonitor()
# Convenience functions for easy access
def monitor_function(func_name: Optional[str] = None):
"""Convenience decorator for monitoring function performance."""
return performance_monitor.monitor_function(func_name)
def track_api_call(api_name: str, endpoint: Optional[str] = None):
"""Convenience context manager for tracking API calls."""
return performance_monitor.track_api_call(api_name, endpoint)
def get_performance_report() -> str:
"""Get a comprehensive performance report."""
return performance_monitor.generate_performance_report()
def get_function_stats(func_name: Optional[str] = None) -> Dict[str, Any]:
"""Get function performance statistics."""
return performance_monitor.get_function_performance(func_name)
def get_api_stats(api_name: Optional[str] = None) -> Dict[str, Any]:
"""Get API call performance statistics."""
return performance_monitor.get_api_call_performance(api_name)
def get_slowest_functions(limit: int = 10) -> List[Dict[str, Any]]:
"""Get the slowest performing functions."""
return performance_monitor.get_slowest_functions(limit)
def get_slowest_apis(limit: int = 10) -> List[Dict[str, Any]]:
"""Get the slowest performing API calls."""
return performance_monitor.get_slowest_api_calls(limit)
def clear_performance_data():
"""Clear all performance monitoring data."""
performance_monitor.clear_statistics()
@@ -0,0 +1,571 @@
"""
Performance Analysis Utilities for PraisonAI
This module provides advanced analysis tools for function flow visualization,
performance bottleneck identification, and comprehensive reporting.
Features:
- Function flow analysis and visualization
- Performance bottleneck detection
- Execution path mapping
- Performance trend analysis
- Advanced reporting utilities
"""
import json
from collections import defaultdict
from typing import Dict, Any, List, Optional
from datetime import datetime
import logging
from dataclasses import dataclass
try:
from .performance_monitor import performance_monitor
PERFORMANCE_MONITOR_AVAILABLE = True
except ImportError:
PERFORMANCE_MONITOR_AVAILABLE = False
logger = logging.getLogger(__name__)
# Performance analysis thresholds
BOTTLENECK_THRESHOLD_AVERAGE = 1.0 # seconds - average duration to consider bottleneck
BOTTLENECK_THRESHOLD_MAX = 5.0 # seconds - max duration to consider bottleneck
HIGH_SEVERITY_THRESHOLD = 2.0 # seconds - average duration for high severity bottleneck
class FunctionFlowAnalyzer:
"""
Advanced function flow analysis and visualization.
Provides tools for analyzing function execution patterns,
identifying bottlenecks, and visualizing execution flow.
"""
def __init__(self):
self.logger = logging.getLogger(__name__)
def analyze_execution_flow(self, flow_data: Optional[List[Dict]] = None) -> Dict[str, Any]:
"""
Analyze function execution flow to identify patterns and bottlenecks.
Args:
flow_data: Optional flow data, or None to use current monitor data
Returns:
Analysis results with flow patterns, bottlenecks, and statistics
"""
if flow_data is None:
if not PERFORMANCE_MONITOR_AVAILABLE:
return {"error": "Performance monitor not available and no flow data provided"}
flow_data = performance_monitor.get_function_flow()
if not flow_data:
return {"message": "No flow data available"}
analysis = {
"total_events": len(flow_data),
"execution_patterns": self._analyze_patterns(flow_data),
"bottlenecks": self._identify_bottlenecks(flow_data),
"parallelism": self._analyze_parallelism(flow_data),
"call_chains": self._build_call_chains(flow_data),
"statistics": self._calculate_flow_statistics(flow_data)
}
return analysis
def _analyze_patterns(self, flow_data: List[Dict]) -> Dict[str, Any]:
"""Analyze execution patterns in the flow data."""
patterns = {
"most_frequent_sequences": [],
"recursive_calls": [],
"long_running_chains": [],
"error_patterns": []
}
# Group events by function to find sequences
function_sequences = defaultdict(list)
for event in flow_data:
func_name = event.get('function', 'unknown')
function_sequences[func_name].append(event)
# Find most frequent function sequences
sequence_counts = defaultdict(int)
for i in range(len(flow_data) - 1):
current_func = flow_data[i].get('function')
next_func = flow_data[i + 1].get('function')
if current_func and next_func:
sequence_counts[(current_func, next_func)] += 1
# Sort by frequency
frequent_sequences = sorted(sequence_counts.items(), key=lambda x: x[1], reverse=True)[:5]
patterns["most_frequent_sequences"] = [
{"sequence": f"{seq[0]} -> {seq[1]}", "count": count}
for seq, count in frequent_sequences
]
# Find recursive calls
for func_name, events in function_sequences.items():
nested_level = 0
for event in events:
if event.get('event') == 'start':
nested_level += 1
if nested_level > 1:
patterns["recursive_calls"].append({
"function": func_name,
"max_depth": nested_level,
"timestamp": event.get('timestamp')
})
elif event.get('event') == 'end':
nested_level = max(0, nested_level - 1)
return patterns
def _identify_bottlenecks(self, flow_data: List[Dict]) -> List[Dict[str, Any]]:
"""Identify performance bottlenecks in the execution flow."""
bottlenecks = []
# Group start/end events
function_durations = defaultdict(list)
active_calls = {}
for event in flow_data:
func_name = event.get('function', 'unknown')
event_type = event.get('event')
timestamp = event.get('timestamp')
thread_id = event.get('thread_id', 0)
if event_type == 'start':
call_key = f"{func_name}_{thread_id}_{timestamp}"
active_calls[call_key] = event
elif event_type == 'end':
duration = event.get('duration', 0)
if duration > 0:
function_durations[func_name].append(duration)
# Find functions with consistently slow performance
for func_name, durations in function_durations.items():
if durations:
avg_duration = sum(durations) / len(durations)
max_duration = max(durations)
# Consider it a bottleneck based on defined thresholds
if avg_duration > BOTTLENECK_THRESHOLD_AVERAGE or max_duration > BOTTLENECK_THRESHOLD_MAX:
bottlenecks.append({
"function": func_name,
"average_duration": avg_duration,
"max_duration": max_duration,
"call_count": len(durations),
"severity": "high" if avg_duration > HIGH_SEVERITY_THRESHOLD else "medium"
})
# Sort by severity and duration
bottlenecks.sort(key=lambda x: x["average_duration"], reverse=True)
return bottlenecks
def _analyze_parallelism(self, flow_data: List[Dict]) -> Dict[str, Any]:
"""Analyze parallelism and concurrent execution patterns."""
thread_activities = defaultdict(list)
for event in flow_data:
thread_id = event.get('thread_id', 0)
thread_activities[thread_id].append(event)
# Find peak concurrency
timestamp_activities = defaultdict(int)
for event in flow_data:
if event.get('event') == 'start':
timestamp = event.get('timestamp')
timestamp_activities[timestamp] += 1
peak_concurrency = max(timestamp_activities.values()) if timestamp_activities else 0
return {
"total_threads": len(thread_activities),
"peak_concurrency": peak_concurrency,
"thread_utilization": {
str(thread_id): len(events)
for thread_id, events in thread_activities.items()
}
}
def _build_call_chains(self, flow_data: List[Dict]) -> List[Dict[str, Any]]:
"""Build call chains from flow data."""
# Track call chains per thread more efficiently
thread_chains = defaultdict(list)
call_stacks = defaultdict(list)
current_chains = defaultdict(list)
for event in flow_data:
thread_id = event.get('thread_id', 0)
func_name = event.get('function', 'unknown')
event_type = event.get('event')
if event_type == 'start':
call_stacks[thread_id].append(func_name)
current_chains[thread_id].append(func_name)
elif event_type == 'end' and call_stacks[thread_id] and call_stacks[thread_id][-1] == func_name:
call_stacks[thread_id].pop()
# If this completes a top-level call (stack becomes empty), record the chain
if not call_stacks[thread_id] and current_chains[thread_id]:
chain = current_chains[thread_id].copy()
thread_chains[thread_id].append({
"thread_id": thread_id,
"chain_length": len(chain),
"functions": chain
})
current_chains[thread_id].clear()
# Flatten all chains and return top 10
all_chains = []
for chains in thread_chains.values():
all_chains.extend(chains)
return all_chains[:10]
def _calculate_flow_statistics(self, flow_data: List[Dict]) -> Dict[str, Any]:
"""Calculate comprehensive flow statistics."""
total_events = len(flow_data)
start_events = [e for e in flow_data if e.get('event') == 'start']
end_events = [e for e in flow_data if e.get('event') == 'end']
successful_events = [e for e in end_events if e.get('success', True)]
total_duration = sum(e.get('duration', 0) for e in end_events)
return {
"total_events": total_events,
"function_calls": len(start_events),
"completed_calls": len(end_events),
"successful_calls": len(successful_events),
"success_rate": len(successful_events) / len(end_events) if end_events else 0,
"total_execution_time": total_duration,
"average_execution_time": total_duration / len(end_events) if end_events else 0
}
def visualize_flow(self, flow_data: Optional[List[Dict]] = None,
format: str = "text") -> str:
"""
Create a visual representation of the function execution flow.
Args:
flow_data: Optional flow data, or None to use current monitor data
format: Output format ("text", "mermaid", "json")
Returns:
Formatted visualization string
"""
if flow_data is None:
if not PERFORMANCE_MONITOR_AVAILABLE:
return "Performance monitor not available and no flow data provided"
flow_data = performance_monitor.get_function_flow(50) # Last 50 events
if not flow_data:
return "No flow data available"
if format == "text":
return self._create_text_visualization(flow_data)
elif format == "mermaid":
return self._create_mermaid_diagram(flow_data)
elif format == "json":
return json.dumps(flow_data, indent=2)
else:
return "Unknown format"
def _create_text_visualization(self, flow_data: List[Dict]) -> str:
"""Create a text-based visualization of the execution flow."""
lines = []
lines.append("📊 FUNCTION EXECUTION FLOW VISUALIZATION")
lines.append("=" * 60)
# Group by thread for better visualization
threads = defaultdict(list)
for event in flow_data:
thread_id = event.get('thread_id', 0)
threads[thread_id].append(event)
for thread_id, events in threads.items():
lines.append(f"\n🧵 Thread {thread_id}:")
lines.append("-" * 30)
call_stack = []
for event in events:
func_name = event.get('function', 'unknown')
event_type = event.get('event')
duration = event.get('duration', 0)
success = event.get('success', True)
if event_type == 'start':
call_stack.append(func_name)
indent = " " * (len(call_stack) - 1)
lines.append(f"{indent}🟢 START {func_name}")
elif event_type == 'end':
if call_stack and call_stack[-1] == func_name:
call_stack.pop()
indent = " " * len(call_stack)
status = "✅" if success else "❌"
lines.append(f"{indent}{status} END {func_name} ({duration:.3f}s)")
return "\n".join(lines)
def _create_mermaid_diagram(self, flow_data: List[Dict]) -> str:
"""Create a Mermaid diagram representation of the flow."""
lines = ["graph TD"]
# Build flow connections
node_counter = 0
node_map = {}
for event in flow_data:
if event.get('event') == 'start':
func_name = event.get('function', 'unknown')
if func_name not in node_map:
node_map[func_name] = f"n{node_counter}"
node_counter += 1
# Add nodes
for func_name, node_id in node_map.items():
lines.append(f" {node_id}[{func_name}]")
# Add connections based on call sequence
prev_func = None
for event in flow_data:
if event.get('event') == 'start':
curr_func = event.get('function', 'unknown')
if prev_func and curr_func != prev_func:
prev_node = node_map.get(prev_func)
curr_node = node_map.get(curr_func)
if prev_node and curr_node:
lines.append(f" {prev_node} --> {curr_node}")
prev_func = curr_func
return "\n".join(lines)
class PerformanceAnalyzer:
"""
Comprehensive performance analysis tools.
Provides advanced analysis capabilities for identifying performance
issues, trends, and optimization opportunities.
"""
def __init__(self):
self.flow_analyzer = FunctionFlowAnalyzer()
def analyze_performance_trends(self, hours_back: int = 24) -> Dict[str, Any]:
"""
Analyze performance trends over time.
Args:
hours_back: Number of hours to analyze
Returns:
Trend analysis results
"""
if not PERFORMANCE_MONITOR_AVAILABLE:
return {"error": "Performance monitor not available"}
# Get current performance data
func_stats = performance_monitor.get_function_performance()
api_stats = performance_monitor.get_api_call_performance()
trends = {
"analysis_period_hours": hours_back,
"function_trends": self._analyze_function_trends(func_stats),
"api_trends": self._analyze_api_trends(api_stats),
"recommendations": self._generate_recommendations(func_stats, api_stats)
}
return trends
def _analyze_function_trends(self, func_stats: Dict[str, Any]) -> Dict[str, Any]:
"""Analyze function performance trends."""
trends = {
"improving": [],
"degrading": [],
"stable": []
}
for func_name, stats in func_stats.items():
recent_times = stats.get('recent_times', [])
if len(recent_times) >= 10: # Need sufficient data
# Compare first half to second half
mid_point = len(recent_times) // 2
first_half_avg = sum(recent_times[:mid_point]) / mid_point
second_half_avg = sum(recent_times[mid_point:]) / (len(recent_times) - mid_point)
if first_half_avg != 0:
change_percent = ((second_half_avg - first_half_avg) / first_half_avg) * 100
else:
change_percent = 0.0 # No change if first half average is zero
trend_data = {
"function": func_name,
"change_percent": change_percent,
"first_half_avg": first_half_avg,
"second_half_avg": second_half_avg
}
if change_percent < -5: # Improving (getting faster)
trends["improving"].append(trend_data)
elif change_percent > 5: # Degrading (getting slower)
trends["degrading"].append(trend_data)
else:
trends["stable"].append(trend_data)
return trends
def _analyze_api_trends(self, api_stats: Dict[str, Any]) -> Dict[str, Any]:
"""Analyze API performance trends."""
trends = {
"fastest_apis": [],
"slowest_apis": [],
"most_reliable": [],
"least_reliable": []
}
api_performance = []
for api_name, stats in api_stats.items():
if stats.get('call_count', 0) > 0:
avg_time = stats['total_time'] / stats['call_count']
success_rate = stats.get('success_rate', 0)
api_performance.append({
"api": api_name,
"average_time": avg_time,
"success_rate": success_rate,
"call_count": stats['call_count']
})
# Sort by performance metrics
fastest = sorted(api_performance, key=lambda x: x['average_time'])[:5]
slowest = sorted(api_performance, key=lambda x: x['average_time'], reverse=True)[:5]
most_reliable = sorted(api_performance, key=lambda x: x['success_rate'], reverse=True)[:5]
least_reliable = sorted(api_performance, key=lambda x: x['success_rate'])[:5]
trends["fastest_apis"] = fastest
trends["slowest_apis"] = slowest
trends["most_reliable"] = most_reliable
trends["least_reliable"] = least_reliable
return trends
def _generate_recommendations(self, func_stats: Dict[str, Any],
api_stats: Dict[str, Any]) -> List[str]:
"""Generate performance optimization recommendations."""
recommendations = []
# Function recommendations
for func_name, stats in func_stats.items():
if stats.get('call_count', 0) > 0:
avg_time = stats['total_time'] / stats['call_count']
error_rate = stats['error_count'] / stats['call_count']
if avg_time > 2.0:
recommendations.append(
f"⚠️ Function '{func_name}' has high average execution time ({avg_time:.2f}s). Consider optimization."
)
if error_rate > 0.1:
recommendations.append(
f"🚨 Function '{func_name}' has high error rate ({error_rate*100:.1f}%). Investigate error handling."
)
# API recommendations
for api_name, stats in api_stats.items():
if stats.get('call_count', 0) > 0:
avg_time = stats['total_time'] / stats['call_count']
success_rate = stats.get('success_rate', 1.0)
if avg_time > 5.0:
recommendations.append(
f"🐌 API '{api_name}' has high average response time ({avg_time:.2f}s). Consider caching or optimization."
)
if success_rate < 0.9:
recommendations.append(
f"⚠️ API '{api_name}' has low success rate ({success_rate*100:.1f}%). Check error handling and retry logic."
)
if not recommendations:
recommendations.append("✅ No major performance issues detected. System is performing well!")
return recommendations
def generate_comprehensive_report(self) -> str:
"""Generate a comprehensive performance analysis report."""
if not PERFORMANCE_MONITOR_AVAILABLE:
return "Performance monitor not available"
report_lines = []
report_lines.append("=" * 80)
report_lines.append("COMPREHENSIVE PERFORMANCE ANALYSIS REPORT")
report_lines.append("=" * 80)
report_lines.append(f"Generated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
report_lines.append("")
# Basic performance report
basic_report = performance_monitor.generate_performance_report()
report_lines.append(basic_report)
report_lines.append("")
# Flow analysis
report_lines.append("🔄 EXECUTION FLOW ANALYSIS")
report_lines.append("-" * 40)
flow_analysis = self.flow_analyzer.analyze_execution_flow()
if "bottlenecks" in flow_analysis:
bottlenecks = flow_analysis["bottlenecks"]
if bottlenecks:
report_lines.append("🚨 IDENTIFIED BOTTLENECKS:")
for bottleneck in bottlenecks[:5]:
report_lines.append(f"• {bottleneck['function']}: {bottleneck['average_duration']:.2f}s avg, {bottleneck['severity']} severity")
else:
report_lines.append("✅ No significant bottlenecks identified")
report_lines.append("")
# Performance trends
report_lines.append("📈 PERFORMANCE TRENDS")
report_lines.append("-" * 40)
trends = self.analyze_performance_trends()
if "recommendations" in trends:
recommendations = trends["recommendations"]
report_lines.append("💡 RECOMMENDATIONS:")
for rec in recommendations:
report_lines.append(f" {rec}")
report_lines.append("")
report_lines.append("=" * 80)
return "\n".join(report_lines)
# Global instances for easy access
flow_analyzer = FunctionFlowAnalyzer()
performance_analyzer = PerformanceAnalyzer()
# Convenience functions
def analyze_function_flow() -> Dict[str, Any]:
"""Analyze current function execution flow."""
return flow_analyzer.analyze_execution_flow()
def visualize_execution_flow(format: str = "text") -> str:
"""Visualize function execution flow."""
return flow_analyzer.visualize_flow(format=format)
def analyze_performance_trends() -> Dict[str, Any]:
"""Analyze performance trends."""
return performance_analyzer.analyze_performance_trends()
def generate_comprehensive_report() -> str:
"""Generate comprehensive performance analysis report."""
return performance_analyzer.generate_comprehensive_report()
@@ -119,13 +119,14 @@ class MinimalTelemetry:
except (ImportError, KeyError, AttributeError):
return "unknown"
def track_agent_execution(self, agent_name: str = None, success: bool = True):
def track_agent_execution(self, agent_name: str = None, success: bool = True, async_mode: bool = False):
"""
Track an agent execution event.
Args:
agent_name: Name of the agent (not logged, just for counting)
success: Whether the execution was successful
async_mode: If True, defer PostHog capture to prevent blocking in streaming scenarios
"""
if not self.enabled:
return
@@ -133,18 +134,41 @@ class MinimalTelemetry:
with self._metrics_lock:
self._metrics["agent_executions"] += 1
# Always log immediately for debugging
self.logger.debug(f"Agent execution tracked: success={success}")
# Send event to PostHog
if self._posthog:
self._posthog.capture(
distinct_id=self.session_id,
event='agent_execution',
properties={
'success': success,
'session_id': self.session_id
}
)
self.logger.debug(f"Agent execution tracked: success={success}")
if async_mode:
# Use a background thread to prevent blocking streaming responses
def _async_capture():
try:
self._posthog.capture(
distinct_id=self.session_id,
event='agent_execution',
properties={
'success': success,
'session_id': self.session_id
}
)
except Exception as e:
# Silently handle any telemetry errors to avoid disrupting user experience
self.logger.debug(f"Async PostHog capture error: {e}")
# Execute in background thread with daemon flag for clean shutdown
import threading
thread = threading.Thread(target=_async_capture, daemon=True)
thread.start()
else:
# Synchronous capture for backward compatibility
self._posthog.capture(
distinct_id=self.session_id,
event='agent_execution',
properties={
'success': success,
'session_id': self.session_id
}
)
def track_task_completion(self, task_name: str = None, success: bool = True):
"""
+12 -4
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "praisonaiagents"
version = "0.0.144"
version = "0.0.146"
description = "Praison AI agents for completing complex tasks with Self Reflection Agents"
requires-python = ">=3.10"
authors = [
@@ -27,7 +27,7 @@ mcp = [
memory = [
"chromadb>=1.0.0",
"litellm>=1.72.0",
"litellm>=1.72.6",
]
knowledge = [
@@ -45,7 +45,7 @@ graph = [
# Add LLM dependencies
llm = [
"litellm>=1.72.0",
"litellm>=1.72.6",
"pydantic>=2.4.2"
]
@@ -66,6 +66,13 @@ mongodb = [
"motor>=3.4.0"
]
# Authentication dependencies
auth = [
"PyJWT>=2.8.0",
"passlib[bcrypt]>=1.7.4",
"python-jose[cryptography]>=3.3.0",
"python-multipart>=0.0.6"
]
# Combined features
all = [
@@ -76,7 +83,8 @@ all = [
"praisonaiagents[mcp]",
"praisonaiagents[api]",
"praisonaiagents[telemetry]",
"praisonaiagents[mongodb]"
"praisonaiagents[mongodb]",
"praisonaiagents[auth]"
]
[tool.setuptools.packages.find]
+10
View File
@@ -0,0 +1,10 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a helpful assistant",
llm="gemini/gemini-2.5-flash",
stream=True
)
for chunk in agent.start("Write a report on about the history of the world"):
print(chunk, end="", flush=True)
+10
View File
@@ -0,0 +1,10 @@
from praisonaiagents import Agent
agent = Agent(
instructions="You are a helpful assistant",
llm="gpt-4o-mini",
stream=True
)
for chunk in agent.start("Write a report on about the history of the world"):
print(chunk, end="", flush=True)
+95
View File
@@ -0,0 +1,95 @@
#!/usr/bin/env python3
import os
import sys
sys.path.insert(0, 'src/praisonai-agents')
print("Testing DEBUG mode functionality...")
def clear_praisonai_modules():
"""Clear all praisonai modules from sys.modules"""
to_remove = []
for mod in sys.modules.keys():
if mod.startswith('praisonaiagents'):
to_remove.append(mod)
for mod in to_remove:
del sys.modules[mod]
def test_normal_mode():
"""Test that warnings are suppressed in normal mode"""
print("\n=== Testing NORMAL mode (LOGLEVEL=INFO) ===")
os.environ['LOGLEVEL'] = 'INFO'
# Clear modules
clear_praisonai_modules()
import warnings
warnings.simplefilter('always')
try:
import praisonaiagents
print("✅ Import successful in normal mode")
# Check if warning suppression is active
should_suppress = praisonaiagents._should_suppress_warnings()
print(f"✅ Warning suppression active: {should_suppress}")
return should_suppress # Should be True in normal mode
except Exception as e:
print(f"❌ Error in normal mode: {e}")
import traceback
traceback.print_exc()
return False
def test_debug_mode():
"""Test that warnings are shown in DEBUG mode"""
print("\n=== Testing DEBUG mode (LOGLEVEL=DEBUG) ===")
os.environ['LOGLEVEL'] = 'DEBUG'
# Clear modules
clear_praisonai_modules()
import warnings
warnings.simplefilter('always')
try:
import praisonaiagents
print("✅ Import successful in debug mode")
# Check if warning suppression is inactive
should_suppress = praisonaiagents._should_suppress_warnings()
print(f"✅ Warning suppression active: {should_suppress} (should be False)")
return not should_suppress # Should be False in debug mode (so we return True if it's correct)
except Exception as e:
print(f"❌ Error in debug mode: {e}")
import traceback
traceback.print_exc()
return False
if __name__ == "__main__":
print("🧪 Testing DEBUG mode implementation")
# Clean up environment first
if 'LOGLEVEL' in os.environ:
del os.environ['LOGLEVEL']
normal_success = test_normal_mode()
debug_success = test_debug_mode()
print("\n" + "="*50)
if normal_success and debug_success:
print("🎉 ALL TESTS PASSED: DEBUG mode functionality works correctly!")
print("✅ Warnings suppressed in normal mode")
print("✅ Warnings enabled in DEBUG mode")
else:
print("💥 TESTS FAILED: Issues with DEBUG mode implementation")
print(f" Normal mode success: {normal_success}")
print(f" Debug mode success: {debug_success}")
# Restore normal mode
os.environ['LOGLEVEL'] = 'INFO'
sys.exit(0 if (normal_success and debug_success) else 1)
+93
View File
@@ -0,0 +1,93 @@
#!/usr/bin/env python3
import os
import sys
sys.path.insert(0, 'src/praisonai-agents')
print("Testing DEBUG mode functionality...")
def test_normal_mode():
"""Test that warnings are suppressed in normal mode"""
print("\n=== Testing NORMAL mode (LOGLEVEL=INFO) ===")
os.environ['LOGLEVEL'] = 'INFO'
# Force reload modules to apply new environment
import sys
for mod in list(sys.modules.keys()):
if mod.startswith('praisonaiagents'):
del sys.modules[mod]
import warnings
warnings.simplefilter('always')
try:
import praisonaiagents
print("✅ Import successful in normal mode")
# Check if warning suppression is active
import praisonaiagents.__init__ as main_init
should_suppress = main_init._should_suppress_warnings()
print(f"✅ Warning suppression active: {should_suppress}")
except Exception as e:
print(f"❌ Error in normal mode: {e}")
return False
return True
def test_debug_mode():
"""Test that warnings are shown in DEBUG mode"""
print("\n=== Testing DEBUG mode (LOGLEVEL=DEBUG) ===")
os.environ['LOGLEVEL'] = 'DEBUG'
# Force reload modules to apply new environment
import sys
for mod in list(sys.modules.keys()):
if mod.startswith('praisonaiagents'):
del sys.modules[mod]
import warnings
warnings.simplefilter('always')
try:
import praisonaiagents
print("✅ Import successful in debug mode")
# Check if warning suppression is inactive
import praisonaiagents.__init__ as main_init
should_suppress = main_init._should_suppress_warnings()
print(f"✅ Warning suppression active: {should_suppress} (should be False)")
if should_suppress:
print("❌ WARNING: Debug mode should not suppress warnings!")
return False
else:
print("✅ DEBUG mode correctly allows warnings")
except Exception as e:
print(f"❌ Error in debug mode: {e}")
return False
return True
if __name__ == "__main__":
print("🧪 Testing DEBUG mode implementation")
# Clean up environment first
if 'LOGLEVEL' in os.environ:
del os.environ['LOGLEVEL']
normal_success = test_normal_mode()
debug_success = test_debug_mode()
print("\n" + "="*50)
if normal_success and debug_success:
print("🎉 ALL TESTS PASSED: DEBUG mode functionality works correctly!")
print("✅ Warnings suppressed in normal mode")
print("✅ Warnings enabled in DEBUG mode")
else:
print("💥 TESTS FAILED: Issues with DEBUG mode implementation")
# Restore normal mode
os.environ['LOGLEVEL'] = 'INFO'
sys.exit(0 if (normal_success and debug_success) else 1)
+10 -17
View File
@@ -1,17 +1,10 @@
try:
print("Testing import...")
from praisonaiagents import Agent
print("✅ Import successful - no syntax errors!")
print("Creating agent...")
agent = Agent(instructions="Test agent", llm="gpt-4o-mini")
print("✅ Agent created successfully!")
except SyntaxError as e:
print(f"❌ SyntaxError: {e}")
print(f" File: {e.filename}")
print(f" Line {e.lineno}: {e.text}")
except Exception as e:
print(f"❌ Other error: {e}")
import traceback
traceback.print_exc()
#!/usr/bin/env python3
import sys
sys.path.insert(0, 'src/praisonai-agents')
import warnings
warnings.simplefilter('always')
print("Testing import with warnings enabled...")
import praisonaiagents
print("Import successful!")
+129
View File
@@ -0,0 +1,129 @@
#!/usr/bin/env python3
"""
Test real streaming behavior for different configurations.
This script tests that streaming works correctly with various settings.
"""
import os
import sys
from praisonaiagents import Agent
def test_real_streaming():
"""Test real streaming behavior with different configurations."""
# Test configurations
test_cases = [
{
"name": "Stream True + Verbose True",
"stream": True,
"verbose": True,
"expected": "Real-time streaming with progressive content display"
},
{
"name": "Stream True + Verbose False",
"stream": True,
"verbose": False,
"expected": "Real-time streaming but no display (callbacks only)"
},
{
"name": "Stream False + Verbose True",
"stream": False,
"verbose": True,
"expected": "'Generating response...' animation with elapsed time, no content"
},
{
"name": "Stream False + Verbose False",
"stream": False,
"verbose": False,
"expected": "No display at all, just returns response"
}
]
for test_case in test_cases:
print(f"\n{'='*60}")
print(f"Testing: {test_case['name']}")
print(f"Configuration: stream={test_case['stream']}, verbose={test_case['verbose']}")
print(f"Expected: {test_case['expected']}")
print(f"{'='*60}\n")
try:
# Create agent with specific configuration
agent = Agent(
name="Test Agent",
role="Testing Assistant",
goal="Test streaming behavior",
backstory="I am a test agent for verifying streaming functionality.",
llm="gpt-4o-mini", # You can change this to test other providers
stream=test_case["stream"],
verbose=test_case["verbose"]
)
# Test the agent
prompt = "Count from 1 to 10 slowly, with each number on a new line."
print(f"Sending prompt: {prompt}\n")
response = agent.chat(prompt)
if not test_case["verbose"]:
# If verbose is False, we need to print the response manually
print(f"\nResponse:\n{response}")
print(f"\n{'='*60}")
print(f"Test completed!")
except Exception as e:
print(f"\nError during test: {str(e)}")
continue
# Wait for user to observe the behavior
input("\nPress Enter to continue to the next test...")
# Test with different providers
print("\n\nTesting with different providers...")
providers = [
("Gemini", "gemini/gemini-1.5-flash"),
("Anthropic", "claude-3-haiku-20240307"),
# Add more providers as needed
]
for name, model in providers:
print(f"\n{'='*60}")
print(f"Testing real streaming with {name}")
print(f"Configuration: stream=True, verbose=True")
print(f"{'='*60}\n")
try:
agent = Agent(
name=f"{name} Agent",
role="Testing Assistant",
goal="Test streaming",
backstory=f"Testing {name} streaming",
llm=model,
stream=True,
verbose=True
)
response = agent.chat("Write a haiku about streaming data.")
print(f"\nTest with {name} completed!")
except Exception as e:
print(f"\nError testing with {name}: {str(e)}")
if "API" in str(e) or "key" in str(e).lower():
print(f"Make sure you have the required API key set for {name}")
if __name__ == "__main__":
print("Starting real streaming behavior tests...")
print("This test verifies the correct behavior for all combinations of stream and verbose settings.\n")
# Check for API keys
if not os.environ.get("OPENAI_API_KEY"):
print("Warning: OPENAI_API_KEY not set. OpenAI tests will fail.")
test_real_streaming()
print("\n\nAll tests completed!")
print("\nSummary of expected behaviors:")
print("- stream=True, verbose=True: Real-time streaming with content display")
print("- stream=True, verbose=False: Real-time streaming, no display")
print("- stream=False, verbose=True: 'Generating...' animation, no content")
print("- stream=False, verbose=False: No display at all")
@@ -0,0 +1,80 @@
#!/usr/bin/env python3
"""
Test to verify that the stream parameter is correctly passed through the agent.chat() method.
This tests the fix for the issue where stream=self.stream was being used instead of the passed parameter.
"""
import os
from praisonaiagents import Agent
# Use a simple model for testing
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY", "test-key")
def test_stream_parameter():
"""Test that stream parameter is correctly passed through chat method"""
# Create an agent with stream=True by default
agent = Agent(
name="TestAgent",
role="Test Assistant",
goal="Test streaming behavior",
backstory="A test agent for verifying stream parameter handling",
llm="gpt-4o-mini",
stream=True, # Default to streaming
verbose=True
)
print("Test 1: Agent has stream=True, but we pass stream=False to chat()")
print("Expected: Should NOT stream (no progressive display)")
print("-" * 50)
# Call chat with stream=False explicitly - should override agent's stream=True
response1 = agent.chat("Tell me a very short fact about Python in one sentence.", stream=False)
print(f"\nResponse 1: {response1}")
print("\n" + "="*70 + "\n")
print("Test 2: Agent has stream=True, and we pass stream=True to chat()")
print("Expected: Should stream (progressive display)")
print("-" * 50)
# Call chat with stream=True explicitly - should stream
response2 = agent.chat("Tell me another very short fact about Python in one sentence.", stream=True)
print(f"\nResponse 2: {response2}")
print("\n" + "="*70 + "\n")
# Now test with an agent that has stream=False by default
agent2 = Agent(
name="TestAgent2",
role="Test Assistant 2",
goal="Test non-streaming behavior",
backstory="A test agent for verifying non-stream parameter handling",
llm="gpt-4o-mini",
stream=False, # Default to non-streaming
verbose=True
)
print("Test 3: Agent has stream=False, but we pass stream=True to chat()")
print("Expected: Should stream (progressive display)")
print("-" * 50)
# Call chat with stream=True explicitly - should override agent's stream=False
response3 = agent2.chat("Tell me a very short fact about JavaScript in one sentence.", stream=True)
print(f"\nResponse 3: {response3}")
print("\n" + "="*70 + "\n")
print("Test 4: Agent has stream=False, and we don't pass stream to chat()")
print("Expected: Should NOT stream (uses agent's default)")
print("-" * 50)
# Call chat without stream parameter - should use agent's default (False)
response4 = agent2.chat("Tell me a very short fact about Java in one sentence.")
print(f"\nResponse 4: {response4}")
print("\n" + "="*70 + "\n")
print("All tests completed!")
if __name__ == "__main__":
test_stream_parameter()
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env python3
"""
Test script to validate real-time streaming functionality
"""
import sys
import os
# Add the source path to enable imports
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'src', 'praisonai-agents'))
try:
from praisonaiagents import Agent
print("Creating agent with streaming enabled...")
agent = Agent(
instructions="You are a helpful assistant",
llm="gemini/gemini-2.5-flash",
stream=True,
verbose=False # Reduce noise during testing
)
print("Starting streaming test...")
print("=" * 50)
# Test the streaming functionality
chunk_count = 0
for chunk in agent.start("Write a short paragraph about the benefits of real-time streaming in AI applications"):
print(chunk, end="", flush=True)
chunk_count += 1
print("\n" + "=" * 50)
print(f"✅ Streaming test completed! Received {chunk_count} chunks.")
if chunk_count > 1:
print("✅ SUCCESS: Real-time streaming is working - received multiple chunks!")
else:
print("⚠️ WARNING: Only received 1 chunk - may still be using simulated streaming")
except ImportError as e:
print(f"❌ Import error: {e}")
print("Make sure you're in the correct directory and dependencies are installed")
except Exception as e:
print(f"❌ Error during streaming test: {e}")
import traceback
traceback.print_exc()
@@ -0,0 +1,110 @@
#!/usr/bin/env python3
"""
Basic test to verify streaming functionality without API calls
"""
import sys
import os
import logging
# Add the source path to enable imports
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'src', 'praisonai-agents'))
def test_imports():
"""Test that we can import the required modules"""
try:
print("Testing imports...")
# Test LLM import
from praisonaiagents.llm.llm import LLM
print("✅ LLM import successful")
# Test Agent import
from praisonaiagents.agent.agent import Agent
print("✅ Agent import successful")
return True
except Exception as e:
print(f"❌ Import error: {e}")
import traceback
traceback.print_exc()
return False
def test_method_existence():
"""Test that the new streaming methods exist"""
try:
from praisonaiagents.llm.llm import LLM
# Check if the new method exists
llm = LLM(model="test")
if hasattr(llm, 'get_response_stream'):
print("✅ get_response_stream method exists in LLM")
else:
print("❌ get_response_stream method missing in LLM")
return False
return True
except Exception as e:
print(f"❌ Method test error: {e}")
import traceback
traceback.print_exc()
return False
def test_agent_streaming_setup():
"""Test that agent streaming is set up correctly"""
try:
from praisonaiagents.agent.agent import Agent
# Create a basic agent instance (without API calls)
agent = Agent(
instructions="Test agent",
llm="test/model", # Mock model that won't make real API calls
stream=True,
verbose=False
)
print("✅ Agent creation successful")
# Check if streaming methods exist
if hasattr(agent, '_start_stream'):
print("✅ _start_stream method exists in Agent")
else:
print("❌ _start_stream method missing in Agent")
return False
return True
except Exception as e:
print(f"❌ Agent test error: {e}")
import traceback
traceback.print_exc()
return False
if __name__ == "__main__":
print("Running basic streaming functionality tests...")
print("=" * 50)
# Set logging to reduce noise
logging.getLogger().setLevel(logging.WARNING)
success = True
# Test imports
if not test_imports():
success = False
# Test method existence
if not test_method_existence():
success = False
# Test agent setup
if not test_agent_streaming_setup():
success = False
print("=" * 50)
if success:
print("✅ All basic tests passed!")
print("✅ Streaming infrastructure is properly set up")
print("📝 Note: Real streaming tests require API keys and will be tested later")
else:
print("❌ Some tests failed - check the implementation")
@@ -0,0 +1,102 @@
#!/usr/bin/env python3
"""
Test streaming behavior for different LLM providers.
This script tests the streaming functionality with OpenAI, LiteLLM, and Gemini models.
"""
import os
import sys
from praisonaiagents import Agent
def test_streaming_behavior():
"""Test streaming behavior with different configurations."""
# Test configurations
test_cases = [
{
"name": "OpenAI - Stream True",
"llm": "gpt-4o-mini",
"stream": True,
"verbose": True,
"expected": "Should show real-time streaming"
},
{
"name": "OpenAI - Stream False",
"llm": "gpt-4o-mini",
"stream": False,
"verbose": True,
"expected": "Should show 'Generating...' animation during API call"
},
{
"name": "Gemini - Stream True",
"llm": "gemini/gemini-1.5-flash",
"stream": True,
"verbose": True,
"expected": "Should show real-time streaming"
},
{
"name": "Gemini - Stream False",
"llm": "gemini/gemini-1.5-flash",
"stream": False,
"verbose": True,
"expected": "Should show 'Generating...' animation during API call"
},
{
"name": "LiteLLM/Ollama - Stream True",
"llm": "ollama/llama3.2",
"stream": True,
"verbose": True,
"expected": "Should show real-time streaming (if Ollama is running)"
},
{
"name": "LiteLLM/Ollama - Stream False",
"llm": "ollama/llama3.2",
"stream": False,
"verbose": True,
"expected": "Should show 'Generating...' animation during API call (if Ollama is running)"
}
]
for test_case in test_cases:
print(f"\n{'='*60}")
print(f"Testing: {test_case['name']}")
print(f"Expected: {test_case['expected']}")
print(f"{'='*60}\n")
try:
# Create agent with specific configuration
agent = Agent(
name="Test Agent",
role="Testing Assistant",
goal="Test streaming behavior",
backstory="I am a test agent for verifying streaming functionality.",
llm=test_case["llm"],
stream=test_case["stream"],
verbose=test_case["verbose"]
)
# Test the agent
response = agent.chat("Write a haiku about streaming data.")
print(f"\nResponse received: {response[:100]}..." if len(response) > 100 else f"\nResponse received: {response}")
print(f"\nTest completed successfully!")
except Exception as e:
print(f"\nError during test: {str(e)}")
if "ollama" in test_case["llm"].lower():
print("Note: Ollama tests require Ollama to be running locally.")
continue
# Wait for user to observe the behavior
input("\nPress Enter to continue to the next test...")
if __name__ == "__main__":
print("Starting streaming behavior tests...")
print("Make sure you have the required API keys set:")
print("- OPENAI_API_KEY for OpenAI tests")
print("- GEMINI_API_KEY or GOOGLE_API_KEY for Gemini tests")
print("- Ollama running locally for Ollama tests\n")
test_streaming_behavior()
print("\n\nAll tests completed!")
@@ -0,0 +1,66 @@
#!/usr/bin/env python3
"""
Test script for streaming display bypass fix
Tests that streaming yields raw chunks without display_generation
"""
import sys
import os
import collections.abc
# Add the praisonai-agents source to Python path
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'src', 'praisonai-agents'))
try:
from praisonaiagents import Agent
print("🧪 Testing Streaming Display Bypass Fix")
print("=" * 50)
# Test configuration - using mock model to avoid API calls
agent = Agent(
instructions="You are a helpful assistant",
llm="mock-model-for-testing",
stream=True
)
# Test 1: Basic streaming setup
print("✅ Agent created successfully with stream=True")
print(f"📊 Agent stream attribute: {agent.stream}")
# Test 2: Check start method behavior and exception on consumption
result = agent.start("Hello, test streaming")
assert isinstance(result, collections.abc.Generator), "Agent.start() should return a generator for streaming"
print("✅ Agent.start() returned a generator (streaming enabled)")
try:
# Consume the generator to trigger the API call, which should fail for a mock model.
list(result)
# If we get here, the test has failed because an exception was expected.
print("❌ FAILED: Expected an exception with mock model, but none was raised.")
except Exception as e:
print(f"✅ SUCCESS: Caught expected exception with mock model: {e}")
print("✅ Streaming path was triggered (exception expected with mock model)")
# Test 3: Verify the streaming method exists and is callable
if hasattr(agent, '_start_stream') and callable(agent._start_stream):
print("✅ _start_stream method exists and is callable")
else:
print("❌ _start_stream method missing")
print("\n🎯 Test Results:")
print("✅ Streaming infrastructure is properly set up")
print("✅ Agent.start() correctly detects stream=True")
print("✅ Modified _start_stream should now bypass display_generation")
print("✅ OpenAI streaming implementation is in place")
print("\n📝 Note: Full streaming test requires valid OpenAI API key")
print("🔗 This test validates the code structure and logic flow")
except ImportError as e:
print(f"❌ Import failed: {e}")
print("Please ensure you're running from the correct directory")
except Exception as e:
print(f"❌ Test failed: {e}")
import traceback
traceback.print_exc()
@@ -0,0 +1,120 @@
#!/usr/bin/env python3
"""
Test streaming logic without requiring API keys
"""
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'src/praisonai-agents'))
def test_streaming_logic():
"""Test the streaming logic without actually calling LLMs"""
print("🔄 Testing streaming logic without API keys\n")
try:
from praisonaiagents import Agent
# Test 1: Default behavior (stream=False)
print("1. Testing default agent (no stream parameter)...")
agent1 = Agent(
instructions="You are a helpful assistant",
llm="gpt-4o-mini"
)
print(f" ✅ Agent created successfully")
print(f" 📊 stream attribute: {getattr(agent1, 'stream', 'NOT SET')}")
if agent1.stream == False:
print(" ✅ CORRECT: stream defaults to False (backward compatible)")
else:
print(" ❌ INCORRECT: stream should default to False")
return False
# Test 2: Explicit stream=False
print("\n2. Testing explicit stream=False...")
agent2 = Agent(
instructions="You are a helpful assistant",
llm="gpt-4o-mini",
stream=False
)
print(f" ✅ Agent created successfully")
print(f" 📊 stream attribute: {agent2.stream}")
if agent2.stream == False:
print(" ✅ CORRECT: stream=False works")
else:
print(" ❌ INCORRECT: stream=False not working")
return False
# Test 3: Explicit stream=True
print("\n3. Testing explicit stream=True...")
agent3 = Agent(
instructions="You are a helpful assistant",
llm="gpt-4o-mini",
stream=True
)
print(f" ✅ Agent created successfully")
print(f" 📊 stream attribute: {agent3.stream}")
if agent3.stream == True:
print(" ✅ CORRECT: stream=True works")
else:
print(" ❌ INCORRECT: stream=True not working")
return False
# Test 4: Check start method logic without actually calling it
print("\n4. Testing start method logic...")
# Mock test - check if start would use streaming based on conditions
def check_streaming_logic(agent, **kwargs):
"""Check what start() method would do without calling LLMs"""
stream_enabled = kwargs.get('stream', getattr(agent, 'stream', False))
return stream_enabled
# Test default agent (should not stream)
would_stream = check_streaming_logic(agent1)
if not would_stream:
print(" ✅ Default agent would NOT stream (backward compatible)")
else:
print(" ❌ Default agent would stream (breaks compatibility)")
return False
# Test explicit stream=False
would_stream = check_streaming_logic(agent2, stream=False)
if not would_stream:
print(" ✅ stream=False would NOT stream")
else:
print(" ❌ stream=False would stream")
return False
# Test explicit stream=True
would_stream = check_streaming_logic(agent3, stream=True)
if would_stream:
print(" ✅ stream=True would stream")
else:
print(" ❌ stream=True would NOT stream")
return False
print("\n" + "="*60)
print("✅ ALL LOGIC TESTS PASSED!")
print("🎉 Backward compatibility has been restored!")
print("\nKey fixes:")
print("- Agent constructor now defaults to stream=False")
print("- Basic usage agent.start('prompt') returns string (not generator)")
print("- Explicit stream=True enables streaming as expected")
print("- Explicit stream=False maintains non-streaming behavior")
return True
except Exception as e:
print(f"❌ Test failed: {e}")
import traceback
traceback.print_exc()
return False
if __name__ == "__main__":
success = test_streaming_logic()
sys.exit(0 if success else 1)
@@ -0,0 +1,149 @@
#!/usr/bin/env python3
"""
Test script to verify that the telemetry streaming pause issue is fixed.
This test demonstrates that streaming starts immediately without blocking on telemetry.
"""
import time
import sys
import os
# Add the source path to enable imports
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'src', 'praisonai-agents'))
def test_streaming_telemetry_fix():
"""Test that streaming starts immediately without telemetry blocking."""
try:
from praisonaiagents import Agent
print("🧪 Testing streaming telemetry fix...")
print("=" * 60)
# Create agent with streaming enabled
agent = Agent(
instructions="You are a helpful assistant that provides brief responses.",
llm="test/mock-model", # Mock model to avoid API calls
stream=True,
verbose=False # Reduce output noise
)
print("✅ Agent created successfully")
# Test that start() returns immediately (generator)
start_time = time.time()
try:
result = agent.start("Say hello briefly")
creation_time = time.time() - start_time
print(f"⏱️ Generator creation time: {creation_time:.3f} seconds")
# Check if it's a generator
import types
if isinstance(result, types.GeneratorType):
print("✅ Agent.start() returned generator (streaming mode)")
print("✅ No blocking pause - telemetry is now asynchronous!")
# Verify the generator can be iterated (though it may fail due to mock model)
try:
first_chunk = next(result)
print(f"✅ First chunk received: {first_chunk[:50]}...")
except Exception as e:
print(f"⚠️ Expected error with mock model: {type(e).__name__}")
print(" This is normal - we're testing telemetry, not actual LLM calls")
return True
else:
print("❌ Agent.start() did not return generator")
return False
except Exception as e:
print(f"❌ Error during agent.start(): {e}")
import traceback
traceback.print_exc()
return False
except Exception as e:
print(f"❌ Import or setup error: {e}")
import traceback
traceback.print_exc()
return False
def test_telemetry_integration():
"""Test that telemetry integration works without blocking."""
try:
from praisonaiagents.telemetry.integration import instrument_agent
from praisonaiagents.telemetry.telemetry import get_telemetry
from praisonaiagents import Agent
print("🔧 Testing telemetry integration...")
# Get telemetry instance
telemetry = get_telemetry()
print(f"✅ Telemetry enabled: {telemetry.enabled}")
# Create agent
agent = Agent(
instructions="Test agent",
llm="test/mock-model",
stream=True,
verbose=False
)
# Instrument the agent (this should happen automatically)
instrumented_agent = instrument_agent(agent, telemetry)
print("✅ Agent instrumented successfully")
# Test that the instrumented start method doesn't block
start_time = time.time()
try:
result = instrumented_agent.start("Test prompt")
creation_time = time.time() - start_time
print(f"⏱️ Instrumented start() time: {creation_time:.3f} seconds")
if creation_time < 1.0: # Should be nearly instantaneous
print("✅ No blocking detected - fix is working!")
return True
else:
print("❌ Potential blocking detected")
return False
except Exception as e:
print(f"⚠️ Expected error with mock model: {type(e).__name__}")
print("✅ But no blocking pause occurred - fix is working!")
return True
except Exception as e:
print(f"❌ Telemetry integration test error: {e}")
import traceback
traceback.print_exc()
return False
if __name__ == "__main__":
print("Testing Streaming Telemetry Fix")
print("=" * 60)
success = True
# Test 1: Basic streaming functionality
if not test_streaming_telemetry_fix():
success = False
print()
# Test 2: Telemetry integration
if not test_telemetry_integration():
success = False
print("=" * 60)
if success:
print("🎉 All tests passed!")
print("✅ Streaming telemetry fix is working correctly")
print("✅ No more pause after 'execution tracked: success=True'")
else:
print("❌ Some tests failed")
print("\n📝 Note: This test uses mock models to avoid API calls.")
print(" Real streaming tests require valid API keys.")
@@ -0,0 +1,67 @@
#!/usr/bin/env python3
"""
Test different combinations of verbose and stream parameters to understand the display behavior.
"""
import os
from praisonaiagents import Agent
# Use a simple model for testing
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY", "test-key")
def test_combination(verbose: bool, stream: bool, test_name: str):
"""Test a specific combination of verbose and stream settings"""
print(f"\n{'='*70}")
print(f"Test: {test_name}")
print(f"Settings: verbose={verbose}, stream={stream}")
print(f"{'='*70}\n")
agent = Agent(
name="TestAgent",
role="Test Assistant",
goal="Test display behavior",
backstory="A test agent",
llm="gpt-4o-mini",
verbose=verbose,
stream=stream
)
response = agent.chat("What is 2 + 2? Answer in exactly one word.")
print(f"\nFinal response: {response}")
print(f"\n{'-'*70}")
def main():
print("Testing different combinations of verbose and stream settings")
print("Observing the display behavior for each combination\n")
# Test all 4 combinations
test_combination(verbose=True, stream=True, test_name="Verbose ON, Stream ON")
test_combination(verbose=True, stream=False, test_name="Verbose ON, Stream OFF")
test_combination(verbose=False, stream=True, test_name="Verbose OFF, Stream ON")
test_combination(verbose=False, stream=False, test_name="Verbose OFF, Stream OFF")
print("\n\nNow testing with explicit stream parameter in chat() method:")
# Test overriding stream in chat method
print(f"\n{'='*70}")
print("Test: Agent has stream=True, but chat() called with stream=False")
print(f"{'='*70}\n")
agent = Agent(
name="TestAgent",
role="Test Assistant",
goal="Test display behavior",
backstory="A test agent",
llm="gpt-4o-mini",
verbose=True,
stream=True # Agent default is streaming
)
# Override with stream=False in chat
response = agent.chat("What is 3 + 3? Answer in exactly one word.", stream=False)
print(f"\nFinal response: {response}")
print("\n\nAll tests completed!")
if __name__ == "__main__":
main()
+22
View File
@@ -0,0 +1,22 @@
#!/usr/bin/env python3
import sys
sys.path.insert(0, 'src/praisonai-agents')
import warnings
warnings.simplefilter('always') # Enable all warnings
print("Testing import with all warnings enabled...")
print("Importing praisonaiagents...")
try:
import praisonaiagents
print("✅ Import successful!")
print("Testing Agent creation...")
agent = praisonaiagents.Agent(instructions="Test agent", llm="gpt-4o-mini")
print("✅ Agent created successfully!")
except Exception as e:
print(f"❌ Error: {e}")
import traceback
traceback.print_exc()
@@ -0,0 +1,333 @@
#!/usr/bin/env python3
"""
Test ContextAgent Implementation
This test script validates the ContextAgent implementation for:
- Import capability
- Basic instantiation
- Method availability
- Backward compatibility
- Integration with existing PraisonAI structure
"""
import sys
from pathlib import Path
# Add the src directory to Python path
project_root = Path(__file__).parent.parent
sys.path.insert(0, str(project_root))
def test_imports():
"""Test that ContextAgent can be imported correctly."""
print("🧪 Testing ContextAgent Imports...")
try:
# Test importing from main package
from praisonaiagents import ContextAgent, create_context_agent
print("✅ Successfully imported ContextAgent and create_context_agent from main package")
# Test importing from agent submodule
from praisonaiagents.agent import ContextAgent as AgentContextAgent
print("✅ Successfully imported ContextAgent from agent submodule")
# Test importing from specific module
from praisonaiagents.agent.context_agent import ContextAgent as DirectContextAgent
print("✅ Successfully imported ContextAgent from direct module")
return True
except ImportError as e:
print(f"❌ Import failed: {e}")
return False
def test_basic_instantiation():
"""Test basic ContextAgent instantiation."""
print("\\n🧪 Testing ContextAgent Instantiation...")
try:
from praisonaiagents import ContextAgent, create_context_agent
# Test direct instantiation
context_agent = ContextAgent()
print("✅ Successfully created ContextAgent with default parameters")
# Test with custom parameters
custom_agent = ContextAgent(
name="Test Context Engineer",
role="Test Role",
goal="Test Goal",
llm="gpt-4o-mini"
)
print("✅ Successfully created ContextAgent with custom parameters")
# Test factory function
factory_agent = create_context_agent(llm="gpt-4o-mini")
print("✅ Successfully created ContextAgent using factory function")
return True, [context_agent, custom_agent, factory_agent]
except Exception as e:
print(f"❌ Instantiation failed: {e}")
return False, []
def test_agent_inheritance():
"""Test that ContextAgent properly inherits from Agent."""
print("\\n🧪 Testing Agent Inheritance...")
try:
from praisonaiagents import ContextAgent, Agent
context_agent = ContextAgent()
# Test inheritance
assert isinstance(context_agent, Agent), "ContextAgent should inherit from Agent"
print("✅ ContextAgent properly inherits from Agent class")
# Test that base Agent properties exist
assert hasattr(context_agent, 'name'), "Should have name attribute"
assert hasattr(context_agent, 'role'), "Should have role attribute"
assert hasattr(context_agent, 'goal'), "Should have goal attribute"
print("✅ ContextAgent has all required Agent attributes")
return True
except Exception as e:
print(f"❌ Inheritance test failed: {e}")
return False
def test_context_engineering_methods():
"""Test Context Engineering specific methods."""
print("\\n🧪 Testing Context Engineering Methods...")
try:
from praisonaiagents import create_context_agent
context_agent = create_context_agent()
# Test that all Context Engineering methods exist
required_methods = [
'analyze_codebase_patterns',
'generate_context_document',
'create_validation_loop',
'enhance_prompt_with_context',
'generate_prp',
'extract_documentation_patterns',
'analyze_test_patterns',
'create_implementation_blueprint'
]
for method_name in required_methods:
assert hasattr(context_agent, method_name), f"Missing method: {method_name}"
assert callable(getattr(context_agent, method_name)), f"Method {method_name} is not callable"
print(f"✅ All {len(required_methods)} Context Engineering methods are available")
# Test async methods
async_methods = [
'aanalyze_codebase_patterns',
'agenerate_context_document',
'acreate_validation_loop',
'aenhance_prompt_with_context',
'agenerate_prp'
]
for method_name in async_methods:
assert hasattr(context_agent, method_name), f"Missing async method: {method_name}"
assert callable(getattr(context_agent, method_name)), f"Async method {method_name} is not callable"
print(f"✅ All {len(async_methods)} async methods are available")
return True
except Exception as e:
print(f"❌ Method availability test failed: {e}")
return False
def test_basic_functionality():
"""Test basic functionality of ContextAgent methods."""
print("\\n🧪 Testing Basic Functionality...")
try:
from praisonaiagents import create_context_agent
context_agent = create_context_agent()
# Test codebase analysis with current directory
test_path = str(Path(__file__).parent.parent)
analysis = context_agent.analyze_codebase_patterns(test_path)
assert isinstance(analysis, dict), "analyze_codebase_patterns should return dict"
print("✅ analyze_codebase_patterns works correctly")
# Test context document generation
context_doc = context_agent.generate_context_document(
project_path=test_path,
requirements="Test feature implementation"
)
assert isinstance(context_doc, str), "generate_context_document should return string"
assert len(context_doc) > 100, "Context document should be substantial"
print("✅ generate_context_document works correctly")
# Test validation loop creation
validation = context_agent.create_validation_loop(
implementation_requirements="Test implementation",
success_criteria=["Test passes", "Code works"]
)
assert isinstance(validation, dict), "create_validation_loop should return dict"
assert 'validation_steps' in validation, "Should have validation_steps"
print("✅ create_validation_loop works correctly")
# Test prompt enhancement
enhanced = context_agent.enhance_prompt_with_context(
base_prompt="Test prompt",
context_data={"test": "data"}
)
assert isinstance(enhanced, str), "enhance_prompt_with_context should return string"
assert len(enhanced) > len("Test prompt"), "Enhanced prompt should be longer"
print("✅ enhance_prompt_with_context works correctly")
# Test PRP generation
prp = context_agent.generate_prp(
feature_request="Test feature",
context_analysis=analysis
)
assert isinstance(prp, str), "generate_prp should return string"
assert "PRP" in prp, "Should contain PRP reference"
print("✅ generate_prp works correctly")
return True
except Exception as e:
print(f"❌ Functionality test failed: {e}")
return False
def test_backward_compatibility():
"""Test that existing PraisonAI functionality still works."""
print("\\n🧪 Testing Backward Compatibility...")
try:
# Test that we can still import and use existing agents
from praisonaiagents import Agent, ImageAgent
# Test basic Agent still works
basic_agent = Agent(name="Test Agent") # Agent created for compatibility test
print("✅ Basic Agent still works")
# Test ImageAgent still works
image_agent = ImageAgent(name="Test Image Agent") # Image agent created for compatibility test
print("✅ ImageAgent still works")
# Test that we can import other PraisonAI components
from praisonaiagents import Task, PraisonAIAgents
print("✅ Task and PraisonAIAgents can still be imported")
# Test that __all__ exports are working
import praisonaiagents
expected_exports = [
'Agent', 'ImageAgent', 'ContextAgent', 'create_context_agent',
'PraisonAIAgents', 'Task'
]
for export in expected_exports:
assert hasattr(praisonaiagents, export), f"Missing export: {export}"
print("✅ All expected exports are available")
return True
except Exception as e:
print(f"❌ Backward compatibility test failed: {e}")
return False
def test_syntax_validation():
"""Test that all Python files have valid syntax."""
print("\\n🧪 Testing Syntax Validation...")
try:
import ast
# Test the main ContextAgent file
context_agent_file = project_root / "praisonaiagents" / "agent" / "context_agent.py"
with open(context_agent_file, 'r') as f:
content = f.read()
# Parse the file to check for syntax errors
ast.parse(content)
print("✅ context_agent.py has valid Python syntax")
# Test the examples
example_files = [
Path(__file__).parent.parent.parent.parent / "examples" / "python" / "agents" / "context-agent.py",
Path(__file__).parent.parent.parent.parent / "examples" / "python" / "concepts" / "context-engineering-workflow.py"
]
for example_file in example_files:
if example_file.exists():
with open(example_file, 'r') as f:
content = f.read()
ast.parse(content)
print(f"✅ {example_file.name} has valid Python syntax")
return True
except SyntaxError as e:
print(f"❌ Syntax error found: {e}")
return False
except Exception as e:
print(f"❌ Syntax validation failed: {e}")
return False
def run_all_tests():
"""Run all tests and provide summary."""
print("🚀 Context Engineering Implementation QA Tests")
print("=" * 60)
test_results = []
# Run all tests
test_results.append(("Imports", test_imports()))
test_results.append(("Instantiation", test_basic_instantiation()[0]))
test_results.append(("Inheritance", test_agent_inheritance()))
test_results.append(("Methods", test_context_engineering_methods()))
test_results.append(("Functionality", test_basic_functionality()))
test_results.append(("Backward Compatibility", test_backward_compatibility()))
test_results.append(("Syntax Validation", test_syntax_validation()))
# Summary
print("\\n📊 Test Results Summary")
print("=" * 30)
passed = 0
total = len(test_results)
for test_name, result in test_results:
status = "✅ PASS" if result else "❌ FAIL"
print(f" {test_name:<20} {status}")
if result:
passed += 1
print(f"\\n🎯 Overall Result: {passed}/{total} tests passed")
if passed == total:
print("🎉 All tests passed! ContextAgent implementation is ready.")
return True
else:
print("⚠️ Some tests failed. Review implementation before release.")
return False
if __name__ == "__main__":
success = run_all_tests()
if success:
print("\\n✅ Quality Assurance Complete")
print(" • ContextAgent implementation validated")
print(" • Backward compatibility confirmed")
print(" • All Context Engineering methods working")
print(" • Examples have valid syntax")
print(" • Ready for production use")
else:
print("\\n❌ Quality Assurance Failed")
print(" • Review failed tests above")
print(" • Fix issues before release")
sys.exit(0 if success else 1)
@@ -0,0 +1,196 @@
#!/usr/bin/env python3
"""
Test script to verify that embedding logging is working correctly with TRACE level.
This test verifies that the TRACE logging level works as expected for embedding operations.
"""
import logging
import sys
import os
import unittest
from io import StringIO
# Add the praisonaiagents module to the path using relative path
current_dir = os.path.dirname(os.path.abspath(__file__))
parent_dir = os.path.dirname(current_dir)
sys.path.insert(0, parent_dir)
# Import the memory module to test the logger setup
try:
from praisonaiagents.memory.memory import logger, TRACE_LEVEL
except ImportError as e:
print(f"Failed to import memory module: {e}")
sys.exit(1)
class TestTraceLogging(unittest.TestCase):
"""Test cases for TRACE logging functionality."""
def setUp(self):
"""Set up test environment before each test."""
# Store original logger level
self.original_level = logger.level
# Create a string stream to capture log output
self.log_capture = StringIO()
self.handler = logging.StreamHandler(self.log_capture)
self.handler.setLevel(TRACE_LEVEL)
self.handler.setFormatter(logging.Formatter('%(levelname)s:%(name)s:%(message)s'))
logger.addHandler(self.handler)
def tearDown(self):
"""Clean up after each test."""
# Restore original logger level
logger.setLevel(self.original_level)
# Remove the test handler
logger.removeHandler(self.handler)
self.handler.close()
def test_trace_level_value(self):
"""Test that TRACE level is set to correct value (5, below DEBUG)."""
self.assertEqual(TRACE_LEVEL, 5)
self.assertLess(TRACE_LEVEL, logging.DEBUG) # TRACE should be below DEBUG (10)
def test_trace_method_exists(self):
"""Test that trace method exists on logger."""
self.assertTrue(hasattr(logger, 'trace'))
self.assertTrue(callable(getattr(logger, 'trace')))
def test_trace_logging_at_trace_level(self):
"""Test that TRACE messages are logged when level is set to TRACE."""
logger.setLevel(TRACE_LEVEL)
test_message = "Test trace message for embedding data"
logger.trace(test_message)
log_output = self.log_capture.getvalue()
self.assertIn("TRACE", log_output)
self.assertIn(test_message, log_output)
def test_trace_hidden_at_debug_level(self):
"""Test that TRACE messages are not logged when level is set to DEBUG."""
logger.setLevel(logging.DEBUG)
test_message = "Test trace message should be hidden"
# Clear any previous output
self.log_capture.truncate(0)
self.log_capture.seek(0)
logger.trace(test_message)
log_output = self.log_capture.getvalue()
self.assertNotIn(test_message, log_output)
def test_trace_hidden_at_info_level(self):
"""Test that TRACE messages are not logged when level is set to INFO."""
logger.setLevel(logging.INFO)
test_message = "Test trace message should be hidden at INFO"
# Clear any previous output
self.log_capture.truncate(0)
self.log_capture.seek(0)
logger.trace(test_message)
log_output = self.log_capture.getvalue()
self.assertNotIn(test_message, log_output)
def test_debug_still_works_at_debug_level(self):
"""Test that DEBUG messages still work when level is set to DEBUG."""
logger.setLevel(logging.DEBUG)
test_message = "Test debug message should be visible"
# Clear any previous output
self.log_capture.truncate(0)
self.log_capture.seek(0)
logger.debug(test_message)
log_output = self.log_capture.getvalue()
self.assertIn("DEBUG", log_output)
self.assertIn(test_message, log_output)
def test_info_still_works_at_info_level(self):
"""Test that INFO messages still work when level is set to INFO."""
logger.setLevel(logging.INFO)
test_message = "Test info message should be visible"
# Clear any previous output
self.log_capture.truncate(0)
self.log_capture.seek(0)
logger.info(test_message)
log_output = self.log_capture.getvalue()
self.assertIn("INFO", log_output)
self.assertIn(test_message, log_output)
def test_mixed_logging_levels(self):
"""Test multiple log levels working together at TRACE level."""
logger.setLevel(TRACE_LEVEL)
# Clear any previous output
self.log_capture.truncate(0)
self.log_capture.seek(0)
logger.info("Info message")
logger.debug("Debug message")
logger.trace("Trace message")
log_output = self.log_capture.getvalue()
self.assertIn("INFO", log_output)
self.assertIn("DEBUG", log_output)
self.assertIn("TRACE", log_output)
self.assertIn("Info message", log_output)
self.assertIn("Debug message", log_output)
self.assertIn("Trace message", log_output)
def run_manual_verification():
"""Run manual verification for visual inspection (can be run standalone)."""
print("=== Manual Verification of TRACE Logging ===")
print(f"TRACE_LEVEL = {TRACE_LEVEL}")
print(f"DEBUG level = {logging.DEBUG}")
print()
# Configure logging for manual testing
logging.basicConfig(
level=TRACE_LEVEL,
format='%(levelname)s:%(name)s:%(message)s',
force=True # Override any existing configuration
)
print("--- Testing with TRACE level (all messages should show) ---")
logger.setLevel(TRACE_LEVEL)
logger.info("This is an INFO message")
logger.debug("This is a DEBUG message")
logger.trace("This is a TRACE message (embedding data would be here)")
print("\n--- Testing with DEBUG level (TRACE should be hidden) ---")
logger.setLevel(logging.DEBUG)
logger.info("INFO with DEBUG level")
logger.debug("DEBUG with DEBUG level")
logger.trace("TRACE with DEBUG level (should be hidden)")
print("\n--- Testing with INFO level (DEBUG and TRACE should be hidden) ---")
logger.setLevel(logging.INFO)
logger.info("INFO with INFO level")
logger.debug("DEBUG with INFO level (should be hidden)")
logger.trace("TRACE with INFO level (should be hidden)")
print("\n✅ Manual verification completed!")
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser(description="Test embedding TRACE logging implementation")
parser.add_argument("--manual", action="store_true", help="Run manual verification with visual output")
args = parser.parse_args()
if args.manual:
print("Running manual verification...")
run_manual_verification()
else:
print("Running automated tests...")
# Run the automated tests
unittest.main(verbosity=2)
@@ -0,0 +1,88 @@
#!/usr/bin/env python3
"""
Test script for Gemini streaming JSON parsing fix.
This script tests the robust error handling added to handle malformed JSON chunks
during streaming responses from Gemini models.
"""
from praisonaiagents import Agent
def test_gemini_streaming_robustness():
"""Test Gemini streaming with robust error handling."""
print("🧪 Testing Gemini Streaming Robustness Fix")
print("=" * 60)
try:
# Create agent with Gemini model (using a lightweight model for testing)
agent = Agent(
instructions="You are a helpful assistant. Be concise.",
llm="gemini/gemini-2.5-flash", # Using flash for faster testing
stream=True,
verbose=True # Enable verbose to see the error handling in action
)
print("✅ Agent created successfully")
print(f"📊 Model: {agent.llm}")
print(f"📊 Stream enabled: {agent.stream}")
print(f"📊 Verbose enabled: {agent.verbose}")
print()
# Test streaming with a simple prompt that might cause chunking issues
print("🔄 Testing streaming response...")
prompt = "Explain what real-time streaming is in AI applications, focusing on the benefits and challenges."
chunk_count = 0
response_content = ""
try:
for chunk in agent.start(prompt):
if chunk:
response_content += chunk
chunk_count += 1
print(chunk, end="", flush=True)
except Exception as streaming_error:
print(f"\n❌ Streaming error occurred: {streaming_error}")
print("🔄 This error should now be handled gracefully with fallback to non-streaming mode")
return False
print("\n\n" + "="*60)
print("✅ Streaming completed successfully!")
print(f"📊 Total chunks received: {chunk_count}")
print(f"📊 Total response length: {len(response_content)} characters")
if chunk_count > 1:
print("✅ SUCCESS: Streaming worked with multiple chunks")
else:
print("⚠️ WARNING: Only received 1 chunk (may have fallen back to non-streaming)")
return True
except Exception as e:
print(f"❌ Test failed with error: {e}")
return False
if __name__ == "__main__":
print("🚀 Starting Gemini Streaming Robustness Test")
print("This test validates the JSON parsing error fixes for Gemini streaming")
print()
success = test_gemini_streaming_robustness()
print(f"\n{'='*60}")
if success:
print("🎉 TEST PASSED: Gemini streaming robustness fix is working!")
else:
print("💥 TEST FAILED: Issues detected with streaming robustness")
print()
print("📝 Key improvements tested:")
print(" • Graceful handling of malformed JSON chunks")
print(" • Automatic fallback to non-streaming on repeated errors")
print(" • Better error logging and categorization")
print(" • Chunk-level error recovery")
# Exit with appropriate status code for CI integration
import sys
sys.exit(0 if success else 1)
+76 -7
View File
@@ -634,6 +634,18 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/4c/a3/ac312faeceffd2d8f86bc6dcb5c401188ba5a01bc88e69bed97578a0dfcd/durationpy-0.9-py3-none-any.whl", hash = "sha256:e65359a7af5cedad07fb77a2dd3f390f8eb0b74cb845589fa6c057086834dd38", size = 3461, upload-time = "2024-10-02T17:58:59.349Z" },
]
[[package]]
name = "ecdsa"
version = "0.19.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "six" },
]
sdist = { url = "https://files.pythonhosted.org/packages/c0/1f/924e3caae75f471eae4b26bd13b698f6af2c44279f67af317439c2f4c46a/ecdsa-0.19.1.tar.gz", hash = "sha256:478cba7b62555866fcb3bb3fe985e06decbdb68ef55713c4e5ab98c57d508e61", size = 201793, upload-time = "2025-03-13T11:52:43.25Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/cb/a3/460c57f094a4a165c84a1341c373b0a4f5ec6ac244b998d5021aade89b77/ecdsa-0.19.1-py2.py3-none-any.whl", hash = "sha256:30638e27cf77b7e15c4c4cc1973720149e1033827cfd00661ca5c8cc0cdb24c3", size = 150607, upload-time = "2025-03-13T11:52:41.757Z" },
]
[[package]]
name = "et-xmlfile"
version = "2.0.0"
@@ -1357,7 +1369,7 @@ wheels = [
[[package]]
name = "litellm"
version = "1.72.1"
version = "1.74.7"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "aiohttp" },
@@ -1372,9 +1384,9 @@ dependencies = [
{ name = "tiktoken" },
{ name = "tokenizers" },
]
sdist = { url = "https://files.pythonhosted.org/packages/d0/a7/67da5f515c3847bd121b5fde54ac48ee3e7d55d75c49d2b51a2f9b20ab7a/litellm-1.72.1.tar.gz", hash = "sha256:d5d822988d62db8fe434aca7584e1cf8f5a03c033e957e3b5c1ca24366840951", size = 8095871, upload-time = "2025-06-04T19:09:07.717Z" }
sdist = { url = "https://files.pythonhosted.org/packages/91/4c/e8ffbd01d0f43357315646890524ce53648a3962169498e08a4b8edca6e2/litellm-1.74.7.tar.gz", hash = "sha256:53b809a342154d8543ea96422cf962cd5ea9df293f83dab0cc63b27baadf0ece", size = 9587483, upload-time = "2025-07-20T01:03:11.853Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/10/2d/48dee9e0cc782a74d7361a3a1acf70d9ac8a9278a4fa8d2dcf2ac7359ed8/litellm-1.72.1-py3-none-any.whl", hash = "sha256:858b4458f199dd8d3acd46436cf5fb1de19404e5dcf92ff59ec417c224b52481", size = 7994774, upload-time = "2025-06-04T19:09:04.316Z" },
{ url = "https://files.pythonhosted.org/packages/1c/54/eb5fee089d3e5e07a6d60b2565f798c66d43f46ba8f339e77f78cee98462/litellm-1.74.7-py3-none-any.whl", hash = "sha256:d630785faf07813cf0d5e9fb0bb84aaa18aa728297858c58c56f34c0b9190df1", size = 8652488, upload-time = "2025-07-20T01:03:09.226Z" },
]
[[package]]
@@ -2286,6 +2298,20 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/ab/5f/b38085618b950b79d2d9164a711c52b10aefc0ae6833b96f626b7021b2ed/pandas-2.2.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:ad5b65698ab28ed8d7f18790a0dc58005c7629f227be9ecc1072aa74c0c1d43a", size = 13098436, upload-time = "2024-09-20T13:09:48.112Z" },
]
[[package]]
name = "passlib"
version = "1.7.4"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/b6/06/9da9ee59a67fae7761aab3ccc84fa4f3f33f125b370f1ccdb915bf967c11/passlib-1.7.4.tar.gz", hash = "sha256:defd50f72b65c5402ab2c573830a6978e5f202ad0d984793c8dde2c4152ebe04", size = 689844, upload-time = "2020-10-08T19:00:52.121Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/3b/a4/ab6b7589382ca3df236e03faa71deac88cae040af60c071a78d254a62172/passlib-1.7.4-py2.py3-none-any.whl", hash = "sha256:aa6bca462b8d8bda89c70b382f0c298a20b5560af6cbfa2dce410c0a2fb669f1", size = 525554, upload-time = "2020-10-08T19:00:49.856Z" },
]
[package.optional-dependencies]
bcrypt = [
{ name = "bcrypt" },
]
[[package]]
name = "pdfminer-six"
version = "20240706"
@@ -2396,7 +2422,7 @@ wheels = [
[[package]]
name = "praisonaiagents"
version = "0.0.144"
version = "0.0.146"
source = { editable = "." }
dependencies = [
{ name = "aiohttp" },
@@ -2416,15 +2442,25 @@ all = [
{ name = "mcp" },
{ name = "mem0ai", extra = ["graph"] },
{ name = "motor" },
{ name = "passlib", extra = ["bcrypt"] },
{ name = "posthog" },
{ name = "pydantic" },
{ name = "pyjwt" },
{ name = "pymongo" },
{ name = "python-jose", extra = ["cryptography"] },
{ name = "python-multipart" },
{ name = "uvicorn" },
]
api = [
{ name = "fastapi" },
{ name = "uvicorn" },
]
auth = [
{ name = "passlib", extra = ["bcrypt"] },
{ name = "pyjwt" },
{ name = "python-jose", extra = ["cryptography"] },
{ name = "python-multipart" },
]
graph = [
{ name = "chromadb" },
{ name = "mem0ai", extra = ["graph"] },
@@ -2465,17 +2501,19 @@ requires-dist = [
{ name = "chromadb", marker = "extra == 'memory'", specifier = ">=1.0.0" },
{ name = "fastapi", marker = "extra == 'api'", specifier = ">=0.115.0" },
{ name = "fastapi", marker = "extra == 'mcp'", specifier = ">=0.115.0" },
{ name = "litellm", marker = "extra == 'llm'", specifier = ">=1.72.0" },
{ name = "litellm", marker = "extra == 'memory'", specifier = ">=1.72.0" },
{ name = "litellm", marker = "extra == 'llm'", specifier = ">=1.72.6" },
{ name = "litellm", marker = "extra == 'memory'", specifier = ">=1.72.6" },
{ name = "markitdown", extras = ["all"], marker = "extra == 'knowledge'", specifier = ">=0.1.0" },
{ name = "mcp", marker = "extra == 'mcp'", specifier = ">=1.6.0" },
{ name = "mem0ai", marker = "extra == 'knowledge'", specifier = ">=0.1.0" },
{ name = "mem0ai", extras = ["graph"], marker = "extra == 'graph'", specifier = ">=0.1.0" },
{ name = "motor", marker = "extra == 'mongodb'", specifier = ">=3.4.0" },
{ name = "openai" },
{ name = "passlib", extras = ["bcrypt"], marker = "extra == 'auth'", specifier = ">=1.7.4" },
{ name = "posthog", specifier = ">=3.0.0" },
{ name = "posthog", marker = "extra == 'telemetry'", specifier = ">=3.0.0" },
{ name = "praisonaiagents", extras = ["api"], marker = "extra == 'all'" },
{ name = "praisonaiagents", extras = ["auth"], marker = "extra == 'all'" },
{ name = "praisonaiagents", extras = ["graph"], marker = "extra == 'all'" },
{ name = "praisonaiagents", extras = ["knowledge"], marker = "extra == 'all'" },
{ name = "praisonaiagents", extras = ["llm"], marker = "extra == 'all'" },
@@ -2485,12 +2523,15 @@ requires-dist = [
{ name = "praisonaiagents", extras = ["telemetry"], marker = "extra == 'all'" },
{ name = "pydantic" },
{ name = "pydantic", marker = "extra == 'llm'", specifier = ">=2.4.2" },
{ name = "pyjwt", marker = "extra == 'auth'", specifier = ">=2.8.0" },
{ name = "pymongo", marker = "extra == 'mongodb'", specifier = ">=4.6.3" },
{ name = "python-jose", extras = ["cryptography"], marker = "extra == 'auth'", specifier = ">=3.3.0" },
{ name = "python-multipart", marker = "extra == 'auth'", specifier = ">=0.0.6" },
{ name = "rich" },
{ name = "uvicorn", marker = "extra == 'api'", specifier = ">=0.34.0" },
{ name = "uvicorn", marker = "extra == 'mcp'", specifier = ">=0.34.0" },
]
provides-extras = ["mcp", "memory", "knowledge", "graph", "llm", "api", "telemetry", "mongodb", "all"]
provides-extras = ["mcp", "memory", "knowledge", "graph", "llm", "api", "telemetry", "mongodb", "auth", "all"]
[[package]]
name = "propcache"
@@ -2856,6 +2897,34 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/6a/3e/b68c118422ec867fa7ab88444e1274aa40681c606d59ac27de5a5588f082/python_dotenv-1.0.1-py3-none-any.whl", hash = "sha256:f7b63ef50f1b690dddf550d03497b66d609393b40b564ed0d674909a68ebf16a", size = 19863, upload-time = "2024-01-23T06:32:58.246Z" },
]
[[package]]
name = "python-jose"
version = "3.5.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "ecdsa" },
{ name = "pyasn1" },
{ name = "rsa" },
]
sdist = { url = "https://files.pythonhosted.org/packages/c6/77/3a1c9039db7124eb039772b935f2244fbb73fc8ee65b9acf2375da1c07bf/python_jose-3.5.0.tar.gz", hash = "sha256:fb4eaa44dbeb1c26dcc69e4bd7ec54a1cb8dd64d3b4d81ef08d90ff453f2b01b", size = 92726, upload-time = "2025-05-28T17:31:54.288Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/d9/c3/0bd11992072e6a1c513b16500a5d07f91a24017c5909b02c72c62d7ad024/python_jose-3.5.0-py2.py3-none-any.whl", hash = "sha256:abd1202f23d34dfad2c3d28cb8617b90acf34132c7afd60abd0b0b7d3cb55771", size = 34624, upload-time = "2025-05-28T17:31:52.802Z" },
]
[package.optional-dependencies]
cryptography = [
{ name = "cryptography" },
]
[[package]]
name = "python-multipart"
version = "0.0.20"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/f3/87/f44d7c9f274c7ee665a29b885ec97089ec5dc034c7f3fafa03da9e39a09e/python_multipart-0.0.20.tar.gz", hash = "sha256:8dd0cab45b8e23064ae09147625994d090fa46f5b0d1e13af944c331a7fa9d13", size = 37158, upload-time = "2024-12-16T19:45:46.972Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/45/58/38b5afbc1a800eeea951b9285d3912613f2603bdf897a4ab0f4bd7f405fc/python_multipart-0.0.20-py3-none-any.whl", hash = "sha256:8a62d3a8335e06589fe01f2a3e178cdcc632f3fbe0d492ad9ee0ec35aab1f104", size = 24546, upload-time = "2024-12-16T19:45:44.423Z" },
]
[[package]]
name = "python-pptx"
version = "1.0.2"
+2 -2
View File
@@ -3,8 +3,8 @@ class Praisonai < Formula
desc "AI tools for various AI applications"
homepage "https://github.com/MervinPraison/PraisonAI"
url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.2.71.tar.gz"
sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.2.71.tar.gz | shasum -a 256`.split.first
url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.2.73.tar.gz"
sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.2.73.tar.gz | shasum -a 256`.split.first
license "MIT"
depends_on "[email protected]"
+67 -1
View File
@@ -545,7 +545,7 @@ class PraisonAI:
return default_args
# Define special commands
special_commands = ['chat', 'code', 'call', 'realtime', 'train', 'ui']
special_commands = ['chat', 'code', 'call', 'realtime', 'train', 'ui', 'context']
parser = argparse.ArgumentParser(prog="praisonai", description="praisonAI command-line interface")
parser.add_argument("--framework", choices=["crewai", "autogen", "praisonai"], help="Specify the framework")
@@ -569,6 +569,9 @@ class PraisonAI:
parser.add_argument("--merge", action="store_true", help="Merge existing agents.yaml with auto-generated agents instead of overwriting")
parser.add_argument("--claudecode", action="store_true", help="Enable Claude Code integration for file modifications and coding tasks")
parser.add_argument("--file", "-f", type=str, help="Read input from a file and append it to the prompt")
parser.add_argument("--url", type=str, help="Repository URL for context analysis")
parser.add_argument("--goal", type=str, help="Goal for context engineering")
parser.add_argument("--auto-analyze", action="store_true", help="Enable automatic analysis in context engineering")
# If we're in a test environment, parse with empty args to avoid pytest interference
if in_test_env:
@@ -674,6 +677,25 @@ class PraisonAI:
self.create_chainlit_interface()
sys.exit(0)
elif args.command == 'context':
if not PRAISONAI_AVAILABLE:
print("[red]ERROR: PraisonAI Agents is not installed. Install with:[/red]")
print("\npip install praisonaiagents\n")
sys.exit(1)
if not args.url:
print("[red]ERROR: --url is required for context command[/red]")
print("Usage: praisonai context --url <repository_url> --goal <goal> [--auto-analyze]")
sys.exit(1)
if not args.goal:
print("[red]ERROR: --goal is required for context command[/red]")
print("Usage: praisonai context --url <repository_url> --goal <goal> [--auto-analyze]")
sys.exit(1)
self.handle_context_command(args.url, args.goal, getattr(args, 'auto_analyze', False))
sys.exit(0)
# Only check framework availability for agent-related operations
if not args.command and (args.init or args.auto or args.framework):
if not CREWAI_AVAILABLE and not AUTOGEN_AVAILABLE and not PRAISONAI_AVAILABLE:
@@ -867,6 +889,50 @@ class PraisonAI:
else:
print("ERROR: Realtime UI is not installed. Please install it with 'pip install \"praisonai[realtime]\"' to use the realtime UI.")
def handle_context_command(self, url: str, goal: str, auto_analyze: bool = False) -> str:
"""
Handle the context command by creating a ContextAgent and running it.
Args:
url: Repository URL for context analysis
goal: Goal for context engineering
auto_analyze: Enable automatic analysis (default: False)
Returns:
str: Result from context engineering
"""
try:
from praisonaiagents import ContextAgent
print("[bold green]Starting Context Engineering...[/bold green]")
print(f"URL: {url}")
print(f"Goal: {goal}")
print(f"Auto-analyze: {auto_analyze}")
# Use the same model configuration pattern as other CLI commands
# Priority order: MODEL_NAME > OPENAI_MODEL_NAME for model selection
model_name = os.environ.get("MODEL_NAME") or os.environ.get("OPENAI_MODEL_NAME", "gpt-4o-mini")
# Create ContextAgent with user's LLM configuration
agent = ContextAgent(llm=model_name, auto_analyze=auto_analyze)
# Format input as expected by the start method: "url goal"
input_text = f"{url} {goal}"
# Execute the context engineering
result = agent.start(input_text)
print("\n[bold green]Context Engineering Complete![/bold green]")
print(result)
return result
except ImportError as e:
print(f"[red]ERROR: Failed to import ContextAgent: {e}[/red]")
print("Make sure praisonaiagents is installed: pip install praisonaiagents")
sys.exit(1)
except Exception as e:
print(f"[red]ERROR: Context engineering failed: {e}[/red]")
sys.exit(1)
if __name__ == "__main__":
praison_ai = PraisonAI()
praison_ai.main()
+1 -1
View File
@@ -57,7 +57,7 @@ class CloudDeployer:
file.write("FROM python:3.11-slim\n")
file.write("WORKDIR /app\n")
file.write("COPY . .\n")
file.write("RUN pip install flask praisonai==2.2.71 gunicorn markdown\n")
file.write("RUN pip install flask praisonai==2.2.73 gunicorn markdown\n")
file.write("EXPOSE 8080\n")
file.write('CMD ["gunicorn", "-b", "0.0.0.0:8080", "api:app"]\n')
+15 -15
View File
@@ -1,6 +1,6 @@
[project]
name = "PraisonAI"
version = "2.2.71"
version = "2.2.73"
description = "PraisonAI is an AI Agents Framework with Self Reflection. PraisonAI application combines PraisonAI Agents, AutoGen, and CrewAI into a low-code solution for building and managing multi-agent LLM systems, focusing on simplicity, customisation, and efficient human-agent collaboration."
readme = "README.md"
license = ""
@@ -12,7 +12,7 @@ dependencies = [
"rich>=13.7",
"markdown>=3.5",
"pyparsing>=3.0.0",
"praisonaiagents>=0.0.144",
"praisonaiagents>=0.0.146",
"python-dotenv>=0.19.0",
"instructor>=1.3.3",
"PyYAML>=6.0",
@@ -45,11 +45,11 @@ anthropic = ["langchain-anthropic>=0.3.0"]
cohere = ["langchain-cohere>=0.3.0,<0.4.0"]
chat = [
"chainlit==2.5.5",
"litellm>=1.68.0",
"litellm>=1.72.6",
"aiosqlite>=0.20.0",
"greenlet>=3.0.3",
"tavily-python==0.5.0",
"crawl4ai>=0.6.0",
"crawl4ai>=0.7.0",
"sqlalchemy>=2.0.36",
"playwright>=1.47.0",
"rich",
@@ -57,22 +57,22 @@ chat = [
]
code = [
"chainlit==2.5.5",
"litellm>=1.68.0",
"litellm>=1.72.6",
"aiosqlite>=0.20.0",
"greenlet>=3.0.3",
"tavily-python==0.5.0",
"crawl4ai>=0.6.0",
"crawl4ai>=0.7.0",
"sqlalchemy>=2.0.36",
"playwright>=1.47.0",
"pydantic<=2.10.1"
]
realtime = [
"chainlit==2.5.5",
"litellm>=1.68.0",
"litellm>=1.72.6",
"aiosqlite>=0.20.0",
"greenlet>=3.0.3",
"tavily-python==0.5.0",
"crawl4ai>=0.6.0",
"crawl4ai>=0.7.0",
"websockets>=12.0",
"plotly>=5.24.0",
"yfinance>=0.2.44",
@@ -90,7 +90,7 @@ call = [
"openai>=1.54.0",
]
train = []
crewai = ["crewai>=0.32.0", "praisonai-tools>=0.0.22"]
crewai = ["crewai>=0.148.0", "praisonai-tools>=0.0.22"]
autogen = ["pyautogen==0.2.29", "praisonai-tools>=0.0.22", "crewai"]
autogen-v4 = [
"autogen-agentchat>=0.4.0",
@@ -102,7 +102,7 @@ autogen-v4 = [
[tool.poetry]
name = "PraisonAI"
version = "2.2.71"
version = "2.2.73"
description = "PraisonAI is an AI Agents Framework with Self Reflection. PraisonAI application combines PraisonAI Agents, AutoGen, and CrewAI into a low-code solution for building and managing multi-agent LLM systems, focusing on simplicity, customisation, and efficient human-agent collaboration."
authors = ["Mervin Praison"]
license = ""
@@ -120,7 +120,7 @@ python = ">=3.10,<3.13"
rich = ">=13.7"
markdown = ">=3.5"
pyparsing = ">=3.0.0"
praisonaiagents = ">=0.0.144"
praisonaiagents = ">=0.0.146"
python-dotenv = ">=0.19.0"
instructor = ">=1.3.3"
PyYAML = ">=6.0"
@@ -129,7 +129,7 @@ pyautogen = {version = "==0.2.29", optional = true}
autogen-agentchat = {version = ">=0.4.0", optional = true}
autogen-ext = {version = ">=0.4.0", optional = true}
autogen-core = {version = ">=0.4.0", optional = true}
crewai = {version = ">=0.32.0", optional = true}
crewai = {version = ">=0.148.0", optional = true}
praisonai-tools = {version = ">=0.0.22", optional = true}
chainlit = {version = "==2.5.5", optional = true}
gradio = {version = ">=4.26.0", optional = true}
@@ -139,11 +139,11 @@ langchain-google-genai = {version = ">=2.1.0", optional = true}
langchain-anthropic = {version = ">=0.3.0", optional = true}
langchain-openai = {version = ">=0.2.1,<0.3.0", optional = true}
langchain-cohere = {version = ">=0.3.0,<0.4.0", optional = true}
litellm = {version = ">=1.68.0", optional = true}
litellm = {version = ">=1.72.6", optional = true}
aiosqlite= {version = ">=0.20.0", optional = true}
greenlet = {version = ">=3.0.3", optional = true}
tavily-python = {version = "==0.5.0", optional=true}
crawl4ai = {version = ">=0.6.0", optional = true}
crawl4ai = {version = ">=0.7.0", optional = true}
websockets = {version = ">=12.0", optional = true}
plotly = {version = ">=5.24.0", optional = true}
yfinance = {version = ">=0.2.44", optional = true}
@@ -206,7 +206,7 @@ langchain-google-genai = ">=2.1.0"
langchain-anthropic = ">=0.3.0"
langchain-openai = ">=0.2.1,<0.3.0"
langchain-cohere = ">=0.3.0,<0.4.0"
litellm = ">=1.68.0"
litellm = ">=1.72.6"
aiosqlite = ">=0.20.0"
greenlet = ">=3.0.3"
+43 -60
View File
@@ -319,22 +319,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/5d/35/be73b6015511aa0173ec595fc579133b797ad532996f2998fd6b8d1bbe6b/audioop_lts-0.2.1-cp313-cp313t-win_arm64.whl", hash = "sha256:78bfb3703388c780edf900be66e07de5a3d4105ca8e8720c5c4d67927e0b15d0", size = 23918, upload-time = "2024-08-04T21:14:42.803Z" },
]
[[package]]
name = "auth0-python"
version = "4.9.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "aiohttp" },
{ name = "cryptography" },
{ name = "pyjwt" },
{ name = "requests" },
{ name = "urllib3" },
]
sdist = { url = "https://files.pythonhosted.org/packages/e8/46/1071f1a190b2397874cb4bf6be4daddc2aa3f83618d27e1e83df89a32c29/auth0_python-4.9.0.tar.gz", hash = "sha256:f9b31ea9c906d0a123b9cdc6ccd7bbbb8156123f44789b08571c45947fb21238", size = 75870, upload-time = "2025-04-01T09:31:37.403Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/ac/d1/800ab8dfe15f00836b8d1ea41f68f5e4731a96e8fc19548993996f3b5728/auth0_python-4.9.0-py3-none-any.whl", hash = "sha256:6440c7f74dfd669d9f5cdfe9bb44c4c3b230ce98a82353f55a387e90241fbf5b", size = 135349, upload-time = "2025-04-01T09:31:35.538Z" },
]
[[package]]
name = "autogen-agentchat"
version = "0.6.4"
@@ -921,11 +905,10 @@ wheels = [
[[package]]
name = "crewai"
version = "0.119.0"
version = "0.148.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "appdirs" },
{ name = "auth0-python" },
{ name = "blinker" },
{ name = "chromadb" },
{ name = "click" },
@@ -934,6 +917,7 @@ dependencies = [
{ name = "json5" },
{ name = "jsonref" },
{ name = "litellm" },
{ name = "onnxruntime" },
{ name = "openai" },
{ name = "openpyxl" },
{ name = "opentelemetry-api" },
@@ -941,16 +925,18 @@ dependencies = [
{ name = "opentelemetry-sdk" },
{ name = "pdfplumber" },
{ name = "pydantic" },
{ name = "pyjwt" },
{ name = "python-dotenv" },
{ name = "pyvis" },
{ name = "regex" },
{ name = "tokenizers" },
{ name = "tomli" },
{ name = "tomli-w" },
{ name = "uv" },
]
sdist = { url = "https://files.pythonhosted.org/packages/41/cc/e05b7c1e4c58d47811673aac6f249eb2648b89e8ffdae3e4a9a2f4641ad2/crewai-0.119.0.tar.gz", hash = "sha256:1cba70ad29399d057bb8d75f167cfaa564de561e409c9d0b2dff72e7d127d92c", size = 72059811, upload-time = "2025-05-08T00:44:09.165Z" }
sdist = { url = "https://files.pythonhosted.org/packages/db/46/3247312d55b1d5887d7ce6d0a3cc39c6dc118452c1a1fb9eaaa9d121a5b4/crewai-0.148.0.tar.gz", hash = "sha256:a713477cebc98a641c8dfda0f6415b6449143848fbdf91fa6aa0f90a0e726a49", size = 6315505, upload-time = "2025-07-16T19:36:27.26Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/11/0a/eb1af6368b3f8b28805cfbac7acf2815438c3f0a0f0149c5a2edc62c666d/crewai-0.119.0-py3-none-any.whl", hash = "sha256:c6d3a447e97a30924df2151f7219e1721506415e062fa2d6d13e3c3550cfa94e", size = 308724, upload-time = "2025-05-08T00:44:03.958Z" },
{ url = "https://files.pythonhosted.org/packages/bb/6f/8d0af8154b9b97d0207f76cd8607721536afd22fabef9702ee664ff27923/crewai-0.148.0-py3-none-any.whl", hash = "sha256:6865355d96493db63f29064eefe6397d27be2a6729cf3957aa6a90d7190ee50f", size = 365343, upload-time = "2025-07-16T19:36:24.553Z" },
]
[[package]]
@@ -1963,11 +1949,11 @@ wheels = [
[[package]]
name = "json-repair"
version = "0.44.1"
version = "0.25.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/a8/6b/ed6e92efc5acfbc9c35ccae1676b70e4adb1552421e64f838c2a3f097d9a/json_repair-0.44.1.tar.gz", hash = "sha256:1130eb9733b868dac1340b43cb2effebb519ae6d52dd2d0728c6cca517f1e0b4", size = 32886, upload-time = "2025-04-30T16:09:38.54Z" }
sdist = { url = "https://files.pythonhosted.org/packages/66/cb/50b0bbc3e504ef875aea0062cdc108077e4923fb8c1209c70c80dc043933/json_repair-0.25.2.tar.gz", hash = "sha256:161a56d7e6bbfd4cad3a614087e3e0dbd0e10d402dd20dc7db418432428cb32b", size = 20458, upload-time = "2024-06-27T16:26:15.492Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/82/b4/3cbd27a3240b2962c3b87bbb1c20eb6c56e5b26cde61f141f86ca98e2f68/json_repair-0.44.1-py3-none-any.whl", hash = "sha256:51d82532c3b8263782a301eb7904c75dce5fee8c0d1aba490287fc0ab779ac50", size = 22478, upload-time = "2025-04-30T16:09:37.303Z" },
{ url = "https://files.pythonhosted.org/packages/aa/43/ac6691c7b5aa7191c964a04ae926d2bb06d9297dba1f2287df5b85cb3715/json_repair-0.25.2-py3-none-any.whl", hash = "sha256:51d67295c3184b6c41a3572689661c6128cef6cfc9fb04db63130709adfc5bf0", size = 12740, upload-time = "2024-06-27T16:26:13.823Z" },
]
[[package]]
@@ -2333,7 +2319,7 @@ wheels = [
[[package]]
name = "litellm"
version = "1.68.0"
version = "1.72.6"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "aiohttp" },
@@ -2348,9 +2334,9 @@ dependencies = [
{ name = "tiktoken" },
{ name = "tokenizers" },
]
sdist = { url = "https://files.pythonhosted.org/packages/ba/22/138545b646303ca3f4841b69613c697b9d696322a1386083bb70bcbba60b/litellm-1.68.0.tar.gz", hash = "sha256:9fb24643db84dfda339b64bafca505a2eef857477afbc6e98fb56512c24dbbfa", size = 7314051, upload-time = "2025-05-04T05:41:48.081Z" }
sdist = { url = "https://files.pythonhosted.org/packages/8d/15/df75f278fd998f6d6900f692b9de2fba2814b316c123c99072a813668aac/litellm-1.72.6.tar.gz", hash = "sha256:4e5c7e4273b09b765302d2faaec30f77b42255c0055b427b55ea02b8092b8582", size = 8393603, upload-time = "2025-06-14T21:43:11.023Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/10/af/1e344bc8aee41445272e677d802b774b1f8b34bdc3bb5697ba30f0fb5d52/litellm-1.68.0-py3-none-any.whl", hash = "sha256:3bca38848b1a5236b11aa6b70afa4393b60880198c939e582273f51a542d4759", size = 7684460, upload-time = "2025-05-04T05:41:44.78Z" },
{ url = "https://files.pythonhosted.org/packages/96/c9/4aae0b77632279eef9716dbcb98edd8b36c08a9da070e2470ca9c410c0f8/litellm-1.72.6-py3-none-any.whl", hash = "sha256:e0ae98d25db4910e78b1a0a604f24c0d6875f6cdea02426b264a45d4fbdb8c46", size = 8302810, upload-time = "2025-06-14T21:43:08.628Z" },
]
[[package]]
@@ -2955,7 +2941,7 @@ wheels = [
[[package]]
name = "onnxruntime"
version = "1.20.1"
version = "1.22.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "coloredlogs" },
@@ -2966,27 +2952,24 @@ dependencies = [
{ name = "sympy" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/4e/28/99f903b0eb1cd6f3faa0e343217d9fb9f47b84bca98bd9859884631336ee/onnxruntime-1.20.1-cp310-cp310-macosx_13_0_universal2.whl", hash = "sha256:e50ba5ff7fed4f7d9253a6baf801ca2883cc08491f9d32d78a80da57256a5439", size = 30996314, upload-time = "2024-11-21T00:48:31.43Z" },
{ url = "https://files.pythonhosted.org/packages/6d/c6/c4c0860bee2fde6037bdd9dcd12d323f6e38cf00fcc9a5065b394337fc55/onnxruntime-1.20.1-cp310-cp310-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7b2908b50101a19e99c4d4e97ebb9905561daf61829403061c1adc1b588bc0de", size = 11954010, upload-time = "2024-11-21T00:48:35.254Z" },
{ url = "https://files.pythonhosted.org/packages/63/47/3dc0b075ab539f16b3d8b09df6b504f51836086ee709690a6278d791737d/onnxruntime-1.20.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d82daaec24045a2e87598b8ac2b417b1cce623244e80e663882e9fe1aae86410", size = 13330452, upload-time = "2024-11-21T00:48:40.02Z" },
{ url = "https://files.pythonhosted.org/packages/27/ef/80fab86289ecc01a734b7ddf115dfb93d8b2e004bd1e1977e12881c72b12/onnxruntime-1.20.1-cp310-cp310-win32.whl", hash = "sha256:4c4b251a725a3b8cf2aab284f7d940c26094ecd9d442f07dd81ab5470e99b83f", size = 9813849, upload-time = "2024-11-21T00:48:43.569Z" },
{ url = "https://files.pythonhosted.org/packages/a9/e6/33ab10066c9875a29d55e66ae97c3bf91b9b9b987179455d67c32261a49c/onnxruntime-1.20.1-cp310-cp310-win_amd64.whl", hash = "sha256:d3b616bb53a77a9463707bb313637223380fc327f5064c9a782e8ec69c22e6a2", size = 11329702, upload-time = "2024-11-21T00:48:46.599Z" },
{ url = "https://files.pythonhosted.org/packages/95/8d/2634e2959b34aa8a0037989f4229e9abcfa484e9c228f99633b3241768a6/onnxruntime-1.20.1-cp311-cp311-macosx_13_0_universal2.whl", hash = "sha256:06bfbf02ca9ab5f28946e0f912a562a5f005301d0c419283dc57b3ed7969bb7b", size = 30998725, upload-time = "2024-11-21T00:48:51.013Z" },
{ url = "https://files.pythonhosted.org/packages/a5/da/c44bf9bd66cd6d9018a921f053f28d819445c4d84b4dd4777271b0fe52a2/onnxruntime-1.20.1-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f6243e34d74423bdd1edf0ae9596dd61023b260f546ee17d701723915f06a9f7", size = 11955227, upload-time = "2024-11-21T00:48:54.556Z" },
{ url = "https://files.pythonhosted.org/packages/11/ac/4120dfb74c8e45cce1c664fc7f7ce010edd587ba67ac41489f7432eb9381/onnxruntime-1.20.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5eec64c0269dcdb8d9a9a53dc4d64f87b9e0c19801d9321246a53b7eb5a7d1bc", size = 13331703, upload-time = "2024-11-21T00:48:57.97Z" },
{ url = "https://files.pythonhosted.org/packages/12/f1/cefacac137f7bb7bfba57c50c478150fcd3c54aca72762ac2c05ce0532c1/onnxruntime-1.20.1-cp311-cp311-win32.whl", hash = "sha256:a19bc6e8c70e2485a1725b3d517a2319603acc14c1f1a017dda0afe6d4665b41", size = 9813977, upload-time = "2024-11-21T00:49:00.519Z" },
{ url = "https://files.pythonhosted.org/packages/2c/2d/2d4d202c0bcfb3a4cc2b171abb9328672d7f91d7af9ea52572722c6d8d96/onnxruntime-1.20.1-cp311-cp311-win_amd64.whl", hash = "sha256:8508887eb1c5f9537a4071768723ec7c30c28eb2518a00d0adcd32c89dea3221", size = 11329895, upload-time = "2024-11-21T00:49:03.845Z" },
{ url = "https://files.pythonhosted.org/packages/e5/39/9335e0874f68f7d27103cbffc0e235e32e26759202df6085716375c078bb/onnxruntime-1.20.1-cp312-cp312-macosx_13_0_universal2.whl", hash = "sha256:22b0655e2bf4f2161d52706e31f517a0e54939dc393e92577df51808a7edc8c9", size = 31007580, upload-time = "2024-11-21T00:49:07.029Z" },
{ url = "https://files.pythonhosted.org/packages/c5/9d/a42a84e10f1744dd27c6f2f9280cc3fb98f869dd19b7cd042e391ee2ab61/onnxruntime-1.20.1-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f1f56e898815963d6dc4ee1c35fc6c36506466eff6d16f3cb9848cea4e8c8172", size = 11952833, upload-time = "2024-11-21T00:49:10.563Z" },
{ url = "https://files.pythonhosted.org/packages/47/42/2f71f5680834688a9c81becbe5c5bb996fd33eaed5c66ae0606c3b1d6a02/onnxruntime-1.20.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bb71a814f66517a65628c9e4a2bb530a6edd2cd5d87ffa0af0f6f773a027d99e", size = 13333903, upload-time = "2024-11-21T00:49:12.984Z" },
{ url = "https://files.pythonhosted.org/packages/c8/f1/aabfdf91d013320aa2fc46cf43c88ca0182860ff15df872b4552254a9680/onnxruntime-1.20.1-cp312-cp312-win32.whl", hash = "sha256:bd386cc9ee5f686ee8a75ba74037750aca55183085bf1941da8efcfe12d5b120", size = 9814562, upload-time = "2024-11-21T00:49:15.453Z" },
{ url = "https://files.pythonhosted.org/packages/dd/80/76979e0b744307d488c79e41051117634b956612cc731f1028eb17ee7294/onnxruntime-1.20.1-cp312-cp312-win_amd64.whl", hash = "sha256:19c2d843eb074f385e8bbb753a40df780511061a63f9def1b216bf53860223fb", size = 11331482, upload-time = "2024-11-21T00:49:19.412Z" },
{ url = "https://files.pythonhosted.org/packages/f7/71/c5d980ac4189589267a06f758bd6c5667d07e55656bed6c6c0580733ad07/onnxruntime-1.20.1-cp313-cp313-macosx_13_0_universal2.whl", hash = "sha256:cc01437a32d0042b606f462245c8bbae269e5442797f6213e36ce61d5abdd8cc", size = 31007574, upload-time = "2024-11-21T00:49:23.225Z" },
{ url = "https://files.pythonhosted.org/packages/81/0d/13bbd9489be2a6944f4a940084bfe388f1100472f38c07080a46fbd4ab96/onnxruntime-1.20.1-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fb44b08e017a648924dbe91b82d89b0c105b1adcfe31e90d1dc06b8677ad37be", size = 11951459, upload-time = "2024-11-21T00:49:26.269Z" },
{ url = "https://files.pythonhosted.org/packages/c0/ea/4454ae122874fd52bbb8a961262de81c5f932edeb1b72217f594c700d6ef/onnxruntime-1.20.1-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bda6aebdf7917c1d811f21d41633df00c58aff2bef2f598f69289c1f1dabc4b3", size = 13331620, upload-time = "2024-11-21T00:49:28.875Z" },
{ url = "https://files.pythonhosted.org/packages/d8/e0/50db43188ca1c945decaa8fc2a024c33446d31afed40149897d4f9de505f/onnxruntime-1.20.1-cp313-cp313-win_amd64.whl", hash = "sha256:d30367df7e70f1d9fc5a6a68106f5961686d39b54d3221f760085524e8d38e16", size = 11331758, upload-time = "2024-11-21T00:49:31.417Z" },
{ url = "https://files.pythonhosted.org/packages/d8/55/3821c5fd60b52a6c82a00bba18531793c93c4addfe64fbf061e235c5617a/onnxruntime-1.20.1-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c9158465745423b2b5d97ed25aa7740c7d38d2993ee2e5c3bfacb0c4145c49d8", size = 11950342, upload-time = "2024-11-21T00:49:34.164Z" },
{ url = "https://files.pythonhosted.org/packages/14/56/fd990ca222cef4f9f4a9400567b9a15b220dee2eafffb16b2adbc55c8281/onnxruntime-1.20.1-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0df6f2df83d61f46e842dbcde610ede27218947c33e994545a22333491e72a3b", size = 13337040, upload-time = "2024-11-21T00:49:37.271Z" },
{ url = "https://files.pythonhosted.org/packages/67/3c/c99b21646a782b89c33cffd96fdee02a81bc43f0cb651de84d58ec11e30e/onnxruntime-1.22.0-cp310-cp310-macosx_13_0_universal2.whl", hash = "sha256:85d8826cc8054e4d6bf07f779dc742a363c39094015bdad6a08b3c18cfe0ba8c", size = 34273493, upload-time = "2025-05-09T20:25:55.66Z" },
{ url = "https://files.pythonhosted.org/packages/54/ab/fd9a3b5285008c060618be92e475337fcfbf8689787953d37273f7b52ab0/onnxruntime-1.22.0-cp310-cp310-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:468c9502a12f6f49ec335c2febd22fdceecc1e4cc96dfc27e419ba237dff5aff", size = 14445346, upload-time = "2025-05-09T20:25:41.322Z" },
{ url = "https://files.pythonhosted.org/packages/1f/ca/a5625644bc079e04e3076a5ac1fb954d1e90309b8eb987a4f800732ffee6/onnxruntime-1.22.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:681fe356d853630a898ee05f01ddb95728c9a168c9460e8361d0a240c9b7cb97", size = 16392959, upload-time = "2025-05-09T20:26:09.047Z" },
{ url = "https://files.pythonhosted.org/packages/6d/6b/8267490476e8d4dd1883632c7e46a4634384c7ff1c35ae44edc8ab0bb7a9/onnxruntime-1.22.0-cp310-cp310-win_amd64.whl", hash = "sha256:20bca6495d06925631e201f2b257cc37086752e8fe7b6c83a67c6509f4759bc9", size = 12689974, upload-time = "2025-05-12T21:26:09.704Z" },
{ url = "https://files.pythonhosted.org/packages/7a/08/c008711d1b92ff1272f4fea0fbee57723171f161d42e5c680625535280af/onnxruntime-1.22.0-cp311-cp311-macosx_13_0_universal2.whl", hash = "sha256:8d6725c5b9a681d8fe72f2960c191a96c256367887d076b08466f52b4e0991df", size = 34282151, upload-time = "2025-05-09T20:25:59.246Z" },
{ url = "https://files.pythonhosted.org/packages/3e/8b/22989f6b59bc4ad1324f07a945c80b9ab825f0a581ad7a6064b93716d9b7/onnxruntime-1.22.0-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fef17d665a917866d1f68f09edc98223b9a27e6cb167dec69da4c66484ad12fd", size = 14446302, upload-time = "2025-05-09T20:25:44.299Z" },
{ url = "https://files.pythonhosted.org/packages/7a/d5/aa83d084d05bc8f6cf8b74b499c77431ffd6b7075c761ec48ec0c161a47f/onnxruntime-1.22.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b978aa63a9a22095479c38371a9b359d4c15173cbb164eaad5f2cd27d666aa65", size = 16393496, upload-time = "2025-05-09T20:26:11.588Z" },
{ url = "https://files.pythonhosted.org/packages/89/a5/1c6c10322201566015183b52ef011dfa932f5dd1b278de8d75c3b948411d/onnxruntime-1.22.0-cp311-cp311-win_amd64.whl", hash = "sha256:03d3ef7fb11adf154149d6e767e21057e0e577b947dd3f66190b212528e1db31", size = 12691517, upload-time = "2025-05-12T21:26:13.354Z" },
{ url = "https://files.pythonhosted.org/packages/4d/de/9162872c6e502e9ac8c99a98a8738b2fab408123d11de55022ac4f92562a/onnxruntime-1.22.0-cp312-cp312-macosx_13_0_universal2.whl", hash = "sha256:f3c0380f53c1e72a41b3f4d6af2ccc01df2c17844072233442c3a7e74851ab97", size = 34298046, upload-time = "2025-05-09T20:26:02.399Z" },
{ url = "https://files.pythonhosted.org/packages/03/79/36f910cd9fc96b444b0e728bba14607016079786adf032dae61f7c63b4aa/onnxruntime-1.22.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c8601128eaef79b636152aea76ae6981b7c9fc81a618f584c15d78d42b310f1c", size = 14443220, upload-time = "2025-05-09T20:25:47.078Z" },
{ url = "https://files.pythonhosted.org/packages/8c/60/16d219b8868cc8e8e51a68519873bdb9f5f24af080b62e917a13fff9989b/onnxruntime-1.22.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6964a975731afc19dc3418fad8d4e08c48920144ff590149429a5ebe0d15fb3c", size = 16406377, upload-time = "2025-05-09T20:26:14.478Z" },
{ url = "https://files.pythonhosted.org/packages/36/b4/3f1c71ce1d3d21078a6a74c5483bfa2b07e41a8d2b8fb1e9993e6a26d8d3/onnxruntime-1.22.0-cp312-cp312-win_amd64.whl", hash = "sha256:c0d534a43d1264d1273c2d4f00a5a588fa98d21117a3345b7104fa0bbcaadb9a", size = 12692233, upload-time = "2025-05-12T21:26:16.963Z" },
{ url = "https://files.pythonhosted.org/packages/a9/65/5cb5018d5b0b7cba820d2c4a1d1b02d40df538d49138ba36a509457e4df6/onnxruntime-1.22.0-cp313-cp313-macosx_13_0_universal2.whl", hash = "sha256:fe7c051236aae16d8e2e9ffbfc1e115a0cc2450e873a9c4cb75c0cc96c1dae07", size = 34298715, upload-time = "2025-05-09T20:26:05.634Z" },
{ url = "https://files.pythonhosted.org/packages/e1/89/1dfe1b368831d1256b90b95cb8d11da8ab769febd5c8833ec85ec1f79d21/onnxruntime-1.22.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6a6bbed10bc5e770c04d422893d3045b81acbbadc9fb759a2cd1ca00993da919", size = 14443266, upload-time = "2025-05-09T20:25:49.479Z" },
{ url = "https://files.pythonhosted.org/packages/1e/70/342514ade3a33ad9dd505dcee96ff1f0e7be6d0e6e9c911fe0f1505abf42/onnxruntime-1.22.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9fe45ee3e756300fccfd8d61b91129a121d3d80e9d38e01f03ff1295badc32b8", size = 16406707, upload-time = "2025-05-09T20:26:17.454Z" },
{ url = "https://files.pythonhosted.org/packages/3e/89/2f64e250945fa87140fb917ba377d6d0e9122e029c8512f389a9b7f953f4/onnxruntime-1.22.0-cp313-cp313-win_amd64.whl", hash = "sha256:5a31d84ef82b4b05d794a4ce8ba37b0d9deb768fd580e36e17b39e0b4840253b", size = 12691777, upload-time = "2025-05-12T21:26:20.19Z" },
{ url = "https://files.pythonhosted.org/packages/9f/48/d61d5f1ed098161edd88c56cbac49207d7b7b149e613d2cd7e33176c63b3/onnxruntime-1.22.0-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0a2ac5bd9205d831541db4e508e586e764a74f14efdd3f89af7fd20e1bf4a1ed", size = 14454003, upload-time = "2025-05-09T20:25:52.287Z" },
{ url = "https://files.pythonhosted.org/packages/c3/16/873b955beda7bada5b0d798d3a601b2ff210e44ad5169f6d405b93892103/onnxruntime-1.22.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:64845709f9e8a2809e8e009bc4c8f73b788cee9c6619b7d9930344eae4c9cd36", size = 16427482, upload-time = "2025-05-09T20:26:20.376Z" },
]
[[package]]
@@ -4048,7 +4031,7 @@ wheels = [
[[package]]
name = "praisonai"
version = "2.2.71"
version = "2.2.73"
source = { editable = "." }
dependencies = [
{ name = "instructor" },
@@ -4169,12 +4152,12 @@ requires-dist = [
{ name = "chainlit", marker = "extra == 'code'", specifier = "==2.5.5" },
{ name = "chainlit", marker = "extra == 'realtime'", specifier = "==2.5.5" },
{ name = "chainlit", marker = "extra == 'ui'", specifier = "==2.5.5" },
{ name = "crawl4ai", marker = "extra == 'chat'", specifier = ">=0.6.0" },
{ name = "crawl4ai", marker = "extra == 'code'", specifier = ">=0.6.0" },
{ name = "crawl4ai", marker = "extra == 'realtime'", specifier = ">=0.6.0" },
{ name = "crawl4ai", marker = "extra == 'chat'", specifier = ">=0.7.0" },
{ name = "crawl4ai", marker = "extra == 'code'", specifier = ">=0.7.0" },
{ name = "crawl4ai", marker = "extra == 'realtime'", specifier = ">=0.7.0" },
{ name = "crewai", marker = "extra == 'autogen'" },
{ name = "crewai", marker = "extra == 'autogen-v4'" },
{ name = "crewai", marker = "extra == 'crewai'", specifier = ">=0.32.0" },
{ name = "crewai", marker = "extra == 'crewai'", specifier = ">=0.148.0" },
{ name = "duckduckgo-search", marker = "extra == 'realtime'", specifier = ">=6.3.0" },
{ name = "fastapi", marker = "extra == 'api'", specifier = ">=0.115.0" },
{ name = "fastapi", marker = "extra == 'call'", specifier = ">=0.95.0" },
@@ -4190,9 +4173,9 @@ requires-dist = [
{ name = "langchain-cohere", marker = "extra == 'cohere'", specifier = ">=0.3.0,<0.4.0" },
{ name = "langchain-google-genai", marker = "extra == 'google'", specifier = ">=2.1.0" },
{ name = "langchain-openai", marker = "extra == 'openai'", specifier = ">=0.2.1,<0.3.0" },
{ name = "litellm", marker = "extra == 'chat'", specifier = ">=1.68.0" },
{ name = "litellm", marker = "extra == 'code'", specifier = ">=1.68.0" },
{ name = "litellm", marker = "extra == 'realtime'", specifier = ">=1.68.0" },
{ name = "litellm", marker = "extra == 'chat'", specifier = ">=1.72.6" },
{ name = "litellm", marker = "extra == 'code'", specifier = ">=1.72.6" },
{ name = "litellm", marker = "extra == 'realtime'", specifier = ">=1.72.6" },
{ name = "markdown", specifier = ">=3.5" },
{ name = "mcp", specifier = ">=1.6.0" },
{ name = "openai", marker = "extra == 'call'", specifier = ">=1.54.0" },
@@ -4202,7 +4185,7 @@ requires-dist = [
{ name = "praisonai-tools", marker = "extra == 'autogen'", specifier = ">=0.0.22" },
{ name = "praisonai-tools", marker = "extra == 'autogen-v4'", specifier = ">=0.0.22" },
{ name = "praisonai-tools", marker = "extra == 'crewai'", specifier = ">=0.0.22" },
{ name = "praisonaiagents", specifier = ">=0.0.144" },
{ name = "praisonaiagents", specifier = ">=0.0.146" },
{ name = "pyautogen", marker = "extra == 'autogen'", specifier = "==0.2.29" },
{ name = "pydantic", marker = "extra == 'chat'", specifier = "<=2.10.1" },
{ name = "pydantic", marker = "extra == 'code'", specifier = "<=2.10.1" },
@@ -4257,7 +4240,7 @@ wheels = [
[[package]]
name = "praisonaiagents"
version = "0.0.144"
version = "0.0.146"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "aiohttp" },
@@ -4266,9 +4249,9 @@ dependencies = [
{ name = "pydantic" },
{ name = "rich" },
]
sdist = { url = "https://files.pythonhosted.org/packages/b4/3a/bf0925b182699f4fe9d8b645772ac76126af536755058682922955efe2b1/praisonaiagents-0.0.144.tar.gz", hash = "sha256:5b3870d8bd6679a27a38793d7444f4636fd0bb21b880e4866706dc9f8da783fe", size = 217748, upload-time = "2025-07-18T21:26:38.632Z" }
sdist = { url = "https://files.pythonhosted.org/packages/a0/49/4542900208f5f63b56c64afd2a7fe5bb967ef2f75a1191ce6ed542454a1e/praisonaiagents-0.0.146.tar.gz", hash = "sha256:264fbd93dc291f1ee5b8853481c439a85c96ec5124161cea8a0a2d39890d3001", size = 268919, upload-time = "2025-07-23T22:55:05.915Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/41/73/8a89fcb4d8cd17c7dbe43505a671644c283af859ce3984660ad71dbb71d0/praisonaiagents-0.0.144-py3-none-any.whl", hash = "sha256:9907cc4e02d35a20c99a171c723625384937a02f725f00f20a43b7d09ceb7d14", size = 237310, upload-time = "2025-07-18T21:26:37.074Z" },
{ url = "https://files.pythonhosted.org/packages/14/4b/55a2dae2a9f65e34a1021648c746d1386859a44065e13e129e5ae6cc7d2f/praisonaiagents-0.0.146-py3-none-any.whl", hash = "sha256:10144163cdf3b75f8cfa751d5317a22ee0b65d0ff9449a9354e3f33b3019e844", size = 286525, upload-time = "2025-07-23T22:55:04.217Z" },
]
[[package]]