mirror of
https://github.com/MervinPraison/PraisonAI.git
synced 2026-11-07 12:47:55 -10:00
- Add comprehensive camera integration examples in examples/python/camera/ - camera-basic.py: Single frame capture and analysis - camera-continuous.py: Continuous monitoring system - camera-multi-agent.py: Multi-agent parallel analysis - camera-video-analysis.py: Video recording and temporal analysis - README.md: Complete documentation for all examples - Add camera integration documentation in docs/features/camera-integration.mdx - Complete integration patterns and use cases - Configuration options and best practices - Troubleshooting guide and security considerations - Links to example code - Update docs/mint.json to include camera integration in navigation Addresses issue #238 regarding multimodel agent camera input Co-authored-by: MervinPraison <[email protected]>
173 lines
4.5 KiB
Markdown
173 lines
4.5 KiB
Markdown
# Camera Integration Examples
|
|
|
|
This directory contains examples demonstrating how to integrate camera feeds with PraisonAI multimodal agents for real-time visual analysis.
|
|
|
|
## 📋 Overview
|
|
|
|
PraisonAI supports visual input through the `images` parameter in Tasks. While there's no built-in camera capture, you can integrate camera feeds by capturing frames/videos and passing them to vision agents.
|
|
|
|
## 🛠️ Setup
|
|
|
|
### Requirements
|
|
```bash
|
|
pip install praisonaiagents opencv-python
|
|
```
|
|
|
|
### Environment Variables
|
|
```bash
|
|
export OPENAI_API_KEY=your_openai_api_key
|
|
```
|
|
|
|
## 📁 Examples
|
|
|
|
### 1. `camera-basic.py` - Basic Camera Capture
|
|
- **Purpose**: Capture a single frame and analyze it
|
|
- **Use Case**: Quick analysis, testing camera setup
|
|
- **Features**:
|
|
- Single frame capture
|
|
- Basic object and scene analysis
|
|
- Automatic cleanup
|
|
|
|
**Usage:**
|
|
```bash
|
|
python camera-basic.py
|
|
```
|
|
|
|
### 2. `camera-continuous.py` - Continuous Monitoring
|
|
- **Purpose**: Continuous camera monitoring with periodic analysis
|
|
- **Use Case**: Security monitoring, surveillance systems
|
|
- **Features**:
|
|
- Configurable analysis intervals
|
|
- Real-time monitoring
|
|
- Security-focused analysis
|
|
- Graceful shutdown with Ctrl+C
|
|
|
|
**Usage:**
|
|
```bash
|
|
python camera-continuous.py
|
|
```
|
|
|
|
### 3. `camera-multi-agent.py` - Multi-Agent Analysis
|
|
- **Purpose**: Multiple specialized agents analyzing the same camera feed
|
|
- **Use Case**: Comprehensive analysis from different perspectives
|
|
- **Features**:
|
|
- Security analysis agent
|
|
- Object detection agent
|
|
- Scene analysis agent
|
|
- Parallel processing
|
|
|
|
**Usage:**
|
|
```bash
|
|
python camera-multi-agent.py
|
|
```
|
|
|
|
### 4. `camera-video-analysis.py` - Video Recording & Analysis
|
|
- **Purpose**: Record video segments and analyze temporal events
|
|
- **Use Case**: Activity analysis, event detection
|
|
- **Features**:
|
|
- Video recording with configurable duration
|
|
- Temporal event analysis
|
|
- Timeline extraction
|
|
- Automatic cleanup
|
|
|
|
**Usage:**
|
|
```bash
|
|
python camera-video-analysis.py
|
|
```
|
|
|
|
## 🎯 Supported Input Types
|
|
|
|
- ✅ **Local Images**: `"camera_shot.jpg"`, `"webcam_capture.png"`
|
|
- ✅ **Local Videos**: `"security_feed.mp4"`, `"recording.avi"`
|
|
- ✅ **Image URLs**: `"https://example.com/live_feed.jpg"`
|
|
- ✅ **Multiple Sources**: `["cam1.jpg", "cam2.jpg", "video.mp4"]`
|
|
|
|
## 🔧 Configuration Options
|
|
|
|
### Camera Settings
|
|
```python
|
|
camera_id = 0 # Default camera (change to 1, 2, etc. for other cameras)
|
|
```
|
|
|
|
### Analysis Intervals
|
|
```python
|
|
analysis_interval = 10 # Seconds between analyses for continuous monitoring
|
|
```
|
|
|
|
### Recording Duration
|
|
```python
|
|
recording_duration = 15 # Seconds for video recording
|
|
```
|
|
|
|
## 💡 Integration Patterns
|
|
|
|
### 1. Real-time Processing
|
|
```python
|
|
# Capture frames periodically and process them as separate tasks
|
|
def capture_and_analyze():
|
|
cap = cv2.VideoCapture(0)
|
|
ret, frame = cap.read()
|
|
if ret:
|
|
cv2.imwrite("temp_capture.jpg", frame)
|
|
# Process with PraisonAI agent
|
|
cap.release()
|
|
```
|
|
|
|
### 2. Batch Analysis
|
|
```python
|
|
# Process multiple camera angles simultaneously
|
|
task = Task(
|
|
description="Analyze multiple camera feeds",
|
|
agent=vision_agent,
|
|
images=["cam1.jpg", "cam2.jpg", "cam3.jpg"]
|
|
)
|
|
```
|
|
|
|
### 3. Video Analysis
|
|
```python
|
|
# Save video segments and analyze them
|
|
task = Task(
|
|
description="Analyze this video for activities",
|
|
agent=vision_agent,
|
|
images=["security_footage.mp4"] # Video files work too
|
|
)
|
|
```
|
|
|
|
## 🔒 Security Considerations
|
|
|
|
- **Camera Permissions**: Ensure your application has camera access
|
|
- **Privacy**: Be mindful of privacy when recording/analyzing
|
|
- **Storage**: Clean up temporary files to avoid storage issues
|
|
- **Access Control**: Implement proper access controls for camera systems
|
|
|
|
## 🐛 Troubleshooting
|
|
|
|
### Camera Not Found
|
|
```python
|
|
# Check available cameras
|
|
for i in range(4): # Check first 4 camera indices
|
|
cap = cv2.VideoCapture(i)
|
|
if cap.isOpened():
|
|
print(f"Camera {i} is available")
|
|
cap.release()
|
|
```
|
|
|
|
### Permission Issues
|
|
- On Linux: Add user to `video` group
|
|
- On macOS: Grant camera permissions in System Preferences
|
|
- On Windows: Check camera privacy settings
|
|
|
|
### Performance Optimization
|
|
- Reduce frame size for faster processing
|
|
- Adjust analysis intervals based on requirements
|
|
- Use parallel processing for multiple agents
|
|
|
|
## 📚 Related Documentation
|
|
|
|
- [Multimodal Features](../../docs/features/multimodal.mdx)
|
|
- [Agent Documentation](../../docs/concepts/agents.mdx)
|
|
- [Task Documentation](../../docs/concepts/tasks.mdx)
|
|
|
|
## 🤝 Contributing
|
|
|
|
Feel free to contribute additional camera integration examples or improvements to existing ones! |